Unable to access helpdesk

Incident Report for Gorgias

Postmortem

Starting at 3:09PM UTC yesterday, our hosting provider (Google Cloud) experienced network connectivity issues in one of their datacenters (us-east1-c zone). As a result, our application servers were disconnected from the database and the load balancer serving traffic, which made some Helpdesks completely unavailable.

By 3:38 PM UTC, our hosting provider fixed the network connectivity issues and all the Helpdesk were available again.

Gorgias is running in several datacenters across the globe and ~20% of all our customers routed to that specific us-east1-c datacenter suffered from this incident. We will keep working on minimizing the impact of such infrastructure outages. We apologize for the service disruption.

Posted Feb 02, 2022 - 02:18 UTC

Resolved

This incident has been resolved.

Posted Jan 31, 2022 - 16:51 UTC

Update

We have fully recovered and are monitoring the system.

Posted Jan 31, 2022 - 15:55 UTC

Monitoring

We experienced connectivity issues with one of our 3rd party service providers, GCP. A fix on their side appears to have been implemented and we are in the recovery process and are monitoring the situation.

Posted Jan 31, 2022 - 15:36 UTC

Investigating

We are investigating an issue on the helpdesk in our us-east1 region where users are unable to access their helpdesk.

Posted Jan 31, 2022 - 15:19 UTC

This incident affected: Helpdesk (REST API, Web App, Mobile Apps) and Helpdesk Clusters (us-east1-635c).