Integrations are slowed down (real time data updates and sending/receiving message are impacted)

Incident Report for Gorgias

Postmortem

Due to the slow down from the earlier incident in cluster us-central1-d8ff, we had large backlog of tasks which caused parts of the message broker to exhaust resources. We were able to quickly provide more resources to the message broker which allowed the integrations to speed back up.

No data loss occurred during this degradation and the message broker has more permanent resource headroom to avoid this in the future. We apologize for any inconvenience caused by this incident.

Posted Apr 06, 2022 - 17:22 UTC

Resolved

This incident has been resolved.

Posted Apr 06, 2022 - 01:47 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Apr 06, 2022 - 01:37 UTC

Identified

The issue has been identified and a fix is being implemented.

Posted Apr 06, 2022 - 00:36 UTC

Update

We are continuing to investigate this issue.

Posted Apr 06, 2022 - 00:31 UTC

Investigating

We are currently investigating this issue.

Posted Apr 06, 2022 - 00:29 UTC

This incident affected: Helpdesk Integrations (Email, Mailgun SMTP, Mailgun inbound email, Mailgun outbound email, Mailgun API, Gmail, Outlook, Live Chat, Smooch Core API, Facebook Posts & Comments, Instagram comments, Shopify integration, Shopify API & Mobile, Aircall Public API, Aircall Aircall Apps, Stripe API, Facebook Messenger, ReCharge integration, Native Phone, Instagram Direct Messages, Yotpo) and Helpdesk Clusters (us-central1-d8ff).