Issue receiving emails for some customers using legacy forwarding rules

Incident Report for Gorgias

Postmortem

Incident Post-Mortem: May 14, 2025

Summary

Between May 12 and May 14, 2025, a subset of Gorgias customers experienced an issue where Gorgias did not deliver emails sent or forwarded to specific email addresses to their helpdesks. A configuration change we made disabled our ability to receive emails from certain addresses. We resolved the issue, and no data loss occurred. We successfully recovered and processed all affected emails.

What Happened?

In 2019, Gorgias improved the security of customer email addresses by switching from easily guessable addresses (e.g., support@yourdomain.gorgias.io) to a more secure, randomized format (e.g., 123PUY9qcgsABC@emails.gorgias.com).

Despite this change, we continued to support the older, customer-specific addresses to maintain a smooth experience for our users. Thanks to a fallback mechanism, our system was able to recognize and connect emails sent to either format.

In March 2024, we updated the email setup interface to show the addresses stored in our database, instead of always displaying the new, secure format.

On May 12, 2025, we made another change that stopped accepting emails forwarded to addresses not stored in our database. Our pre-change analysis showed that most traffic to these addresses was spam, and we assumed that they were no longer being actively used.

Unfortunately, we missed a small number of active accounts that were still relying on these older addresses for email delivery. This led to email disruptions for these accounts.

Once we identified this error, we immediately rolled back the change, restored the prior functionality, and re-processed all affected emails, ensuring no data loss.

What We Are Doing to Prevent This in the Future

Following an internal review, we have identified several actions to reduce the likelihood of similar incidents:

  • Properly Provision Missing Addresses: We have provisioned the email addresses that relied on the fallback search mechanism, so that is no longer required.
  • Improved Monitoring and Alerts: We will enhance our monitoring systems to detect disruptions in inbound email processing more quickly, minimizing response times.
  • Stricter Change Management: We will refine our internal processes for identifying and flagging potentially breaking changes, ensuring that our customer support teams are better prepared to respond if issues arise.
  • Customer Communication and Migration Support: We will improve our outreach to ensure customers are fully transitioned to the new email format, reducing reliance on legacy systems.

We sincerely apologize for any disruption this incident caused to your operations. We know how important email delivery is to you. Providing a reliable, resilient platform is our top priority, and we are committed to learning from this event to prevent future disruptions.

Thank you for your continued trust in Gorgias.

Sincerely,

The Gorgias Team

Posted May 16, 2025 - 17:38 UTC

Resolved

Between May 12 and May 14, 2025, a subset of Gorgias customers experienced an issue where Gorgias did not deliver emails sent or forwarded to specific email addresses to their helpdesks. A configuration change we made disabled our ability to receive emails from certain addresses. We resolved the issue, and no data loss occurred. We successfully recovered and processed all affected emails.
Posted May 12, 2025 - 08:00 UTC