The fix that we applied is effective, the spike of events is gone, and all the systems are back to normal.
The incident is now resolved.
Posted Jan 08, 2025 - 08:31 PST
Monitoring
A fix has been implemented and we are monitoring the results.
It seems that one of our providers, Google, had an issue with dispatching all the events to our system, and that lead to delays in our ecom data, disruptions in chat and voice.
Currently we are monitoring if the fix that we applied is fixing the issue
Posted Jan 08, 2025 - 08:02 PST
Identified
We identified the issue. It seems that we've been flooded by a big number of webhooks, and our system took a long time to respond.
Because of this, the ecom data had some delay in getting updated in helpdesk.
The ecom, chat and voice have been affected by this issue. Currently, we increased the resources that we allocated, and it seems that the situation is back in control.