Chat Message Reliability
Incident Report for Kustomer
Postmortem

Incident Documentation

Please find our full report here.

Posted 7 months ago. Jan 29, 2019 - 18:55 EST

Resolved
At 1:46pm ET January 22nd, 2019 our team deployed caching changes via AWS CloudFront that were intended to improve performance. Instead, these changes caused multiple issues affecting chat users. The impact to a portion of chat users may have included crashing mobile apps, failures to send messages, and/or failure to receive messages.

After being notified of crashes on the morning of January 24th, we quickly deployed fixes to resolve mobile app crashes and message sending at 9:20am ET January 24th.

On the evening of January 24th, we were alerted that message delivery to iOS users from the Kustomer agent may be impacted. We investigated and ultimately resolved this issue at 11:20am ET January 25th.
Posted 7 months ago. Jan 25, 2019 - 11:20 EST