Update - AWS has mitigated the underlying issue that caused some network devices in the US-EAST-1 Region to be impaired. They are seeing improvement in availability across most AWS services. All services are now independently working through service-by-service recovery. In order to expedite overall recovery, They have temporarily disabled Event Deliveries for Amazon EventBridge in the US-EAST-1 Region. These events will still be received & accepted, and queued for later delivery.

Services like SSO, Connect, API Gateway, ECS/Fargate, and EventBridge are still experiencing impact. AWS Engineers are actively working on resolving impact to these services.

We are seeing most services restored in Kustomer, with the exception of some workflow items that are still taking some time to process. We are starting our re-driving efforts and will post more information shortly
Dec 7, 18:14 EST

Monitoring - AWS has executed a mitigation which is showing significant recovery in the US-EAST-1 Region. Our Engineers are seeing significant recovery as well and are actively working to bring back our US-EAST-1 server back online. We will provide another update as soon as this becomes available.
Dec 7, 17:46 EST

Update - We are continuing to experience problems across Kustomer with various parts of the platform including sending/receiving messages, events processing, object updates, automation processing, etc. due to an incident with AWS.

Kustomer engineers are actively working on this issue with AWS and independently to find additional ways to restore functionality as quickly as possible. We do not have an ETA on resolution yet. But will post regular updates as we gather more information.
Dec 7, 17:03 EST

Update - We are continuing to experience problems across Kustomer with various parts of the platform including sending/receiving messages, events processing, object updates, automation processing, etc. due to an incident with AWS.

Kustomer engineers are continuing to actively work on finding additional ways to restore functionality as quickly as possible. We do not have an ETA on resolution yet. But will post regular updates as we gather more information.
Dec 7, 15:38 EST

Update - We are continuing to experience problems across Kustomer with various parts of the platform including sending/receiving messages, events processing, object updates, automation processing, etc. due to an incident with AWS.

AWS has identified the root cause of this issue as a problem with several network devices within the internal AWS network. The internal DNS servers that provide resolution for the AWS services were specifically impacted by these issues. Amazon has moved traffic for those servers to another set of network devices, which has resolved those DNS issues. However, the network devices for other non-DNS services are still impacted, and Amazon is still actively working to shift more traffic to fully mitigate this issue.

Kustomer engineers are also actively working on this issue to find additional ways to restore functionality as quickly as possible. We do not have an ETA on resolution yet. But will post regular updates as we gather more information.
Dec 7, 14:26 EST

Update - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing events in the platform. Services include sending and receiving inbound integration events, app events, object updates, conversation correspondence, automation processing, Amazon Connect services, and other areas of the Kustomer platform. The scope of this incident has increased and we are currently investigating further as we work towards a solution.

We have also been informed that the vendor has started resolving the issue on their side, and are starting to see some signs of recovery. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 13:34 EST

Update - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing events in the platform. Services include sending and receiving inbound integration events, app events, object updates, conversation correspondence, automation processing, Amazon Connect services, and other areas of the Kustomer platform. The scope of this incident has increased and we are currently investigating further as we work towards a solution. We have also been informed that the vendor has identified the issue and is working on a resolution from their side as well. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 13:15 EST

Update - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing events in the platform. Services include sending and receiving inbound integration events, app events, object updates, conversation correspondence, automation processing, and Amazon Connect services. The vendor has identified the issue and is working on a resolution. Our team will continue to monitor the status of the vendor. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 12:50 EST

Update - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing events in the platform, Amazon Connect services, and the delivery of messages within the platform at a slower pace than expected. The vendor has identified the issue and is working on a resolution. Our team will continue to monitor the status of the vendor. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 11:58 EST

Identified - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing events in the platform, Amazon Connect services, and the delivery of messages within the platform at a slower pace than expected. The vendor has identified the issue and is working on a resolution. Our team will continue to monitor the status of the vendor. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 11:34 EST

Investigating - Kustomer is aware of an event reported by one of our third-party vendors affecting Prod1 and Prod2 environments that may cause latency of processing conversation changes and the delivery of messages within the platform at a slower pace than expected. Our team is continuing to take action from our side and will continue monitoring the status with the vendor. Please expect further updates within the next hour and reach out to support through email at support@kustomer.com if you have additional questions or concerns.
Dec 7, 11:03 EST