Commerce Layer Status Page

Aug 3, 2026 · resolved Aug 3

# Summary On Aug 3rd, 2026, from \`5:02 AM UTC\` till \`5:13 AM UTC\`, a routine automated update process failed to complete, causing a major downtime of our EU Database cluster. Monitoring system alerted immediately our team, that started a manual override of the process, taking it to a successful end. # Leadup While our recurrent automated DB update process was running on the EU database it encountered an unexpected condition, that prevented the process to complete, leaving the cluster in an unconsistent state. # Fault Some database engine items were not in the expected condition for the update to complete correctly and that caused the update to fail. # Detection Moments after the initial failure of the automated process, our monitoring system alerted our infrastructure team. # Root Cause Logical replication feature was not in the proper state after some previous replication activity in the cluster. The current update process couldn't detect this situation in advance to avoid starting at all. # Mitigation and resolution A few minutes after the beginning of the issue, our operators were able to reset the inconsistent parameters and apply the manual update procedure. # Corrective and Preventative Measures We are introducing a more robust update process that, by design, will not be prone to similar issues and will consistently limit the overall risk of errors during to database updates.

Nov 18, 2025 · resolved Nov 19

The issue has been resolved.

May 23, 2025 · resolved May 23

# **Summary** On May 23rd, 2025, from \`5:50 AM UTC\` till \`7:03 AM UTC\`, after a scheduled deployment,  some customers experienced mismatched data responses on their interactions with our services. Realizing that it was a major issue, we initiated the roll-back procedure, reverting our services to the previous code version. # **Leadup** At  \`5:50 AM UTC\` we concluded a deployment that included an upgrade of a third party provider library. The deployment introduced errors on several sessions. # **Fault** Some sessions with tokens requested during the affected timeframe might have experienced issues with add to cart functionality and checkout. In particular, adding items to cart and placing orders was not always feasible. In addition to this, some placed orders were missing approved\_at and reference data. # **Detection** A few minutes after the release, one of our operators detected an anomaly on the internal metrics dashboard that was confirmed by our clients requests to the support team. # **Root Cause** The upgrade of a third party provider library that manages low level web request wrappers introduced problems in our APIs that have not been detected by our tests in preprod and qa environments. The upgrade was recommended to keep our services up to speed with the latest security patches and compatibility with other components. # **Mitigation and resolution** A few minutes after the release, once we realized the severity of the issue, we decided to roll-back to the previous version of the code in order to immediately restore normal operations and allow a deeper understanding of the root cause offline. In addition to the roll-back, a partial CDN cache purge was performed to reduce the TTL of the wrong data. # **Corrective and Preventative Measures** We are introducing additional automatic tests related to the involved component and other similar patterns.

Oct 1, 2024 · resolved Oct 1

The Dashboard is now completely operational, the test/live toggle that disappeared, is now visible and clickable.

Apr 9, 2024 · resolved Apr 9

# **Summary** On April 9th, from 6:45 AM to 8:55 AM UTC, during the scheduled maintenance window, we encountered a database issue. The database version upgrade process failed and the automatic recovery service provided by our cloud infrastructure provider did not activate. We immediately contacted the infrastructure provider’s support team. They addressed this problem and there was no resulting data loss. # **Leadup** On April 9th, 2024, we began the DB upgrade maintenance at 6:39 AM UTC. At approximately 6:45 AM UTC, the background task provided by the DB cloud service failed, leaving the entire upgrade procedure stuck in an intermediate state. # **Fault** During the incident, all Core API calls resulted in errors. These errors were spread across all endpoints and customers, depending on their traffic distribution and type. The read requests cached on our CDN continued to work without errors. # **Detection** A few minutes after beginning maintenance, our engineers realized that the automatic upgrade process was not responding. The cloud service provides safeguards during upgrades that automatically restores normal operations should a timeout exceed 5 minutes. This expected action did not occur.     # **Root Cause** The service provider's support team confirmed that the upgrade process, once initiated, encountered an unexpected issue within their internal procedure, causing the upgrade to stall. The automatic recovery safeguard began as expected five minutes after the incident. However, it couldn't complete because one of the read replicas was in an inconsistent state. The vendor is still investigating the cause of this issue. Since the database service is fully managed by the provider, our engineers had to wait for the provider's support team to restore the normal level of availability. # **Mitigation and resolution** A few minutes before 7:00 AM UTC, our engineers tried to manually interrupt the process, but the option to cancel it was disabled by the vendor. Our team immediately engaged the service provider’s support team, requesting to interrupt the process and restore the availability of the DB, with the highest urgency. The provider’s support team started investigating through their own tools and after a few minutes suggested a couple of workarounds that didn’t succeed. In parallel, we began our own restoration process from a backup.   At about 8:50 AM UTC, the service provider’s support team was able to restore the DB availability and our services gradually recovered to normal operation levels. At approximately 8:55 AM UTC, the entire resolution was completed. # **Corrective and Preventative Measures** Given the nature of the cloud infrastructure on which our services are built upon, not all operational steps are in our full control. However, we identified improvements that we intend to implement on processes and procedures, together with our service provider:  * We will directly involve the service provider’s representatives in future maintenance operations, from planning through implementation. * We will update our status page as soon as the issue occurs. We will extend this by sending a proactive alert to organization owners.   * We will schedule the maintenance window during a low traffic period on our platform.

Is Commerce Layer down right now?

No — Commerce Layer is up. All systems operational as of Aug 19, 10:41 PM UTC.

What is Commerce Layer's current status?

Commerce Layer: All Systems Operational. Alert24 checks Commerce Layer's status page continuously and can notify you the moment it changes.

How do I get alerted when Commerce Layer goes down?

Alert24 monitors Commerce Layer and 3,700+ other cloud and SaaS providers. When an outage is detected it updates your status page automatically and pages your on-call team — no manual checks. Start free at alert24.net.