Status Page Status Page

Aug 19, 2026 · resolved Aug 19

The platform is operating normally. This incident has been resolved

Aug 18, 2026 · resolved Aug 18

Some GitHub pull request status checks in the EU region were delayed by up to 30 minutes before completing. No checks were lost and no action is needed - delayed checks completed automatically. We identified the cause, and fixed it. We're monitoring closely.

Aug 17, 2026 · resolved Aug 17

GitHub component: Git Operations, Webhooks, API Requests, Pull Requests Original GitHub incident: https://stspg.io/y1fl26l6wpzr

Aug 14, 2026 · resolved Aug 14

**Summary** PR secret scans experienced processing lag, causing delays in processing pull requests. The issue was caused by timeouts when calling the model validation service. The model validation service was experiencing elevated latency and errors because compute nodes were scheduled on a subnet that had exhausted its available IPs. This slowed PR secret scan processing and caused consumer lag to build up. To resolve the issue, we temporarily disabled the feature that produces new PR secret scans so the queue could drain. We also resolved the underlying infrastructure issue by moving nodes to subnets with available free IPs. **Customer Impact** Customers experienced delayed PR secret scans. Other scan types remained operational. **Root Cause** The secret scanner calls the AI model validation service to classify detections during PR scans. The model validation call had a 30-second timeout configured. When the model validation service became slow or unresponsive, each failed request could block a scan consumer for the full 30 seconds before timing out. The underlying issue was in the model-serving infrastructure. One of the private subnets had run out of available IPs, and new nodes were created in that exhausted subnet, leaving model-serving pods in a pending state. This resulted in elevated latency and errors in the model validation service and reduced the processing capacity of PR secret scans. **Contributing Factors** • **Model validation timeouts.** The model validation service timeout was 30 seconds. When the service was degraded, scan consumers waited for the full timeout before the request failed, which created a processing bottleneck. • **No fallback to bypass AI validation for PR scans.** There was no mechanism to skip AI model validation specifically for PR secret scans while allowing the rest of the scan to continue. The available kill switch stopped producing PR secret scans entirely. This meant the mitigation required a tradeoff between allowing processing lag to continue or temporarily stopping PR secret scans. • **Customer communication gap.** Internal alerts for PR scan lag did fire, but the status page was not updated promptly to reflect the degradation. As a result, there was a period in which we had internal visibility into the issue while customers did not have the same visibility through the status page. **Mitigation and Recovery** A pre-existing feature flag was used to stop new PR secret scans from being produced. This stopped new messages from being added to the PR secret scan queue and allowed the existing backlog to drain. The backlog was cleared shortly after the mitigation was enabled. In parallel, the underlying model-serving infrastructure issue was resolved by moving nodes to subnets with available free IPs, restoring model-serving capacity. Once the infrastructure issue was resolved and the backlog had drained, PR secret scanning was restored. **Learnings and Corrective Actions** • **Review the AI model validation timeout.** The 30-second model validation timeout meant that when the model-serving layer degraded, each scan consumer could remain blocked for the full timeout before failing. We are reviewing the timeout and retry behavior. This work must also account for the risk that shorter timeouts and faster retries could increase request volume against an already degraded model-serving service. • **Add a graceful degradation path for AI validation.** The system currently has a kill switch for PR secret scanning, but does not have a graceful degradation mechanism for the AI validation step. We are working on a mechanism that can disable AI model calls specifically for PR scans, allowing scans to continue while avoiding the dependency on a degraded model-serving layer. The goal is to allow scans to continue with reduced functionality rather than stopping PR secret scanning entirely. • **Improve alerting and incident communication.** The alerting infrastructure detected PR scan lag, but the status page was not updated promptly. We are improving the process for escalating service degradation and communicating customer-facing impact through the status page. • **Improve model-serving infrastructure resilience.** The underlying infrastructure issue occurred because nodes were scheduled on a subnet with no available IPs. The immediate issue was resolved by moving nodes to subnets with free IPs. We are continuing to improve the resilience of the model-serving infrastructure to reduce the likelihood that infrastructure capacity issues can affect PR scan processing. **Conclusion** The incident was caused by degradation in the AI model validation service due to an infrastructure issue where nodes were scheduled on a subnet with exhausted IP capacity. The resulting model validation timeouts slowed PR secret scan processing and caused consumer lag to build. We mitigated the incident by temporarily stopping new PR secret scans, allowing the backlog to drain, and restoring model-serving capacity by moving nodes to subnets with available free IPs. We are following up with improvements to model-serving resilience, timeout and retry behavior, runtime configuration, graceful degradation, alerting, and customer communication.

Aug 13, 2026 · resolved Aug 13

GitHub component: Git Operations, Webhooks, Pull Requests Original GitHub incident: https://stspg.io/f0cgtdc0kqxg

Is Status Page down right now?

No — Status Page is up. All systems operational as of Aug 19, 9:41 PM UTC.

What is Status Page's current status?

Status Page: All Systems Operational. Alert24 checks Status Page's status page continuously and can notify you the moment it changes.

How do I get alerted when Status Page goes down?

Alert24 monitors Status Page and 3,700+ other cloud and SaaS providers. When an outage is detected it updates your status page automatically and pages your on-call team — no manual checks. Start free at alert24.net.