Gemini Partial Outage - August 27, 2026

<p> Incident began at <strong>2026-08-20 08:40</strong> and ended at <strong>2026-08-20 12:20</strong> <span>(all times are <strong>US/Pacific</strong>).</span></p><div class="cBIRi14aVDP__status-update-text"><h1>Incident Report</h1> <h2>Summary</h2> <p>On Thursday, 20 August 2026, from 08:00 to 10:22 US/Pacific (15:00 to 17:22 UTC), multiple Google Cloud services in the us-west1 region experienced elevated latency, provisioning failures, increased error rates, and widespread service degradations for a total duration of 2 hours and 22 minutes. The incident impacted a wide range of core services — including Persistent Disk, Google Kubernetes Engine, Google Compute Engine, Cloud Run, Google Cloud Bigtable, Cloud Storage, and Identity and Access Management — across both control plane operations and data plane requests.</p> <p>We sincerely apologize for the disruption this incident caused to your business operations and critical workloads. We recognize the vital role Google Cloud plays in supporting your organization, and we deeply regret the impact on your business operations and critical workloads. Engineering and infrastructure teams are actively implementing measures to address the root causes and strengthen network resiliency to prevent recurrences in the future. Specifically, teams are monitoring recovery progress, refining safety checks for planned optical maintenance, and optimizing regional traffic routing mechanisms to safeguard against unexpected capacity constraints.</p> <h2>Root Cause</h2> <p>The disruption originated during scheduled fiber optic maintenance, which unexpectedly compromised network capacity between data centers within the us-west1 region. Automated rerouting mechanisms failed to properly redistribute traffic to alternate capacity, resulting in network congestion as volumes exceeded available bandwidth in the affected area.</p> <p>This underlying network degradation subsequently impacted higher-level service components through severe packet loss, request throttling, and increased latency across inter-campus dependencies. Core infrastructure services, including Spanner Paxos consensus and the Unified Metadata Server (UMS), experienced significant latency spikes, which cascaded into timeouts and elevated error rates for downstream dependent products such as Cloud Storage, Cloud IAM, Persistent Disk, and Google Kubernetes Engine. Consequently, both control plane operations and data plane requests failed to execute successfully across multiple Google Cloud services in us-west1 throughout the duration of the incident.</p> <h2>Remediation and Prevention</h2> <p>Internal monitoring systems initially detected widespread service anomalies and alerted Google engineers, who promptly confirmed that multiple core services operating across the us-west1 region were severely impacted. In response to the immediate operational risks, engineering teams promptly executed emergency traffic draining protocols to reroute active service workloads away from the compromised inter-campus network infrastructure and minimize further customer disruption. Once teams restored inter-campus fiber network capacity, engineers systematically reintroduced production traffic back to us-west1 through controlled validation stages, ultimately confirming full service normalization across all impacted platforms.</p> <p>Longer term engineering remediations and architectural enhancements are actively being finalized and assigned to owning teams. Key focus areas include reforming scheduled maintenance protocols, establishing stricter safety checks and circuit redundancy requirements prior to routine maintenance, and refining automated regional traffic failover mechanisms to automatically handle sudden capacity degradation without incurring severe network congestion.</p> <h2>Detailed Description of Impact</h2> <p>On Thursday, August 20, between 08:00 and 10:22 US/Pacific, Google Cloud customers in the us-west1 region encountered elevated latency, provisioning failures, and increased error rates.</p> <ul> <li><strong>Impact / Error Rates:</strong> The reduced capacity caused request throttling, latency spikes, and increased retry volume.</li> <li><strong>Scope Exclusion:</strong> Services and workloads operating in regions other than us-west1 remained fully operational and unaffected.</li> </ul> <p><strong>Affected Services and Features</strong></p> <p>The following services experienced elevated latencies and/or increased error rates, across their respective data planes and control planes:</p> <ul> <li>AlloyDB</li> <li>Apache Kafka</li> <li>Apigee Edge Public Cloud</li> <li>Apigee X</li> <li>Artifact Registry</li> <li>BigQuery & BigQuery Data Transfer Service</li> <li>Cloud Build</li> <li>Cloud Data Fusion</li> <li>Cloud Dataflow</li> <li>Cloud Filestore</li> <li>Cloud Key Management Service (KMS)</li> <li>Cloud Monitoring</li> <li>Cloud Run</li> <li>Cloud SQL</li> <li>Contact Center AI Platform</li> <li>Dataproc Metastore</li> <li>Google App Engine</li> <li>Google Cloud Bigtable</li> <li>Google Cloud Pub/Sub</li> <li>Google Cloud Storage (GCS)</li> <li>Google Compute Engine (GCE)</li> <li>Google Kubernetes Engine (GKE)</li> <li>Identity and Access Management (IAM)</li> <li>Managed Airflow (Cloud Composer)</li> <li>Managed Service for Apache Spark (Dataproc)</li> <li>Persistent Disk</li> </ul> <p>The regional incident in us-west1 followed a structured three-phase recovery dictated by platform dependency layers:</p> <p>While core platform connectivity and live request serving were restored by 10:22 US/Pacific, certain services took additional time to fully recover due to asynchronous backlog processing and localized control-plane state reconciliation for a very small set of customers.</p> <p>For event-driven and pipeline services, inbound error rates dropped to 0 immediately, but some operations experienced elevated latency while workers processed through backlogs accumulated during the outage, clearing later for Cloud Pub/Sub, Cloud Build & Deploy, Cloud Dataflow, and Cloud Storage lifecycle deletions.</p> <p>Concurrently, while primary read/write traffic was healthy across the region, specific long-running lifecycle workflows required extra time to clear locks and reconcile distributed state machines. Compute Engine VM provisioning in zone us-west1-c and Cloud Filestore control plane instance allocation locks and resource validation checks took longer to normalize across regional storage backends.</p> </div><hr><p>Affected products: AlloyDB for PostgreSQL, Apigee Edge Public Cloud, Apigee X, Artifact Registry, BigQuery Data Transfer Service, Cloud Build, Cloud Data Fusion, Cloud Filestore, Cloud Key Management Service, Cloud Monitoring, Cloud Run, Contact Center AI Platform, Dataproc Metastore, Google App Engine, Google BigQuery, Google Cloud Bigtable, Google Cloud Composer, Google Cloud Dataflow, Google Cloud Dataproc, Google Cloud Pub/Sub, Google Cloud SQL, Google Cloud Storage, Google Compute Engine, Google Kubernetes Engine, Identity and Access Management, Managed Service for Apache Kafka, Persistent Disk</p><p>Affected locations: Global, Oregon (us-west1)</p>