A botched network configuration update took down some Google Cloud VMware Engine (GCVE) services, the hyperscaler has confirmed.

The service went down for users in Sydney (australia-southeast1), Melbourne (australia-southeast2), and Frankfurt (europe-west3) for almost 12 hours on July 14.

Only stretch clusters, which extend compute resources across distributed sites or data centers, were impacted, with no disruption reported for standard GCVE services. The issue impacted the infrastructure that links the zones within a stretch cluster, which caused synchronization issues. Virtual machines (VMs) on the impacted sites were effectively isolated and left without writable data.

According to the hyperscaler, the fault was a network configuration update, with engineers rolling back the configuration to its last-known good value.

The outage marks the latest in a growing number of incidents occurring as a result of faulty updates.

A new iteration of Cloudflare's tenant API service resulted in Cloudflare essentially DDoS-ing its own dashboard last September. A few weeks later, a database automation tool would take down Cloudflare’s entire platform, resulting in a quarter of the web going dark, even affecting SDxCentral.

A software issue took down Verizon’s network at the turn of the year, while arguably the most famous update botch in recent years was the infamous Microsoft meltdown caused by a faulty sensor configuration update by CrowdStrike.