Workspace services - Oregon is unavailable
Incident window: November 7, 2019 16:30 UTC to 16:45 UTCImpacted Cloud services:
Workspace (Oregon region only)
Other regions are not affected
Impact Level : high
Workspace services in Oregon have become unavailable starting at 16:30 UTC. The availability and performance of Workspace services in other regions is not affected. We are investigating.
[2019-11-07 16:59 UTC] Resolved. The underlying Oregon Workspace database master node had failed and automatically failed over to the replica node, which had been promoted as the new master (and a new replica created after). However, while the database was able to recover with our automation, the Oregon Workspace application did not automatically recover as would be expected and was still yielding errors until we intervened by restarting. Once restarted at 16:45 UTC, the Oregon Workspace services have been available and performing within expected levels. We will be continuing to monitor. We will also be continuing to coordinate with our development team to identify why the Workspace application was unable to automatically recover and to work toward a solution to mitigate these types of incidents requiring our attention beyond our automated incident response controls in the future.