On May 2nd, 2018 beginning at 12:05 UTC, the PingID and PingOne Dock applications began getting errors connecting to a database hosted on AWS RDS. A manual failover of the database node resolved the issue.
PingOne and PingID database infrastructure runs on AWS RDS (which is an Amazon managed service) and is configured for automated failover in the event of a primary database outage. During this incident, the primary database became unhealthy, but the RDS service did not failover the database properly and required a manual step to failover and restore service.
On May 2nd, 2018 beginning at 12:05 UTC, customers experienced the inability to reach the PingOne Dock (desktop.pingone.com) or authenticate with PingID MFA (authenticator.pingone.com) hosted in our North American data centers. Services began recovering at 12:31 UTC at which point PingID authentication sessions were successful. Due to the backlog of requests to the PingOne Dock, additional servers were deployed to handle the increased load and full service was restored at 12:49 UTC.
May 2, 2018 (all times in UTC)
Service restoration occurred after a manual failover of an AWS RDS database instance.