System Status
System Status
Welcome to Ping Identity's system status site.System Uptime
System Uptime
System uptime in the past 90 days.Past Incidents Past Incidents
Welcome to Ping Identity's system status site.
Please subscribe to updates to get up to the minute alert notifications on any service interuptions.
Incident Summary
On June 18th, 2019 beginning at 17:37 UTC, customers experienced the inability to login to the PingOne Directory services (login.pingone.com and directory-api.pingone.com). Services were fully recovered by 18:12 UTC after a database restoration.
At the time of the incident, there were planned maintenance activities on a downstream database cluster. These activities triggered an edge case scenario where AWS terminated all instances in the cluster in a specific AWS region.
Incident Timeline
June 18, 2019 (all times in UTC)
- 17:37 - Underlying branding database nodes are terminated by AWS.
- 17:40 - New database instances created and begin replicating data from other regions.
- 18:00 - SRE performs rolling restart of application servers to point to the new database nodes.
- 18:12 - Services recovered.
Affected Services
- Directory Login (.com)
- Directory API (.com)
Resolution
Service restoration occurred after the new database nodes were created and data was recovered. There was no data lost during this incident.
Ping Action Items
- Remove branding services as critical dependency of Directory login services.
- Ensure instance deletion protection is enabled on all branding database instances to resolve this AWS termination edge case.
For additional questions please contact Ping Identity Technical Support by opening a case through The Community/Support site (https://www.pingidentity.com/en/account/sign-on.html), or follow this incident on https://status.pingidentity.com for real-time service updates.