System Status

System Status

Welcome to Ping Identity's system status site.

System Uptime

System Uptime

System uptime in the past 90 days.

Past Incidents Past Incidents

Welcome to Ping Identity's system status site.

PingOne/PingID Service Interruption (DNS Outage)
Incident Report for Ping Identity
Postmortem

Incident Summary

Beginning at roughly 19:29 UTC on 1/31/19, DNS lookups for the pingidentity.com domain resulted in "Host not found" results. Service was restored with a configuration change on our primary DNS provider.

DNS for the pingidentity.com domain is served by two external DNS providers. In the event of a failure with our primary provider, DNS is configured to be serviced by our secondary provider. In this case instead of queries failing they were returned with no data, and the switchover to our secondary provider did not happen as expected.

Customer Impact

Customers would have been unable to resolve, and therefore connect to, several of our hosted services (see affected services below). As this issue was caused by incorrect DNS lookups, the actual impact to customers will depend on how clients cached the DNS records. In some cases, there was no impact. In other cases, service was interrupted.

Incident Timeline

Jan 31, 2019 (all times in UTC)

* 19:29 - Monitoring systems detect issues with pingidentity.com services. Incident command process initiated.

* 19:36 - Configuration change applied on primary DNS provider.

* 19:40 - Services report as fully restored.

Affected Services

* PingID Service (.com)

* Token Processor Services (All Regions)

* Provisioning Services (.com)

* ADConnect & Routing Services (.com)

* Pingidentity.com Website

Resolution

Service was restored with a configuration change on our primary DNS provider. We are actively engaged with the vendor to determine what caused the issue.

Ping Action Items

* Engage DNS providers on ensuring proper movement of service to secondary provider.

* Audit and test the resiliency of our DNS infrastructure

Posted Feb 01, 2019 - 17:49 UTC

Resolved
This incident has been resolved.
Posted Jan 31, 2019 - 19:57 UTC
Monitoring
Services are recovering and the team is closely monitoring all services. Another update will be posted once everything is confirmed resolved.
Posted Jan 31, 2019 - 19:44 UTC
Investigating
Monitoring systems have detected an issue with multiple PingOne Services. The Site Reliability Engineering team has been notified and is currently working the issue. We will update this message when the incident has been identified. Automated monitoring systems will update affected components and will resolve operational status as systems recover.

For additional questions please contact Ping Identity Technical Support by opening a case through The Community/Support site (https://www.pingidentity.com/en/account/sign-on.html), or follow this incident on https://status.pingidentity.com for real-time service updates.
Posted Jan 31, 2019 - 19:35 UTC
This incident affected: PingOne for Enterprise - Global (Administration API, AD Connect & Routing Service, Administration Portal, OAuth Configuration Service, Single Sign-on), PingOne for Enterprise - United States (.com services) (Directory API, Directory Login, Office365 Service, PingOne Dock, SCIM Provisioning), PingOne for Enterprise - Europe (.eu services) (Directory API, Directory Login, Office365 Service, PingOne Dock, SCIM Provisioning), PingOne for Enterprise - Australia (.com.au services) (Directory API, Directory Login, Office365 Service, PingOne Dock, SCIM Provisioning), PingID - Europe (.eu services) (PingID Authenticator, PingID SDK Server, PingID Server), PingID - Australia (.com.au services) (PingID Authenticator, PingID SDK Server, PingID Server), PingOne and PingOne Services - United States (.com services) (User Login, User Login API, Admin Login, PingOne MFA: API), Pingidentity.com Website (apidocs.pingidentity.com, developer.pingidentity.com, docs.pingidentity.com, support.pingidentity.com), PingID - United States (.com services) (PingID Authenticator, PingID Server, PingID SDK Server), PingID Global (Notification Services, PingID App), and Twilio (SMS).