Ping Identity's main DNS provider, Dyn, came under a large DDoS attack effectively preventing DNS resolution for Ping customers in multiple geographic regions.
Customers in regions where Dyn was under attack would be unable to resolve Ping's DNS domains. This resulted in an outage of all Ping services for customers in impacted regions.
05:16 - Pingdom pages on EU PingOne Dock being down and SRE begins investigation 05:25 - Ping Identity Administration Portal moves to Degraded Performance 05:28 - Synthetic transactions failure in US East Coast region pages the on-call engineer - licensing.pingidentity.com moves to Degraded - documentation.pingidentity.com moves to Degraded 06:05 - First public reports of Dyn DNS being targeted by a DDoS attack 06:07 - First status posting made by SRE 06:42 - SRE begins preparation for move to new DNS provider 07:00 - SRE decides not to migrate to a secondary DNS provider. Given the limited scope of the attack, SRE decided the risk of migrating outweighed the benefits 07:20 - DDoS attack stops and services resume to normal 07:37 - Status page updated as resolved 09:53 - Second DDoS attack launched against Dyn 10:19 - SRE begins configuring zones on a secondary DNS provider 10:22 - New status page posting is made 10:31 - Status page updated to reflect the intermittent nature of the outage and attack 10:42 - SRE begins increasing TTLs on DNS to 1 day as an attempt to mitigate failed lookups 11:06 - Status page updated to reflect mitigation efforts 11:36 - TTL increase complete 12:04 - Status update made to reflect TTL mitigation. The status page is erroneously updated to report the attack is subsiding. 12:54 - Status Page updated to reflect the continued attack. Impact is now considered to include all services 13:10 - SRE makes the decision to migrate off of Dyn DNS. 13:33 - Migration of pingone.com, pingone.eu and pingidentity.eu complete 14:28 - Migration of pingidentity.com to alternate DNS provider complete 14:34 - Pingdom reports that all endpoints are back up 14:50 - Migration of pingone.com.au and pingidentity.com.au complete 16:47 - Dyn reports all DDoS attacks are mitigated
North America Critical Path Europe Critical Path Australia Critical Path Admin Portal Monitor Directory API Single Sign-on Single Sign-On (PingOne SSO for SaaS Apps/APS) Administration Portal OAuth Service Administration API AD Connect & Routing Service PingOne dock (North America) PingOne dock (Europe) PingOne dock (Australia) Directory Login (North America) Directory Login (Europe) Directory Login (Australia) Directory API (North America) Directory API (Europe) Directory API (Australia) Office365 Service (North America) Office365 Service (Europe) Office365 Service (Australia) SCIM Provisioning (North America) SCIM Provisioning (Europe) SCIM Provisioning (Australia)
PingID App PingID Authenticator (North America) PingID Authenticator (Europe) PingID Authenticator (Australia) PingID Server (North America) PingID Server (Europe) PingID Server (Australia)
Pingidentity.com Critical Path www.pingidentity.com docs.pingidentity.com documentation.pingidentity.com licensing.pingidentity.com updates.pingidentity.com