Mobile App Issues Detected
Incident Report for Ping Identity

Incident Summary

Google Firebase Cloud Messaging experienced an outage, which caused push notifications in PingID Mobile Authentication App to stop working on Android devices.

Customer Impact

Some customers using Android devices could not receive push notifications for PingID Mobile Authentication App.

The root cause from our provider can be found at this URL: https://status.firebase.google.com/incident/Cloud%20Messaging/17005.

Incident Timeline

  • 1425 - Google reports issues with Firebase Cloud Messaging service
  • 1435 - Engineering and SRE teams confirm PingID service degradation
  • 1524 - Google reports that they are still investigating the issue
  • 1710 - Google reports that services should be restored
  • 2121 - Google Firebase Status Dashboard updated showing services green

Affected Services

  • PingID Android Mobile Authenticator
  • In total, 11% of the push notifications on the PingID Android authenticator failed during the outage, where 89% succeeded.

Resolution

Resolution centered on Google restoring availability to Firebase Cloud Messaging service.

Ping Action Items

  • Improve failover mechanism to detect all Push Notification errors and immediately fall back to One Time Password authentication.
  • Ensure that when there is a manual workaround customers can initiate, it will be published on the status page (in this case, turn Swipe off on the mobile app).
Posted 3 months ago. May 22, 2017 - 12:14 MDT

Resolved
This issue has been resolved.
Posted 3 months ago. May 17, 2017 - 08:08 MDT
Identified
Monitoring systems have detected an issue with the PingID Mobile Authentication App. The Site Reliability Engineering team has been notified and found that this is due to a service outage at google. Please see the following link for status at google: https://status.firebase.google.com/incident/Cloud%20Messaging/17005

For additional questions please contact support@pingidentity.com, or follow this incident on https://status.pingidentity.com for real-time service updates.
Posted 3 months ago. May 16, 2017 - 15:27 MDT
This incident affected: PingID Services (PingID App).