Google Workspace Status Dashboard
Incident affecting Google Meet
Incident began at 2022-10-10 22:18 and ended at 2022-11-07 15:20 (times are in Coordinated Universal Time (UTC)).
Date | Time | Description | |
---|---|---|---|
| Nov 30, 2022 | 7:04 PM UTC | Incident ReportSummaryOn 10 October 2022 at 15:18 PT, Google was alerted to a subset of users experiencing issues with Duo phone number registration with our Google Meet iOS application. Once the root cause was identified, Google pushed out an update to the App Store on 4 November 2022. We believe most users (66%) updated their app by 7 November 2022. To our Google Workspace customers who were impacted during this outage, we sincerely apologize. We have conducted an internal investigation and are taking steps to improve our service. Root CauseThe database version for our Google Meet backend infrastructure uses a specific library to complete the database operations to register or reregister a Duo calling account. An issue with the error-handling mechanism in the library resulted in the backend operations not functioning as intended. This caused affected users to be unable to register or reregister phone numbers, preventing the use of calling features, on the Meet iOS app. Remediation and PreventionGoogle engineers were alerted to the situation on 10 October 2022 at 15:18 and immediately started an investigation. Engineers reviewed the Google Meet for iOS app’s feedback logs and identified the root cause as related to registration workflow. Code improvements were identified to handle the app error seen when a user tries to register their account. The fix was deployed in a new version of the iOS app, which was rolled out to the App Store by 4 November 2022. At least 66% of users upgraded to the new version by 7 November 2022. Google is committed to preventing a repeat of this issue in the future and is completing the following actions:
We apologize for the length and severity of this incident. We are taking immediate steps to prevent a recurrence and improve reliability in the future. Detailed Description of ImpactFrom 10 October 2022 at 15:18 to 7 November 2022 at 08:20, Google Meet iOS app users may have encountered an “unable to add calling” error when trying to register or reregister their Duo calling account, which resulted in phone numbers not registering as intended. |
| Nov 8, 2022 | 7:29 PM UTC | Mini Incident ReportWe apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213. (All Times US/Pacific) Incident Start: 28 October 2022 09:22 Incident End: 04 November 2022 11:05 Duration: 7 days, 1hour, 43 minutes Affected Services and Features: Google Meet - iOS App Regions/Zones: Global Description: A subset of Google Meet iOS App users experienced issues with Phone Number registration. From preliminary analysis, the root cause is a latent issue at the time of registration. The latest version of Google Meet iOS App which contains the fix was released to the AppStore on 04 November 2022. Customer Impact: Google Meet iOS App users may have encountered an “unable to add calling” error which resulted in Phone number registration failures. |
| Nov 3, 2022 | 6:51 PM UTC | The problem with Google Meet has been resolved. We apologize for the inconvenience and thank you for your patience and continued support. The fix for the issue is now available in the latest version of Meet iOS app and we recommend customers to upgrade to the latest version of Meet iOS app. |
| Oct 31, 2022 | 5:14 PM UTC | We expect to resolve the problem affecting a significant subset of users of Google Meet at Nov 4, 2022, 8:00 PM UTC. Please note that this time frame is an estimate and may change. We will provide more information by Wednesday, 2022-11-02 7:00 PM UTC on progress of the mitigation. Customer Symptoms: Some users of Meet iOS app may be unable to complete Phone Number registration and as a result can not use calling features. The affected users should still be able to join Meet sessions No workaround at this time. |
| Oct 31, 2022 | 1:37 AM UTC | We expect to resolve the problem affecting a significant subset of users of Google Meet at Nov 4, 2022, 8:00 PM UTC. Please note that this time frame is an estimate and may change. We will provide more information by Monday, 2022-10-31 7:00 PM UTC on progress of the mitigation. Customer Symptoms: Some users of Meet iOS app may be unable to complete Phone Number registration and as a result can not use calling features. The affected users should still be able to join Meet sessions No workaround at this time. |
| Oct 31, 2022 | 1:29 AM UTC | We're aware of a problem with Google Meet affecting a significant subset of users. We will provide an update by Oct 31, 2022, 2:00 AM UTC detailing when we expect to resolve the problem. Please note that this resolution time is an estimate and may change. Some users of Meet iOS app may be unable to complete Phone Number registration and as a result can not use calling features. The affected users should still be able to join Meet sessions No workaround at this time. |
- Times are listed in Coordinated Universal Time (UTC)