Google Workspace Status Dashboard
-
-
-
-
Incident affecting Google Calendar
Incident began at 2023-09-21 13:30 and ended at 2023-09-21 16:30 (times are in Coordinated Universal Time (UTC)).
| Date | Time | Description | |
|---|---|---|---|
| | Sep 28, 2023 | 6:54 PM UTC | Incident ReportSummaryOn 21 Sept at 06:30 US/Pacific, a limited number of Google Calendar web and mobile users experienced service disruption. These users may have seen “500 Internal” errors when trying to access Calendar on the web main page and while interacting with events for a duration of 3 hours and 15 minutes. This is not the level of quality and reliability we strive to offer you, and we are taking immediate steps to improve the platform’s performance and availability. Root CauseA recent update to Google Calendar triggered an unintended bug that slowed down the service. This was compounded by an increase in invalid internet Calendar Scheduling (ICS) from Gmail to Calendar. As a result, some users may have seen intermittent 500 internal errors while using Google Calendar. Remediation and PreventionGoogle engineers were alerted to this issue via internal monitoring on 21 Sept at 06:30 US/Pacific and immediately started an investigation. Engineers mitigated the issue by scaling up Calendar resources and filtering traffic reaching Calendar backends. In addition, we rolled back the recent change in ICS file processing, resulting in traffic returning to normal levels. The issue was fully resolved on 21 Sept at 09:30 US/Pacific. Google is committed preventing a repeat of this issue in the future and is completing the following actions:
Detailed Description of ImpactOn 21 Sept from 06:30 to 09:30 US/Pacific, a limited number of Calendar web users experienced intermittent error rates when accessing Calendar, leading to some users not being able to see their event information updated on Calendar. |
| | Sep 21, 2023 | 7:00 PM UTC | Mini Incident ReportWe apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213. (All Times US/Pacific) Incident Start: 21 September 2023 at 06:32 Incident End: 21 September 2023 at 09:47 Duration: 3 hours, 15 minutes Affected Services and Features: Workspace - Google Calendar Regions/Zones: Global Description: Google Calendar Web and Mobile users experienced a global service disruption where users saw “500 Internal’ errors when accessing Calendar On Web main page or interacting with events for a duration of 3 hours, 15 minutes. From preliminary analysis, the root cause of the issue is due to an increase in Internet Calendar Scheduling (ICS) traffic from Gmail to Calendar backends. The issue was mitigated by scaling up the Calendar backend. Google will complete a full incident report in the following days that will provide a detailed root cause. Customer Impact: Web and Mobile users received “500 Internal" errors when accessing Calendar On Web main page or interacting with events. |
| | Sep 21, 2023 | 4:49 PM UTC | The problem with Google Calendar has been resolved. We apologize for the inconvenience and thank you for your patience and continued support. |
| | Sep 21, 2023 | 4:05 PM UTC | Google Calendar service has already been restored for some users, and we expect a resolution for all users within the next 1 hour. Please note this time frame is an estimate and may change. The affected users are unable to access Google Calendar, but we've provided a workaround below. Rollback is completed, and our engineers are seeing recovery. They are monitoring further for full resolution. Customers who are having issues loading their Calendar, may be able to fix this by refreshing their browser |
| | Sep 21, 2023 | 3:28 PM UTC | We are continuing to investigate this issue. We will provide an update by Sep 21, 2023, 4:00 PM UTC detailing when we expect to resolve the problem. Customer symptom: Web and Mobile users are experiencing "500 internal" errors when trying to access Google Calendar and/or interact with events. Workaround: Customers who are having issues loading their Calendar, may be able to fix this by refreshing their browser |
| | Sep 21, 2023 | 3:12 PM UTC | We are continuing to investigate this issue. We will provide an update by Sep 21, 2023, 3:45 PM UTC detailing when we expect to resolve the problem. The affected users are able to access Google Calendar, but are seeing error messages, high latency, and/or other unexpected behavior. Customer symptom: Customers can see 500 errors when trying to access Google Calendar and/or interact with events. Workaround: Customers who are having issues loading their Calendar, may be able to fix this by refreshing their browser |
| | Sep 21, 2023 | 2:35 PM UTC | Our team is continuing to investigate this issue. We will provide an update by Sep 21, 2023, 3:15 PM UTC with more information about this problem. Thank you for your patience. The affected users are able to access Google Calendar, but are seeing error messages, high latency, and/or other unexpected behavior. Customer symptoms: Customers may see errors when clicking on events. Workaround: Customers who are having issues loading their Calendar, may be able to fix this by refreshing their browser |
| | Sep 21, 2023 | 2:17 PM UTC | We're investigating reports of an issue with Google Calendar. We will provide more information shortly. The affected users are able to access Google Calendar, but are seeing error messages, high latency, and/or other unexpected behavior. Customer symptoms: Customers may see errors when clicking on events. Workaround: Customers who are having issues loading their Calendar, may be able to fix this by refreshing their browser |
- Times are listed in Coordinated Universal Time (UTC)