Important: COVID-19 related customer communication 
menu
menu

Feed available - Subscribe to our feed to stay up to date on upcoming maintenance and incidents.

Reporting Services - delayed processing

Incident window: May 14, 2018 14:16 UTC to May 17, 2018 02:17 UTC

Impacted Cloud services:

Impact Level : minor

We have approximately 10,000 files that are slated for ingestion. This data is expected to be ingested by 16:00 UTC today, as a sync point. Once those files are processed, the ingestion of all queued events will be processed. We expect to be current on events by end of day today. More accurate ETA information will be posted in the next update.

[2018-05-14 16:56 UTC] The 10,000 files have been ingested and processed. Events prior to 2018-05-12 16:00 UTC should now be available within reports. We are performing additional database maintenance and then will begin processing data since 2018-05-12 16:00 UTC.

[2018-05-15 16:51 UTC] Events at least up to 2018-05-15 07:00 UTC should be available within reports. We are continuing to monitor closely until all remaining files have been processed. We will provide another update once completed. If we observe that file processing will require significant time to complete, we will also provide an ETA at that time.

[2018-05-16 11:21 UTC] We expect that all remaining files up to the current date and time will be processed by 2018-05-16 20:00 UTC.

[2018-05-16 21:15 UTC] We still have approximately 2,500,000 messages remaining in our queue to process into files to be loaded into our data warehouse and approximately 2,600 files we are currently loading to the data warehouse. We anticipate that the remaining messages and files will be processed into the data warehouse by 2018-05-17 00:15 UTC at which point analytics will be available close to real-time (approximately 5-minutes behind from when they are collected).

[2018-05-17 03:28 UTC] Resolved. The backlog of messages and files required additional time to finish processing, but have completed as of 2018-05-17 02:17 UTC. Analytics up until the current date and time should now be available within reports. We will be continuing to monitor closely over the next few days to ensure that the analytics processing rate is operating at expected levels.