Kony Cloud status
Current status and incident report
Engagement Services Release
Maintenance window: June 4, 2018 00:01 to 04:00Workspace services
Supporting functionality for Cloud Management Console Release
Developer portal
Supporting functionality for Cloud Management Console Release
Impact Level : high
We expect that Workspace services will be unavailable for some portion of the maintenance window. No downtime is expected for Developer portal while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.
Cloud Management Console Release
Maintenance window: June 4, 2018 04:00 to 08:00 UTC Impacted Cloud services:Cloud Management Console
Ability to reupload swagger file for ApiProxy service.
Stabilization and defect fixes.
Impact Level : minor
No downtime is expected while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.
Engagement Services Release
Maintenance window: June 4, 2018 00:01 to 04:00Engagement services
Performance improvements in push message creation and sending to improve overall throughput.
Fix an issue where creation of segments was failing when done on the Adhoc and Campaign pages.
Fix an issue where deletion, modification, and activation fails for certain imported events.
Impact Level : minor
No downtime is expected while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.
Reporting Services - delayed processing
Incident window: May 25, 2018 13:15 to June 8, 2018 20:00 UTC Impacted Cloud services:Reporting Services
Delayed processing of analytics messages
Impact Level : medium
As communicated in our updates yesterday regarding database maintenance (https://www.kony.com/status/reporting-services-database-maintenance), we accumulated a backlog of analytics messages to process when ingestion of messages was paused. Analytics that are available within reports will not contain the entire set of analytics sent from May 24 and today until the backlog can be processed. Note that no analytics data that has been sent has been lost. We are continuing to monitor and will continue to provide updates until the backlog can be fully processed and that data can be made available within reports. At the current processing rate, we currently estimate that the backlog will be processed by May 26 16:00 UTC.
[2018-05-27 15:15 UTC] The backlog of analytics data has not yet been fully processed. At the current file count and processing rate, we estimate that it will be processed by May 30 13:00 UTC. We will continue to monitor and provide updates until the backlog has been fully processed and data can be made available within reports.
[2018-05-31 16:00 UTC] The backlog of analytics data has not yet been fully processed. At the current file count and processing rate, we estimate processing to complete by June 1 11:00 UTC. We will continue to monitor and provide updates until the backlog has been fully processed and data can be made available within reports.
[2018-06-07 19:25 UTC] The backlog of analytics data has not yet been fully processed and we have turned off message collection to perform maintenance on the underlying database servers. We will provide an update once the database maintenance has been completed and we have re-enabled message collection.
[2018-06-08 00:14 UTC] We have completed our database maintenance and have resumed message collection. We have observed that message collection is behaving as expected. We will continue to monitor and provide updates until the backlog has been fully processed and past data can be made available within reports.
[2018-06-08 11:20 UTC] The backlog of analytics data has not yet been fully processed. At the current file count and processing rate, we estimate processing to complete by June 8 20:20 UTC. We will continue to monitor and provide updates until the backlog has been fully processed and data can be made available within reports.
[2018-06-11 16:12 UTC] Resolved. The backlog of analytics data has been fully processed and should be available within reports. The file processing rate is operating at normal levels and we will be continuing to monitor.
Reporting Services - Database maintenance
Maintenance window: May 22, 2018 20:00 UTC to May 25, 2018 00:00 UTC Impacted Cloud services:Reporting services
Database maintenance
Impact Level : medium
We anticipate that performance for reporting services may be degraded during portions of this maintenance window and may be momentarily unavailable. Other services are not expected to be affected. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.
[2018-05-24 12:13 UTC] The maintenance window has been extended to apply additional changes that will improve reporting services performance. To apply these changes, we have paused the processing of analytics messages since May 24 04:00 UTC. We anticipate being able to resume ingestion of analytics in the next 4 hours, but will continue to provide additional updates.
[2018-05-24 21:50 UTC] The maintenance has been completed and the window has ended. We resumed analytics message processing at 21:00 UTC and anticipate that we will process the backlog of analytics messages within the next 16 hours.
Identity services, Workspace services, and Developer Portal Releases
Maintenance window: May 21, 2018 04:00 to 08:00 UTCIdentity services
Fix intermittent republish failure issue by skipping deletion of users stored in Identity
- Users stored in Identity were previously being deleted during republish
Add Access-Control-Expose-Header for ‘X-Kony-Integrity’ to response
- Users were previously unable to use Integrity feature in SPA apps due to this bug
Fix internal errors to improve stability and improve quality of internal logs
Fix to require app key and secret for custom Identity login in Swagger UI
- Swagger UI was previously not allowing app key and secret headers to custom Identity login to be provided.
Workspace services
Supporting functionality for Cloud Management Console Release</a>
Developer Portal
Supporting functionality for Cloud Management Console Release</a>
Impact Level : minor
Cloud Management Console Release
Maintenance window: May 21, 2018 04:00 to 08:00 UTC Impacted Cloud services:Cloud Management Console
Sample Code for Object services & verbs from Fabric Console
Enhancements to Test Functionality UX from Fabric Console
Stabilization and defect fixes
Fix Swagger file documentation content issues
Impact Level : minor
No downtime is expected while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.
Reporting Services - delayed processing
Incident window: May 14, 2018 14:16 UTC to May 17, 2018 02:17 UTC Impacted Cloud services:Cloud Reports and analytics
Delayed processing of reporting / analytics messages
Impact Level : minor
We have approximately 10,000 files that are slated for ingestion. This data is expected to be ingested by 16:00 UTC today, as a sync point. Once those files are processed, the ingestion of all queued events will be processed. We expect to be current on events by end of day today. More accurate ETA information will be posted in the next update.
[2018-05-14 16:56 UTC] The 10,000 files have been ingested and processed. Events prior to 2018-05-12 16:00 UTC should now be available within reports. We are performing additional database maintenance and then will begin processing data since 2018-05-12 16:00 UTC.
[2018-05-15 16:51 UTC] Events at least up to 2018-05-15 07:00 UTC should be available within reports. We are continuing to monitor closely until all remaining files have been processed. We will provide another update once completed. If we observe that file processing will require significant time to complete, we will also provide an ETA at that time.
[2018-05-16 11:21 UTC] We expect that all remaining files up to the current date and time will be processed by 2018-05-16 20:00 UTC.
[2018-05-16 21:15 UTC] We still have approximately 2,500,000 messages remaining in our queue to process into files to be loaded into our data warehouse and approximately 2,600 files we are currently loading to the data warehouse. We anticipate that the remaining messages and files will be processed into the data warehouse by 2018-05-17 00:15 UTC at which point analytics will be available close to real-time (approximately 5-minutes behind from when they are collected).
[2018-05-17 03:28 UTC] Resolved. The backlog of messages and files required additional time to finish processing, but have completed as of 2018-05-17 02:17 UTC. Analytics up until the current date and time should now be available within reports. We will be continuing to monitor closely over the next few days to ensure that the analytics processing rate is operating at expected levels.
Engagement Services Hotfix
Maintenance window: May 14, 2018 00:01 to 04:00Engagement services
Fix performance degradation issue
Impact Level : minor
No downtime is expected while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.