Engagement Services Hotfix - OAuth API authorization

• Engagement services

  • Fix authorization errors (HTTP 401 responses) when using OAuth to invoke Engagement service APIs.

No downtime is expected while this maintenance is being performed. The scheduled maintenance is designed to mitigate disruptions to service availability and performance. However, it is possible for the impacted service(s) to be unavailable and/or performance degraded for a short period of time during the maintenance window.

Cloud Management Console Release

• Cloud Management Console

  • Stabilization and defect fixes

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for the impacted service(s) to be unavailable for a short period of time during the maintenance window.

Engagement and Identity Services Hotfixes

• Engagement services

  • Fix for issue where under some circustances push notifications were not being delivered

  • Other minor bugfixes

• Identity services

  • Stabilization and defect fixes

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for your service(s) to be unavailable for a short period of time during the maintenance window.

Cloud SSL Certificate Updates

• Cloud SSL certificates for Identity services (*.auth.konycloud.com), App services (*.konycloud.com), and Sync services (*.sync.konycloud.com)

• Note: If you have not pinned Kony certificates in your application, no application updates will be necessary. Customers that have pinned SSL certificates will need to download the new certificates and update their applications to trust old and new certificates and distribute their updated applications. The new certificates can be downloaded by executing the following commands:

  • Identity: openssl s client -showcerts -connect konycertificatepreview.auth.konycloud.com:443

  • App: openssl s client -showcerts -connect konycertificatepreview.konycloud.com:8443

  • Sync: openssl s client -showcerts -connect konycertificatepreview.sync.konycloud.com:9443

• Customers who have pinned the public key instead of the full certificate may not be required to update their applications. The updated certificates will have the same public keys as the existing certificates.

• If necessary, you can submit your applications for expedited approval (e.g., Apple has an expedited approval process for critical bugs, or in this case, pinned certificates).

Customer applications that have pinned SSL certificates will need to be updated as described above.

Security Exposures - CVE-2017-5715, CVE-2017-5753, CVE-2017-5754

• AWS Elastic Compute instances

  • Speculative execution vulnerabilities: CVE-2017-5715, CVE-2017-5753, CVE-2017-5754.

  • Kony Cloud and our AWS Custom Hosting teams are addressing the above mentioned security vulnerabilities that were made public over the last few days; also colorfully referred to as ‘Meltdown’ and ‘Spectre’. Kony and AWS have already taken proactive steps to protect our customers’ systems on our AWS Cloud and AWS Custom Hosted environments. While the initial risk has been addressed, there will be additional steps taken over the coming days to further implement protections as various O/S patches are released or refined.

  • To be more granular, foreign code must be executing directly on our servers, in some cases with high privilege, to be able to execute the attack. Kony is not a generic hosting platform nor do we run any unknown code in the Kony management infrastructure. Customers could potentially upload code for their specific application servers that would execute such an attack, but that code is already intended to process customer data and so there is little to gain in doing so. There are no Kony credentials or private data that would be accessible from these systems. Also, each customer environment is deployed into a separate AWS account so there is no possibility of reaching across systems or VMs to access any other customer’s data.

  • In addition to AWS already taking necessary action to protect the EC2 fleet, Kony has reviewed these CVEs to assess the risk to customer execution environments. We have determined that, in concert with the AWS actions, the risk is extremely low that any attacker would be able to exploit these vulnerabilities. In line with our normal security procedures, Kony will be applying additional patches to operating systems over the next 30 days to further strengthen protections. Customer instances that cannot be automatically patched by Kony will be scheduled for necessary maintenance. Customers will be individually contacted as necessary to schedule these maintenance windows.

Management Services Hotfix

• Management services

  • Fix EMM Store Android 7 Wrapping issue

  • Fix issue preventing login to download Launchpad

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for your service(s) to be unavailable for a short period of time during the maintenance window.

Management Services - Launchpad upgrades in non-US regions

• Management services

  • Upgrade Launchpad for non-US region customers

  • Mangement service users in US region will not be affected

  • If you would like to schedule your Launchpad upgrade prior to this maintenance window, please open a support ticket from the Cloud Management Console.

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for your service(s) to be unavailable for a short period of time during the maintenance window.

Identity Services Rollback to v8 - Expired token gateway exceptions

• Identity services

  • A handful of customers using older SDKs were experiencing a high frequency of invalid / expired token gateway exceptions have been encountered in the latest version that was deployed earlier today. We are rolling back to prior version of Identity services to mitigate these issues until our development teams can investigate further.

  • After further review, the invalid / expired token gateway exceptions were due to a recent clock drift issue that affected specific hardware. Kony mitigated the issue and has since replaced all of the systems running on the affected hardware.

  • The rollback of Identity services was proactive and done out of an abundance of caution. The clock drift issue led to expired claims tokens being returned from Identity service API calls. The V8 SP1 version of Identity was not a contributor to any of the issues observed. We will schedule the redeployment of V8 SP1 in the next maintenance window.

  • Until we replaced affected hardware, some customers also experienced issues when attempting to access the Cloud Management Console.

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for your service(s) to be unavailable for a short period of time during the maintenance window.

[2017-12-18 23:20 UTC] These exceptions appear to be occurring for customers who are on older SDKs.

[2017-12-19 15:27 UTC] After further review, the invalid / expired token gateway exceptions were due to a recent clock drift issue for specific hardware, which was acknowledged by our infrastructure provider. Our provider anticipates being able to deploy a fix in 48 hours. Once we understood that this was only affecting specific hardware, earlier today, we were able to reprovision onto new hardware that was not subject to the clock drift issue. Since the hardware was replaced at 04:00 UTC, the issue with system clocks affecting the validity of claims tokens has been resolved. Customers who had opened support tickets also confirmed that their applications were behaving properly after the hardware change.

[2017-12-19 19:25 UTC] Our review of existing systems has shown that the time drift patches have been applied to the affected hardware categories by the infrastructure provider.

[2017-12-19 19:35 UTC] Until we replaced the affected hardware, some customers also experienced issues when attempting to access the Cloud Management Console.

Engagement Services Hotfix

• Engagement services

  • Add new permission for dynamic user attribute creation and modification

No significant downtime is expected. The scheduled maintenance is designed for minimal disruption of service availability; however, it is possible for your service(s) to be unavailable for a short period of time during the maintenance window.