March 25, 2022

General Information and Timeline

What happened?

On January 20, 2022, the Okta Security team was alerted that a new factor had been added to a Sitel customer support engineer’s Okta account. This factor was a password. Although that individual attempt was unsuccessful, out of an abundance of caution, we reset the account and notified Sitel, a third-party vendor that assists us in providing customer support. Sitel then engaged a leading forensic firm to conduct an investigation.

The following timeline outlines the key milestones relating to the January incident:

Timeline (times in UTC)

• January 20, 2022, 23:18 | Okta Security received an alert that a new factor was added to a Sitel employee’s Okta account from a new location. The target did not accept an MFA challenge, preventing access to the Okta account.
• January 20, 2022, at 23:46 | Okta Security investigated the alert and escalated it to a security incident.
• January 21, 2022, at 00:18 | The Okta Service Desk was added to the incident to assist with containing the user’s account.
• January 21, 2022, at 00:28 | The Okta Service Desk terminated the user’s Okta sessions and suspended the account until the root cause of suspicious activity could be identified and remediated.
• January 21, 2022, at 18:00 | Okta Security shared indicators of compromise with Sitel. Sitel informed us that they retained outside support from a leading forensic firm.
• January 21, 2022, to March 10, 2022 | The forensic firm’s investigation and analysis of the incident was conducted until February 28, 2022, with its report to Sitel dated March 10, 2022.
• March 17, 2022 | Okta received a summary report about the incident from Sitel.
• March 22, 2022, at 03:30 | Screenshots shared online by LAPSUS$
• March 22, 2022, at 05:00 | Okta Security determined that the screenshots were related to the January incident at Sitel.
• March 22, 2022, at 12:27 | Okta received the complete investigation report from Sitel.

FAQs

Why do we think this is constrained to these five days?

We are confident that the activity was constrained to this five-day period because the forensic report from Sitel’s vendor (a leading forensic firm) confirmed this time period, and we verified the time period by reviewing our own logs. 

Okta references a January 20 date when Okta Security noticed the attacker, but what happened from January 16 through January 20?

The five-day window was part of the conclusion reached in the forensic report by a leading forensic firm that Sitel commissioned. This is consistent with the activity Okta detected on January 20, as well as the screenshots provided by the threat actors on March 21, 2022.

On January 20, Okta saw an attempt to directly access Okta using a Sitel employee's Okta account. This activity was detected and blocked by Okta, and we promptly notified Sitel, per the timeline above.

Outside of that attempted access, there was no other evidence of suspicious activity in Okta systems.

Why did Okta not notify customers in January?

Okta would like to acknowledge that we made an error. Sitel is our service provider for which we are ultimately responsible. 

In January, Okta was not aware of the extent of the Sitel issue – only that we had detected and prevented an account takeover attempt and that Sitel had retained a third-party forensic firm to investigate. At that time, we didn’t recognize the risk to Okta and our customers. We should have more actively and forcefully compelled information from Sitel.

In light of the evidence we have gathered over the last week, it is clear that we would have made a different decision if we had been in possession of all the facts we have today.

What was the extent of the compromise that occurred in January, i.e., what data/information was accessed?

The compromise of Sitel was limited to a five-day window in January. The final forensic report that we received from Sitel on March 22, 2022, concluded that there was a five-day period, from January 16 to 21, 2022, during which an attacker had access to Sitel. This is consistent with the screenshots that the threat actor posted on March 21, 2022.

When assessing the potential extent of the compromise, it is essential to remember that, by design, Sitel’s support engineers have limited access. They are unable to create or delete users or download customer databases. Support engineers can facilitate the resetting of passwords and multi-factor authentication factors for users, but they are unable to choose those passwords. In other words, an individual with this level of access could repeatedly trigger a password reset for users, but would not be able to log in to the service.

As outlined in more detail on Okta's blog, we have identified the customers that may have been impacted and have already contacted them. There is no impact on Auth0 or AtSpoke customers, and no impact on HIPAA and FedRAMP customers.

What does Super User access mean, and what does it allow?

The potential impact to Okta customers is limited to the access that support engineers have.

The “Super User” application, as shown in the screenshots, is an internal Okta support tool that provides basic support access for customer support engineers; it is not synonymous with “superadmin.” This does not provide “god-like access” to all its users. This application is built with least privilege in mind, ensuring that support engineers are granted only the specific access they require to perform their roles.

These engineers are unable to create or delete users or download customer databases. Support engineers do have access to limited data, for example, Jira tickets and lists of users, as shown in the screenshots. Support engineers are also able to facilitate the resetting of passwords and MFA factors for users, but are unable to obtain those passwords.

Who was the 3rd party service provider?

Sykes Enterprises, Inc. (which was acquired by Sitel in September 2021) is the third-party service provider that provides customer support engineering on behalf of Okta. As of Saturday, March 26, we are no longer working with Sykes/Sitel and have terminated their account access.

How did Okta determine the number of customers potentially impacted?

In attempting to scope the blast RADIUS for this incident, our team assumed the worst-case scenario and examined all access performed by all Sitel employees to the SuperUser application over the five-day period during which Sitel was compromised. We have determined that the maximum potential impact is 366 (approximately 2.5% of) customers, which reflects the total number of Okta customers whose Okta tenant was accessed by Sitel during that time period. That number is overinclusive because it includes all of the legitimate Sitel customer support access during that time period. 
 

Are there specific means, such as logs, to detect whether any data was accessed or exfiltrated?

Okta is actively continuing our investigation, utilizing logs and other data sources.

What assurances exist that the issue has been remediated by our 3rd party service provider? 

Sitel confirmed to Okta that they have remediated this issue. 

Is it necessary to reset passwords?

Okta is confident in our conclusion that the Okta service has not been breached, and therefore, no corrective actions are necessary for our customers.

Okta is confident in this conclusion because Sitel (and therefore the threat actor who only had the access that Sitel had) was unable to create or delete users, or download customer databases. 

Support engineers are also able to facilitate the resetting of passwords and multi-factor authentication factors for users, but are unable to choose those passwords.

To take advantage of this access, an attacker would need to gain independent access to a compromised email account belonging to the target user.

How can it be determined that a specific Okta org was not impacted?

Okta has reached out to all customers who have been potentially impacted. Additionally, we have notified non-impacted customers.

What are the roles and responsibilities of an Okta Technical Support Engineer (TSE)?

Okta published a Knowledge Base article that describes the roles, responsibilities, and regular activities from the perspective of an Okta Technical Support Engineer (TSE) working as the primary contact with customers, resolving their support inquiries at Okta.

Where can information be found about auditing customer support actions in an Okta tenant using System Log?

Okta published a Knowledge Base article that provides details on how to audit the Okta System Log for any resets or updates to passwords and other factors by your Okta Administrator or by a TSE (Technical Support Engineer) using Okta’s customer support tool, Super User (SU). This article also explains how to search for queries for impersonation access by a TSE, which can only be granted by the Okta Administrator during an engagement with Okta Support. 

Where is additional information available?

Several assets have been published on okta.com/transparency to provide as much information as possible, including a video between CEO Todd McKinnon and CSO David Bradbury, a summary of the investigation, an article on security best practices for Okta admins, and a more detailed analysis of the role of a Technical Support Engineer. We will continue to post new resources on this page as they are available.