Root Cause Analysis:
Okta DNS Disruption - October 21st, 2016
On October 21st, 2016, at approximately 4:10am PDT, customers hosted in Okta's US infrastructure began experiencing intermittent connectivity, authentication, MFA, and API issues as the result of a distributed denial-of-service (DDoS) attack against Okta’s primary Domain Name Service (DNS) provider Dyn. Many customers reported a partial or complete outage in accessing the Okta service. Following Okta's remediation of the DNS provider issue, some customers continued to experience residual connectivity issues due to a server overload condition, locally cached DNS values, or connection issues to Verizon.
Beginning at 4:10am PDT on October 21, 2016, customers hosted in Okta's US service infrastructure experienced intermittent connectivity issues when attempting to connect to the Okta service. Okta immediately began investigating the connection failures.
At approximately 5:40am PDT, Okta identified the root cause of the issue as a distributed denial-of-service (DDoS) attack against the Domain Name Service (DNS) provider Dyn.
At 6:12am PDT, Okta successfully migrated Oktapreview DNS records to a secondary DNS provider unaffected by the DDoS attack to validate the corrective action plan. At 6:18am PDT, Okta migrated all remaining US infrastructure DNS records to this secondary DNS provider. Following the completion of the DNS provider migration, a subset of customers continued to experience connectivity issues resulting from one of three different root causes:
Ultimately, our architectural and vendor choices are our responsibility and we understand that any disruption in service can have a significant impact for our customers. In order to protect our customers and mitigate the impact to our infrastructure in the event of a similar attack, Okta is in the process of implementing the following preventative measures:
The statements contained in this article that are not purely historical are forward-looking statements, including statements regarding Okta's future operating results, long-term business prospects, future product acceptance, and expectations, beliefs, intentions or strategies regarding the future. All forward-looking statements included in this article are based upon information available to Okta as of the date hereof, and Okta assumes no obligation to update any such forward-looking statements. Forward-looking statements involve risks and uncertainties, which could cause actual results to differ materially from those projected. The forward looking product roadmap does not represent a commitment, obligation, or promise to deliver any product and is intended to only outline the general product development plans. Customers should not rely on roadmaps to make a purchasing decision.