This article outlines the functionality of Integrated Windows Authentication (IWA) and provides a structured approach to troubleshooting common post-deployment issues. Before performing these steps, ensure all client browser settings conform to the Configuring Desktop SSO documentation.

Overview of IWA and Okta Desktop SSO

IWA is a Microsoft technology that extends domain trust to third-party applications. The Okta IWA service uses Kerberos and NTLM authentication to allow users to log in to Okta services without entering a username and password. Successful authentication using IWA requires the domain to be accessible to the client via the network. If the Okta IWA server, DNS, or Okta AD Agent cannot be reached from the client, authentication fails.

NOTE: It is technically possible to extend IWA access to machines that do not have direct access to the domain network, but that is beyond the scope of this article. This configuration requires heavy development work to implement TLS / Schannel SSP along with private Communications Transport (PCT) protocol.  Any request to implement this type of authentication should involve a Professional Services engagement.

Standard domain-based authentication utilizes two primary methods, determined by the priority configured in Internet Information Services (IIS) Manager:

• Negotiate: This refers to Kerberos authentication and is required for a seamless Single Sign-On (SSO) experience.
• NTLM: This method serves as a challenge-response failover, prompting users for a username and password if Kerberos fails.

The priority of Authentication Providers is configured within IIS. The method with the highest priority will be used first for authentication. In most deployments with a new IIS install, Negotiate will be listed first and NTLM second.

NOTE: If IIS is configured to use Negotiate and authentication fails, the user will be authenticated via NTLM as a failover method.

• In Diagram 1 below, the Negotiate (Kerberos) authentication attempt is accepted, and the NTLM method is therefore not used. In Diagram 2, the Kerberos authentication fails, and NTLM is used to complete authentication.

Diagram 1	Diagram 2

Anonymous Authentication and IWA Failover

Anonymous authentication allows access to a site's public areas without requiring client credentials. Instead, IIS provides the stored credentials of the IUSR_machinename account to Windows.

When IIS controls the password for this account, it bypasses the standard Windows OS credential check. A sub-authentication DLL (iissuba.dll) informs Windows that the password is valid and logs the user in using a Network Logon. This specific mechanism can trigger a standard network logon prompt (similar to accessing a shared folder) if the authentication fails.

Kerberos and NTLM are used exclusively by Windows Authentication. If Anonymous Authentication is enabled at the Default Web Site level, IIS may attempt this IUSR_machinename logon instead of utilizing the expected Windows Authentication. The Okta IWA flow often fails with a 401 Access is Denied error if the failover from Anonymous Authentication to Windows Authentication does not execute properly.

NOTE: If Kerberos is functioning correctly, an administrator can disable Anonymous Authentication to ensure that Single Sign-On (SSO) attempts strictly utilize Windows Authentication.

IIS Settings to Validate

Verify the following configurations within the IIS Manager to ensure the environment supports the authentication flow:

• IWA Site > Authentication > Providers: Ensure Negotiate is at the top of the list and NTLM is at the bottom.
• IWA Site > Authentication > Advanced Settings: Set Extended Protection to Off. NOTE: Enabling this can impact Mozilla Firefox support.
• Default Web Site: Only Windows Authentication should be enabled.
• IWA Site: Both Windows Authentication and Anonymous Authentication should be enabled.

Troubleshooting Kerberos-Based Authentication Failures

Kerberos errors generate specific messages that assist in diagnosis. Wireshark is the most reliable tool for identifying these errors within authentication packets.

1. Launch Wireshark and use the following filter: dns || kerberos || ip.addr == <Target_Machine_IP>

1. Resolve the host name for the target system to an IP address by checking the HOSTS file, DNS, or the LMHOSTS file.
2. Ping the remote system to verify connectivity.
3. Attempt the IWA flow from the browser.
4. Stop the capture immediately after completion to avoid data pollution.
5. Inspect the Response items in the capture to identify specific Kerberos error codes.

Below are examples of Wireshark capturing packets while performing the above actions:

• Resolving the name

• Pinging the remote system:

• Negotiating Authentication using IWA flow:

The capture above indicates successful authentication protocol negotiation and remote system response. Select Response to present more details:

• Requesting a Kerberos ticket:

NOTE: Refer to the Kerberos errors in network captures article for descriptions of the most frequent errors that may occur when requesting a Kerberos ticket.

Service Principal Name (SPN) Configuration

Service Principal Names (SPN) uniquely identify services running on servers. An SPN is required for Okta IWA to link the network DNS name of the IWA server to the specific service account running the IIS Application Pool. If this mapping is missing or incorrect, the browser cannot retrieve a Kerberos ticket, forcing a fallback to a username/password prompt.

The SPN must be registered to the domain service account used as the Identity for the Okta IWA Application Pool (for example, oktaservice). Below is an example of an SPN used for an Okta IWA instance:

How to configure the SPN: Use the setspn utility to register both the short hostname and the Fully Qualified Domain Name (FQDN) of the IWA server. Open a command prompt with administrative privileges and run the commands below, replacing the hostname and service account names. The commands shown use the example host and SPN shown above:

• setspn -s HTTP/OKTAICE-IWA otkaice\oktaservice
• setspn -s HTTP/OKTAICE-IWA.oktaice.local oktaice\oktaservice

Either method below can be used to confirm that the SPN is correctly applied to the service account. The command shown references the service account example from above, which should be replaced with the appropriate value in the environment.

• Run the command: setspn -L oktaice\oktaservice. The output should list both the hostname and FQDN entries.
• Use ADSI Edit to locate the service account, select Properties, and view the servicePrincipalName attribute.

Once the SPN is manually set, disable Kernel-mode Authentication in the Windows Authentication configuration for the IWA web application:

1. Open IIS Manager and select the IWA Site.
2. Double-click Authentication.
3. Select Windows Authentication.
4. In the Actions pane (right side), select Advanced Settings.
5. Clear the Enable Kernel-mode authentication checkbox.

6. Select OK.