Okta Access Gateway Log Download Fails Due to Size or Timeout Limits

Okta Access Gateway (OAG) log downloads fail when the log files exceed default size limits or when the polling process times out. The configuration and UI logging messages were added with the 2023.4 version. Administrators must increase the maximum allowed log size or the polling timeout values in the configuration file to resolve this issue. When downloading logs from the OAG Admin Console, Okta displays one of the following errors:

minimum.disk.space.not.available=Not enough disk space available on the server to generate log file

disk.space.not.available=Log File is too large, not enough disk space available on the server to generate log file

failed.logs.not.available.for.given.date=Log files are not available for download for the given date

could.not.prepare.tar.gz.file=Failed to prepare log file

failed.remote.log.preparation.event.creation=Failed to complete log download task on worker node(s)

failed.remote.log.file.download.failed=Failed to download logs from worker node(s)

failed.remote.log.preparation.task.timeout=Log file size in worker node(s) too large, failed to download logs from worker(s)

• Okta Identity Engine (OIE)
• Okta Classic Engine
• Okta Access Gateway (OAG)
• Log Download

The log files exceed the default maximum allowed log size or disk space requirements, or the polling process times out before the worker nodes finish generating the logs.

The File Size or Timeout Limits Require Adjustments in the Configuration File

Verify system utilization permits larger files, open the configuration file in the Okta Access Gateway shell, and increase either the file size or the timeout limits, depending on the encountered error, by following these steps:

1. Check the system utilization to ensure space permits and estimate the directory size by running the following commands:

   sudo df -h

   ls -lh /opt/oag/logs/all/ics_all.log*

   ls -lh /opt/oag/logs/archive/all/

   du -sh /opt/oag/logs/*

   NOTE:
   • The archive for a specific date on an appliance contains all logs from every directory under /opt/oag/logs/. Okta recommends using log forwarders and keeping minimal logging on the cluster. For more information, refer to Manage log verbosity.
   • Use the following workaround only as a temporary solution, and revert the changes once the log download succeeds.
2. Sign in to the Okta Access Gateway shell.
3. Open the configuration file using a text editor or run the sed command to update the values directly.
   • To open the file in a text editor, run the following command:

     sudo vi /opt/oag/bin/logPrepareConfig.sh

   • To update the values directly, run the following command:
     sudo sed -i -e 's/DISK_SPACE_REQUIRED=.*/DISK_SPACE_REQUIRED=20000000/' -e 's/MAX_ALLOWED_LOG_SIZE=.*/MAX_ALLOWED_LOG_SIZE=20000000/' -e 's/MAX_ATTEMP_TO_GET_REMOTE_STATUS=.*/MAX_ATTEMP_TO_GET_REMOTE_STATUS=150/' -e 's/SLEEP_BETWEEN_ATTEMPTS=.*/SLEEP_BETWEEN_ATTEMPTS=2/' /opt/oag/bin/logPrepareConfig.sh
4. For file size errors on all Okta Access Gateway nodes, increase the MAX_ALLOWED_LOG_SIZE and DISK_SPACE_REQUIRED values to 20000000 (20GB) if using the text editor.
5. For timeout errors on the Admin node, increase the MAX_ATTEMP_TO_GET_REMOTE_STATUS value to 150 (a 5-minute timeout) if using the text editor.
   NOTE:
   • Okta does not recommend increasing the SLEEP_BETWEEN_ATTEMPTS value.
   • Increasing the MAX_ATTEMP_TO_GET_REMOTE_STATUS on the worker nodes does not change the behavior.

How are the default settings restored?

Run the following command on any node to revert the configuration file to the default settings.

How is the log preparation process monitored?

Set the global log level to info in the Admin Console, trigger the log download, and monitor the log preparation process using the tail command by following these steps.

1. Navigate to the Backups and logs tab, select Log storage, and set the Global verbosity to info.
   
2. Run the following command to view the logs:

   tail -F /opt/oag/logs/all/ics_all.log | grep LOG_PREPARE

3. Prompt the log download from the Admin Console.
4. Observe the log messages to verify the polling status.

   LOG_PREPARE_OPERATION SERVICE LOG_PREPARE_OPERATION INFO Polling log file on node myserver.lab, status=2

   NOTE: A status of 2 indicates that the log file generation on the target server has not completed. The process continues until it reaches the MAX_ATTEMP_TO_GET_REMOTE_STATUS value or observes a status of 0. A status of 0 indicates completion. If the process fails with a Failed.remote.log.preparation.task.failed error, access the last polled node to investigate the failure.