If the server storage is filled, services will not be able to run properly. This is typically due to logs filling up storage. Logging should be changed to a lower level so this does not happen again. The steps to change the logging level can be found here.

• Okta Access Gateway (OAG)

1. Login to the appliance shell.
2. Validate the storage on the server.

   sudo df -h

3. Navigate to the log storage location.

   cd /opt/oag/logs/all

4. Find the largest log files. The following command shows the log files greater than 1 MB. 

   sudo find . -type f -size +1M | sudo xargs ls -lrthS 

   The output will be sorted as per the size, with the largest one at the end. For example: 

   -rw-------+ 1 root root 1.4M Jul 15 15:11 ./audit/audit.log-20240715
   -rw-------+ 1 spgw spgw 2.6M Jul 15 16:20 ./all/ics_all.log-20240715
   -rw-r-xr--+ 1 spgw spgw 3.0M Jul 15 15:11 ./saml_engine/saml.log

5. The current logs can be cleared safely by using the following commands so the services can log again in the same file without any issues. NOTE: Logs once deleted or truncated cannot be retrieved. Please copy the logfiles to another server in case it needs to be preserved. 
   • To clear all logs from current log files:

     • sudo sh -c '> <filename>'
       
       Using filename from the example output above:
        
       sudo sh -c '> ./saml_engine/saml.log'

       For the rotated logs, i.e., logfiles appended with "-<date>" or the ones in  "/opt/oag/logs/archive" - a truncate will not help much as the services will no longer be writing to it. So, to clear out the space, such logs can be deleted. 
   •  To clear logs from a specific date: 

     • sudo sed -i  '/^2024-04-09\|2024-04-10/d' ics_all.log

       In the example above, all log messages starting from 2024-04-09 and 2024-04-10 will be deleted. The command will filter out and write the logs to a new file, so it may not help when the disk is 100% full.