Okta Access Gateway(OAG) Admin node does not have redundancy like worker nodes. This article shows the steps to restore the Admin node in the event of failure and when a snapshot is unavailable.

Starting from 2024.1.1 OAG admin backup will be sent to all the worker nodes under /opt/oag/backups/admin_backup . In case Admin server is no longer available then admins can restore the appliance using the same backup from worker. 

NOTE: 

• On a worker node, only the backup file from /opt/oag/backups/admin_backup can be used for restoration. Backups will be retained for 60 days under the same dir. 
• If the admin node is available, then the latest backup file from /opt/oag/backups can also be used.
• Do not transfer the backup file outside the OAG environment to any other non-OAG server. The archive will contain certificates and keys that can create security issues when moved outside the OAG environment.  

• Okta Access Gateway (OAG)

Prerequisites: 

• Restore the existing admin: 

  • Preserve the contents of the backup, if available,  from /opt/oag/backups in users $HOME through shell.

    sudo cp -pr /opt/oag/backups $HOME/backup_`date +%m%d%y`

  • In case backup files are not available on admin, then it can be copied from any worker node using oagha user. On the worker node switch to oagha user:

    sudo su - oagha

    Copy the latest backup file to the old admin node: 

    scp /opt/oag/backups/admin_backup/oagConfigBackup.<timestamp>.tar.bz2 <Admin_server>:/opt/oag/backups/admin_backup/

  • Login to the old admin node and preserve the file to avoid getting deleted through reset: 

    sudo cp -p /opt/oag/backups/admin_backup/oagConfigBackup.<timestamp>.tar.bz2  $HOME/oagConfigBackup.<timestamp>.tar.bz2

• Restore an existing worker as admin:

  • Preserve the contents of admin_backup from /opt/oag/backups/admin_backup in users $HOME through shell:

    sudo cp -pr /opt/oag/backups/admin_backup $HOME/admin_backup_`date +%m%d%y`

  • Enable oag-admin service:

    sudo /usr/bin/systemctl enable --now oag-admin 

• Restore a newly initialized OAG node:

  • If a new node needs to be used, make sure a system user has been created on that to copy the backup file. For instructions on how to set SCP files in the OAG appliance, refer to SCP Files from the OAG Appliance. 

Steps To Restore: 

1. Log in to one of the worker nodes through the launch shell. Copy (scp) the latest backup file from /opt/oag/backups/admin_backup to the node where restore needs to be executed. Skip this step if the node is an existing worker or old admin since files would have been copied as mentioned in the prerequisites above. To copy the file on the new host:

   scp /opt/oag/backups/admin_backup/oagConfigBackup.<timestamp>.tar.bz2 <scpuser@new_oag_node>:

2. Log in to the node where restore needs to be executed and run reset steps. 

3. Through shell login with sudo to set the correct file permissions required for restore: 

   sudo su

4. Execute restore using the following command. Replace <location> with the location where the backup file has been copied:

   nohup /opt/oag/scripts/oag_restore.sh -H -d -b -f <location>/oagConfigBackup.<timestamp>.tar.bz2 &

5. Monitor the progress through /opt/oag/logs/all/ics_all.log using the following command:
   

   sudo tail -f /opt/oag/logs/all/ics_all.log | grep OAG_RESTORE

<timestamp> <hostname> OAG_RESTORE CLEANUP CLEANUP INFO - [-] Cleanup successful

5. Once the process is completed, validate the applications and the Admin UI after mapping the domains to the node IP.

6. Add the worker nodes to the restored admin. 

Related References

• OAG Deployment
• Add host entry for an application
• Admin renomination
• SCP Files from the OAG Appliance