My name is Silviu and I am a Technical Support Engineer (Tier II) at Okta. It depends on your use-case for each and every profile. You can consider the "deactivated" status as being a back-up for all the attributes, app assignments, groups related to that user, etc. In case you know for sure those users will not use those accounts anymore, then I can't see any reason for not deleting them once and for all. But if you have doubts about it, let's say consider re-hiring them after a period of time, or you used the "deactivated" status for those just to reflect some sort of a suspended mode, then you may keep those so you won't have to recreate them from scratch. Other than that, there are credentials stored for the chicklets (username and password stored for apps, in dashboard) which will be lost if you delete the accounts, but again, if you are sure those account will not be used anymore, then I don't see any reason for not having them deleted.
Wish you all the best in your work!
Thanks, Silviu Muraru Technical Support Engineer | Okta