Some of what you are seeing is by-design. LCM is more about the Deactivation of a User up in an App, and less about what happens when they are removed from a Group. (ie "User is deactivated in Okta, and also Deactivated as a result in any provisioning-supported App they were assigned." Most apps simply leave the User up in the app, but simply mark them Inactive (the idea there is Reactivation later on).

Lots of things in play, including making sure you have the Plus edition/subscription of Slack; Using Group Push, the below caveat which otherwise makes Group Push goodness not happen:

• Using the same Okta group for App assignments and for Group Push is not supported. To maintain consistent group membership between Okta and the downstream app, you need to create a separate group that is configured to push groups to the target app.

Also this

• If you're using Group Push Enhancements for the Slack app and see that updates are not pushed to Slack side, you need to do perform a Push Now for your group mapping. It will force-sync group memberships from Okta to Slack, so those users who are assigned to a group on Slack side but not assigned in Okta, may be removed.

Also quite common is for those Inactive users left up in the app, to have a bulk-delete/cleanup operation available that is run on the app's backend, but this varies by SP.

Finally, something most forget:

Group Push does not create Users in the app-- it only creates a Group in the App w the same name as the Group in Okta, and then populates it with List of Members (Names).

(This is probably the reason for the Group-Push Caveat above)

This almost sounds like a job for a Workflow (but can't tell where you want the User De-Provisioned: is it only up in the App? or in Okta as well?

If it's also in Okta, Workflow might do it.