I did this in the past using the steps outlined by @Stephen plus one extra: disable login by email address and password. (I can’t remember the exact setting name off the top of my head).
We also changed a setting to help deactivate accounts more quickly when someone leaves the org (not perfect, but gets close):