To prevent users from uploading future avatars, you can disable allow uploaded avatars and enable the selectable avatars site setting with a large selection.
I think you may need to upload a replacement placeholder in order for automatic deletion - the user record is still keeping the uploaded avatar image alive even though it’s not in use.
Upload files are named based on the SHA1 of the image content - that may help you to find the backing file?