Thanks @JammyDodger!
I somehow missed that when searching.
Though it appears that the email used for this setting must be listed publicly on the /about page.
I had left ‘contact_email’ empty (opting for using contact_url), which explains why I have not been receiving these emails.
I do think that the email used for administrative critical alerts shouldn’t be the same as the email listed publicly for support/general contact emails though.
One email would make more sense to go to a support ticketing system, while the other should go to the server admin.