It’s not that easy to be honest.
You need to either make a custom javascript file to check if “Oops! That page doesn’t exist or is private.” exists, or change the 404 template to fire an event. So no, this is not out of the box and also it’s a bit fragile.
I would agree that this feature is useful; especially in the days directly after a migration.