Have a policy against it, issue warnings, and start banning people who keep doing it? I am unsure why this needs to be a “feature”, when it is a site policy.
You could also lower the allowed time for old edits, I think out of the box we default to 3 months. Feel free to change that to 5 days or whatever you feel is better.