Yeah. I always recommend just doing a clean install of 20.04 and moving to a clean install. As you suggest, things can go wrong and there is lots of downtime for an OS upgrade, even when everything goes just right. Lots faster to just move to a new server.