Can I just tar the whole /var/discourse and run it on a new server?

I think you’ll need to delete the .tar file left in /var/discourse/shared/standalone/backups/default

It sounds like you have more uploads than will fit in a backup. You’ll need to (1) get a bigger disk, (2) move assets to Spaces or S3, (3) move uploads to a volume, (4) don’t backup uploads

The simplest immediate solution, after you delete the .tar file, is to not backup uploads.