Perhaps check the timezone on the server to see when this is likely to occur for your install
Can you get more details on how long and exactly what times this occurs?
Also perhaps try running a backup manually, to see how long the backups take on your configuration and to see the effect for yourself.