Have you read Understanding Discourse release channels and Configure a supported tracking branch to get Discourse software updates? I would describe the difference more as having access to the latest changes right away, or receiving them a little later. The latter can be very helpful for custom developments that need to be adapted first. Otherwise, I would prefer access to the latest features and fixes. Of course, this carries a risk of new bugs, but the version that was frozen three weeks ago also contains bugs which might have already been fixed in latest, but usually not significant enough to warrant backporting to the last release.
Also keep in mind that downgrading is not supported, so if you are on a version after the current ESR, you need to wait for the next ESR to be published.