You will need to rebuild via the command line. I don’t know why this caching issue exists. I have a default implementation with very little customisation. My guess is something historically has broken the ability to clear caches containing CSS when upgrading via the UI.
Last time this happened, I didn’t rebuild via the command line and the issue cleared up on a subsequent upgrade some weeks later.