It’s one of the downsides of caching, since that page is shown when it can’t connect universally, as a client has no way of automatically detecting the difference.
I’d personally be in favor of enhancing the offline page, and giving some more details, and possibly with a button/link that uninstalls the service worker in the event that the visitor actually wants to remove it because they know the site’s gone.