Could it be possible my ad blocker plays into this?
When looking a the browser console after upgrading, I saw two entries:
/message-bus/fc3f44b30aea4975be751a4cc8736f76/poll:1 Failed to load resource: net::ERR_HTTP2_PROTOCOL_ERROR
docker/upgrade:1 Failed to load resource: the server responded with a status of 504 ()
I see the same. Indeed, whatever the ‘Start Upgrading’ button turns into when the upgrade starts - I think it used to change into some kind of Cancel button - I notice it pretty quickly went back to ‘Start Upgrading’ while the upgrade was still in progress, and remained there even after the upgrade was successfully completed. I’m running Chrome Version 86.0.4240.80 (Official Build) (x86_64) on a Mac.
I saw this on a previous upgrade. The present upgrade is to 2.6.0.beta4 from 2.6.0.beta3. I think the immediately previous upgrade required an app rebuild.
Edit: on my other forum, I just saw the same, and took some screenshots. The button flipped back while the log was still showing one of the many Waiting for Unicorn to reload messages at the beginning. (I speculate that the flip is based on time having elapsed, not to do with the progress of the upgrade. Edit: 60 seconds from button press to reversion. Whereas the progress bar first moved at about 90 seconds.) See below:
Rebooting the machine (perhaps just the Docker container) or restarting redis will fix it, I think. We don’t have clear repro steps for this but I have seen it as well, so it definitely happens.
It’s mostly cosmetic so not a high priority. If we had clear consistent repro steps…
I was able to reproduce this. The client code hasn’t changed in a meaningful way. It looks like the problem is that a call to /admin/docker/upgrade is now throwing a 504 gateway timeout.
Our error handler then tells it to mark its state as “not upgrading”, which means when the message bus notification for complete comes through that it’s not marked as finished.
It seems to me the root cause here is the 504 timeout that we didn’t see before. I suspect some kind of proxy/rails change here. Maybe something in our docker image? @sam are you aware of anything or can maybe assign to dev ops?
I can see some odd things happening on the client that are leading to a bit of a surprise:
Why are we making any HTTP requests to the server during upgrade? We appear to be making a request to /admin/docker/upgrade mid way through upgrade which is confusing to me. We should simply be waiting on the message bus. I ran this in firefox so I have limited debugging.
Messagebus is not long polling, it is only short polling leading to rate limits
“Go to next upgrade” is a bit confusing, we should just say done when you are done upgrading a piece instead of quickly switching you to the other piece.
@Osama can you spend some time debugging / refining / upgrading ember and so on? I think the majority here appears to be client work.
Note to people on this topic, we will get this sorted but it will probably take 2-4 weeks. Since this has been going on for a month now I think we can wait a bit.
I’ve never seen that - is it part of ‘Upgrade All’?
That’s fine for me, of course. I thought I saw someone say that they’d accidentally clicked Start Upgrading when it reappeared - hopefully that’s harmless. I’ve never used ‘Reset Upgrade’ but presumably it’s a safety feature in case the upgrade gets stuck.