Refreshing fixes it, yes. But in this case it’s not a matter of using the wrong key, as I only used the key registered in icloud keychain. The prompt was just never shown to me at all.
Although I wasn’t able to trigger this with any browser other than safari.