Turns out it was a ridiculously simple mistake - apparently there is a “Save Embedding Settings” button at the very far bottom left of the screen on the settings. Even though it looks like clicking the blue check mark next to the host does the “save”, you still need to find that save button and click it.
Additionally, once I did find it I was alerted to the fact that I needed to enter a username.
Honestly, that save button is really unintuitive - it is difficult to find, there is no penalty for not hitting it, the page seems to save my settings regardless (but not apply them I guess?) and none of the guides on this mention that I need to find it and hit it.