This is an issue even if the proxy is sending the X-Forwarded-Proto header and all.
Toggling the force_https site setting will not update the logo URL protocol until the icon_manager cache has been cleared. Adding force_https to the watched settings for the icon manager is probably the easiest solution.