No `Referer:`, no embedding

AI-generated summary

Discourse embedding fails when the browser suppresses the Referer: header, resulting in a 400 Bad Request error. This issue is observed in private Firefox and Safari windows. AxelN notes that enabling “embed any origin” and having comments for only one embedding site does not resolve the issue.

AxelN suspects the problem may be related to the difference in domain names between the blog and the forum. However, he later finds a simpler solution by adding a <meta> tag to the embedding page, allowing cross-site embedding to work AxelN.

Gja suggests an alternative approach, proposing to accept requests that match specific conditions, including the presence of discourseEmbedUrl and Referer headers. However, AxelN cautions that this approach may not be robust enough, as Safari and Firefox/private browsers suppress the Referer header completely, and recommends adding the <meta> tag instead.

Hi,

We use Discourse embedding for Jekyll at New ROOT Web Site! - ROOT - works like a charm! Unless - the browser is suppressing the Referer: header, in which case we get a 400 Bad Request. Try yourself - if you switch to a private Firefox window (which suppresses referrers) then embedding is broken. Same for Safari which seems to be a bit stricter on handing out referrers.

I cross checked by looking at New ROOT Web Site! - Blog - ROOT Forum in a Firefox private window which gives me a HTTP 400; then editing the request to add Referer: https://root.cern/blog/new-web-site/ - and the 400 turns into a 200 happiness.

We do have “embed any origin” enabled. We have comments for exactly one embedding site. Is there a way we can make that work?

Cheers, Axel.

Yes, I can reproduce that.

It’s interesting that embedded comments at https://blog.codinghorror.com/the-cloud-is-just-someone-elses-computer/ work without issue in a Firefox private window. Could the issue be related to the domain? Your blog is on root.cern and your form is on root-forum.cern.ch.

Hi Simon,

Thanks for looking at this! It could very well be related to the difference in domain names: I could imagine that browsers are providing referrers for a request to the same domain, but not to a different domain. But while that’s interesting I don’t yet see what we can do to fix this - we won’t be able to use root.cern as the forum domain, and that cross-domain embedding seemed a key and super useful feature… Is this simply at risk these days, given improved privacy, or can you think of a way out?

Something I might want to try is reverse proxying from root.cern/forum to root-forum.cern.ch. That way, the embedding can be done as if it’s on the same host, and we keep a referrer…? I’ll be back :slight_smile:

Cheers, Axel

Hi,

I now have a way simpler solution. It’s something that should probably be called out in Embed Discourse comments on another website via Javascript :

For cross-site embedding, in the embedding pages, add either

  • <meta name="referrer" content="strict-origin"> with Path Allowlist set to /.* (because no path will be provided), or
  • <meta name="referrer" content="no-referrer-when-downgrade"> with the actual Path Allowlist.

As mentioned in Referrer-Policy header - HTTP | MDN, “There is effort from browsers in moving to a stricter default value,” and Discourse embedding relies on the old default for cross-host embedding.

Cheers, Axel.

Given that this is going to be the default browser standard from Chrome 85 onwards (A new default Referrer-Policy for Chrome - strict-origin-when-cross-origin  |  Blog  |  Chrome for Developers), should we look at an alternative here?

Can we accept requests which match all of the following three conditions

  • Request contains both discourseEmbedUrl and Referer
  • The referer is set to the origin of the discourseEmbedUrl
  • The discourseEmbedUrl matches one of the of the embedding rules

I think passing the discourseEmbedUrl would be easy to do for a majority of users.

Just a warning, Tejas - AFAIK without the meta tag, Safari and Firefox / Private suppress the Referer header completely. What you suggest might work for Chrome >= 85, but adding the meta tag seems more robust / general and I suspect is sufficiently trivial for most embeddings?