Cannot get embedding to work

support
1

Hello, I’m trying to embed Discourse comments on my website, following the embedding guide, and I’ve run into a wall :frowning:

Symptoms

I’ve tried in Firefox and Chrome. In both cases, it loads the “Loading Discussion…” discourse iframe but hangs there, with recurring JavaScript errors in the developer console.

On Firefox, I get an error about X-Frame-Options:

Invalid X-Frame-Options header was found when loading “https://discourse.29th.local/embed/comments?embed_url=https%3A%2F%2Fpersonnel.29th.local%2F%23enlistments%2F11927”: “ALLOWALL” is not a valid directive.

Followed by a DOMException error in embed-application.js:7:

Uncaught DOMException: An invalid or illegal string was specified

These two errors repeat every 30 seconds or so. There are no failed requests in the network tab.

In Chrome, I don’t get the X-Frame-Options error. After a few seconds, I get an error about the target origin not matching the recipient window’s origin:

Failed to execute 'postMessage' on 'DOMWindow': The target origin provided ('https://discourse.29th.local') does not match the recipient window's origin ('https://personnel.29th.local').

I’ve seen plenty of topics on meta about this error, and I’ve tried all the troubleshooting steps, to no avail.

My setup

I followed the discourse for mac setup guide with a slight exception: instead of installing postgres, redis, and mailcatcher globally on my laptop, I have them running in docker containers, with the ports publicly exposed. Discourse has no idea they’re running in docker containers instead of bare metal. Rails/discourse is installed globally, and not running in a docker container.

Entirely separately, my custom web application is running in a docker compose stack. Part of that stack includes an nginx server which routes personnel.29th.local to the appropriate upstream container, and discourse.29th.local to host.docker.internal:3000 (that’s the magic hostname that docker containers can use to reach the host’s localhost).

(As I mention below, I’ve removed the nginx layer from the equation and ended up with the same error)

One possible gotcha here is that my web app is a JavaScript single page app. The page where discourse comments are being embedded is https://personnel.29th.local/#enlistments/1234 and there is no server-side rendering. If that were a problem, I would expect an error with the crawler, at which point I’d settle for Discourse simply linking to my app, rather than crawling it. But the errors it shows don’t appear to relate to crawling failures.

Troubleshooting

I’ve set the embeddable host in Admin > Customize > Embedding to personnel.29th.local. At first, the sample embed code showed http://localhost:3000/ for the discourseUrl, so I fired up rails console and ran:

SiteSetting.force_hostname = "discourse.29th.local"
SiteSetting.port = 443

And turned on “force https” in the admin dashboard. This fixed the url in the sample embed code.

screenshot of admin dashboard embed page
screenshot of admin dashboard embed page1116×507 57 KB

I also added https://personnel.29th.local as a CORS domain in the settings’ cors origins section.

I’m now starting discourse with the following command:

DISCOURSE_DEV_HOSTS=discourse.29th.local,host.docker.internal DISCOURSE_ENABLE_CORS=true bundle exec rails server

I also tried disabling the Content Security Policy in the settings dashboard.

I’ve looked on https://discourse.29th.local/logs/ but I’ve seen no errors, and nothing about Sidekiq.

On the Sidekiq note, I do have a message on the admin dashboard about updates:

A check for updates has not been performed. Ensure sidekiq is running.

So I ran Sidekiq.redis { |r| puts r.flushall } in the rails console and got OK, restarted the rails server, and no change to the message nor the overall problem. I’ve poked around in the redis cache and I don’t see anything about this page.

I’ve also attempted to simplify things by taking the nginx layer out of the equation: reverting SiteSetting.force_hostname and SiteSetting.port to nil, turning off force https, accessing my web app and discourse via localhost, and adding my web app to Discourse’s embeddable hosts and CORS hostnames (http://localhost:8080), but I got the same error, just with different hosts:

Failed to execute 'postMessage' on 'DOMWindow': The target origin provided ('http://localhost:3000') does not match the recipient window's origin ('http://localhost:8080').

I’m running version 2.6.0.beta6 ( 60bc38e6a8 ), which I got by cloning the master branch as per the discourse for mac setup guide a couple weeks ago and running git pull origin master today.

I’ve also removed the tmp directory and restarted the server.

I’ve also gone for a walk, shouted into a pillow, and cried under my desk.

Hopefully this covers all the bases. I hope someone can help!

2

I’m sorry to hear you’re having such a tough time setting it up.

Discourse is not smart enough to crawl a SPA so this sounds the fishiest to me. Can you try and reproduce with a site that contains static content?

It is impossible for us to support every bespoke install, so I recommend you try simplifying your stack further until it works, then add on pieces from there.

3 Likes
3

Thanks for your reply, and no worries I know it will be worthwhile!

I’ve simplified things to use a server-rendered (rails) site and omit the nginx layer all around. I have my app running on port 3001 and discourse running on port 3000.

My embed code is rendered as this:

<script type="text/javascript">
      DiscourseEmbed = { discourseUrl: 'http://localhost:3000/',
                         discourseEmbedUrl: 'http://localhost:3001/enlistments/1' };
    
      (function() {
        var d = document.createElement('script'); d.type = 'text/javascript'; d.async = true;
        d.src = DiscourseEmbed.discourseUrl + 'javascripts/embed.js';
        (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(d);
      })();
</script>

I added localhost:3001 to the embeddable hosts in Admin > Customize > Embedding and http://localhost:3001 to the hostnames in Admin > Settings > cors.

The error is the same, but with the hostnames updated:

Failed to execute 'postMessage' on 'DOMWindow': The target origin provided ('http://localhost:3000') does not match the recipient window's origin ('http://localhost:3001').

The stack is about as simple as it gets now :thinking: I assume that means it’s a configuration issue of some sort? Any ideas?

4

Some further debugging findings:

I manually created a topic and replaced discourseEmbedUrl: 'http://localhost:3001/enlistments/<%= @enlistment.id %>' with topicId: 14 in my JavaScript snippet, and it loaded the comments. That suggests it’s not a CORS or X-Frame- issue, but (a) something to do with scraping, and (b) possibly an issue with how errors are handled whilst embedding.

To investigate the scraping issue, I visited a new page that I hadn’t tried to access before (and therefore no scraping should have been attempted on it). I watched my app’s rails console and loaded the page. I saw /enlistments/6 in the log once. I waited until the error message was thrown in the JavaScript console, by which point Discourse should have attempted to scrape it, but my app’s rails console did not show any other access attempt logs.

There were no errors in discourse’s /logs endpoint, nor any I could make out in discourse’s rails log.

I thought maybe Discourse couldn’t access my website, so I logged into the rails console on my discourse app and ran:

± |master U:3 ?:2 ✗| → rails c
Loading development environment (Rails 6.0.3.3)
[1] pry(main)> require "net/http"
=> false
[2] pry(main)> url = URI.parse("http://localhost:3001/enlistments/6")
=> #<URI::HTTP http://localhost:3001/enlistments/6>
[3] pry(main)> req = Net::HTTP.new(url.host, url.port)
=> #<Net::HTTP localhost:3001 open=false>
[4] pry(main)> res = req.request_head(url.path)
4=> #<Net::HTTPOK 200 OK readbody=true>
[5] pry(main)>

As I did that, I saw the access log in my app’s rails server. This confirmed Discourse can reach my app.

Now I’m thinking it must be an issue with sidekiq or job scheduling? :man_shrugging: I’m not sure how to debug that, though. I’ve never used sidekiq before.

I took another look through the redis data (using TablePlus, a database GUI that works with redis), and I’m seeing about 3 rows with a key of values like default:logster-env-96404aef1da0c422fc32e3bb82d85fbc and a value with values like

[
  {
    "hostname": "myhostname",
    "process_id": 7188,
    "application_version": "60bc38e6a8914a10341a32ff9909e69faa65ffef",
    "params": {
      "embed_url": "http: //localhost:3001/enlistments/11927"
    },
    "HTTP_HOST": "localhost:3000",
    "REQUEST_URI": "/embed/comments?embed_url=http%3A%2F%2Flocalhost%3A3001%2Fenlistments%2F11927",
    "REQUEST_METHOD": "GET",
    "HTTP_USER_AGENT": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.60 Safari/537.36",
    "HTTP_ACCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "HTTP_REFERER": "http://localhost:3000/embed/comments?embed_url=http%3A%2F%2Flocalhost%3A3001%2Fenlistments%2F11927",
    "time": 1606253787041
  }
]

the type is LIST and ttl is -1. I guess that means the job is getting initiated?

I’ve poked around in /sidekiq but I see no mention of this job nor any kind of queue called RetrieveTopic :frowning:

Definitely narrowing things down but could use a hand if anything comes to mind!

5

Hey @eviltrout, any ideas for further troubleshooting, now that I’ve simplified the setup to the basics?

6

It’s almost certainly a configuration issue. Perhaps your Discourse at localhost:3000 thinks it has a different hostname. You can check in a console using:

Discourse.base_url

Another thing to check is the sidekiq logs at /sidekiq.

7

@eviltrout I’ve added some debugging lines into the TopicRetriever library, and confirmed that invalid_url? is false (suggesting it’s valid). Discourse.base_url is indeed set to http://localhost:3000. I believe the RetrieveTopic job is failing silently somewhere, and I’m trying to track down where. There are no error logs in /logs, and /sidekiq has no reference to retrieving topics or logs of any sort.

8

Sorry, I have no more ideas at this time. I know the code is currently working in production so there is something environmental, or a plugin or setup issue.

1 Like
9

Hello, thanks for the investigation. I believe I have exactly the same issue (embed works for existing topic but fails at topic creation). This works on my production environment but not on my dev machine.

My setup is a docker stack, and I made sure everything is visible from both discourse and sidekiq. At this point, I’m starting to think that when Discourse tries to parse a URL (it also fails when onebox tries to pull a preview of a link in a post), it somehow depends on an external service that fails to see local instances… Would that be possible ?

@wilson29thid did you find anything on your side since then ?

10

I came across the same issue on my production machine, not my developer machine. Something makes me think that it doesn’t exactly matter as to which one you’re using.

1 Like
11

No, I’m afraid I never figured this out and ended up using discourse’s api to create a topic manually :pensive:

1 Like
12

I might actually do that too. The good thing is that you can control the amount of threads being created before people start commenting on your articles…