Issues setting up subfolder support in Discourse


(Mike King) #1

Hey all,

Thanks for all this thread as it has been educational and valuable in a lot of ways and I wonder if other people might be running into an issue that I haven’t been able to find an explicit answer to.

First, Discourse seems like a very elegant solution to my message board system needs so thanks for building it and making it open source. I recently purchased a 20 year old website with a forum that is custom built in ASP classic and I’m excited to move off of it onto something modern. So thank y’all for all the efforts you put into building a great product.

Second, I do want to be forthcoming and say, I’ve been involved in the SEO space for the last 11 years (developer for longer than that) so the whole “ARCANE SEO MAGICKS” and disbelief in how subdomains vs. subdirectories are handled by search engines makes me snicker. Based on my own experience, 100% of the time that we’ve had a client (of any size) move content from a subdirectory to a subfolder we see increases in Organic Search traffic for the content they’ve moved as well as the content on the root domain. This is largely because the 301 redirects from the moved content carry link equity with them which makes the root domain more authoritative and because the root domain tends to already have more links pointing to than most given subdomains. I agree that just because it’s on a on subdomain vs. subdirectory probably doesn’t make as much difference in and of itself, but the link equity is a key factor in the differences in performance and even Matt Cutts would agree with that. This is also the case if you move everything from the root domain to a subdomain with the right redirects. Conversely, the aforementioned site is on Shopify and we wanted to put WP as the root and reverse proxy shopify in the /store directory pursuant to this. Unfortunately, I found out the hard way that Shopify throttles IPs and because the reverse proxy requests everything through one IP, it kills the site if you get any meaningful amount of traffic. So we had to move the store to a subdomain and traffic has effectively tanked across the board. If @codinghorror or @neil have any interest, I’m happy to share the details and analytics on that and when we move back to the main domain once we make our switch to WooCommerce so ya’ll can see for yourself what the impact is.

Third, the real reason I’m posting is that I’m having some difficulty with the instructions above and all of the image and JS resources are 404ing:

Here’s the relevant portion of my NGINX configuration file:

location /forum/ {
    proxy_pass http://unix:/var/discourse/shared/standalone/nginx.http.sock:;
    proxy_set_header Host $http_host;
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    rewrite ^/(.*)$ /forum/$1 break;
    rewrite ^/forum/?(.*)$ /$1;
}

I made the changes to the config file myself because the site has a series of pretty involved rules in place for the various different things running to comprise it.

Here’s my app.yml (it pretty much matches what is on this page):

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/web.socketed.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  db_shared_buffers: "2048MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed
    
    env:
      LANG: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  UNICORN_WORKERS: 4

  ## TODO: The domain name this Discourse instance will respond to
  DISCOURSE_HOSTNAME: undergroundhiphop.com
  DISCOURSE_RELATIVE_URL_ROOT: /forum

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1 at example.com,user2 at example.com' <--- changed because it said I couldn't mentioned more than 2 users in a post
  DISCOURSE_DEVELOPER_EMAILS: 'mike at ughh.com'  # changed because it said I couldn't mentioned more than 2 users in a post

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  DISCOURSE_SMTP_ADDRESS: smtp.elasticemail.com
  DISCOURSE_SMTP_PORT: 2525
  DISCOURSE_SMTP_USER_NAME: blah
  DISCOURSE_SMTP_PASSWORD: "aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaaaaa"
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  LETSENCRYPT_ACCOUNT_EMAIL: mike at ughh.com # changed because it said I couldn't mentioned more than 2 users in a post

  ## The CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: //discourse-cdn.example.com

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec:
        echo "Beginning of custom commands"
  - exec:
        cd: $home
        cmd:
          - mkdir -p public/forum
          - cd public/forum && ln -s ../uploads && ln -s ../backups

  - exec: rails r "SiteSetting.long_polling_base_url='/forum/'"

  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  • exec: echo “End of custom commands”

When it runs I see references to /etc/nginx/conf.d/discourse.conf, but this file does not exist.

I, [2017-11-19T18:02:59.816376 #14]  INFO -- : Replacing (?-mix:server.+{) with limit_req_zone $binary_remote_addr zone=flood:10m rate=$reqs_per_secondr/s;
limit_req_zone $binary_remote_addr zone=bot:10m rate=$reqs_per_minuter/m;
limit_req_status 429;
limit_conn_zone $binary_remote_addr zone=connperip:10m;
limit_conn_status 429;
server {
 in /etc/nginx/conf.d/discourse.conf
I, [2017-11-19T18:02:59.816629 #14]  INFO -- : Replacing (?-mix:location at discourse {) with location at discourse { #     changed because it said I couldn't mentioned more than 2 users in a post
limit_conn connperip $conn_per_ip;
limit_req zone=flood burst=$burst_per_second nodelay;
limit_req zone=bot burst=$burst_per_minute nodelay; in /etc/nginx/conf.d/discourse.conf
I, [2017-11-19T18:02:59.819948 #14]  INFO -- : File > /etc/runit/1.d/remove-old-socket  chmod: +x
I, [2017-11-19T18:02:59.822705 #14]  INFO -- : File > /etc/runit/3.d/remove-old-socket  chmod: +x
I, [2017-11-19T18:02:59.822873 #14]  INFO -- : Replacing (?-mix:listen 80;) with listen unix:/shared/nginx.http.sock;
set_real_ip_from unix:;
 in /etc/nginx/conf.d/discourse.conf
I, [2017-11-19T18:02:59.823093 #14]  INFO -- : Replacing (?-mix:listen 443 ssl http2;) with listen     unix:/shared/nginx.https.sock ssl http2;
set_real_ip_from unix:; in /etc/nginx/conf.d/discourse.conf

Also, the custom commands are not executing correctly and I’m not seeing in any indication that they are executing (yes I’m running as root). For instance, cd $home is placing the script at /var/www/discourse which does not exist and then the mkdir also never executes. But, the root for the server is /var/www/html anyway.

I, [2017-11-19T18:02:59.823755 #14]  INFO -- : > echo "Beginning of custom commands"
I, [2017-11-19T18:02:59.824996 #14]  INFO -- : Beginning of custom commands

I, [2017-11-19T18:02:59.825228 #14]  INFO -- : > cd /var/www/discourse && mkdir -p public/forum
I, [2017-11-19T18:02:59.827254 #14]  INFO -- :
I, [2017-11-19T18:02:59.827322 #14]  INFO -- : > cd /var/www/discourse && cd public/forum && ln -s ../uploads && ln -s ../backups
I, [2017-11-19T18:02:59.830304 #14]  INFO -- :
I, [2017-11-19T18:02:59.830465 #14]  INFO -- : > rails r "SiteSetting.long_polling_base_url='/forum/'"
I, [2017-11-19T18:03:06.721146 #14]  INFO -- :
I, [2017-11-19T18:03:06.721357 #14]  INFO -- : > echo "End of custom commands"
I, [2017-11-19T18:03:06.723219 #14]  INFO -- : End of custom commands

So, I’d appreciate it if someone could let me know what I’m doing wrong. Naturally, I had no problems installing this on a subdomain and perhaps I may just need to set it up on a subdomain and reverse proxy it that way, but since I’m new to the software, I don’t if it will allow me to set rel-canonical tags back to the URL that I want.

Thanks!

-Mike


Subfolder support with Docker
(Sam Saffron) #2

I am confused, did you see the instructions in the first post, it has a rewrite rule in the internal nginx, not seeing that in the config you pasted


(Mike King) #3

Hey @sam, I’m certainly not above oversights or complete misunderstandings. :slight_smile:

Which ones do you mean? As I have these two:

rewrite ^/(.*)$ /forum/$1 break;
rewrite ^/forum/?(.*)$ /$1; 

Or do you mean that the first one is meant to go outside the location specification?

Thanks for taking the time to respond.


(Sam Saffron) #4

I mean the whole run section in the first post, not seeing it in your config you need to add it


(Jeff Atwood) #5

If your argument is that it’s easier to cheat by tacking a bunch of unrelated content on an existing website, I don’t think that’s relevant. Further reading:

and

https://www.websitemagazine.com/blog/seo-inquiry-subdomains-subdirectories

and

The problem with subdirectory is that it is a vastly more complex hosting scenario, and thus it is more expensive and more complex to maintain. It certainly messed you up, and that’s exactly why we don’t recommend it.


(Mike King) #6

That’s not my argument at all actually, but this a message board so of course you’d go in that direction.

My actual argument is that the link equity from external links pointing to the content on a subdomain are then consolidated on the root domain by 301 redirecting the target URLs from a subdomain to a subdirectory which thereby makes all of the content perform better. It’s essentially the fundamental concept of PageRank that is powering the improvement of all of those pages’ visibility.

John Mueller is often wrong about a lot of things. If I were you, I wouldn’t place my faith in a guy that says 301 and 302 redirects are treated the same. Michael Martinez is a known troll. What else you got? I encourage you to cite case studies rather than rhetoric.


(Mike King) #7

So, you mean this?

I did run it as is when I first got started, but as I said in my post /etc/nginx/conf.d/discourse.conf doesn’t exist and I’m not using fastly, so it’s not relevant.


(Sam Saffron) #8

I think you are misunderstanding how discourse docker works and how stuff is configured.

Those command run inside the container, the container itself runs nginx, so the commands need to run against the nginx file that is internal to the container during the bootstrap phase.

Note, this strongly reinforces why this is not a “recommended” setup for us. It requires understanding quite a lot about how Discourse is distributed and how to configure reverse proxies so you do not lose IP addresses.

For our hosting we always charge a surcharge on enterprise customers needing this cause it simply is a complicated setup.


(Mike King) #9

Got it. Unfortunately, it didn’t work when I ran it that way, but I appreciate the heads up.

FAILED
--------------------
Errno::ENOENT: No such file or directory @ rb_sysopen - /etc/nginx/conf.d/discourse.con
Location of failure: /pups/lib/pups/replace_command.rb:8:in `read'
replace failed with the params {"filename"=>"/etc/nginx/conf.d/discourse.con", "from"=>"etag off;", "to"=>"etag off;\nlocation /forum {\n   rewrite ^/forum/?(.*)$ /$1;\n}\n"}
39345f84a962846006de69a8d27743a9a18ee3de86b8fc0666bdda4c28e53f75
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

I’ll continue to tinker with it and will make a post for those that will inevitably run into this issue in the future.


(Mike King) #10

Thanks @sam for the clarification. That helped me figure out where I was going wrong.

Ok, just in case anyone else runs into this issue in the future. My issue definitely came down to my lack of understanding of the Docker container as @sam indicated. I did not understand that there was a separate server in the container that the nginx rules referred to. Those of you that will visit this thread after following these instructions as your key entry point into installing Discourse may potentially be as confused as I was since these instructions don’t mention Nginx at all. However, that may be inherent in the use of Docker. I obviously don’t know.

So you should be able to follow those instructions and these Nginx rules that @jmg provided and be all set.

location /forum/ {
    proxy_pass http://unix:/var/discourse/shared/standalone/nginx.http.sock:;
    proxy_set_header Host $http_host;
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    rewrite ^/(.*)$ /forum/$1 break;
    rewrite ^/forum/?(.*)$ /$1;
}

Also, in the latest error that I posted there was a copy and paste fail on my behalf wherein an f was left off of discourse.conf in one of the lines.


(Jeff Atwood) #11

And you, some random SEO guy on the internet, are somehow more credible? That’s… a stretch, to say the least.

Regardless, setting that aside, you’ve already proven my primary point:

I am shocked, shocked to learn that the additional (and considerable) technical complexity of subfolder caused problems for you. How could anyone have possibly predicted this outcome? :roll_eyes: