Uploads broken with subfolder installation

bug
1

Similar to Uploaded avatars and Gravatar not working with subfolder installation…

All uploads are broken on my subfolder install. The uploads make it into the actual uploads directory, but when rendering the posts, all images get src="".

Making a post…
https://i.imgur.com/ofOUY4e.png

After posting…
https://i.imgur.com/EBmnD6e.png

Somewhat amazingly, if I then move to a different browser (chrome now), pull up the topic (where the image is still broken), but then click edit, the image renders in the edit preview once again!

https://i.imgur.com/3rQirhc.png

This confirms it’s successfully uploading to the server, which I’ve verified:

root@cs6991:/var/discourse# ./launcher enter app
x86_64 arch detected.
root@cs6991-app:/var/www/discourse# ls 'public/~cs6991/forum'
backups  uploads
root@cs6991-app:/var/www/discourse# ls 'public/~cs6991/forum/uploads'
default
root@cs6991-app:/var/www/discourse# ls 'public/~cs6991/forum/uploads/default/original/1X/'
08335563eac3a393e60a902d4d38cffdfa6d967d.png  3eee67e6460792667bab4f2248ad4643be4feae3.png
29e403dabcfee32379629fb6d844354193e278ba.png  42ecfcb27b534acc9f3436fa7d291c2fca106e57.png

But it just doesn’t seem to be rendering on the actual page.
Same issue occurs with other uploads, such as avatars.

Some info:

Subfolder: /~cs6991/forum

app.yml

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "80:80"   # http
  - "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  db_shared_buffers: "128MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  UNICORN_WORKERS: 2

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'cgi.cse.unsw.edu.au'

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: '<<REDACTED>>'

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: email-smtp.ap-southeast-2.amazonaws.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: <<REDACTED>>
  DISCOURSE_SMTP_PASSWORD: <<REDACTED>>
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  #DISCOURSE_SMTP_DOMAIN: discourse.example.com    # (required by some providers)
  DISCOURSE_NOTIFICATION_EMAIL: discourse@cs6991.email    # (address to send notifications from)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com

  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

  DISCOURSE_RELATIVE_URL_ROOT: '/~cs6991/forum'

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec:
      cd: $home
      cmd:
        - mkdir -p public/~cs6991/forum
        - cd public/~cs6991/forum && ln -s ../../uploads && ln -s ../../backups
  - replace:
     global: true
     filename: /etc/nginx/conf.d/discourse.conf
     from: proxy_pass http://discourse;
     to: |
        rewrite ^/(.*)$ /~cs6991/forum/$1 break;
        proxy_pass http://discourse;
  - replace:
     filename: /etc/nginx/conf.d/discourse.conf
     from: etag off;
     to: |
        etag off;
        location /~cs6991/forum {
           rewrite ^/~cs6991/forum/?(.*)$ /$1;
        }
  - replace:
       filename: /etc/nginx/conf.d/discourse.conf
       from: $proxy_add_x_forwarded_for
       to: $http_your_original_ip_header
       global: true
  - exec: echo "End of custom commands"

Everything else, as far as I can tell, appears to be working correctly – just upload rendering that’s acting rather peculiar.

I’ve verified this behaviour on a completely fresh build – that is, rm -rf /var/discourse, completely nuking docker, and following cloud install + subfolder instructions.

If there’s further investigation I can pursue, I’m happy to take those steps. (sorry for imgur links – I’m not allowed 2+ image embeds here yet!)

Cheers!

1 Like
3

Some extra info – looks like the src is already stripped away before rendering, as it is missing in the production database:

# sudo -u postgres psql discourse
discourse=# select * from posts where id=13;
 id | user_id | topic_id | post_number |                             raw                             |                                                  cooked                                                  |        created_at         |        updated_at         | reply_to_post_number | reply_count | quote_count |         deleted_at         | off_topic_count | like_count | incoming_link_count | bookmark_count | score | reads | post_type | sort_order | last_editor_id | hidden | hidden_reason_id | notify_moderators_count | spam_count | illegal_count | inappropriate_count |      last_version_at       | user_deleted | reply_to_user_id | percent_rank | notify_user_count | like_score | deleted_by_id | edit_reason | word_count | version | cook_method | wiki |          baked_at          | baked_version | hidden_at | self_edits | reply_quoted | via_email | raw_email | public_version | action_code | locked_by_id | image_upload_id
----+---------+----------+-------------+-------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+---------------------------+---------------------------+----------------------+-------------+-------------+----------------------------+-----------------+------------+---------------------+----------------+-------+-------+-----------+------------+----------------+--------+------------------+-------------------------+------------+---------------+---------------------+----------------------------+--------------+------------------+--------------+-------------------+------------+---------------+-------------+------------+---------+-------------+------+----------------------------+---------------+-----------+------------+--------------+-----------+-----------+----------------+-------------+--------------+-----------------
 13 |       1 |        7 |           2 | ![ferris|690x459](upload://5YA5Y9vjz0iQmn2DErtUBrHCKng.png) | <p><img src="" alt="ferris" data-base62-sha1="5YA5Y9vjz0iQmn2DErtUBrHCKng" width="690" height="459"></p> | 2022-09-01 19:30:38.97281 | 2022-09-01 19:30:38.97281 |                      |           0 |           0 | 2022-09-01 19:47:34.612042 |               0 |          0 |                   0 |              0 |   0.2 |     1 |         1 |          2 |              1 | f      |                  |                       0 |          0 |             0 |                   0 | 2022-09-01 19:30:38.993775 | f            |                  |          0.5 |                 0 |          0 |             1 |             |          5 |       1 |           1 | f    | 2022-09-01 19:30:38.972751 |             2 |           |          0 | f            | f         |           |              1 |             |              |
4

Also from the network tab creating a reply with an image:

raw	"My+image+is+inserted+next:\n\n![ferris|690x459](upload://5YA5Y9vjz0iQmn2DErtUBrHCKng.png)\n\n\nThe+image+is+above."
unlist_topic	"false"
category	"4"
topic_id	"7"
is_warning	"false"
archetype	"regular"
typing_duration_msecs	"7500"
composer_open_duration_msecs	"14116"
featured_link	""
shared_draft	"false"
draft_key	"topic_7"
image_sizes[https://cgi.cse.unsw.edu.au/~cs6991/forum/uploads/default/original/1X/29e403dabcfee32379629fb6d844354193e278ba.png][width]	"1200"
image_sizes[https://cgi.cse.unsw.edu.au/~cs6991/forum/uploads/default/original/1X/29e403dabcfee32379629fb6d844354193e278ba.png][height]	"800"
nested_post	"true"

or the raw query string if preferred:

raw=My+image+is+inserted+next%3A%0A%0A!%5Bferris%7C690x459%5D(upload%3A%2F%2F5YA5Y9vjz0iQmn2DErtUBrHCKng.png)%0A%0A%0AThe+image+is+above.&unlist_topic=false&category=4&topic_id=7&is_warning=false&archetype=regular&typing_duration_msecs=7500&composer_open_duration_msecs=14116&featured_link=&shared_draft=false&draft_key=topic_7&image_sizes%5Bhttps%3A%2F%2Fcgi.cse.unsw.edu.au%2F~cs6991%2Fforum%2Fuploads%2Fdefault%2Foriginal%2F1X%2F29e403dabcfee32379629fb6d844354193e278ba.png%5D%5Bwidth%5D=1200&image_sizes%5Bhttps%3A%2F%2Fcgi.cse.unsw.edu.au%2F~cs6991%2Fforum%2Fuploads%2Fdefault%2Foriginal%2F1X%2F29e403dabcfee32379629fb6d844354193e278ba.png%5D%5Bheight%5D=800&nested_post=true`

the response seems to echo the post back, and you can see the src is already stripped by that point:

{
  "action": "create_post",
  "post": {
    "id": 16,
    "name": null,
    "username": "z.kologlu",
    "avatar_template": "/~cs6991/forum/letter_avatar_proxy/v4/letter/z/b9bd4f/{size}.png",
    "created_at": "2022-09-02T05:37:25.680Z",
    "cooked": "<p>My image is inserted next:</p>\n<p><img src=\"\" alt=\"ferris\" data-base62-sha1=\"5YA5Y9vjz0iQmn2DErtUBrHCKng\" width=\"690\" height=\"459\"></p>\n<p>The image is above.</p>",
    "post_number": 5,
    "post_type": 1,
    "updated_at": "2022-09-02T05:37:25.680Z",
    "reply_count": 0,
    "reply_to_post_number": null,
    "quote_count": 0,
    "incoming_link_count": 0,
    "reads": 0,
    "readers_count": 0,
    "score": 0,
    "yours": true,
    "topic_id": 7,
    "topic_slug": "welcome-to-discourse",
    "display_username": null,
    "primary_group_name": null,
    "flair_name": null,
    "flair_url": null,
    "flair_bg_color": null,
    "flair_color": null,
    "version": 1,
    "can_edit": true,
    "can_delete": true,
    "can_recover": false,
    "can_wiki": true,
    "user_title": null,
    "bookmarked": false,
    "raw": "My image is inserted next:\n\n![ferris|690x459](upload://5YA5Y9vjz0iQmn2DErtUBrHCKng.png)\n\n\nThe image is above.",
    "actions_summary": [
      {
        "id": 3,
        "can_act": true
      },
      {
        "id": 4,
        "can_act": true
      },
      {
        "id": 8,
        "can_act": true
      },
      {
        "id": 7,
        "can_act": true
      }
    ],
    "moderator": false,
    "admin": true,
    "staff": true,
    "user_id": 1,
    "draft_sequence": 12,
    "hidden": false,
    "trust_level": 1,
    "deleted_at": null,
    "user_deleted": false,
    "edit_reason": null,
    "can_view_edit_history": true,
    "wiki": false,
    "reviewable_id": null,
    "reviewable_score_count": 0,
    "reviewable_score_pending_count": 0
  },
  "success": true
}
5

Tracked this issue down further to this function: discourse/post.rb at main · discourse/discourse · GitHub

Modified my local function:

  def cook(raw, opts = {})
    Rails.logger.info("Cooking post with raw: #{raw}")
    # For some posts, for example those imported via RSS, we support raw HTML. In that
    # case we can skip the rendering pipeline.
    return raw if cook_method == Post.cook_methods[:raw_html]

    options = opts.dup
    options[:cook_method] = cook_method

    post_user = self.user
    options[:user_id] = post_user.id if post_user
    options[:omit_nofollow] = true if omit_nofollow?

    if self.with_secure_media?
      each_upload_url do |url|
        uri = URI.parse(url)
        if FileHelper.is_supported_media?(File.basename(uri.path))
          raw = raw.sub(
            url, Rails.application.routes.url_for(
              controller: "uploads", action: "show_secure", path: uri.path[1..-1], host: Discourse.current_hostname
            )
          )
        end
      end
    end

    cooked = post_analyzer.cook(raw, options)

    Rails.logger.info("Cooked into: #{cooked}")

    new_cooked = Plugin::Filter.apply(:after_post_cook, self, cooked)

    if post_type == Post.types[:regular]
      if new_cooked != cooked && new_cooked.blank?
        Rails.logger.debug("Plugin is blanking out post: #{self.url}\nraw: #{raw}")
      elsif new_cooked.blank?
        Rails.logger.debug("Blank post detected post: #{self.url}\nraw: #{raw}")
      end
    end

    Rails.logger.info("New cooked into: #{new_cooked}")

    new_cooked
  end

Output:

Completed 200 OK in 335ms (Views: 0.4ms | ActiveRecord: 0.0ms | Allocations: 78316)
done
done
Cooking post with raw: ![ferris|690x459](upload://5YA5Y9vjz0iQmn2DErtUBrHCKng.png)
Started POST "/~cs6991/forum/presence/update" for 127.0.0.1 at 2022-09-02 05:55:33 +0000
Processing by PresenceController#update as */*
  Parameters: {"client_id"=>"16308337827949548cb8b156301a493b", "leave_channels"=>["/discourse-presence/reply/7"]}
Completed 200 OK in 19ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 6182)
done
Cooked into: <p><img src="" alt="ferris" data-base62-sha1="5YA5Y9vjz0iQmn2DErtUBrHCKng" width="690" height="459"></p>
New cooked into: <p><img src="" alt="ferris" data-base62-sha1="5YA5Y9vjz0iQmn2DErtUBrHCKng" width="690" height="459"></p>
done
6

Managed to track post uploads down to this exact line:

in particular,

  // relative urls
  if (/^\/[\w\.\-]+/i.test(href)) {
    return href;
  }

breaks because since my forum URL is being served out of an apache user-webdir (which I don’t have control over), it starts with a ~, which breaks that regex.
I’ve confirmed that modifying the character class to include ~ (as [\w\.\-~]) fixes the post uploads, but painfully avatar uploads are still broken!

1 Like
7

…aaand the other broken regex:

same ordeal – needs a ~ in the character class
which fixes my avatars

1 Like
8

If Core is not so keen on a PR, you could probably permanently resolve that with a custom plugin for your specific site.

1 Like