Discourse login "unknown error" with socketed operation

I have a problem with “unknown error” error message during login. From the logfiles, it looks like some kind of CSRF issue when opening the session (POST operation causes 403 response), but I dont understand the real root cause.

The access error inside the web_only container (/var/log/nginx/access.log):

[29/Mar/2022:18:08:30 +0000] "forum.netzwissen.de" 87.154.170.198 "POST /message-bus/e65d4728665448f4a47a8fc74a0a6478/poll HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0" "-" 200 538 "https://forum.netzwissen.de/" 25.002 25.002 "-" "-" "-" "-" "-" "-" "-"
[29/Mar/2022:18:08:41 +0000] "forum.netzwissen.de" 87.154.170.198 "GET /session/csrf HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0" "session/csrf" 200 716 "https://forum.netzwissen.de/" 0.029 0.030 "-" "-" "-" "-" "-" "-" "-"
[29/Mar/2022:18:08:41 +0000] "forum.netzwissen.de" 87.154.170.198 "POST /session HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0" "-" 403 494 "https://forum.netzwissen.de/" 0.007 0.006 "-" "-" "-" "-" "-" "-" "-"
[29/Mar/2022:18:08:56 +0000] "forum.netzwissen.de" 87.154.170.198 "POST /message-bus/e65d4728665448f4a47a8fc74a0a6478/poll HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0" "-" 200 538 "https://forum.netzwissen.de/" 25.003 25.002 "-" "-" "-" "-" "-" "-" "-

"

The operational setup:

  • load balancer haproxy is used for ssl termination
  • apache is used as rev proxy on the local docker host, delivering traffic to a local websocket (web.socketed.template.yml)
  • current Discourse with separate containers for data and web_only
  • original IP adresses are delivered from haproxy (forwardfor) to apache and from there through RemoteIPHeader X-Forwarded-For to the nginx inside the web_only container

I would need some ideas how to further debug this setup. Config for the web_only container:

templates:
  - "templates/web.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/web.socketed.template.yml"

#expose:
##  - "127.0.0.1:84:80"   # http
##  - "443:443" # https

# Use 'links' key to link containers together, aka use Docker --link flag.
links:
  - link:
      name: data
      alias: data

params:
  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LANG: de_DE.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  UNICORN_WORKERS: 4
  DISCOURSE_HOSTNAME: 'forum.netzwissen.de'
  DOCKER_USE_HOSTNAME: true
  DISCOURSE_DEVELOPER_EMAILS: 'admin@netzwissen.de,support@netzwissen.de'

  DISCOURSE_SMTP_ADDRESS: mail.netzwissen.de
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: discourse@netzwissen.de
  DISCOURSE_SMTP_PASSWORD: xxxxxxxxxxxxxxx
  DISCOURSE_DB_SOCKET: ''
  DISCOURSE_DB_USERNAME: discourse
  DISCOURSE_DB_PASSWORD: xxxxxxxxxxxxxxx
  DISCOURSE_DB_HOST: data
  DISCOURSE_REDIS_HOST: data


volumes:
  - volume:
      host: /var/discourse/shared/web-only
      guest: /shared
  - volume:
      host: /var/discourse/shared/web-only/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
## in our case plugins are part of the web container
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-shared-edits.git
          - git clone https://github.com/discourse/discourse-chat-integration
          - git clone https://github.com/discourse/discourse-feature-voting
          - git clone https://github.com/discourse/wp-discourse
          - git clone https://github.com/discourse/discourse-openid-connect
          - git clone https://github.com/discourse/discourse-calendar
          - git clone https://github.com/discourse/discourse-data-explorer
          - git clone https://github.com/paviliondev/discourse-events
          - git clone https://github.com/paviliondev/discourse-locations

## Remember, this is YAML syntax - you can only have one block with a name
run:
  - exec: echo "Beginning of custom commands"
  - exec: rails r "SiteSetting.notification_email='discourse@netzwissen.de'"
  - replace:
      filename: /etc/nginx/conf.d/discourse.conf
      from: "types {"
      to: |
        set_real_ip_from 127.0.0.1/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;
        proxy_set_header Host $http_host;
        proxy_set_header X-Request-Start “t=${msec}”;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https; # replaced $thescheme;
        types {

  ## If you want to configure password login for root, uncomment and change:
  ## Use only one of the following lines:
  - exec: /usr/sbin/usermod -p 'xxxxxxxxxxxxxxxxxx' root
  
  - exec: echo "End of custom commands"