CSRF login error after upgrade to 2.5.0.beta4

After upgrade to 2.5.0.beta4 I see CSRF errors in the production log:

Processing by SessionController#csrf as JSON
Completed 200 OK in 1ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 351)
Started POST "/session" for 127.0.0.1 at 2020-05-05 09:25:17 +0000
Processing by SessionController#create as */*
  Parameters: {"login"=>"admin", "password"=>"[FILTERED]", "second_factor_method"=>"1", "timezone"=>"Europe/Berlin"}
Can't verify CSRF token authenticity.
  Rendering text template
  Rendered text template (Duration: 0.0ms | Allocations: 1)
Filter chain halted as :verify_authenticity_token rendered or redirected
Completed 403 Forbidden in 2ms (Views: 0.7ms | Allocations: 1100)

And discourse doctor shows

========================================
Discourse 2.5.0.beta4
Discourse version at forum.netzwissen.de: Discourse 2.5.0.beta4
Discourse version at localhost: NOT FOUND
==================== DNS PROBLEM ====================
This server reports NOT FOUND, but forum.netzwissen.de reports Discourse 2.5.0.beta4 .
This suggests that you have a DNS problem or that an intermediate proxy is to blame.
If you are using Cloudflare, or a CDN, it may be improperly configured.

Question: the server itself hosts multiple services with different DNS names. In front of discourse we have a haproxy server to handle ssl termination. I don’t understand the error message

“Discourse version at localhost: NOT FOUND”

Is it possible that the csrf error has to do with this error message?

Discourse-doctor doesn’t pretend to be able to diagnose a complex setup like yours. It’s comparing only whether local host and the dns return the same value. For your setup it’s expected that they be different.

I don’t have any hints on your actual problem, though. Sorry.

1 Like

Hi,
ok, I tested with another account and get the same eror message, seems like logins are blocked totally now and the CSRF error may be the root cause …

Any ideas for futher debugging? My app.yml ist pretty standard except that

expose:
  - "127.0.0.1:884:80"   # http

Incoming requests are forwarded from a haproxy server to the discourse conttainer at 884. ssl/https is done by haproxy.

When registering a new user through oauth2 (Google), I also get a csrf error:

 Rendered common/_discourse_stylesheet.html.erb (Duration: 0.4ms | Allocations: 206)
  Rendered application/_header.html.erb (Duration: 0.3ms | Allocations: 142)
Completed 200 OK in 23ms (Views: 20.4ms | ActiveRecord: 0.0ms | Allocations: 4636)
Started GET "/latest.json?order=default" for 127.0.0.1 at 2020-05-05 11:43:08 +0000
Processing by ListController#latest as JSON
  Parameters: {"order"=>"default"}
Completed 200 OK in 30ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 10224)
Started GET "/u/hp.json" for 127.0.0.1 at 2020-05-05 11:43:08 +0000
Processing by UsersController#get_honeypot_value as JSON
Completed 200 OK in 3ms (Views: 0.1ms | ActiveRecord: 0.0ms | Allocations: 1049)
Started GET "/session/csrf" for 127.0.0.1 at 2020-05-05 11:43:38 +0000
Processing by SessionController#csrf as JSON
Completed 200 OK in 1ms (Views: 0.2ms | ActiveRecord: 0.0ms | Allocations: 355)
Started POST "/auth/google_oauth2" for 127.0.0.1 at 2020-05-05 11:43:38 +0000
(google_oauth2) Setup endpoint detected, running now.
(google_oauth2) Request phase initiated.
Started GET "/auth/failure?message=csrf_detected" for 127.0.0.1 at 2020-05-05 11:43:38 +0000
Processing by Users::OmniauthCallbacksController#failure as HTML
  Parameters: {"message"=>"csrf_detected"}
  Rendering users/omniauth_callbacks/failure.html.erb within layouts/no_ember
  Rendered users/omniauth_callbacks/failure.html.erb within layouts/no_ember (Duration: 0.1ms | Allocations: 20)
  Rendered layouts/_head.html.erb (Duration: 11.7ms | Allocations: 3551)
  Rendered common/_discourse_stylesheet.html.erb (Duration: 0.5ms | Allocations: 213)
  Rendered application/_header.html.erb (Duration: 0.9ms | Allocations: 555)
Completed 200 OK in 19ms (Views: 16.4ms | ActiveRecord: 0.0ms | Allocations: 7652)

I’ve had exactly the same issue after the upgrade to 2.5.0.beta4 (Moved site behind proxy, favicon and header not using https anymore - #7 by rossierd).

Were you able to fix the issue? I can imagine that the upgrade came with a new version of nginx (or its config) which leads to this issue (but pure hypothetical ;-))
I tried to find a way to disable CSRF in nginx (https://github.com/gartnera/nginx_csrf_prevent) but I think nginx must be recompiled, and I don’t know if we need the complete development environment of Discourse to do that.

Unfortunately the problem is still not solved here. Login fails with “unknown error” and upon each try I see this in the log:

root@develd:/var/discourse# tail -f /var/log/discourse-rails/production.log
Processing by SessionController#csrf as JSON
Completed 200 OK in 1ms (Views: 0.1ms | Allocations: 351)
Started POST "/session" for 127.0.0.1 at 2020-06-07 06:58:19 +0000
Processing by SessionController#create as */*
  Parameters: {"login"=>"admin", "password"=>"[FILTERED]", "second_factor_method"=>"1", "timezone"=>"Europe/Berlin"}
Can't verify CSRF token authenticity.
  Rendering text template
  Rendered text template (Duration: 0.0ms | Allocations: 1)
Filter chain halted as :verify_authenticity_token rendered or redirected
Completed 403 Forbidden in 2ms (Views: 0.8ms | ActiveRecord: 0.0ms | Allocations: 1100)
Started GET "/session/csrf" for 127.0.0.1 at 2020-06-07 07:00:45 +0000
Processing by SessionController#csrf as JSON
Completed 200 OK in 1ms (Views: 0.2ms | Allocations: 351)
Started POST "/session" for 127.0.0.1 at 2020-06-07 07:00:45 +0000
Processing by SessionController#create as */*
  Parameters: {"login"=>"admin", "password"=>"[FILTERED]", "second_factor_method"=>"1", "timezone"=>"Europe/Berlin"}
Can't verify CSRF token authenticity.
  Rendering text template
  Rendered text template (Duration: 0.0ms | Allocations: 1)
Filter chain halted as :verify_authenticity_token rendered or redirected
Completed 403 Forbidden in 2ms (Views: 0.9ms | Allocations: 1100)

The app.yml has

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  ## - exec: rails r "SiteSetting.notification_email='noreply-discourse@netzwissen.de'"
  - replace:
      filename: /etc/nginx/conf.d/discourse.conf
      from: "types {"
      to: |
        set_real_ip_from 127.0.0.0/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;
        types {
  - exec: echo "End of custom commands"

as it was recommended in Haproxy and Discourse IP issue

Try this : Moved site behind proxy, favicon and header not using https anymore

I got successful login with that, but take care where you patch the proxy_pass command. It only works in location @discourse { … }

1 Like

ok, just for understanding: we are talking about the nginx inside the “container zoo”, correct? Because, before the update to 2.5.0beta4 there was no requirement for patching this, it just worked smootly.

Yes, if by zoo you mean the container which is running Discourse. You can enter the container with “rails enter app”

We did some further debugging here: the problem starts at the nginx server inside the container. It does not understand the proxy_pass directive and therefore seems to crash, but why:

root@develd:/var/discourse# docker ps -a
CONTAINER ID        IMAGE                              COMMAND                  CREATED             STATUS                     PORTS                   NAMES
f8f6103a036d        local_discourse/app                "/sbin/boot"             35 seconds ago      Up 32 seconds              127.0.0.1:884->80/tcp   app
43406c37f403        discourse/base:2.0.20200512-1735   "ruby -e 'require 'y…"   2 hours ago         Created


docker exec -it f8f6103a036d /bin/bash

root@forum:/# tail -f /var/log/nginx/error.log
2020/06/08 19:05:03 [emerg] 288#288: "proxy_pass" directive is not allowed here in /etc/nginx/conf.d/discourse.conf:10

I am using this nginx config in app.yml:

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  ## - exec: rails r "SiteSetting.notification_email='noreply-discourse@netzwissen.de'"
  - replace:
      filename: /etc/nginx/conf.d/discourse.conf
      from: "types {"
      to: |
        set_real_ip_from 127.0.0.0/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;
        proxy_set_header Host $http_host;
        proxy_set_header X-Request-Start “t=${msec}”;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https; # $thescheme; <-- What I modified
        proxy_pass http://discourse;
        types {
  - exec: echo "End of custom commands"

I think that you don’t need that proxy_pass right there in your app.yml. My stanza like that looks like this:

  after_bundle_exec:
    # This is the magic to get IP numbers transmitted to Discourse
    # See https://meta.discourse.org/t/last-ip-address-and-action-dispatch-trusted-proxies/50098/3?u=pfaffman
    - replace:
        filename: /etc/nginx/conf.d/discourse.conf
        from: "types {"
        to: |
          set_real_ip_from 192.168.1.0/24;
          set_real_ip_from 172.18.0.0/24;
          set_real_ip_from 172.17.0.0/24;
          real_ip_recursive on;
          real_ip_header X-Forwarded-For;
          types {

But my code may be from before the switch to Debian in the containers.

You might try just editing that file inside the container and restarting nginx.

2 Likes

Just to make sure, in the container itself, edit /etc/nginx/conf.d/discourse.conf and patch the file as above.
Then restart nginx as follows:

$ service nginx stop
$ service nginx start

and look what happens…

Jays hint was correct: I just removed proxy_pass and then it worked, final config in app.yml is:

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  ## - exec: rails r "SiteSetting.notification_email='noreply-discourse@netzwissen.de'"
  - replace:
      filename: /etc/nginx/conf.d/discourse.conf
      from: "types {"
      to: |
        set_real_ip_from 127.0.0.1/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;
        proxy_set_header Host $http_host;
        proxy_set_header X-Request-Start “t=${msec}”;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https; # $thescheme; <-- What I modified
        types {
  - exec: echo "End of custom commands"
3 Likes