I have a feature request. I have a hardware appliance that runs in-line with my webserver. When activated it intercepts HTTP and HTTPS requests. The problem being it does not support HTTP/2 or WebSockets. When it’s activated certain actions like post edits fail to complete. It’s only activated during DDoS Attacks using HTTP/HTTPS floods (it won’t activate in the case of a syn flood that’s handled differently) however legacy support for HTTP/1.1 without WebSockets would be a nice enhancement useful for websites that are under attack. Thoughts?
Wait, are you sure Discourse even uses WebSockets?
It does use Message Bus, which is based on HTTP long polling, and HTTP2 – but long polling is just a special way of using HTTP, and HTTP2 isn’t mandatory (you can still use HTTP, it’s just less efficient)
To be fair I have the websocketed template enabled, I suppose I could disable it but not sure if that’d cause a negative effect or not. The site gets attacked by trolls every so often and I have to enable Voxality’s hardware appliance for mitigating HTTP(s) floods, it’s janky but it works. If I remove the web socket template will post edits work without HTTP/2? I’m not concerned about speed, only uptime. Before I make changes I’d like Discourse staff to confirm for me.
## this is the all-in-one, standalone Discourse Docker container template ## ## After making changes to this file, you MUST rebuild ## /var/discourse/launcher rebuild app ## ## BE *VERY* CAREFUL WHEN EDITING! ## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT! ## visit http://www.yamllint.com/ to validate this file as needed templates: - "templates/postgres.template.yml" - "templates/redis.template.yml" - "templates/web.template.yml" - "templates/web.ratelimited.template.yml" - "templates/web.socketed.template.yml" #- "templates/cloudflare.template.yml" #- "templates/web.ssl.template.yml" #- "templates/web.letsencrypt.ssl.template.yml" ## which TCP/IP ports should this container expose? ## If you want Discourse to share a port with another webserver like Apache or nginx, ## see https://meta.discourse.org/t/17247 for details #expose: #- "80:80" # http #- "443:443" # https params: db_default_text_search_config: "pg_catalog.english" ## Set db_shared_buffers to a max of 25% of the total memory. ## will be set automatically by bootstrap based on detected RAM, or you can override #db_shared_buffers: "256MB" ## can improve sorting performance, but adds memory usage per-connection #db_work_mem: "40MB" ## Which Git revision should this container use? (default: tests-passed) #version: tests-passed env: LANG: en_US.UTF-8 # DISCOURSE_DEFAULT_LOCALE: en # Setup Proxy Server #http_proxy: http://127.0.0.1:8118 #https_proxy: https://127.0.0.1:8118 ## How many concurrent web requests are supported? Depends on memory and CPU cores. ## will be set automatically by bootstrap based on detected CPUs, or you can override #UNICORN_WORKERS: 3 ## TODO: The domain name this Discourse instance will respond to DISCOURSE_HOSTNAME: 'sinfulforums.net' ## Uncomment if you want the container to be started with the same ## hostname (-h option) as specified above (default "$hostname-$config") #DOCKER_USE_HOSTNAME: true ## TODO: List of comma delimited emails that will be made admin and developer ## on initial signup example 'email@example.com,firstname.lastname@example.org' DISCOURSE_DEVELOPER_EMAILS: 'email@example.com' ## TODO: The SMTP mail server used to validate new accounts and send notifications DISCOURSE_SMTP_ADDRESS: smtp.sendgrid.net # required DISCOURSE_SMTP_PORT: 587 # (optional, default 587) DISCOURSE_SMTP_USER_NAME: apikey # required DISCOURSE_SMTP_PASSWORD: [REDACTED] # required, WARNING the char '#' in pw can cause problems! #DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true) ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate #LETSENCRYPT_ACCOUNT_EMAIL: firstname.lastname@example.org ## The CDN address for this Discourse instance (configured to pull) ## see https://meta.discourse.org/t/14857 for details #DISCOURSE_CDN_URL: //discourse-cdn.example.com ## The Docker container is stateless; all data is stored in /shared volumes: - volume: host: /var/discourse/shared/standalone guest: /shared - volume: host: /var/discourse/shared/standalone/log/var-log guest: /var/log ## Plugins go here ## see https://meta.discourse.org/t/19157 for details hooks: after_code: - exec: cd: $home/plugins cmd: - git clone https://github.com/discourse/docker_manager.git - git clone https://github.com/featheredtoast/discourse-plugin-discord-auth.git - git clone https://github.com/discourse/discourse-spoiler-alert.git - git clone https://github.com/discourse/discourse-canned-replies.git - git clone https://github.com/discourse/discourse-tooltips.git - git clone https://github.com/discourse/discourse-signatures.git ## Any custom commands to run after building run: - exec: echo "Beginning of custom commands" ## If you want to set the 'From' email address for your first registration, uncomment and change: ## After getting the first signup email, re-comment the line. It only needs to run once. #- exec: rails r "SiteSetting.email@example.com'" - exec: echo "End of custom commands"
web.socketed.template.yml has nothing to do with WebSockets. That template causes Nginx to listen on a Unix socket in the shared volume (i.e. a special file that can be used by another web server to proxy requests to) instead of a port.
Nice catch, so does Discourse use websockets at all then? If so can they be disabled? When I enable the hardware appliance post edits fail for some reason. I’d like to be able to keep it active 24/7 but only if no features are broken as collateral damage. For now it’s only enabled during attacks. It’s good to know HTTP/2 is optional.
I’m pretty sure it doesn’t.
What happens when you try to edit a post while proxied trough your appliance? What does you browser’s console in the developer tools say? Do any network requests fail?
One moment I’m enabling the hardware appliance. I will respond with more details in about 5 minutes after traffic is intercepted and filtered.
I get the following error when I try to edit a post.
It looks like any action which uses a ‘PUT’ request gets an empty response, in that case the best solution would be to add a workaround to use POST instead of PUT requests, this would involve updates to Discourse’s API. I think that would resolve the issue. I’ve contact Frantech and asked are they blocking ‘PUT’ requests and seeing if Voxality could allow ‘PUT’ requests.
Okay, so the response to the update request is just blank. That’s not good, and I haven’t seen this kind of issue before.
This means that your appliance is either changing the request in a way which causes Discourse to error out and send an empty reply, or it is mangling the response. I’d guess it’s the latter (and specific to
PUT requests), but either way, it’s most likely an issue with your appliance
To debug this further, assuming you don’t find anything informative in Discourse’s logs, you’d most likely have to capture the requests between your appliance and Discourse as well as between the browser and your appliance to see what your appliance changes.
(I’m pretty sure that your appliance does meddle with the content – the first time I loaded https://sinfulforums.net/, the page refreshed before Discourse loaded up, which isn’t normal behavior.)
I’ve contact Frantech and asked are they blocking ‘PUT’ requests and seeing if Voxality could allow ‘PUT’ requests.
No, it does not. See WebSockets, caution required!
Just out of pure curiosity… What did your site hold or what did you do to attract such levels of ddos attacks?
I’d suppose most people would not bother to attack forums…
Because a lot of my members are in the “twitter hacker” community and send DDoS attacks for the lulz, I’m used to mitigating them but it takes time and no system is perfect (rip my 7pm bedtime ).
Alright, do you think you guys could add a workaround as a setting which allows the client to only send POST and GET requests and to avoid things like PUT requests (legacy ddos filter no likey ).
We are generally a forward thinking project, building for the next
10 6 years of the Internet. So it’s unlikely we’ll bend over backwards to support obsolete stuff.
Alright, the main effected thing during attacks is post edits and using the advanced filter is a last resort. I’m looking into whether Voxality could allow PUT requests or not. I might reconsider Cloudflare and double up the filters or something. Setting the real IP from Cloudflare, then the reverse proxy might be another challenge though.
An update everyone: PUT requests cannot send the cookie that the HTTP(s) filter needs. I’m wondering if a plugin could rewrite all PUT requests to use POST instead however that’d be quite the update.
This makes no sense, PUT, GET, HEAD, POST all can set the
Set-Cookie header and responses can contain the
PUT in general is incredibly similar to POST.
Does it send all cookies by default or is there something added I have to do? Because if Voxality doesn’t get the cookie, the filter blocks the request. For now I have nginx rate limiting plus banning with ipsets and iptables. It’s very fast and resource efficient.
“it” is the browser, the browser always sends cookies that is the HTTP protocol.