Recurring 403 Forbidden error on posts with multiple images


(Edward Iskra) #1

We have a recurring problem on large posts throwing a 403 Forbidden on attempts to edit the post.

image

We’re currently running v2.0.0.beta3+20 in Docker.

The posts in question are large, they contain a couple of images, a little formatting, and a couple of images. The pages are generated strictly using the Discourse editor features. On testing, I find that the problem is related to the images embedded.

For example, I have a large post created by someone else where I was asked to edit text. Even adding a single space to the post and saving throws the Forbidden, even if I’m logged in as admin.

If I enter the editor, copy-paste the entire contents, abandon the edits, and then do a reply, I can paste the entire things, make my edits, and save.

So the “save,” itself, works… however, if I go in to edit that post, even adding a single space, attempting to save the edit will throw the Forbidden.

As an experiment, I just chopped up a post into pieces, and tested which pieces are editable - the pieces which weren’t editable, I chopped down smaller, until they were editable an not throwing errors.

I took a long post which had four images, and ended up with 4 smaller pieces… each of which had one image. If a post has two images, it’s uneditable.

The images were originally inserted with simple image pasting; the editor content looks like this:

![kasse|356x220](upload://fkfT6T1qlWXrolyhjDMSedmhPL6.png)

All the posts - copied or edited - render the images just fine.

How can we troubleshoot this further or address the error?


(Jay Pfaffman) #2

It could be some bug with Trust Level and a limit of how many images you can have. newuser max images defaults to 1. Admin should be immune to that limit, but that’s my guess.

Perhaps @jomaxro would want to try to replicate.


(Edward Iskra) #3

I’ll see if changing the newuser max images setting makes a difference.


(Edward Iskra) #4

No help, unless I needed to restart some service to make it take effect. (I changed the setting, logged out/in, tested to confirm the bug is still around.)


(Joshua Rosenfeld) #5

Hey @MentalNomad, is your Discourse site public? Can you share the link to an affected post? If not, can you upload the entire raw contents of the affected post to something like PasteBin and share it here (or in a PM if the post contains private info)?


(Edward Iskra) #6

The post I’m testing with is not public, but we have public posts that are affected… I’ll find one for you…

Here, this one is affected:


(Joshua Rosenfeld) #7

OK, I’m testing this on try.discourse.org. I was able to make an edit successfully during the grace edit window. Will test again after the window ends.


(Joshua Rosenfeld) #8

I was able to successfully edit the post after the grace window ended.

Given that, my first suggestion would be updating to latest. There may have been a bug fixed between beta3 and beta7 (there’s also been security fixes, so you should always ensure you stay up to date :smiley:). Beyond that, next time this occurs please check the logs (https://forum.bitcoingold.org/logs) to see if there’s anything relevant.


(Edward Iskra) #9

We’re now upgraded to Beta7 (happened about 12 hours ago.)

The error still happens when editing a post with multiple images, or when editing a post with multiple links which auto-expand a link preview with an image (Unsure if it happens with non-image links.)

There are no new entries in the error logs since 12 hours ago (looking at https://forum.bitcoingold.org/logs via GUI under an Admin login.)


(Edward Iskra) #10

Here is an example that you can verify for yourself:

A simple post with two links, made into wiki, editing fails:

Simple post in same thread, one link, made into wiki, editing works:


(Joshua Rosenfeld) #11

Just tested. I could make a single edit to the two link post, but not a second edit. Digging into your site configuration (looking at DNS), am I correct that your site is behind Cloudflare? If yes, I’d be interested in know if your issue is related to Delay after editing a post due to Cloudflare, given that I cannot reporduce it on try.discourse.org. Can you open a support ticket with Cloudflare and link to this topic (and the delayed edit topic) and see what they say?


(Jeff Atwood) #12

Yeah you really ought to disable cloudflare for the Discourse URL – we’ve seen many issues with cloudflare in the past, and it is not required, as all assets are already compressed with Discourse. The only reason to use cloudflare, is if you are sure you will be the target of a massive DDoS attack.


(Edward Iskra) #13

We have been the target of more than one massive DDoS attack, and can expect more, so it is required. Disabling CloudFlare is simply not an option.

That said, we’ve now got a little more detail - the problem seems to be that CloudFlare interprets the activity of editing one of these complex Discourse posts in a way that resembles a SQL injection or XSS attempt, and puts up a Captcha which isn’t actually getting displayed to the user.

Here’s an example of the CloudFlare logs when I was being blocked, provided by our sys admin who has been helping troubleshoot:

Rule(s) Triggered
ID    Description    Group    
981176    Inbound Anomaly Score Exceeded (Total Score: 81, SQLi=9, XSS=45): Last Matched Message: IE XSS Filters - Attack Detected.    OWASP Inbound Blocking    Filter
100013    Filter out any HTML links    Cloudflare Specials    Filter
100022    Block everything that isnt GET or POST    Cloudflare Specials    Filter
100022A    Block everything that isnt GET or POST or HEAD    Cloudflare Specials    Filter
950120    Possible Remote File Inclusion (RFI) Attack: Off-Domain Reference/Link    OWASP Generic Attacks    Filter
950901    SQL Injection Attack: SQL Tautology Detected.    OWASP SQL Injection Attacks    Filter
960024    Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters    OWASP SQL Injection Attacks    Filter
960032    Method is not allowed by policy    OWASP HTTP Policy    Filter
973300    Possible XSS Attack Detected - HTML Tag Handler    OWASP XSS Attacks    Filter
973301    XSS Attack Detected    OWASP XSS Attacks    Filter
973304    XSS Attack Detected    OWASP XSS Attacks    Filter
973306    XSS Attack Detected    OWASP XSS Attacks    Filter
973316    IE XSS Filters - Attack Detected.    OWASP XSS Attacks    Filter
973333    IE XSS Filters - Attack Detected.    OWASP XSS Attacks    Filter
973334    IE XSS Filters - Attack Detected.    OWASP XSS Attacks    Filter
973335    IE XSS Filters - Attack Detected.    OWASP XSS Attacks    Filter
973338    XSS Filter - Category 3: Javascript URI Vector    OWASP XSS Attacks    Filter
981133    Prequalify PM    OWASP Generic Attacks    Filter
981136    Check simple XSS patterns    OWASP XSS Attacks    Filter
981231    SQL Comment Sequence Detected.    OWASP SQL Injection Attacks    Filter
981243    Detects classic SQL injection probings 2/2    OWASP SQL Injection Attacks    Filter
981245    Detects basic SQL authentication bypass attempts 2/3    OWASP SQL Injection Attacks    Filter
981248    Detects chained SQL injection attempts 1/2    OWASP SQL Injection Attacks    Filter
2000001    Skip LFI Rules    OWASP Slr Et Lfi Attacks    Filter
2000003    Skip RFI Rules    OWASP Slr Et RFI Attacks    Filter
2000004    Skip SQLi Rules    OWASP Slr Et SQLi Attacks    Filter
2000006    Skip XSS Rules    OWASP Slr Et XSS Attacks    Filter

I’m told that we already have WAF tuned down to prevent false positives for usability reasons, so tweaking them further won’t be wise. @RyanK - if you have any input here, I’d appreciate it.


(Jeff Atwood) #14

If you are attacked via DDoS regularly then you have no choice, and you’ll have to follow up with Cloudflare on any issues. Very interesting situation you highlighted with the captcha inserted, there is no way this could ever work on a JavaScript app like Discourse. Realistically I think your users might just have to live with it as the price for this protection.


(ljpp) #15

There has been exactly 1 issue with CloudFlare + Discourse during the 2-3 years I have explored into Discourse. The one issue is the current ”delay in edit”, which can be worked around by disabling Brotli in CF settings.

In addition to DDoS protection, the Web Application Firewall is a useful feature of CF.


(Jeff Atwood) #16

There have been many breaks in the past, Rocket Loader for example breaks stuff regularly:

https://meta.discourse.org/search?q=cloudflare%20broken

Unless you have a demonstrated need for industrial DDoS protection, I don’t recommend it as a general rule.