Upload attachment fails

(Crimson Dragon) #1

Hi all,

I’m running discourse 2.1.0.beta2 on CentOS7: https://forum.crimsondragon.de/.
I regularly upload big zip files (about 340 MB) to distribute windows alpha builds of a game we’re working on to our alpha testers (private forum category). But recently suddenly the uploading of attachments fails. First the uploads hangs at 0% and after about 5-10 minutes the uploads aborts without any error message. I already checked the logs and the disk space… everything seems to be ok. Smaller uploads (i.e. 7 byte ZIPs) upload successfully. How can I debug the problem? Does anyone know a solution for this problem?

Best regards,
Thomas from Crimson Dragon

(Sam Saffron) #2

Enormous uploads and downloads is not really ideal on Discourse cause neither the upload nor the download is easily resumable.

(Our backup upload feature is resumable but that is an edge case)

Have you considered using a different service for this and just, say oneboxing a dropbox public share or something?

You can technically enable gigantic uploads by amending NGINX to allow per: Changing Max Attachment Size , but I do not think you will have fun with 300MB uploads.

(Crimson Dragon) #3

Hi Sam,

thanks for your fast response. Resuming downloads/uploads isn’t really important for us, because we had no problems the last 18 months we were uploading about 300 MB every 6-8 weeks.
And yes we have considered using a workaround with dropbox, but this link would be public and we have a closed alpha at the moment. We don’t really want to distribute our alpha version publicly. Using the already existing Discourse user permission was ideally for us. So using dropbox would be our fallback workaround.

We already changed the max attachment size to 500 MB. This was ok for the last 18 months, but now suddenly the upload aborts every time at 0%.

Is there some kind of debugging tool or a verbose mode to help to analyse the problem why the upload aborts? Or how to add showing a error message about the root cause why the upload aborts?

(Sam Saffron) #4

I would recommend first finding where at what size this explodes.

Try 1-10-25-50-100-150-200. Then we can work from there. Also is this a traditional install or do you have an extra proxy in front? Are you using cloudflare?

(Crimson Dragon) #5

I even tried uploading a file I already uploaded successfully on March 29th with 312.4 MB. But today even this re-upload hangs at 0%. We’re using the traditional docker install with HAProxy in front and SSL (Let’s encrypt). No cloudflare.

(Sam Saffron) #6

You are going to have to rule out HAProxy blocking this.

But yeah I would start by finding the breaking threshold.

(Crimson Dragon) #7

regarding the threshold, it’s very very low (we are talking about KB not MB!):
70 KB: upload successful
100 KB: hangs at 100%, aborts and upload is not successful
bigger files hang and abort

What do you mean exactly by you “have to rule out HAProxy blocking this”? I didn’t change the HAProxy config since the last 18 months regarding the Discourse… and our 300 MB uploads where always successful in the past.

(Sam Saffron) #8

That low threshold is a strong indicator that whatever is going on is outside of the discourse base image, be it your networking or HAProxy.

For context, we run the exact same docker image here on meta and it works fine with multi megabyte files.

I would look at OS level changes or HAProxy changed defaults.

(Crimson Dragon) #9

Thanks for the networking hint, Sam. My internet provider’s connection is broken, I have several other issues with lost connections while downloading/uploading on different PCs.