CDN 503 on user avatars and CSS files


(Christian Johnsen) #1

Hey guys

My forum (https://forum.billysbilling.dk) just went down. As far as I can see it’s due to the CDN. I hope someone can help me - maybe @sam or @codinghorror?


Template update not loading and failing resources
Broken Image Links
(Régis Hanol) #2

It’s up for me. Is it still down for you?


(Christian Johnsen) #3

Yes, unfortunately it is.


(Régis Hanol) #4

Can you try clearing your browser’s cache and refreshing?


(Christian Johnsen) #5

That does not help.

I still get this site after several minutes of loading time:


(Nicklas Møller) #6

@zogstrip it isn’t a caching issue. We’re located in Denmark, and right now I’m getting a 504 Gateway Time-out across different devices on different networks (while trying to entering the page - not on loading resources).

Sometimes the page will load, but at most times with 404 and 50* errors on resource loading.

Our customer-base (entering this forum) is based in Denmark also, but I don’t see it only as a CDN issue anymore.


(Sam Saffron) #7

Confirmed … investigating … will have it sorted shortly


(Régis Hanol) #8

@christianjohnsen / @nicklasmoeller is it still down now?


(Nicklas Møller) #9

Seems to work now! Thanks.

Will get back to you, if the problem occurs again :smile:


(Jeff Atwood) #10

Looks OK, apologies for the difficulty – we are having some issues with our glusterfs network file system:

We’ve moved more essential resources (stylesheet, etc) to an internal site path so they won’t be stored in the network file system and vulnerable to issues like this. I believe @sam also moved the default letter avatars to an internal site path.

(It’s OK for a picture of a lolcat in a topic, or even some avatars, to be temporarily missing, but it is much more traumatic if the CSS file for the site is missing!)


#11

Ahh okay. I was in the process of making a topic about this. It’s happening here on Meta and the Imgur community too. Images appear broken from Discourse’s fastly CDN.


(Stephanie) #12

OK, so that’s not just me. Sometimes when my phone is running out of space some images won’t load, and since I was getting the low space warnings I initially chalked it up to that.


(Sam Saffron) #13

Yeah we are still working on fixing the missing images but stylesheets should be very solid now, no longer served from a shared fs.


(Sam Saffron) #14

Note this was sitting in my bookmarks list, this particular failure mode is no longer likely at all:

(this is already deployed to all our customers)

Our filesystem migration is over, if any of our customers reading this see anything amiss PM me.


(Sam Saffron) #15