Delay after editing a post due to Cloudflare

About 24-48 hours ago a strange issue has appeared on our site Tappara.co, without any apparent reason – we have not touched anything, nor there is high traffic/server load currently.

After a post has been edited, there is 10~30 second delay before the post is updated with the changes. This is seen at least of iOS Safari and desktop Chrome. If you press the pencil icon (post has been edited before, thus available), the modification of the post is immediately shown.

What I know so far:

  • I updated to 1.9.4 and rebooted - issue is still there
  • It’s not server load related
  • It is not a browser specific issue, as seen on Chrome and Safari
  • I am behind CloudFlare, but can’t see the same on my other (much smaller) community that is also behind CloudFlare.
  • Pings are good, site runs fine

Any ideas?

2 Likes

How many enqueued jobs are in your Sidekiq? It looks about like this normally:

The URL to open it is something like https://forum.example.com/sidekiq/

1 Like

Zero enqueued. Everything looks normal.

1 Like

Weird thing is that this really seems to be limited to message edits. New post appear instantly on the browser they have been written with and also to other devices/sessions.

This legitimately looks like a bug, presumably related to Ember state management. Discourse, in many cases, will try to do stuff within the GUI “instantly” while performing the actual work in the background, but apparently it’s not actually performing the instant-update here, instead only refreshing the view when the push notification that gets triggered by the edit comes down.

Did some more rigorous testing:

  • It seems I can repro this with my other CloudFlare enabled site
  • The delay seems to be matching site polling interval - something has changed in CloudFlare and long polling is broken? (guessing here)
5 Likes

Maybe related to this?

I did my best to submit a ticket to CloudFlare. They are using Discourse themselves, BTW - are they friendly with our staff here?

@Mittineague No, we like to roll on the stable branch.

1 Like

Do you have the message bus running in a different domain, that doesn’t proxy through Cloudflare?

2 Likes

No I have not, since it has not been necessary during the last 2-3 years I have been using Discourse and CloudFlare - my sites have worke fine, until now.

This is something I would like to investigate now and I looked into those topics last night. It would help narrowing down the issue/root cause. I re-read those topics, but could now quite figure it out how to set it up correctly. If it setup a sub-domain xxx.tappara.co that bypasses CloudFlare’s cache, it produces a SSL/HTTPS certificate error and redirects to my root domain tappara.co.

I would be very interested in the experiences of other CloudFlare users!

  • Can you reproduce the issue?
  • How to successfully configure the domain for long polling?

Edit: Also, testing with two browsers, the new posts are shown immediately across browsers. The issue only impacts editing. (older discussion)

2 Likes

It seems that the same delay is impacting resumed browser sessions - it takes a while before the view is updated with new posts.

This might add to the evidence that CloudFlare has broken the long polling.

2 Likes

@falco @sam

I have tried to work this issue around by configuring the long polling base URL in the site settings and app.yml.

This however leads to HTTPS certificate errors (the site cert is not wildcard *.site.com?).

I tried to read the related posts, so am I missing something here?

My recommendation is to stop passing all traffic via Cloudflare and instead just use it as an origin pull CDN

5 Likes

Okay, worth a shot. Is this configuration documented somewhere – I searched this site and others with no match?

Interesting detail is that this is not repro on CloudFlare’s own Discourse instance. Either they are not eating their own dog food, or their configuration is different in some way.

2 Likes

They are not eating own dog food but we are discussing changes that

5 Likes

Aha, I knew it! They use a “classic” CDN setup, instead of their own. That’s sleazy.

So, they are a customer? That’s fantastic, for you and for the development. Could you pass on the good word that they broke something around last wednesday-thursday or so?

1 Like

They are a hosted customer and we do not support fancy setups like CloudFlare acceleration on everything on our standard and business plans.

We have an open discussion about moving CloudFlare to a dedicated setup where they can enable Argo on everything, but this is going to take a while to happen. I completely agree that it would look way better if CloudFlare forums were behind CloudFlare and @RyanK does want to make this happen. This will nip issues like this in the bud cause they will have significantly higher internal visibility which is good for Discourse, good for CloudFlare and good for the general public.

That said, it is going to take the time it takes.

In the meantime, have you considered opening a ticket with CloudFlare?

6 Likes

Ticket is the first thing I did.

2 Likes

Wheels are in motion. Can’t happen soon enough for me, but need to work through some setup steps.

What is your ticket number?

4 Likes

Hello,

CF support replied today with the ticket: 1490836

I was forced to make an emergency evacuation to KeyCDN, as real-timelyness is mission critical for us and hockey playoffs started today. So my site is not cached by CF anymore to work around the issue.

Here are a lot of topics regarding CloudFlare - I hope fellow web admins could step up to support the investigation. I may be able to arrange a test environment in the near future too.

3 Likes