Attempting to work around the 10 minute delay

Support WordPress
1

I’m planning on using the wp-discourse plugin on a gulf coast weather forecasting site that typically services ~10-20k pageviews/day, but occasionally during severe weather events will push 1.5-2MM PV/day to approx 500-700k visitors. The site + its hosting strategy have been battle-tested through several severe weather events and things work great under pressure, primarily thanks to careful design and a lot of help from Cloudflare.

The site’s users are used to a low-friction commenting experience via native Wordpress comments, so there will be some adjustment (like getting them used tgo clicking the “Continue the discussion at…” link) that the user-facing staff is ready to manage.

However, what they won’t tolerate is the variable 10-minute delay between posting comments and them being shown on the daily Wordpress post’s page. They will want new comments (up to the configured limit) immediately on the homepage upon posting, similar to how native WP comments are displayed immediately after posting.

After screwing around with the built-in options to try to make posts show up immediately without nginx’s fastcgi caching, wordpress, or browser caching interfering with new comments appearing on refresh after they’re posted, the I have added the following two mu-plugins to mitigate this and cause newly posted comments to show up on the Wordpress side on refresh:

wp-discourse-transient-killer.php
wp-discourse-cache-header-fix.php

This has solved my problem: new posts to Wordpress-created Discourse threads now appear below the wordpress posts instantly on refresh.

But I’m kind of at the edge of my competence here — what am I breaking/screwing up/undermining by doing this?

I don’t particularly care about creating additional load on my webhost by having the comment endpoint being hammered by visitors checking the daily weather forecast page (with its embedded Discourse comments) — that is a problem I can solve by throwing money at it. My primary requirement is avoiding 20,000+ users sending me emails asking me why their comments aren’t appearing instantly on the homepage when they post them.

Is this the right approach? Is what I’m doing wise? Does it create additional security or performance issues I haven’t anticipated? Basically, am I screwing things up by doing this?

Thanks :slight_smile:

2

Hmm I’m confused why this works, because

And your plugin

add_action( 'wpdc_after_webhook_post_update', function( $topic_ids ) {
    foreach ( (array)$topic_ids as $topic_id ) {
        delete_transient( 'wpdc_comment_html_' . $topic_id );
    }
}, 11 );

Aren’t you mixing up the (Wordpress) $post_ids passed to the action with the (Discourse) topic_id that serves as the transient key ?

I would expect your plugin to look like this

add_action( 'wpdc_after_webhook_post_update', function( $post_ids) {
    foreach ( (array)$post_ids as $post_id ) {
        $topic_id = get_post_meta( $post_id, 'discourse_topic_id', true );
        delete_transient( 'wpdc_comment_html_' . $topic_id );
    }
}, 11 );

OTOH you’re saying this works? :thinking:

3

Hey @Lee_Ars, can you first confirm that you have the comments webhook setup?

That works like so:

  1. There’s a new post in Discourse.
  2. A webhook payload is sent to Wordpress.
  3. WP discourse updates the comment count on the post and also sets a post custom field wpdc_sync_post_comments.
  4. If wpdc_sync_post_comments is set Discourse comments will sync when a Wordpress post is loaded, regardless of the sync period (i.e. the 10 minute delay).

Before we get into caching, I just want to be sure that is in place first. If it is, perhaps also just turn on “Verbose Webhook Logs” and verify you’re receiving webhook requests when a new post is made in Discourse.

1 Like
4

Hi Angus! That’s affirmative, the “Sync Comment Data” webhook option is enabled on the WP side, and I created the webhook on the Discourse side and pings are successful. The WP plugin’s logging shows comment.INFO: sync_comments.success messages at the right timestamps:

[2025-07-07 14:16:38] connection.INFO: check_connection_status.successful_connection  
[2025-07-07 14:16:38] connection.INFO: check_connection_status.valid_scopes  
[2025-07-07 20:11:31] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:25:03] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:32:14] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 20:44:15] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:00:39] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:01:42] comment.INFO: sync_comments.success {"post_id":786} 
[2025-07-07 21:15:40] comment.INFO: sync_comments.success {"post_id":786}

It’s just that even with those success messages, existing visitors (or at least me, multiply testing on firefox/safari/chrome on a mac, firefox/chrome/edge on a win10 PC, and safari on ios) keep getting a cached /wp-json/wp-discourse/v1/discourse-comments endpoint, with cache-control headers set to a nonzero amount. If I do a ctrl-shift-f5 on Chrome (or the equivalent in other browsers) to force it to bypass local cache on refresh, everything works great and the new posts appear.

With the mu-plugins in place, that endpoint shows cache-control set to no-store no-cache etc, and the problem behavior doesn’t manifest—simply visiting the WP post or refreshing it with the regular refresh button shows new embedded comments.

I’ve turned on verbose webhook logging and posted a test post, and things appear OK when I create a new post:

webhook_topic.INFO: update_topic_content.update_post_metadata_success {"post_ids":"786"}

Everything seems to be working, but I still don’t really understand why the original problem behavior manifested, or if it’s arisen from some overlooked issue on my end. (Very possible!)

D’oh, sorry, I’m sure you’re right about me screwing that up — yesterday was a long day, and like I said, I’m kind of at the limit of my abilities here. It was definitely doing something, though now I’m wondering if the “fixed” behavior I’m seeing is just it being broken in a new way. (I’ve updated the “transient killer” plugin to use the right argument now, thank you!)

5

Thanks, so just one more thing to clarify. Is this WP Discourse setting (in “Commenting”) on?

Screenshot 2025-07-08 at 14.38.56
Screenshot 2025-07-08 at 14.38.561460×156 12.1 KB

6

I’ve tried both with the Cache Comment HTML setting enabled or disabled and it appears to have no impact on the problem behavior. I’ve currently got it disabled enabled (sorry, I was incorrect, it’s currently enabled while I keep poking at it), but I can set it to whatever would be helpful for troubleshooting.

7

If the setting is disabled, you won’t notice the topic_id / post_id mixup, since that plugin does nothing in that case. No caching → it doesn’t matter if you delete the wrong cache.

If the setting is enabled then you should notice that the plugin doesn’t work properly.

I.e. if you want to troubleshoot, you should enable the setting.

1 Like
8

Ok, so as a first measure for your case I’d suggest:

  1. keep Cache Comment HTML disabled; and
  2. remove the https://www.bigdinosaur.org/r/wp-discourse-transient-killer.txt plugin as it’s currently not doing, anything as Richard observes.

If that does not resolve the problem then the problem is to do with another form of caching. The next questions to answer are:

  1. What caching solutions do you (and / or your hosting provider) use for Wordpress?
  2. If https://www.bigdinosaur.org/r/wp-discourse-cache-header-fix.txt fixes the issue, how does it’s specific behaviour invalidate whatever cache(s) are being applied in 1?

Looking at wp-discourse-cache-header-fix I see that one of the fixes is for load-comments.js. Do you have this setting enabled?

Screenshot 2025-07-08 at 16.56.13
Screenshot 2025-07-08 at 16.56.131604×254 30.2 KB

9

This is self-hosted WP on nginx+php-fpm 8.3 with nginx fast-cgi caching for dynamic content and Redis object caching (with the object cache drop-in active). There are no other layers (no CDN, no CF, no on-box Varnish or other local cache beyond nginx’s fast-cgi cache). Dumping the nginx fast cgi cache (aggressively, by going rm -rf /etc/nginx/cache/*) has no effect on the problem behavior — stale results are served even after nuking the cache directory and restarting both nginx and php-fpm.

I do indeed have the Ajax comment loading enabled right now, yes, but again, toggling it off (and dumping nginx’s cache plus restarting nginx and php-fpm just in case) had no effect on the problem behavior. Browsers still got stale comments.

Option toggled, transient-killer removed. No change in problem behavior.

The effect it applies appears to be supplying a no-cache cache-control header instead of one with a cache time specified. WIthout it in place, my browser seems to very much want to serve a stale cached version of the wp-json/wp-discourse/v1/discourse-comments endpoint from its disk cache; as noted, i have to shift-ctrl-f5 (or the equivalent) to force a no-cache refresh.

The problem behavior seems to be on the browser end, rather than in a persistent server cache. It’s just every browser on every OS that I have access to that’s doing it.

10

Ok, just so I’m 100% clear, when you have:

  • verified working comment webhook
  • Cache Comment HTML off
  • AJAX Loading off
  • no cdn
  • no cloudfront
  • no wordpress caching plugin
  • no relevant warnings or errors in the php logs

you’re sure it’s not working?

If it’s not working with that setup, I’m at a bit of a loss without having a closer look and would default to

  • AJAX loading on
  • Your wp-discourse-cache-header-fix.php fixes

which is what I suspect was working for you. If that route is working, then you should go for that.

11

Here’s a quick imgur gallery of screenshots of my current plugin settings, for reference.

Confirm no CDN, no Cloudfront or Cloudflare either one, no caching plugins aside from the Nginx helper (to help WP invalidate nginx fast-cgi cache as needed).

Also confirm nothing relevant in either the php-fpm or the nginx error logs.

God, dude, i wish it were. I’ve been banging my head on this for about 30 hours at this point, with some time off to sleep. I might be going a little crosseyed, heh.

12

Yeah, I feel you. Take a break for a day or so. I’ll try to recreate the issue tomorrow by copying your setup.

2 Likes
13

If I don’t flail my way to a solution, and if you’d like, I’d be happy to grant some temporary local access (to the WP blog, the discourse, and/or the underlying hosts) tomorrow or the next day for troubleshooting. Definitely not trying to get free labor or anything, though. Would be happy to pay for your services if there’s actual time to be spent here.

For reference, both the host holding Wordpress and the host running Discord are t3a.large EC2 instances running AWS’ Ubuntu 24.04 AMI; installed software is minimal (nginx + php8 + fpm + redis on the web host, and nothing beyond git + docker + discourse on the discourse host). This is my personal blog, and I’m testing comment integration here before I roll it out in production on the much larger WP site mentioned in the OP. (I also maintain that site and it has a similar tech stack, though it’s fronted by Cloudflare. I’ll cross that bridge when I come to it!)

If you need/want any more details about the underlying stack, I’m happy to provide them.

14

Ok, here’s a video of me trying (and failing) to repro your issue:

The next thing I want you to try is this filter.

15

Thanks @angus — failing repro actually makes me feel a lot better, because that means I’m doing something wrong rather than there actually being something wrong :smiley:

I will get that filter added today when I have a block of time for troubleshooting and report back :+1:

1 Like