Adding the wp-id parameter to topics created through WordPress

Development

AI-generated summary

The discussion revolves around the WP Discourse plugin and its interaction with Discourse. simon proposes adding a wp-id parameter to topics created through WordPress, which would allow for efficient syncing of updated content between Discourse and WordPress.

simon explains that currently, the plugin stores the topic_id in WordPress post meta data, but this approach is not efficient, especially in multisite installations. He suggests adding a wp_id property to Discourse topics, which would simplify the process of matching updated Discourse topics to their corresponding WordPress posts.

blake questions the need for the wp-id parameter, suggesting that the plugin could use the existing topic_id metadata to achieve the same result. simon responds that using the topic_id metadata is not efficient in multisite installations and proposes alternative solutions.

The discussion continues with blake suggesting that the plugin could use the topic_id metadata and query the Discourse API to retrieve the necessary data. Falco suggests using webhooks to improve efficiency.

simon expresses concerns about using webhooks in multisite installations, citing the need to search the postmeta table of each subsite to match the updated topic. However, he later discovers that a post_event webhook can be used to sync content, and proposes returning the embed_url with the webhook to simplify the process.

Ultimately, simon resolves the issue by creating a topic_id/blog_id database table, which allows the plugin to efficiently sync data between Discourse and WordPress in both normal and multisite installations.

1

When a new topic is created through the WP Discourse plugin, the plugin is sending a wp-id (with a hyphen) parameter to Discourse. It’s been doing this since before I started working on the plugin. It doesn’t seem that it’s used anywhere in Discourse.

Having a wp_id property for topics that are created through WordPress would be useful for syncing updated content between Discourse and WordPress. The way I’m doing this at the moment is to make a request to a Discourse route that’s been added through a plugin. The request is returning the topic_id, title, and posts_count for all topics that have a topic_embed that have been updated within a given time range.

On the WordPress end, the difficulty is matching the returned data with the WordPress post. It can be done with the data that’s being returned, but its a somewhat expensive query. If a wp_id property was added to topics that were created through WordPress, the process would be more efficient.

Would it be possible for this to be added to Discourse?

2

@team any input on this?

3

Can you help me understand why you would need the wp-id if the plugin is storing the topic_id after it creates a new topic?

Are you not able to look up the topic_id in the the wordpress post_meta data?

4

I’m trying to match a list of updated Discourse topics to their corresponding WordPress posts. It’s being done at a regular interval in a cron job, so I’m trying to make it as efficient as possible. I’m especially concerned about multisite WordPress installations with an unknown number of subsites. In that case, it seems important to tie up as little time as possible in processing the cron job. On WordPress, finding a post by the value of a meta-key can be done, but it’s not efficient.

I only added the topic_id as post metadata a couple of weeks ago. For posts created before that time, it’s not available. So on the WordPress end there’s no advantage of using the topic_id metadata over using a new wp_id that’s added as a property to Discourse topics.

If you do add it, the field doesn’t need to be wp_id. It could be something like external_post_id .

5

Is WP_Query not efficient? (I’m not a WP expert, so just curious)

This post might help speed things up

I’m leaning toward not adding an external_post_id because I feel like keeping track of the topic_id which the api returns should be the responsibility of the integrator not Discourse. In order to make looking up a topic by external_post_id efficient in Discourse we would also have to index the column in the db table and create/modify an api endpoint to allow for searching by that param.

If there is no advantage to using one or the other right now, I’m going to push you do use the topic_id in the metadata. I guess I’m having a hard time seeing the downside if you are already tracking the topic_id that you couldn’t just hit the /t/{id}.json endpoint and get the title and the posts_count?

6

Sure, I can make it work without it. I’ll remove the wp-id parameter from the data the plugin sends to Discourse when a topic is published through WordPress.

My use case wasn’t wanting to query Discourse for topics by the external_topic_id. I’m querying a Discourse route that I’ve added through a plugin for topics that have been updated over a given period of time. That data is being used to update the comment counts for all the posts on a site, and to indicate which posts need to sync their comment content with Discourse.

7

Even better with the real topic_id you can listen to webhooks, and be way more efficient than regular API calls to data that didn’t change.

8

I like the idea of using a webhook. It was my initial approach to doing this. The biggest problem with using a webhook with a WordPress multisite installation is that it doesn’t return any data that will allow the updated Discourse topic to be matched with the correct WordPress subsite. For each webhook request, the postmeta table of each WordPress subsite will have to be searched to try to match the topic with a post. For a network with 100 subsites, this could become quite inefficient.

I’m going to give this some more thought and see if there’s a way to make it work.

9

This is basically how the plugin is functioning now. What I’m trying to do is keep the comment numbers and content up to date while making as few API calls to Discourse as possible.

10

A post_event webhook will work for syncing content if the post’s topic.topic_embed.embed_url is returned with the webhook. On a WordPress multisite setup, the webhook only needs to be sent to the main site on the network. If the topic was published from a subsite, the subsite can be found from the embed_url.

Does using a webhook for this seem safe?

Would it be possible to return the embed_url with a post_event webhook? For testing, I just added this to the post_serializer.

  def embed_url
    if object.topic && object.topic.topic_embed
      object.topic.topic_embed.embed_url
    end
  end

If this can’t be done, querying a Discourse route that’s added through a plugin to get a list of updated topics also works well. On the WordPress end, the two ways of doing it use mostly the same code.

11

I should probably change the title of this topic :slight_smile: What I’m looking for is the best way to improve the plugin’s performance that will work in both a normal WordPress installation and a multisite installation.

If using a webhook for keeping data synced between Discourse and WordPress is a good approach, the reason for wanting to have the embed_url returned with the post_event webhook, instead of storing that data on the WordPress end, is that the only way I can see of storing the data on WordPress is to create a new database table. That would be quite simple, but I’m trying to conform to the WordPress VIP coding standards. The hope is that the plugin can eventually be used on sites hosted by wordpress.com. The WordPress VIP coding standards do not allow creating or altering the WordPress database tables: https://vip.wordpress.com/documentation/vip/code-review-what-we-look-for/#database-alteration

12

I ended up creating a topic_id/blog_id database table. It’s only added when a single Discourse forum is associated with multiple WordPress sites, so it should be ok.