[Announce] Search & replace / batch process Discourse posts

==> Discourse-post-process <==

I’m happy to introduce this simple but powerful Node.js script that uses the Discourse API to mass-modify Discourse posts. It essentially performs a regexp-based search & replace across all posts in a forum.

After migrating a few thousand posts from MyBB to Discourse, there was various little tweaks that needed to be made (fix BBCode → Markdown conversion, re-attach attachments, fix @username mentions when the usernames have changed etc.) I’m not a Ruby coder, and writing a simple Ruby script to post-process posts was non-trivial. Using the Discourse API was much easier.

Features

I’ve tested the script manually on ~500+ posts in the Quantified Self Forum / Zeo thread and am quite satisfied with it.

14 Likes

Exactly what I was looking for, thank you!

I’m looking to mass search/replace phpbb [youtube] tags with actual youtube links from the phpbb importer, and this is a great first step.

PR for adding support for [youtube] tags.
https://github.com/dandv/discourse-post-process/pull/3

I’ve thrown a PR your way for dry-run option, too.
https://github.com/dandv/discourse-post-process/pull/2

3 Likes

This is awesome!

Can you tell me, how tailored is this to MYBB. I’m considering running it on PHPBB imported data. And i’m just wondering if there are any gotcha’s i should watch out for?

Seems @anon36895752 ran into some scaling issues, explained in a different topic:

@dandv maybe the description should mention some ballpark numbers on maximum recommended forum size to use this method on.

I’ve run my script in batches just to make sure it wouldn’t break too many posts at once, and without any rate limits added or disabled. I think the largest batch was 1000 posts or so, but I haven’t run into Discourse limits.

Is this one still working with latest Discourse version?

I would imagine so.

See also Replace a string in all posts