Replace a string in all posts

Documentation Self-Hosting
1

Want to replace a string in all the posts? Let’s do it!

Access your site

First connect to your Droplet via SSH, and enter the Docker container for your Discourse instances:

cd /var/discourse
./launcher enter app

:warning: WARNING

We strongly recommend you take a full backup before proceeding, and make sure your string replacement is specific enough to affect only the places you want it to. If this string replacement goes wrong, every post on your site will look broken!

Remap all posts containing a specific string

Basic Case-sensitive Remap
Run the following command, substituting find with the string you wish to replace and replace with the replacement string. Note that this does not respect word boundaries.

rake 'posts:remap[find,replace]'

Example results:

find —> replace
Find —> Find
FIND —> FIND
finders keepers —> replaceers keepers
finding —> replaceing

This method can be useful for tasks such as replacing emojis:

rake 'posts:remap[:slightly_smiling:,:slight_smile:]'

The above command will replace all occurrences of :slightly_smiling: with :slight_smile:.

Case-insensitive Remap

rake 'posts:remap[find,replace,string,true]'

Example results:

find —> replace
Find —> replace
FIND —> replace
finders keepers —> replaceers keepers
finding —> replaceing

Regex Remap

If you need more specificity and feel a little adventurous, you can use regex!

To handle word boundaries, you currently need to format your command like so:

rake 'posts:remap[(?<!\\w)(?=\\w)find(?<=\\w)(?!\\w),replace,regex]'

Example results:

replace —> replace
Find —> Find
FIND —> FIND
finders keepers —> finders keepers
finding —> finding

Remove/Delete all occurrences of a word/string

The concept is the same as above, except you are removing a word entirely instead of replacing it:

Basic Case-sensitive Delete

rake 'posts:delete_word[word-to-delete]'

Case-insensitive Delete

rake 'posts:delete_word[word-to-delete,string,true]'

Regex Delete

rake 'posts:delete_word[\\[color=#[0-9a-fA-F]{3\,6}\\],regex]'

Example results:

[color=#ff3333] —>
[color=#ffffff]testing —> testing
[color=#101010]testing[/color] —> testing[/color]

69 Likes
147

How would I do this for just the topics in a single category?

My use case is an imported RSS feed which annoyingly displays a ? instead of a '. And as the feed is a news feed with lots of quotations, this is a problem!

1 Like
148

You’d do it from rails, and it’ll be hard because posts don’t belong to categories, topics do. Sounds like you need a plugin if this is a continual issue. You could be clever with a join or just loop like

Topics.where(category_id: 123).each do |t|
  posts.where(topic_id: t, post_number: 1).each do |p|
    if p.raw.match("?")
        p.raw.gsub!("?","'")
        p.save
    end
  end
end

If it’s not only in post_number: , don’t include that.

If it’s not a huge number of posts, then this is probably good enough, if it works at all.

1 Like
149

Thanks Jay! That would do the trick nicely.

I do need to troubleshoot it properly and see if there is an upstream fix. Or I might need to bolt something onto the RSS Polling Plugin which is a bit of a tricky beast!

1 Like
150

Just a guess, but are you seeing any other issues with posts created from the RSS feed? Are the ? characters present when you view the feed’s source? I’m wondering if what you are running into is an encoding issue.

2 Likes
151

Yeah, I’m just looking at this now. The ? are in there every time there is an unusual character, so it is a problem at the RSS source end. It turns out that ' is just the most common one, but it is also happening for ā, ", and one or two others.

Unfortunately the software company involved isn’t quite as responsive as the Discourse team :kissing_heart:! Wish me luck.

2 Likes
152

That’s what I was suspecting too. It might be as simple as better detecting the right encoding.

3 Likes
153

Any guidance to replace as below please?

[member=12345] → @

154

I used this example which works for replacing a URL. However, it’s not working for phrases.

Example:

rake posts:remap["I Don't Want This Phrase","But I Do Want This One","string",true]

I keep getting this error:

ERROR: Expecting rake posts:remap['find','replace',type] where type is string or regex

I tried it with the single quotes as shown in the error message as well, but no dice. I even tried by placing dashes between the words like in the delete word example. That gave me the same error.

Any advice?

155

I’m afraid that if you need to do something with quotes it’s best to do it in rails. With the rake task, you’ve got to deal with both bash and rails to escape. You might be able to use a bunch of backslashes (that also need to be escaped), but probably not.

Have a look at the rails example that I have above and see if that makes sense. I’ll to add something to the OP next time I have a laptop.

1 Like
156

Thanks for the response. That looks over my head, I can sure learn though.

It sounds like the quotes in the text string is what caused my issue…

My purpose, I find myself changing titles and urls to my blog, so I like to go back to the forum and update those links, the titles and urls as well. I can see this being a semi-regular task when I’m working on SEO. Any more details would be fantastic.

1 Like
157

There must be something I’m not understanding here. I’m trying to replace occurrences of :THUMBS-UP: with :+1: across almost 3M posts (including PMs). I ran:
rake posts:remap[":THUMBS-UP:",":+1:"]
And after about 50 minutes of ................. it returned 40000 posts remapped! This seems like a suspiciously even number. I did find some occurrences that were replaced with :+1: in both recent posts and posts from many years ago, but there are also a ton of :THUMBS-UP: even in the same topic threads that also have some successful :+1: replacements.

158

That sure seems a bit weird. Did you try to remap again?
You can also easily return the number of posts that still contains :THUMBS-UP: using data-explorer if this info can be of some use.

159

Yeah, it’s weird. I did try running the same command two three more times, and each time it remapped 333 more posts… :question:

In the log it’s showing a bunch of supposed replacements, except that when I reload the page and inspect the actual post it remains unchanged:

Screenshot from 2023-01-06 12-42-28
Screenshot from 2023-01-06 12-42-281066×136 11.1 KB

This seemed to me like a caching issue, so I did a redis-cli flushall and hard-reloaded Discourse, but still no change in the posts that were reported as edited in the log.

And also in the error log a bunch of messages like this:

Screenshot from 2023-01-06 12-47-36
Screenshot from 2023-01-06 12-47-361877×64 19.5 KB

160

This should be fine… but one of the problems is that it looks like you can quote other things such as commas, but you can’t since the quotes are consumed by the shell, not by the rake parser.

e.g.:

→ rake posts:remap["a,b","a+b"]
ERROR: Expecting rake posts:remap['find','replace',type] where type is string or regex

And the brackets should be quoted since they’re shell metachars themselves. You won’t even be able to run the command with a file called posts:remapa in the directory (unlikely as this may be) or with failglob set.

I think we should change these commands - they’re misleading in the sense that they look like you’re quoting the strings being replaced, but we really aren’t. The quotes are consumed by the shell; rails never sees them. No reason to even have the quotes, and if rails does see them they’ll be part of the string:

→ rake 'posts:remap["find","replace"]'
Are you sure you want to replace all string occurrences of '"find"' with '"replace"'? (Y/n)

Some more examples including how to deal with things like commas are:

rake 'posts:remap[find,replace,string,true]'
rake $'posts:remap[string with a quote\',string without a quote]'
rake 'posts:remap[a\, b,a+b]'

though, quirkily, this works:

→ rake 'posts:remap[string with a bracket] either quoted\] or not,string without a bracket]'
Are you sure you want to replace all string occurrences of 'string with a bracket] either quoted] or not' with 'string without a bracket'? (Y/n)
1 Like
161

Is it possible the raw is changed, but the (new) post is not yet baked?

162

I thought about that too, but no, I hit the edit button on those posts and the raw is unchanged. And on the successful replacements the new emoji appears without a rebake.

1 Like
163

So I’m trying to replace 600+ instances of an imported YouTube links, but it seems to only work when it’s on a line by itself.

So I need to insert a newline before the YouTube link itself. Thankfully, I have a “replaceme” type of string that can be replaced with the newline.

Is the appropriate way to do a Ruby newline? Or a Markdown newline?

Would something like this work?

rake posts:remap["replaceme","\n",string,true]

Or do I need to escape the \n a few times?

rake posts:remap["replaceme","\\\n",string,true]

Or should I aim for the markdown newline (which is a single backslash, correct?):

rake posts:remap["replaceme","\",string,true]

I guess I’d have to escape that too?

Any guidance appreciated.

164

I’d do it in rails where you don’t have to worry about so many levels of quoting.

2 Likes
165

Good idea. Escaping things makes my head hurt.

I see your code above is easy enough. Something like this?

Topics.where(category_id: 123).each do |t|
  posts.where(topic_id: t).each do |p|
    p.raw.gsub("replaceme","/")
  end
end

Something like that? I assume the markdown / is better than using CR/LF or \n, or even
or whatever?

How does one run this from the shell? Just type in “rails” and then copy the code in there?