In a migration discussion titled ‘Populating topic_links table,’ stuwest seeks a way to populate the topic_links table during a custom import from bespoke forum software into Discourse. Initially unsure whether a rake task exists for this, pfaffman suggests links are generated during post baking. stuwest explains their manual regex cleanup approach due to time constraints but acknowledges rebaking may be necessary. pfaffman warns against skipping rebaking, recommending testing on sample posts and considering timeline extensions or staging the import to avoid complications. stuwest confirms that stripping HTML tags while preserving key elements like <a> and <b> leaves raw content suitable for rebaking — and verifies that marking posts as uncooked and triggering rebake uncooked posts successfully populates topic_links. They clarify they built a custom importer from scratch. pfaffman celebrates the solution, affirming that useful raw content enables safe, post-launch rebaking. The issue is resolved via rebaking, not a dedicated rake task.
I’m in the middle of writing the process to migrate from bespoke forum software to Discourse. My import script is not currently adding entries to the topic_links table, and I wanted to ask if there is something, like maybe a rake task, that can do that.
(Clicktracking for imported posts is a “nice to have” rather than an essential and granted I don’t see anything for it under rake --tasks but since I can’t see rake search:reindex there either and I’m using that during the import I thought it might be worth checking.)
Any response appreciated, even if it’s just a “no, not currently possible.”
Could be! I have 750,000 posts to import and about one week to get it all finalized (because of hosting company shenanigans, don’t ask) so my current approach is just to clean up the imported posts manually with regexes rather than go through the rebake process. If that is how the links are generated, we can look at rebaking everything at some point in the future.
That is unlikely to end well. Posts will need to be rebaked at some point in the future.
You can check one post with a link and see if a rebaked fixes it. You could then rebake just the ones with links.
Are you starting with one of the existing import scripts?
If you have a database dump, worst case you could extend your time line by putting up a placeholder page for a bit while you finish the import. It’ll be much worse to put up a botched import and allow people to add new posts, as it’s much, much easier to do an import on an empty site. Another solution would be to put up a new forum while you finish your script and then put the forum on hold for a bit while you run the final import to add the old data to the new forum.
As far as I can tell it should be okay. The forum software I’m migrating from basically allowed users to put whatever HTML tags they wanted in their posts, so my method was to strip out all tags, with a handful of exceptions like <b> (and </b>), <a>, <blockquote> and so on. Currently the only difference between cooked and raw posts in my database is that there are newlines instead of p and br tags in the raw version.
I’ve set a couple of hundred posts as uncooked and started a rebake uncooked posts task on my dev server just now and it does seem to be populating the topic_links table, so thanks! Problem solved.
No, I rolled my own, following the steps of one of the bulk importers.