Redirecting old forum URLs to new Discourse URLs

downey
as @neil pointed only none existing links that will result in a 404 will invoke a permalink lookup.
which is much better then checking on each requests.

My suggestion is targeting a simple common migration, where all old links can be identified immediately.
before going via the default path. Based on a single match rule and rewrite to permalinkController. which will allow it to skip any overhead that occur on the default path.

This option though if not supported from the application level (modifiing nginx rules from application space). will require an advance user intervention (editing docker builder config).
But will still need the implementation of a “permalinkController” and route on the application level.

1 Like

Hmmm. Not sure what I saw, then. Can’t check right now, though - maybe I was using an old version of the code?

any easy way to do this in bulk? I want to move from vanillaforums but it seems like a headache to map all discussions.

2 Likes

Is there any URL I can visit that will demonstrate this feature? I’m playing around with some movie editing to showcase Discourse features, but I don’t have any footage for this very handy feature.

I think I know what might be happening.

It looks like if you have any routes defined in a plugin, the permalinks check will trigger when you access that route. I guess the plugin routes are only evaluated after all routes for the main app have been tried. This is turning out to be quite annoying as I’m using a lot of new routes in my ChattyMaps plugin :frowning:

This is what I’ve ended up adding in my plugin as I know I won’t be using permalinks (and I’m the only one using my plugin :wink:

PermalinkConstraint.class_eval do
  def matches?(request)
    return false
  end
end

Will be interested in knowing if there are any problems / disadvantages to doing this.

1 Like

Sorry to go necro on an old topic, but thanks so much!

I had to do this though because rails and ruby weren’t in my PATH:

cd /var/discourse
./launcher enter app
PATH="$PATH:/usr/local/bin"
rails c
Permalink.create(url: ‘/discussion/12345’, topic_id: 987)

Does anyone know how to delete these though?

The following does not seem to remove it:
Permalink.delete(id: 1)

Hmm, that’s weird since at that point you’re inside the docker container…

That’s not how you delete a “record” in rails. Try

Permalink.where(topic: 1234, url: "/bla").destroy
3 Likes

Sorry, but I am rather clueless about how to deal with Rails, and I need some help about how to use this for redirecting categories and subcategories.

What is the category_id of a (sub)category exactly? Is it the “category slug”, or in the case of a subcategory is it “main category slug/subcategory slug”? Or is it really some number that I need to look up somehow?

An example for redirecting subcategories would be tremendously helpful!

I’ve messed up the first Permalink that I’ve created and I don’t know how to delete it again.

didn’t work for me. I get the following error:

Permalink.where(url: ‘/c/old-category’, category_id: ‘new-category’).destroy
ArgumentError: wrong number of arguments (0 for 1)
from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.10/lib/active_record/relation.rb:414:in `destroy’

How can I do this right?

To find the id of a subcategory, you can look it up by the slug like this:

Category.find_by_slug('products').id

To delete the permalink for that url, do this:

Permalink.find_by_url("/blah").destroy

There can be only one permalink record per url, so just search by url.

3 Likes

Is there a way to include this automatically in code when I run the migration script? Struggling with how to manage this for importing from Vanilla Forums…

I’ve just created a topic map from MyBB to Discourse automatically, using the migration script.

MyBB was set to use SEO-friendly URLs without IDs in them. Now for example when I navigate to /thread-foo-bar, nginx redirects to /t/foo-bar/12. Here’s how I did it:

  1. Patch the importer to output lines that end up creating a map file to use for for nginx’s map module. For the MyBB importer, I added this code in create_posts:

    parent = topic_lookup_from_imported_post_id(m['first_post_id'])
    if parent
      puts "\nXXX #{m['topic_id']}: #{parent[:topic_id]},"
    end
    

    After that, I grepped for lines starting with XXX, removed the XXX, and made the file a JSON object, which I pasted into this script. Change the URLs to your forums, run the script, and its output will be a series of nginx map lines. I saved it as /etc/nginx/mybb2discourse.map.

  2. Configure nginx to “run other websites on the same machine as Discourse”, while making the following modifications to the nginx config file (/etc/nginx/conf.d/discourse.conf) in order to point nginx to the map file:

    • insert this at the top of the file:
    map_hash_bucket_size 128;
    map_hash_max_size 50000;  # might have to increase this
    
    map $uri $new {
        include /etc/nginx/mybb2discourse.map;
    }
    
    • then in the server section, add:
    if ($new) {
        rewrite ^ $new permanent;
    }
    
  3. Complete the nginx reload and container rebuild steps from the end of the Configure nginx… post linked above.

Would be great if someone who’s better with Ruby patched the importer to output the topic IDs map (or even better, the nginx map directly).

6 Likes

poor performance …
it will generate millions of regexp, and nginx have to process each of it in every request.

Do you have a better proposal?

Or a performance benchmark? I suspect that up to a pretty high number of regexps, the bottleneck is by far the entire Rails request processing + database lookup + response building stack, not nginx’s entirely memory-contained regexp matching.

After that, I grepped for lines starting with XXX, removed the XXX, and made the file a JSON object, which I pasted into this script…

I just copied this new piece of code into the script. How it works, how to get this json object/file?

this doesn’t seem to capture the threads with 0 replies in my mybb database :frowning: Any idea what the issue could be? Or any suggestions on a cleaner way to get a map of old to new threads?

Permalinks and normalizers are the most frustrating, unclear, under documented feature of Discourse that i’ve run into so far. Having a horrible time setting these up. Just wanted to vent my frustration here. I’ve read Problem with permalinks, or regex? as well as other posts on vbulletin specific importers.

Great feature idea, just wish i could figure out how to use it properly.

3 Likes

I understand your frustration. It took me a while to figure them out.

I think it’s because the feature is used infrequently (just when you do an import) and by relatively few people (people who write importers). And once you’ve figured it out for the current problem, you just move on.

3 Likes

I’m soon to be moving on… to another vbulletin import to Discourse :slight_smile: So I’ll share what I’m doing, and after a couple more of these i’ll compile all my lessons learned somewhere.

I wrote an importer for permalinks that solved my vbulletin4 redirects for old permalinks.

To get it to work - add the following “permalink normalizations” in Admin settings to get these redirects to work.

Example 1, you have urls like this:

/forums/f10/some-thread-title-here-51689/index1.html
/forums/f10/some-thread-title-here-51689/index15.html#323423

Normalization 1. This is for the above 2 examples. Add this normalization into the adminsetting first (order of normalizations is important!)

/(forums)/f[0-9]+/.±([0-9]+)/index[0-9]*.html/\1\2

Example 2, your vbulletin also has permalinks like this:

/forums/f10/some-thread-title-here-51689/

Normalization 2. This is for the above example permalink. Add this one normalization second (order of normalizations is important!)

/(forums)/f[0-9]+/.±([0-9]+)/\1\2

And then run this import script after completing the bulk-import or normal import scripts for vbulletin (btw, i had to use both official import scripts, and modified them because neither solved my needs alone: forum around ~1million posts)

5 Likes

Could you add that to the script and submit a PR?

Also, you can set the permalink normalizations in the script (rather than the web interface) something like this:

SiteSetting.permalink_normalizations='/topic/(.*t)\?.*/\1'

If you don’t know what logic is required to know which permalink normalization to use, just pick one and add the other one as a comment. People running the importer will see the code before they can find this thread. :slight_smile:

1 Like

If you need to delete all Permalinks at once, use Permalink.all.each { |p| p.destroy } from rails console.

3 Likes