Redirecting old forum URLs to new Discourse URLs

import

(Neil Lalonde) #1

If you’ve moved from other forum software to Discourse using one of our import scripts, then you probably want all your hard-earned Google search results to continue pointing to the same content. Discourse has a built-in way to handle this for you as an alternative to writing nginx rules, using the permalinks lookup table.

The permalinks table allows you to set two things: a url to match, and what that url should show. There are a few options for defining where the url should redirect to. Set one of these:

  • topic_id: to show a topic
  • post_id: to show a specific post within a topic
  • category_id: to show a category
  • external_url: to redirect to a url that may not belong to your Discourse instance

For example, if your original forum’s topic urls looked like http://example.com/discussion/12345, and the url for that topic after the import is http://example.com/t/we-moved/987, then you can setup the mapping like this:

cd /var/discourse
./launcher enter app
rails c
Permalink.create(url: '/discussion/12345', topic_id: 987)

Discourse will then perform a redirect with http response status code 301 (permanently moved) to the correct url for topic id 345. The 301 should cause search engines to update their records and start using the new urls.

If you want some urls to redirect away from Discourse, you can do so by setting external_url:

Permalink.create(url: '/discussion/12345', external_url: 'http://archived.example.com/discussion/12345')

Redirect old URLs
Redirecting vbulletin urls to discourse
When a category is renamed, the previous URL should redirect to the new one
301 redirect to my old forum archive
Importing from phpBB3
Correct way to add rewrite rules to Nginx
Custom permalinks
Best practices for URL forwarding/rewriting from a previous (non-Discourse) system
vBulletin migration guide?
Redirect vBulletin URLs to Discourse URLs
Permalink overhead
Redirect vBulletin URLs to Discourse URLs
(Robin Ward) #2

We’ll probably want to create an admin interface to create / review these at some point too for those who aren’t terminal jockeys like us.


(Jeff Atwood) #3

I think a better solution is a plain text file with simple key, value mappings in it… like we do with batch (mass) user invites. Not a priority at the moment though.


(Neil Lalonde) #4

True, but I imagine the typical use is to create thousands of these records, using a script like this for a Drupal import:

PostCustomField.where(name: 'import_id').find_each do |pcf|
  next unless pcf.value[0,3] == 'nid' # if it's a Drupal topic id
  imported_id = pcf.value[4..-1]
  topic = pcf.post.try(:topic)
  if topic
    unless Permalink.where(url: "discussions/#{imported_id}", topic_id: topic.id).exists?
      print '.'
      Permalink.create(url: "discussions/#{imported_id}", topic_id: topic.id)
    end
  else
    print "X"
  end
end


(Robin Ward) #5

Yeah an interface is only good for a few records.


(Kane York) #6

Can we have a site setting to turn on the permalink? As it is, it’s causing a database ‘exists’ query on every page load. (You can see this in the rails console)


(Jeff Atwood) #7

Agree, @neil, can we put this behind a site setting? We need to be careful that we’re not adding query load on every page as we build new stuff.


(lid) #8

I like this feature.

I am thinking perhaps performance could be improved if we dedicate a controller for old links
and using nginx rewrite to direct old links to the “permalink” controller.

This way only links going via the permalink controller will need a db lookup.

  1. Adding a rewrite rule to nginx to match old forum pattern(lowest common denominator)

  2. Permalink controller that will be used to make the db lookups

The challenge will be how to add that rule to nginx from the application.
and to prevent it from conflicting with core paths.

Just some food for thoughts.


(Neil Lalonde) #9

Totally not true. We’ll only look for a permalink record right before rendering the 404 not found page, like this request:

I, [2014-10-12T14:48:33.642383 #67907]  INFO -- : Started GET "/bad/url/for/me" for 127.0.0.1 at 2014-10-12 14:48:33 -0400
D, [2014-10-12T14:48:33.717172 #67907] DEBUG -- :   Permalink Exists (1.2ms)  SELECT  1 AS one FROM "permalinks"  WHERE "permalinks"."url" = 'bad/url/for/me' LIMIT 1
F, [2014-10-12T14:48:33.720664 #67907] FATAL -- : 
ActionController::RoutingError (No route matches [GET] "/bad/url/for/me")

Adding an nginx rule has always been an option, but it should be added through your docker config, not from the application. The method described here is an alternative to writing nginx rules, but definitely not the only way to do it.


(Michael Downey) #10

Wouldn’t it be insanely faster and less resource-intensive to provide nginx a rewrite map file to handle this, instead of making Discourse process every request? Maybe I’m missing something obvious. :slight_smile:


et al.


Correct way to add rewrite rules to Nginx
(lid) #11

downey
as @neil pointed only none existing links that will result in a 404 will invoke a permalink lookup.
which is much better then checking on each requests.

My suggestion is targeting a simple common migration, where all old links can be identified immediately.
before going via the default path. Based on a single match rule and rewrite to permalinkController. which will allow it to skip any overhead that occur on the default path.

This option though if not supported from the application level (modifiing nginx rules from application space). will require an advance user intervention (editing docker builder config).
But will still need the implementation of a “permalinkController” and route on the application level.


Does Discourse support unique post IDs?
(Kane York) #12

Hmmm. Not sure what I saw, then. Can’t check right now, though - maybe I was using an old version of the code?


Best practices for migrating to hosted Discourse from other forum software
(Ricardo Viteri) #13

any easy way to do this in bulk? I want to move from vanillaforums but it seems like a headache to map all discussions.


(Erlend Sogge Heggen) #14

Is there any URL I can visit that will demonstrate this feature? I’m playing around with some movie editing to showcase Discourse features, but I don’t have any footage for this very handy feature.


(etewiah) #15

I think I know what might be happening.

It looks like if you have any routes defined in a plugin, the permalinks check will trigger when you access that route. I guess the plugin routes are only evaluated after all routes for the main app have been tried. This is turning out to be quite annoying as I’m using a lot of new routes in my ChattyMaps plugin :frowning:

This is what I’ve ended up adding in my plugin as I know I won’t be using permalinks (and I’m the only one using my plugin :wink:

PermalinkConstraint.class_eval do
  def matches?(request)
    return false
  end
end

Will be interested in knowing if there are any problems / disadvantages to doing this.


(Daytona) #16

Sorry to go necro on an old topic, but thanks so much!

I had to do this though because rails and ruby weren’t in my PATH:

cd /var/discourse
./launcher enter app
PATH="$PATH:/usr/local/bin"
rails c
Permalink.create(url: ‘/discussion/12345’, topic_id: 987)

Does anyone know how to delete these though?

The following does not seem to remove it:
Permalink.delete(id: 1)


Custom permalinks
(Régis Hanol) #17

Hmm, that’s weird since at that point you’re inside the docker container…

That’s not how you delete a “record” in rails. Try

Permalink.where(topic: 1234, url: "/bla").destroy

(Michael Hrenka) #18

Sorry, but I am rather clueless about how to deal with Rails, and I need some help about how to use this for redirecting categories and subcategories.

What is the category_id of a (sub)category exactly? Is it the “category slug”, or in the case of a subcategory is it “main category slug/subcategory slug”? Or is it really some number that I need to look up somehow?

An example for redirecting subcategories would be tremendously helpful!

I’ve messed up the first Permalink that I’ve created and I don’t know how to delete it again.

didn’t work for me. I get the following error:

Permalink.where(url: ‘/c/old-category’, category_id: ‘new-category’).destroy
ArgumentError: wrong number of arguments (0 for 1)
from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.10/lib/active_record/relation.rb:414:in `destroy’

How can I do this right?


(Neil Lalonde) #19

To find the id of a subcategory, you can look it up by the slug like this:

Category.find_by_slug('products').id

To delete the permalink for that url, do this:

Permalink.find_by_url("/blah").destroy

There can be only one permalink record per url, so just search by url.


(Dylan Burkhardt) #20

Is there a way to include this automatically in code when I run the migration script? Struggling with how to manage this for importing from Vanilla Forums…