Issues while migrating from Kunena

import

(Dave) #1

Hi all,

I’m having great fun with my first Discourse install - it’s a fab bit fo kit! I’m new to Ruby et al and looking for a spot of help with kununa.rb

I can get the script to run, having searched a lot :wink: but there is an issue with the script.

In the latest version of kunena the ‘categories’ table is different - field ‘parent’ is now called ‘parent_id’ and I’ve no idea how the older tables were setup, but when changing the field value in the kunena.rb script the categories import doesn’t work as it should.

Not knowing enough about Ruby I can’t quite figure out how the import process is working. What am I not doing?

Any help greatly appreciated, any more details needed please let me know.

Here’s what’s happening:

changed kunen.rb portion (changed ‘parent’ to ‘parent_id’)

    create_categories(@client.query("SELECT id, parent_id, name, description, ordering FROM jos_kunena_categories ORDER BY parent_id, id;")) do |c|
  h = {id: c['id'], name: c['name'], description: c['description'], position: c['ordering'].to_i}
  if c['parent_id'].to_i > 0
    parent = category_from_imported_category_id(c['parent_id'])
    h[:parent_category_id] = parent.id if parent
  end
  h
end

and the output -

root@acc:/var/www/discourse# RAILS_ENV=production sudo -E -u discourse ruby script/import_scripts/kunena.rb 
loading existing groups...
loading existing users...
loading existing categories...
loading existing posts...
loading existing topics...
fetching Joomla users data from mysql
fetching Kunena user data from mysql
creating users
     1464 / 1464 (100.0%)  	

[..list of category names...]

creating topics and posts
        3 / 23141 (  0.0%)  Parent post 0 doesn't exist. Skipping 5: Moderator Help!
        4 / 23141 (  0.0%)  Parent post 0 doesn't exist. Skipping 6: Back End Login
        5 / 23141 (  0.0%)  Parent post 0 doesn't exist. Skipping 7: Article creators
        6 / 23141 (  0.0%)  Parent post 7 doesn't exist. Skipping 8: Article creators
        7 / 23141 (  0.0%)  Parent post 8 doesn't exist. Skipping 9: Article creators
        8 / 23141 (  0.0%)  Parent post 9 doesn't exist. Skipping 10: Article creators
        9 / 23141 (  0.0%)  Parent post 7 doesn't exist. Skipping 12: Article creators
       10 / 23141 (  0.0%)  Parent post 12 doesn't exist. Skipping 13: Article creators
       11 / 23141 (  0.0%)  Parent post 5 doesn't exist. Skipping 14: Moderator Help!
       12 / 23141 (  0.1%)  Parent post 12 doesn't exist. Skipping 15: Article creators
       13 / 23141 (  0.1%)  Parent post 15 doesn't exist. Skipping 16: Article creators
       14 / 23141 (  0.1%)  Parent post 16 doesn't exist. Skipping 17: Article creators
       15 / 23141 (  0.1%)  Parent post 14 doesn't exist. Skipping 18: Moderator Help!
       16 / 23141 (  0.1%)  Parent post 0 doesn't exist. Skipping 19: Avatars
       17 / 23141 (  0.1%)  Parent post 7 doesn't exist. Skipping 20: Article creators
       21 / 23141 (  0.1%)  Parent post 19 doesn't exist. Skipping 24: Avatars
       23 / 23141 (  0.1%)  Parent post 0 doesn't exist. Skipping 26: Acceptable Useage Policy

it makes the first few categories - not quite correctly then refuses to import any posts - It seems that the initial categories aren’t being created properly so it doesn’t know where to import the posts.

Cheers all,

Dave


Migrating to Discourse from another Forum software
(Dave) #2

I think I might be a step closer!
The importer is working, i think, but my original board has too many levels of sub-category.
I have:
Main Cat > Sub Cat > Sub Sub Cat

it’s those Sub Sub Cats that aren’t being brought across right, therefore the posts aren’t getting imported - I think.

Does that make sense?

the 2 boards are:

http://aircadetcentral.net/acc-clone
http://acc.maia-internet.com

edit

ok, so I cloned the main site and re-arranged the categories and they import fine now.

Howerver the posts still don’t import. What am I missing???

creating topics and posts
        2 / 23143 (  0.0%)  Parent post 2 doesn't exist. Skipping 4: The new mess
        3 / 23143 (  0.0%)  Parent post 0 doesn't exist. Skipping 5: Moderator Help!
        4 / 23143 (  0.0%)  Parent post 0 doesn't exist. Skipping 6: Back End Login
        5 / 23143 (  0.0%)  Parent post 0 doesn't exist. Skipping 7: Article creators

cheers


(Dave) #3

All,

from the debugging i’ve done - and i’m not sure if i’m right, the problem lies with this line:

<127>          parent = topic_lookup_from_imported_post_id(m['parent'])

it’s returning a null value for parent therefore the script doesn’t create any posts as it doesn’t know where to put it!

suggestions please!

cheers

Dave


(Gerhard Schlager) #4

Discourse supports only one level of subcategories.
As a workaround I suggest you rearrange your categories in your old forum before doing the import.


(Dave) #5

Hi Gerhard,

Yes, I found that, as stated above. The categories now import fine, it’s the posts that aren’t being imported!


(Gerhard Schlager) #6

Ah, sorry, I should have read your whole post. :wink:

I’m not using this importer, so just a guess…
It looks like the posts are imported in the wrong order.
Take a look at your database and execute the following query.

Do you see rows where parent > id? If this is the case, the ORDER BY needs to be changed.
What are the values of id and thread of the first post in each topic? They should be the same otherwise the check in line 123 won’t work.


(Dave) #7

Hi Gerhard,

that line <123> was confusing me because I am indeed seeing most rows where parent > id. I’m not at my regular computer at the minute so I’ll try later today and play with the ordering.

Thanks for your help :smile:


(Dave) #8

Hmmm, on the other hand the id field is a 1 up serial nothing to do with the post/parent/thread/category - it seems. Here’s the first few lines of the MySQL output using the query in the script:

id 	thread 	parent 	catid 	userid 	subject 	time 	message 	
2 	2 	0 	2 	717 	The new mess 	1350613779 	
3 	3 	0 	3 	717 	The Crew Room 	1350937488 
4 	2 	2 	2 	718 	The new mess 	1350938408 	
5 	4 	0 	2 	717 	Moderator Help! 	1350939426 	
6 	5 	0 	2 	717 	Back End Login 	1350939575 	
7 	6 	0 	2 	717 	Article creators 	1350941108
8 	6 	7 	2 	721 	Article creators 	1350942433 	
9 	6 	8 	2 	717 	Article creators 	1350942649
10 	6 	9 	2 	720 	Article creators 	1350942806 
12 	6 	7 	2 	717 	Article creators 	1350943114 
13 	6 	12 	2 	721 	Article creators 	1350943209
14 	4 	5 	2 	721 	Moderator Help! 	1350943410
15 	6 	12 	2 	717 	Article creators 	1350943428
16 	6 	15 	2 	721 	Article creators 	1350943550
17 	6 	16 	2 	717 	Article creators 	1350943618
18 	4 	14 	2 	717 	Moderator Help! 	1350943656
19 	7 	0 	2 	717 	Avatars 	1350944230 
20 	6 	7 	2 	720 	Article creators 	1350944408 
21 	2 	4 	2 	723 	The new mess 	1350979104 
22 	2 	21 	2 	717 	The new mess 	1350979893 
23 	2 	22 	2 	717 	The new mess 	1351001556 
24 	7 	19 	2 	717 	Avatars 	1351001703 	
25 	2 	23 	2 	717 	The new mess 	1351002874 
26 	8 	0 	16 	717 	Acceptable Useage Policy 	1351006949 
27 	9 	0 	17 	717 	ACC BETA 	1351114404 	

the first 2 threads get imported - sort of! but the rest all fail hmmm


(Gerhard Schlager) #9

Just by looking at it I’d say change line 123 to the following:

if m['parent'] == 0

I’m not sure if thread is even needed. Maybe someone who knows more about kunena can chime in…


(Dave) #10

Wow!

It’s importing! Let’s see what the results are! … :smile:


(Dave) #11

Well,

most of the posts were imported using @gerhard 's suggestion about line 123 in kunena.rb

I’m guessing that some posts didn’t get created because of the content of the post - am about to investigate

Also the script only got to 65.7% of the import - no reason given but could be an overload (there’s 23000+ posts to import) and it’s only a little test DO droplet.

On the plus side the script appears to work now. Where / who do I see about updating the kunena.rb file, or indeed adding another file for kunena v3.0.6 which is the version i’ve been working with?

cheers and thanks for everyone’s help

Dave


(Gerhard Schlager) #12

@neil Can you please take a look at this? For which version did you write the original importer?


(Neil Lalonde) #13

I’ll find out what version of Kunena that export was from. I have no idea…

There are two options for supporting multiple versions of Kunena:

  1. Instead of updating the existing kunena.rb, submit a new file called something like kunena3.rb.
  2. If there are only a few changes to support 3.0, then put a version constant at the top, like KUNENA_VERSION = 3.0 and the script can check that when it needs to:
if KUNENA_VERSION == 3.0
  # do stuff for 3.0
else
  # do stuff for whatever version Neil worked on
end

(Dave) #14

Thanks @neil I reckon a script for kunena 3 might be the way forward. How do I go about submitting the file?

cheers


(Neil Lalonde) #15

Ok that sounds good. Submit it in a pull request. :smile:


(Dave) #16

Hi guys,

still in the process of sorting out the update for the converter - they updated kunena so I still need to check that everything works.

As a side - how would one then replace the kunena forum with Discourse? In other words, in it’s simplest form, iframe discourse into a joomla frame work? - not worried about users as the import process brings them over and I have ho need for users to log into joomla (not yet anyway).

Your help please :smile:


(Dave) #17

Hi again guys,

here’s the next hurdle to migrating - Private Messages.

Is there a (easy) way of migrating form Uddeim to Discourse?

Cheers


(Dave) #18

Hi @neil @gerhard @Benninger ,

I submitted a pull request and had it merged yesterday for kunena3.rb I finally got it to work for kunena 3+ installs!

It even works running against an exiting import that updates the new from the old.

Hope this helps others out as well.

Cheers

Dave


#19

Hi guys,

I’m struggling with getting this import script to run - if anyone out there is willing to give me a hand it would be greatly appreciated. I have 2 sites, one small and one fairly big that I would love to migrate but I’ve been hitting a brick wall with this the last few weeks.

TIA!

/Kohhal


(Tarek Loubani) #20

@maia_internet Thank you. This worked amazing!

@Kohhal: What is the issue? It would be more useful for you to articulate what the error is rather than leave it for us to guess.

tarek : )