How best to import users to migrate a mailing list to Discourse

(Tobias Eigen) #1

I am in the process of migrating a handful of related mailing lists (google groups, actually) to private categories in a new Discourse instance. These are people (about 50) who all work for the same organization and are required as part of their job to be in these mailing lists. We don’t care about the archives, just the users.

I’ve spent some time searching meta for info on this and seem to be either sent to the import scripts (which are about importing from forums including content and require programming) or to the invite system (which requires acceptance of invitation by the user).

Correct me if I’m wrong, but the invite system seems inappropriate because we just want to copy over the lists and have them working in discourse just as they were working on google groups before. We do not want to have to chase down everyone and make sure they click to accept the invitation before they start getting emails and are able to send emails.

If anyone has used the import scripts with success just to import say a CSV of users along the lines of the invite system, I’d really appreciate a HOWTO on the steps to getting it done.

Is there anyway to add many users by admin?
(blaumeer) #2

I don’t have an answer for you.

However 50 users is a fairly small number that can be managed with the invite system. You are notified every time an user accepts the invitation, and you can resend the invitation when it expires after 30 days. I managed a comparable number of users and found that manageable.

(Michael Downey) #3

This is along the same lines of the problem I’m facing, not able to import all my existing LDAP directory users. Since we use SSO, there doesn’t seem to be any way to automatically create them without having them manually sign in to Discourse.

We won’t be able to successfully migrate our remaining large lists to Discourse until there’s a way to reconcile this problem. Those users need to be created, verified, and set up for email notifications for a specified category automatically.

(blaumeer) #4

Then you could add the users using SQL statements in the database directly?

(Tobias Eigen) #5

Yes, that’s an excellent suggestion. Do you know how to do this and if so can you share the steps?

(phil) #6

I am in the same boat, the problem is that discourse is not based on Mysql but Postgre instead
And I have no clue about Postgre :frowning:
I am surprised nobody build ap login to IMPORT USERS.

I mean there is a perfectly good EXPORT button in the admin / user page

If only we could import user using a structured file, that would be an IMPORTANT feature I think…

(blaumeer) #7

Unfortunately I don’t know how to do it. I would start connecting to the DB as explained here and then explore the db structure and see where user data are stored.

There is a meta or github page examples interacting with D pgsql, but I do not recall it right now.

(Dave McClure) #8

Rather than hack the database directly, I think you should take a look at what other import scripts do for importing users.

I would look at that and the code for the bulk invite system that @techAPJ worked on to see what can be pieced together:

It may be worth opening a separate feature request topic for this since other mailing list systems have this ability:

(Alessio Fattorini) #9

Could you open a feature request? It worth for me, I have the same issue

(Tobias Eigen) #10

I think there are actually good design reasons NOT to have user import available within the UX alongside the invite feature, and value the invite feature to encourage/facilitate the best, most civilized behavior by discourse admins as they ramp up their new forums. Discourse is still best for starting new communities and inviting people is better than just adding them.

So making admins do a little work as they migrate their mailing lists to discourse is a Good Thing. What I’d be very happy with (and perhaps contribute some dollars and attention of my (volunteer) developers to) is an import script and HOWTO for importing users from the command line using a prepared CSV.

That said, if someone else develops an optional plugin to add a tickbox on the invite dialogs to “import instead of invite” I wouldn’t say no to that either. :speak_no_evil:

(phil) #11

Everybody has their own reason wanting to add users.

For example, I want to be able to create a couple of key user and reserve some specific usernames before they are taken, so that my staff can have uniform names.

If I ask my staff to register, then I’ll have to explain each one of them what I need, and I know it will not be done right the first time.

As simple as that.

(Kane York) #12

Keep in mind that admins can change the usernames of users, and delete users, with impunity. It’ll be a bit damaging to do it often, and at all with established accounts, but it’s very available to you.

(Tobias Eigen) #13

… and I think this is a prime example of what I mean when I say it is ok to make admins do a bit of work to set up users. It’s not that much effort to edit the settings of a few users.

I have found so far that most issues can be resolved using existing features and some long handed manual steps, and I am ok with that.

(Vincent Oord) #14

For what it’s worth, I’ve created a fairly simple importing script for importing users based on a CSV file. It’s written mostly for my own requirements (which were very simple), but I’ll happily receive contributions to this script. If anyone wishes to do so, just create a pull request.

Adding members to groups via CSV
(Stefan) #15

@vindia your script is really useful for simple user migrations from other forum platforms as well (IP.Board in my case). But I was wondering if there any way to handle banned user. I’m not a ruby/rails developer, but I think I can change it to handle this if somebody points me in the right direction.

In IP.Board, banned users are basically part of a group. In Discourse it seems things work a bit different because you don’t have negative permissions. So even if I create a banned group, there is no easy way to limit the access for that group. Of course I could limit access for the banned group by not assigning permission to “everyone” and assign permissions to individual groups. But is obvious this is not feasible. So my question is, what should I do to your script to also add some users as banned? An easy solution I guess would be to have approved set to false, but it would be better if I could add them as banned in the real sense.

I need to import banned users because I don’t want to leave those out and then somebody else can come and register a username that was banned in my old forum platform.

[later edit]

Looking at other import scripts I’ve found, b['ban_give_reason']) So maybe something like this in an if statement?

system_user = Discourse.system_user, 'Imported banned user')

I’ll probably try it out this weekend.

(Kane York) #16

You want to set the suspended_at and suspended_till fields on the user.

(Vincent Oord) #17

Yeah, what @riking says: in the method you should add two lines for suspended_at and suspended_till and you should file them with a type of Date, for instance the Date of today which you get in Ruby with the method

suspended_till: + 100 // 100 days in the future

You might want to use a date way in the future if you want to suspend someone “for ever”.


Awesome! Do you have instructions on how to use this script (commands etc) within the docker container? For example, I don’t know how to access lib/tasks

(Vincent Oord) #19

Yeah it’s a basic Rake task as used in any Rails application.

From the root dir of your app, inside the docker container, just do

rake user_importer:check[/path/to/users.csv]


@vindia Thanks for responding! I was referring to your initial instruction on adding the script to lib/tasks. Thus far I’ve only been adding plugins through nano containers/app.yml.

How exactly do I add your script to lib/tasks?