Importing users and assigning a post count


(John R.) #1

Currently I am working on getting the TheDailyWTF forum integration complete, in particular, featuring comments, Side Bar WTF integration, creating topics when articles are created, deleting moderator action posts, and setting topic visibility.

The last piece of the puzzle is importing the existing users. What we’ve decided is to simply import the usernames and email addresses of any users whose combined comment count + forum post count >= 100 (there are 584 of these users total). I have gathered this list, but I’m a little stuck as to how to continue with an import. I’ve looked through this thread to see if there was a standard way, and I tried to create a user by POSTing to /users various combinations of values for name, username, email, and password (not a real one of course) – but that typically errored out with “access denied” or seemed to indicate an email was sent but I never received one.

In addition to that, I am wondering if there is a way to set the post count and trust level via the API for each of these users so long-time users can be distinguished from a new user in multiple ways.


(Kane York) #2

Because you only want to import the users, this should be fairly simple and not fraught with too many edge cases and formatting differences like importing posts is. I gave it a try with some test data.

This will NOT send emails to the users. Also, my file is in a pipe-separated value format, and you may find some other format easier to generate. See below.

My input file, emails:

kane@laptop:~$ cat emails
usernamea1 | email1@example.com | 1
usernameb2 | email2@example.com | 2
....
usernamez26 | email26@example.com | 8
usernamea27 | email27@example.com | 627
usernameb28 | email28@example.com | 28

We want to get this on the server and then get into a Ruby console. Let’s do that:

# This or filezilla or whatever:
$ scp emails forum:~
# This or PuTTY:
$ ssh forum
$ cd /var/docker
$ cp /root/emails shared/emails # if you uploaded the file somewhere else, change the first arg here
$ ./launcher ssh app
# These two lines launch pry instead of irb, which is generally better with its highlighting and everything
$ gem install pry-rails
$ (cd /var/www/discourse && RAILS_ENV=production sudo -E -u discourse bundle exec pry -r ./config/environment)
[0] pry(main)>

Decide how you want to divide up trust level 1 and 2 among the imported users, and fill in your decisions: (It’s not a good idea to automatically assign TL3 quite yet.)
I have a definition for 0 because I wasn’t quite reading your post right - you won’t have to worry about that. Feel free to omit that line.

[20] pry(main)> def trust_from_post_count(count)
[20] pry(main)*   return 0 if count <= 2
[20] pry(main)*   return 1 if count < 200
[20] pry(main)*   2
[20] pry(main)* end  

Okay, so what I’m going to do now is try to parse the first line of the file, and then the code I come up with will be put into the file-reading loop. If you need to make changes so that your format works, repeat these few lines until they do, then put the sequence you get into the loop.

[13] pry(main)> firstline = File.readlines('/shared/emails').first
=> "usernamea1 | email1@example.com | 1\n"
[14] pry(main)> arr = firstline.split '|'
=> ["usernamea1 ", " email1@example.com ", " 1\n"]
[15] pry(main)> arr.each {|x| x.strip! }
=> ["usernamea1", "email1@example.com", "1"]
[16] pry(main)> u = User.new(username: arr[0], name: arr[0], email: arr[1])
=> #<User id: nil, LOTS AND LOTS OF OUTPUT........... >
[17] pry(main)> u.trust_level = trust_from_post_count(arr[2])
ArgumentError: comparison of String with 2 failed
from (pry):17:in `<=`
[18] pry(main)> u.trust_level = trust_from_post_count(arr[2].to_i)
=> 0
[18] pry(main> # u.save

That last line is commented for a reason - you might revise this portion, but you want to write to the DB in one go! Partial migrations / imports are never fun.

So, once you’ve got a User output that looks good, pull the trigger:

[27] pry(main)> File.readlines('/shared/emails').each do |line|
[27] pry(main)*   arr = line.split "|"  
[27] pry(main)*   arr.each { |x| x.strip! }  
[27] pry(main)*   u = User.new(username: arr[0], name: arr[0], email: arr[1])  
[27] pry(main)*   u.trust_level = trust_from_post_count(arr[2].to_i)  
[27] pry(main)*   u.activate
[27] pry(main)*   u.save
[27] pry(main)* end  
=> [ (basically the file contents) ]
[29] pry(main)> File.readlines('/shared/emails').count
=> 28
[30] pry(main)> User.count
=> 36

The counts check out! :smile: (The extras you’re seeing are things like system, me, and my Test Accounts Blue, Orange, Purple, and Green.)


(Sam Saffron) #3

Trust level is easy to set via the API, go to the admin user screen, open chrome inspector, see the API call it makes when changing a trust level

Post counts should not be touched, we are likely to have a scheduled job that will reset it to the correct number at some point.

Regarding user creation. You have a few options:

  • Use the API endpoints that invite users and simply invite them to a topic.
  • Use ruby object model to create the users
  • Implement an admin create user API and use it

Existing API endpoints for user creation will require some sort of account activation.


(John R.) #4

@riking - thanks for the detailed steps! I was able to get the users in the system using your instructions fairly easily.