How to script the setup wizard?

Bonjour,

It sounds like it is a FAQ and I apologize in advance if it is: I apparently did not look hard enough :blush: How could I write a script to do the same as the wizard instead of doing it via the web interface ?

It would start with something that resembles the Fabricate(admin) fixture from the spec tests. And maybe called from custom section of the container.

Thanks in advance for your help!

Wizard is half site settings, half API calls.

For API calls you can read How to reverse engineer the Discourse API and for site settings follow the example here:

2 Likes

But … maybe that’s where I’m lost: in order for me to call the API I need an API key. And therefore an admin user… which I don’t have (yet). Is it possible to call the API without an API key ? Or to obtain an API key that would be detached from a user and have admin credentials ?

It looks like I should use rake:admin to create the admin user. Rather, I think the admin user is already created but not approved so I would need to change it so it is approved.

Or maybe I can create an API key with api_key:create_master and use it to create the admin user ?

root@forum:/var/www/discourse# RAILS_DB=secondsite rake 'api_key:create_master[MASTERKEY]'
ad676e7413778aaaaa5d315c35f188591ef0edb4a4d4b2d644b9247a88421cfa

But it looks like I don’t quite understand how this master key is supposed to be used because this does not work:

# curl -X GET "https://forum2/categories" -H "Accept: application/json" -H "Api-Key: ad676e7413778aa3a5d315c35f91ef0edb4a4d4b2d644b924b7a88421cfa"
{"errors":["You are not permitted to view the requested resource. The API username or key is invalid."],"error_type":"invalid_access"}

However, if I create an admin user:

root@forum:/var/www/discourse# RAILS_DB=secondsite rake 'admin:create' 
Email:  loic@dachary.org
Password:  
Repeat password:  

Ensuring account is active!

Account created successfully with username loic
Do you want to grant Admin privileges to this account? (Y/n)  Y

Your account now has Admin privileges!

and then use the same master key specifying the newly created user, it does work:

curl -X GET "https://forum2/categories" -H "Accept: application/json" -H "Api-Key: ad676e7413778aa3a5d315c358591ef0edb4a4d4b2d644b924b7a88421cfa" -H "Api-Username: loic"
{"category_list":{"can...

and has admin privileges since it can also access /admin/site_settings/category/branding

What are you trying to accomplish exactly? You want to automate creating an admin user and setting some settings?

1 Like

Yes, exactly. I want to create a new forum from scratch and populate it with users and categories and topics, from a script that does not require any browser interaction.

Perhaps just create that database first and restore it as part of the installe process?

I don’t have any database: the users, categories and topics will be created using data from mailman2.

If you want to import the data, just do that as a separate process.

If that was a one shot, that would be ok. But I want to be able to test the import script. And in order for the test to run, it must first create a discourse from scratch. Without requiring some sort of manual intervention in the middle of the automated test :wink:

For context, this is part of the work I’m doing to import mailman2 into discourse. The test would run from tox which would:

  • create a new discourse instance
  • run various tests
  • destroy the discourse instance

Assuming a multisite installation, creating an approved admin user and an admin API key can be done with:

  • docker exec app env RAILS_DB=secondsite rake 'api_key:create_master[MYKEY]'
  • ( echo user1@example.com ; echo $pass ; echo $pass ; echo ) | docker exec -i app env RAILS_DB=secondsite rake 'admin:create'

Note: if not on a multisite installation, just remove env RAILS_DB=secondsite

And then verify that it works with

curl -X GET https://forum2/admin/backups -H "Accept: application/json" -H "Api-Key: 886171a73dd12759b5d6c1915b0f0d4475e8b3fff3d97954b95171200b6" -H "Api-Username: user1"
[]

(special thanks to Jay Pfaffman for the inspiration)

After this is done, discourse no longer requires to run the wizard, although it shows that it should be run.

I make a significant part of my living doing imports. I’m fairly certain that what I describe below is pretty much how everyone who does imports regularly does it.

What I would recommend is

  • configure and run the importer
  • test that it does what you want
  • script re-running the importer to import the data that’s acquired since the first run
  • test that
  • run the final import when all of the above works.
  • restore that data to your production server

The migration task and the setting up a production server task are totally different and have different requirements. Usually the migration task requires stuff that the production server does not (but I think that the mailmain importer is an exception).

Running a migration on a multisite server seems especially foolhardy.

3 Likes

Thanks for the advice, I’ll follow this process. There is over 100GB worth of mbox archives, it will require at least a few tests.

Regarding that… it looks like the mbox importer always creates a new category and is not able to use an existing one. Do you happen to know anything about this ?

I did not know the meaning of foolhardy: I like it and will reuse it :slight_smile:

All the more reason that re-running the import from scratch is a Bad Idea. That’s likely to take weeks. When you re-run the importer it imports only the new data (and I often modify it to entirely skip the old data).

You can probably modify the script to use an existing category by creating a CategoryCustomField. I’d recommend just starting with an empty database and letting the script create it, you can then customize it however you want and when you re-run the script it’ll continue to use it.

2 Likes

The real danger with your import is that there are likely to be differences in the data over the years, so stuff that works for 10 year old data won’t work for 5 year old and so on. But maybe you’ll be lucky.

3 Likes

Made me wonder…

1 Like

Not sure if that’s a compliment :wink:
I always use multisite for imports. It allows for multiple forums to co-exist: one that I am tinkering with while the client can review the import at its neighbour. And it’s easy to copy a database to restart from a certain point.

2 Likes

Wait. . . what!?

Well, knock me over with a feather! You’re pretty much the only person on the planet who doesn’t work for CDCK that I think knows more about imports than I do, and if you use multisite for imports then, well, I’ll consider doing the same. I generally crank up separate containers on the same host with traefik in front, so I guess it’s sort-of the same thing, but not really.

3 Likes