Download and restore of site settings

For sites interested in moving to Discourse from another forum or trying out Discourse before an import, it’s tedious to see that settings get moved from a test site to the live one, or that settings done on a live instance can be easily moved to a restore from an import that started with a clean slate.

Now that themes are supported, part of this problem is solved, but it’s still tedious to copy potentially dozens of settings from one site to another.

I started looking at a clean (yet naive) way to do this and started by looking at what data explorer returns for select * from site_settings. That was a good start, but if I reset a value via the web interface, the value still shows up in data explorer. Is it different in rails?

Edit: Another example of my being crazy or looking at the wrong tab. I see now that the setting I changed is still there (of course) but is now blank (as I’d expect). I’d still welcome input on how to best go about this. A plugin that spits out JSON and sucks in an upload?

Any hints on how to go about this?

6 Likes

Now you can, if you can run a ruby script.

https://github.com/pfaffman/discourse-settings-uploader

To create/retrieve settings

Create a JSON settings dump by visiting https://discourse.example.com/admin/site_settings.json and saving the file somewhere convenient.

Retrieve your site’s API key from https://discourse.example.com/admin/api/keys

To install these settings to another site

Do this:

./discourse-settings-uploader HOSTNAME API_KEY API_USER SETTINGS_FILE

For example:

./discourse-settings-uploader discourse.example.com d35989078a system site_settings.json

Warning

This script does minimal error checking and could destroy your site, cause fires, or hurt people.

You should make a backup of your site before you use this script.

17 Likes

We should pull this into a rake task, cause I want core to ship with it.

7 Likes

I’d be flattered.

Would it restore the settings to the local instance, or, as my script does, push them to a remote instance via the API?

On a related note, a rake task to call create_master_key to make it easier to get the API key from a script would be useful.

I’ve tried

echo "ApiKey.create_master_key" | docker exec -w /var/www/discourse -i app bundle exec rails c

but it fails with FATAL: Peer authentication failed for user "discourse" (PG::ConnectionBad).

6 Likes

I would just use straight json here:

rake site_settings:export > settings.yml
rake site_settings:import < settings.yml

or something like that.

@blake is also interested so I think he can probably help.

9 Likes

A rake task for this now exists!

https://github.com/discourse/discourse/commit/925d1a78697dfb8e9549e0b9ba4f7e7f18235cfe

Here is an example of an import. It checks for bad settings (NOT FOUND), should show any errors (setting a username that doesn’t exist), and show you which settings changed, and will only update a value if the new setting is different from the current value.

$ rake site_settings:import < ~/tmp/settings.yml

starting import...

NOT FOUND: existing site setting not found for blake_key
Changed title FROM: Discourse TO: Blake's Site
Changed site_description FROM:  TO: Blake's test site
ERROR: site_contact_username: There's no user with that username.
Changed invite_only FROM: f TO: true
Changed login_required FROM: f TO: true

Results:
 Updated:   4
 Not Found: 1
 Errors:    1

Afterwards you can also check the Staff Action logs to verify the changes.

@pfaffman I do still really like your api version, and I would be the perfect tool to add to the discourse CLI app that I started working on a couple of years go that I haven’t had time for but would still like to exist.

14 Likes

Cool. Can’t wait to use it.

Will it work on a file downloaded from /admin/site_settings.json?

Ah. That could be useful! I have a handful of such stuff that will be part of my lc-panel that installs and manages discourse installations. Maybe soon I’ll get it up. . .

3 Likes

Nope, you will need to log into the box and run a site_setting:export > settings.yml

and it will look like this:

---
blake_key: awesome
default_locale: en
title: Blake's Site
site_description: Blake's test site
contact_email: ''
contact_url: ''
notification_email: noreply@unconfigured.discourse.org
site_contact_username: 'blake'
...

I decided to go with a single line and just do key/value for each setting, rather than including all the other info like last updated, created date, type, etc because I don’t think we really need those right now.

Anyways you could probably write a simple ruby script that parses your json file to the same format as this yaml file.

Also if there was a way to make all site settings read only from the UI, I think this would lead to a cool feature where all site settings could be controlled via git, and if someone wanted to make a change they would have to make a pr. Or a script could be run everyday setting the site’s setting back in case there were any changes. But these are probably all imaginary problems no one actually has.

6 Likes

In the admin settings on the website there’s a setting to “only show overridden”, it would be great if there was a similar option to export only the changed settings to a file!

In the meantime I guess I can export a copy of the settings from a blank test deployment and do a simple diff on the exported files.

I’m trying to document how our internal forum is set up and since there’s over 600 settings I’m going to have to trim the list a bit in the docs :wink:

8 Likes

Very reasonable feedback!

I changed it so the rake task only exports settings that deviated from default.

https://github.com/discourse/discourse/commit/550e108a8c26d30a0867be8b112bf928734fb65f

13 Likes

Hello, I am trying to use this rake task to import site parameters between two environments but it does not seems to work.

I have a correct settings.yml file (same format as mentioned by @blake) = the export works but when I am trying to import it, it just do not do a thing :frowning:

starting import...


Results:
 Updated:   0
 Not Found: 0
 Errors:    0

settings.yml starts with --- on the first line and contain 40 lines with key/values.

Have you any idea why it could not work ?

Thanks

It will only update items that are different. Could you manually verify a site setting in your yaml file and see if it doesn’t match with something on your new site?

1 Like

Thanks for replying, all settings are different because I am trying to import the settings from an existing discourse on a new “blank” discourse.

Edit : I have just dumped and restored the site_settings table manually to get this sorted.

2 Likes

Sorry for this being a bit “off topic”: I tried the latest discourse-settings-uploader from https://github.com/pfaffman/discourse-settings-uploader and get ruby errors. I am not very familiar with ruby and hope that this is only a small/obvious error.

root@develd:/var/discourse# ./discourse-settings-uploader forum2.netzwissen.de [api_key] /var/discourse/site_settings.json
Traceback (most recent call last):
        2: from ./discourse-settings-uploader:3:in `<main>'
        1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- rest-client (LoadError) 

The OS is ubuntu 18.04 lts, ruby is “ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]”. I do use this import script outside the container, is this correct?

Thnx, Thommie

You need to do a

gem install rest-client
1 Like

Hm, not really successful:

root@develd:/var/discourse# gem install rest-client
Fetching: http-accept-1.7.0.gem (100%)
Successfully installed http-accept-1.7.0
Fetching: unf_ext-0.0.7.6.gem (100%)
Building native extensions. This could take a while...
ERROR:  Error installing rest-client:
        ERROR: Failed to build gem native extension.

    current directory: /var/lib/gems/2.5.0/gems/unf_ext-0.0.7.6/ext/unf_ext
/usr/bin/ruby2.5 -r ./siteconf20200104-16857-11g44cb.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Something is missing, so that the extension can not be build, I assume ??

Installed ruby packages:

root@develd:/var/discourse# dpkg --get-selections | grep ruby
libruby2.5:amd64                                install
ruby                                            install
ruby-did-you-mean                               install
ruby-minitest                                   install
ruby-net-telnet                                 install
ruby-power-assert                               install
ruby-test-unit                                  install
ruby2.5                                         install
rubygems-integration                            install
1 Like

Hmm. Yeah. You need some dev package. What I would do isGoogle

   can't find header files for ruby

… very simple solution, build environment was missing, solved with

apt install build-essential

2 Likes