Download and restore of site settings

For sites interested in moving to Discourse from another forum or trying out Discourse before an import, it’s tedious to see that settings get moved from a test site to the live one, or that settings done on a live instance can be easily moved to a restore from an import that started with a clean slate.

Now that themes are supported, part of this problem is solved, but it’s still tedious to copy potentially dozens of settings from one site to another.

I started looking at a clean (yet naive) way to do this and started by looking at what data explorer returns for select * from site_settings. That was a good start, but if I reset a value via the web interface, the value still shows up in data explorer. Is it different in rails?

Edit: Another example of my being crazy or looking at the wrong tab. I see now that the setting I changed is still there (of course) but is now blank (as I’d expect). I’d still welcome input on how to best go about this. A plugin that spits out JSON and sucks in an upload?

Any hints on how to go about this?

Now you can, if you can run a ruby script.

https://github.com/pfaffman/discourse-settings-uploader

To create/retrieve settings

Create a JSON settings dump by visiting https://discourse.example.com/admin/site_settings.json and saving the file somewhere convenient.

Retrieve your site’s API key from https://discourse.example.com/admin/api/keys

To install these settings to another site

Do this:

./discourse-settings-uploader HOSTNAME API_KEY API_USER SETTINGS_FILE

For example:

./discourse-settings-uploader discourse.example.com d35989078a system site_settings.json

Warning

This script does minimal error checking and could destroy your site, cause fires, or hurt people.

You should make a backup of your site before you use this script.

We should pull this into a rake task, cause I want core to ship with it.

I’d be flattered.

Would it restore the settings to the local instance, or, as my script does, push them to a remote instance via the API?

On a related note, a rake task to call create_master_key to make it easier to get the API key from a script would be useful.

I’ve tried

echo "ApiKey.create_master_key" | docker exec -w /var/www/discourse -i app bundle exec rails c

but it fails with FATAL: Peer authentication failed for user "discourse" (PG::ConnectionBad).

I would just use straight json here:

rake site_settings:export > settings.yml
rake site_settings:import < settings.yml

or something like that.

@blake is also interested so I think he can probably help.

A rake task for this now exists!

https://github.com/discourse/discourse/commit/925d1a78697dfb8e9549e0b9ba4f7e7f18235cfe

Here is an example of an import. It checks for bad settings (NOT FOUND), should show any errors (setting a username that doesn’t exist), and show you which settings changed, and will only update a value if the new setting is different from the current value.

$ rake site_settings:import < ~/tmp/settings.yml

starting import...

NOT FOUND: existing site setting not found for blake_key
Changed title FROM: Discourse TO: Blake's Site
Changed site_description FROM:  TO: Blake's test site
ERROR: site_contact_username: There's no user with that username.
Changed invite_only FROM: f TO: true
Changed login_required FROM: f TO: true

Results:
 Updated:   4
 Not Found: 1
 Errors:    1

Afterwards you can also check the Staff Action logs to verify the changes.

@pfaffman I do still really like your api version, and I would be the perfect tool to add to the discourse CLI app that I started working on a couple of years go that I haven’t had time for but would still like to exist.

Cool. Can’t wait to use it.

Will it work on a file downloaded from /admin/site_settings.json?

Ah. That could be useful! I have a handful of such stuff that will be part of my lc-panel that installs and manages discourse installations. Maybe soon I’ll get it up. . .

Nope, you will need to log into the box and run a site_setting:export > settings.yml

and it will look like this:

---
blake_key: awesome
default_locale: en
title: Blake's Site
site_description: Blake's test site
contact_email: ''
contact_url: ''
notification_email: noreply@unconfigured.discourse.org
site_contact_username: 'blake'
...

I decided to go with a single line and just do key/value for each setting, rather than including all the other info like last updated, created date, type, etc because I don’t think we really need those right now.

Anyways you could probably write a simple ruby script that parses your json file to the same format as this yaml file.

Also if there was a way to make all site settings read only from the UI, I think this would lead to a cool feature where all site settings could be controlled via git, and if someone wanted to make a change they would have to make a pr. Or a script could be run everyday setting the site’s setting back in case there were any changes. But these are probably all imaginary problems no one actually has.

In the admin settings on the website there’s a setting to “only show overridden”, it would be great if there was a similar option to export only the changed settings to a file!

In the meantime I guess I can export a copy of the settings from a blank test deployment and do a simple diff on the exported files.

I’m trying to document how our internal forum is set up and since there’s over 600 settings I’m going to have to trim the list a bit in the docs :wink:

Very reasonable feedback!

I changed it so the rake task only exports settings that deviated from default.

https://github.com/discourse/discourse/commit/550e108a8c26d30a0867be8b112bf928734fb65f

안녕하세요, 이 rake 작업을 사용해 두 환경 간 사이트 매개변수를 가져오려 하고 있지만 작동하지 않습니다.

settings.yml 파일은 올바르게 작성되어 있습니다 (@blake님이 언급한 형식과 동일) = 내보내기는 작동하지만 가져오기를 시도하면 아무 일도 일어나지 않습니다 :frowning:

starting import...


Results:
 Updated:   0
 Not Found: 0
 Errors:    0

settings.yml은 첫 번째 줄에 ---로 시작하며 키/값이 포함된 40줄로 구성되어 있습니다.

왜 작동하지 않는지 혹시 아이디어가 있으신가요?

감사합니다

다르고 변경된 항목만 업데이트됩니다. yaml 파일에서 사이트 설정을 수동으로 확인해 보시고, 새 사이트의 설정과 일치하지 않는 부분이 있는지 확인해 주시겠어요?

답변해 주셔서 감사합니다. 기존 Discourse의 설정을 새 “빈” Discourse에 가져오려고 시도하고 있기 때문에 모든 설정이 서로 다릅니다.

수정: 사이트 설정 테이블(site_settings)을 수동으로 덤프하고 복원하여 이 문제를 해결했습니다.

조금 "오프 토픽"인 점 죄송합니다: GitHub - pfaffman/discourse-settings-uploader: Install a Discourse settings JSON file to a site · GitHub 의 최신 discourse-settings-uploader를 시도해 보았는데 ruby 오류가 발생합니다. ruby에 대해 잘 모르고, 이것이 단순히 작거나 명백한 오류일 것이라 희망합니다.

root@develd:/var/discourse# ./discourse-settings-uploader forum2.netzwissen.de [api_key] /var/discourse/site_settings.json
Traceback (most recent call last):
        2: from ./discourse-settings-uploader:3:in `<main>'
        1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- rest-client (LoadError) 

운영체제는 ubuntu 18.04 lts이고, ruby 버전은 "ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]"입니다. 이 가져오기 스크립트를 컨테이너 외부에서 사용하고 있는데, 이것이 올바른가요?

감사합니다, Thommie

다음 명령을 실행해야 합니다.

gem install rest-client

음, 별로 성공적이지 않네요:

root@develd:/var/discourse# gem install rest-client
Fetching: http-accept-1.7.0.gem (100%)
Successfully installed http-accept-1.7.0
Fetching: unf_ext-0.0.7.6.gem (100%)
Building native extensions. This could take a while...
ERROR:  Error installing rest-client:
        ERROR: Failed to build gem native extension.

    current directory: /var/lib/gems/2.5.0/gems/unf_ext-0.0.7.6/ext/unf_ext
/usr/bin/ruby2.5 -r ./siteconf20200104-16857-11g44cb.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

확장 기능을 빌드할 수 없는 이유는 뭔가 누락되어 있기 때문이라고 추측합니다.

설치된 ruby 패키지:

root@develd:/var/discourse# dpkg --get-selections | grep ruby
libruby2.5:amd64                                install
ruby                                            install
ruby-did-you-mean                               install
ruby-minitest                                   install
ruby-net-telnet                                 install
ruby-power-assert                               install
ruby-test-unit                                  install
ruby2.5                                         install
rubygems-integration                            install

음. 그래. 개발용 패키지가 필요합니다. 제가 하겠다면 구글에서

   ruby 헤더 파일을 찾을 수 없습니다

… 아주 간단한 해결책이었습니다. 빌드 환경이 누락되어 있었고, 다음으로 해결했습니다.

apt install build-essential