I am currently in the process of migrating our server from an existing Bitnami based deployment (quite an old version) to the officially supported install in AWS using the latest Discourse release. This new install has an external Elasticache and RDS instance and also will use S3 for backups and uploads.
I have 2 questions:
The old Discourse server is on a pretty old version - by doing the backup/restore in the new Discourse server will that just run all the relevant upgrade commands?
If I copy the backup file into the new Discourse docker container on the new server and initiate the restore on the command line, will the new bucket values I set in my config be overwritten as part of the restore or will my new values be used? I assume my new DB is populated from the backup and then when I exist the container and run ./launcher rebuild app my new S3 values will be used.
If my assumptions are correct, then before I perform the restore I assume I should copy the contents of the old S3 buckets to the new ones?
Are there any other gotchas for doing this type of migration with a backup?
Maybe I am missing something but, why are you establishing new buckets? A new backup bucket with appropriate lifecycle rules seems ok to do but if your existing Discourse instance is using S3 upload bucket you will have some problems.
I am not 100% sure the migration from Bitnami will work without a hitch so we don’t want to impact any existing data incase we need to abort the migration
We changed our bucket naming so thought it easier to have 2 brand new buckets for this
Point 1 is the one that worries me so just being extra careful.
What issues do you envisage with the new uploads bucket? The old Discourse server will be put into read only mode. The plan is once the new server is up and validated we will stop our old server, change the DNS to the new server and then purge the cache in the CDN (Cloudfront) and then open it up to the public.
I missed that you were going to copy over the data from the old buckets. Took a look at AWS, looks to be straight forward. If it were me I would not muck with changing buckets before moving to AWS or a new server elsewhere.
Officially supported?? Not so sure about that.
Strongly suggest you get Discourse updated before the move to a new server.
On the old instance, suggest you move the S3 config to the app.yml if not there already before moving.
I am very curious about how doing this type of install is a value added proposition.
From the numerous posts around unsupported installs it seems more trouble than benefit unless folks are doing them for learning/experimentation.
Such a setup might be an unsupported install from a Discourse point of view, but from the perspective of an enterprise IT organisation, RDS and Elasticache might be standard and the Discourse standard install would be the odd one out.
As mentioned by @RGJ , our enterprise infra uses external services for things like cache, database etc hence the Elasticache and RDS. It means we can have full backup and redundancy for these services and also help with security controls. This is an official/supported installation from a Discourse point of view - its just using a different set of templates - we use discourse_docker/samples/web_only.yml at main · discourse/discourse_docker · GitHub (maybe using the word standard was a bit misleading, apologies).
So, sounds like we should first update our bucket names for the existing install and then perform the move to the new server. Updating the existing install to the latest version is a no go - we experienced issues with the Bitnami upgrade previously hence the move to the official installation method.
Can I ask though, what issues are likely to happen if we perform the restore with the existing buckets and then update the app.yml to reference the new buckets - don’t all of the DISCOURSE_ env vars take precedence over any config in the database (if applicable)? Or is there something else that could cause a problem?
Since if you do it prior to the migration and things go south, you’ll have the issue on the older Bitnami instance instead of your freshly installed standard instance. And, besides the old version, even the mention of the word Bitnami will make you get far, far less support here on meta.
One other question if I may - when I run the restore from the command line inside the Discourse container, does it use the existing /var/www/discourse/config/discourse.conf config for the details about the database connection, redis connection etc or does it overwrite that config from what is in the backup file?
discourse.conf is generated from app.yml, it is not included in the backup file.
Generally, what’s in discourse.conf overrides the site settings.
So if you have setting_foo in your database, you can override it by defining DISCOURSE_SETTING_FOO in your app.yml which will then generate setting_foo in discourse.conf.
Pointing a Discourse container at an external postgresql and redis server using our container image with our tooling is a supported install.
RDS[1] is Postgresql, and Elasticache is Redis, this is fine.
This should be fine we do this for people moving onto our hosting.
My preference is to keep the process as simple as possible, so if you can run a full backup (including uploads) on the old server and restore it to the new, that lets you fully test out the new setup without any impact on the old one.
I’ll do the backup/restore without changing the bucket names as suggested by @RGJ aswell. Like I said, my concern was to not impact the existing server data in any way but if I backup the existing buckets first and make sure the existing server is in Read Only mode during the migration I think the integrity of the upload data in the buckets is pretty well protected.
