Creating a duplicate of production environment

I think you may want to run a remap domain in there too. That way all of the links do not go out to Production.

4 Likes

Hi Andrew,

I originally thought that was the correct way, but I could not get past step 1. My installation of discourse fails and unfortunately, nobody seems to be able to assist with that.

https://meta.discourse.org/t/failure-when-trying-to-install-older-version-of-discourse/61722

I was hoping it’s the wrong process and there is a different way to do it.

Thank you for the information.

Regards
Henti

I think you could do it by changing git head, but I wasn’t smart enough for that. In my case I was close enough to the live site that the restore ran.

Have you seen this:

That is the process I tried.

I’ll continue trying to get that working today. Thanks again for all the assistance.

Resolved the problem.

I had the failed data in /var/discourse that I didn’t know I needed to clean up.

2 Likes

Just to close the loop. Steps I followed.

On Production:

Get latest backup from /var/discourse/shared/standalone/backups/default

On New/Dev instance :

mkdir /srv/discourse
git clone https://github.com/discourse/discourse_docker.git /srv/discourse
cd /srv/discourse
# If you need a specific commit, do:
# git reset --hard git_commit_hash_goes_here

Copy the app.yml from your production server to new/dev instance. Update app.yml

new_hostname=$(hostname -f)
sudo sed -i "s/^  DISCOURSE_HOSTNAME:.*$/  DISCOURSE_HOSTNAME: $new_hostname/g" containers/app.yml
# If you need to duplicate a specific version of discourse, use the git commit hash, otherwise use the branch name. Below we are using stable branch
sudo sed -i 's/^  #version: .*$/  version: stable/g' containers/app.yml
sudo sed -i "s/^  DISCOURSE_DEVELOPER_EMAILS: .*$/  DISCOURSE_DEVELOPER_EMAILS: 'sysadmins@yourcompany.com'/g" containers/app.yml

Bootstrap, start the app container

sudo ./launcher bootstrap app
sudo ./launcher start app

Copy your backup to restore location

sudo mkdir /var/discourse/shared/standalone/backups/default -p
sudo cp /tmp/discourse-backup/lgtm* /var/discourse/shared/standalone/backups/default

Now we run commands in the container to prepare, and restore the backup, and fix the permissions.

backup=$(basename /var/discourse/shared/standalone/backups/default/*)
sudo /usr/bin/docker exec -i app discourse restore $backup
sudo /usr/bin/docker exec -i app chown discourse.www-data /var/www/discourse/public/backups/default/ -R 

If you get an error about SiteSettings not allowing restore. (I think it’s older version of discourse that needs this)

sudo /usr/bin/docker exec -i app rails runner "SiteSetting.allow_restore = true"

I tried using the Advanced, manual method of manually creating and restoring Discourse backups method for my restore, but it failed with rake db:migrate creating an empty db. Might work for you, I did not test further.

5 Likes

Slight update.

Replacing the DISCOURSE_HOSTNAME with $(hostname) bit me in the arse … I didn’t realise it’s used for url generation, so make sure it’s the url external users will be seeing if you use proxy/load balancers.

H

1 Like

I went about this in a different, although slightly uglier way - I cloned my entire droplet on DigitalOcean, then logged into the droplet and disabled all of the ssl/https settings in the app.yml. This at least gave me a clone of my community content to screw with settings on.

I have no elegant way to migrate changes from my test server to the live server except by manually repeating, so this is still total amateur hour, but it at least let’s me explore without breaking the live server.

Took your approach. Thanks. A quick question: I’ve backed up a production instance and restored it to a local Mac instance for dev, but resolving to localhost:3000 is incomplete. e.g. My logo sits at http://localhost:3000/uploads but Discourse is looking in http://localhost/uploads. I see talk of editing my app.yml, but don’t see one inside my discourse directory. Is there something I’m missing here?

Try running the below in a rails console (probably RAILS_ENV=development bundle exec rails c or bundle exec rails c)

SiteSetting.port = "3000"

It’s a hidden site setting if I remember correctly which is why it doesn’t show up in the settings panel.

1 Like

That worked. Thanks.

I’ve set everything up with a local Rails install vs a local Docker container. Looking at so many examples of local Docker containers I’m wondering, did I make the wrong choice? Does it matter at this point, or will I eventually wish I’m in a Docker container to better mirror my DigitalOcean install?

1 Like

Personally I’d go for the development environment but create a VM with a production style install (if you have the resources to do so).

That way you can test out your changes and work in a development environment. You can then use the production style container to test out your changes in a prod like environment (sometimes the local Rails install suppresses some errors or causes strange behaviour) and get the best of both worlds.

Thanks for that feedback. Makes sense.

Are all options for hosting a Docker container paid options, regardless of traffic? I’m used to using Heroku to stage Rails sites at no cost, but the Docker containers are new to me.

Looks as if Heroku supports Docker, but seems the Community recommends Digital Ocean, which I assume does not have a free tier.

Theoretically if you don’t mind having a private dev instance (i.e. it doesn’t allow access from the internet but can be accessed from the network your PC lives at) just run a VM using virtualbox or somthing similar with a bridged IP. Install ubuntu, set up remote root login and you should be OK to follow the production install guide.

The only thing you’d need to pay for is the internet connection that you probably already have and the power for the computer used. There are no server costs related to running your own Ubuntu Server in a VM that you control on your PC.

2 Likes

Now I’m wondering if we should just add a Droplet to our DigitalOcean account for a staging instance at $5 / month. Is the cleanest, simplest approach just a matter of spinning up another Droplet inside the same DigitalOcean account which can be accessed at staging.ourdomain.org? I see posts about running two instances inside a single Droplet. Is there any reason to do that other than cost?

This is a pretty important post, but could we also get some input and directions for processes that exclude a working backup system? I’m finding that the integrated backup doesn’t appear to have any way to function when you use an external database server. In my particular use case I’d like to take a production Discourse installation with a separated database server and then deploy it to a staging/test server where the database is INTEGRATED into the Dockerized setup.

I’ll post my findings once I get this working and what steps I took, but I figured this is a pretty important consideration for this topic.

I don’t think that your issue belongs here particularly when it is an unlikely scenario for almost every developer, .i.e. most developers will not need to duplicate any production environment beyond an officially supported install which is the gold-standard for reproducing bugs.

Also, you’re effectively spamming this forum with your problem because you will be duplicating the solution to your issue at Pg_dump backup failure for remote pgsql - port & version differentials; what options exist?

3 Likes

@RemahI didn’t say development - I said “staging/testing”. I’m talking about a staging environment to test things such as an importation of messages from another platform, etc.

No, I am not spamming the forum. This is a different thought process altogether. What you mention is regarding a backup problems or flaw that exists in the system and the standard “backup/restore” path that some might consider as an option to this path, however it is not the only path to follow and therefore I’m exploring another option of setting up such a staging environment presently by going down an altogether different path.

My comment was pointed: I said “developer” because you have “developed” a non-standard Discourse install.

We know that such configurations generate a lot of issues that are not relevant to the majority of instances using the officially supported install method.

I referred to the topic you created because it is a case in point.

Remember this topic. It is not a discussion of variations from the norm but a request for standard steps:

Your instance is a long way from such standard set of steps:

This topic becomes muddied the more unsupported configurations that are included here. The official install has a population of hundreds or thousands where advice will generally be applicable to all. Whereas non-standard installs often have a population/sample size of one so it is far better to put them in separate topics because some advice will be misleading for the majority.

2 Likes

I apologise if I’ve come across too strongly. :slightly_smiling_face:

I’m not trying to dissuade you from posting what is likely to be very useful information for some sites. Your contribution will be more useful and more easily referenced with its own title in its own topic.