Migratie naar een zelf-gehoste oplossing vanuit Kubernetes

[stevejr (Steve)]

Hallo allemaal,

Ik draai Discourse al een tijdje in een Kubernetes-cluster met behulp van de niet-ondersteunde Bitnami-image. Nu Bitnami de image aan het afschaffen is en achter een betaalmuur gaat, ben ik van plan onze server te migreren naar een zelf-gehoste oplossing op AWS.

Ik heb een paar vragen waar de community me hopelijk mee kan helpen:

• We gebruiken al een externe Postgres-database voor de installatie en die blijft ook zo. We hebben echter enkele instellingen bijgewerkt via de UI en ook met omgevingsvariabelen die de Bitnami-installatiescripts mappen naar Discourse, bijvoorbeeld DISCOURSE_S3_BACKUP_BUCKET mapt naar S3_BACKUP_BUCKET.
  • Is het voldoende om de Discourse-instellingen in de vereiste yaml-bestanden in te stellen of moeten we nog steeds env-variabelen gebruiken?
  • Als we een back-up maken vanuit de UI, wat wordt er dan precies hersteld - wordt de database bijgewerkt?
  • Is het beter om een gloednieuwe database te maken met een schone installatie en dan een back-up/herstel uit te voeren?
  • Als de nieuwe installatie een latere versie van Discourse is, zal dat dan een probleem veroorzaken als er een herstel wordt uitgevoerd?
• De standaard installatiehandleiding gebruikt Docker - hoe monitor je de containers in een productieomgeving, aangezien de standaard installatie een enkele VM met Docker lijkt te zijn.
• Zijn er documenten die een migratie en eventuele valkuilen beschrijven?

Ik weet zeker dat ik meer vragen zal hebben naarmate de migratie vordert, maar alle adviezen/hulp die gegeven kan worden, zou enorm gewaardeerd worden.

Bedankt.

[pfaffman (Jay Pfaffman)]

@stevejr

We use an external Postgres DB already for the installation and so this will stay in place.

If it wasn’t already your plan, I’d move to a new database (on the same server) for the migration so you don’t break your existing site.

@stevejr

However, we updated some settings via the UI and also using environment variables that the Bitnami installation scripts maps to the Discourse, for example DISCOURSE_S3_BACKUP_BUCKET maps to S3_BACKUP_BUCKET.

I can’t tell quite what you think Bitnami is doing, but the thing you want in your ENV is DISCOURSE_S3_BACKUP_BUCKET. See Configure an S3 compatible object storage provider for uploads for how to properly set those S3 variables in your app.yml.

@stevejr

Is it better to create a brand new database with a fresh install and then do a backup/restore?

If by “better” you mean “will this mean that I won’t break our existing site and leave it in a state where it will never work again?”, the answer is yes.

@stevejr

Is it good enough to set the Discourse settings in the required yaml files or should we still use env vars?

Set them in the YML

@stevejr

If we do a backup from the UI what will that actually restore - does it update the database?

Yes it will update the database. I would recommend that you Restore a backup from the command line

@stevejr

If the new install is a later version of Discourse will that cause a problem if a restore is attempted?

That’s what you want. The place you restore from has to be the same or newer than the backup version. It’ll migrate the database after it’s restored.

This might be all you need to know and we’re happy to help here for free. If you’d like hands-on your-setup-specific attention, you can contact me or ask for help in Marketplace.

Also, another option would be to build images and launch those in k8s. I’ve done that a few times and used github to build the images.

[manning-ncsa (Andrew Manning)]

@pfaffman

@stevejr

Is it better to create a brand new database with a fresh install and then do a backup/restore?

If by “better” you mean “will this mean that I won’t break our existing site and leave it in a state where it will never work again?”, the answer is yes.

My experience concurs. I have seen so many strange little failures over the years that I always maintain full backups such that I can start from scratch and restore the site. Relying on fixing problems in situ will eventually fail you.

@stevejr

• The standard installation guide uses Docker - how do you monitor the containers in a production environment as it looks like the standard install is a single VM with Docker.

Like you, I was left in the lurch when Bitnami stopped offering free images and charts. I had to adapt and migrate so many deployments. One of them was my Discourse deployment. If it is useful to you, here is a link to the replacement Helm chart I created in a very short amount of time (meaning that it works but is far from an ideal design). It is an attempt to use the “official installation method” given that I have seen no “community standard” Helm chart emerge after all these years. (I suppose Bitnami’s chart was effectively that standard, because few of us predicted this abrupt change.) In any case, this new chart that I am running for one of my research communities is basically just a pod with two containers: the official Docker-in-Docker container and a custom container based on python:3, installing Docker and then using the official Discourse installation. Because all the components (Discourse server, Redis, PostgreSQL) run in the black box of the locally image built by the launcher script, there is no scalability or support for high-availability. I did manage to reduce the downtime due to the pod respawning on another node (e.g. when draining node for OS updates or a node crash) by using docker save to store the built image on the persistent volume, and then loading that if local_discourse/app:latest is not found.

But to answer your question, I do not know how to monitor anything in this new deployment. I am running “in production” but my community is small enough and the usage moderate enough that if the forum goes offline for a while, it is not a big deal. Even so, I am very close to abandoning self-hosting and migrating to a service like Communiteq or Discourse.org.