الانتقال من حاوية مستقلة إلى حاويات ويب وبيانات منفصلة

ويكي المجتمع Sysadmins
, , ,
48

Hi Jay @pfaffman Thanks for this post and others on this “two container” topic, including Sam’s writings on this as well.

Question:

We have been trying to set up two containers as you mention, one container for data and one for web-only and have been running into a number snags getting this running on macos.

But before we worry about debugging this “two container config” on the mac or Ubuntu, we would like to make sure we are doing this for the right reason.

The reason we want to do the “two container dance” is so the site will not go down when we rebuild the web app, for example when installing a plugin. Also, when we tweak a homegrown plugin; we noticed that sometimes the only way to insure our changes work is to rebuild (that is a story for another day) I’ve also been struggling getting a “fast and friendly” web dev setup going to my satisfaction as a well; but that is another topic for another day.

So, my question is that does the “two container” setup significantly minimize down time when the web-only part of the app is rebuilt?

That’s the right way to think of this, isn’t that right?

When we install a plugin or tweak one, we need to rebuild only the “web-only” yml file and not the data yml ?

We come from a LAMP forum background so changes to plugins can and are mostly done in runtime on the live site (with no down time unless we fat finger something). Also, we hail from some VueJS web apps where we build on the desktop and then we just upload and move the new app into place and there is virtually no down time upgrade / updating a VueJS part of the site. However, with Discourse we get downtime, which we do not want (even a few seconds).

Does the “two container” solution show significant improvements in downtime when we either (1) rebuild the app (for plugins, tweak code, etc) or (2) restore from a full backup?

I feel like I’m going to get “beat up” (again) for asking this question because we are looking for a way to run Discourse in production and make changes with near zero downtime, and we have not yet found a way to do things which are so easy to do with a LAMP or VueJS app (for example).

Hence, the struggle / interest in the “two container” method which we have yet to get up and running.

Thanks!

49

Yes. The existing web container continues to run while the new container is being built. The downtime, then, is just the time that it takes to spin up the new web server, which is typically under a minute, though by no means a zero-downtime proposition. If you want zero downtime you need a reverse proxy in front that will allow the new container to crank up and start working before you shut down the old one. (And if the database migrations for the new container break things for the old one then you get downtime there unless you go through some other machinations).

No difference on restore from backup.

4 إعجابات
50

Thank you Jay @pfaffman,

You are really a top-shelf valuable resource here, without a doubt!

What do you think about this maybe crazy idea (based on my still limited understanding):

Set up nginx as a reverse proxy on the front end; per this tutorial:

Then have two directories / instances with discourse_docker (standalone) set up, for example:

  1. /var/discourse1
  2. /var/discourse2

In both of these instances set up discourse_docker (standalone) to listen on a different socket, modifying this template in each instance:

 - "templates/web.socketed.template.yml"

So, in a nutshell, we have simply rebuilt production (at some quiet time) to run in a different container on listening on different socket (nginx.https.sock2), so there is no socket conflict; which we can build in standalone mode as well (with the goal of eliminating the need need for two containers, data and web-only).

For example (for discussion / illustration), in web.socketed.template.yml in discourse1:

  - replace:
     filename: "/etc/nginx/conf.d/discourse.conf"
     from: /listen 80;/
     to: |
       listen unix:/shared/nginx.http.sock;
       set_real_ip_from unix:;
  - replace:
     filename: "/etc/nginx/conf.d/discourse.conf"
     from: /listen 443 ssl http2;/
     to: |
       listen unix:/shared/nginx.https.sock ssl http2;
       set_real_ip_from unix:;

and in discourse2:

 - replace:
     filename: "/etc/nginx/conf.d/discourse.conf"
     from: /listen 80;/
     to: |
       listen unix:/shared/nginx.http.sock2;
       set_real_ip_from unix:;
  - replace:
     filename: "/etc/nginx/conf.d/discourse.conf"
     from: /listen 443 ssl http2;/
     to: |
       listen unix:/shared/nginx.https.sock2 ssl http2;
       set_real_ip_from unix:;

However, instead of having the discourse template do the magic, we simply manually
switch sockets in /etc/nginx/conf.d/discourse.conf and restart nginx, so we would remove the replace: directive in the web.socketed.template.yml template.

In this proposed (maybe crazy idea) configuration, we can have two standalone containers listening on two different sockets (not in conflict) and simply configure ngnix to connect to the socket we wish to connect to and restart nginx.

This seems clear, easy and perhaps useful (during a slow period of zero new posts in the live instance) for those who might not want (or need) the complexity two containers (data and web-only) per a single discourse instance (app)

Of course, the most robust configuration (from a data perspective) however, for perfection for busy sites would be the “two container” solution because we would want the data and the web-only instance (now listening on two different sockets, sock and sock2.

In the “two container solution” with the nginx front-end, the “standard configuration” is to have both web-only containers listen on the same socket, so both cannot run at the same time; but if (for example only) we had them listen on a different socket, they could both run at the same time and we could just use the nginx config file (and a nginx restart) to switch between the two.

Is this the right understanding?

Am I starting to (slowly but hopefully surely) understand this?

Thanks!

Followup Note Only: I have the “two container” config working on one of my desktop macs:

Screen Shot 2020-04-11 at 12.41.24 PM
Screen Shot 2020-04-11 at 12.41.24 PM1768×81 42 KB

The only caveats in our install was the need to manually create these directories (and set ownership and perms) as these directories get not get created for some reason by the scripts:

~discourse/discourse/shared/data
~discourse/discourse/shared/web-only

and of course, at first I tried with a blank password for the database, and that did not work (the instructions do say to set a password, but I was just experimenting).

Next, will set up the nginx frontend and try the move to that configuration with websocket for the web-only app.

51

That’s a lot to take in. But there’s no reason to have two discourse directories. Just create multiple yml files in the containers directory. Name them whatever you want.

إعجابَين (2)
52

Thanks for confirming. That is exactly how we are set up (single directory) after experimenting today.

All has gone well on the 2 containers config (2CC) but struggling with the nginx reverse proxy setup on macos.

Cannot get a working connection to the unix domain socket in the /shared directory even though the socket is accessible outside the container. Tried with nginx and also python and socat (testing). Always a 61 connection refused error., hmmmm

Been stuck on “connection refused” all day!!

Tomorrow is another day.

53

I had a small question.
If we had just one container setup (just 'app.yml). And we commanded ./launcher bootstrap app.
Then would our website/front-end stop or not?

If yes, then why doesn’t ‘bootstrap web-only’ stop our website?

If no, then what is the benefit got in two-container setup, in regards to time-gain while rebuilding our container? In other words, if we can keep our website running, even when we’re bootstrapping our sole-container, then why would we need to have two separate containers?

إعجاب واحد (1)
54

Not if you create a new .yml file, call it, for example, new_image

Bootstrapping does not start or stop any service, when configured correctly. Bootstrapped images are not running. That is why they are called “bootstrapped”.

However, you need to create a new yml file because you need to create a new image with a new image name.

So, with a new image name, the new bootstrapped image is not yet running. It’s is only “bootstrapped”.

You can build the new image with a new name (not app) and the building part is done. Let’s call this “new_image”.

Then, if we wanted to replace a running image, let’s call this “old_image”, you can do this:

./launcher stop old_image; ./launcher start new_image

In your case:

./launcher bootstrap new_image          #data image and web "app" image running
./launcher stop app; ./launcher start new_image

Because the data container is already built, you save time (1) not building the data image and (2) no down time when rebuilding the web image because you can build this (bootstrap the image) while the other images are running.

This is much faster; but it is not as fast as using a reverse proxy in front of the containers.

In your original message question, you have an image running called app. If you try to bootstrap that image called app again, then you are rebuilding the same image (name). This is not going to save you any time, as your instincts told you.

Is it clear yet?

If not, please ask. Everyone can learn; and learning is exciting. This is (actually) easy to understand but it does take a bit of time if you are new to the concepts.

إعجابَين (2)
55

To tell the truth, I didn’t understand anything. I tried. But failed.

My question was simple.
If we have 2 container setup, and we ‘bootstrap’ (not rebuild) our ‘web-only’ container, then our current web container keep working (because our website shows as working/ok).

And if we have single container setup, like ‘app.yml’, and then if we bootstrap this ‘app’ container, would our website still keep working?

And, if the explanation is simple, then why so?

إعجاب واحد (1)
56

Maybe you should hire someone to help you?

إعجاب واحد (1)
57

No problem.

It was just a small doubt. Whose one part, perhaps could be answered as ‘yes/no’ also.

Nothing big is there.

إعجاب واحد (1)
58

My suggestion is to just enjoying trying it on your own.

It actually takes less time to try it on your own in your own testbed than to ask a question and wait for an answer in a forum.

You will get the answer to your question if you try it; and so you should try these things in a test scenario so you will not break your production app :slight_smile:

Discourse is open source and free to download and configure, thanks to the generosity of the co-founders. This means you can and should take advantage of this open source and freely create, destroy, recreate, and destroy test apps (as my times as you like).

If you are not willing to put some effort into basic sysadmin task, @codinghorror had a great suggestion about hiring the local talent here.

إعجاب واحد (1)
59

Yes. (unless the bootstrap migrates the database in such a way that the running container can no longer use it). You do have down time while the old container so shut down and the new one is booting up.

No. You can’t have two database processes accessing the same files at the same time.

3 إعجابات
60

Oh!!
Thanks for clearing this thing.

إعجاب واحد (1)
61

So, by having the database not in the container you’re building, you can build a new web container that acts on the database while the other web container continues to work.

إعجاب واحد (1)
62

استفسار سريع بخصوص هذا النهج: كيف يمكن للمرء المضي قدمًا في عمليات إعادة البناء (Rebuilds) مع هذا الإعداد؟

بافتراض أننا سننتقل إلى إعداد الحاويتين من الصفر الذي أضافه @pfaffman، فإننا سنحصل على “app”: “data” و “web_only”.

جميع عمليات إعادة البناء وما إلى ذلك موجهة إلى حاوية “web_only”، لكن هل نقوم بأي شيء مع حاوية “data” أم أن عملية التمهيد (bootstrap) في “web_only” تتولى ذلك؟ (أسأل فقط لأنني أقوم ببعض الاختبارات وحاوية البيانات لا تتوقف عن العمل في أي لحظة).

63

نحن نستخدم “ثلاثة حاويات” ووكيل عكسي nginx:

  1. data
  2. socket1
  3. socket2

لنفترض أننا ندير حاليًا socket1 (ما تسميه ‘ويب فقط’) ونريد إعادة البناء واختبار شيء ما. سنقوم بإعادة بناء socket2 بينما يعمل socket1. لأن كلًا من هذه الحاويات يستخدم منفذ نطاق يونكس، يمكن تشغيلها في نفس الوقت لأن المنافذ المشتركة موجودة في ملفات مختلفة (مواقع مختلفة).

ثم، لنفترض أن socket2 تم بناؤه وجاهز للعمل.

أقوم بالآتي:

ln -sf /var/discourse/shared/socket2/nginx.http.sock /var/run/nginx.http.sock

الآن نعمل مباشرة على socket2.

إذا، على سبيل المثال، كانت هناك مشكلة، يمكنني ببساطة فعل الآتي:

ln -sf /var/discourse/shared/socket1/nginx.http.sock /var/run/nginx.http.sock

ونعود إلى حيث كنا من قبل، نعمل على socket1.

للمتعة، أضفت بعض الكود إلى ملف تعريف الدخول في bash، لذا عندما أبدأ جلسة الدخول إلى الخادم، يخبرني دائمًا بأي منفذ (حاوية) يعمل:

Last login: Fri May 15 09:39:39 2020 from 159.192.33.138
srw-rw---- 1 root docker  0 May  5 07:38 /var/run/docker.sock
lrwxrwxrwx 1 root root   44 May 15 09:16 /var/run/nginx.http.sock -> /var/discourse/shared/socket/nginx.http.sock

أتمنى أن يكون هذا مفيدًا.

في رأيي، هذا (بشكل عام) هو “الطريقة الصحيحة” (وهو “المعيار الافتراضي” الخاص بي، في الإنتاج وليس في بيئة الاختبار أو التطوير)، لكن هذا رأيي فقط، وقد يختلف الأمر من شخص لآخر. يتطلب ذلك بعض العمل الإضافي للإعداد، لكنني أعتقد أنه يستحق العناء (في بيئة الإنتاج).

تنبيه:

من الجيد الاحتفاظ بنسخ من ملفات القالب الأصلية yml في مكان آمن، لأن القوالب قد تُستبدل وقد يكون ذلك مشكلة، لذا نميل إلى “حفظ جميع ملفات yml”، “السحب أولاً” ثم “فحص ملفات yml” ثم التمهيد (حذرًا من أي خطر).

إعجابَين (2)
64

لا تحتاج إلى إعادة بناء حاوية البيانات إلا في حال وجود ترقية لـ PostgreSQL (كما حدث للتو) أو لـ Redis (التي حدثت منذ حوالي 6 أشهر).

في الغالب، ما عليك سوى استبدال web_only بـ app في التعليمات التي تراها. لدي ملاحظات على Managing a Two-Container Installation - Documentation - Literate Computing Support

5 إعجابات
65

إذن، في هذا النهج، لديك نسختان من الويب وتستخدم النسخة الخاملة لاختبار الأشياء بنفس البيانات؟ هذا مثير للاهتمام، وسأجربه بالتأكيد بمجرد أن أستقر على استراتيجية “محاولة توفير المساحة” هذه :stuck_out_tongue:

شكرًا جزيلاً لك. قرأت مقالتك من البداية إلى النهاية، لكن لدي سؤال صغير فقط: هل تقصد هنا “البيانات”، صحيح؟ (أي، إعادة بناء البيانات بدلاً من web_only) كما هو مذكور في المقالة؟ أم ربما هناك حيلة مختلفة أفتقدها (أي: عند وجود ترقية كبيرة، تحتاج إلى دمجها ثم فصلها مرة أخرى)؟

66

نعم، نقوم بالاختبار والإضافة وإعادة البناء في حاوية واحدة تعتمد على المنافذ (socket-based) بينما تعمل الأخرى.

نعم، كلاهما يستخدم حاوية البيانات نفسها. ولا “تهتم” حاوية البيانات بأي تطبيق ويب “تتحدث” إليه.

الأمر بسيط جدًا حقًا، بمجرد أن تفهم الفكرة الأساسية حول كيفية تشغيل الحاويات على منافذ يونكس المشتركة (unix domain sockets) وليس على منافذ TCP/IP المكشوفة خارجيًا.

لا نعتقد أن الأمر يستغرق مساحة كبيرة، ولكن مرة أخرى، نحن لا نعمل (في الإنتاج) بمساحة محدودة لأن مساحة القرص ليست باهظة الثمن.

إعجاب واحد (1)
67

شكرًا على التفاصيل. جربت للتو وجود “حاويتين للتطبيق” تشيران إلى “Data One” في بيئة اختبارية، ويعمل الأمر بسلاسة تامة.

لكن لا يمكنني نقله إلى بيئة الإنتاج (PRD)، لأنني لا أستطيع إعادة بناء حاوية البيانات في بيئة الإنتاج، ولا أريد تغيير أي شيء دون حل هذه المشكلة. الأمر يتسبب فقط في إيقاف العمليات برسالة خطأ: discourse@discourse FATAL: terminating connection due to administrator command

Youtube embeds missing