Using Discourse with external PostgreSQL and Redis

[5hark]

There are multiple websites on my existing server, PostgreSQL (in Docker) and Redis (also in Docker). I would like to reuse existing Postgres and Redis with Discourse.

Here is my containers/app.yml from Discourse:

templates:
    #  - "templates/postgres.template.yml"
    #- "templates/redis.template.yml"
  - "templates/web.template.yml"
  ## Uncomment the next line to enable the IPv6 listener
  #- "templates/web.ipv6.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"
  - "templates/web.socketed.template.yml" # Added

extra_hosts:
    - "host.docker.internal:host-gateway"

expose:
    - "8881:80"   # http
    #  - "443:443" # https

env:
  DISCOURSE_DB_NAME: discourse
  DISCOURSE_DB_USERNAME: discourse
  DISCOURSE_DB_HOST: host.docker.internal
  DISCOURSE_DB_PASSWORD: XXXXXXXXXXX

  DISCOURSE_REDIS_HOST: host.docker.internal
  DISCOURSE_REDIS_PORT: 6379

Other websites run without docker, and they able to succesfully connect to Redis and PostgreSQL through localhost. Here is docker-compose file from postgres/redis:

version: "1.0"
services:
    postgres:
        container_name: postgres_db
        build: postgres_th
        # some settings were removed, to make file shorter for this example
        ports:
            - "5432:5432"
        restart: unless-stopped
        networks:
            - main_network

    redis:
        container_name: redis
        image: redis:7.2.1
        ports:
            - "6379:6379"
        healthcheck:
            test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
        restart: unless-stopped
        networks:
            - main_network

networks:
  main_network:
    external: true

When I am trying to build Discourse docker image, I am getting following error:

./launcher rebuild app

FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 358 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.

I found a solution here Configure Discourse to use a separate PostgreSQL server to use docker with host network.

./launcher rebuild app --docker-args --net=host --skip-mac-address
and setting DISCOURSE_DB_HOST to localhost

in this case everything works fine, the only problem remains that Discourse image is running without exposed port because it uses host network instead of bridge.

Is there any way to connect to external Postgres server somehow without setting Discourse docker to use host network? As an alternate solution, if I use host network, how do I connect external nginx (which runs on the same server but without Docker) to nginx inside Discourse docker?

[5hark]

I set specific hostnames to Postgres and Redis in order to have an ability to connect to them and it works within same docker network:

version: "1.0"
services:
    postgres:
        container_name: postgres_db
        ports:
            - "5432:5432"
        networks:
            - main_network
        hostname: postgres-main-docker
    redis:
        container_name: redis
        image: redis:7.2.1
        ports:
            - "6379:6379"
        healthcheck:
            test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
        restart: unless-stopped
        networks:
            - main_network
        hostname: redis-docker

$ docker exec -it 1705c01635f2 bash
root@postgres-main-docker:/# ping redis-docker
PING redis-docker (172.27.0.2) 56(84) bytes of data.
64 bytes from redis.main_network (172.27.0.2): icmp_seq=1 ttl=64 time=0.128 ms
64 bytes from redis.main_network (172.27.0.2): icmp_seq=2 ttl=64 time=0.073 ms

$ docker network inspect main_network
[
    {
        "Name": "main_network",
        "Id": "8eb1973cccb151bd92f6a3b89589cd7a3e428f3ef5dadc0b80a9bb3ed60278fb",
        "Created": "2023-09-22T12:21:17.935299029+03:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.27.0.0/16",
                    "Gateway": "172.27.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "0cfb64e494057bffe3ada9848d8094517ff3badf9212338bfac869ab022f6856": {
                "Name": "redis",
                "EndpointID": "0434f2475e0137f46f2d4c09ae70862612e3734e234d8511228c1cb2f22ece83",
                "MacAddress": "02:42:ac:1b:00:05",
                "IPv4Address": "172.27.0.2/16",
                "IPv6Address": ""
            },
            "93af234be3ae1ca27824e27ff4a90711d9a8184204ed1fb152863ea29f173c37": {
                "Name": "postgres_db",
                "EndpointID": "250d1f7ce883a62c5518d7bdac3ab837522acabd34c8922de0d014a410e20183",
                "MacAddress": "02:42:ac:1b:00:04",
                "IPv4Address": "172.27.0.4/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "main_network",
            "com.docker.compose.project": "XXXXXXXXXXXXXXX",
            "com.docker.compose.version": "2.20.2"
        }
    }
]

Then I tried to add Discourse to the same Docker network - main_network by adding to containers/app.yml the following:

networks:
  - main_network

DISCOURSE_DB_HOST: postgres-main-docker
DISCOURSE_REDIS_HOST: redis-docker

It did not worked. Any ideas why?

So I tried to start building image with:
./launcher rebuild app --docker-args '--network main_network' and it work, which is probably is a solution (but not very convenient since I have to specify network in command line instead of config file.