Docker sometimes fails to connect to github.com

hiddenseal · January 4, 2023, 1:40am

While trying to upgrade Discourse via ./launcher rebuild app I get an issue with

fatal: unable to access 'https://github.com/discourse/REPO.git/': Failed to connect to github.com port 443: Connection timed out

However, the problem is: sometimes this fails even for discourse. But if you try to do it again, immediately after, with no changes, it may go through, but fail on one of the plugins. If I remove all the plugins, the discourse connection may fail or it may go through and actually build the whole thing.

I’ve seen this

but I don’t know how to debug the container networking.

Ed_S · January 4, 2023, 3:36pm

You don’t have exactly the problem stated in the thread you linked, and indeed you seem to have an intermittent problem, which is always the hardest to diagnose. But maybe you could try a manual fetch. I tried this, which is harmless:

# cd /var/discourse/
# ./launcher enter app
x86_64 arch detected.
# wget -O - -v https://github.com/discourse/logster.git/ |wc
--2023-01-04 15:30:25--  https://github.com/discourse/logster.git/
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/discourse/logster/ [following]
--2023-01-04 15:30:25--  https://github.com/discourse/logster/
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘STDOUT’

-                                                  [ <=>                                                                                                 ] 231.22K  --.-KB/s    in 0.06s   

2023-01-04 15:30:26 (3.99 MB/s) - written to stdout [236767]

   2751   15860  236767
#

We see that “github.com” was translated to a reasonable IP address, was contacted, and returned some data. In fact the command run inside the container behaved very much like it does outside the container, and that’s as expected.

Are you using a supported installation recipe, on a well-known hosting company?

hiddenseal · January 4, 2023, 3:45pm

First try gave

Connecting to github.com (github.com)|140.82.121.4|:443... failed: Connection timed out.
Retrying.

Default installation method

depends on how you define well-known, but this is a fairly big player

Ed_S · January 4, 2023, 3:58pm

OK, so it is at least trying to contact a reasonable IP address. I’d suspect something wrong with routing, or firewall, or something wrong at the hosting company level.

If you rerun the test outside the container, you’ll see if you have a container-specific problem or more of a general linux problem.

hiddenseal · January 4, 2023, 4:02pm

It’s a container specific one. I’m struggling to understand what could have caused this as I haven’t modified the container before and the site is ~1.5 years old

Ed_S · January 4, 2023, 4:13pm

Hmm, so, outside the container, you can run that test reliably?

Perhaps you could share a sanitised version of your app.yml? (I have no ideas as to what might be going on, but maybe someone will spot something.)

How old is the present installation - what version and what ID is it at, according to your dashboard?

hiddenseal · January 4, 2023, 4:20pm

yep

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  - "templates/web.ssl.template.yml"
  - "templates/web.letsencrypt.ssl.template.yml"

expose:
  - "80:80"   # http
  - "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"
  db_shared_buffers: "128MB"
  #db_work_mem: "40MB"
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en
  UNICORN_WORKERS: 2
  DISCOURSE_HOSTNAME: domain.example.com

  #DOCKER_USE_HOSTNAME: true

  DISCOURSE_DEVELOPER_EMAILS: 'some@emails.com'

  DISCOURSE_SMTP_ADDRESS: smtp.domain.com
  DISCOURSE_SMTP_PORT: port
  DISCOURSE_SMTP_USER_NAME: email@email.com
  DISCOURSE_SMTP_PASSWORD: "password"
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  DISCOURSE_SMTP_DOMAIN: domain.com
  DISCOURSE_NOTIFICATION_EMAIL: email@email.com

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  LETSENCRYPT_ACCOUNT_EMAIL: email@email.com

  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/discourse-math.git
          - git clone https://github.com/discourse/discourse-reactions.git
          - git clone https://github.com/discourse/discourse-solved.git
          - git clone https://github.com/discourse/discourse-spoiler-alert.git
          - git clone https://github.com/discourse/discourse-whos-online.git
          - git clone https://github.com/discourse/discourse-assign.git
          - git clone https://github.com/discourse/discourse-cakeday.git
          - git clone https://github.com/discourse/discourse-akismet.git
          - git clone https://github.com/discourse/discourse-data-explorer.git
          - git clone https://github.com/discourse/discourse-gamification
          - git clone https://github.com/discourse/discourse-checklist
          - git clone https://github.com/discourse/discourse-footnote
          - git clone https://github.com/discourse/discourse-yearly-review
          - git clone https://github.com/discourse/discourse-encrypt.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  - exec: echo "End of custom commands"

Note: I’ve tried building app w/o any plugins. It either fails to fetch discourse itself or fails to clone 3rd or 4th plugin in the list (which means that with only 2-3 plugins I can build an app from 3rd-4th attempt).

It was 2.9.0.beta14 just before I tried to upgrade to 3.0.0.beta15, don’t recall specific ID but the last time it was upgraded was sometime Dec25-26.

Stephen · January 4, 2023, 4:39pm

It would definitely help for you to be less vague with regards the provider you’re using. They vary wildly in terms of the complexity of their networks.

It’s likely to take far longer to troubleshoot this blindfolded.

hiddenseal · January 5, 2023, 1:23am

It’s not that I was trying to limit the information available, it’s more about whether the name will tell you anything since it’s not a western company. The instance was hosted by reg.ru, the largest hoster in Russia. Anyway, their support this time^[1] wasn’t helpful either, so I moved my site to digital ocean. With the same settings, same procedure used for docker installation, everything worked (and seems to work) fine.

they’re usually fairly good ↩︎

system · February 4, 2023, 1:23am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Can't get discourse working again Installation	8	1597	February 16, 2020
Newest Discourse Version breaks rebuild, Redis Port already in use Installation	8	1274	January 1, 2021
Installation fails on my Debian server Installation	24	2061	March 18, 2019
Could not resolve host: github.com for SamSaffron/pups.git Support docker	12	11637	November 12, 2020
Discourse website not loading after installation without enabling Let's Encrypt Installation	17	2643	April 9, 2020

Docker sometimes fails to connect to github.com

Related topics