I'm using a Unix socket between Caddy and Discourse instead of TCP, but from time to time I get a 'unix:' error

Support
1

The logs show the following:

PG::InvalidTextRepresentation (ERROR: invalid input syntax for type inet: "unix:" LINE 7: client_ip = 'unix:', ^ ) lib/mini_sql_multisite_connection.rb:109:in 'MiniSqlMult

rack-mini-profiler (4.0.1) lib/patches/db/pg/alias_method.rb:109:in 'PG::Connection#exec'
rack-mini-profiler (4.0.1) lib/patches/db/pg/alias_method.rb:109:in 'PG::Connection#async_exec'
mini_sql (1.6.0) lib/mini_sql/postgres/connection.rb:217:in 'MiniSql::Postgres::Connection#run'
mini_sql (1.6.0) lib/mini_sql/active_record_postgres/connection.rb:38:in 'block in MiniSql::ActiveRecordPostgres::Connection#run'
mini_sql (1.6.0) lib/mini_sql/active_record_postgres/connection.rb:34:in 'block in MiniSql::ActiveRecordPostgres::Connection#with_lock'
activesupport (8.0.5.1) lib/active_support/concurrency/null_lock.rb:9:in 'ActiveSupport::Concurrency::NullLock#synchronize'
mini_sql (1.6.0) lib/mini_sql/active_record_postgres/connection.rb:34:in 'MiniSql::ActiveRecordPostgres::Connection#with_lock'
mini_sql (1.6.0) lib/mini_sql/active_record_postgres/connection.rb:38:in 'MiniSql::ActiveRecordPostgres::Connection#run'
lib/mini_sql_multisite_connection.rb:109:in 'MiniSqlMultisiteConnection#run'
mini_sql (1.6.0) lib/mini_sql/postgres/connection.rb:196:in 'MiniSql::Postgres::Connection#exec'
app/models/user_auth_token.rb:244:in 'UserAuthToken#rotate!'
lib/auth/default_current_user_provider.rb:260:in 'Auth::DefaultCurrentUserProvider#refresh_session'
lib/current_user.rb:52:in 'CurrentUser#refresh_session'
app/controllers/application_controller.rb:91:in 'ApplicationController#perform_refresh_session'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:361:in 'block in ActiveSupport::Callbacks::CallTemplate::MethodCall#make_lambda'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:207:in 'ActiveSupport::Callbacks::Filters::After#call'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:563:in 'block in ActiveSupport::Callbacks::CallbackSequence#invoke_after'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:563:in 'Array#each'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:563:in 'ActiveSupport::Callbacks::CallbackSequence#invoke_after'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:134:in 'block in ActiveSupport::Callbacks#run_callbacks'
app/controllers/application_controller.rb:444:in 'block in ApplicationController#with_resolved_locale'
i18n (1.14.8) lib/i18n.rb:354:in 'I18n::Base#with_locale'
app/controllers/application_controller.rb:444:in 'ApplicationController#with_resolved_locale'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks'
app/controllers/application_controller.rb:1126:in 'ApplicationController#ensure_dont_cache_page'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:140:in 'ActiveSupport::Callbacks#run_callbacks'
actionpack (8.0.5.1) lib/abstract_controller/callbacks.rb:260:in 'AbstractController::Callbacks#process_action'
actionpack (8.0.5.1) lib/action_controller/metal/rescue.rb:27:in 'ActionController::Rescue#process_action'
actionpack (8.0.5.1) lib/action_controller/metal/instrumentation.rb:76:in 'block in ActionController::Instrumentation#process_action'
activesupport (8.0.5.1) lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument'
activesupport (8.0.5.1) lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument'
activesupport (8.0.5.1) lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument'
actionpack (8.0.5.1) lib/action_controller/metal/instrumentation.rb:75:in 'ActionController::Instrumentation#process_action'
actionpack (8.0.5.1) lib/action_controller/metal/params_wrapper.rb:259:in 'ActionController::ParamsWrapper#process_action'
activerecord (8.0.5.1) lib/active_record/railties/controller_runtime.rb:39:in 'ActiveRecord::Railties::ControllerRuntime#process_action'
actionpack (8.0.5.1) lib/abstract_controller/base.rb:152:in 'AbstractController::Base#process'
actionview (8.0.5.1) lib/action_view/rendering.rb:40:in 'ActionView::Rendering#process'
rack-mini-profiler (4.0.1) lib/mini_profiler/profiling_methods.rb:90:in 'block in ActionController::Base#profile_method'
actionpack (8.0.5.1) lib/action_controller/metal.rb:252:in 'ActionController::Metal#dispatch'
actionpack (8.0.5.1) lib/action_controller/metal.rb:335:in 'ActionController::Metal.dispatch'
actionpack (8.0.5.1) lib/action_dispatch/routing/route_set.rb:67:in 'ActionDispatch::Routing::RouteSet::Dispatcher#dispatch'
actionpack (8.0.5.1) lib/action_dispatch/routing/route_set.rb:50:in 'ActionDispatch::Routing::RouteSet::Dispatcher#serve'
actionpack (8.0.5.1) lib/action_dispatch/journey/router.rb:53:in 'block in ActionDispatch::Journey::Router#serve'
actionpack (8.0.5.1) lib/action_dispatch/journey/router.rb:133:in 'block in ActionDispatch::Journey::Router#find_routes'
actionpack (8.0.5.1) lib/action_dispatch/journey/router.rb:126:in 'Array#each'
actionpack (8.0.5.1) lib/action_dispatch/journey/router.rb:126:in 'ActionDispatch::Journey::Router#find_routes'
actionpack (8.0.5.1) lib/action_dispatch/journey/router.rb:34:in 'ActionDispatch::Journey::Router#serve'
actionpack (8.0.5.1) lib/action_dispatch/routing/route_set.rb:908:in 'ActionDispatch::Routing::RouteSet#call'
lib/middleware/omniauth_bypass_middleware.rb:35:in 'Middleware::OmniauthBypassMiddleware#call'
lib/middleware/crawler_hooks.rb:13:in 'Middleware::CrawlerHooks#call'
rack (2.2.23) lib/rack/tempfile_reaper.rb:15:in 'Rack::TempfileReaper#call'
rack (2.2.23) lib/rack/conditional_get.rb:27:in 'Rack::ConditionalGet#call'
rack (2.2.23) lib/rack/head.rb:12:in 'Rack::Head#call'
actionpack (8.0.5.1) lib/action_dispatch/http/permissions_policy.rb:38:in 'ActionDispatch::PermissionsPolicy::Middleware#call'
lib/content_security_policy/middleware.rb:12:in 'ContentSecurityPolicy::Middleware#call'
lib/middleware/anonymous_cache.rb:425:in 'Middleware::AnonymousCache#call'
lib/middleware/csp_script_nonce_injector.rb:13:in 'Middleware::CspScriptNonceInjector#call'
lib/middleware/track_view_session_id_injector.rb:12:in 'Middleware::TrackViewSessionIdInjector#call'
config/initializers/008-rack-cors.rb:14:in 'Discourse::Cors#call'
rack (2.2.23) lib/rack/session/abstract/id.rb:266:in 'Rack::Session::Abstract::Persisted#context'
rack (2.2.23) lib/rack/session/abstract/id.rb:260:in 'Rack::Session::Abstract::Persisted#call'
actionpack (8.0.5.1) lib/action_dispatch/middleware/cookies.rb:706:in 'ActionDispatch::Cookies#call'
actionpack (8.0.5.1) lib/action_dispatch/middleware/callbacks.rb:31:in 'block in ActionDispatch::Callbacks#call'
activesupport (8.0.5.1) lib/active_support/callbacks.rb:100:in 'ActiveSupport::Callbacks#run_callbacks'
actionpack (8.0.5.1) lib/action_dispatch/middleware/callbacks.rb:30:in 'ActionDispatch::Callbacks#call'
actionpack (8.0.5.1) lib/action_dispatch/middleware/debug_exceptions.rb:31:in 'ActionDispatch::DebugExceptions#call'
actionpack (8.0.5.1) lib/action_dispatch/middleware/show_exceptions.rb:32:in 'ActionDispatch::ShowExceptions#call'
logster (2.21.0) lib/logster/middleware/reporter.rb:40:in 'Logster::Middleware::Reporter#call'
lib/middleware/default_headers.rb:13:in 'Middleware::DefaultHeaders#call'
railties (8.0.5.1) lib/rails/rack/logger.rb:41:in 'Rails::Rack::Logger#call_app'
railties (8.0.5.1) lib/rails/rack/logger.rb:29:in 'Rails::Rack::Logger#call'
config/initializers/100-quiet_logger.rb:20:in 'DiscourseRackQuietAssetsLogger#call'
config/initializers/100-silence_logger.rb:29:in 'SilenceLogger#call'
actionpack (8.0.5.1) lib/action_dispatch/middleware/request_id.rb:34:in 'ActionDispatch::RequestId#call'
lib/middleware/enforce_hostname.rb:23:in 'Middleware::EnforceHostname#call'
rack (2.2.23) lib/rack/method_override.rb:24:in 'Rack::Metho

Job exception: ERROR: invalid input syntax for type inet: "unix:" LINE 2: SET ip_address = 'unix:' ^

rack-mini-profiler-4.0.1/lib/patches/db/pg/alias_method.rb:109:in 'PG::Connection#exec'
rack-mini-profiler-4.0.1/lib/patches/db/pg/alias_method.rb:109:in 'PG::Connection#async_exec'
mini_sql-1.6.0/lib/mini_sql/postgres/connection.rb:217:in 'MiniSql::Postgres::Connection#run'
mini_sql-1.6.0/lib/mini_sql/active_record_postgres/connection.rb:38:in 'block in MiniSql::ActiveRecordPostgres::Connection#run'
mini_sql-1.6.0/lib/mini_sql/active_record_postgres/connection.rb:34:in 'block in MiniSql::ActiveRecordPostgres::Connection#with_lock'
activesupport-8.0.5.1/lib/active_support/concurrency/null_lock.rb:9:in 'ActiveSupport::Concurrency::NullLock#synchronize'
mini_sql-1.6.0/lib/mini_sql/active_record_postgres/connection.rb:34:in 'MiniSql::ActiveRecordPostgres::Connection#with_lock'
mini_sql-1.6.0/lib/mini_sql/active_record_postgres/connection.rb:38:in 'MiniSql::ActiveRecordPostgres::Connection#run'
/var/www/discourse/lib/mini_sql_multisite_connection.rb:109:in 'MiniSqlMultisiteConnection#run'
mini_sql-1.6.0/lib/mini_sql/postgres/connection.rb:196:in 'MiniSql::Postgres::Connection#exec'
/var/www/discourse/app/models/user.rb:1145:in 'User.update_ip_address!'
/var/www/discourse/lib/auth/default_current_user_provider.rb:235:in 'block in Auth::DefaultCurrentUserProvider#current_user'
/var/www/discourse/lib/scheduler/defer.rb:135:in 'block in Scheduler::Deferrable#do_work'
rails_multisite-7.0.0/lib/rails_multisite/connection_management/null_instance.rb:49:in 'RailsMultisite::ConnectionManagement::NullInstance#with_connection'
rails_multisite-7.0.0/lib/rails_multisite/connection_management.rb:17:in 'RailsMultisite::ConnectionManagement.with_connection'
/var/www/discourse/lib/scheduler/defer.rb:130:in 'Scheduler::Deferrable#do_work'
/var/www/discourse/lib/scheduler/defer.rb:116:in 'block (2 levels) in Scheduler::Deferrable#start_thread'

When it happens, an “Oops - Error 500” page is displayed. At first, I thought Caddy was the culprit because it wasn’t forwarding the client’s IP address to Discourse, so I tried several different configurations, but none of them solved the issue.

It’s worth mentioning that this doesn’t happen very often. It occurs only occasionally, and when it does, simply refreshing the page immediately recovers the site. It then works normally for quite some time before the error randomly occurs again.

2

I’m not an expert and I did not use Caddy but Nginx on my self-hosted instance, but can I ask for your actual Caddy and Discourse templates configuration in ./app/containers.yml?

Can you remember what you usually did before the error happens? I mean, changing admin settings, posting, using some plugin?

I’m sorry not to give specific help to your issue, but I think your replies can add value to your question for getting a clear and fast response from the community.

3

This means that a request / user has a remote ip pointing to your socket, which means something in your proxy chain is misconfigured.

4

@satonotdead @Falco

app.yml

templates:
  - "templates/postgres.18.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  
  - "templates/web.socketed.template.yml"
  
  - "templates/enable-ruby-yjit.yml"


## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  # - "66:80"   # http
  # - "66:443" # https

params:
  ## Which Git revision should this container use? (default: latest)
  version: latest
  ## Maximum upload size (default: 10m)
  upload_size: 150m
  
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  db_shared_buffers: "2048MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  # DISCOURSE_DEFAULT_LOCALE: en

  ## https://meta.discourse.org/t/rescaling-the-server-which-configs-need-to-be-changed-unicorn-workers-memory-etc/252788
  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  UNICORN_WORKERS: 8

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: example.com

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: 'admin+discourse@example.com'

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS is required
  # WARNING: SMTP password should be wrapped in quotes to avoid problems
  DISCOURSE_SMTP_ADDRESS: smtp.provider.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: noreply@example.com
  DISCOURSE_SMTP_PASSWORD: "***"
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default: true)
  DISCOURSE_SMTP_DOMAIN: example.com # (required by some providers)
  DISCOURSE_NOTIFICATION_EMAIL: noreply@example.com
  #DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: peer        # (optional, default: peer, valid values: none, peer, client_once, fail_if_no_peer_cert)
  #DISCOURSE_SMTP_AUTHENTICATION: plain            # (default: plain, valid values: plain, login, cram_md5)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  # LETSENCRYPT_ACCOUNT_EMAIL: admin+letsencrypt@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com

  ## The maxmind geolocation IP account ID and license key for IP address lookups
  ## see https://meta.discourse.org/t/-/173941 for details
  #DISCOURSE_MAXMIND_ACCOUNT_ID: 123456
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

  # Force HTTPS
  DISCOURSE_FORCE_HTTPS: true
  
  # Request Limits
  DISCOURSE_MAX_REQS_PER_IP_MODE: none
  DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE: 12000
  
  DISCOURSE_MAX_DATA_EXPLORER_API_REQ_MODE: none
  DISCOURSE_MAX_DATA_EXPLORER_API_REQS_PER_10_SECONDS: 1000

  DISCOURSE_YJIT_ENABLED: true

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log
  - volume:
      host: /var/discourse/plugins
      guest: /var/plugins

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git
          - cp -a /var/plugins/. $home/plugins/

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"

Caddyfile

#---------------------------------------------------------------------------------

{
	storage file_system {
		root /var/caddy/data
	}
}

#---------------------------------------------------------------------------------

# To achieve an A+ rating in SSL security
(hsts) {
	header {
		Strict-Transport-Security "max-age=31536000; includeSubDomains"
	}
}

# self-hosted acme-dns credentials
(tls-challenge) {
	tls admin+letsencrypt@example.com {
		dns acmedns {
			username ***
			password ***
			subdomain ***
			server_url http://acme.example.com:5005
		}
	}
}

#---------------------------------------------------------------------------------
# Forum (Discourse)
#---------------------------------------------------------------------------------
example.com {
	#-------------------------------------------------------------------------------
	import hsts
	import tls-challenge
	#-------------------------------------------------------------------------------
	request_header X-Real-IP {remote_host}
	reverse_proxy unix//var/discourse/shared/standalone/nginx.http.sock
	#-------------------------------------------------------------------------------
}

#---------------------------------------------------------------------------------
# Subdominios
#---------------------------------------------------------------------------------

#---------------------------------------------------------------------------------
*.example.com {
	#-------------------------------------------------------------------------------
	import hsts
	import tls-challenge
	#-------------------------------------------------------------------------------

	# Force the non-www version of the domain
	@www host www.example.com
	redir @www https://example.com{uri} permanent
  
	#-------------------------------------------------------------------------------
  
	@store host store.example.com
	handle @store {
		@wc_private {
			path /license.txt
			path /readme.html

			# Archivos críticos
			path /wp-config.php
			path /xmlrpc.php
			path /wp-settings.php
			path /wp-load.php
			path /wp-blog-header.php

			path /wp-admin.php

			path /wp-admin/install.php

			path /wp-content/uploads/wc-logs/*
			path /wp-content/uploads/nuvei-logs/*

			path /wp-content/uploads/woocommerce_uploads/*
		}

		# protect_php
		@wc_private_php {
			not path /wp-includes/ms-files.php

			path_regexp protect_php ^/(wp-includes|wp-admin/includes|wp-content/uploads)/.*\.php$
		}
		respond @wc_private 403
		respond @wc_private_php 403

		root * /usr/share/wordpress
		php_fastcgi unix//run/php/php8.3-fpm.sock
		file_server
	}
  
  #-------------------------------------------------------------------------------
  # Multiple standalone HTML files
  #-------------------------------------------------------------------------------

  @join host join.example.com
  handle @join {
    root * /var/example/join
    file_server
  }

  #-------------------------------------------------------------------------------
  # No matchers
  #-------------------------------------------------------------------------------

	handle {
		respond 404
	}

	#-------------------------------------------------------------------------------
}

To be more specific, this error occurs several hours after the previous one, it doesn’t happen very often. I don’t see anything unusual in app.yml or the Caddyfile.

5

If the connection to the socket is right until it gets dropped, I think you can add explicit timeouts and transport options.

6

I think you can add explicit timeouts and transport options

How?

7

Well, as I said before, I did not use Caddy but you can look into their official docs.

8

I think I found the problem. If I perform a rebuild, I never restart Caddy. From now on, I’m going to do this:

./launcher rebuild app && systemctl reload caddy

I suspect that Caddy uses the socket cache or something similar after a rebuild. But it doesn’t usually happen immediately after a rebuild. We’ll see.