This causes mail delivery to fail, and discourse-doctor confirms the mangled password.
Notes
• Quoting the password explicitly at the prompt does not change the outcome.
• Password contains special characters (@, !), but the problem is not just YAML quoting: the username string is literally concatenated at the start of the password.
• Reproducible across multiple runs of ./discourse-setup.
Yeah. That’s the issue. There are multiple levels of escaping that need to happen, like when bash reads the value, when bash hands the value to sed, when sed replaces it, and then, maybe, when the yml file gets it. It’s a known issue:
עורך את containers/app.yml באמצעות Ruby’s YAML (Psych), כך שציטוט/בריחה של YAML מטופלים על ידי מנתח אמיתי
מוחק את משתני ה-SMTP לכל מפתח כדי למנוע סתירות
החל באמצעות git apply -p0 במאגר discourse_docker.
תיקון 1 -
discourse-setup
(כותב SMTP באמצעות Ruby YAML, לא sed)
--- a/discourse-setup
+++ b/discourse-setup
@@ -867,6 +867,77 @@ write_smtp_settings() {
local app_yml="containers/app.yml"
[[ -f "$app_yml" ]] || die "Cannot find $app_yml. Did you run bootstrap?"
+ # Build a URL-encoded SMTP_URL using Python stdlib (no shell escaping games)
+ urlencode() {
+ python3 - <<'PY'
+import sys, urllib.parse
+print(urllib.parse.quote(sys.stdin.read().strip(), safe='._~-'))
+PY
+ }
+
+ # IMPORTANT: read vars without backslash mangling
+ # (these come from earlier prompts; just ensure -r is used at prompt time)
+ local addr="$smtp_address"
+ local port="$smtp_port"
+ local user_enc pass_enc
+ user_enc="$(printf '%s' "$smtp_user" | urlencode)"
+ pass_enc="$(printf '%s' "$smtp_password" | urlencode)"
+ local smtp_url="smtp://${user_enc}:${pass_enc}@${addr}:${port}"
+
+ # Use Ruby to safely load/modify/dump YAML (kills 3 layers of escaping)
+ ruby - <<'RUBY' "$app_yml" "$smtp_url"
+require "yaml"
+require "psych"
+path, url = ARGV
+doc = YAML.safe_load(File.read(path), permitted_classes: [], aliases: true) || {}
+
+# Ensure top-level structure is a Hash and has env
+unless doc.is_a?(Hash)
+ abort "containers/app.yml does not parse to a Hash"
+end
+doc["env"] ||= {}
+env = doc["env"]
+
+# Write single-line SMTP_URL; remove per-key vars to avoid conflicts
+env["SMTP_URL"] = url
+%w[DISCOURSE_SMTP_ADDRESS DISCOURSE_SMTP_PORT DISCOURSE_SMTP_USER_NAME DISCOURSE_SMTP_PASSWORD].each { |k| env.delete(k) }
+
+# Dump back. (Psych preserves strings safely quoted as needed.)
+File.write(path, Psych.dump(doc))
+RUBY
+
+ # quick sanity check for the classic "password prefixed by username" failure
+ python3 - <<'PY'
+import re, sys
+y = open("containers/app.yml","r",encoding="utf-8").read()
+m = re.search(r'^\\s*SMTP_URL:\\s*(?:\"|\')?([^\r\n\"\'\']+)', y, re.M)
+assert m, "SMTP_URL missing after write"
+creds = m.group(1).split('@',1)[0].split('://',1)[-1]
+assert ":" in creds, "SMTP_URL creds missing ':'"
+u, p = creds.split(':',1)
+assert not p.startswith(u), "Password appears prefixed by username"
+print("SMTP_URL looks sane.")
+PY
+}
+
- # Write per-key SMTP entries (address/port/username/password)
- # (legacy: performed via sed substitutions)
- # NOTE: historically fragile with special chars
- update_setting_yaml "DISCOURSE_SMTP_ADDRESS" "$smtp_address"
- update_setting_yaml "DISCOURSE_SMTP_PORT" "$smtp_port"
- update_setting_yaml "DISCOURSE_SMTP_USER_NAME" "$smtp_user"
- update_setting_yaml "DISCOURSE_SMTP_PASSWORD" "$smtp_password"
-}
+ # (legacy per-key writes removed in favor of SMTP_URL via YAML)
+}
ואז תיקון 2 -
templates/web.template.yml
(כדי לתעד את הנתיב הבטוח יותר)
--- a/templates/web.template.yml
+++ b/templates/web.template.yml
@@ -68,6 +68,14 @@ params:
DISCOURSE_SMTP_ENABLE_START_TLS: true
#DISCOURSE_NOTIFICATION_EMAIL: noreply@example.com
+ ## Preferred single-line SMTP configuration (set by discourse-setup):
+ ## URL-encode username & password; example:
+ ## SMTP_URL: "smtp://user%40example.com:p%40ss%3Aword@smtp.example.com:587"
+ ##
+ #SMTP_URL:
+
## If you cannot use SMTP_URL, you may set per-key variables instead.
## Beware that editing those lines with shell tools can be fragile if values include
## characters like @, :, /, ", \, or newlines.
למה זה עובד (ומה זה נמנע)
• שכבת bash: אנו רק משלבים משתנים פשוטים; סודות מועברים לפייתון/רובי דרך stdin/argv, לא דרך ביטויי sed או הערכות shell.
• שכבת sed: הוסרה לחלוטין.
• שכבת YAML: Ruby/Psych מטפלת בציטוט ובבריחה כראוי; אין ציטוטים שנעשו ידנית.
• אישורי SMTP: קידוד % ב-SMTP_URL הוא המקום הנכון לקודד תווים מיוחדים לאימות.
אם אתה מעדיף לשמור על משתנים לכל מפתח, אני יכול לתת לך תיקון אחות שמשתמש באותה גישת Ruby-YAML כדי להגדיר ישירות את DISCOURSE_SMTP_* (עדיין ללא sed), אבל נתיב ה-SMTP_URL הוא הנקי ביותר מכיוון שהוא מפתח אחד, כתיבה אחת, שלב קידוד אחד.
Got it — no Ruby/Python on the host. Here’s a pure Bash + awk patch that:
• builds a percent-encoded SMTP_URL using only Bash (byte-wise loop; no Python),
• inserts or replaces SMTP_URL under the env: block using awk (no sed),
• removes the per-key DISCOURSE_SMTP_* lines (safe anchored deletes),
• adds a tiny sanity check using grep/awk only.
Apply with git apply -p0 in the discourse_docker repo.
--- a/discourse-setup
+++ b/discourse-setup
@@ -867,6 +867,130 @@ write_smtp_settings() {
local app_yml="containers/app.yml"
[[ -f "$app_yml" ]] || die "Cannot find $app_yml. Did you run bootstrap?"
+ ##############################################
+ # Pure-Bash URL encoder for SMTP credentials #
+ ##############################################
+ # Encodes everything except A-Z a-z 0-9 . _ ~ -
+ # Works byte-by-byte; requires bash and printf.
+ urlencode_cred() {
+ local s="$1" out= i ch o
+ # set C locale to get byte semantics
+ LC_ALL=C
+ for ((i=0; i<${#s}; i++)); do
+ ch="${s:i:1}"
+ case "$ch" in
+ [A-Za-z0-9._~-])
+ out+="$ch"
+ ;;
+ *)
+ # Get byte value: print char, read with od, then format %HH
+ # Avoid external heavy deps; od is in coreutils / busybox.
+ o=$(printf '%s' "$ch" | od -An -tu1 | awk '{$1=$1;print $1}')
+ # If od somehow failed (empty), fall back to hex via printf %02X of first byte
+ if [ -z "$o" ]; then
+ o=$(printf '%s' "$ch" | head -c1 | od -An -tu1 | awk '{$1=$1;print $1}')
+ fi
+ printf -v o '%%%02X' "$o"
+ out+="$o"
+ ;;
+ esac
+ done
+ printf '%s' "$out"
+ }
+
+ # Build SMTP_URL (single line) from collected answers
+ # These vars are gathered earlier in discourse-setup:
+ # $smtp_address $smtp_port $smtp_user $smtp_password
+ local addr="$smtp_address"
+ local port="$smtp_port"
+ local user_enc pass_enc
+ user_enc="$(urlencode_cred "$smtp_user")"
+ pass_enc="$(urlencode_cred "$smtp_password")"
+ local smtp_url="smtp://${user_enc}:${pass_enc}@${addr}:${port}"
+
+ ########################################################
+ # YAML-safe edit via awk (no sed / no external runtimes)
+ # - ensure env: exists
+ # - insert or replace SMTP_URL under env:
+ # - remove DISCOURSE_SMTP_* keys
+ ########################################################
+ awk -v NEWVAL="$smtp_url" '
+ BEGIN{
+ have_env=0; in_env=0; inserted=0
+ }
+ # detect env: line at top level (start of line, possibly indented 0..)
+ # we’ll consider two-space indentation for children.
+ /^[[:space:]]*env:[[:space:]]*$/ {
+ print; have_env=1; in_env=1; next
+ }
+ # leaving env: block when indentation returns to 0 or next top-level key
+ in_env && /^[^[:space:]]/ {
+ if (!inserted) {
+ print " SMTP_URL: \"" NEWVAL "\""
+ inserted=1
+ }
+ in_env=0
+ }
+ # while in env:, handle replacements and deletions
+ in_env {
+ # drop per-key DISCOURSE_SMTP_* lines entirely
+ if ($0 ~ /^[[:space:]]*DISCOURSE_SMTP_(ADDRESS|PORT|USER_NAME|PASSWORD):/) next
+ # replace existing SMTP_URL line
+ if ($0 ~ /^[[:space:]]*SMTP_URL:[[:space:]]*/) {
+ print " SMTP_URL: \"" NEWVAL "\""
+ inserted=1
+ next
+ }
+ print
+ next
+ }
+ { print }
+ END{
+ # If env: never existed, append it with the key
+ if (!have_env) {
+ print ""
+ print "env:"
+ print " SMTP_URL: \"" NEWVAL "\""
+ } else if (in_env && !inserted) {
+ # env: existed and we were still in it at EOF
+ print " SMTP_URL: \"" NEWVAL "\""
+ }
+ }
+ ' "$app_yml" > "$app_yml.tmp.$$" && mv "$app_yml.tmp.$$" "$app_yml"
+
+ ##############################################
+ # Sanity check: basic guard against mangling #
+ ##############################################
+ # 1) SMTP_URL present
+ grep -q '^[[:space:]]*SMTP_URL:' "$app_yml" || die "SMTP_URL not written to $app_yml"
+ # 2) password not prefixed by username (classic failure signature)
+ awk '
+ BEGIN{ok=1}
+ /^[[:space:]]*SMTP_URL:[[:space:]]*"/{
+ line=$0
+ gsub(/^[[:space:]]*SMTP_URL:[[:space:]]*"/,"",line)
+ gsub(/".*$/,"",line)
+ # Extract creds before @ and after scheme
+ # e.g. smtp://user:pass@host:port
+ sub(/^[a-z]+:\/\//,"",line)
+ at=index(line,"@")
+ if (at>0) {
+ creds=substr(line,1,at-1)
+ colon=index(creds,":")
+ if (colon>0) {
+ user=substr(creds,1,colon-1)
+ pass=substr(creds,colon+1)
+ if (index(pass,user)==1) { ok=0 }
+ }
+ }
+ }
+ END{ if (!ok) { print "Password appears prefixed by username"; exit 1 } }
+ ' "$app_yml"
}
- # Write per-key SMTP entries (address/port/username/password)
- # (legacy: performed via sed substitutions)
- # NOTE: historically fragile with special chars
- update_setting_yaml "DISCOURSE_SMTP_ADDRESS" "$smtp_address"
- update_setting_yaml "DISCOURSE_SMTP_PORT" "$smtp_port"
- update_setting_yaml "DISCOURSE_SMTP_USER_NAME" "$smtp_user"
- update_setting_yaml "DISCOURSE_SMTP_PASSWORD" "$smtp_password"
-}
+ # Legacy per-key writes removed in favor of atomic SMTP_URL write above.
+}
Why this meets your constraint
• Zero Ruby/Python on the host: only Bash + awk + od/printf/grep (all standard).
• No sed replacement of secrets: we avoid the “Bash → sed → YAML” multi-escape trap.
• Atomic-ish write: edit into a temp file, then mv over containers/app.yml.
• Backwards-safe: if env: doesn’t exist, we create it minimally; if it does, we update in place and remove conflicting DISCOURSE_SMTP_* lines.
• Credentials are URL-encoded before insertion, so special chars in user/pass won’t break the URL parser in Rails/Net::SMTP.
If you’d prefer to keep the per-key variables instead of SMTP_URL, I can send a sibling patch that (still without sed) uses the same awk strategy to set:
hmmm אני תוהה אם אנחנו יכולים פשוט לארוז את הלוגיקה ב-launcher2 ואז נוכל להשתמש בו כדי להריץ את ההתקנה.
כמות ה"קפיצות" ש-discourse-setup עובר היא מרהיבה.
