설치 복구 시도 중

CentOS 머신에서 여기의 가이드를 따라 Discourse를 설치했습니다.
참고로 NGINX Proxy Manager도 함께 사용하고 있습니다.

설정은 몇 주간 정상적으로 작동하다가, 머신을 재부팅해야 하는 상황이 발생했습니다. 이후에는 시작이 실패했습니다. 아래는 rebuild 명령의 출력 결과입니다:

Ensuring launcher is up to date
Launcher is up-to-date
2.0.20230409-0052: Pulling from discourse/base
Digest: sha256:dd75ceb9322f79629f8b0bf78cfb0f79ad6bb366b7ead3e1cd32dcb8712ec46f
Status: Image is up to date for discourse/base:2.0.20230409-0052
docker.io/discourse/base:2.0.20230409-0052
/usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-04-13T09:55:28.002076 #1]  INFO -- : Reading from stdin
I, [2023-04-13T09:55:28.005509 #1]  INFO -- : > locale-gen $LANG && update-locale
I, [2023-04-13T09:55:28.031953 #1]  INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2023-04-13T09:55:28.032080 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2023-04-13T09:55:28.034322 #1]  INFO -- :
I, [2023-04-13T09:55:28.034470 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-04-13T09:55:28.036190 #1]  INFO -- :
I, [2023-04-13T09:55:28.036297 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2023-04-13T09:55:28.037819 #1]  INFO -- :
I, [2023-04-13T09:55:28.037918 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2023-04-13T09:55:28.039663 #1]  INFO -- :
I, [2023-04-13T09:55:28.039770 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-04-13T09:55:28.041327 #1]  INFO -- :
I, [2023-04-13T09:55:28.041457 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGS
QL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/04/13 09:55:28 socat[19] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file
or directory
I, [2023-04-13T09:55:28.045222 #1]  INFO -- :
I, [2023-04-13T09:55:28.045329 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-04-13T09:55:28.047498 #1]  INFO -- :
I, [2023-04-13T09:55:28.047590 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-04-13T09:55:28.049691 #1]  INFO -- :
I, [2023-04-13T09:55:28.049815 #1]  INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-04-13T09:55:28.051658 #1]  INFO -- :
I, [2023-04-13T09:55:28.051800 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_
stat_tmp
I, [2023-04-13T09:55:28.053477 #1]  INFO -- :
I, [2023-04-13T09:55:28.057301 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown:
I, [2023-04-13T09:55:28.060969 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown:
I, [2023-04-13T09:55:28.064663 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown:
I, [2023-04-13T09:55:28.068319 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown:
I, [2023-04-13T09:55:28.068510 #1]  INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-04-13T09:55:37.064277 #1]  INFO -- :
I, [2023-04-13T09:55:37.064575 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o
I, [2023-04-13T09:55:37.066210 #1]  INFO -- :
I, [2023-04-13T09:55:37.066255 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-04-13T09:55:37.076706 #1]  INFO -- :
I, [2023-04-13T09:55:37.076786 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-04-13T09:55:37.078711 #1]  INFO -- :
I, [2023-04-13T09:55:37.078822 #1]  INFO -- : > /root/upgrade_postgres
I, [2023-04-13T09:55:37.082200 #1]  INFO -- :
I, [2023-04-13T09:55:37.082297 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2023-04-13T09:55:37.083919 #1]  INFO -- :
I, [2023-04-13T09:55:37.084109 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' w
I, [2023-04-13T09:55:37.084484 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addr
I, [2023-04-13T09:55:37.085049 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchrono
I, [2023-04-13T09:55:37.085423 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffer
I, [2023-04-13T09:55:37.085857 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work
I, [2023-04-13T09:55:37.086303 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with d
I, [2023-04-13T09:55:37.086614 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postg
I, [2023-04-13T09:55:37.089011 #1]  INFO -- :
I, [2023-04-13T09:55:37.089245 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoi
I, [2023-04-13T09:55:37.089516 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_co
I, [2023-04-13T09:55:37.089887 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with l
I, [2023-04-13T09:55:37.090322 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) wi
I, [2023-04-13T09:55:37.090509 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all al
I, [2023-04-13T09:55:37.090946 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host a
I, [2023-04-13T09:55:37.091279 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u pos
I, [2023-04-13T09:55:37.092562 #1]  INFO -- : > sleep 5
2023-04-13 09:55:37.239 UTC [42] LOG:  starting PostgreSQL 13.10 (Debian 13.10-1.pgdg110+1) on x86_64-pc
2023-04-13 09:55:37.240 UTC [42] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-04-13 09:55:37.240 UTC [42] LOG:  listening on IPv6 address "::", port 5432
2023-04-13 09:55:37.287 UTC [42] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-04-13 09:55:37.338 UTC [45] LOG:  database system was interrupted; last known up at 2023-04-13 08:3
2023-04-13 09:55:37.795 UTC [45] LOG:  invalid resource manager ID in primary checkpoint record
2023-04-13 09:55:37.795 UTC [45] PANIC:  could not locate a valid checkpoint record
2023-04-13 09:55:38.256 UTC [42] LOG:  startup process (PID 45) was terminated by signal 6: Aborted
2023-04-13 09:55:38.256 UTC [42] LOG:  aborting startup due to startup process failure
2023-04-13 09:55:38.283 UTC [42] LOG:  database system is shut down
I, [2023-04-13T09:55:42.094474 #1]  INFO -- :
I, [2023-04-13T09:55:42.094602 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
createdb: error: could not connect to database template1: connection to server on socket "/var/run/postg
        Is the server running locally and accepting connections on that socket?
I, [2023-04-13T09:55:42.130700 #1]  INFO -- :
I, [2023-04-13T09:55:42.130828 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
        Is the server running locally and accepting connections on that socket?
I, [2023-04-13T09:55:42.166116 #1]  INFO -- :
I, [2023-04-13T09:55:42.166242 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
        Is the server running locally and accepting connections on that socket?
I, [2023-04-13T09:55:42.201461 #1]  INFO -- :
I, [2023-04-13T09:55:42.201617 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
        Is the server running locally and accepting connections on that socket?
I, [2023-04-13T09:55:42.236768 #1]  INFO -- :
I, [2023-04-13T09:55:42.236996 #1]  INFO -- : Terminating async processes

FAILED
--------------------
Pups::ExecError: su postgres -c 'psql discourse -c "alter schema public owner to discourse;"' failed with return #<Process::Status: pid 55 exit 2>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params "su postgres -c 'psql $db_name -c \"alter schema public owner to $db_user;\"'"
bootstrap failed with exit code 2
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
cd04fbb38f1ef61e418d680b969c2c056439e3b1dbfe64608524a8d3361cd91c

discourse-doctor는 실행 중인 Docker 앱을 찾지 못합니다:

==================== SERIOUS PROBLEM!!!! ====================
app not running!
Attempting to rebuild

그리고 재빌드를 시도하여, 결국 위와 동일한 로그로 종료됩니다.

미리 감사드립니다!

PANIC: could not locate a valid checkpoint record

db가 손상된 것 같습니다. 제 생각에는 백업에서 복원하는 것이 가장 좋을 것 같습니다. discourse가 기본적으로 백업을 만드는 것 같은데, discourse/shared/app/backups 같은 곳에서 확인해 보세요.


discourse 백업이 없다면, 이것은 discourse 지원보다는 postgresql 지원에 더 가까운 주제가 됩니다. 저는 전문가가 아니지만, 도움이 될 수 있을 것 같습니다.

아래 링크를 사용해야 할 수도 있습니다:

저도 그렇게 생각합니다. @Norike 님, 해결책을 찾으시길 바랍니다… 그 사이 이 상황이 꽤 걱정됩니다. 소프트웨어에 의한 재부팅은 데이터베이스가 깨끗한 상태로 유지되어야 한다고 생각합니다. 이것이 하드 리부팅, 즉 전원 재가동이었나요?

데이터베이스가 저장된 파일시스템에 따라 차이가 있을지 궁금합니다. df -T 명령을 실행하면 어떤 결과가 나오나요?

감사합니다!

실제로 /var/discourse/shared/standalone/backups/default 폴더에서 몇 가지 백업 파일을 찾았네요. 희망적인 소식입니다.
하지만 명령줄에서 백업 복원하기 가이드에 따르면 데이터베이스를 가져오려면 컨테이너가 실행 중이어야 하는데, ./launcher enter app 명령이 실패합니다:

x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Error response from daemon: No such container: app

아마도 깨끗한 설치로 시작해야 할 것 같습니다. 초기화하고 처음부터 다시 시작하는 가이드가 있을까요? 아래 절차가 올바른가요?

  • /var/discourse/shared/standalone/backups/default 폴더의 백업 파일을 /var/discourse 폴더 외부의 다른 위치로 복사
  • /var/discourse 폴더 삭제
  • Install Cloud 가이드를 따라 Discourse를 처음부터 설치하여 새로 실행되는 인스턴스 확보
  • GUI 또는 위 명령줄 가이드를 사용하여 백업 복원

업데이트를 적용한 후 소프트웨어 재부팅이었습니다.

이것입니다:

Filesystem          Type     1K-blocks     Used Available Use% Mounted on
devtmpfs            devtmpfs      4096        0      4096   0% /dev
tmpfs               tmpfs      3869940        0   3869940   0% /dev/shm
tmpfs               tmpfs      1547980    17600   1530380   2% /run
/dev/mapper/cs-root xfs       73364480 16750976  56613504  23% /
/dev/sda2           xfs        1038336   395812    642524  39% /boot
/dev/mapper/cs-home xfs      893122476 10838052 882284424   2% /home
/dev/sda1           vfat        613160     7644    605516   2% /boot/efi
overlay             overlay   73364480 16750976  56613504  23% /var/lib/docker/overlay2/a9c2622d4167f08ff4697a4c49febf55a7e460e087c41c05e6c1bbd321b13f62/merged
overlay             overlay   73364480 16750976  56613504  23% /var/lib/docker/overlay2/765273939fa75096c994588ba9c9bac6b5a9b909a60a962169b83f7c8a213b7f/merged
tmpfs               tmpfs       773988        4    773984   1% /run/user/1000

감사합니다. XFS는 잘 모르지만, 빠른 검색 결과 사용해도 안전할 것 같습니다. 다만 xfs_info / 명령을 실행하면 유용한 정보를 얻을 수 있을 것 같습니다.

좋습니다. 그렇다면 이 전체 디렉토리를 어딘가에 복사해 두세요:

/var/discourse/shared

이렇게 하면 백업에서 복원하거나, 필요할 경우 손상된 데이터베이스를 수리하는 시도를 할 수 있습니다.

그 다음 도커 컨테이너를 삭제하고, docker image prune -a를 실행한 후 /var/discourse를 삭제하고 discourse를 다시 설치하세요. 마지막으로 가장 최신 백업 파일을 해당 위치에 복사하고 복원을 시도해 보세요.

그렇게 하니까 다시 돌아왔어요. 감사합니다!