恢复失败，ALTER TABLE 挂起，超时

deargle · 2021 年4 月 2 日 23:43

我的 Discourse 服务器发生了磁盘故障，我正在尝试将 S3 上的备份恢复到 AWS EC2 t2.small 实例上。 consistently 出现以下错误。

使用的流程如下：

git clone https://github.com/discourse/discourse_docker /var/discourse

我使用的是这个 app.yml

./launcher bootstrap app
./launcher start app

然后我通过 scp 传输备份文件，执行 ./launcher enter app，接着运行 discourse enable_restore 和 discourse restore <文件>。

随后出现以下情况（开头部分已截断）：

14
(1 row)
setval
--------
1
(1 row)
setval
--------
1
(1 row)
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
 #<Thread:0x0000562604925448 /var/www/discourse/lib/sidekiq/pausable.rb:79 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
        33: from /var/www/discourse/lib/sidekiq/pausable.rb:83:in `block (2 levels) in extend_lease_thread'
        32: from /var/www/discourse/lib/sidekiq/pausable.rb:83:in `synchronize'
        31: from /var/www/discourse/lib/sidekiq/pausable.rb:84:in `block (3 levels) in extend_lease_thread'
        30: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each'
        29: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each_key'
        28: from /var/www/discourse/lib/sidekiq/pausable.rb:85:in `block (4 levels) in extend_lease_thread'
        27: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:76:in `with_connection'
        26: from /var/www/discourse/lib/sidekiq/pausable.rb:86:in `block (5 levels) in extend_lease_thread'
        25: from /var/www/discourse/lib/discourse_redis.rb:59:in `block (2 levels) in <class:DiscourseRedis>'
        24: from /var/www/discourse/lib/discourse_redis.rb:29:in `ignore_readonly'
        23: from /var/www/discourse/lib/discourse_redis.rb:59:in `block (3 levels) in <class:DiscourseRedis>'
        22: from /var/www/discourse/lib/discourse_redis.rb:59:in `public_send'
        21: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:425:in `expire'
        20: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `synchronize'
        19: from /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
        18: from /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
        17: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `block in synchronize'
        16: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:426:in `block in expire'
        15: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.1/lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'
        14: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `call'
        13: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:237:in `process'
        12: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:325:in `logging'
        11: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:238:in `block in process'
        10: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:389:in `ensure_connected'
         9: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:248:in `block (2 levels) in process'
         8: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `block in call'
         7: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:279:in `read'
         6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:268:in `io'
         5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:280:in `block in read'
         4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:378:in `read'
         3: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:45:in `gets'
         2: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:52:in `_read_from_socket'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:52:in `loop'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:56:in `block in _read_from_socket': Redis::TimeoutError (Redis::TimeoutError)
        33: from /var/www/discourse/lib/sidekiq/pausable.rb:83:in `block (2 levels) in extend_lease_thread'
        32: from /var/www/discourse/lib/sidekiq/pausable.rb:83:in `synchronize'
        31: from /var/www/discourse/lib/sidekiq/pausable.rb:84:in `block (3 levels) in extend_lease_thread'
        30: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each'
        29: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each_key'
        28: from /var/www/discourse/lib/sidekiq/pausable.rb:85:in `block (4 levels) in extend_lease_thread'
        27: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:76:in `with_connection'
        26: from /var/www/discourse/lib/sidekiq/pausable.rb:86:in `block (5 levels) in extend_lease_thread'
        25: from /var/www/discourse/lib/discourse_redis.rb:59:in `block (2 levels) in <class:DiscourseRedis>'
        24: from /var/www/discourse/lib/discourse_redis.rb:29:in `ignore_readonly'
        23: from /var/www/discourse/lib/discourse_redis.rb:59:in `block (3 levels) in <class:DiscourseRedis>'
        22: from /var/www/discourse/lib/discourse_redis.rb:59:in `public_send'
        21: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:425:in `expire'
        20: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `synchronize'
        19: from /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
        18: from /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
        17: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `block in synchronize'
        16: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:426:in `block in expire'
        15: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.1/lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'
        14: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `call'
        13: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:237:in `process'
        12: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:325:in `logging'
        11: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:238:in `block in process'
        10: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:389:in `ensure_connected'
         9: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:248:in `block (2 levels) in process'
         8: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `block in call'
         7: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:279:in `read'
         6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:268:in `io'
         5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:280:in `block in read'
         4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:378:in `read'
         3: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:45:in `gets'
         2: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:52:in `_read_from_socket'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:52:in `loop'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/connection/ruby.rb:56:in `block in _read_from_socket': Connection timed out (Redis::TimeoutError)
^[[A  ^[[A^[[B       服务器意外关闭了连接
这可能意味着服务器在处理请求之前或处理过程中异常终止。
与服务器的连接已丢失
异常：psql 失败：与服务器的连接已丢失

consistently，恢复进程总是进行到这一步，然后在最后一个（第 85 个）ALTER TABLE 处挂起。我提取了备份中的 dump.sql 文件，如果我没数错的话，第 85 条是这条关于 „public.reviewables

deargle · 2021 年4 月 3 日 00:57

我想我已经找到问题所在了。我本应该在新的主机上运行一次 ./discourse-setup，以便：(1) 创建交换文件，以及 (2) 设置合理的 db_shared_buffers 值。这两个配置中有一个错误，或者两个都有问题，导致了持续的挂起。

pfaffman · 2021 年4 月 3 日 01:03

好眼力！欢迎回来，也欢迎你加入“自己回答自己的问题”俱乐部！我有一大堆这类主题，不过我相信很多已经被删除了。

另一件事是复制 app.yml，但你还是得记得交换操作。除了这个交换的小麻烦外，如果 S3 配置已包含在 yml 文件中，那么这个 yml 文件就足够了。

很高兴你已恢复正常运行！

话题		回复	浏览量
Help restoring - system hung at midnight Self-hosting	26	2506	2022 年6 月 13 日
Can't restore saved backup Self-hosting	3	1263	2017 年3 月 7 日
Backup failed due to PG/SQL errors Self-hosting	9	2420	2020 年12 月 17 日
Hangs discourse and discourse restore FAILED Self-hosting	24	2083	2020 年8 月 26 日
Restore has failed. when i backup and restore to a new discourse Support	13	1678	2019 年9 月 16 日

恢复失败，ALTER TABLE 挂起，超时

相关话题