恢复问题

基于 discourse_docker 仓库,我编写了一个小脚本来实现在 Vagrant 虚拟机中自动化使用它(运行前加上 set -x 即可看到具体执行的操作)

我是不是把备份文件复制到了错误的位置?

还有这条日志信息
Making sure /var/www/discourse/tmp/restores/default/2020-05-13-190832 exists...
是什么意思?

~/infra/discourse on  master! ⌚ 21:14:07
$ pwd
/home/pihentagy/infra/discourse
~/infra/discourse on  master! ⌚ 21:01:14
$ ./wl.sh start
+ set -e
+ VAGRANT_MACHINE_NAME=guest
+ cd discourse
+ case "$1" in
+ init
+ printf 'Checking out and updating repo…\n'
Checking out and updating repo…
+ cd ..
+ git clone https://github.com/discourse/discourse_docker.git discourse
fatal: destination path 'discourse' already exists and is not an empty directory.
+ printf 'Repo already there\n'
Repo already there
+ cd discourse
+ printf 'Updating repo…\n'
Updating repo…
+ git pull -r
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 2), reused 5 (delta 2), pack-reused 0
Unpacking objects: 100% (6/6), done.
From https://github.com/discourse/discourse_docker
   3e465a2..49ed141  master     -> origin/master
Created autostash: 36aae80
HEAD is now at 3e465a2 Remove all pg12 traces so pg_wrapper doesn't get confused
First, rewinding head to replay your work on top of it...
Fast-forwarded master to 49ed14152971f7f4a7437657987952be44c33c0a.
Applying autostash resulted in conflicts.
Your changes are safe in the stash.
You can run "git stash pop" or "git stash drop" at any time.
+ printf 'Copying config file…\n'
Copying config file…
+ cp ../resources/discourse.yml containers/
+ echo 'Starting Vagrant machine...'
Starting Vagrant machine...
+ vagrant up
Bringing machine 'dockerhost' up with 'virtualbox' provider...
==> dockerhost: Checking if box 'ubuntu/xenial64' is up to date...
==> dockerhost: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> dockerhost: flag to force provisioning. Provisioners marked to run always will still run.
+ vagrant ssh -c 'cd /vagrant;sudo ./launcher start discourse'
2627afdfbaac
Nothing to do, your container has already started!
Connection to 127.0.0.1 closed.
+ exit 0

~/infra/discourse on  master! ⌚ 21:07:56
$ ./wl.sh restore /home/pihentagy/infra/icontest-2020-05-12-033823-v20200506044956.tar.gz
+ set -e
+ VAGRANT_MACHINE_NAME=guest
+ cd discourse
+ case "$1" in
+ shift
+ backup=/home/pihentagy/infra/icontest-2020-05-12-033823-v20200506044956.tar.gz
+ discourse_backup_dir=shared/standalone/backups/default
+ mkdir --parents shared/standalone/backups/default
+ rsync -P --verbose /home/pihentagy/infra/icontest-2020-05-12-033823-v20200506044956.tar.gz shared/standalone/backups/default
icontest-2020-05-12-033823-v20200506044956.tar.gz
    390,774,609 100%  317.41MB/s    0:00:01 (xfr#1, to-chk=0/1)

sent 390,870,133 bytes  received 35 bytes  156,348,067.20 bytes/sec
total size is 390,774,609  speedup is 1.00
+ vagrant ssh -c 'sudo docker exec -w /var/www/discourse -i discourse discourse enable_restore'
Restore are now permitted. Disable them with `disable_restore`
Connection to 127.0.0.1 closed.
+ vagrant ssh -c 'sudo docker exec -w /var/www/discourse -i discourse discourse restore'
You must provide a filename to restore. Did you mean one of the following?

Connection to 127.0.0.1 closed.
+ vagrant ssh -c 'sudo docker exec -w /var/www/discourse -i discourse discourse restore icontest-2020-05-12-033823-v20200506044956.tar.gz'
Starting restore: icontest-2020-05-12-033823-v20200506044956.tar.gz
[STARTED]
'system' has started the restore!
Marking restore as running...
Making sure /var/www/discourse/tmp/restores/default/2020-05-13-190832 exists...
Copying archive to tmp directory...
EXCEPTION: lib/discourse.rb:90:in `exec': Failed to copy archive to tmp directory.
cp: cannot stat '/var/www/discourse/public/backups/default/icontest-2020-05-12-033823-v20200506044956.tar.gz': No such file or directory
lib/discourse.rb:100:in `execute_command'
lib/discourse.rb:90:in `exec'
lib/discourse.rb:40:in `execute_command'
/var/www/discourse/lib/backup_restore/local_backup_store.rb:42:in `download_file'
/var/www/discourse/lib/backup_restore/backup_file_handler.rb:61:in `copy_archive_to_tmp_directory'
/var/www/discourse/lib/backup_restore/backup_file_handler.rb:21:in `decompress'
/var/www/discourse/lib/backup_restore/restorer.rb:42:in `run'
script/discourse:143:in `restore'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
script/discourse:284:in `<top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/exe/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.4/exe/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Trying to rollback...
There was no need to rollback
Cleaning stuff up...
Removing tmp '/var/www/discourse/tmp/restores/default/2020-05-13-190832' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[FAILED]
Restore done.
Connection to 127.0.0.1 closed.

Vagrant 目前不受支持,但以下是一些建议。:wink:

我不太确定 rsync 的具体工作方式。路径是否需要以斜杠结尾?如果文件已复制到正确的目录,请确保服务器有权限读取该文件。

4 个赞

那么 vagrant(VirtualBox)会导致一些问题吗?

~/infra/discourse on  master! ⌚ 23:22:23
$ tree discourse/shared 
discourse/shared
└── standalone
    └── backups
        └── default
            └── icontest-2020-05-12-033823-v20200506044956.tar.gz

3 directories, 1 file

~/infra/discourse on  master! ⌚ 23:22:27
$ ls discourse/shared/standalone/backups/default 
icontest-2020-05-12-033823-v20200506044956.tar.gz

~/infra/discourse on  master! ⌚ 23:22:36
$ ls -l discourse/shared/standalone/backups/default
total 381620
-rw-r--r-- 1 pihentagy pihentagy 390774609 May 13 21:08 icontest-2020-05-12-033823-v20200506044956.tar.gz

看起来该文件对所有用户都是可读的,并且已放置在正确的位置。在 discourse Docker 容器内部,我应该在哪里看到备份?我的无人值守恢复脚本是否正确,还是应该将其复制到 Docker 容器内部?如果是,应该复制到哪里?(是否有关于如何在 Docker 外部自动化 Discourse 的指南?)

+ vagrant ssh -c 'sudo docker exec -w /var/www/discourse -i discourse discourse restore'
You must provide a filename to restore. Did you mean one of the following?

Connection to 127.0.0.1 closed.
+ vagrant ssh -c 'sudo docker exec -w /var/www/discourse -i discourse discourse restore icontest-2020-05-12-033823-v20200506044956.tar.gz'
Starting restore: icontest-2020-05-12-033823-v20200506044956.tar.gz
[

这不是必须的,但不以斜杠结尾是不良做法,因为结果取决于目标目录是否已存在。

如果目标目录已存在,则不需要斜杠,文件会被复制到该目录内。
如果目标目录不存在且末尾没有斜杠,文件会被复制为一个名为“default”的文件。
如果目标目录不存在且末尾有斜杠,则会创建该目录,并将文件复制进去。

在这种情况下,复制似乎成功了(纯属侥幸)。

然而,

由于“Did you mean one of the following?”之后没有任何建议,说明文件不在正确的位置。看起来磁盘挂载到 Docker 容器的方式有误。

你可以尝试在 Docker 内部启动备份(discourse backup),然后查看它在主机文件系统中的位置。

3 个赞

奇怪的是,在主机文件系统中看不到它。这正常吗?

vagrant@ubuntu-xenial:~$ sudo docker exec -w /var/www/discourse -i discourse discourse backup
正在启动备份...
[已启动]
'system' 已启动备份!
将备份标记为运行中...
确保 '/var/www/discourse/tmp/backups/default/2020-05-14-121930' 存在...
确保 '/var/www/discourse/public/backups/default' 存在...
更新元数据...
暂停 sidekiq...
等待 sidekiq 完成正在运行的任务...
转储数据库的公共架构...

……
大量的 pg_dump 输出……

恢复 sidekiq...
完成备份...
创建归档:discourse-2020-05-14-121930-v20200512064023.tar.gz
确保归档文件尚不存在...
创建空归档...
归档数据转储...
归档上传文件...
删除临时目录 '/var/www/discourse/tmp/backups/default/2020-05-14-121930'...
正在对归档文件进行 gzip 压缩,这可能需要一些时间...
执行备份的 after_create_hook...
删除旧备份...
清理杂物...
删除 '.tar' 残留文件...
将备份标记为已完成...
刷新磁盘统计信息...
完成!
[成功]
备份完成。
输出文件位于:/var/www/discourse/public/backups/default/discourse-2020-05-14-121930-v20200512064023.tar.gz

vagrant@ubuntu-xenial:~$ find / -name discourse-2020-05-14-121930-v20200512064023.tar.gz 2>/dev/null
vagrant@ubuntu-xenial:~$ sudo docer exec -w /var/www/discourse -i discourse discourse enable_restore
sudo: docer: 未找到命令
vagrant@ubuntu-xenial:~$ sudo docker exec -w /var/www/discourse -i discourse discourse enable_restore
现在允许恢复。使用 `disable_restore` 可禁用恢复。
vagrant@ubuntu-xenial:~$ sudo docker exec -w /var/www/discourse -i discourse discourse restore
您必须提供要恢复的文件名。您的意思是以下之一吗?

discourse restore discourse-2020-05-14-121930-v20200512064023.tar.gz
discourse restore discourse-2020-05-14-121710-v20200512064023.tar.gz
discourse restore discourse-2020-05-14-120436-v20200512064023.tar.gz

顺便问一下:有什么方法可以在代码块中高亮显示行吗?

```bash

这里的默认值是文本。我相信新安装的默认值是“自动检测语言”,

我的意思是“看,第五行很重要!”,也就是高亮显示特定的行。

通常,/var/discourse/shared/standalone/backups 是容器内可见的目录,其路径为 /var/www/discourse/public/backups(因此名为 shared)。您的 rsync 命令将备份放入该目录,以便在容器内可访问。

反之,如果容器写入 public/backups,则备份应在主机的共享目录中可见。

我上面写的是 /var/discourse/shared...。但看起来您正在 ~/infra/discourse 中操作,因此您正在将文件复制到 ~/infra/discourse/shared/standalone/backups/default

通常容器会映射到 /var/discourse/shared/...

这可能就是问题所在。您能检查一下是否存在 /var/discourse/shared 吗?

不行。在代码块中无法实现这一点,因为所有内容都是逐字呈现的。

好吧,我现在刚做了一个备份,并检查了它是否能从 Docker 机器外部找到,结果发现不行。

vagrant@ubuntu-xenial:~$ sudo docker inspect -f "{{.Mounts}}" discourse
[{bind  /var/discourse/shared/standalone /shared   true rprivate} {bind  /var/discourse/shared/standalone/log/var-log /var/log   true rprivate}]

确实如此,不过我暂时忽略了这个情况。顺便一提,那个 rsync 操作是在运行 Discourse 的(Vagrant)机器外部完成的。

但按照你的建议,我现在做了以下操作:

  • 通过 vagrant ssh 进入该机器,然后在内部:
    • 使用 sudo docker exec -w /var/www/discourse -i discourse discourse backup 执行备份
    • 注意到文件路径:
      输出文件位于:/var/www/discourse/public/backups/default/discourse-2020-05-14-125606-v20200512064023.tar.gz
      
    • 在整个 Vagrant 机器中搜索该特定文件,但未找到
      find / -name discourse-2020-05-14-125606-v20200512064023.tar.gz 2>/dev/null
      
    • 但如果我进入 Docker 容器,文件就在那里
      root@ubuntu-xenial-discourse:/var/www/discourse/public/backups/default# ls
      discourse-2020-05-14-120436-v20200512064023.tar.gz  discourse-2020-05-14-121930-v20200512064023.tar.gz
      discourse-2020-05-14-121710-v20200512064023.tar.gz  discourse-2020-05-14-125606-v20200512064023.tar.gz
      

所以我的问题是:如果我创建了一个备份,我应该能从 Docker 外部看到它吗?

与此同时,我刚刚创建了一个 Vagrant 机器,并在标准的 /var/discourse 目录内执行了 git clone。唯一的“奇怪”之处是,我在容器内有一个 discourse.yml 文件,而不是 app.yml

是的,这和你最初遇到的问题是一样的:
“如果你在 Docker 外部有一个备份,并将其放入共享目录中,你应该能在 Docker 内部看到它”(但你看不到)。

问题出在你的共享目录上,原因如下:

我的意思是:目前我重新创建了一台全新的 Vagrant 虚拟机,没有复制任何之前的备份。完成了引导并启动了 Docker 容器,然后执行了备份。

在 Docker 容器之外,Vagrant 虚拟机中看不到任何内容。

我想我已经找到了问题所在:我在外层的 Vagrant 虚拟机中挂载了这个 Docker 共享文件夹。

config.vm.synced_folder "discourse/", "/var/discourse"

如果我在 Vagrantfile 中注释掉这一行,备份就会“神奇地”出现。

所以问题在于 Vagrant 共享文件夹(向上)和 Docker 共享文件夹(向下)发生了冲突,导致路径无效。

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.