Running launcher rebuild wiped my DB

Hi, I was met with this same page when trying to do the one-click browser upgrade:

So I followed instructions only to have the discourse DB wiped out (in the only container available, as the commands wiped the old container) and be met by this:

I know, I know, backup things before updating and everything, but really!?
Fortunately, there wasn’t much material, but I’d love to hear that the launcher made a backup of the DB somewhere so I can restore everything.

How was this install built?

If you followed the standard install your data including backups still exists at /var/discourse/shared.

There’s nothing at /var/discourse/shared. The original installation was done following the official guide, on an Ubuntu 17.10 machine from Azure.

Are you saying that there’s no /var/discourse? Or no shared?

Even with a new install the shared folder will exist.

The dir exists alright, but there’s nothing in there (except for a hidden .gitkeep).

Do you still have the console output from your install?

If you mean the rebuild from some minutes ago, most of it (it’s really long and it surpassed my terminal’s line capacity). I’ll send it to you right now.

Did you do the install a long time ago and Discourse is in /opt/discourse?

No, everything is under /var/discourse, and /var/discourse/shared exists, but there’s nothing in the latter. And it does work, the new container was up after the rebuild and my app.yml was untouched, so the forum came right up but with everything gone. So I entered the docker image, checked postgres and the discourse DB had been wipe clean (or replaced, most likely, I’d have to check tomorrow at work).

The original install was done in mid July, and I’ve had no problem doing the updates from the UI every couple of weeks or so, though I’ve never updated the image in the way I did today. I know I should’ve made a backup before, but it really is almost a placeholder forum for a service that’s in a kind of beta state, so I just went with it without thinking much. I mention it because I only lost a couple dozen users, categories, settings and just a couple of topics, so it isn’t that bad, but I really didn’t expect the process to go on wiping everything. That just sounds mad.

We’ve never seen this happen, so it’s likely something unusual occurred on your end.

I’ve done hundreds of rebuilds on over a hundred servers and have never lost data.

I’d offer to see if I could see what happened for a fee, but it doesn’t sound like it’s worth it.

1 Like

Thanks, but you’re right, it isn’t.

Do you only have “app.yml” inside /var/discourse/containers/ ?

That’s right, and in case you ask, I don’t recall if there was anything else in there before pulling and rebuilding.

Oh! One thing you can do is run history at the bash prompt to see if something else got run.

1 Like

I know about history, I’m not new to Linux (despite what my failure to backup the forum before updating may suggest :sweat_smile:), but the server was configured to clear history on every session. Don’t mean to be annoying here, just wanted to clear out basics.

Anyway, at least manually, I’m sure I didn’t run anything besides git pull and ./launcher rebuild app, both with sudo (as opposed to be running as root). I didn’t get any prompt or error message in the whole process either.

It’s probably tough to diagnose this as there’s no real log (unless someone points me to where the rebuild logs its output, if it does) other than ~4000 lines I got to rescue, most of them concerning migrations, managing assets and the final boilerplate, which i sent to Stephen. Also, as no one mentioned any DB backup other than the one that should be at /var/discourse/shared (which I don’t have), there’s probably no restoring in my case.

So this is more of a heads up: yes, it could have been something at my end, but it doesn’t hurt to report that the rebuild effectively left me with a fresh DB and lost stuff in case there’s a bug. So feel free to close this whenever you like.

2 Likes

Running those as sudo isn’t necessary, but there aren’t any issues I know of that would cause this kind of loss.

I’m curious, the .gitkeep in shared, what’s the timestamp on it?

Both shared and shared/.gitkeep have timestamp Jul 11 22:14 (so does samples, scripts, containers and bin), while others such as launcher, image, discourse-setup, etc., have yesterday’s timestamp, i.e. Dec 4 15:53. So maybe it was a bad install from the start, though I don’t recall having any trouble following the install guide, and I only got to see the issue when doing the update.

The team can confirm this, but I’m pretty confident no code exists which can eradicate stuff from shared.

Your data is still likely to be on the server, just not at the default path. It’s seen in some very old installs and also when installing using a third party package. It’s also not unheard of when Discourse was inadvertently installed at a different path.

If you docker container ls you only see the new and currently running container listening on :80 and :443?

Yeah, there’s only that one container running:

CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                  NAMES
d87e24dfa0eb        local_discourse/app   "/sbin/boot"        25 hours ago        Up 25 hours         0.0.0.0:8090->80/tcp   app

And these are the available images:

REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
local_discourse/app   latest              53b50c3cafcb        25 hours ago        2.32GB
discourse/base        2.0.20181031        ea31cd77735a        5 weeks ago         1.88GB
<none>                <none>              0038d6156b21        4 months ago        2.89GB
discourse/base        2.0.20180613        97bd31fab216        5 months ago        1.76GB