Missing subfolder images after update (again and again)

(Andy Rothfusz) #1

All of my site images are missing again. This seems to happen almost every time I update. Avatars, uploads, all gone.

Now when I re-upload the same image again, I see that it gets assigned the same hash, and the image is still missing/not showing.

Is there a way to get a list of all the uploads the database thinks are on the site, mapped to the original file names and the pages which refer to them? Is there a way to check for dead images? I can’t tell if the database is broken, the hashing algorithm is broken, or if something is deleting files from the shared directory.

It feels like I have no way to manage the uploads, to see when links and images are dead, or to debug why images disappear. Its extremely frustrating as I’ve been trying to add more images to my site.

Edit Added my subfolder.yml used in ./launcher rebuild subfolder – took out passwords etc.
subfolder-cleaned.yml (3.2 KB)

(Felix Freiberger) #2

Are you running a normal Docker install?
Is there anything special about your installation?

(Andy Rothfusz) #3

I’m running in a subdirectory with a normal single container docker install. There is an nginx proxy in front of the installation.

Thanks for starting to think through it. I’d be happy for any tips, but what I was really looking for were some debugging tools. I want to debug behavior, not code. I’m not a ruby guy or a postgre guy. I’d like an interface for seeing what has been uploaded and whether or not it is still available, when it was last accessed, so I can narrow down what’s gone missing and what might have caused it.

(That said, I can read Ruby and run PostGRE commands, but if the software is going to manage all these uploads for me, then it really needs to provide me with an interface to make it less mysterious)

(Sam Saffron) #4

Nothing deletes files from the uploads directory, you should be safe there.

What does stick out to me here is that you are using a subfolder. This is one of the rougher edges we have, we only have 3 enterprise customers running that setup and always try to avoid it because there is always yet another edge case we keep hitting.

./launcher enter app
% rails c
% Upload.first 

Will give you the first upload with a bunch of info about where it is.

Uploads are all stored in /var/www/discourse/public/uploads this is symlinked to /shared/uploads which is volume on the host.

(Jeff Atwood) #5

In the interests of avoiding pain, I recommend not running in a subfolder unless it is a life-critical requirement that you do.

(Andy Rothfusz) #6

Thanks Jeff – I thought the bugs had been shaken-out. It has been working pretty well except these image disappearances.

(Jeff Atwood) #7

Sure @zogstrip can have a look. But subfolder is never going to be a path of least resistance.

(Andy Rothfusz) #8

Note that it is not the path to the images that is broken (which was an early problem with subfolders). The images are not in the shared directory anymore, at least not by the hash names they’ve been given (e.g. https://partners.example.com/forum/uploads/default/original/1X/dd75cf607631895ccfb2d7f64d571f3d67dcffb5.jpg)

That image in particular is one that was working fine before the update. Now it is just a broken image. If I edit the post and re-upload the image, I get the same hash again, but the image is still not found. I don’t see it anywhere in /shared.

Could the original subfolder.yml I was using have killed the files? There were some rm statements in it (see Uploads broken after moving to subfolder)

(Sam Saffron) #9

Have a search for images on the host in /var/discourse/shared/ ? anything there?

(Andy Rothfusz) #10

I’ve checked – there is no file named dd75cf607631895ccfb2d7f64d571f3d67dcffb5.jpg anywhere on the host, even after re-uploading. I can upload different files successfully, but because this image had been uploaded before, I don’t seem to be able to upload new copies of it. And I don’t know any way to remove it from the database.

(Andy Rothfusz) #11

I’m going to try to reproduce this behavior in a fresh installation of Discourse and see what I can learn that way. Thank you all for your help and debugging ideas.

I still think that if Discourse is going to manage all the uploads for me (storing them, pruning unreferenced uploads, creating thumbnails, etc) then a tool to inspect what is where and if any links are dead would be helpful in tracking down what rebuilds or configuration changes cause problems.

It could even become a search/browse feature. A gallery of all the uploaded images with links to the pages that show them. Some people remember things visually…

Thanks again – let’s put this on the back burner until I can narrow down the problem.

(Justin Gordon) #12

I just updated my installation for TalkSurf.com

And we lost at least 4 images.

This is the upgrade that could not be done inside the web tool. Maybe we were low on disk space?

Any way to troubleshoot what went wrong?

Any way to recover the lost images.

What seems odd is that only a handful of somewhat older images were lost. Not the oldest, and not the newest.