Uploaded files mysteriously going missing


(David Scharf) #1

On our forum at https://discourse.vvvv.org/ we have uploaded files and images disappearing after some time. The images turn up broken (example ) and the file links go into nowhere (example ). I looks like somehow the physical copy of the file get’s deleted after a certain time, but I can’t see a pattern right now. We’re running a standard docker installation. Is there something I might be missing here? Is there some kind of cleanup that discourse does which might be removing those files? I have seen related posts here, but none really seem to give the exact answer or explanation of what might be going on.

Thanks all for your help in advance!


(cpradio) #2

Are you running into this?


(David Scharf) #3

@cpradio no we have never rebaked the posts so far…


(cpradio) #4

Well that also happens automatically in a job (IIRC) – for example, there is a process where it comes back and downloads images to the system that were linked to external sources. So you might not be doing it automatically, but a sidekiq job will be.

There is also a sidekiq job named Jobs::CleanUpUploads


(David Scharf) #5

Ok, the setting was enabled for us. I disabled it and we’ll see what happens. Thanks for pointing me there!


(David Scharf) #6

Bonus question: I copied back the file from the tombstone directory into the default directory. The deleted files are back in default, but they don’t seem to be reachable via their public link. For example this is a dead link on our page:

https://discourse.vvvv.org/uploads/default/original/1X/19c3c4928c977ba74ccadc77f904ab1c1a1fa0d5.zip

But the file

/var/discourse/shared/standalone/uploads/default/original/1X/19c3c4928c977ba74ccadc77f904ab1c1a1fa0d5.zip

exists in the filesystem. I also did a ./launcher restart app, but the link is still not reachable.


(Régis Hanol) #7

What if you “rebuild the html” (also known as rebake) the post that contains the link to this attachment?


(David Scharf) #8

How do I do that for an individual post? Searching for this on meta tells me this option should be in the wrench menu, but there does not seem to be an entry for this (and neither do I find something in the admin section of the page).


(Régis Hanol) #9

You have to click the button under said post, then click the wrench button, then click the Rebuild HTML button :wink:


(David Scharf) #10

Weirdly I do not have that button, but I just rebuilt all the posts now, oh well :slight_smile: . The Problem with this file is still persisting though…


(OG) #11

Have you got .zip upload enabled in Settings (authorized extensions)?

What happens if you upload any .zip file through manual upload to post? Is it placed in directory correctly?

Try it and check for difference compared to your other zip file… (file system permissions?)


(David Scharf) #12

We managed to fix this with big help by @zogstrip. To me it’s not quite clear yet why it happened, but I have disabled the the cleaning up of files for now. It may have to do with a massive import of posts we did when starting of with discourse, as there was some hackery involved from my side :slight_smile: