Uploads disappeared


(Michael - DiscourseHosting.com) #1

One of our forums had some (11) uploaded images that disappeared yesterday.
No trace of this in the logs, a gap in the uploads table ids (155-165), and empty directories in uploads.
All affected directories have a modify date of Nov 23, 11:00 .

drwxr-xr-x  2 discourse discourse 4096 Nov 17 13:14 154
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 155
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 156
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 157
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 158
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 159
drwxr-xr-x  2 discourse discourse 4096 Oct 19 19:00 16
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 160
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 161
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 162
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 163
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 164
drwxr-xr-x  2 discourse discourse 4096 Nov 23 11:00 165
drwxr-xr-x  2 discourse discourse 4096 Nov 24 12:15 166

root# ls -l 165
total 0

Forum software was updated that day at 9:38. At 10:58, image #164 was still there.

2013-11-23T08:04:08+00:00|226684|0.643|200|GET  uploads/db5290/164/b9102eac4e86ab9b.png 
2013-11-23T10:58:19+00:00|226684|2.189|200|GET /uploads/db5290/164/b9102eac4e86ab9b.png 
2013-11-23T15:22:04+00:00|373|0.031|404|GET /uploads/db5290/164/b9102eac4e86ab9b.png 
2013-11-23T17:47:13+00:00|373|0.021|404|GET /uploads/db5290/164/b9102eac4e86ab9b.png 

Does anyone have any idea what could have happened here?


(Sam Saffron) #2

Is:

clean_up_uploads
 Remove orphaned uploads to prevent illegal hosting. WARNING: you might want to make a backup of your /uploads directory before enabled this setting.

checked?

cc @zogstrip


(Steven Webb) #3

Were any user accounts destroyed?

There used to be a dependent destroy on user uploads. See:

http://meta.discourse.org/t/why-does-the-user-model-dependent-destroy-uploads/10951/3


(Michael - DiscourseHosting.com) #4

Yes, that is checked.

@double_u as far as I know: no.


(Sam Saffron) #5

@zogstrip any ideas, how do we diagnose? We need some grace period to revert mistakes here. Eg: tombstone images 4 weeks later, purge.

@michaeld recommend you uncheck that for now


(Régis Hanol) #6

Were these images actually used in a post?


(Kasper Peulen) #7

If by any chance this is my forum, I wouldn’t mind if images got deleted if they are not used anymore in a post.


(Michael - DiscourseHosting.com) #8

@zogstrip, yes, they were used in a (few) post(s).

@kasper, no, it wasn’t your forum :smile:


(Régis Hanol) #9

Just pushed a safety net by adding a tombstone for orphaned uploads

https://github.com/discourse/discourse/commit/52160179f8f2711f00b6ea3106aa91184c0de3ea

Every day, we’ll purge files that are older than 30 days (by default) from this tombstone.


(Jeff Atwood) #10