# Recover\_from\_tombstone insanely slow
**URL:** https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752
**Category:** Support
**Created:** [6 april 2018 om 20:12 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752 "2018-04-06T20:12:32Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [6 april 2018 om 20:12 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/1 "2018-04-06T20:12:32Z")
I’ve got a big import that’s got a bunch of tombstoned images. This happens from time to time and I’m not sure why, but that’s not why I’m here.
I’m running `rake uploads:recover_from_tombstone` and it’s taking well over a minute per upload (I think that’s what those 45471 things that it needs to process are). So, I’m looking at a month for it to finish. 👎
And the 25 that it processed appeared not to have done anything (the most recent one did reported that it was restoring an image to its proper place).
It’s a [standard install](https://meta.discourse.org/t/142537?silent=true) on an 8GB Digital Ocean “optimized” droplet.
I’ve set
```plaintext
db_shared_buffers: "2048MB"
db_work_mem: "80MB"
UNICORN_WORKERS: 4
```
At one point I tried increasing `num_connections` up to 200 (a new trick I learned recently), but that didn’t seem to help either.
```plaintext
total used free shared buff/cache available
Mem: 7.8G 1.7G 133M 1.0G 6.0G 4.8G
Swap: 2.0G 691M 1.3G
```
I’m pretty stumped.
EDIT: I think it’s just that the database is not on SSD, but whatever DO calls their extra space and it’s a huge community with a couple million posts.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [6 april 2018 om 21:29 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/3 "2018-04-06T21:29:45Z")
Possible, that job is pretty “dumb”. I’ll have a look if I can find some quick wins.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [6 april 2018 om 21:32 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/4 "2018-04-06T21:32:03Z")
Thanks, @zogstrip!
My quick solution is to rsync the images back to default out of tombstone and then rebake (I think they may need to rebake anyway). This will un-break the images in all posts immediately and the rebake will see that they don’t get tombstoned again, right?
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [6 april 2018 om 21:36 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/5 "2018-04-06T21:36:33Z")
Not sure actually. This will temporarily show the images but the rebake process won’t create the upload records, so they’ll be back in tombstone when the cleanup job runs.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [6 april 2018 om 21:39 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/6 "2018-04-06T21:39:44Z")
OK, so rsync them over so that the images aren’t broken, and THEN run the rake task to update the database.
It’s running much faster on the production server, but it still looks like 20 days to complete 430K (posts?). The site is mostly about images, so broken images is something of a problem. 🙂
This happens to me on imports a lot. I’m thinking that there must be something to do to update the database that’s not happening in the importer. I should look at what that rake task is doing…
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [7 april 2018 om 07:22 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/7 "2018-04-07T07:22:35Z")
@zogstrip how safe would the process be that Jay proposes here? Discourse is still moving images from the import into the tombstone, I seem to be at 440k now and I’m concerned that processing will take longer than 30 days - meaning images could get deleted permanently.
Also, any speedups in the task would be fantastic!
---
### Author:  [@bartv](https://meta.discourse.org/u/bartv)
#### Post date: [7 april 2018 om 11:52 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/8 "2018-04-07T11:52:51Z")
I have a related question: A little over 1600 entries have been processed now, but not a single file has been moved back from the tombstone into the uploads directory (I verified with a filecount). Does this mean it’s not working?
Edit: apparently almost all of them are in the optimized directory, not in original. Digging a little deeper.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [7 april 2018 om 12:09 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/9 "2018-04-07T12:09:08Z")
The script prints the names of the files or moves. Not all of the posts (?) it’s processing need fixing.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [15 mei 2019 om 21:09 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/10 "2019-05-15T21:09:29Z")
We’ve made some improvements to the `uploads:recover_from_tombstone` rake task.
It even has its own tested class now 😉 thanks to @tgxworld and @vinothkannans
>
---
### Author:  [@JammyDodger](https://meta.discourse.org/u/JammyDodger)
#### Post date: [25 september 2022 om 07:16 UTC](https://meta.discourse.org/t/recover-from-tombstone-insanely-slow/84752/12 "2022-09-25T07:16:40Z")