Unable to restore from S3 backup - Fails

[TehLeo (Chris)]

I’m unable to restore from a S3 Backup of Discourse.

Steps to reproduce:

1. Have a Backup of Discourse on S3.
2. Setup a fresh install of Discourse.
3. Create admin account.
4. Connect Discourse to S3
5. Locate the Discourse backup in my S3 backup lists
6. Enable restore
7. Attempt restore
8. Restore Fails

Log:

I’ve tried everything I could with troubleshooting S3. I even attempted to move the backup to the server, and performed a CLI / Manual restore. It fails with the same log. I’ve tried turning off S3 Uploads. I’m unsure what my next troubleshooting steps are, or how to see exactly what post is causing the issue / remove it.

Thank you for any help!

[Canapin (Coin-coin le Canapin)]

Welcome!

@TehLeo

I’ve tried everything

Does “everything” include looking at similar error logs here?

https://meta.discourse.org/search?q=%22posts%20are%20not%20remapped%22%20%22s3%20migration%20failed%22%20order%3Alatest

This result in particular:

Issues with restoring backup to discourse

This sounds like the problem mentioned in Can't restore a backup if it includes link to Discourse onebox You should be able to fix this during the restore, but you need to do it on the shell. cd /var/discourse ./launcher enter app # start the restore with the --pause argument discourse restore --pause <backup_filename> It will pause the restore process at two points: Before DB migrations are run – you can ignore that; press a key to continue Before Uploads are res…

[TehLeo (Chris)]

I followed these instructions, and I recieved 9 posts from the quarry. The old post doesn’t detail what to do with these posts, or how to correct the issue past this point.

[Canapin (Coin-coin le Canapin)]

This is what ask.discourse.org returns, I cannot guarantee it’s safe or that it even works

---

@DiscourseHelper

During a paused restore, you have an opportunity to temporarily modify or clean up problematic data such as uploads or posts in the database before Discourse proceeds with completing the restoration process. Here’s how you can handle or modify the problematic data during a paused restore step-by-step.

---

Steps to Manually Handle Problematic Data in a Paused Restore

1. Initiate the Restore with Pausing:
   Run the restore with the --pause flag to allow the restoration process to pause before uploads or remappings are applied:

   cd /var/discourse
   ./launcher enter app
   discourse restore --pause <backup_filename>
   

2. Point of Pausing:

   • The restore will pause during specific points:
     1. Before database migrations: Press a key to proceed during this point, as it doesn’t concern problematic uploads or posts.
     2. Before uploads are restored: Stop here to modify any uploads or post content-related issues. This is the critical point where you need to act.

3. Open a New SSH Session:
   While the restoration is paused, open a new SSH session to interact with the paused instance without interrupting the restore process:

   cd /var/discourse
   ./launcher enter app
   

4. Access the Rails Console:
   Enter the Rails console to interact with the database:

   rails c
   

5. Identify Problematic Posts:

   • Use SQL-like queries in Rails to identify posts that include problematic content causing issues during backup restoration. For example:

     current_db = RailsMultisite::ConnectionManagement.current_db
     cdn_path = SiteSetting.cdn_path("/uploads/#{current_db}/original").sub(/https?:/, "")
     Post.where("cooked LIKE '%#{cdn_path}%'")
     

   This will return posts where the cooked content includes problematic upload paths or onebox links.

6. Modify or Delete Problematic Posts:

   • If you locate the problematic post(s), you have two primary options:
     • Edit the Post:

       post = Post.find(ID) # Replace ID with the post's ID
       post.update(cooked: "Fixed content or placeholder text")
       

     • Delete the Post:

       post = Post.find(ID)
       post.destroy
       

7. Modify Uploads if Necessary:
   If the issue stems from specific problematic uploads:

   • Locate them using similar queries in the uploads table (accessible via Rails console).
   • Remove or correct the problematic entries:

     upload = Upload.find(ID) # Replace ID with the problematic upload's ID
     upload.destroy
     

8. Resume the Restore Process:
   After making necessary modifications, return to the paused shell session and press any key to resume the restore process.

---

Important Considerations:

• Always create a backup of the original database file before modifying any content.
• Focus on correcting minimal data—that is, only address problematic content without making unnecessary changes.
• If the upload is external (a URL or CDN-related issue), you can remove the external reference manually during restoration.

[TehLeo (Chris)]

So I was able to just delete all 9 posts using the following Rails command:
Post.find(id#).destroy

[Canapin (Coin-coin le Canapin)]

Did you run this while the restore was paused?