Restoring lost posts from mails

AI-generated summary

A catastrophic failure of the Discourse VM hypervisor resulted in data loss, but a backup was successfully restored. To recover lost posts, gwmngilfen considered importing data from users’ mailing list archives. fefrei suggested either manually re-posting a few lost posts using the change ownership tool or building a custom tool using the API for a large number of posts.

gwmngilfen implemented a solution by importing the archived emails into a staff-only category using the mbox importer and then manually cleaned them up. However, this process had limitations, including all posts being assigned to the same user (<noreply@your.discourse.site>) and difficulty handling posts that should be part of an existing thread. Despite the challenges, gwmngilfen found that the moderation tools, particularly Change Ownership and Merge to Topic, made the process more manageable.

We recently had a catastrophic failure of the hypervisor running or Discourse VM, and had to restore from a previous backup. Huge props to the people who on that, it was trivial to restore the site.

My question is about the data lost in the intervening time. Many of my users have mailing list mode enabled, and as such have a complete archive of the lost data. Is it possible to import that data in some way? have some vague ideas about dumping the data to disk in appropriate maildir / mbox format and using the importers… thoughts?

If it’s just a few posts, you can manually re-post them and use the change ownership tool to assign these to the right users.

If you have too much posts for this to be feasible, building a custom tool to do so using the API might be your only option.

Update: so I did this in a fairly horrible manner. I used the mbox importer to import them into a new staff-only category, and then went through them and cleaned them up by hand. Painful, but at 142 mails, not too big a job.

Biggest pain point is that using the mailing list mode archive means every email is <noreply@your.discourse.site> so all the posts get assigned to the same user. It’s also a pain because it (obviously) can’t handle posts that should be part of an existing thread.

Overall, it’s been a nasty job, but the excellent moderation tools (Change Ownership and Merge to Topic especially) make it fairly straightforward.