We had a server crash and had to create a new server and restore from a backup. It was a harrowing process, details for what happened and how we restored it here. Now after restoring (the key was to disable S3 uploads), all the links to attachments in posts are broken (404 error). I’ve searched the forum and can’t find a solution and hoping someone can point me in the right direction. I have two options Can I fix these broken short-url links which link to attachments embedded in posts (all the broken links are for attachments in posts; embedded images render fine, other internal link work fine)? For example, the URL to the attachment on a post on the forum shows as https://XYZ.com/uploads/short-url/phu1HOLvkE8LWpkKYfnMPSWsvHh.zip This is what I’m seeing in the logs when I click on an attachment link in a post (which leads to a 404). Message (5 copies reported) Failed to process hijacked response correctly : Errno::ENOENT : No such file or directory @ rb_sysopen - /XXXXX.s3.dualstack.us-east-1.amazonaws.com/optimized/1X/46728e07f9819907d1b18387bf02ea7fc25c7981_2_32x32.ico Backtrace /var/www/discourse/app/controllers/static_controller.rb:160:in read' /var/www/discourse/app/controllers/static_controller.rb:160:in block (2 levels) in favicon’ /var/www/discourse/lib/distributed_memoizer.rb:16:in block in memoize' /var/www/discourse/lib/distributed_mutex.rb:33:in block in synchronize’ /var/www/discourse/lib/distributed_mutex.rb:29:in synchronize' /var/www/discourse/lib/distributed_mutex.rb:29:in synchronize’ /var/www/discourse/lib/distributed_mutex.rb:14:in synchronize' /var/www/discourse/lib/distributed_memoizer.rb:12:in memoize’ /var/www/discourse/app/controllers/static_controller.rb:138:in block in favicon' /var/www/discourse/lib/hijack.rb:56:in instance_eval’ I’m really hoping there’s a way to fix these short-url links after disabling the S3 upload option while restoring the server from a backup. A post re-bake did not fix it. If for someone reason this is a dead-end and cannot be fixed en mass, now I’ve had thousands of orphaned attachments on the S3 cloud, is there a way to clean them up and free up the space? Is there a way Discourse can go through it’s S3 upload bucket and clear out all orphaned assets?

It might be, or might have been, possible to figure out how to fix up those links, but figuring out how is beyond the scope of what’s feasible in a forum. Upload.sha1_from_short_url('phu1HOLvkE8LWpkKYfnMPSWsvHh.zip') => "b13050bdcd2d58924ba6ab3e7608b16bfc3cd1b7" See if you have a file called b13050bdcd2d58924ba6ab3e7608b16bfc3cd1b7.zip somewhere in your uploads and/or s3 bucket. If so, then it should be possible, though not easy, to fix things up. Since you didn’t include the actual forum or bucket names, we can’t tell here.

Yes, I did find that under: original/2X/b/b13050bdcd2d58924ba6ab3e7608b16bfc3cd1b7.zip Happy to PM you the links/details What’s very funny is that just the attachments (like files) are broken. Any embedded images display just fine.

[image] RBoy: Yes, I did find tha Then the stuff is all there, you just need to re-write the posts somehow. Not sure why it’s not working, but the files are there.

[image] Jay Pfaffman: you just need to re-write the posts somehow Is there a way to run this en mass from the console or the UI or “download” these files from S3 to local?

I believe so, but I think that someone who’s familiar with discourse and rails will need to write it. I’m not aware of an existing solution to your problem. There are a few topics about moving between S3 buckets that might offer some clues, but I don’t think that your particular problem has been solved before.

[image] RBoy: fix these short-url links after disabling the S3 upload option while restoring the server from a backup After restoring, you should have re-enabled the S3 upload option. It sounds like you did not do that?

And it’s hard to do since it got set on the app.yml that might do it.

Would that fix the issue? Just wondering it’s safe to try.

Well, yes, it’s what I recommended you to do in the first place… If you don’t re-enable S3 uploads then the short-url function will look for those uploads locally, but they are on S3.

I’ll try it but on the other hand when I enabled it it broke the restore (see my other topic). With Jays help I had to figure out how to disable the upload to eventually restore it. Have you been able to successfully restore a server with the option enabled?

Fixing or cleaning up broken links and assets after a restore

Product Support

RGJ (Richard - Communiteq) June 14, 2022, 1:15am 12

You need to

Disable the setting
Restore
Enable the setting

As I described in our PM exchange last week

Topic		Replies	Views
Help restoring - system hung at midnight Self-hosting	26	2927	June 13, 2022
Images after a restore don't have S3 bucket URL Product Support s3 , hosting	11	443	June 12, 2026
S3 assets broken after moving forum Product Support	10	1138	September 29, 2021
ToS3MigrationError: uploads not migrated to S3, leading to broken attachments. How to fix this error? Product Support	2	884	July 13, 2021
Can't restore a backup if it includes link to Discourse onebox Bug	0	136	October 2, 2024

Fixing or cleaning up broken links and assets after a restore

Related topics