After doing some experiments, I found that some of my posts don’t display the images even before doing any ‘remap’ operation.
Rather they display a small icon of the image, which shows the image path if I hover my mouse over it and shows the image only when clicked on it.
And that icon too disappears (and white space/image place holder starts to appear in its place), if I ‘Rebuild Html’ from the 3-dots menu, or even if I do any ‘rebake’ operation from inside the container.
Kindly do guide pls.
If you haven’t done a remap and your S3 bucket is still functional, nothing should have changed from before. Where those images working before you started on this road?
In theory you’ll only lose the connection to those image files if you turn off your S3 bucket or remap incorrectly.
Thanks.
I’ve found that the images icon show the path of /bucket/uploads/optimized/folder/… (and since there is no such image at this path, the image doesn’t show, but only icon does).
But when I click on that image icon, the image is shown/served from ‘Orig’ folder, i.e. /bucket/uploads/original/…
I don’t know how the single image may have 2 diff paths stored for it?!!
Anyways, now the problem has come to, as to how do I find the posts which have images mapped to wrong path (under ‘optimized’). So that I could correct/remap their addresses to correct path under ‘Original’.
Thanks @nathank, @Pravi , @itsbhanusharma Since I confused diff problems/scenes, currently I stand here now:
Some of my posts don’t show the images uploaded to them, rather they either show a small icon with a wrong bucket url/address when I hover mouse on them. Or, some of them show correct image and correct bucket path ONLY WHEN I click on those small icons. And yet some other posts’ images don’t show anything at all, just a blank white space.
And if I ran a remap operation (remap wrongbucketurl correctbucketurl) then I found in one sampled post that even though for once the small icon was replaced by correct image (I became happy), but the following day, that image disappeared completely, no icon even. And just a white space appeared in its place. So I had to restore my website to the previous day.
When I ran this command the result was this:
# rake posts:missing_uploads
Looking for missing uploads on: default
19 post uploads are missing.
19 uploads are missing.
6 of 7792 posts are affected.
You have not yet migrated to local. I have gone through your site and the s3 URL is still there.
The command is something like this,
./launcher enter app
discourse remap //bxyzbucket1.s3.dualstack.ap-south-1.amazonaws.com/uploads/ /uploads/default/
discourse remap //bhdisco.s3.dualstack.ap-south-1.amazonaws.com/uploads/ /uploads/default/
rake posts:rebake
I’ve run variations of this process a few dozen times now and it’s still not working.
Our images aren’t displaying using S3 because management have decided they don’t want an S3 bucket with public access so we need to move back to being local.
Initially I’ve inspected the broken links and from that I’ve managed to figure out the values I need to use in the remap (I thought), and a bunch of images work now. But most, I’d say over 90%, don’t work. But unlike when they have broken S3 links and you can at least inspect them to start to figure out what is going wrong, all I am getting is this:
Does anyone know what might be causing these images to fail? I’ve been stuck on this for days now. I find it incredible that a) there is no way to migrate back and b) someone (not me) migrated us when there is no way to migrate back.
To be clear, I’ve followed the process described above by @nathank. I’ve tried many times, generally slight variations of the paths in the remap step because its not clear to me if those instructions are universal or depend on how your directory looks (ours has a bunch of subdirectories that have successfully copied from S3 using the S3 sync step).
I’d really appreciate some help as I’m about ready to tear my hair out.
I think that what you want to do is to turn on the hidden include_s3_uploads_in_backups and make a backup, then restore it (I’d do it first on a new test server). This is what happens when a CDCK customer cancels their subscription, and those backups restore to a new self-hosted site without any problems.
Hi Jay - thanks for the response, and forgive my ignorance here but how would I go about turning that on?
Sorry. Something like
./launcher enter app
rails c
SiteSettings.include_s3_uploads_in_backups=true
exit
exit
I tried doing that but got:
NoMethodError: undefined method
include_s3_uploads_in_backups=' for SiteSettings:Module from (pry):1:inpry’
Then realised maybe I had to turn S3 uploads back on because I had them turned off, but still got the same error.
Oh we use two containers and I am running this in web_only. Rails command isn’t found in the data container so I assume that’s the correct approach.
I seems that command should have been:
SiteSetting.include_s3_uploads_in_backups=true
I ran that and then created a new backup. Restored from that backup and there is no change - most of the images still display the broken icon from above. I tried rebuilding both containers too but this didn’t make a difference.
When I download and inspect the backup zip, all of those files are definitely there, and post restore they are visible in the filesystem. But Discourse just refuses to recognise and display them.
For the record I eventually got this to work. I started fresh (i.e. from a snapshot of my instance) and I’m fairly certain the process that worked in the end was:
SiteSetting.include_s3_uploads_in_backups=true
discourse remap to update references to my various S3 file locations to a local locationThanks @pfaffman for pointing me in the right direction here.
EDIT
I might as well raise this. After my previous post, I realised that six of our topics still have broken images (though the vast majority are OK now).
They are our six oldest posts and all the original images had a different S3 URL than all the others. Clearly this isn’t coincidence. So I checked that all those files are in the uploads/default/original/1X directory, and they all are. Then I ran a remap command using this unique S3 URL - and that appeared to edit the right number of posts. Then I rebaked and rebuilt the containers, but these topics are still broken. Does anyone have any idea why a small number would fail like this?
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.