My group was running our Discourse on a temporary domain for two months until we agreed on our actual domain name. Yesterday I attempted to migrate all content from the old domain to the new one. All text content, user accounts, and even inter-thread links transferred correctly. But now: All past image embeds appear to be lost. No new images can be uploaded. This was my process: Generated a new Discourse app on a new DigitalOcean droplet. Connected the new domain name to that new droplet. Confirmed both Discourse apps and all plugs were up to date with latest available software versions. Put the old Discourse into read-only mode, to prevent additions of any new content. Ran a backup of the old Discourse. Uploaded the backup to the new Discourse. Updated the Discourse email address from the old domain’s email to the new domain’s email. Ran a test of notifications, and they worked for the new Discourse. Clicked through all Discourse settings to update any mentions of old domain to new domain. Changed the old subdomain to redirect to its proper domain, and temporarily added a note and link to the new Discourse there. As written above, most content appeared to transfer flawlessly. But now, a day later, we’ve noticed old image embeds are lost, and no new images can be uploaded. Only their “alt” content appears. Screenshot of an example follows. [Screen Shot 2020-10-25 at 09.20.39] In googling, it appears there are several lengthy threads about this, but I didn’t see one that includes a change in domain name and an inability to re-upload. I’ve attempted just now to resolve this by: SSH into the machine. CD in the Discourse directory and enter the app. Run rake posts:missing_uploads. This reports: Looking for missing uploads on: default 0 post uploads are missing. Run rake uploads:missing. This reports a long list: /var/www/discourse/public/uploads/default/original/1X/bbc547e72f080561282be277749165709cbb0983.ico /var/www/discourse/public/uploads/default/original/1X/0a421ccd1a08047895e2355f44c332f8b069107d.jpeg /var/www/discourse/public/uploads/default/original/1X/034e0353b7558a26252c82982de53002fda0a33f.jpeg […] /var/www/discourse/public/uploads/default/original/1X/f7a6164ffa55af4ee2706d2386227183ef6c2d61.png 96 of 281 uploads are missing /var/www/discourse/public/uploads/default/optimized/1X/997bc5536763d84a8d035ff7becd98277a158680_2_45x45.png […] /var/www/discourse/public/uploads/default/optimized/1X/8944afba36549c9050ef074b391625ef93d4d0e3_2_1035x582.jpeg /var/www/discourse/public/uploads/default/optimized/1X/8944afba36549c9050ef074b391625ef93d4d0e3_2_10x10.png 247 of 761 optimized_images are missing Run rake uploads:recover_from_tombstone. This outputs nothing. I admit I don’t know what those Rake commands are doing. I can also see within the containers/app.yml file that DISCOURSE_HOSTNAME is the correct (new) subdomain and domain. Running ./launcher rebuild app appears to change nothing. Can anyone help, please? Thank you.

My very basic understand of Discourse (and I am very much a novice) is that uploaded images are stored on your site, and an optimised file size version of the image are also stored. Initially, the original image will be displayed to users, and at some point, the forum switches over to an optimized image (to me that appeared to be after 24 hours). I experimented a little bit with the settings as I was concerned about running out of storage. On your admin dashboard, check them out (Under Files section). There are various settings to control image size, and dimensions. This might be related to your rogue image problem (if I understand it correctly). For example, your reformated image still falls outside the criteria. My issue when I transferred hosts from Digital Ocean To Hetzner was that I had some images (and avatars) that simply were not appearing (similar to your first screenshot). My solution was FTP move them (crude but it worked). Not 100% sure why. At first I moved only the optimized images and it didn’t fix the problem, but when I moved all the images it resolved my issue. Apologies about folder mistake I was doing it from memory. The folder structure is: VAR/Discours/Shared/Standalone/Upload Its very easy to login with an FTP client with Digital ocean Host ; IP address of Discourse Port : 22 Username: root Password: your SSH password As I said I am far from an expert, more someone that has tinkered with it the last couple of months. There are people here that can offer better advice (and insight). But if all else fails it might be worth giving my approach a go.

Have you tried a rebake? cd /var/discourse ./launcher enter app rake posts:rebake [image] Rebake all posts? Self-hosting If the URL to your site changed, yes, it’s a good idea to rebake.

[image] omarfilip: Have you tried a rebake? Thanks for the suggestion. I have input your commands just now, and I see no difference. Old images are still not seen. New images still fail to upload. The web browser console throws a “GET 404” error upon completion of the upload attempt, by the way.

Update: I don’t know which command exactly improved the situation, but it’s improved yet not resolved. I can create and upload new files of type TXT, DOCX, XLSX, JPG, and PNG. Most (but not all) old uploads of all types are now displaying and downloading properly. The rogue uploads were displaying/downloading fine before without complaint. And they refuse to show after re-upload in new test posts. As an example: This image: https://www.frontiersin.org/files/Articles/470644/fphys-10-00944-HTML-r1/image_m/fphys-10-00944-t001.jpg Displays properly (in my Discourse) when entered a URL on its own line, as below here: [image] But it disappears from the post after I finish post and refresh the page, leaving what appears to be only a single blank line. Inspecting the source reveals an image element, with a SRC attr set to a path on my domain structured as https://discourse.my_domain_name.org/uploads/default/original/1X/61ae2bdeff3dfe334ad6803409560b667d7dc246.jpeg. And that path opened in a new tab loads an NGINX 404 page. When I download it from its original source to my laptop, and I upload to my Discourse (as I’ll do here in this Discourse below), it fails on my Discourse. [fphys-10-00944-t001] On my Discourse, it looks like this: [Screen Shot 2020-10-25 at 12.03.48] And if I control-click that failed image and click “Open Image in New Tab”, it loads an NGINX 404 page. I see in this composition dialog right now that Discourse changes the image extension from JPG to JPEG, but I don’t think that would matter. If I open the downloaded image in Affinity Photo app, export it with no compression as a new JPG, and re-upload it to my Discourse, it displays fine on my Discourse. This would suggest to me the problem is still with my Discourse, not with the uploaded files.

I came across this problem during recent transfer. The easiest non tech way is resolving this, was logging on to you original site/host with FileZilla, navigate to VAR/Discourse/Uploads Under here you will find subfolders with images. I simply downloaded them from original host and uploaded them to the new one. An quicker alternative way is to use this method of backup. It’s manual process but worked well also. [image] Manually create and restore Discourse backups Sysadmins As discussed below, a warning for newbies: this is an advanced method! there is an automated, web UI way to restore from backups at /admin/backups which is so much easier. with the method below, no automatic backup is kept of the /var/discourse/containers/app.yml file which contains essential SMTP credentials and SSL customizations. Keep a copy of this offsite whenever you change it. If you are using digital ocean, keep frequent snapshots of your droplets - preferably before each time you run .…

[image] Alec: The easiest non tech way is resolving this, was logging on to you original site/host with FileZilla, navigate to VAR/Discourse/Uploads Thanks for replying. I’ve never used FileZilla app. I use Transmit app for FTP. But I don’t know how to connect to my Discourse app on my DigitalOcean droplet using FTP. I can SSH into my droplet, and I can navigate to /var/discourse/, but there’s no uploads directory there. Even if I were manually to move image files from the old location to the new location, how would that address the failure of re-uploading the same images?

My solution was FTP move them (crude but it worked). Not 100% sure why. At first I moved only the optimized images and it didn’t fix the problem, but when I moved all the images it resolved my issue. Thanks for the more detail instruction on how to FTP into my two Discourses. The uploads file was about 125 MB on the old Discourse, 60 MB on the new one. So I copied the old one to my laptop’s Desktop. Then I copied its contents, folder-by-folder, to the new Discourse, skipping any duplications. And to my surprise, it appears to have solved the problem. All image uploads appear to be fixed, both in threads that pre-date my Discourse migration, and in threads that I created today in troubleshooting. I might theorize that Discourse was re-using pre-existing pointers to content that somehow was lost in the move. So if I re-uploaded the exact same image file, it would re-use a broken pointer and repeatedly fail. But when I resaved it as a new file, it succeeded because it stored a new copy with a new pointer. Perhaps. Thanks again very much.

By default backups do not contain optimised images as they can be recreated. Including them would waste bandwidth and disk space. In your case it would have helped to include them by enabling the include_thumbnails_in_backups site setting before creating the backup. As an alternative, running rake posts:rebake_uncooked_posts after the restore would have worked as well. Otherwise the optimised images are regenerated by a background job, but that takes some time… There’s a log message at the end of the restore which tells you exactly that.

Images lost in backup/restore, change of domain, can't upload new

Product Support

omarfilip October 25, 2020, 5:57pm 2

Have you tried a rebake?

cd /var/discourse
./launcher enter app
rake posts:rebake

Topic		Replies	Views
How can I migrate files from the old upload scheme (default/XXX) to the new one (default/original/{1,2}X)? Product Support	17	3604	May 17, 2019
Old image uploads become broken images Product Support	63	9622	May 24, 2021
Hosted to self-hosted migration: past uploads still reference discourse infra Product Support	15	914	October 16, 2023
How to find any missing images? Product Support	61	8293	July 27, 2019
Images prior 2022 lost Product Support	6	733	February 15, 2023

Images lost in backup/restore, change of domain, can't upload new

Related topics