Understanding theme blob assets behavior

[sarahann (Sarah)]

I am not sure if this is a feature of Discourse or it’s a bug, but maybe let’s try to have a discussion about it!

Prerequisites:

1. Discourse setup to use a blob storage for assets

Steps:

1. Upload “image.png” (either using the discourse theme cli or by clicking ‘add’ button under [your_theme]->uploads->add)
2. Notice that if you inspect the anchor href of your newly added asset (using dev tools) it’ll be “[your_blob]/user-uploads/original/1X/[hash].png”. This works great.
3. Delete [hash].png in the blob storage
4. Repest step 2 - re-upload your asset
5. Repeat step 3 - notice the url is the same ("[your_blob]/user-uploads/original/1X/[hash].png") and the asset is unable to be found.

I have gotten around this issue with images, I’d change the integrity of it (crop it very slightly, change opacity etc…).

But my main problem is with fonts that have been previously uploaded, I cannot just add spaces to them to get Discourse to re-generate a proper hash for the font. So essentially, because the font existed at one time and is now deleted I can’t seem to use that font again (unless I manage to change the integrity which causes the font to not work because it’s not in the right format).

This makes me curious about a few things:

1. Does discourse store asset URLs somewhere? How does it re-generate the same URL for each asset?

• Is the hash determined by hashing the asset (and would therefore always be the same as long as the asset is the same)?

2. Is there a better way to delete assets that have been uploaded to a blob? Possibly deleting it from the blob throughs Discourse’s tracking off

I’ve also noticed that when I change the name of the file in my /assets folder, then re-upload my theme (I use the discourse CLI tool) the asset continues to have the same name in [my_theme]->uploads->add.

Anyways, I was hoping someone could help me understand this a little better, cause I think I must be missing something

[zogstrip (Régis Hanol)]

@sarahann

1. Does discourse store asset URLs somewhere? How does it re-generate the same URL for each asset?

Yes, in the uploads table.

@sarahann

• Is the hash determined by hashing the asset (and would therefore always be the same as long as the asset is the same)?

The hash is the MD5 of the content of the file. So changing the name won’t do anything indeed.

@sarahann

2. Is there a better way to delete assets that have been uploaded to a blob? Possibly deleting it from the blob throughs Discourse’s tracking off

Delete them through Discourse and not directly from the blob.

@sarahann

I’ve also noticed that when I change the name of the file in my /assets folder, then re-upload my theme (I use the discourse CLI tool) the asset continues to have the same name in [my_theme]->uploads->add.

That’s because if an uploaded file has the same hash as a previously uploaded file, we’ll return the record associated to that file. The record includes the filename of the previously uploaded file.

[sarahann (Sarah)]

Thanks so much for the clarification!

I figured that Discourse worked like that but its nice to have a concrete answer!

@zogstrip

Delete them through Discourse and not directly from the blob.

How do you do this? I have a theme and am uploading assets in a /assets/images folder within my theme. The assets are then uploaded to discourse and I can view them if I go to Admin → customize → myTheme and look under the uploads section.

Pressing the ‘x’ doesnt delete the asset immediately (I think)? its flagged for deletion and deleted after some amount of time?

I just want to get the process right so that I know how to deal with this in the future

Thanks so much for you knowledge so far! Its been beyond helpful!

[zogstrip (Régis Hanol)]

@sarahann

Pressing the ‘x’ doesnt delete the asset immediately (I think)? its flagged for deletion and deleted after some amount of time?

Exactly, they’ll automatically be deleted after a few hours.

[sarahann (Sarah)]

So I dug a little deeper into why we were having an issue with our uploaded assets.

We have a development environment of Discourse, and the assets live on a development blob storage. When we were making our production environment, we backed up dev and restored it to our production environment. This way, we could have all of our settings and some forum posts.

But it looks like the uploads table gets brought over as well.
So now we have a production instance, with our uploads table filled with links that point to our development blob storage.

I can’t upload the images into production, because the hashes exist in the uploads table and the url’s will not be updated.

Any ideas on how to get around this in the future? I feel like I am missing something.

[zogstrip (Régis Hanol)]

@sarahann

Any ideas on how to get around this in the future?

If you don’t need any of the uploads from the dev instance, then I’d recommend truncating the uploads and optimzed_images table.

[sarahann (Sarah)]

We found a solution to this!

If anyone has this problem in the future, do not truncate your db tables. It happened to mess a lot of things up for us.

Our solution

1. In https://[your Discourse]/admin/customize/themes/[your theme] click the ‘x’ next to all the assets
2. Go to https://[your Discourse]/sidekiq/scheduler
3. Run the following jobs
   • Jobs::CleanUpUploads
   • Jobs::PurgeDeletedUploads
4. Re-upload your theme assets (we uploaded them manually in https://[your Discourse]/admin/customize/themes/[your theme]. Then clicked the ‘edit css/html’ button and saved some of the files with errors.

Thank you @zogstrip for all your help!

[HeyRR]

This does not work on my side, I uploaded some images to one of my customized theme component and then decide to replace them with some other images (with same variable names), but then I always get to see those old images.

Now I deleted all of them and run the scheduler, but those images are still there, can anyone help me understand why this would happen?