Hi,
Since the latest Discourse 2.5.0 version, I have trouble with thumbnails generation.
It seems thumbnails are now generated from core, and it resulted in all thumbnails being wiped.
I tried various operations to put them back and I’m going to explain situations where it doesn’t work.
Note: there is probably a good reason for that change, but that would be really welcomed to not introduce sudden behavior change breaking functionality without having a way to be prepared, with an upgrade guide and/or a way to opt in this change, please.
Context
- Discourse 2.5.0 beta4 (faeb5793ba)
- Topic List Preview plugin 4.4.0
- WP-Discourse Posts published at it is (full HTML) on Discourse topics (first message).
About a post content, this is an example (formatted for you):
Content
Image HTML as code for convenience:
<img
width="150"
height="84"
src="https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-150x84.jpg"
class="attachment-thumbnail size-thumbnail"
alt=""
srcset="
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-150x84.jpg 150w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-300x169.jpg 300w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-1200x675.jpg 1200w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-1536x864.jpg 1536w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-2048x1152.jpg 2048w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-788x443.jpg 788w,
https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית.jpg 1280w
"
sizes="(max-width: 150px) 100vw, 150px"
/>
<div data-wp>
<a
href="https://www.banggood.com/Xiaomi-Redmi-Router-AC2100-2033Mbps-2_4G-5G-Dual-Band-Wireless-Router-6High-Gain-Antennas-128MB-OpenWRT-WiFi-Router-p-1614038.html"
target="_blank"
><img src="https://zuzu.deals/wp-content/uploads/2020/01/5e3128b4e5da7-150x150.jpg"/>
</a>
<div>
<div data-buy>
<a
href="https://www.banggood.com/Xiaomi-Redmi-Router-AC2100-2033Mbps-2_4G-5G-Dual-Band-Wireless-Router-6High-Gain-Antennas-128MB-OpenWRT-WiFi-Router-p-1614038.html"
target="_blank">קנייה
</a>
<span data-clipboard-text="BG38b2ac" data-coupon>BG38b2ac</span><i></i>
</div>
<div data-price>$43.99</div>
</div>
</div>
<hr />
<p>
<small>
פורסם ב: <a href="https://zuzu.deals/%d7%a7%d7%95%d7%a4%d7%95%d7%9f-%d7%91%d7%9c%d7%a2%d7%93%d7%99-%d7%a8%d7%90%d7%95%d7%98%d7%a8-%d7%97%d7%96%d7%a7-%d7%95%d7%97%d7%93%d7%a9-%d7%a9%d7%9c-%d7%a9%d7%99%d7%90%d7%95%d7%9e%d7%99-xiaomi-re-2/"></a>
</small>
</p>
<br />
<p>נעים להכיר!</p>
Before Discourse update
TLP worked always well on our external images, whatever on topic creation or edition.
However, we had the download remote images to local
option disabled because of markdown issue.
Markdown rendering issue
This not the main issue, and it still happens after Discourse upgrade, here some explanation.
When discourse downloads and replaces an image HTML with its markdown syntax – in our context, this results as:
[...]<a href="<link_here>" target="_blank"></a>[...]
Discourse is unable to render the image
Image
To fix the issue, you need at least an empty line above:
[...]<a href="<link_here>" target="_blank">
</a>[...]
Image
Would it be possible to allow rendering markdown image surrounded by HTML, please?
After Discourse update
- All thumbnails got deleted.
- All images are well displayed in the topic content
- Because of this feature (https://github.com/discourse/discourse/commit/03818e642a1ae871bffdc0c39c10f05f0b8b0398, I think), we were forced to enable
download remote images to local
option as it would appear this no more possible to get a thumbnail from external link… -
download remote images max days old
is set to the max, 10000.
From this, I was told to rebake all posts to get the image downloaded. And it’s where it gets weird.
-
rake posts:rebake
did not have much effect (at least not on the first message of a topic, but it did trigger a lot ofPullHotlinkedImages
)
1.1. Looking at some topics, I thought that the image HTML withclass
orsrcset
attributes were the cause, so I tried to normalize all the images with the following code (don’t know ruby) – It helped for some topics.
1.2. However due to the markdown issue, I had to add newlines to fix. – At least for those topics, thumbnail worked.
Code
Post.where(post_number: 1)
.where("raw LIKE '%<img%/>%'")
.each
do |post|
post.raw.gsub!(/<img[^>]+(src="[^"]+")[^>]+\/>/, "<img \\1 />")
post.save!(validate: true)
post.rebake!
end
Post
.where(post_number: 1)
.where("raw LIKE '%upload://%'").each
do |post|
post.raw.gsub!(/(!\[.*upload:\/\/.*\))/, "\n\n\\1")
post.save!(validate: false);
post.rebake!
end
- Editing manually (without changing content, just saving) an ignored topic works most of the time. Image is downloaded.
2.1. Some topic are ignored even after an edit. I can seePullHotlinkedImages
is triggered but no images are downloaded. (like with this html<img src="https://zuzu.reviews/wp-content/uploads/2020/05/HiZERO-VS-BISSEL-VS-שואב-אלחוטי-VS-שואב-רובוטי-VS-מגב-ודלי-VS-מטאטא-VS-ספונגה-חשמלית-150x84.jpg" />
, link) - After normalizing, and seeing that an edit can help to download the image, I tried to
rake posts:rebake
multiple times – without effects. - Then I tried to use the rails console similar to the code above but with specific topic id and with only
post.rebake!
– without effects - Not all images from a topic are always downloaded…
- @Canapin points me to Download remote images from older posts? - #3 by vinothkannans ; trying to rebake all posts now. – Did not help unfortunately
It’s just crazy why some images works, others not. I don’t think it’s a criteria issue. Image settings are high. I really don’t understand what is the logic behind, this seems random.
Currently, we have still a lot of missing thumbnails. Likely most of them can be fixed manually with editing/saving, but that’s not feasible. I’m doing that for a client, and I have lost already a lot of time trying to fix the issue.
I don’t mind having upload images as thumbnail, but:
- Can you tell me if there are specific reasons why images won’t download? Are there settings which can help? Do we need something? How to debug?
- Is there a way to force download them similar to editing/saving through the console?
- Can you allow markdown image rendering if surrounded by HTML?
Hopefully I was enough precise in the problem description.
Thanks in advance for any help and solution.