Hi! Rebaking all posts is sometimes a recommended action for various reasons. After my own experience of a rebake experience that raised issues and seen/foresee other potential others, I’d like to know how they can be avoided. If I rebake my 2M posts, it will trigger too many requests to Youtube and my IP will be blacklisted, preventing Discourse from generating previews. If oneboxes (with titles, thumbnails, excerpts copied in Discourse database - cooked field) original URLs went to be broken or redirected, it seems that the oneboxes will break and that we will lose this information. I have old Facebook and Instagram oneboxes that display nicely. Nowadays, they don’t (they display in an iframe, see this commit). Additional info: Instagram embedding features and layout (note that the layout has changed a bit since then, and the iframes now are cropping content) and Facebook oneboxes not working on my forums (it seems even Discord can’t have “trusted” IP…? It’s a public link) I decided to drop the support of Facebook (and thus Instagram to my knowledge) on my forums for several reasons. If I rebake all my posts, I suppose that every link that was previously properly oneboxed will break. Is that right? To circumvent certain of these issues, "Onebox Assistant", crawl for those previews reliably! could help, but not for all. For example, it couldn’t help me with Facebook previews. What can be done to preserve existing oneboxed information during a rebake?

[image]What rebake is required for the new autocomplete styling? Is there a pattern perhaps we could use to rebake all posts except the ones with an embed/onebox? It feels to me that we need an enhancement to rebake, to be more careful rate limit on selected sites or perhaps on all sites inherit the original onebox if the refetch fails for any reason In other words, I think we need a non-damaging rebake, at least as a selectable option. (There will be some discourse communities who value an up to date 404, or who don’t value old posts at all, but there will also be communities which very much want to preserve old threads intact.)

Could this maybe be reworded and recategorised as a feature request?

That’s a good thought - it’s more robust, and less likely to fall foul of rate-limiting trouble. Leave all oneboxes alone unless specified.

[image] Coin-coin le Canapin: If I rebake my 2M posts, it will trigger too many requests to Youtube and my IP will be blacklisted , preventing Discourse from generating previews. Thanks for posting that. It could impact something I’m working on. Do you have any idea how many requests it takes to trigger the limit? Looking at the YouTube API docs, it seems that they allow up to 10k GET requests per 24 hours, but this is for requests made with an API key: YouTube Data API Overview | Google for Developers. It’s not clear to me how rate limits for unauthenticated API requests to pull in the video preview images are rate limited. I don’t think there is a rate limit applied to videos being played via an iframe, but it would be good to confirm that: YouTube Player API Reference for iframe Embeds | YouTube IFrame Player API | Google for Developers.

[image] Simon Cossar: Do you have any idea how many requests it takes to trigger the limit? I don’t know. I resolved the issue by using Onebox Assistant without any API. Just the plugin enabled. No idea how it resolved my problem. I don’t know either if it would work nowadays. [image] "Onebox Assistant", crawl for those previews reliably! Plugin Not related, but I used the plugin when rebaking all my posts after a huge import (1,6 M posts), including posts containing Youtube videos. Youtube ended by blacklisting my IP after too much embedding. Installing and enabling Onebox Assistant resolved this issue, and I didn’t need it after the rebake. I disabled it but didn’t remove it (just in case, you know). It was perfect for this task.

How to avoid potential issues when rebaking all posts?

Product Support

Stephen (Stephen) January 3, 2023, 4:46pm 4

Is there any value at all in automatically re-fetching the contents of a tweet? I would be fine with rebake skipping fetching one boxed content again unless a box were ticked.

Topic		Replies	Views
Generate onebox previews for links in imported posts Migration onebox	6	948	March 12, 2023
Youtube embeds missing Product Support	50	4942	September 25, 2021
Youtube videos onebox embedding stopped working Product Support	49	7646	April 5, 2021
Questions about rebake Product Support	3	1261	September 23, 2020
What rebake is required for the new autocomplete styling? Product Support hashtag-autocomplete	19	1930	March 12, 2023

How to avoid potential issues when rebaking all posts?

Related topics