tl;dr I’d like to add that we’re experiencing what seems to be the same problem here. If there is a rate-limit issue due to some recent change then I think other users will start to experience this during migration, re-baking of posts or perhaps just down to a really busy forum. The fact that onebox seemingly fails silently means that these issues aren’t visible until users start to complain that YouTube oneboxes are missing.
Background
We’re on 2.6.0.beta 1
Users were getting messages about non-secure content. On investigation, Chrome seemed to be complaining about images linked from HTTP sites. So I configured Discourse to download all images/media and serve them over HTTPS.
Once I’d changed the setting, this meant doing a re-bake of historic posts. Since that rebake, a large chunk of YouTube videos that were previously onebox’d have now changed back to the linked URL
We have one thread of 10,000 posts that consists of solely YouTube video replies and all posts are URLs and not oneboxes.
During the re-bake, all queued jobs processed organically so it’s not jobs stuck in a queue of deleted jobs.
I haven’t seen the same error messages that @marcozambi described, but I believe we are tripping a rate limit too.
What have I tried?
In support of this rate-limit theory, a small piece of code I wrote to re-bake posts worked (onebox’d) for the first 80+ YouTube videos in a thread then failed to convert the remaining videos.
At that point, even editing the post, making a small amendment and resaving did not force the URL to be onebox ‘expanded’. At the same time, all queues were empty or had minor jobs being instantly processed as I would expect.
Attempts to re-run that code over a 30 minute period failed to force the oneboxing of the links. I don’t think 80 is a magic number here, just what was available from the quota we had.
@marcozambi mentioned that the /embed/ format YouTube link worked when others failed, so I amended the code to use a regex search-and-replace of YouTube links to turn them into the /embed/ format.
The code worked.
Re-running the code to just rebake the posts again failed to turn into onebox representations.
My plan is to experiment with a task that converts all YouTube links in the large thread to the /embed/ YouTube format. If that fails or we trip a higher rate limit, then I’ll take a look at @merefield’s Onebox Assistant.
I’ll post an update later.