Youtube videos onebox embedding stopped working

support
21

I’d love some advice on what to do here, but also have some ideas about how this might be better handled.

What might be happening

One theory is that our server may have been identified by YouTube as potentially farming music videos and we’re getting limited / blocked.

We’re a really unremarkable little forum in the UK with meagre traffic, but we have a couple of threads (actually one split in two due to size) of 10K + 2K posts of music videos. It’s a musical chain where the next poster simply posts a song related, often in some tangential way, to the previous post.

We have other threads with YouTube links, of course, but this one is particularly (~100%) dense with music.

Following a rebake at the weekend, I’m guessing that YouTube looked at the activity of the Oneboxer trying to grab headers for lots of music videos and their algorithm put us on the naughty step.

I subsequently, re-tried to bake posts and that has presumably confirmed YouTube’s suspicions that all we do from this IP address is attempt to download music videos.

Might be Related to Digital Ocean

So Googling 429 errors on YouTube, and ignoring all of the YouTube API results, eventually points to a group of similar sounding issues that all say that their IP addresses have been grey-listed/black-listed/banned as a result of being on virtual servers - Digital Ocean’s name was specifically mentioned.

The suggestion is that YouTube may ‘ban’ a series of IP addresses and that other, legitimate servers, are often pulled in as collateral damage.

Here is one such problem and potential solution.

https://support.google.com/youtube/thread/21697789?hl=en

and here is another from a developer at medium.com who works on the embed.ly API.

https://support.google.com/youtube/thread/21939228?hl=en

The suggested resolution in the first post was to disable ip6 on the droplet.

Does anyone know if that sounds likely, or if it would cause any issues?

I’ve held off doing this at the moment so I can collect any other data that might be needed to aid resolution.

What should happen?

Firstly, I understand the reaction here is likely to be that my experience is an edge-case and that no change is needed.

I can understand that. But if @marcozambi and I have found the same issue in a relatively short period of time then it suggests that it could be something that others will see. Perhaps YouTube has recently become more officious in its policing of embedding?

I’d ask you to remember that at the moment my forum is in complete limbo. I have tens of thousands of links that have failed to be Oneboxed, but I don’t know where they are and I can only think of a total rebake to find them all which will almost certainly get more negative attention from YouTube.

At the very least, Onebox cannot fail silently when it hits a rate-limit error. It has to bring it to the attention of admins and, where possible, pause the process that tripped the rate-limit issue.

Options for bringing this to the attention of the forum owner

I don’t think this is easy. At all.

I see Onebox as playing a number of distinctly different roles

a) it builds Oneboxes in real-time as posts are composed,
b) it responds to rebakes of old posts (with often already known links) in the background and headless
c) it is asked to handle masses of background job posts during a migration / rebake

If a rate-limit error were to occur during scenario a) then the author would recognise that the Onebox hadn’t expanded and may complain to the forum admins - it might be noticed. I guess it’s also possible to grab the single error during composition and push an error or message to admins?

In scenario b) the failure may happen during background processing of the post and therefore it would need to be either retried or if a retry count had been reached it may need to be failed and the forum admins notified.

In scenario c) the wider context of lots of failures happening at the same time would need to be known. At the moment I don’t believe there is any overarching control of the post rebake process such that 10,000 429 errors coming back from Onebox might be able to recognised as a bigger problem than sending 10,000 individual messages to admins.

In reality, a failure such as that would mean that all calls to Onebox for YouTube (might be other providers as well) video expansion would need to be put on hold until we’re no longer rate-limited.

Alternative approaches

Rate limit outgoing requests

I guess the prevention-is-better-than-the-cure axiom might dictate that a Discourse rate-limiter could be put in front of the requests so that Discourse would then hold requests back on some settings-configured basis.

This would play havoc with re-bakes and migrations.

Onebox Assistant

Perhaps we all need commercial relationships with organisations that cache and proxy embeds such as the one used by @merefield in his Onebox Assistant plugin?

Whitelist the 'bot

Could we perhaps find a way to whitelist the Discourse 'bot with YouTube? Probably too open to abuse.

YouTube API

Just like the Onebox does with Twitter, perhaps we could use the YouTube API if the rate limits are greater?

4 Likes
22

Some good research there, thank you.

I don’t know whether you noticed, but I raised that even Signal Messenger has problems with YouTube and it’s still an open issue:

https://meta.discourse.org/t/onebox-assistant-crawl-for-those-previews-reliably/107405/9?u=merefield

It also happens on whatsapp.

3 Likes
23

I just wanted to add that we get regularly “banned” by YT every 2 to 3 weeks, we stay banned for a week or so, and then we get un-banned. Then the cycle repeats.
We have only a few hundreds of links to YT video, our users post no more than a handful per day…

4 Likes
24

So without using the OneBox Assistant plugin, there is no solution to preview the youtu.be links in the base discourse?

25

All formats of YouTube links should be expanded by Onebox.

Is it not working for you?

If not, what problems are you seeing?

26

I’m on the latest discourse and I just made a post with a youtu.be link and it didn’t show the preview for it. Regular youtube links work.

27

Let’s see

https://youtu.be/gLCduDJVksc looks fine to me

28

Yes it works on meta.discourse. I’m trying to figure out why it doesn’t on mine. Is there any settings to configure this from admin panel?

29

Youtube has likely banned your server or IP range, as discussed above.

30

But regular youtube links work. If they banned, wouldn’t all of them not work?

31

Not the case, @AntiMetaman.

As @codinghorror suggested above, YouTube appears to rate-limit / ban certain formats of YouTube links work where others continue to work.

In my case the ‘/embed’ format of link works but others don’t.

It looks like YouTube provides rate limits that are different per format. The ‘/embed’ one seems to allow for greater numbers. That’s purely anecdotal and I haven’t got any hard data on that.

A few questions

Are you hosting on Digital Ocean?

Are you running IPv6?

Have you checked that the video id is correctly formed - 11 characters? (stupid question - but you’d be suprised

For a given YouTube video have you tried each of the three supported formats? - i.e.

Which worked and which didn’t?

I have banged my head against this problem for weeks but managed to get a workable solution - but I really don’t understand why it works.

Assuming that you’re seeing the same problem try installing the Onebox Assistant, then enable it but don’t configure it.

I found I needed to bounce the server after enabling it to ensure it worked.

In this configuration - which really should not work at all - my otherwise rate-limited YouTube videos (in any format) get expanded by Onebox - but I have no idea how this works.

I have been discussing the issue with @merefield (the author) here…

https://meta.discourse.org/t/onebox-assistant-crawl-for-those-previews-reliably/107405/36?u=bletch

IF, and it’s a big IF, this also works for you, it’s not a proper fix as it’s just a quirk of method return values and shouldn’t be relied on long-term.

Separately, I’ve also used the Onebox Assistant plugin as it was intended - by subscribing to embed.rocks and it works like a charm.

3 Likes
32

Can you define regular and non-regular YouTube links please.

33

If that is the case, why can’t Onebox convert into the one format that works before polling the data?

34

It could, I guess, but it’s not a ‘fix’.

All of the formats appear to be rate-limited and eventually fail. Even the ‘/embed’ format failed for me after I tried to re-bake all of the YouTube embeds that had failed during a previous re-bake.

Plus, I’ve only seen two experiences shared on here where the ‘/embed’ route worked where others failed - including my own.

There’s not enough evidence yet, nor enough ‘sufferers’ yet to suggest making a change.

Are you experiencing the problem too, @Terrapop - or are you just an interested party?

35

I’m an interested party as we move our popular community to Discourse soon. Just staging at the moment, it works for now, but we want to have a save setup right from the start, thus we will probably setup with Onebox Assistant to our own endpoint which will relay and cache responses from embeds.rock or if this fails, we will pull from Iframely.

1 Like
36

Ok, so I was just playing around and I was able to reproduce it here on meta.discourse! I believe it has something to do with markdown. My youtu.be links do in fact work my forum as intended just not in the examples below. Youtube isn’t blocking my host at all.

  1. This won’t work even on meta.discourse: - YouTube

See? There’s nothing wrong with the video link. If you put the link on the next line, it still won’t work.

  1. This won’t work either:
    - YouTube

You see? Now if I paste this same link here, it will work:

37

Glad you’re not beling blocked by YouTube. I think everything is working as it should - it’s a formatting issue.

I believe that Onebox expanding needs the link to be a) at the beginning of a line on its own and b) if it is in the middle of a list it needs an additital return / linefeed before it.

If you look at the raw version of your post

https://meta.discourse.org/raw/153748/36

Ok, so I was just playing around and I was able to reproduce it here on meta.discourse! I believe it has something to do with markdown. My youtu.be links do in fact work my forum as intended just not in the examples below. Youtube isn't blocking my host at all.

1. This won't work even on meta.discourse: https://youtu.be/MLpyi-oVoI

See? There's nothing wrong with the video link. If you put the link on the next line, it still won't work.

2. This won't work either:
https://youtu.be/MLpyi-oVoIY

You see? Now if I paste this same link here, it will work:

https://youtu.be/MLpyi-oVoIY

You can see that the first item 1. - the link appears in the middle of the line so it will not expand. The link must be the first thing on the line.

And in item 2. there is only one return / linefeed before the YouTube link so it won’t expand either.

If you look at the formatted version of your post you might be able to see that the YouTube link has a hanging indent…

image
image1402×580 83.4 KB

…which means that markdown is still treating the link as being part of bullet point 2 so it won’t expand it.

You could do this.

  1. This is some filler text
  2. This link will work as I’ve left two return / linefeeds after this line and before the YouTube link
3 Likes
38

Yea I’ll just have to learn to work with markdown and put youtube links on a separate line. My users are still getting used to it and it’s a post made by a user that caught my attention where their youtu.be link didn’t work which is what led to all this.

1 Like
39

Hi, I started rebaking 1M+ posts 2 days ago, and youtube previews don’t work anymore since today, so I guess my IP was banned.
What is advised in this case? Should I just wait for the rebake to finish and contact youtube to see if I can be unbanned?
Or use "Onebox Assistant", crawl for those previews reliably!?

40

It is certainly a way if you can afford it.

1 Like