Youtube videos onebox embedding stopped working

Support
41

How to know if we’re actually IP banned or not?

I did like this:

But it downloaded the video without any issue.

root@ubuntu-8gb-hel1-4:~# youtube-dl https://www.youtube.com/watch?v=Xl-PTTeRsik
[youtube] Xl-PTTeRsik: Downloading webpage
[youtube] Xl-PTTeRsik: Downloading MPD manifest
[download] Destination: Loading cargo into HTV-9 Konutori-Xl-PTTeRsik.mp4
[download] 100% of 11.41MiB in 00:01

Maybe my problem comes from somewhere else then? :confused:

I have a 404 when Discourse tries to oneboxes a youtube link.

42

I haven’t got a lot of time to respond at the moment, but a few comments.

  1. Have all the re-bake jobs worked their way through sidekiq at the moment? Does https://yourforum/sidekiq show lots of jobs? If so, then it could just be that the Onebox-ing is taking time.

  2. Take a look at this diagnostic that I carried out and attempt to reproduce the process and report back.

  1. If YouTube banned you midway through a re-bake then you’re going to have to re-bake again as you won’t know what posts with embedded links have been processed or not. I actually wrote a utility to reformat YouTube posts and rebake in chunks.

  2. Sadly, the Oneboxer seems to fail without a message so it’s difficult to know what the cause of it is. I had to fork and modify @merefield’s plugin to log the error messages to find out that I was getting 429 errors. You’re welcome to use my fork but it’s a bit of a hack and (I’ve just noticed) it’s a couple of commits behind Roberts.

  1. If you are being rate-limited (you see 429 errors from YouTube) then I had success installing @merefield’s Onebox Assistant and enabling it but not changing any settings. i.e. leaving all settings blank. WARNING THIS IS WEIRD AND SHOULDN’T WORK BUT IT DOES.

See my posts here

https://meta.discourse.org/t/onebox-assistant-crawl-for-those-previews-reliably/107405/35?u=bletch

and here

https://meta.discourse.org/t/onebox-assistant-crawl-for-those-previews-reliably/107405/36?u=bletch

  1. Finally, to answer your questions about embed.rocks, I have signed up for an account with embed.rocks, tested it with the Onebox Assistant and it worked perfectly during a partial rebake. I have, however, since unconfigured it because it works with the weird behaviour described in 5.
1 Like
43

In a forum that has been around for a while, it’s common to have once genuine YouTube links go stale. Videos are removed due to copyright or accounts being closed and so those links don’t work. Is it possible you’re seeing this?

44

Hi,

No, the video work :frowning:

There are still 900000 of the 1.3M posts remaining to be rebaked (will take at least a week), but oneboxing was working perfectly on new posts one or two days ago.

Now, if create a new post with a youtube link, or if I rebuild the HTML of post which already contains a youtube link, this is the 404 error I get:

image
image1340×793 58.9 KB

Other oneboxes work well (tried vimeo, twitter…).

edit: re-read posts here, and it appears that I have the same issues as you guys.

When you say they removed your ban in your message, I understand that you didn’t contact them, the ban was lifted without doing anything?

I don’t really care if my old posts don’t have embedded videos, but I care about new posts.
My forum isn’t open yet, and I’d like to offer my user a good experience. :slight_smile:

edit : sorry for the multiple edit guys.
I tried the weird workaround described here: "Onebox Assistant", crawl for those previews reliably! - #35
And it does work. I guess I’ll let the “rebake all” end, and since it seems that Youtube lift IP bans automatically, I’ll just wait a few days/weeks.

2 Likes
45

Hi,
My forum stops one boxing youtube videos.
We started a topic with a lot of music videos a week ago and it seems that we have the same problem of @marcozambi oneboxing gets 404 error.
So, What should I do?

47

hello I have the same problem however :

  • my server doesn’t seem to be ip-banned : the curl request does work with no problem (also I can browse youtube from my server)

  • In my browser console I don’t have a 404 error but a 504 gateway timeout

  • also no version of the url works for me

however I have this SSL error in discourse production.log

Failed to onebox https://www.youtube.com/post/***** hostname "www.youtube.com" does not match the server certificate (OpenSSL::SSL::SSLError) ["/usr/local/lib/ruby/2.7.0/openssl/ssl.rb:395:in 'post_connection_check'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/ssl_socket.rb:146:in 'initialize'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/connection.rb:471:in 'new'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/connection.rb:471:in 'socket'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/connection.rb:118:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/middlewares/mock.rb:57:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/middlewares/instrumentor.rb:34:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/middlewares/idempotent.rb:19:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/middlewares/base.rb:22:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/middlewares/base.rb:22:in 'request_call'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon/connection.rb:283:in 'request'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/excon-0.78.1/lib/excon.rb:250:in 'head'", "/var/www/discourse/lib/final_destination.rb:186:in 'public_send'", "/var/www/discourse/lib/final_destination.rb:186:in 'resolve'", "/var/www/discourse/lib/oneboxer.rb:365:in 'block in external_onebox'", "/var/www/discourse/lib/cache.rb:94:in 'fetch'", "/var/www/discourse/lib/oneboxer.rb:358:in 'external_onebox'", "/var/www/discourse/lib/oneboxer.rb:199:in 'onebox_raw'", "/var/www/discourse/lib/oneboxer.rb:44:in 'preview'", "/var/www/discourse/app/controllers/onebox_controller.rb:27:in 'block in show'", "/var/www/discourse/lib/hijack.rb:56:in 'instance_eval'", "/var/www/discourse/lib/hijack.rb:56:in 'block in hijack'", "/var/www/discourse/lib/scheduler/defer.rb:94:in 'block in do_work'", "/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:76:in 'with_connection'", "/var/www/discourse/lib/scheduler/defer.rb:89:in 'do_work'", "/var/www/discourse/lib/scheduler/defer.rb:79:in 'block (2 levels) in start_thread'"]

I can’t understand why my server would suddenly think youtube.com certificate don’t match the url :face_with_raised_eyebrow:

49

@legrit can you share what is the URL you are trying to embed? Also try to run this on your server:

cd /var/discourse
./launcher enter app
curl -vI https://www.youtube.com

and paste the output here.

3 Likes
50

I fixed the problem

in /etc/resolv.conf in the docker container I had the ip of my server as first address and google 8.8.8.8 as second address
I removed the first address and it worked

not totally sure what was going on
youtube was the only site inaccessible from the container

1 Like
51

My Youtube embeds are failing …

image

Seems like there is redirect to “consent” page. This is working well on meta.

Anything has changed lately?

52

@Overgrow please see: Youtube embeddings have stopped working for servers in Europe

5 Likes