YouTube URIs fail to render thumbnails when oneboxed

I’ve confirmed that:

  1. https://youtube.com/shorts/Cs3sTnLO6EE

  2. https://www.youtube.com/watch?v=Cs3sTnLO6EE&lc=Ugyi1VtrCVd2POTc82N4AaABAg#:~:text=A%20Screencast%20Of%20LinkedIn%20Persona%20Verification%20Failure

…become a transparent box:

The DOM shows:

<img src="" width="480" height="360" title=" - YouTube" style="aspect-ratio: 480 / 360;">

However, they render when posted:


  1. bugs.kde.org/show_bug.cgi?id=511477#c1 ↩︎ ↩︎

1 Like

Confirmed, nice catch, going to put a pr-welcome on this for now and leave a note for the team to triage.

1 Like

This is an interesting one, I can reproduce the issues with the links in the OP here on Meta, but locally and on our internal Discourse site they are fetched and rendered fine:

@rokejulianlockhart are these links not rendering properly on your own site, or just not here on Meta?

1 Like

@martin, some others, too:

Non-Shorts

With https://www.youtube.com/watch?v=Cs3sTnLO6EE&lc=Ugyi1VtrCVd2POTc82N4AaABAg#:~:text=A%20Screencast%20Of%20LinkedIn%20Persona%20Verification%20Failure, this reproduces at:

  1. discuss.kde.org/new-topic

    [1]

  2. forum.fairphone.com/new-topic

    [1:1]

However, at discussion.fedoraproject.org, it renders:

[1:2]

Shorts

With https://youtube.com/shorts/Cs3sTnLO6EE, the undermentioned is reproduced:

…at:

  1. discuss.kde.org/new-topic

    [1:3]

  2. forum.fairphone.com/new-topic

    [1:4]

However, as before, at discussion.fedoraproject.org, it renders:

[1:5]

What are Fedora doing so correctly?


  1. bugs.kde.org/show_bug.cgi?id=511477#c1 ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

2 Likes

Is this issue solved, I opened a topic for issue similar currently going in my Board

@hameedacpa, this is completely irrelevant to that, because this is about the previewer. Additionally, you can confirm that it’s not been remediated by merely utilising the previewer. I suggest that you install a translator.

1 Like

I found this topic including partial issue that I’m facing too

Actually all Youtube videos not working like before and my forums is not having a customization as it is based on the original board

1 Like

@hameedacpa, thanks. That is similar, although I doubt equivalent. The presence of an invalid thumbnail icon appears to demonstrate that, alongside the different aspect ratio of the <img. If you put those details in 387672/1, they’ll be of use there. If they’re merely browser-caused differences, it’ll be triaged as a partial duplicate of this.

On that note, include your browser version, Discourse version, and the actual markup of the rendered content in that cited thread. Currently, it’s devoid of much except a screenshot, and spread over immediately consecutive posts; you need to put the effort in to make it presentable, to receive unpaid assistance.

Something is messing up in our Onebox flow here, I think to do with redirection/FinalDestination. If I use curl like so:

curl -L https://youtube.com/shorts/Cs3sTnLO6EE

I am able to find the title and other meta tags in the response:

curl -L https://youtube.com/shorts/Cs3sTnLO6EE | htmlq 'head > meta'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0
100 947221   0 947221   0     0 406174     0  --:--:--  0:00:02 --:--:-- 504109
<meta content="IE=edge" http-equiv="X-UA-Compatible">
<meta content="ApvK67ociHgr2egd6c2ZjrfPuRs8BHcvSggogIOPQNH7GJ3cVlyJ1NOq/COCdj0+zxskqHt9HgLLETc8qqD+vwsAAABteyJvcmlnaW4iOiJodHRwczovL3lvdXR1YmUuY29tOjQ0MyIsImZlYXR1cmUiOiJQcml2YWN5U2FuZGJveEFkc0FQSXMiLCJleHBpcnkiOjE2OTUxNjc5OTksImlzU3ViZG9tYWluIjp0cnVlfQ==" http-equiv="origin-trial">
<meta content="rgba(255, 255, 255, 0.98)" name="theme-color">
<meta content="A Screencast Of LinkedIn Persona Verification Failure" name="title">
<meta content="For https://www.linkedin.com/help/linkedin/cases/73171318#:~:text=Thanks%20for%20contacting%20us%20about,to%20troubleshoot%20any%20additional%20causes." name="description">
...
 curl -L https://youtube.com/shorts/Cs3sTnLO6EE | htmlq 'head > title'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0
100 992110   0 992110   0     0 445530     0  --:--:--  0:00:02 --:--:-- 739941
<title>A Screencast Of LinkedIn Persona Verification Failure - YouTube</title>

However when I get the response via our oneboxer code, these are the only tags (except script + style ones) I get in the <head> :

uri = FinalDestination.new("https://youtube.com/shorts/Cs3sTnLO6EE", Oneboxer.get_final_destination_options("https://youtube.com/shorts/Cs3sTnLO6EE")).resolve
doc2 = Onebox::Helpers.fetch_response(uri)
Nokogiri.HTML(doc2).css("head").children.each do |headel|
  next if headel.name == "script" || headel.name == "style"
  puts headel.to_s
end; nil;

<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta http-equiv="origin-trial" content="ApvK67ociHgr2egd6c2ZjrfPuRs8BHcvSggogIOPQNH7GJ3cVlyJ1NOq/COCdj0+zxskqHt9HgLLETc8qqD+vwsAAABteyJvcmlnaW4iOiJodHRwczovL3lvdXR1YmUuY29tOjQ0MyIsImZlYXR1cmUiOiJQcml2YWN5U2FuZGJveEFkc0FQSXMiLCJleHBpcnkiOjE2OTUxNjc5OTksImlzU3ViZG9tYWluIjp0cnVlfQ==">
<link rel="shortcut icon" href="https://www.youtube.com/s/desktop/ace6261e/img/favicon.ico" type="image/x-icon">
<link rel="icon" href="https://www.youtube.com/s/desktop/ace6261e/img/favicon_32x32.png" sizes="32x32">
<link rel="icon" href="https://www.youtube.com/s/desktop/ace6261e/img/favicon_48x48.png" sizes="48x48">
<link rel="icon" href="https://www.youtube.com/s/desktop/ace6261e/img/favicon_96x96.png" sizes="96x96">
<link rel="icon" href="https://www.youtube.com/s/desktop/ace6261e/img/favicon_144x144.png" sizes="144x144">
<link rel="stylesheet" href="//fonts.googleapis.com/css2?family=Roboto:wght@300;400;500;700&amp;family=YouTube+Sans:wght@300..900&amp;display=swap" nonce="kFtYVVw9wWKkoPdOJkO9xQ">
<link rel="stylesheet" href="/s/player/65578ad1/www-player.css" nonce="kFtYVVw9wWKkoPdOJkO9xQ">
<link rel="stylesheet" href="https://www.youtube.com/s/desktop/ace6261e/cssbin/www-main-desktop-player-skeleton.css" nonce="kFtYVVw9wWKkoPdOJkO9xQ">
<link rel="stylesheet" href="https://www.youtube.com/s/desktop/ace6261e/cssbin/www-onepick.css" nonce="kFtYVVw9wWKkoPdOJkO9xQ">
<link rel="stylesheet" href="https://www.youtube.com/s/_/ytmainappweb/_/ss/k=ytmainappweb.kevlar_base.dsnGl9m3_bM.L.X.O/am=AAAgAAgk/d=0/rs=AGKMywEVyAGSU99VwQpoLFio5FrCvZ1WpA" nonce="kFtYVVw9wWKkoPdOJkO9xQ">
<meta name="theme-color" content="rgba(255, 255, 255, 0.98)">
<link rel="search" type="application/opensearchdescription+xml" href="https://www.youtube.com/opensearch?locale=en_US" title="YouTube">
<link rel="manifest" href="/manifest.webmanifest" crossorigin="use-credentials">
<link rel="canonical" href="undefined">
<link rel="alternate" media="handheld" href="https://m.youtube.com/shorts/Cs3sTnLO6EE">
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.youtube.com/shorts/Cs3sTnLO6EE">
<title> - YouTube</title>
<meta name="title" content="">
<meta name="description" content="Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.">
<meta name="keywords" content="video, sharing, camera phone, video phone, free, upload">
<link rel="alternate" href="android-app://com.google.android.youtube/http/www.youtube.com/shorts/Cs3sTnLO6EE">
<link rel="alternate" href="ios-app://544007664/vnd.youtube/www.youtube.com/shorts/Cs3sTnLO6EE">

I thought maybe this ignore_redirects was causing it, but if I modify the FD options it made no difference:

I suspect YouTube is probably cutting down on scraping because they are having the same AI scraper problem as everyone else, but not sure what specifically is causing us to get the blank response (that looks like it gets meta/title tags populated via JS the same as if you visit in the browser).

This code here for youtube oneboxing is what is expecting the title + image tags to be populated:

Will keep investigating :eyes:

2 Likes

Could be that they are only shipping what looks necessary to save bandwidth? Our Oneboxes do use a browser user agent, so maybe they just honour that and say: “Cool. You don’t need OpenGraph then”?

2 Likes

Okay I think I have a fix here, we should use oEmbed for this stuff from YouTube instead:

I don’t think so, it seems like 2 things:

  1. Some sort of weird IP rate limiting/redirection, because sometimes for different sites on our hosting the OP link works
  2. This code no longer working, because the JSON that we expect is no longer there discourse/lib/onebox/engine/youtube_onebox.rb at e144ec07c8c17ad0f73427524bccc10c074d5a19 · discourse/discourse · GitHub

Overall the oEmbed URL seems safer, for the OP video it’s https://www.youtube.com/oembed?url=https://youtube.com/shorts/Cs3sTnLO6EE&format=json and you get this kind of response:

2 Likes

Okay with that fix merged, this is working now :slight_smile:

Note that onebox URL responses are cached for 1 day, I had to do this in the console Oneboxer.preview(“https://youtube.com/shorts/Cs3sTnLO6EE”, invalidate_oneboxes: true) to see the new result.

3 Likes

This topic was automatically closed after 2 days. New replies are no longer allowed.