Trouble on adding a simple unit test for Youtube oneboxing

Hi!

I want to add support of /shorts/ Youtube link.

My modification of the YoutubeOnebox class works, but it is required that I add a test in youtube_onebox.rb and my test doesn’t work.

Similar to the “can parse youtube embed results” test, I added this code:

it "can parse youtube shorts results" do
  preview = expect(Onebox.preview('https://www.youtube.com/watch?v=wi2jAtpBl0Y').placeholder_html)
  preview.to match(/From which sound/)
  preview.to match(/hqdefault/)
end

I also added this at the beginning of the file, in before do:

stub_request(:get, "https://www.youtube.com/shorts/wi2jAtpBl0Y").to_return(status: 200, body: onebox_response("youtube-shorts"))

I have never done unit testing before, so I don’t really understand of all of this work.

My test failure:

Failures:

  1) Onebox::Engine::YoutubeOnebox can parse youtube shorts results
     Failure/Error:
               http.request(request) do |response|

                 if cookie = response.get_fields('set-cookie')
                   # HACK: If this breaks again in the future, use HTTP::CookieJar from gem 'http-cookie'
                   # See test: it "does not send cookies to the wrong domain"
                   redir_header = { 'Cookie' => cookie.join('; ') }
                 end

                 redir_header = nil unless redir_header.is_a? Hash

     WebMock::NetConnectNotAllowedError:
       Real HTTP connections are disabled. Unregistered request: GET https://www.youtube.com/embed/wi2jAtpBl0Y with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Discourse Forum Onebox v2.9.0.beta12'}

       You can stub this request with the following snippet:

       stub_request(:get, "https://www.youtube.com/embed/wi2jAtpBl0Y").
         with(
           headers: {
          'Accept'=>'*/*',
          'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
          'User-Agent'=>'Discourse Forum Onebox v2.9.0.beta12'
           }).
         to_return(status: 200, body: "", headers: {})

       registered request stubs:

       stub_request(:get, "https://www.youtube.com/shorts/wi2jAtpBl0Y")
       stub_request(:get, "https://www.youtube.com/embed/KCyIfcevExE")
       stub_request(:get, "https://www.youtube.com/playlist?list=PL5308B2E5749D1696")
       stub_request(:get, "http://www.youtube.com/user/googlechrome")
       stub_request(:get, "https://www.youtube.com/channel/UCL8ZULXASCc1I_oaOT0NaOQ")
       stub_request(:get, "https://www.youtube.com/watch?v=21Lk4YiASMo")
       stub_request(:get, "http://www.youtube.com/watch?v=21Lk4YiASMo")
       stub_request(:get, "https://www.youtube.com/embed/21Lk4YiASMo")
       stub_request(:get, "https://youtu.be/21Lk4YiASMo")
       stub_request(:get, "https://www.youtube.com/watch?feature=player_embedded&v=21Lk4YiASMo")

       ============================================================
     # ./lib/onebox/helpers.rb:83:in `block in fetch_response'
     # ./lib/onebox/helpers.rb:69:in `fetch_response'
     # ./lib/onebox/helpers.rb:28:in `fetch_html_doc'
     # ./lib/onebox/engine/youtube_onebox.rb:21:in `parse_embed_response'
     # ./lib/onebox/engine/youtube_onebox.rb:42:in `placeholder_html'
     # ./lib/onebox/preview.rb:28:in `placeholder_html'
     # ./spec/lib/onebox/engine/youtube_onebox_spec.rb:110:in `block (2 levels) in <main>'
     # ./spec/rails_helper.rb:328:in `block (2 levels) in <top (required)>'

Finished in 0.89418 seconds (files took 5.04 seconds to load)
17 examples, 1 failure

Failed examples:

rspec ./spec/lib/onebox/engine/youtube_onebox_spec.rb:109 # Onebox::Engine::YoutubeOnebox can parse youtube shorts results

Do you have any clue why it fails?

Also, what do these parts do in before do?

onebox_response("youtube-embed")

The “youtube-embed” string doesn’t seem to be used anywhere in Discourse’s code except here. What is its purpose?

Why are there twice /embed/ links and why onebox_response has a different string parameter?

stub_request(:get, "https://www.youtube.com/embed/21Lk4YiASMo")
.to_return(status: 200, body: onebox_response("youtube"))
stub_request(:get, "https://www.youtube.com/embed/KCyIfcevExE")
.to_return(status: 200, body: onebox_response("youtube-embed"))
1 Like

I’m not very good at this either, but you need to make that stub have exacctly the same stuff that youtube would return. So it needs to have some subset of what wget https://www.youtube.com/embed/wi2jAtpBl0Y returns. That’s what your spec needs as its input.

wc wi2jAtpBl0Y 
    6   251 65245 wi2jAtpBl0Y

That’s a lot of text to make sense of and stick in your stub.

1 Like

Can you open a draft PR with your current changes? That will be easier to follow along and help with. With the current information it looks like you need to create the stub files and stub and the outgoing requests.

2 Likes

Sure, here it is!

3 Likes

I fixed it in a new PR that has correct attribution to you as a Co-Author:

To make our test suite not depend on a working internet connection (and working Youtube), we stub all internet requests by making those call return with a cached response. the onebox_response function will call the file with a matching name as the parameter passed from the folder discourse/spec/fixtures/onebox at main · discourse/discourse · GitHub.

I added a new .response file for a shorts so we have a good test, but that was all that was necessary for this. Thanks for the PR!

3 Likes

Testing shorts onebox:

https://youtube.com/shorts/VvoFuaLAslw

1 Like

Video unavailable
The uploader has not made this video available in your country

Besides this, it should work :smile:

1 Like

What about

?

It works for me.

Quick question. Here’s the email notification I got:

image

The URL here is /embed/ and not /shorts/, but I suppose this is normal?

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.