"Onebox Assistant", crawl for those previews reliably!

Just investigating something Ed

It appears the API is failing to return the source now, despite their docs claiming this is still supported.

I’ll let you know what I hear back.

In addition a helpful logging statement in my code was bugged!

1 Like

I got an email from the service author saying

Sorry, it took a bit longer because other changes, but it should now be there (I hope!).

He updated the service, but I didn’t find a difference. Yet you saw it ))

1 Like

This should return a result and include a source field. It returns with a result, but doesn’t include the source key/value pair.

curl -H "x-api-key:$$$my-key$$$" https://api.embed.rocks/api?url=https%3A%2F%2Fnews.bbc.co.uk&include=source
2 Likes

OK it’s starting working for me using:

image

and latest version of plugin (I released a small fix for logging warnings when a response has an empty source field)

Try it now.

2 Likes

Now it’s working. But still there is no activity on the usage dashboard AND the speed is very low. When I have 20 youtube links it takes 2 minutes to retrieve all the videos. Can we speed up it?

1 Like

Great.

How quickly the API responds is not my area. No idea about the dashboard either :slight_smile: Perhaps there’s a lag before it updates.

Cheers.

2 Likes

I would kindly ask you to check another issue with YouTube.
After the last update all youtube thumbnails are created in a changed way.

They are smaller than the video block and these black fields appear. can you check the link with your instance?

https://www.youtube.com/watch?v=m_DS_WHVy48

1 Like

That’s off-topic.

This plugin only changes the route for getting meta-data, you’ll need to take that up elsewhere. Unfortunately I don’t have the luxury of time to get involved with more issues than I own :).

6 Likes

Hello, Robert! Is your plugin working correctly after the last Discourse update?

I found a bug. In a preview window of the composer there is no image, but after the draft is saved the error goes away. So smth with the preview window happened.

1 Like

any plans to support Instagram reels?

embed.rocks…

{
  "url": "https://www.instagram.com/reel/CMDWeC7g7UK/",
  "site": "www.instagram.com",
  "favicon": {
    "url": "https://www.instagram.com/favicon.ico",
    "safe": "https%3A%2F%2Fwww.instagram.com%2Ffavicon.ico"
  },
  "title": "Instagram",
  "description": "abcbrisbane                                                         Verified                                                                                     4,206 posts                   ·                   62.6k followers",
  "article": "<div><div>\n              <p><span>abcbrisbane</span>\n                \n                  <span>\n                    Verified\n                  </span>\n                \n              </p>\n              <p><span>\n                  4,206 posts\n                  &#xB7;\n                  62.6k followers\n                </span>\n              </p>\n            </div></div>",
  "images": [
    {
      "url": "https://scontent-iad3-1.cdninstagram.com/v/t51.2885-15/e35/156768077_174277514307696_333807332882305732_n.jpg?tp=1&_nc_ht=scontent-iad3-1.cdninstagram.com&_nc_cat=100&_nc_ohc=uG5Srww55lEAX9JmzsV&ccb=7-4&oh=459a330309ca6a231241a1ebbcc10815&oe=60548228",
      "safe": "https%3A%2F%2Fscontent-iad3-1.cdninstagram.com%2Fv%2Ft51.2885-15%2Fe35%2F156768077_174277514307696_333807332882305732_n.jpg%3Ftp%3D1%26_nc_ht%3Dscontent-iad3-1.cdninstagram.com%26_nc_cat%3D100%26_nc_ohc%3DuG5Srww55lEAX9JmzsV%26ccb%3D7-4%26oh%3D459a330309ca6a231241a1ebbcc10815%26oe%3D60548228",
      "width": 720,
      "height": 1280
    }
  ],
  "videos": [
    {
      "url": "https://scontent-iad3-1.cdninstagram.com/v/t50.2886-16/156535627_243445060829263_4933319464983284914_n.mp4?_nc_ht=scontent-iad3-1.cdninstagram.com&_nc_cat=107&_nc_ohc=wT9AGKAnNIgAX8xqrbt&ccb=7-4&oe=605492EA&oh=d651f243b067b67ea3dcc1fcc9a576bb",
      "secureUrl": "https://scontent-iad3-1.cdninstagram.com/v/t50.2886-16/156535627_243445060829263_4933319464983284914_n.mp4?_nc_ht=scontent-iad3-1.cdninstagram.com&_nc_cat=107&_nc_ohc=wT9AGKAnNIgAX8xqrbt&ccb=7-4&oe=605492EA&oh=d651f243b067b67ea3dcc1fcc9a576bb",
      "type": "video/mp4"
    }
  ],
  "audios": [],
  "type": "video",
  "oembed": {
    "version": "1.0",
    "author_name": "abcbrisbane",
    "provider_name": "Instagram",
    "provider_url": "https://www.instagram.com/",
    "type": "rich",
    "width": 658,
    "html": "<blockquote class=\"instagram-media\" data-instgrm-captioned data-instgrm-permalink=\"https://www.instagram.com/reel/CMDWeC7g7UK/?utm_source=ig_embed&amp;utm_campaign=loading\" data-instgrm-version=\"13\" style=\" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:658px; min-width:326px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);\"><div style=\"padding:16px;\"> <a href=\"https://www.instagram.com/reel/CMDWeC7g7UK/?utm_source=ig_embed&amp;utm_campaign=loading\" style=\" background:#FFFFFF; line-height:0; padding:0 0; text-align:center; text-decoration:none; width:100%;\" target=\"_blank\"> <div style=\" display: flex; flex-direction: row; align-items: center;\"> <div style=\"background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;\"></div> <div style=\"display: flex; flex-direction: column; flex-grow: 1; justify-content: center;\"> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;\"></div> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;\"></div></div></div><div style=\"padding: 19% 0;\"></div> <div style=\"display:block; height:50px; margin:0 auto 12px; width:50px;\"><svg width=\"50px\" height=\"50px\" viewBox=\"0 0 60 60\" version=\"1.1\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\"><g stroke=\"none\" stroke-width=\"1\" fill=\"none\" fill-rule=\"evenodd\"><g transform=\"translate(-511.000000, -20.000000)\" fill=\"#000000\"><g><path d=\"M556.869,30.41 C554.814,30.41 553.148,32.076 553.148,34.131 C553.148,36.186 554.814,37.852 556.869,37.852 C558.924,37.852 560.59,36.186 560.59,34.131 C560.59,32.076 558.924,30.41 556.869,30.41 M541,60.657 C535.114,60.657 530.342,55.887 530.342,50 C530.342,44.114 535.114,39.342 541,39.342 C546.887,39.342 551.658,44.114 551.658,50 C551.658,55.887 546.887,60.657 541,60.657 M541,33.886 C532.1,33.886 524.886,41.1 524.886,50 C524.886,58.899 532.1,66.113 541,66.113 C549.9,66.113 557.115,58.899 557.115,50 C557.115,41.1 549.9,33.886 541,33.886 M565.378,62.101 C565.244,65.022 564.756,66.606 564.346,67.663 C563.803,69.06 563.154,70.057 562.106,71.106 C561.058,72.155 560.06,72.803 558.662,73.347 C557.607,73.757 556.021,74.244 553.102,74.378 C549.944,74.521 548.997,74.552 541,74.552 C533.003,74.552 532.056,74.521 528.898,74.378 C525.979,74.244 524.393,73.757 523.338,73.347 C521.94,72.803 520.942,72.155 519.894,71.106 C518.846,70.057 518.197,69.06 517.654,67.663 C517.244,66.606 516.755,65.022 516.623,62.101 C516.479,58.943 516.448,57.996 516.448,50 C516.448,42.003 516.479,41.056 516.623,37.899 C516.755,34.978 517.244,33.391 517.654,32.338 C518.197,30.938 518.846,29.942 519.894,28.894 C520.942,27.846 521.94,27.196 523.338,26.654 C524.393,26.244 525.979,25.756 528.898,25.623 C532.057,25.479 533.004,25.448 541,25.448 C548.997,25.448 549.943,25.479 553.102,25.623 C556.021,25.756 557.607,26.244 558.662,26.654 C560.06,27.196 561.058,27.846 562.106,28.894 C563.154,29.942 563.803,30.938 564.346,32.338 C564.756,33.391 565.244,34.978 565.378,37.899 C565.522,41.056 565.552,42.003 565.552,50 C565.552,57.996 565.522,58.943 565.378,62.101 M570.82,37.631 C570.674,34.438 570.167,32.258 569.425,30.349 C568.659,28.377 567.633,26.702 565.965,25.035 C564.297,23.368 562.623,22.342 560.652,21.575 C558.743,20.834 556.562,20.326 553.369,20.18 C550.169,20.033 549.148,20 541,20 C532.853,20 531.831,20.033 528.631,20.18 C525.438,20.326 523.257,20.834 521.349,21.575 C519.376,22.342 517.703,23.368 516.035,25.035 C514.368,26.702 513.342,28.377 512.574,30.349 C511.834,32.258 511.326,34.438 511.181,37.631 C511.035,40.831 511,41.851 511,50 C511,58.147 511.035,59.17 511.181,62.369 C511.326,65.562 511.834,67.743 512.574,69.651 C513.342,71.625 514.368,73.296 516.035,74.965 C517.703,76.634 519.376,77.658 521.349,78.425 C523.257,79.167 525.438,79.673 528.631,79.82 C531.831,79.965 532.853,80.001 541,80.001 C549.148,80.001 550.169,79.965 553.369,79.82 C556.562,79.673 558.743,79.167 560.652,78.425 C562.623,77.658 564.297,76.634 565.965,74.965 C567.633,73.296 568.659,71.625 569.425,69.651 C570.167,67.743 570.674,65.562 570.82,62.369 C570.966,59.17 571,58.147 571,50 C571,41.851 570.966,40.831 570.82,37.631\"></path></g></g></g></svg></div><div style=\"padding-top: 8px;\"> <div style=\" color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;\"> View this post on Instagram</div></div><div style=\"padding: 12.5% 0;\"></div> <div style=\"display: flex; flex-direction: row; margin-bottom: 14px; align-items: center;\"><div> <div style=\"background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(0px) translateY(7px);\"></div> <div style=\"background-color: #F4F4F4; height: 12.5px; transform: rotate(-45deg) translateX(3px) translateY(1px); width: 12.5px; flex-grow: 0; margin-right: 14px; margin-left: 2px;\"></div> <div style=\"background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(9px) translateY(-18px);\"></div></div><div style=\"margin-left: 8px;\"> <div style=\" background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;\"></div> <div style=\" width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)\"></div></div><div style=\"margin-left: auto;\"> <div style=\" width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);\"></div> <div style=\" background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);\"></div> <div style=\" width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);\"></div></div></div> <div style=\"display: flex; flex-direction: column; flex-grow: 1; justify-content: center; margin-bottom: 24px;\"> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 224px;\"></div> <div style=\" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 144px;\"></div></div></a><p style=\" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;\"><a href=\"https://www.instagram.com/reel/CMDWeC7g7UK/?utm_source=ig_embed&amp;utm_campaign=loading\" style=\" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none;\" target=\"_blank\">A post shared by ABC Brisbane (@abcbrisbane)</a></p></div></blockquote>\n<script async src=\"//platform.instagram.com/en_US/embeds.js\"></script>",
    "thumbnail_url": "https://scontent.cdninstagram.com/v/t51.2885-15/sh0.08/e35/p640x640/156768077_174277514307696_333807332882305732_n.jpg?tp=1&_nc_ht=scontent.cdninstagram.com&_nc_cat=100&_nc_ohc=uG5Srww55lEAX9n1PBr&ccb=7-4&oh=e7a86224012bc80afe9e882dbb5722ff&oe=60550A92",
    "thumbnail_width": 640,
    "thumbnail_height": 1137
  },
  "duration": 2545,
  "html": "<div class=\"card\">\n    <video controls loop>\n      <source type=\"video/webm\" src=\"https://scontent-iad3-1.cdninstagram.com/v/t50.2886-16/156535627_243445060829263_4933319464983284914_n.mp4?_nc_ht=scontent-iad3-1.cdninstagram.com&_nc_cat=107&_nc_ohc=wT9AGKAnNIgAX8xqrbt&ccb=7-4&oe=605492EA&oh=d651f243b067b67ea3dcc1fcc9a576bb\">\n    </video>\n    \n    <div class=\"card-text\">\n      <h3>Instagram</h3>\n      <p>abcbrisbane                                                         Verified                                                                                     4,206 posts                   ·                   62.6k followers</p>\n      \n        <p class=\"fav\">\n          <img onerror=\"this.style.display='none'\" class=\"favicon\" src=\"https://www.instagram.com/favicon.ico\">\n          www.instagram.com\n        </p>\n    </div>\n  \n  </div>"
}

discourse…

1 Like

Instagram is an absolute nightmare to work with :confused:

For a social media company, they’re very anti-social.

If photos embed, video fails. If video works, TV fails. If TV works, Reels fail. If etc… it’s a never ending battle :pensive:

To the point where we’ve now stopped worrying/caring about Instagram posts.

4 Likes

Have not been able to repro this. I’ll try again at month end.

3 Likes

If it works in a onebox in a vanilla install, it should work here.

If it doesn’t work in onebox, it won’t work here either.

All this plugin is doing is improving the chances of you successfully scraping the page source data which it then hands off to onebox :).

4 Likes

I’ve installed this plugin but unfortunately haven’t been able to make it work.

Here are my settings:

(the API Key is also set, but not shown in the image)

Is there any way to troubleshoot why the editor isn’t generating any oneboxes? After I paste a link in a new line, it shows the spinner for a moment but nothing comes out of it. I’ve checked the Discourse /logs, but there’s no error there.

I’m using Discourse 2.7.0.beta5 (9be99c055f) and discourse-onebox-assistant 2.0.2. Tried with multiple browsers. (Onebox was working with some sites before installing the plugin)

Try to save the post and check after it. I have a problem now that there is no a preview image but it appears after I saved the post.

1 Like

Just tried, and it didn’t generate after posting. Also tried rebuilding the HTML, but no luck.
I’ve checked at embed.rocks and the URLs are supported by them.

I have the same settings. It works for me. Sorry, don’t know the reasons you can’t make it working.

:frowning: Is there any way to troubleshoot or debug the plugin to see why this is happening?

It’s working for me, perfectly, on a recent rebuild. Can you share the link you are trying to show?

Remember, if you’ve previously attempted to onebox a link, Discourse core will cache the result.

You can add a random querystring on the end to overcome the cache: https://mylink.com/todaynews?random=random

You can also check the API is responding with, e.g.:

curl -X GET "https://api.embed.rocks/api/?url=https%3A%2F%2Fnews.bbc.co.uk%0A&skip=article,description,oembed,imextra&include=source" -H "x-api-key: %%%your-api-key%%%"

Do this from your server.

You need to url encode the site you are calling (the url parameter value) using some site like this (not vouched for!)

3 Likes

Thanks for the in-depth post!

However, it… was fixed without me doing anything special. I had to install another plugin, so I rebuilt Discourse, and it started working shortly after that. I don’t know if that was what fixed the issue, but if it happens again I’ll follow your directions for sure :slight_smile:

2 Likes