OneBox fails for Bookshop.org - crawl blocked?

Support
1

What should I ask of a site that disallows OneBox crawls?

Attempts to OneBox for Bookshop.org book listings result in: “Sorry, we were unable to generate a preview for this web page, because the following oEmbed / OpenGraph tags could not be found: description, image” – despite the required OG tags being present on the destination page.

(At time of posting, the result here on meta is the same as my site):

https://bookshop.org/p/books/parable-of-the-sower-octavia-e-butler/19767724?ean=9781538732182

OG details

The og tags do exist on the page:

image
image619×190 5.4 KB

I’ve tried several OpenGraph validators:

  • most report the URL unreachable.
  • a couple of them retrieve a few OG fields from Bookshop.org, but nothing specific to the requested page.
  • the iFramely debugger and the Facebook Sharing Debugger manage a full preview.

I’m gathering that Bookshop.org is rejecting some crawls. I’d rather not introduce a plugin workaround if there’s something appropriate I might ask Bookshop.org to do, e.g. “Please allow the Discourse user agent to crawl?”

1 Like
2

Cloudflare is blocking onebox from reaching the site.

Please relax you bot protection rule to allow Discourse Onebox ?

3 Likes
3

Thanks for confirming! I’ll ask them.

1 Like