What should I ask of a site that disallows OneBox crawls? Attempts to OneBox for Bookshop.org book listings result in: “Sorry, we were unable to generate a preview for this web page, because the following oEmbed / OpenGraph tags could not be found: description, image” – despite the required OG tags being present on the destination page. (At time of posting, the result here on meta is the same as my site): https://bookshop.org/p/books/parable-of-the-sower-octavia-e-butler/19767724?ean=9781538732182 OG details The og tags do exist on the page: [image] I’ve tried several OpenGraph validators: most report the URL unreachable. a couple of them retrieve a few OG fields from Bookshop.org, but nothing specific to the requested page. the iFramely debugger and the Facebook Sharing Debugger manage a full preview. I’m gathering that Bookshop.org is rejecting some crawls. I’d rather not introduce a plugin workaround if there’s something appropriate I might ask Bookshop.org to do, e.g. “Please allow the Discourse user agent to crawl?”

Cloudflare is blocking onebox from reaching the site. [image] ToddZ: if there’s something appropriate I might ask Bookshop.org to do, e.g. “Please allow the Discourse user agent to crawl?” Please relax you bot protection rule to allow Discourse Onebox ?

Thanks for confirming! I’ll ask them.

OneBox fails for Bookshop.org - crawl blocked?

Product Support

Falco (Falco) February 21, 2025, 7:22pm 2

Cloudflare is blocking onebox from reaching the site.

Please relax you bot protection rule to allow Discourse Onebox ?

Topic		Replies	Views
Onebox issue with a specific site Product Support onebox	13	1880	January 31, 2019
Onebox error but OG tag present Product Support	2	592	January 3, 2021
Onebox previews are not working for my site Product Support onebox	4	343	February 23, 2024
Link preview not working for specific site Product Support	10	2610	December 13, 2019
Onebox Does Not Display - iFramely Passes Product Support	3	1427	May 2, 2019

OneBox fails for Bookshop.org - crawl blocked?

Related topics