What’s weird? I think there was a change at some point where the user has to click “Run Pen”, because some pens can chew up a lot of resources, and not everyone will want them to run by default.
No, just that the preview differs significantly from the final posted output. That’s all. It works as designed.
2 个赞
Onebox implementation allows for 2 different outputs depending on preview/actual. We do that for youtube cause every keypress would re-render which becomes obnoxious. (at least until we can hold markdown output in a virtual dom and apply diffs). I guess this is happening here.
1 个赞
Yup, but we can do much better in this case. Will fix 
3 个赞
This has regressed, again:
1 个赞
@Roman can you have a look?
1 个赞
Looks like we’re being intercepted by CloudFlare when trying to fetch the URL’s HTML and this prevents us from discovering the oEmbed endpoint.
I think our options are:
- Manually adding the oEmbed endpoint URL to onebox without having to fetch the HTML.
- Check if Codepen can allow-list us?
2 个赞
I’m sure we can get it fixed. You hit our oEmbed endpoint manually, yes? What would the referrer look like? Is it potentially different on every site is running Discourse?
3 个赞
Thanks for jumping in so quickly @chriscoyier!
This happens when I try to onebox the link using the gem, the request is being done from Ruby’s Net::HTTP.
The flow is:
- Fetch Codepen’s HTML (e.g. GET to https://codepen.io/web-tiki/full/dNpgrR).
- Discover the oEmbed URL from the
application/json+oembedheader tag. - Fetch oEmbed data and build the box.
We never reach step number 2.
Onebox can also work outside of Discourse since it’s a standalone gem, so I don’t think we can rely on a referrer. On the other hand, we could possibly set a specific user-agent that can be allowed on your side? (Is this acceptable, @sam?)
3 个赞
I believe we set one per:
6 个赞
Thanks for pointing that out!
If we allow a particular user-agent, we’ll have to move that into the gem in order to ensure that Codepen’s oneboxes will always work and setting a different agent will no longer be possible.
3 个赞
How’s this going? Are we still screwing y’all up with blocking or has it resolved itself?
If it’s still a problem, we just need a way on our end to make sure we never block these requests. Discourse is self-hosted right? So we can’t count on any particular referrer URL. So it would probably have to be something unique in the UA?
On our end, we might be able to entirely unblock anything oEmbed related. That’s just tricky these days as anything that is entirely unchallenged is a potential attack vector for DDoS.
Most of our oEmbed usage is ultimately through https://embed.ly/ - not sure if that’s a possibility. Perhaps not perfect for an open source thing.
4 个赞
This is still a problem:
○ → curl --user-agent "Discourse Forum Onebox v2.2.0" https://codepen.io/web-tiki/full/dNpgrR
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
…
If our user-agent could be whitelisted that would solve the problem. The string we use for these requests is:
Discourse Forum Onebox v#{discourse_version}
7 个赞
Thanks for the info there. I’ve got a ticket opened and we’ll get it fixed up.
9 个赞
Give it another try when you have a moment.
1 个赞
Unfortunately it appears that ONLY v2.2.0 was whitelisted:
○ → curl -s --user-agent "Discourse Forum Onebox v2.2.0" https://codepen.io/web-tiki/full/dNpgrR | head
<!doctype html>
<!--[if lte IE 9]>
<html lang="en" class="oldie">
<![endif]-->
<!--[if gt IE 9]><!-->
<html lang="en">
<!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name='viewport' content='width=device-width, initial-scale=1'>
○ → curl -s --user-agent "Discourse Forum Onebox v2.2.1" https://codepen.io/web-tiki/full/dNpgrR | head
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
○ → curl -s --user-agent "Discourse Forum Onebox v2.3.1" https://codepen.io/web-tiki/full/dNpgrR | head
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
To make this work for everyone, the leading string Discourse Forum Onebox should be whitelisted.
5 个赞
I can do that. This is tricky stuff though. This is essentially a hole in our protections (we’ve had DDoS issues lately). Plus, a map to that hole is right here in this public thread on the internet. I’ll whitelist it more broadly for now, but if it gets found and hammered, I’ll have to remove the whitelisting. Sorry for making this complicated.
7 个赞
Wait, you have NO other user agents whitelisted? That can’t be right…
It’s quite common for hosts to whitelist user agents for things like this, for example WPEngine does that routinely for all the wordpress blogs they host, because all their customers pay for the CPU time of each request, and when they get lots of requests from bad or unknown crawlers…
2 个赞
你好 @chriscoyier 
我们仍然被拦截了，请问我们这边能做些什么来协助解决吗？
4 个赞
我们仍在努力找出最佳的处理方式。对此造成的严重延误，我们深表歉意。一旦有消息，我会立即更新。
6 个赞