# Embedding pens from CodePen

**URL:** https://meta.discourse.org/t/embedding-pens-from-codepen/13578
**Category:** Feature
**Created:** [9 maart 2014 om 21:56 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578 "2014-03-09T21:56:57Z")
**Posts on this page:** 20
**Page:** 3

<div class="post-metadata">

### Author: ![ralphm](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ralphm/32/108360_2.png) [@ralphm](https://meta.discourse.org/u/ralphm)
#### Post date: [18 januari 2017 om 12:17 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/45 "2017-01-18T12:17:35Z")

</div>

What’s weird? I think there was a change at some point where the user has to click “Run Pen”, because some pens can chew up a lot of resources, and not everyone will want them to run by default.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [18 januari 2017 om 12:30 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/46 "2017-01-18T12:30:08Z")

</div>

No, just that the preview differs significantly from the final posted output. That’s all. It works as designed.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [18 januari 2017 om 14:30 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/47 "2017-01-18T14:30:59Z")

</div>

Onebox implementation allows for 2 different outputs depending on preview/actual. We do that for youtube cause every keypress would re-render which becomes obnoxious. (at least until we can hold markdown output in a virtual dom and apply diffs). I guess this is happening here.

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [18 januari 2017 om 15:45 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/48 "2017-01-18T15:45:32Z")

</div>

> [@sam](#):
>
> Onebox implementation allows for 2 different outputs depending on preview/actual. (…) I guess this is happening here.

Yup, but we can do **much** better in this case. Will fix ✏

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [15 april 2019 om 06:23 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/49 "2019-04-15T06:23:09Z")

</div>

This has regressed, again:

https://codepen.io/web-tiki/embed/preview/dNpgrR?default-tabs=css%2Cresult&height=300&host=https%3A%2F%2Fcodepen.io&slug-hash=dNpgrR

https://codepen.io/web-tiki/embed/preview/dNpgrR?default-tabs=css%2Cresult&height=300&host=https%3A%2F%2Fcodepen.io&slug-hash=dNpgrR

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [15 april 2019 om 07:25 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/50 "2019-04-15T07:25:18Z")

</div>

☹

@Roman can you have a look?

---

<div class="post-metadata">

### Author: ![Roman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/roman/32/157504_2.png) [@Roman](https://meta.discourse.org/u/Roman)
#### Post date: [15 april 2019 om 14:14 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/52 "2019-04-15T14:14:03Z")

</div>

Looks like we’re being intercepted by CloudFlare when trying to fetch the URL’s HTML and this prevents us from discovering the oEmbed endpoint.

I think our options are:

- Manually adding the oEmbed endpoint URL to onebox without having to fetch the HTML.
- Check if Codepen can allow-list us? 🤔

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [15 april 2019 om 15:08 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/53 "2019-04-15T15:08:21Z")

</div>

I’m sure we can get it fixed. You hit our oEmbed endpoint manually, yes? What would the referrer look like? Is it potentially different on every site is running Discourse?

---

<div class="post-metadata">

### Author: ![Roman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/roman/32/157504_2.png) [@Roman](https://meta.discourse.org/u/Roman)
#### Post date: [15 april 2019 om 17:06 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/54 "2019-04-15T17:06:27Z")

</div>

Thanks for jumping in so quickly @chriscoyier!

This happens when I try to onebox the link using the gem, the request is being done from Ruby’s `Net::HTTP`.

The flow is:

1. Fetch Codepen’s HTML (e.g. GET to [https://codepen.io/web-tiki/full/dNpgrR](https://codepen.io/web-tiki/full/dNpgrR)).
2. Discover the oEmbed URL from the `application/json+oembed` header tag.
3. Fetch oEmbed data and build the box.

We never reach step number 2.

Onebox can also work outside of Discourse since it’s a standalone gem, so I don’t think we can rely on a referrer. On the other hand, we could possibly set a specific user-agent that can be allowed on your side? (Is this acceptable, @sam?)

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [15 april 2019 om 17:15 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/55 "2019-04-15T17:15:23Z")

</div>

> [@Roman](#):
>
> On the other hand, we could possibly set a specific user-agent that can be allowed on your side?

I believe we set one per:

> [@Onebox user agent](https://meta.discourse.org/t/onebox-user-agent/61498/18):
>
> This commit adds a custom User-Agent to discourse oneboxes: The User agent looks like this: Discourse Forum Onebox v1.8.0.beta13

---

<div class="post-metadata">

### Author: ![Roman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/roman/32/157504_2.png) [@Roman](https://meta.discourse.org/u/Roman)
#### Post date: [15 april 2019 om 17:24 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/56 "2019-04-15T17:24:30Z")

</div>

Thanks for pointing that out!

If we allow a particular user-agent, we’ll have to move that into the gem in order to ensure that Codepen’s oneboxes will always work and setting a different agent will no longer be possible.

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [23 april 2019 om 16:55 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/57 "2019-04-23T16:55:41Z")

</div>

How’s this going? Are we still screwing y’all up with blocking or has it resolved itself?

If it’s still a problem, we just need a way on our end to make sure we never block these requests. Discourse is self-hosted right? So we can’t count on any particular referrer URL. So it would probably have to be something unique in the UA?

On our end, we might be able to entirely unblock anything oEmbed related. That’s just tricky these days as anything that is entirely unchallenged is a potential attack vector for DDoS.

Most of our oEmbed usage is ultimately through [https://embed.ly/](https://embed.ly/) - not sure if that’s a possibility. Perhaps not perfect for an open source thing.

---

<div class="post-metadata">

### Author: ![supermathie](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/supermathie/32/507518_2.png) [@supermathie](https://meta.discourse.org/u/supermathie)
#### Post date: [23 april 2019 om 18:07 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/58 "2019-04-23T18:07:22Z")

</div>

This is still a problem:

```plaintext
○ → curl --user-agent "Discourse Forum Onebox v2.2.0" https://codepen.io/web-tiki/full/dNpgrR
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
…

```

If our user-agent could be whitelisted that would solve the problem. The string we use for these requests is:  
`Discourse Forum Onebox v#{discourse_version}`

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [23 april 2019 om 22:40 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/59 "2019-04-23T22:40:45Z")

</div>

Thanks for the info there. I’ve got a ticket opened and we’ll get it fixed up.

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [8 mei 2019 om 18:48 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/60 "2019-05-08T18:48:57Z")

</div>

Give it another try when you have a moment.

---

<div class="post-metadata">

### Author: ![supermathie](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/supermathie/32/507518_2.png) [@supermathie](https://meta.discourse.org/u/supermathie)
#### Post date: [8 mei 2019 om 19:46 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/61 "2019-05-08T19:46:19Z")

</div>

Unfortunately it appears that _ONLY v2.2.0_ was whitelisted:

```plaintext
○ → curl -s --user-agent "Discourse Forum Onebox v2.2.0" https://codepen.io/web-tiki/full/dNpgrR | head
<!doctype html>
<!--[if lte IE 9]>
<html lang="en" class="oldie">
<![endif]-->
<!--[if gt IE 9]><!-->
<html lang="en">
<!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name='viewport' content='width=device-width, initial-scale=1'>

○ → curl -s --user-agent "Discourse Forum Onebox v2.2.1" https://codepen.io/web-tiki/full/dNpgrR | head
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

○ → curl -s --user-agent "Discourse Forum Onebox v2.3.1" https://codepen.io/web-tiki/full/dNpgrR | head
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]> <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]> <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

```

To make this work for everyone, the leading string `Discourse Forum Onebox` should be whitelisted.

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [8 mei 2019 om 20:10 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/62 "2019-05-08T20:10:14Z")

</div>

I can do that. This is tricky stuff though. This is essentially a hole in our protections (we’ve had DDoS issues lately). Plus, a map to that hole is right here in this public thread on the internet. I’ll whitelist it more broadly for now, but if it gets found and hammered, I’ll have to remove the whitelisting. Sorry for making this complicated.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [9 mei 2019 om 08:27 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/63 "2019-05-09T08:27:46Z")

</div>

Wait, you have NO other user agents whitelisted? That can’t be right..

It’s quite common for hosts to whitelist user agents for things like this, for example WPEngine does that routinely for all the wordpress blogs they host, because all their customers pay for the CPU time of each request, and when they get lots of requests from bad or unknown crawlers..

---

<div class="post-metadata">

### Author: ![Roman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/roman/32/157504_2.png) [@Roman](https://meta.discourse.org/u/Roman)
#### Post date: [16 oktober 2019 om 19:33 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/64 "2019-10-16T19:33:53Z")

</div>

Hey @chriscoyier 👋

We’re still being blocked, is there something we can do from our side to help?

---

<div class="post-metadata">

### Author: ![chriscoyier](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/chriscoyier/32/108376_2.png) [@chriscoyier](https://meta.discourse.org/u/chriscoyier)
#### Post date: [4 november 2019 om 16:36 UTC](https://meta.discourse.org/t/embedding-pens-from-codepen/13578/65 "2019-11-04T16:36:05Z")

</div>

We’re still trying to figure out the best way to handle it. Sorry for the incredible delay here. I’ll update with news as I have it.

[Vorige pagina](https://meta.discourse.org/t/embedding-pens-from-codepen/13578.md?page=2)

[Volgende pagina](https://meta.discourse.org/t/embedding-pens-from-codepen/13578.md?page=4)