# Oneboxing of sites with hash (#) in URL not working

**URL:** <https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804>\
**Category:** Bug\
**Created:** [2017年七月25日 18:46 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804 "2017-07-25T18:46:10Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)\
**Post date:** [2017年七月25日 18:46 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/1 "2017-07-25T18:46:10Z")

</div>

I’m pretty sure this used to onebox nicely with a highlighted line

> <https://github.com/discourse/discourse-slack-official/blob/master/lib/discourse_slack/slack.rb#L227>

```plaintext
https://github.com/discourse/discourse-slack-official/blob/master/lib/discourse_slack/slack.rb#L227

```

Works fine without the line number

> <https://github.com/discourse/discourse-slack-official/blob/master/lib/discourse_slack/slack.rb>

---

<div class="post-metadata">

**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\
**Post date:** [2017年七月25日 18:50 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/2 "2017-07-25T18:50:13Z")

</div>

Yeah I recall this used to work @techAPJ can you have a quick look?

---

<div class="post-metadata">

**Author:** ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)\
**Post date:** [2017年七月25日 18:51 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/3 "2017-07-25T18:51:21Z")

</div>

Aha - GitHub serves a “Canonical” URL on pages with specified line numbers.

```plaintext
<link rel="canonical" href="https://github.com/discourse/discourse-slack-official/blob/master/lib/discourse_slack/slack.rb" data-pjax-transient="">

```

~~So now that canonical URLs are prioritised, it's broken :(~~

---

<div class="post-metadata">

**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\
**Post date:** [2017年七月25日 19:13 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/4 "2017-07-25T19:13:20Z")

</div>

Interesting… well we got to special case github here, its one of the only places that uses a `#` to get magic behavior in onebox.

What I don’t understand though is why it is a complete fail.

---

<div class="post-metadata">

**Author:** ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)\
**Post date:** [2017年七月26日 08:51 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/5 "2017-07-26T08:51:23Z")

</div>

Wow, this was a tricky issue to debug. This is not related to “prefer canonical URL” change.

Also, this issue is not specific to GitHub only, but all the sites that uses # magic. For example:

> **[OS X 10.10 Yosemite: The Ars Technica Review](https://arstechnica.com/gadgets/2014/10/os-x-10-10/#page-12)**
>
> For the first time in forever, the Mac could be noticed by someone.

This regression was introduced eleven days ago in [this commit](https://github.com/discourse/discourse/commit/b534778f46ac310d9b59afa6f5390fced267f2f0). Specifically in [this code](https://github.com/discourse/discourse/blob/b534778f46ac310d9b59afa6f5390fced267f2f0/lib/final_destination.rb#L15):

```plaintext
URI(URI.escape(url)) if url

```

Escaping the URL converts/encodes `#` in link to `%23` and some sites does not handle that well, GitHub and Ars Technica included.

Wikipedia handle this just well, for example:

```plaintext
https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters

```

> **[Percent-encoding](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters)**
>
> Percent-encoding, also known as URL encoding, is a method to encode arbitrary data in a uniform resource identifier (URI) using only the US-ASCII characters legal within a URI. Percent-encoding is used to ensure special characters do not interfere with the URI's structure and interpretation. Special characters are replaced with a percent sign (%) followed by two hexadecimal digits representing the character's byte value. For example, a space is commonly encoded as %20:
> Although it is known as UR...

@eviltrout should we remove the URL escaping here? Or add a new case for sites with `404` response?

---

<div class="post-metadata">

**Author:** ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)\
**Post date:** [2017年七月26日 11:42 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/6 "2017-07-26T11:42:28Z")

</div>

`#` should not be encoded on the right hand side of the URL; that’s not correct. I think what’s happening is we are mixing logic for URL paths and querystring stuff.

> **[URI fragment](https://en.wikipedia.org/wiki/Fragment_identifier)**
>
> In computer hypertext, a URI fragment is a string of characters that refers to a resource that is subordinate to another, primary resource. The primary resource is identified by a Uniform Resource Identifier (URI), and the fragment identifier points to the subordinate resource.
> The fragment identifier introduced by a hash mark # is the optional last part of a URL for a document. It is typically used to identify a portion of that document. The generic syntax is specified in RFC 3986. The hash ma...

Operative words beng “the optional _last part of a URL_”

---

<div class="post-metadata">

**Author:** ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout)\
**Post date:** [2017年七月26日 14:17 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/7 "2017-07-26T14:17:08Z")

</div>

Yeah it looks to me like `URI.escape` is not being smart here. I would not remove escaping altogether as it can lead to security issues, but it’s worth coming up with a failing test and fixing the escaping to work properly.

Note that `#` should not be encoded as jeff points out, but `<` and `>` and quotes definitely should be.

---

<div class="post-metadata">

**Author:** ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)\
**Post date:** [2017年七月29日 13:04 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/8 "2017-07-29T13:04:51Z")

</div>

A post was split to a new topic: [Can’t onebox Overwatch Forums topic](https://meta.discourse.org/t/cant-onebox-overwatch-forums-topic/67066)

---

<div class="post-metadata">

**Author:** ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)\
**Post date:** [2017年七月29日 14:09 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/9 "2017-07-29T14:09:54Z")

</div>

Fixed via:

[https://github.com/discourse/discourse/commit/1fe553873caaafb0649d2c96872683d4e5007e50](https://github.com/discourse/discourse/commit/1fe553873caaafb0649d2c96872683d4e5007e50)

* * *

Demo:

> <https://github.com/discourse/discourse/blob/1fe553873caaafb0649d2c96872683d4e5007e50/lib/final_destination.rb#L15>

---

<div class="post-metadata">

**Author:** ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)\
**Post date:** [2017年七月29日 14:19 UTC](https://meta.discourse.org/t/oneboxing-of-sites-with-hash-in-url-not-working/66804/10 "2017-07-29T14:19:15Z")

</div>