# Automatic encoding of parsed URL params

**URL:** <https://meta.discourse.org/t/automatic-encoding-of-parsed-url-params/67232>\
**Category:** Development\
**Created:** [August 1, 2017, 9:53am UTC](https://meta.discourse.org/t/automatic-encoding-of-parsed-url-params/67232 "2017-08-01T09:53:48Z")\
**Posts on this page:** 1\
**Showing post:** 15

<div class="post-metadata">

**Author:** ![elijah](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/elijah/32/104055_2.png) [@elijah](https://meta.discourse.org/u/elijah)\
**Post date:** [August 7, 2017, 5:26am UTC](https://meta.discourse.org/t/automatic-encoding-of-parsed-url-params/67232/15 "2017-08-07T05:26:04Z")

</div>

Digging through a bunch of code, I find that unescape() (by that or a similar name) has been defined in multiple locations throughout the RoR system. There’s even this one in ActiveSupport (active\_support/core\_ext/uri.rb) which includes an explicit test that things decode right:

```ruby
require 'uri'
str = "\xE6\x97\xA5\xE6\x9C\xAC\xE8\xAA\x9E" # Ni-ho-nn-go in UTF-8, means Japanese.
parser = URI::Parser.new

unless str == parser.unescape(parser.escape(str))
  URI::Parser.class_eval do
    remove_method :unescape
    def unescape(str, escaped = /%[a-fA-F\d]{2}/)
      # TODO: Are we actually sure that ASCII == UTF-8?
      # YK: My initial experiments say yes, but let's be sure please
      enc = str.encoding
      enc = Encoding::UTF_8 if enc == Encoding::US_ASCII
      str.gsub(escaped) { [$&[1, 2].hex].pack('C') }.force_encoding(enc)
    end
  end
end

```

Very, very few of the unescapes make any reference to Encoding, relying on the default inside Ruby’s core `uri`:

```ruby
  def self.decode_www_form_component(str, enc=Encoding::UTF_8)

```

But this bit in ActionDispatch (action\_dispatch/journey/router/utils.rb) looks curious:

```ruby
          def unescape_uri(uri)
            encoding = uri.encoding == US_ASCII ? UTF_8 : uri.encoding
            uri.gsub(ESCAPED) { [$&[1, 2].hex].pack('C') }.force_encoding(encoding)
          end

```

If US-ASCII, force it to be UTF-8, _ **otherwise leave as is** _. And the as-is we are seeing here is ASCII-8BIT. (That, incidentally, is the encoding used generally for the url _escape_ functions, as it gives the nice byte-wise %encoding result.)

I haven’t reproduced this bug in my own environment, so I can’t test fixing that. And I’m unclear if this is the exact unescaper that get used here.

---

_[View the full topic](https://meta.discourse.org/t/automatic-encoding-of-parsed-url-params/67232)._