# Onebox 404 on Amazon Link - redirection forbidden

**URL:** https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092
**Category:** Bug
**Created:** [August 2, 2016, 12:49am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092 "2016-08-02T00:49:20Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 12:49am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/1 "2016-08-02T00:49:20Z")

</div>

Hi - I’ve got a new install of Discourse (v1.5.3-Dirty) via Bitnami, hosted on an Azure A1 VM.

When i use amazon with a onebox link, I receive a 404.

I logged into the rails console via ssh, and was able to reproduce it, and received this error (using the sample from the Onebox github)

```
irb(main):002:0> url = "http://www.amazon.com/gp/product/B005T3GRNW/ref=s9_simh_gw_p147_d0_i2"
=> "http://www.amazon.com/gp/product/B005T3GRNW/ref=s9_simh_gw_p147_d0_i2"
irb(main):003:0> preview = Onebox.preview(url)
=> #<Onebox::Preview:0x007f4d96897ed0 @url="http://www.amazon.com/gp/product/B005T3GRNW/ref=s9_simh_gw_p147_d0_i2", @options=#<OpenStruct cache=#<Moneta::Expires:0x007f4d9870b420 @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x007f4d9870b4c0 @adapter=#<Moneta::Adapters::Memory:0x007f4d988a74a0 @backend={}>, @prefix="", @features=[:increment, :create]>, @default_expires=nil>, connect_timeout=5, timeout=10, load_paths=["/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/templates"], twitter_client=TwitterApi>, @cache=#<Moneta::Expires:0x007f4d9870b420 @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x007f4d9870b4c0 @adapter=#<Moneta::Adapters::Memory:0x007f4d988a74a0 @backend={}>, @prefix="", @features=[:increment, :create]>, @default_expires=nil>, @engine_class=Onebox::Engine::AmazonOnebox>
irb(main):004:0> preview.to_s
RuntimeError: redirection forbidden: http://www.amazon.com/gp/aw/d/B005T3GRNW -> https://www.amazon.com/gp/aw/d/B005T3GRNW
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:223:in `open_loop'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:149:in `open_uri'
        from /opt/bitnami/apps/discourse/htdocs/lib/freedom_patches/open_uri_redirections.rb:70:in `open_uri'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:704:in `open'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:34:in `open'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/engine/html.rb:12:in `raw'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/engine/amazon_onebox.rb:50:in `data'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/engine.rb:71:in `block in record'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/moneta-0.8.0/lib/moneta/mixins.rb:165:in `fetch'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/engine.rb:71:in `record'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/layout_support.rb:9:in `layout'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/layout_support.rb:13:in `to_html'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/preview.rb:37:in `engine_html'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/lib/onebox/preview.rb:14:in `to_s'
        from (irb):4
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/console.rb:110:in `start'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/console.rb:9:in `start'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:68:in `console'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands.rb:17:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:45:in `require'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:45:in `block in exec_app_rails'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:34:in `loop'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:34:in `exec_app_rails'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/cli.rb:5:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/bin/rails:9:in `require'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/bin/rails:9:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/bin/rails:23:in `load'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/bin/rails:23:in `<main>'
irb(main):005:0> 

```

Any ideas?

---

<div class="post-metadata">

### Author: ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [August 2, 2016, 12:52am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/2 "2016-08-02T00:52:20Z")

</div>

Use the redirection target link, rather than the link that’s being redirected?

---

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 12:54am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/3 "2016-08-02T00:54:23Z")

</div>

This doesn’t seem correct either (but it doesn’t error at least) …

```
irb(main):008:0> url = "https://www.amazon.com/gp/aw/d/B005T3GRNW"
=> "https://www.amazon.com/gp/aw/d/B005T3GRNW"
irb(main):009:0> preview = Onebox.preview(url)
=> #<Onebox::Preview:0x007f4d98cd28b8 @url="https://www.amazon.com/gp/aw/d/B005T3GRNW", @options=#<OpenStruct cache=#<Moneta::Expires:0x007f4d9870b420 @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x007f4d9870b4c0 @adapter=#<Moneta::Adapters::Memory:0x007f4d988a74a0 @backend={}>, @prefix="", @features=[:increment, :create]>, @default_expires=nil>, connect_timeout=5, timeout=10, load_paths=["/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/onebox-1.5.38/templates"], twitter_client=TwitterApi>, @cache=#<Moneta::Expires:0x007f4d9870b420 @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x007f4d9870b4c0 @adapter=#<Moneta::Adapters::Memory:0x007f4d988a74a0 @backend={}>, @prefix="", @features=[:increment, :create]>, @default_expires=nil>, @engine_class=Onebox::Engine::AmazonOnebox>
irb(main):010:0> preview.to_s
=> ""

```

---

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 12:56am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/4 "2016-08-02T00:56:57Z")

</div>

Just trying the original link here to see what happens

## HTTP Link

> **[Amazon.com: (Old Model) Seagate 1TB Desktop HDD Sata 6Gb/s 64MB Cache 3.5-Inch...](https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW)**
>
> Product Description Seagate Barracuda ST1000DM003 1TB 7200RPM SATA3/SATA 6.0 GB/s 64MB Hard Drive (3.5 inch) From the Manufacturer The Seagate Desktop HDD is the one hard drive for all your desktop...

## HTTPS Link

> **[Amazon.com: (Old Model) Seagate 1TB Desktop HDD Sata 6Gb/s 64MB Cache 3.5-Inch...](https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW)**
>
> Product Description Seagate Barracuda ST1000DM003 1TB 7200RPM SATA3/SATA 6.0 GB/s 64MB Hard Drive (3.5 inch) From the Manufacturer The Seagate Desktop HDD is the one hard drive for all your desktop...

---

<div class="post-metadata">

### Author: ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [August 2, 2016, 12:57am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/5 "2016-08-02T00:57:44Z")

</div>

I get redirected to a different URL, and that one oneboxes up quite nicely:

```
[1] pry(main)> url = "https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW"
=> "https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW"                  
[2] pry(main)> preview = Onebox.preview(url)
=> #<Onebox::Preview:0x00562a9555b5c0       
 @cache=
  #<Moneta::Expires:0x00562a95c4f598
   @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x00562a95c4f638 @adapter=#<Moneta::Adapters::Memory:0x00562a95c15c80 @backend={}>, @features=[:increment, :create], @prefix="">,
   @default_expires=nil>,
 @engine_class=Onebox::Engine::AmazonOnebox,
 @options=
  #<OpenStruct cache=#<Moneta::Expires:0x00562a95c4f598 @adapter=#<Moneta::Transformer::JsonPrefixKeyJsonValue:0x00562a95c4f638 @adapter=#<Moneta::Adapters::Memory:0x00562a95c15c80 @backend={}>, @prefix="", @features=[:increment, :create]>, @default_expires=nil>, connect_timeout=5, timeout=10, load_paths=["/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/onebox-1.5.43/templates"], twitter_client=TwitterApi>,
 @url="https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW">
[3] pry(main)> preview.to_s
=> "<aside class=\"onebox amazon\">\n <header class=\"source\">\n <a href=\"https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW\">\n \n www.amazon.com\n </a>\n </header>\n <article class=\"onebox-body\">\n <img src=\"https://images-na.ssl-images-amazon.com/images/I/51jzvyPoV6L._SY400_.jpg\" class=\"thumbnail\"/>\n\n<h3><a href='https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW'>Seagate 1TB Desktop HDD SATA 6Gb/s 64MB Cache 3.5-Inch Internal Bare Drive (ST1000DM003)</a></h3>\n\n<p>\n\nBy now the world knows that Seagate Barracuda 7200 drives, from the world&#39;s premier storage provider, deliver years of reliable service and high performance. The ideal choice for workstations, desktop ...\n\n</p>\n<p><strong>\n$49.99\n<strong></p>\n\n </article>\n <div class=\"onebox-metadata\">\n \n \n </div>\n <div style=\"clear: both\"></div>\n</aside>\n"

```

---

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 1:05am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/6 "2016-08-02T01:05:16Z")

</div>

What’s strange to me is that even if I pass in an https url, it does a redirect from http.

I just tried using OpenURI directly:

```
 open ("http://www.amazon.com/gp/aw/d/B005T3GRNW")
RuntimeError: redirection forbidden: http://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW -> https://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:223:in `open_loop'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:149:in `open_uri'
        from /opt/bitnami/apps/discourse/htdocs/lib/freedom_patches/open_uri_redirections.rb:70:in `open_uri'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:704:in `open'
        from /opt/bitnami/ruby/lib/ruby/2.1.0/open-uri.rb:34:in `open'
        from (irb):16
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/console.rb:110:in `start'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/console.rb:9:in `start'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:68:in `console'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/commands.rb:17:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:45:in `require'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:45:in `block in exec_app_rails'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:34:in `loop'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/app_rails_loader.rb:34:in `exec_app_rails'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/lib/rails/cli.rb:5:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/bin/rails:9:in `require'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/gems/railties-4.2.6/bin/rails:9:in `<top (required)>'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/bin/rails:23:in `load'
        from /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.1.0/bin/rails:23:in `<main>'

```

but https works:

```
irb(main):017:0> open ("https://www.amazon.com/gp/aw/d/B005T3GRNW")
=> #<Tempfile:/tmp/open-uri20160802-44470-a00ltr>

```

---

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 1:20am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/7 "2016-08-02T01:20:11Z")

</div>

Looking at the Onebox code, it appears that it is forcing it to be http which is causing OpenURI to blow up

> <https://github.com/discourse/onebox/blob/main/lib/onebox/engine/amazon_onebox.rb#L13-L20>

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [August 2, 2016, 1:26am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/8 "2016-08-02T01:26:49Z")

</div>

Aha if that is the case we should use protocol agnostic URL there. Can you add a quick fix to onebox @techAPJ?

---

<div class="post-metadata">

### Author: ![tarr11](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tarr11/32/105425_2.png) [@tarr11](https://meta.discourse.org/u/tarr11)
#### Post date: [August 2, 2016, 1:39am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/9 "2016-08-02T01:39:01Z")

</div>

Hm, I’m not yet sure this is the problem. If it were, this would happen to everyone, which it obviously doesn’t…

---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [August 2, 2016, 3:23am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/10 "2016-08-02T03:23:12Z")

</div>

Just make it always https.

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [August 2, 2016, 7:16am UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/11 "2016-08-02T07:16:10Z")

</div>

It isn’t forcing it to be HTTP but rather maintaining the same scheme as the URL that was provided 🙂

Anyway we’ll now force HTTPS on all URLs that go through Amazon onebox.

[https://github.com/discourse/onebox/commit/0fd5ededa05961833ce45a243e7e9bea7b63f5f7](https://github.com/discourse/onebox/commit/0fd5ededa05961833ce45a243e7e9bea7b63f5f7)

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [August 19, 2016, 10:05pm UTC](https://meta.discourse.org/t/onebox-404-on-amazon-link-redirection-forbidden/48092/12 "2016-08-19T22:05:15Z")

</div>