# Make Discourse play nice with the Wayback Machine

**URL:** https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579
**Category:** Feature
**Created:** [October 16, 2015, 2:39am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579 "2015-10-16T02:39:46Z")
**Posts on this page:** 10
**Page:** 3

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [May 14, 2020, 1:14am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/49 "2020-05-14T01:14:03Z")

</div>



---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [May 14, 2020, 1:14am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/50 "2020-05-14T01:14:29Z")

</div>

Replay is currently broken, the JS runs and the Ember Router breaks due to the pathname change.

Thanks to the improved browser detection from @david, there is an extremely ugly but also tempting fix to get new captures to render properly: just patch browser-detect so it detects the replay and yanks out the `noscript` version.

[https://github.com/riking/discourse/commit/6a83c83bd3acab37f7a3e24f6aa4a14081bb2249](https://github.com/riking/discourse/commit/6a83c83bd3acab37f7a3e24f6aa4a14081bb2249)

The problem is, if we start serving that script, and by some miracle the JS starts being able to run, all the old archived pages are forced into the no-js view.

~~Now that I write that out, you know, that’s probably not too bad of a price to pay for getting working archive playbacks _today_. ([Draft PR](https://github.com/discourse/discourse/pull/9775))~~ I have been talked out of actually doing this.

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [May 14, 2020, 8:50am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/51 "2020-05-14T08:50:10Z")

</div>

> [@riking](#):
>
> improved browser detection from @david

(Was @dan, not me)

Is our existing wayback machine bypass broken?

> <https://github.com/discourse/discourse/blob/cb8f8de422b8b270dc57f3614d3d9d718bfc40ef/lib/crawler_detection.rb#L18-L18>

---

<div class="post-metadata">

### Author: ![dan](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/dan/32/101549_2.png) [@dan](https://meta.discourse.org/u/dan)
#### Post date: [May 14, 2020, 10:34am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/52 "2020-05-14T10:34:58Z")

</div>

> [@david](#):
>
> Is our existing wayback machine bypass broken?

Is there any particular reason we are not checking for their user agent (`archive.org_bot`)? It seems to be a less fragile solution.

> **[Internet Archive: Digital Library of Free & Borrowable Texts, Movies, Music &...](https://archive.org/details%2Farchive.org_bot%2F)**

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [May 14, 2020, 10:45am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/53 "2020-05-14T10:45:51Z")

</div>

Their “liveweb” thing does not send the user agent, I think:

> [@sam](#):
>
> What happens here technically is that they run a proxy in California that intercepts all the traffic from my browser to meta. This proxy uses the **same** user agent as the one I have, so we think this is coming from Firefox and give it a proper desktop view which is not desirable at all.

---

<div class="post-metadata">

### Author: ![dan](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/dan/32/101549_2.png) [@dan](https://meta.discourse.org/u/dan)
#### Post date: [May 14, 2020, 10:56am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/54 "2020-05-14T10:56:22Z")

</div>

I believe some things changed (see the dates). I think we should be checking both of them.

 ![Screen Shot 2020-05-14 at 13.54.10](https://global.discourse-cdn.com/meta/original/3X/1/1/11012409f322451b768febc54077e0daca02b7d5.png)

 ![Screen Shot 2020-05-14 at 13.54.21](https://global.discourse-cdn.com/meta/original/3X/0/f/0fda80da21aa9486ea9d3c8316367afbdc081095.png)

EDIT: Submitted a PR for this:

[https://github.com/discourse/discourse/pull/9777](https://github.com/discourse/discourse/pull/9777)

---

<div class="post-metadata">

### Author: ![aschrijver](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/aschrijver/32/92004_2.png) [@aschrijver](https://meta.discourse.org/u/aschrijver)
#### Post date: [May 31, 2020, 10:04am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/55 "2020-05-31T10:04:20Z")

</div>

> [@sam](#):
>
> I think sorting this out is very important, cause each time people submit pages to wayback machine it is getting “rubbish” that it is considering adding to its index.

> [@dan](#):
>
> EDIT: Submitted a PR for this:

Would be lovely to see this working again. I am [promoting](https://github.com/solid/process/issues/202#issuecomment-630230640) Discourse as a central hub for [Solid Project](https://forum.solidproject.org), especially for core team members and experts working on standardization of [Solid](https://solidproject.org), but this issue is an [important reason](https://github.com/solid/process/issues/202#issuecomment-630273205) for them to be unwilling to do so.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [May 31, 2020, 10:17am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/56 "2020-05-31T10:17:45Z")

</div>

The pr was merged it should be working

---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [June 2, 2020, 1:54am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/57 "2020-06-02T01:54:32Z")

</div>

Just confirmed by doing a “save outlinks” on a /top/yearly… working fully right now.

---

<div class="post-metadata">

### Author: ![riking](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/riking/32/170938_2.png) [@riking](https://meta.discourse.org/u/riking)
#### Post date: [June 2, 2020, 1:54am UTC](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579/58 "2020-06-02T01:54:36Z")

</div>



[Previous page](https://meta.discourse.org/t/make-discourse-play-nice-with-the-wayback-machine/34579.md?page=2)
