Improving Discourse static HTML archive

[saurabhp (Saurabh Patel)]

It is recommended to use HTTrack to take a dump of static HTML and host that as a static archived website. But the layout for crawlers is not very pretty to host it as a static site. I will be working on improving the layout and adding necessary data to the static website. You can see the crawler layout at https://meta.discourse.org/?escaped_fragment which I will try to improve.

This is just a placeholder to link with changes I make so that someone reviewing it can get more context.

Let me know if you have any suggestions on this topic.

Thanks

[saurabhp (Saurabh Patel)]

I have created few pull requests related to this and added screenshots in them:
https://github.com/discourse/discourse/pull/7250
https://github.com/discourse/discourse/pull/7270
https://github.com/discourse/discourse/pull/7286

Let me know if you have any suggestions.

[tgxworld (Alan Tan)]

@saurabhp

It is recommended to use HTTrack to take a dump of static HTML and host that as a static archived website.

Sorry in advanced for my question since I’m not very familiar with HTTrack. Why do we need to use HTTrack to take a dump of the static HTML page and host that as a static archived website?

[saurabhp (Saurabh Patel)]

Hey,
You can go through these links to get more context related to this:

A basic Discourse archival tool

Hi all – just jumping in here to say that @mcmcclur’s code was exactly what I was looking for! So thank you very much for sharing I made a few small modifications (mainly additional code that makes sure to grab all posts in a topic, not just the first twenty) and the code is here: https://github.com/kitsandkats/ArchiveDiscourse, forked from @mcmcclur’s original repo and stored as a python file instead of a Jupyter notebook. I’m very happy with how it turned out. Thanks again!

Archive an old forum "in place" to start a new Discourse forum

When possible, we recommend archiving old forums rather than importing them into Discourse. Sometimes this isn’t possible, for various reasons, but getting a fresh start and moving into a new community platform without carrying all your old baggage across is often appealing. [image] Maybe your community needs a reboot. But how do you: Keep the valuable information at old forums around without keeping that ancient forum software running, too, with all its future security vulnerabilities? A…

HTTrack will basically just crawl your website and create a static HTML dump which you can host as a static website.

Quoting from the link above on why people want it.

@mcmcclur

I’ve been using Discourse for a couple of years now as a discussion board when teaching my college math classes so, every few months, I retire one or two sites and start one or two more. Obviously, the discussions on the retiring sites have value so I really needed some way to save them

Let me know if you have any other questions.

[codinghorror (Jeff Atwood)]

You do not “need” to use httrack tool you can use recursive wget and other similar command line Linuxy spidering tools as well.

[saurabhp (Saurabh Patel)]

Just an update regarding this.

All 3 pull requests have been merged. I’m adding screenshots with the new static archive look here below. Let me know if any of you have any suggestions on things to improve.