יש עדכונים לגבי הדרך הטובה ביותר ליצור ארכיון HTML של אתר סטטי?

סיכום מבוסס בינה מלאכותית

In a discussion titled “Any updates on the best way to create a HTML archive of a static site?”, user justin_gordon sought a method to create browseable HTML files for retiring their Discourse forum. While exploring options like the Discourse Archive and the ArchiveDiscourse script, they encountered issues with broken image links and internal URLs pointing to the decommissioned server justin_gordon. pfaffman suggested using wget with specific flags to mirror the site, though it required obtaining the session cookie _t pfaffman. Another user, 翔_贺, shared a Ruby snippet for serving static files, while supermathie noted that a custom exporter could be built from backups. Finally, manuel recommended discourse-static-archive as a promising solution that handles topics, categories, and search, with pfaffman providing a refined wget command using a cookie extraction script for those preferring that method.

Update!

This might be the answer:
https://github.com/jamesob/discourse-archive

I looked at:

Improving Discourse static HTML archive.

It’s old.

I’m going to retire https://forum.talksurf.com/.

Yes, I’m going to archive a backup.

But what if I just want some browseable HTML files?

Should I just run ArchiveDiscourse/archive-discourse.py at master · kitsandkats/ArchiveDiscourse · GitHub ?

Or is there something better?

Thanks in advance!

CC: @pfaffman

Aloha,

Justin

לייק 1

האם משהו כמו וייבק משין (Wayback Machine) דומה?

This worked. I had to make a slight code update.

https://github.com/kitsandkats/ArchiveDiscourse/pull/4

3 לייקים

But not much older than your Discourse version!

I have had some luck mirroring sites with wget. Something like

wget --mirror --page-requisites --convert-links --adjust-extension --compression=auto --reject-regex "/search" --no-if-modified-since --no-check-certificate --execute robots=off --random-wait --wait=1 --user-agent="Googlebot/2.1 (+http://www.google.com/bot.html)" --no-cookies --header "Cookie: _t=$COOKIE" https://forum.talksurf.com/

But you need to get the cookie named _t

Send me an email and I’ll see what I can do.

לייק 1

עשיתי את זה לאחרונה, וכך עשיתי זאת.

 def serve
    file_path = File.expand_path(
      params[:path]+"."+params[:format],
      File.join(File.dirname(__FILE__), "../../public")
    )
    if File.file?(file_path)
      send_file file_path, type: "text/html", disposition: "inline"
    else
      render plain: "404 Not Found", status: 404
    end
  end

רק כדי לעדכן אותך, זה לא מושך את התמונות עם כתובות URL חדשות. התמונות עדיין יצביעו על השרת שלך (שעומד להיות מושבת!).

Jay kindly sent me the dump, and I compared it to mine.

His technique works better in the sense that it saves the images.

However, his internal links don’t point to the articles, but rather to the decommissioned site. However, the articles can be found with images.

It would be a “nice to have” if Discourse supported a static export. :smile:.

2 לייקים

The good thing is that you have all the data, so one could be written that exported the data directly from a backup if anyone had the inclination to do so.

But we’re not likely to write one :wink:

לייק 1

It shouldn’t be too hard to fix the internal links, add it looks like they just need .html added

I’d still love to see a good archive generator for Discourse.

Just let Claude have a run at it, sharing here for reference and keeping this feature in the loop:

https://github.com/nolosb/discourse-static-archive

Its a basic setup so far: topics, categories, tags and users. Search also works. Uploaded an archive of a local demo instance to a github page:

3 לייקים

That is probably better than this.

Paste this to assign the cookie to COOKIE and then paste the cookies copied to clipboard by the Cooke Chrome extension.


COOKIE=$(cat |jq -r '.[ ]|select(.name == "\_t")|.value')

enter the URL here:

then paste:


wget --mirror --page-requisites --convert-links --adjust-extension --compression=auto --reject-regex "/search" --no-if-modified-since --no-check-certificate --execute robots=off --random-wait --wait=1 --user-agent="Googlebot/2.1 (+http://www.google.com/bot.html)" --no-cookies --header "Cookie: \_t=$COOKIE" =FORUMURL=

4 לייקים