ICS → Discourse 가져오기 도구

iCalendar(ICS) 피드에서 이벤트를 Discourse 카테고리로 지속적으로 동기화하는 작은 유틸리티를 만들었습니다.

이것은 완전한 Discourse 플러그인이 아니라 Discourse 설치와 함께 실행되므로, Customization > Extras 섹션에 올려야 합니다. 외부 소스(예: Google 캘린더, 대학 시간표 피드 등)의 캘린더 이벤트를 Discourse 토픽 내에서 표시하고 싶다면 유용할 것입니다.

저장소

작동 방식

  • 주어진 ICS 피드에서 이벤트를 읽습니다.
  • 기존 토픽과 매칭합니다 (UID 또는 시간/장소로 폴백).
  • 선택한 카테고리에서 토픽을 생성하거나 업데이트합니다.
  • systemd 서비스로 지속적으로 실행할 수 있습니다 (flock을 통해 중복 실행 방지).

요구 사항

  • Ubuntu 24.04 LTS (테스트 완료)

  • Python 3 (Ubuntu 24.04 LTS에 이미 포함됨)

  • Discourse API 키

  • 이벤트 토픽을 대상화할 카테고리 ID

출력 예시

대학 시간표 ICS 피드를 Discourse로 동기화했을 때의 모습은 다음과 같습니다:

빠른 시작

저장소를 클론하고 요구 사항을 설치하세요:

git clone https://github.com/Ethsim12/Discourse-ICS-importer-by-REST-API.git /opt/ics-sync
cd /opt/ics-sync
pip install -r requirements.txt

동기화를 수동으로 한 번 실행하세요:

python3 ics_to_discourse.py \
  --ics "https://example.com/feed.ics" \
  --category-id 4 \
  --site-tz "Europe/London" \
  --static-tags "events,ics"

지속적 동기화를 위해 systemd 서비스/타이머로 설정하세요 (예제 설정은 저장소에 있음).

3개의 좋아요

the tags were annoying me, so i made sure the search.json look for indexed content of the event - first post of each topic/event

https://github.com/Ethsim12/Discourse-ICS-importer-by-REST-API/commit/06cbbdfc0c30f3f30605cecdf7cdba6842a6a001

https://github.com/Ethsim12/Discourse-ICS-importer-by-REST-API/commit/dea52fc1625d54956f9ad59344f5818a0628d3a7

https://github.com/Ethsim12/Discourse-ICS-importer-by-REST-API/commit/ed64d3fef6049913fadec7a5cb6f6d546b54f7a6

1개의 좋아요

Thank you again for the share, this calendar is evolving more and more, getting new features thanks to people like you. I wonder how it will be like in 3-5 years :slight_smile:

1개의 좋아요

Brilliant! Thanks for testing it out. Anyone else who wants to try syncing an ICS feed into Discourse, I’d love feedback on whether your feeds behave the same.

2개의 좋아요

A couple of comments.

If I had any time, I’d probably try converting this to a proper plugin. I think it shouldn’t be too hard to create some settings and convert the Python into Ruby and put it in a job.

Another idea, which could be useful for people who are hosted and want to use this, would be to convert the task into a github action and get it to run the task daily. I did this for some scripts a hosted client needed to run daily a while back and it’s working pretty well. It’s at once harder (it requires learning github workflows and how to deal with secrets instead of a good old cron job) and easier (you don’t have to learn how to muck with installing stuff on a machine via a command line interface).

2개의 좋아요

I haven’t tested it lately, but wrapped up the event bbcode parsing in my latest commit to

https://github.com/Ethsim12/discourse-ics-sync

yes, though it would be nice if the ics_feeds setting were to be broken down, so the admin isn’t inputting a single JSON into UI

1개의 좋아요

to be honest i don’t use cron now, i use systemd on a Ubuntu Server 24.04 LTS.

1개의 좋아요

this is a luxury that as soon as i have the time i will learn to achieve :wink::face_exhaling:

Not having access to a command line is, IMHO, no luxury at all! :rofl:

1개의 좋아요

Haha, to be clear I meant GUI is the real luxury - CLI is the skill I need to work toward.

1개의 좋아요

I guess @angus beat you to that by a few years
https://discourse.angus.blog/t/import-events-with-icalendar/53

3개의 좋아요

Behaviour notes from testing ics_to_discourse.py

I’ve been running a series of tests on this script (with and without --time-only-dedupe) and thought it would be useful to document the update/adoption flow in detail.


1. How uniqueness is determined

  • Default mode: adoption requires start + end + location to match exactly.
  • With --time-only-dedupe: adoption requires only start + end; location is treated as “close enough.”

If no existing topic matches these rules, a new topic is created.


2. The role of the UID marker

  • Every event topic gets a hidden HTML marker in the first post:
  <!-- ICSUID:xxxxxxxxxxxxxxxx -->
  • On subsequent runs, the script looks for that marker first.
  • If found, the topic is considered a UID match and updated directly, regardless of how noisy or stale the DESCRIPTION text might be.
  • This makes the UID the true identity key. Visible description fields don’t affect matching.

3. Update flow with UID match

  1. Script fetches the first post and strips the marker:
old_clean = strip_marker(old_raw)
fresh_clean = strip_marker(fresh_raw)
  1. If old_clean == fresh_clean: no update (avoids churn).
  2. If they differ: check whether the change is “meaningful”:
meaningful = (
    _norm_time(old_attrs.get("start")) != _norm_time(new_attrs.get("start"))
    or _norm_time(old_attrs.get("end")) != _norm_time(new_attrs.get("end"))
    or _norm_loc(old_attrs.get("location")) != _norm_loc(new_attrs.get("location"))
)
  • If meaningful = True → update with bump (topic rises in Latest).

  • If meaningful = False → update quietly (bypass_bump=True → revision only, no bump).

    1. Tags are merged (ensures static/default tags are present, never removes moderator/manual ones).
    2. Title and category are never changed on update.

  1. Update flow with no UID match
    1. Script attempts adoption:
      • Builds candidate triples of start/end/location (or start/end only with --time-only-dedupe).
      • Searches /search.json and /latest.json for an existing event with matching attributes.
      • If found → adopt that topic, retrofit UID marker + tags (body left unchanged at this stage).
      • If not found → create a brand new topic with the marker and tags.
    2. Once adopted or created, all future syncs will resolve directly by UID.

  1. Practical consequences
    • Time changes
    • Default: adoption fails (times differ) → new topic created.
    • With --time-only-dedupe: adoption fails the same way; new topic created.
    • Location changes
    • Default: adoption fails (location differs) → new topic created.
    • With --time-only-dedupe: adoption succeeds (times match), but location difference is flagged as “meaningful” → update with bump.
    • Description changes
    • If DESCRIPTION text changes but start/end/location do not:
    • Body is updated quietly (bypass_bump=True).
    • Topic revision created, but no bump in Latest.
    • If DESCRIPTION is unchanged (or only noise such as Last Updated: that normalizes away), no update occurs at all.
    • UID marker
    • Ensures reliable matching on future syncs.
    • Means noisy DESCRIPTION fields don’t affect whether the correct topic is found.

  1. Why the DESCRIPTION sometimes “stays the same”

The script compares the entire body (minus the UID marker).
If only a volatile line like Last Updated: is different, but it normalizes away (e.g. whitespace, line endings, Unicode), old_clean and fresh_clean appear identical → no update is made.
This is by design, to prevent churn from feed noise.


Summary

  • Time defines uniqueness (always creates new topic when times change).
  • Location changes → visible bump (so users notice venue updates).
  • Description changes → quiet update (revision but no bump).
  • UID marker = reliable identity key, ensures the correct topic is always found, even if DESCRIPTION is stale or noisy.

This strikes a good balance: important changes surface in Latest, unimportant churn stays invisible.

Looking back, it’s kind of hilarious how this whole saga unfolded.
The importer script itself is now rock-solid: UID markers, dedupe logic, meaningful vs. quiet updates, tag namespaces… all the stuff you’d actually want in production. The behaviours line up perfectly with the notes i posted — times define uniqueness, locations trigger a bump, descriptions update quietly, and UID markers keep everything anchored. It’s elegant, it’s predictable, it’s done. :white_check_mark:

Meanwhile, the poor Meta topic that hosted it all was… well, doomed.
It began life replying as a sockpuppet (strong start :socks:), ballooned into a Frankenstein thread of code dumps and screenshots, then evolved into a pseudo-changelog with more commits than the repo itself. And just as the script finally became stable? Scheduled for deletion. :skull:

Honestly, it’s poetic. The script’s entire purpose is to stop duplicate events from cluttering up your forum. The topic itself? Seen as a duplicate, quietly marked for garbage collection. The very fate it was built to prevent became its destiny. :wastebasket:

So here’s to the doomed topic:
You didn’t bump Latest, but you bumped our hearts. :heart:

2개의 좋아요

How did you get on with moving it to a Discourse plugin? Or better yet, as a PR on the existing Discourse Calendar and Events Plugin?

I’m reluctant to jump into the config and maintenance required to run your awesome looking script as is (and suspect that many self-hosters would be in the same boat).

1개의 좋아요

How is this script better than the plugin? (Oh, maybe you can’t install plugins?) If the plugin doesn’t do what’s required, maybe submit a PR?

Thanks for the nudge!

Quick status: I’m currently running three instances of my Python ICS→Discourse importer (Uni timetable, Sports Centre bookings, and an Outlook calendar). I did start wrapping it as a Discourse plugin, but the plugin version fell short of the script’s feature-set — mainly because each feed needs bespoke handling (UID quirks, partial updates, cancellations, noisy revisions, etc.). Angus’s plugin is great for many cases; my use cases seem more “feed-specific”.

I also have an open PR against core aimed at reducing the “Latest” blue button noise during large/bursty ICS updates. With busy feeds (like university timetables) a batch of low-value edits can keep “Latest” bouncing; the PR effectively no-ops the “New Topics” button when Latest has sat open while an automated batch runs. Happy to cross-link that PR here if useful.

Longer term: I’m on self-hosted IONOS right now. If I move to official hosting later, I’d still love a way to keep the Python flow (or an equivalent) without needing Enterprise features, if ICS inbound exists there. I suspect a generic core/plugin solution could work if it allowed pluggable “adapters” per feed while keeping strong idempotency (ICS UID), cancellation handling, and edit-without-bump semantics.

If there’s interest, I can sketch a minimal adapter interface and a migration path from my Python script to a Ruby job, or contribute feed-agnostic pieces (UID mapping, debounce/no-bump updates, cancellation logic) to the calendar/events plugin.

1개의 좋아요

That’s a good question, Nathan — and I think there’s definitely space for a minimal, feed-agnostic approach that could live either as a small extension to the Calendar/Event plugin or as a lightweight core job.

For a PR to be generally useful, the key seems to be making the importer adapter-based rather than feed-specific. Something like:

  • Each feed defines a small adapter (could be Python, YAML, or Ruby) that maps ICS fields → Discourse topic fields (title, body, tags, start, end, location, etc.).
  • Core handles idempotency (UID ↔ topic ID mapping), cancellation (STATUS:CANCELLED), and quiet edits (update without bumping Latest).
  • Plugins or site settings could configure polling interval, tag mappings, and bump policy (always, never, on major change).

That way, institutions with noisy or complex feeds (university timetables, room bookings, Outlook calendars, etc.) can provide an adapter suited to their data without hardcoding anything in core.

If there’s interest, I’d be happy to outline that adapter interface or prototype the core “ICS upsert” helper as a Ruby job that others can build on — so that this can gradually evolve from standalone Python scripts to something maintainable and generic within Discourse’s ecosystem.

2개의 좋아요

no longer with the following commit, Thanks Discourse!

https://github.com/Ethsim12/Discourse-ICS-importer-by-REST-API/commit/5ed4cb67735076d883bb46db643b4445282d9662

3개의 좋아요

동작의 미묘한 차이: --time-only-dedupe는 진정한 의미의 “시간 전용”이 아님

추가 테스트를 통해 확인된 미묘하지만 중요한 세부 사항:

  • --time-only-dedupe를 사용할 경우 매칭이 단순히 시작/종료 시간에만 기반하지 않음
  • 여전히 위치가 “충분히 가깝게”(close_enough_loc()을 통해) 일치해야 함

이로 인해 유용한 동작이 나타남:

  • 위치의 사소한 노이즈(형식, 중복 등) → 동일한 토픽이 업데이트됨
  • 실제 위치 변경(예: C05 → C04) → 새로운 토픽이 생성됨

실질적 효과

즉, 다음과 같은 결과를 낳음:

  • 방 변경은 Latest에 표시됨(새 토픽 생성 → 사용자에게 노출)
  • 피드 노이즈는 보이지 않음(조용한 업데이트 또는 무의미한 작업)

따라서 이 시스템은 신호 대 노이즈 필터 역할을 하게 됨:

  • 시간은 정체성을 정의
  • 위치 변경은 의미 있는 것으로 처리
  • 설명의 빈번한 변경은 무시