Migrated from XenForo to Discourse: Overview of the Migration

[SubStrider]

We’ve successfully migrated our community (techenclave.com) from XenForo to Discourse.

Total Posts: 2.5 million
Total Users: 79K (active and banned)
Total Private Messages: 0.7 million

It is a 20 year old community so it has seen its fair share of migrations from Proboards, Vbulletin, IPB, Xenforo and now Discourse.

Took us 4 days to complete the migration. But it had 4 weeks of migration prework and 4 weeks of custom plugin development prior to that.

Thank you everyone for helping out and guiding wherever we got stuck.
Overall it was a wildly successful migration with almost no useful data left behind.

---

The core of our migration was built upon an enhanced XenForo import script, significantly improving upon a standard base. We also leveraged several specialized supporting scripts to handle specific data transformations and ensure data integrity post-import.

Key Enhancements in the Main XenForo Import Script

Performance Optimization (Keyset Pagination): The most critical enhancement is the adoption of keyset pagination (WHERE id > last_id). This method drastically improves batch processing speed compared to traditional OFFSET queries, especially on large datasets, by using indexed primary keys for fetching the next set of records.

Robust Checkpointing and Resumption: Advanced checkpointing logic (using .json files) was implemented for users, topics, replies, and private messages. This allows the import process to safely resume from the last successfully imported record after interruptions (e.g., server restarts, script errors), saving significant time.

Two-Pass Post Import Strategy: Posts are now imported in two distinct passes:

Topics First: All original XenForo threads (first posts) are imported, ensuring that parent topics exist before replies.

Replies Second: Subsequent posts within threads are then imported, linking them correctly to their newly created Discourse topics. This structured approach minimizes orphaned replies and enhances data consistency.

Comprehensive Reaction/Like Import:

The script now intelligently differentiates between XenForo reactions that map directly to Discourse’s core “Likes” (hearts) and those that are custom reactions.

It supports bulk insertion of both core likes (into post_actions table) and custom reactions (leveraging the discourse-reactions plugin), significantly speeding up the process.

Includes a mapping for XenForo emoji shortnames to Discourse equivalents (e.g., thumbsup to +1, heart_eyes to heart).

Enriched Marketplace Data Import (Custom Plugin):

A dedicated section was added to import detailed marketplace listing data (e.g., price, location, condition, warranty, payment options) from XenForo’s thread_field_value table.

This data is stored in a custom TecencMarket::Listing model and as topic custom fields, enabling richer display and functionality on Discourse.

Marketplace Feedback Import (Custom Plugin):

A new function was introduced to migrate user feedback/ratings (likes/dislikes) associated with marketplace transactions.

Thread Prefix to Tag Conversion: The script now automatically converts XenForo’s thread prefixes into Discourse tags. This is crucial for maintaining content organization and discoverability, using the discourse-tagging plugin. It also correctly maps prefix IDs to human-readable titles using XenForo’s phrase table.

Enhanced Post Content Processing: This method received significant updates for better Markdown conversion and handling of XenForo’s complex BBCode:

Improved Quote Handling: Conversion of XenForo’s [QUOTE] tags, including handling of quoted users and linking to specific Discourse posts/topics when possible.

Corrected Media Embeds: Ensures that all imported media (YouTube, Twitter/X) are on their separate lines to enable Discourse’s oneboxing/embedding feature, and converts old BBCode [MEDIA] tags to standard URLs.

Table Conversion: Convert XenForo’s [TABLE] BBCode into Markdown tables.

Attachment Handling (Inline & Appended): The attachment processing logic was refactored. It now attempts to replace [ATTACH] tags with the correct Markdown for the uploaded file. Crucially, any attachments that were not explicitly tagged [ATTACH] in the post content are now appended at the end of the post under a clear “Attachments:” header, ensuring no files are lost.

Emoji/Smiley Conversion: Expanded mapping for XenForo smileys to native Discourse emojis, improving visual consistency.

General BBCode Cleanup: More comprehensive removal and conversion of various BBCode tags (e.g., [B], [I], [URL], [IMG], [LIST], [CODE], [COLOR], [FONT], [SIZE], [INDENT], [USER]).

Character Encoding & Scrubbing: Includes .scrub! for invalid UTF-8 sequences and CGI.unescapeHTML for proper HTML entity decoding.

Suspended User Handling: Banned XenForo users are now imported as suspended Discourse users, retaining their accounts but limiting their activity, with ban reasons preserved in custom fields.

Bookmarks Import: Import all the bookmarks from XenForo to Discourse. There weren’t many to begin with.

---

A large section of users were not happy after the migration. Which is expected. Its not something you all have not heard before after a migration. Nevertheless, will share a summary of the feedback later in this topic, there might be some food for thought to feed into the future of discourse.

[schneeland]

Thanks for sharing! A lot of stuff on the list that I identified as action items for an ongoing SMF to Discourse migration project

Is your customized importer code available to the public? I would especially be interested in how you handled the two-pass import to ensure correct new links.

[SubStrider]

GitHub - rahul-rakesh/xenforo-migration-scripts

You can refer to the import scripts I used. The file you want to see is this:
xenforo_import_enhanced.rb

[SubStrider]

@schneeland

the two-pass import to ensure correct new links.

The reason we had to do two pass was because over time invariably an old topic will get merged into a new topic. But the posts will have the old ID and will get picked up first. With no corresponding topic imported by then, all those posts get dropped and lost.

[schneeland]

Thanks a lot! Will take a look.

Also, I appreciate the additional comment about the thoughts behind this. That is a problem we luckily didn’t encounter. The main reason I was thinking about a two-pass import is that the hyperlinks in quotes still have the old URL - and while permalinks should also work, I think it would be nicer to use the new URL to posts right away.

[cocococosti (Constanza Abarca)]

Wow! Congrats! And thank you for sharing.

@SubStrider

A large section of users were not happy after the migration.

It is expected, yeah. The way that you handle their feedback now is crucial. Address their comments, make sure they feel heard, and highlight the improvements and new features (I’m sure there are many!). If you have any trouble with the feedback, post it here, and I’d be happy to help.

Take a look at the last section of this article, it’ll give you some pointers: How to prepare your community for a Discourse migration

[bhegnauer (Brent Hegnauer)]

After four failed attempts with the stock importer, your version worked beautifully.

[SubStrider]

@SubStrider

Nevertheless, will share a summary of the feedback later in this topic, there might be some food for thought to feed into the future of discourse.

Sharing this summary as promised. Used Gemini 2.5 Pro to summarize all feedback collected in the last 3 months.

I would say the biggest concern has been a good theme. I wish there was a paid themes marketplace on Discourse like on other forum platforms.

---

Summary

The feedback on the migration to Discourse is strongly divided, but a majority of vocal users express significant frustration, leading to reduced engagement. The most critical issues are centered on Navigation & Usability and UI & Design, particularly for desktop users accustomed to traditional forum layouts. While some users appreciate the modern look and technical improvements, the consensus among critics is that the platform prioritizes form over function, creating a high cognitive load and a less intuitive experience.

---

1. Navigation & Usability (Highest Priority)

This category received the most frequent and detailed negative feedback. Users find the new platform fundamentally harder to use, which is a direct cause of their reduced engagement.

• Key Feedback Points & Weightage:

  • Infinite Scroll is Unpopular: The strongest point of consensus is the dislike for infinite scroll, especially on desktop. Users miss the context and control of page numbers for navigating long threads.
  • Generally Difficult to Navigate: A significant number of users describe the platform as confusing, cluttered, tedious, and having a steep learning curve they don’t wish to invest in for a forum.
  • Difficulty Finding Last/Unread Posts: A common struggle is the inability to intuitively jump to the last post or resume reading from a specific point in a thread, requiring users to learn non-obvious workarounds.
  • Poor Content Discovery: Users miss the old categories page, which provided a clear, hierarchical overview of the forum’s structure and latest activity within sections. The new layout is seen as a “mishmash.”
  • Broken Quoting: The process of quoting text is described as difficult and flaky, especially on mobile and tablets where the quote pop-up can be obscured by the native UI.

• Actionable Insights:

  • Prioritize a “Traditional Forum” Experience: The core issue is the deviation from the expected forum model. A theme or layout that emulates a paginated, more structured view, especially for the categories page, would address the most significant complaints.
  • Improve Navigation Cues: Make jumping to the first, last, and last-unread post an obvious, one-click action. The current scrollbar functionality is not intuitive for many. Consider adding explicit buttons at the top and bottom of threads.
  • Re-evaluate the Default Quoting Behavior: The need to manually select text to quote is a major friction point. Investigate plugins or settings that make quoting a full post the default, one-click action.

---

2. UI & Design (High Priority)

Users feel the current design wastes space and harms readability, directly impacting their ability and desire to consume content.

• Key Feedback Points & Weightage:

  • Excessive Whitespace: This was the most cited design flaw. Users on larger desktop monitors feel the narrow central column wastes significant screen real estate and reduces information density.
  • Desire for Customization: Users are actively seeking ways to improve their experience through custom CSS, different themes (Default over Horizon), and smaller font sizes. This indicates a strong desire for more user-level control over the appearance.
  • Poor Readability: The combination of whitespace, font sizes, and lack of clear separation between posts makes it harder to scan and read content compared to the previous software.

• Actionable Insights:

  • Develop a “Compact” or “Full-Width” Theme: Address the whitespace complaint directly by offering an official theme that utilizes more of the screen width. This would also satisfy the desire for customization and is the most common workaround users are implementing themselves.
  • Refine Visual Separators: Increase the visual distinction between posts. Subtle changes like borders, alternating background colors, or different user-info block layouts can significantly improve readability and reduce the “chat-like” feel.
  • Review Font Sizing and Hierarchy: Conduct a review of the theme’s typography to ensure a more consistent and information-dense hierarchy, particularly for subcategory text and post content.

---

3. General Sentiment & User Engagement (High Impact)

A significant portion of the feedback indicates a direct, negative impact on user activity. While some of this is resistance to change, the consistency of the complaints suggests deeper issues.

• Key Feedback Points & Weightage:

  • Negative Sentiment / Usage Dropped: This is a critical metric. Multiple users explicitly state they visit the forum less often now.
  • Positive Sentiment: A smaller but notable group enjoys the modern feel, speed, and features like the notification system and markdown editor.
  • Adaptive Users: This group misses the old forum but understands the technical reasons for the migration and is willing to adapt.

• Actionable Insights:

  • Acknowledge and Communicate: Publicly acknowledge the user feedback and communicate a clear roadmap for UI/UX improvements. This can help retain users who are currently frustrated but willing to adapt.
  • Prioritize “Quality of Life” Fixes: Focus on low-effort, high-impact changes first (e.g., changing category colors to match the old theme as suggested by one user) to show that feedback is being heard and acted upon.
  • Create a “Getting Started” Guide: While some users resist reading guides, a pinned topic that quickly explains the top 5 most confusing changes (like navigating threads and quoting) could ease the transition for many.

---

4. Core Functionality & Performance (Medium Priority)

While not as frequently mentioned as UI/UX, these issues affect the core purpose of the forum.

• Key Feedback Points & Weightage:

  • Chat-like Feel: The design encourages short, reactive replies rather than long-form, structured discussion, making it feel more like a chat app than a forum.
  • Poor Search: Users report that search does not effectively prioritize results from topic titles, making it hard to find known threads.
  • Performance Issues: One user noted that the text editor becomes laggy and causes their laptop to heat up on long threads.

• Actionable Insights:

  • Tune Search Relevance: Investigate Discourse settings or plugins that can weigh topic titles more heavily in search results. The admin’s suggestion to use advanced syntax is a workaround, not a solution for default search behavior.
  • Monitor Performance on Long Threads: Keep an eye on the performance of the editor and scrolling on threads with thousands of replies to address potential client-side bottlenecks.
  • Enable AI Search: As suggested by the admin, enabling AI-powered search could be a long-term solution to the search relevance problem, but it should be weighed against potential costs.

[NateDhaliwal]

@SubStrider

I wish there was a paid themes marketplace on Discourse like on other forum platforms.

Why not Marketplace ? Though the result is usually private.

[Bas (Bas van Leeuwen)]

@SubStrider

Poor Content Discovery: Users miss the old categories page, which provided a clear, hierarchical overview of the forum’s structure and latest activity within sections. The new layout is seen as a “mishmash.”

For these users, you might want to consider explicitly calling out the categories overview. Or even consider switching that overview to something that they like better. It’s an opportunity to get them involved too.
Share some screenshots of what the different settings of Desktop category page style would look like and ask their favorite?

@SubStrider

Broken Quoting: The process of quoting text is described as difficult and flaky, especially on mobile and tablets where the quote pop-up can be obscured by the native UI.

This feels like feedback we’d like to get explicitly. Could you ask them to report this here on Meta?

@SubStrider

Excessive Whitespace: This was the most cited design flaw. Users on larger desktop monitors feel the narrow central column wastes significant screen real estate and reduces information density.

This is fixable with user themes Consider adding this theme component to a user selectable theme GitHub - discourse/discourse-full-width-component: Make Discourse occupy the full browser width

@SubStrider

Poor Readability: The combination of whitespace, font sizes, and lack of clear separation between posts makes it harder to scan and read content compared to the previous software.

I don’t understand “clear separation of posts”?

@SubStrider

• Negative Sentiment / Usage Dropped: This is a critical metric. Multiple users explicitly state they visit the forum less often now.
• Positive Sentiment: A smaller but notable group enjoys the modern feel, speed, and features like the notification system and markdown editor.
• Adaptive Users: This group misses the old forum but understands the technical reasons for the migration and is willing to adapt.

Change is always hard, people will always complain. If the “negative” group is small enough, I think you could safely ignore them for now and focus on the adapters. Make their experience great and the negative people will fall in line (or leave).

[SubStrider]

With a themes marketplace the cost is divided between multiple buyers. The theme creator also has incentive to create many themes.

[SubStrider]

@Bas

This is fixable with user themes Consider adding this theme component to a user selectable theme

The component does not really offer full width. It bunches the layout to the left, but does not expand it. I have implemented one custom theme which does expand, but wish it was possible for all themes.

With full width component on Horizon

With the custom theme based on Foundation

You are right though, change is hard and a lot of users have adjusted, some are still finding it difficult. its tough to satisfy everyone. Just thought I will share the whole experience.

Will share more details on the quote issue on mobile/tablets.