Hosting many forums on a single Discourse instance with a single database

Contribute Feature

AI-generated summary

The discussion revolves around hosting many forums on a single Discourse instance with a single database. jmay is interested in hosting hundreds or thousands of small forums, and is exploring possible solutions.

jmay outlines the necessary features for implementing this, including scoping topics, categories, groups, badges, and notifications to specific forums. sam suggests that using namespaces could be a possible solution, but notes that it would require significant development work.

jmay considers using the Guardian class to restrict access to subsets of the database, while sam advises against adding scope columns to every table and instead suggests using the apartment gem.

However, jmay is hesitant to use the apartment gem due to the potential complexity and performance issues. tobiaseigen proposes a different approach, where each discourse forum is independent and separate, but can be hosted as a group on a larger forum.

jmay eventually considers using categories and groups to implement the multi-forum feature, which would involve nesting categories and groups, and tying moderator access to categories. sam supports this approach and suggests that implementing category-specific moderators would be a good starting point.

jmay agrees to work on this feature and is instructed by sam to create a detailed post on the feature topic, which can be reviewed and moved to the spec category.

1

I’m trying to understand if this is possible with a typical Discourse installation. I’ve read the topics that discuss multi-site installs - this seems to involve setting up a separate database instance for every subdomain.

I’m interested in hosting a large number (hundreds or thousands) of small (~dozens of participants, peak volume likely tens of messages per day) forums. Would be nice if possible to do this without needing to create a separate database for each forum. Thinking about Discourse as an alternative to creating lots of teeny Yahoo/Google groups.

If this is not possible out-of-the-box, is this a feasible customization challenge?

1개의 좋아요
Multiple forums on a single Discourse instance
2

Multisite could be hacked to use namespaces, but doing something like what you are describing would require a dedicated dev.

If you can afford a very competent developer for 2+ months of work I think you could probably get this off the ground.

1개의 좋아요
3

@sam, are you saying that the namespace hack might take 2 months - or that using namespaces would be quick but there’s some better solution that would require more substantial changes and that would take time.

I would consider myself a competent developer … but without any familiarity with the Discourse code. Perhaps time to get my hands dirty. Presumably this is more complicated than adding a forum-id column to the Topics table but that might be the place to start?

4

Some thoughts on implementation of this feature:

  • any implementation of this must be entirely backward-compatible with the current (single forum) Discourse structure
  • Topics, Categories, Groups, Badges and Notifications must all be scoped to a specific forum. Anything else?
  • Site Settings need to be (all? most?) scoped to a forum
  • user must be able to get a list of forums they are participating in; this probably belongs on the Profile page
  • home page behavior must be different when there are multiple forums; what to show before user chooses a forum to enter?
  • searches must be (always? optionally?) scoped to a forum
  • admin/moderator status must be scoped by forum
  • Profile options probably need to be a combination of global and per-forum
  • email digests should consolidate activity across all forums in which the user participates

Plenty of work here. Are there any other particularly scary aspects that I’m missing?

5

I am experimenting with this feature as I explore the Discourse codebase. Comments welcome!

multiforum feature branch on Github

1개의 좋아요
6

You definitely would not do this, this way. Its too hard, risky and there is no chance we will add a “scope” column to every table, never going to happen.

Instead look at apartment gem and how it does schema switching, you would have to port that in conditionally to rails-multisite.

You would need to handle provisioning and comms and a bunch of other stuff.

3개의 좋아요
7

I’m thinking of an environment where there are thousands of forums, most with very light activity. Creating entire sets of tables for every one of those (as in the apartment gem) is terrifying. e.g. on software updates needing to run the migrations against N tenants. And I want an environment where provisioning a new forum is a fast & lightweight operation.

But I see your point on adding scope columns to the major tables. Probably not to everything, but it would be nice to avoid it even on the main ones. And then needing to add .where(forum: current_forum) everywhere just dirties up the codebase.

I’m going to keep hunting for alternatives. Would like to minimize overall impact.

1개의 좋아요
8

The Guardian class looks very interesting. Perhaps that could be extended to include the concept of restricting access to subsets of the database? Separate set of scoping tables, only used when the multi-forum feature is turned on.

9

You realize we have groups etc?

10

I was gonna say… worth taking a look first at groups, categories and visibility of categories to groups, etc

11

Yes, I know about groups. But there’s no nesting of groups. There’s some other conversation about subgroups on here, but I’m not sure that’s a good approach for this feature.

My mental model for this is for schools. One forum for each school, and groups for things like class membership, after-school activities, carpool groups, PTA, etc.

If it’s possible to stretch the existing framework to handle multi-forums, I’d rather do it that way than add a whole lot of new machinery.

12

I am interested in this discussion as well and would be into exploring the use cases more. It seems to me that each discourse forum should be independent and separate. But maybe not right away. At first as it is being evaluated and new forum owners and members learn the ropes it would be hosted as a group on a forum run by someone else for them, like wordpress.com for new bloggers. Eventually as the group grows and takes wings, and needs grow, it could be moved to its own independent forum.

How this would work I am not sure, but the ability to export part of a forum, Eg a group and all related users, categories, and topics, would do it.

13

Quick question about your model: Do your “schools” have anything in common? That is, do they belong to an overarching larger organization/grouping, (Like a network of districts), or are they simply unconnected groups that would come to you for quick, easy mini-forum setup?

I ask because understanding the relationship between the various “Schools” could help clarify how deeply these communities should be siloed off from each other.

14

I’m assuming no relationship between schools. So by default the siloing of topics should be complete. Allowing for some degree of “leakage” between school forums (e.g. for schools within the same district) is an interesting idea, but not a strict requirement.

User might be members of multiple forums, and I want to be able to show users an aggregated view of activity across all their forums. In the tenancy model this would require merging results from queries across N databases. I’d like to avoid that.

15

I’m warming up to the idea of trying to do this entirely with categories and groups. That will entail:

  • nesting categories more deeply (since I’m stealing the top level for the multiple forums)
  • probably nesting groups
  • moderator access tied to categories
  • possibly tethering trust level to categories as well

I found another open feature request that relates to this:
https://meta.discourse.org/t/category-specific-moderators/11668

16

Category specific moderators is a feature we certainly want in core. It would be a great place to start, you just need to get that feature converted to a spec and @codinghorror can bless it once its good.

1개의 좋아요
17

Great. I’m in. By spec do you mean text or an actual rspec definition? Do I just post it in here or is there a procedure I should follow?

18

see the spec category:

https://meta.discourse.org/c/feature/spec

You can not create spec category topics, only we can, so… create a detailed post on that feature topic with your proposed spec and we can review it and move it to the spec category once ready.

19

posted on the other feature topic at

https://meta.discourse.org/t/category-specific-moderators/11668/18