Hosting many forums on a single Discourse instance with a single database

(Jason May) #1

I’m trying to understand if this is possible with a typical Discourse installation. I’ve read the topics that discuss multi-site installs - this seems to involve setting up a separate database instance for every subdomain.

I’m interested in hosting a large number (hundreds or thousands) of small (~dozens of participants, peak volume likely tens of messages per day) forums. Would be nice if possible to do this without needing to create a separate database for each forum. Thinking about Discourse as an alternative to creating lots of teeny Yahoo/Google groups.

If this is not possible out-of-the-box, is this a feasible customization challenge?

Multiple forums on a single Discourse instance
(Sam Saffron) #2

Multisite could be hacked to use namespaces, but doing something like what you are describing would require a dedicated dev.

If you can afford a very competent developer for 2+ months of work I think you could probably get this off the ground.

(Jason May) #3

@sam, are you saying that the namespace hack might take 2 months - or that using namespaces would be quick but there’s some better solution that would require more substantial changes and that would take time.

I would consider myself a competent developer … but without any familiarity with the Discourse code. Perhaps time to get my hands dirty. Presumably this is more complicated than adding a forum-id column to the Topics table but that might be the place to start?

(Jason May) #4

Some thoughts on implementation of this feature:

  • any implementation of this must be entirely backward-compatible with the current (single forum) Discourse structure
  • Topics, Categories, Groups, Badges and Notifications must all be scoped to a specific forum. Anything else?
  • Site Settings need to be (all? most?) scoped to a forum
  • user must be able to get a list of forums they are participating in; this probably belongs on the Profile page
  • home page behavior must be different when there are multiple forums; what to show before user chooses a forum to enter?
  • searches must be (always? optionally?) scoped to a forum
  • admin/moderator status must be scoped by forum
  • Profile options probably need to be a combination of global and per-forum
  • email digests should consolidate activity across all forums in which the user participates

Plenty of work here. Are there any other particularly scary aspects that I’m missing?

(Jason May) #5

I am experimenting with this feature as I explore the Discourse codebase. Comments welcome!

multiforum feature branch on Github

(Sam Saffron) #6

You definitely would not do this, this way. Its too hard, risky and there is no chance we will add a “scope” column to every table, never going to happen.

Instead look at apartment gem and how it does schema switching, you would have to port that in conditionally to rails-multisite.

You would need to handle provisioning and comms and a bunch of other stuff.

(Jason May) #7

I’m thinking of an environment where there are thousands of forums, most with very light activity. Creating entire sets of tables for every one of those (as in the apartment gem) is terrifying. e.g. on software updates needing to run the migrations against N tenants. And I want an environment where provisioning a new forum is a fast & lightweight operation.

But I see your point on adding scope columns to the major tables. Probably not to everything, but it would be nice to avoid it even on the main ones. And then needing to add .where(forum: current_forum) everywhere just dirties up the codebase.

I’m going to keep hunting for alternatives. Would like to minimize overall impact.

(Jason May) #8

The Guardian class looks very interesting. Perhaps that could be extended to include the concept of restricting access to subsets of the database? Separate set of scoping tables, only used when the multi-forum feature is turned on.

(Sam Saffron) #9

You realize we have groups etc?

(Dave McClure) #10

I was gonna say… worth taking a look first at groups, categories and visibility of categories to groups, etc

(Jason May) #11

Yes, I know about groups. But there’s no nesting of groups. There’s some other conversation about subgroups on here, but I’m not sure that’s a good approach for this feature.

My mental model for this is for schools. One forum for each school, and groups for things like class membership, after-school activities, carpool groups, PTA, etc.

If it’s possible to stretch the existing framework to handle multi-forums, I’d rather do it that way than add a whole lot of new machinery.

(Tobias Eigen) #12

I am interested in this discussion as well and would be into exploring the use cases more. It seems to me that each discourse forum should be independent and separate. But maybe not right away. At first as it is being evaluated and new forum owners and members learn the ropes it would be hosted as a group on a forum run by someone else for them, like for new bloggers. Eventually as the group grows and takes wings, and needs grow, it could be moved to its own independent forum.

How this would work I am not sure, but the ability to export part of a forum, Eg a group and all related users, categories, and topics, would do it.

(Luke S) #13

Quick question about your model: Do your “schools” have anything in common? That is, do they belong to an overarching larger organization/grouping, (Like a network of districts), or are they simply unconnected groups that would come to you for quick, easy mini-forum setup?

I ask because understanding the relationship between the various “Schools” could help clarify how deeply these communities should be siloed off from each other.

(Jason May) #14

I’m assuming no relationship between schools. So by default the siloing of topics should be complete. Allowing for some degree of “leakage” between school forums (e.g. for schools within the same district) is an interesting idea, but not a strict requirement.

User might be members of multiple forums, and I want to be able to show users an aggregated view of activity across all their forums. In the tenancy model this would require merging results from queries across N databases. I’d like to avoid that.

(Jason May) #15

I’m warming up to the idea of trying to do this entirely with categories and groups. That will entail:

  • nesting categories more deeply (since I’m stealing the top level for the multiple forums)
  • probably nesting groups
  • moderator access tied to categories
  • possibly tethering trust level to categories as well

I found another open feature request that relates to this:

(Sam Saffron) #16

Category specific moderators is a feature we certainly want in core. It would be a great place to start, you just need to get that feature converted to a spec and @codinghorror can bless it once its good.

(Jason May) #17

Great. I’m in. By spec do you mean text or an actual rspec definition? Do I just post it in here or is there a procedure I should follow?

(Sam Saffron) #18

see the spec category:

You can not create spec category topics, only we can, so… create a detailed post on that feature topic with your proposed spec and we can review it and move it to the spec category once ready.

(Jason May) #19

posted on the other feature topic at