How to replace the rails_multisite gem with my own gem

(Michael Smyers) #1

I would like to replace the rails_multisite gem that comes with Discourse with my own rails_multisite_federation. The existing one appears to be in the source folder under vendor/gems/rails_multisite

Strategy 1:
Make my own Gem and add a line to Gemfile that reads gem 'rails_multisite_bestest'

  • Bundles well. Can add to the existing bundle manifest pretty easily.
  • Would seem to conflict with the existing built-in multisite gem.
  • Seems like I need to have my Gem be aware of removing the other Gem via railtie.rb and ‘remove’ it.

Strategy 2:

  • Remove the rails_multisite folder and replace it with the rails_multisite_federation folder.
    Pros: Easy to work on while I have vagrant up in my business.
    Cons: I would have to modify the Discourse source and keep it updated on every pull.

Strategy 3

  • Edit the rails_multisite code directly. Same consequences of above.

What do you recommend? Is there a ‘best practices’ path?

(Sam Saffron) #2

Why are you trying to do this ?

(Michael Smyers) #3

First of all, thanks for responding so quickly. Much appreciated.

I need to add/remove/manage hosts without restarting the app. I’m going to remove the data typically found in multisite.yml and replace the lookup to take place in a database. I will resolve the actual database info from a lookup table in redis/sql rather than finding it in a local config file.

Technically since the data is stored in a ruby hash, I could have my Gem tweak the RailsMultisite class cache variables, but that seems really hacky and tied to your code. (ie: difficult to maintain)

I need to host about 100k sites under 1 runtime (though I’ll have a few hundred servers running them in parallel). Let’s assume that they all get very low traffic and are mostly idle. (I’ll bring up a separate discourse topic about connection efficiency, because my run through the code makes me worried that it shuts down connections when it switches sites. That seems a bit inefficient.)

I need to bring up about 1 or 2 new forum sites a second, so restarting the app is not really an option.

(Sam Saffron) #5

I am open to extracting the gem to its own repo, cleaning up the code, adding a proper readme, adding tests and additional interfaces to extend stuff. You, however, are going to have to do all this work.

That is the only really supportable way of moving forward here. We can not afford any regressions here.

(Michael Smyers) #6

I will gladly do this work.

I will create a GitHub project for this. It will be a good place to start getting into development, as I foresee my project requiring intimate knowledge of this great codebase.

Thanks for the advice!

(Michael Smyers) #7

@sam I have completed version 0.1.0 of the documentation of the intended Gem. If you like the way this config lays out, please let me know and I will begin working on it immediately.

(Sam Saffron) #8

This is way too big. Start small.

What is the absolute smallest gem you can extract that provides what we have today and extensibility points needed.

(Michael Smyers) #9

I must have the 3 modes of operation (inactive, YAML (backwards compat), and SQL (what I need)). I’ve modified the document to trim out anything that I don’t specifically need.

Here’s the outline:

##Milestone 1

Clone what we have already in Discourse into a separate Gem. Commit this code as-is.

##Milestone 2

Add ability to dip SQL if not found in YAML

##Milestone 3

Wrap that function into a strategy and then allow for strategy fallback.

Here’s the updated docs for milestone 1 and 2: rails_multisite/ at master · TopSpectrum/rails_multisite · GitHub

(Sam Saffron) #10

architecturally I would prefer multiple gems:

  • rails_multisite (what we have now + extensibility interfaces)
  • rails_multisite_sql (milestone 2 / 3) depends on rails_multisite

(Michael Smyers) #11

I am open to this, though I would need some help getting started. I’m not familiar enough with ruby/rails enough to do this in a thoughtful way. I’m willing to work full time on this project; can you connect with me via chat to kick it off?

I’m a solid developer with a history of success (, though the language/pattern of rails is very unfamiliar to me. If you invest a few minutes into the start, I can do a great job and finish it up. I have my environment setup and ready to go.

Here’s my plan:

  1. Remove the folder vendor/rails_multisite
  2. Add my Gem to my local gem cache.
  3. Reference my Gem in the Gemfile file of Discourse.
  4. Get it to work -as-is- and get your sign off.
  5. Tag the release at GitHub - TopSpectrum/rails_multisite: A Gem that allows you to host multiple sites with a single Rails runtime as production 1.0.0
  6. Publish to from this GitHub location
  7. Work with you to identify ways to extend this Gem
  8. Make a new GitHub project for sql lookup strategies
  9. Tag/Publish this Gem to RubyGems
  10. Add to Discourse in Gemfile
  11. ???
  12. Profit?

(Mark Ellul) #12

Did you end up continuing to build out the gem? or did you go with another approach or drop discourse totally?

Looking forward to your response