What exactly is holding Discourse from being OK with 512mb RAM?

Fact 1. Ukraine is a very ITsh country. One of the leading outsource-force in the world. Yet the average salary in Ukraine for a regular person is $100/mo. Paying $120/year to host Discourse ($10/mo with DigitalOcean) is a lot.

Fact 2. Software developers in Ukraine earn from $800 to $2000 / mo. Paying $120/year can still be considered quite expensive by some.

Fact 3. As we used in PHP-world, most of software can easily run on 256MB RAM, making it very cheap to host a wordpress, a forum, a wiki and something else, all on a single cheapest host possible whatever hosting provider you choose.

I’m not familiar with RoR world, but just very curious - WHY and WHAT is exactly holding Discourse from being 512MB or even 256MB compatible? Is it mostly ineffectiveness of Ruby? Or something else?

Even though we all know that hardware is cheap, it is still not cheap in the eyes of some regions’ representatives around the world.

Even though there is more and more RAM provided by the minimal hosting options among providers, PHP is still staying at a very basic minimum. Making it HHVM and using nginx really beats almost everyone in the script-world.

So, can you explain, please, why and what. And whether there are any chances that Discourse is going to become OK with 512MB for small to medium forums in the nearest feature?

  • Postgres database
  • Redis in memory key value store
  • Ruby
  • Sidekiq for background tasks
  • Nginx webserver
  • Unicorn(s) to serve web

1GB is the minimum supported config. 512MB is only supported for single user testing.

The smartphone you are almost certainly carrying in your pocket has 1 GB RAM. If it is an android device it may have up to 3 GB RAM.

4 Likes

Discourse team member, Sam Saffron, talks a lot about Discourse and overall, performance issues encountered with Ruby on Rails in the video below from last months Red Dot Ruby Conference. I think it’s a good place to start answering your main question as to what’s holding it back.

Also keep in mind that Discourse is probably better described as an EmberJS App first and a Rails app second. As far as the PHP vs Rails issue is concerned, while there are many things that come to mind like maintainability, philosophy and community think about the overall future. What is in PHP’s future? Where is it going? I think one concern about PHP is how Facebook’s Hack for HHVM comes into play. In case you didn’t know, Hack is like a less popular PHP tribute band that plays PHPs songs a little bit better than PHP does.

My two cents anyway.

5 Likes

Thanks for linking to the video.

Nowhere going to bring that PHP vs Ruby discussion again - there is a lot in this forum already. Just wanted to better understand the obstacles from the technical point of view.

The VPS most of people from poor countries buy is nowhere near 1GB RAM, but anyway smartphones are not related to hosting, are they?

In particular we would have to move to a threaded web server like puma and re-engineer the job queue to operate in a thread. Due to the “global interpreter lock” you would only get partial parallelization, still, it would probably be good enough for “el cheapo” instance. You would however lose out on a ton of features that make Discourse so reliable.

  • There would no longer be a proper supervisor process that can tear down misbehaving processes (like we have today)
  • There would no longer be clean isolation between the job queue and web server
  • We would need to spend significant engineering effort redesigning the job queue

So yeah, if we had a huge drive to make this happen, we could probably get 512MB … sort of working … for small forums. However, supporting two architectures for deploying the app does worry me, and honestly we don’t have time to work on a project like this.

4 Likes

And it’s not like any paying customers are asking for this, or that we could have any confidence it would work under high load. Or that anyone would actually use it under high load. For that matter, what would be the procedure for switching from one to the other? Would it even be safe? Plugins developed in one model might not work so well in the other. Lots of potential problems, low benefit for the core team.

4 Likes

But … I never implied supporting two architectures.
Just curious if it by chance is going to decrease the hardware requirements with time.

Current architecture requires a minimum of 3 Ruby processes, with PSS of about 150MB each, so there you have 450MB RSS and that is best case before Postgres,Redis and NGINX.

Once you get load PSS will go up to 200/250MB per process … so that is 750MB, which would have to be swapped out.

So you would have a server swapping like crazy.

4 Likes

So, at the end, does it mean that the “heavy guy” is a ruby process? If it was e.g. 50MB, it would be okay to go with 500MB RAM?

You also have the requirements imposed by Docker. Not that Docker has a noticeable overhead, but it limits the hosting and configuration options. You can install without docker, under penalty of becoming an outcast.

2 Likes

I wouldn’t bet on it. Instead, I would bet on the cost of hardware going down.

3 Likes