Installing Discourse for macOS Development Using asdf and docker-compose

I think a better direction forward here is

  1. As long as asdf does not do shims for ruby I am fine changing the official install script so it uses asdf which simplifies stuff for the end user. One less tool. @techAPJ we certainly need to make sure our setup stuff is under version control somewhere so people can contribute, is it?

  2. The script can do fishing and install stuff via port if you don’t have brew

  3. If macports can not install something and you have no brew, just offer a BYO option.

I just see very high amounts of risk in an install guide that will certainly not withstand the test of time.

We maintain our Mac setup guide, we maintain our Docker setup guide. I have a guarantee this will happen cause we have paid staff doing this.

A third party guide may sort of work today, but on a long time scale will break, there is almost a 100% guarantee.

My strong preference here is to move our official setup script to a state where you could use it and build on it.

Then the howto becomes … “how to bring your own redis and pg when using the official mac install guide”

5 Likes

I may be wrong, but I believe that your official MacOS install script uses rbenv which uses shims for Ruby. Am I misunderstanding something? There’s actually a log message that says “Enable shims and autocompletion …” Strangely, it then goes on to the modify the global Ruby version, which halfway defeats the purpose of using rbenv in the first place since you’ve now frobbed my global environment.

What’s wrong with shims?

I think a stronger option would be to have both the setup script and a docker-compose.yml file that you feel comfortable with. There’s already a d/boot step in the Discourse Docker guide before you can run the Rails server, so you can replace d/boot with docker-compose up.

PS: if you are willing to use docker-compose for redis and postgres and provide a .tool-versions file in the Discourse repository, then the setup script becomes (1) install asdf (2) run asdf install (3) install the image tools.

Yes, this is a motivator for me to move off rbenv, I personally use chruby now, some others use rvm.

shims are terrible cause you pay 50-200ms tax every time you fire up a ruby process, cause it does a bunch of pointless work.

So one guarantee here is that our official recommendation is moving off rbenv, the question is what we move to?

A problem with forcing docker on people is that many people have the same attitude towards docker that you have toward brew. I am not judging you here at all, I use pacman, brew is not pacman.

So just to summarize, some things that will happen:

  1. install-rails/mac at master · techAPJ/install-rails · GitHub will move to the discourse org, cause we want to take contributions here and it is official.

  2. install-rails can have switches and a contribution that makes it play nice with macports is welcome

  3. we will be moving off recommending rbenv installs either to chruby or asdf depending on shims

  4. I am open to having a switch of “don’t fiddle with installing pg/redis” and then I guess if people want to compose redis/pg they can. I want to sort out 1-3 first though.

5 Likes

Hmmm… One of the reasons that I found my way to asdf is that its performance was way, way better than some other version managers I was using. nvm was a particularly terrible offender, and causing noticeable delays when opening new terminal windows.

I’m not a heavy Ruby user so asdf's support has been fine for me. But I have found overall that the performance of asdf is quite good. I do not know how this is achieved, and also have no idea how it compares to chruby or rvm.

FWIW here is ruby --version performance on my system using asdf:

 /usr/bin/time ruby --version
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]
        0.21 real         0.08 user         0.14 sys

I don’t see a 200ms tax, but also don’t have anything to compare it with.

Again, the benefit of asdf is pinning all of the tool versions to your development directory, and making it very easy for new developers to get started: asdf install. Of course at this point it’s only maintaining versions for three things: ruby, Yarn, and the postgres libraries. Is that necessary? I don’t know. I’d be happy to use something else if it’s changes could be confined to my development directory.

Also just to be fair—not everyone uses Docker, sure. But AFAIK it doesn’t actively conflict with other alternatives the way that Brew, MacPorts, and (maybe) PacMan do with each other. Package managers are like that.

I’m really struggling to understand what the benefit is of a global, unversioned install of postgres and redis versus docker-compose? Not having to install Docker? I mean, how far are you going to get in Discourse development without Docker anyway?

2 Likes

What does which ruby return?

In the case of chruby I see:

% which ruby
/home/sam/.rubies/ruby-2.6.5/bin/ruby
% ls -al `which ruby`
-rwxr-xr-x 1 sam sam 23852912 Feb 14 12:24 /home/sam/.rubies/ruby-2.6.5/bin/ruby

That is clear proof I am not using a shim

% time ruby --version 
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
ruby --version  0.00s user 0.00s system 97% cpu 0.004 total
% time ruby --disable=gems -e puts 'hello world'

ruby --disable=gems -e puts 'hello world'  0.00s user 0.00s system 95% cpu 0.009 total

rubygems is a wildcard, ruby boot gets slower the more gems you have installed on the local machine, my times look that way cause I use zsh.

As to recommending compose over bare install, I don’t know, I guess my prefs are colored by the fact I am primarily a discourse developer, it is not just a project I hack on sometimes. I think it is fair that the install script can offer an option … what do you prefer bare install or docker isolated pg/redis?

asdf is clearly shimmed:

$ which ruby
/Users/challen/.asdf/shims/ruby

And the performance suffers:

time ruby --disable=gems -e puts 'hello world'
real	0m0.461s
user	0m0.097s
sys	0m0.169s

So there’s that. TBH, everything about Ruby has always seemed so slow to me that I have a hard time caring about this. But if you think it’s an issue then, sure, fine. Of course, when I was putting together this guide I was referencing the official macOS install script, which was updated recently and says nothing about moving away from shims.

I think you’re on the right track here. Moving away from the Homebrew dependency would be great for whiny babies like me. I think using docker-compose.yml to bring up postgres and redis is the right way to go, not only because it simplifies installation, but you also can pin the versions of these tools that you are using. If you don’t decide to do than then at least pin them in the install script.

The only hiccup here will be that the pg Gem requires pg_config to build for some reason, so I don’t know how to work around that except by using a global postgres installation. I worked around it by using asdf to install postgres with a version matching what was in my container and building based off of that. But if you aren’t using asdf I don’t know how to do this properly. I guess you could install postgres globally but not start it, but then still using pg_config to build the gem. Kind of gross but that should work.

I like using asdf since it’s a nice ecosystem and simplifies installing Yarn and postgres libraries if you are using the containerized approach, and the small performance penalty doesn’t concern me. That’s probably what I’ll keep doing.

Anyway—overall I’m quite disappointed that my contribution here wasn’t received in the same spirit in which it was offered. Someone should feel free to nuke this topic if they want to make sure that people find their way to the official installation guides. I added a disclaimer just in case.

6 Likes

I think much of the reticence here has come from a misunderstanding. I misunderstood what your motivator was. I understand far better now.

A lot of positive can come out of this topic, a more flexible and better install script is a good thing imo.

I just want us to have one “official” guide that does right by end user systems. I want to get there by improving our current install guide.

I am sorry you felt like the community was at war with you here, I realize that you spent a lot of your own free time here hacking and sharing, I appreciate this. I just want to make sure that his contribution is something we can learn and grow from.

12 Likes

I certainly meant no ill intent, I was just making the point that having two guides is really hard to support, and there are many people who are less sophisticated than you who won’t understand the nuances.

4 Likes

Overall a lot of the comments here seem to be from people that already have Discourse set up and working. If you’re in that boat, great! And maybe if you work primarily on Discourse or with Ruby you’re comfortable maintaining global installations of Ruby and postgres.

But a potential plugin developer (like me) is likely to want the following set of things from development environment instructions:

  1. Fast to set up, so that I can get started right away
  2. Fast to iterate, so that I can make progress quickly
  3. Unlikely to conflict with my current development environment, since my goal isn’t to turn my machine into a dedicated Discourse development box
  4. Easy to clean up when I’m done, since I’m going to stop (or at least pause) working on Discourse once my plugin is complete—at least until I have bugs to fix or features to add

I don’t care how you decide to get there, but the two supported alternatives don’t check all of those boxes. The native macOS guide achieves #1 and #2, while the Docker guide achieves everything but #2. My goal was to try and achieve all 4. But again, however you decide to do this, there are a lot of good new tools like asdf and docker-compose that could be part of the solution.

Part of what has been frustrating about this is a lot of the criticism of my approach has been inconsistent with your own native macOS guide. For example:

Criticisms related to maintainability are completely valid, although I’ve offered to help with that and it would not require much effort for Discourse staff to maintain this approach. But if you want to criticize my instructions, at least have some understanding of what your own supported install script is doing and some of the inherent tradeoffs. “It already works on my machine” is not a helpful way to receive a guide meant to help bootstrap new developers.

Honestly, I think that the best route to achieving the goals I outlined above would be to improve the performance of the Docker instructions—since it already achieves 3/4 of the objectives. First run time is painfully slow, and perhaps that has to do with a bunch of pre-run steps that are being done. So couldn’t these be done during container build? And are some of theme (e.g., asset minification) even required for development?

2 Likes

I think that’s the big problem with the above. You only see your contribution, whilst not considering the confusion, complexity and additional burden placed on those of us who have to pick up the pieces down the line.

We don’t need five different approaches to install on Mac, with and without docker are more than adequate.

You can make this about yourself if you want, but that’s not why guides here on meta exist, as @sam said:

Which is what the rest of us are driving for.

I think what @sam was previously saying is that we’re learning some things from your approach that we’d like to integrate into our install script because we’re realizing they’re not the ideal. That includes looking at using different ruby management tools, like chruby, that don’t use shims for a performance increase.

I don’t think anyone is trying to be critical of your approach without looking at our own solution critically.

3 Likes

I’ve offered multiple times above to maintain the small set of changes required to support my approach. These offerings don’t really seem to be being taken seriously, because you and others keep harping on this. I can certainly understand if Discourse wants paid staff to maintain these installation guides, rather than open source volunteers. But that wasn’t obvious to me and also seems to be something that people here are refusing to directly acknowledge.

So to reiterate—I am in fact volunteering to support this approach until either one of the two other approaches improves to the point that this isn’t necessary. I will try to answer questions on this topic to the best of my ability and am happy to check in and maintain the changeset required to support this approach. Scouts Honor!

I understand this better now, and I agree with the goal. However, when I started this there were already two macOS installation guides, one which was brand new another which was updated quite recently. Maybe you could more clearly mark these as Official so that its obvious that they are what you are guaranteeing support for?

I certainly don’t pretend to know nearly enough about Discourse to debug the performance problems running in a Docker container. But again, that would seem like the best way to go.

1 Like

There may be a light at the end of this tunnel given:

On Mac perhaps switching delegated to :cached here, may speed up boot by a lot.

@david maybe you can test that.

We will get the guides for Mac up to scratch in the next couple of weeks, just need a bit of time to digest / decide and analyze.

4 Likes

I was thinking about this more today as well. I found the same page that you posted and examined the same options. However, based on my reading it seemed that delegated—which you are already using—was the right option. Cached v. delegated just seems to affect the direction in which updates are expected to propagate. But it’s probably worth at least benchmarking the other non-consistent option.

That said, I think I can explain the performance difference between my setup and the standard Docker install. I hadn’t remembered that you are actually bind mounting a directory containing the postgres database files in order to persist the database across runs. That directory probably gets heavy IO traffic, hence the slowdown.

A better approach is probably to use a Docker volume, which would enable persistence without needing to be bind mounted. You can configure this easily using docker-compose, and probably directly using the Docker command line tools as well. So you could either (1) set up docker-compose.yml to start one container with postgres, a second with redis, and a third with Ruby running your rails app. Or (2) put everything in one container if that’s your thing. The ruby container would just need to have the appropriate Ruby and bundle and bind mount the source directory as is done currently. docker-compose actually allows you to start and stop services separately, so you could bring the rails container up and down during development without restarting your postgres and redis containers. This might be a good setup.

1 Like

Sigh. That doesn’t seem to help much.

@sam: are you 100% certain that slow IO is your problem? Here’s a waterfall diagram showing that a hard refresh of Discourse running in your Docker development environment takes ~20 seconds to first paint on my machine, which is not that slow (my machine, not Discourse in this setup):

Note that this is not the first load of the page—that took almost a minute :expressionless:.

I’m also seeing worker timeouts:

I, [2020-03-09T23:08:06.198592 #439]  INFO -- : Refreshing Gem list
Starting CSS change watcher
I, [2020-03-09T23:08:19.250249 #439]  INFO -- : listening on addr=0.0.0.0:9292 fd=15
Starting up 1 supervised sidekiqs
Loading Sidekiq in process id 512
I, [2020-03-09T23:09:02.266046 #439]  INFO -- : master process ready
I, [2020-03-09T23:09:07.803551 #529]  INFO -- : worker=0 ready
I, [2020-03-09T23:09:08.577435 #545]  INFO -- : worker=1 ready
I, [2020-03-09T23:09:11.200000 #552]  INFO -- : worker=2 ready
E, [2020-03-09T23:10:14.271174 #439] ERROR -- : worker=0 PID:529 timeout (61s > 60s), killing
E, [2020-03-09T23:10:14.570233 #439] ERROR -- : reaped #<Process::Status: pid 529 SIGKILL (signal 9)> worker=0
I, [2020-03-09T23:10:29.307995 #597]  INFO -- : worker=0 ready
E, [2020-03-09T23:11:20.558583 #439] ERROR -- : worker=1 PID:545 timeout (61s > 60s), killing
E, [2020-03-09T23:11:20.729284 #439] ERROR -- : reaped #<Process::Status: pid 545 SIGKILL (signal 9)> worker=1
I, [2020-03-09T23:11:29.265051 #623]  INFO -- : worker=1 ready

That doesn’t seem good…

As Sam said

And that part is fine…

2 Likes

OK. I kept hacking away at this.

I was able to solve the performance problems using the Docker approach by using docker-sync. AFAICT this creates a Docker volume and a (containerized) process to sync files back and forth to it, avoiding the MacOS volume performance bottleneck. On my system the Docker-based approach now approaches native performance.

Yes, it’s another tool. Happily it’s installable as a Gem.

It requires this configuration file in the Discourse development directory:

version: "2"
options:
  verbose: true
syncs:
  discourse_dev_src:
    src: '.'
    sync_userid: '1000'
    sync_groupid: '1000'
    sync_excludes:
      - .git
      - data
      - log
      - node_modules
      - public/uploads
      - public/backups
      - public/stylesheet-cache
      - tmp

(The ignore list could probably be improved. This was just a first pass.)

Then boot_dev needs to be patched:

-    -v "$SOURCE_DIR:/src:delegated" \
+    --mount source=discourse_dev_src,target=/src \

Then you need to start docker-sync (docker-sync start), wait for it to finish an initial sync, and can then run boot-dev --init as usual.

(Note that it probably still makes sense to move the postgres data files to a separate Docker volume as well, as I suggested above. That’s orthogonal to this part, and probably can be handled differently unless you plan to manually edit the postgres files during development.)

Honestly, this has gotten pretty gross. At this point I feel like it might be inevitable to support two different guides: one using Docker for non-Mac, and a second using some other approach for macOS. It could be docker-sync, or just a more flexible install of Ruby and the rest of the toolchain that doesn’t start by installing brew.

1 Like

Unlikely; Docker needs to fix their Mac performance issues, whatever they are. That’s the best long term fix for everyone involved rather than hard forking and doubling the amount of work.

1 Like

You already have separate Docker and macOS installation guides. My point is that it may not be wise to try and merge those, which was suggested earlier. The performance penalty using your Docker instructions on MacOS is severe enough to deter anyone who might want to do some casual Discourse development—like write a plugin, for example.

Or I guess you could just add the docker-sync.yml file and an addendum about how to set this up to the Docker instructions and abandon the native macOS development guide.

Anyway—I actually finished the plugin development I was trying to do a while ago, and have probably spent several multiples experimenting with different development environments. I will now happily retreat to a friendlier version of the “it works on my machine” attitude I encountered above. But I look forward to seeing any positive changes that would help more people get started hacking on Discourse! It really is a great piece of software.

10 Likes

Agree completely! That’s the goal, and thanks for your help in getting us closer!

The reason you’re seeing us exercise caution here is because we’ve seen how this sort of split effort off-the-beaten-path thing generally goes in the past, several times, and the TL;DR spoiler is not well :wink:

7 Likes