Hello from Gitpod! (installing on google cloud + automated dev setup)

The developer install is much slower and designed for…development. So you’ll want to follow the standard install instructions. It’s easy enough to automate. My install service is now fully automated (except for the dns settings required by mailgun because I don’t have control over client dns).

The hard part of installing is mail configuration.

Also, installation and maintenance are different.

4 Likes

Thanks @pfaffman! Sorry about the confusion, I mixed up two things in my original message: 1) installing Discourse for the Gitpod community, and 2) automating the dev setup for Discourse developers and contributors. I’m thankfully following the developer guide for 2), not for 1). :sweat_smile:

@david Hmm, unfortunately, with:

bundle exec rake db:create db:migrate &&
RAILS_ENV=test LOAD_PLUGINS=1 bundle exec rake db:create db:migrate

It still fails with:

== Seed from /workspace/discourse/db/fixtures/990_settings.rb
Discourse hostname: localhost is not a valid domain for emails!

== Seed from /workspace/discourse/db/fixtures/990_topics.rb
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "polls" does not exist
LINE 8:  WHERE a.attrelid = '"polls"'::regclass
                            ^

Did I understand your suggestion correctly? Would you have other ideas on how to fix it? (I tried to drop the tables beforehand, but that failed with a different error which I don’t remember. Happy to try again though.)

Also, I forgot to mention that my setup instructions worked until last week or so, so the error appeared recently after a Discourse rebase. I think after the migration to Rails 6 maybe? DEV: Upgrade Discourse to Rails 6 (#8083) · discourse/discourse@32b8a2c · GitHub

FYI, you can easily reproduce the error by trying to open my Discourse fork in Gitpod: Gitpod - Online IDE for GitHub . You’ll see my automated setup fail, and get an interactive environment where one can try various Discourse commands in the Terminal.

@sam It looks like FIX: Rails 6 multisite migrations and plugin migrations · discourse/discourse@025d4ee · GitHub broke something.

Contrary to the commit message, I think plugin migrations were working before this commit. They aren’t anymore. It doesn’t matter if the LOAD_PLUGINS=1 is provided or not – plugin migrations aren’t running in my dev environment.

I believe it’s because of this:

--- a/lib/plugin/instance.rb
+++ b/lib/plugin/instance.rb
@@ -516,7 +516,7 @@ class Plugin::Instance
     Rake.add_rakelib(File.dirname(path) + "/lib/tasks")
 
     # Automatically include migrations
-    migration_paths = Rails.configuration.paths["db/migrate"]
+    migration_paths = ActiveRecord::Migrator.migrations_paths
     migration_paths << File.dirname(path) + "/db/migrate"
 
     unless Discourse.skip_post_deployment_migrations?
5 Likes

So that’s INSTALL-cloud.

Imagine that there was a docker container that was included with Discourse and you could just use it without installing anything.

2 Likes

I have seen this error on the multisite migrate, on dev/test db it is fine, going to debug it carefully today

4 Likes

Per:

This should make the dev envs behave right when following guides.

@kris.kotlarek can you have a look at that commit … why did schema dump -> load stop working in Rails 6 with our plugins?

5 Likes

Sure, I will check it tonight

1 Like

I think I found problem for ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "polls" does not exist
When load_config is evaluated on database creation, it overrides migrations_paths

So this is what is happening when we try to evaluate db:create and db:migrate at the same time:

However, if we run them separately, paths are fine:

I think we got two options here:

  1. Change guides and recommend to use two commands separately
  2. Monkey patch and use ||=

What do you think?

4 Likes

Will that sort out?

RAILS_ENV=test bin/rake db:migrate
RAILS_ENV=test bin/rake db:schema:dump
dropdb discourse_test
createdb discourse_test
RAILS_ENV=test bin/rake db:schema:load
RAILS_ENV=test bin/rake db:migrate
3 Likes

This would solve the original problem about missing polls table mentioned here: Hello from Gitpod! (installing on google cloud + automated dev setup)
The same problem was mentioned in that topic: Beginners Guide to Install Discourse on Ubuntu for Development

I started debugging that code about populating test DB but I got lost. For some reasons, after the schema is loaded, when calling db:migrate, Rails still wants to evaluate migrations and we got an error that table already exists. So far, I couldn’t find a reason

3 Likes

I see let’s adjust the guides then for now for the op

5 Likes

Thanks a lot for looking into this bug! :+1: I will use the updated instructions to work around it for now.

Aha, interesting, thanks! I’ll have a look at the existing development Dockerfiles to see if they can simply be spawned in one click by Gitpod. :smile: That would be cool.

4 Likes

Interestingly, splitting db:create and db:migrate into two separate commands, as suggested in Beginners Guide to Install Discourse on Ubuntu for Development, also “worked”.

(But now, DISCOURSE_DEV_HOST=.gitpod.io bundle exec rails s -b 0.0.0.0 seems to crash-loop with:

/workspace/.rvm/gems/activesupport-6.0.0/lib/active_support/dependencies.rb:551:in `load_missing_constant': Unable to autoload constant Version, expected /workspace/discourse/lib/version.rb to define it (LoadError)

and the web server started blocking my Gitpod preview URL again:

Blocked host: 3000-a8a71720-4c30-466b-aea5-5344c97c4e94.ws-eu0.gitpod.io

)

4 Likes

I created a pull request with a fix for that problem: FIX: Remove Versions from Active Record warm up by lis2 · Pull Request #8105 · discourse/discourse · GitHub

Could you check if that solves the problem on your machine? (It is working now on my local)

5 Likes

I just changed that env var to have a s at the end

6 Likes

Hi Jan, if you have time, I’m curious about what were the problems / didn’t work out the way you had hoped, with Spectrum?

I’ve been following Spectrum a little bit, reading about them from time to time.

(I tried to send you a PM about this, but for some reason when I click your profile, there’re no send-private-message button.)

Quick update, in case that’s helpful to anyone – we’ve successfully deployed Discourse on Google Cloud! :tada: It’s running on https://community.gitpod.io and we’re loving it so far.

Details

I mostly followed INSTALL-cloud, and created a g1-small GCE VM (1 vCPU, 1.7 GB memory) with an additional 20GB SSD. (Note: I originally considered a n1-standard-1 VM, but it seemed a bit overkill for Discourse).

For the VM’s location, based on this blog post, we’ve determined gce-us-east4 to be the best location, since most Gitpod users are in North America and Europe, but also a lot in Asia, so latency shouldn’t get too bad there either.

For the email setup, we really wanted to use our Google Apps account. We’ve tried to configure a Gmail SMTP relay, but even after triple-checking that we were using all the correct protocols and allowed them through GCP firewall and got the correct IPs whitelisted, no email ever went through. Discourse Doctor was a great help, but didn’t succeed either. So eventually we gave up and used SendGrid instead, because we wanted to evaluate it anyway for other things. It was super easy to set up and worked on first try. FYI, our Discourse traffic is growing, but still easily fits within SendGrid’s free tier (100 emails / day).

We then set up GitHub OAuth login (the same as for gitpod.io, for convenience) and installed a few useful plugins:

Also, for the anecdote, I wrote a quick and dirty spectrum-to-discourse.js nodejs script to transfer our old Spectrum threads over to Discourse. The quality isn’t 100% optimal, and there may be a few bugs left in the script, but this was enough to seed our new Discouse. We now manually review/fix/improve old topics when we see them becoming popular.

Hope all this info may help someone in the future! :crossed_fingers:

8 Likes

Sure! I’ve shared more details with you via PM. :slight_smile:

5 Likes

FYI, I’ve now opened a Pull Request for this: DEV: Add fully-automated dev setup with Gitpod by jankeromnes · Pull Request #9026 · discourse/discourse · GitHub

I hope you’ll like it. :slightly_smiling_face:

1 Like

As a quick update, Google Cloud is now recommending to increase the performance of our Discourse instance due to “high memory utilization”:

This instance has had high memory utilization recently. Consider switching to the machine type: custom (1 vCPU, 2.75 GB memory). Learn more

Current machine type
g1-small (1 vCPU, 1.7 GB memory)

New machine type
custom (1 vCPU, 2.75 GB memory) Recommended

But as we haven’t noticed any performance problems ourselves, we haven’t upgraded yet. (Just thought it would be useful to mention the hint here.) So we’ll be paying more attention to memory going forward, and I’ll update this post if we end up doing the suggested upgrade.

2 Likes