Hello from Gitpod! (installing on google cloud + automated dev setup)

[jankeromnes]

Hello Discourse community!

I work on gitpod.io, a free one-click IDE for GitHub. We currently use Spectrum for our community, but we have some troubles with it and we’d love to stand up a Discourse instance to play with it and potentially migrate.

We’d like to use our Google Cloud account for that, if possible. Could you please point me in the right direction? E.g. is the new Google Cloud Run a good fit for running a Discourse instance?

Also, since our mission is to fully automate all dev setups (at least for open source projects), I’d like to contribute a fully-automated setup for Discourse, allowing contributors to start a ready-to-code Discourse environment online in one click (as opposed to reading through long lists of setup instructions and manually installing/configuring a bunch of dependencies on your current device). I’ve already started to work on this based on @notriddle’s (hi!) excellent automated Discourse setup for Janitor.

I’m currently facing a problem though, since recently the setup instructions fail on a missing “polls” table. Not sure how to fix that yet, but I’ll continue investigating. Just thought I’d mention it here.

[marianord (Mariano Rodriguez)]

@jankeromnes

is the new Google Cloud Run a good fit for running a Discourse instance?

I could be, but it’s not that easy. Discourse is not a stateless container which is what Google Cloud Run is made for, therefore you’ll need a way to add some kind of Persistent Volume for the data that’s kept in Discourse (uploads, avatars, DB).

I’d use a normal Google Compute instance and that’s it. With some kind of script, or infrastructure management tools you should be able to automate the deployment of a new Discourse instances, this guide Install Discourse on Ubuntu or Debian for Development might also help you.

[david (David Taylor)]

@jankeromnes

I’m currently facing a problem though, since recently the setup instructions fail on a missing “polls” table. Not sure how to fix that yet, but I’ll continue investigating. Just thought I’d mention it here.

Which setup instructions are you using? If the polls table is missing, it suggests you have not run the plugin migrations. They should be run automatically in development mode. In test mode, you need to add LOAD_PLUGINS=1 before the rake db:migrate command

[jankeromnes]

Thanks for your replies @marianord and @david!

@marianord

I’d use a normal Google Compute instance and that’s it.

That does sound more reasonable than using Cloud Run, since extracting all the state from Discourse containers could take some time. Thanks for pointing me to the guide! I’ll try to follow it to set up a Compute instance, and report back here with my findings.

@david

Which setup instructions are you using?

I’m following discourse/docs/DEVELOPER-ADVANCED.md at main · discourse/discourse · GitHub.

The exact error I saw was:

== Seed from /workspace/discourse/db/fixtures/990_settings.rb
Discourse hostname: localhost is not a valid domain for emails!

== Seed from /workspace/discourse/db/fixtures/990_topics.rb
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "polls" does not exist
LINE 8:  WHERE a.attrelid = '"polls"'::regclass
                            ^

While trying to do:

bundle exec rake db:create db:migrate &&
RAILS_ENV=test bundle exec rake db:create db:migrate

I’ve also experimented with adding this line:

RAILS_ENV=development bundle exec rake db:create db:migrate

but to no avail.

@david

If the polls table is missing, it suggests you have not run the plugin migrations. They should be run automatically in development mode. In test mode, you need to add LOAD_PLUGINS=1 before the rake db:migrate command

Thanks a lot for the tip! I’ll try that, and report back here as well. (Sorry for asking two questions in one! Hopefully it won’t make the discussion too confusing.)

[pfaffman (Jay Pfaffman)]

@jankeromnes

I’m following discourse/DEVELOPER-ADVANCED.md at master · discourse/discourse · GitHub

The developer install is much slower and designed for…development. So you’ll want to follow the standard install instructions. It’s easy enough to automate. My install service is now fully automated (except for the dns settings required by mailgun because I don’t have control over client dns).

The hard part of installing is mail configuration.

Also, installation and maintenance are different.

[jankeromnes]

@pfaffman

The developer install is much slower and designed for…development. So you’ll want to follow the standard install instructions.

Thanks @pfaffman! Sorry about the confusion, I mixed up two things in my original message: 1) installing Discourse for the Gitpod community, and 2) automating the dev setup for Discourse developers and contributors. I’m thankfully following the developer guide for 2), not for 1).

@david

If the polls table is missing, it suggests you have not run the plugin migrations. They should be run automatically in development mode. In test mode, you need to add LOAD_PLUGINS=1 before the rake db:migrate command

@david Hmm, unfortunately, with:

bundle exec rake db:create db:migrate &&
RAILS_ENV=test LOAD_PLUGINS=1 bundle exec rake db:create db:migrate

It still fails with:

== Seed from /workspace/discourse/db/fixtures/990_settings.rb
Discourse hostname: localhost is not a valid domain for emails!

== Seed from /workspace/discourse/db/fixtures/990_topics.rb
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "polls" does not exist
LINE 8:  WHERE a.attrelid = '"polls"'::regclass
                            ^

Did I understand your suggestion correctly? Would you have other ideas on how to fix it? (I tried to drop the tables beforehand, but that failed with a different error which I don’t remember. Happy to try again though.)

Also, I forgot to mention that my setup instructions worked until last week or so, so the error appeared recently after a Discourse rebase. I think after the migration to Rails 6 maybe? DEV: Upgrade Discourse to Rails 6 (#8083) · discourse/discourse@32b8a2c · GitHub

FYI, you can easily reproduce the error by trying to open my Discourse fork in Gitpod: Dashboard . You’ll see my automated setup fail, and get an interactive environment where one can try various Discourse commands in the Terminal.

[gerhard (Gerhard Schlager)]

@sam It looks like https://github.com/discourse/discourse/commit/025d4ee91f4727540c749e2162680e1042c34376 broke something.

Contrary to the commit message, I think plugin migrations were working before this commit. They aren’t anymore. It doesn’t matter if the LOAD_PLUGINS=1 is provided or not – plugin migrations aren’t running in my dev environment.

I believe it’s because of this:

--- a/lib/plugin/instance.rb
+++ b/lib/plugin/instance.rb
@@ -516,7 +516,7 @@ class Plugin::Instance
     Rake.add_rakelib(File.dirname(path) + "/lib/tasks")
 
     # Automatically include migrations
-    migration_paths = Rails.configuration.paths["db/migrate"]
+    migration_paths = ActiveRecord::Migrator.migrations_paths
     migration_paths << File.dirname(path) + "/db/migrate"
 
     unless Discourse.skip_post_deployment_migrations?

[pfaffman (Jay Pfaffman)]

@jankeromnes

installing Discourse for the Gitpod community,

So that’s INSTALL-cloud.

@jankeromnes

automating the dev setup for Discourse developers and contributors.

Imagine that there was a docker container that was included with Discourse and you could just use it without installing anything.

Install Discourse for development using Docker

Developing using Docker Since Discourse runs in Docker, you should be able to run Discourse directly from your source directory using a Discourse development container. Pros: No need to install any system dependencies, no configuration needed at all for setting up a development environment quickly. Cons: Will be slightly slower than the native dev environment on Ubuntu, and much slower than a native install on MacOS. Step 1: Install Docker Ubuntu curl -fsSL https:/…

[sam (Sam Saffron)]

I have seen this error on the multisite migrate, on dev/test db it is fine, going to debug it carefully today

[sam (Sam Saffron)]

Per:

https://github.com/discourse/discourse/commit/1ca257be791de2b7806d06354f392585d83108a3

This should make the dev envs behave right when following guides.

@kris.kotlarek can you have a look at that commit … why did schema dump → load stop working in Rails 6 with our plugins?

[kris.kotlarek (Kris)]

Sure, I will check it tonight

[kris.kotlarek (Kris)]

I think I found problem for ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "polls" does not exist
When load_config is evaluated on database creation, it overrides migrations_paths

So this is what is happening when we try to evaluate db:create and db:migrate at the same time:

However, if we run them separately, paths are fine:

I think we got two options here:

1. Change guides and recommend to use two commands separately
2. Monkey patch and use ||=

What do you think?

[sam (Sam Saffron)]

Will that sort out?

RAILS_ENV=test bin/rake db:migrate
RAILS_ENV=test bin/rake db:schema:dump
dropdb discourse_test
createdb discourse_test
RAILS_ENV=test bin/rake db:schema:load
RAILS_ENV=test bin/rake db:migrate

[kris.kotlarek (Kris)]

This would solve the original problem about missing polls table mentioned here: Hello from Gitpod! (installing on google cloud + automated dev setup)
The same problem was mentioned in that topic: Beginners Guide to Install Discourse on Ubuntu for Development

I started debugging that code about populating test DB but I got lost. For some reasons, after the schema is loaded, when calling db:migrate, Rails still wants to evaluate migrations and we got an error that table already exists. So far, I couldn’t find a reason

[sam (Sam Saffron)]

I see let’s adjust the guides then for now for the op

[jankeromnes]

Thanks a lot for looking into this bug! I will use the updated instructions to work around it for now.

@pfaffman

Imagine that there was a docker container that was included with Discourse and you could just use it without installing anything.

Aha, interesting, thanks! I’ll have a look at the existing development Dockerfiles to see if they can simply be spawned in one click by Gitpod. That would be cool.

[jankeromnes]

Interestingly, splitting db:create and db:migrate into two separate commands, as suggested in Beginners Guide to Install Discourse on Ubuntu for Development, also “worked”.

(But now, DISCOURSE_DEV_HOST=.gitpod.io bundle exec rails s -b 0.0.0.0 seems to crash-loop with:

/workspace/.rvm/gems/activesupport-6.0.0/lib/active_support/dependencies.rb:551:in `load_missing_constant': Unable to autoload constant Version, expected /workspace/discourse/lib/version.rb to define it (LoadError)

and the web server started blocking my Gitpod preview URL again:

Blocked host: 3000-a8a71720-4c30-466b-aea5-5344c97c4e94.ws-eu0.gitpod.io

)

[kris.kotlarek (Kris)]

I created a pull request with a fix for that problem: https://github.com/discourse/discourse/pull/8105

Could you check if that solves the problem on your machine? (It is working now on my local)

[sam (Sam Saffron)]

I just changed that env var to have a s at the end

DEV: support multiple hosts in dev

[KajMagnus]

@jankeromnes

We currently use Spectrum for our community, but we have some troubles with it

Hi Jan, if you have time, I’m curious about what were the problems / didn’t work out the way you had hoped, with Spectrum?

I’ve been following Spectrum a little bit, reading about them from time to time.

(I tried to send you a PM about this, but for some reason when I click your profile, there’re no send-private-message button.)