Requiring external gems

Development

AI-generated summary

In a discussion about requiring external gems in Discourse plugins, users explore how to include gems not in Discourse’s default Gemfile. kgish initiates the thread seeking guidance, and Falco points to an example plugin using plugin.rb to declare gems and their dependencies. kgish follows up asking about rake tasks, and zogstrip suggests using LOAD_PLUGINS=1 rake your:rake:task. Later, fzngagan shares a crash log while trying to use the oxford_dictionary gem, revealing dependency issues. merefield explains that all transitive dependencies must be manually declared in plugin.rb, calling it a “lengthy iterative process,” which Falco humorously renames a “lengthy recursive process.” fzngagan adds that require: false doesn’t work — the gem must be explicitly required. Luke_Clancy shares a practical workflow: use Ruby 2.6.5 locally, create a Gemfile, run bundle install, and copy dependencies into plugin.rb. Finally, JDoy99 reports a similar issue with the eth gem failing due to a missing forwardable dependency, despite manually installing the gem — highlighting that Discourse’s plugin gem loader doesn’t auto-resolve dependencies, requiring explicit declaration of all nested dependencies in plugin.rb.

1

For a plugin that I am writing, I would like to use some external gem files that are not present in the default discourse gemfile.

What is the standard way of doing this without being able to modify the discourse gemfile?

Also a similar question reqarding user-defined rake tasks.

2

Keep in mind you have to declare the gems AND dependencies.

3

Thanks, didn’t realize that all dependencies needed to be included as well.

How can I also do this for rake tasks?

4

Have you tried “LOAD_PLUGINS=1 rake your:rake:task”?

5

I am trying to use this ruby gem called oxford_dictionary

gem 'oxford_dictionary', '2.0.1', { require: false }

This is what I get in the console

/bin/bash -c "env RBENV_VERSION=2.6.2 /usr/local/Cellar/rbenv/1.1.2/libexec/rbenv exec ruby /Users/faiz/discenv/discourse/bin/rails server -b 0.0.0.0 -p 3000 -e development"
I, [2019-08-08T14:01:24.177008 #5867]  INFO -- : Refreshing Gem list
Traceback (most recent call last):
	30: from /Users/faiz/discenv/discourse/bin/unicorn:49:in `<main>'
	29: from /Users/faiz/discenv/discourse/bin/unicorn:49:in `load'
	28: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/unicorn-5.5.1/bin/unicorn:128:in `<top (required)>'
	27: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:141:in `start'
	26: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/unicorn-5.5.1/lib/unicorn/http_server.rb:794:in `build_app!'
	25: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/unicorn-5.5.1/lib/unicorn.rb:54:in `block in builder'
	24: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/unicorn-5.5.1/lib/unicorn.rb:54:in `eval'
	23: from config.ru:1:in `<main>'
	22: from config.ru:1:in `new'
	21: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/rack-2.0.7/lib/rack/builder.rb:55:in `initialize'
	20: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/rack-2.0.7/lib/rack/builder.rb:55:in `instance_eval'
	19: from config.ru:7:in `block in <main>'
	18: from config.ru:7:in `require'
	17: from /Users/faiz/discenv/discourse/config/environment.rb:4:in `<top (required)>'
	16: from /Users/faiz/discenv/discourse/config/environment.rb:4:in `require'
	15: from /Users/faiz/discenv/discourse/config/application.rb:57:in `<top (required)>'
	14: from /Users/faiz/discenv/discourse/config/application.rb:58:in `<module:Discourse>'
	13: from /Users/faiz/discenv/discourse/config/application.rb:261:in `<class:Application>'
	12: from lib/discourse.rb:168:in `activate_plugins!'
	11: from lib/discourse.rb:168:in `each'
	10: from lib/discourse.rb:171:in `block in activate_plugins!'
	 9: from /Users/faiz/discenv/discourse/lib/plugin/instance.rb:486:in `activate!'
	 8: from /Users/faiz/discenv/discourse/lib/plugin/instance.rb:486:in `instance_eval'
	 7: from /Users/faiz/discenv/discourse/plugins/discourse-dictionary/plugin.rb:7:in `activate!'
	 6: from /Users/faiz/discenv/discourse/lib/plugin/instance.rb:560:in `gem'
	 5: from /Users/faiz/discenv/discourse/lib/plugin_gem.rb:20:in `load'
	 4: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/2.6.0/rubygems/specification.rb:1420:in `activate'
	 3: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/2.6.0/rubygems/specification.rb:1438:in `activate_dependencies'
	 2: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/2.6.0/rubygems/specification.rb:1438:in `each'
	 1: from /Users/faiz/.rbenv/versions/2.6.2/lib/ruby/2.6.0/rubygems/specification.rb:1449:in `block in activate_dependencies'
/Users/faiz/.rbenv/versions/2.6.2/lib/ruby/2.6.0/rubygems/dependency.rb:311:in `to_specs': Could not find 'plissken' (~> 0.1.0) among 285 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/Users/faiz/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0:/Users/faiz/.gem/ruby/2.6.0', execute `gem env` for more information

Process finished with exit code 1

I think its probably because of discourse not being able to install a dependency required by the gem itself. How do I fix this?

Update
Ok I fixed it. Its dependency called plissken had to be mentioned separately.
Why is that though?

6

Your target gem has a dependency that’s not included in the main Gemfile so you need to include it in plugin.rb

This can be a lengthy iterative process sometimes!

7

I would say a lengthy recursive process :rofl:

8

Also, I cannot use require: false there. I have to make it true and explicitly require it to make it work.

Wish that recursion could be handled by the computer one day :stuck_out_tongue_winking_eye:

9

Based off my own exp, an update:

  • Yes the version number is required
  • To get these dependancies, install ruby 2.6.5 (the current discourse ruby version in use) on your local machine, and create a Gemfile with the gems you want.
  • Execute “bundle install” in the terminal to create Gemfile.lock
  • This lists out all the gems, gem dependancies and version numbers.
  • Do as Rafael notes in plugin.rb

I am not sure what ‘required’ does but I set it to true

10

Hey, I know this was a while ago, but I am trying to install the eth gem and its failing on a dependency with this error:

I, [2023-08-28T15:00:23.737246 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
rake aborted!
Gem::MissingSpecError: Could not find 'forwardable' (~> 1.3) among 231 total gem(s)
Checked in 'GEM_PATH=/var/www/discourse/vendor/bundle/ruby/3.2.0:/var/www/discourse/plugins/discourse-radiant-member/gems/3.2.2' at: /var/www/discourse/plugins/discourse-radiant-member/gems/3.2.2/specifications/eth-0.5.11.gemspec, execute `gem env` for more information
/var/www/discourse/lib/plugin_gem.rb:25:in `load'
/var/www/discourse/lib/plugin/instance.rb:825:in `gem'
/var/www/discourse/plugins/discourse-radiant-member/plugin.rb:10:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:722:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:722:in `activate!'
/var/www/discourse/lib/discourse.rb:346:in `block in activate_plugins!'
/var/www/discourse/lib/discourse.rb:343:in `each'
/var/www/discourse/lib/discourse.rb:343:in `activate_plugins!'
/var/www/discourse/config/application.rb:218:in `block in <class:Application>'
/var/www/discourse/lib/plugin.rb:6:in `initialization_guard'
/var/www/discourse/config/application.rb:218:in `<class:Application>'
/var/www/discourse/config/application.rb:75:in `<module:Discourse>'
/var/www/discourse/config/application.rb:74:in `<top (required)>'
<internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'

Caused by:
Gem::MissingSpecError: Could not find 'forwardable' (~> 1.3) among 231 total gem(s)
Checked in 'GEM_PATH=/var/www/discourse/vendor/bundle/ruby/3.2.0:/var/www/discourse/plugins/discourse-radiant-member/gems/3.2.2' , execute `gem env` for more information
/var/www/discourse/lib/plugin_gem.rb:25:in `load'
/var/www/discourse/lib/plugin/instance.rb:825:in `gem'
/var/www/discourse/plugins/discourse-radiant-member/plugin.rb:10:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:722:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:722:in `activate!'
/var/www/discourse/lib/discourse.rb:346:in `block in activate_plugins!'
/var/www/discourse/lib/discourse.rb:343:in `each'
/var/www/discourse/lib/discourse.rb:343:in `activate_plugins!'
/var/www/discourse/config/application.rb:218:in `block in <class:Application>'
/var/www/discourse/lib/plugin.rb:6:in `initialization_guard'
/var/www/discourse/config/application.rb:218:in `<class:Application>'
/var/www/discourse/config/application.rb:75:in `<module:Discourse>'
/var/www/discourse/config/application.rb:74:in `<top (required)>'
<internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
(See full trace by running task with --trace)
I, [2023-08-28T15:00:26.013232 #1]  INFO -- : gem install eth -v 0.5.11 -i /var/www/discourse/plugins/discourse-radiant-member/gems/3.2.2 --no-document --ignore-dependencies --no-user-install
Successfully installed eth-0.5.11
1 gem installed

I, [2023-08-28T15:00:26.014015 #1]  INFO -- : Terminating async processes
I, [2023-08-28T15:00:26.014342 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
I, [2023-08-28T15:00:26.014700 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 111
2023-08-28 15:00:26.015 UTC [42] LOG:  received fast shutdown request
111:signal-handler (1693234826) Received SIGTERM scheduling shutdown...
2023-08-28 15:00:26.027 UTC [42] LOG:  aborting any active transactions
111:M 28 Aug 2023 15:00:26.030 # User requested shutdown...
111:M 28 Aug 2023 15:00:26.031 * Saving the final RDB snapshot before exiting.
2023-08-28 15:00:26.034 UTC [42] LOG:  background worker "logical replication launcher" (PID 51) exited with exit code 1
2023-08-28 15:00:26.035 UTC [46] LOG:  shutting down
111:M 28 Aug 2023 15:00:26.074 * DB saved on disk
111:M 28 Aug 2023 15:00:26.074 # Redis is now ready to exit, bye bye...
2023-08-28 15:00:26.087 UTC [42] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 1002 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
32dfbed48cf5bbb8f9a2d25b0d75388e296cd78ce7f56d37fb7a59c543888ad1

My repository is here- GitHub - JD0x2e/discourse-radiant-member: Get member status and update group membership · GitHub

I have required the forwardable dependency, among a few others just in case (which I got from my Gemfile.lock), but my app just can’t rebuild without throwing this error, any ideas?