Transforming simple ruby files into a ruby gem

SebastianADW · December 3, 2018, 9:56pm

Hey,

i am trying to create my first plugin for discourse. I already have the ruby code working fine. I have 4 different ruby files with different kind of methods in it, i like to have them splitted. And a Gemfile and a Gemfile.lock
How can i transform them now into a Discourse Plugin?
I already read the Plugin Beginner Tutorial, but it focused on JavaScript Code and not so much on pure ruby code i guess? I am just not sure where to put my files. Is it possible to let them split up and copy the code of the “main” file into the plugin.rb ?
I didnt use any kind of class definitions or something like that just tried to split different functionality in different files and methods.
Where do i have to put the Gemfile and the Gemfile.lock

Tried it local but it didnt find the twitter gem which is necessary for it to work, i dont know where the issue came from.

blake · December 4, 2018, 3:09am

Glad to here you are working on your first plugin . I would look at the examples of other plugins and see how they do the ruby side of things.

I don’t think your plugin actually needs to have a Gemfile, but you need to specify all of your dependencies and sub-dependencies inside of your plugin.rb file. Discourse will then install all of the gems when it loads the plugin. See this example:

github.com

xfalcox/discourse-backups-to-dropbox/blob/master/plugin.rb

# name: discourse-backups-to-dropbox
# about: Backups discourse backups in dropbox
# version: 0.0.2
# authors: Rafael dos Santos Silva <xfalcox@gmail.com>
# url: https://github.com/xfalcox/discourse-backups-to-dropbox

gem 'public_suffix', '2.0.5', {require: false }
gem 'domain_name', '0.5.20170404', {require: false }

gem 'addressable', '2.5.1', {require: false }
gem 'http_parser.rb', '0.6.0', {require: false }
gem 'http-cookie', '1.0.3', {require: false }
gem 'http-form_data', '1.0.1', {require: false }

gem 'http', '2.0.3', {require: false }

gem 'dropbox-sdk-v2', '0.0.3', { require: false }

enabled_site_setting :discourse_backups_to_dropbox_enabled

This file has been truncated. show original

Any files your plugin uses you can put inside of the “lib” folder and then require them inside of plugin.rb.

github.com

discourse/discourse-data-explorer/blob/main/plugin.rb#L579


      
          # frozen_string_literal: true
          
          # name: discourse-data-explorer
          # about: Allows you to make SQL queries against your live database, allowing for up-to-the-minute stats reporting.
          # meta_topic_id: 32566
          # version: 0.3
          # authors: Riking
          # url: https://github.com/discourse/discourse-data-explorer
          
          enabled_site_setting :data_explorer_enabled
          
          register_asset "stylesheets/explorer.scss"
          
          register_svg_icon "caret-down"
          register_svg_icon "caret-right"
          register_svg_icon "chevron-left"
          register_svg_icon "exclamation-circle"
          register_svg_icon "info"
          register_svg_icon "pencil-alt"
          register_svg_icon "upload"
          
          add_admin_route "explorer.title", "explorer"
          
          module ::DiscourseDataExplorer
            PLUGIN_NAME = "discourse-data-explorer"
          
            # This should always match the max value for the
            # data_explorer_query_result_limit site setting
            QUERY_RESULT_MAX_LIMIT = 10_000
          end
          
          require_relative "lib/discourse_data_explorer/engine"
          
          after_initialize do
            require_relative "app/jobs/scheduled/delete_hidden_queries"
            require_relative "lib/discourse_data_explorer/data_explorer"
            require_relative "lib/discourse_data_explorer/parameter"
            require_relative "lib/discourse_data_explorer/queries"
            require_relative "lib/discourse_data_explorer/query_group_bookmarkable"
          
            GlobalSetting.add_default(:max_data_explorer_api_reqs_per_10_seconds, 2)
          
            # Available options:
            #   - warn
            #   - warn+block
            #   - block
            GlobalSetting.add_default(:max_data_explorer_api_req_mode, "warn")
          
            add_to_class(:guardian, :user_is_a_member_of_group?) do |group|
              return false if !current_user
              return true if current_user.admin?
              current_user.group_ids.include?(group.id)
            end
          
            add_to_class(:guardian, :user_can_access_query?) do |query|
              return false if !current_user
              return true if current_user.admin?
              query.groups.blank? || query.groups.any? { |group| user_is_a_member_of_group?(group) }
            end
          
            add_to_class(:guardian, :group_and_user_can_access_query?) do |group, query|
              return false if !current_user
              return true if current_user.admin?
              user_is_a_member_of_group?(group) && query.groups.exists?(id: group.id)
            end
          
            add_to_serializer(
              :group_show,
              :has_visible_data_explorer_queries,
              include_condition: -> { scope.user_is_a_member_of_group?(object) },
            ) { DiscourseDataExplorer::Query.for_group(object).exists? }
          
            register_bookmarkable(DiscourseDataExplorer::QueryGroupBookmarkable)
          
            add_api_key_scope(
              :discourse_data_explorer,
              { run_queries: { actions: %w[discourse_data_explorer/query#run], params: %i[id] } },
            )
          
            require_relative "lib/report_generator"
            require_relative "lib/result_to_markdown"
            reloadable_patch do
              if defined?(DiscourseAutomation)
                add_automation_scriptable("recurring_data_explorer_result_pm") do
                  queries =
                    DiscourseDataExplorer::Query
                      .where(hidden: false)
                      .map { |q| { id: q.id, translated_name: q.name } }
                  field :recipients, component: :email_group_user, required: true
                  field :query_id, component: :choices, required: true, extra: { content: queries }
                  field :query_params, component: :"key-value", accepts_placeholders: true
                  field :skip_empty, component: :boolean
          
                  version 1
                  triggerables [:recurring]
          
                  script do |_, fields, automation|
                    recipients = Array(fields.dig("recipients", "value")).uniq
                    query_id = fields.dig("query_id", "value")
                    query_params = fields.dig("query_params", "value") || {}
                    skip_empty = fields.dig("skip_empty", "value") || false
          
                    unless SiteSetting.data_explorer_enabled
                      Rails.logger.warn "#{DiscourseDataExplorer::PLUGIN_NAME} - plugin must be enabled to run automation #{automation.id}"
                      next
                    end
          
                    if recipients.blank?
                      Rails.logger.warn "#{DiscourseDataExplorer::PLUGIN_NAME} - couldn't find any recipients for automation #{automation.id}"
                      next
                    end
          
                    DiscourseDataExplorer::ReportGenerator
                      .generate(query_id, query_params, recipients, { skip_empty: })
                      .each do |pm|
                        begin
                          utils.send_pm(pm, automation_id: automation.id, prefers_encrypt: false)
                        rescue ActiveRecord::RecordNotSaved => e
                          Rails.logger.warn "#{DiscourseDataExplorer::PLUGIN_NAME} - couldn't send PM for automation #{automation.id}: #{e.message}"
                        end
                      end
                  end
                end
              end
            end
          end

This file has been truncated. show original

SebastianADW · December 5, 2018, 8:33pm

Yeah thanks for your response!
I tried to follow all the advices and format the dependencies like the other plugins.

But i always get that kind of error when i try to start the rails server:

/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:inrequire’: cannot load such file – twitter (LoadError)`

i required twitter in the files where i need it put this in the normal plugin.rb

gem "bundler"
gem "net"
gem "twitter"

(obviously there are more gems i put in like this, thats just for an example)
Do you have any idea why its not working?
I also tried to specify the version of the gem like that gem "twitter", "6.2" and add the required false statement, but still the same error.
Maybe good to know, its a local discourse server

I think i just dont get how the gems are working in the plugin environment, cause i always get errors regarding them, for example i also get: you are specifying gem i18n in … however it does not exist …

blake · December 7, 2018, 8:15pm

Not exactly, but I still think it is dependency issues and not specifying the gems correctly.

I think it has to be the full version number like 6.2.0. I also don’t think you need to specify “bundler” or “net”.

Apparently when writing a plugin you have to go through dependency hell to get it all working and happy. Basically the workflow I go though is start with the main gem that I want. In this case it is the twitter gem. Make sure this one is always listed as the last gem in your plugin.rb file.

gem 'twitter', '6.2.0',  { require: false }

Then I look at all the runtime dependencies for this gem and ONE AT A TIME list each dependency with each subdependency above the twitter gem. Run bundle exec rails s between each dependency and check for (Gem::MissingSpecError) to know which dependency to add next.

Apparently the addressable gem (2.5.2, instead of 2.3.8) is already required by a discourse gem, so it should be safe to ignore. (You can run gem dependency addressable --reverse-dependencies to see which gems require it).

Your plugin.rb should look something like:

gem 'buftok',         '0.2.0',        { require: false }
gem 'equalizer',      '0.0.11',       { require: false }
gem 'domain_name',    '0.5.20180417', { require: false }
gem 'http-cookie',    '1.0.3',        { require: false }
gem 'http-form_data', '2.0.0',        { require: false }
gem 'http_parser.rb', '0.6.0',        { require: false }
gem 'http',           '3.3.0',        { require: false }
gem 'memoizable',     '0.4.2',        { require: false }
gem 'naught',         '1.1.0',        { require: false }
gem 'simple_oauth',   '0.3.1',        { require: false }
gem 'twitter',        '6.2.0',        { require: false }

#gem 'addressable',    '2.3.8', { require: false } # Already required in Discourse
#gem 'multipart-post', '2.0.0', { require: false } # Already required in Discourse

require 'twitter'

client = Twitter::REST::Client.new do |config|
  config.consumer_key        = "YOUR_CONSUMER_KEY"
  config.consumer_secret     = "YOUR_CONSUMER_SECRET"
  config.access_token        = "YOUR_ACCESS_TOKEN"
  config.access_token_secret = "YOUR_ACCESS_SECRET"
end

bundle exec rails s should now run successfully and you can now write your plugin and break things out into separate files if you’d like, but the gem dependencies should stay in the plugin.rb file.

SebastianADW · December 10, 2018, 9:10pm

Thank you so much, honestly i have no clue why its working now, maybe i had a few dependencys wrong or in a wrong order, but now it works totally fine!
A quick follow up question, what would you recommend to automize the file? So for example i want to have it run every 10 secs or something like that.

SebastianADW · December 14, 2018, 4:40pm

Update:

Now i use sidekiq to run my main file.

after_initialize do
class MainPlugin
include Sidekiq

require_relative File.expand_path(‘…/lib/gen_all_posts_url.rb’, FILE)
require_relative File.expand_path(‘…/lib/get_tweets.rb’, FILE)
##morecode

end
end

Sidekiq::Cron::Job.create(name: ‘MainPlugin every minute’, cron: ’ *', class:‘MainPlugin’)

But its not working, the strange thing about it is, that i tried for class:‘plugin.rb’ once, which was obviously wrong - so i switched it back, but i am still getting the error message:

Job exception: wrong constant name plugin.rb

I dont get what i am doing wrong here

Topic		Replies	Views
Discourse can't find gems specified in the plugin Dev	15	2534	March 20, 2022
Requiring external gems Dev	9	1692	August 28, 2023
Problem upgrading multi-site/multi-containers Discourse instance Installation	9	1719	May 13, 2022
Package a Discourse plugin as a gem Developers plugin-guides , how-to	4	502	July 27, 2023
Using "custom" ruby gems Dev	5	1004	April 28, 2022

Transforming simple ruby files into a ruby gem

Related topics