Scheduling jobs from a plugin

dev
1

I am trying to set up a job that runs on a scheduled interval from within a plugin. In my plugin.rb I have

after_initialize do
    
    module ::Jobs
        
        module_function
        def count; @count end
        def count= v; @count=v end
        
        Jobs.count =1
    
        class PluginTestJob < ::Jobs::Scheduled
            every 30.seconds
            
            def execute(args)
                puts "I'm working (this job ran #{Jobs.count} times)"
                Jobs.count += 1
            end
        end
    end
  
end

I am trying to follow the pattern in /app/jobs/scheduled. Is after_initialize too late for the job to get scheduled and I have to manually schedule it? Is this on the right track at all?

1 Like
2

This seems to work.

after_initialize do

  class ::Jobs::ExampleJob < Jobs::Scheduled
    every 30.seconds

    def execute(args)
      puts "THIS IS A TEST"
    end
  end

end
5 Likes
3

This still appears not to be working on my image.

Just to be certain, I did test that the after_initialize block is getting called (i.e., probably not a cache issue, typo, etc).

4

Can you get the example job I put on here to work? You should see the output in the terminal window that you are running Sidekiq in.

1 Like
5

Looks like I failed to look into the documentation thoroughly enough. I did not have Sidekiq running at all, oops.

bundle exec sidekiq

It works now.

3 Likes
6

Addendum for future readers:

Let’s say you have

class ::Jobs::ExampleJob< Jobs::Scheduled
    every 1.day
    # ...
end

and you have started your app. If you change every to daily and restart, the change may not get picked up and the job will not start.

Clearing the tmp/ cache does not fix. I assume there is some cache for sidekiq or cron hiding somewhere that you could clear to fix. I couldn’t find it so I just renamed the class and it worked ¯\_(ツ)_/¯

7

Did you try just restarting sidekiq?

8

Yes. Restarted VM and all.

9

Apologies for using an old topic but it has some useful context and examples.

Can anyone offer advice on queuing a series of non-recurring jobs in a plugin?

I don’t want them to repeat or run at regular intervals, I just want them to process asynchronously from my control logic.

I’m using @angus’ excellent Custom Wizard Plugin to capture some information and then my plugin needs to create up to 77 topics.

Attempting to run this synchronously causes a timeout after 31 secs as the whole job probably needs >100 seconds to complete. This means that the job doesn’t finish and control is never passed back to the Wizard plugin.

What I’d like to do is queue the jobs that will create the topics in the background and then hand control back to the plugin for the topics to be created at Sidekiq’s will.

This code extract from above…

after_initialize do

  class ::Jobs::ExampleJob < Jobs::Scheduled
    every 30.seconds

    def execute(args)
      puts "THIS IS A TEST"
    end
  end

end

…shows me how to execute a recurring job, but I want to iterate over the topics to create and enqueue a job for each. I think I’d like to do something like

topics_to_create.each do |topic_details|
    enqueue_onceoff_topic_job( topics_details )
end

What should enqueue_onceoff_topic_job() look like?

Or would it be best to create one job that is passed the details of the 77 topics? Would that still have timeout issues if it is run in the background?

I’ve seen some Onceoff jobs in the app/jobs/onceoff section of the source, but it’s not clear to me how they are kicked off. Do you invoke the inner execute method? Is that enough to queue the jobs?

Many thanks in advance.

1 Like
10

OK for posterity, those that follow and my ageing memory (in case I need to remember how I did it)…

The problem was that the Wizard plugin was timing out because I was trying to create up to 77 topics synchronously. I needed to enqueue a series of jobs that would create topics asynchronously so that I could return control to the Wizard plugin to stop it timing out.

In looking at the various Job class definitions that I found in the source, I couldn’t figure out how to instantiate the class and put it on a sidekiq queue.

After some digging, this appears to come from Jobs.enqueue(:job_name, args). (I believe there are variations of this that schedule jobs and enqueue repeating jobs.

Anyway, the steps I took to defer the creation of topics…

Create the job that will create the topics

It appears to be a convention with plugins to create a plugin-name\jobs directory and inside this folder, I created a Ruby script called deferred_topic_creation.rb. It looks like this…

# school-points\jobs\deferred_topic_creation.rb
module Jobs
  class DeferredTopicCreation < Jobs::Base
    def execute(args)
      creator_user_id = args[:creator_user_id]
      custom_fields = {
        event_start: args[:event_start],
        event_end: args[:event_end]
      }

      creator_user = User.find(creator_user_id) || User.find(Discourse::SYSTEM_USER_ID)

      post_creator = PostCreator.new(
        creator_user,
        args.except!(:creator_user_id, :event_start, :event_end)
      )
      post = post_creator.create

      topic = Topic.find(post.topic_id)
      topic.custom_fields = custom_fields
      topic.save!
    end
  end
end

The code you want to execute when sidekiq finally processes the jobs seems to go inside the execute method, and, at least with an ordinary job (not scheduled or repeated) it is simply passed the args as a hash as the first parameter.

Gotcha! In keeping with other queueing systems I’ve used (in Python), you can’t pass Discourse / Rails objects through to the job in that args hash. If you do, the object is just serialised as a string. I don’t know how to instantiate the object from the string so my solution was to pass IDs of any discourse objects I needed (the user that would create the topics in my case) and then to .find() the object in the job execute method.

load the job code in your plugin - inside the after_initialize do block like so

after_initialize do
    .
    .
    .
    load File.expand_path('../jobs/deferred_topic_creation.rb', __FILE__)
    .
    .
    .

I don’t know if where it is placed in the block is important but the path must resolve to our job definition in the jobs directory.

Enqueue the jobs to create the topics

With that setup in place, you are then able to enqueue the job using a command like this

opts = {
  title: title,
  raw: content,
  category: child_category[:id],
  creator_user_id: creator_user_id,
  event_start: event_start.to_time.to_i,
  event_end: event_end.to_time.to_i
}
Jobs.enqueue(:deferred_topic_creation, opts)

Note that the name of the job is passed as a symbol here.

In development mode remember to stop your server and kill sidekiq before deleting the tmp directory contents and starting the server and sidekiq again.

8 Likes
11

Hi, If call the deferred_topic_creation.rb file in after_initialize it works fine. but In my case want to start when receiving an ajax request from admin/test/ endpoint.

12

It’s been a while - 5 years apparently but from memory…

The job code, for example:

# school-points\jobs\deferred_topic_creation.rb
module Jobs
  class DeferredTopicCreation < Jobs::Base
    def execute(args)
      creator_user_id = args[:creator_user_id]
      custom_fields = {
        event_start: args[:event_start],
        event_end: args[:event_end]
      }

      creator_user = User.find(creator_user_id) || User.find(Discourse::SYSTEM_USER_ID)

      post_creator = PostCreator.new(
        creator_user,
        args.except!(:creator_user_id, :event_start, :event_end)
      )
      post = post_creator.create

      topic = Topic.find(post.topic_id)
      topic.custom_fields = custom_fields
      topic.save!
    end
  end
end

should be LOADED in the after_initialize block. If this is not done, then the job can not be created. This does not create a topic, instead it simply registers the code as a job that can be enqueued later.

Then, when you actually want to do deferred creation of a topic (for you this is after your AJAX request), you would execute something like this:

opts = {
  title: title,
  raw: content,
  category: child_category[:id],
  creator_user_id: creator_user_id,
  event_start: event_start.to_time.to_i,
  event_end: event_end.to_time.to_i
}
Jobs.enqueue(:deferred_topic_creation, opts)

I hope that makes things clearer.

2 Likes