Scheduling jobs from a plugin

Development
1

I am trying to set up a job that runs on a scheduled interval from within a plugin. In my plugin.rb I have

after_initialize do
    
    module ::Jobs
        
        module_function
        def count; @count end
        def count= v; @count=v end
        
        Jobs.count =1
    
        class PluginTestJob < ::Jobs::Scheduled
            every 30.seconds
            
            def execute(args)
                puts "I'm working (this job ran #{Jobs.count} times)"
                Jobs.count += 1
            end
        end
    end
  
end

I am trying to follow the pattern in /app/jobs/scheduled. Is after_initialize too late for the job to get scheduled and I have to manually schedule it? Is this on the right track at all?

2

This seems to work.

after_initialize do

  class ::Jobs::ExampleJob < Jobs::Scheduled
    every 30.seconds

    def execute(args)
      puts "THIS IS A TEST"
    end
  end

end
3

This still appears not to be working on my image.

Just to be certain, I did test that the after_initialize block is getting called (i.e., probably not a cache issue, typo, etc).

4

Can you get the example job I put on here to work? You should see the output in the terminal window that you are running Sidekiq in.

5

Looks like I failed to look into the documentation thoroughly enough. I did not have Sidekiq running at all, oops.

bundle exec sidekiq

It works now.

6

Addendum for future readers:

Let’s say you have

class ::Jobs::ExampleJob< Jobs::Scheduled
    every 1.day
    # ...
end

and you have started your app. If you change every to daily and restart, the change may not get picked up and the job will not start.

Clearing the tmp/ cache does not fix. I assume there is some cache for sidekiq or cron hiding somewhere that you could clear to fix. I couldn’t find it so I just renamed the class and it worked ¯\_(ツ)_/¯

7

Did you try just restarting sidekiq?

8

Yes. Restarted VM and all.

9

Apologies for using an old topic but it has some useful context and examples.

Can anyone offer advice on queuing a series of non-recurring jobs in a plugin?

I don’t want them to repeat or run at regular intervals, I just want them to process asynchronously from my control logic.

I’m using @angus’ excellent Custom Wizard Plugin to capture some information and then my plugin needs to create up to 77 topics.

Attempting to run this synchronously causes a timeout after 31 secs as the whole job probably needs >100 seconds to complete. This means that the job doesn’t finish and control is never passed back to the Wizard plugin.

What I’d like to do is queue the jobs that will create the topics in the background and then hand control back to the plugin for the topics to be created at Sidekiq’s will.

This code extract from above…

after_initialize do

  class ::Jobs::ExampleJob < Jobs::Scheduled
    every 30.seconds

    def execute(args)
      puts "THIS IS A TEST"
    end
  end

end

…shows me how to execute a recurring job, but I want to iterate over the topics to create and enqueue a job for each. I think I’d like to do something like

topics_to_create.each do |topic_details|
    enqueue_onceoff_topic_job( topics_details )
end

What should enqueue_onceoff_topic_job() look like?

Or would it be best to create one job that is passed the details of the 77 topics? Would that still have timeout issues if it is run in the background?

I’ve seen some Onceoff jobs in the app/jobs/onceoff section of the source, but it’s not clear to me how they are kicked off. Do you invoke the inner execute method? Is that enough to queue the jobs?

Many thanks in advance.

10

OK for posterity, those that follow and my ageing memory (in case I need to remember how I did it)…

The problem was that the Wizard plugin was timing out because I was trying to create up to 77 topics synchronously. I needed to enqueue a series of jobs that would create topics asynchronously so that I could return control to the Wizard plugin to stop it timing out.

In looking at the various Job class definitions that I found in the source, I couldn’t figure out how to instantiate the class and put it on a sidekiq queue.

After some digging, this appears to come from Jobs.enqueue(:job_name, args). (I believe there are variations of this that schedule jobs and enqueue repeating jobs.

Anyway, the steps I took to defer the creation of topics…

Create the job that will create the topics

It appears to be a convention with plugins to create a plugin-name\jobs directory and inside this folder, I created a Ruby script called deferred_topic_creation.rb. It looks like this…

# school-points\jobs\deferred_topic_creation.rb
module Jobs
  class DeferredTopicCreation < Jobs::Base
    def execute(args)
      creator_user_id = args[:creator_user_id]
      custom_fields = {
        event_start: args[:event_start],
        event_end: args[:event_end]
      }

      creator_user = User.find(creator_user_id) || User.find(Discourse::SYSTEM_USER_ID)

      post_creator = PostCreator.new(
        creator_user,
        args.except!(:creator_user_id, :event_start, :event_end)
      )
      post = post_creator.create

      topic = Topic.find(post.topic_id)
      topic.custom_fields = custom_fields
      topic.save!
    end
  end
end

The code you want to execute when sidekiq finally processes the jobs seems to go inside the execute method, and, at least with an ordinary job (not scheduled or repeated) it is simply passed the args as a hash as the first parameter.

Gotcha! In keeping with other queueing systems I’ve used (in Python), you can’t pass Discourse / Rails objects through to the job in that args hash. If you do, the object is just serialised as a string. I don’t know how to instantiate the object from the string so my solution was to pass IDs of any discourse objects I needed (the user that would create the topics in my case) and then to .find() the object in the job execute method.

load the job code in your plugin - inside the after_initialize do block like so

after_initialize do
    .
    .
    .
    load File.expand_path('../jobs/deferred_topic_creation.rb', __FILE__)
    .
    .
    .

I don’t know if where it is placed in the block is important but the path must resolve to our job definition in the jobs directory.

Enqueue the jobs to create the topics

With that setup in place, you are then able to enqueue the job using a command like this

opts = {
  title: title,
  raw: content,
  category: child_category[:id],
  creator_user_id: creator_user_id,
  event_start: event_start.to_time.to_i,
  event_end: event_end.to_time.to_i
}
Jobs.enqueue(:deferred_topic_creation, opts)

Note that the name of the job is passed as a symbol here.

In development mode remember to stop your server and kill sidekiq before deleting the tmp directory contents and starting the server and sidekiq again.

11

안녕하세요. after_initialize에서 deferred_topic_creation.rb 파일을 호출하면 정상적으로 작동합니다. 하지만 제 경우에는 admin/test/ 엔드포인트에서 AJAX 요청을 받을 때 시작되도록 하고 싶습니다.

12

오래간만입니다. 기억에 따르면 5년 전이었네요.

예를 들어, 작업 코드(job code)는 다음과 같습니다:

# school-points\jobs\deferred_topic_creation.rb
module Jobs
  class DeferredTopicCreation < Jobs::Base
    def execute(args)
      creator_user_id = args[:creator_user_id]
      custom_fields = {
        event_start: args[:event_start],
        event_end: args[:event_end]
      }

      creator_user = User.find(creator_user_id) || User.find(Discourse::SYSTEM_USER_ID)

      post_creator = PostCreator.new(
        creator_user,
        args.except!(:creator_user_id, :event_start, :event_end)
      )
      post = post_creator.create

      topic = Topic.find(post.topic_id)
      topic.custom_fields = custom_fields
      topic.save!
    end
  end
end

이 코드는 after_initialize 블록에서 로드(LOAD)되어야 합니다. 이렇게 하지 않으면 작업을 생성할 수 없습니다. 이 과정은 실제로 주제를 생성하는 것이 아니라, 나중에 큐에 추가할 수 있는 작업으로 코드를 등록하는 것일 뿐입니다.

그리고 실제로 주제의 지연 생성(deferred creation)을 수행하고 싶을 때(귀하의 경우 AJAX 요청 이후에 해당합니다), 다음과 같은 코드를 실행하면 됩니다:

opts = {
  title: title,
  raw: content,
  category: child_category[:id],
  creator_user_id: creator_user_id,
  event_start: event_start.to_time.to_i,
  event_end: event_end.to_time.to_i
}
Jobs.enqueue(:deferred_topic_creation, opts)

이 설명이 도움이 되기를 바랍니다.