Are you always trying to give users a better experience when using your website or app? One of the most important ways to achieve this is by reducing the server response time. In this article, we will explore Active Job – which allows you to do so by queuing backends. You can also use the queue to help reduce traffic or upload to the server, allowing work to be done when the server is “free”.
What is Active Job?
Active Job in Rails is a framework that helps create tasks and allows them to run on a number of different queueing backends. The tasks can be regular periodic cleaning, uploading images to the server, sending mail … The most common queue systems used in Rails applications are Sidekiq, Resque and Delayed Job.
Use Active Job
Active Job has a fairly simple interface and installer. Here is how to use its features:
Create Job
Active Job when created via command will include job and necessary stubs.
1 2 3 4 5 | rails g job TweetNotifier invoke test_unit create test/jobs/tweet_notifier_job_test.rb create app/jobs/tweet_notifier_job.rb |
In the generated Job Class, the #perform method is called when the Job is executed, we can pass arbitrary parameters to this method.
1 2 3 4 5 6 7 8 | <span class="token keyword">class</span> <span class="token class-name">TweetNotifierJob</span> <span class="token operator"><</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> queue_as <span class="token symbol">:default</span> <span class="token keyword">def</span> <span class="token function">perform</span> <span class="token punctuation">(</span> user <span class="token punctuation">)</span> user <span class="token punctuation">.</span> update_stats <span class="token keyword">end</span> <span class="token keyword">end</span> |
Add a Job to the queue
We have the option to add jobs to the queue as below:
1 2 3 | <span class="token comment"># Thêm job vào hàng đợi để chạy sớm nhất có thể khi server rảnh</span> <span class="token constant">GuestsCleanupJob</span> <span class="token punctuation">.</span> perform_later guest |
1 2 3 | <span class="token comment"># Thêm job vào hàng đợi để chạy vào thời gian định sẵn</span> <span class="token constant">GuestsCleanupJob</span> <span class="token punctuation">.</span> <span class="token function">set</span> <span class="token punctuation">(</span> wait_until <span class="token punctuation">:</span> <span class="token constant">Date</span> <span class="token punctuation">.</span> tomorrow <span class="token punctuation">.</span> noon <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> guest <span class="token punctuation">)</span> |
1 2 3 | <span class="token comment"># Thêm job vào hàng đợi để chạy sau khoảng thời gian chờ kể từ hiện tại</span> <span class="token constant">GuestsCleanupJob</span> <span class="token punctuation">.</span> <span class="token function">set</span> <span class="token punctuation">(</span> wait <span class="token punctuation">:</span> <span class="token number">1.</span> week <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> guest <span class="token punctuation">)</span> |
1 2 3 | <span class="token comment"># `perform_now` and `perform_later` sẽ gọi method `perform` vì thể có thể truyền tùy ý các tham số</span> <span class="token constant">GuestsCleanupJob</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> guest1 <span class="token punctuation">,</span> guest2 <span class="token punctuation">,</span> filter <span class="token punctuation">:</span> <span class="token string">'some_filter'</span> <span class="token punctuation">)</span> |
Execute Job
If no adapter is set, the job will be executed immediately.
Backends
Active Job provides adapters available for some queue systems such as: Sidekiq, Resque, Delayed Job, … You can see details in the document of ActiveJob :: QueueAdapters
Install Backend
You can set the queue system for your application.
1 2 3 4 5 6 7 8 9 | <span class="token comment"># config/application.rb</span> <span class="token keyword">module</span> <span class="token constant">YourApp</span> <span class="token keyword">class</span> <span class="token class-name">Application</span> <span class="token operator"><</span> <span class="token constant">Rails</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Application</span> <span class="token comment"># Be sure to have the adapter's gem in your Gemfile and follow</span> <span class="token comment"># the adapter's specific installation and deployment instructions.</span> config <span class="token punctuation">.</span> active_job <span class="token punctuation">.</span> queue_adapter <span class="token operator">=</span> <span class="token symbol">:sidekiq</span> <span class="token keyword">end</span> <span class="token keyword">end</span> |
Queue
Most adapters support multiple queues. You can schedule jobs to run on a specific queue
1 2 3 4 5 | <span class="token keyword">class</span> <span class="token class-name">GuestsCleanupJob</span> <span class="token operator"><</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> queue_as <span class="token symbol">:low_priority</span> <span class="token comment">#....</span> <span class="token keyword">end</span> |
If you want to more clearly manage the queue the job will run, you can use the #set method as follows:
1 2 | <span class="token constant">MyJob</span> <span class="token punctuation">.</span> <span class="token function">set</span> <span class="token punctuation">(</span> queue <span class="token punctuation">:</span> <span class="token symbol">:another_queue</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> record <span class="token punctuation">)</span> |
Or want to manage from Job level, you can pass 1 block to the method #queue_as. The block will be executed in the job context (called self.arguments) and must return the queue name.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | <span class="token keyword">class</span> <span class="token class-name">ProcessVideoJob</span> <span class="token operator"><</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> queue_as <span class="token keyword">do</span> video <span class="token operator">=</span> <span class="token keyword">self</span> <span class="token punctuation">.</span> arguments <span class="token punctuation">.</span> first <span class="token keyword">if</span> video <span class="token punctuation">.</span> owner <span class="token punctuation">.</span> premium <span class="token operator">?</span> <span class="token symbol">:premium_videojobs</span> <span class="token keyword">else</span> <span class="token symbol">:videojobs</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token keyword">def</span> <span class="token function">perform</span> <span class="token punctuation">(</span> video <span class="token punctuation">)</span> <span class="token comment"># do process video</span> <span class="token keyword">end</span> <span class="token keyword">end</span> <span class="token constant">ProcessVideoJob</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> <span class="token constant">Video</span> <span class="token punctuation">.</span> last <span class="token punctuation">)</span> |
Callbacks
Active Job provides hooks for the lifetime of a job. Callback allows to attach logic to the job life cycle.
Callbacks are available
- before_enqueue
- around_enqueue
- after_enqueue
- before_perform
- around_perform
- after_perform
How to use callbacks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | <span class="token keyword">class</span> <span class="token class-name">GuestsCleanupJob</span> <span class="token operator"><</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> queue_as <span class="token symbol">:default</span> before_enqueue <span class="token keyword">do</span> <span class="token operator">|</span> job <span class="token operator">|</span> <span class="token comment"># do something with the job instance</span> <span class="token keyword">end</span> around_perform <span class="token keyword">do</span> <span class="token operator">|</span> job <span class="token punctuation">,</span> block <span class="token operator">|</span> <span class="token comment"># do something before perform</span> block <span class="token punctuation">.</span> call <span class="token comment"># do something after perform</span> <span class="token keyword">end</span> <span class="token keyword">def</span> perform <span class="token comment"># Do something later</span> <span class="token keyword">end</span> <span class="token keyword">end</span> |
Some tasks should use Active Job
Send email
Emailing is the most common task possible and should be done in the background job. There is no reason to send an email immediately (before the response is displayed), all emails should be delivered to the queue. Even if the email server responds in 100ms, there are still 100ms that you are causing users to wait unnecessarily. Sending emails through a background job is extremely simple with Active Job, as it is built into ActionMailer. By changing the deliver_now method to deliver_later, the Active Job will automatically send emails in the queue asynchronously.
1 2 | <span class="token constant">UserMailer</span> <span class="token punctuation">.</span> <span class="token function">welcome</span> <span class="token punctuation">(</span> <span class="token variable">@user</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> deliver_later |
Image processing
Images may take some time to be processed. It takes more time if you have several (or more) different photo styles and sizes to create. Fortunately, both Paperclip and CarrierWave have additional gems that can help process these images in the queue instead of at the time of upload.
Paperclip uses a Delayed Paperclip gem, supports Active Job, and CarrierWave uses the CarrierWave Backgrounder gem. With Delayed Paperclip , you just need to call an additional method to let it know what you want to handle in the background and the gem will handle the rest. You can ask it to handle some types immediately, while others handle them in the queue.
1 2 3 4 5 6 | <span class="token keyword">class</span> <span class="token class-name">User</span> <span class="token operator"><</span> <span class="token constant">ActiveRecord</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> has_attached_file <span class="token symbol">:avatar</span> <span class="token punctuation">,</span> styles <span class="token punctuation">:</span> <span class="token punctuation">{</span> small <span class="token punctuation">:</span> <span class="token string">"25x25#"</span> <span class="token punctuation">,</span> medium <span class="token punctuation">:</span> <span class="token string">"50x50#"</span> <span class="token punctuation">,</span> large <span class="token punctuation">:</span> <span class="token string">"200x200#"</span> <span class="token punctuation">}</span> <span class="token punctuation">,</span> only_process <span class="token punctuation">:</span> <span class="token punctuation">[</span> <span class="token symbol">:small</span> <span class="token punctuation">]</span> process_in_background <span class="token symbol">:avatar</span> <span class="token punctuation">,</span> only_process <span class="token punctuation">:</span> <span class="token punctuation">[</span> <span class="token symbol">:medium</span> <span class="token punctuation">,</span> <span class="token symbol">:large</span> <span class="token punctuation">]</span> <span class="token keyword">end</span> |
The example above allows image processing: small immediately, while: medium and: large are done in the background.
Upload content
Usually when you have a user uploading content, it needs to be processed. This may be a CSV file that needs to be imported into the system, an image to create a thumbnail image, or a video to be processed. A large CSV file may take several minutes to process, during which time the connection may be timed out. You should handle most asynchronous upload data in the queue. The usage process is as follows:
- Accept the file and upload it to S3 (or wherever you are hosting user-generated content).
- Add a job to the queue to process this file.
- Users will immediately see a successful page telling them that their file has been submitted for processing.
- The system will download the file, process it and mark it as processed. Another note is that you will want to store an import report in the database. It may include any records not processed due to invalid data. What to do is to create an error message file for each import that the user can download.
Use external API
External APIs may be unstable, slow, and the user experience should not depend on them whenever possible. For example, below where we use the IP address to find out some geographic information using the Telize API. It usually responds in 200ms to 500ms, added to your current response time, which can make a big difference. All external APIs should be handled in the same way: Use background jobs if possible. First, we schedule a job execution, passing in the IP address of the current request.
1 2 | <span class="token constant">LogIpAddressJob</span> <span class="token punctuation">.</span> <span class="token function">perform_later</span> <span class="token punctuation">(</span> request <span class="token punctuation">.</span> remote_ip <span class="token punctuation">)</span> |
1 2 3 4 5 6 7 8 9 | <span class="token keyword">class</span> <span class="token class-name">LogIpAddressJob</span> <span class="token operator"><</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">:</span> <span class="token punctuation">:</span> <span class="token constant">Base</span> queue_as <span class="token symbol">:default</span> <span class="token keyword">def</span> <span class="token function">perform</span> <span class="token punctuation">(</span> ip <span class="token punctuation">)</span> ip <span class="token operator">=</span> <span class="token string">"66.207.202.15"</span> <span class="token keyword">if</span> ip <span class="token operator">==</span> <span class="token string">"::1"</span> <span class="token constant">LogIpAddress</span> <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> ip <span class="token punctuation">)</span> <span class="token keyword">end</span> <span class="token keyword">end</span> |
Here we perform the actual work to be done. We will make a real remote request to the API to show how long this request could take.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | <span class="token keyword">class</span> <span class="token class-name">LogIpAddress</span> <span class="token keyword">def</span> <span class="token keyword">self</span> <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> ip <span class="token punctuation">)</span> <span class="token keyword">self</span> <span class="token punctuation">.</span> <span class="token keyword">new</span> <span class="token punctuation">(</span> ip <span class="token punctuation">)</span> <span class="token punctuation">.</span> log <span class="token keyword">end</span> <span class="token keyword">def</span> <span class="token function">initialize</span> <span class="token punctuation">(</span> ip <span class="token punctuation">)</span> <span class="token variable">@ip</span> <span class="token operator">=</span> ip <span class="token keyword">end</span> <span class="token keyword">def</span> get_geo_info <span class="token constant">HTTParty</span> <span class="token punctuation">.</span> <span class="token function">get</span> <span class="token punctuation">(</span> <span class="token string">"http://www.telize.com/geoip/ <span class="token interpolation"><span class="token delimiter tag">#{</span> @ip <span class="token delimiter tag">}</span></span> "</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> parsed_response <span class="token keyword">end</span> <span class="token keyword">def</span> log geo_info <span class="token operator">=</span> get_geo_info <span class="token constant">Rails</span> <span class="token punctuation">.</span> logger <span class="token punctuation">.</span> <span class="token function">debug</span> <span class="token punctuation">(</span> geo_info <span class="token punctuation">)</span> <span class="token comment"># log response to database </span> <span class="token keyword">end</span> <span class="token keyword">end</span> |
You can see what happened in Rails logs:
1 2 3 4 5 | <span class="token punctuation">[</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">]</span> <span class="token constant">Enqueued</span> <span class="token constant">LogIpAddressJob</span> <span class="token punctuation">(</span> <span class="token constant">Job</span> <span class="token constant">ID</span> <span class="token punctuation">:</span> <span class="token number">839</span> db962 <span class="token operator">-</span> <span class="token number">28</span> a0 <span class="token operator">-</span> <span class="token number">4e9</span> d <span class="token operator">-</span> <span class="token number">9168</span> <span class="token operator">-</span> b08674ba192f <span class="token punctuation">)</span> to <span class="token function">Inline</span> <span class="token punctuation">(</span> default <span class="token punctuation">)</span> with arguments <span class="token punctuation">:</span> <span class="token string">"::1"</span> <span class="token punctuation">[</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token constant">LogIpAddressJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token number">839</span> db962 <span class="token operator">-</span> <span class="token number">28</span> a0 <span class="token operator">-</span> <span class="token number">4e9</span> d <span class="token operator">-</span> <span class="token number">9168</span> <span class="token operator">-</span> b08674ba192f <span class="token punctuation">]</span> <span class="token constant">Performing</span> <span class="token constant">LogIpAddressJob</span> from <span class="token function">Inline</span> <span class="token punctuation">(</span> default <span class="token punctuation">)</span> with arguments <span class="token punctuation">:</span> <span class="token string">"::1"</span> <span class="token punctuation">[</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token constant">LogIpAddressJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token number">839</span> db962 <span class="token operator">-</span> <span class="token number">28</span> a0 <span class="token operator">-</span> <span class="token number">4e9</span> d <span class="token operator">-</span> <span class="token number">9168</span> <span class="token operator">-</span> b08674ba192f <span class="token punctuation">]</span> <span class="token punctuation">{</span> <span class="token string">"longitude"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token operator">-</span> <span class="token number">79.4167</span> <span class="token punctuation">,</span> <span class="token string">"latitude"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token number">43.6667</span> <span class="token punctuation">,</span> <span class="token string">"asn"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"AS21949"</span> <span class="token punctuation">,</span> <span class="token string">"offset"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"-4"</span> <span class="token punctuation">,</span> <span class="token string">"ip"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"66.207.202.15"</span> <span class="token punctuation">,</span> <span class="token string">"area_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"0"</span> <span class="token punctuation">,</span> <span class="token string">"continent_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"NA"</span> <span class="token punctuation">,</span> <span class="token string">"dma_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"0"</span> <span class="token punctuation">,</span> <span class="token string">"city"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"Toronto"</span> <span class="token punctuation">,</span> <span class="token string">"timezone"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"America/Toronto"</span> <span class="token punctuation">,</span> <span class="token string">"region"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"Ontario"</span> <span class="token punctuation">,</span> <span class="token string">"country_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"CA"</span> <span class="token punctuation">,</span> <span class="token string">"isp"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"Beanfield Technologies Inc."</span> <span class="token punctuation">,</span> <span class="token string">"postal_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"M6G"</span> <span class="token punctuation">,</span> <span class="token string">"country"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"Canada"</span> <span class="token punctuation">,</span> <span class="token string">"country_code3"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"CAN"</span> <span class="token punctuation">,</span> <span class="token string">"region_code"</span> <span class="token operator">=</span> <span class="token operator">></span> <span class="token string">"ON"</span> <span class="token punctuation">}</span> <span class="token punctuation">[</span> <span class="token constant">ActiveJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token constant">LogIpAddressJob</span> <span class="token punctuation">]</span> <span class="token punctuation">[</span> <span class="token number">839</span> db962 <span class="token operator">-</span> <span class="token number">28</span> a0 <span class="token operator">-</span> <span class="token number">4e9</span> d <span class="token operator">-</span> <span class="token number">9168</span> <span class="token operator">-</span> b08674ba192f <span class="token punctuation">]</span> <span class="token constant">Performed</span> <span class="token constant">LogIpAddressJob</span> from <span class="token function">Inline</span> <span class="token punctuation">(</span> default <span class="token punctuation">)</span> <span class="token keyword">in</span> <span class="token number">572.39</span> ms |
Conclude
Active Job is a great addition to Rails. It provides a clear and unique interface for adding jobs and handling jobs. If you are starting a new Rails project or adding a queuing system to an existing project, be sure to consider using Active Job instead of queuing directly. Using queues can increase website availability (by reducing response time), providing more responsive server and load times (by transmitting various jobs and servers).