Traditionally, enqueuing multiple jobs involved individual calls, leading to potential performance bottlenecks when dealing with large volumes of tasks.
Below is an example of using
sidekiq as adapter
and to enqueue multiple jobs it makes lot of round trips to
Enter perform_all_later method tailored to streamline the process of enqueuing multiple jobs simultaneously within ActiveJob.
perform_all_later method leverages Sidekiq’s push_bulk functionality, enabling a significant optimization in the enqueuing process. Eliminating repeated Redis round-trips, drastically reduces the overhead associated with queuing numerous jobs, thereby enhancing overall application performance.
perform_all_later method accepts an array of job instances
and doesn’t run any callbacks like ActiveRecord bulk methods.
perform_all_later to Enqueue EmailNotificationJob for multiple users
For scenarios involving enqueuing over 1000 jobs and implementing delays for specific tasks, passing instantiated jobs directly allows adding individual delays for each job.
perform_all_later method allows us to enqueue multiple jobs that belong to different classes.
Few things to consider
If the queuing backend lacks support for bulk enqueuing,
perform_all_latergracefully reverts to sequential job enqueuing.
perform_all_latermethod refrains from executing any callbacks (e.g., before_enqueue, before_perform, after_perform), aligning with ActiveRecord’s bulk import APIs.
At present, no explicit restriction on the quantity of jobs enqueued at once via
perform_all_later. Internally, Sidekiq’s
push_bulkmethod adeptly manages this scenario.
perform_laterthat returns an instance of the queued job class,
perform_all_laterreturns nil upon execution.
perform_all_latermethod introduces a separate event,
enqueue_all.active_job, and does not utilize the existing
For queue adapters that support the new
enqueue_allmethod, such as the Sidekiq adapter, the enqueuing process is further optimized using
perform_all_later brings efficiency
and flexibility to job enqueuing in Rails 7.1. This addition streamlines handling diverse job arguments, supports bulk enqueuing,
and offers enhanced control over job execution.
However, it’s essential to note the current limitations in handling return values, lack of callbacks, and considerations regarding batch sizes. These aspects provide insights into potential areas for improvement and future enhancements within the ActiveJob framework.
perform_all_later offers a promising solution to accelerate job enqueuing processes, paving the way for enhanced performance
and scalability in Rails applications.