Sidekiq Background Jobs: How to Run Async Work in Ruby

Sidekiq background jobs let you move time-consuming work out of the request-response cycle so your users never wait on slow tasks. Instead of handling email delivery, image processing, or external API calls during the web request, you push those jobs to a queue and let Sidekiq workers process them asynchronously in the background. This guide covers everything you need to know to integrate Sidekiq into your Ruby applications.

Why background jobs matter

Web applications often need to perform tasks that take seconds or even minutes to complete. If you handle these synchronously, users experience slow response times and your server threads get blocked. Background jobs solve this by moving these tasks to a separate process that runs independently of the web request.

Sidekiq uses Redis as a message broker to store job data. It processes jobs using worker threads instead of forked processes, making it incredibly efficient. A single Sidekiq process can handle hundreds of jobs per second with minimal memory overhead. For a deeper look at how Ruby threads behave under the GIL, see our Ruby concurrency guide.

Installing Sidekiq

Add Sidekiq to your Gemfile:

gem 'sidekiq'

The Gemfile entry tells Bundler which gem to fetch and which version constraints to apply. When you declare a gem here, you are telling your app that it depends on that library, but nothing is downloaded until you run the install command. Bundler resolves the full dependency tree, pulls in sidekiq along with its own dependencies like redis-client and connection_pool, and writes the resolved versions to Gemfile.lock so every environment gets the same gem versions.

bundle install

Sidekiq uses Redis as its message broker, storing every job payload as a serialized entry in a Redis list and a corresponding entry in a Redis sorted set for scheduled jobs. The Sidekiq server process polls those lists and distributes work across its thread pool using the BRPOP command for efficient blocking reads. You can run Redis locally for development, or point to a managed instance like Redis Cloud or AWS ElastiCache in production. Make sure Redis is running and reachable before you start the Sidekiq process, because the process will fail to boot without a Redis connection.

redis-server

Creating your first worker

Workers are Ruby classes that define the work to be done in the background. Each worker has a perform method that Sidekiq calls when processing the job. A worker class encapsulates one type of background work and gets instantiated fresh for every job execution, so instance variables do not leak between jobs. When you design a worker, think about it as a single-purpose unit of work that receives its inputs through the perform method arguments and produces side effects such as sending an email, updating a database record, or calling an external service.

class HardWorker
  include Sidekiq::Worker

  def perform(id, name)
    # Do something expensive
    puts "Processing #{name} with ID #{id}"
    sleep(5)  # Simulate expensive operation
  end
end

The include Sidekiq::Worker module adds the perform_async class method to your worker. When you call perform_async, Sidekiq serializes the arguments to JSON and pushes them onto a Redis queue. The arguments must be simple types: strings, numbers, booleans, arrays, or hashes of those primitives, because Sidekiq uses JSON under the hood. Complex Ruby objects like ActiveRecord models serialize incorrectly, so always pass record IDs and look them up inside the perform method instead.

# Enqueue a job to be processed asynchronously
HardWorker.perform_async(42, "Alice")

# You can also pass additional arguments
HardWorker.perform_async(42, "Alice", { priority: "high" })

Job execution options

Sidekiq provides several options to control how jobs execute. You can schedule jobs to run in the future or repeat them at intervals. Scheduling gives you precise control over when work happens. For example, you can delay retrying a rate-limited API call, batch notifications to go out at a specific hour, or spread expensive report generation across off-peak times when your server has spare capacity.

# Execute in 10 minutes from now
HardWorker.perform_in(600, 42, "Bob")

# Execute at a specific time
HardWorker.perform_at(1.hour.from_now, 42, "Charlie")

# Repeat every hour, starting now
HardWorker.perform_async(42, "Diana")
# Then use sidekiq-scheduler or sidekiq-cron for recurring jobs

Configuring Sidekiq

Sidekiq configuration goes in an initializer. The server and client configurations serve different purposes: the server block controls the process that picks up and executes jobs, while the client block governs how your application pushes jobs onto the queue. Splitting them lets you tune each side independently. For instance, you might give the server a larger Redis connection pool to match its concurrency setting while keeping the client pool smaller since web requests enqueue jobs one at a time.

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.redis = {
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1'),
    size: ENV.fetch('SIDEKIQ_CONCURRENCY', 25).to_i
  }
end

Sidekiq.configure_client do |config|
  config.redis = {
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1')
  }
end

The concurrency setting controls how many worker threads run simultaneously. Each thread can process one job at a time. Start with 25 threads and adjust based on your workload and server resources.

Queue management

Sidekiq supports multiple queues with different priorities. By default, jobs go to the “default” queue, but you can specify different queues. Grouping jobs into named queues lets you route urgent work to high-priority lanes while keeping slow or background maintenance work in lower-priority queues where it will not block time-sensitive tasks. This separation becomes valuable as your application grows and different teams start adding their own job types.

class EmailWorker
  include Sidekiq::Worker
  sidekiq_options queue: 'mailers'

  def perform(user_id, subject, body)
    # Send email
  end
end

Configure queue priorities in your Sidekiq configuration. The weighted priority system means that in each polling cycle, Sidekiq samples jobs from higher-weight queues more often, which gives critical jobs a statistical advantage without completely starving lower-priority queues. This approach works well in practice because it is simple to reason about and does not require complex scheduling logic at the application level.

# config/sidekiq.yml
:concurrency: 25
:queues:
  - [critical, 3]
  - [default, 2]
  - [mailers, 1]
  - [low, 1]

The numbers indicate priority weight: higher numbers mean higher priority. Jobs in the “critical” queue get processed before “default”, which gets processed before “low”.

Error handling and retries

Sidekiq automatically retries failed jobs with exponential backoff. The retry mechanism is a first-class feature of Sidekiq: by default, a failed job is re-enqueued with an increasing delay between attempts, giving transient failures, such as a brief database outage or a rate-limited API, time to resolve before the next try. You can control both the number of retries and the delay formula per worker class.

class SensitiveWorker
  include Sidekiq::Worker
  sidekiq_options retry: 5  # Maximum 5 retries
  sidekiq_retry_in do |count|
    10 * (count + 1)  # 10s, 20s, 30s, 40s, 50s
  end

  def perform(data)
    # Work that might fail
  end
end

For jobs that should never retry, use sidekiq_options retry: false. This setting is appropriate when a job is not idempotent and re-execution would cause duplicate side effects, or when the failure is deterministic and retrying would waste resources. Common examples include jobs that send push notifications, where a second delivery would annoy the user, or jobs that process one-time tokens that become invalid after the first attempt.

class NoRetryWorker
  include Sidekiq::Worker
  sidekiq_options retry: false

  def perform(id)
    # Don't retry on failure
  end
end

You can also move failed jobs to a dead queue after all retries are exhausted using the dead queue feature.

Testing workers

Testing Sidekiq workers requires a different approach than regular unit tests. The sidekiq/testing helper lets you process jobs synchronously inside your test suite so that assertions run after the job logic completes. This pattern keeps tests deterministic and fast because no real Redis connection or background thread is involved; the job runs inline, and the test can inspect the side effects immediately.

require 'sidekiq/testing'

describe HardWorker do
  it 'performs the job' do
    HardWorker.perform_async(42, "test")
    
    # Process jobs synchronously
    HardWorker.drain
    
    # Assert the job ran
    expect(HardWorker.jobs.size).to eq(0)
  end
end

For integration tests that actually run jobs asynchronously, use Sidekiq::Testing.inline!. This mode executes every enqueued job immediately within the same thread, which is useful for end-to-end tests where you want the full job lifecycle to play out without spinning up a separate Sidekiq process. Just remember to reset the testing mode between test cases so that one test’s configuration does not leak into the next.

Sidekiq::Testing.inline!

Starting Sidekiq

Sidekiq runs as a separate process alongside your web server, so you need to start it explicitly. The command below boots the Sidekiq server, loads your application code, and begins polling Redis for jobs. You can run this in a terminal during development, but in production you will want a process manager to keep it alive across restarts and crashes.

bundle exec sidekiq -C config/sidekiq.yml

In production, use a process manager like systemd or Supervisor to keep Sidekiq running. The -d flag daemonizes the process so it runs in the background, and -P writes the process ID to a file that your process manager can use to monitor and restart Sidekiq if it stops unexpectedly. Without a process manager, a crash takes your background processing offline until someone notices and manually restarts it.

bundle exec sidekiq -d -P /tmp/sidekiq.pid -C config/sidekiq.yml

When to use Sidekiq

Sidekiq excels when you need to process many jobs quickly, handle long-running tasks without blocking web requests, or scale background processing across multiple workers. It’s particularly well-suited for sending emails, processing file uploads, calling third-party APIs, generating reports, and scheduled maintenance tasks.

When not to use Sidekiq

Avoid Sidekiq for very simple tasks that complete in milliseconds. You’ll add unnecessary complexity. If you only need to run jobs occasionally, a simpler solution might suffice. Also, Sidekiq requires Redis, so you need to budget for that infrastructure overhead.

Key takeaways

• Sidekiq moves expensive work out of the request cycle so your app stays responsive.
• Redis stores the queue data, so make sure your deployment includes a reliable Redis service.
• Keep worker classes focused on one job type and one perform method.
• Use queue names and retries intentionally so important jobs are not lost in the noise.
• Test workers with Sidekiq’s testing helpers before you send them to production.

Production checklist

Before deploying Sidekiq, make sure you have a few basics in place. Use environment variables for the Redis URL, keep the queue names documented, and decide which jobs should retry automatically. It also helps to separate long-running jobs from fast ones so a slow task does not delay critical work.

If your app has user-facing actions that trigger jobs, log enough detail to trace the job later. A job that fails silently in production is much harder to debug than one that records the job ID, queue name, and arguments.

It is also worth checking that your Redis instance has persistence and monitoring. Sidekiq itself is lightweight, but the queue is only as reliable as the Redis service behind it.

Common mistakes

One common mistake is putting too much logic inside perform. A worker should do the work, but it should not contain every business rule in the system. If the job starts to grow, move the core logic into a service object and let the worker call it.

Another mistake is enqueuing jobs with large payloads. Sidekiq serializes job arguments, so keep them small and pass IDs instead of whole objects. That keeps the queue lean and avoids serialization headaches.

It is also easy to forget that retries are part of the design. A failed job might run several more times, so make sure your code is idempotent or otherwise safe to execute again.

Frequently asked questions

Should every background task use Sidekiq?

No. Use it when you need durable jobs, retries, or background processing that can survive process restarts. For a one-off script or a task that runs only once a day, a simpler approach might be enough.

How do I keep jobs from duplicating work?

Make the job idempotent whenever possible. That usually means checking whether the work has already been done before you do it again. If the job updates a record, use a stable identifier and a clear state check.

When should I split queues?

Split queues when some jobs are more urgent than others, or when long-running jobs start to delay short ones. Separate queues make the worker behavior easier to reason about and easier to tune.

Conclusion

Sidekiq is a practical default for Ruby apps that need reliable background work. It keeps request threads free, gives you retries and queue control, and fits naturally into Rails and plain Ruby applications alike.

Start with a small worker, keep your arguments simple, and pay attention to Redis health and job idempotency. Those habits make Sidekiq easier to operate and much easier to debug when something goes wrong.

See also

• ActiveJob and Sidekiq: background jobs in Rails covers how ActiveJob wraps Sidekiq in a Rails-native job interface with queue adapters and callbacks.
• Ruby Concurrency: Threads, Fibers, and the GIL explains the threading model Sidekiq relies on for parallel job execution.
• Ruby Async Gem: Fiber-Based Concurrency for I/O explores an alternative concurrency model using fibers for I/O-bound workloads.