When a cron job is the wrong tool: recognizing the job that needs a queue

Cron is seductive until it isn’t. Here’s how to spot the moment a simple schedule becomes a reliability trap, and what to build instead.

The cron comfort zone

Cron excels at three things: reports that run at midnight, cleanup tasks that tolerate delay, and idempotent sweeps where running twice is harmless. If your job fits this profile—fixed time, no external trigger, failure means ‘try again tomorrow’—cron is correct. Do not over-engineer.

The cost of cron is operational opacity. When a job fails, you learn from logs you may not check. When a job overlaps with its previous run, you rely on file locks or database flags you may have forgotten to set. For a two-engineer team, this blindness compounds quickly.

The queue signal: external triggers

The shift begins when work is caused by something outside your system. A user uploads a file. A partner webhook fires. A payment succeeds and triggers provisioning. These events demand response, not schedule. Polling with cron turns real-time needs into batch delays and wastes compute checking for work that usually isn’t there.

A queue inverts this. Work arrives, gets enqueued, workers consume. The architecture matches the semantics: event causes work, work gets done, result is confirmed. This is not about scale. A single worker process handling a Redis list is a queue, and it is the right structure from customer one if the trigger is external.

Exactly-once and at-least-once

Cron jobs that must run exactly once are dangerous. The failure modes are subtle: a deploy restarts the scheduler, a container drifts out of sync, a lock expires during a long query. You end up building ad-hoc distributed systems inside what looked like a simple script.

Queues make delivery guarantees explicit. At-least-once is easy. Exactly-once requires idempotent workers—design the job so running twice is safe, rather than fighting to prevent it. This is the default recommendation: prefer at-least-once with idempotency, never chase exactly-once through fragile locking. The deviation condition is financial or safety-critical workflows where external systems enforce strict deduplication; there, use a transactional outbox pattern with your database.

Visibility as a requirement

A cron job that fails writes to stderr and exits non-zero. Someone has to notice. In practice, for early teams, nobody notices until a customer reports missing data. This is unacceptable for revenue-impacting work.

Queues integrate with observability by design. Dead letter queues surface failed jobs. Retry counts are inspectable. A stuck job is a metric you alert on. The operational tax of a queue—worker processes, connection handling, backoff logic—is lower than the tax of undetected cron failures. Build the minimal queue: one Redis instance, one worker pool, one retry policy with exponential backoff. Add a dead letter queue in week two, not month six.

The migration path

You do not need to replace every cron job today. Start with user-facing work: anything a customer expects to happen after they click a button. Move webhook handlers first, then file processing, then report generation that users trigger.

Keep cron for true periodic tasks: nightly aggregation, log rotation, certificate renewal checks. The test is simple: if a human cares when it completes, it belongs in a queue. If the system cares on a schedule, cron remains valid. This boundary keeps your architecture honest and your sleep schedule intact.