Moving Slow Work Into Background Jobs

A user uploads a document. Your request extracts text, calls an AI model, generates a report, sends an email, and updates three tables before returning. It works in development. In production the platform kills it after thirty seconds, the user retries, and now two expensive reports are running.

Slow work does not belong inside a web request. A background job lets the request validate intent, record durable work, and respond immediately while a worker performs the expensive steps with retries and visibility.

Know what should leave the request path

Move work when it may exceed the hosting timeout, depends on a flaky provider, benefits from retries, consumes substantial CPU or memory, or does not need to finish before the user continues. AI generation, document processing, imports, video work, bulk email, webhook fan-out, and large exports are common candidates.

Keep authentication, authorization, input validation, and the durable creation of the job in the request. The user should receive a job ID only after the system has committed to doing the work. This pattern also addresses many serverless function timeout failures.

Make job creation durable and idempotent

The dangerous gap is writing application state and publishing a queue message as separate operations. If one succeeds and the other fails, the job is lost or duplicated. Use a provider with transactional enqueue support, or write an outbox record in the same database transaction as the business change and have a dispatcher publish it.

Give each logical operation an idempotency key. A retry with the same account, input, and operation should return the existing job rather than create another. Inside the worker, make each external side effect idempotent too. Store the provider's response ID before advancing, and use provider idempotency headers when available.

Exactly-once execution is usually a marketing phrase. Aim for at-least-once delivery with idempotent effects.

Model the job as states, not one giant function

Store queued, running, succeeded, failed, and cancelled states plus attempt count, timestamps, current phase, and a safe error summary. For multi-step work, checkpoint after extraction, model generation, persistence, and notification. A crash after persistence should not call the model again merely because the email was not sent.

The worker should claim a job with a lease. If it dies, another worker may retry after the lease expires. Use heartbeats for legitimately long work and cap total execution time. This makes stuck jobs observable instead of permanently marked running.

Expose progress honestly. "Processing page 12 of 40" is useful when known; "working" is better than a fake percentage. Let the frontend poll a status endpoint or subscribe to updates, and ensure that endpoint authorizes access to the owning account.

Retry only failures that may recover

Network timeouts, provider rate limits, and temporary 5xx responses deserve bounded retries with exponential backoff and jitter. Invalid input, missing authorization, and unsupported file formats do not. Classify errors explicitly so permanent failures reach the user quickly.

Limit attempts and move exhausted jobs to a failed or dead-letter state. Alert on the rate of failures, not every single retry. Store enough context to investigate without storing secrets or entire sensitive documents in logs. The guide to error handling in AI-generated apps covers the user-facing side of the same decision.

Respect provider retry hints and concurrency limits. Fifty workers retrying at the same instant can turn a short outage into a longer one. Per-account limits also stop one large import from delaying every other customer.

Design cancellation, reprocessing, and cleanup

Cancellation is cooperative. Mark the job cancelled and check that state between expensive phases. You cannot always interrupt an in-flight provider request, but you can prevent the next step and ignore a late result safely.

Build an operator action to retry a failed job after its cause is fixed. Reprocessing should create an auditable attempt without losing the original failure. Keep job records long enough to support customers and analyze reliability, then delete payloads according to your retention rules.

When jobs create files, reserve credits, or stage partial records, define cleanup for failure and cancellation. This matters especially when an AI app is burning API credits: a queue can control concurrency, but it cannot compensate for unbounded retries you configured yourself.

Observe the queue as a product dependency

Track queue depth, oldest queued age, run duration by job type, success rate, retry count, dead-letter count, and worker availability. Queue depth alone can look healthy while one old job is stuck behind fresh fast work.

Correlate the initial request, job ID, worker attempt, and provider call. A support ticket should lead to one timeline. Add alerts for rising oldest age, no active workers, and failure-rate changes. The broader observability guide shows how to connect these signals to releases and user impact.

Start with a managed queue or the job system built into your platform. Operate Redis, brokers, and worker autoscaling yourself only when the workload or compliance need justifies it. The hard part is the job semantics, not selecting the most impressive queue logo.

Frequently asked questions

Do I need a queue for every email? Not necessarily, but transactional email should not delay the response or fail the main operation. A provider call after the durable transaction or a small managed queue is usually enough.

How does the frontend learn when a job finishes? Start with polling a protected status endpoint. Add server-sent events or WebSockets when latency or volume makes polling wasteful. The stored job state remains the source of truth either way.

Can a serverless platform run background jobs? Yes, through its queue, workflow, or scheduled-function integrations. Do not start work after returning from an ordinary request and assume the process will remain alive; use a durable trigger the platform supports.

---

If slow AI work still runs inside page requests, timeouts and duplicate charges are already part of the architecture. SprintX designs durable job pipelines with safe retries, progress, and operational visibility. Send us the failing workflow.