The Drupal Batch API: Progress Bars for Long-Running Tasksfor Drupal 11 , and 10

In the last lesson you learned to defer work with a queue — drop items on a list, let something else process them later. But some jobs can't wait for "later": a site administrator clicks "Import 10,000 products" and expects to watch it happen, right now, with a progress bar telling them it's working and roughly how long is left. PHP scripts have a hard time limit though — usually 30 to 60 seconds — so how do you process 10,000 of anything in a single request without timing out? This lesson covers Drupal's answer: the Batch API.

What you'll learn in this lesson

• Why a single HTTP request can't safely process a huge amount of work, and how the Batch API splits it up
• The difference between a simple, single-pass operation and a progressive, multi-pass operation
• How the $context array lets your code talk to the batch engine and carry state between requests
• How to report a final summary once every operation has finished

The problem the Batch API solves

If you tried to loop over 10,000 database rows in one PHP request, one of two things would happen: the web server's execution time limit would kill the script partway through, or, if you raised the limit, your visitor would stare at a blank loading screen for minutes with zero feedback. Neither is acceptable.

The Batch API's trick is to break the work into small chunks called operations, and process only one (or a few) of them per HTTP request. After each chunk, Drupal automatically sends the browser a tiny bit of JavaScript that says "now request the next chunk," updates a progress bar, and repeats — all the way until every operation reports itself done. From the visitor's point of view, it looks like one continuous progress bar; behind the scenes, it's actually dozens or thousands of separate quick requests.

The source file

Path (relative to the Examples module's root): modules/batch_example/batch_example.module

<?php

/**
 * @file
 * Outlines how a module can use the Batch API.
 */

/**
 * @defgroup batch_example Example: Batch API
 * @ingroup examples
 * @{
 * Outlines how a module can use the Batch API.
 *
 * Batches allow heavy processing to be spread out over several page
 * requests, ensuring that the processing does not get interrupted
 * because of a PHP timeout, while allowing the user to receive feedback
 * on the progress of the ongoing operations. It also can reduce out of memory
 * situations.
 *
 * The @link batch_example.install .install file @endlink also shows how the
 * Batch API can be used to handle long-running hook_update_N() functions.
 *
 * Two harmless batches are defined:
 * - batch 1: Load the node with the lowest nid 100 times.
 * - batch 2: Load all nodes, 20 times and uses a progressive op, loading nodes
 *   by groups of 5.
 *
 * @see batch
 */

/**
 * Batch operation for batch 1: one at a time.
 *
 * This is the function that is called on each operation in batch 1.
 */
function batch_example_op_1($id, $operation_details, &$context) {
  // Simulate long process by waiting 1/50th of a second.
  usleep(20000);

  // Store some results for post-processing in the 'finished' callback.
  // The contents of 'results' will be available as $results in the
  // 'finished' function (in this example, batch_example_finished()).
  $context['results'][] = $id;

  // Optional message displayed under the progressbar.
  $context['message'] = t('Running Batch "@id" @details',
    ['@id' => $id, '@details' => $operation_details]
  );
}

/**
 * Batch operation for batch 2: five at a time.
 *
 * This is the function that is called on each operation in batch 2.
 *
 * After each group of 5 control is returned to the batch API for later
 * continuation.
 */
function batch_example_op_2($operation_details, &$context) {
  // Use the $context['sandbox'] at your convenience to store the
  // information needed to track progression between successive calls.
  if (empty($context['sandbox'])) {
    $context['sandbox'] = [];
    $context['sandbox']['progress'] = 0;
    $context['sandbox']['current_node'] = 0;

    // Save node count for the termination message.
    $context['sandbox']['max'] = 30;
  }

  // Process in groups of 5 (arbitrary value).
  // When a group of five is processed, the batch update engine determines
  // whether it should continue processing in the same request or provide
  // progress feedback to the user and wait for the next request.
  // That way even though we're already processing at the operation level
  // the operation itself is interruptible.
  $limit = 5;

  // Retrieve the next group.
  $result = range($context['sandbox']['current_node'] + 1, $context['sandbox']['current_node'] + 1 + $limit);

  foreach ($result as $row) {
    // Here we actually perform our dummy 'processing' on the current node.
    usleep(20000);

    // Store some results for post-processing in the 'finished' callback.
    // The contents of 'results' will be available as $results in the
    // 'finished' function (in this example, batch_example_finished()).
    $context['results'][] = $row . ' ' . $operation_details;

    // Update our progress information.
    $context['sandbox']['progress']++;
    $context['sandbox']['current_node'] = $row;
    $context['message'] = t('Running Batch "@id" @details',
      ['@id' => $row, '@details' => $operation_details]
    );
  }

  // Inform the batch engine that we are not finished,
  // and provide an estimation of the completion level we reached.
  if ($context['sandbox']['progress'] != $context['sandbox']['max']) {
    $context['finished'] = ($context['sandbox']['progress'] >= $context['sandbox']['max']);
  }
}

/**
 * Batch 'finished' callback used by both batch 1 and batch 2.
 */
function batch_example_finished($success, $results, $operations) {
  $messenger = \Drupal::messenger();
  if ($success) {
    // Here we could do something meaningful with the results.
    // We just display the number of nodes we processed...
    $messenger->addMessage(t('@count results processed.', ['@count' => count($results)]));
    $messenger->addMessage(t('The final result was "%final"', ['%final' => end($results)]));
  }
  else {
    // An error occurred.
    // $operations contains the operations that remained unprocessed.
    $error_operation = reset($operations);
    $messenger->addMessage(
      t('An error occurred while processing @operation with arguments : @args',
        [
          '@operation' => $error_operation[0],
          '@args' => print_r($error_operation[0], TRUE),
        ]
      )
    );
  }
}

/**
 * @} End of "defgroup batch_example".
 */

How it works

The $context array — how your code talks to the batch engine

Every batch operation callback receives one special argument: &$context, passed by reference. This array is the entire communication channel between your code and the batch engine, and it has four keys worth knowing:

• $context['results'] — an accumulator that survives across every single operation call. Anything you push onto it is still there when the "finished" callback runs at the very end.
• $context['message'] — a string shown to the user underneath the progress bar. Update it on each pass to keep the feedback meaningful rather than a frozen percentage.
• $context['sandbox'] — a private scratch space that only exists for progressive (multi-pass) operations. Drupal preserves whatever you put in here between calls, so it's exactly where you'd track "how far did I get last time."
• $context['finished'] — a number from 0 to 1 (or a boolean) telling the batch engine how done the current operation is. Set it to 1 or leave it alone and the engine moves on to the next operation; set it below 1 and the exact same callback runs again on the next request.

batch_example_op_1() — a simple, single-pass operation

function batch_example_op_1($id, $operation_details, &$context) {
  usleep(20000);
  $context['results'][] = $id;
  $context['message'] = t('Running Batch "@id" @details', [...]);
}

This is the simplest possible operation: it does its (simulated) work, records a result, sets a status message, and returns. Because it never touches $context['finished'], Drupal assumes it's complete after exactly one call and immediately moves to the next item — of which there are 1,000 in this batch, each its own separate operation in the array.

batch_example_op_2() — a progressive, multi-pass operation

if (empty($context['sandbox'])) {
    $context['sandbox']['progress'] = 0;
    $context['sandbox']['current_node'] = 0;
    $context['sandbox']['max'] = 30;
}
// ... process a chunk of 5 ...
$context['finished'] = ($context['sandbox']['progress'] >= $context['sandbox']['max']);

This is where things get interesting. On the very first call, $context['sandbox'] is empty, so the code initializes its own little tracking state — progress, current_node, and a max of 30. It then processes a chunk of 5 items and updates progress. Critically, it sets $context['finished'] to a boolean based on whether progress has reached max yet. If it hasn't, the batch engine calls this exact same function again on the next HTTP request — and because $context['sandbox'] is restored automatically, the code picks up exactly where it left off, as if no interruption ever happened.

This is the pattern to reach for whenever a single logical unit of work is itself too big for one operation call — it turns "one operation" into "one operation that might quietly take several HTTP requests to finish," entirely hidden from the person watching the progress bar.

The finished callback

function batch_example_finished($success, $results, $operations) {
  $messenger = \Drupal::messenger();
  if ($success) {
    $messenger->addMessage(t('@count results processed.', ['@count' => count($results)]));
  }
  ...
}

Once every operation in the batch reports itself finished, Drupal calls this one function exactly once. $success tells you whether everything completed cleanly; $results is the fully merged $context['results'] array from every single operation that ran; and $operations — only populated on failure — lists whatever didn't get to run, which is invaluable for error diagnostics. This is your one chance to summarize the whole job for the user.

Batch 1 vs. batch 2, side by side

See it for yourself

Visit /examples/batch_example on your DDEV site and click "Go" to start batch 1 (1,000 single-pass operations).

That's the progress page caught mid-run — "Completed 317 of 1000," 31.7% along, with the live status message "Running Batch '317' (Operation 316)" underneath. Every one of those 1000 operations is a separate call to batch_example_op_1(), each contributing its $id to $context['results']. Once it reaches 100%, the "finished" callback fires and reports how many results were processed and what the final one was.

Let it run all the way through and this is what you'll land on: the message batch_example_finished() built from count($results) — proof that every one of the 1,000 individual operations pushed its own entry onto $context['results'], and that the finished callback really does run exactly once, after the very last operation, not once per operation.

Quick check: batch 2 processes 30 items in groups of 5, giving 6 real "rounds" of work — but how many operations does its progress bar count toward its total? If you said 20 (the number of entries in the operations array, not the number of items or chunks), you've understood the distinction the Batch API makes.

Key takeaways

• batch_set() takes a definition array with 'operations', 'finished', 'title', 'init_message', 'progress_message', and 'error_message' keys — call it inside a form's submitForm() and Drupal handles the rest.
• Every operation callback receives &$context by reference: use results to accumulate data, message for live status text, and sandbox to persist state across multiple HTTP requests.
• A simple operation finishes in one call (never touches $context['finished']); a progressive operation sets $context['finished'] to a value below 1 to ask the batch engine to call it again.
• The finished callback receives the fully merged $results array — it's the right place for summary messages and any cleanup.
• The same sandbox pattern (as $sandbox['#finished']) applies directly to hook_update_N() functions in .install files, for safe, interruptible database updates.

Coming up next

Batches are triggered by a person clicking a button. But plenty of background work needs to happen on a schedule, with nobody watching at all. Next up: hook_cron() — Drupal's built-in mechanism for recurring background tasks.