I’m using a batch job in Drupal 8 to scrape pages from another site to import.

I’m using httpClient, and I have it wrapped in a try/catch.

try {   $response = Drupal::httpClient()->get($parts['scheme'] . '://' . $parts['host'] . $parts['path'], [     'query' => $parts['query'],     'headers' => array('Accept'=>'text/xml'),     'verify' => FALSE,     'connect_timeout' => 6,     'CURLOPT_SSL_VERIFYPEER' => false,     'CURLOPT_CERTINFO' => false,   ]);   return (string) $response->getBody(); } catch (Exception $e) {   watchdog_exception('scrape', $e);   return FALSE; } 

However, from time to time the remote site times out, and the guzzle exception is not caught, which causes the batch job to end.

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=44&op=do_nojs&op=do StatusText: Internal Server Error ResponseText: The website encountered an unexpected error. Please try again later.GuzzleHttpExceptionConnectException: cURL error 28: Operation timed out after 30001 milliseconds with 0 bytes received (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) in GuzzleHttpHandlerCurlFactory::createRejection() (line 186 of vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php). GuzzleHttpHandlerCurlFactory::finishError(Object, Object, Object) (Line: 103) 

This job imports a few thousand pages, so when the error happens in the middle of the batch, it’s a major pain.

I am getting ready to write a queue for the batch processor, where the batch pops a set of items off the queue and then marks them as done, so that I can continue the job after an exception.

But then I wondered, is there already a way to tell Drupal to continue a batch from where it left off or was interrupted? I did some googling and didn’t find anything indicating that this feature already existed. I don’t want to re-invent the wheel if I don’t have to.

Can I re-start an incomplete, interrupted batch job?