Skip to content

Read

Batch read

Read many pages in one call, politely.

batch reads a list of URLs and returns one result per URL, in the order you gave them.

  • Runs up to four pages at once, one at a time per site.
  • The first URL for each site goes first. The rest of that site wait for it, then start from the tool it found, instead of each climbing the ladder.
  • Every page gets its own receipt; one failure never fails the batch.
  • Signed-in batches run one page at a time.
pages = await web.batch([
"https://example.com/a",
"https://example.com/b",
"https://another.example/c",
])

Set it when you open the runtime:

Runtime("state", concurrency=4, per_domain=1, domain_delay=0.25)

Reads to one site are also paced across processes. See Pacing.