POST /v1/extract
Web Extract API
Up to 20 web pages or PDFs in, clean Markdown out, fetched in parallel. Add a question and get the passages that answer it instead of whole pages.
Overview
What the Web Extract API does
The Web Extract API reads many pages in one call: the same fetch and Markdown conversion as the Webpage API, PDFs included, for up to 20 URLs at once, fetched in parallel. Each page comes back with its final URL, status, title, published date and metadata, as markdown, text or raw html.
With query and highlights, each page also gets the passages most relevant to your question, ranked with BM25 (deterministic, no model involved). max_tokens caps each page instead, so a batch of pages fits a context window.
The response is 200 even when some URLs fail: each failure is listed in failed with a reason code and costs nothing.
Need data, not text? output_schema fills your JSON Schema from each page (+4 credits per page) and summary: true adds a short summary (+4), both written by Claude. See the AI Data Extraction API.
What you get
-
requestobject - urls (deduplicated), format, query.
-
results[]array - One entry per page read: url (final, after redirects), status_code, title, published_at, metadata, cached, markdown / text / html, and links, images, highlights (text, score, heading) when asked for.
-
results[].summary / summary_errorstring | null - With summary: true: the summary, or null and the reason it failed (not charged).
-
results[].output / output_errorany - With output_schema: the JSON value matching your schema, or null and the reason it failed (not charged).
-
failed[]array - URLs that could not be read: url, error (code, message). Not charged.
-
metaobject - request_id, credits_used, latency_ms, succeeded, failed, and llm (provider, model, tokens) when an AI step ran.
Request
curl https://api.serpkite.com/v1/extract \
-H "Authorization: Bearer $SERPKITE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"urls":["https://en.wikipedia.org/wiki/Web_crawler","https://www.rfc-editor.org/rfc/rfc9309.html"],"query":"robots.txt crawl delay","highlights":3}'Response (illustrative, placeholder domains)
{
"request": {
"endpoint": "extract",
"urls": [
"https://en.wikipedia.org/wiki/Web_crawler",
"https://www.rfc-editor.org/rfc/rfc9309.html"
],
"format": "markdown",
"query": "robots.txt crawl delay"
},
"results": [
{
"url": "https://www.rfc-editor.org/rfc/rfc9309.html",
"status_code": 200,
"title": "RFC 9309: Robots Exclusion Protocol",
"metadata": {
"title": "RFC 9309: Robots Exclusion Protocol",
"content_type": "text/html"
},
"cached": false,
"markdown": "# Robots Exclusion Protocol\n\n…",
"highlights": [
{
"text": "Crawlers SHOULD NOT use the cached version for more than 24 hours, unless the robots.txt is unreachable…",
"score": 6.214,
"heading": "Robots Exclusion Protocol > Caching"
}
]
}
],
"failed": [
{
"url": "https://en.wikipedia.org/wiki/Web_crawler",
"error": {
"code": "upstream_timeout",
"message": "the page did not load in time"
}
}
],
"meta": {
"request_id": "req_01J8ZK4M6Q2V7",
"credits_used": 1,
"latency_ms": 2210,
"succeeded": 1,
"failed": 1
}
}
Every response also carries X-Credits-Used, X-Credits-Remaining,
X-Cache and X-Tokens-Estimate headers.
Full endpoint reference
Reference
Parameters
The fields /v1/extract accepts in its JSON body.
| Name | Type | Default | Description |
|---|---|---|---|
urls
required
|
string[] | – | 1–20 public http(s) URLs of HTML pages or PDFs. |
format
|
string | markdown | What each result carries: markdown, text or the raw html. Values: markdown, text, html. |
query
|
string | – | What highlights are ranked against (up to 4,096 characters). |
highlights
|
integer | 0 | Passages per page ranked against query, 0–10 (about 600 characters each). |
max_tokens
|
integer | – | Trim each page's Markdown or text to about this many tokens (100–100,000). |
include_links
|
boolean | false | Webpage: also return the page's outbound links (url, text). |
include_images
|
boolean | false | Webpage: also return the page's image URLs. |
max_age
|
integer | – | Accept a cached page up to this many seconds old, for 0.5 credit. |
country
|
string | – | Fetch the pages as seen from this country (two-letter ISO code). |
timeout
|
integer | – | Seconds to wait for the pages, 1–90 (default 50, or 75 with summary or output_schema). A page still loading is listed in failed (upstream_timeout) and not charged. |
summary
|
boolean | false | Add a 3–6 sentence AI summary of each page, focused on query when set. +4 credits per page; a failed summary is free. |
output_schema
|
object | – | A JSON Schema (at most 16 KB). Each page's data comes back as JSON matching it in output. +4 credits per page; a failed extraction is free. |
prompt
|
string | – | Guides the output_schema extraction: what to pick and how to normalise it (up to 4,096 characters). Needs output_schema. |
Full reference, error codes and headers are in the /v1/extract reference.
Use cases
What people build with the Web Extract API
RAG over chosen sources
Read the URLs your retriever picked, as Markdown, in one round trip.
Answer-focused context
highlights returns only the passages that match the question, keeping prompts small.
PDF reading
Papers, filings and manuals in PDF come back as text alongside HTML pages.
Pricing
Web Extract API pricing
Cost per call
1 credit
per URL that comes back (0.5 from cache); failed URLs are free
- From $0.60 per 1,000 URLs at volume.
- Credits never expire. No subscription.
- Failed, empty and blocked calls are refunded.
- 2,500 free credits = 2,500 URLs, then 1,000 credits a month.
| Pack | Price | Per 1k credits | Per 1k URLs | URLs per pack |
|---|---|---|---|---|
| Starter | $10 | $1.00 | $1.00 | 10,000 |
| Growth | $50 | $0.80 | $0.80 | 62,500 |
| Pro | $300 | $0.60 | $0.60 | 500,000 |
FAQ
Web Extract API: common questions
Keep exploring
Related APIs and integrations
Start building
Start with the Web Extract API
2,500 free credits, then 1,000 every month. No credit card.