Skip to content

New AI assistant answers: ChatGPT, Gemini, Copilot and AI Mode by API

SerpKite
Get API key
Docs menu / Extract

Extract

Read up to 20 URLs (HTML or PDF) per call as Markdown, with ranked highlights, an AI summary or JSON for your schema. Failed URLs are listed and cost nothing.

POST https://api.serpkite.com/v1/extract Credits: 1 per URL that comes back (0.5 from cache); failed URLs are free
View as Markdown

Extract reads many pages in one call: the same fetch and Markdown conversion as /v1/webpage, PDFs included, for up to 20 URLs at once, fetched in parallel. Duplicate URLs are dropped.

With query and highlights, each page also gets its passages most relevant to the query, ranked with BM25 (a lexical ranker: deterministic, no model involved). Use max_tokens to cap each page instead.

Two optional AI steps run per page: summary: true adds a 3–6 sentence summary (focused on query when set, +4 credits per page), and output_schema returns the page's data as JSON that matches your JSON Schema in output (+4 credits per page), guided by an optional prompt. Claude Haiku 5.5 writes both; a schema whose fields are all booleans or enums is answered by OpenAI's Decisions API instead, which always returns schema-valid values. A failed AI step is reported in summary_error / output_error and not charged. See Structured extraction.

The response is 200 even when some or all URLs fail: each failure is listed in failed with a code (invalid_request, upstream_error, upstream_blocked, upstream_timeout, empty_page) and costs nothing. A bot-check page (a site's "verifying your request" interstitial) is never returned as the page: it is retried through another exit, and listed as upstream_blocked if every exit gets it.

Request body

ParameterTypeDefaultDescription
urls requiredstring[]1–20 public http(s) URLs of HTML pages or PDFs.
formatstringmarkdownWhat each result carries: markdown, text or the raw html. One of: markdown, text, html.
querystringWhat highlights are ranked against (up to 4,096 characters).
highlightsinteger0Passages per page ranked against query, 0–10 (about 600 characters each).
max_tokensintegerTrim each page's Markdown or text to about this many tokens (100–100,000).
include_linksbooleanfalseWebpage: also return the page's outbound links (url, text).
include_imagesbooleanfalseWebpage: also return the page's image URLs. Shopping: data: thumbnails on cards with no image URL.
max_ageintegerAccept a cached page up to this many seconds old, for 0.5 credit.
countrystringFetch the pages as seen from this country (two-letter ISO code).
timeoutintegerSeconds to wait for the pages, 1–90 (default 50, or 75 with summary or output_schema). A page still loading is listed in failed (upstream_timeout) and not charged.
summarybooleanfalseAdd a 3–6 sentence AI summary of each page, focused on query when set. +4 credits per page; a failed summary is free.
output_schemaobjectA JSON Schema (at most 16 KB). Each page's data comes back as JSON matching it in output. +4 credits per page; a failed extraction is free.
promptstringGuides the output_schema extraction: what to pick and how to normalise it (up to 4,096 characters). Needs output_schema.

Example request

Authenticate with Authorization: Bearer $SERPKITE_API_KEY (GET requests may pass ?api_key= instead). The official SDKs read SERPKITE_API_KEY for you. See Authentication.

curl https://api.serpkite.com/v1/extract \
  -H "Authorization: Bearer $SERPKITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://en.wikipedia.org/wiki/Web_crawler","https://www.rfc-editor.org/rfc/rfc9309.html"],"query":"robots.txt crawl delay","highlights":3}'

Response fields

FieldTypeDescription
requestobjecturls (deduplicated), format, query.
results[]arrayOne entry per page read: url (final, after redirects), status_code, title, published_at, metadata, cached, markdown / text / html, and links, images, highlights (text, score, heading) when asked for.
results[].summary / summary_errorstring | nullWith summary: true: the summary, or null and the reason it failed (not charged).
results[].output / output_erroranyWith output_schema: the JSON value matching your schema, or null and the reason it failed (not charged).
failed[]arrayURLs that could not be read: url, error (code, message). Not charged.
metaobjectrequest_id, credits_used, latency_ms, succeeded, failed, and llm (provider, model, tokens) when an AI step ran.

Every billed response also carries credit and latency headers (X-Credits-Used, X-Credits-Remaining, X-Request-Id…).

Example response

200 OK · illustrative
{
  "request": {
    "endpoint": "extract",
    "urls": [
      "https://en.wikipedia.org/wiki/Web_crawler",
      "https://www.rfc-editor.org/rfc/rfc9309.html"
    ],
    "format": "markdown",
    "query": "robots.txt crawl delay"
  },
  "results": [
    {
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html",
      "status_code": 200,
      "title": "RFC 9309: Robots Exclusion Protocol",
      "metadata": {
        "title": "RFC 9309: Robots Exclusion Protocol",
        "content_type": "text/html"
      },
      "cached": false,
      "markdown": "# Robots Exclusion Protocol\n\n…",
      "highlights": [
        {
          "text": "Crawlers SHOULD NOT use the cached version for more than 24 hours, unless the robots.txt is unreachable…",
          "score": 6.214,
          "heading": "Robots Exclusion Protocol > Caching"
        }
      ]
    }
  ],
  "failed": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_crawler",
      "error": {
        "code": "upstream_timeout",
        "message": "the page did not load in time"
      }
    }
  ],
  "meta": {
    "request_id": "req_01J8ZK4M6Q2V7",
    "credits_used": 1,
    "latency_ms": 2210,
    "succeeded": 1,
    "failed": 1
  }
}

Errors

Errors use one shape: {"error":{"code","message","request_id"}}. Errors are never billed. Full list in Errors.

StatusCodeMeaning
400invalid_requestA parameter is missing or invalid.
401unauthorizedThe API key is missing, invalid or revoked.
402insufficient_creditsYour balance is too low. Buy a pack or wait for the monthly free grant.
429rate_limitedToo many requests per second for your plan. Retry after the Retry-After header.
503upstream_errorGoogle could not be fetched or parsed. Not billed; retry after Retry-After.

Notes

  • Pages are fetched in parallel, 5 at a time; the call takes as long as the slowest page or timeout.
  • A short page that turns out to be a bot check, a login wall, a soft 404 or an empty app shell is listed in failed and not billed (a decision model checks short pages before they are charged or cached).
  • With summary or output_schema, query text and page content are sent to the model provider (Anthropic, or OpenAI for decision schemas) under their API terms, which exclude training on API data. See Privacy and data retention.
  • The remote MCP server has an extract tool with summary, output_schema and prompt.