Skip to content

New AI assistant answers: ChatGPT, Gemini, Copilot and AI Mode by API

SerpKite
Get API key

POST /v1/extract

Web Extract API

Up to 20 web pages or PDFs in, clean Markdown out, fetched in parallel. Add a question and get the passages that answer it instead of whole pages.

Get a free API key Read the reference
1 credit per URL that comes back (0.5 from cache); failed URLs are free 2,500 free credits, no card Failed calls are free

Overview

What the Web Extract API does

The Web Extract API reads many pages in one call: the same fetch and Markdown conversion as the Webpage API, PDFs included, for up to 20 URLs at once, fetched in parallel. Each page comes back with its final URL, status, title, published date and metadata, as markdown, text or raw html.

With query and highlights, each page also gets the passages most relevant to your question, ranked with BM25 (deterministic, no model involved). max_tokens caps each page instead, so a batch of pages fits a context window.

The response is 200 even when some URLs fail: each failure is listed in failed with a reason code and costs nothing.

Need data, not text? output_schema fills your JSON Schema from each page (+4 credits per page) and summary: true adds a short summary (+4), both written by Claude. See the AI Data Extraction API.

What you get

request object
urls (deduplicated), format, query.
results[] array
One entry per page read: url (final, after redirects), status_code, title, published_at, metadata, cached, markdown / text / html, and links, images, highlights (text, score, heading) when asked for.
results[].summary / summary_error string | null
With summary: true: the summary, or null and the reason it failed (not charged).
results[].output / output_error any
With output_schema: the JSON value matching your schema, or null and the reason it failed (not charged).
failed[] array
URLs that could not be read: url, error (code, message). Not charged.
meta object
request_id, credits_used, latency_ms, succeeded, failed, and llm (provider, model, tokens) when an AI step ran.

Request

curl https://api.serpkite.com/v1/extract \
  -H "Authorization: Bearer $SERPKITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://en.wikipedia.org/wiki/Web_crawler","https://www.rfc-editor.org/rfc/rfc9309.html"],"query":"robots.txt crawl delay","highlights":3}'

Response (illustrative, placeholder domains)

200 OK · application/json
{
  "request": {
    "endpoint": "extract",
    "urls": [
      "https://en.wikipedia.org/wiki/Web_crawler",
      "https://www.rfc-editor.org/rfc/rfc9309.html"
    ],
    "format": "markdown",
    "query": "robots.txt crawl delay"
  },
  "results": [
    {
      "url": "https://www.rfc-editor.org/rfc/rfc9309.html",
      "status_code": 200,
      "title": "RFC 9309: Robots Exclusion Protocol",
      "metadata": {
        "title": "RFC 9309: Robots Exclusion Protocol",
        "content_type": "text/html"
      },
      "cached": false,
      "markdown": "# Robots Exclusion Protocol\n\n…",
      "highlights": [
        {
          "text": "Crawlers SHOULD NOT use the cached version for more than 24 hours, unless the robots.txt is unreachable…",
          "score": 6.214,
          "heading": "Robots Exclusion Protocol > Caching"
        }
      ]
    }
  ],
  "failed": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_crawler",
      "error": {
        "code": "upstream_timeout",
        "message": "the page did not load in time"
      }
    }
  ],
  "meta": {
    "request_id": "req_01J8ZK4M6Q2V7",
    "credits_used": 1,
    "latency_ms": 2210,
    "succeeded": 1,
    "failed": 1
  }
}

Every response also carries X-Credits-Used, X-Credits-Remaining, X-Cache and X-Tokens-Estimate headers. Full endpoint reference

Reference

Parameters

The fields /v1/extract accepts in its JSON body.

Web Extract API parameters
Name Type Default Description
urls required string[] – 1–20 public http(s) URLs of HTML pages or PDFs.
format string markdown What each result carries: markdown, text or the raw html. Values: markdown, text, html.
query string – What highlights are ranked against (up to 4,096 characters).
highlights integer 0 Passages per page ranked against query, 0–10 (about 600 characters each).
max_tokens integer – Trim each page's Markdown or text to about this many tokens (100–100,000).
include_links boolean false Webpage: also return the page's outbound links (url, text).
include_images boolean false Webpage: also return the page's image URLs.
max_age integer – Accept a cached page up to this many seconds old, for 0.5 credit.
country string – Fetch the pages as seen from this country (two-letter ISO code).
timeout integer – Seconds to wait for the pages, 1–90 (default 50, or 75 with summary or output_schema). A page still loading is listed in failed (upstream_timeout) and not charged.
summary boolean false Add a 3–6 sentence AI summary of each page, focused on query when set. +4 credits per page; a failed summary is free.
output_schema object – A JSON Schema (at most 16 KB). Each page's data comes back as JSON matching it in output. +4 credits per page; a failed extraction is free.
prompt string – Guides the output_schema extraction: what to pick and how to normalise it (up to 4,096 characters). Needs output_schema.

Full reference, error codes and headers are in the /v1/extract reference.

Use cases

What people build with the Web Extract API

01

RAG over chosen sources

Read the URLs your retriever picked, as Markdown, in one round trip.

02

Answer-focused context

highlights returns only the passages that match the question, keeping prompts small.

03

PDF reading

Papers, filings and manuals in PDF come back as text alongside HTML pages.

Pricing

Web Extract API pricing

Cost per call

1 credit

per URL that comes back (0.5 from cache); failed URLs are free

  • From $0.60 per 1,000 URLs at volume.
  • Credits never expire. No subscription.
  • Failed, empty and blocked calls are refunded.
  • 2,500 free credits = 2,500 URLs, then 1,000 credits a month.
See all pricing
Web Extract API cost per pack
Pack Price Per 1k credits Per 1k URLs URLs per pack
Starter $10 $1.00 $1.00 10,000
Growth $50 $0.80 $0.80 62,500
Pro $300 $0.60 $0.60 500,000

FAQ

Web Extract API: common questions

Start building

Start with the Web Extract API

2,500 free credits, then 1,000 every month. No credit card.