Page content

Snippets are often not enough to answer a question. With include_content, SerpKite runs the search and then fetches the top N organic results for you, converts each page to clean Markdown and puts it into results[].content. One request replaces a search plus N separate scrapes, which is the usual first step of a RAG pipeline.

curl https://api.serpkite.com/v1/search \
  -H "Authorization: Bearer $SERPKITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"q":"how does retrieval augmented generation work","country":"us","include_content":3}'

Parameters

• Only on /v1/search. Other endpoints return 400 invalid_request.
• 0 (default) to 5. The first N results are fetched, in position order.
• For a single URL you already have, use /v1/webpage instead.

Response

Each fetched result gets a content field with the page’s main content as Markdown, the same extraction /v1/webpage uses. Results beyond N, and results whose page could not be fetched, have no content.

{
  "request": { "endpoint": "search", "engine": "google", "q": "how does retrieval augmented generation work", "country": "us", "language": "en", "include_content": 3 },
  "results": [
    {
      "position": 1,
      "title": "What is retrieval-augmented generation?",
      "link": "https://www.example.com/rag-explained",
      "domain": "example.com",
      "snippet": "Retrieval-augmented generation (RAG) combines a retriever with a language model…",
      "content": "# What is retrieval-augmented generation?\n\nRetrieval-augmented generation (RAG) is a technique that…"
    },
    {
      "position": 2,
      "title": "RAG, step by step",
      "link": "https://docs.example.org/rag",
      "domain": "docs.example.org",
      "content": "# RAG, step by step\n\n1. Split your documents into chunks…"
    }
  ],
  "related_searches": [],
  "meta": { "request_id": "req_01J8ZK4M6Q2V7", "credits_used": 4, "cached": false, "engine": "google" }
}

include_content combines with the other output options:

• format: "compact" keeps content on each row of results.
• format: "markdown" puts each fetched page, indented and wrapped in a content tag, under its result.
• fields: "results.title,results.link,results.content" returns only what a retriever needs.

Pricing

The search costs its normal price, and each fetched page adds 1 credit:

The maximum is reserved when the request starts. Pages that can’t be fetched (timeouts, blocked or non-HTML pages) are not billed, and the unused part of the reservation is refunded, so meta.credits_used and X-Credits-Used show what you actually paid.

Limits and good practice

• Only public pages are fetched, the same way a logged-out visitor sees them. Nothing behind a login or paywall is retrieved.
• Fetching pages adds latency, since the slowest of the N pages bounds the response. Use a client timeout of at least 60 seconds, or queue the request with POST /v1/batches.
• Page content can be long. Check X-Tokens-Estimate before putting everything into a prompt, and chunk or truncate on your side.

Related: Webpage endpoint, RAG pipeline, Output formats.