Page content
Get the top results as Markdown in the same /v1/search call with include_content: one request instead of a search plus N page fetches, +1 credit per page.
Snippets are often not enough to answer a question. With include_content, SerpKite runs the search and then fetches the top N organic results for you, converts each page to clean Markdown and puts it into results[].content. One request replaces a search plus N separate scrapes, which is the usual first step of a RAG pipeline.
curl https://api.serpkite.com/v1/search \
-H "Authorization: Bearer $SERPKITE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"q":"how does retrieval augmented generation work","country":"us","include_content":3}'Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
include_content | integer | 0 | Also fetch the top N result pages (0–5) as Markdown. +1 credit per page. |
- Only on /v1/search. Other endpoints return
400 invalid_request. 0(default) to5. The first N results are fetched, in position order.- For a single known URL, use /v1/webpage; for 1–20 selected URLs, use /v1/extract. For a site section, see Site ingestion.
Response
Each fetched result gets a content field with the page’s main content as Markdown, the same extraction /v1/webpage uses. Results beyond N, and results whose page could not be fetched, have no content.
{
"request": { "endpoint": "search", "engine": "google", "q": "how does retrieval augmented generation work", "country": "us", "language": "en", "include_content": 3 },
"results": [
{
"position": 1,
"title": "What is retrieval-augmented generation?",
"link": "https://www.example.com/rag-explained",
"domain": "example.com",
"snippet": "Retrieval-augmented generation (RAG) combines a retriever with a language model…",
"content": "# What is retrieval-augmented generation?\n\nRetrieval-augmented generation (RAG) is a technique that…"
},
{
"position": 2,
"title": "RAG, step by step",
"link": "https://docs.example.org/rag",
"domain": "docs.example.org",
"content": "# RAG, step by step\n\n1. Split your documents into chunks…"
}
],
"related_searches": [],
"meta": { "request_id": "req_01J8ZK4M6Q2V7", "credits_used": 4, "cached": false, "engine": "google" }
}
include_content combines with the other output options:
format: "compact"keepscontenton each row ofresults.format: "markdown"puts each fetched page, indented and wrapped in acontenttag, under its result.fields: "results.title,results.link,results.content"returns only what a retriever needs.
Highlights instead of whole pages
highlights: true replaces each page’s content with its 3 passages most relevant to q (results[].highlights: text, score, heading), ranked with BM25. If no passages can be produced, the fetched page can still appear as content. It costs nothing extra and keeps prompts small. See Search filters.
An answer over the pages
Add include_answer: true and Claude writes a short answer from the fetched pages (or their highlights), with [n] citations, in answer.text and answer.citations. It costs +4 credits with include_content (+2 over snippets alone) and is refunded with a meta.warnings entry when the model fails. For an answer-first call with streaming, use /v1/answer. See Cited answers.
Pricing
The search costs its normal price, and each fetched page adds 1 credit:
| Request | Credits |
|---|---|
/v1/search |
1 |
/v1/search + include_content: 3 |
up to 4 |
/v1/search + include_content: 5 |
up to 6 |
/v1/search + include_content: 3 via POST /v1/batches |
up to 2 |
The maximum is reserved when the request starts. Pages that can’t be fetched (timeouts, blocked or empty pages, and short pages that turn out to be a bot check, a login wall or a soft 404) are not billed, and the unused part of the reservation is refunded, so meta.credits_used and X-Credits-Used show what you actually paid.
A cache hit halves the search portion, while each include_content page fetched still adds 1 credit. For example, a cached 10-result search plus 3 successfully fetched pages costs 3.5 credits. Highlights do not change the price.
Limits and good practice
- Only public pages are fetched, the same way a logged-out visitor sees them. Nothing behind a login or paywall is retrieved.
- Pages built by JavaScript, or behind a check that a standard browser passes on its own, are rendered in a browser when the request still has time for it. A page behind a CAPTCHA counts as blocked and isn’t billed.
- Fetching pages adds latency, since the slowest of the N pages bounds the response. Use a client timeout of at least 60 seconds, or queue the request with
POST /v1/batches. - Page content can be long. Check
X-Tokens-Estimatebefore putting everything into a prompt, and chunk or truncate on your side.
Related: Webpage endpoint, RAG pipeline, Output formats.
Last updated: 2026-10-24