# Webhooks

> Receive finished batch jobs from POST /v1/batches as signed HTTP POSTs. Payload format, headers, HMAC-SHA256 signature verification in Node.js, Python, Go and PHP, and the retry schedule.

When a [batch job](https://serpkite.com/docs/batch) finishes, SerpKite can POST the result to a URL you control instead of waiting for you to poll. Deliveries are signed with your account's webhook secret so you can prove they came from SerpKite.

## Set it up

1. ### Create a webhook secret

   Every delivery is signed. Your account gets a secret automatically with its first webhook; to see it, open **Settings → Webhooks** in the dashboard and click **Rotate secret** (**Generate secret** if none exists yet), which shows the new secret once. The secret starts with `whsec_`; store it as an environment variable such as `SERPKITE_WEBHOOK_SECRET`.

2. ### Choose where deliveries go

   Either set a default **Webhook URL** in the same settings screen (used by every batch job), or pass `webhook_url` on the [`POST /v1/batches`](https://serpkite.com/docs/batch) call, which takes precedence for every job in that batch:

   ```bash
   curl https://api.serpkite.com/v1/batches \
     -H "Authorization: Bearer $SERPKITE_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"endpoint":"search","requests":[{"q":"espresso grinder","num":100}],"webhook_url":"https://example.com/hooks/serpkite"}'
   ```

   Webhooks are for batch jobs only. Realtime endpoints such as `/v1/search` return the result in the response and don't accept `webhook_url`.

3. ### Verify and acknowledge

   Your endpoint checks the signature, stores the result, and answers with any `2xx` status within 10 seconds. Do slow work after responding.

## The delivery

SerpKite sends one `POST` per finished job, whether it succeeded or failed.

| Header | Value |
| --- | --- |
| `Content-Type` | `application/json` |
| `User-Agent` | `SerpKite-Webhooks/1.0` |
| `X-SerpKite-Event` | `batch.completed` |
| `X-SerpKite-Delivery` | The job ID. The same on every retry, so use it to deduplicate. |
| `X-SerpKite-Timestamp` | Unix time (seconds) when this attempt was signed. |
| `X-SerpKite-Signature` | `v1=` followed by the hex HMAC-SHA256 signature. |

The body is the job, with the result inline. Fields: `event`, `id`, `status` (`done` or `failed`), `endpoint`, `created_at`, `completed_at`, `credits_used`, and either `result` (the same body the realtime endpoint returns) or `error`.

```json
{
  "event": "batch.completed",
  "id": "0192f7a4-6c1e-7b3a-9d2f-5e8a1c4b7d90",
  "status": "done",
  "endpoint": "/v1/search",
  "created_at": "2026-09-29T08:00:00Z",
  "completed_at": "2026-09-29T08:03:12Z",
  "credits_used": 3.5,
  "result": {
    "request": { "endpoint": "search", "engine": "google", "q": "espresso grinder", "num": 100 },
    "results": ["…"],
    "related_searches": [],
    "meta": { "request_id": "req_01J8ZK4M6Q2V7", "credits_used": 3.5, "cached": false }
  }
}
```

When `status` is `failed`, there is no `result`; instead `error` holds `code` and `message`, and the job's credits were refunded. The result is also available from [`GET /v1/batches/{id}`](https://serpkite.com/docs/endpoints/batches) for 24 hours, so a lost delivery is never lost data.

## Verifying the signature

The signature is an HMAC-SHA256 over the timestamp, a dot, and the **raw request body**, keyed with your webhook secret:

```text
X-SerpKite-Signature: v1=hex( HMAC_SHA256( secret, timestamp + "." + raw_body ) )
```

To verify:

1. Read the raw body bytes before any JSON parsing. Re-serialised JSON will not match.
2. Compute the HMAC with your secret and compare it to the header using a constant-time comparison.
3. Reject timestamps older than a few minutes (5 minutes is a good default) to block replays.
4. Deduplicate on `X-SerpKite-Delivery`, since a retry can arrive after you already processed the job.

### With an SDK

The official SDKs verify the signature and the timestamp (5-minute tolerance) for you. Pass the raw body:

```typescript
import { verifyWebhook } from "serpkite";
const ok = await verifyWebhook(process.env.SERPKITE_WEBHOOK_SECRET!, rawBody, req.headers);
```

```python
from serpkite import verify_webhook
ok = verify_webhook(os.environ["SERPKITE_WEBHOOK_SECRET"], request.get_data(), request.headers)
```

```go
ok := serpkite.VerifyWebhook(os.Getenv("SERPKITE_WEBHOOK_SECRET"), body, r.Header, time.Now())
```

### Node.js

```javascript
import crypto from "node:crypto";
import express from "express";

const app = express();
const SECRET = process.env.SERPKITE_WEBHOOK_SECRET;

app.post("/hooks/serpkite", express.raw({ type: "application/json" }), (req, res) => {
  const ts = req.get("X-SerpKite-Timestamp") ?? "";
  const sig = req.get("X-SerpKite-Signature") ?? "";
  const expected =
    "v1=" + crypto.createHmac("sha256", SECRET).update(`${ts}.`).update(req.body).digest("hex");

  const fresh = Math.abs(Date.now() / 1000 - Number(ts)) < 300;
  const valid =
    sig.length === expected.length && crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected));
  if (!fresh || !valid) return res.status(401).end();

  const job = JSON.parse(req.body.toString("utf8"));
  res.status(204).end(); // acknowledge first, then process
  handleJob(req.get("X-SerpKite-Delivery"), job);
});
```

### Python

```python
import hashlib, hmac, os, time
from flask import Flask, request, abort

app = Flask(__name__)
SECRET = os.environ["SERPKITE_WEBHOOK_SECRET"].encode()

@app.post("/hooks/serpkite")
def serpkite_hook():
    ts = request.headers.get("X-SerpKite-Timestamp", "")
    sig = request.headers.get("X-SerpKite-Signature", "")
    raw = request.get_data()  # raw bytes, not request.json
    expected = "v1=" + hmac.new(SECRET, ts.encode() + b"." + raw, hashlib.sha256).hexdigest()

    if not ts.isdigit() or abs(time.time() - int(ts)) > 300:
        abort(401)
    if not hmac.compare_digest(sig, expected):
        abort(401)

    job = request.get_json()
    enqueue(request.headers["X-SerpKite-Delivery"], job)  # your own queue
    return "", 204
```

### Go

```go
package hooks

import (
	"crypto/hmac"
	"crypto/sha256"
	"encoding/hex"
	"io"
	"math"
	"net/http"
	"os"
	"strconv"
	"time"
)

var secret = []byte(os.Getenv("SERPKITE_WEBHOOK_SECRET"))

func SerpKite(w http.ResponseWriter, r *http.Request) {
	body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 32<<20))
	if err != nil {
		http.Error(w, "bad body", http.StatusBadRequest)
		return
	}
	ts := r.Header.Get("X-SerpKite-Timestamp")
	sec, err := strconv.ParseInt(ts, 10, 64)
	if err != nil || math.Abs(float64(time.Now().Unix()-sec)) > 300 {
		http.Error(w, "stale", http.StatusUnauthorized)
		return
	}
	mac := hmac.New(sha256.New, secret)
	mac.Write([]byte(ts + "."))
	mac.Write(body)
	expected := "v1=" + hex.EncodeToString(mac.Sum(nil))
	if !hmac.Equal([]byte(expected), []byte(r.Header.Get("X-SerpKite-Signature"))) {
		http.Error(w, "bad signature", http.StatusUnauthorized)
		return
	}
	w.WriteHeader(http.StatusNoContent)
	go process(r.Header.Get("X-SerpKite-Delivery"), body)
}
```

### PHP

```php
<?php
$secret = getenv('SERPKITE_WEBHOOK_SECRET');
$raw = file_get_contents('php://input');
$ts = $_SERVER['HTTP_X_SERPKITE_TIMESTAMP'] ?? '';
$sig = $_SERVER['HTTP_X_SERPKITE_SIGNATURE'] ?? '';

$expected = 'v1=' . hash_hmac('sha256', $ts . '.' . $raw, $secret);
if (!ctype_digit($ts) || abs(time() - (int) $ts) > 300 || !hash_equals($expected, $sig)) {
    http_response_code(401);
    exit;
}

$job = json_decode($raw, true);
http_response_code(204);
// store $job keyed by $_SERVER['HTTP_X_SERPKITE_DELIVERY']
```

## Retries

A delivery succeeds when your endpoint answers with any `2xx` status within 10 seconds. Anything else (a timeout, a connection error, a `3xx`, `4xx` or `5xx`) counts as a failure and is retried with exponential backoff: about 1 minute, then 2, 4, 8 and so on, up to roughly an hour apart, for up to 8 attempts in total.

The job's `webhook_status` tracks the outcome and is visible on [`GET /v1/batches/{id}`](https://serpkite.com/docs/endpoints/batches) and in the dashboard:

| `webhook_status` | Meaning |
| --- | --- |
| `pending` | Not delivered yet, or waiting for the next retry. |
| `delivered` | Your endpoint returned `2xx`. |
| `failed` | All attempts failed. Fetch the result by polling within 24 hours. |

Each retry is signed again with a fresh timestamp, so always verify against the headers of the request you received.

## Rotating the secret

Generate a new secret under **Settings → Webhooks** at any time. The new secret takes effect for the next delivery attempt and the old one stops working, so deploy the new value to your receiver first (accepting either secret for a short overlap), then rotate.

## Troubleshooting

- **Signature never matches:** you are hashing parsed-and-reserialised JSON, or a framework decompressed or re-encoded the body. Hash the exact bytes you received.
- **Duplicate processing:** retries reuse `X-SerpKite-Delivery`; store processed IDs and skip repeats.
- **Deliveries time out:** respond before doing the work. Results for `num: 100` or `include_content` can be large, so allow bodies of several megabytes.
- **Local development:** expose your machine with a tunnel (for example `cloudflared tunnel` or `ngrok`) and use that URL as `webhook_url`.

Related: [Batch requests](https://serpkite.com/docs/batch), [Batches endpoint](https://serpkite.com/docs/endpoints/batches), [Spend controls](https://serpkite.com/docs/spend-controls).