A key, an install, one call.
id, kind, status, progress, creditsUsed, data, next — with cursor pagination, Idempotency-Key, X-Request-Id and X-MeshArc-Credits on every response, typed error codes, and an OpenAPI spec committed with every route change.The first page in under a minute.
Make a key
export MESHARC_KEY=mesharc_… # from Settings → API keys
Read one page
curl -X POST https://api.mesharc.dev/v1/scrape \
-H "Authorization: Bearer $MESHARC_KEY" \
-d '{ "url": "https://www.gov.uk/news", "formats": "markdown", "maxCredits": 4 }'
Read the response
HTTP/1.1 200 OK
X-MeshArc-Credits: 1
X-Request-Id: 01J9…
{
"id": "6d1c…", "kind": "scrape", "status": "done", "creditsUsed": 1,
"data": [{
"url": "https://www.gov.uk/news", "verdict": "ok",
"method": "tls", "tier": "http", "credits": 1,
"markdown": "# News and communications\n…"
}],
"next": null
}
Crawl a site, keep it
curl -X POST https://api.mesharc.dev/v1/crawl \
-H "Authorization: Bearer $MESHARC_KEY" \
-H "Idempotency-Key: gov-news-2026-09-18" \
-d '{ "url": "https://www.gov.uk/news", "limit": 500, "useSitemap": true,
"webhook": { "url": "https://hooks.example/mesharc", "events": ["crawl.completed"] } }'
# 202 · poll GET /v1/crawl/{id}?cursor=… until status is done
# then POST /v1/crawl/{id}/keep { "name": "gov.uk news", "schedule": "daily" }
Projects, runs, pages, changes, destinations — under one workspace.
| method | path | what |
|---|---|---|
| POST | /v1/scrape | One URL synchronously, or urls[] as a batch. maxCredits caps a single page. |
| POST | /v1/crawl | A one-shot crawl of an ephemeral project. |
| POST | /v1/crawl/{id}/keep | Promote that crawl to a standing project. |
| POST | /v1/map | The site's URLs from its sitemap. 1 credit per sitemap file read. |
| POST | /v1/projects | Create one project or a batch. |
| PATCH | /v1/projects/{id} | Configure scope, formats, steps, schedule, tier. |
| POST | /v1/projects/{id}/dry-run | Resolve sources and sample URLs without spending. |
| GET | /v1/projects/{id}/discovery | The stored sitemap tree and its sections. |
| POST | /v1/projects/{id}/runs | Start a run. GET to list, DELETE to cancel. |
| GET | /v1/projects/{id}/pages | List, search, fetch content, recrawl, screenshot. |
| GET | /v1/projects/{id}/changes | List, review, page diff, markup diff. |
| GET | /v1/projects/{id}/destinations | Destinations, shapes and sync history. |
| POST | /v1/projects/{id}/export | A dataset as CSV or JSONL, per run or per project. |
| GET | /v1/webhooks/deliveries | What a job or project webhook was sent, attempt by attempt. POST …/{id}/redeliver sends one again. |
| GET | /v1/me | Plan, limits, credits. Keys, usage, billing, proxies, monitor, members. |
- validation
- unauthorized
- not_found
- rate_limited
- credits_exhausted
- plan_limit
- tier_limit
- scopes
- projects
- expiry
- rate limit
- Project
- Run
- Page
- Change
- Source
- Destination
Signed events for every change, and seventeen tools for an agent.
POST https://hooks.example/mesharc
X-MeshArc-Event: page.changed
X-MeshArc-Delivery: 8f2a…
X-MeshArc-Signature: sha256=<hmac-sha256 of the raw body, with the project's secret>
{ "event": "page.changed", "project": { "id": "…", "host": "www.gov.uk" },
"run": { "id": "…" }, "data": { "url": "…", "kind": "modified", "fields": ["title", "published_at"] },
"ts": "2026-09-18T06:30:00Z" }
import hmac, hashlib
def verify(secret: str, body: bytes, header: str) -> bool:
want = "sha256=" + hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
return hmac.compare_digest(want, header)
- page.changed
- page.blocked
- page.orphaned
- page.linked
- field.canonical
- site.failing
- site.recovered
- run.finished
- sitemap.added
- sitemap.removed
- sitemap.updated
- sitemap.file_added
- sitemap.file_removed
- crawl.started
- crawl.page
- crawl.completed
- crawl.failed
- scrape.completed
- scrape.failed
- map.completed
- batch.finished
pip install "mesharc[mcp]"
claude mcp add mesharc -e MESHARC_API_KEY=… -- mesharc-mcp
- scrape_urls
- extract_url
- map_site
- crawl_site
- search_pages
- get_page
- keep_crawl_as_project
- create_project
- describe_project_config
- get_project
- update_project
- list_projects
- start_run
- list_pages
- get_changes
- recrawl_pages
- get_job
Both SDKs are thin on purpose: every method is one API call, or a poll loop where wait is on, and returns the API’s JSON — so the OpenAPI spec is the reference and the SDK never disagrees with it.
A page row says what it is, how it was read, and what it cost.
verdict · shape · warnings
method · tier · credits
fields
bodies
widget · solved
exitBytes
inSitemap · orphan · inLinks
origin
What you would otherwise look up in the spec.
How do I authenticate?
Authorization: Bearer <key>. Keys are made under Settings with a scope (read, write or admin), an optional project allow-list, an expiry and their own requests-per-minute. A key restricted to some projects sees the others as 404, not 403 — existence is never confirmed across the line.
Is /scrape synchronous?
One url waits for its page — 60 seconds by default, 120 at most — and returns it in the same response. Pass urls[] and it is a batch, always 202 with a job to poll. Either way the same envelope comes back: id, kind, status, progress, creditsUsed, data, next.
What does Idempotency-Key do?
Send it on any POST and a retry within 24 hours returns the first answer instead of starting a second job. The key is scoped to the route it was first used on. Batch endpoints accept one per item.
How does pagination work?
Every list takes limit (up to 1,000) and cursor, and answers with next — a ready-made URL, or null when the list is done. A job’s pages page the same way, so a 200,000-page crawl is read in slices without holding the lot.
How are webhooks signed and retried?
Every delivery carries X-MeshArc-Signature: sha256=<HMAC-SHA256 of the raw body with the project’s secret>, plus X-MeshArc-Event and X-MeshArc-Delivery. Answer 2xx within the timeout; anything else is retried six times over about a day (1 m, 5 m, 30 m, 2 h, 6 h, 12 h), then paused and shown as such with its log.
What happens at the rate limit?
A 429 with Retry-After and X-RateLimit-Limit, -Remaining and -Reset on every response. The window is a minute, per key; the plan sets the default and a key can be given a lower one.
1,055 selftest cases · OpenAPI spec committed with every route change
Make the first call.
A one-time 1,000 credits, browser tier included, no card. The response says the mode it took and what it cost.