Docs · The API

The API

Base URL https://api.mesharc.dev/api/v1. The OpenAPI spec is at https://api.mesharc.dev/docs. The app uses these same routes; there is no private API behind it.

Authentication#

Authorization: Bearer mesharc_…

A key is issued under Settings → API keys (or POST /me/keys) and shown once. It carries scopes (read < write < admin), optionally a set of projects it may see, an expiry, and a rate limit. The app signs in with a session cookie instead and reaches the API through its own /api/mesharc proxy; the routes are the same.

A route that changes something needs write (the member role); keys, the rate ceiling, deleting a project, webhook secrets, connections and destinations need admin. A session whose email is not verified is refused those with 403 email_unverified, and one in a workspace that requires two-factor authentication without having it on is refused with 403 mfa_required; a session with two-factor on that last showed its code more than ten minutes ago gets 403 mfa_stale from the handful of actions that ask again (POST /auth/2fa/verify with a code clears it). API keys are unaffected by all three.

A key held to some projects sees only those: any other project is a 404. The routes about the whole workspace — its sub-accounts (/orgs/children) and its saved logins (/logins) — refuse it with 403 project_key, and it may make only keys held to some of its projects (POST /me/keys without projects is 403 project_key).

A key of a workspace with sub-accounts acts on one of them by naming it:

X-MeshArc-Tenant: <sub-account id>

The request is then the sub-account's, with the key's own scopes and rate limit. Only a key of that sub-account's own parent may do this, and only a key that is not held to some projects. A session is told to switch workspace instead (400 tenant_refused); a key held to some projects is refused with 403 tenant_refused; an id that is not one of the key's workspace's sub-accounts — another workspace's, a sibling's, the parent's own, or any id sent with a sub-account's own key — is 404, as anything else not yours is; and a suspended sub-account is 403 suspended. Only the 404 is kept in the sign-in log, as a refused attempt; each use is counted per key, sub-account and day.

Conventions#

Job envelopeEvery asynchronous verb answers the same shape: {id, kind, status, progress: {done, total, queued}, creditsUsed, counts, data: [...], next, cursor, stop, request_id, error}. status is queued / running / done / error / cancelled; next is a ready URL for the page after this one.
Pagination?cursor=&limit= — limit up to 1,000 (25 by default on a crawl's pages; up to 100 without bodies); next carries the cursor. A cursor that is no page of the list (past 2⁶³−1) is 400.
IdempotencyIdempotency-Key: <your key> on POST /scrape, /crawl, /map, /search, /agent, /agent/{id}/continue, /agent/{id}/follow-up, /monitors, /monitors/{id}/run and /orgs/children returns the first answer for 24 hours, scoped to the route, so a retry after a dropped connection never starts a second job. A retry that arrives while the first request with its key is still being taken answers 409 with code in_flight: ask again shortly. POST /projects/{id}/runs needs no key: a second run while one is queued or running answers 409.
Request idsEvery response carries X-Request-Id (yours if you sent one) and the body's request_id; quote it to support.
CreditsX-MeshArc-Credits on a scrape's response says what it cost; every envelope carries creditsUsed.
Rate limitsX-RateLimit-Limit, -Remaining, -Reset per key; 429 when exceeded.
Errors{"error": "<what happened>", "code": "<what it means>", "request_id": "…"}. Codes: validation (400 / 422), unauthorized (401), plan_limit (402 — a plan that does not include something, or credits spent), forbidden (403), email_unverified (403), project_key (403 — a key held to some projects, on a route about the whole workspace), suspended (403 — the workspace, or the sub-account named, is suspended), not_found (404 — including anything the key may not see; existence is never confirmed), conflict (409), in_flight (409 — the first request with this Idempotency-Key is still being taken), sub_account (409 — a billing action on a sub-account, whose billing is its parent's; or a sub-account making sub-accounts), tenant_refused (400 / 403 — X-MeshArc-Tenant from a session, or from a key held to some projects), logins_off (409 — saved logins are not switched on here), login_expired (409), login_host (400 — a login for another site), too_large (413), rate_limited (429), unavailable (503), internal (500).
TimeISO 8601 UTC everywhere.

The verbs#

POST /scrape — URLs in, content out, no project#

{"url": "https://example.com/pricing", "formats": "markdown,links", "timeout": 60,
 "config": {"render_js": "auto"}, "scrapeOptions": {"onlyMainContent": true}, "maxCredits": 4}

One url waits for the page and returns it — up to timeout seconds (60 by default, 120 at most), then 202 with the id to poll, the fetch carrying on either way. A list of urls (up to 500) is always 202: grouped by host, fanned out, polled at GET /scrape/{id} (a row per URL with httpStatus, verdict, credits) or announced to webhook_url (batch.finished, signed with the secret returned once). DELETE /scrape/{id} cancels; GET /scrape lists recent ones. formats picks the bodies returned (markdown,text,cleanHtml,rawHtml). maxCredits caps what one page may cost. destination names one of the workspace's destinations — a database, a warehouse, a bucket — and the result is written there as well as returned: a single URL before it answers, so the response carries destinationSync (status, written, skipped, location, error); a batch once, when its last URL is in, reported on GET /scrape/{id}. The destination is checked before anything is queued: another workspace's is a 404, a paused one a 400.

A page row: url, httpStatus, status, verdict, errorCode, shape, warnings, signals, method, tier, credits, billedAs, climbedTo, attempts, tierSkipped, words, language, markdown, text, cleanHtml, html, links, fields, screenshot, head, reason, crawledAt.

POST /crawl — a one-shot crawl, no project first#

{"url": "https://docs.example.com/", "limit": 200, "maxDepth": 3,
 "includePaths": ["/docs/*"], "crawlMode": "sitemap_first", "maxTier": "browser",
 "scrapeOptions": {"formats": ["markdown", "links"]},
 "webhook": {"url": "https://you.example/hook", "events": ["crawl.completed"], "metadata": {"ref": "job-7"}}}

202 with the envelope; the crawl runs as a hidden, ephemeral project, kept for a day unless you keep it with POST /crawl/{id}/keep. GET /crawl/{id}?cursor=&limit=&formats= pages through its pages oldest first; GET /crawl/{id}/page?url= returns one of them in full (matched under either scheme and with or without a trailing slash); DELETE /crawl/{id} stops it; POST /crawl/{id}/keep {name, schedule, retention} promotes it to a standing project with its run already in place. The short names (limit, maxDepth, includePaths, excludePaths, useSitemap, sitemapMode, crawlMode, allowSubdomains, maxAge, maxTier, delay, concurrency, respectRobots) and scrapeOptions map onto the configuration; config takes any key directly. The webhook's secret comes back once as webhookSecret. destination writes the crawl's pages to one of the workspace's destinations when it is done — every page, as a baseline — and GET /crawl/{id} carries how the write went under destinationSync.

POST /map — the site's URLs from its sitemap#

{"url": "https://www.gov.uk/", "search": "visa", "limit": 500, "sitemapMode": "auto", "timeout": 10}

Synchronous when the tree resolves within timeout (10 s by default, 60 at most), else 202 and GET /map/{id}. Rows: {url, lastmod, changefreq, source, file, section}; totals (files, urls, indexes, blocked), robots, method. limit up to 50,000 (5,000 by default); search filters on the URL. Costs 1 credit per sitemap file read — most sites are one file; gov.uk's 29 are 29.

POST /search — web or news results for a query#

{"query": "rail funding consultation 2026", "limit": 5, "country": "GB", "lang": "en", "freshness": "week",
 "includeDomains": ["gov.uk"], "excludeDomains": [], "scrape": {"formats": ["markdown"], "maxCredits": 16}}

Google's results page, read through the ladder like any page; when Google refuses or reads short, DuckDuckGo's (fetched through the residential exit, as it limits each address; the first results page only, so page 2 and on skip it), then Bing's. These are results pages, not a search API. You get organic results only: {position, url, title, snippet, source, engine}, with no ads, answer boxes or AI summaries. sources is ["web"] (the default) or ["news"], one per search: news reads Google's News tab (Bing News when Google refuses; DuckDuckGo has no news page), and each news result adds its publisher and age as the engine shows it ("20h", "1 day ago"). limit is 1–10 (one results page), and page (1–10) reads further in: page 2 is results 11–20, numbered from 11, each page a search and a charge of its own. freshness is hour, day, week, month or year; domains become site: operators. Waits like /scrape (timeout, 60 s by default, 120 at most), then 202 and GET /search/{id}. With scrape, the results' URLs are scraped as a batch and each result gains its page as it lands; the search reads running until they are in. A search costs at most 4 credits whatever rung read it: Google's results page (read in a browser or through the residential exit), and DuckDuckGo's or Bing's at 1 each when their results are used, within those 4; it shows on the billing page as a search. A search aims to answer within about 35 seconds (SEARCH_BUDGET_S): Google's held-back result links are looked up within what is left of that, never under a floor of their own, and DuckDuckGo is asked while they are. Google's retry on the residential browser after its wall is not held to it (nor its wait, up to 20 seconds, for a residential browser to come free), and a results page read through the ladder keeps its own page budget, so a search can take longer and comes back as a job. A refused results page is free: blocked, with outcome: "no_results", means every engine refused or had nothing about the query, and attempts says why: each engine's rung, credits, listed (results its page listed), outsideDomains (dropped for being outside includeDomains / excludeDomains) and added (results it added after the answer given). When an answer is short of the limit, the next engine is asked, and the answer with most results about the query is given, topped up with the others' results about the query that it does not have (the same URL never twice, tracking parameters such as utm_… and srsltid aside), each result naming its engine. A result is about the query when it carries all of a one- or two-word query and half of a longer one; a number, and the question's own words (do, my, wie, cómo, …), are never required. A page is an answer only when enough of its results each carry half of the query's longer words (four letters or more, so an abbreviation an answer spells out is not held against it): all of a page of one or two, else at least two and a quarter of the page. A page where only a stray result does is an answer to something else (a robot's page) and is not given. A short word in Japanese, Chinese, Thai or Korean is looked for inside a result's text; a phrase written without spaces is not held to either rule. DuckDuckGo is paused a few minutes when it walls the server's own address (its challenge, or a 202, 403 or 429); the attempt says paused for …s. A wall through the residential exit refuses that page only, as the next request leaves from a new address. Bing is never paused. An outage, or a page about something else, is refused without pausing anything. Google's page always names its language and region: lang and country, else English and the US. The same search within the hour (ten minutes for news) is answered from the last one at no charge (cached: true). Reading a results page is the one documented exception to robots.txt; the pages a search scrapes obey it as usual. Add "webhook": "https://you.example/hook" (or {url, events, metadata}) and the search's end is posted to it once, signed with the webhookSecret the POST returns once: search.completed when its results are in (before the pages a scrape reads, which GET /search/{id} shows as they land), search.failed when every engine refused it or had nothing for it, or it failed, an answer from the cache included. The data is {search: {id, query, status, outcome, results, cached, error}} (outcome is no_results when it ended blocked), results being how many; GET /search/{id} has them. With "timeout": 0 the POST answers at once with the id, so a caller that cannot wait half a minute hears from the webhook instead. Takes an Idempotency-Key.

GET /search — past searches#

?limit=25&cursor=&q= → {data, next}, newest first. Each row is {id, query, params, status, engine, rung, cached, resultCount, domains, scraped, creditsUsed, error, createdAt, finishedAt}: domains is the first three result hosts, and creditsUsed includes the scraped pages. q keeps searches whose query contains it, in any case. Reading the list fetches nothing and costs nothing; GET /search/{id} opens one as it was stored. Searches are kept 7 days, but a scraped page's text only 1 day. After that the result's page reads {"status": "expired"}.

POST /agent — ask for data, get it back with its sources#

{"prompt": "List each competitor's pricing plans with the monthly price and what each plan includes",
 "urls": ["https://example.com", "https://example.org"],
 "schema": {"type": "object", "properties": {"tiers": {"type": "array", "items": {"type": "object",
   "properties": {"company": {"type": "string"}, "plan": {"type": "string"}, "price_month": {"type": "number"}}}}}},
 "maxCredits": 2000, "maxSteps": 40, "allowedDomains": ["example.com", "example.org"]}

An agent searches the web, reads the site maps and pages it needs through the same ladder as a crawl, and answers. When what it needs is behind a "Load more" button, a tab, a filter or an endless scroll, it renders the page with browser steps (click, scroll, wait, select; never typing, so it fills no form and logs in nowhere), at most three renders of a page per run, each a step in the trace with its actions and credits. With schema (a JSON Schema of type object or array), data is JSON in that shape; without one, {text}. sources lists the pages it read, and with a schema fieldSources says where each value came from: its path in data (plans[0].price, or [2].name when the answer is a list) to the url and pageId of the page it was read on. Only pages the run read are named; a value without an entry had no page to point to. loginNotices lists the saved logins the run needed and could not use — {host, loginId, name, status}, status expired or deleted — so those sites were read signed out, whatever the answer says; on auto, a login saved again while the run goes on is used from then on. It runs in the background: 202 with next, then poll GET /agent/{id} until status is not queued or running (or pass timeout, up to 120 s, to wait). Statuses: done, credit_limit (stopped at maxCredits; data.partial holds what it had), error, cancelled, and expired once its results are past their keep date (then GET answers 410).

Pages are charged as they are read, at the rung that answered (refused pages are free); model tokens at the model provider's price plus 20%, in credits (the billing page shows the rate per million tokens). maxCredits (2,000 by default) caps the run; a budget too small for even one model call (one costs tens of credits) stops at once, at credit_limit, with nothing spent. allowedDomains keeps it on those sites (and their subdomains; *.example.com reads as example.com). Each entry must be a domain, or the request is refused (400) — never quietly dropped, which would leave the run with no limit; the same holds for a search's includeDomains and excludeDomains. Without allowedDomains the run has no domain limit: it may read any site it finds, and a page it reads could try to send it elsewhere. That is a deliberate default for the API, so a run can follow the web; pass your starting URLs' domains to hold it to them. (The Playground does that by default and warns when a run has no limit.) The agent treats page text as data, never as instructions, every page it reads is in its trace, and maxCredits caps what any detour can cost. Takes an Idempotency-Key.

  • GET /agent/{id}/trace?after=N — the run's steps after step N (model calls, searches, pages read with their rung and credits, maps, waits), {data, last}; poll with after=last.
  • GET /agent — runs, newest first, {data, next}; status keeps one status, model one model's runs (as a run names it, e.g. openai:gpt-5.4-mini), and since / until (an ISO 8601 date or date-time) the runs made from or before then.
  • DELETE /agent/{id} — cancel: a queued run stops at once, a running one before its next step. What it used stays charged.
  • POST /agent/{id}/continue — carry on a run that stopped at its credit limit, with a new maxCredits (and maxSteps; the stopped run's when left out). It starts a new run on the same thread (continuesRunId) that resumes where the stopped one was, keeps the pages it read (not paid for again), its saved login and its data providers (each checked again), and answers in full; the stopped run then names it in continuedBy. A run can be continued once; 409 for a run that did not stop at its limit, 410 past its keep date. A run's webhook carries over.
  • POST /agent/{id}/follow-up — ask a follow-up in a conversation. The body is a POST /agent request (its own prompt, urls, schema, maxCredits, maxSteps, allowedDomains, connectionId, logins, dataConnections, webhook, timeout); the new run is on the same thread (threadId), names the message it follows in followsRunId, and that message names it in followedBy. It answers after everything asked and found in the conversation before it, and may read again, free, any page the conversation read — such a read is a fetch step with reused: true and 0 credits, and the page is one of the follow-up's sources. {id} is the conversation's latest message, finished: 409 while it works, for an earlier message (the error names the latest), or past 30 messages; 410 past its keep date. Each follow-up pays for its own work, and restarts the conversation's 7 days.
  • While a run is running, GET /agent/{id} (and each run of a conversation) carries doing: the step under way now, {kind, text, url, since} — kind is model, search, fetch, render, map, extract or lookup. The trace has a step once it has ended; doing is what it is doing before then. null otherwise.
  • GET /agent/conversations — conversations, the latest message first: {id (the thread, its first run's id), title (its first question), messages, status and lastRunId (its latest run), model, creditsUsed, createdAt, updatedAt, expiresAt}; q keeps those whose first question holds it, limit and cursor page. A table's rows are not listed. GET /agent/conversations/{id} adds runs: every run in it, oldest first, each as GET /agent/{id} answers.

An agent run can read members-only pages with the workspace's saved logins, none of which needs picking: "logins" says which it may use — "auto" by default, every active saved login, or "none", or a list of login ids (at most 20; "loginId", one id, is the older form of a list of one). The agent reads pages signed out, as the public sees them, and signs in page by page where a page needs the account: what only the account shows, or a page that asked it to log in. A public page stays signed out, because a site may show a signed-in visitor something else or send it away from a public page; one run can read a site's public pages signed out and its account pages signed in. The page's URL picks the login: a page on a login's site, or one of its subdomains, is read with it — the most specific when two fit, a login for console.example.com before one for example.com. A login is never sent to another site. Each page read signed in names its login in the trace (login, loginHost), and a page the site answered with another names that one (finalUrl). A page that shows the site's login page signed in is reported as needing a login and that login is marked expired; one met signed out says when the run may read it signed in, or when a saved login for the site exists but has expired, or was not one the run named. An agent monitor's run reads the pages its last run read signed in so again, unless the agent decides otherwise, so a value is not compared with what the public sees. A list naming a login that is expired, deleted or not this workspace's is a 400 when the run is asked for; "auto" uses no login while logins are switched off.

To let it look up company or financial data, pass "dataConnections": up to two of the workspace's data providers, one per provider. Each gives the run its lookup tool, on your own key; MeshArc charges no credits for a lookup.

To run on your own model, pass "connectionId": one of your workspace's LLM connections (any OpenAI-compatible chat endpoint, its model and your key; see Connectors). Your provider bills the tokens to your key, and MeshArc charges 1 credit per 1,000 tokens, as for every model stage on your key, plus the pages read. The model must support tool calls.

Add "webhook": "https://you.example/hook" (or {url, events, metadata}) and the run's life is posted to it, signed like every MeshArc webhook with the webhookSecret the POST returns once: agent.started, agent.action (the steps since the last one, at most every 5 s, with progress), then one of agent.completed (with data, sources and creditsUsed, as GET /agent/{id} answers; a run stopped at its limit is completed with status credit_limit), agent.failed or agent.cancelled. All five by default.

A run, its answer, its steps and its webhook messages are kept for 7 days.

Pass "proposals": true and the agent may propose things to watch, rather than only say so: crawling a site on a schedule, a search kept and run on a schedule, or a question asked again on a schedule — each with why and about what it would cost a month. Nothing is made, scheduled or charged: a proposal waits for a member to approve it (see GET /proposals). Without it, the agent has no way to propose.

A pattern in a schema is at most 200 characters, and a pattern that nests repetitions ((a+)+, (\w+\s?)*) is refused with a 400: answers are checked against it, and that shape can take exponential time on some text. Ordinary patterns — an email, dotted names such as [a-z]+(\.[a-z]+)*, a phone number — pass.

POST /agent/batch — the agent over every row of a table#

{"csv": "Company,Website\nAcme,acme.com\nGlobex,globex.com\n",
 "prompt": "What does {{Company}} charge for its cheapest plan?", "urls": ["{{Website}}"],
 "schema": {"type": "object", "properties": {"plan": {"type": "string"}, "price_month": {"type": "number"}}},
 "maxCreditsPerRow": 300, "concurrency": 3, "name": "Competitor prices", "webhook": "https://you.example/hook"}

One agent run a row: the prompt and starting URLs filled from the row's columns, written as {{column}}, and the answer — shaped by schema, a JSON Schema of type object — the row's new columns. Send the table as csv (its first line names the columns; a cell like acme.com in a URL is read as https://acme.com) or as rows, a list of objects. Your plan sets how many rows a batch may have: 25 on the free plan, 200 on Starter, 500 on Growth, 1,000 on Scale.

Everything is checked before anything runs: every {{column}} must name a column, the prompt must name at least one (or the row must start at its own URL), and the schema is checked once. Then each row is checked as POST /agent would check it; a row that fails — its prompt came out empty, a cell is not a URL — is kept as invalid, with the reason, and never run or charged. A table where no row can run is a 400.

  • Where a row may go: without allowedDomains, each row is held to its own URLs' sites, subdomains included (a row starting at www.acme.com may read shop.acme.com); a list holds every row to it; [] lifts the limit. A row with no URL goes where its search leads.
  • A few at a time: concurrency rows run at once (3 by default, 10 at most, and never more than your plan's agent slots). As each row ends, the next starts.
  • What it costs: maxCreditsPerRow is required: each row's run stops there and pays only for what it used. The batch never spends more than rows × that cap, or your balance when the batch started if that was less; a row it can no longer afford is skipped, free, saying why.
  • Data and logins: dataConnections and logins (or loginId) are as POST /agent takes them, checked once for the batch and given to every row's run.
  • Before you start: POST /agent/batch/preview with the same body checks everything and answers what would run — valid, invalid and the first invalid rows with their reasons, the first prompts as their rows will ask them (with their starting URLs and sites), and an estimate: about what a row costs (your last batch rows' average, or the cap before you have any), the total, the most it could be, and your balance. It starts nothing and spends nothing.

It answers 202 with the batch: counts by row status, budget, creditsUsed, averagePerRow, next. Takes an Idempotency-Key; with a webhook, webhookSecret comes back once and agent.batch.finished is posted once, when the last row has ended, with the totals.

  • GET /agent/batch/{id}?offset=&limit=&status= — the batch and a page of its rows in table order (limit up to 200): each row's values, status (pending, queued, running, done, credit_limit, error, cancelled, invalid, skipped), data, fieldSources, sources, creditsUsed, budget, error and runId (the row's own run, at GET /agent/{runId}). more names the next page; status keeps one status.
  • GET /agent/batch/{id}/export?format=csv|jsonl — each row's number (row, from 1) and status, then the table as you sent it, each answer value a new column (one named like an input column becomes <name> (answer)), then creditsUsed, sources, error and runId. In CSV, a cell that would start a spreadsheet formula is written as text.
  • GET /agent/batch — batches, newest first. A batch's rows are not listed under GET /agent.
  • DELETE /agent/batch/{id} — cancel: rows not started are cancelled, running ones stop before their next step. What rows used stays charged; their answers are kept.

A batch and its rows are kept as long as its runs: 7 days, then 410.

POST /monitors — keep a search or an agent request and run it on a schedule#

{"kind": "search", "schedule": "daily", "request": {"query": "acme pricing", "limit": 10},
 "name": "Acme on Google", "webhook": "https://you.example/hook", "baselineId": "<a finished search's id>",
 "budget": {"perMonth": 500}, "alerts": {"email": true, "chatUrl": "https://hooks.slack.com/services/…"},
 "addToProject": "<a project id>"}

A monitor runs the same POST /search or POST /agent body (request, checked as those endpoints check it) on its schedule: hourly, daily (03:00 UTC) or weekly (Mondays, 03:00 UTC). Each run is an ordinary search or agent run, charged as one and listed with your others. When it ends, it is compared with the last run that answered:

  • A search: results are matched by URL (no fragment, no trailing slash, www. the same site), giving new and dropped results ({position, url, title}, position may be null) and moved ones ({url, title, from, to}). When another engine answered than last time (Google refused and Bing read it), engines rank differently by nature: new results are still reported, but drops and moves are not (comparable: false). A monitored search reads its results page only, so scrape is refused.
  • An agent run: the answer is compared value by value. Each change is {path, kind, before, after, source}: kind is added, changed or removed, and source is the page the value was read on. A list of records is matched by the field that names each record (id, name, title, url, plan, sku, product), so the same records in another order are no change. Text is compared with its spacing and case ignored.

The first run that answers is the baseline. baselineId names a finished search or agent run with the same request to use as that first run. For an agent run, budget and steps may differ. For a search, scrape may differ, but its results limit must be at least the monitor's (a search that read fewer results is refused: the ones it stopped short of would read as new). In the app, "Save and schedule" on a search and "Keep and re-run" on an agent run send the one on screen, so keeping it costs nothing. A run that continued another (POST /agent/{id}/continue) is not offered there: its maxCredits is only the top-up it was given, so run from scratch on a schedule it would stop at its credit limit each time. Nor is a follow-up (POST /agent/{id}/follow-up): it asks about the conversation before it, which a monitor's run would not have. Without it, the first run starts when the monitor is made. A run that did not answer (a blocked results page, an agent stopped at its credit limit, a failure) is withheld: kept and shown, but nothing is compared with it and nothing is announced. A slot runs once. A workspace with nothing left to spend skips the slot, and the run says so (status: "skipped").

alerts tells people too: email mails the workspace's members who take run emails, and chatUrl (a Slack, Discord, Microsoft Teams, Google Chat or Mattermost incoming webhook, https) gets the change in that service's own shape -- the new, dropped and moved results, or the values added, changed and removed. The chat URL comes back masked; a masked value sent back keeps it. addToProject (a search monitor's) names a project of the workspace: the results of the first run, and each later run's new results, that are pages of that project's site are added to its listed URLs, so the next crawl reads them (addedToProject on the run says how many). A monitor's searches are kept 90 days, not a week.

sentiment (a news search monitor's: request.sources is ["news"]) scores each run's new results with one of the workspace's LLM connections (see Connectors): {"connectionId": "<an llm connection>", "only": ["negative"]}. Each new result gains sentiment — positive, neutral or negative — and sentimentReason, a line on why, read from its title, publisher and snippet (the first 20 new results of a run; a baseline has none). The run's diff.sentiment says {model, scored, of, error}. The tokens go to your provider on your key, and MeshArc charges 1 credit per 1,000, counted in the run's credits (so a monthly cap sees them). only keeps the chat channel and email to runs with a new result of those tones; the webhook hears every change, with the tones. A model that fails leaves its error on the run and the change is announced unscored, to everyone. {} turns it off.

budget caps what it spends, in credits (0 or left out is no cap). perMonth: once its runs this calendar month (UTC) have cost that much, a due run is skipped (status: "skipped", error saying the cap is reached) until the next month or a higher cap; Run now is not held to it. perRun is an agent monitor's: each run's maxCredits is the smaller of the request's and this (a monitored search reads one results page, 1 to 4 credits, so perRun is refused for one).

With a webhook, a run that changed something posts search.changed or agent.changed once (and a slot skipped for the monthly cap posts budget.reached, once that month), signed with the webhookSecret returned once (here, or by the PATCH that sets a new one). The data is {monitor: {id, name, kind}, run}, with the run as GET /monitors/{id}/runs shows it. Takes an Idempotency-Key. 201 with the monitor and its first run.

An agent's answer as rows. When an agent monitor's answer is a list of records — each competitor's plans, the open roles, the stockists — rows follows each record on its own, as a project's rows follow the records its pages hold:

{"kind": "agent", "schedule": "daily",
 "request": {"prompt": "List Acme's plans with their monthly price", "urls": ["https://acme.example/pricing"],
             "schema": {"type": "object", "properties": {"plans": {"type": "array", "items": {"type": "object",
               "properties": {"name": {"type": "string"}, "price": {"type": "number"}}}}}}},
 "rows": {"path": "plans", "key": ["name"], "confirmAfter": 2}, "destination": "<a destination id>"}

path is the answer's list (left out: the answer itself when the schema is an array, or its one list of records); key the column or columns that name a record (left out: the first of id, name, title, url, plan, sku, product); the columns are the list's own properties, typed from the schema; rows: true takes every default. Each answered run then says which records are added, which changed (one entry per column, with both values) and which are gone — a run's diff lists them, summary counts them (1 added · 1 changed · 1 withheld), and the webhook also posts row.changed with the records that moved.

An agent cannot say how much of the web it saw, so a record missing from one answer is not gone: it is withheld — still listed, marked withheld, and recorded once as a withheld event — and it is gone only when confirmAfter answers in a row (2 by default, 1 to 10) lack it. An answer that lost half the records or more counts as no miss at all, and an answer that did not finish (its credit limit, a failure) changes nothing. Each value keeps the page it was read on (sources); the same value read from another page is not a change. A new path or key, or rows: {}, starts the records again (the next answer is their baseline).

  • GET /monitors/{id}/rows?active=&q=&limit=&offset= — its records as they stand: {key, attrs, url, firstSeen, lastSeen, active, withheld, misses, sources}; active=false lists the records gone. q searches the records' values (not their column names), in any case; total counts its matches, and truncated: true says the search stopped at its scan limit (100,000 records), so total is of those it read. A monitor that keeps no records (a search monitor, or an agent one without rows) answers 409 no_rows on every rows route.
  • GET /monitors/{id}/rows/events?run_id=&kind= — what moved, newest first; kind is added, changed, gone or withheld, run_id a monitor run's id.
  • GET /monitors/{id}/rows/{key} — one record, its sources, and every event that touched it.
  • GET /monitors/{id}/rows/export?dataset=rows|row-events&format=csv|jsonl — as a file.

With a destination (any of the workspace's), each run's records — the ones it saw or called gone — are written there after the run: monitor_id, monitor_name, key, label, state (active / gone), change (added, changed, gone, same, or baseline on the first), first_seen, last_seen, run_id, and record and sources as JSON text. An upsert destination keeps one row per monitor and record; an append one gets each run's. The write is on the destination's history and on the run (destinationSync).

  • GET /monitors?kind= — the workspace's monitors, newest first, each with lastRun, lastRunAt, nextRunAt, its budget ({perRun, perMonth}, 0 is none) and spentThisMonth. At most 200 per workspace.
  • GET /monitors/overview?kind=crawl|search|agent — everything on a schedule in one list: crawl projects with a schedule (a manual project is not listed) and every monitor, {data, counts}. Each row is {kind, id, name, detail, schedule, status, health, nextRunAt, lastRunAt, lastRun: {at, status, summary, changed, baseline, withheld, error}, skipped, spentThisMonth, estimate: {perMonth, upTo, capped, basis}, caps: {perRun, perMonth}, alerts: {webhook, email, chat}} — detail is the project's host or what the monitor asks, estimate about what a month costs held to the monthly cap, alerts.chat the chat service or "". Active first, soonest next run first; paused last. A key held to some projects sees those, and no monitors.
  • GET /monitors/{id} · PATCH /monitors/{id} {name?, schedule?, status?, webhook?, budget?, alerts?, addToProject?, rows?, destination?, sentiment?} — status is active or paused; webhook: "" removes it; budget and alerts are replaced whole ({} removes them); addToProject: "" stops feeding the project; rows: {} stops following records (and forgets them); destination: "" stops writing them.
  • POST /monitors/{id}/pause · POST /monitors/{id}/resume — a paused monitor is not scheduled; a run under way finishes. Resumed, a slot that passed meanwhile runs once.
  • POST /monitors/{id}/run — run it now, paused or not; the schedule counts on from this run. 409 while a run is under way.
  • GET /monitors/{id}/runs?limit=&cursor= — its runs, newest first: {id, monitorId, kind, trigger, status, refId, credits, answered, changed, summary, diff, error, loginNotices, createdAt, finishedAt} (loginNotices: an agent monitor's run, the saved logins it could not use). refId is the run's own search or agent run (GET /search/{refId}, /agent/{refId}, kept 7 days). The comparison is kept with the run, for the last 200 runs.
  • DELETE /monitors/{id} — the monitor and its runs; answers {id, deleted: true}. The searches and agent runs it made stay in your history; an agent run under way is stopped.

POST /triggers — a change or a new result starts work#

{"name": "Pricing watch", "source": {"kind": "project", "id": "<a project id>"},
 "when": {"kinds": ["modified"], "paths": ["/pricing*"], "minWords": 10},
 "action": {"type": "agent", "prompt": "This page changed: {{url}}
{{changes}}
What changed, and does it matter?", "maxCredits": 300},
 "limits": {"cooldownMinutes": 60, "maxPerDay": 10, "perMonth": 2000}}

A trigger watches a project's change records or a monitor's runs, and when a finished run matches when it does one thing (action.type):

  • agent — starts an agent run from prompt, a template: {{url}}, {{before}} and {{after}} are the first change's, {{changes}} lists every change (a line each), {{source}} is the project's or monitor's name. The page text ({{before}}, {{after}}, {{changes}}) is not spliced into the instruction: the prompt names it, and it follows in <page-data> blocks marked as data, which the agent is told never to take as instructions. It starts on the changed pages and is held to the project's site (a monitor's: the changed pages' sites) unless allowedDomains says otherwise; when no site can be told, the fire is skipped rather than run with no limit (allowedDomains: [] asks for no limit on purpose). maxCredits (300 by default), maxSteps, schema and connectionId are as POST /agent takes them, checked when the trigger is made.
  • recrawl — re-crawls the pages that were added or changed (a project's).
  • monitor — runs a monitor now (monitorId).
  • recheck — for a project's change: re-checks only the records of an agent monitor followed as rows (monitorId) whose value was read on a changed page. Its prompt is narrowed to those records and its answer applied to them alone — the records it did not look at are not missing. A change no record was read on starts nothing.

when, for a project: kinds (added, modified, removed; all by default), paths (globs: /pricing*), fields (head fields such as title or canonical — a field change rather than a page change), minWords (words added and removed on a changed page) and significant (only when the change digest called the change significant). For a monitor: events — a search's new_result, dropped_result, moved_result; an agent's values or records added, changed, removed. Baselines and re-baselines never fire.

Each finished run fires a trigger at most once, read about a minute after it ends, oldest first. A trigger looks from when it was made (or resumed): what came before is not news. A run that a trigger started never fires a trigger itself, so triggers do not chain. limits: cooldownMinutes between fires (60), maxPerDay (10, in any 24 hours), perMonth credits (what the runs it started cost this calendar month; 0 is no cap — an agent action's maxCredits is held to what is left). A match a limit holds back is recorded as a skipped fire, saying why. Runs a trigger started say so: triggeredBy on a monitor run, TriggeredBy on the agent or crawl run.

  • GET /triggers?source_id=&target_id= — triggers, newest first, each with its when, action, limits, spentThisMonth and lastFire.
  • GET /triggers/{id} · PATCH /triggers/{id} {name?, when?, action?, limits?, status?} — status is active or paused; a limit left out keeps its value.
  • GET /triggers/{id}/fires — its firing history, newest first: {at, status (fired, skipped, error), reason, sourceRunId, matched, items, refKind, refId, credits}; refKind is agent, crawl or monitor_run. Kept 90 days.
  • POST /triggers/{id}/preview — what the source's latest run would match, and the prompt an agent action would send; fires nothing.
  • DELETE /triggers/{id} — the trigger and its history. Deleting a project or a monitor deletes the triggers that watch it or run it.

GET /proposals — what agents proposed to watch#

An agent run started with "proposals": true may propose a crawl project, a search monitor or an agent monitor. Each proposal is checked when it is made — a private address, a site the workspace already crawls, a search or question a monitor already keeps, the same thing already waiting, more than 5 from one run — and the agent is told why one was refused.

{"data": [{"id": "…", "kind": "project", "name": "acme.test", "what": "https://acme.test/", "schedule": "daily",
  "reason": "You asked to keep an eye on Acme's site.", "estimate": {"perRun": 100, "perMonth": 3000, "upTo": true, "basis": "…"},
  "spec": {"seed": "https://acme.test/", "name": "acme.test", "schedule": "daily", "config": {"max_pages": 100}},
  "runId": "…", "runPrompt": "…", "status": "proposed", "made": null, "createdAt": "…", "decidedAt": null}]}

status=proposed (the default) lists those waiting for a decision, status=approved the approved, status= all; run_id keeps one run's. spec is what approving makes, as POST /projects or POST /monitors would take it. Proposals are kept 30 days. A key held to some projects sees none.

  • POST /proposals/{id}/approve — member and up. Makes it: the project (as POST /projects makes one: its pages are discovered now, and it crawls at its first scheduled time — the next 03:00 UTC for daily — or when run; 402 at the workspace's project limit), or the monitor (its first run starts now; 402 with no credits left). Answers {proposal, made: {kind: "project" | "monitor", id}}. Once: 409 when it was decided already.
  • POST /proposals/{id}/dismiss — member and up. Deletes one still waiting; 409 for one approved.

GET /estimate — about what scheduled work costs a month#

?kind=crawl|search|agent&schedule=hourly|daily|weekly|manual, and what to price: projectId or monitorId for one that exists, or for one not made yet maxPages and maxTier (a crawl) or maxCredits (an agent). Answers {kind, schedule, perRun, runsPerMonth, perMonth, capped, cap, upTo, basis}: runs a month (hourly 730, daily 30, weekly 4.35) times what a run costs, held to monthCap (or the project's or monitor's own monthly cap). Something that has run is priced at the average of its last three runs; otherwise at the most a run could cost (upTo: true): a crawl's pages at its highest tier's price, a rendered results page (4 credits; 1 when DuckDuckGo or Bing answers), an agent run's budget. basis says how, in words. Free, and nothing is saved.

POST /playground/extract — one URL with every format, for the app#

{url, allow_browser, use_proxy, config} → an id; GET /playground/{id} for the result and its evidence; …/screenshot for the image.

Projects#

RouteWhat
GET /projects · POST /projects {seed, name?, schedule?, retention?, config?} · POST /projects/batch {seeds[], template_project_id?, schedule?, retention?}List; create one (schedule defaults to manual here — the app sends weekly); create many from a seed list with a template's config
GET /projects/{id} · PATCH /projects/{id} {name?, schedule?, retention?, config?, paused?} · DELETE /projects/{id} (admin)Read; change anything — paused: true stops its scheduled runs and keeps the schedule (nextRunAt is null meanwhile; resumed, a slot that passed runs once; Run now still runs); delete with its runs and pages
POST /projects/{id}/archive · POST /projects/{id}/restore (member)Archive: it stops running, no longer counts toward the plan's projects, and is deleted after 30 days (purgeAt) unless restored; a run of it answers 409 archived. What a key without admin -- a connected app -- does in place of a delete. Restore brings it back, its schedule as it was
POST /projects/{id}/config/apply {project_ids[], groups[]}Copy this project's saved config onto others, by group (scope, fetch, content, output, …)
POST /projects/run {project_ids[]}Start several
POST /projects/{id}/dry-runResolve the sources and sample URLs without spending
GET /projects/{id}/sources · POST …/sources/preview {sitemap_include, sitemap_exclude, include_paths, …}The seed, sitemap, URL list, feeds and patterns with what the last run found through each; how many URLs a selection keeps
GET /projects/{id}/discovery · POST …/discovery · GET …/discovery/entries?file=&cursor=The stored sitemap tree (files, sections, counts, lastmod ranges, the diff against the last snapshot); take a new snapshot (a map, billed); page through a file's entries
POST /projects/{id}/selectors/match {selectors[]} · POST …/content/preview {config}Match counts against the last run's stored HTML; the markdown a config would produce for the seed

Runs and pages#

RouteWhat
GET /projects/{id}/runs?limit=&cursor=&v= · POST /projects/{id}/runs {trigger?} · GET …/runs/{run} · POST …/runs/{run}/cancelHistory (25 a page); start (202); one run with its counts, credits, stop reason, engine breakdown, log; cancel
GET /projects/{id}/pages?run_id=&v=&flag=&cursor=&limit=Rows of a run (100 a page, 1,000 at most). v filters by verdict; flag=orphan|unlisted|dead_end. The app's query language (status:403 depth:>2) is applied on the rows
GET /projects/{id}/pages/content?url=&run_id=&format=One page's bodies, head, fields, versions
GET /projects/{id}/pages/screenshot?url=&run_id=The JPEG
POST /projects/{id}/pages/search {q, mode: content|selector, run_id?}Which pages say this (words, "phrases") or contain this (CSS / XPath), scanned on the server
POST /projects/{id}/pages/recrawl {urls[]}A scoped run of just these, now
GET /projects/{id}/pages/removed?run_id=Pages in an earlier run and not this one
GET /projects/{id}/runs/{run}/links?kind=orphans|unlisted|dead_endsThe link graph's findings

Changes#

RouteWhat
GET /projects/{id}/changes?run_id=The change record: {counts: {added, removed, modified, same, withheld, unverified, skipped, fields, selectorBreaks, volatileBlocks}, coverage, baseline, entries[]}
POST /projects/{id}/changes/{change}/review {reviewed: true}Mark reviewed
GET /projects/{id}/changes/page?url=&run_id=The word-level content diff of one page against the run before
GET /projects/{id}/changes/markup?url=&run_id=The markup comparison: DOM diff, noise filter, selector repointing

Exports#

GET /exports lists what can be exported. GET /projects/{id}/export?dataset=pages|markdown|changes|fields|sitemap&format=jsonl|csv&run_id= streams a dataset; POST /projects/{id}/export {dataset, format, urls[]} restricts it to listed URLs.

Two more datasets are text files, for a model rather than a table: dataset=llms is an llms.txt — the run's usable pages (ok, thin) as links with their descriptions, grouped by first path segment, under the site's title and the home page's description — and dataset=llms-full is the llms-full.txt: every usable page's markdown in one file, each under its title and a Source: line. Both take format=txt (the default for them) and the same urls[] selection.

Connections and destinations#

RouteWhat
GET /connections/providersThe catalogue with each provider's fields, secrets, note and verified
GET /connections · POST /connections {provider, name, config, secrets} · GET/PATCH/DELETE /connections/{id} · POST /connections/{id}/testWorkspace connections; secrets are never returned
GET /destinations · POST /destinations · PATCH/DELETE /destinations/{id} · GET …/syncs · POST …/syncShared destinations (scope all or projects) and their syncs
GET /projects/{id}/destinations · POST … · PATCH/DELETE …/{dest} · GET …/{dest}/syncs · POST …/{dest}/syncA project's own destinations
POST /destinations/preview {project_id, shape, columns} · GET /destinations/shape-functionsThe shaped rows of the last run; the function and operator lists

What goes in a destination, and how a shape is written, is on the Connectors and destinations page. A destination is written to a warehouse, database, storage, index or stream connection: one on a model (llm) or a data provider is a 400.

Webhooks#

Project webhooks — webhook_url, webhook_events and webhook_include in the config; GET /projects/{id}/webhooks (the events, recent deliveries, and the signing secret — to an admin or owner; anyone else gets secretHidden: true), POST …/webhooks/test (a run.finished with test: true), POST …/webhooks/secret (rotate, admin).

Subscriptions — a project's events sent to one more URL, beside its own webhook and without touching it: how an app such as Zapier or n8n turns them into a trigger, subscribing when a zap or workflow is switched on and deleting the subscription when it is switched off. POST /webhooks/subscriptions {projectId, url, events?, label?} (member) → 201 {subscription: {id, projectId, url, events, label, createdAt, secret}, created: true}. events are the project events below (page.changed when left out); label names who holds it in the list. Each subscription has a secret of its own, returned here only, and its messages are signed with it exactly as the project's own are. The same URL on the same project again returns that subscription (200, created: false) with the events now named. 400 on a URL that is not public http(s) or an event the project does not send; 409 subscription_limit past 25 on one project. GET /webhooks/subscriptions?projectId= lists the workspace's, or one project's, newest first and never with their secrets; DELETE /webhooks/subscriptions/{id} (member) → 204 stops it, dropping its messages still waiting. A receiver that answers 410 Gone is taken as having removed the hook: the subscription is deleted. Deleting the project deletes its subscriptions.

Job webhooks — the webhook on POST /crawl and webhook_url on a batch /scrape, with their own secret returned once. Events crawl.started, crawl.page (gathered into one message per 50 pages, or every 5 s), crawl.completed, crawl.failed, scrape.completed, scrape.failed, map.completed, batch.finished; metadata comes back on every message. An agent run's webhook (POST /agent) has its own five: agent.started, agent.action, agent.completed, agent.failed, agent.cancelled. A search's webhook (POST /search) is told once how it ended: search.completed or search.failed. A monitor's webhook (POST /monitors) hears search.changed or agent.changed, once a run that changed something, budget.reached once a month when its monthly cap skips a run, and — an agent monitor that follows its answer as rows — row.changed with the records that moved.

Page markdown in the message — off unless asked. "webhook_include": ["markdown"] in a project's config adds each added or modified page's markdown to its page.changed; "include": ["markdown"] in a crawl's webhook ({"url": …, "events": ["crawl.page"], "include": ["markdown"]}) adds each page's to crawl.page. A message carries up to about 1 MB of markdown; a page past that, or one not stored, has markdown: null, markdownOmitted (too_large or not_stored) and a contentUrl — the API path that returns it in full (/projects/{id}/pages/content?url=&run_id=, or /crawl/{id}/page?url=), called with your key. A removed page has none. Subscriptions are sent the message without it. The delivery log (GET /webhooks/deliveries) shows [sent in full] in place of each page's markdown; the receiver is sent it whole, signed as every message is. A crawl's crawl.page deliveries that carried markdown are kept 7 days (a one-shot crawl that is not kept takes its deliveries with it when it expires).

Project events — run.finished (status, pages, counts, coverage, stop), page.changed (every page added / modified / removed with words added and removed, the section, whether it was volatile; a modified page only when its edit reaches notify_min_words), page.blocked, field.canonical (canonical, robots and title changes), site.failing / site.recovered, budget.reached (a scheduled run skipped because the project's max_credits_per_month is spent; once a month), page.orphaned / page.linked, sitemap.added / removed / updated / file_added / file_removed.

The digest — with change_digest: {connection_id, focus} in the config (an llm connection of the workspace; focus is what the reader cares about, e.g. "pricing and plan limits"), a compared run with something in it asks the model for a paragraph on what changed and why it matters, up to five points and a significant flag. It rides on run.finished as data.digest ({summary, points[], significant}, or null), in the run email, in the chat message, and on the change set (GET /projects/{id}/changes → digest). Tokens are charged like enrichment, through your own key.

Chat channels — chat_url in the config is a Slack, Discord, Microsoft Teams (Workflows), Mattermost or Google Chat incoming-webhook URL. When a run finishes with something to say — the same rule as the run email: never for a run that compared clean — the record is posted there in that service's own shape (Block Kit, an embed, an Adaptive Card, or {text}), unsigned, through the same delivery queue and retries as a webhook; site.failing / site.recovered post too. The URL is a secret to the channel, so the API returns it masked (https://hooks.slack.com/services/••••), and a masked value sent back in a PATCH keeps the one stored; send "" to remove it. GET /webhooks lists the channel with its health beside the project's webhook (kind: "chat", service).

The message — POST, JSON {id, event, createdAt, data}, headers X-MeshArc-Event, X-MeshArc-Delivery, X-MeshArc-Signature: sha256=<HMAC-SHA256 of the raw body with the secret>, User-Agent: MeshArc-Webhooks/1.0. Answer 2xx within 10 s.

Retries — no answer, a 5xx, a 408 or a 429 is retried six times over about a day (1 m, 5 m, 30 m, 2 h, 6 h, 12 h), then the delivery is paused. Any other 4xx is your endpoint's answer — a wrong URL, a rejected signature — and the delivery is failed at once. GET /webhooks/deliveries?runId=|projectId=&limit= lists what was sent, attempt by attempt (status, attempts, lastStatus, lastError, nextAt, the log); POST /webhooks/deliveries/{id}/redeliver sends one again now with its attempts kept. GET /webhooks gives every project's webhook with its health.

The workspace#

RouteWhat
GET /me · PATCH /me {max_rpm} (admin)The workspace, its plan and limits, credits used and remaining, what this key may do; the rate ceiling
GET /me/usage · GET /me/monitorPages per day and this month's pages, runs, blocked share, pages by engine; what is queued and running, lane depths
GET /me/keys · POST /me/keys {name, scopes[], projects[], expires_in_days, rpm} (admin) · DELETE /me/keys/{id}Keys; the secret is in the create response only. Leave projects out for a key to the whole workspace; an empty list is refused (400), never read as that. A key's rpm only lowers the plan's, and is shown as the limit it meets
GET /me/billing · POST /me/billing/subscribe · /change · /confirm · /card · /cancel · /keep · /credits/buy · PATCH /me/billing/cap · GET /me/billing/invoices · /receipts · /credit-notes/{id}The plan, the meter, the ledger, the price list; subscribing, changing plan and card through Razorpay Checkout (confirm verifies Checkout's answer), cancelling and undoing it, buying credits, the overage cap; invoices, receipts and credit notes
GET /me/notifications · POST /me/notifications/{id}/seenSigned-in members only: while the server has no mail host (emailConfigured: false), the run records and monitor alerts email would have carried — {id, kind, subject, text, link, createdAt}, unseen, a week at most — and dismissing one. Empty once mail is set up, and for a key
GET /orgs/children · POST /orgs/children {name, monthlyCap, externalId} · GET /orgs/children/{id} · PATCH /orgs/children/{id} {name, monthlyCap, suspended, reason} · GET/POST /orgs/children/{id}/keys · DELETE /orgs/children/{id}/keys/{key} (admin)Sub-accounts: made, capped, suspended, given keys; one in full is its month, its spend by kind, its latest charges and which of your keys acted on it
GET /me/proxiesThe pool with per-exit parity and success
GET /metaVerdict meanings, engine costs, the config defaults, the ladder
/auth/*POST /auth/signup, /login, /logout, GET /session, POST /switch, /verify/request, /verify, /forgot, /reset, PATCH /profile, POST /password, GET/DELETE /sessions, GET /members, POST /invites, DELETE /invites/{id}, GET /invites/{token}, POST /invites/accept, PATCH/DELETE /members/{user}, PATCH /workspace, GET /sso, POST /sso/{provider}/start, POST /sso/{provider}/finish, GET /identities, DELETE /identities/{provider}

Sub-accounts#

For a platform that runs MeshArc for its own customers: a workspace of its own for each one, made by yours. A sub-account has its own keys, projects, runs, monitors and data, and sees nothing of your workspace or of its siblings; your workspace sees its spend and can act on it.

  • Plans. Sub-accounts come with Scale (25) and Enterprise (500, or what was agreed); on another plan POST /orgs/children is 402 plan_limit. A sub-account is on your plan — whatever your plan is now — with its limits, and has the plan's concurrency of its own. It cannot have sub-accounts. Moving to a plan without them keeps the ones you have; it stops you making more.
  • Credits. A sub-account has no credits of its own: it spends yours, and what it spends is on your bill — your month's counters.credits includes it, and counters.pool says how much of it was your sub-accounts'. Give one a monthlyCap and it stops at that many credits a month, whatever you have left; with none (0) it may spend what you have. Run out, or reach its cap, and the next job it starts is refused with 402, saying which; what it has already stored stays readable. Its own GET /me/billing says who pays (month.billedTo) and refuses the billing actions — plan, card, credits, overage cap — with 409 sub_account.
  • Keys. POST /orgs/children/{id}/keys makes it a key, shown once, for your customer: it reaches that sub-account and nothing else. Or act on it with one of your own keys and the X-MeshArc-Tenant header (Authentication).
  • Suspending. PATCH {"suspended": true, "reason": "…"} refuses its keys and your header, and cancels what it has queued or running; false lifts it. One that MeshArc suspended, or whose parent is suspended, is stopped: not yours to lift.
  • Deleting. Not offered: suspend one instead. Its data follows the retention of its projects, as any workspace's does.
# Make one for a customer, at most 50,000 credits a month
curl -X POST https://api.mesharc.dev/api/v1/orgs/children \
  -H "Authorization: Bearer $MESHARC_KEY" -H "Idempotency-Key: cust-42" \
  -d '{"name": "Acme Ltd", "monthlyCap": 50000, "externalId": "cust-42"}'

# Scrape on its behalf, with your own key
curl -X POST https://api.mesharc.dev/api/v1/scrape \
  -H "Authorization: Bearer $MESHARC_KEY" -H "X-MeshArc-Tenant: <its id>" \
  -d '{"urls": ["https://example.com"]}'

Saved logins#

A workspace's signed-in session on a site, saved from a live browser, used by the projects and scrapes that name it (login_id in their config) and by agent runs, which sign in with one where a page on its site needs the account (logins on POST /agent and POST /agent/batch says which they may use). Switched on per deployment: while it is off, GET /logins says so (enabled: false), saving or testing one answers 409 logins_off, and a project, scrape or agent run that names one is refused (400). Owners and admins only.

RouteWhat
GET /logins{data, enabled, why}: each login's id, host, name, status (active or expired), how many cookies it holds, who saved it and when, the terms they accepted, uses and lastUsedAt, and why it expired. Never the session
PATCH /logins/{id} {name, expired}Rename it; expired: true stops using it now
DELETE /logins/{id}Delete it. Projects and runs that name it stop, saying so; none reads the site signed out instead
POST /logins/{id}/testOne plain fetch of the site's front page with the login: {status, finalUrl, loginPage, reason}. Free
GET /logins/termsThe terms a person accepts before saving one, and whether logins are on
POST /logins/live {host, name, acknowledged, renew?}For a person signed in to the app (not a key): a single-use ticket, good for a minute, and the wsUrl of the live browser's socket. renew is the id of the login the sign-in saves again (it must be this workspace's, for that site)

How a login is used. Only on its host and that host's subdomains: its cookies go where a browser would send them — one the site set for its own host stays on that host, never sent to a subdomain or across a redirect to another host — and a page rendered with it gets a browser of its own. A login is for a site's name: an IP address, or a name many unrelated sites share (co.uk, github.io), is refused. A run that meets the site's login page with it marks it expired (status: expired, expiredReason) and emails the workspace's owners and admins once; runs that name it then fail until it is saved again — and signing in again to that login (Sign in again, or renew with its id) renews it, under the same id, so the projects, monitors and runs that name it work again. Without renew, a save renews the site's expired login only when the site has no other, or when it is given that login's exact name; otherwise it is a new login, so one account's session is never saved under another's. A key held to some projects cannot list or change the workspace's logins (403 project_key).

The live browser's socket. Open wsUrl from the app's own pages. The server sends {"type": "frame", data, width, height} (a JPEG of the page, base64), {"type": "page", url, title} after each step, {"type": "saved", login}, {"type": "error", message} and {"type": "closed", reason}. Send {"type": "click" | "dblclick", x, y} (in the frame's pixels), {"type": "text", text}, {"type": "key", key} (Enter, Tab, Shift+Tab, Backspace, Escape, the arrows, Home, End, PageUp, PageDown), {"type": "scroll", dy}, {"type": "navigate", url}, {"type": "back"}, {"type": "reload"}, then {"type": "save", name} once signed in — or {"type": "close"}. A save keeps the site's own cookies and local storage, encrypted; the password typed into the site's form is never kept. The browser opens only public sites: an address on this deployment's own network — typed, linked or loaded by a page, a page's WebSocket included — is never reached, and every connection is judged on the address it is made to, so a name cannot answer one address to the check and another to the browser. The browser closes after a few minutes, or sooner with no input; one per person.

Data providers#

Company and financial data the agent looks up on the workspace's own key, from two providers. Connect one as any connection is (POST /connections, {"provider": "pdl" | "fmp", "secrets": {"api_key": "…"}}; owners and admins), then name it on a run with dataConnections: POST /agent, a follow-up, POST /agent/batch, or an agent monitor's request. The run is given that provider's tool; a run naming none has neither.

ProviderThe agent's toolWhat it looks up
People Data Labs (pdl)company_dataOne company, matched by its website (best), LinkedIn page or ticker, or loosely by name and country: name, size and employee count, industry, type, founding year, headquarters, LinkedIn, summary and tags, funding raised, last round and its date, inferred revenue. With PDL's likelihood (1–10) of the match
Financial Modeling Prep (fmp)financial_dataA listed company: search (a name → its ticker symbols), profile, quote, and income, balance, cashflow and metrics (annual or quarterly, newest first, up to 5)
  • What it costs: no MeshArc credits. The provider counts each lookup on your plan — People Data Labs bills a match (a lookup that matches nothing is free), and Financial Modeling Prep counts a call. A run makes at most 25 lookups, and the same lookup twice in a run is one call.
  • Where a value came from: each record found is one of the run's sources, {url, title, provider} — url is the request it came from (https://api.peopledatalabs.com/v5/company/enrich?website=stripe.com), for fieldSources to point to. Your key is sent in a header and is never in a source, the trace, the answer or a webhook.
  • In the trace: each lookup is a lookup step with the provider, what was looked up, found, likelihood (PDL) or the error, and 0 credits.
  • When it cannot answer: a refused or spent key, a plan that does not include the data, or the provider limiting requests is the tool's answer, said plainly; the run goes on with what it has. A connection deleted after a run was asked for leaves that run without its tool, and its trace says so. A 429 is waited out once (up to 10 s).
  • Testing a connection: POST /connections/{id}/test makes one request: People Data Labs' free Autocomplete, or one Financial Modeling Prep symbol search (one of your plan's calls).

Both are built to the providers' current documentation and marked unverified until run against a live account.

What a page row says#

When a limit stopped any page of an answer, the crawl, scrape, batch and search answers also carry notices: one line per reason, with pages (how many), needs, up to five examples, unlockedBy, upgradeUrl and a message a script can print as it is. A search's notice also names the engine the limit kept from answering, and says which engine answered instead. The workspace's owners and admins are emailed too: when 80% of the free plan's stealth trial is spent and once when it runs out, and at most once a week when a run has pages the plan, the trial or the credits left stopped. A cap the workspace set itself (a project's or request's max_tier or max_credits_per_page) is told in the answer, not emailed, and its notice has no unlockedBy; a search's notices are in its answer only.

FieldValues
verdictok, thin (short but a page; kept), blocked (refused, or a 404 / soft 404 — see errorCode), skipped (under the word floor or outside the language list; the row stays, out of the change record), robots, cached
errorCodeOK, SHELL, BLOCKED, NOT_FOUND, RATE_LIMITED, CAPTCHA (the page, rendered or not, is still a widget to solve; reason names it: cloudflare_turnstile, recaptcha, hcaptcha, datadome_captcha, perimeterx_captcha, arkose_funcaptcha or captcha), LOGIN_REQUIRED, GEO_RESTRICTED, TIER_LIMIT (a higher tier than allowed would have read it), BUDGET_EXHAUSTED (the credits the workspace had left could not buy the engine that would have read it), TIMEOUT, NETWORK, DNS_FAILED (the site's name does not resolve: the page's status is unreachable, where a page the site says it does not have is missing). A short page is OK with a warning
limitSet when a limit, not the site, stopped the page (TIER_LIMIT, BUDGET_EXHAUSTED); otherwise null. needs (the tier it needed), reason (stealth_trial_used, tier_ceiling or credits_left), ceiling (the tier it was held to), unlockedBy (the plan that includes it; null for credits and for a cap the workspace set itself), upgradeUrl, a plain message, and outlook {tier, likely, why}: whether that tier is likely to read the page (yes, no or unknown) and why — this site's own record on it, or what the wall is (Cloudflare's check, which a browser usually passes; a captcha, which needs the solver)
shapelisting, table, form, or empty — what a short page's markup says it is
warningsshort; no_main_content on a page that began as an app shell and rendered to its site's frame alone (header, footer, menus, an accessibility panel, a closed dialog): kept as thin, its reason saying so
signalsWhy the judge decided as it did: nav, 30 links, json-ld, open graph, main, no script, app shell, listing, render confirmed, soft 404, host fingerprint, soft404?
method, tier, climbedToThe engine that answered and its tier; the highest engine the page was tried on when it had to climb
attemptsEvery engine the climb tried, cheapest first: {engine, ok, verdict, reason, ms}, ms being how long it took. Empty when the first engine answered. A captcha or Cloudflare's "Just a moment" check skips the plain http engines, which no client gets past (tierSkipped says so)
tierSkippedEach engine the climb passed over, with why: {"browser-residential": "no residential exit is configured", "solver": "no token solver is configured", "unblocker": "not within the browser tier"}. A page refused with a stealth engine unused says here why it was not used
credits, billedAsWhat the page cost; the rung it is priced at when not the one that fetched it
widget, solved, exitBytesA captcha widget on the page and what was done; bytes through the residential exit
foundVia, firstReferer, inSitemap, inLinks, orphan, unlistedHow the crawl reached it and what the link graph says