Overview — your side of it

AISA is a Pay-per-Crawl platform. Publishers price their deep content per URL directory; crawlers pay per request from a prepaid balance, signalled by the HTTP 402 Payment Required standard. As a crawler, your world has exactly two planes:

YOU (crawler / agent) PUBLISHER SITE AISA (this host) | (fronted by an edge gateway, | e.g. Alibaba ESA) | GET /research/report.data.json | | X-AISA-Crawler-Token: aisa_crawler_... | | User-Agent: <your genuine UA> | +------------------------------------------>| | |<-- the edge authorizes & bills your | | token with AISA (internal -- you | | never see or call it) |<------------------------------------------+ | 200 content | 402 payment JSON | 403 blocked JSON | | ACCOUNT PLANE -- the only APIs you call directly, on this host: +--> POST /api/agent/auth/register create account, get crawlerToken (+ possible trial credit) +--> POST /api/agent/auth/login recover token, check balance +--> GET /api/agent/balance real balance, using your crawler token (one call, same credential) +--> POST /api/agent/topup/create-session fund balance via Stripe +--> GET /api/agent/info account details (session auth)
Golden rule: you never call any AISA verification API. Authorization and billing are performed by the publisher's edge gateway on your behalf, per request. If you have seen documentation about a /api/v1/access/verify endpoint — that is gateway-and-publisher-side integration material (see /cdn/docs.html), not something a crawler uses. Your entire crawling contract is: send your token header and your honest User-Agent; read what comes back.

Key Concepts

ConceptDescription
Crawler Token Your crawl credential. Format: aisa_crawler_<64 hex>. Sent as header X-AISA-Crawler-Token on every request to publisher sites. Obtained at registration.
Session Token Short-lived token from register/login, used only for account APIs on this host (Authorization: Bearer <session>). Never used for crawling.
Balance Prepaid USD balance. Each granted access to a priced resource deducts its per-request price. New accounts may start with a trial credit.
402 Response Payment-side action needed. The JSON body is self-describing: error says why, guidance says exactly what to do next.
403 access_blocked The publisher forbids crawler access to this resource. Payment does not unlock it.
guidance Machine-readable block inside every denial: onboarding/docs/llms.txt/skill links, the register API, numbered steps, and (when active) the trial-credit notice.

Quick Start (3 Minutes)

1

Register

Create an account via API. No email verification. You receive a crawlerToken immediately — and possibly a trial credit (signupBonus).

2

Fund (if needed)

If a trial credit was granted you can start right away. Otherwise top up via Stripe (minimum $0.50).

3

Crawl

Request publisher URLs with ONE extra header: X-AISA-Crawler-Token. The edge handles payment; you get content or a self-describing denial.

Complete Example

bash
# Step 1: Register — SAVE crawler.crawlerToken from the response
curl -s -X POST "https://cdn.aisa.one/api/agent/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "mybot@example.com",
    "password": "MySecurePass123",
    "name": "MyBot",
    "crawlerType": "custom"
  }'
# → { "crawler": { "crawlerToken": "aisa_crawler_...", "balance": 0.10 }, "signupBonus": {...}? }
# Trial credit present? Skip straight to Step 3.

# Step 2: Top up when needed (returns a Stripe checkout URL — open it to pay)
curl -s -X POST "https://cdn.aisa.one/api/agent/topup/create-session" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer SESSION_TOKEN_FROM_REGISTER_OR_LOGIN" \
  -d '{ "amount": 5.00 }'

# Step 3: Crawl any AISA-protected publisher URL — one extra header
curl "https://PUBLISHER-SITE/research/report.data.json" \
  -H "X-AISA-Crawler-Token: aisa_crawler_..." \
  -A "MyBot/1.0 (+https://example.com/bot)"

# Step 4: Verify what it cost
curl -s -X POST "https://cdn.aisa.one/api/agent/auth/login" \
  -H "Content-Type: application/json" \
  -d '{ "email": "mybot@example.com", "password": "MySecurePass123" }'
# → crawler.balance
If a 402 response carries guidance.registerApi pointing at a different AISA host, prefer that host — it is always correct for the site you are crawling.

Registration API

Create a new crawler account programmatically. No human intervention required.

FieldTypeRequiredDescription
emailstringYesValid email address (used as login ID). Already registered → 409, use login
passwordstringYesMinimum 8 characters
namestringNoDisplay name for your crawler/agent
crawlerTypestringNoYour real identity if you have one: gptbot, perplexitybot, amazonbot...; else custom (default)
userAgentstringNoYour crawler's User-Agent string (used for identification)
companyNamestringNoOrganization name
bash
curl -s -X POST "https://cdn.aisa.one/api/agent/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "bot@example.com",
    "password": "SecurePass888",
    "name": "MySearchBot",
    "crawlerType": "custom",
    "userAgent": "MySearchBot/1.0",
    "companyName": "Acme AI Labs"
  }'

Success Response

json
{
  "success": true,
  "token": "eyJjcmF3bGVySWQ...",       // Session token (account APIs / dashboard — NOT for crawling)
  "crawler": {
    "id": 42,
    "crawlerUuid": "crawler-a1b2c3d4-1712345678",
    "email": "bot@example.com",
    "name": "MySearchBot",
    "crawlerType": "custom",
    "crawlerToken": "aisa_crawler_abc123def456...",  // Your crawl credential — store securely
    "balance": 0.10,                                 // non-zero when a signup trial credit is active
    "totalSpent": 0
  },
  "signupBonus": {                                   // present only when a trial credit is active
    "amount": 0.10,
    "currency": "USD",
    "note": "Trial credit granted — you can access paid resources immediately ..."
  }
}
Trial credit: when active, new accounts start with a small balance (e.g. USD 0.10 — enough for 5–20 paid fetches at typical prices), so you can complete the whole paid loop before ever touching a payment form. The field is dynamic: rely on the register response, not on documentation.

Login (Existing Account)

bash
curl -s -X POST "https://cdn.aisa.one/api/agent/auth/login" \
  -H "Content-Type: application/json" \
  -d '{ "email": "bot@example.com", "password": "SecurePass888" }'
# → { "token": "<session>", "crawler": { "crawlerToken": "...", "balance": 4.98 } }

Two Tokens — Don't Mix Them Up

 Crawler tokenSession token
Looks likeaisa_crawler_<64 hex>opaque base64
Where you get itcrawler.crawlerToken in register/login responsetoken in register/login response
What it's forCrawling publisher sites — header X-AISA-Crawler-TokenAccount APIs on this host — header Authorization: Bearer <session> (top-up, info)
LifetimeStable (until account reset)Expires; re-login to refresh
Never send your crawler token in Authorization: Bearer — that channel is reserved for other purposes and your request will be treated as unauthenticated (402 invalid_token). The crawler token travels only in the X-AISA-Crawler-Token header. And never print the full token in logs or reports — show a prefix like aisa_crawler_9a4d....
Token format: aisa_crawler_ followed by 64 hex characters. Tokens do not expire but can be revoked from the dashboard.

Balance & Top-up

bash
# Real balance — ONE call, SAME crawler token you crawl with (no email/password, no session):
curl -s "https://cdn.aisa.one/api/agent/balance" \
  -H "X-AISA-Crawler-Token: aisa_crawler_YOUR_TOKEN"
# → { "success": true, "crawler": { "id": 123, "email": "...", "balance": 0.06, "totalSpent": 0.04, ... } }
# (also available in every login response's crawler.balance, or GET /api/agent/info with a session token)

# Top up via Stripe (minimum 0.50 USD)
curl -s -X POST "https://cdn.aisa.one/api/agent/topup/create-session" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer SESSION_TOKEN" \
  -d '{ "amount": 5.00 }'
# → { "checkout_url": "https://checkout.stripe.com/..." }
#   open it, pay, balance credits automatically
Always FETCH the balance — never quote it from memory. A remembered value (e.g. "the account has $0.10 from the signup bonus") goes stale the moment you spend. Call GET /api/agent/balance before reporting any balance, and especially before/after a top-up: post-top-up balance = (real current balance) + amount. Example bug to avoid: account already spent $0.04 (real balance $0.06), user tops up $100 → the answer is $100.06, not "$100.10". Reading balance with the crawler token is safe — that token can already spend, so reading is strictly less sensitive, and it keeps you on the ONE account you crawl and pay with (never re-register mid-task).
Autonomous agents don't fund themselves. Completing a Stripe payment needs a human — when the balance runs out, surface the checkout_url to your user and wait for them to pay. (POST /api/agent/balance/add exists only for a human operator to pre-seed a test account before a test run — it is not part of the crawl flow and an agent must never call it to top itself up; production has no such endpoint.)

Crawling: The One Header That Matters

http
GET https://PUBLISHER-SITE/any/path
X-AISA-Crawler-Token: aisa_crawler_YOUR_TOKEN
User-Agent: YourBot/1.0 (+https://example.com/bot)
You sendWhy
X-AISA-Crawler-Token Your identity + billing account. The edge gateway forwards it to AISA verbatim. Omit it and priced resources answer 402.
Your genuine User-Agent The gateway passes your original UA through for crawler identification and audit. Identify honestly: claiming another bot's UA gets you verified: false internally — publishers offering free passes to verified bots won't extend them to you (paid access unaffected).

Possible outcomes for any URL

HTTPMeaningCost
200 (free tier)HTML pages, .md summaries, discovery files, or directories the publisher opened to crawlers$0
200 (paid)A priced resource your token just paid for. Gateways may attach receipt headers: X-AISA-Charged: true, X-AISA-Charge-Amount: 0.02the per-request price
402Payment-side action needed — the body tells you exactly what (next section)$0 (denials never charge)
403 (blocked)Publisher forbids crawler access to this resource — payment does not unlock it$0
Under the hood (so your model is clear — no action needed): the site's edge gateway intercepts your request before origin, asks AISA to authorize your token against the publisher's pricing rules, deducts the price from your balance on success, and only then serves the content. One request from you = at most one charge. You never talk to that machinery directly.

Handling 402 / 403 Responses

Denial bodies are self-describing JSON. Check error first:

HTTPerrorMeaningYour action
402invalid_tokenNo / unknown / inactive tokenRegister or fix the header; retry the same URL
402insufficient_balancecrawler.shortfall = how much more you needTop up; retry. Tell your user the shortfall.
402price_required / price_mismatch / price_too_highPrice negotiation states (normally handled by the gateway; rarely seen)The quote is in price + currency
403access_blockedPublisher rule withholds this from crawlers — a policy choice, not a payment problemDon't retry or top up for this URL — but don't stop the task either: pivot to the site's purchasable pay-mode directories (its /pricing.json lists them) and explain the difference to your user in plain language

A real 402 (stranger crawler, no token)

json
{
  "accessGranted": false,
  "statusCode": 402,
  "error": "invalid_token",
  "message": "No crawler token provided. Send your registered token in the X-AISA-Crawler-Token header ...",
  "onboarding": "https://cdn.aisa.one/agent/guide.html",
  "guidance": {
    "onboarding":  ".../agent/guide.html",
    "docs":        ".../agent/docs.html",
    "llmsTxt":     ".../llms.txt",
    "skill":       ".../agent/skill.md",
    "registerApi": "POST .../api/agent/auth/register {\"email\",\"password\",\"name\",\"crawlerType\"}",
    "steps": [ "1. Register ...", "2. Already registered? ...", "3. trial credit / top up ...",
               "4. Retry this exact URL with header X-AISA-Crawler-Token" ],
    "signupBonus": "New accounts receive USD 0.10 free trial credit ...",   // present only when active
    "agentAdvice": "If you are an AI agent: narrate these steps to your user ... report whether the
                    content was free or paid, the exact amount charged, and your remaining balance."
  },
  "rule": { "urlPattern": "^/research/.*", "accessMode": "pay", "price": 0.02,
            "currency": "USD", "billingType": "per-request" },
  "crawlerIdentification": { "recognized": true, "identifier": "gptbot", "verified": true }
}
Follow guidance.steps verbatim — it is generated by the same system that will judge your retry, and its links always point at the correct AISA host for the site you are crawling. Field-tested: an agent with zero prior knowledge of AISA completes the full loop in 4 steps using only this block.

Pricing, Discovery & Budgeting

PrincipleDetail
Per-request billingEach granted fetch of a priced resource deducts that resource's price once. Fetch it twice, pay twice.
Publisher-set pricesPrices are per URL-directory, typically $0.005–$0.02 per resource. The exact price appears in rule.price of a 402 — price discovery is free (denials never charge).
Cost landscape firstMany publishers expose /pricing.json (directory → price/mode map) and describe tiers in /llms.txt. Fetch those first — they are free.
Free tier firstHTML pages and .md summaries are usually free; only structured deep files (.data.json, .full.md) are priced. Read summaries to decide what is worth buying.

Crawling a Whole Site or Directory

Real tasks are rarely one exact file. The proven strategy (also encoded in the agent skill):

StepWhatCost
1Discovery: GET /robots.txt, /llms.txt, /sitemap.xml, /pricing.json — map the content and the price/mode of every directory$0
2Free-tier survey: read .md summaries / HTML of candidate items$0
3Selective purchase: buy only what the task needs; keep a running total; confirm budget with your user before bulk buysper item
4Skip blocked directories (403) — no retries, no spend$0
5Stop on insufficient_balance: report the shortfall and surface the top-up option instead of hammering retries$0

Error Reference

HTTPerrorWhereMeaning
400missing_fields / validation messagesaccount APIsBad request body (e.g. password < 8 chars)
402invalid_tokencrawlingToken missing / unknown / account inactive
402insufficient_balancecrawlingBalance below resource price; see crawler.shortfall
402price_required · price_mismatch · price_too_highcrawlingPrice negotiation states (gateway-managed)
403access_blockedcrawlingPublisher blocks crawlers — payment does not unlock
409registerEmail already registered → use login
500internal_erroranyServer fault — retry later

Machine-Readable Resources

ResourceWhat it is
/llms.txtSite index for AI agents (both roles)
/agent/llms.txtThis whole flow as plain text with exact JSON — feed it to your model
/agent/skill.mdDrop-in agent skill (SKILL.md format): the full crawl-register-pay-report behavior, installable or readable as instructions
/agent/guide.htmlThe onboarding page 402 responses point to (human-readable + JSON-LD)
/agent/workflow.htmlEnd-to-end walkthrough with a live demo site, field-tested agent prompts, and skill comparisons

Playground — Live Account Demo

Runs against this host's real account APIs. Register a throwaway crawler, watch the trial credit arrive, check the balance. The playground never sends your crawler token anywhere except this host, and never initiates payment.

🧪 Try the Account APIs
Prefilled with a random throwaway address — change if you like

Code Examples

Python — crawl with auto-onboarding

python
import requests, os

BASE  = os.getenv("AISA_BASE_URL", "https://cdn.aisa.one")   # account APIs; prefer the host from 402 guidance
TOKEN = os.getenv("AISA_CRAWLER_TOKEN")
UA    = "MyCrawler/1.0 (+https://example.com/bot)"

def ensure_account(email, password):
    """Register (or login) and return (crawler_token, balance)."""
    r = requests.post(f"{BASE}/api/agent/auth/register", json={
        "email": email, "password": password, "name": "My Crawler", "crawlerType": "custom"})
    if r.status_code == 409:
        r = requests.post(f"{BASE}/api/agent/auth/login", json={"email": email, "password": password})
    d = r.json()["crawler"]
    return d["crawlerToken"], float(d["balance"])

def crawl(url, token):
    """Fetch a publisher URL; returns (status, body_or_denial_json, charged_amount)."""
    r = requests.get(url, headers={"X-AISA-Crawler-Token": token, "User-Agent": UA})
    charged = float(r.headers.get("X-AISA-Charge-Amount", 0) or 0)
    if r.status_code == 200:
        return 200, r.content, charged
    d = r.json()                       # denial bodies are JSON
    if d.get("error") == "insufficient_balance":
        print(f"[AISA] need ${d['crawler']['shortfall']} more — top up, then retry")
    elif d.get("error") == "access_blocked":
        print("[AISA] publisher blocks this resource; paying will not help — skipping")
    elif d.get("error") == "invalid_token":
        print("[AISA] token rejected — follow d['guidance']['steps']")
    return r.status_code, d, 0.0

Node.js — same flow

javascript
const BASE  = process.env.AISA_BASE_URL || "https://cdn.aisa.one";
const TOKEN = process.env.AISA_CRAWLER_TOKEN;

async function crawl(url) {
  const r = await fetch(url, { headers: {
    "X-AISA-Crawler-Token": TOKEN,
    "User-Agent": "MyCrawler/1.0 (+https://example.com/bot)",
  }});
  if (r.ok) {
    return { ok: true, charged: Number(r.headers.get("x-aisa-charge-amount") || 0), body: await r.text() };
  }
  const denial = await r.json();          // { error, guidance, crawler?.shortfall, ... }
  return { ok: false, status: r.status, denial };
}

Related Documentation

ResourceURLDescription
Onboarding Guide /agent/guide.html The page 402 responses point to — full flow, copy-pasteable
Workflow Demo /agent/workflow.html End-to-end walkthrough with a live demo site and field-tested prompts
Agent Dashboard /agent/ View balance, transactions, manage your account