Web Markdown API
Pass a public URL, get markdown back in the shared Social Fetch envelope — same API key as TikTok, Instagram, and LinkedIn lookups.
GET /v1/web/markdown1 credit per successful request.
About this endpoint
RAG pipelines and research agents rarely stop at social posts. A creator bio links to a Shopify store. A competitor tweet points at a pricing page. Docs and changelogs sit on ordinary public URLs that your social scrapers never touch.
`GET /v1/web/markdown` takes a required `url` (http or https) and returns markdown in the shared `{ data, meta }` envelope. On `lookupStatus: found`, `data.markdown` holds `raw` text and optional `fit` (readability-oriented) content; optional `metadata` may include fields such as title. Optional `filter` is `fit` (default), `raw`, or `bm25`. When `filter=bm25`, pass a `query` string so the extractor can rank relevant passages. Optional `cacheMode` is `enabled` (default), `bypass`, or `write_only`. Bill from `meta.creditsCharged`.
This route extracts one page you already know. For keyword discovery first, use `GET /v1/web/search`. For cleaned HTML, grounded Q&A, or a small URL batch, use `GET /v1/web/html`, `GET /v1/web/ask`, or `GET /v1/web/crawl`. All share one Social Fetch API key.
Web Markdown API FAQ
What does the Web Markdown API return?
A data object with lookupStatus (found or restricted), the fetched url, an HTTP status when available, and markdown content when found (raw plus optional fit, withCitations, and references). Optional metadata may include page fields such as title. The response uses the shared Social Fetch envelope with meta.requestId and meta.creditsCharged.
How do I call web markdown?
Send GET /v1/web/markdown with x-api-key and a required url query parameter (public http or https, up to 2083 characters). Optionally add filter (fit, raw, or bm25), query (required when filter is bm25), and cacheMode. Full enums and examples are in the API docs.
How much does web markdown cost?
Pricing is documented on the operation in the API registry (shown on this page). Confirm on every response with meta.creditsCharged — that field is the billing source of truth.
What do fit, raw, and bm25 mean?
filter selects how content is shaped. fit (default) favors readable main content for LLM use. raw keeps a fuller markdown dump of the page. bm25 ranks passages by relevance to a query string you pass — query is required when filter is bm25.
What does lookupStatus restricted mean?
restricted means bot or access protection blocked automated fetching. Content fields are null. The request can still bill as a completed lookup — read meta.creditsCharged and treat restricted as a hard stop for that URL, not as empty markdown you can ignore.
Is this the same as web search or ask?
No. GET /v1/web/markdown needs a URL and returns page text as markdown. GET /v1/web/search finds candidate URLs by keyword. GET /v1/web/ask takes a URL plus a question and returns an LLM answer. Typical flow: search for candidates, then markdown (or ask) on the URLs you keep.
Do I need a separate scraping vendor for public pages?
Not for this job. Authenticate with your Social Fetch API key. Web markdown shares the same credit wallet and response envelope as the social marketplace routes, so agents can fetch a TikTok profile and a linked landing page without a second vendor stack.