Headers
Get your API keyAPI key (`sfk_...`)
URLs to crawl. Repeat the `url` query parameter for multiple pages (max 5).
When true, scroll the page to load dynamically appended content (infinite scroll). Default false.
Wait for a CSS selector before extraction. Must be prefixed with "css:" (e.g. css:main). JavaScript wait conditions are not supported.
max 200 chars
Most expensive proxy tier to use. The page is tried on `datacenter` first and moves up only if it is blocked, stopping at this tier. Default: every tier (`mobile`).
one of: datacenter, residential, mobile
Most credits each URL may cost (1–25). Default 1. The batch shares one budget of this times the number of URLs; you're charged for what the pages use, at least 1 credit per URL.
≥ 1 · ≤ 25
Endpoint-specific response payload.
Per-URL crawl results.
Final URL associated with this crawl result.
HTTP status code reported for the page fetch.
Whether the page was crawled successfully.
Markdown extracted from the page when available.
Raw markdown for the crawled page.
Filtered markdown for the crawled page.
HTML content for the page when returned by the crawler.
Page metadata such as title or description when available.
Error message when the page crawl failed.
Links discovered on the page when available.
Same-host links discovered on the page.
Absolute or page-relative link href.
Anchor text when available.
Title attribute when available.
Cross-host links discovered on the page.
Absolute or page-relative link href.
Anchor text when available.
Title attribute when available.
Media assets discovered on the page when available.
Images on the page.
Image source URL.
Alt text when available.
Optional relevance score from the crawler.
Videos on the page.
Audio elements on the page.
Summary counts for the crawl batch.
Number of URLs requested in the crawl batch.
≥ 0
Number of URLs that crawled successfully.
≥ 0
Number of URLs that failed to crawl.
≥ 0
Proxy tier the page was loaded on: the tier that served it, or the last one tried. Null when not measured.
one of: datacenter, residential, mobile, null
Whether the page needed more than `maxCredits` allows. Loading stopped early, so content may be incomplete.
Data the page loaded through the proxy, in bytes, across every tier tried. Null when not measured.
≥ 0
Metadata describing the request and billing outcome.
Unique request identifier for tracing this API call.
min 1 chars
Credits charged for this request.
≥ 0
Public API version that served the response.
one of: v1
Present, and `true`, only when the source couldn't be reached and this is the last good answer for the same request instead, up to 24 hours old. Absent on every live answer.
one of: true
When a stale answer (`stale: true`) was originally fetched (ISO 8601, UTC). Absent on every live answer.
Code example
curl "https://api.socialfetch.dev/v1/web/crawl?url=https://www.socialfetch.dev" \
-H "x-api-key: $SOCIALFETCH_API_KEY"Responses
Crawl results.