Social Fetch
Web

Crawl web pages

Crawl a small set of web pages synchronously.

GET/v1/web/crawl
x-api-key*stringheader

API key (`sfk_...`)

Parameters
1
url*arrayquery

URLs to crawl. Repeat the `url` query parameter for multiple pages (max 5).

Response fields
19

Endpoint-specific response payload.

Per-URL crawl results.

dataresults[]url
string

Final URL associated with this crawl result.

dataresults[]status
integer

HTTP status code reported for the page fetch.

dataresults[]success
boolean

Whether the page was crawled successfully.

Markdown extracted from the page when available.

dataresults[]markdownraw
stringoptional

Raw markdown for the crawled page.

dataresults[]markdownfit
stringoptional

Filtered markdown for the crawled page.

dataresults[]html
stringoptional

HTML content for the page when returned by the crawler.

dataresults[]metadata
objectoptional

Page metadata such as title or description when available.

dataresults[]errorMessage
stringoptional

Provider error message when the page crawl failed.

Summary counts for the crawl batch.

datasummaryrequestedUrls
integer

Number of URLs requested in the crawl batch.

≥ 0

datasummarysucceeded
integer

Number of URLs that crawled successfully.

≥ 0

datasummaryfailed
integer

Number of URLs that failed to crawl.

≥ 0

Metadata describing the request and billing outcome.

metarequestId
string

Unique request identifier for tracing this API call.

min 1 chars

metacreditsCharged
integer

Credits charged for this request.

≥ 0

metaversion
string

Public API version that served the response.

one of: v1

Code example

curl "https://api.socialfetch.dev/v1/web/crawl?url=https://www.socialfetch.dev" \
  -H "x-api-key: YOUR_API_KEY"

Responses

Crawl results.

On this page

No Headings