> **For coding agents and LLMs:** This is one page from the Social Fetch docs (markdown export). For curated orientation and workflow guidance, start with [`/llms.txt`](https://www.socialfetch.dev/llms.txt); for agent onboarding and crawl rules, use [`/agents.txt`](https://www.socialfetch.dev/agents.txt); for the full endpoint list with links to pages like this one, use [`/llms-endpoints.txt`](https://www.socialfetch.dev/llms-endpoints.txt); for one platform's parameters and curls, use [`/llms-{platform}.txt`](https://www.socialfetch.dev/llms-tiktok.txt); use [`/llms.json`](https://www.socialfetch.dev/llms.json) when you need structured JSON for tool registration.

## This page

- **On-site (HTML):** <https://www.socialfetch.dev/docs/api/v1/web/markdown/get>
- **Markdown (.mdx) URL:** <https://www.socialfetch.dev/docs/api/v1/web/markdown/get.mdx>

## API base URL and authentication

- **API origin (from OpenAPI `servers`):** `https://api.socialfetch.dev`
- **Authentication:** send `x-api-key: sfk_...` on `/v1/**` and `/v2/**` routes unless the operation is explicitly anonymous (check OpenAPI `security`, the [API reference hub](https://www.socialfetch.dev/docs/api.mdx), [`/llms.txt`](https://www.socialfetch.dev/llms.txt), or [`/llms.json`](https://www.socialfetch.dev/llms.json) for each route).
- **OpenAPI JSON:** <https://www.socialfetch.dev/openapi.json>

## Recommended docs entrypoints (this site)

- [Documentation overview](https://www.socialfetch.dev/docs.mdx) — top-level orientation (markdown).
- [Quickstart](https://www.socialfetch.dev/docs/quickstart.mdx) — authenticate with `x-api-key`, validate auth with `whoami`, and understand the JSON envelope.
- [SDK](https://www.socialfetch.dev/docs/sdk.mdx) — official TypeScript SDK guide, including `SocialFetchClient`, `Result`, and `unwrap()`.
- [Capability matrix](https://www.socialfetch.dev/docs/capability-matrix.mdx) — every operation with identifiers, pagination, outcomes, media download, credits, and its SDK method. Generated from OpenAPI, so use it for route selection instead of scanning individual pages.
- [Recipes](https://www.socialfetch.dev/docs/recipes.mdx) — copyable workflows (brand monitoring, transcripts, Ad Library, creator scoring, Reddit research) with credit callouts and SDK examples.
- [Integrations](https://www.socialfetch.dev/docs/integrations.mdx) — MCP for AI clients, n8n verified node, Apify Store Actors, Make custom app, SDK, and REST API connection paths.
- [MCP product page](https://www.socialfetch.dev/mcp) — hosted MCP overview, OAuth, Skills install.
- [MCP integration](https://www.socialfetch.dev/docs/integrations/mcp.mdx) — hosted `/mcp` server, OAuth, Cursor/VS Code/Claude install snippets, 264 endpoint tools, plus docs_search/docs_read for implementation help.
- [n8n integration](https://www.socialfetch.dev/docs/integrations/n8n.mdx) — install `n8n-nodes-socialfetch`, credentials, and workflow examples.
- [Apify integration](https://www.socialfetch.dev/docs/integrations/apify.mdx) — Store Actors under @social-fetch, PPE billing, dataset export, and quick start.
- [Make integration](https://www.socialfetch.dev/docs/integrations/make.mdx) — custom app modules for Make scenarios, API key credentials, and module catalog.
- [`/llms-endpoints.txt`](https://www.socialfetch.dev/llms-endpoints.txt) — every documented operation with a direct link to that route's agent-readable markdown page (prefer this over parsing OpenAPI).
- [`/llms-{platform}.txt`](https://www.socialfetch.dev/llms-tiktok.txt) — per-platform endpoint files generated from OpenAPI (parameters, credits, curls).
- [`/agents.txt`](https://www.socialfetch.dev/agents.txt) — agent crawl/onboarding file with capabilities, auth rules, and allowlist.
- [`/llms.json`](https://www.socialfetch.dev/llms.json) — structured machine-readable operation inventory with parameter names, pagination, outcomes, credits, and SDK mapping.
- [API reference hub](https://www.socialfetch.dev/docs/api.mdx) — human-friendly index of operations with links into generated pages.
- [Errors](https://www.socialfetch.dev/docs/errors.mdx) — shared error envelope and HTTP status guidance.
- [Credits](https://www.socialfetch.dev/docs/credits.mdx) — metering, `402`, and planning batch jobs.
- Outcome semantics such as `found`, `not_found`, and `private` are documented in [Errors](https://www.socialfetch.dev/docs/errors.mdx) and on operation pages when present in the OpenAPI contract.

## Markdown docs convention

- Every docs page has a markdown twin: append **`.mdx`** to the docs pathname (for example `/docs/quickstart` → `/docs/quickstart.mdx`).
- Agents that send `Accept: text/markdown` on `/docs/**` HTML URLs may receive markdown directly (same URL, `Vary: Accept`).
- Published blog posts use the same convention: `/blog/{slug}` → `/blog/{slug}.mdx`, or `Accept: text/markdown` on the HTML URL (`Vary: Accept`).

---
# Generate web page markdown (https://www.socialfetch.dev/docs/api/v1/web/markdown/get)

## Summary

Convert a web page URL into clean markdown.

**Tags:** `Web`

## HTTP

- **Method:** GET
- **Path:** `/v1/web/markdown`
- **Base URL:** `https://api.socialfetch.dev`

## Capability summary

- **SDK mapping:** `client.web.getMarkdown({ url: "https://www.socialfetch.dev/" })`
- **Accepted identifiers:** `url` (query), `query` (query)
- **Pagination:** none
- **Business outcome field:** `data.lookupStatus` with values `found`, `restricted`

## Agent notes

- **Empty results:** `lookupStatus: restricted` means bot/access protection blocked the fetch; content fields are null.

## Credits

- **Base:** 1 credit per successful lookup.
- **Maximum on success (200):** 1 credit.
- **Failed requests (`lookup_failed`, `503`):** not charged.
- **Authoritative field:** `meta.creditsCharged`.

## Authentication

- **`x-api-key`**: API key (`sfk_...`)

## Parameters

### `url` (query)

- **Required:** yes
- **Constraints:** type `string`; minLength: 1; maxLength: 2083
- **Description:** Web page URL to fetch.

### `filter` (query)

- **Required:** no
- **Constraints:** type `string`; enum: fit, raw, bm25; default: `fit`
- **Description:** Markdown extraction filter. `fit`: strip boilerplate and extract the main readable content. `raw`: full unfiltered page markdown, no content pruning. `bm25`: rank and return only the content most relevant to `query`, using the BM25 keyword-relevance algorithm — requires `query` to be set.

### `query` (query)

- **Required:** no
- **Constraints:** type `string`; maxLength: 500
- **Description:** Optional query string used by the bm25 filter to rank relevant content.

### `cacheMode` (query)

- **Required:** no
- **Constraints:** type `string`; enum: enabled, bypass, write_only; default: `enabled`
- **Description:** Cache behavior. `enabled`: read from cache if present, else fetch and write to cache. `bypass`: always fetch fresh, ignoring and not updating the cache. `write_only`: always fetch fresh, but write the result to cache without reading from it first. Default: `enabled`.

### `scanFullPage` (query)

- **Required:** no
- **Constraints:** type `boolean`
- **Description:** When true, scroll the page to load dynamically appended content (infinite scroll). Default false.

### `waitFor` (query)

- **Required:** no
- **Constraints:** type `string`; maxLength: 200
- **Description:** Wait for a CSS selector before extraction. Must be prefixed with "css:" (e.g. css:main). JavaScript wait conditions are not supported.

## Responses (status codes)

- **200**: Markdown extraction result.
- **400**: Invalid query parameters or disallowed URL
- **401**: Missing or invalid API key
- **402**: Insufficient credits
- **500**: Unexpected or billing error
- **502**: Extraction could not be completed.
- **503**: Service temporarily unavailable; safe to retry with backoff.

## Response body (200)

Markdown extraction result.

### Field outline

- **data** (required) — type `object`. Endpoint-specific response payload.
  - **lookupStatus** (required) — type `string`; enum: found, restricted. Whether page content could be extracted. Restricted means bot protection or similar access controls blocked automated fetching.
  - **url** (required) — type `string`. URL that was fetched.
  - **status** (required) — type `integer`; nullable. HTTP status code reported for the page fetch when available; null when restricted.
  - **markdown** (required) — type `object`; nullable. Markdown content when lookupStatus is found; null when restricted.
    - **raw** (required) — type `string`. Primary markdown text extracted from the page.
    - **fit** (optional) — type `string`. Filtered markdown optimized for LLM consumption.
    - **withCitations** (optional) — type `string`. Markdown with numbered citations for outbound links.
    - **references** (optional) — type `string`. Reference list for cited links in the markdown output.
  - **metadata** (optional) — type `object`. Page metadata such as title when available.
  - **links** (optional) — type `object`. Links discovered on the page when available.
    - **internal** (required) — type `array`. Same-host links discovered on the page.
      - _items:_
    - **external** (required) — type `array`. Cross-host links discovered on the page.
      - _items:_
  - **media** (optional) — type `object`. Media assets discovered on the page when available.
    - **images** (optional) — type `array`. Images on the page.
      - _items:_
    - **videos** (optional) — type `array`. Videos on the page.
      - _items:_
    - **audios** (optional) — type `array`. Audio elements on the page.
      - _items:_
- **meta** (required) — type `object`. Metadata describing the request and billing outcome.
  - **requestId** (required) — type `string`; minLength: 1. Unique request identifier for tracing this API call.
  - **creditsCharged** (required) — type `integer`; minimum: 0. Credits charged for this request.
  - **version** (required) — type `string`; enum: v1. Public API version that served the response.
  - **cached** (optional) — type `boolean`. True when served from shared response cache. Credits still apply (full endpoint price); Age header may be present.

### Example JSON (OpenAPI example)

```json
{
  "data": {
    "lookupStatus": "found",
    "url": "https://www.socialfetch.dev/",
    "status": 200,
    "markdown": {
      "raw": "# Social media scraper API for every major platform.\nScrape profiles, posts, comments, videos, transcripts, and metrics from TikTok, Instagram, YouTube, X, LinkedIn, and more.",
      "fit": "Social media scraper API for every major platform. Scrape profiles, posts, comments, videos, transcripts, and metrics from TikTok, Instagram, YouTube, X, LinkedIn, and more."
    }
  },
  "meta": {
    "requestId": "req_01example",
    "creditsCharged": 1,
    "version": "v1"
  }
}
```

### Machine-readable error codes

When an error JSON body is returned, it may include one of these `error.code` values (derived from the OpenAPI schemas for this operation; additional codes may exist at runtime):

- `bad_request`

## Error handling & retries

Interpret HTTP status codes using the descriptions below. Do not assume a JSON body unless the OpenAPI schema defines one for that status.

- **400**: Invalid query parameters or disallowed URL **Retry:** Fix the request; retrying the same invalid payload will not help.
- **401**: Missing or invalid API key **Retry:** Fix the API key first; retrying without changes will not help.
- **402**: Insufficient credits **Retry:** Do not retry without resolving billing/credits (retrying the same request will not help).
- **500**: Unexpected or billing error
- **502**: Extraction could not be completed. **Retry:** May be transient; a few retries with backoff are reasonable.
- **503**: Service temporarily unavailable; safe to retry with backoff. **Retry:** Usually safe to retry with exponential backoff and jitter.

### Suggested client defaults

- Send the API key using the `x-api-key` header on every request.
- On `503` (and sometimes `502`), retry with backoff; cap retries and surface a clear error to the user.
- On `402`, surface an actionable billing message rather than blind retries.

## Examples

### TypeScript SDK

```typescript
import { SocialFetchClient } from "@socialfetch/sdk";

const client = new SocialFetchClient({
  apiKey: process.env.SOCIALFETCH_API_KEY!,
});

const result = await client.web.getMarkdown({
  url: "https://www.socialfetch.dev",
});

if (!result.ok) {
  console.error(result.error);
} else {
  console.log(result.value.data);
}
```

### Node.js

```javascript
const response = await fetch(
  "https://api.socialfetch.dev/v1/web/markdown?url=https://www.socialfetch.dev",
  {
    headers: {
      "x-api-key": "YOUR_API_KEY"
    }
  }
);

const data = await response.json();
console.log(data);
```

### cURL

```bash
curl "https://api.socialfetch.dev/v1/web/markdown?url=https://www.socialfetch.dev" \
  -H "x-api-key: YOUR_API_KEY"
```

### Python

```python
import requests

response = requests.get(
    "https://api.socialfetch.dev/v1/web/markdown?url=https://www.socialfetch.dev",
    headers={"x-api-key": "YOUR_API_KEY"},
)
data = response.json()
print(data)
```