> **For coding agents and LLMs:** This is one Social Fetch marketing page (markdown export). For API orientation start with [`/llms.txt`](https://www.socialfetch.dev/llms.txt); for auth and crawl rules use [`/agents.txt`](https://www.socialfetch.dev/agents.txt); for every operation use [`/llms-endpoints.txt`](https://www.socialfetch.dev/llms-endpoints.txt).

## This page

- **On-site (HTML):** <https://www.socialfetch.dev/platforms/web/map>
- **Markdown (.mdx) URL:** <https://www.socialfetch.dev/platforms/web/map.mdx>

---

# Web Map API

Pass a site; get back its URLs, ready to filter and feed into markdown, crawl or SEO calls.

`GET /v1/web/map` finds a site's sitemaps through its robots.txt (or at `/sitemap.xml`), follows sitemap indexes, and returns the URLs they list with `lastmod`, `changefreq` and `priority` where the sitemap gives them. Gzipped sitemaps work too. You can also pass a sitemap URL directly. A site with no sitemap is mapped from the links on the page you pass (`source: links`).

Narrow the list with `query` (only URLs containing that text, such as `/blog/`), keep or drop subdomains with `includeSubdomains`, and set `limit` (default 1000, up to 5000). `truncated: true` says more URLs were left; `sitemaps` lists every file read, with its status and how many entries it holds. A slow site stops at the time limit with the URLs found so far (`timedOut: true`), and a sitemap that points at a non-public address is skipped and named in `warnings`.

A call costs 1 credit, which reads about 1.6 MB of sitemaps on the standard proxy, tens of thousands of URLs. A larger site stops early (`budgetExceeded: true`) with what was read unless you raise `maxCredits` (up to 25); then you're charged only for the data used. Read `meta.creditsCharged` on every response.

## Request

- **Endpoint:** `GET /v1/web/map`
- **Credits:** 1 credit
+24 credits for maxCredits
Max 25 credits
You're only charged for what's returned
- **Input:** by site URL
- **Full reference:** [docs](https://www.socialfetch.dev/docs/api/v1/web/map/get)
- **Platform page:** [/platforms/web](https://www.socialfetch.dev/platforms/web)

## FAQ

### What does the Web Map API return?

lookupStatus, the requested url, source (sitemap, links or none), urls (each with url, lastmod, changefreq, priority and the sitemap that listed it), truncated, sitemaps (each file read, with status, kind, urlCount and error), robotsTxt, timedOut, warnings, and the proxy tier, budgetExceeded and bytesTransferred.

### How do I call it?

Send GET /v1/web/map with x-api-key and a site URL, a page on it, or a sitemap URL. Optional: limit, query, sitemap (include, only or skip), includeSubdomains, ignoreQueryParameters, proxy and maxCredits.

### How much does it cost?

1 credit per call by default, which loads about 1.6 MB on the standard proxy. A page that needs more stops early (budgetExceeded: true) unless you raise maxCredits (up to 25); then you're charged only for the data it uses. Blocked sites move up to residential or mobile proxies automatically; those use the budget about nine times faster. meta.creditsCharged is the billing source of truth.

### What if a site has no sitemap?

With the default sitemap=include, the page you pass is read once and its links on the same site become the map, marked source: links. Use sitemap=only to get sitemap URLs alone, or sitemap=skip to use only the page's links.

### Does it load the pages it lists?

No. It reads robots.txt and sitemaps (or one page's links). To get the pages' content, pass the URLs to GET /v1/web/markdown, GET /v1/web/crawl or GET /v1/web/seo.

### What does lookupStatus restricted mean?

Bot protection blocked the site on every proxy tier allowed, so urls is empty. It bills like the other web routes. If you set proxy=datacenter, try without it.
