On this page
If your product touches social networks (marketing analytics, creator CRM, brand safety, AI research), you have probably watched "fetch public posts" turn into a permanent side project.
A CSS selector moves on Instagram and your ingestion job returns empty arrays. A platform renames a field inside a script tag and your TypeScript types lie for a week before anyone notices. You get paged at 2 a.m. for a parser you wrote six months ago and forgot existed.
We built Social Fetch because that maintenance work belongs in an API layer, not on your product roadmap forever. One REST surface covering 20 platforms. Stable JSON shapes. A requestId on every response so support can trace a call without guessing which worker ran it.
The boring API manifesto
"Exciting" social integrations mean headless browsers, proxy rotation, and schema drift in production. Teams need the opposite: stable field names, HTTP errors you can retry on, rotatable API keys, billing that matches completed lookups.
Product teams should ship features, not babysit scrapers. One integration path beats five hacks — especially when someone asks for a second platform by Friday. If we can't debug a failed call from a request ID, the API isn't finished.
The best social data pipeline is the one your future self does not recognize in
git blame, because it has not needed a fix in two years.
Smoke test before you spend credits
Before you hit a metered route, call GET /v1/whoami. It checks your API key and costs nothing. Same smoke test as the Quickstart.
TypeScript version. Keys go in the x-api-key header (server-side only, not a browser token):
Metered routes bill differently. Credits charge when a lookup completes, including not_found and private outcomes that come back as HTTP 200 with a lookupStatus in the body. Pre-send validation errors, lookup_failed, and 503 temporarily_unavailable do not bill the same way. Reconcile your ledger against meta.creditsCharged, not your request count. Full rules: Credits.
One envelope for success and errors
Successful responses wrap payloads in { data, meta }. meta carries requestId, creditsCharged, and the API version. Errors use the same shape so you always know what to paste into a support ticket.
Log meta.requestId on every call. We learned that the hard way; see The Scrape Job That Never Timed Out. Typed error bodies live in the Errors guide. Per-route credit costs are in the API reference; plan volume on Pricing.
When a custom scraper still makes sense
Sometimes a one-off script is the right tool. You need one obscure page, or you are proving something over a weekend. Fine.
Where to go next
Check Pricing before you commit to a ship date. Wire auth and error handling before you parallelize heavy fetches (Quickstart, Errors). Skim route coverage in the API reference before you lock a database schema.
Boring plumbing. Interesting products on top. That is the split we want.
Founder of Social Fetch. Writes about social-data APIs, reliability, and the production incidents that forced better timeouts, envelopes, and billing.
Related articles
Questions about this post? Contact us.