What is bulk social data?
Bulk social data is high-volume public lookup — thousands of handles, posts, or transcripts through the same synchronous API as a single call. On Social Fetch that's REST, credits, and { data, meta } per request, not a batch upload or async job queue.
What bulk actually means here
No separate bulk product. Same typed REST route, called many times with your own concurrency. 5,000 profiles = 5,000 calls (or however many list pages you walk), not one 5,000-row file.
Overview: /bulk-social-data. Catalog: 21 platforms, 165 operations.
Pagination and a concrete pull
List endpoints page via data.page.nextCursor and data.page.hasMore — each page is its own billed request with its own meta.requestId.
Fan-out from a CSV: bounded worker pool (50–200 in flight, not 3,000 at once), retry per handle, not per batch.
Concurrency and credit budgeting
No published RPS cap on paid routes — credit balance is the ceiling. ~500 concurrent requests is reliability guidance.
Budget before you start. Completed lookups charge including not_found. Only lookup_failed and 503 don't charge. Reconcile meta.creditsCharged and GET /v1/balance.
How this compares to run-and-poll batch jobs
Apify-style actors: submit inputs, poll until done, download a dataset — good for minute-to-hour jobs.
Social Fetch: synchronous request/response every time. You own orchestration; partial results are whichever requests finished.
Common mistakes
Unbounded parallelism → timeouts, not faster completion. Use a worker pool.
Retrying not_found in a loop — burns credits for the same answer.
Using Ask for fan-out — call the typed route directly once you know it.
What it is not
Not a nightly CSV of the whole network. Not async run-ID polling. Not arbitrary open-web crawling — see compare pages for crawler tools.
FAQ
How do I bulk fetch social media data with Social Fetch?
Same typed endpoint repeatedly with bounded concurrency (50–500 in flight). Lists: loop nextCursor until hasMore is false.
Does Social Fetch support batch or async job requests?
No. Every request is synchronous; you handle concurrency and retries client-side.
Is there a rate limit for bulk pulls?
No published RPS cap; credits are the limit. ~500 concurrent recommended. See /glossary/rate-limits.
How do I paginate through a large list endpoint?
Pass data.page.nextCursor until data.page.hasMore is false. Each page bills separately.
How much will a large bulk pull cost?
~one charge per completed request (lists/search may cost more). not_found bills; lookup_failed and 503 don't. Check route docs and meta.creditsCharged.
Do bulk jobs get a volume discount?
Larger credit packs lower per-credit cost — Growth ~$1.88/1k, Scale ~$1.65/1k. See /pricing.
Should I use Ask for bulk lookups?
No. Use the typed route directly at volume.
What happens if a bulk job runs out of credits partway through?
Further calls return 402. Check balance before large runs; handle 402 in retry logic.