Web Scraping Specialist
Keep public social data flowing when platforms change the rules overnight — and when they change them again on Friday.
- Location
- Remote — worldwide
- Employment
- Full-time
- Salary
- £55,000 – £85,000 / year
- Hours
- Full-time · flexible async hours
- Visa sponsorship
- Case-by-case — happy to discuss during hiring.
Competitive for role and experience. Final offer depends on location and background.
About the role
Social Fetch normalises public social data into a single REST API. Our customers integrate once and read `followerCount` whether the underlying platform called it something else. When Instagram changes a field name or TikTok rotates an anti-bot challenge, they shouldn't find out from a broken cron job.
You'll own the fetch layer — researching how platforms serve data, implementing and maintaining scrapers, and hardening them when things break in production. That covers profiles, posts, search, comments, transcripts, and ad-library lookups across TikTok, Instagram, YouTube, Reddit, LinkedIn, Facebook, and more. The stack is TypeScript throughout: monorepo, Playwright where we need a browser, plain HTTP where we don't.
Team & how we work
You'll work directly with the engineers who built the API surface and the support team who see customer failures first. When a platform drifts, support often spots the pattern before monitoring does — you'll get Slack threads with requestIds, sample URLs, and "this worked yesterday" context.
We're fully remote with flexible hours. Incidents don't respect timezones, so sometimes you'll jump on an urgent fix outside your usual window — but day-to-day work is async. Write up what you found, open a PR, leave notes for the next person. We don't have a dedicated SRE team; reliability is everyone's job, and yours most of all.
What you'll do
- Build and maintain scrapers and integrations for major social platforms — profiles, posts, search, media, comments, engagement metrics, and platform-specific endpoints.
- Diagnose fetch failures: HTTP errors, schema drift, rate limits, bot detection, geo blocks, CAPTCHAs, and responses that look fine but parse to garbage.
- Design fetch strategies that survive bad days — retries with backoff, fallbacks between mobile and web APIs, session handling, proxy rotation when it's worth the cost.
- Keep normalized API field mappings stable when platforms rename or nest fields differently.
- Write internal notes on platform quirks (auth flows, pagination traps, fields that lie) so the next person doesn't start from zero.
- Watch production quality signals — error rates, empty fields, latency spikes — and fix regressions before customers open tickets.
- Review support escalations that smell like platform drift, not user error.
What we're looking for
- Several years of hands-on web scraping experience. Tutorial-level Playwright scripts aren't enough — we need someone who's been paged when a site changed its HTML.
- Real familiarity with anti-bot systems, browser automation (Playwright/Puppeteer), HTTP clients, and parsing messy HTML and JSON (including data hidden in script tags).
- Comfort with production debugging: logs, reproducing a failing URL, shipping a fix, watching the error rate drop.
- Strong TypeScript or JavaScript — our codebase is TypeScript and you'll be in it daily.
- Pragmatic instincts about public data, ToS grey areas, and when to stop digging.
- Ability to work independently in a remote, async team — you unblock yourself, document what you learned, and ask for help when you're stuck.
Nice to have
- Experience maintaining scrapers across many platforms, not just one site you know inside out.
- Familiarity with proxy providers, CAPTCHA services, and reverse-engineering mobile or web API calls.
- Prior work at a data or API product company where uptime mattered to paying customers.
- Exposure to observability tooling — structured logs, request IDs, error tracking.
Benefits
Full-time team members receive:
- Flexible hours — Async-first. Ship your work; we don't track screen time or mandate overlap windows unless a role specifically needs it.
- Unlimited PTO — Take the time you need. Give the team a heads-up for longer breaks so support and on-call handoffs stay covered.
- Equipment stipend — Home office setup — desk, monitor, chair, whatever actually makes you productive.
- Learning budget — Courses, books, and conferences tied to your role. Scraping conferences, writing workshops, support tooling — all fair game.
- Health benefits — Health coverage for full-time team members. Details and eligibility depend on location; we walk through it during hiring.
How to apply
- Put the role title in the email subject line.
- Include a short intro and relevant experience.
- Link to GitHub or examples of scrapers you've built or maintained (don't share anything proprietary).
- Optional: describe one platform change that broke your scraper and how you fixed it.
GitHub repos, blog posts, or a brief write-up on the hardest scraping problem you've solved — the weirder the platform behaviour, the better.