How to scrape Hacker News data with an API

This guide covers Hacker News search, story, and user lookups via /v1/hackernews/**. HN's public read APIs work for standalone scripts, but product monitors still need feed hydration, comment paging, search mapping, and HTML-only surfaces like favorites — see Social Fetch vs DIY scraping for when that glue is worth owning.

The shortest path is a search:

curl -sS \
  -H "x-api-key: $SOCIALFETCH_API_KEY" \
  -G "https://api.socialfetch.dev/v1/hackernews/search" \
  --data-urlencode "query=Dropbox"

You'll need an API key and curl or the TypeScript SDK. New to the API? Start with the Quickstart.

Using the Social Fetch API

Social Fetch exposes Hacker News under documented /v1/hackernews/** routes — search, feeds, stories, comments, items, users, favorites, Who is Hiring, and thin update helpers. One header authenticates every call.

Search mentions

Brand and launch monitors start here — query the product name, optionally narrow with author, domain, or minPoints, then hydrate only the ids you keep:

const params = new URLSearchParams({"query":"Dropbox"});

const response = await fetch(
  `https://api.socialfetch.dev/v1/hackernews/search?${params.toString()}`,
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

Docs: Search Hacker News. Credit-metered per completed page — trust meta.creditsCharged. An empty hits array with no further pages is a completed empty result, not a transport failure.

Get a story

Classic Show HN Dropbox (8863) is a good smoke-test id:

const response = await fetch(
  "https://api.socialfetch.dev/v1/hackernews/stories/8863",
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

Branch on data.lookupStatus (found, not_found, not_story). Comment trees are a separate paginated route: story comments.

Get a user

Author analysis starts with the user card, then siblings for submissions, comments, or public favorites:

const response = await fetch(
  "https://api.socialfetch.dev/v1/hackernews/users/pg",
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

Reading the response

Search returns hits plus page metadata:

{
  "data": {
    "query": "Dropbox",
    "hits": [
      {
        "id": 8863,
        "type": "story",
        "author": "dhouston",
        "title": "My YC app: Dropbox - Throw away your USB drive",
        "score": 104,
        "itemUrl": "https://news.ycombinator.com/item?id=8863"
      }
    ],
    "page": {
      "page": 0,
      "pageSize": 20,
      "returned": 1,
      "hasMore": false,
      "nextPage": null
    }
  },
  "meta": {
    "requestId": "req_01example",
    "creditsCharged": 1,
    "version": "v1"
  }
}

Story gets return a typed card:

{
  "data": {
    "lookupStatus": "found",
    "story": {
      "id": 8863,
      "author": "dhouston",
      "title": "My YC app: Dropbox - Throw away your USB drive",
      "url": "http://www.getdropbox.com/u/2/screencast.html",
      "score": 104,
      "commentCount": 71
    }
  },
  "meta": {
    "requestId": "req_01example",
    "creditsCharged": 1,
    "version": "v1"
  }
}

The one gotcha: HTTP 200 does not always mean "persist this row." Empty search hits are a completed empty result; story/user routes expose lookupStatus for misses and wrong item types. Every response carries meta.requestId. See Errors and Credits.

What you can build

• Launch and brand monitors — search product names, then pull stories and comments for the hits that matter.
• Front-page digests — ranked feeds with hydrated items for Slack/email roundups.
• Hiring intel — structured Who is Hiring rows with best-effort company/role/location fields (plain text remains source of truth).
• Author analysis — user cards plus submissions, comments, and public favorites.

Vendor shopping: Best Hacker News APIs & scrapers in 2026. Related: Reddit product research.

FAQ

Is scraping Hacker News legal?

It depends on your jurisdiction, what data you collect, and how you use it. HN publishes public read surfaces and public HTML. You remain responsible for Y Combinator's terms and applicable law. This is not legal advice.

Can't I just use Hacker News's public APIs myself?

Yes for HN-only scripts. You still assemble hydration, comment paging, search mapping, and HTML-only surfaces. For multi-platform enrichment, a managed data API is often shorter.

How fresh is the data?

Each request fetches live at call time. meta.requestId traces a specific lookup.

What happens if a story or user doesn't exist?

Check data.lookupStatus where present. Don't treat 200 alone as found.

How are credits charged?

Credits charge when a lookup completes, including empty search pages and typed misses. Each HN request or cursor page is credit-metered. Always reconcile against meta.creditsCharged.

How does this compare to other providers?

See Best Hacker News APIs & scrapers in 2026, vs Apify, and the compare hub.

Ready to try it? Get an API key — new accounts include 100 free credits. Platform hub: /platforms/hackernews.