How to scrape Hacker News data with an API

Use Social Fetch to search Hacker News, read a story and look up a public user profile. The examples below use Dropbox mentions, its original launch post and Paul Graham's pg profile.

Hacker News's public read APIs also work for standalone scripts. See Social Fetch vs DIY scraping if you are deciding which approach to use.

The Dropbox launch post is one example of the public data you can read:

The shortest path is a search:

curl -sS \
  -H "x-api-key: $SOCIALFETCH_API_KEY" \
  -G "https://api.socialfetch.dev/v1/hackernews/search" \
  --data-urlencode "query=Dropbox"

You'll need an API key and curl or the TypeScript SDK. New to the API? Start with the Quickstart.

Using the Social Fetch API

Send your API key in the x-api-key header. The Playground uses your signed-in Social Fetch account, so you can try the requests there without pasting a key. Playground requests spend credits too.

Each completed search page, story lookup and user lookup below costs 1 credit, including an empty page or a missing story or user. Check meta.creditsCharged on each response.

Search mentions

1. Open the Playground search endpoint.
2. Enter Dropbox in query, then click Send.
3. Open Response and read data.hits. Each hit has an item id, content type and an itemUrl linking to the Hacker News discussion.

Use the same request in your code:

const params = new URLSearchParams({"query":"Dropbox"});

const response = await fetch(
  `https://api.socialfetch.dev/v1/hackernews/search?${params.toString()}`,
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

The search can return stories and comments. Set type=story if you only want stories, or use author, domain and minPoints to narrow the results. Full parameters: Search Hacker News.

An empty hits array with no further pages is a completed empty result. Try a broader query or remove filters if you expected matches.

Get the next search page

Search pages start at 0. Read data.page.hasMore and data.page.nextPage, then pass nextPage as the next request's page parameter. Keep the query and filters the same.

Stop when hasMore is false or nextPage is null. The API limits how far you can page through a search; narrow your query if you need results beyond the accessible range. Treat totalHits as an estimate when totalHitsExact is false.

Get a story

The Dropbox launch post has item ID 8863. On the public Hacker News page, its title, points and comment count correspond to the story fields returned by the API.

1. In the Playground, open Hacker News, then Story.
2. Enter 8863 in id and click Send.
3. Check data.lookupStatus before reading data.story.

const response = await fetch(
  "https://api.socialfetch.dev/v1/hackernews/stories/8863",
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

The story is null when lookupStatus is not_found or not_story. A comment also has an item ID, but it cannot be read as a story. For example, 9224 is a comment on the Dropbox post:

Comment text comes from a separate paginated route: story comments. childIds alone is a list of top-level comment IDs, not the whole discussion.

Get a user

1. In the Playground, open Hacker News, then User.
2. Enter pg in username and click Send.
3. When data.lookupStatus is found, read data.profile.

The profile has a username, creation date, karma, about text and public profile URL. Submissions, comments and public favourites use separate routes; see Get Hacker News user.

const response = await fetch(
  "https://api.socialfetch.dev/v1/hackernews/users/pg",
  {
    headers: {
      "x-api-key": process.env.SOCIALFETCH_API_KEY,
    },
  }
);

const body = await response.json();

console.log(response.status, body);

Reading the response

Search returns hits plus page metadata. This shortened example shows the response shape; your query can return different hits and page counts:

{
  "data": {
    "query": "Dropbox",
    "hits": [
      {
        "id": 8863,
        "type": "story",
        "author": "dhouston",
        "title": "My YC app: Dropbox - Throw away your USB drive",
        "score": 104,
        "itemUrl": "https://news.ycombinator.com/item?id=8863"
      }
    ],
    "page": {
      "page": 0,
      "pageSize": 20,
      "returned": 1,
      "hasMore": false,
      "nextPage": null
    }
  },
  "meta": {
    "requestId": "req_01example",
    "creditsCharged": 1,
    "version": "v1"
  }
}

A story lookup returns a story object:

{
  "data": {
    "lookupStatus": "found",
    "story": {
      "id": 8863,
      "author": "dhouston",
      "title": "My YC app: Dropbox - Throw away your USB drive",
      "url": "http://www.getdropbox.com/u/2/screencast.html",
      "score": 104,
      "commentCount": 71
    }
  },
  "meta": {
    "requestId": "req_01example",
    "creditsCharged": 1,
    "version": "v1"
  }
}

Keep meta.requestId when asking support about a response. See Errors and Credits.

What you can build

• Launch and brand monitors: search product names, then pull stories and comments for the hits that matter.
• Front-page digests: ranked feeds with hydrated items for Slack/email roundups.
• Hiring intel: structured Who is Hiring rows with best-effort company/role/location fields (plain text remains source of truth).
• Author analysis: user cards plus submissions, comments, and public favourites.

Compare options: Best Hacker News APIs & scrapers in 2026. Related: Reddit product research.

FAQ

Do I need a Hacker News login?

No. These examples read public data. You need a Social Fetch account with credits to run them in the Playground, or a Social Fetch API key to call them from code.

Can't I just use Hacker News's public APIs myself?

Yes. For a script that only needs Hacker News, you can call its public APIs directly. Social Fetch gives you the same response and billing conventions as the other platforms you use through it.

How fresh is the data?

Story lookups can return a cached result. Check meta.cached and the Age response header when present. Keep meta.requestId if you need support to trace a lookup.

What happens if a story or user doesn't exist?

Check data.lookupStatus where present. Don't treat 200 alone as found.

How are credits charged?

Credits charge when a lookup completes, including empty search pages and typed misses. Each search page, story lookup and user lookup in this guide costs 1 credit. Always reconcile against meta.creditsCharged.

How does this compare to other providers?

See Best Hacker News APIs & scrapers in 2026, vs Apify, and the compare hub.