> **For coding agents and LLMs:** This is one published Social Fetch blog post (markdown export). Product docs and API orientation live in [`/llms.txt`](https://www.socialfetch.dev/llms.txt). The HTML article is at the on-site URL below.

## This page

- **On-site (HTML):** [https://www.socialfetch.dev/blog/effect-typescript-production](https://www.socialfetch.dev/blog/effect-typescript-production)
- **Markdown (.mdx) URL:** [https://www.socialfetch.dev/blog/effect-typescript-production.mdx](https://www.socialfetch.dev/blog/effect-typescript-production.mdx)
- **Blog:** [https://www.socialfetch.dev/blog](https://www.socialfetch.dev/blog)

---

# Effect in production at Social Fetch

Social Fetch pulls public data from TikTok, Instagram, YouTube, X, Reddit, and a dozen other platforms. Each has its own rate limits, failure modes, and habit of changing response shapes without notice.

That's not one hard problem. It's repeated decisions: what counts as failure, how long to retry, how many requests at once, what to do when a response doesn't match expectations. We moved that logic onto [Effect](https://effect.website/). Here's what it buys us — and what it costs. ok

The request layer described here runs on Effect, the TypeScript library for typed errors, Schema, and structured concurrency.

[https://effect.website](https://effect.website)

Here's the shape of one request, fanning out to several platform adapters at once:

```mermaid
flowchart TD
    A[Incoming request] --> B{Fan out with bounded concurrency}
    B --> C[TikTok adapter]
    B --> D[Instagram adapter]
    B --> E[Reddit adapter]
    C --> F{Schema.decode the response}
    D --> F
    E --> F
    F -->|Valid| G[Typed data]
    F -->|RateLimited| H[Retry via Schedule]
    H --> C
    F -->|Blocked| I[Fail, no retry]
    F -->|MalformedResponse| J[Typed error, logged]
    G --> K[Aggregate and return]
```

Each box in that diagram is a specific Effect feature. Here's what each one is doing.

## Typed errors instead of thrown exceptions

The default TypeScript pattern for calling an external API is a function that returns a `Promise` and throws on failure. Every caller then wraps it in a `try/catch` and guesses at the shape of whatever got thrown, or lets it bubble up and hopes something upstream handles it. Nothing in the function's signature tells you what can actually go wrong.

Effect encodes failure in the type itself. An `Effect<Success, Error, Requirements>` value declares its success type and its error type side by side. [Effect's error management docs](https://effect.website/docs/error-management/expected-errors/) put it directly: "the `Effect` type captures not only what the program returns on success but also what type of error it might produce." `Effect<Data, PlatformError>` tells you exactly what a call can return and exactly what can go wrong before you've read the implementation.

In practice that means a distinct error per failure mode, not one generic catch-all:

```typescript
class RateLimited extends Data.TaggedError("RateLimited")<{
  readonly retryAfterMs: number
}> {}

class Blocked extends Data.TaggedError("Blocked")<{}> {}

class MalformedResponse extends Data.TaggedError("MalformedResponse")<{
  readonly raw: unknown
}> {}
```

`Data.TaggedError` adds a `_tag` field to each class, described in the docs as "a discriminant for the error." That discriminant is what lets calling code switch on the specific failure and decide what to do: retry a rate limit, bail on a block, log and move on for a malformed response, instead of parsing an error message string and hoping the wording hasn't changed since the last platform update.

## Validating unreliable platform responses with Schema

Typed errors handle the case where a request fails outright. They don't handle the more common case for a scraping API: the request succeeds, and the response comes back looking almost right. A field that's usually a number arrives as a string. An array that's usually populated comes back empty. A nested object that's always been there is missing this week.

[Effect's Schema module](https://effect.website/docs/schema/introduction/) is built for exactly that boundary: converting untrusted external data into a value you can actually trust the type of.

```typescript
import { Schema } from "effect"

const TikTokPost = Schema.Struct({
  id: Schema.String,
  likeCount: Schema.Number,
  caption: Schema.String
})

const result = Schema.decodeUnknown(TikTokPost)(rawResponse)
```

If `rawResponse` doesn't match the shape, decoding fails with a structured `ParseError` instead of a silent `undefined` three functions downstream. That failure composes with the typed errors above — a schema mismatch goes through the same path as a rate limit or a block. Catch shape mismatches at the boundary, not at whatever line of business logic touches the bad field first.

## Retrying with Schedule instead of a hand-rolled loop

Every platform adapter used to have its own retry logic: a `for` loop, a `setTimeout`, a retry count checked by hand. Small differences crept in every time someone copied it into a new file, and there was no single place to see what the actual retry behavior was.

Effect separates retry timing from the effect being retried. A `Schedule` describes when and how often to retry, and `Effect.retry` applies it:

```typescript
const backoff = Schedule.intersect(
  Schedule.exponential("10 millis"),
  Schedule.recurs(5)
)

const result = Effect.retry(fetchFromPlatform(id), backoff)
```

[Effect's retrying guide](https://www.effect.website/docs/v3/error-management/retrying) explains that `Effect.retry` "takes an effect and a Schedule policy, and will automatically retry the effect if it fails, following the rules of the policy." `Schedule.exponential` sets the backoff curve, and `Schedule.intersect` combines it with `Schedule.recurs(5)` so the delay grows but the retries still stop after five attempts: 10ms, 20ms, 40ms, 80ms, 160ms, then done. Because errors are typed, the retry can also be conditional on which failure happened: retry a `RateLimited`, don't bother retrying a `Blocked`. The schedule is a value we can read, test, and reuse across every adapter, not a loop buried separately in each one.

## Bounded concurrency instead of unbounded Promise.all

A single request to Social Fetch can mean fetching from several platforms at once. The obvious way to do that in plain TypeScript is `Promise.all`, which runs everything at full concurrency with no way to cap it and no clean way to cancel the rest if one branch fails.

`Effect.all` takes a `concurrency` option that controls exactly that:

```typescript
const results = Effect.all(
  platforms.map((p) => fetchFromPlatform(p)),
  { concurrency: 4 }
)
```

The [concurrency docs](https://effect.website/docs/concurrency/basic-concurrency/) lay out the options: a number caps how many effects run at once, `"unbounded"` runs all of them concurrently, and the default with no option set is sequential. `Effect.forEach` takes the same option. The difference from `Promise.all` isn't just the cap. Effect's structured concurrency means that if one branch fails or the whole request gets interrupted, the others are cleanly shut down instead of left running with nothing listening for the result.

## Testable platform clients with Context and Layer

Each platform adapter is a service: something with a `fetch` method that talks to a real API in production and something else entirely in a test. Effect represents that as a `Context.Tag`:

```typescript
class TikTokClient extends Context.Tag("TikTokClient")<
  TikTokClient,
  { readonly fetch: (id: string) => Effect.Effect<Data, PlatformError> }
>() {}
```

A `Layer` supplies the implementation. `Layer.succeed` for something with no dependencies of its own, `Layer.effect` when the implementation needs to pull in other services first:

```typescript
const TikTokClientLive = Layer.effect(
  TikTokClient,
  Effect.gen(function* () {
    const http = yield* HttpClient
    return { fetch: (id) => http.get(`/tiktok/${id}`) }
  })
)

const TikTokClientTest = Layer.succeed(TikTokClient, {
  fetch: (id) => Effect.succeed(fakeResponseFor(id))
})
```

[Effect's layers documentation](https://effect.website/docs/requirements-management/layers/) describes this as keeping service interfaces dependency-free while managing how they're wired up through composition instead of manual mocking. Swapping `TikTokClientLive` for `TikTokClientTest` in a test doesn't touch anything in the code that calls `TikTokClient.fetch`, because that code never knew which implementation it was going to get.

## What Effect costs you

None of this is free, and the people who've actually written about using Effect are pretty consistent about where the cost shows up.

On the [Hacker News thread for Effect's launch](https://news.ycombinator.com/item?id=40682149), the praise and the criticism came from people who'd clearly used it, not just skimmed the homepage. Commenter srhtftw called it "a clear path out of callback-hell that is more reliable than promises and async/await" after a year of use, and mind-blight liked that it "forces these errors into compile-time checks instead of runtime checks." But apozem compared the learning curve to RxJS and called it "brutal" when teaching it to a team, and johnfn argued that swapping `for...of` for `Effect.forEach` and `await` for `Effect.runPromise` costs real readability and debuggability. noname120 and ivanjermakov both flagged the same specific pain: stepping through Effect code in a debugger means stepping through Effect's own internals, not your business logic, and "execution flow is not obvious."

[Rob Bertram's write-up of his first week with Effect](https://robbertram.com/blog/effect-ts-first-impressions/) lands in a similar place from a solo-developer angle. He liked that "side effects, errors, resource lifecycles, and concurrency patterns are all modeled at the type level," and that the strictness rules out whole categories of runtime bugs. But he was candid that he'd "have a really hard time suggesting this technology at work," because most TypeScript developers coming from React or plain Node don't have the functional programming background the library assumes.

We didn't run into all of that equally. The debugging complaint is real, stepping through a retry schedule is not as legible as stepping through a `for` loop. The onboarding cost is real too, someone new to the codebase needs to learn `Effect.gen`, `Layer`, and `Schedule` before they're productive in the request layer specifically, even if the rest of the codebase is plain TypeScript. What made it worth that cost for us is narrow: a request layer that talks to a dozen APIs we don't control, where every one of the problems above (untyped failures, hand-rolled retries, unbounded fan-out, and now unvalidated responses) was already a real, recurring source of bugs before we changed anything. That's a specific shape of problem, not a blanket case for rewriting a whole backend in Effect.

## Where this leaves us

Typed errors, Schema at the boundary, Schedule-based retries, and bounded concurrency solve the same problem: explicit, reusable failure handling instead of copy-pasted conventions. Effect fits a request layer that talks to a dozen APIs we don't control. That's why the request layer stays on it.