Social Fetch is an API for pulling public data off social platforms: profiles, posts, comments, transcripts, across TikTok, Instagram, YouTube, X, Reddit, and a dozen others. Every one of those platforms has its own rate limits, its own way of telling you a request failed, and its own habit of quietly changing a response shape without telling anyone. A request to one platform can succeed while the identical request to another gets rate-limited, and a third comes back with a field that used to be a string and is now null.
None of that is one hard problem. It's a lot of small, repeated decisions: what counts as a failure, how long to wait before trying again, how many requests run at once, what to do when a response doesn't match what you expected. We moved the core of that decision-making onto Effect, and here's what it actually buys us, with the real APIs, the docs behind them, and what other people building with it have said, good and bad.
The request layer described here runs on Effect, the TypeScript library for typed errors, Schema, and structured concurrency.
Here's the shape of one request, fanning out to several platform adapters at once:
Each box in that diagram is a specific Effect feature. Here's what each one is doing.
Typed errors instead of thrown exceptions
The default TypeScript pattern for calling an external API is a function that returns a Promise and throws on failure. Every caller then wraps it in a try/catch and guesses at the shape of whatever got thrown, or lets it bubble up and hopes something upstream handles it. Nothing in the function's signature tells you what can actually go wrong.
Effect encodes failure in the type itself. An Effect<Success, Error, Requirements> value declares its success type and its error type side by side. Effect's error management docs put it directly: "the Effect type captures not only what the program returns on success but also what type of error it might produce." Effect<Data, PlatformError> tells you exactly what a call can return and exactly what can go wrong before you've read the implementation.
In practice that means a distinct error per failure mode, not one generic catch-all:
Data.TaggedError adds a _tag field to each class, described in the docs as "a discriminant for the error." That discriminant is what lets calling code switch on the specific failure and decide what to do: retry a rate limit, bail on a block, log and move on for a malformed response, instead of parsing an error message string and hoping the wording hasn't changed since the last platform update.
Validating unreliable platform responses with Schema
Typed errors handle the case where a request fails outright. They don't handle the more common case for a scraping API: the request succeeds, and the response comes back looking almost right. A field that's usually a number arrives as a string. An array that's usually populated comes back empty. A nested object that's always been there is missing this week.
Effect's Schema module is built for exactly that boundary: converting untrusted external data into a value you can actually trust the type of.
If rawResponse doesn't match the shape, decoding fails with a structured ParseError instead of a silent undefined showing up three functions downstream. That failure is itself just another typed error, so it composes with everything in the previous section: a schema mismatch can go through the same typed-error handling as a rate limit or a block, instead of needing its own bespoke validation layer bolted on separately. For an API whose entire job is trusting responses from services we don't control, catching a shape mismatch at the boundary instead of at whatever line of business logic happens to touch the bad field first is the difference between a clean typed error and a 2am page.
Retrying with Schedule instead of a hand-rolled loop
Every platform adapter used to have its own retry logic: a for loop, a setTimeout, a retry count checked by hand. Small differences crept in every time someone copied it into a new file, and there was no single place to see what the actual retry behavior was.
Effect separates retry timing from the effect being retried. A Schedule describes when and how often to retry, and Effect.retry applies it:
Effect's retrying guide explains that Effect.retry "takes an effect and a Schedule policy, and will automatically retry the effect if it fails, following the rules of the policy." Schedule.exponential sets the backoff curve, and Schedule.intersect combines it with Schedule.recurs(5) so the delay grows but the retries still stop after five attempts: 10ms, 20ms, 40ms, 80ms, 160ms, then done. Because errors are typed, the retry can also be conditional on which failure happened: retry a RateLimited, don't bother retrying a Blocked. The schedule is a value we can read, test, and reuse across every adapter, not a loop buried separately in each one.
Bounded concurrency instead of unbounded Promise.all
A single request to Social Fetch can mean fetching from several platforms at once. The obvious way to do that in plain TypeScript is Promise.all, which runs everything at full concurrency with no way to cap it and no clean way to cancel the rest if one branch fails.
Effect.all takes a concurrency option that controls exactly that:
The concurrency docs lay out the options: a number caps how many effects run at once, "unbounded" runs all of them concurrently, and the default with no option set is sequential. Effect.forEach takes the same option. The difference from Promise.all isn't just the cap. Effect's structured concurrency means that if one branch fails or the whole request gets interrupted, the others are cleanly shut down instead of left running with nothing listening for the result.
Testable platform clients with Context and Layer
Each platform adapter is a service: something with a fetch method that talks to a real API in production and something else entirely in a test. Effect represents that as a Context.Tag:
A Layer supplies the implementation. Layer.succeed for something with no dependencies of its own, Layer.effect when the implementation needs to pull in other services first:
Effect's layers documentation describes this as keeping service interfaces dependency-free while managing how they're wired up through composition instead of manual mocking. Swapping TikTokClientLive for TikTokClientTest in a test doesn't touch anything in the code that calls TikTokClient.fetch, because that code never knew which implementation it was going to get.
What Effect costs you
None of this is free, and the people who've actually written about using Effect are pretty consistent about where the cost shows up.
On the Hacker News thread for Effect's launch, the praise and the criticism came from people who'd clearly used it, not just skimmed the homepage. Commenter srhtftw called it "a clear path out of callback-hell that is more reliable than promises and async/await" after a year of use, and mind-blight liked that it "forces these errors into compile-time checks instead of runtime checks." But apozem compared the learning curve to RxJS and called it "brutal" when teaching it to a team, and johnfn argued that swapping for...of for Effect.forEach and await for Effect.runPromise costs real readability and debuggability. noname120 and ivanjermakov both flagged the same specific pain: stepping through Effect code in a debugger means stepping through Effect's own internals, not your business logic, and "execution flow is not obvious."
Rob Bertram's write-up of his first week with Effect lands in a similar place from a solo-developer angle. He liked that "side effects, errors, resource lifecycles, and concurrency patterns are all modeled at the type level," and that the strictness rules out whole categories of runtime bugs. But he was candid that he'd "have a really hard time suggesting this technology at work," because most TypeScript developers coming from React or plain Node don't have the functional programming background the library assumes.
We didn't run into all of that equally. The debugging complaint is real, stepping through a retry schedule is not as legible as stepping through a for loop. The onboarding cost is real too, someone new to the codebase needs to learn Effect.gen, Layer, and Schedule before they're productive in the request layer specifically, even if the rest of the codebase is plain TypeScript. What made it worth that cost for us is narrow: a request layer that talks to a dozen APIs we don't control, where every one of the problems above (untyped failures, hand-rolled retries, unbounded fan-out, and now unvalidated responses) was already a real, recurring source of bugs before we changed anything. That's a specific shape of problem, not a blanket case for rewriting a whole backend in Effect.
Where this leaves us
Typed errors, Schema at the boundary, Schedule-based retries, and bounded concurrency all point at the same thing: an API that depends on a dozen services we don't control needs failure handling that's explicit and reusable, not implicit and copy-pasted. Effect gives each of those a typed, composable answer instead of a convention someone has to remember to follow correctly every time. That's the specific problem it's good at, and it's why the request layer is staying on it.