When Bright Data is the wrong solution

I want to be fair to the managed vendors up front, because this gets read as a hit piece and it isn't one. Bright Data, Zyte, Apify, Oxylabs, the managed scraping APIs in general are genuinely good at the specific problems they were built for. I reach for them myself. The mistake isn't using them. The mistake is using them for the thing they're quietly bad at, because the sales page doesn't draw that line and you find out after you've built on top of them.

So here's the line, from the buyer's side.

What they're actually good at

Managed vendors win when your problem is breadth without depth. You need a hundred different sites, none of them especially defended, none of them business-critical on their own, and you do not want to operate a hundred crawlers. Paying someone else to keep all hundred alive is a great deal. Your engineers do something more valuable than babysitting scrapers.

They also win as a fallback tier and as a fast start. When I build a system, a managed unblocking API is often a legitimate rung in the fallback stack: try the cheap path, then the heavier path, then hand the genuinely nasty requests to a vendor that specializes in them. And on day one of a project, a managed API gets you data this week while you decide what's worth building for real. Renting before you build is good discipline.

If that's your situation, stop reading and go buy the thing. The rest of this is for the situations where buying is the expensive choice wearing a cheap coat.

Where it goes wrong: the source that is your business

The clearest case to build rather than buy is the source that is your product. If your company sells data from one or two sources, those sources are your moat, and you've just rented your moat from a vendor who rents the same access to your competitors. You don't control the cost, the freshness, the coverage, or the roadmap. When the vendor raises prices or the source changes, you find out when your product breaks, and your fix is a support ticket.

I've watched a data company discover that its entire differentiation was a Bright Data endpoint that three competitors also used. They weren't a data company. They were a reseller with a UI. The build-versus-buy decision on your core source isn't a cost decision, it's a "do we have a business" decision.

Where it goes wrong: cost at volume

Managed scraping is priced per request or per record, and that pricing is wonderful at low volume and brutal at high volume. The per-unit rate that's a rounding error at ten thousand records a month is a serious monthly number at fifty million. At ZoomInfo scale, processing a billion pages a month, no managed vendor's per-request pricing is even in the conversation. You build, because at that volume owning the infrastructure is cheaper by an order of magnitude.

The trap is the slope. Volume creeps up, the bill creeps up with it, and there's never a single day where buying obviously became the wrong call. So nobody revisits it. The vendor invoice grows into a number that would have funded the in-house build several times over, and it's been growing for a year. If your managed-scraping bill has a comma in it and your volume is still climbing, run the build math. It's probably overdue.

Where it goes wrong: depth on a hard, specific target

Managed APIs are generalists. When you need deep, reliable coverage of one heavily-defended target, with specific fields, specific freshness, and a low tolerance for gaps, a generalist vendor will get you most of the way and stall on the last mile. The last mile is the part that matters and the part they can't prioritize for you, because you're one customer among thousands and your weird edge case isn't on their roadmap. A purpose-built crawler for that one target, tuned to its specific defenses, beats a general solution every time on the source you actually care about.

A decision framework you can use in a meeting

Buy when:

• You need many sources, each individually low-stakes
• The data isn't your core differentiator
• Volume is low or spiky and you don't want fixed infrastructure
• You need something live this week and can refine later
• You want a fallback tier for the genuinely nasty requests

Build when:

• The source is your product or moat
• Volume is high enough that per-unit pricing has gone insane (run the actual math, don't eyeball it)
• You need deep, reliable, specific coverage of a hard target
• Freshness and coverage gaps directly cost you revenue
• You need control over cost and roadmap because the business depends on it

Most real stacks are a blend, and that's the right answer. Buy the long tail, build the core, and use managed APIs as a fallback rung inside your own system rather than as the system. The failure I see is binary thinking: "we're a buy shop" or "we build everything," when the actual answer is source-by-source and changes as you scale.

The vendors won't tell you where their product stops being the right call, because that's not their job. Knowing where that line sits for your sources, at your volume, is most of what a good build-versus-buy review produces.