"A bug costs 100 times more to fix in production." It turns up on conference slides and in most shift left testing pitches, and almost nobody who quotes it has seen the data behind it. As far as anyone can tell, there isn't much.
That doesn't mean late bugs are cheap. Anyone who's watched a feature pass development, wait a week for a test cycle and come back with a defect list knows the cost: the developer has moved on, the context is gone, the release slips. Worse is the bug a customer finds, which pulls support, product and engineering into the same thread.
Shift left is about moving checks earlier so that most defects are caught minutes after the code is written, not weeks later. For CTOs it's mostly an economic call, so it's worth knowing what the evidence actually says before you plan the change.
Where the "100x" figure really comes from
In July 2021, The Register reported that researcher Laurent Bossavit had traced the widely cited 100x chart to course notes from an IBM "Systems Sciences Institute", which turned out to be an internal training programme. No data was available to support the figures, and any original data was no more recent than 1981. The chart spread through Roger Pressman's 1987 textbook and was then repeated as if it were a study.
The same article mentions a 2016 study of 171 projects that found time to resolve issues at different stages was usually not significantly different. Hillel Wayne, quoted in the piece, said the research tentatively points toward later bugs costing more, but that it depends on what you mean by "bugs" and "more expensive", and that the costly bugs tend to be design issues.
The most cited economic study is older and broader. NIST's June 2002 planning report on inadequate software testing infrastructure estimated the annual cost to the US economy at about US$59.5 billion, and that feasible improvements could recover about US$22.2 billion of it. It also said over half of software bugs weren't found until downstream in development. Treat those as what they are: US estimates from 2002, extrapolated from two industry case studies.
So we don't quote a multiplier. The mechanism is what holds up. A defect found while the code is still open in the developer's editor gets fixed in minutes by one person. The same defect after release means triage, a support conversation, a hotfix, a redeploy and sometimes data repair. Design flaws found late are the worst case, because the rework spreads.
Daily merges make late testing unworkable
DORA's continuous integration guidance says teams do better when developers merge into trunk at least daily, with automated test runs kept to about 10 minutes. A test phase at the end of a sprint can't keep pace with daily integration.
Security has gone the same way. DORA's page on pervasive security describes shifting security left to cut the cost and delay of fixing problems found late. It recommends automated security tests in the CI pipeline, and suggests security review can gate the move from design to development.
AI-assisted coding adds volume: more code per week means more change to verify, which only works if most of the checking runs automatically and early. We look at that in testing AI-generated code.
The shift left testing stack, check by check
Shift left isn't a tool. It's a set of checks, each placed at the earliest point where it can catch its class of defect.
| Check | When it runs | What it catches |
|---|---|---|
| Acceptance criteria with concrete examples | Before code is written | Misunderstood requirements and design gaps |
| Linting, type checks, static analysis | In the editor and on commit | Whole classes of simple errors |
| Unit tests | On every commit, locally and in CI | Logic errors in business rules |
| Contract tests | In CI for both consumer and provider | Breaking API changes between services |
| Dependency, secret and static security scans | In CI on every change | Known vulnerable packages, leaked keys, risky code patterns |
| API and integration tests with test doubles | In CI | Wiring and data handling errors |
| A few end-to-end and performance smoke tests | After deploy to a test environment | Broken critical journeys and obvious slowdowns |
A cheap first step most teams skip: run the linter and type checker in a pre-commit hook and again in CI with warnings treated as errors. In TypeScript, turning on strict in tsconfig.json catches a surprising number of null and undefined bugs before a test even runs.
Unit tests
Unit tests are the base. DORA's test automation guidance says most errors should be caught by unit tests, and when a later stage or an exploratory session finds a bug, you add a unit test so it's caught earlier next time. Keep that habit for a few months and your detection point moves left on its own.
Contract tests
Most products are several services, a mobile app and a few third-party APIs. Broken integrations are a classic late defect because they only show up when everything is deployed together. Pact describes contract testing as checking each side of an integration point in isolation, so you can confirm services work together "without having to deploy the world first".
Ham Vocke's Practical Test Pyramid recommends consumer-driven contracts: consuming teams write tests capturing what they need, and the providing team runs them continuously in its pipeline. A provider change that would break a consumer fails in the provider's build, before release. With Pact, the usual gate before deploying is the can-i-deploy check against the Pact Broker.
Pipeline checks
The pipeline is where shift left becomes enforceable. DORA says every commit should trigger a build and automated tests, fixing a broken build takes priority over other work, and a change that can't be fixed within a few minutes should be reverted. It also says developers should get feedback from acceptance and performance tests daily.
Early defect detection: four numbers to track
You don't need an industry multiplier to know whether this is working. Track your own:
- Where defects are found. The share caught before merge, in CI, in test environments and in production. You want that moving steadily toward the earlier stages.
- Escaped defects. Production bugs per release, and whether each now has a test at the lowest level that reproduces it.
- Feedback time. Minutes from commit to test result, against DORA's 10-minute guide.
- Time to fix a red build. Long-running red builds mean the pipeline is being ignored.
Two things shift left doesn't mean
It doesn't mean getting rid of testers or handing everything to developers. DORA keeps manual exploratory and usability testing running throughout delivery, and has testers working alongside developers on the automated suite.
It also doesn't mean testing only early. Production monitoring and solid incident response still catch what tests miss. Shift left changes where most defects get found, not whether you keep looking.
How JRC Software moves your checks earlier
Our quality engineering team starts with data you already have. In week one we tag your last few months of bugs by where they were found and what kind they were, using the issue tracker and pipeline history. That usually shows one or two defect classes that keep escaping, such as integration breaks between two services, or validation logic nobody unit tested.
Then we add the cheapest check that would have caught each class earlier: unit tests on the riskiest business rules, contract tests between services, security scans in CI, and a short set of end-to-end checks. We pair with your developers so they own the tests, and keep pipeline gates fast enough that people don't route around them. If your suite is already large and slow, our test automation strategy guide covers what to cut first.
Resist the urge to add a long new gate to every pull request on day one. Start with the checks that run in seconds.
If bugs keep reaching UAT or production, talk to a quality engineer and we'll show you where to move your checks first.
Frequently asked questions
What is shift left testing?
Shift left testing means moving testing earlier in delivery, so defects are found close to when the code or design is created. In practice it combines clear acceptance criteria, unit tests, contract tests, static and security analysis, and automated pipeline checks on every commit, with a small set of end-to-end tests after deployment.
Is it true that bugs cost 10 or 100 times more to fix in production?
The popular multipliers have weak sources. The widely cited 100x chart was traced to old IBM internal training notes with no supporting data. Many practitioners still accept that some late defects, especially design flaws, cost much more to fix, but the ratio varies a lot. Measure where your own defects are found instead of quoting a multiplier.
Does shift left testing replace a QA team?
No. It changes what testers spend their time on. Developers write and maintain most automated tests, while quality engineers design the test strategy, build contract and integration coverage, curate the suite, and lead exploratory and usability testing. DORA's guidance describes testers and developers working together rather than in separate phases.
Where should a team start with shift left?
Start with the defects you already have. Find where your recent bugs were caught, then add the cheapest check that would have caught each one earlier: often a unit test on a business rule, a contract test between two services, or a dependency scan in CI. Keep pipeline feedback to about 10 minutes so developers keep using it.
Software Testing
Talk to a quality engineer
Tell us where you're stuck and we'll come back with practical next steps.
Get in touchSources
- The Register: Everyone cites that 'bugs are 100x more expensive to fix in production' research, but the study might not even exist
- NIST: The Economic Impacts of Inadequate Infrastructure for Software Testing, Planning Report 02-3 summary
- DORA: Capabilities: Continuous integration
- DORA: Capabilities: Pervasive security
- DORA: Capabilities: Test automation
- Pact: Introduction to contract testing
- Martin Fowler: The Practical Test Pyramid (Ham Vocke)