When the Good Test Results Lie to You
You can pass every test and still walk straight into a go-live disaster. This is the story of how we nearly did.
Key Takeaways
✅ Test scripts don’t fail when your processes are wrong — they fail when your assumptions are.
⚠️ You can duplicate the same material, across orders, suppliers… but what caused it?
❌ Data chaos doesn’t always look like chaos. Sometimes, it looks like clean stock with hidden rot.
🧠 Your system builds what the master data tells it to. If that’s wrong? Everything’s wrong.
🔍 The scariest SAP problems don’t explode. They quietly pass testing. Until the day they don’t.
If your gut says your data’s too clean to be real…
👉 Grab the FIX KIT — the mid-project rescue plan for SAP teams facing failure in slow motion.
What You’ll Get:
💡 7 short emails with field-tested insights.
🧠 “Universal AI Prompt”, the FREE eBook that fixes specs, logs & chaos before they harden.
🚀 A smarter way to work — and maybe even reclaim your weekends.
The Test Scripts Passed.
Then We Found the Duplicates.
Then We Found the Real Problem.
Everything Was Fine (Which Should’ve Been the First Red Flag)
UAT week four.
Things were holding.
Procurement flows: ✅
Goods receipt: ✅
Valuation reporting: ✅
People were cautiously optimistic — that dangerous mix of exhaustion and pride.
We’d made it through the worst of testing.
No red flags. No missing authorisations.
We were back on timeline after sprint two nearly derailed the whole thing.
The team was starting to talk about cutover plans.
The PM had even mentioned a Friday off.
That was the mood.
Right before it broke.
Then Finance Asked a Question and Ruined Everything (Bless Them)
It started in finance.
(Where all the best SAP horror stories begin.)
“Hey… why does this supplier have two active POs for what looks like the same component?”
Now, to be fair — we’d seen this kind of thing before.
Sometimes it’s just a text error.
Or someone fat-fingered the material description.
Or the test load created a weird artefact that never made it to production.
But this time?
Nope.
We checked the POs.
Two different material numbers.
Descriptions that were almost twins.
Prices that were definitely not.
They both passed testing.
They both triggered goods receipt.
They both landed safely in test inventory.
And no one noticed — because the test script didn’t care.
The business description was just background noise to the system.
From Duplicate Parts to Existential Panic in 3 Clicks
That’s when panic set in.
We ran stock reports.
Safety stock was enabled on both materials.
Which meant the system quietly built up buffer on duplicate items — assuming they were different.
The purchase history was split.
The valuation was skewed.
And had this made it to go-live?
We’d have overstocked.
Overpaid.
And probably blamed procurement for being careless.
But the real villain wasn’t the PO.
Or the goods receipt.
Or even the duplicated material.
Not yet.
We brought in the master data team.
Asked them to check if this was a one-off.
It wasn’t.
They found four more material pairs — all near-matches.
Different numbers. Different vendors. Same thing.
And the stock levels were all wrong.
At this point, we were sure we’d found it:
“Master data inconsistency. Rookie mistake. Let’s clean up and move on.”
Except…
One developer — the quiet kind, the “I’ll just check something” type — ran a consistency check across the master recipes.
And that’s when the real problem surfaced.
We Put Down the Party Hats and Picked Up a Magnifying Glass
Turns out, the duplicated components didn’t just live in the material master.
They were wired into the recipes.
Copied.
Pasted.
Adjusted ever so slightly to suit the process engineer’s last-minute request.
No central review.
No data governance enforcement.
So even if we’d cleaned up the PO data, stock levels, or material codes — the recipes would’ve re-spawned the duplicates at the next planning cycle.
We weren’t just duplicating inventory.
We were duplicating the truth about how our products were built.
And no one thought to check.
Because the recipes had passed technical validation.
Because “production testing” was scoped for something else.
Because — let’s be honest — everyone assumes the recipes are someone else’s problem.
Turns Out the Master Recipe Was the Villain All Along
Had we gone live?
- We’d be building identical components under different names.
- Our costings would fragment.
- Our production orders would bounce unpredictably.
- MRP would plan based on ghosts.
- And our post-go-live stocktake would look like a data exorcism.
And all of it — every single thread — would have led back to one unassuming mistake:
No one checked the components in the master recipes.
No one asked if the system was building two versions of the same thing.
What we changed
- We added recipe audits to the UAT closure checklist.
- Cross-functional data reviews became mandatory — yes, even if it meant dragging production planners into UAT.
- We created a tool to flag recipe-component mismatches across plants.
- We stopped treating master recipes like sacred scrolls. (Spoiler: they’re more like sticky notes with divine authority.)
FIT Framework Link: 🎯 FOCUS
This wasn’t a config error.
It wasn’t a bug.
It wasn’t even a missed test case.
It was focus drift.
We focused so hard on testing what the system does,
we forgot to validate what it knows.
And the system believed two nearly identical components were different.
Because that’s what the recipes told it.
FOCUS means asking the uncomfortable questions.
Especially the ones that start with:
“What are we assuming is fine, simply because no one’s looked?”
Caught in your own version of this?
👉 Download the FIX KIT
Not a guidebook. Not a policy.
A tactical blueprint for PMs in the high-stakes middle,
where the green test ticks don’t match your gut feeling.
Bring it into the war room.
Use it before go-live.
Before something small becomes expensively invisible.
🛠️ Fix what matters.
Start with what you’ve ignored.