Why tiny errors matter when you're automating assessment at scale.
Vaidy Bala
December 07, 2025
After months of anticipation, we finally opened up the quiz generation feature to Unni and Amrutha - the two high-school tutors we've been building Spark with.
We need two kinds of people in our lives:
The ones who trust us blindly.
And the ones who support us by investing time to make our work better.
Unni is the first kind. He clicked a few buttons, saw no explosions, and waved the green flag.
But Amrutha? She went through everything with a fine-comb. And she found something off.
The answer she selected (4th option marked in red - see image above) is the correct one - but the AI marked a different option as right.
Here's what happened:
We have a maker AI that generates questions and answer options.
And a checker AI that verifies what the maker produces.
The maker AI created a sensible question and good options…
…but marked the wrong answer as correct.
Apparently, the carbon chain length does not change.
(Who knew? Certainly not me.)
When Amrutha reported this, I felt two things:
Gratitude. Without her, I would never have spotted this. My chemistry expertise ends at spelling "Chemistry."
Dread. Some bugs are easy. You give the same input, you get the same output. AI bugs? They're like humans - unpredictable.
My brain immediately went:
"How am I going to reproduce this?"
"Where in our 100+ steps video processing pipeline is the issue hiding?"
Deep breath. Dive in.
Thankfully, within 48 hours, the culprit surfaced:
Spark has internal maker-checker QA across the 100+ step pipeline (because AI hallucinations are real).
But I had forgotten to add a checker in one section of the quiz generation flow.
A few more hours later, the fix was in place.
That's the good news.
The great news? Quizzes are now live - and already making an impact.
Lessons from this episode
Building AI products is an endless loop of polishing and improving.
Early adopters are priceless - especially when they're subject-matter experts.
A robust foundation pays off. It turned out that our 100+ step pipeline is significantly more robust than even I'd given it credit for. Otherwise, finding this issue would have become a wild goose chase.
If you'd like early access to Spark and set up your training content in one click, reply and I'll set you up.