The beauty of this setup is that the tester in me LOVES certain characteristics of this test set. The call is an automated one -- in this case, I know that the caller is saying the *same exact* thing every time. The tone is the same, the inflections are the same, it's the same voice source data. However, over 3 days, I've gotten 3 different transcriptions from Google Voice!
The voice mail *actually* says this:
Hello. This is the attendance office at Durant Road Middle school, calling to inform you that your child, Steven Cannan, was marked absent today. Please send a signed note upon returning to school, explaining the reason for the absence. Thank you.
And following are THREE Google Voice transcriptions ... they crack me UP!
Day 1:
Hello, this is the attendance office ed to Ron corrode middle school calling to inform you that your child. Hey, it's Dawn Cannan was marked absent today. Please send a signed note on returning to school, explaining the reason for the absence. Thank you.
Day 2:
Hello, this is the attendance office and to Ron corrode middle school calling to inform you that your child. Jeanne cannon. You're a smart ass in today. Please send a signed note up on returning to school, explaining the reason for the absence. Thank you.
Day 3:
Hello, this is the attendance office at Deron corrode middle school calling to inform you that your child G intended You're a smart ass IN today. Please send a signed note up. I'm returning to school, explaining the reason for the absence. Thank you.
I *kind of* think they did the best job on the first day :)
The voice mail *actually* says this:
Hello. This is the attendance office at Durant Road Middle school, calling to inform you that your child, Steven Cannan, was marked absent today. Please send a signed note upon returning to school, explaining the reason for the absence. Thank you.
And following are THREE Google Voice transcriptions ... they crack me UP!
Day 1:
Hello, this is the attendance office ed to Ron corrode middle school calling to inform you that your child. Hey, it's Dawn Cannan was marked absent today. Please send a signed note on returning to school, explaining the reason for the absence. Thank you.
Day 2:
Hello, this is the attendance office and to Ron corrode middle school calling to inform you that your child. Jeanne cannon. You're a smart ass in today. Please send a signed note up on returning to school, explaining the reason for the absence. Thank you.
Day 3:
Hello, this is the attendance office at Deron corrode middle school calling to inform you that your child G intended You're a smart ass IN today. Please send a signed note up. I'm returning to school, explaining the reason for the absence. Thank you.
I *kind of* think they did the best job on the first day :)
But this brings up an interesting point for me, and it's one that I have encountered several times in my career as a tester.
Roughly, I can ask this question as something like this: How can you test something that is not feasible to prove, logically or mathematically, is *correct* every time?
I can think of so many examples where testing isn't cut and dry. I started working as a tester in a company that did software that processed genetics algorithms. How did we know that the genetics algorithms were correct? At that time, I started off by having all of my testers know how to do the algorithms by hand, so that results that weren't right would just stick out to them.
At one point, I was testing a search engine. I was lucky in this case to have knowledge of the source data in a database. One approach I took here was to find objects in the source data that exhibited certain special (and easy to identify) characteristics. Then, I could perform searches that I *knew* would return those objects and look for them early in the test results. But even then, I didn't really know that the search engine was performing correctly.
Recently, I read a set of slides by Harry Robinson, talking about testing Google Maps. Here's another good example: How do you test something that has multiple right answers, in such a way that you feel confident releasing it to customers?
I've given two approaches that I have used, but I'd like to know how others have solved similar issues. Please, let's talk, I'd love to hear more from you.
