This site presents a collection of ‘maths’ problems to be solved by computer - 250+ of them and the site allows you to check your answers etc. You don’t need to be a mathematician really, but you do need to be a good algorithm designer/programmer. Haven’t tackled any of the problems yet – but I expect I’ll have a crack at a few in time.
But it also made me think about something else. Could the problems be used as ‘testing’ problems too? The neat thing about some of them is that testing them isn’t easy. Some problems have only one answer – they aren’t useful at all - there is only one test case. But others like problem 22 for example provide input files to process http://projecteuler.net/index.php?section=problems&id=22
The input file could be edited to generate variations – i.e. test cases to demonstrate the code works in general, not just a specific situation. Because some problems must work for infinite cases, simple test techniques probably aren’t enough (are they ever?)
These problem statements aren’t designed for a specific technique – it’s much closer to a real and challenging problem. Also, the algortihm used to solve the problem is a mystery - and there may be many many ways of solving the same problem. (cf screens that simply update records in a database - pretty routine by comparison). So the implementation doesn’t influence our tests - its a true black box problem.
In teaching testing, we start from the wrong end. We teach a technique, formulate a requirement and say, “prepare tests using X technique”. But life isn’t like that - the requirements come first and don’t usually tell you which technique to use! Which makes me think that maybe we could identify a set of problem statements (not necessarily ‘mathematical’) that don’t just decompose to partitions and boundaries, states or decisions and we should use these to teach and train.
So, should training (and certification) be driven by the need to solve problems rather than trot out prepared statements of folklore? Both practice and exams could consist of some written instructions on a pc and some software to test.
Isn’t this the best way to evaluate someone’s ability as a tester?
What stops us doing it? Is it because really - we aren’t as advanced as we think we are? Our test techniques will never prove correctness (we know that well) - they are just heuristic, (usually) more systematic ways of selecting? Are these heuristics really clerical aids for bureaucratic processes rather than effective methods for defect detection and evidence collection? Where’s the proof that say they are more effective? More effective - than what?
But who is looking at how one selects a heuristic? Is it just gut feel, IQ, luck?
Or is there a method that could be defined and taught? Examined? Isn’t that worth pursuing?
