Software Testing

How utterly remiss of me to not have done a post about my paid profession!

I ran into the lovely Toby from Disqover the other day. Toby was my trainer when I did the foundation and the Test Analyst ISTQB courses (this is a certification in testing proficiency. I managed to pass the Test Analyst Test and that was a world of pain – I’m not too good with exams!).

I did a painting of one of our training sessions, which I’d totally forgotten I’d done:

Test training day

Test training day

We are learning Boundary Value Analysis, which is something that I use all the freaking time.

Those courses were excellent. I was already a test analyst when I did them, the foundation course put labels and names to things that all testers do naturally, and the second course gave a number of tools and methods on how to further test. I generally just zen/go by feel these days, which is known in the industry as ‘experience based testing’. You just kind of know where it’s going to fail. You smash it till it does.

01011149

Testing to me is a mixture of process and artform. I am not the sort of tester who likes to follow rules or processes, though I have found you do have to be careful in setting up coverage and tick off what you’ve done – I like to question things and break things and fiddle and play and do dreadful things to people’s hard work. The Frenchman who I work with gave me the compliment once of ‘You are VERY GOOD at breaking things!’. And I’m chuffed at that, I like to smash code and processes and go ‘ok where is this going to fail’ and get in there and make it fail.

The picture above is how i ended up feeling about one part of the system I became very at one with. Testing it involved getting into it’s head and feeling your way using your spidey-sense and working it out by that method as much as you work it out by scripting or design.

The automation testers

The automation testers

My company has gone all out with automation testing. We have very rules driven systems – lots of interacting rules and everyone on the system is an exception. It’s a bit mad sometimes. There are a lot of good people doing a lot of automation of the testing, especially the regression suite (which is run over code to make sure we have not broken anything unexpected). I turned out to be not so good in automation as it’s pretty much programming, and I’m way better at destroying programming than I am at creating it.

What do i look like, the reply fairy?

What do i look like, the reply fairy?

Very often in the testing process you need clarification around something, which is when you contact the developer or the analyst or the SME (subject matter expert). I once didn’t get a reply from one of my devs and I went to find out why, his comment was “What do I look like, the reply fairy?”.

YES. YOU DO.  And here is your portrait!

Sold - Project meeting

Sold – Project meeting

It’s much better to get together and have a meeting about these things.

I did this painting on a system replacement project. Not a lot of the people in the painting felt flattered by it, but I thought it was a good likeness of everyone.

sold - two days till release

sold – two days till release

My current project has been under the pump for months to get things finished and has been a bit panicky and hard core for a while now. Release date is soon. It’s as ready as it’s going to be and we’ve done a good job. All projects get merged together and then tested together to make sure they all play nice together – this is  pre release testing. All releases turn into chaos just before due date, this is a pretty accurate visual description of the process, including the headband dude who is about to throw up in fear.

Sold - we found a production defect

Sold – we found a production defect

Then of course there is when a big defect gets into production. This is both always unwelcome  – and usually expected to be found – there is usually something. Testing mitigates risk, it does not remove all bugs – it cannot without infinite time and infinite resources. You do what you can and usually we do a good job. I know, for example, my current project is going to burn in the firey hell for about 2 months as it’s an entirely different way of doing business (real time processing instead of batch) and the unknowns are huge – and we don’t know how it’s going to work as a live system. Once real users start using real data, is when you go ‘oh wow i didn’t know it was going to do that’ – or COMBINATIONS of factors combine to explode in all sorts of interesting ways that as a tester you could try and predict but not always predict.

As a tester you kind of know whether your product is going to be ok. I know my current one is ok in one aspect, but the new infrastructure around it has done some unpredictable things, and this makes me deeply uneasy about what we are going to find when we go live.  But we simply cannot mimic a real system in a test environment with what we have to work with. So. We call that ‘production testing’.

It’s frustrating as a tester in one way, because you are judged only on your failures. If a release goes in smoothly, the assumption is that the work was easy and you didn’t really add value because there are no defects there, no matter how shit it was when the code was delivered for testing (defects come not just in code but also flawed design and missing bits that no one thought of). If a release goes in with a few bugs that were not picked up in testing, the fingers all get pointed and the ‘WHY DID YOU NOT FIND THIS’ questions start and you have to justify not looking at something that you didn’t know could be there and not finding something that you’d have to do 8 unlikely combinations of data/processing to find. you might have done all these things but not in the exact order the bug then turned up in, so on, so forth . Well…because we didn’t find it, we are not nasa and we dont spend 4 million and 2 years testing every release! We will add that to things to check next time and learn from that. That’s all you can do.