The 1st Myth of Software Testing - Anyone can test

How hard can it be?

 

If you spend long enough in, or around, an ‘average’ organisation that develops software then these may resonate with you. I would be very surprised if you haven’t heard of at least one of these (or a derivative) at some point in time.

1. Anyone can test, to a degree — Context is king

2. Testing == Automation == Quality — Just no with a side of no

3. Testers are responsible for the quality of the delivered product — Let’s go back to the future

4. QA specialists aren’t needed, developers can do the testing just fine — Again, context is king…Right up until you suffer a catastrophic failure, your reputation and share price tumbles and the whole dev department scours vacancies for dev work on Candy crush

5. A zero defect policy is achievable — The universe disagrees

1. Anyone can test software

“…”

Hmmm….this one is a tricky one to defend, mostly because it’s true to a certain extent. Apply this thinking to safety critical software, how comfortable would you feel sitting on an aircraft about to take off and the person next to you (who has already won the battle of the arm-rest) whispers “This should be ok, I tested the flight control software, I’m reasonably confident that we won’t suddenly fall out of the sky in a ball of fire”

or

A loved one is about to undergo surgery when a nurse leans in and states “I’ve had a look at the monitoring equipment software, you should be ok whilst you’re unconscious” 

or

It’s the end of the month, you’re eagerly awaiting your salary to be paid into your account, money is tight, you login (hopefully without issue and hopefully in a secure manner…but that’s ok, a person at the bank has told you it looked ok).

Your salary isn’t there…oops Now your banking details are accessible by anyone with an inkling of how to manipulate URI’s…

oh…wait, both of those do & have happened….and that testing was apparently undertaken by a professional QA team.

Oopsy.

Anyone can test software, consumers of software inadvertently and subconsciously do it on a daily basis…when you checkout you verify that the number in your shopping basket are as expected, you verify the amount tallies with your expectations, you verify that what is in your basket is what you expect to be there…and many, many, more...the question is whether you would trust that the same approach would result in the aircraft staying in the sky, the surgeon being alerted to a problem with either the equipment or the software running on it or your salary appearing in your account?

Testing to an accepted standard is highly challenging:

• Deciphering expected behaviour

• Questioning assumptions

• Planning based on business needs

• Creating scenarios

• Designing tests

• Scripting tests

• Experimentation

• Creating representative test data

• Collaborating with your peers

• Documentation and resolution

• Exploring the product

• Experiencing the product

Just a few of the tasks that require an analytical, detail oriented and exploratory approach.

• Modelling the product and the test space

• Risks

• Procedures

• Oracles

• Coverage

• Seeking and obtaining useful information

• Note-taking

• Recording and reporting

• What-if's

These skills require practice. These skills require experience. These skills require a different mindset. These skills require a skilled tester.

Context is king. 

Safety critical - you trust that the people who developed and implemented have a clue. 

You want to know that the company complies with legislation and meets or exceeds industry-wide standards.

Candy crush - meh, the worst that can happen is Bob loses progress and has to do the same repetitive levels until he hits his previous level or previous points tally he was on before (if he actually notices). 

So, yes, anyone can test. The real question is, would you stake your life, or your job, or your income on it? Most of us are somewhere in the middle.