Humans have big brains and two thumbs, and we’ve used them to conquer the world. But even we admit that, in some cases, computer power is superior to human power. Take software quality assurance (QA). In many cases, testing software can be done quicker and more efficiently by using automated tests rather than with upright bipeds.
You might worry that, eventually, the machines will take over and eliminate humans from the QA process. So why does QA need humans, when humans need OSHA-approved working conditions and time-consuming bathroom breaks?
Because in some cases, the reverse is true: Humans are better at QA than is automation. We have five situations to prove it.
QA engineers, take note: Your robot overlords won’t replace you any time soon. Just keep feeding the machinery with the souls of programmers who create bugs, and you should be fine.
Humans can judge user experiences
Didn’t you love the internet of the 1990s, where walls of white text on a black background were broken up only by flashing lights and underlined/bolded/italicized words? Me neither. When we developed tools to make the internet a prettier place to read fanfic, we used those tools to the best of our human ability.
Computers certainly can’t discern if a page is pleasing to the eye. Humans can. “Humans are using these tools, and when you look at a banking application, you look at its aesthetic value. That’s why American Express Gold Cards are gold,” says Sondra Moyer, a senior test engineer. “I don’t know how much you can automate a review of a printed page to see if that’s what people will like.” A human can.
With user acceptance testing (UAT), people try the system and reveal that the programmers’ idea of easy is actually button-click hell. For bonus points, UX professionals and QA engineers – humans, again – build an observation lab that employs eye-tracking to see exactly why the user isn’t clicking the meant-to-be obvious button and instead keeps logging out. By accident. Presumably.
Automated tests can detect if a website’s link is broken. But they can’t determine how to easily navigate and use your application. “Designing a piece of software might make perfect sense to the architects and programmers,” says Russell Myers, former QA analyst. But if the average user finds the software confusing or unhelpful, he adds, “That’s going to piss them off, and it’s really going to affect usability.”
There’s nothing on Earth better at judgment than humans. Not even Skynet — and Skynet created Judgment Day.
Humans are better at spotting errors
When you want to run repetitive tests across multiple web browsers on different operating systems, you want a computer. When you want creativity, you want a meatbag. I mean, a human.
QA tester Arbel Grossman encountered an example of such a situation when he was testing a screen reader, a program that reads text aloud. The text on the page displayed the day and month, a separator bar, and the amount of a financial transaction…but the screen reader only spoke the date and the financial transaction, sans bar.
“The computer said, ‘Jan13743 dollars.’ The computer couldn’t distinguish which part was the date and which part was the amount of the transaction,” says Grossman.
As far as the test was concerned, there was no error, because computers only spot the errors they’re designed to catch. Grossman says, “If the text is wrong or a word is misspelled, a computer wouldn’t know that; a computer doesn’t know the meaning of what it sees.”
This proves that certain flaws, such as a font change or a misplaced image, are only obvious to human QA testers. The Matrix wouldn’t even notice a glitch.
Humans write better error messages
Automated testing can weed out software defects in the time it takes to run a script. But even though a developer’s code may be error-free, experienced testers still recognize when a problem exists between keyboard and chair (PEBKAC), that is, the issue originates with a user.
What happens if a user duplicates someone else’s login name? What if they decide to use “Password” as their password? You can’t shoot the user with a lightning bolt through their Wi-Fi connection (because the IEEE standards committees don’t know how to have fun).
However, you can send a human to the rescue. QA professionals may recognize a common foible and can recreate error messages for every inaccuracy. A good message details how a user can fix the error of their ways.
Even error messages should be quality assured. According to Sheri Byrne, a head of accessibility, “It requires a human to test whether an error message explains to the user what to do next.”
This means when your business needs someone to craft a useful, actionable step-by-step guide to righting a user’s wrongs, Robby the Robot need not apply.
Humans can save time and money
Automation exists to relieve humans of time-consuming and repetitive tasks. These include loading the same assets across different browsers on different operating systems on different platforms again and again and ahhh! My eyes! But it sometimes is less expensive and time-consuming to run tests manually.
There isn’t much point in automating one-off scenarios. Before incurring the cost of diverting a developer off of a current project, quality engineer Jason Sendaula asks himself the following questions: “You’re going to need developers to…automate those tests. How hard is it to automate? How many hours am I going to need to put in to make this work? How long is it going to be working for?”
After adding up the costs, you might determine that the projected time savings isn’t worth the development effort of making Tron fight for the users. I mean, automating a test.
If your developers are crunching to meet a deadline — or bug-hunting your latest release — it may be more efficient to go old-school with your test. Insert an old-school human here. Then, with foresight for the next iteration, invest in QA automation software.
Humans communicate
Software development is sometimes spread across multiple teams. This complicated work is made even more complicated if one team is outsourced and another team is off-shored. This makes it difficult to synchronize project work.
But it is not impossible. “One reason that QA needs humans is to communicate,” says Byrne.
It turns out that human mouths aren’t just for snacking. They’re useful for forming words. “QA engineers not only find bugs; they provide valuable feedback on documentation and training,” Byrne says. “In the end, the QA engineers are users, and they can make enhancement requests like any other user.”
And if you’re very good at QA and at being a human, you can provide feedback while snacking.
End result: Automation is wonderful. It can save humans from the tedium that makes our minds wander away from work, and did I leave the lights on in the kitchen? But humans are a required part of the process, and they will be for the foreseeable future.