Automated testing tools were supposed to make things easier.
Getting them to execute pre-scripted tests — instead of relying on
QA pros to carry out tests by hand — saves time and money. Moving
to automated testing is a no-brainer, right?
In theory, the case for automated testing tools makes sense. And yet, in years of writing about software testing, I haven’t heard a single test manager say that automation saves time and money and boosts software quality.
What I hear over and over again are variants on the same theme: Test automation projects are far more challenging than they initially appear. They require scripting skills, which some testers lack. And while the initial stage of setting up tools and writing test scripts is the most demanding phase of an automation project, the most successful projects never really go on autopilot. They are never done.
Don’t get me wrong, I am not building a case against automated testing. Nor am I saying that automation doesn’t make sense for the long run. Virtually all QA pros I talk to believe it does, and they also say there is a certain inevitability about making the move to automation. Certain tests are better left to computers rather than to human testers.
In this installment of Quality Time, I discuss four reasons why automated testing projects are challenging. Understanding these challenges can help QA managers avoid some common pitfalls, accurately assess their readiness for automated testing and set more realistic expectations for upcoming projects.
Reason 1: Test automation projects are software development projects.
Failure to fully grasp this notion is a recipe for disaster. You can’t set up automated testing tools without team members who can write code. As Lisa Crispin, a resident SearchSoftwareQuality expert, explained in her tip on devising a test automation strategy, “automating tests is software development. It must be done with the same care and thought that goes into writing production code.”
This poses a significant hurdle for testers who lack coding skills. That challenge is best met by mastering those skills, not by shying away from projects that demand them. “Learn how to script tests — learn a scripting language like Ruby,” Coveros CEO Jeff Payne said, addressing an audience of software testers at STAREAST 2013 earlier this year. Companies are looking for testers who can automate, and those without coding skills will ultimately be left behind, he said.
Reason 2: You can’t succeed at automated testing until you succeed at manual testing.
The success of any test project, manual or automated, ultimately hinges on testing the set of things most likely to deliver the highest-quality software to the user. If you don’t know what to test, you are not ready to automate the testing process, according to expert Mike Kelly, in our story, When to use manual vs. automated software testing tools: “I see a lot of teams focus on automation, as a cost-lowering technique, when what they really need to focus on first is testing the right things.”
Kelly’s commonsense rule reminds me of a rule art students are often asked to follow in painting 101: No abstract paintings until you prove your ability to paint realistically. In other words, don’t tackle the variation on the theme until you have mastered the theme itself.
Reason 3: Test automation does not mean testing with automation added.
A key misconception about test automation is that it’s easy, said test expert Hans Buwalda, addressing an audience of software testers in the “Lightning Strikes the Keynotes” session at STAREAST 2013. “But I am still waiting for my easy project,” said Buwalda, chief technology officer for software testing service firm LogiGear in San Mateo, Calif. He believes automated testing is harder than manual testing. You can’t simply add automated tests to an existing test process. Instead, you have to rethink your entire approach. Which tests are best automated? Which ones should remain manual? Answering those questions requires creative thinking, he said. “Don’t lose your creativity [when you move to automated testing],” he told the audience.
Reason 4: Avoid automated testing autopilot.
When teams successfully complete the initial stage of setting up automated testing, there is a tendency to keep on running the same tests over and over again. Test expert Robert Galen of RGalen Consulting Group in North Carolina, said this is a big mistake. Running more tests faster does not result in higher-quality software. Better software is the result of running the right tests and continually re-evaluating which tests are the right ones, Galen explained in our story, Test automation ROI commoditizes testers, hurts test organizations.
In other words, the most successful automated testing projects are never complete. You keep on asking which tests will help produce the best software, and then script them accordingly. This approach helps ensure success with automated testing tools. But when you start running the same old tests in autopilot mode, trouble lies ahead.
In theory, the case for automated testing tools makes sense. And yet, in years of writing about software testing, I haven’t heard a single test manager say that automation saves time and money and boosts software quality.
What I hear over and over again are variants on the same theme: Test automation projects are far more challenging than they initially appear. They require scripting skills, which some testers lack. And while the initial stage of setting up tools and writing test scripts is the most demanding phase of an automation project, the most successful projects never really go on autopilot. They are never done.
Don’t get me wrong, I am not building a case against automated testing. Nor am I saying that automation doesn’t make sense for the long run. Virtually all QA pros I talk to believe it does, and they also say there is a certain inevitability about making the move to automation. Certain tests are better left to computers rather than to human testers.
In this installment of Quality Time, I discuss four reasons why automated testing projects are challenging. Understanding these challenges can help QA managers avoid some common pitfalls, accurately assess their readiness for automated testing and set more realistic expectations for upcoming projects.
Reason 1: Test automation projects are software development projects.
Failure to fully grasp this notion is a recipe for disaster. You can’t set up automated testing tools without team members who can write code. As Lisa Crispin, a resident SearchSoftwareQuality expert, explained in her tip on devising a test automation strategy, “automating tests is software development. It must be done with the same care and thought that goes into writing production code.”
This poses a significant hurdle for testers who lack coding skills. That challenge is best met by mastering those skills, not by shying away from projects that demand them. “Learn how to script tests — learn a scripting language like Ruby,” Coveros CEO Jeff Payne said, addressing an audience of software testers at STAREAST 2013 earlier this year. Companies are looking for testers who can automate, and those without coding skills will ultimately be left behind, he said.
Reason 2: You can’t succeed at automated testing until you succeed at manual testing.
The success of any test project, manual or automated, ultimately hinges on testing the set of things most likely to deliver the highest-quality software to the user. If you don’t know what to test, you are not ready to automate the testing process, according to expert Mike Kelly, in our story, When to use manual vs. automated software testing tools: “I see a lot of teams focus on automation, as a cost-lowering technique, when what they really need to focus on first is testing the right things.”
Kelly’s commonsense rule reminds me of a rule art students are often asked to follow in painting 101: No abstract paintings until you prove your ability to paint realistically. In other words, don’t tackle the variation on the theme until you have mastered the theme itself.
Reason 3: Test automation does not mean testing with automation added.
A key misconception about test automation is that it’s easy, said test expert Hans Buwalda, addressing an audience of software testers in the “Lightning Strikes the Keynotes” session at STAREAST 2013. “But I am still waiting for my easy project,” said Buwalda, chief technology officer for software testing service firm LogiGear in San Mateo, Calif. He believes automated testing is harder than manual testing. You can’t simply add automated tests to an existing test process. Instead, you have to rethink your entire approach. Which tests are best automated? Which ones should remain manual? Answering those questions requires creative thinking, he said. “Don’t lose your creativity [when you move to automated testing],” he told the audience.
Reason 4: Avoid automated testing autopilot.
When teams successfully complete the initial stage of setting up automated testing, there is a tendency to keep on running the same tests over and over again. Test expert Robert Galen of RGalen Consulting Group in North Carolina, said this is a big mistake. Running more tests faster does not result in higher-quality software. Better software is the result of running the right tests and continually re-evaluating which tests are the right ones, Galen explained in our story, Test automation ROI commoditizes testers, hurts test organizations.
In other words, the most successful automated testing projects are never complete. You keep on asking which tests will help produce the best software, and then script them accordingly. This approach helps ensure success with automated testing tools. But when you start running the same old tests in autopilot mode, trouble lies ahead.
No comments:
Post a Comment