Labour represents the highest cost to any system. When paying for a person’s time, you need to compensate them for massive amounts of resources that are effectively unused (food, clothing, and shelter).
Accuracy is another issue with manual testing procedures. Humans get tired, lazy, biased, complacent, and any number of other things that will cause them to make mistakes. This is especially true when involved in highly redundant tasks: people get bored and start taking shortcuts.
There are two solutions to solving this problem. Firstly, we can clearly outline the tasks to be undertaken by the individual. By defining the process in painstaking detail, we are able to encourage people to take the necessary steps to assure quality. By explicitly listing the steps a tester must take, you reduce the ability for the individual to create excuses for themselves, or to take shortcuts. The level of detail required for this type of documentation leads us to the ability to automate testing.
Automation harnesses all the advantages that computing has offered. The benefits of computers as given in my High School “Computing” class were simple: they never get tired, they never get bored, and they never vary their process. Essentially, computers are far more effective at being meticulous when working through checklists and taking readings.
Naturally, this is contrasted against a human’s ability to be creative, the one weakness of automated testing.
Automated testing systems cannot perform tests outside of their predefined parameters. It takes a human’s creativity to identify new tests. Further, computers cannot determine human desires. At the end of the day, software is built for humans, therefore a certain amount of testing must be done to gain the “human impression”.
3.3.1. Pros/Cons
3.3.1.1. Humans are Creative
This should be used to create tests. High-order employees are hired for their ability to identify the tests that need to be created and defined. The detailed definition of tests is, like Business Analysis, a very thought intensive process. Until a test has been accurately defined, it cannot be trusted to the infinitely stupid box that sits on top of our desks. The creativity of humans is a valuable asset that is required of the test creator.
3.3.1.2. Some activities are difficult to automate
Some tasks are just plain difficult to automate. At some point, the amount of effort required to allow a computer to accurately determine whether something is successful or not far exceeds the time it would take for a human to just sit down and check it themselves.
???Include chart showing cost curves???
3.3.1.3. Regression testing is Expensive
The volume of activity involved in a complete regression test should be phenomenal (if you are doing it thoroughly). If the organization has done an effective job of defining the boundaries and operations of their system, there should be a lot of tests to perform. This means that it is basically impossible for a complete regression test to be performed without a large number of testers.
Automation overcomes this problem in a number of ways. Firstly, computers can perform tests faster than humans; also, computers can work 24 hours a day. This means that rather than a complete regression test requiring months to complete, the effort can be reduced to hours, or even minutes. Also, as the deadline approaches, the computer does not feel the pressure to start taking shortcuts to meet forced deadlines. This ensures that the readings taken from the system are accurate and represent reality.
3.3.2. Examples of Good Automation Candidates
Latency Scan
Heartbeat Monitor/Deadman’s Switch
Simple CRUD Operations
3.3.3. Examples of Poor Automation Candidates
Poorly understood or defined requirements
- the objective should be to have humans test it in an effort to determine what they are doing
- having a human work through this a few times will likely help define the requirements
User Acceptance Testing
- computers don’t have “impressions” or “feelings”
- won’t pick up on a “font just feels too small” based on context
- computers can’t test “Not Look Stupid” (my favourite test)
