One of the first things I do when coming to any company is to implement an automated test framework for the consistent development of Unit Tests.
5.2.0.1. CompanyTests Project
The project should be self-contained to allow for maximum portability between all developers, testers, and anyone else we can convince to write tests. That means it should be a simple matter of checking the project out of SVN and opening it with Eclipse.
5.2.1. Sample Test Run
The simplest way to verify that your environment is configured correctly is to run a couple of tests. There are two types of tests stored in the project: system scans, and unit tests. Since the purpose differs, the way they will typically be used is different. It is a good idea to run through each of the two types of runs.
5.2.1.1. Sample 1: Run smoke tests
Tests can be grouped through the use of XML configuration files that select specific tests to run, and pass parameters into the system.
The Smoke Tests push information into various parts of the system and check for problems. They are not a detailed check but scan large portions of the system.
- Open: CompanyTests > suites > smoke.xml
- Check configuration parameters
- SampleSize: 1.1
- BaseURL: http://www-staging.cms.example.ca
- In Package Explorer (left hand menu), right click on “smoke.xml”
- Select “Debug As” > TestNG Suite
In the console you should see “[TestNG] Running:”. Next to the console tab, there is a “Results of running suite”, this should have a list of tests that have been run, are running, and have failed.
5.2.1.2. Sample 2: Run siteSearch
When working on a particular component, the developer should keep a copy of the tests open, as they make changes to the system, they can run the tests associated with the changes they are making to ensure the test and the code actually agree with one another.
- Navigate: CompanyTests > src > ca.company.web.test.components.company > sitesearch.java > testSearchWroking
- Right Click on “testSearchWorking”
- Select “Debug As” > TestNG Suite
5.2.2. Project Layout
There are four main sections to the test packages:
+
|
Components
|
ca.company.web.tests.components.*
|
-
|
Templates
|
ca.company.web.tests.templates.*
|
+
|
System
|
ca.company.web.tests.crawlers.*
|
+
|
Utilities
|
ca.company.web.tests.*
|
Each of these represents a “testable” area, with the exception of the Utilities which are helper classes used to manage the tests in general.
Within each of these areas, the project is laid out to mirror the development project. In particular, for each component path in Dev, there is a corresponding component path in CompanyTests. This allows a developer to easily switch between the two code bases.
5.2.3. Writing Tests
Once a test has been documented, there is certainly an opportunity to automate this test. Automation is faster and more reliable, so we should attempt to automate every test.
Before writing the actual test steps, it is necessary to get a browser instance. Locally, we have a helper routine for getting browser instances (BrowserPool). BrowserPool and the drivers it returns are sensitive to setup and teardown requirements and will take care of much of the setup and releasing of the browser instances.
It is also best to set up any constants we are going to use right away. This is to match the pattern found in the documentation.
- search - the text we are going to search for
- suggest - the text we expect to get back
- pageUrl - the URL of the page we are going to use for testing.
@Test(testName="rm1242")
public void testSearchSpellCheck() {
final String search = "webteam";
final String suggest = "web team";
final String pageUrl = baseUrl + this.pageUrl + "?q=" + search + "&stype=main";
try(BrowserPooledDriver pooleddriver = browsers.borrowObject()){
WebDriver driver = pooleddriver.getDriver();
// actual test steps go here
}
}
|
At this point, we are ready to actually automate the test.
For this, the documentation offers excellent pseudocode. The step-by-step procedure has been tested by humans running it, and is very detailed. Really, it is code for than runs on a “MeatBag Processor Unit”.
try(BrowserPooledDriver pooleddriver = browsers.borrowObject()){
WebDriver driver = pooleddriver.getDriver();
// STEP 1: GOTO: http://example.ca/search.html?q=webteam
// RESULT: search suggests "Did you mean: web team"</li>
// RESULT: "web team" should be a link
}
|
At this point, I would strongly suggest running the code. This would be a really convenient time to find out that you have made a mistake in the administrative code. Much more convenient than after you have put a bunch of test code in place.
To run the code:
- Find the method in the “Package Explorer” or “Outline” view of Eclipse.
- Right-click on the test
- Select “Debug As” -> TestNG
You should see it kick in and run the test. Hopefully you get a green icon!
Now that we have the baseline in place, we can move on to actually automating the test steps. Our description can easily be translated into automated steps (all one of them), and the checks for results we are expecting.
try(BrowserPooledDriver pooleddriver = browsers.borrowObject()){
WebDriver driver = pooleddriver.getDriver();
// STEP 1: GOTO: http://example.ca/search.html?q=webteam
driver.get(pageUrl);
// RESULT: search suggests "Did you mean: web team"</li>
elem = driver.findElement(By.xpath("//div[@class='spelling']/a"));
Assert.assertNotNull(elem, "Presence of new search link");
// RESULT: "web team" should be a link
elem = driver.findElement(By.xpath("//div[@class='spelling']/a"));
Assert.assertEquals(elem,suggest,"Spelling suggestion made");
}
|
We use the Selenium Webdriver to operate a web browser instance. In our example above, we can see the driver does a “findElement” call, this looks up the webpage element (any HTML tag). For a parameter, you need to call one of the methods of the “By” class, which is how we address the element we are attempting to look up. In this case, we are attempting to find an “anchor” tag, by its xpath.
It is worth noting that Selenium is not the only way to gather data about our environment. In strict UnitTesting, we would be directly calling the methods on each class we write in our library. In the case we are checking header information, Selenium can’t help, we need to turn to other HTTP communication mechanisms. Perhaps we want to test a terminal application, we could use a command streamer to act as our interface.
Once we have obtained the element we can run some Assertions on it. Assertions are the falsifiable statements about the data we have gathered. TestNG gives us an Assert class that contains several assert methods that allow us to make different kinds of statements about the data. Every assertion should have a textual description associated with it. In the first case, we are going to assert that we found the element at all (did not get null); the second case checks to see that it contains the text we are anticipating.
Now would be a good time to run the test again.
Assuming you got a green-light from the test, it is worth taking a moment to look back on what we now have.
- Automated Test - obviously we have created an automated test. Faster and more reliable.
- Manual Test - using JavaDoc, we are able to maintain steps for the test to be performed and reviewed by non-technical individuals. If automation were not available, or not possible, we still have a test plan.
- Regression Library - we have a complete regression library. This library can be used to review tests over time. When a major release is performed, we will have a list of known issues that need to be checked against. Stakeholders have a listing of everything that is being checked and can highlight areas that are being overlooked. This also gives the foundation for effort estimation as we can identify how long tests take to run and compare that to the number of tests required for a given run.
- Complete Specifications - The tests in a regression library offer a complete description of the specifications of the system. If the specifications need to change, there is a source of information for understanding what the original functionality was, and what ramifications may be present.
