This is something I wrote more than a year ago. Although I have not worked much on web UI testing for some time I believe points it makes are still valid.
Abstract
Here I try to address the problems in automating web user interface tests as a part of continuous integration process and propose some improvements to lower the cost of automated UI tests.I suggest a top down approach where web user interface testing will be written with minimal consideration of html elements but with more focus on business requirements.
Basics
Test coverage is one of the most important indicator of good quality software. An enterprise software typically must have the unit, integration, acceptance and stress tests. However writing and maintaining good quality tests has it’s costs as well as it’s benefits.Unit Tests
Unit tests typically test the most basic, independent function blocks of the application. They run typically fast. A good functioning unit test must fail when the function it tests changes behaviour.There are a good number of tools for unit testing which helps the development and maintenance like the JUnit, NUnit. Also modern IDE’s support running of these tests without much effort.
Integration Tests
Integration tests, test how the dependent blocks of the application work together. Most modern enterprise applications are designed using more than one often complex frameworks and libraries. Integration test typically run longer than unit tests and more costly than unit tests. Often mock objects are created to be able to test certain parts of the application.Acceptance Tests
Acceptance tests, are aimed to test directly on the final product and it’s often the most complex and costly to maintain. Different products for both desktop and web testing is available. java.awt.Robot, Watir and Selenium are examples.Stress Tests
Stress tests, are aimed to make sure the final product meets the non-functional requirements. Non functional requirements of the applications are just as important as the functional if not more. Producing load and making sure the system performs well under load is especially important.Web Acceptance Tests And Reducing It’s Costs
Directly testing a web application is quite challenging. Most basic approach to acceptance testing may be considered “bottom up” trying to fulfill the business cases by thinking in terms of the basic html elements. Typical problems with this approach are:Bottom Up Approaches to Web Acceptance Tests
Referencing the web elements
Referencing the elements is required in order to perform functions, clicking, entering text and such, directly from the browser. Although lots of alternatives exists to select elements, referencing id’s, certain text or xpath queries, they are cumbersome and hard to maintain. XPath approach is further harder to understand.Non Data Centric
Most of the test frameworks don’t come with basic data populators needed. So most tests are run with specific data against an empty databases or they have to be cleaned up.Easing the Bottom Up Approach
To overcome the cumbersomeness of writing acceptance tests tools such as Watir and Selenium, have recorder software which would record the users browser actions and produce scripts. Although this eases the initial creation of the test scripts does not resolve the maintenance and data population issues.Top Down Approach
Web tests should be thought of what might be called “top down” approach. Which is minimal thinking of html elements more on business requirements and if they are met.Most web application are made of forms and applications mostly performs by filling and submitting these forms.
Minimal amount of referencing the elements should be made through the test page preferably just for the forms which would be filled and submitted.
Most of the user data required to fill out forms could be filled out using either guessing, based on id’s, or elements neighbour labels, or through extra invisible information to the user. These extra information could be easily integrated with modern component based web frameworks. Also they could easily be switched on for testing and off for production use.
This approach would remove the burden of maintaining change of id’s and places of html elements.
Meta-Data Required for Finding and Filling Html Input Elements
Typical meta-data for an html element would be related with identifying it and what kind of data it will be required to fill it. For a name field meta-data would be, an type id, an actual html id allowing the system to find it, it’s validation requirements like, if it’s required and what it’s max length is.Recognizing the Errors
Every enterprise application has standard way’s to inform the users of errors through html elements.System must be able to recognize errors. Both for testing if validations work and trying out new values where needed.
Integrating the Meta-Data System
This meta-data required by top-down approach could be integrated in component based, declarative web frameworks.A typical JSF, standard user interface framework for Java, component holds nearly all the information required by the system;
<tow:inputtext id="name" value="#{new.name}" label="#{lbl.name}"
required="true" maxLength="30" />
An extra information could be added for determining the components type id.
Component meta-data required could be either generated as invisible html elements embedded in the page. Or source code could be supplied for the test executor which would parse the source and extract the meta-data it requires.
Code Samples for Both Approaches
Bottom Up Script
browser = Watir::Browser.start "http://localhost:8181/petclinic/"browser.link(:text, "Find owner").click
browser.link(:text, "Add Owner").click
browser.button(:value, "Add Owner").click
if !browser.text.include?("is required ")
puts "required validation failed"
end
browser.text_field(:id, "telephone").set("asdf")
browser.button(:value, "Add Owner").click
if !browser.text.include?("must be all numeric ")
puts "numeric validation failed"
end
browser.text_field(:id, "firstName").set("Ortega")
browser.text_field(:id, "lastName").set("Junior")
browser.text_field(:id, "address").set("Argentina")
browser.text_field(:id, "city").set("Buenos Aries")
browser.text_field(:id, "telephone").set("999")
browser.button(:value, "Add Owner").click
if !browser.text.include?("Owner Information")
puts "Failed to create Owner"
end
Top Down Script
tow = TOW.newtow.goto 'http://localhost:8181/petclinic/'
tow.click "Find owner"
tow.click "Add Owner"
tow.testForm FormSpec.new({:id => "owner", :allRequired => true})
tow.assertNoErrors
Both scripts are for creating an owner on a petclinic application. Petclinic application is a enterprise example application freely available over the web.
Bottom up script is generated mostly as it’s using TestWise recorder. The script as it’s, is highly coupled. Change in any one of the 9 access selectors, firstName, lastName and such, will cause the script to fail. Furthermore running the script twice could cause the uniqueness constraints of the db fail.
Top down script uses a sample framework, which is much less coupled with both the data and html elements. It only address 3 html elements. Further it could be run without the fear of running into data uniqueness issues since it uses data dictionaries to populate the data. It also does test both the required and numeric validations.
No comments:
Post a Comment