Usability Testing, Expert Reviews, Surveys, and Continuing Assessments
Introduction
Designers can become so entranced with their creations that they may
fail to evaluate them adequately. Experienced designers have attained the
wisdom and humility to know that extensive testing is a necessity. If feedback
is the "breakfast of champions" then testing is the "dinner
of the gods." However, careful choices must be made from the large
menu of evaluation possibilities in order to get a balanced meal.
The determinants of the evaluation plan include:
- stage of design (early, middle, late)
- novelty of project (well defined vs. exploratory)
- number of expected users
- criticality of the interface (life-critical medical system vs. museum
exhibit support)
- costs of product and finances allocated for testing
- time available
- experience of the design and evaluation team
The range of evaluation plans might be from an ambitious two-year test with
multiple phases for a new national air-traffic control system to a few days
test with 3-6 users for a small internal accounting system. The range of
costs might be from 10% of a project down to 1%.
A few years ago it was good advice to get ahead of the competition by focusing
on usability and doing testing, but the rapid growth of interest in usability
now means that failure to do so would be risky. The dangers are not only
that the competition has strengthened, but that customary engineering practice
now requires adequate testing. Failure to perform and document testing could
lead to failed contract proposals or malpractice lawsuits from users when
errors arise. At this point it is irresponsible to bypass some form of usability
testing.
One troubling aspect of testing is the uncertainty that remains even after
exhaustive testing by multiple methods. Perfection is not possible in complex
human endeavors, so planning must include continuing methods to assess and
repair problems during the lifecycle of an interface. Second, even though
problems may continue to be found, some decision has to be made about completing
prototype testing and delivering the product. Third, most testing methods
will appropriately account for normal usage, but performance with high levels
of input such as in reactor or air-traffic control emergencies is extremely
difficult to test. Development of testing methods to deal with stressful
situations and even partial equipment failures will have to be undertaken
as user interfaces are developed for an increasing number of life-critical
applications.
The Usability Professionals Association was founded in 1991 to exchange
information among workers in this arena. The annual conference focuses attention
on forms of usability evaluations and provides a forum for exchanges of
ideas among the more than 4000 members.
 |
Please send
comments and suggestions to the Booksite
Director
Last Updated:
11 December 2002
|
