What is Usability Testing?

Usability testing is the observation of typical users performing tasks with a product, conducted for the purpose of determining what changes need to be made to the content, presentation, or user-interface for that product.

THE practice of Usability Testing for Websites have come to simplify the way we use the internet to conduct business, exchange information, share knowledge, or play games. It is a method of understanding the user requirements and building websites strictly to comply with the specific needs of the targetted users. This makes the user involved in the development cycle of a product. Usability testing has a direct application within instructional design when it is used as a formative evaluation technique for e-learning products.

Usability testing reduces risk and maximizes ROI by injecting realistic and accurate measures into the design and development process

A usability test has a user perform a set of tasks in a controlled environment where the user's actions and comments can be observed. A usability test usually has the following three events:

· Planning meeting
· Observations sessions
· Action meeting

A good scenario has the following characteristics:

· It situates the task within a user context.
· It establishes a goal to be accomplished.
· It has a clear ending point.

:: Benefits

The main benefit of usability testing is that it lets the designer understand how the user makes sense of the user interface within the user's context. This is part of the formative evaluation process and can inform design changes that make the interface more intuitive for the learner. The user's focus during an e-learning event should be on mastering the content, not on learning the e-learning software.

Another benefit of usability testing is that it helps reveal learning requirements that might have been otherwise missed. This applies to both the direct and indirect application of usability testing within the instructional design model. During needs assessments, users are not always able to articulate their true learning needs. Within the action context of the usability test scenarios, however, these needs can be clearly discerned.

:: Usability Test Elements

A usability test has a user perform a set of tasks in a controlled environment where the user's actions and comments can be observed. "User testing with real users is the most fundamental usability method and is in some sense irreplaceable, since it provides direct information about how people use computers and what their exact problems are with the concrete interface being tested" (Nielsen, 1993, p. 165). This way of evaluating usability is considered to be more rigorous than merely inspecting the user interface against design guidelines, where empirical evidence supporting those guidelines is scarce (Hannafin, Hannafin, Hooper, Rieber, & Kini, 1996; Nielsen, 1994).

Formal tests use a lab that allows the test to be videotaped and observed through monitors and a one-way mirror. Some labs allow additional viewing rooms for executives and other interested parties who are not directly associated with the current test. Figure 3 shows the layout of a usability lab.

In simple projects, such as a course website that has been designed by the professor, informal tests can be performed by having the designer sit with a user as the user goes through a prescribed set of scenarios.

:: Protocols

The user employs a think-out-loud protocol, that is, says out loud what he or she is thinking and interpreting from the product. Nielsen (1993) argues that the think-out-loud protocol is the "single most valuable usability engineering method" (p. 195). If a user is not talkative or if elaboration is needed, the test facilitator can use a teach-back protocol at the end of the scenario, that is, ask the user to go over what he or she did and explain it to the facilitator. During the test itself, however, the facilitator generally minimizes contact with the user. "The experimenter should normally refrain from interacting with the user, and should certainly not express any personal opinions or indicate whether the user is doing well or poorly" (Nielsen, 1993, p. 190). Sometimes, a facilitator may sit afterwards with the participant and review the tape of the session, asking the participant to talk about what he or she was doing and why. This is called a guided recall protocol.

:: Data Analysis

The data yielded by the usability test are analyzed to find themes and patterns. Because of large quantity of qualitative data generated in a usability test, a systematic method for analyzing those data must be applied. The following techniques help provide a rigorous review of the data and an efficient mechanism for identifying themes and key issues.

1. During the user session, one of the observation team members keeps a time-stamped computer log of critical events, comments, or problems that occur. Other members in the observation room can relay specific observations to the logger as well as keep their own notes. These entries are the observation notes of the logger, but are generally detailed enough to serve as an abbreviated transcript, especially when reviewed immediately after the event.

2. Immediately after each user session, the observation team holds a findings meeting in which it reviews printouts of the log and individual notes kept during the user tasks. Items that represent possible usability problems are noted on flip-chart sheets that are taped to the meeting room wall. Each problem's time-stamp is recorded alongside its description on the flip chart so that there is a direct tie-in to the original data. This information is color-coded according to each user. As different users encounter the same item, the new notation is added below the original notation for the first user who encountered that problem. In this way, problems encountered by several users stand out due to the multicolored entries.

3. After all the tests have been conducted, the data are summarized into categories, such as 'navigation,' 'response time,' 'browser issues,' etc. These categories become helpful in the solution stage, organizing problems around resources or pointing out where a single solution might be common to several problems.

4. The team meets soon after the last user session (generally the next day, to give the facilitator time to organize and type the findings) to review the list of findings and decide what features of the product or documentation to change. Team members are assigned to solutions and completion dates are established.

:: Informal Testing

Formal tests, although worthwhile for the results they yield, can be expensive and require the resources of a permanent or portable lab. Often, the assistance of a usability professional is required to manage and facilitate the test. In reality, not every instructional design project can bear the burden of these requirements.

Less formal tests can still provide a lot of valuable usability data if done correctly. These tests generally involve the facilitator, usually the person who designed or developed the website or program, sitting with a user while the user goes through the product. The following elements are critical, however, to running a useful test:

• Selection of realistic users
• Providing good test scenarios
• A non-intrusive facilitator who employs non-directive interview techniques
  

The first two items are no different for an informal test than for a formal test. Real users doing realistic tasks is the core of usability testing. The last item is particularly important in informal tests for two reasons:

• The facilitator is often the designer and therefore is not as objective as an impartial usability consultant would be.
• The facilitator is not separated from the user as he or she would be in a lab, and it is more likely that the user will ask the facilitator for guidance during the scenarios.
  

The facilitator must suppress his or her natural desire to give information or be directive and remember that the purpose of the test is to collect information. This requires the development of an interviewing style that is designed to help the user reflect on how he or she is making sense of the user interface. In many respects, it resembles the non-directive therapeutic style of the psychologist Carl Rogers, whose technique included reflecting back to the patient the feelings the patient had just described.

Next Article: Know your users