The cognitive walkthrough is a usability evaluation method in which one or more evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
The focus of the cognitive walkthrough is on understanding the system's learnability for new or infrequent users. The cognitive walkthrough was originally designed as a tool to evaluate walk-up-and-use systems like postal kiosks, automated teller machines (ATMs), and interactive exhibits in museums where users would have little or no training. However, the cognitive walkthrough has been employed successfully with more complex systems like CAD software and software development tools to understand the first experience of new users.
Benefits, Advantages and Disadvantages
Who Should Be Involved?
The cognitive walkthrough can be conducted by an individual or group. In a group evaluation, the important roles are:
The cognitive walkthrough does not provide much guidance about choosing tasks that represent what real users will do (Jeffries, Miller, Wharton, & Uyeda, 1991). The 1994 practitioner guide suggests that tasks be chosen on the basis of market studies, needs analysis, and requirements, which are all second hand sources of information. Wharton, Bradford, Jeffries, and Franzke (1992, p. 387) made some specific recommendations regarding tasks:
Solutions from the cognitive walkthrough may be suboptimal. The cognitive walkthrough emphasizes solutions for specific problems encountered in the action sequence of a task, but does not deal with more general or higher-level solutions that might be applicable across different tasks.
Analyses tend to draw attention to superficial aspects of design (such as labels and verbiage) rather than deep aspects such as the appropriateness of the task structures and ease of error recovery.
The cognitive walkthrough has gone through several versions with each version an attempt to simplify the method. The original version,(Lewis, Polson, Wharton, & Riemen, 1990) was viewed as requiring substantial background in cognitive psychology (Wharton, Rieman, Lewis, & Polson, 1994) and cumbersome to apply in real-world environments. A variation of the original cognitive walkthrough incorporated detailed forms and instructions to simplify the method for practitioners who were not cognitive psychologists. However, these changes made the cognitive walkthrough procedure too laborious (and nearly as complex as the original version) for most practitioners. The 1994 version (Wharton, Rieman, Lewis, & Polson, 1994), was written as “a practitioner’s guide” and considered the primary reference for those who wanted to conduct cognitive walkthroughs.
Spencer (2000) proposed an even more simplified version, the “streamlined cognitive walkthrough,” for fast-paced development efforts. Spencer reduced the number of questions that evaluators asked as they walked through the action sequences for each task. Instead of asking the four questions of the cognitive walkthrough, Spencer simply asked these two questions:
Spencer also recommended strict ground rules (for example, "no design discussions") to keep develoment teams from jumping into design debates about the most appropriate solutions.
Spencer provides some limited discussion on the effectiveness of the streamlined cognitive walkthrough, but despite widespread use by practitioners, this variation has not received much published validation.
Data Analysis Approach
This technique involves asking well defined questions about every step of (analyst-defined) tasks based on an agreed system description. The primary data from the cognitive walkthrough are success or failure stories for each correct action in an action sequence. A failure occurs when an analyst answers “no” to any of the questions that are asked about each correct action. For each failure, an explanation based on assumptions about the hypothetical user is recorded and used to generate design solutions.
The cognitive walkthrough assumes exploratory use, but is not tailored to any particular sector. There are different versions of CW tailored to different application areas as well as the generic approach.