During usability test sessions, we usually ask the participants to say out loud what they are thinking. What they say ends up being important data to justify what is right or wrong about a software page design. We trust that what they say as veridical, at least as long as it is consistent with our own perceptions of the events.
But what is happening when participants talk? Are they reporting events as they occur? Are they filtering what they perceive and reporting their interpretation of it? Is their interpretation biased by what they expect to see or what they think we expect them to see?
This issue reminds me of a book, The Electric Kool-Aid Acid Test, by Tom Wolfe published in 1968. In this work of new journalism, Wolfe describes the adventures of Ken Kesey. Kesey, the author of One Flew Over the Cuckoo's Nest, was an early advocate of the use of LSD, before it became illegal. He believed that when you take LSD your brain chemistry changes and you are the closest to “being in the moment” that a human can be. He believed in something called “the gap.” When we perceive an event, is has already happened, hence the gap. There are gaps between the event and when we sense it; between the event and when we perceive it and between the event and our feelings about it. He and the people who gathered around him, called the Merry Pranksters, would take LSD to try to get as close as they could to being in the moment, that is to minimizing the gap between when an event occurs and their perception of it. They lived for living in the moment.
So if there is a gap between events and their perceptions, what is a participant in a usability test talking about? It can’t be what is happening. At best it is their memory of what just happened. Also we know that people can think many times faster than they can talk. So there is plenty of time to interpret what is perceived, especially if you slow down your talking. This gap is why cognitive psychology researchers don’t trust what people say as reflecting what is going on in their heads when they are allowed to report/interpret whatever they choose.
So what’s up with thinking aloud?During usability test sessions, we usually ask the participants to say out loud what they are thinking. What they say ends up being important data to justify what is right or wrong about a software page design. We trust that what they say as veridical, at least as long as it is consistent with our own perceptions of the events.
But what is happening when participants talk? Are they reporting events as they occur? Are they filtering what they perceive and reporting their interpretation of it? Is their interpretation biased by what they expect to see or what they think we expect them to see?
This issue reminds me of a book, The Electric Kool-Aid Acid Test, by Tom Wolfe published in 1968. In this work of new journalism, Wolfe describes the adventures of Ken Kesey. Kesey, the author of One Flew Over the Cuckoo's Nest, was an early advocate of the use of LSD, before it became illegal. He believed that when you take LSD your brain chemistry changes and you are the closest to “being in the moment” that a human can be. He believed in something called “the gap.” When we perceive an event, is has already happened, hence the gap. There are gaps between the event and when we sense it; between the event and when we perceive it and between the event and our feelings about it. He and the people who gathered around him, called the Merry Pranksters, would take LSD to try to get as close as they could to being in the moment, that is to minimizing the gap between when an event occurs and their perception of it. They lived for living in the moment.
So if there is a gap between events and their perceptions, what is a participant in a usability test talking about? It can’t be what is happening. At best it is their memory of what just happened. Also we know that people can think many times faster than they can talk. So there is plenty of time to interpret what is perceived, especially if you slow down your talking. This gap is why cognitive psychology researchers don’t trust what people say as reflecting what is going on in their heads when they are allowed to report/interpret whatever they choose.
So what’s up with thinking aloud?