I’ve been a bit of a pizza snob ever since my first job making pizza at the age of 15. I’ve always had a pretty low opinion of Domino’s Pizza®, so I was very impressed when I saw Domino’s recent efforts to reinvent both their pizza and their brand identity by tackling their problems head on.

When I first saw their ads and online video, which you can see in Figure 1, I was amazed that they had chosen to use actual user feedback that denigrated their product. It’s extremely rare to see companies admit their own faults so bluntly and publicly, but the best way to deal with the elephant in the room is to acknowledge it, confront it, and ultimately overcome it. We have yet to see whether Domino’s can successfully overcome their negative brand perception, but the message they’ve sent has been enough to motivate me—someone who had previously sworn never to eat Domino’s Pizza again—to give them another try.

Domino’s Brutally Honest Re-branding Effort

Their rebranding effort has brought up quite a few memories for me, because I’ve seen the pain on the faces of designers and engineers when users eviscerate the products they’ve spent months or years developing. I also know convincing stakeholders they need to rethink a product can be a painful and costly endeavor. However, one of my most salient memories involves interacting with research participants who already had their minds made up about the quality of a product before they ever laid eyes on it. People’s preconceived notions can be another elephant in the room—a barrier to achieving accurate and actionable feedback on a concept or design.

The Angry Elephant

“To show these participants we were truly ready to make serious changes, we had to … be brutally honest about the failings of the existing design.”

A short time ago, my business partner, Bryan McClain, and I were giving a talk about communication and research at an ecommerce company, and someone asked a very interesting question: What was our most difficult interaction with a research participant? The question instantly brought to mind this memory: Some time ago, we were asked to conduct a focus-group session with a specialized group of participants. In this particular case, the focus-group participants had previously tested builds of the product, and their feedback had been overwhelmingly negative. Our client, to their credit, had already decided to redesign the product from the ground up and realized they needed direction. The company had chosen to bring in a fresh research team, so they hired our company to engage users, find out exactly what was wrong with the current design, and provide guidance on how they could redesign the product.

I’ll never forget the experience of walking into the room where the focus group was to take place to greet this group of participants and being met with glares bordering on open hostility. We explained that we were performing research the company intended to use to improve the design of the product, but that wasn’t anything they hadn’t heard before, and they were understandably skeptical. Our initial attempts to extract meaningful information fell flat as the participants just spat out short comments, then quickly clammed up. It was obvious we would have to win them over and show them we were on their side. Vague phrases such as we recognize that there are problems with the design were not enough to assuage their doubts. To show these participants we were truly ready to make serious changes, we had to do what Domino’s has done in their recent advertisements: be brutally honest about the failings of the existing design. So, after going through an extemporaneous list of the product’s design shortcomings, we were finally able to show the participants that we understood their pain, cared about their feedback, and were ready to make changes.

This kind of experience interacting with user research participants emphasizes two essential parts of successfully communicating with participants: establishing objectivity and building alliances.

Establishing Objectivity

“When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design.”

When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design. If you have any biases or a personal connection with the product design, participants may feel uncomfortable giving you negative feedback, because they unconsciously sense your disappointment. For this reason, we commonly advise that designers not lead user research for products they’ve designed. If a dedicated user researcher is not an option, we typically advise companies to have a different designer do the user research—one who has no personal attachment to the project. When we are interacting with participants, very early on in our conversations with them, we make a point of letting them know that we did not design the product, so anything they might say will not hurt our feelings. This gives participants the freedom to offer negative feedback without the social stigma of being mean or hurtful.

Building Alliances

“A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team.”

When taking on the role of user researcher—whether you are actually a user researcher or usability professional or are a designer, engineer, product manager, or in an another profession—it is important to take to heart that, for that period of time, you are on the side of the users. A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team. In doing this, a researcher builds alliances with participants, so participants feel that the researcher—and thus, the company—values their feedback and will use it to influence the design of the product. While establishing objectivity can help participants feel free to speak openly, building alliances can really motivate them to participate actively in a user research session. It’s another strategy you can use to maximize the amount of actionable user feedback you’ll get from a research session.

The Happy Elephant

“A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product….”

Of course, preconceived notions about the quality of a product can also swing in the other direction. A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product, because participants have already made up their minds to love the product before they even lay eyes on it. We’ve tested products with participants who were brand loyalists and overlooked numerous faults in a product’s design. In one particular instance, a participant had difficulty performing a majority of the tasks during a usability test, but reported that he loved the design, so we shouldn’t change a thing. It was only through our pointing out and inquiring about specific areas of difficulty he had encountered that he finally admitted that there were aspects of the user interface that should change. There are two methods that come to mind for dealing with this type of issue: anonymous testing and prioritizing objective data.

Anonymous Testing

“If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so.”

The easiest way to overcome such brand bias—whether user research participants have positive or negative perceptions of a company’s brands—is to simply avoid it entirely. If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so. Often, by not associating a brand with a product or concept, we can acquire a more accurate understanding of users’ reactions to the product itself.

As consultants, we can accomplish this by performing user research under our own company name. We can bring participants to our offices and use prototypes and builds that show no company logos. It can be much more difficult to establish anonymity if we are testing at our clients’ offices, but, of course, it isn’t always necessary to do so. We usually do suggest some type of anonymous testing when we’re working with companies that are trying to overcome a strong negative or positive brand perception. However, anonymous testing isn’t necessary for companies that are less well-known or have a more neutral brand perception.

Prioritizing Objective Data

“In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research.”

Objective data refers to the information user researchers collect through direct observation of participants, while subjective data refers to data researchers collect through participants’ verbal reports. In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research. For example, during a usability test, we note specific instances in which a participant has difficulty with a device or user interface. Then, when later interviewing the participant, we typically make statements like this: It’s great that you loved the Web site, but you seemed to have difficulty doing XYZ. A participant usually responds by acknowledging the difficulty, then making comments about their expectations or suggesting improvements to the user interface.

There are also ways to objectively document emotional responses such as frustration, disgust, and happiness. Learn to recognize such emotional cues, document them, and inquire about them when doing any type of product research, including generative user research such as ethnography. Typically, objective data is more accurate than subjective data, so it is important to prioritize data accordingly when findings are inconsistent.

Conclusion

“Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them to use.”

Participant biases, both positive and negative, can get in the way of obtaining useful feedback when performing concept or usability testing. There are a number of ways to work around these biases, as follows:

• Establish objectivity. Show participants that you are not personally attached to a concept or design, allowing them to provide feedback without hurting your feelings.
• Build alliances with participants. Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them.
• Use anonymous testing. Filter usability testing through a subsidiary or a contracted agency to entirely avoid any brand bias.
• Prioritize objective data. What users actually do is more important that what they say they do.

If you take these lessons to heart and apply them appropriately, they can be enormously helpful in making some of your more difficult interactions with users much more productive.