Reinventing user testing with a relaxed environment

By Christine Del Castillo

We’re now in the thick of the “design and experiment” phase of a new prototype we’re developing here at Fidelity Labs. Week after week, my team has been digging deep to find out what really matters to our target market. And our process is a rapidly executed series of cycles: Ask questions, get feedback from real people, iterate on the feedback.

Put another way: Done? Do it again. Do it better!

However, in our last cycle of user input, we didn’t just evolve our prototype, we evolved our entire approach to user testing. Our previous sessions were one-on-one interviews with testers held during working hours in conference rooms. But Bill Stewart, who leads design strategy on our small team, was eager to shake things up.

What on earth is a “Whine and Dine”?

Bill previously organized “Whine and Dine” events while at IDEO, which he referred to as “unfocus groups.” These events are designed to be the antithesis of precisely moderated, seriously buttoned-up focus groups in beige rooms with two-way mirrors. He proposed a new format to us, painting the picture of it as a treat: We’d invite potential Fidelity customers to a relaxed evening in a lounge-like space with music, wine, and cheese boards. The ambience would encourage a generous and generative mood.

“Instead of inviting them to be critical, we’re inviting them to help us. They can find the holes, and tell us how to improve,” said Bill.

The team was willing to try it, and eager to take it to the next level. Ryan, who is co-leading experience design for our team, suggested separate stations to demo different versions of our prototype. Meredith, our other experience design lead, suggested time limits for each station, similar to speed dating. The setup was a dynamic way to introduce our concepts to our testers.

The game plan

We transformed the Fidelity Labs Design Thinking workspace into the lounge environment. In addition to an hors d’oeuvres table and drink station, we constructed five stations, wherein five different parts of our prototype were shared with five testers. Each tester had 10 minutes at each station to give us their snap judgments. When the testers concluded their journey, they enjoyed a short break and casual conversations.

Then everyone reconvened on the couches to discuss poster-sized printouts of our prototype. We reviewed each concept again, and encouraged the group to share their thoughts with us and with each other: the good, the bad, and especially the ugly. Some of the side conversations—such as the mom who explained retirement plans to a twenty-something tester—were both informative and heartening. It was exceptionally candid and natural feedback.

We also passed out score sheets—a kind of secret ballot, with only two questions on the page: “Please rate this concept overall” and “Would you tell a friend?” This was a quick and effective way to gauge their overall interest in the product we’re developing.

Lessons learned

Our primary goal was to boost engagement and, in that sense, the event was a wild success. This new format—with its quick bursts of activity, interspersed snacking, socializing, and group discussion—was noticeably more exciting for our testers. On paper, an after-work event can seem exhausting. But in practice, this format kept our team’s energy level high.

A future version of this event—a prototype in itself—may involve experimenting with other aspects of the event, such as sending in one tester at a time to experience each station in a specific order, from first to last. Also, trying an external location, such as a restaurant, could be another way to create an inviting atmosphere to get more candid and spontaneous input from our testers.