Get the best results out of your interviews with a four-step analysis

Getting valuable insights from your qualitative research is harder than it sounds. If you thought interviews and user tests are easier than ‘hard numerical science’, you’re wrong!

This is not an interview guide. I’m assuming you already did the hard work: you set your research goals, confirmed them with your stakeholders, wrote the interview plan, recruited the right participants and did the interviews / user tests. This is about what happens after.

You’re staring at your notes wondering how the @€£$# are you going to extract meaningful conclusions out of this mess. It’s easy to fall into research traps during your analysis: confirmation bias (seeing what you expected to see), remembering only the first and last participant or forgetting the context of remarks.

Here are the five steps I go through to get usable conclusions:

  1. Collect useful notes
  2. Determine pertinence
  3. Cluster and tag
  4. Draw conclusions The nice thing is: you can do all of this using free tooling. I suggest Microsoft Word, but Google Docs, Bear and Pages are personal favorites as well.

1. Collecting useful notes

In machine learning there’s the saying: “garbage in, garbage out”. It means if your starting data (your notes in our case) isn’t good, no AI-magic in the world will deliver you useful outcomes. The same goes for analyzing data by hand.

Writing good notes is hard. It doesn’t come automatically for most people. But preparation helps and you will get better at it over time. I always try to write my interview questions together with the person who’ll take notes and I do the analysis of the conversations with them as well. The best way to learn how to write useful notes is to do the analysis of a conversation.

I talk about a separate note-taker, because I’m a big fan of writing notes during the user test or interview instead of transcribing a recording afterwards. Transcribing video is a lot more work and you’ll miss important context cues. That’s why there are always three people at an interview: the person being interviewed, the experiment leader and the scribe.

Some tips for taking notes:

2. Determine Pertinence

Step 2 in our process is determining pertinence. Not everything in our notes is gold. We want to mark the relevant bits, the things that are pertinent to our situation.

Paste the notes of all participants underneath each other in one file, and use a big header to separate one participant from another. Highlight the (parts of) paragraphs that seem useful, and be gracious: better select too much than too little. I prefer Microsoft Word for this because as soon as you select text the mini-toolbar appears and you can easily highlight your selection.

Highlighting text in the Word mini-bar (sadly, no drinks in the mini-bar) Highlighting text in the Word mini-bar (sadly, no drinks in the mini-bar)

I use this color scheme:

Now on to the next step!

3. Cluster and tag

Warning: this step involves manual labor

Copy (not cut!) all pertinent bits to the bottom part of your file on new lines, leaving an empty line in between. Add the name / number of the participant (“P1: ”) before each bit.

This results in a colorful mess: a lot of colored paragraphs with blank spaces in between.

Now we’ll do something called an affinity diagram. Copy a paragraph and paste it together with another paragraph that seems like they belong together. Congratulations: you created a cluster. Keep adding bits to the clusters until no paragraphs are left unclustered. Use double lines to mark off a cluster.

Whenever you feel like it, name your clusters. You can give them tags or headings. Feel free to rename or reshuffle until you feel right about the clusters.

Some grouped comments. I’m seeing patterns already ;) Some grouped comments. I’m seeing patterns already ;)

Refuse the urge to do easy grouping (like grouping all the positive things together or grouping based on the interview question), try to look for deeper patterns instead. Make clusters with multiple ‘colors’.

Tip: another nice thing of Microsoft Word is that you can do this stuff together with other researchers with the cloud package.

4. Drawing conclusions

By now you’ve wrestled with your data so much, you’re probably seeing patterns already.

Write your conclusions down per cluster and support them with quotes / video / audio. Going back to your recordings is important to objectively check if you put your findings in the right context. Tip: if you upload your video to youtube, they can automatically sync existing notes to the video.

When drawing conclusions, be critical:

I’ve shared my approach to retrieving useful conclusions from qualitative research, but that’s not end of this story! I’m really curious to hear how you do it or if you have any feedback on my methods! Please leave a comment on the article, or PM me on LinkedIn

Good luck with your research! 😇

This article was previously published on Medium: Get the best results out of your interviews with a four-step analysis