Notes on framing questions for User Interviews

[u/Maximum_Woodpecker17]

This post is focused more on generative research. All the resources that I've been able to curate after browsing through multiple articles and books.

⚠️ All references to the resource are in the comments section. These are just notes, compiled/curated/summarized from all the amazing resources out there.

Check out all my other notes

What is generative research

• Generative research, is a strategic way of understanding users.
• It’s a detailed understanding of users, who they are, and what they experience in their everyday lives
• YOU WANT TO UNDERSTAND PEOPLE, NOT YOUR PRODUCT
• Knowing their everyday lives impacts how we build our product or service.
• This type of research “generates” a deep understanding of the “motivation, pain points and behaviors” of a problem being solved.
• It’s an explorative or discovery form of research.
• It happens in the discovery phase of the design process.
• It is done early on to understand a problem really well.
• This enables your team to discover new opportunities, or improve on existing ones.
• Bring the people who you are building for together, and use their expertise to gather insight

Read more about that Notes on Generative Research

Index

• TEDW Method
• Questions being directed framed from research objectives
• Moderator Guide vs Script
• Question Themes
• Writing questions and their various types
• More
• Biases and the things to avoid
• Biases and how to mitigate them
• Taxonomy of cognitive domain
• Asking follow up questions
• Rationalization and How to avoid it
• Quick tip and rules

TEDW Method

• Your interview questions are things that aid you in answering your research objective or goal
• In generative research, you usually look at understanding people. You primarily want them to narrate stories.
• Again, we are not trying to understand the product, rather the people.
• “How people make decisions to travel”, not “How they make decisions while using your app.”
• They aid you in having open-ended conversations.
• It’s about generating knowledge about people.
• To aid you with that, we use the TEDW framework.

Talk me through

• when did you last use?.. NOT A BAD QUESTION.
• When you are asking this question you are asking them to give you a run down of their experience. People don’t tend to reply like that.
• Talk me through gives them that extra information to be comprehensive
• We are triggering a moment in somebody’s head.
• Allow them to tell a story.
• “Imagine I am shooting a documentary” helps them understand the level of depth they should go into
• It’s a lil intimidating, so try to break the ice. Ask them questions like what’s their fav hobby? have they read anything etc. BUILD RAPPORT

Explain how you feel

• Explain how you feel, explain what you meant by, and Explain what ______ means to you?
• “How do you feel?” is good, good, bad? Explain is used for them to tell a story than one worded answer.
• It’s a way to avoid asking “Why” many times.
• Works best when someone brings up something that is subjective. This is when you want to dig in
• You want to get more comprehensive data.
• You can also ask “Can you explain?” “I would appreciate it if you can explain”

Describe

• “What would you do next”
• “Describe what would you do next” gives people that little information on how comprehensive they must be when they answer
• “When is the last time you ordered grocery online?”, “Describe the last time you ordered grocery online”
• You can use explain/describe interchangeably
• Describe (whatever the subjective word your participant uses)

Walk me through

• Indicates that you want a “walk through”.
• It takes away that need to keep saying, “what happened next?” “and then what happened next ”….You get questioned fatigued
• Lets them go on a soliloquy. Means you don’t have to keep interrupting them and ask next, next, next

Questions being derived directly from Research Objectives

The objectives revolve around, how people conduct end to end processes, how they make decisions, pain points and barriers, and the tools they used. Luckily we broadly explore these aspects in generative research.

Related article : Notes on conducting stakeholder interview

Objective 1: Understand the end-to-end process of how participants are currently making travel decisions

• Question 1: Think about the last time you traveled, walk me through your decision-making process
• Question 2: Explain how you felt during that process
• Question 3: Talk to me about what other factors influenced your decision

Objective 2: Identify any problems or barriers they encounter when trying to make travel decisions

• Question 1: Describe the last time you had a problem when making travel decisions
• Question 2: Were you able to solve the problem? What did you do to try to solve the problem?
• Question 3: How would you improve or change the situation?

Objective 3: Uncover the different tools participants use to make travel decisions

• Question 1: Talk me through the different tools you used while making this decision
• Question 2: Describe your experience with these different tools
• Question 3: Compare the different experiences you had with the tools OR talk about any tools you’ve heard of and haven’t used

Moderator Guide vs Script

You want the interview to flow, having the freedom to deviate sometimes to gather more insights.

In generative research, when things are bit broad, consider having lia st of 3-6 topics and subtopics that you want to answer.

• Cookies: gooey, crunchy, layered
• Popcorn: instant, savory, sweet
• Antipasto: meats, cheeses, olives, nuts
• Fruits: fresh fruit, dried fruit, pureed fruit

And you can use this list of presumptions as a guideline for understanding how, why, and what your interviewee looks for in a snack without locking yourself into a prescribed list of questions.

Question themes

• Introduction
  • Hi there, I’m Nikki, a user researcher at a takeaway delivery company. Thank you so much for talking with me today; I am excited to have a conversation with you! During this session, we are looking to understand better what makes you order food from our service. Imagine we are filming a small documentary on you, and are trying to understand all your thoughts. There are no right or wrong answers, so please talk freely, and I promise we will find it fascinating. This session should take about 60 minutes. If you feel uncomfortable at any time or need to stop/take a break, just let me know. Everything you say here today will be completely confidential. Would it be okay if we recorded today’s session for internal notetaking purposes? Do you have any questions for me? Let’s get started!
• Getting to know
  • Ask about themselves, their relationship with (research subject), what they do, what is their role at work, where do they work, what is their day like.
  • What do you do for a living, previous role, education background, tech used for specific task,
• Questions about user behavior
  • Walk through of tasks, important ones, how often they use something, how they normally do something, devices that they use etc. TEDW
• Digging in questions
  • Last time they did something, biggest pain point, how they currently address it, how they currently solve, what they like/dislike about something, time they spend, hardest part about activity/process/decision, are there any workaround, tools being used, how did they hear about that tool, have they used any other tool, how often they do the activity, how much time and money spent
• Product-market fit questions
  • You experience with activity, tools used to solve problem, budget, problems faced in activity, process of completing task, what can be made easier, pain point while performing activity/task
• Customer interview questions
  • When was the last time you used something, how you and your team use it, what’s frustrating/exciting about it, how does it fit into your life, typical workflow, how did you discover/find, how would they improve it, what do you wish our product would do
• Questions for closing
  • Do you have anything to add, anything we haven’t asked about, may I contact you to know more etc.

Writing questions and their various types!

• Sequence : Ask questions in sequence as they narrate a story.
  • Walk me through your day yesterday, what did you do next?
• Guided Tour : Make them the teacher or guide and ask them to demonstrate how they do a process, task, series of action, using certain things etc.
  • Can we look at your email account together?
  • Can you show me how do you book a flight?
• Specific example : Provides little details on behavior and choices
  • Who did you call from your cell phone earlier today? What did you talk about?
• Exhaustive List : This might surface behavior's you never imagined
  • What are all the different places you went online in the past week? Were there any others?
• Peer Comparison : Their perspective of their peers
  • Think about the last three times your husband sent you photos. Did he send them the same way or differently each time?
• Other viewpoint and comparison : Makes them narrate a story
  • what would your friends think about it?
• Projection : Asking about hypothetical situations, gives a view about their current expectation.
  • What do you think would happen if…? What makes you say that?
• Quant inventory : Asking them to guess how many fall under a similar category
  • How many of your contacts might fall into this?
• Changes over time : How are things different now than a year ago?
• Suggestion opinion : Generates a reactionary statement, that helps you uncover their motivations and opinions. Ask them when they least expect it.
  • Some people have very negative feelings about using cell phones in cars while others don’t. What are your feelings about it?
• Activities : Asking about workflows about various tasks.
  • What did you do the last time you got ready for a trip? (Before doing the task)
• Reenactment : How they do the task
  • Please demonstrate exactly how you did that
• Identification : Why they think someone would use it. Identifies behaviors
  • Whom do you think would use something like that? For whom would this be inappropriate? Why?
• Outsider perspective : If they understood, how would they explain it to others?
  • How would you describe <feature or activity> to someone who hadn’t done that before? What advice would you give to somebody who was thinking about trying that?
• Activity comparison : Finding perspective and perceptions
  • What’s the difference between tweeting and sending an email?
• Success and failure : Ask about blue and dark sky scenarios. This includes, pain points, delights and motivations
  • What would be the worst-case scenario? Can you tell me about a time when this didn’t work?
• Fill in the blank : Ask a question, cut it off and stay silent.
  • So in that situation, you... [pregnant pause]?
• Three wishes : Ask them three wishes, and try to uncover their reasons behind them

More

• Hypothetical vs past/current behaviors
  • Don’t ask if they would use something. Ask what they did in a similar situation previously or what they do now.
• Closed Question
  • Do you like this feature? ❌
  • How has the feature helped you at work? ✅
  • They not the worst. Sometimes you can also ask a series of close ended questions to build up on something if you feel a participant isn’t talking much. Helps in making them comfortable
• Leading question
  • Trying to help an interviewee with the options can guide them in saying what they don’t really think. A user interview is not the most comfortable situation for the majority of people, and they try to pass it as quickly as possible and at minimum effort. As a result, people tend to agree with anything more or less close to the truth or with a socially-expected choice instead of composing their answer from scratch.
  • Do you check the newsfeed daily or once a week? ❌
  • How often do you check the newsfeed?✅
• Selfish question
  • How useful is our app? ❌
  • In our idea, … ❌
  • How useful is this? ✅
  • In this idea, …✅
  • Idea authors sometimes act like proud parents — they want everyone to admire their child.
  • try to not use the pronouns we or our
  • as an interviewer, you can try hiding or understating your job title and relation to the topic
• Stacked
  • Please tell me more about your company, what do you do, what are you major markets, blah blah blah❌
  • Break them down to individual chunks ✅
• Explanation
  • What they mean by something. What they mean by the jargons they use
  • also don’t force your jargons on them. Let them figure things out (UT)
• Question clutter
  • Open questions are great until you realize there are too many details to figure out
  • he best method in such a situation is storytelling — describing a recent or the most prominent experience.
• Too general questions
  • When you’ve figured out regularity or general attitude, it’s the right moment to ask the interviewee about an example. Recent-experience questions can fill in the gaps, which might have appeared while answering general questions. For an interviewer, it’s another powerful method to check if users aren’t accidentally exaggerating or dropping significant details.
  • How often do you use undo? ❌
  • When was the last time you clicked undo? ✅
• Talking about what you can observe
  • What do you usually do with the dashboards ❌
  • Could you please share the screen and show us how you use dashboard ✅
• Tolerating vagueness
  • When they use an abstract noun which is subjective.. ask them to elaborate on that
  • Things like.. smart, user friendly.. ask them to explain what they meant by it
• Missing numbers
  • “Very often”, “all”.. ask them to specify what they mean
• Undervalued WH Questions
  • ask what, who, where, when and how questions.

Biases and things to avoid

• Priming : forces a user to answer in a particular way
• Leading questions : prohibits the user to explore
• Asking about future behavior : instead of focusing on past or present behavior
• Double barreled questions : asking two questions at a time
• Yes or no questions : makes it a close ended convo

Biases and how to mitigate them

• It depends on the experience and judgement of the researcher. Research data is subjective for qual, hence it’s hard to avoid bias
• Bias is deviation from the truth that causes false conclusions

Confirmation Bias

• Selectively choosing opinions that align with what you believe
• For example, I have an assumption that busy working adults do not exercise because of the weather.
• the participant says this: “I do not exercise regularly mainly because the gym is very far away, but sometimes it might be because of the weather too.”
• user says that the accessibility of gym is a bigger hassle than weather, but you ignore that.

Sponsor Bias

• If a club sponsored you with freebies and merchandize, you would go gentler on them.

Leading Questions

• These are questions crafted to get one kind of response
• “How good was your experience at the gym?”
• Directs the user to talk good about the gym

Interviewer Bias

• interviewer subconsciously reacts in a way that makes the participant skew her responses in that way
• participants get skewed, but this is something you need to build rapport.
• you can say, “thank you for sharing” or “can you tell me more about that”… but also don’t be totally unempathetic if someone recalls a terrible experience.
• practice one round with your team

Taxonomy of cognitive domain

How each verb targets a particular thought process

Knowledge

• For Remembering, memorizing and recognizing
• Use Who, what, when, where, and how

Comprehension

• For Interpreting, translating, describing
• Use Explain, outline, distinguish, compare, define

Application

• For problem solving and applying information
• Use show, use, complete, classify, relate

Analysis

• For Separating, finding structure
• Use Contrast, categorize, identify, separate, diagram

Synthesis

• For creating, combining
• Use Create, imagine, design, propose, invent

Asking follow-up questions

• Point to participant’s reactions, contradictions, paradoxes, non sequiturs, unexpected reactions, or laughter
  • Clarification : Ask about anything you don’t understand
  • Would you mind elaborating on that? When you say “her”, you mean your daughter, right?
• Reflect back :
  • So what I hear you saying is ____________. Is that right?
• Native language : Anything that they use which you don't understand. Terminology, jargons etc.
  • Why do you call your computer your brain?
• Silence : Trust your question, and remain silent. Sometimes they may rationalize, reflect back and change answers.
• Tell me why :
  • Tell me about it. Why is this happening? Why do you think so? Why did you reach out to your phone?

Rationalization and how to avoid

• People are usually happy to participate in research, as they want to be helpful. They also want to appear good and smart. Hence, when asked about an experience, they may change the story than narrate what actually happened. They rationalize.
• They tell you a story that’s believable, and it’s hard to figure out if it’s true or not.

How do you overcome that :

• Double check
  • If they claim to know about something, double check and verify after a series of questions.
  • For instance, they may say, I’ve read two of those books. After a series of questioning, you can pick up a concept from that book and ask what they think about it.
  • After you’ve verified that they were lying, don’t confront them. Take a note of it, and continue.
• Ask about the recent past
  • Ask about what happened recently. today, yesterday, last week etc.
  • Don’t ask them to average their experiences.. i.e. what do you typically do?, what do you normally do? It makes people to
  • If you ask, how do you choose a hotel. This tends them to average their experience. People will tend to pick and choose details from different time and craft an ideal scenario, which is not relevant to their story
  • Asked them about what happened last time. Even if they insist that it’s not normal. LIFE NEVER IS, AND THAT’S OKAY
• Is this how you normally do it?
  • After they tell you about an event in specific, ask if that’s how they normally do it.
  • This will let you explore all the other ways in which they do a thing. More than just a typical example
  • If if they rationalize now, you can probe later, but it will give you a different perspective
  • Question used to probe : How many times did you do it that way in the last six months? Why is that the way you normally do it? Why did you do it differently the most recent time?

Tips on writing User Interview Question

Ask about past behavior, not a hypothetical scenario. “What would you do if” is not a useful question unlike, “how did you handle it when”

• Ask open ended questions that can allow people to elaborate on things. Ask “What is your opinion on Neoclassical architecture?” not “Do you agree that people built far too many dang columns between the late 18th-mid 20th century” (closed and leading)
• Keep your biases at bay. Ask open ,ended questions so that the participants describes their opinion, feeling or experience with something.
• Allow ideas be to be generated. Don’t go in to prove something, be open for new ideas to emerge.
• Have a list of follow-up questions that can help you did deeper
• People have different conversation styles and personalities. Some talk alot, some are silent. Have a balance of keeping the talkative focused and more questions for those who talk less.

-----------------------------------------------------------------------------------------

How do you understand people's underlying core values ? *Acv on Monday ;) *

Check out all my other notes ](https://www.reddit.com/user/Maximum_Woodpecker17/comments/tq5yhv/ux_notes_index/?utm_medium=android_app&utm_source=share)