Interviewing Follow-Up, Scenario-Based Evaluation

Persona Exercise
Interviews:
Hardest Parts:

• designing interview questions
• non-leading questions

Suggestions:

• open-ended questions to just get them talking
• ex. How do you usually perform this task?
• If there is an existing product like the one you are creating, get them to use it and watch them. How are they using it? What problems do they encounter?
• If the interview goes in an unexpected direction, go with it for a while and find out why.
• It's okay to ask more specific questions that are related to your project, just avoid leading them and making them feel like they are supposed to give a specific answer.
• It really helps to record your interviews, but also take notes so you can have a quick record if you don't have time to listen to the entire interview again.
• If working as a team, one person can interview and one can record. It allows the interviewer to be more focused on good follow-up questions and the person rather than writing.
  
• Interview 6 to 10 people for your capstone project.

Deriving the Persona:

• If you interview enough people, the key themes tend to come together.
• You definitely may find that there are a couple personas, but you are going to need to decide on one persona for your design at least to begin with.
  

How did you decide who to interview?

• Develop a profile: who are the general users? What user characteristics are relevant to design and testing? What characteristics should all users have in common? (ex. Do they all have to be able to use e-mail?) What characteristics will vary? How will you define them?
• Try to interview between 6 and 10 people. Less makes it hard to identify patterns, more makes the info. not as useful.
• Having trouble finding people? Ask the first few users to suggest others.
• Do contextual interviews. Go to the users' relevant environment. (home, office, classroom)

How did you decide what to ask?

• Interview as a team. One interviewer, one scribe
• Explain why you are talking to them and they can end at any time for any reason
• Ask open-ended questions that allow elaboration
• Look for opportunities to probe interesting responses
• Be specific in getting how people already do tasks: verbal - tell me about the last time you... observational - walk me through how you'd....
• You can change/add questions as needed
• Ask about contrasts: best experience? worst? likes? dislikes?
• Last question: Can I contact you for a follow up interview? (It's okay if they say no)
• Don't use yes/no questions
• Be wary of leading questions -- Yes, Prime Minister youtube clip
  
• Avoid speculative questions: (ex. Would you use an electronic calendar?) If you must ask, do so at the very end of the interview. Don't put too much stake in this answer.
• Avoid specific design questions. Look for data to inform your design. Don't expect your interviewee to be able to solve your design problems.
  

What lessons learned for Capstone project?

• It will be beneficial to interview 6-10 people.
• Develop a user profile to help you decide who to interview and what questions to ask.
  
• You may have to narrow your project to one persona to be your primary persona.
• Are you going to focus on novice, intermediate, or advanced users? If the system is optional, you can focus on whichever group you want. But, if your system is required of a larger group of varied users, you may need to focus on the novice and just put in some short cuts to help the expert move through more quickly. (For me - links to get to what they want quickly, but detailed instructions for those who need it.
• Encourage and reassure your interviewees and later your testing and usability people so they don't get nervous and feel like they're not doing a good job.
• Adjust your questions for future interviews to avoid problems encountered in earlier interviews.

Analyzing your interview data:

• Flesh out notes from interview IMMEDIATELY.
• In teams, have each member start by analyzing separately. Look for patterns that concern design priorities, design issues, and deal-breaker (show-stoppers). Reconcile key themes identified by each team member.
• Use data to identify priorities
• Then derive tasks

Now you have personas & tasks … now what?!

Scenario-Based Evaluation
What is a scenario?
•Task + Persona = Scenario
•Idealized but detailed
•Describes a specific instance of interaction
•Tries to balance generality & accuracy
–Use persona
–Use common, general tasks
–Situate use in your design
•Scenario-Based Development
–Scenario-Based Evaluation

How to write a scenario
•Describe the situation of use that people (e.g., your persona) would experience
•Write it for what your persona would want (or need) to do
–Several scenarios for common tasks
•Include specific, realistic details from your data collection
•Scenarios tell a short story
–Represent the conditions, situations, and people that you’ll be designing for

*It's okay, initially, to have your scenario be idealistic because you will analyze problems with the system and where reality deviates from the scenario later during the analysis phase.

Scenarios for Design
•Usually, a collection of scenarios are used
–Should represent key priorities of your design
•Scenarios help you perform evaluations without the users
–Cognitive Walkthroughs
•Scenarios help justify & communicate decisions

HCI Exercise #3: Scenarios
•Due next week
•May work in teams
•Steps
–Use persona you created for HCI Ex #2
–Develop two common tasks
•Use an educational technology of your choice
•If possible, use your Capstone project
–Write two scenarios of use (one for each task), describing how your persona would engage with the technology to perform the task(s).

My Group's Practice Scenario:
Striving Stuart is currently enrolled in the Earth, Space and Geophysical Science course. His teacher, Ms. Rawlings, has just finished presenting the material on the forces that cause erosion to the class and assigned groups of students to work on a related project activity. Stuart needs to work with two other students, Bitter Betsy and Enigmatic Emily, to create a slide presentation reviewing the forces learned in class. Stuart is responsible for creating slides about wind, Betsy will create the slides about water, and Emily is in charge of slides about temperature. Stuart secretly has a crush on Emily and plans on sneaking in comments about how hot she is on her temperature slides. Betsy is very jealous of Emily. She plans on throwing a bucket of water on Emily. Since all three students have different schedules and they have existing gmail accounts, the group decides to create their presentation on Google Docs. Stuart creates a new presentation on Google Docs. He invites his group members to join the presentation via email. Using the information from his class notes, Stuart creates his slides about wind while the girls create their own. After every member has submitted their information, they add photos, proof read each other’s slides, and edit. Stuart watches to see when Emily signs on so he can daydream about her typing. He knows she is there because her name comes up on the screen. He waits until she is finished with her slides before trying to edit them since two people can’t edit the same slide at the same time. In class, the group simply pulls up their presentation on the web and each member takes turns speaking about their slides. The class enjoys the slide show, particularly because Stuart started the presentation with an “Earth, Wind and Fire” music video. After the presentation, Stuart asks Emily if she wants to ditch 8th period to go surfing with him.

Focus on key points of the system. Not to the point of task analysis, but do mention the general things he will do to accomplish the task.