User testing validation
XC Staff
Product teams work together to test their hypotheses. User Testing Validation refers to the process of testing the hypotheses we've made with the aim of learning more about our users. There are several methods that teams can use to gather information from intended users. The end-goal is to gather strong enough evidence to draw a conclusion regarding a given hypothesis. How and why we perform user testing will depend on the product stage.
Why we do it
Learn
In the beginning of the product lifecycle, user testing can be used to generate product concepts. During this phase, teams employ user feedback to learn if they should pursue work in a problem space, and, if so, what they might build.
Confirm
Product teams continuously make product decisions based on user feedback. Not all user feedback directly translates to one clear-cut path forward, so product teams make decisions based on the information they have at hand. Once a team has implemented the decision, user testing can be used to confirm whether they got it right (or not).
Types
Interviews: These are one-on-one conversations with users, which the product team has structured to drive out learnings. Interviews are useful for obtaining feedback in both the concept generation phase and the product development phase.
Usability testing: This type of feedback session is used to evaluate a feature, product or service by testing it with participants representing intended users. Usability testing is usually conducted during the product build phase, as this is when the team has something tangible that they can show users in order to gather feedback.
Asynchronous feedback: Product teams might seek to learn more about their users' experiences and opinions using online platforms. Here, users can respond to survey or open-ended questions. Two platforms that the XC uses are Alpha and UserZoom. Asynchronous feedback methods are often used when teams need responses from a relatively large number of participants, and/or when they need access to audiences they might not otherwise be able to reach.
User testing sessions: The PM's role
Logistics: Depending on how the product team has divvied up responsibilities, the PM might be the point person for logistics related to user testing sessions. For instance, the PM should understand the team's research budget as well as the cost of the team's planned research activities so that the team doesn't overspend. When it comes to research interviews, the PM might work with a recruiting agency to find participants. This could involve writing questions that screen-in appropriate participants as well as providing the recruiter a research schedule with dates and times.
Testing Script: The user research script should reflect the key themes and ideas the team has agreed it would like to learn more about. Product managers might pair with designers on the development of the script and its questions and/or provide feedback to designers on the document while they are drafting it.
Research Session: Product managers can take both leading and supporting roles during a user research session. Using the script as a guide, a PM might be the lead facilitator of a session. Otherwise, he or she can support the lead facilitator -- usually a designer -- by taking detailed notes and asking follow up questions when prompted by the facilitator.
Synthesis: Individual user research observations on their own do not make for useful learnings. That's why team members consider what they heard and saw across multiple research sessions in order to uncover insights. Anyone who was present during a research session should regroup with the team after to contribute to the team's collective notes and analysis.