A mixed methods approach: a little more science and a little less art

With over 55 million reservations booked to date, ClassPass has a lot of information on where, when and how customers are using the app (or website), but they don’t know all of the whys behind the numbers.  To mine actionable insights, my research team employed a mixed methods approach grounded in quantitative behavioral data.

Study overview

• Participants from 4 quantitatively derived clusters 
• Data from pre and post interview surveys
• 29 user interviews
• Synthesis of qualitative data through affinity mapping
• Iteration of cluster approach through attribute comparisons
• Development of 4 core personas
• Presentation of findings and recommendations to cross-functional teams

Recruiting the "right" users

Working alongside a data scientist, our research team refined user criteria based on four clusters groups previously identified through cluster analysis to ensure that our interviewees were indeed representative of actual users.  To further screen participants and collect additional pre-interview information, we deployed a questionnaire.

Interviewing

Our research team recruited and interviewed a total of 29 people. Each interview was one hour long. The majority of interviews happened over video, although several NYC based users came in for in-person interviews.

When n=29

The initial plan was to affinity map within the four cluster groups-- one affinity map per cluster group with a focus on behavioral differences as opposed to demographic differences. 

Post-its for just 7 out of 29 user interviews 

Sifting through the data proved to be a daunting endeavor. The first attempt to make sense out of this overwhelming mass of sticky notes failed for two reasons: 1. There wasn’t enough pattern overlap in the initial clusters, and 2. There were 2,500 post-it notes.  We regrouped by literally regrouping the clusters.  

Iterating on the approach

We created an "attribute tally" spreadsheet to track salient data points across the 29 interviewees. The resulting document tracked over 20 user attributes, from age to workout schedule. By coding these data points, we could more readily determine key variables that differentiate segments and extract nuanced patterns.

Data-driven personas

Convincing, representative personas speak to known and unknown user pain points and opportunities. The four user personas that emerged from our research findings captured the motives, behaviors, goals, priorities of ClassPass users.  Each persona artifact included a short narrative from the persona’s perspective, providing both depth and personality.

Each persona represents a significant portion of ClassPass users in the real world, enabling product designers to focus on a manageable and memorable cast of characters.

Presentation matters: Transferring insights and knowledge

I find presenting work to be both immensely satisfying and completely daunting.  Satisfying because you're finally able to share something tangible and actionable for your efforts; daunting because startups move at a fast clip (and humans generally have limited attention spans), which means you need to capture your audience's attention early and keep it.  Knowing this reality all too well, we relied on storytelling and visuals to convey findings and insights and saved the nitty gritty details for the research report. Quotes from user interviews were particularly impactful and led to many follow up questions and animated discussions.

The presentation was well received. The deliverables, particularly the personas, are being used to brainstorm and prioritize features.  They are also helping to define product strategy. To ensure that our research lived on, we created a file repository where product and marketing team members could go to quickly access high level insights, individual observations as well as raw research data.