Archetype creation

Introduction

Handshake is a three-sided marketplace made up of students, employers, and academic institutions. The student UXR team at Handshake had finished an archetype creation study to help the company better understand the types of students using the platform prior to some major redesigns of student facing product. The success of these five student archetypes and their subsequent use led to the desire to understand employer users at a similar level.

My Role

I was tasked with handling this project from end-to-end. This includeded meeting with stakeholders, selecting the methodology, and ultimately presenting the resulting archetypes.

Method
Round 1 – Interviews

Interviews were selected because they are a great way to learn what we don’t know by providing a small framework in the form of a moderator guide, but allowing me to dive fully into anything interesting that shows up. I sent my moderator guide around to a few of the stakeholders to ensure that it was covering areas that would be of interest to them as part of the archetype creation. These interviews were reocrded on Zoom over the course of 3 weeks and were transcribed and stored on a Reduct.video repository for coding. 

My teammate was brought into the project here to help interviews and coding move forward while I was on PTO and then assisted with the thematic analysis after the completion of the interviews. She and I took the themes generated to a workshop with our fellow design team members to have them rate their familiarity with themes and identify any that seemed previously unknown to them. [PLACEHOLDER] 

Round 2 – Survey

Following the thematic analysis, 25 statements were generated for employers to rate on a 7-point scale how accurate the statement was for them. These statements were added to a Qualtrics survey, in a few matrix questions, with additional demographic focused questions, such as the size of their recruitment budget, the amount of company revenue, the size of their hiring team, industry, and a number of other questions that could help us understand how these themes present themselves in our vastly diverse employer userbase.

Following the survey, I plotted the within cluster sum of squares (WCSS) for a variety of k-means clusters using R Studio and determined the optimal k value using a combination of hierarchical clustering, elbow method, and silhouette method. After determing the optimal k value, I conducted the cluster analysis to determine the mean values of the different groups across the 25 thematic statements.

Participants
Round 1  – Interviews

22 interviews total:

  • 7 small employer users <150 employees
  • 6 medium employer users 100-1000 employees
  • 4 large employer users 1000+ employees
  • 5 employers who don’t use Handshake
Round 2 – Survey

Goal: 2000 responses, mix of 33% small, 33% medium, and 33% large employers to overrepresent the employers who paid for our product, at the of a request of a key stakeholder. 

I surveyed 80,000 employer users and received 1,786 completed responses with a mix of 45% small, 40% medium, and 15% large employers. 

example Reduct.video videoboard
Results

Following the 22 interviews conducted, my colleague and I were able to code and clip nearly 400 highlights that we organized into themes over the course of 1 week. We took these themes into workshops with different stakeholder groups including designers, product managers, and sales, ultimately narrowing them down to 25 themes.

We worked with our content writers to develop the 25 themes into statements that we could ask with a 7-point Likert scale in a survey. The survey I created on qualtrics included these 25 statements broken up into smaller matrix style questions in combination with some demographic questions about the user and their company’s hiring team make up and needs. We let the survey run for a week with a few reminder prompts and achieved enough responses to be able to statistically compare groups of employers across a variety of variables including size, industry, location, hiring needs, etc.

The within cluster sum of squares (WCSS) helped me determine the optimal number of clusters to be 8, which is what I then used to conduct a k-means cluster analysis, forming the responses into 8 different clusters based on their responses to the 25 thematic statements. Once the 8 groups were established, demographic details were studied in combination with the mean scores of the cluster on each statement to get a fuller understanding of the archetype. Then we came up with summary statements about each archetype and took them to our content team to help us come up with names that evoked the ideas in those summaries.

While I am not able to share the eight groups here, I can say that they covered the majority of known employer user needs and goals from a research perspective. However, stakeholders were intimidated by the number of archetypes. Coming off of the 5 student archetypes, having 8 employer archetypes felt difficult to remember and act upon. I repackaged the archetypes following this feedback, creating a Venn diagram showing how certain archetypes overlapped with others in terms of resources and experience in order to highlight the few that each product team would be interacting with. At Handshake the teams working on enterprise clients (typically larger paying users) were separate from teams that focused on our free and mid-tier employer users. This allowed me to reduce the amount of archetypes that each business segment had to focus on to five, with a few that overlapped with the other business segment.

Example Venn diagram. Here I may have A represented by Large employer, B as small employer, and C as strong recruiting knowledge
Reflection

This project was by far one of the largest I have undertaken in my career as a UXR and ultimately will serve as a lesson for many research projects in the future. There were a lot of things that went really well during the project including:

  • Participant recruitment
  • Interviews
  • Survey

And a few that didn’t go as smoothly:

  • Thematic analysis
  • Cluster analysis
  • Packaging the research findings

I think the main reason there were parts that didn’t go as smoothly were because the peer researcher was brought in as a co-owner late to the project. This led to both me and the additional co-owner having to compromise on both insight prioritization and data interpretation, then being asked to share out the knowledge that we didn’t whole-heartedly believe in.

In addition, we both were primary users of SPSS for data analysis and the company didn’t have the finances to pay for a license. This led to me working with ChatGPT to write R-code to conduct a k-means cluster analysis in R Studio which I haven’t done before. However, my academic classes in R code led to skills in at least reading the code and understanding that the data analysis was accurate.