NBA Top shot
Scoring the perfect content mix for NBA Top Shot
Scope of work
Goal: Identify which type of personalized content users prefer and why
Research Approach: Preference testing
Role: Planning, recruiting, executing, synthesizing and reporting
Timeframe: 2 days
Methods: Survey
Tools: Alchemer, DoveTail
Project Background
NBA Top Shot is a digital collectible marketplace that lets basketball fans buy, own and trade officially licensed video highlights (Moments) of their favourite players or teams in the form of NFTs. Moments can either be purchased through limited-time pack drops, earned by competing in challenges, or through the Top Shot marketplace, where fans from all over the world buy and sell around the clock.
The NBA Top Shot team wanted to update the authenticated homepage (what a site visitor sees after signing in) as a way to increase overall site engagement, and was exploring the idea of adding relevant and personalized content that would be tailored to each user.
The content a user would see would be based on what specific actions they’ve taken on the site, ie. if they are close to completing a set, they may see a message prompting them to complete it with a link to the Moments that they need to purchase.
From a business perspective, showing relevant and personalized content would help the NBA Top Shot team increase engagement, conversion rates and overall user satisfaction. For users, it would improve their overall experience of the site by ensuring that the content they see is tailored to them and their interests.
Proposed list of personalized and relevant content
- Archive set completions are up 10%! You only need 2 Moments to complete!
- Youʼve almost completed a set! Finish it with these 3 Moments
- You have a challenge entry partially completed. Click here to complete it
- A Moment on your watch list just dropped in price
- Reserve your spot in the next pack drop
- Complete your favorite teamʼs nightly challenge
- Earn these new collectible ways to personalize your profile to show your love for your favorite team
- Find out how high youʼre ranked amongst fans of your favorite team.
- See the collection of your favorite teamʼs top collector!
- Watch these videos from top collectors to see their stories, tips, and more!
- See a breakdown of your collection and how it stacks up to other collectors
- See the biggest pulls collectors made in last weekʼs drop!
- Read breaking Top Shot News
- You gained this many spots in your leaderboard this week
- Watch video recaps of the latest VIP collector event
The challenge
The NBA Top Shot team had compiled a list of personalized content types based on insights from previous user research.
Implementing each type would require significant cross-functional effort to gather and use the right data, so they asked me to help determine the optimal content mix by:
Evaluating which content types users actually prefer
Determining the relative importance of each type
my approach
After an initial kick-off meeting to better understand goals and desired outcomes, I proposed and got alignment on the following research plan:
Research Questions
What type of personalized content do
users prefer and why?
Methodology
MaxDiff Analysis
to measure user preferences
Recruitment
250 existing users of
NBA Top Shot
methodology deep dive
MaxDiff Analysis
A MaxDiff Analysis is a survey-based research technique used to reveal the order and strength of importance of all items. Users are shown a set of items, and are asked to choose which item is most important and least important. When results are displayed, each item is scored, indicating the order of preference.
For this research initiative, users were asked to respond to the following MaxDiff survey question:
I chose to deploy a MaxDiff Analysis survey method for this research initiative for the following reasons:
The team wanted to know which personalized content types users found most and least important. This required feedback from a large number of users in order to ensure statistical significance and data integrity.
The large number of personalized content types would cause fatigue if users were asked to sort by other method, e.g. ranking, card sort, etc.
Asking users to rank their preferred content types in order of priority both avoids the issue of scale bias and forces users to prioritize.
Key takeaways
Of the 250 NBA Top Shot users that took the survey, a general theme is that messages and actions that help users grow their collections ranked as being most important, while messages that speak to what others are doing on the platform ranked as least important.
Reason for Most Important Selection
Reserve your spot in the next pack drop
Users consider Drops to be the most exciting aspect of Top Shot. They would like to see this type of messaging on the site as it would give them extra reassurance that they arenʼt missing out.
“Pack drops and ripping packs continue to be one of the more exciting aspects of Top Shot. As people are in and out of the Top Shot website throughout the week, being aware of upcoming drops are of high importance – especially if you have to reserve by a cutoff time.“
Reason for Least Important Selection
See the collection of your favourite teamʼs top collector!
For the average user, seeing the collection of top collectors would discourage and demotivate them. Ultimately, it would make them feel bad for not having joined sooner.
“I could care less about seeing my teams’ top whale account. In fact it would make me feel bad to see that regularly on my page. If I want to see this I will seek it out myself.”
Outcome
As a result of the research findings, the NBA Top Shot team had the data to inform what personalized content they should focus on implementing on the authenticated homepage.
The research also helped inform what types of personalized content and messaging would discourage users from wanting to continue using the site and helped the team identify additional user pain-points, challenges and needs.