Usability Study: Amazon Comparison Chart

E-Commerce

A usability study uncovering what prevents customers from making confident purchase decisions with Amazon's comparison chart, and how to fix it.

Impact

We surfaced 6 prioritized usability issues and presented findings and actionable recommendations to Amazon researchers, PMs, and designers. Key impact:

  • Guided redesign direction: This study surfaced discoverability, scannability, and information overload as the top barriers, and delivered a severity-rated roadmap to help the team prioritize fixes across desktop and mobile.

  • Shaped the research roadmap: We mapped out what was already known vs what needed further investigation, helping the team understand where to invest next.

  • Informed product strategy: It uncovered a link between shopping stage and chart engagement, suggesting the need to tailor chart experiences for early vs late stage shoppers.

Skills

Usability Study, Competitive Analysis, Mixed-Methods, Stakeholder Engagement, B2C

Role

Lead UXR

Team

Amazon Devices team & UW HCDE

Timeline

January - March 2024

A well-designed comparison chart shortens the path to purchase, giving shoppers the confidence to decide without searching for information elsewhere.

I led a remote moderated usability study comparing Amazon's Fire Tablet chart against Apple, Samsung, and Best Buy, and included Blink (Amazon's home security brand) as an internal benchmark. Findings gave the team a prioritized, evidence-backed roadmap to reduce friction and improve decision-to-purchase conversion.

* This study was sponsored by Amazon and completed in a team of 4 as part of a graduate course at the University of Washington.

Impact

We surfaced 6 prioritized usability issues and presented findings and actionable recommendations to Amazon researchers, PMs, and designers. Key impact included:

  • Guided redesign direction: This study surfaced discoverability, scannability, and information overload as the top barriers, and delivered a severity-rated roadmap to help the team prioritize fixes across desktop and mobile.

  • Shaped the research roadmap: We mapped out what was already known vs what needed further investigation, helping the team understand where to invest next.

  • Informed product strategy: It uncovered a link between shopping stage and chart engagement, suggesting the need to tailor chart experiences for early vs late stage shoppers.

Details

Context

In 2023, Amazon launched an updated comparison chart for its Fire Tablet product page, but it was unclear whether the chart was helping customers make purchase decisions. This raised a question: what works, what doesn't, and what can Amazon learn from competitors?

Working with the Amazon Device Team, we evaluated the Fire Tablet chart against Apple, Samsung, Best Buy, and Blink to understand how it's landing and deliver redesign recommendations for the team.

Research Questions

  • What aspects of Amazon's comparison chart help or hinder customers from making confident purchase decisions?

  • What are competitors doing better, and what can Amazon learn to improve its chart experience?

  • How does a shopper's stage in the buying journey affect how they engage with the comparison chart?

Methods

I recruited 12 participants through UserTesting. Participants were evenly split across:

  • Desktop and mobile users

  • Amazon device owners and non-owners

  • Early, mid, and late stage shoppers

The study combined two methods:

01 Moderated Usability Testing

To identify what helps or hinders users when navigating the chart in a realistic shopping context

I chose moderated sessions over unmoderated because comparison charts involve nuanced decision-making behavior that's hard to interpret without being able to probe in the moment.

Participants completed think-aloud tasks navigating Amazon's Fire Tablet chart alongside 1 to 2 competitor charts and the Blink subscription chart. To reduce order bias and priming effects, we randomized the order charts were presented to each participant.

02 Post-task Interviews and Likert Scale Ratings

To benchmark the Fire Tablet chart against competitors and quantify user experience across key usability dimensions

Following each task, participants answered Likert scale rating questions across six dimensions: usefulness, visual appeal, ease of use, information clarity, information sufficiency, and confidence.

We followed up with a 15-min post-task interview to dig deeper into their experiences and preferences across charts.

Selected Findings & Recommendations

The current chart experience has friction that slows shoppers down when making purchase decisions.

There are concrete opportunities to fix this: first, get users to the chart more quickly; then, tailor the experience so shoppers at different stages can make decisions more efficiently; and finally, make the information easy enough to act on. Addressing these could shorten the path to purchase and reduce drop-off risks.

Finding 01 - Low Chart Discoverability Compared to Competitors [Severity 3 - Major]

  • 50% users never found the Amazon Fire Tablet comparison chart on their own. Those who did had to scroll through most of the page to reach it — a bigger problem on mobile due to smaller screens.

  • By contrast, Blink and Samsung placed their charts higher on the page, and all users found them without any prompting.

“The Amazon chart took forever to find. It's under the ocean like 3,000 feet down — there's no way I'm getting down there, especially on a phone.” — P7, Mobile

“The Amazon chart took forever to find. It's under the ocean like 3,000 feet down — there's no way I'm getting down there, especially on a phone.” — P7, Mobile

Why it Matters

  • Every user who doesn't find the chart has to make a purchase decision with less information, increasing the risk of hesitation or abandonment before they make the final purchase. Low discoverability means Amazon may lose an opportunity to influence purchase decisions at a high-intent moment.

Recommendations

  • Adding a click-through link higher on the page is the low-hanging fruit. We recommend prioritizing this first, as it doesn't require repositioning the chart and can be implemented quickly.

  • Moving the chart itself higher requires evaluating what content gets displaced. We recommend A/B testing both approaches to confirm which drives more chart engagement and conversion before committing to a permanent layout change.

Finding 02 - Shopping Stage Could Shape Chart Engagement [Severity 1 - Cosmetic]

  • All early-stage shoppers stopped at the "see the difference" feature near the top to get a quick overview and narrow down their options, without ever scrolling down to the full comparison chart.

  • Of the 5 users who interacted with "see the difference," 4 indicated it gave them enough information to feel satisfied at that point in their journey.

“I saw the “see the differences”. And to me I thought that was replacing the chart in my mind.” — P5, Desktop

“I saw the “see the differences”. And to me I thought that was replacing the chart in my mind.” — P5, Desktop

Why it Matters

  • The full comparison chart may have been designed for users who already know what they want to compare. But early-stage shoppers have a different need: they want a quick overview to narrow down options, not a detailed spec breakdown.

  • This raises an important question: does "see the difference" replace the full chart for early-stage shoppers entirely, or do they return to the full chart later when they are ready to make a final decision with real money on the line? Understanding this would help Amazon decide where to invest in the chart experience.

Recommendations

  • The key open question is whether early-stage shoppers return to the full chart later in a real purchase decision, or whether "see the difference" is enough to get them to buy. An in-app survey triggered after a user interacts with "see the difference" could help clarify this in a real shopping context.

  • If they do need to return to the full chart to make a final decision, we could consider adding a prompt within "see the difference" that guides users to the full chart.

Finding 03 - Users Need Everyday Language Over Technical Specs to Compare and Decide [Severity 2 - Minor]

  • Half of participants struggled to interpret technical specs like GHz for CPU speed or mAh for battery life, making it hard to compare products without leaving the site to google what they meant.

  • They preferred everyday descriptions instead (e.g., "up to 10 hours of battery life" over "1000mAh").

“The Gigahertz doesn't really help me understand how fast it is. I'm not sure what that even means." — P1, Desktop

“The Gigahertz doesn't really help me understand how fast it is. I'm not sure what that even means." — P1, Desktop

Why it Matters

  • When users can't interpret the specs, they may lose confidence and turn to external sources to research what they mean. This could pull them away from the page and potentially delay the purchase decision.

  • For most shoppers, technical specs could slow down the path to purchase. Presenting information in simpler, actionable terms could be a lever for improving decision making speed and conversion.

Recommendations

  • The low-hanging fruit is translating technical specs into real-world terms. For example, replacing mAh with hours of battery life, and GHz with performance descriptions like "handles multitasking and streaming smoothly".

  • For specs that are harder to simplify, add info icons that let users access plain-language explanations on demand without cluttering the default chart view. The Blink chart's info icon approach received consistently positive feedback and is a strong reference point.

Reflection

What I Learned

Narrowing scope earlier leads to deeper insights. Covering many variables (e.g., desktop versus mobile, third-party charts, and subscription charts) gave us a helpful overview, but some findings could have been stronger with a more focused scope.

What I'd Do Differently

Scope research around business priorities. Before designing the study, I'd align with stakeholders on the 1-2 most pressing questions, explain the tradeoff between breadth and depth, and scope accordingly. Covering less ground but going deeper on the right question could be more valuable and impactful than broad but shallow coverage.

I'm available

Let's Connect

Contact Me

Tammy Yan

I'm available

Let's Connect

Contact Me

Tammy Yan

Usability Study: Amazon Comparison Chart

E-Commerce

A usability study uncovering what prevents customers from making confident purchase decisions with Amazon's comparison chart, and how to fix it.

Tags

Usability Study, Competitive Analysis, Mixed-Methods, Stakeholder Engagement, B2C

Role

Lead UXR

Team

Amazon Devices team & UW HCDE

Timeline

January - March 2024

A well-designed comparison chart shortens the path to purchase, giving shoppers the confidence to decide without searching for information elsewhere.

I led a remote moderated usability study comparing Amazon's Fire Tablet chart against Apple, Samsung, and Best Buy, and included Blink (Amazon's home security brand) as an internal benchmark. Findings gave the team a prioritized, evidence-backed roadmap to reduce friction and improve decision-to-purchase conversion.

* This study was sponsored by Amazon and completed in a team of 4 as part of a graduate course at the University of Washington.

Impact

We surfaced 6 prioritized usability issues and presented findings and actionable recommendations to Amazon researchers, PMs, and designers. Key impact:

  • Guided redesign direction: This study surfaced discoverability, scannability, and information overload as the top barriers, and delivered a severity-rated roadmap to help the team prioritize fixes across desktop and mobile.

  • Shaped the research roadmap: We mapped out what was already known vs what needed further investigation, helping the team understand where to invest next.

  • Informed product strategy: It uncovered a link between shopping stage and chart engagement, suggesting the need to tailor chart experiences for early vs late stage shoppers.

Details

Context

In 2023, Amazon launched an updated comparison chart for its Fire Tablet product page, but it was unclear whether the chart was helping customers make purchase decisions. This raised a question: what works, what doesn't, and what can Amazon learn from competitors?

Working with the Amazon Device Team, we evaluated the Fire Tablet chart against Apple, Samsung, Best Buy, and Blink to understand how it's landing and deliver redesign recommendations for the team.

Research Questions

  • What aspects of Amazon's comparison chart help or hinder customers from making confident purchase decisions?

  • What are competitors doing better, and what can Amazon learn to improve its chart experience?

  • How does a shopper's stage in the buying journey affect how they engage with the comparison chart?

Methods

I recruited 12 participants through UserTesting. Participants were evenly split across:

  • Desktop and mobile users

  • Amazon device owners and non-owners

  • Early, mid, and late stage shoppers

The study combined two methods:

01 Moderated Usability Testing

To identify what helps or hinders users when navigating the chart in a realistic shopping context

I chose moderated sessions over unmoderated because comparison charts involve nuanced decision-making behavior that's hard to interpret without being able to probe in the moment.

Participants completed think-aloud tasks navigating Amazon's Fire Tablet chart alongside 1 to 2 competitor charts and the Blink subscription chart. To reduce order bias and priming effects, we randomized the order charts were presented to each participant.

02 Post-task Interviews and Likert Scale Ratings

To benchmark the Fire Tablet chart against competitors and quantify user experience across key usability dimensions

Following each task, participants answered Likert scale rating questions across six dimensions: usefulness, visual appeal, ease of use, information clarity, information sufficiency, and confidence.

We followed up with a 15-min post-task interview to dig deeper into their experiences and preferences across charts

Reflection

What I Learned

Narrowing scope earlier leads to deeper insights. Covering many variables (e.g., desktop versus mobile, third-party charts, and subscription charts) gave us a helpful overview, but some findings could have been stronger with a more focused scope.

What I'd Do Differently

Scope research around business priorities. Before designing the study, I'd align with stakeholders on the 1-2 most pressing questions, explain the tradeoff between breadth and depth, and scope accordingly. Covering less ground but going deeper on the right question could be more valuable and impactful than broad but shallow coverage.

Selected Findings & Recommendations

The current chart experience has friction that slows shoppers down when making purchase decisions.

There are concrete opportunities to fix this: first, get users to the chart more quickly; then, tailor the experience so shoppers at different stages can make decisions more efficiently; and finally, make the information easy enough to act on. Addressing these could shorten the path to purchase and reduce drop-off risks.

Finding 01 - Low Chart Discoverability Compared to Competitors [Severity 3 - Major]

  • 50% users never found the Amazon Fire Tablet comparison chart on their own. Those who did had to scroll through most of the page to reach it — a bigger problem on mobile due to smaller screens.

  • By contrast, Blink and Samsung placed their charts higher on the page, and all users found them without any prompting.

“The Amazon chart took forever to find. It's under the ocean like 3,000 feet down — there's no way I'm getting down there, especially on a phone.” — P7, Mobile

Why it Matters

  • Every user who doesn't find the chart has to make a purchase decision with less information, increasing the risk of hesitation or abandonment before they make the final purchase. Low discoverability means Amazon may lose an opportunity to influence purchase decisions at a high-intent moment.

Recommendations

  • Adding a click-through link higher on the page is the low-hanging fruit. We recommend prioritizing this first, as it doesn't require repositioning the chart and can be implemented quickly.

  • Moving the chart itself higher requires evaluating what content gets displaced. We recommend A/B testing both approaches to confirm which drives more chart engagement and conversion before committing to a permanent layout change.

Finding 02 - Shopping Stage Could Shape Chart Engagement [Severity 1 - Cosmetic]

  • All early-stage shoppers stopped at the "see the difference" feature near the top to get a quick overview and narrow down their options, without ever scrolling down to the full comparison chart.

  • Of the 5 users who interacted with "see the difference," 4 indicated it gave them enough information to feel satisfied at that point in their journey.

“I saw the “see the differences”. And to me I thought that was replacing the chart in my mind.” — P5, Desktop

Why it Matters

  • The full comparison chart may have been designed for users who already know what they want to compare. But early-stage shoppers have a different need: they want a quick overview to narrow down options, not a detailed spec breakdown.

  • This raises an important question: does "see the difference" replace the full chart for early-stage shoppers entirely, or do they return to the full chart later when they are ready to make a final decision with real money on the line? Understanding this would help Amazon decide where to invest in the chart experience.

Recommendations

  • The key open question is whether early-stage shoppers return to the full chart later in a real purchase decision, or whether "see the difference" is enough to get them to buy. An in-app survey triggered after a user interacts with "see the difference" could help clarify this in a real shopping context.

  • If they do need to return to the full chart to make a final decision, we could consider adding a prompt within "see the difference" that guides users to the full chart.

Finding 03 - Users Need Everyday Language Over Technical Specs to Compare and Decide [Severity 2 - Minor]

  • Half of participants struggled to interpret technical specs like GHz for CPU speed or mAh for battery life, making it hard to compare products without leaving the site to google what they meant.

  • They preferred everyday descriptions instead (e.g., "up to 10 hours of battery life" over "1000mAh").

“The Gigahertz doesn't really help me understand how fast it is. I'm not sure what that even means." — P1, Desktop

Why it Matters

  • When users can't interpret the specs, they may lose confidence and turn to external sources to research what they mean. This could pull them away from the page and potentially delay the purchase decision.

  • For most shoppers, technical specs could slow down the path to purchase. Presenting information in simpler, actionable terms could be a lever for improving decision making speed and conversion.

Recommendations

  • The low-hanging fruit is translating technical specs into real-world terms. For example, replacing mAh with hours of battery life, and GHz with performance descriptions like "handles multitasking and streaming smoothly".

  • For specs that are harder to simplify, add info icons that let users access plain-language explanations on demand without cluttering the default chart view. The Blink chart's info icon approach received consistently positive feedback and is a strong reference point.

I'm available

Let's Connect

Contact Me

Tammy Yan