This is a sanitized version of a usability testing report to a client from my time at Xngage, a B2B e-commerce consultancy. The client, “Client X” is a wholesaler to trade professionals; their customer, “ABC Heating”, is an HVAC company. References to Insite are to the e-commerc platform at the time known as InsiteCommerce, now known as B2B Commerce Cloud (from Optimizely).
Executive Summary
I conducted a usability test of the “eClientX 2.0” website with 3 participants from your customer, ABC Heating. Top-level findings and recommendations are:
- The website’s usability is strong. While there is always a learning curve as customers adjust to a new site and new functions, we do not anticipate major problems. The participants in the usability test were able to complete the sample scenarios and did not get stuck anywhere.
- Client X should provide training for its customers. As a counterpoint to the above, even the best designed websites are much easier to use when a knowledgeable partner takes the time to explain them.
- Pay special attention to features that are roughly equivalent to the previous website, because your customers will have built up expectations about that functionality that may no longer be true.
- New features of the Insite platform should be conveyed to customers through a blend of person-to-person training, short videos, and marketing emails.
- Continue to evaluate your website over time, through a blend of customer feedback, informal usability tests, and web analytics. The best website experiences are built on understanding your customers!
Background and Method
The purposes of this usability test were:
• Learn the participant’s behaviors and expectations.
• Uncover opportunities for the future, based on participant feedback.
• Validate that the participant can successfully complete the most important tasks.
I conducted a remote, moderated usability test facilitated through a conference calling / screen sharing application.
The participants were shown a clickable mockup of the Client X website and asked to complete various tasks related to finding and purchasing products.
My original intent was to cover the following 4 scenarios; however, I adjusted on the fly to de-emphasize all but the first scenario, because the others did not apply to how ABC Heating conducts their business.
- path to purchase (simple)
- Find a product on the site
- Add the product to the cart
- Complete the checkout process
- path to purchase (complex)
- Find and add a product to the cart, then…
- Ship To – change to something other than the default address
- for a job with a one-time order
- for a job with multiple future orders (testing the ‘Save this address…’ checkbox)
- Carrier and/or Shipping Method – explore understanding of these options
- Payment Method – select among various methods; add a new method
- Review screen – edit the order itself to add a line level note
- Auto Reorder
- receive a product every [X time period], automatically
- explore understanding of this functionality
- 1-Click Order
- Find a product and immediately purchase it (skip ‘Add to Cart’)
- Clarify understanding that you are immediately on the Checkout – Review page
Detailed Findings
This section lists findings in chronological order.
Approximate timestamps are provided for the associated video file in minutes:seconds format (for example, 3:15 means to start watching at the 3 minutes, 15 seconds mark).
Initial Impressions
• (2:25) Initial impressions begin.
• Clear recognition that this is the unauthenticated site (3:15)
Sign In
• Mention of searching or finding a product prior to sign in (3:37). It’s common for people to use an e-commerce website while un-authenticated, only signing in when required.
• Signed in (4:28) but did not check the “Remember Me” box. Possibly deliberate behavior in an office environment in which multiple people share the same computer.
Authenticated Homepage
• (6:20) Discussion of “Shop By Category” and how it means different things to different people: brands / manufacturers, or product categories.
• If additional research shows you have many customers who want to shop by brand (rather than by true product category), you should re-label this to “Shop by Product Category” and creat a new menu item for “Shop by Brand”.
• Reorder Pad (7:10) including a desire (7:40) to see items reordered within a certain timeframe, which is part of the “Search Reorder Pad” functionality.
• Find the Products that Fit Your Project (9:00) – some initial confusion over this section, that it may be based on recent search history, but I clarified that these are marketing placements.
• (10:40) For the Recently Viewed Products section, questions about whether these were recently viewed by the user or by the account. The concern was that with multiple users, the recently viewed list could be irrelevant to all but one person. I clarified that the recently viewed list is based on the signed in user during that user’s session. As long as each person has their own sign in (and doesn’t share), there should be no “cross pollination” of recently viewed products.
Path to Purchase – Simple
• (14:00) Beginning of the scenario to find a Honeywell 24V automatic rectangular damper.
• (14:58) Search. However, unlike in the mockup, the participant (15:12) would have used the model number (and this will also be successful in the working website).
o These participants were strongly “search first” – however, if we repeated this usability test with other participants, we would likely find some who are “navigate first and only search when that fails”.
• A desire to see recently searched items in the search bar (16:00). I clarified (later, at 19:10) that this is part of the Insite functionality.
• The participant was able to add the item to the minicart and proceed to checkout (17:30).
• Ability to see how many are available and where, on the product detail page (18:25).
• There was some concern about the need to select a carrier (21:00) and shipping method. I clarified (much later, at 38:40) that these will be defaulted based on the user’s profile.
• (22:35) The participants stated that if something is not in stock, they would actually call Client X to have Client X place the order on their behalf. This is an example of where you could teach customers how to self-serve, if desired.
• There was a discussion around shipping notes (23:30) and the various ways the participants use them. This may be an opportunity for Client X to clarify how you want customers to use this field.
• (26:00) I clarified that in reality, the Need by Date will default to a blank value, so that customers will not inadvertently select same-day orders.
• These participants never use credit cards for Client X Supply purchases, always purchase orders. (26:50) Again, the actual Insite functionality will default the Payment Method to the user profile.
• (27:40) The participants were slightly confused (possibly because of the unfamiliar scenario) about the need to select a payment method, and they attempted to input a purchase order number into the Promotion Code field.
o If one group of people is confused by something, it’s highly likely that other groups will also be confused. I recommend that you monitor this part of the checkout process (using Google Analytics and Tag Management) to ensure that customers who do use different payment types know how to complete this screen.
• When viewing the Review screen (30:40), the participants noted that the existing website “has fewer steps” because the process is “all on one screen”.
o It is common for people to perceive a process they are familiar with to be simpler than one they are unfamiliar with, even though objectively the unfamiliar process may be better designed!
• (31:50) Desire to see where the order is shipping from, because of the impact to lead time.
• I acknowledged that in reality (but not in the mockup) the Submit Order button will follow the user down the screen – it’s not anchored to the top of the page. (34:05)
• (36:3) Upon order completion, the participants noticed the Order Number and Status message (Processing). They also expect to receive an email (37:55), which is part of the Insite functionality.
Path to Purchase – Complex
• We discussed (34:45) that ABC Heating does not currently have orders shipped to job sites… although they are aware this is a possibility.
• This is an example of an Insite function that customers could be reminded of in marketing emails, so that when they are ready to use that function, they remember it exists!
Auto Reorder
• (41:30) I explained the auto reorder scenario, but it turned out this is not how ABC Heating keeps items in stock (42:30).
• When asked where they would find the auto reorder functionality (44:20), the participants thought to look under Quick Order.
• Quick Order vs. Bulk Order (44:55) – the participants weren’t sure if these are the same functionality or not. You could consider removing one of these labels to prevent confusion.
• When asked to describe Quick Order (45:08), the participant responded:
o “A quick order is something that we order all the time, but we manually place that order.” … “If you click on quick order, it would pull up the things that we ordered most frequently – not necessarily the most volume, but the most frequently.” … “Bulk order would be the [auto reorder].”
o “I would expect to hit Quick Order and then not have to [populate anything]” (47:45)
• Because this is definitely not how Quick Order works, you should plan to educate customers about this functionality. I clarified the actual Quick Order at 53:20.
• In addition, the Reorder Pad (48:25) – which actually does retain a history of products the customer purchased – was perceived to be “a marketing thing like Amazon” [recommended products]. Again, you should plan to educate customers about the differences between the Reorder Pad and the Find the Products that Fits Your Project.
1-Click Order
• (51:50) The participants clearly understood the difference between Add to Cart and Buy Now With 1-Click…
• (52:15) … and also recognized that the 1-Click takes you to “the end” of the purchase process.