Usability Test – One Time Passcodes

A large bank wondered: which version of a one time passcode design (those 6 digit numbers you get via text message or email) would be easier for customers to use when they forget their password? Because of the short turnaround time on this project, I chose to conduct a remote unmoderated usability test of both versions on a cloud-based testing platform.

Findings and Recommendations

Both designs were equally easy to use.

  • For Version A (radio buttons), 5 of 6 users rated the task extremely easy, and the one who did not had a complaint about the mockup, not the OTP process itself.
  • For Version B (flat buttons), 5 of 5 users rated the task easy or extremely easy.

Users clearly understand the purpose of a One Time Passcode.

  • 100% (11 of 11) selected “It validates your identity without your banking password” from a list of multiple choices.

Eliminate the redundancy of inputting User ID on the home screen and then again on the ‘Forgot Password’ screen

  • A few participants commented that it is redundant to input your User ID on the Forgot Password screen when you’ve just entered it on the previous screen.
  • We should carry over the User ID that the customer has already entered.

Avoid or alleviate concerns about inputting Social Security Number (SSN)

  • Several participants expressed concerns about inputting their SSN as part of the one time passcode process.
  • We should explore other ways to validate customer identity, or explain more thoroughly why we absolutely need their SSN as part of this process.
  • This video clip is a representative sample of the reluctance that participants had to divulge their SSN.

Retain the ability to select ‘Forgot Password’ prior to inputting User ID

  • 6 of 11 participants attempted to select ‘Forgot Password’ prior to inputting their User ID. Within the confines of the usability test mockup, they were not able to proceed until they clicked to enter the sample User ID.
  • On the real bank website, you can select ‘Forgot?’ immediately. You don’t have to input a User ID first. We should retain this functionality!

Usability Test Summary

I tested two versions of the one time passcode (OTP) process with 11 people:

6 participants saw Version A (radio buttons)

One time passcode selection screen with radio buttons.
Version A – radio buttons

5 participants saw Version B (flat buttons)

One time passcode selection screen with separate, flat buttons.
Version B – flat buttons

 

 

 

 

Participants

Participant Profiles
I used our standard screeners to screen for frequent users of online banking between the ages of 18 and 99.

Recruiting
I used the cloud based testing platform’s automated recruiting service, which accesses millions of potential participants throughout the United States.

Study Design

Methodology
We tested two groups of participants, using unmoderated think-aloud studies on the cloud-based testing platform.

Tasks
The single task was to obtain an OTP:

“Imagine that you are trying to access your checking account online, but you’ve forgotten your password. Follow the instructions to validate your identity by using a one-time security code. When you’ve completed this task, click Success. If you get stuck and can’t continue, click Abandon.”

Timeframe
This usability test was started and completed in a one day time period in 3Q2018.

Leave a Reply