Project Digital Payments
Context
​Due to legal restrictions, a new digital payment platform involving a 3rd party replaced a major process that directly impacts over 70,000 of our users.
​
Problem
This new platform contains a fee to utilize and is deterring a concerning amount of customers from switching over resulting in loss of revenue.
​
Short-term Goal
For each page of the setup process, we were looking to understand users initial reaction and why, their expectations, and any questions, concerns, and surprises they encountered.
​​
Long-term Goal
Tracking trend using benchmarking methods to identify increased / decreased satisfaction and effort scores between different releases then identifying what specifically impacted these scores. From here, our goal was to increase confidence levels of opting-in, reduce overall support call volume regarding this platform, and ultimately to uncover opportunities of “How might we limit resistance of our users not adopting this new platform?”
Stage of Research
-
Discovery
-
Exploratory
-
Testing
-
Listening
Methodologies
-
Survey- Qualtrics screener
-
Usability Study- 30 minute remote sessions via Microsoft Teams
Participants / Criteria
-
9 Current users
-
5 primarily make payments using the current digital payment platform
-
4 primarily make payments via paper checks
-
5 Tested on desktop / laptop device
-
4 Tested on a mobile device
-
-
Criteria
-
Primary manager of their account (i.e., not their spouse or another member on the membership)​
-
Expressed interest in joining the Beta via the screener
-
Tested on the device they primarily use for signing in (mobile or desktop / laptop)
-
Contributors
-
Stakeholders- Company leadership, current members, and support staff
-
Research requester- Program manager
-
​Communication- Mass comms and BI
-
Note takers- Design team
-
Moderator / planner- Myself
Context
Executive Summary of Insights
Account setup issues​
-
Fee
-
Users expressed confusion and / or frustration with the fee resulting in users opting out
-
-
Unable to setup account
-
If the user's verification fails a second time, the user is unable to complete setup
-
-
"Continue" button broken
-
The "continue" button was not working for a handful of users to complete setup
-
-
Unable to edit profile
-
After clicking "continue", the option to edit personal information is no longer an option​​​
-
Planning
-
Understanding goals- Scheduled a meeting with our stakeholder to understand their goals, the "whys"​, and any constraints to be aware of​
-
Success metrics- Worked with stakeholder and designers to determine what we would use to define this study as a success or failure
-
Methodologies- Determined the appropriate methodologies. In this case, a survey followed by a usability study
-
Criteria- Determined participant criteria
Scope
Survey
-
Identified users- Utilizing our list of Member Insights Group (users who have expressed interest in joining research sessions), we identified a current users to send the survey to
-
Gaging interest- Gathered a list of users who were interested in opting in to this new digital sharing platform
-
Understanding specifics- Using open text fields, we gained insights into these users' motivation for opting in
-
Distribution- Sent survey to 137 current users via Qualtrics, completed by 114 of those users
-
Incentive-Requested incentive / budget approval
-
Booking sessions- Created Microsoft Booking session for scheduling the usability study
-
Identify users- After reviewing results of the survey, was able to identify potential users to request an interview with
-
NDA / media waiver- Sent NDA / Media waivers along with a confirmation email prior to study
-
Recruitment- Recruited 9 current members segmented by their current primary payment method along with primary sign-in device (mobile or desktop / laptop)
Recruitment
-
Questions- Questions asked were based on answering our success metrics determined earlier
-
Follow up- Leveraged insights discovered from the survey for follow-up questions
Moderator Guide
Insights / Impact / Outcomes
Account setup issues
​
​
1. Fee
Problem- Users expressed frustration with seeing the fee for the first time during setup. This resulted in a handful of users opting out of the new digital sharing platform stating the current payment system is "free".
Perceived Impact Rating- High
Outcome- While eliminating the fee is not an option at this point, our support team is continuing to monitor and provide feedback to inform leadership. Additionally, our communication department is actively working to explaining to our users what the fee is used for, how it works, and why it's necessary.​​​​​​​
​
2. Unable to Setup Account
Problem- If the user's authentication fails during the second attempt of a manual KYC (Know Your Customer) verification stage, users are unable to complete the setup process and must contact our support team to issue them a new link to attempt completing setup at a later date.
Perceived Impact Rating- High
Outcome- Our software developers are actively working with the 3rd party platform developers towards a more efficient solution.
​
3. "Continue" Button Broken
Problem- On the final page of setup, the "continue" button was not working for a handful of our users. This users were then unable to complete the setup process
Perceived Impact Rating- High
Outcome- Our software team reissued a new link to those users. We then followed up with those users and discovered the issue did not happen again. While we are not certain of what caused the issue, we found a temporary workaround solution in the meantime.
​
​​​​​​​​​​​​4. Unable to Edit Profile
Problem- After the user confirms their profile information is correct (including their legal names, address, and driver's license number), the users is unable to edit this information later which may result in a failed KYC verification process.
Perceived Impact Rating- High
Outcome- While this was not a high enough issue to require immediate action, our the design, software, and support team are continuing to monitor user feedback regarding this issue.​​​​​​​​​​​​​
​
5. Verification Documents
Problem- When users fail the initial KYC verification, they are required to upload photo documents of their personal information. However, the uploading process does not specific what format types are accepted resulting in an error message with no further explanation or next steps provided.
Perceived Impact Rating- Medium
Outcome- Our communication and design team have since added language to the page explaining which formats are acceptable
​
6. Resetting Password
Problem- When a user resets their password while using the link provided to setup an account, the option to setup the new digital sharing platform can only be accessed through the original link. This resulted in users expecting the setup link to be accessible from their profile page. However, this was not the case. The user is required to reuse the link provided to move forward with the process resulting in a less than intuitive process.
Perceived Impact Rating- Medium
Outcome- Our communication and design team have since added additional locations to setup the account in additional locations based on insights from user expectations.
​
7. Next Steps / FAQ Location
Problem- After completing the process, there were several users who mentioned not knowing what to expect for sending in their next monthly payment or if any additional steps were needed on their end.
Perceived Impact Rating- Low
Outcome- Our communication and design team have since added a FAQ page at the end of the setup process explaining next steps and what to expect moving forward.
Mobile Specific Issues
8. Reload / Back Button
Problem- Regardless of which page the user is on, if the user scrolls down too quickly or accidentally goes back to the previous screen, the page reloads resulting in the users being required to start the process over completely.
Perceived Impact Rating- Medium
Outcome- 3rd Party software developers worked with our design team to add a failsafe message confirming the action was intentional along with an option to cancel the request.
​
9. Drag and Drop Feature
Problem- Due to the small screen, the interface made uploading documents challenging compared to the desktop view.​
Perceived Impact Rating- Medium
Outcome- Our software development and testing team have since addressed the issue by increasing the drag and drop space on mobile.
​
​
Retro / Lessons Learned
Confirmation Email
Problem- We had a handful of participants who were late to the session or forgot about the session completely. During this process we discovered the booking platform we used with an automatic confirmation email was going to users' spam box.
Opportunity for Improvement- Rather than relying on the booking platform for confirmation emails, we are instead manually sending confirmation emails using our department email.
​
"Is this the real thing?"
Problem- While the recruitment email mentioned testing our Beta digital sharing platform, we did not clarify the user was actually setting up an account (not a prototype usability study). This resulted in users being confused and a humbling "oops" moment on my part.
Opportunity for Improvement- In the future, clarifying that while we are testing the platform, users will actually setup an account while on the call with us.
​
​