Content testing and research for the Handshake App
Background
In early May and June 2021, I participated in the UX Writer Collective's certificate program for content testing and research. For my final assignment, I presented my analysis and content recommendations for an imaginary mobile app called HandShake.
This page covers key insights from my presentation.
About HandShake and this project
HandShake is a payment app that lets business owners pay their freelancers directly. The study aims to see how users perceive the copy on the interface for areas of improvement.
Overall findings
While participants could navigate the apps with some ease, the ambiguous text, lack of context, and unclear user flow make the user experience difficult.
Methodology: Participants, scenario & tasks
Each user is given five tasks. Participants are measured by:
-
Successful task completion (Y/N)
-
Ease of use (Self-assessed on 1-10 scale)
-
Confidence (Researcher assessed on 1-10 scale)
Each participant is also asked follow-up questions about how they felt about each specific task and allowed to comment in general.
Observations are also noted by the tester (me) alongside users’ testing results.
Participants
-
Total number: 5 (small sample size)
-
Job: Business Owners for various industries
-
Age: 25-41 years old
Tasks
-
Create account
-
Set up project
-
Navigate tabs
-
Make payment
-
Payment confirmation comprehension
Follow up questions
-
Tell me about your experience in completing that task. Did anything, in particular, stand out?
-
Do you have anything to say about the words or the language in this task?
-
Is there anything that could be done here to make the experience a better one?
User flow
For this usability testing, the focus was the Business Owner flow (see below).
Describe your image
Describe your image
Describe your image
Describe your image
User content data
Here are the results that I organized from the raw data that was provided.
Key findings
Ambiguous language
Some of the text confused users, including these:
Sign In / Sign Up / First Time User? (Task 1)
Many users struggled with the Sign In / Sign Up and “First time user” text on the project screen. It’s unclear if first-time users should fill out the form fields or click the CTA button for first time users or both.
“I’m good” on the payment confirmation screen (Task 5)
Users weren’t sure whether this meant to cancel the payment or if clicking “Cancel” meant cancelling the notification or the actual payment.
Poor copy placement
Some instances where users were confused with what action to perform next because of the copy placement:
Not ready to send a message to “workers” (Task 2)
When users want to add a freelancer for their project, some of them don’t feel ready to send a message right away. Having the Send Message button for the Set Up Project task might not be the right place.
Where to cancel payment? (Task 5)
It’s unclear if clicking the X next to each payment would mean cancelling the payment.
Missing information
There were a few instances where users were feeling less confident due to missing information:
No explanation for Send Message CTAs (Task 2)
It’s unclear why users are asked to send a message when setting up a project. There should be some explanation for this, whether it’s because they won’t be notified otherwise or for any additional notes to make?
No payment set up instructions (Task 4)
Currently there are no payment set up instructions asking users how to set up their payment system.
Task completion
Yes
No
“These labels aren't very clear. I think it’s because I don’t use the word 'project' in my business so that confuses me”
- Ashley, Owner of Fashionista
Recommendations
Here are 3 content recommendations to improve the UX of the app:
1. Create distinct user flows for logging in or creating an account
It seems like logging in and creating an account involves the same user flow, where users have to fill out the same form. I highly recommend taking users to a different screen depending on their user intent.
2. Either remove the “Send Message” CTA or explain it
It’s unclear why users are prompted to send a message to the freelancers once they set up the project. It’s best not to have vague CTAs if there’s no context provided (e.g., why do they need to do this?). I recommend modifying the “Send Message" button to “Send Request” so it makes sense what the message is for.
3. Replace Requests for Payment with “Payments due.”
Changing this copy would also make it more obvious for users that this is where they can make their payments.
Recommended follow-up tests
Hypothesis
If we change the call-to-actions by making the Sign In / Sign Up text as distinct actions to take, we can expect a higher completion rate for this page as it will be less confusing for users what action they need to take.
Supporting data
See user responses for Task 1
Methodology
-
Writer: Catherine
-
Customer segment to test: Business owners
-
Percent of traffic for A&B: 50 / 50
-
Minimum stat significance: 90%
-
Success metric: 10% more sign ups.
A/B test #1
Source page (Sign In)
Control shows Sign in or Sign up text followed by a form field.
Variant to test
Variant will show Log In CTA button and Create an Account CTA button after the form field.
Control
Variant
Hypothesis
If we change replace the text “Cancel” and “I’m good” with “Cancel Payment” and “View Receipt” respectively, we can expect a higher completion rate for this page because it will be clearer for users what action they’re taking.
Supporting data
See user responses for Task 5.
Methodology
-
Writer: Catherine
-
Customer segment to test: Business owners
-
Percent of traffic for A&B: 50 / 50
-
Minimum stat significance: 90%
-
Success metrics: 20% reduction in support tickets volume
A/B test #2
Source page - Confirmation
Control shows “Cancel” and “I’m good” CTAs.
Variant to test
Variant will show “View Receipt” or “Cancel Payment”
Control
Variant
