A Realiable Message
By the start of 2020, WhatsApp took a
strong-willed step to limit the sending forwards from 5 people to a single person.
This shift inspired me to research for a new experience to make a reliable forward message for that single person.
This project is a part of UC Berkeley Extension course (UX Research) in Summer 2020.
What I did
- Generative Survey Research
- Participant Recruitment
- Remote Usability Test
- Data Analysis
- Presenting Insights
- Wireframes (Low & High Fidelity)
- Eye Tracking Plan
Deliverables
- Wireframes
- Prototype
- Research Plan
Tools
- Adobe XD
- Qualtrics
- Zoom
Project Type
- Individual
Duration
- April 2020 - June 2020
Impact
- 60% of the participants felt new forward experience is reliable
- Benchmark user behavior for next research study
The Story
With overwhelming information forwarded in WhatsApp, people trust or mistrust the information without verifying a message;
this leads to losing trust in vital information or with the sender. Below are the headlines from different news sources that call on the seriousness of misinformation in WhatsApp groups.
So, for my research project for the class, I choose to dive into this topic to find better ways to provide reliable information in forward messages.
Research Question
How might we improve reliability and engagement with forward text messages?
Outcome
In the new experience, sending a forward message takes a few steps with an obligation to include source information.
Also, the message with a source link contains a tag that says "forwarded with source"
OUTPUT
Solution
Research Process
Background
According to the CSDS Mood of the Nation (MOTN) survey done in 2018, 61% of users mistrust information shared on WhatsApp. Indeed, the lack of trust is due to the large volume of forwarding messages spread without any “control”.
Additionally, I hypothesize that providing friction and verify before sending a message allows us to spread reliable information. Also, WhatsApp witnessed a 70% drop in 'highly forwarded messages' after imposing limits in April 2020. Thus, it sets the perfect opportunity to test my hypothesis and spread a reliable message.
Generative survey research
I started the project with generative research to understand current users' perceptions of WhatsApp messages; I surveyed with 25 current WhatsApp users. The survey respondents were from family and friends groups distributed across India and the USA.
Connect 25 Responses
After analyzing the survey responses, I found that the multiple user groups are interested to read and verify the information in the message. I divided these user groups based on the sender, message received type, and user age range. Then I plotted a covariate task matrix to analyze the frequency of tasks. For example:
- Sender typeStranger, friend, family, authentic sources.
- Message received type Group, Individual
- Occupation | age Students (18-24), working professional (25-42), entrepreneurs (42- 65).
- Task type The reader would either like to read and verify the message (examination model), ignore the message (neglect model), and provide an opinion (impression model).
From this exploration, I observed a few patterns:
- Messages from a group often get unnoticed.
- Source, sender, and topic of interest are essential to motivate the user in reading a message.
- Users prefer to educate senders about misinformation in the message.
- A few user groups tend to verify the information in a message using the internet.
How might we...
"help a multitasker to obtain important information from a WhatsApp forward message?”
“assist a WhatsApp user to verify the information in a forward message?”
“let a WhatsApp user provide an opinion to a forward message?”
Ideas Kickstarter
Using the survey insights about user perceptions and needs, I made a proof of concept. I sketched the ideas and divided the concept-sketch as follows:
- Improve transparency Adding forwarded count and the message read | unread status to be visually recognized.
- Improve reliabilityFact-check option for WhatsApp messages.
- Improve engagementTo provide opinion on WhatsApp messages. So, I explored a misinformation scale.
Low Fidelity
As my concepts were prepared, I evaluated those initial sketches with a few expert users. Based on their feedback, I changed the scale from six to three -- a detailed scale induces more learning curve, instead to keep it simple (useful, not useful, and funny).
Usability Test
Using Low-Fi wireframes, I conducted a remote moderated usability test for the current and new design. Comparing results between current and new design provides evidence to benchmark the interface performance for future iterations.
Study Procedure
- To avoid order effects and promote consistent learningEach participant is exposed to both current and new design in the same order .
- Think-out loud protocol to understand mental models Participants are instructed to think-out loud method along with answering cognitive probes.
- Post-testPost-session participant completed a post-task questionnaire , post-test questionnaire (SUPR-Q) , product desirability questionnaire for the current and new design.
Analysis
The usability test provides in-depth knowledge of the low-fi wireframes in the form of qualitative data, quantitative data, attitudinal data, and usability issues. I coded the data into four categories - design ideas, surprises, quotes, issues, empirical data for analysis. The top pain points evident from the data were:
- People acknowledge to read a message based on the received type ( friends, family, group): Before the study, 3 of 5 participants said that they ignore the reading long messages when it comes from sources like groups and family. This is 30% less than the expected 90% mark.
- People give importance to validate information using google search to find source of information. 2 of 5 participants said validating information is somewhat easy in the new design. This is 20% less than the expected 60% mark.
- People give value to use the summary feature only if a message is genuine. 4 of 5 participants completed the task with a 20% reduction in time in the new design.
- People prefer to verify using a source link instead of third-party fact-checkers. 3 of 5 participants completed the task with a 31% increase in time-on-task with the new design.
Solution Prioritization
With these insights and usability pain points. I created a prioritization matrix based on the impact and frequency of the problems. This method guided in making an informed decision aligning with the research question.
Medium Fidelity Wireframes
With the insights prioritized, I created medium fiedlity wireframes with a new flow for forwading a long text message. The steps follows the concept called information ‘chunks’ ; to reduce cognitive load for the user as well simpler steps to improve effciency of the sending a message.
Future Directions
EyeTracking plan
To identify what captures user attention.
Accessibility plan
To make interface universal for all user groups.
Engaging stakeholders plan
To make research a success.
EyeTracking
Incorporating an eye-tracking method helps us to understand whether is it the information in a tag or an icon that captures user attention to read a message. It could be interesting if people miss seeing it or they overlook it due to the content or the icon.
Eyetracking plan document
Accessibility test
According to the US Statista survey done in 2019, 13% of users belong to 46-55 years, 7% of users are above 56+ years, a typical age to develop vision disorders.So, it is crucial to make the design accessible to all users. Below is the accessibility plan for WhatsApp new version that I plan to conduct with the new design, after testing with the current WhatsApp design (provides room for improvement).
Involving stakeholders
Last, I believe that making research success requires the involvement of the stakeholders during the research. It helps to align better with user needs. It promotes empathy for users and solves problems faster . Below is my stakeholder engagement plan for this project.
Learnings
This was my project as a part of the UX Research program at UC Berkeley Extension. It was a great learning experience with Jennifer Romano Bergstrom taught us methods to plan and conduct research in the remote & in-person settings. These are the top four things I can reflect upon after the usability test:
- Test with enough users Data from five participants(n=5) is hard to determine statistical significance, so it is required to test with more users to achieve significance. Also, I plan to recruit more participants to avoid no-shows.
- Recruit diverse user groups Including age groups above 55 provides data variability that could be generalized for a scalable audience. I plan to recruit for the next iteration
- Conduct pilot studyA dry run of the procedure helped me to identify gaps in the research protocol and with other materials.
- Data analysis after each session Process data after each session provides user insights to improve further usability sessions.
