Usability Study – Is 5 Enough?

Usability_study_is_5_enough_Ekta_Rohra_Jafri_IBM_Interactive_Experience (1)

As a start-up girl I know that often usability studies have to be conducted as the sole facilitator, especially when design decisions have to be tested and time is never enough. I have often pushed my limits on legibility and speed writing when conducting a usability study and I still do. Also I’ve learnt to rely on instinct and impressions, letting my subconscious to do the analytics while I gather as much information as I can. But lately I tried mobile video recording as the extra eye on the job. There are many advantages of having a video recording of your usability test.

The first one is that you get two pairs of eyes reading your data from two angles. You from across the desk reading facial expressions and eye movement and the video from over the shoulder recording screen interactions and the user’s ‘think aloud’ thoughts. The second is that you get a tangible record of actual screen interactions to add to your inferences which you can keep going back to and watching again and again. And you get a tool to sell your inferences forward to stakeholders. The 3rd one may not sound like much but put it together with the prioritisation engine that is project management, and you probably have the most powerful tool in your hands. Lastly, it gives you validation with even a really small set of 3 or 5 users.

But is 3 or 5 enough? The answer that I have realised is – Yes. Absolutely. When testing mid development on a half baked product, even 3 is more than enough. With video and two pairs of eyes, you often get repeat impressions that may manifest in emotions and reactions, or screen hesitation and interactions or words. The biggest problems are faced by the largest number of users, its simple mathematics. The more clues you get to form impressions with, the more validation or invalidation of hypotheses you can arrive at. So in essence, the addition of a casual video observer strengthens and speeds up usability study to a very large extent.

But really is 5 enough? When you cover your target audience personas well in a broad spectrum group, you can arrive at decisions on things like screen affordance, understanding, expectations from the product lightening fast. After all, if a button looks like a banner ad, it will look the same to most people. If finding the next step is difficult, it will be more or less difficult for every one. If the screen has contrast issues, most people will have legibility problems with it. The biggest usability flaws come from the consistency of human computer interactions. And finding the 20 biggest usability flaws fixes 80% of the usability issues in an application (The Pareto Principle of UX).

In fact, in my experience and that is consistently felt across the globe also, more than 7 users is a waste of time and energy. Sure, you may find more issues with more people, but the consistency of the issues decreases with more users. And the biggest flaws have the most repeats. So in essence with more than 7 users, you are repeating the same thing again and again and impressions are getting more and more diffuse and difficult to form.

So if I had any advice for my next usability test, or yours, I’d say – crank up the mobile camera, wipe your SD card clean, decrease the quality of video, and shoot away. Video is a friend. And yes, 5 users are enough!

Original Story Published on:

Written By
Ekta Rohra Jafri,
Design Director-Group Manager at IBM Interactive Experience
Ekta is an HFI certified Usability Analyst, IBM Design Thinking Practitioner, Designer & Strategist with expertise in creation of content & products. A story teller who works to translate your vision into a usable, monetizable product. With a stellar 22 yrs of experience out of which 13 yrs in design & user experience of digital entities, she has managed and delivered design projects on platforms from web to mobile to TVs, short form text to 3D reality. An outcome oriented expert who can create, diagnose and repair experiences through research led analysis. She has successfully driven product innovations to drive monetization, guided product strategy to match business goals. Adept in all kinds of observatory and generative research from contextual studies to traffic or social analysis, she is also a mentor & team builder who has set up & led UX teams for COEs. A Design Thinker who loves to bring people to the same platform in terms of thought and action and a futurist who reimagines the future every time she designs.