As a start-up girl I know that often usability studies have to be conducted as the sole facilitator, especially when design decisions have to be tested and time is never enough. I have often pushed my limits on legibility and speed writing when conducting a usability study and I still do. Also I’ve learnt to rely on instinct and impressions, letting my subconscious to do the analytics while I gather as much information as I can. But lately I tried mobile video recording as the extra eye on the job. There are many advantages of having a video recording of your usability test.
The first one is that you get two pairs of eyes reading your data from two angles. You from across the desk reading facial expressions and eye movement and the video from over the shoulder recording screen interactions and the user’s ‘think aloud’ thoughts. The second is that you get a tangible record of actual screen interactions to add to your inferences which you can keep going back to and watching again and again. And you get a tool to sell your inferences forward to stakeholders. The 3rd one may not sound like much but put it together with the prioritisation engine that is project management, and you probably have the most powerful tool in your hands. Lastly, it gives you validation with even a really small set of 3 or 5 users.
But is 3 or 5 enough? The answer that I have realised is – Yes. Absolutely. When testing mid development on a half baked product, even 3 is more than enough. With video and two pairs of eyes, you often get repeat impressions that may manifest in emotions and reactions, or screen hesitation and interactions or words. The biggest problems are faced by the largest number of users, its simple mathematics. The more clues you get to form impressions with, the more validation or invalidation of hypotheses you can arrive at. So in essence, the addition of a casual video observer strengthens and speeds up usability study to a very large extent.
But really is 5 enough? When you cover your target audience personas well in a broad spectrum group, you can arrive at decisions on things like screen affordance, understanding, expectations from the product lightening fast. After all, if a button looks like a banner ad, it will look the same to most people. If finding the next step is difficult, it will be more or less difficult for every one. If the screen has contrast issues, most people will have legibility problems with it. The biggest usability flaws come from the consistency of human computer interactions. And finding the 20 biggest usability flaws fixes 80% of the usability issues in an application (The Pareto Principle of UX).
In fact, in my experience and that is consistently felt across the globe also, more than 7 users is a waste of time and energy. Sure, you may find more issues with more people, but the consistency of the issues decreases with more users. And the biggest flaws have the most repeats. So in essence with more than 7 users, you are repeating the same thing again and again and impressions are getting more and more diffuse and difficult to form.
So if I had any advice for my next usability test, or yours, I’d say – crank up the mobile camera, wipe your SD card clean, decrease the quality of
Original Story Published on:
Ekta Rohra Jafri,
Design Director-Group Manager at IBM Interactive Experience
Ekta is an HFI