As a parent during a pandemic, I have been exposed to a fair number of apps teachers use to communicate with parents and keep up with students’ assignments. As I interacted with them, I was lost wondering how exactly I was supposed to know what is where or what goes to who? I admire the technology and the usefulness of the app as a learning and communication tool, but it’s not easy to use! My experience left me overwhelmed, confused, and a little frustrated. This can be the case for any product that has not been properly usability tested. Whether you are publishing your own website or creating an app, usability testing is a benefit to you and your users.
You Are Not Your User
Usability Expert, Steve Krug, who has over 25 years of experience consulting on usability, says usability testing can see if “an average person can successfully use the thing for the intended purpose without more trouble than it’s worth” in his talk featured on Digitial.Gov, Usability Testing with Steve Krug (June 2018). I think about this as asking someone to proofread something I have written. I have read and re-read the same sentences over, I spent time creating something and emotionally invested myself in the work. Having someone else read over it from an outside perspective (someone unfamiliar with it) adds valuable insight to what is being read or used. This allows feedback on how clear a point is or is not.
Moderated vs. Unmoderated Testing
At the end of Krug’s talk, he wrapped up with a few tips and resources where he mentioned moderated (remote) testing and unmoderated testing. I wanted to know more about what the differences were and if one would be better than the other. After a quick search, I came across “Usability Testing: Moderated vs Unmoderated” by Nick Babich, UX Architect and contributing writer on the Adobe XD Ideas Blog. He gave an easy-to-understand explanation of the two types of testing and the pros and cons of each.
What’s the difference?
| Moderated | Unmoderated |
|---|---|
| +Controlled +User Engagement -Costly -Takes Time |
+Less Costly +Less chance of bias -Lack of guidance for user -No opportunity for immediate follow-up questions |
| Working directly with a user to “guide” them through the testing. Testing in a lab, workplace, or remotely through screen share. |
No moderator present at testing. Testing takes place in the user’s own environment. Can be more useful to test specific sections or parts. |
Which one should I use?
Between the moderated and unmoderated tests, I found it came down to cost as an outweighing factor. Krug and Badich mention that moderated testing has added expenses such as paying testers, renting a space, or hiring an outside firm to conduct the tests. This shouldn’t keep you from doing it though. Remote testing is easy to do with screen sharing and video chats with a moderator and a user. If you “keep it simple” as Krug suggests, you can find volunteers willing to try your product. This is one solution that could work for cutting the cost
Both testing methods can be time-consuming. A successful test depends on the preparation put into the test. Writing a detailed list of tasks for the user, preparing the testing space, and framing the testing to the guidelines of your goal requires time invested upfront. My conclusion is that it’s not better or worse to use one over the other, but rather it’s worse to have done none and best to have both. If your site is live and your users have trouble navigating the pages or can’t find what they are looking for they will, as the tester from Krug’s talk says, “bail”. Without the usability test, you lack the insight that would have prevented this frustrating and confusing experience. You are not your user; you know too much and have too much invested in the product to see it as your user would.
The value of moderated and unmoderated testing can’t be compared one over the other because they are answering different questions. Applying both types of testing to a product offers a full picture of how your user will interact and react to it. Moderated answers question “why” and “what” as you get live immediate feedback from the user, where unmoderated answers “how many?”. How many testers experienced the same roadblocks or asked the same questions? How user-friendly is it for you?
This chapter is a revised version of a blog post titled “ Take to Usability Test” on Mau Does Web Publishing.
