User Experience (UX) experts are divided on the matter. One camp sees remote user testing
as a Holy Grail of sorts, given that is more cost-effective, allows access to a much greater pool of potential participants to recruit, and avoids a host of hassles, from transport issues to scheduling the flow of participants to the testing space3
. The other camp believes face-to-face testing cannot be undervalued for the nuances it reveals in user behaviour. The reality is that Covid-19 leaves us no choice: user-centred design must find ways to adapt to remote testing and research.
Illustrations done by or sourced by: Nikita Golubev and Freepik
Things may not, however, be so black and white. A middle road that has been suggested by some in the field - including a Joburg-based UX design company Experio4
- is finding ways to conduct user testing and research with as much caution as possible. Testing spaces are highly sanitised, masks are worn, and conversations with users are held through Perspex screens. The upside is that in-person testing and interviews can take place, but at what cost? It could be argued that screens and strict hygiene measures, while a necessity, put some participants on edge and interrupt the rapport that is vital for establishing a relationship between researcher and participant.
Without the benefit of being able to observe a user in as relaxed and ‘normal’ a state as possible, some view remote testing as preferable over the ‘face-to-face with caution’ alternative. This isn’t, however, the end of the discussion around this approach. As design expert Nick Bowmast5
outlines, the UX community needs to keep in conversation by sharing research, reflecting on experiences from the field, and documenting what’s been learned to arrive at best practice approaches. After all, if there’s any industry that understands how iterative processes work, it’s user-centred design6
So we’re back where we started: with no choice but to embrace remote usability testing, but making the best of the situation. One way to make the most is to take the hint from Nick Bowmast and use this time to study remote testing itself and share those findings. If you hadn’t guessed yet, that’s what the rest of this article will summarise: how the team at one Cape Town-based UX and design-thinking company has approached remote testing and what they’ve learnt.
That company is How Might We
, who over the course of lockdown have completed over forty remote user research sessions 7
, each of approximately one hour and all testing digital products from the healthcare, insurance and investment sectors. Testing falls under two umbrella categories: usability testing8
(can customers find the buttons on your app, for example) and ethnographic research9
(how your digital products interface with the motivations, beliefs and cultural tensions of your users, for instance).
Let’s begin with a case study that looks at ethnographic research for a funeral cover product aimed at individuals from lower-resource settings. Testing was conducted entirely over mobile to reach participants without desktop access. The first snag was smartphone penetration, which prevented the research from engaging with a host of intended users. Second, due to varying degrees of connectivity, it was necessary to send participants mobile data. This also necessitated inviting more participants than required, since a few dropped off due to connectivity failures.
For some users, these types of technical issues adversely affected their ability to feel safe. For many others, however, the experience of talking on the phone is second nature, so they readily relaxed in their home environments and became candid – often much more so than How Might We
has found in their office environment in town. Issues around researchers not being able to read body language were offset by the ability to read tone of voice from participants who opened up as if chatting to a friend. One must consider that the nature of this research project meant that over-the-phone testing was a good fit, because participants were simply being asked how they felt about the topic and how it relates to their family.
The second research setting involved usability testing around a short-term insurance product aimed at individuals from higher-resource settings. This is perhaps the best fit for remote testing, since participants have access to the smartphones and connectivity that enables this research from a distance. Screen-recording allows researchers to analyse how participants engaged with the interface, providing rich data on usability issues.
The major benefit here was that these higher-income individuals availed themselves more readily for remote testing than they typically do for in-person studies, and they afforded the researchers more of their time. This is likely since for these users, working from home provides them with greater flexibility to pursue tasks at their leisure. The drawback experienced by users in this scenario was around privacy and security, with some expressing concerns that the research team would have ongoing access to their phone and their financial information. To overcome this, the How Might We researchers found that they needed to allocate more time than normal to explain clearly what was being recorded and what was not.
Face-to-face usability testing and research will likely always be preferable, but it can only be preferable when it’s a viable option. Covid-19 presents us with an opportunity to refine remote testing methodology and implementation, share our findings and move collectively toward a set of best practices.
Furthermore, Covid-19 presents the right time to conduct this research, since our lives have fundamentally changed. Usability testing and ethnographic research are critical to understand the shifting motivations and behaviours of users navigating their way through the new landscape. As digital takes centre stage in consumers’ lives, it couldn’t be a better moment to unpack and address the challenges of remote usability testing and ethnographic research.
If your business is looking to get real feedback from your customers on your current product or future concepts, get in touch
with How Might We
. A team of experts will guide you to discover how usability tests and research form the most important part of customer-centric design.
Phone: +27 21 010 1700