Running London-based experiments for Intelligent Maps – The difficulties of participant recruitment

Extreme Citizen Science is a philosophy of situated bottom-up initiatives and local practices which take into account local needs, practices and cultures to work with broad networks of people to design and build new devices and knowledge creation processes which can transform the world. In ExCiteS we have worked with various communities all over the world for these purposes; examples include our work with BaAka hunter-gatherers in Cameroon collecting data for animal monitoring and wildlife crime mapping and collecting data for monitoring cattle invasions with the Ju/‘hoansi of Nyae Nyae Conservancy in Namibia. 

In the work we do in ExCiteS and for the tools that we develop we face various challenges. First, we work with people from various cultural backgrounds, people who face their political and economic challenges and who have trust concerns (especially when it comes to getting engaged and participating in this type of projects). Although there is some research that focuses on geographic data collection in similar contexts, user experiences and design issues remain mostly anecdotal evidence, which is hardly communicated outside the boundaries of specific projects. Second, we work in places where literacy cannot be taken for granted – several users are low-literate (this includes people who don’t read or write either by choice or chance and people who can write or read with difficulty). Third, the people we work with may have limited or no prior experience in interacting with digital technologies (incl. smartphones). We, therefore, need to implement designs which have the potential to help overcome these potential digital divides. 

Implementing a user-centred design approach can be extremely complex; travelling to these areas to integrate user input at all stages and evaluate prototypes can be very costly. From a methodological point of view, conventional HCI methods are not culturally universal and therefore an ’out-of-the-box’ implementation is mostly inappropriate for cross-cultural HCI (Human-Computer Interaction) and HCI4D (HCI for Development) research. For example, a major obstacle in using participatory methodologies when working with egalitarian societies is that it might not be appropriate to isolate specific participants and work with them since it is much more suitable for everyone in the community to participate. Also in specific cultural contexts ‘criticism’ is not considered appropriate neither it is interrupting people to ask questions about specific features; therefore it is extremely difficult to get feedback.  Inspired by colleagues who have designed and run HCI experiments in urban centres with low-literate users (e.g., Knoche and Huang, 2012) we decided to explore the same approach and work with London-based participants. Our aim was to test interface design features or develop prototypes that would subsequently inform the design of our interfaces and which we would then further evaluate in the field with indigenous communities. In subsequent blog posts we will detail our experience carrying out these experiments, but before that, we report on the process of identifying suitable participants and proxy users, which is the first major challenge. 

London experiment 1: February – September 2018 – Investigating the use of linear structures for Sapelli, our data collector app

Several studies demonstrate low-literate people’s difficulties in understanding and using menus that are based on hierarchies and instead recommend linear structures with up and down button or scrollbars to navigate them. In this experiment, we took into account these research findings to develop and further evaluate four user interfaces with low-literate participants in the UK. Caroline Trimm (UCLIC MSc student) conducted these experiments to evaluate Sapelli’s hierarchical structure against Tap&Map and two linear structure interfaces. For more information about this study, you may refer to our paper presented in the ECSCW ‘19 conference. 

Identifying and working with low-literate participants in London was not simple. Participants were recruited through two main channels – word of mouth and advertisement fliers that were put out through adult literacy learning groups such as the Literacy Trust. In total, over 50 groups were contacted across the UK, including adult learning centres, adult literacy learning groups, citizens advice bureau, Job Centres, churches, community centres, local radio stations, and garden volunteering charities. Posters were placed in libraries, supermarkets, community notice boards and local theatres across London, while adverts were placed on social media. Ten adult learning colleges, literacy groups and community centres were visited to encourage participants to take part. Most participants were recruited using the snowball sampling method, whereby recruited participants recommend other possible participants. The recruitment process lasted approximately two months and the 13 participants recruited were given a £20 cash incentive for taking part in the study which lasted an hour per session. To assess their literacy skills we asked them questions from the UK Government 2011 Skills for Life Survey, which reviews literacy, numeracy and ICT levels in England. 

London experiment 2: January – July 2019 – Developing prototypes for Intelligent Maps, out data visualisation platform

From the results of our London experiment 1, we evaluated that people with limited or no ICT experience (especially in the context of data collection) was a particularly suitable user group for the purposes of the ExCiteS research. Also we found that in terms of interacting with our technologies there were many similarities across these audiences (e.g., older people had difficulties browsing the NFC cards due to arthritis, while those in the field had the same problems due to carrying other equipment or being on a boat fishing at the same time; rough skin was found in both contexts to prevent seamless interaction with the touchscreen). An experiment with older London-based participants was designed to get preliminary user requirements for the data visualisation tool which is currently under development in ExCiteS.

Sanayah Malik, MSc student at UCL Anthropology and Carol Iglesias MA student at the Goldsmiths Centre for Research Architecture and intern at ExCiteS carried out this experiment. In order to find the participants, thorough research of about 70 relevant organisations and neighborhood centers that offered activities for people of third age/retired people took place. Several organisations were not allowed to share any contact details for their members and did not feel comfortable offering the opportunity to be part of an experiment with their members. Therefore, a visit in person distributing leaflets in the centers as well as in other public spaces (libraries, cafes, etc.) was required. Two centres – i.e. Calthorpe Garden and Somerstown Community Center – responded positively and a meeting was arranged to discuss in person and present further details of the project. This whole process lasted about two months and a total of 13 participants were finally recruited (eight were non-English speakers). All participants were given a £30 cash incentive for taking part in the study, which this time involved collecting data for a period of two weeks, followed by interviews and focus groups to design paper prototypes for visualisations of the data they have collected. 

Conclusions 

Despite the difficulties and long timescales required to identify and recruit suitable participants the findings of our studies are very exciting. The results of the first London experiment informed the design of a similar experiment with indigenous communities in Namibia in which we found very similar results. The analysis and results of the second experiment is work still in progress and we are planning to take them into account to inform the design of a low-fidelity prototype to visualise the data collected, which will be also tested in the field. 

Lessons Learned

• Start by creating a list of relevant organisations that can be contacted. Expect low response rates (around 3%). For this, we recommend visiting the organisations and other public spaces to distribute information fliers and explain the project in person. 
• Start the recruitment process 2-3 months in advance. 
• Providing incentives, may not be necessary depending on the nature of the project and the benefits it has for participants, but we found that providing money incentives significantly improved our recruitment rates. 
• Design the data collection experiment around a topic which will interest participants and keep them motivated. 
• Keep your participants and organisation representatives informed at all stages about the process and project outcomes. They might encourage participation and help with recruitment in future projects. 
• It is harder to identify low-literate participants via relevant organisations due to the associated stigma. It is easier to identify and recruit them via word of mouth and snowball sampling might work particularly well. 
• Designing short in duration experiments, rather than experiments which require longer timescales, to achieve higher recruitment rates. 
• When dealing with longer timescales, we found that it helps if the project is embedded in pre-existing communities/groups, where the links between participants help them maintain interest in the project. 
• When groups have varying technological literacy levels, participants who feel less confident often tend to intervene less in focus groups. It is important to stress that it is their experiences that are particularly helpful and at times necessary to schedule individual interviews to ensure everyone’s experience is communicated.

Do you have any experience running similar experiments? Do you have any suggestions that could help improve participant recruitment? If so, please get in touch with us to share and discuss our experiences!