Evaluating an online asynchronous Multiple Mini Interview (MMI) with ten principles of fairness built-in: findings from a cross-sectional feasibility study
More details
Hide details
School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, United Kingdom
Publication date: 2023-10-24
Corresponding author
Alison J Callwood   

School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, United Kingdom
Eur J Midwifery 2023;7(Supplement 1):A41
Ensuring equity, inclusivity, and diversity in selection to health professions is becoming increasingly recognised as an ethical and practical imperitive1. However, fulfilling our responsibility to ensure fair selection has never been more challenging to enact. This is because of unintended biases that are intrinsic to human assessment compounded by unprecedented Covid-driven adoption of online systems in the absence of published evidence.

Material and Methods:
We undertook a rapid review of published literature within and outside health professions2 (2011- September 2021) to identify key principles optimising online interviews. Our online interview system grounded in Multiple Mini Interview (MMI) methodology was customised accordingly and applicants underwent a seven station, four-minute circuit. We aimed to explore reliability for all users, fairness for a random stratified sample (gender, age, UK/non-UK, disability) usability and acceptability with applicants to Nursing and Midwifery programmes at one UK university.

Ten principles key to building-in fairness were elicited from 49 articles identified and reviewed. These included: • Incorporate language that supports the affirmation of values e.g., “well done for getting this far”. • Use encouraging words/phrases in the interview e.g., “good luck”. • Soften language instructions e.g., “when you are ready …” • Minimise verbal load of interview content. • Provide prior opportunities for candidates to familiarise themselves with the online format. • Recommend generic, blank backgrounds for video-based interviews. • Accommodate access and engagement for neurodiverse applicants. • Ensure diversity of interviewers. • Avoid culturally sensitive subjects in interview content. • Ensure the use of gender-neutral language and pronouns. Data were analysed from 712 applicants. Cronbach’s α were good-excellent across questions within each scenario (mean Cronbach’s α 0.72, range 0.64-0.89,). Random sample sub-group analyses (n=284) showed similarly positive results with α female/male: 0.74/0.87; age: <20years/ >21 years 0.76/0.83, disability/non-disability: 0.78/0.88 and UK/non-UK 0.78/0.77 respectively. Applicants found the instructions helpful/very helpful (89%) and qualitatively viewed the flexibility, use of different staff, Covid safety, a relaxed environment and cost savings as advantages. Interviewers (96%) found the system intuitive and easy to use, and the flexibility and time-savings (70%) reduced stress.

With ten principles for equity built in, these preliminary findings suggest the online interview is reliable, fair, time-efficient, and acceptable. The MMI interviews were pre-recorded using inclusive language by diverse staff, representative of the University community. This promotion of equity was positively evaluated by applicants. Additional time and an intuitive system user interface met the needs of neurodiverse applicants. Many organisations continue to use online interviews post-Covid. Best practice guidance on their design and configuration to optimise performance and mitigate potential unfairness issues is not readily available. These findings therefore provide unique internationally applicable insights necessary to embed fairness into online selection approaches across health professional selection. Funding: Part of this study was funded by United Kingdom Research and innovation (UKRI). Conflict of interest: Alison Callwood is CEO of the UK University spinout company set up with funding from Innovate UK to build the online interview.

Council of Deans of Health. Equality, Diversity and Inclusion. Accessed February 2023.
Linos E, Reinhard J, Ruda S. Levelling the playing field in police recruitment: Evidence from a field experiment on test performance. Public Administration. 2017;95(4):943-956. doi:10.1111/padm.12344
Journals System - logo
Scroll to top