Training in the health professions has benefited greatly from integrating simulation as a training tool. Simulation has supported the acquisition of skills for procedures that are high-risk and infrequent through exercises that mimic patients and environments without compromising patient safety. In addition, it has been shown to improve trainees' skills, knowledge, and behavior,1 which are essential attributes of health care. Simulation can be successfully used to train multidisciplinary teams in complex scenarios and to assess clinical skills.2
Technological advances have enhanced simulation in health education by introducing simulators with a high degree of realism. Physical realism includes “factors such as environment, equipment, and related tools.”3 It is recognized that physical realism occurs on a continuum from high (ie, extremely realistic, complex, interactive experiences) to low (ie, simplistic, low-tech, not requiring programming). As such, we sought to include studies where high/low may be a comparison of 2 closely related or vastly different simulation modalities. High physical realism simulators are reported to produce better results than low physical realism simulators.1,4
The World Bank uses a categorization framework to evaluate and classify nations into 4 distinct income strata contingent on their gross national income per capita.5 These encompass low-income (<$1135), lower middle-income ($1136–$4465), upper middle-income ($4466–$13,845), and high-income nations (>$13,845). The first 3 strata are defined cumulatively as low- and middle-income countries (LMICs).
In LMIC settings with relatively few skilled health care providers, simulation-based training could have a remarkable impact.6 However in such locales, the capital and ongoing maintenance costs of high-fidelity systems are generally unaffordable at the scale that would be needed.7 The LMICs face many constraints that hinder their ability to acquire high-fidelity manikins for medical training effectively. Although establishing advanced simulation centers is crucial for promoting enduring progress in medical and surgical training, it is imperative to recognize that this undertaking is characterized by demanding financial resources and time investments. A review of the strengths and limitations of surgical simulation initiatives based on an international survey encompassing 42 simulation centers prioritized the need for robust public funding allocations, dedicated simulation technicians, and unwavering endorsement and support from institutional leadership.8 Moreover, it is noteworthy that using high-fidelity models in this context in all settings has yet to conclusively demonstrate a substantial advantage over their low-fidelity counterparts in pedagogical outcomes.9
There are few reports comparing high and low physical realism simulation specifically in LMICs. Therefore, this systematic review examined whether the level of physical realism of simulators/task trainers makes a difference in clinical, educational, and procedural outcomes in these settings.
METHODSThe Society for Simulation in Healthcare commissioned this review as part of an effort to establish practice guidelines. The review was meticulously planned, executed, and reported in strict accordance with the quality standards set forth by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) for reporting metaanalyses (see Table, Supplemental Digital Content 1, https://links.lww.com/SIH/A988, which presents the completed PRISMA checklist).10 Although a study protocol had not been publicly disclosed before this review, an a priori protocol was created and diligently followed (see Table, Supplemental Digital Content 2, https://links.lww.com/SIH/A989, which presents the review protocol). Because this was a systematic review of previously published studies, there was no need for ethical approval or patient consent.
Research QuestionThe PICO format (Population, Intervention, Control, Outcomes)11 was used to formulate the research question: In health care providers and/or health care trainees/students engaging in simulation training in LMICs (P), does a higher physical realism simulator or task trainer approach (I), as opposed to a lower physical realism simulator or task trainer approach (C), affect educational outcomes (immediate and retention): participant satisfaction, knowledge, skills, and attitudes; clinical outcomes: changes in health care practitioner behaviors and patient outcomes; and process outcomes: cost (O).
Study EligibilityWe included all comparative prospective and retrospective randomized and observational studies that have been published since 2011 to ensure the relevance of the evidence. We aimed to include publications in all languages, provided they had an available English abstract. Excluded from consideration were studies involving unpublished findings, trial protocols, commentaries, editorials, and review articles.
Data SourcesWith the assistance of an information specialist, we devised search strategies using a blend of keywords and standardized index terms focused on assessing the level of physical realism in simulation settings within LMICs, involving comparative analyses (see Table, Supplemental Digital Content 3, https://links.lww.com/SIH/A990, which presents the search strategy). These searches were initially executed on January 29, 2022, across CINAHL via EBSCO (1963+), Ovid Embase (1974+), and Ovid Medline (1946+ encompassing epub ahead of print, in-process, and other nonindexed citations). We limited the search to commence from 2011, and ineligible studies were removed from consideration. An update of these searches was conducted on January 24, 2023.
Study SelectionThe titles and abstracts of all potentially eligible studies underwent a dual and independent screening for inclusion. Subsequently, the selected studies underwent a more comprehensive, duplicate screening for eligibility, in accordance with predefined inclusion and exclusion criteria. In cases where discrepancies arose between the reviewers, resolution was achieved through discussion. In addition, the included articles were thoroughly examined for any supplementary citations.
Data CollectionEach reviewer independently extracted data from every study, and any discrepancies were addressed through discussion until a consensus was reached. The data included study design, country, World Bank classification, study population, study duration, description of intervention and control, and outcomes measured.
AnalysisGiven the significant heterogeneity in clinical and methodological aspects among the identified studies in our search, conducting a metaanalysis was considered impractical. Instead, we opted for a narrative summary. To evaluate the risk of bias, we conducted independent and duplicate assessments using the revised Cochrane risk-of-bias tool (RoB2)12 for randomized trials and the ROBINS-I tool13 for nonrandomized studies. In addition, we assessed the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology.14
RESULTS Study SelectionThe search identified 2311 articles after removal of duplicates. Of these, 2198 were excluded leaving 113 full-text articles to be screened for eligibility (see Fig. 1). In total, 11 studies were identified for inclusion.15–25
FIGURE 1:Flow chart of inclusion of articles.
Study CharacteristicsNine randomized controlled studies with 627 participants15–23 and 2 case-controlled studies with 159 participants24,25 were conducted between 2006 and 2022 (Table 1). Most of the studies15–22,25 were conducted in upper middle-income countries, as designated by the World Bank classification.5 One study was conducted in a lower middle-income country24 and 1 was conducted in a low-income country.23 Participants included students and residents from both nursing and medicine. Higher physical realism was compared with lower physical realism training for a broad range of educational interventions including basic surgical skills, intramuscular injection skills, nurse training, life support awareness and skills, clinical examination skills, and more complex medical interventions (fiber optic bronchoscopy). Outcomes identified in the studies included participant satisfaction, participant confidence, skills acquisition, and performance in a clinical environment.
TABLE 1 - Study Characteristics Study Design Setting/Date Participants Research Question Intervention Control Outcomes Conclusion Limitations Akdemir 2014 RCT Türkiye (UMIC)ACLS, advanced cardiac life support; ATLS, advanced trauma life support; World Bank classification: EMS, emergency medical service; LIC, lower income country; LowMIC, lower middle-income country; UMIC, upper middle-income country.
The risk-of-bias assessments are summarized in Tables 2 and 3. There were no studies that were assessed to be low risk of bias. The main issues identified with the studies related to inadequate blinding of participants and assessors, inadequate randomization, and unclear selection processes. The certainty of evidence was judged to be very low for all outcomes, downgraded for very serious risk of bias and inconsistency.
TABLE 2 - Risk-of-Bias Assessment Using ROB2 for Randomized Controlled Studies Study Total (N) Population Allocation Generation Allocation Concealment Blinding Participants Blinding Assessors Outcome Complete Outcome Selective Other Bias Overall Denadai 2012 36 Medical students Some concerns Low Low Low Low Low Low Some concerns Akdemir 2014 60 Gynecology residents Low Low High High Low Low Low High Denadai & Oschiiwa 2014 40 Medical students Some concerns Low Low Low Low Low Low Some concerns Denadai & Saad-Hosne 2014 60 Medical students Some concerns Low Low Low Low Low Low Some concerns Jiang 2018 46 Anesthesia residents Low Low Low High Low Low Low High Garcia 2019 36 ATLS candidates High High High High Low Low Low High Amanak 2020 73 Midwifery students Low High High High Low Low Low High Murthy 2020 214 Medical students & residents Low Low High High Low Low Low High Labuschagne 2022 53 Medical students Some concerns Low Low Low Low Low Low Some concernsATLS, advanced trauma life support.
One randomized controlled study (n = 45) compared higher physical realism with lower physical realism mannequins for Advanced Trauma Life Support training. Although the higher physical realism mannequin performed better than lower physical realism mannequin in most procedures, students and instructors found that both were acceptable for teaching and learning Advanced Trauma Life Support surgical skills.20 Data were presented graphically for this study and not in tabulated format, meaning that absolute numbers for each group are unclear.
Two case-controlled studies24,25 (n = 159) showed mixed levels of satisfaction between higher and lower physical realism models. When used for teaching an Advanced Life Support protocol, the high physical realism group gave higher satisfaction scores on all items except the one about the variety of learning materials and activities used during simulation.24 With regard to nursing education, the students' satisfaction scores were high in both groups. The satisfaction score in the control group (mean [SD], 4.50 [0.35]) was significantly higher than that in the intervention group (4.23 [0.33]), t = 3.048, P = 0.003.25
Participant ConfidenceFour randomized controlled studies (n = 182) showed higher confidence levels for higher levels of physical realism.17–19,21 Three studies from the same center involved the use of didactic training only (control), ethylene-vinyl acetate bench model (low physical realism), or animal bench model (high physical realism) to teach medical students a variety of suturing skills. In the study addressing basic suturing skills, students felt more confident (P = 0.00) to perform both types of sutures after training [simple suturing: control 1.67 ± 0.65, low 3.17 ± 0.72, high 3.25 ± 0.75; subdermal suturing: control 1.33 ± 0.49, low 2.83 ± 0.72, high 2.92 ± 0.67].17 With regard to elliptical suture skills, students felt more confident to perform the skill in both simulation groups (control: 2.5 ± 0.52; low: 4.25 ± 0.73 and 4.13 ± 0.64; high: 4.13 ± 0.64 and 4.25 ± 0.71; P = 0.00).18 For rhomboid flap skills, both simulation groups showed similar confidence rates and superior to the didactic training group (control: 1.75 ± 0.75; low: 3.17 ± 0.83 and 3.25 ± 0.62; high: 3.68 ± 0.67 and 3.33 ± 0.78; P < 0.05 between intervention groups; P > 0.05 intervention vs. control).19 Finally, a study comparing a virtual reality mannequin with a standard mannequin for fiber optic bronchoscopy showed that participants' confidence increased after training but did not differ significantly between groups.21
Two case-controlled studies (n = 159) showed mixed results for confidence between higher and lower physical realism models.24,25 The high physical realism group teaching Advanced Life Support awareness showed greater confidence in all parameters except the one about the resourcefulness of the techniques in learning simulation.24 For nursing education, the mean self-confidence score in the control group (4.08 [0.38]) was significantly higher than that in the intervention group (3.79 [0.34]), t = 3.120, P = 0.003.25
Skills AcquisitionSix randomized controlled studies (n = 469) examined skills acquisition, including self-efficacy scoring. There were no differences in skills acquisition between higher and lower physical realism models for intramuscular injections, suturing, fiber optic intubation, and breast examination.16–19,21,23 One study addressing teaching of intramuscular injection skills showed that the hybrid simulation model (intervention) group had better total General Self-Efficacy Scale scores than the standard (control) model group (91.1 ± 14.98 vs. 85.23 ± 12.05, P < 0.05). The mean score of the intervention group in the State-Trait Anxiety Inventory was lower than that of the control group (32.02 ± 8.81 vs. 35.12 ± 7.82, P < 0.05). The mean score of the intervention group was higher in the Guide to Performing Intramuscular Injections compared with the control group (28.02 ± 5.36 vs. 21.86 ± 2.85, P < 0.05)16 One study using higher versus lower physical realism of bench models for simple and subdermal suturing training showed that animal and organic bench model groups demonstrated similar performance. These in turn were both better in the Global Rating Scale evaluation (P = 0.00) compared with the didactic-only (control) [simple suturing mean difference (MD): control 3.27, low 14.56, high 14.77; subdermal suturing MD: control 1.63, low 13.17, high 14.00].17 One study that looked at elliptical suture training showed no significant difference between groups (Mean difference for low physical realism groups: 14.57 and 14.75, high physical realism groups 15.13 and 15.00; P > 0.05).18 Both high and low physical realism groups of rhomboid flap skills had similar posttraining results and were better than didactic-only (control: 13.82 ± 1.11; low: 24.42 ± 1.11 and 24.58 ± 1.24; high: 24.42 ± 1.50 and 24.5 ± 1.24; P < 0.05 between intervention groups; P > 0.05 intervention vs. control).19
One study looking at virtual reality versus standard mannequin training for fiber optic intubation showed that plateaus in the learning curves were achieved more swiftly after 19 (95% confidence interval, 15–26) practice sessions in the intervention group as opposed to 24 (95% confidence interval, 20–32) in the control group. No significant difference was found between the groups in procedure time [13.7 (6.6) vs. 11.9 (4.1) secs, t =1.101, P = 0.278] or global rating scale [3.9 (0.4) vs. 3.8 (0.4), t = 0.791, P = 0.433].21 Finally, another study using higher versus lower physical realism to teach clinical breast examination skills showed that the mean difference in examination scores were not significantly different [postlecture MD 0.86 ± 0.69, P = 0.16; postlecture and sim MD 0.03 ± 0.38, P = 0.66; crossover examination MD 0.1 ± 0.37, P = 0.29]23
Performance in Clinical EnvironmentTwo randomized controlled studies (n = 113) measured provider performance in a clinical environment. There was no difference in provider performance between higher and lower physical realism models of laparoscopic surgery and cardiopulmonary resuscitation (CPR).15,22 A study comparing a higher with a lower physical realism simulation trainer for laparoscopic surgery showed that the median (range) total score of the general skills part of the objective structured assessment in laparoscopic surgery rating scale was 17 (15–19) points in the higher physical realism group, 17 (16–18) points in the lower physical realism group, and 11.5 (10–14) points in the control group (senior residents with previous laparoscopic experience). The median operation time to complete the procedure was 340 (260–400) seconds in the higher physical realism group, 340 (270–430) seconds in the lower physical realism group, and 425 (320–530) seconds in the control group. There were no differences between the higher and lower physical realism groups15 One study comparing higher with lower physical realism mannequins for CPR training showed that the median flow fraction for the higher physical realism group was 78.0% (interquartile range [IQR], 63–89) and for the lower physical realism group 80.0% (IQR, 74–85). The median number of compressions for the higher physical realism group was 104 (IQR, 101–109) and for the lower physical realism group 107 (IQR, 79–124). Both groups achieved a 100% compression rate with adequate depth. The maximum total effectiveness of both groups was 99%. No statistically significant difference was seen for the overall percentage of compression effectiveness between the groups22
DISCUSSIONThe findings of this review suggest that lower as opposed to higher physical realism simulators/task trainers should be used for health care simulation training in LMICs. The lack of evidence for any clear benefit or desirable effects of higher physical realism combined with its higher cost argue against recommending its use in most LMIC settings.
In addition to cost-effectiveness, the impact of resources required and equity are counterintuitive to higher physical realism training. In any setting it is critical to consider the feasibility, sustainability, and scalability of implementation, which in LMIC settings favor lower physical/low-cost realism in the implementation of simulation-based programs. If low-cost higher physical realism training equipment were to become available, then this would impact the principle of feasibility, but further directed research is needed to identify such a solution.
Our review confirmed that simulation-based medical education is an effective methodology in LMICs compared with no simulation. We found that simulation is effective in acquisition and retention of suturing, CPR, and other surgical/procedural skills. Unfortunately, implementation is often prohibited by the human resources and financial cost required.
There was limited evidence in this review for desirable effects in participant satisfaction20 and confidence.17–19,21 The remaining studies presented mixed results for these outcomes.24,25 Four studies demonstrated no difference in skills acquisition between higher versus lower physical realism models,17–19,23 and 2 studies found that hybrid physical realism or higher physical realism models were better than the lower physical realism models.16,21 Two studies examined learner performance with simulated laparoscopic skills and quality of CPR skills, which showed no difference between higher and lower physical realism models.15,22
Given the gaps in opportunities present in LMIC settings, it is important to consider the value of training equity over physical realism. The associated resources and costs may prohibit high physical realism simulation training in some LMIC settings.26,27 If implementation of high physical realism simulation was to be prioritized in these settings, it would likely come at the expense of other important health care educational interventions in these low-resource settings. This should be balanced against the benefits of improving patient outcomes with potentially little cost and lower resources if low physical realism models are used. It is also notable that significant inequity already exists within health care and medical education in LMIC settings due to constraints in resource allocation and variable support from local government and nongovernmental entities and ministries of health.28 Although physical realism does play a role in learner and stakeholder acceptability, the high comparative cost may not be acceptable to stakeholders. Health ministries and nongovernmental organizations must inevitably consider factors of scalability and logistics of implementation of simulation in addition to cost. Most levels of health care trainees, from the viewpoint of learners as stakeholders, perceive acceptability of any form of simulation to be an acceptable benefit.29 Due to high cost, feasibility of program implementation and sustainability for high physical realism simulation will inevitably be negatively impacted. This is frequently amplified by the lack of trained individuals for repair and maintenance of equipment in LMIC settings, creating issues with dissemination and sustainability of high-technology equipment.
Limitations and Future ResearchOur review was limited by the number of studies comparing lower physical realism with higher physical realism in LMICs. In addition, most studies were conducted in upper middle-income settings, highlighting the need for more scrutiny of low- and lower middle-income settings. The paucity of evidence precluded any meaningful subgroup comparisons between LMIC settings.
All studies examined had significant issues with multiple confounding factors, methodological validity, and significant heterogeneity of learner populations, ranging from medical students to midwifery students to practicing surgeons. The definitions of higher versus lower physical realism were heterogeneous. Generalizability was low due to metrics used to assess main study outcomes. Most of these studies did not directly compare high versus low physical realism mannequins, and those that did failed to show significant differences between the 2 groups. Studies with outcomes of skill acquisition and performance in clinical practice were downgraded for risk of bias and inconsistency, resulting in very low certainty of evidence.
The overall balance of effects does not favor higher versus lower physical realism simulation in LMICs, but it is important to consider that there are major gaps in simulation research in LMICs. None of the studies examined patient outcomes, making the pragmatic value of study outcomes more limited in clinical practice applications. None of the studies discussed previously examined cost efficacy, resulting in our inference that equivocal efficacy combined with large financial and human resource costs undermines current implementation strategies for high physical realism simulation training in LMICs.
CONCLUSIONOur systematic review suggests that lower as opposed to higher physical realism simulators/task trainers should be used for health care simulation training of individuals in LMICs. Further research is needed to identify a solution that considers physical realism balanced with cost, equity, impact of resources, sustainability, and scalability. Future studies in LMICs should focus on appropriate study populations and interventions and be adequately powered to address relevant study outcomes. In addition, it will be important for future studies to use consistent and standardized definitions of “high” and “low” physical realism and to conduct robust comparative studies between the 2 modalities, given the paucity of high-quality literature currently available to address this research question.
ACKNOWLEDGMENTThe authors would like to thank Danielle Gerberi (information specialist) for her help with this review. We would also like to thank the leadership team from the Society for Simulation in Healthcare (Dimitrios Stefanidis, Sharon Decker, Sharon Muret-Wagstaff, David Cook, Mohammad Kalantar, Mohammed Ansari, Kathryn Adams, and Kristyn Gadlage) for their support and guidance in this process.
REFERENCES 1. Cheng A, Lang TR, Starr SR, Pusic M, Cook DA. Technology-enhanced simulation and pediatric education: a meta-analysis. Pediatrics 2014;133(5):e1313–e1323. 2. Aggarwal R, Mytton OT, Derbrew M, et al. Training and simulation for patient safety. BMJ Qual Saf 2010;19(Suppl 2):i34–i43. 3. Lioce L, Loprieato J, Downing D, et al; the Terminolog
Comments (0)