INTRODUCTION

Hospital visits are a crucial component of the early clinical and professional development course at the medical schools in the University of Nottingham, UK. In this programme medical students achieve competencies in basic clinical skills and ethics by hands-on experience of performing clinical examinations on patients. Early clinical and professional development course was established in the medical school with the aim to address broader issues of professional awareness at an early stage as advocated in the GMC publication Tomorrow’s Doctors (2009) by integrating clinical practice with teaching of basic sciences. Similar programme for fresher medical students was introduced in 1971 in the department of Family Practice at Howard University College of Medicine to expose medical students to patient care early in their careers. Almost 89% students had rated this programme as good to excellent.1

The structure of the hospital visits comprises an introductory lecture on history taking and clinical examination of the relevant system, briefing and demonstration of the examination by the teaching fellows (TF) in smaller groups. This was followed by hands-on patient examination by students under supervision. Medical students are expected to attend practice sessions at the skills centre before the visit.

Hospital visit programme is not only beneficial to the medical students but also helps the participating junior doctors and teaching fellows to develop presentation and communication skills through teaching.2 Clinical experience is an important part of the integrated medical curriculum. It provides a foundation for experiential learning in the early medical years by which the students understand the relevance of the basic science knowledge and clinical practice in a real hospital setting.3,4 During the debriefing, medical students can put forward questions and hence identify gaps in knowledge so as to address and improve one’s learning by consolidating the understanding of clinical topics.5

A systematic review by Dornan et al. included 73 studies done between 1992 and 2001 evaluating the influence of early clinical experience in hospital and community settings in developing professional attitudes, empathy, confidence and motivation in medical students. The American studies in this review had noted that early clinical experience in preclinical years led to increased recruitment to primary care and rural medical practice.6 However, the gap in the research is the need to evaluate the outcomes of the early clinical experience programme to understand their effectiveness in shaping the clinical careers of medical students and plan the medical curriculum. Hakim et al. in 2014 emphasized the role of integrated model of medical curriculum by incorporating clinical experience earlier in medical education to reinforce cognitive, psychomotor and affective domains of learning and promote professional and social development.7 Johnson et al. noted positive relationship of early clinical exposure and students’ satisfaction with medical education by significance tests.8 Clinical elective licence (CEL), a practical course for early medical students in Austria was highly rated by 1st and 2nd-year students in their feedback based on evaluation in 2011. The students had reported increased confidence with patient interaction and communication.9 The Medical Council of India ‘Vision 2015’ aims to produce medical graduates at par with global standards by reforming the preclinical curriculum with early clinical exposure. Govindarajan in a study on learning objectives of 150 preclinical students posted to various specialist clinics and intervention laboratories noted significant increase in post-test scores in all the specialist areas where early medical teaching was associated with clinical experience. This was based on analysis of open feedback from students, which showed significant impact on their cognitive, psychomotor and affective domains. This study model has introduced new medical students to professional, interpersonal, scientific and ethical aspects of medicine earlier in their careers.10 Study on preceptorship in internal medicine at preclinical level has shown to improve the students’ performance during internal medicine clerkship.11 Further studies on clinical experience achieved by introduction of various programmes to promote clinical skills in preclinical year students have reported improved performance, professionalism and satisfaction with the curriculum.12,13 Although all these studies have shown improved student experience and satisfaction with learning clinical skills, it is crucial to conduct course evaluation to understand the effectiveness of the programme. Goldfarb and Morrison in 2014 showed the effectiveness of collecting students’ feedback in real time, that is while the course is ongoing rather than wait till the end of the course to enable course modification effectively. This maximizes student involvement in course development, improvements in curriculum and identifying weaknesses in the programme and in students’ experience.14 A study on students’ course evaluation has mentioned that courses which are well organized, clearly communicated with helpful teaching staff and delivered goals and objectives of curriculum were generally rated highly by medical students. This study has further described the limitations of students’ evaluation mainly that the quantitative ratings are subjective; students may not appreciate value of the curriculum content which ultimately should be finalized by the medical educators and not just based on students’ ratings. Hence, the end of the course evaluation questionnaires should include effectiveness of faculty teaching, educational material, course organization and achievement of the course objective.15 Benton and Cashin, 2014, have recommended that students’ evaluation of teaching (SET) is effective for assessing the teaching programme or faculty. SET is a multidimensional tool which includes several factors such as course organization, planning, communication skills, teacher– student interactions, course difficulty and grading systems. These factors are scored, so far a single global score has not been identified to validate teaching effectiveness.16

With the literature review, we have identified the gap in the research related to the effectiveness of the end of the course students’ feedbacks for evaluation of the preclinical skills programme. We hope to bridge this gap by analysing the medical students’ end-of-course feedback suggestions for hospital visit programme and implementing their suggestions to modify the course based on their perceptions and further measuring the effectiveness of the modifications with feedback from the students’ experiencing the changes to the training programme.

Our study was to evaluate the preclinical (1st and 2nd year) medical students’ experience of hospital-based clinical teaching programme between 2011 and 2013 on their feedback forms based on the students’ feedback. The students’ suggestions were implemented to bring about suitable modifications to the hospital visit programme. The modified programme was similarly evaluated with students’ feedback. The objective of the research was to gauge the effectiveness of the SET by comparing their pre- and post-modification ratings of the programme.

METHODS

This quasi-experimental pre- and post-test intervention design study involved comparisons of students’ feedback obtained before and after suitable modifications in the hospital visit programme with 337 feedback forms available in each group. The pre-test group feedbacks were collected from February 2011 to December 2011 and the post-test group feedbacks from February 2012 to May 2013 (Table I). This work was carried out in accordance with the Declaration of Helsinki with no potential harm to participants, and the anonymity of participants was maintained. As this study was not a part of a clinical trial and did not involve patients or research participants, ethics approval was not sought.

TABLE I. Pre-test and post-test scores

Pre-test score 1, 2, 3, 4, 5 Questions Post-test score 1490 Introductory talk was useful 1551 88 7 Number of patients allocated 1074 1521 Rate quality of your visit 1581 Yes 327, No 10 Were you supervised? Yes 366, No 1 Yes 323, No 14 Opportunities to see more than one patient Yes 337, No 0 Yes 323, No 2 Was there a debrief at the end? Yes 335, No 2

The pre-test group had experienced hospital visits based on an old format, and the post-test group had experienced modifications in the programme based on the feedback suggestions of the previous group. In the pre-test hospital visit programme, all the year group students had an introductory lecture, following which groups of 7 students were formed and each group was supervised by one TF. Based on the feedback, it was noted that the number of patients seen by each student was unpredictable, ranging from 1 to 4; also, the amount of time spent on the ward was left to the discretion of the TF. This resulted in dissatisfaction among the students, which was reflected in their feedback. A major issue identified was of a high student to TF ratio which made it difficult to provide one-to-one supervision for students in the group. To improve on the old format, modifications in the hospital visit programme were introduced in the first quarter of 2012. The important changes incorporated were 2 TFs were allocated to a group of 7–8 students; hospital visits time was stipulated from 9 a.m. to 12 noon and not finishing before 12 noon; each student had to see no less than 3 patients; debriefing was conducted at the end of hospital visits by the TFs; TFs were mandated to collect student feedback at the end of the hospital visit.

The feedback form used is shown in Table II. The questionnaire format of the feedback form is similar to a tool for the evaluation of clinical learning environment questionnaire. Five main factors considered by this tool are cases, authenticity of the clinical learning experience, supervision, organization of the doctor–patient encounter and motivation to learn.17

TABLE II. Hospital visit questionnaire

Date: When ranking, please note: 1=Poor, 5=Excellent The introductory talk was useful 1, 2, 3, 4, 5 State the name of the doctor undertaking the ward element of the visit. Number of patients you saw, this includes those patients that you examined yourself, that a fellow student examined, that you just observed and also that the teacher demonstrated on? 1, 2, 3, 4, 5 Were you supervised when on the wards and at some point, undertaking your examination? Answer: Yes or no Rate the overall quality of the ward-based part of your visit 1, 2, 3, 4, 5 Free text comments What did you enjoy about the hospital visit? What could be improved?

In the free text section, a total of 898 comments were noted for both the groups. These comments were on two questions asked in the feedback, first ‘What did you enjoy about the hospital visit?’ and ‘What could be improved?’ These comments or codes were grouped into pre-test and post-test with four main themes of introductory talk, hospital visit organization, patient experience, and briefing and debriefing. Salient comments are shown in Tables IIIa and IIIb.

Quantitative response questions in the feedback were statistically analysed using Welch’s t-test, and free text questions were qualitatively analysed and grouped into themes.

RESULTS

The feedback forms of the preclinical 1st and 2nd-year medical students for the hospital visit programme were studied retrospectively before and after the changes were introduced in the programme.

Quantitative analysis

As shown in Table I, the post-test group had more positive response for all six questions compared to the pre-test score. Welch’s t-test comparing pre- and post-test scores showed significantly positive difference between the groups (p<0.001).

Thematic analysis of free text

For the free text question regarding ‘What did you enjoy about the hospital visit?’ the appreciative comments were on getting hands-on experience with real patients and briefing and debriefing by the TFs (Table IIIa). Most of the students had responded to the question ‘What could be improved?’ with the suggestions of having more patients for examination and to be able to spend more time on the ward (Table IIIb). The negative feedback comments from the hospital visit programme 2011–12 were identified and necessary modifications implemented in the year 2012–13. These comments were usually around the issues of high students to teacher (TF) ratio of approximately 7:1; inadequate and variable time duration devoted to the hospital visit for different groups of students and the number of patients seen by each student. These were flagged as the main concerns and reasons for student dissatisfaction with the programme. Although the feedback forms were administered over a period of 2 years to different groups of students, it was interesting how these issues were consistently flagged by the students.

TABLE IIIa. Free text comments regarding what went well for both pre-test and post-test hospital visit groups Themes identified

Themes identified What went well (pre-test February 2011– December 2011) What went well (post-test February 2012–May 2013) Introductory talk – Good introductory lecture was really helpful to – Really good introduction by the teacher who give us a review on abdominal examination demonstrated one-to-one on patient and then before actually performing it supervised on two other patients by a senior medical student – Structured lecture at the start left me feeling completely capable in history taking, it leaves you feeling confident Patient experience – Got to practise examination skills and gain confidence – Examining patient as a group with registrar gave equal learning opportunity – – – – – Chance to see more than one patient The summary session at the end was particularly useful to ask questions Very interactive and well-planned visit Talking to patients Learning how to improve and create structure to the consultation – – – Long time spent with each patient going through their cases and do all aspects of investigations Hands-on experience with real patients was enjoyable as we were supervised by the junior doctors who were helpful Plenty of time to examine patients; all of the patients were happy to help and none of them were – Being allowed to interview a patient unsupervised Taking histories and getting feedbacks Interacting with patients and senior members of staff – – being moved around like in A and E Presenting complaints Very interesting cases, learnt how to take structured + concise history and communicate better with – Meeting a man who had communication difficulties – patients Gave us more practice with patients’ history taking and understand the different levels of communication first Briefing and debriefing – Good guidance – Pre-practical teaching of history taking not just – Excellent method of teaching describing so feel more confident when taking – Having sufficient time for feedback and having a history consultant that was genuinely interested in our – Doctors made the visit fun development – Enjoyed briefing and debriefing – Really good feedback/summarizing after seeing – Learning from constructive feedback given by the the patients doctor – Good to practice taking histories on 2 patients – The feedback at the end of the session was very and then feedback to the group helpful. The patients were friendly – Well structured, patients ready in advance, more – Learning about mistakes I made in history taking than one patient. Got feedback on each history – Quality of teaching plus feedback on the wards presented – Opportunity to practice skills – Constructive feedback highly appreciable – Got to work as a team with another medical student, gained feedback from each other Hospital visit organization – Actually bridging the gap between theory and – Good feedback session, proper sit down and practical by performing clinical examination discussion – Feedback given inbetween the patients for – Lots of help and feedback from facilitators 5 minutes each – One-to-one help from 4th year was really useful – New information+learn how to do examination – Well organized informative session, good variety of and also practise with real patients+teacher had patients seen a lot of time for us; did not feel rushed – 4th-year students were very helpful

TABLE IIIb. Free text comments on improvements needed for both pre-test and post-test hospital visit groups Themes identified

Themes identified Improvements needed (pre-test February 2011– December 2011) Improvements needed (post-test February 2012– May 2013) Introductory talk – Presentation was not relevant to the practical – More explanations on importance of each – Introductory talk could have more practical examination information about how you take examination – Level was slightly higher than required for 1st year rather than a lot of theory – Could have revised the examination procedure – Talk at the beginning could be more relevant to – Could have been shorter our OSCE practical – The talk at the beginning explained the – Computer did not work, which made the examination but it would be nice to know the theory introductory talk less effective also. – Could contain overview/example of how to do the examination. – Could include more relevance to the practical aspect of our visit Patient experience – More patients with different pathologies – Potentially examine more patients with respiratory – We can get to practise on more than one patient conditions – Maybe we could see more patients – More time so we could see more patients – It would have been good to have more than one – Opportunity to see normal and abnormal patient and a mixture of men and women – More patients for examination – Patients should be selected beforehand – Timing: we overran a bit – Practice on more patients – Lecture seemed unnecessary and only saw 3 patients – More patients to examine or interview of – Number of patients available for interview, rotating different ages etc. between patients would have been useful – One or two more patients to speak to – I guess just more time to see patients and practice Briefing and debriefing – Others have someone observe while examining – More explanations/debrief or quickly run through the examination – Have all the notes available so you can compare practically rather than a lecture what you find with what the patient actually has – Smaller groups per supervisor – Watch the doctor take a history first – Checklist of how to do the examination should – Perhaps getting to personally interview a second be provided. Sheet explaining tasks patient after the feedback – Run through the actual examination process as a – Perhaps give the students a brief of structure before recap before we had to do it and more chance to talk to the patients. More – Seeing an examination start to finish without constructive criticism on our examinations. Perhaps stop a second individual attempt at an examination to improve on the effort Hospital visit organization – Being given equipment such as a torch to carry – Some clearly did not want to be examined out cranial nerve examinations – More independent examination – They tried to show us 3 patients, unfortunately, – Organization leading up to the visit they were not feeling well but nothing could have – Information about where to go only came up on been done and the doctors tried really hard to get NLE a few minutes before we were supposed to meet to see more – It would be nice for more information to be put on NLE so I could have prepared better – Felt slightly imposing with 6 students around 1 patient Introductory talk

In the pre-test group, there were a few positive comments on usefulness of the introductory talk for patients’ history taking. However, most comments were on improvements by including details of clinical skills and technique relevant to the system rather than theoretical knowledge only. In the post-test group, the students had appreciated the changes to the introductory talk regarding detailed teaching of clinical examination and demonstration of examination technique on a volunteer, which they found useful. The improvements requested were shortening the length of talk, and also including theoretical knowledge in a concise manner.

Patient experience

The pre-test group acknowledged that they got to examine at least one patient on the ward and were able to interact with patients and elicit history. Most students expressed the desire to see more than one patient to improve their examination techniques, ‘Number of patients available for interview, rotating between patients would have been useful’. The students in the post-test group had largely appreciated the opportunity to examine more than one patient during their hospital visits and had felt that more time should have been allocated to allow systematic examination of the allocated patients, for example ‘I guess just more time to see and practice on patients’. However, due to technical issues, the hospital visit hours on the ward could not be extended beyond 12 noon as it affected the ward patients’ meal time.

Brief and debrief

The pre-test group had commented positively on the briefing and debriefing at the end of the visit; however, they had noted that these could be standardized by providing a checklist of examination techniques before seeing patients and also proposed getting one-to-one feedback on the patient examination performed. The students in the post-test group felt that briefing and the feedback experience was improved by having 2 TFs per group. Students particularly appreciated receiving feedback after every examination as it helped in improving the technique on the next patient, ‘I really liked the debriefing session after each session, we took and presented the histories, it gave us the opportunity to improve before seeing our next patient’.

Hospital visit organization

In general, the pre-test group appreciated the hospital visit programme for the opportunity to gain hands-on experience with clinical history and undertaking patient examinations. However, they had commented on the areas requiring improvement such as smaller student groups, identifying appropriate patients for examination, avoiding those patients who had surgeries and improving time management on the ward. Pre-selection of patients would save time and allow students to focus on their examinations as soon as they got to the ward. The post-test group students had remarked on the better patient selection for examinations, good briefing and debriefing and active involvement of the TFs. Students felt that the introductory talk and the clinical examinations for the systems should be uploaded to their online education network ‘NLE’ (Nation of learning excellence) in advance as it would facilitate their understanding and improve their confidence in performing patient examination in a short period of time.

DISCUSSION

The changes brought about in our preclinical hospital visit programme were based on the recommendations of pre-test students’ feedback, mainly improving students to TF’s ratio, committing at least 3 patients to each student for bedside examination and hospital visit session lasting not less than 3 hours. Evaluation of the post-test group feedback forms has shown positive response for the modified programme. This was mainly brought about by motivation of the students in both the groups to actively provide feedback on the teaching programme.

The student’s giving feedback identified and rectified the factors affecting the current programme to overall improve the management of the current programme, course content and grades. As the feedbacks were anonymised, the students were more open about their views. The advantages of teaching feedbacks were to give teachers the opportunity to improve the learning and teaching experience of the students.18 We noted that to improve the hospital visit programme and the post-test feedbacks of the students, the TFs had brought about changes to the programme based on pre-test students feedback such as modifying the introductory lecture to include demonstration of practical skills, coordinating with other TFs to improvise oneto-one observation of students performing clinical skills, ensuring enough number of patients are made available to students for examination; these modifications in the teaching programme had positively influenced the feedbacks from the post-test student groups. Hence, our study shows that SET programme has motivational influence on the teaching staff thus improving the teaching programme. These findings illustrated that the students were keen and eager to learn and gain the maximum from their clinical visits. The feedback was taken seriously, the issues addressed and improvements implemented in early 2012. Analyses of the post-test feedbacks revealed positive responses and satisfaction with the programme. Our study shows the validity and reliability of student feedback for improvement of various programmes in higher education.2 The students who provided the initial feedback were not the same who experienced the changed programme and provided the subsequent feedback. This illustrates that it was not just a question of one group of students ‘getting what they asked for’ but a genuine improvement in the programme as experienced by the second set of students who had no knowledge of the concerns of the first group of students and the changes implemented as a consequence. Wong and Moni had observed that SET is an effective tool for improving the education programme only when the teaching and training staff take into account the feedback and implement appropriate actions. Salient features of this study are institutional requirements, operational practices, personal biases and provision of support.19 There is limited evidence about how the teachers in higher and medical education in particular, systematically apply student feedback to improve the quality of their teaching practice.20 In our study, the concerted effort of the programme organizers to introduce and implement changes as suggested by the students’ feedback had resulted in major improvement in the post-test groups feedback. Our findings are supported by observations from studies by Scot et al. and Schiekirka and Raupach that the concept of pre-test and post-test feedback design is relatively simple and effective for constant improvements of concise medical education modules.21,22

This is one of the first studies involving clinical experience of preclinical medical students to demonstrate effective utilization of students’ feedback to evaluate and accordingly modify the programme leading to increased students’ satisfaction. It was performed retrospectively due to which certain biases experienced by the participating students could not be eliminated or controlled by the researchers. It was also noted that due to organizational limitations of resources, not all students’ requirements were fulfilled in spite of the popular demand such as each student getting to examine more than 3 patients independently in every hospital visit or allowing students more than 3 hours on the ward for each visit as it compromised patients’ eating and resting times (Table IIIb).

The evaluation of medical education based on students rating assesses three of the four main features, structures, processes and teacher characteristics. Schiekirka and Raupach have noted that evaluation activities are not mandatory at all the universities; hence, selected students providing course ratings might create a bias.22 However, at our medical school, the preclinical students attending the hospital visit programme had to compulsorily submit the feedback to evaluate the programme. Evaluation studies have also valued evaluating learning outcomes. However, difficulty in generating valid and reliable tools to assess this feature has been noted. In our study, learning outcomes were not evaluated as there was no summative assessment at the end of the programme.16 SET was effectively used to evaluate the hospital visit programme in preclinical years by assessing important factors of students’ satisfaction such as teaching performance, introductory talk, demonstration of the systems examination, briefing, debriefing, available facilities, number of patients seen, and time spent on the ward.

Future work

Evidence-based medicine (EBM) is more important than ever with research in all areas of medicine leading to changes in investigations and management of health conditions eventually replacing the conventional treatments. As EBM aims to provide most effective care to improve patients’ outcomes, it becomes crucial that EBM is introduced earlier at preclinical level of medical education if it was to be incorporated in routine medical practice of new generation of doctors.23

Conclusion

The benefits accrued from a hospital visit programme for the preclinical medical students are determined by the available infrastructure, consistence of allocated time, adequate patient contact and the ratio of teacher to the taught. Potential benefits to the students and staff participating in the hospital visits are the development of professional attitudes and encouraging independent learning. Appropriate follow-up action on students’ feedback such as effective training for student tutors and introduction of innovative education methods is important to make the programme robust and maximize the learning outcome, students’ experience and satisfaction with the programme.