The aim of this study was to investigate the effect of training on the arthroscopic performance of a group of medical students and to determine whether all students could be trained to competence. Thirty-three medical students with no previous experience of arthroscopy were randomised to a ‘Trained’ or an ‘Untrained’ cohort. They were required to carry out 30 episodes of two simulated arthroscopic tasks (one shoulder and one knee). The primary outcome variable was task success at each episode. Individuals achieved competence when their learning curve stabilised. The secondary outcome was technical dexterity, assessed objectively using a validated motion analysis system. Six subjects in the ‘Untrained’ cohort failed to achieve competence in the shoulder task, compared with one in the ‘Trained’ cohort. During the knee task, two subjects in each cohort failed to achieve competence. Based on the objective motion analysis parameters, the ‘Trained’ cohort performed better on the shoulder task (p < 0.05) but there was no significant difference for the knee task (p > 0.05).
Although specific training improved the arthroscopic performance of novices, there were individuals who could not achieve competence despite focused training.These findings may have an impact on the selection process for trainees and influence individual career choices.
The teaching and assessment of surgical skills has been a key focus for surgical educators. The restructuring of modern surgical education from the traditional time-based apprenticeship1 to a competence-driven system, together with restrictions on surgical trainees’ working hours, have resulted in a reduction in the average number of operative cases per trainee.2,3 Furthermore, pressure from service-driven health-care systems has resulted in a dramatic reduction in the time available to train a surgeon: approximately 80% in the United Kingdom.4 In addition, high-profile negligence cases have placed a demand on the surgical community to demonstrate that surgeons have the necessary skills.5
Recent changes in postgraduate medical training in the United Kingdom have encouraged early career choices.6 However, current Foundation programmes focus on the development of generic skills, and, more importantly, selection into core surgical training programmes requires no previous demonstration of technical ability. Therefore, surgical trainers have a difficult role in trying to select the most suitable candidates for entry into higher surgical training. With regard to trauma and orthopaedic surgery, a wide range of factors are associated with successful progress.7-9 Although there is no formal consensus on how to identify the most suitable candidates, it is assumed and expected that those selected must have an aptitude for the development of technical skills.
There is a growing body of evidence calling for more formal integration of simulation into surgical training and assessment.10-12 Simulation lends itself well to the acquisition of arthroscopic skills as it enables the reproduction of the narrow field of vision, loss of tactile perception and the two-dimensional representation of a three-dimensional surgical environment. Furthermore, some authors have confirmed the use of simulation in orthopaedic training, having shown the transfer validity of arthroscopic training on a simple knee simulator.13
Previous studies using laparoscopic simulators suggest that there are individuals who are unable to achieve proficiency at laparoscopic tasks despite practice.14 However, this important area has not been well investigated in orthopaedic surgery. We have recently shown that there is significant variation in the innate arthroscopic ability of medical students.15 This study highlighted that although these ‘novice’ arthroscopists improved their basic skills with repetition, some were unable to achieve proficiency in basic tasks.
Given the growing importance of selecting appropriate candidates for orthopaedic higher surgical training programmes, the aim of this study was to determine the effect of focused arthroscopic training on the performance of such novice arthroscopists and to establish whether all novices can achieve task competence with training and repetition compared to repetition alone.
Subjects and setting
Thirty-three penultimate-year medical students were invited to participate in this study during their orthopaedic rotation. This rotation was a compulsory attachment undertaken by all undergraduate students studying medicine at our institution. Participation in the study was entirely voluntary. Despite this, all the students who were attached to the department during this particular semester agreed to participate. Therefore, there was no selection bias towards students interested in orthopaedic surgery. None had previously performed or assisted in any arthroscopic procedures. Using sealed opaque envelopes they were randomised to a ‘Trained’ cohort who received specific training (n = 16) on two basic arthroscopic tasks, and a control ‘Untrained’ cohort who received no training (n = 17) (Fig. 1). The study took place over a four-month period in the surgical skills laboratory of a university teaching hospital. All subjects gave informed consent to participate and institutional approval was obtained for the study.
Two standardised simulator-based arthroscopic tasks with construct validity15 were set up. Participants watched an instructional video explaining the tasks and were allowed five minutes to become familiar with the equipment. Each participant performed 30 episodes of a task (with a one-week interval after each ten episodes).
Shoulder arthroscopy task
This was a narrow-field triangulation task performed on a Hillway Shoulder benchtop simulator (Model HSSH-01; Hillway Surgical Limited, Selsey, United Kingdom) using a standard 30° arthroscope, an arthroscopic camera and a display system (Smith & Nephew Endoscopy, Huntingdon, United Kingdom). It involved passing a needle via the anterior aspect of the shoulder (through the rotator interval) to enter the joint through a pre-drawn 10 mm diameter circle within the shoulder capsule. This circle was clearly visible via the arthroscopic view. While performing this task, the subjects were able to view the shoulder model both externally and arthroscopically. The aim of the task was to represent the skill involved during anterior portal placement in real shoulder arthroscopy. Placement of the needle in the predefined circle within ten attempts was considered ‘success’ at the task.
Knee arthroscopy task
This simple ‘locate and retrieve’ task was designed to represent retrieval of a loose body (represented by a 3 mm diameter metal ball bearing) using graspers (Alligator Grasper; Smith & Nephew Endoscopy). The ball bearing was placed in a repeatable position on the tibial plateau, leaning against the posterior horn of the medial meniscus. Due to the posterior slope of the tibia there was only a single position the ball bearing could rest without sliding to the back of the knee and this was used as the standardised starting point. The task was performed on a knee arthroscopy simulator (Model 1413; Sawbones, Malmö, Sweden) using a 30° arthroscope, camera and display system (Smith & Nephew Endoscopy). The participants were deemed ‘successful’ at the task if they were able to retrieve the ball bearing from the knee joint within 3 minutes.15
In addition to the standard instructions, the participants in the Trained group received a 30-minute one-to-one tutorial relevant to the two tasks. The training was delivered by a senior clinical fellow (SA) with extensive experience in both undergraduate and postgraduate orthopaedic teaching. This focused on the relevant anatomy, the fundamentals of arthroscopic triangulation, familiarisation with the 30° arthroscope, a detailed breakdown of the tasks, and tips on avoiding common pitfalls.
Performance evaluation based on stabilisation of surgical learning curves
The participants were judged to have stabilised their learning curves adequately, thereby becoming ‘competent’ at the tasks, if they did not fail the tasks beyond the 20th episode (Table I). This cut-off point was selected based on previously published work.15
Evaluation of surgical dexterity
Arthroscopic dexterity was assessed objectively using a motion analysis system (Patriot; Polhemus, Colchester, Vermont) and post-processed by custom software (MATLAB v6.5; The MathWorks, Natick, Massachusetts). This three-dimensional tracking system has been extensively validated and is a reliable indicator of dexterity.16 The system provides three outputs: total ‘time taken’ to complete the task, total ‘path length travelled’ by a subject’s hands and the total ‘number of hand movements’ during the task. The motion analysis parameters of individuals with greater dexterity are demonstrated by more efficient hand movements over shorter distances and, as a consequence, in a shorter time.17
All data were analysed by the statistical software package SPSS 18.0 (SPSS Inc., Chicago, Illinois) using non-parametric tests. The performances of the Untrained and Trained cohorts were compared using the Mann-Whitney U test for ‘motion analysis parameters’ and the chi-squared test for ‘ability to achieve competence’. A p-value < 0.05 was considered statistically significant.
The two cohorts were evenly matched in terms of demographics (Table II).
Performance evaluation based on achieving task competenceShoulder arthroscopy task
Seven subjects were unable to achieve competence, one in the Trained cohort and six in the Untrained cohort (Table III). The performance of the Trained cohort during this task was significantly better (p = 0.041).
Knee arthroscopy task
Four subjects were unable to achieve competence, including two in the Trained cohort and two in the Untrained cohort (Table III) (p = 0.948).
Performance evaluation based on motion analysis parametersShoulder arthroscopy task
Motion analysis results showed that the performance of the Trained cohort was significantly better than that of the Untrained cohort for all three parameters (Fig. 2). The median time taken in the Trained cohort was 549.7 seconds (interquartile range (IQR) 479.6 to 613.9) compared with 881.2 seconds (IQR 770.7 to 1281) in the Untrained cohort (p < 0.0001). The median path length in the Trained cohort was 2712 mm (IQR 2477 to 3117) compared with 3530 mm (IQR 3249 to 4818) in the Untrained cohort (p < 0.0001). The median number of hand movements in the Trained cohort was 207 (IQR 168 to 229) compared with 341 (IQR 241 to 436) in the Untrained cohort (p < 0.001).
Knee arthroscopy task
Motion analysis results during this task revealed that although there was a trend towards an improved performance by the Trained cohort, there was no significant difference in performance between cohorts (Fig. 3). The median time taken in the Trained cohort was 1257 seconds (IQR 1104 to 1385) compared with 1505 seconds (IQR 1190 to 1819) in the Untrained cohort (p = 0.081). The median path length in the Trained cohort was 14 014 mm (IQR 12 126 to 16 214) compared with 14 585 mm (IQR 13 009 to 16 863) in the Untrained cohort (p = 0.256). The median number of hand movements in the Trained cohort was 946 (IQR 885 to 1092) compared with 987 (IQR 853 to 1138) in the Untrained cohort (p = 0.958).
This study demonstrates that, despite training, some medical students are unable to achieve competence in basic simulated arthroscopic tasks. The findings suggest the existence of three groups based on arthroscopic ability: those with innate arthroscopic ability, i.e. those in the Untrained cohort who ‘self-learned’ and achieved competence with simple repetition; an intermediate group with moderate ability, i.e. those in the Trained cohort who were able to perform as well as the naturally skilled individuals following appropriate training (making them ‘trainable’ at arthroscopy); and those who appeared unable to perform basic arthroscopic tasks proficiently despite training and repetition.
The effect of technical skills training in non-orthopaedic surgical specialties has previously been reported, demonstrating similar findings to ours. Eversbusch and Grantcharov18 studied the performance of novices on a gastrointestinal virtual reality (VR) simulator and showed superior performance by the ‘trained’ cohort. Further studies using laparoscopic VR simulation identified subgroups of trainees who could not reach a predefined level of proficiency despite repeated practice.14,19 Although it is reassuring to see that training did have a positive impact on the arthroscopic ability of most medical students (during the shoulder task), the inability of a small subgroup who could not reach basic proficiency raises concerns. This particular study did not aim to identify why some medical students could not achieve competence at the arthroscopic tasks. However, previous attempts have been made to identify predictors of surgical performance and question why certain individuals possess superior innate surgical ability.20-22 Visuospatial ability and fine motor dexterity have been highlighted as important predictors of performance in novices,23-25 but the relative importance of these factors appears to diminish with experience and practice, thus becoming less influential on the ability of more senior surgeons.26
We acknowledge that the study has limitations. First, possible underpowering could explain why training did not appear to improve performance during the knee task. However, a study population of 33 is larger than in many surgical skills studies, and logistical problems limited the number of medical students rotating through our institution. The knee task was a ‘large field of view’ task compared to the shoulder task and therefore easier, as indicated by the large proportion of participants who achieved competence even without training. This unexpected simplicity may have rendered the training for the knee task less effective because of a ceiling effect. However, both arthroscopic tasks possessed construct validity and had appeared sufficiently complex for novices. Selection of more complex tasks in future studies could avoid any ceiling effect. Another possible limitation is that we only used one method of skills training. Previous studies have demonstrated that techniques such as video feedback27 or the practice schedule of a task,28 e.g. blocked versus random practice, can influence the effectiveness of skills acquisition. Future studies should aim to further assess the effect of differing training modalities in novices.
A small number of studies in orthopaedics13 and other surgical specialties29,30 have demonstrated the transfer validity of performance on simulators to the operating room. This study highlights novices that might not be trainable in the ‘arthroscopically’ focused subspecialties. The importance of these findings will only be known when a longitudinal study follows such individuals through a training programme with annual ‘follow-up’ to identify whether the amount and type of training allowed them to reach competence at these tasks. Only then could such tasks be used as selection criteria for specialty training in orthopaedics.
There have been calls for more objective methods of selecting surgeons,9,31 including a technical component. The Royal College of Surgeons in Ireland now includes testing of technical skills and fundamental abilities (psychomotor skills, visuospatial ability and depth perception) in the selection of higher surgical trainees from shortlisted candidates.32 In agreement with our study, they found that many of the shortlisted candidates struggled with basic surgical tasks.
Our findings also raise the question of the timing of selection for orthopaedic training. The shift from ‘run-through’ to ‘uncoupling’ of surgical training, as advised by the Tooke report,33 enables a two-year period of generic or themed surgical training (CT1-2) followed by competitive entry into ST3 (higher surgical training). One advantage of this approach is to allow ‘early-years’ trainees to demonstrate their ability to acquire and develop technical skills during a period of training. A recent survey of orthopaedic trainers in the United Kingdom indicated, however, that only 3% favoured run-through training that would commence directly after the Foundation Years.34 However, it is also recognised that a large proportion of eventual surgical trainees make their career choice by the end of the first postgraduate year.6 Therefore, an alternative is to change the generic nature of Foundation Training so that those with an early interest in a surgical career can select appropriate rotations, and show an aptitude for both the technical and non-technical skills required in craft specialties.
In conclusion, this study highlights that, despite training, some novices cannot master basic arthroscopic skills on simulators. It is a sensible national aim to determine the surgical ability of individuals in a robust and objective manner at an early stage in surgical training, so that those who lack the required attributes can be given focused training or advised to pursue careers in alternative specialties. However, more research is still needed in this area before robust evidence is available to make these important decisions about a trainee’s career.
This study was supported by our NIHR Biomedical Research Unit and NHS Education South Central.
No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.
- Received July 7, 2011.
- Accepted August 17, 2011.
- ©2011 British Editorial Society of Bone and Joint Surgery