Overview
The Skills Log has been developed in-house over the last 12 months. The log is an attempt to improve the way Clinical Skills teaching is managed and assessed over the five years of the undergraduate programme.
Traditionally Clinical Skills have been taught in specialist Clinical Skills labs. Skills have then been practiced, first in the skills labs, and later in the clinical environment under the supervision of a Healthcare professional.
Assessment of skills has been done using OSCE examinations, with students requiring at least one of each of the clinical skills to be observed by a Healthcare Professional in a clinical setting during the final year.
The purpose of the Skills Log is track the opportunites students have to practice their clinical skills in the clinical setting, which is where much of the real learning occurs. The purpose of this tracking is many-fold:
- To help students track their own progress and take responsibility for gaining opportunities to practices skills, allowing them to identify short comings in their own experience and competence.
- To help the School monitor the opportunities students are getting to practice skills and resolve any short comings identified
- To help the School assess the Clinical Skills teaching being provided by looking at overall student performance for each skill.
The log as a system currently comprises a paper-based log book, with tear-off sheets, and the online website, where data is eventually stored and analysed. In the clinical setting students will use the paper-based log book to capture feedback on their performance from themselves, peers and Health-care Professionals. This information is later transcribed into the online log.
Currently the system is in pilot. We have already gathered feedback from the 2011 Diagnostic Synthesis placement, where the log was first used by students. This feedback has resulted in version 2 of the log being released, which addresses a number of concerns raised.
In particular the system has moved away from attempting to verify every transcribed log entry by email. While this was requested by Honorary Senior Lecturers, the clinicians on the ground have been less receptive to this idea and have resented the volume of email being sent. We have also encountered problems with emails being filtered. To this end the system has been modified to only spot check around 10% of entries, which is being trialled now. Hopefully this will appease clinicians, make it less awkward for students to ask for feedback, and make the system more manageable for school administrators.
Future plans
There are number of areas for refinement and development, as well as a need to look at the educational philosophy associated with the tool and its use.
Use of paper-based log books
The current use of paper-based log books and the need to transcribe that data into the online log at a later time is a little clumsy. This can almost certainly improved (though not entirely removed) through the use of mobile technologies. An increasing number of students have smart phones and we have been looking to develop a mobile interface for the log to allow digital capture of the initial data in place of the paper-based log book. Currently work has started (but is on hold) on a mobile version of the Skills Log website, with support for offline access and data storage. Long term, native mobile apps may be a better option.
CUSUM analysis
CUSUM analysis is a statistical technique that can be used for demonstrating competence. This is currently being evaluated and has been implemented and enabled for tutors and administrators to view for individual students. Work will need to be done to refine the parameters being used for the analysis and to validate the outputs of the analysis against OSCE results and feedback from Health-care professionals on a student’s capability.
Educational philosophy
As a pilot this tool suffers from not being well integrated into the teaching and assessment methodology for clinical skills. We are asking students to gather lots of data and lots of signatures, and encouraging them to gain lots of practice, while still requiring them to complete traditional assessments. In short, the tool is not yet well embedded into the Clinical Skills teaching and assessment. Furthermore, the system is being sold as a student aid (to help them manage their own learning) while requiring signatures and then attempting to validate each entry with Health-care Professionals. This mixed philosophy is to be expected in the pilot but will need addressing long term. Some points to be clarified include:
- What is the log attempting to do or demonstrate?
- Is it a student aid to learning or a summative assessment tool?
