Assessment Strategy for 2021 Anaesthetics Curriculum

Formative assessment is assessment for learning. The goal of formative assessment is to monitor progress in order to offer on-going constructive feedback with the aim of improving performance. In formative assessment there is no grade or mark, no pass or fail. Formative assessment must provide good quality feedback; without this the process loses its purpose. Formative assessment encourages reflection on learning by the trainee and demonstrates to both the learner and trainer how the learner is progressing.

SLEs have been in use for over ten years and in that time have been revised so that they emphasise their formative function (Norcini, J. and Burch, V. (2007), ‘Workplace Based Assessments as an Educational Tool: AMEE Guide 31’. Medical Teacher, vol. 29, pp 855-871 ). Integral to the SLEs are reflection on the learning event by the anaesthetist in training and feedback from the assessor. The purpose of feedback is to inform the learner about their work in relation to what is expected and direct them on how to improve. As part of this feedback the assessor can indicate what level of supervision the anaesthetist in training requires for that task or case and how they can improve in order to reach the level of supervision required. To facilitate this levels of supervision have been developed and a supervision/entrustment scale is included on some of the SLEs.

The levels of supervision/entrustment are 1 to 4.

A supervision scale will be used in a formative way to demonstrate progress by the trainee. It will be used to inform summative assessments such as the IAC and IACOA.

Figure 5  – the levels of supervision

 

The educational supervisor should review the SLE with the anaesthetist in training to see how they are progressing and to ensure that they are acting on feedback received. The main formative assessments used in the curriculum are the following SLEs:

Anaesthesia Clinical Evaluation Exercise (A-CEX)

The A-CEX is used during clinical sessions, and the assessments are based on the observed performance of the anaesthetist in training’s skills, attitudes and behaviours, and knowledge. It looks at the anaesthetist in training’s performance in a case rather than focusing on a specific procedure, for example the anaesthetic management of a patient with renal failure.

Anaesthesia List Management Tool (ALMAT)

Similar to the A-CEX, the ALMAT is designed to assess and facilitate feedback on an anaesthetist in training’s performance during their practice. When undertaking an ALMAT, an anaesthetist in training is given responsibility for the running of a surgical list according to their level of competence. This tool is particularly appropriate for more senior anaesthetists in training and allows assessment of both clinical and non-clinical skills. Anaesthetists in training should request this assessment before the start of the list, and they may be assessed either by the trainer with direct responsibility for that list, or it may be possible for an anaesthetist in training working with indirect supervision to be assessed by the nominated supervising consultant for that area. 

Directly Observed Procedural Skills (DOPS)

The DOPS tool is used for assessing performance in procedures, such as arterial cannulation or epidural insertion. This tool is therefore more suited to Stage 1 training rather than Stage 2 or 3, except for new areas of anaesthetic practice, which should focus on higher level skills. They are useful for assessing anaesthetists in training who are learning a new skill e.g. nerve block. 

Case-Based Discussion (CBD)

The CBD is usually used away from the clinical environment – it allows the assessor to question the anaesthetist in training about a clinical episode in order to assess their knowledge and rationale for their actions, or what they would do if presented with the clinical scenario. When undertaking a CBD, the anaesthetist in training should bring the case notes and/or anaesthetic chart of a case that they wish to discuss in retrospect. The conduct and management of the case as well as the standards of documentation and follow up should be discussed. CBDs offer an opportunity to discuss a case in depth and to explore thinking, judgement and knowledge. They also provide a useful forum for reflecting on practice, especially in cases of critical incidents. 

Logbook

The LLp integrated logbook allows the anaesthetist in training’s development as assessed by certain WBAs to be placed in context. It is not a formal assessment in its own right, but anaesthetists in training are required to keep a log of all anaesthetic, pain and ICM procedures they have undertaken including the level of supervision required on each occasion. The logbook demonstrates breadth of experience and a logbook review should consider the mix of cases, level of supervision and balance of elective and emergency cases, if relevant, for the learning outcome. 

Multi-Source Feedback (MSF)

The MSF, unlike the other WBAs, provides specific feedback on generic skills such as communication, leadership, team working, reliability, etc., across the domains of Good Medical Practice from a wide range of individuals who have worked with the anaesthetist in training in the current training year. Other WBAs are a snap shot in time covering a clinical episode, where the MSF is used to measure a anaesthetist in training’s performance across a broader period of time and informs the assessment of achievement of learning outcomes. Anaesthetists in training are required to have at least one MSF completed for each training year and MSFs can be conducted in anaesthesia, pain medicine or ICM. The anaesthetist in training identifies a minimum of 12 people (who should be from a mixture of disciplines) with whom they have worked, for example, consultants, theatre staff, recovery staff, ODPs, midwives and administrative staff, and sends a request through the LLp.

Anaesthetic Quality Improvement Project Assessment Tool (A-QIPAT)

Quality improvement is a key element of professional practice. The A-QIPAT form is introduced in this curriculum to enhance assessment of this learning outcome. This assessment allows individuals who have worked with the anaesthetist in training to comment on their performance as part of a quality improvement project. This is a very useful way to provide the anaesthetist with feedback that is specific to their performance in quality improvement projects (AoMRC Final QI Curriculum January 2019).

Multiple Trainer Reports (MTRs)

Consultant feedback is a mandatory part of completing a learning outcome, and should assure whoever signs the HALO form that the trainee is considered competent to provide anaesthesia and peri-operative care to the required level in this learning outcome.

The MTRs differs from an MSF as it concerns an anaesthetist’s training progress with key capabilities and learning outcomes. MSFs seek feedback from the multidisciplinary team, including consultants, on overall professional behaviour and attitude.

The current RCoA consultant feedback form has been developed to provide reports that give feedback across all the learning outcomes. Consultant feedback will be collated through the LLp and will form part of the Educational Supervisor’s Structured Report (ESSR). At least one MTR will be required per year of training, and for certain areas of training specific MTRs will be required. This includes paediatric, cardiac, neuro and obstetric anaesthesia.

Consultant feedback will be collated, linked to the learning outcome and presented in the ESSR at ARCP. It should be discussed with the trainee during or at the end of a learning outcome prior to sign-off.