2021 Curriculum for a CCT in Anaesthetics

The following methods will provide evidence of progress in the integrated programme of assessment. The requirements for each training year/level are stipulated in the ARCP decision guidance; checklists for anaesthetists in training and for educational supervisors can be found on the College website. Evidence is a crucial concept in this curriculum, and as well as the methods listed below, can include other sources, such as the Personal Development Plan, quality improvement project or logbook summaries. The anaesthetist in training will collect evidence to support their acquisition of the requirements for each of the domains, and the Educational Supervisor will use it to make a global judgement indicating whether the anaesthetist in training has made satisfactory progress for the defined stage of training. These methods are described briefly below. More information and guidance for anaesthetists in training and trainers will be available in the ‘Guide to Anaesthetics Training’.

Summative assessment

Summative assessment is an assessment of learning and results in a mark or grade, pass or fail. The goal of summative assessment is to test knowledge or performance against set criteria. The summative assessment in the anaesthetic training programme takes the following forms:

• Fellowship of the Royal College of Anaesthetists (FRCA) examinations: Primary and Final
• Initial Assessment of Competence (IAC)
• Initial Assessment of Competence in Obstetric Anaesthesia (IACOA)
• Holistic Assessment of Learning (HALO) form
• Multiple Trainer Reports (MTR)
• Educational supervisors structured report (ESSR)
• Entrustable Professional Activities (EPAs).

Formative assessment

Formative assessment is assessment for learning. The goal of formative assessment is to monitor progress in order to offer ongoing constructive feedback with the aim of improving performance. In formative assessment there is no grade or mark, no pass or fail. Formative assessment must provide good quality feedback; without this the process loses its purpose. SLEs have been in use for over ten years and in that time have been revised so that they emphasise their formative function¹⁵. Integral to the SLEs are reflection on the learning event by the anaesthetist in training and feedback from the assessor. The purpose of feedback is to inform the learner about their work in relation to what is expected and direct them on how to improve. As part of this feedback the assessor can indicate what level of supervision the anaesthetist in training requires for that task or case and how they can improve in order to reach the level of supervision required. To facilitate this, levels of supervision have been developed and a supervision/entrustment scale is included on some of the SLEs.

A supervision scale will be used in a formative way to demonstrate progress by the trainee. It will be used to inform summative assessments such as the IAC and IACOA.

Figure 6 – The levels of supervision

The educational supervisor should review the SLE with the anaesthetist in training to see how they are progressing and to ensure that they are acting on feedback received.

The main formative assessments used in the curriculum are the SLEs:

• Anaesthetic Clinical Evaluation Exercise [A-CEX]
• Anaesthetic List/Clinic/Ward Management Assessment Tool [ALMAT]
• Direct Observation of Procedural Skills [DOPS]
• Case Based Discussion [CBD]
• Logbook
• Multi-Source Feedback [MSF]
• Anaesthetic Quality Improvement Project Assessment Tool [A-QIPAT]
• Multiple Trainer Report (MTR)

SLEs

Each individual SLE is designed to assess a range of important aspects of performance in different training situations. Taken together they can assess the breadth of knowledge, skills and performance described in the curriculum. The SLEs described in this curriculum have been in use for over ten years and are now an established component of training.

The SLE methodology is designed to meet the following criteria:

• Validity – the assessment actually does test what is intended; that methods are relevant to the actual clinical practice; that performance in increasingly complex tasks is reflected in the assessment outcome
• Reliability – multiple measures of performance using different assessors in different training situations produce a consistent picture of performance over time
• Feasibility – methods are designed to be practical by fitting into the training and working environment
• Cost-effectiveness – the only additional significant costs should be in the training of trainers and the time invested needed for feedback and regular appraisal, which should be factored into trainer job plans
• Opportunities for feedback – structured feedback is a fundamental component
• Impact on learning – the educational feedback from trainers should lead to anaesthetists in training’ reflections on practice in order to address learning needs.

SLEs use different trainers’ direct observations of anaesthetists in training to assess the actual performance of anaesthetists as they manage different clinical situations in different clinical settings and provide more granular formative assessment in crucial areas of the curriculum than does the more global assessment provided by supervisors’ reports. SLEs are primarily aimed at providing constructive feedback to anaesthetists in training in important areas of the curriculum throughout each placement in all phases of training. It is normal for anaesthetists in training to have some assessments that identify areas for development because their performance is not yet at the standard for the completion of that training.

How many SLEs?

In order to complete a learning outcome, anaesthetists in training should undertake SLEs that contribute to evidence showing key capabilities at each of the three stages of training. There are several key capabilities within each domain and stage of training and a single assessment may provide evidence to satisfy multiple key capabilities across a range of domains.

The SLE blueprints are found in section 5.10. These show how the SLEs could be used to provide evidence towards demonstration of attainment of each learning outcome.

There are no requirements for minimum numbers of SLEs. The SLEs should be used in a formative way to demonstrate reflection on learning and progress by the trainee. The SLEs allow the trainer to indicate what level of supervision is required for the trainee. For the IAC and IACOA trainers need to be satisfied that the anaesthetist is able to perform with the required a certain level of supervision in order to complete these training requirements.

The Faculty of ICM has set the number of assessments for ICM, which are listed in the ICM Curriculum, and some assessments achieved in the anaesthesia curriculum may be cross counted to satisfy ICM competences and vice-versa.

Who can assess?

Consultants, specialty anaesthetists, and senior anaesthetists in training can assess SLEs. In accordance with GMC standards, assessors must possess expertise in the area to be assessed and be familiar with the assessment process. Senior anaesthetists in training and non-medical staff may assess SLEs if they have completed appropriate training, and if the educational supervisor (ES) considers it appropriate. The ES may need to enter the assessment in the Lifelong Learning platform (LLp). Anaesthetists in training must not perform assessments for the IAC and the IACOA.

The supervised learning event process:

• feedback is the most important element of a SLE
• anaesthetists in training should undertake SLEs relevant to their current practice
• areas for learning should be identified prior to starting a list, clinic, ward-round, etc., and the anaesthetist in training should ask the trainer in advance to perform a SLE
• requesting SLEs retrospectively is considered poor practice and is not acceptable, except in Case-Based Discussions
• the anaesthetist in training should reflect on the learning event in the SLE
• the trainer should observe the performance of the anaesthetist in training, and give immediate verbal feedback as well as suggestions for future development, further reading etc.; they will indicate what level of supervision the anaesthetist requires for that activity
• trainers should comment on clinical and non-clinical aspects of performance, such as professionalism and team-working
• if facilities exist and it is safe to do so, the assessment can be documented on the LLp at this time
• if the online form cannot be completed at this time, the anaesthetist in training will send a request for a SLE to the trainer electronically
• verbal feedback should always take place at the time of the SLE
• the trainer should complete the online form as soon as possible
• the anaesthetist in training should link the form to the relevant learning outcome so that the SLE can be used as evidence for the HALO
• linking a SLE to more than one unit of training may be appropriate, if it demonstrates relevant progress.

Local education providers/hospitals and Deaneries often provide training in supervised learning events. The College provides training in the ‘Anaesthetists as Educators’ courses, and online materials are available on the RCoA website.

Supervisor reports

Consultant feedback, and feedback from other approved anaesthetist trainers, is a fundamental source of evidence when assessing anaesthetists in training’ performance. This means of assessment is valuable in identifying anaesthetists in training who are performing above and below the standard expected for their level.

All of these methods are described briefly below and include feedback opportunities as an integral part of the programme of assessment. A record of the assessment, including feedback, should be recorded in the anaesthetist in training’s LLp. More information and guidance for anaesthetists in training and trainers is available in the ‘2021 Curriculum Assessment Guidance’.

FRCA

The FRCA examination is a two-part ‘high-stakes’ national assessment. Its major focus is on the knowledge required for practice but the structured oral examination [SOE] and objectively structured clinical examination [OSCE] test decision-making, understanding of procedure and practical elements (including the use of simulation).

The Primary examination is divided into two parts: the MCQ and the OSCE/SOE. Possession of the Primary FRCA is a mandatory requirement for entry into Stage 2 (ST4).

The Final FRCA also consists of two parts: the written and the SOE. The Final examination must be successfully completed in order to progress to Stage 3 (ST6).

Further details on the examinations are available on the Examinations pages on the RCoA website.

IAC

The IAC is the first critical progression point in the anaesthetic curriculum, and the anaesthetic element of the ACCS curriculum. The purpose of the IAC is to signify that the anaesthetist in training has achieved a basic understanding of anaesthesia and is able to give anaesthetics at a level of supervision commensurate with the individual anaesthetist in training’s skills and the clinical case; and the anaesthetist in training can be added to the on-call rota for anaesthesia. The IAC is not a licence for independent anaesthetic practice.

IACOA

The IACOA must be obtained by all anaesthetists in training before being considered safe to work in an obstetric unit without direct supervision. Achieving the IACOA does not signal the completion of training in obstetric anaesthesia during Stage 1. Further training will be required in order to attain the required key capabilities.

Holistic Assessment of Learning Outcomes (HALOs)

A satisfactorily completed HALO form provides evidence that an anaesthetist in training has achieved the key capabilities required to demonstrate attainment of a stage learning outcome, in order to progress to the next. Supervisors should draw upon a range of evidence including the logbook of cases completed, SLEs, illustrations set out in the curriculum document, and consultant feedback to inform their decision as to whether the stage learning outcome has been achieved. The logbook review should consider the mix of cases, level of supervision and balance of elective and emergency cases, if relevant, for the stage learning outcome. Evidence for achievement of key capabilities and learning outcomes will be uploaded to the LLp and will be linked by the anaesthetist in training to the relevant stage learning outcome. The supervisor will be able to review this evidence at the end of a stage of training to complete the HALO but it is expected that the evidence will be collected and linked throughout the stage of training period so that educational supervisors and ARCP panels are able to review progress.

All hospitals must identify appropriate designated trainers to sign the HALO form for each stage learning outcome. Each trainer should be familiar with the requirements for the stage learning outcome and be able to provide guidance for anaesthetists in training who have not yet achieved the learning outcomes. It is anticipated that the HALOs for the generic professional capability based stage learning outcomes will be signed by the anaesthetist’s educational supervisor. The professional judgement of the supervisor will ultimately determine whether it is appropriate to sign the HALO form for an anaesthetist in training.

Stage 1 Training Certificate

The Stage 1 Training Certificate signifies that an anaesthetist in training has achieved the required HALOs in all learning outcomes for that stage of training, has passed the Primary FRCA, and is eligible to progress to Stage 2.

Stage 2 Training Certificate

The Stage 2 Training Certificate signifies that an anaesthetist in training has achieved the required HALOs in all learning outcomes for that stage of training, has passed the Final FRCA, and is eligible to progress to Stage 3.

Stage 3 Training Certificate

The Stage 3 Training Certificate signifies that an anaesthetist in training has achieved the required HALOs in all learning outcomes for that stage of training and is eligible for the award of a CCT or CESR[CP].

Anaesthesia Clinical Evaluation Exercise (A-CEX)

The A-CEX is used during clinical sessions, and the assessments are based on the observed performance of the anaesthetist in training’s skills, attitudes and behaviours, and knowledge. It looks at the anaesthetist in training’s performance in a case rather than focusing on a specific procedure, for example the anaesthetic management of a patient with renal failure.

Anaesthesia List Management Tool (ALMAT)

Similar to the A-CEX, the ALMAT is designed to assess and facilitate feedback on an anaesthetist in training’s performance during their practice. When undertaking an ALMAT, an anaesthetist in training is given responsibility for the running of a surgical list according to their level of competence. This tool is particularly appropriate for more senior anaesthetists in training and allows assessment of both clinical and non-clinical skills. Anaesthetists in training should request this assessment before the start of the list, and they may be assessed either by the trainer with direct responsibility for that list, or it may be possible for an anaesthetist in training working with indirect supervision to be assessed by the nominated supervising consultant for that area.

Directly Observed Procedural Skills (DOPS)

The DOPS tool is used for assessing performance in procedures, such as arterial cannulation or epidural insertion. This tool is therefore more suited to Stage 1 training rather than Stage 2 or 3, except for new areas of anaesthetic practice, which should focus on higher level skills. They are useful for assessing anaesthetists in training who are learning a new skill e.g. nerve block.

Case-Based Discussion (CBD)

The CBD is usually used away from the clinical environment – it allows the assessor to question the anaesthetist in training about a clinical episode in order to assess their knowledge and rationale for their actions, or what they would do if presented with the clinical scenario. When undertaking a CBD, the anaesthetist in training should bring the case notes and/or anaesthetic chart of a case that they wish to discuss in retrospect. The conduct and management of the case as well as the standards of documentation and follow up should be discussed. CBDs offer an opportunity to discuss a case in depth and to explore thinking, judgement and knowledge. They also provide a useful forum for reflecting on practice, especially in cases of critical incidents.

Logbook

The LLp integrated logbook allows the anaesthetist in training’s development as assessed by certain SLEs to be placed in context. It is not a formal assessment in its own right, but anaesthetists in training are required to keep a log of all anaesthetic, pain and ICM procedures they have undertaken including the level of supervision required on each occasion. The logbook demonstrates breadth of experience and a logbook review should consider the mix of cases, level of supervision and balance of elective and emergency cases, if relevant, for the learning outcome.

Multi-Source Feedback (MSF)

The MSF, unlike the other SLEs, provides specific feedback on generic skills such as communication, leadership, team working, reliability, etc., across the domains of Good Medical Practice from a wide range of individuals who have worked with the anaesthetist in training in the current training year. Other SLEs are a snapshot in time covering a clinical episode, where the MSF is used to measure an anaesthetist in training’s performance across a broader period of time and informs the assessment of achievement of learning outcomes.

Anaesthetists in training are required to have at least one MSF completed for each training year and MSFs can be conducted in anaesthesia, pain medicine or ICM. The anaesthetist in training identifies a minimum of 12 people (who should be from a mixture of disciplines) with whom they have worked, for example, consultants, theatre staff, recovery staff, ODPs, midwives and administrative staff, and sends a request through the LLp.

Anaesthetic Quality Improvement Project Assessment Tool (A-QIPAT)

Quality improvement is a key element of professional practice. The A-QIPAT form is introduced in this curriculum to enhance assessment of this learning outcome. This assessment allows individuals who have worked with the anaesthetist in training to comment on their performance as part of a quality improvement project. This is a very useful way to provide the anaesthetist with feedback that is specific to their performance in quality improvement projects.¹⁶

Multiple Trainer Reports (MTRs)

Consultant feedback is a mandatory part of completing a learning outcome and should assure whoever signs the HALO form that the trainee is considered competent to provide anaesthesia and peri-operative care to the required level in this learning outcome.

The MTRs differs from the MSF as it concerns an anaesthetist’s training progress with key capabilities and learning outcomes. MSFs seek feedback from the multidisciplinary team, including consultants, on overall professional behaviour and attitude.

The current RCoA consultant feedback form has been developed to provide reports that give feedback across all the learning outcomes. Consultant feedback will be collated through the LLp and will form part of the Educational Supervisor’s Structured Report (ESSR). At least one MTR will be required per year of training, and for certain areas of training specific MTRs will be required. This includes paediatric, cardiac, neuro and obstetric anaesthesia.

Consultant feedback will be collated, linked to the learning outcome and presented in the ESSR at ARCP. It should be discussed with the trainee during or at the end of a learning outcome prior to sign-off.

Educational supervisors structured report (ESSR)

The LLp system allows for multiple ESSRs per year that can be completed at intervals reflective of individual training programmes, as agreed between an anaesthetist in training and an educational supervisor. These will all subsequently feed into an ARCP.

The ESSR will periodically (at least annually) record a longitudinal, global report of an anaesthetist in training’s progress based on a range of assessment, potentially including exams and observations in practice or reflection on behaviour by those who have appropriate expertise and experience. The ESSR can incorporate commentary or reports from longitudinal observations, such as from supervisors or formative assessments demonstrating progress over time.

Entrustable Professional Activities (EPAs)

The RCoA utilises supervision/entrustment scales for SLEs (DOPS, A-CEX, ALMAT, and CBDs) within the curriculum to provide formative assessment and meaningful feedback on the level of supervision that was required for anaesthetists in training undertaking clinical activities. Entrustable professional activities (EPAs) involve looking across a range of different skills and behaviours to make global decisions about an anaesthetist in training’s suitability to take on particular responsibilities or tasks and help to establish an increase in autonomy and responsibility for the unsupervised practice of key activities. (ten Cate, 2013) Unlike conventional SLEs that assess previous activity, EPAs focus on an anaesthetist in training’s ability to cope with future situations and challenges. (Peters, 2017¹⁷)

This curriculum embeds EPAs at two critical progression points to make summative decisions on defined areas of practice confirming that the trainee is able to undertake specific responsibilities safely and independently. These summative assessments will be undertaken by ‘training faculty members‘ who have observed an anaesthetist in training’s performance on multiple occasions and who utilise all available sources of relevant information including; SLEs, clinical logbook, supervisor reports, MSF, and MTRs. Utilising all the relevant information available at each progression point for individual anaesthetists in training will ensure that the curriculum is underpinned by a programmatic approach to assessment¹⁸ .

The EPAs are centred on an anaesthetist in training’s ability to join the on-call rotas for general and obstetric anaesthesia and are widely recognised as priority areas in which entrustment decisions are required to ensure patient safety.

Initial Assessment of Competence

This comprises three arenas of professional activity:

• safe general anaesthesia with spontaneous respiration to ASA 1-2 patients for uncomplicated surgery in the supine position
• safe rapid sequence induction for ASA 1-2 patients aged 16 or older and failed intubation routine
• safe perioperative care to ASA 1E – 2E patients requiring uncomplicated emergency surgery.

Initial Assessment of Obstetric Competence

This comprises four arenas of professional activity:

• safe administration of epidural/CSE for pain relief in labour
• safe administration of epidural top-up for an emergency caesarean section
• safe administration of spinal/CSE for elective or emergency caesarean section
• safe administration of general anaesthesia for elective or emergency caesarean section.