Independent Review of the Assessment Processes of the RCoA
The Royal College of Anaesthetists and the Faculties of Intensive Care and Pain Medicine are committed to delivering fair, robust and high-quality examinations that are aligned with GMC standards and best contemporary practice and that uphold and improve standards of patient care.
Two reviews of examinations delivered by the College were completed during the period January 2020 to July 2022. The first was an internal review with the scope to look at all aspects of the FRCA examination against current best practice in postgraduate medical assessment.
The second was an independent review commissioned by the College to provide independent assurance and insights into the development and delivery of our assessment processes, alongside our regular internal reviews of the FRCA. The review considered all aspects of the FRCA, FFICM and FFPMRCA exams, including an error in the delivery of the September 2021 FRCA Final Written Exam and an unexpectedly low pass rate for the October 2021 FFICM Objective Structured Clinical Examination (OSCE).
The independent review was conducted by Professor John McLachlan and we published his report in full on 16 February 2023. We welcome the findings of this review and encourage members to read our full response.
We will address the recommendations, many of which align closely with those of our recent internal review of the FRCA, working closely with the Faculties of Pain and Intensive Care Medicine. The reviews will inform our developmental roadmap, and we will implement changes with care over the coming months and years, taking into account the needs of candidates and consulting with stakeholders.
While it is likely that changes will be made to the FRCA exam format in future, the current FRCA examinations remain valid, reliable, fit for purpose and are approved by the GMC. We will ensure exam candidates are given at least 12 months’ notice of any significant changes to the FRCA exams in order to facilitate good preparation and support. Therefore, anaesthetists in training and prospective candidates should not change the way they prepare for the current sets of exams until formally notified of any changes.
We have published a summary of our proposed timetable for implementation of the changes resulting from the review, and we highlight the main themes relevant to the FRCA examinations here. We will keep our members and the GMC informed of our progress through regular updates on our website and member communications.
Our programme of improvement
In considering the recommendations from both the independent review and our internal review, we will:
- Invest additional resources in our examinations function to implement changes arising from the review and strengthen our capacity to deliver exams. We have established an Examinations Development and Assurance Group (EDAG) to lead the development, quality and alignment of exam processes across the FRCA and Faculty exams, where appropriate. This group will have an independent chair and comprise examiners from all exams delivered by the College, members of the exams team, lay and trainee representatives and a specialist educational assessment expert.
- Give candidates and anaesthetists in training a greater role in our assessment processes, including more representation on College committees engaged in assessment. These representatives will also play a central role in supporting communication with candidates.
- Review the purpose of our assessments in accordance with the changes made in the curriculum and the changing nature of clinical practice in our specialties. We will consult with members and stakeholders to define this.
- Implement a unified approach to the use of, and training in, standard-setting across the FRCA exams and all assessments delivered by the College.
- Undertake new research to inform our assessment design process. We are committed to investigating the impact of gender, ethnicity and educational background on exam performance. We also aim to undertake validity research to compare exam performance with performance in the workplace as estimated by trainers and supervisors with performance in RCoA assessments.