In February 2023 we published our independent and internal reviews of the FRCA examinations together with a commitment to develop our examinations and update our processes. This article provides an overview of the positive steps that have taken place since February 2023 to enhance the quality and effectiveness of the examinations in the RCoA, FICM, and FPM.

Action plans have been developed to focus on addressing the recommendations ensuring validity, reliability, and utility while reducing the assessment burden.

Enhancing capacity – supporting our staff

We have increased capacity within the examinations department through the creation of new posts. This move ensures additional support and stability in key examination processes and functions. The enhanced staffing will allow staff to better contribute their expertise to the implementation of recommendations from the reviews and the development of the exams while managing the numerous activities associated with exam delivery.

Collaboration and expertise – establishment of the Examinations Development and Assurance Group

A new Examinations Development and Assurance Group (EDAG) has been established as a working group to oversee the implementation of recommendations from the exam reviews. This group was formed at the end of the academic year 2022/2023 and has representation from all three examinations. The Group operates as a forum for sharing good practices and expertise from across the examinations and for exploring opportunities for collaboration. It has an independent chair, and comprises leads from each exam, an expert in postgraduate medical assessment, and (in line with our aim to give candidates a greater role in our assessment) doctors-in-training representatives for each exam. Their collective efforts will drive positive change and create a more authentic and future-proofed examination system.

So far, the Group has approved new statements that describe the purpose of the FRCA and its constituent parts, Primary and Final, and an outline of new ways to assess clinical application of knowledge in the Primary examination. The intention is for EDAG to have approved all revised formats by the end of this academic year ready for further consultation with stakeholders, followed by initial trials and pilots which we anticipate starting in the academic year 2024/2025.

Empowering inclusive decision-making

To ensure transparent and inclusive decision-making, the Head of Examinations and relevant staff within the examinations department have now been made full voting members in all relevant assessment committees. This change in the terms of reference allows a greater voice for the assessment, statistics, and logistical expertise and knowledge to actively contribute to the exams, aligning their roles with other committee members. This step reinforces the commitment to create an inclusive environment for decision-making processes.

Education fellows and research

Last August, we welcomed to the College two education fellows who joined the examinations department to help address the research recommendations within the independent review, and to support work to reduce the attainment gap in RCoA examinations. The latter has entailed the successful delivery of two SOE master classes, starting in the North West but rolling out to other regions. Early discussions are underway to research the validity of the examination by comparing examination outcomes with workplace indicators such as ARCP outcomes. A third fellow started in February of this year to support the development of the Primary written exam. By engaging in research, these fellows will contribute to the ongoing improvement of the examination system, ensuring its continuous development and alignment with best practices.

Engaging with, and listening to, our stakeholders

The voices of examiners, doctors-in-training, and the wider membership continue to be vital in shaping the future of our College and faculty examinations. Two listening events allowed examiners and candidates to raise questions and provide valuable feedback on our development plans. These interactive sessions and surveys create a platform for meaningful engagement, fostering a sense of collective responsibility in the examination process, and we intend to continue with similar engagement as plans move on.

To ensure greater input from doctors-in-training and SAS doctors, in October 2023 we launched a new Affiliate Examiner role to initially support the development of Single Best Answer questions and provide input to the written components of the FRCA exam, and later to support the design, development and piloting of new clinical oral exams for the FRCA. We had a tremendous response, and 24 new Affiliate Examiners will be welcomed to the RCoA examinations department in May of this year.

Bridging expertise – the power of externality

To gather insights from renowned experts in postgraduate medical assessment, Professor Rikki Goddard-Fuller, an independent assessment expert, has joined EDAG to provide valuable advice and guidance on exam design and development. Leveraging external expertise helps ensure that the examination system remains at the forefront of best practices.

Continuous professional development for examiners

Examiner training and professional development are critical for maintaining high examination standards. To address this, examiner CPD days have been implemented, covering a range of topics including the attainment gap, equality diversity and inclusion, assessment needs of neurodivergent candidates, and a question-development masterclass. The CPD initiative will empower examiners to adapt to evolving assessment methodologies and ensure consistency across all examinations.