Exams during the pandemic

The COVID-19 pandemic has had a large impact on the whole of the anaesthetic profession including anaesthetists in training. As well as the re-deployments to ICUs and disruption to training there has been a disproportionate effect on those anaesthetists early on their career with regards to recruitment and exams. In our last blog we outlined the disruption to national recruitment and the efforts by the College to re-commence a fair process as soon as possible. This piece runs through the effect these unprecedented times have had on exams.

The Primary and Final FRCA written exams had fortunately concluded before the onset of national lockdown in March and therefore were relatively unaffected. As a by-product of the lockdown there was however a delay in the release of the Final written results. This was due to the fact that the CRQ portion of the exam requires examiners to physically mark candidates’ papers (rather than being optically read). These papers are stored at the College and normally reviewed by examiners in person. Lockdown meant that this was not possible and therefore there was a necessary delay until staff could return to the College and scan in all the papers for examiners to mark remotely.

Following lockdown it became quickly apparent that the Primary OSCE/SOE (May) and Final SOE (June) exams would have to be cancelled due the restrictions of movement and ongoing clinical commitment of both candidates and examiners during the peak of ICU admissions. These decisions were made in conjunction with the GMC, Academy of Medical Royal Colleges (AoMRC) and the statutory education bodies (HEE, NES, NIMDTA and HEIW). At this time, the College began early work to ensure that future exams could take place to minimise further disruption to anaesthetists in training.

As with many other Colleges, three main methods of running the exams were considered

  • To continue the current format with social distancing measures in place if necessary. This was felt to be unfeasible due to a number of reasons:
    • For the numbers of candidates, particularly for the written exams, it was readily apparent that finding venues with large enough space to ensure appropriate social distancing would be extremely difficult
    • Restrictions in place at the time meant that it could not be guaranteed that these venues would be available
    • Future national or regional lockdowns could restrict candidates abilities to travel to the exams
    • It was unclear whether hotels would be available for candidates travelling a large distance.
  • To increase the number of exam venues so that exams could be held on a more regional basis. Again this was felt to be difficult because of potential recurrent lockdown as well as the large increase in admin support required in this situation
  • To investigate the possibility of online/remote exams

There was an early decision by the College Examinations Committee (which includes anaesthetists in training representation) that remote running of the written exam at least was possible. This possibility had been previously investigated by the College. A well-established remote examining and invigilation company called TestReach was approached and visited by members of the Examinations Committee at the beginning of 2020.

TestReach are extremely experienced at running remotely invigilated exams across the world. Candidates can be examined in their home or any suitably quiet place of their choosing and will be remotely invigilated by a TestReach employee who is specifically trained to monitor candidates, as in a face-to-face exam, for any behaviour that could indicate cheating. College staff can then be alerted and appropriate action taken if required. As in the pen and paper exam, a comfort break is allowed and candidates are allowed one piece of A4 paper to make notes which will be reviewed by the invigilator at the end of the exam. Prior to the exam, candidates must conduct a system check to ensure that the TestReach App functions on their chosen device. The content and format of the exams remain unchanged. Further information about taking the Primary and Final written exams online can be found here: https://www.rcoa.ac.uk/node/11036

Unfortunately, during the first set of remotely invigilated Primary written exams on the 28 August 2020 many candidates experienced issues with their connection with the TestReach system resulting in incidences of delayed starts and interruptions with their exam. These problems were understandably incredibly stressful and upsetting for the candidates affected. In the following days this issue was investigated thoroughly and changes made by TestReach to reduce the chance of a similar issue in the future. The College’s latest update regarding the response to this can be found here: https://rcoa.ac.uk/news/update-primary-frca-mcq-proctored-examination-candidates . Since this time, candidates have sat the Final written, via the TestReach system, without incident.

The format of the Primary and Final SOEs also allows easy transitioning to delivery online and development of this is in progress.  The format, content and length of the exams will remain the same but extra time will be factored in for any connection or IT issues. The Primary OSCE exam is more challenging to transition to an online format, however the Examinations team have been working hard to make changes to allow this, while ensuring the validity of the exam. Like the written exams, there will be measures in place to ensure that candidates take the test alone and do not have any materials in the vicinity that could help them in the exam.  However, emphasis is placed on the candidate to ensure that they remain under examination conditions and professional throughout the examination.

The College has signed a contract with Fry-IT for the delivery of our clinical examinations online.  Fry-IT are a software company whose assessment platform, Practique, is used by universities and professional bodies, and who have a vast amount of experience working with Royal Colleges including RCEM, RCPsych, RCPCH and RCGP. In the time of COVID-19, they have swiftly developed their platform to enable a full virtual delivery of exams by combining the candidate scheduling and paper build parts of Practique with a video-streaming package. Further details can be found here: https://www.rcoa.ac.uk/node/11151

A risk analysis has been performed by the College comparing socially distanced, hybrid and fully remote OSCE/VIVA exams. Although no solution is without risk, on the basis of this analysis, it is felt that a fully remote OSCE/VIVA gives candidates the best chance of being able to take their examinations with minimal disruption. We accept however, that it is likely than some candidates may have temporary issues with connection, due to local or national issues. The College will be providing more details on what candidates should do in this eventuality, but time is being factored in to the examination period to account for this and candidates will not be penalised for events out of their control.    

With the help of extra commitment from the College examiners, the November 2020 and January 2021 Primary OSCE/SOE will run for ten working days, rather than the usual five or six, in order to accommodate the extra time required to run the exam in its new format and to allow for any potential connection issues that may occur. The December 2020 sitting of the Final SOE was unable to be extended due to examiners’ clinical commitments as well as the proximity to Christmas, therefore an extra sitting has been arranged in February 2021.

Despite these extra sittings, it is likely that there will be a large number of candidates applying for the upcoming sittings. The College has therefore updated its exam prioritisation regulations and it is unfortunately likely that some candidates may not be able to sit the exam in their preferred sitting. Further information on this and updates on exam fees can be found here: https://www.rcoa.ac.uk/news/prioritisation-applications-frca-examinations-academic-year-20202021

The College is very aware of the impact that both the exam cancellations and the changes to future exams has had on candidates. The COVID-19 pandemic is unprecedented and the College has had to adapt its exam system rapidly to ensure that the disruption to training is minimised. We are working hard to put in place robust plans to ensure the continuity of a fair and rigorous exam process which will maintain the high standards of anaesthetists in the UK. The Examinations Committee has input and representation from the Anaesthetists in Training Committee who want to continue to air your views. If you have any questions regarding exams that cannot be answered via the College website, please get in touch with your local Anaesthetists in Training Representative who can be found on our website https://www.rcoa.ac.uk/training-careers/college-representatives/anaesthetists-training-representation

 

Author:

Dr Alister Seaton, RCoA Anaesthetists in Training Committee member

With acknowledgement to:

Dr Fiona Donald, RCoA Vice President
Dr Chris Carey, RCoA Council Member, Training, Curriculum and Assessment Committee chair
Dr Mark Forrest, RCoA Council Member, Examinations Committee chair

Russell Ampofo, Director of Education, Training and Examinations

Fiona Daniels, Head of Examinations