FRCR Part 2B (Oncology) - CO2B - scoring system

1. Context

The Final FRCR (Part B) Exam consists of 12 assessment stations with clinical style questions. Each exam is conducted and marked independently by two examiners. There is no ranking of candidates, other than to determine the recipient of the Gold Award as described below.

2. Preparation

Questions for the stations are prepared in advance, with guidelines provided to examiners on their features and scoring.

3. Marking scheme

Within each station, examiners will assess candidates against three clinical skill domains. The clinical skill domains that are assessed across the exam are: A. Communication B. Clinical judgement C. Managing patients on radiotherapy D. Interpretation E. Patient-centred care F. Managing patients receiving systemic anticancer therapies (SACT) Candidates will be rated for each domain using the following rating scale:

• Definite fail
• Just fail
• Just pass
• Definite pass

Generic anchor statements describing the characteristics shown by candidates that would lead to different ratings for each clinical skill domain are provided to examiners. Additionally, case-specific marking criteria is developed and provided to examiners. Further information on the clinical skill domains is available in the CO2B exam blueprint. 

All stations are double marked. For the majority of stations, the first examiner conducts the exam while marking in real-time. The second examiner watches the video of the exam and marks independently. The exception is the communication station, which is not live marked – instead, both examiners mark independently using the video.

After the exam ends, ratings provided for each domain are converted into numerical scores (ranging from 0 to 3). Per station, each examiner can score a candidate up to a maximum of nine marks. If there is more than a four mark difference between the first and second examiner marks for a station, the senior examiner will ask the two examiners to discuss their ratings for that station. Having conferred, the ratings can be altered if necessary and this is fed back to the senior examiner.

At each station, an examiner also provides a global judgement for each candidate. This judgement uses the following ratings:

• Very poor fail
• Clear fail
• Borderline fail
• Borderline pass
• Clear pass
• Very good pass

The global judgement is an overall impression of the candidate’s performance across the whole station and is used for standard setting for the exam only, as described below.

4. Calibration

A number of processes are undertaken to ensure there is calibration across all examiners. All examiners have prior sight of the cases used within the exam stations and have the opportunity to feedback on the cases’ content. Marking criteria are developed for each station. These cover the clinical skill domains and expected characteristics displayed by the candidate for each of the domain ratings (Definite fail / Just fail / Just pass / Definite pass). At the end of the first round of assessments, the senior examiner will liaise with all examiners to check if any issues have arisen that had not been recognised at the start of the day. If a problem is encountered, such as an element of the question which was interpreted in an unanticipated way by all candidates, then the senior examiner may give permission to adjust the recommendations and rating of the candidates who have been through the station.

5. Exam pass / fail decisions

Borderline regression, an established and recognised standard setting methodology, is used to determine the pass mark for the exam.

Station (domain) scores are regressed against global ratings, allowing a ’cut-score’ to be calculated for each station. This cut-score is based on the station score of a ‘borderline’ candidate. An overall pass mark is obtained by summing the cut-scores of all 12 stations. Candidates who achieve the pass mark or higher will pass.

6. The Gold Award

The Gold Award can be awarded to an outstanding candidate on the recommendation of the Exam Board, based on performance, at each sitting of the exam. To identify the best-placed candidate to be recommended, the following criteria are applied in this order:

• Candidates who have passed all sections of the Final FRCR at their first attempt and all in one round (i.e. sitting Part A and B consecutively).
• Candidates will be ranked according to score on each of the two exams - Part A (SBA paper) and Part B (multi-station exam) - and given a ranking number from 1 for the highest score. The candidate who has the highest combined placing in the exam components (i.e. the lowest combined ranking numbers) will be recommended.
• In the event of a tie, the tied candidate who achieved the highest combined score in Part B will be recommended.
• If still tied, the tied candidate who achieved the highest score in Part A will be recommended.