Less than half of group of Italian radiologists were able to successfully complete a mammography on-line self-assessment test.
The detection of breast cancer requires the correct interpretation of mammography scans and guidelines on screening require that robust quality control and quality assurance are in place. Although an assessment of performance against standards can be achieved through audits, such audits provide little insight of the diagnostic ability of individual radiologists. The use of on-line self-assessment, using pre-diagnosed mammograms, would therefore offer a means of identifying any knowledge gaps and could serve as an important educational tool.
The use of an on-line self-assessment tool was adopted by the Italian Society of Medical Radiology in 2018 and incorporated as part of a national on-line self-assessment program for interpretation of mammograms. The test included a set of 24 pathology-confirmed cancers combined with 108 mammograms which were reported on as negative after double reading. All cases were acquired with full-field digital mammography and came from women aged 50 – 69 years, who represent those at average risk of breast cancer. For the assessment, the society determined a pass threshold for sensitivity of 62% (which amounted to at least 15/24 cancers correctly identified) and 86% specificity (93/108 negative mammograms correctly identified). Radiologists registering with the on-line self-assessment provided information on demographics, place of work (public or private sector), years of breast imaging experience and a qualitative self-judgement of mammography interpretative skills as “beginner”, “average” and “expert”. As with usual mammography practice, the radiologists were required to submit a dichotomous diagnosis (i.e., positive/negative) for every case. In addition to reporting on the radiologist’s success in the test, the authors employed regression analysis and odds ratios to identify the most important predictor variables for diagnostic accuracy.
Findings
A total of 685 radiologists registered for the on-line self-assessment and 49.9% (342) with a mean age of 46 years (69% female), completed test. Among those completing, two-thirds (64%) self-judged their interpretative skills as “average” and respondents had a median of 8 years breast imaging experience although 38.3% reporting having more than 10 years’ experience. Participants reported a median number of 1501 mammographic interpretations per year and the majority (68.7%) worked in the public rather than private sector.
When examining the proportion who successfully completed the assessment, only 28.7% of radiologists (98/342) passed on their first attempt. After an initial failure, 138 of the remaining 244 radiologists, re-took the test of whom, only 35.5% (49/138) passed. In fact, the authors reported that overall, only 44.2% (151/342) of radiologists who completed the on-line self-assessment were successful.
Using regression analysis, a significant association for diagnostic accuracy was found only for the assessment of > 3,000 mammograms per year compared to < 1,000 (Odds ratio, OR = 3.88, 95% CI 1.07 – 14.14, p = 0.04) and working the in public rather than private sector (OR = 1.65). Other variables such as age, self-judged interpretative skills and years of breast imaging experience had no significant effect.
The authors concluded that their study suggested that breast imaging experience does not guarantee diagnostic accuracy in screening reading. They also noted that their on-line self-assessment test could be included as a criterion for the accreditation process of breast units.
Citation
Brancato B et al. Mammography self?evaluation online test for screening readers: an Italian Society of Medical Radiology (SIRM) initiative. Eur Radiol 2021
