RSNA Announces Winners of Intracranial Hemorrhage AI Challenge

OAK BROOK, Ill. (December 2, 2019) — The Radiological Society of North America (RSNA) has announced the official results of its latest artificial intelligence (AI) challenge.

The RSNA Intracranial Hemorrhage Detection and Classification Challenge required teams to develop algorithms that can identify and classify subtypes of hemorrhages on head CT scans. The data set, which comprises more than 25,000 head CT scans contributed by several research institutions, is the first multiplanar dataset used in an RSNA AI Challenge.

The Machine Learning Steering Subcommittee and the Machine Learning Data Standards Subcommittee worked with volunteer specialists from the American Society of Neuroradiology (ASNR) to label these exams for the presence of five subtypes of intracranial hemorrhage — an effort of unprecedented scope in the radiology community.

“The dedication of the volunteers who contributed data and who worked on annotating the dataset was truly remarkable,” said Charles E. Kahn Jr., M.D., M.S., chair of the RSNA Radiology Informatics Committee (RIC) and editor of the journal Radiology: AI. “They have created a resource of tremendous value for imaging research.”

The award-winning teams in the RSNA Intracranial Hemorrhage Detection and Classification Challenge are:

1. SeuTao
2. NoBrainer
3. takuoko
4. GZ
5. Keep Digging Gold
6. BRAINSCAN.AI
7. Big Head
8. 賞金で焼肉
9. Mind Blowers
10. VinBDI.MedicalImagingTeam

“The results produced by the winning teams achieved really impressive performance,” said Luciano M. Prevedello, M.D., M.P.H., chair of the Machine Learning Steering Subcommittee of the RIC. “The challenge demonstrates the increasing sophistication of the imaging AI research community and the real potential of this technology to improve the efficiency and quality of care in radiology.”

The challenge was run on a platform provided by Kaggle, Inc. (a subsidiary of Alphabet, Inc., also the parent company of Google). The Kaggle platform provides access to datasets, a discussion forum for participants, the repository of submitted results and a leaderboard that runs throughout the challenge. Kaggle also provided $25,000 in prize money to be shared among the winning entries.

In September the first wave of data was released to researchers who worked to develop and “train” algorithms. Participants used a training dataset that includes the radiologists’ labels to develop algorithms that replicate those annotations. During the evaluation phase, from Nov. 4 to Nov. 11, participants applied their algorithms to the testing portion of the dataset, which was provided to them with the annotations withheld. Their results were then compared to the annotations on the testing dataset, and an evaluation metric was applied to rate their performance and determine the winners.

The winners will be recognized in a presentation at the AI Showcase Theater on Monday, Dec. 2 (3:30 – 5 p.m.), during the RSNA Scientific Assembly and Annual Meeting (RSNA 2019, Dec. 1-6, McCormick Place, Chicago, #RSNA19).

“The challenge recognition event provides a great opportunity for AI researchers, the imaging industry and radiologists to meet, exchange ideas and mark the progress toward implementing AI in medical imaging,” said Curtis P. Langlotz, M.D., Ph.D., RSNA Board Liaison for Information Technology and Annual Meeting. “The setting of the event in the AI Showcase area of the RSNA annual meeting really gives a tangible sense of the growing impact of this revolutionary technology.”

For more information on the challenge, visit RSNA.org/AI-image-challenge or https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/overview/description.