RSNA Recognizes Winners of Pneumonia Detection Machine Learning Challenge

OAK BROOK, Ill. (November 26, 2018) — The Radiological Society of North America (RSNA) has announced the official results of its second annual machine learning challenge.

The RSNA Pneumonia Detection Challenge required teams to develop algorithms to identify and localize pneumonia in chest X-rays. Over 1,400 teams took part in the challenge, and 346 submitted results during the evaluation phase of the competition.

The challenge made use of a publicly available chest X-ray dataset from the National Institutes of Health. The RSNA Machine Learning Steering Committee collaborated with volunteers from the Society of Thoracic Radiology, led by Carol Wu, M.D., to annotate the dataset, identifying instances of probable pneumonia. The annotated dataset provided the "ground truth" for participants to train their algorithms and to evaluate their submissions in the final phase of the challenge. Annotation of the datasets was organized and validated using tools provided by MD.ai under the leadership of George Shih, M.D., and Anouk Stein, M.D.

"A successful machine learning challenge needs to begin with a dataset accurate and large enough to provide ground truth," said Safwan Halabi, M.D., medical director of Radiology Informatics at Stanford Children's Health and chair of the RSNA Machine Learning Data Standards Committee. "Developers build their models by training them on the dataset, and challenge organizers use a segment of the dataset to measure their performance. One of the main goals of the competition is to advance the use of machine learning as a tool to improve diagnostic accuracy and efficiency with the ultimate goal of improving patient care."

The challenge was run on a platform provided by Kaggle, Inc. (a subsidiary of Alphabet, Inc., also the parent company of Google). The Kaggle platform provides access to datasets, a discussion forum for participants, the repository of submitted results and a leaderboard that runs throughout the challenge. Kaggle also provided $30,000 in prize money to be shared among the winning entries.

The winning teams in the RSNA Pneumonia Detection Challenge are:

1. Ian Pan & Alexandre Cadrin
2. Dmytro Poplavskiy [ods.ai]
3. Phillip Cheng
4. 16bit.ai / layer6
5. JRS_HP
6. PFNeumonia
7. DEEPRADIOLOGY
8. Mu Song
9. DASA-FIDI-IARA
10. DancingBears

The winners will be recognized at a Machine Learning Challenge Awards ceremony held today at 2:00-3:30 p.m. in the Machine Learning Showcase (North Building, Hall B) during the RSNA 2018 annual meeting at McCormick Place in Chicago.

"The expectation that artificial intelligence will soon provide valuable tools for radiology continues to grow," said Luciano Prevedello, M.D., M.P.H., chief of the Division of Medical Imaging Informatics at The Ohio State University and chair of the Machine Learning Steering Subcommittee of the RSNA Radiology Informatics Committee. "By organizing machine learning data challenges, RSNA is playing an important role in fostering and demonstrating these capabilities."

Full results and detailed information on the challenge is available on the Kaggle site: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge.