A game-based crowdsourcing platform for rapidly training middle and high school students to perform biomedical image analysis

Steve Feng; Min-jae Woo; Hannah Kim; Eunso Kim; Sojung Ki; Lei Shao; Aydogan Ozcan

doi:10.1117/12.2212310

Translator Disclaimer

11 March 2016 A game-based crowdsourcing platform for rapidly training middle and high school students to perform biomedical image analysis

Steve Feng, Min-jae Woo, Hannah Kim, Eunso Kim, Sojung Ki, Lei Shao, Aydogan Ozcan

Author Affiliations +

Proceedings Volume 9699, Optics and Biophotonics in Low-Resource Settings II; 96990T (2016) https://doi.org/10.1117/12.2212310
Event: SPIE BiOS, 2016, San Francisco, California, United States

Abstract

We developed an easy-to-use and widely accessible crowd-sourcing tool for rapidly training humans to perform biomedical image diagnostic tasks and demonstrated this platform’s ability on middle and high school students in South Korea to diagnose malaria infected red-blood-cells (RBCs) using Giemsa-stained thin blood smears imaged under light microscopes. We previously used the same platform (i.e., BioGames) to crowd-source diagnostics of individual RBC images, marking them as malaria positive (infected), negative (uninfected), or questionable (insufficient information for a reliable diagnosis). Using a custom-developed statistical framework, we combined the diagnoses from both expert diagnosticians and the minimally trained human crowd to generate a gold standard library of malaria-infection labels for RBCs. Using this library of labels, we developed a web-based training and educational toolset that provides a quantified score for diagnosticians/users to compare their performance against their peers and view misdiagnosed cells. We have since demonstrated the ability of this platform to quickly train humans without prior training to reach high diagnostic accuracy as compared to expert diagnosticians. Our initial trial group of 55 middle and high school students has collectively played more than 170 hours, each demonstrating significant improvements after only 3 hours of training games, with diagnostic scores that match expert diagnosticians’. Next, through a national-scale educational outreach program in South Korea we recruited >1660 students who demonstrated a similar performance level after 5 hours of training. We plan to further demonstrate this tool’s effectiveness for other diagnostic tasks involving image labeling and aim to provide an easily-accessible and quickly adaptable framework for online training of new diagnosticians.

Conference Presentation

Citation Download Citation

Steve Feng, Min-jae Woo, Hannah Kim, Eunso Kim, Sojung Ki, Lei Shao, and Aydogan Ozcan "A game-based crowdsourcing platform for rapidly training middle and high school students to perform biomedical image analysis", Proc. SPIE 9699, Optics and Biophotonics in Low-Resource Settings II, 96990T (11 March 2016); https://doi.org/10.1117/12.2212310

ACCESS THE FULL ARTICLE