Background

Although considerable efforts are being made to automate image processing and analysis through machine learning and other computational approaches (Gulshan et al., 2016; Janowczyk and Madabhushi, 2016; Esteva et al., 2017; Bychkov et al., 2018), many biomedical subfields currently require expert visual image classification due to the complexity of the phenotypes being studied. Furthermore, machine learning approaches require many examples to train, so developing automated approaches to score newly discovered or rare phenotypes may take time. Examples of expert image scoring include but are not limited to: medical diagnostic scores for tumors and other pathologies (Dhyani et al., 2015; Fuchs et al., 2018), veterinary applications (Barton et al., 2018), and research that requires quantification of physiological and cellular morphologies (Passeri et al., 2009; Bretman et al., 2010; Green et al., 2011; Gonzalez-Hunt et al., 2014; Riley et al., 2016).

Expert image scoring requires that the experimenter be blinded to the treatment group in order to avoid unintentional bias in the qualitative visual scoring. Researchers commonly achieve this in one of three ways. First, some labs choose to have multiple experts involved in the preparation of slides/specimens, image acquisition, and image processing/scoring so that treatment groups are not identifiable between tasks (Bretman et al., 2010; Green et al., 2011). A second approach is to have a single experimenter prepare specimens, acquire images, and score images, but have a second experimenter manually rename images to blind the scoring to treatment group. A third approach is to create an in-house command line script to blind the image names to avoid the manual manipulation of file names (Riley et al., 2016). All three of these approaches ultimately can lead to a robust and unbiased classification of images, but are labor-intensive, time-consuming, and can lead to errors in assignment of scores.

In this protocol, we introduce a simple, freely available software application we developed to allow the user to load unblinded images, be shown those images in a randomized order, easily assign scores, and receive summarized results in a readily exportable file format. All of these steps are carried out within the software using a simple graphical user interface. The software also includes an optional Quality Control mechanism to help identify scoring inconsistencies. We have recently reported exercise effects on mitochondrial morphology in body wall muscle of the nematode Caenorhabditis elegans that were classified by blind visual categorical scoring using this software (Hartman et al., 2018). Herein, we describe the basic methods for using the software as well as caveats and considerations that should be taken into account when using blind visual scoring of images.