fidoptData Science Bowl 2018, nuclei of many experiments
One 2D light-microscopy image of stained nuclei, fluorescence or histology, from one biological experiment of the 2018 Data Science Bowl, every nucleus drawn by hand (BBBC038v1).
Read before comparing (10)
- Experiments. The collection names no image's experiment. Its metadata.xlsx gives each experiment's split, image count, size and stain, and within a split, image size and grey or colour reproduce every count (prepare.py --pin). Experiments of one size are one case: michesonn, annedm and three Fraunhofer fish (40 images), nelson and frias (10), the six images of p_18_cc0_public_set1.
- Second stage. Its 15 experiments are held out by the paper, but the table also lists p_16ure_pig, p_17Marinaccio and p_17Moscow among the training images and p_17_forum_nelson_diff in the first-stage test: no development case shares their image size and colour, so bbbc002, bbbc018, p_16ure_human and the colour p_15Stachler images go too.
- Development. jager and mito are first-stage test images, drawn in GIMP like the test; isbi, keller, stachler_bw and eggert_bbbc022 (p_06Eggert and BBBC022, one size) are training images, drawn with superpixels whose edges, the paper says, need not follow the nucleus. Known errors of the training masks (github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes) are not corrected.
- Images. As stored, grey fluorescence written as red, green and blue alike, histology in colour; an alpha channel, 255 everywhere, is left out. 205 x 694 to 1272 x 603 pixels.
- Labels. The solution files' nuclei, one run-length code each, made one label: touching nuclei become one region and the corpus scores no instances.
- Units. The files give no pixel size, and magnifications differ from one experiment to the next: distances are in pixels and the surface tolerance is one pixel.
- Start. On the 10 test images the nuclei cover 20% of an image on average (2 to 36%) and the whole image scores a Dice of 0.31. Otsu's threshold finds most of them: its bright pixels score 0.86 (0.66 to 0.98) on the eight grey fluorescence images, its dark pixels 0.75 and 0.90 on the two colour ones, with a surface Dice of 0.58 at one pixel over the ten. TotalSegmentator has no microscopy task.
- Left out. The second stage's 2,913 unscored images, decoys without truth; the 12 Wikimedia images; the other images of each group.
- Outside numbers. The challenge scored instances (mean average precision over IoU thresholds), which the corpus does not; its winner and every image of both stages are public, so a model trained on BBBC038 may have seen the test cases.
- Overlap. bbbc002, bbbc018 and bbbc022 are crops of BBBC002, BBBC018 and BBBC022, and p_14ISBI of the ISBI 2014 cervical cytology challenge: no dataset of the corpus holds them.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 px | Time s | RAM GB | GPU GB | Images |
| 1 | Otsu nucleiCPUapp | 0.861 ±0.081 | 0.612 ±0.193 | 51.12 ±66.44 | 5 | 0.8 | ||
| 2 | InstanSegCPUapp | 0.371 ±0.267 | 0.316 ±0.247 | 124.46 ±141.56 | 12 | 1.4 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 px, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- The test cases are the second stage's 15 held-out experiments, 106 scored images, in the 10 groups their image size and colour tell apart, one image a group. The development cases are six groups of first-stage experiments that share no image size and colour with a second-stage experiment (p_06Eggert and BBBC022 share one size), one image a group.
- Truth
- contours by experts every nucleus of the image drawn pixel by pixel in GIMP by one of the Broad Institute's expert biologists and reviewed by the others (test and first-stage images), with a superpixel tool on the training images; the corpus merges the instances into one label
- Source
- bbbc.broadinstitute.org/BBBC038
data.broadinstitute.org/bbbc/BBBC038/metadata.xlsx
doi.org/10.1038/s41592-019-0612-7
github.com/carpenterlab/2019_caicedo_dsb - Licence
- CC0 1.0 (the collection's page: 'the various contributors of the imagesets have waived all copyright and related or neighboring rights to BBBC038v1')
6 cases for tuning, never tested
jager, stachler_bw, eggert_bbbc022, isbi, mito, keller