fidoptLNDb
Chest CT of an adult from the São João University Hospital Centre in Porto, with the lung nodules of 3 mm or more segmented by one radiologist (LNDb, the training set of the LNDb challenge).
Read before comparing (9)
- Cohort. Chest CTs collected retrospectively at the Centro Hospitalar e Universitário de São João (CHUSJ), Porto, from 2016 to 2018, one a patient, on three Siemens scanners. Following LIDC-IDRI: adults without a known history of cancer, no intravenous contrast, slices of 1 mm or less; CTs with another lung disease, noise, motion, artefacts, more than six nodules or a nodule wider than 30 mm left out.
- Reading. Five radiologists of the hospital read the CTs, one to three a CT, each once and blind to the others, with no review or consensus. Each painted every nodule of 3 mm or more with a brush on the axial slices and marked smaller nodules and non-nodules by their centre alone.
- Truth. Label 1 is one radiologist's painted nodules: when several read the CT, the first in the dataset's numbering who painted one, a rule that looks at no outline. A case's Dice pools its nodules, so the largest weighs most. The ten test truths drawn first hold 22 nodules of 15 to 383 mm³; the ten added later, larger, 36 connected parts of 8 to 1,866 mm³.
- Left out. Nodules under 3 mm, non-nodules and nodules that only the CT's other radiologists painted are not in the truth: a method that segments one is charged with it. LNDb's radiologists often find different nodules: the paper's detection agreement between two of them is 0.23 to 0.40 (its table II).
- Spread. Against the truth, the other radiologists' maps on the six of the ten test CTs drawn first that were read more than once: three of seven empty, the four others at a Dice of 0.01, 0.20, 0.64 and 0.86, an HD95 of 0.7 mm on LNDb-0264, where both painted one nodule, and 114 to 227 mm elsewhere. No method should be expected to beat another radiologist; HD95 mostly counts nodules found or missed.
- Images. The archives' MetaImage files written as NIfTI, untouched, in 16-bit Hounsfield units. The Somatom go.Up's CTs pad the field with -8192 HU; values reach 23439 on LNDb-0280 and -28,165 to 28,598 outside LNDb-0154's reconstructed field. Some CTs are wider than 512 pixels, and some have axes turned from the world's, by 1 to 5 degrees on the go.Up's.
- TotalSegmentator. Version 2.18 (task lung_nodules) on two test cases: on LNDb-0099 a Dice of 0.57 and an HD95 of 3.5 mm, its two nodules inside the truth's two but drawn smaller; on LNDb-0215 a Dice of 0.008. On neither does it find a nodule the truth lacks.
- Outside numbers. None: LNDb's sub-challenge B gave the nodule centres and scored each outline against each radiologist's. LNDb sets no surface tolerance, so the surface Dice is at 1 mm, one slice. The test CTs are public with their masks, so a model trained on LNDb has seen them; TotalSegmentator's README does not say whether its lung_nodules model, trained partly on LIDC-IDRI, saw LNDb.
- Overlap. No other dataset of the corpus holds these CTs: LNDb's come from CHUSJ alone, and no descriptor names LNDb or CHUSJ as a source.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | KonfAI segmentation app · TotalSegmentator (lung_nodules), the authors' packageGPUapp | 0.276 ±0.116 | 0.353 ±0.133 | 127.46 ±40.11 | 30.3 | 5.7 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 20 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- 20 test CTs drawn with seed 2026 among the 191 outside development, then 18 development CTs, the last of the same seeded order. The pool is the 209 of LNDb's 236 training CTs in which a radiologist painted a nodule; the 27 others hold only centres or nothing. The masks of LNDb's 58 test CTs are not public.
- Truth
- contours by experts the lung nodules of 3 mm or more that one radiologist painted with a brush on the axial slices, reading the CT once, blind to the others and without review: of the CT's one to three radiologists, the first in the dataset's numbering who painted a nodule
- Source
- zenodo.org/records/8348419
arxiv.org/abs/1911.08434
doi.org/10.1016/j.media.2021.102027
lndb.grand-challenge.org/Evaluation
lndb.grand-challenge.org/Rules
doi.org/10.1038/s41597-024-03345-6 - Licence
- CC BY-NC-ND 4.0 (the record's description, its README and the challenge's rules); the record's licence field says CC BY 4.0
18 cases for tuning, never tested
LNDb-0171, LNDb-0016, LNDb-0121, LNDb-0049, LNDb-0034, LNDb-0109, LNDb-0112, LNDb-0258, LNDb-0143, LNDb-0280, LNDb-0020, LNDb-0003, LNDb-0251, LNDb-0265, LNDb-0188, LNDb-0268, LNDb-0055, LNDb-0174