fidoptCT-ORG
Abdomen-pelvis, chest-abdomen-pelvis or whole-body CT of a patient with lesions in the liver or another labelled organ, with the liver, bladder, lungs, kidneys, bones and brain segmented (CT-ORG).
10 test caseswhole bodyimage CT
Read before comparing (9)
- Pool. 140 CTs of 140 patients, 131 from the LiTS challenge's sites and 9 PET-CTs from Stanford Healthcare, included for lesions in a labelled organ, mostly the liver. Eight test cases are abdomen-pelvis scans that stop in the lower chest and two run from the pelvis to the lung apices; the lungs are whole on case_13 and case_15 only.
- Truth. A graduate student made or oversaw the liver, kidneys, bladder and brain on every scan in ITK-SNAP, by active contours then hand correction. On the 21 test scans the lungs were made the same way and the bones began as a morphological algorithm's output, corrected by hand; a board-certified radiologist reviewed all 21 and 9 were refined. Graded semi-automatic.
- Development. The training scans' lungs and bones are the algorithm's alone, which the README says 'should not be considered reliable for evaluation': fit for tuning only. The lungs are air below -150 HU, trachea included, and the bones miss the ribs an abdominal scan cuts off before the spine; on case_114 the lungs hold a 70 ml piece of bowel gas.
- Sources. The README's paragraph on the splits says the test scans' organs were labelled with its automatic method B and their bones with A then B; its own description and the data paper say the opposite, and the corpus follows the paper.
- Labels. 1 liver, 2 bladder, 3 lungs, 4 kidneys, 5 bone, 6 brain, which no case built shows; a lesion inside the liver is liver. The lungs hold the trachea on some test cases (case_13) and not others (case_15). The bone label, the skeleton with its marrow, matches no TotalSegmentator structure: a method scores on it only by naming a label bone.
- Images. The collection's files as delivered. On case_10 and case_15 the header's scaling caps the image at 1602 and 2976 HU, and case_0 reads -3024 outside the reconstructed field. TCIA warns that some volumes appear left-right flipped, but each header tells its voxel order: the liver lies 48 to 94 mm right of the body's centre on all 20 cases built.
- TotalSegmentator. Its task total on case_13 scores a Dice of 0.968 on the liver, 0.943 on the bladder, 0.978 on the lungs and 0.957 on the kidneys, and 0 on the bone, a name it does not write, so its line reads 0.769 over five structures.
- Overlap. No other dataset of the corpus holds these scans: none is built from LiTS, the decathlon's liver task or 3D-IRCADb-01. A dataset built from those later would share CT-ORG's scans.
- Outside numbers. None compare: the data paper scores all 21 test scans at 3 mm, the corpus ten at their own resolution. The test scans are public with their labels and in the training collections of VISTA3D, SAT, SegVol and SAM-Med3D, and models trained on LiTS or the decathlon's liver task hold its LiTS scans: such models may have seen the test cases.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | lungmask · lobesGPUapp1 of 6 structures | 0.975 ±0.007 | 0.933 ±0.025 | 2.04 ±1.83 | 281 | 6.7 | 5.3 | |
| 22–3 | KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp | 0.933 ±0.014 | 0.780 ±0.031 | 6.46 ±3.26 | 22 | 2.9 | 6.2 | |
| 32–3 | KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp | 0.931 ±0.015 | 0.775 ±0.031 | 5.36 ±2.65 | 27.3 | 5.7 | ||
| 44–5 | KonfAI segmentation app · IMPACTSeg body, KonfAI appGPUapp2 of 6 structures | 0.859 ±0.012 | 0.522 ±0.029 | 15.20 ±3.71 | 6 | 1.8 | 7.2 | |
| 54–5 | CT-FM whole-body segmentationGPUapp | 0.764 ±0.147 | 0.605 ±0.142 | 29.01 ±17.35 | 214 | 14.8 | 7.2 | |
| 66–7 | MONAI wholeBody CTGPUapp9 failed | 0.089 ±0.201 | 0.070 ±0.158 | 8.27 | 18 | 3.7 | 8.9 | |
| 76–7 | KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp4 of 6 structures | 0.675 ±0.076 | 0.436 ±0.060 | 78.98 ±14.30 | 15.3 | 8.8 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
2 results not ranked: an older run, cases still missing, or a failure on every case
| Accuracy | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Machine | Images |
| lungmask · lobesGPUapp1 of 6 structuresolder run | 0.975 ±0.007 | 0.933 ±0.025 | 2.04 ±1.83 | 455 | 4.0 | 5.0 | Laptop | ||
| MONAI wholeBody CTGPUappolder run | 0.926 ±0.016 | 0.758 ±0.034 | 6.37 ±2.79 | 36 | 3.8 | 17.6 | Laptop | ||
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
The first test caseClick to look at every case.
- Test cases
- 10 test cases drawn with seed 2026 among the provider's 21 test scans (case_0 to case_20), whose labels were all drawn or corrected by hand; 10 development cases among its training scans (case_21 to case_139). Training scans 28 to 47, 3D-IRCADb-01's 20, are left out: their headers lost the pixel size and slice spacing, and IRCAD's page puts them under CC BY-NC-ND 4.0.
- Truth
- contours by a program, verified by people the dataset's six labels: the liver, the bladder, the lungs, the kidneys (both in one label), the bones (the skeleton in one label) and the brain where a scan shows it; on the test cases all drawn or corrected by hand in ITK-SNAP and reviewed by a radiologist, on the development cases the lungs and the bones made by a morphological algorithm alone
- Source
- www.cancerimagingarchive.net/collection/ct-org
doi.org/10.7937/tcia.2019.tt7f4v7o
www.cancerimagingarchive.net/wp-content/uploads/ctorg_README.txt
doi.org/10.1038/s41597-020-00715-8
github.com/bbrister/ctOrganSegmentation
arxiv.org/abs/1901.04056
www.ircad.fr/research/data-sets/liver-segmentation-3d-ircadb-01 - Licence
- CC BY 3.0
10 cases for tuning, never tested
case_48, case_63, case_24, case_97, case_99, case_75, case_114, case_122, case_69, case_25