fidoptTotalSegmentator CT
Routine clinical CT of any region of the body from the University Hospital Basel, with the 117 structures of TotalSegmentator's task total labelled (TotalSegmentator dataset 2.0.1).
83 test caseswhole bodyimage CT
Read before comparing (11)
- Images. CTs drawn at random from the Basel archive, of any region, phase and pathology, resampled to 1.5 mm, the face blurred, in Hounsfield units. GE CTs such as s1119 hold -3024 outside the reconstructed circle; the top slices of s0802 hold the scanner's padding, -8192, with 249 voxels of the left humerus' mask on it. No mask is given.
- Truth. Public models and an nnU-Net retrained as the work went drafted the masks, refined by hand until every CT was reviewed and corrected where needed, under two physicians. Version 2 corrected some classes and added 19 without saying how they were drawn; how the 24 CTs it added to the test split, s1406 to s1429, were annotated is not described. Graded semi-automatic: no agreement is given.
- Known flaws. The provider says a small part of each rib at the spine is always missing, the ends of the ribs at the costal cartilages are not properly defined, and colon and small bowel are sometimes impossible to tell apart.
- TotalSegmentator. Its task total held the 89 test CTs out, but the truth began as networks' predictions corrected where wrong, so it shares their boundaries: on test CT s1176 it reaches a median Dice of 0.98. Its line here is not a fair comparison. Whether total_v3 and models such as VISTA3D, SAT, SegVol and MedSAM held the test split out is theirs to say.
- Overlaps. The 117 masks overlap at some borders, 0.04 to 0.6 % of a CT's labelled voxels, most often the heart and the inferior vena cava; prepare.py gives such a voxel to the later class, as the provider's nnU-Net conversion does.
- Left out. Five test CTs hold a mask piece of 1 mL or more inside another organ's mask: the heart of s1121 in the liver, the left lower lobe of s1412 and s1413 in the spleen, the small bowel of s1414 in the left kidney, the stomach of s1426 in the colon.
- Screening. Every case kept was screened the same way. The pieces left lie where two structures meet, are a kidney cyst in its kidney, or are small, the largest misplaced one 0.5 mL of the left lower lobe in s0802's stomach; on s0308 two pieces of the small bowel lie 69 and 88 % inside the colon at their junction, kept.
- Field of view. Each CT holds the structures it covers, cut where it ends: 5 to 114 of the 117 on the test CTs, 5 on s1406.
- Leakage. The release holds 1,204 scans of 1,134 subjects, not one CT a patient: a subject's two scans may fall on both sides of the provider's split. A model trained on the training and validation splits has seen no test CT; TotalSegmentator, and any model trained on the training split, has seen the development CTs.
- Tolerance. The surface Dice is at 3 mm, that of the provider's own evaluation.
- Overlap. No other dataset of the corpus holds these CTs; Basel's MR images, other images, are in TotalSegMRI.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp | 0.928 ±0.009 | 0.961 ±0.008 | 2.82 ±0.80 | 85 | 20.5 | 4.3 | |
| 2 | KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp | 0.867 ±0.024 | 0.916 ±0.019 | 4.86 ±1.21 | 29 | 3.1 | 5.5 | |
| 3 | KonfAI segmentation app · IMPACTSeg body, KonfAI appGPUapp2 of 117 structures | 0.789 ±0.028 | 0.905 ±0.034 | 8.34 ±1.83 | 25 | 2.0 | 3.3 | |
| 4 | lungmask · lobesGPUapp5 of 117 structures | 0.657 ±0.083 | 0.632 ±0.080 | 32.36 ±34.00 | 48 | 3.4 | 4.4 | |
| 5 | KonfAI segmentation app · TotalSegmentator (kidney_cysts), the authors' packageGPUapp2 of 117 structures | 0.058 ±0.069 | 0.065 ±0.082 | 19.75 ±55.06 | 27 | 14.1 | 2.2 | |
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 83 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
4 results not ranked: an older run, cases still missing, or a failure on every case
| Accuracy | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Machine | Images |
| KonfAI segmentation app · IMPACTSeg body, KonfAI appGPUapp2 of 117 structuresolder run | 0.789 ±0.028 | 0.905 ±0.034 | 8.33 ±1.83 | 15 | 1.8 | 5.5 | Laptop | ||
| KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUappolder run | 0.928 ±0.009 | 0.961 ±0.008 | 2.82 ±0.80 | 22.6 | 4.8 | Laptop | |||
| KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp1 failedolder run | 0.856 ±0.031 | 0.905 ±0.029 | 4.88 ±1.22 | 28 | 2.7 | 9.3 | Laptop | ||
| lungmask · lobesGPUapp5 of 117 structuresolder run | 0.657 ±0.083 | 0.632 ±0.080 | 32.36 ±34.00 | 3.0 | 5.5 | Laptop | |||
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
The first test caseClick to look at every case.
- Test cases
- The provider's test split taken whole but for six CTs: 83 of meta.csv's 89, five left out for a broken truth and s0735 for a mask off its image's grid. 30 development CTs, the last of its training split in the seeded order.
- Truth
- contours by a program, verified by people the dataset's masks of the 117 structures of TotalSegmentator's task total on each CT, merged into one map in its numbering (a voxel two masks claim, 0.04 to 0.6 % of a CT's labelled voxels, goes to the later class, as the provider's own conversion does); a test CT holds 5 to 114 of them, those of its field of view
- Source
- zenodo.org/records/10047292
doi.org/10.1148/ryai.230024
arxiv.org/abs/2208.05868
github.com/wasserth/TotalSegmentator/blob/master/resources/improvements_in_v2.md
github.com/wasserth/TotalSegmentator/blob/master/resources/convert_dataset_to_nnunet.py
github.com/wasserth/TotalSegmentator/blob/master/resources/evaluate.py
zenodo.org/records/6802614 - Licence
- CC BY 4.0
30 cases for tuning, never tested
s1038, s0170, s1358, s0492, s0250, s1252, s1001, s0383, s0495, s0248, s0346, s0194, s1146, s0109, s1147, s0330, s0670, s0370, s0891, s0896, s1189, s0595, s0804, s0530, s0636, s0601, s0647, s0863, s0228, s0576