fidopt Interactive view
Abdomen CT, segmentation

C4KC-KiTS

Late arterial abdominal CT of a patient before the removal of a kidney tumour at the University of Minnesota, with the kidneys and the tumour segmented (C4KC-KiTS on TCIA, KiTS19's training set).

17 test casesabdomenimage CT
Look at the images
Read before comparing (10)
  • Cohort. KiTS19's 300 patients had a partial or radical nephrectomy for a kidney tumour at the University of Minnesota from 2010 to mid-2018; the 210 released with their labels are C4KC-KiTS. More than half of the CTs come from over 50 referring institutions.
  • Truth. Medical students supervised by a urologic oncologist drew contours on about 50 transverse slices a patient; a program set the boundary with fat at -30 HU, filled the hilum and interpolated between slices, and each case was checked against the radiology and pathology reports. Repeated on 30 cases, the process agreed with itself at a Dice of 0.983 for kidney with tumour and 0.923 for tumour.
  • Labels. Label 1 is both kidneys without their tumours but with their cysts and, above -30 HU, the hilum's contents: KiTS19 has no cyst label. The benchmark scores it as kidneys, TotalSegmentator's kidney_left and kidney_right without its kidney cysts. A method that segments the whole kidney with its tumour loses where the tumour is.
  • Corrected cases. TCIA's SEGs hold the labels as KiTS19 froze them for the challenge, so the four cases its authors corrected in December 2019 (KiTS-00015, -00023, -00037 and -00125) are left out. On the 20 cases built first, every label map equals the KiTS19 repository's corrected segmentation voxel for voxel.
  • Images. TCIA's late arterial series in Hounsfield units, slices 0.5 to 5 mm apart and pixels 0.55 to 0.98 mm on the 20 built first. The scanners' padding at -1024 or -2048 HU stays; test case KiTS-00097 also holds about 32,000 voxels between -6573 and -1025 HU.
  • Tumours. The test tumours are mostly small, a median of 17 ml against 35 ml over KiTS19's 300 patients, but include KiTS-00042, a left tumour of 1.4 l, and KiTS-00070, a right one of 451 ml. The development tumours hold 4.6 to 702 ml.
  • Checks. Development case KiTS-00005 has a horseshoe kidney. TotalSegmentator on test case KiTS-00083 scores a kidney Dice of 0.916, an HD95 of 5.3 mm and a surface Dice of 0.803 at 1 mm; it has no tumour class, and its kidneys hold 80% of the tumour's voxels.
  • Room. On KiTS19's test set the winning entry reached a Dice of 0.974 for the kidney with its tumour and 0.851 for the tumour, the teams' means 0.915 and 0.580: the tumour leaves most of the room. The benchmark scores each label apart, so its kidney Dice is not KiTS19's; the surface Dice is at 1 mm, KiTS19 setting no tolerance.
  • Outside numbers. None: KiTS19 ranks its entries on its hidden test set. The test cases are KiTS19 training cases, public with their labels, so a network trained on KiTS19 has seen them.
  • Overlap. With AorticTreeCTA's KiTS19 CTs, named by TCIA patient in its prepare.py, left out of the pool, no other dataset of the corpus holds these CTs.
AccuracyCost
#MethodDiceSurface DiceHD95 mmTime sRAM GBGPU GBImages
11–3KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp1 of 2 structures0.890 ±0.0240.749 ±0.0377.94 ±2.5412.57.5
21–3KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp1 of 2 structures0.890 ±0.0230.739 ±0.0368.18 ±3.3920.54.7
31–3KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp1 of 2 structures0.884 ±0.0290.737 ±0.0369.29 ±5.05253.26.8
4MONAI wholeBody CTGPUapp1 of 2 structures12 failed0.261 ±0.2140.213 ±0.1758.54 ±4.54194.18.4

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 17 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.

1 result not ranked: an older run, cases still missing, or a failure on every case
AccuracyCost
#MethodDiceSurface DiceHD95 mmTime sRAM GBGPU GBMachineImages
MONAI wholeBody CTGPUapp1 of 2 structuresolder run0.894 ±0.0190.734 ±0.0347.00 ±1.65162.912.3Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test case of C4KC-KiTS
The first test caseClick to look at every case.
Test cases
17 test CTs drawn with seed 2026 among the 162 patients outside development, and 16 development CTs, from a pool of 178. Left out: AorticTreeCTA's 20 patients, the 8 C4KC-KiTS patients SAROSBodyCT then held and four cases whose labels were corrected after TCIA's copy. SAROSBodyCT, grown since to 35 C4KC-KiTS patients, holds none of these cases. The labels of KiTS19's 90 test cases are not public.
Truth
contours by a program, verified by people KiTS19's two labels, 1 the kidneys without their tumours, cysts and the hilum's contents included, and 2 the tumours: contours drawn by hand by medical students under a urologic oncologist's supervision on about 50 transverse slices a patient, completed by a program (the boundary with fat at -30 HU, the hilum filled, the slices between interpolated), each case reviewed by a second student and checked against the radiology and pathology reports
Licence
CC BY 3.0 (TCIA) and CC BY-NC-SA 4.0 (the KiTS19 repository)
16 cases for tuning, never tested

KiTS-00164, KiTS-00057, KiTS-00071, KiTS-00155, KiTS-00005, KiTS-00034, KiTS-00000, KiTS-00185, KiTS-00102, KiTS-00130, KiTS-00051, KiTS-00190, KiTS-00065, KiTS-00043, KiTS-00016, KiTS-00131