Prostate-Anatomical-Edge-Cases

Radiotherapy simulation CT of the pelvis of a man with prostate cancer, most with an anatomical variant chosen to stress autosegmentation, the prostate, bladder, rectum and femoral heads contoured by one radiation oncologist (Prostate-Anatomical-Edge-Cases).

12 test casespelvisimage CT

Look at the images

Read before comparing (10)

Cohort. 131 men treated by external beam radiotherapy at one institution: 112 edge cases with anatomical variants such as an enlarged prostate, drooping seminal vesicles, a hip arthroplasty, a Foley catheter or a hydrogel spacer, and 19 normal cases. One simulation CT each, on one Philips scanner, in Hounsfield units as delivered. Each patient's variants come from the paper's Supplementary Table 2.
Truth. The clinical contours, drawn by hand by one radiation oncologist with MR for reference where available, peer-reviewed and used for treatment planning. No agreement between readers is reported: graded expert.
Left out. The rectum is empty on AEC-009, 116, 117, 119, 125, 127 and 134 and the prostate on AEC-072, organs the CT shows, which a method would be charged for. AEC-020 and AEC-111 have a femoral head without contours but are not listed as arthroplasties; neither was drawn.
Femoral heads. A head without contours, as on the prosthesis side of test case AEC-037, is absent from the truth: labelling the femur there, as TotalSegmentator does, scores a false positive at 0. The heads stop near the lesser trochanter; the slices below the lowest structure are an ignore region, never shown or scored, so the shafts cost nothing.
Rectum. The clinic's contour, 38 to 81 mm long on the 20 CTs built first, short of the whole rectum: a method that segments its full length is charged for the rest. It differs from the paper's volume on 23 of the 123 patients, smaller on 21 (test CT AEC-081: 34.5 ml against 344.9 ml).
Shared voxels. The prostate and the bladder share 0.07 to 13.9 ml on the 20 CTs built first, most around AEC-023's catheter. The prostate keeps them, so the bladder label is the bladder outside the prostate.
Checks. On the test CTs the femoral heads lie on their sides and the prostate in front of the rectum and below the bladder. TotalSegmentator 2.18 on the normal test case AEC-094 scores a prostate Dice of 0.674 and a bladder Dice of 0.927; it has no rectum, which scores 0.
Room. The paper's commercial tool reached a mean Dice of 0.78 on the normal cohort and 0.12 less on edge cases with several variants; hip prostheses degraded every structure and a catheter the bladder. The surface Dice is at 1 mm, the provider setting no tolerance.
Exposure. Public since May 2023 and listed by the MedSAM dataset list, IMed-361M and SegBook: a model trained on it may have seen these test cases.
Overlap. No descriptor of the corpus names the collection, its authors or their institutions; the corpus's pelvic radiotherapy CTs come from Beaumont Health and the SynthRAD centres.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
11–3	MONAI wholeBody CTGPUapp3 of 5 structures1 failed	0.810 ±0.174	0.599 ±0.149	5.32 ±1.52	20	3.2	15.6	Laptop
21–3	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp4 of 5 structures	0.840 ±0.051	0.568 ±0.073	6.31 ±1.10	35	7.6	7.6	Workstation
31–3	KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp4 of 5 structures	0.839 ±0.051	0.566 ±0.071	6.16 ±1.11		19.4	5.8	Workstation
4	CT-FM whole-body segmentationGPUapp4 of 5 structures	0.719 ±0.124	0.460 ±0.101	23.89 ±22.44	103	16.9	6.9	Workstation

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 12 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it.

2 results not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp4 of 5 structuresolder run	0.840 ±0.051	0.568 ±0.073	6.31 ±1.10	38	4.7	10.8	Laptop
	MONAI wholeBody CTGPUapp3 of 5 structuresfailed on every case	0.000 ±0.000	0.000 ±0.000					Workstation

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10	11	12
MONAI wholeBody CT	failed	0.556	0.352	0.559	0.562	0.560	0.564	0.543	0.510	0.529	0.563	0.535
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.638	0.659	0.483	0.686	0.715	0.710	0.707	0.694	0.687	0.720	0.705	0.659
KonfAI segmentation app · TotalSegmentator (total), the authors' package	0.634	0.655	0.484	0.679	0.718	0.714	0.702	0.694	0.692	0.717	0.701	0.663
CT-FM whole-body segmentation	0.572	0.616	0.467	0.673	0.729	0.679	0.299	0.659	0.656	0.249	0.689	0.612

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test case of Prostate-Anatomical-Edge-Cases

The first test caseClick to look at every case.

Test cases: 12 test CTs drawn with seed 2026 by the provider's cohorts among the 112 patients outside development: 2 of the 15 normal cases, 1 of the 11 hip arthroplasties and 9 of the 97 other edge cases. 11 development CTs, the last of the same seeded order, each cohort in proportion: 1 normal, 1 hip arthroplasty and 9 other edge cases. The pool is 123 patients: eight whose prostate or rectum has no contour are left out.
Truth: contours by experts the clinical structure set, drawn by hand on the simulation CT by one radiation oncologist for all the patients, MR for reference where available, peer-reviewed and used for treatment planning: 1 the prostate, 2 the bladder outside it, 3 the rectum over the length the clinic contours, 4 and 5 the femoral heads down to about the lesser trochanters, none on a side a hip prosthesis replaced; the slices below the lowest structure are the ignore region
Source: www.cancerimagingarchive.net/collection/prostate-anatomical-edge-cases
doi.org/10.7937/QSTF-ST65
doi.org/10.1016/j.phro.2023.100413
pmc.ncbi.nlm.nih.gov/articles/PMC9922913
Licence: CC BY 4.0

11 cases for tuning, never tested

Prostate-AEC-103, Prostate-AEC-045, Prostate-AEC-036, Prostate-AEC-095, Prostate-AEC-074, Prostate-AEC-126, Prostate-AEC-071, Prostate-AEC-063, Prostate-AEC-115, Prostate-AEC-038, Prostate-AEC-057