fidoptHVSMR-2.0
Whole-heart 3D cardiovascular MR at 1.5 T of one child or adult with congenital heart disease, cropped around the heart, with its four chambers and four great vessels labelled (HVSMR-2.0).
Read before comparing (8)
- Pool. 60 scans of 60 patients aged under 1 to 52 (mean 11.6) from clinical routine at Boston Children's Hospital, most free-breathing SSFP with a respiratory navigator and ECG gating, many with gadolinium. The provider grades the malformations of 12 as mild, 11 as moderate and 37 as severe; the test cases hold 1, 3 and 6. hvsmr_clinical.csv has no row for pat59, a development case.
- Images. The provider's cropped release, which its own evaluation used: each image cut by hand to a tight box around the heart, at its acquired resolution (0.38 to 1.60 mm), with the scanner's intensities as delivered. The full field of view and the normalised crop are not used.
- Truth. On the 20 scans of HVSMR 2016, trained raters divided a manual blood pool, reviewed by two clinicians, into the eight structures. On the 40 added scans people corrected the map of four networks trained on those 20, 4 to 8 hours a scan. A paediatric cardiologist reviewed all 60, hence the grade semi-automatic. The truth meets the bar of a main leaderboard.
- Protocol. The labels follow the provider's protocol (Table 2 of its paper), not TotalSegmentator's: the left atrium takes the pulmonary veins up to their branching, papillary muscles and trabeculations stay out of the ventricles, and after a Glenn or Fontan operation the pulmonary stump is right ventricle. There is no myocardium label.
- Chambers. They are named by morphology, not side: the anatomic left ventricle lies on the patient's right in 15 scans, 4 of them test cases. Most single ventricles are labelled left ventricle and a common atrium left atrium, the other label left empty, so 4 test cases hold no right atrium.
- Ignore zones. The provider marks optional zones at the ends of the great vessels and pulmonary veins, to be removed before scoring. The corpus zeroes them in both maps for every structure, where the protocol takes each out of its own vessel only, and never shows them to methods. On the test cases they remove a quarter of the venae cavae's labelled volume (16 to 38% a scan), an eighth of the pulmonary artery's (6 to 22%) and 8% of the left atrium's.
- Room. A network trained on the first 20 scans reached a Dice of 0.88 on the mild and moderate scans of the 40 others and 0.65 on the severe ones (Pace et al. 2022). TotalSegmentator's task total_mr makes only the aorta and the inferior vena cava: on one test case they score 0.72 and 0.80.
- Outside numbers. None compare: the corpus draws its own split and scores each scan at its own resolution, where Pace et al. 2022 resized the crops.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 11–2 | KonfAI segmentation app · TotalSegmentator (total_mr), KonfAI appGPUapp2 of 8 structures | 0.363 ±0.247 | 0.280 ±0.194 | 31.94 ±11.38 | 25 | 2.2 | 1.8 | |
| 21–2 | KonfAI segmentation app · TotalSegmentator (total_mr), the authors' packageGPUapp2 of 8 structures | 0.329 ±0.257 | 0.250 ±0.196 | 31.05 ±14.50 | 18 | 12.7 | 2.2 | |
| 3 | KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp2 of 8 structures | 0.000 ±0.000 | 0.000 ±0.000 | 118.27 | 2.4 | 3.5 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- 10 test scans drawn with seed 2026 among the 50 outside development, from the two sources in proportion, 3 of them from the 20 scans of HVSMR 2016, and 10 development scans. The splits of HVSMR 2016 and of Pace et al. 2022 are not used.
- Truth
- contours by a program, verified by people the dataset's eight structures on each scan, the blood pool of the four chambers and of the four great vessels, no myocardium: divided by hand on the 20 scans of HVSMR 2016, a network's map corrected by hand on the 40 others, every scan reviewed by a pediatric cardiologist
- Source
- doi.org/10.6084/m9.figshare.c.7074755.v2
figshare.com/articles/dataset/HVSMR-2_0_cropped_/25226366
doi.org/10.1038/s41597-024-03469-9
doi.org/10.1016/j.media.2022.102469
segchd.csail.mit.edu - Licence
- CC BY 4.0
10 cases for tuning, never tested
pat21, pat20, pat2, pat42, pat33, pat19, pat59, pat53, pat12, pat38