fidoptACDC
Cine MR short-axis stack of one patient's heart at end diastole (fixed) and end systole (moving), both ventricular cavities and the myocardium drawn at both phases (ACDC, the MICCAI 2017 Automated Cardiac Diagnosis Challenge).
Read before comparing (7)
- Truth. An experienced user traced the contours on both frames; a cardiologist with 10 years of experience corrected and validated them, then a second from another university hospital (20 years), reaching a consensus where they disagreed. The papillary muscles lie inside the left cavity; the contours stop at the aortic valve. The training and test pages speak instead of 'manual references given by one clinical expert'.
- Slices. The test stacks hold 8 to 20 short-axis slices 10 mm apart (5 mm on patient140 and patient147), with pixels of 1.37 to 1.74 mm. Each label is a 2D drawing on its slice, so a label voxel stands for a slab of 5 to 10 mm.
- Through-plane. The labels check motion within the slices better than across them: at end systole the left cavity loses one to three slices on 8 of the 10 test cases, the right ventricle one or two at the base on all ten. A slice gained or lost weighs a whole slab in the Dice and 5 to 10 mm in the HD95; between slices nothing is checked.
- Volume. The motion depends on the pathology: the left cavity loses 15 % of its volume on patient133, a dilated ventricle, and 82 % on patient142, a hypertrophic one. The myocardium label also changes volume between frames, by -2 % to +30 % (patient138), so a field that matches the drawn wall has to expand it.
- Images. Cine SSFP at 1.5 T or 3 T on two Siemens scanners, University Hospital of Dijon. The two frames are time points of one cine on one grid: end diastole is frame 1 on every test case, end systole frame 9 to 12. The headers disagree, so prepare.py rewrites each file on the geometry ITK reads, no voxel changed. No patient orientation is given.
- Baseline. With no registration the test pairs overlap with a mean Dice of 0.56 a pair (0.45 to 0.79; right ventricle 0.63, myocardium 0.45, left ventricle 0.59) and an HD95 of 12.9 mm (8.0 to 17.7).
- Development. Two normal, one previous myocardial infarction, two dilated, four hypertrophic and one abnormal right ventricle. The training set has no common intensity scale: patient007, patient023 and patient026 are 16-bit integers up to 1415, the seven others, like the test set, 32-bit floats holding whole numbers up to 255. Values are kept as stored.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | multiGradICONGPUapp | 0.817 ±0.029 | +0.259 | 8.90 ±2.47 | 0.031 | 0.00 | 76 | 3.0 | 12.4 | |
| 2 | anatomix + ConvexAdamGPUapp | 0.672 ±0.076 | +0.114 | 9.95 ±1.39 | 0.022 | 0.00 | 9 | 2.1 | 5.5 | |
| 3 | ConvexAdam (MIND)GPUapp9 failed | 0.583 ±0.080 | +0.025 | 12.33 ±2.18 | 0.004 | 0.00 | 7 | 2.0 | 0.5 | |
| 3 | DemonsCPUapp | 0.731 ±0.080 | +0.173 | 10.20 ±2.63 | 0.035 | 0.03 | 1 | 0.2 | ||
| No registration | 0.558 ±0.079 | 12.94 ±1.83 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test pairs drawn with seed 2026 among the 50 patients of the challenge's test set, two from each of its five pathology groups. Seven of them were left out of the draw (draw.held_out): they were the development cases when it was made. The 10 development pairs are the first ten of the reverse seeded order over the challenge's training set (patient001 to patient100, both frames labelled on all).
- Truth
- contours by experts the right ventricular cavity (1), the myocardium (2) and the left ventricular cavity (3), drawn on each short-axis slice of the two frames and checked by two cardiologists
- Source
- humanheart-project.creatis.insa-lyon.fr/database/#collection/637218c173e9f0047faa00fb
www.creatis.insa-lyon.fr/Challenge/acdc/databases.html
www.creatis.insa-lyon.fr/Challenge/acdc/evaluation.html
doi.org/10.1109/TMI.2018.2837502 - Licence
- CC BY-NC-SA 4.0 (the ACDC paper, Bernard et al., IEEE TMI 2018, must be cited)
10 pairs for tuning, never tested
patient099, patient069, patient075, patient023, patient007, patient026, patient039, patient041, patient034, patient009