fidoptCAMUS
2D echocardiography of one patient: an apical two- or four-chamber view at end diastole (fixed) and end systole (moving), with the left ventricular cavity, myocardium and left atrium drawn by a cardiologist on both (CAMUS).
Read before comparing (9)
- Truth. One cardiologist chose the frames and drew all 500 patients; the portal gives only these contours. On a fold's 40 good and medium patients, two cardiologists agree at a Dice of 0.82 to 0.92 on the cavity and 0.88 to 0.94 inside the epicardium, 2.2 to 4.0 mm apart on average; the first, drawing twice, at 0.93 to 0.96. No figure is given for the atrium or the myocardium alone.
- Labels. 1 is the left ventricular cavity, trabeculae and papillary muscles included, closed at the mitral valve plane; 2 the myocardium; 3 the left atrium. The right heart, the valves and the rest of the sector are unlabelled: the board measures the left heart.
- Sector. The acquisitions centre on the left ventricle, so the sector's far edge often cuts the atrium, whose label ends there. Where the wall lay outside the sector the cardiologist continued the epicardium: on patient0047_2CH, patient0197_2CH, patient0214_4CH and patient0215_2CH, 10 to 35 % of the end-diastolic myocardium label lies where both images are black and nothing guides a method.
- Motion. Between the frames the cavity label loses 24 to 61 % of its area on the test pairs, the atrium label gains up to 173 % as it fills, and the myocardium label changes by -19 to +13 %: a field that matches the drawn wall must grow or shrink it in the plane.
- Baseline. With no registration the three structures overlap at a mean Dice of 0.73 a pair (0.58 to 0.85; cavity 0.75, myocardium 0.66, atrium 0.76), with an HD95 of 9.9 mm (5.8 to 13.4). On patient0047_2CH (0.85), patient0194_4CH (0.81), patient0214_4CH and patient0261_2CH (0.80), what is left is close to two cardiologists' disagreement.
- Images. Routine scans of 500 patients at the University Hospital of St Etienne on GE Vivid E95 scanners, half with an ejection fraction under 45 %. Both frames, where the left ventricle looked largest and smallest, come from one cine on one grid, used as delivered. The portal's square pixels of 0.308 mm are not the grid the paper describes; how they were made is not said.
- Pool. The provider's test split of 50 patients, so a network trained on its training and validation splits has not seen a test pair. A patient's case is the first of their views not rated poor, in a seeded hash order; the 5 patients with two poor views give none. The test pairs: three two-chamber good, three two-chamber medium, two four-chamber good, two four-chamber medium.
- Overlap. The portal's TED collection holds the four-chamber full cycles of 98 CAMUS patients under its own numbering, no file naming a CAMUS patient: a TED dataset would have to leave out these cases' images.
- Outside numbers. None: the challenge ranks segmentation, not registration.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | vismatch matchers · RoMaGPU | 0.758 ±0.061 | +0.033 | 9.03 ±2.05 | 0.014 | 0.00 | 12 | 3.2 | 6.9 | |
| 22–6 | vismatch matchers · LoFTRGPU | 0.726 ±0.062 | +0.001 | 9.78 ±2.04 | 0.002 | 0.00 | 3 | 2.0 | 1.4 | |
| 22–6 | vismatch matchers · XFeatGPU | 0.741 ±0.066 | +0.016 | 9.30 ±1.97 | 0.019 | 0.00 | 3 | 2.2 | 0.4 | |
| 42–6 | vismatch matchers · SIFT + LightGlueGPU | 0.713 ±0.086 | −0.012 | 10.31 ±2.40 | 0.027 | 0.00 | 3 | 2.0 | 0.5 | |
| 52–6 | vismatch matchers · DISK + LightGlueGPU | 0.680 ±0.103 | −0.045 | 12.56 ±3.34 | 0.073 | 0.00 | 3 | 2.1 | 3.1 | |
| 62–6 | vismatch matchers · ALIKED + LightGlueGPU | 0.663 ±0.098 | −0.062 | 14.00 ±6.26 | 0.096 | 0.00 | 3 | 2.3 | 1.9 | |
| No registration | 0.725 ±0.063 | 9.87 ±2.05 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test pairs drawn with seed 2026 from the 45 views of the provider's test split, one view a patient, in proportion to view and image quality; ten views once used for development are held out. 10 development pairs come from the 385 views of the provider's training split, one a patient, so they share no patient with the test pairs. Poor-quality views are left out of both pools.
- Truth
- contours by experts the left ventricular cavity (1), the myocardium (2) and the left atrium (3), drawn by one cardiologist on the end-diastolic and end-systolic frames of each view
- Source
- humanheart-project.creatis.insa-lyon.fr/database/#collection/6373703d73e9f0047faa1bc8
www.creatis.insa-lyon.fr/Challenge/camus/databases.html
www.creatis.insa-lyon.fr/Challenge/camus/evaluation.html
doi.org/10.1109/TMI.2019.2900516
arxiv.org/abs/1908.06948 - Licence
- CC BY-NC-SA 4.0 (the CAMUS paper, Leclerc et al., IEEE TMI 2019, must be cited)
10 pairs for tuning, never tested
patient0498_4CH, patient0079_2CH, patient0454_2CH, patient0344_2CH, patient0400_2CH, patient0195_4CH, patient0411_4CH, patient0013_2CH, patient0470_4CH, patient0005_4CH