fidoptATM22
One chest CT without contrast from the Shanghai Chest Hospital, from healthy people to patients with severe lung disease, with the airway tree labelled down to the small bronchi (ATM'22 challenge, first training batch).
Read before comparing (8)
- Source. ATM'22 (MICCAI 2022) trains on two Zenodo archives of chest CTs from LIDC-IDRI and the Shanghai Chest Hospital. The first holds 149 CTs of the hospital and one of LIDC-IDRI; only the hospital's are used.
- Patients. The provider's table gives no patient identifier for the hospital's CTs, and the paper does not say whether anyone was scanned twice: one CT is taken as one patient, which nothing confirms.
- Truth. Three networks trained on the BAS dataset drew each tree, merged by majority vote, then three radiologists delineated and double-checked it by hand. Graded semi-automatic: a method close to those networks may be favoured. Each scan has one label, and the paper gives no agreement between the radiologists.
- Specks. On 18 of the 20 cases built first, the mask is one tree plus 1 to 96 loose specks of a few voxels, 0.3 % of the airway at most. They are scored as labelled.
- Images. Axial CTs of 0.63 to 0.91 mm in plane and 0.5 mm slices, in Hounsfield units, copied byte for byte with their masks, which lie on the same grid.
- Orientation. As ITK reads the headers, the direction is the identity. On test case ATM_041, TotalSegmentator 2.18 puts the liver on the right and the heart, spleen and stomach on the left, and its lung_airways class (lung_vessels task) reaches a Dice of 0.84 against the tree, precision 0.76, recall 0.95.
- Scoring. The challenge ranks tree length and branches detected, Dice and precision on a prediction's largest connected piece. The corpus scores the whole mask by Dice, HD95 and surface Dice, which see a missed peripheral branch only by its few voxels. The challenge's numbers are on its hidden sets: none compares.
- Overlap. No other dataset of the corpus holds the hospital's CTs. LIDC-IDRI, which other datasets of the corpus are built on, is left out.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | KonfAI segmentation app · TotalSegmentator (lung_vessels), the authors' packageGPUapp | 0.862 ±0.011 | 0.940 ±0.019 | 5.71 ±2.92 | 47.3 | 5.3 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 14 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- 14 test CTs drawn with seed 2026 outside development, then 13 development CTs, the last of the same seeded order, from a pool of 148: the first training batch's 149 Shanghai Chest Hospital CTs less ATM_164, whose label the provider says is misaligned. No CT of LIDC-IDRI or EXACT'09 is used: ATM_172, from LIDC-IDRI, and the whole second batch stay out.
- Truth
- contours by a program, verified by people the airway tree, trachea and bronchi: drawn first by an ensemble of three networks, then delineated and double-checked by hand by three radiologists with more than five years of experience, 60 to 90 minutes a scan (the challenge's paper)
- Source
- zenodo.org/records/7949582
atm22.grand-challenge.org
doi.org/10.1016/j.media.2023.102957
arxiv.org/abs/2303.05745 - Licence
- CC BY 4.0
13 cases for tuning, never tested
ATM_089, ATM_060, ATM_083, ATM_033, ATM_079, ATM_010, ATM_127, ATM_015, ATM_042, ATM_026, ATM_121, ATM_028, ATM_019