fidopt Interactive view
Electron microscopy, organelles, segmentation

MitoNet TEM benchmark

A 2D transmission electron micrograph of cells or tissue, its mitochondria segmented by hand: one of the 100 TEM benchmark images MitoNet's authors kept out of their training data (EMPIAR-10982).

10 test casescellimage EM2D, distances in px
Look at the images
Read before comparing (8)
  • Truth. The paper: 'Ground truth segmentations were created manually by two experienced researchers'. It does not say whether each image was drawn by one or both, and gives no agreement between them. The 100 images are in none of the authors' training sets (CEM1.5M, CEM-MitoLab).
  • Sources. The laboratory's own TEM repositories at the National Cancer Institute (65), the Cell Image Library (17), Nanotomy (10) and the WormImage database and WormBook (8); 92 by TEM and 8 by STEM, from human and mouse to C. elegans.
  • Units. The files give no pixel size, so every distance is in pixels and the surface tolerance is one pixel; a pixel is not the same length from one image to the next. The test images are 640 x 451 to 1275 x 898 pixels, 8-bit grey as stored.
  • Labels. The provider numbers each mitochondrion; prepare.py gives every one the label 1, so two that touch become one region and a method is not asked to tell them apart. Mitochondria cover 4 to 32 % of a test image.
  • Samples. Some images share a sample (five of 'H9 cells', three of one clam's hemocytes, four from one Nanotomy publication) and 27 NCI images name none: a test and a development image may show one sample. No test image shares its source page with a development image.
  • Strictness. On the test images the truth grown by two pixels scores a Dice of 0.94 but a surface Dice of 0.27 at one pixel: the surface Dice asks for the outline almost to the pixel. The dark pixels of the image (Otsu) score a Dice of 0.28.
  • Outside numbers. The paper's model reaches a semantic IoU of 0.72 on the 100 images, about a Dice of 0.84; the paper scores instances, which the corpus does not.
  • Left out. The entry's six FIB-SEM volumes: fewer than the ten test cases a main leaderboard needs, labelled otherwise, and Lucchi++ is the EPFL hippocampus data other datasets would hold.

No method has a result on every case of this dataset yet.

Case by case

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test case of MitoNet TEM benchmark
The first test caseClick to look at every case.
Test cases
10 test images drawn with seed 2026 among the 100, from the four sources in proportion: six NCI, two Cell Image Library, one Nanotomy and one WormImage. The 10 development images are the last of the same seeded order, the sources in proportion: six NCI, two Cell Image Library, one Nanotomy and one WormImage; 80 images are unused.
Truth
contours by experts the mitochondria of the image, segmented by hand by two experienced researchers of the authors' laboratory, one mitochondrion an instance; the corpus merges the instances into one label
Licence
CC0 1.0 (EMPIAR's policy for all its data; the MitoNet paper, Conrad and Narayan, Cell Systems 2023, is the reference to cite)
10 cases for tuning, never tested

yn4W2WociugOlXWN83Ps, qOA9ml35XYXQjtWmFvvg, JbZdraW8jUWOkWYenUQW, MTeOXdixYD6o3ysTjtlY, lCH8UM6Mv2AG7zDPrqHA, 1unGsOC0dfeB02LCtQwi, YnjZLcFb5mRSIQR7oTJ1, X1DAYM5MVaMXPhpeeTBT, XsisEi7EmDiq7esbhl3k, qqRmOpdujNXIqEuWrFLk