# Evaluation

Home | Details | Rules | Important Dates | Registration | Download | Evaluation | Submission | Results | Contact | Organisers | Q&A | Program | Presentations

Each team will be assigned three ranking numbers based on the three criteria below, one ranking number per criterion, using a standard competition ranking. The sum of these numbers will be used for the final ranking. Matlab code for calculating the evaluation metrics is available here.

### Detection

The ground truth for each segmented object is the object in the manual annotation that has maximum overlap with that segmented object.

A segmented glandular object that intersects with at least 50% of its ground truth will be considered as true positive, otherwise it will be considered as false positive. A ground truth glandular object that has no corresponding segmented object or has less than 50% of its area overlapped by its corresponding segmented object will be considered as false negative.

Let

- be the number of true positives,
- be the number of false positives,
- be the number of false negatives.

A metric for gland detection is the F1-score, defined by

where

,

### Segmentation

Given a set of pixels annotated as a ground truth obect and a set of pixels segmented as a glandular object, Dice index is defined as follows

Further, let

- denote a set of ground truth objects in image .
- denote a set of segmented objects in image .
- denote the th segmented object in image .
- denote a ground truth object that maximally overlaps in image .
- denote the th ground truth object in image .
- denote a segmented object that maximally overlaps in image .
- a set of all ground truth objects.
- a set of all segmented objects.
- denote the total number of segmented objects in .
- denote the total number of ground truth objects in .

We define the object-level Dice index as

where

The object-level Dice index will be used to evaluate the performance of segmentation.

### Shape Similarity

Let denote a set of pixels annotated as ground truth and denote a set of pixels segmented as glandular objects. A Hausdorff distance between and is defined as

.

Now, let

- denote a set of ground truth objects in image .
- denote a set of segmented objects in image .
- denote the th segmented object in image .
- denote a ground truth object that maximally overlaps in image . If there is no ground truth object overlapping , is defined as ground truth object that has the minimum Hausdorff distance from .
- denote the th ground truth object in image .
- denote a segmented object that maximally overlaps in image . If there is no segmented object overlapping , is defined as segmented object that has the minimum Hausdorff distance from .
- a set of all ground truth objects.
- a set of all segmented objects.
- denote the total number of segmented objects in .
- denote the total number of ground truth objects in .

We measure the shape similarity between all segmented objects in and all ground truth objects in using the object-level Hausdorff distance

where