In this task the goal is to determine the gender of the persons depicted in the individual images.
Evaluation Metrics
We propose to use the following evaluation metrics:
- Accuracy (ACC)
- True Positive Rate (TPR)
- True Negative Rate (TNR)
- Area under the Receiver Operator Characteristic Curve (AUC)
Data Format
The label/fold files are structured such that there is one line per image. Each line starts with the filename followed by the fold ID and the gender. Each of the values is separated by a tab. The gender can be either M for male or F for female. An example line looks like the following:
224896_00M25.JPG 0 M
where 224896_00M25.JPG is the filename of the image, 0 is the fold ID, and M shows that the person depicted on the image is of male gender.
Benchmarking Protocol
For evaluating gender classification approaches 5-fold cross-validation shall be used for both conditions. To prevent algorithms from learning the identity of the persons in the training set rather than the gender it has to be made sure that all images of individual subjects are only in one fold at a time. Additionally, the folds are selected in such a way that the distribution of age, gender and ethnicity in the folds is similar to the distribution in the whole database. The file lists for these folds can be found in the Downloads section below.
Controlled Condition
We propose to use the MORPH-II database for the controlled labratory condition for gender classification.
Uncontrolled Condition
For the uncontrolled condition we decided to use the Labeled Faces in the Wild (LFW) dataset.
Downloads
- Complete Evaluation Guidelines
- Folds for controlled condition of gender classification
- Folds for controlled condition of gender classification on balanced dataset
- Folds for uncontrolled condition of gender classification
- Folds for uncontrolled condition of gender classification on balanced dataset
Author
- Tobias Gehrig, Karlsruhe Institute of Technology, Germany
Discussion
If you want to take part in the discussion about benchmarks on gender classification, please feel free to subscribe to the BeFIT-Gender mailinglist.
