Update results README to include robustness and real labels tests. Include labels files used in results gen.

This commit is contained in:
Ross Wightman 2020-07-27 14:08:44 -07:00
parent 6c17d57a2c
commit 6be878c22a
3 changed files with 1014 additions and 1 deletions

View File

@ -4,7 +4,7 @@ This folder contains validation results for the models in this collection having
## Datasets
There are currently results for the ImageNet validation set and 3 additional test sets.
There are currently results for the ImageNet validation set and 5 additional test/label sets.
### ImageNet Validation - [`results-imagenet.csv`](results-imagenet.csv)
@ -34,5 +34,17 @@ An ImageNet test set of 10,000 images sampled from new images roughly 10 years a
A collection of 7500 images covering 200 of the 1000 ImageNet classes. Images are naturally occuring adversarial examples that confuse typical ImageNet classifiers. This is a challenging dataset, your typical ResNet-50 will score 0% top-1.
### ImageNet-Robustness - [`results-imagenet-r.csv`](results-imagenet-r.csv)
* Source: https://github.com/hendrycks/imagenet-r
* Paper: "The Many Faces of Robustness" - https://arxiv.org/abs/2006.16241
Renditions of 200 ImageNet classes resulting in 30,000 images for testing robustness.
### ImageNet-"Real Labels" - [`results-imagenet-real.csv`](results-imagenet-real.csv)
* Source: https://github.com/google-research/reassessed-imagenet
* Paper: "Are we done with ImageNet?" - https://arxiv.org/abs/2006.07159
## TODO
* Explore adding a reduced version of ImageNet-C (Corruptions) and ImageNet-P (Perturbations) from https://github.com/hendrycks/robustness. The originals are huge and image size specific.

File diff suppressed because it is too large Load Diff

1
results/real.json Normal file

File diff suppressed because one or more lines are too long