144 lines
4.2 KiB
Markdown
144 lines
4.2 KiB
Markdown
|
# Log Collector
|
|||
|
|
|
|||
|
|
## Function
|
|||
|
|
|
|||
|
|
Automatically collect logs and write the result in a json file or markdown file.
|
|||
|
|
|
|||
|
|
If there are several `.log.json` files in one folder, Log Collector assumes that the `.log.json` files other than the first one are resume from the preceding `.log.json` file. Log Collector returns the result considering all `.log.json` files.
|
|||
|
|
|
|||
|
|
## Usage:
|
|||
|
|
|
|||
|
|
To use log collector, you need to write a config file to configure the log collector first.
|
|||
|
|
|
|||
|
|
For example:
|
|||
|
|
|
|||
|
|
example_config.py:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# The work directory that contains folders that contains .log.json files.
|
|||
|
|
work_dir = '../../work_dirs'
|
|||
|
|
# The metric used to find the best evaluation.
|
|||
|
|
metric = 'mIoU'
|
|||
|
|
|
|||
|
|
# **Don't specify the log_items and ignore_keywords at the same time.**
|
|||
|
|
# Specify the log files we would like to collect in `log_items`.
|
|||
|
|
# The folders specified should be the subdirectories of `work_dir`.
|
|||
|
|
log_items = [
|
|||
|
|
'segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup',
|
|||
|
|
'segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr',
|
|||
|
|
'segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr',
|
|||
|
|
'segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr'
|
|||
|
|
]
|
|||
|
|
# Or specify `ignore_keywords`. The folders whose name contain one
|
|||
|
|
# of the keywords in the `ignore_keywords` list(e.g., `'segformer'`)
|
|||
|
|
# won't be collected.
|
|||
|
|
# ignore_keywords = ['segformer']
|
|||
|
|
|
|||
|
|
# Other log items in .log.json that you want to collect.
|
|||
|
|
# should not include metric.
|
|||
|
|
other_info_keys = ["mAcc"]
|
|||
|
|
# The output markdown file's name.
|
|||
|
|
markdown_file ='markdowns/lr_in_trans.json.md'
|
|||
|
|
# The output json file's name. (optional)
|
|||
|
|
json_file = 'jsons/trans_in_cnn.json'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The structure of the work-dir directory should be like：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
├── work-dir
|
|||
|
|
│ ├── folder1
|
|||
|
|
│ │ ├── time1.log.json
|
|||
|
|
│ │ ├── time2.log.json
|
|||
|
|
│ │ ├── time3.log.json
|
|||
|
|
│ │ ├── time4.log.json
|
|||
|
|
│ ├── folder2
|
|||
|
|
│ │ ├── time5.log.json
|
|||
|
|
│ │ ├── time6.log.json
|
|||
|
|
│ │ ├── time7.log.json
|
|||
|
|
│ │ ├── time8.log.json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Then , cd to the log collector folder.
|
|||
|
|
|
|||
|
|
Now you can run log_collector.py by using command:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python log_collector.py ./example_config.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The output markdown file is like:
|
|||
|
|
|
|||
|
|
|exp_num|method|mIoU best|best index|mIoU last|last index|last iter num|
|
|||
|
|
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
|||
|
|
|1|segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup|0.2776|10|0.2776|10|160000|
|
|||
|
|
|2|segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr|0.2802|10|0.2802|10|160000|
|
|||
|
|
|3|segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr|0.4943|11|0.4943|11|160000|
|
|||
|
|
|4|segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr|0.4883|11|0.4883|11|160000|
|
|||
|
|
|
|||
|
|
The output json file is like:
|
|||
|
|
```json
|
|||
|
|
[
|
|||
|
|
{
|
|||
|
|
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup",
|
|||
|
|
"metric_used": "mIoU",
|
|||
|
|
"last_iter": 160000,
|
|||
|
|
"last eval": {
|
|||
|
|
"eval_index": 10,
|
|||
|
|
"mIoU": 0.2776,
|
|||
|
|
"mAcc": 0.3779
|
|||
|
|
},
|
|||
|
|
"best eval": {
|
|||
|
|
"eval_index": 10,
|
|||
|
|
"mIoU": 0.2776,
|
|||
|
|
"mAcc": 0.3779
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr",
|
|||
|
|
"metric_used": "mIoU",
|
|||
|
|
"last_iter": 160000,
|
|||
|
|
"last eval": {
|
|||
|
|
"eval_index": 10,
|
|||
|
|
"mIoU": 0.2802,
|
|||
|
|
"mAcc": 0.3764
|
|||
|
|
},
|
|||
|
|
"best eval": {
|
|||
|
|
"eval_index": 10,
|
|||
|
|
"mIoU": 0.2802,
|
|||
|
|
"mAcc": 0.3764
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"method": "segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr",
|
|||
|
|
"metric_used": "mIoU",
|
|||
|
|
"last_iter": 160000,
|
|||
|
|
"last eval": {
|
|||
|
|
"eval_index": 11,
|
|||
|
|
"mIoU": 0.4943,
|
|||
|
|
"mAcc": 0.6097
|
|||
|
|
},
|
|||
|
|
"best eval": {
|
|||
|
|
"eval_index": 11,
|
|||
|
|
"mIoU": 0.4943,
|
|||
|
|
"mAcc": 0.6097
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"method": "segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr",
|
|||
|
|
"metric_used": "mIoU",
|
|||
|
|
"last_iter": 160000,
|
|||
|
|
"last eval": {
|
|||
|
|
"eval_index": 11,
|
|||
|
|
"mIoU": 0.4883,
|
|||
|
|
"mAcc": 0.6061
|
|||
|
|
},
|
|||
|
|
"best eval": {
|
|||
|
|
"eval_index": 11,
|
|||
|
|
"mIoU": 0.4883,
|
|||
|
|
"mAcc": 0.6061
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
]
|
|||
|
|
```
|