144 lines
4.2 KiB
Markdown
144 lines
4.2 KiB
Markdown
# Log Collector
|
||
|
||
## Function
|
||
|
||
Automatically collect logs and write the result in a json file or markdown file.
|
||
|
||
If there are several `.log.json` files in one folder, Log Collector assumes that the `.log.json` files other than the first one are resume from the preceding `.log.json` file. Log Collector returns the result considering all `.log.json` files.
|
||
|
||
## Usage:
|
||
|
||
To use log collector, you need to write a config file to configure the log collector first.
|
||
|
||
For example:
|
||
|
||
example_config.py:
|
||
|
||
```python
|
||
# The work directory that contains folders that contains .log.json files.
|
||
work_dir = '../../work_dirs'
|
||
# The metric used to find the best evaluation.
|
||
metric = 'mIoU'
|
||
|
||
# **Don't specify the log_items and ignore_keywords at the same time.**
|
||
# Specify the log files we would like to collect in `log_items`.
|
||
# The folders specified should be the subdirectories of `work_dir`.
|
||
log_items = [
|
||
'segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup',
|
||
'segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr',
|
||
'segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr',
|
||
'segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr'
|
||
]
|
||
# Or specify `ignore_keywords`. The folders whose name contain one
|
||
# of the keywords in the `ignore_keywords` list(e.g., `'segformer'`)
|
||
# won't be collected.
|
||
# ignore_keywords = ['segformer']
|
||
|
||
# Other log items in .log.json that you want to collect.
|
||
# should not include metric.
|
||
other_info_keys = ["mAcc"]
|
||
# The output markdown file's name.
|
||
markdown_file ='markdowns/lr_in_trans.json.md'
|
||
# The output json file's name. (optional)
|
||
json_file = 'jsons/trans_in_cnn.json'
|
||
```
|
||
|
||
The structure of the work-dir directory should be like：
|
||
|
||
```text
|
||
├── work-dir
|
||
│ ├── folder1
|
||
│ │ ├── time1.log.json
|
||
│ │ ├── time2.log.json
|
||
│ │ ├── time3.log.json
|
||
│ │ ├── time4.log.json
|
||
│ ├── folder2
|
||
│ │ ├── time5.log.json
|
||
│ │ ├── time6.log.json
|
||
│ │ ├── time7.log.json
|
||
│ │ ├── time8.log.json
|
||
```
|
||
|
||
Then , cd to the log collector folder.
|
||
|
||
Now you can run log_collector.py by using command:
|
||
|
||
```bash
|
||
python log_collector.py ./example_config.py
|
||
```
|
||
|
||
The output markdown file is like:
|
||
|
||
|exp_num|method|mIoU best|best index|mIoU last|last index|last iter num|
|
||
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
||
|1|segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup|0.2776|10|0.2776|10|160000|
|
||
|2|segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr|0.2802|10|0.2802|10|160000|
|
||
|3|segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr|0.4943|11|0.4943|11|160000|
|
||
|4|segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr|0.4883|11|0.4883|11|160000|
|
||
|
||
The output json file is like:
|
||
```json
|
||
[
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2776,
|
||
"mAcc": 0.3779
|
||
},
|
||
"best eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2776,
|
||
"mAcc": 0.3779
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2802,
|
||
"mAcc": 0.3764
|
||
},
|
||
"best eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2802,
|
||
"mAcc": 0.3764
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4943,
|
||
"mAcc": 0.6097
|
||
},
|
||
"best eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4943,
|
||
"mAcc": 0.6097
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4883,
|
||
"mAcc": 0.6061
|
||
},
|
||
"best eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4883,
|
||
"mAcc": 0.6061
|
||
}
|
||
}
|
||
]
|
||
```
|