145 lines
4.5 KiB
Markdown
145 lines
4.5 KiB
Markdown
# Log Collector
|
||
|
||
## Function
|
||
|
||
Automatically collect logs and write the result in a json file or markdown file.
|
||
|
||
If there are several `.log.json` files in one folder, Log Collector assumes that the `.log.json` files other than the first one are resume from the preceding `.log.json` file. Log Collector returns the result considering all `.log.json` files.
|
||
|
||
## Usage:
|
||
|
||
To use log collector, you need to write a config file to configure the log collector first.
|
||
|
||
For example:
|
||
|
||
example_config.py:
|
||
|
||
```python
|
||
# The work directory that contains folders that contains .log.json files.
|
||
work_dir = '../../work_dirs'
|
||
# The metric used to find the best evaluation.
|
||
metric = 'mIoU'
|
||
|
||
# **Don't specify the log_items and ignore_keywords at the same time.**
|
||
# Specify the log files we would like to collect in `log_items`.
|
||
# The folders specified should be the subdirectories of `work_dir`.
|
||
log_items = [
|
||
'segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup',
|
||
'segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr',
|
||
'segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr',
|
||
'segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr'
|
||
]
|
||
# Or specify `ignore_keywords`. The folders whose name contain one
|
||
# of the keywords in the `ignore_keywords` list(e.g., `'segformer'`)
|
||
# won't be collected.
|
||
# ignore_keywords = ['segformer']
|
||
|
||
# Other log items in .log.json that you want to collect.
|
||
# should not include metric.
|
||
other_info_keys = ["mAcc"]
|
||
# The output markdown file's name.
|
||
markdown_file ='markdowns/lr_in_trans.json.md'
|
||
# The output json file's name. (optional)
|
||
json_file = 'jsons/trans_in_cnn.json'
|
||
```
|
||
|
||
The structure of the work-dir directory should be like：
|
||
|
||
```text
|
||
├── work-dir
|
||
│ ├── folder1
|
||
│ │ ├── time1.log.json
|
||
│ │ ├── time2.log.json
|
||
│ │ ├── time3.log.json
|
||
│ │ ├── time4.log.json
|
||
│ ├── folder2
|
||
│ │ ├── time5.log.json
|
||
│ │ ├── time6.log.json
|
||
│ │ ├── time7.log.json
|
||
│ │ ├── time8.log.json
|
||
```
|
||
|
||
Then , cd to the log collector folder.
|
||
|
||
Now you can run log_collector.py by using command:
|
||
|
||
```bash
|
||
python log_collector.py ./example_config.py
|
||
```
|
||
|
||
The output markdown file is like:
|
||
|
||
| exp_num | method | mIoU best | best index | mIoU last | last index | last iter num |
|
||
| :-----: | :-----------------------------------------------------: | :-------: | :--------: | :-------: | :--------: | :-----------: |
|
||
| 1 | segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup | 0.2776 | 10 | 0.2776 | 10 | 160000 |
|
||
| 2 | segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr | 0.2802 | 10 | 0.2802 | 10 | 160000 |
|
||
| 3 | segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr | 0.4943 | 11 | 0.4943 | 11 | 160000 |
|
||
| 4 | segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr | 0.4883 | 11 | 0.4883 | 11 | 160000 |
|
||
|
||
The output json file is like:
|
||
|
||
```json
|
||
[
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_lr_with_warmup",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2776,
|
||
"mAcc": 0.3779
|
||
},
|
||
"best eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2776,
|
||
"mAcc": 0.3779
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_cnn_no_warmup_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2802,
|
||
"mAcc": 0.3764
|
||
},
|
||
"best eval": {
|
||
"eval_index": 10,
|
||
"mIoU": 0.2802,
|
||
"mAcc": 0.3764
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_mit_trans_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4943,
|
||
"mAcc": 0.6097
|
||
},
|
||
"best eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4943,
|
||
"mAcc": 0.6097
|
||
}
|
||
},
|
||
{
|
||
"method": "segformer_mit-b5_512x512_160k_ade20k_swin_trans_lr",
|
||
"metric_used": "mIoU",
|
||
"last_iter": 160000,
|
||
"last eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4883,
|
||
"mAcc": 0.6061
|
||
},
|
||
"best eval": {
|
||
"eval_index": 11,
|
||
"mIoU": 0.4883,
|
||
"mAcc": 0.6061
|
||
}
|
||
}
|
||
]
|
||
```
|