diff --git a/docs/en/user_guides/useful_tools.md b/docs/en/user_guides/useful_tools.md index a52743bd..1956f58d 100644 --- a/docs/en/user_guides/useful_tools.md +++ b/docs/en/user_guides/useful_tools.md @@ -115,23 +115,23 @@ Description of the script's functions: The data required by each sub function is obtained through the data preparation of `main()`. Function 1: Generated by the sub function `show_bbox_num` to display the distribution of categories and bbox instances. - + Function 2: Generated by the sub function `show_bbox_wh` to display the width and height distribution of categories and bbox instances. - + Function 3: Generated by the sub function `show_bbox_wh_ratio` to display the width to height ratio distribution of categories and bbox instances. - + Function 3: Generated by the sub function `show_bbox_area` to display the distribution map of category and bbox instance area based on area rules. - + -Print List: Generated by the sub function `show_class_list` and `show_data_lis`. +Print List: Generated by the sub function `show_class_list` and `show_data_list`. - + ```shell python tools/analysis_tools/dataset_analysis.py ${CONFIG} \ @@ -145,44 +145,44 @@ python tools/analysis_tools/dataset_analysis.py ${CONFIG} \ E,g： -1.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, By default,the data loadingt type is `train_dataset`, the area rule is `[0,32,96,1e5]`, generate a result graph containing all functions and save the graph to the current running directory `./dataset_analysis` folder: +1.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, By default,the data loadingt type is `train_dataset`, the area rule is `[0,32,96,1e5]`, generate a result graph containing all functions and save the graph to the current running directory `./dataset_analysis` folder: ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py ``` -2.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, change the data loading type from the default `train_dataset` to `val_dataset` through the `--val-dataset` setting: +2.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, change the data loading type from the default `train_dataset` to `val_dataset` through the `--val-dataset` setting: ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --val-dataset ``` -3.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, change the display of all generated classes to specific classes. Take the display of `person` classes as an example: +3.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, change the display of all generated classes to specific classes. Take the display of `person` classes as an example: ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --class-name person ``` -4.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, redefine the area rule through `--area-rule` . Take `30 70 125` as an example, the area rule becomes `[0,30,70,125,1e5]`： +4.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, redefine the area rule through `--area-rule` . Take `30 70 125` as an example, the area rule becomes `[0,30,70,125,1e5]`： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ - --area-rule 30 70 120 +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ + --area-rule 30 70 125 ``` -5.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, change the display of four function renderings to only display `Function 1` as an example: +5.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, change the display of four function renderings to only display `Function 1` as an example: ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --func show_bbox_num ``` -6.Use `config` file `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` analyze the dataset, modify the picture saving address to `work_ir/dataset_analysis`: +6.Use `config` file `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` analyze the dataset, modify the picture saving address to `work_ir/dataset_analysis`: ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --output-dir work_dir/dataset_analysis ``` diff --git a/docs/zh_cn/user_guides/useful_tools.md b/docs/zh_cn/user_guides/useful_tools.md index 4e324d0f..baece672 100644 --- a/docs/zh_cn/user_guides/useful_tools.md +++ b/docs/zh_cn/user_guides/useful_tools.md @@ -108,30 +108,30 @@ python tools/analysis_tools/browse_dataset.py 'configs/yolov5/yolov5_s-v61_syncb --not-show ``` -### 可视化数据集分 +### 可视化数据集分析 脚本 `tools/analysis_tools/dataset_analysis.py` 能够帮助用户得到四种功能的结果图，并将图片保存到当前运行目录下的 `dataset_analysis` 文件夹中。 关于该脚本的功能的说明： 通过 `main()` 的数据准备，得到每个子函数所需要的数据。 功能一：显示类别和 bbox 实例个数的分布图，通过子函数 `show_bbox_num` 生成。 - + 功能二：显示类别和 bbox 实例宽、高的分布图，通过子函数 `show_bbox_wh` 生成。 - + 功能三：显示类别和 bbox 实例宽/高比例的分布图，通过子函数 `show_bbox_wh_ratio` 生成。 - + 功能四：基于面积规则下，显示类别和 bbox 实例面积的分布图，通过子函数 `show_bbox_area` 生成。 - + -打印列表显示，通过脚本中子函数 `show_class_list` 和 `show_data_lis` 生成。 +打印列表显示，通过脚本中子函数 `show_class_list` 和 `show_data_list` 生成。 - + ```shell python tools/analysis_tools/dataset_analysis.py ${CONFIG} \ @@ -145,44 +145,44 @@ python tools/analysis_tools/dataset_analysis.py ${CONFIG} \ 例子： -1.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，其中默认设置:数据加载类型为 `train_dataset` ，面积规则设置为 `[0,32,96,1e5]` ,生成包含所有类的结果图并将图片保存到当前运行目录下 `./dataset_analysis` 文件夹中： +1.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，其中默认设置:数据加载类型为 `train_dataset` ，面积规则设置为 `[0,32,96,1e5]` ,生成包含所有类的结果图并将图片保存到当前运行目录下 `./dataset_analysis` 文件夹中： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py ``` -2.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，通过 `--val-dataset` 设置将数据加载类型由默认的 `train_dataset` 改为 `val_dataset`： +2.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，通过 `--val-dataset` 设置将数据加载类型由默认的 `train_dataset` 改为 `val_dataset`： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --val-dataset ``` -3.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，通过 `--class-name` 设置将生成所有类改为特定类显示，以显示 `person` 为例： +3.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，通过 `--class-name` 设置将生成所有类改为特定类显示，以显示 `person` 为例： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --class-name person ``` -4.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，通过 `--area-rule` 重新定义面积规则，以 `30 70 125` 为例,面积规则变为 `[0,30,70,125,1e5]`： +4.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，通过 `--area-rule` 重新定义面积规则，以 `30 70 125` 为例,面积规则变为 `[0,30,70,125,1e5]`： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --area-rule 30 70 125 ``` -5.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，通过 `--func` 设置，将显示四个功能效果图改为只显示 `功能一` 为例： +5.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，通过 `--func` 设置，将显示四个功能效果图改为只显示 `功能一` 为例： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --func show_bbox_num ``` -6.使用 `config` 文件 `configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py` 分析数据集，通过 `--output-dir` 设置修改图片保存地址，以 `work_ir/dataset_analysis` 地址为例： +6.使用 `config` 文件 `configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py` 分析数据集，通过 `--output-dir` 设置修改图片保存地址，以 `work_ir/dataset_analysis` 地址为例： ```shell -python tools/analysis_tools/dataset_analysis.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py \ +python tools/analysis_tools/dataset_analysis.py configs/yolov5/voc/yolov5_s-v61_fast_1xb64-50e_voc.py \ --output-dir work_dir/dataset_analysis ``` diff --git a/tools/analysis_tools/dataset_analysis.py b/tools/analysis_tools/dataset_analysis.py index 3cb7ccd6..ae0bd114 100644 --- a/tools/analysis_tools/dataset_analysis.py +++ b/tools/analysis_tools/dataset_analysis.py @@ -7,6 +7,7 @@ import matplotlib.patches as mpatches import matplotlib.pyplot as plt import numpy as np from mmengine.config import Config +from mmengine.dataset.dataset_wrapper import ConcatDataset from mmengine.utils import ProgressBar from prettytable import PrettyTable @@ -61,12 +62,12 @@ def show_bbox_num(cfg, args, fig_set, class_name, class_num): print('\n\nDrawing bbox_num figure:') # Draw designs fig = plt.figure( - figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=600) + figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=300) plt.bar(class_name, class_num, align='center') # Draw titles, labels and so on for x, y in enumerate(class_num): - plt.text(x, y, '%s' % y, ha='center', fontsize=fig_set['fontsize']) + plt.text(x, y, '%s' % y, ha='center', fontsize=fig_set['fontsize'] + 3) plt.xticks(rotation=fig_set['xticks_angle']) plt.xlabel('Category Name') plt.ylabel('Num of instances') @@ -77,7 +78,10 @@ def show_bbox_num(cfg, args, fig_set, class_name, class_num): if not os.path.exists(out_dir): os.makedirs(out_dir) out_name = fig_set['out_name'] - fig.savefig(f'{out_dir}/{out_name}_bbox_num.jpg') # Save Image + fig.savefig( + f'{out_dir}/{out_name}_bbox_num.jpg', + bbox_inches='tight', + pad_inches=0.1) # Save Image plt.close() print(f'End and save in {out_dir}/{out_name}_bbox_num.jpg') @@ -88,7 +92,7 @@ def show_bbox_wh(args, fig_set, class_bbox_w, class_bbox_h, class_name): print('\n\nDrawing bbox_wh figure:') # Draw designs fig, ax = plt.subplots( - figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=600) + figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=300) # Set the position of the map and label on the x-axis positions_w = list(range(0, 12 * len(class_name), 12)) @@ -165,7 +169,10 @@ def show_bbox_wh(args, fig_set, class_bbox_w, class_bbox_h, class_name): if not os.path.exists(out_dir): os.makedirs(out_dir) out_name = fig_set['out_name'] - fig.savefig(f'{out_dir}/{out_name}_bbox_wh.jpg') # Save Image + fig.savefig( + f'{out_dir}/{out_name}_bbox_wh.jpg', + bbox_inches='tight', + pad_inches=0.1) # Save Image plt.close() print(f'End and save in {out_dir}/{out_name}_bbox_wh.jpg') @@ -176,7 +183,7 @@ def show_bbox_wh_ratio(args, fig_set, class_name, class_bbox_ratio): print('\n\nDrawing bbox_wh_ratio figure:') # Draw designs fig, ax = plt.subplots( - figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=600) + figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=300) # Set the position of the map and label on the x-axis positions = list(range(0, 6 * len(class_name), 6)) @@ -222,7 +229,10 @@ def show_bbox_wh_ratio(args, fig_set, class_name, class_bbox_ratio): if not os.path.exists(out_dir): os.makedirs(out_dir) out_name = fig_set['out_name'] - fig.savefig(f'{out_dir}/{out_name}_bbox_ratio.jpg') # Save Image + fig.savefig( + f'{out_dir}/{out_name}_bbox_ratio.jpg', + bbox_inches='tight', + pad_inches=0.1) # Save Image plt.close() print(f'End and save in {out_dir}/{out_name}_bbox_ratio.jpg') @@ -240,7 +250,7 @@ def show_bbox_area(args, fig_set, area_rule, class_name, bbox_area_num): # Draw designs fig = plt.figure( - figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=600) + figsize=(fig_set['figsize'][0], fig_set['figsize'][1]), dpi=300) for i in range(len(area_rule) - 1): area_num = [bbox_area_num[idx][i] for idx in range(len(class_name))] plt.bar( @@ -251,7 +261,11 @@ def show_bbox_area(args, fig_set, area_rule, class_name, bbox_area_num): color=colors[i]) for idx, (x, y) in enumerate(zip(positions.tolist(), area_num)): plt.text( - x + width * i, y, y, ha='center', fontsize=fig_set['fontsize']) + x + width * i, + y, + y, + ha='center', + fontsize=fig_set['fontsize'] - 1) # Draw titles, labels and so on plt.xticks(rotation=fig_set['xticks_angle']) @@ -276,7 +290,10 @@ def show_bbox_area(args, fig_set, area_rule, class_name, bbox_area_num): if not os.path.exists(out_dir): os.makedirs(out_dir) out_name = fig_set['out_name'] - fig.savefig(f'{out_dir}/{out_name}_bbox_area.jpg') # Save Image + fig.savefig( + f'{out_dir}/{out_name}_bbox_area.jpg', + bbox_inches='tight', + pad_inches=0.1) # Save Image plt.close() print(f'End and save in {out_dir}/{out_name}_bbox_area.jpg') @@ -331,19 +348,19 @@ def show_data_list(args, area_rule): print(data_info) -def show_dataset_classes(dataset_classes): +def show_data_classes(data_classes): """When printing an error, all class names of the dataset.""" print('\n\nThe name of the class contained in the dataset:') data_classes_info = PrettyTable() data_classes_info.title = 'Information of dataset class' # List Print Settings # If the quantity is too large, 25 rows will be displayed in each column - if len(dataset_classes) < 25: - data_classes_info.add_column('Class name', dataset_classes) - elif len(dataset_classes) % 25 != 0 and len(dataset_classes) > 25: - col_num = int(len(dataset_classes) / 25) + 1 - data_name_list = list(dataset_classes) - for i in range(0, (col_num * 25) - len(dataset_classes)): + if len(data_classes) < 25: + data_classes_info.add_column('Class name', data_classes) + elif len(data_classes) % 25 != 0 and len(data_classes) > 25: + col_num = int(len(data_classes) / 25) + 1 + data_name_list = list(data_classes) + for i in range(0, (col_num * 25) - len(data_classes)): data_name_list.append('') for i in range(0, len(data_name_list), 25): data_classes_info.add_column('Class name', @@ -367,12 +384,21 @@ def main(): elif args.val_dataset is True: dataset = DATASETS.build(cfg.val_dataloader.dataset) - data_list = dataset.load_data_list() + # Determine whether the dataset is ConcatDataset + if isinstance(dataset, ConcatDataset): + datasets = dataset.datasets + data_list = [] + for idx in range(len(datasets)): + datasets_list = datasets[idx].load_data_list() + data_list += datasets_list + else: + data_list = dataset.load_data_list() + # 2.Prepare data # Drawing settings fig_all_set = { - 'figsize': [45, 18], - 'fontsize': 4, + 'figsize': [35, 18], + 'fontsize': int(10 - 0.08 * len(dataset.metainfo['CLASSES'])), 'xticks_angle': 70, 'out_name': cfg.dataset_type } @@ -393,8 +419,8 @@ def main(): classes_idx = [dataset.metainfo['CLASSES'].index(args.class_name)] fig_set = fig_one_set else: - dataset_classes = dataset.metainfo['CLASSES'] - show_dataset_classes(dataset_classes) + data_classes = dataset.metainfo['CLASSES'] + show_data_classes(data_classes) raise RuntimeError(f'Expected args.class_name to be one of the list,' f'but got "{args.class_name}"')