diff --git a/model_zoo/official/cv/yolov5/README.md b/model_zoo/official/cv/yolov5/README.md index 4203e2debf8..fb8d8b39ce7 100644 --- a/model_zoo/official/cv/yolov5/README.md +++ b/model_zoo/official/cv/yolov5/README.md @@ -1,361 +1,253 @@ -# Contents +# Inference ProcessContents - [YOLOv5 Description](#YOLOv5-description) - [Model Architecture](#model-architecture) - [Dataset](#dataset) -- [Environment Requirements](#environment-requirements) - [Quick Start](#quick-start) - [Script Description](#script-description) - [Script and Sample Code](#script-and-sample-code) - [Script Parameters](#script-parameters) - [Training Process](#training-process) - [Training](#training) - - [Testing Process](#testing-process) - - [Evaluation](#testing) + - [Distributed Training](#distributed-training) - [Evaluation Process](#evaluation-process) - [Evaluation](#evaluation) - - [Inference Process](#inference-process) - - [Export MindIR](#export-mindir) - - [Infer on Ascend310](#infer-on-ascend310) - - [result](#result) - [Model Description](#model-description) - - [Performance](#performance) - - [Evaluation Performance](#evaluation-performance) - - [Inference Performance](#inference-performance) - - [310 Inference Performance](#310-inference-performance) +- [Performance](#performance) + - [Evaluation Performance](#evaluation-performance) + - [Inference Performance](#inference-performance) + - [Transfer Learning](#transfer-learning) +- [Description of Random Situation](#description-of-random-situation) - [ModelZoo Homepage](#modelzoo-homepage) # [YOLOv5 Description](#contents) -YOLOv5 is a state-of-the-art detector which is faster (FPS) and more accurate (MS COCO AP50...95 and AP50) than all available alternative detectors. -YOLOv5 has verified a large number of features, and selected for use such of them for improving the accuracy of both the classifier and the detector. -These features can be used as best-practice for future studies and developments. +Published in April 2020, YOLOv5 achieved state of the art performance on the COCO dataset for object detection. It is an important improvement of YoloV3, the implementation of a new architecture in the **Backbone** and the modifications in the **Neck** have improved the **mAP**(mean Average Precision) by **10%** and the number of **FPS**(Frame per Second) by **12%**. -[Code](https://github.com/ultralytics/yolov5) +[code](https://github.com/ultralytics/yolov5) # [Model Architecture](#contents) -YOLOv5 choose CSP with Focus backbone, SPP additional module, PANet path-aggregation neck, and YOLOv5 (anchor based) head as the architecture of YOLOv5. +The YOLOv5 network is mainly composed of CSP and Focus as a backbone, spatial pyramid pooling(SPP) additional module, PANet path-aggregation neck and YOLOv3 head. [CSP](https://arxiv.org/abs/1911.11929) is a novel backbone that can enhance the learning capability of CNN. The [spatial pyramid pooling](https://arxiv.org/abs/1406.4729) block is added over CSP to increase the receptive field and separate out the most significant context features. Instead of Feature pyramid networks (FPN) for object detection used in YOLOv3, the PANet is used as the method for parameter aggregation for different detector levels. To be more specifical, CSPDarknet53 contains 5 CSP modules which use the convolution **C** with kernel size k=3x3, stride s = 2x2; Within the PANet and SPP, **1x1, 5x5, 9x9, 13x13 max poolings are applied. # [Dataset](#contents) -Dataset support: [MS COCO] or datasetd with the same format as MS COCO -Annotation support: [MS COCO] or annotation as the same format as MS COCO +Dataset used: [COCO2017]() -- The directory structure is as follows, the name of directory and file is user define: - - ```shell - ├── dataset - ├── YOLOv5 - ├── annotations - │ ├─ train.json - │ └─ val.json - ├─ images - ├─ train - │ └─images - │ ├─picture1.jpg - │ ├─ ... - │ └─picturen.jpg - └─ val - └─images - ├─picture1.jpg - ├─ ... - └─picturen.jpg - ``` - -we suggest user to use MS COCO dataset to experience our model, -other datasets need to use the same format as MS COCO. - -# [Environment Requirements](#contents) - -- Hardware(Ascend) - - Prepare hardware environment with Ascend processor. -- Framework - - [MindSpore](https://www.mindspore.cn/install/en) -- For more information, please check the resources below: - - [MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/master/index.html) - - [MindSpore Python API](https://www.mindspore.cn/docs/api/en/master/index.html) +Note that you can run the scripts with **COCO2017 **or any other datasets with the same format as MS COCO Annotation. But we do suggest user to use MS COCO dataset to experience our model. # [Quick Start](#contents) After installing MindSpore via the official website, you can start training and evaluation as follows: -``` shell -# The parameter of training_shape define image shape for network, default is [640, 640], -``` - -```shell +```bash #run training example(1p) by python command python train.py \ - --data_dir=./dataset/xxx \ + --data_dir=xxx/dataset \ --is_distributed=0 \ - --lr=0.01 \ - --T_max=320 \ - --max_epoch=320 \ - --warmup_epochs=4 \ - --training_shape=640 \ + --yolov5_version='yolov5s' \ + --lr=0.02 \ + --max_epoch=300 \ + --warmup_epochs=20 \ + --per_batch_size=128 \ --lr_scheduler=cosine_annealing > log.txt 2>&1 & ``` -```shell -# standalone training example(1p) by shell script -bash run_standalone_train.sh dataset/xxx -``` - -```shell +```bash # For Ascend device, distributed training example(8p) by shell script -bash run_distribute_train.sh dataset/xxx rank_table_8p.json +bash run_distribute_train.sh xxx/dateset/ xxx/cspdarknet.ckpt rank_table_8pcs.json ``` -```python +```bash # run evaluation by python command python eval.py \ - --data_dir=./dataset/xxx \ - --pretrained=yolov5.ckpt \ - --testing_shape=640 > log.txt 2>&1 & +    --data_dir=xxx/dataset \ +    --testing_shape=640 > log.txt 2>&1 & ``` -```python +```bash # run evaluation by shell script -bash run_eval.sh dataset/xxx checkpoint/xxx.ckpt +bash run_eval.sh xxx/dataset xxx/yolov5.ckpt ``` # [Script Description](#contents) ## [Script and Sample Code](#contents) -```python -└─yolov5 - ├─README.md - ├─mindspore_hub_conf.md # config for mindspore hub - ├─ascend310_infer # application for 310 inference - ├─scripts - ├─run_standalone_train.sh # launch standalone training(1p) in ascend - ├─run_distribute_train.sh # launch distributed training(8p) in ascend - ├─run_infer_310.sh # launch 310 inference in ascend - └─run_eval.sh # launch evaluating in ascend - ├─src - ├─__init__.py # python init file - ├─config.py # parameter configuration - ├─yolov5_backbone.py # backbone of network - ├─distributed_sampler.py # iterator of dataset - ├─initializer.py # initializer of parameters - ├─logger.py # log function - ├─loss.py # loss function - ├─lr_scheduler.py # generate learning rate - ├─transforms.py # Preprocess data - ├─util.py # util function - ├─yolo.py # yolov5 network - ├─yolo_dataset.py # create dataset for YOLOV5 - ├─eval.py # evaluate val results - ├─export.py # convert mindspore model to air model - ├─postprocess.py # postprocess script - └─train.py # train net +```bash +├── model_zoo + ├── README.md // descriptions about all the models + ├── yolov5 + ├── README.md // descriptions about yolov5 + ├── scripts + │ ├──run_distribute_train.sh // launch distributed training(8p) in ascend + │ ├──run_eval.sh // shell script for evaluation + │ ├──rank_table_8pcs.json // the example of rank table settings for 8p training + ├── src + │ ├──config.py // parameter configuration + │ ├──backbone.py // backbone of network + │ ├──distributed_sampler.py // iterator of dataset + │ ├──initializer.py // initializer of parameters + │ ├──logger.py // log function + │ ├──loss.py // loss function + │ ├──lr_scheduler.py // generate learning rate + │ ├──transforms.py // Preprocess data + │ ├──util.py // util function + │ ├──yolo.py // yolov5 network + │ ├──yolo_dataset.py // create dataset for YOLOV5 + ├── train.py // training script + ├── eval.py // evaluation script + ├── export.py // export script ``` ## [Script Parameters](#contents) -Major parameters train.py as follows: +```python +Major parameters in train.py are: -```shell optional arguments: - -h, --help show this help message and exit - --device_target device where the code will be implemented: "Ascend" | "GPU", default is "Ascend" - --data_dir DATA_DIR Train dataset directory. - --per_batch_size PER_BATCH_SIZE - Batch size for Training. Default: 8. - --pretrained_backbone PRETRAINED_BACKBONE - The backbone file of yolov5. Default: "". - --resume_yolov5 RESUME_YOLOV5 - The ckpt file of YOLOv5, which used to fine tune. - Default: "" - --lr_scheduler LR_SCHEDULER - Learning rate scheduler, options: exponential, - cosine_annealing. Default: exponential - --lr LR Learning rate. Default: 0.01 - --lr_epochs LR_EPOCHS - Epoch of changing of lr changing, split with ",". - Default: 220,250 - --lr_gamma LR_GAMMA Decrease lr by a factor of exponential lr_scheduler. - Default: 0.1 - --eta_min ETA_MIN Eta_min in cosine_annealing scheduler. Default: 0 - --T_max T_MAX T-max in cosine_annealing scheduler. Default: 320 - --max_epoch MAX_EPOCH - Max epoch num to train the model. Default: 320 - --warmup_epochs WARMUP_EPOCHS - Warmup epochs. Default: 0 - --weight_decay WEIGHT_DECAY - Weight decay factor. Default: 0.0005 - --momentum MOMENTUM Momentum. Default: 0.9 - --loss_scale LOSS_SCALE - Static loss scale. Default: 1024 - --label_smooth LABEL_SMOOTH - Whether to use label smooth in CE. Default:0 - --label_smooth_factor LABEL_SMOOTH_FACTOR - Smooth strength of original one-hot. Default: 0.1 - --log_interval LOG_INTERVAL - Logging interval steps. Default: 100 - --ckpt_path CKPT_PATH - Checkpoint save location. Default: outputs/ - --ckpt_interval CKPT_INTERVAL - Save checkpoint interval. Default: None - --is_save_on_master IS_SAVE_ON_MASTER - Save ckpt on master or all rank, 1 for master, 0 for - all ranks. Default: 1 - --is_distributed IS_DISTRIBUTED - Distribute train or not, 1 for yes, 0 for no. Default: - 1 - --rank RANK Local rank of distributed. Default: 0 - --group_size GROUP_SIZE - World size of device. Default: 1 - --need_profiler NEED_PROFILER - Whether use profiler. 0 for no, 1 for yes. Default: 0 - --training_shape TRAINING_SHAPE - Fix training shape. Default: "" - --resize_rate RESIZE_RATE - Resize rate for multi-scale training. Default: None + + --device_target device where the code will be implemented: "Ascend", default is "Ascend" + --data_dir Train dataset directory. + --per_batch_size Batch size for Training. Default: 8. + --pretrained_backbone The ckpt file of CSPDarknet53. Default: "". + --resume_yolov5 The ckpt file of YOLOv5, which used to fine tune.Default: "" + --lr_scheduler Learning rate scheduler, options: exponential,cosine_annealing. + Default: cosine_annealing + --lr Learning rate. Default: 0.02 + --lr_epochs Epoch of changing of lr changing, split with ",". Default: '220,250' + --lr_gamma Decrease lr by a factor of exponential lr_scheduler. Default: 0.1 + --eta_min Eta_min in cosine_annealing scheduler. Default: 0. + --t_max T-max in cosine_annealing scheduler. Default: 320 + --max_epoch Max epoch num to train the model. Default: 320 + --warmup_epochs Warmup epochs. Default: 20 + --weight_decay Weight decay factor. Default: 0.0005 + --momentum Momentum. Default: 0.9 + --loss_scale Static loss scale. Default: 64 + --label_smooth Whether to use label smooth in CE. Default:0 + --label_smooth_factor Smooth strength of original one-hot. Default: 0.1 + --log_interval Logging interval steps. Default: 100 + --ckpt_path Checkpoint save location. Default: outputs/ + --ckpt_interval Save checkpoint interval. Default: None + --is_save_on_master Save ckpt on master or all rank, 1 for master, 0 for all ranks. Default: 1 + --is_distributed Distribute train or not, 1 for yes, 0 for no. Default: 1 + --rank Local rank of distributed. Default: 0 + --group_size World size of device. Default: 1 + --need_profiler Whether use profiler. 0 for no, 1 for yes. Default: 0 + --training_shape Fix training shape. Default: "" + --resize_rate Resize rate for multi-scale training. Default: 10 ``` ## [Training Process](#contents) ### Training +For Ascend device, standalone training can be started like this: + ```python +#run training example(1p) by python command python train.py \ - --data_dir=/dataset/xxx \ + --data_dir=xxx/dataset \ + --yolov5_version='yolov5s' \ --is_distributed=0 \ - --lr=0.01 \ - --T_max=320 \ - --max_epoch=320 \ - --warmup_epochs=4 \ - --training_shape=640 \ + --lr=0.02 \ + --max_epoch=300 \ + --warmup_epochs=20 \ + --per_batch_size=128 \ --lr_scheduler=cosine_annealing > log.txt 2>&1 & ``` -The python command above will run in the background, you can view the results through the file log.txt. +The python command above will run in the background, you can view the results through the file `log.txt`. -After training, you'll get some checkpoint files under the outputs folder by default. The loss value will be achieved as follows: +After training, you'll get some checkpoint files under the **outputs** folder by default. The loss value will be achieved as follows: -```shell -# grep "loss:" train/log.txt -2021-05-13 20:50:25,617:INFO:epoch[0], iter[100], loss:loss:2648.764910, fps:61.59 imgs/sec, lr:1.7226087948074564e-05 -2021-05-13 20:50:39,821:INFO:epoch[0], iter[200], loss:loss:764.535622, fps:56.33 imgs/sec, lr:3.4281620173715055e-05 -2021-05-13 20:50:53,287:INFO:epoch[0], iter[300], loss:loss:494.950782, fps:59.47 imgs/sec, lr:5.1337152399355546e-05 -2021-05-13 20:51:06,138:INFO:epoch[0], iter[400], loss:loss:393.339678, fps:62.25 imgs/sec, lr:6.839268462499604e-05 -2021-05-13 20:51:17,985:INFO:epoch[0], iter[500], loss:loss:329.976604, fps:67.57 imgs/sec, lr:8.544822048861533e-05 -2021-05-13 20:51:29,359:INFO:epoch[0], iter[600], loss:loss:294.734397, fps:70.37 imgs/sec, lr:0.00010250374907627702 -2021-05-13 20:51:40,634:INFO:epoch[0], iter[700], loss:loss:281.497078, fps:70.98 imgs/sec, lr:0.00011955928493989632 -2021-05-13 20:51:52,307:INFO:epoch[0], iter[800], loss:loss:264.300707, fps:68.54 imgs/sec, lr:0.0001366148208035156 -2021-05-13 20:52:05,479:INFO:epoch[0], iter[900], loss:loss:261.971103, fps:60.76 imgs/sec, lr:0.0001536703493911773 -2021-05-13 20:52:17,362:INFO:epoch[0], iter[1000], loss:loss:264.591175, fps:67.33 imgs/sec, lr:0.00017072587797883898 +```python +# grep "loss:" log.txt +2021-08-06 15:30:15,798:INFO:epoch[0], iter[600], loss:296.308071, fps:44.44 imgs/sec, lr:0.00010661844862625003 +2021-08-06 15:31:21,119:INFO:epoch[0], iter[700], loss:276.071959, fps:48.99 imgs/sec, lr:0.00012435863027349114 +2021-08-06 15:32:26,185:INFO:epoch[0], iter[800], loss:266.955208, fps:49.18 imgs/sec, lr:0.00014209879736881703 +2021-08-06 15:33:30,507:INFO:epoch[0], iter[900], loss:252.610914, fps:49.75 imgs/sec, lr:0.00015983897901605815 +2021-08-06 15:34:42,176:INFO:epoch[0], iter[1000], loss:243.106683, fps:44.65 imgs/sec, lr:0.00017757914611138403 +2021-08-06 15:35:47,429:INFO:epoch[0], iter[1100], loss:240.498834, fps:49.04 imgs/sec, lr:0.00019531932775862515 +2021-08-06 15:36:48,945:INFO:epoch[0], iter[1200], loss:245.711473, fps:52.02 imgs/sec, lr:0.00021305949485395104 +2021-08-06 15:37:51,293:INFO:epoch[0], iter[1300], loss:231.388255, fps:51.33 imgs/sec, lr:0.00023079967650119215 +2021-08-06 15:38:55,680:INFO:epoch[0], iter[1400], loss:238.904242, fps:49.70 imgs/sec, lr:0.00024853984359651804 +2021-08-06 15:39:57,419:INFO:epoch[0], iter[1500], loss:232.161600, fps:51.83 imgs/sec, lr:0.00026628002524375916 +2021-08-06 15:41:03,808:INFO:epoch[0], iter[1600], loss:227.844698, fps:48.20 imgs/sec, lr:0.00028402020689100027 +2021-08-06 15:42:06,155:INFO:epoch[0], iter[1700], loss:226.668858, fps:51.33 imgs/sec, lr:0.00030176035943441093 ... ``` ### Distributed Training -For Ascend device, distributed training example(8p) by shell script +For Ascend device, distributed training example(8p) by shell script: -```shell -bash run_distribute_train.sh dataset/coco2017 rank_table_8p.json +```bash +# For Ascend device, distributed training example(8p) by shell script +bash run_distribute_train.sh xxx/dateset/ xxx/cspdarknet.ckpt rank_table_8pcs.json ``` The above shell script will run distribute training in the background. You can view the results through the file train_parallel[X]/log.txt. The loss value will be achieved as follows: -```shell -# distribute training result(8p) +```bash +# distribute training result(8p, dynamic shape) ... -2021-05-13 21:08:41,992:INFO:epoch[0], iter[600], loss:247.577421, fps:469.29 imgs/sec, lr:0.0001640283880988136 -2021-05-13 21:08:56,291:INFO:epoch[0], iter[700], loss:235.298894, fps:447.67 imgs/sec, lr:0.0001913209562189877 -2021-05-13 21:09:10,431:INFO:epoch[0], iter[800], loss:239.481037, fps:452.78 imgs/sec, lr:0.00021861353889107704 -2021-05-13 21:09:23,517:INFO:epoch[0], iter[900], loss:232.826709, fps:489.15 imgs/sec, lr:0.0002459061215631664 -2021-05-13 21:09:36,407:INFO:epoch[0], iter[1000], loss:224.734599, fps:496.65 imgs/sec, lr:0.0002731987042352557 -2021-05-13 21:09:49,072:INFO:epoch[0], iter[1100], loss:232.334771, fps:505.34 imgs/sec, lr:0.0003004912578035146 -2021-05-13 21:10:03,597:INFO:epoch[0], iter[1200], loss:242.001476, fps:440.69 imgs/sec, lr:0.00032778384047560394 -2021-05-13 21:10:18,237:INFO:epoch[0], iter[1300], loss:225.391021, fps:437.20 imgs/sec, lr:0.0003550764231476933 -2021-05-13 21:10:33,027:INFO:epoch[0], iter[1400], loss:228.738176, fps:432.76 imgs/sec, lr:0.0003823690058197826 -2021-05-13 21:10:47,424:INFO:epoch[0], iter[1500], loss:225.712950, fps:444.54 imgs/sec, lr:0.0004096615593880415 -2021-05-13 21:11:02,077:INFO:epoch[0], iter[1600], loss:221.249353, fps:436.77 imgs/sec, lr:0.00043695414206013083 -2021-05-13 21:11:16,631:INFO:epoch[0], iter[1700], loss:222.449119, fps:439.89 imgs/sec, lr:0.00046424672473222017 +2021-08-05 16:01:34,116:INFO:epoch[0], iter[200], loss:415.453676, fps:580.07 imgs/sec, lr:0.0002742903889156878 +2021-08-05 16:01:57,588:INFO:epoch[0], iter[300], loss:273.358383, fps:545.96 imgs/sec, lr:0.00041075327317230403 +2021-08-05 16:02:26,247:INFO:epoch[0], iter[400], loss:244.621502, fps:446.64 imgs/sec, lr:0.0005472161574289203 +2021-08-05 16:02:55,532:INFO:epoch[0], iter[500], loss:234.524876, fps:437.10 imgs/sec, lr:0.000683679012581706 +2021-08-05 16:03:25,046:INFO:epoch[0], iter[600], loss:235.185213, fps:434.08 imgs/sec, lr:0.0008201419259421527 +2021-08-05 16:03:54,585:INFO:epoch[0], iter[700], loss:228.878598, fps:433.48 imgs/sec, lr:0.0009566047810949385 +2021-08-05 16:04:23,932:INFO:epoch[0], iter[800], loss:219.259134, fps:436.29 imgs/sec, lr:0.0010930676944553852 +2021-08-05 16:04:52,707:INFO:epoch[0], iter[900], loss:225.741833, fps:444.84 imgs/sec, lr:0.001229530549608171 +2021-08-05 16:05:21,872:INFO:epoch[1], iter[1000], loss:218.811336, fps:438.91 imgs/sec, lr:0.0013659934047609568 +2021-08-05 16:05:51,216:INFO:epoch[1], iter[1100], loss:219.491889, fps:436.50 imgs/sec, lr:0.0015024563763290644 +2021-08-05 16:06:20,546:INFO:epoch[1], iter[1200], loss:219.895906, fps:436.57 imgs/sec, lr:0.0016389192314818501 +2021-08-05 16:06:49,521:INFO:epoch[1], iter[1300], loss:218.516680, fps:441.79 imgs/sec, lr:0.001775382086634636 +2021-08-05 16:07:18,303:INFO:epoch[1], iter[1400], loss:209.922935, fps:444.79 imgs/sec, lr:0.0019118449417874217 +2021-08-05 16:07:47,702:INFO:epoch[1], iter[1500], loss:210.997816, fps:435.60 imgs/sec, lr:0.0020483077969402075 +2021-08-05 16:08:16,482:INFO:epoch[1], iter[1600], loss:210.678421, fps:444.88 imgs/sec, lr:0.002184770768508315 +2021-08-05 16:08:45,568:INFO:epoch[1], iter[1700], loss:203.285874, fps:440.07 imgs/sec, lr:0.0023212337400764227 +2021-08-05 16:09:13,947:INFO:epoch[1], iter[1800], loss:203.014775, fps:451.11 imgs/sec, lr:0.0024576964788138866 +2021-08-05 16:09:42,954:INFO:epoch[2], iter[1900], loss:194.683969, fps:441.28 imgs/sec, lr:0.0025941594503819942 ... ``` ## [Evaluation Process](#contents) -### Valid +### Evaluation + +Before running the command below, please check the checkpoint path used for evaluation. The file **yolov5.ckpt** used in the follow script is the last saved checkpoint file, but we renamed it to "yolov5.ckpt". ```python +# run evaluation by python command python eval.py \ - --data_dir=./dataset/coco2017 \ - --pretrained=yolov5.ckpt \ - --testing_shape=640 > log.txt 2>&1 & +    --data_dir=xxx/dataset \ +    --pretrained=xxx/yolov5.ckpt \ +    --testing_shape=640 > log.txt 2>&1 & OR -bash run_eval.sh dataset/coco2017 checkpoint/yolov5.ckpt +# run evaluation by shell script +bash run_eval.sh xxx/dataset xxx/yolov5.ckpt ``` The above python command will run in the background. You can view the results through the file "log.txt". The mAP of the test dataset will be as follows: -```shell +```python # log.txt =============coco eval reulst========= - Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.372 - Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.574 - Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.403 - Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.219 - Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.426 - Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.480 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.302 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.504 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.560 - Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.399 - Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.619 - Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.674 -``` - -## [Inference process](#contents) - -### Export MindIR - -```shell -python export.py --ckpt_file [CKPT_PATH] --file_format [EXPORT_FORMAT] --batch_size [BATCH_SIZE] -``` - -The ckpt_file parameter is required, -`EXPORT_FORMAT` should be in ["AIR", "MINDIR"].Current model only support CPU MODE. -`BATCH_SIZE` current batch_size can only be set to 1. - -### Infer on Ascend310 - -Before performing inference, the mindir file must be exported by `export.py` script. We only provide an example of inference using MINDIR model. -Current batch_size can only be set to 1. - -```shell -# Ascend310 inference -bash run_infer_310.sh [MINDIR_PATH] [DATA_PATH] [ANN_FILE] [DVPP] [DEVICE_ID] -``` - -- `ANN_FILE` annotations file path. -- `DVPP` is mandatory, and must choose from ["DVPP", "CPU"], it's case-insensitive. Current model only support CPU MODE. -- `DEVICE_ID` is optional, default value is 0. - -### result - -Inference result is saved in current path, you can find result like this in acc.log file. - -```bash -# acc.log -=============coco 310 infer reulst========= - Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.369 - Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.571 - Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.398 - Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.216 - Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.421 - Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.487 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.301 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.502 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.558 - Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.388 - Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.617 - Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.677 +Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.369 +Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.573 +Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.395 +Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.218 +Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.418 +Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.482 +Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.298 +Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.501 +Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.557 +Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.395 +Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.619 +Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.677 +2020-12-21 17:16:40,322:INFO:testing cost time 0.35h ``` # [Model Description](#contents) @@ -366,56 +258,41 @@ Inference result is saved in current path, you can find result like this in acc. YOLOv5 on 118K images(The annotation and data format must be the same as coco2017) -| Parameters | YOLOv5s | -| -------------------------- | ----------------------------------------------------------- | -| Resource | Ascend 910; CPU 2.60GHz, 192cores; Memory, 755G | -| uploaded Date | 5/14/2021 (month/day/year) | -| MindSpore Version | 1.0.0-alpha | -| Dataset | 118K images | -| Training Parameters | epoch=320, batch_size=8, lr=0.01, momentum=0.9 | -| Optimizer | Momentum | -| Loss Function | Sigmoid Cross Entropy with logits, Giou Loss | -| outputs | heatmaps | -| Loss | 53 | -| Speed | 1p 55 img/s 8p 440 img/s(shape=640) | -| Total time | 24h(8pcs) | -| Checkpoint for Fine tuning | 58M (.ckpt file) | -| Scripts | | +| Parameters | YOLOv5 | +| -------------------------- | ------------------------------------------------------------ | +| Resource | Ascend 910 ;CPU 2.60GHz,192cores; Memory, 755G | +| uploaded Date | 7/12/2021 (month/day/year) | +| MindSpore Version | 1.2.0 | +| Dataset | 118K images | +| Training Parameters | epoch=300, batch_size=8, lr=0.02,momentum=0.9,warmup_epoch=20 | +| Optimizer | Momentum | +| Loss Function | Sigmoid Cross Entropy with logits, Giou Loss | +| outputs | boxes and label | +| Loss | 111.970097 | +| Speed | 8p about 450 FPS | +| Total time | 8p 21h28min | +| Checkpoint for Fine tuning | 53.62M (.ckpt file) | +| Scripts | https://gitee.com/mindspore/mindspore/tree/master/model_zoo/ | ### Inference Performance -YOLOv5 on 5K images(The annotation and data format must be the same as coco val2017 ) - -| Parameters | YOLOv5s | -| -------------------------- | ----------------------------------------------------------- | -| Resource | Ascend 910; CPU 2.60GHz, 192cores; Memory, 755G | -| uploaded Date | 5/14/2021 (month/day/year) | -| MindSpore Version | 1.2.0 | -| Dataset | 5K images | -| batch_size | 1 | -| outputs | box position and sorces, and probability | -| Accuracy | map=36.8~37.2%(shape=640) | -| Model for inference | 58M (.ckpt file) | - -### 310 Inference Performance - -| Parameters | Ascend | -| ------------------- | ---------------------------------------- | -| Model Version | YOLOv5s | -| Resource | Ascend 310; CentOS 3.10 | -| Uploaded Date | 07/06/2021 (month/day/year) | -| MindSpore Version | 1.2.0 | -| Dataset | Coco2017 5K images | -| batch_size | 1 | +| Parameters | YOLOv5 | +| ------------------- | --------------------------- | +| Resource | Ascend 910 ;CPU 2.60GHz,192cores; Memory, 755G | +| Uploaded Date | 7/12/2021 (month/day/year) | +| MindSpore Version | 1.2.0 | +| Dataset | 20K images | +| batch_size | 1 | | outputs | box position and sorces, and probability | -| Accuracy | Accuracy=0.71654 | -| Model for inference | 58M(.ckpt file) | +| Accuracy | mAP >= 36.7%(shape=640) | +| Model for inference | 56.67M (.ckpt file) | + +### Transfer Learning # [Description of Random Situation](#contents) -In dataset.py, we set the seed inside ```create_dataset``` function. -In var_init.py, we set seed for weight initialization +In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py. # [ModelZoo Homepage](#contents) -Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo). + Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo). diff --git a/model_zoo/official/cv/yolov5/README_CN.md b/model_zoo/official/cv/yolov5/README_CN.md deleted file mode 100644 index c9d7a5861ec..00000000000 --- a/model_zoo/official/cv/yolov5/README_CN.md +++ /dev/null @@ -1,422 +0,0 @@ -# 目录 - -- [YOLOv5说明](#yolov5说明) -- [模型架构](#模型架构) -- [数据集](#数据集) -- [环境要求](#环境要求) -- [快速入门](#快速入门) -- [脚本说明](#脚本说明) - - [脚本和示例代码](#脚本和示例代码) - - [脚本参数](#脚本参数) - - [训练过程](#训练过程) - - [训练](#训练) - - [测试过程](#测试过程) - - [测试](#测试) - - [评估过程](#评估过程) - - [评估](#评估) - - [推理过程](#推理过程) - - [导出MindIR](#导出mindir) - - [在Ascend310执行推理](#在ascend310执行推理) - - [结果](#结果) -- [模型说明](#模型说明) - - [性能](#性能) - - [评估性能](#评估性能) - - [推理性能](#推理性能) - - [310推理性能](#310推理性能) -- [ModelZoo主页](#modelzoo主页) - -# [YOLOv5描述](#目录) - -YOLOv5作为先进的检测器,它比所有可用的替代检测器更快(FPS)并且更准确(MS COCO AP50 ... 95和AP50)。 -本文已经验证了大量的特征,并选择使用这些特征来提高分类和检测的精度。 -这些特性可以作为未来研究和开发的最佳实践。 - -[代码](https://github.com/ultralytics/yolov5) - -# [模型架构](#目录) - -选择CSP Focus主干、SPP附加模块、PANet路径聚合网络和YOLOv5(基于锚点)头作为YOLOv5架构。 - -# [数据集](#目录) - -支持的数据集:[MS COCO]或与MS COCO格式相同的数据集 -支持的标注:[MS COCO]或与MS COCO相同格式的标注 - -- 目录结构如下,由用户定义目录和文件的名称: - - ```shell - ├── dataset - ├── YOLOv5 - ├── annotations - │ ├─ train.json - │ └─ val.json - ├─ images - ├─ train - │ └─images - │ ├─picture1.jpg - │ ├─ ... - │ └─picturen.jpg - └─ val - └─images - ├─picture1.jpg - ├─ ... - └─picturen.jpg - ``` - -建议用户使用MS COCO数据集来体验模型, -其他数据集需要使用与MS COCO相同的格式。 - -# [环境要求](#目录) - -- 硬件 Ascend - - 使用Ascend处理器准备硬件环境。 -- 框架 - - [MindSpore](https://www.mindspore.cn/install) -- 更多关于Mindspore的信息,请查看以下资源: - - [MindSpore教程](https://www.mindspore.cn/tutorials/zh-CN/master/index.html) - - [MindSpore API](https://www.mindspore.cn/docs/api/zh-CN/master/index.html) - -# [快速入门](#目录) - -通过官方网站安装MindSpore后,您可以按照如下步骤进行训练和评估: - -``` shell -# training_shape参数定义网络图像形状,默认为[640, 640]。 -``` - -```shell -# python命令执行训练示例(1卡) -python train.py \ - --data_dir=./dataset/xxx \ - --is_distributed=0 \ - --lr=0.01 \ - --T_max=320 \ - --max_epoch=320 \ - --warmup_epochs=4 \ - --training_shape=640 \ - --lr_scheduler=cosine_annealing > log.txt 2>&1 & -``` - -```shell -# shell脚本单机训练示例(1卡) -bash run_standalone_train.sh dataset/xxx -``` - -```shell -# 对于Ascend设备,使用shell脚本分布式训练示例(8卡) -bash run_distribute_train.sh dataset/xxx rank_table_8p.json -``` - -```python -# 使用python命令评估 -python eval.py \ - --data_dir=./dataset/xxx \ - --pretrained=yolov5.ckpt \ - --testing_shape=640 > log.txt 2>&1 & -``` - -```python -# shell脚本执行评估 -bash run_eval.sh dataset/xxx checkpoint/xxx.ckpt -``` - -# [脚本说明](#目录) - -## [脚本和示例代码](#目录) - -```python -└─yolov5 - ├─README.md - ├─mindspore_hub_conf.md # Mindspore Hub配置 - ├─ascend310_infer # 用于310推理 - ├─scripts - ├─run_standalone_train.sh # 在Ascend中启动单机训练(1卡) - ├─run_distribute_train.sh # 在Ascend中启动分布式训练(8卡) - ├─run_infer_310.sh # 在Ascend中启动310推理 - ├─run_eval.sh # 在Ascend中启动评估 - ├─src - ├─__init__.py # Python初始化文件 - ├─config.py # 参数配置 - ├─yolov5_backbone.py # 网络骨干 - ├─distributed_sampler.py # 数据集迭代器 - ├─initializer.py # 参数初始化器 - ├─logger.py # 日志函数 - ├─loss.py # 损失函数 - ├─lr_scheduler.py # 生成学习率 - ├─transforms.py # 预处理数据 - ├─util.py # 工具函数 - ├─yolo.py # YOLOv5网络 - ├─yolo_dataset.py # 为YOLOv5创建数据集 - - ├─eval.py # 评估验证结果 - ├─export.py # 将MindSpore模型转换为AIR模型 - ├─preprocess.py # 310推理前处理脚本 - ├─postprocess.py # 310推理后处理脚本 - ├─train.py # 训练网络 -``` - -## [脚本参数](#目录) - -train.py中主要参数如下: - -```shell -可选参数: - -h, --help 显示此帮助消息并退出 - --device_target 实现代码的设备:“Ascend”(默认值)|“GPU” - --data_dir DATA_DIR 训练数据集目录 - --per_batch_size PER_BATCH_SIZE - 训练的批处理大小。 默认值:8。 - --pretrained_backbone PRETRAINED_BACKBONE - YOLOv5主干文件。 默认值:""。 - --resume_yolov5 RESUME_YOLOV5 - YOLOv5的ckpt文件,用于微调。 - 默认值:"" - --lr_scheduler LR_SCHEDULER - 学习率调度器,取值选项:exponential, - cosine_annealing。 默认值:exponential - --lr LR 学习率。 默认值:0.01 - --lr_epochs LR_EPOCHS - LR变化轮次,用“,”分隔。 - 默认值:220,250 - --lr_gamma LR_GAMMA 将LR降低一个exponential lr_scheduler因子。 - 默认值:0.1 - --eta_min ETA_MIN cosine_annealing调度器中的eta_min。 默认值:0 - --T_max T_MAX cosine_annealing调度器中的T-max。 默认值:320 - --max_epoch MAX_EPOCH - 训练模型的最大轮次数。 默认值:320 - --warmup_epochs WARMUP_EPOCHS - 热身轮次。 默认值:0 - --weight_decay WEIGHT_DECAY - 权重衰减因子。 默认值:0.0005 - --momentum MOMENTUM 动量。 默认值:0.9 - --loss_scale LOSS_SCALE - 静态损失尺度。 默认值:1024 - --label_smooth LABEL_SMOOTH - CE中是否使用标签平滑。 默认值:0 - --label_smooth_factor LABEL_SMOOTH_FACTOR - 原one-hot的光滑强度。 默认值:0.1 - --log_interval LOG_INTERVAL - 日志记录间隔步数。 默认值:100 - --ckpt_path CKPT_PATH - Checkpoint保存位置。 默认值:outputs/ - --ckpt_interval CKPT_INTERVAL - 保存checkpoint间隔。 默认值:None - --is_save_on_master IS_SAVE_ON_MASTER - 在master或all rank上保存ckpt,1代表master,0代表 - all ranks。 默认值:1 - --is_distributed IS_DISTRIBUTED - 是否分发训练,1代表是,0代表否。 默认值: - 1 - --rank RANK 分布式本地进程序号。 默认值:0 - --group_size GROUP_SIZE - 设备进程总数。 默认值:1 - --need_profiler NEED_PROFILER - 是否使用profiler。 0表示否,1表示是。 默认值:0 - --training_shape TRAINING_SHAPE - 恢复训练形状。 默认值:"" - --resize_rate RESIZE_RATE - 多尺度训练的缩放速率。 默认值:None -``` - -## [训练过程](#目录) - -### 训练 - -```python -python train.py \ - --data_dir=/dataset/xxx \ - --is_distributed=0 \ - --lr=0.01 \ - --T_max=320 \ - --max_epoch=320 \ - --warmup_epochs=4 \ - --training_shape=640 \ - --lr_scheduler=cosine_annealing > log.txt 2>&1 & -``` - -上述python命令将在后台运行,您可以通过log.txt文件查看结果。 - -训练结束后,您可在默认输出文件夹下找到checkpoint文件。 得到如下损失值: - -```shell -# grep "loss:" train/log.txt -2021-05-13 20:50:25,617:INFO:epoch[0], iter[100], loss:loss:2648.764910, fps:61.59 imgs/sec, lr:1.7226087948074564e-05 -2021-05-13 20:50:39,821:INFO:epoch[0], iter[200], loss:loss:764.535622, fps:56.33 imgs/sec, lr:3.4281620173715055e-05 -2021-05-13 20:50:53,287:INFO:epoch[0], iter[300], loss:loss:494.950782, fps:59.47 imgs/sec, lr:5.1337152399355546e-05 -2021-05-13 20:51:06,138:INFO:epoch[0], iter[400], loss:loss:393.339678, fps:62.25 imgs/sec, lr:6.839268462499604e-05 -2021-05-13 20:51:17,985:INFO:epoch[0], iter[500], loss:loss:329.976604, fps:67.57 imgs/sec, lr:8.544822048861533e-05 -2021-05-13 20:51:29,359:INFO:epoch[0], iter[600], loss:loss:294.734397, fps:70.37 imgs/sec, lr:0.00010250374907627702 -2021-05-13 20:51:40,634:INFO:epoch[0], iter[700], loss:loss:281.497078, fps:70.98 imgs/sec, lr:0.00011955928493989632 -2021-05-13 20:51:52,307:INFO:epoch[0], iter[800], loss:loss:264.300707, fps:68.54 imgs/sec, lr:0.0001366148208035156 -2021-05-13 20:52:05,479:INFO:epoch[0], iter[900], loss:loss:261.971103, fps:60.76 imgs/sec, lr:0.0001536703493911773 -2021-05-13 20:52:17,362:INFO:epoch[0], iter[1000], loss:loss:264.591175, fps:67.33 imgs/sec, lr:0.00017072587797883898 -... -``` - -### 分布式训练 - -对于Ascend设备,使用shell脚本分布式训练示例(8卡) - -```shell -bash run_distribute_train.sh dataset/coco2017 rank_table_8p.json -``` - -上述shell脚本将在后台运行分布式训练。 您可以通过train_parallel[X]/log.txt文件查看结果。 得到如下损失值: - -```shell -# 分布式训练示例(8卡) -... -2021-05-13 21:08:41,992:INFO:epoch[0], iter[600], loss:247.577421, fps:469.29 imgs/sec, lr:0.0001640283880988136 -2021-05-13 21:08:56,291:INFO:epoch[0], iter[700], loss:235.298894, fps:447.67 imgs/sec, lr:0.0001913209562189877 -2021-05-13 21:09:10,431:INFO:epoch[0], iter[800], loss:239.481037, fps:452.78 imgs/sec, lr:0.00021861353889107704 -2021-05-13 21:09:23,517:INFO:epoch[0], iter[900], loss:232.826709, fps:489.15 imgs/sec, lr:0.0002459061215631664 -2021-05-13 21:09:36,407:INFO:epoch[0], iter[1000], loss:224.734599, fps:496.65 imgs/sec, lr:0.0002731987042352557 -2021-05-13 21:09:49,072:INFO:epoch[0], iter[1100], loss:232.334771, fps:505.34 imgs/sec, lr:0.0003004912578035146 -2021-05-13 21:10:03,597:INFO:epoch[0], iter[1200], loss:242.001476, fps:440.69 imgs/sec, lr:0.00032778384047560394 -2021-05-13 21:10:18,237:INFO:epoch[0], iter[1300], loss:225.391021, fps:437.20 imgs/sec, lr:0.0003550764231476933 -2021-05-13 21:10:33,027:INFO:epoch[0], iter[1400], loss:228.738176, fps:432.76 imgs/sec, lr:0.0003823690058197826 -2021-05-13 21:10:47,424:INFO:epoch[0], iter[1500], loss:225.712950, fps:444.54 imgs/sec, lr:0.0004096615593880415 -2021-05-13 21:11:02,077:INFO:epoch[0], iter[1600], loss:221.249353, fps:436.77 imgs/sec, lr:0.00043695414206013083 -2021-05-13 21:11:16,631:INFO:epoch[0], iter[1700], loss:222.449119, fps:439.89 imgs/sec, lr:0.00046424672473222017 -... -``` - -## [评估过程](#目录) - -### 验证 - -```python -python eval.py \ - --data_dir=./dataset/coco2017 \ - --pretrained=yolov5.ckpt \ - --testing_shape=640 > log.txt 2>&1 & -OR -bash run_eval.sh dataset/coco2017 checkpoint/yolov5.ckpt -``` - -上述python命令将在后台运行。 您可以通过log.txt文件查看结果。 测试数据集的mAP如下: - -```shell -# log.txt -=============coco eval reulst========= - Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.372 - Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.574 - Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.403 - Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.219 - Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.426 - Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.480 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.302 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.504 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.560 - Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.399 - Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.619 - Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.674 -``` - -## [推理过程](#目录) - -### 导出MindIR - -```shell -python export.py --ckpt_file [CKPT_PATH] --file_format [EXPORT_FORMAT] --batch_size [BATCH_SIZE] -``` - -参数ckpt_file为必填项, -`EXPORT_FORMAT` 必须在 ["AIR", "MINDIR"]中选择。 -`BATCH_SIZE` 目前仅支持batch_size为1的推理。 - -### 在Ascend310执行推理 - -在执行推理前,mindir文件必须通过`export.py`脚本导出。以下展示了使用mindir模型执行推理的示例。 - -```shell -# Ascend310 inference -bash run_infer_310.sh [MINDIR_PATH] [DATA_PATH] [ANN_FILE] [DVPP] [DEVICE_ID] -``` - -- `ANN_FILE` Annotations 文件路径。 -- `DVPP` 为必填项,需要在["DVPP", "CPU"]选择,大小写均可。目前仅支持CPU算子推理。 -- `DEVICE_ID` 可选,默认值为0。 - -### 结果 - -推理结果保存在脚本执行的当前路径,你可以在acc.log中看到以下精度计算结果。 - -```bash -=============coco 310 infer reulst========= - Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.369 - Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.571 - Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.398 - Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.216 - Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.421 - Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.487 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.301 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.502 - Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.558 - Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.388 - Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.617 - Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.677 -``` - -# [模型说明](#目录) - -## [性能](#目录) - -### 评估性能 - -YOLOv5应用于118000张图像上(标注和数据格式必须与COCO 2017相同) - -|参数| YOLOv5s | -| -------------------------- | ----------------------------------------------------------- | -| 资源 | Ascend 910;CPU 2.60GHz,192核;内存:755G | -|上传日期| 2021年05月14日 | -| MindSpore版本|1.0.0-alpha| -|数据集|118000张图像| -|训练参数|epoch=320, batch_size=8, lr=0.01, momentum=0.9| -| 优化器 | Momentum | -|损失函数|Sigmoid Cross Entropy with logits, Giou Loss| -|输出|heatmaps | -| 损失 | 53 | -|速度| 1卡:55 img/s;8卡:440 img/s(shape=640)| -| 总时长 | 24小时(8卡) | -| 微调检查点 | 58M (.ckpt文件) | -|脚本| | - -### 推理性能 - -YOLOv5应用于5000张图像上(标注和数据格式必须与COCO val 2017相同) - -|参数| YOLOv5s | -| -------------------------- | ----------------------------------------------------------- | -| 资源 | Ascend 910;CPU 2.60GHz,192核;内存:755G | -|上传日期| 2021年05月14日 | -| MindSpore版本 | 1.2.0 | -|数据集|5000张图像| -|批处理大小|1| -|输出|边框位置和分数,以及概率| -|精度|map=36.8~37.2%(shape=640)| -|推理模型| 58M(.ckpt文件)| - -### 310推理性能 - -YOLOv5应用于5000张图像上(标注和数据格式必须与COCO val 2017相同) - -|参数| YOLOv5s | -| -------------------------- | ----------------------------------------------------------- | -| 资源 | Ascend 310;CPU 2.60GHz,192核;内存:755G | -|上传日期| 2021年06月28日 | -| MindSpore版本 | 1.2.0 | -|数据集|5000张图像| -|批处理大小|1| -|输出|边框位置和分数,以及概率| -|精度|map=36.9%(shape=640)| -|推理模型| 58M(.ckpt文件)| - -# [随机情况说明](#目录) - -在dataset.py中,我们设置了“create_dataset”函数内的种子。 -在var_init.py中,我们设置了权重初始化的种子。 - -# [ModelZoo主页](#目录) - -请浏览官网[主页](https://gitee.com/mindspore/mindspore/tree/master/model_zoo)。 diff --git a/model_zoo/official/cv/yolov5/eval.py b/model_zoo/official/cv/yolov5/eval.py index c9f5329d5e8..8be991e01ac 100644 --- a/model_zoo/official/cv/yolov5/eval.py +++ b/model_zoo/official/cv/yolov5/eval.py @@ -31,7 +31,7 @@ from mindspore import context from mindspore.train.serialization import load_checkpoint, load_param_into_net import mindspore as ms -from src.yolo import YOLOV5s +from src.yolo import YOLOV5 from src.logger import get_logger from src.yolo_dataset import create_yolo_dataset from src.config import ConfigYOLOV5 @@ -43,11 +43,13 @@ parser.add_argument('--device_target', type=str, default='Ascend', help='device where the code will be implemented. (Default: Ascend)') # dataset related -parser.add_argument('--data_dir', type=str, default='', help='train data dir') +parser.add_argument('--data_dir', type=str, default='/data/coco', help='train data dir') parser.add_argument('--per_batch_size', default=1, type=int, help='batch size for per gpu') # network related parser.add_argument('--pretrained', default='', type=str, help='model_path, local pretrained model to load') +parser.add_argument('--yolov5_version', default='yolov5s', type=str, + help='The version of YOLOv5, options: yolov5s, yolov5m, yolov5l, yolov5x') # logging related parser.add_argument('--log_path', type=str, default='outputs/', help='checkpoint save location') @@ -59,11 +61,37 @@ parser.add_argument('--testing_shape', type=str, default='', help='shape for tes parser.add_argument('--ignore_threshold', type=float, default=0.001, help='threshold to throw low quality boxes') parser.add_argument('--multi_label', type=ast.literal_eval, default=True, help='whether to use multi label') parser.add_argument('--multi_label_thresh', type=float, default=0.1, help='threshhold to throw low quality boxes') +parser.add_argument('--is_modelArts', type=int, default=0, + help='Trainning in modelArts or not, 1 for yes, 0 for no. Default: 0') args, _ = parser.parse_known_args() +args.rank = 0 -args.data_root = os.path.join(args.data_dir, 'val2017') -args.ann_file = os.path.join(args.data_dir, 'annotations/instances_val2017.json') +if args.is_modelArts: + args.data_root = os.path.join(args.data_dir, 'val2017') + args.ann_file = os.path.join(args.data_dir, 'annotations') + import moxing as mox + + local_data_url = os.path.join('/cache/data', str(args.rank)) + local_annFile = os.path.join('/cache/data', str(args.rank)) + local_pretrained = os.path.join('/cache/data', str(args.rank)) + + temp_str = args.pretrained.split('/')[-1] + args.pretrained = args.pretrained[0:args.pretrained.rfind('/')] + + mox.file.copy_parallel(args.data_root, local_data_url) + args.data_root = local_data_url + + mox.file.copy_parallel(args.ann_file, local_annFile) + args.ann_file = os.path.join(local_data_url, 'instances_val2017.json') + + mox.file.copy_parallel(args.pretrained, local_pretrained) + args.pretrained = os.path.join(local_data_url, temp_str) +else: + args.data_root = os.path.join(args.data_dir, 'val2017') + args.ann_file = os.path.join( + args.data_dir, + 'annotations/instances_val2017.json') class Redirct: @@ -103,7 +131,7 @@ class DetectionEngine: self.nms_thresh = args_detection.nms_thresh self.multi_label = args_detection.multi_label self.multi_label_thresh = args_detection.multi_label_thresh - # self.coco_catids = self._coco.getCatIds() + self.coco_catids = self._coco.getCatIds() self.coco_catIds = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77, 78, 79, 80, @@ -111,18 +139,15 @@ class DetectionEngine: def do_nms_for_results(self): """Get result boxes.""" - # np.save('/opt/disk1/hjc/yolov5_positive_policy/result.npy', self.results) for img_id in self.results: for clsi in self.results[img_id]: dets = self.results[img_id][clsi] dets = np.array(dets) keep_index = self._diou_nms(dets, thresh=self.nms_thresh) - keep_box = [{'image_id': int(img_id), - 'category_id': int(clsi), + keep_box = [{'image_id': int(img_id), 'category_id': int(clsi), 'bbox': list(dets[i][:4].astype(float)), - 'score': dets[i][4].astype(float)} - for i in keep_index] + 'score': dets[i][4].astype(float)} for i in keep_index] self.det_boxes.extend(keep_box) def _nms(self, predicts, threshold): @@ -149,7 +174,8 @@ class DetectionEngine: intersect_w = np.maximum(0.0, min_x2 - max_x1 + 1) intersect_h = np.maximum(0.0, min_y2 - max_y1 + 1) intersect_area = intersect_w * intersect_h - ovr = intersect_area / (areas[i] + areas[order[1:]] - intersect_area) + ovr = intersect_area / \ + (areas[i] + areas[order[1:]] - intersect_area) indexes = np.where(ovr <= threshold)[0] order = order[indexes + 1] @@ -289,8 +315,8 @@ class DetectionEngine: flag[i, c] = True confidence = cls_emb[flag] * conf - for x_lefti, y_lefti, wi, hi, confi, clsi in zip(x_top_left, y_top_left, w, h, confidence, - cls_argmax): + for x_lefti, y_lefti, wi, hi, confi, clsi in zip(x_top_left, y_top_left, + w, h, confidence, cls_argmax): if confi < self.ignore_threshold: continue if img_id not in self.results: @@ -317,9 +343,8 @@ if __name__ == "__main__": context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target, device_id=device_id) # logger - args.outputs_dir = os.path.join(args.log_path, - datetime.datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S')) - rank_id = int(os.environ.get('RANK_ID')) if os.environ.get('RANK_ID') else 0 + args.outputs_dir = os.path.join(args.log_path, datetime.datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S')) + rank_id = int(os.getenv('DEVICE_ID', '0')) args.logger = get_logger(args.outputs_dir, rank_id) context.reset_auto_parallel_context() @@ -327,7 +352,8 @@ if __name__ == "__main__": context.set_auto_parallel_context(parallel_mode=parallel_mode, gradients_mean=True, device_num=1) args.logger.info('Creating Network....') - network = YOLOV5s(is_training=False) + dict_version = {'yolov5s': 0, 'yolov5m': 1, 'yolov5l': 2, 'yolov5x': 3} + network = YOLOV5(is_training=False, version=dict_version[args.yolov5_version]) args.logger.info(args.pretrained) if os.path.isfile(args.pretrained): @@ -355,8 +381,7 @@ if __name__ == "__main__": config.test_img_shape = convert_testing_shape(args.testing_shape) ds, data_size = create_yolo_dataset(data_root, ann_file, is_training=False, batch_size=args.per_batch_size, - max_epoch=1, device_num=1, rank=rank_id, shuffle=False, - config=config) + max_epoch=1, device_num=1, rank=rank_id, shuffle=False, config=config) args.logger.info('testing shape : {}'.format(config.test_img_shape)) args.logger.info('total {} images to eval'.format(data_size)) diff --git a/model_zoo/official/cv/yolov5/export.py b/model_zoo/official/cv/yolov5/export.py index ba4a237c33a..a228cef97e5 100644 --- a/model_zoo/official/cv/yolov5/export.py +++ b/model_zoo/official/cv/yolov5/export.py @@ -18,18 +18,21 @@ import numpy as np import mindspore from mindspore import context, Tensor from mindspore.train.serialization import export, load_checkpoint, load_param_into_net -from src.config import ConfigYOLOV5 + from src.yolo import YOLOV5s_Infer parser = argparse.ArgumentParser(description='yolov5 export') parser.add_argument("--device_id", type=int, default=0, help="Device id") parser.add_argument("--batch_size", type=int, default=1, help="batch size") +parser.add_argument('--yolov5_version', default='yolov5s', type=str, + help='The version of YOLOv5, options: yolov5s, yolov5m, yolov5l, yolov5x') + parser.add_argument("--testing_shape", type=int, default=640, help="test shape") parser.add_argument("--ckpt_file", type=str, required=True, help="Checkpoint file path.") parser.add_argument("--file_name", type=str, default="yolov5", help="output file name.") -parser.add_argument('--file_format', type=str, choices=["AIR", "MINDIR"], default='AIR', help='file format') -parser.add_argument("--device_target", type=str, choices=["Ascend", "GPU", "CPU"], default="Ascend", - help="device target") +parser.add_argument('--file_format', type=str, choices=["AIR", "ONNX", "MINDIR"], default='MINDIR', help='file format') +parser.add_argument("--device_target", type=str, choices=["Ascend", "GPU", "CPU"], + default="Ascend", help="device target") args = parser.parse_args() context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target) @@ -37,16 +40,18 @@ if args.device_target == "Ascend": context.set_context(device_id=args.device_id) if __name__ == "__main__": - config = ConfigYOLOV5() - if args.testing_shape: - config.test_img_shape = [int(args.testing_shape), int(args.testing_shape)] - ts_shape = config.test_img_shape[0] + ts_shape = args.testing_shape // 2 - network = YOLOV5s_Infer(config.test_img_shape) + dict_version = {'yolov5s': 0, 'yolov5m': 1, 'yolov5l': 2, 'yolov5x': 3} + args.file_name = args.file_name + '_' + args.yolov5_version + + network = YOLOV5s_Infer(args.testing_shape, version=dict_version[args.yolov5_version]) + network.set_train(False) param_dict = load_checkpoint(args.ckpt_file) load_param_into_net(network, param_dict) - input_data = Tensor(np.zeros([args.batch_size, 12, int(ts_shape/2), int(ts_shape/2)]), mindspore.float32) + input_data = Tensor(np.zeros([args.batch_size, 12, ts_shape, ts_shape]), mindspore.float32) export(network, input_data, file_name=args.file_name, file_format=args.file_format) + print('==========success export===============') diff --git a/model_zoo/official/cv/yolov5/postprocess.py b/model_zoo/official/cv/yolov5/postprocess.py index f128389c71b..9cd53c2ed0d 100644 --- a/model_zoo/official/cv/yolov5/postprocess.py +++ b/model_zoo/official/cv/yolov5/postprocess.py @@ -83,7 +83,6 @@ class DetectionEngine: self.nms_thresh = args_detection.nms_thresh self.multi_label = args_detection.multi_label self.multi_label_thresh = args_detection.multi_label_thresh - # self.coco_catids = self._coco.getCatIds() self.coco_catIds = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77, 78, 79, 80, @@ -91,18 +90,15 @@ class DetectionEngine: def do_nms_for_results(self): """Get result boxes.""" - # np.save('/opt/disk1/hjc/yolov5_positive_policy/result.npy', self.results) for image_id in self.results: for clsi in self.results[image_id]: dets = self.results[image_id][clsi] dets = np.array(dets) keep_index = self._diou_nms(dets, thresh=self.nms_thresh) - keep_box = [{'image_id': int(image_id), - 'category_id': int(clsi), + keep_box = [{'image_id': int(image_id), 'category_id': int(clsi), 'bbox': list(dets[i][:4].astype(float)), - 'score': dets[i][4].astype(float)} - for i in keep_index] + 'score': dets[i][4].astype(float)} for i in keep_index] self.det_boxes.extend(keep_box) def _nms(self, predicts, threshold): diff --git a/model_zoo/official/cv/yolov5/scripts/run_distribute_train.sh b/model_zoo/official/cv/yolov5/scripts/run_distribute_train.sh index 82214ae19b9..053c8b73b25 100644 --- a/model_zoo/official/cv/yolov5/scripts/run_distribute_train.sh +++ b/model_zoo/official/cv/yolov5/scripts/run_distribute_train.sh @@ -63,6 +63,7 @@ do python train.py \ --data_dir=$DATASET_PATH \ --is_distributed=1 \ + --yolov5_version='yolov5s' \ --lr=0.02 \ --T_max=300 \ --max_epoch=300 \ diff --git a/model_zoo/official/cv/yolov5/scripts/run_eval.sh b/model_zoo/official/cv/yolov5/scripts/run_eval.sh index a092a24945d..600690de2bb 100644 --- a/model_zoo/official/cv/yolov5/scripts/run_eval.sh +++ b/model_zoo/official/cv/yolov5/scripts/run_eval.sh @@ -61,6 +61,7 @@ env > env.log echo "start inferring for device $DEVICE_ID" python eval.py \ --data_dir=$DATASET_PATH \ + --yolov5_version='yolov5s' \ --pretrained=$CHECKPOINT_PATH \ --testing_shape=640 > log.txt 2>&1 & cd .. diff --git a/model_zoo/official/cv/yolov5/scripts/run_standalone_train.sh b/model_zoo/official/cv/yolov5/scripts/run_standalone_train.sh index 4db3ec380e4..0d7d8bf7c76 100644 --- a/model_zoo/official/cv/yolov5/scripts/run_standalone_train.sh +++ b/model_zoo/official/cv/yolov5/scripts/run_standalone_train.sh @@ -58,6 +58,7 @@ env > env.log python train.py \ --data_dir=$DATASET_PATH \ --is_distributed=0 \ + --yolov5_version='yolov5s' \ --lr=0.01 \ --T_max=320 \ --max_epoch=320 \ diff --git a/model_zoo/official/cv/yolov5/src/backbone.py b/model_zoo/official/cv/yolov5/src/backbone.py new file mode 100644 index 00000000000..49339576d98 --- /dev/null +++ b/model_zoo/official/cv/yolov5/src/backbone.py @@ -0,0 +1,172 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""DarkNet model.""" +import mindspore.nn as nn +from mindspore.ops import operations as P + + +class Bottleneck(nn.Cell): + # Standard bottleneck + # ch_in, ch_out, shortcut, groups, expansion + def __init__(self, c1, c2, shortcut=True, e=0.5): + super(Bottleneck, self).__init__() + c_ = int(c2 * e) # hidden channels + self.conv1 = Conv(c1, c_, 1, 1) + self.conv2 = Conv(c_, c2, 3, 1) + self.add = shortcut and c1 == c2 + + def construct(self, x): + c1 = self.conv1(x) + c2 = self.conv2(c1) + out = c2 + if self.add: + out = x + out + return out + + +class BottleneckCSP(nn.Cell): + # CSP Bottleneck with 3 convolutions + def __init__(self, c1, c2, n=1, shortcut=True, e=0.5): + super(BottleneckCSP, self).__init__() + c_ = int(c2 * e) # hidden channels + self.conv1 = Conv(c1, c_, 1, 1) + self.conv2 = Conv(c1, c_, 1, 1) + self.conv3 = Conv(2 * c_, c2, 1) # act=FReLU(c2) + self.m = nn.SequentialCell( + [Bottleneck(c_, c_, shortcut, e=1.0) for _ in range(n)]) + self.concat = P.Concat(axis=1) + + def construct(self, x): + c1 = self.conv1(x) + c2 = self.m(c1) + c3 = self.conv2(x) + c4 = self.concat((c2, c3)) + c5 = self.conv3(c4) + + return c5 + + +class SPP(nn.Cell): + # Spatial pyramid pooling layer used in YOLOv3-SPP + def __init__(self, c1, c2, k=(5, 9, 13)): + super(SPP, self).__init__() + c_ = c1 // 2 # hidden channels + self.conv1 = Conv(c1, c_, 1, 1) + self.conv2 = Conv(c_ * (len(k) + 1), c2, 1, 1) + + self.maxpool1 = nn.MaxPool2d(kernel_size=5, stride=1, pad_mode='same') + self.maxpool2 = nn.MaxPool2d(kernel_size=9, stride=1, pad_mode='same') + self.maxpool3 = nn.MaxPool2d(kernel_size=13, stride=1, pad_mode='same') + self.concat = P.Concat(axis=1) + + def construct(self, x): + c1 = self.conv1(x) + m1 = self.maxpool1(c1) + m2 = self.maxpool2(c1) + m3 = self.maxpool3(c1) + c4 = self.concat((c1, m1, m2, m3)) + c5 = self.conv2(c4) + return c5 + + +class Focus(nn.Cell): + # Focus wh information into c-space + def __init__(self, c1, c2, k=1, s=1, p=None, act=True): + super(Focus, self).__init__() + self.conv = Conv(c1 * 4, c2, k, s, p, act) + + def construct(self, x): + c1 = self.conv(x) + return c1 + + +class SiLU(nn.Cell): + def __init__(self): + super(SiLU, self).__init__() + self.sigmoid = P.Sigmoid() + + def construct(self, x): + return x * self.sigmoid(x) + + +def auto_pad(k, p=None): # kernel, padding + # Pad to 'same' + if p is None: + p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad + return p + + +class Conv(nn.Cell): + # Standard convolution + def __init__(self, c1, c2, k=1, s=1, p=None, + dilation=1, + alpha=0.1, + momentum=0.97, + eps=1e-3, + pad_mode="same", + act=True): # ch_in, ch_out, kernel, stride, padding + super(Conv, self).__init__() + self.padding = auto_pad(k, p) + self.pad_mode = None + if self.padding == 0: + self.pad_mode = 'same' + elif self.padding == 1: + self.pad_mode = 'pad' + self.conv = nn.Conv2d( + c1, + c2, + k, + s, + padding=self.padding, + pad_mode=self.pad_mode, + has_bias=False) + self.bn = nn.BatchNorm2d(c2, momentum=momentum, eps=eps) + self.act = SiLU() if act is True else ( + act if isinstance(act, nn.Cell) else P.Identity()) + + def construct(self, x): + return self.act(self.bn(self.conv(x))) + + +class YOLOv5Backbone(nn.Cell): + def __init__(self, shape): + super(YOLOv5Backbone, self).__init__() + self.focus = Focus(shape[0], shape[1], k=3, s=1) + self.conv1 = Conv(shape[1], shape[2], k=3, s=2) + self.CSP1 = BottleneckCSP(shape[2], shape[2], n=1 * shape[6]) + self.conv2 = Conv(shape[2], shape[3], k=3, s=2) + self.CSP2 = BottleneckCSP(shape[3], shape[3], n=3 * shape[6]) + self.conv3 = Conv(shape[3], shape[4], k=3, s=2) + self.CSP3 = BottleneckCSP(shape[4], shape[4], n=3 * shape[6]) + self.conv4 = Conv(shape[4], shape[5], k=3, s=2) + self.spp = SPP(shape[5], shape[5], k=[5, 9, 13]) + self.CSP4 = BottleneckCSP(shape[5], shape[5], n=1 * shape[6], shortcut=False) + + def construct(self, x): + """construct method""" + c1 = self.focus(x) + c2 = self.conv1(c1) + c3 = self.CSP1(c2) + c4 = self.conv2(c3) + # out + c5 = self.CSP2(c4) + c6 = self.conv3(c5) + # out + c7 = self.CSP3(c6) + c8 = self.conv4(c7) + c9 = self.spp(c8) + # out + c10 = self.CSP4(c9) + return c5, c7, c10 diff --git a/model_zoo/official/cv/yolov5/src/config.py b/model_zoo/official/cv/yolov5/src/config.py index 9f9777bcc99..3414afa90cd 100644 --- a/model_zoo/official/cv/yolov5/src/config.py +++ b/model_zoo/official/cv/yolov5/src/config.py @@ -30,9 +30,22 @@ class ConfigYOLOV5: jitter = 0.3 resize_rate = 10 - multi_scale = [[320, 320], [352, 352], [384, 384], [416, 416], [448, 448], - [480, 480], [512, 512], [544, 544], [576, 576], [608, 608], - [640, 640], [672, 672], [704, 704], [736, 736], [768, 768]] + multi_scale = [ + [320, 320], + [352, 352], + [384, 384], + [416, 416], + [448, 448], + [480, 480], + [512, 512], + [544, 544], + [576, 576], + [608, 608], + [640, 640], + [672, 672], + [704, 704], + [736, 736], + [768, 768]] num_classes = 80 max_box = 150 @@ -51,5 +64,10 @@ class ConfigYOLOV5: (459, 401)] out_channel = 3 * (num_classes + 5) + input_shape = [[3, 32, 64, 128, 256, 512, 1], + [3, 48, 96, 192, 384, 768, 2], + [3, 64, 128, 256, 512, 1024, 3], + [3, 80, 160, 320, 640, 1280, 4]] + # test_param test_img_shape = [640, 640] diff --git a/model_zoo/official/cv/yolov5/src/distributed_sampler.py b/model_zoo/official/cv/yolov5/src/distributed_sampler.py index 5c4a507b9a3..4f4015aff66 100644 --- a/model_zoo/official/cv/yolov5/src/distributed_sampler.py +++ b/model_zoo/official/cv/yolov5/src/distributed_sampler.py @@ -20,6 +20,7 @@ import numpy as np class DistributedSampler: """Distributed sampler.""" + def __init__(self, dataset_size, num_replicas=None, rank=None, shuffle=True): if num_replicas is None: print("***********Setting world_size to 1 since it is not passed in ******************") @@ -39,7 +40,8 @@ class DistributedSampler: # deterministically shuffle based on epoch if self.shuffle: indices = np.random.RandomState(seed=self.epoch).permutation(self.dataset_size) - # np.array type. number from 0 to len(dataset_size)-1, used as index of dataset + # np.array type. number from 0 to len(dataset_size)-1, used as + # index of dataset indices = indices.tolist() self.epoch += 1 # change to list type diff --git a/model_zoo/official/cv/yolov5/src/initializer.py b/model_zoo/official/cv/yolov5/src/initializer.py index 237dcd7c882..2f49dbfd413 100644 --- a/model_zoo/official/cv/yolov5/src/initializer.py +++ b/model_zoo/official/cv/yolov5/src/initializer.py @@ -20,7 +20,7 @@ from mindspore.common import initializer as init from mindspore.common.initializer import Initializer as MeInitializer from mindspore.train.serialization import load_checkpoint, load_param_into_net import mindspore.nn as nn -from .util import load_backbone + def calculate_gain(nonlinearity, param=None): r"""Return the recommended gain value for the given nonlinearity function. @@ -58,7 +58,8 @@ def calculate_gain(nonlinearity, param=None): # True/False are instances of int, hence check above negative_slope = param else: - raise ValueError("negative_slope {} not a valid number".format(param)) + raise ValueError( + "negative_slope {} not a valid number".format(param)) return math.sqrt(2.0 / (1 + negative_slope ** 2)) raise ValueError("Unsupported nonlinearity {}".format(nonlinearity)) @@ -118,7 +119,8 @@ def kaiming_uniform_(arr, a=0, mode='fan_in', nonlinearity='leaky_relu'): fan = _calculate_correct_fan(arr, mode) gain = calculate_gain(nonlinearity, a) std = gain / math.sqrt(fan) - bound = math.sqrt(3.0) * std # Calculate uniform bounds from standard deviation + # Calculate uniform bounds from standard deviation + bound = math.sqrt(3.0) * std return np.random.uniform(-bound, bound, arr.shape) @@ -141,6 +143,7 @@ def _calculate_fan_in_and_fan_out(arr): class KaimingUniform(MeInitializer): """Kaiming uniform initializer.""" + def __init__(self, a=0, mode='fan_in', nonlinearity='leaky_relu'): super(KaimingUniform, self).__init__() self.a = a @@ -156,34 +159,23 @@ def default_recurisive_init(custom_cell): """Initialize parameter.""" for _, cell in custom_cell.cells_and_names(): if isinstance(cell, nn.Conv2d): - cell.weight.set_data(init.initializer(KaimingUniform(a=math.sqrt(5)), - cell.weight.shape, - cell.weight.dtype)) + cell.weight.set_data(init.initializer(KaimingUniform(a=math.sqrt(5)), cell.weight.shape, cell.weight.dtype)) if cell.bias is not None: fan_in, _ = _calculate_fan_in_and_fan_out(cell.weight) bound = 1 / math.sqrt(fan_in) - cell.bias.set_data(init.initializer(init.Uniform(bound), - cell.bias.shape, - cell.bias.dtype)) + cell.bias.set_data(init.initializer(init.Uniform(bound), cell.bias.shape, cell.bias.dtype)) elif isinstance(cell, nn.Dense): - cell.weight.set_data(init.initializer(KaimingUniform(a=math.sqrt(5)), - cell.weight.shape, - cell.weight.dtype)) + cell.weight.set_data(init.initializer(KaimingUniform(a=math.sqrt(5)), cell.weight.shape, cell.weight.dtype)) if cell.bias is not None: fan_in, _ = _calculate_fan_in_and_fan_out(cell.weight) bound = 1 / math.sqrt(fan_in) - cell.bias.set_data(init.initializer(init.Uniform(bound), - cell.bias.shape, - cell.bias.dtype)) + cell.bias.set_data(init.initializer(init.Uniform(bound), cell.bias.shape, cell.bias.dtype)) elif isinstance(cell, (nn.BatchNorm2d, nn.BatchNorm1d)): pass + def load_yolov5_params(args, network): """Load yolov5 backbone parameter from checkpoint.""" - if args.pretrained_backbone: - network = load_backbone(network, args.pretrained_backbone, args) - args.logger.info('load pre-trained backbone {} into network'.format(args.pretrained_backbone)) - if args.resume_yolov5: param_dict = load_checkpoint(args.resume_yolov5) param_dict_new = {} @@ -200,3 +192,20 @@ def load_yolov5_params(args, network): args.logger.info('resume finished') load_param_into_net(network, param_dict_new) args.logger.info('load_model {} success'.format(args.resume_yolov5)) + + if args.pretrained_backbone: + param_dict = load_checkpoint(args.pretrained_backbone) + param_dict_new = {} + for key, values in param_dict.items(): + if key.startswith('moments.'): + continue + elif key.startswith('yolo_network.'): + param_dict_new[key[13:]] = values + args.logger.info('in resume {}'.format(key)) + else: + param_dict_new[key] = values + args.logger.info('in resume {}'.format(key)) + + args.logger.info('pretrained finished') + load_param_into_net(network, param_dict_new) + args.logger.info('load_model {} success'.format(args.pretrained_backbone)) diff --git a/model_zoo/official/cv/yolov5/src/logger.py b/model_zoo/official/cv/yolov5/src/logger.py index d9f924ad96c..23597901712 100644 --- a/model_zoo/official/cv/yolov5/src/logger.py +++ b/model_zoo/official/cv/yolov5/src/logger.py @@ -27,6 +27,7 @@ class LOGGER(logging.Logger): logger_name: String. Logger name. rank: Integer. Rank id. """ + def __init__(self, logger_name, rank=0): super(LOGGER, self).__init__(logger_name) self.rank = rank @@ -65,11 +66,11 @@ class LOGGER(logging.Logger): if self.isEnabledFor(logging.INFO) and self.rank == 0: line_width = 2 important_msg = '\n' - important_msg += ('*'*70 + '\n')*line_width - important_msg += ('*'*line_width + '\n')*2 - important_msg += '*'*line_width + ' '*8 + msg + '\n' - important_msg += ('*'*line_width + '\n')*2 - important_msg += ('*'*70 + '\n')*line_width + important_msg += ('*' * 70 + '\n') * line_width + important_msg += ('*' * line_width + '\n') * 2 + important_msg += '*' * line_width + ' ' * 8 + msg + '\n' + important_msg += ('*' * line_width + '\n') * 2 + important_msg += ('*' * 70 + '\n') * line_width self.info(important_msg, *args, **kwargs) diff --git a/model_zoo/official/cv/yolov5/src/loss.py b/model_zoo/official/cv/yolov5/src/loss.py index 75f08f7f810..6cc09927903 100644 --- a/model_zoo/official/cv/yolov5/src/loss.py +++ b/model_zoo/official/cv/yolov5/src/loss.py @@ -16,8 +16,10 @@ from mindspore.ops import operations as P import mindspore.nn as nn + class ConfidenceLoss(nn.Cell): """Loss for confidence.""" + def __init__(self): super(ConfidenceLoss, self).__init__() self.cross_entropy = P.SigmoidCrossEntropyWithLogits() @@ -32,6 +34,7 @@ class ConfidenceLoss(nn.Cell): class ClassLoss(nn.Cell): """Loss for classification.""" + def __init__(self): super(ClassLoss, self).__init__() self.cross_entropy = P.SigmoidCrossEntropyWithLogits() diff --git a/model_zoo/official/cv/yolov5/src/lr_scheduler.py b/model_zoo/official/cv/yolov5/src/lr_scheduler.py index ceb4e22b384..672fd8d5a6c 100644 --- a/model_zoo/official/cv/yolov5/src/lr_scheduler.py +++ b/model_zoo/official/cv/yolov5/src/lr_scheduler.py @@ -76,7 +76,7 @@ def warmup_cosine_annealing_lr(lr, steps_per_epoch, warmup_epochs, max_epoch, T_ if i < warmup_steps: lr = linear_warmup_lr(i + 1, warmup_steps, base_lr, warmup_init_lr) else: - lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi*last_epoch / T_max)) / 2 + lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi * last_epoch / T_max)) / 2 lr_each_step.append(lr) return np.array(lr_each_step).astype(np.float32) @@ -92,7 +92,7 @@ def warmup_cosine_annealing_lr_V2(lr, steps_per_epoch, warmup_epochs, max_epoch, last_lr = 0 last_epoch_V1 = 0 - T_max_V2 = int(max_epoch*1/3) + T_max_V2 = int(max_epoch * 1 / 3) lr_each_step = [] for i in range(total_steps): @@ -100,13 +100,13 @@ def warmup_cosine_annealing_lr_V2(lr, steps_per_epoch, warmup_epochs, max_epoch, if i < warmup_steps: lr = linear_warmup_lr(i + 1, warmup_steps, base_lr, warmup_init_lr) else: - if i < total_steps*2/3: - lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi*last_epoch / T_max)) / 2 + if i < total_steps * 2 / 3: + lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi * last_epoch / T_max)) / 2 last_lr = lr last_epoch_V1 = last_epoch else: base_lr = last_lr - last_epoch = last_epoch-last_epoch_V1 + last_epoch = last_epoch - last_epoch_V1 lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi * last_epoch / T_max_V2)) / 2 lr_each_step.append(lr) @@ -118,8 +118,8 @@ def warmup_cosine_annealing_lr_sample(lr, steps_per_epoch, warmup_epochs, max_ep start_sample_epoch = 60 step_sample = 2 tobe_sampled_epoch = 60 - end_sampled_epoch = start_sample_epoch + step_sample*tobe_sampled_epoch - max_sampled_epoch = max_epoch+tobe_sampled_epoch + end_sampled_epoch = start_sample_epoch + step_sample * tobe_sampled_epoch + max_sampled_epoch = max_epoch + tobe_sampled_epoch T_max = max_sampled_epoch base_lr = lr @@ -137,7 +137,7 @@ def warmup_cosine_annealing_lr_sample(lr, steps_per_epoch, warmup_epochs, max_ep if i < warmup_steps: lr = linear_warmup_lr(i + 1, warmup_steps, base_lr, warmup_init_lr) else: - lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi*last_epoch / T_max)) / 2 + lr = eta_min + (base_lr - eta_min) * (1. + math.cos(math.pi * last_epoch / T_max)) / 2 lr_each_step.append(lr) assert total_steps == len(lr_each_step) @@ -147,34 +147,17 @@ def warmup_cosine_annealing_lr_sample(lr, steps_per_epoch, warmup_epochs, max_ep def get_lr(args): """generate learning rate.""" if args.lr_scheduler == 'exponential': - lr = warmup_step_lr(args.lr, - args.lr_epochs, - args.steps_per_epoch, - args.warmup_epochs, - args.max_epoch, - gamma=args.lr_gamma, - ) + lr = warmup_step_lr(args.lr, args.lr_epochs, args.steps_per_epoch, args.warmup_epochs, args.max_epoch, + gamma=args.lr_gamma) elif args.lr_scheduler == 'cosine_annealing': - lr = warmup_cosine_annealing_lr(args.lr, - args.steps_per_epoch, - args.warmup_epochs, - args.max_epoch, - args.T_max, - args.eta_min) + lr = warmup_cosine_annealing_lr(args.lr, args.steps_per_epoch, args.warmup_epochs, + args.max_epoch, args.T_max, args.eta_min) elif args.lr_scheduler == 'cosine_annealing_V2': - lr = warmup_cosine_annealing_lr_V2(args.lr, - args.steps_per_epoch, - args.warmup_epochs, - args.max_epoch, - args.T_max, - args.eta_min) + lr = warmup_cosine_annealing_lr_V2(args.lr, args.steps_per_epoch, args.warmup_epochs, + args.max_epoch, args.T_max, args.eta_min) elif args.lr_scheduler == 'cosine_annealing_sample': - lr = warmup_cosine_annealing_lr_sample(args.lr, - args.steps_per_epoch, - args.warmup_epochs, - args.max_epoch, - args.T_max, - args.eta_min) + lr = warmup_cosine_annealing_lr_sample(args.lr, args.steps_per_epoch, args.warmup_epochs, + args.max_epoch, args.T_max, args.eta_min) else: raise NotImplementedError(args.lr_scheduler) return lr diff --git a/model_zoo/official/cv/yolov5/src/transforms.py b/model_zoo/official/cv/yolov5/src/transforms.py index 61609f44c35..e3bbbb89e01 100644 --- a/model_zoo/official/cv/yolov5/src/transforms.py +++ b/model_zoo/official/cv/yolov5/src/transforms.py @@ -22,6 +22,7 @@ from PIL import Image import cv2 import mindspore.dataset.vision.py_transforms as PV + def _rand(a=0., b=1.): return np.random.rand() * (b - a) + a @@ -64,7 +65,7 @@ def statistic_normalize_img(img, statistic_norm): # img: RGB if isinstance(img, Image.Image): img = np.array(img) - img = img/255. + img = img / 255. mean = np.array([0.485, 0.456, 0.406]) std = np.array([0.229, 0.224, 0.225]) if statistic_norm: @@ -137,16 +138,15 @@ def _preprocess_true_boxes(true_boxes, anchors, in_shape, num_classes, max_boxes """ Introduction ------------ - 对训练数据的ground truth box进行预处理 + preprocessing ground truth box Parameters ---------- - true_boxes: ground truth box 形状为[boxes, 5], x_min, y_min, x_max, y_max, class_id + true_boxes: ground truth box shape as [boxes, 5], x_min, y_min, x_max, y_max, class_id """ anchors = np.array(anchors) num_layers = anchors.shape[0] // 3 anchor_mask = [[6, 7, 8], [3, 4, 5], [0, 1, 2]] true_boxes = np.array(true_boxes, dtype='float32') - # input_shape = np.array([in_shape, in_shape], dtype='int32') input_shape = np.array(in_shape, dtype='int32') boxes_xy = (true_boxes[..., 0:2] + true_boxes[..., 2:4]) // 2. # trans to box center point @@ -160,17 +160,14 @@ def _preprocess_true_boxes(true_boxes, anchors, in_shape, num_classes, max_boxes y_true = [np.zeros((grid_shapes[l][0], grid_shapes[l][1], len(anchor_mask[l]), 5 + num_classes), dtype='float32') for l in range(num_layers)] # y_true [gridy, gridx] - # 这里扩充维度是为了后面应用广播计算每个图中所有box的anchor互相之间的iou anchors = np.expand_dims(anchors, 0) anchors_max = anchors / 2. anchors_min = -anchors_max - # 因为之前对box做了padding, 因此需要去除全0行 valid_mask = boxes_wh[..., 0] > 0 wh = boxes_wh[valid_mask] if wh.size != 0: - # 为了应用广播扩充维度 wh = np.expand_dims(wh, -2) - # wh 的shape为[box_num, 1, 2] + # wh shape[box_num, 1, 2] boxes_max = wh / 2. boxes_min = -boxes_max intersect_min = np.maximum(boxes_min, anchors_min) @@ -180,10 +177,8 @@ def _preprocess_true_boxes(true_boxes, anchors, in_shape, num_classes, max_boxes box_area = wh[..., 0] * wh[..., 1] anchor_area = anchors[..., 0] * anchors[..., 1] iou = intersect_area / (box_area + anchor_area - intersect_area) - #topk iou - # 找出和ground truth box的iou最大的anchor box, - # 然后将对应不同比例的负责该ground turth box 的位置置为ground truth box坐标 + # topk iou topk = 4 topk_flag = iou.argsort() topk_flag = topk_flag >= topk_flag.shape[1] - topk @@ -210,7 +205,7 @@ def _preprocess_true_boxes(true_boxes, anchors, in_shape, num_classes, max_boxes y_true[l][j, i, k, 5 + c] = 1 - label_smooth_factor else: y_true[l][j, i, k, 5 + c] = 1. - #best anchor for gt + # best anchor for gt best_anchor = np.argmax(iou, axis=-1) for t, n in enumerate(best_anchor): for l in range(num_layers): @@ -352,9 +347,7 @@ def _choose_candidate_by_constraints(max_trial, input_w, input_h, image_w, image (None, 1), ) else: - constraints = ( - (None, None), - ) + constraints = ((None, None),) # add default candidate candidates = [(0, 0, input_w, input_h)] for constraint in constraints: @@ -411,7 +404,8 @@ def _correct_bbox_by_candidates(candidates, input_w, input_h, image_w, if allow_outside_center: pass else: - t_box = t_box[np.logical_and((t_box[:, 0] + t_box[:, 2])/2. >= 0., (t_box[:, 1] + t_box[:, 3])/2. >= 0.)] + t_box = t_box[ + np.logical_and((t_box[:, 0] + t_box[:, 2]) / 2. >= 0., (t_box[:, 1] + t_box[:, 3]) / 2. >= 0.)] t_box = t_box[np.logical_and((t_box[:, 0] + t_box[:, 2]) / 2. <= input_w, (t_box[:, 1] + t_box[:, 3]) / 2. <= input_h)] @@ -455,24 +449,12 @@ def _data_aug(image, box, jitter, hue, sat, val, image_input_size, max_boxes, flip = _rand() < .5 box_data = np.zeros((max_boxes, 5)) - candidates = _choose_candidate_by_constraints(use_constraints=False, - max_trial=max_trial, - input_w=input_w, - input_h=input_h, - image_w=image_w, - image_h=image_h, - jitter=jitter, - box=box) - box_data, candidate = _correct_bbox_by_candidates(candidates=candidates, - input_w=input_w, - input_h=input_h, - image_w=image_w, - image_h=image_h, - flip=flip, - box=box, - box_data=box_data, - allow_outside_center=True, - max_boxes=max_boxes) + candidates = _choose_candidate_by_constraints(use_constraints=False, max_trial=max_trial, input_w=input_w, + input_h=input_h, image_w=image_w, image_h=image_h, + jitter=jitter, box=box) + box_data, candidate = _correct_bbox_by_candidates(candidates=candidates, input_w=input_w, input_h=input_h, + image_w=image_w, image_h=image_h, flip=flip, box=box, + box_data=box_data, allow_outside_center=True, max_boxes=max_boxes) dx, dy, nw, nh = candidate interp = get_interp_method(interp=10) image = image.resize((nw, nh), pil_image_reshape(interp)) @@ -514,6 +496,7 @@ def reshape_fn(image, img_id, config): class MultiScaleTrans: """Multi scale transform.""" + def __init__(self, config, device_num): self.config = config self.seed = 0 diff --git a/model_zoo/official/cv/yolov5/src/util.py b/model_zoo/official/cv/yolov5/src/util.py index 3e439232133..109057bd36e 100644 --- a/model_zoo/official/cv/yolov5/src/util.py +++ b/model_zoo/official/cv/yolov5/src/util.py @@ -13,8 +13,6 @@ # limitations under the License. # ============================================================================ """Util class or function.""" -from mindspore.train.serialization import load_checkpoint -import mindspore.nn as nn import mindspore.common.dtype as mstype from .yolo import YoloLossBlock @@ -54,65 +52,6 @@ class AverageMeter: return fmtstr.format(**self.__dict__) -def load_backbone(net, ckpt_path, args): - """Load cspdarknet53 backbone checkpoint.""" - param_dict = load_checkpoint(ckpt_path) - yolo_backbone_prefix = 'feature_map.backbone' - darknet_backbone_prefix = 'backbone' - find_param = [] - not_found_param = [] - net.init_parameters_data() - for name, cell in net.cells_and_names(): - if name.startswith(yolo_backbone_prefix): - name = name.replace(yolo_backbone_prefix, darknet_backbone_prefix) - if isinstance(cell, (nn.Conv2d, nn.Dense)): - darknet_weight = '{}.weight'.format(name) - darknet_bias = '{}.bias'.format(name) - if darknet_weight in param_dict: - cell.weight.set_data(param_dict[darknet_weight].data) - find_param.append(darknet_weight) - else: - not_found_param.append(darknet_weight) - if darknet_bias in param_dict: - cell.bias.set_data(param_dict[darknet_bias].data) - find_param.append(darknet_bias) - else: - not_found_param.append(darknet_bias) - elif isinstance(cell, (nn.BatchNorm2d, nn.BatchNorm1d)): - darknet_moving_mean = '{}.moving_mean'.format(name) - darknet_moving_variance = '{}.moving_variance'.format(name) - darknet_gamma = '{}.gamma'.format(name) - darknet_beta = '{}.beta'.format(name) - if darknet_moving_mean in param_dict: - cell.moving_mean.set_data(param_dict[darknet_moving_mean].data) - find_param.append(darknet_moving_mean) - else: - not_found_param.append(darknet_moving_mean) - if darknet_moving_variance in param_dict: - cell.moving_variance.set_data(param_dict[darknet_moving_variance].data) - find_param.append(darknet_moving_variance) - else: - not_found_param.append(darknet_moving_variance) - if darknet_gamma in param_dict: - cell.gamma.set_data(param_dict[darknet_gamma].data) - find_param.append(darknet_gamma) - else: - not_found_param.append(darknet_gamma) - if darknet_beta in param_dict: - cell.beta.set_data(param_dict[darknet_beta].data) - find_param.append(darknet_beta) - else: - not_found_param.append(darknet_beta) - - args.logger.info('================found_param {}========='.format(len(find_param))) - args.logger.info(find_param) - args.logger.info('================not_found_param {}========='.format(len(not_found_param))) - args.logger.info(not_found_param) - args.logger.info('=====load {} successfully ====='.format(ckpt_path)) - - return net - - def default_wd_filter(x): """default weight decay filter.""" parameter_name = x.name @@ -120,10 +59,12 @@ def default_wd_filter(x): # all bias not using weight decay return False if parameter_name.endswith('.gamma'): - # bn weight bias not using weight decay, be carefully for now x not include BN + # bn weight bias not using weight decay, be carefully for now x not + # include BN return False if parameter_name.endswith('.beta'): - # bn weight bias not using weight decay, be carefully for now x not include BN + # bn weight bias not using weight decay, be carefully for now x not + # include BN return False return True @@ -139,19 +80,23 @@ def get_param_groups(network): # all bias not using weight decay no_decay_params.append(x) elif parameter_name.endswith('.gamma'): - # bn weight bias not using weight decay, be carefully for now x not include BN + # bn weight bias not using weight decay, be carefully for now x not + # include BN no_decay_params.append(x) elif parameter_name.endswith('.beta'): - # bn weight bias not using weight decay, be carefully for now x not include BN + # bn weight bias not using weight decay, be carefully for now x not + # include BN no_decay_params.append(x) else: decay_params.append(x) - return [{'params': no_decay_params, 'weight_decay': 0.0}, {'params': decay_params}] + return [{'params': no_decay_params, 'weight_decay': 0.0}, + {'params': decay_params}] class ShapeRecord: """Log image shape.""" + def __init__(self): self.shape_record = { 416: 0, @@ -178,7 +123,7 @@ class ShapeRecord: def show(self, logger): for key in self.shape_record: rate = self.shape_record[key] / float(self.shape_record['total']) - logger.info('shape {}: {:.2f}%'.format(key, rate*100)) + logger.info('shape {}: {:.2f}%'.format(key, rate * 100)) def keep_loss_fp32(network): diff --git a/model_zoo/official/cv/yolov5/src/yolo.py b/model_zoo/official/cv/yolov5/src/yolo.py index c514fb81c28..245566fe5f9 100644 --- a/model_zoo/official/cv/yolov5/src/yolo.py +++ b/model_zoo/official/cv/yolov5/src/yolo.py @@ -24,29 +24,28 @@ from mindspore.ops import operations as P from mindspore.ops import functional as F from mindspore.ops import composite as C -from src.yolov5_backbone import YOLOv5Backbone, Conv, C3 +from src.backbone import YOLOv5Backbone, Conv, BottleneckCSP from src.config import ConfigYOLOV5 from src.loss import ConfidenceLoss, ClassLoss -class YOLOv5(nn.Cell): - def __init__(self, backbone, out_channel): - super(YOLOv5, self).__init__() - self.out_channel = out_channel +class YOLO(nn.Cell): + def __init__(self, backbone, shape): + super(YOLO, self).__init__() self.backbone = backbone + self.config = ConfigYOLOV5() - self.conv1 = Conv(512, 256, k=1, s=1) # 10 - self.C31 = C3(512, 256, n=1, shortcut=False) # 11 - self.conv2 = Conv(256, 128, k=1, s=1) - self.C32 = C3(256, 128, n=1, shortcut=False) # 13 - self.conv3 = Conv(128, 128, k=3, s=2) - self.C33 = C3(256, 256, n=1, shortcut=False) # 15 - self.conv4 = Conv(256, 256, k=3, s=2) - self.C34 = C3(512, 512, n=1, shortcut=False) # 17 - - self.backblock1 = YoloBlock(128, 255) - self.backblock2 = YoloBlock(256, 255) - self.backblock3 = YoloBlock(512, 255) + self.conv1 = Conv(shape[5], shape[4], k=1, s=1) + self.CSP5 = BottleneckCSP(shape[5], shape[4], n=1*shape[6], shortcut=False) + self.conv2 = Conv(shape[4], shape[3], k=1, s=1) + self.CSP6 = BottleneckCSP(shape[4], shape[3], n=1*shape[6], shortcut=False) + self.conv3 = Conv(shape[3], shape[3], k=3, s=2) + self.CSP7 = BottleneckCSP(shape[4], shape[4], n=1*shape[6], shortcut=False) + self.conv4 = Conv(shape[4], shape[4], k=3, s=2) + self.CSP8 = BottleneckCSP(shape[5], shape[5], n=1*shape[6], shortcut=False) + self.back_block1 = YoloBlock(shape[3], self.config.out_channel) + self.back_block2 = YoloBlock(shape[4], self.config.out_channel) + self.back_block3 = YoloBlock(shape[5], self.config.out_channel) self.concat = P.Concat(axis=1) @@ -57,29 +56,32 @@ class YOLOv5(nn.Cell): feature_map2 is (batch_size, backbone_shape[3], h/16, w/16) feature_map3 is (batch_size, backbone_shape[4], h/32, w/32) """ - img_hight = P.Shape()(x)[2] * 2 + img_height = P.Shape()(x)[2] * 2 img_width = P.Shape()(x)[3] * 2 - backbone4, backbone6, backbone9 = self.backbone(x) + feature_map1, feature_map2, feature_map3 = self.backbone(x) - cv1 = self.conv1(backbone9) # 10 - ups1 = P.ResizeNearestNeighbor((img_hight / 16, img_width / 16))(cv1) - concat1 = self.concat((ups1, backbone6)) - bcsp1 = self.C31(concat1) # 13 - cv2 = self.conv2(bcsp1) - ups2 = P.ResizeNearestNeighbor((img_hight / 8, img_width / 8))(cv2) # 15 - concat2 = self.concat((ups2, backbone4)) - bcsp2 = self.C32(concat2) # 17 - cv3 = self.conv3(bcsp2) + c1 = self.conv1(feature_map3) + ups1 = P.ResizeNearestNeighbor((img_height // 16, img_width // 16))(c1) + c2 = self.concat((ups1, feature_map2)) + c3 = self.CSP5(c2) + c4 = self.conv2(c3) + ups2 = P.ResizeNearestNeighbor((img_height // 8, img_width // 8))(c4) + c5 = self.concat((ups2, feature_map1)) + # out + c6 = self.CSP6(c5) + c7 = self.conv3(c6) - concat3 = self.concat((cv3, cv2)) - bcsp3 = self.C33(concat3) # 20 - cv4 = self.conv4(bcsp3) - concat4 = self.concat((cv4, cv1)) - bcsp4 = self.C34(concat4) # 23 - small_object_output = self.backblock1(bcsp2) # h/8, w/8 - medium_object_output = self.backblock2(bcsp3) # h/16, w/16 - big_object_output = self.backblock3(bcsp4) # h/32, w/32 + c8 = self.concat((c7, c4)) + # out + c9 = self.CSP7(c8) + c10 = self.conv4(c9) + c11 = self.concat((c10, c1)) + # out + c12 = self.CSP8(c11) + small_object_output = self.back_block1(c6) + medium_object_output = self.back_block2(c9) + big_object_output = self.back_block3(c12) return small_object_output, medium_object_output, big_object_output @@ -98,19 +100,17 @@ class YoloBlock(nn.Cell): YoloBlock(12, 255) """ - def __init__(self, in_channels, out_channels): super(YoloBlock, self).__init__() - self.cv = nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, has_bias=True) + self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, has_bias=True) def construct(self, x): """construct method""" - out = self.cv(x) + out = self.conv(x) return out - class DetectionBlock(nn.Cell): """ YOLOv5 detection Network. It will finally output the detection result. @@ -146,13 +146,14 @@ class DetectionBlock(nn.Cell): raise KeyError("Invalid scale value for DetectionBlock") self.anchors = Tensor([self.config.anchor_scales[i] for i in idx], ms.float32) self.num_anchors_per_scale = 3 - self.num_attrib = 4 + 1 + self.config.num_classes + self.num_attrib = 4+1+self.config.num_classes self.lambda_coord = 1 self.sigmoid = nn.Sigmoid() self.reshape = P.Reshape() self.tile = P.Tile() self.concat = P.Concat(axis=-1) + self.pow = P.Pow() self.conf_training = is_training def construct(self, x, input_shape): @@ -190,6 +191,7 @@ class DetectionBlock(nn.Cell): P.Cast()(F.tuple_to_array((grid_size[1], grid_size[0])), ms.float32) # box_wh is w->h box_wh = P.Exp()(box_wh) * self.anchors / input_shape + box_confidence = self.sigmoid(box_confidence) box_probs = self.sigmoid(box_probs) @@ -200,7 +202,6 @@ class DetectionBlock(nn.Cell): class Iou(nn.Cell): """Calculate the iou of boxes""" - def __init__(self): super(Iou, self).__init__() self.min = P.Minimum() @@ -214,8 +215,8 @@ class Iou(nn.Cell): """ box1_xy = box1[:, :, :, :, :, :2] box1_wh = box1[:, :, :, :, :, 2:4] - box1_mins = box1_xy - box1_wh / F.scalar_to_array(2.0) # topLeft - box1_maxs = box1_xy + box1_wh / F.scalar_to_array(2.0) # rightDown + box1_mins = box1_xy - box1_wh / F.scalar_to_array(2.0) # topLeft + box1_maxs = box1_xy + box1_wh / F.scalar_to_array(2.0) # rightDown box2_xy = box2[:, :, :, :, :, :2] box2_wh = box2[:, :, :, :, :, 2:4] @@ -239,7 +240,6 @@ class YoloLossBlock(nn.Cell): """ Loss block cell of YOLOV5 network. """ - def __init__(self, scale, config=ConfigYOLOV5()): super(YoloLossBlock, self).__init__() self.config = config @@ -261,7 +261,7 @@ class YoloLossBlock(nn.Cell): self.class_loss = ClassLoss() self.reduce_sum = P.ReduceSum() - self.giou = Giou() + self.g_iou = GIou() def construct(self, prediction, pred_xy, pred_wh, y_true, gt_box, input_shape): """ @@ -315,15 +315,15 @@ class YoloLossBlock(nn.Cell): pred_boxes_me = P.Reshape()(pred_boxes_me, (-1, 4)) true_boxes_me = xywh2x1y1x2y2(true_boxes) true_boxes_me = P.Reshape()(true_boxes_me, (-1, 4)) - ciou = self.giou(pred_boxes_me, true_boxes_me) - ciou_loss = object_mask_me * box_loss_scale_me * (1 - ciou) - ciou_loss_me = self.reduce_sum(ciou_loss, ()) - loss = ciou_loss_me * 4 + confidence_loss + class_loss + c_iou = self.g_iou(pred_boxes_me, true_boxes_me) + c_iou_loss = object_mask_me * box_loss_scale_me * (1 - c_iou) + c_iou_loss_me = self.reduce_sum(c_iou_loss, ()) + loss = c_iou_loss_me * 4 + confidence_loss + class_loss batch_size = P.Shape()(prediction)[0] return loss / batch_size -class YOLOV5s(nn.Cell): +class YOLOV5(nn.Cell): """ YOLOV5 network. @@ -337,13 +337,13 @@ class YOLOV5s(nn.Cell): YOLOV5s(True) """ - def __init__(self, is_training): - super(YOLOV5s, self).__init__() + def __init__(self, is_training, version=0): + super(YOLOV5, self).__init__() self.config = ConfigYOLOV5() # YOLOv5 network - self.feature_map = YOLOv5(backbone=YOLOv5Backbone(), - out_channel=self.config.out_channel) + self.shape = self.config.input_shape[version] + self.feature_map = YOLO(backbone=YOLOv5Backbone(shape=self.shape), shape=self.shape) # prediction on the default anchor boxes self.detect_1 = DetectionBlock('l', is_training=is_training) @@ -364,18 +364,17 @@ class YOLOV5s_Infer(nn.Cell): YOLOV5 Infer. """ - def __init__(self, inputshape): + def __init__(self, input_shape, version=0): super(YOLOV5s_Infer, self).__init__() - self.network = YOLOV5s(is_training=False) - self.inputshape = inputshape + self.network = YOLOV5(is_training=False, version=version) + self.input_shape = input_shape def construct(self, x): - return self.network(x, self.inputshape) + return self.network(x, self.input_shape) class YoloWithLossCell(nn.Cell): """YOLOV5 loss.""" - def __init__(self, network): super(YoloWithLossCell, self).__init__() self.yolo_network = network @@ -398,7 +397,6 @@ class YoloWithLossCell(nn.Cell): class TrainingWrapper(nn.Cell): """Training wrapper.""" - def __init__(self, network, optimizer, sens=1.0): super(TrainingWrapper, self).__init__(auto_prefix=False) self.network = network @@ -427,15 +425,13 @@ class TrainingWrapper(nn.Cell): grads = self.grad(self.network, weights)(*args, sens) if self.reducer_flag: grads = self.grad_reducer(grads) - self.optimizer(grads) - return loss + return F.depend(loss, self.optimizer(grads)) -class Giou(nn.Cell): +class GIou(nn.Cell): """Calculating giou""" - def __init__(self): - super(Giou, self).__init__() + super(GIou, self).__init__() self.cast = P.Cast() self.reshape = P.Reshape() self.min = P.Minimum() diff --git a/model_zoo/official/cv/yolov5/train.py b/model_zoo/official/cv/yolov5/train.py index aad1b659145..8f6c42cc38b 100644 --- a/model_zoo/official/cv/yolov5/train.py +++ b/model_zoo/official/cv/yolov5/train.py @@ -19,135 +19,126 @@ import argparse import datetime import mindspore as ms from mindspore.context import ParallelMode -from mindspore.nn.optim.momentum import Momentum +from mindspore.nn import Momentum from mindspore import Tensor from mindspore import context from mindspore.communication.management import init, get_rank, get_group_size from mindspore.train.callback import ModelCheckpoint, RunContext from mindspore.train.callback import _InternalCallbackParam, CheckpointConfig -from src.yolo import YOLOV5s, YoloWithLossCell, TrainingWrapper +from src.yolo import YOLOV5, YoloWithLossCell, TrainingWrapper from src.logger import get_logger from src.util import AverageMeter, get_param_groups from src.lr_scheduler import get_lr from src.yolo_dataset import create_yolo_dataset from src.initializer import default_recurisive_init, load_yolov5_params from src.config import ConfigYOLOV5 + ms.set_seed(1) +parser = argparse.ArgumentParser('mindspore coco training') -def parse_args(cloud_args=None): - """Parse train arguments.""" - parser = argparse.ArgumentParser('mindspore coco training') +# device related +parser.add_argument('--device_target', type=str, default='Ascend', help='device where the code will be implemented.') - # device related - parser.add_argument('--device_target', type=str, default='Ascend', - help='device where the code will be implemented.') +# dataset related +parser.add_argument('--data_dir', default='/data/coco', type=str, help='Train dataset directory.') +parser.add_argument('--per_batch_size', default=32, type=int, help='Batch size for Training. Default: 8') - # dataset related - parser.add_argument('--data_dir', type=str, help='Train dataset directory.') - parser.add_argument('--per_batch_size', default=8, type=int, help='Batch size for Training. Default: 8') +# network related +parser.add_argument('--yolov5_version', default='yolov5s', type=str, + help='The version of YOLOv5, options: yolov5s, yolov5m, yolov5l, yolov5x') +parser.add_argument('--pretrained_backbone', default='', type=str, help='The pretrained file of yolov5. Default: "".') +parser.add_argument('--resume_yolov5', default='', type=str, + help='The ckpt file of YOLOv5, which used to fine tune. Default: ""') - # network related - parser.add_argument('--pretrained_backbone', default='', type=str, - help='The backbone file of YOLOv5. Default: "".') - parser.add_argument('--resume_yolov5', default='', type=str, - help='The ckpt file of YOLOv5, which used to fine tune. Default: ""') +# optimizer and lr related +parser.add_argument('--lr_scheduler', default='cosine_annealing', type=str, + help='Learning rate scheduler, options: exponential, cosine_annealing. Default: exponential') +parser.add_argument('--lr', default=0.013, type=float, help='Learning rate. Default: 0.01') +parser.add_argument('--lr_epochs', type=str, default='220,250', + help='Epoch of changing of lr changing, split with ",". Default: 220,250') +parser.add_argument('--lr_gamma', type=float, default=0.1, + help='Decrease lr by a factor of exponential lr_scheduler. Default: 0.1') +parser.add_argument('--eta_min', type=float, default=0., help='Eta_min in cosine_annealing scheduler. Default: 0') +parser.add_argument('--T_max', type=int, default=300, help='T-max in cosine_annealing scheduler. Default: 320') +parser.add_argument('--max_epoch', type=int, default=300, help='Max epoch num to train the model. Default: 320') +parser.add_argument('--warmup_epochs', default=20, type=float, help='Warmup epochs. Default: 0') +parser.add_argument('--weight_decay', type=float, default=0.0005, help='Weight decay factor. Default: 0.0005') +parser.add_argument('--momentum', type=float, default=0.9, help='Momentum. Default: 0.9') - # optimizer and lr related - parser.add_argument('--lr_scheduler', default='cosine_annealing', type=str, - help='Learning rate scheduler, options: exponential, cosine_annealing. Default: exponential') - parser.add_argument('--lr', default=0.013, type=float, help='Learning rate. Default: 0.01') - parser.add_argument('--lr_epochs', type=str, default='220,250', - help='Epoch of changing of lr changing, split with ",". Default: 220,250') - parser.add_argument('--lr_gamma', type=float, default=0.1, - help='Decrease lr by a factor of exponential lr_scheduler. Default: 0.1') - parser.add_argument('--eta_min', type=float, default=0., help='Eta_min in cosine_annealing scheduler. Default: 0') - parser.add_argument('--T_max', type=int, default=300, help='T-max in cosine_annealing scheduler. Default: 320') - parser.add_argument('--max_epoch', type=int, default=300, help='Max epoch num to train the model. Default: 320') - parser.add_argument('--warmup_epochs', default=20, type=float, help='Warmup epochs. Default: 0') - parser.add_argument('--weight_decay', type=float, default=0.0005, help='Weight decay factor. Default: 0.0005') - parser.add_argument('--momentum', type=float, default=0.9, help='Momentum. Default: 0.9') +# loss related +parser.add_argument('--loss_scale', type=int, default=1024, help='Static loss scale. Default: 1024') +parser.add_argument('--label_smooth', type=int, default=0, help='Whether to use label smooth in CE. Default:0') +parser.add_argument('--label_smooth_factor', type=float, default=0.1, + help='Smooth strength of original one-hot. Default: 0.1') - # loss related - parser.add_argument('--loss_scale', type=int, default=1024, help='Static loss scale. Default: 1024') - parser.add_argument('--label_smooth', type=int, default=0, help='Whether to use label smooth in CE. Default:0') - parser.add_argument('--label_smooth_factor', type=float, default=0.1, - help='Smooth strength of original one-hot. Default: 0.1') +# logging related +parser.add_argument('--log_interval', type=int, default=100, help='Logging interval steps. Default: 100') +parser.add_argument('--ckpt_path', type=str, default='outputs/', help='Checkpoint save location. Default: outputs/') +parser.add_argument('--ckpt_interval', type=int, default=None, help='Save checkpoint interval. Default: None') - # logging related - parser.add_argument('--log_interval', type=int, default=100, help='Logging interval steps. Default: 100') - parser.add_argument('--ckpt_path', type=str, default='outputs/', help='Checkpoint save location. Default: outputs/') - parser.add_argument('--ckpt_interval', type=int, default=10, help='Save checkpoint interval. Default: 10') +parser.add_argument('--is_save_on_master', type=int, default=1, + help='Save ckpt on master or all rank, 1 for master, 0 for all ranks. Default: 1') - parser.add_argument('--is_save_on_master', type=int, default=1, - help='Save ckpt on master or all rank, 1 for master, 0 for all ranks. Default: 1') +# distributed related +parser.add_argument('--is_distributed', type=int, default=0, + help='Distribute train or not, 1 for yes, 0 for no. Default: 1') +parser.add_argument('--rank', type=int, default=0, help='Local rank of distributed. Default: 0') +parser.add_argument('--group_size', type=int, default=1, help='World size of device. Default: 1') - # distributed related - parser.add_argument('--is_distributed', type=int, default=1, - help='Distribute train or not, 1 for yes, 0 for no. Default: 1') - parser.add_argument('--rank', type=int, default=0, help='Local rank of distributed. Default: 0') - parser.add_argument('--group_size', type=int, default=1, help='World size of device. Default: 1') +# roma obs +parser.add_argument('--train_url', type=str, default="", help='train url') +# profiler init +parser.add_argument('--need_profiler', type=int, default=0, + help='Whether use profiler. 0 for no, 1 for yes. Default: 0') - # roma obs - parser.add_argument('--train_url', type=str, default="", help='train url') - # profiler init - parser.add_argument('--need_profiler', type=int, default=0, - help='Whether use profiler. 0 for no, 1 for yes. Default: 0') +# reset default config +parser.add_argument('--training_shape', type=str, default="", help='Fix training shape. Default: ""') +parser.add_argument('--resize_rate', type=int, default=10, help='Resize rate for multi-scale training. Default: None') +parser.add_argument('--is_modelArts', type=int, default=0, + help='Trainning in modelArts or not, 1 for yes, 0 for no. Default: 0') - # reset default config - parser.add_argument('--training_shape', type=str, default="", help='Fix training shape. Default: ""') - parser.add_argument('--resize_rate', type=int, default=10, - help='Resize rate for multi-scale training. Default: None') +args, _ = parser.parse_known_args() - args, _ = parser.parse_known_args() - args = merge_args(args, cloud_args) - if args.lr_scheduler == 'cosine_annealing' and args.max_epoch > args.T_max: - args.T_max = args.max_epoch +if args.lr_scheduler == 'cosine_annealing' and args.max_epoch > args.T_max: + args.T_max = args.max_epoch - args.lr_epochs = list(map(int, args.lr_epochs.split(','))) +args.lr_epochs = list(map(int, args.lr_epochs.split(','))) + +if args.is_modelArts: args.data_root = os.path.join(args.data_dir, 'train2017') - args.annFile = os.path.join(args.data_dir, 'annotations/instances_train2017.json') + args.annFile = os.path.join(args.data_dir, 'annotations') + outputs_dir = os.path.join('/cache', args.ckpt_path) +else: + args.data_root = os.path.join(args.data_dir, 'train2017') + args.annFile = os.path.join( + args.data_dir, 'annotations/instances_train2017.json') + outputs_dir = args.ckpt_path - devid = int(os.getenv('DEVICE_ID', '0')) - context.set_context(mode=context.GRAPH_MODE, enable_auto_mixed_precision=True, - device_target=args.device_target, save_graphs=False, device_id=devid) - # init distributed - if args.is_distributed: - if args.device_target == "Ascend": - init() - else: - init("nccl") - args.rank = get_rank() - args.group_size = get_group_size() - - # select for master rank save ckpt or all rank save, compatible for model parallel - args.rank_save_ckpt_flag = 0 - if args.is_save_on_master: - if args.rank == 0: - args.rank_save_ckpt_flag = 1 +deviced = int(os.getenv('DEVICE_ID', '0')) +context.set_context(mode=context.GRAPH_MODE, enable_auto_mixed_precision=True, device_target=args.device_target, + save_graphs=False, device_id=deviced) +# init distributed +if args.is_distributed: + if args.device_target == "Ascend": + init() else: + init("nccl") + args.rank = get_rank() + args.group_size = get_group_size() + +args.rank_save_ckpt_flag = 0 +if args.is_save_on_master: + if args.rank == 0: args.rank_save_ckpt_flag = 1 +else: + args.rank_save_ckpt_flag = 1 - # logger - args.outputs_dir = os.path.join(args.ckpt_path, - datetime.datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S')) - args.logger = get_logger(args.outputs_dir, args.rank) - args.logger.save_args(args) - - return args - -def merge_args(args, cloud_args): - args_dict = vars(args) - if isinstance(cloud_args, dict): - for key in cloud_args.keys(): - val = cloud_args[key] - if key in args_dict and val: - arg_type = type(args_dict[key]) - if arg_type is not type(None): - val = arg_type(val) - args_dict[key] = val - return args +# logger +args.outputs_dir = os.path.join(outputs_dir, datetime.datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S')) +args.logger = get_logger(args.outputs_dir, args.rank) +args.logger.save_args(args) def convert_training_shape(args_training_shape): @@ -155,111 +146,114 @@ def convert_training_shape(args_training_shape): return training_shape -def train(cloud_args=None): - args = parse_args(cloud_args) - loss_meter = AverageMeter('loss') +loss_meter = AverageMeter('loss') - context.reset_auto_parallel_context() - parallel_mode = ParallelMode.STAND_ALONE - degree = 1 - if args.is_distributed: - parallel_mode = ParallelMode.DATA_PARALLEL - degree = get_group_size() - context.set_auto_parallel_context(parallel_mode=parallel_mode, gradients_mean=True, device_num=degree) +if args.is_modelArts: + import moxing as mox + local_data_url = os.path.join('/cache/data', str(args.rank)) + local_annFile = os.path.join('/cache/data', str(args.rank)) + mox.file.copy_parallel(args.data_root, local_data_url) + args.data_root = local_data_url - network = YOLOV5s(is_training=True) - # default is kaiming-normal - default_recurisive_init(network) - load_yolov5_params(args, network) + mox.file.copy_parallel(args.annFile, local_annFile) + args.annFile = os.path.join(local_data_url, 'instances_train2017.json') - network = YoloWithLossCell(network) - config = ConfigYOLOV5() +context.reset_auto_parallel_context() +parallel_mode = ParallelMode.STAND_ALONE +degree = 1 +if args.is_distributed: + parallel_mode = ParallelMode.DATA_PARALLEL + degree = get_group_size() +context.set_auto_parallel_context(parallel_mode=parallel_mode, gradients_mean=True, device_num=degree) - config.label_smooth = args.label_smooth - config.label_smooth_factor = args.label_smooth_factor +dict_version = {'yolov5s': 0, 'yolov5m': 1, 'yolov5l': 2, 'yolov5x': 3} +network = YOLOV5(is_training=True, version=dict_version[args.yolov5_version]) +# default is kaiming-normal +default_recurisive_init(network) +load_yolov5_params(args, network) - if args.training_shape: - config.multi_scale = [convert_training_shape(args.training_shape)] - if args.resize_rate: - config.resize_rate = args.resize_rate +network = YoloWithLossCell(network) +config = ConfigYOLOV5() - ds, data_size = create_yolo_dataset(image_dir=args.data_root, anno_path=args.annFile, is_training=True, - batch_size=args.per_batch_size, max_epoch=args.max_epoch, - device_num=args.group_size, rank=args.rank, config=config) - args.logger.info('Finish loading dataset') +config.label_smooth = args.label_smooth +config.label_smooth_factor = args.label_smooth_factor - args.steps_per_epoch = int(data_size / args.per_batch_size / args.group_size) +if args.training_shape: + config.multi_scale = [convert_training_shape(args.training_shape)] +if args.resize_rate: + config.resize_rate = args.resize_rate - if not args.ckpt_interval: - args.ckpt_interval = args.steps_per_epoch +ds, data_size = create_yolo_dataset(image_dir=args.data_root, anno_path=args.annFile, is_training=True, + batch_size=args.per_batch_size, max_epoch=args.max_epoch, + device_num=args.group_size, rank=args.rank, config=config) - lr = get_lr(args) +args.logger.info('Finish loading dataset') - opt = Momentum(params=get_param_groups(network), - learning_rate=Tensor(lr), - momentum=args.momentum, - weight_decay=args.weight_decay, - loss_scale=args.loss_scale) +args.steps_per_epoch = int(data_size / args.per_batch_size / args.group_size) - network = TrainingWrapper(network, opt, args.loss_scale // 2) - network.set_train() +if not args.ckpt_interval: + args.ckpt_interval = args.steps_per_epoch + +lr = get_lr(args) + +opt = Momentum(params=get_param_groups(network), momentum=args.momentum, learning_rate=Tensor(lr), + weight_decay=args.weight_decay, loss_scale=args.loss_scale) + +network = TrainingWrapper(network, opt, args.loss_scale // 2) +network.set_train() + +if args.rank_save_ckpt_flag: + # checkpoint save + ckpt_max_num = args.max_epoch * args.steps_per_epoch // args.ckpt_interval + ckpt_config = CheckpointConfig(save_checkpoint_steps=args.ckpt_interval, keep_checkpoint_max=1) + save_ckpt_path = os.path.join(args.outputs_dir, 'ckpt_' + str(args.rank) + '/') + ckpt_cb = ModelCheckpoint(config=ckpt_config, directory=save_ckpt_path, prefix='{}'.format(args.rank)) + cb_params = _InternalCallbackParam() + cb_params.train_network = network + cb_params.epoch_num = ckpt_max_num + cb_params.cur_epoch_num = 1 + run_context = RunContext(cb_params) + ckpt_cb.begin(run_context) + +old_progress = -1 +t_end = time.time() +data_loader = ds.create_dict_iterator(output_numpy=True, num_epochs=1) + +for i, data in enumerate(data_loader): + images = data["image"] + input_shape = images.shape[2:4] + images = Tensor.from_numpy(images) + batch_y_true_0 = Tensor.from_numpy(data['bbox1']) + batch_y_true_1 = Tensor.from_numpy(data['bbox2']) + batch_y_true_2 = Tensor.from_numpy(data['bbox3']) + batch_gt_box0 = Tensor.from_numpy(data['gt_box1']) + batch_gt_box1 = Tensor.from_numpy(data['gt_box2']) + batch_gt_box2 = Tensor.from_numpy(data['gt_box3']) + input_shape = Tensor(tuple(input_shape[::-1]), ms.float32) + loss = network(images, batch_y_true_0, batch_y_true_1, batch_y_true_2, batch_gt_box0, batch_gt_box1, + batch_gt_box2, input_shape) + loss_meter.update(loss.asnumpy()) if args.rank_save_ckpt_flag: - # checkpoint save - ckpt_max_num = args.max_epoch * args.steps_per_epoch // args.ckpt_interval - ckpt_config = CheckpointConfig(save_checkpoint_steps=args.ckpt_interval, - keep_checkpoint_max=ckpt_max_num) - save_ckpt_path = os.path.join(args.outputs_dir, 'ckpt_' + str(args.rank) + '/') - ckpt_cb = ModelCheckpoint(config=ckpt_config, - directory=save_ckpt_path, - prefix='{}'.format(args.rank)) - cb_params = _InternalCallbackParam() - cb_params.train_network = network - cb_params.epoch_num = ckpt_max_num - cb_params.cur_epoch_num = 1 - run_context = RunContext(cb_params) - ckpt_cb.begin(run_context) + # ckpt progress + cb_params.cur_step_num = i + 1 # current step number + cb_params.batch_num = i + 2 + ckpt_cb.step_end(run_context) - old_progress = -1 - t_end = time.time() - data_loader = ds.create_dict_iterator(output_numpy=True, num_epochs=1) + if i % args.log_interval == 0: + time_used = time.time() - t_end + epoch = int(i / args.steps_per_epoch) + fps = args.per_batch_size * (i - old_progress) * args.group_size / time_used + if args.rank == 0: + args.logger.info('epoch[{}], iter[{}], {}, fps:{:.2f} imgs/sec, ' + 'lr:{}'.format(epoch, i, loss_meter, fps, lr[i])) + t_end = time.time() + loss_meter.reset() + old_progress = i - for i, data in enumerate(data_loader): - images = data["image"] - input_shape = images.shape[2:4] - images = Tensor.from_numpy(images) - batch_y_true_0 = Tensor.from_numpy(data['bbox1']) - batch_y_true_1 = Tensor.from_numpy(data['bbox2']) - batch_y_true_2 = Tensor.from_numpy(data['bbox3']) - batch_gt_box0 = Tensor.from_numpy(data['gt_box1']) - batch_gt_box1 = Tensor.from_numpy(data['gt_box2']) - batch_gt_box2 = Tensor.from_numpy(data['gt_box3']) - input_shape = Tensor(tuple(input_shape[::-1]), ms.float32) - loss = network(images, batch_y_true_0, batch_y_true_1, batch_y_true_2, batch_gt_box0, batch_gt_box1, - batch_gt_box2, input_shape) - loss_meter.update(loss.asnumpy()) + if (i + 1) % args.steps_per_epoch == 0 and args.rank_save_ckpt_flag: + cb_params.cur_epoch_num += 1 - if args.rank_save_ckpt_flag: - # ckpt progress - cb_params.cur_step_num = i + 1 # current step number - cb_params.batch_num = i + 2 - ckpt_cb.step_end(run_context) - - if i % args.log_interval == 0: - time_used = time.time() - t_end - epoch = int(i / args.steps_per_epoch) - fps = args.per_batch_size * (i - old_progress) * args.group_size / time_used - if args.rank == 0: - args.logger.info( - 'epoch[{}], iter[{}], {}, fps:{:.2f} imgs/sec, lr:{}'.format(epoch, i, loss_meter, fps, lr[i])) - t_end = time.time() - loss_meter.reset() - old_progress = i - - if (i + 1) % args.steps_per_epoch == 0 and args.rank_save_ckpt_flag: - cb_params.cur_epoch_num += 1 - - args.logger.info('==========end training===============') - -if __name__ == "__main__": - train() +if args.is_modelArts: + mox.file.copy_parallel(src_url='/cache/outputs/', dst_url='obs://hit-cyf/yolov5_npu/outputs/') +args.logger.info('==========end training===============')