new add of fastscnn

This commit is contained in:
谭华林 2021-07-05 17:39:10 +08:00
parent f8f1f0f84d
commit 10eb574da6
27 changed files with 3389 additions and 0 deletions

View File

@ -0,0 +1,360 @@
# FastSCNN
<!-- TOC -->
- [FastSCNN](#FastSCNN)
- [FastSCNN介绍](#FastSCNN介绍)
- [模型结构](#模型结构)
- [数据集](#数据集)
- [环境要求](#环境要求)
- [快速入门](#快速入门)
- [脚本说明](#脚本说明)
- [脚本及样例代码](#脚本及样例代码)
- [脚本参数](#脚本参数)
- [训练过程](#训练过程)
- [训练](#训练)
- [评估过程](#评估过程)
- [评估](#评估)
- [310推理](#310推理)
- [模型描述](#模型描述)
- [性能](#性能)
- [评估性能](#评估性能)
- [随机情况说明](#随机情况说明)
- [ModelZoo主页](#modelzoo主页)
<!-- /TOC -->
## FastSCNN介绍
FastSCNN 于 2019 年发表在 BMVC 是英国剑桥大学与东芝欧洲研究院联合研究的快速语义分割算法。它在高分辨率1024×2048图像上的 **实时语义分割** 能到达 123.5 FPS 的帧率与 68% 的准确率。作者指出由于 Fast-SCNN 的参数量很小,所以积极的使用数据增强技术不太可能会带来过拟合。作者还通过实验证明,对于 Fast-SCNN 这种小型网络,与大型网络的趋势相反,仅通过预训练或其他带有粗略标记的训练数据,对于最后的分割结果精度上提升效果不明显,而在训练网络时,多训练几个周期就能达到和有预训练辅助一样的分割精度。
[论文](https://arxiv.org/abs/1902.04502)Poudel R , Liwicki S , Cipolla R . Fast-SCNN: Fast Semantic Segmentation Network[J]. 2019.
## 模型结构
Fast-SCNN 包括学习下采样模块、精细化全局特征提取模块、特征融合模块以及标准分类器四个部分。
下采样学习模块:包括三个卷积层,第一层因为输入图片是三通道的原因,采用普通卷积计算,其他两层都是深度可分离卷积;
全局特征提取模块:使用 MobileNetV2 中的高效瓶颈残差块,将其中卷积都换成深度可分离卷积层,最后加入一个金字塔池化模块聚合基于不同区域的上下文信息;
特征融合模块:用于融合 2 个分支的输出特征,与 ICNet 和 ContextNet 相同的是,作者倾向于简单添加功能以确保效率;
分类器:两个深度可分离卷积层加上一个逐点卷积。
## 数据集
数据集:[**Cityscapes**](<https://www.cityscapes-dataset.com/>)
Cityscapes 数据集,即城市景观数据集,包含来自 50 个不同城市的街道场景中记录的多种立体视频序列。
该数据集包含如下images_base 和 annotations_base 分别对应着文件夹 leftImg8bit5,030 items, totalling 11.6 GBfactually 5000 items和 gtFine30,030 items, totalling 1.1 GB。里面都包含三个文件夹train、val、test。总共 5000 张精细标注2975 张训练图500 张验证图和 1525 张测试图。
## 环境要求
- 硬件Ascend/ModelArts
- 准备Ascend或ModelArts处理器搭建硬件环境。
- 框架
- [MindSpore](https://www.mindspore.cn/install)
- 如需查看详情,请参见如下资源:
- [MindSpore教程](https://www.mindspore.cn/tutorials/zh-CN/master/index.html)
- [MindSpore Python API](https://www.mindspore.cn/docs/api/zh-CN/master/api_python/mindspore.html)
## 快速入门
通过官方网站安装 MindSpore 后,您可以按照如下步骤进行训练和评估:
```python
#通过 python 命令行运行单卡训练脚本。
python train.py \
--dataset=xxx/dataset/ \
--base_size=1024 \
--epochs=1000 \
--batch_size=2 \
--lr=0.001 \
--use_modelarts=0 \
--output_path=./outputs/ \
--is_distributed=0 > log.txt 2>&1 &
#通过 bash 命令启动单卡训练。
bash ./scripts/run_train.sh [train_code_path] [dataset] [epochs] [batch_size] [lr] [output_path]
#Ascend多卡训练
bash ./scripts/run_distribute_train.sh [train_code_path] [dataset] [epochs] [batch_size] [lr] [output_path]
# 通过 python 命令行运行推理脚本。
# resume_path 指 ckpt 所在目录,为了兼容 modelarts将其拆分为了 “路径” 与 “文件名”
python eval.py \
--dataset=xxx/dataset/ \
--resume_path=xxx/ \
--resume_name=fastscnn.ckpt \
--output_path=./outputs/ \
--is_distributed=0 > log.txt 2>&1 &
#通过 bash 命令启动推理。
bash ./scripts/run_eval.sh [train_code_path] [dataset] [resume_path] [resume_name] [output_path]
```
Ascend训练生成[RANK_TABLE_FILE](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/utils/hccl_tools)
## 脚本说明
### 脚本及样例代码
```bash
├── model_zoo
├── README.md // 所有模型的说明文件
├── fastscnn
├──ascend310_infer // 310 推理代码目录C++
│ ├──inc
│ │ ├──utils.h // 工具包头文件
│ ├──src
│ │ ├──main.cc // 310推理代码
│ │ ├──utils.cc // 工具包
│ ├──build.sh // 代码编译脚本
│ ├──CMakeLists.txt // 代码编译设置
│ ├──fusion_switch.cfg // 配置文件
├──cal_mIoU.py // 310 推理时计算mIoU的脚本
├──preprocess.py // 310 推理时预处理验证集的脚本
├──score.py // 310 推理时计算mIoU的脚本
├── README_CN.md // fastscnn 的说明文件
├── scripts
│ ├──run_distribute_train.sh // Ascend 8卡训练脚本
│ ├──run_eval.sh // 推理启动脚本
│ ├──run_train.sh // 训练启动脚本
│ ├──run_infer_310.sh // 启动310推理的脚本
├── src
│ ├──dataloader.py // 数据集处理
│ ├──distributed_sampler.py // 8卡并行时的数据集切分操作
│ ├──fast_scnn.py // 模型结构
│ ├──logger.py // 日志打印文件
│ ├──loss.py // 损失函数
│ ├──lr_scheduler.py // 学习率衰减策略
│ ├──score.py // 推理时的 mIoU 计算脚本
│ ├──seg_data_base.py // 语义分割数据集通用处理脚本
│ ├──util.py // 边训练边验证时的 mIoU 计算脚本
│ ├──visualize.py // 分割结果可视化脚本
├── export.py // 将权重文件导出为 MINDIR 等格式的脚本
├── train.py // 训练脚本
├── eval.py // 推理脚本
```
### 脚本参数
```bash
train.py 中的主要参数如下:
--dataset: 数据集路径
--base_size: 图片初始大小图片缩放基准0.5~2倍缩放
--crop_size: 剪切尺寸
--train_split: 训练类型('test''train','val','testval'
--aux: 是否使用辅助损失
--aux_weight: 辅助损失的权重
--epochs: 训练次数
--save_every: 保存ckpt的频率
--resume_path: 预训练文件路径(接着该文件继续训练)
--resume_name: 预训练文件名
--batch_size: 批次大小
--lr: 学习率
--momentum: SGD的momentum
--weight_decay: SGD的weight_decay
--eval_while_train: 是否边训练边验证1 for True, 0 for False
--eval_steps: 验证频率(边训练边验证)
--eval_start_epoch: 边训练边验证时的起始 epoch
--use_modelarts: 是否使用 modelarts1 for True, 0 for False; 设置为 1 时将使用 moxing 从 obs 拷贝数据)
--train_url: modelsarts 需要的参数,但因该名称存在歧义而在代码中未使用)
--data_url: modelsarts 需要的参数,但因该名称存在歧义而在代码中未使用)
--output_path: 日志等文件输出目录
--outer_path: 输出到 obs 外部的目录(仅在 modelarts 上运行时有效)
--device_target: 运行设备(默认 "Ascend"
--is_distributed: 是否多卡运行
--rank: Local rank of distributed. Default: 0
--group_size: World size of device. Default: 1
--is_save_on_master: 是否仅保存 0 卡上的运行结果
--ckpt_save_max: ckpt 保存的最多文件数
eval.py 中的主要参数如下:
--dataset: 数据集路径
--base_size: 图片初始大小图片缩放基准0.5~2倍缩放
--crop_size: 剪切尺寸
--resume_path: 推理文件路径
--resume_name: 推理文件名
--use_modelarts: 是否使用 modelarts1 for True, 0 for False; 设置为 1 时将使用 moxing 从 obs 拷贝数据)
--train_url: modelsarts 需要的参数,但因该名称存在歧义而在代码中未使用)
--data_url: modelsarts 需要的参数,但因该名称存在歧义而在代码中未使用)
--output_path: 日志等文件输出目录
--outer_path: 输出到 obs 外部的目录(仅在 modelarts 上运行时有效)
--device_target: 运行设备(默认 "Ascend"
--is_distributed: 是否多卡运行
--rank: Local rank of distributed. Default: 0
--group_size: World size of device. Default: 1
export.py 中的主要参数如下:
--batch_size: 批次大小
--aux: 是否使用辅助损失
--image_height: 图片高度
--image_width: 图片宽度
--ckpt_file: 权重文件路径
--file_name: 权重文件名称
--file_format: 待转文件格式choices=["AIR", "ONNX", "MINDIR"]
--device_target: 运行设备(默认 "Ascend"
--device_id: 运行设备id
preprocess.py 中的主要参数如下:
--out_dir: 保存处理后的图片及标签的路径
--image_path: 测试集路径根目录
--image_height: 切割高度
--image_width: 切割宽度
cal_mIoU.py 中的主要参数如下:
--label_path: 标签文件路径
--output_path: 模型推理完成后的结果保存路径,默认为 xx/scripts/result_Files/
--image_width: 图片宽度
--image_height: 图片高度
--save_mask:是否保存语义分割可视化结果01默认保存至--output_path参数指定的路径下
```
### 训练过程
#### 训练
- Ascend处理器环境运行
```python
#通过 python 命令行运行单卡训练脚本。
python train.py \
--dataset=xxx/dataset/ \
--base_size=1024 \
--epochs=1000 \
--batch_size=2 \
--lr=0.001 \
--use_modelarts=0 \
--output_path=./outputs/ \
--is_distributed=0 > log.txt 2>&1 &
#通过 bash 命令启动单卡训练。
bash ./scripts/run_train.sh [train_code_path] [dataset] [epochs] [batch_size] [lr] [output_path]
#上述命令均会使脚本在后台运行,日志将输出到 log.txt可通过查看该文件了解训练详情
#Ascend多卡训练(2、4、8卡配置请自行修改run_distribute_train.sh默认8卡)
bash ./scripts/run_distribute_train.sh [train_code_path] [dataset] [epochs] [batch_size] [lr] [output_path]
```
训练完成后,您可以在 --output_path 参数指定的目录下找到保存的权重文件,训练过程中的部分 loss 收敛情况如下4卡并行
```bash
# grep "epoch time:" log.txt
epoch: 1 step: 372, loss is 1.3456033
epoch time: 137732.853 ms, per step time: 370.250 ms
epoch: 2 step: 372, loss is 1.0044098
epoch time: 58415.648 ms, per step time: 157.031 ms
epoch: 3 step: 372, loss is 1.18629
epoch time: 58427.821 ms, per step time: 157.064 ms
epoch: 4 step: 372, loss is 1.2148521
epoch time: 58462.224 ms, per step time: 157.157 ms
epoch: 5 step: 372, loss is 1.2190971
epoch time: 58443.678 ms, per step time: 157.107 ms
epoch: 6 step: 372, loss is 1.3678352
epoch time: 58433.127 ms, per step time: 157.078 ms
epoch: 7 step: 372, loss is 1.1452634
epoch time: 58486.977 ms, per step time: 157.223 ms
epoch: 8 step: 372, loss is 0.97296643
epoch time: 58435.751 ms, per step time: 157.085 ms
epoch: 9 step: 372, loss is 1.3209964
epoch time: 58425.310 ms, per step time: 157.057 ms
epoch: 10 step: 372, loss is 3.6610103
epoch time: 58471.895 ms, per step time: 157.183 ms
2021-06-25 09:59:00,682 :INFO: epoch: 10, pixAcc: 83.403519720267, mIou: 24.45742576986472
2021-06-25 09:59:00,682 :INFO: update best result: 24.45742576986472
2021-06-25 09:59:00,880 :INFO: update best checkpoint at: ./outputs/2021-06-25_time_09_46_46/best_map.ckpt
epoch: 11 step: 372, loss is 0.4546556
epoch time: 58473.429 ms, per step time: 157.187 ms
epoch: 12 step: 372, loss is 0.8289163
epoch time: 58415.030 ms, per step time: 157.030 ms
epoch: 13 step: 372, loss is 2.704109
epoch time: 58482.305 ms, per step time: 157.210 ms
epoch: 14 step: 372, loss is 0.6193013
epoch time: 58430.010 ms, per step time: 157.070 ms
epoch: 15 step: 372, loss is 1.2098892
epoch time: 58479.622 ms, per step time: 157.203 ms
epoch: 16 step: 372, loss is 1.0399697
epoch time: 58434.526 ms, per step time: 157.082 ms
epoch: 17 step: 372, loss is 0.70629436
epoch time: 58419.096 ms, per step time: 157.041 ms
epoch: 18 step: 372, loss is 0.9555901
epoch time: 58483.321 ms, per step time: 157.213 ms
epoch: 19 step: 372, loss is 0.60520625
epoch time: 58427.472 ms, per step time: 157.063 ms
epoch: 20 step: 372, loss is 1.1268346
epoch time: 58429.871 ms, per step time: 157.070 ms
2021-06-25 10:09:28,363 :INFO: epoch: 20, pixAcc: 89.35572496631883, mIou: 31.57923493725986
2021-06-25 10:09:28,363 :INFO: update best result: 31.57923493725986
2021-06-25 10:09:28,541 :INFO: update best checkpoint at: ./outputs/2021-06-25_time_09_46_46/best_map.ckpt
...
```
### 评估过程
#### 评估
在运行以下命令之前,请检查用于推理评估的权重文件路径是否正确。
- Ascend处理器环境运行
```python
#通过 python 命令启动评估
python eval.py \
--dataset=xxx/dataset/ \
--resume_path=xxx/ \
--resume_name=fastscnn.ckpt \
--output_path=./outputs/ \
--is_distributed=0 > log.txt 2>&1 &
#通过 bash 命令启动推理。
bash ./scripts/run_eval.sh [train_code_path] [dataset] [resume_path] [resume_name] [output_path]
```
运行完成后,您可以在 --output_path 指定的目录下找到最终语义分割结果的效果图;各种类别物体的 iou 值也保存在该文件夹下的 eval_results.txt 文件中。
#### 310 推理
- 在 Ascend 310 处理器环境运行
```python
#通过 bash 命令启动推理
bash run_infer_310.sh [model_path] [data_path] [out_image_path] [image_height] [image_width] [device_id]
#上述命令将完成推理所需的全部工作。执行完成后,将产生 preprocess.log、infer.log、acc.log 三个日志文件。
#如果您需要单独执行各部分代码,可以参照 run_infer_310.sh 内的流程分别进行编译、图片预处理、推理和 mIoU 计算,请注意核对各部分所需参数!
```
## 模型描述
### 性能
#### 评估性能
FastSCNN on “Cityscapes ”
| Parameters | FastSCNN |
| -------------------------- | ------------------------------------------------------------ |
| Resource | Ascend 910 CPU 2.60GHz192cores; Memory, 755G |
| uploaded Date | 6/25/2021 (month/day/year) |
| MindSpore Version | master |
| Dataset | Cityscapes |
| Training Parameters | epoch=1000, batch_size=2, lr=0.001 |
| Optimizer | SGD |
| Loss Function | MixSoftmaxCrossEntropyLoss |
| outputs | image with segmentation mask |
| Loss | 0.4 |
| Accuracy | 55.48% |
| Total time | 8p8h20m |
| Checkpoint for Fine tuning | 8p: 14.51MB(.ckpt file) |
| Scripts | https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/fastscnn |
## 随机情况说明
train.py中设置了随机种子。
## ModelZoo主页
请浏览官网[主页](https://gitee.com/mindspore/mindspore/tree/master/model_zoo)。

View File

@ -0,0 +1,14 @@
cmake_minimum_required(VERSION 3.14.1)
project(Ascend310Infer)
add_compile_definitions(_GLIBCXX_USE_CXX11_ABI=0)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O0 -g -std=c++17 -Werror -Wall -fPIE -Wl,--allow-shlib-undefined")
set(PROJECT_SRC_ROOT ${CMAKE_CURRENT_LIST_DIR}/)
option(MINDSPORE_PATH "mindspore install path" "")
include_directories(${MINDSPORE_PATH})
include_directories(${MINDSPORE_PATH}/include)
include_directories(${PROJECT_SRC_ROOT})
find_library(MS_LIB libmindspore.so ${MINDSPORE_PATH}/lib)
file(GLOB_RECURSE MD_LIB ${MINDSPORE_PATH}/_c_dataengine*)
add_executable(main src/main.cc src/utils.cc)
target_link_libraries(main ${MS_LIB} ${MD_LIB} gflags)

View File

@ -0,0 +1,29 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
if [ -d out ]; then
rm -rf out
fi
mkdir out
cd out || exit
if [ -f "Makefile" ]; then
make clean
fi
cmake .. \
-DMINDSPORE_PATH="`pip3.7 show mindspore-ascend | grep Location | awk '{print $2"/mindspore"}' | xargs realpath`"
make

View File

@ -0,0 +1 @@
ConvBatchnormFusionPass:off

View File

@ -0,0 +1,32 @@
/**
* Copyright 2021 Huawei Technologies Co., Ltd
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef MINDSPORE_INFERENCE_UTILS_H_
#define MINDSPORE_INFERENCE_UTILS_H_
#include <sys/stat.h>
#include <dirent.h>
#include <vector>
#include <string>
#include <memory>
#include "include/api/types.h"
std::vector<std::string> GetAllFiles(std::string_view dirName);
DIR *OpenDir(std::string_view dirName);
std::string RealPath(std::string_view path);
mindspore::MSTensor ReadFileToTensor(const std::string &file);
int WriteResult(const std::string& imageFile, const std::vector<mindspore::MSTensor> &outputs);
#endif

View File

@ -0,0 +1,166 @@
/**
* Copyright 2021 Huawei Technologies Co., Ltd
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <sys/time.h>
#include <gflags/gflags.h>
#include <dirent.h>
#include <iostream>
#include <string>
#include <algorithm>
#include <iosfwd>
#include <vector>
#include <fstream>
#include <sstream>
#include <random>
#include <ctime>
#include "include/api/model.h"
#include "include/api/context.h"
#include "include/api/types.h"
#include "include/api/serialization.h"
#include "include/minddata/dataset/include/vision_ascend.h"
#include "include/minddata/dataset/include/execute.h"
#include "include/minddata/dataset/include/transforms.h"
#include "include/minddata/dataset/include/constants.h"
#include "include/minddata/dataset/include/vision.h"
#include "inc/utils.h"
using mindspore::Serialization;
using mindspore::Model;
using mindspore::Context;
using mindspore::Status;
using mindspore::ModelType;
using mindspore::Graph;
using mindspore::GraphCell;
using mindspore::kSuccess;
using mindspore::MSTensor;
using mindspore::DataType;
using mindspore::dataset::Execute;
using mindspore::dataset::TensorTransform;
using mindspore::dataset::vision::Decode;
using mindspore::dataset::vision::Resize;
using mindspore::dataset::vision::HWC2CHW;
using mindspore::dataset::vision::Normalize;
using mindspore::dataset::transforms::TypeCast;
DEFINE_string(model_path, "", "model path");
DEFINE_string(dataset_path, "", "dataset path");
DEFINE_int32(device_id, 0, "device id");
DEFINE_string(fusion_switch_path, ".", "fusion switch path");
int main(int argc, char **argv) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
if (RealPath(FLAGS_model_path).empty()) {
std::cout << "Invalid mindir" << std::endl;
return 1;
}
auto context = std::make_shared<Context>();
auto ascend310_info = std::make_shared<mindspore::Ascend310DeviceInfo>();
ascend310_info->SetDeviceID(FLAGS_device_id);
context->MutableDeviceInfo().push_back(ascend310_info);
Graph graph;
Status ret = Serialization::Load(FLAGS_model_path, ModelType::kMindIR, &graph);
if (ret != kSuccess) {
std::cout << "Load model failed." << std::endl;
return 1;
}
if (RealPath(FLAGS_fusion_switch_path).empty()) {
std::cout << "Invalid fusion switch path" << std::endl;
return 1;
}
if (!FLAGS_fusion_switch_path.empty()) {
ascend310_info->SetFusionSwitchConfigPath(FLAGS_fusion_switch_path);
}
Model model;
ret = model.Build(GraphCell(graph), context);
if (ret != kSuccess) {
std::cout << "ERROR: Build failed." << std::endl;
return 1;
}
std::vector<MSTensor> modelInputs = model.GetInputs();
auto all_files = GetAllFiles(FLAGS_dataset_path);
if (all_files.empty()) {
std::cout << "ERROR: no input data." << std::endl;
return 1;
}
std::map<double, double> costTime_map;
size_t size = all_files.size();
// Define transform
for (size_t i = 0; i < size; ++i) {
struct timeval start;
struct timeval end;
double startTime_ms;
double endTime_ms;
std::vector<MSTensor> inputs;
std::vector<MSTensor> outputs;
std::vector<MSTensor> outputs0; // when aux==1, there would be 3 outputs, but we only use the first one
// when aux==0, there would be 1 output
std::cout << "Start predict input files:" << all_files[i] << std::endl;
auto image = ReadFileToTensor(all_files[i]);
inputs.emplace_back(modelInputs[0].Name(), modelInputs[0].DataType(), modelInputs[0].Shape(),
image.Data().get(), image.DataSize());
gettimeofday(&start, NULL);
ret = model.Predict(inputs, &outputs);
outputs0.emplace_back(outputs[0]); // when aux==1, outputs0's shape would be (1,19,768,768)
// when aux==0, outputs0's shape would be (19,768,768)
gettimeofday(&end, NULL);
if (ret != kSuccess) {
std::cout << "Predict " << all_files[i] << " failed." << std::endl;
return 1;
}
startTime_ms = (1.0 * start.tv_sec * 1000000 + start.tv_usec) / 1000;
endTime_ms = (1.0 * end.tv_sec * 1000000 + end.tv_usec) / 1000;
costTime_map.insert(std::pair<double, double>(startTime_ms, endTime_ms));
WriteResult(all_files[i], outputs0);
}
double average = 0.0;
int infer_cnt = 0;
for (auto iter = costTime_map.begin(); iter != costTime_map.end(); iter++) {
double diff = 0.0;
diff = iter->second - iter->first;
average += diff;
infer_cnt++;
}
average = average / infer_cnt;
std::stringstream timeCost;
timeCost << "NN inference cost average time: "<< average << " ms of infer_count " << infer_cnt << std::endl;
std::cout << "NN inference cost average time: "<< average << "ms of infer_count " << infer_cnt << std::endl;
std::string file_name = "./time_Result" + std::string("/test_perform_static.txt");
std::ofstream file_stream(file_name.c_str(), std::ios::trunc);
file_stream << timeCost.str();
file_stream.close();
costTime_map.clear();
return 0;
}

View File

@ -0,0 +1,127 @@
/**
* Copyright 2021 Huawei Technologies Co., Ltd
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "inc/utils.h"
#include <fstream>
#include <algorithm>
#include <iostream>
using mindspore::MSTensor;
using mindspore::DataType;
std::vector<std::string> GetAllFiles(std::string_view dirName) {
struct dirent *filename;
DIR *dir = OpenDir(dirName);
if (dir == nullptr) {
return {};
}
std::vector<std::string> res;
while ((filename = readdir(dir)) != nullptr) {
std::string dName = std::string(filename->d_name);
if (dName == "." || dName == ".." || filename->d_type != DT_REG) {
continue;
}
res.emplace_back(std::string(dirName) + "/" + filename->d_name);
}
std::sort(res.begin(), res.end());
for (auto &f : res) {
std::cout << "image file: " << f << std::endl;
}
return res;
}
int WriteResult(const std::string& imageFile, const std::vector<MSTensor> &outputs) {
std::string homePath = "./result_Files";
for (size_t i = 0; i < outputs.size(); ++i) {
size_t outputSize;
std::shared_ptr<const void> netOutput = outputs[i].Data();
outputSize = outputs[i].DataSize();
int pos = imageFile.rfind('/');
std::string fileName(imageFile, pos + 1);
fileName.replace(fileName.find('.'), fileName.size() - fileName.find('.'), '_' + std::to_string(i) + ".bin");
std::string outFileName = homePath + "/" + fileName;
FILE *outputFile = fopen(outFileName.c_str(), "wb");
fwrite(netOutput.get(), outputSize, sizeof(char), outputFile);
fclose(outputFile);
outputFile = nullptr;
}
return 0;
}
mindspore::MSTensor ReadFileToTensor(const std::string &file) {
if (file.empty()) {
std::cout << "Pointer file is nullptr" << std::endl;
return mindspore::MSTensor();
}
std::ifstream ifs(file);
if (!ifs.good()) {
std::cout << "File: " << file << " is not exist" << std::endl;
return mindspore::MSTensor();
}
if (!ifs.is_open()) {
std::cout << "File: " << file << "open failed" << std::endl;
return mindspore::MSTensor();
}
ifs.seekg(0, std::ios::end);
size_t size = ifs.tellg();
mindspore::MSTensor buffer(file, mindspore::DataType::kNumberTypeUInt8, {static_cast<int64_t>(size)}, nullptr, size);
ifs.seekg(0, std::ios::beg);
ifs.read(reinterpret_cast<char *>(buffer.MutableData()), size);
ifs.close();
return buffer;
}
DIR *OpenDir(std::string_view dirName) {
if (dirName.empty()) {
std::cout << " dirName is null ! " << std::endl;
return nullptr;
}
std::string realPath = RealPath(dirName);
struct stat s;
lstat(realPath.c_str(), &s);
if (!S_ISDIR(s.st_mode)) {
std::cout << "dirName is not a valid directory !" << std::endl;
return nullptr;
}
DIR *dir = opendir(realPath.c_str());
if (dir == nullptr) {
std::cout << "Can not open dir " << dirName << std::endl;
return nullptr;
}
std::cout << "Successfully opened the dir " << dirName << std::endl;
return dir;
}
std::string RealPath(std::string_view path) {
char realPathMem[PATH_MAX] = {0};
char *realPathRet = nullptr;
realPathRet = realpath(path.data(), realPathMem);
if (realPathRet == nullptr) {
std::cout << "File: " << path << " is not exist.";
return "";
}
std::string realPath(realPathMem);
std::cout << path << " realpath is: " << realPath << std::endl;
return realPath;
}

View File

@ -0,0 +1,106 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
import argparse
import os
import time
import glob
import numpy as np
from tabulate import tabulate
from score import SegmentationMetric
## Params
parser = argparse.ArgumentParser()
parser.add_argument('--label_path', type=str
, help='directory of dataset label')
parser.add_argument('--output_path', default='', type=str,
help='path of the predict files that generated by the model')
parser.add_argument('--image_width', default=768, type=int, help='image_width')
parser.add_argument('--image_height', default=768, type=int, help='image_height')
parser.add_argument('--save_mask', default=0, type=int, help='0 for False, 1 for True')
args = parser.parse_args()
cityspallete = [
128, 64, 128,
244, 35, 232,
70, 70, 70,
102, 102, 156,
190, 153, 153,
153, 153, 153,
250, 170, 30,
220, 220, 0,
107, 142, 35,
152, 251, 152,
0, 130, 180,
220, 20, 60,
255, 0, 0,
0, 0, 142,
0, 0, 70,
0, 60, 100,
0, 80, 100,
0, 0, 230,
119, 11, 32,
]
classes = ('road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light',
'traffic sign', 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car',
'truck', 'bus', 'train', 'motorcycle', 'bicycle')
def cal_mIoU(label_path, output_path, image_width, image_height, save_mask):
file_list = glob.glob(label_path+'*') # label_path must end by '/'
start_time = time.time()
metric = SegmentationMetric(19)
metric.reset()
index = 0
for file in file_list:
label = np.fromfile(file, dtype=np.int32)
label = label.reshape(image_height, image_width)
filename = file.split(os.sep)[-1][:-10] # get the name of image file
predict_path = os.path.join(output_path, filename+"_img_0.bin")
predict = np.fromfile(predict_path, dtype=np.float32)
predict = predict.reshape(1, 19, image_height, image_width)
metric.update(predict, label)
pixAcc, mIoU = metric.get()
print("[EVAL] Sample: {:d}, pixAcc: {:.3f}, mIoU: {:.3f}".format(index + 1, pixAcc * 100, mIoU * 100))
index += 1
if save_mask == 1:
output = np.argmax(predict[0], axis=0)
out_img = Image.fromarray(output.astype('uint8'))
out_img.putpalette(cityspallete)
outname = str(index) + '.png'
out_img.save(os.path.join(output_path, outname))
pixAcc, mIoU, category_iou = metric.get(return_category_iou=True)
print('End validation pixAcc: {:.3f}, mIoU: {:.3f}'.format(pixAcc * 100, mIoU * 100))
txtName = os.path.join(output_path, "eval_results.txt")
with open(txtName, "w") as f:
string = 'validation pixAcc:' + str(pixAcc * 100) + ', mIoU:' + str(mIoU * 100)
f.write(string)
f.write('\n')
headers = ['class id', 'class name', 'iou']
table = []
for i, cls_name in enumerate(classes):
table.append([cls_name, category_iou[i]])
string = 'class name: ' + cls_name + ' iou: ' + str(category_iou[i]) + '\n'
f.write(string)
print('Category iou: \n {}'.format(tabulate(table, headers, \
tablefmt='grid', showindex="always", numalign='center', stralign='center')))
time_used = time.time() - start_time
print("Time cost:"+str(time_used)+" seconds!")
if __name__ == '__main__':
cal_mIoU(args.label_path, args.output_path, args.image_width, args.image_height, args.save_mask)

View File

@ -0,0 +1,198 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
'''eval.py'''
import os
import argparse
from PIL import Image
from tabulate import tabulate
from mindspore import context
import mindspore.ops as ops
from mindspore.context import ParallelMode
from mindspore import load_checkpoint, load_param_into_net
from mindspore.communication.management import init, get_rank, get_group_size
from mindspore.dataset.transforms.py_transforms import Compose
from mindspore.dataset.vision.py_transforms import ToTensor, Normalize
from src.dataloader import create_CitySegmentation
from src.fast_scnn import FastSCNN
from src.score import SegmentationMetric
from src.logger import get_logger
import src.visualize as visualize
def parse_args():
"""Training Options for Segmentation Experiments"""
parser = argparse.ArgumentParser(description='Fast-SCNN on mindspore')
parser.add_argument('--dataset', type=str, default='/data/dataset/citys/',
help='dataset name (default: /data/dataset/citys/)')
parser.add_argument('--base_size', type=int, default=1024, help='base image size')
parser.add_argument('--crop_size', type=int, default=(768, 768), help='crop image size')
parser.add_argument('--train_split', type=str, default='train',
help='dataset train split (default: train)')
parser.add_argument('--aux', action='store_true', default=True, help='Auxiliary loss')
parser.add_argument('--aux_weight', type=float, default=0.4,
help='auxiliary loss weight')
parser.add_argument('--epochs', type=int, default=1000, metavar='N',
help='number of epochs to train (default: 1000)')
parser.add_argument('--save_every', type=int, default=1, metavar='N',
help='save ckpt every N epoch')
parser.add_argument('--resume_path', type=str, default=None,
help='put the path to resuming file if needed')
parser.add_argument('--resume_name', type=str, default=None,
help='resuming file name')
parser.add_argument('--batch_size', type=int, default=2, metavar='N',
help='input batch size for training (default: 2)')
parser.add_argument('--lr', type=float, default=0.001, metavar='LR',
help='base learning rate (default: 0.045)')
parser.add_argument('--momentum', type=float, default=0.9, metavar='M',
help='momentum (default: 0.9)')
parser.add_argument('--weight_decay', type=float, default=4e-5, metavar='M',
help='w-decay (default: 4e-5)')
parser.add_argument('--eval_while_train', type=int, default=1, help='eval while training')
parser.add_argument('--eval_steps', type=int, default=10, help='each N epochs we eval')
parser.add_argument('--eval_start_epoch', type=int, default=850, help='eval_start_epoch')
parser.add_argument('--use_modelarts', type=int, default=0,
help='when set True, we should load dataset from obs with moxing')
parser.add_argument('--train_url', type=str, default='train_url/',
help='needed by modelarts, but we donot use it because the name is ambiguous')
parser.add_argument('--data_url', type=str, default='data_url/',
help='needed by modelarts, but we donot use it because the name is ambiguous')
parser.add_argument('--output_path', type=str, default='./outputs/',
help='output_path,when use_modelarts is set True, it will be cache/output/')
parser.add_argument('--outer_path', type=str, default='s3://output/',
help='obs path,to store e.g ckpt files ')
parser.add_argument('--device_target', type=str, default='Ascend',
help='device where the code will be implemented. (Default: Ascend)')
parser.add_argument('--is_distributed', type=int, default=0, help='if multi device')
parser.add_argument('--rank', type=int, default=0, help='local rank of distributed')
parser.add_argument('--group_size', type=int, default=1, help='world size of distributed')
parser.add_argument('--is_save_on_master', type=int, default=1,
help='save ckpt on master or all rank')
parser.add_argument('--ckpt_save_max', type=int, default=800,
help='Maximum number of checkpoint files can be saved. Default: 5.')
# the parser
args_ = parser.parse_args()
return args_
args = parse_args()
save_dir = args.output_path
device_id = int(os.getenv('DEVICE_ID', '0'))
context.set_context(mode=context.GRAPH_MODE,
device_target=args.device_target, save_graphs=False)
def validation():
'''validation'''
if args.is_distributed:
assert args.device_target == "Ascend"
init()
context.set_context(device_id=device_id)
args.rank = get_rank()
args.group_size = get_group_size()
device_num = args.group_size
context.reset_auto_parallel_context()
context.set_auto_parallel_context(device_num=device_num, parallel_mode=ParallelMode.DATA_PARALLEL,
mirror_mean=True)
else:
if args.device_target in ["Ascend", "GPU"]:
context.set_context(device_id=device_id)
# select for master rank save ckpt or all rank save, compatible for model parallel
args.rank_save_ckpt_flag = 0
if args.is_save_on_master:
if args.rank == 0:
args.rank_save_ckpt_flag = 1
else:
args.rank_save_ckpt_flag = 1
metric = SegmentationMetric(19)
metric.reset()
# create network
model = FastSCNN(num_classes=19, aux=True)
if args.resume_path:
if args.use_modelarts:
import moxing as mox
args.logger.info("copying resume checkpoint from obs to cache....")
mox.file.copy_parallel(args.resume_path, 'cache/resume_path')
args.logger.info("copying resume checkpoint finished....")
args.resume_path = 'cache/resume_path/'
args.resume_path = os.path.join(args.resume_path, args.resume_name)
args.logger.info('loading resume checkpoint {} into network'.format(args.resume_path))
load_param_into_net(model, load_checkpoint(args.resume_path))
args.logger.info('loaded resume checkpoint {} into network'.format(args.resume_path))
model.set_train(False)
# image transform
input_transform = Compose([
ToTensor(),
Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])
if args.use_modelarts:
import moxing as mox
args.logger.info("copying dataset from obs to cache....")
mox.file.copy_parallel(args.dataset, 'cache/dataset')
args.logger.info("copying dataset finished....")
args.dataset = 'cache/dataset/'
val_dataset, _ = create_CitySegmentation(args, data_path=args.dataset, \
split='val', mode='val', transform=input_transform, \
base_size=args.base_size, crop_size=args.crop_size, \
batch_size=1, device_num=args.group_size, \
rank=args.rank, shuffle=False)
classes = ('road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light',
'traffic sign', 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car',
'truck', 'bus', 'train', 'motorcycle', 'bicycle')
data_loader = val_dataset.create_dict_iterator()
for i, data in enumerate(data_loader):
images = data["image"]
targets = data["label"]
output = model(images)[0]
metric.update(output, targets)
pixAcc, mIoU = metric.get()
args.logger.info("[EVAL] Sample: {:d}, pixAcc: {:.3f}, mIoU: {:.3f}".format(i + 1, pixAcc * 100, mIoU * 100))
output = ops.Argmax(axis=0)(output[0]).asnumpy()
out_img = Image.fromarray(output.astype('uint8'))
out_img.putpalette(visualize.cityspallete)
outname = str(i) + '.png'
out_img.save(os.path.join(save_dir, outname))
pixAcc, mIoU = metric.get()
args.logger.info("[EVAL END] pixAcc: {:.3f}, mIoU: {:.3f}".format(pixAcc * 100, mIoU * 100))
pixAcc, mIoU, category_iou = metric.get(return_category_iou=True)
args.logger.info('End validation pixAcc: {:.3f}, mIoU: {:.3f}'.format(pixAcc * 100, mIoU * 100))
txtName = os.path.join(save_dir, "eval_results.txt")
with open(txtName, "w") as f:
string = 'validation pixAcc:' + str(pixAcc * 100) + ', mIoU:' + str(mIoU * 100)
f.write(string)
f.write('\n')
headers = ['class id', 'class name', 'iou']
table = []
for i, cls_name in enumerate(classes):
table.append([cls_name, category_iou[i]])
string = 'class name: ' + cls_name + ' iou: ' + str(category_iou[i]) + '\n'
f.write(string)
args.logger.info('Category iou: \n {}'.format(tabulate(table, headers, \
tablefmt='grid', showindex="always", numalign='center', stralign='center')))
if __name__ == '__main__':
args.logger = get_logger(save_dir, "Fast_SCNN", args.rank)
args.logger.save_args(args)
validation()

View File

@ -0,0 +1,54 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""
##############export checkpoint file into air, onnx, mindir models#################
python export.py
"""
import argparse
import numpy as np
import mindspore as ms
from mindspore import context, Tensor, load_checkpoint, load_param_into_net, export
from src.fast_scnn import FastSCNN
## Params
parser = argparse.ArgumentParser()
parser.add_argument('--batch_size', default=1, type=int, help='batch size')
parser.add_argument('--aux', action='store_true', default=True, help='Auxiliary loss')
parser.add_argument("--image_height", type=int, default=768, help="Image height.")
parser.add_argument("--image_width", type=int, default=768, help="Image width.")
parser.add_argument("--ckpt_file", type=str, required=True, help="Checkpoint file path.")
parser.add_argument("--file_name", type=str, default="fastscnn", help="output file name.")
parser.add_argument("--file_format", type=str, choices=["AIR", "ONNX", "MINDIR"], default="AIR", help="file format")
parser.add_argument('--device_target', type=str, default='Ascend'
, help='device where the code will be implemented. (Default: Ascend)')
parser.add_argument("--device_id", type=int, default=0, help="Device id")
args_opt = parser.parse_args()
context.set_context(mode=context.GRAPH_MODE, device_target=args_opt.device_target, device_id=args_opt.device_id)
if __name__ == '__main__':
net = FastSCNN(num_classes=19, aux=args_opt.aux)
param_dict = load_checkpoint(args_opt.ckpt_file)
load_param_into_net(net, param_dict)
input_arr = Tensor(np.zeros([args_opt.batch_size, 3, \
args_opt.image_height, args_opt.image_width]), ms.float32)
export(net, input_arr, file_name=args_opt.file_name, file_format=args_opt.file_format)

View File

@ -0,0 +1,152 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
import argparse
import os
import numpy as np
import PIL.Image as Image
parser = argparse.ArgumentParser()
parser.add_argument('--out_dir', type=str, required=True
, help='directory to store the image after being cropped')
parser.add_argument('--image_path', type=str, required=True,
help='directory of image to crop')
parser.add_argument('--image_height', type=int, default=768
, help='image height after being cropped')
parser.add_argument('--image_width', type=int, default=768
, help='image width after being cropped')
args = parser.parse_args()
valid_classes = [7, 8, 11, 12, 13, 17, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 31, 32, 33]
_key = np.array([-1, -1, -1, -1, -1, -1,
-1, -1, 0, 1, -1, -1,
2, 3, 4, -1, -1, -1,
5, -1, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15,
-1, -1, 16, 17, 18])
_mapping = np.array(range(-1, len(_key) - 1)).astype('int32')
def _get_city_pairs(folder, split='train'):
'''_get_city_pairs'''
def get_path_pairs(img_folder, mask_folder):
img_paths = []
mask_paths = []
for root, _, files in os.walk(img_folder):
for filename in files:
if filename.startswith('._'):
continue
if filename.endswith('.png'):
imgpath = os.path.join(root, filename)
foldername = os.path.basename(os.path.dirname(imgpath))
maskname = filename.replace('leftImg8bit', 'gtFine_labelIds')
maskpath = os.path.join(mask_folder, foldername, maskname)
if os.path.isfile(imgpath) and os.path.isfile(maskpath):
img_paths.append(imgpath)
mask_paths.append(maskpath)
else:
print('cannot find the mask or image:', imgpath, maskpath)
print('Found {} images in the folder {}'.format(len(img_paths), img_folder))
return img_paths, mask_paths
if split in ('train', 'val'):
img_folder = os.path.join(folder, 'leftImg8bit' + os.sep + split)
mask_folder = os.path.join(folder, 'gtFine' + os.sep + split)
img_paths, mask_paths = get_path_pairs(img_folder, mask_folder)
return img_paths, mask_paths
assert split == 'trainval'
print('trainval set')
train_img_folder = os.path.join(folder, 'leftImg8bit' + os.sep + 'train')
train_mask_folder = os.path.join(folder, 'gtFine' + os.sep + 'train')
val_img_folder = os.path.join(folder, 'leftImg8bit' + os.sep + 'val')
val_mask_folder = os.path.join(folder, 'gtFine' + os.sep + 'val')
train_img_paths, train_mask_paths = get_path_pairs(train_img_folder, train_mask_folder)
val_img_paths, val_mask_paths = get_path_pairs(val_img_folder, val_mask_folder)
img_paths = train_img_paths + val_img_paths
mask_paths = train_mask_paths + val_mask_paths
return img_paths, mask_paths
def _val_sync_transform(outsize, img, mask):
'''_val_sync_transform'''
short_size = min(outsize)
w, h = img.size
if w > h:
oh = short_size
ow = int(1.0 * w * oh / h)
else:
ow = short_size
oh = int(1.0 * h * ow / w)
img = img.resize((ow, oh), Image.BILINEAR)
mask = mask.resize((ow, oh), Image.NEAREST)
# center crop
w, h = img.size
x1 = int(round((w - outsize[1]) / 2.))
y1 = int(round((h - outsize[0]) / 2.))
img = img.crop((x1, y1, x1 + outsize[1], y1 + outsize[0]))
mask = mask.crop((x1, y1, x1 + outsize[1], y1 + outsize[0]))
# final transform
img, mask = np.array(img), _mask_transform(mask)
return img, mask
def _class_to_index(mask):
# assert the value
values = np.unique(mask)
for value in values:
assert value in _mapping
index = np.digitize(mask.ravel(), _mapping, right=True)
return _key[index].reshape(mask.shape)
def _mask_transform(mask):
target = _class_to_index(np.array(mask).astype('int32'))
return np.array(target).astype('int32')
def crop_imageAndLabel(out_dir, image_path, image_height, image_width):
if not os.path.exists(os.path.join(out_dir, "images")):
os.makedirs(os.path.join(out_dir, "images"))
if not os.path.exists(os.path.join(out_dir, "labels")):
os.makedirs(os.path.join(out_dir, "labels"))
assert os.path.exists(image_path), "Please put dataset in {SEG_ROOT}/datasets/cityscapes"
images, mask_paths = _get_city_pairs(image_path, 'val')
assert len(images) == len(mask_paths)
if not images:
raise RuntimeError("Found 0 images in subfolders of:" + image_path + "\n")
for index in range(len(images)):
print("Processing ", images[index])
img = Image.open(images[index]).convert('RGB')
mask = Image.open(mask_paths[index])
img, mask = _val_sync_transform((image_height, image_width), img, mask)
img = img.astype(np.float32)
mask = mask.astype(np.int32)
mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]
img = img.transpose((2, 0, 1))#HWC->CHW
for channel, _ in enumerate(img):
# Normalization
img[channel] /= 255
img[channel] -= mean[channel]
img[channel] /= std[channel]
img = np.expand_dims(img, 0)#NCHW
mask = np.expand_dims(mask, 0)#NHW
filename = images[index].split(os.sep)[-1].split('.')[0] # get the name of image file
img.tofile(os.path.join(os.path.join(out_dir, "images"), filename+'_img.bin'))
mask.tofile(os.path.join(os.path.join(out_dir, "labels"), filename+'_label.bin'))
if __name__ == "__main__":
crop_imageAndLabel(args.out_dir, args.image_path, args.image_height, args.image_width)

View File

@ -0,0 +1,160 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Evaluation Metrics for Semantic Segmentation"""
import numpy as np
__all__ = ['SegmentationMetric', 'batch_pix_accuracy', 'batch_intersection_union',
'pixelAccuracy', 'intersectionAndUnion', 'hist_info', 'compute_score']
class SegmentationMetric():
"""Computes pixAcc and mIoU metric scores
"""
def __init__(self, nclass):
super(SegmentationMetric, self).__init__()
self.nclass = nclass
self.reset()
def update(self, preds, labels):
"""Updates the internal evaluation result.
Parameters
----------
labels : 'NumpyArray' or list of `NumpyArray`
The labels of the data.
preds : 'NumpyArray' or list of `NumpyArray`
Predicted values.
"""
def evaluate_worker(self, pred, label):
correct, labeled = batch_pix_accuracy(pred, label)
inter, union = batch_intersection_union(pred, label, self.nclass)
self.total_correct += correct
self.total_label += labeled
self.total_inter += inter
self.total_union += union
evaluate_worker(self, preds, labels)
def get(self, return_category_iou=False):
"""Gets the current evaluation result.
Returns
-------
metrics : tuple of float
pixAcc and mIoU
"""
# remove np.spacing(1)
pixAcc = 1.0 * self.total_correct / (2.220446049250313e-16 + self.total_label)
IoU = 1.0 * self.total_inter / (2.220446049250313e-16 + self.total_union)
mIoU = IoU.mean().item()
if return_category_iou:
return pixAcc, mIoU, IoU
return pixAcc, mIoU
def reset(self):
"""Resets the internal evaluation result to initial state."""
self.total_inter = np.zeros(self.nclass)
self.total_union = np.zeros(self.nclass)
self.total_correct = 0
self.total_label = 0
def batch_pix_accuracy(output, target):
"""PixAcc"""
# inputs are numpy array, output 4D NCHW where 'C' means label classes, target 3D NHW
predict = np.argmax(output.astype(np.int64), 1) + 1
target = target.astype(np.int64) + 1
pixel_labeled = (target > 0).sum()
pixel_correct = ((predict == target) * (target > 0)).sum()
assert pixel_correct <= pixel_labeled, "Correct area should be smaller than Labeled"
return pixel_correct, pixel_labeled
def batch_intersection_union(output, target, nclass):
"""mIoU"""
# inputs are numpy array, output 4D, target 3D
mini = 1
maxi = nclass
nbins = nclass
predict = np.argmax(output.astype(np.float32), 1) + 1
target = target.astype(np.float32) + 1
predict = predict.astype(np.float32) * (target > 0).astype(np.float32)
intersection = predict * (predict == target).astype(np.float32)
# areas of intersection and union
# element 0 in intersection occur the main difference from np.bincount. set boundary to -1 is necessary.
area_inter, _ = np.histogram(intersection, bins=nbins, range=(mini, maxi))
area_pred, _ = np.histogram(predict, bins=nbins, range=(mini, maxi))
area_lab, _ = np.histogram(target, bins=nbins, range=(mini, maxi))
area_union = area_pred + area_lab - area_inter
assert (area_inter > area_union).sum() == 0, "Intersection area should be smaller than Union area"
return area_inter.astype(np.float32), area_union.astype(np.float32)
def pixelAccuracy(imPred, imLab):
"""
This function takes the prediction and label of a single image, returns pixel-wise accuracy
To compute over many images do:
for i = range(Nimages):
(pixel_accuracy[i], pixel_correct[i], pixel_labeled[i]) = \
pixelAccuracy(imPred[i], imLab[i])
mean_pixel_accuracy = 1.0 * np.sum(pixel_correct) / (np.spacing(1) + np.sum(pixel_labeled))
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
pixel_labeled = np.sum(imLab >= 0)
pixel_correct = np.sum((imPred == imLab) * (imLab >= 0))
pixel_accuracy = 1.0 * pixel_correct / pixel_labeled
return (pixel_accuracy, pixel_correct, pixel_labeled)
def intersectionAndUnion(imPred, imLab, numClass):
"""
This function takes the prediction and label of a single image,
returns intersection and union areas for each class
To compute over many images do:
for i in range(Nimages):
(area_intersection[:,i], area_union[:,i]) = intersectionAndUnion(imPred[i], imLab[i])
IoU = 1.0 * np.sum(area_intersection, axis=1) / np.sum(np.spacing(1)+area_union, axis=1)
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
imPred = imPred * (imLab >= 0)
# Compute area intersection:
intersection = imPred * (imPred == imLab)
(area_intersection, _) = np.histogram(intersection, bins=numClass, range=(1, numClass))
# Compute area union:
(area_pred, _) = np.histogram(imPred, bins=numClass, range=(1, numClass))
(area_lab, _) = np.histogram(imLab, bins=numClass, range=(1, numClass))
area_union = area_pred + area_lab - area_intersection
return (area_intersection, area_union)
def hist_info(pred, label, num_cls):
assert pred.shape == label.shape
k = (label >= 0) & (label < num_cls)
labeled = np.sum(k)
correct = np.sum((pred[k] == label[k]))
return np.bincount(num_cls * label[k].astype(int) + pred[k], \
minlength=num_cls ** 2).reshape(num_cls, num_cls), labeled, correct
def compute_score(hist, correct, labeled):
iu = np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist))
mean_IU = np.nanmean(iu)
mean_IU_no_back = np.nanmean(iu[1:])
mean_pixel_acc = correct / labeled
return iu, mean_IU, mean_IU_no_back, mean_pixel_acc

View File

@ -0,0 +1,74 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
if [ $# != 6 ]; then
echo "Usage: sh run_distribute_train.sh [train_code_path] [dataset]" \
"[epochs] [batch_size] [lr] [output_path]"
exit 1
fi
get_real_path() {
if [ "${1:0:1}" == "/" ]; then
echo "$1"
else
echo "$(realpath -m $PWD/$1)/"
fi
}
train_code_path=$(get_real_path $1)
echo "train_code_path: "$train_code_path
dataset=$(get_real_path $2)
echo "dataset: "$dataset
if [ ! -d $train_code_path ]
then
echo "error: train_code_path=$train_code_path is not a dictionary."
exit 1
fi
if [ ! -d $dataset ]
then
echo "error: dataset=$dataset is not a dictionary."
exit 1
fi
ulimit -c unlimited
export SLOG_PRINT_TO_STDOUT=0
export RANK_TABLE_FILE=${train_code_path}scripts/hccl_8p_01234567_10.155.170.118.json
export RANK_SIZE=8
export RANK_START_ID=0
for((i=0;i<=$RANK_SIZE-1;i++));
do
export RANK_ID=${i}
export DEVICE_ID=$((i + RANK_START_ID))
echo 'start rank='${i}', device id='${DEVICE_ID}'...'
if [ -d ${train_code_path}/device${DEVICE_ID} ]; then
rm -rf ${train_code_path}/device${DEVICE_ID}
fi
mkdir ${train_code_path}/device${DEVICE_ID}
cd ${train_code_path}/device${DEVICE_ID} || exit
nohup python ${train_code_path}train.py --is_distributed=1 \
--dataset=${dataset} \
--epochs=$3 \
--batch_size=$4 \
--lr=$5 \
--output_path=$6 > log.txt 2>&1 &
done

View File

@ -0,0 +1,62 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
if [ $# != 5 ]; then
echo "Usage: sh run_eval.sh [train_code_path] [dataset] [resume_path]" \
"[resume_name] [output_path]"
exit 1
fi
get_real_path() {
if [ "${1:0:1}" == "/" ]; then
echo "$1"
else
echo "$(realpath -m $PWD/$1)/"
fi
}
train_code_path=$(get_real_path $1)
echo "train_code_path: "$train_code_path
dataset=$(get_real_path $2)
echo "dataset: "$dataset
resume_path=$(get_real_path $3)
echo "resume_path: "$resume_path
if [ ! -d $train_code_path ]
then
echo "error: train_code_path=$train_code_path is not a dictionary."
exit 1
fi
if [ ! -d $dataset ]
then
echo "error: dataset=$dataset is not a dictionary."
exit 1
fi
if [ ! -d $resume_path ]
then
echo "error: resume_path=$resume_path is not a dictionary."
exit 1
fi
python ${train_code_path}eval.py --is_distributed=0 \
--dataset=$dataset \
--resume_path=$resume_path \
--resume_name=$4 \
--output_path=$5 > log.txt 2>&1 &

View File

@ -0,0 +1,132 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
if [ $# != 6 ]; then
echo "Usage: sh run_infer_310.sh [model_path] [data_path]" \
"[out_image_path] [image_height] [image_width] [device_id]"
exit 1
fi
get_real_path_name() {
if [ "${1:0:1}" == "/" ]; then
echo "$1"
else
echo "$(realpath -m $PWD/$1)"
fi
}
get_real_path() {
if [ "${1:0:1}" == "/" ]; then
echo "$1"
else
echo "$(realpath -m $PWD/$1)/"
fi
}
model=$(get_real_path_name $1)
data_path=$(get_real_path $2)
out_image_path=$(get_real_path $3)
image_height=$4
image_width=$5
device_id=$6
echo "model path: "$model
echo "dataset path: "$data_path
echo "out image path: "$out_image_path
echo "image_height: "$image_height
echo "image_width: "$image_width
echo "device id: "$device_id
export ASCEND_HOME=/usr/local/Ascend/
if [ -d ${ASCEND_HOME}/ascend-toolkit ]; then
export PATH=$ASCEND_HOME/ascend-toolkit/latest/fwkacllib/ccec_compiler/bin:$ASCEND_HOME/ascend-toolkit/latest/atc/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/lib:$ASCEND_HOME/ascend-toolkit/latest/atc/lib64:$ASCEND_HOME/ascend-toolkit/latest/fwkacllib/lib64:$ASCEND_HOME/driver/lib64:$ASCEND_HOME/add-ons:$LD_LIBRARY_PATH
export TBE_IMPL_PATH=$ASCEND_HOME/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe
export PYTHONPATH=${TBE_IMPL_PATH}:$ASCEND_HOME/ascend-toolkit/latest/fwkacllib/python/site-packages:$PYTHONPATH
export ASCEND_OPP_PATH=$ASCEND_HOME/ascend-toolkit/latest/opp
else
export PATH=$ASCEND_HOME/atc/ccec_compiler/bin:$ASCEND_HOME/atc/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/lib:$ASCEND_HOME/atc/lib64:$ASCEND_HOME/acllib/lib64:$ASCEND_HOME/driver/lib64:$ASCEND_HOME/add-ons:$LD_LIBRARY_PATH
export PYTHONPATH=$ASCEND_HOME/atc/python/site-packages:$PYTHONPATH
export ASCEND_OPP_PATH=$ASCEND_HOME/opp
fi
function compile_app()
{
cd ../ascend310_infer/ || exit
if [ -f "Makefile" ]; then
make clean
fi
sh build.sh &> build.log
}
function preprocess()
{
cd ../ || exit
echo "\nstart preprocess"
echo "waitting for preprocess finish..."
python3.7 preprocess.py --out_dir=$out_image_path --image_path=$data_path --image_height=$image_height --image_width=$image_width > preprocess.log 2>&1
echo "preprocess finished! you can see the log in preprocess.log!"
}
function infer()
{
cd ./scripts || exit
if [ -d result_Files ]; then
rm -rf ./result_Files
fi
if [ -d time_Result ]; then
rm -rf ./time_Result
fi
mkdir result_Files
mkdir time_Result
echo "\nstart infer..."
echo "waitting for infer finish..."
../ascend310_infer/out/main --model_path=$model --dataset_path=$out_image_path/images/ --device_id=$device_id > infer.log 2>&1
echo "infer finished! you can see the log in infer.log!"
}
function cal_mIoU()
{
echo "\nstart calculate mIoU..."
echo "waitting for calculate finish..."
python3.7 ../cal_mIoU.py --label_path=$out_image_path/labels/ --output_path=./result_Files --image_height=$image_height --image_width=$image_width --save_mask=0 >acc.log 2>&1
echo "infer finished! you can see the log in acc.log\n"
}
compile_app
if [ $? -ne 0 ]; then
echo "compile app code failed"
exit 1
fi
preprocess
if [ $? -ne 0 ]; then
echo "execute preprocess failed"
exit 1
fi
infer
if [ $? -ne 0 ]; then
echo " execute inference failed"
exit 1
fi
cal_mIoU
if [ $? -ne 0 ]; then
echo "calculate mIoU failed"
exit 1
fi

View File

@ -0,0 +1,57 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
if [ $# != 6 ]; then
echo "Usage: sh run_train.sh [train_code_path] [dataset]" \
"[epochs] [batch_size] [lr] [output_path]"
exit 1
fi
get_real_path() {
if [ "${1:0:1}" == "/" ]; then
echo "$1"
else
echo "$(realpath -m $PWD/$1)/"
fi
}
train_code_path=$(get_real_path $1)
echo "train_code_path: "$train_code_path
dataset=$(get_real_path $2)
echo "dataset: "$dataset
if [ ! -d $train_code_path ]
then
echo "error: train_code_path=$train_code_path is not a dictionary."
exit 1
fi
if [ ! -d $dataset ]
then
echo "error: dataset=$dataset is not a dictionary."
exit 1
fi
nohup python ${train_code_path}train.py --is_distributed=0 \
--dataset=${dataset} \
--epochs=$3 \
--batch_size=$4 \
--lr=$5 \
--output_path=$6 > log.txt 2>&1 &
echo 'Train task has been started successfully!'
echo 'Please check the log at log.txt'

View File

@ -0,0 +1,196 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Prepare Cityscapes dataset"""
import os
import numpy as np
from PIL import Image
import mindspore.dataset as ds
import mindspore.dataset.vision.c_transforms as CV
from src.seg_data_base import SegmentationDataset
from src.distributed_sampler import DistributedSampler
__all__ = ['CitySegmentation']
class CitySegmentation(SegmentationDataset):
"""Cityscapes Semantic Segmentation Dataset.
Parameters
----------
root : string
Path to Cityscapes folder. Default is './datasets/cityscapes'
split: string
'train', 'val' or 'test'
transform : callable, optional
A function that transforms the image
Examples
--------
>>> # Transforms for Normalization
>>> input_transform = transforms.Compose([
>>> transforms.ToTensor(),
>>> transforms.Normalize((.485, .456, .406), (.229, .224, .225)),
>>> ])
>>> # Create Dataset
>>> trainset = CitySegmentation(split='train', transform=input_transform)
>>> # Create Training Loader
>>> train_data = data.DataLoader(
>>> trainset, 4, shuffle=True,
>>> num_workers=4)
"""
BASE_DIR = 'cityscapes'
NUM_CLASS = 19
def __init__(self, args, root='/data/Fast_SCNN/dataset/', split='train', mode=None, **kwargs):
super(CitySegmentation, self).__init__(root, split, mode, **kwargs)
self.args = args
assert os.path.exists(self.root), "Please put dataset in {SEG_ROOT}/datasets/cityscapes"
self.images, self.mask_paths = _get_city_pairs(args, self.root, self.split)
assert len(self.images) == len(self.mask_paths)
if not self.images:
raise RuntimeError("Found 0 images in subfolders of:" + root + "\n")
self.valid_classes = [7, 8, 11, 12, 13, 17, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 31, 32, 33]
self._key = np.array([-1, -1, -1, -1, -1, -1,
-1, -1, 0, 1, -1, -1,
2, 3, 4, -1, -1, -1,
5, -1, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15,
-1, -1, 16, 17, 18])
self._mapping = np.array(range(-1, len(self._key) - 1)).astype('int32')
def _class_to_index(self, mask):
# assert the value
values = np.unique(mask)
for value in values:
assert value in self._mapping
index = np.digitize(mask.ravel(), self._mapping, right=True)
return self._key[index].reshape(mask.shape)
def __getitem__(self, index):
img = Image.open(self.images[index]).convert('RGB')
if self.mode == 'test':
return img, os.path.basename(self.images[index])
mask = Image.open(self.mask_paths[index])
# synchrosized transform
if self.mode == 'train':
img, mask = self._sync_transform(img, mask)
elif self.mode == 'val':
img, mask = self._val_sync_transform(img, mask)
else:
assert self.mode == 'testval'
img, mask = self._img_transform(img), self._mask_transform(mask)
return img, mask
def _mask_transform(self, mask):
target = self._class_to_index(np.array(mask).astype('int32'))
return np.array(target).astype('int32')
def __len__(self):
return len(self.images)
@property
def pred_offset(self):
return 0
@property
def classes(self):
"""Category names."""
return ('road', 'sidewalk', 'building', 'wall', 'fence', 'pole', 'traffic light',
'traffic sign', 'vegetation', 'terrain', 'sky', 'person', 'rider', 'car',
'truck', 'bus', 'train', 'motorcycle', 'bicycle')
def _get_city_pairs(args, folder, split='train'):
'''_get_city_pairs'''
def get_path_pairs(img_folder, mask_folder):
img_paths = []
mask_paths = []
for root, _, files in os.walk(img_folder):
for filename in files:
if filename.startswith('._'):
continue
if filename.endswith('.png'):
imgpath = os.path.join(root, filename)
foldername = os.path.basename(os.path.dirname(imgpath))
maskname = filename.replace('leftImg8bit', 'gtFine_labelIds')
maskpath = os.path.join(mask_folder, foldername, maskname)
if os.path.isfile(imgpath) and os.path.isfile(maskpath):
img_paths.append(imgpath)
mask_paths.append(maskpath)
else:
args.logger.info('cannot find the mask or image:', imgpath, maskpath)
args.logger.info('Found {} images in the folder {}'.format(len(img_paths), img_folder))
return img_paths, mask_paths
if split in ('train', 'val'):
img_folder = os.path.join(folder, 'leftImg8bit/' + split)
mask_folder = os.path.join(folder, 'gtFine/' + split)
img_paths, mask_paths = get_path_pairs(img_folder, mask_folder)
return img_paths, mask_paths
assert split == 'trainval'
args.logger.info('trainval set')
train_img_folder = os.path.join(folder, 'leftImg8bit/train')
train_mask_folder = os.path.join(folder, 'gtFine/train')
val_img_folder = os.path.join(folder, 'leftImg8bit/val')
val_mask_folder = os.path.join(folder, 'gtFine/val')
train_img_paths, train_mask_paths = get_path_pairs(train_img_folder, train_mask_folder)
val_img_paths, val_mask_paths = get_path_pairs(val_img_folder, val_mask_folder)
img_paths = train_img_paths + val_img_paths
mask_paths = train_mask_paths + val_mask_paths
return img_paths, mask_paths
def create_CitySegmentation(args, data_path='../dataset/', split='train', mode=None, \
transform=None, base_size=1024, crop_size=(512, 1024), \
batch_size=2, device_num=1, rank=0, shuffle=True):
'''create_CitySegmentation'''
dataset = CitySegmentation(args, root=data_path, split=split, mode=mode, \
base_size=base_size, crop_size=crop_size)
dataset_len = len(dataset)
distributed_sampler = DistributedSampler(dataset_len, device_num, rank, shuffle=shuffle)
data_set = ds.GeneratorDataset(dataset, column_names=["image", "label"], num_parallel_workers=8, \
shuffle=shuffle, sampler=distributed_sampler)
# general resize, normalize and toTensor
if transform is not None:
data_set = data_set.map(input_columns=["image"], operations=transform, num_parallel_workers=8)
else:
hwc_to_chw = CV.HWC2CHW()
data_set = data_set.map(input_columns=["image"], operations=hwc_to_chw, num_parallel_workers=8)
data_set = data_set.batch(batch_size, drop_remainder=True)
return data_set, dataset_len
if __name__ == '__main__':
ds = create_CitySegmentation(
data_path=r"./dataset/leftImg8bit_trainvaltest",
split='train',
mode='train',
base_size=1024,
crop_size=(512, 1024),
batch_size=1,
device_num=1,
rank=0,
shuffle=True)
data_loader = ds.create_dict_iterator()
for i, data in enumerate(data_loader):
print(data['image'])
print(data['label'])
print(i)

View File

@ -0,0 +1,60 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""distributed sampler."""
from __future__ import division
import math
import numpy as np
class DistributedSampler:
"""Distributed sampler."""
def __init__(self, dataset_size, num_replicas=None, rank=None, shuffle=True):
if num_replicas is None:
print("***********Setting world_size to 1 since it is not passed in ******************")
num_replicas = 1
if rank is None:
print("***********Setting rank to 0 since it is not passed in ******************")
rank = 0
self.dataset_size = dataset_size
self.num_replicas = num_replicas
self.rank = rank
self.epoch = 0
self.num_samples = int(math.ceil(dataset_size * 1.0 / self.num_replicas))
self.total_size = self.num_samples * self.num_replicas
self.shuffle = shuffle
def __iter__(self):
# deterministically shuffle based on epoch
if self.shuffle:
indices = np.random.RandomState(seed=self.epoch).permutation(self.dataset_size)
# np.array type. number from 0 to len(dataset_size)-1, used as index of dataset
indices = indices.tolist()
self.epoch += 1
# change to list type
else:
indices = list(range(self.dataset_size))
# add extra samples to make it evenly divisible
indices += indices[:(self.total_size - len(indices))]
assert len(indices) == self.total_size
# subsample
indices = indices[self.rank:self.total_size:self.num_replicas]
assert len(indices) == self.num_samples
return iter(indices)
def __len__(self):
return self.num_samples

View File

@ -0,0 +1,264 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Fast Segmentation Convolutional Neural Network"""
import mindspore.nn as nn
from mindspore.ops import operations as P
from src.loss import MixSoftmaxCrossEntropyLoss
__all__ = ['FastSCNN', 'FastSCNNWithLossCell']
class FastSCNN(nn.Cell):
'''FastSCNN'''
def __init__(self, num_classes, aux=False):
super(FastSCNN, self).__init__()
self.aux = aux
self.learning_to_downsample = LearningToDownsample(32, 48, 64)
self.global_feature_extractor = GlobalFeatureExtractor(64, \
[64, 96, 128], 128, 6, [3, 3, 3])
self.feature_fusion = FeatureFusionModule(64, 128, 128)
self.classifier = Classifier(128, num_classes)
if self.aux:
self.auxlayer1 = nn.SequentialCell(
[nn.Conv2d(64, 32, 3, pad_mode='pad', padding=1, has_bias=False),
nn.BatchNorm2d(32),
nn.ReLU(),
nn.Dropout(0.9),#1-0.9=0.1
nn.Conv2d(32, num_classes, 1, has_bias=True)]
)
self.auxlayer2 = nn.SequentialCell(
[nn.Conv2d(128, 32, 3, pad_mode='pad', padding=1, has_bias=False),
nn.BatchNorm2d(32),
nn.ReLU(),
nn.Dropout(0.9),#1-0.9=0.1
nn.Conv2d(32, num_classes, 1, has_bias=True)]
)
self.ResizeBilinear = nn.ResizeBilinear()
def construct(self, x):
'''construct'''
size = x.shape[2:]
higher_res_features = self.learning_to_downsample(x)
lower_res_features = self.global_feature_extractor(higher_res_features)
x = self.feature_fusion(higher_res_features, lower_res_features)
x = self.classifier(x)
x = self.ResizeBilinear(x, size, align_corners=True)
if self.aux:
auxout = self.auxlayer1(higher_res_features)
auxout = self.ResizeBilinear(auxout, size, align_corners=True)
auxout2 = self.auxlayer2(lower_res_features)
auxout2 = self.ResizeBilinear(auxout2, size, align_corners=True)
return x, auxout, auxout2
return x
class _ConvBNReLU(nn.Cell):
"""Conv-BN-ReLU"""
def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, padding=0):
super(_ConvBNReLU, self).__init__()
self.conv = nn.SequentialCell(
[nn.Conv2d(in_channels=in_channels, out_channels=out_channels, \
kernel_size=kernel_size, stride=stride, padding=padding, has_bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU()]
)
def construct(self, x):
return self.conv(x)
class _DSConv(nn.Cell):
"""Depthwise Separable Convolutions"""
def __init__(self, dw_channels, out_channels, stride=1):
super(_DSConv, self).__init__()
self.conv = nn.SequentialCell(
[nn.Conv2d(in_channels=dw_channels, out_channels=dw_channels, \
kernel_size=3, stride=stride, pad_mode="pad", \
padding=1, group=dw_channels, has_bias=False),
nn.BatchNorm2d(dw_channels),
nn.ReLU(),
nn.Conv2d(in_channels=dw_channels, out_channels=out_channels, \
kernel_size=1, has_bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU()]
)
def construct(self, x):
return self.conv(x)
class _DWConv(nn.Cell):
'''_DWConv'''
def __init__(self, dw_channels, out_channels, stride=1):
super(_DWConv, self).__init__()
self.conv = nn.SequentialCell(
[nn.Conv2d(in_channels=dw_channels, out_channels=out_channels, \
kernel_size=3, stride=stride, pad_mode="pad", padding=1, \
group=dw_channels, has_bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU()]
)
def construct(self, x):
return self.conv(x)
class LinearBottleneck(nn.Cell):
"""LinearBottleneck used in MobileNetV2"""
def __init__(self, in_channels, out_channels, t=6, stride=2):
super(LinearBottleneck, self).__init__()
self.use_shortcut = (stride == 1) and (in_channels == out_channels)
self.block = nn.SequentialCell([
# pw
_ConvBNReLU(in_channels, in_channels * t, 1),
# dw
_DWConv(in_channels * t, in_channels * t, stride),
# pw-linear
nn.Conv2d(in_channels * t, out_channels, 1, has_bias=False),
nn.BatchNorm2d(out_channels)])
def construct(self, x):
out = self.block(x)
if self.use_shortcut:
out = x + out
return out
class PyramidPooling(nn.Cell):
"""Pyramid pooling module"""
def __init__(self, in_channels, out_channels):
super(PyramidPooling, self).__init__()
inter_channels = int(in_channels / 4)
self.conv1 = _ConvBNReLU(in_channels, inter_channels, 1)
self.conv2 = _ConvBNReLU(in_channels, inter_channels, 1)
self.conv3 = _ConvBNReLU(in_channels, inter_channels, 1)
self.conv4 = _ConvBNReLU(in_channels, inter_channels, 1)
self.out = _ConvBNReLU(in_channels * 2, out_channels, 1)
self.concat = P.Concat(axis=1)
self.pool24 = nn.AvgPool2d(kernel_size=24, stride=24)
self.pool12 = nn.AvgPool2d(kernel_size=12, stride=12)
self.pool8 = nn.AvgPool2d(kernel_size=8, stride=8)
self.pool6 = nn.AvgPool2d(kernel_size=4, stride=4)
self.resizeBilinear = nn.ResizeBilinear()
def _AdaptiveAvgPool2d(self, x, output_size):
#NCHW, for NCx24x24 and size in (1,2,3,6) only
if output_size == 1:
return self.pool24(x)
if output_size == 2:
return self.pool12(x)
if output_size == 3:
return self.pool8(x)
return self.pool6(x)
def upsample(self, x, size):
return self.resizeBilinear(x, size, align_corners=True)
def construct(self, x):
size = x.shape[2:]
feat1 = self.upsample(self.conv1(self._AdaptiveAvgPool2d(x, 1)), size)
feat2 = self.upsample(self.conv2(self._AdaptiveAvgPool2d(x, 2)), size)
feat3 = self.upsample(self.conv3(self._AdaptiveAvgPool2d(x, 3)), size)
feat4 = self.upsample(self.conv4(self._AdaptiveAvgPool2d(x, 6)), size)
x = self.concat((x, feat1, feat2, feat3, feat4))
x = self.out(x)
return x
class LearningToDownsample(nn.Cell):
"""Learning to downsample module"""
def __init__(self, dw_channels1=32, dw_channels2=48, out_channels=64):
super(LearningToDownsample, self).__init__()
self.conv = _ConvBNReLU(3, dw_channels1, 3, 2)
self.dsconv1 = _DSConv(dw_channels1, dw_channels2, 2)
self.dsconv2 = _DSConv(dw_channels2, out_channels, 2)
def construct(self, x):
x = self.conv(x)
x = self.dsconv1(x)
x = self.dsconv2(x)
return x
class GlobalFeatureExtractor(nn.Cell):
"""Global feature extractor module"""
def __init__(self, in_channels=64, block_channels=(64, 96, 128),
out_channels=128, t=6, num_blocks=(3, 3, 3)):
super(GlobalFeatureExtractor, self).__init__()
self.bottleneck1 = self._make_layer(LinearBottleneck, in_channels, \
block_channels[0], num_blocks[0], t, 2)
self.bottleneck2 = self._make_layer(LinearBottleneck, block_channels[0], \
block_channels[1], num_blocks[1], t, 2)
self.bottleneck3 = self._make_layer(LinearBottleneck, block_channels[1], \
block_channels[2], num_blocks[2], t, 1)
self.ppm = PyramidPooling(block_channels[2], out_channels)
def _make_layer(self, block, inplanes, planes, blocks, t=6, stride=1):
layers = []
layers.append(block(inplanes, planes, t, stride))
for _ in range(1, blocks):
layers.append(block(planes, planes, t, 1))
return nn.SequentialCell(layers)
def construct(self, x):
x = self.bottleneck1(x)
x = self.bottleneck2(x)
x = self.bottleneck3(x)
x = self.ppm(x)
return x
class FeatureFusionModule(nn.Cell):
"""Feature fusion module"""
def __init__(self, highter_in_channels, lower_in_channels, out_channels, scale_factor=4):
super(FeatureFusionModule, self).__init__()
self.scale_factor = scale_factor
self.dwconv = _DWConv(lower_in_channels, out_channels, 1)
self.conv_lower_res = nn.SequentialCell([
nn.Conv2d(out_channels, out_channels, 1),
nn.BatchNorm2d(out_channels)])
self.conv_higher_res = nn.SequentialCell([
nn.Conv2d(highter_in_channels, out_channels, 1),
nn.BatchNorm2d(out_channels)])
self.relu = nn.ReLU()
self.ResizeBilinear = nn.ResizeBilinear()
def construct(self, higher_res_feature, lower_res_feature):
lower_res_feature = self.ResizeBilinear(lower_res_feature, \
scale_factor=4, align_corners=True)
lower_res_feature = self.dwconv(lower_res_feature)
lower_res_feature = self.conv_lower_res(lower_res_feature)
higher_res_feature = self.conv_higher_res(higher_res_feature)
out = higher_res_feature + lower_res_feature
return self.relu(out)
class Classifier(nn.Cell):
"""Classifier"""
def __init__(self, dw_channels, num_classes, stride=1, **kwargs):
super(Classifier, self).__init__()
self.dsconv1 = _DSConv(dw_channels, dw_channels, stride)
self.dsconv2 = _DSConv(dw_channels, dw_channels, stride)
self.conv = nn.SequentialCell([
nn.Dropout(0.9), # 1-0.9=0.1
nn.Conv2d(dw_channels, num_classes, 1)])
def construct(self, x):
x = self.dsconv1(x)
x = self.dsconv2(x)
x = self.conv(x)
return x
class FastSCNNWithLossCell(nn.Cell):
"""FastSCNN loss, MixSoftmaxCrossEntropyLoss."""
def __init__(self, network, args):
super(FastSCNNWithLossCell, self).__init__()
self.network = network
self.aux = args.aux
self.loss = MixSoftmaxCrossEntropyLoss(args, aux=args.aux, aux_weight=args.aux_weight)
def construct(self, images, targets):
outputs = self.network(images)
if self.aux:
return self.loss(outputs[0], outputs[1], outputs[2], targets)
return self.loss(outputs, targets)

View File

@ -0,0 +1,80 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Custom Logger."""
import os
import sys
import logging
from datetime import datetime
class LOGGER(logging.Logger):
"""
Logger.
Args:
logger_name: String. Logger name.
rank: Integer. Rank id.
"""
def __init__(self, logger_name, rank=0):
super(LOGGER, self).__init__(logger_name)
self.rank = rank
if rank % 8 == 0:
console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s')
console.setFormatter(formatter)
self.addHandler(console)
def setup_logging_file(self, log_dir, rank=0):
"""Setup logging file."""
self.rank = rank
if not os.path.exists(log_dir):
os.makedirs(log_dir, exist_ok=True)
log_name = datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S') + '_rank_{}.log'.format(rank)
self.log_fn = os.path.join(log_dir, log_name)
fh = logging.FileHandler(self.log_fn)
fh.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s')
fh.setFormatter(formatter)
self.addHandler(fh)
def info(self, msg, *args, **kwargs):
if self.isEnabledFor(logging.INFO):
self._log(logging.INFO, msg, args, **kwargs)
def save_args(self, args):
self.info('Args:')
args_dict = vars(args)
for key in args_dict.keys():
self.info('--> %s: %s', key, args_dict[key])
self.info('')
def important_info(self, msg, *args, **kwargs):
if self.isEnabledFor(logging.INFO) and self.rank == 0:
line_width = 2
important_msg = '\n'
important_msg += ('*'*70 + '\n')*line_width
important_msg += ('*'*line_width + '\n')*2
important_msg += '*'*line_width + ' '*8 + msg + '\n'
important_msg += ('*'*line_width + '\n')*2
important_msg += ('*'*70 + '\n')*line_width
self.info(important_msg, *args, **kwargs)
def get_logger(path, logger_name, rank):
"""Get Logger."""
logger = LOGGER(logger_name, rank)
logger.setup_logging_file(path, rank)
return logger

View File

@ -0,0 +1,66 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Custom losses."""
import mindspore
import mindspore.nn as nn
import mindspore.ops as ops
from mindspore.common.tensor import Tensor
from mindspore.nn import SoftmaxCrossEntropyWithLogits
__all__ = ['MixSoftmaxCrossEntropyLoss']
class MixSoftmaxCrossEntropyLoss(nn.Cell):
'''MixSoftmaxCrossEntropyLoss'''
def __init__(self, args, ignore_label=-1, aux=True, aux_weight=0.4, \
sparse=True, reduction='none', one_d_length=2*768*768, **kwargs):
super(MixSoftmaxCrossEntropyLoss, self).__init__()
self.ignore_label = ignore_label
self.weight = aux_weight if aux else 1.0
self.select = ops.Select()
self.reduceSum = ops.ReduceSum(keep_dims=False)
self.div_no_nan = ops.DivNoNan()
self.mul = ops.Mul()
self.reshape = ops.Reshape()
self.cast = ops.Cast()
self.transpose = ops.Transpose()
self.zero_tensor = Tensor([0]*one_d_length, mindspore.float32)
self.SoftmaxCrossEntropyWithLogits = \
SoftmaxCrossEntropyWithLogits(sparse=sparse, reduction="none")
args.logger.info('using MixSoftmaxCrossEntropyLoss....')
args.logger.info('self.ignore_label:' + str(self.ignore_label))
args.logger.info('self.aux:' + str(aux))
args.logger.info('self.weight:' + str(self.weight))
args.logger.info('one_d_length:' + str(one_d_length))
def construct(self, *inputs, **kwargs):
'''construct'''
preds, target = inputs[:-1], inputs[-1]
target = self.reshape(target, (-1,))
valid_flag = target != self.ignore_label
num_valid = self.reduceSum(self.cast(valid_flag, mindspore.float32))
z = self.transpose(preds[0], (0, 2, 3, 1))#move the C-dim to the last, then reshape it.
#This operation is vital, or the data would be soiled.
loss = self.SoftmaxCrossEntropyWithLogits(self.reshape(z, (-1, 19)), target)
loss = self.select(valid_flag, loss, self.zero_tensor)
loss = self.reduceSum(loss)
loss = self.div_no_nan(loss, num_valid)
for i in range(1, len(preds)):
z = self.transpose(preds[i], (0, 2, 3, 1))
aux_loss = self.SoftmaxCrossEntropyWithLogits(self.reshape(z, (-1, 19)), target)
aux_loss = self.select(valid_flag, aux_loss, self.zero_tensor)
aux_loss = self.reduceSum(aux_loss)
aux_loss = self.div_no_nan(aux_loss, num_valid)
loss += self.mul(self.weight, aux_loss)
return loss

View File

@ -0,0 +1,113 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Popular Learning Rate Schedulers"""
import math
class LRScheduler():
r"""Learning Rate Scheduler
Parameters
----------
mode : str
Modes for learning rate scheduler.
Currently it supports 'constant', 'step', 'linear', 'poly' and 'cosine'.
base_lr : float
Base learning rate, i.e. the starting learning rate.
target_lr : float
Target learning rate, i.e. the ending learning rate.
With constant mode target_lr is ignored.
niters : int
Number of iterations to be scheduled.
nepochs : int
Number of epochs to be scheduled.
iters_per_epoch : int
Number of iterations in each epoch.
offset : int
Number of iterations before this scheduler.
power : float
Power parameter of poly scheduler.
step_iter : list
A list of iterations to decay the learning rate.
step_epoch : list
A list of epochs to decay the learning rate.
step_factor : float
Learning rate decay factor.
"""
def __init__(self, mode, base_lr=0.01, target_lr=0, niters=0, nepochs=0, iters_per_epoch=0,
offset=0, power=2, step_iter=None, step_epoch=None, step_factor=0.1):
super(LRScheduler, self).__init__()
assert (mode in ['constant', 'step', 'linear', 'poly', 'cosine'])
self.mode = mode
if mode == 'step':
assert (step_iter is not None or step_epoch is not None)
self.base_lr = base_lr
self.target_lr = target_lr
if self.mode == 'constant':
self.target_lr = self.base_lr
self.niters = niters
self.step = step_iter
epoch_iters = nepochs * iters_per_epoch
if epoch_iters > 0:
self.niters = epoch_iters
if step_epoch is not None:
self.step = [s * iters_per_epoch for s in step_epoch]
self.offset = offset
self.power = power
self.step_factor = step_factor
def __call__(self, total_steps):
lr_each_step = []
for i in range(total_steps):
self.update(i)
lr_each_step.append(self.learning_rate)
return lr_each_step
def update(self, num_update):
'''update'''
N = self.niters - 1
T = num_update - self.offset
T = min(max(0, T), N)
if self.mode == 'constant':
factor = 0
elif self.mode == 'linear':
factor = 1 - T / N
elif self.mode == 'poly':
factor = pow(1 - T / N, self.power)
elif self.mode == 'cosine':
factor = (1 + math.cos(math.pi * T / N)) / 2
elif self.mode == 'step':
if self.step is not None:
count = sum([1 for s in self.step if s <= T])
factor = pow(self.step_factor, count)
else:
factor = 1
else:
raise NotImplementedError
if self.mode == 'step':
self.learning_rate = self.base_lr * factor
else:
self.learning_rate = self.target_lr + (self.base_lr - self.target_lr) * factor
if __name__ == '__main__':
lr_scheduler = LRScheduler(mode='poly', base_lr=0.01, nepochs=60,
iters_per_epoch=176, power=0.9)
lr = lr_scheduler(200)
print(lr)

View File

@ -0,0 +1,167 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Evaluation Metrics for Semantic Segmentation"""
import numpy as np
from mindspore.common.tensor import Tensor
__all__ = ['SegmentationMetric', 'batch_pix_accuracy', 'batch_intersection_union',
'pixelAccuracy', 'intersectionAndUnion', 'hist_info', 'compute_score']
class SegmentationMetric():
"""Computes pixAcc and mIoU metric scores
"""
def __init__(self, nclass):
super(SegmentationMetric, self).__init__()
self.nclass = nclass
self.reset()
def update(self, preds, labels):
"""Updates the internal evaluation result.
Parameters
----------
labels : 'NumpyArray' or list of `NumpyArray`
The labels of the data.
preds : 'NumpyArray' or list of `NumpyArray`
Predicted values.
"""
def evaluate_worker(self, pred, label):
correct, labeled = batch_pix_accuracy(pred.asnumpy(), label.asnumpy())
inter, union = batch_intersection_union(pred.asnumpy(), label.asnumpy(), self.nclass)
self.total_correct += correct
self.total_label += labeled
self.total_inter += inter
self.total_union += union
if isinstance(preds, Tensor):
evaluate_worker(self, preds, labels)
elif isinstance(preds, (list, tuple)):
for (pred, label) in zip(preds, labels):
evaluate_worker(self, pred, label)
def get(self, return_category_iou=False):
"""Gets the current evaluation result.
Returns
-------
metrics : tuple of float
pixAcc and mIoU
"""
# remove np.spacing(1)
pixAcc = 1.0 * self.total_correct / (2.220446049250313e-16 + self.total_label)
IoU = 1.0 * self.total_inter / (2.220446049250313e-16 + self.total_union)
mIoU = IoU.mean().item()
if return_category_iou:
return pixAcc, mIoU, IoU
return pixAcc, mIoU
def reset(self):
"""Resets the internal evaluation result to initial state."""
self.total_inter = np.zeros(self.nclass)
self.total_union = np.zeros(self.nclass)
self.total_correct = 0
self.total_label = 0
def batch_pix_accuracy(output, target):
"""PixAcc"""
# inputs are numpy array, output 4D NCHW where 'C' means label classes, target 3D NHW
predict = np.argmax(output.astype(np.int64), 1) + 1
target = target.astype(np.int64) + 1
pixel_labeled = (target > 0).sum()
pixel_correct = ((predict == target) * (target > 0)).sum()
assert pixel_correct <= pixel_labeled, "Correct area should be smaller than Labeled"
return pixel_correct, pixel_labeled
def batch_intersection_union(output, target, nclass):
"""mIoU"""
# inputs are numpy array, output 4D, target 3D
mini = 1
maxi = nclass
nbins = nclass
predict = np.argmax(output.astype(np.float32), 1) + 1
target = target.astype(np.float32) + 1
predict = predict.astype(np.float32) * (target > 0).astype(np.float32)
intersection = predict * (predict == target).astype(np.float32)
# areas of intersection and union
# element 0 in intersection occur the main difference from np.bincount. set boundary to -1 is necessary.
area_inter, _ = np.histogram(intersection, bins=nbins, range=(mini, maxi))
area_pred, _ = np.histogram(predict, bins=nbins, range=(mini, maxi))
area_lab, _ = np.histogram(target, bins=nbins, range=(mini, maxi))
area_union = area_pred + area_lab - area_inter
assert (area_inter > area_union).sum() == 0, "Intersection area should be smaller than Union area"
return area_inter.astype(np.float32), area_union.astype(np.float32)
def pixelAccuracy(imPred, imLab):
"""
This function takes the prediction and label of a single image, returns pixel-wise accuracy
To compute over many images do:
for i = range(Nimages):
(pixel_accuracy[i], pixel_correct[i], pixel_labeled[i]) = \
pixelAccuracy(imPred[i], imLab[i])
mean_pixel_accuracy = 1.0 * np.sum(pixel_correct) / (np.spacing(1) + np.sum(pixel_labeled))
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
pixel_labeled = np.sum(imLab >= 0)
pixel_correct = np.sum((imPred == imLab) * (imLab >= 0))
pixel_accuracy = 1.0 * pixel_correct / pixel_labeled
return (pixel_accuracy, pixel_correct, pixel_labeled)
def intersectionAndUnion(imPred, imLab, numClass):
"""
This function takes the prediction and label of a single image,
returns intersection and union areas for each class
To compute over many images do:
for i in range(Nimages):
(area_intersection[:,i], area_union[:,i]) = intersectionAndUnion(imPred[i], imLab[i])
IoU = 1.0 * np.sum(area_intersection, axis=1) / np.sum(np.spacing(1)+area_union, axis=1)
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
imPred = imPred * (imLab >= 0)
# Compute area intersection:
intersection = imPred * (imPred == imLab)
(area_intersection, _) = np.histogram(intersection, bins=numClass, range=(1, numClass))
# Compute area union:
(area_pred, _) = np.histogram(imPred, bins=numClass, range=(1, numClass))
(area_lab, _) = np.histogram(imLab, bins=numClass, range=(1, numClass))
area_union = area_pred + area_lab - area_intersection
return (area_intersection, area_union)
def hist_info(pred, label, num_cls):
assert pred.shape == label.shape
k = (label >= 0) & (label < num_cls)
labeled = np.sum(k)
correct = np.sum((pred[k] == label[k]))
return np.bincount(num_cls * label[k].astype(int) + pred[k], \
minlength=num_cls ** 2).reshape(num_cls, num_cls), labeled, correct
def compute_score(hist, correct, labeled):
iu = np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist))
mean_IU = np.nanmean(iu)
mean_IU_no_back = np.nanmean(iu[1:])
mean_pixel_acc = correct / labeled
return iu, mean_IU, mean_IU_no_back, mean_pixel_acc

View File

@ -0,0 +1,118 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Base segmentation dataset"""
import random
import numpy as np
from PIL import Image, ImageOps, ImageFilter
__all__ = ['SegmentationDataset']
class SegmentationDataset():
"""Segmentation Base Dataset"""
def __init__(self, root, split, mode, base_size=520, crop_size=480):
super(SegmentationDataset, self).__init__()
self.root = root
self.split = split
self.mode = mode if mode is not None else split
self.base_size = base_size
self.crop_size = self.to_tuple(crop_size)
def to_tuple(self, size):
if isinstance(size, (list, tuple)):
return tuple(size)
if isinstance(size, (int, float)):
return tuple((size, size))
raise ValueError('Unsupport datatype: {}'.format(type(size)))
def _val_sync_transform(self, img, mask):
'''_val_sync_transform'''
outsize = self.crop_size
short_size = min(outsize)
w, h = img.size
if w > h:
oh = short_size
ow = int(1.0 * w * oh / h)
else:
ow = short_size
oh = int(1.0 * h * ow / w)
img = img.resize((ow, oh), Image.BILINEAR)
mask = mask.resize((ow, oh), Image.NEAREST)
# center crop
w, h = img.size
x1 = int(round((w - outsize[1]) / 2.))
y1 = int(round((h - outsize[0]) / 2.))
img = img.crop((x1, y1, x1 + outsize[1], y1 + outsize[0]))
mask = mask.crop((x1, y1, x1 + outsize[1], y1 + outsize[0]))
# final transform
img, mask = self._img_transform(img), self._mask_transform(mask)
return img, mask
def _sync_transform(self, img, mask):
'''_sync_transform'''
# random mirror
if random.random() < 0.5:
img = img.transpose(Image.FLIP_LEFT_RIGHT)
mask = mask.transpose(Image.FLIP_LEFT_RIGHT)
crop_size = self.crop_size
# random scale (short edge)
short_size = random.randint(int(self.base_size * 0.5), int(self.base_size * 2.0))
w, h = img.size
if h > w:
ow = short_size
oh = int(1.0 * h * ow / w)
else:
oh = short_size
ow = int(1.0 * w * oh / h)
img = img.resize((ow, oh), Image.BILINEAR)
mask = mask.resize((ow, oh), Image.NEAREST)
# pad crop
if short_size < min(crop_size):
padh = crop_size[0] - oh if oh < crop_size[0] else 0
padw = crop_size[1] - ow if ow < crop_size[1] else 0
img = ImageOps.expand(img, border=(0, 0, padw, padh), fill=0)
mask = ImageOps.expand(mask, border=(0, 0, padw, padh), fill=-1)
# random crop crop_size
w, h = img.size
x1 = random.randint(0, w - crop_size[1])
y1 = random.randint(0, h - crop_size[0])
img = img.crop((x1, y1, x1 + crop_size[1], y1 + crop_size[0]))
mask = mask.crop((x1, y1, x1 + crop_size[1], y1 + crop_size[0]))
# gaussian blur as in PSP
if random.random() < 0.5:
img = img.filter(ImageFilter.GaussianBlur(
radius=random.random()))
# final transform
img, mask = self._img_transform(img), self._mask_transform(mask)
return img, mask
def _img_transform(self, img):
return np.array(img)
def _mask_transform(self, mask):
return np.array(mask).astype('int32')
@property
def num_class(self):
"""Number of categories."""
return self.NUM_CLASS
@property
def pred_offset(self):
return 0

View File

@ -0,0 +1,260 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Util class or function."""
import os
import stat
from datetime import datetime
import numpy as np
from mindspore import nn
from mindspore import save_checkpoint
from mindspore import log as logger
from mindspore.train.callback import Callback
from mindspore.common.tensor import Tensor
def apply_eval(eval_param_dict):
"""run Evaluation"""
model = eval_param_dict["model"]
dataset = eval_param_dict["dataset"]
eval_score = model.eval(dataset, dataset_sink_mode=False)["SegmentationMetric"]
return eval_score
class TempLoss(nn.Cell):
"""A temp loss cell."""
def construct(self, *inputs, **kwargs):
return 0.1
class SegmentationMetric(nn.Metric):
"""FastSCNN Metric, computes pixAcc and mIoU metric scores."""
def __init__(self, nclass):
super(SegmentationMetric, self).__init__()
self.nclass = nclass
self.clear()
def clear(self):
"""Resets the internal evaluation result to initial state."""
self.total_inter = np.zeros(self.nclass)
self.total_union = np.zeros(self.nclass)
self.total_correct = 0
self.total_label = 0
def update(self, *inputs):
"""Updates the internal evaluation result.
Parameters
----------
labels : 'NumpyArray' or list of `NumpyArray`
The labels of the data.
preds : 'NumpyArray' or list of `NumpyArray`
Predicted values.
"""
preds, labels = inputs[0], inputs[-1]
preds = preds[0]
#print("preds:",preds)
#print("labels:",labels)
def evaluate_worker(self, pred, label):
correct, labeled = batch_pix_accuracy(pred.asnumpy(), label.asnumpy())
inter, union = batch_intersection_union(pred.asnumpy(), label.asnumpy(), self.nclass)
self.total_correct += correct
self.total_label += labeled
self.total_inter += inter
self.total_union += union
if isinstance(preds, Tensor):
evaluate_worker(self, preds, labels)
elif isinstance(preds, (list, tuple)):
for (pred, label) in zip(preds, labels):
evaluate_worker(self, pred, label)
def eval(self):
"""Gets the current evaluation result.
Returns
-------
metrics : tuple of float
pixAcc and mIoU
"""
# remove np.spacing(1)
pixAcc = 1.0 * self.total_correct / (2.220446049250313e-16 + self.total_label)
IoU = 1.0 * self.total_inter / (2.220446049250313e-16 + self.total_union)
mIoU = IoU.mean().item()
return pixAcc, mIoU
class EvalCallBack(Callback):
"""
Evaluation callback when training.
Args:
eval_function (function): evaluation function.
eval_param_dict (dict): evaluation parameters' configure dict.
interval (int): run evaluation interval, default is 1.
eval_start_epoch (int): evaluation start epoch, default is 1.
save_best_ckpt (bool): Whether to save best checkpoint, default is True.
besk_ckpt_name (str): bast checkpoint name, default is `best.ckpt`.
metrics_name (str): evaluation metrics name, default is `acc`.
Returns:
None
Examples:
>>> EvalCallBack(eval_function, eval_param_dict)
"""
def __init__(self, eval_function, eval_param_dict, interval=1, eval_start_epoch=1, \
save_best_ckpt=True, ckpt_directory="./", besk_ckpt_name="best.ckpt", metrics_name="acc"):
super(EvalCallBack, self).__init__()
self.eval_param_dict = eval_param_dict
self.eval_function = eval_function
self.eval_start_epoch = eval_start_epoch
if interval < 1:
raise ValueError("interval should >= 1.")
self.interval = interval
self.save_best_ckpt = save_best_ckpt
self.best_res = 0
self.best_epoch = 0
if not os.path.isdir(ckpt_directory):
os.makedirs(ckpt_directory)
self.bast_ckpt_path = os.path.join(ckpt_directory, besk_ckpt_name)
self.metrics_name = metrics_name
def remove_ckpoint_file(self, file_name):
"""Remove the specified checkpoint file from this checkpoint manager and also from the directory."""
try:
os.chmod(file_name, stat.S_IWRITE)
os.remove(file_name)
except OSError:
logger.warning("OSError, failed to remove the older ckpt file %s.", file_name)
except ValueError:
logger.warning("ValueError, failed to remove the older ckpt file %s.", file_name)
def epoch_end(self, run_context):
"""Callback when epoch end."""
cb_params = run_context.original_args()
cur_epoch = cb_params.cur_epoch_num
if cur_epoch >= self.eval_start_epoch and (cur_epoch - self.eval_start_epoch) % self.interval == 0:
res = self.eval_function(self.eval_param_dict)
print(datetime.now().strftime('%Y-%m-%d %H:%M:%S,%f')[:-3],\
":INFO: epoch: {}, {}: {}, {}: {}".format(cur_epoch, self.metrics_name[0], \
res[0]*100, self.metrics_name[1], res[1]*100), flush=True)
if res[1] >= self.best_res:
self.best_res = res[1]
self.best_epoch = cur_epoch
print(datetime.now().strftime('%Y-%m-%d %H:%M:%S,%f')[:-3],\
":INFO: update best result: {}".format(res[1]*100), flush=True)
if self.save_best_ckpt:
if os.path.exists(self.bast_ckpt_path):
self.remove_ckpoint_file(self.bast_ckpt_path)
save_checkpoint(cb_params.train_network, self.bast_ckpt_path)
print(datetime.now().strftime('%Y-%m-%d %H:%M:%S,%f')[:-3],\
":INFO: update best checkpoint at: {}".format(self.bast_ckpt_path), flush=True)
def end(self, run_context):
print(datetime.now().strftime('%Y-%m-%d %H:%M:%S,%f')[:-3],\
":INFO: End training, the best {0} is: {1}, it's epoch is {2}".format(self.metrics_name[1],\
self.best_res*100, self.best_epoch), flush=True)
def batch_pix_accuracy(output, target):
"""PixAcc"""
# inputs are numpy array, output 4D NCHW where 'C' means label classes, target 3D NHW
predict = np.argmax(output.astype(np.int64), 1) + 1
target = target.astype(np.int64) + 1
pixel_labeled = (target > 0).sum()
pixel_correct = ((predict == target) * (target > 0)).sum()
assert pixel_correct <= pixel_labeled, "Correct area should be smaller than Labeled"
return pixel_correct, pixel_labeled
def batch_intersection_union(output, target, nclass):
"""mIoU"""
# inputs are numpy array, output 4D, target 3D
mini = 1
maxi = nclass
nbins = nclass
predict = np.argmax(output.astype(np.float32), 1) + 1
target = target.astype(np.float32) + 1
predict = predict.astype(np.float32) * (target > 0).astype(np.float32)
intersection = predict * (predict == target).astype(np.float32)
# areas of intersection and union
# element 0 in intersection occur the main difference from np.bincount. set boundary to -1 is necessary.
area_inter, _ = np.histogram(intersection, bins=nbins, range=(mini, maxi))
area_pred, _ = np.histogram(predict, bins=nbins, range=(mini, maxi))
area_lab, _ = np.histogram(target, bins=nbins, range=(mini, maxi))
area_union = area_pred + area_lab - area_inter
assert (area_inter > area_union).sum() == 0, "Intersection area should be smaller than Union area"
return area_inter.astype(np.float32), area_union.astype(np.float32)
def pixelAccuracy(imPred, imLab):
"""
This function takes the prediction and label of a single image, returns pixel-wise accuracy
To compute over many images do:
for i = range(Nimages):
(pixel_accuracy[i], pixel_correct[i], pixel_labeled[i]) = \
pixelAccuracy(imPred[i], imLab[i])
mean_pixel_accuracy = 1.0 * np.sum(pixel_correct) / (np.spacing(1) + np.sum(pixel_labeled))
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
pixel_labeled = np.sum(imLab >= 0)
pixel_correct = np.sum((imPred == imLab) * (imLab >= 0))
pixel_accuracy = 1.0 * pixel_correct / pixel_labeled
return (pixel_accuracy, pixel_correct, pixel_labeled)
def intersectionAndUnion(imPred, imLab, numClass):
"""
This function takes the prediction and label of a single image,
returns intersection and union areas for each class
To compute over many images do:
for i in range(Nimages):
(area_intersection[:,i], area_union[:,i]) = intersectionAndUnion(imPred[i], imLab[i])
IoU = 1.0 * np.sum(area_intersection, axis=1) / np.sum(np.spacing(1)+area_union, axis=1)
"""
# Remove classes from unlabeled pixels in gt image.
# We should not penalize detections in unlabeled portions of the image.
imPred = imPred * (imLab >= 0)
# Compute area intersection:
intersection = imPred * (imPred == imLab)
(area_intersection, _) = np.histogram(intersection, bins=numClass, range=(1, numClass))
# Compute area union:
(area_pred, _) = np.histogram(imPred, bins=numClass, range=(1, numClass))
(area_lab, _) = np.histogram(imLab, bins=numClass, range=(1, numClass))
area_union = area_pred + area_lab - area_intersection
return (area_intersection, area_union)
def hist_info(pred, label, num_cls):
assert pred.shape == label.shape
k = (label >= 0) & (label < num_cls)
labeled = np.sum(k)
correct = np.sum((pred[k] == label[k]))
return np.bincount(num_cls * label[k].astype(int) + pred[k], minlength=num_cls ** 2).\
reshape(num_cls, num_cls), labeled, correct
def compute_score(hist, correct, labeled):
iu = np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist))
mean_IU = np.nanmean(iu)
mean_IU_no_back = np.nanmean(iu[1:])
#freq = hist.sum(1) / hist.sum()
# freq_IU = (iu[freq > 0] * freq[freq > 0]).sum()
mean_pixel_acc = correct / labeled
return iu, mean_IU, mean_IU_no_back, mean_pixel_acc

View File

@ -0,0 +1,114 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Visualization Utils"""
from PIL import Image
__all__ = ['get_color_pallete']
def get_color_pallete(npimg, dataset='citys'):
"""Visualize image.
Parameters
----------
npimg : numpy.ndarray
Single channel image with shape `H, W, 1`.
dataset : str, default: 'pascal_voc'
The dataset that model pretrained on. ('pascal_voc', 'ade20k')
Returns
-------
out_img : PIL.Image
Image with color palette
"""
# recovery boundary
if dataset in ('pascal_voc', 'pascal_aug'):
npimg[npimg == -1] = 255
# put colormap
if dataset == 'ade20k':
npimg = npimg + 1
out_img = Image.fromarray(npimg.astype('uint8'))
out_img.putpalette(adepallete)
return out_img
if dataset == 'citys':
out_img = Image.fromarray(npimg.astype('uint8'))
out_img.putpalette(cityspallete)
return out_img
out_img = Image.fromarray(npimg.astype('uint8'))
out_img.putpalette(vocpallete)
return out_img
def _getvocpallete(num_cls):
'''_getvocpallete'''
n = num_cls
palette = [0] * (n * 3)
for j in range(0, n):
lab = j
palette[j * 3 + 0] = 0
palette[j * 3 + 1] = 0
palette[j * 3 + 2] = 0
i = 0
while lab > 0:
palette[j * 3 + 0] |= (((lab >> 0) & 1) << (7 - i))
palette[j * 3 + 1] |= (((lab >> 1) & 1) << (7 - i))
palette[j * 3 + 2] |= (((lab >> 2) & 1) << (7 - i))
i = i + 1
lab >>= 3
return palette
vocpallete = _getvocpallete(256)
adepallete = [
0, 0, 0, 120, 120, 120, 180, 120, 120, 6, 230, 230, 80, 50, 50, 4, 200, 3, 120, 120, 80, 140, 140, 140, 204,
5, 255, 230, 230, 230, 4, 250, 7, 224, 5, 255, 235, 255, 7, 150, 5, 61, 120, 120, 70, 8, 255, 51, 255, 6, 82,
143, 255, 140, 204, 255, 4, 255, 51, 7, 204, 70, 3, 0, 102, 200, 61, 230, 250, 255, 6, 51, 11, 102, 255, 255,
7, 71, 255, 9, 224, 9, 7, 230, 220, 220, 220, 255, 9, 92, 112, 9, 255, 8, 255, 214, 7, 255, 224, 255, 184, 6,
10, 255, 71, 255, 41, 10, 7, 255, 255, 224, 255, 8, 102, 8, 255, 255, 61, 6, 255, 194, 7, 255, 122, 8, 0, 255,
20, 255, 8, 41, 255, 5, 153, 6, 51, 255, 235, 12, 255, 160, 150, 20, 0, 163, 255, 140, 140, 140, 250, 10, 15,
20, 255, 0, 31, 255, 0, 255, 31, 0, 255, 224, 0, 153, 255, 0, 0, 0, 255, 255, 71, 0, 0, 235, 255, 0, 173, 255,
31, 0, 255, 11, 200, 200, 255, 82, 0, 0, 255, 245, 0, 61, 255, 0, 255, 112, 0, 255, 133, 255, 0, 0, 255, 163,
0, 255, 102, 0, 194, 255, 0, 0, 143, 255, 51, 255, 0, 0, 82, 255, 0, 255, 41, 0, 255, 173, 10, 0, 255, 173, 255,
0, 0, 255, 153, 255, 92, 0, 255, 0, 255, 255, 0, 245, 255, 0, 102, 255, 173, 0, 255, 0, 20, 255, 184, 184, 0,
31, 255, 0, 255, 61, 0, 71, 255, 255, 0, 204, 0, 255, 194, 0, 255, 82, 0, 10, 255, 0, 112, 255, 51, 0, 255, 0,
194, 255, 0, 122, 255, 0, 255, 163, 255, 153, 0, 0, 255, 10, 255, 112, 0, 143, 255, 0, 82, 0, 255, 163, 255,
0, 255, 235, 0, 8, 184, 170, 133, 0, 255, 0, 255, 92, 184, 0, 255, 255, 0, 31, 0, 184, 255, 0, 214, 255, 255,
0, 112, 92, 255, 0, 0, 224, 255, 112, 224, 255, 70, 184, 160, 163, 0, 255, 153, 0, 255, 71, 255, 0, 255, 0,
163, 255, 204, 0, 255, 0, 143, 0, 255, 235, 133, 255, 0, 255, 0, 235, 245, 0, 255, 255, 0, 122, 255, 245, 0,
10, 190, 212, 214, 255, 0, 0, 204, 255, 20, 0, 255, 255, 255, 0, 0, 153, 255, 0, 41, 255, 0, 255, 204, 41, 0,
255, 41, 255, 0, 173, 0, 255, 0, 245, 255, 71, 0, 255, 122, 0, 255, 0, 255, 184, 0, 92, 255, 184, 255, 0, 0,
133, 255, 255, 214, 0, 25, 194, 194, 102, 255, 0, 92, 0, 255]
cityspallete = [
128, 64, 128,
244, 35, 232,
70, 70, 70,
102, 102, 156,
190, 153, 153,
153, 153, 153,
250, 170, 30,
220, 220, 0,
107, 142, 35,
152, 251, 152,
0, 130, 180,
220, 20, 60,
255, 0, 0,
0, 0, 142,
0, 0, 70,
0, 60, 100,
0, 80, 100,
0, 0, 230,
119, 11, 32,
]

View File

@ -0,0 +1,227 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
'''train.py'''
import os
import math
import argparse
import datetime
import mindspore
import mindspore.nn as nn
from mindspore import context
from mindspore.train import Model
from mindspore.common import set_seed
from mindspore.common.tensor import Tensor
from mindspore.context import ParallelMode
from mindspore import FixedLossScaleManager
from mindspore import load_checkpoint, load_param_into_net
from mindspore.dataset.transforms.py_transforms import Compose
from mindspore.dataset.vision.py_transforms import ToTensor, Normalize
from mindspore.communication.management import init, get_rank, get_group_size
from mindspore.train.callback import TimeMonitor, LossMonitor, CheckpointConfig, ModelCheckpoint
from src.logger import get_logger
from src.lr_scheduler import LRScheduler
from src.dataloader import create_CitySegmentation
from src.fast_scnn import FastSCNN, FastSCNNWithLossCell
from src.util import SegmentationMetric, EvalCallBack, apply_eval, TempLoss
def parse_args():
"""Training Options for Segmentation Experiments"""
parser = argparse.ArgumentParser(description='Fast-SCNN on mindspore')
parser.add_argument('--dataset', type=str, default='/data/dataset/citys/',
help='dataset name (default: /data/dataset/citys/)')
parser.add_argument('--base_size', type=int, default=1024, help='base image size')
parser.add_argument('--crop_size', type=int, default=(768, 768), help='crop image size')
parser.add_argument('--train_split', type=str, default='train',
help='dataset train split (default: train)')
parser.add_argument('--aux', action='store_true', default=True, help='Auxiliary loss')
parser.add_argument('--aux_weight', type=float, default=0.4,
help='auxiliary loss weight')
parser.add_argument('--epochs', type=int, default=1000, metavar='N',
help='number of epochs to train (default: 1000)')
parser.add_argument('--save_every', type=int, default=1, metavar='N',
help='save ckpt every N epoch')
parser.add_argument('--resume_path', type=str, default=None,
help='put the path to resuming file if needed')
parser.add_argument('--resume_name', type=str, default=None,
help='resuming file name')
parser.add_argument('--batch_size', type=int, default=2, metavar='N',
help='input batch size for training (default: 2)')
parser.add_argument('--lr', type=float, default=0.001, metavar='LR',
help='base learning rate (default: 0.045)')
parser.add_argument('--momentum', type=float, default=0.9, metavar='M',
help='momentum (default: 0.9)')
parser.add_argument('--weight_decay', type=float, default=4e-5, metavar='M',
help='w-decay (default: 4e-5)')
parser.add_argument('--eval_while_train', type=int, default=1, help='eval while training')
parser.add_argument('--eval_steps', type=int, default=10, help='each N epochs we eval')
parser.add_argument('--eval_start_epoch', type=int, default=850, help='eval_start_epoch')
parser.add_argument('--use_modelarts', type=int, default=0,
help='when set True, we should load dataset from obs with moxing')
parser.add_argument('--train_url', type=str, default='train_url/',
help='needed by modelarts, but we donot use it because the name is ambiguous')
parser.add_argument('--data_url', type=str, default='data_url/',
help='needed by modelarts, but we donot use it because the name is ambiguous')
parser.add_argument('--output_path', type=str, default='./outputs/',
help='output_path,when use_modelarts is set True, it will be cache/output/')
parser.add_argument('--outer_path', type=str, default='s3://output/',
help='obs path,to store e.g ckpt files ')
parser.add_argument('--device_target', type=str, default='Ascend',
help='device where the code will be implemented. (Default: Ascend)')
parser.add_argument('--is_distributed', type=int, default=0, help='if multi device')
parser.add_argument('--rank', type=int, default=0, help='local rank of distributed')
parser.add_argument('--group_size', type=int, default=1, help='world size of distributed')
parser.add_argument('--is_save_on_master', type=int, default=1,
help='save ckpt on master or all rank')
parser.add_argument('--ckpt_save_max', type=int, default=800,
help='Maximum number of checkpoint files can be saved. Default: 800')
# the parser
args_ = parser.parse_args()
return args_
args = parse_args()
set_seed(1)
device_id = int(os.getenv('DEVICE_ID', '0'))
context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target, save_graphs=False)
save_dir = os.path.join(args.output_path, datetime.datetime.now().strftime('%Y-%m-%d_time_%H_%M_%S'))
def train():
'''train'''
if args.is_distributed:
assert args.device_target == "Ascend"
context.set_context(device_id=device_id)
init()
args.rank = get_rank()
args.group_size = get_group_size()
device_num = args.group_size
context.reset_auto_parallel_context()
context.set_auto_parallel_context(device_num=device_num, parallel_mode=ParallelMode.DATA_PARALLEL)
else:
if args.device_target in ["Ascend", "GPU"]:
context.set_context(device_id=device_id)
# select for master rank save ckpt or all rank save, compatible for model parallel
args.rank_save_ckpt_flag = 0
if args.is_save_on_master:
if args.rank == 0:
args.rank_save_ckpt_flag = 1
else:
args.rank_save_ckpt_flag = 1
args.logger = get_logger(save_dir, "Fast_SCNN", args.rank)
args.logger.save_args(args)
# image transform
input_transform = Compose([
ToTensor(),
Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])
if args.use_modelarts:
import moxing as mox
args.logger.info("copying dataset from obs to cache....")
mox.file.copy_parallel(args.dataset, 'cache/dataset')
args.logger.info("copying dataset finished....")
args.dataset = 'cache/dataset/'
train_dataset, train_dataset_len = create_CitySegmentation(args, data_path=args.dataset, \
split=args.train_split, mode='train', transform=input_transform, \
base_size=args.base_size, crop_size=args.crop_size, batch_size=args.batch_size, \
device_num=args.group_size, rank=args.rank, shuffle=True)
args.steps_per_epoch = math.ceil(train_dataset_len / args.batch_size / args.group_size)
# create network
f_model = FastSCNN(num_classes=19, aux=args.aux)
# resume checkpoint if needed
# resume checkpoint if needed
if args.resume_path:
if args.use_modelarts:
import moxing as mox
args.logger.info("copying resume checkpoint from obs to cache....")
mox.file.copy_parallel(args.resume_path, 'cache/resume_path')
args.logger.info("copying resume checkpoint finished....")
args.resume_path = 'cache/resume_path/'
args.resume_path = os.path.join(args.resume_path, args.resume_name)
args.logger.info('loading resume checkpoint {} into network'.format(args.resume_path))
load_param_into_net(f_model, load_checkpoint(args.resume_path))
args.logger.info('loaded resume checkpoint {} into network'.format(args.resume_path))
model = FastSCNNWithLossCell(f_model, args)
model.set_train()
# lr scheduling
lr_list = LRScheduler(mode='cosine', base_lr=args.lr, nepochs=args.epochs, \
iters_per_epoch=args.steps_per_epoch, power=0.9)(args.epochs*args.steps_per_epoch)
# optimizer
optimizer = nn.SGD(params=model.trainable_params(), momentum=args.momentum, \
learning_rate=Tensor(lr_list, mindspore.float32), \
weight_decay=args.weight_decay, loss_scale=1024)
loss_scale = FixedLossScaleManager(1024, drop_overflow_update=False)
model = Model(model, optimizer=optimizer, loss_scale_manager=loss_scale, amp_level="O0")
# define callbacks
if args.rank == 0:
time_cb = TimeMonitor(data_size=train_dataset_len)
loss_cb = LossMonitor()
callbacks = [time_cb, loss_cb]
else:
callbacks = None
if args.rank_save_ckpt_flag:
ckpt_config = CheckpointConfig(save_checkpoint_steps=args.steps_per_epoch*args.save_every,
keep_checkpoint_max=args.ckpt_save_max)
save_ckpt_path = os.path.join(save_dir, 'ckpt_' + str(args.rank) + '/')
ckpt_cb = ModelCheckpoint(config=ckpt_config,
directory=save_ckpt_path,
prefix='rank_'+str(args.rank))
callbacks.append(ckpt_cb)
if args.eval_while_train == 1 and args.rank == 0:
val_dataset, _ = create_CitySegmentation(args, data_path=args.dataset, \
split='val', mode='val', transform=input_transform, \
base_size=args.base_size, crop_size=args.crop_size, \
batch_size=1, device_num=1, \
rank=args.rank, shuffle=False)
loss_f = TempLoss()
network_eval = Model(f_model, loss_fn=loss_f, metrics={"SegmentationMetric": SegmentationMetric(19)})
eval_param_dict = {"model": network_eval, "dataset": val_dataset}
eval_cb = EvalCallBack(apply_eval, eval_param_dict, interval=args.eval_steps,
eval_start_epoch=args.eval_start_epoch, save_best_ckpt=True,
ckpt_directory=save_dir, besk_ckpt_name="best_map.ckpt",
metrics_name=("pixAcc", "mIou"))
callbacks.append(eval_cb)
model.train(args.epochs, train_dataset, callbacks=callbacks, dataset_sink_mode=True)
args.logger.info("training finished....")
if args.use_modelarts:
import moxing as mox
args.logger.info("copying files from cache to obs....")
mox.file.copy_parallel(save_dir, args.outer_path)
args.logger.info("copying finished....")
if __name__ == '__main__':
print('Starting training, Total Epochs: %d' % (args.epochs))
train()