diff --git a/model_zoo/official/lite/image_segmentation/README.en.md b/model_zoo/official/lite/image_segmentation/README.en.md index d069b244c7e..7b5098b1abe 100644 --- a/model_zoo/official/lite/image_segmentation/README.en.md +++ b/model_zoo/official/lite/image_segmentation/README.en.md @@ -1,6 +1,6 @@ ## Demo of Image Segmentation -The following describes how to use the MindSpore Lite C++ APIs (Android JNIs) and MindSpore Lite image segmentation models to perform on-device inference, classify the content captured by a device camera, and display the most possible segmentation result on the application's image preview screen. +The following describes how to use the MindSpore Lite JAVA APIs and MindSpore Lite image segmentation models to perform on-device inference, classify the content captured by a device camera, and display the most possible segmentation result on the application's image preview screen. ### Running Dependencies @@ -34,8 +34,6 @@ The following describes how to use the MindSpore Lite C++ APIs (Android JNIs) an 3. Continue the installation on the Android device. After the installation is complete, you can view the content captured by a camera and the inference result. - ![result](images/app_result.jpg) - 4. The solutions of Android Studio configuration problems: | | Warning | Solution | @@ -49,28 +47,26 @@ The following describes how to use the MindSpore Lite C++ APIs (Android JNIs) an ## Detailed Description of the Sample Program -This image segmentation sample program on the Android device includes a Java layer and a JNI layer. At the Java layer, the Android Camera 2 API is used to enable a camera to obtain image frames and process images. At the JNI layer, the model inference process is completed in [Runtime](https://www.mindspore.cn/tutorial/lite/en/master/use/runtime.html). +This image segmentation sample program on the Android device is implemented through Java. At the Java layer, the Android Camera 2 API is used to enable a camera to obtain image frames and process images. Then Java API is called to infer.[Runtime](https://www.mindspore.cn/tutorial/lite/en/master/use/runtime.html). ### Sample Program Structure ```text app -│ ├── src/main │ ├── assets # resource files | | └── deeplabv3.ms # model file │ | -│ ├── cpp # main logic encapsulation classes for model loading and prediction -| | | -| | ├── MindSporeNetnative.cpp # JNI methods related to MindSpore calling -│ | └── MindSporeNetnative.h # header file -│ | │ ├── java # application code at the Java layer -│ │ └── com.mindspore.himindsporedemo -│ │ ├── gallery.classify # implementation related to image processing and MindSpore JNI calling -│ │ │ └── ... -│ │ └── widget # implementation related to camera enabling and drawing -│ │ └── ... +│ │ └── com.mindspore.imagesegmentation +│ │ ├── help # pre-process of image and inference of model +│ │ │ └── ImageUtils # image pre-process +│ │ │ └── ModelTrackingResult # post-process of result of inference +│ │ │ └── TrackingMobile # load model, compile graph and perform +│ │ └── BitmapUtils # image process +│ │ └── MainActivity # interactive page +│ │ └── OnBackgroundImageListener # get images from the photo album +│ │ └── StyleRecycleViewAdapter # get images from the photo album │ │ │ ├── res # resource files related to Android │ └── AndroidManifest.xml # Android configuration file @@ -84,7 +80,7 @@ app ### Configuring MindSpore Lite Dependencies -When MindSpore C++ APIs are called at the Android JNI layer, related library files are required. You can use MindSpore Lite [source code compilation](https://www.mindspore.cn/tutorial/lite/en/master/use/build.html) to generate the MindSpore Lite version. In this case, you need to use the compile command of generate with image preprocessing module. +When MindSpore Java APIs are called, related library files are required. You can use MindSpore Lite [source code compilation](https://www.mindspore.cn/tutorial/lite/en/master/use/build.html) to generate the MindSpore Lite version. In this case, you need to use the compile command of generate with image preprocessing module. In this example, the build process automatically downloads the `mindspore-lite-1.0.1-runtime-arm64-cpu` by the `app/download.gradle` file and saves in the `app/src/main/cpp` directory. diff --git a/model_zoo/official/lite/image_segmentation/README.md b/model_zoo/official/lite/image_segmentation/README.md index dd6dd974316..9ff53a44287 100644 --- a/model_zoo/official/lite/image_segmentation/README.md +++ b/model_zoo/official/lite/image_segmentation/README.md @@ -1,6 +1,6 @@ ## MindSpore Lite 端侧图像分割demo（Android） -本示例程序演示了如何在端侧利用MindSpore Lite C++ API（Android JNI）以及MindSpore Lite 图像分割模型完成端侧推理，实现对设备摄像头捕获的内容进行分割，并在App图像预览界面中显示出最可能的分割结果。 +本示例程序演示了如何在端侧利用MindSpore Lite Java API 以及MindSpore Lite 图像分割模型完成端侧推理，实现对设备摄像头捕获的内容进行分割，并在App图像预览界面中显示出最可能的分割结果。 ### 运行依赖 @@ -38,10 +38,6 @@ ![install](images/install.jpg) - 如下图所示，识别出的概率最高的物体是植物。 - - ![result](images/app_result.jpg) - 4. Android Studio 配置问题解决方案可参考下表： | | 报错 | 解决方案 | @@ -55,9 +51,9 @@ ## 示例程序详细说明 -本端侧图像分割Android示例程序分为JAVA层和JNI层，其中，JAVA层主要通过Android Camera 2 API实现摄像头获取图像帧，以及相应的图像处理等功能；JNI层完成模型推理的过程。 +本端侧图像分割Android示例程序使用Java实现，Java层主要通过Android Camera 2 API实现摄像头获取图像帧，进行相应的图像处理，之后调用Java API 完成模型推理。 -> 此处详细说明示例程序的JNI层实现，JAVA层运用Android Camera 2 API实现开启设备摄像头以及图像帧处理等功能，需读者具备一定的Android开发基础知识。 +> 此处详细说明示例程序的Java层图像处理及模型推理实现，Java层运用Android Camera 2 API实现开启设备摄像头以及图像帧处理等功能，需读者具备一定的Android开发基础知识。 ### 示例程序结构 @@ -67,19 +63,16 @@ app │ ├── assets # 资源文件 | | └── deeplabv3.ms # 存放模型文件 │ | -│ ├── cpp # 模型加载和预测主要逻辑封装类 -| | ├── .. -| | ├── mindspore_lite_x.x.x-minddata-arm64-cpu #MindSpore Lite版本 -| | ├── MindSporeNetnative.cpp # MindSpore调用相关的JNI方法 -│ | └── MindSporeNetnative.h # 头文件 -| | └── MsNetWork.cpp # MindSpre接口封装 -│ | │ ├── java # java层应用代码 -│ │ └── com.mindspore.himindsporedemo -│ │ ├── gallery.classify # 图像处理及MindSpore JNI调用相关实现 -│ │ │ └── ... -│ │ └── widget # 开启摄像头及绘制相关实现 -│ │ └── ... +│ │ └── com.mindspore.imagesegmentation +│ │ ├── help # 图像处理及MindSpore Java调用相关实现 +│ │ │ └── ImageUtils # 图像预处理 +│ │ │ └── ModelTrackingResult # 推理数据后处理 +│ │ │ └── TrackingMobile # 模型加载、构建计算图和推理 +│ │ └── BitmapUtils # 图像处理 +│ │ └── MainActivity # 交互主页面 +│ │ └── OnBackgroundImageListener # 获取相册图像 +│ │ └── StyleRecycleViewAdapter # 获取相册图像 │ │ │ ├── res # 存放Android相关的资源文件 │ └── AndroidManifest.xml # Android配置文件 @@ -93,7 +86,7 @@ app ### 配置MindSpore Lite依赖项 -Android JNI层调用MindSpore C++ API时，需要相关库文件支持。可通过MindSpore Lite[源码编译](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/build.html)生成`mindspore-lite-{version}-minddata-{os}-{device}.tar.gz`库文件包并解压缩（包含`libmindspore-lite.so`库文件和相关头文件），在本例中需使用生成带图像预处理模块的编译命令。 +Android 调用MindSpore Java API时，需要相关库文件支持。可通过MindSpore Lite[源码编译](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/build.html)生成`mindspore-lite-{version}-minddata-{os}-{device}.tar.gz`库文件包并解压缩（包含`libmindspore-lite.so`库文件和相关头文件），在本例中需使用生成带图像预处理模块的编译命令。 > version：输出件版本号，与所编译的分支代码对应的版本一致。 > diff --git a/model_zoo/official/lite/image_segmentation/images/home.png b/model_zoo/official/lite/image_segmentation/images/home.png new file mode 100644 index 00000000000..29e954a425c Binary files /dev/null and b/model_zoo/official/lite/image_segmentation/images/home.png differ diff --git a/model_zoo/official/lite/image_segmentation/images/install.jpg b/model_zoo/official/lite/image_segmentation/images/install.jpg new file mode 100644 index 00000000000..c98ee71dae7 Binary files /dev/null and b/model_zoo/official/lite/image_segmentation/images/install.jpg differ diff --git a/model_zoo/official/lite/image_segmentation/images/project_structure.png b/model_zoo/official/lite/image_segmentation/images/project_structure.png new file mode 100644 index 00000000000..6f71294479c Binary files /dev/null and b/model_zoo/official/lite/image_segmentation/images/project_structure.png differ diff --git a/model_zoo/official/lite/image_segmentation/images/run_app.PNG b/model_zoo/official/lite/image_segmentation/images/run_app.PNG new file mode 100644 index 00000000000..2557b6293de Binary files /dev/null and b/model_zoo/official/lite/image_segmentation/images/run_app.PNG differ diff --git a/model_zoo/official/lite/image_segmentation/images/sdk_management.png b/model_zoo/official/lite/image_segmentation/images/sdk_management.png new file mode 100644 index 00000000000..faf694bd2e6 Binary files /dev/null and b/model_zoo/official/lite/image_segmentation/images/sdk_management.png differ