[DOCS] Updating Interactive Tutorials (#22815)
Updating Interactive Tutorials - adding new notebooks.
This commit is contained in:
Sebastian Golebiewski
GitHub
parent
b1c699aab5
commit
bf36b03bd2
|
|
@ -59,6 +59,24 @@ The Jupyter notebooks are categorized into following classes:
|
||||||
Below you will find a selection of recommended tutorials that demonstrate inference on a particular model. These tutorials are guaranteed to provide a great experience with inference in OpenVINO:
|
Below you will find a selection of recommended tutorials that demonstrate inference on a particular model. These tutorials are guaranteed to provide a great experience with inference in OpenVINO:
|
||||||
|
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 284-openvoice
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/284-openvoice/284-openvoice.png
|
||||||
|
|
||||||
|
Voice tone cloning with OpenVoice and OpenVINO.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 283-photo-maker
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/283-photo-maker/283-photo-maker.gif
|
||||||
|
|
||||||
|
Text-to-image generation using PhotoMaker and OpenVINO.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 281-kosmos2-multimodal-large-language-model
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/281-kosmos2-multimodal-large-language-model/281-kosmos2-multimodal-large-language-model.png
|
||||||
|
|
||||||
|
Kosmos-2: Multimodal Large Language Model and OpenVINO.
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 280-depth-anything
|
:title: 280-depth-anything
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
||||||
|
|
@ -83,14 +101,12 @@ Below you will find a selection of recommended tutorials that demonstrate infere
|
||||||
|
|
||||||
LLM Instruction-following pipeline with OpenVINO.
|
LLM Instruction-following pipeline with OpenVINO.
|
||||||
|
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 274-efficient-sam
|
:title: 274-efficient-sam
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/274-efficient-sam/274-efficient-sam.png
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/274-efficient-sam/274-efficient-sam.png
|
||||||
|
|
||||||
Object segmentations with EfficientSAM and OpenVINO.
|
Object segmentations with EfficientSAM and OpenVINO.
|
||||||
|
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 273-stable-zephyr-3b-chatbot
|
:title: 273-stable-zephyr-3b-chatbot
|
||||||
:img: _static/images/notebook_eye.png
|
:img: _static/images/notebook_eye.png
|
||||||
|
|
@ -104,25 +120,6 @@ Below you will find a selection of recommended tutorials that demonstrate infere
|
||||||
|
|
||||||
Paint by Example using Stable Diffusion and OpenVINO.
|
Paint by Example using Stable Diffusion and OpenVINO.
|
||||||
|
|
||||||
.. showcase::
|
|
||||||
:title: 271-sdxl-turbo
|
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/271-sdxl-turbo/271-sdxl-turbo.png
|
|
||||||
|
|
||||||
Single step image generation using SDXL-turbo and OpenVINO.
|
|
||||||
|
|
||||||
.. showcase::
|
|
||||||
:title: 270-sound-generation-audioldm2
|
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/270-sound-generation-audioldm2/270-sound-generation-audioldm2.png
|
|
||||||
|
|
||||||
Sound Generation with AudioLDM2 and OpenVINO.
|
|
||||||
|
|
||||||
.. showcase::
|
|
||||||
:title: 269-film-slowmo
|
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/269-film-slowmo/269-film-slowmo.gif
|
|
||||||
|
|
||||||
Frame interpolation using FILM and OpenVINO.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
If there are any issues while running the notebooks, refer to the **Troubleshooting** and **FAQ** sections in the :doc:`Installation Guide <notebooks_installation>` or start a GitHub
|
If there are any issues while running the notebooks, refer to the **Troubleshooting** and **FAQ** sections in the :doc:`Installation Guide <notebooks_installation>` or start a GitHub
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,12 @@ Convert & Optimize
|
||||||
|
|
||||||
Tutorials that explain how to optimize and quantize models with OpenVINO tools.
|
Tutorials that explain how to optimize and quantize models with OpenVINO tools.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 127-tensorflow-bit-image-classification-nncf-quantization
|
||||||
|
:img: _static/images/notebook_eye.png
|
||||||
|
|
||||||
|
BiT Image Classification OpenVINO IR model Quantization with NNCF.
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 126-tensorflow-hub
|
:title: 126-tensorflow-hub
|
||||||
:img: _static/images/notebook_eye.png
|
:img: _static/images/notebook_eye.png
|
||||||
|
|
@ -11,6 +11,31 @@ Model Demos
|
||||||
|
|
||||||
Demos that demonstrate inference on a particular model.
|
Demos that demonstrate inference on a particular model.
|
||||||
|
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 284-openvoice
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/284-openvoice/284-openvoice.png
|
||||||
|
|
||||||
|
Voice tone cloning with OpenVoice and OpenVINO.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 283-photo-maker
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/283-photo-maker/283-photo-maker.gif
|
||||||
|
|
||||||
|
Text-to-image generation using PhotoMaker and OpenVINO.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 282-siglip-zero-shot-image-classification
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/228-clip-zero-shot-image-classification/228-clip-zero-shot-convert.png
|
||||||
|
|
||||||
|
Zero-shot Image Classification with SigLIP.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 281-kosmos2-multimodal-large-language-model
|
||||||
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/281-kosmos2-multimodal-large-language-model/281-kosmos2-multimodal-large-language-model.png
|
||||||
|
|
||||||
|
Kosmos-2: Multimodal Large Language Model and OpenVINO.
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 280-depth-anything
|
:title: 280-depth-anything
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
||||||
|
|
@ -441,11 +466,6 @@ Demos that demonstrate inference on a particular model.
|
||||||
|
|
||||||
Use pre-trained models to detect and recognize vehicles and their attributes with OpenVINO.
|
Use pre-trained models to detect and recognize vehicles and their attributes with OpenVINO.
|
||||||
|
|
||||||
.. showcase::
|
|
||||||
:title: 217-vision-deblur
|
|
||||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/217-vision-deblur/217-vision-deblur.png
|
|
||||||
|
|
||||||
Deblur images with DeblurGAN-v2.
|
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 216-attention-center
|
:title: 216-attention-center
|
||||||
|
|
@ -23,6 +23,12 @@ Tutorials that include code to train neural networks.
|
||||||
|
|
||||||
Use Neural Network Compression Framework (NNCF) to quantize PyTorch model.
|
Use Neural Network Compression Framework (NNCF) to quantize PyTorch model.
|
||||||
|
|
||||||
|
.. showcase::
|
||||||
|
:title: 301-tensorflow-training-openvino
|
||||||
|
:img: _static/images/notebook_eye.png
|
||||||
|
|
||||||
|
From Training to Deployment with TensorFlow and OpenVINO™.
|
||||||
|
|
||||||
.. showcase::
|
.. showcase::
|
||||||
:title: 301-tensorflow-training-openvino-nncf
|
:title: 301-tensorflow-training-openvino-nncf
|
||||||
:img: _static/images/notebook_eye.png
|
:img: _static/images/notebook_eye.png
|
||||||
|
|
@ -5,7 +5,7 @@ repo_directory = "notebooks"
|
||||||
repo_owner = "openvinotoolkit"
|
repo_owner = "openvinotoolkit"
|
||||||
repo_name = "openvino_notebooks"
|
repo_name = "openvino_notebooks"
|
||||||
repo_branch = "tree/main"
|
repo_branch = "tree/main"
|
||||||
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/"
|
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240209220807/dist/rst_files/"
|
||||||
blacklisted_extensions = ['.xml', '.bin']
|
blacklisted_extensions = ['.xml', '.bin']
|
||||||
notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/main/"
|
notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/main/"
|
||||||
notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath="
|
notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath="
|
||||||
|
|
|
||||||
|
|
@ -68,11 +68,11 @@ def fetch_colab_list(colab_list_file) -> list:
|
||||||
|
|
||||||
|
|
||||||
def add_glob_directive():
|
def add_glob_directive():
|
||||||
"""This function modifies toctrees of the five node articles in tutorials
|
"""This function modifies toctrees of the five node articles in tutorials
|
||||||
section. It adds the notebooks found in docs/notebooks directory to the menu.
|
section. It adds the notebooks found in docs/notebooks directory to the menu.
|
||||||
"""
|
"""
|
||||||
tutorials_path = Path('../../docs/articles_en/learn_openvino/tutorials').resolve(strict=True)
|
tutorials_path = Path('../../docs/articles_en/learn-openvino/interactive-tutorials-python').resolve(strict=True)
|
||||||
tutorials_files = [x for x in os.listdir(tutorials_path) if re.match("notebooks_section_[0-9]{1}\.", x)]
|
tutorials_files = [x for x in os.listdir(tutorials_path) if re.match("notebooks-section-[0-9]{1}.*?\.rst", x)]
|
||||||
for tutorials_file in tutorials_files:
|
for tutorials_file in tutorials_files:
|
||||||
file_name = os.path.join(tutorials_path, tutorials_file)
|
file_name = os.path.join(tutorials_path, tutorials_file)
|
||||||
with open(file_name, 'r+', encoding='cp437') as section_file:
|
with open(file_name, 'r+', encoding='cp437') as section_file:
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ This basic introduction to OpenVINO™ shows how to do inference with an
|
||||||
image classification model.
|
image classification model.
|
||||||
|
|
||||||
A pre-trained `MobileNetV3
|
A pre-trained `MobileNetV3
|
||||||
model <https://docs.openvino.ai/2023.0/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
model <https://docs.openvino.ai/2023.3/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||||
from `Open Model
|
from `Open Model
|
||||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used in
|
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used in
|
||||||
this tutorial. For more information about how OpenVINO IR models are
|
this tutorial. For more information about how OpenVINO IR models are
|
||||||
|
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/001-hello-world-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/001-hello-world-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="001-hello-world-with-output_11_1.png">001-hello-world-with-output_11_1.png</a> 26-Jan-2024 01:05 387941
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -61,47 +61,43 @@ Table of contents:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.31.0)
|
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.31.0)
|
||||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.1)
|
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.1)
|
||||||
Requirement already satisfied: ipywidgets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (8.1.1)
|
Requirement already satisfied: ipywidgets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (8.1.2)
|
||||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.3.2)
|
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.3.2)
|
||||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.6)
|
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.6)
|
||||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2.1.0)
|
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2.2.0)
|
||||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2023.11.17)
|
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2024.2.2)
|
||||||
|
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (0.2.1)
|
||||||
|
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (8.12.3)
|
||||||
|
Requirement already satisfied: traitlets>=4.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (5.14.1)
|
||||||
|
Requirement already satisfied: widgetsnbextension~=4.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (4.0.10)
|
||||||
|
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (3.0.10)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (0.2.1)
|
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.2.0)
|
||||||
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (8.12.3)
|
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (5.1.1)
|
||||||
Requirement already satisfied: traitlets>=4.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (5.14.1)
|
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.19.1)
|
||||||
Requirement already satisfied: widgetsnbextension~=4.0.9 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (4.0.9)
|
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.6)
|
||||||
Requirement already satisfied: jupyterlab-widgets~=3.0.9 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (3.0.9)
|
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.7.5)
|
||||||
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.2.0)
|
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.43)
|
||||||
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (5.1.1)
|
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (2.17.2)
|
||||||
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.19.1)
|
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.3)
|
||||||
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.6)
|
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||||
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.7.5)
|
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||||
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.43)
|
|
||||||
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (2.17.2)
|
|
||||||
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.3)
|
|
||||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.3)
|
||||||
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.3)
|
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)
|
||||||
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)
|
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets) (0.2.13)
|
||||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets) (0.2.13)
|
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.0.1)
|
||||||
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.0.1)
|
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.4.1)
|
||||||
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.4.1)
|
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (0.2.2)
|
||||||
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (0.2.2)
|
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets) (1.16.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets) (1.16.0)
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -218,7 +214,7 @@ notebooks.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -267,7 +263,7 @@ points to the filename of an ONNX model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.onnx')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.onnx')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -323,7 +319,7 @@ without any conversion step. Pass the filename with extension to
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/inference.pdiparams')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/inference.pdiparams')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -367,7 +363,7 @@ TensorFlow models saved in frozen graph format can also be passed to
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.pb')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.pb')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -419,7 +415,7 @@ It is pre-trained model optimized to work with TensorFlow Lite.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.tflite')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.tflite')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -504,7 +500,7 @@ Information about the inputs and outputs of the model are in
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -714,7 +710,7 @@ produced data as values.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -903,7 +899,7 @@ input shape.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.bin')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.bin')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1066,7 +1062,7 @@ the cache.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ Hello Image Segmentation
|
||||||
A very basic introduction to using segmentation models with OpenVINO™.
|
A very basic introduction to using segmentation models with OpenVINO™.
|
||||||
|
|
||||||
In this tutorial, a pre-trained
|
In this tutorial, a pre-trained
|
||||||
`road-segmentation-adas-0001 <https://docs.openvino.ai/2023.0/omz_models_model_road_segmentation_adas_0001.html>`__
|
`road-segmentation-adas-0001 <https://docs.openvino.ai/2023.3/omz_models_model_road_segmentation_adas_0001.html>`__
|
||||||
model from the `Open Model
|
model from the `Open Model
|
||||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used.
|
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used.
|
||||||
ADAS stands for Advanced Driver Assistance Services. The model
|
ADAS stands for Advanced Driver Assistance Services. The model
|
||||||
|
|
@ -182,7 +182,7 @@ is provided.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7ffab9d92970>
|
<matplotlib.image.AxesImage at 0x7f11c8142580>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -209,7 +209,7 @@ Do Inference
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7ffa747040d0>
|
<matplotlib.image.AxesImage at 0x7f11c8051eb0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/003-hello-segmentation-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/003-hello-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="003-hello-segmentation-with-output_11_2.png">003-hello-segmentation-with-output_11_2.png</a> 26-Jan-2024 01:05 249032
|
|
||||||
<a href="003-hello-segmentation-with-output_13_1.png">003-hello-segmentation-with-output_13_1.png</a> 26-Jan-2024 01:05 20550
|
|
||||||
<a href="003-hello-segmentation-with-output_17_0.png">003-hello-segmentation-with-output_17_0.png</a> 26-Jan-2024 01:05 260045
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -5,7 +5,7 @@ A very basic introduction to using object detection models with
|
||||||
OpenVINO™.
|
OpenVINO™.
|
||||||
|
|
||||||
The
|
The
|
||||||
`horizontal-text-detection-0001 <https://docs.openvino.ai/2023.0/omz_models_model_horizontal_text_detection_0001.html>`__
|
`horizontal-text-detection-0001 <https://docs.openvino.ai/2023.3/omz_models_model_horizontal_text_detection_0001.html>`__
|
||||||
model from `Open Model
|
model from `Open Model
|
||||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used. It
|
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used. It
|
||||||
detects horizontal text in images and returns a blob of data in the
|
detects horizontal text in images and returns a blob of data in the
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/004-hello-detection-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/004-hello-detection-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="004-hello-detection-with-output_11_1.png">004-hello-detection-with-output_11_1.png</a> 26-Jan-2024 01:05 305482
|
|
||||||
<a href="004-hello-detection-with-output_16_0.png">004-hello-detection-with-output_16_0.png</a> 26-Jan-2024 01:05 457214
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -2,7 +2,7 @@ Convert a TensorFlow Model to OpenVINO™
|
||||||
=======================================
|
=======================================
|
||||||
|
|
||||||
This short tutorial shows how to convert a TensorFlow
|
This short tutorial shows how to convert a TensorFlow
|
||||||
`MobileNetV3 <https://docs.openvino.ai/2023.0/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
`MobileNetV3 <https://docs.openvino.ai/2023.3/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||||
image classification model to OpenVINO `Intermediate
|
image classification model to OpenVINO `Intermediate
|
||||||
Representation <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_IR_and_opsets.html>`__
|
Representation <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_IR_and_opsets.html>`__
|
||||||
(OpenVINO IR) format, using `Model Conversion
|
(OpenVINO IR) format, using `Model Conversion
|
||||||
|
|
@ -75,14 +75,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:33:56.723840: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:34:07.759850: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:33:56.757735: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:34:07.794264: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:33:57.273134: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:34:08.310440: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
Settings
|
Settings
|
||||||
|
|
@ -122,12 +122,12 @@ and save it to the disk.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:34:00.136277: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
2024-02-09 22:34:11.190073: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||||
2024-01-25 22:34:00.136313: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
2024-02-09 22:34:11.190106: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||||
2024-01-25 22:34:00.136317: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
2024-02-09 22:34:11.190111: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||||
2024-01-25 22:34:00.136451: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
2024-02-09 22:34:11.190249: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||||
2024-01-25 22:34:00.136466: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
2024-02-09 22:34:11.190264: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||||
2024-01-25 22:34:00.136470: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
2024-02-09 22:34:11.190268: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -137,13 +137,13 @@ and save it to the disk.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:34:04.279915: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
2024-02-09 22:34:15.411337: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||||
[[{{node inputs}}]]
|
[[{{node inputs}}]]
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:34:07.400979: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
2024-02-09 22:34:18.568762: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||||
[[{{node inputs}}]]
|
[[{{node inputs}}]]
|
||||||
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 54). These functions will not be directly callable after loading.
|
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 54). These functions will not be directly callable after loading.
|
||||||
|
|
||||||
|
|
@ -362,5 +362,5 @@ performance.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
IR model in OpenVINO Runtime/CPU: 0.0011 seconds per image, FPS: 933.95
|
IR model in OpenVINO Runtime/CPU: 0.0011 seconds per image, FPS: 926.34
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/101-tensorflow-classification-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/101-tensorflow-classification-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="101-tensorflow-classification-to-openvino-with-output_19_1.png">101-tensorflow-classification-to-openvino-with-..></a> 26-Jan-2024 01:05 387941
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -576,17 +576,17 @@ performance.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU: 0.042 seconds per image, FPS: 24.02
|
PyTorch model on CPU: 0.040 seconds per image, FPS: 24.69
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
ONNX model in OpenVINO Runtime/CPU: 0.030 seconds per image, FPS: 33.66
|
ONNX model in OpenVINO Runtime/CPU: 0.018 seconds per image, FPS: 56.48
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.029 seconds per image, FPS: 35.01
|
OpenVINO IR model in OpenVINO Runtime/CPU: 0.018 seconds per image, FPS: 55.11
|
||||||
|
|
||||||
|
|
||||||
**Show Device Information**
|
**Show Device Information**
|
||||||
|
|
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-onnx-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-onnx-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="102-pytorch-onnx-to-openvino-with-output_22_0.png">102-pytorch-onnx-to-openvino-with-output_22_0.png</a> 26-Jan-2024 01:05 465692
|
|
||||||
<a href="102-pytorch-onnx-to-openvino-with-output_27_0.png">102-pytorch-onnx-to-openvino-with-output_27_0.png</a> 26-Jan-2024 01:05 465695
|
|
||||||
<a href="102-pytorch-onnx-to-openvino-with-output_29_0.png">102-pytorch-onnx-to-openvino-with-output_29_0.png</a> 26-Jan-2024 01:05 465692
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -248,7 +248,7 @@ Benchmark PyTorch Model Inference
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
16.4 ms ± 673 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
17.6 ms ± 52.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Convert PyTorch Model to OpenVINO Intermediate Representation
|
Convert PyTorch Model to OpenVINO Intermediate Representation
|
||||||
|
|
@ -407,7 +407,7 @@ Benchmark OpenVINO Model Inference
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
3.31 ms ± 28.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
3.42 ms ± 7.33 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Convert PyTorch Model with Static Input Shape
|
Convert PyTorch Model with Static Input Shape
|
||||||
|
|
@ -537,7 +537,7 @@ Benchmark OpenVINO Model Inference with Static Input Shape
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2.89 ms ± 38.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
2.9 ms ± 17.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Convert TorchScript Model to OpenVINO Intermediate Representation
|
Convert TorchScript Model to OpenVINO Intermediate Representation
|
||||||
|
|
@ -632,7 +632,7 @@ Benchmark Scripted Model Inference
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
12.8 ms ± 6.97 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
13 ms ± 50.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Convert PyTorch Scripted Model to OpenVINO Intermediate Representation
|
Convert PyTorch Scripted Model to OpenVINO Intermediate Representation
|
||||||
|
|
@ -691,7 +691,7 @@ Benchmark OpenVINO Model Inference Converted From Scripted Model
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
3.41 ms ± 6.84 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
3.42 ms ± 6.53 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Traced Model
|
Traced Model
|
||||||
|
|
@ -767,7 +767,7 @@ Benchmark Traced Model Inference
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
12.2 ms ± 29 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
13.4 ms ± 4.67 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
||||||
Convert PyTorch Traced Model to OpenVINO Intermediate Representation
|
Convert PyTorch Traced Model to OpenVINO Intermediate Representation
|
||||||
|
|
@ -826,5 +826,5 @@ Benchmark OpenVINO Model Inference Converted From Traced Model
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
3.4 ms ± 3.43 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
3.47 ms ± 10.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,20 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_11_0.jpg">102-pytorch-to-openvino-with-output_11_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_11_0.png">102-pytorch-to-openvino-with-output_11_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_20_0.jpg">102-pytorch-to-openvino-with-output_20_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_20_0.png">102-pytorch-to-openvino-with-output_20_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_31_0.jpg">102-pytorch-to-openvino-with-output_31_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_31_0.png">102-pytorch-to-openvino-with-output_31_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_35_0.jpg">102-pytorch-to-openvino-with-output_35_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_35_0.png">102-pytorch-to-openvino-with-output_35_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_39_0.jpg">102-pytorch-to-openvino-with-output_39_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_39_0.png">102-pytorch-to-openvino-with-output_39_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_43_0.jpg">102-pytorch-to-openvino-with-output_43_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_43_0.png">102-pytorch-to-openvino-with-output_43_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_47_0.jpg">102-pytorch-to-openvino-with-output_47_0.jpg</a> 26-Jan-2024 01:05 54874
|
|
||||||
<a href="102-pytorch-to-openvino-with-output_47_0.png">102-pytorch-to-openvino-with-output_47_0.png</a> 26-Jan-2024 01:05 542516
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -94,7 +94,7 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
--2024-01-25 22:35:57-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
--2024-02-09 22:36:08-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
||||||
Resolving proxy-mu.intel.com (proxy-mu.intel.com)... 10.217.247.236
|
Resolving proxy-mu.intel.com (proxy-mu.intel.com)... 10.217.247.236
|
||||||
Connecting to proxy-mu.intel.com (proxy-mu.intel.com)|10.217.247.236|:911... connected.
|
Connecting to proxy-mu.intel.com (proxy-mu.intel.com)|10.217.247.236|:911... connected.
|
||||||
Proxy request sent, awaiting response...
|
Proxy request sent, awaiting response...
|
||||||
|
|
@ -102,7 +102,7 @@ Imports
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
404 Not Found
|
404 Not Found
|
||||||
2024-01-25 22:35:57 ERROR 404: Not Found.
|
2024-02-09 22:36:08 ERROR 404: Not Found.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -135,12 +135,12 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:35:59 INFO: Loading faiss with AVX2 support.
|
2024-02-09 22:36:10 INFO: Loading faiss with AVX2 support.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:35:59 INFO: Successfully loaded faiss with AVX2 support.
|
2024-02-09 22:36:10 INFO: Successfully loaded faiss with AVX2 support.
|
||||||
|
|
||||||
|
|
||||||
Settings
|
Settings
|
||||||
|
|
@ -224,7 +224,7 @@ inference on that image, and then show the top three prediction results.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[2024/01/25 22:36:20] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
[2024/02/09 22:36:38] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -294,7 +294,7 @@ clipping values.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:36:20 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
2024-02-09 22:36:39 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -306,7 +306,7 @@ clipping values.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7f72946bbac0>
|
<matplotlib.image.AxesImage at 0x7f9b4c389670>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -489,7 +489,7 @@ Note that many optimizations are possible to improve the performance.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PaddlePaddle model on CPU: 0.0073 seconds per image, FPS: 137.36
|
PaddlePaddle model on CPU: 0.0075 seconds per image, FPS: 133.16
|
||||||
|
|
||||||
PaddlePaddle result:
|
PaddlePaddle result:
|
||||||
Labrador retriever, 0.75138
|
Labrador retriever, 0.75138
|
||||||
|
|
@ -553,7 +553,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0031 seconds per image, FPS: 322.19
|
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0030 seconds per image, FPS: 328.87
|
||||||
|
|
||||||
OpenVINO result:
|
OpenVINO result:
|
||||||
Labrador retriever, 0.74909
|
Labrador retriever, 0.74909
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/103-paddle-to-openvino-classification-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/103-paddle-to-openvino-classification-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="103-paddle-to-openvino-classification-with-output_15_3.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 120883
|
|
||||||
<a href="103-paddle-to-openvino-classification-with-output_23_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
|
||||||
<a href="103-paddle-to-openvino-classification-with-output_27_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
|
||||||
<a href="103-paddle-to-openvino-classification-with-output_30_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
|
||||||
<a href="103-paddle-to-openvino-classification-with-output_8_2.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -43,7 +43,7 @@ Table of contents:
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
%pip install -q "nncf>=2.5.0"
|
%pip install -q "nncf>=2.5.0"
|
||||||
%pip install -q "transformers" datasets evaluate --extra-index-url https://download.pytorch.org/whl/cpu
|
%pip install -q transformers datasets evaluate --extra-index-url https://download.pytorch.org/whl/cpu
|
||||||
%pip install -q "openvino>=2023.1.0"
|
%pip install -q "openvino>=2023.1.0"
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -96,14 +96,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:37:51.906403: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:38:10.763464: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:37:51.939998: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:38:10.797722: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:37:52.576201: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:38:11.441310: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -191,7 +191,7 @@ PyTorch model formats are supported:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||||
return self.fget.__get__(instance, owner)()
|
return self.fget.__get__(instance, owner)()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -286,13 +286,15 @@ The optimization process contains the following steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -303,12 +305,15 @@ The optimization process contains the following steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -329,13 +334,15 @@ The optimization process contains the following steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -346,13 +353,15 @@ The optimization process contains the following steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -548,17 +557,17 @@ Frames Per Second (FPS) for images.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU: 0.074 seconds per sentence, SPS: 13.56
|
PyTorch model on CPU: 0.073 seconds per sentence, SPS: 13.77
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 48.16
|
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 47.89
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 109.03
|
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 109.72
|
||||||
|
|
||||||
|
|
||||||
Finally, measure the inference performance of OpenVINO ``FP32`` and
|
Finally, measure the inference performance of OpenVINO ``FP32`` and
|
||||||
|
|
@ -566,7 +575,7 @@ Finally, measure the inference performance of OpenVINO ``FP32`` and
|
||||||
Tool <https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html>`__
|
Tool <https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html>`__
|
||||||
in OpenVINO.
|
in OpenVINO.
|
||||||
|
|
||||||
**Note**: The ``benchmark_app`` tool is able to measure the
|
**NOTE**: The ``benchmark_app`` tool is able to measure the
|
||||||
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
||||||
models only. For more accurate performance, run ``benchmark_app`` in
|
models only. For more accurate performance, run ``benchmark_app`` in
|
||||||
a terminal/command prompt after closing other applications. Run
|
a terminal/command prompt after closing other applications. Run
|
||||||
|
|
@ -599,7 +608,7 @@ in OpenVINO.
|
||||||
Device with "device" name is not registered in the OpenVINO Runtime
|
Device with "device" name is not registered in the OpenVINO Runtime
|
||||||
|
|
||||||
Traceback (most recent call last):
|
Traceback (most recent call last):
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||||
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
||||||
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
||||||
Exception from src/inference/src/dev/core_impl.cpp:560:
|
Exception from src/inference/src/dev/core_impl.cpp:560:
|
||||||
|
|
@ -632,7 +641,7 @@ in OpenVINO.
|
||||||
Device with "device" name is not registered in the OpenVINO Runtime
|
Device with "device" name is not registered in the OpenVINO Runtime
|
||||||
|
|
||||||
Traceback (most recent call last):
|
Traceback (most recent call last):
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||||
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
||||||
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
||||||
Exception from src/inference/src/dev/core_impl.cpp:560:
|
Exception from src/inference/src/dev/core_impl.cpp:560:
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ device <https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_AU
|
||||||
(or AUTO in short) selects the most suitable device for inference by
|
(or AUTO in short) selects the most suitable device for inference by
|
||||||
considering the model precision, power efficiency and processing
|
considering the model precision, power efficiency and processing
|
||||||
capability of the available `compute
|
capability of the available `compute
|
||||||
devices <https://docs.openvino.ai/2023.3/openvino_docs_Runtime_Inference_Modes_Overview.html>`__.
|
devices <https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_Supported_Devices.html>`__.
|
||||||
The model precision (such as ``FP32``, ``FP16``, ``INT8``, etc.) is the
|
The model precision (such as ``FP32``, ``FP16``, ``INT8``, etc.) is the
|
||||||
first consideration to filter out the devices that cannot run the
|
first consideration to filter out the devices that cannot run the
|
||||||
network efficiently.
|
network efficiently.
|
||||||
|
|
@ -81,13 +81,13 @@ Import modules and create Core
|
||||||
|
|
||||||
import time
|
import time
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
|
|
||||||
from IPython.display import Markdown, display
|
from IPython.display import Markdown, display
|
||||||
|
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
|
|
||||||
if "GPU" not in core.available_devices:
|
if "GPU" not in core.available_devices:
|
||||||
display(Markdown('<div class="alert alert-block alert-danger"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>'))
|
display(Markdown('<div class="alert alert-block alert-danger"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>'))
|
||||||
|
|
||||||
|
|
@ -127,11 +127,11 @@ For more information about model conversion API, see this
|
||||||
|
|
||||||
import torchvision
|
import torchvision
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
base_model_dir = Path("./model")
|
base_model_dir = Path("./model")
|
||||||
base_model_dir.mkdir(exist_ok=True)
|
base_model_dir.mkdir(exist_ok=True)
|
||||||
model_path = base_model_dir / "resnet50.xml"
|
model_path = base_model_dir / "resnet50.xml"
|
||||||
|
|
||||||
if not model_path.exists():
|
if not model_path.exists():
|
||||||
pt_model = torchvision.models.resnet50(weights="DEFAULT")
|
pt_model = torchvision.models.resnet50(weights="DEFAULT")
|
||||||
ov_model = ov.convert_model(pt_model, input=[[1,3,224,224]])
|
ov_model = ov.convert_model(pt_model, input=[[1,3,224,224]])
|
||||||
|
|
@ -164,24 +164,24 @@ By default, ``compile_model`` API will select **AUTO** as
|
||||||
|
|
||||||
# Set LOG_LEVEL to LOG_INFO.
|
# Set LOG_LEVEL to LOG_INFO.
|
||||||
core.set_property("AUTO", {"LOG_LEVEL":"LOG_INFO"})
|
core.set_property("AUTO", {"LOG_LEVEL":"LOG_INFO"})
|
||||||
|
|
||||||
# Load the model onto the target device.
|
# Load the model onto the target device.
|
||||||
compiled_model = core.compile_model(ov_model)
|
compiled_model = core.compile_model(ov_model)
|
||||||
|
|
||||||
if isinstance(compiled_model, ov.CompiledModel):
|
if isinstance(compiled_model, ov.CompiledModel):
|
||||||
print("Successfully compiled model without a device_name.")
|
print("Successfully compiled model without a device_name.")
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[22:41:12.8903]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
||||||
[22:41:12.8903]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
||||||
[22:41:12.8904]I[plugin.cpp:536][AUTO] device:CPU, config:PERF_COUNT=NO
|
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERF_COUNT=NO
|
||||||
[22:41:12.8904]I[plugin.cpp:541][AUTO] device:CPU, priority:0
|
[22:41:31.9445]I[plugin.cpp:541][AUTO] device:CPU, priority:0
|
||||||
[22:41:12.8904]I[schedule.cpp:17][AUTO] scheduler starting
|
[22:41:31.9446]I[schedule.cpp:17][AUTO] scheduler starting
|
||||||
[22:41:12.8904]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
[22:41:31.9446]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
||||||
[22:41:13.0500]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
[22:41:32.0858]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
||||||
[22:41:13.0502]I[plugin.cpp:569][AUTO] underlying hardware does not support hardware context
|
[22:41:32.0860]I[plugin.cpp:569][AUTO] underlying hardware does not support hardware context
|
||||||
Successfully compiled model without a device_name.
|
Successfully compiled model without a device_name.
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -195,7 +195,7 @@ By default, ``compile_model`` API will select **AUTO** as
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Deleted compiled_model
|
Deleted compiled_model
|
||||||
[22:41:13.0623]I[schedule.cpp:303][AUTO] scheduler ending
|
[22:41:32.0982]I[schedule.cpp:303][AUTO] scheduler ending
|
||||||
|
|
||||||
|
|
||||||
Explicitly pass AUTO as device_name to Core::compile_model API
|
Explicitly pass AUTO as device_name to Core::compile_model API
|
||||||
|
|
@ -210,9 +210,9 @@ improve readability of your code.
|
||||||
|
|
||||||
# Set LOG_LEVEL to LOG_NONE.
|
# Set LOG_LEVEL to LOG_NONE.
|
||||||
core.set_property("AUTO", {"LOG_LEVEL":"LOG_NONE"})
|
core.set_property("AUTO", {"LOG_LEVEL":"LOG_NONE"})
|
||||||
|
|
||||||
compiled_model = core.compile_model(model=ov_model, device_name="AUTO")
|
compiled_model = core.compile_model(model=ov_model, device_name="AUTO")
|
||||||
|
|
||||||
if isinstance(compiled_model, ov.CompiledModel):
|
if isinstance(compiled_model, ov.CompiledModel):
|
||||||
print("Successfully compiled model using AUTO.")
|
print("Successfully compiled model using AUTO.")
|
||||||
|
|
||||||
|
|
@ -271,16 +271,16 @@ function, we will reuse it for preparing input data.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
|
|
||||||
# Download the image from the openvino_notebooks storage
|
# Download the image from the openvino_notebooks storage
|
||||||
image_filename = download_file(
|
image_filename = download_file(
|
||||||
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco.jpg",
|
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco.jpg",
|
||||||
directory="data"
|
directory="data"
|
||||||
)
|
)
|
||||||
|
|
||||||
image = Image.open(str(image_filename))
|
image = Image.open(str(image_filename))
|
||||||
input_transform = torchvision.models.ResNet50_Weights.DEFAULT.transforms()
|
input_transform = torchvision.models.ResNet50_Weights.DEFAULT.transforms()
|
||||||
|
|
||||||
input_tensor = input_transform(image)
|
input_tensor = input_transform(image)
|
||||||
input_tensor = input_tensor.unsqueeze(0).numpy()
|
input_tensor = input_tensor.unsqueeze(0).numpy()
|
||||||
image
|
image
|
||||||
|
|
@ -307,14 +307,14 @@ Load the model to GPU device and perform inference
|
||||||
|
|
||||||
if "GPU" not in core.available_devices:
|
if "GPU" not in core.available_devices:
|
||||||
print(f"A GPU device is not available. Available devices are: {core.available_devices}")
|
print(f"A GPU device is not available. Available devices are: {core.available_devices}")
|
||||||
else :
|
else :
|
||||||
# Start time.
|
# Start time.
|
||||||
gpu_load_start_time = time.perf_counter()
|
gpu_load_start_time = time.perf_counter()
|
||||||
compiled_model = core.compile_model(model=ov_model, device_name="GPU") # load to GPU
|
compiled_model = core.compile_model(model=ov_model, device_name="GPU") # load to GPU
|
||||||
|
|
||||||
# Execute the first inference.
|
# Execute the first inference.
|
||||||
results = compiled_model(input_tensor)[0]
|
results = compiled_model(input_tensor)[0]
|
||||||
|
|
||||||
# Measure time to the first inference.
|
# Measure time to the first inference.
|
||||||
gpu_fil_end_time = time.perf_counter()
|
gpu_fil_end_time = time.perf_counter()
|
||||||
gpu_fil_span = gpu_fil_end_time - gpu_load_start_time
|
gpu_fil_span = gpu_fil_end_time - gpu_load_start_time
|
||||||
|
|
@ -340,11 +340,11 @@ executed on CPU until GPU is ready.
|
||||||
# Start time.
|
# Start time.
|
||||||
auto_load_start_time = time.perf_counter()
|
auto_load_start_time = time.perf_counter()
|
||||||
compiled_model = core.compile_model(model=ov_model) # The device_name is AUTO by default.
|
compiled_model = core.compile_model(model=ov_model) # The device_name is AUTO by default.
|
||||||
|
|
||||||
# Execute the first inference.
|
# Execute the first inference.
|
||||||
results = compiled_model(input_tensor)[0]
|
results = compiled_model(input_tensor)[0]
|
||||||
|
|
||||||
|
|
||||||
# Measure time to the first inference.
|
# Measure time to the first inference.
|
||||||
auto_fil_end_time = time.perf_counter()
|
auto_fil_end_time = time.perf_counter()
|
||||||
auto_fil_span = auto_fil_end_time - auto_load_start_time
|
auto_fil_span = auto_fil_end_time - auto_load_start_time
|
||||||
|
|
@ -353,7 +353,7 @@ executed on CPU until GPU is ready.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Time to load model using AUTO device and get first inference: 0.15 seconds.
|
Time to load model using AUTO device and get first inference: 0.16 seconds.
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -406,11 +406,11 @@ Class and callback definition
|
||||||
"""
|
"""
|
||||||
self.fps = 0
|
self.fps = 0
|
||||||
self.latency = 0
|
self.latency = 0
|
||||||
|
|
||||||
self.start_time = time.perf_counter()
|
self.start_time = time.perf_counter()
|
||||||
self.latency_list = []
|
self.latency_list = []
|
||||||
self.interval = interval
|
self.interval = interval
|
||||||
|
|
||||||
def update(self, infer_request: ov.InferRequest) -> bool:
|
def update(self, infer_request: ov.InferRequest) -> bool:
|
||||||
"""
|
"""
|
||||||
Update the metrics if current ongoing @interval seconds duration is expired. Record the latency only if it is not expired.
|
Update the metrics if current ongoing @interval seconds duration is expired. Record the latency only if it is not expired.
|
||||||
|
|
@ -432,12 +432,12 @@ Class and callback definition
|
||||||
return True
|
return True
|
||||||
else :
|
else :
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
class InferContext:
|
class InferContext:
|
||||||
"""
|
"""
|
||||||
Inference context. Record and update peforamnce metrics via @metrics, set @feed_inference to False once @remaining_update_num <=0
|
Inference context. Record and update peforamnce metrics via @metrics, set @feed_inference to False once @remaining_update_num <=0
|
||||||
:member: metrics: instance of class PerformanceMetrics
|
:member: metrics: instance of class PerformanceMetrics
|
||||||
:member: remaining_update_num: the remaining times for peforamnce metrics updating.
|
:member: remaining_update_num: the remaining times for peforamnce metrics updating.
|
||||||
:member: feed_inference: if feed inference request is required or not.
|
:member: feed_inference: if feed inference request is required or not.
|
||||||
"""
|
"""
|
||||||
|
|
@ -452,7 +452,7 @@ Class and callback definition
|
||||||
self.metrics = PerformanceMetrics(update_interval)
|
self.metrics = PerformanceMetrics(update_interval)
|
||||||
self.remaining_update_num = num
|
self.remaining_update_num = num
|
||||||
self.feed_inference = True
|
self.feed_inference = True
|
||||||
|
|
||||||
def update(self, infer_request: ov.InferRequest):
|
def update(self, infer_request: ov.InferRequest):
|
||||||
"""
|
"""
|
||||||
Update the context. Set @feed_inference to False if the number of remaining performance metric updates (@remaining_update_num) reaches 0
|
Update the context. Set @feed_inference to False if the number of remaining performance metric updates (@remaining_update_num) reaches 0
|
||||||
|
|
@ -461,13 +461,13 @@ Class and callback definition
|
||||||
"""
|
"""
|
||||||
if self.remaining_update_num <= 0 :
|
if self.remaining_update_num <= 0 :
|
||||||
self.feed_inference = False
|
self.feed_inference = False
|
||||||
|
|
||||||
if self.metrics.update(infer_request) :
|
if self.metrics.update(infer_request) :
|
||||||
self.remaining_update_num = self.remaining_update_num - 1
|
self.remaining_update_num = self.remaining_update_num - 1
|
||||||
if self.remaining_update_num <= 0 :
|
if self.remaining_update_num <= 0 :
|
||||||
self.feed_inference = False
|
self.feed_inference = False
|
||||||
|
|
||||||
|
|
||||||
def completion_callback(infer_request: ov.InferRequest, context) -> None:
|
def completion_callback(infer_request: ov.InferRequest, context) -> None:
|
||||||
"""
|
"""
|
||||||
callback for the inference request, pass the @infer_request to @context for updating
|
callback for the inference request, pass the @infer_request to @context for updating
|
||||||
|
|
@ -476,8 +476,8 @@ Class and callback definition
|
||||||
:returns: None
|
:returns: None
|
||||||
"""
|
"""
|
||||||
context.update(infer_request)
|
context.update(infer_request)
|
||||||
|
|
||||||
|
|
||||||
# Performance metrics update interval (seconds) and number of times.
|
# Performance metrics update interval (seconds) and number of times.
|
||||||
metrics_update_interval = 10
|
metrics_update_interval = 10
|
||||||
metrics_update_num = 6
|
metrics_update_num = 6
|
||||||
|
|
@ -493,29 +493,29 @@ Loop for inference and update the FPS/Latency every
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
THROUGHPUT_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
THROUGHPUT_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
||||||
|
|
||||||
print("Compiling Model for AUTO device with THROUGHPUT hint")
|
print("Compiling Model for AUTO device with THROUGHPUT hint")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"THROUGHPUT"})
|
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"THROUGHPUT"})
|
||||||
|
|
||||||
infer_queue = ov.AsyncInferQueue(compiled_model, 0) # Setting to 0 will query optimal number by default.
|
infer_queue = ov.AsyncInferQueue(compiled_model, 0) # Setting to 0 will query optimal number by default.
|
||||||
infer_queue.set_callback(completion_callback)
|
infer_queue.set_callback(completion_callback)
|
||||||
|
|
||||||
print(f"Start inference, {metrics_update_num: .0f} groups of FPS/latency will be measured over {metrics_update_interval: .0f}s intervals")
|
print(f"Start inference, {metrics_update_num: .0f} groups of FPS/latency will be measured over {metrics_update_interval: .0f}s intervals")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
while THROUGHPUT_hint_context.feed_inference:
|
while THROUGHPUT_hint_context.feed_inference:
|
||||||
infer_queue.start_async(input_tensor, THROUGHPUT_hint_context)
|
infer_queue.start_async(input_tensor, THROUGHPUT_hint_context)
|
||||||
|
|
||||||
infer_queue.wait_all()
|
infer_queue.wait_all()
|
||||||
|
|
||||||
# Take the FPS and latency of the latest period.
|
# Take the FPS and latency of the latest period.
|
||||||
THROUGHPUT_hint_fps = THROUGHPUT_hint_context.metrics.fps
|
THROUGHPUT_hint_fps = THROUGHPUT_hint_context.metrics.fps
|
||||||
THROUGHPUT_hint_latency = THROUGHPUT_hint_context.metrics.latency
|
THROUGHPUT_hint_latency = THROUGHPUT_hint_context.metrics.latency
|
||||||
|
|
||||||
print("Done")
|
print("Done")
|
||||||
|
|
||||||
del compiled_model
|
del compiled_model
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -531,32 +531,32 @@ Loop for inference and update the FPS/Latency every
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 184.87fps, latency: 30.81ms, time interval: 10.01s
|
throughput: 179.69fps, latency: 31.58ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 185.21fps, latency: 31.62ms, time interval: 10.02s
|
throughput: 182.30fps, latency: 32.10ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 185.47fps, latency: 31.54ms, time interval: 10.00s
|
throughput: 180.62fps, latency: 32.36ms, time interval: 10.02s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 185.75fps, latency: 31.53ms, time interval: 10.01s
|
throughput: 179.76fps, latency: 32.61ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 185.09fps, latency: 31.64ms, time interval: 10.00s
|
throughput: 180.36fps, latency: 32.36ms, time interval: 10.02s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 184.42fps, latency: 31.76ms, time interval: 10.00s
|
throughput: 179.77fps, latency: 32.58ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -575,30 +575,30 @@ Loop for inference and update the FPS/Latency for each
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
LATENCY_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
LATENCY_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
||||||
|
|
||||||
print("Compiling Model for AUTO Device with LATENCY hint")
|
print("Compiling Model for AUTO Device with LATENCY hint")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"LATENCY"})
|
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"LATENCY"})
|
||||||
|
|
||||||
# Setting to 0 will query optimal number by default.
|
# Setting to 0 will query optimal number by default.
|
||||||
infer_queue = ov.AsyncInferQueue(compiled_model, 0)
|
infer_queue = ov.AsyncInferQueue(compiled_model, 0)
|
||||||
infer_queue.set_callback(completion_callback)
|
infer_queue.set_callback(completion_callback)
|
||||||
|
|
||||||
print(f"Start inference, {metrics_update_num: .0f} groups fps/latency will be out with {metrics_update_interval: .0f}s interval")
|
print(f"Start inference, {metrics_update_num: .0f} groups fps/latency will be out with {metrics_update_interval: .0f}s interval")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
while LATENCY_hint_context.feed_inference:
|
while LATENCY_hint_context.feed_inference:
|
||||||
infer_queue.start_async(input_tensor, LATENCY_hint_context)
|
infer_queue.start_async(input_tensor, LATENCY_hint_context)
|
||||||
|
|
||||||
infer_queue.wait_all()
|
infer_queue.wait_all()
|
||||||
|
|
||||||
# Take the FPS and latency of the latest period.
|
# Take the FPS and latency of the latest period.
|
||||||
LATENCY_hint_fps = LATENCY_hint_context.metrics.fps
|
LATENCY_hint_fps = LATENCY_hint_context.metrics.fps
|
||||||
LATENCY_hint_latency = LATENCY_hint_context.metrics.latency
|
LATENCY_hint_latency = LATENCY_hint_context.metrics.latency
|
||||||
|
|
||||||
print("Done")
|
print("Done")
|
||||||
|
|
||||||
del compiled_model
|
del compiled_model
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -614,32 +614,32 @@ Loop for inference and update the FPS/Latency for each
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 138.57fps, latency: 6.67ms, time interval: 10.00s
|
throughput: 139.27fps, latency: 6.65ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 141.04fps, latency: 6.66ms, time interval: 10.00s
|
throughput: 141.22fps, latency: 6.62ms, time interval: 10.01s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 140.74fps, latency: 6.66ms, time interval: 10.00s
|
throughput: 140.71fps, latency: 6.64ms, time interval: 10.01s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 141.60fps, latency: 6.68ms, time interval: 10.01s
|
throughput: 141.11fps, latency: 6.63ms, time interval: 10.01s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 141.73fps, latency: 6.68ms, time interval: 10.00s
|
throughput: 141.26fps, latency: 6.62ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
throughput: 141.45fps, latency: 6.67ms, time interval: 10.00s
|
throughput: 141.18fps, latency: 6.63ms, time interval: 10.00s
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -655,21 +655,21 @@ Difference in FPS and latency
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
TPUT = 0
|
TPUT = 0
|
||||||
LAT = 1
|
LAT = 1
|
||||||
labels = ["THROUGHPUT hint", "LATENCY hint"]
|
labels = ["THROUGHPUT hint", "LATENCY hint"]
|
||||||
|
|
||||||
fig1, ax1 = plt.subplots(1, 1)
|
fig1, ax1 = plt.subplots(1, 1)
|
||||||
fig1.patch.set_visible(False)
|
fig1.patch.set_visible(False)
|
||||||
ax1.axis('tight')
|
ax1.axis('tight')
|
||||||
ax1.axis('off')
|
ax1.axis('off')
|
||||||
|
|
||||||
cell_text = []
|
cell_text = []
|
||||||
cell_text.append(['%.2f%s' % (THROUGHPUT_hint_fps," FPS"), '%.2f%s' % (THROUGHPUT_hint_latency, " ms")])
|
cell_text.append(['%.2f%s' % (THROUGHPUT_hint_fps," FPS"), '%.2f%s' % (THROUGHPUT_hint_latency, " ms")])
|
||||||
cell_text.append(['%.2f%s' % (LATENCY_hint_fps," FPS"), '%.2f%s' % (LATENCY_hint_latency, " ms")])
|
cell_text.append(['%.2f%s' % (LATENCY_hint_fps," FPS"), '%.2f%s' % (LATENCY_hint_latency, " ms")])
|
||||||
|
|
||||||
table = ax1.table(cellText=cell_text, colLabels=["FPS (Higher is better)", "Latency (Lower is better)"], rowLabels=labels,
|
table = ax1.table(cellText=cell_text, colLabels=["FPS (Higher is better)", "Latency (Lower is better)"], rowLabels=labels,
|
||||||
rowColours=["deepskyblue"] * 2, colColours=["deepskyblue"] * 2,
|
rowColours=["deepskyblue"] * 2, colColours=["deepskyblue"] * 2,
|
||||||
cellLoc='center', loc='upper left')
|
cellLoc='center', loc='upper left')
|
||||||
table.auto_set_font_size(False)
|
table.auto_set_font_size(False)
|
||||||
|
|
@ -677,7 +677,7 @@ Difference in FPS and latency
|
||||||
table.auto_set_column_width(0)
|
table.auto_set_column_width(0)
|
||||||
table.auto_set_column_width(1)
|
table.auto_set_column_width(1)
|
||||||
table.scale(1, 3)
|
table.scale(1, 3)
|
||||||
|
|
||||||
fig1.tight_layout()
|
fig1.tight_layout()
|
||||||
plt.show()
|
plt.show()
|
||||||
|
|
||||||
|
|
@ -691,28 +691,28 @@ Difference in FPS and latency
|
||||||
# Output the difference.
|
# Output the difference.
|
||||||
width = 0.4
|
width = 0.4
|
||||||
fontsize = 14
|
fontsize = 14
|
||||||
|
|
||||||
plt.rc('font', size=fontsize)
|
plt.rc('font', size=fontsize)
|
||||||
fig, ax = plt.subplots(1,2, figsize=(10, 8))
|
fig, ax = plt.subplots(1,2, figsize=(10, 8))
|
||||||
|
|
||||||
rects1 = ax[0].bar([0], THROUGHPUT_hint_fps, width, label=labels[TPUT], color='#557f2d')
|
rects1 = ax[0].bar([0], THROUGHPUT_hint_fps, width, label=labels[TPUT], color='#557f2d')
|
||||||
rects2 = ax[0].bar([width], LATENCY_hint_fps, width, label=labels[LAT])
|
rects2 = ax[0].bar([width], LATENCY_hint_fps, width, label=labels[LAT])
|
||||||
ax[0].set_ylabel("frames per second")
|
ax[0].set_ylabel("frames per second")
|
||||||
ax[0].set_xticks([width / 2])
|
ax[0].set_xticks([width / 2])
|
||||||
ax[0].set_xticklabels(["FPS"])
|
ax[0].set_xticklabels(["FPS"])
|
||||||
ax[0].set_xlabel("Higher is better")
|
ax[0].set_xlabel("Higher is better")
|
||||||
|
|
||||||
rects1 = ax[1].bar([0], THROUGHPUT_hint_latency, width, label=labels[TPUT], color='#557f2d')
|
rects1 = ax[1].bar([0], THROUGHPUT_hint_latency, width, label=labels[TPUT], color='#557f2d')
|
||||||
rects2 = ax[1].bar([width], LATENCY_hint_latency, width, label=labels[LAT])
|
rects2 = ax[1].bar([width], LATENCY_hint_latency, width, label=labels[LAT])
|
||||||
ax[1].set_ylabel("milliseconds")
|
ax[1].set_ylabel("milliseconds")
|
||||||
ax[1].set_xticks([width / 2])
|
ax[1].set_xticks([width / 2])
|
||||||
ax[1].set_xticklabels(["Latency (ms)"])
|
ax[1].set_xticklabels(["Latency (ms)"])
|
||||||
ax[1].set_xlabel("Lower is better")
|
ax[1].set_xlabel("Lower is better")
|
||||||
|
|
||||||
fig.suptitle('Performance Hints')
|
fig.suptitle('Performance Hints')
|
||||||
fig.legend(labels, fontsize=fontsize)
|
fig.legend(labels, fontsize=fontsize)
|
||||||
fig.tight_layout()
|
fig.tight_layout()
|
||||||
|
|
||||||
plt.show()
|
plt.show()
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:1b8fb134deae365a3c5b358b032e2ad1e5ccaccae51cb176f250ba946461d8d6
|
oid sha256:acc35ef9a1d6c2acf74782263f65bd81b65d7fd6663a146ffb88c9a9b64e343d
|
||||||
size 25724
|
size 26542
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:d311a41fc8902df2f1ca223ffcdcf1ff634d160376b890b3a571326912b1d88a
|
oid sha256:ab4df6ece8c31c9c5d5ab7d51b7d628ef7fb5df7193d859b8cfd9547273b76e3
|
||||||
size 40017
|
size 39977
|
||||||
|
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/106-auto-device-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/106-auto-device-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="106-auto-device-with-output_14_1.jpg">106-auto-device-with-output_14_1.jpg</a> 26-Jan-2024 01:05 121563
|
|
||||||
<a href="106-auto-device-with-output_14_1.png">106-auto-device-with-output_14_1.png</a> 26-Jan-2024 01:05 869661
|
|
||||||
<a href="106-auto-device-with-output_27_0.png">106-auto-device-with-output_27_0.png</a> 26-Jan-2024 01:05 25724
|
|
||||||
<a href="106-auto-device-with-output_28_0.png">106-auto-device-with-output_28_0.png</a> 26-Jan-2024 01:05 40017
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -360,14 +360,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -378,14 +379,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -403,13 +405,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -420,14 +424,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -50,84 +50,88 @@ Table of contents:
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
|
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
|
||||||
Requirement already satisfied: datasets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.16.1)
|
Requirement already satisfied: datasets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.17.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Collecting torchmetrics>=0.11.0
|
Collecting torchmetrics>=0.11.0
|
||||||
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl.metadata (20 kB)
|
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl.metadata (20 kB)
|
||||||
Requirement already satisfied: torch>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.1.0+cpu)
|
Requirement already satisfied: torch>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.1.0+cpu)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.13.1)
|
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.13.1)
|
||||||
Requirement already satisfied: numpy>=1.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (1.23.5)
|
Requirement already satisfied: numpy>=1.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (1.23.5)
|
||||||
Requirement already satisfied: pyarrow>=8.0.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (15.0.0)
|
Requirement already satisfied: pyarrow>=12.0.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (15.0.0)
|
||||||
Requirement already satisfied: pyarrow-hotfix in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.6)
|
Requirement already satisfied: pyarrow-hotfix in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.6)
|
||||||
Requirement already satisfied: dill<0.3.8,>=0.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.3.7)
|
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.3.8)
|
||||||
Requirement already satisfied: pandas in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.0.3)
|
Requirement already satisfied: pandas in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.0.3)
|
||||||
Requirement already satisfied: requests>=2.19.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.31.0)
|
Requirement already satisfied: requests>=2.19.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.31.0)
|
||||||
Requirement already satisfied: tqdm>=4.62.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (4.66.1)
|
Requirement already satisfied: tqdm>=4.62.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (4.66.1)
|
||||||
Requirement already satisfied: xxhash in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.4.1)
|
Requirement already satisfied: xxhash in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.4.1)
|
||||||
Requirement already satisfied: multiprocess in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.70.15)
|
Requirement already satisfied: multiprocess in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.70.16)
|
||||||
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets) (2023.10.0)
|
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets) (2023.10.0)
|
||||||
|
Requirement already satisfied: aiohttp in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.9.3)
|
||||||
|
Requirement already satisfied: huggingface-hub>=0.19.4 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.20.3)
|
||||||
.. parsed-literal::
|
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (23.2)
|
||||||
|
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (6.0.1)
|
||||||
Requirement already satisfied: aiohttp in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.9.1)
|
|
||||||
Requirement already satisfied: huggingface-hub>=0.19.4 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.20.3)
|
|
||||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (23.2)
|
|
||||||
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (6.0.1)
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Collecting lightning-utilities>=0.8.0 (from torchmetrics>=0.11.0)
|
Collecting lightning-utilities>=0.8.0 (from torchmetrics>=0.11.0)
|
||||||
Using cached lightning_utilities-0.10.1-py3-none-any.whl.metadata (4.8 kB)
|
Using cached lightning_utilities-0.10.1-py3-none-any.whl.metadata (4.8 kB)
|
||||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torchmetrics>=0.11.0) (4.9.0)
|
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torchmetrics>=0.11.0) (4.9.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (1.12)
|
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (1.12)
|
||||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1)
|
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1)
|
||||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1.3)
|
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1.3)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: attrs>=17.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (23.2.0)
|
Requirement already satisfied: aiosignal>=1.1.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.1)
|
||||||
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (6.0.4)
|
Requirement already satisfied: attrs>=17.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (23.2.0)
|
||||||
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.9.4)
|
Requirement already satisfied: frozenlist>=1.1.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.4.1)
|
||||||
Requirement already satisfied: frozenlist>=1.1.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.4.1)
|
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (6.0.5)
|
||||||
Requirement already satisfied: aiosignal>=1.1.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.1)
|
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.9.4)
|
||||||
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (4.0.3)
|
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (4.0.3)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: setuptools in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from lightning-utilities>=0.8.0->torchmetrics>=0.11.0) (69.0.3)
|
Requirement already satisfied: setuptools in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from lightning-utilities>=0.8.0->torchmetrics>=0.11.0) (69.0.3)
|
||||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.3.2)
|
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.3.2)
|
||||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.6)
|
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.6)
|
||||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2.1.0)
|
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2.2.0)
|
||||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2023.11.17)
|
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2024.2.2)
|
||||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1.0) (2.1.4)
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2.8.2)
|
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1.0) (2.1.5)
|
||||||
Requirement already satisfied: pytz>=2020.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.3.post1)
|
|
||||||
Requirement already satisfied: tzdata>=2022.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.4)
|
|
||||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1.0) (1.3.0)
|
.. parsed-literal::
|
||||||
|
|
||||||
|
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2.8.2)
|
||||||
|
Requirement already satisfied: pytz>=2020.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2024.1)
|
||||||
|
Requirement already satisfied: tzdata>=2022.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.4)
|
||||||
|
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1.0) (1.3.0)
|
||||||
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)
|
|
||||||
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl (840 kB)
|
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl (840 kB)
|
||||||
Using cached lightning_utilities-0.10.1-py3-none-any.whl (24 kB)
|
Using cached lightning_utilities-0.10.1-py3-none-any.whl (24 kB)
|
||||||
|
|
||||||
|
|
@ -220,9 +224,9 @@ IR).
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:593: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:593: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:632: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:632: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -260,7 +264,7 @@ dataset.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1429: FutureWarning: The repository for patrickvonplaten/librispeech_asr_dummy contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/patrickvonplaten/librispeech_asr_dummy
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1454: FutureWarning: The repository for patrickvonplaten/librispeech_asr_dummy contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/patrickvonplaten/librispeech_asr_dummy
|
||||||
You can avoid this message in future by passing the argument `trust_remote_code=True`.
|
You can avoid this message in future by passing the argument `trust_remote_code=True`.
|
||||||
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
|
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
|
||||||
warnings.warn(
|
warnings.warn(
|
||||||
|
|
@ -334,14 +338,14 @@ steps:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:43:51.310601: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:44:10.617295: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:43:51.341489: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:44:10.648558: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:43:51.934215: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:44:11.243901: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -351,14 +355,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -369,14 +374,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -422,14 +428,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -440,14 +447,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -582,7 +590,7 @@ quantized model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:62: FutureWarning: Importing `WordErrorRate` from `torchmetrics` was deprecated and will be removed in 2.0. Import `WordErrorRate` from `torchmetrics.text` instead.
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:62: FutureWarning: Importing `WordErrorRate` from `torchmetrics` was deprecated and will be removed in 2.0. Import `WordErrorRate` from `torchmetrics.text` instead.
|
||||||
_future_warning(
|
_future_warning(
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -659,35 +667,35 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Read model took 54.55 ms
|
[ INFO ] Read model took 56.59 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
||||||
[ INFO ] Model outputs:
|
[ INFO ] Model outputs:
|
||||||
[ INFO ] logits , 1170 (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [?,?,32]
|
[ INFO ] 1170 , logits (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [?,?,32]
|
||||||
[Step 5/11] Resizing model to match image sizes and given batch
|
[Step 5/11] Resizing model to match image sizes and given batch
|
||||||
[ INFO ] Model batch size: 1
|
[ INFO ] Model batch size: 1
|
||||||
[ INFO ] Reshaping model: 'input_values': [1,30480]
|
[ INFO ] Reshaping model: 'input_values': [1,30480]
|
||||||
[ INFO ] Reshape model took 30.95 ms
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
[ INFO ] Reshape model took 30.54 ms
|
||||||
[Step 6/11] Configuring input of the model
|
[Step 6/11] Configuring input of the model
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
||||||
[ INFO ] Model outputs:
|
[ INFO ] Model outputs:
|
||||||
[ INFO ] logits , 1170 (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [1,95,32]
|
[ INFO ] 1170 , logits (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [1,95,32]
|
||||||
[Step 7/11] Loading the model to the device
|
[Step 7/11] Loading the model to the device
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 560.02 ms
|
[ INFO ] Compile model took 559.12 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: Model0
|
[ INFO ] NETWORK_NAME: Model0
|
||||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ INFO ] NUM_STREAMS: 6
|
[ INFO ] NUM_STREAMS: 6
|
||||||
[ INFO ] AFFINITY: Affinity.CORE
|
[ INFO ] AFFINITY: Affinity.CORE
|
||||||
[ INFO ] INFERENCE_NUM_THREADS: 24
|
[ INFO ] INFERENCE_NUM_THREADS: 24
|
||||||
|
|
@ -711,21 +719,21 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 58.98 ms
|
[ INFO ] First inference took 56.43 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 2766 iterations
|
[ INFO ] Count: 2778 iterations
|
||||||
[ INFO ] Duration: 60197.24 ms
|
[ INFO ] Duration: 60216.97 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 130.42 ms
|
[ INFO ] Median: 129.97 ms
|
||||||
[ INFO ] Average: 130.38 ms
|
[ INFO ] Average: 129.78 ms
|
||||||
[ INFO ] Min: 64.93 ms
|
[ INFO ] Min: 110.04 ms
|
||||||
[ INFO ] Max: 148.03 ms
|
[ INFO ] Max: 151.03 ms
|
||||||
[ INFO ] Throughput: 45.95 FPS
|
[ INFO ] Throughput: 46.13 FPS
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -759,7 +767,7 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Read model took 70.41 ms
|
[ INFO ] Read model took 67.98 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
||||||
|
|
@ -772,7 +780,7 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Reshape model took 37.13 ms
|
[ INFO ] Reshape model took 37.53 ms
|
||||||
[Step 6/11] Configuring input of the model
|
[Step 6/11] Configuring input of the model
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
||||||
|
|
@ -783,7 +791,7 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 999.07 ms
|
[ INFO ] Compile model took 1060.01 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: Model0
|
[ INFO ] NETWORK_NAME: Model0
|
||||||
|
|
@ -803,10 +811,6 @@ models.
|
||||||
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
||||||
[ INFO ] CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
|
[ INFO ] CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
|
||||||
[Step 9/11] Creating infer requests and preparing input tensors
|
[Step 9/11] Creating infer requests and preparing input tensors
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ WARNING ] No input files were given for input 'input_values'!. This input will be filled with random values!
|
[ WARNING ] No input files were given for input 'input_values'!. This input will be filled with random values!
|
||||||
[ INFO ] Fill input 'input_values' with random values
|
[ INFO ] Fill input 'input_values' with random values
|
||||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 60000 ms duration)
|
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 60000 ms duration)
|
||||||
|
|
@ -815,19 +819,19 @@ models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 56.74 ms
|
[ INFO ] First inference took 62.07 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 4284 iterations
|
[ INFO ] Count: 4296 iterations
|
||||||
[ INFO ] Duration: 60076.08 ms
|
[ INFO ] Duration: 60070.01 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 84.26 ms
|
[ INFO ] Median: 83.98 ms
|
||||||
[ INFO ] Average: 83.96 ms
|
[ INFO ] Average: 83.75 ms
|
||||||
[ INFO ] Min: 73.47 ms
|
[ INFO ] Min: 45.06 ms
|
||||||
[ INFO ] Max: 111.27 ms
|
[ INFO ] Max: 106.14 ms
|
||||||
[ INFO ] Throughput: 71.31 FPS
|
[ INFO ] Throughput: 71.52 FPS
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -218,7 +218,7 @@ for that property.
|
||||||
indent = len(max(supported_properties, key=len))
|
indent = len(max(supported_properties, key=len))
|
||||||
|
|
||||||
for property_key in supported_properties:
|
for property_key in supported_properties:
|
||||||
if property_key not in ('SUPPORTED_PROPERTIES'):
|
if property_key not in ('SUPPORTED_METRICS', 'SUPPORTED_CONFIG_KEYS', 'SUPPORTED_PROPERTIES'):
|
||||||
try:
|
try:
|
||||||
property_val = core.get_property(device, property_key)
|
property_val = core.get_property(device, property_key)
|
||||||
except TypeError:
|
except TypeError:
|
||||||
|
|
@ -1337,7 +1337,7 @@ Process Results
|
||||||
# Draw info at the top left such as current fps, the devices and the performance hint being used
|
# Draw info at the top left such as current fps, the devices and the performance hint being used
|
||||||
cv2.putText(frame, f"fps {str(round(frame_fps[current_frame], 2))}", (5, 20), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
cv2.putText(frame, f"fps {str(round(frame_fps[current_frame], 2))}", (5, 20), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||||
cv2.putText(frame, f"device {device_name}", (5, 40), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
cv2.putText(frame, f"device {device_name}", (5, 40), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||||
cv2.putText(frame, f"hint {compiled_model.get_property('PERFORMANCE_HINT').name}", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
cv2.putText(frame, f"hint {compiled_model.get_property('PERFORMANCE_HINT')}", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||||
|
|
||||||
# prediction contains [image_id, label, conf, x_min, y_min, x_max, y_max] according to model
|
# prediction contains [image_id, label, conf, x_min, y_min, x_max, y_max] according to model
|
||||||
for prediction in np.squeeze(results[current_frame]):
|
for prediction in np.squeeze(results[current_frame]):
|
||||||
|
|
|
||||||
|
|
@ -27,7 +27,7 @@ The quantization and pre-post-processing API are not included here as
|
||||||
they change the precision (quantization) or processing graph
|
they change the precision (quantization) or processing graph
|
||||||
(prepostprocessor). You can find examples of how to apply them to
|
(prepostprocessor). You can find examples of how to apply them to
|
||||||
optimize performance on OpenVINO IR files in
|
optimize performance on OpenVINO IR files in
|
||||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__ and
|
`111-detection-quantization <111-detection-quantization-with-output.html>`__ and
|
||||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||||
|
|
||||||
|image0|
|
|image0|
|
||||||
|
|
@ -141,7 +141,7 @@ requirements of this particular object detection model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<DisplayHandle display_id=b3ed024cde96c857177f3da66878c56a>
|
<DisplayHandle display_id=c7531cff1487c41296f1ac25e2e96b93>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -202,62 +202,54 @@ PyTorch Hub and small enough to see the difference in performance.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
5%|▌ | 208k/3.87M [00:00<00:01, 2.11MB/s]
|
7%|▋ | 272k/3.87M [00:00<00:01, 2.23MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
15%|█▍ | 576k/3.87M [00:00<00:01, 3.05MB/s]
|
19%|█▉ | 752k/3.87M [00:00<00:00, 3.64MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
24%|██▍ | 952k/3.87M [00:00<00:00, 3.36MB/s]
|
29%|██▉ | 1.13M/3.87M [00:00<00:00, 3.87MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
33%|███▎ | 1.30M/3.87M [00:00<00:00, 3.51MB/s]
|
39%|███▉ | 1.52M/3.87M [00:00<00:00, 3.61MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
43%|████▎ | 1.67M/3.87M [00:00<00:00, 3.60MB/s]
|
49%|████▉ | 1.89M/3.87M [00:00<00:00, 3.68MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
52%|█████▏ | 2.02M/3.87M [00:00<00:00, 3.16MB/s]
|
61%|██████▏ | 2.38M/3.87M [00:00<00:00, 4.06MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
61%|██████ | 2.36M/3.87M [00:00<00:00, 3.26MB/s]
|
72%|███████▏ | 2.77M/3.87M [00:00<00:00, 3.82MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
70%|███████ | 2.72M/3.87M [00:00<00:00, 3.38MB/s]
|
83%|████████▎ | 3.23M/3.87M [00:00<00:00, 4.07MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
80%|███████▉ | 3.09M/3.87M [00:00<00:00, 3.49MB/s]
|
94%|█████████▍| 3.63M/3.87M [00:01<00:00, 3.85MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
89%|████████▉ | 3.46M/3.87M [00:01<00:00, 3.58MB/s]
|
100%|██████████| 3.87M/3.87M [00:01<00:00, 3.87MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
|
|
||||||
99%|█████████▉| 3.84M/3.87M [00:01<00:00, 3.67MB/s]
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
|
|
||||||
100%|██████████| 3.87M/3.87M [00:01<00:00, 3.45MB/s]
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
@ -473,12 +465,12 @@ optimizations applied. We will treat it as our baseline.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU. First inference time: 0.0268 seconds
|
PyTorch model on CPU. First inference time: 0.0280 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU: 0.0213 seconds per image (46.98 FPS)
|
PyTorch model on CPU: 0.0218 seconds per image (45.96 FPS)
|
||||||
|
|
||||||
|
|
||||||
ONNX model
|
ONNX model
|
||||||
|
|
@ -528,12 +520,12 @@ Representation (IR) to leverage the OpenVINO Runtime.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
ONNX model on CPU. First inference time: 0.0186 seconds
|
ONNX model on CPU. First inference time: 0.0174 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
ONNX model on CPU: 0.0123 seconds per image (81.52 FPS)
|
ONNX model on CPU: 0.0136 seconds per image (73.63 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model
|
OpenVINO IR model
|
||||||
|
|
@ -570,12 +562,12 @@ accuracy drop. That’s why we skip that step in this notebook.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU. First inference time: 0.0166 seconds
|
OpenVINO model on CPU. First inference time: 0.0153 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU: 0.0122 seconds per image (81.86 FPS)
|
OpenVINO model on CPU: 0.0122 seconds per image (82.17 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model on GPU
|
OpenVINO IR model on GPU
|
||||||
|
|
@ -636,12 +628,12 @@ If it is the case, don’t use it.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + more threads on CPU. First inference time: 0.0153 seconds
|
OpenVINO model + more threads on CPU. First inference time: 0.0150 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + more threads on CPU: 0.0122 seconds per image (82.19 FPS)
|
OpenVINO model + more threads on CPU: 0.0122 seconds per image (81.82 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model in latency mode
|
OpenVINO IR model in latency mode
|
||||||
|
|
@ -672,12 +664,12 @@ devices as well.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO. First inference time: 0.0154 seconds
|
OpenVINO model on AUTO. First inference time: 0.0153 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO: 0.0125 seconds per image (80.26 FPS)
|
OpenVINO model on AUTO: 0.0125 seconds per image (80.25 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model in latency mode + shared memory
|
OpenVINO IR model in latency mode + shared memory
|
||||||
|
|
@ -712,12 +704,12 @@ performance!
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + shared memory on AUTO. First inference time: 0.0124 seconds
|
OpenVINO model + shared memory on AUTO. First inference time: 0.0113 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + shared memory on AUTO: 0.0054 seconds per image (184.61 FPS)
|
OpenVINO model + shared memory on AUTO: 0.0054 seconds per image (186.01 FPS)
|
||||||
|
|
||||||
|
|
||||||
Other tricks
|
Other tricks
|
||||||
|
|
@ -728,8 +720,8 @@ Other tricks
|
||||||
There are other tricks for performance improvement, such as quantization
|
There are other tricks for performance improvement, such as quantization
|
||||||
and pre-post-processing or dedicated to throughput mode. To get even
|
and pre-post-processing or dedicated to throughput mode. To get even
|
||||||
more from your model, please visit
|
more from your model, please visit
|
||||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__,
|
`111-detection-quantization <111-detection-quantization-with-output.html>`__,
|
||||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__, and
|
`118-optimize-preprocessing <118-optimize-preprocessing>`__, and
|
||||||
`109-throughput-tricks <109-throughput-tricks-with-output.html>`__.
|
`109-throughput-tricks <109-throughput-tricks-with-output.html>`__.
|
||||||
|
|
||||||
Performance comparison
|
Performance comparison
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:4f157b7f097e536fab10e5a103ae836be062794d8d7b4f9b16185e2e6900d2f5
|
oid sha256:ae331df2cb3993ba23bef0c441831024024f4f1f946c2da17a2ef6a2e6a68d8f
|
||||||
size 56983
|
size 52876
|
||||||
|
|
|
||||||
|
|
@ -1,14 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-latency-tricks-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-latency-tricks-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="109-latency-tricks-with-output_15_0.jpg">109-latency-tricks-with-output_15_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_18_0.jpg">109-latency-tricks-with-output_18_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_20_0.jpg">109-latency-tricks-with-output_20_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_24_0.jpg">109-latency-tricks-with-output_24_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_26_0.jpg">109-latency-tricks-with-output_26_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_28_0.jpg">109-latency-tricks-with-output_28_0.jpg</a> 26-Jan-2024 01:04 162715
|
|
||||||
<a href="109-latency-tricks-with-output_31_0.png">109-latency-tricks-with-output_31_0.png</a> 26-Jan-2024 01:04 56983
|
|
||||||
<a href="109-latency-tricks-with-output_5_0.jpg">109-latency-tricks-with-output_5_0.jpg</a> 26-Jan-2024 01:04 155828
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -24,7 +24,7 @@ The quantization and pre-post-processing API are not included here as
|
||||||
they change the precision (quantization) or processing graph
|
they change the precision (quantization) or processing graph
|
||||||
(prepostprocessor). You can find examples of how to apply them to
|
(prepostprocessor). You can find examples of how to apply them to
|
||||||
optimize performance on OpenVINO IR files in
|
optimize performance on OpenVINO IR files in
|
||||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__ and
|
`111-detection-quantization <111-detection-quantization-with-output.html>`__ and
|
||||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||||
|
|
||||||
|image0|
|
|image0|
|
||||||
|
|
@ -93,7 +93,7 @@ Prerequisites
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, List, Tuple
|
from typing import Any, List, Tuple
|
||||||
|
|
||||||
# Fetch `notebook_utils` module
|
# Fetch `notebook_utils` module
|
||||||
import urllib.request
|
import urllib.request
|
||||||
urllib.request.urlretrieve(
|
urllib.request.urlretrieve(
|
||||||
|
|
@ -116,24 +116,24 @@ object detection model.
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import cv2
|
import cv2
|
||||||
|
|
||||||
FRAMES_NUMBER = 1024
|
FRAMES_NUMBER = 1024
|
||||||
|
|
||||||
IMAGE_WIDTH = 640
|
IMAGE_WIDTH = 640
|
||||||
IMAGE_HEIGHT = 480
|
IMAGE_HEIGHT = 480
|
||||||
|
|
||||||
# load image
|
# load image
|
||||||
image = utils.load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg")
|
image = utils.load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg")
|
||||||
image = cv2.resize(image, dsize=(IMAGE_WIDTH, IMAGE_HEIGHT), interpolation=cv2.INTER_AREA)
|
image = cv2.resize(image, dsize=(IMAGE_WIDTH, IMAGE_HEIGHT), interpolation=cv2.INTER_AREA)
|
||||||
|
|
||||||
# preprocess it for YOLOv5
|
# preprocess it for YOLOv5
|
||||||
input_image = image / 255.0
|
input_image = image / 255.0
|
||||||
input_image = np.transpose(input_image, axes=(2, 0, 1))
|
input_image = np.transpose(input_image, axes=(2, 0, 1))
|
||||||
input_image = np.expand_dims(input_image, axis=0)
|
input_image = np.expand_dims(input_image, axis=0)
|
||||||
|
|
||||||
# simulate video with many frames
|
# simulate video with many frames
|
||||||
video_frames = np.tile(input_image, (FRAMES_NUMBER, 1, 1, 1, 1))
|
video_frames = np.tile(input_image, (FRAMES_NUMBER, 1, 1, 1, 1))
|
||||||
|
|
||||||
# show the image
|
# show the image
|
||||||
utils.show_array(image)
|
utils.show_array(image)
|
||||||
|
|
||||||
|
|
@ -146,7 +146,7 @@ object detection model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<DisplayHandle display_id=27f64c41c38d04ac2269516dbbad4c95>
|
<DisplayHandle display_id=bbb34b7fd1ad545280d19661bf0bd4c3>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -164,13 +164,13 @@ PyTorch Hub and small enough to see the difference in performance.
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
from IPython.utils import io
|
from IPython.utils import io
|
||||||
|
|
||||||
# directory for all models
|
# directory for all models
|
||||||
base_model_dir = Path("model")
|
base_model_dir = Path("model")
|
||||||
|
|
||||||
model_name = "yolov5n"
|
model_name = "yolov5n"
|
||||||
model_path = base_model_dir / model_name
|
model_path = base_model_dir / model_name
|
||||||
|
|
||||||
# load YOLOv5n from PyTorch Hub
|
# load YOLOv5n from PyTorch Hub
|
||||||
pytorch_model = torch.hub.load("ultralytics/yolov5", "custom", path=model_path, device="cpu", skip_validation=True)
|
pytorch_model = torch.hub.load("ultralytics/yolov5", "custom", path=model_path, device="cpu", skip_validation=True)
|
||||||
# don't print full model architecture
|
# don't print full model architecture
|
||||||
|
|
@ -186,12 +186,12 @@ PyTorch Hub and small enough to see the difference in performance.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
YOLOv5 🚀 2023-4-21 Python-3.8.10 torch-2.1.0+cpu CPU
|
YOLOv5 🚀 2023-4-21 Python-3.8.10 torch-2.1.0+cpu CPU
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Fusing layers...
|
Fusing layers...
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -201,7 +201,7 @@ PyTorch Hub and small enough to see the difference in performance.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Adding AutoShape...
|
Adding AutoShape...
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -223,10 +223,10 @@ benchmarking process.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
|
|
||||||
# initialize OpenVINO
|
# initialize OpenVINO
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
|
|
||||||
# print available devices
|
# print available devices
|
||||||
for device in core.available_devices:
|
for device in core.available_devices:
|
||||||
device_name = core.get_property(device, "FULL_DEVICE_NAME")
|
device_name = core.get_property(device, "FULL_DEVICE_NAME")
|
||||||
|
|
@ -250,8 +250,8 @@ second (FPS).
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from openvino.runtime import AsyncInferQueue
|
from openvino.runtime import AsyncInferQueue
|
||||||
|
|
||||||
|
|
||||||
def benchmark_model(model: Any, frames: np.ndarray, async_queue: AsyncInferQueue = None, benchmark_name: str = "OpenVINO model", device_name: str = "CPU") -> float:
|
def benchmark_model(model: Any, frames: np.ndarray, async_queue: AsyncInferQueue = None, benchmark_name: str = "OpenVINO model", device_name: str = "CPU") -> float:
|
||||||
"""
|
"""
|
||||||
Helper function for benchmarking the model. It measures the time and prints results.
|
Helper function for benchmarking the model. It measures the time and prints results.
|
||||||
|
|
@ -264,7 +264,7 @@ second (FPS).
|
||||||
end = time.perf_counter()
|
end = time.perf_counter()
|
||||||
first_infer_time = end - start
|
first_infer_time = end - start
|
||||||
print(f"{benchmark_name} on {device_name}. First inference time: {first_infer_time :.4f} seconds")
|
print(f"{benchmark_name} on {device_name}. First inference time: {first_infer_time :.4f} seconds")
|
||||||
|
|
||||||
# benchmarking
|
# benchmarking
|
||||||
start = time.perf_counter()
|
start = time.perf_counter()
|
||||||
for batch in frames:
|
for batch in frames:
|
||||||
|
|
@ -273,15 +273,15 @@ second (FPS).
|
||||||
if async_queue:
|
if async_queue:
|
||||||
async_queue.wait_all()
|
async_queue.wait_all()
|
||||||
end = time.perf_counter()
|
end = time.perf_counter()
|
||||||
|
|
||||||
# elapsed time
|
# elapsed time
|
||||||
infer_time = end - start
|
infer_time = end - start
|
||||||
|
|
||||||
# print second per image and FPS
|
# print second per image and FPS
|
||||||
mean_infer_time = infer_time / FRAMES_NUMBER
|
mean_infer_time = infer_time / FRAMES_NUMBER
|
||||||
mean_fps = FRAMES_NUMBER / infer_time
|
mean_fps = FRAMES_NUMBER / infer_time
|
||||||
print(f"{benchmark_name} on {device_name}: {mean_infer_time :.4f} seconds per image ({mean_fps :.2f} FPS)")
|
print(f"{benchmark_name} on {device_name}: {mean_infer_time :.4f} seconds per image ({mean_fps :.2f} FPS)")
|
||||||
|
|
||||||
return mean_fps
|
return mean_fps
|
||||||
|
|
||||||
The following functions aim to post-process results and draw boxes on
|
The following functions aim to post-process results and draw boxes on
|
||||||
|
|
@ -300,21 +300,21 @@ the image.
|
||||||
"cell phone", "microwave", "oven", "oaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
|
"cell phone", "microwave", "oven", "oaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
|
||||||
"hair drier", "toothbrush"
|
"hair drier", "toothbrush"
|
||||||
]
|
]
|
||||||
|
|
||||||
# Colors for the classes above (Rainbow Color Map).
|
# Colors for the classes above (Rainbow Color Map).
|
||||||
colors = cv2.applyColorMap(
|
colors = cv2.applyColorMap(
|
||||||
src=np.arange(0, 255, 255 / len(classes), dtype=np.float32).astype(np.uint8),
|
src=np.arange(0, 255, 255 / len(classes), dtype=np.float32).astype(np.uint8),
|
||||||
colormap=cv2.COLORMAP_RAINBOW,
|
colormap=cv2.COLORMAP_RAINBOW,
|
||||||
).squeeze()
|
).squeeze()
|
||||||
|
|
||||||
|
|
||||||
def postprocess(detections: np.ndarray) -> List[Tuple]:
|
def postprocess(detections: np.ndarray) -> List[Tuple]:
|
||||||
"""
|
"""
|
||||||
Postprocess the raw results from the model.
|
Postprocess the raw results from the model.
|
||||||
"""
|
"""
|
||||||
# candidates - probability > 0.25
|
# candidates - probability > 0.25
|
||||||
detections = detections[detections[..., 4] > 0.25]
|
detections = detections[detections[..., 4] > 0.25]
|
||||||
|
|
||||||
boxes = []
|
boxes = []
|
||||||
labels = []
|
labels = []
|
||||||
scores = []
|
scores = []
|
||||||
|
|
@ -328,22 +328,22 @@ the image.
|
||||||
)
|
)
|
||||||
labels.append(int(label))
|
labels.append(int(label))
|
||||||
scores.append(float(score))
|
scores.append(float(score))
|
||||||
|
|
||||||
# Apply non-maximum suppression to get rid of many overlapping entities.
|
# Apply non-maximum suppression to get rid of many overlapping entities.
|
||||||
# See https://paperswithcode.com/method/non-maximum-suppression
|
# See https://paperswithcode.com/method/non-maximum-suppression
|
||||||
# This algorithm returns indices of objects to keep.
|
# This algorithm returns indices of objects to keep.
|
||||||
indices = cv2.dnn.NMSBoxes(
|
indices = cv2.dnn.NMSBoxes(
|
||||||
bboxes=boxes, scores=scores, score_threshold=0.25, nms_threshold=0.5
|
bboxes=boxes, scores=scores, score_threshold=0.25, nms_threshold=0.5
|
||||||
)
|
)
|
||||||
|
|
||||||
# If there are no boxes.
|
# If there are no boxes.
|
||||||
if len(indices) == 0:
|
if len(indices) == 0:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
# Filter detected objects.
|
# Filter detected objects.
|
||||||
return [(labels[idx], scores[idx], boxes[idx]) for idx in indices.flatten()]
|
return [(labels[idx], scores[idx], boxes[idx]) for idx in indices.flatten()]
|
||||||
|
|
||||||
|
|
||||||
def draw_boxes(img: np.ndarray, boxes):
|
def draw_boxes(img: np.ndarray, boxes):
|
||||||
"""
|
"""
|
||||||
Draw detected boxes on the image.
|
Draw detected boxes on the image.
|
||||||
|
|
@ -355,7 +355,7 @@ the image.
|
||||||
x2 = box[0] + box[2]
|
x2 = box[0] + box[2]
|
||||||
y2 = box[1] + box[3]
|
y2 = box[1] + box[3]
|
||||||
cv2.rectangle(img=img, pt1=box[:2], pt2=(x2, y2), color=color, thickness=2)
|
cv2.rectangle(img=img, pt1=box[:2], pt2=(x2, y2), color=color, thickness=2)
|
||||||
|
|
||||||
# Draw a label name inside the box.
|
# Draw a label name inside the box.
|
||||||
cv2.putText(
|
cv2.putText(
|
||||||
img=img,
|
img=img,
|
||||||
|
|
@ -367,17 +367,17 @@ the image.
|
||||||
thickness=1,
|
thickness=1,
|
||||||
lineType=cv2.LINE_AA,
|
lineType=cv2.LINE_AA,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def show_result(results: np.ndarray):
|
def show_result(results: np.ndarray):
|
||||||
"""
|
"""
|
||||||
Postprocess the raw results, draw boxes and show the image.
|
Postprocess the raw results, draw boxes and show the image.
|
||||||
"""
|
"""
|
||||||
output_img = image.copy()
|
output_img = image.copy()
|
||||||
|
|
||||||
detections = postprocess(results)
|
detections = postprocess(results)
|
||||||
draw_boxes(output_img, detections)
|
draw_boxes(output_img, detections)
|
||||||
|
|
||||||
utils.show_array(output_img)
|
utils.show_array(output_img)
|
||||||
|
|
||||||
Optimizations
|
Optimizations
|
||||||
|
|
@ -400,7 +400,7 @@ optimizations applied. We will treat it as our baseline.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
|
|
||||||
with torch.no_grad():
|
with torch.no_grad():
|
||||||
result = pytorch_model(torch.as_tensor(video_frames[0])).detach().numpy()[0]
|
result = pytorch_model(torch.as_tensor(video_frames[0])).detach().numpy()[0]
|
||||||
show_result(result)
|
show_result(result)
|
||||||
|
|
@ -413,12 +413,12 @@ optimizations applied. We will treat it as our baseline.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU. First inference time: 0.0201 seconds
|
PyTorch model on CPU. First inference time: 0.0224 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PyTorch model on CPU: 0.0192 seconds per image (52.19 FPS)
|
PyTorch model on CPU: 0.0221 seconds per image (45.32 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model
|
OpenVINO IR model
|
||||||
|
|
@ -438,23 +438,23 @@ step in this notebook.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
onnx_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}").with_suffix(".onnx")
|
onnx_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}").with_suffix(".onnx")
|
||||||
|
|
||||||
# export PyTorch model to ONNX if it doesn't already exist
|
# export PyTorch model to ONNX if it doesn't already exist
|
||||||
if not onnx_path.exists():
|
if not onnx_path.exists():
|
||||||
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||||
torch.onnx.export(pytorch_model, dummy_input, onnx_path)
|
torch.onnx.export(pytorch_model, dummy_input, onnx_path)
|
||||||
|
|
||||||
# convert ONNX model to IR, use FP16
|
# convert ONNX model to IR, use FP16
|
||||||
ov_model = ov.convert_model(onnx_path)
|
ov_model = ov.convert_model(onnx_path)
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_cpu_model = core.compile_model(ov_model, device_name="CPU")
|
ov_cpu_model = core.compile_model(ov_model, device_name="CPU")
|
||||||
|
|
||||||
result = ov_cpu_model(video_frames[0])[ov_cpu_model.output(0)][0]
|
result = ov_cpu_model(video_frames[0])[ov_cpu_model.output(0)][0]
|
||||||
show_result(result)
|
show_result(result)
|
||||||
ov_cpu_fps = benchmark_model(model=ov_cpu_model, frames=video_frames, benchmark_name="OpenVINO model")
|
ov_cpu_fps = benchmark_model(model=ov_cpu_model, frames=video_frames, benchmark_name="OpenVINO model")
|
||||||
|
|
||||||
del ov_cpu_model # release resources
|
del ov_cpu_model # release resources
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -464,12 +464,12 @@ step in this notebook.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU. First inference time: 0.0134 seconds
|
OpenVINO model on CPU. First inference time: 0.0142 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU: 0.0070 seconds per image (142.35 FPS)
|
OpenVINO model on CPU: 0.0071 seconds per image (141.49 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model + bigger batch
|
OpenVINO IR model + bigger batch
|
||||||
|
|
@ -488,13 +488,13 @@ hardware and model.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
batch_size = 4
|
batch_size = 4
|
||||||
|
|
||||||
onnx_batch_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}_batch_{batch_size}").with_suffix(".onnx")
|
onnx_batch_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}_batch_{batch_size}").with_suffix(".onnx")
|
||||||
|
|
||||||
if not onnx_batch_path.exists():
|
if not onnx_batch_path.exists():
|
||||||
dummy_input = torch.randn(batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
dummy_input = torch.randn(batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||||
torch.onnx.export(pytorch_model, dummy_input, onnx_batch_path)
|
torch.onnx.export(pytorch_model, dummy_input, onnx_batch_path)
|
||||||
|
|
||||||
# export the model with the bigger batch size
|
# export the model with the bigger batch size
|
||||||
ov_batch_model = ov.convert_model(onnx_batch_path)
|
ov_batch_model = ov.convert_model(onnx_batch_path)
|
||||||
|
|
||||||
|
|
@ -510,13 +510,13 @@ hardware and model.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_cpu_batch_model = core.compile_model(ov_batch_model, device_name="CPU")
|
ov_cpu_batch_model = core.compile_model(ov_batch_model, device_name="CPU")
|
||||||
|
|
||||||
batched_video_frames = video_frames.reshape([-1, batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH])
|
batched_video_frames = video_frames.reshape([-1, batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH])
|
||||||
|
|
||||||
result = ov_cpu_batch_model(batched_video_frames[0])[ov_cpu_batch_model.output(0)][0]
|
result = ov_cpu_batch_model(batched_video_frames[0])[ov_cpu_batch_model.output(0)][0]
|
||||||
show_result(result)
|
show_result(result)
|
||||||
ov_cpu_batch_fps = benchmark_model(model=ov_cpu_batch_model, frames=batched_video_frames, benchmark_name="OpenVINO model + bigger batch")
|
ov_cpu_batch_fps = benchmark_model(model=ov_cpu_batch_model, frames=batched_video_frames, benchmark_name="OpenVINO model + bigger batch")
|
||||||
|
|
||||||
del ov_cpu_batch_model # release resources
|
del ov_cpu_batch_model # release resources
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -526,12 +526,12 @@ hardware and model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + bigger batch on CPU. First inference time: 0.0435 seconds
|
OpenVINO model + bigger batch on CPU. First inference time: 0.0486 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model + bigger batch on CPU: 0.0068 seconds per image (147.59 FPS)
|
OpenVINO model + bigger batch on CPU: 0.0068 seconds per image (146.44 FPS)
|
||||||
|
|
||||||
|
|
||||||
Asynchronous processing
|
Asynchronous processing
|
||||||
|
|
@ -561,17 +561,17 @@ the pipeline.
|
||||||
result = infer_request.get_output_tensor(0).data[0]
|
result = infer_request.get_output_tensor(0).data[0]
|
||||||
show_result(result)
|
show_result(result)
|
||||||
pass
|
pass
|
||||||
|
|
||||||
infer_queue = ov.AsyncInferQueue(ov_model)
|
infer_queue = ov.AsyncInferQueue(ov_model)
|
||||||
infer_queue.set_callback(callback) # set callback to post-process (show) results
|
infer_queue.set_callback(callback) # set callback to post-process (show) results
|
||||||
|
|
||||||
infer_queue.start_async(video_frames[0])
|
infer_queue.start_async(video_frames[0])
|
||||||
infer_queue.wait_all()
|
infer_queue.wait_all()
|
||||||
|
|
||||||
# don't show output for the remaining frames
|
# don't show output for the remaining frames
|
||||||
infer_queue.set_callback(lambda x, y: {})
|
infer_queue.set_callback(lambda x, y: {})
|
||||||
fps = benchmark_model(model=infer_queue.start_async, frames=video_frames, async_queue=infer_queue, benchmark_name=benchmark_name, device_name=device_name)
|
fps = benchmark_model(model=infer_queue.start_async, frames=video_frames, async_queue=infer_queue, benchmark_name=benchmark_name, device_name=device_name)
|
||||||
|
|
||||||
del infer_queue # release resources
|
del infer_queue # release resources
|
||||||
return fps
|
return fps
|
||||||
|
|
||||||
|
|
@ -592,9 +592,9 @@ feature, which sets the batch size to the optimal level.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_cpu_through_model = core.compile_model(ov_model, device_name="CPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
ov_cpu_through_model = core.compile_model(ov_model, device_name="CPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||||
|
|
||||||
ov_cpu_through_fps = benchmark_async_mode(ov_cpu_through_model, benchmark_name="OpenVINO model", device_name="CPU (THROUGHPUT)")
|
ov_cpu_through_fps = benchmark_async_mode(ov_cpu_through_model, benchmark_name="OpenVINO model", device_name="CPU (THROUGHPUT)")
|
||||||
|
|
||||||
del ov_cpu_through_model # release resources
|
del ov_cpu_through_model # release resources
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -604,12 +604,12 @@ feature, which sets the batch size to the optimal level.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU (THROUGHPUT). First inference time: 0.0260 seconds
|
OpenVINO model on CPU (THROUGHPUT). First inference time: 0.0221 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on CPU (THROUGHPUT): 0.0040 seconds per image (248.03 FPS)
|
OpenVINO model on CPU (THROUGHPUT): 0.0040 seconds per image (249.65 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model in throughput mode on GPU
|
OpenVINO IR model in throughput mode on GPU
|
||||||
|
|
@ -633,9 +633,9 @@ execution.
|
||||||
if "GPU" in core.available_devices:
|
if "GPU" in core.available_devices:
|
||||||
# compile for GPU
|
# compile for GPU
|
||||||
ov_gpu_model = core.compile_model(ov_model, device_name="GPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
ov_gpu_model = core.compile_model(ov_model, device_name="GPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||||
|
|
||||||
ov_gpu_fps = benchmark_async_mode(ov_gpu_model, benchmark_name="OpenVINO model", device_name="GPU (THROUGHPUT)")
|
ov_gpu_fps = benchmark_async_mode(ov_gpu_model, benchmark_name="OpenVINO model", device_name="GPU (THROUGHPUT)")
|
||||||
|
|
||||||
del ov_gpu_model # release resources
|
del ov_gpu_model # release resources
|
||||||
|
|
||||||
OpenVINO IR model in throughput mode on AUTO
|
OpenVINO IR model in throughput mode on AUTO
|
||||||
|
|
@ -651,9 +651,9 @@ performance hint.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_auto_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
ov_auto_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||||
|
|
||||||
ov_auto_fps = benchmark_async_mode(ov_auto_model, benchmark_name="OpenVINO model", device_name="AUTO (THROUGHPUT)")
|
ov_auto_fps = benchmark_async_mode(ov_auto_model, benchmark_name="OpenVINO model", device_name="AUTO (THROUGHPUT)")
|
||||||
|
|
||||||
del ov_auto_model # release resources
|
del ov_auto_model # release resources
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -663,12 +663,12 @@ performance hint.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO (THROUGHPUT). First inference time: 0.0244 seconds
|
OpenVINO model on AUTO (THROUGHPUT). First inference time: 0.0235 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO (THROUGHPUT): 0.0040 seconds per image (250.62 FPS)
|
OpenVINO model on AUTO (THROUGHPUT): 0.0040 seconds per image (249.90 FPS)
|
||||||
|
|
||||||
|
|
||||||
OpenVINO IR model in cumulative throughput mode on AUTO
|
OpenVINO IR model in cumulative throughput mode on AUTO
|
||||||
|
|
@ -685,7 +685,7 @@ activate all devices.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_auto_cumulative_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "CUMULATIVE_THROUGHPUT"})
|
ov_auto_cumulative_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "CUMULATIVE_THROUGHPUT"})
|
||||||
|
|
||||||
ov_auto_cumulative_fps = benchmark_async_mode(ov_auto_cumulative_model, benchmark_name="OpenVINO model", device_name="AUTO (CUMULATIVE THROUGHPUT)")
|
ov_auto_cumulative_fps = benchmark_async_mode(ov_auto_cumulative_model, benchmark_name="OpenVINO model", device_name="AUTO (CUMULATIVE THROUGHPUT)")
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -695,12 +695,12 @@ activate all devices.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT). First inference time: 0.0211 seconds
|
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT). First inference time: 0.0227 seconds
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT): 0.0040 seconds per image (249.40 FPS)
|
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT): 0.0040 seconds per image (250.47 FPS)
|
||||||
|
|
||||||
|
|
||||||
Other tricks
|
Other tricks
|
||||||
|
|
@ -714,7 +714,7 @@ mode. To get even more from your model, please visit `advanced
|
||||||
throughput
|
throughput
|
||||||
options <https://docs.openvino.ai/2023.3/openvino_docs_deployment_optimization_guide_tput_advanced.html>`__,
|
options <https://docs.openvino.ai/2023.3/openvino_docs_deployment_optimization_guide_tput_advanced.html>`__,
|
||||||
`109-latency-tricks <109-latency-tricks-with-output.html-with-output.html>`__,
|
`109-latency-tricks <109-latency-tricks-with-output.html-with-output.html>`__,
|
||||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__, and
|
`111-detection-quantization <111-detection-quantization-with-output.html>`__, and
|
||||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||||
|
|
||||||
Performance comparison
|
Performance comparison
|
||||||
|
|
@ -733,20 +733,20 @@ steps, just skip them.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from matplotlib import pyplot as plt
|
from matplotlib import pyplot as plt
|
||||||
|
|
||||||
labels = ["PyTorch model", "OpenVINO IR model", "OpenVINO IR model + bigger batch", "OpenVINO IR model in throughput mode", "OpenVINO IR model in throughput mode on GPU",
|
labels = ["PyTorch model", "OpenVINO IR model", "OpenVINO IR model + bigger batch", "OpenVINO IR model in throughput mode", "OpenVINO IR model in throughput mode on GPU",
|
||||||
"OpenVINO IR model in throughput mode on AUTO", "OpenVINO IR model in cumulative throughput mode on AUTO"]
|
"OpenVINO IR model in throughput mode on AUTO", "OpenVINO IR model in cumulative throughput mode on AUTO"]
|
||||||
|
|
||||||
fps = [pytorch_fps, ov_cpu_fps, ov_cpu_batch_fps, ov_cpu_through_fps, ov_gpu_fps, ov_auto_fps, ov_auto_cumulative_fps]
|
fps = [pytorch_fps, ov_cpu_fps, ov_cpu_batch_fps, ov_cpu_through_fps, ov_gpu_fps, ov_auto_fps, ov_auto_cumulative_fps]
|
||||||
|
|
||||||
bar_colors = colors[::10] / 255.0
|
bar_colors = colors[::10] / 255.0
|
||||||
|
|
||||||
fig, ax = plt.subplots(figsize=(16, 8))
|
fig, ax = plt.subplots(figsize=(16, 8))
|
||||||
ax.bar(labels, fps, color=bar_colors)
|
ax.bar(labels, fps, color=bar_colors)
|
||||||
|
|
||||||
ax.set_ylabel("Throughput [FPS]")
|
ax.set_ylabel("Throughput [FPS]")
|
||||||
ax.set_title("Performance difference")
|
ax.set_title("Performance difference")
|
||||||
|
|
||||||
plt.xticks(rotation='vertical')
|
plt.xticks(rotation='vertical')
|
||||||
plt.show()
|
plt.show()
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:8e1932b29ece3712ada3af1ea8df832692a5416111c7caeb3c884cd3ee628110
|
oid sha256:c054b14da27759eeaf1a1791c919f28eade5a92ea995b1f07690c71c435b2092
|
||||||
size 62463
|
size 62458
|
||||||
|
|
|
||||||
|
|
@ -1,14 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-throughput-tricks-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-throughput-tricks-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="109-throughput-tricks-with-output_15_0.jpg">109-throughput-tricks-with-output_15_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_18_0.jpg">109-throughput-tricks-with-output_18_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_21_0.jpg">109-throughput-tricks-with-output_21_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_25_0.jpg">109-throughput-tricks-with-output_25_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_29_0.jpg">109-throughput-tricks-with-output_29_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_31_0.jpg">109-throughput-tricks-with-output_31_0.jpg</a> 26-Jan-2024 01:05 162715
|
|
||||||
<a href="109-throughput-tricks-with-output_34_0.png">109-throughput-tricks-with-output_34_0.png</a> 26-Jan-2024 01:05 62463
|
|
||||||
<a href="109-throughput-tricks-with-output_5_0.jpg">109-throughput-tricks-with-output_5_0.jpg</a> 26-Jan-2024 01:05 155828
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -18,7 +18,7 @@ This notebook needs a quantized OpenVINO IR model and images from the
|
||||||
`KiTS-19 <https://github.com/neheller/kits19>`__ dataset, converted to
|
`KiTS-19 <https://github.com/neheller/kits19>`__ dataset, converted to
|
||||||
2D images. (To learn how the model is quantized, see the `Convert and
|
2D images. (To learn how the model is quantized, see the `Convert and
|
||||||
Quantize a UNet Model and Show Live
|
Quantize a UNet Model and Show Live
|
||||||
Inference <110-ct-segmentation-quantize-nncf-with-output.html>`__ tutorial.)
|
Inference <110-ct-segmentation-quantize-nncf.ipynb>`__ tutorial.)
|
||||||
|
|
||||||
This notebook provides a pre-trained model, trained for 20 epochs with
|
This notebook provides a pre-trained model, trained for 20 epochs with
|
||||||
the full KiTS-19 frames dataset, which has an F1 score on the validation
|
the full KiTS-19 frames dataset, which has an F1 score on the validation
|
||||||
|
|
@ -80,14 +80,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:50:13.572016: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:50:38.323593: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:50:13.606068: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:50:38.357752: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:50:14.166493: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:50:38.922511: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
Settings
|
Settings
|
||||||
|
|
@ -140,7 +140,7 @@ Tool <https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html>`__
|
||||||
is a command-line application that can be run in the notebook with
|
is a command-line application that can be run in the notebook with
|
||||||
``! benchmark_app`` or ``%sx benchmark_app`` commands.
|
``! benchmark_app`` or ``%sx benchmark_app`` commands.
|
||||||
|
|
||||||
**Note**: The ``benchmark_app`` tool is able to measure the
|
**NOTE**: The ``benchmark_app`` tool is able to measure the
|
||||||
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
||||||
models only. For more accurate performance, run ``benchmark_app`` in
|
models only. For more accurate performance, run ``benchmark_app`` in
|
||||||
a terminal/command prompt after closing other applications. Run
|
a terminal/command prompt after closing other applications. Run
|
||||||
|
|
@ -198,7 +198,7 @@ is a command-line application that can be run in the notebook with
|
||||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||||
[Step 4/11] Reading model files
|
[Step 4/11] Reading model files
|
||||||
[ INFO ] Loading model files
|
[ INFO ] Loading model files
|
||||||
[ INFO ] Read model took 13.17 ms
|
[ INFO ] Read model took 13.02 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input.1 (node: input.1) : f32 / [...] / [1,1,512,512]
|
[ INFO ] input.1 (node: input.1) : f32 / [...] / [1,1,512,512]
|
||||||
|
|
@ -216,7 +216,7 @@ is a command-line application that can be run in the notebook with
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 235.31 ms
|
[ INFO ] Compile model took 232.64 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: pretrained_unet_kits19
|
[ INFO ] NETWORK_NAME: pretrained_unet_kits19
|
||||||
|
|
@ -252,21 +252,21 @@ is a command-line application that can be run in the notebook with
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 24.14 ms
|
[ INFO ] First inference took 24.68 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 1355 iterations
|
[ INFO ] Count: 1366 iterations
|
||||||
[ INFO ] Duration: 15007.81 ms
|
[ INFO ] Duration: 15004.33 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 10.84 ms
|
[ INFO ] Median: 10.75 ms
|
||||||
[ INFO ] Average: 10.89 ms
|
[ INFO ] Average: 10.80 ms
|
||||||
[ INFO ] Min: 10.58 ms
|
[ INFO ] Min: 10.53 ms
|
||||||
[ INFO ] Max: 14.29 ms
|
[ INFO ] Max: 12.59 ms
|
||||||
[ INFO ] Throughput: 90.29 FPS
|
[ INFO ] Throughput: 91.04 FPS
|
||||||
|
|
||||||
|
|
||||||
Download and Prepare Data
|
Download and Prepare Data
|
||||||
|
|
@ -503,7 +503,7 @@ Create asynchronous inference queue and perform it
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Loaded model to Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO') in 0.24 seconds.
|
Loaded model to Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO') in 0.23 seconds.
|
||||||
Total time to infer all frames: 2.762s
|
Total time to infer all frames: 2.588s
|
||||||
Time per frame: 0.040619s (24.619 FPS)
|
Time per frame: 0.038061s (26.274 FPS)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-scan-live-inference-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-scan-live-inference-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="110-ct-scan-live-inference-with-output_21_0.png">110-ct-scan-live-inference-with-output_21_0.png</a> 26-Jan-2024 01:05 48780
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -24,9 +24,9 @@ This third tutorial in the series shows how to:
|
||||||
All notebooks in this series:
|
All notebooks in this series:
|
||||||
|
|
||||||
- `Data Preparation for 2D Segmentation of 3D Medical
|
- `Data Preparation for 2D Segmentation of 3D Medical
|
||||||
Data <data-preparation-ct-scan-with-output.html>`__
|
Data <data-preparation-ct-scan.ipynb>`__
|
||||||
- `Train a 2D-UNet Medical Imaging Model with PyTorch
|
- `Train a 2D-UNet Medical Imaging Model with PyTorch
|
||||||
Lightning <pytorch-monai-training-with-output.html>`__
|
Lightning <pytorch-monai-training.ipynb>`__
|
||||||
- Convert and Quantize a Segmentation Model and Show Live Inference
|
- Convert and Quantize a Segmentation Model and Show Live Inference
|
||||||
(this notebook)
|
(this notebook)
|
||||||
- `Live Inference and Benchmark CT-scan
|
- `Live Inference and Benchmark CT-scan
|
||||||
|
|
@ -39,7 +39,7 @@ This notebook needs a trained UNet model. We provide a pre-trained
|
||||||
model, trained for 20 epochs with the full
|
model, trained for 20 epochs with the full
|
||||||
`Kits-19 <https://github.com/neheller/kits19>`__ frames dataset, which
|
`Kits-19 <https://github.com/neheller/kits19>`__ frames dataset, which
|
||||||
has an F1 score on the validation set of 0.9. The training code is
|
has an F1 score on the validation set of 0.9. The training code is
|
||||||
available in `this notebook <pytorch-monai-training-with-output.html>`__.
|
available in `this notebook <pytorch-monai-training.ipynb>`__.
|
||||||
|
|
||||||
NNCF for PyTorch models requires a C++ compiler. On Windows, install
|
NNCF for PyTorch models requires a C++ compiler. On Windows, install
|
||||||
`Microsoft Visual Studio
|
`Microsoft Visual Studio
|
||||||
|
|
@ -182,14 +182,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:50:47.254337: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:51:11.599112: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:50:47.287377: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:51:11.634091: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:50:47.852256: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:51:12.223414: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -205,7 +205,7 @@ Settings
|
||||||
By default, this notebook will download one CT scan from the KITS19
|
By default, this notebook will download one CT scan from the KITS19
|
||||||
dataset that will be used for quantization. To use the full dataset, set
|
dataset that will be used for quantization. To use the full dataset, set
|
||||||
``BASEDIR`` to the path of the dataset, as prepared according to the
|
``BASEDIR`` to the path of the dataset, as prepared according to the
|
||||||
`Data Preparation <data-preparation-ct-scan-with-output.html>`__ notebook.
|
`Data Preparation <data-preparation-ct-scan.ipynb>`__ notebook.
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -226,7 +226,7 @@ notebook is a
|
||||||
`BasicUNet <https://docs.monai.io/en/stable/networks.html#basicunet>`__
|
`BasicUNet <https://docs.monai.io/en/stable/networks.html#basicunet>`__
|
||||||
model from `MONAI <https://monai.io>`__. We provide a pre-trained
|
model from `MONAI <https://monai.io>`__. We provide a pre-trained
|
||||||
checkpoint. To see how this model performs, check out the `training
|
checkpoint. To see how this model performs, check out the `training
|
||||||
notebook <pytorch-monai-training-with-output.html>`__.
|
notebook <pytorch-monai-training.ipynb>`__.
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -304,7 +304,7 @@ Dataset
|
||||||
The ``KitsDataset`` class in the next cell expects images and masks in
|
The ``KitsDataset`` class in the next cell expects images and masks in
|
||||||
the *``basedir``* directory, in a folder per patient. It is a simplified
|
the *``basedir``* directory, in a folder per patient. It is a simplified
|
||||||
version of the Dataset class in the `training
|
version of the Dataset class in the `training
|
||||||
notebook <pytorch-monai-training-with-output.html>`__.
|
notebook <pytorch-monai-training.ipynb>`__.
|
||||||
|
|
||||||
Images are loaded with MONAI’s
|
Images are loaded with MONAI’s
|
||||||
`LoadImage <https://docs.monai.io/en/stable/transforms.html#loadimage>`__,
|
`LoadImage <https://docs.monai.io/en/stable/transforms.html#loadimage>`__,
|
||||||
|
|
@ -481,7 +481,7 @@ this notebook.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -489,7 +489,7 @@ this notebook.
|
||||||
advanced algorithms for Neural Networks inference optimization in
|
advanced algorithms for Neural Networks inference optimization in
|
||||||
OpenVINO with minimal accuracy drop.
|
OpenVINO with minimal accuracy drop.
|
||||||
|
|
||||||
**Note**: NNCF Post-training Quantization is available in OpenVINO
|
**NOTE**: NNCF Post-training Quantization is available in OpenVINO
|
||||||
2023.0 release.
|
2023.0 release.
|
||||||
|
|
||||||
Create a quantized model from the pre-trained ``FP32`` model and the
|
Create a quantized model from the pre-trained ``FP32`` model and the
|
||||||
|
|
@ -532,13 +532,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -549,14 +551,15 @@ steps:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -575,11 +578,11 @@ model and save it.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
return self._level_low.item()
|
return self._level_low.item()
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
return self._level_high.item()
|
return self._level_high.item()
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -692,7 +695,7 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
||||||
[Step 4/11] Reading model files
|
[Step 4/11] Reading model files
|
||||||
[ INFO ] Loading model files
|
[ INFO ] Loading model files
|
||||||
[ INFO ] Read model took 26.68 ms
|
[ INFO ] Read model took 26.51 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] x (node: x) : f32 / [...] / [?,?,?,?]
|
[ INFO ] x (node: x) : f32 / [...] / [?,?,?,?]
|
||||||
|
|
@ -710,7 +713,7 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 86.15 ms
|
[ INFO ] Compile model took 87.61 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: Model0
|
[ INFO ] NETWORK_NAME: Model0
|
||||||
|
|
@ -732,9 +735,9 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
[Step 9/11] Creating infer requests and preparing input tensors
|
[Step 9/11] Creating infer requests and preparing input tensors
|
||||||
[ ERROR ] Input x is dynamic. Provide data shapes!
|
[ ERROR ] Input x is dynamic. Provide data shapes!
|
||||||
Traceback (most recent call last):
|
Traceback (most recent call last):
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
||||||
data_queue = get_input_data(paths_to_input, app_inputs_info)
|
data_queue = get_input_data(paths_to_input, app_inputs_info)
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
||||||
raise Exception(f"Input {info.name} is dynamic. Provide data shapes!")
|
raise Exception(f"Input {info.name} is dynamic. Provide data shapes!")
|
||||||
Exception: Input x is dynamic. Provide data shapes!
|
Exception: Input x is dynamic. Provide data shapes!
|
||||||
|
|
||||||
|
|
@ -762,7 +765,7 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
||||||
[Step 4/11] Reading model files
|
[Step 4/11] Reading model files
|
||||||
[ INFO ] Loading model files
|
[ INFO ] Loading model files
|
||||||
[ INFO ] Read model took 13.10 ms
|
[ INFO ] Read model took 13.17 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] x.1 (node: x.1) : f32 / [...] / [1,1,512,512]
|
[ INFO ] x.1 (node: x.1) : f32 / [...] / [1,1,512,512]
|
||||||
|
|
@ -772,10 +775,6 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
[ INFO ] Model batch size: 1
|
[ INFO ] Model batch size: 1
|
||||||
[Step 6/11] Configuring input of the model
|
[Step 6/11] Configuring input of the model
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ INFO ] x.1 (node: x.1) : f32 / [N,C,H,W] / [1,1,512,512]
|
[ INFO ] x.1 (node: x.1) : f32 / [N,C,H,W] / [1,1,512,512]
|
||||||
[ INFO ] Model outputs:
|
[ INFO ] Model outputs:
|
||||||
[ INFO ] 571 (node: 571) : f32 / [...] / [1,1,512,512]
|
[ INFO ] 571 (node: 571) : f32 / [...] / [1,1,512,512]
|
||||||
|
|
@ -784,7 +783,7 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 188.53 ms
|
[ INFO ] Compile model took 190.56 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: main_graph
|
[ INFO ] NETWORK_NAME: main_graph
|
||||||
|
|
@ -812,21 +811,21 @@ be run in the notebook with ``! benchmark_app`` or
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 30.02 ms
|
[ INFO ] First inference took 30.25 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 964 iterations
|
[ INFO ] Count: 973 iterations
|
||||||
[ INFO ] Duration: 15009.54 ms
|
[ INFO ] Duration: 15006.05 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 15.33 ms
|
[ INFO ] Median: 15.16 ms
|
||||||
[ INFO ] Average: 15.36 ms
|
[ INFO ] Average: 15.22 ms
|
||||||
[ INFO ] Min: 14.97 ms
|
[ INFO ] Min: 14.87 ms
|
||||||
[ INFO ] Max: 17.10 ms
|
[ INFO ] Max: 17.88 ms
|
||||||
[ INFO ] Throughput: 64.23 FPS
|
[ INFO ] Throughput: 64.84 FPS
|
||||||
|
|
||||||
|
|
||||||
Visually Compare Inference Results
|
Visually Compare Inference Results
|
||||||
|
|
@ -911,7 +910,7 @@ seed is displayed to enable reproducing specific runs of this cell.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Visualizing results with seed 1706219510
|
Visualizing results with seed 1707515536
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -994,8 +993,8 @@ performs inference, and displays the results on the frames loaded in
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Loaded model to CPU in 0.21 seconds.
|
Loaded model to CPU in 0.18 seconds.
|
||||||
Total time for 68 frames: 2.62 seconds, fps:26.34
|
Total time for 68 frames: 2.68 seconds, fps:25.70
|
||||||
|
|
||||||
|
|
||||||
References
|
References
|
||||||
|
|
@ -1003,16 +1002,20 @@ References
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**OpenVINO**
|
**OpenVINO** - `NNCF
|
||||||
|
Repository <https://github.com/openvinotoolkit/nncf/>`__ - `Neural
|
||||||
|
Network Compression Framework for fast model
|
||||||
|
inference <https://arxiv.org/abs/2002.08679>`__ - `OpenVINO API
|
||||||
|
Tutorial <002-openvino-api-with-output.html>`__ - `OpenVINO
|
||||||
|
PyPI (pip install
|
||||||
|
openvino-dev) <https://pypi.org/project/openvino-dev/>`__
|
||||||
|
|
||||||
- `NNCF Repository <https://github.com/openvinotoolkit/nncf/>`__
|
**Kits19 Data** - `Kits19 Challenge
|
||||||
- `Neural Network Compression Framework for fast model inference <https://arxiv.org/abs/2002.08679>`__
|
Homepage <https://kits19.grand-challenge.org/>`__ - `Kits19 GitHub
|
||||||
- `OpenVINO API Tutorial <002-openvino-api-with-output.html>`__
|
Repository <https://github.com/neheller/kits19>`__ - `The KiTS19
|
||||||
- `OpenVINO PyPI (pip install openvino-dev) <https://pypi.org/project/openvino-dev/>`__
|
Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT
|
||||||
|
Semantic Segmentations, and Surgical
|
||||||
**Kits19 Data**
|
Outcomes <https://arxiv.org/abs/1904.00445>`__ - `The state of the art
|
||||||
|
in kidney and kidney tumor segmentation in contrast-enhanced CT imaging:
|
||||||
- `Kits19 Challenge Homepage <https://kits19.grand-challenge.org/>`__
|
Results of the KiTS19
|
||||||
- `Kits19 GitHub Repository <https://github.com/neheller/kits19>`__
|
challenge <https://www.sciencedirect.com/science/article/pii/S1361841520301857>`__
|
||||||
- `The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes <https://arxiv.org/abs/1904.00445>`__
|
|
||||||
- `The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge <https://www.sciencedirect.com/science/article/pii/S1361841520301857>`__
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:8b31829da55a95766d9f5f4e8b93cec1e5bac3f1147fd2a5cea2e8d7f7b12abc
|
oid sha256:fa3a987453cb9be2a1e6e3c9d6ef943c29914c0a1f740e2eac55ebad8bc9ac4a
|
||||||
size 386985
|
size 387367
|
||||||
|
|
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-segmentation-quantize-nncf-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-segmentation-quantize-nncf-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="110-ct-segmentation-quantize-nncf-with-output_15_1.png">110-ct-segmentation-quantize-nncf-with-output_1..></a> 26-Jan-2024 01:05 158997
|
|
||||||
<a href="110-ct-segmentation-quantize-nncf-with-output_37_1.png">110-ct-segmentation-quantize-nncf-with-output_3..></a> 26-Jan-2024 01:05 386985
|
|
||||||
<a href="110-ct-segmentation-quantize-nncf-with-output_42_0.jpg">110-ct-segmentation-quantize-nncf-with-output_4..></a> 26-Jan-2024 01:05 73812
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -67,10 +67,6 @@ Preparations
|
||||||
Note: you may need to restart the kernel to use updated packages.
|
Note: you may need to restart the kernel to use updated packages.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
Note: you may need to restart the kernel to use updated packages.
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -182,7 +178,7 @@ Settings
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/112-pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/112-pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -376,7 +372,8 @@ Create and load original uncompressed model
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
ResNet-50 from the `torchivision repository <https://github.com/pytorch/vision>`__ is pre-trained on
|
ResNet-50 from the ```torchivision``
|
||||||
|
repository <https://github.com/pytorch/vision>`__ is pre-trained on
|
||||||
ImageNet with more prediction classes than Tiny ImageNet, so the model
|
ImageNet with more prediction classes than Tiny ImageNet, so the model
|
||||||
is adjusted by swapping the last FC layer to one with fewer output
|
is adjusted by swapping the last FC layer to one with fewer output
|
||||||
values.
|
values.
|
||||||
|
|
@ -474,47 +471,47 @@ I. Evaluate the loaded model
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [ 0/79] Time 0.249 (0.249) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
Test: [ 0/79] Time 0.283 (0.283) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [10/79] Time 0.223 (0.231) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
Test: [10/79] Time 0.242 (0.242) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [20/79] Time 0.230 (0.231) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
Test: [20/79] Time 0.237 (0.241) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [30/79] Time 0.229 (0.230) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
Test: [30/79] Time 0.238 (0.241) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [40/79] Time 0.245 (0.229) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
Test: [40/79] Time 0.244 (0.241) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [50/79] Time 0.223 (0.229) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
Test: [50/79] Time 0.270 (0.245) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [60/79] Time 0.227 (0.229) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
Test: [60/79] Time 0.246 (0.245) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [70/79] Time 0.227 (0.230) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
Test: [70/79] Time 0.263 (0.244) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
* Acc@1 60.740 Acc@5 83.960 Total time: 18.038
|
* Acc@1 60.740 Acc@5 83.960 Total time: 19.092
|
||||||
Test accuracy of FP32 model: 60.740
|
Test accuracy of FP32 model: 60.740
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -557,14 +554,14 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:57:45.269741: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 22:53:51.860179: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 22:57:45.300230: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 22:53:51.891244: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 22:57:45.813067: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 22:53:52.407039: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -584,13 +581,15 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -611,13 +610,15 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||||
|
</pre>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -634,48 +635,48 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [ 0/79] Time 0.430 (0.430) Acc@1 81.25 (81.25) Acc@5 89.84 (89.84)
|
Test: [ 0/79] Time 0.435 (0.435) Acc@1 82.81 (82.81) Acc@5 92.19 (92.19)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [10/79] Time 0.392 (0.397) Acc@1 56.25 (66.26) Acc@5 85.16 (87.14)
|
Test: [10/79] Time 0.391 (0.395) Acc@1 54.69 (66.34) Acc@5 85.94 (87.50)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [20/79] Time 0.393 (0.395) Acc@1 68.75 (63.80) Acc@5 84.38 (86.98)
|
Test: [20/79] Time 0.389 (0.395) Acc@1 69.53 (63.91) Acc@5 84.38 (87.09)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [30/79] Time 0.393 (0.396) Acc@1 52.34 (62.15) Acc@5 75.78 (85.01)
|
Test: [30/79] Time 0.388 (0.395) Acc@1 52.34 (62.22) Acc@5 75.78 (84.90)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [40/79] Time 0.392 (0.395) Acc@1 67.19 (60.75) Acc@5 89.84 (84.26)
|
Test: [40/79] Time 0.392 (0.393) Acc@1 67.97 (60.75) Acc@5 89.84 (84.30)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [50/79] Time 0.392 (0.395) Acc@1 57.81 (60.68) Acc@5 88.28 (84.15)
|
Test: [50/79] Time 0.398 (0.393) Acc@1 60.16 (60.72) Acc@5 88.28 (84.30)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [60/79] Time 0.393 (0.394) Acc@1 64.06 (60.32) Acc@5 85.94 (83.58)
|
Test: [60/79] Time 0.390 (0.393) Acc@1 66.41 (60.27) Acc@5 86.72 (83.75)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [70/79] Time 0.392 (0.394) Acc@1 54.69 (60.17) Acc@5 78.12 (83.08)
|
Test: [70/79] Time 0.388 (0.392) Acc@1 54.69 (60.06) Acc@5 80.47 (83.29)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
* Acc@1 60.690 Acc@5 83.740 Total time: 30.882
|
* Acc@1 60.570 Acc@5 83.950 Total time: 30.736
|
||||||
Accuracy of initialized INT8 model: 60.690
|
Accuracy of initialized INT8 model: 60.570
|
||||||
|
|
||||||
|
|
||||||
It should be noted that the inference time for the quantized PyTorch
|
It should be noted that the inference time for the quantized PyTorch
|
||||||
|
|
@ -724,20 +725,20 @@ For more information about model conversion, refer to this
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
return self._level_low.item()
|
return self._level_low.item()
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
return self._level_high.item()
|
return self._level_high.item()
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||||
Tensor-likes are not close!
|
Tensor-likes are not close!
|
||||||
|
|
||||||
Mismatched elements: 25573 / 25600 (99.9%)
|
Mismatched elements: 25587 / 25600 (99.9%)
|
||||||
Greatest absolute difference: 0.5424436330795288 at index (1, 149) (up to 1e-05 allowed)
|
Greatest absolute difference: 0.5083470344543457 at index (42, 14) (up to 1e-05 allowed)
|
||||||
Greatest relative difference: 42.99047422811133 at index (90, 158) (up to 1e-05 allowed)
|
Greatest relative difference: 79.27243410505909 at index (126, 158) (up to 1e-05 allowed)
|
||||||
_check_trace(
|
_check_trace(
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -778,47 +779,47 @@ Evaluate the FP32 and INT8 models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [ 0/79] Time 0.184 (0.184) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
Test: [ 0/79] Time 0.185 (0.185) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [10/79] Time 0.139 (0.143) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
Test: [10/79] Time 0.136 (0.143) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [20/79] Time 0.139 (0.141) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
Test: [20/79] Time 0.141 (0.141) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [30/79] Time 0.139 (0.140) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
Test: [30/79] Time 0.140 (0.140) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [40/79] Time 0.135 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
Test: [40/79] Time 0.140 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [50/79] Time 0.139 (0.140) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
Test: [50/79] Time 0.139 (0.139) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [60/79] Time 0.139 (0.139) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
Test: [60/79] Time 0.138 (0.139) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [70/79] Time 0.138 (0.139) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
Test: [70/79] Time 0.136 (0.139) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
* Acc@1 60.740 Acc@5 83.960 Total time: 10.882
|
* Acc@1 60.740 Acc@5 83.960 Total time: 10.886
|
||||||
Accuracy of FP32 IR model: 60.740
|
Accuracy of FP32 IR model: 60.740
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -831,48 +832,48 @@ Evaluate the FP32 and INT8 models.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [ 0/79] Time 0.133 (0.133) Acc@1 81.25 (81.25) Acc@5 91.41 (91.41)
|
Test: [ 0/79] Time 0.145 (0.145) Acc@1 82.03 (82.03) Acc@5 91.41 (91.41)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [10/79] Time 0.076 (0.084) Acc@1 54.69 (66.41) Acc@5 85.94 (87.64)
|
Test: [10/79] Time 0.076 (0.083) Acc@1 55.47 (66.76) Acc@5 86.72 (87.36)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [20/79] Time 0.077 (0.080) Acc@1 71.09 (64.10) Acc@5 84.38 (87.05)
|
Test: [20/79] Time 0.076 (0.080) Acc@1 70.31 (64.43) Acc@5 85.16 (87.02)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [30/79] Time 0.075 (0.079) Acc@1 52.34 (62.17) Acc@5 75.00 (84.98)
|
Test: [30/79] Time 0.076 (0.079) Acc@1 53.12 (62.40) Acc@5 75.78 (84.93)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [40/79] Time 0.077 (0.079) Acc@1 67.19 (60.67) Acc@5 89.84 (84.22)
|
Test: [40/79] Time 0.077 (0.078) Acc@1 67.19 (60.84) Acc@5 90.62 (84.20)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [50/79] Time 0.077 (0.078) Acc@1 60.16 (60.63) Acc@5 88.28 (84.22)
|
Test: [50/79] Time 0.078 (0.078) Acc@1 59.38 (60.83) Acc@5 88.28 (84.15)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [60/79] Time 0.076 (0.078) Acc@1 65.62 (60.31) Acc@5 86.72 (83.67)
|
Test: [60/79] Time 0.076 (0.078) Acc@1 64.84 (60.40) Acc@5 87.50 (83.63)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Test: [70/79] Time 0.076 (0.077) Acc@1 53.12 (60.01) Acc@5 78.91 (83.20)
|
Test: [70/79] Time 0.077 (0.078) Acc@1 53.12 (60.16) Acc@5 80.47 (83.14)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
* Acc@1 60.540 Acc@5 83.840 Total time: 6.060
|
* Acc@1 60.680 Acc@5 83.770 Total time: 6.075
|
||||||
Accuracy of INT8 IR model: 60.540
|
Accuracy of INT8 IR model: 60.680
|
||||||
|
|
||||||
|
|
||||||
IV. Compare performance of INT8 model and FP32 model in OpenVINO
|
IV. Compare performance of INT8 model and FP32 model in OpenVINO
|
||||||
|
|
@ -941,25 +942,25 @@ throughput (frames per second) values.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Throughput: 38.82 FPS
|
[ INFO ] Throughput: 38.33 FPS
|
||||||
Benchmark INT8 model (OpenVINO IR)
|
Benchmark INT8 model (OpenVINO IR)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Throughput: 157.27 FPS
|
[ INFO ] Throughput: 155.58 FPS
|
||||||
Benchmark FP32 model (OpenVINO IR) synchronously
|
Benchmark FP32 model (OpenVINO IR) synchronously
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Throughput: 40.19 FPS
|
[ INFO ] Throughput: 39.95 FPS
|
||||||
Benchmark INT8 model (OpenVINO IR) synchronously
|
Benchmark INT8 model (OpenVINO IR) synchronously
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Throughput: 137.35 FPS
|
[ INFO ] Throughput: 137.77 FPS
|
||||||
|
|
||||||
|
|
||||||
Show device Information for reference:
|
Show device Information for reference:
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/113-image-classification-quantization-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/113-image-classification-quantization-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="113-image-classification-quantization-with-output_30_5.png">113-image-classification-quantization-with-outp..></a> 26-Jan-2024 01:05 14855
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -122,24 +122,20 @@ each frame of the video.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 12%, 32 KB, 1452 KB/s, 0 seconds passed
|
... 12%, 32 KB, 1341 KB/s, 0 seconds passed
|
||||||
|
... 25%, 64 KB, 1341 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 25%, 64 KB, 1385 KB/s, 0 seconds passed
|
... 38%, 96 KB, 1143 KB/s, 0 seconds passed
|
||||||
... 38%, 96 KB, 1632 KB/s, 0 seconds passed
|
... 51%, 128 KB, 1326 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 51%, 128 KB, 1424 KB/s, 0 seconds passed
|
... 64%, 160 KB, 1323 KB/s, 0 seconds passed
|
||||||
... 64%, 160 KB, 1393 KB/s, 0 seconds passed
|
... 77%, 192 KB, 1444 KB/s, 0 seconds passed
|
||||||
... 77%, 192 KB, 1664 KB/s, 0 seconds passed
|
... 89%, 224 KB, 1678 KB/s, 0 seconds passed
|
||||||
... 89%, 224 KB, 1795 KB/s, 0 seconds passed
|
... 100%, 248 KB, 1859 KB/s, 0 seconds passed
|
||||||
... 100%, 248 KB, 1986 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
|
|
||||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
|
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
|
||||||
|
|
@ -147,174 +143,177 @@ each frame of the video.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 0%, 32 KB, 1284 KB/s, 0 seconds passed
|
... 0%, 32 KB, 1301 KB/s, 0 seconds passed
|
||||||
... 1%, 64 KB, 1284 KB/s, 0 seconds passed
|
|
||||||
... 2%, 96 KB, 1894 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 3%, 128 KB, 1699 KB/s, 0 seconds passed
|
... 1%, 64 KB, 1298 KB/s, 0 seconds passed
|
||||||
... 4%, 160 KB, 2092 KB/s, 0 seconds passed
|
... 2%, 96 KB, 1907 KB/s, 0 seconds passed
|
||||||
... 5%, 192 KB, 2274 KB/s, 0 seconds passed
|
... 3%, 128 KB, 1722 KB/s, 0 seconds passed
|
||||||
... 6%, 224 KB, 2623 KB/s, 0 seconds passed
|
... 4%, 160 KB, 2133 KB/s, 0 seconds passed
|
||||||
... 7%, 256 KB, 2328 KB/s, 0 seconds passed
|
|
||||||
... 8%, 288 KB, 2599 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 9%, 320 KB, 2693 KB/s, 0 seconds passed
|
... 5%, 192 KB, 2234 KB/s, 0 seconds passed
|
||||||
... 9%, 352 KB, 2939 KB/s, 0 seconds passed
|
... 6%, 224 KB, 2577 KB/s, 0 seconds passed
|
||||||
... 10%, 384 KB, 2655 KB/s, 0 seconds passed
|
... 7%, 256 KB, 2312 KB/s, 0 seconds passed
|
||||||
... 11%, 416 KB, 2858 KB/s, 0 seconds passed
|
... 8%, 288 KB, 2581 KB/s, 0 seconds passed
|
||||||
... 12%, 448 KB, 2918 KB/s, 0 seconds passed
|
... 9%, 320 KB, 2614 KB/s, 0 seconds passed
|
||||||
... 13%, 480 KB, 3100 KB/s, 0 seconds passed
|
... 9%, 352 KB, 2855 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 14%, 512 KB, 2858 KB/s, 0 seconds passed
|
... 10%, 384 KB, 2608 KB/s, 0 seconds passed
|
||||||
... 15%, 544 KB, 2904 KB/s, 0 seconds passed
|
... 11%, 416 KB, 2811 KB/s, 0 seconds passed
|
||||||
... 16%, 576 KB, 3065 KB/s, 0 seconds passed
|
... 12%, 448 KB, 2817 KB/s, 0 seconds passed
|
||||||
... 17%, 608 KB, 3219 KB/s, 0 seconds passed
|
... 13%, 480 KB, 2999 KB/s, 0 seconds passed
|
||||||
... 18%, 640 KB, 2998 KB/s, 0 seconds passed
|
... 14%, 512 KB, 2785 KB/s, 0 seconds passed
|
||||||
... 18%, 672 KB, 3133 KB/s, 0 seconds passed
|
... 15%, 544 KB, 2949 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 19%, 704 KB, 3164 KB/s, 0 seconds passed
|
... 16%, 576 KB, 2947 KB/s, 0 seconds passed
|
||||||
... 20%, 736 KB, 3293 KB/s, 0 seconds passed
|
... 17%, 608 KB, 3097 KB/s, 0 seconds passed
|
||||||
... 21%, 768 KB, 3098 KB/s, 0 seconds passed
|
... 18%, 640 KB, 2903 KB/s, 0 seconds passed
|
||||||
... 22%, 800 KB, 3120 KB/s, 0 seconds passed
|
... 18%, 672 KB, 3035 KB/s, 0 seconds passed
|
||||||
... 23%, 832 KB, 3228 KB/s, 0 seconds passed
|
... 19%, 704 KB, 3035 KB/s, 0 seconds passed
|
||||||
|
... 20%, 736 KB, 3158 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 24%, 864 KB, 3062 KB/s, 0 seconds passed
|
... 21%, 768 KB, 2990 KB/s, 0 seconds passed
|
||||||
... 25%, 896 KB, 3170 KB/s, 0 seconds passed
|
... 22%, 800 KB, 3107 KB/s, 0 seconds passed
|
||||||
... 26%, 928 KB, 3274 KB/s, 0 seconds passed
|
... 23%, 832 KB, 3100 KB/s, 0 seconds passed
|
||||||
... 27%, 960 KB, 3291 KB/s, 0 seconds passed
|
... 24%, 864 KB, 3209 KB/s, 0 seconds passed
|
||||||
... 27%, 992 KB, 3388 KB/s, 0 seconds passed
|
|
||||||
... 28%, 1024 KB, 3231 KB/s, 0 seconds passed
|
|
||||||
... 29%, 1056 KB, 3322 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 30%, 1088 KB, 3331 KB/s, 0 seconds passed
|
... 25%, 896 KB, 3055 KB/s, 0 seconds passed
|
||||||
... 31%, 1120 KB, 3189 KB/s, 0 seconds passed
|
... 26%, 928 KB, 3153 KB/s, 0 seconds passed
|
||||||
... 32%, 1152 KB, 3275 KB/s, 0 seconds passed
|
... 27%, 960 KB, 3148 KB/s, 0 seconds passed
|
||||||
... 33%, 1184 KB, 3360 KB/s, 0 seconds passed
|
... 27%, 992 KB, 3242 KB/s, 0 seconds passed
|
||||||
... 34%, 1216 KB, 3370 KB/s, 0 seconds passed
|
... 28%, 1024 KB, 3104 KB/s, 0 seconds passed
|
||||||
... 35%, 1248 KB, 3448 KB/s, 0 seconds passed
|
... 29%, 1056 KB, 3191 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 36%, 1280 KB, 3316 KB/s, 0 seconds passed
|
... 30%, 1088 KB, 3186 KB/s, 0 seconds passed
|
||||||
... 36%, 1312 KB, 3390 KB/s, 0 seconds passed
|
... 31%, 1120 KB, 3271 KB/s, 0 seconds passed
|
||||||
... 37%, 1344 KB, 3401 KB/s, 0 seconds passed
|
... 32%, 1152 KB, 3144 KB/s, 0 seconds passed
|
||||||
... 38%, 1376 KB, 3472 KB/s, 0 seconds passed
|
... 33%, 1184 KB, 3225 KB/s, 0 seconds passed
|
||||||
... 39%, 1408 KB, 3349 KB/s, 0 seconds passed
|
... 34%, 1216 KB, 3217 KB/s, 0 seconds passed
|
||||||
... 40%, 1440 KB, 3418 KB/s, 0 seconds passed
|
... 35%, 1248 KB, 3293 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 41%, 1472 KB, 3426 KB/s, 0 seconds passed
|
... 36%, 1280 KB, 3177 KB/s, 0 seconds passed
|
||||||
... 42%, 1504 KB, 3488 KB/s, 0 seconds passed
|
... 36%, 1312 KB, 3250 KB/s, 0 seconds passed
|
||||||
... 43%, 1536 KB, 3372 KB/s, 0 seconds passed
|
... 37%, 1344 KB, 3243 KB/s, 0 seconds passed
|
||||||
... 44%, 1568 KB, 3437 KB/s, 0 seconds passed
|
... 38%, 1376 KB, 3311 KB/s, 0 seconds passed
|
||||||
... 45%, 1600 KB, 3446 KB/s, 0 seconds passed
|
... 39%, 1408 KB, 3204 KB/s, 0 seconds passed
|
||||||
... 45%, 1632 KB, 3505 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 46%, 1664 KB, 3400 KB/s, 0 seconds passed
|
... 40%, 1440 KB, 3198 KB/s, 0 seconds passed
|
||||||
... 47%, 1696 KB, 3457 KB/s, 0 seconds passed
|
... 41%, 1472 KB, 3260 KB/s, 0 seconds passed
|
||||||
... 48%, 1728 KB, 3468 KB/s, 0 seconds passed
|
... 42%, 1504 KB, 3324 KB/s, 0 seconds passed
|
||||||
... 49%, 1760 KB, 3520 KB/s, 0 seconds passed
|
... 43%, 1536 KB, 3225 KB/s, 0 seconds passed
|
||||||
... 50%, 1792 KB, 3417 KB/s, 0 seconds passed
|
... 44%, 1568 KB, 3218 KB/s, 0 seconds passed
|
||||||
... 51%, 1824 KB, 3472 KB/s, 0 seconds passed
|
... 45%, 1600 KB, 3279 KB/s, 0 seconds passed
|
||||||
|
... 45%, 1632 KB, 3339 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 52%, 1856 KB, 3478 KB/s, 0 seconds passed
|
... 46%, 1664 KB, 3246 KB/s, 0 seconds passed
|
||||||
... 53%, 1888 KB, 3383 KB/s, 0 seconds passed
|
... 47%, 1696 KB, 3234 KB/s, 0 seconds passed
|
||||||
... 54%, 1920 KB, 3437 KB/s, 0 seconds passed
|
... 48%, 1728 KB, 3291 KB/s, 0 seconds passed
|
||||||
... 54%, 1952 KB, 3487 KB/s, 0 seconds passed
|
... 49%, 1760 KB, 3349 KB/s, 0 seconds passed
|
||||||
... 55%, 1984 KB, 3495 KB/s, 0 seconds passed
|
|
||||||
... 56%, 2016 KB, 3544 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 57%, 2048 KB, 3451 KB/s, 0 seconds passed
|
... 50%, 1792 KB, 3263 KB/s, 0 seconds passed
|
||||||
... 58%, 2080 KB, 3500 KB/s, 0 seconds passed
|
... 51%, 1824 KB, 3249 KB/s, 0 seconds passed
|
||||||
... 59%, 2112 KB, 3506 KB/s, 0 seconds passed
|
... 52%, 1856 KB, 3303 KB/s, 0 seconds passed
|
||||||
... 60%, 2144 KB, 3554 KB/s, 0 seconds passed
|
... 53%, 1888 KB, 3357 KB/s, 0 seconds passed
|
||||||
... 61%, 2176 KB, 3466 KB/s, 0 seconds passed
|
... 54%, 1920 KB, 3280 KB/s, 0 seconds passed
|
||||||
|
... 54%, 1952 KB, 3328 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 62%, 2208 KB, 3510 KB/s, 0 seconds passed
|
... 55%, 1984 KB, 3314 KB/s, 0 seconds passed
|
||||||
... 63%, 2240 KB, 3513 KB/s, 0 seconds passed
|
... 56%, 2016 KB, 3365 KB/s, 0 seconds passed
|
||||||
... 64%, 2272 KB, 3561 KB/s, 0 seconds passed
|
... 57%, 2048 KB, 3294 KB/s, 0 seconds passed
|
||||||
... 64%, 2304 KB, 3478 KB/s, 0 seconds passed
|
... 58%, 2080 KB, 3337 KB/s, 0 seconds passed
|
||||||
... 65%, 2336 KB, 3518 KB/s, 0 seconds passed
|
... 59%, 2112 KB, 3325 KB/s, 0 seconds passed
|
||||||
... 66%, 2368 KB, 3521 KB/s, 0 seconds passed
|
... 60%, 2144 KB, 3372 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 67%, 2400 KB, 3444 KB/s, 0 seconds passed
|
... 61%, 2176 KB, 3306 KB/s, 0 seconds passed
|
||||||
... 68%, 2432 KB, 3486 KB/s, 0 seconds passed
|
... 62%, 2208 KB, 3345 KB/s, 0 seconds passed
|
||||||
... 69%, 2464 KB, 3495 KB/s, 0 seconds passed
|
... 63%, 2240 KB, 3335 KB/s, 0 seconds passed
|
||||||
... 70%, 2496 KB, 3528 KB/s, 0 seconds passed
|
... 64%, 2272 KB, 3379 KB/s, 0 seconds passed
|
||||||
|
... 64%, 2304 KB, 3316 KB/s, 0 seconds passed
|
||||||
|
... 65%, 2336 KB, 3351 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 71%, 2528 KB, 3454 KB/s, 0 seconds passed
|
... 66%, 2368 KB, 3343 KB/s, 0 seconds passed
|
||||||
... 72%, 2560 KB, 3496 KB/s, 0 seconds passed
|
... 67%, 2400 KB, 3385 KB/s, 0 seconds passed
|
||||||
... 73%, 2592 KB, 3505 KB/s, 0 seconds passed
|
... 68%, 2432 KB, 3322 KB/s, 0 seconds passed
|
||||||
... 73%, 2624 KB, 3515 KB/s, 0 seconds passed
|
... 69%, 2464 KB, 3312 KB/s, 0 seconds passed
|
||||||
... 74%, 2656 KB, 3463 KB/s, 0 seconds passed
|
... 70%, 2496 KB, 3349 KB/s, 0 seconds passed
|
||||||
... 75%, 2688 KB, 3502 KB/s, 0 seconds passed
|
... 71%, 2528 KB, 3390 KB/s, 0 seconds passed
|
||||||
... 76%, 2720 KB, 3514 KB/s, 0 seconds passed
|
|
||||||
... 77%, 2752 KB, 3523 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 78%, 2784 KB, 3474 KB/s, 0 seconds passed
|
... 72%, 2560 KB, 3331 KB/s, 0 seconds passed
|
||||||
... 79%, 2816 KB, 3510 KB/s, 0 seconds passed
|
... 73%, 2592 KB, 3322 KB/s, 0 seconds passed
|
||||||
... 80%, 2848 KB, 3522 KB/s, 0 seconds passed
|
... 73%, 2624 KB, 3356 KB/s, 0 seconds passed
|
||||||
... 81%, 2880 KB, 3530 KB/s, 0 seconds passed
|
... 74%, 2656 KB, 3395 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 82%, 2912 KB, 3482 KB/s, 0 seconds passed
|
... 75%, 2688 KB, 3338 KB/s, 0 seconds passed
|
||||||
... 82%, 2944 KB, 3518 KB/s, 0 seconds passed
|
... 76%, 2720 KB, 3328 KB/s, 0 seconds passed
|
||||||
... 83%, 2976 KB, 3529 KB/s, 0 seconds passed
|
... 77%, 2752 KB, 3363 KB/s, 0 seconds passed
|
||||||
... 84%, 3008 KB, 3535 KB/s, 0 seconds passed
|
... 78%, 2784 KB, 3399 KB/s, 0 seconds passed
|
||||||
... 85%, 3040 KB, 3489 KB/s, 0 seconds passed
|
... 79%, 2816 KB, 3344 KB/s, 0 seconds passed
|
||||||
... 86%, 3072 KB, 3523 KB/s, 0 seconds passed
|
... 80%, 2848 KB, 3335 KB/s, 0 seconds passed
|
||||||
... 87%, 3104 KB, 3532 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 88%, 3136 KB, 3540 KB/s, 0 seconds passed
|
... 81%, 2880 KB, 3369 KB/s, 0 seconds passed
|
||||||
... 89%, 3168 KB, 3498 KB/s, 0 seconds passed
|
... 82%, 2912 KB, 3321 KB/s, 0 seconds passed
|
||||||
... 90%, 3200 KB, 3530 KB/s, 0 seconds passed
|
... 82%, 2944 KB, 3352 KB/s, 0 seconds passed
|
||||||
... 91%, 3232 KB, 3539 KB/s, 0 seconds passed
|
... 83%, 2976 KB, 3342 KB/s, 0 seconds passed
|
||||||
... 91%, 3264 KB, 3544 KB/s, 0 seconds passed
|
... 84%, 3008 KB, 3374 KB/s, 0 seconds passed
|
||||||
|
... 85%, 3040 KB, 3407 KB/s, 0 seconds passed
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 92%, 3296 KB, 3504 KB/s, 0 seconds passed
|
... 86%, 3072 KB, 3360 KB/s, 0 seconds passed
|
||||||
... 93%, 3328 KB, 3536 KB/s, 0 seconds passed
|
... 87%, 3104 KB, 3351 KB/s, 0 seconds passed
|
||||||
... 94%, 3360 KB, 3544 KB/s, 0 seconds passed
|
... 88%, 3136 KB, 3380 KB/s, 0 seconds passed
|
||||||
... 95%, 3392 KB, 3550 KB/s, 0 seconds passed
|
... 89%, 3168 KB, 3411 KB/s, 0 seconds passed
|
||||||
... 96%, 3424 KB, 3509 KB/s, 0 seconds passed
|
... 90%, 3200 KB, 3364 KB/s, 0 seconds passed
|
||||||
... 97%, 3456 KB, 3540 KB/s, 0 seconds passed
|
|
||||||
... 98%, 3488 KB, 3548 KB/s, 0 seconds passed
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
... 99%, 3520 KB, 3560 KB/s, 0 seconds passed
|
... 91%, 3232 KB, 3357 KB/s, 0 seconds passed
|
||||||
... 100%, 3549 KB, 3521 KB/s, 1 seconds passed
|
... 91%, 3264 KB, 3384 KB/s, 0 seconds passed
|
||||||
|
... 92%, 3296 KB, 3342 KB/s, 0 seconds passed
|
||||||
|
... 93%, 3328 KB, 3368 KB/s, 0 seconds passed
|
||||||
|
... 94%, 3360 KB, 3358 KB/s, 1 seconds passed
|
||||||
|
... 95%, 3392 KB, 3388 KB/s, 1 seconds passed
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
... 96%, 3424 KB, 3346 KB/s, 1 seconds passed
|
||||||
|
... 97%, 3456 KB, 3372 KB/s, 1 seconds passed
|
||||||
|
... 98%, 3488 KB, 3363 KB/s, 1 seconds passed
|
||||||
|
... 99%, 3520 KB, 3392 KB/s, 1 seconds passed
|
||||||
|
... 100%, 3549 KB, 3419 KB/s, 1 seconds passed
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -512,7 +511,7 @@ Test performance in Sync Mode
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Source ended
|
Source ended
|
||||||
average throuput in sync mode: 43.51 fps
|
average throuput in sync mode: 47.04 fps
|
||||||
|
|
||||||
|
|
||||||
Async Mode
|
Async Mode
|
||||||
|
|
@ -651,7 +650,7 @@ Test the performance in Async Mode
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Source ended
|
Source ended
|
||||||
average throuput in async mode: 74.01 fps
|
average throuput in async mode: 74.61 fps
|
||||||
|
|
||||||
|
|
||||||
Compare the performance
|
Compare the performance
|
||||||
|
|
@ -794,5 +793,5 @@ Test the performance with ``AsyncInferQueue``
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
average throughput in async mode with async infer queue: 110.03 fps
|
average throughput in async mode with async infer queue: 113.01 fps
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:a6ef5964af038dfaf0c794dfc9a36258293ea7a6f8318f5f84490e8953a10479
|
oid sha256:8c51610ffd478d0bc5358648e0475259dc8eb04d090c6f55cb860d2e9b8a8cb6
|
||||||
size 30402
|
size 30356
|
||||||
|
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/115-async-api-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/115-async-api-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="115-async-api-with-output_15_0.png">115-async-api-with-output_15_0.png</a> 26-Jan-2024 01:05 4307
|
|
||||||
<a href="115-async-api-with-output_19_0.png">115-async-api-with-output_19_0.png</a> 26-Jan-2024 01:05 4307
|
|
||||||
<a href="115-async-api-with-output_21_0.png">115-async-api-with-output_21_0.png</a> 26-Jan-2024 01:05 30402
|
|
||||||
<a href="115-async-api-with-output_27_0.png">115-async-api-with-output_27_0.png</a> 26-Jan-2024 01:05 4307
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -85,14 +85,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:06:01.802535: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 23:02:05.779349: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 23:06:01.837209: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 23:02:05.814537: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:06:02.398511: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:02:06.378496: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
Download, quantize and sparsify the model, using Hugging Face Optimum API
|
Download, quantize and sparsify the model, using Hugging Face Optimum API
|
||||||
|
|
@ -228,7 +228,7 @@ as an example. It is recommended to tune based on your applications.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Read model took 60.22 ms
|
[ INFO ] Read model took 62.38 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
||||||
|
|
@ -239,7 +239,7 @@ as an example. It is recommended to tune based on your applications.
|
||||||
[Step 5/11] Resizing model to match image sizes and given batch
|
[Step 5/11] Resizing model to match image sizes and given batch
|
||||||
[ INFO ] Model batch size: 1
|
[ INFO ] Model batch size: 1
|
||||||
[ INFO ] Reshaping model: 'input_ids': [1,64], 'attention_mask': [1,64], 'token_type_ids': [1,64]
|
[ INFO ] Reshaping model: 'input_ids': [1,64], 'attention_mask': [1,64], 'token_type_ids': [1,64]
|
||||||
[ INFO ] Reshape model took 23.06 ms
|
[ INFO ] Reshape model took 23.14 ms
|
||||||
[Step 6/11] Configuring input of the model
|
[Step 6/11] Configuring input of the model
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
||||||
|
|
@ -252,7 +252,7 @@ as an example. It is recommended to tune based on your applications.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 1064.23 ms
|
[ INFO ] Compile model took 1107.64 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: torch_jit
|
[ INFO ] NETWORK_NAME: torch_jit
|
||||||
|
|
@ -284,21 +284,21 @@ as an example. It is recommended to tune based on your applications.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 27.61 ms
|
[ INFO ] First inference took 30.14 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 8900 iterations
|
[ INFO ] Count: 8852 iterations
|
||||||
[ INFO ] Duration: 60039.72 ms
|
[ INFO ] Duration: 60038.32 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 26.68 ms
|
[ INFO ] Median: 26.79 ms
|
||||||
[ INFO ] Average: 26.74 ms
|
[ INFO ] Average: 26.86 ms
|
||||||
[ INFO ] Min: 25.09 ms
|
[ INFO ] Min: 24.76 ms
|
||||||
[ INFO ] Max: 39.62 ms
|
[ INFO ] Max: 42.20 ms
|
||||||
[ INFO ] Throughput: 148.24 FPS
|
[ INFO ] Throughput: 147.44 FPS
|
||||||
|
|
||||||
|
|
||||||
Benchmark quantized sparse inference performance
|
Benchmark quantized sparse inference performance
|
||||||
|
|
@ -363,7 +363,7 @@ for which a layer will be enabled.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Read model took 67.79 ms
|
[ INFO ] Read model took 71.12 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
||||||
|
|
@ -378,7 +378,7 @@ for which a layer will be enabled.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Reshape model took 23.92 ms
|
[ INFO ] Reshape model took 23.54 ms
|
||||||
[Step 6/11] Configuring input of the model
|
[Step 6/11] Configuring input of the model
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
||||||
|
|
@ -392,9 +392,9 @@ for which a layer will be enabled.
|
||||||
Wrong value for property key CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE. Expected only float numbers
|
Wrong value for property key CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE. Expected only float numbers
|
||||||
|
|
||||||
Traceback (most recent call last):
|
Traceback (most recent call last):
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 408, in main
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 408, in main
|
||||||
compiled_model = benchmark.core.compile_model(model, benchmark.device, device_config)
|
compiled_model = benchmark.core.compile_model(model, benchmark.device, device_config)
|
||||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/runtime/ie_api.py", line 547, in compile_model
|
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/runtime/ie_api.py", line 547, in compile_model
|
||||||
super().compile_model(model, device_name, {} if config is None else config),
|
super().compile_model(model, device_name, {} if config is None else config),
|
||||||
RuntimeError: Exception from src/inference/src/core.cpp:99:
|
RuntimeError: Exception from src/inference/src/core.cpp:99:
|
||||||
[ GENERAL_ERROR ] Exception from src/plugins/intel_cpu/src/config.cpp:158:
|
[ GENERAL_ERROR ] Exception from src/plugins/intel_cpu/src/config.cpp:158:
|
||||||
|
|
|
||||||
|
|
@ -269,7 +269,7 @@ Check whether the OVMS container is running normally:
|
||||||
|
|
||||||
The required Model Server parameters are listed below. For additional
|
The required Model Server parameters are listed below. For additional
|
||||||
configuration options, see the `Model Server Parameters
|
configuration options, see the `Model Server Parameters
|
||||||
section <https://docs.openvino.ai/2023.2/ovms_docs_parameters.html>`__.
|
section <https://docs.openvino.ai/2023.3/ovms_docs_parameters.html>`__.
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
|
|
@ -928,6 +928,6 @@ References
|
||||||
|
|
||||||
|
|
||||||
1. `OpenVINO™ Model Server
|
1. `OpenVINO™ Model Server
|
||||||
documentation <https://docs.openvino.ai/2023.0/ovms_what_is_openvino_model_server.html>`__
|
documentation <https://docs.openvino.ai/2023.3/ovms_what_is_openvino_model_server.html>`__
|
||||||
2. `OpenVINO™ Model Server GitHub
|
2. `OpenVINO™ Model Server GitHub
|
||||||
repository <https://github.com/openvinotoolkit/model_server/>`__
|
repository <https://github.com/openvinotoolkit/model_server/>`__
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/117-model-server-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/117-model-server-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="117-model-server-with-output_23_2.png">117-model-server-with-output_23_2.png</a> 26-Jan-2024 01:04 305482
|
|
||||||
<a href="117-model-server-with-output_28_1.png">117-model-server-with-output_28_1.png</a> 26-Jan-2024 01:04 458294
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -106,14 +106,14 @@ Imports
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:07:18.341652: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 23:03:22.538807: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 23:07:18.375926: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 23:03:22.573455: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:07:18.888977: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:03:23.087260: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
Setup image and device
|
Setup image and device
|
||||||
|
|
@ -194,12 +194,12 @@ and save it to the disk.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:07:22.553946: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
2024-02-09 23:03:26.685646: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||||
2024-01-25 23:07:22.553985: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
2024-02-09 23:03:26.685683: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||||
2024-01-25 23:07:22.553989: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
2024-02-09 23:03:26.685688: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||||
2024-01-25 23:07:22.554129: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
2024-02-09 23:03:26.685821: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||||
2024-01-25 23:07:22.554144: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
2024-02-09 23:03:26.685836: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||||
2024-01-25 23:07:22.554147: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
2024-02-09 23:03:26.685841: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -360,7 +360,7 @@ for mean/scale normalization.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7f5ce40abdb0>
|
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7ff8b271c1b0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -391,7 +391,7 @@ may be specified is input data
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7f5ce40abb70>
|
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7ff7b1f9dd70>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -432,7 +432,7 @@ then such conversion will be added explicitly.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7f5ce40abe70>
|
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7ff7b1f9d230>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -651,10 +651,10 @@ Compare performance
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0149 seconds per image, FPS: 67.09
|
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0153 seconds per image, FPS: 65.53
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0182 seconds per image, FPS: 54.89
|
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0183 seconds per image, FPS: 54.77
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/118-optimize-preprocessing-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/118-optimize-preprocessing-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="118-optimize-preprocessing-with-output_14_1.png">118-optimize-preprocessing-with-output_14_1.png</a> 26-Jan-2024 01:04 387941
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -106,7 +106,7 @@ Download TFLite model
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/119-tflite-to-openvino/model/efficientnet_lite0_fp32_2.tflite')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/119-tflite-to-openvino/model/efficientnet_lite0_fp32_2.tflite')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -281,7 +281,11 @@ GPU.
|
||||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
|
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||||
[Step 4/11] Reading model files
|
[Step 4/11] Reading model files
|
||||||
[ INFO ] Loading model files
|
[ INFO ] Loading model files
|
||||||
[ INFO ] Read model took 10.06 ms
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
[ INFO ] Read model took 21.35 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] images (node: images) : f32 / [...] / [1,224,224,3]
|
[ INFO ] images (node: images) : f32 / [...] / [1,224,224,3]
|
||||||
|
|
@ -299,9 +303,13 @@ GPU.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 138.70 ms
|
[ INFO ] Compile model took 147.38 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] NETWORK_NAME: TensorFlow_Lite_Frontend_IR
|
[ INFO ] NETWORK_NAME: TensorFlow_Lite_Frontend_IR
|
||||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||||
[ INFO ] NUM_STREAMS: 6
|
[ INFO ] NUM_STREAMS: 6
|
||||||
|
|
@ -323,19 +331,19 @@ GPU.
|
||||||
[ INFO ] Fill input 'images' with random values
|
[ INFO ] Fill input 'images' with random values
|
||||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
||||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||||
[ INFO ] First inference took 7.39 ms
|
[ INFO ] First inference took 7.28 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 17544 iterations
|
[ INFO ] Count: 17502 iterations
|
||||||
[ INFO ] Duration: 15006.83 ms
|
[ INFO ] Duration: 15007.36 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 5.01 ms
|
[ INFO ] Median: 5.02 ms
|
||||||
[ INFO ] Average: 5.00 ms
|
[ INFO ] Average: 5.02 ms
|
||||||
[ INFO ] Min: 2.90 ms
|
[ INFO ] Min: 3.02 ms
|
||||||
[ INFO ] Max: 14.27 ms
|
[ INFO ] Max: 13.94 ms
|
||||||
[ INFO ] Throughput: 1169.07 FPS
|
[ INFO ] Throughput: 1166.23 FPS
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/119-tflite-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/119-tflite-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="119-tflite-to-openvino-with-output_16_1.jpg">119-tflite-to-openvino-with-output_16_1.jpg</a> 26-Jan-2024 01:05 68170
|
|
||||||
<a href="119-tflite-to-openvino-with-output_16_1.png">119-tflite-to-openvino-with-output_16_1.png</a> 26-Jan-2024 01:05 621006
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -24,23 +24,32 @@ After creating the OpenVINO IR, load the model in `OpenVINO
|
||||||
Runtime <https://docs.openvino.ai/nightly/openvino_docs_OV_UG_OV_Runtime_User_Guide.html>`__
|
Runtime <https://docs.openvino.ai/nightly/openvino_docs_OV_UG_OV_Runtime_User_Guide.html>`__
|
||||||
and do inference with a sample image.
|
and do inference with a sample image.
|
||||||
|
|
||||||
**Table of contents:**
|
Table of contents:
|
||||||
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
- `Prerequisites <#prerequisites>`__
|
- `Prerequisites <#prerequisites>`__
|
||||||
- `Imports <#imports>`__
|
- `Imports <#imports>`__
|
||||||
- `Settings <#settings>`__
|
- `Settings <#settings>`__
|
||||||
- `Download Model from TensorFlow Hub <#download-model-from-tensorflow-hub>`__
|
- `Download Model from TensorFlow
|
||||||
- `Convert Model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
Hub <#download-model-from-tensorflow-hub>`__
|
||||||
- `Test Inference on the Converted Model <#test-inference-on-the-converted-model>`__
|
- `Convert Model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||||
- `Select inference device <#select-inference-device>`__
|
- `Test Inference on the Converted
|
||||||
- `Load the Model <#load-the-model>`__
|
Model <#test-inference-on-the-converted-model>`__
|
||||||
- `Get Model Information <#get-model-information>`__
|
- `Select inference device <#select-inference-device>`__
|
||||||
- `Get an Image for Test Inference <#get-an-image-for-test-inference>`__
|
|
||||||
- `Perform Inference <#perform-inference>`__
|
- `Load the Model <#load-the-model>`__
|
||||||
- `Inference Result Visualization <#inference-result-visualization>`__
|
- `Get Model Information <#get-model-information>`__
|
||||||
- `Next Steps <#next-steps>`__
|
- `Get an Image for Test
|
||||||
- `Async inference pipeline <#async-inference-pipeline>`__
|
Inference <#get-an-image-for-test-inference>`__
|
||||||
- `Integration preprocessing to model <#integration-preprocessing-to-model>`__
|
- `Perform Inference <#perform-inference>`__
|
||||||
|
- `Inference Result
|
||||||
|
Visualization <#inference-result-visualization>`__
|
||||||
|
|
||||||
|
- `Next Steps <#next-steps>`__
|
||||||
|
|
||||||
|
- `Async inference pipeline <#async-inference-pipeline>`__
|
||||||
|
- `Integration preprocessing to
|
||||||
|
model <#integration-preprocessing-to-model>`__
|
||||||
|
|
||||||
Prerequisites
|
Prerequisites
|
||||||
-------------
|
-------------
|
||||||
|
|
@ -355,7 +364,7 @@ Read the image, resize and convert it to the input shape of the network:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7f5f2fe7a310>
|
<matplotlib.image.AxesImage at 0x7f39e4396eb0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:ed008919b91d0e231d8872fb4fae77bdfc3a513547568b9448b69de0065625c5
|
oid sha256:8e7e8a03ae8a07e661d3a25306fe3dca53093416ecf98222fe478edaf13d07df
|
||||||
size 393453
|
size 393160
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-instance-segmentation-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-instance-segmentation-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="120-tensorflow-instance-segmentation-to-openvino-with-output_25_1.png">120-tensorflow-instance-segmentation-to-openvin..></a> 26-Jan-2024 01:05 395346
|
|
||||||
<a href="120-tensorflow-instance-segmentation-to-openvino-with-output_39_0.png">120-tensorflow-instance-segmentation-to-openvin..></a> 26-Jan-2024 01:05 393453
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -157,7 +157,7 @@ from TensorFlow Hub:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/model/tf/faster_rcnn_resnet50_v1_640x640.tar.gz')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/model/tf/faster_rcnn_resnet50_v1_640x640.tar.gz')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -343,7 +343,7 @@ Load and save an image:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_bike.jpg')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_bike.jpg')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -371,7 +371,7 @@ Read the image, resize and convert it to the input shape of the network:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7f8f0a2c3d00>
|
<matplotlib.image.AxesImage at 0x7f62f45088e0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -635,7 +635,7 @@ Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_91cl.txt')
|
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_91cl.txt')
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:a5b7ad5bc51c70dfab0037ca885aeb4e613947df3fd7fdff8d055c8d64aefd4a
|
oid sha256:69ba7f4bdc1e3d85a98b34863980a347ab3bdf2aedb7d9d78a2f8d43484127b6
|
||||||
size 391815
|
size 391578
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-object-detection-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-object-detection-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="120-tensorflow-object-detection-to-openvino-with-output_25_1.png">120-tensorflow-object-detection-to-openvino-wit..></a> 26-Jan-2024 01:05 395346
|
|
||||||
<a href="120-tensorflow-object-detection-to-openvino-with-output_38_0.png">120-tensorflow-object-detection-to-openvino-wit..></a> 26-Jan-2024 01:05 391815
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -35,7 +35,7 @@ Table of contents:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (23.3.2)
|
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -196,19 +196,19 @@ NLP model from Hugging Face and export it in ONNX format:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:11:32.267646: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 23:07:37.212192: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 23:11:32.303561: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 23:07:37.247733: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:11:32.956586: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:07:37.883428: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -699,12 +699,12 @@ frameworks conversion guides.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:11:51.061555: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
2024-02-09 23:07:56.008045: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||||
2024-01-25 23:11:51.061587: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
2024-02-09 23:07:56.008076: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||||
2024-01-25 23:11:51.061592: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
2024-02-09 23:07:56.008081: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||||
2024-01-25 23:11:51.061773: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
2024-02-09 23:07:56.008267: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||||
2024-01-25 23:11:51.061789: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
2024-02-09 23:07:56.008284: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||||
2024-01-25 23:11:51.061793: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
2024-02-09 23:07:56.008287: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||||
|
|
||||||
|
|
||||||
Migration from Legacy conversion API
|
Migration from Legacy conversion API
|
||||||
|
|
|
||||||
|
|
@ -118,7 +118,7 @@ documentation.
|
||||||
conversion into IR. The legacy Frontend is Python
|
conversion into IR. The legacy Frontend is Python
|
||||||
based and is available for TensorFlow*, ONNX*, MXNet*,
|
based and is available for TensorFlow*, ONNX*, MXNet*,
|
||||||
Caffe*, and Kaldi* models.
|
Caffe*, and Kaldi* models.
|
||||||
--input_model INPUT_MODEL, -w INPUT_MODEL, -m INPUT_MODEL
|
--input_model INPUT_MODEL, -m INPUT_MODEL, -w INPUT_MODEL
|
||||||
Tensorflow*: a file with a pre-trained model (binary
|
Tensorflow*: a file with a pre-trained model (binary
|
||||||
or text .pb file after freezing). Caffe*: a model
|
or text .pb file after freezing). Caffe*: a model
|
||||||
proto file with model weights.
|
proto file with model weights.
|
||||||
|
|
@ -256,11 +256,7 @@ documentation.
|
||||||
Paths or a comma-separated list of paths to libraries
|
Paths or a comma-separated list of paths to libraries
|
||||||
(.so or .dll) with extensions. For the legacy MO path
|
(.so or .dll) with extensions. For the legacy MO path
|
||||||
(if `--use_legacy_frontend` is used), a directory or a
|
(if `--use_legacy_frontend` is used), a directory or a
|
||||||
comma
|
comma-separated list of directories with extensions
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
-separated list of directories with extensions
|
|
||||||
are supported. To disable all extensions including
|
are supported. To disable all extensions including
|
||||||
those that are placed at the default location, pass an
|
those that are placed at the default location, pass an
|
||||||
empty string.
|
empty string.
|
||||||
|
|
@ -744,19 +740,19 @@ NLP model from Hugging Face and export it in ONNX format:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:12:13.362144: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 23:08:18.586507: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 23:12:13.397165: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 23:08:18.621399: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:12:14.029251: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:08:19.256172: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1027,8 +1023,8 @@ To convert a model to OpenVINO IR, use the following command:
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1129,8 +1125,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -1154,8 +1150,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1211,8 +1207,8 @@ sequence length dimension for inputs:
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1262,8 +1258,8 @@ dimension:
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1336,8 +1332,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -1361,8 +1357,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1438,13 +1434,9 @@ Resnet50 model that was exported to the ONNX format:
|
||||||
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.0/openvino_2_0_transition_guide.html
|
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.0/openvino_2_0_transition_guide.html
|
||||||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1495,8 +1487,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -1520,8 +1512,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1578,8 +1570,8 @@ that the preprocessing takes negligible time for inference.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -1603,8 +1595,8 @@ that the preprocessing takes negligible time for inference.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1658,8 +1650,8 @@ the color channels before inference.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -1709,8 +1701,8 @@ models, this decrease is negligible.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ SUCCESS ] Generated IR version 11 model.
|
[ SUCCESS ] Generated IR version 11 model.
|
||||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
|
||||||
|
|
@ -303,6 +303,7 @@ value 25 to speed up the execution.
|
||||||
INFO:nncf:Validation of initial model was started
|
INFO:nncf:Validation of initial model was started
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
INFO:nncf:Elapsed Time: 00:00:00
|
INFO:nncf:Elapsed Time: 00:00:00
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
Convert Detectron2 Models to OpenVINO™
|
Convert Detectron2 Models to OpenVINO™
|
||||||
=========================================
|
=======================================
|
||||||
|
|
||||||
`Detectron2 <https://github.com/facebookresearch/detectron2>`__ is
|
`Detectron2 <https://github.com/facebookresearch/detectron2>`__ is
|
||||||
Facebook AI Research’s library that provides state-of-the-art detection
|
Facebook AI Research’s library that provides state-of-the-art detection
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:b9a59f5b6bc77d33212bcadd69faacc6c286d6eac7d7ed88ce28ccefe227f010
|
oid sha256:97f44896da4cb3dee863ed6869c16015702bf26050467e2ff84110920318665b
|
||||||
size 59043
|
size 58781
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:2db4d40d09643561d54ed996c0e42df721272e7bc17220c20240a81142df67eb
|
oid sha256:4398d08f29dcb59dd57f4cd453308897ac75f6f8b860558fb21cfdbcd479b6e3
|
||||||
size 508506
|
size 509675
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:44dbb04db8f71ea61f1896b341ac6811246dd60d600942296da3918b7f82c715
|
oid sha256:ae3a4df87c84baa406e0d2ed491e25f28bd9d8e6e9fe649bd74c0b7eae35ac07
|
||||||
size 55966
|
size 55630
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
version https://git-lfs.github.com/spec/v1
|
version https://git-lfs.github.com/spec/v1
|
||||||
oid sha256:b9e00a023f05b67dd28e92c59ea377d5c9dc85f2c4f183d44e03227b25384749
|
oid sha256:c832d80fe116434db304523dbc29d6e292e7a5733b70ae709e8eb2a0674506e6
|
||||||
size 457794
|
size 457756
|
||||||
|
|
|
||||||
|
|
@ -1,12 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/123-detectron2-to-openvino-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/123-detectron2-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_22_0.jpg">123-detectron2-to-openvino-with-output_22_0.jpg</a> 26-Jan-2024 01:05 59043
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_22_0.png">123-detectron2-to-openvino-with-output_22_0.png</a> 26-Jan-2024 01:05 508506
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_32_0.jpg">123-detectron2-to-openvino-with-output_32_0.jpg</a> 26-Jan-2024 01:05 55966
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_32_0.png">123-detectron2-to-openvino-with-output_32_0.png</a> 26-Jan-2024 01:05 457794
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_8_0.jpg">123-detectron2-to-openvino-with-output_8_0.jpg</a> 26-Jan-2024 01:05 46858
|
|
||||||
<a href="123-detectron2-to-openvino-with-output_8_0.png">123-detectron2-to-openvino-with-output_8_0.png</a> 26-Jan-2024 01:05 503218
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
🤗 Hugging Face Model Hub with OpenVINO™
|
Hugging Face Model Hub with OpenVINO™
|
||||||
=========================================
|
=========================================
|
||||||
|
|
||||||
The Hugging Face (HF) `Model Hub <https://huggingface.co/models>`__ is a
|
The Hugging Face (HF) `Model Hub <https://huggingface.co/models>`__ is a
|
||||||
|
|
@ -129,7 +129,7 @@ tutorials <https://huggingface.co/learn/nlp-course/chapter2/2?fw=pt#behind-the-p
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||||
return self.fget.__get__(instance, owner)()
|
return self.fget.__get__(instance, owner)()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -268,7 +268,7 @@ Converting a Model Using the Optimum Intel Package
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
🤗 Optimum Intel is the interface between the 🤗 Transformers and
|
Optimum Intel is the interface between the Transformers and
|
||||||
Diffusers libraries and the different tools and libraries provided by
|
Diffusers libraries and the different tools and libraries provided by
|
||||||
Intel to accelerate end-to-end pipelines on Intel architectures.
|
Intel to accelerate end-to-end pipelines on Intel architectures.
|
||||||
|
|
||||||
|
|
@ -343,14 +343,14 @@ documentation <https://huggingface.co/docs/optimum/intel/inference>`__.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:14:43.057378: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
2024-02-09 23:10:50.826096: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||||
2024-01-25 23:14:43.091971: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
2024-02-09 23:10:50.861099: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:14:43.657635: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:10:51.428729: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
Initialize and Convert the Model Automatically using OVModel class
|
Initialize and Convert the Model Automatically using OVModel class
|
||||||
|
|
@ -478,7 +478,7 @@ Full list of supported arguments available via ``--help``
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:14:55.440371: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:11:03.409282: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -490,6 +490,7 @@ Full list of supported arguments available via ``--help``
|
||||||
[--int8]
|
[--int8]
|
||||||
[--weight-format {fp32,fp16,int8,int4_sym_g128,int4_asym_g128,int4_sym_g64,int4_asym_g64}]
|
[--weight-format {fp32,fp16,int8,int4_sym_g128,int4_asym_g128,int4_sym_g64,int4_asym_g64}]
|
||||||
[--ratio RATIO] [--disable-stateful]
|
[--ratio RATIO] [--disable-stateful]
|
||||||
|
[--convert-tokenizer]
|
||||||
output
|
output
|
||||||
|
|
||||||
optional arguments:
|
optional arguments:
|
||||||
|
|
@ -506,21 +507,20 @@ Full list of supported arguments available via ``--help``
|
||||||
--task TASK The task to export the model for. If not specified,
|
--task TASK The task to export the model for. If not specified,
|
||||||
the task will be auto-inferred based on the model.
|
the task will be auto-inferred based on the model.
|
||||||
Available tasks depend on the model, but are among:
|
Available tasks depend on the model, but are among:
|
||||||
['stable-diffusion', 'audio-xvector', 'text-
|
['sentence-similarity', 'object-detection', 'question-
|
||||||
classification', 'automatic-speech-recognition',
|
answering', 'text-to-audio', 'audio-xvector', 'stable-
|
||||||
'image-segmentation', 'audio-classification',
|
diffusion-xl', 'feature-extraction', 'image-to-image',
|
||||||
'multiple-choice', 'image-to-text', 'stable-diffusion-
|
'text-generation', 'mask-generation', 'text-
|
||||||
xl', 'zero-shot-image-classification', 'text-to-
|
classification', 'image-segmentation', 'automatic-
|
||||||
audio', 'fill-mask', 'object-detection', 'image-to-
|
speech-recognition', 'text2text-generation', 'stable-
|
||||||
image', 'mask-generation', 'image-classification',
|
diffusion', 'audio-classification', 'semantic-
|
||||||
'feature-extraction', 'semantic-segmentation', 'text-
|
segmentation', 'fill-mask', 'depth-estimation', 'zero-
|
||||||
generation', 'question-answering', 'conversational',
|
shot-image-classification', 'image-to-text', 'zero-
|
||||||
'zero-shot-object-detection', 'masked-im', 'sentence-
|
shot-object-detection', 'multiple-choice',
|
||||||
similarity', 'audio-frame-classification', 'text2text-
|
'conversational', 'image-classification', 'masked-im',
|
||||||
generation', 'depth-estimation', 'token-
|
'audio-frame-classification', 'token-classification'].
|
||||||
classification']. For decoder models, use `xxx-with-
|
For decoder models, use `xxx-with-past` to export the
|
||||||
past` to export the model using past key values in the
|
model using past key values in the decoder.
|
||||||
decoder.
|
|
||||||
--cache_dir CACHE_DIR
|
--cache_dir CACHE_DIR
|
||||||
Path indicating where to store cache.
|
Path indicating where to store cache.
|
||||||
--framework {pt,tf} The framework to use for the export. If not provided,
|
--framework {pt,tf} The framework to use for the export. If not provided,
|
||||||
|
|
@ -558,6 +558,8 @@ Full list of supported arguments available via ``--help``
|
||||||
a stateless model, for example, to be compatible with
|
a stateless model, for example, to be compatible with
|
||||||
existing OpenVINO native inference code that expects
|
existing OpenVINO native inference code that expects
|
||||||
kv-cache inputs and outputs in the model.
|
kv-cache inputs and outputs in the model.
|
||||||
|
--convert-tokenizer Add converted tokenizer and detokenizer with OpenVINO
|
||||||
|
Tokenizers
|
||||||
|
|
||||||
|
|
||||||
The command line export for model from example above with FP16 weights
|
The command line export for model from example above with FP16 weights
|
||||||
|
|
@ -578,7 +580,7 @@ compression:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:14:59.738992: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
2024-02-09 23:11:07.691775: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -593,7 +595,7 @@ compression:
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||||
return self.fget.__get__(instance, owner)()
|
return self.fget.__get__(instance, owner)()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -658,13 +660,20 @@ Model inference is exactly the same as for the original model!
|
||||||
3) negative 0.0031
|
3) negative 0.0031
|
||||||
|
|
||||||
|
|
||||||
You can find more examples of using Optimum Intel here:
|
You can find more examples of using Optimum Intel here: 1. `Accelerate
|
||||||
|
Inference of Sparse Transformer
|
||||||
1. `Accelerate Inference of Sparse Transformer Models <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/116-sparsity-optimization>`__
|
Models <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/116-sparsity-optimization>`__
|
||||||
2. `Grammatical Error Correction with OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction>`__
|
2. `Grammatical Error Correction with
|
||||||
3. `Stable Diffusion v2.1 using Optimum-Intel OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/236-stable-diffusion-v2/236-stable-diffusion-v2-optimum-demo.ipynb>`__
|
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction>`__
|
||||||
4. `Image generation with Stable Diffusion XL <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/248-stable-diffusion-xl>`__
|
3. `Stable Diffusion v2.1 using Optimum-Intel
|
||||||
5. `Instruction following using Databricks Dolly 2.0 <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following>`__
|
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/236-stable-diffusion-v2/236-stable-diffusion-v2-optimum-demo.ipynb>`__
|
||||||
6. `Create LLM-powered Chatbot using OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot>`__
|
4. `Image generation with Stable Diffusion
|
||||||
7. `Document Visual Question Answering Using Pix2Struct and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/260-pix2struct-docvqa>`__
|
XL <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/248-stable-diffusion-xl>`__
|
||||||
8. `Automatic speech recognition using Distil-Whisper and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
|
5. `Instruction following using Databricks Dolly
|
||||||
|
2.0 <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following>`__
|
||||||
|
6. `Create LLM-powered Chatbot using
|
||||||
|
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot>`__
|
||||||
|
7. `Document Visual Question Answering Using Pix2Struct and
|
||||||
|
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/260-pix2struct-docvqa>`__
|
||||||
|
8. `Automatic speech recognition using Distil-Whisper and
|
||||||
|
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
|
||||||
|
|
|
||||||
|
|
@ -37,8 +37,10 @@ Table of contents:
|
||||||
- `Use the OpenVINO IR model to run an
|
- `Use the OpenVINO IR model to run an
|
||||||
inference <#use-the-openvino-ir-model-to-run-an-inference>`__
|
inference <#use-the-openvino-ir-model-to-run-an-inference>`__
|
||||||
|
|
||||||
Prerequisites\
|
Prerequisites
|
||||||
-------------------------------------------------------
|
-------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -144,12 +146,14 @@ wight <https://pytorch.org/vision/stable/models.html#using-the-pre-trained-model
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
|
||||||
warnings.warn(
|
warnings.warn(
|
||||||
|
|
||||||
|
|
||||||
Use the original model to run an inference\
|
Use the original model to run an inference
|
||||||
------------------------------------------------------------------------------------
|
------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -188,7 +192,7 @@ And print results
|
||||||
|
|
||||||
Predicted Class: 281
|
Predicted Class: 281
|
||||||
Predicted Label: n02123045 tabby, tabby cat
|
Predicted Label: n02123045 tabby, tabby cat
|
||||||
Predicted Probability: 0.6184040307998657
|
Predicted Probability: 0.5800774693489075
|
||||||
|
|
||||||
|
|
||||||
Convert the model to OpenVINO Intermediate representation format
|
Convert the model to OpenVINO Intermediate representation format
|
||||||
|
|
@ -260,8 +264,10 @@ Select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
compiled_model = core.compile_model(ov_model_xml_path, device_name=device.value)
|
compiled_model = core.compile_model(ov_model_xml_path, device_name=device.value)
|
||||||
|
|
||||||
Use the OpenVINO IR model to run an inference\
|
Use the OpenVINO IR model to run an inference
|
||||||
---------------------------------------------------------------------------------------
|
---------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-convnext-classification-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-convnext-classification-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="125-convnext-classification-with-output_4_0.jpg">125-convnext-classification-with-output_4_0.jpg</a> 26-Jan-2024 01:05 63187
|
|
||||||
<a href="125-convnext-classification-with-output_4_0.png">125-convnext-classification-with-output_4_0.png</a> 26-Jan-2024 01:05 723717
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -22,27 +22,28 @@ plant, sheep, sofa, train, tv monitor**
|
||||||
More information about the model is available in the `torchvision
|
More information about the model is available in the `torchvision
|
||||||
documentation <https://pytorch.org/vision/main/models/lraspp.html>`__
|
documentation <https://pytorch.org/vision/main/models/lraspp.html>`__
|
||||||
|
|
||||||
|
Table of contents:
|
||||||
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
- `Prerequisites) <#prerequisites>`__
|
||||||
**Table of contents:**
|
|
||||||
|
|
||||||
- `Prerequisites <#prerequisites>`__
|
|
||||||
- `Get a test image <#get-a-test-image>`__
|
- `Get a test image <#get-a-test-image>`__
|
||||||
- `Download and prepare a model <#download-and-prepare-a-model>`__
|
- `Download and prepare a model <#download-and-prepare-a-model>`__
|
||||||
- `Define a preprocessing and prepare an input
|
- `Define a preprocessing and prepare an input
|
||||||
data <#define-a-preprocessing-and-prepare-an-input-data>`__
|
data <#define-a-preprocessing-and-prepare-an-input-data>`__
|
||||||
- `Run an inference on the PyTorch
|
- `Run an inference on the PyTorch
|
||||||
model <#run-an-inference-on-the-pytorch-model>`__
|
model) <#run-an-inference-on-the-pytorch-model>`__
|
||||||
- `Convert the original model to OpenVINO IR
|
- `Convert the original model to OpenVINO IR
|
||||||
Format <#convert-the-original-model-to-openvino-ir-format>`__
|
Format <#convert-the-original-model-to-openvino-ir-format>`__
|
||||||
- `Run an inference on the OpenVINO
|
- `Run an inference on the OpenVINO
|
||||||
model <#run-an-inference-on-the-openvino-model>`__
|
model) <#run-an-inference-on-the-openvino-model>`__
|
||||||
- `Show results <#show-results>`__
|
- `Show results <#show-results>`__
|
||||||
- `Show results for the OpenVINO IR
|
- `Show results for the OpenVINO IR
|
||||||
model <#show-results-for-the-openvino-ir-model>`__
|
model) <#show-results-for-the-openvino-ir-model>`__
|
||||||
|
|
||||||
|
Prerequisites
|
||||||
|
-------------
|
||||||
|
|
||||||
|
|
||||||
Prerequisites\
|
|
||||||
-------------------------------------------------------
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -69,24 +70,25 @@ Prerequisites\
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
import torch
|
import torch
|
||||||
|
|
||||||
Get a test image
|
Get a test image
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
First of all lets get a test
|
|
||||||
image from an open dataset.
|
|
||||||
|
First of all lets get a test image from an open dataset.
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import urllib.request
|
import urllib.request
|
||||||
|
|
||||||
from torchvision.io import read_image
|
from torchvision.io import read_image
|
||||||
import torchvision.transforms as transforms
|
import torchvision.transforms as transforms
|
||||||
|
|
||||||
|
|
||||||
img_path = 'cats_image.jpeg'
|
img_path = 'cats_image.jpeg'
|
||||||
urllib.request.urlretrieve(
|
urllib.request.urlretrieve(
|
||||||
url='https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg',
|
url='https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg',
|
||||||
|
|
@ -103,10 +105,11 @@ image from an open dataset.
|
||||||
Download and prepare a model
|
Download and prepare a model
|
||||||
----------------------------
|
----------------------------
|
||||||
|
|
||||||
Define width and height of the
|
|
||||||
image that will be used by the network during inference. According to
|
|
||||||
the input transforms function, the model is pre-trained on images with a
|
Define width and height of the image that will be used by the network
|
||||||
height of 480 and width of 640.
|
during inference. According to the input transforms function, the model
|
||||||
|
is pre-trained on images with a height of 480 and width of 640.
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -119,12 +122,12 @@ models <https://pytorch.org/vision/stable/models.html#listing-and-retrieving-ava
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import torchvision.models as models
|
import torchvision.models as models
|
||||||
|
|
||||||
# List available models
|
# List available models
|
||||||
all_models = models.list_models()
|
all_models = models.list_models()
|
||||||
# List of models by type
|
# List of models by type
|
||||||
segmentation_models = models.list_models(module=models.segmentation)
|
segmentation_models = models.list_models(module=models.segmentation)
|
||||||
|
|
||||||
print(segmentation_models)
|
print(segmentation_models)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -155,24 +158,27 @@ but there is only one for this model.
|
||||||
Define a preprocessing and prepare an input data
|
Define a preprocessing and prepare an input data
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
|
|
||||||
You can use
|
|
||||||
``torchvision.transforms`` to make a preprocessing or
|
|
||||||
|
You can use ``torchvision.transforms`` to make a preprocessing or
|
||||||
use\ `preprocessing transforms from the model
|
use\ `preprocessing transforms from the model
|
||||||
wight <https://pytorch.org/vision/stable/models.html#using-the-pre-trained-models>`__.
|
wight <https://pytorch.org/vision/stable/models.html#using-the-pre-trained-models>`__.
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
preprocess = models.segmentation.LRASPP_MobileNet_V3_Large_Weights.COCO_WITH_VOC_LABELS_V1.transforms()
|
preprocess = models.segmentation.LRASPP_MobileNet_V3_Large_Weights.COCO_WITH_VOC_LABELS_V1.transforms()
|
||||||
preprocess.resize_size = (IMAGE_HEIGHT, IMAGE_WIDTH) # change to an image size
|
preprocess.resize_size = (IMAGE_HEIGHT, IMAGE_WIDTH) # change to an image size
|
||||||
|
|
||||||
input_data = preprocess(image)
|
input_data = preprocess(image)
|
||||||
input_data = np.expand_dims(input_data, axis=0)
|
input_data = np.expand_dims(input_data, axis=0)
|
||||||
|
|
||||||
Run an inference on the PyTorch model\
|
Run an inference on the PyTorch model
|
||||||
-------------------------------------------------------------------------------
|
-------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
@ -193,8 +199,8 @@ directory. For more information on how to convert models, see this
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
ov_model_xml_path = Path('models/ov_lraspp_model.xml')
|
ov_model_xml_path = Path('models/ov_lraspp_model.xml')
|
||||||
|
|
||||||
|
|
||||||
if not ov_model_xml_path.exists():
|
if not ov_model_xml_path.exists():
|
||||||
ov_model_xml_path.parent.mkdir(parents=True, exist_ok=True)
|
ov_model_xml_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||||
|
|
@ -203,15 +209,17 @@ directory. For more information on how to convert models, see this
|
||||||
else:
|
else:
|
||||||
print(f"IR model {ov_model_xml_path} already exists.")
|
print(f"IR model {ov_model_xml_path} already exists.")
|
||||||
|
|
||||||
Run an inference on the OpenVINO model\
|
Run an inference on the OpenVINO model
|
||||||
--------------------------------------------------------------------------------
|
--------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Select device from dropdown list for running inference using OpenVINO
|
Select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import ipywidgets as widgets
|
import ipywidgets as widgets
|
||||||
|
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
device = widgets.Dropdown(
|
device = widgets.Dropdown(
|
||||||
options=core.available_devices + ["AUTO"],
|
options=core.available_devices + ["AUTO"],
|
||||||
|
|
@ -219,7 +227,7 @@ Select device from dropdown list for running inference using OpenVINO
|
||||||
description='Device:',
|
description='Device:',
|
||||||
disabled=False,
|
disabled=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
device
|
device
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -244,9 +252,10 @@ Run an inference
|
||||||
Show results
|
Show results
|
||||||
------------
|
------------
|
||||||
|
|
||||||
Confirm that the segmentation
|
|
||||||
results look as expected by comparing model predictions on the OpenVINO
|
|
||||||
IR and PyTorch models.
|
Confirm that the segmentation results look as expected by comparing
|
||||||
|
model predictions on the OpenVINO IR and PyTorch models.
|
||||||
|
|
||||||
You can use `pytorch
|
You can use `pytorch
|
||||||
tutorial <https://pytorch.org/vision/0.12/auto_examples/plot_visualization_utils.html#sphx-glr-auto-examples-plot-visualization-utils-py>`__
|
tutorial <https://pytorch.org/vision/0.12/auto_examples/plot_visualization_utils.html#sphx-glr-auto-examples-plot-visualization-utils-py>`__
|
||||||
|
|
@ -257,13 +266,13 @@ visualize the image with a ``cat`` mask for the PyTorch model.
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
import torchvision.transforms.functional as F
|
import torchvision.transforms.functional as F
|
||||||
|
|
||||||
|
|
||||||
plt.rcParams["savefig.bbox"] = 'tight'
|
plt.rcParams["savefig.bbox"] = 'tight'
|
||||||
|
|
||||||
|
|
||||||
def show(imgs):
|
def show(imgs):
|
||||||
if not isinstance(imgs, list):
|
if not isinstance(imgs, list):
|
||||||
imgs = [imgs]
|
imgs = [imgs]
|
||||||
|
|
@ -284,11 +293,11 @@ Prepare and display a cat mask.
|
||||||
'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor'
|
'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor'
|
||||||
]
|
]
|
||||||
sem_class_to_idx = {cls: idx for (idx, cls) in enumerate(sem_classes)}
|
sem_class_to_idx = {cls: idx for (idx, cls) in enumerate(sem_classes)}
|
||||||
|
|
||||||
normalized_mask = torch.nn.functional.softmax(result_torch, dim=1)
|
normalized_mask = torch.nn.functional.softmax(result_torch, dim=1)
|
||||||
|
|
||||||
cat_mask = normalized_mask[0, sem_class_to_idx['cat']]
|
cat_mask = normalized_mask[0, sem_class_to_idx['cat']]
|
||||||
|
|
||||||
show(cat_mask)
|
show(cat_mask)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -313,7 +322,7 @@ And now we can plot a boolean mask on top of the original image.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from torchvision.utils import draw_segmentation_masks
|
from torchvision.utils import draw_segmentation_masks
|
||||||
|
|
||||||
show(draw_segmentation_masks(image, masks=boolean_cat_mask, alpha=0.7, colors='yellow'))
|
show(draw_segmentation_masks(image, masks=boolean_cat_mask, alpha=0.7, colors='yellow'))
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -321,8 +330,10 @@ And now we can plot a boolean mask on top of the original image.
|
||||||
.. image:: 125-lraspp-segmentation-with-output_files/125-lraspp-segmentation-with-output_32_0.png
|
.. image:: 125-lraspp-segmentation-with-output_files/125-lraspp-segmentation-with-output_32_0.png
|
||||||
|
|
||||||
|
|
||||||
Show results for the OpenVINO IR model\
|
Show results for the OpenVINO IR model
|
||||||
--------------------------------------------------------------------------------
|
--------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-lraspp-segmentation-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-lraspp-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="125-lraspp-segmentation-with-output_28_0.png">125-lraspp-segmentation-with-output_28_0.png</a> 26-Jan-2024 01:05 87576
|
|
||||||
<a href="125-lraspp-segmentation-with-output_32_0.png">125-lraspp-segmentation-with-output_32_0.png</a> 26-Jan-2024 01:05 385522
|
|
||||||
<a href="125-lraspp-segmentation-with-output_34_0.png">125-lraspp-segmentation-with-output_34_0.png</a> 26-Jan-2024 01:05 385520
|
|
||||||
<a href="125-lraspp-segmentation-with-output_5_0.jpg">125-lraspp-segmentation-with-output_5_0.jpg</a> 26-Jan-2024 01:05 63187
|
|
||||||
<a href="125-lraspp-segmentation-with-output_5_0.png">125-lraspp-segmentation-with-output_5_0.png</a> 26-Jan-2024 01:05 723717
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -21,22 +21,28 @@ You have the flexibility to run this tutorial notebook in its entirety
|
||||||
or selectively execute specific sections, as each section operates
|
or selectively execute specific sections, as each section operates
|
||||||
independently.
|
independently.
|
||||||
|
|
||||||
**Table of contents:**
|
Table of contents:
|
||||||
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
- `Image classification <#image-classification>`__
|
- `Image classification <#image-classification>`__
|
||||||
- `Install required packages <#install-required-packages>`__
|
|
||||||
- `Import libraries <#import-libraries>`__
|
- `Install required packages <#install-required-packages>`__
|
||||||
- `Download the classifier <#download-the-classifier>`__
|
- `Import libraries <#import-libraries>`__
|
||||||
- `Download a single image to try the model on <#download-a-single-image-to-try-the-model-on>`__
|
- `Download the classifier <#download-the-classifier>`__
|
||||||
- `Convert model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
- `Download a single image to try the model
|
||||||
- `Select inference device <#select-inference-device>`__
|
on <#download-a-single-image-to-try-the-model-on>`__
|
||||||
- `Inference <#inference>`__
|
- `Convert model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||||
- `Image style transfer <#image-style-transfer>`__
|
- `Select inference device <#select-inference-device>`__
|
||||||
- `Install required packages <#install-required-packages>`__
|
- `Inference <#inference>`__
|
||||||
- `Load the model <#load-the-model>`__
|
|
||||||
- `Convert the model to OpenVINO IR <#convert-the-model-to-openvino-ir>`__
|
- `Image style transfer <#image-style-transfer>`__
|
||||||
- `Select inference device <#select-inference-device>`__
|
|
||||||
- `Inference <#inference>`__
|
- `Install required packages <#install-required-packages>`__
|
||||||
|
- `Load the model <#load-the-model>`__
|
||||||
|
- `Convert the model to OpenVINO
|
||||||
|
IR <#convert-the-model-to-openvino-ir>`__
|
||||||
|
- `Select inference device <#select-inference-device>`__
|
||||||
|
- `Inference <#inference>`__
|
||||||
|
|
||||||
Image classification
|
Image classification
|
||||||
--------------------
|
--------------------
|
||||||
|
|
@ -125,8 +131,8 @@ and wrap it as a Keras layer with ``hub.KerasLayer``.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:15:55.944757: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
2024-02-09 23:12:03.569013: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||||
2024-01-25 23:15:55.944938: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
2024-02-09 23:12:03.569190: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||||
|
|
||||||
|
|
||||||
Download a single image to try the model on
|
Download a single image to try the model on
|
||||||
|
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/126-tensorflow-hub-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/126-tensorflow-hub-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="126-tensorflow-hub-with-output_11_0.jpg">126-tensorflow-hub-with-output_11_0.jpg</a> 26-Jan-2024 01:05 10479
|
|
||||||
<a href="126-tensorflow-hub-with-output_11_0.png">126-tensorflow-hub-with-output_11_0.png</a> 26-Jan-2024 01:05 92843
|
|
||||||
<a href="126-tensorflow-hub-with-output_26_0.png">126-tensorflow-hub-with-output_26_0.png</a> 26-Jan-2024 01:05 203738
|
|
||||||
<a href="126-tensorflow-hub-with-output_45_0.png">126-tensorflow-hub-with-output_45_0.png</a> 26-Jan-2024 01:05 538743
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,3 @@
|
||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:37f1a76ccf411397019d83e8344b79cdebf644f0f6ba6598141e72c6ee50c62b
|
||||||
|
size 224517
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:33f7798bd5dfbfa423dcca1d37eb7e29d0474e331105347cf0efdcfd695b850a
|
||||||
|
size 335047
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6b354964f6b453eb9454e0e21ccb3e1275071464698af4693fc94235a1ca563e
|
||||||
|
size 296205
|
||||||
|
|
@ -3,7 +3,7 @@ Monodepth Estimation with OpenVINO
|
||||||
|
|
||||||
This tutorial demonstrates Monocular Depth Estimation with MidasNet in
|
This tutorial demonstrates Monocular Depth Estimation with MidasNet in
|
||||||
OpenVINO. Model information can be found
|
OpenVINO. Model information can be found
|
||||||
`here <https://docs.openvino.ai/2023.0/omz_models_model_midasnet.html>`__.
|
`here <https://docs.openvino.ai/2023.3/omz_models_model_midasnet.html>`__.
|
||||||
|
|
||||||
.. figure:: https://user-images.githubusercontent.com/36741649/127173017-a0bbcf75-db24-4d2c-81b9-616e04ab7cd9.gif
|
.. figure:: https://user-images.githubusercontent.com/36741649/127173017-a0bbcf75-db24-4d2c-81b9-616e04ab7cd9.gif
|
||||||
:alt: monodepth
|
:alt: monodepth
|
||||||
|
|
@ -69,7 +69,7 @@ Install requirements
|
||||||
|
|
||||||
%pip install -q "openvino>=2023.1.0"
|
%pip install -q "openvino>=2023.1.0"
|
||||||
%pip install -q matplotlib opencv-python requests tqdm
|
%pip install -q matplotlib opencv-python requests tqdm
|
||||||
|
|
||||||
# Fetch `notebook_utils` module
|
# Fetch `notebook_utils` module
|
||||||
import urllib.request
|
import urllib.request
|
||||||
urllib.request.urlretrieve(
|
urllib.request.urlretrieve(
|
||||||
|
|
@ -92,7 +92,7 @@ Install requirements
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
('notebook_utils.py', <http.client.HTTPMessage at 0x7fed08307550>)
|
('notebook_utils.py', <http.client.HTTPMessage at 0x7fe5c4f7e130>)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -105,7 +105,7 @@ Imports
|
||||||
|
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import cv2
|
import cv2
|
||||||
import matplotlib.cm
|
import matplotlib.cm
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
|
|
@ -120,7 +120,7 @@ Imports
|
||||||
display,
|
display,
|
||||||
)
|
)
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
|
|
||||||
from notebook_utils import download_file, load_image
|
from notebook_utils import download_file, load_image
|
||||||
|
|
||||||
Download the model
|
Download the model
|
||||||
|
|
@ -134,14 +134,14 @@ The model is in the `OpenVINO Intermediate Representation
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
model_folder = Path('model')
|
model_folder = Path('model')
|
||||||
|
|
||||||
ir_model_url = 'https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/depth-estimation-midas/FP32/'
|
ir_model_url = 'https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/depth-estimation-midas/FP32/'
|
||||||
ir_model_name_xml = 'MiDaS_small.xml'
|
ir_model_name_xml = 'MiDaS_small.xml'
|
||||||
ir_model_name_bin = 'MiDaS_small.bin'
|
ir_model_name_bin = 'MiDaS_small.bin'
|
||||||
|
|
||||||
download_file(ir_model_url + ir_model_name_xml, filename=ir_model_name_xml, directory=model_folder)
|
download_file(ir_model_url + ir_model_name_xml, filename=ir_model_name_xml, directory=model_folder)
|
||||||
download_file(ir_model_url + ir_model_name_bin, filename=ir_model_name_bin, directory=model_folder)
|
download_file(ir_model_url + ir_model_name_bin, filename=ir_model_name_bin, directory=model_folder)
|
||||||
|
|
||||||
model_xml_path = model_folder / ir_model_name_xml
|
model_xml_path = model_folder / ir_model_name_xml
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -167,13 +167,13 @@ Functions
|
||||||
def normalize_minmax(data):
|
def normalize_minmax(data):
|
||||||
"""Normalizes the values in `data` between 0 and 1"""
|
"""Normalizes the values in `data` between 0 and 1"""
|
||||||
return (data - data.min()) / (data.max() - data.min())
|
return (data - data.min()) / (data.max() - data.min())
|
||||||
|
|
||||||
|
|
||||||
def convert_result_to_image(result, colormap="viridis"):
|
def convert_result_to_image(result, colormap="viridis"):
|
||||||
"""
|
"""
|
||||||
Convert network result of floating point numbers to an RGB image with
|
Convert network result of floating point numbers to an RGB image with
|
||||||
integer values from 0-255 by applying a colormap.
|
integer values from 0-255 by applying a colormap.
|
||||||
|
|
||||||
`result` is expected to be a single network result in 1,H,W shape
|
`result` is expected to be a single network result in 1,H,W shape
|
||||||
`colormap` is a matplotlib colormap.
|
`colormap` is a matplotlib colormap.
|
||||||
See https://matplotlib.org/stable/tutorials/colors/colormaps.html
|
See https://matplotlib.org/stable/tutorials/colors/colormaps.html
|
||||||
|
|
@ -184,8 +184,8 @@ Functions
|
||||||
result = cmap(result)[:, :, :3] * 255
|
result = cmap(result)[:, :, :3] * 255
|
||||||
result = result.astype(np.uint8)
|
result = result.astype(np.uint8)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
def to_rgb(image_data) -> np.ndarray:
|
def to_rgb(image_data) -> np.ndarray:
|
||||||
"""
|
"""
|
||||||
Convert image_data from BGR to RGB
|
Convert image_data from BGR to RGB
|
||||||
|
|
@ -202,7 +202,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import ipywidgets as widgets
|
import ipywidgets as widgets
|
||||||
|
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
device = widgets.Dropdown(
|
device = widgets.Dropdown(
|
||||||
options=core.available_devices + ["AUTO"],
|
options=core.available_devices + ["AUTO"],
|
||||||
|
|
@ -210,7 +210,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
description='Device:',
|
description='Device:',
|
||||||
disabled=False,
|
disabled=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
device
|
device
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -237,10 +237,10 @@ output keys and the expected input shape for the model.
|
||||||
core.set_property({'CACHE_DIR': '../cache'})
|
core.set_property({'CACHE_DIR': '../cache'})
|
||||||
model = core.read_model(model_xml_path)
|
model = core.read_model(model_xml_path)
|
||||||
compiled_model = core.compile_model(model=model, device_name=device.value)
|
compiled_model = core.compile_model(model=model, device_name=device.value)
|
||||||
|
|
||||||
input_key = compiled_model.input(0)
|
input_key = compiled_model.input(0)
|
||||||
output_key = compiled_model.output(0)
|
output_key = compiled_model.output(0)
|
||||||
|
|
||||||
network_input_shape = list(input_key.shape)
|
network_input_shape = list(input_key.shape)
|
||||||
network_image_height, network_image_width = network_input_shape[2:]
|
network_image_height, network_image_width = network_input_shape[2:]
|
||||||
|
|
||||||
|
|
@ -262,10 +262,10 @@ H=height, W=width).
|
||||||
|
|
||||||
IMAGE_FILE = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg"
|
IMAGE_FILE = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg"
|
||||||
image = load_image(path=IMAGE_FILE)
|
image = load_image(path=IMAGE_FILE)
|
||||||
|
|
||||||
# Resize to input shape for network.
|
# Resize to input shape for network.
|
||||||
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
||||||
|
|
||||||
# Reshape the image to network input shape NCHW.
|
# Reshape the image to network input shape NCHW.
|
||||||
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
||||||
|
|
||||||
|
|
@ -280,11 +280,11 @@ original image shape.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
result = compiled_model([input_image])[output_key]
|
result = compiled_model([input_image])[output_key]
|
||||||
|
|
||||||
# Convert the network result of disparity map to an image that shows
|
# Convert the network result of disparity map to an image that shows
|
||||||
# distance as colors.
|
# distance as colors.
|
||||||
result_image = convert_result_to_image(result=result)
|
result_image = convert_result_to_image(result=result)
|
||||||
|
|
||||||
# Resize back to original image shape. The `cv2.resize` function expects shape
|
# Resize back to original image shape. The `cv2.resize` function expects shape
|
||||||
# in (width, height), [::-1] reverses the (height, width) shape to match this.
|
# in (width, height), [::-1] reverses the (height, width) shape to match this.
|
||||||
result_image = cv2.resize(result_image, image.shape[:2][::-1])
|
result_image = cv2.resize(result_image, image.shape[:2][::-1])
|
||||||
|
|
@ -292,7 +292,7 @@ original image shape.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/tmp/ipykernel_809788/2076527990.py:15: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
|
/tmp/ipykernel_2841459/2076527990.py:15: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
|
||||||
cmap = matplotlib.cm.get_cmap(colormap)
|
cmap = matplotlib.cm.get_cmap(colormap)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -346,7 +346,7 @@ Video Settings
|
||||||
# Try the `THEO` encoding if you have FFMPEG installed.
|
# Try the `THEO` encoding if you have FFMPEG installed.
|
||||||
# FOURCC = cv2.VideoWriter_fourcc(*"THEO")
|
# FOURCC = cv2.VideoWriter_fourcc(*"THEO")
|
||||||
FOURCC = cv2.VideoWriter_fourcc(*"vp09")
|
FOURCC = cv2.VideoWriter_fourcc(*"vp09")
|
||||||
|
|
||||||
# Create Path objects for the input video and the result video.
|
# Create Path objects for the input video and the result video.
|
||||||
output_directory = Path("output")
|
output_directory = Path("output")
|
||||||
output_directory.mkdir(exist_ok=True)
|
output_directory.mkdir(exist_ok=True)
|
||||||
|
|
@ -369,11 +369,11 @@ compute values for these properties for the monodepth video.
|
||||||
raise ValueError(f"The video at {VIDEO_FILE} cannot be read.")
|
raise ValueError(f"The video at {VIDEO_FILE} cannot be read.")
|
||||||
input_fps = cap.get(cv2.CAP_PROP_FPS)
|
input_fps = cap.get(cv2.CAP_PROP_FPS)
|
||||||
input_video_frame_height, input_video_frame_width = image.shape[:2]
|
input_video_frame_height, input_video_frame_width = image.shape[:2]
|
||||||
|
|
||||||
target_fps = input_fps / ADVANCE_FRAMES
|
target_fps = input_fps / ADVANCE_FRAMES
|
||||||
target_frame_height = int(input_video_frame_height * SCALE_OUTPUT)
|
target_frame_height = int(input_video_frame_height * SCALE_OUTPUT)
|
||||||
target_frame_width = int(input_video_frame_width * SCALE_OUTPUT)
|
target_frame_width = int(input_video_frame_width * SCALE_OUTPUT)
|
||||||
|
|
||||||
cap.release()
|
cap.release()
|
||||||
print(
|
print(
|
||||||
f"The input video has a frame width of {input_video_frame_width}, "
|
f"The input video has a frame width of {input_video_frame_width}, "
|
||||||
|
|
@ -403,10 +403,10 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
input_video_frame_nr = 0
|
input_video_frame_nr = 0
|
||||||
start_time = time.perf_counter()
|
start_time = time.perf_counter()
|
||||||
total_inference_duration = 0
|
total_inference_duration = 0
|
||||||
|
|
||||||
# Open the input video
|
# Open the input video
|
||||||
cap = cv2.VideoCapture(str(VIDEO_FILE))
|
cap = cv2.VideoCapture(str(VIDEO_FILE))
|
||||||
|
|
||||||
# Create a result video.
|
# Create a result video.
|
||||||
out_video = cv2.VideoWriter(
|
out_video = cv2.VideoWriter(
|
||||||
str(result_video_path),
|
str(result_video_path),
|
||||||
|
|
@ -414,36 +414,36 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
target_fps,
|
target_fps,
|
||||||
(target_frame_width * 2, target_frame_height),
|
(target_frame_width * 2, target_frame_height),
|
||||||
)
|
)
|
||||||
|
|
||||||
num_frames = int(NUM_SECONDS * input_fps)
|
num_frames = int(NUM_SECONDS * input_fps)
|
||||||
total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT) if num_frames == 0 else num_frames
|
total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT) if num_frames == 0 else num_frames
|
||||||
progress_bar = ProgressBar(total=total_frames)
|
progress_bar = ProgressBar(total=total_frames)
|
||||||
progress_bar.display()
|
progress_bar.display()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
while cap.isOpened():
|
while cap.isOpened():
|
||||||
ret, image = cap.read()
|
ret, image = cap.read()
|
||||||
if not ret:
|
if not ret:
|
||||||
cap.release()
|
cap.release()
|
||||||
break
|
break
|
||||||
|
|
||||||
if input_video_frame_nr >= total_frames:
|
if input_video_frame_nr >= total_frames:
|
||||||
break
|
break
|
||||||
|
|
||||||
# Only process every second frame.
|
# Only process every second frame.
|
||||||
# Prepare a frame for inference.
|
# Prepare a frame for inference.
|
||||||
# Resize to the input shape for network.
|
# Resize to the input shape for network.
|
||||||
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
||||||
# Reshape the image to network input shape NCHW.
|
# Reshape the image to network input shape NCHW.
|
||||||
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
||||||
|
|
||||||
# Do inference.
|
# Do inference.
|
||||||
inference_start_time = time.perf_counter()
|
inference_start_time = time.perf_counter()
|
||||||
result = compiled_model([input_image])[output_key]
|
result = compiled_model([input_image])[output_key]
|
||||||
inference_stop_time = time.perf_counter()
|
inference_stop_time = time.perf_counter()
|
||||||
inference_duration = inference_stop_time - inference_start_time
|
inference_duration = inference_stop_time - inference_start_time
|
||||||
total_inference_duration += inference_duration
|
total_inference_duration += inference_duration
|
||||||
|
|
||||||
if input_video_frame_nr % (10 * ADVANCE_FRAMES) == 0:
|
if input_video_frame_nr % (10 * ADVANCE_FRAMES) == 0:
|
||||||
clear_output(wait=True)
|
clear_output(wait=True)
|
||||||
progress_bar.display()
|
progress_bar.display()
|
||||||
|
|
@ -457,7 +457,7 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
f"({1/inference_duration:.2f} FPS)"
|
f"({1/inference_duration:.2f} FPS)"
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Transform the network result to a RGB image.
|
# Transform the network result to a RGB image.
|
||||||
result_frame = to_rgb(convert_result_to_image(result))
|
result_frame = to_rgb(convert_result_to_image(result))
|
||||||
# Resize the image and the result to a target frame shape.
|
# Resize the image and the result to a target frame shape.
|
||||||
|
|
@ -467,13 +467,13 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
stacked_frame = np.hstack((image, result_frame))
|
stacked_frame = np.hstack((image, result_frame))
|
||||||
# Save a frame to the video.
|
# Save a frame to the video.
|
||||||
out_video.write(stacked_frame)
|
out_video.write(stacked_frame)
|
||||||
|
|
||||||
input_video_frame_nr = input_video_frame_nr + ADVANCE_FRAMES
|
input_video_frame_nr = input_video_frame_nr + ADVANCE_FRAMES
|
||||||
cap.set(1, input_video_frame_nr)
|
cap.set(1, input_video_frame_nr)
|
||||||
|
|
||||||
progress_bar.progress = input_video_frame_nr
|
progress_bar.progress = input_video_frame_nr
|
||||||
progress_bar.update()
|
progress_bar.update()
|
||||||
|
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("Processing interrupted.")
|
print("Processing interrupted.")
|
||||||
finally:
|
finally:
|
||||||
|
|
@ -483,7 +483,7 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
cap.release()
|
cap.release()
|
||||||
end_time = time.perf_counter()
|
end_time = time.perf_counter()
|
||||||
duration = end_time - start_time
|
duration = end_time - start_time
|
||||||
|
|
||||||
print(
|
print(
|
||||||
f"Processed {processed_frames} frames in {duration:.2f} seconds. "
|
f"Processed {processed_frames} frames in {duration:.2f} seconds. "
|
||||||
f"Total FPS (including video processing): {processed_frames/duration:.2f}."
|
f"Total FPS (including video processing): {processed_frames/duration:.2f}."
|
||||||
|
|
@ -494,7 +494,7 @@ Do Inference on a Video and Create Monodepth Video
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Processed 60 frames in 37.40 seconds. Total FPS (including video processing): 1.60.Inference FPS: 43.54
|
Processed 60 frames in 37.50 seconds. Total FPS (including video processing): 1.60.Inference FPS: 43.40
|
||||||
Monodepth Video saved to 'output/Coco%20Walking%20in%20Berkeley_monodepth.mp4'.
|
Monodepth Video saved to 'output/Coco%20Walking%20in%20Berkeley_monodepth.mp4'.
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -524,8 +524,8 @@ Display Monodepth Video
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Showing monodepth video saved at
|
Showing monodepth video saved at
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/201-vision-monodepth/output/Coco%20Walking%20in%20Berkeley_monodepth.mp4
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/201-vision-monodepth/output/Coco%20Walking%20in%20Berkeley_monodepth.mp4
|
||||||
If you cannot see the video in your browser, please click on the following link to download the video
|
If you cannot see the video in your browser, please click on the following link to download the video
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/201-vision-monodepth-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/201-vision-monodepth-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="201-vision-monodepth-with-output_18_0.png">201-vision-monodepth-with-output_18_0.png</a> 26-Jan-2024 01:05 959858
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -5,7 +5,7 @@ Super Resolution is the process of enhancing the quality of an image by
|
||||||
increasing the pixel count using deep learning. This notebook shows the
|
increasing the pixel count using deep learning. This notebook shows the
|
||||||
Single Image Super Resolution (SISR) which takes just one low resolution
|
Single Image Super Resolution (SISR) which takes just one low resolution
|
||||||
image. A model called
|
image. A model called
|
||||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.0/omz_models_model_single_image_super_resolution_1032.html>`__,
|
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.3/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||||
which is available in Open Model Zoo, is used in this tutorial. It is
|
which is available in Open Model Zoo, is used in this tutorial. It is
|
||||||
based on the research paper cited below.
|
based on the research paper cited below.
|
||||||
|
|
||||||
|
|
@ -774,8 +774,8 @@ as total time to process each patch.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Processed 42 patches in 4.63 seconds. Total patches per second (including processing): 9.06.
|
Processed 42 patches in 4.64 seconds. Total patches per second (including processing): 9.05.
|
||||||
Inference patches per second: 17.81
|
Inference patches per second: 17.92
|
||||||
|
|
||||||
|
|
||||||
Save superresolution image and the bicubic image
|
Save superresolution image and the bicubic image
|
||||||
|
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/202-vision-superresolution-image-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/202-vision-superresolution-image-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="202-vision-superresolution-image-with-output_15_1.png">202-vision-superresolution-image-with-output_15..></a> 26-Jan-2024 01:05 272963
|
|
||||||
<a href="202-vision-superresolution-image-with-output_17_1.png">202-vision-superresolution-image-with-output_17..></a> 26-Jan-2024 01:05 356735
|
|
||||||
<a href="202-vision-superresolution-image-with-output_23_1.png">202-vision-superresolution-image-with-output_23..></a> 26-Jan-2024 01:05 2896276
|
|
||||||
<a href="202-vision-superresolution-image-with-output_27_1.png">202-vision-superresolution-image-with-output_27..></a> 26-Jan-2024 01:05 3207711
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -5,7 +5,7 @@ Super Resolution is the process of enhancing the quality of an image by
|
||||||
increasing the pixel count using deep learning. This notebook applies
|
increasing the pixel count using deep learning. This notebook applies
|
||||||
Single Image Super Resolution (SISR) to frames in a 360p (480×360) video
|
Single Image Super Resolution (SISR) to frames in a 360p (480×360) video
|
||||||
in 360p resolution. A model called
|
in 360p resolution. A model called
|
||||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.0/omz_models_model_single_image_super_resolution_1032.html>`__,
|
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.3/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||||
which is available in Open Model Zoo, is used in this tutorial. It is
|
which is available in Open Model Zoo, is used in this tutorial. It is
|
||||||
based on the research paper cited below.
|
based on the research paper cited below.
|
||||||
|
|
||||||
|
|
@ -81,7 +81,7 @@ Imports
|
||||||
|
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import cv2
|
import cv2
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from IPython.display import (
|
from IPython.display import (
|
||||||
|
|
@ -120,7 +120,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import ipywidgets as widgets
|
import ipywidgets as widgets
|
||||||
|
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
device = widgets.Dropdown(
|
device = widgets.Dropdown(
|
||||||
options=core.available_devices + ["AUTO"],
|
options=core.available_devices + ["AUTO"],
|
||||||
|
|
@ -128,7 +128,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
description='Device:',
|
description='Device:',
|
||||||
disabled=False,
|
disabled=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
device
|
device
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -144,20 +144,20 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
# 1032: 4x superresolution, 1033: 3x superresolution
|
# 1032: 4x superresolution, 1033: 3x superresolution
|
||||||
model_name = 'single-image-super-resolution-1032'
|
model_name = 'single-image-super-resolution-1032'
|
||||||
|
|
||||||
base_model_dir = Path('./model').expanduser()
|
base_model_dir = Path('./model').expanduser()
|
||||||
|
|
||||||
model_xml_name = f'{model_name}.xml'
|
model_xml_name = f'{model_name}.xml'
|
||||||
model_bin_name = f'{model_name}.bin'
|
model_bin_name = f'{model_name}.bin'
|
||||||
|
|
||||||
model_xml_path = base_model_dir / model_xml_name
|
model_xml_path = base_model_dir / model_xml_name
|
||||||
model_bin_path = base_model_dir / model_bin_name
|
model_bin_path = base_model_dir / model_bin_name
|
||||||
|
|
||||||
if not model_xml_path.exists():
|
if not model_xml_path.exists():
|
||||||
base_url = f'https://storage.openvinotoolkit.org/repositories/open_model_zoo/2023.0/models_bin/1/{model_name}/FP16/'
|
base_url = f'https://storage.openvinotoolkit.org/repositories/open_model_zoo/2023.0/models_bin/1/{model_name}/FP16/'
|
||||||
model_xml_url = base_url + model_xml_name
|
model_xml_url = base_url + model_xml_name
|
||||||
model_bin_url = base_url + model_bin_name
|
model_bin_url = base_url + model_bin_name
|
||||||
|
|
||||||
download_file(model_xml_url, model_xml_path)
|
download_file(model_xml_url, model_xml_path)
|
||||||
download_file(model_bin_url, model_bin_path)
|
download_file(model_bin_url, model_bin_path)
|
||||||
else:
|
else:
|
||||||
|
|
@ -180,7 +180,7 @@ Functions
|
||||||
"""
|
"""
|
||||||
Convert network result of floating point numbers to image with integer
|
Convert network result of floating point numbers to image with integer
|
||||||
values from 0-255. Values outside this range are clipped to 0 and 255.
|
values from 0-255. Values outside this range are clipped to 0 and 255.
|
||||||
|
|
||||||
:param result: a single superresolution network result in N,C,H,W shape
|
:param result: a single superresolution network result in N,C,H,W shape
|
||||||
"""
|
"""
|
||||||
result = result.squeeze(0).transpose(1, 2, 0)
|
result = result.squeeze(0).transpose(1, 2, 0)
|
||||||
|
|
@ -215,18 +215,18 @@ resolution version of the image in 1920x1080.
|
||||||
# dictionaries.
|
# dictionaries.
|
||||||
original_image_key, bicubic_image_key = compiled_model.inputs
|
original_image_key, bicubic_image_key = compiled_model.inputs
|
||||||
output_key = compiled_model.output(0)
|
output_key = compiled_model.output(0)
|
||||||
|
|
||||||
# Get the expected input and target shape. The `.dims[2:]` function returns the height
|
# Get the expected input and target shape. The `.dims[2:]` function returns the height
|
||||||
# and width.The `resize` function of OpenCV expects the shape as (width, height),
|
# and width.The `resize` function of OpenCV expects the shape as (width, height),
|
||||||
# so reverse the shape with `[::-1]` and convert it to a tuple.
|
# so reverse the shape with `[::-1]` and convert it to a tuple.
|
||||||
input_height, input_width = list(original_image_key.shape)[2:]
|
input_height, input_width = list(original_image_key.shape)[2:]
|
||||||
target_height, target_width = list(bicubic_image_key.shape)[2:]
|
target_height, target_width = list(bicubic_image_key.shape)[2:]
|
||||||
|
|
||||||
upsample_factor = int(target_height / input_height)
|
upsample_factor = int(target_height / input_height)
|
||||||
|
|
||||||
print(f"The network expects inputs with a width of {input_width}, " f"height of {input_height}")
|
print(f"The network expects inputs with a width of {input_width}, " f"height of {input_height}")
|
||||||
print(f"The network returns images with a width of {target_width}, " f"height of {target_height}")
|
print(f"The network returns images with a width of {target_width}, " f"height of {target_height}")
|
||||||
|
|
||||||
print(
|
print(
|
||||||
f"The image sides are upsampled by a factor of {upsample_factor}. "
|
f"The image sides are upsampled by a factor of {upsample_factor}. "
|
||||||
f"The new image is {upsample_factor**2} times as large as the "
|
f"The new image is {upsample_factor**2} times as large as the "
|
||||||
|
|
@ -264,7 +264,7 @@ Settings
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
OUTPUT_DIR = "output"
|
OUTPUT_DIR = "output"
|
||||||
|
|
||||||
Path(OUTPUT_DIR).mkdir(exist_ok=True)
|
Path(OUTPUT_DIR).mkdir(exist_ok=True)
|
||||||
# Maximum number of frames to read from the input video. Set to 0 to read all frames.
|
# Maximum number of frames to read from the input video. Set to 0 to read all frames.
|
||||||
NUM_FRAMES = 100
|
NUM_FRAMES = 100
|
||||||
|
|
@ -290,10 +290,10 @@ Download and Prepare Video
|
||||||
filename = Path(stream.default_filename.encode("ascii", "ignore").decode("ascii")).stem
|
filename = Path(stream.default_filename.encode("ascii", "ignore").decode("ascii")).stem
|
||||||
stream.download(output_path=OUTPUT_DIR, filename=filename)
|
stream.download(output_path=OUTPUT_DIR, filename=filename)
|
||||||
print(f"Video {filename} downloaded to {OUTPUT_DIR}")
|
print(f"Video {filename} downloaded to {OUTPUT_DIR}")
|
||||||
|
|
||||||
# Create Path objects for the input video and the resulting videos.
|
# Create Path objects for the input video and the resulting videos.
|
||||||
video_path = Path(stream.get_file_path(filename, OUTPUT_DIR))
|
video_path = Path(stream.get_file_path(filename, OUTPUT_DIR))
|
||||||
|
|
||||||
# Path names for the result videos.
|
# Path names for the result videos.
|
||||||
superres_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_superres.mp4")
|
superres_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_superres.mp4")
|
||||||
bicubic_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_bicubic.mp4")
|
bicubic_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_bicubic.mp4")
|
||||||
|
|
@ -314,14 +314,14 @@ Download and Prepare Video
|
||||||
raise ValueError(f"The video at '{video_path}' cannot be read.")
|
raise ValueError(f"The video at '{video_path}' cannot be read.")
|
||||||
fps = cap.get(cv2.CAP_PROP_FPS)
|
fps = cap.get(cv2.CAP_PROP_FPS)
|
||||||
frame_count = cap.get(cv2.CAP_PROP_FRAME_COUNT)
|
frame_count = cap.get(cv2.CAP_PROP_FRAME_COUNT)
|
||||||
|
|
||||||
if NUM_FRAMES == 0:
|
if NUM_FRAMES == 0:
|
||||||
total_frames = frame_count
|
total_frames = frame_count
|
||||||
else:
|
else:
|
||||||
total_frames = min(frame_count, NUM_FRAMES)
|
total_frames = min(frame_count, NUM_FRAMES)
|
||||||
|
|
||||||
original_frame_height, original_frame_width = image.shape[:2]
|
original_frame_height, original_frame_width = image.shape[:2]
|
||||||
|
|
||||||
cap.release()
|
cap.release()
|
||||||
print(
|
print(
|
||||||
f"The input video has a frame width of {original_frame_width}, "
|
f"The input video has a frame width of {original_frame_width}, "
|
||||||
|
|
@ -389,10 +389,10 @@ video.
|
||||||
start_time = time.perf_counter()
|
start_time = time.perf_counter()
|
||||||
frame_nr = 0
|
frame_nr = 0
|
||||||
total_inference_duration = 0
|
total_inference_duration = 0
|
||||||
|
|
||||||
progress_bar = ProgressBar(total=total_frames)
|
progress_bar = ProgressBar(total=total_frames)
|
||||||
progress_bar.display()
|
progress_bar.display()
|
||||||
|
|
||||||
cap = cv2.VideoCapture(filename=str(video_path))
|
cap = cv2.VideoCapture(filename=str(video_path))
|
||||||
try:
|
try:
|
||||||
while cap.isOpened():
|
while cap.isOpened():
|
||||||
|
|
@ -400,22 +400,22 @@ video.
|
||||||
if not ret:
|
if not ret:
|
||||||
cap.release()
|
cap.release()
|
||||||
break
|
break
|
||||||
|
|
||||||
if frame_nr >= total_frames:
|
if frame_nr >= total_frames:
|
||||||
break
|
break
|
||||||
|
|
||||||
# Resize the input image to the network shape and convert it from (H,W,C) to
|
# Resize the input image to the network shape and convert it from (H,W,C) to
|
||||||
# (N,C,H,W).
|
# (N,C,H,W).
|
||||||
resized_image = cv2.resize(src=image, dsize=(input_width, input_height))
|
resized_image = cv2.resize(src=image, dsize=(input_width, input_height))
|
||||||
input_image_original = np.expand_dims(resized_image.transpose(2, 0, 1), axis=0)
|
input_image_original = np.expand_dims(resized_image.transpose(2, 0, 1), axis=0)
|
||||||
|
|
||||||
# Resize and reshape the image to the target shape with bicubic
|
# Resize and reshape the image to the target shape with bicubic
|
||||||
# interpolation.
|
# interpolation.
|
||||||
bicubic_image = cv2.resize(
|
bicubic_image = cv2.resize(
|
||||||
src=image, dsize=(target_width, target_height), interpolation=cv2.INTER_CUBIC
|
src=image, dsize=(target_width, target_height), interpolation=cv2.INTER_CUBIC
|
||||||
)
|
)
|
||||||
input_image_bicubic = np.expand_dims(bicubic_image.transpose(2, 0, 1), axis=0)
|
input_image_bicubic = np.expand_dims(bicubic_image.transpose(2, 0, 1), axis=0)
|
||||||
|
|
||||||
# Do inference.
|
# Do inference.
|
||||||
inference_start_time = time.perf_counter()
|
inference_start_time = time.perf_counter()
|
||||||
result = compiled_model(
|
result = compiled_model(
|
||||||
|
|
@ -427,19 +427,19 @@ video.
|
||||||
inference_stop_time = time.perf_counter()
|
inference_stop_time = time.perf_counter()
|
||||||
inference_duration = inference_stop_time - inference_start_time
|
inference_duration = inference_stop_time - inference_start_time
|
||||||
total_inference_duration += inference_duration
|
total_inference_duration += inference_duration
|
||||||
|
|
||||||
# Transform the inference result into an image.
|
# Transform the inference result into an image.
|
||||||
result_frame = convert_result_to_image(result=result)
|
result_frame = convert_result_to_image(result=result)
|
||||||
|
|
||||||
# Write the result image and the bicubic image to a video file.
|
# Write the result image and the bicubic image to a video file.
|
||||||
superres_video.write(image=result_frame)
|
superres_video.write(image=result_frame)
|
||||||
bicubic_video.write(image=bicubic_image)
|
bicubic_video.write(image=bicubic_image)
|
||||||
|
|
||||||
stacked_frame = np.hstack((bicubic_image, result_frame))
|
stacked_frame = np.hstack((bicubic_image, result_frame))
|
||||||
comparison_video.write(image=stacked_frame)
|
comparison_video.write(image=stacked_frame)
|
||||||
|
|
||||||
frame_nr = frame_nr + 1
|
frame_nr = frame_nr + 1
|
||||||
|
|
||||||
# Update the progress bar and the status message.
|
# Update the progress bar and the status message.
|
||||||
progress_bar.progress = frame_nr
|
progress_bar.progress = frame_nr
|
||||||
progress_bar.update()
|
progress_bar.update()
|
||||||
|
|
@ -453,8 +453,8 @@ video.
|
||||||
f"({1/inference_duration:.2f} FPS)"
|
f"({1/inference_duration:.2f} FPS)"
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("Processing interrupted.")
|
print("Processing interrupted.")
|
||||||
finally:
|
finally:
|
||||||
|
|
@ -480,13 +480,13 @@ video.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Processed frame 100. Inference time: 0.05 seconds (19.51 FPS)
|
Processed frame 100. Inference time: 0.06 seconds (17.00 FPS)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Video's saved to output directory.
|
Video's saved to output directory.
|
||||||
Processed 100 frames in 242.29 seconds. Total FPS (including video processing): 0.41. Inference FPS: 18.44.
|
Processed 100 frames in 243.08 seconds. Total FPS (including video processing): 0.41. Inference FPS: 17.69.
|
||||||
|
|
||||||
|
|
||||||
Show Side-by-Side Video of Bicubic and Superresolution Version
|
Show Side-by-Side Video of Bicubic and Superresolution Version
|
||||||
|
|
|
||||||
|
|
@ -623,7 +623,7 @@ bounds of input batch size.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7f3c87e408e0>
|
<matplotlib.image.AxesImage at 0x7f476c1d6fd0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -788,7 +788,7 @@ Get the reading result on the meter picture
|
||||||
.. image:: 203-meter-reader-with-output_files/203-meter-reader-with-output_24_1.png
|
.. image:: 203-meter-reader-with-output_files/203-meter-reader-with-output_24_1.png
|
||||||
|
|
||||||
|
|
||||||
Try it with your meter photos
|
Try it with your meter photos!
|
||||||
-----------------------------------------------
|
------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/203-meter-reader-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/203-meter-reader-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="203-meter-reader-with-output_16_1.png">203-meter-reader-with-output_16_1.png</a> 26-Jan-2024 01:04 170121
|
|
||||||
<a href="203-meter-reader-with-output_18_1.png">203-meter-reader-with-output_18_1.png</a> 26-Jan-2024 01:04 190271
|
|
||||||
<a href="203-meter-reader-with-output_20_1.png">203-meter-reader-with-output_20_1.png</a> 26-Jan-2024 01:04 26914
|
|
||||||
<a href="203-meter-reader-with-output_22_1.png">203-meter-reader-with-output_22_1.png</a> 26-Jan-2024 01:04 8966
|
|
||||||
<a href="203-meter-reader-with-output_24_1.png">203-meter-reader-with-output_24_1.png</a> 26-Jan-2024 01:04 170338
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -26,20 +26,31 @@ paper: `Segmenter: Transformer for Semantic
|
||||||
Segmentation <https://arxiv.org/abs/2105.05633>`__ or in the
|
Segmentation <https://arxiv.org/abs/2105.05633>`__ or in the
|
||||||
`repository <https://github.com/rstrudel/segmenter>`__.
|
`repository <https://github.com/rstrudel/segmenter>`__.
|
||||||
|
|
||||||
|
Table of contents:
|
||||||
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
**Table of contents:**
|
- `Get and prepare PyTorch model <#get-and-prepare-pytorch-model>`__
|
||||||
|
|
||||||
- `Get and prepare PyTorch model <#get-and-prepare-pytorch-model>`__
|
- `Prerequisites <#prerequisites>`__
|
||||||
- `Prerequisites <#prerequisites>`__
|
- `Loading PyTorch model <#loading-pytorch-model>`__
|
||||||
- `Loading PyTorch model <#loading-pytorch-model>`__
|
|
||||||
- `Preparing preprocessing and visualization functions <#preparing-preprocessing-and-visualization-functions>`__
|
- `Preparing preprocessing and visualization
|
||||||
- `Preprocessing <#preprocessing>`__
|
functions <#preparing-preprocessing-and-visualization-functions>`__
|
||||||
- `visualization <#visualization>`__
|
|
||||||
- `Validation of inference of original model <#validation-of-inference-of-original-model>`__
|
- `Preprocessing <#preprocessing>`__
|
||||||
- `Convert PyTorch model to OpenVINO Intermediate Representation (IR) <#convert-pytorch-model-to-openvino-intermediate-representation-ir>`__
|
- `Visualization <#visualization>`__
|
||||||
- `Verify converted model inference <#verify-converted-model-inference>`__
|
|
||||||
- `Select inference device <#select-inference-device>`__
|
- `Validation of inference of original
|
||||||
- `Benchmarking performance of converted model <#benchmarking-performance-of-converted-model>`__
|
model <#validation-of-inference-of-original-model>`__
|
||||||
|
- `Convert PyTorch model to OpenVINO Intermediate Representation
|
||||||
|
(IR) <#convert-pytorch-model-to-openvino-intermediate-representation-ir>`__
|
||||||
|
- `Verify converted model
|
||||||
|
inference <#verify-converted-model-inference>`__
|
||||||
|
|
||||||
|
- `Select inference device <#select-inference-device>`__
|
||||||
|
|
||||||
|
- `Benchmarking performance of converted
|
||||||
|
model <#benchmarking-performance-of-converted-model>`__
|
||||||
|
|
||||||
.. |Segmenter diagram| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/93932510/f57979e7-fd3b-449f-bf01-afe0f965abbc
|
.. |Segmenter diagram| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/93932510/f57979e7-fd3b-449f-bf01-afe0f965abbc
|
||||||
|
|
||||||
|
|
@ -79,13 +90,13 @@ Prerequisites
|
||||||
|
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# clone Segmenter repo
|
# clone Segmenter repo
|
||||||
if not Path("segmenter").exists():
|
if not Path("segmenter").exists():
|
||||||
!git clone https://github.com/rstrudel/segmenter
|
!git clone https://github.com/rstrudel/segmenter
|
||||||
else:
|
else:
|
||||||
print("Segmenter repo already cloned")
|
print("Segmenter repo already cloned")
|
||||||
|
|
||||||
# include path to Segmenter repo to use its functions
|
# include path to Segmenter repo to use its functions
|
||||||
sys.path.append("./segmenter")
|
sys.path.append("./segmenter")
|
||||||
|
|
||||||
|
|
@ -108,10 +119,7 @@ Receiving objects: 6% (17/268)
|
||||||
Receiving objects: 7% (19/268)
|
Receiving objects: 7% (19/268)
|
||||||
Receiving objects: 8% (22/268)
|
Receiving objects: 8% (22/268)
|
||||||
Receiving objects: 9% (25/268)
|
Receiving objects: 9% (25/268)
|
||||||
|
Receiving objects: 10% (27/268)
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
Receiving objects: 10% (27/268)
|
|
||||||
Receiving objects: 11% (30/268)
|
Receiving objects: 11% (30/268)
|
||||||
Receiving objects: 12% (33/268)
|
Receiving objects: 12% (33/268)
|
||||||
Receiving objects: 13% (35/268)
|
Receiving objects: 13% (35/268)
|
||||||
|
|
@ -135,118 +143,122 @@ Receiving objects: 22% (59/268)
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 24% (65/268), 2.55 MiB | 2.48 MiB/s
|
Receiving objects: 24% (65/268), 3.68 MiB | 3.63 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 24% (66/268), 6.21 MiB | 3.02 MiB/s
|
Receiving objects: 24% (66/268), 7.47 MiB | 3.70 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 25% (67/268), 6.21 MiB | 3.02 MiB/s
|
Receiving objects: 25% (67/268), 7.47 MiB | 3.70 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 25% (68/268), 9.88 MiB | 3.19 MiB/s
|
Receiving objects: 25% (68/268), 11.26 MiB | 3.73 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 26% (70/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 26% (70/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 27% (73/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 27% (73/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 28% (76/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 28% (76/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 29% (78/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 29% (78/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 30% (81/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 30% (81/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 31% (84/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 31% (84/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 32% (86/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 32% (86/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 33% (89/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 33% (89/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 34% (92/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 34% (92/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 35% (94/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 35% (94/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 36% (97/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 36% (97/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 37% (100/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 37% (100/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 38% (102/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 38% (102/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 39% (105/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 39% (105/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 40% (108/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 40% (108/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 41% (110/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 41% (110/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 42% (113/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 42% (113/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 43% (116/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 43% (116/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 44% (118/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 44% (118/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 45% (121/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 45% (121/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 46% (124/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 46% (124/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 47% (126/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 47% (126/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 48% (129/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 48% (129/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 49% (132/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 49% (132/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 50% (134/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 50% (134/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 51% (137/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 51% (137/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 52% (140/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 52% (140/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 53% (143/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 53% (143/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 54% (145/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 54% (145/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 55% (148/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 55% (148/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 56% (151/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 56% (151/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 57% (153/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 57% (153/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 58% (156/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 58% (156/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 59% (159/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 59% (159/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 60% (161/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 60% (161/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 61% (164/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 61% (164/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 62% (167/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 62% (167/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 63% (169/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 63% (169/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 64% (172/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 64% (172/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 65% (175/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 65% (175/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 66% (177/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 66% (177/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 67% (180/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 67% (180/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 68% (183/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 68% (183/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 69% (185/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 69% (185/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 70% (188/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 70% (188/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 71% (191/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 71% (191/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 72% (193/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 72% (193/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 73% (196/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 73% (196/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 74% (199/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 74% (199/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 75% (201/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 75% (201/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 76% (204/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 76% (204/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 77% (207/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 77% (207/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 78% (210/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 78% (210/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 79% (212/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 79% (212/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 80% (215/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 80% (215/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 81% (218/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 81% (218/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 82% (220/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 82% (220/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 83% (223/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 83% (223/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 84% (226/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 84% (226/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 85% (228/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 85% (228/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 86% (231/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 86% (231/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 87% (234/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 87% (234/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 88% (236/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 88% (236/268), 11.26 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 89% (239/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 89% (239/268), 11.26 MiB | 3.73 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 90% (242/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 90% (242/268), 11.26 MiB | 3.73 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 91% (244/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 91% (244/268), 11.26 MiB | 3.73 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 91% (246/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 92% (247/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 92% (247/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 93% (250/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 93% (250/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 94% (252/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 94% (252/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 95% (255/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 95% (255/268), 11.71 MiB | 3.24 MiB/s
|
Receiving objects: 96% (258/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 96% (258/268), 11.71 MiB | 3.24 MiB/s
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Receiving objects: 97% (260/268), 13.54 MiB | 3.28 MiB/s
|
Receiving objects: 97% (260/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 98% (263/268), 13.54 MiB | 3.28 MiB/s
|
Receiving objects: 98% (263/268), 13.11 MiB | 3.72 MiB/s
|
||||||
Receiving objects: 99% (266/268), 13.54 MiB | 3.28 MiB/s
|
Receiving objects: 99% (266/268), 13.11 MiB | 3.72 MiB/s
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
Receiving objects: 99% (267/268), 15.03 MiB | 3.73 MiB/s
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
remote: Total 268 (delta 0), reused 0 (delta 0), pack-reused 268[K
|
remote: Total 268 (delta 0), reused 0 (delta 0), pack-reused 268[K
|
||||||
Receiving objects: 100% (268/268), 15.25 MiB | 3.29 MiB/s
|
Receiving objects: 100% (268/268), 15.03 MiB | 3.73 MiB/s
|
||||||
Receiving objects: 100% (268/268), 15.34 MiB | 3.30 MiB/s, done.
|
Receiving objects: 100% (268/268), 15.34 MiB | 3.73 MiB/s, done.
|
||||||
Resolving deltas: 0% (0/117)
|
Resolving deltas: 0% (0/117)
|
||||||
Resolving deltas: 1% (2/117)
|
Resolving deltas: 1% (2/117)
|
||||||
Resolving deltas: 2% (3/117)
|
Resolving deltas: 2% (3/117)
|
||||||
|
Resolving deltas: 3% (4/117)
|
||||||
Resolving deltas: 5% (6/117)
|
Resolving deltas: 5% (6/117)
|
||||||
Resolving deltas: 7% (9/117)
|
Resolving deltas: 7% (9/117)
|
||||||
Resolving deltas: 8% (10/117)
|
Resolving deltas: 8% (10/117)
|
||||||
|
|
@ -254,15 +266,15 @@ Resolving deltas: 9% (11/117)
|
||||||
Resolving deltas: 10% (12/117)
|
Resolving deltas: 10% (12/117)
|
||||||
Resolving deltas: 11% (14/117)
|
Resolving deltas: 11% (14/117)
|
||||||
Resolving deltas: 13% (16/117)
|
Resolving deltas: 13% (16/117)
|
||||||
Resolving deltas: 14% (17/117)
|
Resolving deltas: 15% (18/117)
|
||||||
Resolving deltas: 26% (31/117)
|
Resolving deltas: 26% (31/117)
|
||||||
Resolving deltas: 32% (38/117)
|
Resolving deltas: 33% (39/117)
|
||||||
|
Resolving deltas: 54% (64/117)
|
||||||
Resolving deltas: 56% (66/117)
|
Resolving deltas: 56% (66/117)
|
||||||
Resolving deltas: 58% (69/117)
|
Resolving deltas: 76% (90/117)
|
||||||
Resolving deltas: 73% (86/117)
|
Resolving deltas: 80% (94/117)
|
||||||
Resolving deltas: 76% (89/117)
|
|
||||||
Resolving deltas: 77% (91/117)
|
|
||||||
Resolving deltas: 81% (95/117)
|
Resolving deltas: 81% (95/117)
|
||||||
|
Resolving deltas: 82% (96/117)
|
||||||
Resolving deltas: 100% (117/117)
|
Resolving deltas: 100% (117/117)
|
||||||
Resolving deltas: 100% (117/117), done.
|
Resolving deltas: 100% (117/117), done.
|
||||||
|
|
||||||
|
|
@ -281,9 +293,9 @@ Resolving deltas: 100% (117/117), done.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: torch in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 1)) (2.1.0+cpu)
|
Requirement already satisfied: torch in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 1)) (2.1.0+cpu)
|
||||||
Requirement already satisfied: click in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 2)) (8.1.7)
|
Requirement already satisfied: click in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 2)) (8.1.7)
|
||||||
Requirement already satisfied: numpy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 3)) (1.23.5)
|
Requirement already satisfied: numpy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 3)) (1.23.5)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -296,9 +308,9 @@ Resolving deltas: 100% (117/117), done.
|
||||||
|
|
||||||
Collecting python-hostlist (from -r segmenter/requirements.txt (line 5))
|
Collecting python-hostlist (from -r segmenter/requirements.txt (line 5))
|
||||||
Using cached python_hostlist-1.23.0-py3-none-any.whl
|
Using cached python_hostlist-1.23.0-py3-none-any.whl
|
||||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 6)) (4.66.1)
|
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 6)) (4.66.1)
|
||||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 7)) (2.31.0)
|
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 7)) (2.31.0)
|
||||||
Requirement already satisfied: pyyaml in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 8)) (6.0.1)
|
Requirement already satisfied: pyyaml in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 8)) (6.0.1)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -316,49 +328,57 @@ Resolving deltas: 100% (117/117), done.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Collecting mmsegmentation==0.14.1 (from -r segmenter/requirements.txt (line 11))
|
Collecting mmsegmentation==0.14.1 (from -r segmenter/requirements.txt (line 11))
|
||||||
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
Using cached mmsegmentation-0.14.1-py3-none-any.whl (201 kB)
|
Using cached mmsegmentation-0.14.1-py3-none-any.whl (201 kB)
|
||||||
Requirement already satisfied: torchvision in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from timm==0.4.12->-r segmenter/requirements.txt (line 9)) (0.16.0+cpu)
|
Requirement already satisfied: torchvision in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from timm==0.4.12->-r segmenter/requirements.txt (line 9)) (0.16.0+cpu)
|
||||||
Requirement already satisfied: addict in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (2.4.0)
|
Requirement already satisfied: addict in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (2.4.0)
|
||||||
Requirement already satisfied: Pillow in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (10.2.0)
|
Requirement already satisfied: Pillow in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (10.2.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Collecting yapf (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10))
|
Collecting yapf (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10))
|
||||||
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
Using cached yapf-0.40.2-py3-none-any.whl.metadata (45 kB)
|
Using cached yapf-0.40.2-py3-none-any.whl.metadata (45 kB)
|
||||||
Requirement already satisfied: matplotlib in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.7.4)
|
Requirement already satisfied: matplotlib in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.7.4)
|
||||||
Requirement already satisfied: prettytable in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.9.0)
|
Requirement already satisfied: prettytable in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.9.0)
|
||||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.13.1)
|
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.13.1)
|
||||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (4.9.0)
|
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (4.9.0)
|
||||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (1.12)
|
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (1.12)
|
||||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1)
|
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1)
|
||||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1.3)
|
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1.3)
|
||||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (2023.10.0)
|
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (2023.10.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.3.2)
|
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.3.2)
|
||||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.6)
|
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.6)
|
||||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2.1.0)
|
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2.2.0)
|
||||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2023.11.17)
|
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2024.2.2)
|
||||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch->-r segmenter/requirements.txt (line 1)) (2.1.4)
|
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch->-r segmenter/requirements.txt (line 1)) (2.1.5)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: contourpy>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.1.1)
|
Requirement already satisfied: contourpy>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.1.1)
|
||||||
Requirement already satisfied: cycler>=0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.12.1)
|
Requirement already satisfied: cycler>=0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.12.1)
|
||||||
Requirement already satisfied: fonttools>=4.22.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (4.47.2)
|
Requirement already satisfied: fonttools>=4.22.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (4.48.1)
|
||||||
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.4.5)
|
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.4.5)
|
||||||
Requirement already satisfied: packaging>=20.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (23.2)
|
Requirement already satisfied: packaging>=20.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (23.2)
|
||||||
Requirement already satisfied: pyparsing>=2.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.1.1)
|
Requirement already satisfied: pyparsing>=2.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.1.1)
|
||||||
Requirement already satisfied: python-dateutil>=2.7 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (2.8.2)
|
Requirement already satisfied: python-dateutil>=2.7 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (2.8.2)
|
||||||
Requirement already satisfied: importlib-resources>=3.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (6.1.1)
|
Requirement already satisfied: importlib-resources>=3.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (6.1.1)
|
||||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.2.13)
|
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.2.13)
|
||||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch->-r segmenter/requirements.txt (line 1)) (1.3.0)
|
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch->-r segmenter/requirements.txt (line 1)) (1.3.0)
|
||||||
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (7.0.1)
|
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (7.0.1)
|
||||||
Requirement already satisfied: platformdirs>=3.5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (4.1.0)
|
Requirement already satisfied: platformdirs>=3.5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (4.2.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -369,8 +389,12 @@ Resolving deltas: 100% (117/117), done.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Requirement already satisfied: zipp>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from importlib-metadata>=6.6.0->yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (3.17.0)
|
Requirement already satisfied: zipp>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from importlib-metadata>=6.6.0->yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (3.17.0)
|
||||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.16.0)
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
|
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.16.0)
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -408,7 +432,7 @@ Resolving deltas: 100% (117/117), done.
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
# Fetch the notebook utils script from the openvino_notebooks repo
|
# Fetch the notebook utils script from the openvino_notebooks repo
|
||||||
import urllib.request
|
import urllib.request
|
||||||
urllib.request.urlretrieve(
|
urllib.request.urlretrieve(
|
||||||
|
|
@ -429,13 +453,13 @@ config for our model.
|
||||||
# here we use tiny model, there are also better but larger models available in repository
|
# here we use tiny model, there are also better but larger models available in repository
|
||||||
WEIGHTS_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/checkpoint.pth"
|
WEIGHTS_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/checkpoint.pth"
|
||||||
CONFIG_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/variant.yml"
|
CONFIG_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/variant.yml"
|
||||||
|
|
||||||
MODEL_DIR = Path("model/")
|
MODEL_DIR = Path("model/")
|
||||||
MODEL_DIR.mkdir(exist_ok=True)
|
MODEL_DIR.mkdir(exist_ok=True)
|
||||||
|
|
||||||
download_file(WEIGHTS_LINK, directory=MODEL_DIR, show_progress=True)
|
download_file(WEIGHTS_LINK, directory=MODEL_DIR, show_progress=True)
|
||||||
download_file(CONFIG_LINK, directory=MODEL_DIR, show_progress=True)
|
download_file(CONFIG_LINK, directory=MODEL_DIR, show_progress=True)
|
||||||
|
|
||||||
WEIGHT_PATH = MODEL_DIR / "checkpoint.pth"
|
WEIGHT_PATH = MODEL_DIR / "checkpoint.pth"
|
||||||
CONFIG_PATH = MODEL_DIR / "variant.yaml"
|
CONFIG_PATH = MODEL_DIR / "variant.yaml"
|
||||||
|
|
||||||
|
|
@ -473,7 +497,7 @@ initialize the model.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from segmenter.segm.model.factory import load_model
|
from segmenter.segm.model.factory import load_model
|
||||||
|
|
||||||
pytorch_model, config = load_model(WEIGHT_PATH)
|
pytorch_model, config = load_model(WEIGHT_PATH)
|
||||||
# put model into eval mode, to set it for inference
|
# put model into eval mode, to set it for inference
|
||||||
pytorch_model.eval()
|
pytorch_model.eval()
|
||||||
|
|
@ -536,12 +560,12 @@ normalized with given mean and standard deviation provided in
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
import torch
|
import torch
|
||||||
import torchvision.transforms.functional as F
|
import torchvision.transforms.functional as F
|
||||||
|
|
||||||
|
|
||||||
def preprocess(im: Image, normalization: dict) -> torch.Tensor:
|
def preprocess(im: Image, normalization: dict) -> torch.Tensor:
|
||||||
"""
|
"""
|
||||||
Preprocess image: scale, normalize and unsqueeze
|
Preprocess image: scale, normalize and unsqueeze
|
||||||
|
|
||||||
:param im: input image
|
:param im: input image
|
||||||
:param normalization: dictionary containing normalization data from config file
|
:param normalization: dictionary containing normalization data from config file
|
||||||
:return:
|
:return:
|
||||||
|
|
@ -553,7 +577,7 @@ normalized with given mean and standard deviation provided in
|
||||||
im = F.normalize(im, normalization["mean"], normalization["std"])
|
im = F.normalize(im, normalization["mean"], normalization["std"])
|
||||||
# change dim from [C, H, W] to [1, C, H, W]
|
# change dim from [C, H, W] to [1, C, H, W]
|
||||||
im = im.unsqueeze(0)
|
im = im.unsqueeze(0)
|
||||||
|
|
||||||
return im
|
return im
|
||||||
|
|
||||||
Visualization
|
Visualization
|
||||||
|
|
@ -577,29 +601,29 @@ corresponding to the inferred labels.
|
||||||
|
|
||||||
from segmenter.segm.data.utils import dataset_cat_description, seg_to_rgb
|
from segmenter.segm.data.utils import dataset_cat_description, seg_to_rgb
|
||||||
from segmenter.segm.data.ade20k import ADE20K_CATS_PATH
|
from segmenter.segm.data.ade20k import ADE20K_CATS_PATH
|
||||||
|
|
||||||
|
|
||||||
def apply_segmentation_mask(pil_im: Image, results: torch.Tensor) -> Image:
|
def apply_segmentation_mask(pil_im: Image, results: torch.Tensor) -> Image:
|
||||||
"""
|
"""
|
||||||
Combine segmentation masks with the image
|
Combine segmentation masks with the image
|
||||||
|
|
||||||
:param pil_im: original input image
|
:param pil_im: original input image
|
||||||
:param results: tensor containing segmentation masks for each pixel
|
:param results: tensor containing segmentation masks for each pixel
|
||||||
:return:
|
:return:
|
||||||
pil_blend: image with colored segmentation masks overlay
|
pil_blend: image with colored segmentation masks overlay
|
||||||
"""
|
"""
|
||||||
cat_names, cat_colors = dataset_cat_description(ADE20K_CATS_PATH)
|
cat_names, cat_colors = dataset_cat_description(ADE20K_CATS_PATH)
|
||||||
|
|
||||||
# 3D array, where each pixel has values for all classes, take index of max as label
|
# 3D array, where each pixel has values for all classes, take index of max as label
|
||||||
seg_map = results.argmax(0, keepdim=True)
|
seg_map = results.argmax(0, keepdim=True)
|
||||||
# transform label id to colors
|
# transform label id to colors
|
||||||
seg_rgb = seg_to_rgb(seg_map, cat_colors)
|
seg_rgb = seg_to_rgb(seg_map, cat_colors)
|
||||||
seg_rgb = (255 * seg_rgb.cpu().numpy()).astype(np.uint8)
|
seg_rgb = (255 * seg_rgb.cpu().numpy()).astype(np.uint8)
|
||||||
pil_seg = Image.fromarray(seg_rgb[0])
|
pil_seg = Image.fromarray(seg_rgb[0])
|
||||||
|
|
||||||
# overlay segmentation mask over original image
|
# overlay segmentation mask over original image
|
||||||
pil_blend = Image.blend(pil_im, pil_seg, 0.5).convert("RGB")
|
pil_blend = Image.blend(pil_im, pil_seg, 0.5).convert("RGB")
|
||||||
|
|
||||||
return pil_blend
|
return pil_blend
|
||||||
|
|
||||||
Validation of inference of original model
|
Validation of inference of original model
|
||||||
|
|
@ -613,15 +637,15 @@ example image ``coco_hollywood.jpg``.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
from segmenter.segm.model.utils import inference
|
from segmenter.segm.model.utils import inference
|
||||||
|
|
||||||
# load image with PIL
|
# load image with PIL
|
||||||
image = load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_hollywood.jpg")
|
image = load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_hollywood.jpg")
|
||||||
# load_image reads the image in BGR format, [:,:,::-1] reshape transfroms it to RGB
|
# load_image reads the image in BGR format, [:,:,::-1] reshape transfroms it to RGB
|
||||||
pil_image = Image.fromarray(image[:,:,::-1])
|
pil_image = Image.fromarray(image[:,:,::-1])
|
||||||
|
|
||||||
# preprocess image with normalization params loaded in previous steps
|
# preprocess image with normalization params loaded in previous steps
|
||||||
image = preprocess(pil_image, normalization)
|
image = preprocess(pil_image, normalization)
|
||||||
|
|
||||||
# inference function needs some meta parameters, where we specify that we don't flip images in inference mode
|
# inference function needs some meta parameters, where we specify that we don't flip images in inference mode
|
||||||
im_meta = dict(flip=False)
|
im_meta = dict(flip=False)
|
||||||
# perform inference with function from repository
|
# perform inference with function from repository
|
||||||
|
|
@ -641,7 +665,7 @@ previous steps.
|
||||||
|
|
||||||
# combine segmentation mask with image
|
# combine segmentation mask with image
|
||||||
blended_image = apply_segmentation_mask(pil_image, original_results)
|
blended_image = apply_segmentation_mask(pil_image, original_results)
|
||||||
|
|
||||||
# show image with segmentation mask overlay
|
# show image with segmentation mask overlay
|
||||||
blended_image
|
blended_image
|
||||||
|
|
||||||
|
|
@ -688,15 +712,15 @@ they are not a problem.
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
|
|
||||||
# get input sizes from config file
|
# get input sizes from config file
|
||||||
batch_size = 2
|
batch_size = 2
|
||||||
channels = 3
|
channels = 3
|
||||||
image_size = config["dataset_kwargs"]["image_size"]
|
image_size = config["dataset_kwargs"]["image_size"]
|
||||||
|
|
||||||
# make dummy input with correct shapes obtained from config file
|
# make dummy input with correct shapes obtained from config file
|
||||||
dummy_input = torch.randn(batch_size, channels, image_size, image_size)
|
dummy_input = torch.randn(batch_size, channels, image_size, image_size)
|
||||||
|
|
||||||
model = ov.convert_model(pytorch_model, example_input=dummy_input, input=([batch_size, channels, image_size, image_size], ))
|
model = ov.convert_model(pytorch_model, example_input=dummy_input, input=([batch_size, channels, image_size, image_size], ))
|
||||||
# serialize model for saving IR
|
# serialize model for saving IR
|
||||||
ov.save_model(model, MODEL_DIR / "segmenter.xml")
|
ov.save_model(model, MODEL_DIR / "segmenter.xml")
|
||||||
|
|
@ -704,21 +728,21 @@ they are not a problem.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if H % patch_size > 0:
|
if H % patch_size > 0:
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if W % patch_size > 0:
|
if W % patch_size > 0:
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/vit.py:122: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/vit.py:122: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if x.shape[1] != pos_embed.shape[1]:
|
if x.shape[1] != pos_embed.shape[1]:
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/decoder.py:100: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/decoder.py:100: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
masks = rearrange(masks, "b (h w) n -> b n h w", h=int(GS))
|
masks = rearrange(masks, "b (h w) n -> b n h w", h=int(GS))
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if extra_h > 0:
|
if extra_h > 0:
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||||
if extra_w > 0:
|
if extra_w > 0:
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -739,7 +763,7 @@ any additional custom code required to process input.
|
||||||
class SegmenterOV:
|
class SegmenterOV:
|
||||||
"""
|
"""
|
||||||
Class containing OpenVINO model with all attributes required to work with inference function.
|
Class containing OpenVINO model with all attributes required to work with inference function.
|
||||||
|
|
||||||
:param model: compiled OpenVINO model
|
:param model: compiled OpenVINO model
|
||||||
:type model: CompiledModel
|
:type model: CompiledModel
|
||||||
:param output_blob: output blob used in inference
|
:param output_blob: output blob used in inference
|
||||||
|
|
@ -750,14 +774,14 @@ any additional custom code required to process input.
|
||||||
:type n_cls: int
|
:type n_cls: int
|
||||||
:param normalization:
|
:param normalization:
|
||||||
:type normalization: dict
|
:type normalization: dict
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, model_path: Path, device:str = "CPU"):
|
def __init__(self, model_path: Path, device:str = "CPU"):
|
||||||
"""
|
"""
|
||||||
Constructor method.
|
Constructor method.
|
||||||
Initializes OpenVINO model and sets all required attributes
|
Initializes OpenVINO model and sets all required attributes
|
||||||
|
|
||||||
:param model_path: path to model's .xml file, also containing variant.yml
|
:param model_path: path to model's .xml file, also containing variant.yml
|
||||||
:param device: device string for selecting inference device
|
:param device: device string for selecting inference device
|
||||||
"""
|
"""
|
||||||
|
|
@ -767,23 +791,23 @@ any additional custom code required to process input.
|
||||||
model_xml = core.read_model(model_path)
|
model_xml = core.read_model(model_path)
|
||||||
self.model = core.compile_model(model_xml, device)
|
self.model = core.compile_model(model_xml, device)
|
||||||
self.output_blob = self.model.output(0)
|
self.output_blob = self.model.output(0)
|
||||||
|
|
||||||
# load model configs
|
# load model configs
|
||||||
variant_path = Path(model_path).parent / "variant.yml"
|
variant_path = Path(model_path).parent / "variant.yml"
|
||||||
with open(variant_path, "r") as f:
|
with open(variant_path, "r") as f:
|
||||||
self.config = yaml.load(f, Loader=yaml.FullLoader)
|
self.config = yaml.load(f, Loader=yaml.FullLoader)
|
||||||
|
|
||||||
# load normalization specs from config
|
# load normalization specs from config
|
||||||
normalization_name = self.config["dataset_kwargs"]["normalization"]
|
normalization_name = self.config["dataset_kwargs"]["normalization"]
|
||||||
self.normalization = STATS[normalization_name]
|
self.normalization = STATS[normalization_name]
|
||||||
|
|
||||||
# load number of classes from config
|
# load number of classes from config
|
||||||
self.n_cls = self.config["net_kwargs"]["n_cls"]
|
self.n_cls = self.config["net_kwargs"]["n_cls"]
|
||||||
|
|
||||||
def forward(self, data: torch.Tensor) -> torch.Tensor:
|
def forward(self, data: torch.Tensor) -> torch.Tensor:
|
||||||
"""
|
"""
|
||||||
Perform inference on data and return the result in Tensor format
|
Perform inference on data and return the result in Tensor format
|
||||||
|
|
||||||
:param data: input data to model
|
:param data: input data to model
|
||||||
:return: data inferred by model
|
:return: data inferred by model
|
||||||
"""
|
"""
|
||||||
|
|
@ -802,7 +826,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import ipywidgets as widgets
|
import ipywidgets as widgets
|
||||||
|
|
||||||
core = ov.Core()
|
core = ov.Core()
|
||||||
device = widgets.Dropdown(
|
device = widgets.Dropdown(
|
||||||
options=core.available_devices + ["AUTO"],
|
options=core.available_devices + ["AUTO"],
|
||||||
|
|
@ -810,7 +834,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
description='Device:',
|
description='Device:',
|
||||||
disabled=False,
|
disabled=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
device
|
device
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -842,7 +866,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
# combine segmentation mask with image
|
# combine segmentation mask with image
|
||||||
converted_blend = apply_segmentation_mask(pil_image, results)
|
converted_blend = apply_segmentation_mask(pil_image, results)
|
||||||
|
|
||||||
# show image with segmentation mask overlay
|
# show image with segmentation mask overlay
|
||||||
converted_blend
|
converted_blend
|
||||||
|
|
||||||
|
|
@ -904,17 +928,17 @@ to measure the inference performance of the model.
|
||||||
[ WARNING ] Default duration 120 seconds is used for unknown device AUTO
|
[ WARNING ] Default duration 120 seconds is used for unknown device AUTO
|
||||||
[ INFO ] OpenVINO:
|
[ INFO ] OpenVINO:
|
||||||
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
||||||
[ INFO ]
|
[ INFO ]
|
||||||
[ INFO ] Device info:
|
[ INFO ] Device info:
|
||||||
[ INFO ] AUTO
|
[ INFO ] AUTO
|
||||||
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
||||||
[ INFO ]
|
[ INFO ]
|
||||||
[ INFO ]
|
[ INFO ]
|
||||||
[Step 3/11] Setting device configuration
|
[Step 3/11] Setting device configuration
|
||||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||||
[Step 4/11] Reading model files
|
[Step 4/11] Reading model files
|
||||||
[ INFO ] Loading model files
|
[ INFO ] Loading model files
|
||||||
[ INFO ] Read model took 22.64 ms
|
[ INFO ] Read model took 23.09 ms
|
||||||
[ INFO ] Original model I/O parameters:
|
[ INFO ] Original model I/O parameters:
|
||||||
[ INFO ] Model inputs:
|
[ INFO ] Model inputs:
|
||||||
[ INFO ] im (node: im) : f32 / [...] / [2,3,512,512]
|
[ INFO ] im (node: im) : f32 / [...] / [2,3,512,512]
|
||||||
|
|
@ -932,7 +956,7 @@ to measure the inference performance of the model.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] Compile model took 392.81 ms
|
[ INFO ] Compile model took 385.39 ms
|
||||||
[Step 8/11] Querying optimal runtime parameters
|
[Step 8/11] Querying optimal runtime parameters
|
||||||
[ INFO ] Model:
|
[ INFO ] Model:
|
||||||
[ INFO ] NETWORK_NAME: Model0
|
[ INFO ] NETWORK_NAME: Model0
|
||||||
|
|
@ -940,10 +964,6 @@ to measure the inference performance of the model.
|
||||||
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
|
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
|
||||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||||
[ INFO ] MULTI_DEVICE_PRIORITIES: CPU
|
[ INFO ] MULTI_DEVICE_PRIORITIES: CPU
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ INFO ] CPU:
|
[ INFO ] CPU:
|
||||||
[ INFO ] AFFINITY: Affinity.CORE
|
[ INFO ] AFFINITY: Affinity.CORE
|
||||||
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
||||||
|
|
@ -965,30 +985,30 @@ to measure the inference performance of the model.
|
||||||
[ INFO ] LOADED_FROM_CACHE: False
|
[ INFO ] LOADED_FROM_CACHE: False
|
||||||
[Step 9/11] Creating infer requests and preparing input tensors
|
[Step 9/11] Creating infer requests and preparing input tensors
|
||||||
[ WARNING ] No input files were given for input 'im'!. This input will be filled with random values!
|
[ WARNING ] No input files were given for input 'im'!. This input will be filled with random values!
|
||||||
[ INFO ] Fill input 'im' with random values
|
[ INFO ] Fill input 'im' with random values
|
||||||
|
|
||||||
|
|
||||||
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 120000 ms duration)
|
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 120000 ms duration)
|
||||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[ INFO ] First inference took 203.86 ms
|
[ INFO ] First inference took 210.45 ms
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[Step 11/11] Dumping statistics report
|
[Step 11/11] Dumping statistics report
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
|
||||||
|
|
||||||
[ INFO ] Execution Devices:['CPU']
|
[ INFO ] Execution Devices:['CPU']
|
||||||
[ INFO ] Count: 1704 iterations
|
[ INFO ] Count: 1686 iterations
|
||||||
[ INFO ] Duration: 120454.83 ms
|
[ INFO ] Duration: 120531.12 ms
|
||||||
[ INFO ] Latency:
|
[ INFO ] Latency:
|
||||||
[ INFO ] Median: 423.45 ms
|
[ INFO ] Median: 429.25 ms
|
||||||
[ INFO ] Average: 423.78 ms
|
[ INFO ] Average: 428.34 ms
|
||||||
[ INFO ] Min: 358.59 ms
|
[ INFO ] Min: 354.96 ms
|
||||||
[ INFO ] Max: 506.30 ms
|
[ INFO ] Max: 506.55 ms
|
||||||
[ INFO ] Throughput: 28.29 FPS
|
[ INFO ] Throughput: 27.98 FPS
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,10 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/204-segmenter-semantic-segmentation-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/204-segmenter-semantic-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="204-segmenter-semantic-segmentation-with-output_21_0.jpg">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 72352
|
|
||||||
<a href="204-segmenter-semantic-segmentation-with-output_21_0.png">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 909669
|
|
||||||
<a href="204-segmenter-semantic-segmentation-with-output_32_0.jpg">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 72372
|
|
||||||
<a href="204-segmenter-semantic-segmentation-with-output_32_0.png">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 909654
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -24,8 +24,7 @@ Table of contents:
|
||||||
- `Settings <#settings>`__
|
- `Settings <#settings>`__
|
||||||
- `Load the U2-Net Model <#load-the-u2-net-model>`__
|
- `Load the U2-Net Model <#load-the-u2-net-model>`__
|
||||||
|
|
||||||
- `Convert PyTorch U2-Net model to OpenVINO
|
- `Convert PyTorch U2-Net model to OpenVINO IR <#convert-pytorch-u2-net-model-to-openvino-ir>`__
|
||||||
IR <#convert-pytorch-u2-net-model-to-openvino-ir>`__
|
|
||||||
- `Load and Pre-Process Input
|
- `Load and Pre-Process Input
|
||||||
Image <#load-and-pre-process-input-image>`__
|
Image <#load-and-pre-process-input-image>`__
|
||||||
- `Select inference device <#select-inference-device>`__
|
- `Select inference device <#select-inference-device>`__
|
||||||
|
|
@ -190,52 +189,52 @@ next cell loads the model and the pre-trained weights.
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
11%|█ | 524k/4.68M [00:00<00:01, 3.36MB/s]
|
11%|█ | 524k/4.68M [00:00<00:01, 3.35MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
22%|██▏ | 1.05M/4.68M [00:00<00:00, 3.74MB/s]
|
22%|██▏ | 1.05M/4.68M [00:00<00:00, 3.71MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
34%|███▎ | 1.57M/4.68M [00:00<00:00, 3.84MB/s]
|
34%|███▎ | 1.57M/4.68M [00:00<00:00, 3.78MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
45%|████▍ | 2.10M/4.68M [00:00<00:00, 3.89MB/s]
|
45%|████▍ | 2.10M/4.68M [00:00<00:00, 3.88MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
56%|█████▌ | 2.62M/4.68M [00:00<00:00, 3.92MB/s]
|
56%|█████▌ | 2.62M/4.68M [00:00<00:00, 3.96MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
67%|██████▋ | 3.15M/4.68M [00:00<00:00, 3.94MB/s]
|
67%|██████▋ | 3.15M/4.68M [00:00<00:00, 3.95MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
78%|███████▊ | 3.67M/4.68M [00:00<00:00, 3.95MB/s]
|
78%|███████▊ | 3.67M/4.68M [00:00<00:00, 3.97MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
90%|████████▉ | 4.19M/4.68M [00:01<00:00, 3.96MB/s]
|
90%|████████▉ | 4.19M/4.68M [00:01<00:00, 4.00MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 4.05MB/s]
|
100%|██████████| 4.68M/4.68M [00:01<00:00, 4.15MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
||||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 3.93MB/s]
|
100%|██████████| 4.68M/4.68M [00:01<00:00, 3.96MB/s]
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
|
|
@ -282,7 +281,7 @@ OpenVINO IR format. Executing the following command may take a while.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/functional.py:3769: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
|
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/functional.py:3769: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
|
||||||
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
|
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -382,7 +381,7 @@ Load the OpenVINO IR model to OpenVINO Runtime and do inference.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Inference finished. Inference time: 0.113 seconds, FPS: 8.89.
|
Inference finished. Inference time: 0.110 seconds, FPS: 9.05.
|
||||||
|
|
||||||
|
|
||||||
Visualize Results
|
Visualize Results
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/205-vision-background-removal-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/205-vision-background-removal-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="205-vision-background-removal-with-output_22_0.png">205-vision-background-removal-with-output_22_0.png</a> 26-Jan-2024 01:05 279572
|
|
||||||
<a href="205-vision-background-removal-with-output_24_0.png">205-vision-background-removal-with-output_24_0.png</a> 26-Jan-2024 01:05 927043
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -204,7 +204,7 @@ source of the function.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[01/25 23:27:13] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/animeganv2_hayao.pdparams
|
[02/09 23:41:27] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/animeganv2_hayao.pdparams
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -345,7 +345,7 @@ succeeds, the output of the next cell will include
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:27:21 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
2024-02-09 23:41:36 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -356,12 +356,12 @@ succeeds, the output of the next cell will include
|
||||||
[Paddle2ONNX] Start to parsing Paddle model...
|
[Paddle2ONNX] Start to parsing Paddle model...
|
||||||
[Paddle2ONNX] Use opset_version = 11 for ONNX export.
|
[Paddle2ONNX] Use opset_version = 11 for ONNX export.
|
||||||
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
||||||
2024-01-25 23:27:21 [INFO] ONNX model saved in model/paddlegan_anime.onnx.
|
2024-02-09 23:41:36 [INFO] ONNX model saved in model/paddlegan_anime.onnx.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
I0125 23:27:21.862668 812000 program_interpreter.cc:212] New Executor is Running.
|
I0209 23:41:36.202327 2843665 program_interpreter.cc:212] New Executor is Running.
|
||||||
|
|
||||||
|
|
||||||
Convert to OpenVINO IR
|
Convert to OpenVINO IR
|
||||||
|
|
@ -643,12 +643,12 @@ measure inference on one image. For more accurate benchmarking, use
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.418 seconds per image, FPS: 2.39
|
OpenVINO IR model in OpenVINO Runtime/CPU: 0.427 seconds per image, FPS: 2.34
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
PaddleGAN model on CPU: 6.256 seconds per image, FPS: 0.16
|
PaddleGAN model on CPU: 6.182 seconds per image, FPS: 0.16
|
||||||
|
|
||||||
|
|
||||||
References
|
References
|
||||||
|
|
|
||||||
|
|
@ -1,8 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/206-vision-paddlegan-anime-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/206-vision-paddlegan-anime-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="206-vision-paddlegan-anime-with-output_15_0.png">206-vision-paddlegan-anime-with-output_15_0.png</a> 26-Jan-2024 01:05 1810982
|
|
||||||
<a href="206-vision-paddlegan-anime-with-output_37_0.png">206-vision-paddlegan-anime-with-output_37_0.png</a> 26-Jan-2024 01:05 1931637
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -161,7 +161,7 @@ source code.
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
[01/25 23:27:55] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/DF2K_JPEG.pdparams
|
[02/09 23:42:09] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/DF2K_JPEG.pdparams
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -289,12 +289,12 @@ Convert PaddlePaddle Model to ONNX
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:28:02 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
2024-02-09 23:42:16 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
I0125 23:28:01.927830 812342 program_interpreter.cc:212] New Executor is Running.
|
I0209 23:42:16.006111 2843987 program_interpreter.cc:212] New Executor is Running.
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
@ -303,13 +303,13 @@ Convert PaddlePaddle Model to ONNX
|
||||||
[Paddle2ONNX] Model file path: model/paddle_model_static_onnx_temp_dir/model.pdmodel
|
[Paddle2ONNX] Model file path: model/paddle_model_static_onnx_temp_dir/model.pdmodel
|
||||||
[Paddle2ONNX] Paramters file path: model/paddle_model_static_onnx_temp_dir/model.pdiparams
|
[Paddle2ONNX] Paramters file path: model/paddle_model_static_onnx_temp_dir/model.pdiparams
|
||||||
[Paddle2ONNX] Start to parsing Paddle model...
|
[Paddle2ONNX] Start to parsing Paddle model...
|
||||||
[Paddle2ONNX] Use opset_version = 13 for ONNX export.
|
|
||||||
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
|
||||||
|
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
2024-01-25 23:28:05 [INFO] ONNX model saved in model/paddlegan_sr.onnx.
|
[Paddle2ONNX] Use opset_version = 13 for ONNX export.
|
||||||
|
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
||||||
|
2024-02-09 23:42:19 [INFO] ONNX model saved in model/paddlegan_sr.onnx.
|
||||||
|
|
||||||
|
|
||||||
Convert ONNX Model to OpenVINO IR with `Model Conversion Python API <https://docs.openvino.ai/2023.3/openvino_docs_model_processing_introduction.html>`__
|
Convert ONNX Model to OpenVINO IR with `Model Conversion Python API <https://docs.openvino.ai/2023.3/openvino_docs_model_processing_introduction.html>`__
|
||||||
|
|
@ -394,7 +394,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7ff10225de20>
|
<matplotlib.image.AxesImage at 0x7fe9ebed32e0>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -423,7 +423,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
Inference duration: 3.26 seconds
|
Inference duration: 3.27 seconds
|
||||||
|
|
||||||
|
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
@ -446,7 +446,7 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
|
|
||||||
.. parsed-literal::
|
.. parsed-literal::
|
||||||
|
|
||||||
<matplotlib.image.AxesImage at 0x7ff0cc1894f0>
|
<matplotlib.image.AxesImage at 0x7fe9b40e3580>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,9 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/207-vision-paddlegan-superresolution-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/207-vision-paddlegan-superresolution-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="207-vision-paddlegan-superresolution-with-output_26_1.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 436999
|
|
||||||
<a href="207-vision-paddlegan-superresolution-with-output_30_1.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 476190
|
|
||||||
<a href="207-vision-paddlegan-superresolution-with-output_32_0.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 2835305
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,20 +0,0 @@
|
||||||
<html>
|
|
||||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/208-optical-character-recognition-with-output_files/</title></head>
|
|
||||||
<body bgcolor="white">
|
|
||||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/208-optical-character-recognition-with-output_files/</h1><hr><pre><a href="../">../</a>
|
|
||||||
<a href="208-optical-character-recognition-with-output_16_0.png">208-optical-character-recognition-with-output_1..></a> 26-Jan-2024 01:05 305482
|
|
||||||
<a href="208-optical-character-recognition-with-output_26_0.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 923631
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_0.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1996
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_0.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 11367
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_10.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1990
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_10.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 11142
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_2.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1630
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_2.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 8428
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_4.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 949
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_4.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 2274
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_6.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 817
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_6.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1559
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_8.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 838
|
|
||||||
<a href="208-optical-character-recognition-with-output_28_8.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1487
|
|
||||||
</pre><hr></body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -8,9 +8,9 @@ Latin alphabet is available in `notebook
|
||||||
This model is capable of processing only one line of symbols at a time.
|
This model is capable of processing only one line of symbols at a time.
|
||||||
|
|
||||||
The models used in this notebook are
|
The models used in this notebook are
|
||||||
`handwritten-japanese-recognition-0001 <https://docs.openvino.ai/2023.0/omz_models_model_handwritten_japanese_recognition_0001.html>`__
|
`handwritten-japanese-recognition-0001 <https://docs.openvino.ai/2023.3/omz_models_model_handwritten_japanese_recognition_0001.html>`__
|
||||||
and
|
and
|
||||||
`handwritten-simplified-chinese-0001 <https://docs.openvino.ai/2023.0/omz_models_model_handwritten_simplified_chinese_recognition_0001.html>`__.
|
`handwritten-simplified-chinese-0001 <https://docs.openvino.ai/2023.3/omz_models_model_handwritten_simplified_chinese_recognition_0001.html>`__.
|
||||||
To decode model outputs as readable text
|
To decode model outputs as readable text
|
||||||
`kondate_nakayosi <https://github.com/openvinotoolkit/open_model_zoo/blob/master/data/dataset_classes/kondate_nakayosi.txt>`__
|
`kondate_nakayosi <https://github.com/openvinotoolkit/open_model_zoo/blob/master/data/dataset_classes/kondate_nakayosi.txt>`__
|
||||||
and
|
and
|
||||||
|
|
@ -62,12 +62,12 @@ Imports
|
||||||
|
|
||||||
from collections import namedtuple
|
from collections import namedtuple
|
||||||
from itertools import groupby
|
from itertools import groupby
|
||||||
|
|
||||||
import cv2
|
import cv2
|
||||||
import matplotlib.pyplot as plt
|
import matplotlib.pyplot as plt
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import openvino as ov
|
import openvino as ov
|
||||||
|
|
||||||
# Fetch `notebook_utils` module
|
# Fetch `notebook_utils` module
|
||||||
import urllib.request
|
import urllib.request
|
||||||
urllib.request.urlretrieve(
|
urllib.request.urlretrieve(
|
||||||
|
|
@ -89,7 +89,7 @@ Set up all constants and folders used in this notebook
|
||||||
base_models_dir = "models"
|
base_models_dir = "models"
|
||||||
data_folder = "data"
|
data_folder = "data"
|
||||||
charlist_folder = f"{data_folder}/text"
|
charlist_folder = f"{data_folder}/text"
|
||||||
|
|
||||||
# Precision used by the model.
|
# Precision used by the model.
|
||||||
precision = "FP16"
|
precision = "FP16"
|
||||||
|
|
||||||
|
|
@ -127,9 +127,9 @@ If you want to perform OCR on a text in Japanese, set
|
||||||
|
|
||||||
# Select the language by using either language="chinese" or language="japanese".
|
# Select the language by using either language="chinese" or language="japanese".
|
||||||
language = "chinese"
|
language = "chinese"
|
||||||
|
|
||||||
languages = {"chinese": chinese_files, "japanese": japanese_files}
|
languages = {"chinese": chinese_files, "japanese": japanese_files}
|
||||||
|
|
||||||
selected_language = languages.get(language)
|
selected_language = languages.get(language)
|
||||||
|
|
||||||
Download the Model
|
Download the Model
|
||||||
|
|
@ -196,14 +196,14 @@ select device from dropdown list for running inference using OpenVINO
|
||||||
.. code:: ipython3
|
.. code:: ipython3
|
||||||
|
|
||||||
import ipywidgets as widgets
|
import ipywidgets as widgets
|
||||||
|
|
||||||
device = widgets.Dropdown(
|
device = widgets.Dropdown(
|
||||||
options=core.available_devices + ["AUTO"],
|
options=core.available_devices + ["AUTO"],
|
||||||
value='AUTO',
|
value='AUTO',
|
||||||
description='Device:',
|
description='Device:',
|
||||||
disabled=False,
|
disabled=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
device
|
device
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -252,32 +252,32 @@ keep letters proportional and meet input shape.
|
||||||
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/" + selected_language.demo_image_name,
|
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/" + selected_language.demo_image_name,
|
||||||
directory=data_folder
|
directory=data_folder
|
||||||
)
|
)
|
||||||
|
|
||||||
# Text detection models expect an image in grayscale format.
|
# Text detection models expect an image in grayscale format.
|
||||||
# IMPORTANT! This model enables reading only one line at time.
|
# IMPORTANT! This model enables reading only one line at time.
|
||||||
|
|
||||||
# Read the image.
|
# Read the image.
|
||||||
image = cv2.imread(filename=str(file_name), flags=cv2.IMREAD_GRAYSCALE)
|
image = cv2.imread(filename=str(file_name), flags=cv2.IMREAD_GRAYSCALE)
|
||||||
|
|
||||||
# Fetch the shape.
|
# Fetch the shape.
|
||||||
image_height, _ = image.shape
|
image_height, _ = image.shape
|
||||||
|
|
||||||
# B,C,H,W = batch size, number of channels, height, width.
|
# B,C,H,W = batch size, number of channels, height, width.
|
||||||
_, _, H, W = recognition_input_layer.shape
|
_, _, H, W = recognition_input_layer.shape
|
||||||
|
|
||||||
# Calculate scale ratio between the input shape height and image height to resize the image.
|
# Calculate scale ratio between the input shape height and image height to resize the image.
|
||||||
scale_ratio = H / image_height
|
scale_ratio = H / image_height
|
||||||
|
|
||||||
# Resize the image to expected input sizes.
|
# Resize the image to expected input sizes.
|
||||||
resized_image = cv2.resize(
|
resized_image = cv2.resize(
|
||||||
image, None, fx=scale_ratio, fy=scale_ratio, interpolation=cv2.INTER_AREA
|
image, None, fx=scale_ratio, fy=scale_ratio, interpolation=cv2.INTER_AREA
|
||||||
)
|
)
|
||||||
|
|
||||||
# Pad the image to match input size, without changing aspect ratio.
|
# Pad the image to match input size, without changing aspect ratio.
|
||||||
resized_image = np.pad(
|
resized_image = np.pad(
|
||||||
resized_image, ((0, 0), (0, W - resized_image.shape[1])), mode="edge"
|
resized_image, ((0, 0), (0, W - resized_image.shape[1])), mode="edge"
|
||||||
)
|
)
|
||||||
|
|
||||||
# Reshape to network input shape.
|
# Reshape to network input shape.
|
||||||
input_image = resized_image[None, None, :, :]
|
input_image = resized_image[None, None, :, :]
|
||||||
|
|
||||||
|
|
@ -335,10 +335,10 @@ Chinese and Japanese models.
|
||||||
|
|
||||||
# Get a dictionary to encode the output, based on model documentation.
|
# Get a dictionary to encode the output, based on model documentation.
|
||||||
used_charlist = selected_language.charlist_name
|
used_charlist = selected_language.charlist_name
|
||||||
|
|
||||||
# With both models, there should be blank symbol added at index 0 of each charlist.
|
# With both models, there should be blank symbol added at index 0 of each charlist.
|
||||||
blank_char = "~"
|
blank_char = "~"
|
||||||
|
|
||||||
with used_charlist_file.open(mode="r", encoding="utf-8") as charlist:
|
with used_charlist_file.open(mode="r", encoding="utf-8") as charlist:
|
||||||
letters = blank_char + "".join(line.strip() for line in charlist)
|
letters = blank_char + "".join(line.strip() for line in charlist)
|
||||||
|
|
||||||
|
|
@ -380,7 +380,7 @@ Finally, get the symbols from corresponding indexes in the charlist.
|
||||||
|
|
||||||
# Remove a batch dimension.
|
# Remove a batch dimension.
|
||||||
predictions = np.squeeze(predictions)
|
predictions = np.squeeze(predictions)
|
||||||
|
|
||||||
# Run the `argmax` function to pick the symbols with the highest probability.
|
# Run the `argmax` function to pick the symbols with the highest probability.
|
||||||
predictions_indexes = np.argmax(predictions, axis=1)
|
predictions_indexes = np.argmax(predictions, axis=1)
|
||||||
|
|
||||||
|
|
@ -388,13 +388,13 @@ Finally, get the symbols from corresponding indexes in the charlist.
|
||||||
|
|
||||||
# Use the `groupby` function to remove concurrent letters, as required by CTC greedy decoding.
|
# Use the `groupby` function to remove concurrent letters, as required by CTC greedy decoding.
|
||||||
output_text_indexes = list(groupby(predictions_indexes))
|
output_text_indexes = list(groupby(predictions_indexes))
|
||||||
|
|
||||||
# Remove grouper objects.
|
# Remove grouper objects.
|
||||||
output_text_indexes, _ = np.transpose(output_text_indexes, (1, 0))
|
output_text_indexes, _ = np.transpose(output_text_indexes, (1, 0))
|
||||||
|
|
||||||
# Remove blank symbols.
|
# Remove blank symbols.
|
||||||
output_text_indexes = output_text_indexes[output_text_indexes != 0]
|
output_text_indexes = output_text_indexes[output_text_indexes != 0]
|
||||||
|
|
||||||
# Assign letters to indexes from the output array.
|
# Assign letters to indexes from the output array.
|
||||||
output_text = [letters[letter_index] for letter_index in output_text_indexes]
|
output_text = [letters[letter_index] for letter_index in output_text_indexes]
|
||||||
|
|
||||||
|
|
@ -411,7 +411,7 @@ the image with predicted text printed below.
|
||||||
plt.figure(figsize=(20, 1))
|
plt.figure(figsize=(20, 1))
|
||||||
plt.axis("off")
|
plt.axis("off")
|
||||||
plt.imshow(resized_image, cmap="gray", vmin=0, vmax=255)
|
plt.imshow(resized_image, cmap="gray", vmin=0, vmax=255)
|
||||||
|
|
||||||
print("".join(output_text))
|
print("".join(output_text))
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue