[DOCS] Updating Interactive Tutorials (#22815)
Updating Interactive Tutorials - adding new notebooks.
This commit is contained in:
Sebastian Golebiewski
GitHub
parent
b1c699aab5
commit
bf36b03bd2
|
|
@ -59,6 +59,24 @@ The Jupyter notebooks are categorized into following classes:
|
|||
Below you will find a selection of recommended tutorials that demonstrate inference on a particular model. These tutorials are guaranteed to provide a great experience with inference in OpenVINO:
|
||||
|
||||
|
||||
.. showcase::
|
||||
:title: 284-openvoice
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/284-openvoice/284-openvoice.png
|
||||
|
||||
Voice tone cloning with OpenVoice and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 283-photo-maker
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/283-photo-maker/283-photo-maker.gif
|
||||
|
||||
Text-to-image generation using PhotoMaker and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 281-kosmos2-multimodal-large-language-model
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/281-kosmos2-multimodal-large-language-model/281-kosmos2-multimodal-large-language-model.png
|
||||
|
||||
Kosmos-2: Multimodal Large Language Model and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 280-depth-anything
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
||||
|
|
@ -83,14 +101,12 @@ Below you will find a selection of recommended tutorials that demonstrate infere
|
|||
|
||||
LLM Instruction-following pipeline with OpenVINO.
|
||||
|
||||
|
||||
.. showcase::
|
||||
:title: 274-efficient-sam
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/274-efficient-sam/274-efficient-sam.png
|
||||
|
||||
Object segmentations with EfficientSAM and OpenVINO.
|
||||
|
||||
|
||||
.. showcase::
|
||||
:title: 273-stable-zephyr-3b-chatbot
|
||||
:img: _static/images/notebook_eye.png
|
||||
|
|
@ -104,25 +120,6 @@ Below you will find a selection of recommended tutorials that demonstrate infere
|
|||
|
||||
Paint by Example using Stable Diffusion and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 271-sdxl-turbo
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/271-sdxl-turbo/271-sdxl-turbo.png
|
||||
|
||||
Single step image generation using SDXL-turbo and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 270-sound-generation-audioldm2
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/270-sound-generation-audioldm2/270-sound-generation-audioldm2.png
|
||||
|
||||
Sound Generation with AudioLDM2 and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 269-film-slowmo
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/269-film-slowmo/269-film-slowmo.gif
|
||||
|
||||
Frame interpolation using FILM and OpenVINO.
|
||||
|
||||
|
||||
|
||||
.. note::
|
||||
If there are any issues while running the notebooks, refer to the **Troubleshooting** and **FAQ** sections in the :doc:`Installation Guide <notebooks_installation>` or start a GitHub
|
||||
|
|
|
|||
|
|
@ -11,6 +11,12 @@ Convert & Optimize
|
|||
|
||||
Tutorials that explain how to optimize and quantize models with OpenVINO tools.
|
||||
|
||||
.. showcase::
|
||||
:title: 127-tensorflow-bit-image-classification-nncf-quantization
|
||||
:img: _static/images/notebook_eye.png
|
||||
|
||||
BiT Image Classification OpenVINO IR model Quantization with NNCF.
|
||||
|
||||
.. showcase::
|
||||
:title: 126-tensorflow-hub
|
||||
:img: _static/images/notebook_eye.png
|
||||
|
|
@ -11,6 +11,31 @@ Model Demos
|
|||
|
||||
Demos that demonstrate inference on a particular model.
|
||||
|
||||
|
||||
.. showcase::
|
||||
:title: 284-openvoice
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/284-openvoice/284-openvoice.png
|
||||
|
||||
Voice tone cloning with OpenVoice and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 283-photo-maker
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/283-photo-maker/283-photo-maker.gif
|
||||
|
||||
Text-to-image generation using PhotoMaker and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 282-siglip-zero-shot-image-classification
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/228-clip-zero-shot-image-classification/228-clip-zero-shot-convert.png
|
||||
|
||||
Zero-shot Image Classification with SigLIP.
|
||||
|
||||
.. showcase::
|
||||
:title: 281-kosmos2-multimodal-large-language-model
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/281-kosmos2-multimodal-large-language-model/281-kosmos2-multimodal-large-language-model.png
|
||||
|
||||
Kosmos-2: Multimodal Large Language Model and OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 280-depth-anything
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/280-depth-anything/280-depth-anything.gif
|
||||
|
|
@ -441,11 +466,6 @@ Demos that demonstrate inference on a particular model.
|
|||
|
||||
Use pre-trained models to detect and recognize vehicles and their attributes with OpenVINO.
|
||||
|
||||
.. showcase::
|
||||
:title: 217-vision-deblur
|
||||
:img: https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/main/notebooks/217-vision-deblur/217-vision-deblur.png
|
||||
|
||||
Deblur images with DeblurGAN-v2.
|
||||
|
||||
.. showcase::
|
||||
:title: 216-attention-center
|
||||
|
|
@ -23,6 +23,12 @@ Tutorials that include code to train neural networks.
|
|||
|
||||
Use Neural Network Compression Framework (NNCF) to quantize PyTorch model.
|
||||
|
||||
.. showcase::
|
||||
:title: 301-tensorflow-training-openvino
|
||||
:img: _static/images/notebook_eye.png
|
||||
|
||||
From Training to Deployment with TensorFlow and OpenVINO™.
|
||||
|
||||
.. showcase::
|
||||
:title: 301-tensorflow-training-openvino-nncf
|
||||
:img: _static/images/notebook_eye.png
|
||||
|
|
@ -5,7 +5,7 @@ repo_directory = "notebooks"
|
|||
repo_owner = "openvinotoolkit"
|
||||
repo_name = "openvino_notebooks"
|
||||
repo_branch = "tree/main"
|
||||
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/"
|
||||
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240209220807/dist/rst_files/"
|
||||
blacklisted_extensions = ['.xml', '.bin']
|
||||
notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/main/"
|
||||
notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath="
|
||||
|
|
|
|||
|
|
@ -68,11 +68,11 @@ def fetch_colab_list(colab_list_file) -> list:
|
|||
|
||||
|
||||
def add_glob_directive():
|
||||
"""This function modifies toctrees of the five node articles in tutorials
|
||||
"""This function modifies toctrees of the five node articles in tutorials
|
||||
section. It adds the notebooks found in docs/notebooks directory to the menu.
|
||||
"""
|
||||
tutorials_path = Path('../../docs/articles_en/learn_openvino/tutorials').resolve(strict=True)
|
||||
tutorials_files = [x for x in os.listdir(tutorials_path) if re.match("notebooks_section_[0-9]{1}\.", x)]
|
||||
tutorials_path = Path('../../docs/articles_en/learn-openvino/interactive-tutorials-python').resolve(strict=True)
|
||||
tutorials_files = [x for x in os.listdir(tutorials_path) if re.match("notebooks-section-[0-9]{1}.*?\.rst", x)]
|
||||
for tutorials_file in tutorials_files:
|
||||
file_name = os.path.join(tutorials_path, tutorials_file)
|
||||
with open(file_name, 'r+', encoding='cp437') as section_file:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ This basic introduction to OpenVINO™ shows how to do inference with an
|
|||
image classification model.
|
||||
|
||||
A pre-trained `MobileNetV3
|
||||
model <https://docs.openvino.ai/2023.0/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||
model <https://docs.openvino.ai/2023.3/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||
from `Open Model
|
||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used in
|
||||
this tutorial. For more information about how OpenVINO IR models are
|
||||
|
|
|
|||
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/001-hello-world-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/001-hello-world-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="001-hello-world-with-output_11_1.png">001-hello-world-with-output_11_1.png</a> 26-Jan-2024 01:05 387941
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -61,47 +61,43 @@ Table of contents:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.31.0)
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.1)
|
||||
Requirement already satisfied: ipywidgets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (8.1.1)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2.1.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2023.11.17)
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.31.0)
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.1)
|
||||
Requirement already satisfied: ipywidgets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (8.1.2)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2.2.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests) (2024.2.2)
|
||||
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (0.2.1)
|
||||
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (8.12.3)
|
||||
Requirement already satisfied: traitlets>=4.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (5.14.1)
|
||||
Requirement already satisfied: widgetsnbextension~=4.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (4.0.10)
|
||||
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (3.0.10)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (0.2.1)
|
||||
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (8.12.3)
|
||||
Requirement already satisfied: traitlets>=4.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (5.14.1)
|
||||
Requirement already satisfied: widgetsnbextension~=4.0.9 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (4.0.9)
|
||||
Requirement already satisfied: jupyterlab-widgets~=3.0.9 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets) (3.0.9)
|
||||
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.2.0)
|
||||
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (5.1.1)
|
||||
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.19.1)
|
||||
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.6)
|
||||
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.7.5)
|
||||
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.43)
|
||||
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (2.17.2)
|
||||
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.3)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.2.0)
|
||||
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (5.1.1)
|
||||
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.19.1)
|
||||
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.6)
|
||||
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.7.5)
|
||||
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.43)
|
||||
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (2.17.2)
|
||||
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.3)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
|
||||
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.3)
|
||||
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets) (0.2.13)
|
||||
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.0.1)
|
||||
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.4.1)
|
||||
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (0.2.2)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets) (1.16.0)
|
||||
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.3)
|
||||
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets) (0.2.13)
|
||||
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.0.1)
|
||||
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (2.4.1)
|
||||
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets) (0.2.2)
|
||||
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets) (1.16.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -218,7 +214,7 @@ notebooks.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -267,7 +263,7 @@ points to the filename of an ONNX model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.onnx')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.onnx')
|
||||
|
||||
|
||||
|
||||
|
|
@ -323,7 +319,7 @@ without any conversion step. Pass the filename with extension to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/inference.pdiparams')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/inference.pdiparams')
|
||||
|
||||
|
||||
|
||||
|
|
@ -367,7 +363,7 @@ TensorFlow models saved in frozen graph format can also be passed to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.pb')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.pb')
|
||||
|
||||
|
||||
|
||||
|
|
@ -419,7 +415,7 @@ It is pre-trained model optimized to work with TensorFlow Lite.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.tflite')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.tflite')
|
||||
|
||||
|
||||
|
||||
|
|
@ -504,7 +500,7 @@ Information about the inputs and outputs of the model are in
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -714,7 +710,7 @@ produced data as values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -903,7 +899,7 @@ input shape.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/segmentation.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -1066,7 +1062,7 @@ the cache.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/002-openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ Hello Image Segmentation
|
|||
A very basic introduction to using segmentation models with OpenVINO™.
|
||||
|
||||
In this tutorial, a pre-trained
|
||||
`road-segmentation-adas-0001 <https://docs.openvino.ai/2023.0/omz_models_model_road_segmentation_adas_0001.html>`__
|
||||
`road-segmentation-adas-0001 <https://docs.openvino.ai/2023.3/omz_models_model_road_segmentation_adas_0001.html>`__
|
||||
model from the `Open Model
|
||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used.
|
||||
ADAS stands for Advanced Driver Assistance Services. The model
|
||||
|
|
@ -182,7 +182,7 @@ is provided.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7ffab9d92970>
|
||||
<matplotlib.image.AxesImage at 0x7f11c8142580>
|
||||
|
||||
|
||||
|
||||
|
|
@ -209,7 +209,7 @@ Do Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7ffa747040d0>
|
||||
<matplotlib.image.AxesImage at 0x7f11c8051eb0>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/003-hello-segmentation-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/003-hello-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="003-hello-segmentation-with-output_11_2.png">003-hello-segmentation-with-output_11_2.png</a> 26-Jan-2024 01:05 249032
|
||||
<a href="003-hello-segmentation-with-output_13_1.png">003-hello-segmentation-with-output_13_1.png</a> 26-Jan-2024 01:05 20550
|
||||
<a href="003-hello-segmentation-with-output_17_0.png">003-hello-segmentation-with-output_17_0.png</a> 26-Jan-2024 01:05 260045
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -5,7 +5,7 @@ A very basic introduction to using object detection models with
|
|||
OpenVINO™.
|
||||
|
||||
The
|
||||
`horizontal-text-detection-0001 <https://docs.openvino.ai/2023.0/omz_models_model_horizontal_text_detection_0001.html>`__
|
||||
`horizontal-text-detection-0001 <https://docs.openvino.ai/2023.3/omz_models_model_horizontal_text_detection_0001.html>`__
|
||||
model from `Open Model
|
||||
Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__ is used. It
|
||||
detects horizontal text in images and returns a blob of data in the
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/004-hello-detection-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/004-hello-detection-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="004-hello-detection-with-output_11_1.png">004-hello-detection-with-output_11_1.png</a> 26-Jan-2024 01:05 305482
|
||||
<a href="004-hello-detection-with-output_16_0.png">004-hello-detection-with-output_16_0.png</a> 26-Jan-2024 01:05 457214
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -2,7 +2,7 @@ Convert a TensorFlow Model to OpenVINO™
|
|||
=======================================
|
||||
|
||||
This short tutorial shows how to convert a TensorFlow
|
||||
`MobileNetV3 <https://docs.openvino.ai/2023.0/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||
`MobileNetV3 <https://docs.openvino.ai/2023.3/omz_models_model_mobilenet_v3_small_1_0_224_tf.html>`__
|
||||
image classification model to OpenVINO `Intermediate
|
||||
Representation <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_IR_and_opsets.html>`__
|
||||
(OpenVINO IR) format, using `Model Conversion
|
||||
|
|
@ -75,14 +75,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:33:56.723840: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:33:56.757735: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:34:07.759850: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:34:07.794264: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:33:57.273134: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:34:08.310440: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Settings
|
||||
|
|
@ -122,12 +122,12 @@ and save it to the disk.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:34:00.136277: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-01-25 22:34:00.136313: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-01-25 22:34:00.136317: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-01-25 22:34:00.136451: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-01-25 22:34:00.136466: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-01-25 22:34:00.136470: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-02-09 22:34:11.190073: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-02-09 22:34:11.190106: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-02-09 22:34:11.190111: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-02-09 22:34:11.190249: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-02-09 22:34:11.190264: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-02-09 22:34:11.190268: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -137,13 +137,13 @@ and save it to the disk.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:34:04.279915: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||
2024-02-09 22:34:15.411337: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||
[[{{node inputs}}]]
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:34:07.400979: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||
2024-02-09 22:34:18.568762: I tensorflow/core/common_runtime/executor.cc:1197] [/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: You must feed a value for placeholder tensor 'inputs' with dtype float and shape [?,1,1,1024]
|
||||
[[{{node inputs}}]]
|
||||
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 54). These functions will not be directly callable after loading.
|
||||
|
||||
|
|
@ -362,5 +362,5 @@ performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
IR model in OpenVINO Runtime/CPU: 0.0011 seconds per image, FPS: 933.95
|
||||
IR model in OpenVINO Runtime/CPU: 0.0011 seconds per image, FPS: 926.34
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/101-tensorflow-classification-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/101-tensorflow-classification-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="101-tensorflow-classification-to-openvino-with-output_19_1.png">101-tensorflow-classification-to-openvino-with-..></a> 26-Jan-2024 01:05 387941
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -576,17 +576,17 @@ performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.042 seconds per image, FPS: 24.02
|
||||
PyTorch model on CPU: 0.040 seconds per image, FPS: 24.69
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
ONNX model in OpenVINO Runtime/CPU: 0.030 seconds per image, FPS: 33.66
|
||||
ONNX model in OpenVINO Runtime/CPU: 0.018 seconds per image, FPS: 56.48
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.029 seconds per image, FPS: 35.01
|
||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.018 seconds per image, FPS: 55.11
|
||||
|
||||
|
||||
**Show Device Information**
|
||||
|
|
|
|||
|
|
@ -1,9 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-onnx-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-onnx-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="102-pytorch-onnx-to-openvino-with-output_22_0.png">102-pytorch-onnx-to-openvino-with-output_22_0.png</a> 26-Jan-2024 01:05 465692
|
||||
<a href="102-pytorch-onnx-to-openvino-with-output_27_0.png">102-pytorch-onnx-to-openvino-with-output_27_0.png</a> 26-Jan-2024 01:05 465695
|
||||
<a href="102-pytorch-onnx-to-openvino-with-output_29_0.png">102-pytorch-onnx-to-openvino-with-output_29_0.png</a> 26-Jan-2024 01:05 465692
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -248,7 +248,7 @@ Benchmark PyTorch Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
16.4 ms ± 673 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
17.6 ms ± 52.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -407,7 +407,7 @@ Benchmark OpenVINO Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.31 ms ± 28.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.42 ms ± 7.33 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Model with Static Input Shape
|
||||
|
|
@ -537,7 +537,7 @@ Benchmark OpenVINO Model Inference with Static Input Shape
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2.89 ms ± 38.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
2.9 ms ± 17.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert TorchScript Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -632,7 +632,7 @@ Benchmark Scripted Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
12.8 ms ± 6.97 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
13 ms ± 50.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Scripted Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -691,7 +691,7 @@ Benchmark OpenVINO Model Inference Converted From Scripted Model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.41 ms ± 6.84 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.42 ms ± 6.53 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Traced Model
|
||||
|
|
@ -767,7 +767,7 @@ Benchmark Traced Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
12.2 ms ± 29 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
13.4 ms ± 4.67 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Traced Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -826,5 +826,5 @@ Benchmark OpenVINO Model Inference Converted From Traced Model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.4 ms ± 3.43 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.47 ms ± 10.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,20 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/102-pytorch-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="102-pytorch-to-openvino-with-output_11_0.jpg">102-pytorch-to-openvino-with-output_11_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_11_0.png">102-pytorch-to-openvino-with-output_11_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_20_0.jpg">102-pytorch-to-openvino-with-output_20_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_20_0.png">102-pytorch-to-openvino-with-output_20_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_31_0.jpg">102-pytorch-to-openvino-with-output_31_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_31_0.png">102-pytorch-to-openvino-with-output_31_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_35_0.jpg">102-pytorch-to-openvino-with-output_35_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_35_0.png">102-pytorch-to-openvino-with-output_35_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_39_0.jpg">102-pytorch-to-openvino-with-output_39_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_39_0.png">102-pytorch-to-openvino-with-output_39_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_43_0.jpg">102-pytorch-to-openvino-with-output_43_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_43_0.png">102-pytorch-to-openvino-with-output_43_0.png</a> 26-Jan-2024 01:05 542516
|
||||
<a href="102-pytorch-to-openvino-with-output_47_0.jpg">102-pytorch-to-openvino-with-output_47_0.jpg</a> 26-Jan-2024 01:05 54874
|
||||
<a href="102-pytorch-to-openvino-with-output_47_0.png">102-pytorch-to-openvino-with-output_47_0.png</a> 26-Jan-2024 01:05 542516
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -94,7 +94,7 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
--2024-01-25 22:35:57-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
||||
--2024-02-09 22:36:08-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
||||
Resolving proxy-mu.intel.com (proxy-mu.intel.com)... 10.217.247.236
|
||||
Connecting to proxy-mu.intel.com (proxy-mu.intel.com)|10.217.247.236|:911... connected.
|
||||
Proxy request sent, awaiting response...
|
||||
|
|
@ -102,7 +102,7 @@ Imports
|
|||
.. parsed-literal::
|
||||
|
||||
404 Not Found
|
||||
2024-01-25 22:35:57 ERROR 404: Not Found.
|
||||
2024-02-09 22:36:08 ERROR 404: Not Found.
|
||||
|
||||
|
||||
|
||||
|
|
@ -135,12 +135,12 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:35:59 INFO: Loading faiss with AVX2 support.
|
||||
2024-02-09 22:36:10 INFO: Loading faiss with AVX2 support.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:35:59 INFO: Successfully loaded faiss with AVX2 support.
|
||||
2024-02-09 22:36:10 INFO: Successfully loaded faiss with AVX2 support.
|
||||
|
||||
|
||||
Settings
|
||||
|
|
@ -224,7 +224,7 @@ inference on that image, and then show the top three prediction results.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[2024/01/25 22:36:20] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
||||
[2024/02/09 22:36:38] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -294,7 +294,7 @@ clipping values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:36:20 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
||||
2024-02-09 22:36:39 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -306,7 +306,7 @@ clipping values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f72946bbac0>
|
||||
<matplotlib.image.AxesImage at 0x7f9b4c389670>
|
||||
|
||||
|
||||
|
||||
|
|
@ -489,7 +489,7 @@ Note that many optimizations are possible to improve the performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PaddlePaddle model on CPU: 0.0073 seconds per image, FPS: 137.36
|
||||
PaddlePaddle model on CPU: 0.0075 seconds per image, FPS: 133.16
|
||||
|
||||
PaddlePaddle result:
|
||||
Labrador retriever, 0.75138
|
||||
|
|
@ -553,7 +553,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0031 seconds per image, FPS: 322.19
|
||||
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0030 seconds per image, FPS: 328.87
|
||||
|
||||
OpenVINO result:
|
||||
Labrador retriever, 0.74909
|
||||
|
|
|
|||
|
|
@ -1,11 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/103-paddle-to-openvino-classification-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/103-paddle-to-openvino-classification-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="103-paddle-to-openvino-classification-with-output_15_3.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 120883
|
||||
<a href="103-paddle-to-openvino-classification-with-output_23_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
||||
<a href="103-paddle-to-openvino-classification-with-output_27_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
||||
<a href="103-paddle-to-openvino-classification-with-output_30_1.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
||||
<a href="103-paddle-to-openvino-classification-with-output_8_2.png">103-paddle-to-openvino-classification-with-outp..></a> 26-Jan-2024 01:05 224886
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -43,7 +43,7 @@ Table of contents:
|
|||
.. code:: ipython3
|
||||
|
||||
%pip install -q "nncf>=2.5.0"
|
||||
%pip install -q "transformers" datasets evaluate --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
%pip install -q transformers datasets evaluate --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
%pip install -q "openvino>=2023.1.0"
|
||||
|
||||
|
||||
|
|
@ -96,14 +96,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:37:51.906403: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:37:51.939998: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:38:10.763464: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:38:10.797722: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:37:52.576201: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:38:11.441310: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -191,7 +191,7 @@ PyTorch model formats are supported:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
return self.fget.__get__(instance, owner)()
|
||||
|
||||
|
||||
|
|
@ -286,13 +286,15 @@ The optimization process contains the following steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -303,12 +305,15 @@ The optimization process contains the following steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -329,13 +334,15 @@ The optimization process contains the following steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -346,13 +353,15 @@ The optimization process contains the following steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -548,17 +557,17 @@ Frames Per Second (FPS) for images.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.074 seconds per sentence, SPS: 13.56
|
||||
PyTorch model on CPU: 0.073 seconds per sentence, SPS: 13.77
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 48.16
|
||||
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 47.89
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 109.03
|
||||
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 109.72
|
||||
|
||||
|
||||
Finally, measure the inference performance of OpenVINO ``FP32`` and
|
||||
|
|
@ -566,7 +575,7 @@ Finally, measure the inference performance of OpenVINO ``FP32`` and
|
|||
Tool <https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html>`__
|
||||
in OpenVINO.
|
||||
|
||||
**Note**: The ``benchmark_app`` tool is able to measure the
|
||||
**NOTE**: The ``benchmark_app`` tool is able to measure the
|
||||
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
||||
models only. For more accurate performance, run ``benchmark_app`` in
|
||||
a terminal/command prompt after closing other applications. Run
|
||||
|
|
@ -599,7 +608,7 @@ in OpenVINO.
|
|||
Device with "device" name is not registered in the OpenVINO Runtime
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
||||
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
||||
Exception from src/inference/src/dev/core_impl.cpp:560:
|
||||
|
|
@ -632,7 +641,7 @@ in OpenVINO.
|
|||
Device with "device" name is not registered in the OpenVINO Runtime
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 166, in main
|
||||
supported_properties = benchmark.core.get_property(device, properties.supported_properties())
|
||||
RuntimeError: Exception from src/inference/src/core.cpp:228:
|
||||
Exception from src/inference/src/dev/core_impl.cpp:560:
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ device <https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_AU
|
|||
(or AUTO in short) selects the most suitable device for inference by
|
||||
considering the model precision, power efficiency and processing
|
||||
capability of the available `compute
|
||||
devices <https://docs.openvino.ai/2023.3/openvino_docs_Runtime_Inference_Modes_Overview.html>`__.
|
||||
devices <https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_Supported_Devices.html>`__.
|
||||
The model precision (such as ``FP32``, ``FP16``, ``INT8``, etc.) is the
|
||||
first consideration to filter out the devices that cannot run the
|
||||
network efficiently.
|
||||
|
|
@ -81,13 +81,13 @@ Import modules and create Core
|
|||
|
||||
import time
|
||||
import sys
|
||||
|
||||
|
||||
import openvino as ov
|
||||
|
||||
|
||||
from IPython.display import Markdown, display
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
|
||||
if "GPU" not in core.available_devices:
|
||||
display(Markdown('<div class="alert alert-block alert-danger"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>'))
|
||||
|
||||
|
|
@ -127,11 +127,11 @@ For more information about model conversion API, see this
|
|||
|
||||
import torchvision
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
base_model_dir = Path("./model")
|
||||
base_model_dir.mkdir(exist_ok=True)
|
||||
model_path = base_model_dir / "resnet50.xml"
|
||||
|
||||
|
||||
if not model_path.exists():
|
||||
pt_model = torchvision.models.resnet50(weights="DEFAULT")
|
||||
ov_model = ov.convert_model(pt_model, input=[[1,3,224,224]])
|
||||
|
|
@ -164,24 +164,24 @@ By default, ``compile_model`` API will select **AUTO** as
|
|||
|
||||
# Set LOG_LEVEL to LOG_INFO.
|
||||
core.set_property("AUTO", {"LOG_LEVEL":"LOG_INFO"})
|
||||
|
||||
|
||||
# Load the model onto the target device.
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
|
||||
if isinstance(compiled_model, ov.CompiledModel):
|
||||
print("Successfully compiled model without a device_name.")
|
||||
print("Successfully compiled model without a device_name.")
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[22:41:12.8903]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
||||
[22:41:12.8903]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
||||
[22:41:12.8904]I[plugin.cpp:536][AUTO] device:CPU, config:PERF_COUNT=NO
|
||||
[22:41:12.8904]I[plugin.cpp:541][AUTO] device:CPU, priority:0
|
||||
[22:41:12.8904]I[schedule.cpp:17][AUTO] scheduler starting
|
||||
[22:41:12.8904]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
||||
[22:41:13.0500]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
||||
[22:41:13.0502]I[plugin.cpp:569][AUTO] underlying hardware does not support hardware context
|
||||
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
||||
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
||||
[22:41:31.9445]I[plugin.cpp:536][AUTO] device:CPU, config:PERF_COUNT=NO
|
||||
[22:41:31.9445]I[plugin.cpp:541][AUTO] device:CPU, priority:0
|
||||
[22:41:31.9446]I[schedule.cpp:17][AUTO] scheduler starting
|
||||
[22:41:31.9446]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
||||
[22:41:32.0858]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
||||
[22:41:32.0860]I[plugin.cpp:569][AUTO] underlying hardware does not support hardware context
|
||||
Successfully compiled model without a device_name.
|
||||
|
||||
|
||||
|
|
@ -195,7 +195,7 @@ By default, ``compile_model`` API will select **AUTO** as
|
|||
.. parsed-literal::
|
||||
|
||||
Deleted compiled_model
|
||||
[22:41:13.0623]I[schedule.cpp:303][AUTO] scheduler ending
|
||||
[22:41:32.0982]I[schedule.cpp:303][AUTO] scheduler ending
|
||||
|
||||
|
||||
Explicitly pass AUTO as device_name to Core::compile_model API
|
||||
|
|
@ -210,9 +210,9 @@ improve readability of your code.
|
|||
|
||||
# Set LOG_LEVEL to LOG_NONE.
|
||||
core.set_property("AUTO", {"LOG_LEVEL":"LOG_NONE"})
|
||||
|
||||
|
||||
compiled_model = core.compile_model(model=ov_model, device_name="AUTO")
|
||||
|
||||
|
||||
if isinstance(compiled_model, ov.CompiledModel):
|
||||
print("Successfully compiled model using AUTO.")
|
||||
|
||||
|
|
@ -271,16 +271,16 @@ function, we will reuse it for preparing input data.
|
|||
.. code:: ipython3
|
||||
|
||||
from PIL import Image
|
||||
|
||||
|
||||
# Download the image from the openvino_notebooks storage
|
||||
image_filename = download_file(
|
||||
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco.jpg",
|
||||
directory="data"
|
||||
)
|
||||
|
||||
|
||||
image = Image.open(str(image_filename))
|
||||
input_transform = torchvision.models.ResNet50_Weights.DEFAULT.transforms()
|
||||
|
||||
|
||||
input_tensor = input_transform(image)
|
||||
input_tensor = input_tensor.unsqueeze(0).numpy()
|
||||
image
|
||||
|
|
@ -307,14 +307,14 @@ Load the model to GPU device and perform inference
|
|||
|
||||
if "GPU" not in core.available_devices:
|
||||
print(f"A GPU device is not available. Available devices are: {core.available_devices}")
|
||||
else :
|
||||
else :
|
||||
# Start time.
|
||||
gpu_load_start_time = time.perf_counter()
|
||||
compiled_model = core.compile_model(model=ov_model, device_name="GPU") # load to GPU
|
||||
|
||||
|
||||
# Execute the first inference.
|
||||
results = compiled_model(input_tensor)[0]
|
||||
|
||||
|
||||
# Measure time to the first inference.
|
||||
gpu_fil_end_time = time.perf_counter()
|
||||
gpu_fil_span = gpu_fil_end_time - gpu_load_start_time
|
||||
|
|
@ -340,11 +340,11 @@ executed on CPU until GPU is ready.
|
|||
# Start time.
|
||||
auto_load_start_time = time.perf_counter()
|
||||
compiled_model = core.compile_model(model=ov_model) # The device_name is AUTO by default.
|
||||
|
||||
|
||||
# Execute the first inference.
|
||||
results = compiled_model(input_tensor)[0]
|
||||
|
||||
|
||||
|
||||
|
||||
# Measure time to the first inference.
|
||||
auto_fil_end_time = time.perf_counter()
|
||||
auto_fil_span = auto_fil_end_time - auto_load_start_time
|
||||
|
|
@ -353,7 +353,7 @@ executed on CPU until GPU is ready.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Time to load model using AUTO device and get first inference: 0.15 seconds.
|
||||
Time to load model using AUTO device and get first inference: 0.16 seconds.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -406,11 +406,11 @@ Class and callback definition
|
|||
"""
|
||||
self.fps = 0
|
||||
self.latency = 0
|
||||
|
||||
|
||||
self.start_time = time.perf_counter()
|
||||
self.latency_list = []
|
||||
self.interval = interval
|
||||
|
||||
|
||||
def update(self, infer_request: ov.InferRequest) -> bool:
|
||||
"""
|
||||
Update the metrics if current ongoing @interval seconds duration is expired. Record the latency only if it is not expired.
|
||||
|
|
@ -432,12 +432,12 @@ Class and callback definition
|
|||
return True
|
||||
else :
|
||||
return False
|
||||
|
||||
|
||||
|
||||
|
||||
class InferContext:
|
||||
"""
|
||||
Inference context. Record and update peforamnce metrics via @metrics, set @feed_inference to False once @remaining_update_num <=0
|
||||
:member: metrics: instance of class PerformanceMetrics
|
||||
:member: metrics: instance of class PerformanceMetrics
|
||||
:member: remaining_update_num: the remaining times for peforamnce metrics updating.
|
||||
:member: feed_inference: if feed inference request is required or not.
|
||||
"""
|
||||
|
|
@ -452,7 +452,7 @@ Class and callback definition
|
|||
self.metrics = PerformanceMetrics(update_interval)
|
||||
self.remaining_update_num = num
|
||||
self.feed_inference = True
|
||||
|
||||
|
||||
def update(self, infer_request: ov.InferRequest):
|
||||
"""
|
||||
Update the context. Set @feed_inference to False if the number of remaining performance metric updates (@remaining_update_num) reaches 0
|
||||
|
|
@ -461,13 +461,13 @@ Class and callback definition
|
|||
"""
|
||||
if self.remaining_update_num <= 0 :
|
||||
self.feed_inference = False
|
||||
|
||||
|
||||
if self.metrics.update(infer_request) :
|
||||
self.remaining_update_num = self.remaining_update_num - 1
|
||||
if self.remaining_update_num <= 0 :
|
||||
self.feed_inference = False
|
||||
|
||||
|
||||
|
||||
|
||||
def completion_callback(infer_request: ov.InferRequest, context) -> None:
|
||||
"""
|
||||
callback for the inference request, pass the @infer_request to @context for updating
|
||||
|
|
@ -476,8 +476,8 @@ Class and callback definition
|
|||
:returns: None
|
||||
"""
|
||||
context.update(infer_request)
|
||||
|
||||
|
||||
|
||||
|
||||
# Performance metrics update interval (seconds) and number of times.
|
||||
metrics_update_interval = 10
|
||||
metrics_update_num = 6
|
||||
|
|
@ -493,29 +493,29 @@ Loop for inference and update the FPS/Latency every
|
|||
.. code:: ipython3
|
||||
|
||||
THROUGHPUT_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
||||
|
||||
|
||||
print("Compiling Model for AUTO device with THROUGHPUT hint")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"THROUGHPUT"})
|
||||
|
||||
|
||||
infer_queue = ov.AsyncInferQueue(compiled_model, 0) # Setting to 0 will query optimal number by default.
|
||||
infer_queue.set_callback(completion_callback)
|
||||
|
||||
|
||||
print(f"Start inference, {metrics_update_num: .0f} groups of FPS/latency will be measured over {metrics_update_interval: .0f}s intervals")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
while THROUGHPUT_hint_context.feed_inference:
|
||||
infer_queue.start_async(input_tensor, THROUGHPUT_hint_context)
|
||||
|
||||
|
||||
infer_queue.wait_all()
|
||||
|
||||
|
||||
# Take the FPS and latency of the latest period.
|
||||
THROUGHPUT_hint_fps = THROUGHPUT_hint_context.metrics.fps
|
||||
THROUGHPUT_hint_latency = THROUGHPUT_hint_context.metrics.latency
|
||||
|
||||
|
||||
print("Done")
|
||||
|
||||
|
||||
del compiled_model
|
||||
|
||||
|
||||
|
|
@ -531,32 +531,32 @@ Loop for inference and update the FPS/Latency every
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 184.87fps, latency: 30.81ms, time interval: 10.01s
|
||||
throughput: 179.69fps, latency: 31.58ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 185.21fps, latency: 31.62ms, time interval: 10.02s
|
||||
throughput: 182.30fps, latency: 32.10ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 185.47fps, latency: 31.54ms, time interval: 10.00s
|
||||
throughput: 180.62fps, latency: 32.36ms, time interval: 10.02s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 185.75fps, latency: 31.53ms, time interval: 10.01s
|
||||
throughput: 179.76fps, latency: 32.61ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 185.09fps, latency: 31.64ms, time interval: 10.00s
|
||||
throughput: 180.36fps, latency: 32.36ms, time interval: 10.02s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 184.42fps, latency: 31.76ms, time interval: 10.00s
|
||||
throughput: 179.77fps, latency: 32.58ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -575,30 +575,30 @@ Loop for inference and update the FPS/Latency for each
|
|||
.. code:: ipython3
|
||||
|
||||
LATENCY_hint_context = InferContext(metrics_update_interval, metrics_update_num)
|
||||
|
||||
|
||||
print("Compiling Model for AUTO Device with LATENCY hint")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
compiled_model = core.compile_model(model=ov_model, config={"PERFORMANCE_HINT":"LATENCY"})
|
||||
|
||||
|
||||
# Setting to 0 will query optimal number by default.
|
||||
infer_queue = ov.AsyncInferQueue(compiled_model, 0)
|
||||
infer_queue.set_callback(completion_callback)
|
||||
|
||||
|
||||
print(f"Start inference, {metrics_update_num: .0f} groups fps/latency will be out with {metrics_update_interval: .0f}s interval")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
while LATENCY_hint_context.feed_inference:
|
||||
infer_queue.start_async(input_tensor, LATENCY_hint_context)
|
||||
|
||||
|
||||
infer_queue.wait_all()
|
||||
|
||||
|
||||
# Take the FPS and latency of the latest period.
|
||||
LATENCY_hint_fps = LATENCY_hint_context.metrics.fps
|
||||
LATENCY_hint_latency = LATENCY_hint_context.metrics.latency
|
||||
|
||||
|
||||
print("Done")
|
||||
|
||||
|
||||
del compiled_model
|
||||
|
||||
|
||||
|
|
@ -614,32 +614,32 @@ Loop for inference and update the FPS/Latency for each
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 138.57fps, latency: 6.67ms, time interval: 10.00s
|
||||
throughput: 139.27fps, latency: 6.65ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 141.04fps, latency: 6.66ms, time interval: 10.00s
|
||||
throughput: 141.22fps, latency: 6.62ms, time interval: 10.01s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 140.74fps, latency: 6.66ms, time interval: 10.00s
|
||||
throughput: 140.71fps, latency: 6.64ms, time interval: 10.01s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 141.60fps, latency: 6.68ms, time interval: 10.01s
|
||||
throughput: 141.11fps, latency: 6.63ms, time interval: 10.01s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 141.73fps, latency: 6.68ms, time interval: 10.00s
|
||||
throughput: 141.26fps, latency: 6.62ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
throughput: 141.45fps, latency: 6.67ms, time interval: 10.00s
|
||||
throughput: 141.18fps, latency: 6.63ms, time interval: 10.00s
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -655,21 +655,21 @@ Difference in FPS and latency
|
|||
.. code:: ipython3
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
|
||||
TPUT = 0
|
||||
LAT = 1
|
||||
labels = ["THROUGHPUT hint", "LATENCY hint"]
|
||||
|
||||
fig1, ax1 = plt.subplots(1, 1)
|
||||
|
||||
fig1, ax1 = plt.subplots(1, 1)
|
||||
fig1.patch.set_visible(False)
|
||||
ax1.axis('tight')
|
||||
ax1.axis('off')
|
||||
|
||||
ax1.axis('tight')
|
||||
ax1.axis('off')
|
||||
|
||||
cell_text = []
|
||||
cell_text.append(['%.2f%s' % (THROUGHPUT_hint_fps," FPS"), '%.2f%s' % (THROUGHPUT_hint_latency, " ms")])
|
||||
cell_text.append(['%.2f%s' % (LATENCY_hint_fps," FPS"), '%.2f%s' % (LATENCY_hint_latency, " ms")])
|
||||
|
||||
table = ax1.table(cellText=cell_text, colLabels=["FPS (Higher is better)", "Latency (Lower is better)"], rowLabels=labels,
|
||||
|
||||
table = ax1.table(cellText=cell_text, colLabels=["FPS (Higher is better)", "Latency (Lower is better)"], rowLabels=labels,
|
||||
rowColours=["deepskyblue"] * 2, colColours=["deepskyblue"] * 2,
|
||||
cellLoc='center', loc='upper left')
|
||||
table.auto_set_font_size(False)
|
||||
|
|
@ -677,7 +677,7 @@ Difference in FPS and latency
|
|||
table.auto_set_column_width(0)
|
||||
table.auto_set_column_width(1)
|
||||
table.scale(1, 3)
|
||||
|
||||
|
||||
fig1.tight_layout()
|
||||
plt.show()
|
||||
|
||||
|
|
@ -691,28 +691,28 @@ Difference in FPS and latency
|
|||
# Output the difference.
|
||||
width = 0.4
|
||||
fontsize = 14
|
||||
|
||||
|
||||
plt.rc('font', size=fontsize)
|
||||
fig, ax = plt.subplots(1,2, figsize=(10, 8))
|
||||
|
||||
|
||||
rects1 = ax[0].bar([0], THROUGHPUT_hint_fps, width, label=labels[TPUT], color='#557f2d')
|
||||
rects2 = ax[0].bar([width], LATENCY_hint_fps, width, label=labels[LAT])
|
||||
ax[0].set_ylabel("frames per second")
|
||||
ax[0].set_xticks([width / 2])
|
||||
ax[0].set_xticks([width / 2])
|
||||
ax[0].set_xticklabels(["FPS"])
|
||||
ax[0].set_xlabel("Higher is better")
|
||||
|
||||
|
||||
rects1 = ax[1].bar([0], THROUGHPUT_hint_latency, width, label=labels[TPUT], color='#557f2d')
|
||||
rects2 = ax[1].bar([width], LATENCY_hint_latency, width, label=labels[LAT])
|
||||
ax[1].set_ylabel("milliseconds")
|
||||
ax[1].set_xticks([width / 2])
|
||||
ax[1].set_xticklabels(["Latency (ms)"])
|
||||
ax[1].set_xlabel("Lower is better")
|
||||
|
||||
|
||||
fig.suptitle('Performance Hints')
|
||||
fig.legend(labels, fontsize=fontsize)
|
||||
fig.tight_layout()
|
||||
|
||||
|
||||
plt.show()
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1b8fb134deae365a3c5b358b032e2ad1e5ccaccae51cb176f250ba946461d8d6
|
||||
size 25724
|
||||
oid sha256:acc35ef9a1d6c2acf74782263f65bd81b65d7fd6663a146ffb88c9a9b64e343d
|
||||
size 26542
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d311a41fc8902df2f1ca223ffcdcf1ff634d160376b890b3a571326912b1d88a
|
||||
size 40017
|
||||
oid sha256:ab4df6ece8c31c9c5d5ab7d51b7d628ef7fb5df7193d859b8cfd9547273b76e3
|
||||
size 39977
|
||||
|
|
|
|||
|
|
@ -1,10 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/106-auto-device-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/106-auto-device-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="106-auto-device-with-output_14_1.jpg">106-auto-device-with-output_14_1.jpg</a> 26-Jan-2024 01:05 121563
|
||||
<a href="106-auto-device-with-output_14_1.png">106-auto-device-with-output_14_1.png</a> 26-Jan-2024 01:05 869661
|
||||
<a href="106-auto-device-with-output_27_0.png">106-auto-device-with-output_27_0.png</a> 26-Jan-2024 01:05 25724
|
||||
<a href="106-auto-device-with-output_28_0.png">106-auto-device-with-output_28_0.png</a> 26-Jan-2024 01:05 40017
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -360,14 +360,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -378,14 +379,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -403,13 +405,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -420,14 +424,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -50,84 +50,88 @@ Table of contents:
|
|||
.. parsed-literal::
|
||||
|
||||
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
|
||||
Requirement already satisfied: datasets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.16.1)
|
||||
Requirement already satisfied: datasets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.17.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Collecting torchmetrics>=0.11.0
|
||||
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl.metadata (20 kB)
|
||||
Requirement already satisfied: torch>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.1.0+cpu)
|
||||
Requirement already satisfied: torch>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.1.0+cpu)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.13.1)
|
||||
Requirement already satisfied: numpy>=1.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (1.23.5)
|
||||
Requirement already satisfied: pyarrow>=8.0.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (15.0.0)
|
||||
Requirement already satisfied: pyarrow-hotfix in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.6)
|
||||
Requirement already satisfied: dill<0.3.8,>=0.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.3.7)
|
||||
Requirement already satisfied: pandas in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.0.3)
|
||||
Requirement already satisfied: requests>=2.19.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.31.0)
|
||||
Requirement already satisfied: tqdm>=4.62.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (4.66.1)
|
||||
Requirement already satisfied: xxhash in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.4.1)
|
||||
Requirement already satisfied: multiprocess in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.70.15)
|
||||
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets) (2023.10.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: aiohttp in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.9.1)
|
||||
Requirement already satisfied: huggingface-hub>=0.19.4 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.20.3)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (23.2)
|
||||
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (6.0.1)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.13.1)
|
||||
Requirement already satisfied: numpy>=1.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (1.23.5)
|
||||
Requirement already satisfied: pyarrow>=12.0.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (15.0.0)
|
||||
Requirement already satisfied: pyarrow-hotfix in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.6)
|
||||
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.3.8)
|
||||
Requirement already satisfied: pandas in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.0.3)
|
||||
Requirement already satisfied: requests>=2.19.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (2.31.0)
|
||||
Requirement already satisfied: tqdm>=4.62.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (4.66.1)
|
||||
Requirement already satisfied: xxhash in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.4.1)
|
||||
Requirement already satisfied: multiprocess in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.70.16)
|
||||
Requirement already satisfied: fsspec<=2023.10.0,>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets) (2023.10.0)
|
||||
Requirement already satisfied: aiohttp in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (3.9.3)
|
||||
Requirement already satisfied: huggingface-hub>=0.19.4 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (0.20.3)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (23.2)
|
||||
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from datasets) (6.0.1)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Collecting lightning-utilities>=0.8.0 (from torchmetrics>=0.11.0)
|
||||
Using cached lightning_utilities-0.10.1-py3-none-any.whl.metadata (4.8 kB)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torchmetrics>=0.11.0) (4.9.0)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torchmetrics>=0.11.0) (4.9.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1.3)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1.0) (3.1.3)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: attrs>=17.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (23.2.0)
|
||||
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (6.0.4)
|
||||
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.9.4)
|
||||
Requirement already satisfied: frozenlist>=1.1.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.4.1)
|
||||
Requirement already satisfied: aiosignal>=1.1.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.1)
|
||||
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (4.0.3)
|
||||
Requirement already satisfied: aiosignal>=1.1.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.3.1)
|
||||
Requirement already satisfied: attrs>=17.3.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (23.2.0)
|
||||
Requirement already satisfied: frozenlist>=1.1.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.4.1)
|
||||
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (6.0.5)
|
||||
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (1.9.4)
|
||||
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from aiohttp->datasets) (4.0.3)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: setuptools in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from lightning-utilities>=0.8.0->torchmetrics>=0.11.0) (69.0.3)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2.1.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2023.11.17)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1.0) (2.1.4)
|
||||
Requirement already satisfied: setuptools in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from lightning-utilities>=0.8.0->torchmetrics>=0.11.0) (69.0.3)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2.2.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.19.0->datasets) (2024.2.2)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2.8.2)
|
||||
Requirement already satisfied: pytz>=2020.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.3.post1)
|
||||
Requirement already satisfied: tzdata>=2022.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.4)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1.0) (1.3.0)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1.0) (2.1.5)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2.8.2)
|
||||
Requirement already satisfied: pytz>=2020.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2024.1)
|
||||
Requirement already satisfied: tzdata>=2022.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pandas->datasets) (2023.4)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1.0) (1.3.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)
|
||||
Using cached torchmetrics-1.3.0.post0-py3-none-any.whl (840 kB)
|
||||
Using cached lightning_utilities-0.10.1-py3-none-any.whl (24 kB)
|
||||
|
||||
|
|
@ -220,9 +224,9 @@ IR).
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:593: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:593: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:632: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py:632: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||
|
||||
|
||||
|
|
@ -260,7 +264,7 @@ dataset.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1429: FutureWarning: The repository for patrickvonplaten/librispeech_asr_dummy contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/patrickvonplaten/librispeech_asr_dummy
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1454: FutureWarning: The repository for patrickvonplaten/librispeech_asr_dummy contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/patrickvonplaten/librispeech_asr_dummy
|
||||
You can avoid this message in future by passing the argument `trust_remote_code=True`.
|
||||
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
|
||||
warnings.warn(
|
||||
|
|
@ -334,14 +338,14 @@ steps:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:43:51.310601: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:43:51.341489: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:44:10.617295: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:44:10.648558: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:43:51.934215: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:44:11.243901: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
|
||||
|
|
@ -351,14 +355,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -369,14 +374,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -422,14 +428,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -440,14 +447,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -582,7 +590,7 @@ quantized model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:62: FutureWarning: Importing `WordErrorRate` from `torchmetrics` was deprecated and will be removed in 2.0. Import `WordErrorRate` from `torchmetrics.text` instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:62: FutureWarning: Importing `WordErrorRate` from `torchmetrics` was deprecated and will be removed in 2.0. Import `WordErrorRate` from `torchmetrics.text` instead.
|
||||
_future_warning(
|
||||
|
||||
|
||||
|
|
@ -659,35 +667,35 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Read model took 54.55 ms
|
||||
[ INFO ] Read model took 56.59 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
||||
[ INFO ] Model outputs:
|
||||
[ INFO ] logits , 1170 (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [?,?,32]
|
||||
[ INFO ] 1170 , logits (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [?,?,32]
|
||||
[Step 5/11] Resizing model to match image sizes and given batch
|
||||
[ INFO ] Model batch size: 1
|
||||
[ INFO ] Reshaping model: 'input_values': [1,30480]
|
||||
[ INFO ] Reshape model took 30.95 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Reshape model took 30.54 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
||||
[ INFO ] Model outputs:
|
||||
[ INFO ] logits , 1170 (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [1,95,32]
|
||||
[ INFO ] 1170 , logits (node: __module.lm_head/aten::linear/Add) : f32 / [...] / [1,95,32]
|
||||
[Step 7/11] Loading the model to the device
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 560.02 ms
|
||||
[ INFO ] Compile model took 559.12 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] NUM_STREAMS: 6
|
||||
[ INFO ] AFFINITY: Affinity.CORE
|
||||
[ INFO ] INFERENCE_NUM_THREADS: 24
|
||||
|
|
@ -711,21 +719,21 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 58.98 ms
|
||||
[ INFO ] First inference took 56.43 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 2766 iterations
|
||||
[ INFO ] Duration: 60197.24 ms
|
||||
[ INFO ] Count: 2778 iterations
|
||||
[ INFO ] Duration: 60216.97 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 130.42 ms
|
||||
[ INFO ] Average: 130.38 ms
|
||||
[ INFO ] Min: 64.93 ms
|
||||
[ INFO ] Max: 148.03 ms
|
||||
[ INFO ] Throughput: 45.95 FPS
|
||||
[ INFO ] Median: 129.97 ms
|
||||
[ INFO ] Average: 129.78 ms
|
||||
[ INFO ] Min: 110.04 ms
|
||||
[ INFO ] Max: 151.03 ms
|
||||
[ INFO ] Throughput: 46.13 FPS
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -759,7 +767,7 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Read model took 70.41 ms
|
||||
[ INFO ] Read model took 67.98 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [?,?]
|
||||
|
|
@ -772,7 +780,7 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Reshape model took 37.13 ms
|
||||
[ INFO ] Reshape model took 37.53 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_values (node: input_values) : f32 / [...] / [1,30480]
|
||||
|
|
@ -783,7 +791,7 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 999.07 ms
|
||||
[ INFO ] Compile model took 1060.01 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -803,10 +811,6 @@ models.
|
|||
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
||||
[ INFO ] CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
|
||||
[Step 9/11] Creating infer requests and preparing input tensors
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ WARNING ] No input files were given for input 'input_values'!. This input will be filled with random values!
|
||||
[ INFO ] Fill input 'input_values' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 60000 ms duration)
|
||||
|
|
@ -815,19 +819,19 @@ models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 56.74 ms
|
||||
[ INFO ] First inference took 62.07 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 4284 iterations
|
||||
[ INFO ] Duration: 60076.08 ms
|
||||
[ INFO ] Count: 4296 iterations
|
||||
[ INFO ] Duration: 60070.01 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 84.26 ms
|
||||
[ INFO ] Average: 83.96 ms
|
||||
[ INFO ] Min: 73.47 ms
|
||||
[ INFO ] Max: 111.27 ms
|
||||
[ INFO ] Throughput: 71.31 FPS
|
||||
[ INFO ] Median: 83.98 ms
|
||||
[ INFO ] Average: 83.75 ms
|
||||
[ INFO ] Min: 45.06 ms
|
||||
[ INFO ] Max: 106.14 ms
|
||||
[ INFO ] Throughput: 71.52 FPS
|
||||
|
||||
|
|
|
|||
|
|
@ -218,7 +218,7 @@ for that property.
|
|||
indent = len(max(supported_properties, key=len))
|
||||
|
||||
for property_key in supported_properties:
|
||||
if property_key not in ('SUPPORTED_PROPERTIES'):
|
||||
if property_key not in ('SUPPORTED_METRICS', 'SUPPORTED_CONFIG_KEYS', 'SUPPORTED_PROPERTIES'):
|
||||
try:
|
||||
property_val = core.get_property(device, property_key)
|
||||
except TypeError:
|
||||
|
|
@ -1337,7 +1337,7 @@ Process Results
|
|||
# Draw info at the top left such as current fps, the devices and the performance hint being used
|
||||
cv2.putText(frame, f"fps {str(round(frame_fps[current_frame], 2))}", (5, 20), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||
cv2.putText(frame, f"device {device_name}", (5, 40), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||
cv2.putText(frame, f"hint {compiled_model.get_property('PERFORMANCE_HINT').name}", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||
cv2.putText(frame, f"hint {compiled_model.get_property('PERFORMANCE_HINT')}", (5, 60), cv2.FONT_ITALIC, 0.6, (0, 0, 0), 1, cv2.LINE_AA)
|
||||
|
||||
# prediction contains [image_id, label, conf, x_min, y_min, x_max, y_max] according to model
|
||||
for prediction in np.squeeze(results[current_frame]):
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ The quantization and pre-post-processing API are not included here as
|
|||
they change the precision (quantization) or processing graph
|
||||
(prepostprocessor). You can find examples of how to apply them to
|
||||
optimize performance on OpenVINO IR files in
|
||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__ and
|
||||
`111-detection-quantization <111-detection-quantization-with-output.html>`__ and
|
||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||
|
||||
|image0|
|
||||
|
|
@ -141,7 +141,7 @@ requirements of this particular object detection model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<DisplayHandle display_id=b3ed024cde96c857177f3da66878c56a>
|
||||
<DisplayHandle display_id=c7531cff1487c41296f1ac25e2e96b93>
|
||||
|
||||
|
||||
|
||||
|
|
@ -202,62 +202,54 @@ PyTorch Hub and small enough to see the difference in performance.
|
|||
.. parsed-literal::
|
||||
|
||||
|
||||
5%|▌ | 208k/3.87M [00:00<00:01, 2.11MB/s]
|
||||
7%|▋ | 272k/3.87M [00:00<00:01, 2.23MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
15%|█▍ | 576k/3.87M [00:00<00:01, 3.05MB/s]
|
||||
19%|█▉ | 752k/3.87M [00:00<00:00, 3.64MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
24%|██▍ | 952k/3.87M [00:00<00:00, 3.36MB/s]
|
||||
29%|██▉ | 1.13M/3.87M [00:00<00:00, 3.87MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
33%|███▎ | 1.30M/3.87M [00:00<00:00, 3.51MB/s]
|
||||
39%|███▉ | 1.52M/3.87M [00:00<00:00, 3.61MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
43%|████▎ | 1.67M/3.87M [00:00<00:00, 3.60MB/s]
|
||||
49%|████▉ | 1.89M/3.87M [00:00<00:00, 3.68MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
52%|█████▏ | 2.02M/3.87M [00:00<00:00, 3.16MB/s]
|
||||
61%|██████▏ | 2.38M/3.87M [00:00<00:00, 4.06MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
61%|██████ | 2.36M/3.87M [00:00<00:00, 3.26MB/s]
|
||||
72%|███████▏ | 2.77M/3.87M [00:00<00:00, 3.82MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
70%|███████ | 2.72M/3.87M [00:00<00:00, 3.38MB/s]
|
||||
83%|████████▎ | 3.23M/3.87M [00:00<00:00, 4.07MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
80%|███████▉ | 3.09M/3.87M [00:00<00:00, 3.49MB/s]
|
||||
94%|█████████▍| 3.63M/3.87M [00:01<00:00, 3.85MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
89%|████████▉ | 3.46M/3.87M [00:01<00:00, 3.58MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
100%|██████████| 3.87M/3.87M [00:01<00:00, 3.87MB/s]
|
||||
|
||||
|
||||
99%|█████████▉| 3.84M/3.87M [00:01<00:00, 3.67MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
100%|██████████| 3.87M/3.87M [00:01<00:00, 3.45MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
|
@ -473,12 +465,12 @@ optimizations applied. We will treat it as our baseline.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU. First inference time: 0.0268 seconds
|
||||
PyTorch model on CPU. First inference time: 0.0280 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.0213 seconds per image (46.98 FPS)
|
||||
PyTorch model on CPU: 0.0218 seconds per image (45.96 FPS)
|
||||
|
||||
|
||||
ONNX model
|
||||
|
|
@ -528,12 +520,12 @@ Representation (IR) to leverage the OpenVINO Runtime.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
ONNX model on CPU. First inference time: 0.0186 seconds
|
||||
ONNX model on CPU. First inference time: 0.0174 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
ONNX model on CPU: 0.0123 seconds per image (81.52 FPS)
|
||||
ONNX model on CPU: 0.0136 seconds per image (73.63 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model
|
||||
|
|
@ -570,12 +562,12 @@ accuracy drop. That’s why we skip that step in this notebook.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU. First inference time: 0.0166 seconds
|
||||
OpenVINO model on CPU. First inference time: 0.0153 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU: 0.0122 seconds per image (81.86 FPS)
|
||||
OpenVINO model on CPU: 0.0122 seconds per image (82.17 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model on GPU
|
||||
|
|
@ -636,12 +628,12 @@ If it is the case, don’t use it.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + more threads on CPU. First inference time: 0.0153 seconds
|
||||
OpenVINO model + more threads on CPU. First inference time: 0.0150 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + more threads on CPU: 0.0122 seconds per image (82.19 FPS)
|
||||
OpenVINO model + more threads on CPU: 0.0122 seconds per image (81.82 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model in latency mode
|
||||
|
|
@ -672,12 +664,12 @@ devices as well.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO. First inference time: 0.0154 seconds
|
||||
OpenVINO model on AUTO. First inference time: 0.0153 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO: 0.0125 seconds per image (80.26 FPS)
|
||||
OpenVINO model on AUTO: 0.0125 seconds per image (80.25 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model in latency mode + shared memory
|
||||
|
|
@ -712,12 +704,12 @@ performance!
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + shared memory on AUTO. First inference time: 0.0124 seconds
|
||||
OpenVINO model + shared memory on AUTO. First inference time: 0.0113 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + shared memory on AUTO: 0.0054 seconds per image (184.61 FPS)
|
||||
OpenVINO model + shared memory on AUTO: 0.0054 seconds per image (186.01 FPS)
|
||||
|
||||
|
||||
Other tricks
|
||||
|
|
@ -728,8 +720,8 @@ Other tricks
|
|||
There are other tricks for performance improvement, such as quantization
|
||||
and pre-post-processing or dedicated to throughput mode. To get even
|
||||
more from your model, please visit
|
||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__,
|
||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__, and
|
||||
`111-detection-quantization <111-detection-quantization-with-output.html>`__,
|
||||
`118-optimize-preprocessing <118-optimize-preprocessing>`__, and
|
||||
`109-throughput-tricks <109-throughput-tricks-with-output.html>`__.
|
||||
|
||||
Performance comparison
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4f157b7f097e536fab10e5a103ae836be062794d8d7b4f9b16185e2e6900d2f5
|
||||
size 56983
|
||||
oid sha256:ae331df2cb3993ba23bef0c441831024024f4f1f946c2da17a2ef6a2e6a68d8f
|
||||
size 52876
|
||||
|
|
|
|||
|
|
@ -1,14 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-latency-tricks-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-latency-tricks-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="109-latency-tricks-with-output_15_0.jpg">109-latency-tricks-with-output_15_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_18_0.jpg">109-latency-tricks-with-output_18_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_20_0.jpg">109-latency-tricks-with-output_20_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_24_0.jpg">109-latency-tricks-with-output_24_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_26_0.jpg">109-latency-tricks-with-output_26_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_28_0.jpg">109-latency-tricks-with-output_28_0.jpg</a> 26-Jan-2024 01:04 162715
|
||||
<a href="109-latency-tricks-with-output_31_0.png">109-latency-tricks-with-output_31_0.png</a> 26-Jan-2024 01:04 56983
|
||||
<a href="109-latency-tricks-with-output_5_0.jpg">109-latency-tricks-with-output_5_0.jpg</a> 26-Jan-2024 01:04 155828
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -24,7 +24,7 @@ The quantization and pre-post-processing API are not included here as
|
|||
they change the precision (quantization) or processing graph
|
||||
(prepostprocessor). You can find examples of how to apply them to
|
||||
optimize performance on OpenVINO IR files in
|
||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__ and
|
||||
`111-detection-quantization <111-detection-quantization-with-output.html>`__ and
|
||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||
|
||||
|image0|
|
||||
|
|
@ -93,7 +93,7 @@ Prerequisites
|
|||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, List, Tuple
|
||||
|
||||
|
||||
# Fetch `notebook_utils` module
|
||||
import urllib.request
|
||||
urllib.request.urlretrieve(
|
||||
|
|
@ -116,24 +116,24 @@ object detection model.
|
|||
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
|
||||
FRAMES_NUMBER = 1024
|
||||
|
||||
|
||||
IMAGE_WIDTH = 640
|
||||
IMAGE_HEIGHT = 480
|
||||
|
||||
|
||||
# load image
|
||||
image = utils.load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg")
|
||||
image = cv2.resize(image, dsize=(IMAGE_WIDTH, IMAGE_HEIGHT), interpolation=cv2.INTER_AREA)
|
||||
|
||||
|
||||
# preprocess it for YOLOv5
|
||||
input_image = image / 255.0
|
||||
input_image = np.transpose(input_image, axes=(2, 0, 1))
|
||||
input_image = np.expand_dims(input_image, axis=0)
|
||||
|
||||
|
||||
# simulate video with many frames
|
||||
video_frames = np.tile(input_image, (FRAMES_NUMBER, 1, 1, 1, 1))
|
||||
|
||||
|
||||
# show the image
|
||||
utils.show_array(image)
|
||||
|
||||
|
|
@ -146,7 +146,7 @@ object detection model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<DisplayHandle display_id=27f64c41c38d04ac2269516dbbad4c95>
|
||||
<DisplayHandle display_id=bbb34b7fd1ad545280d19661bf0bd4c3>
|
||||
|
||||
|
||||
|
||||
|
|
@ -164,13 +164,13 @@ PyTorch Hub and small enough to see the difference in performance.
|
|||
|
||||
import torch
|
||||
from IPython.utils import io
|
||||
|
||||
|
||||
# directory for all models
|
||||
base_model_dir = Path("model")
|
||||
|
||||
|
||||
model_name = "yolov5n"
|
||||
model_path = base_model_dir / model_name
|
||||
|
||||
|
||||
# load YOLOv5n from PyTorch Hub
|
||||
pytorch_model = torch.hub.load("ultralytics/yolov5", "custom", path=model_path, device="cpu", skip_validation=True)
|
||||
# don't print full model architecture
|
||||
|
|
@ -186,12 +186,12 @@ PyTorch Hub and small enough to see the difference in performance.
|
|||
.. parsed-literal::
|
||||
|
||||
YOLOv5 🚀 2023-4-21 Python-3.8.10 torch-2.1.0+cpu CPU
|
||||
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Fusing layers...
|
||||
Fusing layers...
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -201,7 +201,7 @@ PyTorch Hub and small enough to see the difference in performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Adding AutoShape...
|
||||
Adding AutoShape...
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -223,10 +223,10 @@ benchmarking process.
|
|||
.. code:: ipython3
|
||||
|
||||
import openvino as ov
|
||||
|
||||
|
||||
# initialize OpenVINO
|
||||
core = ov.Core()
|
||||
|
||||
|
||||
# print available devices
|
||||
for device in core.available_devices:
|
||||
device_name = core.get_property(device, "FULL_DEVICE_NAME")
|
||||
|
|
@ -250,8 +250,8 @@ second (FPS).
|
|||
.. code:: ipython3
|
||||
|
||||
from openvino.runtime import AsyncInferQueue
|
||||
|
||||
|
||||
|
||||
|
||||
def benchmark_model(model: Any, frames: np.ndarray, async_queue: AsyncInferQueue = None, benchmark_name: str = "OpenVINO model", device_name: str = "CPU") -> float:
|
||||
"""
|
||||
Helper function for benchmarking the model. It measures the time and prints results.
|
||||
|
|
@ -264,7 +264,7 @@ second (FPS).
|
|||
end = time.perf_counter()
|
||||
first_infer_time = end - start
|
||||
print(f"{benchmark_name} on {device_name}. First inference time: {first_infer_time :.4f} seconds")
|
||||
|
||||
|
||||
# benchmarking
|
||||
start = time.perf_counter()
|
||||
for batch in frames:
|
||||
|
|
@ -273,15 +273,15 @@ second (FPS).
|
|||
if async_queue:
|
||||
async_queue.wait_all()
|
||||
end = time.perf_counter()
|
||||
|
||||
|
||||
# elapsed time
|
||||
infer_time = end - start
|
||||
|
||||
|
||||
# print second per image and FPS
|
||||
mean_infer_time = infer_time / FRAMES_NUMBER
|
||||
mean_fps = FRAMES_NUMBER / infer_time
|
||||
print(f"{benchmark_name} on {device_name}: {mean_infer_time :.4f} seconds per image ({mean_fps :.2f} FPS)")
|
||||
|
||||
|
||||
return mean_fps
|
||||
|
||||
The following functions aim to post-process results and draw boxes on
|
||||
|
|
@ -300,21 +300,21 @@ the image.
|
|||
"cell phone", "microwave", "oven", "oaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
|
||||
"hair drier", "toothbrush"
|
||||
]
|
||||
|
||||
|
||||
# Colors for the classes above (Rainbow Color Map).
|
||||
colors = cv2.applyColorMap(
|
||||
src=np.arange(0, 255, 255 / len(classes), dtype=np.float32).astype(np.uint8),
|
||||
colormap=cv2.COLORMAP_RAINBOW,
|
||||
).squeeze()
|
||||
|
||||
|
||||
|
||||
|
||||
def postprocess(detections: np.ndarray) -> List[Tuple]:
|
||||
"""
|
||||
Postprocess the raw results from the model.
|
||||
"""
|
||||
# candidates - probability > 0.25
|
||||
detections = detections[detections[..., 4] > 0.25]
|
||||
|
||||
|
||||
boxes = []
|
||||
labels = []
|
||||
scores = []
|
||||
|
|
@ -328,22 +328,22 @@ the image.
|
|||
)
|
||||
labels.append(int(label))
|
||||
scores.append(float(score))
|
||||
|
||||
|
||||
# Apply non-maximum suppression to get rid of many overlapping entities.
|
||||
# See https://paperswithcode.com/method/non-maximum-suppression
|
||||
# This algorithm returns indices of objects to keep.
|
||||
indices = cv2.dnn.NMSBoxes(
|
||||
bboxes=boxes, scores=scores, score_threshold=0.25, nms_threshold=0.5
|
||||
)
|
||||
|
||||
|
||||
# If there are no boxes.
|
||||
if len(indices) == 0:
|
||||
return []
|
||||
|
||||
|
||||
# Filter detected objects.
|
||||
return [(labels[idx], scores[idx], boxes[idx]) for idx in indices.flatten()]
|
||||
|
||||
|
||||
|
||||
|
||||
def draw_boxes(img: np.ndarray, boxes):
|
||||
"""
|
||||
Draw detected boxes on the image.
|
||||
|
|
@ -355,7 +355,7 @@ the image.
|
|||
x2 = box[0] + box[2]
|
||||
y2 = box[1] + box[3]
|
||||
cv2.rectangle(img=img, pt1=box[:2], pt2=(x2, y2), color=color, thickness=2)
|
||||
|
||||
|
||||
# Draw a label name inside the box.
|
||||
cv2.putText(
|
||||
img=img,
|
||||
|
|
@ -367,17 +367,17 @@ the image.
|
|||
thickness=1,
|
||||
lineType=cv2.LINE_AA,
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
def show_result(results: np.ndarray):
|
||||
"""
|
||||
Postprocess the raw results, draw boxes and show the image.
|
||||
"""
|
||||
output_img = image.copy()
|
||||
|
||||
|
||||
detections = postprocess(results)
|
||||
draw_boxes(output_img, detections)
|
||||
|
||||
|
||||
utils.show_array(output_img)
|
||||
|
||||
Optimizations
|
||||
|
|
@ -400,7 +400,7 @@ optimizations applied. We will treat it as our baseline.
|
|||
.. code:: ipython3
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
with torch.no_grad():
|
||||
result = pytorch_model(torch.as_tensor(video_frames[0])).detach().numpy()[0]
|
||||
show_result(result)
|
||||
|
|
@ -413,12 +413,12 @@ optimizations applied. We will treat it as our baseline.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU. First inference time: 0.0201 seconds
|
||||
PyTorch model on CPU. First inference time: 0.0224 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.0192 seconds per image (52.19 FPS)
|
||||
PyTorch model on CPU: 0.0221 seconds per image (45.32 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model
|
||||
|
|
@ -438,23 +438,23 @@ step in this notebook.
|
|||
.. code:: ipython3
|
||||
|
||||
onnx_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}").with_suffix(".onnx")
|
||||
|
||||
|
||||
# export PyTorch model to ONNX if it doesn't already exist
|
||||
if not onnx_path.exists():
|
||||
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||
torch.onnx.export(pytorch_model, dummy_input, onnx_path)
|
||||
|
||||
|
||||
# convert ONNX model to IR, use FP16
|
||||
ov_model = ov.convert_model(onnx_path)
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
ov_cpu_model = core.compile_model(ov_model, device_name="CPU")
|
||||
|
||||
|
||||
result = ov_cpu_model(video_frames[0])[ov_cpu_model.output(0)][0]
|
||||
show_result(result)
|
||||
ov_cpu_fps = benchmark_model(model=ov_cpu_model, frames=video_frames, benchmark_name="OpenVINO model")
|
||||
|
||||
|
||||
del ov_cpu_model # release resources
|
||||
|
||||
|
||||
|
|
@ -464,12 +464,12 @@ step in this notebook.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU. First inference time: 0.0134 seconds
|
||||
OpenVINO model on CPU. First inference time: 0.0142 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU: 0.0070 seconds per image (142.35 FPS)
|
||||
OpenVINO model on CPU: 0.0071 seconds per image (141.49 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model + bigger batch
|
||||
|
|
@ -488,13 +488,13 @@ hardware and model.
|
|||
.. code:: ipython3
|
||||
|
||||
batch_size = 4
|
||||
|
||||
|
||||
onnx_batch_path = base_model_dir / Path(f"{model_name}_{IMAGE_WIDTH}_{IMAGE_HEIGHT}_batch_{batch_size}").with_suffix(".onnx")
|
||||
|
||||
|
||||
if not onnx_batch_path.exists():
|
||||
dummy_input = torch.randn(batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||
torch.onnx.export(pytorch_model, dummy_input, onnx_batch_path)
|
||||
|
||||
|
||||
# export the model with the bigger batch size
|
||||
ov_batch_model = ov.convert_model(onnx_batch_path)
|
||||
|
||||
|
|
@ -510,13 +510,13 @@ hardware and model.
|
|||
.. code:: ipython3
|
||||
|
||||
ov_cpu_batch_model = core.compile_model(ov_batch_model, device_name="CPU")
|
||||
|
||||
|
||||
batched_video_frames = video_frames.reshape([-1, batch_size, 3, IMAGE_HEIGHT, IMAGE_WIDTH])
|
||||
|
||||
|
||||
result = ov_cpu_batch_model(batched_video_frames[0])[ov_cpu_batch_model.output(0)][0]
|
||||
show_result(result)
|
||||
ov_cpu_batch_fps = benchmark_model(model=ov_cpu_batch_model, frames=batched_video_frames, benchmark_name="OpenVINO model + bigger batch")
|
||||
|
||||
|
||||
del ov_cpu_batch_model # release resources
|
||||
|
||||
|
||||
|
|
@ -526,12 +526,12 @@ hardware and model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + bigger batch on CPU. First inference time: 0.0435 seconds
|
||||
OpenVINO model + bigger batch on CPU. First inference time: 0.0486 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model + bigger batch on CPU: 0.0068 seconds per image (147.59 FPS)
|
||||
OpenVINO model + bigger batch on CPU: 0.0068 seconds per image (146.44 FPS)
|
||||
|
||||
|
||||
Asynchronous processing
|
||||
|
|
@ -561,17 +561,17 @@ the pipeline.
|
|||
result = infer_request.get_output_tensor(0).data[0]
|
||||
show_result(result)
|
||||
pass
|
||||
|
||||
|
||||
infer_queue = ov.AsyncInferQueue(ov_model)
|
||||
infer_queue.set_callback(callback) # set callback to post-process (show) results
|
||||
|
||||
|
||||
infer_queue.start_async(video_frames[0])
|
||||
infer_queue.wait_all()
|
||||
|
||||
|
||||
# don't show output for the remaining frames
|
||||
infer_queue.set_callback(lambda x, y: {})
|
||||
fps = benchmark_model(model=infer_queue.start_async, frames=video_frames, async_queue=infer_queue, benchmark_name=benchmark_name, device_name=device_name)
|
||||
|
||||
|
||||
del infer_queue # release resources
|
||||
return fps
|
||||
|
||||
|
|
@ -592,9 +592,9 @@ feature, which sets the batch size to the optimal level.
|
|||
.. code:: ipython3
|
||||
|
||||
ov_cpu_through_model = core.compile_model(ov_model, device_name="CPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||
|
||||
|
||||
ov_cpu_through_fps = benchmark_async_mode(ov_cpu_through_model, benchmark_name="OpenVINO model", device_name="CPU (THROUGHPUT)")
|
||||
|
||||
|
||||
del ov_cpu_through_model # release resources
|
||||
|
||||
|
||||
|
|
@ -604,12 +604,12 @@ feature, which sets the batch size to the optimal level.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU (THROUGHPUT). First inference time: 0.0260 seconds
|
||||
OpenVINO model on CPU (THROUGHPUT). First inference time: 0.0221 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on CPU (THROUGHPUT): 0.0040 seconds per image (248.03 FPS)
|
||||
OpenVINO model on CPU (THROUGHPUT): 0.0040 seconds per image (249.65 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model in throughput mode on GPU
|
||||
|
|
@ -633,9 +633,9 @@ execution.
|
|||
if "GPU" in core.available_devices:
|
||||
# compile for GPU
|
||||
ov_gpu_model = core.compile_model(ov_model, device_name="GPU", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||
|
||||
|
||||
ov_gpu_fps = benchmark_async_mode(ov_gpu_model, benchmark_name="OpenVINO model", device_name="GPU (THROUGHPUT)")
|
||||
|
||||
|
||||
del ov_gpu_model # release resources
|
||||
|
||||
OpenVINO IR model in throughput mode on AUTO
|
||||
|
|
@ -651,9 +651,9 @@ performance hint.
|
|||
.. code:: ipython3
|
||||
|
||||
ov_auto_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "THROUGHPUT"})
|
||||
|
||||
|
||||
ov_auto_fps = benchmark_async_mode(ov_auto_model, benchmark_name="OpenVINO model", device_name="AUTO (THROUGHPUT)")
|
||||
|
||||
|
||||
del ov_auto_model # release resources
|
||||
|
||||
|
||||
|
|
@ -663,12 +663,12 @@ performance hint.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO (THROUGHPUT). First inference time: 0.0244 seconds
|
||||
OpenVINO model on AUTO (THROUGHPUT). First inference time: 0.0235 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO (THROUGHPUT): 0.0040 seconds per image (250.62 FPS)
|
||||
OpenVINO model on AUTO (THROUGHPUT): 0.0040 seconds per image (249.90 FPS)
|
||||
|
||||
|
||||
OpenVINO IR model in cumulative throughput mode on AUTO
|
||||
|
|
@ -685,7 +685,7 @@ activate all devices.
|
|||
.. code:: ipython3
|
||||
|
||||
ov_auto_cumulative_model = core.compile_model(ov_model, device_name="AUTO", config={"PERFORMANCE_HINT": "CUMULATIVE_THROUGHPUT"})
|
||||
|
||||
|
||||
ov_auto_cumulative_fps = benchmark_async_mode(ov_auto_cumulative_model, benchmark_name="OpenVINO model", device_name="AUTO (CUMULATIVE THROUGHPUT)")
|
||||
|
||||
|
||||
|
|
@ -695,12 +695,12 @@ activate all devices.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT). First inference time: 0.0211 seconds
|
||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT). First inference time: 0.0227 seconds
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT): 0.0040 seconds per image (249.40 FPS)
|
||||
OpenVINO model on AUTO (CUMULATIVE THROUGHPUT): 0.0040 seconds per image (250.47 FPS)
|
||||
|
||||
|
||||
Other tricks
|
||||
|
|
@ -714,7 +714,7 @@ mode. To get even more from your model, please visit `advanced
|
|||
throughput
|
||||
options <https://docs.openvino.ai/2023.3/openvino_docs_deployment_optimization_guide_tput_advanced.html>`__,
|
||||
`109-latency-tricks <109-latency-tricks-with-output.html-with-output.html>`__,
|
||||
`111-detection-quantization <111-yolov5-quantization-migration-with-output.html>`__, and
|
||||
`111-detection-quantization <111-detection-quantization-with-output.html>`__, and
|
||||
`118-optimize-preprocessing <118-optimize-preprocessing-with-output.html>`__.
|
||||
|
||||
Performance comparison
|
||||
|
|
@ -733,20 +733,20 @@ steps, just skip them.
|
|||
.. code:: ipython3
|
||||
|
||||
from matplotlib import pyplot as plt
|
||||
|
||||
|
||||
labels = ["PyTorch model", "OpenVINO IR model", "OpenVINO IR model + bigger batch", "OpenVINO IR model in throughput mode", "OpenVINO IR model in throughput mode on GPU",
|
||||
"OpenVINO IR model in throughput mode on AUTO", "OpenVINO IR model in cumulative throughput mode on AUTO"]
|
||||
|
||||
|
||||
fps = [pytorch_fps, ov_cpu_fps, ov_cpu_batch_fps, ov_cpu_through_fps, ov_gpu_fps, ov_auto_fps, ov_auto_cumulative_fps]
|
||||
|
||||
|
||||
bar_colors = colors[::10] / 255.0
|
||||
|
||||
|
||||
fig, ax = plt.subplots(figsize=(16, 8))
|
||||
ax.bar(labels, fps, color=bar_colors)
|
||||
|
||||
|
||||
ax.set_ylabel("Throughput [FPS]")
|
||||
ax.set_title("Performance difference")
|
||||
|
||||
|
||||
plt.xticks(rotation='vertical')
|
||||
plt.show()
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8e1932b29ece3712ada3af1ea8df832692a5416111c7caeb3c884cd3ee628110
|
||||
size 62463
|
||||
oid sha256:c054b14da27759eeaf1a1791c919f28eade5a92ea995b1f07690c71c435b2092
|
||||
size 62458
|
||||
|
|
|
|||
|
|
@ -1,14 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-throughput-tricks-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/109-throughput-tricks-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="109-throughput-tricks-with-output_15_0.jpg">109-throughput-tricks-with-output_15_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_18_0.jpg">109-throughput-tricks-with-output_18_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_21_0.jpg">109-throughput-tricks-with-output_21_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_25_0.jpg">109-throughput-tricks-with-output_25_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_29_0.jpg">109-throughput-tricks-with-output_29_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_31_0.jpg">109-throughput-tricks-with-output_31_0.jpg</a> 26-Jan-2024 01:05 162715
|
||||
<a href="109-throughput-tricks-with-output_34_0.png">109-throughput-tricks-with-output_34_0.png</a> 26-Jan-2024 01:05 62463
|
||||
<a href="109-throughput-tricks-with-output_5_0.jpg">109-throughput-tricks-with-output_5_0.jpg</a> 26-Jan-2024 01:05 155828
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -18,7 +18,7 @@ This notebook needs a quantized OpenVINO IR model and images from the
|
|||
`KiTS-19 <https://github.com/neheller/kits19>`__ dataset, converted to
|
||||
2D images. (To learn how the model is quantized, see the `Convert and
|
||||
Quantize a UNet Model and Show Live
|
||||
Inference <110-ct-segmentation-quantize-nncf-with-output.html>`__ tutorial.)
|
||||
Inference <110-ct-segmentation-quantize-nncf.ipynb>`__ tutorial.)
|
||||
|
||||
This notebook provides a pre-trained model, trained for 20 epochs with
|
||||
the full KiTS-19 frames dataset, which has an F1 score on the validation
|
||||
|
|
@ -80,14 +80,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:50:13.572016: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:50:13.606068: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:50:38.323593: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:50:38.357752: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:50:14.166493: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:50:38.922511: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Settings
|
||||
|
|
@ -140,7 +140,7 @@ Tool <https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html>`__
|
|||
is a command-line application that can be run in the notebook with
|
||||
``! benchmark_app`` or ``%sx benchmark_app`` commands.
|
||||
|
||||
**Note**: The ``benchmark_app`` tool is able to measure the
|
||||
**NOTE**: The ``benchmark_app`` tool is able to measure the
|
||||
performance of the OpenVINO Intermediate Representation (OpenVINO IR)
|
||||
models only. For more accurate performance, run ``benchmark_app`` in
|
||||
a terminal/command prompt after closing other applications. Run
|
||||
|
|
@ -198,7 +198,7 @@ is a command-line application that can be run in the notebook with
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 13.17 ms
|
||||
[ INFO ] Read model took 13.02 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input.1 (node: input.1) : f32 / [...] / [1,1,512,512]
|
||||
|
|
@ -216,7 +216,7 @@ is a command-line application that can be run in the notebook with
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 235.31 ms
|
||||
[ INFO ] Compile model took 232.64 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: pretrained_unet_kits19
|
||||
|
|
@ -252,21 +252,21 @@ is a command-line application that can be run in the notebook with
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 24.14 ms
|
||||
[ INFO ] First inference took 24.68 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 1355 iterations
|
||||
[ INFO ] Duration: 15007.81 ms
|
||||
[ INFO ] Count: 1366 iterations
|
||||
[ INFO ] Duration: 15004.33 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 10.84 ms
|
||||
[ INFO ] Average: 10.89 ms
|
||||
[ INFO ] Min: 10.58 ms
|
||||
[ INFO ] Max: 14.29 ms
|
||||
[ INFO ] Throughput: 90.29 FPS
|
||||
[ INFO ] Median: 10.75 ms
|
||||
[ INFO ] Average: 10.80 ms
|
||||
[ INFO ] Min: 10.53 ms
|
||||
[ INFO ] Max: 12.59 ms
|
||||
[ INFO ] Throughput: 91.04 FPS
|
||||
|
||||
|
||||
Download and Prepare Data
|
||||
|
|
@ -503,7 +503,7 @@ Create asynchronous inference queue and perform it
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Loaded model to Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO') in 0.24 seconds.
|
||||
Total time to infer all frames: 2.762s
|
||||
Time per frame: 0.040619s (24.619 FPS)
|
||||
Loaded model to Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO') in 0.23 seconds.
|
||||
Total time to infer all frames: 2.588s
|
||||
Time per frame: 0.038061s (26.274 FPS)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-scan-live-inference-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-scan-live-inference-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="110-ct-scan-live-inference-with-output_21_0.png">110-ct-scan-live-inference-with-output_21_0.png</a> 26-Jan-2024 01:05 48780
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -24,9 +24,9 @@ This third tutorial in the series shows how to:
|
|||
All notebooks in this series:
|
||||
|
||||
- `Data Preparation for 2D Segmentation of 3D Medical
|
||||
Data <data-preparation-ct-scan-with-output.html>`__
|
||||
Data <data-preparation-ct-scan.ipynb>`__
|
||||
- `Train a 2D-UNet Medical Imaging Model with PyTorch
|
||||
Lightning <pytorch-monai-training-with-output.html>`__
|
||||
Lightning <pytorch-monai-training.ipynb>`__
|
||||
- Convert and Quantize a Segmentation Model and Show Live Inference
|
||||
(this notebook)
|
||||
- `Live Inference and Benchmark CT-scan
|
||||
|
|
@ -39,7 +39,7 @@ This notebook needs a trained UNet model. We provide a pre-trained
|
|||
model, trained for 20 epochs with the full
|
||||
`Kits-19 <https://github.com/neheller/kits19>`__ frames dataset, which
|
||||
has an F1 score on the validation set of 0.9. The training code is
|
||||
available in `this notebook <pytorch-monai-training-with-output.html>`__.
|
||||
available in `this notebook <pytorch-monai-training.ipynb>`__.
|
||||
|
||||
NNCF for PyTorch models requires a C++ compiler. On Windows, install
|
||||
`Microsoft Visual Studio
|
||||
|
|
@ -182,14 +182,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:50:47.254337: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:50:47.287377: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:51:11.599112: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:51:11.634091: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:50:47.852256: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:51:12.223414: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -205,7 +205,7 @@ Settings
|
|||
By default, this notebook will download one CT scan from the KITS19
|
||||
dataset that will be used for quantization. To use the full dataset, set
|
||||
``BASEDIR`` to the path of the dataset, as prepared according to the
|
||||
`Data Preparation <data-preparation-ct-scan-with-output.html>`__ notebook.
|
||||
`Data Preparation <data-preparation-ct-scan.ipynb>`__ notebook.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -226,7 +226,7 @@ notebook is a
|
|||
`BasicUNet <https://docs.monai.io/en/stable/networks.html#basicunet>`__
|
||||
model from `MONAI <https://monai.io>`__. We provide a pre-trained
|
||||
checkpoint. To see how this model performs, check out the `training
|
||||
notebook <pytorch-monai-training-with-output.html>`__.
|
||||
notebook <pytorch-monai-training.ipynb>`__.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -304,7 +304,7 @@ Dataset
|
|||
The ``KitsDataset`` class in the next cell expects images and masks in
|
||||
the *``basedir``* directory, in a folder per patient. It is a simplified
|
||||
version of the Dataset class in the `training
|
||||
notebook <pytorch-monai-training-with-output.html>`__.
|
||||
notebook <pytorch-monai-training.ipynb>`__.
|
||||
|
||||
Images are loaded with MONAI’s
|
||||
`LoadImage <https://docs.monai.io/en/stable/transforms.html#loadimage>`__,
|
||||
|
|
@ -481,7 +481,7 @@ this notebook.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||
|
||||
|
||||
|
|
@ -489,7 +489,7 @@ this notebook.
|
|||
advanced algorithms for Neural Networks inference optimization in
|
||||
OpenVINO with minimal accuracy drop.
|
||||
|
||||
**Note**: NNCF Post-training Quantization is available in OpenVINO
|
||||
**NOTE**: NNCF Post-training Quantization is available in OpenVINO
|
||||
2023.0 release.
|
||||
|
||||
Create a quantized model from the pre-trained ``FP32`` model and the
|
||||
|
|
@ -532,13 +532,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -549,14 +551,15 @@ steps:
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -575,11 +578,11 @@ model and save it.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_low.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_high.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:179: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||
|
||||
|
||||
|
|
@ -692,7 +695,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 26.68 ms
|
||||
[ INFO ] Read model took 26.51 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x (node: x) : f32 / [...] / [?,?,?,?]
|
||||
|
|
@ -710,7 +713,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 86.15 ms
|
||||
[ INFO ] Compile model took 87.61 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -732,9 +735,9 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[Step 9/11] Creating infer requests and preparing input tensors
|
||||
[ ERROR ] Input x is dynamic. Provide data shapes!
|
||||
Traceback (most recent call last):
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
||||
data_queue = get_input_data(paths_to_input, app_inputs_info)
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
||||
raise Exception(f"Input {info.name} is dynamic. Provide data shapes!")
|
||||
Exception: Input x is dynamic. Provide data shapes!
|
||||
|
||||
|
|
@ -762,7 +765,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 13.10 ms
|
||||
[ INFO ] Read model took 13.17 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x.1 (node: x.1) : f32 / [...] / [1,1,512,512]
|
||||
|
|
@ -772,10 +775,6 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ INFO ] Model batch size: 1
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] x.1 (node: x.1) : f32 / [N,C,H,W] / [1,1,512,512]
|
||||
[ INFO ] Model outputs:
|
||||
[ INFO ] 571 (node: 571) : f32 / [...] / [1,1,512,512]
|
||||
|
|
@ -784,7 +783,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 188.53 ms
|
||||
[ INFO ] Compile model took 190.56 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: main_graph
|
||||
|
|
@ -812,21 +811,21 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 30.02 ms
|
||||
[ INFO ] First inference took 30.25 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 964 iterations
|
||||
[ INFO ] Duration: 15009.54 ms
|
||||
[ INFO ] Count: 973 iterations
|
||||
[ INFO ] Duration: 15006.05 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 15.33 ms
|
||||
[ INFO ] Average: 15.36 ms
|
||||
[ INFO ] Min: 14.97 ms
|
||||
[ INFO ] Max: 17.10 ms
|
||||
[ INFO ] Throughput: 64.23 FPS
|
||||
[ INFO ] Median: 15.16 ms
|
||||
[ INFO ] Average: 15.22 ms
|
||||
[ INFO ] Min: 14.87 ms
|
||||
[ INFO ] Max: 17.88 ms
|
||||
[ INFO ] Throughput: 64.84 FPS
|
||||
|
||||
|
||||
Visually Compare Inference Results
|
||||
|
|
@ -911,7 +910,7 @@ seed is displayed to enable reproducing specific runs of this cell.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Visualizing results with seed 1706219510
|
||||
Visualizing results with seed 1707515536
|
||||
|
||||
|
||||
|
||||
|
|
@ -994,8 +993,8 @@ performs inference, and displays the results on the frames loaded in
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Loaded model to CPU in 0.21 seconds.
|
||||
Total time for 68 frames: 2.62 seconds, fps:26.34
|
||||
Loaded model to CPU in 0.18 seconds.
|
||||
Total time for 68 frames: 2.68 seconds, fps:25.70
|
||||
|
||||
|
||||
References
|
||||
|
|
@ -1003,16 +1002,20 @@ References
|
|||
|
||||
|
||||
|
||||
**OpenVINO**
|
||||
**OpenVINO** - `NNCF
|
||||
Repository <https://github.com/openvinotoolkit/nncf/>`__ - `Neural
|
||||
Network Compression Framework for fast model
|
||||
inference <https://arxiv.org/abs/2002.08679>`__ - `OpenVINO API
|
||||
Tutorial <002-openvino-api-with-output.html>`__ - `OpenVINO
|
||||
PyPI (pip install
|
||||
openvino-dev) <https://pypi.org/project/openvino-dev/>`__
|
||||
|
||||
- `NNCF Repository <https://github.com/openvinotoolkit/nncf/>`__
|
||||
- `Neural Network Compression Framework for fast model inference <https://arxiv.org/abs/2002.08679>`__
|
||||
- `OpenVINO API Tutorial <002-openvino-api-with-output.html>`__
|
||||
- `OpenVINO PyPI (pip install openvino-dev) <https://pypi.org/project/openvino-dev/>`__
|
||||
|
||||
**Kits19 Data**
|
||||
|
||||
- `Kits19 Challenge Homepage <https://kits19.grand-challenge.org/>`__
|
||||
- `Kits19 GitHub Repository <https://github.com/neheller/kits19>`__
|
||||
- `The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes <https://arxiv.org/abs/1904.00445>`__
|
||||
- `The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge <https://www.sciencedirect.com/science/article/pii/S1361841520301857>`__
|
||||
**Kits19 Data** - `Kits19 Challenge
|
||||
Homepage <https://kits19.grand-challenge.org/>`__ - `Kits19 GitHub
|
||||
Repository <https://github.com/neheller/kits19>`__ - `The KiTS19
|
||||
Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT
|
||||
Semantic Segmentations, and Surgical
|
||||
Outcomes <https://arxiv.org/abs/1904.00445>`__ - `The state of the art
|
||||
in kidney and kidney tumor segmentation in contrast-enhanced CT imaging:
|
||||
Results of the KiTS19
|
||||
challenge <https://www.sciencedirect.com/science/article/pii/S1361841520301857>`__
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8b31829da55a95766d9f5f4e8b93cec1e5bac3f1147fd2a5cea2e8d7f7b12abc
|
||||
size 386985
|
||||
oid sha256:fa3a987453cb9be2a1e6e3c9d6ef943c29914c0a1f740e2eac55ebad8bc9ac4a
|
||||
size 387367
|
||||
|
|
|
|||
|
|
@ -1,9 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-segmentation-quantize-nncf-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/110-ct-segmentation-quantize-nncf-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="110-ct-segmentation-quantize-nncf-with-output_15_1.png">110-ct-segmentation-quantize-nncf-with-output_1..></a> 26-Jan-2024 01:05 158997
|
||||
<a href="110-ct-segmentation-quantize-nncf-with-output_37_1.png">110-ct-segmentation-quantize-nncf-with-output_3..></a> 26-Jan-2024 01:05 386985
|
||||
<a href="110-ct-segmentation-quantize-nncf-with-output_42_0.jpg">110-ct-segmentation-quantize-nncf-with-output_4..></a> 26-Jan-2024 01:05 73812
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -67,10 +67,6 @@ Preparations
|
|||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -182,7 +178,7 @@ Settings
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/112-pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/112-pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
||||
|
||||
|
||||
|
||||
|
|
@ -376,7 +372,8 @@ Create and load original uncompressed model
|
|||
|
||||
|
||||
|
||||
ResNet-50 from the `torchivision repository <https://github.com/pytorch/vision>`__ is pre-trained on
|
||||
ResNet-50 from the ```torchivision``
|
||||
repository <https://github.com/pytorch/vision>`__ is pre-trained on
|
||||
ImageNet with more prediction classes than Tiny ImageNet, so the model
|
||||
is adjusted by swapping the last FC layer to one with fewer output
|
||||
values.
|
||||
|
|
@ -474,47 +471,47 @@ I. Evaluate the loaded model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.249 (0.249) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [ 0/79] Time 0.283 (0.283) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [10/79] Time 0.223 (0.231) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [10/79] Time 0.242 (0.242) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [20/79] Time 0.230 (0.231) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [20/79] Time 0.237 (0.241) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [30/79] Time 0.229 (0.230) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [30/79] Time 0.238 (0.241) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [40/79] Time 0.245 (0.229) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [40/79] Time 0.244 (0.241) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [50/79] Time 0.223 (0.229) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [50/79] Time 0.270 (0.245) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [60/79] Time 0.227 (0.229) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
Test: [60/79] Time 0.246 (0.245) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [70/79] Time 0.227 (0.230) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
Test: [70/79] Time 0.263 (0.244) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 18.038
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 19.092
|
||||
Test accuracy of FP32 model: 60.740
|
||||
|
||||
|
||||
|
|
@ -557,14 +554,14 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:57:45.269741: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 22:57:45.300230: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 22:53:51.860179: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 22:53:51.891244: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 22:57:45.813067: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 22:53:52.407039: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -584,13 +581,15 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -611,13 +610,15 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
|
||||
</pre>
|
||||
|
||||
|
||||
|
||||
|
|
@ -634,48 +635,48 @@ Guide <https://docs.openvino.ai/2023.3/basic_quantization_flow.html>`__.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.430 (0.430) Acc@1 81.25 (81.25) Acc@5 89.84 (89.84)
|
||||
Test: [ 0/79] Time 0.435 (0.435) Acc@1 82.81 (82.81) Acc@5 92.19 (92.19)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [10/79] Time 0.392 (0.397) Acc@1 56.25 (66.26) Acc@5 85.16 (87.14)
|
||||
Test: [10/79] Time 0.391 (0.395) Acc@1 54.69 (66.34) Acc@5 85.94 (87.50)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [20/79] Time 0.393 (0.395) Acc@1 68.75 (63.80) Acc@5 84.38 (86.98)
|
||||
Test: [20/79] Time 0.389 (0.395) Acc@1 69.53 (63.91) Acc@5 84.38 (87.09)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [30/79] Time 0.393 (0.396) Acc@1 52.34 (62.15) Acc@5 75.78 (85.01)
|
||||
Test: [30/79] Time 0.388 (0.395) Acc@1 52.34 (62.22) Acc@5 75.78 (84.90)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [40/79] Time 0.392 (0.395) Acc@1 67.19 (60.75) Acc@5 89.84 (84.26)
|
||||
Test: [40/79] Time 0.392 (0.393) Acc@1 67.97 (60.75) Acc@5 89.84 (84.30)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [50/79] Time 0.392 (0.395) Acc@1 57.81 (60.68) Acc@5 88.28 (84.15)
|
||||
Test: [50/79] Time 0.398 (0.393) Acc@1 60.16 (60.72) Acc@5 88.28 (84.30)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [60/79] Time 0.393 (0.394) Acc@1 64.06 (60.32) Acc@5 85.94 (83.58)
|
||||
Test: [60/79] Time 0.390 (0.393) Acc@1 66.41 (60.27) Acc@5 86.72 (83.75)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [70/79] Time 0.392 (0.394) Acc@1 54.69 (60.17) Acc@5 78.12 (83.08)
|
||||
Test: [70/79] Time 0.388 (0.392) Acc@1 54.69 (60.06) Acc@5 80.47 (83.29)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
* Acc@1 60.690 Acc@5 83.740 Total time: 30.882
|
||||
Accuracy of initialized INT8 model: 60.690
|
||||
* Acc@1 60.570 Acc@5 83.950 Total time: 30.736
|
||||
Accuracy of initialized INT8 model: 60.570
|
||||
|
||||
|
||||
It should be noted that the inference time for the quantized PyTorch
|
||||
|
|
@ -724,20 +725,20 @@ For more information about model conversion, refer to this
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_low.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:342: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_high.item()
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
Tensor-likes are not close!
|
||||
|
||||
Mismatched elements: 25573 / 25600 (99.9%)
|
||||
Greatest absolute difference: 0.5424436330795288 at index (1, 149) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 42.99047422811133 at index (90, 158) (up to 1e-05 allowed)
|
||||
Mismatched elements: 25587 / 25600 (99.9%)
|
||||
Greatest absolute difference: 0.5083470344543457 at index (42, 14) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 79.27243410505909 at index (126, 158) (up to 1e-05 allowed)
|
||||
_check_trace(
|
||||
|
||||
|
||||
|
|
@ -778,47 +779,47 @@ Evaluate the FP32 and INT8 models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.184 (0.184) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [ 0/79] Time 0.185 (0.185) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [10/79] Time 0.139 (0.143) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [10/79] Time 0.136 (0.143) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [20/79] Time 0.139 (0.141) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [20/79] Time 0.141 (0.141) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [30/79] Time 0.139 (0.140) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [30/79] Time 0.140 (0.140) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [40/79] Time 0.135 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [40/79] Time 0.140 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [50/79] Time 0.139 (0.140) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [50/79] Time 0.139 (0.139) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [60/79] Time 0.139 (0.139) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
Test: [60/79] Time 0.138 (0.139) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [70/79] Time 0.138 (0.139) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
Test: [70/79] Time 0.136 (0.139) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 10.882
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 10.886
|
||||
Accuracy of FP32 IR model: 60.740
|
||||
|
||||
|
||||
|
|
@ -831,48 +832,48 @@ Evaluate the FP32 and INT8 models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.133 (0.133) Acc@1 81.25 (81.25) Acc@5 91.41 (91.41)
|
||||
Test: [ 0/79] Time 0.145 (0.145) Acc@1 82.03 (82.03) Acc@5 91.41 (91.41)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [10/79] Time 0.076 (0.084) Acc@1 54.69 (66.41) Acc@5 85.94 (87.64)
|
||||
Test: [10/79] Time 0.076 (0.083) Acc@1 55.47 (66.76) Acc@5 86.72 (87.36)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [20/79] Time 0.077 (0.080) Acc@1 71.09 (64.10) Acc@5 84.38 (87.05)
|
||||
Test: [20/79] Time 0.076 (0.080) Acc@1 70.31 (64.43) Acc@5 85.16 (87.02)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [30/79] Time 0.075 (0.079) Acc@1 52.34 (62.17) Acc@5 75.00 (84.98)
|
||||
Test: [30/79] Time 0.076 (0.079) Acc@1 53.12 (62.40) Acc@5 75.78 (84.93)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [40/79] Time 0.077 (0.079) Acc@1 67.19 (60.67) Acc@5 89.84 (84.22)
|
||||
Test: [40/79] Time 0.077 (0.078) Acc@1 67.19 (60.84) Acc@5 90.62 (84.20)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [50/79] Time 0.077 (0.078) Acc@1 60.16 (60.63) Acc@5 88.28 (84.22)
|
||||
Test: [50/79] Time 0.078 (0.078) Acc@1 59.38 (60.83) Acc@5 88.28 (84.15)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [60/79] Time 0.076 (0.078) Acc@1 65.62 (60.31) Acc@5 86.72 (83.67)
|
||||
Test: [60/79] Time 0.076 (0.078) Acc@1 64.84 (60.40) Acc@5 87.50 (83.63)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [70/79] Time 0.076 (0.077) Acc@1 53.12 (60.01) Acc@5 78.91 (83.20)
|
||||
Test: [70/79] Time 0.077 (0.078) Acc@1 53.12 (60.16) Acc@5 80.47 (83.14)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
* Acc@1 60.540 Acc@5 83.840 Total time: 6.060
|
||||
Accuracy of INT8 IR model: 60.540
|
||||
* Acc@1 60.680 Acc@5 83.770 Total time: 6.075
|
||||
Accuracy of INT8 IR model: 60.680
|
||||
|
||||
|
||||
IV. Compare performance of INT8 model and FP32 model in OpenVINO
|
||||
|
|
@ -941,25 +942,25 @@ throughput (frames per second) values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Throughput: 38.82 FPS
|
||||
[ INFO ] Throughput: 38.33 FPS
|
||||
Benchmark INT8 model (OpenVINO IR)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Throughput: 157.27 FPS
|
||||
[ INFO ] Throughput: 155.58 FPS
|
||||
Benchmark FP32 model (OpenVINO IR) synchronously
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Throughput: 40.19 FPS
|
||||
[ INFO ] Throughput: 39.95 FPS
|
||||
Benchmark INT8 model (OpenVINO IR) synchronously
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Throughput: 137.35 FPS
|
||||
[ INFO ] Throughput: 137.77 FPS
|
||||
|
||||
|
||||
Show device Information for reference:
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/113-image-classification-quantization-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/113-image-classification-quantization-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="113-image-classification-quantization-with-output_30_5.png">113-image-classification-quantization-with-outp..></a> 26-Jan-2024 01:05 14855
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -122,24 +122,20 @@ each frame of the video.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
... 12%, 32 KB, 1452 KB/s, 0 seconds passed
|
||||
... 12%, 32 KB, 1341 KB/s, 0 seconds passed
|
||||
... 25%, 64 KB, 1341 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 25%, 64 KB, 1385 KB/s, 0 seconds passed
|
||||
... 38%, 96 KB, 1632 KB/s, 0 seconds passed
|
||||
... 38%, 96 KB, 1143 KB/s, 0 seconds passed
|
||||
... 51%, 128 KB, 1326 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 51%, 128 KB, 1424 KB/s, 0 seconds passed
|
||||
... 64%, 160 KB, 1393 KB/s, 0 seconds passed
|
||||
... 77%, 192 KB, 1664 KB/s, 0 seconds passed
|
||||
... 89%, 224 KB, 1795 KB/s, 0 seconds passed
|
||||
... 100%, 248 KB, 1986 KB/s, 0 seconds passed
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
... 64%, 160 KB, 1323 KB/s, 0 seconds passed
|
||||
... 77%, 192 KB, 1444 KB/s, 0 seconds passed
|
||||
... 89%, 224 KB, 1678 KB/s, 0 seconds passed
|
||||
... 100%, 248 KB, 1859 KB/s, 0 seconds passed
|
||||
|
||||
|
||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
|
||||
|
|
@ -147,174 +143,177 @@ each frame of the video.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
... 0%, 32 KB, 1284 KB/s, 0 seconds passed
|
||||
... 1%, 64 KB, 1284 KB/s, 0 seconds passed
|
||||
... 2%, 96 KB, 1894 KB/s, 0 seconds passed
|
||||
... 0%, 32 KB, 1301 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 3%, 128 KB, 1699 KB/s, 0 seconds passed
|
||||
... 4%, 160 KB, 2092 KB/s, 0 seconds passed
|
||||
... 5%, 192 KB, 2274 KB/s, 0 seconds passed
|
||||
... 6%, 224 KB, 2623 KB/s, 0 seconds passed
|
||||
... 7%, 256 KB, 2328 KB/s, 0 seconds passed
|
||||
... 8%, 288 KB, 2599 KB/s, 0 seconds passed
|
||||
... 1%, 64 KB, 1298 KB/s, 0 seconds passed
|
||||
... 2%, 96 KB, 1907 KB/s, 0 seconds passed
|
||||
... 3%, 128 KB, 1722 KB/s, 0 seconds passed
|
||||
... 4%, 160 KB, 2133 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 9%, 320 KB, 2693 KB/s, 0 seconds passed
|
||||
... 9%, 352 KB, 2939 KB/s, 0 seconds passed
|
||||
... 10%, 384 KB, 2655 KB/s, 0 seconds passed
|
||||
... 11%, 416 KB, 2858 KB/s, 0 seconds passed
|
||||
... 12%, 448 KB, 2918 KB/s, 0 seconds passed
|
||||
... 13%, 480 KB, 3100 KB/s, 0 seconds passed
|
||||
... 5%, 192 KB, 2234 KB/s, 0 seconds passed
|
||||
... 6%, 224 KB, 2577 KB/s, 0 seconds passed
|
||||
... 7%, 256 KB, 2312 KB/s, 0 seconds passed
|
||||
... 8%, 288 KB, 2581 KB/s, 0 seconds passed
|
||||
... 9%, 320 KB, 2614 KB/s, 0 seconds passed
|
||||
... 9%, 352 KB, 2855 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 14%, 512 KB, 2858 KB/s, 0 seconds passed
|
||||
... 15%, 544 KB, 2904 KB/s, 0 seconds passed
|
||||
... 16%, 576 KB, 3065 KB/s, 0 seconds passed
|
||||
... 17%, 608 KB, 3219 KB/s, 0 seconds passed
|
||||
... 18%, 640 KB, 2998 KB/s, 0 seconds passed
|
||||
... 18%, 672 KB, 3133 KB/s, 0 seconds passed
|
||||
... 10%, 384 KB, 2608 KB/s, 0 seconds passed
|
||||
... 11%, 416 KB, 2811 KB/s, 0 seconds passed
|
||||
... 12%, 448 KB, 2817 KB/s, 0 seconds passed
|
||||
... 13%, 480 KB, 2999 KB/s, 0 seconds passed
|
||||
... 14%, 512 KB, 2785 KB/s, 0 seconds passed
|
||||
... 15%, 544 KB, 2949 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 19%, 704 KB, 3164 KB/s, 0 seconds passed
|
||||
... 20%, 736 KB, 3293 KB/s, 0 seconds passed
|
||||
... 21%, 768 KB, 3098 KB/s, 0 seconds passed
|
||||
... 22%, 800 KB, 3120 KB/s, 0 seconds passed
|
||||
... 23%, 832 KB, 3228 KB/s, 0 seconds passed
|
||||
... 16%, 576 KB, 2947 KB/s, 0 seconds passed
|
||||
... 17%, 608 KB, 3097 KB/s, 0 seconds passed
|
||||
... 18%, 640 KB, 2903 KB/s, 0 seconds passed
|
||||
... 18%, 672 KB, 3035 KB/s, 0 seconds passed
|
||||
... 19%, 704 KB, 3035 KB/s, 0 seconds passed
|
||||
... 20%, 736 KB, 3158 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 24%, 864 KB, 3062 KB/s, 0 seconds passed
|
||||
... 25%, 896 KB, 3170 KB/s, 0 seconds passed
|
||||
... 26%, 928 KB, 3274 KB/s, 0 seconds passed
|
||||
... 27%, 960 KB, 3291 KB/s, 0 seconds passed
|
||||
... 27%, 992 KB, 3388 KB/s, 0 seconds passed
|
||||
... 28%, 1024 KB, 3231 KB/s, 0 seconds passed
|
||||
... 29%, 1056 KB, 3322 KB/s, 0 seconds passed
|
||||
... 21%, 768 KB, 2990 KB/s, 0 seconds passed
|
||||
... 22%, 800 KB, 3107 KB/s, 0 seconds passed
|
||||
... 23%, 832 KB, 3100 KB/s, 0 seconds passed
|
||||
... 24%, 864 KB, 3209 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 30%, 1088 KB, 3331 KB/s, 0 seconds passed
|
||||
... 31%, 1120 KB, 3189 KB/s, 0 seconds passed
|
||||
... 32%, 1152 KB, 3275 KB/s, 0 seconds passed
|
||||
... 33%, 1184 KB, 3360 KB/s, 0 seconds passed
|
||||
... 34%, 1216 KB, 3370 KB/s, 0 seconds passed
|
||||
... 35%, 1248 KB, 3448 KB/s, 0 seconds passed
|
||||
... 25%, 896 KB, 3055 KB/s, 0 seconds passed
|
||||
... 26%, 928 KB, 3153 KB/s, 0 seconds passed
|
||||
... 27%, 960 KB, 3148 KB/s, 0 seconds passed
|
||||
... 27%, 992 KB, 3242 KB/s, 0 seconds passed
|
||||
... 28%, 1024 KB, 3104 KB/s, 0 seconds passed
|
||||
... 29%, 1056 KB, 3191 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 36%, 1280 KB, 3316 KB/s, 0 seconds passed
|
||||
... 36%, 1312 KB, 3390 KB/s, 0 seconds passed
|
||||
... 37%, 1344 KB, 3401 KB/s, 0 seconds passed
|
||||
... 38%, 1376 KB, 3472 KB/s, 0 seconds passed
|
||||
... 39%, 1408 KB, 3349 KB/s, 0 seconds passed
|
||||
... 40%, 1440 KB, 3418 KB/s, 0 seconds passed
|
||||
... 30%, 1088 KB, 3186 KB/s, 0 seconds passed
|
||||
... 31%, 1120 KB, 3271 KB/s, 0 seconds passed
|
||||
... 32%, 1152 KB, 3144 KB/s, 0 seconds passed
|
||||
... 33%, 1184 KB, 3225 KB/s, 0 seconds passed
|
||||
... 34%, 1216 KB, 3217 KB/s, 0 seconds passed
|
||||
... 35%, 1248 KB, 3293 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 41%, 1472 KB, 3426 KB/s, 0 seconds passed
|
||||
... 42%, 1504 KB, 3488 KB/s, 0 seconds passed
|
||||
... 43%, 1536 KB, 3372 KB/s, 0 seconds passed
|
||||
... 44%, 1568 KB, 3437 KB/s, 0 seconds passed
|
||||
... 45%, 1600 KB, 3446 KB/s, 0 seconds passed
|
||||
... 45%, 1632 KB, 3505 KB/s, 0 seconds passed
|
||||
... 36%, 1280 KB, 3177 KB/s, 0 seconds passed
|
||||
... 36%, 1312 KB, 3250 KB/s, 0 seconds passed
|
||||
... 37%, 1344 KB, 3243 KB/s, 0 seconds passed
|
||||
... 38%, 1376 KB, 3311 KB/s, 0 seconds passed
|
||||
... 39%, 1408 KB, 3204 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 46%, 1664 KB, 3400 KB/s, 0 seconds passed
|
||||
... 47%, 1696 KB, 3457 KB/s, 0 seconds passed
|
||||
... 48%, 1728 KB, 3468 KB/s, 0 seconds passed
|
||||
... 49%, 1760 KB, 3520 KB/s, 0 seconds passed
|
||||
... 50%, 1792 KB, 3417 KB/s, 0 seconds passed
|
||||
... 51%, 1824 KB, 3472 KB/s, 0 seconds passed
|
||||
... 40%, 1440 KB, 3198 KB/s, 0 seconds passed
|
||||
... 41%, 1472 KB, 3260 KB/s, 0 seconds passed
|
||||
... 42%, 1504 KB, 3324 KB/s, 0 seconds passed
|
||||
... 43%, 1536 KB, 3225 KB/s, 0 seconds passed
|
||||
... 44%, 1568 KB, 3218 KB/s, 0 seconds passed
|
||||
... 45%, 1600 KB, 3279 KB/s, 0 seconds passed
|
||||
... 45%, 1632 KB, 3339 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 52%, 1856 KB, 3478 KB/s, 0 seconds passed
|
||||
... 53%, 1888 KB, 3383 KB/s, 0 seconds passed
|
||||
... 54%, 1920 KB, 3437 KB/s, 0 seconds passed
|
||||
... 54%, 1952 KB, 3487 KB/s, 0 seconds passed
|
||||
... 55%, 1984 KB, 3495 KB/s, 0 seconds passed
|
||||
... 56%, 2016 KB, 3544 KB/s, 0 seconds passed
|
||||
... 46%, 1664 KB, 3246 KB/s, 0 seconds passed
|
||||
... 47%, 1696 KB, 3234 KB/s, 0 seconds passed
|
||||
... 48%, 1728 KB, 3291 KB/s, 0 seconds passed
|
||||
... 49%, 1760 KB, 3349 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 57%, 2048 KB, 3451 KB/s, 0 seconds passed
|
||||
... 58%, 2080 KB, 3500 KB/s, 0 seconds passed
|
||||
... 59%, 2112 KB, 3506 KB/s, 0 seconds passed
|
||||
... 60%, 2144 KB, 3554 KB/s, 0 seconds passed
|
||||
... 61%, 2176 KB, 3466 KB/s, 0 seconds passed
|
||||
... 50%, 1792 KB, 3263 KB/s, 0 seconds passed
|
||||
... 51%, 1824 KB, 3249 KB/s, 0 seconds passed
|
||||
... 52%, 1856 KB, 3303 KB/s, 0 seconds passed
|
||||
... 53%, 1888 KB, 3357 KB/s, 0 seconds passed
|
||||
... 54%, 1920 KB, 3280 KB/s, 0 seconds passed
|
||||
... 54%, 1952 KB, 3328 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 62%, 2208 KB, 3510 KB/s, 0 seconds passed
|
||||
... 63%, 2240 KB, 3513 KB/s, 0 seconds passed
|
||||
... 64%, 2272 KB, 3561 KB/s, 0 seconds passed
|
||||
... 64%, 2304 KB, 3478 KB/s, 0 seconds passed
|
||||
... 65%, 2336 KB, 3518 KB/s, 0 seconds passed
|
||||
... 66%, 2368 KB, 3521 KB/s, 0 seconds passed
|
||||
... 55%, 1984 KB, 3314 KB/s, 0 seconds passed
|
||||
... 56%, 2016 KB, 3365 KB/s, 0 seconds passed
|
||||
... 57%, 2048 KB, 3294 KB/s, 0 seconds passed
|
||||
... 58%, 2080 KB, 3337 KB/s, 0 seconds passed
|
||||
... 59%, 2112 KB, 3325 KB/s, 0 seconds passed
|
||||
... 60%, 2144 KB, 3372 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 67%, 2400 KB, 3444 KB/s, 0 seconds passed
|
||||
... 68%, 2432 KB, 3486 KB/s, 0 seconds passed
|
||||
... 69%, 2464 KB, 3495 KB/s, 0 seconds passed
|
||||
... 70%, 2496 KB, 3528 KB/s, 0 seconds passed
|
||||
... 61%, 2176 KB, 3306 KB/s, 0 seconds passed
|
||||
... 62%, 2208 KB, 3345 KB/s, 0 seconds passed
|
||||
... 63%, 2240 KB, 3335 KB/s, 0 seconds passed
|
||||
... 64%, 2272 KB, 3379 KB/s, 0 seconds passed
|
||||
... 64%, 2304 KB, 3316 KB/s, 0 seconds passed
|
||||
... 65%, 2336 KB, 3351 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 71%, 2528 KB, 3454 KB/s, 0 seconds passed
|
||||
... 72%, 2560 KB, 3496 KB/s, 0 seconds passed
|
||||
... 73%, 2592 KB, 3505 KB/s, 0 seconds passed
|
||||
... 73%, 2624 KB, 3515 KB/s, 0 seconds passed
|
||||
... 74%, 2656 KB, 3463 KB/s, 0 seconds passed
|
||||
... 75%, 2688 KB, 3502 KB/s, 0 seconds passed
|
||||
... 76%, 2720 KB, 3514 KB/s, 0 seconds passed
|
||||
... 77%, 2752 KB, 3523 KB/s, 0 seconds passed
|
||||
... 66%, 2368 KB, 3343 KB/s, 0 seconds passed
|
||||
... 67%, 2400 KB, 3385 KB/s, 0 seconds passed
|
||||
... 68%, 2432 KB, 3322 KB/s, 0 seconds passed
|
||||
... 69%, 2464 KB, 3312 KB/s, 0 seconds passed
|
||||
... 70%, 2496 KB, 3349 KB/s, 0 seconds passed
|
||||
... 71%, 2528 KB, 3390 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 78%, 2784 KB, 3474 KB/s, 0 seconds passed
|
||||
... 79%, 2816 KB, 3510 KB/s, 0 seconds passed
|
||||
... 80%, 2848 KB, 3522 KB/s, 0 seconds passed
|
||||
... 81%, 2880 KB, 3530 KB/s, 0 seconds passed
|
||||
... 72%, 2560 KB, 3331 KB/s, 0 seconds passed
|
||||
... 73%, 2592 KB, 3322 KB/s, 0 seconds passed
|
||||
... 73%, 2624 KB, 3356 KB/s, 0 seconds passed
|
||||
... 74%, 2656 KB, 3395 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 82%, 2912 KB, 3482 KB/s, 0 seconds passed
|
||||
... 82%, 2944 KB, 3518 KB/s, 0 seconds passed
|
||||
... 83%, 2976 KB, 3529 KB/s, 0 seconds passed
|
||||
... 84%, 3008 KB, 3535 KB/s, 0 seconds passed
|
||||
... 85%, 3040 KB, 3489 KB/s, 0 seconds passed
|
||||
... 86%, 3072 KB, 3523 KB/s, 0 seconds passed
|
||||
... 87%, 3104 KB, 3532 KB/s, 0 seconds passed
|
||||
... 75%, 2688 KB, 3338 KB/s, 0 seconds passed
|
||||
... 76%, 2720 KB, 3328 KB/s, 0 seconds passed
|
||||
... 77%, 2752 KB, 3363 KB/s, 0 seconds passed
|
||||
... 78%, 2784 KB, 3399 KB/s, 0 seconds passed
|
||||
... 79%, 2816 KB, 3344 KB/s, 0 seconds passed
|
||||
... 80%, 2848 KB, 3335 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 88%, 3136 KB, 3540 KB/s, 0 seconds passed
|
||||
... 89%, 3168 KB, 3498 KB/s, 0 seconds passed
|
||||
... 90%, 3200 KB, 3530 KB/s, 0 seconds passed
|
||||
... 91%, 3232 KB, 3539 KB/s, 0 seconds passed
|
||||
... 91%, 3264 KB, 3544 KB/s, 0 seconds passed
|
||||
... 81%, 2880 KB, 3369 KB/s, 0 seconds passed
|
||||
... 82%, 2912 KB, 3321 KB/s, 0 seconds passed
|
||||
... 82%, 2944 KB, 3352 KB/s, 0 seconds passed
|
||||
... 83%, 2976 KB, 3342 KB/s, 0 seconds passed
|
||||
... 84%, 3008 KB, 3374 KB/s, 0 seconds passed
|
||||
... 85%, 3040 KB, 3407 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 92%, 3296 KB, 3504 KB/s, 0 seconds passed
|
||||
... 93%, 3328 KB, 3536 KB/s, 0 seconds passed
|
||||
... 94%, 3360 KB, 3544 KB/s, 0 seconds passed
|
||||
... 95%, 3392 KB, 3550 KB/s, 0 seconds passed
|
||||
... 96%, 3424 KB, 3509 KB/s, 0 seconds passed
|
||||
... 97%, 3456 KB, 3540 KB/s, 0 seconds passed
|
||||
... 98%, 3488 KB, 3548 KB/s, 0 seconds passed
|
||||
... 86%, 3072 KB, 3360 KB/s, 0 seconds passed
|
||||
... 87%, 3104 KB, 3351 KB/s, 0 seconds passed
|
||||
... 88%, 3136 KB, 3380 KB/s, 0 seconds passed
|
||||
... 89%, 3168 KB, 3411 KB/s, 0 seconds passed
|
||||
... 90%, 3200 KB, 3364 KB/s, 0 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 99%, 3520 KB, 3560 KB/s, 0 seconds passed
|
||||
... 100%, 3549 KB, 3521 KB/s, 1 seconds passed
|
||||
... 91%, 3232 KB, 3357 KB/s, 0 seconds passed
|
||||
... 91%, 3264 KB, 3384 KB/s, 0 seconds passed
|
||||
... 92%, 3296 KB, 3342 KB/s, 0 seconds passed
|
||||
... 93%, 3328 KB, 3368 KB/s, 0 seconds passed
|
||||
... 94%, 3360 KB, 3358 KB/s, 1 seconds passed
|
||||
... 95%, 3392 KB, 3388 KB/s, 1 seconds passed
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
... 96%, 3424 KB, 3346 KB/s, 1 seconds passed
|
||||
... 97%, 3456 KB, 3372 KB/s, 1 seconds passed
|
||||
... 98%, 3488 KB, 3363 KB/s, 1 seconds passed
|
||||
... 99%, 3520 KB, 3392 KB/s, 1 seconds passed
|
||||
... 100%, 3549 KB, 3419 KB/s, 1 seconds passed
|
||||
|
||||
|
||||
|
||||
|
|
@ -512,7 +511,7 @@ Test performance in Sync Mode
|
|||
.. parsed-literal::
|
||||
|
||||
Source ended
|
||||
average throuput in sync mode: 43.51 fps
|
||||
average throuput in sync mode: 47.04 fps
|
||||
|
||||
|
||||
Async Mode
|
||||
|
|
@ -651,7 +650,7 @@ Test the performance in Async Mode
|
|||
.. parsed-literal::
|
||||
|
||||
Source ended
|
||||
average throuput in async mode: 74.01 fps
|
||||
average throuput in async mode: 74.61 fps
|
||||
|
||||
|
||||
Compare the performance
|
||||
|
|
@ -794,5 +793,5 @@ Test the performance with ``AsyncInferQueue``
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
average throughput in async mode with async infer queue: 110.03 fps
|
||||
average throughput in async mode with async infer queue: 113.01 fps
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a6ef5964af038dfaf0c794dfc9a36258293ea7a6f8318f5f84490e8953a10479
|
||||
size 30402
|
||||
oid sha256:8c51610ffd478d0bc5358648e0475259dc8eb04d090c6f55cb860d2e9b8a8cb6
|
||||
size 30356
|
||||
|
|
|
|||
|
|
@ -1,10 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/115-async-api-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/115-async-api-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="115-async-api-with-output_15_0.png">115-async-api-with-output_15_0.png</a> 26-Jan-2024 01:05 4307
|
||||
<a href="115-async-api-with-output_19_0.png">115-async-api-with-output_19_0.png</a> 26-Jan-2024 01:05 4307
|
||||
<a href="115-async-api-with-output_21_0.png">115-async-api-with-output_21_0.png</a> 26-Jan-2024 01:05 30402
|
||||
<a href="115-async-api-with-output_27_0.png">115-async-api-with-output_27_0.png</a> 26-Jan-2024 01:05 4307
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -85,14 +85,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:06:01.802535: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 23:06:01.837209: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 23:02:05.779349: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 23:02:05.814537: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:06:02.398511: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:02:06.378496: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Download, quantize and sparsify the model, using Hugging Face Optimum API
|
||||
|
|
@ -228,7 +228,7 @@ as an example. It is recommended to tune based on your applications.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Read model took 60.22 ms
|
||||
[ INFO ] Read model took 62.38 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
||||
|
|
@ -239,7 +239,7 @@ as an example. It is recommended to tune based on your applications.
|
|||
[Step 5/11] Resizing model to match image sizes and given batch
|
||||
[ INFO ] Model batch size: 1
|
||||
[ INFO ] Reshaping model: 'input_ids': [1,64], 'attention_mask': [1,64], 'token_type_ids': [1,64]
|
||||
[ INFO ] Reshape model took 23.06 ms
|
||||
[ INFO ] Reshape model took 23.14 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
||||
|
|
@ -252,7 +252,7 @@ as an example. It is recommended to tune based on your applications.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 1064.23 ms
|
||||
[ INFO ] Compile model took 1107.64 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: torch_jit
|
||||
|
|
@ -284,21 +284,21 @@ as an example. It is recommended to tune based on your applications.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 27.61 ms
|
||||
[ INFO ] First inference took 30.14 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 8900 iterations
|
||||
[ INFO ] Duration: 60039.72 ms
|
||||
[ INFO ] Count: 8852 iterations
|
||||
[ INFO ] Duration: 60038.32 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 26.68 ms
|
||||
[ INFO ] Average: 26.74 ms
|
||||
[ INFO ] Min: 25.09 ms
|
||||
[ INFO ] Max: 39.62 ms
|
||||
[ INFO ] Throughput: 148.24 FPS
|
||||
[ INFO ] Median: 26.79 ms
|
||||
[ INFO ] Average: 26.86 ms
|
||||
[ INFO ] Min: 24.76 ms
|
||||
[ INFO ] Max: 42.20 ms
|
||||
[ INFO ] Throughput: 147.44 FPS
|
||||
|
||||
|
||||
Benchmark quantized sparse inference performance
|
||||
|
|
@ -363,7 +363,7 @@ for which a layer will be enabled.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Read model took 67.79 ms
|
||||
[ INFO ] Read model took 71.12 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
|
||||
|
|
@ -378,7 +378,7 @@ for which a layer will be enabled.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Reshape model took 23.92 ms
|
||||
[ INFO ] Reshape model took 23.54 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,64]
|
||||
|
|
@ -392,9 +392,9 @@ for which a layer will be enabled.
|
|||
Wrong value for property key CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE. Expected only float numbers
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 408, in main
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 408, in main
|
||||
compiled_model = benchmark.core.compile_model(model, benchmark.device, device_config)
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/runtime/ie_api.py", line 547, in compile_model
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/runtime/ie_api.py", line 547, in compile_model
|
||||
super().compile_model(model, device_name, {} if config is None else config),
|
||||
RuntimeError: Exception from src/inference/src/core.cpp:99:
|
||||
[ GENERAL_ERROR ] Exception from src/plugins/intel_cpu/src/config.cpp:158:
|
||||
|
|
|
|||
|
|
@ -269,7 +269,7 @@ Check whether the OVMS container is running normally:
|
|||
|
||||
The required Model Server parameters are listed below. For additional
|
||||
configuration options, see the `Model Server Parameters
|
||||
section <https://docs.openvino.ai/2023.2/ovms_docs_parameters.html>`__.
|
||||
section <https://docs.openvino.ai/2023.3/ovms_docs_parameters.html>`__.
|
||||
|
||||
.. raw:: html
|
||||
|
||||
|
|
@ -928,6 +928,6 @@ References
|
|||
|
||||
|
||||
1. `OpenVINO™ Model Server
|
||||
documentation <https://docs.openvino.ai/2023.0/ovms_what_is_openvino_model_server.html>`__
|
||||
documentation <https://docs.openvino.ai/2023.3/ovms_what_is_openvino_model_server.html>`__
|
||||
2. `OpenVINO™ Model Server GitHub
|
||||
repository <https://github.com/openvinotoolkit/model_server/>`__
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/117-model-server-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/117-model-server-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="117-model-server-with-output_23_2.png">117-model-server-with-output_23_2.png</a> 26-Jan-2024 01:04 305482
|
||||
<a href="117-model-server-with-output_28_1.png">117-model-server-with-output_28_1.png</a> 26-Jan-2024 01:04 458294
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -106,14 +106,14 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:07:18.341652: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 23:07:18.375926: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 23:03:22.538807: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 23:03:22.573455: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:07:18.888977: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:03:23.087260: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Setup image and device
|
||||
|
|
@ -194,12 +194,12 @@ and save it to the disk.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:07:22.553946: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-01-25 23:07:22.553985: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-01-25 23:07:22.553989: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-01-25 23:07:22.554129: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-01-25 23:07:22.554144: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-01-25 23:07:22.554147: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-02-09 23:03:26.685646: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-02-09 23:03:26.685683: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-02-09 23:03:26.685688: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-02-09 23:03:26.685821: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-02-09 23:03:26.685836: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-02-09 23:03:26.685841: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -360,7 +360,7 @@ for mean/scale normalization.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7f5ce40abdb0>
|
||||
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7ff8b271c1b0>
|
||||
|
||||
|
||||
|
||||
|
|
@ -391,7 +391,7 @@ may be specified is input data
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7f5ce40abb70>
|
||||
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7ff7b1f9dd70>
|
||||
|
||||
|
||||
|
||||
|
|
@ -432,7 +432,7 @@ then such conversion will be added explicitly.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7f5ce40abe70>
|
||||
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7ff7b1f9d230>
|
||||
|
||||
|
||||
|
||||
|
|
@ -651,10 +651,10 @@ Compare performance
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0149 seconds per image, FPS: 67.09
|
||||
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0153 seconds per image, FPS: 65.53
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0182 seconds per image, FPS: 54.89
|
||||
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0183 seconds per image, FPS: 54.77
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/118-optimize-preprocessing-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/118-optimize-preprocessing-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="118-optimize-preprocessing-with-output_14_1.png">118-optimize-preprocessing-with-output_14_1.png</a> 26-Jan-2024 01:04 387941
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -106,7 +106,7 @@ Download TFLite model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/119-tflite-to-openvino/model/efficientnet_lite0_fp32_2.tflite')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/119-tflite-to-openvino/model/efficientnet_lite0_fp32_2.tflite')
|
||||
|
||||
|
||||
|
||||
|
|
@ -281,7 +281,11 @@ GPU.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 10.06 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Read model took 21.35 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] images (node: images) : f32 / [...] / [1,224,224,3]
|
||||
|
|
@ -299,9 +303,13 @@ GPU.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 138.70 ms
|
||||
[ INFO ] Compile model took 147.38 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] NETWORK_NAME: TensorFlow_Lite_Frontend_IR
|
||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||
[ INFO ] NUM_STREAMS: 6
|
||||
|
|
@ -323,19 +331,19 @@ GPU.
|
|||
[ INFO ] Fill input 'images' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 7.39 ms
|
||||
[ INFO ] First inference took 7.28 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 17544 iterations
|
||||
[ INFO ] Duration: 15006.83 ms
|
||||
[ INFO ] Count: 17502 iterations
|
||||
[ INFO ] Duration: 15007.36 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 5.01 ms
|
||||
[ INFO ] Average: 5.00 ms
|
||||
[ INFO ] Min: 2.90 ms
|
||||
[ INFO ] Max: 14.27 ms
|
||||
[ INFO ] Throughput: 1169.07 FPS
|
||||
[ INFO ] Median: 5.02 ms
|
||||
[ INFO ] Average: 5.02 ms
|
||||
[ INFO ] Min: 3.02 ms
|
||||
[ INFO ] Max: 13.94 ms
|
||||
[ INFO ] Throughput: 1166.23 FPS
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/119-tflite-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/119-tflite-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="119-tflite-to-openvino-with-output_16_1.jpg">119-tflite-to-openvino-with-output_16_1.jpg</a> 26-Jan-2024 01:05 68170
|
||||
<a href="119-tflite-to-openvino-with-output_16_1.png">119-tflite-to-openvino-with-output_16_1.png</a> 26-Jan-2024 01:05 621006
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -24,23 +24,32 @@ After creating the OpenVINO IR, load the model in `OpenVINO
|
|||
Runtime <https://docs.openvino.ai/nightly/openvino_docs_OV_UG_OV_Runtime_User_Guide.html>`__
|
||||
and do inference with a sample image.
|
||||
|
||||
**Table of contents:**
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Imports <#imports>`__
|
||||
- `Settings <#settings>`__
|
||||
- `Download Model from TensorFlow Hub <#download-model-from-tensorflow-hub>`__
|
||||
- `Convert Model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||
- `Test Inference on the Converted Model <#test-inference-on-the-converted-model>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Load the Model <#load-the-model>`__
|
||||
- `Get Model Information <#get-model-information>`__
|
||||
- `Get an Image for Test Inference <#get-an-image-for-test-inference>`__
|
||||
- `Perform Inference <#perform-inference>`__
|
||||
- `Inference Result Visualization <#inference-result-visualization>`__
|
||||
- `Next Steps <#next-steps>`__
|
||||
- `Async inference pipeline <#async-inference-pipeline>`__
|
||||
- `Integration preprocessing to model <#integration-preprocessing-to-model>`__
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Imports <#imports>`__
|
||||
- `Settings <#settings>`__
|
||||
- `Download Model from TensorFlow
|
||||
Hub <#download-model-from-tensorflow-hub>`__
|
||||
- `Convert Model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||
- `Test Inference on the Converted
|
||||
Model <#test-inference-on-the-converted-model>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
|
||||
- `Load the Model <#load-the-model>`__
|
||||
- `Get Model Information <#get-model-information>`__
|
||||
- `Get an Image for Test
|
||||
Inference <#get-an-image-for-test-inference>`__
|
||||
- `Perform Inference <#perform-inference>`__
|
||||
- `Inference Result
|
||||
Visualization <#inference-result-visualization>`__
|
||||
|
||||
- `Next Steps <#next-steps>`__
|
||||
|
||||
- `Async inference pipeline <#async-inference-pipeline>`__
|
||||
- `Integration preprocessing to
|
||||
model <#integration-preprocessing-to-model>`__
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
|
@ -355,7 +364,7 @@ Read the image, resize and convert it to the input shape of the network:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f5f2fe7a310>
|
||||
<matplotlib.image.AxesImage at 0x7f39e4396eb0>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ed008919b91d0e231d8872fb4fae77bdfc3a513547568b9448b69de0065625c5
|
||||
size 393453
|
||||
oid sha256:8e7e8a03ae8a07e661d3a25306fe3dca53093416ecf98222fe478edaf13d07df
|
||||
size 393160
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-instance-segmentation-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-instance-segmentation-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="120-tensorflow-instance-segmentation-to-openvino-with-output_25_1.png">120-tensorflow-instance-segmentation-to-openvin..></a> 26-Jan-2024 01:05 395346
|
||||
<a href="120-tensorflow-instance-segmentation-to-openvino-with-output_39_0.png">120-tensorflow-instance-segmentation-to-openvin..></a> 26-Jan-2024 01:05 393453
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -157,7 +157,7 @@ from TensorFlow Hub:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/model/tf/faster_rcnn_resnet50_v1_640x640.tar.gz')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/model/tf/faster_rcnn_resnet50_v1_640x640.tar.gz')
|
||||
|
||||
|
||||
|
||||
|
|
@ -343,7 +343,7 @@ Load and save an image:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_bike.jpg')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_bike.jpg')
|
||||
|
||||
|
||||
|
||||
|
|
@ -371,7 +371,7 @@ Read the image, resize and convert it to the input shape of the network:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f8f0a2c3d00>
|
||||
<matplotlib.image.AxesImage at 0x7f62f45088e0>
|
||||
|
||||
|
||||
|
||||
|
|
@ -635,7 +635,7 @@ Zoo <https://github.com/openvinotoolkit/open_model_zoo/>`__:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_91cl.txt')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/120-tensorflow-object-detection-to-openvino/data/coco_91cl.txt')
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a5b7ad5bc51c70dfab0037ca885aeb4e613947df3fd7fdff8d055c8d64aefd4a
|
||||
size 391815
|
||||
oid sha256:69ba7f4bdc1e3d85a98b34863980a347ab3bdf2aedb7d9d78a2f8d43484127b6
|
||||
size 391578
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-object-detection-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/120-tensorflow-object-detection-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="120-tensorflow-object-detection-to-openvino-with-output_25_1.png">120-tensorflow-object-detection-to-openvino-wit..></a> 26-Jan-2024 01:05 395346
|
||||
<a href="120-tensorflow-object-detection-to-openvino-with-output_38_0.png">120-tensorflow-object-detection-to-openvino-wit..></a> 26-Jan-2024 01:05 391815
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -35,7 +35,7 @@ Table of contents:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (23.3.2)
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -196,19 +196,19 @@ NLP model from Hugging Face and export it in ONNX format:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:11:32.267646: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 23:11:32.303561: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 23:07:37.212192: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 23:07:37.247733: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:11:32.956586: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:07:37.883428: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||
|
||||
|
||||
|
|
@ -699,12 +699,12 @@ frameworks conversion guides.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:11:51.061555: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-01-25 23:11:51.061587: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-01-25 23:11:51.061592: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-01-25 23:11:51.061773: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-01-25 23:11:51.061789: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-01-25 23:11:51.061793: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-02-09 23:07:56.008045: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-02-09 23:07:56.008076: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-02-09 23:07:56.008081: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-02-09 23:07:56.008267: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-02-09 23:07:56.008284: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-02-09 23:07:56.008287: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
Migration from Legacy conversion API
|
||||
|
|
|
|||
|
|
@ -118,7 +118,7 @@ documentation.
|
|||
conversion into IR. The legacy Frontend is Python
|
||||
based and is available for TensorFlow*, ONNX*, MXNet*,
|
||||
Caffe*, and Kaldi* models.
|
||||
--input_model INPUT_MODEL, -w INPUT_MODEL, -m INPUT_MODEL
|
||||
--input_model INPUT_MODEL, -m INPUT_MODEL, -w INPUT_MODEL
|
||||
Tensorflow*: a file with a pre-trained model (binary
|
||||
or text .pb file after freezing). Caffe*: a model
|
||||
proto file with model weights.
|
||||
|
|
@ -256,11 +256,7 @@ documentation.
|
|||
Paths or a comma-separated list of paths to libraries
|
||||
(.so or .dll) with extensions. For the legacy MO path
|
||||
(if `--use_legacy_frontend` is used), a directory or a
|
||||
comma
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
-separated list of directories with extensions
|
||||
comma-separated list of directories with extensions
|
||||
are supported. To disable all extensions including
|
||||
those that are placed at the default location, pass an
|
||||
empty string.
|
||||
|
|
@ -744,19 +740,19 @@ NLP model from Hugging Face and export it in ONNX format:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:12:13.362144: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 23:12:13.397165: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 23:08:18.586507: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 23:08:18.621399: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:12:14.029251: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:08:19.256172: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:246: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||
|
||||
|
||||
|
|
@ -1027,8 +1023,8 @@ To convert a model to OpenVINO IR, use the following command:
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1129,8 +1125,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1154,8 +1150,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1211,8 +1207,8 @@ sequence length dimension for inputs:
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1262,8 +1258,8 @@ dimension:
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1336,8 +1332,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1361,8 +1357,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1438,13 +1434,9 @@ Resnet50 model that was exported to the ONNX format:
|
|||
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.0/openvino_2_0_transition_guide.html
|
||||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1495,8 +1487,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1520,8 +1512,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1578,8 +1570,8 @@ that the preprocessing takes negligible time for inference.
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1603,8 +1595,8 @@ that the preprocessing takes negligible time for inference.
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1658,8 +1650,8 @@ the color channels before inference.
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1709,8 +1701,8 @@ models, this decrease is negligible.
|
|||
.. parsed-literal::
|
||||
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/121-convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
|
|||
|
|
@ -303,6 +303,7 @@ value 25 to speed up the execution.
|
|||
INFO:nncf:Validation of initial model was started
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
INFO:nncf:Elapsed Time: 00:00:00
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
Convert Detectron2 Models to OpenVINO™
|
||||
=========================================
|
||||
=======================================
|
||||
|
||||
`Detectron2 <https://github.com/facebookresearch/detectron2>`__ is
|
||||
Facebook AI Research’s library that provides state-of-the-art detection
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b9a59f5b6bc77d33212bcadd69faacc6c286d6eac7d7ed88ce28ccefe227f010
|
||||
size 59043
|
||||
oid sha256:97f44896da4cb3dee863ed6869c16015702bf26050467e2ff84110920318665b
|
||||
size 58781
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2db4d40d09643561d54ed996c0e42df721272e7bc17220c20240a81142df67eb
|
||||
size 508506
|
||||
oid sha256:4398d08f29dcb59dd57f4cd453308897ac75f6f8b860558fb21cfdbcd479b6e3
|
||||
size 509675
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:44dbb04db8f71ea61f1896b341ac6811246dd60d600942296da3918b7f82c715
|
||||
size 55966
|
||||
oid sha256:ae3a4df87c84baa406e0d2ed491e25f28bd9d8e6e9fe649bd74c0b7eae35ac07
|
||||
size 55630
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b9e00a023f05b67dd28e92c59ea377d5c9dc85f2c4f183d44e03227b25384749
|
||||
size 457794
|
||||
oid sha256:c832d80fe116434db304523dbc29d6e292e7a5733b70ae709e8eb2a0674506e6
|
||||
size 457756
|
||||
|
|
|
|||
|
|
@ -1,12 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/123-detectron2-to-openvino-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/123-detectron2-to-openvino-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="123-detectron2-to-openvino-with-output_22_0.jpg">123-detectron2-to-openvino-with-output_22_0.jpg</a> 26-Jan-2024 01:05 59043
|
||||
<a href="123-detectron2-to-openvino-with-output_22_0.png">123-detectron2-to-openvino-with-output_22_0.png</a> 26-Jan-2024 01:05 508506
|
||||
<a href="123-detectron2-to-openvino-with-output_32_0.jpg">123-detectron2-to-openvino-with-output_32_0.jpg</a> 26-Jan-2024 01:05 55966
|
||||
<a href="123-detectron2-to-openvino-with-output_32_0.png">123-detectron2-to-openvino-with-output_32_0.png</a> 26-Jan-2024 01:05 457794
|
||||
<a href="123-detectron2-to-openvino-with-output_8_0.jpg">123-detectron2-to-openvino-with-output_8_0.jpg</a> 26-Jan-2024 01:05 46858
|
||||
<a href="123-detectron2-to-openvino-with-output_8_0.png">123-detectron2-to-openvino-with-output_8_0.png</a> 26-Jan-2024 01:05 503218
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
🤗 Hugging Face Model Hub with OpenVINO™
|
||||
Hugging Face Model Hub with OpenVINO™
|
||||
=========================================
|
||||
|
||||
The Hugging Face (HF) `Model Hub <https://huggingface.co/models>`__ is a
|
||||
|
|
@ -129,7 +129,7 @@ tutorials <https://huggingface.co/learn/nlp-course/chapter2/2?fw=pt#behind-the-p
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
return self.fget.__get__(instance, owner)()
|
||||
|
||||
|
||||
|
|
@ -268,7 +268,7 @@ Converting a Model Using the Optimum Intel Package
|
|||
|
||||
|
||||
|
||||
🤗 Optimum Intel is the interface between the 🤗 Transformers and
|
||||
Optimum Intel is the interface between the Transformers and
|
||||
Diffusers libraries and the different tools and libraries provided by
|
||||
Intel to accelerate end-to-end pipelines on Intel architectures.
|
||||
|
||||
|
|
@ -343,14 +343,14 @@ documentation <https://huggingface.co/docs/optimum/intel/inference>`__.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:14:43.057378: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-01-25 23:14:43.091971: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-02-09 23:10:50.826096: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-02-09 23:10:50.861099: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:14:43.657635: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:10:51.428729: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Initialize and Convert the Model Automatically using OVModel class
|
||||
|
|
@ -478,7 +478,7 @@ Full list of supported arguments available via ``--help``
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:14:55.440371: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:11:03.409282: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -490,6 +490,7 @@ Full list of supported arguments available via ``--help``
|
|||
[--int8]
|
||||
[--weight-format {fp32,fp16,int8,int4_sym_g128,int4_asym_g128,int4_sym_g64,int4_asym_g64}]
|
||||
[--ratio RATIO] [--disable-stateful]
|
||||
[--convert-tokenizer]
|
||||
output
|
||||
|
||||
optional arguments:
|
||||
|
|
@ -506,21 +507,20 @@ Full list of supported arguments available via ``--help``
|
|||
--task TASK The task to export the model for. If not specified,
|
||||
the task will be auto-inferred based on the model.
|
||||
Available tasks depend on the model, but are among:
|
||||
['stable-diffusion', 'audio-xvector', 'text-
|
||||
classification', 'automatic-speech-recognition',
|
||||
'image-segmentation', 'audio-classification',
|
||||
'multiple-choice', 'image-to-text', 'stable-diffusion-
|
||||
xl', 'zero-shot-image-classification', 'text-to-
|
||||
audio', 'fill-mask', 'object-detection', 'image-to-
|
||||
image', 'mask-generation', 'image-classification',
|
||||
'feature-extraction', 'semantic-segmentation', 'text-
|
||||
generation', 'question-answering', 'conversational',
|
||||
'zero-shot-object-detection', 'masked-im', 'sentence-
|
||||
similarity', 'audio-frame-classification', 'text2text-
|
||||
generation', 'depth-estimation', 'token-
|
||||
classification']. For decoder models, use `xxx-with-
|
||||
past` to export the model using past key values in the
|
||||
decoder.
|
||||
['sentence-similarity', 'object-detection', 'question-
|
||||
answering', 'text-to-audio', 'audio-xvector', 'stable-
|
||||
diffusion-xl', 'feature-extraction', 'image-to-image',
|
||||
'text-generation', 'mask-generation', 'text-
|
||||
classification', 'image-segmentation', 'automatic-
|
||||
speech-recognition', 'text2text-generation', 'stable-
|
||||
diffusion', 'audio-classification', 'semantic-
|
||||
segmentation', 'fill-mask', 'depth-estimation', 'zero-
|
||||
shot-image-classification', 'image-to-text', 'zero-
|
||||
shot-object-detection', 'multiple-choice',
|
||||
'conversational', 'image-classification', 'masked-im',
|
||||
'audio-frame-classification', 'token-classification'].
|
||||
For decoder models, use `xxx-with-past` to export the
|
||||
model using past key values in the decoder.
|
||||
--cache_dir CACHE_DIR
|
||||
Path indicating where to store cache.
|
||||
--framework {pt,tf} The framework to use for the export. If not provided,
|
||||
|
|
@ -558,6 +558,8 @@ Full list of supported arguments available via ``--help``
|
|||
a stateless model, for example, to be compatible with
|
||||
existing OpenVINO native inference code that expects
|
||||
kv-cache inputs and outputs in the model.
|
||||
--convert-tokenizer Add converted tokenizer and detokenizer with OpenVINO
|
||||
Tokenizers
|
||||
|
||||
|
||||
The command line export for model from example above with FP16 weights
|
||||
|
|
@ -578,7 +580,7 @@ compression:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:14:59.738992: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-02-09 23:11:07.691775: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -593,7 +595,7 @@ compression:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
|
||||
return self.fget.__get__(instance, owner)()
|
||||
|
||||
|
||||
|
|
@ -658,13 +660,20 @@ Model inference is exactly the same as for the original model!
|
|||
3) negative 0.0031
|
||||
|
||||
|
||||
You can find more examples of using Optimum Intel here:
|
||||
|
||||
1. `Accelerate Inference of Sparse Transformer Models <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/116-sparsity-optimization>`__
|
||||
2. `Grammatical Error Correction with OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction>`__
|
||||
3. `Stable Diffusion v2.1 using Optimum-Intel OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/236-stable-diffusion-v2/236-stable-diffusion-v2-optimum-demo.ipynb>`__
|
||||
4. `Image generation with Stable Diffusion XL <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/248-stable-diffusion-xl>`__
|
||||
5. `Instruction following using Databricks Dolly 2.0 <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following>`__
|
||||
6. `Create LLM-powered Chatbot using OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot>`__
|
||||
7. `Document Visual Question Answering Using Pix2Struct and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/260-pix2struct-docvqa>`__
|
||||
8. `Automatic speech recognition using Distil-Whisper and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
|
||||
You can find more examples of using Optimum Intel here: 1. `Accelerate
|
||||
Inference of Sparse Transformer
|
||||
Models <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/116-sparsity-optimization>`__
|
||||
2. `Grammatical Error Correction with
|
||||
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction>`__
|
||||
3. `Stable Diffusion v2.1 using Optimum-Intel
|
||||
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/236-stable-diffusion-v2/236-stable-diffusion-v2-optimum-demo.ipynb>`__
|
||||
4. `Image generation with Stable Diffusion
|
||||
XL <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/248-stable-diffusion-xl>`__
|
||||
5. `Instruction following using Databricks Dolly
|
||||
2.0 <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following>`__
|
||||
6. `Create LLM-powered Chatbot using
|
||||
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot>`__
|
||||
7. `Document Visual Question Answering Using Pix2Struct and
|
||||
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/260-pix2struct-docvqa>`__
|
||||
8. `Automatic speech recognition using Distil-Whisper and
|
||||
OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
|
||||
|
|
|
|||
|
|
@ -37,8 +37,10 @@ Table of contents:
|
|||
- `Use the OpenVINO IR model to run an
|
||||
inference <#use-the-openvino-ir-model-to-run-an-inference>`__
|
||||
|
||||
Prerequisites\
|
||||
-------------------------------------------------------
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -144,12 +146,14 @@ wight <https://pytorch.org/vision/stable/models.html#using-the-pre-trained-model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
|
||||
warnings.warn(
|
||||
|
||||
|
||||
Use the original model to run an inference\
|
||||
------------------------------------------------------------------------------------
|
||||
Use the original model to run an inference
|
||||
------------------------------------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -188,7 +192,7 @@ And print results
|
|||
|
||||
Predicted Class: 281
|
||||
Predicted Label: n02123045 tabby, tabby cat
|
||||
Predicted Probability: 0.6184040307998657
|
||||
Predicted Probability: 0.5800774693489075
|
||||
|
||||
|
||||
Convert the model to OpenVINO Intermediate representation format
|
||||
|
|
@ -260,8 +264,10 @@ Select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
compiled_model = core.compile_model(ov_model_xml_path, device_name=device.value)
|
||||
|
||||
Use the OpenVINO IR model to run an inference\
|
||||
---------------------------------------------------------------------------------------
|
||||
Use the OpenVINO IR model to run an inference
|
||||
---------------------------------------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-convnext-classification-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-convnext-classification-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="125-convnext-classification-with-output_4_0.jpg">125-convnext-classification-with-output_4_0.jpg</a> 26-Jan-2024 01:05 63187
|
||||
<a href="125-convnext-classification-with-output_4_0.png">125-convnext-classification-with-output_4_0.png</a> 26-Jan-2024 01:05 723717
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -22,27 +22,28 @@ plant, sheep, sofa, train, tv monitor**
|
|||
More information about the model is available in the `torchvision
|
||||
documentation <https://pytorch.org/vision/main/models/lraspp.html>`__
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
|
||||
**Table of contents:**
|
||||
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Prerequisites) <#prerequisites>`__
|
||||
- `Get a test image <#get-a-test-image>`__
|
||||
- `Download and prepare a model <#download-and-prepare-a-model>`__
|
||||
- `Define a preprocessing and prepare an input
|
||||
data <#define-a-preprocessing-and-prepare-an-input-data>`__
|
||||
- `Run an inference on the PyTorch
|
||||
model <#run-an-inference-on-the-pytorch-model>`__
|
||||
model) <#run-an-inference-on-the-pytorch-model>`__
|
||||
- `Convert the original model to OpenVINO IR
|
||||
Format <#convert-the-original-model-to-openvino-ir-format>`__
|
||||
- `Run an inference on the OpenVINO
|
||||
model <#run-an-inference-on-the-openvino-model>`__
|
||||
model) <#run-an-inference-on-the-openvino-model>`__
|
||||
- `Show results <#show-results>`__
|
||||
- `Show results for the OpenVINO IR
|
||||
model <#show-results-for-the-openvino-ir-model>`__
|
||||
model) <#show-results-for-the-openvino-ir-model>`__
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
|
||||
Prerequisites\
|
||||
-------------------------------------------------------
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -69,24 +70,25 @@ Prerequisites\
|
|||
.. code:: ipython3
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
import openvino as ov
|
||||
import torch
|
||||
|
||||
Get a test image
|
||||
----------------
|
||||
|
||||
First of all lets get a test
|
||||
image from an open dataset.
|
||||
|
||||
|
||||
First of all lets get a test image from an open dataset.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import urllib.request
|
||||
|
||||
|
||||
from torchvision.io import read_image
|
||||
import torchvision.transforms as transforms
|
||||
|
||||
|
||||
|
||||
|
||||
img_path = 'cats_image.jpeg'
|
||||
urllib.request.urlretrieve(
|
||||
url='https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg',
|
||||
|
|
@ -103,10 +105,11 @@ image from an open dataset.
|
|||
Download and prepare a model
|
||||
----------------------------
|
||||
|
||||
Define width and height of the
|
||||
image that will be used by the network during inference. According to
|
||||
the input transforms function, the model is pre-trained on images with a
|
||||
height of 480 and width of 640.
|
||||
|
||||
|
||||
Define width and height of the image that will be used by the network
|
||||
during inference. According to the input transforms function, the model
|
||||
is pre-trained on images with a height of 480 and width of 640.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -119,12 +122,12 @@ models <https://pytorch.org/vision/stable/models.html#listing-and-retrieving-ava
|
|||
.. code:: ipython3
|
||||
|
||||
import torchvision.models as models
|
||||
|
||||
|
||||
# List available models
|
||||
all_models = models.list_models()
|
||||
# List of models by type
|
||||
segmentation_models = models.list_models(module=models.segmentation)
|
||||
|
||||
|
||||
print(segmentation_models)
|
||||
|
||||
|
||||
|
|
@ -155,24 +158,27 @@ but there is only one for this model.
|
|||
Define a preprocessing and prepare an input data
|
||||
------------------------------------------------
|
||||
|
||||
You can use
|
||||
``torchvision.transforms`` to make a preprocessing or
|
||||
|
||||
|
||||
You can use ``torchvision.transforms`` to make a preprocessing or
|
||||
use\ `preprocessing transforms from the model
|
||||
wight <https://pytorch.org/vision/stable/models.html#using-the-pre-trained-models>`__.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
|
||||
|
||||
preprocess = models.segmentation.LRASPP_MobileNet_V3_Large_Weights.COCO_WITH_VOC_LABELS_V1.transforms()
|
||||
preprocess.resize_size = (IMAGE_HEIGHT, IMAGE_WIDTH) # change to an image size
|
||||
|
||||
|
||||
input_data = preprocess(image)
|
||||
input_data = np.expand_dims(input_data, axis=0)
|
||||
|
||||
Run an inference on the PyTorch model\
|
||||
-------------------------------------------------------------------------------
|
||||
Run an inference on the PyTorch model
|
||||
-------------------------------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
@ -193,8 +199,8 @@ directory. For more information on how to convert models, see this
|
|||
.. code:: ipython3
|
||||
|
||||
ov_model_xml_path = Path('models/ov_lraspp_model.xml')
|
||||
|
||||
|
||||
|
||||
|
||||
if not ov_model_xml_path.exists():
|
||||
ov_model_xml_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
|
||||
|
|
@ -203,15 +209,17 @@ directory. For more information on how to convert models, see this
|
|||
else:
|
||||
print(f"IR model {ov_model_xml_path} already exists.")
|
||||
|
||||
Run an inference on the OpenVINO model\
|
||||
--------------------------------------------------------------------------------
|
||||
Run an inference on the OpenVINO model
|
||||
--------------------------------------
|
||||
|
||||
|
||||
|
||||
Select device from dropdown list for running inference using OpenVINO
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
|
|
@ -219,7 +227,7 @@ Select device from dropdown list for running inference using OpenVINO
|
|||
description='Device:',
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -244,9 +252,10 @@ Run an inference
|
|||
Show results
|
||||
------------
|
||||
|
||||
Confirm that the segmentation
|
||||
results look as expected by comparing model predictions on the OpenVINO
|
||||
IR and PyTorch models.
|
||||
|
||||
|
||||
Confirm that the segmentation results look as expected by comparing
|
||||
model predictions on the OpenVINO IR and PyTorch models.
|
||||
|
||||
You can use `pytorch
|
||||
tutorial <https://pytorch.org/vision/0.12/auto_examples/plot_visualization_utils.html#sphx-glr-auto-examples-plot-visualization-utils-py>`__
|
||||
|
|
@ -257,13 +266,13 @@ visualize the image with a ``cat`` mask for the PyTorch model.
|
|||
|
||||
import torch
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
|
||||
import torchvision.transforms.functional as F
|
||||
|
||||
|
||||
|
||||
|
||||
plt.rcParams["savefig.bbox"] = 'tight'
|
||||
|
||||
|
||||
|
||||
|
||||
def show(imgs):
|
||||
if not isinstance(imgs, list):
|
||||
imgs = [imgs]
|
||||
|
|
@ -284,11 +293,11 @@ Prepare and display a cat mask.
|
|||
'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor'
|
||||
]
|
||||
sem_class_to_idx = {cls: idx for (idx, cls) in enumerate(sem_classes)}
|
||||
|
||||
|
||||
normalized_mask = torch.nn.functional.softmax(result_torch, dim=1)
|
||||
|
||||
|
||||
cat_mask = normalized_mask[0, sem_class_to_idx['cat']]
|
||||
|
||||
|
||||
show(cat_mask)
|
||||
|
||||
|
||||
|
|
@ -313,7 +322,7 @@ And now we can plot a boolean mask on top of the original image.
|
|||
.. code:: ipython3
|
||||
|
||||
from torchvision.utils import draw_segmentation_masks
|
||||
|
||||
|
||||
show(draw_segmentation_masks(image, masks=boolean_cat_mask, alpha=0.7, colors='yellow'))
|
||||
|
||||
|
||||
|
|
@ -321,8 +330,10 @@ And now we can plot a boolean mask on top of the original image.
|
|||
.. image:: 125-lraspp-segmentation-with-output_files/125-lraspp-segmentation-with-output_32_0.png
|
||||
|
||||
|
||||
Show results for the OpenVINO IR model\
|
||||
--------------------------------------------------------------------------------
|
||||
Show results for the OpenVINO IR model
|
||||
--------------------------------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-lraspp-segmentation-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/125-lraspp-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="125-lraspp-segmentation-with-output_28_0.png">125-lraspp-segmentation-with-output_28_0.png</a> 26-Jan-2024 01:05 87576
|
||||
<a href="125-lraspp-segmentation-with-output_32_0.png">125-lraspp-segmentation-with-output_32_0.png</a> 26-Jan-2024 01:05 385522
|
||||
<a href="125-lraspp-segmentation-with-output_34_0.png">125-lraspp-segmentation-with-output_34_0.png</a> 26-Jan-2024 01:05 385520
|
||||
<a href="125-lraspp-segmentation-with-output_5_0.jpg">125-lraspp-segmentation-with-output_5_0.jpg</a> 26-Jan-2024 01:05 63187
|
||||
<a href="125-lraspp-segmentation-with-output_5_0.png">125-lraspp-segmentation-with-output_5_0.png</a> 26-Jan-2024 01:05 723717
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -21,22 +21,28 @@ You have the flexibility to run this tutorial notebook in its entirety
|
|||
or selectively execute specific sections, as each section operates
|
||||
independently.
|
||||
|
||||
**Table of contents:**
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- `Image classification <#image-classification>`__
|
||||
- `Install required packages <#install-required-packages>`__
|
||||
- `Import libraries <#import-libraries>`__
|
||||
- `Download the classifier <#download-the-classifier>`__
|
||||
- `Download a single image to try the model on <#download-a-single-image-to-try-the-model-on>`__
|
||||
- `Convert model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Inference <#inference>`__
|
||||
- `Image style transfer <#image-style-transfer>`__
|
||||
- `Install required packages <#install-required-packages>`__
|
||||
- `Load the model <#load-the-model>`__
|
||||
- `Convert the model to OpenVINO IR <#convert-the-model-to-openvino-ir>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Inference <#inference>`__
|
||||
- `Image classification <#image-classification>`__
|
||||
|
||||
- `Install required packages <#install-required-packages>`__
|
||||
- `Import libraries <#import-libraries>`__
|
||||
- `Download the classifier <#download-the-classifier>`__
|
||||
- `Download a single image to try the model
|
||||
on <#download-a-single-image-to-try-the-model-on>`__
|
||||
- `Convert model to OpenVINO IR <#convert-model-to-openvino-ir>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Inference <#inference>`__
|
||||
|
||||
- `Image style transfer <#image-style-transfer>`__
|
||||
|
||||
- `Install required packages <#install-required-packages>`__
|
||||
- `Load the model <#load-the-model>`__
|
||||
- `Convert the model to OpenVINO
|
||||
IR <#convert-the-model-to-openvino-ir>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Inference <#inference>`__
|
||||
|
||||
Image classification
|
||||
--------------------
|
||||
|
|
@ -125,8 +131,8 @@ and wrap it as a Keras layer with ``hub.KerasLayer``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:15:55.944757: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-01-25 23:15:55.944938: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-02-09 23:12:03.569013: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-02-09 23:12:03.569190: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
Download a single image to try the model on
|
||||
|
|
|
|||
|
|
@ -1,10 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/126-tensorflow-hub-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/126-tensorflow-hub-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="126-tensorflow-hub-with-output_11_0.jpg">126-tensorflow-hub-with-output_11_0.jpg</a> 26-Jan-2024 01:05 10479
|
||||
<a href="126-tensorflow-hub-with-output_11_0.png">126-tensorflow-hub-with-output_11_0.png</a> 26-Jan-2024 01:05 92843
|
||||
<a href="126-tensorflow-hub-with-output_26_0.png">126-tensorflow-hub-with-output_26_0.png</a> 26-Jan-2024 01:05 203738
|
||||
<a href="126-tensorflow-hub-with-output_45_0.png">126-tensorflow-hub-with-output_45_0.png</a> 26-Jan-2024 01:05 538743
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:37f1a76ccf411397019d83e8344b79cdebf644f0f6ba6598141e72c6ee50c62b
|
||||
size 224517
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:33f7798bd5dfbfa423dcca1d37eb7e29d0474e331105347cf0efdcfd695b850a
|
||||
size 335047
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6b354964f6b453eb9454e0e21ccb3e1275071464698af4693fc94235a1ca563e
|
||||
size 296205
|
||||
|
|
@ -3,7 +3,7 @@ Monodepth Estimation with OpenVINO
|
|||
|
||||
This tutorial demonstrates Monocular Depth Estimation with MidasNet in
|
||||
OpenVINO. Model information can be found
|
||||
`here <https://docs.openvino.ai/2023.0/omz_models_model_midasnet.html>`__.
|
||||
`here <https://docs.openvino.ai/2023.3/omz_models_model_midasnet.html>`__.
|
||||
|
||||
.. figure:: https://user-images.githubusercontent.com/36741649/127173017-a0bbcf75-db24-4d2c-81b9-616e04ab7cd9.gif
|
||||
:alt: monodepth
|
||||
|
|
@ -69,7 +69,7 @@ Install requirements
|
|||
|
||||
%pip install -q "openvino>=2023.1.0"
|
||||
%pip install -q matplotlib opencv-python requests tqdm
|
||||
|
||||
|
||||
# Fetch `notebook_utils` module
|
||||
import urllib.request
|
||||
urllib.request.urlretrieve(
|
||||
|
|
@ -92,7 +92,7 @@ Install requirements
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
('notebook_utils.py', <http.client.HTTPMessage at 0x7fed08307550>)
|
||||
('notebook_utils.py', <http.client.HTTPMessage at 0x7fe5c4f7e130>)
|
||||
|
||||
|
||||
|
||||
|
|
@ -105,7 +105,7 @@ Imports
|
|||
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
import cv2
|
||||
import matplotlib.cm
|
||||
import matplotlib.pyplot as plt
|
||||
|
|
@ -120,7 +120,7 @@ Imports
|
|||
display,
|
||||
)
|
||||
import openvino as ov
|
||||
|
||||
|
||||
from notebook_utils import download_file, load_image
|
||||
|
||||
Download the model
|
||||
|
|
@ -134,14 +134,14 @@ The model is in the `OpenVINO Intermediate Representation
|
|||
.. code:: ipython3
|
||||
|
||||
model_folder = Path('model')
|
||||
|
||||
|
||||
ir_model_url = 'https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/depth-estimation-midas/FP32/'
|
||||
ir_model_name_xml = 'MiDaS_small.xml'
|
||||
ir_model_name_bin = 'MiDaS_small.bin'
|
||||
|
||||
|
||||
download_file(ir_model_url + ir_model_name_xml, filename=ir_model_name_xml, directory=model_folder)
|
||||
download_file(ir_model_url + ir_model_name_bin, filename=ir_model_name_bin, directory=model_folder)
|
||||
|
||||
|
||||
model_xml_path = model_folder / ir_model_name_xml
|
||||
|
||||
|
||||
|
|
@ -167,13 +167,13 @@ Functions
|
|||
def normalize_minmax(data):
|
||||
"""Normalizes the values in `data` between 0 and 1"""
|
||||
return (data - data.min()) / (data.max() - data.min())
|
||||
|
||||
|
||||
|
||||
|
||||
def convert_result_to_image(result, colormap="viridis"):
|
||||
"""
|
||||
Convert network result of floating point numbers to an RGB image with
|
||||
integer values from 0-255 by applying a colormap.
|
||||
|
||||
|
||||
`result` is expected to be a single network result in 1,H,W shape
|
||||
`colormap` is a matplotlib colormap.
|
||||
See https://matplotlib.org/stable/tutorials/colors/colormaps.html
|
||||
|
|
@ -184,8 +184,8 @@ Functions
|
|||
result = cmap(result)[:, :, :3] * 255
|
||||
result = result.astype(np.uint8)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
|
||||
def to_rgb(image_data) -> np.ndarray:
|
||||
"""
|
||||
Convert image_data from BGR to RGB
|
||||
|
|
@ -202,7 +202,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
|
|
@ -210,7 +210,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
description='Device:',
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -237,10 +237,10 @@ output keys and the expected input shape for the model.
|
|||
core.set_property({'CACHE_DIR': '../cache'})
|
||||
model = core.read_model(model_xml_path)
|
||||
compiled_model = core.compile_model(model=model, device_name=device.value)
|
||||
|
||||
|
||||
input_key = compiled_model.input(0)
|
||||
output_key = compiled_model.output(0)
|
||||
|
||||
|
||||
network_input_shape = list(input_key.shape)
|
||||
network_image_height, network_image_width = network_input_shape[2:]
|
||||
|
||||
|
|
@ -262,10 +262,10 @@ H=height, W=width).
|
|||
|
||||
IMAGE_FILE = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_bike.jpg"
|
||||
image = load_image(path=IMAGE_FILE)
|
||||
|
||||
|
||||
# Resize to input shape for network.
|
||||
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
||||
|
||||
|
||||
# Reshape the image to network input shape NCHW.
|
||||
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
||||
|
||||
|
|
@ -280,11 +280,11 @@ original image shape.
|
|||
.. code:: ipython3
|
||||
|
||||
result = compiled_model([input_image])[output_key]
|
||||
|
||||
|
||||
# Convert the network result of disparity map to an image that shows
|
||||
# distance as colors.
|
||||
result_image = convert_result_to_image(result=result)
|
||||
|
||||
|
||||
# Resize back to original image shape. The `cv2.resize` function expects shape
|
||||
# in (width, height), [::-1] reverses the (height, width) shape to match this.
|
||||
result_image = cv2.resize(result_image, image.shape[:2][::-1])
|
||||
|
|
@ -292,7 +292,7 @@ original image shape.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/tmp/ipykernel_809788/2076527990.py:15: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
|
||||
/tmp/ipykernel_2841459/2076527990.py:15: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.
|
||||
cmap = matplotlib.cm.get_cmap(colormap)
|
||||
|
||||
|
||||
|
|
@ -346,7 +346,7 @@ Video Settings
|
|||
# Try the `THEO` encoding if you have FFMPEG installed.
|
||||
# FOURCC = cv2.VideoWriter_fourcc(*"THEO")
|
||||
FOURCC = cv2.VideoWriter_fourcc(*"vp09")
|
||||
|
||||
|
||||
# Create Path objects for the input video and the result video.
|
||||
output_directory = Path("output")
|
||||
output_directory.mkdir(exist_ok=True)
|
||||
|
|
@ -369,11 +369,11 @@ compute values for these properties for the monodepth video.
|
|||
raise ValueError(f"The video at {VIDEO_FILE} cannot be read.")
|
||||
input_fps = cap.get(cv2.CAP_PROP_FPS)
|
||||
input_video_frame_height, input_video_frame_width = image.shape[:2]
|
||||
|
||||
|
||||
target_fps = input_fps / ADVANCE_FRAMES
|
||||
target_frame_height = int(input_video_frame_height * SCALE_OUTPUT)
|
||||
target_frame_width = int(input_video_frame_width * SCALE_OUTPUT)
|
||||
|
||||
|
||||
cap.release()
|
||||
print(
|
||||
f"The input video has a frame width of {input_video_frame_width}, "
|
||||
|
|
@ -403,10 +403,10 @@ Do Inference on a Video and Create Monodepth Video
|
|||
input_video_frame_nr = 0
|
||||
start_time = time.perf_counter()
|
||||
total_inference_duration = 0
|
||||
|
||||
|
||||
# Open the input video
|
||||
cap = cv2.VideoCapture(str(VIDEO_FILE))
|
||||
|
||||
|
||||
# Create a result video.
|
||||
out_video = cv2.VideoWriter(
|
||||
str(result_video_path),
|
||||
|
|
@ -414,36 +414,36 @@ Do Inference on a Video and Create Monodepth Video
|
|||
target_fps,
|
||||
(target_frame_width * 2, target_frame_height),
|
||||
)
|
||||
|
||||
|
||||
num_frames = int(NUM_SECONDS * input_fps)
|
||||
total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT) if num_frames == 0 else num_frames
|
||||
progress_bar = ProgressBar(total=total_frames)
|
||||
progress_bar.display()
|
||||
|
||||
|
||||
try:
|
||||
while cap.isOpened():
|
||||
ret, image = cap.read()
|
||||
if not ret:
|
||||
cap.release()
|
||||
break
|
||||
|
||||
|
||||
if input_video_frame_nr >= total_frames:
|
||||
break
|
||||
|
||||
|
||||
# Only process every second frame.
|
||||
# Prepare a frame for inference.
|
||||
# Resize to the input shape for network.
|
||||
resized_image = cv2.resize(src=image, dsize=(network_image_height, network_image_width))
|
||||
# Reshape the image to network input shape NCHW.
|
||||
input_image = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0)
|
||||
|
||||
|
||||
# Do inference.
|
||||
inference_start_time = time.perf_counter()
|
||||
result = compiled_model([input_image])[output_key]
|
||||
inference_stop_time = time.perf_counter()
|
||||
inference_duration = inference_stop_time - inference_start_time
|
||||
total_inference_duration += inference_duration
|
||||
|
||||
|
||||
if input_video_frame_nr % (10 * ADVANCE_FRAMES) == 0:
|
||||
clear_output(wait=True)
|
||||
progress_bar.display()
|
||||
|
|
@ -457,7 +457,7 @@ Do Inference on a Video and Create Monodepth Video
|
|||
f"({1/inference_duration:.2f} FPS)"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# Transform the network result to a RGB image.
|
||||
result_frame = to_rgb(convert_result_to_image(result))
|
||||
# Resize the image and the result to a target frame shape.
|
||||
|
|
@ -467,13 +467,13 @@ Do Inference on a Video and Create Monodepth Video
|
|||
stacked_frame = np.hstack((image, result_frame))
|
||||
# Save a frame to the video.
|
||||
out_video.write(stacked_frame)
|
||||
|
||||
|
||||
input_video_frame_nr = input_video_frame_nr + ADVANCE_FRAMES
|
||||
cap.set(1, input_video_frame_nr)
|
||||
|
||||
|
||||
progress_bar.progress = input_video_frame_nr
|
||||
progress_bar.update()
|
||||
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("Processing interrupted.")
|
||||
finally:
|
||||
|
|
@ -483,7 +483,7 @@ Do Inference on a Video and Create Monodepth Video
|
|||
cap.release()
|
||||
end_time = time.perf_counter()
|
||||
duration = end_time - start_time
|
||||
|
||||
|
||||
print(
|
||||
f"Processed {processed_frames} frames in {duration:.2f} seconds. "
|
||||
f"Total FPS (including video processing): {processed_frames/duration:.2f}."
|
||||
|
|
@ -494,7 +494,7 @@ Do Inference on a Video and Create Monodepth Video
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Processed 60 frames in 37.40 seconds. Total FPS (including video processing): 1.60.Inference FPS: 43.54
|
||||
Processed 60 frames in 37.50 seconds. Total FPS (including video processing): 1.60.Inference FPS: 43.40
|
||||
Monodepth Video saved to 'output/Coco%20Walking%20in%20Berkeley_monodepth.mp4'.
|
||||
|
||||
|
||||
|
|
@ -524,8 +524,8 @@ Display Monodepth Video
|
|||
.. parsed-literal::
|
||||
|
||||
Showing monodepth video saved at
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/201-vision-monodepth/output/Coco%20Walking%20in%20Berkeley_monodepth.mp4
|
||||
If you cannot see the video in your browser, please click on the following link to download the video
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/201-vision-monodepth/output/Coco%20Walking%20in%20Berkeley_monodepth.mp4
|
||||
If you cannot see the video in your browser, please click on the following link to download the video
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/201-vision-monodepth-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/201-vision-monodepth-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="201-vision-monodepth-with-output_18_0.png">201-vision-monodepth-with-output_18_0.png</a> 26-Jan-2024 01:05 959858
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -5,7 +5,7 @@ Super Resolution is the process of enhancing the quality of an image by
|
|||
increasing the pixel count using deep learning. This notebook shows the
|
||||
Single Image Super Resolution (SISR) which takes just one low resolution
|
||||
image. A model called
|
||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.0/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.3/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||
which is available in Open Model Zoo, is used in this tutorial. It is
|
||||
based on the research paper cited below.
|
||||
|
||||
|
|
@ -774,8 +774,8 @@ as total time to process each patch.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Processed 42 patches in 4.63 seconds. Total patches per second (including processing): 9.06.
|
||||
Inference patches per second: 17.81
|
||||
Processed 42 patches in 4.64 seconds. Total patches per second (including processing): 9.05.
|
||||
Inference patches per second: 17.92
|
||||
|
||||
|
||||
Save superresolution image and the bicubic image
|
||||
|
|
|
|||
|
|
@ -1,10 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/202-vision-superresolution-image-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/202-vision-superresolution-image-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="202-vision-superresolution-image-with-output_15_1.png">202-vision-superresolution-image-with-output_15..></a> 26-Jan-2024 01:05 272963
|
||||
<a href="202-vision-superresolution-image-with-output_17_1.png">202-vision-superresolution-image-with-output_17..></a> 26-Jan-2024 01:05 356735
|
||||
<a href="202-vision-superresolution-image-with-output_23_1.png">202-vision-superresolution-image-with-output_23..></a> 26-Jan-2024 01:05 2896276
|
||||
<a href="202-vision-superresolution-image-with-output_27_1.png">202-vision-superresolution-image-with-output_27..></a> 26-Jan-2024 01:05 3207711
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -5,7 +5,7 @@ Super Resolution is the process of enhancing the quality of an image by
|
|||
increasing the pixel count using deep learning. This notebook applies
|
||||
Single Image Super Resolution (SISR) to frames in a 360p (480×360) video
|
||||
in 360p resolution. A model called
|
||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.0/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||
`single-image-super-resolution-1032 <https://docs.openvino.ai/2023.3/omz_models_model_single_image_super_resolution_1032.html>`__,
|
||||
which is available in Open Model Zoo, is used in this tutorial. It is
|
||||
based on the research paper cited below.
|
||||
|
||||
|
|
@ -81,7 +81,7 @@ Imports
|
|||
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from IPython.display import (
|
||||
|
|
@ -120,7 +120,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
|
|
@ -128,7 +128,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
description='Device:',
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -144,20 +144,20 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
# 1032: 4x superresolution, 1033: 3x superresolution
|
||||
model_name = 'single-image-super-resolution-1032'
|
||||
|
||||
|
||||
base_model_dir = Path('./model').expanduser()
|
||||
|
||||
|
||||
model_xml_name = f'{model_name}.xml'
|
||||
model_bin_name = f'{model_name}.bin'
|
||||
|
||||
|
||||
model_xml_path = base_model_dir / model_xml_name
|
||||
model_bin_path = base_model_dir / model_bin_name
|
||||
|
||||
|
||||
if not model_xml_path.exists():
|
||||
base_url = f'https://storage.openvinotoolkit.org/repositories/open_model_zoo/2023.0/models_bin/1/{model_name}/FP16/'
|
||||
model_xml_url = base_url + model_xml_name
|
||||
model_bin_url = base_url + model_bin_name
|
||||
|
||||
|
||||
download_file(model_xml_url, model_xml_path)
|
||||
download_file(model_bin_url, model_bin_path)
|
||||
else:
|
||||
|
|
@ -180,7 +180,7 @@ Functions
|
|||
"""
|
||||
Convert network result of floating point numbers to image with integer
|
||||
values from 0-255. Values outside this range are clipped to 0 and 255.
|
||||
|
||||
|
||||
:param result: a single superresolution network result in N,C,H,W shape
|
||||
"""
|
||||
result = result.squeeze(0).transpose(1, 2, 0)
|
||||
|
|
@ -215,18 +215,18 @@ resolution version of the image in 1920x1080.
|
|||
# dictionaries.
|
||||
original_image_key, bicubic_image_key = compiled_model.inputs
|
||||
output_key = compiled_model.output(0)
|
||||
|
||||
|
||||
# Get the expected input and target shape. The `.dims[2:]` function returns the height
|
||||
# and width.The `resize` function of OpenCV expects the shape as (width, height),
|
||||
# so reverse the shape with `[::-1]` and convert it to a tuple.
|
||||
input_height, input_width = list(original_image_key.shape)[2:]
|
||||
target_height, target_width = list(bicubic_image_key.shape)[2:]
|
||||
|
||||
|
||||
upsample_factor = int(target_height / input_height)
|
||||
|
||||
|
||||
print(f"The network expects inputs with a width of {input_width}, " f"height of {input_height}")
|
||||
print(f"The network returns images with a width of {target_width}, " f"height of {target_height}")
|
||||
|
||||
|
||||
print(
|
||||
f"The image sides are upsampled by a factor of {upsample_factor}. "
|
||||
f"The new image is {upsample_factor**2} times as large as the "
|
||||
|
|
@ -264,7 +264,7 @@ Settings
|
|||
.. code:: ipython3
|
||||
|
||||
OUTPUT_DIR = "output"
|
||||
|
||||
|
||||
Path(OUTPUT_DIR).mkdir(exist_ok=True)
|
||||
# Maximum number of frames to read from the input video. Set to 0 to read all frames.
|
||||
NUM_FRAMES = 100
|
||||
|
|
@ -290,10 +290,10 @@ Download and Prepare Video
|
|||
filename = Path(stream.default_filename.encode("ascii", "ignore").decode("ascii")).stem
|
||||
stream.download(output_path=OUTPUT_DIR, filename=filename)
|
||||
print(f"Video {filename} downloaded to {OUTPUT_DIR}")
|
||||
|
||||
|
||||
# Create Path objects for the input video and the resulting videos.
|
||||
video_path = Path(stream.get_file_path(filename, OUTPUT_DIR))
|
||||
|
||||
|
||||
# Path names for the result videos.
|
||||
superres_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_superres.mp4")
|
||||
bicubic_video_path = Path(f"{OUTPUT_DIR}/{video_path.stem}_bicubic.mp4")
|
||||
|
|
@ -314,14 +314,14 @@ Download and Prepare Video
|
|||
raise ValueError(f"The video at '{video_path}' cannot be read.")
|
||||
fps = cap.get(cv2.CAP_PROP_FPS)
|
||||
frame_count = cap.get(cv2.CAP_PROP_FRAME_COUNT)
|
||||
|
||||
|
||||
if NUM_FRAMES == 0:
|
||||
total_frames = frame_count
|
||||
else:
|
||||
total_frames = min(frame_count, NUM_FRAMES)
|
||||
|
||||
|
||||
original_frame_height, original_frame_width = image.shape[:2]
|
||||
|
||||
|
||||
cap.release()
|
||||
print(
|
||||
f"The input video has a frame width of {original_frame_width}, "
|
||||
|
|
@ -389,10 +389,10 @@ video.
|
|||
start_time = time.perf_counter()
|
||||
frame_nr = 0
|
||||
total_inference_duration = 0
|
||||
|
||||
|
||||
progress_bar = ProgressBar(total=total_frames)
|
||||
progress_bar.display()
|
||||
|
||||
|
||||
cap = cv2.VideoCapture(filename=str(video_path))
|
||||
try:
|
||||
while cap.isOpened():
|
||||
|
|
@ -400,22 +400,22 @@ video.
|
|||
if not ret:
|
||||
cap.release()
|
||||
break
|
||||
|
||||
|
||||
if frame_nr >= total_frames:
|
||||
break
|
||||
|
||||
|
||||
# Resize the input image to the network shape and convert it from (H,W,C) to
|
||||
# (N,C,H,W).
|
||||
resized_image = cv2.resize(src=image, dsize=(input_width, input_height))
|
||||
input_image_original = np.expand_dims(resized_image.transpose(2, 0, 1), axis=0)
|
||||
|
||||
|
||||
# Resize and reshape the image to the target shape with bicubic
|
||||
# interpolation.
|
||||
bicubic_image = cv2.resize(
|
||||
src=image, dsize=(target_width, target_height), interpolation=cv2.INTER_CUBIC
|
||||
)
|
||||
input_image_bicubic = np.expand_dims(bicubic_image.transpose(2, 0, 1), axis=0)
|
||||
|
||||
|
||||
# Do inference.
|
||||
inference_start_time = time.perf_counter()
|
||||
result = compiled_model(
|
||||
|
|
@ -427,19 +427,19 @@ video.
|
|||
inference_stop_time = time.perf_counter()
|
||||
inference_duration = inference_stop_time - inference_start_time
|
||||
total_inference_duration += inference_duration
|
||||
|
||||
|
||||
# Transform the inference result into an image.
|
||||
result_frame = convert_result_to_image(result=result)
|
||||
|
||||
|
||||
# Write the result image and the bicubic image to a video file.
|
||||
superres_video.write(image=result_frame)
|
||||
bicubic_video.write(image=bicubic_image)
|
||||
|
||||
|
||||
stacked_frame = np.hstack((bicubic_image, result_frame))
|
||||
comparison_video.write(image=stacked_frame)
|
||||
|
||||
|
||||
frame_nr = frame_nr + 1
|
||||
|
||||
|
||||
# Update the progress bar and the status message.
|
||||
progress_bar.progress = frame_nr
|
||||
progress_bar.update()
|
||||
|
|
@ -453,8 +453,8 @@ video.
|
|||
f"({1/inference_duration:.2f} FPS)"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("Processing interrupted.")
|
||||
finally:
|
||||
|
|
@ -480,13 +480,13 @@ video.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Processed frame 100. Inference time: 0.05 seconds (19.51 FPS)
|
||||
Processed frame 100. Inference time: 0.06 seconds (17.00 FPS)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Video's saved to output directory.
|
||||
Processed 100 frames in 242.29 seconds. Total FPS (including video processing): 0.41. Inference FPS: 18.44.
|
||||
Processed 100 frames in 243.08 seconds. Total FPS (including video processing): 0.41. Inference FPS: 17.69.
|
||||
|
||||
|
||||
Show Side-by-Side Video of Bicubic and Superresolution Version
|
||||
|
|
|
|||
|
|
@ -623,7 +623,7 @@ bounds of input batch size.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f3c87e408e0>
|
||||
<matplotlib.image.AxesImage at 0x7f476c1d6fd0>
|
||||
|
||||
|
||||
|
||||
|
|
@ -788,7 +788,7 @@ Get the reading result on the meter picture
|
|||
.. image:: 203-meter-reader-with-output_files/203-meter-reader-with-output_24_1.png
|
||||
|
||||
|
||||
Try it with your meter photos
|
||||
-----------------------------------------------
|
||||
Try it with your meter photos!
|
||||
------------------------------
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/203-meter-reader-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/203-meter-reader-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="203-meter-reader-with-output_16_1.png">203-meter-reader-with-output_16_1.png</a> 26-Jan-2024 01:04 170121
|
||||
<a href="203-meter-reader-with-output_18_1.png">203-meter-reader-with-output_18_1.png</a> 26-Jan-2024 01:04 190271
|
||||
<a href="203-meter-reader-with-output_20_1.png">203-meter-reader-with-output_20_1.png</a> 26-Jan-2024 01:04 26914
|
||||
<a href="203-meter-reader-with-output_22_1.png">203-meter-reader-with-output_22_1.png</a> 26-Jan-2024 01:04 8966
|
||||
<a href="203-meter-reader-with-output_24_1.png">203-meter-reader-with-output_24_1.png</a> 26-Jan-2024 01:04 170338
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -26,20 +26,31 @@ paper: `Segmenter: Transformer for Semantic
|
|||
Segmentation <https://arxiv.org/abs/2105.05633>`__ or in the
|
||||
`repository <https://github.com/rstrudel/segmenter>`__.
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
**Table of contents:**
|
||||
- `Get and prepare PyTorch model <#get-and-prepare-pytorch-model>`__
|
||||
|
||||
- `Get and prepare PyTorch model <#get-and-prepare-pytorch-model>`__
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Loading PyTorch model <#loading-pytorch-model>`__
|
||||
- `Preparing preprocessing and visualization functions <#preparing-preprocessing-and-visualization-functions>`__
|
||||
- `Preprocessing <#preprocessing>`__
|
||||
- `visualization <#visualization>`__
|
||||
- `Validation of inference of original model <#validation-of-inference-of-original-model>`__
|
||||
- `Convert PyTorch model to OpenVINO Intermediate Representation (IR) <#convert-pytorch-model-to-openvino-intermediate-representation-ir>`__
|
||||
- `Verify converted model inference <#verify-converted-model-inference>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
- `Benchmarking performance of converted model <#benchmarking-performance-of-converted-model>`__
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Loading PyTorch model <#loading-pytorch-model>`__
|
||||
|
||||
- `Preparing preprocessing and visualization
|
||||
functions <#preparing-preprocessing-and-visualization-functions>`__
|
||||
|
||||
- `Preprocessing <#preprocessing>`__
|
||||
- `Visualization <#visualization>`__
|
||||
|
||||
- `Validation of inference of original
|
||||
model <#validation-of-inference-of-original-model>`__
|
||||
- `Convert PyTorch model to OpenVINO Intermediate Representation
|
||||
(IR) <#convert-pytorch-model-to-openvino-intermediate-representation-ir>`__
|
||||
- `Verify converted model
|
||||
inference <#verify-converted-model-inference>`__
|
||||
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
|
||||
- `Benchmarking performance of converted
|
||||
model <#benchmarking-performance-of-converted-model>`__
|
||||
|
||||
.. |Segmenter diagram| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/93932510/f57979e7-fd3b-449f-bf01-afe0f965abbc
|
||||
|
||||
|
|
@ -79,13 +90,13 @@ Prerequisites
|
|||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
# clone Segmenter repo
|
||||
if not Path("segmenter").exists():
|
||||
!git clone https://github.com/rstrudel/segmenter
|
||||
else:
|
||||
print("Segmenter repo already cloned")
|
||||
|
||||
|
||||
# include path to Segmenter repo to use its functions
|
||||
sys.path.append("./segmenter")
|
||||
|
||||
|
|
@ -108,10 +119,7 @@ Receiving objects: 6% (17/268)
|
|||
Receiving objects: 7% (19/268)
|
||||
Receiving objects: 8% (22/268)
|
||||
Receiving objects: 9% (25/268)
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 10% (27/268)
|
||||
Receiving objects: 10% (27/268)
|
||||
Receiving objects: 11% (30/268)
|
||||
Receiving objects: 12% (33/268)
|
||||
Receiving objects: 13% (35/268)
|
||||
|
|
@ -135,118 +143,122 @@ Receiving objects: 22% (59/268)
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 24% (65/268), 2.55 MiB | 2.48 MiB/s
|
||||
Receiving objects: 24% (65/268), 3.68 MiB | 3.63 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 24% (66/268), 6.21 MiB | 3.02 MiB/s
|
||||
Receiving objects: 24% (66/268), 7.47 MiB | 3.70 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 25% (67/268), 6.21 MiB | 3.02 MiB/s
|
||||
Receiving objects: 25% (67/268), 7.47 MiB | 3.70 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 25% (68/268), 9.88 MiB | 3.19 MiB/s
|
||||
Receiving objects: 25% (68/268), 11.26 MiB | 3.73 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 26% (70/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 27% (73/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 28% (76/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 29% (78/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 30% (81/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 31% (84/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 32% (86/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 33% (89/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 34% (92/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 35% (94/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 36% (97/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 37% (100/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 38% (102/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 39% (105/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 40% (108/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 41% (110/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 42% (113/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 43% (116/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 44% (118/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 45% (121/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 46% (124/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 47% (126/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 48% (129/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 49% (132/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 50% (134/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 51% (137/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 52% (140/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 53% (143/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 54% (145/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 55% (148/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 56% (151/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 57% (153/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 58% (156/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 59% (159/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 60% (161/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 61% (164/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 62% (167/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 63% (169/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 64% (172/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 65% (175/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 66% (177/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 67% (180/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 68% (183/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 69% (185/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 70% (188/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 71% (191/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 72% (193/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 73% (196/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 74% (199/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 75% (201/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 76% (204/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 77% (207/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 78% (210/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 79% (212/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 80% (215/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 81% (218/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 82% (220/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 83% (223/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 84% (226/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 85% (228/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 86% (231/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 87% (234/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 88% (236/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 89% (239/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 26% (70/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 27% (73/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 28% (76/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 29% (78/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 30% (81/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 31% (84/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 32% (86/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 33% (89/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 34% (92/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 35% (94/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 36% (97/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 37% (100/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 38% (102/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 39% (105/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 40% (108/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 41% (110/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 42% (113/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 43% (116/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 44% (118/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 45% (121/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 46% (124/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 47% (126/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 48% (129/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 49% (132/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 50% (134/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 51% (137/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 52% (140/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 53% (143/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 54% (145/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 55% (148/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 56% (151/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 57% (153/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 58% (156/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 59% (159/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 60% (161/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 61% (164/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 62% (167/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 63% (169/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 64% (172/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 65% (175/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 66% (177/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 67% (180/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 68% (183/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 69% (185/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 70% (188/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 71% (191/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 72% (193/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 73% (196/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 74% (199/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 75% (201/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 76% (204/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 77% (207/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 78% (210/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 79% (212/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 80% (215/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 81% (218/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 82% (220/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 83% (223/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 84% (226/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 85% (228/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 86% (231/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 87% (234/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 88% (236/268), 11.26 MiB | 3.73 MiB/s
|
||||
Receiving objects: 89% (239/268), 11.26 MiB | 3.73 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 90% (242/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 90% (242/268), 11.26 MiB | 3.73 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 91% (244/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 91% (244/268), 11.26 MiB | 3.73 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 91% (246/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 92% (247/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 93% (250/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 94% (252/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 95% (255/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 96% (258/268), 11.71 MiB | 3.24 MiB/s
|
||||
Receiving objects: 92% (247/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 93% (250/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 94% (252/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 95% (255/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 96% (258/268), 13.11 MiB | 3.72 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 97% (260/268), 13.54 MiB | 3.28 MiB/s
|
||||
Receiving objects: 98% (263/268), 13.54 MiB | 3.28 MiB/s
|
||||
Receiving objects: 99% (266/268), 13.54 MiB | 3.28 MiB/s
|
||||
Receiving objects: 97% (260/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 98% (263/268), 13.11 MiB | 3.72 MiB/s
|
||||
Receiving objects: 99% (266/268), 13.11 MiB | 3.72 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Receiving objects: 99% (267/268), 15.03 MiB | 3.73 MiB/s
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
remote: Total 268 (delta 0), reused 0 (delta 0), pack-reused 268[K
|
||||
Receiving objects: 100% (268/268), 15.25 MiB | 3.29 MiB/s
|
||||
Receiving objects: 100% (268/268), 15.34 MiB | 3.30 MiB/s, done.
|
||||
Receiving objects: 100% (268/268), 15.03 MiB | 3.73 MiB/s
|
||||
Receiving objects: 100% (268/268), 15.34 MiB | 3.73 MiB/s, done.
|
||||
Resolving deltas: 0% (0/117)
|
||||
Resolving deltas: 1% (2/117)
|
||||
Resolving deltas: 2% (3/117)
|
||||
Resolving deltas: 3% (4/117)
|
||||
Resolving deltas: 5% (6/117)
|
||||
Resolving deltas: 7% (9/117)
|
||||
Resolving deltas: 8% (10/117)
|
||||
|
|
@ -254,15 +266,15 @@ Resolving deltas: 9% (11/117)
|
|||
Resolving deltas: 10% (12/117)
|
||||
Resolving deltas: 11% (14/117)
|
||||
Resolving deltas: 13% (16/117)
|
||||
Resolving deltas: 14% (17/117)
|
||||
Resolving deltas: 15% (18/117)
|
||||
Resolving deltas: 26% (31/117)
|
||||
Resolving deltas: 32% (38/117)
|
||||
Resolving deltas: 33% (39/117)
|
||||
Resolving deltas: 54% (64/117)
|
||||
Resolving deltas: 56% (66/117)
|
||||
Resolving deltas: 58% (69/117)
|
||||
Resolving deltas: 73% (86/117)
|
||||
Resolving deltas: 76% (89/117)
|
||||
Resolving deltas: 77% (91/117)
|
||||
Resolving deltas: 76% (90/117)
|
||||
Resolving deltas: 80% (94/117)
|
||||
Resolving deltas: 81% (95/117)
|
||||
Resolving deltas: 82% (96/117)
|
||||
Resolving deltas: 100% (117/117)
|
||||
Resolving deltas: 100% (117/117), done.
|
||||
|
||||
|
|
@ -281,9 +293,9 @@ Resolving deltas: 100% (117/117), done.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: torch in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 1)) (2.1.0+cpu)
|
||||
Requirement already satisfied: click in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 2)) (8.1.7)
|
||||
Requirement already satisfied: numpy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 3)) (1.23.5)
|
||||
Requirement already satisfied: torch in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 1)) (2.1.0+cpu)
|
||||
Requirement already satisfied: click in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 2)) (8.1.7)
|
||||
Requirement already satisfied: numpy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 3)) (1.23.5)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -296,9 +308,9 @@ Resolving deltas: 100% (117/117), done.
|
|||
|
||||
Collecting python-hostlist (from -r segmenter/requirements.txt (line 5))
|
||||
Using cached python_hostlist-1.23.0-py3-none-any.whl
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 6)) (4.66.1)
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 7)) (2.31.0)
|
||||
Requirement already satisfied: pyyaml in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 8)) (6.0.1)
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 6)) (4.66.1)
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 7)) (2.31.0)
|
||||
Requirement already satisfied: pyyaml in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from -r segmenter/requirements.txt (line 8)) (6.0.1)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -316,49 +328,57 @@ Resolving deltas: 100% (117/117), done.
|
|||
.. parsed-literal::
|
||||
|
||||
Collecting mmsegmentation==0.14.1 (from -r segmenter/requirements.txt (line 11))
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Using cached mmsegmentation-0.14.1-py3-none-any.whl (201 kB)
|
||||
Requirement already satisfied: torchvision in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from timm==0.4.12->-r segmenter/requirements.txt (line 9)) (0.16.0+cpu)
|
||||
Requirement already satisfied: addict in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (2.4.0)
|
||||
Requirement already satisfied: Pillow in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (10.2.0)
|
||||
Requirement already satisfied: torchvision in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from timm==0.4.12->-r segmenter/requirements.txt (line 9)) (0.16.0+cpu)
|
||||
Requirement already satisfied: addict in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (2.4.0)
|
||||
Requirement already satisfied: Pillow in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (10.2.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Collecting yapf (from mmcv==1.3.8->-r segmenter/requirements.txt (line 10))
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Using cached yapf-0.40.2-py3-none-any.whl.metadata (45 kB)
|
||||
Requirement already satisfied: matplotlib in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.7.4)
|
||||
Requirement already satisfied: prettytable in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.9.0)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.13.1)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (4.9.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1.3)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (2023.10.0)
|
||||
Requirement already satisfied: matplotlib in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.7.4)
|
||||
Requirement already satisfied: prettytable in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.9.0)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.13.1)
|
||||
Requirement already satisfied: typing-extensions in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (4.9.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (3.1.3)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch->-r segmenter/requirements.txt (line 1)) (2023.10.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2.1.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2023.11.17)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch->-r segmenter/requirements.txt (line 1)) (2.1.4)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (3.6)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2.2.0)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->-r segmenter/requirements.txt (line 7)) (2024.2.2)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch->-r segmenter/requirements.txt (line 1)) (2.1.5)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: contourpy>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.1.1)
|
||||
Requirement already satisfied: cycler>=0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.12.1)
|
||||
Requirement already satisfied: fonttools>=4.22.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (4.47.2)
|
||||
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.4.5)
|
||||
Requirement already satisfied: packaging>=20.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (23.2)
|
||||
Requirement already satisfied: pyparsing>=2.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.1.1)
|
||||
Requirement already satisfied: python-dateutil>=2.7 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (2.8.2)
|
||||
Requirement already satisfied: importlib-resources>=3.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (6.1.1)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.2.13)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch->-r segmenter/requirements.txt (line 1)) (1.3.0)
|
||||
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (7.0.1)
|
||||
Requirement already satisfied: platformdirs>=3.5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (4.1.0)
|
||||
Requirement already satisfied: contourpy>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.1.1)
|
||||
Requirement already satisfied: cycler>=0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.12.1)
|
||||
Requirement already satisfied: fonttools>=4.22.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (4.48.1)
|
||||
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.4.5)
|
||||
Requirement already satisfied: packaging>=20.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (23.2)
|
||||
Requirement already satisfied: pyparsing>=2.3.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (3.1.1)
|
||||
Requirement already satisfied: python-dateutil>=2.7 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (2.8.2)
|
||||
Requirement already satisfied: importlib-resources>=3.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (6.1.1)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (0.2.13)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch->-r segmenter/requirements.txt (line 1)) (1.3.0)
|
||||
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (7.0.1)
|
||||
Requirement already satisfied: platformdirs>=3.5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (4.2.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -369,8 +389,12 @@ Resolving deltas: 100% (117/117), done.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: zipp>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from importlib-metadata>=6.6.0->yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (3.17.0)
|
||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.16.0)
|
||||
Requirement already satisfied: zipp>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from importlib-metadata>=6.6.0->yapf->mmcv==1.3.8->-r segmenter/requirements.txt (line 10)) (3.17.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: six>=1.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.14.1->-r segmenter/requirements.txt (line 11)) (1.16.0)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -408,7 +432,7 @@ Resolving deltas: 100% (117/117), done.
|
|||
|
||||
import numpy as np
|
||||
import yaml
|
||||
|
||||
|
||||
# Fetch the notebook utils script from the openvino_notebooks repo
|
||||
import urllib.request
|
||||
urllib.request.urlretrieve(
|
||||
|
|
@ -429,13 +453,13 @@ config for our model.
|
|||
# here we use tiny model, there are also better but larger models available in repository
|
||||
WEIGHTS_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/checkpoint.pth"
|
||||
CONFIG_LINK = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/segmenter/checkpoints/ade20k/seg_tiny_mask/variant.yml"
|
||||
|
||||
|
||||
MODEL_DIR = Path("model/")
|
||||
MODEL_DIR.mkdir(exist_ok=True)
|
||||
|
||||
|
||||
download_file(WEIGHTS_LINK, directory=MODEL_DIR, show_progress=True)
|
||||
download_file(CONFIG_LINK, directory=MODEL_DIR, show_progress=True)
|
||||
|
||||
|
||||
WEIGHT_PATH = MODEL_DIR / "checkpoint.pth"
|
||||
CONFIG_PATH = MODEL_DIR / "variant.yaml"
|
||||
|
||||
|
|
@ -473,7 +497,7 @@ initialize the model.
|
|||
.. code:: ipython3
|
||||
|
||||
from segmenter.segm.model.factory import load_model
|
||||
|
||||
|
||||
pytorch_model, config = load_model(WEIGHT_PATH)
|
||||
# put model into eval mode, to set it for inference
|
||||
pytorch_model.eval()
|
||||
|
|
@ -536,12 +560,12 @@ normalized with given mean and standard deviation provided in
|
|||
from PIL import Image
|
||||
import torch
|
||||
import torchvision.transforms.functional as F
|
||||
|
||||
|
||||
|
||||
|
||||
def preprocess(im: Image, normalization: dict) -> torch.Tensor:
|
||||
"""
|
||||
Preprocess image: scale, normalize and unsqueeze
|
||||
|
||||
|
||||
:param im: input image
|
||||
:param normalization: dictionary containing normalization data from config file
|
||||
:return:
|
||||
|
|
@ -553,7 +577,7 @@ normalized with given mean and standard deviation provided in
|
|||
im = F.normalize(im, normalization["mean"], normalization["std"])
|
||||
# change dim from [C, H, W] to [1, C, H, W]
|
||||
im = im.unsqueeze(0)
|
||||
|
||||
|
||||
return im
|
||||
|
||||
Visualization
|
||||
|
|
@ -577,29 +601,29 @@ corresponding to the inferred labels.
|
|||
|
||||
from segmenter.segm.data.utils import dataset_cat_description, seg_to_rgb
|
||||
from segmenter.segm.data.ade20k import ADE20K_CATS_PATH
|
||||
|
||||
|
||||
|
||||
|
||||
def apply_segmentation_mask(pil_im: Image, results: torch.Tensor) -> Image:
|
||||
"""
|
||||
Combine segmentation masks with the image
|
||||
|
||||
|
||||
:param pil_im: original input image
|
||||
:param results: tensor containing segmentation masks for each pixel
|
||||
:return:
|
||||
pil_blend: image with colored segmentation masks overlay
|
||||
"""
|
||||
cat_names, cat_colors = dataset_cat_description(ADE20K_CATS_PATH)
|
||||
|
||||
|
||||
# 3D array, where each pixel has values for all classes, take index of max as label
|
||||
seg_map = results.argmax(0, keepdim=True)
|
||||
# transform label id to colors
|
||||
seg_rgb = seg_to_rgb(seg_map, cat_colors)
|
||||
seg_rgb = (255 * seg_rgb.cpu().numpy()).astype(np.uint8)
|
||||
pil_seg = Image.fromarray(seg_rgb[0])
|
||||
|
||||
|
||||
# overlay segmentation mask over original image
|
||||
pil_blend = Image.blend(pil_im, pil_seg, 0.5).convert("RGB")
|
||||
|
||||
|
||||
return pil_blend
|
||||
|
||||
Validation of inference of original model
|
||||
|
|
@ -613,15 +637,15 @@ example image ``coco_hollywood.jpg``.
|
|||
.. code:: ipython3
|
||||
|
||||
from segmenter.segm.model.utils import inference
|
||||
|
||||
|
||||
# load image with PIL
|
||||
image = load_image("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco_hollywood.jpg")
|
||||
# load_image reads the image in BGR format, [:,:,::-1] reshape transfroms it to RGB
|
||||
pil_image = Image.fromarray(image[:,:,::-1])
|
||||
|
||||
|
||||
# preprocess image with normalization params loaded in previous steps
|
||||
image = preprocess(pil_image, normalization)
|
||||
|
||||
|
||||
# inference function needs some meta parameters, where we specify that we don't flip images in inference mode
|
||||
im_meta = dict(flip=False)
|
||||
# perform inference with function from repository
|
||||
|
|
@ -641,7 +665,7 @@ previous steps.
|
|||
|
||||
# combine segmentation mask with image
|
||||
blended_image = apply_segmentation_mask(pil_image, original_results)
|
||||
|
||||
|
||||
# show image with segmentation mask overlay
|
||||
blended_image
|
||||
|
||||
|
|
@ -688,15 +712,15 @@ they are not a problem.
|
|||
.. code:: ipython3
|
||||
|
||||
import openvino as ov
|
||||
|
||||
|
||||
# get input sizes from config file
|
||||
batch_size = 2
|
||||
channels = 3
|
||||
image_size = config["dataset_kwargs"]["image_size"]
|
||||
|
||||
|
||||
# make dummy input with correct shapes obtained from config file
|
||||
dummy_input = torch.randn(batch_size, channels, image_size, image_size)
|
||||
|
||||
|
||||
model = ov.convert_model(pytorch_model, example_input=dummy_input, input=([batch_size, channels, image_size, image_size], ))
|
||||
# serialize model for saving IR
|
||||
ov.save_model(model, MODEL_DIR / "segmenter.xml")
|
||||
|
|
@ -704,21 +728,21 @@ they are not a problem.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if H % patch_size > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if W % patch_size > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/vit.py:122: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/vit.py:122: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x.shape[1] != pos_embed.shape[1]:
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/decoder.py:100: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/decoder.py:100: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
masks = rearrange(masks, "b (h w) n -> b n h w", h=int(GS))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if extra_h > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/notebooks/204-segmenter-semantic-segmentation/./segmenter/segm/model/utils.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if extra_w > 0:
|
||||
|
||||
|
||||
|
|
@ -739,7 +763,7 @@ any additional custom code required to process input.
|
|||
class SegmenterOV:
|
||||
"""
|
||||
Class containing OpenVINO model with all attributes required to work with inference function.
|
||||
|
||||
|
||||
:param model: compiled OpenVINO model
|
||||
:type model: CompiledModel
|
||||
:param output_blob: output blob used in inference
|
||||
|
|
@ -750,14 +774,14 @@ any additional custom code required to process input.
|
|||
:type n_cls: int
|
||||
:param normalization:
|
||||
:type normalization: dict
|
||||
|
||||
|
||||
"""
|
||||
|
||||
|
||||
def __init__(self, model_path: Path, device:str = "CPU"):
|
||||
"""
|
||||
Constructor method.
|
||||
Initializes OpenVINO model and sets all required attributes
|
||||
|
||||
|
||||
:param model_path: path to model's .xml file, also containing variant.yml
|
||||
:param device: device string for selecting inference device
|
||||
"""
|
||||
|
|
@ -767,23 +791,23 @@ any additional custom code required to process input.
|
|||
model_xml = core.read_model(model_path)
|
||||
self.model = core.compile_model(model_xml, device)
|
||||
self.output_blob = self.model.output(0)
|
||||
|
||||
|
||||
# load model configs
|
||||
variant_path = Path(model_path).parent / "variant.yml"
|
||||
with open(variant_path, "r") as f:
|
||||
self.config = yaml.load(f, Loader=yaml.FullLoader)
|
||||
|
||||
|
||||
# load normalization specs from config
|
||||
normalization_name = self.config["dataset_kwargs"]["normalization"]
|
||||
self.normalization = STATS[normalization_name]
|
||||
|
||||
|
||||
# load number of classes from config
|
||||
self.n_cls = self.config["net_kwargs"]["n_cls"]
|
||||
|
||||
|
||||
def forward(self, data: torch.Tensor) -> torch.Tensor:
|
||||
"""
|
||||
Perform inference on data and return the result in Tensor format
|
||||
|
||||
|
||||
:param data: input data to model
|
||||
:return: data inferred by model
|
||||
"""
|
||||
|
|
@ -802,7 +826,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
|
|
@ -810,7 +834,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
description='Device:',
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -842,7 +866,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
# combine segmentation mask with image
|
||||
converted_blend = apply_segmentation_mask(pil_image, results)
|
||||
|
||||
|
||||
# show image with segmentation mask overlay
|
||||
converted_blend
|
||||
|
||||
|
|
@ -904,17 +928,17 @@ to measure the inference performance of the model.
|
|||
[ WARNING ] Default duration 120 seconds is used for unknown device AUTO
|
||||
[ INFO ] OpenVINO:
|
||||
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ] Device info:
|
||||
[ INFO ] AUTO
|
||||
[ INFO ] Build ................................. 2023.3.0-13775-ceeafaf64f3-releases/2023/3
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[Step 3/11] Setting device configuration
|
||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 22.64 ms
|
||||
[ INFO ] Read model took 23.09 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] im (node: im) : f32 / [...] / [2,3,512,512]
|
||||
|
|
@ -932,7 +956,7 @@ to measure the inference performance of the model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Compile model took 392.81 ms
|
||||
[ INFO ] Compile model took 385.39 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -940,10 +964,6 @@ to measure the inference performance of the model.
|
|||
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
|
||||
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
|
||||
[ INFO ] MULTI_DEVICE_PRIORITIES: CPU
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] CPU:
|
||||
[ INFO ] AFFINITY: Affinity.CORE
|
||||
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
|
||||
|
|
@ -965,30 +985,30 @@ to measure the inference performance of the model.
|
|||
[ INFO ] LOADED_FROM_CACHE: False
|
||||
[Step 9/11] Creating infer requests and preparing input tensors
|
||||
[ WARNING ] No input files were given for input 'im'!. This input will be filled with random values!
|
||||
[ INFO ] Fill input 'im' with random values
|
||||
[ INFO ] Fill input 'im' with random values
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 120000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] First inference took 203.86 ms
|
||||
[ INFO ] First inference took 210.45 ms
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[Step 11/11] Dumping statistics report
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 1704 iterations
|
||||
[ INFO ] Duration: 120454.83 ms
|
||||
[ INFO ] Count: 1686 iterations
|
||||
[ INFO ] Duration: 120531.12 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 423.45 ms
|
||||
[ INFO ] Average: 423.78 ms
|
||||
[ INFO ] Min: 358.59 ms
|
||||
[ INFO ] Max: 506.30 ms
|
||||
[ INFO ] Throughput: 28.29 FPS
|
||||
[ INFO ] Median: 429.25 ms
|
||||
[ INFO ] Average: 428.34 ms
|
||||
[ INFO ] Min: 354.96 ms
|
||||
[ INFO ] Max: 506.55 ms
|
||||
[ INFO ] Throughput: 27.98 FPS
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/204-segmenter-semantic-segmentation-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/204-segmenter-semantic-segmentation-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="204-segmenter-semantic-segmentation-with-output_21_0.jpg">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 72352
|
||||
<a href="204-segmenter-semantic-segmentation-with-output_21_0.png">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 909669
|
||||
<a href="204-segmenter-semantic-segmentation-with-output_32_0.jpg">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 72372
|
||||
<a href="204-segmenter-semantic-segmentation-with-output_32_0.png">204-segmenter-semantic-segmentation-with-output..></a> 26-Jan-2024 01:05 909654
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -24,8 +24,7 @@ Table of contents:
|
|||
- `Settings <#settings>`__
|
||||
- `Load the U2-Net Model <#load-the-u2-net-model>`__
|
||||
|
||||
- `Convert PyTorch U2-Net model to OpenVINO
|
||||
IR <#convert-pytorch-u2-net-model-to-openvino-ir>`__
|
||||
- `Convert PyTorch U2-Net model to OpenVINO IR <#convert-pytorch-u2-net-model-to-openvino-ir>`__
|
||||
- `Load and Pre-Process Input
|
||||
Image <#load-and-pre-process-input-image>`__
|
||||
- `Select inference device <#select-inference-device>`__
|
||||
|
|
@ -190,52 +189,52 @@ next cell loads the model and the pre-trained weights.
|
|||
.. parsed-literal::
|
||||
|
||||
|
||||
11%|█ | 524k/4.68M [00:00<00:01, 3.36MB/s]
|
||||
11%|█ | 524k/4.68M [00:00<00:01, 3.35MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
22%|██▏ | 1.05M/4.68M [00:00<00:00, 3.74MB/s]
|
||||
22%|██▏ | 1.05M/4.68M [00:00<00:00, 3.71MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
34%|███▎ | 1.57M/4.68M [00:00<00:00, 3.84MB/s]
|
||||
34%|███▎ | 1.57M/4.68M [00:00<00:00, 3.78MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
45%|████▍ | 2.10M/4.68M [00:00<00:00, 3.89MB/s]
|
||||
45%|████▍ | 2.10M/4.68M [00:00<00:00, 3.88MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
56%|█████▌ | 2.62M/4.68M [00:00<00:00, 3.92MB/s]
|
||||
56%|█████▌ | 2.62M/4.68M [00:00<00:00, 3.96MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
67%|██████▋ | 3.15M/4.68M [00:00<00:00, 3.94MB/s]
|
||||
67%|██████▋ | 3.15M/4.68M [00:00<00:00, 3.95MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
78%|███████▊ | 3.67M/4.68M [00:00<00:00, 3.95MB/s]
|
||||
78%|███████▊ | 3.67M/4.68M [00:00<00:00, 3.97MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
90%|████████▉ | 4.19M/4.68M [00:01<00:00, 3.96MB/s]
|
||||
90%|████████▉ | 4.19M/4.68M [00:01<00:00, 4.00MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 4.05MB/s]
|
||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 4.15MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 3.93MB/s]
|
||||
100%|██████████| 4.68M/4.68M [00:01<00:00, 3.96MB/s]
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
|
|
@ -282,7 +281,7 @@ OpenVINO IR format. Executing the following command may take a while.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-598/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/functional.py:3769: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-609/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/functional.py:3769: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
|
||||
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
|
||||
|
||||
|
||||
|
|
@ -382,7 +381,7 @@ Load the OpenVINO IR model to OpenVINO Runtime and do inference.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Inference finished. Inference time: 0.113 seconds, FPS: 8.89.
|
||||
Inference finished. Inference time: 0.110 seconds, FPS: 9.05.
|
||||
|
||||
|
||||
Visualize Results
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/205-vision-background-removal-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/205-vision-background-removal-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="205-vision-background-removal-with-output_22_0.png">205-vision-background-removal-with-output_22_0.png</a> 26-Jan-2024 01:05 279572
|
||||
<a href="205-vision-background-removal-with-output_24_0.png">205-vision-background-removal-with-output_24_0.png</a> 26-Jan-2024 01:05 927043
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -204,7 +204,7 @@ source of the function.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[01/25 23:27:13] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/animeganv2_hayao.pdparams
|
||||
[02/09 23:41:27] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/animeganv2_hayao.pdparams
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -345,7 +345,7 @@ succeeds, the output of the next cell will include
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:27:21 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||
2024-02-09 23:41:36 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -356,12 +356,12 @@ succeeds, the output of the next cell will include
|
|||
[Paddle2ONNX] Start to parsing Paddle model...
|
||||
[Paddle2ONNX] Use opset_version = 11 for ONNX export.
|
||||
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
||||
2024-01-25 23:27:21 [INFO] ONNX model saved in model/paddlegan_anime.onnx.
|
||||
2024-02-09 23:41:36 [INFO] ONNX model saved in model/paddlegan_anime.onnx.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
I0125 23:27:21.862668 812000 program_interpreter.cc:212] New Executor is Running.
|
||||
I0209 23:41:36.202327 2843665 program_interpreter.cc:212] New Executor is Running.
|
||||
|
||||
|
||||
Convert to OpenVINO IR
|
||||
|
|
@ -643,12 +643,12 @@ measure inference on one image. For more accurate benchmarking, use
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.418 seconds per image, FPS: 2.39
|
||||
OpenVINO IR model in OpenVINO Runtime/CPU: 0.427 seconds per image, FPS: 2.34
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
PaddleGAN model on CPU: 6.256 seconds per image, FPS: 0.16
|
||||
PaddleGAN model on CPU: 6.182 seconds per image, FPS: 0.16
|
||||
|
||||
|
||||
References
|
||||
|
|
|
|||
|
|
@ -1,8 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/206-vision-paddlegan-anime-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/206-vision-paddlegan-anime-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="206-vision-paddlegan-anime-with-output_15_0.png">206-vision-paddlegan-anime-with-output_15_0.png</a> 26-Jan-2024 01:05 1810982
|
||||
<a href="206-vision-paddlegan-anime-with-output_37_0.png">206-vision-paddlegan-anime-with-output_37_0.png</a> 26-Jan-2024 01:05 1931637
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -161,7 +161,7 @@ source code.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[01/25 23:27:55] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/DF2K_JPEG.pdparams
|
||||
[02/09 23:42:09] ppgan INFO: Found /opt/home/k8sworker/.cache/ppgan/DF2K_JPEG.pdparams
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -289,12 +289,12 @@ Convert PaddlePaddle Model to ONNX
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:28:02 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||
2024-02-09 23:42:16 [INFO] Static PaddlePaddle model saved in model/paddle_model_static_onnx_temp_dir.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
I0125 23:28:01.927830 812342 program_interpreter.cc:212] New Executor is Running.
|
||||
I0209 23:42:16.006111 2843987 program_interpreter.cc:212] New Executor is Running.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -303,13 +303,13 @@ Convert PaddlePaddle Model to ONNX
|
|||
[Paddle2ONNX] Model file path: model/paddle_model_static_onnx_temp_dir/model.pdmodel
|
||||
[Paddle2ONNX] Paramters file path: model/paddle_model_static_onnx_temp_dir/model.pdiparams
|
||||
[Paddle2ONNX] Start to parsing Paddle model...
|
||||
[Paddle2ONNX] Use opset_version = 13 for ONNX export.
|
||||
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-01-25 23:28:05 [INFO] ONNX model saved in model/paddlegan_sr.onnx.
|
||||
[Paddle2ONNX] Use opset_version = 13 for ONNX export.
|
||||
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
|
||||
2024-02-09 23:42:19 [INFO] ONNX model saved in model/paddlegan_sr.onnx.
|
||||
|
||||
|
||||
Convert ONNX Model to OpenVINO IR with `Model Conversion Python API <https://docs.openvino.ai/2023.3/openvino_docs_model_processing_introduction.html>`__
|
||||
|
|
@ -394,7 +394,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7ff10225de20>
|
||||
<matplotlib.image.AxesImage at 0x7fe9ebed32e0>
|
||||
|
||||
|
||||
|
||||
|
|
@ -423,7 +423,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Inference duration: 3.26 seconds
|
||||
Inference duration: 3.27 seconds
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -446,7 +446,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7ff0cc1894f0>
|
||||
<matplotlib.image.AxesImage at 0x7fe9b40e3580>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/207-vision-paddlegan-superresolution-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/207-vision-paddlegan-superresolution-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="207-vision-paddlegan-superresolution-with-output_26_1.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 436999
|
||||
<a href="207-vision-paddlegan-superresolution-with-output_30_1.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 476190
|
||||
<a href="207-vision-paddlegan-superresolution-with-output_32_0.png">207-vision-paddlegan-superresolution-with-outpu..></a> 26-Jan-2024 01:04 2835305
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,20 +0,0 @@
|
|||
<html>
|
||||
<head><title>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/208-optical-character-recognition-with-output_files/</title></head>
|
||||
<body bgcolor="white">
|
||||
<h1>Index of /projects/ov-notebook/0.1.0-latest/20240125220808/dist/rst_files/208-optical-character-recognition-with-output_files/</h1><hr><pre><a href="../">../</a>
|
||||
<a href="208-optical-character-recognition-with-output_16_0.png">208-optical-character-recognition-with-output_1..></a> 26-Jan-2024 01:05 305482
|
||||
<a href="208-optical-character-recognition-with-output_26_0.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 923631
|
||||
<a href="208-optical-character-recognition-with-output_28_0.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1996
|
||||
<a href="208-optical-character-recognition-with-output_28_0.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 11367
|
||||
<a href="208-optical-character-recognition-with-output_28_10.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1990
|
||||
<a href="208-optical-character-recognition-with-output_28_10.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 11142
|
||||
<a href="208-optical-character-recognition-with-output_28_2.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1630
|
||||
<a href="208-optical-character-recognition-with-output_28_2.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 8428
|
||||
<a href="208-optical-character-recognition-with-output_28_4.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 949
|
||||
<a href="208-optical-character-recognition-with-output_28_4.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 2274
|
||||
<a href="208-optical-character-recognition-with-output_28_6.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 817
|
||||
<a href="208-optical-character-recognition-with-output_28_6.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1559
|
||||
<a href="208-optical-character-recognition-with-output_28_8.jpg">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 838
|
||||
<a href="208-optical-character-recognition-with-output_28_8.png">208-optical-character-recognition-with-output_2..></a> 26-Jan-2024 01:05 1487
|
||||
</pre><hr></body>
|
||||
</html>
|
||||
|
|
@ -8,9 +8,9 @@ Latin alphabet is available in `notebook
|
|||
This model is capable of processing only one line of symbols at a time.
|
||||
|
||||
The models used in this notebook are
|
||||
`handwritten-japanese-recognition-0001 <https://docs.openvino.ai/2023.0/omz_models_model_handwritten_japanese_recognition_0001.html>`__
|
||||
`handwritten-japanese-recognition-0001 <https://docs.openvino.ai/2023.3/omz_models_model_handwritten_japanese_recognition_0001.html>`__
|
||||
and
|
||||
`handwritten-simplified-chinese-0001 <https://docs.openvino.ai/2023.0/omz_models_model_handwritten_simplified_chinese_recognition_0001.html>`__.
|
||||
`handwritten-simplified-chinese-0001 <https://docs.openvino.ai/2023.3/omz_models_model_handwritten_simplified_chinese_recognition_0001.html>`__.
|
||||
To decode model outputs as readable text
|
||||
`kondate_nakayosi <https://github.com/openvinotoolkit/open_model_zoo/blob/master/data/dataset_classes/kondate_nakayosi.txt>`__
|
||||
and
|
||||
|
|
@ -62,12 +62,12 @@ Imports
|
|||
|
||||
from collections import namedtuple
|
||||
from itertools import groupby
|
||||
|
||||
|
||||
import cv2
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import openvino as ov
|
||||
|
||||
|
||||
# Fetch `notebook_utils` module
|
||||
import urllib.request
|
||||
urllib.request.urlretrieve(
|
||||
|
|
@ -89,7 +89,7 @@ Set up all constants and folders used in this notebook
|
|||
base_models_dir = "models"
|
||||
data_folder = "data"
|
||||
charlist_folder = f"{data_folder}/text"
|
||||
|
||||
|
||||
# Precision used by the model.
|
||||
precision = "FP16"
|
||||
|
||||
|
|
@ -127,9 +127,9 @@ If you want to perform OCR on a text in Japanese, set
|
|||
|
||||
# Select the language by using either language="chinese" or language="japanese".
|
||||
language = "chinese"
|
||||
|
||||
|
||||
languages = {"chinese": chinese_files, "japanese": japanese_files}
|
||||
|
||||
|
||||
selected_language = languages.get(language)
|
||||
|
||||
Download the Model
|
||||
|
|
@ -196,14 +196,14 @@ select device from dropdown list for running inference using OpenVINO
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
value='AUTO',
|
||||
description='Device:',
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -252,32 +252,32 @@ keep letters proportional and meet input shape.
|
|||
"https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/" + selected_language.demo_image_name,
|
||||
directory=data_folder
|
||||
)
|
||||
|
||||
|
||||
# Text detection models expect an image in grayscale format.
|
||||
# IMPORTANT! This model enables reading only one line at time.
|
||||
|
||||
|
||||
# Read the image.
|
||||
image = cv2.imread(filename=str(file_name), flags=cv2.IMREAD_GRAYSCALE)
|
||||
|
||||
|
||||
# Fetch the shape.
|
||||
image_height, _ = image.shape
|
||||
|
||||
|
||||
# B,C,H,W = batch size, number of channels, height, width.
|
||||
_, _, H, W = recognition_input_layer.shape
|
||||
|
||||
|
||||
# Calculate scale ratio between the input shape height and image height to resize the image.
|
||||
scale_ratio = H / image_height
|
||||
|
||||
|
||||
# Resize the image to expected input sizes.
|
||||
resized_image = cv2.resize(
|
||||
image, None, fx=scale_ratio, fy=scale_ratio, interpolation=cv2.INTER_AREA
|
||||
)
|
||||
|
||||
|
||||
# Pad the image to match input size, without changing aspect ratio.
|
||||
resized_image = np.pad(
|
||||
resized_image, ((0, 0), (0, W - resized_image.shape[1])), mode="edge"
|
||||
)
|
||||
|
||||
|
||||
# Reshape to network input shape.
|
||||
input_image = resized_image[None, None, :, :]
|
||||
|
||||
|
|
@ -335,10 +335,10 @@ Chinese and Japanese models.
|
|||
|
||||
# Get a dictionary to encode the output, based on model documentation.
|
||||
used_charlist = selected_language.charlist_name
|
||||
|
||||
|
||||
# With both models, there should be blank symbol added at index 0 of each charlist.
|
||||
blank_char = "~"
|
||||
|
||||
|
||||
with used_charlist_file.open(mode="r", encoding="utf-8") as charlist:
|
||||
letters = blank_char + "".join(line.strip() for line in charlist)
|
||||
|
||||
|
|
@ -380,7 +380,7 @@ Finally, get the symbols from corresponding indexes in the charlist.
|
|||
|
||||
# Remove a batch dimension.
|
||||
predictions = np.squeeze(predictions)
|
||||
|
||||
|
||||
# Run the `argmax` function to pick the symbols with the highest probability.
|
||||
predictions_indexes = np.argmax(predictions, axis=1)
|
||||
|
||||
|
|
@ -388,13 +388,13 @@ Finally, get the symbols from corresponding indexes in the charlist.
|
|||
|
||||
# Use the `groupby` function to remove concurrent letters, as required by CTC greedy decoding.
|
||||
output_text_indexes = list(groupby(predictions_indexes))
|
||||
|
||||
|
||||
# Remove grouper objects.
|
||||
output_text_indexes, _ = np.transpose(output_text_indexes, (1, 0))
|
||||
|
||||
|
||||
# Remove blank symbols.
|
||||
output_text_indexes = output_text_indexes[output_text_indexes != 0]
|
||||
|
||||
|
||||
# Assign letters to indexes from the output array.
|
||||
output_text = [letters[letter_index] for letter_index in output_text_indexes]
|
||||
|
||||
|
|
@ -411,7 +411,7 @@ the image with predicted text printed below.
|
|||
plt.figure(figsize=(20, 1))
|
||||
plt.axis("off")
|
||||
plt.imshow(resized_image, cmap="gray", vmin=0, vmax=255)
|
||||
|
||||
|
||||
print("".join(output_text))
|
||||
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue