Compare commits
6 Commits
master
...
itikhono/p
| Author | SHA1 | Date |
|---|---|---|
Ivan Tikhonov
|
c5bd13a63c | |
|
Sebastian Golebiewski
|
65c3b1783c | |
|
Zlobin Vladimir
|
457b0dc288 | |
|
Sergey Lyalin
|
cff550d5ac | |
|
Alina Kladieva
|
5fd6bbbc07 | |
|
Tatiana Savina
|
67271cd1ef |
|
|
@ -56,7 +56,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: ${{ env.OPENVINO_CONTRIB_REPO }}
|
||||
submodules: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Dependencies
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@ jobs:
|
|||
with:
|
||||
repository: 'openvinotoolkit/openvino_tokenizers'
|
||||
path: ${{ env.OPENVINO_TOKENIZERS_REPO }}
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
- name: Download OpenVINO package
|
||||
uses: actions/download-artifact@v4
|
||||
|
|
|
|||
|
|
@ -129,7 +129,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: ${{ env.OPENVINO_CONTRIB_REPO }}
|
||||
submodules: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
@ -632,7 +632,7 @@ jobs:
|
|||
with:
|
||||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: ${{ env.OPENVINO_CONTRIB_REPO }}
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Dependencies
|
||||
|
|
|
|||
|
|
@ -117,7 +117,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: ${{ env.OPENVINO_CONTRIB_REPO }}
|
||||
submodules: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
|
|||
|
|
@ -115,7 +115,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/testdata'
|
||||
path: ${{ env.MODELS_PATH }}
|
||||
lfs: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
@ -287,7 +287,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/testdata'
|
||||
path: ${{ env.MODELS_PATH }}
|
||||
lfs: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
- name: Download selective build statistics package
|
||||
uses: actions/download-artifact@v4
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: ${{ env.OPENVINO_CONTRIB_REPO }}
|
||||
submodules: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ jobs:
|
|||
with:
|
||||
repository: 'openvinotoolkit/openvino_contrib'
|
||||
path: 'openvino_contrib'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
|
|||
|
|
@ -81,7 +81,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/testdata'
|
||||
path: 'testdata'
|
||||
lfs: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
#
|
||||
# Print system info
|
||||
|
|
@ -284,7 +284,7 @@ jobs:
|
|||
repository: 'openvinotoolkit/testdata'
|
||||
path: 'testdata'
|
||||
lfs: 'true'
|
||||
ref: 'master'
|
||||
ref: 'releases/2024/2'
|
||||
|
||||
- name: Download selective build statistics package
|
||||
uses: actions/download-artifact@v4
|
||||
|
|
|
|||
|
|
@ -1,2 +0,0 @@
|
|||
|
||||
> **NOTE**: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.
|
||||
|
|
@ -1,8 +1,5 @@
|
|||
# OpenVINO™ Development Tools
|
||||
|
||||
<!--- The note below is intended for master branch only for pre-release purpose. Remove it for official releases. --->
|
||||
> **NOTE**: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.
|
||||
|
||||
> **NOTE**: OpenVINO™ Development Tools package has been deprecated and will be discontinued with 2025.0 release. To learn more, refer to the [OpenVINO Legacy Features and Components page](https://docs.openvino.ai/2024/documentation/legacy-features.html).
|
||||
|
||||
Intel® Distribution of OpenVINO™ toolkit is an open-source toolkit for optimizing and deploying AI inference. It can be used to develop applications and solutions based on deep learning tasks, such as: emulation of human vision, automatic speech recognition, natural language processing, recommendation systems, etc. It provides high-performance and rich deployment options, from edge to cloud.
|
||||
|
|
|
|||
|
|
@ -1,8 +1,5 @@
|
|||
# OpenVINO™
|
||||
|
||||
<!--- The note below is intended for master branch only for pre-release purpose. Remove it for official releases. --->
|
||||
> **NOTE**: This version is pre-release software and has not undergone full release validation or qualification. No support is offered on pre-release software and APIs/behavior are subject to change. It should NOT be incorporated into any production software/solution and instead should be used only for early testing and integration while awaiting a final release version of this software.
|
||||
|
||||
Intel® Distribution of OpenVINO™ toolkit is an open-source toolkit for optimizing and deploying AI inference. It can be used to develop applications and solutions based on deep learning tasks, such as: emulation of human vision, automatic speech recognition, natural language processing, recommendation systems, etc. It provides high-performance and rich deployment options, from edge to cloud.
|
||||
|
||||
If you have already finished developing your models and converting them to the OpenVINO model format, you can install OpenVINO Runtime to deploy your applications on various devices. The [OpenVINO™](https://docs.openvino.ai/2024/openvino-workflow/running-inference.html) Python package includes a set of libraries for an easy inference integration with your products.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ repo_directory = "notebooks"
|
|||
repo_owner = "openvinotoolkit"
|
||||
repo_name = "openvino_notebooks"
|
||||
repo_branch = "tree/main"
|
||||
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240506220807/dist/rst_files/"
|
||||
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240515220822/dist/rst_files/"
|
||||
blacklisted_extensions = ['.xml', '.bin']
|
||||
notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/"
|
||||
notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath="
|
||||
|
|
|
|||
|
|
@ -74,61 +74,61 @@ Lab instead.**
|
|||
Using cached https://download.pytorch.org/whl/cpu/torch-2.3.0%2Bcpu-cp38-cp38-linux_x86_64.whl (190.4 MB)
|
||||
Collecting onnx
|
||||
Using cached onnx-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (16 kB)
|
||||
Requirement already satisfied: ipywidgets>=7.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (8.1.2)
|
||||
Requirement already satisfied: ipywidgets>=7.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (8.1.2)
|
||||
Collecting ipydatawidgets>=1.1.1 (from pythreejs)
|
||||
Using cached ipydatawidgets-4.3.5-py2.py3-none-any.whl.metadata (1.4 kB)
|
||||
Collecting numpy (from pythreejs)
|
||||
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)
|
||||
Requirement already satisfied: traitlets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (5.14.3)
|
||||
Requirement already satisfied: defusedxml>=0.7.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (0.7.1)
|
||||
Requirement already satisfied: traitlets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (5.14.3)
|
||||
Requirement already satisfied: defusedxml>=0.7.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (0.7.1)
|
||||
Collecting networkx<=3.1.0 (from openvino-dev>=2024.0.0)
|
||||
Using cached networkx-3.1-py3-none-any.whl.metadata (5.3 kB)
|
||||
Collecting openvino-telemetry>=2023.2.1 (from openvino-dev>=2024.0.0)
|
||||
Using cached openvino_telemetry-2024.1.0-py3-none-any.whl.metadata (2.3 kB)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (24.0)
|
||||
Requirement already satisfied: pyyaml>=5.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (6.0.1)
|
||||
Requirement already satisfied: requests>=2.25.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (2.31.0)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (24.0)
|
||||
Requirement already satisfied: pyyaml>=5.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (6.0.1)
|
||||
Requirement already satisfied: requests>=2.25.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (2.31.0)
|
||||
Collecting openvino==2024.1.0 (from openvino-dev>=2024.0.0)
|
||||
Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl.metadata (8.8 kB)
|
||||
Collecting filelock (from torch)
|
||||
Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (4.11.0)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (4.11.0)
|
||||
Collecting sympy (from torch)
|
||||
Using cached https://download.pytorch.org/whl/sympy-1.12-py3-none-any.whl (5.7 MB)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (3.1.4)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (3.1.4)
|
||||
Collecting fsspec (from torch)
|
||||
Using cached fsspec-2024.3.1-py3-none-any.whl.metadata (6.8 kB)
|
||||
Downloading fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB)
|
||||
Collecting protobuf>=3.20.2 (from onnx)
|
||||
Using cached protobuf-5.26.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
|
||||
Collecting traittypes>=0.2.0 (from ipydatawidgets>=1.1.1->pythreejs)
|
||||
Using cached traittypes-0.2.1-py2.py3-none-any.whl.metadata (1.0 kB)
|
||||
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (0.2.2)
|
||||
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (8.12.3)
|
||||
Requirement already satisfied: widgetsnbextension~=4.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (4.0.10)
|
||||
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (3.0.10)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.7)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2.2.1)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2024.2.2)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch) (2.1.5)
|
||||
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (0.2.2)
|
||||
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (8.12.3)
|
||||
Requirement already satisfied: widgetsnbextension~=4.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (4.0.10)
|
||||
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (3.0.10)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.7)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2.2.1)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2024.2.2)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch) (2.1.5)
|
||||
Collecting mpmath>=0.19 (from sympy->torch)
|
||||
Using cached https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB)
|
||||
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.0)
|
||||
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (5.1.1)
|
||||
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.19.1)
|
||||
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.1.7)
|
||||
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.5)
|
||||
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (3.0.43)
|
||||
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.18.0)
|
||||
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.6.3)
|
||||
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (4.9.0)
|
||||
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.8.4)
|
||||
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.0)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.13)
|
||||
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.0.1)
|
||||
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.4.1)
|
||||
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.2)
|
||||
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (1.16.0)
|
||||
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.0)
|
||||
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (5.1.1)
|
||||
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.19.1)
|
||||
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.1.7)
|
||||
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.5)
|
||||
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (3.0.43)
|
||||
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.18.0)
|
||||
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.6.3)
|
||||
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (4.9.0)
|
||||
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.8.4)
|
||||
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.0)
|
||||
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.13)
|
||||
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.0.1)
|
||||
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.4.1)
|
||||
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.2)
|
||||
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (1.16.0)
|
||||
Using cached pythreejs-2.4.2-py3-none-any.whl (3.4 MB)
|
||||
Using cached openvino_dev-2024.1.0-15008-py3-none-any.whl (4.7 MB)
|
||||
Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl (38.7 MB)
|
||||
|
|
@ -140,10 +140,11 @@ Lab instead.**
|
|||
Using cached openvino_telemetry-2024.1.0-py3-none-any.whl (23 kB)
|
||||
Using cached protobuf-5.26.1-cp37-abi3-manylinux2014_x86_64.whl (302 kB)
|
||||
Using cached filelock-3.14.0-py3-none-any.whl (12 kB)
|
||||
Using cached fsspec-2024.3.1-py3-none-any.whl (171 kB)
|
||||
Downloading fsspec-2024.5.0-py3-none-any.whl (316 kB)
|
||||
[2K ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.1/316.1 kB 2.6 MB/s eta 0:00:00
|
||||
Using cached traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB)
|
||||
Installing collected packages: openvino-telemetry, mpmath, traittypes, sympy, protobuf, numpy, networkx, fsspec, filelock, torch, openvino, opencv-python, onnx, openvino-dev, ipydatawidgets, pythreejs
|
||||
Successfully installed filelock-3.14.0 fsspec-2024.3.1 ipydatawidgets-4.3.5 mpmath-1.3.0 networkx-3.1 numpy-1.24.4 onnx-1.16.0 opencv-python-4.9.0.80 openvino-2024.1.0 openvino-dev-2024.1.0 openvino-telemetry-2024.1.0 protobuf-5.26.1 pythreejs-2.4.2 sympy-1.12 torch-2.3.0+cpu traittypes-0.2.1
|
||||
Successfully installed filelock-3.14.0 fsspec-2024.5.0 ipydatawidgets-4.3.5 mpmath-1.3.0 networkx-3.1 numpy-1.24.4 onnx-1.16.0 opencv-python-4.9.0.80 openvino-2024.1.0 openvino-dev-2024.1.0 openvino-telemetry-2024.1.0 protobuf-5.26.1 pythreejs-2.4.2 sympy-1.12 torch-2.3.0+cpu traittypes-0.2.1
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
|
|
@ -251,18 +252,18 @@ IR format.
|
|||
.. parsed-literal::
|
||||
|
||||
========== Converting human-pose-estimation-3d-0001 to ONNX
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=model/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=model/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx
|
||||
|
||||
ONNX check passed successfully.
|
||||
|
||||
========== Converting human-pose-estimation-3d-0001 to IR (FP32)
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 256, 448]' --compress_to_fp16=False
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 256, 448]' --compress_to_fp16=False
|
||||
|
||||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.bin
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -216,7 +216,7 @@ chair for example.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/tmp/ipykernel_2840820/2434168836.py:12: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
|
||||
/tmp/ipykernel_16799/2434168836.py:12: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
|
||||
ax.scatter3D(X, Y, Z, s=5, cmap="jet", marker="o", label="chair")
|
||||
|
||||
|
||||
|
|
@ -317,7 +317,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/tmp/ipykernel_2840820/2804603389.py:23: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
|
||||
/tmp/ipykernel_16799/2804603389.py:23: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
|
||||
ax.scatter(XCur, YCur, ZCur, s=5, cmap="jet", marker="o", label=classes[i])
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d81ad2656ac29b01ccc926bde66d6d66d05d187c5bf95b2767317a6355ad86fb
|
||||
size 68213
|
||||
oid sha256:1f9f7a4ff050de5ac1c035ff13546573f526abbc3d4b1e157edb3a278caba746
|
||||
size 69060
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ notebooks/detectron2-to-openvino/detectron2-to-openvino.ipynb
|
|||
notebooks/distilbert-sequence-classification/distilbert-sequence-classification.ipynb
|
||||
notebooks/distil-whisper-asr/distil-whisper-asr.ipynb
|
||||
notebooks/dolly-2-instruction-following/dolly-2-instruction-following.ipynb
|
||||
notebooks/dynamicrafter-animating-images/dynamicrafter-animating-images.ipynb
|
||||
notebooks/efficient-sam/efficient-sam.ipynb
|
||||
notebooks/encodec-audio-compression/encodec-audio-compression.ipynb
|
||||
notebooks/fast-segment-anything/fast-segment-anything.ipynb
|
||||
|
|
|
|||
|
|
@ -78,7 +78,7 @@ Load and run the original pipeline
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -202,29 +202,29 @@ Convert the Text Encoder
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input_shape[-1] > 1 or self.sliding_window is not None:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if past_key_values_length > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:620: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:620: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
encoder_states = () if output_hidden_states else None
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:625: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:625: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if output_hidden_states:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:648: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:648: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if output_hidden_states:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:651: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:651: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if not return_dict:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:742: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:742: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if not return_dict:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:1227: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:1227: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if not return_dict:
|
||||
|
||||
|
||||
|
|
@ -333,13 +333,13 @@ suitable. This function repeats part of ``AmusedPipeline``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/tmp/ipykernel_2841622/3779428577.py:34: TracerWarning: Converting a tensor to a Python list might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/tmp/ipykernel_17572/3779428577.py:34: TracerWarning: Converting a tensor to a Python list might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
shape=shape.tolist(),
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/vq_model.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/vq_model.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if not force_not_quantize:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if hidden_states.shape[0] >= 64:
|
||||
|
||||
|
||||
|
|
@ -477,7 +477,7 @@ And insert wrappers instances in the pipeline:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
|
||||
|
||||
|
||||
|
|
@ -692,7 +692,7 @@ model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1486: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1486: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
|
||||
You can avoid this message in future by passing the argument `trust_remote_code=True`.
|
||||
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
|
||||
warnings.warn(
|
||||
|
|
@ -706,7 +706,7 @@ model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
|
||||
|
||||
|
||||
|
|
@ -784,17 +784,17 @@ model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
|
||||
return Tensor(self.data * unwrap_tensor_data(other))
|
||||
|
||||
|
||||
|
|
@ -826,7 +826,7 @@ Demo generation with quantized pipeline
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
|
||||
|
||||
|
||||
|
|
@ -910,7 +910,7 @@ a rough estimate of generation quality.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Metric `InceptionScore` will save all extracted features in buffer. For large datasets this may lead to large memory footprint.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Metric `InceptionScore` will save all extracted features in buffer. For large datasets this may lead to large memory footprint.
|
||||
warnings.warn(\*args, \*\*kwargs) # noqa: B028
|
||||
|
||||
|
||||
|
|
@ -922,9 +922,9 @@ a rough estimate of generation quality.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
|
||||
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/image/inception.py:176: UserWarning: std(): degrees of freedom is <= 0. Correction should be strictly less than the reduction factor (input numel divided by output numel). (Triggered internally at ../aten/src/ATen/native/ReduceOps.cpp:1807.)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/image/inception.py:176: UserWarning: std(): degrees of freedom is <= 0. Correction should be strictly less than the reduction factor (input numel divided by output numel). (Triggered internally at ../aten/src/ATen/native/ReduceOps.cpp:1807.)
|
||||
return kl.mean(), kl.std()
|
||||
|
||||
|
||||
|
|
@ -941,7 +941,7 @@ a rough estimate of generation quality.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Quantized pipeline Inception Score: 9.630990028381348
|
||||
Quantized pipeline Inception Score: 9.630992889404297
|
||||
Quantization speed-up: 2.10x
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -1,3 +0,0 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3d53c86b99c2ec0b3b3cfc9f9acc9e1a44643013b4cfdf7ba27231bb70150be7
|
||||
size 1456471
|
||||
|
|
@ -0,0 +1,638 @@
|
|||
Asynchronous Inference with OpenVINO™
|
||||
=====================================
|
||||
|
||||
This notebook demonstrates how to use the `Async
|
||||
API <https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/general-optimizations.html>`__
|
||||
for asynchronous execution with OpenVINO.
|
||||
|
||||
OpenVINO Runtime supports inference in either synchronous or
|
||||
asynchronous mode. The key advantage of the Async API is that when a
|
||||
device is busy with inference, the application can perform other tasks
|
||||
in parallel (for example, populating inputs or scheduling other
|
||||
requests) rather than wait for the current inference to complete first.
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- `Imports <#imports>`__
|
||||
- `Prepare model and data
|
||||
processing <#prepare-model-and-data-processing>`__
|
||||
|
||||
- `Download test model <#download-test-model>`__
|
||||
- `Load the model <#load-the-model>`__
|
||||
- `Create functions for data
|
||||
processing <#create-functions-for-data-processing>`__
|
||||
- `Get the test video <#get-the-test-video>`__
|
||||
|
||||
- `How to improve the throughput of video
|
||||
processing <#how-to-improve-the-throughput-of-video-processing>`__
|
||||
|
||||
- `Sync Mode (default) <#sync-mode-default>`__
|
||||
- `Test performance in Sync Mode <#test-performance-in-sync-mode>`__
|
||||
- `Async Mode <#async-mode>`__
|
||||
- `Test the performance in Async
|
||||
Mode <#test-the-performance-in-async-mode>`__
|
||||
- `Compare the performance <#compare-the-performance>`__
|
||||
|
||||
- `AsyncInferQueue <#asyncinferqueue>`__
|
||||
|
||||
- `Setting Callback <#setting-callback>`__
|
||||
- `Test the performance with
|
||||
AsyncInferQueue <#test-the-performance-with-asyncinferqueue>`__
|
||||
|
||||
Imports
|
||||
-------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import platform
|
||||
|
||||
%pip install -q "openvino>=2023.1.0"
|
||||
%pip install -q opencv-python
|
||||
if platform.system() != "windows":
|
||||
%pip install -q "matplotlib>=3.4"
|
||||
else:
|
||||
%pip install -q "matplotlib>=3.4,<3.7"
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import cv2
|
||||
import time
|
||||
import numpy as np
|
||||
import openvino as ov
|
||||
from IPython import display
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# Fetch the notebook utils script from the openvino_notebooks repo
|
||||
import requests
|
||||
|
||||
r = requests.get(
|
||||
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py",
|
||||
)
|
||||
open("notebook_utils.py", "w").write(r.text)
|
||||
|
||||
import notebook_utils as utils
|
||||
|
||||
Prepare model and data processing
|
||||
---------------------------------
|
||||
|
||||
|
||||
|
||||
Download test model
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
We use a pre-trained model from OpenVINO’s `Open Model
|
||||
Zoo <https://docs.openvino.ai/2024/documentation/legacy-features/model-zoo.html>`__
|
||||
to start the test. In this case, the model will be executed to detect
|
||||
the person in each frame of the video.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
# directory where model will be downloaded
|
||||
base_model_dir = "model"
|
||||
|
||||
# model name as named in Open Model Zoo
|
||||
model_name = "person-detection-0202"
|
||||
precision = "FP16"
|
||||
model_path = f"model/intel/{model_name}/{precision}/{model_name}.xml"
|
||||
download_command = f"omz_downloader " f"--name {model_name} " f"--precision {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}"
|
||||
! $download_command
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
################|| Downloading person-detection-0202 ||################
|
||||
|
||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.xml
|
||||
|
||||
|
||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
|
||||
|
||||
|
||||
|
||||
|
||||
Select inference device
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Dropdown(description='Device:', options=('CPU', 'AUTO'), value='CPU')
|
||||
|
||||
|
||||
|
||||
Load the model
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
# initialize OpenVINO runtime
|
||||
core = ov.Core()
|
||||
|
||||
# read the network and corresponding weights from file
|
||||
model = core.read_model(model=model_path)
|
||||
|
||||
# compile the model for the CPU (you can choose manually CPU, GPU etc.)
|
||||
# or let the engine choose the best available device (AUTO)
|
||||
compiled_model = core.compile_model(model=model, device_name=device.value)
|
||||
|
||||
# get input node
|
||||
input_layer_ir = model.input(0)
|
||||
N, C, H, W = input_layer_ir.shape
|
||||
shape = (H, W)
|
||||
|
||||
Create functions for data processing
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
def preprocess(image):
|
||||
"""
|
||||
Define the preprocess function for input data
|
||||
|
||||
:param: image: the orignal input frame
|
||||
:returns:
|
||||
resized_image: the image processed
|
||||
"""
|
||||
resized_image = cv2.resize(image, shape)
|
||||
resized_image = cv2.cvtColor(np.array(resized_image), cv2.COLOR_BGR2RGB)
|
||||
resized_image = resized_image.transpose((2, 0, 1))
|
||||
resized_image = np.expand_dims(resized_image, axis=0).astype(np.float32)
|
||||
return resized_image
|
||||
|
||||
|
||||
def postprocess(result, image, fps):
|
||||
"""
|
||||
Define the postprocess function for output data
|
||||
|
||||
:param: result: the inference results
|
||||
image: the orignal input frame
|
||||
fps: average throughput calculated for each frame
|
||||
:returns:
|
||||
image: the image with bounding box and fps message
|
||||
"""
|
||||
detections = result.reshape(-1, 7)
|
||||
for i, detection in enumerate(detections):
|
||||
_, image_id, confidence, xmin, ymin, xmax, ymax = detection
|
||||
if confidence > 0.5:
|
||||
xmin = int(max((xmin * image.shape[1]), 10))
|
||||
ymin = int(max((ymin * image.shape[0]), 10))
|
||||
xmax = int(min((xmax * image.shape[1]), image.shape[1] - 10))
|
||||
ymax = int(min((ymax * image.shape[0]), image.shape[0] - 10))
|
||||
cv2.rectangle(image, (xmin, ymin), (xmax, ymax), (0, 255, 0), 2)
|
||||
cv2.putText(
|
||||
image,
|
||||
str(round(fps, 2)) + " fps",
|
||||
(5, 20),
|
||||
cv2.FONT_HERSHEY_SIMPLEX,
|
||||
0.7,
|
||||
(0, 255, 0),
|
||||
3,
|
||||
)
|
||||
return image
|
||||
|
||||
Get the test video
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
video_path = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/video/CEO%20Pat%20Gelsinger%20on%20Leading%20Intel.mp4"
|
||||
|
||||
How to improve the throughput of video processing
|
||||
-------------------------------------------------
|
||||
|
||||
|
||||
|
||||
Below, we compare the performance of the synchronous and async-based
|
||||
approaches:
|
||||
|
||||
Sync Mode (default)
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
Let us see how video processing works with the default approach. Using
|
||||
the synchronous approach, the frame is captured with OpenCV and then
|
||||
immediately processed:
|
||||
|
||||
.. figure:: https://user-images.githubusercontent.com/91237924/168452573-d354ea5b-7966-44e5-813d-f9053be4338a.png
|
||||
:alt: drawing
|
||||
|
||||
drawing
|
||||
|
||||
::
|
||||
|
||||
while(true) {
|
||||
// capture frame
|
||||
// populate CURRENT InferRequest
|
||||
// Infer CURRENT InferRequest
|
||||
//this call is synchronous
|
||||
// display CURRENT result
|
||||
}
|
||||
|
||||
\``\`
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
def sync_api(source, flip, fps, use_popup, skip_first_frames):
|
||||
"""
|
||||
Define the main function for video processing in sync mode
|
||||
|
||||
:param: source: the video path or the ID of your webcam
|
||||
:returns:
|
||||
sync_fps: the inference throughput in sync mode
|
||||
"""
|
||||
frame_number = 0
|
||||
infer_request = compiled_model.create_infer_request()
|
||||
player = None
|
||||
try:
|
||||
# Create a video player
|
||||
player = utils.VideoPlayer(source, flip=flip, fps=fps, skip_first_frames=skip_first_frames)
|
||||
# Start capturing
|
||||
start_time = time.time()
|
||||
player.start()
|
||||
if use_popup:
|
||||
title = "Press ESC to Exit"
|
||||
cv2.namedWindow(title, cv2.WINDOW_GUI_NORMAL | cv2.WINDOW_AUTOSIZE)
|
||||
while True:
|
||||
frame = player.next()
|
||||
if frame is None:
|
||||
print("Source ended")
|
||||
break
|
||||
resized_frame = preprocess(frame)
|
||||
infer_request.set_tensor(input_layer_ir, ov.Tensor(resized_frame))
|
||||
# Start the inference request in synchronous mode
|
||||
infer_request.infer()
|
||||
res = infer_request.get_output_tensor(0).data
|
||||
stop_time = time.time()
|
||||
total_time = stop_time - start_time
|
||||
frame_number = frame_number + 1
|
||||
sync_fps = frame_number / total_time
|
||||
frame = postprocess(res, frame, sync_fps)
|
||||
# Display the results
|
||||
if use_popup:
|
||||
cv2.imshow(title, frame)
|
||||
key = cv2.waitKey(1)
|
||||
# escape = 27
|
||||
if key == 27:
|
||||
break
|
||||
else:
|
||||
# Encode numpy array to jpg
|
||||
_, encoded_img = cv2.imencode(".jpg", frame, params=[cv2.IMWRITE_JPEG_QUALITY, 90])
|
||||
# Create IPython image
|
||||
i = display.Image(data=encoded_img)
|
||||
# Display the image in this notebook
|
||||
display.clear_output(wait=True)
|
||||
display.display(i)
|
||||
# ctrl-c
|
||||
except KeyboardInterrupt:
|
||||
print("Interrupted")
|
||||
# Any different error
|
||||
except RuntimeError as e:
|
||||
print(e)
|
||||
finally:
|
||||
if use_popup:
|
||||
cv2.destroyAllWindows()
|
||||
if player is not None:
|
||||
# stop capturing
|
||||
player.stop()
|
||||
return sync_fps
|
||||
|
||||
Test performance in Sync Mode
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
sync_fps = sync_api(source=video_path, flip=False, fps=30, use_popup=False, skip_first_frames=800)
|
||||
print(f"average throuput in sync mode: {sync_fps:.2f} fps")
|
||||
|
||||
|
||||
|
||||
.. image:: async-api-with-output_files/async-api-with-output_17_0.png
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Source ended
|
||||
average throuput in sync mode: 43.35 fps
|
||||
|
||||
|
||||
Async Mode
|
||||
~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
Let us see how the OpenVINO Async API can improve the overall frame rate
|
||||
of an application. The key advantage of the Async approach is as
|
||||
follows: while a device is busy with the inference, the application can
|
||||
do other things in parallel (for example, populating inputs or
|
||||
scheduling other requests) rather than wait for the current inference to
|
||||
complete first.
|
||||
|
||||
.. figure:: https://user-images.githubusercontent.com/91237924/168452572-c2ff1c59-d470-4b85-b1f6-b6e1dac9540e.png
|
||||
:alt: drawing
|
||||
|
||||
drawing
|
||||
|
||||
In the example below, inference is applied to the results of the video
|
||||
decoding. So it is possible to keep multiple infer requests, and while
|
||||
the current request is processed, the input frame for the next is being
|
||||
captured. This essentially hides the latency of capturing, so that the
|
||||
overall frame rate is rather determined only by the slowest part of the
|
||||
pipeline (decoding vs inference) and not by the sum of the stages.
|
||||
|
||||
::
|
||||
|
||||
while(true) {
|
||||
// capture frame
|
||||
// populate NEXT InferRequest
|
||||
// start NEXT InferRequest
|
||||
// this call is async and returns immediately
|
||||
// wait for the CURRENT InferRequest
|
||||
// display CURRENT result
|
||||
// swap CURRENT and NEXT InferRequests
|
||||
}
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
def async_api(source, flip, fps, use_popup, skip_first_frames):
|
||||
"""
|
||||
Define the main function for video processing in async mode
|
||||
|
||||
:param: source: the video path or the ID of your webcam
|
||||
:returns:
|
||||
async_fps: the inference throughput in async mode
|
||||
"""
|
||||
frame_number = 0
|
||||
# Create 2 infer requests
|
||||
curr_request = compiled_model.create_infer_request()
|
||||
next_request = compiled_model.create_infer_request()
|
||||
player = None
|
||||
async_fps = 0
|
||||
try:
|
||||
# Create a video player
|
||||
player = utils.VideoPlayer(source, flip=flip, fps=fps, skip_first_frames=skip_first_frames)
|
||||
# Start capturing
|
||||
start_time = time.time()
|
||||
player.start()
|
||||
if use_popup:
|
||||
title = "Press ESC to Exit"
|
||||
cv2.namedWindow(title, cv2.WINDOW_GUI_NORMAL | cv2.WINDOW_AUTOSIZE)
|
||||
# Capture CURRENT frame
|
||||
frame = player.next()
|
||||
resized_frame = preprocess(frame)
|
||||
curr_request.set_tensor(input_layer_ir, ov.Tensor(resized_frame))
|
||||
# Start the CURRENT inference request
|
||||
curr_request.start_async()
|
||||
while True:
|
||||
# Capture NEXT frame
|
||||
next_frame = player.next()
|
||||
if next_frame is None:
|
||||
print("Source ended")
|
||||
break
|
||||
resized_frame = preprocess(next_frame)
|
||||
next_request.set_tensor(input_layer_ir, ov.Tensor(resized_frame))
|
||||
# Start the NEXT inference request
|
||||
next_request.start_async()
|
||||
# Waiting for CURRENT inference result
|
||||
curr_request.wait()
|
||||
res = curr_request.get_output_tensor(0).data
|
||||
stop_time = time.time()
|
||||
total_time = stop_time - start_time
|
||||
frame_number = frame_number + 1
|
||||
async_fps = frame_number / total_time
|
||||
frame = postprocess(res, frame, async_fps)
|
||||
# Display the results
|
||||
if use_popup:
|
||||
cv2.imshow(title, frame)
|
||||
key = cv2.waitKey(1)
|
||||
# escape = 27
|
||||
if key == 27:
|
||||
break
|
||||
else:
|
||||
# Encode numpy array to jpg
|
||||
_, encoded_img = cv2.imencode(".jpg", frame, params=[cv2.IMWRITE_JPEG_QUALITY, 90])
|
||||
# Create IPython image
|
||||
i = display.Image(data=encoded_img)
|
||||
# Display the image in this notebook
|
||||
display.clear_output(wait=True)
|
||||
display.display(i)
|
||||
# Swap CURRENT and NEXT frames
|
||||
frame = next_frame
|
||||
# Swap CURRENT and NEXT infer requests
|
||||
curr_request, next_request = next_request, curr_request
|
||||
# ctrl-c
|
||||
except KeyboardInterrupt:
|
||||
print("Interrupted")
|
||||
# Any different error
|
||||
except RuntimeError as e:
|
||||
print(e)
|
||||
finally:
|
||||
if use_popup:
|
||||
cv2.destroyAllWindows()
|
||||
if player is not None:
|
||||
# stop capturing
|
||||
player.stop()
|
||||
return async_fps
|
||||
|
||||
Test the performance in Async Mode
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
async_fps = async_api(source=video_path, flip=False, fps=30, use_popup=False, skip_first_frames=800)
|
||||
print(f"average throuput in async mode: {async_fps:.2f} fps")
|
||||
|
||||
|
||||
|
||||
.. image:: async-api-with-output_files/async-api-with-output_21_0.png
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Source ended
|
||||
average throuput in async mode: 73.97 fps
|
||||
|
||||
|
||||
Compare the performance
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
width = 0.4
|
||||
fontsize = 14
|
||||
|
||||
plt.rc("font", size=fontsize)
|
||||
fig, ax = plt.subplots(1, 1, figsize=(10, 8))
|
||||
|
||||
rects1 = ax.bar([0], sync_fps, width, color="#557f2d")
|
||||
rects2 = ax.bar([width], async_fps, width)
|
||||
ax.set_ylabel("frames per second")
|
||||
ax.set_xticks([0, width])
|
||||
ax.set_xticklabels(["Sync mode", "Async mode"])
|
||||
ax.set_xlabel("Higher is better")
|
||||
|
||||
fig.suptitle("Sync mode VS Async mode")
|
||||
fig.tight_layout()
|
||||
|
||||
plt.show()
|
||||
|
||||
|
||||
|
||||
.. image:: async-api-with-output_files/async-api-with-output_23_0.png
|
||||
|
||||
|
||||
``AsyncInferQueue``
|
||||
-------------------
|
||||
|
||||
|
||||
|
||||
Asynchronous mode pipelines can be supported with the
|
||||
`AsyncInferQueue <https://docs.openvino.ai/2024/openvino-workflow/running-inference/integrate-openvino-with-your-application/python-api-exclusives.html#asyncinferqueue>`__
|
||||
wrapper class. This class automatically spawns the pool of
|
||||
``InferRequest`` objects (also called “jobs”) and provides
|
||||
synchronization mechanisms to control the flow of the pipeline. It is a
|
||||
simpler way to manage the infer request queue in Asynchronous mode.
|
||||
|
||||
Setting Callback
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
When ``callback`` is set, any job that ends inference calls upon the
|
||||
Python function. The ``callback`` function must have two arguments: one
|
||||
is the request that calls the ``callback``, which provides the
|
||||
``InferRequest`` API; the other is called “user data”, which provides
|
||||
the possibility of passing runtime values.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
def callback(infer_request, info) -> None:
|
||||
"""
|
||||
Define the callback function for postprocessing
|
||||
|
||||
:param: infer_request: the infer_request object
|
||||
info: a tuple includes original frame and starts time
|
||||
:returns:
|
||||
None
|
||||
"""
|
||||
global frame_number
|
||||
global total_time
|
||||
global inferqueue_fps
|
||||
stop_time = time.time()
|
||||
frame, start_time = info
|
||||
total_time = stop_time - start_time
|
||||
frame_number = frame_number + 1
|
||||
inferqueue_fps = frame_number / total_time
|
||||
|
||||
res = infer_request.get_output_tensor(0).data[0]
|
||||
frame = postprocess(res, frame, inferqueue_fps)
|
||||
# Encode numpy array to jpg
|
||||
_, encoded_img = cv2.imencode(".jpg", frame, params=[cv2.IMWRITE_JPEG_QUALITY, 90])
|
||||
# Create IPython image
|
||||
i = display.Image(data=encoded_img)
|
||||
# Display the image in this notebook
|
||||
display.clear_output(wait=True)
|
||||
display.display(i)
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
def inferqueue(source, flip, fps, skip_first_frames) -> None:
|
||||
"""
|
||||
Define the main function for video processing with async infer queue
|
||||
|
||||
:param: source: the video path or the ID of your webcam
|
||||
:retuns:
|
||||
None
|
||||
"""
|
||||
# Create infer requests queue
|
||||
infer_queue = ov.AsyncInferQueue(compiled_model, 2)
|
||||
infer_queue.set_callback(callback)
|
||||
player = None
|
||||
try:
|
||||
# Create a video player
|
||||
player = utils.VideoPlayer(source, flip=flip, fps=fps, skip_first_frames=skip_first_frames)
|
||||
# Start capturing
|
||||
start_time = time.time()
|
||||
player.start()
|
||||
while True:
|
||||
# Capture frame
|
||||
frame = player.next()
|
||||
if frame is None:
|
||||
print("Source ended")
|
||||
break
|
||||
resized_frame = preprocess(frame)
|
||||
# Start the inference request with async infer queue
|
||||
infer_queue.start_async({input_layer_ir.any_name: resized_frame}, (frame, start_time))
|
||||
except KeyboardInterrupt:
|
||||
print("Interrupted")
|
||||
# Any different error
|
||||
except RuntimeError as e:
|
||||
print(e)
|
||||
finally:
|
||||
infer_queue.wait_all()
|
||||
player.stop()
|
||||
|
||||
Test the performance with ``AsyncInferQueue``
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
frame_number = 0
|
||||
total_time = 0
|
||||
inferqueue(source=video_path, flip=False, fps=30, skip_first_frames=800)
|
||||
print(f"average throughput in async mode with async infer queue: {inferqueue_fps:.2f} fps")
|
||||
|
||||
|
||||
|
||||
.. image:: async-api-with-output_files/async-api-with-output_29_0.png
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
average throughput in async mode with async infer queue: 111.33 fps
|
||||
|
||||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:deee8ff5a3fba807c811d63eeb7f516cbdc82d47f6b4c4ae797742ff6327a54d
|
||||
size 30483
|
||||
oid sha256:c6eb6b07a2e43cfab480087829f6babef1e7050550997c85a7a6824f8c308cc3
|
||||
size 30403
|
||||
|
|
|
|||
|
|
@ -186,15 +186,15 @@ By default, ``compile_model`` API will select **AUTO** as
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[23:25:36.8706]I[plugin.cpp:418][AUTO] device:CPU, config:LOG_LEVEL=LOG_INFO
|
||||
[23:25:36.8707]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
||||
[23:25:36.8707]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
||||
[23:25:36.8707]I[plugin.cpp:418][AUTO] device:CPU, config:PERF_COUNT=NO
|
||||
[23:25:36.8707]I[plugin.cpp:423][AUTO] device:CPU, priority:0
|
||||
[23:25:36.8707]I[schedule.cpp:17][AUTO] scheduler starting
|
||||
[23:25:36.8707]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
||||
[23:25:37.0101]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
||||
[23:25:37.0103]I[plugin.cpp:451][AUTO] underlying hardware does not support hardware context
|
||||
[23:28:01.3129]I[plugin.cpp:418][AUTO] device:CPU, config:LOG_LEVEL=LOG_INFO
|
||||
[23:28:01.3129]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
|
||||
[23:28:01.3130]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
|
||||
[23:28:01.3130]I[plugin.cpp:418][AUTO] device:CPU, config:PERF_COUNT=NO
|
||||
[23:28:01.3130]I[plugin.cpp:423][AUTO] device:CPU, priority:0
|
||||
[23:28:01.3130]I[schedule.cpp:17][AUTO] scheduler starting
|
||||
[23:28:01.3130]I[auto_schedule.cpp:131][AUTO] select device:CPU
|
||||
[23:28:01.4657]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
|
||||
[23:28:01.4659]I[plugin.cpp:451][AUTO] underlying hardware does not support hardware context
|
||||
Successfully compiled model without a device_name.
|
||||
|
||||
|
||||
|
|
@ -208,7 +208,7 @@ By default, ``compile_model`` API will select **AUTO** as
|
|||
.. parsed-literal::
|
||||
|
||||
Deleted compiled_model
|
||||
[23:25:37.0205]I[schedule.cpp:303][AUTO] scheduler ending
|
||||
[23:28:01.4767]I[schedule.cpp:303][AUTO] scheduler ending
|
||||
|
||||
|
||||
Explicitly pass AUTO as device_name to Core::compile_model API
|
||||
|
|
@ -366,7 +366,7 @@ executed on CPU until GPU is ready.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Time to load model using AUTO device and get first inference: 0.19 seconds.
|
||||
Time to load model using AUTO device and get first inference: 0.18 seconds.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -538,12 +538,12 @@ Loop for inference and update the FPS/Latency every
|
|||
|
||||
Compiling Model for AUTO device with THROUGHPUT hint
|
||||
Start inference, 6 groups of FPS/latency will be measured over 10s intervals
|
||||
throughput: 177.50fps, latency: 32.10ms, time interval: 10.00s
|
||||
throughput: 179.46fps, latency: 32.64ms, time interval: 10.01s
|
||||
throughput: 179.28fps, latency: 32.70ms, time interval: 10.00s
|
||||
throughput: 177.92fps, latency: 32.86ms, time interval: 10.01s
|
||||
throughput: 178.98fps, latency: 32.68ms, time interval: 10.02s
|
||||
throughput: 178.91fps, latency: 32.77ms, time interval: 10.01s
|
||||
throughput: 177.51fps, latency: 32.04ms, time interval: 10.01s
|
||||
throughput: 179.73fps, latency: 32.54ms, time interval: 10.01s
|
||||
throughput: 178.74fps, latency: 32.73ms, time interval: 10.01s
|
||||
throughput: 179.46fps, latency: 32.59ms, time interval: 10.01s
|
||||
throughput: 178.98fps, latency: 32.74ms, time interval: 10.02s
|
||||
throughput: 178.58fps, latency: 32.79ms, time interval: 10.01s
|
||||
Done
|
||||
|
||||
|
||||
|
|
@ -589,12 +589,12 @@ Loop for inference and update the FPS/Latency for each
|
|||
|
||||
Compiling Model for AUTO Device with LATENCY hint
|
||||
Start inference, 6 groups fps/latency will be out with 10s interval
|
||||
throughput: 135.86fps, latency: 6.81ms, time interval: 10.00s
|
||||
throughput: 138.93fps, latency: 6.82ms, time interval: 10.01s
|
||||
throughput: 138.89fps, latency: 6.82ms, time interval: 10.00s
|
||||
throughput: 138.82fps, latency: 6.81ms, time interval: 10.01s
|
||||
throughput: 138.99fps, latency: 6.82ms, time interval: 10.00s
|
||||
throughput: 139.01fps, latency: 6.82ms, time interval: 10.01s
|
||||
throughput: 136.40fps, latency: 6.83ms, time interval: 10.01s
|
||||
throughput: 137.96fps, latency: 6.81ms, time interval: 10.00s
|
||||
throughput: 137.97fps, latency: 6.80ms, time interval: 10.00s
|
||||
throughput: 137.97fps, latency: 6.80ms, time interval: 10.00s
|
||||
throughput: 138.06fps, latency: 6.80ms, time interval: 10.00s
|
||||
throughput: 133.29fps, latency: 7.06ms, time interval: 10.01s
|
||||
Done
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2ce598bcda980dc39d1fc49884a02509af4d0f599c4dd674e07b994af39cf533
|
||||
size 27041
|
||||
oid sha256:d644b71f335dd26763dfad14f93ba1ff32ffe30cfdbe3ac06d1c7346aaba3985
|
||||
size 27550
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1efc7ab0c2433842a374eb59dc1be264bb613039696dbdceb5b51c97c8ad666b
|
||||
size 39972
|
||||
oid sha256:b3e6b932de9fd81a384cccede0ed571c920f68e6a9176ce2d84cee626bb03f05
|
||||
size 40115
|
||||
|
|
|
|||
|
|
@ -559,7 +559,7 @@ As discussed before, the model consists of several blocks which can be
|
|||
reused for building pipelines for different tasks. In the diagram below,
|
||||
you can see how image captioning works:
|
||||
|
||||
|image6|
|
||||
|image01|
|
||||
|
||||
The visual model accepts the image preprocessed by ``BlipProcessor`` as
|
||||
input and produces image embeddings, which are directly passed to the
|
||||
|
|
@ -573,12 +573,12 @@ tokenized by ``BlipProcessor`` are provided to the text encoder and then
|
|||
multimodal question embedding is passed to the text decoder for
|
||||
performing generation of answers.
|
||||
|
||||
|image7|
|
||||
|image11|
|
||||
|
||||
The next step is implementing both pipelines using OpenVINO models.
|
||||
|
||||
.. |image6| image:: https://user-images.githubusercontent.com/29454499/221865836-a56da06e-196d-449c-a5dc-4136da6ab5d5.png
|
||||
.. |image7| image:: https://user-images.githubusercontent.com/29454499/221868167-d0081add-d9f3-4591-80e7-4753c88c1d0a.png
|
||||
.. |image01| image:: https://user-images.githubusercontent.com/29454499/221865836-a56da06e-196d-449c-a5dc-4136da6ab5d5.png
|
||||
.. |image11| image:: https://user-images.githubusercontent.com/29454499/221868167-d0081add-d9f3-4591-80e7-4753c88c1d0a.png
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ Table of contents:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
|
@ -181,13 +181,13 @@ NLP model from Hugging Face and export it in ONNX format:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-06 23:46:51.110172: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-06 23:46:51.145296: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-15 23:49:16.064636: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-15 23:49:16.099980: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-06 23:46:51.660347: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
2024-05-15 23:49:16.617080: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||
|
||||
|
||||
|
|
@ -664,12 +664,12 @@ frameworks conversion guides.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-06 23:47:11.917183: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-05-06 23:47:11.917219: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-05-06 23:47:11.917224: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-05-06 23:47:11.917431: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-05-06 23:47:11.917454: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-05-06 23:47:11.917459: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-05-15 23:49:36.583572: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-05-15 23:49:36.583606: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
|
||||
2024-05-15 23:49:36.583610: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
|
||||
2024-05-15 23:49:36.583842: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
|
||||
2024-05-15 23:49:36.583866: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
|
||||
2024-05-15 23:49:36.583871: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
Migration from Legacy conversion API
|
||||
|
|
|
|||
|
|
@ -183,7 +183,7 @@ And print results
|
|||
|
||||
Predicted Class: 281
|
||||
Predicted Label: n02123045 tabby, tabby cat
|
||||
Predicted Probability: 0.5510364174842834
|
||||
Predicted Probability: 0.4661690592765808
|
||||
|
||||
|
||||
Convert the model to OpenVINO Intermediate representation format
|
||||
|
|
|
|||
|
|
@ -197,7 +197,7 @@ which in a raw format looks like this:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
'\ufeffThe Project Gutenberg eBook of Anna Karenina\r\n \r\nThis ebook is for the use of anyone anywhere in the United States and\r\nmost other parts of the world at no cost and with almost no restrictions\r\nwhatsoever. You may copy it, give it away or re-use it under the terms\r\nof the Project Gutenberg License included with this ebook or online\r\nat www.gutenberg.org. If you are not located in the United States,\r\nyou will have to check the laws of the country where you are located\r\nbefore using this eBook.\r\n\r\nTitle: Anna Karenina\r\n\r\n\r\nAuthor: graf Leo Tolstoy\r\n\r\nTranslator: Constance Garnett\r\n\r\nRelease date: July 1, 1998 [eBook #1399]\r\n Most recently updated: April 9, 2023\r\n\r\nLanguage: English\r\n\r\n\r\n\r\n\*\*\* START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA \*\*\*\r\n[Illustration]\r\n\r\n\r\n\r\n\r\n ANNA KARENINA \r\n\r\n by Leo Tolstoy \r\n\r\n Translated by Constance Garnett \r\n\r\nContents\r\n\r\n\r\n PART ONE\r\n PART TWO\r\n PART THREE\r\n PART FOUR\r\n PART FIVE\r\n PART SIX\r\n PART SEVEN\r\n PART EIGHT\r\n\r\n\r\n\r\n\r\nPART ONE\r\n\r\nChapter 1\r\n\r\n\r\nHappy families are all alike; every unhappy family is unhappy in its\r\nown way.\r\n\r\nEverything was in confusion in the Oblonskys’ house. The wife had\r\ndiscovered that the husband was carrying on an intrigue with a French\r\ngirl, who had been a governess in their family, and she had announced\r\nto her husband that she could not go on living in the same house with\r\nhim. This position of affairs had now lasted three days, and not only\r\nthe husband and wife themselves, but all the me'
|
||||
'\ufeffThe Project Gutenberg eBook of Anna Karenina\r\n \r\nThis ebook is for the use of anyone anywhere in the United States and\r\nmost other parts of the world at no cost and with almost no restrictions\r\nwhatsoever. You may copy it, give it away or re-use it under the terms\r\nof the Project Gutenberg License included with this ebook or online\r\nat www.gutenberg.org. If you are not located in the United States,\r\nyou will have to check the laws of the country where you are located\r\nbefore using this eBook.\r\n\r\nTitle: Anna Karenina\r\n\r\n\r\nAuthor: graf Leo Tolstoy\r\n\r\nTranslator: Constance Garnett\r\n\r\nRelease date: July 1, 1998 [eBook #1399]\r\n Most recently updated: April 9, 2023\r\n\r\nLanguage: English\r\n\r\n\r\n\r\n*** START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA \*\*\*\r\n[Illustration]\r\n\r\n\r\n\r\n\r\n ANNA KARENINA \r\n\r\n by Leo Tolstoy \r\n\r\n Translated by Constance Garnett \r\n\r\nContents\r\n\r\n\r\n PART ONE\r\n PART TWO\r\n PART THREE\r\n PART FOUR\r\n PART FIVE\r\n PART SIX\r\n PART SEVEN\r\n PART EIGHT\r\n\r\n\r\n\r\n\r\nPART ONE\r\n\r\nChapter 1\r\n\r\n\r\nHappy families are all alike; every unhappy family is unhappy in its\r\nown way.\r\n\r\nEverything was in confusion in the Oblonskys’ house. The wife had\r\ndiscovered that the husband was carrying on an intrigue with a French\r\ngirl, who had been a governess in their family, and she had announced\r\nto her husband that she could not go on living in the same house with\r\nhim. This position of affairs had now lasted three days, and not only\r\nthe husband and wife themselves, but all the me'
|
||||
|
||||
|
||||
|
||||
|
|
@ -210,7 +210,7 @@ which in a raw format looks like this:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
'The Project Gutenberg EBook of Anna Karenina, 1. Band, by Leo N. Tolstoi\r\n\r\nThis eBook is for the use of anyone anywhere at no cost and with\r\nalmost no restrictions whatsoever. You may copy it, give it away or\r\nre-use it under the terms of the Project Gutenberg License included\r\nwith this eBook or online at www.gutenberg.org\r\n\r\n\r\nTitle: Anna Karenina, 1. Band\r\n\r\nAuthor: Leo N. Tolstoi\r\n\r\nRelease Date: February 18, 2014 [EBook #44956]\r\n\r\nLanguage: German\r\n\r\nCharacter set encoding: ISO-8859-1\r\n\r\n\*\*\* START OF THIS PROJECT GUTENBERG EBOOK ANNA KARENINA, 1. BAND \*\*\*\r\n\r\n\r\n\r\n\r\nProduced by Norbert H. Langkau, Jens Nordmann and the\r\nOnline Distributed Proofreading Team at http://www.pgdp.net\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n Anna Karenina.\r\n\r\n\r\n Roman aus dem Russischen\r\n\r\n des\r\n\r\n Grafen Leo N. Tolstoi.\r\n\r\n\r\n\r\n Nach der siebenten Auflage übersetzt\r\n\r\n von\r\n\r\n Hans Moser.\r\n\r\n\r\n Erster Band.\r\n\r\n\r\n\r\n Leipzig\r\n\r\n Druck und Verlag von Philipp Reclam jun.\r\n\r\n \* \* \* \* \*\r\n\r\n\r\n\r\n\r\n Erster Teil.\r\n\r\n »Die Rache ist mein, ich will vergelten.«\r\n\r\n 1.\r\n\r\n\r\nAlle glücklichen Familien sind einander ähnlich; jede unglücklich'
|
||||
'The Project Gutenberg EBook of Anna Karenina, 1. Band, by Leo N. Tolstoi\r\n\r\nThis eBook is for the use of anyone anywhere at no cost and with\r\nalmost no restrictions whatsoever. You may copy it, give it away or\r\nre-use it under the terms of the Project Gutenberg License included\r\nwith this eBook or online at www.gutenberg.org\r\n\r\n\r\nTitle: Anna Karenina, 1. Band\r\n\r\nAuthor: Leo N. Tolstoi\r\n\r\nRelease Date: February 18, 2014 [EBook #44956]\r\n\r\nLanguage: German\r\n\r\nCharacter set encoding: ISO-8859-1\r\n\r\n\*\*\* START OF THIS PROJECT GUTENBERG EBOOK ANNA KARENINA, 1. BAND \*\*\*\r\n\r\n\r\n\r\n\r\nProduced by Norbert H. Langkau, Jens Nordmann and the\r\nOnline Distributed Proofreading Team at http://www.pgdp.net\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n Anna Karenina.\r\n\r\n\r\n Roman aus dem Russischen\r\n\r\n des\r\n\r\n Grafen Leo N. Tolstoi.\r\n\r\n\r\n\r\n Nach der siebenten Auflage übersetzt\r\n\r\n von\r\n\r\n Hans Moser.\r\n\r\n\r\n Erster Band.\r\n\r\n\r\n\r\n Leipzig\r\n\r\n Druck und Verlag von Philipp Reclam jun.\r\n\r\n \* \* \* \* *\r\n\r\n\r\n\r\n\r\n Erster Teil.\r\n\r\n »Die Rache ist mein, ich will vergelten.«\r\n\r\n 1.\r\n\r\n\r\nAlle glücklichen Familien sind einander ähnlich; jede unglücklich'
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ This notebook needs a trained UNet model. We provide a pre-trained
|
|||
model, trained for 20 epochs with the full
|
||||
`Kits-19 <https://github.com/neheller/kits19>`__ frames dataset, which
|
||||
has an F1 score on the validation set of 0.9. The training code is
|
||||
available in `this notebook <pytorch-monai-training-with-output.html>`__.
|
||||
available in `this notebook <pytorch-monai-training.ipynb>`__.
|
||||
|
||||
NNCF for PyTorch models requires a C++ compiler. On Windows, install
|
||||
`Microsoft Visual Studio
|
||||
|
|
@ -88,9 +88,9 @@ Table of contents:
|
|||
.. code:: ipython3
|
||||
|
||||
import platform
|
||||
|
||||
|
||||
%pip install -q "openvino>=2023.3.0" "monai>=0.9.1" "torchmetrics>=0.11.0" "nncf>=2.8.0" "opencv-python" torch tqdm --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
|
||||
if platform.system() != "Windows":
|
||||
%pip install -q "matplotlib>=3.4"
|
||||
else:
|
||||
|
|
@ -118,9 +118,9 @@ Imports
|
|||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Union
|
||||
|
||||
|
||||
warnings.filterwarnings("ignore", category=UserWarning)
|
||||
|
||||
|
||||
import cv2
|
||||
import matplotlib.pyplot as plt
|
||||
import monai
|
||||
|
|
@ -132,19 +132,19 @@ Imports
|
|||
from nncf.common.logging.logger import set_log_level
|
||||
from torchmetrics import F1Score as F1
|
||||
import requests
|
||||
|
||||
|
||||
|
||||
|
||||
set_log_level(logging.ERROR) # Disables all NNCF info and warning messages
|
||||
|
||||
|
||||
# Fetch `notebook_utils` module
|
||||
r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py")
|
||||
open("notebook_utils.py", "w").write(r.text)
|
||||
from notebook_utils import download_file
|
||||
|
||||
|
||||
if not Path("./custom_segmentation.py").exists():
|
||||
download_file(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/notebooks/ct-segmentation-quantize/custom_segmentation.py")
|
||||
from custom_segmentation import SegmentationModel
|
||||
|
||||
|
||||
if not Path("./async_pipeline.py").exists():
|
||||
download_file(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/notebooks/ct-segmentation-quantize/async_pipeline.py")
|
||||
from async_pipeline import show_live_inference
|
||||
|
|
@ -152,10 +152,10 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-06 23:48:33.144412: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-06 23:48:33.181396: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-15 23:50:56.536543: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-15 23:50:56.573924: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-06 23:48:33.764576: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-15 23:50:57.174238: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -197,13 +197,13 @@ notebook <pytorch-monai-training.ipynb>`__.
|
|||
state_dict_url = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/models/kidney-segmentation-kits19/unet_kits19_state_dict.pth"
|
||||
state_dict_file = download_file(state_dict_url, directory="pretrained_model")
|
||||
state_dict = torch.load(state_dict_file, map_location=torch.device("cpu"))
|
||||
|
||||
|
||||
new_state_dict = {}
|
||||
for k, v in state_dict.items():
|
||||
new_key = k.replace("_model.", "")
|
||||
new_state_dict[new_key] = v
|
||||
new_state_dict.pop("loss_function.pos_weight")
|
||||
|
||||
|
||||
model = monai.networks.nets.BasicUNet(spatial_dims=2, in_channels=1, out_channels=1).eval()
|
||||
model.load_state_dict(new_state_dict)
|
||||
|
||||
|
|
@ -285,8 +285,8 @@ method to display the images in the expected orientation:
|
|||
def rotate_and_flip(image):
|
||||
"""Rotate `image` by 90 degrees and flip horizontally"""
|
||||
return cv2.flip(cv2.rotate(image, rotateCode=cv2.ROTATE_90_CLOCKWISE), flipCode=1)
|
||||
|
||||
|
||||
|
||||
|
||||
class KitsDataset:
|
||||
def __init__(self, basedir: str):
|
||||
"""
|
||||
|
|
@ -295,35 +295,35 @@ method to display the images in the expected orientation:
|
|||
with each subdirectory containing directories imaging_frames, with jpg images, and
|
||||
segmentation_frames with segmentation masks as png files.
|
||||
See [data-preparation-ct-scan](./data-preparation-ct-scan.ipynb)
|
||||
|
||||
|
||||
:param basedir: Directory that contains the prepared CT scans
|
||||
"""
|
||||
masks = sorted(BASEDIR.glob("case_*/segmentation_frames/*png"))
|
||||
|
||||
|
||||
self.basedir = basedir
|
||||
self.dataset = masks
|
||||
print(f"Created dataset with {len(self.dataset)} items. " f"Base directory for data: {basedir}")
|
||||
|
||||
|
||||
def __getitem__(self, index):
|
||||
"""
|
||||
Get an item from the dataset at the specified index.
|
||||
|
||||
|
||||
:return: (image, segmentation_mask)
|
||||
"""
|
||||
mask_path = self.dataset[index]
|
||||
image_path = str(mask_path.with_suffix(".jpg")).replace("segmentation_frames", "imaging_frames")
|
||||
|
||||
|
||||
# Load images with MONAI's LoadImage to match data loading in training notebook
|
||||
mask = LoadImage(image_only=True, dtype=np.uint8)(str(mask_path)).numpy()
|
||||
img = LoadImage(image_only=True, dtype=np.float32)(str(image_path)).numpy()
|
||||
|
||||
|
||||
if img.shape[:2] != (512, 512):
|
||||
img = cv2.resize(img.astype(np.uint8), (512, 512)).astype(np.float32)
|
||||
mask = cv2.resize(mask, (512, 512))
|
||||
|
||||
|
||||
input_image = np.expand_dims(img, axis=0)
|
||||
return input_image, mask
|
||||
|
||||
|
||||
def __len__(self):
|
||||
return len(self.dataset)
|
||||
|
||||
|
|
@ -341,10 +341,10 @@ kidney pixels to verify that the annotations look correct:
|
|||
image_data, mask = next(item for item in dataset if np.count_nonzero(item[1]) > 5000)
|
||||
# Remove extra image dimension and rotate and flip the image for visualization
|
||||
image = rotate_and_flip(image_data.squeeze())
|
||||
|
||||
|
||||
# The data loader returns annotations as (index, mask) and mask in shape (H,W)
|
||||
mask = rotate_and_flip(mask)
|
||||
|
||||
|
||||
fig, ax = plt.subplots(1, 2, figsize=(12, 6))
|
||||
ax[0].imshow(image, cmap="gray")
|
||||
ax[1].imshow(mask, cmap="gray");
|
||||
|
|
@ -422,7 +422,7 @@ this notebook.
|
|||
.. code:: ipython3
|
||||
|
||||
fp32_ir_path = MODEL_DIR / Path("unet_kits19_fp32.xml")
|
||||
|
||||
|
||||
fp32_ir_model = ov.convert_model(model, example_input=torch.ones(1, 1, 512, 512, dtype=torch.float32))
|
||||
ov.save_model(fp32_ir_model, str(fp32_ir_path))
|
||||
|
||||
|
|
@ -435,7 +435,7 @@ this notebook.
|
|||
.. parsed-literal::
|
||||
|
||||
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||
|
||||
|
||||
|
|
@ -467,8 +467,8 @@ steps:
|
|||
"""
|
||||
images, _ = data_item
|
||||
return images
|
||||
|
||||
|
||||
|
||||
|
||||
data_loader = torch.utils.data.DataLoader(dataset)
|
||||
calibration_dataset = nncf.Dataset(data_loader, transform_fn)
|
||||
quantized_model = nncf.quantize(
|
||||
|
|
@ -534,18 +534,18 @@ Convert quantized model to OpenVINO IR model and save it.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_low.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_high.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
Tensor-likes are not close!
|
||||
|
||||
Mismatched elements: 249823 / 262144 (95.3%)
|
||||
Greatest absolute difference: 4.744992733001709 at index (0, 0, 242, 231) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 26823.613314473136 at index (0, 0, 124, 22) (up to 1e-05 allowed)
|
||||
|
||||
Mismatched elements: 247888 / 262144 (94.6%)
|
||||
Greatest absolute difference: 6.055658936500549 at index (0, 0, 158, 273) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 13834.454856259192 at index (0, 0, 343, 273) (up to 1e-05 allowed)
|
||||
_check_trace(
|
||||
|
||||
|
||||
|
|
@ -570,7 +570,7 @@ Compare File Size
|
|||
|
||||
fp32_ir_model_size = fp32_ir_path.with_suffix(".bin").stat().st_size / 1024
|
||||
quantized_model_size = int8_ir_path.with_suffix(".bin").stat().st_size / 1024
|
||||
|
||||
|
||||
print(f"FP32 IR model size: {fp32_ir_model_size:.2f} KB")
|
||||
print(f"INT8 model size: {quantized_model_size:.2f} KB")
|
||||
|
||||
|
|
@ -591,16 +591,16 @@ Select Inference Device
|
|||
core = ov.Core()
|
||||
# By default, benchmark on MULTI:CPU,GPU if a GPU is available, otherwise on CPU.
|
||||
device_list = ["MULTI:CPU,GPU" if "GPU" in core.available_devices else "AUTO"]
|
||||
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + device_list,
|
||||
value=device_list[0],
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -621,7 +621,7 @@ Compare Metrics for the original model and the quantized model to be sure that t
|
|||
|
||||
int8_compiled_model = core.compile_model(int8_ir_model, device.value)
|
||||
int8_f1 = compute_f1(int8_compiled_model, dataset)
|
||||
|
||||
|
||||
print(f"FP32 F1: {fp32_f1:.3f}")
|
||||
print(f"INT8 F1: {int8_f1:.3f}")
|
||||
|
||||
|
|
@ -669,17 +669,17 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[Step 2/11] Loading OpenVINO Runtime
|
||||
[ INFO ] OpenVINO:
|
||||
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ] Device info:
|
||||
[ INFO ] AUTO
|
||||
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[Step 3/11] Setting device configuration
|
||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 8.87 ms
|
||||
[ INFO ] Read model took 8.68 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x (node: x) : f32 / [...] / [?,?,?,?]
|
||||
|
|
@ -693,7 +693,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [?,1,16..,16..]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 148.60 ms
|
||||
[ INFO ] Compile model took 149.48 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -728,9 +728,9 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[Step 9/11] Creating infer requests and preparing input tensors
|
||||
[ ERROR ] Input x is dynamic. Provide data shapes!
|
||||
Traceback (most recent call last):
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
|
||||
data_queue = get_input_data(paths_to_input, app_inputs_info)
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
||||
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
|
||||
raise Exception(f"Input {info.name} is dynamic. Provide data shapes!")
|
||||
Exception: Input x is dynamic. Provide data shapes!
|
||||
|
||||
|
|
@ -748,17 +748,17 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[Step 2/11] Loading OpenVINO Runtime
|
||||
[ INFO ] OpenVINO:
|
||||
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ] Device info:
|
||||
[ INFO ] AUTO
|
||||
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[ INFO ]
|
||||
[Step 3/11] Setting device configuration
|
||||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 10.46 ms
|
||||
[ INFO ] Read model took 10.82 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x (node: x) : f32 / [...] / [1,1,512,512]
|
||||
|
|
@ -772,7 +772,7 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [1,1,512,512]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 253.55 ms
|
||||
[ INFO ] Compile model took 257.40 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model49
|
||||
|
|
@ -806,20 +806,20 @@ be run in the notebook with ``! benchmark_app`` or
|
|||
[ INFO ] PERF_COUNT: False
|
||||
[Step 9/11] Creating infer requests and preparing input tensors
|
||||
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
|
||||
[ INFO ] Fill input 'x' with random values
|
||||
[ INFO ] Fill input 'x' with random values
|
||||
[Step 10/11] Measuring performance (Start inference synchronously, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 28.04 ms
|
||||
[ INFO ] First inference took 27.38 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 969 iterations
|
||||
[ INFO ] Duration: 15000.48 ms
|
||||
[ INFO ] Count: 959 iterations
|
||||
[ INFO ] Duration: 15011.96 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 15.24 ms
|
||||
[ INFO ] Average: 15.28 ms
|
||||
[ INFO ] Min: 14.97 ms
|
||||
[ INFO ] Max: 17.08 ms
|
||||
[ INFO ] Throughput: 64.60 FPS
|
||||
[ INFO ] Median: 15.40 ms
|
||||
[ INFO ] Average: 15.45 ms
|
||||
[ INFO ] Min: 15.18 ms
|
||||
[ INFO ] Max: 17.18 ms
|
||||
[ INFO ] Throughput: 63.88 FPS
|
||||
|
||||
|
||||
Visually Compare Inference Results
|
||||
|
|
@ -853,11 +853,11 @@ seed is displayed to enable reproducing specific runs of this cell.
|
|||
# to binary segmentation masks
|
||||
def sigmoid(x):
|
||||
return np.exp(-np.logaddexp(0, -x))
|
||||
|
||||
|
||||
|
||||
|
||||
num_images = 4
|
||||
colormap = "gray"
|
||||
|
||||
|
||||
# Load FP32 and INT8 models
|
||||
core = ov.Core()
|
||||
fp_model = core.read_model(fp32_ir_path)
|
||||
|
|
@ -866,18 +866,18 @@ seed is displayed to enable reproducing specific runs of this cell.
|
|||
compiled_model_int8 = core.compile_model(int8_model, device_name=device.value)
|
||||
output_layer_fp = compiled_model_fp.output(0)
|
||||
output_layer_int8 = compiled_model_int8.output(0)
|
||||
|
||||
|
||||
# Create subset of dataset
|
||||
background_slices = (item for item in dataset if np.count_nonzero(item[1]) == 0)
|
||||
kidney_slices = (item for item in dataset if np.count_nonzero(item[1]) > 50)
|
||||
data_subset = random.sample(list(background_slices), 2) + random.sample(list(kidney_slices), 2)
|
||||
|
||||
|
||||
# Set seed to current time. To reproduce specific results, copy the printed seed
|
||||
# and manually set `seed` to that value.
|
||||
seed = int(time.time())
|
||||
random.seed(seed)
|
||||
print(f"Visualizing results with seed {seed}")
|
||||
|
||||
|
||||
fig, ax = plt.subplots(nrows=num_images, ncols=4, figsize=(24, num_images * 4))
|
||||
for i, (image, mask) in enumerate(data_subset):
|
||||
display_image = rotate_and_flip(image.squeeze())
|
||||
|
|
@ -886,13 +886,13 @@ seed is displayed to enable reproducing specific runs of this cell.
|
|||
input_image = np.expand_dims(image, 0)
|
||||
res_fp = compiled_model_fp([input_image])
|
||||
res_int8 = compiled_model_int8([input_image])
|
||||
|
||||
|
||||
# Process inference outputs and convert to binary segementation masks
|
||||
result_mask_fp = sigmoid(res_fp[output_layer_fp]).squeeze().round().astype(np.uint8)
|
||||
result_mask_int8 = sigmoid(res_int8[output_layer_int8]).squeeze().round().astype(np.uint8)
|
||||
result_mask_fp = rotate_and_flip(result_mask_fp)
|
||||
result_mask_int8 = rotate_and_flip(result_mask_int8)
|
||||
|
||||
|
||||
# Display images, annotations, FP32 result and INT8 result
|
||||
ax[i, 0].imshow(display_image, cmap=colormap)
|
||||
ax[i, 1].imshow(target_mask, cmap=colormap)
|
||||
|
|
@ -904,7 +904,7 @@ seed is displayed to enable reproducing specific runs of this cell.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Visualizing results with seed 1715032183
|
||||
Visualizing results with seed 1715809926
|
||||
|
||||
|
||||
|
||||
|
|
@ -946,7 +946,7 @@ overlay of the segmentation mask on the original image/frame.
|
|||
.. code:: ipython3
|
||||
|
||||
CASE = 117
|
||||
|
||||
|
||||
segmentation_model = SegmentationModel(ie=core, model_path=int8_ir_path, sigmoid=True, rotate_and_flip=True)
|
||||
case_path = BASEDIR / f"case_{CASE:05d}"
|
||||
image_paths = sorted(case_path.glob("imaging_frames/*jpg"))
|
||||
|
|
@ -987,8 +987,8 @@ performs inference, and displays the results on the frames loaded in
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Loaded model to AUTO in 0.23 seconds.
|
||||
Total time for 68 frames: 2.67 seconds, fps:25.86
|
||||
Loaded model to AUTO in 0.21 seconds.
|
||||
Total time for 68 frames: 2.72 seconds, fps:25.33
|
||||
|
||||
|
||||
References
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:16b3bd0c7d257596831f1d01b363b5f52edfd6db56c147651414f6ac0f77d958
|
||||
size 382301
|
||||
oid sha256:28010020834c2072a301b1a4eab4743fe594249fd6868e415af89e0dbc74892e
|
||||
size 383860
|
||||
|
|
|
|||
|
|
@ -79,9 +79,9 @@ Prerequisites
|
|||
remote: Counting objects: 100% (144/144), done.[K
|
||||
remote: Compressing objects: 100% (105/105), done.[K
|
||||
remote: Total 421 (delta 101), reused 43 (delta 39), pack-reused 277[K
|
||||
Receiving objects: 100% (421/421), 237.89 MiB | 26.48 MiB/s, done.
|
||||
Receiving objects: 100% (421/421), 237.89 MiB | 27.94 MiB/s, done.
|
||||
Resolving deltas: 100% (144/144), done.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
WARNING: typer 0.12.3 does not provide the extra 'all'
|
||||
|
|
@ -273,13 +273,13 @@ loading on device using ``core.complie_model``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/dinov2/layers/patch_embed.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/dinov2/layers/patch_embed.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert H % patch_H == 0, f"Input image height {H} is not a multiple of patch height {patch_H}"
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/dinov2/layers/patch_embed.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/dinov2/layers/patch_embed.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert W % patch_W == 0, f"Input image width {W} is not a multiple of patch width: {patch_W}"
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/vision_transformer.py:183: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/torchhub/facebookresearch_dinov2_main/vision_transformer.py:183: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if npatch == N and w == h:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/depth_anything/dpt.py:133: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/depth_anything/dpt.py:133: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
out = F.interpolate(out, (int(patch_h * 14), int(patch_w * 14)), mode="bilinear", align_corners=True)
|
||||
|
||||
|
||||
|
|
@ -571,7 +571,7 @@ Run inference on video
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Processed 60 frames in 13.33 seconds. Total FPS (including video processing): 4.50.Inference FPS: 10.52
|
||||
Processed 60 frames in 13.28 seconds. Total FPS (including video processing): 4.52.Inference FPS: 10.54
|
||||
Video saved to 'output/Coco Walking in Berkeley_depth_anything.mp4'.
|
||||
|
||||
|
||||
|
|
@ -598,7 +598,7 @@ Run inference on video
|
|||
.. parsed-literal::
|
||||
|
||||
Showing video saved at
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/output/Coco Walking in Berkeley_depth_anything.mp4
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/output/Coco Walking in Berkeley_depth_anything.mp4
|
||||
If you cannot see the video in your browser, please click on the following link to download the video
|
||||
|
||||
|
||||
|
|
@ -735,10 +735,10 @@ quantization code below may take some time.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-06 23:52:17.005825: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-06 23:52:17.039208: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-15 23:54:08.057348: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-15 23:54:08.091208: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-06 23:52:17.604619: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-15 23:54:08.654880: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
|
||||
|
|
@ -902,10 +902,10 @@ data.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Processed 60 frames in 12.84 seconds. Total FPS (including video processing): 4.67.Inference FPS: 12.79
|
||||
Processed 60 frames in 12.70 seconds. Total FPS (including video processing): 4.73.Inference FPS: 12.89
|
||||
Video saved to 'output/Coco Walking in Berkeley_depth_anything_int8.mp4'.
|
||||
Showing video saved at
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/output/Coco Walking in Berkeley_depth_anything.mp4
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/depth-anything/Depth-Anything/output/Coco Walking in Berkeley_depth_anything.mp4
|
||||
If you cannot see the video in your browser, please click on the following link to download the video
|
||||
|
||||
|
||||
|
|
@ -985,8 +985,8 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
FP16 Throughput: 10.69 FPS
|
||||
INT8 Throughput: 14.04 FPS
|
||||
FP16 Throughput: 10.63 FPS
|
||||
INT8 Throughput: 13.92 FPS
|
||||
Speed-up: 1.31
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5d10e6ad91e4f0d5c9b289d8f112f86c47143a98ce4603fcc99487dcb882d558
|
||||
size 58513
|
||||
oid sha256:402297f642d1ae24c5f34c947b243308e80a2353a297ff4fbc29a80a91447f3d
|
||||
size 57749
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1a040b682a9984544752dc46565957f9cbc02d450723acb6a9fcfd9a9963ad84
|
||||
size 509276
|
||||
oid sha256:bfdce03dd80512c030e07a93860311e984178cc6c00aea85904d97a9d85fd91b
|
||||
size 507627
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3fe0cf1ddc8c8ebbe68052a133d2d8dcc9478b4837f8b2e40a06696df7a04d52
|
||||
size 57234
|
||||
oid sha256:d467c3878c59234b1e763ec286626e3036584d037e32857396ec873b73347d3c
|
||||
size 53573
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:73da85bb37edfdfd377d92a77b9f588565f13dede569ce50d652e486f40c8bcd
|
||||
size 459409
|
||||
oid sha256:ef9d6efc8183a218693431a458cd84831e4820d02518cfd9d67f6c532ffc6f53
|
||||
size 456551
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -36,31 +36,31 @@ Imports
|
|||
.. parsed-literal::
|
||||
|
||||
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
|
||||
Requirement already satisfied: openvino>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2024.1.0)
|
||||
Requirement already satisfied: transformers in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.40.2)
|
||||
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu)
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.4)
|
||||
Requirement already satisfied: numpy<2.0.0,>=1.16.6 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (1.23.5)
|
||||
Requirement already satisfied: openvino-telemetry>=2023.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (2024.1.0)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (24.0)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (3.14.0)
|
||||
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.23.0)
|
||||
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (6.0.1)
|
||||
Requirement already satisfied: regex!=2019.12.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2024.4.28)
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2.31.0)
|
||||
Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.19.1)
|
||||
Requirement already satisfied: safetensors>=0.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.4.3)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.7)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2.2.1)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2024.2.2)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
|
||||
Requirement already satisfied: openvino>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2024.1.0)
|
||||
Requirement already satisfied: transformers in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.40.2)
|
||||
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu)
|
||||
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.4)
|
||||
Requirement already satisfied: numpy<2.0.0,>=1.16.6 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (1.23.5)
|
||||
Requirement already satisfied: openvino-telemetry>=2023.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (2024.1.0)
|
||||
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (24.0)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (3.14.0)
|
||||
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.23.0)
|
||||
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (6.0.1)
|
||||
Requirement already satisfied: regex!=2019.12.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2024.5.15)
|
||||
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2.31.0)
|
||||
Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.19.1)
|
||||
Requirement already satisfied: safetensors>=0.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.4.3)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
|
||||
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.3.2)
|
||||
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.7)
|
||||
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2.2.1)
|
||||
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2024.2.2)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
|
|
@ -90,7 +90,7 @@ model from Hugging Face.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -117,7 +117,7 @@ understand the context of a sentence. Here, we will use
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -157,9 +157,9 @@ optimal execution on end-point target devices.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||
|
||||
|
||||
|
|
@ -272,7 +272,7 @@ For a single input sentence
|
|||
.. parsed-literal::
|
||||
|
||||
Label: POSITIVE
|
||||
Total Time: 0.02 seconds
|
||||
Total Time: 0.03 seconds
|
||||
|
||||
|
||||
Read from a text file
|
||||
|
|
|
|||
|
|
@ -0,0 +1,758 @@
|
|||
Animating Open-domain Images with DynamiCrafter and OpenVINO
|
||||
============================================================
|
||||
|
||||
Animating a still image offers an engaging visual experience.
|
||||
Traditional image animation techniques mainly focus on animating natural
|
||||
scenes with stochastic dynamics (e.g. clouds and fluid) or
|
||||
domain-specific motions (e.g. human hair or body motions), and thus
|
||||
limits their applicability to more general visual content. To overcome
|
||||
this limitation, `DynamiCrafter
|
||||
team <https://doubiiu.github.io/projects/DynamiCrafter/>`__ explores the
|
||||
synthesis of dynamic content for open-domain images, converting them
|
||||
into animated videos. The key idea is to utilize the motion prior of
|
||||
text-to-video diffusion models by incorporating the image into the
|
||||
generative process as guidance. Given an image, DynamiCrafter team first
|
||||
projects it into a text-aligned rich context representation space using
|
||||
a query transformer, which facilitates the video model to digest the
|
||||
image content in a compatible fashion. However, some visual details
|
||||
still struggle to be preserved in the resultant videos. To supplement
|
||||
with more precise image information, DynamiCrafter team further feeds
|
||||
the full image to the diffusion model by concatenating it with the
|
||||
initial noises. Experimental results show that the proposed method can
|
||||
produce visually convincing and more logical & natural motions, as well
|
||||
as higher conformity to the input image.
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<table class="center">
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<tr>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td colspan="2">
|
||||
|
||||
“bear playing guitar happily, snowing”
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td colspan="2">
|
||||
|
||||
“boy walking on the street”
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</tr>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<tr>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</td>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</tr>
|
||||
|
||||
.. raw:: html
|
||||
|
||||
</table >
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Load the original model <#load-the-original-model>`__
|
||||
- `Convert the model to OpenVINO
|
||||
IR <#convert-the-model-to-openvino-ir>`__
|
||||
|
||||
- `Convert CLIP text encoder <#convert-clip-text-encoder>`__
|
||||
- `Convert CLIP image encoder <#convert-clip-image-encoder>`__
|
||||
- `Convert AE encoder <#convert-ae-encoder>`__
|
||||
- `Convert Diffusion U-Net model <#convert-diffusion-u-net-model>`__
|
||||
- `Convert AE decoder <#convert-ae-decoder>`__
|
||||
|
||||
- `Compiling models <#compiling-models>`__
|
||||
- `Building the pipeline <#building-the-pipeline>`__
|
||||
- `Interactive inference <#interactive-inference>`__
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
%pip uninstall -q -y openvino-dev openvino openvino-nightly
|
||||
%pip install -q --upgrade openvino-nightly
|
||||
%pip install -q "gradio>=4.19" omegaconf decord einops pytorch_lightning kornia open_clip_torch transformers av opencv-python torch --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
WARNING: Skipping openvino-nightly as it is not installed.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
dynamicrafter_path = Path("dynamicrafter")
|
||||
|
||||
if not dynamicrafter_path.exists():
|
||||
dynamicrafter_path.mkdir(parents=True, exist_ok=True)
|
||||
!git clone https://github.com/Doubiiu/DynamiCrafter.git dynamicrafter
|
||||
|
||||
sys.path.append(str(dynamicrafter_path))
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Cloning into 'dynamicrafter'...
|
||||
remote: Enumerating objects: 288, done.[K
|
||||
remote: Counting objects: 100% (125/125), done.[K
|
||||
remote: Compressing objects: 100% (71/71), done.[K
|
||||
remote: Total 288 (delta 74), reused 71 (delta 54), pack-reused 163
|
||||
Receiving objects: 100% (288/288), 72.40 MiB | 24.98 MiB/s, done.
|
||||
Resolving deltas: 100% (89/89), done.
|
||||
|
||||
|
||||
Load and run the original pipeline
|
||||
----------------------------------
|
||||
|
||||
|
||||
|
||||
We will use model for 256x256 resolution as example. Also, models for
|
||||
320x512 and 576x1024 are
|
||||
`available <https://github.com/Doubiiu/DynamiCrafter?tab=readme-ov-file#-models>`__.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import os
|
||||
|
||||
from huggingface_hub import hf_hub_download
|
||||
from omegaconf import OmegaConf
|
||||
|
||||
from dynamicrafter.scripts.evaluation.funcs import load_model_checkpoint
|
||||
from dynamicrafter.utils.utils import instantiate_from_config
|
||||
|
||||
|
||||
def download_model():
|
||||
REPO_ID = "Doubiiu/DynamiCrafter"
|
||||
if not os.path.exists("./checkpoints/dynamicrafter_256_v1/"):
|
||||
os.makedirs("./checkpoints/dynamicrafter_256_v1/")
|
||||
local_file = os.path.join("./checkpoints/dynamicrafter_256_v1/model.ckpt")
|
||||
if not os.path.exists(local_file):
|
||||
hf_hub_download(repo_id=REPO_ID, filename="model.ckpt", local_dir="./checkpoints/dynamicrafter_256_v1/", local_dir_use_symlinks=False)
|
||||
|
||||
ckpt_path = "checkpoints/dynamicrafter_256_v1/model.ckpt"
|
||||
config_file = "dynamicrafter/configs/inference_256_v1.0.yaml"
|
||||
config = OmegaConf.load(config_file)
|
||||
model_config = config.pop("model", OmegaConf.create())
|
||||
model_config["params"]["unet_config"]["params"]["use_checkpoint"] = False
|
||||
model = instantiate_from_config(model_config)
|
||||
model = load_model_checkpoint(model, ckpt_path)
|
||||
model.eval()
|
||||
|
||||
return model
|
||||
|
||||
|
||||
model = download_model()
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.
|
||||
For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
model.ckpt: 0%| | 0.00/10.4G [00:00<?, ?B/s]
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
AE working on z of shape (1, 4, 32, 32) = 4096 dimensions.
|
||||
>>> model checkpoint loaded.
|
||||
|
||||
|
||||
Convert the model to OpenVINO IR
|
||||
--------------------------------
|
||||
|
||||
|
||||
|
||||
Let’s define the conversion function for PyTorch modules. We use
|
||||
``ov.convert_model`` function to obtain OpenVINO Intermediate
|
||||
Representation object and ``ov.save_model`` function to save it as XML
|
||||
file.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import torch
|
||||
|
||||
import openvino as ov
|
||||
|
||||
|
||||
def convert(model: torch.nn.Module, xml_path: str, example_input, input_shape=None):
|
||||
xml_path = Path(xml_path)
|
||||
if not xml_path.exists():
|
||||
xml_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with torch.no_grad():
|
||||
if not input_shape:
|
||||
converted_model = ov.convert_model(model, example_input=example_input)
|
||||
else:
|
||||
converted_model = ov.convert_model(model, example_input=example_input, input=input_shape)
|
||||
ov.save_model(converted_model, xml_path, compress_to_fp16=False)
|
||||
|
||||
# cleanup memory
|
||||
torch._C._jit_clear_class_registry()
|
||||
torch.jit._recursive.concrete_type_store = torch.jit._recursive.ConcreteTypeStore()
|
||||
torch.jit._state._clear_class_state()
|
||||
|
||||
Flowchart of DynamiCrafter proposed in `the
|
||||
paper <https://arxiv.org/abs/2310.12190>`__:
|
||||
|
||||
|schema| Description: > During inference, our model can generate
|
||||
animation clips from noise conditioned on the input still image.
|
||||
|
||||
Let’s convert models from the pipeline one by one.
|
||||
|
||||
.. |schema| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/76171391/d1033876-c664-4345-a254-0649edbf1906
|
||||
|
||||
Convert CLIP text encoder
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
from dynamicrafter.lvdm.modules.encoders.condition import FrozenOpenCLIPEmbedder
|
||||
|
||||
|
||||
COND_STAGE_MODEL_OV_PATH = Path("models/cond_stage_model.xml")
|
||||
|
||||
|
||||
class FrozenOpenCLIPEmbedderWrapper(FrozenOpenCLIPEmbedder):
|
||||
|
||||
def forward(self, tokens):
|
||||
z = self.encode_with_transformer(tokens.to(self.device))
|
||||
return z
|
||||
|
||||
|
||||
cond_stage_model = FrozenOpenCLIPEmbedderWrapper(device="cpu")
|
||||
|
||||
|
||||
convert(
|
||||
cond_stage_model,
|
||||
COND_STAGE_MODEL_OV_PATH,
|
||||
example_input=torch.ones([1, 77], dtype=torch.long),
|
||||
)
|
||||
|
||||
Convert CLIP image encoder
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
``FrozenOpenCLIPImageEmbedderV2`` model accepts images of various
|
||||
resolutions.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
EMBEDDER_OV_PATH = Path("models/embedder_ir.xml")
|
||||
|
||||
|
||||
dummy_input = torch.rand([1, 3, 767, 767], dtype=torch.float32)
|
||||
|
||||
model.embedder.model.visual.input_patchnorm = None # fix error: visual model has not attribute 'input_patchnorm'
|
||||
convert(model.embedder, EMBEDDER_OV_PATH, example_input=dummy_input, input_shape=[1, 3, -1, -1])
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/utils/image.py:226: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input.numel() == 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:573: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if size == input_size:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:579: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
antialias = antialias and (max(factors) > 1)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:581: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if antialias:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:584: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
sigmas = (max((factors[0] - 1.0) / 2.0, 0.001), max((factors[1] - 1.0) / 2.0, 0.001))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
sigma = tensor([sigma], device=input.device, dtype=input.dtype)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
sigma = tensor([sigma], device=input.device, dtype=input.dtype)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/core/check.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if x_shape_to_check[i] != dim:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/kernels.py:92: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mean = tensor([[mean]], device=sigma.device, dtype=sigma.dtype)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if len(mean.shape) == 0 or mean.shape[0] == 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:103: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if len(std.shape) == 0 or std.shape[0] == 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:107: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if mean.shape and mean.shape[0] != 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:108: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if mean.shape[0] != data.shape[1] and mean.shape[:2] != data.shape[:2]:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:112: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if std.shape and std.shape[0] != 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:113: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if std.shape[0] != data.shape[1] and std.shape[:2] != data.shape[:2]:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:116: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mean = torch.as_tensor(mean, device=data.device, dtype=data.dtype)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:117: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
std = torch.as_tensor(std, device=data.device, dtype=data.dtype)
|
||||
|
||||
|
||||
Convert AE encoder
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
ENCODER_FIRST_STAGE_OV_PATH = Path("models/encode_first_stage_ir.xml")
|
||||
|
||||
|
||||
dummy_input = torch.rand([1, 3, 256, 256], dtype=torch.float32)
|
||||
|
||||
convert(
|
||||
model.first_stage_model.encoder,
|
||||
ENCODER_FIRST_STAGE_OV_PATH,
|
||||
example_input=dummy_input,
|
||||
)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/ae_modules.py:67: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
w_ = w_ * (int(c)**(-0.5))
|
||||
|
||||
|
||||
Convert Diffusion U-Net model
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
MODEL_OV_PATH = Path("models/model_ir.xml")
|
||||
|
||||
|
||||
class ModelWrapper(torch.nn.Module):
|
||||
def __init__(self, diffusion_model):
|
||||
super().__init__()
|
||||
self.diffusion_model = diffusion_model
|
||||
|
||||
def forward(self, xc, t, context=None, fs=None, temporal_length=None):
|
||||
outputs = self.diffusion_model(xc, t, context=context, fs=fs, temporal_length=temporal_length)
|
||||
return outputs
|
||||
|
||||
|
||||
convert(
|
||||
ModelWrapper(model.model.diffusion_model),
|
||||
MODEL_OV_PATH,
|
||||
example_input={
|
||||
"xc": torch.rand([1, 8, 16, 32, 32], dtype=torch.float32),
|
||||
"t": torch.tensor([1]),
|
||||
"context": torch.rand([1, 333, 1024], dtype=torch.float32),
|
||||
"fs": torch.tensor([3]),
|
||||
"temporal_length": torch.tensor([16]),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:556: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if l_context == 77 + t*16: ## !!! HARD CODE here
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:205: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if batch_size:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:232: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if self.use_temporal_conv and batch_size:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:76: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert x.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:99: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert x.shape[1] == self.channels
|
||||
|
||||
|
||||
Convert AE decoder
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
``Decoder`` receives a
|
||||
``bfloat16`` tensor. numpy doesn’t support this type. To avoid problems
|
||||
with the conversion lets replace ``decode`` method to convert bfloat16
|
||||
to float32.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import types
|
||||
|
||||
|
||||
def decode(self, z, **kwargs):
|
||||
z = self.post_quant_conv(z)
|
||||
z = z.float()
|
||||
dec = self.decoder(z)
|
||||
return dec
|
||||
|
||||
|
||||
model.first_stage_model.decode = types.MethodType(decode, model.first_stage_model)
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
DECODER_FIRST_STAGE_OV_PATH = Path("models/decoder_first_stage_ir.xml")
|
||||
|
||||
|
||||
dummy_input = torch.rand([16, 4, 32, 32], dtype=torch.float32)
|
||||
|
||||
convert(
|
||||
model.first_stage_model.decoder,
|
||||
DECODER_FIRST_STAGE_OV_PATH,
|
||||
example_input=dummy_input,
|
||||
)
|
||||
|
||||
Compiling models
|
||||
----------------
|
||||
|
||||
|
||||
|
||||
Select device from dropdown list for running inference using OpenVINO.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
core = ov.Core()
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
value="AUTO",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO')
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
compiled_cond_stage_model = core.compile_model(COND_STAGE_MODEL_OV_PATH, device.value)
|
||||
compiled_encode_first_stage = core.compile_model(ENCODER_FIRST_STAGE_OV_PATH, device.value)
|
||||
compiled_embedder = core.compile_model(EMBEDDER_OV_PATH, device.value)
|
||||
compiled_model = core.compile_model(MODEL_OV_PATH, device.value)
|
||||
compiled_decoder_first_stage = core.compile_model(DECODER_FIRST_STAGE_OV_PATH, device.value)
|
||||
|
||||
Building the pipeline
|
||||
---------------------
|
||||
|
||||
|
||||
|
||||
Let’s create callable wrapper classes for compiled models to allow
|
||||
interaction with original pipelines. Note that all of wrapper classes
|
||||
return ``torch.Tensor``\ s instead of ``np.array``\ s.
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import open_clip
|
||||
|
||||
|
||||
class CondStageModelWrapper(torch.nn.Module):
|
||||
def __init__(self, cond_stage_model):
|
||||
super().__init__()
|
||||
self.cond_stage_model = cond_stage_model
|
||||
|
||||
def encode(self, tokens):
|
||||
if isinstance(tokens, list):
|
||||
tokens = open_clip.tokenize(tokens[0])
|
||||
outs = self.cond_stage_model(tokens)[0]
|
||||
|
||||
return torch.from_numpy(outs)
|
||||
|
||||
|
||||
class EncoderFirstStageModelWrapper(torch.nn.Module):
|
||||
def __init__(self, encode_first_stage):
|
||||
super().__init__()
|
||||
self.encode_first_stage = encode_first_stage
|
||||
|
||||
def forward(self, x):
|
||||
outs = self.encode_first_stage(x)[0]
|
||||
|
||||
return torch.from_numpy(outs)
|
||||
|
||||
|
||||
class EmbedderWrapper(torch.nn.Module):
|
||||
def __init__(self, embedder):
|
||||
super().__init__()
|
||||
self.embedder = embedder
|
||||
|
||||
def forward(self, x):
|
||||
outs = self.embedder(x)[0]
|
||||
|
||||
return torch.from_numpy(outs)
|
||||
|
||||
|
||||
class CModelWrapper(torch.nn.Module):
|
||||
def __init__(self, diffusion_model, out_channels):
|
||||
super().__init__()
|
||||
self.diffusion_model = diffusion_model
|
||||
self.out_channels = out_channels
|
||||
|
||||
def forward(self, xc, t, context, fs, temporal_length):
|
||||
inputs = {
|
||||
"xc": xc,
|
||||
"t": t,
|
||||
"context": context,
|
||||
"fs": fs,
|
||||
}
|
||||
outs = self.diffusion_model(inputs)[0]
|
||||
|
||||
return torch.from_numpy(outs)
|
||||
|
||||
|
||||
class DecoderFirstStageModelWrapper(torch.nn.Module):
|
||||
def __init__(self, decoder_first_stage):
|
||||
super().__init__()
|
||||
self.decoder_first_stage = decoder_first_stage
|
||||
|
||||
def forward(self, x):
|
||||
x.float()
|
||||
outs = self.decoder_first_stage(x)[0]
|
||||
|
||||
return torch.from_numpy(outs)
|
||||
|
||||
And insert wrappers instances in the pipeline:
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
model.cond_stage_model = CondStageModelWrapper(compiled_cond_stage_model)
|
||||
model.first_stage_model.encoder = EncoderFirstStageModelWrapper(compiled_encode_first_stage)
|
||||
model.embedder = EmbedderWrapper(compiled_embedder)
|
||||
model.model.diffusion_model = CModelWrapper(compiled_model, model.model.diffusion_model.out_channels)
|
||||
model.first_stage_model.decoder = DecoderFirstStageModelWrapper(compiled_decoder_first_stage)
|
||||
|
||||
Interactive inference
|
||||
---------------------
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import time
|
||||
|
||||
from einops import repeat
|
||||
from pytorch_lightning import seed_everything
|
||||
import torchvision.transforms as transforms
|
||||
|
||||
from dynamicrafter.scripts.evaluation.funcs import save_videos, batch_ddim_sampling, get_latent_z
|
||||
from dynamicrafter.lvdm.models.samplers.ddim import DDIMSampler
|
||||
|
||||
|
||||
def register_buffer(self, name, attr):
|
||||
if isinstance(attr, torch.Tensor):
|
||||
if attr.device != torch.device("cpu"):
|
||||
attr = attr.to(torch.device("cpu"))
|
||||
setattr(self, name, attr)
|
||||
|
||||
|
||||
# monkey patching to replace the original method 'register_buffer' that uses CUDA
|
||||
DDIMSampler.register_buffer = types.MethodType(register_buffer, DDIMSampler)
|
||||
|
||||
|
||||
def get_image(image, prompt, steps=5, cfg_scale=7.5, eta=1.0, fs=3, seed=123, model=model):
|
||||
result_dir = "results"
|
||||
if not os.path.exists(result_dir):
|
||||
os.mkdir(result_dir)
|
||||
|
||||
seed_everything(seed)
|
||||
transform = transforms.Compose(
|
||||
[
|
||||
transforms.Resize(min((256, 256))),
|
||||
transforms.CenterCrop((256, 256)),
|
||||
]
|
||||
)
|
||||
# torch.cuda.empty_cache()
|
||||
print("start:", prompt, time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(time.time())))
|
||||
start = time.time()
|
||||
if steps > 60:
|
||||
steps = 60
|
||||
model = model.cpu()
|
||||
batch_size = 1
|
||||
channels = model.model.diffusion_model.out_channels
|
||||
frames = model.temporal_length
|
||||
h, w = 256 // 8, 256 // 8
|
||||
noise_shape = [batch_size, channels, frames, h, w]
|
||||
|
||||
# text cond
|
||||
with torch.no_grad(), torch.cpu.amp.autocast():
|
||||
text_emb = model.get_learned_conditioning([prompt])
|
||||
|
||||
# img cond
|
||||
img_tensor = torch.from_numpy(image).permute(2, 0, 1).float().to(model.device)
|
||||
img_tensor = (img_tensor / 255.0 - 0.5) * 2
|
||||
|
||||
image_tensor_resized = transform(img_tensor) # 3,h,w
|
||||
videos = image_tensor_resized.unsqueeze(0) # bchw
|
||||
|
||||
z = get_latent_z(model, videos.unsqueeze(2)) # bc,1,hw
|
||||
|
||||
img_tensor_repeat = repeat(z, "b c t h w -> b c (repeat t) h w", repeat=frames)
|
||||
|
||||
cond_images = model.embedder(img_tensor.unsqueeze(0)) # blc
|
||||
|
||||
img_emb = model.image_proj_model(cond_images)
|
||||
|
||||
imtext_cond = torch.cat([text_emb, img_emb], dim=1)
|
||||
|
||||
fs = torch.tensor([fs], dtype=torch.long, device=model.device)
|
||||
cond = {"c_crossattn": [imtext_cond], "fs": fs, "c_concat": [img_tensor_repeat]}
|
||||
|
||||
## inference
|
||||
batch_samples = batch_ddim_sampling(model, cond, noise_shape, n_samples=1, ddim_steps=steps, ddim_eta=eta, cfg_scale=cfg_scale)
|
||||
## b,samples,c,t,h,w
|
||||
prompt_str = prompt.replace("/", "_slash_") if "/" in prompt else prompt
|
||||
prompt_str = prompt_str.replace(" ", "_") if " " in prompt else prompt_str
|
||||
prompt_str = prompt_str[:40]
|
||||
if len(prompt_str) == 0:
|
||||
prompt_str = "empty_prompt"
|
||||
|
||||
save_videos(batch_samples, result_dir, filenames=[prompt_str], fps=8)
|
||||
print(f"Saved in {prompt_str}. Time used: {(time.time() - start):.2f} seconds")
|
||||
|
||||
return os.path.join(result_dir, f"{prompt_str}.mp4")
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
import gradio as gr
|
||||
|
||||
|
||||
i2v_examples_256 = [
|
||||
["dynamicrafter/prompts/256/art.png", "man fishing in a boat at sunset", 50, 7.5, 1.0, 3, 234],
|
||||
["dynamicrafter/prompts/256/boy.png", "boy walking on the street", 50, 7.5, 1.0, 3, 125],
|
||||
["dynamicrafter/prompts/256/dance1.jpeg", "two people dancing", 50, 7.5, 1.0, 3, 116],
|
||||
["dynamicrafter/prompts/256/fire_and_beach.jpg", "a campfire on the beach and the ocean waves in the background", 50, 7.5, 1.0, 3, 111],
|
||||
["dynamicrafter/prompts/256/guitar0.jpeg", "bear playing guitar happily, snowing", 50, 7.5, 1.0, 3, 122],
|
||||
]
|
||||
|
||||
|
||||
def dynamicrafter_demo():
|
||||
css = """#input_img {max-width: 256px !important} #output_vid {max-width: 256px; max-height: 256px}"""
|
||||
|
||||
with gr.Blocks(analytics_enabled=False, css=css) as dynamicrafter_iface:
|
||||
with gr.Tab(label="Image2Video_256x256"):
|
||||
with gr.Column():
|
||||
with gr.Row():
|
||||
with gr.Column():
|
||||
with gr.Row():
|
||||
i2v_input_image = gr.Image(label="Input Image", elem_id="input_img")
|
||||
with gr.Row():
|
||||
i2v_input_text = gr.Text(label="Prompts")
|
||||
with gr.Row():
|
||||
i2v_seed = gr.Slider(label="Random Seed", minimum=0, maximum=10000, step=1, value=123)
|
||||
i2v_eta = gr.Slider(minimum=0.0, maximum=1.0, step=0.1, label="ETA", value=1.0, elem_id="i2v_eta")
|
||||
i2v_cfg_scale = gr.Slider(minimum=1.0, maximum=15.0, step=0.5, label="CFG Scale", value=7.5, elem_id="i2v_cfg_scale")
|
||||
with gr.Row():
|
||||
i2v_steps = gr.Slider(minimum=1, maximum=60, step=1, elem_id="i2v_steps", label="Sampling steps", value=50)
|
||||
i2v_motion = gr.Slider(minimum=1, maximum=4, step=1, elem_id="i2v_motion", label="Motion magnitude", value=3)
|
||||
i2v_end_btn = gr.Button("Generate")
|
||||
with gr.Row():
|
||||
i2v_output_video = gr.Video(label="Generated Video", elem_id="output_vid", autoplay=True, show_share_button=True)
|
||||
|
||||
gr.Examples(
|
||||
examples=i2v_examples_256,
|
||||
inputs=[i2v_input_image, i2v_input_text, i2v_steps, i2v_cfg_scale, i2v_eta, i2v_motion, i2v_seed],
|
||||
outputs=[i2v_output_video],
|
||||
fn=get_image,
|
||||
cache_examples=False,
|
||||
)
|
||||
i2v_end_btn.click(
|
||||
inputs=[i2v_input_image, i2v_input_text, i2v_steps, i2v_cfg_scale, i2v_eta, i2v_motion, i2v_seed],
|
||||
outputs=[i2v_output_video],
|
||||
fn=get_image,
|
||||
)
|
||||
|
||||
return dynamicrafter_iface
|
||||
|
||||
|
||||
demo = dynamicrafter_demo()
|
||||
|
||||
|
||||
try:
|
||||
demo.queue().launch(debug=False)
|
||||
except Exception:
|
||||
demo.queue().launch(debug=False, share=True)
|
||||
# if you are launching remotely, specify server_name and server_port
|
||||
# demo.launch(server_name='your server name', server_port='server port in int')
|
||||
# Read more in the docs: https://gradio.app/docs/
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Running on local URL: http://127.0.0.1:7860
|
||||
|
||||
To create a public link, set `share=True` in `launch()`.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -84,7 +84,6 @@ Prerequisites
|
|||
.. parsed-literal::
|
||||
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
WARNING: typer 0.12.3 does not provide the extra 'all'
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
|
|
@ -106,9 +105,9 @@ Prerequisites
|
|||
remote: Counting objects: 100% (85/85), done.[K
|
||||
remote: Compressing objects: 100% (33/33), done.[K
|
||||
remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339[K
|
||||
Receiving objects: 100% (424/424), 262.14 MiB | 24.49 MiB/s, done.
|
||||
Receiving objects: 100% (424/424), 262.14 MiB | 26.62 MiB/s, done.
|
||||
Resolving deltas: 100% (246/246), done.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM
|
||||
|
||||
|
||||
Load PyTorch model
|
||||
|
|
@ -363,23 +362,23 @@ disk using ``openvino.save_model``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:220: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:220: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if (
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:241: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:241: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert (
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:163: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:163: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
size = int(math.sqrt(xy_num))
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:164: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:164: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert size * size == xy_num
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:166: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:166: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if size != h or size != w:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:251: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:251: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert x.shape[2] == num_patches
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if num_pts > self.decoder_max_num_input_points:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:92: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:92: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
elif num_pts < self.decoder_max_num_input_points:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:126: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:126: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if output_w > 0 and output_h > 0:
|
||||
|
||||
|
||||
|
|
@ -643,10 +642,10 @@ architecture type, we should specify ``transformer`` in ``model_type``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:01:29.399785: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:01:29.433596: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:09:26.533883: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:09:26.567663: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:01:30.076398: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:09:27.185648: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
|
||||
|
|
@ -851,7 +850,7 @@ models, we use ``bencmark_app``.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 40.36 ms
|
||||
[ INFO ] Read model took 40.50 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?]
|
||||
|
|
@ -871,7 +870,7 @@ models, we use ``bencmark_app``.
|
|||
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?]
|
||||
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 1355.34 ms
|
||||
[ INFO ] Compile model took 1359.16 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -912,17 +911,17 @@ models, we use ``bencmark_app``.
|
|||
[ INFO ] Fill input 'batched_point_labels' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop).
|
||||
[ INFO ] First inference took 645.58 ms
|
||||
[ INFO ] First inference took 644.87 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 49 iterations
|
||||
[ INFO ] Duration: 15813.12 ms
|
||||
[ INFO ] Duration: 15741.55 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 1908.46 ms
|
||||
[ INFO ] Average: 1882.77 ms
|
||||
[ INFO ] Min: 609.20 ms
|
||||
[ INFO ] Max: 1971.02 ms
|
||||
[ INFO ] Throughput: 3.10 FPS
|
||||
[ INFO ] Median: 1892.64 ms
|
||||
[ INFO ] Average: 1876.78 ms
|
||||
[ INFO ] Min: 664.09 ms
|
||||
[ INFO ] Max: 2004.53 ms
|
||||
[ INFO ] Throughput: 3.11 FPS
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -948,7 +947,7 @@ models, we use ``bencmark_app``.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 53.68 ms
|
||||
[ INFO ] Read model took 54.09 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?]
|
||||
|
|
@ -968,7 +967,7 @@ models, we use ``bencmark_app``.
|
|||
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?]
|
||||
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 1854.44 ms
|
||||
[ INFO ] Compile model took 1870.95 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -1009,17 +1008,17 @@ models, we use ``bencmark_app``.
|
|||
[ INFO ] Fill input 'batched_point_labels' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop).
|
||||
[ INFO ] First inference took 596.95 ms
|
||||
[ INFO ] First inference took 583.07 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 55 iterations
|
||||
[ INFO ] Duration: 16443.47 ms
|
||||
[ INFO ] Duration: 16526.72 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 1775.29 ms
|
||||
[ INFO ] Average: 1756.44 ms
|
||||
[ INFO ] Min: 630.26 ms
|
||||
[ INFO ] Max: 1849.40 ms
|
||||
[ INFO ] Throughput: 3.34 FPS
|
||||
[ INFO ] Median: 1791.04 ms
|
||||
[ INFO ] Average: 1766.59 ms
|
||||
[ INFO ] Min: 586.80 ms
|
||||
[ INFO ] Max: 1852.64 ms
|
||||
[ INFO ] Throughput: 3.33 FPS
|
||||
|
||||
|
||||
Interactive segmentation demo
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f283354c61f2d5c9aea2fce146bd130ae9133059c0d90b6fadea3cb6207466da
|
||||
size 1261054
|
||||
oid sha256:6023d12350dd3ca910572f4dad89f563abcd31328ac27846f5ddc64705b4daf5
|
||||
size 1260186
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c36f8c8a366405e85e109c43647bb0cf0f0a92d10d7ddeb70c19e87a6818485b
|
||||
size 1259311
|
||||
oid sha256:ac4eb9a2fe78ace641f00796a0d59f9383c3ee6904c2fa69a6e2eec1af1e779e
|
||||
size 1259619
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0979318f5d2a5bdb8e7deac91c7ab319d24efcca8be6e5612c4dca27a802ca95
|
||||
size 1262900
|
||||
oid sha256:faa1d90c28aeb3a308002ab6430ef3f26c078f561f1e776770f2461bed61b074
|
||||
size 1262279
|
||||
|
|
|
|||
|
|
@ -129,7 +129,7 @@ bandwidth.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
|
||||
warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
|
||||
|
||||
|
||||
|
|
@ -289,7 +289,7 @@ similar as possible to the original.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
|
||||
warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
|
||||
|
||||
|
||||
|
|
@ -389,13 +389,13 @@ with ``ov.save_model``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:60: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:60: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
ideal_length = (math.ceil(n_frames) - 1) * stride + (kernel_size - padding_total)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert padding_left >= 0 and padding_right >= 0, (padding_left, padding_right)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
max_pad = max(padding_left, padding_right)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:89: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:89: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if length <= max_pad:
|
||||
|
||||
|
||||
|
|
@ -415,11 +415,11 @@ with ``ov.save_model``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/quantization/core_vq.py:358: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/quantization/core_vq.py:358: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
quantized_out = torch.tensor(0.0, device=q_indices.device)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/quantization/core_vq.py:359: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/quantization/core_vq.py:359: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
|
||||
for i, indices in enumerate(q_indices):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:103: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/encodec/modules/conv.py:103: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert (padding_left + padding_right) <= x.shape[-1]
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -145,7 +145,7 @@ model and generate a segmentation map.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
100%|██████████| 138M/138M [00:01<00:00, 95.8MB/s]
|
||||
100%|██████████| 138M/138M [00:03<00:00, 45.4MB/s]
|
||||
|
||||
|
||||
|
||||
|
|
@ -157,8 +157,8 @@ model and generate a segmentation map.
|
|||
.. parsed-literal::
|
||||
|
||||
|
||||
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 768x1024 37 objects, 628.0ms
|
||||
Speed: 3.4ms preprocess, 628.0ms inference, 27.9ms postprocess per image at shape (1, 3, 768, 1024)
|
||||
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 768x1024 37 objects, 655.2ms
|
||||
Speed: 4.0ms preprocess, 655.2ms inference, 30.8ms postprocess per image at shape (1, 3, 768, 1024)
|
||||
|
||||
|
||||
The model returns segmentation maps for all the objects on the image.
|
||||
|
|
@ -201,10 +201,10 @@ tracing. The FastSAM model itself is based on YOLOv8 model.
|
|||
PyTorch: starting from 'FastSAM-x.pt' with input shape (1, 3, 1024, 1024) BCHW and output shape(s) ((1, 37, 21504), (1, 32, 256, 256)) (138.2 MB)
|
||||
|
||||
OpenVINO: starting export with openvino 2024.1.0-15008-f4afc983258-releases/2024/1...
|
||||
OpenVINO: export success ✅ 6.1s, saved as 'FastSAM-x_openvino_model/' (276.1 MB)
|
||||
OpenVINO: export success ✅ 6.2s, saved as 'FastSAM-x_openvino_model/' (276.1 MB)
|
||||
|
||||
Export complete (9.0s)
|
||||
Results saved to /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/fast-segment-anything
|
||||
Export complete (9.2s)
|
||||
Results saved to /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything
|
||||
Predict: yolo predict task=segment model=FastSAM-x_openvino_model imgsz=1024
|
||||
Validate: yolo val task=segment model=FastSAM-x_openvino_model imgsz=1024 data=ultralytics/datasets/sa.yaml
|
||||
Visualize: https://netron.app
|
||||
|
|
@ -313,8 +313,8 @@ pipeline.
|
|||
.. parsed-literal::
|
||||
|
||||
|
||||
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 1024x1024 42 objects, 510.3ms
|
||||
Speed: 6.8ms preprocess, 510.3ms inference, 34.2ms postprocess per image at shape (1, 3, 1024, 1024)
|
||||
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 1024x1024 42 objects, 512.0ms
|
||||
Speed: 6.5ms preprocess, 512.0ms inference, 29.3ms postprocess per image at shape (1, 3, 1024, 1024)
|
||||
|
||||
|
||||
One can observe the converted model outputs in the next cell, they is
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -48,14 +48,12 @@ Clone repositories and install requirements
|
|||
|
||||
.. code:: ipython3
|
||||
|
||||
%pip install -q timm --extra-index-url https://download.pytorch.org/whl/cpu # is needed for torch
|
||||
%pip install -q "openvino>=2024.0" "torch>=2.1" opencv-python supervision transformers yapf pycocotools addict "gradio>=4.19" tqdm
|
||||
%pip install -q "openvino>=2024.0" "torch>=2.1" opencv-python supervision transformers yapf pycocotools addict "gradio>=4.19" tqdm timm --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
For faster computation and to limit RAM by default we use
|
||||
|
|
@ -113,17 +111,17 @@ segmentation you can select vanilla ``SAM``.
|
|||
|
||||
Cloning into 'GroundingDINO'...
|
||||
remote: Enumerating objects: 379, done.[K
|
||||
remote: Counting objects: 100% (176/176), done.[K
|
||||
remote: Compressing objects: 100% (65/65), done.[K
|
||||
remote: Total 379 (delta 136), reused 111 (delta 111), pack-reused 203[K
|
||||
Receiving objects: 100% (379/379), 14.03 MiB | 20.06 MiB/s, done.
|
||||
Resolving deltas: 100% (195/195), done.
|
||||
remote: Counting objects: 100% (175/175), done.[K
|
||||
remote: Compressing objects: 100% (63/63), done.[K
|
||||
remote: Total 379 (delta 135), reused 112 (delta 112), pack-reused 204[K
|
||||
Receiving objects: 100% (379/379), 14.03 MiB | 27.68 MiB/s, done.
|
||||
Resolving deltas: 100% (194/194), done.
|
||||
Cloning into 'EfficientSAM'...
|
||||
remote: Enumerating objects: 424, done.[K
|
||||
remote: Counting objects: 100% (85/85), done.[K
|
||||
remote: Compressing objects: 100% (33/33), done.[K
|
||||
remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339[K
|
||||
Receiving objects: 100% (424/424), 262.14 MiB | 28.00 MiB/s, done.
|
||||
Receiving objects: 100% (424/424), 262.14 MiB | 24.66 MiB/s, done.
|
||||
Resolving deltas: 100% (246/246), done.
|
||||
|
||||
|
||||
|
|
@ -360,24 +358,6 @@ Convert GroundingDINO to OpenVINO IR format
|
|||
TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
output layer_id 0 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
output layer_id 1 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
output layer_id 2 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
output layer_id 3 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
output layer_id 4 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
output layer_id 5 is nan
|
||||
num_nan 230400, num_inf 0
|
||||
|
||||
|
||||
Run OpenVINO optimized GroundingDINO
|
||||
|
|
@ -527,10 +507,10 @@ class, but the inference will be done using OpenVINO optimized model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:14:36.448862: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:14:36.488990: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:24:34.510973: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:24:34.551326: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:14:37.051985: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:24:35.297555: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
Convert predicted boxes to supervision box detections format
|
||||
|
|
|
|||
|
|
@ -194,7 +194,7 @@ is provided.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f34a03f2a60>
|
||||
<matplotlib.image.AxesImage at 0x7fc77535e280>
|
||||
|
||||
|
||||
|
||||
|
|
@ -221,7 +221,7 @@ Do Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f34a0321cd0>
|
||||
<matplotlib.image.AxesImage at 0x7fc76c77f280>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -121,7 +121,7 @@ tutorials <https://huggingface.co/learn/nlp-course/chapter2/2?fw=pt#behind-the-p
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
|
||||
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
|
||||
|
|
@ -187,7 +187,7 @@ Note how we reuse our real ``encoded_input``, passing it to the
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -325,11 +325,11 @@ documentation <https://huggingface.co/docs/optimum/intel/inference>`__.
|
|||
To disable this warning, you can either:
|
||||
- Avoid using `tokenizers` before the fork if possible
|
||||
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
|
||||
2024-05-07 00:16:16.370934: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:16:16.406467: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:26:16.053025: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:26:16.088586: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:16:17.026247: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
2024-05-16 00:26:16.708352: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
|
||||
|
||||
|
|
@ -366,7 +366,7 @@ inference run.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Framework not specified. Using pt to export the model.
|
||||
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
|
||||
|
|
@ -384,7 +384,7 @@ inference run.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
Compiling the model to AUTO ...
|
||||
|
||||
|
|
@ -440,7 +440,7 @@ Full list of supported arguments available via ``--help``
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:16:28.374572: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:26:28.476626: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
usage: optimum-cli export openvino [-h] -m MODEL [--task TASK]
|
||||
[--cache_dir CACHE_DIR]
|
||||
[--framework {pt,tf}] [--trust-remote-code]
|
||||
|
|
@ -469,21 +469,20 @@ Full list of supported arguments available via ``--help``
|
|||
--task TASK The task to export the model for. If not specified,
|
||||
the task will be auto-inferred based on the model.
|
||||
Available tasks depend on the model, but are among:
|
||||
['feature-extraction', 'zero-shot-image-
|
||||
classification', 'conversational', 'text2text-
|
||||
generation', 'sentence-similarity', 'stable-
|
||||
diffusion', 'text-to-audio', 'stable-diffusion-xl',
|
||||
'fill-mask', 'image-to-text', 'text-generation',
|
||||
'semantic-segmentation', 'automatic-speech-
|
||||
recognition', 'mask-generation', 'token-
|
||||
classification', 'audio-classification', 'multiple-
|
||||
choice', 'question-answering', 'masked-im', 'zero-
|
||||
shot-object-detection', 'audio-xvector', 'image-
|
||||
segmentation', 'object-detection', 'image-
|
||||
classification', 'audio-frame-classification', 'image-
|
||||
to-image', 'depth-estimation', 'text-classification'].
|
||||
For decoder models, use `xxx-with-past` to export the
|
||||
model using past key values in the decoder.
|
||||
['sentence-similarity', 'fill-mask', 'semantic-
|
||||
segmentation', 'depth-estimation', 'zero-shot-image-
|
||||
classification', 'zero-shot-object-detection', 'audio-
|
||||
frame-classification', 'token-classification', 'image-
|
||||
segmentation', 'question-answering', 'text2text-
|
||||
generation', 'image-classification', 'text-to-audio',
|
||||
'stable-diffusion', 'mask-generation', 'feature-
|
||||
extraction', 'audio-classification', 'stable-
|
||||
diffusion-xl', 'automatic-speech-recognition', 'audio-
|
||||
xvector', 'text-classification', 'conversational',
|
||||
'text-generation', 'image-to-image', 'multiple-
|
||||
choice', 'object-detection', 'image-to-text', 'masked-
|
||||
im']. For decoder models, use `xxx-with-past` to
|
||||
export the model using past key values in the decoder.
|
||||
--cache_dir CACHE_DIR
|
||||
Path indicating where to store cache.
|
||||
--framework {pt,tf} The framework to use for the export. If not provided,
|
||||
|
|
@ -560,12 +559,12 @@ compression:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:16:33.077270: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:26:33.213783: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
`--fp16` option is deprecated and will be removed in a future version. Use `--weight-format` instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Framework not specified. Using pt to export the model.
|
||||
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
|
||||
|
|
@ -574,7 +573,7 @@ compression:
|
|||
Using framework PyTorch: 2.3.0+cpu
|
||||
Overriding 1 configuration item(s)
|
||||
- use_cache -> False
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
OpenVINO Tokenizers is not available. To deploy models in production with C++ code, please follow installation instructions: https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#installation
|
||||
|
||||
|
|
|
|||
|
|
@ -101,7 +101,7 @@ Model preparation stage has the following steps:
|
|||
remote: Counting objects: 100% (281/281), done.[K
|
||||
remote: Compressing objects: 100% (96/96), done.[K
|
||||
remote: Total 282 (delta 135), reused 269 (delta 128), pack-reused 1[K
|
||||
Receiving objects: 100% (282/282), 9.22 MiB | 24.51 MiB/s, done.
|
||||
Receiving objects: 100% (282/282), 9.22 MiB | 12.60 MiB/s, done.
|
||||
Resolving deltas: 100% (135/135), done.
|
||||
|
||||
|
||||
|
|
@ -173,7 +173,7 @@ Preprocessing for model obtained from training
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
100%|██████████| 170498071/170498071 [00:07<00:00, 24352667.80it/s]
|
||||
100%|██████████| 170498071/170498071 [00:07<00:00, 22872546.75it/s]
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -245,10 +245,10 @@ about supported parameters can be found on this
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:17:12.728714: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:17:12.761090: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:27:14.913330: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:27:14.947018: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:17:13.290828: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:27:15.477320: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
|
||||
|
|
@ -431,7 +431,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 9.73 ms
|
||||
[ INFO ] Read model took 9.70 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x (node: x) : f32 / [...] / [1,3,32,32]
|
||||
|
|
@ -445,7 +445,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 203.61 ms
|
||||
[ INFO ] Compile model took 203.56 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model2
|
||||
|
|
@ -482,17 +482,17 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ INFO ] Fill input 'x' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 3.03 ms
|
||||
[ INFO ] First inference took 2.69 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 88560 iterations
|
||||
[ INFO ] Duration: 15003.27 ms
|
||||
[ INFO ] Count: 88608 iterations
|
||||
[ INFO ] Duration: 15001.90 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 1.85 ms
|
||||
[ INFO ] Average: 1.85 ms
|
||||
[ INFO ] Min: 1.22 ms
|
||||
[ INFO ] Max: 17.13 ms
|
||||
[ INFO ] Throughput: 5902.71 FPS
|
||||
[ INFO ] Min: 1.19 ms
|
||||
[ INFO ] Max: 8.66 ms
|
||||
[ INFO ] Throughput: 5906.45 FPS
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -518,7 +518,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 14.98 ms
|
||||
[ INFO ] Read model took 14.43 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] x (node: x) : f32 / [...] / [1,3,32,32]
|
||||
|
|
@ -532,7 +532,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 356.81 ms
|
||||
[ INFO ] Compile model took 331.43 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model2
|
||||
|
|
@ -569,17 +569,17 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
|
|||
[ INFO ] Fill input 'x' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 2.04 ms
|
||||
[ INFO ] First inference took 2.03 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 166524 iterations
|
||||
[ INFO ] Duration: 15000.87 ms
|
||||
[ INFO ] Count: 167064 iterations
|
||||
[ INFO ] Duration: 15001.83 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 1.01 ms
|
||||
[ INFO ] Average: 1.04 ms
|
||||
[ INFO ] Min: 0.73 ms
|
||||
[ INFO ] Max: 7.02 ms
|
||||
[ INFO ] Throughput: 11100.96 FPS
|
||||
[ INFO ] Min: 0.76 ms
|
||||
[ INFO ] Max: 13.10 ms
|
||||
[ INFO ] Throughput: 11136.24 FPS
|
||||
|
||||
|
||||
Compare results on four pictures
|
||||
|
|
|
|||
|
|
@ -50,7 +50,6 @@ additional part demonstrates how to run optimization with
|
|||
`NNCF <https://github.com/openvinotoolkit/nncf/>`__ to speed up
|
||||
pipeline.
|
||||
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
|
|
@ -62,12 +61,13 @@ Table of contents:
|
|||
- `Perform Face Identity extraction <#perform-face-identity-extraction>`__
|
||||
|
||||
- `Prepare InstantID pipeline <#prepare-instantid-pipeline>`__
|
||||
- `Convert InstantID pipeline components to OpenVINO Intermediate Representation format <#convert-instantid-pipeline-components-to-openvino-intermediate-representation-format>`__
|
||||
- `Convert InstantID pipeline components to OpenVINO Intermediate Representation
|
||||
format <#convert-instantid-pipeline-components-to-openvino-intermediate-representation-format>`__
|
||||
|
||||
- `ControlNet <#controlnet>`__
|
||||
- `unet <#unet>`__
|
||||
- `Unet <#unet>`__
|
||||
- `VAE Decoder <#vae-decoder>`__
|
||||
- `text encoders <#text-encoders>`__
|
||||
- `Text Encoders <#text-encoders>`__
|
||||
- `Image Projection Model <#image-projection-model>`__
|
||||
|
||||
- `Prepare OpenVINO InstantID Pipeline <#prepare-openvino-instantid-pipeline>`__
|
||||
|
|
@ -75,7 +75,7 @@ Table of contents:
|
|||
|
||||
- `Select inference device for InstantID <#select-inference-device-for-instantid>`__
|
||||
- `Create pipeline <#create-pipeline>`__
|
||||
- `run inference <#run-inference>`__
|
||||
- `Run inference <#run-inference>`__
|
||||
|
||||
- `Quantization <#quantization>`__
|
||||
|
||||
|
|
@ -87,7 +87,8 @@ Table of contents:
|
|||
- `Run Weights Compression <#run-weights-compression>`__
|
||||
|
||||
- `Compare model file sizes <#compare-model-file-sizes>`__
|
||||
- `Compare inference time of the FP16 and INT8 pipelines <#compare-inference-time-of-the-fp16-and-int8-pipelines>`__
|
||||
- `Compare inference time of the FP16 and INT8 pipelines
|
||||
<#compare-inference-time-of-the-fp16-and-int8-pipelines>`__
|
||||
|
||||
- `Interactive demo <#interactive-demo>`__
|
||||
|
||||
|
|
|
|||
|
|
@ -222,7 +222,7 @@ Download Model Checkpoint
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/knowledge-graphs-conve/models/conve.pt')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/knowledge-graphs-conve/models/conve.pt')
|
||||
|
||||
|
||||
|
||||
|
|
@ -384,7 +384,7 @@ typical to use metrics such as Mean Reciprocal Rank, Hits@10 etc.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Average time taken for inference: 0.7582604885101318 ms
|
||||
Average time taken for inference: 0.6809830665588379 ms
|
||||
Mean accuracy of the model on the test dataset: 0.875
|
||||
|
||||
|
||||
|
|
@ -531,7 +531,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Average time taken for inference: 0.675062338511149 ms
|
||||
Average time taken for inference: 0.6866355737050375 ms
|
||||
Mean accuracy of the model on the test dataset: 0.10416666666666667
|
||||
|
||||
|
||||
|
|
@ -550,7 +550,7 @@ Determine the platform specific speedup obtained through OpenVINO graph optimiza
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Speedup with OpenVINO optimizations: 1.12 X
|
||||
Speedup with OpenVINO optimizations: 0.99 X
|
||||
|
||||
|
||||
Benchmark the converted OpenVINO model using benchmark app
|
||||
|
|
@ -595,7 +595,7 @@ inference can also be obtained by looking at the benchmark app results.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 4.87 ms
|
||||
[ INFO ] Read model took 5.32 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] e1 (node: e1) : i64 / [...] / []
|
||||
|
|
@ -611,7 +611,7 @@ inference can also be obtained by looking at the benchmark app results.
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] ***NO_NAME*** (node: aten::softmax/Softmax) : f32 / [...] / [1,271]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 69.30 ms
|
||||
[ INFO ] Compile model took 66.93 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -641,17 +641,17 @@ inference can also be obtained by looking at the benchmark app results.
|
|||
[ INFO ] Fill input 'rel' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 10000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 1.25 ms
|
||||
[ INFO ] First inference took 1.44 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 101688 iterations
|
||||
[ INFO ] Duration: 10000.88 ms
|
||||
[ INFO ] Count: 100644 iterations
|
||||
[ INFO ] Duration: 10001.37 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 1.01 ms
|
||||
[ INFO ] Average: 1.02 ms
|
||||
[ INFO ] Min: 0.70 ms
|
||||
[ INFO ] Max: 8.75 ms
|
||||
[ INFO ] Throughput: 10167.91 FPS
|
||||
[ INFO ] Median: 1.02 ms
|
||||
[ INFO ] Average: 1.03 ms
|
||||
[ INFO ] Min: 0.65 ms
|
||||
[ INFO ] Max: 8.70 ms
|
||||
[ INFO ] Throughput: 10063.02 FPS
|
||||
|
||||
|
||||
Conclusions
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ Install requirements
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
|
@ -133,13 +133,13 @@ example <https://huggingface.co/microsoft/kosmos-2-patch14-224>`__
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:19:14.165808: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:19:14.200484: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:29:11.534432: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:29:11.569240: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:19:14.695568: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
2024-05-16 00:29:12.061235: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
|
||||
|
||||
|
|
@ -360,11 +360,11 @@ Vision model accept ``pixel_values`` and returns ``image_embeds``.
|
|||
.. parsed-literal::
|
||||
|
||||
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:469: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:469: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:509: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:509: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||
|
||||
|
||||
|
|
@ -392,7 +392,7 @@ Convert Image To Text Projection model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
|
||||
if a.grad is not None:
|
||||
|
||||
|
||||
|
|
@ -527,13 +527,13 @@ generated text by ``AutoProcessor``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:808: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:808: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if max_pos > self.weights.size(0):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1117: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1117: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input_shape[-1] > 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:924: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:924: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attention_mask.size() != (batch_size, 1, seq_length, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1210: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1210: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if past_key_values_length > 0:
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9ab062bc4b0b22c2815ec0f5c3a2e7237c9ae71ac83cc94e3363de499d8a03c6
|
||||
size 118458
|
||||
oid sha256:55f4e723ec2322d6837bbe04816e3b48ef5ab3446204ac5194544233f059f30f
|
||||
size 119957
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a3943e3c5bcdf600ff01c05b9f60dc88b9bc7bd38474d51c247c4e80544d5964
|
||||
size 1150936
|
||||
oid sha256:e15ceb0ae875340a0d9ce3082a02fcafcbe810e86863cbb69f14fccce49e97e5
|
||||
size 1151174
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0541792d494f341f9b36f073472c0f6bc248297506ada281f7dfbb3feaf0a5af
|
||||
size 120137
|
||||
oid sha256:6d515c7b0d0e34137797eb090964a6ba4b2fb724bbd3003c8c31c7bd0b7add6f
|
||||
size 122393
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1273f178881d3158052a8c494b40a5bb426a972130242bc0f84145c5bb2d98d6
|
||||
size 1150777
|
||||
oid sha256:e3ae8247baa4ef9148ed9c3a6a9bfc9b732f976b19042f530c34cb7f8cf5bd36
|
||||
size 1150593
|
||||
|
|
|
|||
|
|
@ -90,10 +90,10 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:20:31.324929: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:20:31.359787: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:30:29.627218: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:30:29.661916: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:20:31.955321: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:30:30.261372: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -200,7 +200,7 @@ PyTorch model formats are supported:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -532,9 +532,9 @@ Frames Per Second (FPS) for images.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.072 seconds per sentence, SPS: 13.84
|
||||
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 47.97
|
||||
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 110.23
|
||||
PyTorch model on CPU: 0.071 seconds per sentence, SPS: 14.02
|
||||
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 48.10
|
||||
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 108.86
|
||||
|
||||
|
||||
Finally, measure the inference performance of OpenVINO ``FP32`` and
|
||||
|
|
@ -575,7 +575,7 @@ in OpenVINO.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 19.23 ms
|
||||
[ INFO ] Read model took 18.56 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?]
|
||||
|
|
@ -586,7 +586,7 @@ in OpenVINO.
|
|||
[Step 5/11] Resizing model to match image sizes and given batch
|
||||
[ INFO ] Model batch size: 1
|
||||
[ INFO ] Reshaping model: 'input_ids': [1,128], '36': [1,128], 'token_type_ids': [1,128]
|
||||
[ INFO ] Reshape model took 5.65 ms
|
||||
[ INFO ] Reshape model took 5.68 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128]
|
||||
|
|
@ -595,7 +595,7 @@ in OpenVINO.
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 376.44 ms
|
||||
[ INFO ] Compile model took 351.77 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -636,17 +636,17 @@ in OpenVINO.
|
|||
[ INFO ] Fill input 'token_type_ids' with random values
|
||||
[Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 22.61 ms
|
||||
[ INFO ] First inference took 21.49 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 6217 iterations
|
||||
[ INFO ] Duration: 120004.55 ms
|
||||
[ INFO ] Count: 6147 iterations
|
||||
[ INFO ] Duration: 120009.26 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 19.20 ms
|
||||
[ INFO ] Average: 19.21 ms
|
||||
[ INFO ] Min: 18.57 ms
|
||||
[ INFO ] Max: 23.34 ms
|
||||
[ INFO ] Throughput: 51.81 FPS
|
||||
[ INFO ] Median: 19.24 ms
|
||||
[ INFO ] Average: 19.43 ms
|
||||
[ INFO ] Min: 18.60 ms
|
||||
[ INFO ] Max: 22.12 ms
|
||||
[ INFO ] Throughput: 51.22 FPS
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -673,27 +673,27 @@ in OpenVINO.
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 24.76 ms
|
||||
[ INFO ] Read model took 24.21 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?]
|
||||
[ INFO ] 36 , attention_mask (node: attention_mask) : i64 / [...] / [1,?]
|
||||
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,?]
|
||||
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,?]
|
||||
[ INFO ] Model outputs:
|
||||
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
|
||||
[Step 5/11] Resizing model to match image sizes and given batch
|
||||
[ INFO ] Model batch size: 1
|
||||
[ INFO ] Reshaping model: 'input_ids': [1,128], '36': [1,128], 'token_type_ids': [1,128]
|
||||
[ INFO ] Reshape model took 7.38 ms
|
||||
[ INFO ] Reshape model took 7.31 ms
|
||||
[Step 6/11] Configuring input of the model
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128]
|
||||
[ INFO ] 36 , attention_mask (node: attention_mask) : i64 / [...] / [1,128]
|
||||
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,128]
|
||||
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,128]
|
||||
[ INFO ] Model outputs:
|
||||
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 1183.71 ms
|
||||
[ INFO ] Compile model took 1079.78 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: Model0
|
||||
|
|
@ -734,15 +734,15 @@ in OpenVINO.
|
|||
[ INFO ] Fill input 'token_type_ids' with random values
|
||||
[Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 15.91 ms
|
||||
[ INFO ] First inference took 15.94 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 11978 iterations
|
||||
[ INFO ] Duration: 120006.11 ms
|
||||
[ INFO ] Count: 12483 iterations
|
||||
[ INFO ] Duration: 120004.49 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 10.29 ms
|
||||
[ INFO ] Average: 9.93 ms
|
||||
[ INFO ] Min: 8.15 ms
|
||||
[ INFO ] Max: 11.91 ms
|
||||
[ INFO ] Throughput: 99.81 FPS
|
||||
[ INFO ] Median: 9.59 ms
|
||||
[ INFO ] Average: 9.52 ms
|
||||
[ INFO ] Min: 8.18 ms
|
||||
[ INFO ] Max: 11.10 ms
|
||||
[ INFO ] Throughput: 104.02 FPS
|
||||
|
||||
|
|
|
|||
|
|
@ -121,10 +121,10 @@ https://huggingface.co/docs/diffusers/en/api/pipelines/latent_consistency_models
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:26:31.942146: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:26:31.978132: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:36:40.510949: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:36:40.546299: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:26:32.469596: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:36:41.041474: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f55f400d3c6132e0208b817685c99756778f625b48a68f0a061909eb21b17b73
|
||||
size 30125
|
||||
oid sha256:0bd7464e563caf1e06e0c1cbd3a3469c284a6e9d7372f2c8d1828f9df001abe0
|
||||
size 29753
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b73d5a12608ef2be6bed0c105a19e7375b187853f502d6659c3fd84e0d968759
|
||||
size 416282
|
||||
oid sha256:ca79e22459b6d83468505496f54f4f227721878ffec5293e1564c3332bf8e439
|
||||
size 429952
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:66b5aca06c991e57d8cf17b73f6260878086185c9a7f5901d054fe53637adf3a
|
||||
size 36310
|
||||
oid sha256:27e0663303ea64ceb8505c4d59c5c86a6afb20e2ff2a10166b0c47ccff2bc52b
|
||||
size 33705
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bfb27d57b247ea9ea90b3a6d194f3ad1d874c657ad6ec204b0fffc0623c09442
|
||||
size 454841
|
||||
oid sha256:5ee2f1136215766a50f4539f33ba7ac6040d545fcf22c7f80dca523bffdc92f9
|
||||
size 442908
|
||||
|
|
|
|||
|
|
@ -119,7 +119,7 @@ documentation.
|
|||
conversion into IR. The legacy Frontend is Python
|
||||
based and is available for TensorFlow*, ONNX*, MXNet*,
|
||||
Caffe*, and Kaldi* models.
|
||||
--input_model INPUT_MODEL, -m INPUT_MODEL, -w INPUT_MODEL
|
||||
--input_model INPUT_MODEL, -w INPUT_MODEL, -m INPUT_MODEL
|
||||
Tensorflow*: a file with a pre-trained model (binary
|
||||
or text .pb file after freezing). Caffe*: a model
|
||||
proto file with model weights.
|
||||
|
|
@ -735,13 +735,13 @@ NLP model from Hugging Face and export it in ONNX format:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-06 23:47:34.384632: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-06 23:47:34.420652: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-15 23:49:58.968544: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-15 23:49:59.004440: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-06 23:47:35.064940: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
2024-05-15 23:49:59.646592: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
|
||||
mask, torch.tensor(torch.finfo(scores.dtype).min)
|
||||
|
||||
|
||||
|
|
@ -1004,8 +1004,8 @@ To convert a model to OpenVINO IR, use the following command:
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1100,8 +1100,8 @@ guide <https://docs.openvino.ai/2024/openvino-workflow/model-preparation/setting
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1119,8 +1119,8 @@ guide <https://docs.openvino.ai/2024/openvino-workflow/model-preparation/setting
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1168,8 +1168,8 @@ sequence length dimension for inputs:
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1213,8 +1213,8 @@ dimension:
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1281,8 +1281,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1300,8 +1300,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1376,8 +1376,8 @@ Resnet50 model that was exported to the ONNX format:
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1422,8 +1422,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1441,8 +1441,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1491,8 +1491,8 @@ that the preprocessing takes negligible time for inference.
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -1510,8 +1510,8 @@ that the preprocessing takes negligible time for inference.
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1557,8 +1557,8 @@ the color channels before inference.
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -1602,8 +1602,8 @@ models, this decrease is negligible.
|
|||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
|
|||
|
|
@ -1058,8 +1058,12 @@ Select device from dropdown list for running inference using OpenVINO.
|
|||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="AUTO",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
|
|||
|
|
@ -136,8 +136,8 @@ Download PyTorch model
|
|||
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
|
||||
|
||||
|
||||
OpenVINO## Convert model to OpenVINO Intermediate Representation
|
||||
|
||||
OpenVINO## Convert model to OpenVINO Intermediate Representation `back
|
||||
to top ⬆️ <#Table-of-contents:>`__
|
||||
|
||||
OpenVINO supports PyTorch models via conversion to OpenVINO Intermediate
|
||||
Representation (IR). `OpenVINO model conversion
|
||||
|
|
@ -1183,8 +1183,12 @@ Select device
|
|||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices,
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
|
|||
|
|
@ -233,8 +233,13 @@ Select inference device for LLM
|
|||
import ipywidgets as widgets
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
|
|||
|
|
@ -436,7 +436,7 @@ Convert model using Optimum-CLI tool
|
|||
|
||||
|
||||
`Optimum Intel <https://huggingface.co/docs/optimum/intel/index>`__ is
|
||||
the interface between the
|
||||
the interface between the
|
||||
`Transformers <https://huggingface.co/docs/transformers/index>`__ and
|
||||
`Diffusers <https://huggingface.co/docs/diffusers/index>`__ libraries
|
||||
and OpenVINO to accelerate end-to-end pipelines on Intel architectures.
|
||||
|
|
@ -487,10 +487,11 @@ to make it
|
|||
`symmetric <https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Quantization.md#symmetric-quantization>`__
|
||||
you can add ``--sym``.
|
||||
|
||||
For INT4 quantization you can also specify the following arguments : -
|
||||
The ``--group-size`` parameter will define the group size to use for
|
||||
quantization, -1 it will results in per-column quantization. - The
|
||||
``--ratio`` parameter controls the ratio between 4-bit and 8-bit
|
||||
For INT4 quantization you can also specify the following arguments :
|
||||
|
||||
- The ``--group-size`` parameter will define the group size to use for
|
||||
quantization, -1 it will results in per-column quantization.
|
||||
- The ``--ratio`` parameter controls the ratio between 4-bit and 8-bit
|
||||
quantization. If set to 0.9, it means that 90% of the layers will be
|
||||
quantized to int4 while 10% will be quantized to int8.
|
||||
|
||||
|
|
@ -708,8 +709,12 @@ Select device for inference and model variant
|
|||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
@ -722,7 +727,7 @@ Select device for inference and model variant
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Dropdown(description='Device:', options=('CPU', 'GPU.0', 'GPU.1', 'AUTO'), value='CPU')
|
||||
Dropdown(description='Device:', options=('CPU', 'GPU', 'AUTO'), value='CPU')
|
||||
|
||||
|
||||
|
||||
|
|
@ -1339,6 +1344,7 @@ answers.https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide.html
|
|||
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
# please uncomment and run this cell for stopping gradio interface
|
||||
|
|
|
|||
|
|
@ -529,8 +529,13 @@ Select device for inference and model variant
|
|||
.. code:: ipython3
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
|
|||
|
|
@ -620,8 +620,13 @@ Select device for embedding model inference
|
|||
.. code:: ipython3
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
support_devices = core.available_devices
|
||||
if "NPU" in support_devices:
|
||||
support_devices.remove("NPU")
|
||||
|
||||
embedding_device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
@ -656,7 +661,7 @@ Select device for rerank model inference
|
|||
.. code:: ipython3
|
||||
|
||||
rerank_device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
@ -691,7 +696,7 @@ Select device for LLM model inference
|
|||
.. code:: ipython3
|
||||
|
||||
llm_device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
options=support_devices + ["AUTO"],
|
||||
value="CPU",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
|
|
@ -966,6 +971,7 @@ The most common full sequence from raw data to answer looks like:
|
|||
|
||||
.. code:: ipython3
|
||||
|
||||
import re
|
||||
from typing import List
|
||||
from langchain.text_splitter import (
|
||||
CharacterTextSplitter,
|
||||
|
|
|
|||
|
|
@ -42,7 +42,8 @@ post <https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-effic
|
|||
In this tutorial we consider how to bring OpenVINO power into Magika.
|
||||
|
||||
Table of contents:
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
------------------
|
||||
|
||||
|
||||
- `Prerequisites <#prerequisites>`__
|
||||
- `Define model loading class <#define-model-loading-class>`__
|
||||
|
|
@ -58,7 +59,7 @@ Table of contents:
|
|||
Prerequisites
|
||||
-------------
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
|
|
|
|||
|
|
@ -641,7 +641,7 @@ bounds of input batch size.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7fe343b14c10>
|
||||
<matplotlib.image.AxesImage at 0x7f9941fc2040>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ Prerequisites
|
|||
remote: Counting objects: 100% (45/45), done.[K
|
||||
remote: Compressing objects: 100% (36/36), done.[K
|
||||
remote: Total 45 (delta 9), reused 44 (delta 8), pack-reused 0[K
|
||||
Unpacking objects: 100% (45/45), 428.50 KiB | 3.15 MiB/s, done.
|
||||
Unpacking objects: 100% (45/45), 428.50 KiB | 3.25 MiB/s, done.
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -438,8 +438,8 @@ Perform search
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Image encoding took 0.0943 ms
|
||||
Text encoding took 0.0107 ms
|
||||
Image encoding took 0.0983 ms
|
||||
Text encoding took 0.0152 ms
|
||||
|
||||
|
||||
|
||||
|
|
@ -517,7 +517,7 @@ be used separately. Let’s convert each part to OpenVINO.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/mobileclip/modules/common/transformer.py:125: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/mobileclip/modules/common/transformer.py:125: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if seq_len != self.num_embeddings:
|
||||
|
||||
|
||||
|
|
@ -609,8 +609,8 @@ Perform search
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Image encoding took 0.0293 ms
|
||||
Text encoding took 0.0058 ms
|
||||
Image encoding took 0.0315 ms
|
||||
Text encoding took 0.0059 ms
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ Install requirements
|
|||
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
|
||||
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible.
|
||||
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible.
|
||||
optimum-intel 1.17.0.dev0+9bb4334 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
|
||||
optimum-intel 1.17.0.dev0+8c2b787 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
|
|
@ -110,13 +110,13 @@ Import required packages
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
2024-05-07 00:30:21.457260: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:30:21.492393: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:40:31.564225: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:40:31.599009: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:30:22.002379: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
2024-05-16 00:40:32.110586: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
|
||||
|
||||
|
|
@ -333,15 +333,15 @@ compression instead of INT8 weight compression.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:595: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:595: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input_shape[-1] > 1:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if seq_len > self.max_seq_len_cached:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:348: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:348: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz, self.num_heads, q_len, kv_seq_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:355: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:355: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attention_mask.size() != (bsz, 1, q_len, kv_seq_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:365: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:365: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz, self.num_heads, q_len, self.head_dim):
|
||||
|
||||
|
||||
|
|
@ -423,7 +423,7 @@ compression instead of INT8 weight compression.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
|
||||
if a.grad is not None:
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -225,20 +225,20 @@ Converting mobilenet-v2-pytorch…
|
|||
.. parsed-literal::
|
||||
|
||||
========== Converting mobilenet-v2-pytorch to ONNX
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-name=mobilenet_v2 --weights=model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth --import-module=torchvision.models --input-shape=1,3,224,224 --output-file=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx --input-names=data --output-names=prob
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-name=mobilenet_v2 --weights=model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth --import-module=torchvision.models --input-shape=1,3,224,224 --output-file=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx --input-names=data --output-names=prob
|
||||
|
||||
ONNX check passed successfully.
|
||||
|
||||
========== Converting mobilenet-v2-pytorch to IR (FP16)
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/mobilenet-v2-pytorch/FP16 --model_name=mobilenet-v2-pytorch --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.624,57.12,57.375]' --reverse_input_channels --output=prob --input_model=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 224, 224]' --compress_to_fp16=True
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/mobilenet-v2-pytorch/FP16 --model_name=mobilenet-v2-pytorch --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.624,57.12,57.375]' --reverse_input_channels --output=prob --input_model=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 224, 224]' --compress_to_fp16=True
|
||||
|
||||
[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.
|
||||
Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html
|
||||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.bin
|
||||
|
||||
|
||||
|
||||
|
|
@ -290,8 +290,8 @@ information in a dictionary.
|
|||
'description': 'MobileNet V2 is image classification model pre-trained on ImageNet dataset. This is a PyTorch* implementation of MobileNetV2 architecture as described in the paper "Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation" <https://arxiv.org/abs/1801.04381>.\nThe model input is a blob that consists of a single image of "1, 3, 224, 224" in "RGB" order.\nThe model output is typical object classifier for the 1000 different classifications matching with those in the ImageNet database.',
|
||||
'framework': 'pytorch',
|
||||
'license_url': 'https://raw.githubusercontent.com/pytorch/vision/master/LICENSE',
|
||||
'accuracy_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/accuracy-check.yml',
|
||||
'model_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/model.yml',
|
||||
'accuracy_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/accuracy-check.yml',
|
||||
'model_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/model.yml',
|
||||
'precisions': ['FP16', 'FP32'],
|
||||
'subdirectory': 'public/mobilenet-v2-pytorch',
|
||||
'task_type': 'classification',
|
||||
|
|
@ -368,7 +368,7 @@ seconds…
|
|||
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
|
||||
[Step 4/11] Reading model files
|
||||
[ INFO ] Loading model files
|
||||
[ INFO ] Read model took 27.49 ms
|
||||
[ INFO ] Read model took 28.84 ms
|
||||
[ INFO ] Original model I/O parameters:
|
||||
[ INFO ] Model inputs:
|
||||
[ INFO ] data (node: data) : f32 / [N,C,H,W] / [1,3,224,224]
|
||||
|
|
@ -382,7 +382,7 @@ seconds…
|
|||
[ INFO ] Model outputs:
|
||||
[ INFO ] prob (node: prob) : f32 / [...] / [1,1000]
|
||||
[Step 7/11] Loading the model to the device
|
||||
[ INFO ] Compile model took 145.46 ms
|
||||
[ INFO ] Compile model took 147.59 ms
|
||||
[Step 8/11] Querying optimal runtime parameters
|
||||
[ INFO ] Model:
|
||||
[ INFO ] NETWORK_NAME: main_graph
|
||||
|
|
@ -410,17 +410,17 @@ seconds…
|
|||
[ INFO ] Fill input 'data' with random values
|
||||
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
|
||||
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
|
||||
[ INFO ] First inference took 5.83 ms
|
||||
[ INFO ] First inference took 6.31 ms
|
||||
[Step 11/11] Dumping statistics report
|
||||
[ INFO ] Execution Devices:['CPU']
|
||||
[ INFO ] Count: 20130 iterations
|
||||
[ INFO ] Duration: 15004.99 ms
|
||||
[ INFO ] Count: 20214 iterations
|
||||
[ INFO ] Duration: 15004.27 ms
|
||||
[ INFO ] Latency:
|
||||
[ INFO ] Median: 4.34 ms
|
||||
[ INFO ] Average: 4.35 ms
|
||||
[ INFO ] Min: 3.10 ms
|
||||
[ INFO ] Max: 12.27 ms
|
||||
[ INFO ] Throughput: 1341.55 FPS
|
||||
[ INFO ] Median: 4.32 ms
|
||||
[ INFO ] Average: 4.33 ms
|
||||
[ INFO ] Min: 2.48 ms
|
||||
[ INFO ] Max: 12.25 ms
|
||||
[ INFO ] Throughput: 1347.22 FPS
|
||||
|
||||
|
||||
Benchmark with Different Settings
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -72,7 +72,7 @@ Install requirements
|
|||
magika 0.5.1 requires numpy<2.0,>=1.24; python_version >= "3.8" and python_version < "3.9", but you have numpy 1.23.5 which is incompatible.
|
||||
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible.
|
||||
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible.
|
||||
optimum-intel 1.17.0.dev0+9bb4334 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
|
||||
optimum-intel 1.17.0.dev0+8c2b787 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:31feb975df6ad5a4fefa385cf8bfeb4fa4d58a306ece031f9cc4ed754a1806a9
|
||||
size 174999
|
||||
oid sha256:0aa26914ef65f9ab1061c4195a73b0b8c1d098a02bb4cb01c12997aed53843ad
|
||||
size 175079
|
||||
|
|
|
|||
|
|
@ -194,7 +194,7 @@ notebooks.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -243,7 +243,7 @@ points to the filename of an ONNX model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.onnx')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.onnx')
|
||||
|
||||
|
||||
|
||||
|
|
@ -303,7 +303,7 @@ without any conversion step. Pass the filename with extension to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/inference.pdiparams')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/inference.pdiparams')
|
||||
|
||||
|
||||
|
||||
|
|
@ -347,7 +347,7 @@ TensorFlow models saved in frozen graph format can also be passed to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.pb')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.pb')
|
||||
|
||||
|
||||
|
||||
|
|
@ -403,7 +403,7 @@ It is pre-trained model optimized to work with TensorFlow Lite.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.tflite')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.tflite')
|
||||
|
||||
|
||||
|
||||
|
|
@ -484,7 +484,7 @@ Information about the inputs and outputs of the model are in
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -690,7 +690,7 @@ produced data as values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -879,7 +879,7 @@ input shape.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -1031,7 +1031,7 @@ the cache.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
|
||||
|
||||
|
||||
|
||||
|
|
@ -1061,7 +1061,7 @@ the cache.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Loading the network to the CPU device took 0.17 seconds.
|
||||
Loading the network to the CPU device took 0.18 seconds.
|
||||
|
||||
|
||||
After running the previous cell, we know the model exists in the cache
|
||||
|
|
|
|||
|
|
@ -106,7 +106,9 @@ use ``pip install openvino-tokenizers[transformers]``.
|
|||
Found existing installation: openvino 2024.1.0
|
||||
Uninstalling openvino-2024.1.0:
|
||||
Successfully uninstalled openvino-2024.1.0
|
||||
WARNING: Skipping openvino-nightly as it is not installed.
|
||||
Found existing installation: openvino-nightly 2024.2.0.dev20240513
|
||||
Uninstalling openvino-nightly-2024.2.0.dev20240513:
|
||||
Successfully uninstalled openvino-nightly-2024.2.0.dev20240513
|
||||
Found existing installation: openvino-dev 2024.1.0
|
||||
Uninstalling openvino-dev-2024.1.0:
|
||||
Successfully uninstalled openvino-dev-2024.1.0
|
||||
|
|
@ -116,15 +118,15 @@ use ``pip install openvino-tokenizers[transformers]``.
|
|||
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
|
||||
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.14.0)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
|
||||
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu)
|
||||
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.14.0)
|
||||
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0)
|
||||
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12)
|
||||
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
|
||||
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
|
||||
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
|
||||
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
|
||||
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
||||
|
|
@ -157,7 +159,7 @@ constructor.
|
|||
.. parsed-literal::
|
||||
|
||||
Loading Huggingface Tokenizer...
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Converting Huggingface Tokenizer to OpenVINO...
|
||||
Saved OpenVINO Tokenizer: tokenizer/openvino_tokenizer.xml, tokenizer/openvino_tokenizer.bin
|
||||
|
|
@ -193,7 +195,7 @@ The other method is to pass HuggingFace ``hf_tokenizer`` object to
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -282,10 +284,10 @@ one:
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:39:47.665228: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:39:47.699909: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:50:00.049334: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:50:00.085249: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:39:48.279101: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:50:00.664160: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -497,17 +499,17 @@ models and tokenizers simplifies memory management.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:40:23.074270: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:50:37.102688: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Framework not specified. Using pt to export the model.
|
||||
Using framework PyTorch: 2.3.0+cpu
|
||||
Overriding 1 configuration item(s)
|
||||
- use_cache -> False
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
Detokenizer is not supported, convert tokenizer only.
|
||||
|
||||
|
|
@ -523,7 +525,7 @@ models and tokenizers simplifies memory management.
|
|||
.. parsed-literal::
|
||||
|
||||
Loading Huggingface Tokenizer...
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
Converting Huggingface Tokenizer to OpenVINO...
|
||||
Saved OpenVINO Tokenizer: bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.xml, bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.bin
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -64,7 +64,7 @@ Table of contents:
|
|||
.. parsed-literal::
|
||||
|
||||
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
|
||||
openvino-tokenizers 2024.2.0.0.dev20240506 requires openvino~=2024.2.0.0.dev, but you have openvino 2024.1.0 which is incompatible.
|
||||
openvino-tokenizers 2024.2.0.0.dev20240513 requires openvino~=2024.2.0.0.dev, but you have openvino 2024.1.0 which is incompatible.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
Note: you may need to restart the kernel to use updated packages.
|
||||
|
||||
|
|
@ -368,20 +368,20 @@ Converting text-recognition-resnet-fc…
|
|||
.. parsed-literal::
|
||||
|
||||
========== Converting text-recognition-resnet-fc to ONNX
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/text-recognition-resnet-fc --model-path=model/public/text-recognition-resnet-fc --model-name=get_model --import-module=model '--model-param=file_config=r"model/public/text-recognition-resnet-fc/vedastr/configs/resnet_fc.py"' '--model-param=weights=r"model/public/text-recognition-resnet-fc/vedastr/ckpt/resnet_fc.pth"' --input-shape=1,1,32,100 --input-names=input --output-names=output --output-file=model/public/text-recognition-resnet-fc/resnet_fc.onnx
|
||||
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/text-recognition-resnet-fc --model-path=model/public/text-recognition-resnet-fc --model-name=get_model --import-module=model '--model-param=file_config=r"model/public/text-recognition-resnet-fc/vedastr/configs/resnet_fc.py"' '--model-param=weights=r"model/public/text-recognition-resnet-fc/vedastr/ckpt/resnet_fc.pth"' --input-shape=1,1,32,100 --input-names=input --output-names=output --output-file=model/public/text-recognition-resnet-fc/resnet_fc.onnx
|
||||
|
||||
ONNX check passed successfully.
|
||||
|
||||
========== Converting text-recognition-resnet-fc to IR (FP16)
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/text-recognition-resnet-fc/FP16 --model_name=text-recognition-resnet-fc --input=input '--mean_values=input[127.5]' '--scale_values=input[127.5]' --output=output --input_model=model/public/text-recognition-resnet-fc/resnet_fc.onnx '--layout=input(NCHW)' '--input_shape=[1, 1, 32, 100]' --compress_to_fp16=True
|
||||
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/text-recognition-resnet-fc/FP16 --model_name=text-recognition-resnet-fc --input=input '--mean_values=input[127.5]' '--scale_values=input[127.5]' --output=output --input_model=model/public/text-recognition-resnet-fc/resnet_fc.onnx '--layout=input(NCHW)' '--input_shape=[1, 1, 32, 100]' --compress_to_fp16=True
|
||||
|
||||
[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.
|
||||
Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html
|
||||
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
|
||||
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
|
||||
[ SUCCESS ] Generated IR version 11 model.
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.bin
|
||||
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.xml
|
||||
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.bin
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -207,8 +207,8 @@ and save it to the disk.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:42:55.865083: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-05-07 00:42:55.865267: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
2024-05-16 00:53:11.397090: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
|
||||
2024-05-16 00:53:11.397273: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -366,7 +366,7 @@ for mean/scale normalization.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7f96d4115270>
|
||||
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7fd96c504970>
|
||||
|
||||
|
||||
|
||||
|
|
@ -397,7 +397,7 @@ may be specified is input data
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7f96d4115f70>
|
||||
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7fd96c504830>
|
||||
|
||||
|
||||
|
||||
|
|
@ -435,7 +435,7 @@ then such conversion will be added explicitly.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7f96944cd4f0>
|
||||
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7fd96c532a70>
|
||||
|
||||
|
||||
|
||||
|
|
@ -649,6 +649,6 @@ Compare performance
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0153 seconds per image, FPS: 65.50
|
||||
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0185 seconds per image, FPS: 53.99
|
||||
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0152 seconds per image, FPS: 65.63
|
||||
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0184 seconds per image, FPS: 54.32
|
||||
|
||||
|
|
|
|||
|
|
@ -200,7 +200,7 @@ Download the Model for Text **Detection**
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/paddle-o…
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/paddle-o…
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -246,7 +246,7 @@ Download the Model for Text **Recognition**
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/paddle-o…
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/paddle-o…
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3427940d246df617e9e394b03aae4704047ce6a8c924024afd5d3aed63ed5bca
|
||||
size 592987
|
||||
oid sha256:a7f1428e52cc0f79f7ee67a644231f522aa713e6d54a73f8511693e1a28ec610
|
||||
size 590583
|
||||
|
|
|
|||
|
|
@ -78,11 +78,11 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
--2024-05-07 00:44:40-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
||||
--2024-05-16 00:54:56-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
|
||||
Resolving proxy-dmz.intel.com (proxy-dmz.intel.com)... 10.241.208.166
|
||||
Connecting to proxy-dmz.intel.com (proxy-dmz.intel.com)|10.241.208.166|:911... connected.
|
||||
Proxy request sent, awaiting response... 404 Not Found
|
||||
2024-05-07 00:44:41 ERROR 404: Not Found.
|
||||
2024-05-16 00:54:56 ERROR 404: Not Found.
|
||||
|
||||
dpkg: error: cannot access archive 'libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb': No such file or directory
|
||||
|
||||
|
|
@ -113,8 +113,8 @@ Imports
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:44:42 INFO: Loading faiss with AVX512 support.
|
||||
2024-05-07 00:44:42 INFO: Successfully loaded faiss with AVX512 support.
|
||||
2024-05-16 00:54:58 INFO: Loading faiss with AVX512 support.
|
||||
2024-05-16 00:54:58 INFO: Successfully loaded faiss with AVX512 support.
|
||||
|
||||
|
||||
Settings
|
||||
|
|
@ -198,7 +198,7 @@ inference on that image, and then show the top three prediction results.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[2024/05/07 00:45:02] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
||||
[2024/05/16 00:55:27] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
|
||||
Labrador retriever, 0.75138
|
||||
German short-haired pointer, 0.02373
|
||||
Great Dane, 0.01848
|
||||
|
|
@ -264,7 +264,7 @@ clipping values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:45:02 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
||||
2024-05-16 00:55:27 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -276,7 +276,7 @@ clipping values.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
<matplotlib.image.AxesImage at 0x7f7e383109a0>
|
||||
<matplotlib.image.AxesImage at 0x7fa3f4283910>
|
||||
|
||||
|
||||
|
||||
|
|
@ -455,7 +455,7 @@ Note that many optimizations are possible to improve the performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PaddlePaddle model on CPU: 0.0074 seconds per image, FPS: 134.45
|
||||
PaddlePaddle model on CPU: 0.0076 seconds per image, FPS: 132.29
|
||||
|
||||
PaddlePaddle result:
|
||||
Labrador retriever, 0.75138
|
||||
|
|
@ -516,7 +516,7 @@ select device from dropdown list for running inference using OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0029 seconds per image, FPS: 344.91
|
||||
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0029 seconds per image, FPS: 340.41
|
||||
|
||||
OpenVINO result:
|
||||
Labrador retriever, 0.74909
|
||||
|
|
|
|||
|
|
@ -118,10 +118,10 @@ Table of contents:
|
|||
.. code:: ipython3
|
||||
|
||||
import platform
|
||||
|
||||
|
||||
%pip install -q "openvino-dev>=2024.0.0"
|
||||
%pip install -q opencv-python requests scipy tqdm
|
||||
|
||||
|
||||
if platform.system() != "Windows":
|
||||
%pip install -q "matplotlib>=3.4"
|
||||
else:
|
||||
|
|
@ -145,7 +145,7 @@ Imports
|
|||
import collections
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
|
||||
import numpy as np
|
||||
import cv2
|
||||
from IPython import display
|
||||
|
|
@ -155,17 +155,17 @@ Imports
|
|||
.. code:: ipython3
|
||||
|
||||
# Import local modules
|
||||
|
||||
|
||||
if not Path("./notebook_utils.py").exists():
|
||||
# Fetch `notebook_utils` module
|
||||
import requests
|
||||
|
||||
|
||||
r = requests.get(
|
||||
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py",
|
||||
)
|
||||
|
||||
|
||||
open("notebook_utils.py", "w").write(r.text)
|
||||
|
||||
|
||||
import notebook_utils as utils
|
||||
from deepsort_utils.tracker import Tracker
|
||||
from deepsort_utils.nn_matching import NearestNeighborDistanceMetric
|
||||
|
|
@ -218,46 +218,46 @@ replace the name of the model in the code below.
|
|||
precision = "FP16"
|
||||
# The name of the model from Open Model Zoo
|
||||
detection_model_name = "person-detection-0202"
|
||||
|
||||
|
||||
download_command = (
|
||||
f"omz_downloader " f"--name {detection_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}"
|
||||
)
|
||||
! $download_command
|
||||
|
||||
|
||||
detection_model_path = f"model/intel/{detection_model_name}/{precision}/{detection_model_name}.xml"
|
||||
|
||||
|
||||
|
||||
|
||||
reidentification_model_name = "person-reidentification-retail-0287"
|
||||
|
||||
|
||||
download_command = (
|
||||
f"omz_downloader " f"--name {reidentification_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}"
|
||||
)
|
||||
! $download_command
|
||||
|
||||
|
||||
reidentification_model_path = f"model/intel/{reidentification_model_name}/{precision}/{reidentification_model_name}.xml"
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
################|| Downloading person-detection-0202 ||################
|
||||
|
||||
|
||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.xml
|
||||
|
||||
|
||||
|
||||
|
||||
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
|
||||
|
||||
|
||||
|
||||
|
||||
################|| Downloading person-reidentification-retail-0287 ||################
|
||||
|
||||
|
||||
========== Downloading model/intel/person-reidentification-retail-0287/person-reidentification-retail-0267.onnx
|
||||
|
||||
|
||||
|
||||
|
||||
========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml
|
||||
|
||||
|
||||
|
||||
|
||||
========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.bin
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Load model
|
||||
|
|
@ -282,18 +282,18 @@ performance, but slightly longer startup time).
|
|||
.. code:: ipython3
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
|
||||
|
||||
|
||||
class Model:
|
||||
"""
|
||||
This class represents a OpenVINO model object.
|
||||
|
||||
|
||||
"""
|
||||
|
||||
|
||||
def __init__(self, model_path, batchsize=1, device="AUTO"):
|
||||
"""
|
||||
Initialize the model object
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
model_path: path of inference model
|
||||
|
|
@ -305,18 +305,18 @@ performance, but slightly longer startup time).
|
|||
self.input_shape = self.input_layer.shape
|
||||
self.height = self.input_shape[2]
|
||||
self.width = self.input_shape[3]
|
||||
|
||||
|
||||
for layer in self.model.inputs:
|
||||
input_shape = layer.partial_shape
|
||||
input_shape[0] = batchsize
|
||||
self.model.reshape({layer: input_shape})
|
||||
self.compiled_model = core.compile_model(model=self.model, device_name=device)
|
||||
self.output_layer = self.compiled_model.output(0)
|
||||
|
||||
|
||||
def predict(self, input):
|
||||
"""
|
||||
Run inference
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
input: array of input data
|
||||
|
|
@ -334,14 +334,14 @@ select device from dropdown list for running inference using OpenVINO
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
value="AUTO",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -364,10 +364,11 @@ Data Processing
|
|||
|
||||
|
||||
|
||||
Data Processing includes data preprocess and postprocess functions. -
|
||||
Data preprocess function is used to change the layout and shape of input
|
||||
data, according to requirement of the network input format. - Data
|
||||
postprocess function is used to extract the useful information from
|
||||
Data Processing includes data preprocess and postprocess functions.
|
||||
|
||||
- Data preprocess function is used to change the layout and shape of input
|
||||
data, according to requirement of the network input format.
|
||||
- Data postprocess function is used to extract the useful information from
|
||||
network’s original output and visualize it.
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -375,7 +376,7 @@ network’s original output and visualize it.
|
|||
def preprocess(frame, height, width):
|
||||
"""
|
||||
Preprocess a single image
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
frame: input frame
|
||||
|
|
@ -386,12 +387,12 @@ network’s original output and visualize it.
|
|||
resized_image = resized_image.transpose((2, 0, 1))
|
||||
input_image = np.expand_dims(resized_image, axis=0).astype(np.float32)
|
||||
return input_image
|
||||
|
||||
|
||||
|
||||
|
||||
def batch_preprocess(img_crops, height, width):
|
||||
"""
|
||||
Preprocess batched images
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
img_crops: batched input images
|
||||
|
|
@ -400,12 +401,12 @@ network’s original output and visualize it.
|
|||
"""
|
||||
img_batch = np.concatenate([preprocess(img, height, width) for img in img_crops], axis=0)
|
||||
return img_batch
|
||||
|
||||
|
||||
|
||||
|
||||
def process_results(h, w, results, thresh=0.5):
|
||||
"""
|
||||
postprocess detection results
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
h, w: original height and width of input image
|
||||
|
|
@ -432,18 +433,18 @@ network’s original output and visualize it.
|
|||
)
|
||||
labels.append(int(label))
|
||||
scores.append(float(score))
|
||||
|
||||
|
||||
if len(boxes) == 0:
|
||||
boxes = np.array([]).reshape(0, 4)
|
||||
scores = np.array([])
|
||||
labels = np.array([])
|
||||
return np.array(boxes), np.array(scores), np.array(labels)
|
||||
|
||||
|
||||
|
||||
|
||||
def draw_boxes(img, bbox, identities=None):
|
||||
"""
|
||||
Draw bounding box in original image
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
img: original image
|
||||
|
|
@ -469,12 +470,12 @@ network’s original output and visualize it.
|
|||
2,
|
||||
)
|
||||
return img
|
||||
|
||||
|
||||
|
||||
|
||||
def cosin_metric(x1, x2):
|
||||
"""
|
||||
Calculate the consin distance of two vector
|
||||
|
||||
|
||||
Parameters
|
||||
----------
|
||||
x1, x2: input vectors
|
||||
|
|
@ -501,19 +502,19 @@ Visualize data
|
|||
image_indices = ["1_1.png", "1_2.png", "2_1.png"]
|
||||
image_paths = [utils.download_file(base_file_link + image_index, directory="data") for image_index in image_indices]
|
||||
image1, image2, image3 = [cv2.cvtColor(cv2.imread(str(image_path)), cv2.COLOR_BGR2RGB) for image_path in image_paths]
|
||||
|
||||
|
||||
# Define titles with images.
|
||||
data = {"Person 1": image1, "Person 2": image2, "Person 3": image3}
|
||||
|
||||
|
||||
# Create a subplot to visualize images.
|
||||
fig, axs = plt.subplots(1, len(data.items()), figsize=(5, 5))
|
||||
|
||||
|
||||
# Fill the subplot.
|
||||
for ax, (name, image) in zip(axs, data.items()):
|
||||
ax.axis("off")
|
||||
ax.set_title(name)
|
||||
ax.imshow(image)
|
||||
|
||||
|
||||
# Display an image.
|
||||
plt.show(fig)
|
||||
|
||||
|
|
@ -582,7 +583,7 @@ video file.
|
|||
2. Prepare a set of frames for person tracking.
|
||||
3. Run AI inference for person tracking.
|
||||
4. Visualize the results.
|
||||
|
||||
|
||||
Parameters:
|
||||
----------
|
||||
source: The webcam number to feed the video stream with primary webcam set to "0", or the video path.
|
||||
|
|
@ -605,7 +606,7 @@ video file.
|
|||
if use_popup:
|
||||
title = "Press ESC to Exit"
|
||||
cv2.namedWindow(winname=title, flags=cv2.WINDOW_GUI_NORMAL | cv2.WINDOW_AUTOSIZE)
|
||||
|
||||
|
||||
processing_times = collections.deque()
|
||||
while True:
|
||||
# Grab the frame.
|
||||
|
|
@ -614,11 +615,11 @@ video file.
|
|||
print("Source ended")
|
||||
break
|
||||
# If the frame is larger than full HD, reduce size to improve the performance.
|
||||
|
||||
|
||||
# Resize the image and change dims to fit neural network input.
|
||||
h, w = frame.shape[:2]
|
||||
input_image = preprocess(frame, detector.height, detector.width)
|
||||
|
||||
|
||||
# Measure processing time.
|
||||
start_time = time.time()
|
||||
# Get the results.
|
||||
|
|
@ -627,21 +628,21 @@ video file.
|
|||
processing_times.append(stop_time - start_time)
|
||||
if len(processing_times) > 200:
|
||||
processing_times.popleft()
|
||||
|
||||
|
||||
_, f_width = frame.shape[:2]
|
||||
# Mean processing time [ms].
|
||||
processing_time = np.mean(processing_times) * 1100
|
||||
fps = 1000 / processing_time
|
||||
|
||||
|
||||
# Get poses from detection results.
|
||||
bbox_xywh, score, label = process_results(h, w, results=output)
|
||||
|
||||
|
||||
img_crops = []
|
||||
for box in bbox_xywh:
|
||||
x1, y1, x2, y2 = xywh_to_xyxy(box, h, w)
|
||||
img = frame[y1:y2, x1:x2]
|
||||
img_crops.append(img)
|
||||
|
||||
|
||||
# Get reidentification feature of each person.
|
||||
if img_crops:
|
||||
# preprocess
|
||||
|
|
@ -649,17 +650,17 @@ video file.
|
|||
features = extractor.predict(img_batch)
|
||||
else:
|
||||
features = np.array([])
|
||||
|
||||
|
||||
# Wrap the detection and reidentification results together
|
||||
bbox_tlwh = xywh_to_tlwh(bbox_xywh)
|
||||
detections = [Detection(bbox_tlwh[i], features[i]) for i in range(features.shape[0])]
|
||||
|
||||
|
||||
# predict the position of tracking target
|
||||
tracker.predict()
|
||||
|
||||
|
||||
# update tracker
|
||||
tracker.update(detections)
|
||||
|
||||
|
||||
# update bbox identities
|
||||
outputs = []
|
||||
for track in tracker.tracks:
|
||||
|
|
@ -671,14 +672,14 @@ video file.
|
|||
outputs.append(np.array([x1, y1, x2, y2, track_id], dtype=np.int32))
|
||||
if len(outputs) > 0:
|
||||
outputs = np.stack(outputs, axis=0)
|
||||
|
||||
|
||||
# draw box for visualization
|
||||
if len(outputs) > 0:
|
||||
bbox_tlwh = []
|
||||
bbox_xyxy = outputs[:, :4]
|
||||
identities = outputs[:, -1]
|
||||
frame = draw_boxes(frame, bbox_xyxy, identities)
|
||||
|
||||
|
||||
cv2.putText(
|
||||
img=frame,
|
||||
text=f"Inference time: {processing_time:.1f}ms ({fps:.1f} FPS)",
|
||||
|
|
@ -689,7 +690,7 @@ video file.
|
|||
thickness=1,
|
||||
lineType=cv2.LINE_AA,
|
||||
)
|
||||
|
||||
|
||||
if use_popup:
|
||||
cv2.imshow(winname=title, mat=frame)
|
||||
key = cv2.waitKey(1)
|
||||
|
|
@ -704,7 +705,7 @@ video file.
|
|||
# Display the image in this notebook.
|
||||
display.clear_output(wait=True)
|
||||
display.display(i)
|
||||
|
||||
|
||||
# ctrl-c
|
||||
except KeyboardInterrupt:
|
||||
print("Interrupted")
|
||||
|
|
@ -758,11 +759,11 @@ will work.
|
|||
.. code:: ipython3
|
||||
|
||||
USE_WEBCAM = False
|
||||
|
||||
|
||||
cam_id = 0
|
||||
video_file = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/video/people.mp4"
|
||||
source = cam_id if USE_WEBCAM else video_file
|
||||
|
||||
|
||||
run_person_tracking(source=source, flip=USE_WEBCAM, use_popup=False)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e1516e12fb1b935d115f20141dfd9ff90fa0f9abbf2e39a85e96de0954fab7de
|
||||
size 218937
|
||||
oid sha256:b26360db4003d149dd356ef7b559c7a7bfb8b4b4b627822abe1dffb8c2f2931a
|
||||
size 218887
|
||||
|
|
|
|||
|
|
@ -81,10 +81,10 @@ Clone PhotoMaker repository
|
|||
|
||||
Cloning into 'PhotoMaker'...
|
||||
remote: Enumerating objects: 236, done.[K
|
||||
remote: Counting objects: 100% (145/145), done.[K
|
||||
remote: Compressing objects: 100% (97/97), done.[K
|
||||
remote: Total 236 (delta 114), reused 65 (delta 48), pack-reused 91[K
|
||||
Receiving objects: 100% (236/236), 9.31 MiB | 26.64 MiB/s, done.
|
||||
remote: Counting objects: 100% (143/143), done.[K
|
||||
remote: Compressing objects: 100% (96/96), done.[K
|
||||
remote: Total 236 (delta 113), reused 64 (delta 47), pack-reused 93[K
|
||||
Receiving objects: 100% (236/236), 9.31 MiB | 24.77 MiB/s, done.
|
||||
Resolving deltas: 100% (120/120), done.
|
||||
|
||||
|
||||
|
|
@ -158,10 +158,10 @@ PhotoMaker to generate the original PhotoMaker pipeline.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:45:51.268817: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:45:51.303612: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 00:56:15.653138: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 00:56:15.688553: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:45:51.957581: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 00:56:16.345251: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
@ -175,7 +175,7 @@ PhotoMaker to generate the original PhotoMaker pipeline.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -356,13 +356,13 @@ output(text embeddings) which will be the input for U-Net model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/photo-maker/PhotoMaker/photomaker/model.py:84: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/photo-maker/PhotoMaker/photomaker/model.py:84: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert class_tokens_mask.sum() == stacked_id_embeds.shape[0], f"{class_tokens_mask.sum()} != {stacked_id_embeds.shape[0]}"
|
||||
|
||||
|
||||
|
|
@ -442,11 +442,11 @@ sequence of latent text embeddings.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input_shape[-1] > 1 or self.sliding_window is not None:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if past_key_values_length > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
|
||||
|
||||
|
||||
|
|
@ -574,15 +574,15 @@ original Stable Diffusion XL model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_condition.py:1110: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_condition.py:1110: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if dim % default_overall_up_factor != 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:137: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:137: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if hidden_states.shape[0] >= 64:
|
||||
|
||||
|
||||
|
|
@ -697,7 +697,7 @@ VAE decoder.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
6286
|
||||
6285
|
||||
|
||||
|
||||
|
||||
|
|
@ -1070,6 +1070,7 @@ Interactive Demo
|
|||
|
||||
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
||||
demo.close()
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1a74b5a0589bb13eb1e3e6be207a0682193cf5b0a3b721075a78c5145bd207cb
|
||||
size 108163
|
||||
oid sha256:1a4f362ade6abe7487381a15c7a75c9caed0404082776188ee121c324329bde7
|
||||
size 108011
|
||||
|
|
|
|||
|
|
@ -156,11 +156,11 @@ hub <https://huggingface.co/pyannote/speaker-diarization>`__.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:55:33.958421: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:55:33.992823: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 01:06:05.899253: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 01:06:05.934768: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:55:34.728268: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/pyannote/audio/core/io.py:42: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
|
||||
2024-05-16 01:06:06.671600: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/pyannote/audio/core/io.py:42: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
|
||||
torchaudio.set_audio_backend("soundfile")
|
||||
|
||||
|
||||
|
|
@ -251,7 +251,7 @@ pipeline
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Diarization pipeline took 17.11 s
|
||||
Diarization pipeline took 17.27 s
|
||||
|
||||
|
||||
The result of running the pipeline can be represented as a diagram
|
||||
|
|
@ -424,7 +424,7 @@ Run speaker diarization with OpenVINO
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Diarization pipeline took 16.62 s
|
||||
Diarization pipeline took 16.68 s
|
||||
|
||||
|
||||
.. code:: ipython3
|
||||
|
|
|
|||
|
|
@ -566,9 +566,9 @@ performance.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PyTorch model on CPU: 0.039 seconds per image, FPS: 25.93
|
||||
ONNX model in OpenVINO Runtime/AUTO: 0.018 seconds per image, FPS: 54.11
|
||||
OpenVINO IR model in OpenVINO Runtime/AUTO: 0.028 seconds per image, FPS: 35.60
|
||||
PyTorch model on CPU: 0.040 seconds per image, FPS: 24.97
|
||||
ONNX model in OpenVINO Runtime/AUTO: 0.018 seconds per image, FPS: 55.35
|
||||
OpenVINO IR model in OpenVINO Runtime/AUTO: 0.029 seconds per image, FPS: 34.17
|
||||
|
||||
|
||||
**Show Device Information**
|
||||
|
|
|
|||
|
|
@ -150,7 +150,7 @@ Settings
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/pytorch-post-training-quantization-nncf/model/resnet50_fp32.pth')
|
||||
|
||||
|
||||
|
||||
|
|
@ -440,15 +440,15 @@ I. Evaluate the loaded model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.249 (0.249) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [10/79] Time 0.223 (0.231) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [20/79] Time 0.224 (0.227) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [30/79] Time 0.223 (0.232) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [40/79] Time 0.225 (0.232) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [50/79] Time 0.224 (0.231) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [60/79] Time 0.214 (0.231) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
Test: [70/79] Time 0.218 (0.230) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 17.917
|
||||
Test: [ 0/79] Time 0.247 (0.247) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [10/79] Time 0.230 (0.232) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [20/79] Time 0.227 (0.232) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [30/79] Time 0.229 (0.231) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [40/79] Time 0.231 (0.231) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [50/79] Time 0.229 (0.231) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [60/79] Time 0.228 (0.231) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
Test: [70/79] Time 0.225 (0.233) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 18.168
|
||||
Test accuracy of FP32 model: 60.740
|
||||
|
||||
|
||||
|
|
@ -491,10 +491,10 @@ Guide <https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 00:57:16.172977: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 00:57:16.205211: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 01:08:40.762510: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 01:08:40.795475: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 00:57:16.762420: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 01:08:41.328093: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -563,16 +563,16 @@ Guide <https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.419 (0.419) Acc@1 80.47 (80.47) Acc@5 91.41 (91.41)
|
||||
Test: [10/79] Time 0.397 (0.397) Acc@1 53.12 (66.26) Acc@5 87.50 (87.86)
|
||||
Test: [20/79] Time 0.394 (0.397) Acc@1 68.75 (63.76) Acc@5 85.94 (87.43)
|
||||
Test: [30/79] Time 0.397 (0.396) Acc@1 50.78 (61.97) Acc@5 74.22 (85.28)
|
||||
Test: [40/79] Time 0.396 (0.396) Acc@1 68.75 (60.54) Acc@5 89.06 (84.32)
|
||||
Test: [50/79] Time 0.395 (0.396) Acc@1 59.38 (60.48) Acc@5 87.50 (84.16)
|
||||
Test: [60/79] Time 0.413 (0.396) Acc@1 65.62 (60.27) Acc@5 85.16 (83.53)
|
||||
Test: [70/79] Time 0.393 (0.405) Acc@1 53.91 (60.00) Acc@5 79.69 (83.14)
|
||||
* Acc@1 60.530 Acc@5 83.770 Total time: 31.659
|
||||
Accuracy of initialized INT8 model: 60.530
|
||||
Test: [ 0/79] Time 0.418 (0.418) Acc@1 81.25 (81.25) Acc@5 91.41 (91.41)
|
||||
Test: [10/79] Time 0.395 (0.396) Acc@1 53.12 (66.12) Acc@5 86.72 (87.64)
|
||||
Test: [20/79] Time 0.395 (0.395) Acc@1 68.75 (63.73) Acc@5 85.94 (87.31)
|
||||
Test: [30/79] Time 0.391 (0.394) Acc@1 51.56 (62.10) Acc@5 75.00 (85.11)
|
||||
Test: [40/79] Time 0.392 (0.394) Acc@1 67.97 (60.69) Acc@5 89.84 (84.18)
|
||||
Test: [50/79] Time 0.395 (0.394) Acc@1 59.38 (60.55) Acc@5 87.50 (84.08)
|
||||
Test: [60/79] Time 0.393 (0.394) Acc@1 66.41 (60.30) Acc@5 85.94 (83.52)
|
||||
Test: [70/79] Time 0.393 (0.393) Acc@1 53.12 (59.99) Acc@5 78.91 (83.11)
|
||||
* Acc@1 60.520 Acc@5 83.740 Total time: 30.806
|
||||
Accuracy of initialized INT8 model: 60.520
|
||||
|
||||
|
||||
It should be noted that the inference time for the quantized PyTorch
|
||||
|
|
@ -621,16 +621,16 @@ For more information about model conversion, refer to this
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_low.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
return self._level_high.item()
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
|
||||
Tensor-likes are not close!
|
||||
|
||||
Mismatched elements: 25557 / 25600 (99.8%)
|
||||
Greatest absolute difference: 0.21502065658569336 at index (75, 14) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 47.884264953188925 at index (81, 88) (up to 1e-05 allowed)
|
||||
Mismatched elements: 25563 / 25600 (99.9%)
|
||||
Greatest absolute difference: 0.17899155616760254 at index (4, 14) (up to 1e-05 allowed)
|
||||
Greatest relative difference: 195.06354978354977 at index (43, 158) (up to 1e-05 allowed)
|
||||
_check_trace(
|
||||
|
||||
|
||||
|
|
@ -671,15 +671,15 @@ Evaluate the FP32 and INT8 models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.185 (0.185) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [10/79] Time 0.138 (0.143) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [20/79] Time 0.138 (0.141) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [30/79] Time 0.139 (0.140) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [40/79] Time 0.140 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [50/79] Time 0.141 (0.140) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [ 0/79] Time 0.197 (0.197) Acc@1 81.25 (81.25) Acc@5 92.19 (92.19)
|
||||
Test: [10/79] Time 0.136 (0.144) Acc@1 56.25 (66.97) Acc@5 86.72 (87.50)
|
||||
Test: [20/79] Time 0.139 (0.142) Acc@1 67.97 (64.29) Acc@5 85.16 (87.35)
|
||||
Test: [30/79] Time 0.139 (0.141) Acc@1 53.12 (62.37) Acc@5 77.34 (85.33)
|
||||
Test: [40/79] Time 0.141 (0.140) Acc@1 67.19 (60.86) Acc@5 90.62 (84.51)
|
||||
Test: [50/79] Time 0.138 (0.140) Acc@1 60.16 (60.80) Acc@5 88.28 (84.42)
|
||||
Test: [60/79] Time 0.139 (0.140) Acc@1 66.41 (60.46) Acc@5 86.72 (83.79)
|
||||
Test: [70/79] Time 0.136 (0.139) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 10.900
|
||||
Test: [70/79] Time 0.140 (0.140) Acc@1 52.34 (60.21) Acc@5 80.47 (83.33)
|
||||
* Acc@1 60.740 Acc@5 83.960 Total time: 10.934
|
||||
Accuracy of FP32 IR model: 60.740
|
||||
|
||||
|
||||
|
|
@ -692,16 +692,16 @@ Evaluate the FP32 and INT8 models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.143 (0.143) Acc@1 80.47 (80.47) Acc@5 91.41 (91.41)
|
||||
Test: [10/79] Time 0.077 (0.085) Acc@1 53.91 (66.55) Acc@5 86.72 (87.86)
|
||||
Test: [20/79] Time 0.078 (0.082) Acc@1 68.75 (64.03) Acc@5 85.94 (87.39)
|
||||
Test: [30/79] Time 0.077 (0.081) Acc@1 50.00 (62.25) Acc@5 74.22 (85.28)
|
||||
Test: [40/79] Time 0.078 (0.080) Acc@1 68.75 (60.90) Acc@5 89.06 (84.36)
|
||||
Test: [50/79] Time 0.077 (0.080) Acc@1 60.94 (60.78) Acc@5 87.50 (84.22)
|
||||
Test: [60/79] Time 0.077 (0.079) Acc@1 66.41 (60.51) Acc@5 85.94 (83.61)
|
||||
Test: [70/79] Time 0.078 (0.079) Acc@1 52.34 (60.19) Acc@5 79.69 (83.21)
|
||||
* Acc@1 60.660 Acc@5 83.790 Total time: 6.202
|
||||
Accuracy of INT8 IR model: 60.660
|
||||
Test: [ 0/79] Time 0.144 (0.144) Acc@1 80.47 (80.47) Acc@5 91.41 (91.41)
|
||||
Test: [10/79] Time 0.080 (0.084) Acc@1 50.78 (66.12) Acc@5 86.72 (87.71)
|
||||
Test: [20/79] Time 0.076 (0.081) Acc@1 68.75 (63.65) Acc@5 85.94 (87.31)
|
||||
Test: [30/79] Time 0.078 (0.080) Acc@1 50.78 (62.10) Acc@5 74.22 (85.23)
|
||||
Test: [40/79] Time 0.078 (0.079) Acc@1 68.75 (60.79) Acc@5 89.84 (84.39)
|
||||
Test: [50/79] Time 0.076 (0.079) Acc@1 60.16 (60.72) Acc@5 87.50 (84.25)
|
||||
Test: [60/79] Time 0.078 (0.078) Acc@1 65.62 (60.46) Acc@5 85.94 (83.66)
|
||||
Test: [70/79] Time 0.078 (0.078) Acc@1 53.12 (60.18) Acc@5 79.69 (83.26)
|
||||
* Acc@1 60.670 Acc@5 83.850 Total time: 6.116
|
||||
Accuracy of INT8 IR model: 60.670
|
||||
|
||||
|
||||
IV. Compare performance of INT8 model and FP32 model in OpenVINO
|
||||
|
|
@ -766,13 +766,13 @@ throughput (frames per second) values.
|
|||
.. parsed-literal::
|
||||
|
||||
Benchmark FP32 model (OpenVINO IR)
|
||||
[ INFO ] Throughput: 38.47 FPS
|
||||
[ INFO ] Throughput: 38.09 FPS
|
||||
Benchmark INT8 model (OpenVINO IR)
|
||||
[ INFO ] Throughput: 154.26 FPS
|
||||
[ INFO ] Throughput: 157.16 FPS
|
||||
Benchmark FP32 model (OpenVINO IR) synchronously
|
||||
[ INFO ] Throughput: 39.64 FPS
|
||||
[ INFO ] Throughput: 39.29 FPS
|
||||
Benchmark INT8 model (OpenVINO IR) synchronously
|
||||
[ INFO ] Throughput: 135.44 FPS
|
||||
[ INFO ] Throughput: 134.04 FPS
|
||||
|
||||
|
||||
Show device Information for reference:
|
||||
|
|
|
|||
|
|
@ -155,7 +155,7 @@ models will be stored.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/pytorch-quantization-aware-training/model/resnet18_fp32.pth')
|
||||
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/pytorch-quantization-aware-training/model/resnet18_fp32.pth')
|
||||
|
||||
|
||||
|
||||
|
|
@ -460,9 +460,9 @@ section at the top of this notebook.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
|
||||
warnings.warn(msg)
|
||||
|
||||
|
||||
|
|
@ -572,10 +572,10 @@ about supported parameters can be found on this
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 01:02:39.722072: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 01:02:39.758422: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 01:14:13.011381: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 01:14:13.048138: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 01:02:40.294932: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
2024-05-16 01:14:13.581913: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -644,14 +644,14 @@ demonstrated here.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Test: [ 0/79] Time 0.202 (0.202) Loss 1.005 (1.005) Acc@1 78.91 (78.91) Acc@5 88.28 (88.28)
|
||||
Test: [10/79] Time 0.164 (0.169) Loss 1.992 (1.625) Acc@1 44.53 (60.37) Acc@5 79.69 (83.66)
|
||||
Test: [20/79] Time 0.163 (0.167) Loss 1.814 (1.705) Acc@1 60.94 (58.04) Acc@5 80.47 (82.66)
|
||||
Test: [30/79] Time 0.163 (0.166) Loss 2.287 (1.795) Acc@1 50.78 (56.48) Acc@5 68.75 (80.97)
|
||||
Test: [40/79] Time 0.171 (0.167) Loss 1.615 (1.832) Acc@1 60.94 (55.43) Acc@5 82.81 (80.43)
|
||||
Test: [50/79] Time 0.165 (0.166) Loss 1.952 (1.833) Acc@1 57.03 (55.51) Acc@5 75.00 (80.16)
|
||||
Test: [60/79] Time 0.197 (0.167) Loss 1.794 (1.856) Acc@1 57.03 (55.16) Acc@5 84.38 (79.84)
|
||||
Test: [70/79] Time 0.192 (0.169) Loss 2.371 (1.889) Acc@1 46.88 (54.68) Acc@5 74.22 (79.14)
|
||||
Test: [ 0/79] Time 0.217 (0.217) Loss 1.005 (1.005) Acc@1 78.91 (78.91) Acc@5 88.28 (88.28)
|
||||
Test: [10/79] Time 0.170 (0.175) Loss 1.992 (1.625) Acc@1 44.53 (60.37) Acc@5 79.69 (83.66)
|
||||
Test: [20/79] Time 0.169 (0.171) Loss 1.814 (1.705) Acc@1 60.94 (58.04) Acc@5 80.47 (82.66)
|
||||
Test: [30/79] Time 0.169 (0.170) Loss 2.287 (1.795) Acc@1 50.78 (56.48) Acc@5 68.75 (80.97)
|
||||
Test: [40/79] Time 0.186 (0.170) Loss 1.615 (1.832) Acc@1 60.94 (55.43) Acc@5 82.81 (80.43)
|
||||
Test: [50/79] Time 0.166 (0.171) Loss 1.952 (1.833) Acc@1 57.03 (55.51) Acc@5 75.00 (80.16)
|
||||
Test: [60/79] Time 0.165 (0.170) Loss 1.794 (1.856) Acc@1 57.03 (55.16) Acc@5 84.38 (79.84)
|
||||
Test: [70/79] Time 0.166 (0.170) Loss 2.371 (1.889) Acc@1 46.88 (54.68) Acc@5 74.22 (79.14)
|
||||
* Acc@1 55.040 Acc@5 79.730
|
||||
Accuracy of initialized INT8 model: 55.040
|
||||
|
||||
|
|
@ -685,29 +685,29 @@ training pipeline are required. Here is a simple example.
|
|||
.. parsed-literal::
|
||||
|
||||
Epoch:[0][ 0/782] Time 0.416 (0.416) Loss 0.917 (0.917) Acc@1 76.56 (76.56) Acc@5 93.75 (93.75)
|
||||
Epoch:[0][ 50/782] Time 0.369 (0.369) Loss 0.628 (0.812) Acc@1 87.50 (80.33) Acc@5 96.09 (93.92)
|
||||
Epoch:[0][100/782] Time 0.364 (0.368) Loss 0.759 (0.806) Acc@1 79.69 (80.55) Acc@5 93.75 (94.12)
|
||||
Epoch:[0][150/782] Time 0.363 (0.368) Loss 0.865 (0.799) Acc@1 82.81 (80.71) Acc@5 92.97 (94.18)
|
||||
Epoch:[0][200/782] Time 0.363 (0.368) Loss 0.581 (0.787) Acc@1 86.72 (80.96) Acc@5 97.66 (94.32)
|
||||
Epoch:[0][250/782] Time 0.363 (0.368) Loss 0.720 (0.782) Acc@1 83.59 (81.00) Acc@5 93.75 (94.42)
|
||||
Epoch:[0][300/782] Time 0.363 (0.368) Loss 0.739 (0.777) Acc@1 78.91 (81.11) Acc@5 93.75 (94.40)
|
||||
Epoch:[0][350/782] Time 0.364 (0.368) Loss 0.819 (0.767) Acc@1 78.12 (81.36) Acc@5 92.97 (94.52)
|
||||
Epoch:[0][400/782] Time 0.375 (0.368) Loss 0.787 (0.766) Acc@1 80.47 (81.42) Acc@5 94.53 (94.51)
|
||||
Epoch:[0][450/782] Time 0.366 (0.368) Loss 0.733 (0.763) Acc@1 82.03 (81.55) Acc@5 96.88 (94.54)
|
||||
Epoch:[0][500/782] Time 0.368 (0.368) Loss 0.728 (0.760) Acc@1 82.81 (81.59) Acc@5 94.53 (94.57)
|
||||
Epoch:[0][550/782] Time 0.371 (0.368) Loss 0.777 (0.758) Acc@1 83.59 (81.63) Acc@5 95.31 (94.59)
|
||||
Epoch:[0][600/782] Time 0.372 (0.368) Loss 0.725 (0.756) Acc@1 80.47 (81.67) Acc@5 97.66 (94.60)
|
||||
Epoch:[0][650/782] Time 0.363 (0.368) Loss 0.920 (0.755) Acc@1 76.56 (81.66) Acc@5 92.97 (94.62)
|
||||
Epoch:[0][700/782] Time 0.365 (0.369) Loss 0.648 (0.753) Acc@1 84.38 (81.70) Acc@5 92.97 (94.63)
|
||||
Epoch:[0][750/782] Time 0.366 (0.368) Loss 0.782 (0.750) Acc@1 80.47 (81.71) Acc@5 94.53 (94.66)
|
||||
Test: [ 0/79] Time 0.145 (0.145) Loss 1.094 (1.094) Acc@1 76.56 (76.56) Acc@5 86.72 (86.72)
|
||||
Test: [10/79] Time 0.143 (0.144) Loss 1.848 (1.527) Acc@1 49.22 (62.93) Acc@5 80.47 (84.30)
|
||||
Test: [20/79] Time 0.143 (0.144) Loss 1.540 (1.597) Acc@1 64.06 (60.64) Acc@5 81.25 (83.82)
|
||||
Test: [30/79] Time 0.146 (0.144) Loss 2.052 (1.692) Acc@1 56.25 (59.20) Acc@5 71.88 (82.21)
|
||||
Test: [40/79] Time 0.143 (0.144) Loss 1.515 (1.745) Acc@1 64.06 (57.79) Acc@5 85.16 (81.44)
|
||||
Test: [50/79] Time 0.143 (0.144) Loss 1.915 (1.751) Acc@1 53.91 (57.60) Acc@5 77.34 (81.14)
|
||||
Test: [60/79] Time 0.143 (0.144) Loss 1.585 (1.786) Acc@1 67.19 (57.01) Acc@5 85.16 (80.69)
|
||||
Test: [70/79] Time 0.142 (0.144) Loss 2.454 (1.812) Acc@1 44.53 (56.57) Acc@5 74.22 (80.24)
|
||||
Epoch:[0][ 50/782] Time 0.391 (0.370) Loss 0.628 (0.812) Acc@1 87.50 (80.33) Acc@5 96.09 (93.92)
|
||||
Epoch:[0][100/782] Time 0.370 (0.377) Loss 0.759 (0.806) Acc@1 79.69 (80.55) Acc@5 93.75 (94.12)
|
||||
Epoch:[0][150/782] Time 0.365 (0.374) Loss 0.865 (0.799) Acc@1 82.81 (80.71) Acc@5 92.97 (94.18)
|
||||
Epoch:[0][200/782] Time 0.371 (0.373) Loss 0.581 (0.787) Acc@1 86.72 (80.96) Acc@5 97.66 (94.32)
|
||||
Epoch:[0][250/782] Time 0.373 (0.372) Loss 0.720 (0.782) Acc@1 83.59 (81.00) Acc@5 93.75 (94.42)
|
||||
Epoch:[0][300/782] Time 0.369 (0.371) Loss 0.739 (0.777) Acc@1 78.91 (81.11) Acc@5 93.75 (94.40)
|
||||
Epoch:[0][350/782] Time 0.369 (0.371) Loss 0.819 (0.767) Acc@1 78.12 (81.36) Acc@5 92.97 (94.52)
|
||||
Epoch:[0][400/782] Time 0.369 (0.371) Loss 0.787 (0.766) Acc@1 80.47 (81.42) Acc@5 94.53 (94.51)
|
||||
Epoch:[0][450/782] Time 0.366 (0.371) Loss 0.733 (0.763) Acc@1 82.03 (81.55) Acc@5 96.88 (94.54)
|
||||
Epoch:[0][500/782] Time 0.369 (0.370) Loss 0.728 (0.760) Acc@1 82.81 (81.59) Acc@5 94.53 (94.57)
|
||||
Epoch:[0][550/782] Time 0.369 (0.370) Loss 0.777 (0.758) Acc@1 83.59 (81.63) Acc@5 95.31 (94.59)
|
||||
Epoch:[0][600/782] Time 0.364 (0.370) Loss 0.725 (0.756) Acc@1 80.47 (81.67) Acc@5 97.66 (94.60)
|
||||
Epoch:[0][650/782] Time 0.389 (0.370) Loss 0.920 (0.755) Acc@1 76.56 (81.66) Acc@5 92.97 (94.62)
|
||||
Epoch:[0][700/782] Time 0.370 (0.370) Loss 0.648 (0.753) Acc@1 84.38 (81.70) Acc@5 92.97 (94.63)
|
||||
Epoch:[0][750/782] Time 0.368 (0.370) Loss 0.782 (0.750) Acc@1 80.47 (81.71) Acc@5 94.53 (94.66)
|
||||
Test: [ 0/79] Time 0.142 (0.142) Loss 1.094 (1.094) Acc@1 76.56 (76.56) Acc@5 86.72 (86.72)
|
||||
Test: [10/79] Time 0.142 (0.143) Loss 1.848 (1.527) Acc@1 49.22 (62.93) Acc@5 80.47 (84.30)
|
||||
Test: [20/79] Time 0.142 (0.143) Loss 1.540 (1.597) Acc@1 64.06 (60.64) Acc@5 81.25 (83.82)
|
||||
Test: [30/79] Time 0.143 (0.144) Loss 2.052 (1.692) Acc@1 56.25 (59.20) Acc@5 71.88 (82.21)
|
||||
Test: [40/79] Time 0.142 (0.144) Loss 1.515 (1.745) Acc@1 64.06 (57.79) Acc@5 85.16 (81.44)
|
||||
Test: [50/79] Time 0.142 (0.144) Loss 1.915 (1.751) Acc@1 53.91 (57.60) Acc@5 77.34 (81.14)
|
||||
Test: [60/79] Time 0.142 (0.144) Loss 1.585 (1.786) Acc@1 67.19 (57.01) Acc@5 85.16 (80.69)
|
||||
Test: [70/79] Time 0.145 (0.144) Loss 2.454 (1.812) Acc@1 44.53 (56.57) Acc@5 74.22 (80.24)
|
||||
* Acc@1 56.970 Acc@5 80.830
|
||||
Accuracy of tuned INT8 model: 56.970
|
||||
Accuracy drop of tuned INT8 model over pre-trained FP32 model: -1.450
|
||||
|
|
@ -800,9 +800,9 @@ throughput (frames per second) values.
|
|||
.. parsed-literal::
|
||||
|
||||
Benchmark FP32 model (IR)
|
||||
[ INFO ] Throughput: 2943.57 FPS
|
||||
[ INFO ] Throughput: 2954.77 FPS
|
||||
Benchmark INT8 model (IR)
|
||||
[ INFO ] Throughput: 11901.74 FPS
|
||||
[ INFO ] Throughput: 11676.13 FPS
|
||||
|
||||
|
||||
Show Device Information for reference.
|
||||
|
|
|
|||
|
|
@ -251,7 +251,7 @@ Benchmark PyTorch Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
16.9 ms ± 130 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
18.6 ms ± 32.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -410,7 +410,7 @@ Benchmark OpenVINO Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.16 ms ± 11 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.3 ms ± 10 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Model with Static Input Shape
|
||||
|
|
@ -540,7 +540,7 @@ Benchmark OpenVINO Model Inference with Static Input Shape
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2.95 ms ± 6.16 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.04 ms ± 21.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert TorchScript Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -635,7 +635,7 @@ Benchmark Scripted Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
14.1 ms ± 67.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
14.3 ms ± 18.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Scripted Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -694,7 +694,7 @@ Benchmark OpenVINO Model Inference Converted From Scripted Model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.27 ms ± 6.74 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.23 ms ± 25.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Traced Model
|
||||
|
|
@ -770,7 +770,7 @@ Benchmark Traced Model Inference
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
14.7 ms ± 193 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
14.7 ms ± 635 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
||||
Convert PyTorch Traced Model to OpenVINO Intermediate Representation
|
||||
|
|
@ -829,5 +829,5 @@ Benchmark OpenVINO Model Inference Converted From Traced Model
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
3.4 ms ± 3.83 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
3.36 ms ± 32.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
|
||||
|
||||
|
|
|
|||
|
|
@ -101,7 +101,7 @@ it may take some time.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -222,7 +222,7 @@ function or directly loading on device using ``core.complie_model``.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -110,17 +110,17 @@ pipeline on disk.
|
|||
from diffusers import LCMScheduler, AutoPipelineForText2Image
|
||||
import gc
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
model_id = "segmind/Segmind-Vega"
|
||||
adapter_id = "segmind/Segmind-VegaRT"
|
||||
pt_model_dir = Path("segmind-vegart")
|
||||
|
||||
|
||||
if not pt_model_dir.exists():
|
||||
pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
|
||||
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
|
||||
pipe.load_lora_weights(adapter_id)
|
||||
pipe.fuse_lora()
|
||||
|
||||
|
||||
pipe.save_pretrained("segmind-vegart")
|
||||
del pipe
|
||||
gc.collect()
|
||||
|
|
@ -176,7 +176,7 @@ back to image format.
|
|||
.. code:: ipython3
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
model_dir = Path("openvino-segmind-vegart")
|
||||
sdxl_model_id = "./segmind-vegart"
|
||||
tae_id = "madebyollin/taesdxl"
|
||||
|
|
@ -188,26 +188,26 @@ back to image format.
|
|||
import openvino as ov
|
||||
from diffusers import AutoencoderTiny
|
||||
import gc
|
||||
|
||||
|
||||
|
||||
|
||||
class VAEEncoder(torch.nn.Module):
|
||||
def __init__(self, vae):
|
||||
super().__init__()
|
||||
self.vae = vae
|
||||
|
||||
|
||||
def forward(self, sample):
|
||||
return self.vae.encode(sample)
|
||||
|
||||
|
||||
|
||||
|
||||
class VAEDecoder(torch.nn.Module):
|
||||
def __init__(self, vae):
|
||||
super().__init__()
|
||||
self.vae = vae
|
||||
|
||||
|
||||
def forward(self, latent_sample):
|
||||
return self.vae.decode(latent_sample)
|
||||
|
||||
|
||||
|
||||
|
||||
def convert_tiny_vae(model_id, output_path):
|
||||
tiny_vae = AutoencoderTiny.from_pretrained(model_id)
|
||||
tiny_vae.eval()
|
||||
|
|
@ -222,8 +222,8 @@ back to image format.
|
|||
del tiny_vae
|
||||
del ov_model
|
||||
gc.collect()
|
||||
|
||||
|
||||
|
||||
|
||||
if not skip_convert_model:
|
||||
!optimum-cli export openvino --model $sdxl_model_id --task stable-diffusion-xl $model_dir --fp16
|
||||
convert_tiny_vae(tae_id, model_dir)
|
||||
|
|
@ -245,7 +245,7 @@ For saving time, we will not cover image-to-image generation in this
|
|||
notebook. As we already mentioned, Segmind-Vega is compatible with
|
||||
Stable Diffusion XL pipeline, the steps required to run Stable Diffusion
|
||||
XL inference for image-to-image task were discussed in this
|
||||
`notebook <stable-dffision-xl-with-output.html>`__.
|
||||
`notebook <stable-dffision-xl.ipynb>`__.
|
||||
|
||||
Select inference device for text-to-image generation
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -255,16 +255,16 @@ Select inference device for text-to-image generation
|
|||
.. code:: ipython3
|
||||
|
||||
import ipywidgets as widgets
|
||||
|
||||
|
||||
core = ov.Core()
|
||||
|
||||
|
||||
device = widgets.Dropdown(
|
||||
options=core.available_devices + ["AUTO"],
|
||||
value="AUTO",
|
||||
description="Device:",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
device
|
||||
|
||||
|
||||
|
|
@ -279,7 +279,7 @@ Select inference device for text-to-image generation
|
|||
.. code:: ipython3
|
||||
|
||||
from optimum.intel.openvino import OVStableDiffusionXLPipeline
|
||||
|
||||
|
||||
text2image_pipe = OVStableDiffusionXLPipeline.from_pretrained(model_dir, device=device.value)
|
||||
|
||||
|
||||
|
|
@ -301,9 +301,9 @@ Select inference device for text-to-image generation
|
|||
.. code:: ipython3
|
||||
|
||||
from transformers import set_seed
|
||||
|
||||
|
||||
set_seed(23)
|
||||
|
||||
|
||||
prompt = "A cinematic highly detailed shot of a baby Yorkshire terrier wearing an intricate Italian priest robe, with crown"
|
||||
image = text2image_pipe(prompt, num_inference_steps=4, height=512, width=512, guidance_scale=0.5).images[0]
|
||||
image.save("dog.png")
|
||||
|
|
@ -363,7 +363,7 @@ improve model inference speed.
|
|||
description="Quantization",
|
||||
disabled=False,
|
||||
)
|
||||
|
||||
|
||||
to_quantize
|
||||
|
||||
|
||||
|
|
@ -379,14 +379,14 @@ improve model inference speed.
|
|||
|
||||
# Fetch `skip_kernel_extension` module
|
||||
import requests
|
||||
|
||||
|
||||
r = requests.get(
|
||||
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py",
|
||||
)
|
||||
open("skip_kernel_extension.py", "w").write(r.text)
|
||||
|
||||
|
||||
int8_pipe = None
|
||||
|
||||
|
||||
if to_quantize.value and "GPU" in device.value:
|
||||
to_quantize.value = False
|
||||
42
|
||||
|
|
@ -405,8 +405,8 @@ model inputs for calibration we should customize ``CompiledModel``.
|
|||
.. code:: ipython3
|
||||
|
||||
UNET_INT8_OV_PATH = model_dir / "optimized_unet" / "openvino_model.xml"
|
||||
|
||||
|
||||
|
||||
|
||||
def disable_progress_bar(pipeline, disable=True):
|
||||
if not hasattr(pipeline, "_progress_bar_config"):
|
||||
pipeline._progress_bar_config = {"disable": disable}
|
||||
|
|
@ -416,31 +416,31 @@ model inputs for calibration we should customize ``CompiledModel``.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
import datasets
|
||||
import numpy as np
|
||||
from tqdm.notebook import tqdm
|
||||
from transformers import set_seed
|
||||
from typing import Any, Dict, List
|
||||
|
||||
|
||||
set_seed(1)
|
||||
|
||||
|
||||
class CompiledModelDecorator(ov.CompiledModel):
|
||||
def __init__(self, compiled_model: ov.CompiledModel, data_cache: List[Any] = None):
|
||||
super().__init__(compiled_model)
|
||||
self.data_cache = data_cache if data_cache else []
|
||||
|
||||
|
||||
def __call__(self, *args, **kwargs):
|
||||
self.data_cache.append(*args)
|
||||
return super().__call__(*args, **kwargs)
|
||||
|
||||
|
||||
def collect_calibration_data(pipe, subset_size: int) -> List[Dict]:
|
||||
original_unet = pipe.unet.request
|
||||
pipe.unet.request = CompiledModelDecorator(original_unet)
|
||||
|
||||
|
||||
dataset = datasets.load_dataset("conceptual_captions", split="train").shuffle(seed=42)
|
||||
disable_progress_bar(pipe)
|
||||
|
||||
|
||||
# Run inference for data collection
|
||||
pbar = tqdm(total=subset_size)
|
||||
diff = 0
|
||||
|
|
@ -462,7 +462,7 @@ model inputs for calibration we should customize ``CompiledModel``.
|
|||
break
|
||||
pbar.update(collected_subset_size - diff)
|
||||
diff = collected_subset_size
|
||||
|
||||
|
||||
calibration_dataset = pipe.unet.request.data_cache
|
||||
disable_progress_bar(pipe, disable=False)
|
||||
pipe.unet.request = original_unet
|
||||
|
|
@ -471,7 +471,7 @@ model inputs for calibration we should customize ``CompiledModel``.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
if not UNET_INT8_OV_PATH.exists():
|
||||
text2image_pipe = OVStableDiffusionXLPipeline.from_pretrained(model_dir, device=device.value)
|
||||
unet_calibration_data = collect_calibration_data(text2image_pipe, subset_size=200)
|
||||
|
|
@ -492,10 +492,10 @@ sensitive ``Convolution`` layers in FP16 precision.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
import nncf
|
||||
from nncf.scopes import IgnoredScope
|
||||
|
||||
|
||||
UNET_OV_PATH = model_dir / "unet" / "openvino_model.xml"
|
||||
if not UNET_INT8_OV_PATH.exists():
|
||||
unet = core.read_model(UNET_OV_PATH)
|
||||
|
|
@ -516,7 +516,7 @@ sensitive ``Convolution`` layers in FP16 precision.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
def create_int8_pipe(model_dir, unet_int8_path, device, core, unet_device='CPU'):
|
||||
int8_pipe = OVStableDiffusionXLPipeline.from_pretrained(model_dir, device=device, compile=True)
|
||||
del int8_pipe.unet.request
|
||||
|
|
@ -525,12 +525,12 @@ sensitive ``Convolution`` layers in FP16 precision.
|
|||
int8_pipe.unet.model = core.read_model(unet_int8_path)
|
||||
int8_pipe.unet.request = core.compile_model(int8_pipe.unet.model, unet_device or device)
|
||||
return int8_pipe
|
||||
|
||||
|
||||
int8_text2image_pipe = create_int8_pipe(model_dir, UNET_INT8_OV_PATH, device.value, core)
|
||||
|
||||
|
||||
|
||||
|
||||
set_seed(23)
|
||||
|
||||
|
||||
image = int8_text2image_pipe(prompt, num_inference_steps=4, height=512, width=512, guidance_scale=0.5).images[0]
|
||||
display(image)
|
||||
|
||||
|
|
@ -563,10 +563,10 @@ Compare UNet file size
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
fp16_ir_model_size = UNET_OV_PATH.with_suffix(".bin").stat().st_size / 1024
|
||||
quantized_model_size = UNET_INT8_OV_PATH.with_suffix(".bin").stat().st_size / 1024
|
||||
|
||||
|
||||
print(f"FP16 model size: {fp16_ir_model_size:.2f} KB")
|
||||
print(f"INT8 model size: {quantized_model_size:.2f} KB")
|
||||
print(f"Model compression rate: {fp16_ir_model_size / quantized_model_size:.3f}")
|
||||
|
|
@ -594,9 +594,9 @@ pipelines, we use median inference time on the calibration subset.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
import time
|
||||
|
||||
|
||||
validation_size = 7
|
||||
calibration_dataset = datasets.load_dataset("conceptual_captions", split="train")
|
||||
validation_data = []
|
||||
|
|
@ -605,11 +605,11 @@ pipelines, we use median inference time on the calibration subset.
|
|||
break
|
||||
prompt = batch["caption"]
|
||||
validation_data.append(prompt)
|
||||
|
||||
|
||||
def calculate_inference_time(pipe, dataset):
|
||||
inference_time = []
|
||||
disable_progress_bar(pipe)
|
||||
|
||||
|
||||
for prompt in dataset:
|
||||
start = time.perf_counter()
|
||||
image = pipe(
|
||||
|
|
@ -634,15 +634,15 @@ pipelines, we use median inference time on the calibration subset.
|
|||
.. code:: ipython3
|
||||
|
||||
%%skip not $to_quantize.value
|
||||
|
||||
|
||||
int8_latency = calculate_inference_time(int8_text2image_pipe, validation_data)
|
||||
|
||||
|
||||
del int8_text2image_pipe
|
||||
gc.collect()
|
||||
|
||||
|
||||
text2image_pipe = OVStableDiffusionXLPipeline.from_pretrained(model_dir, device=device.value)
|
||||
fp_latency = calculate_inference_time(text2image_pipe, validation_data)
|
||||
|
||||
|
||||
del text2image_pipe
|
||||
gc.collect()
|
||||
print(f"FP16 pipeline latency: {fp_latency:.3f}")
|
||||
|
|
@ -684,13 +684,13 @@ launch the interactive demo.
|
|||
.. code:: ipython3
|
||||
|
||||
quantized_model_present = UNET_INT8_OV_PATH.exists()
|
||||
|
||||
|
||||
use_quantized_model = widgets.Checkbox(
|
||||
value=quantized_model_present,
|
||||
description="Use quantized model",
|
||||
disabled=not quantized_model_present,
|
||||
)
|
||||
|
||||
|
||||
use_quantized_model
|
||||
|
||||
|
||||
|
|
@ -705,16 +705,16 @@ launch the interactive demo.
|
|||
.. code:: ipython3
|
||||
|
||||
import gradio as gr
|
||||
|
||||
|
||||
if use_quantized_model.value:
|
||||
if not quantized_model_present:
|
||||
raise RuntimeError("Quantized model not found.")
|
||||
text2image_pipe = create_int8_pipe(model_dir, UNET_INT8_OV_PATH, device.value, core)
|
||||
|
||||
|
||||
else:
|
||||
text2image_pipe = OVStableDiffusionXLPipeline.from_pretrained(model_dir, device=device.value)
|
||||
|
||||
|
||||
|
||||
|
||||
def generate_from_text(text, seed, num_steps, height, width):
|
||||
set_seed(seed)
|
||||
result = text2image_pipe(
|
||||
|
|
@ -725,8 +725,8 @@ launch the interactive demo.
|
|||
width=width,
|
||||
).images[0]
|
||||
return result
|
||||
|
||||
|
||||
|
||||
|
||||
with gr.Blocks() as demo:
|
||||
with gr.Column():
|
||||
positive_input = gr.Textbox(label="Text prompt")
|
||||
|
|
@ -772,7 +772,7 @@ launch the interactive demo.
|
|||
],
|
||||
[positive_input, seed_input],
|
||||
)
|
||||
|
||||
|
||||
# if you are launching remotely, specify server_name and server_port
|
||||
# demo.launch(server_name='your server name', server_port='server port in int')
|
||||
# Read more in the docs: https://gradio.app/docs/
|
||||
|
|
|
|||
|
|
@ -117,13 +117,13 @@ tokenizer and preparing the images.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
2024-05-07 01:10:53.448821: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-07 01:10:53.484554: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
2024-05-16 01:22:30.035493: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
|
||||
2024-05-16 01:22:30.071294: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
|
||||
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
|
||||
2024-05-07 01:10:54.080932: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
2024-05-16 01:22:30.670592: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -260,11 +260,11 @@ object ready to load on the device and start making predictions.
|
|||
.. parsed-literal::
|
||||
|
||||
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/siglip/modeling_siglip.py:354: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/siglip/modeling_siglip.py:354: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (batch_size, self.num_heads, q_len, k_v_seq_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/siglip/modeling_siglip.py:372: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/siglip/modeling_siglip.py:372: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (batch_size, self.num_heads, q_len, self.head_dim):
|
||||
|
||||
|
||||
|
|
@ -470,7 +470,7 @@ model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1486: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1486: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
|
||||
You can avoid this message in future by passing the argument `trust_remote_code=True`.
|
||||
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
|
||||
warnings.warn(
|
||||
|
|
@ -647,7 +647,7 @@ model are similar to the PyTorch model.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
[{'dog': 1.0}, {'horse': 0.0}, {'cat': 0.0}, {'tiger': 0.0}, {'wolf': 0.0}]
|
||||
[{'dog': 0.99}, {'horse': 0.0}, {'cat': 0.0}, {'tiger': 0.0}, {'wolf': 0.0}]
|
||||
|
||||
|
||||
|
||||
|
|
@ -722,7 +722,7 @@ approximately estimate the speed up of the dynamic quantized models.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
Performance speed up: 2.092
|
||||
Performance speed up: 2.078
|
||||
|
||||
|
||||
Interactive inference
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a8a6f33d09e139009ae19dbf3aa4942040d04c4abf091e3a31213d6c140eb580
|
||||
size 580932
|
||||
oid sha256:ef68ecd56d6879cabb231490beedcce22eac99a8d7ebb4a1e92c249da43a2112
|
||||
size 580945
|
||||
|
|
|
|||
|
|
@ -92,9 +92,9 @@ and install required packages.
|
|||
remote: Counting objects: 100% (70/70), done.[K
|
||||
remote: Compressing objects: 100% (26/26), done.[K
|
||||
remote: Total 205 (delta 53), reused 44 (delta 44), pack-reused 135[K
|
||||
Receiving objects: 100% (205/205), 31.89 MiB | 8.08 MiB/s, done.
|
||||
Receiving objects: 100% (205/205), 31.89 MiB | 22.14 MiB/s, done.
|
||||
Resolving deltas: 100% (96/96), done.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/notebooks/sketch-to-image-pix2pix-turbo/img2img-turbo
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/sketch-to-image-pix2pix-turbo/img2img-turbo
|
||||
|
||||
|
||||
Load PyTorch model
|
||||
|
|
@ -355,9 +355,9 @@ diagram indicate trainable layers. Semi-transparent layers are frozen.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -375,9 +375,9 @@ diagram indicate trainable layers. Semi-transparent layers are frozen.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
|
||||
torch.utils._pytree._register_pytree_node(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
@ -388,7 +388,7 @@ diagram indicate trainable layers. Semi-transparent layers are frozen.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
100%|██████████| 525M/525M [08:38<00:00, 1.01MiB/s]
|
||||
100%|██████████| 525M/525M [10:13<00:00, 855kiB/s]
|
||||
|
||||
|
||||
.. parsed-literal::
|
||||
|
|
@ -432,35 +432,35 @@ on disk using ``ov.save_model`` in compressed to FP16 format.
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if input_shape[-1] > 1 or self.sliding_window is not None:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if past_key_values_length > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:135: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:135: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unet_2d_condition.py:915: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unet_2d_condition.py:915: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if dim % default_overall_up_factor != 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
assert hidden_states.shape[1] == self.channels
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if hidden_states.shape[0] >= 64:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:433: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:433: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if model_output.shape[1] == sample.shape[1] * 2 and self.variance_type in ["learned", "learned_range"]:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:440: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:440: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
alpha_prod_t_prev = self.alphas_cumprod[prev_t] if prev_t >= 0 else self.one
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:479: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:479: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
if t > 0:
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:330: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/schedulers/scheduling_ddpm.py:330: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
|
||||
alpha_prod_t_prev = self.alphas_cumprod[prev_t] if prev_t >= 0 else self.one
|
||||
|
||||
|
||||
|
|
@ -904,11 +904,11 @@ Download results using download button
|
|||
|
||||
.. parsed-literal::
|
||||
|
||||
/tmp/ipykernel_2886802/1555011934.py:259: GradioDeprecationWarning: 'scale' value should be an integer. Using 0.4 will cause issues.
|
||||
/tmp/ipykernel_64378/1555011934.py:259: GradioDeprecationWarning: 'scale' value should be an integer. Using 0.4 will cause issues.
|
||||
with gr.Column(elem_id="column_process", min_width=50, scale=0.4):
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/gradio/utils.py:776: UserWarning: Expected 1 arguments for function <function <lambda> at 0x7f72861f9dc0>, received 0.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/gradio/utils.py:776: UserWarning: Expected 1 arguments for function <function <lambda> at 0x7f4b658d8670>, received 0.
|
||||
warnings.warn(
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-674/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/gradio/utils.py:780: UserWarning: Expected at least 1 arguments for function <function <lambda> at 0x7f72861f9dc0>, received 0.
|
||||
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/gradio/utils.py:780: UserWarning: Expected at least 1 arguments for function <function <lambda> at 0x7f4b658d8670>, received 0.
|
||||
warnings.warn(
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f98dd0a0866fe24d9076f351114c7f1810169a0c0c3dc87d19ffbf71e97da7ff
|
||||
size 23769
|
||||
oid sha256:033d1867d9ab107466af9dbfece37b9d3ce4ffeff93e5233023fabeb922095f7
|
||||
size 22253
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue