natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DOCS] Updating interactive tutorials (#24886)

This commit is contained in:
Sebastian Golebiewski 2024-06-10 07:34:50 +02:00 committed by GitHub
parent aaa634874c
commit 1144a66a9b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
182 changed files with 8221 additions and 3023 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

185
docs/articles_en/learn-openvino/interactive-tutorials-python/notebooks-installation.rst
View File

@ -32,15 +32,17 @@ The table below lists the supported operating systems and Python versions.
| | (64-bit | | | (64-bit |
| | ) <https://www.python.org/>`__ | | | ) <https://www.python.org/>`__ |
+=====================================+================================+ +=====================================+================================+
| Ubuntu 18.04 LTS | 3.8, 3.9, 3.10. 3.11 | | Ubuntu 20.04 LTS, 64-bit | 3.8, 3.9, 3.10. 3.11 |
+-------------------------------------+--------------------------------+ +-------------------------------------+--------------------------------+
| Ubuntu 20.04 LTS | 3.8, 3.9, 3.10, 3.11 | | Ubuntu 22.04 LTS, 64-bit | 3.8, 3.9, 3.10, 3.11 |
+-------------------------------------+--------------------------------+ +-------------------------------------+--------------------------------+
| Red Hat Enterprise Linux 8 | 3.8, 3.9, 3.10, 3.11 | | Red Hat Enterprise Linux 8 | 3.8, 3.9, 3.10, 3.11 |
+-------------------------------------+--------------------------------+ +-------------------------------------+--------------------------------+
| macOS 12.6.x versions | 3.8, 3.9, 3.10, 3.11 | | CentOS 7, 64 bit | 3.8, 3.9, 3.10, 3.11 |
+-------------------------------------+--------------------------------+ +-------------------------------------+--------------------------------+
| Windows 10 Pro, Enterprise | 3.8, 3.9, 3.10, 3.11 | | macOS 10.15.x versions or higher | 3.8, 3.9, 3.10, 3.11 |
+-------------------------------------+--------------------------------+
| Windows 10, 64-bit Pro, Enterprise | 3.8, 3.9, 3.10, 3.11 |
| or Education editions | | | or Education editions | |
+-------------------------------------+--------------------------------+ +-------------------------------------+--------------------------------+
| Windows Server 2016 or higher | 3.8, 3.9, 3.10, 3.11 | | Windows Server 2016 or higher | 3.8, 3.9, 3.10, 3.11 |
@ -64,6 +66,7 @@ Installing prerequisites
Run the installer by double clicking it. Follow the installation steps to set up the software. Run the installer by double clicking it. Follow the installation steps to set up the software.
While installing, make sure you check the box to *add Python to system PATH*. While installing, make sure you check the box to *add Python to system PATH*.
Also, it is recommended to use the installer option to disable the PATH length limit.
.. note:: .. note::
@ -81,6 +84,12 @@ Installing prerequisites
Run the installer by double clicking it. Follow the installation steps to set up the software. Run the installer by double clicking it. Follow the installation steps to set up the software.
4. (Optional) Install FFMPEG
Download FFMPEG binary from `here <https://ffmpeg.org/download.html>`__
Set FFMPEG's path (e.g., ``C:\ffmpeg\bin``) to the PATH environmental variable on Windows.
.. tab-item:: Linux .. tab-item:: Linux
:sync: linux :sync: linux
@ -96,7 +105,7 @@ Installing prerequisites
sudo apt-get update sudo apt-get update
sudo apt-get upgrade sudo apt-get upgrade
sudo apt-get install python3-venv build-essential python3-dev git-all sudo apt-get install python3-venv build-essential python3-dev git-all libgl1-mesa-dev ffmpeg
For an Intel Integrated Graphics Card, you can install the `Intel Graphics Compute Runtime <https://github.com/intel/compute-runtime>`__ to enable inference on this device. The command for Ubuntu 20.04 is: For an Intel Integrated Graphics Card, you can install the `Intel Graphics Compute Runtime <https://github.com/intel/compute-runtime>`__ to enable inference on this device. The command for Ubuntu 20.04 is:
@ -133,7 +142,8 @@ Installing prerequisites
.. code-block:: sh .. code-block:: sh
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
After you install it, follow the instructions from the Homebrew installation to set it up.
After you install it, follow the instructions from the Homebrew installation to set it up.
3. **Install Python and dependencies** 3. **Install Python and dependencies**
@ -142,6 +152,8 @@ Installing prerequisites
brew install python@3.9 brew install python@3.9
brew install protobuf brew install protobuf
# optional but recommended
brew install ffmpeg
Run each step below in a terminal. Run each step below in a terminal.
@ -250,6 +262,56 @@ Installing prerequisites
CMD /tmp/scripts/run CMD /tmp/scripts/run
.. tab-item:: Amazon SageMaker
:sync: amazon-sagemaker
.. note::
An `AWS <https://console.aws.amazon.com/console/home?nc2=h_ct&src=header-signin>`__
account and access to
`Amazon SageMaker Studio <https://aws.amazon.com/sagemaker/studio/>`__
are required.
1. **Log into your Amazon SageMaker Studio Environment and** ``Add user``.
|amazon-studio-1|
2. **Choose desired user profile name**
|amazon-studio-2|
3. **Choose Jupyter Lab version 3.0**
|amazon-studio-3|
4. **Choose the remaining default setting and click "Submit" to add a user.**
5. **Launch the Amazon SageMaker Studio environment.**
Click "Open Studio" to start the environment:
|amazon-studio-4|
.. note::
You are using an ``ml.t3.medium`` instance, which is for free for
250 hours per month for the first 2 months on Studio notebook.
6. **Wait for a couple of minutes for your environment to load.**
You should be able to see the following screen:
|amazon-studio-5|
7. **Select a SageMaker image.**
Choose ``Data Science 3.0`` in "Select a SageMaker image" drop-down under
"Notebooks and compute resources".
Then, click **+** on "Image Terminal" to start a terminal session:
|amazon-studio-6|
Installing notebooks Installing notebooks
++++++++++++++++++++ ++++++++++++++++++++
@ -294,6 +356,19 @@ Installing notebooks
pip install -r requirements.txt pip install -r requirements.txt
.. important::
In case of problems with accessing HuggingFace in PRC, set-up the networking
environment before you launch the notebooks:
.. code-block::
pip install -U huggingface_hub
set HF_ENDPOINT = https://hf-mirror.com
For more information, visit `HF-Mirror HuggingFace <https://hf-mirror.com>`__.
.. tab-item:: Linux .. tab-item:: Linux
:sync: linux :sync: linux
@ -333,6 +408,18 @@ Installing notebooks
pip install -r requirements.txt pip install -r requirements.txt
.. important::
In case of problems with accessing HuggingFace in PRC, set-up the networking
environment before you launch the notebooks:
.. code-block::
pip install -U huggingface_hub
set HF_ENDPOINT = https://hf-mirror.com
For more information, visit `HF-Mirror HuggingFace <https://hf-mirror.com>`__.
.. tab-item:: macOS .. tab-item:: macOS
:sync: macos :sync: macos
@ -475,6 +562,69 @@ Installing notebooks
While running the container on Windows and macOS, only CPU devices can be used. To access the iGPU, install the notebooks locally, following the instructions above. While running the container on Windows and macOS, only CPU devices can be used. To access the iGPU, install the notebooks locally, following the instructions above.
.. tab-item:: Amazon SageMaker
:sync: amazon-sagemaker
**Use the terminal and follow the steps below.**
|amazon-studio-7|
1. **Install few system dependencies.**
.. code-block::
apt update
apt install build-essential -y
apt install libpython3.9-dev -y
apt install libgl1-mesa-glx -y
2. **Setup OpenVINO conda environment.**
.. code-block::
conda create --name openvino_env python=3.9
conda activate openvino_env
conda install ipykernel
set PATH="/anaconda/envs/openvino_env/bin;%PATH%"
3. **Setup OpenVINO Notebooks.**
.. code-block::
git clone https://github.com/openvinotoolkit/openvino_notebooks.git
cd openvino_notebooks
# Install OpenVINO and OpenVINO notebook Requirements
python -m pip install --upgrade pip
pip install -r requirements.txt
4. **Run the Notebooks**
* To run the notebooks, click the top level "openvino_notebooks" folder
and navigate to your example:
|amazon-studio-8|
* Choose "Image" - ``Data Science 3.0``,
"Kernel" - ``Python [conda env:openvino_env],``
"Instance type"- your desired compute instance.
|amazon-studio-9|
|amazon-studio-10|
|amazon-studio-11|
.. note::
Make sure you use the ``Python [conda env:openvino_env]``
environment (not ``Python 3``).
* Next, run the cells of the notebook. You may try other notebooks to
explore OpenVINO features and examples.
Run the Notebooks Run the Notebooks
################# #################
@ -614,6 +764,27 @@ Additional Resources
.. |ml-studio-2| image:: https://user-images.githubusercontent.com/15709723/117582205-b6f4d580-b0b5-11eb-9b83-eb2004ad9b19.png .. |ml-studio-2| image:: https://user-images.githubusercontent.com/15709723/117582205-b6f4d580-b0b5-11eb-9b83-eb2004ad9b19.png
.. |amazon-studio-1| image:: https://user-images.githubusercontent.com/4837253/199801883-7bb64ad2-bb7f-4477-ace1-25111d4fd43c.png
.. |amazon-studio-2| image:: https://user-images.githubusercontent.com/4837253/199802173-8d65c851-604b-4b92-bafa-cae86b17d1ec.png
.. |amazon-studio-3| image:: https://user-images.githubusercontent.com/4837253/199802353-14c17233-3dae-4649-bbfe-59b8a598450c.png
.. |amazon-studio-4| image:: https://user-images.githubusercontent.com/4837253/199802726-97c85732-ff25-4cdd-ad6e-d491b4ed122b.png
.. |amazon-studio-5| image:: https://user-images.githubusercontent.com/15709723/199784252-c8581c73-342a-4c70-9207-5543d7b87346.png
.. |amazon-studio-6| image:: https://user-images.githubusercontent.com/4837253/199805717-5d102d27-e92e-4426-8d14-0484fd5ba24c.png
.. |amazon-studio-7| image:: https://user-images.githubusercontent.com/4837253/199807022-3cc5dd9e-f9f0-445d-be5e-d429dc1b752c.png
.. |amazon-studio-8| image:: https://user-images.githubusercontent.com/4837253/199810405-0f6748e1-d5f5-469e-8305-a96724dfffba.png
.. |amazon-studio-9| image:: https://user-images.githubusercontent.com/4837253/199812540-c52ea429-9d53-4bdb-aec1-a0b8616c6fcc.png
.. |amazon-studio-10| image:: https://user-images.githubusercontent.com/4837253/199812587-20c3e360-3a31-4032-b17a-8b242d6ccc26.png
.. |amazon-studio-11| image:: https://user-images.githubusercontent.com/4837253/199812713-32074aa7-8190-43c8-815c-231542c7b286.png
.. |docker-terminal-1| image:: https://user-images.githubusercontent.com/15709723/127793994-355e4d29-d131-432d-a12a-b08ca6131223.png .. |docker-terminal-1| image:: https://user-images.githubusercontent.com/15709723/127793994-355e4d29-d131-432d-a12a-b08ca6131223.png

2
docs/nbdoc/consts.py
View File

@ -6,7 +6,7 @@ repo_directory = "notebooks"
repo_owner = "openvinotoolkit" repo_owner = "openvinotoolkit"
repo_name = "openvino_notebooks" repo_name = "openvino_notebooks"
repo_branch = "tree/main" repo_branch = "tree/main"
artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240515220822/dist/rst_files/" artifacts_link = "http://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20240605220807/dist/rst_files/"
blacklisted_extensions = ['.xml', '.bin'] blacklisted_extensions = ['.xml', '.bin']
notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/" notebooks_repo = "https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/"
notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=" notebooks_binder = "https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath="

100
docs/notebooks/3D-pose-estimation-with-output.rst
View File

@ -69,82 +69,82 @@ Lab instead.**
Collecting openvino-dev>=2024.0.0 Collecting openvino-dev>=2024.0.0
Using cached openvino_dev-2024.1.0-15008-py3-none-any.whl.metadata (16 kB) Using cached openvino_dev-2024.1.0-15008-py3-none-any.whl.metadata (16 kB)
Collecting opencv-python Collecting opencv-python
Using cached opencv_python-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB) Using cached opencv_python-4.10.0.82-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Collecting torch Collecting torch
Using cached https://download.pytorch.org/whl/cpu/torch-2.3.0%2Bcpu-cp38-cp38-linux_x86_64.whl (190.4 MB) Using cached https://download.pytorch.org/whl/cpu/torch-2.3.1%2Bcpu-cp38-cp38-linux_x86_64.whl (190.4 MB)
Collecting onnx Collecting onnx
Using cached onnx-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (16 kB) Using cached onnx-1.16.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (16 kB)
Requirement already satisfied: ipywidgets>=7.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (8.1.2) Requirement already satisfied: ipywidgets>=7.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (8.1.3)
Collecting ipydatawidgets>=1.1.1 (from pythreejs) Collecting ipydatawidgets>=1.1.1 (from pythreejs)
Using cached ipydatawidgets-4.3.5-py2.py3-none-any.whl.metadata (1.4 kB) Using cached ipydatawidgets-4.3.5-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting numpy (from pythreejs) Collecting numpy (from pythreejs)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB) Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)
Requirement already satisfied: traitlets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (5.14.3) Requirement already satisfied: traitlets in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pythreejs) (5.14.3)
Requirement already satisfied: defusedxml>=0.7.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (0.7.1) Requirement already satisfied: defusedxml>=0.7.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (0.7.1)
Collecting networkx<=3.1.0 (from openvino-dev>=2024.0.0) Collecting networkx<=3.1.0 (from openvino-dev>=2024.0.0)
Using cached networkx-3.1-py3-none-any.whl.metadata (5.3 kB) Using cached networkx-3.1-py3-none-any.whl.metadata (5.3 kB)
Collecting openvino-telemetry>=2023.2.1 (from openvino-dev>=2024.0.0) Collecting openvino-telemetry>=2023.2.1 (from openvino-dev>=2024.0.0)
Using cached openvino_telemetry-2024.1.0-py3-none-any.whl.metadata (2.3 kB) Using cached openvino_telemetry-2024.1.0-py3-none-any.whl.metadata (2.3 kB)
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (24.0) Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (24.0)
Requirement already satisfied: pyyaml>=5.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (6.0.1) Requirement already satisfied: pyyaml>=5.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (6.0.1)
Requirement already satisfied: requests>=2.25.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (2.31.0) Requirement already satisfied: requests>=2.25.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino-dev>=2024.0.0) (2.32.0)
Collecting openvino==2024.1.0 (from openvino-dev>=2024.0.0) Collecting openvino==2024.1.0 (from openvino-dev>=2024.0.0)
Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl.metadata (8.8 kB) Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl.metadata (8.8 kB)
Collecting filelock (from torch) Collecting filelock (from torch)
Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB) Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB)
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (4.11.0) Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (4.12.1)
Collecting sympy (from torch) Collecting sympy (from torch)
Using cached https://download.pytorch.org/whl/sympy-1.12-py3-none-any.whl (5.7 MB) Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (3.1.4) Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch) (3.1.4)
Collecting fsspec (from torch) Collecting fsspec (from torch)
Downloading fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB) Using cached fsspec-2024.6.0-py3-none-any.whl.metadata (11 kB)
Collecting protobuf>=3.20.2 (from onnx) Collecting protobuf>=3.20.2 (from onnx)
Using cached protobuf-5.26.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (592 bytes) Using cached protobuf-5.27.0-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
Collecting traittypes>=0.2.0 (from ipydatawidgets>=1.1.1->pythreejs) Collecting traittypes>=0.2.0 (from ipydatawidgets>=1.1.1->pythreejs)
Using cached traittypes-0.2.1-py2.py3-none-any.whl.metadata (1.0 kB) Using cached traittypes-0.2.1-py2.py3-none-any.whl.metadata (1.0 kB)
Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (0.2.2) Requirement already satisfied: comm>=0.1.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (0.2.2)
Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (8.12.3) Requirement already satisfied: ipython>=6.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (8.12.3)
Requirement already satisfied: widgetsnbextension~=4.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (4.0.10) Requirement already satisfied: widgetsnbextension~=4.0.11 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (4.0.11)
Requirement already satisfied: jupyterlab-widgets~=3.0.10 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (3.0.10) Requirement already satisfied: jupyterlab-widgets~=3.0.11 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipywidgets>=7.2.1->pythreejs) (3.0.11)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.3.2) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.7) Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2.2.1) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2024.2.2) Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev>=2024.0.0) (2024.6.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch) (2.1.5) Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch) (2.1.5)
Collecting mpmath>=0.19 (from sympy->torch) Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch)
Using cached https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB) Using cached https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB)
Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.0) Requirement already satisfied: backcall in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.0)
Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (5.1.1) Requirement already satisfied: decorator in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (5.1.1)
Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.19.1) Requirement already satisfied: jedi>=0.16 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.19.1)
Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.1.7) Requirement already satisfied: matplotlib-inline in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.1.7)
Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.5) Requirement already satisfied: pickleshare in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.5)
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (3.0.43) Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (3.0.46)
Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.18.0) Requirement already satisfied: pygments>=2.4.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.18.0)
Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.6.3) Requirement already satisfied: stack-data in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.6.3)
Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (4.9.0) Requirement already satisfied: pexpect>4.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (4.9.0)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.8.4) Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.8.4)
Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.0) Requirement already satisfied: ptyprocess>=0.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.7.0)
Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.13) Requirement already satisfied: wcwidth in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.13)
Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.0.1) Requirement already satisfied: executing>=1.2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.4.1) Requirement already satisfied: asttokens>=2.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (2.4.1)
Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.2) Requirement already satisfied: pure-eval in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (0.2.2)
Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (1.16.0) Requirement already satisfied: six>=1.12.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from asttokens>=2.1.0->stack-data->ipython>=6.1.0->ipywidgets>=7.2.1->pythreejs) (1.16.0)
Using cached pythreejs-2.4.2-py3-none-any.whl (3.4 MB) Using cached pythreejs-2.4.2-py3-none-any.whl (3.4 MB)
Using cached openvino_dev-2024.1.0-15008-py3-none-any.whl (4.7 MB) Using cached openvino_dev-2024.1.0-15008-py3-none-any.whl (4.7 MB)
Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl (38.7 MB) Using cached openvino-2024.1.0-15008-cp38-cp38-manylinux2014_x86_64.whl (38.7 MB)
Using cached opencv_python-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (62.2 MB) Using cached opencv_python-4.10.0.82-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (62.5 MB)
Using cached onnx-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB) Using cached onnx-1.16.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)
Using cached ipydatawidgets-4.3.5-py2.py3-none-any.whl (271 kB) Using cached ipydatawidgets-4.3.5-py2.py3-none-any.whl (271 kB)
Using cached networkx-3.1-py3-none-any.whl (2.1 MB) Using cached networkx-3.1-py3-none-any.whl (2.1 MB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB) Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached openvino_telemetry-2024.1.0-py3-none-any.whl (23 kB) Using cached openvino_telemetry-2024.1.0-py3-none-any.whl (23 kB)
Using cached protobuf-5.26.1-cp37-abi3-manylinux2014_x86_64.whl (302 kB) Using cached protobuf-5.27.0-cp38-abi3-manylinux2014_x86_64.whl (309 kB)
Using cached filelock-3.14.0-py3-none-any.whl (12 kB) Using cached filelock-3.14.0-py3-none-any.whl (12 kB)
Downloading fsspec-2024.5.0-py3-none-any.whl (316 kB) Using cached fsspec-2024.6.0-py3-none-any.whl (176 kB)
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.1/316.1 kB 2.6 MB/s eta 0:00:00 Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB)
Using cached traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB) Using cached traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB)
Installing collected packages: openvino-telemetry, mpmath, traittypes, sympy, protobuf, numpy, networkx, fsspec, filelock, torch, openvino, opencv-python, onnx, openvino-dev, ipydatawidgets, pythreejs Installing collected packages: openvino-telemetry, mpmath, traittypes, sympy, protobuf, numpy, networkx, fsspec, filelock, torch, openvino, opencv-python, onnx, openvino-dev, ipydatawidgets, pythreejs
Successfully installed filelock-3.14.0 fsspec-2024.5.0 ipydatawidgets-4.3.5 mpmath-1.3.0 networkx-3.1 numpy-1.24.4 onnx-1.16.0 opencv-python-4.9.0.80 openvino-2024.1.0 openvino-dev-2024.1.0 openvino-telemetry-2024.1.0 protobuf-5.26.1 pythreejs-2.4.2 sympy-1.12 torch-2.3.0+cpu traittypes-0.2.1 Successfully installed filelock-3.14.0 fsspec-2024.6.0 ipydatawidgets-4.3.5 mpmath-1.3.0 networkx-3.1 numpy-1.24.4 onnx-1.16.1 opencv-python-4.10.0.82 openvino-2024.1.0 openvino-dev-2024.1.0 openvino-telemetry-2024.1.0 protobuf-5.27.0 pythreejs-2.4.2 sympy-1.12.1 torch-2.3.1+cpu traittypes-0.2.1
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -252,18 +252,18 @@ IR format.
.. parsed-literal:: .. parsed-literal::
========== Converting human-pose-estimation-3d-0001 to ONNX ========== Converting human-pose-estimation-3d-0001 to ONNX
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=model/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=model/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx
ONNX check passed successfully. ONNX check passed successfully.
========== Converting human-pose-estimation-3d-0001 to IR (FP32) ========== Converting human-pose-estimation-3d-0001 to IR (FP32)
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 256, 448]' --compress_to_fp16=False Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=model/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 256, 448]' --compress_to_fp16=False
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/3D-pose-estimation-webcam/model/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.bin

4
docs/notebooks/3D-segmentation-point-clouds-with-output.rst
View File

@ -216,7 +216,7 @@ chair for example.
.. parsed-literal:: .. parsed-literal::
/tmp/ipykernel_16799/2434168836.py:12: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored /tmp/ipykernel_3063563/2434168836.py:12: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
ax.scatter3D(X, Y, Z, s=5, cmap="jet", marker="o", label="chair") ax.scatter3D(X, Y, Z, s=5, cmap="jet", marker="o", label="chair")
@ -317,7 +317,7 @@ select device from dropdown list for running inference using OpenVINO
.. parsed-literal:: .. parsed-literal::
/tmp/ipykernel_16799/2804603389.py:23: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored /tmp/ipykernel_3063563/2804603389.py:23: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
ax.scatter(XCur, YCur, ZCur, s=5, cmap="jet", marker="o", label=classes[i]) ax.scatter(XCur, YCur, ZCur, s=5, cmap="jet", marker="o", label=classes[i])

4
docs/notebooks/action-recognition-webcam-with-output_files/action-recognition-webcam-with-output_22_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:1f9f7a4ff050de5ac1c035ff13546573f526abbc3d4b1e157edb3a278caba746 oid sha256:f9519b2a12072147ebf54e1d7a840ccde81b965fa1844f42f79e66c6513d844a
size 69060 size 68147

6
docs/notebooks/all_notebooks_paths.txt
View File

@ -15,6 +15,7 @@ notebooks/convert-to-openvino/convert-to-openvino.ipynb
notebooks/convert-to-openvino/legacy-mo-convert-to-openvino.ipynb notebooks/convert-to-openvino/legacy-mo-convert-to-openvino.ipynb
notebooks/cross-lingual-books-alignment/cross-lingual-books-alignment.ipynb notebooks/cross-lingual-books-alignment/cross-lingual-books-alignment.ipynb
notebooks/ct-segmentation-quantize/ct-segmentation-quantize-nncf.ipynb notebooks/ct-segmentation-quantize/ct-segmentation-quantize-nncf.ipynb
notebooks/ddcolor-image-colorization/ddcolor-image-colorization.ipynb
notebooks/decidiffusion-image-generation/decidiffusion-image-generation.ipynb notebooks/decidiffusion-image-generation/decidiffusion-image-generation.ipynb
notebooks/depth-anything/depth-anything.ipynb notebooks/depth-anything/depth-anything.ipynb
notebooks/detectron2-to-openvino/detectron2-to-openvino.ipynb notebooks/detectron2-to-openvino/detectron2-to-openvino.ipynb
@ -63,6 +64,7 @@ notebooks/model-server/model-server.ipynb
notebooks/model-tools/model-tools.ipynb notebooks/model-tools/model-tools.ipynb
notebooks/music-generation/music-generation.ipynb notebooks/music-generation/music-generation.ipynb
notebooks/named-entity-recognition/named-entity-recognition.ipynb notebooks/named-entity-recognition/named-entity-recognition.ipynb
notebooks/nano-llava-multimodal-chatbot/nano-llava-multimodal-chatbot.ipynb
notebooks/object-detection-webcam/object-detection.ipynb notebooks/object-detection-webcam/object-detection.ipynb
notebooks/oneformer-segmentation/oneformer-segmentation.ipynb notebooks/oneformer-segmentation/oneformer-segmentation.ipynb
notebooks/openvino-api/openvino-api.ipynb notebooks/openvino-api/openvino-api.ipynb
@ -73,6 +75,7 @@ notebooks/optimize-preprocessing/optimize-preprocessing.ipynb
notebooks/paddle-ocr-webcam/paddle-ocr-webcam.ipynb notebooks/paddle-ocr-webcam/paddle-ocr-webcam.ipynb
notebooks/paddle-to-openvino/paddle-to-openvino-classification.ipynb notebooks/paddle-to-openvino/paddle-to-openvino-classification.ipynb
notebooks/paint-by-example/paint-by-example.ipynb notebooks/paint-by-example/paint-by-example.ipynb
notebooks/person-counting-webcam/person-counting.ipynb
notebooks/person-tracking-webcam/person-tracking.ipynb notebooks/person-tracking-webcam/person-tracking.ipynb
notebooks/photo-maker/photo-maker.ipynb notebooks/photo-maker/photo-maker.ipynb
notebooks/pix2struct-docvqa/pix2struct-docvqa.ipynb notebooks/pix2struct-docvqa/pix2struct-docvqa.ipynb
@ -80,6 +83,7 @@ notebooks/pose-estimation-webcam/pose-estimation.ipynb
notebooks/pyannote-speaker-diarization/pyannote-speaker-diarization.ipynb notebooks/pyannote-speaker-diarization/pyannote-speaker-diarization.ipynb
notebooks/pytorch-post-training-quantization-nncf/pytorch-post-training-quantization-nncf.ipynb notebooks/pytorch-post-training-quantization-nncf/pytorch-post-training-quantization-nncf.ipynb
notebooks/pytorch-quantization-aware-training/pytorch-quantization-aware-training.ipynb notebooks/pytorch-quantization-aware-training/pytorch-quantization-aware-training.ipynb
notebooks/pytorch-quantization-sparsity-aware-training/pytorch-quantization-sparsity-aware-training.ipynb
notebooks/pytorch-to-openvino/pytorch-onnx-to-openvino.ipynb notebooks/pytorch-to-openvino/pytorch-onnx-to-openvino.ipynb
notebooks/pytorch-to-openvino/pytorch-to-openvino.ipynb notebooks/pytorch-to-openvino/pytorch-to-openvino.ipynb
notebooks/qrcode-monster/qrcode-monster.ipynb notebooks/qrcode-monster/qrcode-monster.ipynb
@ -87,6 +91,7 @@ notebooks/quantizing-model-with-accuracy-control/speech-recognition-quantization
notebooks/quantizing-model-with-accuracy-control/yolov8-quantization-with-accuracy-control.ipynb notebooks/quantizing-model-with-accuracy-control/yolov8-quantization-with-accuracy-control.ipynb
notebooks/riffusion-text-to-music/riffusion-text-to-music.ipynb notebooks/riffusion-text-to-music/riffusion-text-to-music.ipynb
notebooks/rmbg-background-removal/rmbg-background-removal.ipynb notebooks/rmbg-background-removal/rmbg-background-removal.ipynb
notebooks/s3d-mil-nce-text-to-video-retrieval/s3d-mil-nce-text-to-video-retrieval.ipynb
notebooks/sdxl-turbo/sdxl-turbo.ipynb notebooks/sdxl-turbo/sdxl-turbo.ipynb
notebooks/segment-anything/segment-anything.ipynb notebooks/segment-anything/segment-anything.ipynb
notebooks/siglip-zero-shot-image-classification/siglip-zero-shot-image-classification.ipynb notebooks/siglip-zero-shot-image-classification/siglip-zero-shot-image-classification.ipynb
@ -135,6 +140,7 @@ notebooks/vision-paddlegan-superresolution/vision-paddlegan-superresolution.ipyn
notebooks/whisper-subtitles-generation/whisper-convert.ipynb notebooks/whisper-subtitles-generation/whisper-convert.ipynb
notebooks/whisper-subtitles-generation/whisper-nncf-quantize.ipynb notebooks/whisper-subtitles-generation/whisper-nncf-quantize.ipynb
notebooks/wuerstchen-image-generation/wuerstchen-image-generation.ipynb notebooks/wuerstchen-image-generation/wuerstchen-image-generation.ipynb
notebooks/yolov10-optimization/yolov10-optimization.ipynb
notebooks/yolov7-optimization/yolov7-optimization.ipynb notebooks/yolov7-optimization/yolov7-optimization.ipynb
notebooks/yolov8-optimization/yolov8-instance-segmentation.ipynb notebooks/yolov8-optimization/yolov8-instance-segmentation.ipynb
notebooks/yolov8-optimization/yolov8-keypoint-detection.ipynb notebooks/yolov8-optimization/yolov8-keypoint-detection.ipynb

74
docs/notebooks/amused-lightweight-text-to-image-with-output.rst
View File

@ -78,8 +78,8 @@ Load and run the original pipeline
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/transformers/transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
warnings.warn( deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
@ -202,29 +202,29 @@ Convert the Text Encoder
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if input_shape[-1] > 1 or self.sliding_window is not None: if input_shape[-1] > 1 or self.sliding_window is not None:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if past_key_values_length > 0: if past_key_values_length > 0:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:620: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:622: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
encoder_states = () if output_hidden_states else None encoder_states = () if output_hidden_states else None
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:625: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:627: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if output_hidden_states: if output_hidden_states:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:276: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len): if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:284: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len): if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:316: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim): if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:648: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:650: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if output_hidden_states: if output_hidden_states:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:651: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:653: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not return_dict: if not return_dict:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:742: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:745: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not return_dict: if not return_dict:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:1227: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:1230: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not return_dict: if not return_dict:
@ -333,13 +333,13 @@ suitable. This function repeats part of ``AmusedPipeline``.
.. parsed-literal:: .. parsed-literal::
/tmp/ipykernel_17572/3779428577.py:34: TracerWarning: Converting a tensor to a Python list might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /tmp/ipykernel_3064357/3779428577.py:34: TracerWarning: Converting a tensor to a Python list might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
shape=shape.tolist(), shape=shape.tolist(),
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/vq_model.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/vq_model.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not force_not_quantize: if not force_not_quantize:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert hidden_states.shape[1] == self.channels assert hidden_states.shape[1] == self.channels
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if hidden_states.shape[0] >= 64: if hidden_states.shape[0] >= 64:
@ -477,7 +477,7 @@ And insert wrappers instances in the pipeline:
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:140: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False) deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
@ -541,12 +541,8 @@ improve model inference speed.
QUANTIZED_TRANSFORMER_OV_PATH = Path(str(TRANSFORMER_OV_PATH).replace(".xml", "_quantized.xml")) QUANTIZED_TRANSFORMER_OV_PATH = Path(str(TRANSFORMER_OV_PATH).replace(".xml", "_quantized.xml"))
to_quantize = widgets.Checkbox( skip_for_device = "GPU" in device.value
value=True, to_quantize = widgets.Checkbox(value=not skip_for_device, description="Quantization", disabled=skip_for_device)
description="Quantization",
disabled=False,
)
to_quantize to_quantize
@ -692,7 +688,7 @@ model.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1486: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1491: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
You can avoid this message in future by passing the argument `trust_remote_code=True`. You can avoid this message in future by passing the argument `trust_remote_code=True`.
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`. Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
warnings.warn( warnings.warn(
@ -706,7 +702,7 @@ model.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:140: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False) deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
@ -784,17 +780,17 @@ model.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/experimental/tensor/tensor.py:84: RuntimeWarning: invalid value encountered in multiply
return Tensor(self.data * unwrap_tensor_data(other)) return Tensor(self.data * unwrap_tensor_data(other))
@ -826,7 +822,7 @@ Demo generation with quantized pipeline
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:140: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False) deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
@ -910,7 +906,11 @@ a rough estimate of generation quality.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Metric `InceptionScore` will save all extracted features in buffer. For large datasets this may lead to large memory footprint. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/datasets/load.py:1491: FutureWarning: The repository for conceptual_captions contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/conceptual_captions
You can avoid this message in future by passing the argument `trust_remote_code=True`.
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:43: UserWarning: Metric `InceptionScore` will save all extracted features in buffer. For large datasets this may lead to large memory footprint.
warnings.warn(\*args, \*\*kwargs) # noqa: B028 warnings.warn(\*args, \*\*kwargs) # noqa: B028
@ -922,9 +922,9 @@ a rough estimate of generation quality.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:139: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/configuration_utils.py:140: FutureWarning: Accessing config attribute `_execution_device` directly via 'AmusedPipeline' object attribute is deprecated. Please access '_execution_device' over 'AmusedPipeline's config object instead, e.g. 'scheduler.config._execution_device'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False) deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/image/inception.py:176: UserWarning: std(): degrees of freedom is <= 0. Correction should be strictly less than the reduction factor (input numel divided by output numel). (Triggered internally at ../aten/src/ATen/native/ReduceOps.cpp:1807.) /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torchmetrics/image/inception.py:176: UserWarning: std(): degrees of freedom is <= 0. Correction should be strictly less than the reduction factor (input numel divided by output numel). (Triggered internally at ../aten/src/ATen/native/ReduceOps.cpp:1807.)
return kl.mean(), kl.std() return kl.mean(), kl.std()

27
docs/notebooks/animate-anyone-with-output.rst
View File

File diff suppressed because one or more lines are too long

6
docs/notebooks/async-api-with-output.rst
View File

@ -352,7 +352,7 @@ Test performance in Sync Mode
.. parsed-literal:: .. parsed-literal::
Source ended Source ended
average throuput in sync mode: 43.35 fps average throuput in sync mode: 60.54 fps
Async Mode Async Mode
@ -491,7 +491,7 @@ Test the performance in Async Mode
.. parsed-literal:: .. parsed-literal::
Source ended Source ended
average throuput in async mode: 73.97 fps average throuput in async mode: 103.70 fps
Compare the performance Compare the performance
@ -634,5 +634,5 @@ Test the performance with ``AsyncInferQueue``
.. parsed-literal:: .. parsed-literal::
average throughput in async mode with async infer queue: 111.33 fps average throughput in async mode with async infer queue: 148.11 fps

4
docs/notebooks/async-api-with-output_files/async-api-with-output_23_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:c6eb6b07a2e43cfab480087829f6babef1e7050550997c85a7a6824f8c308cc3 oid sha256:0d82d618fb2b2ef25ecd8ad941de1d1173b3e21a3340314cc584dca9b32d6c55
size 30403 size 29416

50
docs/notebooks/auto-device-with-output.rst
View File

@ -96,7 +96,7 @@ Import modules and create Core
core = ov.Core() core = ov.Core()
if "GPU" not in core.available_devices: if not any("GPU" in device for device in core.available_devices):
display( display(
Markdown( Markdown(
'<div class="alert alert-block alert-danger"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>' '<div class="alert alert-block alert-danger"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>'
@ -186,15 +186,15 @@ By default, ``compile_model`` API will select **AUTO** as
.. parsed-literal:: .. parsed-literal::
[23:28:01.3129]I[plugin.cpp:418][AUTO] device:CPU, config:LOG_LEVEL=LOG_INFO [23:27:27.6972]I[plugin.cpp:418][AUTO] device:CPU, config:LOG_LEVEL=LOG_INFO
[23:28:01.3129]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY [23:27:27.6973]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT=LATENCY
[23:28:01.3130]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0 [23:27:27.6973]I[plugin.cpp:418][AUTO] device:CPU, config:PERFORMANCE_HINT_NUM_REQUESTS=0
[23:28:01.3130]I[plugin.cpp:418][AUTO] device:CPU, config:PERF_COUNT=NO [23:27:27.6973]I[plugin.cpp:418][AUTO] device:CPU, config:PERF_COUNT=NO
[23:28:01.3130]I[plugin.cpp:423][AUTO] device:CPU, priority:0 [23:27:27.6973]I[plugin.cpp:423][AUTO] device:CPU, priority:0
[23:28:01.3130]I[schedule.cpp:17][AUTO] scheduler starting [23:27:27.6973]I[schedule.cpp:17][AUTO] scheduler starting
[23:28:01.3130]I[auto_schedule.cpp:131][AUTO] select device:CPU [23:27:27.6973]I[auto_schedule.cpp:131][AUTO] select device:CPU
[23:28:01.4657]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished [23:27:27.8462]I[auto_schedule.cpp:109][AUTO] device:CPU compiling model finished
[23:28:01.4659]I[plugin.cpp:451][AUTO] underlying hardware does not support hardware context [23:27:27.8464]I[plugin.cpp:451][AUTO] underlying hardware does not support hardware context
Successfully compiled model without a device_name. Successfully compiled model without a device_name.
@ -208,7 +208,7 @@ By default, ``compile_model`` API will select **AUTO** as
.. parsed-literal:: .. parsed-literal::
Deleted compiled_model Deleted compiled_model
[23:28:01.4767]I[schedule.cpp:303][AUTO] scheduler ending [23:27:27.8575]I[schedule.cpp:303][AUTO] scheduler ending
Explicitly pass AUTO as device_name to Core::compile_model API Explicitly pass AUTO as device_name to Core::compile_model API
@ -318,7 +318,7 @@ Load the model to GPU device and perform inference
.. code:: ipython3 .. code:: ipython3
if "GPU" not in core.available_devices: if not any("GPU" in device for device in core.available_devices):
print(f"A GPU device is not available. Available devices are: {core.available_devices}") print(f"A GPU device is not available. Available devices are: {core.available_devices}")
else: else:
# Start time. # Start time.
@ -366,7 +366,7 @@ executed on CPU until GPU is ready.
.. parsed-literal:: .. parsed-literal::
Time to load model using AUTO device and get first inference: 0.18 seconds. Time to load model using AUTO device and get first inference: 0.16 seconds.
.. code:: ipython3 .. code:: ipython3
@ -538,12 +538,12 @@ Loop for inference and update the FPS/Latency every
Compiling Model for AUTO device with THROUGHPUT hint Compiling Model for AUTO device with THROUGHPUT hint
Start inference, 6 groups of FPS/latency will be measured over 10s intervals Start inference, 6 groups of FPS/latency will be measured over 10s intervals
throughput: 177.51fps, latency: 32.04ms, time interval: 10.01s throughput: 177.73fps, latency: 32.02ms, time interval: 10.01s
throughput: 179.73fps, latency: 32.54ms, time interval: 10.01s throughput: 179.52fps, latency: 32.63ms, time interval: 10.00s
throughput: 178.74fps, latency: 32.73ms, time interval: 10.01s throughput: 178.56fps, latency: 32.79ms, time interval: 10.00s
throughput: 179.46fps, latency: 32.59ms, time interval: 10.01s throughput: 177.70fps, latency: 32.99ms, time interval: 10.01s
throughput: 178.98fps, latency: 32.74ms, time interval: 10.02s throughput: 178.80fps, latency: 32.69ms, time interval: 10.02s
throughput: 178.58fps, latency: 32.79ms, time interval: 10.01s throughput: 177.72fps, latency: 33.00ms, time interval: 10.01s
Done Done
@ -589,12 +589,12 @@ Loop for inference and update the FPS/Latency for each
Compiling Model for AUTO Device with LATENCY hint Compiling Model for AUTO Device with LATENCY hint
Start inference, 6 groups fps/latency will be out with 10s interval Start inference, 6 groups fps/latency will be out with 10s interval
throughput: 136.40fps, latency: 6.83ms, time interval: 10.01s throughput: 135.52fps, latency: 6.87ms, time interval: 10.01s
throughput: 137.96fps, latency: 6.81ms, time interval: 10.00s throughput: 137.89fps, latency: 6.85ms, time interval: 10.00s
throughput: 137.97fps, latency: 6.80ms, time interval: 10.00s throughput: 137.71fps, latency: 6.82ms, time interval: 10.01s
throughput: 137.97fps, latency: 6.80ms, time interval: 10.00s throughput: 137.83fps, latency: 6.83ms, time interval: 10.01s
throughput: 138.06fps, latency: 6.80ms, time interval: 10.00s throughput: 137.80fps, latency: 6.83ms, time interval: 10.01s
throughput: 133.29fps, latency: 7.06ms, time interval: 10.01s throughput: 138.34fps, latency: 6.84ms, time interval: 10.00s
Done Done

4
docs/notebooks/auto-device-with-output_files/auto-device-with-output_27_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:d644b71f335dd26763dfad14f93ba1ff32ffe30cfdbe3ac06d1c7346aaba3985 oid sha256:50f5fe192a152bfb8036a782ee93ab26543c9a30fc626e67aab5e5a7dba9e35a
size 27550 size 27240

4
docs/notebooks/auto-device-with-output_files/auto-device-with-output_28_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:b3e6b932de9fd81a384cccede0ed571c920f68e6a9176ce2d84cee626bb03f05 oid sha256:5a4bfe1e79236b08d4848e6ff59eccf1e2c459d74cb8940b54e8cad3ef01a948
size 40115 size 40033

104
docs/notebooks/clip-zero-shot-classification-with-output.rst
View File

@ -102,9 +102,9 @@ tokenizer and preparing the images.
.. code:: ipython3 .. code:: ipython3
import platform import platform
%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu "gradio>=4.19" "openvino>=2023.1.0" "transformers[torch]>=4.30" "datasets" "nncf>=2.6.0" "torch>=2.1" Pillow %pip install -q --extra-index-url https://download.pytorch.org/whl/cpu "gradio>=4.19" "openvino>=2023.1.0" "transformers[torch]>=4.30" "datasets" "nncf>=2.6.0" "torch>=2.1" Pillow
if platform.system() != "Windows": if platform.system() != "Windows":
%pip install -q "matplotlib>=3.4" %pip install -q "matplotlib>=3.4"
else: else:
@ -119,7 +119,7 @@ tokenizer and preparing the images.
.. code:: ipython3 .. code:: ipython3
from transformers import CLIPProcessor, CLIPModel from transformers import CLIPProcessor, CLIPModel
# load pre-trained model # load pre-trained model
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch16") model = CLIPModel.from_pretrained("openai/clip-vit-base-patch16")
# load preprocessor for model input # load preprocessor for model input
@ -141,8 +141,8 @@ tokenizer and preparing the images.
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
import numpy as np import numpy as np
from PIL import Image from PIL import Image
def visualize_result(image: Image, labels: List[str], probs: np.ndarray, top: int = 5): def visualize_result(image: Image, labels: List[str], probs: np.ndarray, top: int = 5):
""" """
Utility function for visualization classification results Utility function for visualization classification results
@ -160,7 +160,7 @@ tokenizer and preparing the images.
plt.subplot(8, 8, 1) plt.subplot(8, 8, 1)
plt.imshow(image) plt.imshow(image)
plt.axis("off") plt.axis("off")
plt.subplot(8, 8, 2) plt.subplot(8, 8, 2)
y = np.arange(top_probs.shape[-1]) y = np.arange(top_probs.shape[-1])
plt.grid() plt.grid()
@ -189,17 +189,17 @@ similarity score for the final result.
import requests import requests
from pathlib import Path from pathlib import Path
sample_path = Path("data/coco.jpg") sample_path = Path("data/coco.jpg")
sample_path.parent.mkdir(parents=True, exist_ok=True) sample_path.parent.mkdir(parents=True, exist_ok=True)
r = requests.get("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco.jpg") r = requests.get("https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/image/coco.jpg")
with sample_path.open("wb") as f: with sample_path.open("wb") as f:
f.write(r.content) f.write(r.content)
image = Image.open(sample_path) image = Image.open(sample_path)
input_labels = [ input_labels = [
"cat", "cat",
"dog", "dog",
@ -213,9 +213,9 @@ similarity score for the final result.
"computer", "computer",
] ]
text_descriptions = [f"This is a photo of a {label}" for label in input_labels] text_descriptions = [f"This is a photo of a {label}" for label in input_labels]
inputs = processor(text=text_descriptions, images=[image], return_tensors="pt", padding=True) inputs = processor(text=text_descriptions, images=[image], return_tensors="pt", padding=True)
results = model(**inputs) results = model(**inputs)
logits_per_image = results["logits_per_image"] # this is the image-text similarity score logits_per_image = results["logits_per_image"] # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1).detach().numpy() # we can take the softmax to get the label probabilities probs = logits_per_image.softmax(dim=1).detach().numpy() # we can take the softmax to get the label probabilities
@ -243,10 +243,10 @@ save it on disk for the next usage with ``ov.save_model``.
.. code:: ipython3 .. code:: ipython3
import openvino as ov import openvino as ov
fp16_model_path = Path("clip-vit-base-patch16.xml") fp16_model_path = Path("clip-vit-base-patch16.xml")
model.config.torchscript = True model.config.torchscript = True
if not fp16_model_path.exists(): if not fp16_model_path.exists():
ov_model = ov.convert_model(model, example_input=dict(inputs)) ov_model = ov.convert_model(model, example_input=dict(inputs))
ov.save_model(ov_model, fp16_model_path) ov.save_model(ov_model, fp16_model_path)
@ -263,7 +263,7 @@ same input data from the example above with PyTorch.
.. code:: ipython3 .. code:: ipython3
from scipy.special import softmax from scipy.special import softmax
# create OpenVINO core object instance # create OpenVINO core object instance
core = ov.Core() core = ov.Core()
@ -277,14 +277,14 @@ select device from dropdown list for running inference using OpenVINO
.. code:: ipython3 .. code:: ipython3
import ipywidgets as widgets import ipywidgets as widgets
device = widgets.Dropdown( device = widgets.Dropdown(
options=core.available_devices + ["AUTO"], options=core.available_devices + ["AUTO"],
value="AUTO", value="AUTO",
description="Device:", description="Device:",
disabled=False, disabled=False,
) )
device device
@ -317,8 +317,6 @@ Great! Looks like we got the same result.
Quantize model to INT8 using NNCF Quantize model to INT8 using NNCF
--------------------------------- ---------------------------------
## Quantize model to INT8 using
NNCF
The goal of this part of tutorial is to demonstrate how to speed up the The goal of this part of tutorial is to demonstrate how to speed up the
model by applying 8-bit post-training quantization from model by applying 8-bit post-training quantization from
@ -349,7 +347,7 @@ inference faster. The optimization process contains the following steps:
description="Quantization", description="Quantization",
disabled=False, disabled=False,
) )
to_quantize to_quantize
@ -368,7 +366,7 @@ inference faster. The optimization process contains the following steps:
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py", url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py",
) )
open("skip_kernel_extension.py", "w").write(r.text) open("skip_kernel_extension.py", "w").write(r.text)
%load_ext skip_kernel_extension %load_ext skip_kernel_extension
Prepare datasets Prepare datasets
@ -384,16 +382,16 @@ model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import requests import requests
from io import BytesIO from io import BytesIO
import numpy as np import numpy as np
from PIL import Image from PIL import Image
from requests.packages.urllib3.exceptions import InsecureRequestWarning from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning) requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
max_length = model.config.text_config.max_position_embeddings max_length = model.config.text_config.max_position_embeddings
def check_text_data(data): def check_text_data(data):
""" """
Check if the given data is text-based. Check if the given data is text-based.
@ -403,7 +401,7 @@ model.
if isinstance(data, list): if isinstance(data, list):
return all(isinstance(x, str) for x in data) return all(isinstance(x, str) for x in data)
return False return False
def get_pil_from_url(url): def get_pil_from_url(url):
""" """
Downloads and converts an image from a URL to a PIL Image object. Downloads and converts an image from a URL to a PIL Image object.
@ -411,7 +409,7 @@ model.
response = requests.get(url, verify=False, timeout=20) response = requests.get(url, verify=False, timeout=20)
image = Image.open(BytesIO(response.content)) image = Image.open(BytesIO(response.content))
return image.convert("RGB") return image.convert("RGB")
def collate_fn(example, image_column="image_url", text_column="caption"): def collate_fn(example, image_column="image_url", text_column="caption"):
""" """
Preprocesses an example by loading and transforming image and text data. Preprocesses an example by loading and transforming image and text data.
@ -422,10 +420,10 @@ model.
""" """
assert len(example) == 1 assert len(example) == 1
example = example[0] example = example[0]
if not check_text_data(example[text_column]): if not check_text_data(example[text_column]):
raise ValueError("Text data is not valid") raise ValueError("Text data is not valid")
url = example[image_column] url = example[image_column]
try: try:
image = get_pil_from_url(url) image = get_pil_from_url(url)
@ -434,7 +432,7 @@ model.
return None return None
except Exception: except Exception:
return None return None
inputs = processor(text=example[text_column], images=[image], return_tensors="pt", padding=True) inputs = processor(text=example[text_column], images=[image], return_tensors="pt", padding=True)
if inputs['input_ids'].shape[1] > max_length: if inputs['input_ids'].shape[1] > max_length:
return None return None
@ -443,11 +441,11 @@ model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import torch import torch
from datasets import load_dataset from datasets import load_dataset
from tqdm.notebook import tqdm from tqdm.notebook import tqdm
def prepare_calibration_data(dataloader, init_steps): def prepare_calibration_data(dataloader, init_steps):
""" """
This function prepares calibration data from a dataloader for a specified number of initialization steps. This function prepares calibration data from a dataloader for a specified number of initialization steps.
@ -470,8 +468,8 @@ model.
} }
) )
return data return data
def prepare_dataset(opt_init_steps=50, max_train_samples=1000): def prepare_dataset(opt_init_steps=50, max_train_samples=1000):
""" """
Prepares a vision-text dataset for quantization. Prepares a vision-text dataset for quantization.
@ -485,14 +483,14 @@ model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import logging import logging
import nncf import nncf
core = ov.Core() core = ov.Core()
nncf.set_log_level(logging.ERROR) nncf.set_log_level(logging.ERROR)
int8_model_path = 'clip-vit-base-patch16_int8.xml' int8_model_path = 'clip-vit-base-patch16_int8.xml'
calibration_data = prepare_dataset() calibration_data = prepare_dataset()
ov_model = core.read_model(fp16_model_path) ov_model = core.read_model(fp16_model_path)
@ -535,12 +533,12 @@ Create a quantized model from the pre-trained ``FP16`` model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
if len(calibration_data) == 0: if len(calibration_data) == 0:
raise RuntimeError( raise RuntimeError(
'Calibration dataset is empty. Please check internet connection and try to download images manually.' 'Calibration dataset is empty. Please check internet connection and try to download images manually.'
) )
calibration_dataset = nncf.Dataset(calibration_data) calibration_dataset = nncf.Dataset(calibration_data)
quantized_model = nncf.quantize( quantized_model = nncf.quantize(
model=ov_model, model=ov_model,
@ -654,7 +652,7 @@ the same input data that we used before.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
# compile model for loading on device # compile model for loading on device
compiled_model = core.compile_model(quantized_model, device.value) compiled_model = core.compile_model(quantized_model, device.value)
# run inference on preprocessed data and get image-text similarity score # run inference on preprocessed data and get image-text similarity score
@ -679,9 +677,9 @@ Compare File Size
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
from pathlib import Path from pathlib import Path
fp16_ir_model_size = Path(fp16_model_path).with_suffix(".bin").stat().st_size / 1024 / 1024 fp16_ir_model_size = Path(fp16_model_path).with_suffix(".bin").stat().st_size / 1024 / 1024
quantized_model_size = Path(int8_model_path).with_suffix(".bin").stat().st_size / 1024 / 1024 quantized_model_size = Path(int8_model_path).with_suffix(".bin").stat().st_size / 1024 / 1024
print(f"FP16 IR model size: {fp16_ir_model_size:.2f} MB") print(f"FP16 IR model size: {fp16_ir_model_size:.2f} MB")
@ -711,9 +709,9 @@ up of the dynamic quantized models.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import time import time
def calculate_inference_time(model_path, calibration_data): def calculate_inference_time(model_path, calibration_data):
model = core.compile_model(model_path, device.value) model = core.compile_model(model_path, device.value)
inference_time = [] inference_time = []
@ -728,7 +726,7 @@ up of the dynamic quantized models.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
fp16_latency = calculate_inference_time(fp16_model_path, calibration_data) fp16_latency = calculate_inference_time(fp16_model_path, calibration_data)
int8_latency = calculate_inference_time(int8_model_path, calibration_data) int8_latency = calculate_inference_time(int8_model_path, calibration_data)
print(f"Performance speed up: {fp16_latency / int8_latency:.3f}") print(f"Performance speed up: {fp16_latency / int8_latency:.3f}")
@ -742,8 +740,6 @@ up of the dynamic quantized models.
Interactive demo Interactive demo
---------------- ----------------
## Interactive demo
Now, it is your turn! You can provide your own image and comma-separated Now, it is your turn! You can provide your own image and comma-separated
list of labels for zero-shot classification. list of labels for zero-shot classification.
@ -754,13 +750,13 @@ example, ``cat,dog,bird``)
.. code:: ipython3 .. code:: ipython3
import gradio as gr import gradio as gr
model_path = Path("clip-vit-base-patch16-int8.xml") model_path = Path("clip-vit-base-patch16-int8.xml")
if not model_path.exists(): if not model_path.exists():
model_path = Path("clip-vit-base-patch16.xml") model_path = Path("clip-vit-base-patch16.xml")
compiled_model = core.compile_model(model_path, device.value) compiled_model = core.compile_model(model_path, device.value)
def classify(image, text): def classify(image, text):
"""Classify image using classes listing. """Classify image using classes listing.
Args: Args:
@ -774,10 +770,10 @@ example, ``cat,dog,bird``)
inputs = processor(text=text_descriptions, images=[image], return_tensors="np", padding=True) inputs = processor(text=text_descriptions, images=[image], return_tensors="np", padding=True)
ov_logits_per_image = compiled_model(dict(inputs))[0] ov_logits_per_image = compiled_model(dict(inputs))[0]
probs = softmax(ov_logits_per_image, axis=1)[0] probs = softmax(ov_logits_per_image, axis=1)[0]
return {label: float(prob) for label, prob in zip(labels, probs)} return {label: float(prob) for label, prob in zip(labels, probs)}
demo = gr.Interface( demo = gr.Interface(
classify, classify,
[ [

24
docs/notebooks/convert-to-openvino-with-output.rst
View File

@ -35,7 +35,7 @@ Table of contents:
.. parsed-literal:: .. parsed-literal::
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0) Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -181,13 +181,11 @@ NLP model from Hugging Face and export it in ONNX format:
.. parsed-literal:: .. parsed-literal::
2024-05-15 23:49:16.064636: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-05 23:48:38.001731: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-15 23:49:16.099980: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-05 23:48:38.036985: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 23:49:16.617080: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-05 23:48:38.551703: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:231: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
mask, torch.tensor(torch.finfo(scores.dtype).min) mask, torch.tensor(torch.finfo(scores.dtype).min)
@ -664,12 +662,12 @@ frameworks conversion guides.
.. parsed-literal:: .. parsed-literal::
2024-05-15 23:49:36.583572: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW 2024-06-05 23:48:58.596260: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
2024-05-15 23:49:36.583606: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07 2024-06-05 23:48:58.596295: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:168] retrieving CUDA diagnostic information for host: iotg-dev-workstation-07
2024-05-15 23:49:36.583610: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07 2024-06-05 23:48:58.596299: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:175] hostname: iotg-dev-workstation-07
2024-05-15 23:49:36.583842: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2 2024-06-05 23:48:58.596508: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:199] libcuda reported version is: 470.223.2
2024-05-15 23:49:36.583866: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3 2024-06-05 23:48:58.596524: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:203] kernel reported version is: 470.182.3
2024-05-15 23:49:36.583871: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration 2024-06-05 23:48:58.596528: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
Migration from Legacy conversion API Migration from Legacy conversion API

2
docs/notebooks/convnext-classification-with-output.rst
View File

@ -183,7 +183,7 @@ And print results
Predicted Class: 281 Predicted Class: 281
Predicted Label: n02123045 tabby, tabby cat Predicted Label: n02123045 tabby, tabby cat
Predicted Probability: 0.4661690592765808 Predicted Probability: 0.5808374285697937
Convert the model to OpenVINO Intermediate representation format Convert the model to OpenVINO Intermediate representation format

8
docs/notebooks/cross-lingual-books-alignment-with-output.rst
View File

@ -197,7 +197,7 @@ which in a raw format looks like this:
.. parsed-literal:: .. parsed-literal::
'\ufeffThe Project Gutenberg eBook of Anna Karenina\r\n \r\nThis ebook is for the use of anyone anywhere in the United States and\r\nmost other parts of the world at no cost and with almost no restrictions\r\nwhatsoever. You may copy it, give it away or re-use it under the terms\r\nof the Project Gutenberg License included with this ebook or online\r\nat www.gutenberg.org. If you are not located in the United States,\r\nyou will have to check the laws of the country where you are located\r\nbefore using this eBook.\r\n\r\nTitle: Anna Karenina\r\n\r\n\r\nAuthor: graf Leo Tolstoy\r\n\r\nTranslator: Constance Garnett\r\n\r\nRelease date: July 1, 1998 [eBook #1399]\r\n Most recently updated: April 9, 2023\r\n\r\nLanguage: English\r\n\r\n\r\n\r\n*** START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA \*\*\*\r\n[Illustration]\r\n\r\n\r\n\r\n\r\n ANNA KARENINA \r\n\r\n by Leo Tolstoy \r\n\r\n Translated by Constance Garnett \r\n\r\nContents\r\n\r\n\r\n PART ONE\r\n PART TWO\r\n PART THREE\r\n PART FOUR\r\n PART FIVE\r\n PART SIX\r\n PART SEVEN\r\n PART EIGHT\r\n\r\n\r\n\r\n\r\nPART ONE\r\n\r\nChapter 1\r\n\r\n\r\nHappy families are all alike; every unhappy family is unhappy in its\r\nown way.\r\n\r\nEverything was in confusion in the Oblonskys’ house. The wife had\r\ndiscovered that the husband was carrying on an intrigue with a French\r\ngirl, who had been a governess in their family, and she had announced\r\nto her husband that she could not go on living in the same house with\r\nhim. This position of affairs had now lasted three days, and not only\r\nthe husband and wife themselves, but all the me' '\ufeffThe Project Gutenberg eBook of Anna Karenina\r\n \r\nThis ebook is for the use of anyone anywhere in the United States and\r\nmost other parts of the world at no cost and with almost no restrictions\r\nwhatsoever. You may copy it, give it away or re-use it under the terms\r\nof the Project Gutenberg License included with this ebook or online\r\nat www.gutenberg.org. If you are not located in the United States,\r\nyou will have to check the laws of the country where you are located\r\nbefore using this eBook.\r\n\r\nTitle: Anna Karenina\r\n\r\n\r\nAuthor: graf Leo Tolstoy\r\n\r\nTranslator: Constance Garnett\r\n\r\nRelease date: July 1, 1998 [eBook #1399]\r\n Most recently updated: April 9, 2023\r\n\r\nLanguage: English\r\n\r\n\r\n\r\n\*\*\* START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA \*\*\*\r\n[Illustration]\r\n\r\n\r\n\r\n\r\n ANNA KARENINA \r\n\r\n by Leo Tolstoy \r\n\r\n Translated by Constance Garnett \r\n\r\nContents\r\n\r\n\r\n PART ONE\r\n PART TWO\r\n PART THREE\r\n PART FOUR\r\n PART FIVE\r\n PART SIX\r\n PART SEVEN\r\n PART EIGHT\r\n\r\n\r\n\r\n\r\nPART ONE\r\n\r\nChapter 1\r\n\r\n\r\nHappy families are all alike; every unhappy family is unhappy in its\r\nown way.\r\n\r\nEverything was in confusion in the Oblonskys’ house. The wife had\r\ndiscovered that the husband was carrying on an intrigue with a French\r\ngirl, who had been a governess in their family, and she had announced\r\nto her husband that she could not go on living in the same house with\r\nhim. This position of affairs had now lasted three days, and not only\r\nthe husband and wife themselves, but all the me'
@ -210,7 +210,7 @@ which in a raw format looks like this:
.. parsed-literal:: .. parsed-literal::
'The Project Gutenberg EBook of Anna Karenina, 1. Band, by Leo N. Tolstoi\r\n\r\nThis eBook is for the use of anyone anywhere at no cost and with\r\nalmost no restrictions whatsoever. You may copy it, give it away or\r\nre-use it under the terms of the Project Gutenberg License included\r\nwith this eBook or online at www.gutenberg.org\r\n\r\n\r\nTitle: Anna Karenina, 1. Band\r\n\r\nAuthor: Leo N. Tolstoi\r\n\r\nRelease Date: February 18, 2014 [EBook #44956]\r\n\r\nLanguage: German\r\n\r\nCharacter set encoding: ISO-8859-1\r\n\r\n\*\*\* START OF THIS PROJECT GUTENBERG EBOOK ANNA KARENINA, 1. BAND \*\*\*\r\n\r\n\r\n\r\n\r\nProduced by Norbert H. Langkau, Jens Nordmann and the\r\nOnline Distributed Proofreading Team at http://www.pgdp.net\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n Anna Karenina.\r\n\r\n\r\n Roman aus dem Russischen\r\n\r\n des\r\n\r\n Grafen Leo N. Tolstoi.\r\n\r\n\r\n\r\n Nach der siebenten Auflage übersetzt\r\n\r\n von\r\n\r\n Hans Moser.\r\n\r\n\r\n Erster Band.\r\n\r\n\r\n\r\n Leipzig\r\n\r\n Druck und Verlag von Philipp Reclam jun.\r\n\r\n \* \* \* \* *\r\n\r\n\r\n\r\n\r\n Erster Teil.\r\n\r\n »Die Rache ist mein, ich will vergelten.«\r\n\r\n 1.\r\n\r\n\r\nAlle glücklichen Familien sind einander ähnlich; jede unglücklich' 'The Project Gutenberg EBook of Anna Karenina, 1. Band, by Leo N. Tolstoi\r\n\r\nThis eBook is for the use of anyone anywhere at no cost and with\r\nalmost no restrictions whatsoever. You may copy it, give it away or\r\nre-use it under the terms of the Project Gutenberg License included\r\nwith this eBook or online at www.gutenberg.org\r\n\r\n\r\nTitle: Anna Karenina, 1. Band\r\n\r\nAuthor: Leo N. Tolstoi\r\n\r\nRelease Date: February 18, 2014 [EBook #44956]\r\n\r\nLanguage: German\r\n\r\nCharacter set encoding: ISO-8859-1\r\n\r\n\*\*\* START OF THIS PROJECT GUTENBERG EBOOK ANNA KARENINA, 1. BAND \*\*\*\r\n\r\n\r\n\r\n\r\nProduced by Norbert H. Langkau, Jens Nordmann and the\r\nOnline Distributed Proofreading Team at http://www.pgdp.net\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n Anna Karenina.\r\n\r\n\r\n Roman aus dem Russischen\r\n\r\n des\r\n\r\n Grafen Leo N. Tolstoi.\r\n\r\n\r\n\r\n Nach der siebenten Auflage übersetzt\r\n\r\n von\r\n\r\n Hans Moser.\r\n\r\n\r\n Erster Band.\r\n\r\n\r\n\r\n Leipzig\r\n\r\n Druck und Verlag von Philipp Reclam jun.\r\n\r\n * * * * *\r\n\r\n\r\n\r\n\r\n Erster Teil.\r\n\r\n »Die Rache ist mein, ich will vergelten.«\r\n\r\n 1.\r\n\r\n\r\nAlle glücklichen Familien sind einander ähnlich; jede unglücklich'
@ -407,12 +407,12 @@ languages. It has the same architecture as the BERT model but has been
trained on a different task: to produce identical embeddings for trained on a different task: to produce identical embeddings for
translation pairs. translation pairs.
|image1| |image01|
This makes LaBSE a great choice for our task and it can be reused for This makes LaBSE a great choice for our task and it can be reused for
different language pairs still producing good results. different language pairs still producing good results.
.. |image1| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/627d3a39-7076-479f-a7b1-392f49a0b83e .. |image01| image:: https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/627d3a39-7076-479f-a7b1-392f49a0b83e
.. code:: ipython3 .. code:: ipython3

56
docs/notebooks/ct-segmentation-quantize-nncf-with-output.rst
View File

@ -152,10 +152,10 @@ Imports
.. parsed-literal:: .. parsed-literal::
2024-05-15 23:50:56.536543: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-05 23:50:19.239580: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-15 23:50:56.573924: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-05 23:50:19.274774: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 23:50:57.174238: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-05 23:50:19.860831: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
.. parsed-literal:: .. parsed-literal::
@ -435,7 +435,7 @@ this notebook.
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x_e.shape[-i - 1] != x_0.shape[-i - 1]: if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
@ -534,18 +534,18 @@ Convert quantized model to OpenVINO IR model and save it.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:337: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return self._level_low.item() return self._level_low.item()
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/nncf/torch/quantization/layers.py:345: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return self._level_high.item() return self._level_high.item()
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/monai/networks/nets/basic_unet.py:168: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x_e.shape[-i - 1] != x_0.shape[-i - 1]: if x_e.shape[-i - 1] != x_0.shape[-i - 1]:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:1116: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
Tensor-likes are not close! Tensor-likes are not close!
Mismatched elements: 247888 / 262144 (94.6%) Mismatched elements: 249914 / 262144 (95.3%)
Greatest absolute difference: 6.055658936500549 at index (0, 0, 158, 273) (up to 1e-05 allowed) Greatest absolute difference: 4.319500803947449 at index (0, 0, 125, 295) (up to 1e-05 allowed)
Greatest relative difference: 13834.454856259192 at index (0, 0, 343, 273) (up to 1e-05 allowed) Greatest relative difference: 10952.699411314505 at index (0, 0, 220, 387) (up to 1e-05 allowed)
_check_trace( _check_trace(
@ -679,7 +679,7 @@ be run in the notebook with ``! benchmark_app`` or
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 8.68 ms [ INFO ] Read model took 8.75 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [?,?,?,?] [ INFO ] x (node: x) : f32 / [...] / [?,?,?,?]
@ -693,7 +693,7 @@ be run in the notebook with ``! benchmark_app`` or
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [?,1,16..,16..] [ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [?,1,16..,16..]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 149.48 ms [ INFO ] Compile model took 150.71 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -728,9 +728,9 @@ be run in the notebook with ``! benchmark_app`` or
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ ERROR ] Input x is dynamic. Provide data shapes! [ ERROR ] Input x is dynamic. Provide data shapes!
Traceback (most recent call last): Traceback (most recent call last):
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/main.py", line 486, in main
data_queue = get_input_data(paths_to_input, app_inputs_info) data_queue = get_input_data(paths_to_input, app_inputs_info)
File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data File "/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/openvino/tools/benchmark/utils/inputs_filling.py", line 123, in get_input_data
raise Exception(f"Input {info.name} is dynamic. Provide data shapes!") raise Exception(f"Input {info.name} is dynamic. Provide data shapes!")
Exception: Input x is dynamic. Provide data shapes! Exception: Input x is dynamic. Provide data shapes!
@ -758,7 +758,7 @@ be run in the notebook with ``! benchmark_app`` or
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 10.82 ms [ INFO ] Read model took 10.69 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [1,1,512,512] [ INFO ] x (node: x) : f32 / [...] / [1,1,512,512]
@ -772,7 +772,7 @@ be run in the notebook with ``! benchmark_app`` or
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [1,1,512,512] [ INFO ] ***NO_NAME*** (node: __module.final_conv/aten::_convolution/Add) : f32 / [...] / [1,1,512,512]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 257.40 ms [ INFO ] Compile model took 275.30 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model49 [ INFO ] NETWORK_NAME: Model49
@ -809,17 +809,17 @@ be run in the notebook with ``! benchmark_app`` or
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference synchronously, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference synchronously, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 27.38 ms [ INFO ] First inference took 29.48 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 959 iterations [ INFO ] Count: 969 iterations
[ INFO ] Duration: 15011.96 ms [ INFO ] Duration: 15001.31 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 15.40 ms [ INFO ] Median: 15.23 ms
[ INFO ] Average: 15.45 ms [ INFO ] Average: 15.28 ms
[ INFO ] Min: 15.18 ms [ INFO ] Min: 14.97 ms
[ INFO ] Max: 17.18 ms [ INFO ] Max: 17.10 ms
[ INFO ] Throughput: 63.88 FPS [ INFO ] Throughput: 64.59 FPS
Visually Compare Inference Results Visually Compare Inference Results
@ -904,7 +904,7 @@ seed is displayed to enable reproducing specific runs of this cell.
.. parsed-literal:: .. parsed-literal::
Visualizing results with seed 1715809926 Visualizing results with seed 1717624288
@ -987,8 +987,8 @@ performs inference, and displays the results on the frames loaded in
.. parsed-literal:: .. parsed-literal::
Loaded model to AUTO in 0.21 seconds. Loaded model to AUTO in 0.23 seconds.
Total time for 68 frames: 2.72 seconds, fps:25.33 Total time for 68 frames: 2.33 seconds, fps:29.56
References References

4
docs/notebooks/ct-segmentation-quantize-nncf-with-output_files/ct-segmentation-quantize-nncf-with-output_37_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:28010020834c2072a301b1a4eab4743fe594249fd6868e415af89e0dbc74892e oid sha256:1a4a6ad8cf666b4ce12cb07fa53b22a2ba0257697d926d9846ee4c18752fc553
size 383860 size 380300

778
docs/notebooks/ddcolor-image-colorization-with-output.rst Normal file
View File

@ -0,0 +1,778 @@
Colorize grayscale images using DDColor and OpenVINO
======================================================
Image colorization is the process of adding color to grayscale images.
Initially captured in black and white, these images are transformed into
vibrant, lifelike representations by estimating RGB colors. This
technology enhances both aesthetic appeal and perceptual quality.
Historically, artists manually applied colors to monochromatic
photographs, a painstaking task that could take up to a month for a
single image. However, with advancements in information technology and
the rise of deep neural networks, automated image colorization has
become increasingly important.
DDColor is one of the most progressive methods of image colorization in
our days. It is a novel approach using dual decoders: a pixel decoder
and a query-based color decoder, that stands out in its ability to
produce photo-realistic colorization, particularly in complex scenes
with multiple objects and diverse contexts. |image0|
More details about this approach can be found in original model
`repository <https://github.com/piddnad/DDColor>`__ and
`paper <https://arxiv.org/abs/2212.11613>`__.
In this tutorial we consider how to convert and run DDColor using
OpenVINO. Additionally, we will demonstrate how to optimize this model
using `NNCF <https://github.com/openvinotoolkit/nncf/>`__.
🪄 Let’s start to explore magic of image colorization!
Table of contents:
^^^^^^^^^^^^^^^^^^
- `Prerequisites <#prerequisites>`__
- `Load PyTorch model <#load-pytorch-model>`__
- `Run PyTorch model inference <#run-pytorch-model-inference>`__
- `Convert PyTorch model to OpenVINO Intermediate
Representation <#convert-pytorch-model-to-openvino-intermediate-representation>`__
- `Run OpenVINO model inference <#run-openvino-model-inference>`__
- `Optimize OpenVINO model using
NNCF <#optimize-openvino-model-using-nncf>`__
- `Collect quantization dataset <#collect-quantization-dataset>`__
- `Perform model quantization <#perform-model-quantization>`__
- `Run INT8 model inference <#run-int8-model-inference>`__
- `Compare FP16 and INT8 model
size <#compare-fp16-and-int8-model-size>`__
- `Compare inference time of the FP16 and INT8
models <#compare-inference-time-of-the-fp16-and-int8-models>`__
- `Interactive inference <#interactive-inference>`__
.. |image0| image:: https://github.com/piddnad/DDColor/raw/master/assets/network_arch.jpg
Prerequisites
-------------
.. code:: ipython3
import platform
import os
os.environ["GIT_CLONE_PROTECTION_ACTIVE"] = "false"
%pip install -q timm "torch>=2.1" "torchvision" "opencv_python" "pillow" "PyYAML" "scipy" "scikit-image" "datasets" "gradio>=4.19" --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -Uq --pre "openvino" --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q "git+https://github.com/openvinotoolkit/nncf.git"
if platform.python_version_tuple()[1] in ["8", "9"]:
%pip install -q "gradio-imageslider<=0.0.17" "typing-extensions>=4.9.0"
else:
%pip install -q "gradio-imageslider"
.. parsed-literal::
Note: you may need to restart the kernel to use updated packages.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
openvino-dev 2024.1.0 requires openvino==2024.1.0, but you have openvino 2024.3.0.dev20240605 which is incompatible.
Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages.
.. code:: ipython3
import sys
from pathlib import Path
repo_dir = Path("DDColor")
if not repo_dir.exists():
!git clone https://github.com/piddnad/DDColor.git
sys.path.append(str(repo_dir))
.. parsed-literal::
Cloning into 'DDColor'...
remote: Enumerating objects: 223, done.
remote: Counting objects: 100% (69/69), done.
remote: Compressing objects: 100% (35/35), done.
remote: Total 223 (delta 51), reused 36 (delta 33), pack-reused 154
Receiving objects: 100% (223/223), 13.34 MiB | 22.50 MiB/s, done.
Resolving deltas: 100% (72/72), done.
.. code:: ipython3
try:
from inference.colorization_pipeline_hf import DDColorHF, ImageColorizationPipelineHF
except Exception:
from inference.colorization_pipeline_hf import DDColorHF, ImageColorizationPipelineHF
Load PyTorch model
------------------
There are several models from DDColor’s family provided in `model
repository <https://github.com/piddnad/DDColor/blob/master/MODEL_ZOO.md>`__.
We will use DDColor-T, the most lightweight version of ddcolor model,
but demonstrated in the tutorial steps are also applicable to other
models from DDColor family.
.. code:: ipython3
import torch
model_name = "ddcolor_paper_tiny"
ddcolor_model = DDColorHF.from_pretrained(f"piddnad/{model_name}")
colorizer = ImageColorizationPipelineHF(model=ddcolor_model, input_size=512)
ddcolor_model.to("cpu")
colorizer.device = torch.device("cpu")
.. parsed-literal::
config.json: 0%| | 0.00/258 [00:00<?, ?B/s]
.. parsed-literal::
pytorch_model.bin: 0%| | 0.00/220M [00:00<?, ?B/s]
Run PyTorch model inference
---------------------------
.. code:: ipython3
import cv2
import PIL
IMG_PATH = "DDColor/assets/test_images/Ansel Adams _ Moore Photography.jpeg"
img = cv2.imread(IMG_PATH)
PIL.Image.fromarray(img[:, :, ::-1])
.. image:: ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_8_0.png
.. code:: ipython3
image_out = colorizer.process(img)
PIL.Image.fromarray(image_out[:, :, ::-1])
.. image:: ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_9_0.png
Convert PyTorch model to OpenVINO Intermediate Representation
-------------------------------------------------------------
OpenVINO supports PyTorch models via conversion to OpenVINO Intermediate
Representation (IR). OpenVINO model conversion API should be used for
these purposes. ``ov.convert_model`` function accepts original PyTorch
model instance and example input for tracing and returns ``ov.Model``
representing this model in OpenVINO framework. Converted model can be
used for saving on disk using ``ov.save_model`` function or directly
loading on device using ``core.complie_model``.
.. code:: ipython3
import openvino as ov
import torch
OV_COLORIZER_PATH = Path("ddcolor.xml")
if not OV_COLORIZER_PATH.exists():
ov_model = ov.convert_model(ddcolor_model, example_input=torch.ones((1, 3, 512, 512)), input=[1, 3, 512, 512])
ov.save_model(ov_model, OV_COLORIZER_PATH)
Run OpenVINO model inference
----------------------------
Select one of supported devices for inference using dropdown list.
.. code:: ipython3
import ipywidgets as widgets
core = ov.Core()
device = widgets.Dropdown(
options=core.available_devices + ["AUTO"],
value="AUTO",
description="Device:",
disabled=False,
)
device
.. parsed-literal::
Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO')
.. code:: ipython3
compiled_model = core.compile_model(OV_COLORIZER_PATH, device.value)
.. code:: ipython3
import cv2
import numpy as np
import torch
import torch.nn.functional as F
def process(img, compiled_model):
# Preprocess input image
height, width = img.shape[:2]
# Normalize to [0, 1] range
img = (img / 255.0).astype(np.float32)
orig_l = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)[:, :, :1] # (h, w, 1)
# Resize rgb image -> lab -> get grey -> rgb
img = cv2.resize(img, (512, 512))
img_l = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)[:, :, :1]
img_gray_lab = np.concatenate((img_l, np.zeros_like(img_l), np.zeros_like(img_l)), axis=-1)
img_gray_rgb = cv2.cvtColor(img_gray_lab, cv2.COLOR_LAB2RGB)
# Transpose HWC -> CHW and add batch dimension
tensor_gray_rgb = torch.from_numpy(img_gray_rgb.transpose((2, 0, 1))).float().unsqueeze(0)
# Run model inference
output_ab = compiled_model(tensor_gray_rgb)[0]
# Postprocess result
# resize ab -> concat original l -> rgb
output_ab_resize = F.interpolate(torch.from_numpy(output_ab), size=(height, width))[0].float().numpy().transpose(1, 2, 0)
output_lab = np.concatenate((orig_l, output_ab_resize), axis=-1)
output_bgr = cv2.cvtColor(output_lab, cv2.COLOR_LAB2BGR)
output_img = (output_bgr * 255.0).round().astype(np.uint8)
return output_img
.. code:: ipython3
ov_processed_img = process(img, compiled_model)
PIL.Image.fromarray(ov_processed_img[:, :, ::-1])
.. image:: ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_16_0.png
Optimize OpenVINO model using NNCF
----------------------------------
`NNCF <https://github.com/openvinotoolkit/nncf/>`__ enables
post-training quantization by adding quantization layers into model
graph and then using a subset of the training dataset to initialize the
parameters of these additional quantization layers. Quantized operations
are executed in ``INT8`` instead of ``FP32``/``FP16`` making model
inference faster.
The optimization process contains the following steps:
1. Create a calibration dataset for quantization.
2. Run ``nncf.quantize()`` to obtain quantized model.
3. Save the ``INT8`` model using ``openvino.save_model()`` function.
Please select below whether you would like to run quantization to
improve model inference speed.
.. code:: ipython3
to_quantize = widgets.Checkbox(
value=True,
description="Quantization",
disabled=False,
)
to_quantize
.. parsed-literal::
Checkbox(value=True, description='Quantization')
.. code:: ipython3
import requests
OV_INT8_COLORIZER_PATH = Path("ddcolor_int8.xml")
compiled_int8_model = None
r = requests.get(
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py",
)
open("skip_kernel_extension.py", "w").write(r.text)
%load_ext skip_kernel_extension
Collect quantization dataset
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We use a portion of
`ummagumm-a/colorization_dataset <https://huggingface.co/datasets/ummagumm-a/colorization_dataset>`__
dataset from Hugging Face as calibration data.
.. code:: ipython3
%%skip not $to_quantize.value
from datasets import load_dataset
subset_size = 300
calibration_data = []
if not OV_INT8_COLORIZER_PATH.exists():
dataset = load_dataset("ummagumm-a/colorization_dataset", split="train").shuffle(seed=42)
for idx, batch in enumerate(dataset):
if idx >= subset_size:
break
img = np.array(batch["conditioning_image"])
img = (img / 255.0).astype(np.float32)
img = cv2.resize(img, (512, 512))
img_l = cv2.cvtColor(np.stack([img, img, img], axis=2), cv2.COLOR_BGR2Lab)[:, :, :1]
img_gray_lab = np.concatenate((img_l, np.zeros_like(img_l), np.zeros_like(img_l)), axis=-1)
img_gray_rgb = cv2.cvtColor(img_gray_lab, cv2.COLOR_LAB2RGB)
image = np.expand_dims(img_gray_rgb.transpose((2, 0, 1)).astype(np.float32), axis=0)
calibration_data.append(image)
.. parsed-literal::
Downloading readme: 0%| | 0.00/574 [00:00<?, ?B/s]
.. parsed-literal::
Downloading data: 0%| | 0.00/127M [00:00<?, ?B/s]
.. parsed-literal::
Generating train split: 0%| | 0/1000 [00:00<?, ? examples/s]
Perform model quantization
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: ipython3
%%skip not $to_quantize.value
import nncf
if not OV_INT8_COLORIZER_PATH.exists():
ov_model = core.read_model(OV_COLORIZER_PATH)
quantized_model = nncf.quantize(
model=ov_model,
subset_size=subset_size,
calibration_dataset=nncf.Dataset(calibration_data),
)
ov.save_model(quantized_model, OV_INT8_COLORIZER_PATH)
.. parsed-literal::
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
.. parsed-literal::
2024-06-05 23:53:12.619257: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-06-05 23:53:12.657603: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-05 23:53:13.253249: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
.. parsed-literal::
Output()
.. raw:: html
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
.. raw:: html
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
</pre>
.. parsed-literal::
Output()
.. raw:: html
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"></pre>
.. raw:: html
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">
</pre>
Run INT8 model inference
------------------------
.. code:: ipython3
from IPython.display import display
if OV_INT8_COLORIZER_PATH.exists():
compiled_int8_model = core.compile_model(OV_INT8_COLORIZER_PATH, device.value)
img = cv2.imread("DDColor/assets/test_images/Ansel Adams _ Moore Photography.jpeg")
img_out = process(img, compiled_int8_model)
display(PIL.Image.fromarray(img_out[:, :, ::-1]))
.. image:: ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_25_0.png
Compare FP16 and INT8 model size
--------------------------------
.. code:: ipython3
fp16_ir_model_size = OV_COLORIZER_PATH.with_suffix(".bin").stat().st_size / 2**20
print(f"FP16 model size: {fp16_ir_model_size:.2f} MB")
if OV_INT8_COLORIZER_PATH.exists():
quantized_model_size = OV_INT8_COLORIZER_PATH.with_suffix(".bin").stat().st_size / 2**20
print(f"INT8 model size: {quantized_model_size:.2f} MB")
print(f"Model compression rate: {fp16_ir_model_size / quantized_model_size:.3f}")
.. parsed-literal::
FP16 model size: 104.89 MB
INT8 model size: 52.97 MB
Model compression rate: 1.980
Compare inference time of the FP16 and INT8 models
--------------------------------------------------
To measure the inference performance of OpenVINO FP16 and INT8 models,
use `Benchmark
Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-tool.html>`__.
**NOTE**: For the most accurate performance estimation, it is
recommended to run ``benchmark_app`` in a terminal/command prompt
after closing other applications.
.. code:: ipython3
!benchmark_app -m $OV_COLORIZER_PATH -d $device.value -api async -shape "[1,3,512,512]" -t 15
.. parsed-literal::
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ]
[ INFO ] Device info:
[ INFO ] AUTO
[ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 42.73 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [1,3,512,512]
[ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.refine_net.0.0/aten::_convolution/Add) : f32 / [...] / [1,2,512,512]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'x': [1,3,512,512]
[ INFO ] Reshape model took 0.03 ms
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ] x (node: x) : u8 / [N,C,H,W] / [1,3,512,512]
[ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.refine_net.0.0/aten::_convolution/Add) : f32 / [...] / [1,2,512,512]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 1527.31 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0
[ INFO ] EXECUTION_DEVICES: ['CPU']
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
[ INFO ] MULTI_DEVICE_PRIORITIES: CPU
[ INFO ] CPU:
[ INFO ] AFFINITY: Affinity.CORE
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
[ INFO ] CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
[ INFO ] DYNAMIC_QUANTIZATION_GROUP_SIZE: 0
[ INFO ] ENABLE_CPU_PINNING: True
[ INFO ] ENABLE_HYPER_THREADING: True
[ INFO ] EXECUTION_DEVICES: ['CPU']
[ INFO ] EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE
[ INFO ] INFERENCE_NUM_THREADS: 24
[ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float32'>
[ INFO ] KV_CACHE_PRECISION: <Type: 'float16'>
[ INFO ] LOG_LEVEL: Level.NO
[ INFO ] MODEL_DISTRIBUTION_POLICY: set()
[ INFO ] NETWORK_NAME: Model0
[ INFO ] NUM_STREAMS: 6
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
[ INFO ] PERFORMANCE_HINT: THROUGHPUT
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ] PERF_COUNT: NO
[ INFO ] SCHEDULING_CORE_TYPE: SchedulingCoreType.ANY_CORE
[ INFO ] MODEL_PRIORITY: Priority.MEDIUM
[ INFO ] LOADED_FROM_CACHE: False
[ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 548.36 ms
[Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 72 iterations
[ INFO ] Duration: 16675.11 ms
[ INFO ] Latency:
[ INFO ] Median: 1385.45 ms
[ INFO ] Average: 1386.39 ms
[ INFO ] Min: 1326.48 ms
[ INFO ] Max: 1447.89 ms
[ INFO ] Throughput: 4.32 FPS
.. code:: ipython3
if OV_INT8_COLORIZER_PATH.exists():
!benchmark_app -m $OV_INT8_COLORIZER_PATH -d $device.value -api async -shape "[1,3,512,512]" -t 15
.. parsed-literal::
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ]
[ INFO ] Device info:
[ INFO ] AUTO
[ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 67.54 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [1,3,512,512]
[ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.refine_net.0.0/aten::_convolution/Add) : f32 / [...] / [1,2,512,512]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'x': [1,3,512,512]
[ INFO ] Reshape model took 0.03 ms
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ] x (node: x) : u8 / [N,C,H,W] / [1,3,512,512]
[ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: __module.refine_net.0.0/aten::_convolution/Add) : f32 / [...] / [1,2,512,512]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 2704.43 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0
[ INFO ] EXECUTION_DEVICES: ['CPU']
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
[ INFO ] MULTI_DEVICE_PRIORITIES: CPU
[ INFO ] CPU:
[ INFO ] AFFINITY: Affinity.CORE
[ INFO ] CPU_DENORMALS_OPTIMIZATION: False
[ INFO ] CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
[ INFO ] DYNAMIC_QUANTIZATION_GROUP_SIZE: 0
[ INFO ] ENABLE_CPU_PINNING: True
[ INFO ] ENABLE_HYPER_THREADING: True
[ INFO ] EXECUTION_DEVICES: ['CPU']
[ INFO ] EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE
[ INFO ] INFERENCE_NUM_THREADS: 24
[ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float32'>
[ INFO ] KV_CACHE_PRECISION: <Type: 'float16'>
[ INFO ] LOG_LEVEL: Level.NO
[ INFO ] MODEL_DISTRIBUTION_POLICY: set()
[ INFO ] NETWORK_NAME: Model0
[ INFO ] NUM_STREAMS: 6
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 6
[ INFO ] PERFORMANCE_HINT: THROUGHPUT
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ] PERF_COUNT: NO
[ INFO ] SCHEDULING_CORE_TYPE: SchedulingCoreType.ANY_CORE
[ INFO ] MODEL_PRIORITY: Priority.MEDIUM
[ INFO ] LOADED_FROM_CACHE: False
[ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 290.81 ms
[Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 150 iterations
[ INFO ] Duration: 15775.52 ms
[ INFO ] Latency:
[ INFO ] Median: 627.63 ms
[ INFO ] Average: 626.88 ms
[ INFO ] Min: 535.42 ms
[ INFO ] Max: 721.71 ms
[ INFO ] Throughput: 9.51 FPS
Interactive inference
---------------------
.. code:: ipython3
import gradio as gr
from gradio_imageslider import ImageSlider
from functools import partial
def generate(image, use_int8=True):
image_in = cv2.imread(image)
image_out = process(image_in, compiled_model if not use_int8 else compiled_int8_model)
image_in_pil = PIL.Image.fromarray(cv2.cvtColor(image_in, cv2.COLOR_BGR2RGB))
image_out_pil = PIL.Image.fromarray(cv2.cvtColor(image_out, cv2.COLOR_BGR2RGB))
return (image_in_pil, image_out_pil)
with gr.Blocks() as demo:
with gr.Row(equal_height=False):
image = gr.Image(type="filepath")
with gr.Column():
output_image = ImageSlider(show_label=True, type="filepath", interactive=False, label="FP16 model output")
button = gr.Button(value="Run{}".format(" FP16 model" if compiled_int8_model is not None else ""))
with gr.Column(visible=compiled_int8_model is not None):
output_image_int8 = ImageSlider(show_label=True, type="filepath", interactive=False, label="INT8 model output")
button_i8 = gr.Button(value="Run INT8 model")
button.click(fn=partial(generate, use_int8=False), inputs=[image], outputs=[output_image])
button_i8.click(fn=partial(generate, use_int8=True), inputs=[image], outputs=[output_image_int8])
examples = gr.Examples(
[
"DDColor/assets/test_images/New York Riverfront December 15, 1931.jpg",
"DDColor/assets/test_images/Audrey Hepburn.jpg",
"DDColor/assets/test_images/Acrobats Balance On Top Of The Empire State Building, 1934.jpg",
],
inputs=[image],
)
if __name__ == "__main__":
try:
demo.queue().launch(debug=False)
except Exception:
demo.queue().launch(share=True, debug=False)
# if you are launching remotely, specify server_name and server_port
# demo.launch(server_name='your server name', server_port='server port in int')
# Read more in the docs: https://gradio.app/docs/
.. parsed-literal::
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_16_0.jpg Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e152195f834e9ac26e7df50e12c80cf13ee8d747e62dd29edf85b6fabb98cf84
size 164898

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_16_0.png Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f5d2b934619e33836937b6f9fe32987a37ca7506ecaf7d044d8286b6212bebe5
size 1497558

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_25_0.jpg Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c70e32cab8d811d577a589a2212a382d02aa7c51f0093e8537ee86828314c592
size 167271

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_25_0.png Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:634bf7305f368aea78f429424e11e7f00fc341a485d74823434ac1d624a6515d
size 1521926

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_8_0.jpg Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:614b930200ec51d4631b7c2a3cc743e18a8f3f94c3ea47673ef1fca37028af31
size 152058

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_8_0.png Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c105b02bbf0b3c82c253b6b4e92d7b28518775279197cbf239ca40dd52c9e9e
size 792834

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_9_0.jpg Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a92ff02af12364a2b15169f96657de93c6a172e20174f7b8a857578d58405f9
size 164917

3
docs/notebooks/ddcolor-image-colorization-with-output_files/ddcolor-image-colorization-with-output_9_0.png Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:82894174823bc19a70eb250cf7238d542f2d81127e6bd3760d210c94796f2968
size 1497540

8
docs/notebooks/decidiffusion-image-generation-with-output.rst
View File

@ -1174,12 +1174,8 @@ improve model inference speed.
.. code:: ipython3 .. code:: ipython3
to_quantize = widgets.Checkbox( skip_for_device = "GPU" in device.value
value=True, to_quantize = widgets.Checkbox(value=not skip_for_device, description="Quantization", disabled=skip_for_device)
description="Quantization",
disabled=False,
)
to_quantize to_quantize

36
docs/notebooks/depth-anything-with-output.rst
View File

File diff suppressed because one or more lines are too long

4
docs/notebooks/depth-anything-with-output_files/depth-anything-with-output_44_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:5c42b9045954f576a8bdcc0318d85ee8132c8086e27359768e6e5e8158c412de oid sha256:325b2ce95cbb3fcfc44950baeefaf565052b4ba44f51f2a6afc76f6fa74ea5c3
size 451541 size 451387

2
docs/notebooks/detectron2-to-openvino-with-output.rst
View File

@ -52,7 +52,7 @@ Install required packages for running model
.. code:: ipython3 .. code:: ipython3
%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu torch torchvision %pip install -q "torch" "torchvision" "opencv-python" "wheel" --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -q "git+https://github.com/facebookresearch/detectron2.git" --extra-index-url https://download.pytorch.org/whl/cpu %pip install -q "git+https://github.com/facebookresearch/detectron2.git" --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -q "openvino>=2023.1.0" %pip install -q "openvino>=2023.1.0"

4
docs/notebooks/detectron2-to-openvino-with-output_files/detectron2-to-openvino-with-output_22_0.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:402297f642d1ae24c5f34c947b243308e80a2353a297ff4fbc29a80a91447f3d oid sha256:eb6a3c05535408d011ad8aa46258978d309afda0fe57ab8c97ea81078386469c
size 57749 size 58366

4
docs/notebooks/detectron2-to-openvino-with-output_files/detectron2-to-openvino-with-output_22_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:bfdce03dd80512c030e07a93860311e984178cc6c00aea85904d97a9d85fd91b oid sha256:84897b3dce254b9f229b11ab01345853e5a8397bb232c2dbf9e9062f6a3d3857
size 507627 size 509075

4
docs/notebooks/detectron2-to-openvino-with-output_files/detectron2-to-openvino-with-output_32_0.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:d467c3878c59234b1e763ec286626e3036584d037e32857396ec873b73347d3c oid sha256:c558219ed9c6dae7fa76cbd8791cdb30ea5ee7d88af767ff82b989067e11ff6b
size 53573 size 54727

4
docs/notebooks/detectron2-to-openvino-with-output_files/detectron2-to-openvino-with-output_32_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:ef9d6efc8183a218693431a458cd84831e4820d02518cfd9d67f6c532ffc6f53 oid sha256:a3202c1e5f4b8914b4290f1189e147f965b87d6556b8ed8d59d12ebf7812db9a
size 456551 size 457983

206
docs/notebooks/distil-whisper-asr-with-output.rst
View File

File diff suppressed because one or more lines are too long

68
docs/notebooks/distilbert-sequence-classification-with-output.rst
View File

@ -36,31 +36,31 @@ Imports
.. parsed-literal:: .. parsed-literal::
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
Requirement already satisfied: openvino>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2024.1.0) Requirement already satisfied: openvino>=2023.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2024.3.0.dev20240605)
Requirement already satisfied: transformers in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.40.2) Requirement already satisfied: transformers in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.41.2)
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu) Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.1+cpu)
Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.4) Requirement already satisfied: tqdm in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (4.66.4)
Requirement already satisfied: numpy<2.0.0,>=1.16.6 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (1.23.5) Requirement already satisfied: numpy<2.0.0,>=1.16.6 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (1.23.5)
Requirement already satisfied: openvino-telemetry>=2023.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (2024.1.0) Requirement already satisfied: openvino-telemetry>=2023.2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (2024.1.0)
Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (24.0) Requirement already satisfied: packaging in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from openvino>=2023.1.0) (24.0)
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (3.14.0) Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (3.14.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.23.0) Requirement already satisfied: huggingface-hub<1.0,>=0.23.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.23.3)
Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (6.0.1) Requirement already satisfied: pyyaml>=5.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2024.5.15) Requirement already satisfied: regex!=2019.12.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2024.5.15)
Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2.31.0) Requirement already satisfied: requests in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (2.32.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.19.1) Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.19.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.4.3) Requirement already satisfied: safetensors>=0.4.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from transformers) (0.4.3)
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0) Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.12.1)
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12) Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12.1)
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1) Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4) Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1) Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5) Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.3.2) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.7) Requirement already satisfied: idna<4,>=2.5 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2.2.1) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2024.2.2) Requirement already satisfied: certifi>=2017.4.17 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from requests->transformers) (2024.6.2)
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0) Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -87,13 +87,6 @@ model from Hugging Face.
checkpoint = "distilbert-base-uncased-finetuned-sst-2-english" checkpoint = "distilbert-base-uncased-finetuned-sst-2-english"
model = AutoModelForSequenceClassification.from_pretrained(pretrained_model_name_or_path=checkpoint) model = AutoModelForSequenceClassification.from_pretrained(pretrained_model_name_or_path=checkpoint)
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Initializing the Tokenizer Initializing the Tokenizer
-------------------------- --------------------------
@ -114,13 +107,6 @@ understand the context of a sentence. Here, we will use
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=checkpoint) tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=checkpoint)
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Convert Model to OpenVINO Intermediate Representation format Convert Model to OpenVINO Intermediate Representation format
------------------------------------------------------------ ------------------------------------------------------------
@ -157,9 +143,9 @@ optimal execution on end-point target devices.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:231: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
mask, torch.tensor(torch.finfo(scores.dtype).min) mask, torch.tensor(torch.finfo(scores.dtype).min)

6
docs/notebooks/dolly-2-instruction-following-with-output.rst
View File

@ -124,8 +124,10 @@ documentation <https://huggingface.co/docs/optimum/intel/inference>`__.
.. code:: ipython3 .. code:: ipython3
%pip unsinstall -q -y openvino openvino-dev openvino-nightly optimum optimum-intel %pip install -Uq pip
%pip install -q "diffusers>=0.16.1" "transformers>=4.33.0" "torch>=2.1" "openvino-nightly" "nncf>=2.10.0" onnx "gradio>=4.19" --extra-index-url https://download.pytorch.org/whl/cpu %pip uninstall -q -y optimum optimum-intel
%pip install --pre -Uq openvino openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q "diffusers>=0.16.1" "transformers>=4.33.0" "torch>=2.1" "nncf>=2.10.0" onnx "gradio>=4.19" --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -q "git+https://github.com/huggingface/optimum-intel.git" %pip install -q "git+https://github.com/huggingface/optimum-intel.git"
Convert model using Optimum-CLI tool Convert model using Optimum-CLI tool

69
docs/notebooks/dynamicrafter-animating-images-with-output.rst
View File

@ -123,14 +123,13 @@ Prerequisites
.. code:: ipython3 .. code:: ipython3
%pip uninstall -q -y openvino-dev openvino openvino-nightly %pip install -Uq pip
%pip install -q --upgrade openvino-nightly %pip install --pre -Uq openvino --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q "gradio>=4.19" omegaconf decord einops pytorch_lightning kornia open_clip_torch transformers av opencv-python torch --extra-index-url https://download.pytorch.org/whl/cpu %pip install -q "gradio>=4.19" omegaconf decord einops pytorch_lightning kornia open_clip_torch transformers av opencv-python torch --extra-index-url https://download.pytorch.org/whl/cpu
.. parsed-literal:: .. parsed-literal::
WARNING: Skipping openvino-nightly as it is not installed.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -154,12 +153,12 @@ Prerequisites
.. parsed-literal:: .. parsed-literal::
Cloning into 'dynamicrafter'... Cloning into 'dynamicrafter'...
remote: Enumerating objects: 288, done. remote: Enumerating objects: 294, done.
remote: Counting objects: 100% (125/125), done. remote: Counting objects: 100% (131/131), done.
remote: Compressing objects: 100% (71/71), done. remote: Compressing objects: 100% (75/75), done.
remote: Total 288 (delta 74), reused 71 (delta 54), pack-reused 163 remote: Total 294 (delta 79), reused 76 (delta 56), pack-reused 163
Receiving objects: 100% (288/288), 72.40 MiB | 24.98 MiB/s, done. Receiving objects: 100% (294/294), 72.40 MiB | 26.63 MiB/s, done.
Resolving deltas: 100% (89/89), done. Resolving deltas: 100% (94/94), done.
Load and run the original pipeline Load and run the original pipeline
@ -207,7 +206,7 @@ We will use model for 256x256 resolution as example. Also, models for
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.
For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder. For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.
warnings.warn( warnings.warn(
@ -316,43 +315,43 @@ resolutions.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/utils/image.py:226: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/utils/image.py:226: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if input.numel() == 0: if input.numel() == 0:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:573: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:573: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if size == input_size: if size == input_size:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:579: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:579: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
antialias = antialias and (max(factors) > 1) antialias = antialias and (max(factors) > 1)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:581: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:581: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if antialias: if antialias:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:584: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:584: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
sigmas = (max((factors[0] - 1.0) / 2.0, 0.001), max((factors[1] - 1.0) / 2.0, 0.001)) sigmas = (max((factors[0] - 1.0) / 2.0, 0.001), max((factors[1] - 1.0) / 2.0, 0.001))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3)) ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/geometry/transform/affwarp.py:589: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3)) ks = int(max(2.0 * 2 * sigmas[0], 3)), int(max(2.0 * 2 * sigmas[1], 3))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
sigma = tensor([sigma], device=input.device, dtype=input.dtype) sigma = tensor([sigma], device=input.device, dtype=input.dtype)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/gaussian.py:55: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
sigma = tensor([sigma], device=input.device, dtype=input.dtype) sigma = tensor([sigma], device=input.device, dtype=input.dtype)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/core/check.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/core/check.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x_shape_to_check[i] != dim: if x_shape_to_check[i] != dim:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/kernels.py:92: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/filters/kernels.py:92: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
mean = tensor([[mean]], device=sigma.device, dtype=sigma.dtype) mean = tensor([[mean]], device=sigma.device, dtype=sigma.dtype)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if len(mean.shape) == 0 or mean.shape[0] == 1: if len(mean.shape) == 0 or mean.shape[0] == 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:103: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:103: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if len(std.shape) == 0 or std.shape[0] == 1: if len(std.shape) == 0 or std.shape[0] == 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:107: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:107: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if mean.shape and mean.shape[0] != 1: if mean.shape and mean.shape[0] != 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:108: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:108: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if mean.shape[0] != data.shape[1] and mean.shape[:2] != data.shape[:2]: if mean.shape[0] != data.shape[1] and mean.shape[:2] != data.shape[:2]:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:112: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:112: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if std.shape and std.shape[0] != 1: if std.shape and std.shape[0] != 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:113: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:113: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if std.shape[0] != data.shape[1] and std.shape[:2] != data.shape[:2]: if std.shape[0] != data.shape[1] and std.shape[:2] != data.shape[:2]:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:116: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:116: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
mean = torch.as_tensor(mean, device=data.device, dtype=data.dtype) mean = torch.as_tensor(mean, device=data.device, dtype=data.dtype)
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:117: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/kornia/enhance/normalize.py:117: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
std = torch.as_tensor(std, device=data.device, dtype=data.dtype) std = torch.as_tensor(std, device=data.device, dtype=data.dtype)
@ -377,7 +376,7 @@ Convert AE encoder
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/ae_modules.py:67: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/ae_modules.py:67: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
w_ = w_ * (int(c)**(-0.5)) w_ = w_ * (int(c)**(-0.5))
@ -416,15 +415,15 @@ Convert Diffusion U-Net model
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:556: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:556: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if l_context == 77 + t*16: ## !!! HARD CODE here if l_context == 77 + t*16: ## !!! HARD CODE here
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:205: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:205: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if batch_size: if batch_size:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:232: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:232: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if self.use_temporal_conv and batch_size: if self.use_temporal_conv and batch_size:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:76: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:76: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert x.shape[1] == self.channels assert x.shape[1] == self.channels
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:99: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/dynamicrafter-animating-images/dynamicrafter/lvdm/modules/networks/openaimodel3d.py:99: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert x.shape[1] == self.channels assert x.shape[1] == self.channels

88
docs/notebooks/efficient-sam-with-output.rst
View File

@ -105,9 +105,9 @@ Prerequisites
remote: Counting objects: 100% (85/85), done. remote: Counting objects: 100% (85/85), done.
remote: Compressing objects: 100% (33/33), done. remote: Compressing objects: 100% (33/33), done.
remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339 remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339
Receiving objects: 100% (424/424), 262.14 MiB | 26.62 MiB/s, done. Receiving objects: 100% (424/424), 262.14 MiB | 26.65 MiB/s, done.
Resolving deltas: 100% (246/246), done. Resolving deltas: 100% (246/246), done.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM
Load PyTorch model Load PyTorch model
@ -362,23 +362,23 @@ disk using ``openvino.save_model``.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:220: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:220: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if ( if (
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:241: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:241: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert ( assert (
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:163: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:163: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
size = int(math.sqrt(xy_num)) size = int(math.sqrt(xy_num))
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:164: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:164: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert size * size == xy_num assert size * size == xy_num
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:166: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:166: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if size != h or size != w: if size != h or size != w:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:251: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam_encoder.py:251: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert x.shape[2] == num_patches assert x.shape[2] == num_patches
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if num_pts > self.decoder_max_num_input_points: if num_pts > self.decoder_max_num_input_points:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:92: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:92: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
elif num_pts < self.decoder_max_num_input_points: elif num_pts < self.decoder_max_num_input_points:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:126: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/efficient-sam/EfficientSAM/efficient_sam/efficient_sam.py:126: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if output_w > 0 and output_h > 0: if output_w > 0 and output_h > 0:
@ -642,10 +642,10 @@ architecture type, we should specify ``transformer`` in ``model_type``.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:09:26.533883: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:16:32.701538: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:09:26.567663: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:16:32.734051: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:09:27.185648: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:16:33.371409: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
@ -839,26 +839,26 @@ models, we use ``bencmark_app``.
[ INFO ] Parsing input parameters [ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1 [ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] AUTO [ INFO ] AUTO
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1 [ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 40.50 ms [ INFO ] Read model took 29.27 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?] [ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?]
[ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?] [ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?]
[ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?] [ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,3,?,?]
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,3]
[Step 5/11] Resizing model to match image sizes and given batch [Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1 [ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model [Step 6/11] Configuring input of the model
@ -867,10 +867,10 @@ models, we use ``bencmark_app``.
[ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?] [ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?]
[ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?] [ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,3,?,?]
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,3]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 1359.16 ms [ INFO ] Compile model took 1448.36 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -911,17 +911,17 @@ models, we use ``bencmark_app``.
[ INFO ] Fill input 'batched_point_labels' with random values [ INFO ] Fill input 'batched_point_labels' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop). [ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop).
[ INFO ] First inference took 644.87 ms [ INFO ] First inference took 677.52 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 49 iterations [ INFO ] Count: 49 iterations
[ INFO ] Duration: 15741.55 ms [ INFO ] Duration: 15956.35 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 1892.64 ms [ INFO ] Median: 1926.16 ms
[ INFO ] Average: 1876.78 ms [ INFO ] Average: 1900.48 ms
[ INFO ] Min: 664.09 ms [ INFO ] Min: 644.61 ms
[ INFO ] Max: 2004.53 ms [ INFO ] Max: 1989.53 ms
[ INFO ] Throughput: 3.11 FPS [ INFO ] Throughput: 3.07 FPS
.. code:: ipython3 .. code:: ipython3
@ -936,26 +936,26 @@ models, we use ``bencmark_app``.
[ INFO ] Parsing input parameters [ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1 [ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] AUTO [ INFO ] AUTO
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1 [ INFO ] Build ................................. 2024.3.0-15599-de4d00a5970
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 54.09 ms [ INFO ] Read model took 42.94 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?] [ INFO ] batched_images (node: batched_images) : f32 / [...] / [?,?,?,?]
[ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?] [ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?]
[ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?] [ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,3,?,?]
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,3]
[Step 5/11] Resizing model to match image sizes and given batch [Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1 [ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model [Step 6/11] Configuring input of the model
@ -964,10 +964,10 @@ models, we use ``bencmark_app``.
[ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?] [ INFO ] batched_points (node: batched_points) : i64 / [...] / [?,?,?,?]
[ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?] [ INFO ] batched_point_labels (node: batched_point_labels) : i64 / [...] / [?,?,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_3) : f32 / [...] / [?,?,3,?,?]
[ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,?] [ INFO ] ***NO_NAME*** (node: aten::reshape/Reshape_2) : f32 / [...] / [?,?,3]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 1870.95 ms [ INFO ] Compile model took 1975.21 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -1008,17 +1008,17 @@ models, we use ``bencmark_app``.
[ INFO ] Fill input 'batched_point_labels' with random values [ INFO ] Fill input 'batched_point_labels' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop). [ INFO ] Benchmarking in full mode (inputs filling are included in measurement loop).
[ INFO ] First inference took 583.07 ms [ INFO ] First inference took 609.28 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 55 iterations [ INFO ] Count: 55 iterations
[ INFO ] Duration: 16526.72 ms [ INFO ] Duration: 16666.59 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 1791.04 ms [ INFO ] Median: 1799.56 ms
[ INFO ] Average: 1766.59 ms [ INFO ] Average: 1780.10 ms
[ INFO ] Min: 586.80 ms [ INFO ] Min: 641.70 ms
[ INFO ] Max: 1852.64 ms [ INFO ] Max: 1867.14 ms
[ INFO ] Throughput: 3.33 FPS [ INFO ] Throughput: 3.30 FPS
Interactive segmentation demo Interactive segmentation demo

4
docs/notebooks/efficient-sam-with-output_files/efficient-sam-with-output_16_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:6023d12350dd3ca910572f4dad89f563abcd31328ac27846f5ddc64705b4daf5 oid sha256:45a2aec656ead3328284564339cc490e379c5ad16d1a1a4922f99eb84037b818
size 1260186 size 1259656

4
docs/notebooks/efficient-sam-with-output_files/efficient-sam-with-output_24_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:ac4eb9a2fe78ace641f00796a0d59f9383c3ee6904c2fa69a6e2eec1af1e779e oid sha256:7dd1210236f949071574ecd9097962a3db3156b0c44810aa2f2b6d03576f6ca3
size 1259619 size 1260203

4
docs/notebooks/efficient-sam-with-output_files/efficient-sam-with-output_35_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:faa1d90c28aeb3a308002ab6430ef3f26c078f561f1e776770f2461bed61b074 oid sha256:89fc9efa319522df2dd0638e28e020e8695a8c791db61da44f75703b1d493470
size 1262279 size 1262684

20
docs/notebooks/encodec-audio-compression-with-output.rst
View File

File diff suppressed because one or more lines are too long

4
docs/notebooks/encodec-audio-compression-with-output_files/encodec-audio-compression-with-output_38_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:c02accf9f1cf8a5dff3d4c78aa47ae41802250a855310353044c2f58a5689c10 oid sha256:aea9089d7a4630b53481b1277bbf8e7f52f1c992ed61d1e998250980f59df5ab
size 44203 size 44186

29
docs/notebooks/fast-segment-anything-with-output.rst
View File

@ -66,7 +66,7 @@ Install requirements
.. code:: ipython3 .. code:: ipython3
%pip install -q "ultralytics==8.1.42" onnx tqdm --extra-index-url https://download.pytorch.org/whl/cpu %pip install -q "ultralytics==8.2.24" onnx tqdm --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -q "openvino-dev>=2024.0.0" %pip install -q "openvino-dev>=2024.0.0"
%pip install -q "nncf>=2.9.0" %pip install -q "nncf>=2.9.0"
%pip install -q "gradio>=4.13" %pip install -q "gradio>=4.13"
@ -140,12 +140,12 @@ model and generate a segmentation map.
.. parsed-literal:: .. parsed-literal::
Downloading https://github.com/ultralytics/assets/releases/download/v8.1.0/FastSAM-x.pt to 'FastSAM-x.pt'... Downloading https://github.com/ultralytics/assets/releases/download/v8.2.0/FastSAM-x.pt to 'FastSAM-x.pt'...
.. parsed-literal:: .. parsed-literal::
100%|██████████| 138M/138M [00:03<00:00, 45.4MB/s] 100%|██████████| 138M/138M [00:01<00:00, 100MB/s]
@ -157,8 +157,8 @@ model and generate a segmentation map.
.. parsed-literal:: .. parsed-literal::
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 768x1024 37 objects, 655.2ms image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 768x1024 37 objects, 658.4ms
Speed: 4.0ms preprocess, 655.2ms inference, 30.8ms postprocess per image at shape (1, 3, 768, 1024) Speed: 4.0ms preprocess, 658.4ms inference, 586.5ms postprocess per image at shape (1, 3, 768, 1024)
The model returns segmentation maps for all the objects on the image. The model returns segmentation maps for all the objects on the image.
@ -196,7 +196,7 @@ tracing. The FastSAM model itself is based on YOLOv8 model.
.. parsed-literal:: .. parsed-literal::
Ultralytics YOLOv8.1.42 🚀 Python-3.8.10 torch-2.3.0+cpu CPU (Intel Core(TM) i9-10920X 3.50GHz) Ultralytics YOLOv8.2.24 🚀 Python-3.8.10 torch-2.3.1+cpu CPU (Intel Core(TM) i9-10920X 3.50GHz)
PyTorch: starting from 'FastSAM-x.pt' with input shape (1, 3, 1024, 1024) BCHW and output shape(s) ((1, 37, 21504), (1, 32, 256, 256)) (138.2 MB) PyTorch: starting from 'FastSAM-x.pt' with input shape (1, 3, 1024, 1024) BCHW and output shape(s) ((1, 37, 21504), (1, 32, 256, 256)) (138.2 MB)
@ -204,7 +204,7 @@ tracing. The FastSAM model itself is based on YOLOv8 model.
OpenVINO: export success ✅ 6.2s, saved as 'FastSAM-x_openvino_model/' (276.1 MB) OpenVINO: export success ✅ 6.2s, saved as 'FastSAM-x_openvino_model/' (276.1 MB)
Export complete (9.2s) Export complete (9.2s)
Results saved to /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything Results saved to /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/fast-segment-anything
Predict: yolo predict task=segment model=FastSAM-x_openvino_model imgsz=1024 Predict: yolo predict task=segment model=FastSAM-x_openvino_model imgsz=1024
Validate: yolo val task=segment model=FastSAM-x_openvino_model imgsz=1024 data=ultralytics/datasets/sa.yaml Validate: yolo val task=segment model=FastSAM-x_openvino_model imgsz=1024 data=ultralytics/datasets/sa.yaml
Visualize: https://netron.app Visualize: https://netron.app
@ -313,8 +313,8 @@ pipeline.
.. parsed-literal:: .. parsed-literal::
image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 1024x1024 42 objects, 512.0ms image 1/1 /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/fast-segment-anything/coco_bike.jpg: 1024x1024 42 objects, 503.6ms
Speed: 6.5ms preprocess, 512.0ms inference, 29.3ms postprocess per image at shape (1, 3, 1024, 1024) Speed: 7.6ms preprocess, 503.6ms inference, 38.0ms postprocess per image at shape (1, 3, 1024, 1024)
One can observe the converted model outputs in the next cell, they is One can observe the converted model outputs in the next cell, they is
@ -513,6 +513,11 @@ repo <../yolov8-optimization/>`__.
preset=nncf.QuantizationPreset.MIXED) preset=nncf.QuantizationPreset.MIXED)
.. parsed-literal::
<string>:7: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
.. parsed-literal:: .. parsed-literal::
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
@ -651,9 +656,9 @@ calibration dataset to measure the performance.
.. parsed-literal:: .. parsed-literal::
Segmented in 22 seconds Segmented in 23 seconds
Resulting in 5.82 fps Resulting in 5.57 fps
That is 3.09 times faster! That is 2.96 times faster!
Try out the converted pipeline Try out the converted pipeline

192
docs/notebooks/freevc-voice-conversion-with-output.rst
View File

File diff suppressed because one or more lines are too long

29
docs/notebooks/grounded-segment-anything-with-output.rst
View File

@ -48,7 +48,7 @@ Clone repositories and install requirements
.. code:: ipython3 .. code:: ipython3
%pip install -q "openvino>=2024.0" "torch>=2.1" opencv-python supervision transformers yapf pycocotools addict "gradio>=4.19" tqdm timm --extra-index-url https://download.pytorch.org/whl/cpu %pip install -q "openvino>=2024.0" "torch>=2.1" opencv-python "supervision[desktop]" transformers yapf pycocotools addict "gradio>=4.19" tqdm timm --extra-index-url https://download.pytorch.org/whl/cpu
.. parsed-literal:: .. parsed-literal::
@ -111,17 +111,17 @@ segmentation you can select vanilla ``SAM``.
Cloning into 'GroundingDINO'... Cloning into 'GroundingDINO'...
remote: Enumerating objects: 379, done. remote: Enumerating objects: 379, done.
remote: Counting objects: 100% (175/175), done. remote: Counting objects: 100% (177/177), done.
remote: Compressing objects: 100% (63/63), done. remote: Compressing objects: 100% (64/64), done.
remote: Total 379 (delta 135), reused 112 (delta 112), pack-reused 204 remote: Total 379 (delta 137), reused 113 (delta 113), pack-reused 202
Receiving objects: 100% (379/379), 14.03 MiB | 27.68 MiB/s, done. Receiving objects: 100% (379/379), 14.03 MiB | 23.40 MiB/s, done.
Resolving deltas: 100% (194/194), done. Resolving deltas: 100% (195/195), done.
Cloning into 'EfficientSAM'... Cloning into 'EfficientSAM'...
remote: Enumerating objects: 424, done. remote: Enumerating objects: 424, done.
remote: Counting objects: 100% (85/85), done. remote: Counting objects: 100% (85/85), done.
remote: Compressing objects: 100% (33/33), done. remote: Compressing objects: 100% (33/33), done.
remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339 remote: Total 424 (delta 76), reused 52 (delta 52), pack-reused 339
Receiving objects: 100% (424/424), 262.14 MiB | 24.66 MiB/s, done. Receiving objects: 100% (424/424), 262.14 MiB | 23.25 MiB/s, done.
Resolving deltas: 100% (246/246), done. Resolving deltas: 100% (246/246), done.
@ -248,15 +248,6 @@ GroundingDINO imports
.. parsed-literal:: .. parsed-literal::
final text_encoder_type: bert-base-uncased final text_encoder_type: bert-base-uncased
.. parsed-literal::
FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
.. parsed-literal::
final text_encoder_type: bert-base-uncased final text_encoder_type: bert-base-uncased
@ -507,10 +498,10 @@ class, but the inference will be done using OpenVINO optimized model.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:24:34.510973: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:30:58.009326: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:24:34.551326: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:30:58.048500: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:24:35.297555: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:30:58.610355: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Convert predicted boxes to supervision box detections format Convert predicted boxes to supervision box detections format

106
docs/notebooks/hello-npu-with-output.rst
View File

@ -120,7 +120,7 @@ appear (ensure that the driver is installed successfully).
.. code:: ipython3 .. code:: ipython3
import openvino as ov import openvino as ov
core = ov.Core() core = ov.Core()
core.available_devices core.available_devices
@ -149,7 +149,7 @@ To get the value of a property, such as the device name, we can use the
.. code:: ipython3 .. code:: ipython3
device = "NPU" device = "NPU"
core.get_property(device, "FULL_DEVICE_NAME") core.get_property(device, "FULL_DEVICE_NAME")
@ -173,7 +173,7 @@ for that property.
print(f"{device} SUPPORTED_PROPERTIES:\n") print(f"{device} SUPPORTED_PROPERTIES:\n")
supported_properties = core.get_property(device, "SUPPORTED_PROPERTIES") supported_properties = core.get_property(device, "SUPPORTED_PROPERTIES")
indent = len(max(supported_properties, key=len)) indent = len(max(supported_properties, key=len))
for property_key in supported_properties: for property_key in supported_properties:
if property_key not in ("SUPPORTED_METRICS", "SUPPORTED_CONFIG_KEYS", "SUPPORTED_PROPERTIES"): if property_key not in ("SUPPORTED_METRICS", "SUPPORTED_CONFIG_KEYS", "SUPPORTED_PROPERTIES"):
try: try:
@ -233,20 +233,20 @@ Classification model from torchvision.
.. code:: ipython3 .. code:: ipython3
from pathlib import Path from pathlib import Path
# create a directory for resnet model file # create a directory for resnet model file
MODEL_DIRECTORY_PATH = Path("model") MODEL_DIRECTORY_PATH = Path("model")
MODEL_DIRECTORY_PATH.mkdir(exist_ok=True) MODEL_DIRECTORY_PATH.mkdir(exist_ok=True)
model_name = "resnet50" model_name = "resnet50"
.. code:: ipython3 .. code:: ipython3
from torchvision.models import resnet50, ResNet50_Weights from torchvision.models import resnet50, ResNet50_Weights
# create model object # create model object
pytorch_model = resnet50(weights=ResNet50_Weights.DEFAULT) pytorch_model = resnet50(weights=ResNet50_Weights.DEFAULT)
# switch model from training to inference mode # switch model from training to inference mode
pytorch_model.eval(); pytorch_model.eval();
@ -264,9 +264,9 @@ see this
.. code:: ipython3 .. code:: ipython3
precision = "FP16" precision = "FP16"
model_path = MODEL_DIRECTORY_PATH / "ir_model" / f"{model_name}_{precision.lower()}.xml" model_path = MODEL_DIRECTORY_PATH / "ir_model" / f"{model_name}_{precision.lower()}.xml"
model = None model = None
if not model_path.exists(): if not model_path.exists():
model = ov.convert_model(pytorch_model, input=[[1, 3, 224, 224]]) model = ov.convert_model(pytorch_model, input=[[1, 3, 224, 224]])
@ -349,10 +349,10 @@ To see how UMD caching see the following example:
import time import time
from pathlib import Path from pathlib import Path
start = time.time() start = time.time()
core = ov.Core() core = ov.Core()
# Compile the model as before # Compile the model as before
model = core.read_model(model=model_path) model = core.read_model(model=model_path)
compiled_model = core.compile_model(model, device) compiled_model = core.compile_model(model, device)
@ -368,7 +368,7 @@ To see how UMD caching see the following example:
start = time.time() start = time.time()
core = ov.Core() core = ov.Core()
# Compile the model once again to see UMD Caching # Compile the model once again to see UMD Caching
model = core.read_model(model=model_path) model = core.read_model(model=model_path)
compiled_model = core.compile_model(model, device) compiled_model = core.compile_model(model, device)
@ -393,24 +393,24 @@ as follow
# Create cache folder # Create cache folder
cache_folder = Path("cache") cache_folder = Path("cache")
cache_folder.mkdir(exist_ok=True) cache_folder.mkdir(exist_ok=True)
start = time.time() start = time.time()
core = ov.Core() core = ov.Core()
# Set cache folder # Set cache folder
core.set_property({"CACHE_DIR": cache_folder}) core.set_property({"CACHE_DIR": cache_folder})
# Compile the model # Compile the model
model = core.read_model(model=model_path) model = core.read_model(model=model_path)
compiled_model = core.compile_model(model, device) compiled_model = core.compile_model(model, device)
print(f"Cache enabled (first time) - compile time: {time.time() - start}s") print(f"Cache enabled (first time) - compile time: {time.time() - start}s")
start = time.time() start = time.time()
core = ov.Core() core = ov.Core()
# Set cache folder # Set cache folder
core.set_property({"CACHE_DIR": cache_folder}) core.set_property({"CACHE_DIR": cache_folder})
# Compile the model as before # Compile the model as before
model = core.read_model(model=model_path) model = core.read_model(model=model_path)
compiled_model = core.compile_model(model, device) compiled_model = core.compile_model(model, device)
@ -529,12 +529,12 @@ NPU vs CPU with Latency Hint
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] CPU [ INFO ] CPU
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
@ -577,7 +577,7 @@ NPU vs CPU with Latency Hint
[ INFO ] KV_CACHE_PRECISION: <Type: 'float16'> [ INFO ] KV_CACHE_PRECISION: <Type: 'float16'>
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 28.95 ms [ INFO ] First inference took 28.95 ms
@ -605,12 +605,12 @@ NPU vs CPU with Latency Hint
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] NPU [ INFO ] NPU
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
@ -631,20 +631,20 @@ NPU vs CPU with Latency Hint
[ INFO ] Compile model took 2302.40 ms [ INFO ] Compile model took 2302.40 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] DEVICE_ID: [ INFO ] DEVICE_ID:
[ INFO ] ENABLE_CPU_PINNING: False [ INFO ] ENABLE_CPU_PINNING: False
[ INFO ] EXECUTION_DEVICES: NPU.3720 [ INFO ] EXECUTION_DEVICES: NPU.3720
[ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'> [ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'>
[ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'} [ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'}
[ INFO ] LOADED_FROM_CACHE: False [ INFO ] LOADED_FROM_CACHE: False
[ INFO ] NETWORK_NAME: [ INFO ] NETWORK_NAME:
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1 [ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ] PERFORMANCE_HINT: PerformanceMode.LATENCY [ INFO ] PERFORMANCE_HINT: PerformanceMode.LATENCY
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1 [ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1
[ INFO ] PERF_COUNT: False [ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 7.94 ms [ INFO ] First inference took 7.94 ms
@ -681,12 +681,12 @@ time:
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] NPU [ INFO ] NPU
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
@ -707,20 +707,20 @@ time:
[ INFO ] Compile model took 2157.58 ms [ INFO ] Compile model took 2157.58 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] DEVICE_ID: [ INFO ] DEVICE_ID:
[ INFO ] ENABLE_CPU_PINNING: False [ INFO ] ENABLE_CPU_PINNING: False
[ INFO ] EXECUTION_DEVICES: NPU.3720 [ INFO ] EXECUTION_DEVICES: NPU.3720
[ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'> [ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'>
[ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'} [ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'}
[ INFO ] LOADED_FROM_CACHE: False [ INFO ] LOADED_FROM_CACHE: False
[ INFO ] NETWORK_NAME: [ INFO ] NETWORK_NAME:
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1 [ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ] PERFORMANCE_HINT: PerformanceMode.LATENCY [ INFO ] PERFORMANCE_HINT: PerformanceMode.LATENCY
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1 [ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1
[ INFO ] PERF_COUNT: False [ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 7.94 ms [ INFO ] First inference took 7.94 ms
@ -757,12 +757,12 @@ NPU vs CPU with Throughput Hint
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] CPU [ INFO ] CPU
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
@ -805,7 +805,7 @@ NPU vs CPU with Throughput Hint
[ INFO ] KV_CACHE_PRECISION: <Type: 'float16'> [ INFO ] KV_CACHE_PRECISION: <Type: 'float16'>
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 31.62 ms [ INFO ] First inference took 31.62 ms
@ -833,12 +833,12 @@ NPU vs CPU with Throughput Hint
[Step 2/11] Loading OpenVINO Runtime [Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: [ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] Device info: [ INFO ] Device info:
[ INFO ] NPU [ INFO ] NPU
[ INFO ] Build ................................. 2024.1.0-14992-621b025bef4 [ INFO ] Build ................................. 2024.1.0-14992-621b025bef4
[ INFO ] [ INFO ]
[ INFO ] [ INFO ]
[Step 3/11] Setting device configuration [Step 3/11] Setting device configuration
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
@ -859,20 +859,20 @@ NPU vs CPU with Throughput Hint
[ INFO ] Compile model took 2265.07 ms [ INFO ] Compile model took 2265.07 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] DEVICE_ID: [ INFO ] DEVICE_ID:
[ INFO ] ENABLE_CPU_PINNING: False [ INFO ] ENABLE_CPU_PINNING: False
[ INFO ] EXECUTION_DEVICES: NPU.3720 [ INFO ] EXECUTION_DEVICES: NPU.3720
[ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'> [ INFO ] INFERENCE_PRECISION_HINT: <Type: 'float16'>
[ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'} [ INFO ] INTERNAL_SUPPORTED_PROPERTIES: {'CACHING_PROPERTIES': 'RO'}
[ INFO ] LOADED_FROM_CACHE: False [ INFO ] LOADED_FROM_CACHE: False
[ INFO ] NETWORK_NAME: [ INFO ] NETWORK_NAME:
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 4 [ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 4
[ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT [ INFO ] PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1 [ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 1
[ INFO ] PERF_COUNT: False [ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'x'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'x'!. This input will be filled with random values!
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 7.95 ms [ INFO ] First inference took 7.95 ms
@ -908,13 +908,17 @@ its properties, and even tailor the model performance through the
different performance hints. different performance hints.
Discover the power of Neural Processing Unit (NPU) with OpenVINO through Discover the power of Neural Processing Unit (NPU) with OpenVINO through
these interactive Jupyter notebooks: ##### Introduction - these interactive Jupyter notebooks:
`hello-world <https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/hello-world>`__:
Start your OpenVINO journey by performing inference on an OpenVINO IR Introduction
model. - ''''''''''''
`hello-segmentation <https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/hello-segmentation>`__:
Dive into inference with a segmentation model and explore image - `hello-world <https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/hello-world>`__:
segmentation capabilities. Start your OpenVINO journey by performing inference on an OpenVINO IR
model.
- `hello-segmentation <https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/hello-segmentation>`__:
Dive into inference with a segmentation model and explore image
segmentation capabilities.
Model Optimization and Conversion Model Optimization and Conversion
''''''''''''''''''''''''''''''''' '''''''''''''''''''''''''''''''''

4
docs/notebooks/hello-segmentation-with-output.rst
View File

@ -194,7 +194,7 @@ is provided.
.. parsed-literal:: .. parsed-literal::
<matplotlib.image.AxesImage at 0x7fc77535e280> <matplotlib.image.AxesImage at 0x7f35bceac760>
@ -221,7 +221,7 @@ Do Inference
.. parsed-literal:: .. parsed-literal::
<matplotlib.image.AxesImage at 0x7fc76c77f280> <matplotlib.image.AxesImage at 0x7f3580514910>

65
docs/notebooks/hugging-face-hub-with-output.rst
View File

@ -121,7 +121,7 @@ tutorials <https://huggingface.co/learn/nlp-course/chapter2/2?fw=pt#behind-the-p
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn( warnings.warn(
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight'] Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
@ -187,7 +187,7 @@ Note how we reuse our real ``encoded_input``, passing it to the
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
@ -325,11 +325,11 @@ documentation <https://huggingface.co/docs/optimum/intel/inference>`__.
To disable this warning, you can either: To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible - Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false) - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
2024-05-16 00:26:16.053025: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:32:38.966766: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:26:16.088586: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:32:39.002435: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:26:16.708352: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:32:39.621181: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node( torch.utils._pytree._register_pytree_node(
@ -366,13 +366,13 @@ inference run.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn( warnings.warn(
Framework not specified. Using pt to export the model. Framework not specified. Using pt to export the model.
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight'] Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). - This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
Overriding 1 configuration item(s) Overriding 1 configuration item(s)
- use_cache -> False - use_cache -> False
@ -384,7 +384,7 @@ inference run.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
Compiling the model to AUTO ... Compiling the model to AUTO ...
@ -440,7 +440,7 @@ Full list of supported arguments available via ``--help``
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:26:28.476626: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:32:51.454579: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
usage: optimum-cli export openvino [-h] -m MODEL [--task TASK] usage: optimum-cli export openvino [-h] -m MODEL [--task TASK]
[--cache_dir CACHE_DIR] [--cache_dir CACHE_DIR]
[--framework {pt,tf}] [--trust-remote-code] [--framework {pt,tf}] [--trust-remote-code]
@ -449,7 +449,8 @@ Full list of supported arguments available via ``--help``
[--weight-format {fp32,fp16,int8,int4,int4_sym_g128,int4_asym_g128,int4_sym_g64,int4_asym_g64}] [--weight-format {fp32,fp16,int8,int4,int4_sym_g128,int4_asym_g128,int4_sym_g64,int4_asym_g64}]
[--ratio RATIO] [--sym] [--ratio RATIO] [--sym]
[--group-size GROUP_SIZE] [--group-size GROUP_SIZE]
[--dataset DATASET] [--disable-stateful] [--dataset DATASET] [--all-layers]
[--disable-stateful]
[--disable-convert-tokenizer] [--disable-convert-tokenizer]
[--convert-tokenizer] [--convert-tokenizer]
[--library {transformers,diffusers,timm,sentence_transformers}] [--library {transformers,diffusers,timm,sentence_transformers}]
@ -469,20 +470,21 @@ Full list of supported arguments available via ``--help``
--task TASK The task to export the model for. If not specified, --task TASK The task to export the model for. If not specified,
the task will be auto-inferred based on the model. the task will be auto-inferred based on the model.
Available tasks depend on the model, but are among: Available tasks depend on the model, but are among:
['sentence-similarity', 'fill-mask', 'semantic- ['zero-shot-object-detection', 'automatic-speech-
segmentation', 'depth-estimation', 'zero-shot-image- recognition', 'semantic-segmentation', 'text2text-
classification', 'zero-shot-object-detection', 'audio- generation', 'stable-diffusion', 'audio-
frame-classification', 'token-classification', 'image- classification', 'mask-generation', 'image-to-text',
segmentation', 'question-answering', 'text2text- 'text-to-audio', 'image-to-image', 'zero-shot-image-
generation', 'image-classification', 'text-to-audio', classification', 'token-classification', 'feature-
'stable-diffusion', 'mask-generation', 'feature- extraction', 'image-classification', 'image-
extraction', 'audio-classification', 'stable- segmentation', 'conversational', 'sentence-
diffusion-xl', 'automatic-speech-recognition', 'audio- similarity', 'audio-frame-classification', 'text-
xvector', 'text-classification', 'conversational', generation', 'text-classification', 'fill-mask',
'text-generation', 'image-to-image', 'multiple- 'stable-diffusion-xl', 'audio-xvector', 'multiple-
choice', 'object-detection', 'image-to-text', 'masked- choice', 'object-detection', 'masked-im', 'question-
im']. For decoder models, use `xxx-with-past` to answering', 'depth-estimation']. For decoder models,
export the model using past key values in the decoder. use `xxx-with-past` to export the model using past key
values in the decoder.
--cache_dir CACHE_DIR --cache_dir CACHE_DIR
Path indicating where to store cache. Path indicating where to store cache.
--framework {pt,tf} The framework to use for the export. If not provided, --framework {pt,tf} The framework to use for the export. If not provided,
@ -520,6 +522,9 @@ Full list of supported arguments available via ``--help``
['conceptual_captions','laion/220k-GPT4Vision- ['conceptual_captions','laion/220k-GPT4Vision-
captions-from-LIVIS','laion/filtered-wit'] for captions-from-LIVIS','laion/filtered-wit'] for
diffusion models. diffusion models.
--all-layers Whether embeddings and last MatMul layers should be
compressed to INT4. If not provided an weight
compression is applied, they are compressed to INT8.
--disable-stateful Disable stateful converted models, stateless models --disable-stateful Disable stateful converted models, stateless models
will be generated instead. Stateful models are will be generated instead. Stateful models are
produced by default when this key is not used. In produced by default when this key is not used. In
@ -559,21 +564,21 @@ compression:
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:26:33.213783: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:32:56.261281: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node( torch.utils._pytree._register_pytree_node(
`--fp16` option is deprecated and will be removed in a future version. Use `--weight-format` instead. `--fp16` option is deprecated and will be removed in a future version. Use `--weight-format` instead.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn( warnings.warn(
Framework not specified. Using pt to export the model. Framework not specified. Using pt to export the model.
Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight'] Some weights of the model checkpoint at cardiffnlp/twitter-roberta-base-sentiment-latest were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). - This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
Overriding 1 configuration item(s) Overriding 1 configuration item(s)
- use_cache -> False - use_cache -> False
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
OpenVINO Tokenizers is not available. To deploy models in production with C++ code, please follow installation instructions: https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#installation OpenVINO Tokenizers is not available. To deploy models in production with C++ code, please follow installation instructions: https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#installation

140
docs/notebooks/image-bind-with-output.rst
View File

File diff suppressed because one or more lines are too long

42
docs/notebooks/image-classification-quantization-with-output.rst
View File

@ -101,7 +101,7 @@ Model preparation stage has the following steps:
remote: Counting objects: 100% (281/281), done. remote: Counting objects: 100% (281/281), done.
remote: Compressing objects: 100% (96/96), done. remote: Compressing objects: 100% (96/96), done.
remote: Total 282 (delta 135), reused 269 (delta 128), pack-reused 1 remote: Total 282 (delta 135), reused 269 (delta 128), pack-reused 1
Receiving objects: 100% (282/282), 9.22 MiB | 12.60 MiB/s, done. Receiving objects: 100% (282/282), 9.22 MiB | 23.02 MiB/s, done.
Resolving deltas: 100% (135/135), done. Resolving deltas: 100% (135/135), done.
@ -173,7 +173,7 @@ Preprocessing for model obtained from training
.. parsed-literal:: .. parsed-literal::
100%|██████████| 170498071/170498071 [00:07<00:00, 22872546.75it/s] 100%|██████████| 170498071/170498071 [00:06<00:00, 24821501.62it/s]
.. parsed-literal:: .. parsed-literal::
@ -245,10 +245,10 @@ about supported parameters can be found on this
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:27:14.913330: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:33:36.759161: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:27:14.947018: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:33:36.792130: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:27:15.477320: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:33:37.462996: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
@ -431,7 +431,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 9.70 ms [ INFO ] Read model took 9.77 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [1,3,32,32] [ INFO ] x (node: x) : f32 / [...] / [1,3,32,32]
@ -445,7 +445,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10] [ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 203.56 ms [ INFO ] Compile model took 181.72 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model2 [ INFO ] NETWORK_NAME: Model2
@ -482,17 +482,17 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 2.69 ms [ INFO ] First inference took 2.97 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 88608 iterations [ INFO ] Count: 88560 iterations
[ INFO ] Duration: 15001.90 ms [ INFO ] Duration: 15003.70 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 1.85 ms [ INFO ] Median: 1.85 ms
[ INFO ] Average: 1.85 ms [ INFO ] Average: 1.85 ms
[ INFO ] Min: 1.19 ms [ INFO ] Min: 1.57 ms
[ INFO ] Max: 8.66 ms [ INFO ] Max: 9.36 ms
[ INFO ] Throughput: 5906.45 FPS [ INFO ] Throughput: 5902.54 FPS
.. code:: ipython3 .. code:: ipython3
@ -518,7 +518,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 14.43 ms [ INFO ] Read model took 14.86 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] x (node: x) : f32 / [...] / [1,3,32,32] [ INFO ] x (node: x) : f32 / [...] / [1,3,32,32]
@ -532,7 +532,7 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10] [ INFO ] x.17 (node: aten::linear/Add) : f32 / [...] / [1,10]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 331.43 ms [ INFO ] Compile model took 335.02 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model2 [ INFO ] NETWORK_NAME: Model2
@ -569,17 +569,17 @@ Tool <https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-to
[ INFO ] Fill input 'x' with random values [ INFO ] Fill input 'x' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 2.03 ms [ INFO ] First inference took 1.95 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 167064 iterations [ INFO ] Count: 166536 iterations
[ INFO ] Duration: 15001.83 ms [ INFO ] Duration: 15001.86 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 1.01 ms [ INFO ] Median: 1.01 ms
[ INFO ] Average: 1.04 ms [ INFO ] Average: 1.04 ms
[ INFO ] Min: 0.76 ms [ INFO ] Min: 0.74 ms
[ INFO ] Max: 13.10 ms [ INFO ] Max: 7.29 ms
[ INFO ] Throughput: 11136.24 FPS [ INFO ] Throughput: 11101.03 FPS
Compare results on four pictures Compare results on four pictures

17
docs/notebooks/instant-id-with-output.rst
View File

@ -50,10 +50,10 @@ additional part demonstrates how to run optimization with
`NNCF <https://github.com/openvinotoolkit/nncf/>`__ to speed up `NNCF <https://github.com/openvinotoolkit/nncf/>`__ to speed up
pipeline. pipeline.
Table of contents: Table of contents:
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
- `Prerequisites <#prerequisites>`__ - `Prerequisites <#prerequisites>`__
- `Convert and prepare Face IdentityNet <#convert-and-prepare-face-identitynet>`__ - `Convert and prepare Face IdentityNet <#convert-and-prepare-face-identitynet>`__
@ -61,8 +61,7 @@ Table of contents:
- `Perform Face Identity extraction <#perform-face-identity-extraction>`__ - `Perform Face Identity extraction <#perform-face-identity-extraction>`__
- `Prepare InstantID pipeline <#prepare-instantid-pipeline>`__ - `Prepare InstantID pipeline <#prepare-instantid-pipeline>`__
- `Convert InstantID pipeline components to OpenVINO Intermediate Representation - `Convert InstantID pipeline components to OpenVINO Intermediate Representation format <#convert-instantid-pipeline-components-to-openvino-intermediate-representation-format>`__
format <#convert-instantid-pipeline-components-to-openvino-intermediate-representation-format>`__
- `ControlNet <#controlnet>`__ - `ControlNet <#controlnet>`__
- `Unet <#unet>`__ - `Unet <#unet>`__
@ -87,8 +86,7 @@ Table of contents:
- `Run Weights Compression <#run-weights-compression>`__ - `Run Weights Compression <#run-weights-compression>`__
- `Compare model file sizes <#compare-model-file-sizes>`__ - `Compare model file sizes <#compare-model-file-sizes>`__
- `Compare inference time of the FP16 and INT8 pipelines - `Compare inference time of the FP16 and INT8 pipelines <#compare-inference-time-of-the-fp16-and-int8-pipelines>`__
<#compare-inference-time-of-the-fp16-and-int8-pipelines>`__
- `Interactive demo <#interactive-demo>`__ - `Interactive demo <#interactive-demo>`__
@ -501,9 +499,6 @@ Now, let’s see models inference result
Select Inference Device for Face Recognition Select Inference Device for Face Recognition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### Select Inference Device for
Face Recognition
.. code:: ipython3 .. code:: ipython3
import openvino as ov import openvino as ov
@ -1694,7 +1689,6 @@ Select inference device for InstantID
Create pipeline Create pipeline
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
### Create pipeline
.. code:: ipython3 .. code:: ipython3
@ -1713,7 +1707,6 @@ Create pipeline
Run inference Run inference
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
### Run inference
.. code:: ipython3 .. code:: ipython3
@ -1775,8 +1768,8 @@ improve model inference speed.
.. code:: ipython3 .. code:: ipython3
to_quantize = widgets.Checkbox(value=True, description="Quantization") skip_for_device = "GPU" in device.value
to_quantize = widgets.Checkbox(value=not skip_for_device, description="Quantization", disabled=skip_for_device)
to_quantize to_quantize

22
docs/notebooks/knowledge-graphs-conve-with-output.rst
View File

@ -222,7 +222,7 @@ Download Model Checkpoint
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/knowledge-graphs-conve/models/conve.pt') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/knowledge-graphs-conve/models/conve.pt')
@ -384,7 +384,7 @@ typical to use metrics such as Mean Reciprocal Rank, Hits@10 etc.
.. parsed-literal:: .. parsed-literal::
Average time taken for inference: 0.6809830665588379 ms Average time taken for inference: 0.6391008694966634 ms
Mean accuracy of the model on the test dataset: 0.875 Mean accuracy of the model on the test dataset: 0.875
@ -531,7 +531,7 @@ select device from dropdown list for running inference using OpenVINO
.. parsed-literal:: .. parsed-literal::
Average time taken for inference: 0.6866355737050375 ms Average time taken for inference: 0.7032553354899088 ms
Mean accuracy of the model on the test dataset: 0.10416666666666667 Mean accuracy of the model on the test dataset: 0.10416666666666667
@ -550,7 +550,7 @@ Determine the platform specific speedup obtained through OpenVINO graph optimiza
.. parsed-literal:: .. parsed-literal::
Speedup with OpenVINO optimizations: 0.99 X Speedup with OpenVINO optimizations: 0.91 X
Benchmark the converted OpenVINO model using benchmark app Benchmark the converted OpenVINO model using benchmark app
@ -595,7 +595,7 @@ inference can also be obtained by looking at the benchmark app results.
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 5.32 ms [ INFO ] Read model took 5.04 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] e1 (node: e1) : i64 / [...] / [] [ INFO ] e1 (node: e1) : i64 / [...] / []
@ -611,7 +611,7 @@ inference can also be obtained by looking at the benchmark app results.
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] ***NO_NAME*** (node: aten::softmax/Softmax) : f32 / [...] / [1,271] [ INFO ] ***NO_NAME*** (node: aten::softmax/Softmax) : f32 / [...] / [1,271]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 66.93 ms [ INFO ] Compile model took 64.51 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -641,17 +641,17 @@ inference can also be obtained by looking at the benchmark app results.
[ INFO ] Fill input 'rel' with random values [ INFO ] Fill input 'rel' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 10000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 12 inference requests, limits: 10000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 1.44 ms [ INFO ] First inference took 1.24 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 100644 iterations [ INFO ] Count: 100548 iterations
[ INFO ] Duration: 10001.37 ms [ INFO ] Duration: 10001.41 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 1.02 ms [ INFO ] Median: 1.02 ms
[ INFO ] Average: 1.03 ms [ INFO ] Average: 1.03 ms
[ INFO ] Min: 0.65 ms [ INFO ] Min: 0.65 ms
[ INFO ] Max: 8.70 ms [ INFO ] Max: 8.40 ms
[ INFO ] Throughput: 10063.02 FPS [ INFO ] Throughput: 10053.38 FPS
Conclusions Conclusions

28
docs/notebooks/kosmos2-multimodal-large-language-model-with-output.rst
View File

@ -66,7 +66,7 @@ Install requirements
.. parsed-literal:: .. parsed-literal::
Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0) Requirement already satisfied: pip in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (24.0)
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -133,14 +133,10 @@ example <https://huggingface.co/microsoft/kosmos-2-patch14-224>`__
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:29:11.534432: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:35:34.439294: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:29:11.569240: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:35:34.473205: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:29:12.061235: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:35:34.967434: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
@ -360,11 +356,11 @@ Vision model accept ``pixel_values`` and returns ``image_embeds``.
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:469: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:466: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len): if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:509: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:506: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim): if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
@ -392,7 +388,7 @@ Convert Image To Text Projection model
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.) /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
if a.grad is not None: if a.grad is not None:
@ -527,13 +523,13 @@ generated text by ``AutoProcessor``.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:808: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:805: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if max_pos > self.weights.size(0): if max_pos > self.weights.size(0):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1117: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1114: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if input_shape[-1] > 1: if input_shape[-1] > 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:924: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:921: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attention_mask.size() != (batch_size, 1, seq_length, src_len): if attention_mask.size() != (batch_size, 1, seq_length, src_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1210: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/kosmos2/modeling_kosmos2.py:1207: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if past_key_values_length > 0: if past_key_values_length > 0:

4
docs/notebooks/kosmos2-multimodal-large-language-model-with-output_files/kosmos2-multimodal-large-language-model-with-output_29_0.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:55f4e723ec2322d6837bbe04816e3b48ef5ab3446204ac5194544233f059f30f oid sha256:cf1443764f4f5d14409c30f1c2e1cc52862f9d279d4ec76371894162beb45353
size 119957 size 121469

4
docs/notebooks/kosmos2-multimodal-large-language-model-with-output_files/kosmos2-multimodal-large-language-model-with-output_29_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:e15ceb0ae875340a0d9ce3082a02fcafcbe810e86863cbb69f14fccce49e97e5 oid sha256:4d7ff778d8a6131d02d03afc15c4ff72695e117e44e9e41d0cd1ade176d07190
size 1151174 size 1150902

4
docs/notebooks/kosmos2-multimodal-large-language-model-with-output_files/kosmos2-multimodal-large-language-model-with-output_8_0.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:6d515c7b0d0e34137797eb090964a6ba4b2fb724bbd3003c8c31c7bd0b7add6f oid sha256:57dd2c11fbd048e2e211c2caccadf63b3b844a99eb21d7585d3ab8b15e0d577e
size 122393 size 123612

4
docs/notebooks/kosmos2-multimodal-large-language-model-with-output_files/kosmos2-multimodal-large-language-model-with-output_8_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:e3ae8247baa4ef9148ed9c3a6a9bfc9b732f976b19042f530c34cb7f8cf5bd36 oid sha256:a6f20b1ac6e439ce83cdf50c7a44d13853072fb966d5ca52e1a83279be35b44a
size 1150593 size 1150696

88
docs/notebooks/language-quantize-bert-with-output.rst
View File

@ -90,10 +90,10 @@ Imports
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:30:29.627218: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:36:52.607166: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:30:29.661916: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:36:52.641403: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:30:30.261372: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:36:53.239122: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
.. parsed-literal:: .. parsed-literal::
@ -191,16 +191,7 @@ PyTorch model formats are supported:
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
.. parsed-literal::
WARNING:nncf:NNCF provides best results with torch==2.2.*, while current torch version is 2.3.0+cpu. If you encounter issues, consider switching to torch==2.2.*
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
@ -313,7 +304,6 @@ The optimization process contains the following steps:
.. parsed-literal:: .. parsed-literal::
INFO:nncf:36 ignored nodes were found by name in the NNCFGraph
INFO:nncf:50 ignored nodes were found by name in the NNCFGraph INFO:nncf:50 ignored nodes were found by name in the NNCFGraph
@ -532,9 +522,9 @@ Frames Per Second (FPS) for images.
.. parsed-literal:: .. parsed-literal::
PyTorch model on CPU: 0.071 seconds per sentence, SPS: 14.02 PyTorch model on CPU: 0.068 seconds per sentence, SPS: 14.61
IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 48.10 IR FP32 model in OpenVINO Runtime/AUTO: 0.021 seconds per sentence, SPS: 47.63
OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.009 seconds per sentence, SPS: 108.86 OpenVINO IR INT8 model in OpenVINO Runtime/AUTO: 0.010 seconds per sentence, SPS: 103.25
Finally, measure the inference performance of OpenVINO ``FP32`` and Finally, measure the inference performance of OpenVINO ``FP32`` and
@ -575,27 +565,27 @@ in OpenVINO.
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 18.56 ms [ INFO ] Read model took 19.23 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?] [ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?]
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,?] [ INFO ] 63 , attention_mask (node: attention_mask) : i64 / [...] / [1,?]
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,?] [ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2] [ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
[Step 5/11] Resizing model to match image sizes and given batch [Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1 [ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'input_ids': [1,128], '36': [1,128], 'token_type_ids': [1,128] [ INFO ] Reshaping model: 'input_ids': [1,128], '63': [1,128], 'token_type_ids': [1,128]
[ INFO ] Reshape model took 5.68 ms [ INFO ] Reshape model took 10.36 ms
[Step 6/11] Configuring input of the model [Step 6/11] Configuring input of the model
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128] [ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128]
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,128] [ INFO ] 63 , attention_mask (node: attention_mask) : i64 / [...] / [1,128]
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,128] [ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,128]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2] [ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 351.77 ms [ INFO ] Compile model took 362.31 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -629,24 +619,24 @@ in OpenVINO.
[ INFO ] PERF_COUNT: False [ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'input_ids'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'input_ids'!. This input will be filled with random values!
[ WARNING ] No input files were given for input '36'!. This input will be filled with random values! [ WARNING ] No input files were given for input '63'!. This input will be filled with random values!
[ WARNING ] No input files were given for input 'token_type_ids'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'token_type_ids'!. This input will be filled with random values!
[ INFO ] Fill input 'input_ids' with random values [ INFO ] Fill input 'input_ids' with random values
[ INFO ] Fill input '36' with random values [ INFO ] Fill input '63' with random values
[ INFO ] Fill input 'token_type_ids' with random values [ INFO ] Fill input 'token_type_ids' with random values
[Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration) [Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 21.49 ms [ INFO ] First inference took 21.08 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 6147 iterations [ INFO ] Count: 6222 iterations
[ INFO ] Duration: 120009.26 ms [ INFO ] Duration: 120015.61 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 19.24 ms [ INFO ] Median: 19.16 ms
[ INFO ] Average: 19.43 ms [ INFO ] Average: 19.20 ms
[ INFO ] Min: 18.60 ms [ INFO ] Min: 18.52 ms
[ INFO ] Max: 22.12 ms [ INFO ] Max: 23.65 ms
[ INFO ] Throughput: 51.22 FPS [ INFO ] Throughput: 51.84 FPS
.. code:: ipython3 .. code:: ipython3
@ -673,27 +663,27 @@ in OpenVINO.
[ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY. [ WARNING ] Performance hint was not explicitly specified in command line. Device(AUTO) performance hint will be set to PerformanceMode.LATENCY.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 24.21 ms [ INFO ] Read model took 25.23 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?] [ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,?]
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,?] [ INFO ] attention_mask , 63 (node: attention_mask) : i64 / [...] / [1,?]
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,?] [ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,?]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2] [ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
[Step 5/11] Resizing model to match image sizes and given batch [Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1 [ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'input_ids': [1,128], '36': [1,128], 'token_type_ids': [1,128] [ INFO ] Reshaping model: 'input_ids': [1,128], '63': [1,128], 'token_type_ids': [1,128]
[ INFO ] Reshape model took 7.31 ms [ INFO ] Reshape model took 12.77 ms
[Step 6/11] Configuring input of the model [Step 6/11] Configuring input of the model
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128] [ INFO ] input_ids (node: input_ids) : i64 / [...] / [1,128]
[ INFO ] attention_mask , 36 (node: attention_mask) : i64 / [...] / [1,128] [ INFO ] attention_mask , 63 (node: attention_mask) : i64 / [...] / [1,128]
[ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,128] [ INFO ] token_type_ids (node: token_type_ids) : i64 / [...] / [1,128]
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2] [ INFO ] logits (node: __module.classifier/aten::linear/Add) : f32 / [...] / [1,2]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 1079.78 ms [ INFO ] Compile model took 1122.18 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: Model0 [ INFO ] NETWORK_NAME: Model0
@ -727,22 +717,22 @@ in OpenVINO.
[ INFO ] PERF_COUNT: False [ INFO ] PERF_COUNT: False
[Step 9/11] Creating infer requests and preparing input tensors [Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'input_ids'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'input_ids'!. This input will be filled with random values!
[ WARNING ] No input files were given for input '36'!. This input will be filled with random values! [ WARNING ] No input files were given for input '63'!. This input will be filled with random values!
[ WARNING ] No input files were given for input 'token_type_ids'!. This input will be filled with random values! [ WARNING ] No input files were given for input 'token_type_ids'!. This input will be filled with random values!
[ INFO ] Fill input 'input_ids' with random values [ INFO ] Fill input 'input_ids' with random values
[ INFO ] Fill input '36' with random values [ INFO ] Fill input '63' with random values
[ INFO ] Fill input 'token_type_ids' with random values [ INFO ] Fill input 'token_type_ids' with random values
[Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration) [Step 10/11] Measuring performance (Start inference synchronously, limits: 120000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 15.94 ms [ INFO ] First inference took 18.38 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 12483 iterations [ INFO ] Count: 11981 iterations
[ INFO ] Duration: 120004.49 ms [ INFO ] Duration: 120000.10 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 9.59 ms [ INFO ] Median: 10.28 ms
[ INFO ] Average: 9.52 ms [ INFO ] Average: 9.93 ms
[ INFO ] Min: 8.18 ms [ INFO ] Min: 8.10 ms
[ INFO ] Max: 11.10 ms [ INFO ] Max: 11.42 ms
[ INFO ] Throughput: 104.02 FPS [ INFO ] Throughput: 99.84 FPS

11
docs/notebooks/latent-consistency-models-image-generation-with-output.rst
View File

@ -839,12 +839,8 @@ improve model inference speed.
.. code:: ipython3 .. code:: ipython3
to_quantize = widgets.Checkbox( skip_for_device = "GPU" in device.value
value=True, to_quantize = widgets.Checkbox(value=not skip_for_device, description="Quantization", disabled=skip_for_device)
description="Quantization",
disabled=False,
)
to_quantize to_quantize
@ -863,9 +859,6 @@ Let’s load ``skip magic`` extension to skip quantization if
int8_pipe = None int8_pipe = None
if to_quantize.value and "GPU" in device.value:
to_quantize.value = False
# Fetch `skip_kernel_extension` module # Fetch `skip_kernel_extension` module
import requests import requests

14
docs/notebooks/latent-consistency-models-optimum-demo-with-output.rst
View File

@ -121,10 +121,10 @@ https://huggingface.co/docs/diffusers/en/api/pipelines/latent_consistency_models
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:36:40.510949: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:42:53.373765: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:36:40.546299: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:42:53.408053: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:36:41.041474: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:42:53.902704: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
@ -235,7 +235,7 @@ and there is no need to do it manually
.. parsed-literal:: .. parsed-literal::
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
.. parsed-literal:: .. parsed-literal::
@ -246,9 +246,9 @@ and there is no need to do it manually
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu

4
docs/notebooks/latent-consistency-models-optimum-demo-with-output_files/latent-consistency-models-optimum-demo-with-output_15_1.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:0bd7464e563caf1e06e0c1cbd3a3469c284a6e9d7372f2c8d1828f9df001abe0 oid sha256:ede98f62d1d27162d7d604073e9f8de2fb5d951d493981d40fed6cde05723048
size 29753 size 33866

4
docs/notebooks/latent-consistency-models-optimum-demo-with-output_files/latent-consistency-models-optimum-demo-with-output_15_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:ca79e22459b6d83468505496f54f4f227721878ffec5293e1564c3332bf8e439 oid sha256:9010559959f775533c438237a49d197fa05cab0e22c7ed546dba031a90a5b4eb
size 429952 size 441523

4
docs/notebooks/latent-consistency-models-optimum-demo-with-output_files/latent-consistency-models-optimum-demo-with-output_8_1.jpg
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:27e0663303ea64ceb8505c4d59c5c86a6afb20e2ff2a10166b0c47ccff2bc52b oid sha256:9246fabbebb05dc5ecc65e4f4e61c860fe49e4b175a3efdcbd4351689720b594
size 33705 size 38529

4
docs/notebooks/latent-consistency-models-optimum-demo-with-output_files/latent-consistency-models-optimum-demo-with-output_8_1.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:5ee2f1136215766a50f4539f33ba7ac6040d545fcf22c7f80dca523bffdc92f9 oid sha256:c5809256a03aff9396dc484356572bdc0a532c31a43a3a25ee663ca04899068b
size 442908 size 463403

8
docs/notebooks/lcm-lora-controlnet-with-output.rst
View File

@ -1366,12 +1366,8 @@ improve model inference speed.
.. code:: ipython3 .. code:: ipython3
is_gpu_device = "GPU" in device.value skip_for_device = "GPU" in device.value
to_quantize = widgets.Checkbox( to_quantize = widgets.Checkbox(value=not skip_for_device, description="Quantization", disabled=skip_for_device)
value=not is_gpu_device,
description="Quantization",
disabled=is_gpu_device,
)
to_quantize to_quantize

68
docs/notebooks/legacy-mo-convert-to-openvino-with-output.rst
View File

@ -119,7 +119,7 @@ documentation.
conversion into IR. The legacy Frontend is Python conversion into IR. The legacy Frontend is Python
based and is available for TensorFlow*, ONNX*, MXNet*, based and is available for TensorFlow*, ONNX*, MXNet*,
Caffe*, and Kaldi* models. Caffe*, and Kaldi* models.
--input_model INPUT_MODEL, -w INPUT_MODEL, -m INPUT_MODEL --input_model INPUT_MODEL, -m INPUT_MODEL, -w INPUT_MODEL
Tensorflow*: a file with a pre-trained model (binary Tensorflow*: a file with a pre-trained model (binary
or text .pb file after freezing). Caffe*: a model or text .pb file after freezing). Caffe*: a model
proto file with model weights. proto file with model weights.
@ -735,13 +735,11 @@ NLP model from Hugging Face and export it in ONNX format:
.. parsed-literal:: .. parsed-literal::
2024-05-15 23:49:58.968544: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-05 23:49:21.145736: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-15 23:49:59.004440: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-05 23:49:21.179983: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-15 23:49:59.646592: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-05 23:49:21.824078: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:231: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/distilbert/modeling_distilbert.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
mask, torch.tensor(torch.finfo(scores.dtype).min) mask, torch.tensor(torch.finfo(scores.dtype).min)
@ -1004,8 +1002,8 @@ To convert a model to OpenVINO IR, use the following command:
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. code:: ipython3 .. code:: ipython3
@ -1100,8 +1098,8 @@ guide <https://docs.openvino.ai/2024/openvino-workflow/model-preparation/setting
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. parsed-literal:: .. parsed-literal::
@ -1119,8 +1117,8 @@ guide <https://docs.openvino.ai/2024/openvino-workflow/model-preparation/setting
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. code:: ipython3 .. code:: ipython3
@ -1168,8 +1166,8 @@ sequence length dimension for inputs:
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. code:: ipython3 .. code:: ipython3
@ -1213,8 +1211,8 @@ dimension:
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. code:: ipython3 .. code:: ipython3
@ -1281,8 +1279,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. parsed-literal:: .. parsed-literal::
@ -1300,8 +1298,8 @@ guide <https://docs.openvino.ai/2023.3/openvino_docs_MO_DG_prepare_model_convert
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/distilbert.bin
.. code:: ipython3 .. code:: ipython3
@ -1376,8 +1374,8 @@ Resnet50 model that was exported to the ONNX format:
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. code:: ipython3 .. code:: ipython3
@ -1422,8 +1420,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. parsed-literal:: .. parsed-literal::
@ -1441,8 +1439,8 @@ presented by input data. Use either ``layout`` or ``source_layout`` with
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. code:: ipython3 .. code:: ipython3
@ -1491,8 +1489,8 @@ that the preprocessing takes negligible time for inference.
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. parsed-literal:: .. parsed-literal::
@ -1510,8 +1508,8 @@ that the preprocessing takes negligible time for inference.
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. code:: ipython3 .. code:: ipython3
@ -1557,8 +1555,8 @@ the color channels before inference.
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. code:: ipython3 .. code:: ipython3
@ -1602,8 +1600,8 @@ models, this decrease is negligible.
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/convert-to-openvino/model/resnet.bin
.. code:: ipython3 .. code:: ipython3

250
docs/notebooks/llava-next-multimodal-chatbot-with-output.rst
View File

@ -69,12 +69,12 @@ Prerequisites
.. code:: ipython3 .. code:: ipython3
from pathlib import Path from pathlib import Path
MODEL_DIR = Path("model") MODEL_DIR = Path("model")
IMAGE_ENCODER_PATH = MODEL_DIR / "image_encoder.xml" IMAGE_ENCODER_PATH = MODEL_DIR / "image_encoder.xml"
INPUT_EMBEDDING_PATH = MODEL_DIR / "input_embeddings.xml" INPUT_EMBEDDING_PATH = MODEL_DIR / "input_embeddings.xml"
LANGUAGE_MODEL_PATH = MODEL_DIR / "language_model.xml" LANGUAGE_MODEL_PATH = MODEL_DIR / "language_model.xml"
requires_pt_model_loading = not all([p.exists() for p in [IMAGE_ENCODER_PATH, INPUT_EMBEDDING_PATH, LANGUAGE_MODEL_PATH]]) requires_pt_model_loading = not all([p.exists() for p in [IMAGE_ENCODER_PATH, INPUT_EMBEDDING_PATH, LANGUAGE_MODEL_PATH]])
Download PyTorch model Download PyTorch model
@ -87,18 +87,18 @@ Download PyTorch model
from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
import torch import torch
import gc import gc
processor = LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf") processor = LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf")
image_encoder_model, input_embedding_model, language_model = None, None, None image_encoder_model, input_embedding_model, language_model = None, None, None
class ImageEncoder(torch.nn.Module): class ImageEncoder(torch.nn.Module):
def __init__(self, config, vision_tower, multi_modal_projector): def __init__(self, config, vision_tower, multi_modal_projector):
super().__init__() super().__init__()
self.config = config self.config = config
self.vision_tower = vision_tower self.vision_tower = vision_tower
self.multi_modal_projector = multi_modal_projector self.multi_modal_projector = multi_modal_projector
def forward(self, pixel_values): def forward(self, pixel_values):
batch_size, num_patches, num_channels, height, width = pixel_values.shape batch_size, num_patches, num_channels, height, width = pixel_values.shape
reshaped_pixel_values = pixel_values.view(batch_size * num_patches, num_channels, height, width) reshaped_pixel_values = pixel_values.view(batch_size * num_patches, num_channels, height, width)
@ -110,8 +110,8 @@ Download PyTorch model
selected_image_feature = selected_image_feature selected_image_feature = selected_image_feature
image_features = self.multi_modal_projector(selected_image_feature) image_features = self.multi_modal_projector(selected_image_feature)
return image_features return image_features
if requires_pt_model_loading: if requires_pt_model_loading:
model = LlavaNextForConditionalGeneration.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", low_cpu_mem_usage=True) model = LlavaNextForConditionalGeneration.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", low_cpu_mem_usage=True)
model.config.save_pretrained(MODEL_DIR) model.config.save_pretrained(MODEL_DIR)
@ -136,8 +136,8 @@ Download PyTorch model
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
OpenVINO## Convert model to OpenVINO Intermediate Representation `back Convert model to OpenVINO Intermediate Representation
to top ⬆️ <#Table-of-contents:>`__ -----------------------------------------------------
OpenVINO supports PyTorch models via conversion to OpenVINO Intermediate OpenVINO supports PyTorch models via conversion to OpenVINO Intermediate
Representation (IR). `OpenVINO model conversion Representation (IR). `OpenVINO model conversion
@ -208,8 +208,8 @@ Image Encoder is represented in LLaVA by pretrained CLIP model.
import torch import torch
import openvino as ov import openvino as ov
import gc import gc
def cleanup_torchscript_cache(): def cleanup_torchscript_cache():
""" """
Helper for removing cached model representation Helper for removing cached model representation
@ -217,14 +217,14 @@ Image Encoder is represented in LLaVA by pretrained CLIP model.
torch._C._jit_clear_class_registry() torch._C._jit_clear_class_registry()
torch.jit._recursive.concrete_type_store = torch.jit._recursive.ConcreteTypeStore() torch.jit._recursive.concrete_type_store = torch.jit._recursive.ConcreteTypeStore()
torch.jit._state._clear_class_state() torch.jit._state._clear_class_state()
if not IMAGE_ENCODER_PATH.exists(): if not IMAGE_ENCODER_PATH.exists():
ov_image_encoder = ov.convert_model(image_encoder_model, example_input=torch.zeros((1, 5, 3, 336, 336))) ov_image_encoder = ov.convert_model(image_encoder_model, example_input=torch.zeros((1, 5, 3, 336, 336)))
ov.save_model(ov_image_encoder, IMAGE_ENCODER_PATH) ov.save_model(ov_image_encoder, IMAGE_ENCODER_PATH)
del ov_image_encoder del ov_image_encoder
cleanup_torchscript_cache() cleanup_torchscript_cache()
del image_encoder_model del image_encoder_model
gc.collect(); gc.collect();
@ -242,16 +242,16 @@ use it separately.
.. code:: ipython3 .. code:: ipython3
llm_input = None llm_input = None
if not LANGUAGE_MODEL_PATH.exists(): if not LANGUAGE_MODEL_PATH.exists():
llm_input = input_embedding_model(torch.ones((2, 2), dtype=torch.int64)) llm_input = input_embedding_model(torch.ones((2, 2), dtype=torch.int64))
if not INPUT_EMBEDDING_PATH.exists(): if not INPUT_EMBEDDING_PATH.exists():
ov_input_embeddings_model = ov.convert_model(input_embedding_model, example_input=torch.ones((2, 2), dtype=torch.int64)) ov_input_embeddings_model = ov.convert_model(input_embedding_model, example_input=torch.ones((2, 2), dtype=torch.int64))
ov.save_model(ov_input_embeddings_model, INPUT_EMBEDDING_PATH) ov.save_model(ov_input_embeddings_model, INPUT_EMBEDDING_PATH)
del ov_input_embeddings_model del ov_input_embeddings_model
cleanup_torchscript_cache() cleanup_torchscript_cache()
del input_embedding_model del input_embedding_model
gc.collect(); gc.collect();
@ -302,28 +302,28 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
from typing import Optional, Tuple, List from typing import Optional, Tuple, List
from openvino.runtime import opset13 from openvino.runtime import opset13
import numpy as np import numpy as np
def model_has_state(ov_model: ov.Model): def model_has_state(ov_model: ov.Model):
# TODO: Provide a better way based on the variables availability, but OV Python API doesn't expose required methods # TODO: Provide a better way based on the variables availability, but OV Python API doesn't expose required methods
return len(ov_model.get_sinks()) > 0 return len(ov_model.get_sinks()) > 0
def model_has_input_output_name(ov_model: ov.Model, name: str): def model_has_input_output_name(ov_model: ov.Model, name: str):
""" """
Helper function for checking that model has specified input or output name Helper function for checking that model has specified input or output name
Parameters: Parameters:
ov_model (ov.Model): # TODO: Can we derive the dimensions from the model topology? ov_model (ov.Model): # TODO: Can we derive the dimensions from the model topology?
name (str): name (str):
name of input or output name of input or output
Returns: Returns:
True if input or output with requested name exists else False True if input or output with requested name exists else False
""" """
return name in sum([list(t.get_names()) for t in ov_model.inputs + ov_model.outputs], []) return name in sum([list(t.get_names()) for t in ov_model.inputs + ov_model.outputs], [])
def fuse_cache_reorder( def fuse_cache_reorder(
ov_model: ov.Model, ov_model: ov.Model,
not_kv_inputs: List[str], not_kv_inputs: List[str],
@ -332,14 +332,14 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
): ):
""" """
Fuses reored_cache during generate cycle into ov.Model. Used with stateful models, because we can not modify model state directly. Fuses reored_cache during generate cycle into ov.Model. Used with stateful models, because we can not modify model state directly.
Adds a new beam_idx parameter and Gather op per each kv-cache input in a given model. Adds a new beam_idx parameter and Gather op per each kv-cache input in a given model.
Should be run before make_stateful. Implements optimumum's _reorder_cache Should be run before make_stateful. Implements optimumum's _reorder_cache
inside the model in the beginning of each iteration. inside the model in the beginning of each iteration.
Gather works along given gather_dim dimension that may vary from model to model. Gather works along given gather_dim dimension that may vary from model to model.
KV-cache inputs are identified based on names in key_value_input_names. KV-cache inputs are identified based on names in key_value_input_names.
Append the new beam_idx parameter to not_kv_inputs. Append the new beam_idx parameter to not_kv_inputs.
Parameters: Parameters:
ov_model (`ov.Model`): ov_model (`ov.Model`):
openvino model for processing openvino model for processing
@ -350,7 +350,7 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
gather_dim (int): gather_dim (int):
dimension for gathering cache during reorder pass dimension for gathering cache during reorder pass
""" """
if model_has_input_output_name(ov_model, "beam_idx"): if model_has_input_output_name(ov_model, "beam_idx"):
raise ValueError("Model already has fused cache") raise ValueError("Model already has fused cache")
input_batch = ov_model.input("inputs_embeds").get_partial_shape()[0] input_batch = ov_model.input("inputs_embeds").get_partial_shape()[0]
@ -366,12 +366,12 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
for consumer in consumers: for consumer in consumers:
consumer.replace_source_output(gather.output(0)) consumer.replace_source_output(gather.output(0))
ov_model.validate_nodes_and_infer_types() ov_model.validate_nodes_and_infer_types()
def build_state_initializer(ov_model: ov.Model, batch_dim: int): def build_state_initializer(ov_model: ov.Model, batch_dim: int):
""" """
Build initialization ShapeOf Expression for all ReadValue ops Build initialization ShapeOf Expression for all ReadValue ops
Parameters: Parameters:
ov_model (ov.Model): ov_model (ov.Model):
openvino model openvino model
@ -393,8 +393,8 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
broadcast = opset13.broadcast(opset13.constant(0.0, dtype=op.get_output_element_type(0)), shape) broadcast = opset13.broadcast(opset13.constant(0.0, dtype=op.get_output_element_type(0)), shape)
op.set_arguments([broadcast]) op.set_arguments([broadcast])
ov_model.validate_nodes_and_infer_types() ov_model.validate_nodes_and_infer_types()
def make_stateful( def make_stateful(
ov_model: ov.Model, ov_model: ov.Model,
not_kv_inputs: List[str], not_kv_inputs: List[str],
@ -406,7 +406,7 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
): ):
""" """
Hides kv-cache inputs and outputs inside the model as variables. Hides kv-cache inputs and outputs inside the model as variables.
Parameters: Parameters:
ov_model (ov.Model): ov_model (ov.Model):
openvino model openvino model
@ -424,9 +424,9 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
precalculated number of beams and batch for shapes initialization precalculated number of beams and batch for shapes initialization
""" """
from openvino._offline_transformations import apply_make_stateful_transformation from openvino._offline_transformations import apply_make_stateful_transformation
input_output_map = {} input_output_map = {}
if num_beams_and_batch is not None: if num_beams_and_batch is not None:
# Set batch size for input_ids and attention mask to avoid dynamic dimension got propagated from the end of the model back to ReadValue # Set batch size for input_ids and attention mask to avoid dynamic dimension got propagated from the end of the model back to ReadValue
for input in not_kv_inputs: for input in not_kv_inputs:
@ -441,16 +441,16 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
shape = input.get_partial_shape() shape = input.get_partial_shape()
shape[batch_dim] = num_beams_and_batch * num_attention_heads shape[batch_dim] = num_beams_and_batch * num_attention_heads
input.get_node().set_partial_shape(shape) input.get_node().set_partial_shape(shape)
if num_beams_and_batch is not None: if num_beams_and_batch is not None:
# Re-validation model if shapes are altered above # Re-validation model if shapes are altered above
ov_model.validate_nodes_and_infer_types() ov_model.validate_nodes_and_infer_types()
apply_make_stateful_transformation(ov_model, input_output_map) apply_make_stateful_transformation(ov_model, input_output_map)
if num_beams_and_batch is None: if num_beams_and_batch is None:
build_state_initializer(ov_model, batch_dim) build_state_initializer(ov_model, batch_dim)
def patch_stateful(ov_model): def patch_stateful(ov_model):
key_value_input_names = [key.get_any_name() for key in ov_model.inputs[2:-1]] key_value_input_names = [key.get_any_name() for key in ov_model.inputs[2:-1]]
key_value_output_names = [key.get_any_name() for key in ov_model.outputs[1:]] key_value_output_names = [key.get_any_name() for key in ov_model.outputs[1:]]
@ -459,7 +459,7 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
return return
batch_dim = 0 batch_dim = 0
num_attention_heads = 1 num_attention_heads = 1
fuse_cache_reorder(ov_model, not_kv_inputs, key_value_input_names, batch_dim) fuse_cache_reorder(ov_model, not_kv_inputs, key_value_input_names, batch_dim)
make_stateful( make_stateful(
ov_model, ov_model,
@ -475,7 +475,7 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
make_stateful_model = True make_stateful_model = True
core = ov.Core() core = ov.Core()
if not LANGUAGE_MODEL_PATH.exists(): if not LANGUAGE_MODEL_PATH.exists():
pkv = language_model(inputs_embeds=llm_input, attention_mask=torch.ones((2, 2), dtype=torch.int64))[1] pkv = language_model(inputs_embeds=llm_input, attention_mask=torch.ones((2, 2), dtype=torch.int64))[1]
model_inputs = ["attention_mask", "position_ids"] model_inputs = ["attention_mask", "position_ids"]
@ -495,10 +495,10 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/running-inference
"position_ids": position_ids, "position_ids": position_ids,
}, },
) )
for input, input_name in zip(ov_model.inputs, model_inputs): for input, input_name in zip(ov_model.inputs, model_inputs):
input.get_tensor().set_names({input_name}) input.get_tensor().set_names({input_name})
for output, output_name in zip(ov_model.outputs, model_outputs): for output, output_name in zip(ov_model.outputs, model_outputs):
output.get_tensor().set_names({output_name}) output.get_tensor().set_names({output_name})
if make_stateful_model: if make_stateful_model:
@ -556,13 +556,13 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/model-optimizatio
.. code:: ipython3 .. code:: ipython3
import ipywidgets as widgets import ipywidgets as widgets
to_compress_weights = widgets.Checkbox( to_compress_weights = widgets.Checkbox(
value=True, value=True,
description="Weights Compression", description="Weights Compression",
disabled=False, disabled=False,
) )
to_compress_weights to_compress_weights
@ -577,13 +577,13 @@ documentation <https://docs.openvino.ai/2024/openvino-workflow/model-optimizatio
.. code:: ipython3 .. code:: ipython3
import nncf import nncf
compression_configuration = { compression_configuration = {
"mode": nncf.CompressWeightsMode.INT4_SYM, "mode": nncf.CompressWeightsMode.INT4_SYM,
"group_size": 64, "group_size": 64,
"ratio": 0.6, "ratio": 0.6,
} }
LANGUAGE_MODEL_PATH_INT4 = LANGUAGE_MODEL_PATH.parent / LANGUAGE_MODEL_PATH.name.replace(".xml", "-int4.xml") LANGUAGE_MODEL_PATH_INT4 = LANGUAGE_MODEL_PATH.parent / LANGUAGE_MODEL_PATH.name.replace(".xml", "-int4.xml")
if to_compress_weights.value and not LANGUAGE_MODEL_PATH_INT4.exists(): if to_compress_weights.value and not LANGUAGE_MODEL_PATH_INT4.exists():
ov_model = core.read_model(LANGUAGE_MODEL_PATH) ov_model = core.read_model(LANGUAGE_MODEL_PATH)
@ -631,7 +631,7 @@ inference faster. The optimization process contains the following steps:
description="Quantization", description="Quantization",
disabled=False, disabled=False,
) )
to_quantize to_quantize
@ -646,15 +646,15 @@ inference faster. The optimization process contains the following steps:
.. code:: ipython3 .. code:: ipython3
IMAGE_ENCODER_PATH_INT8 = IMAGE_ENCODER_PATH.parent / IMAGE_ENCODER_PATH.name.replace(".xml", "-int8.xml") IMAGE_ENCODER_PATH_INT8 = IMAGE_ENCODER_PATH.parent / IMAGE_ENCODER_PATH.name.replace(".xml", "-int8.xml")
import requests import requests
r = requests.get( r = requests.get(
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py", url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/skip_kernel_extension.py",
) )
open("skip_kernel_extension.py", "w").write(r.text) open("skip_kernel_extension.py", "w").write(r.text)
%load_ext skip_kernel_extension %load_ext skip_kernel_extension
Prepare datasets Prepare datasets
@ -670,15 +670,15 @@ model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import requests import requests
from io import BytesIO from io import BytesIO
import numpy as np import numpy as np
from PIL import Image from PIL import Image
from requests.packages.urllib3.exceptions import InsecureRequestWarning from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning) requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
def get_pil_from_url(url): def get_pil_from_url(url):
""" """
Downloads and converts an image from a URL to a PIL Image object. Downloads and converts an image from a URL to a PIL Image object.
@ -686,7 +686,7 @@ model.
response = requests.get(url, verify=False, timeout=20) response = requests.get(url, verify=False, timeout=20)
image = Image.open(BytesIO(response.content)) image = Image.open(BytesIO(response.content))
return image.convert("RGB") return image.convert("RGB")
def collate_fn(example, image_column="image_url"): def collate_fn(example, image_column="image_url"):
""" """
Preprocesses an example by loading and transforming image and text data. Preprocesses an example by loading and transforming image and text data.
@ -705,18 +705,18 @@ model.
return None return None
except Exception: except Exception:
return None return None
inputs = processor.image_processor(images=[image], return_tensors="pt") inputs = processor.image_processor(images=[image], return_tensors="pt")
return inputs return inputs
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
import torch import torch
from datasets import load_dataset from datasets import load_dataset
from tqdm.notebook import tqdm from tqdm.notebook import tqdm
def prepare_calibration_data(dataloader, init_steps): def prepare_calibration_data(dataloader, init_steps):
""" """
This function prepares calibration data from a dataloader for a specified number of initialization steps. This function prepares calibration data from a dataloader for a specified number of initialization steps.
@ -737,8 +737,8 @@ model.
} }
) )
return data return data
def prepare_dataset(opt_init_steps=50, max_train_samples=1000): def prepare_dataset(opt_init_steps=50, max_train_samples=1000):
""" """
Prepares a vision-text dataset for quantization. Prepares a vision-text dataset for quantization.
@ -752,7 +752,7 @@ model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
vcalibration_data = [] vcalibration_data = []
if not IMAGE_ENCODER_PATH_INT8.exists(): if not IMAGE_ENCODER_PATH_INT8.exists():
calibration_data = prepare_dataset() calibration_data = prepare_dataset()
@ -770,14 +770,14 @@ Create a quantized model from the pre-trained model.
.. code:: ipython3 .. code:: ipython3
%%skip not $to_quantize.value %%skip not $to_quantize.value
if not IMAGE_ENCODER_PATH_INT8.exists(): if not IMAGE_ENCODER_PATH_INT8.exists():
if len(calibration_data) == 0: if len(calibration_data) == 0:
raise RuntimeError( raise RuntimeError(
'Calibration dataset is empty. Please check internet connection and try to download images manually.' 'Calibration dataset is empty. Please check internet connection and try to download images manually.'
) )
ov_model = core.read_model(IMAGE_ENCODER_PATH) ov_model = core.read_model(IMAGE_ENCODER_PATH)
calibration_dataset = nncf.Dataset(calibration_data) calibration_dataset = nncf.Dataset(calibration_data)
quantized_model = nncf.quantize( quantized_model = nncf.quantize(
@ -821,8 +821,8 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
unpad_image, unpad_image,
) )
import openvino as ov import openvino as ov
class OVLlavaForCausalLM(GenerationMixin): class OVLlavaForCausalLM(GenerationMixin):
def __init__( def __init__(
self, self,
@ -851,11 +851,11 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
self.image_newline = torch.zeros(self.config.text_config.hidden_size, dtype=torch.float32) self.image_newline = torch.zeros(self.config.text_config.hidden_size, dtype=torch.float32)
self.pad_token_id = self.config.pad_token_id if self.config.pad_token_id is not None else -1 self.pad_token_id = self.config.pad_token_id if self.config.pad_token_id is not None else -1
self.past_len = 0 self.past_len = 0
def can_generate(self): def can_generate(self):
"""Returns True to validate the check that the model using `GenerationMixin.generate()` can indeed generate.""" """Returns True to validate the check that the model using `GenerationMixin.generate()` can indeed generate."""
return True return True
def __call__( def __call__(
self, self,
input_ids: torch.LongTensor, input_ids: torch.LongTensor,
@ -875,7 +875,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
image_sizes, image_sizes,
**kwargs, **kwargs,
) )
def forward( def forward(
self, self,
input_ids: torch.LongTensor, input_ids: torch.LongTensor,
@ -900,49 +900,49 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
# inputs["attention_mask"] = attention_mask # inputs["attention_mask"] = attention_mask
if "beam_idx" in self.input_names: if "beam_idx" in self.input_names:
inputs["beam_idx"] = self.next_beam_idx if self.next_beam_idx is not None else np.arange(batch_size, dtype=int) inputs["beam_idx"] = self.next_beam_idx if self.next_beam_idx is not None else np.arange(batch_size, dtype=int)
if not self.stateful: if not self.stateful:
first_layer_past_key_value = torch.from_numpy(past_key_values[0][0][:, :, :, 0]) first_layer_past_key_value = torch.from_numpy(past_key_values[0][0][:, :, :, 0])
else: else:
first_layer_past_key_value = torch.from_numpy(self.request.query_state()[0].state.data[:, :, :, 0]) first_layer_past_key_value = torch.from_numpy(self.request.query_state()[0].state.data[:, :, :, 0])
# Sum all dimensions of head_dim (-2) to avoid random errors such as: https://github.com/huggingface/transformers/pull/28032#issuecomment-1863691941 # Sum all dimensions of head_dim (-2) to avoid random errors such as: https://github.com/huggingface/transformers/pull/28032#issuecomment-1863691941
batch_index, non_attended_tokens = torch.where(first_layer_past_key_value.float().sum(-2) == 0) batch_index, non_attended_tokens = torch.where(first_layer_past_key_value.float().sum(-2) == 0)
# Get the target length # Get the target length
target_length = input_ids.shape[1] target_length = input_ids.shape[1]
past_length = first_layer_past_key_value.shape[-1] past_length = first_layer_past_key_value.shape[-1]
extended_attention_mask = torch.ones( extended_attention_mask = torch.ones(
(attention_mask.shape[0], past_length), (attention_mask.shape[0], past_length),
dtype=attention_mask.dtype, dtype=attention_mask.dtype,
device=attention_mask.device, device=attention_mask.device,
) )
# Filter out only the tokens that can be un-attended, this can happen # Filter out only the tokens that can be un-attended, this can happen
# if one uses Llava + Fused modules where the cache on the # if one uses Llava + Fused modules where the cache on the
# first iteration is already big enough, or if one passes custom cache # first iteration is already big enough, or if one passes custom cache
valid_indices = non_attended_tokens < extended_attention_mask.size(-1) valid_indices = non_attended_tokens < extended_attention_mask.size(-1)
new_batch_index = batch_index[valid_indices] new_batch_index = batch_index[valid_indices]
new_non_attended_tokens = non_attended_tokens[valid_indices] new_non_attended_tokens = non_attended_tokens[valid_indices]
# Zero-out the places where we don't need to attend # Zero-out the places where we don't need to attend
extended_attention_mask[new_batch_index, new_non_attended_tokens] = 0 extended_attention_mask[new_batch_index, new_non_attended_tokens] = 0
attention_mask = torch.cat((extended_attention_mask, attention_mask[:, -target_length:]), dim=1) attention_mask = torch.cat((extended_attention_mask, attention_mask[:, -target_length:]), dim=1)
position_ids = torch.sum(attention_mask, dim=1).unsqueeze(-1) - 1 position_ids = torch.sum(attention_mask, dim=1).unsqueeze(-1) - 1
inputs["attention_mask"] = attention_mask inputs["attention_mask"] = attention_mask
inputs["position_ids"] = position_ids inputs["position_ids"] = position_ids
else: else:
inputs = self.prepare_multimodal_input(input_ids, pixel_values, attention_mask, position_ids, image_sizes) inputs = self.prepare_multimodal_input(input_ids, pixel_values, attention_mask, position_ids, image_sizes)
# Run inference # Run inference
self.request.start_async(inputs, share_inputs=True) self.request.start_async(inputs, share_inputs=True)
self.request.wait() self.request.wait()
logits = torch.from_numpy(self.request.get_tensor(self.output_names[0]).data) logits = torch.from_numpy(self.request.get_tensor(self.output_names[0]).data)
if not self.stateful: if not self.stateful:
# Tuple of length equal to : number of layer * number of past_key_value per decoder layer (2 corresponds to the self-attention layer) # Tuple of length equal to : number of layer * number of past_key_value per decoder layer (2 corresponds to the self-attention layer)
past_key_values = tuple(self.request.get_tensor(key).data for key in self.key_value_output_names) past_key_values = tuple(self.request.get_tensor(key).data for key in self.key_value_output_names)
@ -952,7 +952,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
past_key_values = ((),) past_key_values = ((),)
self.past_len += inputs["inputs_embeds"].shape[1] self.past_len += inputs["inputs_embeds"].shape[1]
return CausalLMOutputWithPast(logits=logits, past_key_values=past_key_values) return CausalLMOutputWithPast(logits=logits, past_key_values=past_key_values)
def prepare_multimodal_input(self, input_ids, pixel_values, attention_mask, position_ids, image_sizes=None): def prepare_multimodal_input(self, input_ids, pixel_values, attention_mask, position_ids, image_sizes=None):
"""Preprocessing function for embedding multimodal data""" """Preprocessing function for embedding multimodal data"""
inputs = {} inputs = {}
@ -974,7 +974,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
# Set initial value for the next beam_idx input that will be used at the current iteration # Set initial value for the next beam_idx input that will be used at the current iteration
# and will be optionally updated by _reorder_cache at the next iterations if beam_search is used # and will be optionally updated by _reorder_cache at the next iterations if beam_search is used
self.next_beam_idx = np.arange(batch_size, dtype=int) self.next_beam_idx = np.arange(batch_size, dtype=int)
if "beam_idx" in self.input_names: if "beam_idx" in self.input_names:
inputs["beam_idx"] = self.next_beam_idx if self.next_beam_idx is not None else np.arange(batch_size, dtype=int) inputs["beam_idx"] = self.next_beam_idx if self.next_beam_idx is not None else np.arange(batch_size, dtype=int)
if pixel_values is None: if pixel_values is None:
@ -988,16 +988,16 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
image_features = torch.from_numpy(res[0]) image_features = torch.from_numpy(res[0])
split_sizes = [image.shape[0] for image in pixel_values] split_sizes = [image.shape[0] for image in pixel_values]
image_features = torch.split(image_features, split_sizes, dim=0) image_features = torch.split(image_features, split_sizes, dim=0)
# NOTE we only support multimodal_patch_merge_type == "spatial_unpad" # NOTE we only support multimodal_patch_merge_type == "spatial_unpad"
height = width = self.config.vision_config.image_size // self.config.vision_config.patch_size height = width = self.config.vision_config.image_size // self.config.vision_config.patch_size
new_image_features = [] new_image_features = []
for image_idx, image_feature in enumerate(image_features): for image_idx, image_feature in enumerate(image_features):
if image_feature.shape[0] > 1: if image_feature.shape[0] > 1:
base_image_feature = image_feature[0] base_image_feature = image_feature[0]
image_feature = image_feature[1:] image_feature = image_feature[1:]
if height * width != base_image_feature.shape[0]: if height * width != base_image_feature.shape[0]:
raise ValueError("The number of patches is not consistent with the image size.") raise ValueError("The number of patches is not consistent with the image size.")
num_patch_height, num_patch_width = get_anyres_image_grid_shape( num_patch_height, num_patch_width = get_anyres_image_grid_shape(
@ -1023,7 +1023,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
image_feature = torch.cat((image_feature, self.image_newline[None]), dim=0) image_feature = torch.cat((image_feature, self.image_newline[None]), dim=0)
new_image_features.append(image_feature) new_image_features.append(image_feature)
image_features = torch.stack(new_image_features, dim=0) image_features = torch.stack(new_image_features, dim=0)
( (
inputs_embeds, inputs_embeds,
attention_mask, attention_mask,
@ -1032,9 +1032,9 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
inputs["inputs_embeds"] = inputs_embeds inputs["inputs_embeds"] = inputs_embeds
inputs["attention_mask"] = attention_mask inputs["attention_mask"] = attention_mask
inputs["position_ids"] = position_ids inputs["position_ids"] = position_ids
return inputs return inputs
def _merge_input_ids_with_image_features(self, image_features, inputs_embeds, input_ids, attention_mask, labels): def _merge_input_ids_with_image_features(self, image_features, inputs_embeds, input_ids, attention_mask, labels):
num_images, num_image_patches, embed_dim = image_features.shape num_images, num_image_patches, embed_dim = image_features.shape
batch_size, sequence_length = input_ids.shape batch_size, sequence_length = input_ids.shape
@ -1045,7 +1045,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
# Compute the maximum embed dimension # Compute the maximum embed dimension
max_embed_dim = (num_special_image_tokens.max() * (num_image_patches - 1)) + sequence_length max_embed_dim = (num_special_image_tokens.max() * (num_image_patches - 1)) + sequence_length
batch_indices, non_image_indices = torch.where(input_ids != self.config.image_token_index) batch_indices, non_image_indices = torch.where(input_ids != self.config.image_token_index)
# 2. Compute the positions where text should be written # 2. Compute the positions where text should be written
# Calculate new positions for text tokens in merged image-text sequence. # Calculate new positions for text tokens in merged image-text sequence.
# `special_image_token_mask` identifies image tokens. Each image token will be replaced by `nb_text_tokens_per_images - 1` text tokens. # `special_image_token_mask` identifies image tokens. Each image token will be replaced by `nb_text_tokens_per_images - 1` text tokens.
@ -1056,7 +1056,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
if left_padding: if left_padding:
new_token_positions += nb_image_pad[:, None] # offset for left padding new_token_positions += nb_image_pad[:, None] # offset for left padding
text_to_overwrite = new_token_positions[batch_indices, non_image_indices] text_to_overwrite = new_token_positions[batch_indices, non_image_indices]
# 3. Create the full embedding, already padded to the maximum position # 3. Create the full embedding, already padded to the maximum position
final_embedding = torch.zeros( final_embedding = torch.zeros(
batch_size, batch_size,
@ -1080,14 +1080,14 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
text_to_overwrite.to(target_device), text_to_overwrite.to(target_device),
) )
attention_mask = attention_mask.to(target_device) attention_mask = attention_mask.to(target_device)
# 4. Fill the embeddings based on the mask. If we have ["hey" "<image>", "how", "are"] # 4. Fill the embeddings based on the mask. If we have ["hey" "<image>", "how", "are"]
# we need to index copy on [0, 577, 578, 579] for the text and [1:576] for the image features # we need to index copy on [0, 577, 578, 579] for the text and [1:576] for the image features
final_embedding[batch_indices, text_to_overwrite] = inputs_embeds[batch_indices, non_image_indices] final_embedding[batch_indices, text_to_overwrite] = inputs_embeds[batch_indices, non_image_indices]
final_attention_mask[batch_indices, text_to_overwrite] = attention_mask[batch_indices, non_image_indices] final_attention_mask[batch_indices, text_to_overwrite] = attention_mask[batch_indices, non_image_indices]
if labels is not None: if labels is not None:
final_labels[batch_indices, text_to_overwrite] = labels[batch_indices, non_image_indices] final_labels[batch_indices, text_to_overwrite] = labels[batch_indices, non_image_indices]
# 5. Fill the embeddings corresponding to the images. Anything that is still zeros needs filling # 5. Fill the embeddings corresponding to the images. Anything that is still zeros needs filling
image_to_overwrite = torch.all(final_embedding == 0, dim=-1) image_to_overwrite = torch.all(final_embedding == 0, dim=-1)
image_to_overwrite &= image_to_overwrite.cumsum(-1) - 1 >= nb_image_pad[:, None].to(target_device) image_to_overwrite &= image_to_overwrite.cumsum(-1) - 1 >= nb_image_pad[:, None].to(target_device)
@ -1096,19 +1096,19 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
f"The input provided to the model are wrong. The number of image tokens is {torch.sum(special_image_token_mask)} while" f"The input provided to the model are wrong. The number of image tokens is {torch.sum(special_image_token_mask)} while"
f" the number of image given to the model is {num_images}. This prevents correct indexing and breaks batch generation." f" the number of image given to the model is {num_images}. This prevents correct indexing and breaks batch generation."
) )
final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device) final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device)
final_attention_mask |= image_to_overwrite final_attention_mask |= image_to_overwrite
position_ids = (final_attention_mask.cumsum(-1) - 1).masked_fill_((final_attention_mask == 0), 1) position_ids = (final_attention_mask.cumsum(-1) - 1).masked_fill_((final_attention_mask == 0), 1)
# 6. Mask out the embedding at padding positions, as we later use the past_key_value value to determine the non-attended tokens. # 6. Mask out the embedding at padding positions, as we later use the past_key_value value to determine the non-attended tokens.
batch_indices, pad_indices = torch.where(input_ids == self.pad_token_id) batch_indices, pad_indices = torch.where(input_ids == self.pad_token_id)
indices_to_mask = new_token_positions[batch_indices, pad_indices] indices_to_mask = new_token_positions[batch_indices, pad_indices]
final_embedding[batch_indices, indices_to_mask] = 0 final_embedding[batch_indices, indices_to_mask] = 0
return final_embedding, final_attention_mask, position_ids return final_embedding, final_attention_mask, position_ids
def prepare_inputs_for_generation( def prepare_inputs_for_generation(
self, self,
input_ids, input_ids,
@ -1124,7 +1124,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
cache_length = past_length = past_key_values[0][0].shape[2] cache_length = past_length = past_key_values[0][0].shape[2]
else: else:
cache_length = past_length = self.past_len cache_length = past_length = self.past_len
# Keep only the unprocessed tokens: # Keep only the unprocessed tokens:
# 1 - If the length of the attention_mask exceeds the length of input_ids, then we are in a setting where # 1 - If the length of the attention_mask exceeds the length of input_ids, then we are in a setting where
# some of the inputs are exclusively passed as part of the cache (e.g. when passing input_embeds as # some of the inputs are exclusively passed as part of the cache (e.g. when passing input_embeds as
@ -1142,7 +1142,7 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
# older attention values, as their corresponding values are not part of the input. # older attention values, as their corresponding values are not part of the input.
if cache_length < past_length and attention_mask is not None: if cache_length < past_length and attention_mask is not None:
attention_mask = attention_mask[:, -(cache_length + input_ids.shape[1]) :] attention_mask = attention_mask[:, -(cache_length + input_ids.shape[1]) :]
position_ids = kwargs.get("position_ids", None) position_ids = kwargs.get("position_ids", None)
if attention_mask is not None and position_ids is None: if attention_mask is not None and position_ids is None:
# create position_ids on the fly for batch gllavaenerationsubset_siz # create position_ids on the fly for batch gllavaenerationsubset_siz
@ -1150,13 +1150,13 @@ documentation <https://huggingface.co/docs/transformers/main_classes/text_genera
position_ids.masked_fill_(attention_mask == 0, 1) position_ids.masked_fill_(attention_mask == 0, 1)
if past_key_values: if past_key_values:
position_ids = position_ids[:, -input_ids.shape[1] :] position_ids = position_ids[:, -input_ids.shape[1] :]
# if `inputs_embeds` are passed, we only want to use them in the 1st generation step # if `inputs_embeds` are passed, we only want to use them in the 1st generation step
if inputs_embeds is not None and past_key_values is None: if inputs_embeds is not None and past_key_values is None:
model_inputs = {"inputs_embeds": inputs_embeds} model_inputs = {"inputs_embeds": inputs_embeds}
else: else:
model_inputs = {"input_ids": input_ids} model_inputs = {"input_ids": input_ids}
model_inputs.update( model_inputs.update(
{ {
"position_ids": position_ids, "position_ids": position_ids,
@ -1182,18 +1182,18 @@ Select device
.. code:: ipython3 .. code:: ipython3
core = ov.Core() core = ov.Core()
support_devices = core.available_devices support_devices = core.available_devices
if "NPU" in support_devices: if "NPU" in support_devices:
support_devices.remove("NPU") support_devices.remove("NPU")
device = widgets.Dropdown( device = widgets.Dropdown(
options=support_devices + ["AUTO"], options=support_devices + ["AUTO"],
value="CPU", value="CPU",
description="Device:", description="Device:",
disabled=False, disabled=False,
) )
device device
@ -1212,7 +1212,7 @@ Select device
description="INT4 language model", description="INT4 language model",
disabled=not LANGUAGE_MODEL_PATH_INT4.exists(), disabled=not LANGUAGE_MODEL_PATH_INT4.exists(),
) )
use_int4_lang_model use_int4_lang_model
@ -1231,7 +1231,7 @@ Select device
description="INT8 image encoder", description="INT8 image encoder",
disabled=not IMAGE_ENCODER_PATH_INT8.exists(), disabled=not IMAGE_ENCODER_PATH_INT8.exists(),
) )
use_int8_image_encoder use_int8_image_encoder
@ -1247,23 +1247,23 @@ Select device
lang_model_path = LANGUAGE_MODEL_PATH_INT4 if use_int4_lang_model.value else LANGUAGE_MODEL_PATH lang_model_path = LANGUAGE_MODEL_PATH_INT4 if use_int4_lang_model.value else LANGUAGE_MODEL_PATH
image_encoder_path = IMAGE_ENCODER_PATH_INT8 if use_int8_image_encoder.value else IMAGE_ENCODER_PATH image_encoder_path = IMAGE_ENCODER_PATH_INT8 if use_int8_image_encoder.value else IMAGE_ENCODER_PATH
ov_llava_model = OVLlavaForCausalLM(core, image_encoder_path, INPUT_EMBEDDING_PATH, lang_model_path, device.value) ov_llava_model = OVLlavaForCausalLM(core, image_encoder_path, INPUT_EMBEDDING_PATH, lang_model_path, device.value)
.. code:: ipython3 .. code:: ipython3
from PIL import Image from PIL import Image
import requests import requests
from transformers import TextStreamer from transformers import TextStreamer
url = "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11" url = "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11"
image = Image.open(requests.get(url, stream=True).raw) image = Image.open(requests.get(url, stream=True).raw)
question = "What is unusual on this image?" question = "What is unusual on this image?"
prompt = f"[INST] <image>\n{question}[/INST]" prompt = f"[INST] <image>\n{question}[/INST]"
streamer = TextStreamer(processor, skip_special_tokens=True, skip_prompt=True) streamer = TextStreamer(processor, skip_special_tokens=True, skip_prompt=True)
inputs = processor(prompt, image, return_tensors="pt") inputs = processor(prompt, image, return_tensors="pt")
print(f"Question:\n{question}") print(f"Question:\n{question}")
image image
@ -1311,7 +1311,7 @@ Interactive demo
from threading import Thread from threading import Thread
from PIL import Image from PIL import Image
import torch import torch
example_image_urls = [ example_image_urls = [
( (
"https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/1d6a0188-5613-418d-a1fd-4560aae1d907", "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/1d6a0188-5613-418d-a1fd-4560aae1d907",
@ -1324,8 +1324,8 @@ Interactive demo
] ]
for url, file_name in example_image_urls: for url, file_name in example_image_urls:
Image.open(requests.get(url, stream=True).raw).save(file_name) Image.open(requests.get(url, stream=True).raw).save(file_name)
def bot_streaming(message, history): def bot_streaming(message, history):
print(message) print(message)
if message["files"]: if message["files"]:
@ -1336,28 +1336,28 @@ Interactive demo
for hist in history: for hist in history:
if isinstance(hist[0], tuple): if isinstance(hist[0], tuple):
image = hist[0][0] image = hist[0][0]
if image is None: if image is None:
gr.Error("You need to upload an image for LLaVA to work.") gr.Error("You need to upload an image for LLaVA to work.")
prompt = f"[INST] <image>\n{message['text']} [/INST]" prompt = f"[INST] <image>\n{message['text']} [/INST]"
image = Image.open(image).convert("RGB") image = Image.open(image).convert("RGB")
inputs = processor(prompt, image, return_tensors="pt") inputs = processor(prompt, image, return_tensors="pt")
streamer = TextIteratorStreamer(processor, **{"skip_special_tokens": True}) streamer = TextIteratorStreamer(processor, **{"skip_special_tokens": True})
generation_kwargs = dict(inputs, streamer=streamer, max_new_tokens=100) generation_kwargs = dict(inputs, streamer=streamer, max_new_tokens=100)
thread = Thread(target=ov_llava_model.generate, kwargs=generation_kwargs) thread = Thread(target=ov_llava_model.generate, kwargs=generation_kwargs)
thread.start() thread.start()
text_prompt = f"[INST] \n{message['text']} [/INST]" text_prompt = f"[INST] \n{message['text']} [/INST]"
buffer = "" buffer = ""
for new_text in streamer: for new_text in streamer:
buffer += new_text buffer += new_text
generated_text_without_prompt = buffer[len(text_prompt) :] generated_text_without_prompt = buffer[len(text_prompt) :]
yield generated_text_without_prompt yield generated_text_without_prompt
demo = gr.ChatInterface( demo = gr.ChatInterface(
fn=bot_streaming, fn=bot_streaming,
title="LLaVA NeXT", title="LLaVA NeXT",
@ -1369,7 +1369,7 @@ Interactive demo
stop_btn="Stop Generation", stop_btn="Stop Generation",
multimodal=True, multimodal=True,
) )
try: try:
demo.launch(debug=False) demo.launch(debug=False)
except Exception: except Exception:

11
docs/notebooks/llm-agent-langchain-with-output.rst
View File

@ -54,16 +54,21 @@ Prerequisites
.. code:: ipython3 .. code:: ipython3
%pip uninstall -q -y openvino-dev openvino openvino-nightly optimum optimum-intel import os
os.environ["GIT_CLONE_PROTECTION_ACTIVE"] = "false"
%pip install -Uq pip
%pip uninstall -q -y optimum optimum-intel
%pip install --pre -Uq openvino openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\ %pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\
"git+https://github.com/huggingface/optimum-intel.git"\ "git+https://github.com/huggingface/optimum-intel.git"\
"git+https://github.com/openvinotoolkit/nncf.git"\ "git+https://github.com/openvinotoolkit/nncf.git"\
"torch>=2.1"\ "torch>=2.1"\
"datasets"\ "datasets"\
"accelerate"\ "accelerate"\
"openvino-nightly"\
"gradio"\ "gradio"\
"transformers>=4.38.1" "langchain>=0.1.14" "wikipedia" "transformers>=4.38.1" "langchain>=0.2.0" "langchain-community>=0.2.0" "wikipedia"
Create a tools Create a tools
-------------- --------------

35
docs/notebooks/llm-chatbot-with-output.rst
View File

@ -65,14 +65,19 @@ Install required dependencies
.. code:: ipython3 .. code:: ipython3
%pip uninstall -q -y openvino-dev openvino openvino-nightly optimum optimum-intel import os
os.environ["GIT_CLONE_PROTECTION_ACTIVE"] = "false"
%pip install -Uq pip
%pip uninstall -q -y optimum optimum-intel
%pip install --pre -Uq openvino openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\ %pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\
"git+https://github.com/huggingface/optimum-intel.git"\ "git+https://github.com/huggingface/optimum-intel.git"\
"git+https://github.com/openvinotoolkit/nncf.git"\ "git+https://github.com/openvinotoolkit/nncf.git"\
"torch>=2.1"\ "torch>=2.1"\
"datasets" \ "datasets" \
"accelerate"\ "accelerate"\
"openvino-nightly"\
"gradio>=4.19"\ "gradio>=4.19"\
"onnx" "einops" "transformers_stream_generator" "tiktoken" "transformers>=4.38.1" "bitsandbytes" "onnx" "einops" "transformers_stream_generator" "tiktoken" "transformers>=4.38.1" "bitsandbytes"
@ -81,6 +86,7 @@ Install required dependencies
import os import os
from pathlib import Path from pathlib import Path
import requests import requests
import shutil
# fetch model configuration # fetch model configuration
@ -89,7 +95,18 @@ Install required dependencies
if not config_dst_path.exists(): if not config_dst_path.exists():
if config_shared_path.exists(): if config_shared_path.exists():
os.symlink(config_shared_path, config_dst_path) try:
os.symlink(config_shared_path, config_dst_path)
except Exception:
shutil.copy(config_shared_path, config_dst_path)
else:
r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py")
with open("llm_config.py", "w") as f:
f.write(r.text)
elif not os.path.islink(config_dst_path):
print("LLM config will be updated")
if config_shared_path.exists():
shutil.copy(config_shared_path, config_dst_path)
else: else:
r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py") r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py")
with open("llm_config.py", "w") as f: with open("llm_config.py", "w") as f:
@ -436,7 +453,7 @@ Convert model using Optimum-CLI tool
`Optimum Intel <https://huggingface.co/docs/optimum/intel/index>`__ is `Optimum Intel <https://huggingface.co/docs/optimum/intel/index>`__ is
the interface between the the interface between the
`Transformers <https://huggingface.co/docs/transformers/index>`__ and `Transformers <https://huggingface.co/docs/transformers/index>`__ and
`Diffusers <https://huggingface.co/docs/diffusers/index>`__ libraries `Diffusers <https://huggingface.co/docs/diffusers/index>`__ libraries
and OpenVINO to accelerate end-to-end pipelines on Intel architectures. and OpenVINO to accelerate end-to-end pipelines on Intel architectures.
@ -487,11 +504,10 @@ to make it
`symmetric <https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Quantization.md#symmetric-quantization>`__ `symmetric <https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Quantization.md#symmetric-quantization>`__
you can add ``--sym``. you can add ``--sym``.
For INT4 quantization you can also specify the following arguments : For INT4 quantization you can also specify the following arguments : -
The ``--group-size`` parameter will define the group size to use for
- The ``--group-size`` parameter will define the group size to use for quantization, -1 it will results in per-column quantization. - The
quantization, -1 it will results in per-column quantization. ``--ratio`` parameter controls the ratio between 4-bit and 8-bit
- The ``--ratio`` parameter controls the ratio between 4-bit and 8-bit
quantization. If set to 0.9, it means that 90% of the layers will be quantization. If set to 0.9, it means that 90% of the layers will be
quantized to int4 while 10% will be quantized to int8. quantized to int4 while 10% will be quantized to int8.
@ -1344,7 +1360,6 @@ answers.https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide.html
.. code:: ipython3 .. code:: ipython3
# please uncomment and run this cell for stopping gradio interface # please uncomment and run this cell for stopping gradio interface

6
docs/notebooks/llm-question-answering-with-output.rst
View File

@ -66,8 +66,10 @@ Prerequisites
.. code:: ipython3 .. code:: ipython3
%pip uninstall -q -y openvino openvino-dev openvino-nightly optimum optimum-intel %pip install -Uq pip
%pip install -q "torch>=2.1" openvino-nightly "nncf>=2.7" "transformers>=4.36.0" onnx "optimum>=1.16.1" "accelerate" "datasets>=2.14.6" "gradio>=4.19" "git+https://github.com/huggingface/optimum-intel.git" --extra-index-url https://download.pytorch.org/whl/cpu %pip uninstall -q -y optimum optimum-intel
%pip install --pre -Uq openvino openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q "torch>=2.1" "nncf>=2.7" "transformers>=4.36.0" onnx "optimum>=1.16.1" "accelerate" "datasets>=2.14.6" "gradio>=4.19" "git+https://github.com/huggingface/optimum-intel.git" --extra-index-url https://download.pytorch.org/whl/cpu
Select model for inference Select model for inference
-------------------------- --------------------------

74
docs/notebooks/llm-rag-langchain-with-output.rst
View File

@ -81,21 +81,25 @@ Install required dependencies
.. code:: ipython3 .. code:: ipython3
%pip uninstall -q -y openvino-dev openvino openvino-nightly optimum optimum-intel import os
os.environ["GIT_CLONE_PROTECTION_ACTIVE"] = "false"
%pip install -Uq pip
%pip uninstall -q -y optimum optimum-intel
%pip install --pre -Uq openvino openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\ %pip install -q --extra-index-url https://download.pytorch.org/whl/cpu\
"git+https://github.com/huggingface/optimum-intel.git"\ "git+https://github.com/huggingface/optimum-intel.git"\
"git+https://github.com/openvinotoolkit/nncf.git"\ "git+https://github.com/openvinotoolkit/nncf.git"\
"datasets"\ "datasets"\
"accelerate"\ "accelerate"\
"openvino-nightly"\
"gradio"\ "gradio"\
"onnx" "einops" "transformers_stream_generator" "tiktoken" "transformers>=4.38.1" "bitsandbytes" "chromadb" "sentence_transformers" "langchain>=0.1.15" "langchainhub" "unstructured" "scikit-learn" "python-docx" "pypdf" "onnx" "einops" "transformers_stream_generator" "tiktoken" "transformers>=4.38.1" "bitsandbytes" "faiss-cpu" "sentence_transformers" "langchain>=0.2.0" "langchain-community>=0.2.0" "langchainhub" "unstructured" "scikit-learn" "python-docx" "pypdf"
.. parsed-literal:: .. parsed-literal::
WARNING: Skipping openvino-dev as it is not installed. WARNING: Skipping openvino-dev as it is not installed.
WARNING: Skipping openvino as it is not installed.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -105,6 +109,7 @@ Install required dependencies
import os import os
from pathlib import Path from pathlib import Path
import requests import requests
import shutil
import io import io
# fetch model configuration # fetch model configuration
@ -118,11 +123,23 @@ Install required dependencies
if not config_dst_path.exists(): if not config_dst_path.exists():
if config_shared_path.exists(): if config_shared_path.exists():
os.symlink(config_shared_path, config_dst_path) try:
os.symlink(config_shared_path, config_dst_path)
except Exception:
shutil.copy(config_shared_path, config_dst_path)
else: else:
r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py") r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py")
with open("llm_config.py", "w") as f: with open("llm_config.py", "w") as f:
f.write(r.text) f.write(r.text)
elif not os.path.islink(config_dst_path):
print("LLM config will be updated")
if config_shared_path.exists():
shutil.copy(config_shared_path, config_dst_path)
else:
r = requests.get(url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/llm_config.py")
with open("llm_config.py", "w") as f:
f.write(r.text)
if not text_example_en_path.exists(): if not text_example_en_path.exists():
r = requests.get(url=text_example_en) r = requests.get(url=text_example_en)
@ -768,14 +785,14 @@ of LangChain.
.. parsed-literal:: .. parsed-literal::
2024-04-28 21:05:33.318682: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-05-24 00:13:06.057342: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-28 21:05:33.322370: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2024-05-24 00:13:06.061389: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-04-28 21:05:33.366644: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-05-24 00:13:06.108453: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-04-28 21:05:33.366676: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-05-24 00:13:06.108490: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-04-28 21:05:33.366714: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-05-24 00:13:06.108542: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-04-28 21:05:33.376052: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-05-24 00:13:06.120406: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-28 21:05:34.068587: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-05-24 00:13:06.938926: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Compiling the model to CPU ... Compiling the model to CPU ...
@ -910,7 +927,6 @@ inference framework.
.. parsed-literal:: .. parsed-literal::
Compiling the model to CPU ... Compiling the model to CPU ...
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
@ -1065,7 +1081,7 @@ which will help to create a chain to connect RAG components including:
.. code:: ipython3 .. code:: ipython3
from langchain.prompts import PromptTemplate from langchain.prompts import PromptTemplate
from langchain.vectorstores import Chroma from langchain_community.vectorstores import FAISS
from langchain.chains.retrieval import create_retrieval_chain from langchain.chains.retrieval import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.docstore.document import Document from langchain.docstore.document import Document
@ -1074,6 +1090,7 @@ which will help to create a chain to connect RAG components including:
import gradio as gr import gradio as gr
stop_tokens = llm_model_configuration.get("stop_tokens") stop_tokens = llm_model_configuration.get("stop_tokens")
rag_prompt_template = llm_model_configuration["rag_prompt_template"]
class StopOnTokens(StoppingCriteria): class StopOnTokens(StoppingCriteria):
@ -1151,7 +1168,7 @@ which will help to create a chain to connect RAG components including:
texts = text_splitter.split_documents(documents) texts = text_splitter.split_documents(documents)
global db global db
db = Chroma.from_documents(texts, embedding) db = FAISS.from_documents(texts, embedding)
global retriever global retriever
if search_method == "similarity_score_threshold": if search_method == "similarity_score_threshold":
@ -1162,7 +1179,7 @@ which will help to create a chain to connect RAG components including:
if run_rerank: if run_rerank:
reranker.top_n = vector_search_top_n reranker.top_n = vector_search_top_n
retriever = ContextualCompressionRetriever(base_compressor=reranker, base_retriever=retriever) retriever = ContextualCompressionRetriever(base_compressor=reranker, base_retriever=retriever)
prompt = PromptTemplate.from_template(llm_model_configuration["rag_prompt_template"]) prompt = PromptTemplate.from_template(rag_prompt_template)
global combine_docs_chain global combine_docs_chain
combine_docs_chain = create_stuff_documents_chain(llm, prompt) combine_docs_chain = create_stuff_documents_chain(llm, prompt)
@ -1214,7 +1231,7 @@ which will help to create a chain to connect RAG components including:
return "", history + [[message, ""]] return "", history + [[message, ""]]
def bot(history, temperature, top_p, top_k, repetition_penalty, hide_full_prompt): def bot(history, temperature, top_p, top_k, repetition_penalty, hide_full_prompt, do_rag):
""" """
callback function for running chatbot on submit button click callback function for running chatbot on submit button click
@ -1226,6 +1243,7 @@ which will help to create a chain to connect RAG components including:
top_k: parameter for control the range of tokens considered by the AI model based on their cumulative probability, selecting number of tokens with highest probability. top_k: parameter for control the range of tokens considered by the AI model based on their cumulative probability, selecting number of tokens with highest probability.
repetition_penalty: parameter for penalizing tokens based on how frequently they occur in the text. repetition_penalty: parameter for penalizing tokens based on how frequently they occur in the text.
hide_full_prompt: whether to show searching results in promopt. hide_full_prompt: whether to show searching results in promopt.
do_rag: whether do RAG when generating texts.
""" """
streamer = TextIteratorStreamer( streamer = TextIteratorStreamer(
@ -1246,7 +1264,11 @@ which will help to create a chain to connect RAG components including:
if stop_tokens is not None: if stop_tokens is not None:
llm.pipeline._forward_params["stopping_criteria"] = StoppingCriteriaList(stop_tokens) llm.pipeline._forward_params["stopping_criteria"] = StoppingCriteriaList(stop_tokens)
t1 = Thread(target=rag_chain.invoke, args=({"input": history[-1][0]},)) if do_rag:
t1 = Thread(target=rag_chain.invoke, args=({"input": history[-1][0]},))
else:
input_text = rag_prompt_template.format(input=history[-1][0], context="")
t1 = Thread(target=llm.invoke, args=(input_text,))
t1.start() t1.start()
# Initialize an empty string to store the generated text # Initialize an empty string to store the generated text
@ -1301,8 +1323,8 @@ which will help to create a chain to connect RAG components including:
chunk_size = gr.Slider( chunk_size = gr.Slider(
label="Chunk size", label="Chunk size",
value=700, value=400,
minimum=100, minimum=50,
maximum=2000, maximum=2000,
step=50, step=50,
interactive=True, interactive=True,
@ -1311,7 +1333,7 @@ which will help to create a chain to connect RAG components including:
chunk_overlap = gr.Slider( chunk_overlap = gr.Slider(
label="Chunk overlap", label="Chunk overlap",
value=100, value=50,
minimum=0, minimum=0,
maximum=400, maximum=400,
step=10, step=10,
@ -1324,6 +1346,12 @@ which will help to create a chain to connect RAG components including:
value="Vector Store is Not ready", value="Vector Store is Not ready",
interactive=False, interactive=False,
) )
do_rag = gr.Checkbox(
value=True,
label="RAG is ON",
interactive=True,
info="Whether to do RAG for generation",
)
with gr.Accordion("Generation Configuration", open=False): with gr.Accordion("Generation Configuration", open=False):
with gr.Row(): with gr.Row():
with gr.Column(): with gr.Column():
@ -1454,13 +1482,13 @@ which will help to create a chain to connect RAG components including:
) )
submit_event = msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then( submit_event = msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then(
bot, bot,
[chatbot, temperature, top_p, top_k, repetition_penalty, hide_context], [chatbot, temperature, top_p, top_k, repetition_penalty, hide_context, do_rag],
chatbot, chatbot,
queue=True, queue=True,
) )
submit_click_event = submit.click(user, [msg, chatbot], [msg, chatbot], queue=False).then( submit_click_event = submit.click(user, [msg, chatbot], [msg, chatbot], queue=False).then(
bot, bot,
[chatbot, temperature, top_p, top_k, repetition_penalty, hide_context], [chatbot, temperature, top_p, top_k, repetition_penalty, hide_context, do_rag],
chatbot, chatbot,
queue=True, queue=True,
) )

5
docs/notebooks/magika-content-type-recognition-with-output.rst
View File

@ -42,8 +42,7 @@ post <https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-effic
In this tutorial we consider how to bring OpenVINO power into Magika. In this tutorial we consider how to bring OpenVINO power into Magika.
Table of contents: Table of contents:
------------------ ^^^^^^^^^^^^^^^^^^
- `Prerequisites <#prerequisites>`__ - `Prerequisites <#prerequisites>`__
- `Define model loading class <#define-model-loading-class>`__ - `Define model loading class <#define-model-loading-class>`__
@ -59,8 +58,6 @@ Table of contents:
Prerequisites Prerequisites
------------- -------------
.. code:: ipython3 .. code:: ipython3
%pip install -q magika "openvino>=2024.1.0" "gradio>=4.19" %pip install -q magika "openvino>=2024.1.0" "gradio>=4.19"

2
docs/notebooks/meter-reader-with-output.rst
View File

@ -641,7 +641,7 @@ bounds of input batch size.
.. parsed-literal:: .. parsed-literal::
<matplotlib.image.AxesImage at 0x7f9941fc2040> <matplotlib.image.AxesImage at 0x7f9eff572a90>

161
docs/notebooks/mobileclip-video-search-with-output.rst
View File

@ -47,14 +47,13 @@ Table of contents:
Prerequisites Prerequisites
------------- -------------
## Prerequisites
.. code:: ipython3 .. code:: ipython3
from pathlib import Path from pathlib import Path
repo_dir = Path("./ml-mobileclip") repo_dir = Path("./ml-mobileclip")
if not repo_dir.exists(): if not repo_dir.exists():
!git clone https://github.com/apple/ml-mobileclip.git !git clone https://github.com/apple/ml-mobileclip.git
@ -66,15 +65,15 @@ Prerequisites
remote: Counting objects: 100% (45/45), done. remote: Counting objects: 100% (45/45), done.
remote: Compressing objects: 100% (36/36), done. remote: Compressing objects: 100% (36/36), done.
remote: Total 45 (delta 9), reused 44 (delta 8), pack-reused 0 remote: Total 45 (delta 9), reused 44 (delta 8), pack-reused 0
Unpacking objects: 100% (45/45), 428.50 KiB | 3.25 MiB/s, done. Unpacking objects: 100% (45/45), 428.50 KiB | 3.17 MiB/s, done.
.. code:: ipython3 .. code:: ipython3
%pip install -q "./ml-mobileclip" --no-deps %pip install -q "./ml-mobileclip" --no-deps
%pip install -q "clip-benchmark>=1.4.0" "datasets>=2.8.0" "open-clip-torch>=2.20.0" "timm>=0.9.5" "torch>=1.13.1" "torchvision>=0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu %pip install -q "clip-benchmark>=1.4.0" "datasets>=2.8.0" "open-clip-torch>=2.20.0" "timm>=0.9.5" "torch>=1.13.1" "torchvision>=0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
%pip install -q "openvino>=2024.0.0" "gradio>=4.19" "matplotlib" "Pillow" "altair" "pandas" "opencv-python" "tqdm" %pip install -q "openvino>=2024.0.0" "gradio>=4.19" "matplotlib" "Pillow" "altair" "pandas" "opencv-python" "tqdm"
@ -82,8 +81,8 @@ Prerequisites
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible. mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.1+cpu which is incompatible.
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible. mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.1+cpu which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -132,9 +131,9 @@ comparison purposes, you can select different models among:
.. code:: ipython3 .. code:: ipython3
import ipywidgets as widgets import ipywidgets as widgets
model_dir = Path("checkpoints") model_dir = Path("checkpoints")
supported_models = { supported_models = {
"MobileCLIP": { "MobileCLIP": {
"mobileclip_s0": { "mobileclip_s0": {
@ -203,8 +202,8 @@ comparison purposes, you can select different models among:
}, },
}, },
} }
model_type = widgets.Dropdown(options=supported_models.keys(), default="MobileCLIP", description="Model type:") model_type = widgets.Dropdown(options=supported_models.keys(), default="MobileCLIP", description="Model type:")
model_type model_type
@ -220,13 +219,13 @@ comparison purposes, you can select different models among:
.. code:: ipython3 .. code:: ipython3
available_models = supported_models[model_type.value] available_models = supported_models[model_type.value]
model_checkpoint = widgets.Dropdown( model_checkpoint = widgets.Dropdown(
options=available_models.keys(), options=available_models.keys(),
default=list(available_models), default=list(available_models),
description="Model:", description="Model:",
) )
model_checkpoint model_checkpoint
@ -241,15 +240,15 @@ comparison purposes, you can select different models among:
.. code:: ipython3 .. code:: ipython3
import requests import requests
r = requests.get( r = requests.get(
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py", url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py",
) )
open("notebook_utils.py", "w").write(r.text) open("notebook_utils.py", "w").write(r.text)
from notebook_utils import download_file from notebook_utils import download_file
model_config = available_models[model_checkpoint.value] model_config = available_models[model_checkpoint.value]
Run model inference Run model inference
@ -284,8 +283,8 @@ Prepare image gallery
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
import numpy as np import numpy as np
from PIL import Image from PIL import Image
def visualize_result(images: List, query: str = "", selected: List[int] = None): def visualize_result(images: List, query: str = "", selected: List[int] = None):
""" """
Utility function for visualization classification results Utility function for visualization classification results
@ -313,8 +312,8 @@ Prepare image gallery
mask = np.ones_like(np.array(images[idx])) mask = np.ones_like(np.array(images[idx]))
a.imshow(mask, "jet", interpolation="none", alpha=0.75) a.imshow(mask, "jet", interpolation="none", alpha=0.75)
return fig return fig
images_urls = [ images_urls = [
"https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/282ce53e-912d-41aa-ab48-2a001c022d74", "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/282ce53e-912d-41aa-ab48-2a001c022d74",
"https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/9bb40168-82b5-4b11-ada6-d8df104c736c", "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/9bb40168-82b5-4b11-ada6-d8df104c736c",
@ -324,17 +323,17 @@ Prepare image gallery
image_names = ["red_panda.png", "cat.png", "raccoon.png", "dog.png"] image_names = ["red_panda.png", "cat.png", "raccoon.png", "dog.png"]
sample_path = Path("data") sample_path = Path("data")
sample_path.mkdir(parents=True, exist_ok=True) sample_path.mkdir(parents=True, exist_ok=True)
images = [] images = []
for image_name, image_url in zip(image_names, images_urls): for image_name, image_url in zip(image_names, images_urls):
image_path = sample_path / image_name image_path = sample_path / image_name
if not image_path.exists(): if not image_path.exists():
download_file(image_url, filename=image_name, directory=sample_path) download_file(image_url, filename=image_name, directory=sample_path)
images.append(Image.open(image_path).convert("RGB").resize((640, 420))) images.append(Image.open(image_path).convert("RGB").resize((640, 420)))
input_labels = ["cat"] input_labels = ["cat"]
text_descriptions = [f"This is a photo of a {label}" for label in input_labels] text_descriptions = [f"This is a photo of a {label}" for label in input_labels]
visualize_result(images, "image gallery"); visualize_result(images, "image gallery");
@ -381,7 +380,7 @@ preprocessing utilities
from PIL import Image from PIL import Image
import mobileclip import mobileclip
import open_clip import open_clip
# instantiate model # instantiate model
model_name = model_config["model_name"] model_name = model_config["model_name"]
pretrained = model_config["pretrained"] pretrained = model_config["pretrained"]
@ -411,8 +410,8 @@ Perform search
image_tensor = torch.stack([preprocess(image) for image in images]) image_tensor = torch.stack([preprocess(image) for image in images])
text = tokenizer(text_descriptions) text = tokenizer(text_descriptions)
with torch.no_grad(): with torch.no_grad():
# calculate image embeddings # calculate image embeddings
image_encoding_start = time.perf_counter() image_encoding_start = time.perf_counter()
@ -424,22 +423,22 @@ Perform search
text_features = model.encode_text(text) text_features = model.encode_text(text)
text_encoding_end = time.perf_counter() text_encoding_end = time.perf_counter()
print(f"Text encoding took {text_encoding_end - text_encoding_start:.3} ms") print(f"Text encoding took {text_encoding_end - text_encoding_start:.3} ms")
# normalize embeddings # normalize embeddings
image_features /= image_features.norm(dim=-1, keepdim=True) image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True) text_features /= text_features.norm(dim=-1, keepdim=True)
# calcualte similarity score # calcualte similarity score
image_probs = (100.0 * text_features @ image_features.T).softmax(dim=-1) image_probs = (100.0 * text_features @ image_features.T).softmax(dim=-1)
selected_image = [torch.argmax(image_probs).item()] selected_image = [torch.argmax(image_probs).item()]
visualize_result(images, input_labels[0], selected_image); visualize_result(images, input_labels[0], selected_image);
.. parsed-literal:: .. parsed-literal::
Image encoding took 0.0983 ms Image encoding took 0.1 ms
Text encoding took 0.0152 ms Text encoding took 0.0107 ms
@ -466,8 +465,8 @@ be used separately. Let’s convert each part to OpenVINO.
import types import types
import torch.nn.functional as F import torch.nn.functional as F
def se_block_forward(self, inputs): def se_block_forward(self, inputs):
"""Apply forward pass.""" """Apply forward pass."""
b, c, h, w = inputs.size() b, c, h, w = inputs.size()
@ -483,12 +482,12 @@ be used separately. Let’s convert each part to OpenVINO.
import openvino as ov import openvino as ov
import gc import gc
ov_models_dir = Path("ov_models") ov_models_dir = Path("ov_models")
ov_models_dir.mkdir(exist_ok=True) ov_models_dir.mkdir(exist_ok=True)
image_encoder_path = ov_models_dir / f"{model_checkpoint.value}_im_encoder.xml" image_encoder_path = ov_models_dir / f"{model_checkpoint.value}_im_encoder.xml"
if not image_encoder_path.exists(): if not image_encoder_path.exists():
if "mobileclip_s" in model_name: if "mobileclip_s" in model_name:
model.image_encoder.model.conv_exp.se.forward = types.MethodType(se_block_forward, model.image_encoder.model.conv_exp.se) model.image_encoder.model.conv_exp.se.forward = types.MethodType(se_block_forward, model.image_encoder.model.conv_exp.se)
@ -501,23 +500,23 @@ be used separately. Let’s convert each part to OpenVINO.
ov.save_model(ov_image_encoder, image_encoder_path) ov.save_model(ov_image_encoder, image_encoder_path)
del ov_image_encoder del ov_image_encoder
gc.collect() gc.collect()
text_encoder_path = ov_models_dir / f"{model_checkpoint.value}_text_encoder.xml" text_encoder_path = ov_models_dir / f"{model_checkpoint.value}_text_encoder.xml"
if not text_encoder_path.exists(): if not text_encoder_path.exists():
model.forward = model.encode_text model.forward = model.encode_text
ov_text_encoder = ov.convert_model(model, example_input=text, input=[-1, text.shape[1]]) ov_text_encoder = ov.convert_model(model, example_input=text, input=[-1, text.shape[1]])
ov.save_model(ov_text_encoder, text_encoder_path) ov.save_model(ov_text_encoder, text_encoder_path)
del ov_text_encoder del ov_text_encoder
gc.collect() gc.collect()
del model del model
gc.collect(); gc.collect();
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/mobileclip/modules/common/transformer.py:125: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/mobileclip/modules/common/transformer.py:125: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if seq_len != self.num_embeddings: if seq_len != self.num_embeddings:
@ -534,16 +533,16 @@ Select device for image encoder
.. code:: ipython3 .. code:: ipython3
core = ov.Core() core = ov.Core()
import ipywidgets as widgets import ipywidgets as widgets
device = widgets.Dropdown( device = widgets.Dropdown(
options=core.available_devices + ["AUTO"], options=core.available_devices + ["AUTO"],
value="AUTO", value="AUTO",
description="Device:", description="Device:",
disabled=False, disabled=False,
) )
device device
@ -600,17 +599,17 @@ Perform search
print(f"Text encoding took {text_encoding_end - text_encoding_start:.3} ms") print(f"Text encoding took {text_encoding_end - text_encoding_start:.3} ms")
image_features /= image_features.norm(dim=-1, keepdim=True) image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True) text_features /= text_features.norm(dim=-1, keepdim=True)
image_probs = (100.0 * text_features @ image_features.T).softmax(dim=-1) image_probs = (100.0 * text_features @ image_features.T).softmax(dim=-1)
selected_image = [torch.argmax(image_probs).item()] selected_image = [torch.argmax(image_probs).item()]
visualize_result(images, input_labels[0], selected_image); visualize_result(images, input_labels[0], selected_image);
.. parsed-literal:: .. parsed-literal::
Image encoding took 0.0315 ms Image encoding took 0.0309 ms
Text encoding took 0.0059 ms Text encoding took 0.00588 ms
@ -646,14 +645,14 @@ models can require different optimal threshold for search.
ToTensor, ToTensor,
) )
from open_clip.transform import image_transform from open_clip.transform import image_transform
current_device = device.value current_device = device.value
current_model = image_encoder_path.name.split("_im_encoder")[0] current_model = image_encoder_path.name.split("_im_encoder")[0]
available_converted_models = [model_file.name.split("_im_encoder")[0] for model_file in ov_models_dir.glob("*_im_encoder.xml")] available_converted_models = [model_file.name.split("_im_encoder")[0] for model_file in ov_models_dir.glob("*_im_encoder.xml")]
available_devices = list(core.available_devices) + ["AUTO"] available_devices = list(core.available_devices) + ["AUTO"]
download_file( download_file(
"https://github.com/intel-iot-devkit/sample-videos/raw/master/car-detection.mp4", "https://github.com/intel-iot-devkit/sample-videos/raw/master/car-detection.mp4",
directory=sample_path, directory=sample_path,
@ -663,8 +662,8 @@ models can require different optimal threshold for search.
directory=sample_path, directory=sample_path,
filename="coco.mp4", filename="coco.mp4",
) )
def get_preprocess_and_tokenizer(model_name): def get_preprocess_and_tokenizer(model_name):
if "mobileclip" in model_name: if "mobileclip" in model_name:
resolution = supported_models["MobileCLIP"][model_name]["image_size"] resolution = supported_models["MobileCLIP"][model_name]["image_size"]
@ -685,10 +684,10 @@ models can require different optimal threshold for search.
resize_size = model_configs[model_name]["image_size"] resize_size = model_configs[model_name]["image_size"]
preprocess = image_transform((resize_size, resize_size), is_train=False, resize_mode="longest") preprocess = image_transform((resize_size, resize_size), is_train=False, resize_mode="longest")
tokenizer = open_clip.get_tokenizer(model_configs[model_name]["model_name"]) tokenizer = open_clip.get_tokenizer(model_configs[model_name]["model_name"])
return preprocess, tokenizer return preprocess, tokenizer
def run( def run(
path: str, path: str,
text_search: str, text_search: str,
@ -707,7 +706,7 @@ models can require different optimal threshold for search.
global tokenizer global tokenizer
global ov_compiled_image_encoder global ov_compiled_image_encoder
global ov_compiled_text_encoder global ov_compiled_text_encoder
if current_model != model_name or device != current_device: if current_model != model_name or device != current_device:
ov_compiled_image_encoder = core.compile_model(ov_models_dir / f"{model_name}_im_encoder.xml", device) ov_compiled_image_encoder = core.compile_model(ov_models_dir / f"{model_name}_im_encoder.xml", device)
ov_compiled_text_encoder = core.compile_model(ov_models_dir / f"{model_name}_text_encoder.xml", device) ov_compiled_text_encoder = core.compile_model(ov_models_dir / f"{model_name}_text_encoder.xml", device)
@ -717,7 +716,7 @@ models can require different optimal threshold for search.
# Load video # Load video
dataset = LoadVideo(path, transforms=preprocess, vid_stride=stride) dataset = LoadVideo(path, transforms=preprocess, vid_stride=stride)
dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=False, num_workers=0) dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=False, num_workers=0)
# Get image query features # Get image query features
if image_search: if image_search:
image = preprocess(image_search).unsqueeze(0) image = preprocess(image_search).unsqueeze(0)
@ -737,11 +736,11 @@ models can require different optimal threshold for search.
for image, orig, frame, timestamp in dataloader: for image, orig, frame, timestamp in dataloader:
with torch.no_grad(): with torch.no_grad():
image_features = torch.from_numpy(ov_compiled_image_encoder(image)[0]) image_features = torch.from_numpy(ov_compiled_image_encoder(image)[0])
image_features /= image_features.norm(dim=-1, keepdim=True) image_features /= image_features.norm(dim=-1, keepdim=True)
probs = query_features.cpu().numpy() @ image_features.cpu().numpy().T probs = query_features.cpu().numpy() @ image_features.cpu().numpy().T
probs = probs[0] probs = probs[0]
# Save frame similarity values # Save frame similarity values
df = pd.DataFrame( df = pd.DataFrame(
{ {
@ -751,15 +750,15 @@ models can require different optimal threshold for search.
} }
) )
res = pd.concat([res, df]) res = pd.concat([res, df])
# Check if frame is over threshold # Check if frame is over threshold
for i, p in enumerate(probs): for i, p in enumerate(probs):
if p > thresh: if p > thresh:
matches.append(to_pil_image(orig[i])) matches.append(to_pil_image(orig[i]))
matches_probs.append(p) matches_probs.append(p)
print(f"Frames: {frame.tolist()} - Probs: {probs}") print(f"Frames: {frame.tolist()} - Probs: {probs}")
# Create plot of similarity values # Create plot of similarity values
lines = ( lines = (
alt.Chart(res) alt.Chart(res)
@ -770,16 +769,16 @@ models can require different optimal threshold for search.
) )
).properties(width=600) ).properties(width=600)
rule = alt.Chart().mark_rule(strokeDash=[6, 3], size=2).encode(y=alt.datum(thresh)) rule = alt.Chart().mark_rule(strokeDash=[6, 3], size=2).encode(y=alt.datum(thresh))
selected_frames = np.argsort(-1 * np.array(matches_probs))[:20] selected_frames = np.argsort(-1 * np.array(matches_probs))[:20]
matched_sorted_frames = [matches[idx] for idx in selected_frames] matched_sorted_frames = [matches[idx] for idx in selected_frames]
return ( return (
lines + rule, lines + rule,
matched_sorted_frames, matched_sorted_frames,
) # Only return up to 20 images to not crash the UI ) # Only return up to 20 images to not crash the UI
class LoadVideo(Dataset): class LoadVideo(Dataset):
def __init__(self, path, transforms, vid_stride=1): def __init__(self, path, transforms, vid_stride=1):
self.transforms = transforms self.transforms = transforms
@ -787,31 +786,31 @@ models can require different optimal threshold for search.
self.cur_frame = 0 self.cur_frame = 0
self.cap = cv2.VideoCapture(path) self.cap = cv2.VideoCapture(path)
self.total_frames = int(self.cap.get(cv2.CAP_PROP_FRAME_COUNT) / self.vid_stride) self.total_frames = int(self.cap.get(cv2.CAP_PROP_FRAME_COUNT) / self.vid_stride)
def __getitem__(self, _): def __getitem__(self, _):
# Read video # Read video
# Skip over frames # Skip over frames
for _ in range(self.vid_stride): for _ in range(self.vid_stride):
self.cap.grab() self.cap.grab()
self.cur_frame += 1 self.cur_frame += 1
# Read frame # Read frame
_, img = self.cap.retrieve() _, img = self.cap.retrieve()
timestamp = self.cap.get(cv2.CAP_PROP_POS_MSEC) timestamp = self.cap.get(cv2.CAP_PROP_POS_MSEC)
# Convert to PIL # Convert to PIL
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = Image.fromarray(np.uint8(img)) img = Image.fromarray(np.uint8(img))
# Apply transforms # Apply transforms
img_t = self.transforms(img) img_t = self.transforms(img)
return img_t, to_tensor(img), self.cur_frame, timestamp return img_t, to_tensor(img), self.cur_frame, timestamp
def __len__(self): def __len__(self):
return self.total_frames return self.total_frames
desc_text = """ desc_text = """
Search the content's of a video with a text description. Search the content's of a video with a text description.
__Note__: Long videos (over a few minutes) may cause UI performance issues. __Note__: Long videos (over a few minutes) may cause UI performance issues.
@ -840,7 +839,7 @@ models can require different optimal threshold for search.
examples=[[sample_path / "car-detection.mp4", "white car"]], examples=[[sample_path / "car-detection.mp4", "white car"]],
allow_flagging="never", allow_flagging="never",
) )
desc_image = """ desc_image = """
Search the content's of a video with an image query. Search the content's of a video with an image query.
__Note__: Long videos (over a few minutes) may cause UI performance issues. __Note__: Long videos (over a few minutes) may cause UI performance issues.
@ -874,8 +873,8 @@ models can require different optimal threshold for search.
tab_names=["Text Query Search", "Image Query Search"], tab_names=["Text Query Search", "Image Query Search"],
title="CLIP Video Content Search", title="CLIP Video Content Search",
) )
try: try:
demo.launch(debug=False) demo.launch(debug=False)
except Exception: except Exception:
@ -900,7 +899,7 @@ models can require different optimal threshold for search.
.. parsed-literal:: .. parsed-literal::
Running on local URL: http://127.0.0.1:7860 Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`. To create a public link, set `share=True` in `launch()`.

37
docs/notebooks/mobilevlm-language-assistant-with-output.rst
View File

@ -56,9 +56,9 @@ Install requirements
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible. mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.1+cpu which is incompatible.
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible. mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.1+cpu which is incompatible.
optimum-intel 1.17.0.dev0+8c2b787 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible. optimum-intel 1.18.0.dev0+2c79d98 requires transformers<4.42.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -110,13 +110,13 @@ Import required packages
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node( torch.utils._pytree._register_pytree_node(
2024-05-16 00:40:31.564225: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:46:44.654609: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:40:31.599009: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:46:44.688475: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:40:32.110586: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:46:45.198736: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node( torch.utils._pytree._register_pytree_node(
@ -324,24 +324,15 @@ compression instead of INT8 weight compression.
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:595: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
.. parsed-literal::
WARNING:nncf:NNCF provides best results with torch==2.2.*, while current torch version is 2.3.0+cpu. If you encounter issues, consider switching to torch==2.2.*
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:595: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if input_shape[-1] > 1: if input_shape[-1] > 1:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if seq_len > self.max_seq_len_cached: if seq_len > self.max_seq_len_cached:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:348: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:348: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (bsz, self.num_heads, q_len, kv_seq_len): if attn_weights.size() != (bsz, self.num_heads, q_len, kv_seq_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:355: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:355: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attention_mask.size() != (bsz, 1, q_len, kv_seq_len): if attention_mask.size() != (bsz, 1, q_len, kv_seq_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:365: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py:365: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (bsz, self.num_heads, q_len, self.head_dim): if attn_output.size() != (bsz, self.num_heads, q_len, self.head_dim):
@ -423,7 +414,7 @@ compression instead of INT8 weight compression.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.) /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/torch/jit/_trace.py:165: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)
if a.grad is not None: if a.grad is not None:

32
docs/notebooks/model-tools-with-output.rst
View File

@ -225,20 +225,20 @@ Converting mobilenet-v2-pytorch…
.. parsed-literal:: .. parsed-literal::
========== Converting mobilenet-v2-pytorch to ONNX ========== Converting mobilenet-v2-pytorch to ONNX
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-name=mobilenet_v2 --weights=model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth --import-module=torchvision.models --input-shape=1,3,224,224 --output-file=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx --input-names=data --output-names=prob Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-name=mobilenet_v2 --weights=model/public/mobilenet-v2-pytorch/mobilenet_v2-b0353104.pth --import-module=torchvision.models --input-shape=1,3,224,224 --output-file=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx --input-names=data --output-names=prob
ONNX check passed successfully. ONNX check passed successfully.
========== Converting mobilenet-v2-pytorch to IR (FP16) ========== Converting mobilenet-v2-pytorch to IR (FP16)
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/mobilenet-v2-pytorch/FP16 --model_name=mobilenet-v2-pytorch --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.624,57.12,57.375]' --reverse_input_channels --output=prob --input_model=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 224, 224]' --compress_to_fp16=True Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/mobilenet-v2-pytorch/FP16 --model_name=mobilenet-v2-pytorch --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.624,57.12,57.375]' --reverse_input_channels --output=prob --input_model=model/public/mobilenet-v2-pytorch/mobilenet-v2.onnx '--layout=data(NCHW)' '--input_shape=[1, 3, 224, 224]' --compress_to_fp16=True
[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False. [ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.
Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/model-tools/model/public/mobilenet-v2-pytorch/FP16/mobilenet-v2-pytorch.bin
@ -290,8 +290,8 @@ information in a dictionary.
'description': 'MobileNet V2 is image classification model pre-trained on ImageNet dataset. This is a PyTorch* implementation of MobileNetV2 architecture as described in the paper "Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation" <https://arxiv.org/abs/1801.04381>.\nThe model input is a blob that consists of a single image of "1, 3, 224, 224" in "RGB" order.\nThe model output is typical object classifier for the 1000 different classifications matching with those in the ImageNet database.', 'description': 'MobileNet V2 is image classification model pre-trained on ImageNet dataset. This is a PyTorch* implementation of MobileNetV2 architecture as described in the paper "Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation" <https://arxiv.org/abs/1801.04381>.\nThe model input is a blob that consists of a single image of "1, 3, 224, 224" in "RGB" order.\nThe model output is typical object classifier for the 1000 different classifications matching with those in the ImageNet database.',
'framework': 'pytorch', 'framework': 'pytorch',
'license_url': 'https://raw.githubusercontent.com/pytorch/vision/master/LICENSE', 'license_url': 'https://raw.githubusercontent.com/pytorch/vision/master/LICENSE',
'accuracy_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/accuracy-check.yml', 'accuracy_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/accuracy-check.yml',
'model_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/model.yml', 'model_config': '/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/mobilenet-v2-pytorch/model.yml',
'precisions': ['FP16', 'FP32'], 'precisions': ['FP16', 'FP32'],
'subdirectory': 'public/mobilenet-v2-pytorch', 'subdirectory': 'public/mobilenet-v2-pytorch',
'task_type': 'classification', 'task_type': 'classification',
@ -368,7 +368,7 @@ seconds…
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT. [ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files [Step 4/11] Reading model files
[ INFO ] Loading model files [ INFO ] Loading model files
[ INFO ] Read model took 28.84 ms [ INFO ] Read model took 27.44 ms
[ INFO ] Original model I/O parameters: [ INFO ] Original model I/O parameters:
[ INFO ] Model inputs: [ INFO ] Model inputs:
[ INFO ] data (node: data) : f32 / [N,C,H,W] / [1,3,224,224] [ INFO ] data (node: data) : f32 / [N,C,H,W] / [1,3,224,224]
@ -382,7 +382,7 @@ seconds…
[ INFO ] Model outputs: [ INFO ] Model outputs:
[ INFO ] prob (node: prob) : f32 / [...] / [1,1000] [ INFO ] prob (node: prob) : f32 / [...] / [1,1000]
[Step 7/11] Loading the model to the device [Step 7/11] Loading the model to the device
[ INFO ] Compile model took 147.59 ms [ INFO ] Compile model took 162.51 ms
[Step 8/11] Querying optimal runtime parameters [Step 8/11] Querying optimal runtime parameters
[ INFO ] Model: [ INFO ] Model:
[ INFO ] NETWORK_NAME: main_graph [ INFO ] NETWORK_NAME: main_graph
@ -410,17 +410,17 @@ seconds…
[ INFO ] Fill input 'data' with random values [ INFO ] Fill input 'data' with random values
[Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration) [Step 10/11] Measuring performance (Start inference asynchronously, 6 inference requests, limits: 15000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop). [ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).
[ INFO ] First inference took 6.31 ms [ INFO ] First inference took 5.52 ms
[Step 11/11] Dumping statistics report [Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU'] [ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 20214 iterations [ INFO ] Count: 20316 iterations
[ INFO ] Duration: 15004.27 ms [ INFO ] Duration: 15008.24 ms
[ INFO ] Latency: [ INFO ] Latency:
[ INFO ] Median: 4.32 ms [ INFO ] Median: 4.30 ms
[ INFO ] Average: 4.33 ms [ INFO ] Average: 4.30 ms
[ INFO ] Min: 2.48 ms [ INFO ] Min: 2.64 ms
[ INFO ] Max: 12.25 ms [ INFO ] Max: 12.94 ms
[ INFO ] Throughput: 1347.22 FPS [ INFO ] Throughput: 1353.66 FPS
Benchmark with Different Settings Benchmark with Different Settings

39
docs/notebooks/music-generation-with-output.rst
View File

File diff suppressed because one or more lines are too long

1263
docs/notebooks/nano-llava-multimodal-chatbot-with-output.rst Normal file
View File

File diff suppressed because it is too large Load Diff

3
docs/notebooks/nano-llava-multimodal-chatbot-with-output_files/nano-llava-multimodal-chatbot-with-output_7_1.jpg Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:795e7d90a3a734604a845cbab2e038d9ee5b713ab33390183562af970605fe42
size 65057

3
docs/notebooks/nano-llava-multimodal-chatbot-with-output_files/nano-llava-multimodal-chatbot-with-output_7_1.png Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:425c949d0a4a907c2ffdaeef53fcd948f1b184cfe325bf69b76994a5a9b6c1d0
size 490750

2
docs/notebooks/notebooks_with_colab_buttons.txt
View File

@ -23,6 +23,7 @@ mobileclip-video-search
model-tools model-tools
music-generation music-generation
named-entity-recognition named-entity-recognition
nano-llava-multimodal-chatbot
object-detection object-detection
openvino-api openvino-api
openvino-tokenizers openvino-tokenizers
@ -30,6 +31,7 @@ optical-character-recognition
optimize-preprocessing optimize-preprocessing
paddle-ocr-webcam paddle-ocr-webcam
paddle-to-openvino-classification paddle-to-openvino-classification
person-counting
person-tracking person-tracking
pix2struct-docvqa pix2struct-docvqa
pytorch-onnx-to-openvino pytorch-onnx-to-openvino

5
docs/notebooks/object-detection-with-output.rst
View File

@ -70,9 +70,8 @@ Install requirements
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
magika 0.5.1 requires numpy<2.0,>=1.24; python_version >= "3.8" and python_version < "3.9", but you have numpy 1.23.5 which is incompatible. magika 0.5.1 requires numpy<2.0,>=1.24; python_version >= "3.8" and python_version < "3.9", but you have numpy 1.23.5 which is incompatible.
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible. mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.1+cpu which is incompatible.
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible. mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.1+cpu which is incompatible.
optimum-intel 1.17.0.dev0+8c2b787 requires transformers<4.41.0,>=4.36.0, but you have transformers 4.33.3 which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.

4
docs/notebooks/object-detection-with-output_files/object-detection-with-output_19_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:0aa26914ef65f9ab1061c4195a73b0b8c1d098a02bb4cb01c12997aed53843ad oid sha256:0ed603a97f1687a8b9ce577079909662f0ef54ba3324885625da2057b1255286
size 175079 size 175062

20
docs/notebooks/openvino-api-with-output.rst
View File

@ -194,7 +194,7 @@ notebooks.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
@ -243,7 +243,7 @@ points to the filename of an ONNX model.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.onnx') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.onnx')
@ -303,7 +303,7 @@ without any conversion step. Pass the filename with extension to
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/inference.pdiparams') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/inference.pdiparams')
@ -347,7 +347,7 @@ TensorFlow models saved in frozen graph format can also be passed to
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.pb') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.pb')
@ -403,7 +403,7 @@ It is pre-trained model optimized to work with TensorFlow Lite.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.tflite') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.tflite')
@ -484,7 +484,7 @@ Information about the inputs and outputs of the model are in
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
@ -690,7 +690,7 @@ produced data as values.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
@ -879,7 +879,7 @@ input shape.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.bin') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/segmentation.bin')
@ -1031,7 +1031,7 @@ the cache.
.. parsed-literal:: .. parsed-literal::
PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin') PosixPath('/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/openvino-api/model/classification.bin')
@ -1061,7 +1061,7 @@ the cache.
.. parsed-literal:: .. parsed-literal::
Loading the network to the CPU device took 0.18 seconds. Loading the network to the CPU device took 0.15 seconds.
After running the previous cell, we know the model exists in the cache After running the previous cell, we know the model exists in the cache

60
docs/notebooks/openvino-tokenizers-with-output.rst
View File

@ -95,7 +95,6 @@ use ``pip install openvino-tokenizers[transformers]``.
.. code:: ipython3 .. code:: ipython3
%pip install -Uq pip %pip install -Uq pip
%pip uninstall -y openvino openvino-nightly openvino-dev
%pip install --pre -Uq openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly %pip install --pre -Uq openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
%pip install "torch>=2.1" --extra-index-url https://download.pytorch.org/whl/cpu %pip install "torch>=2.1" --extra-index-url https://download.pytorch.org/whl/cpu
@ -103,30 +102,19 @@ use ``pip install openvino-tokenizers[transformers]``.
.. parsed-literal:: .. parsed-literal::
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Found existing installation: openvino 2024.1.0
Uninstalling openvino-2024.1.0:
Successfully uninstalled openvino-2024.1.0
Found existing installation: openvino-nightly 2024.2.0.dev20240513
Uninstalling openvino-nightly-2024.2.0.dev20240513:
Successfully uninstalled openvino-nightly-2024.2.0.dev20240513
Found existing installation: openvino-dev 2024.1.0
Uninstalling openvino-dev-2024.1.0:
Successfully uninstalled openvino-dev-2024.1.0
Note: you may need to restart the kernel to use updated packages.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mobileclip 0.1.0 requires torch==1.13.1, but you have torch 2.3.0+cpu which is incompatible. openvino-dev 2024.1.0 requires openvino==2024.1.0, but you have openvino 2024.3.0.dev20240605 which is incompatible.
mobileclip 0.1.0 requires torchvision==0.14.1, but you have torchvision 0.18.0+cpu which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.0+cpu) Requirement already satisfied: torch>=2.1 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (2.3.1+cpu)
Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.14.0) Requirement already satisfied: filelock in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.14.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.11.0) Requirement already satisfied: typing-extensions>=4.8.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (4.12.1)
Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12) Requirement already satisfied: sympy in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (1.12.1)
Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1) Requirement already satisfied: networkx in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1)
Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4) Requirement already satisfied: jinja2 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (3.1.4)
Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1) Requirement already satisfied: fsspec in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from torch>=2.1) (2024.3.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5) Requirement already satisfied: MarkupSafe>=2.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from jinja2->torch>=2.1) (2.1.5)
Requirement already satisfied: mpmath>=0.19 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0) Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages (from sympy->torch>=2.1) (1.3.0)
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -159,8 +147,6 @@ constructor.
.. parsed-literal:: .. parsed-literal::
Loading Huggingface Tokenizer... Loading Huggingface Tokenizer...
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Converting Huggingface Tokenizer to OpenVINO... Converting Huggingface Tokenizer to OpenVINO...
Saved OpenVINO Tokenizer: tokenizer/openvino_tokenizer.xml, tokenizer/openvino_tokenizer.bin Saved OpenVINO Tokenizer: tokenizer/openvino_tokenizer.xml, tokenizer/openvino_tokenizer.bin
Saved OpenVINO Detokenizer: tokenizer/openvino_detokenizer.xml, tokenizer/openvino_detokenizer.bin Saved OpenVINO Detokenizer: tokenizer/openvino_detokenizer.xml, tokenizer/openvino_detokenizer.bin
@ -193,12 +179,6 @@ The other method is to pass HuggingFace ``hf_tokenizer`` object to
ov_tokenizer, ov_detokenizer ov_tokenizer, ov_detokenizer
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
.. parsed-literal:: .. parsed-literal::
@ -262,7 +242,7 @@ tasks, but not suitable for text generation.
Token ids: [[ 1 4321] Token ids: [[ 1 4321]
[ 1 6031]] [ 1 6031]]
Detokenized text: ['<s> Test' '<s> strings'] Detokenized text: ['Test' 'strings']
We can compare the result of converted (de)tokenizer with the original We can compare the result of converted (de)tokenizer with the original
@ -284,10 +264,10 @@ one:
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:50:00.049334: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 00:58:07.002270: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:50:00.085249: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 00:58:07.036768: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:50:00.664160: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:58:07.614844: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
.. parsed-literal:: .. parsed-literal::
@ -499,17 +479,17 @@ models and tokenizers simplifies memory management.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:50:37.102688: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 00:58:44.506889: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, tensorflow, onnx, openvino
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node( torch.utils._pytree._register_pytree_node(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn( warnings.warn(
Framework not specified. Using pt to export the model. Framework not specified. Using pt to export the model.
Using framework PyTorch: 2.3.0+cpu Using framework PyTorch: 2.3.1+cpu
Overriding 1 configuration item(s) Overriding 1 configuration item(s)
- use_cache -> False - use_cache -> False
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
Detokenizer is not supported, convert tokenizer only. Detokenizer is not supported, convert tokenizer only.
@ -525,8 +505,6 @@ models and tokenizers simplifies memory management.
.. parsed-literal:: .. parsed-literal::
Loading Huggingface Tokenizer... Loading Huggingface Tokenizer...
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Converting Huggingface Tokenizer to OpenVINO... Converting Huggingface Tokenizer to OpenVINO...
Saved OpenVINO Tokenizer: bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.xml, bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.bin Saved OpenVINO Tokenizer: bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.xml, bert-tiny-finetuned-sms-spam-detection/openvino_tokenizer.bin

73
docs/notebooks/openvoice-with-output.rst
View File

File diff suppressed because one or more lines are too long

10
docs/notebooks/optical-character-recognition-with-output.rst
View File

@ -64,7 +64,7 @@ Table of contents:
.. parsed-literal:: .. parsed-literal::
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
openvino-tokenizers 2024.2.0.0.dev20240513 requires openvino~=2024.2.0.0.dev, but you have openvino 2024.1.0 which is incompatible. openvino-tokenizers 2024.3.0.0.dev20240605 requires openvino~=2024.3.0.0.dev, but you have openvino 2024.1.0 which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -368,20 +368,20 @@ Converting text-recognition-resnet-fc…
.. parsed-literal:: .. parsed-literal::
========== Converting text-recognition-resnet-fc to ONNX ========== Converting text-recognition-resnet-fc to ONNX
Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/text-recognition-resnet-fc --model-path=model/public/text-recognition-resnet-fc --model-name=get_model --import-module=model '--model-param=file_config=r"model/public/text-recognition-resnet-fc/vedastr/configs/resnet_fc.py"' '--model-param=weights=r"model/public/text-recognition-resnet-fc/vedastr/ckpt/resnet_fc.pth"' --input-shape=1,1,32,100 --input-names=input --output-names=output --output-file=model/public/text-recognition-resnet-fc/resnet_fc.onnx Conversion to ONNX command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/internal_scripts/pytorch_to_onnx.py --model-path=/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/omz_tools/models/public/text-recognition-resnet-fc --model-path=model/public/text-recognition-resnet-fc --model-name=get_model --import-module=model '--model-param=file_config=r"model/public/text-recognition-resnet-fc/vedastr/configs/resnet_fc.py"' '--model-param=weights=r"model/public/text-recognition-resnet-fc/vedastr/ckpt/resnet_fc.pth"' --input-shape=1,1,32,100 --input-names=input --output-names=output --output-file=model/public/text-recognition-resnet-fc/resnet_fc.onnx
ONNX check passed successfully. ONNX check passed successfully.
========== Converting text-recognition-resnet-fc to IR (FP16) ========== Converting text-recognition-resnet-fc to IR (FP16)
Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/text-recognition-resnet-fc/FP16 --model_name=text-recognition-resnet-fc --input=input '--mean_values=input[127.5]' '--scale_values=input[127.5]' --output=output --input_model=model/public/text-recognition-resnet-fc/resnet_fc.onnx '--layout=input(NCHW)' '--input_shape=[1, 1, 32, 100]' --compress_to_fp16=True Conversion command: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/python -- /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/bin/mo --framework=onnx --output_dir=model/public/text-recognition-resnet-fc/FP16 --model_name=text-recognition-resnet-fc --input=input '--mean_values=input[127.5]' '--scale_values=input[127.5]' --output=output --input_model=model/public/text-recognition-resnet-fc/resnet_fc.onnx '--layout=input(NCHW)' '--input_shape=[1, 1, 32, 100]' --compress_to_fp16=True
[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False. [ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.
Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html
[ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API. [ INFO ] MO command line tool is considered as the legacy conversion API as of OpenVINO 2023.2 release. Please use OpenVINO Model Converter (OVC). OVC represents a lightweight alternative of MO and provides simplified model conversion API.
Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html Find more information about transition from MO to OVC at https://docs.openvino.ai/2023.2/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html
[ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.xml [ SUCCESS ] XML file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.xml
[ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.bin [ SUCCESS ] BIN file: /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/optical-character-recognition/model/public/text-recognition-resnet-fc/FP16/text-recognition-resnet-fc.bin

14
docs/notebooks/optimize-preprocessing-with-output.rst
View File

@ -207,8 +207,8 @@ and save it to the disk.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:53:11.397090: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW 2024-06-06 01:01:17.864372: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE: forward compatibility was attempted on non supported HW
2024-05-16 00:53:11.397273: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration 2024-06-06 01:01:17.864557: E tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:312] kernel version 470.182.3 does not match DSO version 470.223.2 -- cannot find working devices in this configuration
.. parsed-literal:: .. parsed-literal::
@ -366,7 +366,7 @@ for mean/scale normalization.
.. parsed-literal:: .. parsed-literal::
<openvino._pyopenvino.preprocess.InputTensorInfo at 0x7fd96c504970> <openvino._pyopenvino.preprocess.InputTensorInfo at 0x7f4ec064a1f0>
@ -397,7 +397,7 @@ may be specified is input data
.. parsed-literal:: .. parsed-literal::
<openvino._pyopenvino.preprocess.InputModelInfo at 0x7fd96c504830> <openvino._pyopenvino.preprocess.InputModelInfo at 0x7f4ec0650b70>
@ -435,7 +435,7 @@ then such conversion will be added explicitly.
.. parsed-literal:: .. parsed-literal::
<openvino._pyopenvino.preprocess.PreProcessSteps at 0x7fd96c532a70> <openvino._pyopenvino.preprocess.PreProcessSteps at 0x7f4ec0650330>
@ -649,6 +649,6 @@ Compare performance
.. parsed-literal:: .. parsed-literal::
IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0152 seconds per image, FPS: 65.63 IR model in OpenVINO Runtime/CPU with manual image preprocessing: 0.0153 seconds per image, FPS: 65.57
IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0184 seconds per image, FPS: 54.32 IR model in OpenVINO Runtime/CPU with preprocessing API: 0.0140 seconds per image, FPS: 71.22

4
docs/notebooks/paddle-ocr-webcam-with-output.rst
View File

@ -200,7 +200,7 @@ Download the Model for Text **Detection**
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/paddle-o… /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/paddle-o…
.. parsed-literal:: .. parsed-literal::
@ -246,7 +246,7 @@ Download the Model for Text **Recognition**
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/paddle-o… /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/paddle-o…
.. parsed-literal:: .. parsed-literal::

4
docs/notebooks/paddle-ocr-webcam-with-output_files/paddle-ocr-webcam-with-output_30_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:a7f1428e52cc0f79f7ee67a644231f522aa713e6d54a73f8511693e1a28ec610 oid sha256:7769f0a5ccc478c3f33005eb556365ab725f495ef55a296d7576f716c2cdb99a
size 590583 size 591026

20
docs/notebooks/paddle-to-openvino-classification-with-output.rst
View File

@ -64,7 +64,7 @@ Imports
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
paddleclas 2.5.2 requires easydict, which is not installed. paddleclas 2.5.2 requires easydict, which is not installed.
paddleclas 2.5.2 requires gast==0.3.3, but you have gast 0.4.0 which is incompatible. paddleclas 2.5.2 requires gast==0.3.3, but you have gast 0.4.0 which is incompatible.
paddleclas 2.5.2 requires opencv-python==4.6.0.66, but you have opencv-python 4.9.0.80 which is incompatible. paddleclas 2.5.2 requires opencv-python==4.6.0.66, but you have opencv-python 4.10.0.82 which is incompatible.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages. Note: you may need to restart the kernel to use updated packages.
@ -78,11 +78,11 @@ Imports
.. parsed-literal:: .. parsed-literal::
--2024-05-16 00:54:56-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb --2024-06-06 01:02:58-- http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb
Resolving proxy-dmz.intel.com (proxy-dmz.intel.com)... 10.241.208.166 Resolving proxy-dmz.intel.com (proxy-dmz.intel.com)... 10.241.208.166
Connecting to proxy-dmz.intel.com (proxy-dmz.intel.com)|10.241.208.166|:911... connected. Connecting to proxy-dmz.intel.com (proxy-dmz.intel.com)|10.241.208.166|:911... connected.
Proxy request sent, awaiting response... 404 Not Found Proxy request sent, awaiting response... 404 Not Found
2024-05-16 00:54:56 ERROR 404: Not Found. 2024-06-06 01:02:58 ERROR 404: Not Found.
dpkg: error: cannot access archive 'libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb': No such file or directory dpkg: error: cannot access archive 'libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb': No such file or directory
@ -113,8 +113,8 @@ Imports
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:54:58 INFO: Loading faiss with AVX512 support. 2024-06-06 01:03:00 INFO: Loading faiss with AVX512 support.
2024-05-16 00:54:58 INFO: Successfully loaded faiss with AVX512 support. 2024-06-06 01:03:00 INFO: Successfully loaded faiss with AVX512 support.
Settings Settings
@ -198,7 +198,7 @@ inference on that image, and then show the top three prediction results.
.. parsed-literal:: .. parsed-literal::
[2024/05/16 00:55:27] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead. [2024/06/06 01:03:28] ppcls WARNING: The current running environment does not support the use of GPU. CPU has been used instead.
Labrador retriever, 0.75138 Labrador retriever, 0.75138
German short-haired pointer, 0.02373 German short-haired pointer, 0.02373
Great Dane, 0.01848 Great Dane, 0.01848
@ -264,7 +264,7 @@ clipping values.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:55:27 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers). 2024-06-06 01:03:29 WARNING: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
.. parsed-literal:: .. parsed-literal::
@ -276,7 +276,7 @@ clipping values.
.. parsed-literal:: .. parsed-literal::
<matplotlib.image.AxesImage at 0x7fa3f4283910> <matplotlib.image.AxesImage at 0x7fd0b0255130>
@ -455,7 +455,7 @@ Note that many optimizations are possible to improve the performance.
.. parsed-literal:: .. parsed-literal::
PaddlePaddle model on CPU: 0.0076 seconds per image, FPS: 132.29 PaddlePaddle model on CPU: 0.0075 seconds per image, FPS: 133.88
PaddlePaddle result: PaddlePaddle result:
Labrador retriever, 0.75138 Labrador retriever, 0.75138
@ -516,7 +516,7 @@ select device from dropdown list for running inference using OpenVINO
.. parsed-literal:: .. parsed-literal::
OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0029 seconds per image, FPS: 340.41 OpenVINO IR model in OpenVINO Runtime (AUTO): 0.0029 seconds per image, FPS: 340.10
OpenVINO result: OpenVINO result:
Labrador retriever, 0.74909 Labrador retriever, 0.74909

151
docs/notebooks/person-tracking-with-output.rst
View File

@ -118,10 +118,10 @@ Table of contents:
.. code:: ipython3 .. code:: ipython3
import platform import platform
%pip install -q "openvino-dev>=2024.0.0" %pip install -q "openvino-dev>=2024.0.0"
%pip install -q opencv-python requests scipy tqdm %pip install -q opencv-python requests scipy tqdm
if platform.system() != "Windows": if platform.system() != "Windows":
%pip install -q "matplotlib>=3.4" %pip install -q "matplotlib>=3.4"
else: else:
@ -145,7 +145,7 @@ Imports
import collections import collections
from pathlib import Path from pathlib import Path
import time import time
import numpy as np import numpy as np
import cv2 import cv2
from IPython import display from IPython import display
@ -155,17 +155,17 @@ Imports
.. code:: ipython3 .. code:: ipython3
# Import local modules # Import local modules
if not Path("./notebook_utils.py").exists(): if not Path("./notebook_utils.py").exists():
# Fetch `notebook_utils` module # Fetch `notebook_utils` module
import requests import requests
r = requests.get( r = requests.get(
url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py", url="https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py",
) )
open("notebook_utils.py", "w").write(r.text) open("notebook_utils.py", "w").write(r.text)
import notebook_utils as utils import notebook_utils as utils
from deepsort_utils.tracker import Tracker from deepsort_utils.tracker import Tracker
from deepsort_utils.nn_matching import NearestNeighborDistanceMetric from deepsort_utils.nn_matching import NearestNeighborDistanceMetric
@ -218,46 +218,46 @@ replace the name of the model in the code below.
precision = "FP16" precision = "FP16"
# The name of the model from Open Model Zoo # The name of the model from Open Model Zoo
detection_model_name = "person-detection-0202" detection_model_name = "person-detection-0202"
download_command = ( download_command = (
f"omz_downloader " f"--name {detection_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}" f"omz_downloader " f"--name {detection_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}"
) )
! $download_command ! $download_command
detection_model_path = f"model/intel/{detection_model_name}/{precision}/{detection_model_name}.xml" detection_model_path = f"model/intel/{detection_model_name}/{precision}/{detection_model_name}.xml"
reidentification_model_name = "person-reidentification-retail-0287" reidentification_model_name = "person-reidentification-retail-0287"
download_command = ( download_command = (
f"omz_downloader " f"--name {reidentification_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}" f"omz_downloader " f"--name {reidentification_model_name} " f"--precisions {precision} " f"--output_dir {base_model_dir} " f"--cache_dir {base_model_dir}"
) )
! $download_command ! $download_command
reidentification_model_path = f"model/intel/{reidentification_model_name}/{precision}/{reidentification_model_name}.xml" reidentification_model_path = f"model/intel/{reidentification_model_name}/{precision}/{reidentification_model_name}.xml"
.. parsed-literal:: .. parsed-literal::
################|| Downloading person-detection-0202 ||################ ################|| Downloading person-detection-0202 ||################
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.xml ========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.xml
========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin ========== Downloading model/intel/person-detection-0202/FP16/person-detection-0202.bin
################|| Downloading person-reidentification-retail-0287 ||################ ################|| Downloading person-reidentification-retail-0287 ||################
========== Downloading model/intel/person-reidentification-retail-0287/person-reidentification-retail-0267.onnx ========== Downloading model/intel/person-reidentification-retail-0287/person-reidentification-retail-0267.onnx
========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml ========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml
========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.bin ========== Downloading model/intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.bin
Load model Load model
@ -282,18 +282,18 @@ performance, but slightly longer startup time).
.. code:: ipython3 .. code:: ipython3
core = ov.Core() core = ov.Core()
class Model: class Model:
""" """
This class represents a OpenVINO model object. This class represents a OpenVINO model object.
""" """
def __init__(self, model_path, batchsize=1, device="AUTO"): def __init__(self, model_path, batchsize=1, device="AUTO"):
""" """
Initialize the model object Initialize the model object
Parameters Parameters
---------- ----------
model_path: path of inference model model_path: path of inference model
@ -305,18 +305,18 @@ performance, but slightly longer startup time).
self.input_shape = self.input_layer.shape self.input_shape = self.input_layer.shape
self.height = self.input_shape[2] self.height = self.input_shape[2]
self.width = self.input_shape[3] self.width = self.input_shape[3]
for layer in self.model.inputs: for layer in self.model.inputs:
input_shape = layer.partial_shape input_shape = layer.partial_shape
input_shape[0] = batchsize input_shape[0] = batchsize
self.model.reshape({layer: input_shape}) self.model.reshape({layer: input_shape})
self.compiled_model = core.compile_model(model=self.model, device_name=device) self.compiled_model = core.compile_model(model=self.model, device_name=device)
self.output_layer = self.compiled_model.output(0) self.output_layer = self.compiled_model.output(0)
def predict(self, input): def predict(self, input):
""" """
Run inference Run inference
Parameters Parameters
---------- ----------
input: array of input data input: array of input data
@ -334,14 +334,14 @@ select device from dropdown list for running inference using OpenVINO
.. code:: ipython3 .. code:: ipython3
import ipywidgets as widgets import ipywidgets as widgets
device = widgets.Dropdown( device = widgets.Dropdown(
options=core.available_devices + ["AUTO"], options=core.available_devices + ["AUTO"],
value="AUTO", value="AUTO",
description="Device:", description="Device:",
disabled=False, disabled=False,
) )
device device
@ -364,11 +364,10 @@ Data Processing
Data Processing includes data preprocess and postprocess functions. Data Processing includes data preprocess and postprocess functions. -
Data preprocess function is used to change the layout and shape of input
- Data preprocess function is used to change the layout and shape of input data, according to requirement of the network input format. - Data
data, according to requirement of the network input format. postprocess function is used to extract the useful information from
- Data postprocess function is used to extract the useful information from
network’s original output and visualize it. network’s original output and visualize it.
.. code:: ipython3 .. code:: ipython3
@ -376,7 +375,7 @@ network’s original output and visualize it.
def preprocess(frame, height, width): def preprocess(frame, height, width):
""" """
Preprocess a single image Preprocess a single image
Parameters Parameters
---------- ----------
frame: input frame frame: input frame
@ -387,12 +386,12 @@ network’s original output and visualize it.
resized_image = resized_image.transpose((2, 0, 1)) resized_image = resized_image.transpose((2, 0, 1))
input_image = np.expand_dims(resized_image, axis=0).astype(np.float32) input_image = np.expand_dims(resized_image, axis=0).astype(np.float32)
return input_image return input_image
def batch_preprocess(img_crops, height, width): def batch_preprocess(img_crops, height, width):
""" """
Preprocess batched images Preprocess batched images
Parameters Parameters
---------- ----------
img_crops: batched input images img_crops: batched input images
@ -401,12 +400,12 @@ network’s original output and visualize it.
""" """
img_batch = np.concatenate([preprocess(img, height, width) for img in img_crops], axis=0) img_batch = np.concatenate([preprocess(img, height, width) for img in img_crops], axis=0)
return img_batch return img_batch
def process_results(h, w, results, thresh=0.5): def process_results(h, w, results, thresh=0.5):
""" """
postprocess detection results postprocess detection results
Parameters Parameters
---------- ----------
h, w: original height and width of input image h, w: original height and width of input image
@ -433,18 +432,18 @@ network’s original output and visualize it.
) )
labels.append(int(label)) labels.append(int(label))
scores.append(float(score)) scores.append(float(score))
if len(boxes) == 0: if len(boxes) == 0:
boxes = np.array([]).reshape(0, 4) boxes = np.array([]).reshape(0, 4)
scores = np.array([]) scores = np.array([])
labels = np.array([]) labels = np.array([])
return np.array(boxes), np.array(scores), np.array(labels) return np.array(boxes), np.array(scores), np.array(labels)
def draw_boxes(img, bbox, identities=None): def draw_boxes(img, bbox, identities=None):
""" """
Draw bounding box in original image Draw bounding box in original image
Parameters Parameters
---------- ----------
img: original image img: original image
@ -470,12 +469,12 @@ network’s original output and visualize it.
2, 2,
) )
return img return img
def cosin_metric(x1, x2): def cosin_metric(x1, x2):
""" """
Calculate the consin distance of two vector Calculate the consin distance of two vector
Parameters Parameters
---------- ----------
x1, x2: input vectors x1, x2: input vectors
@ -502,19 +501,19 @@ Visualize data
image_indices = ["1_1.png", "1_2.png", "2_1.png"] image_indices = ["1_1.png", "1_2.png", "2_1.png"]
image_paths = [utils.download_file(base_file_link + image_index, directory="data") for image_index in image_indices] image_paths = [utils.download_file(base_file_link + image_index, directory="data") for image_index in image_indices]
image1, image2, image3 = [cv2.cvtColor(cv2.imread(str(image_path)), cv2.COLOR_BGR2RGB) for image_path in image_paths] image1, image2, image3 = [cv2.cvtColor(cv2.imread(str(image_path)), cv2.COLOR_BGR2RGB) for image_path in image_paths]
# Define titles with images. # Define titles with images.
data = {"Person 1": image1, "Person 2": image2, "Person 3": image3} data = {"Person 1": image1, "Person 2": image2, "Person 3": image3}
# Create a subplot to visualize images. # Create a subplot to visualize images.
fig, axs = plt.subplots(1, len(data.items()), figsize=(5, 5)) fig, axs = plt.subplots(1, len(data.items()), figsize=(5, 5))
# Fill the subplot. # Fill the subplot.
for ax, (name, image) in zip(axs, data.items()): for ax, (name, image) in zip(axs, data.items()):
ax.axis("off") ax.axis("off")
ax.set_title(name) ax.set_title(name)
ax.imshow(image) ax.imshow(image)
# Display an image. # Display an image.
plt.show(fig) plt.show(fig)
@ -583,7 +582,7 @@ video file.
2. Prepare a set of frames for person tracking. 2. Prepare a set of frames for person tracking.
3. Run AI inference for person tracking. 3. Run AI inference for person tracking.
4. Visualize the results. 4. Visualize the results.
Parameters: Parameters:
---------- ----------
source: The webcam number to feed the video stream with primary webcam set to "0", or the video path. source: The webcam number to feed the video stream with primary webcam set to "0", or the video path.
@ -606,7 +605,7 @@ video file.
if use_popup: if use_popup:
title = "Press ESC to Exit" title = "Press ESC to Exit"
cv2.namedWindow(winname=title, flags=cv2.WINDOW_GUI_NORMAL | cv2.WINDOW_AUTOSIZE) cv2.namedWindow(winname=title, flags=cv2.WINDOW_GUI_NORMAL | cv2.WINDOW_AUTOSIZE)
processing_times = collections.deque() processing_times = collections.deque()
while True: while True:
# Grab the frame. # Grab the frame.
@ -615,11 +614,11 @@ video file.
print("Source ended") print("Source ended")
break break
# If the frame is larger than full HD, reduce size to improve the performance. # If the frame is larger than full HD, reduce size to improve the performance.
# Resize the image and change dims to fit neural network input. # Resize the image and change dims to fit neural network input.
h, w = frame.shape[:2] h, w = frame.shape[:2]
input_image = preprocess(frame, detector.height, detector.width) input_image = preprocess(frame, detector.height, detector.width)
# Measure processing time. # Measure processing time.
start_time = time.time() start_time = time.time()
# Get the results. # Get the results.
@ -628,21 +627,21 @@ video file.
processing_times.append(stop_time - start_time) processing_times.append(stop_time - start_time)
if len(processing_times) > 200: if len(processing_times) > 200:
processing_times.popleft() processing_times.popleft()
_, f_width = frame.shape[:2] _, f_width = frame.shape[:2]
# Mean processing time [ms]. # Mean processing time [ms].
processing_time = np.mean(processing_times) * 1100 processing_time = np.mean(processing_times) * 1100
fps = 1000 / processing_time fps = 1000 / processing_time
# Get poses from detection results. # Get poses from detection results.
bbox_xywh, score, label = process_results(h, w, results=output) bbox_xywh, score, label = process_results(h, w, results=output)
img_crops = [] img_crops = []
for box in bbox_xywh: for box in bbox_xywh:
x1, y1, x2, y2 = xywh_to_xyxy(box, h, w) x1, y1, x2, y2 = xywh_to_xyxy(box, h, w)
img = frame[y1:y2, x1:x2] img = frame[y1:y2, x1:x2]
img_crops.append(img) img_crops.append(img)
# Get reidentification feature of each person. # Get reidentification feature of each person.
if img_crops: if img_crops:
# preprocess # preprocess
@ -650,17 +649,17 @@ video file.
features = extractor.predict(img_batch) features = extractor.predict(img_batch)
else: else:
features = np.array([]) features = np.array([])
# Wrap the detection and reidentification results together # Wrap the detection and reidentification results together
bbox_tlwh = xywh_to_tlwh(bbox_xywh) bbox_tlwh = xywh_to_tlwh(bbox_xywh)
detections = [Detection(bbox_tlwh[i], features[i]) for i in range(features.shape[0])] detections = [Detection(bbox_tlwh[i], features[i]) for i in range(features.shape[0])]
# predict the position of tracking target # predict the position of tracking target
tracker.predict() tracker.predict()
# update tracker # update tracker
tracker.update(detections) tracker.update(detections)
# update bbox identities # update bbox identities
outputs = [] outputs = []
for track in tracker.tracks: for track in tracker.tracks:
@ -672,14 +671,14 @@ video file.
outputs.append(np.array([x1, y1, x2, y2, track_id], dtype=np.int32)) outputs.append(np.array([x1, y1, x2, y2, track_id], dtype=np.int32))
if len(outputs) > 0: if len(outputs) > 0:
outputs = np.stack(outputs, axis=0) outputs = np.stack(outputs, axis=0)
# draw box for visualization # draw box for visualization
if len(outputs) > 0: if len(outputs) > 0:
bbox_tlwh = [] bbox_tlwh = []
bbox_xyxy = outputs[:, :4] bbox_xyxy = outputs[:, :4]
identities = outputs[:, -1] identities = outputs[:, -1]
frame = draw_boxes(frame, bbox_xyxy, identities) frame = draw_boxes(frame, bbox_xyxy, identities)
cv2.putText( cv2.putText(
img=frame, img=frame,
text=f"Inference time: {processing_time:.1f}ms ({fps:.1f} FPS)", text=f"Inference time: {processing_time:.1f}ms ({fps:.1f} FPS)",
@ -690,7 +689,7 @@ video file.
thickness=1, thickness=1,
lineType=cv2.LINE_AA, lineType=cv2.LINE_AA,
) )
if use_popup: if use_popup:
cv2.imshow(winname=title, mat=frame) cv2.imshow(winname=title, mat=frame)
key = cv2.waitKey(1) key = cv2.waitKey(1)
@ -705,7 +704,7 @@ video file.
# Display the image in this notebook. # Display the image in this notebook.
display.clear_output(wait=True) display.clear_output(wait=True)
display.display(i) display.display(i)
# ctrl-c # ctrl-c
except KeyboardInterrupt: except KeyboardInterrupt:
print("Interrupted") print("Interrupted")
@ -759,11 +758,11 @@ will work.
.. code:: ipython3 .. code:: ipython3
USE_WEBCAM = False USE_WEBCAM = False
cam_id = 0 cam_id = 0
video_file = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/video/people.mp4" video_file = "https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/video/people.mp4"
source = cam_id if USE_WEBCAM else video_file source = cam_id if USE_WEBCAM else video_file
run_person_tracking(source=source, flip=USE_WEBCAM, use_popup=False) run_person_tracking(source=source, flip=USE_WEBCAM, use_popup=False)

4
docs/notebooks/person-tracking-with-output_files/person-tracking-with-output_25_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:b26360db4003d149dd356ef7b559c7a7bfb8b4b4b627822abe1dffb8c2f2931a oid sha256:760c838d2fd407c95d5c8df5e0b82b7e132ab78fe46ae939e2319655a1865164
size 218887 size 219822

52
docs/notebooks/photo-maker-with-output.rst
View File

@ -84,7 +84,7 @@ Clone PhotoMaker repository
remote: Counting objects: 100% (143/143), done. remote: Counting objects: 100% (143/143), done.
remote: Compressing objects: 100% (96/96), done. remote: Compressing objects: 100% (96/96), done.
remote: Total 236 (delta 113), reused 64 (delta 47), pack-reused 93 remote: Total 236 (delta 113), reused 64 (delta 47), pack-reused 93
Receiving objects: 100% (236/236), 9.31 MiB | 24.77 MiB/s, done. Receiving objects: 100% (236/236), 9.31 MiB | 25.70 MiB/s, done.
Resolving deltas: 100% (120/120), done. Resolving deltas: 100% (120/120), done.
@ -158,10 +158,12 @@ PhotoMaker to generate the original PhotoMaker pipeline.
.. parsed-literal:: .. parsed-literal::
2024-05-16 00:56:15.653138: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-06-06 01:04:44.222965: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-16 00:56:15.688553: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2024-06-06 01:04:44.256910: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-16 00:56:16.345251: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2024-06-06 01:04:44.912971: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/transformers/transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
.. code:: ipython3 .. code:: ipython3
@ -173,12 +175,6 @@ PhotoMaker to generate the original PhotoMaker pipeline.
pipe = load_original_pytorch_pipeline_components(photomaker_path, base_model_id) pipe = load_original_pytorch_pipeline_components(photomaker_path, base_model_id)
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
.. parsed-literal:: .. parsed-literal::
@ -347,22 +343,13 @@ output(text embeddings) which will be the input for U-Net model.
.. parsed-literal:: .. parsed-literal::
[ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s. [ WARNING ] Please fix your imports. Module %s has been moved to %s. The old module will be deleted in version %s.
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4481: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
.. parsed-literal::
WARNING:nncf:NNCF provides best results with torch==2.2.*, while current torch version is 2.3.0+cpu. If you encounter issues, consider switching to torch==2.2.*
.. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_utils.py:4371: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
warnings.warn( warnings.warn(
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:276: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len): if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:319: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:316: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim): if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/notebooks/photo-maker/PhotoMaker/photomaker/model.py:84: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/notebooks/photo-maker/PhotoMaker/photomaker/model.py:84: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert class_tokens_mask.sum() == stacked_id_embeds.shape[0], f"{class_tokens_mask.sum()} != {stacked_id_embeds.shape[0]}" assert class_tokens_mask.sum() == stacked_id_embeds.shape[0], f"{class_tokens_mask.sum()} != {stacked_id_embeds.shape[0]}"
@ -401,7 +388,7 @@ output(text embeddings) which will be the input for U-Net model.
.. parsed-literal:: .. parsed-literal::
19445 19039
@ -442,11 +429,11 @@ sequence of latent text embeddings.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:86: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if input_shape[-1] > 1 or self.sliding_window is not None: if input_shape[-1] > 1 or self.sliding_window is not None:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if past_key_values_length > 0: if past_key_values_length > 0:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:287: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:284: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len): if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
@ -574,15 +561,15 @@ original Stable Diffusion XL model.
.. parsed-literal:: .. parsed-literal::
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_condition.py:1110: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_condition.py:1114: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if dim % default_overall_up_factor != 0: if dim % default_overall_up_factor != 0:
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:137: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:136: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert hidden_states.shape[1] == self.channels assert hidden_states.shape[1] == self.channels
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/downsampling.py:145: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert hidden_states.shape[1] == self.channels assert hidden_states.shape[1] == self.channels
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert hidden_states.shape[1] == self.channels assert hidden_states.shape[1] == self.channels
/opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-681/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! /opt/home/k8sworker/ci-ai/cibuilds/ov-notebook/OVNotebookOps-697/.workspace/scm/ov-notebook/.venv/lib/python3.8/site-packages/diffusers/models/upsampling.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if hidden_states.shape[0] >= 64: if hidden_states.shape[0] >= 64:
@ -1070,7 +1057,6 @@ Interactive Demo
.. code:: ipython3 .. code:: ipython3
demo.close() demo.close()

4
docs/notebooks/photo-maker-with-output_files/photo-maker-with-output_33_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:ba2722c4b968ff96e1195ce80d01324804b30f97ba21a7edeb9e6d736bb3f507 oid sha256:1c87da41df00051b45c0f4d64d7b6875c6d20586c68ea6f375f02317d38b6522
size 357391 size 357510

4
docs/notebooks/pose-estimation-with-output_files/pose-estimation-with-output_22_0.png
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:1a4f362ade6abe7487381a15c7a75c9caed0404082776188ee121c324329bde7 oid sha256:4d3d81a3fef30928eb8d98479cc71a865e7acbdb41cc1aa8e4d0aed7143e81ac
size 108011 size 108056

Some files were not shown because too many files have changed in this diff Show More