2bf8d910f6
* [Docs][PyOV] update python snippets * first snippet * Fix samples debug * Fix linter * part1 * Fix speech sample * update model state snippet * add serialize * add temp dir * CPU snippets update (#134) * snippets CPU 1/6 * snippets CPU 2/6 * snippets CPU 3/6 * snippets CPU 4/6 * snippets CPU 5/6 * snippets CPU 6/6 * make module TODO: REMEMBER ABOUT EXPORTING PYTONPATH ON CIs ETC * Add static model creation in snippets for CPU * export_comp_model done * leftovers * apply comments * apply comments -- properties * small fixes * rempve debug info * return IENetwork instead of Function * apply comments * revert precision change in common snippets * update opset * [PyOV] Edit docs for the rest of plugins (#136) * modify main.py * GNA snippets * GPU snippets * AUTO snippets * MULTI snippets * HETERO snippets * Added properties * update gna * more samples * Update docs/OV_Runtime_UG/model_state_intro.md * Update docs/OV_Runtime_UG/model_state_intro.md * attempt1 fix ci * new approach to test * temporary remove some files from run * revert cmake changes * fix ci * fix snippet * fix py_exclusive snippet * fix preprocessing snippet * clean-up main * remove numpy installation in gha * check for GPU * add logger * iexclude main * main update * temp * Temp2 * Temp2 * temp * Revert temp * add property execution devices * hide output from samples --------- Co-authored-by: p-wysocki <przemyslaw.wysocki@intel.com> Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> |
||
|---|---|---|
| .. | ||
| README.md | ||
| bert_benchmark.py | ||
| requirements.txt | ||
README.md
Bert Benchmark Python Sample
@sphinxdirective
.. meta:: :description: Learn how to estimate performance of a Bert model using Asynchronous Inference Request (Python) API.
This sample demonstrates how to estimate performance of a Bert model using Asynchronous Inference Request API. Unlike :doc:demos <omz_demos> this sample doesn't have configurable command line arguments. Feel free to modify sample's source code to try out different options.
The following Python API is used in the application:
.. tab-set::
.. tab-item:: Python API
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Feature | API | Description |
+================================+=================================================+==============================================+
| OpenVINO Runtime Version | [openvino.runtime.get_version] | Get Openvino API version. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Basic Infer Flow | [openvino.runtime.Core], | Common API to do inference: compile a model. |
| | [openvino.runtime.Core.compile_model] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], | Do asynchronous inference. |
| | [openvino.runtime.AsyncInferQueue.start_async], | |
| | [openvino.runtime.AsyncInferQueue.wait_all] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Model Operations | [openvino.runtime.CompiledModel.inputs] | Get inputs of a model. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/benchmark/bert_benchmark/bert_benchmark.py
:language: python
How It Works ####################
The sample downloads a model and a tokenizer, export the model to onnx, reads the exported model and reshapes it to enforce dynamic input shapes, compiles the resulting model, downloads a dataset and runs benchmarking on the dataset.
You can see the explicit description of
each sample step at :doc:Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application> section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running ####################
Install the openvino Python package:
.. code-block:: sh
python -m pip install openvino
Install packages from requirements.txt:
.. code-block:: sh
python -m pip install -r requirements.txt
Run the sample
.. code-block:: sh
python bert_benchmark.py
Sample Output ####################
The sample outputs how long it takes to process a dataset.
See Also ####################
- :doc:
Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application> - :doc:
Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview> - :doc:
Model Downloader <omz_tools_downloader> - :doc:
Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>
@endsphinxdirective