70 lines
2.2 KiB
ReStructuredText
70 lines
2.2 KiB
ReStructuredText
.. {#openvino_sample_bert_benchmark}
|
|
|
|
Bert Benchmark Python Sample
|
|
============================
|
|
|
|
|
|
.. meta::
|
|
:description: Learn how to estimate performance of a Bert model using Asynchronous Inference Request (Python) API.
|
|
|
|
|
|
This sample demonstrates how to estimate performance of a Bert model using Asynchronous
|
|
Inference Request API. Unlike `demos <https://docs.openvino.ai/2024/omz_demos.html>`__ this sample does not have
|
|
configurable command line arguments. Feel free to modify sample's source code to
|
|
try out different options.
|
|
|
|
|
|
How It Works
|
|
####################
|
|
|
|
The sample downloads a model and a tokenizer, exports the model to ONNX format, reads the
|
|
exported model and reshapes it to enforce dynamic input shapes. Then, it compiles the
|
|
resulting model, downloads a dataset and runs a benchmark on the dataset.
|
|
|
|
.. scrollbox::
|
|
|
|
.. doxygensnippet:: samples/python/benchmark/bert_benchmark/bert_benchmark.py
|
|
:language: python
|
|
|
|
|
|
You can see the explicit description of each sample step at
|
|
:doc:`Integration Steps <../../openvino-workflow/running-inference/integrate-openvino-with-your-application>`
|
|
section of "Integrate OpenVINO™ Runtime with Your Application" guide.
|
|
|
|
Running
|
|
####################
|
|
|
|
1. Install the ``openvino`` Python package:
|
|
|
|
.. code-block:: console
|
|
|
|
python -m pip install openvino
|
|
|
|
|
|
2. Install packages from ``requirements.txt``:
|
|
|
|
.. code-block:: console
|
|
|
|
python -m pip install -r requirements.txt
|
|
|
|
3. Run the sample
|
|
|
|
.. code-block:: console
|
|
|
|
python bert_benchmark.py
|
|
|
|
|
|
Sample Output
|
|
####################
|
|
|
|
The sample outputs how long it takes to process a dataset.
|
|
|
|
Additional Resources
|
|
####################
|
|
|
|
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <../../openvino-workflow/running-inference/integrate-openvino-with-your-application>`
|
|
- :doc:`Get Started with Samples <get-started-demos>`
|
|
- :doc:`Using OpenVINO Samples <../openvino-samples>`
|
|
- :doc:`Convert a Model <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api>`
|
|
- `Bert Benchmark Python Sample on Github <https://github.com/openvinotoolkit/openvino/blob/master/samples/python/benchmark/bert_benchmark/README.md>`__
|