Commit Graph

362 Commits

Author SHA1 Message Date
Ilya Naumov 149c43044c
Set less strict numpy requirements (#6039) 2021-06-04 20:55:22 +03:00
Pavel Esir d9d63d0d7e
[MO] fix MarkNodesWithShapeValues (#5808)
* fix marking of nodes with shape values: corrected 'leaky' starting condition

* cleaned up code

* fixed t2t type collision error: removed strict assert from Pow; now type_infer is same as for other eltwise ops; added warnings about type alignment

* type cast for Pow according to ONNX spec

* fix highlight in dump_graph_for_graphviz

* soft_get('op')

* Revert "soft_get('op')"

This reverts commit cadfe18f

* Revert "type cast for Pow according to ONNX spec"

This reverts commit bf85ebf8

* applied review comments

* comment for ShapeOf bfs, removed Error from elementwise op

* reverted back dump_graph_for_graphviz: fill_color -> node_attrs

* applied last review comments: returned back Error, added type_infer into unit-test utils
2021-06-04 18:49:49 +03:00
Evgeny Lazarev 0db9d3e2c5
Added extractor for ONNX operation Size (#6031)
* Added extractor for ONNX operation Size

* Moved transformation of Size operation from TF specific to generic front phase

* Updated list of supported ONNX operation

* Moved unit test for Size decomposition to a new location
2021-06-04 17:18:01 +03:00
Eugeny Volosenkov 859a3b8a30
Deprecated warnings fix in MO (#6016)
* Add code style fixes

* Revert "Add code style fixes"

This reverts commit 490934f243.

* Fix Invalid escape sequence

* Fix Invalid escape sequence
2021-06-04 16:53:30 +03:00
Anastasia Popova 3825e90ae1
Fix of RollWithEmptyAxesReplacer transformation. (#5937)
* Fixed Roll output shape bug in RollWithEmptyAxesReplacer transformations.

* Added check in unit test.
2021-06-03 14:45:05 +03:00
Anton Chetverikov f88222933a
[MO] Add extractor for ONNX *OrEqual operations (#5932)
* Add extractor for ONNX GreaterOrEqual operation

* Add more extractors

* Fix extractor

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-06-02 09:06:41 +03:00
Anastasia Popova b1e7a5cdb1
Fixed duplicate names appearance in graph after fuse mul transformation. (#5842)
* Added manual set of unique name to the data node.

* Added attributes checks in unit test.

* Removed redundant line break.
2021-05-28 16:37:43 +03:00
Anton Chetverikov 9d3780648f
Add missed attributes for Parameter layer (#5676)
* Add shape and element_type to ir_v10_attrs structure

* Add comment

* Update add_input_op_input_port_with_data() and add_input_op_output_port_with_data() functions with new GraphAPI

* Remove incorrectly added test

* Apply comments

* Update missed tests

* Add more checks to tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-05-28 10:22:06 +03:00
Gleb Kazantaev 98ac1d04c3
Fix IE Python API search inside MO for Windows platform (#5862) 2021-05-28 09:57:16 +03:00
Anastasia Popova a9230a916b
LSTMCell tensor names fix (#5696)
* Added handling of debug information in create_node().

* Code refactoring.

* Checks fixed.

* Added comments, added unit test.

* Renamed unit test class.

* Fixed port number in unit test.
2021-05-27 15:13:44 +03:00
Evgenya Stepyreva ffc9c95a2b
Revert "Fuse mul transformation fix (#5518)" (#5831)
This reverts commit 84b94c9834.
2021-05-26 16:07:46 +03:00
Maxim Vafin d8d9819b5f
Fix security issue with XML parsing (#5686)
* Fix security issue with XML parsing

* Apply review feedback

* Rework defusing stdlib solution

* Apply review feedback
2021-05-26 14:17:55 +03:00
Gleb Kazantaev 7933e9f61d
Find ie api based on platform (#5820)
Co-authored-by: Gleb Kazantaev <gleb@nncv666el.inn.intel.com>
2021-05-26 09:07:37 +02:00
Evgenya Stepyreva 74293c54df
Transpose FQ optimization (#5763)
* Transpose FQ optimization

* Tests added
2021-05-25 16:42:21 +03:00
Evgeny Lazarev 43bdf6b3c1
Fix pad op with int input 2021.4 (#5772)
* Added operation ConvertLike to the MO

* Fixed transformations with Pad which insert Const with pad value of incorrect type

* Added constant folding to ConvertLike operation

* Fixed unit tests for Pad transformations to include ConverLike operations

* Update copyright year

* nGraph code style fix
2021-05-25 11:55:27 +03:00
Anastasia Popova 04de4e34bc
Additional telemetry events in MO. (#5662)
* Added additional telemetry events.

* Separated sending tf1 and tf2.

* Small correction.

* Unit test fix.

* Added op_names_statistic field in graph. Added op names saving in loop ext, while ext.

* Optimize imports.

* Added debug print.

* Added comments, removed debug print.

* Added comment.

* Renamed dynamic shapes event label to partially defined, added unit tests.

* Added attribute checks, moved telemetry methods to separate file.

* Small corrections.

* Updated BOM file.
2021-05-24 15:21:29 +03:00
Vladimir Gavrilov 2bc69da1d1
Fix in model-optimizer/mo/back/ie_ir_ver_2/emitter.py. (#5742) 2021-05-21 18:58:03 +03:00
Yegor Kruglov 40c5d116e3
Updated condition in metadata depersonalize function (#5655)
* updated condition in metadata depersonalize function

* renamed list with keys

* removed unnecessary key
2021-05-21 18:08:57 +03:00
Svetlana Dolinina 8e956654bf
fix stride for 1d maxpooling (#5710)
* fix stride for 1d maxpooling

* fix test
2021-05-21 11:57:40 +03:00
Roman Kazantsev d98e954f5d
Make order of port names determined in IR (#5701)
* Make order of port names determined in IR

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Make port names in determined order and adopted tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-05-20 13:44:06 +03:00
Anastasia Kazantaeva e755232a7f
fix date (#5694) 2021-05-19 22:21:51 +03:00
Anton Chetverikov 538f407829
Fix errors for VariadicSplit layer restored from serialized IR (#5613)
* Fix errors in VariadicSplit layer restored from serialized IR

* Update VariadicSplit specification and error message to allow 1D tensors on 1st input

* Update spec

* Resolve comments

* Apply comments, add unit tests

* Update unit tests
2021-05-19 19:03:58 +03:00
Evgeny Lazarev 369b427657
Fixed MO CMakeList to remove unit_tests directory (#5670) 2021-05-18 20:37:54 +03:00
Gleb Kazantaev 8be740a5aa
Enable Pruning transformation by default inside Model Optimizer (#5633) 2021-05-18 12:48:01 +03:00
iliya mironov 642a62d0e2
Add axis support (#5242)
* Add axis support

* Update dequant extractor

* Update qdeq ops

* Refactoring quantize

* Update dequantize resolver

* Update dequantize op

* Refactoring dequantize

* Some fixes for quantize and dequantize

* Update unit tests

* Reafctoring quantize/dequantize axis support

* Move quantize/dequantize resolvers to middle

* hot fix

* Fix unit tests

* Fix unit tests

* Update quintize resolver comment

* Refactoring code according to code review

* Fix according to review

* Change order for transforms quantize pipline
2021-05-17 15:03:22 +03:00
Evgeny Lazarev 717499cf2f
Common telemetry (#5032)
* Moved telemetry to repo root directory from MO

* New telemetry package in the "openvino" sub-directory

* Removed telemetry from the MO BOM

* Updated MO BOM and added stub file for telemetry

* Fixed license header

* Fixed license headers and cleaned up the telemetry setup.py

* Fixed import

* Added temporary dependency for openvino-telemetry

* Added ignore for pylint issues

* Fixed import statements

* Updated imports in the telemetry library

* Removed telemetry library. Added link to another private repo

* Removed redundant start_session event for the MO

* Changed approach to import the telemetry library

* Minor code refactoring

* Updated MO telemetry events sending messages

* Refactor sending events for the IE runtime check

* Disable forcing warnings for deprecated methods

* Removed changes from the requirements.txt to install telemetry library to avoid merge conflicts

* Update copyright in the model-optimizer/mo/utils/telemetry_stub.py

Co-authored-by: Gleb Kazantaev <gleb.nnstu@gmail.com>

Co-authored-by: Gleb Kazantaev <gleb.nnstu@gmail.com>
2021-05-14 21:56:03 +03:00
Ilya Naumov 3e076a88aa
Align MO pip requirements (#5171)
* Align MO pip requirements

* fix missing mxnet on platforms other than Windows or Linux

* remove spaces near environment markers
2021-05-14 16:05:25 +03:00
Sergey Lyubimtsev 5af767a0a2
Add MO extras requires for supported DL frameworks (#5615)
* Add extras requires for supported DL frameworks

* fix pylint issues

* codestyle (PEP8)

* Indentation
2021-05-13 20:20:41 +03:00
Anton Chetverikov f2f44ce160
Add uint64 to data types maps (#5603) 2021-05-12 18:50:38 +03:00
Gleb Kazantaev b4565b7b4f
Add Model Optimizer --transform option (#5504)
* Execute MO stages inside subprocess to have single IE check

* Add --transform key

* Updated ofline transformations to execute user specified passes; updated logic to raise when --transform is used

* Parametrize LowLatency transformation with num_iterations key

* Fixed MO and IE versions comparision

* Use subprocess for offline transformations execution to catch errors

* remove ie_is_available from IR; fixed typo

* Fix for old IE versions

* Update parse_transform key unit tests

* Show available transformations

* Fixed typo

* Fix review comments

* Fix python2 compatibility

* Fixed review comments

* Fixed __main__ import
2021-05-12 18:42:56 +03:00
iliya mironov fe5ca28b6e
Smal fix with tf env_setup (#5476)
* Smal fix with tf env_setup

* Fix tf loader

* Fix version checker
2021-05-12 15:33:03 +03:00
Pavel Esir 0b22d6c51c
Add Gather-7 to MO (#5264)
* initial solution

* added unit-tests + some corrections

* axis getting improvements

* fixed MO IR reader for old IR's

* a couple of corrections

* applied review comments

* corrected negative batch_dims normalization for shape calculation, for IR original negative values are kept

* added additional checks and negative tests
2021-05-11 22:29:59 +03:00
Roman Kazantsev dc22c177d5
Extend MO for operation Einsum-7 (#5401)
* Extend MO for operation Einsum-7

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Add extractor for einsum and optimize code based on review feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix the code based on the review: correct code, tests and comments; move insert_transpose

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix LayoutChangeForEinsum transformation condition

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update third-party dependencies

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-05-11 21:36:04 +03:00
Anastasia Kazantaeva 42226fccae
Upgrade message for 2021.4 lts (#5471) 2021-05-11 15:58:32 +03:00
Anastasia Popova 84b94c9834
Fuse mul transformation fix (#5518)
* Changed fuse_mul behaviour for proper data node connection.

* Corrected the comment.

* Corrected the comment.

* Added permutation attribute saving.

* Added comment.

* Added unit tests, comments corrections.
2021-05-07 17:01:31 +03:00
Anastasia Popova 4ea09b1f98
ReverseInputChannels mapping fix (#5523)
* Fixed attributes saving to keep tensor debug info in Parameter node.

* Added comment and unit tests.

* Small correction.

* Small correction of unit test.

* Comment corrected.
2021-05-07 16:29:38 +03:00
Vladimir Gavrilov 0c288d506c
MO support for operations DFT and IDFT (#5197)
* Written MO classes for DFT and IDFT operations.

* Added class to read TF (I)FFT operations.

* Written extractors for TF operations FFT, FFT2D, FFT3D, IFFT, IFFT2D, IFFT3D.

* Written MO Roll operation and TF Roll operation extractor.

* Started to write needed transformations.

* Written transformation StridedSlices + Complex + Roll + (i)FFTxD + Roll + (Imag, Real) + Pack -> Roll + (I)DFT + Roll.

* Written transformation for Complex + ComplexAbs.

* Written correction of axes of Roll.

* Small fix.

* Small fix.

* Some fixes.

* Some changes.

* Now TF Roll is read as TFRoll. Written inserting Transposes before and after (I)DFT.

* Small fix.

* Written tests for the transformation TFRollToRoll.

* Added comments to some transformations.

* Deleted redundant import.

* Written tests for the transformation TransposeDFT.

* Fixes in MO IR Reader to read/write (I)DFT.

* Fixes in the list of supported TF layers.

* Started to write tests for SSliceComplexRolledFFTPackBlockReplacement transformation.

* Written tests for the MO transformation SSliceComplexRolledFFTPackBlockReplacement.

* Written tests for the MO transformation ComplexAbs.

* Tests for transformations were moved into unit_tests directory.

* All extractors for (I)FFTxD are in one file now.

* Deleted redundant transformations.

* Fixed extractor for TF Roll: now this operation is read as MO Roll.

* Added comments to TFFFT operation.

* The method insert_transpose of classes TransposeDFT and LayoutChangeForGatherND was moved into the separate function in the file model-optimizer/extensions/middle/InsertLayoutPropagationTransposes.py.

* Fixed comment for the transformation TransposeDFT.

* Small fix.

* Some fixes.

* Deleted shape infer function for the operation TFFFT. Sorted imports in complex_abs.py.

* Small fixes.

* Deleted redundant import.

* Fixes in some asserts.

* Small fix.

* Added names for created nodes in the transformation ComplexAbs.

* Added comments to the method canonicalize_axes.

* The transformation SSliceComplexRolledFFTPackBlockReplacement was split into the sequence of transformations SSliceComplexRollReplacement -> RollRealImagPackReplacement -> TFFFTToDFT.

* Written tests for the transformation SSliceComplexRollReplacement.

* Written tests for the transformation RollRealImagPackReplacement.

* Written tests for the transformation TFFFTToDFT.

* Deleted commented code.

* Fixed types of constants in the transformation ComplexAbs.

* Written tests for canonicalization of signal_size value.

* Deleted 'Replacement' from names of files and classes.

* Used comarison of ids, not names.

* replace_sub_graph was replaced with find_and_replace_pattern.

* Now the transformation RollRealImagPack is executed before running transformation model-optimizer/extensions/front/Pack.py.

* The body of the function create_dft_from_tffft is a part of the transformation TFFFTToDFT body now.

* Now method correct_roll_axes of classes RollRealImagPack and SSliceComplexRoll is moved to the function in mo/front/tf/graph_utils.py.

* Small changes.

* Added comment before mark_input_as_in_correct_layout(roll, 2).

* Now the functions correct_roll_axes generates sub-graph in the input port 2 of Roll.

* Corrected tests for the transformation SSliceComplexRoll.

* Corrected tests for the transformation RollRealImagPack.

* Deleted commented code.

* Some renaming.

* Added decomposition of the separate operation ComplexAbs (without Complex before it).

* Added comment to the transformation ComplexAbsAfterComplex.

* Optimized imports for the transformation TFFFTToDFT.

* The transformation SSliceComplexRoll was split into the sequence SSliceComplex -> CorrectRollAxes and disabled.

* Written tests for the transformation ComplexAbs.

* Written tests for the transformation SSliceComplex.

* Written tests for the transformation CorrectRollAxes.

* Deleted the transformation SSliceComplexRoll.

* Deleted renaming nodes.

* Fixed comment.

* Small fixes.

* Small fix.

* The attribute need_correction was renamed as input_rank_changed.

* Small fixes.

* Deleted commented code.

* Now we iterate over all complex_node.out_port(0).get_connection().get_destinations() input ports and mark the corresponding nodes with the marker attribute.

* Added the attribute 'in_ports_count' into the class FFTBase.

* Tests for the transformation TransposeDFT were rewritten using helper functions.

* Now the transformation RollRealImagPack uses existing Roll node instead of creating new one.

* Small fixes.

* Fix in the documentation.

* Written class to read MxNet (I)FFT operations. Written corresponding extractors.

* Corrected shape infer function for MXFFT operation. Written transformation to convert MXFFT to (I)DFT.

* Fixed shape infer function.

* Fixed the conversion MXFFT to (I)DFT.

* Written tests for the transformation MXFFTToDFT.

* The function correct_roll_axes was replaced with more generic function add_constant_to_negative_values.

* Fixes in classes TFFFT, FFTBase, DFT, IDFT, MXFFT.

* Added asserts in constructors of operations TFFFT and MXFFT.

* Refactored transformation MXFFTToDFT: conversion of DFT and IDFT were moved into separated functions.

* Moved some commented code.

* Fixed BOM file.

* Written function convert_ifft_to_dft.

* Started to rewrite tests for MXFFTToDFT transformations, in the case is_inverse=False.

* Small fixes.

* Fixes in the transformation RollRealImagPack.

* Renaming tests class for the transformation SSliceComplex.

* Fixes in the function compare_graphs. Now we get all output nodes of op node, and these output nodes are sorted by names.

* Fixed tests for the transformation MXFFTToDFT.

* Fix in the transformation ThresholdedReluDecomposition: added disconnect for trelu input port.

* Fixes in test for the transformation TFSliceToSlice.

* Small fix in the transformation ObjectDetectionAPIPreprocessor2Replacement.

* Small fix in comment.

* Optimized imports.

* Used remove_node in the transformation ThresholdedReluDecomposition and remove_nodes_from in the transformation RollRealImagPack, instead of ports disconnection.

* Deleted commented code.

* Deleted test case test_slice_replacer_begin_with_2_inputs.
2021-05-07 09:44:24 +03:00
Anton Chetverikov b3c1dd9de3
Add sys_platform environment marker to version checker (#5437)
* Add sys_platform environment marker

* Update sys_platform check

* Add unit tests for sys_platform marker

* apply review comments

* Fix typo

* Update checker and tests, apply comments

* Update comments parsing and tests

* Fix commrnt

* Resolve comments and update check logic

* Update tests and fix bug with negative tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-05-06 16:20:37 +03:00
Yegor Kruglov abb1ca657e
Implementation of ArgMin ONNX + TF extractors (#5126)
* implement argmin extractors

* reconsidering argmax to topk

* arg ops refactoring

* rename ArgMaxToTopK

* added unittests

* update docs

* move unittest file to new folder

* conversations resolving

* revert changes with argmax.py, move argmin op to a new file

* rename ArgMaxSqueeze

* updated BOM file

* little fix

* code refactoring in ArgMaxOp, updated unittests

Co-authored-by: yegor.kruglov <ykruglov@nnlvdp-mkaglins.inn.intel.com>
2021-05-06 13:41:49 +03:00
Yegor Kruglov e3ea9bf4dd
fix incorrect input names for mean values (#5508) 2021-05-06 13:33:51 +03:00
Evgeny Lazarev 98783b88ac
Removed transformation which removes Const->Result sub-graphs (#5295)
* Removed transformation which removes Const->Result sub-graphs

* Removed one more MO transformation which removes Const->Result sub-graph during the front phase
2021-05-05 14:24:05 +03:00
Gleb Kazantaev b291ca8cfa
Use Serialization as a default engine in MO (#5347)
* Use Serialization as a default engine in MO

* Added cmd option to use old serialization

* Added mapping file generation

* Test mapping file generation

* Fix setBatchsize parameters order; fix mapping file generation

* Added FrameworkNode; added method to read models with custom ops but without extensions

* Added python API for read_network_without_extensions function; updated mo not to use IECore

* Added read_model_without_extensions to IReader and IParser

* Fix V7 IR reader

* Fix pword value

* Fix dllexport macro usage

* Add metainfo to IR

* Fix nGraph code style

* Fix license header

* Restore prepare_emit_ir behaviour

* Fix compare_function to resolve situation when Result input port has multiple names

* Update Compare Functions

* Fix FrameworkNode validation

* Self-review

* CodeStyle check

* --use_fallback -> --use_legacy_ir_generation

* Sort imports in main.py

* --path_to_model -> --input_model

* Use logging instead of print

* Code simplifucation&cleanup

* Fix offline_Transformations key

* Fix GeneraeMappingFile comments

* Use Extension approach to work with custom ops

* Fix versions check

* Code clean-up

* Moved FrameworkNode to inference_engine_transformations library

* Fix FrameworkNode includes

* Code clean-up
2021-05-04 16:40:20 +03:00
Svetlana Dolinina 7b52e3155a
Tdnnf (#5255)
* initial changes (IR not generated)

* extractor fix

* convert tdnnf (with correct infer)

* refactoring + comments in code

* added unit tests + couple fixes based on tests

* change order for old convolutions

* fix pylint

* small refactoring

* added if to remove changes in old irs

* doc updated

* fix layout and kernel shapes for old convolutions

* fixed test

* moved test

* fix import in test

* fixed backward compatibility

* review fixes
2021-04-29 14:38:08 +03:00
Anastasia Popova 07214d0a47
Changed "out_port_id" attribute setting in mapping file to store tensor names. (#5344)
* Removed port id from fw_tensor_debug_info attribute.

* Added port number to tensor names in kaldi, mxnet. Fixed Const naming.

* Sort imports.
2021-04-29 14:05:35 +03:00
Svetlana Dolinina 4daa74303a
changed output finding + add error to catch problem in future (#5368)
* changed output finding + add error to catch problem in future

* added unit test

* review fixes

* review fixes

* fix typo
2021-04-28 12:59:59 +03:00
Anton Chetverikov 8395ce1b0f
Update modules version checker in MO to support `~=` (#5342)
* Add check for ~= sign for modules versions check

* Add unit test

* Update check for environment markers

* Update check for ~= sign

* Update unit tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-28 11:23:08 +03:00
Sergey Lyubimtsev 7d5fd3d51c
MO Transformation configs are missing in openvino-dev package (#5302)
* MO Transformation configs are missing in openvino-dev package

* fix pylint issues

* add escape

Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>

* add escape

Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>

* add mo_caffe.proto

* Add MO package_data to openvino-dev package

Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>
2021-04-28 10:15:22 +03:00
Svetlana Dolinina 3028c78594
fix typo + test (#5392) 2021-04-27 13:37:02 +03:00
Anton Chetverikov 576e692b1d
Update errors raising cases in modules versions checker in MO (#5141)
* Change error raising to message printing

* Update modules versions check

* Update check and add unit test

* Fix wrong function call

* Update unit tests

* Fix test

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-23 13:23:06 +03:00
Yegor Kruglov d2c4d068b5
fixed mismatch of input names between framework and MO .mapping file (#5256) 2021-04-21 18:46:49 +03:00