* Model caching support - Core part
Introducing model caching support
Use core.SetConfig({{CONFIG_KEY(CACHE_DIR), <dir>}}); to enable caching of models
OpenVINO will try to create caching folder if it doesn't exist, but it is recommended for client to create caching folder with necessary permissions before enabling cache in config
For caching, plugins shall support import/export functionality
Plugin requirements:
- Add METRIC_KEY(IMPORT_EXPORT_SUPPORT) in SUPPORTED_METRICS to support caching
If plugin has different device architectures with different caches, i.e.
For "GNA.0" - one cache, for "GNA.10" - another cache
In this case plugin shall support DEVICE_ARCHITECTURE metric and return different strings for different DEVICE_ID's
Added functional tests
* Fix CentOS build issues
* Few updates according to code review
* Revert unnecessary changes for Import/Export core implementation
These changes affect old behavior and may be undesired
For caching support these is no need to change anything in this area
If needed, such removal of 'Magic' usage can be done under separate task in future
* More tests:
1) Verify that Imported data from stream is the same as was exported
2) Verify that cache is not loaded when config in LoadNetwork is changed
3) Verify that if CNN Network is changed between ReadNetwork and LoadNetwork - cache is not loaded
* Update of NetworkCompilationContext
Put back functionality of calculating hash based on runtime information, weights
Implemented OstreamHashWrapper to avoid serialization to buffer
* Correction of CACHE_DIR key description
* Unit tests for compilation_context
Changes:
1) Improved handling of OstreamHashAdapter
2) Improved runtime info serialization (not just PrimitivesPriority and affinity)
3) Removed redundant weights hash calculation
* Fix GCC 4.8 build issues
* Compilation context updates
1) Use hash of sum of serialized data to get hash of network. It is more efficient comparing to weights sum calculation
2) CalculateFileInfo - convert path to absolute ("./test.blob" and "test.blob" shall give same hash)
* Hash - added more rt_info attributes + tests
- PrimitivesPriority
- FusedNames
- Dequantization
* Moved "get_absolute_path" macro to file_utils.h
* Make 'absoluteFilePath' a library API, not macro
* One more unit test for fileName hashing
* Fix compilation error after merge with latest master
* Allow tests to be executed in parallel (stress mode)
* More minor updates for stress testing
Now it allows to execute tests with '--repeat=100' option where one test is executed in multiple processes simultaneously
Example:
./gtest-parallel <openvino_dir>/bin/intel64/Debug/ieFuncTests --gtest_filter=CachingTest* --repeat=10
* Use absolute model file path for calculating blob name
* Added 'createDirectoryRecursive' API to plugin_api/file_utils
* added unit test
* added python test
* using pword approach
* Added passing path to onnx reader
* support for wstring
* Added more tests
* Apply suggestions from code review
Co-authored-by: Michał Karzyński <4430709+postrational@users.noreply.github.com>
* fix build for Windows
* styles applied
* Fixed Windows tests
* styles applied
* fixed styles in tests
* review remarks
* cmake order
* Used target_compile_definitions instead of add_definitions
* Move ONNX_TEST_MODELS to other scope
Co-authored-by: Michał Karzyński <4430709+postrational@users.noreply.github.com>
* Implement unicode conversion using Windows native functions
* NOCPPLINT
* Fixed deprecated c++ api usage in tests
* Moved impl to cpp
* Moved Unicode utils to Plugin API
* Added missed include for Windows
* Fixes for unit tests; CentOS fixes
* Fixed Windows compilation
* Fixed unit tests on Unix
* Fixed unix 2
Ilya Lavrenov