Commit Graph

116 Commits

Author SHA1 Message Date
Maxim Vafin b8ced3dfc7
Change opt-125m-gptq model (#24899)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-06-07 14:50:30 +04:00
Andrii Staikov be5fd8b6d4
Add PagedAttention tests to Precommit (#24634)
### Details:
Add tests for the SPDAToPagedAttention transformation to Precommit
1) Check if PagedAttentionExtension node appeared in the model after the
transformation
2) Check 2 last dimensions of 'key_cache.' and 'value_cache.' inputs to
be static.

### Tickets:
 - [CVS-138943](https://jira.devtools.intel.com/browse/CVS-138943)

Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
2024-05-30 08:58:09 +00:00
Ilya Lavrenov 72028b7bbc
Use CPU torch (#24658)
### Details:
- See
https://github.com/openvinotoolkit/openvino/actions/runs/9195359216/job/25292054679#step:16:224
2024-05-25 22:10:38 +04:00
Maxim Vafin e9d42dd60b
[PT FE] Fix regression in easyocr model (#24586)
### Details:
- *After support for i64 was added in frontend `aten::adaptive_avg_pool`
started to fail if shape input is list.*

### Tickets:
 - *CVS-141335*
2024-05-20 16:51:30 +00:00
Roman Kazantsev da66abcbe9
[Model Hub Tests] Fix caching for read_model scenario (#24355)
**Details:** Fix caching for read_model scenario

**Ticket:** TBD

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-05-15 13:58:54 +00:00
Roman Kazantsev fb838f2f20
[TF FE] Support TF Hub openimages-v4-ssd-mobilenet-v2 model (#24474)
**Details:** LookupTableImport operation node can be terminating in a
graph and needs to be preserved in a graph for further HashTable
operation resolving.

**Ticket:** TBD

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-05-13 12:48:16 +00:00
Andrey Babushkin 62d55aafd1
Switch Tensorflow models tests back on Azure runners (#24038)
### Details:
This switches TF Hub model tests back to self-hosted runners in AKS and
stores models in persistent storage in Azure
2024-04-25 18:43:03 +00:00
Evgeny Kotov 07fd0f97df
read_model vs ovc.convert performance tests fix OVVP output data (#23724)
### Details:
 - send tests start datetime in version field

### Tickets:
 - 137070
2024-04-23 12:56:41 +00:00
Roman Kazantsev aa038b8776
[TF FE] Support non-default signature and add read_model scenario to validation (#24153)
**Details:** Fix model input and output names in case absence of serving
signature. mil-nce model from TF Hub does not have default signature
("serving") so use user names from other signatures to resolve internal
tensor names.
Also, it adds validation for `read_model` scenario.

**Tickets:** 139245, 124401

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-04-23 07:33:58 +00:00
Maxim Vafin 03fa47a2ff
Refactor hf GHA tests (#24048)
### Details:
 - *Refactor hf tests*

### Tickets:
 - *ticket-id*
2024-04-16 14:27:34 +00:00
Maxim Vafin 2eb09a4d07
[FX] Remove unused code from FX decoder (#23954)
### Details:
 - *Remove unused code from FX decoder*

### Tickets:
 - *ticket-id*
2024-04-11 12:42:52 +00:00
Anastasiia Pnevskaia 81599b790b
[TF FE] Increased HF nightly scope. (#23480)
### Details:
 - Increased HF nightly scope.

### Tickets:
 -
2024-04-05 12:48:44 +00:00
Maxim Vafin 4ef40f0f0e
[PT FE] Fix GHA model tests (#23846)
### Details:
 - *Added `kornia` dependency for lightglue model*
 - *Disable GFPGAN since it doesn't support latest `torchvision`*
 - *Remove `transfo-xl-wt103` as not supported on latest `transformers`*
 - *Remove XFAIL from `facebook/levit-128S`*

### Tickets:
 - *ticket-id*
2024-04-05 06:30:50 +00:00
Maxim Vafin ee2962a50c
[OVC] Allow models obtained from torch.export (#23815)
### Details:
 - *Allow models obtained from `torch.export`*

### Tickets:
 - *ticket-id*
2024-04-03 05:56:46 +00:00
Maxim Vafin fba93854cb
[PT FE] Fix issue with aten.copy in FX graph (#23711)
### Details:
 - *Fix translation for `aten.copy.default`*
 - *Add support for `aten.rand.default`*
 - *Support `vit-mae` model*
 - *Update hf model list with newly added models since list creation*

### Tickets:
 - *CVS-133732*
2024-03-28 13:01:06 +00:00
Maxim Vafin 99715bf875
[PT FE] Add more operators for FX graph (#23698)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-27 07:46:53 +00:00
Evgeny Kotov ab64489d9b
Implement performance tests in nightly validation to compare read_model and ovc.convert with OVVP output (#23303)
### Details:
 - add nightly performance tests with skip list
 - [restored PR](https://github.com/openvinotoolkit/openvino/pull/21353)
- [restored PR
#2](https://github.com/openvinotoolkit/openvino/pull/22325)
 - added OVVP support
 - extend skip list with the models that have unstable results

### Tickets:
 - 126282
2024-03-27 06:02:01 +00:00
Maxim Vafin b10877f6e8
[PT FE] Resolve i64 usage in frontend (#23518)
### Details:
 - *Support lightglue*
 - *Support int64 as scalar in PyTorch Frontend*

### Tickets:
 - *CVS-127326, CVS-134328*
2024-03-26 15:57:00 +00:00
Roman Kazantsev 1ab9bedd81
[TF FE] Support TensorFlow 2.16 (#23562)
**Details:** Support TensorFlow 2.16

**Ticket:** TBD

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-03-21 20:39:39 +00:00
Anastasiia Pnevskaia 432ea63e0c
[TF FE] Fixed memory sharing for HF models. (#23318)
### Details:
 There were two problems with memory sharing, which affected HF models. 
`value()` method of class `ResourceVariable` copies value of variable in
some cases. This results in sharing of memory with a copied value, which
is deallocated when `GraphIterator` finishes its work.
Replaced `value()` with `read_value_no_copy()` which should not copy
value in any case.

Another problem is that original model inference moves weights to new
location which leads to corruption of shared memory in OV model.
So the original model should be inferred after the OV inference to
prevent any changes in original model.

### Tickets:
 - 134787, 134408

---------

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-03-15 08:46:20 +00:00
Anastasiia Pnevskaia 579ffc3a7b
[TF FE] Fixed comparison method in TF Hugging Face tests for case when output structure is not the same as in FW (#23269)
### Details:
- Tests for Hugging Face models fail when output structure has
dictionary of tuples. For such case comparison method was changed to
compare outputs using original model output signature, which has inner
tensor names that can be matched with OV output.

### Tickets:
 - 134405

---------

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-03-14 12:50:54 +00:00
Jacek Pawlak 239466ca5d
copyright year update (#23370)
New PR due to merge
conflicts(https://github.com/openvinotoolkit/openvino/pull/22917)

Updated the copyright year from 2018-2023 to 2018-2024 in all openvino
files

Ref. to script: CVS-101144

Command used:
```bash
git grep -lz '2018-2023 Intel Corporation' | xargs -0 sed -i '' -e 's/2018-2023 Intel Corporation/2018-2024 Intel Corporation/g'
```
2024-03-14 09:37:02 +00:00
Maxim Vafin ad8d6ef35d
[PT FE] Support aten.reflection_pad{1,2,3}d.default (#23379)
### Details:
- *Support `aten.reflection_pad1d.default`,
`aten.reflection_pad2d.default` and `aten.reflection_pad3d.default`*

### Tickets:
 - *ticket-id*
2024-03-13 22:39:37 +00:00
Anastasiia Pnevskaia f55e967dd5
[TF FE] Remove paraphrase model from precommit (#23299)
### Details:
- sentence-transformers/paraphrase-mpnet-base-v2 randomly fails on
accuracy.
 - Ticket for fixing - 134787

### Tickets:
 -

---------

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-03-06 19:11:28 +00:00
Maxim Vafin 4f5c2a7a93
Change model since previous persimmon model was removed (#23201)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-05 10:29:11 +00:00
Maxim Vafin 4edb040869
Fix timm filter and run timm models in parallel (#23116)
### Details:
 - *Update list of timm models to comply with new version of timm*
- *Run timm models and torchvision models in trace and export model in
parallel*

### Tickets:
 - *ticket-id*
2024-03-01 10:59:32 +00:00
Maxim Vafin 8b792927fa
[PT FE] Support multiple FX operations (#23112)
### Details:
 - *Unpack nested tuples at model outputs*
 - *Support for `aten.cat.default` with default axis*
 - *Support `aten::reciprocal_` and `aten::abs_` for TorchScript case*
 - *Support:*
    *  `aten.adaptive_max_pool3d.default`
    *  `aten.adaptive_max_pool{N}d.default`
    *  `aten.amin.default`
    *  `aten.argmin.default`
    *  `aten.bitwise_{XXX}.default`
    *  `aten.clamp_max.default`
    *  `aten.clamp_{max/min}.Tensor`
    *  `aten.fill.Tensor`
    *  `aten.flip.default`
    *  `aten.fmod.{Scalar/Tensor}`
    *  `aten.ge.{Scalar/Tensor}`
    *  `aten.gt.Tensor`
    *  `aten.index_select.default`
    *  `aten.le.{Scalar/Tensor}`
    *  `aten.log{10/1p/2}.default`
    *  `aten.{max/maximum/mean/min/minimum}.default`
    *  `aten.min.dim`
    *  `aten.ne.Tensor`
    *  `aten.pow.{Scalar/Tensor_Tensor}`
    *  `aten.reciprocal.default`
    *  `aten.rsub.Tensor`
    *  `aten.scatter.value`
    *  `aten.select_scatter.default`
    *  `aten.sign.default`
    *  `aten.sqrt.default`
    *  `aten.sum.default`
    *  `aten.unfold.default`
    *  `aten.var.correction`
    *  *trigonometric ops*

 - *Enabled layer tests for added ops for torch.export case*

### Tickets:
 - *ticket-id*
2024-02-29 16:14:31 +00:00
Anastasiia Pnevskaia a698628c7a
[TF FE] Add TF Hugging Face model tests into nightly (#23095)
### Details:
 - Added Hugging Face models to nightly

### Tickets:
 - 132904

---------

Co-authored-by: Andrei Kashchikhin <andrey.kashchikhin@intel.com>
2024-02-28 16:03:40 +00:00
Maxim Vafin ed6b3a5333
Disable torch.export for nightly (#23114)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-02-27 21:08:11 +04:00
Anastasiia Pnevskaia a819605986
[TF FE] Test TF Hugging Face models (#22970)
### Details:
 - TF Hugging face tests

### Tickets:
 - 132904

---------

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-02-26 12:41:09 +00:00
Maxim Vafin 92e165aa4c
[PT FE] Add support for aten::inverse (#22880)
### Details:
 - *Add support for `aten::inverse`*
- *Tests: couldn't replicate the operation in layer tests, but it exists
in TPSMM model, so test for this model is included*

### Tickets:
 - *CVS-123613*
2024-02-16 19:45:49 +00:00
Maxim Vafin 77d56395db
Update torch models in GHA tests (#22855)
### Details:
 - *Update torch models in GHA tests*

### Tickets:
 - *None*
2024-02-16 15:28:36 +00:00
Roman Kazantsev aca2f76e42
[TF FE] Fix bad cast for tf.TensorShape to ov.PartialShape (#22813)
**Details:** Fix bad cast for tf.TensorShape to ov.PartialShape. The
error occurred in mobile-object-localizer-v1 model.

**Ticket:** CVS-132455

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-02-14 11:48:16 +00:00
Anastasiia Pnevskaia 0affac4c30
[TF FE] Fixed reading of string attributes in TF Decoder (#22752)
### Details:
 - Non utf-8 strings are passed as bytes.
 - String attr type now passed as ov.string.

### Tickets:
 - 131659
2024-02-09 12:46:31 +00:00
Roman Kazantsev a6fd315e02
[TF Hub] Update test for MUSE model (#22647)
**Details:** Update test for MUSE model

**Ticket:** TBD

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-02-05 17:31:50 +00:00
Roman Kazantsev c7b4ba34c3
[GHA][Model Hub Tests] Use openvino-tokenizers in model hub tests (#22638)
**Details:** Use openvino-tokenizers in model hub tests.

**Ticket:** TBD

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-02-04 07:30:36 +00:00
Maxim Vafin 44f4d9b18e
[GHA] Disable export in timm tests until problem with timeout is solved (#22563) 2024-02-01 02:41:48 +00:00
Maxim Vafin d018779add
[PT FE] torch.export support (#22397)
* [PT FE] torch.export support

* Apply code style

* Fix build

* Support torch <2.2

* Support more operations

* Update tests/model_hub_tests/torch_tests/torch_utils.py

* Support for model operations

* Support is_causal as kwarg for SDPA

* Update src/frontends/pytorch/src/op/addcmul.cpp

* Update tests/model_hub_tests/torch_tests/test_timm.py

* Support only decoder passed to convert_model

* Fix tests

* Update src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py

* Update src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py

Co-authored-by: Ekaterina Aidova <ekaterina.aidova@intel.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Improve testing with caching model

* Apply suggestions from code review

---------

Co-authored-by: Ekaterina Aidova <ekaterina.aidova@intel.com>
2024-01-30 11:50:18 +00:00
abhinav231-valisetti 51cbbab2f4
Fixed IndexError list index out of range for Squeeze (#22326)
* fixed list index out of range error in node_decoder.py

* Improved and Modified as per review feedback

- Added a Model in the precommit_models for precommit validation.
- Removed skip and xfail marrks for the issue (129232 IndexError list
  index out of range for Squeeze)
- Added explicilty return None in node_decoder.py

* Update tests/model_hub_tests/tf_hub_tests/nightly_models

* Update tests/model_hub_tests/tf_hub_tests/nightly_models

* Update tests/model_hub_tests/tf_hub_tests/nightly_models

* Apply suggestions from code review

* Update tests/model_hub_tests/tf_hub_tests/precommit_models

* Update tests/model_hub_tests/tf_hub_tests/precommit_models

---------

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-01-24 05:36:19 +00:00
Pavel Esir 28b6fc9394
uncomment persimmon-8b-base, tvp-base, pvt-tiny-224 (#22212) 2024-01-17 16:52:19 +01:00
Roman Kazantsev 14183ae7d1
[TF Hub] Mark up models failing due to unsupported operations (#22207)
* [TF Hub] Mark up models failing due to unsupported operations

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Update tests/model_hub_tests/tf_hub_tests/nightly_models

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-17 11:19:22 +01:00
Maxim Vafin d699522843
[GHA] Update torchbench tests due to remove of phi_1_5 (#22191) 2024-01-16 23:50:10 +01:00
Roman Kazantsev 464c35181a
[TF FE] Add testing model with tokenizer to precommit (#22141)
* [TF FE] Add testing model with tokenizer to precommit

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Fix typo

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-16 10:36:29 +04:00
Pavel Esir 938600fbf7
[tests] resolve HF models 4th batch (#21907)
* add inspect pytorch modules

* add infer for each submodule

* skip -> xfail in several models

* successfull separate modules execution

* revert jukebox, whisper

* ready for rewiew

* rely on tags insted of names

* fix import error
2024-01-15 12:57:24 +04:00
Roman Kazantsev 994438b898
[TF Hub] Move Notebooks tests to Kaggle links (#22108)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-12 10:32:56 +04:00
Roman Kazantsev 2a4ae7c5c0
[TF Hub] Mark up models failing with IndexError for Squeeze (#22070)
* [TF Hub] Mark up models failing with IndexError for Squeeze

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Add xfail for nightly

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-10 18:46:43 +01:00
Roman Kazantsev ce4bbd75b4
[TF Hub] Mark up models failing with timeout error (#22050)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-09 20:28:55 +01:00
Roman Kazantsev 3052310d3b
[TF Hub] Mark up models failing with InvalidArgumentError and AttributeError (#22010)
* [TF Hub] Mark tests failing with InvalidArgumentError

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* [TF Hub] Mark test models failing with AttributeError

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-08 13:29:28 +01:00
Roman Kazantsev 51eb10743a
[TF Hub] Markup models failing with error "Model references undeclared parameters" (#21975)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-05 08:33:01 +00:00
Roman Kazantsev 7953dfafa1
[TF Hub] Markup models failing due to inference results mismatch (#21936)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-01-03 15:27:12 +01:00