diff --git a/docs/doxygen/ie_docs.xml b/docs/doxygen/ie_docs.xml index 503f9f38986..c12cc4d22be 100644 --- a/docs/doxygen/ie_docs.xml +++ b/docs/doxygen/ie_docs.xml @@ -194,6 +194,7 @@ limitations under the License. + diff --git a/docs/ops/sort/MulticlassNMS_8.md b/docs/ops/sort/MulticlassNMS_8.md new file mode 100644 index 00000000000..16997a81397 --- /dev/null +++ b/docs/ops/sort/MulticlassNMS_8.md @@ -0,0 +1,161 @@ +## MulticlassNonMaxSuppression {#openvino_docs_ops_sort_MulticlassNonMaxSuppression_8} + +**Versioned name**: *MulticlassNonMaxSuppression-8* + +**Category**: *Sorting and maximization* + +**Short description**: *MulticlassNonMaxSuppression* performs multi-class non-maximum suppression of the boxes with predicted scores. + +**Detailed description**: *MulticlassNonMaxSuppression* is a multi-phase operation. It implements non-maximum suppression algorithm as described below: + +1. Let `B = [b_0,...,b_n]` be the list of initial detection boxes, `S = [s_0,...,s_N]` be the list of corresponding scores. +2. Let `D = []` be an initial collection of resulting boxes. Let `adaptive_threshold = iou_threshold`. +3. If `B` is empty, go to step 9. +4. Take the box with highest score. Suppose that it is the box `b` with the score `s`. +5. Delete `b` from `B`. +6. If the score `s` is greater than or equal to `score_threshold`, add `b` to `D`, else go to step 9. +7. If `nms_eta < 1` and `adaptive_threshold > 0.5`, update `adaptive_threshold *= nms_eta`. +8. For each input box `b_i` from `B` and the corresponding score `s_i`, set `s_i = 0` when `iou(b, b_i) > adaptive_threshold`, and go to step 3. +9. Return `D`, a collection of the corresponding scores `S`, and the number of elements in `D`. + +This algorithm is applied independently to each class of each batch element. The operation feeds at most `nms_top_k` scoring candidate boxes to this algorithm. +The total number of output boxes of each batch element must not exceed `keep_top_k`. +Boxes of `background_class` are skipped and thus eliminated. + +**Attributes**: + +* *sort_result* + + * **Description**: *sort_result* specifies the order of output elements. + * **Range of values**: `class`, `score`, `none` + * *class* - sort selected boxes by class id (ascending). + * *score* - sort selected boxes by score (descending). + * *none* - do not guarantee the order. + * **Type**: `string` + * **Default value**: `none` + * **Required**: *No* + +* *sort_result_across_batch* + + * **Description**: *sort_result_across_batch* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not. + * **Range of values**: true or false + * *true* - sort selected boxes across batches. + * *false* - do not sort selected boxes across batches (boxes are sorted per batch element). + * **Type**: boolean + * **Default value**: false + * **Required**: *No* + +* *output_type* + + * **Description**: the tensor type of outputs `selected_indices` and `valid_outputs`. + * **Range of values**: `i64` or `i32` + * **Type**: `string` + * **Default value**: `i64` + * **Required**: *No* + +* *iou_threshold* + + * **Description**: intersection over union threshold. + * **Range of values**: a floating-point number + * **Type**: `float` + * **Default value**: `0` + * **Required**: *No* + +* *score_threshold* + + * **Description**: minimum score to consider box for the processing. + * **Range of values**: a floating-point number + * **Type**: `float` + * **Default value**: `0` + * **Required**: *No* + +* *nms_top_k* + + * **Description**: maximum number of boxes to be selected per class. + * **Range of values**: an integer + * **Type**: `int` + * **Default value**: `-1` meaning to keep all boxes + * **Required**: *No* + +* *keep_top_k* + + * **Description**: maximum number of boxes to be selected per batch element. + * **Range of values**: an integer + * **Type**: `int` + * **Default value**: `-1` meaning to keep all boxes + * **Required**: *No* + +* *background_class* + + * **Description**: the background class id. + * **Range of values**: an integer + * **Type**: `int` + * **Default value**: `-1` meaning to keep all classes. + * **Required**: *No* + +* *nms_eta* + + * **Description**: eta parameter for adaptive NMS. + * **Range of values**: a floating-point number in close range `[0, 1.0]`. + * **Type**: `float` + * **Default value**: `1.0` + * **Required**: *No* + +**Inputs**: + +* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. The box coordinates are layout as `[xmin, ymin, xmax, ymax]`. **Required.** + +* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. **Required.** + +**Outputs**: + +* **1**: `selected_outputs` - tensor of type *T_THRESHOLDS* and shape `[number of selected boxes, 6]` containing the selected boxes with score and class as tuples `[class_id, box_score, xmin, ymin, xmax, ymax]`. + +* **2**: `selected_indices` - tensor of type *T_IND* and shape `[number of selected boxes, 1]` the selected indices in the flattened `boxes`, which are absolute values cross batches. Therefore possible valid values are in the range `[0, num_batches * num_boxes - 1]`. + +* **3**: `selected_num` - 1D tensor of type *T_IND* and shape `[num_batches]` representing the number of selected boxes for each batch element. + +When there is no box selected, `selected_num` is filled with `0`. `selected_outputs` is an empty tensor of shape `[0, 6]`, and `selected_indices` is an empty tensor of shape `[0, 1]`. + +**Types** + +* *T*: floating point type. + +* *T_MAX_BOXES*: integer type. + +* *T_THRESHOLDS*: floating point type. + +* *T_IND*: `int64` or `int32`. + +**Example** + +```xml + + + + + 3 + 100 + 4 + + + 3 + 5 + 100 + + + + + -1 + 6 + + + -1 + 1 + + + 3 + + + +```