mirrors/mmcv
1
0
Fork 0
mirror of https://github.com/open-mmlab/mmcv.git synced 2025-06-03 21:54:52 +08:00
Code Issues Projects Releases Wiki Activity
mmcv/docs/en/deployment/tensorrt_custom_ops.md
Zaida Zhou b326a219a7
[Docs] Replace markdownlint with mdformat for avoiding installing ruby (#1936) 
* Use mdformat pre-commit hook

* allows consecutive numbering

* improve .mdformat.toml

* test mdformat

* format markdown

* minor fix

* fix codespecll

* fix circleci

* add linkify-it-py dependency for cicleci

* add comments

* replace flake8 url

* add mdformat-myst dependency

* remove mdformat-myst dependency

* update contributing.md
2022-05-16 20:47:56 +08:00

17 KiB
Raw Blame History

TensorRT Custom Ops

MMCVRoIAlign

Description

Perform RoIAlign on output feature, used in bbox_head of most two stage detectors.

Parameters

Type Parameter Description
int output_height height of output roi
int output_width width of output roi
float spatial_scale used to scale the input boxes
int sampling_ratio number of input samples to take for each output sample. 0 means to take samples densely for current models.
str mode pooling mode in each bin. avg or max
int aligned If aligned=0, use the legacy implementation in MMDetection. Else, align the results more perfectly.

Inputs

inputs[0]: T
Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
inputs[1]: T
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of inputs[0].

Outputs

outputs[0]: T
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element output[0][r-1] is a pooled feature map corresponding to the r-th RoI inputs[1][r-1].

Type Constraints

  • T:tensor(float32, Linear)

ScatterND

Description

ScatterND takes three inputs data tensor of rank r >= 1, indices tensor of rank q >= 1, and updates tensor of rank q + r - indices.shape[-1] - 1. The output of the operation is produced by creating a copy of the input data, and then updating its value to values specified by updates at specific index positions specified by indices. Its output shape is the same as the shape of data. Note that indices should not have duplicate entries. That is, two or more updates for the same index-location is not supported.

The output is calculated via the following equation:

  output = np.copy(data)
  update_indices = indices.shape[:-1]
  for idx in np.ndindex(update_indices):
      output[indices[idx]] = updates[idx]

Parameters

None

Inputs

inputs[0]: T
Tensor of rank r>=1.
inputs[1]: tensor(int32, Linear)
Tensor of rank q>=1.
inputs[2]: T
Tensor of rank q + r - indices_shape[-1] - 1.

Outputs

outputs[0]: T
Tensor of rank r >= 1.

Type Constraints

  • T:tensor(float32, Linear), tensor(int32, Linear)

NonMaxSuppression

Description

Filter out boxes has high IoU overlap with previously selected boxes or low score. Output the indices of valid boxes. Indices of invalid boxes will be filled with -1.

Parameters

Type Parameter Description
int center_point_box 0 - the box data is supplied as [y1, x1, y2, x2], 1-the box data is supplied as [x_center, y_center, width, height].
int max_output_boxes_per_class The maximum number of boxes to be selected per batch per class. Default to 0, number of output boxes equal to number of input boxes.
float iou_threshold The threshold for deciding whether boxes overlap too much with respect to IoU. Value range [0, 1]. Default to 0.
float score_threshold The threshold for deciding when to remove boxes based on score.
int offset 0 or 1, boxes' width or height is (x2 - x1 + offset).

Inputs

inputs[0]: T
Input boxes. 3-D tensor of shape (num_batches, spatial_dimension, 4).
inputs[1]: T
Input scores. 3-D tensor of shape (num_batches, num_classes, spatial_dimension).

Outputs

outputs[0]: tensor(int32, Linear)
Selected indices. 2-D tensor of shape (num_selected_indices, 3) as [[batch_index, class_index, box_index], ...].
num_selected_indices=num_batches* num_classes* min(max_output_boxes_per_class, spatial_dimension).
All invalid indices will be filled with -1.

Type Constraints

  • T:tensor(float32, Linear)

MMCVDeformConv2d

Description

Perform Deformable Convolution on input feature, read Deformable Convolutional Network for detail.

Parameters

Type Parameter Description
list of ints stride The stride of the convolving kernel. (sH, sW)
list of ints padding Paddings on both sides of the input. (padH, padW)
list of ints dilation The spacing between kernel elements. (dH, dW)
int deformable_group Groups of deformable offset.
int group Split input into groups. input_channel should be divisible by the number of groups.
int im2col_step DeformableConv2d use im2col to compute convolution. im2col_step is used to split input and offset, reduce memory usage of column.

Inputs

inputs[0]: T
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
inputs[1]: T
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
inputs[2]: T
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).

Outputs

outputs[0]: T
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).

Type Constraints

  • T:tensor(float32, Linear)

grid_sampler

Description

Perform sample from input with pixel locations from grid.

Parameters

Type Parameter Description
int interpolation_mode Interpolation mode to calculate output values. (0: bilinear , 1: nearest)
int padding_mode Padding mode for outside grid values. (0: zeros, 1: border, 2: reflection)
int align_corners If align_corners=1, the extrema (-1 and 1) are considered as referring to the center points of the input's corner pixels. If align_corners=0, they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic.

Inputs

inputs[0]: T
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
inputs[1]: T
Input offset; 4-D tensor of shape (N, outH, outW, 2), where outH and outW is the height and width of offset and output.

Outputs

outputs[0]: T
Output feature; 4-D tensor of shape (N, C, outH, outW).

Type Constraints

  • T:tensor(float32, Linear)

cummax

Description

Returns a namedtuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim. And indices is the index location of each maximum value found in the dimension dim.

Parameters

Type Parameter Description
int dim The dimension to do the operation over.

Inputs

inputs[0]: T
The input tensor.

Outputs

outputs[0]: T
Output values.
outputs[1]: (int32, Linear)
Output indices.

Type Constraints

  • T:tensor(float32, Linear)

cummin

Description

Returns a namedtuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim. And indices is the index location of each minimum value found in the dimension dim.

Parameters

Type Parameter Description
int dim The dimension to do the operation over.

Inputs

inputs[0]: T
The input tensor.

Outputs

outputs[0]: T
Output values.
outputs[1]: (int32, Linear)
Output indices.

Type Constraints

  • T:tensor(float32, Linear)

MMCVInstanceNormalization

Description

Carries out instance normalization as described in the paper https://arxiv.org/abs/1607.08022.

y = scale * (x - mean) / sqrt(variance + epsilon) + B, where mean and variance are computed per instance per channel.

Parameters

Type Parameter Description
float epsilon The epsilon value to use to avoid division by zero. Default is 1e-05

Inputs

input: T
Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size.
scale: T
The input 1-dimensional scale tensor of size C.
B: T
The input 1-dimensional bias tensor of size C.

Outputs

output: T
The output tensor of the same shape as input.

Type Constraints

  • T:tensor(float32, Linear)

MMCVModulatedDeformConv2d

Description

Perform Modulated Deformable Convolution on input feature, read Deformable ConvNets v2: More Deformable, Better Results for detail.

Parameters

Type Parameter Description
list of ints stride The stride of the convolving kernel. (sH, sW)
list of ints padding Paddings on both sides of the input. (padH, padW)
list of ints dilation The spacing between kernel elements. (dH, dW)
int deformable_group Groups of deformable offset.
int group Split input into groups. input_channel should be divisible by the number of groups.

Inputs

inputs[0]: T
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
inputs[1]: T
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
inputs[2]: T
Input mask; 4-D tensor of shape (N, deformable_group* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
inputs[3]: T
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
inputs[4]: T, optional
Input weight; 1-D tensor of shape (output_channel).

Outputs

outputs[0]: T
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).

Type Constraints

  • T:tensor(float32, Linear)