Data types of generic Op must be fully registered(English Version)

Data types of generic Op must be fully registered

---

OP书写规范：通用类Op数据类型须注册完整 (中文版)

---

Specification summary：

• Section 1, Background
• Section 2, Characteristics of generic Op
• Section 3, Relevant Instructions of CI

Additional instructions:

During the implementation process, the specifications may find aspects that are not considered by the existing specifications, and need to be supplemented and improved in the implementation process. Please also give positive feedback.

1. Background

At present, all Ops in Paddle framework register the data types supported by CPU or GPU through the REGISTER_OP_CPU_KERNEL and REGISTER_OP_CUDA_KERNEL macros. The data types supported by different Ops vary according to the functional requirements and usage scenarios, such as Conv2d convolution only supports float and double, and sum Op needs to support int, int64_t, float, and double.

An op such as sum, whose computational logic does not depend on a particular data type, is called a generic Op. The data types supported by the generic Op must be fully registered at least four data types: int, int64_t, float, and double.

2. Characteristics of generic Op

Generic Op mainly has the following characteristics:

1. Mathematical operation function Op

• Including abs, round, some trigonometric functions, elementwise related op, activation function, value comparison Ops.
  • absolute value: fluid.layers.abs(x)
  • Exponential calculation: fluid.layers.pow(x, ...)
  • Trigonometric functions: fluid.layers.acos(x)
  • elementwise: elementwise_add、elementwise_mul、elementwise_div et.al.
  • ReLU: fluid.layers.relu(x,...)
  • value comparison: fluid.layers.less_than(x, y,...)

2. Data reduce or index operation Op

• Including Ops that perform reduce calculations by axis parameters or index related Ops.
  • index related: fluid.layers.argmin(x,axis=0)
  • minimum: fluid.layers.reduce_min(input, ...)
  • mean: fluid.layers.reduce_mean(input,...)
  • topK: fluid.layers.top(input, ...)
  • sort: fluid.layers.argsort(input,...)

3. Tensor formation or other transformation dependence

• Mainly including fill_batch_like, fill_constant and other Op with Tensor generating custom value and shape, and Op with flatten, reshape, reverse operation.
  • fill: fluid.layers.fill_constant_batch_size_like(input, shape,dtype, ...)
  • reshape: fluid.layers.reshape(x, shape)
  • assignment: fluid.layers.assign(input,..)
  • mask: fluid.layers.sequence_mask(x,...)
  • squeeze: fluid.layers.squeeze(x, ...)

How can you tell if an Op is a generic Op? You can refer to the following three principles:

1. The Op calculation logic is not strongly dependent on a particular data type.

The input of loss Op is usually the predicted probability distribution, and the output is of float and double types. Therefore it is not necessary to register int32 and int64 types(But label should support the both types). However, for ops related to pow and elementwise, the computational logic is decoupled from the data type, so the above four types must be registered at least.

2. The Op operations are independent of the data region of the input Tensor.

Such as the Ops with Tensor reshape, squeeze, mask, should register at least this four types.

3. Refer to the data types supported by other frameworks

You can refer to the Op implementations of other frameworks with the same functionality to align kernel-supported data types.

3. Relevant Instructions of CI

Incremental checking of this specification has been enabled in PR_CI_CPU_Py2. If the new OP does not register int/int64_t, float/double, or all four types, the check will fail.

Details of data type registration:

Data types	state	instructions
int/int64_t/float/double	Allowed	Fully Registered
int/int64_t	Allowed	Only support integer
float/double	Allowed	Only support float
float	Not Allowed	Need support double
int	Not Allowed	Need support int64_t

An error message similar to the following will appear in the Build Log:

[08:49:34]	[Step 1/1] ****************
[08:49:34]	[Step 1/1] 0. You must have one RD (Aurelius84 (Recommend) or liym27 or zhhsplendid)approval for the data_type registration of new operator. More data_type of new operator should be registered in your PR. Please make sure that both float/double (or int/int64_t) have been registered.
[08:49:34]	[Step 1/1]  For more details, please click [https://github.com/PaddlePaddle/Paddle/wiki/Data-types-of-generic-Op-must-be-fully-registered].
[08:49:34]	[Step 1/1] 
[08:49:34]	[Step 1/1] There are 1 approved errors.
[08:49:34]	[Step 1/1] ****************
[08:49:36]	[Step 1/1] expand_test_a only supports [int float] now, but lacks [double int64_t].
[08:49:36]	[Step 1/1] expand_test_b only supports [int double float] now, but lacks [int64_t].
[08:49:36]	[Step 1/1] expand_test_c only supports [float] now, but lacks [double].
[08:49:42]	[Step 1/1] Process exited with code 1

Please modify your codes according to the error message to achieve the purpose of compatibility upgrade. If it is confirmed that the upgrade cannot be compatible, please find the relevant approvers(there is a list of approvers in CI Build Log) to review the code and at least one approval is required.

If you want to repeat the process of CI, please follow these steps:

1. Compile and install paddle from develop branch
2. Print list of all data types registered with the Op from develop branch, run:
   python tools/check_op_register_type.py > OP_TYPE_DEV.spec
3. Compile and install paddle from PR branch
4. Print list of all data types registered with the Op from PR branch, run:
   python tools/check_op_register_type.py > OP_TYPE_PR.spec
5. Compare the two op desc, run:
   python tools/check_op_register_type.py OP_TYPE_DEV.spec OP_TYPE_PR.spec

If you have problems, please contact @ Aurelius84.