SeetaResearch / Dragon

Summary:
This commit correctly uses the distribution commands to collect the
python package and the compiled dynamic libraries.

Summary:
This commit fuses the weight decay and mixed precision conversion
into update kernels to get lower training latency.

Summary:
This commit adds a feature to map memory and load data with offset.
Tensor takes mapped memory is readonly and will drop it on next mutation.

Summary:
This commit adds extra CUDA softmax kernels using warp reduce.
Warp reduce leads to better performance when dimension <= 256,
which is preferred for the recent vision transformers.

Summary:
This commit uses "math::Transpose" instead of "kernels::Transpose"
for more possible optimized routines in the future.

Summary:
This commit instantiates CUDA kernels by using constant dimensions
to enable the optimization during compiler-time.

Summary:
This commit allows transpose to compute in-place by leveraging buffer.
We also adds CRD mode for space-depth transpose (i.e., pixel shuffle).

Summary:
This commit removes the default cuBLAS tensor core math mode
when CUDA >= 11.0 on ampere devices to avoid the FP32 downcast math.

Summary:
This commit adds im2col operator to unfold input to depth.

Summary:
This commit renames triangular operator to trilu following the ONNX.

Summary:
This commit fixes the alpha and beta to 1/6 and 0.5 for hardswish,
the same behavior as ONNX scheme.

Summary:
This commit adds reverse or flip operator.

Summary:
This commit adds GELU activation to compute output
via approximate or naive mode.

Summary:
This commit adds scatter and gather operator
to remap elements along the given dimension of indices.

Summary:
The new frontend makes an union of two execution modes, while starts from
a single tensor class. Besides, it emits the operator execution through
a common path that works both for dragon and torch.

Summary:
This commit generalizes the fully-connected operation into GEMM,
and enhances the matmul operation via batched Dot, GEMV and GEMM.
New representations and attributes have been consistent with ONNX.

Summary:
For the purpose of consistency on getting CUDNN convolution algorithms,
CUDNN v6 (mainly relied by CUDA 8.0) is now dropped.

Summary:
This commit adds the sysconfig module to get the build information.
Build information is helpful to select tests or report issues.

Summary:
This commit adds 1D and 3D support for vision operators
via a generalized ND implementation.

Summary:
This commit fixes the two compiling issues on win32:
1. Remove the constexpr in generating protobuf headers.
2. Enforce the "/MT" flag to overwrite the "/MD" always.

Summary:
This commit tests the correctness of shape inference and data type
blended by autograph module.

Summary:
We found it unstable when defining CUB storages in a union.
More surveys should be taken to understand this issue.

Summary:
This commit moves the parser into ArgHelper which designed
to add the descriptors before.

Summary:
This commit feeds the repeated tensor arguments with
the entire array instead of the piecewise scalars.

Summary:
This commit redesigns the ``vm.onnx`` by referring the official repository.
Frontends and backends are aligned with identical API for dragon, torch and tensorrt.

Summary:
This commit removes the unused reference of GradientTape to
avoid the circular reference issue.

Summary:

This commit attaches input and output together in assign operators,
which fixes the missing input defs due to identity from input to output.

Summary:
This commit adds pseudo FP16 kernels with FP32 conversions
for DepthwiseConv2d and SyncBN operator.

Summary:
This commit adds a fallback with FP32 accumulator
for FP16 ReduceSum to avoid dropping too many small values.
Besides, FP16 kernels for arch < 530 are almost available.

Summary:
This commit adds the hardsigmoid, hardswish and swish op with specialized kernel,
there are widely used in MobileNetV3 and EfficientNet.

Summary:
This commit uses CopyMatrix to implement concat and split generically
instead of specialized kernels.

Summary:
This commit uses unique tensors to provide workspace data
to avoid the corruption between operator and kernel.

Summary:
This commit adds the linspace op for dragon, torch and tensorflow.
And, a workaround for truncated int interval is made to range/linspace (up to 2**57).

Summary:
This commit reimplements the cuda argmax/argmin via BlockReduce,
instead of the naive reduction in kernel loop.

Summary:
This commit adds the sort op for dragon, torch and tensorflow.
Besides, cuda implementation of topk op is now available.

Summary:
This commit uses local(thread or stream) workspace for Context,
which provides a more elegant way to dispatch kernels requiring scratch.
Besides, TF32 math type is provided as a cuDNN option for Ampere device.

Summary:
This commit adds the unique op for dragon, torch, tensorflow and onnx.
Besides, fixes the bug that gets the wrong workspace size in cached cudnn convolution.

Summary:
This commit reimplements the default shuffle policy of data reader with
sequential sampling (be consistent with DALI) instead of chunk permutation (MXNet solution).
Sequential sampling is tuned by argument ``initial_fill`` only, and works both for HDD and SSD.

Summary:
This commit renames the operator argument getter to ``GetArgument``
whatever an argument is single or repeated.