Skip to content

SeetaResearch / Dragon

Go to a project
Commit aa2ec8c3 by Ting PAN
Options

Make the consistency for shape and data type 

Summary:
This commit enforces the shape and data type inherited from the same metaclass,
which ensures the consistency between different styles.
1 parent 0ab14f30
Expand all
Inline Side-by-side
Showing with 442 additions and 435 deletions
docs/api/python/dragon.rst
...@@ -15,9 +15,6 @@ dragon ...@@ -15,9 +15,6 @@ dragon
`class Tensor <dragon/Tensor.html>`_ `class Tensor <dragon/Tensor.html>`_
: Tensor abstraction for graph executing. : Tensor abstraction for graph executing.
`class TensorSpec <dragon/TensorSpec.html>`_
: Spec to describe properties of a tensor.
`class Workspace <dragon/Workspace.html>`_ `class Workspace <dragon/Workspace.html>`_
: Sandbox to isolate the resources and computations. : Sandbox to isolate the resources and computations.
...@@ -213,7 +210,6 @@ dragon ...@@ -213,7 +210,6 @@ dragon
dragon/stack dragon/stack
dragon/stop_gradient dragon/stop_gradient
dragon/Tensor dragon/Tensor
dragon/TensorSpec
dragon/tile dragon/tile
dragon/transpose dragon/transpose
dragon/where dragon/where
......
docs/api/python/dragon/TensorSpec.rst deleted 100644 → 0
TensorSpec
==========
.. autoclass:: dragon.TensorSpec
__init__
--------
.. automethod:: dragon.TensorSpec.__init__
Properties
----------
dtype
#####
.. autoattribute:: dragon.TensorSpec.dtype
name
####
.. autoattribute:: dragon.TensorSpec.name
shape
#####
.. autoattribute:: dragon.TensorSpec.shape
Methods
-------
is_compatible_with
##################
.. automethod:: dragon.TensorSpec.is_compatible_with
.. raw:: html
<style>
h1:before {
content: "dragon.";
color: #103d3e;
}
</style>
docs/api/python/tensorflow.rst
...@@ -9,6 +9,9 @@ vm.tensorflow ...@@ -9,6 +9,9 @@ vm.tensorflow
`class GradientTape <tensorflow/GradientTape.html>`_ `class GradientTape <tensorflow/GradientTape.html>`_
: Record the operations for auto differentiation. : Record the operations for auto differentiation.
`class TensorShape <tensorflow/TensorShape.html>`_
: Represent the a sequence of dimensions.
`class TensorSpec <tensorflow/TensorSpec.html>`_ `class TensorSpec <tensorflow/TensorSpec.html>`_
: Spec to describe properties of a tensor. : Spec to describe properties of a tensor.
...@@ -124,6 +127,7 @@ vm.tensorflow ...@@ -124,6 +127,7 @@ vm.tensorflow
tensorflow/slice tensorflow/slice
tensorflow/split tensorflow/split
tensorflow/squeeze tensorflow/squeeze
tensorflow/TensorShape
tensorflow/TensorSpec tensorflow/TensorSpec
tensorflow/transpose tensorflow/transpose
tensorflow/zeros tensorflow/zeros
......
docs/api/python/tensorflow/TensorShape.rst 0 → 100644
TensorShape
===========
.. autoclass:: dragon.vm.tensorflow.TensorShape
__init__
--------
.. automethod:: dragon.vm.tensorflow.TensorShape.__init__
Properties
----------
dims
####
.. autoattribute:: dragon.vm.tensorflow.TensorShape.dims
ndims
#####
.. autoattribute:: dragon.vm.tensorflow.TensorShape.ndims
rank
####
.. autoattribute:: dragon.vm.tensorflow.TensorShape.rank
Methods
-------
as_list
#######
.. automethod:: dragon.vm.tensorflow.TensorShape.as_list
.. raw:: html
<style>
h1:before {
content: "tf.";
color: #103d3e;
}
</style>
docs/api/python/tensorflow/TensorSpec.rst
...@@ -26,7 +26,7 @@ shape ...@@ -26,7 +26,7 @@ shape
<style> <style>
h1:before { h1:before {
content: "dragon."; content: "tf.";
color: #103d3e; color: #103d3e;
} }
</style> </style>
docs/api/python/tensorflow/dtypes/DType.rst
...@@ -7,6 +7,76 @@ __init__ ...@@ -7,6 +7,76 @@ __init__
-------- --------
.. automethod:: dragon.vm.tensorflow.dtypes.DType.__init__ .. automethod:: dragon.vm.tensorflow.dtypes.DType.__init__
Properties
----------
as_numpy_dtype
##############
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.as_numpy_dtype
as_datatype_enum
################
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.as_datatype_enum
base_dtype
##########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.base_dtype
is_numpy_compatible
###################
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_numpy_compatible
is_bool
#######
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_bool
is_complex
##########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_complex
is_floating
###########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_floating
is_integer
##########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_integer
is_quantized
############
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_quantized
is_unsigned
###########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.is_unsigned
limits
######
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.limits
max
###
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.max
min
###
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.min
name
####
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.name
real_dtype
##########
.. autoattribute:: dragon.vm.tensorflow.dtypes.DType.real_dtype
Methods
-------
is_compatible_with
##################
.. automethod:: dragon.vm.tensorflow.dtypes.DType.is_compatible_with
.. raw:: html .. raw:: html
<style> <style>
......
docs/api/python/torch.rst
...@@ -7,7 +7,10 @@ vm.torch ...@@ -7,7 +7,10 @@ vm.torch
####### #######
`class device <torch/device.html>`_ `class device <torch/device.html>`_
: Represent the device where tensor will be allocated. : Represent the device spec.
`class dtype <torch/device.html>`_
: The basic data type.
`class enable_grad <torch/enable_grad.html>`_ `class enable_grad <torch/enable_grad.html>`_
: Context-manager to enable gradient calculation. : Context-manager to enable gradient calculation.
...@@ -265,6 +268,7 @@ vm.torch ...@@ -265,6 +268,7 @@ vm.torch
torch/cumsum torch/cumsum
torch/device torch/device
torch/div torch/div
torch/dtype
torch/empty torch/empty
torch/enable_grad torch/enable_grad
torch/eq torch/eq
......
docs/api/python/torch/dtype.rst 0 → 100644
dtype
=====
.. autoclass:: dragon.vm.torch.dtype
__init__
--------
.. automethod:: dragon.vm.torch.dtype.__init__
.. raw:: html
<style>
h1:before {
content: "torch.";
color: #103d3e;
}
</style>
dragon/operators/array/arange_op.cc
...@@ -21,7 +21,7 @@ void ArangeOp<Context>::DoRunWithType() { ...@@ -21,7 +21,7 @@ void ArangeOp<Context>::DoRunWithType() {
// Determine the generating range // Determine the generating range
// Values are in a half-open interval: [start, stop) // Values are in a half-open interval: [start, stop)
auto count = (int64_t)std::round((stop - start) / step); auto count = (int64_t)std::ceil((stop - start) / step);
CHECK_GT(count, 0) << "\nInvalid generating range: " CHECK_GT(count, 0) << "\nInvalid generating range: "
<< "[" << start << ", " << stop << ") with step = " << step << "[" << start << ", " << stop << ") with step = " << step
<< "."; << ".";
......
dragon/python/__init__.py
...@@ -39,7 +39,6 @@ from dragon import vm ...@@ -39,7 +39,6 @@ from dragon import vm
from dragon.core.autograph.tensor import Tensor from dragon.core.autograph.tensor import Tensor
from dragon.core.eager.tensor import EagerTensor from dragon.core.eager.tensor import EagerTensor
from dragon.core.eager.backprop import GradientTape from dragon.core.eager.backprop import GradientTape
from dragon.core.framework.tensor_spec import TensorSpec
from dragon.core.framework.workspace import Workspace from dragon.core.framework.workspace import Workspace
# Function # Function
......
dragon/python/core/autograph/def_function.py
...@@ -254,7 +254,7 @@ class FunctionGuard(object): ...@@ -254,7 +254,7 @@ class FunctionGuard(object):
) )
shape = input_signature[i].shape shape = input_signature[i].shape
dtype = input_signature[i].dtype dtype = input_signature[i].dtype
inputs.append(Tensor(name, shape, dtype).constant()) inputs.append(Tensor(shape, dtype, name).constant())
with context.name_scope('${%d}' % id(self)), eager_context.graph_mode(): with context.name_scope('${%d}' % id(self)), eager_context.graph_mode():
returns = nest.flatten(self._python_function(*inputs)) returns = nest.flatten(self._python_function(*inputs))
outputs, dummies = [], [] outputs, dummies = [], []
...@@ -328,8 +328,8 @@ def function(func=None, input_signature=None): ...@@ -328,8 +328,8 @@ def function(func=None, input_signature=None):
```python ```python
@dragon.function(input_signature=[ @dragon.function(input_signature=[
dragon.TensorSpec(shape=[], dtype='float32'), dragon.Tensor(shape=[], dtype='float32'),
dragon.TensorSpec(shape=[], dtype='float32') dragon.Tensor(shape=[], dtype='float32'),
]) ])
def foo(x, y): def foo(x, y):
return dragon.math.add([x + y, x]) return dragon.math.add([x + y, x])
...@@ -341,8 +341,8 @@ def function(func=None, input_signature=None): ...@@ -341,8 +341,8 @@ def function(func=None, input_signature=None):
---------- ----------
func : callable, optional func : callable, optional
The function to be compiled. The function to be compiled.
input_signature : Sequence[dragon.TensorSpec], optional input_signature : Sequence[dragon.Tensor], optional
The specs to hint the input info. The tensors to hint the input info.
Returns Returns
------- -------
......
dragon/python/core/autograph/function_lib.py
...@@ -20,9 +20,10 @@ import os ...@@ -20,9 +20,10 @@ import os
from dragon.core.autograph import grad_maker from dragon.core.autograph import grad_maker
from dragon.core.autograph.op_def import OpDef from dragon.core.autograph.op_def import OpDef
from dragon.core.autograph.op_def import OpInfo from dragon.core.autograph.op_def import OpInfo
from dragon.core.autograph.tensor import Tensor from dragon.core.autograph.tensor import TensorRef
from dragon.core.framework import config from dragon.core.framework import config
from dragon.core.framework import proto_util from dragon.core.framework import proto_util
from dragon.core.framework import types
from dragon.core.framework import workspace from dragon.core.framework import workspace
from dragon.core.proto import dragon_pb2 from dragon.core.proto import dragon_pb2
from dragon.core.util import logging from dragon.core.util import logging
...@@ -107,9 +108,7 @@ def add_update_defs(graph_def, optimizer): ...@@ -107,9 +108,7 @@ def add_update_defs(graph_def, optimizer):
name=OpDef.get_name(), name=OpDef.get_name(),
operation='MEAN', operation='MEAN',
communication='ALLREDUCE', communication='ALLREDUCE',
**process_group.arguments **process_group.arguments))
)
)
graph_def.op.extend(update_defs) graph_def.op.extend(update_defs)
...@@ -147,14 +146,11 @@ class Function(object): ...@@ -147,14 +146,11 @@ class Function(object):
if givens is not None: if givens is not None:
name_dict = {} name_dict = {}
for k, v in givens.items(): for k, v in givens.items():
if isinstance(v, Tensor): if types.is_symbolic_tensor(v):
name_dict[k.id] = v.id name_dict[k.id] = v.id
op_info.merge_from(v) op_info.merge_from(v)
else: else:
raise ValueError( raise ValueError('Excepted a Tensor, got {}.'.format(type(v).__name__))
'Excepted a Tensor, '
'got {}.'.format(type(v).__name__)
)
# Update the original defs. # Update the original defs.
op_info = copy.deepcopy(op_info) op_info = copy.deepcopy(op_info)
for k in op_info._defs.keys(): for k in op_info._defs.keys():
...@@ -257,8 +253,8 @@ class Function(object): ...@@ -257,8 +253,8 @@ class Function(object):
The self. The self.
""" """
self.outputs = [Tensor(name) for name in graph_def.output] self.outputs = [TensorRef(name) for name in graph_def.output]
self.inputs = [Tensor(name).constant() for name in graph_def.input] self.inputs = [TensorRef(name).constant() for name in graph_def.input]
# Fill with all known graph elements. # Fill with all known graph elements.
add_device_option(graph_def) add_device_option(graph_def)
...@@ -293,7 +289,7 @@ def create_function(inputs=None, outputs=None, givens=None, optimizer=None): ...@@ -293,7 +289,7 @@ def create_function(inputs=None, outputs=None, givens=None, optimizer=None):
Tensors that catch any operators can be used to create a graph: Tensors that catch any operators can be used to create a graph:
```python ```python
x = dragon.Tensor('x', dtype='float32').constant() x = dragon.Tensor(dtype='float32').constant()
y = x * 2 y = x * 2
f = dragon.create_function(outputs=y) f = dragon.create_function(outputs=y)
``` ```
...@@ -315,20 +311,20 @@ def create_function(inputs=None, outputs=None, givens=None, optimizer=None): ...@@ -315,20 +311,20 @@ def create_function(inputs=None, outputs=None, givens=None, optimizer=None):
Specify ``givens`` to substitute tensors before creating: Specify ``givens`` to substitute tensors before creating:
```python ```python
x = dragon.Tensor('x', dtype='float32').constant() x = dragon.Tensor(dtype='float32').constant()
y = x * 2 y = x * 2
foo = dragon.create_function(outputs=y) foo = dragon.create_function(outputs=y)
# "bar" takes "x2" as input, and also writes to "y" # "bar" takes "x2" as input, and also writes to "y"
x2 = dragon.Tensor('x2', dtype='float32').constant() x2 = dragon.Tensor(dtype='float32').constant()
bar = dragon.create_function(outputs=y, givens={x: x2}) bar = dragon.create_function(outputs=y, givens={x: x2})
``` ```
Specify ``optimizer`` to make a graph applying parameter updates: Specify ``optimizer`` to make a graph applying parameter updates:
```python ```python
x = dragon.Tensor('x', dtype='float32').set_value(1) x = dragon.Tensor(dtype='float32').set_value(1)
x_grad = dragon.Tensor('x_grad', dtype='float32').set_value(1) x_grad = dragon.Tensor(dtype='float32').set_value(1)
optimizer = dragon.optimizers.SGD(base_lr=0.01) optimizer = dragon.optimizers.SGD(base_lr=0.01)
optimizer.apply_gradients(values_and_grads=[(x, x_grad)]) optimizer.apply_gradients(values_and_grads=[(x, x_grad)])
......
dragon/python/core/autograph/tensor.py
...@@ -27,17 +27,17 @@ from dragon.core.util import nest ...@@ -27,17 +27,17 @@ from dragon.core.util import nest
class Tensor(types.TensorMetaclass): class Tensor(types.TensorMetaclass):
"""Tensor abstraction for graph executing.""" """Tensor abstraction for graph executing."""
def __init__(self, name=None, shape=None, dtype=None): def __init__(self, shape=None, dtype=None, name=None):
"""Create a ``Tensor``. """Create a ``Tensor``.
Parameters Parameters
---------- ----------
name : str, optional shape : Sequence[int], optional
The optional tensor name.
shape : sequence, optional
The optional tensor shape. The optional tensor shape.
dtype : str, optional dtype : str, optional
The optional data type. The optional data type.
name : str, optional
The optional tensor name.
""" """
self._op, self._grad = None, None self._op, self._grad = None, None
...@@ -66,7 +66,7 @@ class Tensor(types.TensorMetaclass): ...@@ -66,7 +66,7 @@ class Tensor(types.TensorMetaclass):
The data type to set. The data type to set.
""" """
self._dtype = value self._dtype = str(value) if value else value
@property @property
def id(self): def id(self):
...@@ -131,8 +131,8 @@ class Tensor(types.TensorMetaclass): ...@@ -131,8 +131,8 @@ class Tensor(types.TensorMetaclass):
Returns Returns
------- -------
Sequence[int] Tuple[int]
The shape. The tensor shape.
""" """
return self._shape return self._shape
...@@ -150,10 +150,9 @@ class Tensor(types.TensorMetaclass): ...@@ -150,10 +150,9 @@ class Tensor(types.TensorMetaclass):
if value is not None: if value is not None:
if not nest.is_sequence(value): if not nest.is_sequence(value):
raise TypeError( raise TypeError(
'The <shape> should be a Sequence. ' 'The <shape> should be a sequence. Got {}.'
'Got {}.'.format(type(value)) .format(type(value).__name__))
) self._shape = tuple(nest.flatten(value))
self._shape = nest.flatten(value)
else: else:
self._shape = value self._shape = value
...@@ -475,7 +474,7 @@ class Tensor(types.TensorMetaclass): ...@@ -475,7 +474,7 @@ class Tensor(types.TensorMetaclass):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.Tensor] item : Union[slice, int, dragon.Tensor]
The index. The index.
Returns Returns
...@@ -643,7 +642,7 @@ class Tensor(types.TensorMetaclass): ...@@ -643,7 +642,7 @@ class Tensor(types.TensorMetaclass):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.Tensor] key : Union[slice, int, dragon.Tensor]
The index. The index.
value : Union[dragon.Tensor, number] value : Union[dragon.Tensor, number]
The value to set. The value to set.
...@@ -685,6 +684,6 @@ class TensorRef(object): ...@@ -685,6 +684,6 @@ class TensorRef(object):
"""Create a reference not involved with name scope.""" """Create a reference not involved with name scope."""
def __new__(cls, name, shape=None, dtype=None): def __new__(cls, name, shape=None, dtype=None):
tensor = Tensor('', shape=shape, dtype=dtype) tensor_ref = Tensor(shape=shape, dtype=dtype, name='')
tensor._name = name tensor_ref._name = name
return tensor return tensor_ref
dragon/python/core/eager/tensor.py
...@@ -144,15 +144,15 @@ class EagerTensor(Tensor): ...@@ -144,15 +144,15 @@ class EagerTensor(Tensor):
@property @property
def shape(self): def shape(self):
"""Return the shape of this tensor. """Return tensor shape.
Returns Returns
------- -------
Sequence[int] Tuple[int]
The shape. The tensor shape.
""" """
return self._impl.dims return tuple(self._impl.dims)
@shape.setter @shape.setter
def shape(self, value): def shape(self, value):
...@@ -451,7 +451,7 @@ class EagerTensor(Tensor): ...@@ -451,7 +451,7 @@ class EagerTensor(Tensor):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.EagerTensor] item : Union[slice, int, dragon.EagerTensor]
The index. The index.
Returns Returns
...@@ -668,7 +668,7 @@ class EagerTensor(Tensor): ...@@ -668,7 +668,7 @@ class EagerTensor(Tensor):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.EagerTensor] key : Union[slice, int, dragon.EagerTensor]
The index. The index.
value : Union[dragon.EagerTensor, number] value : Union[dragon.EagerTensor, number]
The value to set. The value to set.
......
dragon/python/core/framework/proto_util.py
...@@ -36,7 +36,7 @@ if sys.version_info >= (3, 0): ...@@ -36,7 +36,7 @@ if sys.version_info >= (3, 0):
argument.i = value argument.i = value
elif type(value) is bytes: elif type(value) is bytes:
argument.s = value argument.s = value
elif type(value) is str: elif isinstance(value, str):
argument.s = str.encode(value) argument.s = str.encode(value)
elif isinstance(value, Message): elif isinstance(value, Message):
argument.s = value.SerializeToString() argument.s = value.SerializeToString()
...@@ -63,7 +63,7 @@ else: ...@@ -63,7 +63,7 @@ else:
argument.f = value argument.f = value
elif type(value) in (bool, int, long, numpy.int64): elif type(value) in (bool, int, long, numpy.int64):
argument.i = value argument.i = value
elif type(value) is str: elif isinstance(value, str):
argument.s = value argument.s = value
elif type(value) is unicode: elif type(value) is unicode:
argument.s = str(value) argument.s = str(value)
......
dragon/python/core/framework/tensor_spec.py deleted 100644 → 0
# ------------------------------------------------------------
# Copyright (c) 2017-present, SeetaTech, Co.,Ltd.
#
# Licensed under the BSD 2-Clause License.
# You should have received a copy of the BSD 2-Clause License
# along with the software. If not, See,
#
# <https://opensource.org/licenses/BSD-2-Clause>
#
# ------------------------------------------------------------
"""Structure to represent a tensor."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from dragon.core.util import nest
class TensorSpec(object):
"""Spec to describe properties of a tensor."""
def __init__(self, shape, dtype='float32', name=None):
"""Create a TensorSpec.
Parameters
----------
shape : Sequence[int], required
The dimensions.
dtype : str, optional, default='float32'
The optional data type.
name : str, optional
The optional name.
"""
self._shape, self._dtype, self._name = shape, dtype, name
if shape is not None:
self._shape = nest.flatten(shape)
@property
def dtype(self):
"""Return the data type.
Returns
-------
str
The data type.
"""
return self._dtype
@property
def name(self):
"""Return the spec name.
Returns
-------
str
The spec name.
"""
return self._name
@property
def shape(self):
"""Return the dimensions.
Returns
-------
Sequence[int]
The dimensions.
"""
return self._shape
def is_compatible_with(self, spec_or_tensor):
"""Return a bool indicating whether given the spec is compatible.
Returns
-------
bool
**True** if ``shape`` and ``dtype`` are
both compatible otherwise **False**.
"""
def dtype_is_compatible_with(spec_or_tensor):
return self._dtype == spec_or_tensor.dtype
def shape_is_compatible_with(spec_or_tensor):
shape = spec_or_tensor.shape
if self._shape is not None and shape is not None:
if len(self._shape) != len(shape):
return False
for x_dim, y_dim in zip(self._shape, shape):
if x_dim != y_dim:
return False
return True
return \
dtype_is_compatible_with(spec_or_tensor) and \
shape_is_compatible_with(spec_or_tensor)
dragon/python/core/framework/workspace.py
...@@ -180,7 +180,7 @@ class Workspace(backend.Workspace): ...@@ -180,7 +180,7 @@ class Workspace(backend.Workspace):
```python ```python
# Define a named tensor to feed # Define a named tensor to feed
x = dragon.Tensor('x') x = dragon.Tensor(name='x')
dragon.get_workspace().feed_tensor(x, 0) dragon.get_workspace().feed_tensor(x, 0)
# Feed by specifying a tensor name # Feed by specifying a tensor name
......
dragon/python/core/ops/rnn_ops.py
...@@ -18,7 +18,7 @@ import itertools ...@@ -18,7 +18,7 @@ import itertools
import numpy import numpy
import warnings import warnings
from dragon.core.autograph.tensor import Tensor from dragon.core.autograph.tensor import TensorRef
from dragon.core.eager import context from dragon.core.eager import context
from dragon.core.eager.tensor import EagerTensor from dragon.core.eager.tensor import EagerTensor
from dragon.core.framework import config from dragon.core.framework import config
...@@ -191,7 +191,7 @@ class RNNModule(object): ...@@ -191,7 +191,7 @@ class RNNModule(object):
self._weights_count += int(numpy.prod(shape)) self._weights_count += int(numpy.prod(shape))
# Create the flat float32 weights. # Create the flat float32 weights.
self._weights = EagerTensor(shape=[self._weights_count], trainable=True) self._weights = EagerTensor(shape=[self._weights_count], trainable=True)
self._weights_ref = Tensor(self._weights.name) self._weights_ref = TensorRef(self._weights.name)
def _uniform_init(self, shape, dtype='float32'): def _uniform_init(self, shape, dtype='float32'):
stdv = 1. / numpy.sqrt(self.hidden_size) stdv = 1. / numpy.sqrt(self.hidden_size)
......
dragon/python/core/ops/tensorbind_eager.py
...@@ -146,7 +146,7 @@ def getitem(self, item): ...@@ -146,7 +146,7 @@ def getitem(self, item):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.EagerTensor] item : Union[slice, int, dragon.EagerTensor]
The index. The index.
Returns Returns
...@@ -490,7 +490,7 @@ def setitem(self, key, value): ...@@ -490,7 +490,7 @@ def setitem(self, key, value):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.EagerTensor] key : Union[slice, int, dragon.EagerTensor]
The index. The index.
value : Union[dragon.EagerTensor, number] value : Union[dragon.EagerTensor, number]
The value to set. The value to set.
......
dragon/python/core/ops/tensorbind_symbol.py
...@@ -121,7 +121,7 @@ def getitem(self, item): ...@@ -121,7 +121,7 @@ def getitem(self, item):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.Tensor] item : Union[slice, int, dragon.Tensor]
The index. The index.
Returns Returns
...@@ -324,7 +324,7 @@ def setitem(self, key, value): ...@@ -324,7 +324,7 @@ def setitem(self, key, value):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.Tensor] key : Union[slice, int, dragon.Tensor]
The index. The index.
value : Union[dragon.Tensor, number] value : Union[dragon.Tensor, number]
The value to set. The value to set.
......
tensorflow/__init__.py
...@@ -58,8 +58,8 @@ from dragon.vm.tensorflow.core.framework.dtypes import variant ...@@ -58,8 +58,8 @@ from dragon.vm.tensorflow.core.framework.dtypes import variant
from dragon.vm.tensorflow.core.framework.ops import convert_to_tensor from dragon.vm.tensorflow.core.framework.ops import convert_to_tensor
from dragon.vm.tensorflow.core.framework.ops import device from dragon.vm.tensorflow.core.framework.ops import device
from dragon.vm.tensorflow.core.framework.ops import name_scope from dragon.vm.tensorflow.core.framework.ops import name_scope
from dragon.vm.tensorflow.core.framework.tensor_spec import TensorSpec
from dragon.vm.tensorflow.core.framework.tensor_shape import TensorShape from dragon.vm.tensorflow.core.framework.tensor_shape import TensorShape
from dragon.vm.tensorflow.core.framework.tensor_spec import TensorSpec
from dragon.vm.tensorflow.core.module.module import Module from dragon.vm.tensorflow.core.module.module import Module
from dragon.vm.tensorflow.core.ops.array_ops import broadcast_to from dragon.vm.tensorflow.core.ops.array_ops import broadcast_to
from dragon.vm.tensorflow.core.ops.array_ops import concat from dragon.vm.tensorflow.core.ops.array_ops import concat
......
tensorflow/core/framework/constant_op.py
...@@ -52,9 +52,9 @@ def constant(value, dtype=None, shape=None, name='Const'): ...@@ -52,9 +52,9 @@ def constant(value, dtype=None, shape=None, name='Const'):
""" """
if dtype is not None: if dtype is not None:
if isinstance(value, numpy.ndarray): if isinstance(value, numpy.ndarray):
value = value.astype(str(dtype)) value = value.astype(dtype)
else: else:
value = numpy.array(value, str(dtype)) value = numpy.array(value, dtype)
else: else:
if not isinstance(value, numpy.ndarray): if not isinstance(value, numpy.ndarray):
value = numpy.array(value) value = numpy.array(value)
......
tensorflow/core/framework/framework_lib.py
...@@ -18,5 +18,4 @@ from dragon.vm.tensorflow.core.framework.constant_op import * ...@@ -18,5 +18,4 @@ from dragon.vm.tensorflow.core.framework.constant_op import *
from dragon.vm.tensorflow.core.framework.dtypes import * from dragon.vm.tensorflow.core.framework.dtypes import *
from dragon.vm.tensorflow.core.framework.ops import device from dragon.vm.tensorflow.core.framework.ops import device
from dragon.vm.tensorflow.core.framework.ops import convert_to_tensor from dragon.vm.tensorflow.core.framework.ops import convert_to_tensor
from dragon.vm.tensorflow.core.framework.tensor_shape import Dimension
from dragon.vm.tensorflow.core.framework.tensor_shape import TensorShape from dragon.vm.tensorflow.core.framework.tensor_shape import TensorShape
tensorflow/core/framework/tensor_shape.py
...@@ -12,127 +12,87 @@ ...@@ -12,127 +12,87 @@
# <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/tensor_shape.py> # <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/tensor_shape.py>
# #
# ------------------------------------------------------------ # ------------------------------------------------------------
"""Tensor shape utilities."""
from __future__ import absolute_import from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
from dragon.core.autograph.tensor import Tensor
from dragon.core.eager.tensor import EagerTensor
class TensorShape(tuple):
"""Represent the a sequence of dimensions."""
class Dimension(object): def __init__(self, dims):
def __init__(self, value): """Create a ``TensorShape``.
if value is None:
self._value = None
else:
self._value = int(value)
if self._value < 0:
raise ValueError("Dimension %d must be >= 0" % self._value)
@property Parameters
def value(self): ----------
return self._value dims : Sequence[int]
The dimensions.
def __repr__(self): """
return "Dimension(%s)" % repr(self._value) super(TensorShape, self).__init__()
def __str__(self):
value = self._value
return "?" if value is None else str(value)
def __eq__(self, other):
try:
other = as_dimension(other)
except (TypeError, ValueError):
return NotImplemented
if self._value is None or other.value is None:
return None
return self._value == other.value
def __ne__(self, other):
try:
other = as_dimension(other)
except (TypeError, ValueError):
return NotImplemented
if self._value is None or other.value is None:
return None
return self._value != other.value
def __int__(self):
return self._value
def as_dimension(value):
if isinstance(value, Dimension):
return value
else:
return Dimension(value)
class TensorShape(object):
def __init__(self, dims):
if dims is None:
self._dims = None
elif isinstance(dims, TensorShape):
self._dims = dims.dims
else:
try:
dims_iter = iter(dims)
except TypeError:
self._dims = [as_dimension(dims)]
else:
self._dims = [as_dimension(d) for d in dims_iter]
@property @property
def dims(self): def dims(self):
return self._dims """Return the list of dimensions.
Returns
-------
List[int]
The dimensions.
"""
return list(self)
@property @property
def ndims(self): def ndims(self):
if self._dims is None: """Return the number of dimensions.
return None
else:
return len(self._dims)
def as_list(self): Deprecated. See ``TensorShape.rank``.
if self._dims is None:
raise ValueError("as_list() is not defined on an unknown TensorShape.")
return [dim.value for dim in self._dims]
def __repr__(self): Returns
return "TensorShape(%r)" % self._dims -------
int
The number of dimensions.
def __str__(self): """
if self.ndims is None: return len(self)
return "<unknown>"
elif self.ndims == 1:
return "(%s,)" % self._dims[0]
else:
return "(%s)" % ", ".join(str(d) for d in self._dims)
def __getitem__(self, key): @property
if self._dims is not None: def rank(self):
if isinstance(key, slice): """Return the rank of shape.
return TensorShape(self._dims[key])
else:
return self._dims[key]
else:
return Dimension(None)
Returns
-------
int
The rank.
def dimension_value(dimension): """
"""Return the value of specified dimension.""" return len(self)
if isinstance(dimension, Dimension):
return dimension.value
return dimension
def as_list(self):
"""Return the list of dimensions.
def get_shape(self): Returns
"""Construct the shape descriptor.""" -------
return TensorShape(self.shape) List[int]
The dimensions.
"""
return list(self)
# The Monkey Patching. def __repr__(self):
EagerTensor.get_shape = get_shape return "TensorShape({})".format(list(self))
Tensor.get_shape = get_shape
def __str__(self):
if self.ndims == 1:
return "(%s,)" % self.dims[0]
else:
return "(%s)" % ", ".join(str(d) for d in self.dims)
def __getitem__(self, key):
if isinstance(key, slice):
return TensorShape(self.dims[key])
else:
return self.dims[key]
tensorflow/core/framework/tensor_spec.py
...@@ -17,16 +17,26 @@ from __future__ import absolute_import ...@@ -17,16 +17,26 @@ from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
from dragon.core.framework import tensor_spec
from dragon.vm.tensorflow.core.framework import dtypes from dragon.vm.tensorflow.core.framework import dtypes
from dragon.vm.tensorflow.core.framework import tensor_shape from dragon.vm.tensorflow.core.framework import tensor_shape
class TensorSpec(tensor_spec.TensorSpec): class TensorSpec(object):
"""Spec to describe properties of a tensor.""" """Spec to describe properties of a tensor."""
def __init__(self, shape, dtype=dtypes.float32, name=None): def __init__(self, shape, dtype='float32', name=None):
"""Create a ``TensorSpec``.""" """Create a TensorSpec.
Parameters
----------
shape : Sequence[int], required
The dimensions.
dtype : str, optional, default='float32'
The optional data type.
name : str, optional
The optional name.
"""
self._shape = tensor_shape.TensorShape(shape) self._shape = tensor_shape.TensorShape(shape)
try: try:
self._shape_tuple = tuple(self._shape.as_list()) self._shape_tuple = tuple(self._shape.as_list())
...@@ -45,7 +55,7 @@ class TensorSpec(tensor_spec.TensorSpec): ...@@ -45,7 +55,7 @@ class TensorSpec(tensor_spec.TensorSpec):
The data type. The data type.
""" """
return self._dtype.name return str(self._dtype)
@property @property
def name(self): def name(self):
...@@ -70,3 +80,29 @@ class TensorSpec(tensor_spec.TensorSpec): ...@@ -70,3 +80,29 @@ class TensorSpec(tensor_spec.TensorSpec):
""" """
return self._shape.as_list() return self._shape.as_list()
def is_compatible_with(self, spec_or_tensor):
"""Return a bool whether given the spec is compatible.
Returns
-------
bool
**True** if compatible otherwise **False**.
"""
def dtype_is_compatible_with(spec_or_tensor):
return self.dtype == spec_or_tensor.dtype
def shape_is_compatible_with(spec_or_tensor):
shape = spec_or_tensor.shape
if self._shape is not None and shape is not None:
if len(self.shape) != len(shape):
return False
for x_dim, y_dim in zip(self.shape, shape):
if x_dim != y_dim:
return False
return True
return \
dtype_is_compatible_with(spec_or_tensor) and \
shape_is_compatible_with(spec_or_tensor)
tensorflow/core/keras/engine/input_layer.py
...@@ -43,8 +43,8 @@ def Input( ...@@ -43,8 +43,8 @@ def Input(
x = tf.keras.Input(shape=(8,), batch_size=8, dtype='float32') x = tf.keras.Input(shape=(8,), batch_size=8, dtype='float32')
# Create a placeholder aliasing an existing tensor # Create a placeholder aliasing an existing tensor
x = dragon.Tensor('x', shape=(8,), dtype='float32').constant() x = dragon.Tensor(shape=(8,), dtype='float32').constant()
xx = tf.keras.Input(tensor=x) y = tf.keras.Input(tensor=x)
``` ```
Parameters Parameters
...@@ -69,38 +69,29 @@ def Input( ...@@ -69,38 +69,29 @@ def Input(
if 'batch_shape' in kwargs: if 'batch_shape' in kwargs:
batch_shape = kwargs.pop('batch_shape') batch_shape = kwargs.pop('batch_shape')
if shape and batch_shape: if shape and batch_shape:
raise ValueError( raise ValueError('Specify <shape> or <batch_shape>, not both.')
'Specify <shape> or '
'<batch_shape>, not both.'
)
shape = batch_shape shape = batch_shape
else: else:
if shape is not None: if shape is not None:
shape = (batch_size,) + tuple(shape) shape = (batch_size,) + tuple(shape)
if kwargs: if kwargs:
raise ValueError('Unrecognized keyword arguments:', kwargs.keys()) raise ValueError('Unrecognized keyword arguments:', kwargs.keys())
if dtype is None: if dtype is None:
if tensor is not None: if tensor is not None:
dtype = tensor.dtype dtype = tensor.dtype
else: else:
dtype = 'float32' dtype = 'float32'
if shape is None: if shape is None:
if tensor is None: if tensor is None:
raise ValueError('Specify either <shape> or <tensor>.') raise ValueError('Specify either <shape> or <tensor>.')
else: else:
shape = tensor.shape shape = tensor.shape
if isinstance(shape, tensor_shape.TensorShape): if isinstance(shape, tensor_shape.TensorShape):
shape = tuple(shape.as_list()) shape = tuple(shape.as_list())
elif isinstance(shape, six.integer_types): elif isinstance(shape, six.integer_types):
shape = (shape,) shape = (shape,)
placeholder = array_ops.placeholder( placeholder = array_ops.placeholder(
dtype=dtype, shape=shape, name=name if name else 'input') dtype=dtype, shape=shape, name=name if name else 'input')
if tensor is not None: if tensor is not None:
workspace.get_workspace().register_alias(tensor, placeholder.id) workspace.get_workspace().register_alias(tensor, placeholder.id)
return placeholder return placeholder
tensorflow/core/keras/engine/input_spec.py
...@@ -78,8 +78,7 @@ def assert_input_compatibility(input_spec, inputs, layer_name): ...@@ -78,8 +78,7 @@ def assert_input_compatibility(input_spec, inputs, layer_name):
raise ValueError( raise ValueError(
'Layer ' + layer_name + ' expects ' + 'Layer ' + layer_name + ' expects ' +
str(len(input_spec)) + ' inputs, ' str(len(input_spec)) + ' inputs, '
'but it received ' + str(len(inputs)) + ' input tensors.' 'but it received ' + str(len(inputs)) + ' input tensors.')
)
# For each pair of input and spec. # For each pair of input and spec.
for input_index, (x, spec) in enumerate(zip(inputs, input_spec)): for input_index, (x, spec) in enumerate(zip(inputs, input_spec)):
if spec is None: if spec is None:
......
tensorflow/core/keras/layers/convolutional.py
...@@ -72,13 +72,11 @@ class Conv(Layer): ...@@ -72,13 +72,11 @@ class Conv(Layer):
def build(self, input_shape): def build(self, input_shape):
input_shape = tensor_shape.TensorShape(input_shape) input_shape = tensor_shape.TensorShape(input_shape)
channel_axis = self._get_channel_axis() channel_axis = self._get_channel_axis()
if input_shape.dims[channel_axis].value is None: if input_shape.dims[channel_axis] is None:
raise ValueError( raise ValueError(
'The channel dimension of the input ' 'The channel dimension of the input '
'should be determined, got None.' 'should be determined, got None.')
)
input_dim = int(input_shape[channel_axis]) input_dim = int(input_shape[channel_axis])
# Assume that kernel is packed into NCHW format # Assume that kernel is packed into NCHW format
# for computing the fans correctly # for computing the fans correctly
if self.filters > 0: if self.filters > 0:
...@@ -86,7 +84,6 @@ class Conv(Layer): ...@@ -86,7 +84,6 @@ class Conv(Layer):
else: else:
self.filters = input_dim self.filters = input_dim
kernel_shape = (input_dim, 1) + self.kernel_size kernel_shape = (input_dim, 1) + self.kernel_size
self.kernel = self.add_weight( self.kernel = self.add_weight(
name='kernel', name='kernel',
shape=kernel_shape, shape=kernel_shape,
...@@ -106,7 +103,6 @@ class Conv(Layer): ...@@ -106,7 +103,6 @@ class Conv(Layer):
) )
else: else:
self.bias = None self.bias = None
self.built = True self.built = True
def call(self, inputs): def call(self, inputs):
...@@ -280,17 +276,15 @@ class Conv2DTranspose(Conv2D): ...@@ -280,17 +276,15 @@ class Conv2DTranspose(Conv2D):
def build(self, input_shape): def build(self, input_shape):
input_shape = tensor_shape.TensorShape(input_shape) input_shape = tensor_shape.TensorShape(input_shape)
channel_axis = self._get_channel_axis() channel_axis = self._get_channel_axis()
if input_shape.dims[channel_axis].value is None: if input_shape.dims[channel_axis] is None:
raise ValueError( raise ValueError(
'The channel dimension of the inputs ' 'The channel dimension of the inputs '
'should be determined, got None.' 'should be determined, got None.'
) )
input_dim = int(input_shape[channel_axis]) input_dim = int(input_shape[channel_axis])
# Assume that kernel is packed into NCHW format, # Assume that kernel is packed into NCHW format,
# for computing the fans correctly. # for computing the fans correctly.
kernel_shape = (input_dim, self.filters) + self.kernel_size kernel_shape = (input_dim, self.filters) + self.kernel_size
self.kernel = self.add_weight( self.kernel = self.add_weight(
name='kernel', name='kernel',
shape=kernel_shape, shape=kernel_shape,
...@@ -310,7 +304,6 @@ class Conv2DTranspose(Conv2D): ...@@ -310,7 +304,6 @@ class Conv2DTranspose(Conv2D):
) )
else: else:
self.bias = None self.bias = None
self.built = True self.built = True
def call(self, inputs): def call(self, inputs):
...@@ -320,11 +313,9 @@ class Conv2DTranspose(Conv2D): ...@@ -320,11 +313,9 @@ class Conv2DTranspose(Conv2D):
h_axis, w_axis = 2, 3 h_axis, w_axis = 2, 3
else: else:
h_axis, w_axis = 1, 2 h_axis, w_axis = 1, 2
height, width = inputs_shape[h_axis], inputs_shape[w_axis] height, width = inputs_shape[h_axis], inputs_shape[w_axis]
kernel_h, kernel_w = self.kernel_size kernel_h, kernel_w = self.kernel_size
stride_h, stride_w = self.strides stride_h, stride_w = self.strides
if self.output_padding is None: if self.output_padding is None:
out_pad_h = out_pad_w = None out_pad_h = out_pad_w = None
else: else:
...@@ -349,7 +340,6 @@ class Conv2DTranspose(Conv2D): ...@@ -349,7 +340,6 @@ class Conv2DTranspose(Conv2D):
output_shape = (batch_size, self.filters, out_height, out_width) output_shape = (batch_size, self.filters, out_height, out_width)
else: else:
output_shape = (batch_size, out_height, out_width, self.filters) output_shape = (batch_size, out_height, out_width, self.filters)
outputs = nn_ops.conv_transpose( outputs = nn_ops.conv_transpose(
input=inputs, input=inputs,
filters=self.kernel, filters=self.kernel,
......
tensorflow/core/keras/layers/core.py
...@@ -83,20 +83,17 @@ class Dense(Layer): ...@@ -83,20 +83,17 @@ class Dense(Layer):
if not (dtype.is_floating or dtype.is_complex): if not (dtype.is_floating or dtype.is_complex):
raise TypeError( raise TypeError(
'Unable to build `Dense` layer with non-floating point ' 'Unable to build `Dense` layer with non-floating point '
'dtype %s' % (dtype,) 'dtype %s' % (dtype,))
)
if self.input_dim is None: if self.input_dim is None:
input_shape = tensor_shape.TensorShape(input_shape) input_shape = tensor_shape.TensorShape(input_shape)
if tensor_shape.dimension_value(input_shape[-1]) is None: if input_shape[-1] is None:
raise ValueError( raise ValueError(
'The last dimension of the inputs should be defined.\n' 'The last dimension of the inputs should be defined.\n'
'Or you should specify <input_dim> in the constructor.' 'Or you should specify <input_dim> in the constructor.')
) last_dim = input_shape[-1]
last_dim = tensor_shape.dimension_value(input_shape[-1])
else: else:
last_dim = self.input_dim last_dim = self.input_dim
self.input_spec = InputSpec(min_ndim=2, axes={-1: last_dim}) self.input_spec = InputSpec(min_ndim=2, axes={-1: last_dim})
self.kernel = self.add_weight( self.kernel = self.add_weight(
'kernel', 'kernel',
shape=[last_dim, self.units], shape=[last_dim, self.units],
......
tensorflow/core/keras/layers/normalization.py
...@@ -97,7 +97,7 @@ class BatchNormalization(Layer): ...@@ -97,7 +97,7 @@ class BatchNormalization(Layer):
input_shape = tensor_shape.TensorShape(input_shape) input_shape = tensor_shape.TensorShape(input_shape)
if not input_shape.ndims: if not input_shape.ndims:
raise ValueError('Input has undefined rank:', input_shape) raise ValueError('Input has undefined rank:', input_shape)
param_shape = [input_shape.dims[self.axis].value] param_shape = [input_shape.dims[self.axis]]
self.input_spec = InputSpec( self.input_spec = InputSpec(
# Each layer should adapt to the: # Each layer should adapt to the:
# 1) The number of dimensions. # 1) The number of dimensions.
......
tensorflow/core/ops/array_ops.py
...@@ -217,12 +217,7 @@ def fill(dims, value=0, dtype=None, name=None): ...@@ -217,12 +217,7 @@ def fill(dims, value=0, dtype=None, name=None):
dtype = 'int32' dtype = 'int32'
elif dtype == numpy.float64: elif dtype == numpy.float64:
dtype = 'float32' dtype = 'float32'
return init_ops.fill( return init_ops.fill(shape=dims, value=value, dtype=dtype, name=name)
shape=dims,
value=value,
dtype=str(dtype),
name=name,
)
def gather(params, indices, axis=0, name=None): def gather(params, indices, axis=0, name=None):
...@@ -315,7 +310,7 @@ def ones(shape, dtype='float32', name=None): ...@@ -315,7 +310,7 @@ def ones(shape, dtype='float32', name=None):
A optional name for the operation. A optional name for the operation.
""" """
return init_ops.fill(shape, value=1, dtype=str(dtype), name=name) return init_ops.fill(shape, value=1, dtype=dtype, name=name)
def ones_like(input, dtype='float32', name=None): def ones_like(input, dtype='float32', name=None):
...@@ -340,7 +335,7 @@ def ones_like(input, dtype='float32', name=None): ...@@ -340,7 +335,7 @@ def ones_like(input, dtype='float32', name=None):
A optional name for the operation. A optional name for the operation.
""" """
return init_ops.ones_like(input, dtype=str(dtype), name=name) return init_ops.ones_like(input, dtype=dtype, name=name)
def one_hot( def one_hot(
...@@ -488,13 +483,13 @@ def placeholder(dtype=None, shape=None, name=None): ...@@ -488,13 +483,13 @@ def placeholder(dtype=None, shape=None, name=None):
workspace.get_workspace().unique_name( workspace.get_workspace().unique_name(
context.get_name_scope() + name if name else 'Placeholder', context.get_name_scope() + name if name else 'Placeholder',
suffix=':0', namespace='Tensor'), suffix=':0', namespace='Tensor'),
dtype=str(dtype) if dtype else dtype, dtype=dtype if dtype else dtype,
shape=shape, shape=shape,
).constant() ).constant()
def reshape(tensor, shape, name=None): def reshape(tensor, shape, name=None):
r"""Change the dimensions of input. """Change the dimensions of input.
Examples: Examples:
...@@ -527,7 +522,7 @@ def reshape(tensor, shape, name=None): ...@@ -527,7 +522,7 @@ def reshape(tensor, shape, name=None):
def shape(input, name=None): def shape(input, name=None):
r"""Return the shape of input. """Return the shape of input.
Examples: Examples:
...@@ -641,7 +636,7 @@ def split( ...@@ -641,7 +636,7 @@ def split(
axis=0, axis=0,
name=None, name=None,
): ):
r"""Split input into chunks along the given axis. """Split input into chunks along the given axis.
Either number or size of splits will be accepted: Either number or size of splits will be accepted:
...@@ -775,7 +770,7 @@ def zeros(shape, dtype='float32', name=None): ...@@ -775,7 +770,7 @@ def zeros(shape, dtype='float32', name=None):
A optional name for the operation. A optional name for the operation.
""" """
return init_ops.fill(shape, value=0., dtype=str(dtype), name=name) return init_ops.fill(shape, value=0., dtype=dtype, name=name)
def zeros_like(input, dtype='float32', name=None): def zeros_like(input, dtype='float32', name=None):
...@@ -800,4 +795,4 @@ def zeros_like(input, dtype='float32', name=None): ...@@ -800,4 +795,4 @@ def zeros_like(input, dtype='float32', name=None):
A optional name for the operation. A optional name for the operation.
""" """
return init_ops.zeros_like(input, dtype=str(dtype), name=name) return init_ops.zeros_like(input, dtype=dtype, name=name)
tensorflow/core/ops/init_ops.py
...@@ -79,7 +79,7 @@ class Constant(Initializer): ...@@ -79,7 +79,7 @@ class Constant(Initializer):
The output tensor. The output tensor.
""" """
dtype = str(self.dtype) if dtype is None else str(dtype) dtype = str(self.dtype) if dtype is None else dtype
return init_ops.fill(shape, value=self.value, dtype=dtype) return init_ops.fill(shape, value=self.value, dtype=dtype)
...@@ -125,7 +125,7 @@ class RandomNormal(Initializer): ...@@ -125,7 +125,7 @@ class RandomNormal(Initializer):
shape=shape, shape=shape,
mean=self.mean, mean=self.mean,
std=self.stddev, std=self.stddev,
dtype=str(self.dtype) if dtype is None else str(dtype), dtype=str(self.dtype) if dtype is None else dtype,
) )
...@@ -167,12 +167,12 @@ class RandomUniform(Initializer): ...@@ -167,12 +167,12 @@ class RandomUniform(Initializer):
The output tensor. The output tensor.
""" """
dtype = str(self.dtype) if dtype is None else str(dtype) dtype = str(self.dtype) if dtype is None else dtype
return init_ops.random_uniform( return init_ops.random_uniform(
shape=shape, shape=shape,
low=self.minval, low=self.minval,
high=self.maxval, high=self.maxval,
dtype=str(self.dtype) if dtype is None else str(dtype), dtype=str(self.dtype) if dtype is None else dtype,
) )
...@@ -218,7 +218,7 @@ class TruncatedNormal(Initializer): ...@@ -218,7 +218,7 @@ class TruncatedNormal(Initializer):
shape=shape, shape=shape,
mean=self.mean, mean=self.mean,
std=self.stddev, std=self.stddev,
dtype=str(self.dtype) if dtype is None else str(dtype), dtype=str(self.dtype) if dtype is None else dtype,
) )
...@@ -280,14 +280,14 @@ class VarianceScaling(Initializer): ...@@ -280,14 +280,14 @@ class VarianceScaling(Initializer):
shape=shape, shape=shape,
mode=self.mode, mode=self.mode,
scale=self.scale * 2.0, scale=self.scale * 2.0,
dtype=str(self.dtype) if dtype is None else str(dtype) dtype=str(self.dtype) if dtype is None else dtype
) )
else: else:
return init_ops.glorot_uniform( return init_ops.glorot_uniform(
shape=shape, shape=shape,
mode=self.mode, mode=self.mode,
scale=self.scale * 3.0, scale=self.scale * 3.0,
dtype=str(self.dtype) if dtype is None else str(dtype) dtype=str(self.dtype) if dtype is None else dtype
) )
...@@ -374,8 +374,8 @@ class Ones(Initializer): ...@@ -374,8 +374,8 @@ class Ones(Initializer):
The output tensor. The output tensor.
""" """
dtype = str(self.dtype) if dtype is None else str(dtype) dtype = str(self.dtype) if dtype is None else dtype
return init_ops.fill(shape, value=1, dtype=str(dtype)) return init_ops.fill(shape, value=1, dtype=dtype)
class Zeros(Initializer): class Zeros(Initializer):
...@@ -412,7 +412,7 @@ class Zeros(Initializer): ...@@ -412,7 +412,7 @@ class Zeros(Initializer):
The output tensor. The output tensor.
""" """
dtype = str(self.dtype) if dtype is None else str(dtype) dtype = str(self.dtype) if dtype is None else dtype
return init_ops.fill(shape, value=0, dtype=dtype) return init_ops.fill(shape, value=0, dtype=dtype)
......
tensorflow/core/ops/linalg_ops.py
...@@ -48,4 +48,4 @@ def eye(num_rows, num_columns=None, dtype='float32', name=None): ...@@ -48,4 +48,4 @@ def eye(num_rows, num_columns=None, dtype='float32', name=None):
The output tensor. The output tensor.
""" """
return init_ops.eye(num_rows, num_columns, dtype=str(dtype), name=name) return init_ops.eye(num_rows, num_columns, dtype=dtype, name=name)
tensorflow/core/ops/math_ops.py
...@@ -209,7 +209,7 @@ def cast(x, dtype, name=None): ...@@ -209,7 +209,7 @@ def cast(x, dtype, name=None):
The output tensor. The output tensor.
""" """
return array_ops.cast(x, dtype=str(dtype), name=name) return array_ops.cast(x, dtype=dtype, name=name)
def ceil(x, name=None): def ceil(x, name=None):
...@@ -890,7 +890,7 @@ def range(start, limit=None, delta=1, dtype='int64', name=None): ...@@ -890,7 +890,7 @@ def range(start, limit=None, delta=1, dtype='int64', name=None):
start=start, start=start,
stop=limit, stop=limit,
step=delta, step=delta,
dtype=str(dtype), dtype=dtype,
name=name, name=name,
) )
......
tensorflow/core/ops/nn_ops.py
...@@ -56,12 +56,11 @@ def avg_pool( ...@@ -56,12 +56,11 @@ def avg_pool(
The output tensor. The output tensor.
""" """
num_total_dims = input.get_shape().ndims if input.shape is not None:
if num_total_dims is None: num_total_dims = len(input.shape)
else:
num_total_dims = len(ksize) num_total_dims = len(ksize)
num_spatial_dims = num_total_dims - 2 num_spatial_dims = num_total_dims - 2
# Make default parameters
data_format = data_format if data_format else 'NHWC' data_format = data_format if data_format else 'NHWC'
start_axis = 2 if data_format.startswith('NC') else 1 start_axis = 2 if data_format.startswith('NC') else 1
normalize_spatial_args = \ normalize_spatial_args = \
...@@ -74,7 +73,6 @@ def avg_pool( ...@@ -74,7 +73,6 @@ def avg_pool(
ksize = normalize_spatial_args('ksize', ksize) ksize = normalize_spatial_args('ksize', ksize)
strides = normalize_spatial_args('strides', strides) strides = normalize_spatial_args('strides', strides)
padding, pads = normalize_spatial_args('padding', padding) padding, pads = normalize_spatial_args('padding', padding)
return getattr(vision_ops, 'pool{}d'.format(num_spatial_dims))( return getattr(vision_ops, 'pool{}d'.format(num_spatial_dims))(
[input], [input],
kernel_shape=ksize.shape[start_axis:start_axis + num_spatial_dims], kernel_shape=ksize.shape[start_axis:start_axis + num_spatial_dims],
...@@ -173,12 +171,11 @@ def convolution( ...@@ -173,12 +171,11 @@ def convolution(
The output tensor. The output tensor.
""" """
num_total_dims = filters.get_shape().ndims if filters.shape is not None:
if num_total_dims is None: num_total_dims = len(filters.shape)
raise ValueError('Rank of `filters` must be determined.') else:
raise ValueError('Rank of <filters> must be determined.')
num_spatial_dims = num_total_dims - 2 num_spatial_dims = num_total_dims - 2
# Make default parameters
data_format = data_format if data_format else 'NHWC' data_format = data_format if data_format else 'NHWC'
start_axis = 2 if data_format.startswith('NC') else 1 start_axis = 2 if data_format.startswith('NC') else 1
normalize_spatial_args = \ normalize_spatial_args = \
...@@ -191,7 +188,6 @@ def convolution( ...@@ -191,7 +188,6 @@ def convolution(
strides = normalize_spatial_args('strides', strides) strides = normalize_spatial_args('strides', strides)
dilations = normalize_spatial_args('dilations', dilations) dilations = normalize_spatial_args('dilations', dilations)
padding, pads = normalize_spatial_args('padding', padding) padding, pads = normalize_spatial_args('padding', padding)
return getattr(vision_ops, '{}{}d'.format( return getattr(vision_ops, '{}{}d'.format(
kwargs.get('conv_type', 'conv'), num_spatial_dims))( kwargs.get('conv_type', 'conv'), num_spatial_dims))(
[input, filters], [input, filters],
...@@ -241,14 +237,13 @@ def conv_transpose( ...@@ -241,14 +237,13 @@ def conv_transpose(
The output tensor. The output tensor.
""" """
num_total_dims = filters.get_shape().ndims if filters.shape is not None:
if num_total_dims is None: num_total_dims = len(filters.shape)
num_total_dims = input.get_shape().ndims elif input.shape is not None:
if num_total_dims is None: num_total_dims = len(input.shape)
raise ValueError("rank of input or filters must be known.") else:
raise ValueError('Rank of <input> or <filters> must be known.')
num_spatial_dims = num_total_dims - 2 num_spatial_dims = num_total_dims - 2
# Make default parameters
data_format = data_format if data_format else 'NHWC' data_format = data_format if data_format else 'NHWC'
start_axis = 2 if data_format.startswith('NC') else 1 start_axis = 2 if data_format.startswith('NC') else 1
normalize_spatial_args = \ normalize_spatial_args = \
...@@ -264,7 +259,6 @@ def conv_transpose( ...@@ -264,7 +259,6 @@ def conv_transpose(
if padding == 'SAME' and output_shape is None: if padding == 'SAME' and output_shape is None:
raise ValueError('Excepted <output_shape> for same padding.') raise ValueError('Excepted <output_shape> for same padding.')
output_shape = normalize_spatial_args('output_shape', output_shape) output_shape = normalize_spatial_args('output_shape', output_shape)
return getattr(vision_ops, 'conv{}d_transpose'.format(num_spatial_dims))( return getattr(vision_ops, 'conv{}d_transpose'.format(num_spatial_dims))(
[input, filters], [input, filters],
kernel_shape=filters.shape[2:], kernel_shape=filters.shape[2:],
...@@ -608,12 +602,11 @@ def max_pool( ...@@ -608,12 +602,11 @@ def max_pool(
The output tensor. The output tensor.
""" """
num_total_dims = input.get_shape().ndims if input.shape is not None:
if num_total_dims is None: num_total_dims = len(input.shape)
else:
num_total_dims = len(ksize) num_total_dims = len(ksize)
num_spatial_dims = num_total_dims - 2 num_spatial_dims = num_total_dims - 2
# Make default parameters
data_format = data_format if data_format else 'NHWC' data_format = data_format if data_format else 'NHWC'
start_axis = 2 if data_format.startswith('NC') else 1 start_axis = 2 if data_format.startswith('NC') else 1
normalize_spatial_args = \ normalize_spatial_args = \
...@@ -626,7 +619,6 @@ def max_pool( ...@@ -626,7 +619,6 @@ def max_pool(
ksize = normalize_spatial_args('ksize', ksize) ksize = normalize_spatial_args('ksize', ksize)
strides = normalize_spatial_args('strides', strides) strides = normalize_spatial_args('strides', strides)
padding, pads = normalize_spatial_args('padding', padding) padding, pads = normalize_spatial_args('padding', padding)
return getattr(vision_ops, 'pool{}d'.format(num_spatial_dims))( return getattr(vision_ops, 'pool{}d'.format(num_spatial_dims))(
[input], [input],
kernel_shape=ksize[start_axis:start_axis + num_spatial_dims], kernel_shape=ksize[start_axis:start_axis + num_spatial_dims],
......
tensorflow/core/ops/random_ops.py
...@@ -50,7 +50,7 @@ def random_normal( ...@@ -50,7 +50,7 @@ def random_normal(
The output tensor. The output tensor.
""" """
_, dtype, init_fn = seed, str(dtype), init_ops.random_normal _, dtype, init_fn = seed, dtype, init_ops.random_normal
return init_fn(shape, mean, stddev, dtype=dtype, name=name) return init_fn(shape, mean, stddev, dtype=dtype, name=name)
...@@ -87,7 +87,7 @@ def random_uniform( ...@@ -87,7 +87,7 @@ def random_uniform(
The output tensor. The output tensor.
""" """
_, dtype, init_fn = seed, str(dtype), init_ops.random_uniform _, dtype, init_fn = seed, dtype, init_ops.random_uniform
return init_fn(shape, minval, maxval, dtype=dtype, name=name) return init_fn(shape, minval, maxval, dtype=dtype, name=name)
...@@ -125,5 +125,5 @@ def truncated_normal( ...@@ -125,5 +125,5 @@ def truncated_normal(
The output tensor. The output tensor.
""" """
_, dtype, init_fn = seed, str(dtype), init_ops.truncated_normal _, dtype, init_fn = seed, dtype, init_ops.truncated_normal
return init_fn(shape, mean, stddev, dtype=dtype, name=name) return init_fn(shape, mean, stddev, dtype=dtype, name=name)
tensorflow/core/ops/variables.py
...@@ -43,27 +43,21 @@ class Variable(VariableMetaclass, EagerTensor): ...@@ -43,27 +43,21 @@ class Variable(VariableMetaclass, EagerTensor):
): ):
"""Create a ``Variable``.""" """Create a ``Variable``."""
super(Variable, self).__init__(trainable=trainable) super(Variable, self).__init__(trainable=trainable)
name = name if name else 'Variable' name = name if name else 'Variable'
self._name = context.get_name_scope() + name + ':0' self._name = context.get_name_scope() + name + ':0'
# Determine th value. # Determine th value.
if isinstance(initial_value, EagerTensor): if isinstance(initial_value, EagerTensor):
initial_value = initial_value.numpy() initial_value = initial_value.numpy()
elif isinstance(initial_value, Tensor): elif isinstance(initial_value, Tensor):
initial_value = initial_value.get_value() initial_value = initial_value.get_value()
# Determine the data type. # Determine the data type.
if not isinstance(initial_value, numpy.ndarray): if not isinstance(initial_value, numpy.ndarray):
initial_value = numpy.array( initial_value = numpy.array(initial_value, dtype if dtype else dtype)
initial_value, str(dtype) if dtype else dtype)
elif dtype is not None: elif dtype is not None:
initial_value = initial_value.astype(str(dtype)) initial_value = initial_value.astype(dtype)
# Determine the tensor shape. # Determine the tensor shape.
if shape is not None: if shape is not None:
initial_value = initial_value.reshape(shape) initial_value = initial_value.reshape(shape)
self._from_numpy(initial_value, copy=False) self._from_numpy(initial_value, copy=False)
@property @property
...@@ -96,7 +90,6 @@ def get_default_initializer(name, shape=None, dtype=dtypes.float32): ...@@ -96,7 +90,6 @@ def get_default_initializer(name, shape=None, dtype=dtypes.float32):
# Defaults: float32. # Defaults: float32.
if dtype is None: if dtype is None:
dtype = dtypes.float32 dtype = dtypes.float32
# Xavier for float16, float32, float64. # Xavier for float16, float32, float64.
if dtype.is_floating: if dtype.is_floating:
initializer = init_ops.glorot_uniform_initializer() initializer = init_ops.glorot_uniform_initializer()
......
test/dragon/core/test_framework.py
...@@ -67,20 +67,20 @@ class TestTensor(unittest.TestCase): ...@@ -67,20 +67,20 @@ class TestTensor(unittest.TestCase):
self.assertEqual(dragon.Tensor().ndim, 0) self.assertEqual(dragon.Tensor().ndim, 0)
self.assertEqual(dragon.Tensor(shape=(2,)).ndim, 1) self.assertEqual(dragon.Tensor(shape=(2,)).ndim, 1)
self.assertEqual(dragon.Tensor().shape, None) self.assertEqual(dragon.Tensor().shape, None)
self.assertEqual(dragon.Tensor(shape=(2,)).shape, [2]) self.assertEqual(dragon.Tensor(shape=(2,)).shape, (2,))
self.assertEqual(dragon.Tensor().size, 0) self.assertEqual(dragon.Tensor().size, 0)
self.assertEqual(dragon.Tensor(shape=(2, None)).size, math.inf) self.assertEqual(dragon.Tensor(shape=(2, None)).size, math.inf)
self.assertEqual(dragon.Tensor(shape=(2,)).size, 2) self.assertEqual(dragon.Tensor(shape=(2,)).size, 2)
self.assertEqual(dragon.Tensor().dtype, None) self.assertEqual(dragon.Tensor().dtype, None)
self.assertEqual(dragon.Tensor(dtype='float32').dtype, 'float32') self.assertEqual(dragon.Tensor(dtype='float32').dtype, 'float32')
self.assertEqual(dragon.EagerTensor(shape=(2,)).ndim, 1) self.assertEqual(dragon.EagerTensor(shape=(2,)).ndim, 1)
self.assertEqual(dragon.EagerTensor(shape=(2,)).shape, [2]) self.assertEqual(dragon.EagerTensor(shape=(2,)).shape, (2,))
self.assertEqual(dragon.EagerTensor(shape=(2,)).size, 2) self.assertEqual(dragon.EagerTensor(shape=(2,)).size, 2)
self.assertEqual(dragon.EagerTensor(shape=(2,), dtype='float32').dtype, 'float32') self.assertEqual(dragon.EagerTensor(shape=(2,), dtype='float32').dtype, 'float32')
self.assertEqual(dragon.EagerTensor().device, dragon.EagerTensor().device) self.assertEqual(dragon.EagerTensor().device, dragon.EagerTensor().device)
self.assertNotEqual(a.__hash__(), b.__hash__()) self.assertNotEqual(a.__hash__(), b.__hash__())
self.assertNotEqual(a.__repr__(), b.__repr__()) self.assertNotEqual(a.__repr__(), b.__repr__())
self.assertNotEqual(b.__repr__(), dragon.EagerTensor([2]).__repr__()) self.assertNotEqual(b.__repr__(), dragon.EagerTensor((2,)).__repr__())
self.assertEqual(int(a.constant().set_value(1)), 1) self.assertEqual(int(a.constant().set_value(1)), 1)
self.assertEqual(float(dragon.Tensor.convert_to(1)), 1.) self.assertEqual(float(dragon.Tensor.convert_to(1)), 1.)
self.assertEqual(int(b.set_value(1)), 1) self.assertEqual(int(b.set_value(1)), 1)
...@@ -117,7 +117,7 @@ class TestTensor(unittest.TestCase): ...@@ -117,7 +117,7 @@ class TestTensor(unittest.TestCase):
x = dragon.EagerTensor(data, copy=True) x = dragon.EagerTensor(data, copy=True)
x_to_dlpack = dragon.dlpack.to_dlpack(x) x_to_dlpack = dragon.dlpack.to_dlpack(x)
x_from_dlpack = dragon.dlpack.from_dlpack(x_to_dlpack) x_from_dlpack = dragon.dlpack.from_dlpack(x_to_dlpack)
self.assertEqual(x_from_dlpack.shape, list(data.shape)) self.assertEqual(x_from_dlpack.shape, data.shape)
self.assertEqual(x_from_dlpack.dtype, str(data.dtype)) self.assertEqual(x_from_dlpack.dtype, str(data.dtype))
self.assertLessEqual(np.abs(x_from_dlpack.numpy() - data).max(), 1e-5) self.assertLessEqual(np.abs(x_from_dlpack.numpy() - data).max(), 1e-5)
...@@ -130,7 +130,7 @@ class TestTensor(unittest.TestCase): ...@@ -130,7 +130,7 @@ class TestTensor(unittest.TestCase):
x_from_dlpack = dragon.dlpack.from_dlpack(x_to_dlpack) x_from_dlpack = dragon.dlpack.from_dlpack(x_to_dlpack)
self.assertEqual(x_from_dlpack.device.type, 'cuda') self.assertEqual(x_from_dlpack.device.type, 'cuda')
self.assertEqual(x_from_dlpack.device.index, 0) self.assertEqual(x_from_dlpack.device.index, 0)
self.assertEqual(x_from_dlpack.shape, list(data.shape)) self.assertEqual(x_from_dlpack.shape, data.shape)
self.assertEqual(x_from_dlpack.dtype, str(data.dtype)) self.assertEqual(x_from_dlpack.dtype, str(data.dtype))
self.assertLessEqual(np.abs(x_from_dlpack.numpy() - data).max(), 1e-5) self.assertLessEqual(np.abs(x_from_dlpack.numpy() - data).max(), 1e-5)
...@@ -150,7 +150,7 @@ class TestWorkspace(unittest.TestCase): ...@@ -150,7 +150,7 @@ class TestWorkspace(unittest.TestCase):
w = dragon.Workspace() w = dragon.Workspace()
with w.as_default(): with w.as_default():
v1, v2 = dragon.EagerTensor(1), np.array(2) v1, v2 = dragon.EagerTensor(1), np.array(2)
x = dragon.Tensor('test_feed_tensor/x') x = dragon.Tensor(name='test_feed_tensor/x')
w.feed_tensor(x, v1) w.feed_tensor(x, v1)
self.assertEqual(int(x), 1) self.assertEqual(int(x), 1)
w.feed_tensor(x, v2) w.feed_tensor(x, v2)
...@@ -159,7 +159,7 @@ class TestWorkspace(unittest.TestCase): ...@@ -159,7 +159,7 @@ class TestWorkspace(unittest.TestCase):
def test_merge_form(self): def test_merge_form(self):
w1, w2 = dragon.Workspace(), dragon.Workspace() w1, w2 = dragon.Workspace(), dragon.Workspace()
with w1.as_default(): with w1.as_default():
x = dragon.Tensor('test_merge_from/x').set_value(0) x = dragon.Tensor(name='test_merge_from/x').set_value(0)
w2.merge_from(w1) w2.merge_from(w1)
with w2.as_default(): with w2.as_default():
self.assertEqual(int(x), 0) self.assertEqual(int(x), 0)
......
test/dragon/core/test_ops.py
...@@ -67,7 +67,7 @@ class OpTestCase(unittest.TestCase): ...@@ -67,7 +67,7 @@ class OpTestCase(unittest.TestCase):
dtype = symbols[i][1].dtype dtype = symbols[i][1].dtype
shape = symbols[i][1].shape shape = symbols[i][1].shape
super(OpTestCase, self).assertEqual(dtype, str(values[i].dtype)) super(OpTestCase, self).assertEqual(dtype, str(values[i].dtype))
super(OpTestCase, self).assertEqual(shape, list(shape)) super(OpTestCase, self).assertEqual(shape, values[i].shape)
inputs[symbols[i][0]] = values[i] inputs[symbols[i][0]] = values[i]
first = inputs[:num_first] if num_first > 1 else inputs[0] first = inputs[:num_first] if num_first > 1 else inputs[0]
second = inputs[num_first:len(inputs)] if num_second > 1 else inputs[num_first] second = inputs[num_first:len(inputs)] if num_second > 1 else inputs[num_first]
...@@ -239,7 +239,7 @@ class TestActivationOps(OpTestCase): ...@@ -239,7 +239,7 @@ class TestActivationOps(OpTestCase):
for x_shape, w_shape, data_format in entries: for x_shape, w_shape, data_format in entries:
data1 = uniform(x_shape) data1 = uniform(x_shape)
data2 = np.ones(w_shape, 'float32') * 0.25 data2 = np.ones(w_shape, 'float32') * 0.25
x, w = new_tensor(data1), new_tensor(data2) x, w = new_tensor(data1), new_tensor(data2.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w]) tape.watch([x, w])
y = dragon.nn.prelu([x, w], data_format=data_format) y = dragon.nn.prelu([x, w], data_format=data_format)
...@@ -632,7 +632,7 @@ class TestArrayOps(OpTestCase): ...@@ -632,7 +632,7 @@ class TestArrayOps(OpTestCase):
tape.watch(x) tape.watch(x)
y = dragon.masked_select([x, x > 2]) y = dragon.masked_select([x, x > 2])
dx = tape.gradient(y, [x], output_gradients=[y])[0] dx = tape.gradient(y, [x], output_gradients=[y])[0]
self.assertEqual([y, dx], [data[data > 2], grad]) self.assertEqual([y, dx], [data[data > 2], grad], test_symbols=False)
@unittest.skipIf(not TEST_CUDA, 'CUDA unavailable') @unittest.skipIf(not TEST_CUDA, 'CUDA unavailable')
def test_masked_select_cuda(self): def test_masked_select_cuda(self):
...@@ -645,7 +645,8 @@ class TestArrayOps(OpTestCase): ...@@ -645,7 +645,8 @@ class TestArrayOps(OpTestCase):
data = arange((2, 3)) data = arange((2, 3))
x = new_tensor(data) x = new_tensor(data)
y = dragon.nonzero(x > 2) y = dragon.nonzero(x > 2)
self.assertEqual(y, np.stack(np.nonzero(data > 2), axis=1)) self.assertEqual(
y, np.stack(np.nonzero(data > 2), axis=1), test_symbols=False)
@unittest.skipIf(not TEST_CUDA, 'CUDA unavailable') @unittest.skipIf(not TEST_CUDA, 'CUDA unavailable')
def test_non_zero_cuda(self): def test_non_zero_cuda(self):
...@@ -699,7 +700,7 @@ class TestArrayOps(OpTestCase): ...@@ -699,7 +700,7 @@ class TestArrayOps(OpTestCase):
tape.watch(x) tape.watch(x)
y = dragon.repeat(x, axis, repeats) y = dragon.repeat(x, axis, repeats)
grad = arange(y.shape) grad = arange(y.shape)
grad_shape = y.shape[:-1] + [y.shape[-1] // 2, 2] grad_shape = y.shape[:-1] + (y.shape[-1] // 2, 2)
dy = new_tensor(grad) dy = new_tensor(grad)
dx = tape.gradient(y, [x], output_gradients=[dy])[0] dx = tape.gradient(y, [x], output_gradients=[dy])[0]
self.assertEqual( self.assertEqual(
...@@ -2271,8 +2272,8 @@ class TestNormalizationOps(OpTestCase): ...@@ -2271,8 +2272,8 @@ class TestNormalizationOps(OpTestCase):
data4, data5 = arange(w_shape) * .1, arange(w_shape, 1) * .1 data4, data5 = arange(w_shape) * .1, arange(w_shape, 1) * .1
data6 = uniform(x_shape) data6 = uniform(x_shape)
x = new_tensor(data1) x = new_tensor(data1)
w, b = new_tensor(data2), new_tensor(data3) w, b = new_tensor(data2.flatten()), new_tensor(data3.flatten())
rm, rv = new_tensor(data4), new_tensor(data5) rm, rv = new_tensor(data4.flatten()), new_tensor(data5.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w, b]) tape.watch([x, w, b])
y = dragon.nn.batch_norm( y = dragon.nn.batch_norm(
...@@ -2330,7 +2331,7 @@ class TestNormalizationOps(OpTestCase): ...@@ -2330,7 +2331,7 @@ class TestNormalizationOps(OpTestCase):
data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1 data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1
data6 = arange(x_shape) * .1 data6 = arange(x_shape) * .1
x = new_tensor(data1) x = new_tensor(data1)
w, b = new_tensor(data2), new_tensor(data3) w, b = new_tensor(data2.flatten()), new_tensor(data3.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w, b]) tape.watch([x, w, b])
y = dragon.nn.group_norm( y = dragon.nn.group_norm(
...@@ -2374,7 +2375,7 @@ class TestNormalizationOps(OpTestCase): ...@@ -2374,7 +2375,7 @@ class TestNormalizationOps(OpTestCase):
data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1 data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1
data6 = arange(x_shape) * 10. data6 = arange(x_shape) * 10.
x = new_tensor(data1) x = new_tensor(data1)
w, b = new_tensor(data2), new_tensor(data3) w, b = new_tensor(data2.flatten()), new_tensor(data3.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w, b]) tape.watch([x, w, b])
y = dragon.nn.instance_norm([x, w, b], axis=axis, eps=eps) y = dragon.nn.instance_norm([x, w, b], axis=axis, eps=eps)
...@@ -2417,7 +2418,7 @@ class TestNormalizationOps(OpTestCase): ...@@ -2417,7 +2418,7 @@ class TestNormalizationOps(OpTestCase):
data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1 data2, data3 = arange(w_shape, 1) * .1, arange(w_shape) * .1
data6 = arange(x_shape) * 10. data6 = arange(x_shape) * 10.
x = new_tensor(data1) x = new_tensor(data1)
w, b = new_tensor(data2), new_tensor(data3) w, b = new_tensor(data2.flatten()), new_tensor(data3.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w, b]) tape.watch([x, w, b])
y = dragon.nn.layer_norm([x, w, b], axis=axis, eps=eps) y = dragon.nn.layer_norm([x, w, b], axis=axis, eps=eps)
...@@ -2587,7 +2588,7 @@ class TestTensorOps(OpTestCase): ...@@ -2587,7 +2588,7 @@ class TestTensorOps(OpTestCase):
grad[data > 2] = 1 grad[data > 2] = 1
grad *= data grad *= data
x = new_tensor(data) x = new_tensor(data)
self.assertEqual(x[x > 2], data[data > 2]) self.assertEqual(x[x > 2], data[data > 2], test_symbols=False)
entries = [0, entries = [0,
slice(None, None, None), slice(None, None, None),
slice(0, None, None), slice(0, None, None),
...@@ -2885,13 +2886,13 @@ class TestVisionOps(OpTestCase): ...@@ -2885,13 +2886,13 @@ class TestVisionOps(OpTestCase):
with execution_context().mode(execution): with execution_context().mode(execution):
for x_shape, b_shape, data_format in entries: for x_shape, b_shape, data_format in entries:
data1, data2 = arange(x_shape), arange(b_shape) data1, data2 = arange(x_shape), arange(b_shape)
x, w = new_tensor(data1), new_tensor(data2) x, b = new_tensor(data1), new_tensor(data2.flatten())
with dragon.GradientTape() as tape: with dragon.GradientTape() as tape:
tape.watch([x, w]) tape.watch([x, b])
y = dragon.nn.bias_add([x, w], data_format) y = dragon.nn.bias_add([x, b], data_format)
dx, dw = tape.gradient(y, [x, w], output_gradients=[x]) dx, db = tape.gradient(y, [x, b], output_gradients=[x])
self.assertEqual( self.assertEqual(
[y, dx, dw], [y, dx, db],
[data1 + data2, data1, reduce_like(data1, data2).flatten()]) [data1 + data2, data1, reduce_like(data1, data2).flatten()])
@unittest.skipIf(not TEST_CUDA, 'CUDA unavailable') @unittest.skipIf(not TEST_CUDA, 'CUDA unavailable')
...@@ -3422,7 +3423,7 @@ def new_tensor(data): ...@@ -3422,7 +3423,7 @@ def new_tensor(data):
"""Create a new tensor for current execution.""" """Create a new tensor for current execution."""
if execution_context().executing_eagerly(): if execution_context().executing_eagerly():
return dragon.EagerTensor(data, copy=True) return dragon.EagerTensor(data, copy=True)
return dragon.Tensor(None, data.shape, str(data.dtype)).set_value(data) return dragon.Tensor(data.shape, str(data.dtype)).set_value(data)
def process_indices(item): def process_indices(item):
......
torch/__init__.py
...@@ -24,6 +24,7 @@ from dragon.vm.torch import vision ...@@ -24,6 +24,7 @@ from dragon.vm.torch import vision
# Classes # Classes
from dragon.vm.torch.autograd import Variable from dragon.vm.torch.autograd import Variable
from dragon.vm.torch.cpp import device from dragon.vm.torch.cpp import device
from dragon.vm.torch.cpp import dtype
from dragon.vm.torch.cpp import Size from dragon.vm.torch.cpp import Size
from dragon.vm.torch.tensor import ByteTensor from dragon.vm.torch.tensor import ByteTensor
from dragon.vm.torch.tensor import CharTensor from dragon.vm.torch.tensor import CharTensor
...@@ -116,3 +117,21 @@ from dragon.vm.torch.serialization import load ...@@ -116,3 +117,21 @@ from dragon.vm.torch.serialization import load
from dragon.vm.torch.serialization import save from dragon.vm.torch.serialization import save
from dragon.vm.torch.tensor import empty from dragon.vm.torch.tensor import empty
from dragon.vm.torch.tensor import tensor from dragon.vm.torch.tensor import tensor
# Aliases
bool = dtype('bool')
int8 = dtype('int8')
uint8 = dtype('uint8')
int16 = short = dtype('int16')
int32 = int = dtype('int32')
int64 = long = dtype('int64')
qint8 = dtype('qint8')
quint8 = dtype('quint8')
qint32 = dtype('qint32')
bfloat16 = dtype('bfloat16')
float16 = half = dtype('float16')
float32 = float = dtype('float32')
float64 = double = dtype('float64')
complex32 = dtype('complex32')
complex64 = dtype('complex64')
complex128 = dtype('complex128')
torch/cpp.py
...@@ -58,7 +58,7 @@ class Size(tuple): ...@@ -58,7 +58,7 @@ class Size(tuple):
class device(object): class device(object):
"""Represent the device where tensor will be allocated.""" """Represent the device spec."""
def __init__(self, type='cpu', index=0): def __init__(self, type='cpu', index=0):
"""Create a ``device``. """Create a ``device``.
...@@ -100,6 +100,57 @@ class device(object): ...@@ -100,6 +100,57 @@ class device(object):
return 'device(type={}, index={})'.format(self.type, self.index) return 'device(type={}, index={})'.format(self.type, self.index)
class dtype(str):
"""The basic data type.
Following data types are defined:
* ``torch.float16`` or ``torch.half``: 16-bit half-precision floating-point.
* ``torch.float32`` or ``torch.float``: 32-bit single-precision floating-point.
* ``torch.float64`` or ``torch.double``: 64-bit double-precision floating-point.
* ``torch.bfloat16``: 16-bit truncated floating-point.
* ``torch.complex32``: 32-bit single-precision complex.
* ``torch.complex64``: 64-bit single-precision complex.
* ``torch.complex128``: 128-bit double-precision complex.
* ``torch.int8``: 8-bit signed integer.
* ``torch.uint8``: 8-bit unsigned integer.
* ``torch.int16`` or ``torch.short``: 16-bit signed integer.
* ``torch.int32`` or ``torch.int``: 32-bit signed integer.
* ``torch.int64`` or ``torch.long``: 64-bit signed integer.
* ``torch.bool``: Boolean.
* ``torch.qint8``: Quantized 8-bit signed integer.
* ``torch.quint8``: Quantized 8-bit unsigned integer.
* ``torch.qint32``: Quantized 32-bit signed integer.
"""
def __init__(self, s):
"""Create a ``dtype``.
Parameters
----------
s : str
The data type descriptor.
"""
super(dtype, self).__init__()
def from_numpy(array): def from_numpy(array):
"""Create a tensor from the given numpy array. """Create a tensor from the given numpy array.
......
torch/ops/tensorbind.py
...@@ -636,7 +636,7 @@ def getitem(self, item): ...@@ -636,7 +636,7 @@ def getitem(self, item):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.vm.torch.Tensor] item : Union[slice, int, dragon.vm.torch.Tensor]
The index. The index.
Returns Returns
...@@ -1318,7 +1318,7 @@ def setitem(self, key, value): ...@@ -1318,7 +1318,7 @@ def setitem(self, key, value):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.vm.torch.Tensor] key : Union[slice, int, dragon.vm.torch.Tensor]
The index. The index.
value : Union[dragon.vm.torch.Tensor, number] value : Union[dragon.vm.torch.Tensor, number]
The value to set. The value to set.
......
torch/tensor.py
...@@ -1940,7 +1940,7 @@ class Tensor(object): ...@@ -1940,7 +1940,7 @@ class Tensor(object):
Parameters Parameters
---------- ----------
item : Union[int, slice, dragon.vm.torch.Tensor] item : Union[slice, int, dragon.vm.torch.Tensor]
The index. The index.
Returns Returns
...@@ -2198,7 +2198,7 @@ class Tensor(object): ...@@ -2198,7 +2198,7 @@ class Tensor(object):
Parameters Parameters
---------- ----------
key : Union[int, slice, dragon.vm.torch.Tensor] key : Union[slice, int, dragon.vm.torch.Tensor]
The index. The index.
value : Union[dragon.vm.torch.Tensor, number] value : Union[dragon.vm.torch.Tensor, number]
The value to set. The value to set.
......
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!