Initializer that generates tensors without scaling variance.
`tf.uniform_unit_scaling_initializer`Compat aliases for migration
See Migration guide for more details.
tf.initializers.uniform_unit_scaling( factor=1.0, seed=None, dtype=tf.dtypes.float32 )
When initializing a deep network, it is in principle advantageous to keep the scale of the input variance constant, so it does not explode or diminish by reaching the final layer. If the input is
x and the operation
x * W, and we want to initialize
W uniformly at random, we need to pick
[-sqrt(3) / sqrt(dim), sqrt(3) / sqrt(dim)]
to keep the scale intact, where
dim = W.shape (the size of the input). A similar calculation for convolutional networks gives an analogous result with
dim equal to the product of the first 3 dimensions. When nonlinearities are present, we need to multiply this by a constant
factor. See (Sussillo et al., 2014) for deeper motivation, experiments and the calculation of constants. In section 2.3 there, the constants were numerically computed: for a linear layer it's 1.0, relu: ~1.43, tanh: ~1.15.
| ||Float. A multiplicative factor by which the values will be scaled.|
| || A Python integer. Used to create random seeds. See |
| || Default data type, used if no |
@classmethod from_config( config )
Instantiates an initializer from a configuration dictionary.
initializer = RandomUniform(-1, 1) config = initializer.get_config() initializer = RandomUniform.from_config(config)
| || A Python dictionary. It will typically be the output of |
|An Initializer instance.|
Returns the configuration of the initializer as a JSON-serializable dict.
|A JSON-serializable Python dict.|
__call__( shape, dtype=None, partition_info=None )
Returns a tensor object initialized as specified by the initializer.
| ||Shape of the tensor.|
| ||Optional dtype of the tensor. If not provided use the initializer dtype.|
| ||Optional information about the possible partitioning of a tensor.|
© 2020 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.