ConvDiagonalFactor
Inherits From: DiagonalFactor
Defined in tensorflow/contrib/kfac/python/ops/fisher_factors.py
.
FisherFactor for a diagonal approx of a convolutional layer's Fisher.
name
__init__
__init__( inputs, outputs_grads, filter_shape, strides, padding, data_format=None, dilations=None, has_bias=False )
Creates a ConvDiagonalFactor object.
inputs
: List of Tensors of shape [batch_size, height, width, in_channels]. Input activations to this layer. List index is towers.outputs_grads
: List of Tensors, each of shape [batch_size, height, width, out_channels], which are the gradients of the loss with respect to the layer's outputs. First index is source, second index is tower.filter_shape
: Tuple of 4 ints: (kernel_height, kernel_width, in_channels, out_channels). Represents shape of kernel used in this layer.strides
: The stride size in this layer (1-D Tensor of length 4).padding
: The padding in this layer (1-D of Tensor length 4).data_format
: None or str. Format of conv2d inputs.dilations
: None or tuple of 4 ints.has_bias
: Python bool. If True, the layer is assumed to have a bias parameter in addition to its filter parameter.ValueError
: If inputs, output_grads, and filter_shape do not agree on in_channels or out_channels.ValueError
: If strides, dilations are not length-4 lists of ints.ValueError
: If data_format does not put channel last.get_cov
get_cov()
Get full covariance matrix.
Tensor of shape [n, n]. Represents all parameter-parameter correlations captured by this FisherFactor.
get_cov_var
get_cov_var()
Get variable backing this FisherFactor.
May or may not be the same as self.get_cov()
Variable of shape self._cov_shape.
instantiate_cov_variables
instantiate_cov_variables()
Makes the internal cov variable(s).
instantiate_inv_variables
instantiate_inv_variables()
Makes the internal "inverse" variable(s).
left_multiply_matpower
left_multiply_matpower( x, exp, damping_func )
Left multiplies 'x' by matrix power of this factor (w/ damping applied).
This calculation is essentially: (C + damping * I)exp * x where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.
x can represent either a matrix or a vector. For some factors, 'x' might represent a vector but actually be stored as a 2D matrix for convenience.
x
: Tensor. Represents a single vector. Shape depends on implementation.exp
: float. The matrix exponent to use.damping_func
: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().Tensor of same shape as 'x' representing the result of the multiplication.
make_covariance_update_op
make_covariance_update_op(ema_decay)
Constructs and returns the covariance update Op.
ema_decay
: The exponential moving average decay (float or Tensor).An Op for updating the covariance Variable referenced by _cov.
make_inverse_update_ops
make_inverse_update_ops()
Create and return update ops corresponding to registered computations.
register_matpower
register_matpower( exp, damping_func )
right_multiply_matpower
right_multiply_matpower( x, exp, damping_func )
Right multiplies 'x' by matrix power of this factor (w/ damping applied).
This calculation is essentially: x * (C + damping * I)exp where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.
Unlike left_multiply_matpower, x will always be a matrix.
x
: Tensor. Represents a single vector. Shape depends on implementation.exp
: float. The matrix exponent to use.damping_func
: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().Tensor of same shape as 'x' representing the result of the multiplication.
© 2018 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/contrib/kfac/fisher_factors/ConvDiagonalFactor