FullyConnectedKroneckerFactor
Inherits From: InverseProvidingFactor
Defined in tensorflow/contrib/kfac/python/ops/fisher_factors.py
.
Kronecker factor for the input or output side of a fully-connected layer.
name
__init__
__init__( tensors, has_bias=False )
Instantiate FullyConnectedKroneckerFactor.
tensors
: List of list of Tensors, each of shape [batch_size, n]. The Tensors are typically either a layer's inputs or its output's gradients. The first list index is source, the second is tower.has_bias
: bool. If True, append '1' to each row.get_cov
get_cov()
Get full covariance matrix.
Tensor of shape [n, n]. Represents all parameter-parameter correlations captured by this FisherFactor.
get_cov_var
get_cov_var()
Get variable backing this FisherFactor.
May or may not be the same as self.get_cov()
Variable of shape self._cov_shape.
get_eigendecomp
get_eigendecomp()
Creates or retrieves eigendecomposition of self._cov.
get_inverse
get_inverse(damping_func)
get_matpower
get_matpower( exp, damping_func )
instantiate_cov_variables
instantiate_cov_variables()
Makes the internal cov variable(s).
instantiate_inv_variables
instantiate_inv_variables()
Makes the internal "inverse" variable(s).
left_multiply_matpower
left_multiply_matpower( x, exp, damping_func )
Left multiplies 'x' by matrix power of this factor (w/ damping applied).
This calculation is essentially: (C + damping * I)exp * x where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.
x can represent either a matrix or a vector. For some factors, 'x' might represent a vector but actually be stored as a 2D matrix for convenience.
x
: Tensor. Represents a single vector. Shape depends on implementation.exp
: float. The matrix exponent to use.damping_func
: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().Tensor of same shape as 'x' representing the result of the multiplication.
make_covariance_update_op
make_covariance_update_op(ema_decay)
Constructs and returns the covariance update Op.
ema_decay
: The exponential moving average decay (float or Tensor).An Op for updating the covariance Variable referenced by _cov.
make_inverse_update_ops
make_inverse_update_ops()
Create and return update ops corresponding to registered computations.
register_inverse
register_inverse(damping_func)
register_matpower
register_matpower( exp, damping_func )
Registers a matrix power to be maintained and served on demand.
This creates a variable and signals make_inverse_update_ops to make the corresponding update op. The variable can be read via the method get_matpower.
exp
: float. The exponent to use in the matrix power.damping_func
: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().right_multiply_matpower
right_multiply_matpower( x, exp, damping_func )
Right multiplies 'x' by matrix power of this factor (w/ damping applied).
This calculation is essentially: x * (C + damping * I)exp where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.
Unlike left_multiply_matpower, x will always be a matrix.
x
: Tensor. Represents a single vector. Shape depends on implementation.exp
: float. The matrix exponent to use.damping_func
: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().Tensor of same shape as 'x' representing the result of the multiplication.
© 2018 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/contrib/kfac/fisher_factors/FullyConnectedKroneckerFactor