W3cubDocs

/TensorFlow Python

tf.contrib.kfac.fisher_factors.FullyConnectedKroneckerFactor

Class `FullyConnectedKroneckerFactor`

Inherits From: `InverseProvidingFactor`

Kronecker factor for the input or output side of a fully-connected layer.

Methods

`__init__`

```__init__(
tensors,
has_bias=False
)
```

Instantiate FullyConnectedKroneckerFactor.

Args:

• `tensors`: List of list of Tensors, each of shape [batch_size, n]. The Tensors are typically either a layer's inputs or its output's gradients. The first list index is source, the second is tower.
• `has_bias`: bool. If True, append '1' to each row.

`get_cov`

```get_cov()
```

Get full covariance matrix.

Returns:

Tensor of shape [n, n]. Represents all parameter-parameter correlations captured by this FisherFactor.

`get_cov_var`

```get_cov_var()
```

Get variable backing this FisherFactor.

May or may not be the same as self.get_cov()

Returns:

Variable of shape self._cov_shape.

`get_eigendecomp`

```get_eigendecomp()
```

Creates or retrieves eigendecomposition of self._cov.

`get_inverse`

```get_inverse(damping_func)
```

`get_matpower`

```get_matpower(
exp,
damping_func
)
```

`instantiate_cov_variables`

```instantiate_cov_variables()
```

Makes the internal cov variable(s).

`instantiate_inv_variables`

```instantiate_inv_variables()
```

Makes the internal "inverse" variable(s).

`left_multiply_matpower`

```left_multiply_matpower(
x,
exp,
damping_func
)
```

Left multiplies 'x' by matrix power of this factor (w/ damping applied).

This calculation is essentially: (C + damping * I)exp * x where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.

x can represent either a matrix or a vector. For some factors, 'x' might represent a vector but actually be stored as a 2D matrix for convenience.

Args:

• `x`: Tensor. Represents a single vector. Shape depends on implementation.
• `exp`: float. The matrix exponent to use.
• `damping_func`: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().

Returns:

Tensor of same shape as 'x' representing the result of the multiplication.

`make_covariance_update_op`

```make_covariance_update_op(ema_decay)
```

Constructs and returns the covariance update Op.

Args:

• `ema_decay`: The exponential moving average decay (float or Tensor).

Returns:

An Op for updating the covariance Variable referenced by _cov.

`make_inverse_update_ops`

```make_inverse_update_ops()
```

Create and return update ops corresponding to registered computations.

`register_inverse`

```register_inverse(damping_func)
```

`register_matpower`

```register_matpower(
exp,
damping_func
)
```

Registers a matrix power to be maintained and served on demand.

This creates a variable and signals make_inverse_update_ops to make the corresponding update op. The variable can be read via the method get_matpower.

Args:

• `exp`: float. The exponent to use in the matrix power.
• `damping_func`: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().

`right_multiply_matpower`

```right_multiply_matpower(
x,
exp,
damping_func
)
```

Right multiplies 'x' by matrix power of this factor (w/ damping applied).

This calculation is essentially: x * (C + damping * I)exp where * is matrix-multiplication, is matrix power, I is the identity matrix, and C is the matrix represented by this factor.

Unlike left_multiply_matpower, x will always be a matrix.

Args:

• `x`: Tensor. Represents a single vector. Shape depends on implementation.
• `exp`: float. The matrix exponent to use.
• `damping_func`: A function that computes a 0-D Tensor or a float which will be the damping value used. i.e. damping = damping_func().

Returns:

Tensor of same shape as 'x' representing the result of the multiplication.