View source on GitHub |

Computes Kullback-Leibler divergence loss between `y_true`

and `y_pred`

.

tf.keras.losses.KLDivergence( reduction=losses_utils.ReductionV2.AUTO, name='kl_divergence' )

`loss = y_true * log(y_true / y_pred)`

See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

y_true = [[0, 1], [0, 0]] y_pred = [[0.6, 0.4], [0.4, 0.6]] # Using 'auto'/'sum_over_batch_size' reduction type. kl = tf.keras.losses.KLDivergence() kl(y_true, y_pred).numpy() 0.458

# Calling with 'sample_weight'. kl(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy() 0.366

# Using 'sum' reduction type. kl = tf.keras.losses.KLDivergence( reduction=tf.keras.losses.Reduction.SUM) kl(y_true, y_pred).numpy() 0.916

# Using 'none' reduction type. kl = tf.keras.losses.KLDivergence( reduction=tf.keras.losses.Reduction.NONE) kl(y_true, y_pred).numpy() array([0.916, -3.08e-06], dtype=float32)

Usage with the `compile()`

API:

model.compile(optimizer='sgd', loss=tf.keras.losses.KLDivergence())

Args | |
---|---|

`reduction` | (Optional) Type of `tf.keras.losses.Reduction` to apply to loss. Default value is `AUTO` . `AUTO` indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE` . When used with `tf.distribute.Strategy` , outside of built-in training loops such as `tf.keras` `compile` and `fit` , using `AUTO` or `SUM_OVER_BATCH_SIZE` will raise an error. Please see this custom training tutorial for more details. |

`name` | Optional name for the op. Defaults to 'kl_divergence'. |

`from_config`

@classmethod from_config( config )

Instantiates a `Loss`

from its config (output of `get_config()`

).

Args | |
---|---|

`config` | Output of `get_config()` . |

Returns | |
---|---|

A `Loss` instance. |

`get_config`

get_config()

Returns the config dictionary for a `Loss`

instance.

`__call__`

__call__( y_true, y_pred, sample_weight=None )

Invokes the `Loss`

instance.

Args | |
---|---|

`y_true` | Ground truth values. shape = `[batch_size, d0, .. dN]` , except sparse loss functions such as sparse categorical crossentropy where shape = `[batch_size, d0, .. dN-1]` |

`y_pred` | The predicted values. shape = `[batch_size, d0, .. dN]` |

`sample_weight` | Optional `sample_weight` acts as a coefficient for the loss. If a scalar is provided, then the loss is simply scaled by the given value. If `sample_weight` is a tensor of size `[batch_size]` , then the total loss for each sample of the batch is rescaled by the corresponding element in the `sample_weight` vector. If the shape of `sample_weight` is `[batch_size, d0, .. dN-1]` (or can be broadcasted to this shape), then each loss element of `y_pred` is scaled by the corresponding value of `sample_weight` . (Note on`dN-1` : all loss functions reduce by 1 dimension, usually axis=-1.) |

Returns | |
---|---|

Weighted loss float `Tensor` . If `reduction` is `NONE` , this has shape `[batch_size, d0, .. dN-1]` ; otherwise, it is scalar. (Note `dN-1` because all loss functions reduce by 1 dimension, usually axis=-1.) |

Raises | |
---|---|

`ValueError` | If the shape of `sample_weight` is invalid. |

© 2020 The TensorFlow Authors. All rights reserved.

Licensed under the Creative Commons Attribution License 3.0.

Code samples licensed under the Apache 2.0 License.

https://www.tensorflow.org/versions/r2.3/api_docs/python/tf/keras/losses/KLDivergence