View source on GitHub |

Optimizer that implements the Adamax algorithm.

Inherits From: `Optimizer`

tf.keras.optimizers.Adamax( learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, name='Adamax', **kwargs )

It is a variant of Adam based on the infinity norm. Default parameters follow those provided in the paper. Adamax is sometimes superior to adam, specially in models with embeddings.

References see Section 7 of Kingma et al., 2014 (pdf).

Args | |
---|---|

`learning_rate` | A Tensor or a floating point value. The learning rate. |

`beta_1` | A float value or a constant float tensor. The exponential decay rate for the 1st moment estimates. |

`beta_2` | A float value or a constant float tensor. The exponential decay rate for the exponentially weighted infinity norm. |

`epsilon` | A small constant for numerical stability. |

`name` | Optional name for the operations created when applying gradients. Defaults to "Adamax". |

`**kwargs` | keyword arguments. Allowed to be {`clipnorm` , `clipvalue` , `lr` , `decay` }. `clipnorm` is clip gradients by norm; `clipvalue` is clip gradients by value, `decay` is included for backward compatibility to allow time inverse decay of learning rate. `lr` is included for backward compatibility, recommended to use `learning_rate` instead. |

Attributes | |
---|---|

`iterations` | Variable. The number of training steps this Optimizer has run. |

`weights` | Returns variables of this Optimizer based on the order created. |

`add_slot`

add_slot( var, slot_name, initializer='zeros' )

Add a new slot variable for `var`

.

`add_weight`

add_weight( name, shape, dtype=None, initializer='zeros', trainable=None, synchronization=tf.VariableSynchronization.AUTO, aggregation=tf.VariableAggregation.NONE )

`apply_gradients`

apply_gradients( grads_and_vars, name=None )

Apply gradients to variables.

This is the second part of `minimize()`

. It returns an `Operation`

that applies gradients.

Args | |
---|---|

`grads_and_vars` | List of (gradient, variable) pairs. |

`name` | Optional name for the returned operation. Default to the name passed to the `Optimizer` constructor. |

Returns | |
---|---|

An `Operation` that applies the specified gradients. The `iterations` will be automatically increased by 1. |

Raises | |
---|---|

`TypeError` | If `grads_and_vars` is malformed. |

`ValueError` | If none of the variables have gradients. |

`from_config`

@classmethod from_config( config, custom_objects=None )

Creates an optimizer from its config.

This method is the reverse of `get_config`

, capable of instantiating the same optimizer from the config dictionary.

Arguments | |
---|---|

`config` | A Python dictionary, typically the output of get_config. |

`custom_objects` | A Python dictionary mapping names to additional Python objects used to create this optimizer, such as a function used for a hyperparameter. |

Returns | |
---|---|

An optimizer instance. |

`get_config`

get_config()

Returns the config of the optimimizer.

An optimizer config is a Python dictionary (serializable) containing the configuration of an optimizer. The same optimizer can be reinstantiated later (without any saved state) from this configuration.

Returns | |
---|---|

Python dictionary. |

`get_gradients`

get_gradients( loss, params )

Returns gradients of `loss`

with respect to `params`

.

Arguments | |
---|---|

`loss` | Loss tensor. |

`params` | List of variables. |

Returns | |
---|---|

List of gradient tensors. |

Raises | |
---|---|

`ValueError` | In case any gradient cannot be computed (e.g. if gradient function not implemented). |

`get_slot`

get_slot( var, slot_name )

`get_slot_names`

get_slot_names()

A list of names for this optimizer's slots.

`get_updates`

get_updates( loss, params )

`get_weights`

get_weights()

`minimize`

minimize( loss, var_list, grad_loss=None, name=None )

Minimize `loss`

by updating `var_list`

.

This method simply computes gradient using `tf.GradientTape`

and calls `apply_gradients()`

. If you want to process the gradient before applying then call `tf.GradientTape`

and `apply_gradients()`

explicitly instead of using this function.

Args | |
---|---|

`loss` | A callable taking no arguments which returns the value to minimize. |

`var_list` | list or tuple of `Variable` objects to update to minimize `loss` , or a callable returning the list or tuple of `Variable` objects. Use callable when the variable list would otherwise be incomplete before `minimize` since the variables are created at the first time `loss` is called. |

`grad_loss` | Optional. A `Tensor` holding the gradient computed for `loss` . |

`name` | Optional name for the returned operation. |

Returns | |
---|---|

An Operation that updates the variables in `var_list` . If `global_step` was not `None` , that operation also increments `global_step` . |

Raises | |
---|---|

`ValueError` | If some of the variables are not `Variable` objects. |

`set_weights`

set_weights( weights )

`variables`

variables()

Returns variables of this Optimizer based on the order created.

© 2020 The TensorFlow Authors. All rights reserved.

Licensed under the Creative Commons Attribution License 3.0.

Code samples licensed under the Apache 2.0 License.

https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/optimizers/Adamax