DEPRECATED.
tf.keras.preprocessing.text.Tokenizer(
num_words=None,
filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
lower=True,
split=' ',
char_level=False,
oov_token=None,
analyzer=None,
**kwargs
)
| Used in the tutorials |
|---|
fit_on_sequencesfit_on_sequences(
sequences
)
fit_on_textsfit_on_texts(
texts
)
get_configget_config()
sequences_to_matrixsequences_to_matrix(
sequences, mode='binary'
)
sequences_to_textssequences_to_texts(
sequences
)
sequences_to_texts_generatorsequences_to_texts_generator(
sequences
)
texts_to_matrixtexts_to_matrix(
texts, mode='binary'
)
texts_to_sequencestexts_to_sequences(
texts
)
texts_to_sequences_generatortexts_to_sequences_generator(
texts
)
to_jsonto_json(
**kwargs
)
© 2022 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 4.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer