W3cubDocs

/TensorFlow Python

tf.keras.preprocessing.text.text_to_word_sequence

tf.keras.preprocessing.text.text_to_word_sequence(
    text,
    filters='!"#$%&()*+,-./:;<=>[email protected][\\]^_`{|}~\t\n',
    lower=True,
    split=' '
)

Defined in tensorflow/python/keras/_impl/keras/preprocessing/text.py.

Converts a text to a sequence of words (or tokens).

Arguments:

  • text: Input text (string).
  • filters: Sequence of characters to filter out.
  • lower: Whether to convert the input to lowercase.
  • split: Sentence split marker (string).

Returns:

A list of words (or tokens).

© 2018 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/text_to_word_sequence