tf.keras.utils.SequenceEnqueuer

View source on GitHub

Base class to enqueue inputs.

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.keras.utils.SequenceEnqueuer

tf.keras.utils.SequenceEnqueuer(
    sequence, use_multiprocessing=False
)

The task of an Enqueuer is to use parallelism to speed up preprocessing. This is done with processes or threads.

Example:

enqueuer = SequenceEnqueuer(...)
enqueuer.start()
datas = enqueuer.get()
for data in datas:
    # Use the inputs; training, evaluating, predicting.
    # ... stop sometime.
enqueuer.stop()

The enqueuer.get() should be an infinite stream of data.

Methods

get

View source

get()

Creates a generator to extract data from the queue.

Skip the data if it is None. Returns: Generator yielding tuples (inputs, targets) or (inputs, targets, sample_weights).

is_running

View source

is_running()

start

View source

start(
    workers=1, max_queue_size=10
)

Starts the handler's workers.

Args
workers	Number of workers.
max_queue_size	queue size (when full, workers could block on put())

stop

View source

stop(
    timeout=None
)

Stops running threads and wait for them to exit, if necessary.

Should be called by the same thread which called start().

Args
timeout	maximum time to wait on thread.join()