tf.raw_ops.ExperimentalAutoShardDataset

Creates a dataset that shards the input dataset.

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.raw_ops.ExperimentalAutoShardDataset

tf.raw_ops.ExperimentalAutoShardDataset(
    input_dataset, num_workers, index, output_types, output_shapes,
    auto_shard_policy=0, name=None
)

Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).

This dataset will throw a NotFound error if we cannot shard the dataset automatically.

Args
input_dataset	A Tensor of type variant. A variant tensor representing the input dataset.
num_workers	A Tensor of type int64. A scalar representing the number of workers to distribute this dataset across.
index	A Tensor of type int64. A scalar representing the index of the current worker out of num_workers.
output_types	A list of tf.DTypes that has length >= 1.
output_shapes	A list of shapes (each a tf.TensorShape or list of ints) that has length >= 1.
auto_shard_policy	An optional int. Defaults to 0.
name	A name for the operation (optional).

Returns
A Tensor of type variant.