This document describes how to run TensorFlow on S3 file system.
We assume that you are familiar with reading data.
To use S3 with TensorFlow, change the file paths you use to read and write data to an S3 path. For example:
filenames = ["s3://bucketname/path/to/file1.tfrecord", "s3://bucketname/path/to/file2.tfrecord"] dataset = tf.data.TFRecordDataset(filenames)
When reading or writing data on S3 with your TensorFlow program, the behavior could be controlled by various environmental variables:
AWS_REGIONis not specified, then
To read or write objects in a bucket that is no publicly accessible, AWS credentials must be provided through one of the following methods:
~/.aws/credentialson Linux, macOS, or Unix, or
© 2018 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.