Google Cloud Storage Import

Prerequisites

The Google Cloud Storage (GCS) can be used via the httpfs extension. This can be installed with the INSTALL httpfs SQL command. This only needs to be run once.

Credentials and Configuration

You need to create HMAC keys and declare them:

CREATE SECRET (
    TYPE GCS,
    KEY_ID 'AKIAIOSFODNN7EXAMPLE',
    SECRET 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'
);

Querying

After setting up the GCS credentials, you can query the GCS data using:

SELECT *
FROM read_parquet('gs://⟨gcs_bucket⟩/⟨file.parquet⟩');

Attaching to a Database

You can attach to a database file in read-only mode:

LOAD httpfs;
ATTACH 'gs://⟨gcs_bucket⟩/⟨file.duckdb⟩' AS ⟨duckdb_database⟩ (READ_ONLY);

Databases in Google Cloud Storage can only be attached in read-only mode.