Python (h5py)

h5s3 may be used from Python with h5py. By using h5py, you are able to pass S3 backed files to existing code programmed against the h5py interface.

Currently this requires installing h5py from github because the driver registration feature has not been put in a release yet.

Example

import h5s3
import h5py

# register our driver with h5py
h5s3.register()

f = h5py.File(
    's3://bucket/name.h5s3',
    'r+'
    driver='h5s3',
    aws_access_key='<your-access-key>',
    aws_secret_key='<your-secret-key>',
)

print(f)
# <HDF5 file "name.h5s3" (mode r+)>

# now use f like any other hdf5 file!

API

h5s3.register()

Register the h5s3 driver with h5py.

See also

h5s3.unregiser()

h5s3.unregiser()

Unregister the h5s3 driver with h5py.

See also

h5s3.regiser()

h5s3.set_fapl(plist, aws_access_key, aws_secret_key, aws_region='us-east-1', page_size=0, page_cache_size=0, host='s3.amazonaws.com', use_tls=True)

Set the fapl for the h5s3 driver.

Parameters:

plist : PropFAID

The property list.

aws_access_key : str

Your amazon access key id.

aws_secret_key : str

Your amazon secret key.

page_size : int, optional

The size of a data page.

page_cache_size : int, optional

The number of pages to cache in memory.

host : str, optional

The host for the aws API to ues.

use_tls : bool, optional

Connect to the aws API with TLS.

Notes

All arguments except for plist will be forwarded from h5py.File.