trx.fetcher

Test data management for downloading and verifying test assets.

Attributes

TEST_DATA_REPO
TEST_DATA_TAG
TEST_DATA_API_URL
TEST_DATA_BASE_URL

Functions

get_home()	Return a user-writeable file-system location to put files.
get_testing_files_dict()	Return dictionary linking zip file to their GitHub release URL and checksums.
md5sum(filename)	Compute the MD5 checksum of a file.
sha256sum(filename)	Compute the SHA256 checksum of a file.
fetch_data(files_dict[, keys])	Download files to folder and check their md5 checksums.

Module Contents

trx.fetcher.TEST_DATA_REPO = 'tee-ar-ex/trx-test-data'

trx.fetcher.TEST_DATA_TAG = 'v0.1.0'

trx.fetcher.TEST_DATA_API_URL = 'https://api.github.com/repos/tee-ar-ex/trx-test-data/releases/tags/v0.1.0'

trx.fetcher.TEST_DATA_BASE_URL = 'https://github.com/tee-ar-ex/trx-test-data/releases/download/v0.1.0'

trx.fetcher.get_home()

Return a user-writeable file-system location to put files.

Returns:

str

Path to the TRX home directory.

trx.fetcher.get_testing_files_dict()

Return dictionary linking zip file to their GitHub release URL and checksums.

Assets are hosted under the v0.1.0 release of tee-ar-ex/trx-test-data. If URLs change, check TEST_DATA_API_URL to discover the latest asset locations.

Returns:

dict

Mapping of filenames to (url, md5, sha256) tuples.

trx.fetcher.md5sum(filename)

Compute the MD5 checksum of a file.

Parameters:

filenamestr

Path to file to hash.

Returns:

str

Hexadecimal MD5 digest.

trx.fetcher.sha256sum(filename)

Compute the SHA256 checksum of a file.

Parameters:

filenamestr

Path to file to hash.

Returns:

str

Hexadecimal SHA256 digest.

trx.fetcher.fetch_data(files_dict, keys=None)

Download files to folder and check their md5 checksums.

Parameters:

files_dictdict

For each file in files_dict the value should be (url, md5). The file will be downloaded from url, if the file does not already exist or if the file exists but the md5 checksum does not match. Zip files are automatically unzipped and its contents are md5 checked.

keyslist of str or str or None, optional

Subset of keys from files_dict to download. When None, all keys are downloaded.

Raises:

ValueError

Raises if the md5 checksum of the file does not match the expected value. The downloaded file is not deleted when this error is raised.