query

Implement the following REST endpoints.

1. Create a storage element.
2. Fetch storage elements.
3. Toggle active state of storage element.

check_dataset_fully_transformed async

Python

check_dataset_fully_transformed(session, name, scope, storage_name, transformation, test_mode=False)

Check if dataset has been fully transformed.

check_replication

Python

check_replication(item)

Check if item needs to be replicated.

compare_checksum

Python

compare_checksum(file1, file2)

Compare checksum of two files.

dataset_belongs_to

Python

dataset_belongs_to(event_number, data_type)

Query the L4 action picker database to get the dataset name.

dataset_scout

Python

dataset_scout(session, dataset_name, dataset_scope)

Return the number of file replicas for each storage element.

Parameters:

Name	Type	Description	Default
session		Postgres session.	required
dataset_name	str	Name of dataset.	required
dataset_scope	str	Name of dataset scope.	required

Returns:

Name	Type	Description
count	Dict	Number of file replicas for each storage element.

file_info async

Python

file_info(session, payload, test_mode=False)

Get information about a file.

find_dataset async

Python

find_dataset(session, payload, test_mode=False)

Find location of the dataset.

find_dataset_by_id async

Python

find_dataset_by_id(session, id, test_mode=False)

Find dataset by id.

find_datasets_at_storage_element async

Python

find_datasets_at_storage_element(session, payload, test_mode=False)

Find all datasets at a storage element.

find_file_by_id async

Python

find_file_by_id(session, id)

Find file by id.

Parameters:

Name	Type	Description	Default
session	Union[AsyncSession, sessionmaker]	Postgres session.	required
id	int	File's id number.	required

Returns:

Name	Type	Description
file	Dict	Dictionary containing file's info.

find_num_replicas_by_deletion_status async

Python

find_num_replicas_by_deletion_status(session, status, duration, test_mode=False)

Find number of file replicas matching deletion status.

find_num_replicas_by_replication_status async

Python

find_num_replicas_by_replication_status(session, status, duration, test_mode=False)

Find number of file replicas matching replication status.

find_replica_by_id async

Python

find_replica_by_id(session, id)

Find file replica by id.

Parameters:

Name	Type	Description	Default
session	Union[AsyncSession, sessionmaker]	Postgres session.	required
id	int	File replica's id number.	required

Returns:

Name	Type	Description
replica	Dict	Dictionary containing replica's info.

find_replica_info_by_deletion_status async

Python

find_replica_info_by_deletion_status(session, status, duration, limit=100, test_mode=False)

Find number of file replicas matching deletion status.

find_replica_info_by_replication_status async

Python

find_replica_info_by_replication_status(session, status, duration, limit=100, test_mode=False)

Find number of file replicas matching replication status.

get_dataset async

Python

get_dataset(session, scope, name, test_mode=False)

Get a specific dataset.

get_dataset_by_file_replica_deletion_status async

Python

get_dataset_by_file_replica_deletion_status(session, status, storage_name, num_datasets=10)

Find datasets by file replica deletion status.

Parameters:

Name	Type	Description	Default
session	Union[AsyncSession, sessionmaker]	Postgres session.	required
status	str	Deletion status.	required
storage_name	str	Storage name.	required
num_datasets	int	Number of datasets.	10

Returns:

Name	Type	Description
datasets	List[Dataset]	List of datasets.

get_dataset_children async

Python

get_dataset_children(session, scope, name, test_mode=False)

Get dataset's children.

get_dataset_deletion_date async

Python

get_dataset_deletion_date(session, dataset)

List deletion date and state for dataset in all scopes dataset exists in.

Parameters:

Name	Type	Description	Default
session	Union[AsyncSession, sessionmaker]	Postgres session.	required
dataset	str	Name of dataset.	required

Returns:

Name	Type	Description
deletion_info	Dict[str, Dict[str, Any]]	Dictionary of scope containing storage elements contain deletion date and state.

get_dataset_scopes async

Python

get_dataset_scopes(session, dataset_name='', test_mode=False)

Get all scopes.

get_datasets async

Python

get_datasets(session, scope, test_mode=False)

Get all datasets for a scope.

get_datasets_files async

Python

get_datasets_files(session, payload)

Get all datasets and files.

get_file_replicas_by_deletion_status async

Python

get_file_replicas_by_deletion_status(session, status, storage_name, num_replicas=1000)

Find file replicas by deletion status.

get_file_replicas_by_replication_status async

Python

get_file_replicas_by_replication_status(session, status, storage_name, replicate_to, num_replicas=1000)

Find file replicas by replication status.

Useful for finding replicas that are at replicate_to, but do not have an entry in the Datatrail database.

get_files async

Python

get_files(session, scope, name, test_mode=False)

Fetch files from a dataset.

get_larger_datasets async

Python

get_larger_datasets(session, scope, test_mode=False)

Get the larger datasets for a given scope.

Parameters:

Name	Type	Description	Default
session	Union[AsyncSession, sessionmaker]	Postgres session.	required
scope	str	Name of scope.	required
test_mode	bool	Activate test mode.	False

get_state_updated_after

Python

get_state_updated_after(duration)

Get state updated after date.

get_storage_elements async

Python

get_storage_elements(session)

Get all storage elements.

query_tsar_verification

Python

query_tsar_verification(event_number)

Fetch tsar verification from frb-master's verifications.