Datatrail Admin CLI Reference

This page provides documentation for the Datatrail Admin CLI.

datatrail-admin

Datatrail :: Command Line Interface.

Usage:

Text Only

datatrail-admin [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin action

Action tools.

Usage:

Text Only

datatrail-admin action [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin action clear

Clear all files from a dataset from a storage element.

Usage:

Text Only

datatrail-admin action clear [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the dataset to pull. This dataset must be registered into datatrail.	_required
--scope, -s	text	Scope of the dataset to pull. This dataset must be registered into datatrail.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin action delete

Delete a file from a storage element.

Usage:

Text Only

datatrail-admin action delete [OPTIONS]

Options:

Name	Type	Description	Default
--storage_element	text	Storage element to delete from.	_required
--path, -p	text	Location of the file to delete.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin action pull

Pull a file from a storage element.

Usage:

Text Only

datatrail-admin action pull [OPTIONS]

Options:

Name	Type	Description	Default
--storage_element	text	Storage element to pull from.	_required
--source, -s	text	Location of the file to pull.	_required
--destination, -d	text	Location to save the file to.	None
--help	boolean	Show this message and exit.	False

datatrail-admin action pull-dataset

Pull all files from a dataset from a storage element.

Usage:

Text Only

datatrail-admin action pull-dataset [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the dataset to pull. This dataset must be registered into datatrail.	_required
--scope, -s	text	Scope of the dataset to pull. This dataset must be registered into datatrail.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin action push

Push a file to a storage element.

Usage:

Text Only

datatrail-admin action push [OPTIONS]

Options:

Name	Type	Description	Default
--storage_element	text	Storage element to push to.	_required
--source, -s	text	Location of the file to push.	_required
--destination, -d	text	Location to save the file to.	None
--help	boolean	Show this message and exit.	False

datatrail-admin commit

Commit tools.

Usage:

Text Only

datatrail-admin commit [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin deletion

Deletion tools.

Usage:

Text Only

datatrail-admin deletion [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin deletion check-dataset-fully-deleted

Check if all files from dataset have been deleted

Usage:

Text Only

datatrail-admin deletion check-dataset-fully-deleted [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the dataset to pull. This dataset must be registered into datatrail. eg. event number	_required
--scope, -s	text	Scope of the dataset to pull. This dataset must be registered into datatrail. eg. chime.event.baseband.raw	_required
--storage-element, -e	text	Storage element from where the files were to be deleted. eg. chime/minoc.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin deletion deleter-status

Show deleter's currently queued and running files.

Usage:

Text Only

datatrail-admin deletion deleter-status [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin deletion reset-replica-state

Change the state from failed/staged to available.

Usage:

Text Only

datatrail-admin deletion reset-replica-state [OPTIONS]

Options:

Name	Type	Description	Default
--files, -f	text	Text file with list of files to reset replica state.	_required
--storage-name, -s	text	Name of the storage element file is replicated from.	_required
--replicate-to, -r	text	Name of the storage element file is replicated to.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin deletion show-bucket-datatrail-info

Display bucket and datatrail information for a file name.

Usage:

Text Only

datatrail-admin deletion show-bucket-datatrail-info [OPTIONS] FILE_NAME

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin deletion show-status

Show num and info for failed and completed file replicas.

Usage:

Text Only

datatrail-admin deletion show-status [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin deletion stage-healing

Create work in Buckets for deletion healers.

Usage:

Text Only

datatrail-admin deletion stage-healing [OPTIONS] NUM_DATASETS SITE

Options:

Name	Type	Description	Default
-t, --transformation-status	text	Transformation status: staged, failed, cancelled	staged
--help	boolean	Show this message and exit.	False

datatrail-admin deletion start

Start the automated deletion worker.

Usage:

Text Only

datatrail-admin deletion start [OPTIONS] {chime|kko|gbo|hco}

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin query

Query tools.

Usage:

Text Only

datatrail-admin query [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin query find

Find all files from a dataset.

Usage:

Text Only

datatrail-admin query find [OPTIONS]

Options:

Name	Type	Description	Default
--scope, -s	text	Scope of the dataset to find.	_required
--name, -n	text	Name of the dataset to find.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin query find-datasets-at-storage-element

Find all datasets from a storage element.

Usage:

Text Only

datatrail-admin query find-datasets-at-storage-element [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the storage element (eg. chime, minoc, arc).	_required
--help	boolean	Show this message and exit.	False

datatrail-admin query info

Find information about a file.

Usage:

Text Only

datatrail-admin query info [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the file.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin query ls

List the datasets or files in a dataset.

Usage:

Text Only

datatrail-admin query ls [OPTIONS] SCOPE [DATASET]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin queue

Queue tools.

Usage:

Text Only

datatrail-admin queue [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin queue transfer-processed-data-from-arc

Transfer processed files one at a time from arc.

Usage:

Text Only

datatrail-admin queue transfer-processed-data-from-arc [OPTIONS]

Options:

Name	Type	Description	Default
--source, -s	text	Location of the directory from which to transfer files.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin registration

Registration tools.

Usage:

Text Only

datatrail-admin registration [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin registration register-datasets

Register datasets into datatrail.

Usage:

Text Only

datatrail-admin registration register-datasets [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin registration register-events

Register events from the frb archivers.

Usage:

Text Only

datatrail-admin registration register-events [OPTIONS]

Options:

Name	Type	Description	Default
--date	text	Date in YYYY-mm-dd format to register the data for.	None
--run-until	text	Date in YYYY-mm-dd format to register the data for.	None
--dry-run	boolean	This will not actually register the events. Useful for testing.	False
--test-mode	boolean	Are you currently in dev mode?	False
--verbose, -v	boolean	Do you want the system to be verbose?	False
--help	boolean	Show this message and exit.	False

datatrail-admin registration register-failed-datasets

Retry registering failed datasets into datatrail.

Usage:

Text Only

datatrail-admin registration register-failed-datasets [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin registration show-last-event-date

Find date that the events have been registered until.

Usage:

Text Only

datatrail-admin registration show-last-event-date [OPTIONS]

Options:

Name	Type	Description	Default
--url	text	Server where pipeline results are stored.	https://frb.chimenet.ca/results/
--help	boolean	Show this message and exit.	False

datatrail-admin registration show-unregistered-datasets

Find all unregistered events.

Usage:

Text Only

datatrail-admin registration show-unregistered-datasets [OPTIONS]

Options:

Name	Type	Description	Default
--url	text	Server where pipeline results are stored.	https://frb.chimenet.ca/results/
--help	boolean	Show this message and exit.	False

datatrail-admin registration show-unregistered-events

Find all unregistered events.

Usage:

Text Only

datatrail-admin registration show-unregistered-events [OPTIONS]

Options:

Name	Type	Description	Default
--url	text	Server where pipeline results are stored.	https://frb.chimenet.ca/results/
--help	boolean	Show this message and exit.	False

datatrail-admin replication

Replication tools.

Usage:

Text Only

datatrail-admin replication [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin replication check-dataset-fully-replicated

Check if all files from dataset have been replicated

Usage:

Text Only

datatrail-admin replication check-dataset-fully-replicated 
    [OPTIONS]

Options:

Name	Type	Description	Default
--name, -n	text	Name of the dataset to pull. This dataset must be registered into datatrail. eg. event number	_required
--scope, -s	text	Scope of the dataset to pull. This dataset must be registered into datatrail. eg. chime.event.baseband.raw	_required
--storage-element, -e	text	Storage element from where the files were to be replicated. eg. chime/minoc.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin replication replicator-status

Show replicator's currently queued and running files.

Usage:

Text Only

datatrail-admin replication replicator-status [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin replication reset-replica-state

Change the state from failed/staged to available.

Usage:

Text Only

datatrail-admin replication reset-replica-state [OPTIONS]

Options:

Name	Type	Description	Default
--files, -f	text	Text file with list of files to reset replica state.	_required
--storage-name, -s	text	Name of the storage element file is replicated from.	_required
--replicate-to, -r	text	Name of the storage element file is replicated to.	_required
--help	boolean	Show this message and exit.	False

datatrail-admin replication show-bucket-datatrail-info

Display bucket and datatrail information for a file name.

Usage:

Text Only

datatrail-admin replication show-bucket-datatrail-info [OPTIONS] FILE_NAME

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin replication show-status

Show num and info for failed and completed file replicas.

Usage:

Text Only

datatrail-admin replication show-status [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin replication stage-healing

Create work in Buckets for replication healers.

Usage:

Text Only

datatrail-admin replication stage-healing [OPTIONS] NUM_REPLICAS SITE

Options:

Name	Type	Description	Default
-t, --transformation-status	text	Transformation status: staged, failed, cancelled	staged
--help	boolean	Show this message and exit.	False

datatrail-admin replication start

Start replicating events from {site} to minoc.

Usage:

Text Only

datatrail-admin replication start [OPTIONS] {chime|kko|gbo|hco}

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin utility

Utility tools.

Usage:

Text Only

datatrail-admin utility [OPTIONS] COMMAND [ARGS]...

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False

datatrail-admin utility check-service-status

Show status of services that datatrail client depends upon..

Usage:

Text Only

datatrail-admin utility check-service-status [OPTIONS]

Options:

Name	Type	Description	Default
--help	boolean	Show this message and exit.	False