Datatrail Admin CLI Reference¶
This page provides documentation for the Datatrail Admin CLI.
datatrail-admin¶
Datatrail :: Command Line Interface.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin action¶
Action tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin action pull-dataset¶
Pull all files from a dataset from a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the dataset to pull. This dataset must be registered into datatrail. | _required |
--scope, -s |
text | Scope of the dataset to pull. This dataset must be registered into datatrail. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin action pull¶
Pull a file from a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--storage_element |
text | Storage element to pull from. | _required |
--source, -s |
text | Location of the file to pull. | _required |
--destination, -d |
text | Location to save the file to. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin action push¶
Push a file to a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--storage_element |
text | Storage element to push to. | _required |
--source, -s |
text | Location of the file to push. | _required |
--destination, -d |
text | Location to save the file to. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin action delete¶
Delete a file from a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--storage_element |
text | Storage element to delete from. | _required |
--path, -p |
text | Location of the file to delete. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin action clear¶
Clear all files from a dataset from a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the dataset to pull. This dataset must be registered into datatrail. | _required |
--scope, -s |
text | Scope of the dataset to pull. This dataset must be registered into datatrail. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin commit¶
Commit tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion¶
Deletion tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion start¶
Start the automated deletion worker.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
site |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion show-status¶
Show num and info for failed and completed file replicas.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion show-bucket-datatrail-info¶
Display bucket and datatrail information for a file name.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
file_name |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion deleter-status¶
Show deleter's currently queued and running files.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion check-dataset-fully-deleted¶
Check if all files from dataset have been deleted
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the dataset to pull. This dataset must be registered into datatrail. eg. event number | _required |
--scope, -s |
text | Scope of the dataset to pull. This dataset must be registered into datatrail. eg. chime.event.baseband.raw | _required |
--storage-element, -e |
text | Storage element from where the files were to be deleted. eg. chime/minoc. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion reset-replica-state¶
Change the state from failed/staged to available.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--files, -f |
text | Text file with list of files to reset replica state. | _required |
--storage-name, -s |
text | Name of the storage element file is replicated from. | _required |
--replicate-to, -r |
text | Name of the storage element file is replicated to. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin deletion stage-healing¶
Create work in Buckets for deletion healers.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
-t, --transformation-status |
text | Transformation status: staged, failed, cancelled | staged |
num_datasets |
text | _required | |
site |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin query¶
Query tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin query info¶
Find information about a file.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the file. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin query ls¶
List the datasets or files in a dataset.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
scope |
text | _required | |
dataset |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin query find¶
Find all files from a dataset.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--scope, -s |
text | Scope of the dataset to find. | _required |
--name, -n |
text | Name of the dataset to find. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin query find-datasets-at-storage-element¶
Find all datasets from a storage element.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the storage element (eg. chime, minoc, arc). | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin queue¶
Queue tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin queue transfer-processed-data-from-arc¶
Transfer processed files one at a time from arc.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--source, -s |
text | Location of the directory from which to transfer files. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration¶
Registration tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration register-events¶
Register events from the frb archivers.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--date |
text | Date in YYYY-mm-dd format to register the data for. | _required |
--run-until |
text | Date in YYYY-mm-dd format to register the data for. | _required |
--dry-run |
text | This will not actually register the events. Useful for testing. | False |
--test-mode |
text | Are you currently in dev mode? | False |
--verbose, -v |
text | Do you want the system to be verbose? | False |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration register-datasets¶
Register datasets into datatrail.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration register-failed-datasets¶
Retry registering failed datasets into datatrail.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration show-unregistered-events¶
Find all unregistered events.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--url |
text | Server where pipeline results are stored. | https://frb.chimenet.ca/results/ |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration show-unregistered-datasets¶
Find all unregistered events.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--url |
text | Server where pipeline results are stored. | https://frb.chimenet.ca/results/ |
--query, -q |
text | Query string to filter the datasets, e.g. --query site chime. | _required |
--limit |
text | Limit the number of datasets to show. | 1000 |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration show-last-event-date¶
Find date that the events have been registered until.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--url |
text | Server where pipeline results are stored. | https://frb.chimenet.ca/results/ |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration start¶
Start the registration of baseband and n_squared data.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
site |
text | _required | |
storage_element |
text | _required | |
datatypes |
text | _required | |
--run_until |
text | Date in YYYY-mm-dd format to register data until. | _required |
--span |
text | Number of days to reattempt. | 30 |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration single-dataset¶
Register a single dataset.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
storage_element |
text | _required | |
datatype |
text | _required | |
--name |
text | Name of the dataset to be registered, will be used to search for path. | _required |
--year |
text | Year of the dataset to be registered. | _required |
--month |
text | Month of the dataset to be registered. | _required |
--day |
text | Day of the dataset to be registered. | _required |
--path |
text | Path of the dataset to be registered. | _required |
--yes, -y |
text | Do not ask for confirmation. | False |
--help |
boolean | Show this message and exit. | False |
datatrail-admin registration single-date¶
Register all events for a single date.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
storage_element |
text | _required | |
datatype |
text | _required | |
date |
text | _required | |
--yes, -y |
text | Do not ask for confirmation. | False |
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication¶
Replication tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication start¶
Start replicating events from {SE} to minoc.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
replicate_from |
text | _required | |
replicate_to |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication stage-healing¶
Create work in Buckets for replication healers.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
-t, --transformation-status |
text | Transformation status: staged, failed, cancelled | staged |
num_replicas |
text | _required | |
storage_element |
text | _required | |
site |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication replicator-status¶
Show replicator's currently queued and running files.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication show-status¶
Show num and info for failed and completed file replicas.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication check-dataset-fully-replicated¶
Check if all files from dataset have been replicated
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--name, -n |
text | Name of the dataset to pull. This dataset must be registered into datatrail. eg. event number | _required |
--scope, -s |
text | Scope of the dataset to pull. This dataset must be registered into datatrail. eg. chime.event.baseband.raw | _required |
--storage-element, -e |
text | Storage element from where the files were to be replicated. eg. chime/minoc. | _required |
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication show-bucket-datatrail-info¶
Display bucket and datatrail information for a file name.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
file_name |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin replication reset-replica-state¶
Change the state from failed/staged to available.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--files, -f |
text | Text file with list of files to reset replica state. | _required |
--storage-name, -s |
text | Name of the storage element file is replicated from. | _required |
--replicate-to, -r |
text | Name of the storage element file is replicated to. | _required |
site |
text | _required | |
--help |
boolean | Show this message and exit. | False |
datatrail-admin utility¶
Utility tools.
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |
datatrail-admin utility check-service-status¶
Show status of services that datatrail client depends upon..
Usage:
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help |
boolean | Show this message and exit. | False |