Skip to content

A Telescope-Agnostic FRB Virtual Observatory Event (VOEvent) Service

FRB VOE is a telescope-agnostic server for publishing, broadcasting, and recording Virtual Observatory Events (VOEvents) for detections of Fast Radio Bursts (FRBs). The code base consists of a backend server that is managed by the maestro package, and a suite of command line interface (CLI) tools that allow a client to interact with the server.

Telescopes and observatories that are actively detecting or following up FRBs in the radio regime can establish a subscription-based FRB VOEvent service using frb-voe. The core functionality of the code base is the following tasks:

  • Start and stop a Comet VOEvent broker.
  • Publish VOEvents that follow an extension to the FRB VOEvent Standard originally prescribed in Petroff et al. 2017 and broadcast them using the Comet broker.
  • Maintain a database of subscribers that can receive the VOEvent broadcast from the Comet broker.
  • Submit, search for, and update existing FRB objects on the Transient Name Server (TNS).

Overview

There are currently two divergent branches: mcgill and master. The mcgill branch is a legacy code base designed to run on the chimefrb node on the calys network at McGill University. The master branch is the precursor to a forthcoming code package that will be made accessible to the public as a telescope-agnostic VOEvent publication server.

CHIME/FRB VOEvent McGill Server

Starting the CHIME/FRB VOEvent McGill Server

Login to the calys network, part of the Physics Department network at McGill University.

Text Only
$ ssh calys.physics.mcgill.ca
Login as user chimefrb to the chimefrb node on this network.
Text Only
$ ssh chimefrb@chimefrb
The FRB VOEvent server is configured as two systemd services, each with their own .service file. These are user specific but can only be modified by the System Administrator in the McGill Physics Department. Start the services one-by-one in the following order:
Text Only
$ systemctl --user start frb-voe-app.service
$ systemctl --user start comet.service
Confirm healthy start up by viewing the logs with either systemctl or journalctl. Examples:
Text Only
$ systemctl --user status -l frb-voe-app.service
$ journalctl -xe

Stopping the CHIME/FRB VOEvent McGill Server

In case you need to stop the server, for instance to prevent broadcasting VOEvents during a storm of false positives from the CHIME/FRB real-time pipeline, follow these instructions. First, login to the node as described in the previous section. Then stop the services one-by-one in the following order:

Text Only
$ systemctl --user stop frb-voe-app.service
$ systemctl --user stop comet.service
Confirm healthy shutdown by viewing the logs, as in the previous section.

Updating the CHIME/FRB VOEvent Allow-list

Susbcriptions are acquired through the webform on the public webpage. They are added to the Google Cloud database automatically, but need to be fetched from there routinely so that the allow-list for the Comet broker is updated. Currently this requires a hard restart of the McGill server.

First, login to the node as described previously. Then activate the frb_voe environment:

Text Only
$ cd /venvs/frb_voe
$ source ./bin/activate
Next, move to the folder containing the code base.
Text Only
$ cd utilities/
N.B. This is where the GitHub repository has been cloned into. Run the script to fetch all data from the Google Cloud DB.
Text Only
$ python3 get_all_cloud_data.py
N.B. You must use python3 since the chimefrb node has been configured with multiple versions of python installed on it. You will be prompted to enter a date to collect subscriptions since. Select a date in the past, generally equal to the day after the last day that this process was performed. When in doubt, use October 1, 2021, since before this date the VOEvent Service was not public.
Text Only
Fetch Google Cloud entries since (YYYY-MM-DD): 2022-05-01
This will create three .csv files locally that you can look at. You only need to update the Google Group that represents the mailing list for email version of the VOEvents. This process is manual currently but could be automated in the future using Google's API for Groups. Nonetheless, read the contents of cloud_db_mailing_list.csv. Select any emails under Susbcriber Email that have not already been made members of the Google Group. Use the Google Group GUI to add these email addresses as members.

Finally, you need to stop and then start (effectively, restart) both services. Consult the directions in previous sections.

Plotting Published CHIME/FRB VOEvent Statistics

Use the IPython notebook in utilities/chimefrb_voevent_service_data_visualization.ipynb to create plots of published CHIME/FRB VOEvents from a select date range.

TNS Service

Registering on the TNS

The TNS is the official IAU-recognized distributor and maintainer of the naming scheme for FRBs, as previously announced in the FRB 2020 virtual conference (watch the first presentation in Session 11 from the conference here).

All programmatic requests to the TNS require authentication. This in turn requires registration on the TNS web portal. To complete the registration process and obtain required authorization, follow these steps:

  1. Request a User Account here. If you had additional team members who will manage your TNS data, they should also create a User Account.
  2. Log in here.
  3. Edit your TNS Group members here.
  4. Add a TNS bot here. This is required in order to obtain the credentials that are needed for making programmatic requests to the TNS via the API that is embedded in frb-voe.
  5. Once your TNS bot has been created, it will appear in the table on this page. In the rightmost column you can click edit to view or change its properties. In particular, the Edit Bot page allows one to create a new API key for the bot at any time.
  6. From the Edit Bot page, one can obtain values for the environment variables that are required to use the TNS functionality of frb-voe.
    • TNS_API_KEY from the API alphanumeric key value.
    • TNS_API_ID from the "tns_id" key in the User-Agent specification.
    • TNS_API_BOT_NAME from the "name" key in the User-Agent specification.
  7. set the following environment variables using the values that were provided to you or your organization when registering for usage of the TNS.
    Text Only
    export TNS_API_KEY=""       # Alphanumeric API key for your TNS bot
export TNS_API_TNS_ID=""    # ID number for your TNS bot
export TNS_API_BOT_NAME=""  # Name of your TNS bot

Dependencies Installation

The following are instructions for installing the frb-voe package and running the TNS service.

First, cd into your preferred path /my/path and clone the repository.

Text Only
git clone git@github.com:CHIMEFRB/frb-voe.git

Now, install poetry with

Text Only
pip install poetry

Then install the requirements using poetry by running the following in the top-level folder where the poetry.lock file is found.

Text Only
poetry install

MongoDB Setup

Docker is a reliable way to set up a containerized MongoDB server. Follow directions to install Docker Desktop on your machine. A MongoDB server is required to host the databases used by frb-voe and this can be set up to run in a docker container on a standard port.

Launch the frb-voe backend

In your current session, launch the server by moving into the top-level directory first and then running maestro. Assume that you've cloned the repo into ~.

Text Only
cd ~/frb-voe
poetry run maestro start

This will run the server in the current terminal session in a non-daemonized mode. Namely, if the session is quitted, the server will die. The following output indicates normal start-up:

Text Only
Maestro Backend Startup

 ↪ config → ~/frb-voe/backend.yml
 ↪ debug  → False
 ↪ mode   → development

Confirm Backend Startup → [y/N]:

Select y at the prompt. The server will startup and initialize based on its configuration. You should confirm that your environment variables were loaded properly by looking for the following in the output:

Text Only
[2022-06-22 11:10:25,377] INFO tns::set_api_key():l129: Setting TNS API key
[2022-06-22 11:10:25,377] INFO tns::setup():l76: TNS API key was set to None
[2022-06-22 11:10:25,377] INFO tns::set_tns_id():l149: Setting TNS bot ID
[2022-06-22 11:10:25,377] INFO tns::setup():l80: TNS API bot ID was set to None
[2022-06-22 11:10:25,377] INFO tns::set_bot_name():l169: Setting TNS bot name
[2022-06-22 11:10:25,377] INFO tns::setup():l84: TNS API bot name was set to None
[2022-06-22 11:10:25,377] INFO tns::setup():l87: TNS marker was set to: tns_marker{"tns_id": "None", "type": "bot", "name": "None"}

If you see None values, your environment variables have not been made accesible. Confirm the process outlined in the previous subsection.

The server is now running on localhost at port 4357.

Stop the frb-voe backend

The server can be stopped by issuing CTRL+C in the terminal where it is running. The following indicates a successful shutdown:

Text Only
[2022-06-22 11:21:12 -0400] [60857] [INFO] Stopping worker [60857]
[2022-06-22 11:21:12,031] INFO server::serve():l914: Stopping worker [60857]
[2022-06-22 11:21:12 -0400] [60857] [INFO] Server Stopped
[2022-06-22 11:21:12,047] INFO app::run():l1121: Server Stopped

Interact with the TNS

Once the frb-voe backend is started, a dedicated CLI can be used for all interactions with the Transient Name Server. The command signature is the following:

Text Only
poetry run frb-voe tns [COMMAND] [OPTIONS]

For CHIME/FRB Users

If you are running the server and using the CLI on your local machine, you need to set up a tunnel to site in order to access the CHIME/FRB Workflow system. Use one of the following two commands to do this, subsitituting your LDAP username.

Text Only
ssh -L 8004:frb-vsop.chime:8004 <USER>@tubular.chimenet.ca
Text Only
ssh -L 8004:frb-vsop.chime:8004 <USER>@liberty.chimenet.ca
Close the tunnel with exit when you are finished with the TNS interactions (e.g., submissions or search queries).

Now that you have the tunnel open, before you follow the below instructions, also make sure that 1. The environment variables are properly set 2. You're in the frb-voe conda environment 3. Run poetry run maestro start to start the server

Submit an FRB

When you have the event number of a FRB Candidate that needs to be submitted to the TNS, use the following command to acquire the TNS name.

Text Only
poetry run frb-voe tns chimefrb-submit --help

The help dialogue will explain what options are required and what data is needed from the user. These include:

  • event number
  • proprietary period length (in years)

Optionally, one can practice the submission by setting the --sandbox flag in the call signature.

For Other Users

TBC.

Search for an Existing FRB

Many observatories keep a record of their detections under an internal name before the data is submitted for official naming from the TNS. Assuming this internal name was provided during the submission to the TNS, one can also search the TNS by the internal name to obtain the TNS name.

Text Only
poetry run frb-voe tns search -i chimefrb_1234567890

In this example, the internal name is chimefrb_1234567890. If successful, the command will print the TNS name in the format YYYMMDDx to the screen. Some (older or manually submitted) CHIME/FRB events have their internal names as 1234567890.

Update the Proprietary Period of an Existing FRB

Submissions to the TNS can be made proprietary for a period of time. The submit CLI allows this period to be specified in an integer number of years. When it comes time to publish the FRB in a paper, the owner of the data should stage the data for release to the public on the TNS. This can be done by updating the date on which the proprietary period will end.

Text Only
poetry run frb-voe tns change --name YYYYMMDDx --group-id XX --date 2022-01-01

In this example, the FRB with a TNS name YYYYMMDDx and owned by the TNS group with ID number XX will become public on the TNS on January 1, 2022.