Skip to content

Developer Guide

Contributing to frb-master

If you are thinking of making code changes and feature development (or even a simple bug fix), please first open a Github Issue, then click on Development in the issue page to create a new branch that you can work on.

Pre-requisites

  • Python 3.9
  • Make sure you have the following packages installed in your machine:
  • curl
  • rsync
  • libldap2-dev
  • libsasl2-dev
  • git
  • gcc
  • make
  • openssh-client

You can install the above packages using apt or apt-get:

Text Only
sudo apt-get install curl rsync libldap2-dev libsasl2-dev git gcc make openssh-client

How to install the project

To clone the frb-master repo in your local environment, run the following command in your local terminal and inside a folder of your choice:

Text Only
git clone https://github.com/CHIMEFRB/frb-master.git
cd frb-master/

If you have a branch you would like to work on, run the following command:

Text Only
git checkout <issue-branch-name>

frb-master uses poetry for package management. To install the dependencies, run

Text Only
poetry shell
poetry install

Please make sure you have installed commitizen, or, if you prefer GUI over command line, have installed any Conventional Commit extensions in VSCode so that all the commits follow the conventional commit standard.

And now you are all set up to make code changes!

After you have completed your changes (and added tests and documentations!), please open a Pull Request on GitHub for your changes to be reviewed before deploying onsite.

Environment Variables

Below are some of the environment variables used:

  • SANIC_FRB_MASTER_HOST -- Host Sanic binds to for local runs, set to 0.0.0.0
  • SANIC_FRB_MASTER_PORT -- Port Sanic listens on for local runs, set to 8001
  • SANIC_FRB_MASTER_WORKERS -- Number of Sanic worker processes, for local testing, it is set to 1
  • MDB_HOST -- MongoDB host for the main frb-master database
  • MARIADB_HOST -- MariaDB host used by the L4-related database layer
  • POSTGRESDB_HOST -- PostgreSQL host used by the datatrail database layer
  • SANIC_LDAP_IP_ADDRESS -- LDAP endpoint used for local authentication testing (not used in prod)
  • MDB_PORT -- MongoDB port
  • DOCKER_BASE_URL -- Docker Engine endpoint used by the local cluster container, set to tcp://cluster-node:3333
  • DISABLE_AUTHORIZATION -- Disables authorization checks when set to a truthy value
  • DEBUG -- Enables debug mode and local auto-reload when set to a truthy value
  • MARIADB_DATABASE -- MariaDB database name
  • MARIADB_USERNAME -- MariaDB username
  • MARIADB_PASSWORD -- MariaDB password
  • MARIADB_PORT -- MariaDB port
  • POSTGRESDB_PORT -- PostgreSQL port
  • POSTGRESDB_USERNAME -- PostgreSQL username
  • POSTGRESDB_PASSWORD -- PostgreSQL password
  • SANIC_TESTING -- Enables testing-specific behavior, including local LDAP and L4 settings
  • TNS_API_KEY -- API key used to access the Transient Name Server (TNS)
  • TNS_API_ID -- TNS bot identifier
  • TNS_API_BOT_NAME -- TNS bot name
  • JWT_SECRETS_FILE -- Path to the JWT secrets file used for authentication

frb-master also supports additional environment variables, including secret-backed database credentials, workflow database settings, Docker registry credentials, and Slack configuration. Not all of them are documented here. Check out the production stack file in frb-devops.

Running tests locally

To run tests locally, run the following steps: (make sure you are in the frb-master/ folder)

Text Only
docker compose up
A stream of logs will appear as different Docker containers start up. Once you see that frb-master is running: frb-master container running

In another terminal (make sure you are in the frb-master/ folder), run

Text Only
docker exec frb_master pytest
will execute all the tests under the tests/ folder. To run a specific test file, add /path/to/test after docker exec frb_master pytest.

test_swarm.py failing

When docker compose up is executed locally, it does not start a Docker Swarm; therefore, tests/test_swarm.py is expected to fail.

To stop, run:

Text Only
docker compose down

Deployment Guide

To deploy any changes onsite, first make sure that the latest releases are merged (look for any open Pull Request by github-actions like this) so that a new Docker image will be generated automatically by the Conitunuous Deployment GitHub Action.

Update Docker stack on frb-vsop

Now that a new Docker image has been generated, run the following command on frb-vsop.chime node ( ssh as frbadmin)

Text Only
cd frb-devops/stacks/core
git pull
docker stack deploy -c frb-master.yml frb_master --with-registry-auth
The Docker stack frb_master will be updated to run with the latest frb-master Docker image.

Roll Back Deployment

To roll back the version deployed onsite to the previous stable version, first identify the running version by going to https://frb.chimenet.ca/frb-master/version. To identify the last stable version, go to https://github.com/CHIMEFRB/frb-master/releases and locate the version tag before the latest release.

Version tags convention

frb-master uses conventional commit in order for release tags and CHANGELOG.md to be automatically generated/updated. That means, the version before v1.2.0 might be v1.1.1234 or v1.1.0; please double check the version number before rolling back.

With the newly deployed version identified, ssh into frb-vsop.chime as frbadmin, and run

Text Only
cd frb-devops/stacks/core
git pull
Then use nano or vim to change this line (L5) in frb-master.yml:
Text Only
    image: chimefrb/frb-master:latest
from latest to the last stable version (with 'v' included). After the change is saved, run
Text Only
docker stack deploy -c frb-master.yml frb_master --with-registry-auth

Remember to REVERT the Roll Back

The above changes to the frb-master.yml file is uncommitted and diverge from the main branch in frb-devops repo. In principle, any uncommitted change is prohibited from being deployed. After another stable version of frb-master is ready (with bug fixes) and a new Docker image generated, please revert the uncommitted changes by ssh-ing into frb-vsop.chime as frbadmin and run the following command:

Text Only
cd frb-devops/stacks/core/
git restore frb-master.yml
git pull
docker stack deploy -c frb-master.yml frb_master --with-registry-auth