Containerized Front Arena Dev/Test environments

August 20, 2024 - Rohan van der Walt

FIS Cross-Asset Trading and Risk Platform - Formerly Front Arena

This guide outlines the process of creating a containerized FIS Front Arena environment, drawing from my experience at a Cryptocurrency exchange that used Front Arena for Position and Risk Management.

These containers can be stored & run on cloud container services, like AWS ECR & ECS, if this is available in your organization.

Running the Front Arena backend entirely within Docker containers can streamline development and testing workflows, enabling faster development cycles and easier server administration.

Common Use Cases

  • Isolated Development Environments: Quickly create consistent, reproducible isolated environments.
  • ATF Testing: Set up dedicated environments tailored for Arena Testing Framework (ATF) - Based on RobotFramework
  • Testing New Releases: Easily build, install, and test new extension modules, including unit and integration tests.
  • Automated Upgrade Testing: Prepare for upgrade releases by automating the rollout and testing processes in a controlled environment.

Scope of This Guide

In this guide, we will focus on building a minimal environment that will allow a Prime client to log in, which includes setting up the Arena Data Server (ADS) and the database (ADM).

If you have any questions or need to run additional services such as AMB, AMBA, APS, ATS, ArenaWeb, etc., please feel free to reach out on LinkedIn or Telegram.

Advantages of Dockerizing Front Arena

  • Reproducibility: Docker allows you to easily and quickly reproduce development environments from snapshots. This is especially valuable when you need to reset environments to a clean state, which can be a frustrating and time-consuming task with traditional installations.
  • Streamlined Testing: When testing changes that involve static data, traditional setups often require manual cleanup, which is both time-consuming and error-prone. Docker simplifies this process, enabling you to reset environments efficiently.
  • Continuous Integration Pipelines: Docker images and volumes seamlessly integrate with standard CI/CD workflows. This allows for the automated installation of extension modules, streamlined environment building, and overall smoother integration into continuous deployment pipelines.
  • Simplified Administration: Docker simplifies the management of services by making it easy to start and stop them, create database backups, and duplicate services or environments. The rich set of Docker commands provides powerful tools for managing these tasks.

Simplified System Architecture

Typical Front Arena System Architecture

Step 1: Create a Customized Docker Image for Front Arena ADS

To begin, we'll create a customized Docker image, named fa, which will serve as the foundation for the Front Arena ADS service. This image will also be used to connect to the ADM and populate it with the initial data. You need to provide your license file to the ADS service either by copying it to the image in this step, or mapping it to the container at runtime.

Here's how to set it up:

  1. Set Environment Variables:

  2. Install Dependencies:

    • Install Python
    • Install unixODBC for database connectivity.

  3. Copy and Install RPM Packages:

    • Copy the required RPM packages into the Docker image. Downloadable from your FIS Client Portal account.
    • This includes ADS, ATS, PrimeUtil, ADM, AMBA, and DATAUPLOAD packages, the exact list may vary depending on your specific requirements.
    • Install these packages within the image.

Below is a Dockerfile that illustrates some of these steps:

FROM centos:7

ARG FA_INST_ADS_DIR=fa_inst/2020.2/ADS
ARG FA_INST_ADS_FILE=ADS2020.2.1.b122-4.15.122.0-0.x86_64.rpm

ARG FA_INST_PRIMEUTIL_DIR=fa_inst/2020.2/PRIME
ARG FA_INST_PRIMEUTIL_FILE=PRIMEUTIL2020.2.1.b942-4.41.942.0-0.x86_64.rpm

# Add other installation files as needed
ARG FA_INST_ATS_DIR=fa_inst/2020.2/ATS
ARG FA_INST_ATS_FILE=ATS2020.2.2.b979-4.41.979.0-0.x86_64.rpm

ARG ADS_VERSION=2020.2.1.b122
…
RUN rpm -ivh /ADS.rpm
RUN rpm -ivh /ATS.rpm
RUN rpm -ivh /PRIMEUTIL.rpm

RUN rpm -ivh /ADM.rpm
RUN rpm -ivh /AMBA.rpm
RUN rpm -ivh /DATAUPLOAD.rpm

This Dockerfile only serves as a preview, and would need to be adjusted for your specific environment.

For help with setting this up in your environment, get in contact on LinkedIn or Telegram.

Step 2: Start an Empty MS SQL Server

In this step, we will start an empty Microsoft SQL Server instance. For this, you can use the official Docker image provided by Microsoft.

Here's the Dockerfile to get started:

# Use the official MS SQL Server 2019 image from Microsoft
FROM mcr.microsoft.com/mssql/server:2019-latest

# Set environment variables for MS SQL Server
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=%SECRETPASS%

# Expose the default SQL Server port
EXPOSE 1433

Step 3: Create Database, Start ADS, and Install Arena Data Model + DataUpload

In this step, we'll set up the necessary database and user, start the Arena Data Server (ADS), and install the Arena Data Model. To automate these tasks, we'll create an entrypoint.sh script within the fa image. This script will connect to the MS SQL Server instance (referred to as mssql), create the data schema, and install the Arena Data Model.

Here's an example of an entrypoint.sh script:

# Use sqlcmd to connect to your MSSQL database, and create the front arena database and user.
# In this case “ADM_PROD”
sqlcmd -S mssql -U sa -P $SA_PASSWORD -Q "CREATE DATABASE ADM_PROD ON PRIMARY;
CREATE LOGIN ADM_PROD WITH PASSWORD = '%SECRETPASS%', DEFAULT_DATABASE = ADM_PROD;
ALTER AUTHORIZATION ON DATABASE::ADM_PROD TO ADM_PROD;"

# Install ADM
/opt/front/arena/sbin/install_adm_mssql prod
# Enter the database owner login for database ADM_PROD ..
# Use a tool like expect if you need to automatically handle this.

Step 4: Upload the DataUpload Package

With the database and Arena Data Model set up, the next step is to populate the database using the "DataUpload" package. This package will ensure that the database is initialized with the required data for a clean Front Arena environment.

# Rerun intas_revusers after user creation
sqlcmd -S mssql -U sa -P $SA_PASSWORD -Q "EXECUTE ADM_PROD.dbo.intas_revusers '%SECRETPASS%'"

# Apply All Components
cd /opt/front/arena/sbin
write_appl prod -server fa-ads:9000 -user ARENASYS

# Run DataUpload
/opt/front/arena/sbin/import_dataupld prod intas ARENASYS -a

# Enter accounting currency (CHF, DKK, EUR, GBP, JPY, NOK, SEK, USD, ZAR) (default: EUR)
# Use a tool like `expect` if you need to automate this
# Rerun intas_revusers
sqlcmd -S mssql -U sa -P $SA_PASSWORD -Q "EXECUTE ADM_PROD.dbo.intas_revusers '%SECRETPASS%'"

DataUpload

An example docker-compose.yml file that sets up both the MS SQL Server (ADM) and the Front Arena (ADS) service. This configuration will automatically handle starting the database server, and running the ADS. Remember to map your license file to the fa service if you didnt copy it to the base image.

services:
  mssql:
    image: mcr.microsoft.com/mssql/server:2019-latest
    environment:
      - ACCEPT_EULA=Y
      - SA_PASSWORD=$SA_PASSWORD
    volumes:
      - fa-db-persist:/var/opt/mssql  # Docker Volume Persistent storage for the database

  fa:
    image: fa
    environment:
      - SA_PASSWORD=$SA_PASSWORD
    depends_on:
      - mssql  # Ensure MSSQL service is up before starting the 'fa' service
    entrypoint: /usr/local/bin/entrypoint.sh  # Custom entrypoint script for setup and initialization - map this as needed

volumes:
  fa-db-persist:

Docker containers for FA

Step 5: Access and Configure Front Arena

  1. Connect to ADS:
    • Use the IP address and port of your Docker container to access the Arena Data Server (ADS). For example, localhost:9000.

Front Arena Prime Login Screen

  1. Login:

    • Start your Front Arena Prime client and connect to the ADS service.
    • Log in using the ARENASYS system user for initial setup.

  2. Initial Setup:

    • If you see missing extension modules on the first login, use the Extension Editor to add them.

Extension context is missing some modules
  1. Alternative Setup:
    • Optionally, use a snapshot from an existing instance (e.g., Prod) to populate your environment.

With these steps, your Front Arena backend should be up and running.

Duplicating the environment

Potential Next Steps

  • Create Users/Groups: Set up User Profiles, Groups, and Users.
  • Configure Portfolios: Define Portfolio and Tree structures.
  • Set Up Instruments: Add and configure instruments.
  • Price Feeds: Integrate and configure price feeds.
  • Define Curves/Surfaces: Set up Yield Curves and Volatility Surfaces.
  • Additional Services: Configure AMB, AMBA, ATS, and other integrations.

Screenshot of the newly created environment:

Extension context is missing some modules

Keywords:

#FIS #FrontArena #CrossAssetTradingAndRisk #Prime #PortfolioManagement #Containerisation #Docker