Skip to content

Offline installation

CodeScoring supports operation in air-gapped environments. In this mode, access to the package and vulnerability database is provided by the Index API Offline service.

Product prerequisites

Before using Index API Offline, the CodeScoring platform must already be deployed on the server using Docker or Kubernetes.

Installation and update resources

The resource address for the installation files, Docker Registry, and offline database files can be obtained from the vendor.

The examples below use the following placeholders:

  • <REGISTRY_URL> — HTTPS address of the distribution resource;
  • <REGISTRY_HOST> — Docker Registry address without the protocol.

The following resources are available:

  • Docker Registry with CodeScoring and Index API Offline images
  • Full offline Index API database: <REGISTRY_URL>/db/index-api.db
  • Incremental offline database updates: <REGISTRY_URL>/#browse/browse:codescoring-offline-files:db%2Fv1%2Fupdates

The offline database is provided as an encrypted SQLite file. Incremental updates are distributed as WAL files.

Downloading the database

  1. Select a server with at least 300 GB of available disk space.

  2. Create a curl.config file with the following credentials:

    user = <REGISTRY_LOGIN>:<REGISTRY_PASSWORD>

  3. Start downloading the database:

    curl --config curl.config \
     -C - \
     --output index-api.db \
     <REGISTRY_URL>/db/index-api.db

It is recommended to perform the download inside screen or tmux, as the process may take a significant amount of time.

Installing Index API Offline

  1. Log in to the Docker Registry:

    docker login <REGISTRY_HOST>

  2. Download the archive with installation files and extract it:

    curl -u <REGISTRY_LOGIN>:<REGISTRY_PASSWORD> \
     -C - \
     <REGISTRY_URL>/repository/codescoring-offline-files/index-api/docker-compose/<INDEX_API_OFFLINE_VERSION>.tar.gz \
     -o index-api-offline.tar.gz

  3. Move the downloaded database file to the service directory, for example:

    mv index-api.db index-api-offline/db

  4. Navigate to the configuration directory:

    cd index-api-offline/

  5. Copy the .env.template configuration template to .env and configure it:

    Main parameters:

    • INDEX_API_OFFLINE_VERSION — Index API Offline version;
    • NGINX_SSL_ENABLED — enable SSL (true | false);
    • NGINX_HOST — hostname for nginx;
    • OFFLINE_DB_UPDATE_ENABLED — enable automatic database updates;
    • WAL_CHECK_INTERVAL — update check interval;
    • NEXUS_HOST — update repository address;
    • NEXUS_USERNAME — Nexus username;
    • NEXUS_PASSWORD — Nexus password;
    • CODESCORING_ACTIVATION_KEY — license key.

    Authentication parameters for database updates and Prometheus metrics access (optional):

    • MAINTENANCE_USERNAME — Basic Auth username for /system/update_database;
    • MAINTENANCE_PASSWORD — Basic Auth password for /system/update_database;
    • METRICS_USERNAME — Basic Auth username for /metrics;
    • METRICS_PASSWORD — Basic Auth password for /metrics.

  6. When SSL is enabled (NGINX_SSL_ENABLED=true), place the certificate and key files in the ssl directory:

    Important

    The certificate must be in PEM format. The certificate file must have the .crt extension, and the private key file must have the .key extension.

  7. Start the service:

docker compose up -d --remove-orphans

After installation and startup, the CodeScoring interface displays the version of the latest WAL file uploaded to Index API Offline.

Database Updates

Automatic offline database updates

The database update process can be performed automatically.

The update mechanism works as follows:

  1. The running Index API Offline service periodically checks the update repository.
  2. The service stores the current database version number (for example, 72).
  3. When the next available update is detected (for example, 73), the file is downloaded automatically.
  4. The downloaded WAL file is applied to the local database.

The check interval is defined by the WAL_CHECK_INTERVAL parameter.

If OFFLINE_DB_UPDATE_ENABLED is disabled, automatic updates will not be performed.

Manual database update

Manual updating is used when automatic WAL file downloading is disabled (OFFLINE_DB_UPDATE_ENABLED=false) or is not possible due to network restrictions.

Updating using individual update files

If you have deployed the first version of an offline database, you must sequentially download and upload all available WAL update files, starting with the version following the current database version.

The update files must be placed in the directory specified by the WAL_DIR parameter in the Index API Offline configuration. The service monitors files in this directory and applies them to the database.

The update package includes:

  • *.wal — a file containing the changes (the main update data);
  • *.shm — an auxiliary service file required by SQLite for correct WAL application.

Updating using the full database version

If you are downloading the latest full offline database file, you must do the following:

  1. Stop the Index API Offline service.
  2. Move the current database file to a backup directory (it is recommended to save a copy until the upgrade is complete).
  3. Place the new database file in the directory specified in the DB_FILE parameter.
  4. Start the Index API Offline service.
  5. Verify that the service has started successfully.

After this, if necessary, you can continue the upgrade using WAL files, placing them in the WAL_DIR directory in the standard order.

Starting the upgrade process

When using manual upgrade mode, the upgrade process is also started manually.

  1. Specify authentication parameters in the .env file:

    MAINTENANCE_USERNAME=
MAINTENANCE_PASSWORD=

  2. Place the update files (*.wal, *.shm) in the directory specified in the WAL_DIR parameter.

To start the update process, run the following query:

curl -X GET --location "{{INDEX_API_URL}}/system/update_database" \
-H "Accept: application/json" \
-H "Authorization: Basic {{metrics_auth}}"

Where:

  • {{INDEX_API_URL}} is the value from the INDEX_API_URL variable in .env;
  • {{metrics_auth}} — a Base64-encoded string similar to user:password.

The update status is tracked in the Index API Offline service logs.

Possible messages:

  • WAL files download scheduler started with interval: — the update service has started;
  • missing mandatory Nexus configuration parameters — a valid message for manual mode;
  • The WAL file [%s] was successfully loaded into the main database file — the update was successfully applied;
  • Failed to checkpoint WAL file: — an error applying the WAL file;
  • No WAL files to checkpoint — there are no files to apply;
  • Error getting latest database update: — an error retrieving update information.
Страница была полезна?