Русский

Installation in Kubernetes¶

Installation using Helm chart with default parameters¶

Important!: This installation option does not provide an ability to scale CodeScoring horizontally. To install CodeScoring with horizontal scaling support, please refer to the relevant documentation section below.

Important!: You must have a configured default StorageClass in the cluster. By default, volumes are created with a capacity of 20 GiB

Installation order:

Create a namespace.
```
kubectl create namespace codescoring
```
Create a secret to access the private registry of Docker images of the "CodeScoring" system, using the address (REGISTRY_URL), login (USERNAME) and password (PASSWORD) received from the vendor.
```
kubectl create secret docker-registry codescoring-regcred --docker-server=REGISTRY_URL --docker-username=USERNAME --docker-password=PASSWORD -n codescoring
```
Install Helm using your preferred method.

Run the following commands to add the current Helm repository to the local machine:

helm repo add codescoring-org https://registry-one.codescoring.ru/repository/helm/ --username USERNAME --password PASSWORD
helm repo update

Create a values.yaml file with the following content:

Important!: Please replace the values in the sensitive data fields with your own. These fields include secretKey, defaultSuperuserUsername, defaultSuperuserPassword, defaultSuperuserEmail, and all fields containing username or password. It is also important to note that all such variables are mandatory.

codescoring:
 config:
   ## codescoring-backend configuration parameters
   siteScheme: https # site scheme http or https
   siteHost: "codescoring.k8s.local" # domain where CodeScoring will be available
   djangoCSRFTrustedOptions: "https://codescoring.k8s.local" # Domain where CodeScoring will be available, including schema
   secretKey: "" # secret key for the application backend, random string of characters
   defaultSuperuserUsername: "admin" # administrator name on the system
   defaultSuperuserPassword: "changeme" # system administrator password
   defaultSuperuserEmail: "mail@example.com" # e-mail of the administrator in the system
   databaseHost: ipcs-pgcat
   databasePort: 5432
   postgresqlDatabase: "codescoring"
   postgresqlUsername: "codescoring"
   postgresqlPassword: "changeme" # password must match the password for pgcat.postgresql.password

pgcat:
 adminPassword: "changeme"

postgresql:
 host: "codescoring-postgresql"
 port: 5432
 username: "codescoring"
 password: "changeme" # password must match the password in codescoring.postgresqlPassword
 database: "codescoring"


frontend:
 ingress:
   enabled: true
   className: "nginx"
   hosts:
   - host: codescoring.k8s.local # domain where CodeScoring will be available
   paths:
   - path: /
   pathType: ImplementationSpecific

Run the command to install the chart

helm install codescoring codescoring-org/codescoring -n codescoring -f values.yaml --create-namespace --atomic --version CHART_VERSION

Changing the admin password¶

To change the admin password without manually editing the values.yaml file, you can use the following command:

kubectl exec -it your-backend-pod -- python manage.py changepassword <user_name>

Advanced settings for Helm chart parameters¶

Important: It is highly recommended that you make the necessary changes before installing CodeScoring, otherwise a complete system reinstall may be required. These instructions assume that the specialist has experience working with a Kubernetes cluster and the Helm utility.

To easily edit CodeScoring parameters, you can download and unpack the Helm chart source code with the command:

helm pull codescoring-org/codescoring --version CHART_VERSION --untar --untardir codescoring-src && cd codescoring-src

In the values.yaml file you can edit the necessary variables, and after that run the installation command in the directory with the Helm chart source code:

helm install codescoring . -f values.yaml -n codescoring --atomic --version CHART_VERSION

Connecting to external PostgreSQL and Redis¶

By default, PostgreSQL and Redis run in separate StatefulSets. This option may not be suitable for usage in a production environment, because it is not fault tolerant.

Connecting to external Redis¶

To connect to external Redis, you must do the following:

Disable Redis deployment by specifying the variable - redis.enabled: false
In the codescoring.config.djangoCachesRedisUrls and codescoring.config.hueyRedisUrl variables, specify the connection strings for external Redis.

Connecting to external Redis via TLS¶

To connect to external Redis, in addition to abovementioned you must do the following:

Set the codescoring.trustedCA.enabled variable to true
Add the Redis-server root certificate to codescoring.trustedCA.certificates
In the codescoring.config.djangoCachesRedisUrls and codescoring.config.hueyRedisUrl variables, specify the connection strings for external Redis using the following format: rediss://redis.example.com:6379/0, where 0 is the Redis database number.

Connecting to PostgreSQL via PgCat pooler¶

Important!: Connecting to external PostgreSQL must be done using a connection pooler.

This option is suitable if PostgreSQL is already deployed in the existing infrastructure, but the connection pool is not used. Helm-chart will deploy the PgCat puller and connect it to the existing PostgreSQL. You need to do the following:

Disable PostgreSQL deployment by specifying the variable - postgresql.enabled: false

Connect the PgCat pooler to external PostgreSQL, replacing the appropriate parameters with the required ones:

codescoring:
  config:
    postgresqlDatabase: "codescoring"
    postgresqlUsername: "codescoring"
    postgresqlPassword: "changeme"
  pgcat:
    postgresql:
      host: "postgresql.example.host"
      port: 5432
      username: "codescoring"
      password: "changeme"
      database: "codescoring"

Connecting to an external PostgreSQL pooler¶

This option is suitable if PostgreSQL and a connection pooler (for example, PgBouncer) are already deployed in the existing infrastructure. In this case, deployment of the PgCat pooler is not required. You need to do the following:

Disable PostgreSQL deployment by specifying the variable - postgresql.enabled: false
Disable PgCat deployment by specifying the variable - codescoring.pgcat.enabled: false
Connect codescoring directly to an external pooler, in the codescoring.config section the parameters are:

posgtresqlHost: ipcs-pgcat
posgtresqlPort: 5432
postgresqlDatabase: "codescoring"
postgresqlUsername: "codescoring"
postgresqlPassword: "changeme"

4. Enable an external pooler and adjust the connection settings in the codescoring.config.externalPooler section:

Important!: An external pooler must support both transaction and session work modes. The connection settings for the pools must be adjusted in corresponding sections: externalPooler.transactionPool and externalPooler.sessionPool.

externalPooler:
  enabled: true
  transctionPool:
    host: ipcs-pgcat
    port: 5432
    username: "codescoring"
    password: "changeme"
  sessionPool:
    host: ipcs-pgcat
    port: 5432
    username: "codescoring"
    password: "changeme"

Setting up volumes (PV)¶

By default, the chart creates the required volumes via Dynamic Volume Provisioning using the default StorageClass. If this volume deployment option is not suitable, you can configure volume creation in several ways.

Important!: The options described below are mutually exclusive. You must select ONLY ONE deployment option for each volume. You can select different deployment options for different volumes.

Note

To change the size of created volumes (except for local ones), you need to change the size parameter in the corresponding sections.

Dynamic Volume Provisioning using the required StorageClass¶

You can set the required StorageClass in the following variables:

codescoring.persistentVolumes.analysisRoot.storageClass
codescoring.persistentVolumes.mediaRoot.storageClass
codescoring.persistentVolumes.djangoStatic.storageClass
codescoring.backup.persistentVolume.storageClass
redis.persistentVolume.storageClass (if using built-in Redis)
postgresql.persistentVolume.storageClass (if using built-in PostgreSQL)

In this case, volumes will be created using the specified StorageClass

PersistentVolumeClaim for pre-created PersistentVolumes¶

The name of pre-created volumes can be set in the following variables:

codescoring.persistentVolumes.analysisRoot.volumeName
codescoring.persistentVolumes.mediaRoot.volumeName
codescoring.persistentVolumes.djangoStatic.volumeName
codescoring.backup.persistentVolume.volumeName
redis.persistentVolume.volumeName (if using built-in Redis)
postgresql.persistentVolume.volumeName (if using built-in PostgreSQL)

In this case, only PersistentVolumeClaim will be created for the volumes specified in these variables

Using pre-created PersistentVolumeClaim¶

The name of pre-created PVCs can be set in the following variables:

codescoring.persistentVolumes.analysisRoot.existingClaim
codescoring.persistentVolumes.mediaRoot.existingClaim
codescoring.persistentVolumes.djangoStatic.existingClaim
codescoring.backup.persistentVolume.existingClaim
redis.persistentVolume.existingClaim (if using built-in Redis)
postgresql.persistentVolume.exsistingClaim (if using built-in PostgreSQL)

In this case, the specified PVC name will be inserted directly into the volumes section for Pod.

Using local volumes¶

If there is no external data storage in the Kubernetes cluster, it is possible to run CodeScoring using local volumes. In this case, the data will be stored on one of the cluster nodes.

To create local volumes, you must perform the following steps:

Assign the value true to the following variables:
codescoring.persistentVolumes.analysisRoot.localVolume.enabled
codescoring.persistentVolumes.mediaRoot.localVolume.enabled
codescoring.persistentVolumes.djangoStatic.localVolume.enabled
codescoring.backup.persistentVolume.localVolume.enabled
redis.persistentVolume.localVolume.enabled (if using built-in Redis)
postgresql.persistentVolume.localVolume.enabled (if using built-in PostgreSQL)
Set the path to the directory on the cluster node, in which the data in the following variables will be placed:
codescoring.persistentVolumes.analysisRoot.localVolume.path
codescoring.persistentVolumes.mediaRoot.localVolume.path
codescoring.persistentVolumes.djangoStatic.localVolume.path
codescoring.backup.persistentVolume.localVolume.path
redis.persistentVolume.localVolume.path (if using built-in Redis)
postgresql.persistentVolume.localVolume.path (if using built-in PostgreSQL)
Specify the name of the node on which the local volume will be created in the following variables:
codescoring.persistentVolumes.analysisRoot.localVolume.nodeHostname
codescoring.persistentVolumes.djangoStatic.localVolume.nodeHostname
codescoring.backup.persistentVolume.localVolume.nodeHostname
redis.persistentVolume.localVolume.nodeHostname (if using built-in Redis)
postgresql.persistentVolume.localVolume.nodeHostname (if using built-in PostgreSQL)

It is allowed to use different nodes for different volumes.

Setting up storage for temporary scan files¶

By default, temporary files during the scanning process are stored in the /tmp directory inside containers, to which Ephemeral Volumes of type emptyDir are mounted:

codescoring.huey.ipcsQueue.ephemeralVolumes
codescoring.huey.tasksOsaContainerImageScan.ephemeralVolumes
codescoring.huey.tasksOsaPackageScan.ephemeralVolumes

However, in some cases you may need to use Persistent Volume instead of Ephemeral Volume. In this case, you should comment out the appropriate sections in ephemeralVolumes for one or more services, depending on which services require volumes to be mounted:

codescoring:
  huey:
    ipcsQueue:
      ephemeralVolumes:
        volumeMounts:
        # - mountPath: /tmp
        #   name: ipcs-queue-tmp
        - mountPath: /etc/ssl/certs
          name: ipcs-queue-ssl-certs
        volumes:
        # - name: ipcs-queue-tmp
        #   emptyDir: {}
        - name: ipcs-queue-ssl-certs
          emptyDir: {}

    tasksOsaContainerImageScan:
      ephemeralVolumes:
        volumeMounts:
        # - mountPath: /tmp
        #   name: container-image-scan-tmp
        - mountPath: /etc/ssl/certs
          name: container-image-scan-ssl-certs
        volumes:
        # - name: container-image-scan-tmp
        #   emptyDir: {}
        - name: container-image-scan-ssl-certs
          emptyDir: {}

    tasksOsaPackageScan:
      ephemeralVolumes:
        volumeMounts:
        # - mountPath: /tmp
        #   name: package-scan-tmp
        - mountPath: /etc/ssl/certs
          name: package-scan-ssl-certs
        volumes:
        # - name: package-scan-tmp
        #   emptyDir: {}
        - name: package-scan-ssl-certs
          emptyDir: {}

Then you need to set the value enabled: true in one or more of the following sections:

codescoring.huey.persistentVolumes.hueyTmp
codescoring.huey.persistentVolumes.hueyPackageScanTmp
codescoring.huey.persistentVolumes.hueyContainerImageScanTmp

As a result, PersistentVolumeClaim will be created for the corresponding services. It is worth noting that the possibilities for configuring these volumes fully correspond to those described in the section Configuring volumes (PV).

When horizontally scaling services, you need to configure volumes in accordance with the instructions in the Horizontal scaling CodeScoring section.

Horizontal scaling CodeScoring¶

Important!: To horizontally scale the CodeScoring system, the Kubernetes cluster must have the ability to create volumes with the access type ReadWriteMany (RWX)

To scale CodeScoring horizontally, you need to create analysis-root, media-root and django-static volumes with an access type of ReadWriteMany.

To do this, you need to replace the value of ReadWriteOnce with ReadWriteMany in the variables:

codescoring.persistentVolumes.analysisRoot.accessModes
codescoring.persistentVolumes.mediaRoot.accessModes
codescoring.persistentVolumes.djangoStatic.accessModes

Then, you need to comment out the variables: - codescoring.backend.affinity - codescoring.frontend.affinity

If this is not done, then all pods will be launched on only one node in the cluster.

Setting resource limits¶

By default, requests and limits are not specified. This is done to ensure the ability to run the CodeScoring system in clusters with a small number of resources (for example, minikube) for testing purposes. When running in a production environment, you may need to configure resource limits. This can be done by setting the following variables:

postgresql.resources (when using built-in PostgreSQL)
redis.resources (when using built-in Redis)
codescoring.backend.resources
codescoring.frontend.resources
codescoring.huey.highPriorityQueue.resources
codescoring.huey.ipcsQueue.resources
codescoring.huey.tasksOsaContainerImageScan.resources
codescoring.huey.tasksOsaPackageScan.resources
codescoring.huey.tasksPolicy.resources
codescoring.huey.tasksMedia.resources

It is possible to specify both resources and limits together or separately, for example:

codescoring:
  backend:
    resources:
      limits:
        cpu: 1000m
        memory: 2000Mi
  huey:
    ipcsQueue:
      resources:
        limits:
          cpu: 2000m
          memory: 3000Mi
        requests:
          cpu: 1000m
          memory: 1000Mi

Below are approximate limits values for a CodeScoring installation

with 8-10 projects: href="#__codelineno-13-1">codescoring: backend: resources: limits: cpu: 250m memory: 2500Mi huey: ipcsQueue: scheduler: resources: limits: cpu: 500m memory: 500Mi resources: limits: cpu: 2250m memory: 4000Mi highPriorityQueue: resources: limits: cpu: 2250m memory: 4000Mi tasksOsaContainerImageScan: resources: limits: cpu: 2250m memory: 4000Mi tasksOsaPackageScan: resources: limits: cpu: 2250m memory: 4000Mi tasksOsaPackageScan: resources: limits: cpu: 2250m memory: 4000Mi tasksPolicy: resources: limits: cpu: 2250m memory: 4000Mi tasksTqi: resources: limits: cpu: 2250m memory: 4000Mi tasksMedia: resources: limits: cpu: 1000m memory: 1500Mi frontend: resources: limits: cpu: 250m memory: 500Mi redis: resources: limits: cpu: 1000m memory: 2000Mi postgresql: resources: limits: cpu: 1000m memory: 2000Mi

Adding a Certificate Authority (CA) certificate¶

To access CodeScoring resources with TLS certificates signed by a corporate certification authority (CA), you must:

Set the codescoring.trustedCA.enabled variable to true
Add the root certificate of the certification authority (RootCA) in PEM format to the codescoring.trustedCA.certificates variable in the key: value format, where key is the name of the certificate file, including the .crt extension, value is the certificate in PEM format.

For example:

codescoring:
  trustedCA:
    enabled: true
    certificates:
        ## THIS IS AN EXAMPLE ONE! DO NOT USE IN PRODUCTION!
        my-root-ca.crt: |-
          -----BEGIN CERTIFICATE-----
          MIIDTDCCAjSgAwIBAgIBATANBgkqhkiG9w0BAQUFADA3MQswCQYDVQQGEwJERTEP
          MA0GA1UEChMGZWR1UEtJMRcwFQYDVQQDEw5lZHVQS0kgVGVzdCBDQTAeFw0xMDAz
          MzExMjIwMjRaFw0zMDAzMjYxMjIwMjRaMDcxCzAJBgNVBAYTAkRFMQ8wDQYDVQQK
          EwZlZHVQS0kxFzAVBgNVBAMTDmVkdVBLSSBUZXN0IENBMIIBIjANBgkqhkiG9w0B
          AQEFAAOCAQ8AMIIBCgKCAQEAt5IxCk/NQPOLqeA1lGuB3pvqHGQPxRQ1udYGcXQY
          t7EuSMFymUR9m5TsifG1ktktJTtOWyaWFC4ac0vai49wGVeuDYptfZBoHLIUvCwN
          DOofLYHxk04WzfrtSiUTptn1o6QPOw8YR0XH30MEi1zgD8fLMZmVTJ+XwA5Eus6c
          XtTmI4XhNrHUtvWt4UsNgLmp5/djUgRMpNqxIdrpFQzl+XycRJRAaoAwUzHFl14t
          49qwBhGChxQ8AdDMQGA7kv6VR8o0ktCPv3a4GQbs8+z0cX0w5dC+XhJ1xpqW6TOg
          qAY9XBFIDe5j21hjKmNZ39rsODVGUS2wUtNEhSz+3YqxLwIDAQABo2MwYTAdBgNV
          HQ4EFgQUqHe3saMjZZLan8RlFJs+Xuz4yiAwHwYDVR0jBBgwFoAUqHe3saMjZZLa
          n8RlFJs+Xuz4yiAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwDQYJ
          KoZIhvcNAQEFBQADggEBAEjQGyHZQis47c2kf+zXJJoDDlRgFzr9xfcnrHFaJvYx
          nuqNE0T+xmujnwGm3VrgddeAQJuW3sD6y0Ox8NgL4z886VFeaDQ0GmFPI6HEVtg6
          mixMhi+YzdkC+PFrEdYUeVNNwVO+bvJb1Rc08BYU4v7VtTkssHjru76E2/ahn/Ct
          kaVTEojEWeRaxsw5/0VLkgyf8SwDaukM2aamqgEzfsw5GTdSAh7ERZKc+zF7Sr5s
          DY8c5lOmyCwuNh9ODuw4cAThICrn7G8bh8ZyxLyj4Znxh0X45SwMZKTmYLfy9ab8
          b/j7FK8uBNRL+pXl9HGBWAFA01uJw4HkYK+Uo+RcAzo=
          -----END CERTIFICATE-----

If you have multiple root CAs, you need to add them to separate keys, for example:

codescoring:
  trustedCA:
    enabled: true
    certificates:
        my-root-ca.crt: |-
          ...
        my-root-ca-2.crt: |-
          ...

Secret Management¶

By default, the Secret objects are created for ipcs-backend, ipcs-pgcat and postgresql templates. The values for their variables are provided in the values file.

It is also possible to use external secret storages. To achieve that, it is necessary to have an External Secrets Operator (ESO) installed in the cluster. It adds all required CRDs (Custom Resource Definitions) to the cluster and connects to an external secret provider.

In order to connect ESO to an external secret storage, you must configure a provider for the SecretStore in the codescoring.secretStore block.

Next, you need to configure the ExternalSecret objects to fetch secrets from the external provider in the codescoring.config.externalSecret, codescoring.pgcat.externalSecret, postgresql.externalSecret blocks.

All the configurations must be made in accordance with the official ESO documentation.

Important!: Some providers may bill requests to secret storages. The request interval can be configured using the externalSecret.refreshInterval parameter for each particular service.

Monitoring¶

For metrics collection purpose there are the ServiceMonitor resources provided in the chart. The metrics are collected from the backend and the osa-api services.

The ServiceMonitor can be configured in the following values sections: codescoring.backend.prometheus.serviceMonitor, codescoring.osa_api.prometheus.serviceMonitor.

In addition there are PrometheusRule resources provided for the abovementioned services. They are required for the alerting rule setup and can be configured in the following values sections: codescoring.backend.prometheus.alerts, codescoring.osa_api.prometheus.alerts.

All the settings must be adjusted in accordance with the official Prometheus Operator documentation.

Upgrading CodeScoring¶

In order to upgrade CodeScoring you need to actualize the helm repository by running

helm repo update

and then upgrade the installation with the following command, where the CHART_NAME variable must contain the version you're going to upgrade to

helm upgrade codescoring codescoring-org/codescoring -n codescoring -f values.yaml --version CHART_VERSION