Upgrade SD Elements

Determine required changes to values.custom.yaml. See Compatibility documention for more details.
Steps:

  1. Determine if a new Chart version is needed. See Download the SD Elements Helm chart for more information.

  2. Run helm upgrade with the same parameters used with helm install …​ in the previous section

Compatibility

This page provides information on changes to helm customizations that may cause confusion or backwards incompatibility with older charts. Additional information on the customizations can be found in the default values.yaml packaged in the SD Elements helm chart.

Any changes marked as "Important" will contain new changes requiring mandatory modification to your custom overlays. All other changes are provided by default in the provided values.yaml packaged with the SD Elements helm chart.

Upgrade path planning

Certain SD Elements versions contain changes to our architecture and necessitate making breaking change. Upgrades to the latest version require intermediate upgrades to prerequisite versions (e.g. to perform data migration, prevent data loss, etc.). Ensure your upgrade path includes each prerequisite version listed below

Prerequisite VersionComments
2022.2.71Data migration from `ReadWriteMany` volumes to S3 object storage

SD Elements 2023.1

Summary

Below are the changes an administrator of SD Elements should verify before attempting an upgrade to version 2023.1.

  • Update datastore subsection customizations to sc-datastore

  • [If Applicable] Allow access to repository.securitycompass.com

datastore subsection changes

As part of this change, performing the upgrade while cron jobs are running may put the cron job pod into a waiting state and unable to find a key in the secrets. Please refer to the Troubleshooting guide to remediate this issue.

The provided datastore has been refactored to clarify parameters and application ownership. Upgrading to this version and above requires a restructure of the values.yaml regarding the datastore section.

Old ParameterNew ParameterComments
datastore.*sc-datastore.*

Parameters that are not listed in the above table are unmodified.

New artifact distribution infrastructure

New infrastructure has been put in place to deliver artifacts (e.g. SD Elements containers and helm charts) to customers. If your infrastructure configuration requires network restrictions, ensure this new hostname, repository.securitycompass.com, is allowed.

Additionally, helm installation instructions have been updated. See Deploy SD Elements using Helm for more details

If you’ve previously configured New Image Registry for Older Versions, revert to the default configuration below:

global:
  imageRegistry: repository.securitycompass.com
  imageRegistryFormat: "%s/sde-docker-%s/%s/%s:%s"
  imageOrganization: prod
  imageSource: sde

SD Elements 2022.3

Summary

All volumes using ReadWriteMany access modes are removed starting with SD Elements version 2022.3. Ensure your SD Elements deployment has been upgraded to 2022.2 before proceeding to 2022.3.

S3 and RWX volumes

SD Elements no longer uses volumes that use ReadWriteMany access modes in this version, and the section global.rwx is deprecated and is safe to remove.

SD Elements 2022.2

Summary

Below are the changes an administrator of SD Elements should verify before attempting an upgrade to version 2022.2.

  • S3 Bucket is created and credentials for S3 are passed under global.sharedStorage (See S3 Configuration)

  • Set global.rwx.enabled

    • true if updating from previous release

    • false if performing a new installation

  • The mail subsection is deprecated and replaced with sc-mail

RWX volumes

When updating to 2022.2, set global.rwx.enabled=true to ensure data is not lost.

Version 2022.2 removes requirement for Read-Write-Many volumes. Upgrades to this version require global.rwx.enabled=true to retain previous behaviour and data migration (more under Scheduled backup changes). New installations are safe to set global.rwx.enabled=false.

Scheduled backup changes

Backups are now stored in the configured S3 bucket. By default this job * Generates a full backup every 20 minutes * Removes backups older than 14 days * Stores backups under /backup directory in the S3 bucket These backup timing and retention period can be configured in job.specifications.database-backup

For usage details on backup and restore, see Backup-Restore in the API documentation

mail subsection changes

The included mail transfer agent (MTA), postfix, is replaced with exim. As a result all previous configuration in the mail subsection needs to be re-configured under the sc-mail.config subsection. For more information, see the values.yaml of the Helm chart.

SD Elements 5.16

Summary

Below are the changes an administrator of SD Elements should verify before attempting an upgrade to version 5.16.

  • broker subsection customizations to be changed to sc-broker

  • database subsection customizations to be changed to sc-database

  • Secrets must be retrieved and explicitly defined as customizations

  • sc-database.clientUser to be set to sde

  • global.nameOverride to be set to sde

  • job subsection customizations to be defined as map instead of list

broker and database subsection changes

The provided broker and databse has been refactored to clarify parameters and application ownership. Upgrading to this version and later requires a restructure of the values.yaml regarding the broker section.

Old ParameterNew ParameterComments
broker.*sc-broker.*
database.*sc-database.*

Parameters that are not listed in the above table are unmodified.
As part of this change, performing the upgrade while cron jobs are running may put the cron job pod into a waiting state and is unable to find a key in the secrets. Please refer to the Troubleshooting guide to remediate this issue.

Explicitly defined secrets

All secrets must be explicitly defined. Below is an example of an overlay file with all new secrets expected to be defined These secrets should be managed and maintained like any software that requires version control.

sde:
  jwtSecret: SDE_JWT_SECRET
  secretKey: SDE_SECRET_KEY
  superuserPassword: SDE_SUPERUSER_PASSWORD
sc-database:
  clientPassword: SDE_DATABASE_PASSWORD
datastore:
  clientPassword: SDE_DATASTORE_PASSWORD
sc-broker:
  clientPassword: SDE_BROKER_PASSWORD

The secrets must match the existing deployment. By default the secrets are base 64 encoded and require decoding before input into above overlay.

 # Syntax
 $ kubectl get secrets "<RELEASE_NAME>-sde-secrets" --output json > sde-secrets.json
 # Example
 $ kubectl get secrets "prod-sde-secrets" --output json > sde-secrets.json

Database user update

Default db user has changed for upgrades from previous versions, and requires sc-database.clientUser=sde to be set. Fresh installs do not require this setting.

Labels update

Default labels have changed for upgrades from previous versions, and requires global.nameOverride=sde to be set. Fresh installs do not require this setting.

job subsection changes

Job customizations are now a map instead of a list, allowing for easier overrides. Below is an example of how a job was previous configured, and how it is configured now. Fresh installations and uncustomized jobs do not require any changes.

 # Previous configuration
 job:
   - name: alm-hourly
     schedule: "42 * * * *"
     niceness: 19
     command: ["/bin/sde.sh"]
     args:
       - "almsync"
       - "hourly"
 # Updated configuration
 job:
   specification:
     alm-hourly:
       schedule: "42 * * * *"
       niceness: 19
       command: ["/bin/sde.sh"]
       args:
         - "almsync"
         - "hourly"

Resources can also be customized in the job section. They can be added for all jobs or for specific jobs.

 # For all jobs
 job:
   resources:
     requests:
       cpu: 500m
       memory: 512Mi
 # For specific jobs:
 job:
   specifications:
     alm-hourly:
       resources:
         requests:
           cpu: 500m
           memory: 512Mi

results matching ""

    No results matching ""