Virtual Machine (VM) deployment overview

As of SD Elements 2024.2, the container deployment is the only deployment model offered for self managed instances of SD Elements. See the Container Migration Guide for information on migrating your VM instance to a Kubernetes and the Container Deployment section for documentation on installing and maintaining an SD Elements container deployment.

SD Elements is deployed as a virtual machine and managed using a combination of SSH console and web browser.

The guest virtual machine runs CentOS Linux with SELinux enabled. It is configured with strict controls to meet the security demands of government and enterprise organizations.

Virtual machine prerequisites

The SD Elements virtual machine has the following requirements:

System requirements:

  • VMWare ESXi 4.1 or more recent

  • Minimum 8 GB memory

  • At least 4 CPU cores

  • Minimum 100 GB of disk space

  • One Static Internal version 4 IP address

  • DNS name (e.g. sde.intranet.example.com)

  • TLS/SSL Certificate recognized and trusted by browsers (OpenSSL/Apache compatible format)

  • IPv4 inbound ports:

    • TCP port 80 (HTTP) (only used for redirect to port 443 during initial connection)

    • TCP port 443 (HTTPS)

    • TCP port 22 (SSH) for maintenance/management

  • IPv4 outbound ports:

    • TCP port 25 (SMTP) to a relay server for sending email

    • TCP port 443 ( Updates, patches and integrations with Issue Trackers or SAML authentication), at the very minimum the system needs access to updates.sdelements.com (application) and anvil.sdelements.com (system/OS)

    • TCP port 389/636 for LDAP/LDAPS if LDAP authentication is used

Required skills

An SD Elements administrator should be comfortable with the following concepts and skill-sets:

Required experience:

  1. TCP/IP networking

  2. Linux Administration

  3. Single Sign-on solutions like LDAP and SAML.

Components

The SD Elements server is composed of the following major components:

Name Process name Purpose

sde

sde

  • Standard way to manage the server and perform upgrades

Nginx

nginx

  • Exposes ports 80 and 443 to SD Elements application users

  • Acts as a reverse proxy to Apache

  • Hosts TLS/SSL for the web application

  • Serves public static content directly to clients

Apache

httpd

  • Processes application requests from Nginx

  • Runs the Django Python application in WSGI processes

PostreSQL

postgres

  • Database server for the application

  • Accessed by the Apache WSGI processes

Celery

celery

  • Runs application jobs (Issue Tracker, Scanner integration, emails)

RabbitMQ

rabbitmq

  • Queuing system for jobs (Issue Tracker, Scanner integration, emails)

Cron

  • Scheduling mechanism for running regular application jobs (hourly, daily, weekly, monthly)

Python

python

  • Web application, celery and updater are written in Python

Puppet

puppet

  • Configuration management tool for handling system configuration in a consistent and repeatable way.

Postfix

postfix

  • Handles emails initiated by the application.

sde_admin

  • Python package for sde

sdetools

  • Python package for the integration components

    • Issue Tracker, Scanner, and LDAP integration

Log files

Files stored in /docs/sde/log generally track the application events associated with SD Elements. Other system processes that run on the server store their logs according to the default CentOS location, typically /var/log/.

In the table below VERSION denotes the application version; for example 4.14.
Component File Scope

SD Elements web application

/docs/sde/log/sdlc.log

Main source of application logs

SAML support

/docs/sde/log/saml.log

SAML incoming requests

Celery

/docs/sde/log/celery/celery_docs_sde_VERSION.log

Logs for integration (Issue Tracker, scanner tools) and email jobs.

Nginx

  • /docs/sde/log/nginx_docs_sde_VERSION_access.log

  • /docs/sde/log/nginx_docs_sde_VERSION_error.log

  • Contains web application request logs

  • Contains web application error logs

Apache

  • /docs/sde/log/apache_access_main_docs_sde_VERSION.log

  • /docs/sde/log/apache_error_main_docs_sde_VERSION.log

  • Contains core application request logs

  • Contains core application error logs

sde

/docs/sde/log/deploy.log

Logs collected during upgrades

LDAP

/docs/sde/log/ldap.log

Contains LDAP logs

Notable system files

File Purpose

/docs/sde/local_settings

Custom application settings that survive upgrades. These override and extend core application settings

/docs/sde/updater.cfg

Contains username, password and URL location for downloading SD Elements updates.

/etc/sde/custom.yaml

A YAML file of custom settings used to configure the server. See Supported system configuration for the list of available settings. The settings are applied after each upgrade or when sde reprovision is run.

Supported system configuration

The system can be configured with a wide array of settings and options that are correlated during upgrades or when sde reprovision is run. For a complete list of officially supported options, see /etc/sde/custom.yaml.example. A sample of options is provided below.

Any other system configuration changes may be lost or overwritten when the system is reprovisioned or during an upgrade.

Name Settings

NTP time service

 
ntp::enable:
  - true
ntp::restrict:
  - 127.0.0.1
ntp::autoupdate: false
ntp::servers:
  - 0.us.pool.ntp.org iburst
  - 1.us.pool.ntp.org iburst
  - 2.us.pool.ntp.org iburst
  - 3.us.pool.ntp.org iburst

The SD Elements activation key for access to update repositories for subscription-manager (supplied by SD Elements Support)

 
role::sdelements_server::activation_key: 'Acme-rhel7-81ab92bb-54ff-4aaaa-9122-ab4ac1e1241e'

Application Server TLS/SSL

 
role::sdelements_server::ssl_key: '/etc/apache2/ssl/apache.key'
role::sdelements_server::ssl_cert: '/etc/apache2/ssl/apache.crt'

Custom certificates

 
role::server::custom_ca_certs: '/etc/sde/custom_ca_certs/'

Manage firewall

 
role::server::manage_firewall: false
# Add firewall rules to iptables to open custom ports
#profile::managed_firewall::custom_ports:
# - 8099
# - 8299

Log rotation

 
role::server::manage_logrotate: false

Server admin email

 
role::sdelements_server::admin_email: 'support@sdelements.com'

SSH daemon

 
sshd::permit_root_login: 'no'
sshd::password_authentication: 'no'
sshd::kerberos_authentication: 'no'
sshd::gssapi_authentication: 'no'
sshd::agent_forwarding: 'no'
sshd::tcp_forwarding: 'no'

Update configuration

 
profile::sde_instance::instance_upgrade_user: <username>
profile::sde_instance::instance_upgrade_password: <supersecret>
profile::sde_instance::instance_upgrade_url: 'https://update.sdelements.com/sde/prod/4/'

Application worker processes

 
profile::sde_instance::wsgi_processes: 4
profile::sde_instance::wsgi_threads: 2
profile::sde_instance::wsgi_threads: 1
profile::sde_instance::wsgi_threads: 1
profile::sde_instance::wsgi_batch_processes: 2
profile::sde_instance::wsgi_batch_threads: 1
profile::sde_instance::wsgi_batch_priority: 20

Webserver settings

 
sde::instance::sde_admin_apache_vhost_port: 8099

TLS/SSL Config

 
sde::instance::ssl_protocols: TLSv1, TLSv1.1, TLSv1.2
sde::instance::ssl_ciphers: ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256

Nginx worker processes

 
nginx::worker_processes: 4

Apache mod_rewrite

 
# Enable and configure Apache mod_rewrite rules.
# See: https://httpd.apache.org/docs/current/mod/mod_rewrite.html
profile::sde_instance::apache_rewrites:
 - comment: 'Always send back a 204 for OPTIONS'
   rewrite_cond:
    - "%{literal('%')}{REQUEST_METHOD} OPTIONS"
   rewrite_rule:
    - '^(.*)$ $1 [R=204,L]'

CORS Support

 
# The following Apache headers are configurable to support Cross-Origin Resource
# Sharing (CORS). 'Access-Control-Allow-Headers' and 'Access-Control-Allow-Methods'
# are added with default values (below) if 'Access-Control-Allow-Origin' is set
# - Access-Control-Allow-Origin
# - Access-Control-Allow-Headers
# - Access-Control-Allow-Methods
#
#  Defaults:
#profile::sde_instance::apache_ac_allow_origin: null
#profile::sde_instance::apache_ac_allow_headers: 'authorization'
#profile::sde_instance::apache_ac_allow_methods: 'POST, GET, HEAD'
profile::sde_instance::apache_ac_allow_origin: 'https://cors.example.com/'
profile::sde_instance::apache_ac_allow_headers: 'authorization'
profile::sde_instance::apache_ac_allow_methods: 'POST, GET, HEAD'

Email relay server

 
postfix::server::relayhost: mail.example.com

DNS servers and settings

 
classes:
  - '::resolv_conf'
resolvconf::nameservers:
  - '198.51.100.1'
  - '198.51.100.2'
resolvconf::searchpath:
  - 'subdomain1.example.com'
  - 'subdomain2.example.com'
resolvconf::domain:
  - 'example.com'

Custom cron jobs

 
cron::job:
  'first_job':
    command:  '/bin/echo "This is run as root every 12 hours"'
    hour:     '*/12'
    mode:     '0644'

  'second_job':
    command:  '/bin/echo "This is run as puppet every 12 hours"'
    hour:     '*/12'
    mode:     '0644'
    user:     puppet

  'once_a_day':
    command:  '/bin/echo "Today is $(/bin/date)"'
    interval: 'daily'
    environment:
      - 'MAILTO=root'
# Another variation of cron::job
cron::job::weekly:
  'weekly_job':
     command: '/usr/bin/echo "I run weekly"'
     minute:  0
     hour:    0
     weekday: 0

SD Elements VM architecture

VM SDE Architecture Current

results matching ""

    No results matching ""