Troubleshooting

The sections below covers common issues that may arise on a system and how to address them.

System

The table below provides guidance for resolving certain system-related issues that may arise.

Symptom Reason Verification Next steps

Application shuts down regularly

Nginx TLS config and logrotate conflict

  • Check if logrotate is enabled.

  • Check if the Nginx TLS certificate requires a password on startup.

Choose one:

  • Disable logrotate

  • Remove password from Nginx TLS certificate

Error running sde reprovision

No outbound access to updates.sdelements.com or anvil.sdelements.com

Run in offline mode: sde reprovision --offline-mode

Nginx TLS/SSL error

Check if Nginx certificate specifies the server’s fully qualified domain name (FQDN)

Update certificate for server’s FQDN. Re-run reprovision.

Malformed /etc/hosts file

Check if localhost is defined in /etc/hosts

Update the hosts file. Re-run reprovision.

Limited diskspace

Lingering PostgreSQL temp files

Check the pgsql_tmp directory for large temp files

Remove the temp files

Upgrades

The table below provides steps to resolve upgrade issues.

Symptom Reason Verification Next steps

Connection problem to updates.sdelements.com or anvil.sdelements.com

No outbound network access from the server to updates.sdelements.com or anvil.sdelements.com

Run commands:

ping updates.sdelements.com
ping anvil.sdelements.com

  • If ping is successful then network access is confirmed: a firewall exception may be required.

  • Request a firewall exception from your IT team or configure an HTTP proxy.

Network access limited by local firewall

Run command:

iptables -nL

Network access limited by firewall

Run commands:

tracepath updates.sdelements.com/443
tracepath anvil.sdelements.com/443

  • If tracepath return successfully then updates server should be accessible.

  • Request a firewall exception from your IT team or [Configure an HTTP proxy].

Update fails

An unexpected issue encountered in the updater.

Upgrade the SD Elements updater and try the upgrade again.

Check /docs/sde/log/deploy.log for errors and reach out to SD Elements Support with the details.

ALM, Scanner & LDAP integration

The table below provides troubleshooting guidance for issues regarding integration with an ALM, scanner or LDAP server.

Symptom Reason Verification Next steps

Invalid server or server unreachable

Connection details are invalid

Verify the connection details are correct.

Update the connection with the correct information and retry.

No network access to server.host.name

Run command:

ping server.host.name

  • If ping is successful then network access is confirmed: a firewall exception may only be required.

  • If ping fails then DNS cannot resolve the IP of the other server or the server is unreachable.

    • Request a firewall exception from your IT team or configure an HTTP proxy.

    • Update the system so that server.host.name resolves to an IP.

Network access limited by local firewall

Run command:

iptables -nL

Network access limited by firewall

Run command:

tracepath server.host.name/port

  • If tracepath returns successfully then server should be accessible. Try the integration again.

  • If tracepath fails, request a firewall exception from your IT team or configure an HTTP proxy.

Network access is limited by a transparent proxy

A transparent proxy may be at issue if outbound network access is already confirmed for other external systems but not for this server.

Transparent proxies allow companies to control traffic without burdening systems with configuration. Contact the IT team for details and request a whitelist to the desired endpoint, if needed.

Network access is limited by an IPS (Intrusion Prevention System)

Check with IT team if traffic to server.host.name:port is filtered by an IPS rule.

Investigate the cause for rule being triggered. Request an exception for the specific server.

TLS/SSL validation error

HTTPS connection fails certificate validation

Verify the TLS/SSL connection

Add the certificate to the system

Connection relies on a proxy that rewrites TLS/SSL certificates or its own certificate is untrusted.

Check that the proxy’s certificate or its CA certificate is trusted by the system: Validate TLS/SSL connection to the proxy.

Add the HTTPS proxy certificate to the system

TLS/SSL connection error

HTTPS connection to server requires Server Name Indication (SNI) support

Contact SD Elements product team to prioritize SNI support

HTTPS connection fails due to cipher or protocol error

Validate a TLS/SSL connection.

Investigate whether the target server supports minimum TLS security settings. For example, SSLv3 is not supported.

Jobs stuck or not working

Celery needs a restart

Application shows jobs stuck in status "Waiting…​" for more than 10 minutes

On the SSH console run:

sde supervisor restart all

Inconsistent connection

DNS issue

If connection to a server fails intermittently, the problem may be due to a flaky DNS lookup.

Add an entry to /etc/hosts for the server and retry.

Job unexpectedly fails

Integration issue or unsupported server

Examine celery logs for the error.

Timeout reached

Examine celery logs for a SoftTimeLimitExceeded() or similar error.

Extend the job timeout

Integration server error

Examine celery logs.

  • If the error is of the form "HTTP 5xx" it is a server or gateway error.

    • Contact the integration server owner or network team to investigate.

    • Verify that the integration server is patched and up-to-date.

    • If the problem persists and no resolution is found then open a support case about the error

LDAP SSO error

Use the in-app troubleshooting mechanism.

  • Use ldapsearch if the web interface cannot surface the underlying issue.

Missing LDAP configuration

Examine /docs/sde/local_settings for LDAP configuration

Update local_settings with the appropriate configuration. Restart Apache.

Capture detailed integration logs

Diagnosing integration issues is aided greatly by detailed logs between SD Elements and the other server. Follow the steps below to collect verbose logs for a problematic integration.

Prerequisites:

  • SSH credentials for sde_admin or sudo access.

  • Application Super User access.

Steps:

  1. Login to the SD Elements web application as a Super User.

  2. Open the problematic integration connection.

  3. Enable option Debug Mode.

  4. Access the SD Elements server SSH console as sde_admin.

  5. Run command:

    sde manage_django run_session_capture_server | tee /docs/sde/log/debug_integration.log

  6. Run the problematic integration until it completes.

  7. Disable option Debug Mode on the integration.

  8. Cancel the run_session_capture_server command by entering Ctrl-C.

The full integration logs are captured in file debug_integration.log.

 Credentials are stored as cleartext in the log file. Remove the file from the system as soon as possible.

Upgrade the integration components

An updated version of the integration components sdetools may be provided to address unexpected or environmental issues. Follow the steps below to apply an update to your system.

Prerequisites:

  • SSH credentials for sde_admin or sudo access.

  • Outbound HTTPS access to updates.sdelements.com

  • The version number from SD Elements Support for a new release of sdetools.

Steps:

  1. Access the SD Elements server SSH console as sde_admin.

  2. Run the commands below, assuming that the new version of sdetools is 5.14.2:

    sde pip live install sdetools==5.14.2
sde supervisor restart all
sde apache restart

Modify the application job timeout

Integrations with ALM systems, scanning tools, and LDAP servers are run by the Celery process. By default these jobs time out after 10 minutes.

To modify the job timeout to 15 minutes, for example, follow the steps below.

Prerequisites:

  • SSH credentials for sde_admin

Steps:

  1. Access the SD Elements server SSH console as sde_admin.

  2. Update file /docs/sde/local_settings set:

    CELERY_JOB_TASK_SOFT_TIME_LIMIT = 15 * 60

  3. Save the file and run:

    sde supervisor restart all
sde apache restart

New jobs are configured to expire after 15 minutes.

SAML

The table below provides troubleshooting guidance for issues arising from SAML integration.

Symptom Reason Verification Next steps

IdP is posting to the wrong SP URL.

The SP URL has the wrong domain.

Check the SP URL on the SAML configuration page

Update the domain on the Domain Settings page. Re-enable SAML.

The user is not redirected to the correct Login URL

The login URL is heavily cached or not specified.

Confirm the Login URL if IdP initiated is configured. Restart the web application:

sde apache restart

Technical issues error page

IdP is configured for endpoint /sso/saml2/acs

Check the IdP configuration for the SP URL.

Ensure the IdP configuration posts to URL with a trailing slash. For example, /sso/saml2/acs/

The IdP has specified an unexpected entity ID.

Check the SAML logs for a "Not for me" error

Make sure the IdP has configured the SP for entity ID https://<your-sd-elements-domain>/sso/saml2/metadata/

An entity ID mismatch between the SAML assertion and the IdP metadata.

Check the SAML logs for a "Missing key" error

Ensure the entity ID in the IdP metadata is correct.

System time may be out of sync with the time on the SAML token.

Check the SAML logs for a "Can’t use it yet" error

Ensure NTP settings are correct and working.

The certificate inside the SAML request may be invalid. Verify that it is in unix format and does not contain DOS line endings.

Check the SAML logs for a "Strange beginning of PEM File" error

You can also validate it using the following command:

openssl x509 -in filename.crt -text -noout

Unexpected attribute mapping. Verify the value of SAML_ATTRIBUTE_MAPPING in the /docs/sde/local_settings file.

Check SAML logs for "The user is none" or "Could not find saml_user_value" error

Update the SAML_ATTRIBUTE_MAPPING for expected input.

Decryption of the encrypted assertion has failed. The IdP metadata is missing or the IdP used an incorrect public encryption key for the SP.

Check the SAML logs for "NoneType' object has no attribute 'authn_statement'" error

  • Regenerate the SP metadata and re-import into the IdP.

  • Reinstall the IdP metadata.

Open a support case using the support portal

If the system is acting abnormally, reach out to SD Elements Support with details about the issue. You will receive a response within 3 business hours.

Prerequisites:
Steps:

  1. Open the support portal https://support.sdelements.com

  2. Click on Submit a request

  3. Enter your details:

    • Email address: A way for the support team to respond to you directly.

    • Subject: A brief description of the issue

    • Description: The issue you experienced as well as steps to reproduce. include your system versions.

    • Attachments: Screenshots, log files, or other information helpful to better understand and diagnose your issue.

  4. Click Submit.

A new support ticket is created for your issue. You will receive an email soon as confirmation.

Open a support case using email

If the system is acting abnormally, reach out to SD Elements Support with details about the issue. You will receive a response within 3 business hours.

Prerequisites:
Steps:

  1. Compose a new email to support@sdelements.com

    • Subject: A brief description of the issue

    • Body: The issue you experienced as well as steps to reproduce. include your system versions.

    • Attachments: Screenshots, log files, or other information helpful to better understand and diagnose your issue.

  2. Click Send.

A new support ticket is automatically created for your issue. You will receive an email soon as confirmation.

results matching ""

    No results matching ""