Scan a Repository

SD Elements has introduced Scan a Repository, a feature that allows users to connect their Git repositories, scan them for relevant data, and map these data points to survey answers, while also helping to generate diagram components.

Purpose

The main purpose of this feature is to streamline the SD Elements project onboarding process. This is done by scanning an external repository to answer the SD Elements survey.

Canvas controls

This user guide will provide detailed instructions on how to connect and use the “Scan a Repository” feature within SD Elements. It covers everything from setting up connections, to scanning a repository and mapping the results to answers within a draft project survey.

Enable Scan a Repository

Before Scan a Repository becomes available to use within a project, administrators must enable the feature. In order to use this feature an administrator will need turn it on using either the UI or API by selecting the Scan a Repository feature flag. The administrator will need to have the "Turn On Feature Flags" permission to do this.

Prerequisites:

  • The user has the permission Organization → Manage features.

Steps:

  1. Navigate to the System (gear icon) tab.

  2. Select Features.

  3. Under Team Onboarding, Check the Scan a Repository checkbox.

    Canvas controls

  4. Click Save.

The scan a repository feature will be activated and available for use in this SD Elements instance, both via UI and API.

Scan a Repository Permissions

Once the feature has been enabled within the SD Elements instance, only users with specific permissions will be able to manage this functionality. Users will require the permissions to manage a project and the survey.

Prerequisites:

  • The user has the permission Project role→Project Management→Edit project survey.

Git Access

If the end user has the appropriate permissions, they will be able to create a user-based connection to their desired Git platform, which can then be used for any given project.

Supported Git Connections

Scan a Repository currently supports connections to GitHub and GitLab. See Limitations & Restrictions for more information.

GitHub Access

Users can connect to GitHub either via OAuth through the SD Elements Platform or using a Personal Access Token (PAT) via the API. These options are available for GitHub Cloud (https://github.com).

In order to connect to GitHub on Premise, please refer to the below section "Connecting via API".

GitLab Access

Users can connect to GitLab either via OAuth through the SD Elements Platform or using an access token via the API. These options are available for GitLab Cloud (https://gitlab.com).

In order to connect to GitLab on Premise, please refer to the below section "Connecting via API".

Creating a Repository Connection

Before a user initiates a repository scan, they must authenticate and configure the connection details.

Connecting Git Account within a Project

  1. Users should select "Scan a Repository" from the Getting Started Card.

  2. Next, users should choose the platform they want to connect to (e.g., GitHub).

  3. After selecting a platform, users will be prompted to authenticate and grant SD Elements permission to clone and read repositories.

    Canvas controls

  4. Users can remove this connection via the Scan A Repository page at any point. Doing so will require the user to re-authenticate if they wish to run a subsequent scan or before running any new scans within other SD Elements projects.

    Canvas controls

Running a Scan within a Project

After establishing a connection for a user, they will be redirected to set up the repository and branch for the project.

  1. Users will select a repository, which will then display the branches within the selected repository.

  2. Once users have selected the repository and branch they can click the Scan button to initiate the scan, which will then redirect users to the survey.

    Canvas Control

  3. While a scan is running, the project remains locked.

    Canvas controls

  4. Once the scan is completed, users will need to click on the Unlock & Continue button to unlock the project and update the survey with the answers found during the scan. After doing this a user can make any manual modifications to the survey that they wish before continuing on.

    Canvas controls

To learn more about setting up a connection and trigger a scan via API, please refer to:

Limitations & Restrictions

  • The API workflow must be used for for self-hosted GitHub or GitLab intances.

  • If you are using a SaaS environment with repositories hosted on GitHub.com or GitLab.com, you can take advantage of the Team Onboarding OAuth User Experience (UX) to streamline access and integration.

Project Scans:

  • Projects are locked during scans to ensure that only one scan occurs within a project at a time.

  • The repository size for scans must be under 2 GB when scanning within SD Elements (via UX).

Connection Restrictions:

  • The GitHub connection name cannot dynamically reflect the actual repository name. It remains as a static "GitHub" for identification.

  • Organizations using custom URL domains for SD Elements (such as: abc.nameofCorp.com or app.custom_name.us) cannot use the UX flow and must exclusively configure connections via the API.

Additional limitations for OSD Customers:

Onsite Scan a Repository

We offer an alternative solution that allows customers to run repository scans directly within their own local environment. This approach ensures full control and security over your scanning processes while leveraging the scanning functionality that is built into SD Elements.

This combination provides flexibility, enhanced data privacy, and streamlined workflows tailored to meet your unique requirements.

Download On-Site Script

Users have the ability to download the binary file containing the repository scanning script, as well provide binary files for various OS systems.

Running On-Site Script

Supported Platforms

To access the binary files, sign in to the Customer Portal and navigate to the Knowledge Base article available at Downloading Onsite Repository Scanning Script. This article includes both the binary files and detailed instructions on how to run the on-site scanning script.

Upload On-Site Script Results

  1. Once the scan is complete, the results JSON file can be directly uploaded to SD Elements via the UX.

  2. Navigate to the Upload a Scan page using the menu option in the project survey page.

    Canvas controls

  3. Upload the generated results file to the Upload JSON File field. Note that the file should be in JSON format and less than 5 MB

    Canvas controls

  4. Click the Upload button to upload the file.

  5. Once the file is uploaded, the users will be redirected to the project survey page where the project is locked.

  6. After the file processed successfully, users will need to click on the Unlock & Continue button to unlock the project and update the survey with the answers found during the scan. After doing this a user can make any manual modifications to the survey that they wish before continuing on.

    Canvas controls

  7. If there are any validation errors during the upload, they will be displayed on the screen. Below are examples of common validation errors:

    • File Format Error: This error occurs if the uploaded file is not in JSON format.

      Canvas controls

    • File Size Error: This error occurs if the uploaded file exceeds the 5 MB size limit.

    Canvas controls

Upload On-Site Script Results via API

For Users who are interested in submitting the JSON results over via the API, please refer to API Docs for reference.

Network Restrictions

Network restrictions can be configured to mitigate Server-Side Request Forgery attacks, which help to prevent exposing access to internal resources if you do not have network level protection.

By default, the Scan a Repository feature has restricted network access to local or private internal IPs and certain ports during Git repository scanning to secure the SD Elements environment from SSRF attacks.

Enabling network restrictions will have the following effects:

  • Block access to IPv6 addresses

  • Block access to multicast, link-local, loopback, IETF reserved, and private network addresses

  • Block access to restricted hosts (default: localhost)

  • Block access to restricted ports (default: 22, 23)

During Scan a Repository feature, any requests to connect to or scan Git repositories hosted at a restricted address directly or via a redirect will fail.

The set of restricted hosts and ports can be configured. In the deployment setup, add or modify the following lines to the desired values:

extraEnvVars:
  - name: SDE_TOB_IGNORE_INTERNAL_NETWORK_RESTRICTION
    value: "False"
  - name: SDE_TOB_NETWORK_RESTRICTION_DOMAIN_BLOCK_LIST
    value: "example.com"
  - name: SDE_TOB_NETWORK_RESTRICTION_PORT_BLOCK_LIST
    value: "22,23"

results matching ""

    No results matching ""