Scan a Repository
SD Elements has introduced Scan a Repository, a feature that allows users to connect their Git repositories, scan them for relevant data, and map these data points to survey answers, while also helping to generate diagram components.
Purpose
The main purpose of this feature is to streamline the SD Elements project onboarding process. This is done by scanning an external repository to answer the SD Elements survey.
This user guide will provide detailed instructions on how to connect and use the “Scan a Repository” feature within SD Elements. It covers everything from setting up connections, to scanning a repository and mapping the results to answers within a draft project survey.
Enable Scan a Repository
Before Scan a Repository becomes available to use within a project, administrators must enable the feature. In order to use this feature an administrator will need turn it on using either the UI or API by selecting the Scan a Repository feature flag. The administrator will need to have the "Turn On Feature Flags" permission to do this.
-
The user has the permission Organization → Manage features.
-
Navigate to the System (gear icon) tab.
-
Select Features.
-
Under Team Onboarding, Check the Scan a Repository checkbox.
-
Click Save.
The scan a repository feature will be activated and available for use in this SD Elements instance, both via UI and API.
Scan a Repository Permissions
Once the feature has been enabled within the SD Elements instance, only users with specific permissions will be able to manage this functionality. Users will require the permissions to manage a project and the survey.
-
The user has the permission Project role→Project Management→Edit project survey.
Git Access
If the end user has the appropriate permissions, they will be able to create a user-based connection to their desired Git platform, which can then be used for any given project.
Supported Git Connections
Scan a Repository currently supports connections to GitHub and GitLab. See Limitations & Restrictions for more information.
GitHub Access
Users can connect to GitHub either via OAuth through the SD Elements Platform or using a Personal Access Token (PAT) via the API. These options are available for GitHub Cloud (https://github.com).
In order to connect to GitHub on Premise, please refer to the below section "Connecting via API".
GitLab Access
Users can connect to GitLab either via OAuth through the SD Elements Platform or using an access token via the API. These options are available for GitLab Cloud (https://gitlab.com).
In order to connect to GitLab on Premise, please refer to the below section "Connecting via API".
Creating a Repository Connection
Before a user initiates a repository scan, they must authenticate and configure the connection details.
Connecting Git Account within a Project
-
Users should select "Scan a Repository" from the Getting Started Card.
-
Next, users should choose the platform they want to connect to (e.g., GitHub).
-
After selecting a platform, users will be prompted to authenticate and grant SD Elements permission to clone and read repositories.
-
Users can remove this connection via the Scan A Repository page at any point. Doing so will require the user to re-authenticate if they wish to run a subsequent scan or before running any new scans within other SD Elements projects.
Running a Scan within a Project
After establishing a connection for a user, they will be redirected to set up the repository and branch for the project.
-
Users will select a repository, which will then display the branches within the selected repository.
-
Once users have selected the repository and branch they can click the Scan button to initiate the scan, which will then redirect users to the survey.
-
While a scan is running, the project remains locked.
-
Once the scan is completed, users will need to click on the Unlock & Continue button to unlock the project and update the survey with the answers found during the scan. After doing this a user can make any manual modifications to the survey that they wish before continuing on.
To learn more about setting up a connection and trigger a scan via API, please refer to:
Limitations & Restrictions
-
The API workflow must be used for for self-hosted GitHub or GitLab intances.
-
If you are using a SaaS environment with repositories hosted on GitHub.com or GitLab.com, you can take advantage of the Team Onboarding OAuth User Experience (UX) to streamline access and integration.
Project Scans:
-
Projects are locked during scans to ensure that only one scan occurs within a project at a time.
-
The repository size for scans must be under 2 GB when scanning within SD Elements (via UX).
Connection Restrictions:
-
The GitHub connection name cannot dynamically reflect the actual repository name. It remains as a static "GitHub" for identification.
-
Organizations using custom URL domains for SD Elements (such as: abc.nameofCorp.com or app.custom_name.us) cannot use the UX flow and must exclusively configure connections via the API.
Additional limitations for OSD Customers:
-
The API workflow to scan a repository is only supported.
Onsite Scan a Repository
We offer an alternative solution that allows customers to run repository scans directly within their own local environment. This approach ensures full control and security over your scanning processes while leveraging the scanning functionality that is built into SD Elements.
This combination provides flexibility, enhanced data privacy, and streamlined workflows tailored to meet your unique requirements.
Download On-Site Script
Users have the ability to download the binary file containing the repository scanning script, as well provide binary files for various OS systems.
Running On-Site Script
Supported Platforms
-
macOS
-
Linux
-
Windows (via Windows Subsystem for Linux (WSL))
-
For Windows users, the Linux binary file can be utilized through WSL. Learn more about WSL and installation steps.
-
To access the binary files, sign in to the Customer Portal and navigate to the Knowledge Base article available at Downloading Onsite Repository Scanning Script. This article includes both the binary files and detailed instructions on how to run the on-site scanning script.
Upload On-Site Script Results
-
Once the scan is complete, the results JSON file can be directly uploaded to SD Elements via the UX.
-
Navigate to the Upload a Scan page using the menu option in the project survey page.
-
Upload the generated results file to the Upload JSON File field. Note that the file should be in JSON format and less than 5 MB
-
Click the Upload button to upload the file.
-
Once the file is uploaded, the users will be redirected to the project survey page where the project is locked.
-
After the file processed successfully, users will need to click on the Unlock & Continue button to unlock the project and update the survey with the answers found during the scan. After doing this a user can make any manual modifications to the survey that they wish before continuing on.
-
If there are any validation errors during the upload, they will be displayed on the screen. Below are examples of common validation errors:
-
File Format Error: This error occurs if the uploaded file is not in JSON format.
-
File Size Error: This error occurs if the uploaded file exceeds the 5 MB size limit.
-
Upload On-Site Script Results via API
For Users who are interested in submitting the JSON results over via the API, please refer to API Docs for reference.