Supported ecosystems and analysis methods¶
Manifest files¶
To find dependencies, CodeScoring primarily relies on parsing package manager manifest files. The system supports analysis of the following technologies:
Language | Package manager or build tool | File format |
---|---|---|
Java and Kotlin | Gradle, Maven | pom.xml ivy.xml maven-dependency-tree.txt gradle-dependency-tree.txt *.gradle < br/>*.gradle.kts gradle.lockfile |
JavaScript and TypeScript | npm, yarn | package.json package-lock.json npm-shrinkwrap.json yarn.lock pnpm-lock.yaml |
Python | pip, Poetry, Pipenv, Conda | setup.py Pipfile Pipfile.lock pyproject.toml poetry.lock requirements.txt requirements.pip requires.txt environment.yml meta.yml conda-lock.yml |
C and C++ | Conan | conanfile.txt conan.lock conanfile.py |
Go | Go Modules | go.mod go.sum |
PHP | Composer | composer.json composer.lock |
Ruby | RubyGems | Gemfile Gemfile.lock *.gemspec gems.locked |
C# | Nuget | *.nuspec packages.lock.json Project.json Project.lock.json packages.config paket.dependencies paket.lock *.csproj project.assets.json |
Objective-C and Swift | CocoaPods | Podfile Podfile.lock *.podspec |
Rust | Cargo | Cargo.lock Cargo.toml |
Scala | sbt | scala-dependency-tree.txt sbt-dependency-tree.txt |
The best result is achieved through a combination of a manifest file and a corresponding lock file (if it is provided by the package manager mechanism).
Resolution mechanism in the absence of a lock file¶
If there is no lock file the system will try to resolve transitive OSS dependencies itself:
- Maven
- for pom.xml and build.gradle format generation of
maven-dependency-tree
via the corresponding maven plugin - Maven version 3.8.3 and OpenJDK version 11 are used
- PyPi
- generation of
poetry.lock
using the Poetry package manager - Python version 3.8 is used
- N.P.M.
- generation of
yarn.lock
using the Yarn package manager - uses Node.js version 16
- Nuget
- for csproj format generation of
project.assets.json
using built-in nuget tools - uses .NET SDK version 8 -Packagist
- generation of
composer.lock
using the Composer package manager - PHP version 8 is used -Rubygems
- generation of
Gemfile.lock
using the Bundler package manager - uses Ruby version 3
Generation of lock files by the system does not produce the best results in every case, since it often depends on the environment.
Resolving dependencies in the build environment¶
Some package managers do not include transitive dependencies in their manifests by default. For high-quality compositional analysis when working with them, it is recommended to use a dependency resolution mechanism in the build environment.
When resolving dependencies in the environment, the agent checks for the absence of a lock file, independently launches the package manager or build tool, and generates a complete list of components taking into account the correct version of the build. The functionality is currently available for the following ecosystems:
- .NET
- Go
- Gradle
- Maven
- npm
- Poetry
- sbt
- yarn
- Conda
Mechanism for searching dependencies using hashes¶
The second dependency search mechanism implemented in CodeScoring is a hash search – a search for direct inclusion of libraries in project code. As part of this mechanism, all project files are hashed and these signatures are compared with known open source libraries.
Currently, hash searches occur for the following package manager indexes and the following file types:
- Maven
.jar
.war
.ear
-npm.min.js
- PyPI
.whl
.egg
- Nuget
.nupkg
Hashes of files whose size does not exceed 512 bytes are not sent to the cloud.
Build scanning for C and C++¶
If the Conan package manager and corresponding manifests are not used to build a C/C++ project, a special mode for analyzing the build process output can be used to get a list of used libraries.
In this mode, the Johnny console agent analyzes the build process using compiler flags and identifies the libraries used. Then, using the system cache, the location of the libraries and their source are determined.