GitInsight - Knowledge Graph-based analysis of software development processes
GitInsight stores information generated during software development in a semantic knowledge graph, enabling complex analysis such as which team member is best suited to work on a new issue.
Initial situation
Modern software development processes generate numerous artefacts, such as issues, commits, pull requests, test cases and releases. Consistently linking this data serves as the basis for in-depth analyses to reveal hidden connections and dependencies. Understanding the complex evolution of software projects becomes increasingly difficult.
Problem statement / Project goal
In practice, however, these artefacts are usually located in different systems (Git, Jira, CI/CD, test management) and are rarely integrated in a uniform manner. As a result, a large part of the necessary traceability is maintained manually and many analyses (e.g. of causes or responsibilities) are carried out without automated support. However, in heavily regulated industries such as medical technology, aviation and the automotive industry, standards and approval procedures often require complete traceability from requirements through implementation to testing. The goal of this project is to address this fragmentation by developing GitInsight. A system designed to extract, structure and link these data sources into a semantic knowledge graph.
Impression 1
The figure illustrates data that could be stored by GitInsight. Through querying one can traverse the nodes connecting entities and exploring information that was previously hard to aggregate and analyze. For example one could find all classes that were modified in commits made by author "david". The query would specify all commits containing "david" as the value for its author attribute and from there follow the references to reach the classes.
Impression 2
Query interface to interact with GitInsight. The figure displays a query that retrieves all fully qualified class names.
Impression 3
Schema of the entities present in GitInsight.
Solution developed and its benefits
To achieve this, a system facilitating extraction from different data sources was designed. The approach focuses on modularity to enable the integration of future data sources and utilizes technologies with temporal capabilities to model the complex evolution of software projects. The project integrates three concrete data sources:
- Version control (Git): Commits, diffs and tags
- Collaborative development platform (GitLab): Issues, merge requests and pipelines
- Static code analysis: Classes and methods
The primary result of this work is the implementation of the GitInsight application. The system successfully integrates the targeted data sources, creating a multi-layered representation of the software development process. It enables users to perform complex, cross-domain queries that were previously difficult to aggregate manually. For instance, the system allows for the traversal of nodes to link specific developer activities e.g. commits by a specific author directly to structural code changes e.g. modified classes, revealing hidden connections and dependencies within a project.
Key terms
- Knowledge GraphStructural representation of data, highlighting relationships between entities
- ScalaProgramming language used to develop GitInsight
- DatomicDatabase used to store knowledge graph
- DatalogQuery language used to perform queries on the graph
- SootUpLibrary used to perform static code analysis on the JVM