Automated Feature Catalogue Extraction from Long-Lived Software

When software stays in service for decades, it is doing something right. This project uses locally hosted AI models to read 1.3 million lines of proven C code and distil it into a clear catalogue of everything the system can do - fully locally, without a single line of code leaving the machine.

Initial situation

Some software is so successful that it stays in service for decades. Our industry partner, who wishes to stay anonymous, operates such a system: around 1.3 million lines of C code that have grown with the business over many years and reliably support its daily operations. A system of this maturity embodies an enormous amount of domain knowledge - and much of that knowledge lives in the code itself rather than in up-to-date documentation, as is natural for any product that has been continuously developed for so long. As the company plans the next generation of its software, it wants to make sure that every capability the current system has earned over the years is carried forward. Compiling that inventory by hand would mean months of work for experienced developers and domain experts.

Problem statement / Project goal

The goal of this project was to automate that exploration: a pipeline that reads the source code of a mature C/GTK system and produces a human-readable feature catalogue - a structured list of everything the system can do for its users, grouped into thematic areas, with references back to the source files. Because the source code is valuable intellectual property, everything has to run locally: no cloud services, no external APIs. The pipeline also has to cope with German function and variable names, and it has to scale from small test projects to millions of lines of code.

Solution developed and its benefits

We developed a fully local extraction pipeline that turns raw C source code into a readable feature catalogue in eight stages: chunking the code into individual functions, embedding them with a multilingual AI model that understands German identifiers, structuring the embeddings into candidate features, labelling each feature with a locally hosted language model, consolidating near-duplicates, grouping related features into thematic areas, labelling those groups, and rendering the final report. Two strategies were compared: a bottom-up approach that clusters functions within files, and a top-down approach that follows the system's entry points - the places where a user actually interacts with the software.

• a complete, structured feature catalogue generated directly from source code - no existing documentation required
• fully local processing: the source code never leaves the company's infrastructure
• features described from the user's perspective, understandable for product owners, not just developers
• every feature cites the source files it was extracted from, so results can be verified
• technical helper functions are separated into an appendix, keeping the main catalogue readable
• a caching system that makes runs on large codebases resumable and repeatable
• an automated evaluation tool that scores generated catalogues against a known reference

On the reference codebase, the best configuration recovered roughly 90% of the specified features with very few invented entries. Applied to the partner's production system, the pipeline processed the full 1.3 million lines of code and produced a structured catalogue of several thousand features - a solid foundation for designing the next generation of the software, ensuring that the strengths built up over decades are carried into the future.

Testimonial
«As the client for the IP5 project, I was impressed by the professionalism and dedication of the team. Communication was clear and collaborative throughout, and the final result exceeded our expectations in both quality and execution. It was a pleasure working with such a motivated and capable team, and I would gladly work with them again.»
industry partner

Key terms

• Feature catalogueA structured list of everything a system can do for its users, with a reference to the source files each entry came from.
• Reverse engineeringRecovering what a system does from its source code, when no up-to-date documentation exists.
• Legacy systemSoftware that has been in productive use and under continuous development for decades.
• C/GTKThe C programming language combined with GTK, an open-source toolkit for graphical user interfaces.
• ChunkingSplitting the source code into individual functions by parsing it into a syntax tree (Tree-sitter).
• EmbeddingA numeric vector representing a piece of code or text, positioned so that similar meaning lands close together.
• ClusteringGrouping items by the similarity of their embeddings, without knowing the groups in advance (k-means, Ward, HDBSCAN).
• Leaf featureThe smallest catalogue entry: a few related functions that together provide one capability.
• Feature area / sub-groupThe upper catalogue levels, formed by clustering the generated feature descriptions rather than the code.
• Bottom-up vs. top-downThe two structuring strategies compared: grouping functions within a file, or starting from the system's user-facing entry points.
• Large language model (LLM)An AI model that reads and writes text and code; here it names and describes each feature.
• Prompt engineeringDesigning the instructions given to the language model so its output stays consistent and faithful to the code.
• Local / offline inferenceRunning the AI models on the company's own hardware, without any cloud service or external API.
• HallucinationA plausible-sounding catalogue entry describing a capability the code does not actually have.
• Coverage and F1The evaluation metrics: how much of the known functionality is captured, and how well entries match the reference one-to-one.
• Reference codebaseA purpose-built C project with 71 known features, used as ground truth for automatic scoring.
• Caching / resumabilityStoring each stage's result under a fingerprint of its inputs, so long runs continue instead of starting over.
• GPU / CUDA / vLLMThe infrastructure for local inference: models run on graphics processors, served efficiently by vLLM on larger GPUs.

Customer

This project was carried out in cooperation with a Swiss industry partner who wishes to remain anonymous.

Team