Interactive Annotation of Wrist Fracture CTs
We developed an interactive segmentation model to mark and separate 3D fracture segments directly from CT scans. By adapting state-of-the-art foundation models to our case and integrating them into an application for 3D viewing, we created a workflow that speeds up and simplifies annotation while delivering the highest annotation quality to date.
Initial situation
Wrist fractures are among the most common injuries treated in hand surgery. For initial diagnosis, X-ray images of the wrist are usually sufficient. However, if there is no clear diagnosis or when surgical intervention is planned, computed tomography (CT) is used to generate detailed three-dimensional images of the wrist bones. These 3D images are crucial for accurate diagnosis and surgical planning, as wrist fractures and subsequent complications resulting from surgery can significantly affect hand function and overall quality of life. To enable surgeons to understand fractures more quickly, a detailed 3D model of the fractures is crucial. Such a model can facilitate faster diagnosis, support more effective surgical planning and ultimately improve surgical outcomes. In recent research, conducted in collaboration between FHNW and AINEAS GmbH, automatic segmentation methods have been investigated to create such models. However, these methods are not yet sufficiently robust for clinical application, particularly in the case of wrist fractures, which often present a wide variety of fracture patterns as well as small, complex fragments and structures.
Problem statement
A major limitation for developing reliable models is the lack of annotated training data. Creating high-quality manual annotations requires expert knowledge and is a slow, resource-intensive process. AINEAS GmbH, which creates those ground truth annotations, needs on average 106 minutes and as much as 280 minutes to annotate a single patient's wrist. Furthermore, the large variability in fracture patterns makes it difficult for models to generalise effectively across different cases. The segmentation of fine anatomical structures adds another layer of complexity, as even small inaccuracies can impact clinical decision-making.
Project goal
The objective of this work is to design and implement an interactive, intelligent tool aimed at accelerating the annotation of wrist CT scans. To achieve this, a dedicated extension for the 3D Slicer software will be developed, allowing expert users to generate an automated proposal for a complete annotation by placing clicks or “scribbles” on individual fragments.
Initial CT scan
High-resolution CT imaging of the wrist provides a detailed 3D view of the bone structures, serving as the starting point for the segmentation process.
Annotation Process
Using our interactive segmentation tools, fracture fragments are precisely marked in the CT images with minimal effort, aided by model predictions based on user input and refined over multiple iterations, with adaptations made through additional inputs to improve each prediction.
Final Result
The completed 3D segmentation separates each fracture fragment into its own layer, enabling clear visualization and analysis for diagnosis or research, and possibly serving as ground truth for future work on automatic segmentation.
Results
We developed and integrated an interactive segmentation model for wrist fracture CTs into a dedicated 3D Slicer extension. Starting from an evaluation of five foundation models, we identified nnInteractive as the most promising and fine-tuned it for our use case using a dynamic training pipeline that simulates user interactions.
The resulting extension offers the following features:- Integration of the fine-tuned interactive model
- Local refinement strategy that preserves correct regions during corrections
- Flagging system to mark fractures as complete and prevent oversegmentation
- Loading of predefined fracture sets with ready-to-use empty segmentation layers
- Functionality to flexibly add specific fragment types, such as ENK fragments, as needed
- Shortcuts to facilitate viewing and handling of annotations with many fragments
Our best model achieved DSC 0.870 and NSD 0.926 on an independent test set. Expert feedback confirmed that the tool is accurate, responsive, and easy to use, enabling faster and more precise annotation.
Key terms
- Medical Image Analysis Processing and analysis of CT scans to extract detailed anatomical information for clinical and research purposes.
- Wrist Fractures Complex injuries to the wrist bones, often requiring high-resolution imaging and precise segmentation for diagnosis and treatment planning.
- 3D Slicer Open-source software platform for medical image visualization, analysis, and custom extension development.
- Segmentation The process of dividing medical images into meaningful regions, such as individual bone fragments in a fracture.
- Annotation Creation of labeled image data by marking relevant structures, used to train and evaluate AI models.
- Interactive Segmentation A human-in-the-loop approach where users guide the segmentation process through targeted inputs for improved accuracy and efficiency.
Customer
AINEAS GmbH
Dr. med. Marco Keller
Theaterplatz 8
5400 Baden