Thigh Muscle Segmentation in MRI under Sparse and Inconsistent Labeling Conditions

Outlining the thigh muscles in an MRI scan by hand costs about five hours per subject. We measured how much of that work a neural network can take over, and how little annotation it actually needs.

Initial situation

Muscle volume and the amount of fat inside a muscle are used in orthopaedics and sports medicine to plan surgery, guide rehabilitation and estimate injury risk. To measure them, a specialist has to trace every single muscle in every slice of an MRI scan by hand. For four thigh muscles this takes roughly five hours per subject, and even two trained annotators do not fully agree with each other.

Deep learning can take this over, but the usual methods assume that every scan is completely annotated. Real clinical datasets look different. In the data of this project, from the GCAP study at the Balgrist Campus Zürich, some subjects are traced on one thigh only, some muscles appear in just a few subjects, and frequently only part of the slices carry a label.

The scans themselves are study data and are not reproduced on this page. The figures below are schematics and aggregated results.

Problem statement

These gaps are not simply less data, they push the training in the wrong direction. Standard loss functions treat every unlabelled pixel as background. A muscle that is plainly visible in the image but was never drawn is therefore taught to the model as the absence of muscle. The sparser the annotation, the more systematically the model learns something false.

Which countermeasures actually help against this, and how little annotation is enough, had not been compared under identical conditions. That is the gap this thesis addresses.

Approach

These gaps are not simply less data, they push the training in the wrong direction. Standard loss functions treat every unlabelled pixel as background. A muscle that is plainly visible in the image but was never drawn is therefore taught to the model as the absence of muscle. The sparser the annotation, the more systematically the model learns something false.

For each form of gap we varied how much annotation was left, from nearly complete down to very little, and trained a two dimensional U-Net at every level. The model was always scored on exactly that annotation it had never been shown. In total more than 100 configurations were trained across two cohorts. Architecture, data splits, hyperparameters and random seed were identical everywhere, so that the measured differences can only come from the annotation gap and from the countermeasure applied to it.

The experiment in five steps. The complete annotation is kept aside untouched and used only to score the model, so that every measured drop can be traced to the gap that was introduced on purpose.

The three annotation gaps, simulated under control. Blue is what the model was allowed to learn from, grey is what was withheld and later used for scoring. On the left a whole thigh is missing, in the middle one muscle is missing in half the group, on the right only part of the slices keep their label.

Results

The loss function decides before anything else. Simply excluding unlabelled pixels from the error is not enough. The background is then left unsupervised, and the smallest muscle spreads into it: its accuracy collapses from 0.899 to 0.014 while the other seven muscles remain unchanged. A marginal cross-entropy term taken from the literature repairs this without pretending that unlabelled pixels are background.

A countermeasure works when it matches the shape of the gap. If a thigh is missing, mirroring the annotated side lifts the withheld side from 0.535 to 0.826 Dice. If slices are missing, feeding the network the neighbouring slices helps. If a muscle is missing that other subjects do annotate, nothing helps, because the class is already anchored elsewhere in the group. Self training, which is often recommended in general terms, was neutral to harmful in all three cases.

Every countermeasure against the baseline of the same annotation budget. Orange is better, blue is worse, a dash means the study design does not define that combination. Mirroring gains a lot where a thigh is missing, extra slice context gains where slices are missing, and pseudo labelling loses in every single column.

The annotation requirement is far lower than expected. One bilaterally annotated pair of thighs, a quarter to a third of the group per muscle, and a fifth of the slices are sufficient. Translated into practice: instead of about 226 slices per thigh, roughly 45 traced by hand cost about 0.001 in accuracy, and even 23 slices cost only about 0.013. The accuracy reached lies in the range that two human annotators reach when compared with one another.

As an outside reference we ran a freely available pretrained model on the same data without any local training. It reached 0.868 against 0.895 under full local supervision, which puts a concrete price on annotating your own data.

Solution developed and its benefits

The outcome is not a clinical product but a recipe that a research group can follow when its annotation budget is limited. Decide first how the annotation may be incomplete and set the budget accordingly, then train with the combined loss and add the prior that matches the gap, and finally score only on annotation that was held back. If the result is below the level reported between two human annotators, annotate the next increment.

The benefit is that the annotation effort, by far the most expensive part of such a project, can be planned instead of estimated. For the group at the Balgrist Campus this means concretely that a new cohort no longer has to be traced slice by slice.

The recommended procedure, with the measured figures behind each step.

Key terms

• LanguagePython
• LibrariesMONAI (2D U-Net, metrics)
• Infrastructurei4DS Slurm cluster
• DataDixon MRI water channel, GCAP study, Balgrist Campus Zürich
• KeywordsMedical image segmentation, sparse annotation, partial supervision, U-Net, annotation budget, musculoskeletal MRI

Acknowledgements

The authors acknowledge the Swiss Center for Musculoskeletal Imaging, SCMI, Balgrist Campus AG, Zürich, for providing the MRI imaging data used in this study.

Customer

Balgrist Campus Zürich
The MRI scans and the associated annotations come from the GCAP study at the Balgrist Campus Zürich.
Company website

Team