In recent years, the usage of large language models (LLMs) has significantly increased. LLMs simplify processes and assist users with daily tasks but pose risks due to overconfident generated responses. This study explores methods to elicit confidence scores from LLM responses, aiming to bridge the gap between user trust and model reliability. Accurate confidence scores enable users to make informed decisions about trusting a model's response or seeking further verification.
The goal of this project is to develop a comprehensive pipeline for comparing confidence levels and strategies for assessing uncertainty in various LLMs. By enabling systematic analysis, we hope to understand how different strategies affect the confidence and reliability of model-generated results.
The motivation of this project is the increasing use of LLMs and the need to improve confidence in generated responses. Previous studies have shown that LLMs often display overconfidence, leading to errors. Insights on confidence score of generated responses is crucial for when LLMs are used in critical areas like healthcare or finance.
Investigating shows that verbalized confidence tend to be overconfident. GPT 3-5 outperforms Llama 2 and excells in arithemtic reasoning. Overall the calibration for the confidence predictions seem to be no better than random guessing when confidence is verbalized.
Daniel Perruchoud, Professor of Data Science
Clara Tran
Daniel Perruchoud, joseph.weibel@fhnw.ch
Joseph Weibel, daniel.perruchoud@fhnw.ch
