Reliability-Aware LLM Reasoning: Handling Uncertainty in Robot Perception

Robots operating in human environments must often make decisions based on perceptual information that
is uncertain, incomplete, or ambiguous. This paper proposes a reliability-aware reasoning framework that enables large
language models (LLMs) to account for perceptual uncertainty when selecting actions in human-robot interaction scenarios.
The environment is represented as a structured scene composed of detected objects enriched with confidence estimates, attribute reliability, and spatial uncertainty information. Using this representation, the LLM evaluates candidate objects through a reliability scoring mechanism that integrates multiple sources of perceptual evidence and supports uncertainty-aware decision making.The proposed approach is evaluated using perception episodes with controlled levels of uncertainty and compared with a baseline LLM-based matching strategy that ignores perceptual
reliability. Experimental results show that incorporating uncertainty-aware reasoning substantially improves decision
robustness under medium and high uncertainty conditions while reducing safety-critical behaviors caused by overconfident
autonomous decisions.