User Mental Models as a Lens for Evaluating Explainable-AI System Outputs

Traditional evaluation methods for Explainable AI (XAI) often emphasize perceived clarity or task performance, overlooking the cognitive processes by which domain experts construct and refine their understanding. In this work, we propose a mental model-based evaluation framework that, grounded in cognitive science and Human-Computer Interaction (HCI), defines five analytical dimensions to assess how end users engage with XAI systems. Our goal is to support the design and evaluation of XAI interfaces that enable informed, trustworthy, and reflective decision-making in professional practice. We illustrate this perspective outlining how the mental model-based framework could be used to assess explanation effectiveness of Brain Age Predictor, an interactive interface for brain age prediction based on structural Magnetic Resonance Imaging (MRI) data.