Explainable AI: Theoretical Models and Applications in Heterogeneous Multi-Attribute Data
General description of the project
These limitations mean that, when using AI in high-risk decision-making processes, the intervention and final validation of a healthcare professional is essential. n this regard, the European Artificial Intelligence Act reinforces this requirement by classifying medical applications as high-risk systems, establishing the obligation of continuous human supervision of algorithms. Such supervision not only seeks to mitigate risks, but also to guarantee and enhance optimal performance derived from effective collaboration between humans and AI systems.
This raises a central question: how can effective collaboration between healthcare personnel and artificial intelligence systems be consolidated if professionals lack sufficient information about the internal workings of these models? In this context, Explainable Artificial Intelligence (XAI) becomes relevant, a field of research that develops methods aimed at increasing the transparency and interpretability of models traditionally considered ‘black boxes’.
In its early stages, XAI was conceived primarily as a tool for developers to identify errors, biases and undesirable behaviour in predictions, contributing to the improvement and validation of models. However, its scope has gradually expanded to include end users who make decisions based on these predictions, particularly in high-impact contexts such as healthcare. In this context, XAI facilitates more effective collaboration with AI systems by providing understandable explanations of how they work and their results.
XAI is not limited to providing technical information on how models work, but requires adapting these explanations to the characteristics of the end user. This adaptation must consider factors such as the specific needs of the recipient, the language used, the most appropriate type of visualisation and the level of detail required to promote effective understanding and responsible use of the system.
Recognising that different healthcare personnel profiles require different types of explanations, at NAIR we address specific use cases and actively involve healthcare personnel from the initial stages of problem identification. This approach takes the form of user-centred design, in which healthcare professionals participate across all phases of solution development, from requirements definition to final validation.
Objectives
The project focuses on the design, development, and evaluation of an XAI framework specifically aimed at healthcare professionals. This framework aims to transform the traditional ‘black box’ of AI systems into an understandable, useful, and clinically relevant tool, allowing healthcare personnel to adequately gauge their level of confidence in the predictions and recommendations generated by these models.
To achieve this goal, a comprehensive methodological approach based on three complementary axes is adopted. First, a real user-centred design is applied, through the development of explanations aligned with existing clinical workflows and the creation of intuitive interfaces that respect the time and cognitive constraints of healthcare professionals.
Secondly, multimodal explanations are incorporated, combining different sources and formats of information in order to offer a richer and more robust understanding of the behaviour of AI models, while ensuring their technical reliability in critical clinical environments.
Finally, the framework integrates ethical and diversity considerations across the board, promoting the active mitigation of algorithmic biases and ensuring compliance with current European regulations on Artificial Intelligence.
Funding
Project funded by the Call for Grants for the recruitment of doctoral students by companies, research centres and technology centres: Industrial Doctorates 2024.
More projects
Calle del Sadar, s/n
31006 Pamplona (Navarra)
info@naircenter.eu
