METHOD FOR VERIFYING THE BALANCE OF MENTAL MODELS OF AN INTELLIGENT SYSTEM’S DECISION
DOI:
https://doi.org/10.20998/2079-0023.2026.01.13Keywords:
explanation, artificial intelligence system, explainable artificial intelligence, dependencies, mental model, verificationAbstract
The subject of the study is the process of verifying mental models of an intelligent system’s decision. The aim of the work is to develop an approach for assessing the balance of explanations of intelligent systems’ decisions with respect to their negative and positive aspects. In accordance with this aim, the following main tasks are addressed: to develop an approach to assessing the balance of explanations of intelligent systems’ decisions based on the proportional representation of negative and positive characteristics in an explanation; to develop a general method for verifying the balance of mental models of an intelligent system’s decision that takes into account the structural and weighted coverage of the set of negative aspects by the decision’s mental models; to carry out an experimental evaluation of the proposed method using a set of user reviews of the operation of a recommender system that contain information about mental models of the proposed decisions. An approach to assessing the balance of explanations of intelligent systems’ decisions that accounts for negative aspects is proposed. The approach involves constructing a reference weighted set of essential negative aspects of a decision, extracting negative elements of the explanation, computing indicators of structural and weighted coverage, and assessing the proportionality of the presentation of negative information. This approach provides a quantitative estimate of the extent to which an explanation reflects the limitations and potential negative consequences of applying a decision in relation to its positive properties. A method for verifying the balance of mental models of an intelligent system’s decision is proposed. The method includes the stages of constructing a reference set of negative aspects of a decision based on the analysis of user reviews, extracting the negative component of the decision’s mental models, computing indicators of structural and weighted coverage, assessing the proportionality and relevance of the presentation of negative aspects, and forming an integral measure of the balance of mental models. The method enables refinement of a mental model taking into account shortcomings of the practical application of a decision by the user that arise due to incompleteness of the models. Experimental evaluation of the method based on a set of user reviews has shown that using reviews as a source of information about users’ mental models makes it possible to construct a reference set of negative aspects of an intelligent system’s decision that reflects usage problems and risks important to users.
References
Goyal P., Ferrara E. Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems, 2018, vol. 151, pp. 78–94. DOI: 10.1016/j.knosys.2018.03.022.
Brown D. E. Introduction to data mining for medical informatics. Clinics in Laboratory Medicine, 2008, vol. 28, no. 1, pp. 17–26. DOI: 10.1016/j.cll.2007.10.005.
Guidotti R., Monreale A., Ruggieri S., Turini F., Giannotti F., Pedreschi D. A survey of methods for explaining black box models. ACM Computing Surveys, 2019, vol. 51, no. 5, art. 93. DOI: 10.1145/3236009.
Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 2019, vol. 267, pp. 1–38. DOI: 10.1016/j.artint.2018.07.007.
Doshi-Velez F., Kortz M., Budish R. et al. Accountability of AI under the law: The role of explanation. arXiv preprint, 2017, arXiv:1711.01134.
Ribeiro M. T., Singh S., Guestrin C. “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, 2016, pp. 1135–1144. DOI: 10.1145/2939672.2939778.
Zhang Y., Chen X. Explainable recommendation: A survey and new perspectives. Foundations and Trends in Information Retrieval, 2020, vol. 14, no. 1, pp. 1–101. DOI: 10.1561/1500000066.
Tintarev N., Masthoff J. Explaining recommendations: Design and evaluation. In: Recommender Systems Handbook, 2nd ed. Boston, Springer, 2015, pp. 353–382. DOI: 10.1007/978-1-4899-7637-6_10.
Lundberg S. M., Lee S.-I. A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems 30 (NIPS 2017). Long Beach, 2017, pp. 4765–4774.
Samek W., Montavon G., Vedaldi A., Hansen L. K., Müller K.-R., eds. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham, Springer, 2019. 433 p. DOI: 10.1007/978-3-030-28954-6.
Kulesza T., Stumpf S., Wong W.-K. et al. Explanatory debugging: Supporting end-user debugging of machine-learned programs. In: Proceedings of the 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Pittsburgh, 2011, pp. 41– 48. DOI: 10.1109/VLHCC.2011.6070395.
Eiband M., Schneider H., Bilandzic M., Fazekas-Con Z., Haug M., Hussmann H. Bringing transparency design into practice: User interfaces for explainable AI. In: Proceedings of the 2018 Conference on Human Factors in Computing Systems (CHI’18 Workshop on Explainable Smart Systems). Montreal, 2018.
Garcez A. d’Avila, Besold T. R., De Raedt L. et al. Neural-symbolic learning and reasoning: A survey and interpretation. Neurocomputing, 2019, vol. 339, pp. 3–13. DOI: 10.1016/j.neucom.2019.01.034.
Besold T. R., d’Avila Garcez A., Bader S. et al. Neural-symbolic learning and reasoning: Contributions and challenges. In: Proceedings of the AAAI Spring Symposium on Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches. Stanford, 2017.
Zhang L., Wang S., Liu B. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2018, vol. 8, no. 4, e1253. DOI: 10.1002/widm.1253.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
