CONCEPTUALIZING THE SPACE: HOW NATURAL AND ARTIFICIAL COGNITIVE AGENTS USE TOPOLOGICAL SEMANTICS SCHEMES (BASED ON DESCRIPTIONS OF PAINTINGS FROM THE HERMITAGE COLLECTION)
DOI:
https://doi.org/10.5840/eps202562113Keywords:
conceptualization of space, topological semantics, artificial cognitive agent, natural cognitive agent, figure and background, large language modelsAbstract
The article is devoted to the description of the differences in the conceptualization of space observed in informants, large language models and computer vision models capable of generating a text describing what they “saw”. We use the concept of a cognitive agent and substantiate the distinction between “natural vs artificial cognitive agent”: the first is understood as a person, the second is an AI model capable of making decisions and performing tasks adequately in a given situation. The aim of the study is to compare the ways of understanding the location of an object in space in natural cognitive agents and artificial cognitive agents of two types: large language models and models created for Image to Text task. The main methods are the method of linguistic experiment and the method of semantic description based on the theory of topological semantics by L. Talmi. As an incentive material, six paintings from the collection of the State Hermitage Museum were used, divided into three groups: portraits, monofigure paintings on mythological or religious themes, and multifigure compositions. The participants of the experiments were: 63 informants (Mean age = 19.1, 48 females, 15 males), 5 LLMs, 6 Image to Text models based on computer vision technology and capable of generating descriptions of recognized images in English. Using the typology of configurational topological schemes and “figure – background” type schemes, we compared the ways of understanding space that the models rely on. As a result, we have formulated a number of conclusions, the most important of which is that natural cognitive agents differ from artificial cognitive agents in its ability to integrate the process of conceptualization of an object in space into other cognitive processes: entity recognition and categorization, attention mechanisms, awareness of cause-and-effect relationships. Artificial cognitive agents are only learning such integrativity and mutual coordination, for example, when generative models conceptualize those objects in which they are not sure, since these are products of hallucination, as objects with fuzzy boundaries, and Image to Text models combine into a single heterogeneous human object and the most striking original detail of its environment, because they “believe” that this is the most important thing for description tasks.
References
Витяев Е.Е., Гончаров С.С., Свириденко Д.И. О задачном подходе в искусственном интеллекте и когнитивных науках // Сибирский философский журнал. 2020. № 2. С. 5–29. https://doi.org/10.25205/2541-75172020-18-2-5-29
Калуцкая А.П. Моделирование информационного взаимодействия когнитивного агента с внешней средой на основе псевдофизических логик и обобщенных ограничений: автореф. … канд. тех. наук. М., 2010. 25 с.
Касавин И.Т. Эпистемический лидер в динамике культуры. Пример искусственного интеллекта // Цифровой ученый: лаборатория философа. 2024. Т. 7. № 1. С. 62–73. DOI: 10.32326/2618-9267-2024-7-1-62-73.
Кравченко А.В. Экологическая парадигма в исследованиях языка: смена познавательных установок // Лингвоэкология: проблемы и пути их решения: монография / Под ред. А.П. Сковородникова, Г.А. Копниной. Красноярск: Сиб. федер. ун-т, 2022. С. 5–27.
Поспелов Д.А. Ситуационное управление. Теория и практика. М.: Наука, 1986. 288 с.
Прасолов В.В. Наглядная топология. М., 2012. 112 с.
Рахилина Е.В. Основные идеи когнитивной семантики // Современная американская лингвистика: Фундаментальные направления. М., 2002. С. 370–388.
Скребцова Т.Г. Когнитивная лингвистика: классические теории, новые подходы. М.: ЯСК, 2018. 391 с.
Vityaev, E.E., Goncharov, S.S., Sviridenko, D.I. “O zadachnom podkhode v iskusstvennom intellekte i kognitivnykh naukakh” [On the Problem Approach in Artificial Intelligence and Cognitive Sciences], Sibirskii filosofskii zhurnal [Sibirskii filosofskii zhurnal], 2020, no. 2, pp. 5–29. https://doi.org/10.25205/2541-7517-2020-18-2-529. (In Russian)
Kalutskaya, A.P. Modelirovanie informatsionnogo vzaimodeistviya kognitivnogo agenta s vneshnei sredoi na osnove psevdofizicheskikh logik i obobshchennykh ogranichenii: avtoref. … kand. tekh. nauk [Modeling of Information Interaction of a Cognitive Agent with the External Environment Based on Pseudophysical Logics and Generalized Constraints: Abstract of the Candidate of Technical Sciences]. Мoscow, 2010. (In Russian)
Kasavin, I.T. “Jepistemicheskij lider v dinamike kul’tury. Primer iskusstvennogo intellekta” [Epistemic Leader in the Dynamics of Culture. Example of Artificial Intelligence], The Digital Scholar: Philosopher’s Lab, 2024, vol. 7, no. 1, pp. 62–73. https://doi.org/10.32326/2618-9267-2024-7-1-62-73. (In Russian)
Kravchenko, A.V. “Ekologicheskaya paradigma v issledovaniyakh yazyka: smena poznavatel’nykh ustanovok” [The Ecological Paradigm in Language Research: Changing Cognitive Attitudes], in: A.P. Skovorodnikov, G.A. Kopnina (eds.): Lingvoekologiya: problemy i puti ikh resheniya [Linguoecology: Problems and Solutions]. Krasnoyarsk: Sib. feder. un-t, 2022, pp. 5–27. (In Russian)
Pospelov, D.A. Situatsionnoe upravlenie. Teoriya i praktika [Situational Management. Theory and Practice]. Moscow: Nauka, 1986. (In Russian)
Prasolov, V.V. Naglyadnaya topologiya [Visual Topology]. Moscow, 2012, 112 pp. (In Russian)
Rakhilina, E.V. “Osnovnye idei kognitivnoi semantiki” [The Main Ideas of Cognitive Semantics], in: Sovremennaya amerikanskaya lingvistika: Fundamental’nye napravleniya [Modern American linguistics: Fundamental directions]. Moscow, 2002, pp. 370–388. (In Russian)
Skrebtsova, T.G. Kognitivnaya lingvistika: klassicheskie teorii, novye podkhody [Cognitive Linguistics: Classical Theories and New Approaches]. Moscow: YaSK, 2018. (In Russian)
Chemero, A. Radical Embodied Cognitive Science. MIT Press, 2011.
Filimon, F., Nelson, J.D., Hagler, D.J., & Sereno, M.I. “Human Cortical Representations for Reaching: Mirror Neurons for Execution, Observation, and Imagery”, NeuroImage, 2007, no. 37 (4), pp. 1315–1328. DOI: https://doi.org/10.1016/j.neuroimage.2007.06.008.
Ji, X., Elmoznino, E., Deane, G., Constant, A., Dumas, G., Lajoie, G., Simon, J., Bengio, Y. Sources of Richness and Ineffability for Phenomenally Conscious States // arXiv:2302.064032023. https://doi.org/10.48550/ arXiv.2302.064
Kravchenko, A.V. “Essential Properties of Language, or, Why Language Is Not a Code”, Language Sciences, 2007, vol. 29, no. 5, pp. 650–671. https://doi.org/10.1016/j.langsci.2007.01.004
Lillicrap, T., Santoro, A., Marris, L., Akerman, C.J., Hinton, G. “Backpropagation and the Brain”, Nat Rev Neurosci, 2020, no. 21 (6), pp. 335–346. DOI: 10.1038/s41583-020-0277-3.
Maturana, H.R., Varela, F.J. The Tree of Knowledge: The Biological Roots of Human Understanding. Boston: Shambhala Publications, 1987.
Nagel, T. “What Is It Like to Be a Bat?”, The Philosophical Review, 1974, no. 83 (4), pp. 435–450.
Russell, S., Norvig, P. Artificial Intelligence: A Modern Approach, Pearson, 4th Edition, 2020.
Schatten, M., Ðurić, B. O. and Peharda, T. “A Cognitive Agent for University Student Support”, 2021 IEEE Technology & Engineering Management Conference – Europe (TEMSCON-EUR), Dubrovnik, Croatia, pp. 1–6. DOI: https://doi.org/10.1109/TEMSCON-EUR52034.2021.9488627.
Talmy, L. Towards a Cognitive Semantics. Vol. 1. Concept Structuring Systems. MIT Press, 2000.
Talmy, L. Towards a Cognitive Semantics. Vol. 2. Typology and Process in Concept Structuring. MIT Press, 2000.
Talmy, L. The Targeting System of Language. MIT Press, 2018.
Touvron, H., Lavril, T., Izacard, G, Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N, Hambro, E., Azhar, F., Rodriguez, Au., Joulin, A., Grave, E., and Lample, G. “Llama: Open and Efficient Foundation Language Models”. ArXiv, abs/2302.13971, 2023. DOI: https://doi.org/ 10.48550/arXiv.2302.13971.
Xue, L., Shu, M., Awadalla, A., Wang, J., Yan, A., Purushwalkam, S., Zhou, H., Prabhu, V., Dai, Y., Ryoo, M.S., Kendre, Sh., Zhang, J.,
Qin, C., Zhang, Sh., Chen, Ch.-Ch., Yu, N., Tan, J., Manoj, T., Heinecke, Sh., Wang, H., Choi, Y., Schmidt, L., Chen, Z., Savarese, S., Niebles, J.-C., Xiong, C., Xu, R. xGen-MM (BLIP-3): A Family of Open Large Multimodal Models. arXiv:2408.08872v2. 2024.
