Observable responses and hidden states in recurrent neural networks to reason about cognitive aspects of language
DOI:
https://doi.org/10.5944/ap.22.1.43347Keywords:
Artificial Neural Networks, Recurrent Networks, LSTM, Hidden States, Surprisal, Language, Event-Related PotentialsAbstract
In order to study the psychological processes involved in language, Cognitive Science investigates the internal representations involved in understanding or producing language. It also postulates the operations that modify those representations given contextual constraints. Thus, context and representation interact to create meanings. Accordingly, there are different hypotheses about how the cognitive system produces language. Just as there are experimental methodologies for their study, different architectures of artificial neural networks make it possible to provide these hypotheses with a formal apparatus. In these models, the representations and operations involved are exhaustively characterized. Recurrent neural networks (RNNs) with LSTM mechanisms and Transformers stand out as particularly useful architectures for modeling the contextual sequentiality of language. This special issue gives us the opportunity to explain how to use their external expressions (outputs) as well as their internal representations (hidden states) to understand, in cognitive terms, the effect that changes of expectations have on different temporal markings of sentences. To do so, we illustrate such formalization using a Sequence-Sequence RNN with encoder and decoder and relate its measures with event-related potentials (ERPs) experiments on a nuclear issue in language: systematic compositionality.
Downloads
References
Anderson, J. R. (2005). Cognitive Psychology and its Implications. Macmillan.
Belinchón, M., Igoa, J. M. y Rivière, Á. (2009). Psicología del Lenguaje. Investigación y Teoría [Psychology of Language: Research and Theory]. Trotta.
Bickerton, D. (1995). Language and Human Behavior. University of Washington Press.
Boltzmann, L. (2012). Studien über das Gleichgewicht der lebendigen Kraft zwischen bewegten materiellen Punkten [Studies on the Equilibrium of Living Force Between Moving Material Points]. En F. Hasenöhrl (Ed.), Wissenschaftliche Abhandlungen (pp. 49–96). Cambridge University Press.
Bridle, J. (1989). Training Stochastic Model Recognition Algorithms as Networks Can Lead to Maximum Mutual Information Estimation of Parameters. In Proceedings of the 2nd International Conference on Neural Information Processing Systems (NIPS-89) (pp. 211–217). Morgan Kaufmann.
Busemeyer, J. R., Wang, Z., Townsend, J. T. y Eidels, A. (2015). The Oxford Handbook of Computational and Mathematical Psychology. Oxford University Press.
Carta, T., Romac, C., Wolf, T., Lamprier, S., Sigaud, O. y Oudeyer, P. Y. (2023, July). Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning. En International Conference on Machine Learning (pp. 3676–3713). PMLR.
Choi, H., Cho, K. y Bengio, Y. (2017). Context-Dependent Word Representation for Neural Machine Translation. Computer Speech & Language, 45, 149–160.
Chomsky, N. (1957). Syntactic Structures. Mouton.
Cong, Y., LaCroix, A. N. y Lee, J. (2024). Clinical Efficacy of Pre-Trained Large Language Models through the Lens of Aphasia. Scientific Reports, 14(1), Artículo 15573. https://doi.org/10.1038/s41598-024-66576-y
De Vega, M., Glenberg, A. y Graesser, A. (2012). Symbols and Embodiment: Debates on Meaning and Cognition. Oxford University Press.
Elman, J.L. (1990). Finding Structure in Time. Cognitive Science, 14(2), 179–211. https://doi.org/10.1207/s15516709cog1402_1
Farrell, S. y Lewandowsky, S. (2010). Computational Models as Aids to Better Reasoning in Psychology. Current Directions in Psychological Science, 19(5), 329–335. https://doi.org/10.1177/0963721410386677
Federmeier, K. D. y Kutas, M. (1999). Right Words and Left Words: Electrophysiological Evidence for Hemispheric Differences in Meaning Processing. Cognitive Brain Research, 8(3), 373–392. https://doi.org/10.1016/S0926-6410(99)00036-1
Fodor J. A. y Pylyshyn, Z. W. (1988). Connectionism and Cognitive Architecture: A Critical Analysis. Cognition, 28(1-2), 3–71. https://doi.org/10.1016/0010-0277(88)90031-5
Heine, B. y Kuteva, T. (2007). The genesis of Grammar: A Reconstruction. Oxford University Press.
Hernandez, E., Sen Sharma, A., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y. y Bau, D. (2023). Linearity of Relation Decoding in Transformer Language Models. ArXiv, https://doi.org/10.48550/arXiv.2308.09124
Hochreiter, S. y Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780.
Hubel, D. H. y Wiesel, T.N. (1968). Receptive Fields and Functional Architecture of Monkey Striate Cortex. Journal of Physiology, 195, 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455
Ivanova, A., Sathe, A., Lipkin, B., Kumar, U., Radkani, S., Clark, T., Kauf, C., Hu, J., Pramod, R., Grand, G., Paulun, V., Ryskina, M., Akyurek, E., Wilcox, E., Rashid, N., Choshen, L., Levy, R., Fedorenko, E., Tenenbaum, J. y Andreas, J. (2024). Elements of World Knowledge (EWOK): A cognition-inspired framework for evaluating basic world knowledge in language models. arXiv https://doi.org/10.48550/arXiv.2405.09605
Jackendoff, R. (1999). Possible Stages in the Evolution of the Language Capacity. Trends in Cognitive Sciences, 3(7), 272–279. https://doi.org/10.1016/S1364-6613(99)01333-9
Jackendoff, R. (1987). Consciousness and the Computacional Mind. MIT Press.
Jordan, M.I. (1997). Serial Order: A Parallel Distributed Processing Approach. Advances in Psychology, 121, 471–495. https://doi.org/10.1016/S0166-4115(97)80111-2
Jorge-Botana, G. (2024). Redes neuronales recurrentes y Transformers para modelos cognitivos del lenguaje [Recurrent Neural Networks and Transformers for Cognitive Language Models]. Ediciones Complutense.
Jurafsky, D. y Martin, J. H. (2023). Speech and Language Processing. An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Pearson.
Kaan, E. (1999). Syntax and Semantics? Trends in Cognitive Sciences, 3(9), Artículo 322. https://doi.org/10.1016/S1364-6613(99)01376-5
Kutas, M. y Federmeier, K.D. (2011). Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP). Annual Review of Psychology, 62(1), 621–647. https://doi.org/10.1146/annurev.psych.093008.131123
Kutas, M. y Hillyard, S. A. (1980). Reading Senseless Sentences: Brain potentials Reflect Semantic Incongruity. Science, 207(4427), 203–205. https://doi.org/10.1126/science.7350657
Lasnik, H. y Lidz, J. (2016). The Argument from the Poverty of the Stimulus. En I. Roberts (Ed.), The Oxford Handbook of Universal Grammar (pp.221–248). Oxford Academic.
Lindquist, K. A. (2021). Language and Emotion: Introduction to the Special Issue. Affective Science, 2(2), 91–98. https://doi.org/10.1007/s42761-021-00049-7
Liñán, J. L. (2009). Sistematicidad, productividad y composicionalidad: Una aproximación pragmatista [Sistematicity, Productivity, and Compositionality: A Pragmatistic Approach]. Revista de Filosofía, 34(1), 51–75.
McClelland, J. L. y Rumelhart, D.E. (1989). Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises. MIT press.
McClelland, J. L., Rumelhart, D. E. y PDP Research Group. (1987). Parallel Distributed Processing, Volume 2: Explorations in the Microstructure of Cognition: Psychological and Biological Models (Vol. 2). MIT press.
Nandi, A., Manning, C. D. y Murty, S. (2024). Sneaking Syntax into Transformer Language Models with Tree Regularization. arXiv preprint arXiv. https://doi.org/10.48550/arXiv.2411.18885
Neisser, U. (1967). Cognitive Psychology. Prentice-Hall.
Oh, B. D. y Schuler, W. (2023). Why Does Surprisal from Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times? Transactions of the Association for Computational Linguistics, 11, 336–350.
Pearl, L. (2022). Poverty of the Stimulus Without Tears. Language Learning and Development, 18(4), 415–454. https://doi.org/10.1080/15475441.2021.1981908
Piantadosi, S. T. (2023). Modern language models refute Chomsky’s approach to language. En E. Gibson y M. Poliak (Eds), From Fieldwork to Linguistic Theory: A Tribute to Dan Everett (pp.353–414). Language Science Press.
Pitt, D. (2022). Mental Representation. En E. N. Zalta y U. Nodelman (Eds.), The Stanford Encyclopedia of Philosophy (Fall 2022 Edition). https://plato.stanford.edu/archives/fall2022/entries/mental-representation/
Rabovsky, M., Hansen, S. S. y McClelland, J. L. (2018). Modelling the N400 Brain Potential as Change in a Probabilistic Representation of Meaning. Nature Human Behaviour, 2(9), 693–705. https://doi.org/10.1038/s41562-018-0406-4
Rabovsky, M. y McClelland, J. L. (2020). Quasi-Compositional Mapping from form to Meaning: A Neural Network-Based Approach to Capturing Neural Responses during human Language Comprehension. Philosophical Transactions of the Royal Society B, 375(1791), Artículo 20190313. https://doi.org/10.1098/rstb.2019.0313
Rayner, K. (Ed.). (2012). Eye Movements in Reading: Perceptual and Language Processes. Academic Press.
Rescorla, R. A. y Wagner, A. R. (1972). A Theory of Pavlovian Conditioning: Variations in the Effectiveness of Reinforcement and Nonreinforcement. En A. H. Black y W. F. Prokasy (Eds.), Classical Conditioning II (pp. 64–99). Appleton-Century-Crofts.
Rumelhart, D. E., McClelland, J. L. y PDP Research Group. (1986). Parallel Distributed Processing, Volume 1: Explorations in the Microstructure of Cognition. Foundations. MIT press.
Skinner, B. F. (1957). Verbal Behavior. Prentice-Hall.
Slaats, S. y Martin, A. E. (2023). What's Surprising about Surprisal. https://osf.io/7pvau/download/
Sterelny, K. (1990). The Representational Theory of Mind. Basil Blackwell.
Sun, R. (2023). The Cambridge Handbook of Computational Cognitive Sciences. Cambridge University Press.
Szabó, Z. G. (2001). Problems in Compositionality. Garland.
Szabó, Z. G. (2020). Compositionality. En E. N. Zalta y U. Nodelman (Eds.), Stanford Encyclopedia of Philosophy. The Metaphysics Research Lab. http://seop.illc.uva.nl/entries/compositionality/
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. y Polosukhin, I. (2017). Attention is all you need. En U. von Luxburg, I. Guyon y S. Bengio (Eds.), Proceedings of the 31st International Conference on Neural Information Processing Systems (pp. 6000–6010). Curran.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Faculty of Psychology. Applied Psychology Service (UNED)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Accion Psicologica is published under Creative Commons Attribution-Non Commercial (CC BY-NC). The opinions and contents of the articles published in Acción Psicológica are responsibility of the authors and do not compromise the scientific and political opinion of the journal. Authors are also responsible for providing copies of the raw data, ratings, and, in general, relevant experimental material to interested readers.
