Cognitive Architecture: Review

The N400 event-related brain potential is elicited by each word in a sentence and offers an important window into the mechanisms of real-time language comprehension. Since the 1980s, studies investigating the N400 have expanded our understanding of how bottom-up linguistic inputs interact with top-down contextual constraints. More recently, a growing body of computational modeling research has aimed to formalize theoretical accounts of the N400 to better understand the neural and functional basis of this component. Here, we provide a comprehensive review of this literature. We discuss “word-level” models that focus on the N400’s sensitivity to lexical factors and simple priming manipulations, as well as more recent sentence-level models that explain its sensitivity to broader context. We discuss each model’s insights and limitations in relation to a set of cognitive and biological constraints that have informed our understanding of language comprehension and the N400 over the past few decades. We then review a novel computational model of the N400 that is based on the principles of predictive coding, which can accurately simulate both word-level and sentence-level phenomena. In this predictive coding account, the N400 is conceptualized as the magnitude of lexico-semantic prediction error produced by incoming words during the process of inferring their meaning. Finally, we highlight important directions for future research, including a discussion of how these computational models can be expanded to explain language-related ERP effects outside the N400 time window, and variation in N400 modulation across different populations.

Since the early 2000s, several ERP studies have challenged the assumption that we always use syntactic contextual information to influence semantic processing of incoming words, as reflected by the N400 component. One approach for explaining these findings is to posit distinct semantic and syntactic processing mechanisms, each with distinct time courses. While this approach can explain specific datasets, it cannot account for the wider body of findings. I propose an alternative explanation: a dynamic generative framework in which our goal is to infer the underlying event that best explains the set of inputs encountered at any given time. Within this framework, combinations of semantic and syntactic cues with varying reliabilities are used as evidence to weight probabilistic hypotheses about this event. I further argue that the computational principles of this framework can be extended to understand how we infer situation models during discourse comprehension, and intended messages during spoken communication.

We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher level representations to predictively pre-activate lower level representations, and whether we ’commit’ in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioral and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher level inferences to predictively pre-activate information at multiple lower representational levels. We also suggest that the degree and level of predictive pre-activation might be a function of the expected utility of prediction, which, in turn, may depend on comprehenders’ goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.

Language and thought dysfunction are central to the schizophrenia syndrome. They are evident in the major symptoms of psychosis itself, particularly as disorganized language output (positive thought disorder) and auditory verbal hallucinations (AVHs), and they also manifest as abnormalities in both high-level semantic and contextual processing and low-level perception. However, the literatures characterizing these abnormalities have largely been separate and have sometimes provided mutually exclusive accounts of aberrant language in schizophrenia. In this review, we propose that recent generative probabilistic frameworks of language processing can provide crucial insights that link these four lines of research. We first outline neural and cognitive evidence that real-time language comprehension and production normally involve internal generative circuits that propagate probabilistic predictions to perceptual cortices - predictions that are incrementally updated based on prediction error signals as new inputs are encountered. We then explain how disruptions to these circuits may compromise communicative abilities in schizophrenia by reducing the efficiency and robustness of both high-level language processing and low-level speech perception. We also argue that such disruptions may contribute to the phenomenology of thought-disordered speech and false perceptual inferences in the language system (i.e., AVHs). This perspective suggests a number of productive avenues for future research that may elucidate not only the mechanisms of language abnormalities in schizophrenia, but also promising directions for cognitive rehabilitation.

Disturbances of thought and language are fundamental to schizophrenia. Cognitive behavioral and electrophysiological research has implicated problems in two different neurocognitive mechanisms: abnormalities in the structure and function of semantic memory, and abnormalities in combining and integrating words together to build up sentence and discourse context. This review discusses recent electrophysiological evidence suggesting that these two deficits are not completely distinct, but rather that language impairment in schizophrenia results from a dysfunctional interaction between these systems in an effort to build up higher-order meaning. Moreover, although language abnormalities are more pronounced in patients with positive thought disorder, they manifest themselves in all patients when increased demands are placed on the comprehension system. Further investigation of language dysfunction may also provide insights into other aspects of psychotic thought.

This overview outlines findings of cognitive and neurocognitive studies on comprehension of verbal, pictorial, and video stimuli in healthy participants and patients with schizophrenia. We present evidence for a distinction between two complementary neurocognitive streams of conceptual analysis during comprehension. In familiar situations, adequate understanding of events may be achieved by mapping the perceived information on the associative and similarity-based connections between concepts in semantic memory - a process reflected by an N400 waveform of event-related electrophysiological potentials (ERPs). However, in less conventional contexts, a more flexible mechanism may be needed. We suggest that this alternative processing stream, reflected by a P600 ERP waveform, may use discrete, rule-like goal-related requirements of real-world actions to comprehend relationships between perceived people, objects, and actions. This neurocognitive model of comprehension is used as a basis in discussing studies in schizophrenia. These studies suggest an imbalanced engagement of the two conceptual streams in schizophrenia, whereby patients may rely on the associative and similarity-based networks in semantic memory even when it would be more adaptive to recruit mechanisms that draw upon goal-related requirements. Finally, we consider the roles that these conceptual mechanisms may play in real-life behavior, and the consequences that their dysfunction may have for disorganized behavior and inability to plan actions to achieve behavioral goals in schizophrenia.

This is the first of two articles that discuss higher-order language and semantic processing in schizophrenia. This article reviews clinical characterizations of language output and the phenomenon of positive thought disorder, as well as more principled characterizations of language output in schizophrenia. It also gives an overview of evidence for the predominant theory of language dysfunction in schizophrenia: that it arises from abnormalities in (a) semantic memory and/or (b) working memory and executive function. The companion article (Part 2) focuses on the study of language in schizophrenia using online psycholinguistic methods and considers how the study of schizophrenia may inform our understanding of normal language processing.

This is the second of two articles that discuss higher-order language and semantic processing in schizophrenia. The companion article (Part 1) gives an introduction to language dysfunction in schizophrenia patients. This article reviews a selection of psycholinguistic studies which suggest that sentence-level abnormalities in schizophrenia may stem from a relative overdependence on semantic associative relationships at the expense of building higher-order meaning. Language disturbances in schizophrenia may be best conceptualized as arising from an imbalance of activity across two streams of processing, one drawing upon semantic relationships within semantic memory and the other involving the use of combinatorial mechanisms to build propositional meaning. I will also discuss some of the ways in which the study of schizophrenia may offer new insights into the cognitive and neural architecture of the normal language system.

Clinically, patients with schizophrenia show prominent abnormalities at the discourse level, with production characterized by tangential and illogical relationships between ideas and unclear references. Despite these clinical manifestations, most studies of language in schizophrenia have focused on semantic relationships between single words and the build-up of meaning within single-clause sentences. The present paper discusses the few studies that have gone beyond clause boundaries to fully understand language impairments in schizophrenia. We also give an overview of a relevant literature that considers the neurocognitive mechanisms by which coherence links are established across clauses in healthy adults, providing a framework that may guide future research in this area.