Lack of combinatorial productivity in language processing with simple recurrent networks

An astronomical set of sentences can be produced in natural language by combining relatively simple sentence structures with a human-size lexicon. These sentences are within the range of human language performance. Here, we investigate the ability of simple recurrent networks (SRNs) to handle such combinatorial productivity. We successfully trained SRNs to process sentences formed by combining sentence structures with different groups of words. Then, we tested the networks with test sentences in which words from different training sentences were combined. The networks failed to process these sentences, even though the sentence structures remained the same and all words appeared on the same syntactic positions as in the training sentences. In these combination cases, the networks produced work

• Type: Journal
• Year: 2004
• Where: CONNECTION
• Authors: Frank van der Velde, Gwendid T. van der Voort van der Kleij, Marc de Kamps
• Keywords: CONNECTION 2004, Human Language Performance, Sentence Structures, Training Sentences