University of Illinois Urbana-Champaign

Artificial Language Evolution on a Dynamical Interaction Network

Swarup, Samarth

Content Files
Artificial Language Evolution on a Dynamical Interaction Network.pdf

Permalink

https://hdl.handle.net/2142/11379

Description

Title
Artificial Language Evolution on a Dynamical Interaction Network
Author(s)
Swarup, Samarth
Issue Date
2007-09
Keyword(s)
computer science
Date of Ingest
2009-04-22T16:33:12Z
Abstract
"This dissertation studies the impact of a dynamical interaction network on the distributed learning of a common language. In recent years there has been much interest is developing algorithms for enabling populations of agents to converge upon a shared language, and in studying the role of the interaction network in this process. The focus so far has been on fixed networks, with various topologies, and simple algorithms, which do not provide a general framework for associating tasks in the environment with language. We try to overcome both these limitations in this work. We derive a new algorithm for generating realistic complex networks, called Noisy Preferential Attachment (NPA). This is a modification of preferential attachment that unifies it with the quasispecies model of molecular evolution. The growing network can now be seen as a process in which the links in the network are undergoing selection, replication, and mutation. We also demonstrate that by varying the mutation rate over time, we can reproduce features of growing networks in the real world. We then model a population of language learning agents on an interaction topology evolving according to NPA and demonstrate that under certain conditions they can converge very rapidly. However, we also note that they always converge to a maximally simple language. This leads us to introduce a method of relating language to task based on an analogy between the agents' hypothesis space and an information channel. Language is represented as a form-meaning association matrix and is learned alongside the neural network that is used to solve the task, by treating the hidden layer nodes as the meanings. We introduce a new ""language game"" which we call the Classification Game. We show that the population, through playing the classification game, converges to a representation which is simple, but not too simple, by balancing the pressures for learnability and functionality. This leads to a form of complexity regularization which corresponds to a search for at minima of the error surface in weight space. We demonstrate that the population can avoid overfitting through this process. The languages that emerge can be either holistic or compositional, depending on the nature of the task and the cognitive capacity of the agents. We then introduce temporal tasks and show that the same setup, using recurrent neural networks and form-meaning association matrices, can generate languages with strict symbol ordering, which is a rudimentary form of syntax. Finally, we bring together language and topology evolution and show that when the classification game is played on a topology evolving according to NPA, very rapid convergence can be achieved at the expense of a small increase in complexity of the solution. We also compare the convergence rates of several other topologies and show that NPA results in the fastest convergence. Regular and small world topologies show very slow convergence, due to the formation of communities which are locally converged but at odds with other communities."
Type of Resource
text
Genre of Resource
Technical Report
Permalink
http://hdl.handle.net/2142/11379
Copyright and License Information
You are granted permission for the non-commercial reproduction, distribution, display, and performance of this technical report in any format, BUT this permission is only for a period of 45 (forty-five) days from the most recent time that you verified that this technical report is still available from the University of Illinois at Urbana-Champaign Computer Science Department under terms that include this permission. All other rights are reserved by the author(s).

Owning Collections

Research and Tech Reports - Computer Science PRIMARY