University of Illinois Urbana-Champaign

Distributional graph: Connecting language towards a representation of knowledge and meaning

Mao, Shufan

Content Files
MAO-DISSERTATION-2023.pdf

Permalink

https://hdl.handle.net/2142/122021

Description

Title
Distributional graph: Connecting language towards a representation of knowledge and meaning
Author(s)
Mao, Shufan
Issue Date
2023-11-30
Director of Research (if dissertation) or Advisor (if thesis)
Willits, Jon A
Doctoral Committee Chair(s)
Willits, Jon A
Committee Member(s)
  • Fisher, Cynthia L
  • Federmeier, Kara D
  • Montag, Jessica L
  • Dell, Gary S
Department of Study
Psychology
Discipline
Psychology
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Semantic memory
  • Distributional semantics
  • Knowledge representation
  • Language comprehension
  • Semantic Network
Abstract
What is the nature of the relationship between Knowledge and Language How does knowledge representation interact with language use? The critical role of world knowledge (lexical semantic) in language processing has long been noticed by psycholinguists, but relatively less attention has been paid to incorporating rich linguistic information in modeling knowledge and semantic representations. Current distributional models do form semantic representations from linguistic input. Despite their success in representing relatively simple semantic relations, these models are less effective in extracting rich linguistic structures from corpus, resulting in limited capability in representing complex lexical dependencies, achieving challenging semantic tasks, and accounting for relatively involved semantic behaviors. In this dissertation, I develop a novel type of distributional model – Distributional Graph – that transforms raw linguistic input into graphical forms and connects the graphlets to build a semantic network. The model encodes distributional patterns of linguistic units by a graphical topology, so that linguistically expressed concepts can be evaluated by network metrics. In particular, I adopt a spreading activation algorithm that gives rise to a graded measure of semantic relatedness on the network. I show, with two groups of studies, that distributional graphs equipped with spreading activation may better represent complex lexical relationships, compared to existing distributional models. In particular, I show that (i) The graphical encoding of co-occurrence leads to effective representation of indirect semantic relations which facilitates generalizing word-word lexical dependencies, and (ii) The explicit encoding of constituent structures in Distributional Graph leads to effective representation of multi-way lexical dependencies and success in compositional generalization tasks. The modeling works are conducted on artificially generated corpora with controlled distributional constraints, leading to a clear mechanistic account for the formal capabilities of the model. Meanwhile, the modeling results have profound implications on theory of language cognition and knowledge development in human. Considerable amount of behavioral studies are needed to validate Distributional Graph as a model for human semantic memory, and the experimental works require scaling up the Distributional Graph approach with naturalistic linguistic input. For the long-term vision, by integrating multi-modal inputs and finer grammatical information, the more full-fledged distributional graphs may give rise to generation of novel concepts that are both grammatical and meaningful. Importantly, as distributional graphs are based on explainable computational mechanisms, such advance may contribute to more interpretable AIs, and further the understanding of knowledge development and innovation in humanity.
Graduation Semester
2023-12
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/122021
Copyright and License Information
Copyright 2023 Shufan Mao

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois