Exploring the power of text-rich graph representation learning

Zhu, Qi

Permalink

https://hdl.handle.net/2142/122007 Copy

Description

Title

Exploring the power of text-rich graph representation learning

Author(s)

Zhu, Qi

Issue Date

2023-11-27

Director of Research (if dissertation) or Advisor (if thesis)

Han, Jiawei

Doctoral Committee Chair(s)

Han, Jiawei

Committee Member(s)

• Tong, Hanghang
• Sundaram, Hari
• Perozzi, Bryan

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• graph neural networks
• text-rich graph
• representation learning
• language model
• generalization

Abstract

During my doctoral research, I observed that many applications related to graphs cannot be captured by existing simple network models. Lots of real networks exhibit massive text information on various type of objects, also known as text-rich or text-attributed networks. Traditional graph representation learning (e.g. network embedding) largely overlook the complex textual information within nodes and edges. Consequently, many recent graph mining algorithms employ supervised representation learning using neural networks on graph-structured data, specifically graph neural networks (GNNs). In this dissertation, I am motivated to bridge the gap between the frontier machine learning techniques and real-world problems on graph structured data (e.g. web-scale retrieval and classification). Two primary obstacles have hindered previous work on text-rich graphs. First, they assume sufficient and well-posed task-specific annotations. Second, an abundant computation budget is required for graph neural network training and inference, which is unrealistic considering large language model with billions parameters. From a practitioner's perspective, my research follows label efficient and parameter efficient principles to design graph representation learning algorithms for effective and efficient modeling of text-rich graphs. The first part of my dissertation work focuses on label-efficient representation learning, which reduces the need for extensive annotation across various tasks and graphs. Then I will introduce my recent efforts on seamless integration of language models and graph neural networks without excessive amount of training parameters. In contrast to existing work that develops powerful architectures for specific applications, my thesis overcomes the barriers to achieving flexibility and efficiency in general graph neural networks. Therefore, the proposed models can not only achieve superior performance in selected applications, but also enhance the capacity of any existing graph mining tasks. Together with all these efforts, the developed algorithms improve the adaptation of existing graph neural networks on more sophisticated text-rich networks and seek a more powerful representation learning paradigm in this area.

Graduation Semester

2023-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Qi Zhu

Owning Collections