ABM: attention-based message passing network for knowledge graph completion

Xu, Weikai

Permalink

https://hdl.handle.net/2142/117646 Copy

Description

Title

ABM: attention-based message passing network for knowledge graph completion

Author(s)

Xu, Weikai

Issue Date

2022-11-21

Director of Research (if dissertation) or Advisor (if thesis)

Tong, Hanghang

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Knowledge Graph Completion
• Attention
• Message Passing

Language

eng

Abstract

Knowledge graphs are ubiquitous and play an important role in many real-world applications, including recommender systems, question-answering, fact-checking, and so on. However, most of the knowledge graphs are incomplete which can hamper their practical usage. Fortunately, knowledge graph completion (KGC) can mitigate this problem by inferring missing edges in the knowledge graph according to the existing information. In this thesis, we propose a novel KGC method named Attention-Based Message passing (ABM) which focuses on predicting the relation between any two entities in a knowledge graph. The proposed ABM consists of three integral parts, including (1) context embedding, (2) structure embedding, and (3) path embedding. In the context embedding, the proposed ABM generalizes the existing message passing neural network to update the node embedding and the edge embedding to assimilate the knowledge of nodes’ neighbors, which captures the relative role information of the edge that we want to predict. In the structure embedding, the proposed method overcomes the shortcomings of the existing Graph Neural Network (GNN) method (i.e., most methods ignore the structural similarity between nodes.) by assigning different attention weights to different nodes while conducting the aggregation. Path embedding generates paths between any two entities and treats these paths as sequences. Then, the sequence can be used as the input of the Transformer to update the embedding of the knowledge graph to gather the global role of the missing edges. By utilizing these three mutually complementary strategies, the proposed ABM is able to capture both local and global information which in turn leads to a superb performance. Experiment results show that ABM outperforms baseline methods on a wide range of datasets.

Graduation Semester

2022-12

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/117646

Copyright and License Information

Copyright 2022 Weikai Xu

Owning Collections