Tailoring large language models for zero-shot relation extraction

Zhou, Sizhe

Permalink

https://hdl.handle.net/2142/129194 Copy

Description

Title

Tailoring large language models for zero-shot relation extraction

Author(s)

Zhou, Sizhe

Issue Date

2025-04-14

Director of Research (if dissertation) or Advisor (if thesis)

Han, Jiawei

Department of Study

Siebel School Comp & Data Sci

Discipline

Computer Science

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• relation extraction
• large language models
• small language models

Abstract

Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both costly and labor-intensive to obtain. Moreover, these models often struggle to adapt to novel or unseen relations. Few-shot learning, which aims to reduce annotation demands, typically provides incomplete and biased supervision for target relations, resulting in degraded and unstable performance. To accurately and explicitly describe relation semantics while minimizing the need for annotations, we explore the definition only zero-shot RE setting, in which only relation definitions expressed in natural language are used to train an RE model. We introduce REPaL, a framework comprising three stages: (1) We leverage large language models (LLMs) to generate initial seed instances from relation definitions and an unlabeled corpus. (2) We then fine-tune a bidirectional small language model (SLM) using these initial seeds to learn relational patterns specific to the target domain. (3) To expand pattern coverage and mitigate biases introduced by the initial seeds, we incorporate feedback derived from the SLM's predictions on the unlabeled corpus and the historical synthesis data. To accomplish this, we leverage the multi-turn conversational ability of LLMs to generate additional instances in follow-up dialogues informed by both the feedback and synthesis history. Our studies reveal that definition-oriented seed synthesis effectively enhances pattern coverage, whereas indiscriminately increasing the number of seeds leads to performance saturation. Experiments on two datasets demonstrate that REPaL significantly improves cost-effective zero-shot performance by substantial margins.

Graduation Semester

2025-05

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129194

Copyright and License Information

Copyright 2025 Sizhe Zhou

Owning Collections