Enhancing diversity in generative commonsense reasoning for explaining relationships between concepts
Liu, Chenzhengyi
Loading…
Permalink
https://hdl.handle.net/2142/120265
Description
Title
Enhancing diversity in generative commonsense reasoning for explaining relationships between concepts
Author(s)
Liu, Chenzhengyi
Issue Date
2023-04-16
Director of Research (if dissertation) or Advisor (if thesis)
Chang, Kevin Chen-Chuan
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Natural Language Processing
Commonsense Reasoning
Diverse Text Generation
Large Language Models
Mixture Of Experts
Cold Decoding
Language
eng
Abstract
The ability to reason based on common sense and knowledge of how things work is crucial for machines to navigate the world. However, large language models (LLMs) often lack explicit representations of relationships between concepts and events, making it challenging to interpret their reasoning processes. To overcome these challenges, in this paper, we propose DimonGen task, which aims to generate diverse sentences describing concept relationships in various everyday scenarios. To support this, we also create a new benchmark dataset for this task by extracting the existing ConceptNet and CommonGen dataset. To address the DimonGen task, we propose two complementary methods: MoREE, a two-stage method that utilizes external knowledge to generate diverse relationship sentences, and DC Decoding, a decoding framework that uses a global energy function to diversify the set of generations. Both methods are evaluated on the benchmark dataset and show significant improvements in the quality and diversity of generated sentences. The results suggest that these methods can generate diverse sentences that reflect relationships between concepts from multiple and varied perspectives. Our code and data for the DimmonGen task are available at https://github.com/liuchenzhengyi/DimonGen.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
