University of Illinois Urbana-Champaign

Introducing builder perspective in minecraft architect utterance generation

Athani, Amit A

Content Files
ATHANI-THESIS-2023.pdf

Permalink

https://hdl.handle.net/2142/120121

Description

Title
Introducing builder perspective in minecraft architect utterance generation
Author(s)
Athani, Amit A
Issue Date
2023-04-27
Director of Research (if dissertation) or Advisor (if thesis)
Hockenmaier, Julia
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
  • NLP
  • Natural Language Processing
  • CNN
  • ROI
Abstract
The task of Collaborative Dialogue in Minecraft involves developing interactive agents that can communicate with humans to collaboratively solve tasks in grounded scenarios. Previous attempts to solve this task have treated the tasks of Architect Utterance and Builder Action Prediction as separate problems, and independent models have been developed to solve each. However, a human architect would typically think about how to build the target structure before giving verbal instructions to the Builder. Therefore, in this work, we focus on the Architect Utterance generation task and explore how the Builder Action Prediction model can be used to improve utterance generation. We achieve this by using a customized builder that can view the Minecraft world through the Architect’s perspective and generate action sequences. These action sequences represent a simulation of what the Architect would do if they were acting as the Builder, allowing the Architect to think in terms of Builder actions. We leverage this Builder-For-Architect model to introduce action sequences and their effects on the world state into the Architect’s decision-making process.
Graduation Semester
2023-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/120121
Copyright and License Information
Copyright 2023 Amit Athani

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois