Narrative comprehension through analogy: A study in cognitive modeling and narrative clustering

Abstract

As the field of natural language processing improves and finds its way into everyday use its current limitations and shortcomings become all the more apparent. The next generation of NLP systems will need to be able to handle tasks at a higher level, drawing together information beyond the lexical and across sentence boundaries. To address this need, research into the field of discourse understanding has emerged as a current hot topic with special attention being drawn to narrative comprehension. We explore cognitive modeling and the application of derived measures of analogy to tasks in the discourse/narrative domains. First, we present improvements to the LISA model, a state-of-the-art cognitive model of analogy, increasing the model’s flexibility and robustness, extending the model’s functionality to include a probabilistic measure of belief, and presenting an algorithm for automatically producing the model’s encoding. Finally we test the utility of narrative analogy as a feature for the Story Cloze Task. We find that narrative analogy is a poor feature on its own, but as part of a composite model with sentiment analysis, it outperforms the best task-given baselines but under-performs state-of-the-art. More importantly, through failure analysis we find that narrative analogy, as conceptualized by the field, is insufficient for such tasks, and researchers must first be able to determine when an analogy should be drawn since simply finding all potential analogies proves insufficient.