Reasoning beyond scale: Structured inference for small language models
Aakriti, -
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/129549
Description
Title
Reasoning beyond scale: Structured inference for small language models
Author(s)
Aakriti, -
Issue Date
2025-04-20
Director of Research (if dissertation) or Advisor (if thesis)
Han, Jiawei
Department of Study
Siebel School Comp & Data Sci
Discipline
Computer Science
Degree Granting Institution
University of Illinois Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Small Language Models
Knowledge Graphs
Question Answering
Claim Verification
Reasoning
Abstract
Recent progress in large language models (LLMs), such as GPT-4 [1], PaLM [2], and LLaMA [3], has substantially advanced the field of natural language processing (NLP), particularly in tasks requiring reasoning, such as multi-hop question answering (QA) and claim verification [4, 5]. Despite these achievements, such models require significant computational and financial resources, limiting their real-world accessibility [6, 7]. This has motivated a growing interest in small language models (SLMs), typically with fewer than 8 billion parameters, which offer more efficient alternatives. However, SLMs face major challenges when performing complex reasoning under long-context and distractor-rich scenarios, often exhibiting difficulties with factual consistency and multi-step inference [8, 9, 10].
This thesis investigates multi-hop reasoning in small models through the lens of structured knowledge integration, focusing on whether small models can remain grounded in relevant context while avoiding hallucination and distractor interference. Multi-hop reasoning requires the model to chain together intermediate facts and establish logical links across multiple documents or sentences [11]. A key question explored is whether explicit knowledge representations, such as knowledge graphs (KGs), can act as a scaffold to improve logical consistency, contextual grounding, and interpretability in reasoning tasks. The study further examines how SLMs manage cross-document coreference, semantic role tracking, and inference reliability when guided by structured signals.
This thesis attempts to identify the linguistic and representational bottlenecks that hinder SLMs from attaining robust reasoning, going beyond surface-level performance. It analyzes the conditions under which models succeed or fail to maintain coherence across reasoning chains, particularly when processing lengthy, noisy input. The thesis also explores how models respond to adversarial distractors and the extent to which structured inputs reduce hallucination rates [12, 13]. Overall, the research contributes to a broader understanding of how reasoning, factuality, and context grounding can be enabled in models – an essential step toward deploying capable and trustworthy NLP systems in real-world environments.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
