Patronus: multi-modal sensing, analytics, and localization assistance for heterogeneous working environments

Tian, Beitong

Permalink

https://hdl.handle.net/2142/129844 Copy

Description

Title

Patronus: multi-modal sensing, analytics, and localization assistance for heterogeneous working environments

Author(s)

Tian, Beitong

Issue Date

2025-07-08

Director of Research (if dissertation) or Advisor (if thesis)

Nahrstedt, Klara

Doctoral Committee Chair(s)

Nahrstedt, Klara

Committee Member(s)

• Caesar, Matthew
• Soltanaghai, Elahe
• Shenoy, Prashant

Department of Study

Siebel School Comp & Data Sci

Discipline

Computer Science

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Internet of Things
• Smart glasses
• Multimodal sensing
• Condition monitoring
• Environmental monitoring
• Visual localization
• Intelligent environments

Abstract

Today’s “Jarvis-like” AI assistants excel at generic, consumer-oriented tasks, yet they remain ill-suited for heterogeneous working environments—dynamic laboratories and industrial facilities where hands-on professionals must juggle safety-critical processes, rapidly changing context, and a deluge of multimodal data. This thesis argues that effective assistance in these settings hinges on the tight co-design of three pillars: (i) a low-cost, scalable, and evolvable sensing infrastructure, (ii) a trustworthy real-time analytics pipeline, and (iii) an accurate, practical localization layer that enables context-aware humandata interaction. In this thesis, we introduce Patronus, a modular framework composed of five interoperable systems that collectively satisfy these requirements. SENSELET++ deploys a plug-and-play sensor network and anomaly analytics for scalable environmental monitoring. MachineStethoscope enables on-device, unsupervised health monitoring for legacy rotating machinery. WeldMon fuses heterogeneous signals and introduces synthetic fault augmentation to improve failure prediction in ultrasonic welding. GaugeTracker digitizes analog gauges entirely on low-cost IoT hardware, leveraging multiple vision and vision language models for robust transcription. Finally, AnyLoc provides energy-efficient visual localization that operates under low-resolution and low-light conditions in cluttered indoor scenes. Together, these systems power MAINTGlasses, a hands-free smart-glasses interface that delivers spatially relevant insights to professionals in real time. Deployments across cleanrooms, nanofabrication labs, and server rooms demonstrate that Patronus fosters safer, faster, and more efficient workflows. By unifying sensing, analytics, and localization, this work charts a practical path toward intelligent environments that actively understand and support complex human work.

Graduation Semester

2025-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129844

Copyright and License Information

Copyright © 2025 Beitong Tian. All rights reserved.

Owning Collections