Energy-efficient spiking neural network-based visual place recognition and reinforcement learning in adversarial scenarios

Akcal, Ugur

Permalink

https://hdl.handle.net/2142/129736 Copy

Description

Title

Energy-efficient spiking neural network-based visual place recognition and reinforcement learning in adversarial scenarios

Author(s)

Akcal, Ugur

Issue Date

2025-04-30

Director of Research (if dissertation) or Advisor (if thesis)

Chowdhary, Girish

Doctoral Committee Chair(s)

Tran, Huy

Committee Member(s)

• Ornik, Melkior
• Gillette, Rhanor

Department of Study

Aerospace Engineering

Discipline

Aerospace Engineering

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Spiking Neural Networks
• Reinforcement Learning
• Localization
• Visual Place Recognition
• Multi-Agent Systems
• Robotics
• Artificial Intelligence

Abstract

Spiking neural networks (SNNs) have attracted significant attention as an emerging third-generation artificial intelligence (AI) technology due to their potential for remarkable computational efficiency when deployed on neuromorphic hardware. Research has shown that SNNs can achieve energy efficiencies several orders of magnitude greater than their conventional artificial neural network (ANN) counterparts. Although there are numerous outstanding demonstrations of ANNs, their deployment on platforms with limited computational resources is quite restricted. In this context, SNNs offer considerable potential for expanding the deployment range of AI-based robotics solutions, driving a growing and sustained interest in SNN research. However, training state-of-the-art (SOTA) SNNs tailored for robotics problems is often intractable, and they typically demonstrate poor real-time performance. This motivates the current work to develop SNNs with tractable training that can yield better performance than existing SOTA SNNs in two domains: 1) Visual Place Recognition (VPR) and 2) Reinforcement Learning (RL) in adversarial scenarios. VPR is the ability to recognize locations in a physical environment based solely on visual inputs—a challenging task due to perceptual aliasing, viewpoint and appearance variations, and the complexity of dynamic scenes. To address the shortcomings of existing SNN approaches, we developed an end-to-end convolutional SNN model for VPR that leverages backpropagation for tractable training. During training, we employ rate-based approximations of leaky integrate-and-fire (LIF) neurons, which are then replaced with spiking LIF neurons during inference. The proposed method significantly outperforms existing SOTA SNNs on challenging datasets such as Nordland and Oxford RobotCar, achieving $78.6\%$ precision at $100\%$ recall on the Nordland dataset (compared to $73.0\%$ from the current SOTA) and $45.7\%$ on the Oxford RobotCar dataset (compared to $20.2\%$ from the current SOTA). Our approach offers a simpler training pipeline while yielding significant improvements in both training and inference times compared to existing SNNs for VPR. Hardware-in-the-loop tests using Intel's neuromorphic USB form factor, Kapoho Bay, show that our on-chip spiking models for VPR trained via the ANN-to-SNN conversion strategy continue to outperform their SNN counterparts, despite a slight but noticeable decrease in performance when transitioning from off-chip to on-chip, while offering significant energy efficiency. The results highlight the outstanding rapid prototyping and real-world deployment capabilities of this approach, showing it to be a substantial step toward more prevalent SNN-based real-world robotics solutions. RL literature has seen a surge in applications of SNNs, due to their computational efficiency when deployed on neuromorphic hardware. Existing work commonly uses population coding, reward-modulated spike timing-dependent plasticity (R-STDP), or other three-factor Hebbian rules. While these methods perform adequately on simple, less stochastic tasks, they falter in more complex and highly stochastic settings, such as a multi-agent Capture-the-Flag (CtF) game. This shortfall arises from various issues, including substantial hyperparameter tuning burdens, vulnerabilities to the dead neuron problem, and vanishing gradients. To address these challenges, we propose S2Act, a computationally lightweight spiking actor-critic network that adopts the ANN-to-SNN conversion paradigm. By employing rate-based approximations of Leaky LIF neurons that mimic Rectified Linear Unit (ReLU) activation functions during training, we mitigate the vanishing gradient problem. After training, we replace the rate-based LIF approximations with the original spiking LIF neurons for inference and deployment on neuromorphic hardware. We evaluated S2Act in a simulated parking task and more challenging multi-agent CtF environment alongside relevant SNN baselines. We also implemented S2Act in a real-world multi-robot CtF demonstration using Intel’s neuromorphic USB form factor Kapoho Bay on TurtleBot platforms. Our experimental results show that S2Act outperforms existing SNN baselines, as it achieves a remarkable improvement in training time and superior task performance with a significantly smaller network size. These findings highlight the potential of S2Act for efficient real-world deployment of RL-based robots in complex tasks.

Graduation Semester

2025-05

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129736

Copyright and License Information

Copyright 2025 Ugur Akcal

Owning Collections