Evaluation and design of AI red-teaming agent for cybersecurity of web applications

Zhu, Yuxuan

Permalink

https://hdl.handle.net/2142/132586 Copy

Description

Title

Evaluation and design of AI red-teaming agent for cybersecurity of web applications

Author(s)

Zhu, Yuxuan

Issue Date

2025-12-08

Director of Research (if dissertation) or Advisor (if thesis)

Kang, Daniel

Department of Study

Siebel School Comp & Data Sci

Discipline

Computer Science

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• artificial intelligence
• benchmark
• agent
• cybersecurity

Abstract

Large language models (LLMs) and LLM-based agents have become increasingly sophisticated, especially in the realm of cybersecurity. Recent studies show that LLM agents are increasingly capable of autonomously conducting cyberattacks, particularly against web applications, which are among the most common targets. This emerging risk highlights the urgent need for a real-world red-teaming framework to systematically assess the capabilities and threats of LLMs in exploiting web application vulnerabilities. However, existing work falls short for two key reasons: (1) there is no real-world benchmark for evaluating LLMs on exploiting web application vulnerabilities, and (2) there is no agentic scaffolding that fully unleashes the potential of LLMs in exploiting web application vulnerabilities, especially under zero-day settings. To close this gap, we first introduce CVE-Bench, a real-world web application cybersecurity benchmark based on critical-severity Common Vulnerabilities and Exposures. In CVE-Bench, we design a sandbox framework that enables LLM agents to exploit vulnerable web applications in scenarios that mimic real-world conditions, while also providing effective evaluation of their exploits. Second, we build HPTSA, an agentic framework that coordinates teams of LLM agents to exploit real-world, zero-day vulnerabilities. While prior agents struggle with exploring many different vulnerabilities and long-range planning when used alone, HPTSA uses a planning agent that can launch subagents. The planning agent explores the system and determines which subagents to call, resolving long-term planning issues when trying different vulnerabilities. Empirically, our multi-agent system improves over prior agent frameworks by up to 4.3×. Using CVE-Bench and HPTSA, we evaluated the cybersecurity capability of frontier LLMs. We show that our state-of-the-art agent framework, HPTSA, can exploit up to 13% of the real-world web application vulnerabilities at an average cost of $1.7 per exploit. These findings highlight the realistic dual-use nature of AI agents: they can both support security testing and maintenance and facilitate web application attacks. We hope that this work encourages frontier LLM providers and stakeholders to carefully consider these dual-use implications when designing, deploying, and governing LLM services.

Graduation Semester

2025-12

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/132586

Copyright and License Information

Copyright 2025 Yuxuan Zhu

Owning Collections