University of Illinois Urbana-Champaign

Reflection-aware static regression test selection

Hadzi-Tanovic, Milica

Content Files
HADZI-TANOVIC-THESIS-2018.pdf

Permalink

https://hdl.handle.net/2142/101065

Description

Title
Reflection-aware static regression test selection
Author(s)
Hadzi-Tanovic, Milica
Issue Date
2018-04-26
Director of Research (if dissertation) or Advisor (if thesis)
Marinov, Darko
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2018-09-04T20:31:54Z
Keyword(s)
  • regression test selection
  • reflection
  • static analysis
Abstract
Regression test selection (RTS) aims to speed up regression testing by rerunning only tests that are affected by code changes. RTS can be performed using dynamic or static analysis techniques. A recent study showed that static and dynamic RTS can perform similarly for some medium-sized Java projects. However, the results also showed that static RTS can be sometimes unsafe, missing to select some tests that dynamic RTS selects, and reflection was the only cause of unsafety among the evaluated projects. In this thesis, we investigate five techniques—three purely static techniques and two hybrid static-dynamic techniques—to make static RTS safe with respect to reflection. We implemented four of these reflection-aware techniques as extensions to one reflection-unaware (RU) static RTS technique in a tool called STARTS. We evaluated the fifth technique but did not yet fully implement it. To assess reflection-aware SRTS techniques, we measured benefits and costs of four implemented reflection-aware techniques by comparing their end-to-end times with the RU technique and with RetestAll—the latter runs all tests after every code change. We also compared safety and precision of all five static RTS techniques relative to Ekstazi, a state-of-the-art dynamic RTS technique. Our results on 805 revisions of 22 open-source Java projects show that all reflection-aware techniques we evaluated can make static RTS safe with respect to reflection, but their costs vary widely. The best purely static technique in our study is based on border analysis with minimal border methods which avoids analyzing JDK and saves, on average, 14.1% of the end-to-end time of RetestAll. Furthermore, the results show that a hybrid technique based on per-test analysis is very promising in terms of safety and precision. On the other hand, the worst techniques were based on string analysis; these techniques are imprecise and often lead to selecting to re-run all tests. Taken together, these results show the need for more research into purely static techniques for making static RTS reflection aware.
Graduation Semester
2018-05
Type of Resource
text
Permalink
http://hdl.handle.net/2142/101065
Copyright and License Information
Copyright 2018 Milica Hadzi-Tanovic

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science