Files in this item



application/pdfDUAN-THESIS-2016.pdf (1MB)
(no description provided)PDF


Title:Effective detection of security compromises in enterprises using feature engineering
Author(s):Duan, Jiayi
Advisor(s):Vasudevan, Shobha
Department / Program:Electrical & Computer Eng
Discipline:Electrical & Computer Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Feature engineering
Abstract:We present a method to effectively detect malicious activity in the data of enterprise logs. Our method involves feature engineering, or generating new features by applying operators on the features of the raw data. We apply the Fourier expansion of Boolean functions to generate parity functions on feature subsets, or parity features. We also investigate a heuristic method of applying Boolean operators to raw data features, generating propositional features. We demonstrate with real data sets that the engineered features enhance the performance of classifiers and clustering algorithms. As compared to classification of raw data features, the engineered features achieve up to 50.6% improvement in malicious recall while sacrificing no more than 0.47% in accuracy. Clustering with respect to the engineered features finds up to 6 "pure" malicious clusters, as compared to 0 "pure" clusters with raw data features. In one case, exactly one (1) engineered feature could achieve higher performance than 91 raw data features. In general, a small number (<10) of engineered features achieve higher performance than raw data features.
Issue Date:2016-12-08
Rights Information:Copyright 2016 Jiayi Duan
Date Available in IDEALS:2017-03-01
Date Deposited:2016-12

This item appears in the following Collection(s)

Item Statistics