Files in this item

FilesDescriptionFormat

application/pdf

application/pdfJacquesDaSilva_Gabriela.pdf (2MB)
(no description provided)PDF

Description

Title:Partial fault tolerance in stream processing applications - methods and evaluation techniques
Author(s):Jacques da Silva, Gabriela
Director of Research:Iyer, Ravishankar K.
Doctoral Committee Chair(s):Iyer, Ravishankar K.
Doctoral Committee Member(s):Lumetta, Steven S.; Nahrstedt, Klara; Sanders, William H.; Vasudevan, Shobha
Department / Program:Electrical & Computer Eng
Discipline:Electrical & Computer Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Degree:Ph.D.
Genre:Dissertation
Subject(s):fault tolerance
fault injection
stream processing
model-based evaluation
high-availability
Abstract:Stream processing emerged as a paradigm to continuously process incoming live data streams, such as audio, video, and business feeds. These applications are assembled as dataflow graphs, where each vertex of the graph is a stream operator and each edge is a stream connection. In this environment, a fault in a stream operator can result in massive data loss or in the generation of inaccurate results. Most of the fault tolerance solutions proposed for streaming applications aim at guaranteeing that no data is lost or that no data item is delivered to the application more than once. These techniques result in high performance overhead, given the need to coordinate the state stored in checkpoints of distributed components or maintain consistency between replicas. In this dissertation, we investigate partial fault tolerance methods, which protect only the most critical stream operators of a streaming application. These methods take advantage of the fact that stream processing algorithms are approximate by nature and, as a result, can still achieve acceptable results under data loss and duplicate data delivery. The methods proposed in this dissertation include a checkpoint-based mechanism and a partial graph replication technique. Both techniques were implemented in System S, IBM Research's stream processing middleware. In addition, this dissertation describes two different fault tolerance evaluation techniques. The first technique is based on fault injection and is used to emulate the effects of partial fault tolerance on a streaming application. With the fault injection results, the developers can understand the impact of faults on the application output and identify the most critical operators on their streaming application. The second evaluation technique is a model-based framework which provides generic abstractions for representing streaming applications with the stochastic activity network formalism. The framework allows the comparison of different fault tolerance techniques under varying fault models. Based on the results, the developers can evaluate the trade-offs that a certain technique provides when applied to their target application.
Issue Date:2011-01-14
URI:http://hdl.handle.net/2142/18383
Rights Information:Copyright 2010 Gabriela Jacques da Silva
Date Available in IDEALS:2011-01-14
Date Deposited:December 2


This item appears in the following Collection(s)

Item Statistics