University of Illinois Urbana-Champaign

Confluence: A System for Lossless Multi-Source Single-Sink Data Collection

Patel, Jay A.; Cho, Brian; Gupta, Indranil

Content Files
paper.pdf

Permalink

https://hdl.handle.net/2142/13158

Description

Title
Confluence: A System for Lossless Multi-Source Single-Sink Data Collection
Author(s)
  • Patel, Jay A.
  • Cho, Brian
  • Gupta, Indranil
Issue Date
2009-07-24
Keyword(s)
  • Distributed Systems
  • Computer Networks
Date of Ingest
2009-07-24T21:28:16Z
Abstract
Distributed environments often require collection of large amounts of critical and raw data from multiple locations to a central clearinghouse, e.g., task results or large datasets from multiple clouds, logs from multiple PlanetLab nodes, video transcripts in tele-immersive settings, etc. We present the design, implementation and evaluation of Confluence, a system for rapid and lossless transfer of unique files from multiple source nodes to a single sink node. First, we formally model the multi-source single-sink data collection problem for a static network and present an optimal solution in terms of total transfer time. Second, we build in mechanisms to make the system workable in dynamic networks. The resulting Confluence system builds an adaptive source-2-source (s2s) overlay amongst participating nodes, which exploits spatial as well as temporal heterogeneity of available bandwidth.We conduct an evaluation of Confluence on PlanetLab traces in ns-2. Results show that Confluence can improve total transfer time by as much as 40% (with up to 50 sources).
Type of Resource
  • text
  • image
Genre of Resource
Technical Report
Language
en
Permalink
http://hdl.handle.net/2142/13158

Owning Collections

Research and Tech Reports - Computer Science PRIMARY