JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers - IRISA_SET Access content directly
Conference Papers Year : 2014

JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers

Abstract

The easily-accessible computation power offered by cloud infrastructures coupled with the revolution of Big Data are expanding the scale and speed at which data analysis is per-formed. In their quest for finding the Value in the 3 Vs of Big Data, applications process larger data sets, within and across clouds. Enabling fast data transfers across geograph-ically distributed sites becomes particularly important for applications which manage continuous streams of events in real time. Scientific applications (e.g. the Ocean Observa-tory Initiative or the ATLAS experiment) as well as com-mercial ones (e.g. Microsoft's Bing and Office 365 large-scale services) operate on tens of data-centers around the globe and follow similar patterns: they aggregate monitoring data, assess the QoS or run global data mining queries based on in-ter site event stream processing. In this paper, we propose a set of strategies for efficient transfers of events between cloud data-centers and we introduce JetStream: a prototype implementing these strategies as a high performance batch-based streaming middleware. JetStream is able to self-adapt to the streaming conditions by modeling and monitoring a set of context parameters. It further aggregates the available bandwidth by enabling multi-route streaming across cloud sites. The prototype was validated on tens of nodes from US and Europe data-centers of the Windows Azure cloud using synthetic benchmarks and with application code from the context of the Alice experiment at CERN. The results show an increase in transfer rate of 250 times over individual event streaming. Besides, introducing an adaptive transfer strategy brings an additional 25% gain. Finally, the transfer rate can further be tripled thanks to the use of multi-route streaming.
Fichier principal
Vignette du fichier
DEBS2014 Camera Ready.pdf (833.62 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01090281 , version 1 (19-12-2014)

Identifiers

Cite

Radu Tudoran, Olivier Nano, Ivo Santos, Alexandru Costan, Hakan Soncu, et al.. JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers. DEBS'14 - The 8th ACM International Conference on Distributed Event-Based Systems, May 2014, Mumbai, India. pp.23 - 34, ⟨10.1145/2611286.2611298⟩. ⟨hal-01090281⟩
787 View
654 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More