Orchestrating safe streaming computations with precise control

Peng Li, Kunal Agrawal, Jeremy Buhler, Roger D. Chamberlain

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Streaming computing is a paradigm of distributed computing that features networked nodes connected by first-in-first-out data channels. Communication between nodes may include not only high-volume data tokens but also infrequent and unpredictable control messages carrying control information, such as data set boundaries, exceptions, or reconfiguration requests. In many applications, it is necessary to order delivery of control messages precisely relative to data tokens, which can be especially challenging when nodes can filter data tokens. Existing approaches, mainly data serialization protocols, do not exploit the low-volume nature of control messages and may not guarantee that synchronization of these messages with data will be free of deadlock. In this paper, we propose an efficient messaging system for adding precisely ordered control messages to streaming applications. We use a credit-based protocol to avoid the need to tag data tokens and control messages. For potential deadlocks caused by filtering behavior and global synchronization, we propose deadlock avoidance solutions and prove their correctness.

Original languageEnglish
Title of host publication2014 20th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2014 - Proceedings
PublisherIEEE Computer Society
Pages1017-1022
Number of pages6
ISBN (Electronic)9781479976157
DOIs
StatePublished - 2014
Event20th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2014 - Hsinchu, Taiwan, Province of China
Duration: Dec 16 2014Dec 19 2014

Publication series

NameProceedings of the International Conference on Parallel and Distributed Systems - ICPADS
Volume2015-April
ISSN (Print)1521-9097

Conference

Conference20th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2014
Country/TerritoryTaiwan, Province of China
CityHsinchu
Period12/16/1412/19/14

Fingerprint

Dive into the research topics of 'Orchestrating safe streaming computations with precise control'. Together they form a unique fingerprint.

Cite this