TY - GEN
T1 - Orchestrating safe streaming computations with precise control
AU - Li, Peng
AU - Agrawal, Kunal
AU - Buhler, Jeremy
AU - Chamberlain, Roger D.
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2014
Y1 - 2014
N2 - Streaming computing is a paradigm of distributed computing that features networked nodes connected by first-in-first-out data channels. Communication between nodes may include not only high-volume data tokens but also infrequent and unpredictable control messages carrying control information, such as data set boundaries, exceptions, or reconfiguration requests. In many applications, it is necessary to order delivery of control messages precisely relative to data tokens, which can be especially challenging when nodes can filter data tokens. Existing approaches, mainly data serialization protocols, do not exploit the low-volume nature of control messages and may not guarantee that synchronization of these messages with data will be free of deadlock. In this paper, we propose an efficient messaging system for adding precisely ordered control messages to streaming applications. We use a credit-based protocol to avoid the need to tag data tokens and control messages. For potential deadlocks caused by filtering behavior and global synchronization, we propose deadlock avoidance solutions and prove their correctness.
AB - Streaming computing is a paradigm of distributed computing that features networked nodes connected by first-in-first-out data channels. Communication between nodes may include not only high-volume data tokens but also infrequent and unpredictable control messages carrying control information, such as data set boundaries, exceptions, or reconfiguration requests. In many applications, it is necessary to order delivery of control messages precisely relative to data tokens, which can be especially challenging when nodes can filter data tokens. Existing approaches, mainly data serialization protocols, do not exploit the low-volume nature of control messages and may not guarantee that synchronization of these messages with data will be free of deadlock. In this paper, we propose an efficient messaging system for adding precisely ordered control messages to streaming applications. We use a credit-based protocol to avoid the need to tag data tokens and control messages. For potential deadlocks caused by filtering behavior and global synchronization, we propose deadlock avoidance solutions and prove their correctness.
UR - http://www.scopus.com/inward/record.url?scp=84988222571&partnerID=8YFLogxK
U2 - 10.1109/PADSW.2014.7097925
DO - 10.1109/PADSW.2014.7097925
M3 - Conference contribution
AN - SCOPUS:84988222571
T3 - Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
SP - 1017
EP - 1022
BT - 2014 20th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2014 - Proceedings
PB - IEEE Computer Society
T2 - 20th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2014
Y2 - 16 December 2014 through 19 December 2014
ER -