Toggle Main Menu Toggle Search

Open Access padlockePrints

Run-time Adaptation of Stream Processing Spanning the Cloud and the Edge

Lookup NU author(s): Adam Cattermole, Jonathan Dowland, Professor Paul WatsonORCiD

Downloads


Licence

This is the authors' accepted manuscript of a conference proceedings (inc. abstract) that has been published in its final definitive form by IEEE, 2021.

For re-use rights please refer to the publisher's terms and conditions.


Abstract

Applications that process streams of events generated by sensors are important in a wide range of domains. It is now common to distribute stream processing across edge devices and the cloud. This exploits processing power near the sensors, reducing the load on the cloud and often the required network bandwidth. In this paper we focus on one challenge in distributed stream processing: automatically adapting the partitioning of the processing between the edge and the cloud without a loss of service. An example is when the event arrival rate increases and the edge processor can no longer meet performance requirements. Re-partitioning without loss of service involves moving computations between the edge and the cloud while events are still being processed. In this paper we describe StrIoT – a stream processing system that supports automatic re-partitioning. It is based on a set of functional stream operators, and the paper describes how the run-time system can automatically adapt applications that use them. A key feature is support for the fission and fusion of operators during adaptations. Performance evaluation shows that StrIoT can move parts of a stream processing application between the cloud and edge with only a low, temporary impact on performance.


Publication metadata

Author(s): Cattermole A, Dowland J, Watson P

Publication type: Conference Proceedings (inc. Abstract)

Publication status: Published

Conference Name: 14th IEEE/ACM International Conference on Utility and Cloud Computing (UCC 2021)

Year of Conference: 2021

Pages: Article no. 18

Online publication date: 07/02/2022

Acceptance date: 14/11/2021

Date deposited: 08/02/2022

Publisher: IEEE

URL: https://doi.org/10.1145/3492323.3495627

DOI: 10.1145/3492323.3495627

Notes: https://www.cs.le.ac.uk/events/UCC2021/conference-programme/Workshop-Programme.htm

Library holdings: Search Newcastle University Library for this item

ISBN: 9781450391634


Share