Toward Real-Time Stream Processing in Edge Devices

Primary Investigator

• KD Kang, Professor, Department of Computer Science, State University of New York at Binghamton

Project Summary

Timely analysis of real-time sensor data streams is essential to key applications in the Internet of Things (IoT), such as smart health, transportation, and energy. Although advanced stream processing engines (SPEs), such as Apache Storm, Flink, and Spark Streaming, provide powerful stream processing frameworks in a cloud, sending sensor data to the SPE for analysis over the wide area network may incur many deadline misses and create bottlenecks in the core Internet. A viable alternative is real-time sensor data stream processing in edge devices; however, it is challenging to support timing constraints using limited resources available in such devices. Real-time scheduling theory is not directly applicable, since it is agnostic to data semantics and usually based on worst-case assumptions for the predictability that would be too pessimistic and resource inefficient in edge devices. The problem is becoming increasingly serious as the number of IoT devices and data volume increases rapidly. The proposed work aims to bridge the widening gap by investigating cost-efficient approaches for soft real-time stream processing at the edge. This project explores novel approaches to scheduling, sensor stream processing, and load sharing to significantly decrease deadline misses and communicational as well as computational resource consumption, while enhancing the reliability of real-time stream processing.

The research is expected to provide an enabling technology for important IoT applications with great societal impacts, such as those in healthcare, transportation, and energy that produce immense real-time sensor data streams, by substantially improving the timeliness and reliability of real-time stream processing with less resource consumptions compared to state-of-the-art SPEs. The investigator will use select research results to continue education and outreach efforts that include broadly disseminating publications and code that will be produced by this project, developing new courses and teaching materials on real-time stream processing, recruiting underrepresented groups of students to work on the project, and encouraging the younger generation to study computer science and pursue careers in industry and academia.

Publications

• Y. Liu, A. Andhare, K. D. Kang, Corun: Concurrent Inference and Continuous Training at the Edge for Cost-Efficient AI-Based Mobile Image Sensing, Sensors, 2024, 24(16), 5262. [paper] [source code]

• Y. Liu, K. D. Kang, AROD: Adaptive Real-Time Object Detection Based on Pixel Motion Speed, IEEE 100th Vehicular Technology Conference (VTC2024-Fall), Washington DC, USA, October 7-10, 2024. [pdf] [code]

• K. D. Kang, Limitations of Validity Intervals in Data Freshness Management, arXiv preprint arXiv:2407.20431, 2024.

• Y. Liu, K. D. Kang, Filtering Empty Video Frames for Efficient Real-Time Object Detection, Sensors, 24(10):3025, 2024. [paper] [code]

• Y. Liu, K. D. Kang, Preprocessing via Deep Learning for Enhancing Real-Time Performance of Object Detection, IEEE 97th Vehicular Technology Conference, Florence, Italy, June 20-23, 2023. [code]

• Y. Liu, K. D. Kang, M. J. Doe, HADD: High-Accuracy Detection of Depressed Mood, MDPI Technologies, 10(6), 123, 2022. paper, source code

• K. D. Kang, A Review of Efficient Real-Time Decision Making in the Internet of Things, MDPI Technologies, 10(1):12, 2022.

• F. Chai, K. D. Kang, Adaptive Deep Learning for Soft Real-Time Image Classification, MDPI Technologies, 9(1):20, 2021.

• Xiaoyang Zhang, Harshit Vadodaria, Na Li, Kyoung-Don Kang, and Yao Liu, A Smartphone Thermal Temperature Analysis for Virtual and Augmented Reality, In Proceedings of the IEEE International Conference on Artificial Intelligence & Virtual Reality, December 14-18, 2020.

• Di Mu, Mo Sha, Kyoung-Don Kang, and Hyungdae Yi, Radio Selection and Data Partitioning for Energy-Efficient Wireless Data Transfer in Real-Time IoT Applications, Ad Hoc Networks, Special Issue on Algorithms, Systems and Applications for Distributed Sensing, vol. 107, pp. 1-11, October 2020. (in press before the funding decision)