ePrints

Toward Distributed, Global, Deep Learning Using IoT Devices

Lookup NU author(s): Professor Raj Ranjan

Downloads

• Published version [.pdf]

Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

Abstract

© 1997-2012 IEEE. Deep learning (DL) using large scale, high-quality IoT datasets can be computationally expensive. Utilizing such datasets to produce a problem-solving model within a reasonable time frame requires a scalable distributed training platform/system. We present a novel approach where to train one DL model on the hardware of thousands of mid-sized IoT devices across the world, rather than the use of GPU cluster available within a data center. We analyze the scalability and model convergence of the subsequently generated model, identify three bottlenecks that are: high computational operations, time consuming dataset loading I/O, and the slow exchange of model gradients. To highlight research challenges for globally distributed DL training and classification, we consider a case study from the video data processing domain. A need for a two-step deep compression method, which increases the training speed and scalability of DL training processing, is also outlined. Our initial experimental validation shows that the proposed method is able to improve the tolerance of the distributed training process to varying internet bandwidth, latency, and Quality of Service metrics.

Publication metadata

Author(s): Sudharsan B, Patel P, Breslin J, Ali MI, Mitra K, Dustdar S, Rana O, Jayaraman PP, Ranjan R

Publication type: Article

Publication status: Published

Journal: IEEE Internet Computing

Year: 2021

Volume: 25

Issue: 3

Pages: 6-12

Online publication date: 20/07/2021

Acceptance date: 02/04/2018

Date deposited: 12/08/2021

ISSN (print): 1089-7801

ISSN (electronic): 1941-0131

Publisher: IEEE

URL: https://doi.org/10.1109/MIC.2021.3053711

DOI: 10.1109/MIC.2021.3053711

Altmetrics

Funding

Share