Browse by author
Lookup NU author(s): Professor Raj Ranjan
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
© 1997-2012 IEEE. Deep learning (DL) using large scale, high-quality IoT datasets can be computationally expensive. Utilizing such datasets to produce a problem-solving model within a reasonable time frame requires a scalable distributed training platform/system. We present a novel approach where to train one DL model on the hardware of thousands of mid-sized IoT devices across the world, rather than the use of GPU cluster available within a data center. We analyze the scalability and model convergence of the subsequently generated model, identify three bottlenecks that are: high computational operations, time consuming dataset loading I/O, and the slow exchange of model gradients. To highlight research challenges for globally distributed DL training and classification, we consider a case study from the video data processing domain. A need for a two-step deep compression method, which increases the training speed and scalability of DL training processing, is also outlined. Our initial experimental validation shows that the proposed method is able to improve the tolerance of the distributed training process to varying internet bandwidth, latency, and Quality of Service metrics.
Author(s): Sudharsan B, Patel P, Breslin J, Ali MI, Mitra K, Dustdar S, Rana O, Jayaraman PP, Ranjan R
Publication type: Article
Publication status: Published
Journal: IEEE Internet Computing
Online publication date: 20/07/2021
Acceptance date: 02/04/2018
Date deposited: 12/08/2021
ISSN (print): 1089-7801
ISSN (electronic): 1941-0131
Altmetrics provided by Altmetric