Resource-constrained Edge Devices (EDs), e.g., IoT sensors and microcontroller units, are expected to make intelligent decisions using Deep Learning (DL) inference at the edge of the network. This generated significant research interest in designing tinyML models – DL models with reduced computation and memory storage requirements – that can be embedded on the EDs. However, tinyML models have lower inference accuracy. The question of how to use the full potential of tinyML without compromising the inference accuracy remains open. Toward this end, in this talk, we explore Hierarchical Inference (HI), a novel approach for performing distributed DL inference between EDs and edge servers. Under HI, for each data sample, an ED first uses a local algorithm (e.g., a tinyML model) for inference. Depending on the application, if the inference provided by the local algorithm is incorrect or further assistance is required from large DL models on edge or cloud, only then the ED offloads the data sample. At the outset, HI seems infeasible as the ED generally cannot know if the local inference is sufficient. Nevertheless, we present the feasibility of implementing HI for image classification applications. We demonstrate its benefits using quantitative analysis and show that HI  provides a better trade-off between offloading cost, throughput, and inference

About Ghina Al Atat

Ghina AlAtat is a Ph.D. student in the Edge Networks Group at IMDEA Networks Institute in Madrid, Spain. Her research focuses on distributed machine learning inference to support ML applications in the IoT and edge computing ecosystem, with a particular interest in edge intelligence. Prior to joining IMDEA, Ghina worked as a research assistant at Suliman Olayan School of Business and as a teaching assistant at the Physics Department at the American University of Beirut, Lebanon, while pursuing her master’s degree in Computational Sciences at AUB. Her thesis work involved modeling and analyzing a novel double-sided stochastic platform using a queuing theoretic approach. She holds a BS in Physics along with minors in Mathematics, Computer Science, and Computational Science, also from AUB.

This event will be conducted in English