Data redundancy management for leaf-edges in connected environments

Abstract

Major advances in the fields of Internet and Communication Technology (ICT), data modeling/processing, and sensing technology have rendered traditional environments (e.g., cities, buildings) more connected. Although the sensed data could be useful for various applications (e.g., event detection in cities, energy management in commercial buildings), it first requires pre-processing to clean various inconsistencies (e.g., anomalies, redundancies, missing values). In this work, we focus on managing data redundancies in connected environments. Existing approaches suffer from (i) disregarding edge data redundancies either at the edge or at the core of the network; (ii) disregarding sensor mobility and the dynamicity of the network; (iii) disregarding the limited resources of edge devices; (iv) disregarding network/infrastructure resources; and (v) disregarding data consumer needs/requirements when cleaning the data redundancies. To address these limitations, we propose here DRMF: Data Redundancy Management for leaF-edges allowing to identify and remove data redundancies in connected environments at the device level. DRMF considers both static and mobile edge devices, and provides two algorithms for temporal and spatio-temporal redundancy detection. Once redundancies are identified, DRMF performs data deduplication taking into account the dynamic requirements of data consumers and device resources (e.g., processing, battery, memory). Experimental results highlight the performance and accuracy of our solution in detecting and eliminating edge data redundancies.