Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE)

Malboubi, Mehdi; Vu, Cuong; Chuah, Chen -Nee; Sharma, Puneet

doi:10.1109/TNET.2015.2475362

Title: Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE)

Abstract

Network inference (or tomography) problems, such as traffic matrix estimation or completion and link loss inference, have been studied rigorously in different networking applications. These problems are often posed as under-determined linear inverse (UDLI) problems and solved in a centralized manner, where all the measurements are collected at a central node, which then applies a variety of inference techniques to estimate the attributes of interest. This paper proposes a novel framework for decentralizing these large-scale under-determined network inference problems by intelligently partitioning it into smaller subproblems and solving them independently and in parallel. The resulting estimates, referred to as multiple descriptions, can then be fused together to compute the global estimate. We apply this Multiple Description and Fusion Estimation (MDFE) framework to three classical problems: traffic matrix estimation, traffic matrix completion, and loss inference. Using real topologies and traces, we demonstrate how MDFE can speed up computation while maintaining (even improving) the estimation accuracy and how it enhances robustness against noise and failures. In conclusion, we also show that our MDFE framework is compatible with a variety of existing inference techniques used to solve the UDLI problems.

Authors:

Malboubi, Mehdi ^[1]; Vu, Cuong ^[1]; Chuah, Chen -Nee ^[1]; Sharma, Puneet ^[2]

Univ. of California, Davis, CA (United States)
HP-Labs, Palo Alto, CA (United States)

Publication Date:: Thu Sep 17 00:00:00 EDT 2015

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC)

OSTI Identifier:: 1525161

Grant/Contract Number:: AC02-05CH11231

Resource Type:: Accepted Manuscript

Journal Name:: IEEE/ACM Transactions on Networking

Additional Journal Information:: Journal Volume: 24; Journal Issue: 4; Journal ID: ISSN 1063-6692

Publisher:: IEEE - ACM

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; distributed estimation; multiple description fusion; network inference; traffic matrix estimation; under-determined inverse problem

Citation Formats


                    Malboubi, Mehdi, Vu, Cuong, Chuah, Chen -Nee, and Sharma, Puneet. Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE).  United States: N. p., 2015. 
Web.  doi:10.1109/TNET.2015.2475362.

Copy to clipboard


                    Malboubi, Mehdi, Vu, Cuong, Chuah, Chen -Nee, & Sharma, Puneet. Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE).  United States.  https://doi.org/10.1109/TNET.2015.2475362

Copy to clipboard


                    Malboubi, Mehdi, Vu, Cuong, Chuah, Chen -Nee, and Sharma, Puneet. Thu .  
"Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE)".  United States.  https://doi.org/10.1109/TNET.2015.2475362.  https://www.osti.gov/servlets/purl/1525161.

Copy to clipboard


                    
@article{osti_1525161,

  title        = {Decentralizing Network Inference Problems With Multiple-Description Fusion Estimation (MDFE)},

  author       = {Malboubi, Mehdi and Vu, Cuong and Chuah, Chen -Nee and Sharma, Puneet},

  abstractNote = {Network inference (or tomography) problems, such as traffic matrix estimation or completion and link loss inference, have been studied rigorously in different networking applications. These problems are often posed as under-determined linear inverse (UDLI) problems and solved in a centralized manner, where all the measurements are collected at a central node, which then applies a variety of inference techniques to estimate the attributes of interest. This paper proposes a novel framework for decentralizing these large-scale under-determined network inference problems by intelligently partitioning it into smaller subproblems and solving them independently and in parallel. The resulting estimates, referred to as multiple descriptions, can then be fused together to compute the global estimate. We apply this Multiple Description and Fusion Estimation (MDFE) framework to three classical problems: traffic matrix estimation, traffic matrix completion, and loss inference. Using real topologies and traces, we demonstrate how MDFE can speed up computation while maintaining (even improving) the estimation accuracy and how it enhances robustness against noise and failures. In conclusion, we also show that our MDFE framework is compatible with a variety of existing inference techniques used to solve the UDLI problems.},

  doi          = {10.1109/TNET.2015.2475362},

  journal      = {IEEE/ACM Transactions on Networking},

  number       = 4,

  volume       = 24,

  place        = {United States},

  year         = {Thu Sep 17 00:00:00 EDT 2015},

  month        = {Thu Sep 17 00:00:00 EDT 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/TNET.2015.2475362

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 3 works

Citation information provided by
Web of Science

Figures / Tables:

Fig. 1: MDFE process: a system perspective.

All figures and tables (27 total)

Save / Share:

Export Metadata

Save to My Library

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

Decentralized Low-Rank State Estimation for Power Distribution Systems

Journal Article Sagan, April ; Liu, Yajing ; Bernstein, Andrey - IEEE Transactions on Smart Grid

This article considers the low-observability state estimation problem in power distribution networks and develops a decentralized state estimation algorithm leveraging the matrix completion methodology. Matrix completion has been shown to be an effective technique in state estimation that exploits the low dimensionality of the power system measurements to recover missing information. This technique can utilize an approximate (linear) load flow model, or it can be used with no physical models in a network where no information about the topology or line admittance is available. The direct application of matrix completion algorithms requires solving a semi-definite programming (SDP) problem, which becomesmore »« less
https://doi.org/10.1109/TSG.2021.3058609
Decentralized Signal Processing and Distributed Control of Collaborative Autonomous Networks (17-ERD-101 LDRD Final Report)

Technical Report Goldhahn, Ryan A.

Intelligent autonomous sensor networks, often comprised of large numbers of sensors, must be capable of jointly exploiting data collected at each agent in the network, and using that data to optimize their future actions towards multiple mission objectives. Centralized signal processing and optimization solutions process all data and determine all future actions at a single agent, and the resulting information and commands are disseminated back to the network. The communications bandwidth this requires and the single point of failure the central agent represents often make these solutions untenable for national security applications. In this project, several fundamental algorithms for solvingmore »« less
https://doi.org/10.2172/1573142

Full Text Available
A learning-based measurement framework for traffic matrix inference in software defined networks

Journal Article Malboubi, Mehdi ; Peng, Shu-Ming ; Sharma, Puneet ; ... - Computers and Electrical Engineering

Here, we propose an intelligent framework for Traffic Matrix (TM) inference in Software Defined Networks (SDN) where the Ternary Content Addressable Memory (TCAM) entries of switches are partitioned into two parts to: (1) effectively aggregate part of incoming flows for aggregate measurements, and (2) de-aggregate and directly measure the most informative flows for per-flow measurements. These measurements are then processed to effectively estimate the size of network flows. Under hard resource constraints of limited TCAM sizes, we show how to design the optimal and efficient-compressed flow aggregation matrices. We propose an optimal Multi-Armed Bandit (MAB) based algorithm to adaptively measuremore »« less
https://doi.org/10.1016/j.compeleceng.2017.11.020

Full Text Available
Decentralized Optimal Dispatch of Photovoltaic Inverters in Residential Distribution Systems

Conference Dall'Anese, Emiliano ; Dhople, Sairaj V. ; Johnson, Brian B. ; ...

Summary form only given. Decentralized methods for computing optimal real and reactive power setpoints for residential photovoltaic (PV) inverters are developed in this paper. It is known that conventional PV inverter controllers, which are designed to extract maximum power at unity power factor, cannot address secondary performance objectives such as voltage regulation and network loss minimization. Optimal power flow techniques can be utilized to select which inverters will provide ancillary services, and to compute their optimal real and reactive power setpoints according to well-defined performance criteria and economic objectives. Leveraging advances in sparsity-promoting regularization techniques and semidefinite relaxation, this papermore »« less
https://doi.org/10.1109/PESGM.2015.7286298
Decentralized Failure-Tolerant Optimization of Electric Vehicle Charging

Journal Article Aravena, Ignacio ; Chapin, Steve J. ; Ponce, Colin - IEEE Transactions on Smart Grid

We present a decentralized failure-tolerant algorithm for optimizing electric vehicle (EV) charging, using charging stations as computing agents. The algorithm is based on the alternating direction method of multipliers (ADMM) and it has the following features: (i) It handles capacity, peak demand, and ancillary services coupling constraints. (ii) It does not require a central agent collecting information and performing coordination (e.g. an aggregator), instead all agents exchange information and computations are carried out in a fully decentralized fashion. (iii) It can withstand the failure of any number of computing agents, as long as the remaining computing agents are in amore »« less
https://doi.org/10.1109/tsg.2021.3080583

Full Text Available

Similar Records