A two-stage multi-objective optimization of erasure coding in overlay networks

Nishant Saurabh, Dragi Kimovski, Francesco Gaetano, Radu Prodan

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

In the recent years, overlay networks have emergedas a crucial platform for deployment of various distributed applications. Many of these applications rely on data redundancy techniques, such as erasure coding, to achieve higher fault tolerance. However, erasure coding applied in large scale overlay networksentails various overheads in terms of storage, latency and datarebuilding costs. These overheads are largely attributed to theselected erasure coding scheme and the encoded chunk placementin the overlay network. This paper explores a multi-objective optimization approach for identifying appropriate erasure codingschemes and encoded chunk placement in overlay networks. Theuniqueness of our approach lies in the consideration of multipleerasure coding objectives such as encoding rate and redundancyfactor, with overlay network performance characteristics likestorage consumption, latency and system reliability. Our approach enables a variety of tradeoff solutions with respect tothese objectives to be identified in the form of a Pareto front. To solve this problem, we propose a novel two stage multi-objective evolutionary algorithm, where the first stage determinesthe optimal set of encoding schemes, while the second stageoptimizes placement of the corresponding encoded data chunksin overlay networks of varying sizes. We study the performanceof our method by generating and analyzing the Pareto optimalsets of tradeoff solutions. Experimental results demonstrate thatthe Pareto optimal set produced by our multi-objective approachincludes and even dominates the chunk placements delivered bya related state-of-The-Art weighted sum method.

Original languageEnglish
Title of host publicationProceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages150-159
Number of pages10
ISBN (Electronic)9781509066100
DOIs
Publication statusPublished - 10 Jul 2017
Event17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017 - Madrid, Spain
Duration: 14 May 201717 May 2017

Publication series

NameProceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017

Conference

Conference17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017
Country/TerritorySpain
CityMadrid
Period14/05/1717/05/17

Keywords

  • Erasure Coding
  • Multi-Objective optimization
  • Overlay Network
  • Pareto optimal set
  • Peer-To-Peer

Fingerprint

Dive into the research topics of 'A two-stage multi-objective optimization of erasure coding in overlay networks'. Together they form a unique fingerprint.

Cite this