面向多源大数据云端处理的成本最小化方法

发布时间：2018-05-29 15:57

本文选题：大数据处理 + 多数据中心　；参考：《软件学报》2017年03期

【摘要】：云计算为大数据处理提供了一种强大而高效的解决方案.在此模式下,数据管理者(data manager,简称DM)可以租用多个数据中心实时处理地理分散的数据.然而,由于数据产生的动态性以及资源价格的波动性,将数据迁移至哪些数据中心并提供合适的计算资源来处理它们,成为DM低成本处理多源数据的一大问题.首先,将以上问题转换成联合随机优化问题;然后,利用李雅普诺夫(Lyapunov)优化框架将原问题分解成两个独立的子问题进行求解;最后,基于求解结果设计在线算法.理论分析结果表明:所提算法可不断趋近线下最优解,并能够保证数据处理时延.在WorldCup98和Youtube数据集上的实验验证了理论分析结果的正确性以及该方法的优越性.
[Abstract]:Cloud computing provides a powerful and efficient solution for big data processing. In this mode, data manager (DMN) can rent multiple data centers to process geographically distributed data in real time. However, due to the dynamic nature of data and the volatility of resource prices, migrating data to which data centers and providing appropriate computing resources to deal with them has become a major problem for DM to process multi-source data at low cost. First, the above problem is transformed into a joint stochastic optimization problem; then, the original problem is decomposed into two independent sub-problems by using Lyapunov Lyapunov-optimization framework. Finally, an online algorithm is designed based on the solution results. The theoretical analysis results show that the proposed algorithm can continuously reach the optimal solution under the line and can guarantee the data processing delay. Experiments on WorldCup98 and Youtube datasets verify the correctness of the theoretical analysis results and the advantages of the method.
【作者单位】：国防科学技术大学信息系统工程重点实验室;Department
【基金】：国家自然科学基金(61572511) 海外及港澳学者合作研究基金(11428101) 国防科学技术大学科研计划项目(ZK16-03-57,ZK16-03-09) 西南电子电信技术研究室公开课题(2015014)~~
【分类号】：TP311.13

【相似文献】