数据中心网络的流量管理和优化问题研究

发布时间：2018-04-08 15:12

本文选题：云计算　切入点：数据中心网络　出处：《北京邮电大学》2013年博士论文

【摘要】：近几年,云计算蓬勃发展并得到了产业界和学术界的广泛关注,已经成为信息化建设领域的热点和未来趋势。与此同时,许多新的互联网在线业务(如搜索、社交网络、即时通信等)的用户量也在迅速增长。在云计算和互联网在线业务的快速发展过程中,作为信息化基础设施的数据中心一直处于核心地位。随着业务的发展和新技术的使用,数据中心正在发生和形成重大的变化及趋势,由此给数据中心网络(Data Center Networks, DCN)带来了新的挑战和问题。首先,新兴的业务需要服务器之间进行大量的一对多和多对多的通信,其结果是数据中心的内部流量急剧增长并呈现出不同于互联网流量的新特性。在当前技术条件下,数据中心网络会频繁发生拥塞,导致丢包增多、时延增大和吞吐量下降,严重影响业务的性能和服务质量。其次,虚拟化技术给数据中心网络带来了新的挑战,虚拟机迁移带来的流量会加重网络负载甚至造成网络拥塞。由此可见,数据中心网络在新的业务模式和新技术发展趋势下已经不能满足需求。为保证业务的性能及服务质量,数据中心网络的流量管理和优化问题已成为当前亟待解决的重要问题。本文首先系统总结了数据中心网络的相关研究背景和最新进展,对当前的数据中心网络各层的相关技术做了比较全面的归纳和总结。接着,以数据中心网络流量作为切入点,从数据中心网络的新特性出发,分别从网络层次的低到高,研究了数据中心网络的流量管理和优化问题,包括流量工程、传输层协议的优化与改进、多虚拟机迁移调度等四个方面,以减少数据中心网络的拥塞,优化网络资源的使用,从而提升业务的性能。具体如下： (一)研究了数据中心网络常见的场景“以太网链路聚合”中的流量优化问题。在数据中心网络流量工程中,只是将被聚合的物理链路看做是一条链路,没有考虑到这些被聚合的物理链路之间如何均衡流量。而目前以太网链路聚合的流量分配算法也是基于会话的,不适用于数据中心网络环境。本文首先分析基于帧的流量分配算法造成帧乱序的原因,发现这些因素在数据中心网络环境下都不再成立或者可以避免。因此,本文提出一个基于字节计数的帧级别流量分配算法,该算法可以达到最优的负载均衡,同时不会造成乱序。唯一需要满足的条件是同一个会话的帧大小要一致,而该条件在数据中心环境下很容易满足。 (二)研究了TCP协议在数据中心网络面临的两个问题：TCP扩展限制(Scaling Limit)和TCP Incast吞吐量崩溃。数据中心带宽高、时延小且出现了新的通信模式,面向广域网设计的TCP在数据中心网络环境面临着性能差的问题。本文提出通过减小分组长度来优化TCP性能。分组长度越小,链路和交换机缓存区可以承载的分组越多,TCP拥塞控制的粒度也越小,同时丢包可能性也越小,从而达到优化TCP性能的目的,包括减轻TCP扩展限制和缓解TCP Incast吞吐量崩溃。减小分组的好处是可以很方便在现有硬件和软件上实现,只需要在服务器或交换机上做简单配置即可。 (三)研究了能提供截止时间知晓(deadline-aware)传输服务的改进TCP协议。数据中心内的很多流都有截止时间,而占了数据中心大部分流量的TCP协议不能提供截止时间知晓的传输服务。本文提出了一种截止时间知晓的改进TCP协议,称为DATCP (Deadline-Aware TCP),来为数据中心承载的应用提供截止时间知晓的传输服务。DATCP根据流的期望速率和实际吞吐量之差来动态调整其其拥塞控制参数,来满足流的截止时间需求。DATCP只需对标准TCP做少量修改,并利用现有商品化数据中心交换机普遍支持的显示拥塞通知ECN机制。 (四)研究了虚拟化数据中心常见的多虚拟机迁移调度问题。前面都是从网络角度来优化流量的,这在互联网研究领域是普遍的。而在数据中心网络,直接控制流量分布来实现流量管理和优化更直接有效。本文通过计算出一个合适的虚拟机迁移的调度顺序,来最小化虚拟机迁移的持续时间。多个虚拟机同时迁移(VM shuffle)调度问题被建模成一个作业调度的最优化问题,该问题等价于旅行商问题。由于该问题是NP-难,本文设计了两个调度算法：基于模拟退火的离线算法和基于贪婪方式的在线算法。两个算法的基本想法都是计算一个合适的虚拟机调度顺序,优先迁移给其他虚拟机带来最大好处的虚拟机,为后续迁移虚拟机提供更大的可用带宽,这样所有虚拟机的迁移持续时间就会减小,从而最小化多虚拟机迁移对数据中心造成的影响。
[Abstract]:In recent years, rapid development of cloud computing and has received wide attention from industry and academia, has become a hot topic in the field of information technology and the trend in the future. At the same time, many new Internet online business (such as search, social networking, instant messaging and so on) the number of users is increasing rapidly. In the process of rapid development of cloud computing the Internet and online business, as the information infrastructure of the data center has been at the core. With the use of business development and new technology, data center is the occurrence and formation of major changes and trends, to the data center network (Data Center Networks, DCN) brings new challenges and problems. First of all, the emerging business needs a large number of servers and many to many communication, the result is the internal flow of data center and the rapid growth of Internet traffic is different from New features. In the current technical conditions, the data center network congestion occur frequently, cause packet loss increase, delay and throughput degradation, seriously affect the performance and quality of service business. Secondly, virtualization technology has brought new challenges to the data center network, virtual machine migration to bring traffic will increase the network load and even cause network congestion. Thus, the data center network in the development of business mode and under the new trend of new technology can not meet the demand. In order to ensure the service performance and quality of service, traffic management and data center network optimization has become an important issue to be solved.
This paper first summarizes the related research background and latest development of the data center network, the relevant technology of each layer of the network data center to do a more comprehensive and summarized. Then, the data center network traffic as a starting point, starting from the characteristics of the new data center network, separately from the network level to low high, traffic management and optimization of the data center network, including traffic engineering, optimization and improvement of transmission protocol, the four aspects of virtual machine migration scheduling, in order to reduce the congestion of data center network, optimize the use of cyber source, so as to enhance business performance. The details are as follows:
(a) the flow optimization problem scene "data center network common Ethernet link aggregation". In the data center network traffic engineering, only the physical link is regarded as a link aggregation, without considering the physical link between these are aggregated as He Junheng flow. And traffic allocation algorithm at Ethernet link the polymerization is also based on the session, does not apply to the data center network environment. This paper first analyzes the traffic allocation algorithm based on frame frame causes out of order, it was found that these factors in the data center network environment are no longer valid or can be avoided. Therefore, this paper proposes a frame level traffic allocation algorithm based on the byte count. The algorithm can achieve optimal load balancing, without causing disorder. The only need to meet the condition that the frame size of the same session to be consistent, and the conditions It is easy to meet in the data center environment.
(two) of the TCP agreement in the two problems in the data center network: TCP (Scaling Limit) expansion limit and TCP Incast throughput collapse. The data center of high bandwidth, small delay and the emergence of new communication mode, Wan oriented design of the TCP in the data center network environment is facing the problem of poor performance in this paper. Put forward to optimize the performance of TCP by reducing the packet length. The packet length is smaller, more packet link and switch buffer can bear, the size of TCP congestion control and packet loss is smaller and less likely, so as to achieve the purpose of optimization of TCP can reduce TCP, including expansion limit and mitigate TCP Incast throughput decreases collapse. Group benefits can be easily realized in the existing hardware and software, only need to do a simple configuration to the server or switch.
(three) the study can provide the deadline aware (Deadline-Aware) improved TCP protocol services. A lot of flow within the data center has a deadline, and accounted for most of the traffic data center TCP protocol can not provide Business Hours cutoff transmission known. This paper presents an improved TCP protocol deadline aware, said DATCP (Deadline-Aware TCP), provided by knowledge transfer service.DATCP according to the flow rate and the difference between the actual throughput expected to dynamically adjust its congestion control parameters used to load the data center, to meet the deadline demand flow.DATCP only make minor modifications to the standard TCP, and the use of existing commercial data general support of the center switch display ECN congestion notification mechanism.
(four) the virtual machine common virtualized data center migration scheduling problem. In front of all is to optimize traffic from the network point of view, this research field is common on the Internet. And in the data center network, traffic management and optimization to achieve a more direct effective direct control flow through the calculation of a distribution. The proper migration of virtual machine scheduling in order to minimize the duration of virtual machine migration. Multiple virtual machines (VM shuffle) at the same time migration scheduling problem is modeled as a scheduling optimization problem, the problem is equivalent to the traveling salesman problem. Because the problem is NP- hard, this paper designs two scheduling algorithm based on the off-line simulated annealing algorithm and greedy algorithm based on online mode. The basic idea of the two algorithms are calculated with an appropriate virtual machine scheduling, priority to other virtual machine migration brings the biggest The benefits of the virtual machine provide greater available bandwidth for subsequent migration virtual machines, so that the migration time of all virtual machines will be reduced, so as to minimize the impact of multi virtual machine migration on data centers.

【学位授予单位】：北京邮电大学
【学位级别】：博士
【学位授予年份】：2013
【分类号】：TP308;TP393.06

【参考文献】