面向数据中心的RDMA高速网络服务通用平台
发布时间：2018-07-20 19:50
【摘要】：网络I/O一直是分布式应用的瓶颈。为此,相关研究者提出高速网络RDMA(Remote Direct Memory Access)技术。RDMA具有卸载TCP/IP协议栈和用户态下处理数据传输的工作方式,它首先在高性能计算领域得到广泛应用。随着其硬件价格的降低和RDMA over Ethernet的出现,数据中心也将RDMA引入。它的低延迟、高带宽、CPU轻负荷的特点,为许多传统分布式应用的设计提供了新思路,也对新兴的大数据处理、云计算服务等起到性能保障。现有的利用RDMA技术加速分布式系统的工作都是将RDMA网络模块与整个系统紧耦合。系统设计者通常以一个高速网络模块的角度来使用RDMA,试图来解决跨节点间的数据传输瓶颈。当将这些系统部署到数据中心或者云计算环境下,就会产生资源浪费、恶性竞争、较差的数据本地性、可扩展性不够等问题。此外,将RDMA网络模块与特定系统紧耦合这种方式,不但增加系统设计者的负担,还在系统维护成本、软件复用、大规模部署等方面有所损害。基于上述原因,我们设计并实现一个提供RDMA高速网络的通用平台RDMAVisor,以守护进程的方式存在,向应用程序提供简单易用的接口来调用RDMA高速网络服务。分布式应用设计者通过这些接口,来获得类似于系统服务一样的RDMA高速网络数据传输服务,从而能够专心于应用逻辑的设计和实现。本文的主要贡献有以下4点:(1)设计并实现面向数据中心的RDMA网络服务通用平台RDMAVisor;(2)对RDMA中各种服务类型和操作进行分析,以自适应调节参数的方式,为各种类型的分布式应用提供用户态下的数据传输服务;(3)通过复用RDMA连接实体,减轻甚至解决了RDMA的可扩展性问题;(4)针对RDMA原生操作如RDMA Remote Atomic的不足,RDMAVisor在守护进程层级向用户提供Remote Atomic操作,以更高效的方式替换掉RDMA原生操作。
[Abstract]:Network I / O has been the bottleneck of distributed applications. For this reason, researchers have proposed that the high-speed network RDMA (remote Direct memory Access) technology. RDMA has the ability to uninstall TCP / IP protocol stack and process data transmission in user state. It is widely used in the field of high performance computing. With the reduction of hardware price and the emergence of RDMA over Ethernet, the data center also introduces RDMA. Its characteristics of low latency and high bandwidth and low CPU load provide new ideas for the design of many traditional distributed applications, and also guarantee the performance of the emerging big data processing, cloud computing services and so on. The existing work of using RDMA technology to accelerate the distributed system is to tightly couple the RDMA network module with the whole system. System designers usually use RDMAs as a high-speed network module to try to solve the bottleneck of data transmission between nodes. When these systems are deployed in the data center or cloud computing environment, there will be problems such as waste of resources, vicious competition, poor data locality, poor scalability and so on. In addition, the tight coupling of RDMA network modules with specific systems not only increases the burden on system designers, but also damages system maintenance costs, software reuse, large-scale deployment, and so on. For the above reasons, we design and implement a general platform for RDMA high-speed network, RDMA Visor, which exists as a daemon and provides an easy to use interface to the application program to invoke the RDMA high-speed network service. Through these interfaces, distributed application designers can obtain RDMA high-speed network data transmission services similar to system services, so that they can concentrate on the design and implementation of application logic. The main contributions of this paper are as follows: (1) designing and implementing the general platform of RDMA network service for data center, (2) analyzing various service types and operations in RDMA, and adjusting the parameters adaptively. Provide data transmission services for various types of distributed applications in user mode; (3) by multiplexing RDMA connection entities, The extensibility of RDMA is alleviated or even solved. (4) RDMA native operations such as RDMA remote Atomic are not suitable for RDMA. RDMA Visor provides remote Atomic operations to users at the daemon level and replaces them with more efficient methods.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.0
[Abstract]:Network I / O has been the bottleneck of distributed applications. For this reason, researchers have proposed that the high-speed network RDMA (remote Direct memory Access) technology. RDMA has the ability to uninstall TCP / IP protocol stack and process data transmission in user state. It is widely used in the field of high performance computing. With the reduction of hardware price and the emergence of RDMA over Ethernet, the data center also introduces RDMA. Its characteristics of low latency and high bandwidth and low CPU load provide new ideas for the design of many traditional distributed applications, and also guarantee the performance of the emerging big data processing, cloud computing services and so on. The existing work of using RDMA technology to accelerate the distributed system is to tightly couple the RDMA network module with the whole system. System designers usually use RDMAs as a high-speed network module to try to solve the bottleneck of data transmission between nodes. When these systems are deployed in the data center or cloud computing environment, there will be problems such as waste of resources, vicious competition, poor data locality, poor scalability and so on. In addition, the tight coupling of RDMA network modules with specific systems not only increases the burden on system designers, but also damages system maintenance costs, software reuse, large-scale deployment, and so on. For the above reasons, we design and implement a general platform for RDMA high-speed network, RDMA Visor, which exists as a daemon and provides an easy to use interface to the application program to invoke the RDMA high-speed network service. Through these interfaces, distributed application designers can obtain RDMA high-speed network data transmission services similar to system services, so that they can concentrate on the design and implementation of application logic. The main contributions of this paper are as follows: (1) designing and implementing the general platform of RDMA network service for data center, (2) analyzing various service types and operations in RDMA, and adjusting the parameters adaptively. Provide data transmission services for various types of distributed applications in user mode; (3) by multiplexing RDMA connection entities, The extensibility of RDMA is alleviated or even solved. (4) RDMA native operations such as RDMA remote Atomic are not suitable for RDMA. RDMA Visor provides remote Atomic operations to users at the daemon level and replaces them with more efficient methods.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.0
【相似文献】
相关期刊论文 前10条
1 刘翔;罗菊;车军;;下一代传输网络通用平台[J];中国新通信;2007年05期
2 任东旺,田文德,姚飞;一种管理信息系统通用平台思想的提出[J];微型电脑应用;2003年03期
3 陈建;李剑峰;李方义;王玉玲;姜峰;马开\，
本文编号：2134589
本文链接：https://www.wllwen.com/guanlilunwen/ydhl/2134589.html
