基于天河2高速互连网络实现混合层次文件系统H~2FS高速通信

发布时间：2018-04-08 09:04

本文选题：混合层次文件系统　切入点：H~FS　出处：《计算机学报》2017年09期

【摘要】：高效通信性能是影响并行文件系统性能与效率的重要因素.该文基于TH-2系统的高性能互连网络——天河2高速互连网络(TH-Express 2),设计并实现了混合层次文件系统H~2FS中的高速通信模块FSE.FSE采用了TH-Express 2的短报文通信MP和远程内存访问RDMA相结合的方式,实现客户端与ION的通信.FSE采用基于动态链表的内存注册池减少通信延迟,提高通信效率.基于可变信用的流量控制,FSE有效降低系统出现拥塞的可能性,提高系统的可扩展性和稳定性.为了充分利用多核处理器的并发处理能力以及多RDMA引擎的传输能力,FSE对报文传输和数据处理功能采用了多维多线程方式,提高数据访问带宽.FSE的自适应端点管理针对通信端点容错能力设计,提高了系统弹性,有助于增强系统稳定性.FSE实现优化了数据传输效率,实现了低延迟、高带宽、高可扩展数据访问.在两个不同实际系统上的测试结果表明,FSE可以充分发掘TH-Express 2的特性,点点数据访问带宽可达8.6Gbps,使得H~2FS具有较高的数据访问带宽和元数据性能,并具有良好的可扩展性.同基于TCP的通信模块实现相比,FSE的读写延迟可低至55%和20%,最大单ION写性能是其3.3倍;同Lustre文件系统相比,FSE更好地利用了高速网的特性来获取较高性能,写延迟可低至其28.6%.
[Abstract]:Efficient communication performance is an important factor affecting the performance and efficiency of parallel file systems.Based on the high performance interconnection network of TH-2 system, Tianhe2 High Speed Interconnect Network (TH-Express 2) is designed and implemented in this paper. The high speed communication module FSE.FSE of hybrid hierarchical file system H~2FS uses TH-Express 2 short message communication MP and remote memory access.Ask about the way RDMA is combined,In order to realize the communication between client and ION, the memory register pool based on dynamic linked list is used to reduce the communication delay and improve the communication efficiency.FSE based on variable credit can effectively reduce the possibility of congestion and improve the scalability and stability of the system.In order to make full use of the concurrency processing ability of multi-core processor and the transmission capability of multi-#en0# engine, the multi-dimensional multithreading mode is adopted for message transmission and data processing.The adaptive endpoint management for improving the bandwidth of data access. FSE is designed for the fault-tolerant ability of communication endpoints, which improves the flexibility of the system, enhances the stability of the system, optimizes the efficiency of data transmission, and realizes low delay and high bandwidth.High extensible data access.The test results on two different systems show that FSE can fully exploit the characteristics of TH-Express 2, and the point data access bandwidth can reach 8.6 Gbps. it makes H~2FS have higher data access bandwidth and metadata performance, and has good scalability.Compared with the implementation of the communication module based on TCP, the delay of reading and writing can be as low as 55% and 20%, and the maximum performance of single ION is 3.3 times higher than that of the Lustre file system. Compared with the Lustre file system, FSE makes better use of the characteristics of high-speed network to obtain higher performance, and the write delay can be as low as 28.6%.
【作者单位】：国防科学技术大学计算机学院;高性能计算国家重点实验室;中山大学国家超算广州中心;
【基金】：国家自然科学基金(61120106005) 国家“八六三”高技术研究发展计划项目基金(2012AA01A301) 国家重点研发计划(2016YFB0200400) 科技部云计算与大数据重大专项(2016YFB1000302)资助~~
【分类号】：TP338

【相似文献】