基于MPI的高并发负载生成系统的研究与实现

发布时间：2018-05-22 17:14

本文选题：高并发负载生成 + 压力测试系统　；参考：《中山大学》2014年硕士论文

【摘要】：压力测试是检验Web应用QOS（Quality of Service）的一种重要方法，测试所需的高并发请求需要借助计算机程序来生成。随着云计算时代的到来，后端服务器的性能得到空前的提升，传统的负载生成方案已经越来越不能满足现代云端服务器压力测试的需求。已有的负载生成方案中，多线程负载生成法由于技术成熟且稳定性好，被大多数商业和开源系统所采用。然而这种每个线程管理一个连接的方式在并发连接数很高时，会因为频繁的线程上下文切换而导致性能急剧恶化。另一类基于异步I/O的负载生成方法虽然能够高效地产生大量负载，但是这种方案通常只使用了单个线程处理所有连接而没能充分发挥多核CPU的处理能力。使用多核优化异步I/O负载生成的设想虽然已经存在，，但是缺少完整的工程实践和有效性证明。另一方面，现代压力测试系统的垂直扩展能力的重要性也与日俱增，如何提高系统的可扩展性也是当前该领域研究的重要课题。本文首先梳理了前人工作中与压力测试系统以及负载生成系统相关的研究，分析总结了已有解决方案的基本原理以及研究结论，归纳出已有的工作中存在的不足以及缺陷。在这样的基础和背景下，本文提出了基于MPI技术的负载生成系统PBench的设计和实现。PBench在使用事件驱动式的异步I/O负载生成技术的基础上，利用MPI并行处理技术在多核以及分布式环境下实现任务级并行。充分发挥多核CPU的性能，提高单个计算节点负载生成的能力，同时也使得系统具备了很强的垂直扩展能力，能够方便地实现集群化。本文详细地介绍了PBench的设计思想以及所用到的关键技术，讲述了PBench的设计要点和实现方式，并通过详尽的实验来证明PBench这种并行化设计确实能够充分发挥多核CPU的处理能力，提高单机负载生成的效率。
[Abstract]:Stress testing is an important method to test Web application QOS(Quality of Service. The high concurrency request for testing needs to be generated by computer program. With the arrival of cloud computing era, the performance of back-end servers has been improved unprecedented, the traditional load generation scheme has become more and more unable to meet the needs of modern cloud server stress testing. Among the existing load generation schemes, multithreaded load generation method has been adopted by most commercial and open source systems because of its mature technology and good stability. However, when the number of concurrent connections is high, the performance of each thread managed by one connection can deteriorate sharply because of frequent thread context switching. The other kind of load generation method based on asynchronous I / O can efficiently generate a large amount of load, but this scheme usually only uses a single thread to handle all connections without fully exerting the processing power of multi-core CPU. Although the idea of multi-core optimization for asynchronous I / O load generation is already in existence, there is a lack of complete engineering practice and validation. On the other hand, the vertical expansion ability of modern stress test system is becoming more and more important, and how to improve the scalability of the system is also an important topic in this field. In this paper, the previous researches related to the stress test system and the load generation system are reviewed, the basic principles and conclusions of the existing solutions are analyzed and summarized, and the shortcomings and defects in the existing work are summed up. On this basis and background, this paper presents the design and implementation of PBench, a load generation system based on MPI technology, based on the use of event-driven asynchronous I / O load generation technology. Task-level parallelism is implemented in multi-core and distributed environments using MPI parallel processing technology. The performance of multi-core CPU is brought into full play to improve the load generation ability of a single computing node. At the same time the system has a strong vertical expansion ability and can easily realize clustering. This paper introduces the design idea and key technology of PBench in detail, describes the key points and implementation methods of PBench, and proves that the parallel design of PBench can give full play to the processing ability of multi-core CPU through detailed experiments. Improve the efficiency of single machine load generation.
【学位授予单位】：中山大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.09

【参考文献】