虚拟化环境下面向分布式非关系型数据管理的通信优化系统

发布时间：2018-05-27 21:37

本文选题：副本 + 通信优化　；参考：《华中科技大学》2013年硕士论文

【摘要】：云计算和虚拟化技术的兴起，使在虚拟机中做海量数据存储和管理受到了越来越多人的关注，Cassandra作为一个高性能分布式非关系型数据管理系统，在虚拟化环境下面临着新的通信问题。对于读请求，从相同物理机上虚拟机的读效率要高于不同物理机上虚拟机的读效率；对于写请求，需要将多个副本写到不同物理机的虚拟机上，避免物理机宕机，以至所有的副本不可访问。虚拟化环境下面向分布式非关系型数据管理的通信优化系统，首先是虚拟机位置感知，识别各个虚拟机之间的位置关系，为了避免单节点故障问题，采用了P2P的Gossip协议，每个物理机获取该节点上虚拟机的位置列表，，通过Gossip将数据传播到其他的节点，每个节点保存所有虚拟机的位置信息；其次使用thrift发布位置信息，并且对数据重新组织和划分，方便快速获取所有虚拟机的位置信息；然后是写节点的选择，当收到写请求时，选择不在同一物理机上的虚拟机节点作为写节点，避免任意物理机宕机造成的数据不可访问；最后是最优读节点的选择，当收到用户的读请求时，使用修改后的节点选择算法，优先选择距离最近的节点读取数据，从而提高读效率。功能测试表明，虚拟化环境下面向分布式非关系型数据管理的通信优化系统能够选择正确的节点写；任意关闭一台物理机，数据都可以读取，数据的可访问性得到提高。性能测试表明，对于优化后的系统，TPS提高了23%左右。
[Abstract]:With the rise of cloud computing and virtualization technology, more and more people pay attention to mass data storage and management in virtual machines. Cassandra is a high-performance distributed non-relational data management system. In the virtualization environment, we are facing new communication problems. For the read request, the reading efficiency of the virtual machine from the same physical machine is higher than that of the different physical machine virtual machine; for the write request, it is necessary to write multiple copies to the virtual machine of the different physical machine to avoid the physical machine downtime. Even all copies are not accessible. In order to avoid the problem of single node fault, the communication optimization system for distributed non-relational data management in virtualization environment is based on the location perception of virtual machine and the recognition of location relationship among virtual machines. In order to avoid the problem of single node fault, P2P Gossip protocol is adopted. Each physical machine obtains a list of locations of virtual machines on the node, propagates the data to other nodes through Gossip, and each node stores the location information of all virtual machines. Secondly, the location information is published using thrift, and the data is reorganized and partitioned. It is convenient to obtain the location information of all virtual machines quickly, and then the selection of write nodes. When the write request is received, the virtual machine nodes that are not on the same physical machine are selected as write nodes, so that the data caused by arbitrary physical outages can not be accessed. Finally, the optimal reading node is selected. When the user's request is received, the modified node selection algorithm is used to select the nearest node to read the data, so as to improve the reading efficiency. The function test shows that the communication optimization system for distributed non-relational data management in virtualization environment can select the correct node write, turn off a physical machine at any time, the data can be read, and the data accessibility is improved. The performance test shows that the TPS of the optimized system is improved by about 23%.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP302

【共引文献】