分布式存储中的资源索引技术研究

发布时间：2018-01-14 04:12

本文关键词：分布式存储中的资源索引技术研究　出处：《浙江大学》2013年硕士论文　论文类型：学位论文

【摘要】：互联网的快速发展导致数据量的不断膨胀,单一节点的存储容量难以满足海量数据存储的要求,用户对容量巨大、安全、高效、可靠的分布式存储系统的需求也越来越大。传统的分布式存储系统通常部署在稳定可靠的节点上,难以有效利用P2P网络中大量的存储空间和计算资源。如何利用这些空闲的存储和计算资源以构建大容量、高可用的存储服务平台是当前分布式存储领域的研究热点。本文分析了P2P网络的特点以及当前分布式存储系统在资源检索方面的特点,提出并实现了一个基于信誉的倒排索引框架ROIN;根据DHT结构的特点,设计了一个基于分组和信誉的负载均衡分布式存储与资源索引框架LBGR。 ROIN框架通过双层DHT网络进行资源检索,实现了关键字模糊搜索,克服了传统DHT结构的不足；针对P2P网络中节点动态性强的特点,建立了节点信誉模型并计算节点的信誉值,在此基础上选取高信誉值的节点作为索引节点,从而避免了将索引信息存储在不可靠节点上,提高了系统的可靠性。通过搜索覆盖率、搜索延迟、索引节点负载和网络扰动等方面的实验,以及一系列与GlusterFS、 Pastry、PCIR的对比实验,证明了ROIN能够在P2P网络上建立一个健壮、有效的资源索引框架。由于ROIN是分布式的资源索引框架,同样存在数据一致性等普遍存在于分布式资源索引系统中的问题,本文提出了一种新型的资源索引框架LBGR。LBGR采用了分组和信誉相结合的资源管理策略,实现了基于DHT结构的资源索引技术。通过信誉系统的评估,LBGR框架选择高信誉值的节点作为超级节点并对恶意节点进行有效遏制；通过基于虚拟节点的负载均衡策略,解决了DHT系统中普遍存在的负载均衡问题。模拟实验验证了所提出的负载均衡算法的有效性,其结果可与集中式负载均衡算法相比拟。通过与GlusterFS的比较分析,LBGR显示了在应对P2P网络节点动态性方面的优势。上述两种不同的资源索引框架均能运行在P2P网络上,各具优势,并通过实验证明能提供可靠高效的服务,具有实际应用价值。
[Abstract]:The rapid development of the Internet leads to the continuous expansion of the data quantity , the storage capacity of a single node is difficult to meet the requirements of the mass data storage , and the requirement of the user to the distributed storage system with large capacity , safety , high efficiency and reliability is more and more large . Traditional distributed storage systems are usually deployed on a stable and reliable node , so that a large amount of storage space and computing resources in the P2P network are difficult to effectively utilize . This paper analyzes the characteristics of P2P network and the characteristics of current distributed storage system in resource retrieval , proposes and implements a reputation - based inverted index framework ROIN , designs a load balance distributed storage and resource index framework LBGR based on packet and reputation according to the characteristics of DHT structure . In order to overcome the disadvantages of traditional DHT structure , a node reputation model is established and the node ' s reputation value is calculated , and a series of experiments with GlusterFS , Pastry and PCIR prove that ROIN can set up a robust and effective resource index framework on P2P network . Because ROIN is a distributed resource index framework , there is also a problem that data consistency exists in distributed resource indexing system . In this paper , a new resource index framework LBGR.LBGR is proposed . A new resource index technology based on DHT is presented . Based on the evaluation of reputation system , the LBGR framework selects the node with high reputation value as the super node and effectively restrains the malicious node . The simulation experiment verifies the effectiveness of the proposed load balancing algorithm . The results can be compared with the centralized load balancing algorithm . By comparison with GlusterFS , LBGR shows the advantages of the P2P network node dynamics . the two different resource index frames can be operated on the P2P network , the advantages are advantageous , and a reliable and efficient service can be provided through experiments to have practical application value .

【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】