MiniQQ项目的数据存储模块的设计与实现

发布时间：2018-01-13 17:11

本文关键词：MiniQQ项目的数据存储模块的设计与实现　出处：《哈尔滨工业大学》2012年硕士论文　论文类型：学位论文

【摘要】：当今的互联网应用呈现出海量数据存储、随机访问和数据库查询频度非常高，而数据更新的频度相对较低的特点。人们对互联网应用的并发性能也提出了更高的要求。面对这些挑战，传统的数据库服务器已经不能满足当前互联网的需求。因此，人们开始考虑使用何种技术与架构来解决这些问题，由此带来了一些新的技术问题：海量数据存储、负载均衡、数据库复制、读写分离、透明访问等。作为一个大型的互联网应用公司，为了给用户提供更好的优质体验，在日常开发中必须解决这些问题，并进行长期的优化。因此公司提出了这样一个mini项目来对此进行研究，为日后的工作与深入研究打下基础。本文首先对整个项目的整体需求进行了系统分析，明确了本课题在实现基本功能的基础上还应解决的核心问题，如海量数据存储、高并发访问、数据一致性、容灾备份等。本文对国内外现状进行研究之后，结合公司的成型技术，在MiniQQ项目的存储模块设计中，设计并实践了一套解决方案，实现了整个系统的基本功能需求，实现了对海量数据存储的支持，能够支持一定量级的高并发访问，确保了用户数据的一致性与容灾。最后通过功能与性能测试证明了该解决方案的可行性与正确性。本文主要围绕这套解决方案的设计与实现进行分析描述，主要包含了整体架构的设计，相关技术的介绍，，数据分片技术在海量数据存储中的应用，复制技术、负载均衡、读写分离在高并发访问和数据一致性上的应用，内存数据库在数据存储模块中的使用。
[Abstract]:In today's Internet Applications presents a massive data storage, random access and database query frequency is very high, and the characteristics of frequency of data update is relatively low. Concurrent performance of people on the Internet application is also put forward higher requirements. In the face of these challenges, the traditional database server has been unable to meet the current needs of the Internet. Therefore, people to solve these problems began to consider what kind of technology and architecture, which brings some new technical problems: massive data storage, load balancing, database replication, read-write separation, transparent access and so on. As a large Internet App Co, in order to provide a better quality of experience, in the day-to-day development of these problems must be solved for a long time, and optimization. So the company proposed to study such a mini project, and further study on after work Lay the foundation.
Firstly, the overall demand for the entire project to carry on the system analysis, the key issue of this paper should also solve on the basis of the realization of the basic functions of the sea quantity data storage, high concurrency, data consistency, disaster recovery and so on. Then this paper makes a research on the status quo at home and abroad, combined with the molding technology company, in the the design of storage module in MiniQQ project design and practice a set of solutions to achieve the basic functional requirements of the whole system, the realization of mass data storage, high concurrency can support a certain magnitude of the visit, to ensure consistency and recovery of user data. Finally, functional testing and performance testing to prove the feasibility of the the solution and correctness.
This paper mainly focuses on the design and implementation of this solution is described, including the overall architecture design, related technology, application, data slicing technology in mass data storage in the replication technology, load balancing, read-write separation in the application of high concurrent access and data consistency of the database, the use of memory in the data storage module.

【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【参考文献】