高并发环境下异构数据源数据服务的研究与实现

发布时间：2019-01-11 10:55

【摘要】：随着科学技术的发展,社会的进步,越来越多的企业都开通了互联网业务,这也就导致这些企业需要拥有自己的数据存储管理系统。因此这些企业储存了大量的数据结构、数据格式各异的跟自己业务需求相关的异构数据源。大数据的时代随之到来,然而这些海量数据的异构性成为利用这些大数据的一大难题。找寻一种方法来屏蔽这些数据源的异构性是大数据运用的前提。随着互联网用户数的迅速增加,超大用户数的访问量对企业的服务器性能提出了极高的要求。单台服务器无论拥有多高的配置都无法永远满足日益增加的用户访问量,因此基于消息总线的异构数据服务是解决以上两个问题的良好办法。本文提出了一个异构数据源的集成方法,从而解决了异构数据源的数据异构问题。并且本论文还提出一种负载均衡算法帮助合理优化服务器集群的任务分配问题,从而使服务器集群的性能达到最优化,使得企业能够有效地处理大用户量的高并发访问。本文通过提出一种基于中间件的异构数据解析方法,用户针对中间件提供的统一数据类型进行数据查询,中间件将查询语句分解为针对不同数据源的查询语句,并通过封装器获取数据。其中统一数据类型的集成是中间件法的核心问题,因此本文将详细阐述异构数据类型到中间类型,中间类型再到统一数据类型的映射方法。并且本文通过提出一种基于消息总线的负载均衡技术来处理服务器集群任务分配不均的问题,将服务器集群中的所有服务器接入消息总线,每台服务器通过消息总线可以获取其他服务器的负载情况从而进行任务的动态分配。从而良好地解决了高并发的问题。在本论文最后进行了异构数据源集成地仿真,将一个关系型数据源和一个XML数据源进行了集成实现,并且使用ActiveMQ消息中间件实现了消息总线,将四台服务器接入消息总线实现了集群服务器负载均衡地模拟。
[Abstract]:With the development of science and technology and the progress of society, more and more enterprises have opened Internet business, which leads to the need for these enterprises to have their own data storage management system. Therefore, these enterprises store a large number of data structures, data formats are different from their own business requirements related to heterogeneous data sources. Big data's time has come, however, the heterogeneity of these massive data has become a big problem in taking advantage of the big data. To find a way to block the heterogeneity of these data sources is big data's premise. With the rapid increase of the number of Internet users, the number of visitors to the large number of users put forward a very high demand for the performance of the enterprise server. No matter how high the configuration of a single server is, it can not always meet the increasing number of users, so the heterogeneous data service based on message bus is a good way to solve the above two problems. In this paper, an integration method of heterogeneous data sources is proposed to solve the problem of heterogeneous data sources. This paper also proposes a load balancing algorithm to help optimize the task allocation of the server cluster so as to optimize the performance of the server cluster and enable the enterprise to deal with the high concurrent access of large users effectively. In this paper, a method of heterogeneous data parsing based on middleware is proposed. Users query the uniform data types provided by middleware, and the middleware decomposes the query statements into query statements for different data sources. And through the wrapper to obtain data. The integration of uniform data types is the core of the middleware method, so this paper will elaborate the mapping method of heterogeneous data types to intermediate types, intermediate types to unified data types. In this paper, a load balancing technique based on message bus is proposed to deal with the problem of uneven distribution of tasks in the server cluster, and all servers in the server cluster are connected to the message bus. Each server can obtain the load of other servers through the message bus to dynamically distribute the tasks. Thus, the problem of high concurrency is solved well. At the end of this paper, the heterogeneous data source is integrated into simulation, a relational data source and a XML data source are integrated, and the message bus is realized by using ActiveMQ message middleware. The four servers are connected to the message bus to realize the load balance simulation of the cluster server.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.13

【参考文献】