分布式服务与服务集群技术研究
发布时间：2018-11-18 14:54
【摘要】：随着互联网技术的进一步的发展与深入,越来越多的企业应用需要应对百万级的高并发用户访问、低延时及7×24×365高可用性等空前的需求难题,应用系统的业务访问量与日俱增,对应用服务器和数据库的要求越来越高。在Web站点以及云平台中存在大量半结构化数据,传统的关系型数据库难以适应半结构化的数据存储访问模式,例如用户Session信息。在面临大规模数据访问时,RAM与ROM在读写速度上相差数个数量级,磁盘的IO操作会堆积RAM中的数据读写请求,造成性能瓶颈,从而导致越来越高的访问响应延迟。分布式缓存使用系统的高速内存(RAM)作为数据对象的存储介质,并采用键值对的形式存储数据。这样就可以在价格低廉、标准化的硬件和软件平台上的扩展系统的内存,专门用于缓存海量的数据,为大容量、业务关键型的事务处理提供业务支撑。随着业务需求的进一步发展,应用规模在不断的扩大,巨无霸应用也就越来越多,传统应用的垂直架构,已经无法应对各种复杂业务带来的各种挑战。只有进行业务梳理,将公共能力抽取成基础服务或公共服务,对复杂应用进行模块化拆分和服务化治理,降低服务消费者与服务提供者之间的耦合度。对采用了传统的垂直架构的系统应用进行服务治理化改造,将应用的各个部分的业务进行服务抽取和分离,形成一个个相对独立的服务节点,其中的每个服务节点只处理一件或多件相关的同类型的业务,如单点登陆服务器,只处理用户登陆相关的业务。应对百万级的并发访问、每秒数千的并发事务处理时,任何单个服务节点都不能做到及时响应。这个时候,考虑的不是增加单个服务器的性能,而是需要对高访问量、高并发量的服务节点架设一个或多个服务器集群分担业务压力,形成多台服务器提供相同的服务,并相互之间相互独立的服务器集群,最终实现整个应用的垂直架构平滑地向分布式服务框架演变。本论文选题来自于本人曾经实习的亚信科技有限公司的服务治理项目组,灵感来源于其对北京移动CRM系统服务化改造的北京移动能力开放平台项目,能力开放平台通过将业务支撑系统业务能力的逐步有序开放与管控,实现内部业务能力对内、外部渠道的共享以及开发者和运营者的利益双赢,使得电信业务开发变得更为容易和高效,促进移动互联网业务的发展和繁荣,支持业务能力持续创新。本文旨在研究和探讨,如何实现通过服务治理的方式将传统应用的垂直架构方案演变成分布式服务架构方案,以达到解耦服务提供方和服务使用方之间的耦合关系的目的,并通过分布式缓存、分布式服务、服务器集群、消息中间件、服务注册中心、数据库分库分表、负载均衡、系统业务分拆等技术手段应对高访问量、高并发量、高可用等方面的应用需求。
[Abstract]:With the further development and deepening of Internet technology, more and more enterprise applications need to deal with the unprecedented difficulties of high concurrent user access, low latency and 7 脳 24 脳 365 high availability, and the number of business visits to the application system is increasing day by day. Application servers and databases are increasingly demanding. There is a large amount of semi-structured data in Web sites and cloud platforms, and traditional relational databases are difficult to adapt to semi-structured data storage access patterns, such as user Session information. When faced with large-scale data access, RAM and ROM differ several orders of magnitude in read and write speed. The IO operation of disk will pile up data read and write requests in RAM, resulting in performance bottleneck, which leads to higher and higher access response delay. The distributed cache uses the high speed memory (RAM) of the system as the storage medium of the data object and stores the data in the form of key-value pairs. In this way, we can extend the memory of the system on the cheap, standardized hardware and software platform, specially used to cache the massive data, and provide the service support for the transaction processing of the large capacity and business critical type. With the further development of business requirements and the continuous expansion of application scale, there are more and more Big Mac applications. The vertical structure of traditional applications can no longer cope with various challenges brought by complex business. Only by combing the business, extracting the public capacity into basic services or public services, the complex applications are divided into modular and service-oriented governance, so as to reduce the coupling degree between service consumers and service providers. The system application with traditional vertical architecture is transformed into service governance, and the services of each part of the application are extracted and separated to form a relatively independent service node. Each service node only handles one or more related services, such as single-point login server, and only deals with user logon related services. For millions of concurrent access and thousands of concurrent transactions per second, no single service node can respond in a timely manner. At this time, the consideration is not to increase the performance of a single server, but to set up one or more server clusters with high traffic and high concurrency to share the business pressure, resulting in multiple servers providing the same service. Finally, the vertical architecture of the whole application evolves smoothly to the distributed service framework. The topic of this thesis comes from the service management project team of Asense Technology Co., Ltd., which I once worked as an intern. The inspiration comes from the Beijing Mobile open platform project for the service transformation of Beijing Mobile CRM system. The open platform realizes the sharing of internal and external channels and the interests of developers and operators through the gradual and orderly opening and control of the business capabilities of the business support system. It makes telecom service development easier and more efficient, promotes the development and prosperity of mobile Internet service, and supports continuous innovation of service capability. The purpose of this paper is to study and discuss how to transform the vertical architecture scheme of traditional application into a distributed service architecture scheme through service governance, in order to decouple the coupling relationship between service provider and service consumer. And through distributed cache, distributed services, server cluster, message middleware, service registry, database table, load balancing, system business separation and other technical means to deal with high traffic, high concurrency, etc. High availability and other application requirements.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.092
本文编号：2340349
[Abstract]:With the further development and deepening of Internet technology, more and more enterprise applications need to deal with the unprecedented difficulties of high concurrent user access, low latency and 7 脳 24 脳 365 high availability, and the number of business visits to the application system is increasing day by day. Application servers and databases are increasingly demanding. There is a large amount of semi-structured data in Web sites and cloud platforms, and traditional relational databases are difficult to adapt to semi-structured data storage access patterns, such as user Session information. When faced with large-scale data access, RAM and ROM differ several orders of magnitude in read and write speed. The IO operation of disk will pile up data read and write requests in RAM, resulting in performance bottleneck, which leads to higher and higher access response delay. The distributed cache uses the high speed memory (RAM) of the system as the storage medium of the data object and stores the data in the form of key-value pairs. In this way, we can extend the memory of the system on the cheap, standardized hardware and software platform, specially used to cache the massive data, and provide the service support for the transaction processing of the large capacity and business critical type. With the further development of business requirements and the continuous expansion of application scale, there are more and more Big Mac applications. The vertical structure of traditional applications can no longer cope with various challenges brought by complex business. Only by combing the business, extracting the public capacity into basic services or public services, the complex applications are divided into modular and service-oriented governance, so as to reduce the coupling degree between service consumers and service providers. The system application with traditional vertical architecture is transformed into service governance, and the services of each part of the application are extracted and separated to form a relatively independent service node. Each service node only handles one or more related services, such as single-point login server, and only deals with user logon related services. For millions of concurrent access and thousands of concurrent transactions per second, no single service node can respond in a timely manner. At this time, the consideration is not to increase the performance of a single server, but to set up one or more server clusters with high traffic and high concurrency to share the business pressure, resulting in multiple servers providing the same service. Finally, the vertical architecture of the whole application evolves smoothly to the distributed service framework. The topic of this thesis comes from the service management project team of Asense Technology Co., Ltd., which I once worked as an intern. The inspiration comes from the Beijing Mobile open platform project for the service transformation of Beijing Mobile CRM system. The open platform realizes the sharing of internal and external channels and the interests of developers and operators through the gradual and orderly opening and control of the business capabilities of the business support system. It makes telecom service development easier and more efficient, promotes the development and prosperity of mobile Internet service, and supports continuous innovation of service capability. The purpose of this paper is to study and discuss how to transform the vertical architecture scheme of traditional application into a distributed service architecture scheme through service governance, in order to decouple the coupling relationship between service provider and service consumer. And through distributed cache, distributed services, server cluster, message middleware, service registry, database table, load balancing, system business separation and other technical means to deal with high traffic, high concurrency, etc. High availability and other application requirements.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.092
【参考文献】
相关博士学位论文 前2条
1 王红斌;Web服务器集群系统的自适应负载均衡调度策略研究[D];吉林大学;2013年
2 卢本捷;分布式消息队列的理论、实现与应用[D];华中科技大学;2004年
相关硕士学位论文 前10条
1 张羽;一种分布式服务治理框架的设计与实现[D];北京交通大学;2016年
2 刘先纺;基于Dubbo的网上订餐系统[D];华南理工大学;2016年
3 张永祥;电子商务系统平台模型的研究与实现[D];电子科技大学;2016年
4 冯博;基于PaaS的电信虚拟运营商系统设计和实现[D];中国科学院大学(工程管理与信息技术学院);2015年
5 王利萍;基于Nginx服务器集群负载均衡技术的研究与改进[D];山东大学;2015年
6 侯yN寒;电信能力开放平台中鉴权框架的设计与实现[D];北京邮电大学;2015年
7 崔解宾;分布式内存缓存技术在数据处理平台中的研究与应用[D];北京邮电大学;2015年
8 闫明;高可用可扩展集群化Redis设计与实现[D];西安电子科技大学;2014年
9 沈伟;云计算平台下分布式缓存系统的性能优化研究[D];西安电子科技大学;2014年
10 吴伟华;基于分布式缓存的内容分发网络[D];西安电子科技大学;2014年
，本文编号：2340349
本文链接：https://www.wllwen.com/guanlilunwen/ydhl/2340349.html
