流程工业分布式实时数据库研究与应用

发布时间：2018-02-12 07:47

本文关键词： 流程工业分布式实时数据库分布式自组织理论动态存储全局索引机制分布式并行处理框架　出处：《浙江大学》2016年博士论文　论文类型：学位论文

【摘要】：随着产业结构优化和集中度提升,流程工业企业的生产规模不断扩大,传统的集中式实时数据库由于数据吞吐量不足、投资成本过高、容错能力较低、难以弹性扩展等问题,已无法满足实际应用需求。针对上述问题,本论文将大规模分布式系统设计思想与实时性、可靠性技术相结合,建立流程工业分布式实时数据库的服务模型,研究海量数据的动态存储索引机制、事务建模与并行处理、高可用性网络实时通信等技术,并实现研究成果在实际工业项目中的应用验证。具体理论研究和技术应用成果如下:(1)在对流程工业应用场景的需求特点进行深入分析的基础上,提出了分布式实时数据库的服务模型作为系统的基础架构,以及基于中心化组织结构和中心节点主从机制的实时数据库分布式自组织理论。提出了基于Raft协议的主从中心节点选举策略,结合服务发现模型、服务部署策略的研究与设计,实现了实时数据库服务内部与系统整体两个层级的自组织。(2)针对流程工业的存储应用场景特点,提出了基于工厂逻辑模型的动态全局索引机制,以及基于二级分页和多版本管理的本地索引机制。重点对数据分区分片策略、节点增删处理方案、数据多重副本机制和存档文件压缩方案进行了研究,解决了异常状态下数据拟合错误和历史数据占用磁盘空间过高的问题,实现了存储系统的可靠性和扩展能力。(3)深入分析流程工业分布式任务请求的特点,建立了分布式实时数据库的事务模型。基于该模型,研究并实现了相应的分布式并行查询模型和分布式并行计算服务,用于解决分布式事务处理面临的事务分解、并行调度、资源管理等问题。提出了分布式实时混合并发控制算法,综合利用多版本并发控制与乐观并发控制的无阻塞特性,能够优化分布式事务执行的实时性。(4)提出了高可用性网络实时通信服务的层次结构,通过冗余连接会话、数据转发通道、网络服务自组织等多种机制和策略,提高了流程工业网络的可用性。设计了针对流程工业网络的时钟同步机制,能够有效处理网络时延抖动和服务器单点失效问题。同时,提出了网络传输数据加密保护方案和基于数字证书的用户身份认证机制,提升了网络信息安全水平。本论文完成的研究成果有效地解决了流程工业实时数据库的基础架构、大容量、实时性、可用性、并发访问等难点问题,并通过系统测试和实际应用案例验证了其功能和性能指标项,应用结果表明本论文的研究成果能够满足大规模流程工业生产的实际需求。
[Abstract]:With the optimization of industrial structure and enhance the degree of concentration, enterprise production scale is expanding constantly, the traditional centralized real-time database due to insufficient data throughput, the investment cost is too high, fault tolerance is low, it is difficult to elastic expansion and other issues, has been unable to meet the needs of practical application. To solve the above problems, this paper will design and real-time large-scale distributed systems reliability, technology, service model of distributed process industrial real-time database, dynamic indexing mechanism of massive data storage, parallel processing and transaction modeling, high availability network real-time communication technology, and realizes the application of the research results are verified in actual industrial projects. The specific theoretical research and technical application of the results are as follows: (1) in-depth analysis based on the demand characteristics of the application of process industry, put forward a distributed real-time database Service model as the basis for the system architecture, and distributed real-time database structure and mechanism of the center center master-slave node based on self-organization theory. Proposed master-slave center node selection strategy based on Raft protocol, combined with the model of service discovery, service deployment research and design strategy, realize the self organization internal and overall system real-time database service two a level. (2) for storage application scenarios of the characteristics of process industry, put forward the dynamic global indexing mechanism based on logical model of plant, and based on the local index system of two level paging and multi version management. Focusing on the data partition strategy, data processing scheme of node deletions, multiple copy mechanism and archive file compression scheme the study, to solve the problem of high disk space occupied under abnormal condition, data fitting error and historical data, realize the storage system System reliability and scalability. (3) analyzes the characteristics of distributed task process industry request, established the transaction model of distributed real-time database. Based on this model, the research and implementation of the corresponding distributed parallel query service and distributed parallel computing model, is used to solve the distributed transaction processing face transaction decomposition, parallel scheduling, resource management other issues. We put forward a real-time hybrid concurrency control algorithm for distributed, comprehensive utilization of multi version concurrency control and optimistic concurrency control of non blocking characteristics, to optimize the distributed transaction execution in real time. (4) put forward a hierarchical structure of real-time communication service high availability network connection, session through redundancy, data transmission channel, network service self organization and other mechanisms and strategies to improve the availability of network of process industry. The design of clock synchronization mechanism for the network process industry, can Enough to effectively deal with network delay jitter and server failure problem. At the same time, the network transmission of data encryption scheme and user identity authentication mechanism based on digital certificate, enhance network information security level. The results of this study can effectively solve the completed infrastructure, process industrial real-time database capacity, real-time, availability, concurrency and other difficult problems, and through the system test and practical application verifies the function and performance indexes. The application results show that the research results of this paper can satisfy the actual demand of large-scale industrial production process.

【学位授予单位】：浙江大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP311.13

【参考文献】