基于PostgreSQL存储引擎的多线程化方法研究与实现

发布时间：2018-05-31 20:02

本文选题：PostgreSQL + 存储引擎　；参考：《华中科技大学》2012年硕士论文

【摘要】：一个高效的存储引擎对数据库系统的性能提升具有重要意义。本论文针对开源数据库管理系统PostgreSQL已有存储引擎的结构上的缺陷，提出一种将多进程结构改造为多线程结构的解决方法。 PostgreSQL是一个多进程的数据库管理系统，每当有用户连接到系统后，就创建一个新进程处理这个用户的连接，用户连接断开后该进程被销毁。进程的创建，调度和销毁的开销都很大，比线程的对应操作开销大很多，为了合理利用系统资源，应该使用多线程的结构。本文对存储引擎多进程结构改造为多线程结构所涉及的几个方面进行了分析与实现。对全局变量的改造提出线程本地存储等方法并讨论了方法之间的优劣，为线程控制改造引入Boost第三方库并设计了跨平台的替换方案，分析了进程通信机制采用的共享消息队列方法的原理，讨论了解决进程资源限制所使用的虚拟文件描述符机制。基于上述分析，对存储引擎多线程结构改造的具体实现进行了较为详细的论述。使用消息队列结合线程信息的方法实现了线程通信机制，将启动后台服务进程分离为启动存储引擎主线程和启动后台服务线程，把后台写进程改造成以线程方式启动，对其正常情况和异常情况下的实现流程做了较详尽的阐述。所有单线程和多线程测试用例在Windows和Linux平台下都通过，，存储引擎多线程改造基本成功。本文研究结果表明，多线程结构的存储引擎能够节省系统资源、充分发挥多核处理器的计算能力，提升数据库系统整体性能，如果进一步实现事务与线程脱离，还可以大幅提高分布式数据库系统的吞吐量。
[Abstract]:An efficient storage engine is of great significance to improve the performance of database systems. In order to overcome the structural defects of the existing storage engine in open source database management system (PostgreSQL), this paper proposes a solution to transform the multi-process structure into multi-thread structure. PostgreSQL is a multi-process database management system. Whenever a user is connected to the system, a new process is created to handle the user's connection, and the process is destroyed after the user's connection is disconnected. The cost of process creation scheduling and destruction is much higher than the corresponding operation cost of threads. In order to make rational use of system resources the multi-thread structure should be used. This paper analyzes and implements several aspects involved in the transformation of multi-process structure of storage engine into multi-thread structure. The methods of thread local storage are put forward and the advantages and disadvantages of the methods are discussed. The Boost third-party library is introduced into the thread control transformation and a cross-platform replacement scheme is designed. This paper analyzes the principle of shared message queue method used in process communication mechanism, and discusses the virtual file descriptor mechanism used to solve process resource limitation. Based on the above analysis, the implementation of multi-thread structure reconstruction of storage engine is discussed in detail. The thread communication mechanism is realized by using message queue combined with thread information. The starting background service process is separated into the main thread of starting storage engine and the starting background service thread, and the background write process is transformed to start by thread. Its normal and abnormal conditions of the implementation of a more detailed description of the flow. All single-threaded and multi-threaded test cases are passed on Windows and Linux platforms, and the storage engine multithreading transformation is basically successful. The results show that the multi-thread storage engine can save system resources, give full play to the computing power of multi-core processors, and improve the overall performance of database system. It can also greatly improve the throughput of distributed database systems.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP311.13;TP333

【参考文献】