面向银行业务的数据平台的设计与实现

发布时间：2018-08-02 08:44

【摘要】：随着经营管理水平的不断提升,黑龙江省农村信用社在客户管理、风险管理、运营管理、财务管理、监管合规等多个业务领域的需求越来越多,对数据的使用提出了更高的要求。黑龙江省农村信用社经过多年的系统建设,积累了大量的业务数据,然而,由于早期农信社的各系统的建设是相对独立的,缺乏完善的数据管控体系以及对数据的统一管理,导致了农信社在数据存储中出现了没有统一的数据源、业务数据分散、各业务系统数据不一致、数据质量不高、安全性低等问题。现有的分散的数据不能满足企业级精细化管理和个性化营销的需求。因此需要建立数据平台,来挖掘并发挥企业数据的最大价值。本课题的研究正是着眼于目前农信社数据存储的弊端,着重研究了数据平台系统在农信社系统中的实际应用。本文以黑龙江省农村信用社数据平台项目为实际项目背景,主要阐述了数据平台项目一期基础平台的设计与实现过程。通过对农信社历史数据及实际业务经营情况的分析得出了黑龙江省农村信用社数据平台数据存储模型,整理并制定了统一的业务数据标准格式,并采用OLAP(On-Line Analytical Processing,联机在线分析处理)、透明网关等技术设计并完成了所有业务数据自源系统生成至存储到数据模型过程中所涉及的数据采集、数据集成加工等功能模块,以及业务数据自数据模型中提取出来到展现在最终用户的过程中涉及的数据聚合,数据展现及分发等功能模块,并且针对数据平台内全部业务数据流转的E(Extract)、T(Transformation)、L(Load)过程各个环节的实际情况,采用了有针对性的处理方式完成了ETL调度模块的设计与实现。通过详细测试本平台上游各业务系统的数据能够成功的加载到数据平台系统中,而且进入数据平台之后经过数据加工处理,数据平台中各层之间的数据比对正确;系统的调度平台能够按照各工作流的预先配置自动发起数据抽取、转换、加载等任务,对业务数据进行批量处理,并且能够完成对任务处理的监控与统计,很好的完成数据平台项目的需求。
[Abstract]:With the continuous improvement of management level, the demand of Heilongjiang rural credit cooperatives in many business areas, such as customer management, risk management, operation management, financial management, regulatory compliance, and so on, has put forward higher requirements for the use of data. The rural credit cooperatives in Heilongjiang province have accumulated a large amount of business after years of system construction. Data, however, due to the relatively independent construction of the early agricultural credit cooperatives, the lack of perfect data management system and the unified management of data, the agricultural credit cooperatives in the data storage have no unified data sources, business data dispersion, different business system data, data quality is not high, low security and so on. The existing scattered data can not meet the needs of the fine enterprise management and personalized marketing. Therefore, it is necessary to establish a data platform to excavate and exert the maximum value of the enterprise data. The research of this subject is focusing on the disadvantages of the data storage of the agricultural credit cooperatives at present, and focuses on the research of the reality of the data platform system in the agricultural credit cooperative system. Using the data platform project of Heilongjiang rural credit cooperatives as the actual project background, this paper mainly expounds the design and implementation process of the basic platform of the data platform project. Through the analysis of the historical data and the actual business operation of the rural credit cooperatives, the data storage model of the data platform of the rural credit cooperatives in Heilongjiang is obtained. The unified business data standard format is formulated, and the OLAP (On-Line Analytical Processing, online online analysis and processing) and transparent gateway are used to design and complete the data collection, data integration processing and other functional modules, as well as the number of services involved in the process of generating all the business data from the source system to the data model. According to the self data model, the function modules such as data aggregation, data presentation and distribution involved in the process of end-user are extracted, and the actual situation of each ring of the E (Extract), T (Transformation), L (Load) process in the data platform is carried out by a targeted processing method. The design and implementation of the L scheduling module can be successfully loaded into the data platform system by testing the data of the upstream business system in this platform, and after data processing, the data alignment between each layer in the data platform is correct; the system scheduling platform can be configured in advance according to the workflow. It automatically initiates data extraction, conversion, loading and other tasks, batch processing of business data, and can complete the monitoring and statistics of task processing, and complete the requirements of the data platform project.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP311.13

【参考文献】