Hadoop构建的银行海量数据存储系统研究

发布时间：2018-06-27 00:28

本文选题：大数据 + 数据迁移　；参考：《哈尔滨理工大学学报》2015年04期

【摘要】：针对目前国内银行的OLAP(Online Analytical Processing)技术滞后于海量数据处理需求的情况,研究了海量数据存储、处理的关键技术,设计并实现了用Hadoop平台和关系型数据库架构的银行海量数据处理系统.介绍了系统架构、文件存储层次设计、数据迁移方案、任务调度算法等关键技术的实现.通过实验验证,系统能够实现数据的高可靠性与数据的高可用性目标,既能满足业务数据实时处理的需求,也能满足海量数据统计分析的需求.
[Abstract]:In view of the fact that the OLAP (online Analytical processing) technology of domestic banks lags behind the demand for massive data processing, the key technologies of mass data storage and processing are studied. A bank mass data processing system based on Hadoop platform and relational database architecture is designed and implemented. The system architecture, file storage hierarchy design, data migration scheme, task scheduling algorithm and other key technologies are introduced. The experimental results show that the system can achieve the goal of high reliability and high availability of data, which can not only meet the requirements of real-time processing of business data, but also meet the needs of statistical analysis of massive data.
【作者单位】：哈尔滨金融学院网络信息中心;
【基金】：黑龙江省科技攻关项目(GC12A307)
【分类号】：TP311.13;TP333

【参考文献】