分布式数据管理平台的设计与实现

发布时间：2018-01-18 01:11

  本文关键词：分布式数据管理平台的设计与实现　出处：《中山大学》2015年硕士论文　论文类型：学位论文

  更多相关文章： 分布式 数据管理平台 标签 数据挖掘 用户画像

【摘要】：伴随着web2.0的到来,以及移动互联网的急速发展,网络上的数据也朝多元化的方向发生了爆炸性的增长。随着云计算与大数据的兴起,企业开始收集用户相关的数据,希望这些数据能够帮助他们赢得更多的市场。但是,他们低效的数据管理方式,使得这些数据的价值并未真正的发挥出来。在市场营销过程中,作为核心的用户数据是必须管理好的。本文主要研究的是如何帮助企业管理海量的用户数据,其主要目的是提供一套完整的能够处理海量用户数据的分布式数据管理平台,该平台可以对用户进行多维度的深度的数据挖掘与分析,并最终为每个用户打上标签生成一个可以全面描述用户的用户画像,然后将这些用户画像供给客户进行检索和数据应用的开发。由于数据管理平台在国内是个非常新的商业领域,因此国内几乎没有完整的论文对数据管理平台进行详细的介绍,因此本文的设计和实现工作都是基于现实中的客户需求来进行的。本方案实现后,提供了一个高效、稳定、可伸缩的分布式数据管理平台,其出众的数据分析和管理能力受到了肯定。分布式数据管理平台自部署以来,3个月的稳定运行,帮助企业管理了近千万用户的上亿条数据,并为这些用户生成用户画像,同时提供用户画像的检索功能和OpenAPI。满足了客户的数据管理需求,并为他们的数据增值提供了强有力的支撑。
[Abstract]:With the arrival of web2.0 and the rapid development of the mobile Internet, the data on the network has also explosive growth in the direction of diversification. With the rise of cloud computing and big data. Companies are starting to collect user-related data in the hope that it will help them win more markets. However, they have inefficient data management methods. Make the value of these data has not really played out. In the marketing process, as the core of user data must be managed well. This paper mainly studies how to help enterprises to manage a large amount of user data. The main purpose of the platform is to provide a complete set of distributed data management platform which can deal with massive user data. Finally, each user is tagged to generate a user portrait that can describe the user comprehensively. Then these user portraits will be supplied to customers for retrieval and data application development, because the data management platform is a very new business field in China. Therefore, there is almost no complete paper on the data management platform for detailed introduction, so the design and implementation of this paper are based on the reality of customer requirements. Provides an efficient, stable, scalable distributed data management platform, whose outstanding data analysis and management capabilities have been recognized. The distributed data management platform has been running steadily for 3 months since its deployment. It helps the enterprise manage hundreds of millions of data of nearly ten million users, and generates the user portrait for these users, at the same time, it provides the retrieval function of the user portrait and OpenAPI. meets the customer's data management needs. And for their data added to provide a strong support.
【学位授予单位】：中山大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP311.52
，

本文编号：1438775