娱乐消费大数据分析系统的研究与实现

发布时间：2019-06-20 03:25

【摘要】：当今社会,人们的生活水平逐渐升高,在消费者消费的同时,会留下相应的足迹,也会产生相应的记录。由于人们每天消费量巨大,会产生大量消费记录。若想从海量数据中快速分析出用户最关心的消费品类和相关行为,用传统的分析方式已是很难满足需求。传统方法耗费时间长,难以实现预求效果。这些消费记录中包含了顾客消费的许多消费信息,如果通过特定的分析工具去分析这些数据,则会从数据中分析出许多有效信息,包括消费者的消费习惯、商品的销售情况等。现今流行的计算框架有Hadoop计算框架和Spark计算框架。其主要针对海量数据进行分析和处理,本文则是使用Hadoop中的HDFS和Spark计算框架对广大消费者的娱乐消费记录进行分析,得出其娱乐消费的实际情况。本文首先对大数据处理分析系统研究现状做了综述,介绍了相关的理论和技术;其次对系统需求进行了具体分析,从中得出系统的总体设计架构,系统主要功能包括聚合统计、随机抽取、Topl0热门品类和Topl0活跃session;然后,对获取数据算法的设计思路和系统主要功能进行了详细设计,并对数据库进行了详细设计;最后,搭建了系统环境并对获取数据算法和系统的主要功能进行了具体实现,实现完成后对其进行了功能性测试和非功能性测试。运行结果表明,系统完全可以正常运行,可以达到预期的目标。
[Abstract]:In today's society, people's living standards gradually rise, at the same time, consumers will leave the corresponding footprints, but also produce the corresponding records. Because people consume a lot of money every day, they will produce a lot of consumption records. In order to quickly analyze the consumer categories and related behaviors that users are most concerned about from the massive data, it is difficult to meet the demand with the traditional analysis method. The traditional method takes a long time and is difficult to achieve the desired effect. These consumption records contain a lot of consumer information about customer consumption. If we analyze these data through specific analysis tools, we will analyze a lot of valid information from the data, including consumer consumption habits, commodity sales and so on. Nowadays, the popular computing frameworks are Hadoop computing framework and Spark computing framework. This paper mainly analyzes and processes the massive data, this paper uses the HDFS and Spark computing framework in Hadoop to analyze the entertainment consumption records of the majority of consumers, and obtains the actual situation of their entertainment consumption. In this paper, the research status of big data processing and analysis system is summarized, and the related theories and technologies are introduced. secondly, the system requirements are analyzed concretely, and the overall design architecture of the system is obtained. the main functions of the system include aggregation statistics, random extraction, Topl0 hot categories and Topl0 active session;. Then, the design idea of the data acquisition algorithm and the main functions of the system are designed in detail, and the database is designed in detail. Finally, the system environment is built and the main functions of the data acquisition algorithm and the system are implemented concretely. After the completion of the implementation, the functional test and non-functional testing are carried out. The running results show that the system can run normally and achieve the expected goal.
【学位授予单位】：内蒙古大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.52

【相似文献】