通信行为分析的大数据处理策略的研究

发布时间：2018-04-01 10:24

本文选题：通信行为分析　切入点：大数据处理策略　出处：《吉林大学》2017年硕士论文

【摘要】：随着计算机技术的蓬勃发展和创新,互联网技术早已深入到了我们生活的方方面面,“互联网+”的概念悄然兴起,已成为各行业发展的“新常态”。2017年1月,中国互联网络信息中心(CNNIC)公布了第39次全国互联网发展统计报告,报告显示,截至2016年12月,我国网民规模已达7.31亿,手机网民规模达6.95亿,较2015年底增长7550万人。移动互联网使用人数持续增多,网络数据的爆发式增长仍未停歇,如何处理对这些大数据进而挖掘其潜在规律和应用价值,越来越受到各行各业的重视。在如今这个时代,数据是信息世界里最有价值的资产,而大数据是信息技术产业的又一次重大技术变革,具有里程碑式的意义,可以预见的是,大数据将越来越多的在各领域产生巨大影响,为该领域发展开拓一条新的道路。从CPU内部指令集间的指令传输到人类的火星探测工程,通信成了信息时代最基本的元素,数据的产生离不开各种各样的通信行为,物联网和移动互联网的飞速发展,使得通信所产生的数据更加多样化,如何准确、快速从天量网络数据中挖掘出用户的通信行为信息,找出并总结其通信行为特点,从而为网络管理、网络监控、设备的策略管理提供帮助成了值得关注的问题,需要注意的是,这里所研究的用户“通信行为”,不仅仅是“人”的,也可以是智能设备、网络终端、虚拟环境等一切可以产生通信行为的“客体”的行为。在这种背景下,本文针对目前大数据的发展趋势,学习研究目前流行的大数据处理架构,确定使用Hadoop分布式架构作为系统的基础平台,并比较各种通信行为分析模式,以了解掌握整个网络环境下各类用户通信行为为目标,深入研究TCP/IP参考模型的各层各类网络协议,针对用户的通联关系、通信规律、常规网络行为等方面,结合Hadoop分布式处理存储技术和网络协议重组还原技术,研究设计并实现了一套通信行为分析的大数据处理系统,实现在大数据中重组还原所需协议并找出相关通联信息等功能,并对网络通联情况、协议使用情况、网站浏览情况、邮件收发情况等网络通信元数据进行统计,实现对整体网络通信情况的掌握,也可针对需要的特定用户、特定通信行为进行分析,了解其通信行为,掌握其通信规律。本系统旨在为网络管理者提供更为详实准确的网络通信行为信息,帮助网络管理者在大数据背景下更好的监督网络通信行为,为科学高效的网络管理模式提供坚实有力的保障。
[Abstract]:With the rapid development and innovation of computer technology, Internet technology has already penetrated into all aspects of our lives. The concept of "Internet" has emerged quietly and has become the "new normal" of the development of various industries. The China Internet Network Information Center (CNNIC) has released the 39th National Statistics report on Internet Development. The report shows that as of December 2016, the number of Internet users in China has reached 731 million and the number of mobile Internet users has reached 695 million. The number of people using the mobile Internet continues to increase, and the explosive growth of network data has not stopped. How to deal with these big data and then tap its potential laws and application value, In this era, data is the most valuable asset in the information world, and big data is another major technological change in the information technology industry. Big data will have more and more tremendous influence in various fields, opening up a new way for the development of this field. From the instruction set within the CPU to the Mars exploration project of mankind, communication has become the most basic element of the information age. The generation of data can not be separated from a variety of communication behaviors. With the rapid development of the Internet of things and the mobile Internet, the data generated by communication become more diversified. To find out and summarize the characteristics of communication behavior, so as to provide help for the policy management of network management, network monitoring and equipment management, has become a matter of concern. What should be noted is that the "communication behavior" of users studied here is not only "human". It can also be the behavior of intelligent devices, network terminals, virtual environments and all other "objects" that can produce communication behavior. In this context, this paper studies and studies the current popular big data processing framework in view of big data's current development trend. It is decided to use the Hadoop distributed architecture as the basic platform of the system, and to compare various communication behavior analysis models, in order to understand all kinds of user communication behavior under the entire network environment as the goal, and deeply study all kinds of network protocols of the TCP/IP reference model. In this paper, a big data processing system for communication behavior analysis is designed and implemented based on Hadoop distributed processing storage technology and network protocol recombination and restore technology, aiming at the connection relationship, communication rules and general network behavior of users. To achieve the functions of reorganizing and restoring the required protocols in big data and finding relevant communication information, and statistics on network communication metadata, such as network communication, protocol usage, website browsing, mail sending and receiving, etc. To realize the mastering of the overall network communication situation, but also to analyze the specific communication behavior of the specific users who need it, to understand their communication behavior, The purpose of this system is to provide network managers with more detailed and accurate information on network communication behavior, and to help network managers better supervise network communication behavior under the background of big data. For the scientific and efficient network management model to provide a solid and effective protection.
【学位授予单位】：吉林大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13

【相似文献】