基于集群架构的移动终端网络流量采集与服务平台研发

发布时间：2018-04-08 18:04

本文选题：网络流量采集　切入点：Android程序遍历　出处：《济南大学》2017年硕士论文

【摘要】：随着移动终端的广泛使用,尤其是智能手机的迅速普及,移动智能终端给现代社会巨大的变革。然而随着移动应用的普及和用户数量爆发式增长,移动智能终端的安全也面临着巨大挑战。在移动终端恶意软件检测领域,近年来学术界和产业界关注的除了静态特征码和动态行为分析方法之外,基于网络流量特征的检测方法日益被业界关注和研究。但这种检测技术由于用到机器学习技术甚至是深度学习技术,因此获取海量的、有标记的网络流量数据成为了研究的首要任务。针对以上问题,为达到快速收集指定应用程序网络流量数据的目的。本文设计并实现了基于集群架构的移动终端网络流量采集与服务平台。该平台分为三部分,第一部分是存储服务器,用于存储应用程序文件和网络流量文件;第二部分是控制及服务系统,用于整个平台的控制;第三部分是采集集群,由采集流量计算机(采集机)组成,用于采集移动终端网络流量。每一台采集计算机中部署多线程网络流量采集程序,采集计算机中开启多个线程,每一个线程通过程序实现全自动的应用程序网络流量收集任务。并且针对收集的网络流量文件,本平台可以对其进行进一步的处理,例如提取网络流量中的DNS数据包、TCP流,提取目的地址为恶意地址的TCP流,网络流量可视化等。应用技术方面,平台使用Python语言搭建了基于集群架构的Android系统网络流量采集系统,并且编写了专门的Android应用程序界面遍历脚本和采用PHP语言编写的WEB端的管理系统。前者使得自动化采集到的网络流量更加接近真实环境下产生的网络流量,后者则方便研究人员操作该平台。本文利用平台共采集68000余个安卓应用程序,采集到网络流量字节数大约21GB,经过进一步处理得到大约106万个TCP网络流和88万个DNS请求,从DNS数据包中提取得到大约1万个请求的域名,再经过检测发现744个属于恶意域名,进一步提取出纯恶意网络流量大约1GB。目前该数据集不仅在本校实验室中使用,还共享给了内布拉斯加大学林肯分校、湖南大学等研究团队。
[Abstract]:With the wide use of mobile terminals, especially the rapid popularization of smart phones, mobile intelligent terminals have brought great changes to modern society.However, with the popularity of mobile applications and the explosive growth of the number of users, the security of mobile intelligent terminals is also facing great challenges.In the field of mobile terminal malware detection, in recent years, in addition to static signature and dynamic behavior analysis methods, the detection methods based on network traffic characteristics have been paid more and more attention and research in academia and industry.However, due to the use of machine learning technology and even deep learning technology, obtaining massive and marked network traffic data becomes the primary task of the research.In view of the above problems, in order to achieve the purpose of fast collection of network traffic data for specified applications.This paper designs and implements a mobile terminal network traffic collection and service platform based on cluster architecture.The platform is divided into three parts, the first part is the storage server, which is used to store application files and network traffic files; the second part is the control and service system for the control of the whole platform; the third part is the collection cluster.By the collection flow computer (acquisition machine), used to collect mobile terminal network traffic.A multithread network traffic collection program is deployed in each acquisition computer, and multiple threads are opened in the acquisition computer. Each thread realizes the automatic network traffic collection task of the application program through the program.For the collected network traffic files, the platform can further process them, such as extracting DNS data packets from network traffic, extracting TCP flows with malicious address, network traffic visualization and so on.In terms of application technology, the platform uses Python language to build a network traffic acquisition system of Android system based on cluster architecture, and compiles a special Android application interface traversal script and a WEB management system written by PHP language.The former makes the automatically collected network traffic closer to the network traffic generated in real environment, while the latter is convenient for researchers to operate the platform.This paper uses the platform to collect more than 68000 Android applications, collects about 21GB of network traffic bytes, and gets about 10.6m TCP network streams and 880,000 DNS requests after further processing.Ten thousand requested domain names were extracted from DNS packets, and 744 domain names were found to be malicious domain names after detection, and the pure malicious network traffic was further extracted about 1 GB.The data set is not only used in our laboratory, but also shared with research teams at the University of Nebraska, Lincoln and Hunan University.
【学位授予单位】：济南大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.06;TP311.52

【参考文献】