面向公共安全的数据分析系统设计与实现

发布时间：2018-06-10 17:25

本文选题：数据采集 + 图数据　；参考：《中国科学院大学(中国科学院工程管理与信息技术学院)》2017年硕士论文

【摘要】：为应对突发性公共安全事件带来的巨大挑战,我国的公共安全处置策略需要从“事后处置”转变到“事前预防”,为此,需要强化数据资源的综合利用以及自动化地、深度地数据分析和信息挖掘手段,以提高信息发现的及时性、前瞻性,防范突发性安全事件于未然。通过建设面向公共安全的数据分析系统,为信息挖掘应用提供监测和预警手段,解决安全保障部门面临的问题。本文在研究了面向特定网站的主动式快速数据采集技术,关系网络分析相关的图数据压缩存储、分析和可视化展示等技术的基础上,设计并实现了一套主动获取数据、自动分析数据的分析系统,实现对特定人员、特定群体、异常行为、安全事件的检测与挖掘。本文的主要工作如下:实现面向特定网站的数据采集。分析了针对公共安全领域内数据查询类网站进行数据采集的技术,研究基于浏览器模拟、浏览器操纵的采集方式,实现用户名密码、PKI证书登录的认证授权,获取数据并进行解析,使用增量式加速、长连接加速、并行加速、分布式加速等加速策略加快采集速度,并使用最近最少使用法、直接映射法对数据去重,实现对公共安全领域内数据查询类网站数据的准确高效的采集,为信息挖掘应用提供基础数据来源。研究数据分析技术,提出一种隐关系发现算法并证明其正确性。研究基于Bloom过滤器和基于Cuckoo过滤器的数据比对加速技术,实现特定人员快速侦测;通过统计方法挖掘隐含在人员活动数据中的人员关系,利用多源异构数据构建人员关系和活动图,实现大规模图数据的压缩,在关系网络上进行治安效价传递、异常行为等分析,并进行可视化展示,使公共安区保障部门能直观地观察分析和处理结果。设计并实现了数据分析系统。通过整合面向特定网站的数据采集技术和数据分析与图可视化技术,设计开发一套主动获取数据、自动分析数据的分析系统,对活动记录进行分析处理,实现对特定人员、特定群体、异常行为、安全事件的检测与挖掘。
[Abstract]:In order to cope with the huge challenge of emergency public safety events, the strategy of public security disposal in China needs to be changed from "post treatment" to "pre prevention". Therefore, it is necessary to strengthen the comprehensive utilization of data resources and the methods of automatic and in-depth data analysis and information mining to improve the timeliness and foresight of information discovery. By building a data analysis system facing public security, it provides monitoring and early warning for information mining applications, and solves the problems faced by the security and security departments. In this paper, the active fast data acquisition technology for specific websites is studied, and the related graph data compression storage is related to the network analysis. On the basis of analysis and visualization techniques, a set of active data acquisition and analysis system is designed and implemented to realize the detection and mining of specific people, specific groups, abnormal behavior and security events. The main work of this paper is as follows: to realize data acquisition for specific websites and analyze public security. The technology of data collection in the field of data query in the field, research based on the browser simulation, the acquisition mode manipulated by the browser, realize the username cipher, the authentication authorization of the PKI certificate, obtain the data and analyze it, use incremental acceleration, accelerate the long connection, accelerate the acceleration, and accelerate the distributed acceleration strategy to speed the acquisition speed. And using the latest least use method, the direct mapping method is used to retrieve data accurately and efficiently in the field of public security. It provides the basic data source for the application of information mining. It studies the data analysis technology and presents a hidden relation discovery algorithm and proves its correctness. The research is based on the Bloom filter and the research. Based on the data comparison of the Cuckoo filter, the rapid detection of specific personnel is realized, the personnel relationship hidden in the personnel activity data is excavated by statistical method, the personnel relationship and activity graph are constructed by using multi source heterogeneous data to realize the compression of the large scale map data, and the security titer transfer and abnormal behavior are carried out on the closed network. Through the analysis and visualization, the Security Department of the public security area can observe and analyze the results intuitively. The data analysis system is designed and implemented. By integrating data acquisition technology and data analysis and graph visualization technology oriented to specific websites, a set of active data acquisition and analysis system is designed and developed. Activity records are analyzed and processed to detect and mine specific people, specific groups, abnormal behaviors and security incidents.
【学位授予单位】：中国科学院大学(中国科学院工程管理与信息技术学院)
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP274.2

【相似文献】