大数据环境下数据分析与可视化核心技术研究

发布时间：2018-05-08 18:34

本文选题：数据分析 + 数据可视化　；参考：《北京邮电大学》2016年硕士论文

【摘要】：步入信息社会以来,数据的爆炸式增长为信息处理与数据分析带来了新的机遇与挑战。IBM Watson指出,每天都产生约25亿GB的非结构化的原始数据,数据中的关键信息往往淹没在庞大的数据量以及复杂的结构之中。因此,需要对数据信息进行有效的过滤处理,其中,数据可视化分析技术是至关重要的部分。本文分析了当前数据可视化分析领域的主要方法和工具,发现当前的可视分析方案仍存在着一些值得探究的问题:人对于数据的认知特点决定了人在分析过程中具有一定的认知局限性,而单一的采用机器计算的方式进行数据分析存在目的性差的问题;在可视化分析过程中存在可视化模型选择困难的现实问题;一些可视化工具片面追求华丽的效果,而轻视了对数据关键信息进行传递的能力等。据此本课题完成了以下的研究与实践工作:1.本文从数据可视化分析的流程入手,结合传统探索式数据发现方法,提出了启发式数据可视化分析策略。该策略重视人对数据的认知特性,强调人机之间的互动与反馈,注重数据本身对可视化分析过程的指导性。2.文章重点论述了构建一个可视化数据分析平台的设计与实现过程。这个平台以B/S结构为基础,结合当前流行的D3.js可视化框架,实现了一个集数据属性分析、可视化模型推荐、动态可视化模型构建为一体的可视化分析应用。3.该平台实践了理论提出的启发式数据分析策略,并提出自动化可视分析图形推荐方法以及可视化模型编码推荐方法等两种可视分析新方法。系统功能简化了分析人员使用数据可视化分析方法的过程,提升了分析效率,并且能够得到更佳的可视化分析结果。本系统架构重视降低模块之间的耦合程度,提供了较强的可扩展性。以该平台的架构为基础,可以进一步实践启发式的数据可视化分析方法,并将系统扩展出更多的具体应用。此外,本系统还可以扩展可视化模型的种类、模型推荐算法等方面继续优化。
[Abstract]:Since entering the information society, the explosive growth of data has brought new opportunities and challenges for information processing and data analysis. IBM Watson points out that about 2.5 billion GB of unstructured raw data is produced every day. The key information in the data is often submerged in the huge amount of data and complex structure. Therefore, it is necessary to filter the data information effectively, among which, the data visualization analysis technology is the most important part. In this paper, the main methods and tools in the field of data visualization analysis are analyzed. It is found that there are still some problems worth exploring in the current visual analysis scheme: the cognitive characteristics of data determine the cognitive limitations in the process of analysis. However, there is a problem of poor purpose in a single method of data analysis using machine computing, a practical problem of difficult choice of visual model in the process of visual analysis, a partial pursuit of ornate effects by some visualization tools, It belittles the ability to transmit the key information of the data. Accordingly, this subject has completed the following research and practice work: 1. This paper starts with the flow of data visualization analysis and proposes a heuristic strategy of data visualization analysis combined with the traditional exploratory data discovery method. The strategy attaches importance to the cognitive characteristics of data, emphasizes the interaction and feedback between man and machine, and emphasizes the guidance of data itself to the visual analysis process. This paper focuses on the design and implementation of a visual data analysis platform. Based on the structure of B / S and the current popular D3.js visualization framework, this platform implements a visual analysis application .3which integrates data attribute analysis, visual model recommendation and dynamic visualization model construction. The platform implements the heuristic data analysis strategy proposed by the theory, and proposes two new visual analysis methods, such as automated visual analysis graphics recommendation method and visual model coding recommendation method. The function of the system simplifies the process of using the data visualization analysis method, improves the efficiency of the analysis, and can obtain better visual analysis results. This system architecture pays attention to reduce the coupling degree between modules, and provides a strong scalability. Based on the architecture of the platform, the heuristic method of data visualization analysis can be further practiced, and the system can be extended to more concrete applications. In addition, the system can also extend the types of visual models, model recommendation algorithms and other aspects continue to optimize.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.13

【参考文献】