基于Spark的大数据热图可视化方法
发布时间：2019-01-03 14:48
【摘要】：针对普通客户端浏览和分析大数据困难的问题,结合Spark和LOD技术,以热图为例提出一种面向大数据可视化技术框架.首先利用Spark平台分层并以瓦片为单位并行计算,然后将结果分布式存储在HDFS上,最后通过web服务器应用Ajax技术结合地理信息提供各种时空分析服务.文中重点解决了数据点位置和地图之间的映射,以及由于并行计算导致的热图瓦片之间边缘偏差这2个问题.实验结果表明,该方法将数据交互操作与数据绘制和计算任务分离,为浏览器端大数据可视化提供了一个新的思路.
[Abstract]:Aiming at the difficult problem of browsing and analyzing big data on the common client side, this paper presents a visualization technology framework for big data by combining Spark and LOD technology, taking thermal graph as an example. Firstly, the Spark platform is used for layered and tile parallel computing, then the results are distributed stored on the HDFS. Finally, the web server uses Ajax technology and geographic information to provide all kinds of spatio-temporal analysis services. In this paper, the mapping between data points and maps, and the edge deviation between tiles caused by parallel computation are mainly solved. The experimental results show that the method separates the data interactive operation from the data rendering and computing tasks, and provides a new way to visualize big data on the browser side.
【作者单位】： 浙江工业大学计算机科学与技术学院;浙江大学CAD&CG国家重点实验室;
【基金】：国家自然科学基金(61303141,61272302) 浙江省自然科学基金(LY12F02034) 国家“八六三”高技术研究发展计划(2015AA016404)
【分类号】：TP391.41
，
本文编号：2399530
[Abstract]:Aiming at the difficult problem of browsing and analyzing big data on the common client side, this paper presents a visualization technology framework for big data by combining Spark and LOD technology, taking thermal graph as an example. Firstly, the Spark platform is used for layered and tile parallel computing, then the results are distributed stored on the HDFS. Finally, the web server uses Ajax technology and geographic information to provide all kinds of spatio-temporal analysis services. In this paper, the mapping between data points and maps, and the edge deviation between tiles caused by parallel computation are mainly solved. The experimental results show that the method separates the data interactive operation from the data rendering and computing tasks, and provides a new way to visualize big data on the browser side.
【作者单位】： 浙江工业大学计算机科学与技术学院;浙江大学CAD&CG国家重点实验室;
【基金】：国家自然科学基金(61303141,61272302) 浙江省自然科学基金(LY12F02034) 国家“八六三”高技术研究发展计划(2015AA016404)
【分类号】：TP391.41
，
本文编号：2399530
本文链接：https://www.wllwen.com/kejilunwen/ruanjiangongchenglunwen/2399530.html
