Webmon整站分析系统的设计与实现

发布时间：2018-03-30 21:33

本文选题：网站分析　切入点：MapReduce　出处：《北京邮电大学》2014年硕士论文

【摘要】：随着互联网高速发展和网站维护成本不断增加,本文提出Webmon整站分析系统。网站维护者通过Webmon整站分析系统,可以实现对网站的全面监控和分析,保证网站质量。针对本课题定位的Webmon整站分析系统的目标,本文对Webmon整站分析系统进行了需求分析并给出了总体设计。Webmon整站分析系统采用Web三层架构,主要功能模块有网站抓取模块、数据分析模块和报告投递模块。本文分析了各个模块的功能需求。本文对系统中的关键技术做了调研分析,主要是MapReduce模型、分布式数据库HBase以及PageRank算法。本文着重从数据类型、数据访问和数据表来完成数据库设计,着重从模块工作流程和关键技术实现来完成各个功能模块的详细设计。本文设计实现基于PageRank算法和基于网站逻辑结构的网站平均性能的计算,并且得出模拟的网站用户体验得分,全方面的考虑了网站的性能数据。本文对Webmon整站分析系统的功能点进行了功能测试,验证了系统的具体实现与预期结果一致。本文在最后进行了整体的总结和展望,并针对系统中存在的不完善的地方,提出了具体的改进方向。
[Abstract]:With the rapid development of the Internet and the increasing cost of website maintenance, this paper puts forward the Webmon whole station analysis system. Website maintainers can realize the overall monitoring and analysis of the website through the Webmon whole station analysis system, and ensure the quality of the website. Aiming at the goal of Webmon whole station analysis system, this paper analyzes the requirement of Webmon whole station analysis system and gives the overall design. Webmon whole station analysis system adopts Web three-layer structure, and the main function module is web crawling module. Data analysis module and report delivery module. This paper analyzes the functional requirements of each module. In this paper, the key technologies in the system are investigated and analyzed, including MapReduce model, distributed database HBase and PageRank algorithm. This paper focuses on data types, data access and data tables to complete the database design, In this paper, we design and implement the average performance of the website based on the PageRank algorithm and the logical structure of the website. And the simulation of the site user experience score, all aspects of the performance of the site data. In this paper, the function point of Webmon whole station analysis system is tested, and the concrete realization of the system is in accordance with the expected result. In the end, the whole summary and prospect are given, and the imperfections in the system are pointed out. The improvement direction is put forward.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092

【参考文献】