云环境中集群监控数据采集与分析系统的研究与实现

发布时间：2018-03-19 04:09

本文选题：云计算　切入点：集群监控　出处：《东南大学》2016年硕士论文　论文类型：学位论文

【摘要】：目前基于云计算技术的数据中心已经被越来越多的企业部署,云数据中心的资源分布高度集中,如何有效地对云环境下的各类资源进行监控,并依据监控数据进行合理分析以提高集群服务质量及可用性成为各企业在建立私有云的过程中所面临的一个重要问题。一些公有云服务提供商如亚马逊等都提供了一定的监控功能,但大多数开源私有云环境如OpenStack提供的监控功能并不完善,为此,本文结合企业实际项目实现了一套云环境中的集群监控数据采集与分析系统。本文实现的这套系统能够采集虚拟机与物理机的基础信息和性能监控数据,支持用户通过添加自定义采集脚本增加自己的采集项,解决了目前私有云环境下的监控系统难以满足多样性数据采集的问题。本文主要研究内容有三个方面。首先,本系统采用目标节点自动注册和心跳机制来同步目标节点状态,实现了监控节点的自动发现和状态维护,并提供了基于Web的管理控制端,可以查看每台节点的当前状态。其次,本系统提供了对监控数据的REST API操作接口,允许第三方组件如数据呈现模块获取数据并呈现到Web视图,做到了服务的独立性和可扩展性。再次,系统提供了性能分析模块,以定时任务的方式每天提交计算性能指标和集群健康度的作业到性能分析平台,利用性能分析的结果使集群获得DRS(Dynamic Resource Scheduler)功能,为运维排障提供参考依据,提高集群整体利用率。本系统各模块之间通过API来进行交互,如此便可对整个系统进行解耦,简化了系统复杂性。经企业实际应用验证,本系统的设计是合理的,功能方面基本满足了预期需求,性能方面可以支持到上万规模的目标机器,性能分析方面给出了较准确的结果,提高了集群的利用率,集群可用性达到了99.98%,是一套切实可行的集群数据监控与分析系统。
[Abstract]:At present, data centers based on cloud computing technology have been deployed by more and more enterprises, and the resources of cloud data centers are highly concentrated. How to effectively monitor all kinds of resources in the cloud environment, And reasonable analysis based on monitoring data to improve the quality and availability of cluster services has become an important problem faced by enterprises in the process of building private cloud. Some public cloud service providers such as Amazon provide. With a certain monitoring function, However, most open source private cloud environments, such as OpenStack, provide inadequate monitoring capabilities. In this paper, a cluster monitoring data acquisition and analysis system in cloud environment is implemented, which can collect basic information and performance monitoring data of virtual machine and physical machine. The user is supported to add their own collection items by adding custom collection script, which solves the problem that the monitoring system under the private cloud environment is difficult to meet the problem of diversity data collection. Firstly, the main research contents of this paper are three aspects. The system uses the automatic registration and heartbeat mechanism of the target node to synchronize the state of the target node, realizes the automatic discovery and state maintenance of the monitoring node, and provides the management control terminal based on Web, which can view the current status of each node. The system provides REST API operation interface for monitoring data, allows third-party components such as data presentation module to obtain data and render to Web view, and achieves the independence and extensibility of service. Thirdly, the system provides performance analysis module. The task of computing performance index and health degree of cluster was submitted to the performance analysis platform every day in the way of timing task. The cluster obtained DRS(Dynamic Resource Scheduler function by the result of performance analysis, which provided the reference basis for operation and maintenance obstacle arrangement. This system can decouple the whole system and simplify the complexity of the system. The design of the system is reasonable, which is verified by the practical application of the enterprise. The function aspect basically meets the expected demand, the performance aspect can support ten thousand scale target machine, the performance analysis aspect has given the more accurate result, has improved the cluster utilization ratio, Cluster availability reaches 99.98, which is a feasible cluster data monitoring and analysis system.
【学位授予单位】：东南大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP393.09;TP274

【相似文献】