硬盘故障预测模型在大型数据中心环境下的验证

发布时间：2018-07-16 08:15

【摘要】：随着互联网的发展、存储规模的骤增,大型数据中心硬盘频繁损坏导致的数据丢失给企业带来的损失已成为不可忽视的重大问题.以往基于硬盘SMART(self-monitoring,analysis and reporting technology)属性建立的包括应用统计学和机器学习等方法在内的各种硬盘故障预测模型,虽然取得了较好的效果,但其数据采集及处理等方面均存在不足之处.基于某真实的互联网大型数据中心环境,提取SMART属性数据,并提出了一种基于神经网络权值矩阵的方法,结合Rank Sum秩和检验、RAT反向安排测试、Z-Score评分3种无参统计学方法,对属性进行选择,应用CART决策树及BP神经网络2种机器学习方法,建立硬盘故障预测模型.实验表明描述的2种硬盘故障预测模型均具有很好的性能,这是机器学习算法在实际应用场景下很好的实践.此外,通过实验以及对实验的分析和解释,得出一些有益的结论,这为下一步的研究工作奠定了基础.
[Abstract]:With the development of Internet and the sudden increase of storage scale, the loss of data caused by frequent damage to hard disk of large data center has become an important problem that can not be ignored. In the past, a variety of hard disk fault prediction models based on smart (self-monitoring analysis and reporting technology) attributes, including applied statistics and machine learning, have been established. Although good results have been achieved, their data acquisition and processing have some shortcomings. Based on a real Internet data center environment, smart attribute data is extracted, and a method based on neural network weight matrix is proposed. Three non-parametric statistical methods are proposed, including Rank Sum rank sum test and Rank Sum rank sum test. Two machine learning methods, cart decision tree and BP neural network, are used to establish the hard disk fault prediction model. The experiments show that the two kinds of hard disk fault prediction models have good performance, which is a good practice of machine learning algorithm in the practical application scenario. In addition, through the experiment and the analysis and explanation of the experiment, some useful conclusions are drawn, which lays the foundation for the next research work.
【作者单位】：南开大学计算机与控制工程学院;北京奇虎科技有限公司;南开大学软件学院;
【基金】：国家自然科学基金项目(61373018;11301288) 教育部新世纪优秀人才支持计划基金项目(NCET130301) 中央高校基础科研费基金项目(65141021)
【分类号】：TP333.35

【相似文献】