面向微博话题的粒子群优化聚类算法研究

发布时间：2018-01-31 04:43

本文关键词： 聚类分析 K-means算法粒子群优化 Spark　出处：《河南理工大学》2016年硕士论文　论文类型：学位论文

【摘要】：Web技术的快速发展使微博成为人们进行沟通与交流的一种新型社交网络平台。在该平台上,用户可以自由地发表对某些话题的观点与看法。微博内容简单、发布容易的特点使其每天产生的信息总量巨大,从这些庞大的微博信息中提取出人们感兴趣的热点话题成为时下聚类分析的研究重点。本文的主要研究内容如下:(1)首先,在对聚类算法、粒子群优化算法(Particle Swarm Optimization,PSO)和微博话题的国内外研究现状进行阐述的基础上,深入研究和分析了聚类算法和PSO算法的基本原理及改进思想,同时根据项目和数据分析的需求成功搭建了用于处理大数据的Spark集群。(2)其次,提出了基于时间因子的混沌粒子群优化K-means算法(the K-means algorithm of Chaotic Particle swarm optimization with Time Factor),即KCPTF算法。在该算法中,为保证粒子的全局寻优能力,引入了反映时间效应的非线性递减时间因子,使粒子在算法初期可快定位在近最优解附近;为防止粒子群陷入局部最优解,引入了混沌寻优技术,利用混沌技术的遍历性增强粒子的全局搜索能力,并确保了粒子群的多样性,同时启用边界缓冲墙技术来动态调整越界粒子;将改进后的PSO算法与K-means算法进行合并。在Matlab上对UCI数据库中的数据集进行仿真测试,对比实验表明,KCPTF算法的聚类结果有更高的精确度。(3)最后,将本文提出的KCPTF算法应用在对新浪微博话题的聚类上,并在Spark平台上开发出了一个基于新浪微博的热点话题原型系统,该系统可聚类出一定时段内的热点话题,达到了项目在聚类方面的预期要求。
[Abstract]:With the rapid development of Web technology, Weibo has become a new social network platform for people to communicate and communicate. On this platform, users can freely express their views and opinions on certain topics. Weibo's content is simple. The ease of publishing makes the amount of information generated every day huge. Extracting hot topics of interest from these huge Weibo information has become the focus of cluster analysis. The main research contents of this paper are as follows: 1) first of all, the clustering algorithm is proposed. Particle Swarm Optimization algorithm (PSO) and Weibo topic research status at home and abroad are discussed. The basic principle and improved idea of clustering algorithm and PSO algorithm are deeply studied and analyzed. At the same time, according to the requirements of the project and data analysis, the Spark cluster used to deal with big data is successfully built. A chaotic particle swarm optimization K-means algorithm based on time factor is proposed. The K-means algorithm of Chaotic Particle swarm optimization with. Time Factor). . In order to ensure the global optimization ability of particles, a nonlinear decrement time factor is introduced, which reflects the time effect, so that the particles can be located near the near optimal solution in the initial stage of the algorithm. In order to prevent particle swarm from falling into local optimal solution, chaos optimization technique is introduced. The global searching ability of particle is enhanced by using the ergodicity of chaos technology, and the diversity of particle swarm is ensured. At the same time, the boundary buffer wall technology is used to dynamically adjust the cross boundary particles; The improved PSO algorithm is combined with K-means algorithm. The data set in UCI database is simulated on Matlab. The clustering result of KCPTF algorithm has higher accuracy. Finally, the KCPTF algorithm proposed in this paper is applied to the clustering of Sina Weibo topic. A hot topic prototype system based on Sina Weibo is developed on Spark platform. The system can cluster the hot topics in a certain period of time and meet the expected requirements of the project in clustering.
【学位授予单位】：河南理工大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP393.092;TP311.13

【相似文献】