面向微博数据流的观点汇总技术的研究与实现

发布时间：2018-07-29 06:09

【摘要】：作为一种日益流行的Web 2.0应用,微博客已逐渐成为人们日常生活中记录身边事件以及交流个人观点过程中不容忽视的载体和不可或缺的平台,并被越来越多的人们所接受和青睐。微博用户可以通过固定或移动的客户端登陆,借助“发布”、“转发”等功能方便快捷的表达自己的个人情感和意愿。微博“方便快捷”、“可读性强”等特点使得其在短时间内得以广泛传播,并形成了其独特的传播快、流量大、实时性强、信息繁杂等特点,这些特点使得对微博数据的处理面临巨大挑战,微博用户迫切希望在第一时间简单快捷的获取某一事件的大众观点及相应的观点倾向。针对这样的需求,本文在充分考虑微博数据“流”形态的基础上,将观点汇总技术与之结合,旨在及时有效地获取用户希望得到的汇总观点。基于上述目标,本文首先利用流数据上的聚类手段,将目标数据流根据其包含的主题予以增量式的实时聚类,形成随时间变迁的动态主题簇。实验表明,在合适的参数范围内,本文提出的方法可以获得一个较为平稳的聚类效果,各主题簇间划分较为清楚。第二,在每个主题簇中建立起基于情感词的主题-观点短语树,以此作为一种压缩手段存储数据流产生的主题及观点信息,在此过程中充分考虑数据流的特性,辅以通过动态分析频繁项集,对形成的短语树进行的剪枝维护,以求将数据规模控制在可控范围。在相关实验中,本文成功构建出主题情感树,并通过依托频繁项集统计的方式实现了对树的规模的有效控制。最后,通过抽取每个主题簇中主题—观点短语树内的最长短语作为簇的代表主题及观点,将各簇中的抽取结果汇总,形成最终的针对整个事件的观点汇总结果。经过与相关方法的对比,五分之三的评测人员认为本文的方法得出的汇总结果更为出色,从侧面反映了本文提出的处理模式的合理性。综上所述,本文提出的工作流程和方法在充分考虑微博数据流实时性特点的同时,可以较全面地涵盖所分析数据的主题。基于情感词的短语树的构建成功解决了主题及观点的存储压缩问题,并在可接受的准确率损失度下形成了合理完整的主题—观点短语,最终分析形成的汇总结果具有较强的可读性以及较为突出的代表性。
[Abstract]:As an increasingly popular Web 2.0 application, Weibo guest has gradually become a carrier and an indispensable platform for people to record events and exchange personal opinions in their daily life. And by more and more people accepted and favored. Weibo users can use fixed or mobile client login, with the help of "publish", "forward" and other functions to express their personal feelings and wishes quickly and conveniently. The characteristics of Weibo, such as "convenient and fast" and "strong readability", make it widely spread in a short period of time, and form its unique characteristics such as fast transmission, large flow, strong real-time and complicated information, etc. These characteristics make the processing of Weibo data face a huge challenge. Weibo users are eager to obtain the public opinion and the corresponding opinion tendency of a certain event in the first time. In view of this demand, this paper combines the viewpoint summarization technology with Weibo data "flow" form, in order to get the summary viewpoint that the user wants in time and effectively. Based on the above objectives, this paper firstly makes use of the clustering method on the stream data to cluster the target data stream incrementally according to the topics it contains, and forms a dynamic cluster of topics over time. The experimental results show that the proposed method can obtain a more stable clustering effect within a suitable range of parameters, and the classification of each topic cluster is relatively clear. Secondly, a topic-viewpoint phrase tree based on affective words is built in each topic cluster, which is used as a compression method to store the topic and viewpoint information generated by the data flow. In the process, the characteristics of the data flow are fully considered. With the help of dynamic analysis of frequent itemsets, the pruning and maintenance of the formed phrase trees is carried out in order to control the data size in a controllable range. In related experiments, this paper successfully constructs the subject emotion tree, and realizes the effective control of the tree scale by relying on frequent itemsets statistics. Finally, by extracting the longest phrase in the topic-viewpoint phrase tree of each topic cluster as the representative topic and viewpoint of the cluster, the extracted results from each cluster are summarized to form the final summary result for the whole event. Through the comparison with the related methods, 3/5 evaluators think that the summary result of this method is more excellent, which reflects the rationality of the processing mode proposed in this paper. To sum up, the workflow and method proposed in this paper can fully consider the real-time characteristics of Weibo data flow, and can cover the subject of the analyzed data more comprehensively at the same time. The construction of phrase tree based on affective words successfully solves the problem of storing and compressing the topic and viewpoint, and forms a reasonable and complete topic-viewpoint phrase under the acceptable loss of accuracy. The result of the final analysis has strong readability and outstanding representativeness.
【学位授予单位】：东北大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092;TP391.1

【相似文献】