基于汉语句法分析的微博聚类方法研究

发布时间：2018-05-28 07:56

本文选题：微博 + 聚类　；参考：《西安电子科技大学》2014年硕士论文

【摘要】：微博是互联网上的一种重要的信息发布工具，以其简短、便捷的方式表达用户观点，已经成为了新的媒体和社交平台。为了更好地提升微博服务质量，如何为不同用户推荐感兴趣的好友和内容就成为了一个值得研究的课题。针对此问题，现有方法主要是从结构和内容两方面入手进行研究。由于微博是一个由文本内容构成的平台，所以从文本入手更加具有普适性以及跨越性，不会受到已有用户关系的影响，可以根据不同用户的语义偏好性来进行聚类。因此本文根据文本内容来研究微博相似度，为后续的聚类处理提供可靠依据。微博是一种短文本形式，如果使用传统的文本聚类方法，就会造成表示文本的向量空间模型（Vector Space Model，VSM）过于稀疏的问题。为了避免微博特征向量过于稀疏，区别于广泛使用的添加外来语义的处理方法，本文从汉语句法结构入手进行分析，明确了构造句法分析器的需求，并依据该需求设计了句法分析器的结构及其处理流程。处理时，，首先对文本进行句法分析，依据不同词性的前后顺序及共现关系，识别出不同类型的短语；然后根据识别出的短语类型，为组成该短语的不同成分赋予相应权值，使得句中关键分词的权值更大；迭代以上步骤，反复归约出新的短语成分，就可以为文本中的核心部分赋予最大的权值，贡献值较小的部分，则权值相应较小。利用这种方法确定的特征向量，可以更加客观地标识出不同成分的重要性。在此基础上，利用向量空间模型计算出不同微博文本之间的相似度。本文在计算相似度时，为了避免出现聚类结果以特征向量非零项数为基准而发生的拥簇现象，特意进行了两次程度不同的聚类，分别使用了不同的相似度函数，这种方法很好地改善了上述问题。本文算法最大限度地挖掘出了微博中有限内容的隐含价值，实验表明，使用该算法得出的特征向量，提高了相似度计算的准确性，并提升了微博聚类的效果。
[Abstract]:Weibo is an important information publishing tool on the Internet. It has become a new media and social platform because of its short and convenient way to express users' views. In order to improve the quality of Weibo service, how to recommend interested friends and content to different users has become a research topic. In order to solve this problem, the existing methods are mainly studied from two aspects: structure and content. Because Weibo is a platform composed of text content, it is more universal and leapfrogging from the text, and can be clustered according to the semantic preference of different users. Therefore, this paper studies Weibo similarity according to text content, and provides reliable basis for subsequent clustering processing. Weibo is a short text form. If the traditional text clustering method is used, the vector space model (VSM) representing text will be too sparse. In order to avoid the Weibo feature vector being too sparse, which is different from the widely used processing method of adding exonyms, this paper starts with the analysis of Chinese syntactic structure, and clarifies the requirements of constructing parser. According to this requirement, the structure of parser and its processing flow are designed. In processing, first of all, the text is analyzed syntactic, and different types of phrases are identified according to the order and co-occurrence of different parts of speech, and then the corresponding weights are assigned to the different components of the phrase according to the identified phrase types. If we iterate the above steps and reduce the new phrase components repeatedly, we can assign the largest weight to the core part of the text, and the smaller contribution value, the smaller the weight value. Using the eigenvector determined by this method, the importance of different components can be identified more objectively. On this basis, the vector space model is used to calculate the similarity between different Weibo texts. In this paper, in order to avoid clustering results based on the non-zero term number of feature vector, clustering with different degrees is carried out in order to avoid clustering, and different similarity functions are used respectively. This method improves the above problem well. In this paper, the hidden value of finite content in Weibo is excavated to the maximum extent. The experiments show that the feature vectors obtained by this algorithm can improve the accuracy of similarity calculation and improve the effect of Weibo clustering.
【学位授予单位】：西安电子科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092;TP391.1

【参考文献】