文本评论数据质量分析方法研究

发布时间：2018-10-22 09:29

【摘要】：随着电子商务的快速兴起,人们越来越倾向于网络购物。一方面由于网购交易双方信息的不对称性,使网购用户鉴定商品质量的困难加大,降低电子商务市场的交易效率;另一方面,用户评论数量巨大,且逐年增长,再加上用户的消费经验、评价态度等存在着巨大差异,导致评论数据中的信息和价值也参差不齐。因此,如何快速的从海量评论中获取有价值的信息是当前电子商务市场一项迫切而重要的任务。本文从文本评论的质量角度出发,基于评论中商品特征词之间的泛化/特化关系,建立商品特征主题层次结构,在该结构的基础上对文本质量进行分析,从而向用户推荐高质量评论数据。本文将每个商品下的所有评论作为一个文档,提取文本评论中用户常用的比较关注的商品特征,根据商品特征词之间的泛化/特化关系,本文提出了一种基于FCA的特征主题层次格构建方法(THL Based on FCA,TBF),以此来构建商品特征主题层次格(Topic Hierarchy Lattice,THL)。TBF方法在采用句法分析方法提取出的商品特征的基础上,利用文档主题生成模型(Latent Dirichlet Allocation,LDA)来进行主题分析,提取出特征主题(该主题是由一些商品特征词按照一定的概率分布组成的词组),并得到文档——主题(doc-topic)和主题——特征词(topic-word)概率矩阵,然后根据商品类别与特征主题之间的二元关系,采用形式概念分析(Formal Concept Analysis,FCA)的方法来构建THL。在THL的基础上,本文提出了文本评论质量的五个影响因素:全面性、专业性、内聚性、相关性和可读性,由此提出了一个商品评论的质量分析模型(Comment Quality Model Based on THL,CQM)来计算每条评论的质量分数。该模型结合了评论质量的五个影响因素,可以比较全面的评估评论的质量。本文选取某B2C电子商务平台2012年交易数据中的116个商品类别、6,212个商品和18,415,146条评论数据为实验数据集,并通过人工打分的形式收集了评论质量评分数据作为实验的测试数据集,实验结果中,CQM的MAE值为0.726,表明CQM对评论质量的预测评分与人工评分比较接近,验证了CQM对评论评分预测的准确性;另外,将CQM与四种常用的分类预测模型进行比较,其中Random Forest的分类效果最好,召回率最高为56.6%,验证了评论质量影响因素的合理性,同时,CQM也有比较好的分类效果,说明本文提出的CQM是有效的。
[Abstract]:With the rapid rise of e-commerce, people are more and more inclined to online shopping. On the one hand, because of the asymmetry of the information between the two sides of the online purchase transaction, it makes it more difficult for the online shopping users to identify the quality of the goods, thus reducing the transaction efficiency in the e-commerce market; on the other hand, the number of users' comments is huge, and the number of comments is increasing year by year. In addition, there are great differences in consumer experience, evaluation attitude and so on, which leads to the uneven information and value in the comment data. Therefore, how to quickly obtain valuable information from mass comments is an urgent and important task in the current e-commerce market. From the point of view of the quality of text comment, based on the generalization / specialization relationship between commodity feature words in the comment, this paper establishes the hierarchical structure of commodity feature topic, and analyzes the text quality on the basis of this structure. To recommend high-quality review data to users. In this paper, all the comments under each item are regarded as a document, which is used by the user in the text comment to extract the commodity features of concern, according to the generalization / specialization relationship between the commodity feature words. In this paper, we propose a method of constructing feature subject hierarchy based on FCA (THL Based on FCA,TBF), which is used to construct commodity feature subject hierarchy (Topic Hierarchy Lattice,THL). TBF method based on syntactic analysis. Using the document topic generation model (Latent Dirichlet Allocation,LDA) for topic analysis, The feature topic (the topic is a phrase composed of some commodity feature words according to a certain probability distribution) is extracted, and the probability matrix of document-topic (doc-topic) and theme-feature word (topic-word) is obtained. Then, according to the binary relationship between commodity category and feature theme, the formal concept is used to analyze (Formal Concept Analysis,FCA) to construct THL.. On the basis of THL, this paper proposes five factors that influence the quality of text review: comprehensiveness, professionalism, cohesion, relevance and readability. A quality analysis model (Comment Quality Model Based on THL,CQM) is proposed to calculate the quality score of each comment. The model combines five factors that influence the quality of comments and can be used to evaluate the quality of comments comprehensively. This paper selects 116 commodity categories, 6212 items and 18415146 comments data from a B2C e-commerce platform's 2012 transaction data as experimental data set, and collects the comment quality score data as the experimental test data set through the form of manual scoring. In the experimental results, the MAE value of CQM is 0.726, which indicates that the prediction score of CQM for comment quality is close to that of manual score, which verifies the accuracy of CQM in predicting comment score. In addition, CQM is compared with four commonly used classification prediction models. The classification effect of Random Forest is the best, and the highest recall rate is 56.6, which verifies the reasonableness of the influencing factors of comment quality. At the same time, CQM also has a better classification effect, which shows that the CQM proposed in this paper is effective.
【学位授予单位】：重庆大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP391.1

【相似文献】