大规模用户在线行为数据分析

发布时间：2018-05-30 06:13

本文选题：用户行为分析 + 用户属性　；参考：《上海交通大学》2013年硕士论文

【摘要】：最近十年来全世界网民的数量呈现快速增长的趋势，，截止2011年1月全世界网民的数量已经超过了20亿。在线用户规模的急剧扩大使得用户在线行为分析在很多研究领域的意义显得越来越重大，如在经济、学术以及社会事务等领域。其中，最具有商业价值要数用户在线商业意图（Online CommercialIntention）的挖掘。通过分析用户的在线行为来挖掘出用户个人兴趣及倾向，为有针对性的广告投放提供有力的依据，从而达到更好的广告推荐效果。文中提出了一种检测用户商业意图的新方法架构，通过分析用户浏览过的网页信息，从中挖掘出用户个性化的购买倾向。借助于大型购物平台上搜索引擎的历史记录以及搜索结果页面上商品的点击分布情况，通过统计分析建立起搜索词与商品类目之间的关联——关键词词典。利用这一词典我们可以从用户浏览过的网页内容中抽取出一系列能够体现用户商业意图的关键词，关键词选取的标准包括关键词的TF、ICF、QF以及关键词本身的长度。每一个关键词根据关键词词典将被映射到一系列的商品类目，其中分值最高的几个商品类目将视为页面的商业意图检测结果。为了消除一些不相关类目对于用户商业意图结果的影响并使得OCI结果更加集中，类目的相似度模型被引入来修正OCI结果中的分值。用户每天的商业意图通过聚合其当天浏览过页面的OCI结果得到。为了体现出商业意图短暂、灵活多变的特点，我们提出了用户时间维度上的商业意图模型来更真实地体现出用户的实时商业意图变化。为了评估所提出OCI检测方法的效果，在实验部分，通过制定统一的评测标准，评测人员对于随机抽取的1036个测试页面的商业意图结果进行了人工标注。从标注结果与系统检测结果的对比中可知，在检测商业意图存在的角度，系统判断的准确率达到了70%，召回率达到了将近90%；而从系统匹商业类目的角度来看，系统匹配商业类目的正确率达到了86%。通过后续的实验我们还发现，通过限制页面关键词的数量以及设定匹配类目最小阙值等方法可以有效地提升系统检测商业意图的整体效果。作为最终的分析结果，系统以一系列商品类目的形式给出了用户在一段时间内的商业意图。根据商品类目在一段时间内的分值变化情况，可以推测出用户在这一段时间内购买某一类商品的倾向及对应程度。通过构建合适的用户购买行为模型，可以对目标用户进行有针对性的商品推荐。借助于大规模分布式计算架构，文中所提到的OCI检测系统流程可以应用于大规模用户行为数据的分析。正在运营的系统每天可以处理十亿级别的用户浏览记录，用来分析得到几千万用户的具体商业意图，每个小时的数据处理量大约在150GB。
[Abstract]:The number of Internet users worldwide has been growing rapidly in the last ten years. By the end of January 2011, the number of Internet users in the world has exceeded 2 billion. The rapid expansion of online user scale makes the analysis of user online behavior more and more significant in many research fields, such as in the fields of economy, academic and social affairs. The most commercial value of the user online business intention (Online CommercialIntention) mining. Through the analysis of the user's online behavior to excavate the user's personal interests and tendencies, to provide a powerful basis for targeted advertising, so as to achieve better advertising recommendation effect.
In this paper, a new method to detect the user's commercial intention is proposed. By analyzing the information of the user's web pages, the user's personalized purchase tendency is excavated. By the historical record of the search engine on the large shopping platform and the point distribution of the goods on the search result page, the search is established by statistical analysis. A keyword dictionary. Using this dictionary, we can use this dictionary to extract a series of key words that can reflect the user's commercial intentions. The criteria for keyword selection include the TF, ICF, QF, and the length of the keyword itself. Each keyword is based on the key words. The dictionary will be mapped to a series of commodity categories, in which the highest number of items will be considered as the result of the business intent detection of the page. In order to eliminate the impact of some unrelated items on the user's commercial intent results and make the OCI results more centralized, the similarity model of the category is introduced to modify the value of the OCI results. In order to reflect the transient and flexible features of the business intention, the business intention of the user's daily business intention is obtained by aggregating the OCI results that have been viewed on the same day. In order to demonstrate the real real time business intention of the user, we propose a business intention model in the user time dimension.
In order to evaluate the effectiveness of the proposed OCI detection method, in the experimental part, the judges manually annotate the results of the commercial intent of 1036 random test pages by formulating a unified evaluation standard. The accuracy rate reached 70% and the recall rate reached nearly 90%. From the point of view of the system goods business category, the correct rate of the system matching business category reached 86%. through the follow-up experiment. We also found that the system detects can be effectively promoted by restricting the number of key words and setting the minimum threshold of the matching category. The overall effect of the purpose of the industry.
As a result of the final analysis, the system gives the user's commercial intent for a period of time in the form of a series of commodity categories. According to the changes in the value of the items within a period of time, the user can speculate on the tendency and corresponding degree of the user to purchase a certain category of goods during this period. As a model, targeted products can be recommended to target users.
With the aid of large-scale distributed computing architecture, the OCI detection system process mentioned in this paper can be applied to the analysis of large-scale user behavior data. The operating system can handle one billion levels of user browsing records every day to analyze the specific business intentions of tens of millions of users, and the amount of data processing per hour is about 1. 50GB.
【学位授予单位】：上海交通大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3

【相似文献】