基于用户分类的隐马尔可夫WEB预取模型及应用研究

发布时间：2018-09-03 18:49

【摘要】：Web预取技术是在分析用户访问的相关数据或行为的基础上,主动预测其下一步可能浏览的页面,通过隐性的请求加载,从而预先取得并存放在缓存中,以备用户访问,从而减少用户访问时因为网络或服务器等各种可能问题造成的时延。基于Web访问模式特性及预取与缓存的基本理论,本文在前人的基础上充分发掘Web访问过程中的规律和特性,采用概率统计、数学分析等方法,改进了包括日志处理,特征词提取,资源预测及资源缓存与替换等算法,建立了一套预取一体化框架。在预取一体化框架的研究中,主要工作包括:(1)在日志数据处理时对路径的补充工作提出了描点法,用以补充日志记录中的访问路径序列遗漏、缺失的信息,以还原一个完整合理的访问行为,该算法简单有效,易于实现;(2)在特征词处理时对传统的特征词提取算法TF-IDF(Term Frequency Inverse Document Frequency)进行了改进,提出了TF-IDF-CD(Term Frequency Inverse Document Frequency-Categorical Description)算法,该算法解决了传统特征词提取过程中对于类别区分能力较弱的问题;(3)资源预测过程是在原马尔可夫预取模型的基础上对用户进行分类,同时对用户访问路径从语义的角度进行分析,提取用户信息需求,从而形成了基于用户分类的隐马尔可夫预取模型,该算法将基于访问路径和基于语义两类算法思想相结合,以达到更好的预测准确率。(4)在资源缓存与替换中,我们在GDS(Greedy-Dual-Size)和GDSF(Greedy-Dual-Size-Frequency)算法的基础上,引入时间频度概念,提出了GDSF-T(Greedy-Dual-Size-Frequency-Time)算法,该算法弥补了时间因素对访问频度的影响。最后,将该框架应用于一个基于微信端的农产品产销平台中,以针对此类移动平台特有的带宽低、延迟高、间歇性连通等网络特点,达到降低访问时延、优化系统性能的目的,并通过实验对系统引入预取框架前后对系统性能的影响进行了测试和分析,测试结果显示各项指标均良好。
[Abstract]:Web prefetching technology is based on the analysis of relevant data or behaviors accessed by the user, and actively predicts the pages it may visit next, and loads the pages through hidden requests, which can be obtained in advance and stored in the cache for the user to visit. Thus, the delay caused by various possible problems such as network or server is reduced when the user accesses. Based on the characteristics of Web access mode and the basic theory of prefetching and caching, this paper fully explores the rules and characteristics of Web access process on the basis of predecessors, and adopts the methods of probability statistics and mathematical analysis to improve the log processing. An integrated prefetching framework is established, such as feature extraction, resource prediction and resource cache and replacement. In the research of pre-fetching integrated framework, the main work includes: (1) in the process of log data processing, a new method is proposed to supplement the missing and missing information of access path sequence in log records. In order to restore a complete and reasonable access behavior, the algorithm is simple and effective and easy to implement. (2) the traditional feature word extraction algorithm TF-IDF (Term Frequency / Inverse Document Frequency) is improved, and the TF-IDF-CD (Term Frequency Inverse Document Frequency-Categorical Description algorithm is proposed. The algorithm solves the problem of weak classification ability in the traditional feature extraction process. (3) the resource prediction process classifies users on the basis of the original Markov prefetching model. At the same time, the user access path is analyzed from the point of view of semantics, and user information requirements are extracted, and a hidden Markov prefetching model based on user classification is formed. The algorithm combines two kinds of algorithms based on access path and semantics. In order to achieve better prediction accuracy. (4) in resource cache and replacement, based on GDS (Greedy-Dual-Size) and GDSF (Greedy-Dual-Size-Frequency) algorithm, we introduce the concept of time frequency, and propose GDSF-T (Greedy-Dual-Size-Frequency-Time) algorithm, which makes up for the influence of time factor on access frequency. Finally, the framework is applied to a production and marketing platform of agricultural products based on WeChat, in order to reduce the access delay and optimize the system performance in view of the unique characteristics of the mobile platform, such as low bandwidth, high delay, intermittent connectivity and so on. The performance of the system is tested and analyzed before and after the introduction of the prefetching frame. The test results show that the performance of the system is good.
【学位授予单位】：西安理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.0;O211.62

【相似文献】