基于数据挖掘的淘宝商品竞争力分析系统的设计与实现

发布时间：2018-01-29 16:27

本文关键词： 网络爬虫数据挖掘电子商务决策支持　出处：《山东大学》2015年硕士论文　论文类型：学位论文

【摘要】：当今社会,科技发展日新月异,网络科技的普及为新兴的电子商务交易注入了崭新的活力,不仅减少了人力物力的损耗,而且创造了巨额的商业利润。在电子商务交易中,顾客通过网络购买商品,只能浏览网页图片信息介绍,不能实际观察商品,对电子商务交易的真实性容易产生怀疑。在这个过程中,一方面需要网络政策法规的约束,另一方面也要对交易商品进行仔细对比和审查。与此同时,商户也需要对出售的商品进行分析,了解同行业商品的优劣和价格走向,根据市场信息做出商业决策。但是目前网络上商品种类繁多,数量巨大,对商品信息的获取并非易事。本文主要通过网络爬虫的方法采集在网络交易中具有代表性的天猫商城在售商品信息,并对商品各种信息进行分析和判断,得到决策支持,从而满足交易双方的需求。本系统主要是应用网络爬虫技术对天猫商城在线商品信息进行采集,并对采集的网页信息进行筛选,提取出商品类别、商品标签、商品品牌、商品详细介绍、评论信息及店铺介绍等信息,并将这些数据经过系统处理,然后存入数据库中。采用主题爬虫的方式进行数据采集,有利于页面信息的分析,方便URL链接的定位,减少提取次数提高提取信息的效率,而且可以灵活设定提取开始的类别及层次,能够有效保证采集数据的真实性和实时性。对于采集的数据主要通过数据挖掘的方式进行分析。通过对商品介绍、用户评价、店铺介绍等信息的整理和分析,可以找出销售、价格、发货等方面的排名情况,进而可以分析商品的潜在价值,获取该商品的竞争力排名,可以成为买家置信商品理论依据。更深的层次可以挖掘交易双方的交易行为等情况,方便商家做出策略调整以提高销售量或应时上新。目前本系统可以正常运行,但工作效率随着网络数据的增加还有提升空间。未来的发展方向是多线程并发,大规模数据采集和整理。下一步工作将着重发展多线程数据采集,对更新数据的定点提取方面进行完善,并加强分析及数据挖掘的范围与深度。
[Abstract]:Nowadays, with the rapid development of science and technology, the popularity of network technology has injected new vitality into the emerging e-commerce transactions, not only reducing the loss of manpower and material resources. And created a huge commercial profit. In e-commerce transactions, customers through the network to buy goods, can only browse the web page picture information introduction, can not actually observe the goods. It is easy to doubt the authenticity of electronic commerce transactions. In this process, on the one hand, we need network policies and regulations, on the other hand, we should carefully compare and examine the traded commodities. At the same time. Merchants also need to analyze the goods sold to understand the advantages and disadvantages of the same industry commodities and price trends, according to market information to make business decisions. But at present, there are many kinds of goods on the network, and the quantity is huge. It is not easy to obtain commodity information. In this paper, we collect the representative information of Tmall in the online transaction through the method of web crawler, and analyze and judge all kinds of commodity information. This system mainly uses web crawler technology to collect Tmall online commodity information and screen the collected web page information. Extract the category, label, brand, detailed description, comment information and shop description, and process these data through the system. Then stored in the database. Using the method of topic crawler to collect data, which is conducive to the analysis of page information, facilitate the location of URL links, reduce the number of times of extraction to improve the efficiency of information extraction. And can flexibly set the beginning of the extraction of categories and levels, can effectively ensure the authenticity of the collected data and real-time. For the collected data mainly through the way of data mining analysis, through the introduction of commodities. User evaluation, shop introduction and other information collation and analysis, can find out the sales, prices, shipping and other aspects of the ranking, and then can analyze the potential value of goods, obtain the competitiveness of the goods ranking. It can be used as the theoretical basis for the buyer to believe in the commodity. A deeper level can be used to excavate the transaction behavior of both sides of the transaction. It is convenient for the merchant to adjust the strategy to increase the sales volume or update the time. At present, the system can run normally, but with the increase of network data, there is still room for improvement. The future development direction is multithreading and concurrency. The next work will focus on the development of multithreaded data acquisition, improve the fixed-point extraction of updated data, and strengthen the scope and depth of analysis and data mining.
【学位授予单位】：山东大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP391.1;F724.6

【参考文献】