面向电商运营平台的业务追踪系统的设计与实现

发布时间：2018-07-05 03:30

本文选题：电子商务 + 数据采集　；参考：《东南大学》2016年硕士论文

【摘要】：随着电子商务的发展,电子商务系统变得越来越复杂。现在的电子商务系统不但需要支持各种各样的商品展示、购买和配送,还需要处理电商大促时产生的巨大交易流量,为了满足这个需求,电子商务系统需要由许多运营系统来支持。一次成功的购物需要各个系统之间相互配合完成,而这期间任何一个系统出现问题都会导致购物的失败,而且问题的定位和排查也非常困难。为了使得在复杂的运营平台下高效、快捷地定位和排查问题,本文分析了电商业务追踪系统的需求,设计并实现了面向电商运营平台的业务追踪系统。通过对日志数据的采集和处理,本系统能够再现整个业务的执行流程和进度,便于问题的定位和排查,同时,根据数据聚类结果与历史数据的比较结果来实现业务风险预警。本文主要工作归纳如下：(1)提出了面向电商运营平台的业务追踪系统的设计方案。根据系统的输入输出,对系统的需求进行了分析,梳理了系统的主要功能,设计了业务追踪系统的总体架构和数据库表结构。(2)提出了电商业务系统中日志数据的采集与处理方法,包括日志数据的读取、筛选、切分、处理和保存方案,其中采用Sunday字符串匹配算法来筛选日志,使用布隆过滤器来进行日志去重,采用分段生成唯一序号的方法来生成本系统的数据库主键等。(3)提出了电商业务追踪与分析方法,包括电商业务追踪和分析过程中的数据查询、排序、聚类和预警方案,其中采用并发查询和排序进行日志的追踪,使用聚类算法对数据进行聚类,通过与历史数据对比来进行业务预警等。(4)实现了电商业务追踪系统,并进行了系统测试,现已在企业真实的场景下进行了部署和使用。
[Abstract]:With the development of e-commerce, e-commerce system becomes more and more complex. Today's e-commerce systems not only need to support the display, purchase and distribution of all kinds of goods, but also need to deal with the huge volume of transactions generated by the promotion of e-commerce, in order to meet this demand. E-commerce systems need to be supported by many operating systems. A successful shopping requires the cooperation between the various systems, and during this period any system problems will lead to the failure of shopping, and the problem location and troubleshooting is very difficult. In order to make the problem of locating and checking efficiently and quickly under the complex operation platform, this paper analyzes the requirements of the e-commerce business tracking system, and designs and implements a business tracking system oriented to the e-commerce operation platform. Through the collection and processing of log data, the system can reproduce the whole business execution process and progress, easy to locate and search problems, at the same time, according to the data clustering results and historical data comparison results to achieve business risk early warning. The main work of this paper is summarized as follows: (1) the design of business tracking system for e-commerce platform is proposed. According to the input and output of the system, the requirements of the system are analyzed, and the main functions of the system are sorted out. The architecture of the service tracking system and the structure of the database table are designed. (2) the methods of collecting and processing the log data in the e-commerce business system are proposed, including the reading, filtering, splitting, processing and saving of the log data. Among them, Sunday string matching algorithm is used to filter log, Bron filter is used to remove log, and the method of creating unique sequence number is used to generate the database primary key of the system. (3) the method of tracking and analyzing e-commerce services is proposed. It includes data query, sorting, clustering and early warning scheme in the process of business tracing and analysis, in which concurrent query and sorting are used for log tracing, and clustering algorithm is used to cluster data. Compared with the historical data to carry on the business early warning and so on. (4) implemented the electronic commerce service tracking system, and carried on the system test, has now carried on the deployment and the use in the enterprise real scene.
【学位授予单位】：东南大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.52

【参考文献】