面向评测基准的社交流数据生成

发布时间：2018-05-06 12:11

本文选题：社交流 + 数据生成器　；参考：《华东师范大学》2016年博士论文

【摘要】：社交流数据指记录了社交实体状态及实体间关系动态变化的数据流。它可以用来表示大量应用中的实体状态变化,如社交媒体数据中用户发布和转发信息的动态、利技文献间的引用、分布式系统中各节点间数据的传输等。社交流数据与传统的网络和流数据不同,它既是一系列是实体状态数据流,又是动态变化的网络数据,它是图数据和流数据的结合。正是由于社交流数据存在图和流数据的复合特点,因此,社交流数据存在巨大的商业和研究价值,有效的数据管理和挖掘是学术界和工业界共同关注的焦点。目前,多种技术可以用来管理或者处理社交流数据,如何针对应用选取合适的数据生成器是评测基准需要解决的问题。然而,出于对隐私、数据量庞大不易转移等问题的考虑,评测基准通常无法提供真实数据评测系统。因此,一个能够灵活地、高效地产生大规模“真实的”人工数据的生成器具有重要意义。本文面向评测基准的需求研究产生社交流数据的方法。提出的产生社交流数据的方法能针对不同类型社交流数据产生与“真实的”社交流数据特征相一致的数据。为了实现高吞吐量的产生大规模数据的目的,本文设计并实现了一个分布式生成生成社交流数据的系统。此外,本文以社交流数据生成器为基础,设计了一个基于社交媒体分析查询的评测基准。总之,全文围绕着社交流生成这一问题而展开,主要贡献具体包括以下四个方面1.提出了一种基于人类动力学模型和时序生长网络模型产生单链接社交流数据的方法单链接社交流中的社交项目最多可连接一个历史项目。该方法利用两个缓冲区的迭代更新而顺序地产生单链接社交流数据。其中一个缓冲区是未来项目缓冲区：它存储生产者未来将来发布的社交项目。另一是近期项目缓冲池,其中保存了指定窗口大小的近期历史项目。在两个缓冲区迭代更新的过程中,该方法使用人类动力学模型为每个生产者产生不带链接信息的社交项目,利用时序生长网络模型确定社交项目的项目链接信息。用户可以通过配置参数产生指定规模、数据分布和类型的数据集。通过实验表明,提出的方法能够以稳定的吞吐量和内存占用持续地产生“真实的”单链接社交流数据。2.提出了一种基于人类动力学模型和网络生成模型产生多链接社交流数据的方法多链接社交流内的社交项目可以连接多个历史项目,因此,在产生多链接社交流数据的过程中对社交项目的链接生成部分有新的要求。该方法在产生单链接社交流数据方法的基础上,同样使用两个缓冲池迭代更新的方法顺序地产生多链接社交流数据。在生产社交项目的链接信息时,扩展的时序生长模型和边复制模型都可以用来产生链接信息。经实验验证分析,利用扩展的时序生长模型产生的社交流数据能更好地匹配真实的数据分布。基于扩展的时序生长模型产生多链接社交流的方法能够以稳定的吞吐量和内存占用持续地产生“真实的”多链接社交流数据。3.采用主从架构,实现了一个分布式生成社交流数据的系统为了实现高吞吐量的产生大规模社交流数据的目的,该系统可分布式地产生单链接和多链接社交流数据。该系统采用单个主机和多个工作节点的架构产生数据。工作节点利用产生单链接和多链接社交流数据的方法,在使用时序生长模型产生链接信息的基础上,产生指定生产者分区内的社交流数据。主机则合并来自各个节点的部分社交流以产生最终的全局社交流数据。该系统利用分布式链接生成方法、异步模型和延迟更新策略具体实现分布式生产数据的细节。通过实验表明,分布式数据生成系统在产生不失真数据的前提下,可以通过增加节点的方式实现生产数据吞吐量的线性增长。4.以社交流数据生成器为基础,设计了一个基于社交媒体分析查询的基准测试社交媒体服务已经成为互联网上最流行的服务之一,社交媒体数据是—类典型的社交流数据。本文设计了一种基于社交媒体数据分析的评测基准BSMA,它包含了数据支持、负载生成器和一个性能测试工具。负载生成器部分定义了社交媒体的数据模型,并在此基础上定义了4类24个查询模版,并提供一个可以根据需求产生不同参数值给查询任务的参数生成器。数据支持部分不仅提供提供真实的新浪微博数据集,还提供能够产生社交流数据的生成器BSMA-Gen。BSMA-Gen使用本文提供的生成社交流数据的方法产生数据。24个查询模版中包含多个基于社交流数据中的时序和链接关系网络的查询,BSMA-Gen可作为该类查询的数据支持。用户可以使用测试工具连接待测系统、配置并执行测试任务,最终根据定义的三个评测指标输出评测结果。综上所述,本文形式化定义了社交流模型以及相关特征。提出产生单链接和多链接社交流数据的架构、模型和生成算法。用户可以根据需求来配置社交流数据生成器,使其产生指定数据分布、指定类型的数据。为了能够高吞吐量产生大规模社交流数据,本文设计并实现了一个分布式生成系统。本文以社交流数据生成器为基础,设计了一个基于社交媒体查询的评测基准。
[Abstract]:Social exchange data refers to a data stream that records dynamic changes in the state of a social entity and the relationship between entities. It can be used to represent the changes in the state of the entity in a large number of applications, such as the dynamic of the user's release and forwarding of information in the social media data, the reference between the benefit and technology documents, the transmission of data among the nodes in the distributed system, and so on. The traditional network and flow data are different. It is a series of entity state data flow and dynamic changing network data. It is the combination of graph data and stream data. It is the complex characteristics of graph and stream data of social exchange data. Therefore, social exchange data has huge commercial and research value, effective data management and excavation. Mining is the focus of both the academia and the industry. At present, a variety of technologies can be used to manage or deal with the exchange of data, and how to select a suitable data generator for applications is a problem to be solved for benchmarking. However, the benchmarks are usually not available for consideration of privacy and the large amount of data that is not easy to transfer. Real data evaluation system. Therefore, a flexible, efficient generation of large scale "real" artificial data is of great significance. In order to achieve high throughput, this paper designs and implements a distributed generation system for generating data exchange. In addition, based on the social communication data generator, this paper designs a benchmarking based on social media analysis query. The full text is focused on the problem of social exchange. The main contributions include the following four aspects: 1., a method based on the human dynamics model and the time series growth network model to generate single chain connection exchange data is proposed. The social projects in single chain connection exchanges can be connected to one historical project at most. The method uses two slow events. One buffer zone is the future project buffer: it stores the future social projects that the producer will release in the future. The other is the recent project buffer pool, which preserves the recent historical project of the size of the specified window. In the process of iteration of the two buffer zones, this party The method uses the human dynamics model to generate social projects without link information for each producer, using the time series growth network model to determine the link information of social projects. Users can generate a specified size, data distribution and type of data set by configuring parameters. Quantity and memory occupy a continuous generation of "real" single chain communication data..2. proposes a method based on human dynamics model and network generation model to generate multi link societies to exchange data. There are new requirements for the link generation of social projects. Based on the method of generating single chain link exchange data, this method also uses two buffer pools to iterate update methods in order to exchange data in the real estate multi link society. In the production of link information for social projects, the extended time sequence growth model and the edge replication model can be used for the production of social project. It produces link information. Through experimental verification, the cooperative data generated by the extended time series growth model can match the real data distribution better. The method of generating multi link communication based on the extended time series growth model can continue to live a real "real" multi link exchange with stable throughput and memory occupation. According to.3., a distributed generation society exchange data system is implemented. In order to achieve high throughput, the system can generate large scale social exchange data. This system can distribute data from single link and multi link society. The system uses a single host and multiple work nodes to generate data. Using the method of generating single link and multi link data exchange data, on the basis of using the time series growth model to generate link information, the cooperative data in the designated producer partition is generated. The host combines the social communication from each node to produce the final global social flow data. The system uses the distributed link generation method. The step model and the delay update strategy implement the details of the distributed production data. The experiment shows that the distributed data generation system can achieve linear growth of production throughput by increasing nodes, based on social communication data generator, based on the premise of producing undistorted data, and design a social based.4. based on social communication data generator. Social media service has become one of the most popular services on the Internet, social media data is a typical social exchange data. This paper designs a benchmarking BSMA based on social media data analysis, which includes data support, load generator, and a performance testing tool. The generator part defines the data model of social media, and on this basis defines 4 classes of 24 query templates, and provides a parameter generator that produces different parameter values to the query task according to the requirements. The data support section not only provides a real Sina micro-blog data set, but also provides the generation of social exchange data. The BSMA-Gen.BSMA-Gen uses the method of exchanging data provided by the generator in this article to produce data.24 query templates containing multiple queries based on time series and link relations based on social communication data. BSMA-Gen can be used as data support for this type of query. In conclusion, this paper formally defines the social communication model and the related features. This paper presents a framework, model and generation algorithm for generating the data of single link and multi link society. Users can configure the social data generator to produce the specified data according to the requirements. A distributed generation system is designed and implemented in this paper to generate large scale social exchange data for high throughput. This paper designs a benchmarking based on social media query based on social media query.

【学位授予单位】：华东师范大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP311.13

【相似文献】