网上专家经济观点自动挖掘研究

发布时间：2018-02-15 04:04

本文关键词： 非结构化数据文本数据自然语言处理技术空间向量模型　出处：《首都经济贸易大学》2017年硕士论文　论文类型：学位论文

【摘要】：前互联网时代,数据的获取受制于时间、空间等因素的影响,由于当时网络不发达,数据获取的难度大、速度慢、来源少,导致人们对于数据的获取感到非常困难。并且在数据处理上,所能处理的数据类型大多是结构化数据,对于非结构化数据还未能从容应对。现在,随着互联网的快速发展,信息获取的广度增加、速度加快,其也不再受制于时间和空间,再加上非结构化数据占比越来越多,传统的结构化数据等早已不能满足人们的分析需要。这时,数据的获取开始受限于信息的整合。因为面对现如今海量的数据,尤其是诸如文本、声音、图像等非结构化数据,如何将其中有用的信息进行整合,已经成为了一个重要的问题。目前,信息整合大多依赖于人力,那么如何实现自动化信息整合就成为了一项挑战。本文研究的目的是以经济形势数据为例,通过python urllib爬虫技术实现数据的获取;利用专家观点筛选公式实现数据的筛选;使用描述统计相关方法实现数据的整体把握;运用自然语言处理技术实现文本数据的处理;结合空间向量模型实现文本的聚类;自建文章自动撰写系统实现数据的整合。通过本研究,使得我们可以自动处理文本数据,从而实现一套自动化处理文本的流程。为人们的决策起到辅助作用,极大提高人们的效率,方便人们的生活。
[Abstract]:In the pre-Internet era, the acquisition of data was affected by time, space and other factors. Because of the underdevelopment of the network at that time, it was difficult to obtain data, slow in speed, and few in sources. It makes it very difficult for people to get data. And in data processing, most of the data types that can be processed are structured data, and they can't deal with unstructured data. Now, with the rapid development of the Internet, The breadth and speed of access to information is increasing, and it is no longer constrained by time and space. In addition, with the increasing proportion of unstructured data, the traditional structured data has long been unable to meet the analytical needs of people. Access to data began to be limited by the integration of information, because in the face of today's massive amounts of data, especially unstructured data such as text, sound, and images, how to integrate useful information into them, At present, information integration mostly depends on manpower, so how to realize automation information integration has become a challenge. The purpose of this paper is to take the economic situation data as an example. Through python urllib crawler technology to achieve data acquisition, using expert view screening formula to achieve data screening, using descriptive statistics related methods to achieve the overall grasp of data, using natural language processing technology to achieve text data processing; Combining spatial vector model to achieve text clustering, self-built automatic writing system to achieve data integration. Through this study, we can automatically process text data, Thus a set of automatic process of text processing is realized, which plays an auxiliary role in people's decision-making, greatly improves people's efficiency and facilitates people's life.
【学位授予单位】：首都经济贸易大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【参考文献】