面向云计算的海量数据检索技术研究与应用

发布时间：2018-01-15 23:30

本文关键词：面向云计算的海量数据检索技术研究与应用　出处：《电子科技大学》2013年硕士论文　论文类型：学位论文

更多相关文章： 云计算 分布式存储 hadoop hbase

【摘要】：近十多年来，互联网产业迅猛发展日新月异，不仅让投资互联网的风险投资者们盈利颇丰，也成就了一批市值百亿美元的大型互联网企业。互联网产业给新兴经济实体带来非常强劲的发展动力。全球数亿的网民、企业、事业及政府部门单位正通过互联网中得到更多的资讯、数据交换、消费和业务的推广和应用。这是一个庞大而且有层次的用户群体。云计算正是互联网产业中一面新兴的旗帜。它结合了互联网络的优势，把大量的企业应用，个人服务等应用通过服务的方式基于互联网提供了广大的用户，比如桌面云、云存储等。面向云计算的应用与服务将是二十一世纪中最有价值、最有前景的技术。本文以云计算为基础，重点研究分布式数据存储下非结构化数据检索技术，并采用该技术设计一套面向广电行业的新闻线索汇聚平台的Sass服务。云平台采用了Hadoop作为其分布式数据存储平台，并构建集群的基础环境。采用hbase作为分布式面向列的非结构化数据库，作为检索引擎。研究过程中，注重实用性和科学性并重的原则。 1.分析Hadoop的分布式数据存储架构，，采用该架构构建检索的基础分布式技术平台。 2.研究MapReduce作业机制，采用该机制使用hbase基于列的分布式数据库进行设计，构建一个非结构化的检索引擎。 3.基于以上技术平台设计并搭建一套新闻线索汇聚业务平台。系统按照云服务的架构设计，支撑海量的新闻线索，并提供快的检索、分类聚类的业务功能。
[Abstract]:In the past more than 10 years, the rapid development of the Internet industry change rapidly, not only let the Internet investment risk investors are profitable, but also the achievements of a number of billions of dollars in market value of large Internet companies. The Internet industry has brought a very strong impetus to the development of the emerging economic entity. Enterprise global hundreds of millions of Internet users, and government departments, business units are more the information through the Internet, data exchange, promotion and application of consumer and business. This is a huge and hierarchical user groups.
Cloud computing is a banner of the emerging Internet industry. It combines the advantages of the Internet, a large number of enterprise applications, personal services and other applications by way of services based on the Internet provides the majority of users, such as desktop cloud, cloud storage. For cloud computing applications and services will be the most valuable in twenty-first Century in the most promising technology.
This paper focuses on the study of cloud based, distributed data storage under unstructured data retrieval technology, and design a set for the broadcasting industry news clues gathering platform Sass service cloud platform. Using the technology of using Hadoop as its distributed data storage platform, and build environment based clusters. Unstructured database using HBase as distributed column oriented, as search engines. In the course of the study, focusing on both practical and scientific principles.
1. analyze the distributed data storage architecture of Hadoop, and use this architecture to build a basic distributed technology platform for retrieval.
2. study the MapReduce operation mechanism, use this mechanism to design a HBase based distributed database, and build an unstructured retrieval engine.
3. design and build a set of news thread convergence business platform based on the above technical platform.
The system is designed in accordance with the architecture of cloud services, supporting massive news clues, and providing fast retrieval and classified clustering business functions.

【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】