DBFS技术及其在远洋运输业务邮件管理中的应用研究

发布时间：2018-07-05 19:38

本文选题：DBFS + Lucene　；参考：《南京航空航天大学》2012年硕士论文

【摘要】：传统的基于目录和文件的层级文件系统沿用至今，虽然这样的树型文件结构给用户提供了简单易用的文件存放和修改方法，但同时也带来了定位文件和目录的困难。随着硬件性能和磁盘存储容量的不断提高，计算机中的文件数目不断增加，而传统的层级文件系统的诸多缺点（如对文件进行描述的元数据信息匮乏且不易扩充）越来越明显。尤其是在单个文件体积小、数量级大的文件管理情形下，，用传统的层级文件系统对文件进行查找和定位非常困难，而这一情形下的文件管理也成为一个难题。本通过对用户态下的DBFS（Database-based File System，数据库文件系统）技术的研究来解决文件体积小、数量级大的文件管理难题，而现有的用户态下的DBFS技术无法很好地解决这一问题，因此本文对现有DBFS技术进行完善，通过对数据库技术和全文检索技术的研究对比，提出利用嵌入式数据库SQLite和全文检索引擎Lucene对现有的DBFS模型进行改进，使其既能提供基于文件元数据也能提供基于文本内容的快速检索。通过对Lucene的深入学习，本文对全文检索的相关技术进行如下研究和改进：1、针对应用领域的用户需求，对倒排索引进行个性化改进；2、结合TF-IDF加权算法对Lucene现有结果排序算法进行改进，使其在体现文档和特征词关联度的基础上，更好地体现用户对不同信息的不同侧重程度，从而更好地满足实际检索需求。最后结合远洋运输业务邮件管理需求，构建了基于DBFS的远洋运输业务邮件管理原型系统，与Uniwell（H.K.）公司的实际业务数据相结合，进行应用研究分析，验证本文改进的DBFS模型在单个文件体积小、数量级大的小文件管理方面的有效性，并且通过实验数据验证了其对于倒排索引的个性化研究和结果排序算法改进的有效性。本课题的研究为类似Uniwell（H.K.）这样的远洋运输公司提供了对积累的海量远洋运输业务邮件的快速查找和管理方法，提高了远洋运输公司在累积的海量信息中迅速获得航次决策信息的效率，从而为有效的航次决策提供支持。综上所述，论文的研究成果具有较高的理论意义和实用价值。
[Abstract]:The traditional hierarchical file system based on directories and files has been used up to now. Although this tree file structure provides users with a simple and easy to use file storage and modification methods, it also brings difficulties in locating files and directories. With the improvement of hardware performance and disk storage capacity, the number of files in the computer is increasing, and many disadvantages of traditional hierarchical file system (such as the lack of metadata information described to the file) are becoming more and more obvious. Especially in the case of single file with small size and large order of magnitude, it is very difficult to find and locate files by traditional hierarchical file system, and file management becomes a difficult problem in this case. In this paper, we study DBFS (Database based File system) technology in user state to solve the file management problem of small size and large order of magnitude, but the existing DBFS technology in user state can not solve this problem very well. In this paper, the existing DBFS technology is improved, and the existing DBFS model is improved by using the embedded database SQLite and the full-text retrieval engine Lucene through the research and comparison of database technology and full-text retrieval technology. It can provide both file-based metadata and text-based content-based fast retrieval. Through the in-depth study of Lucene, this paper carries on the following research and improvement to the related technology of full-text retrieval, aiming at the user demand of the application domain, carries on the personalized improvement to the inverted index; 2. Based on the TF-IDF weighted algorithm, Lucene's existing result sorting algorithm is improved to better reflect the different emphasis degree of users on different information on the basis of reflecting the correlation degree of documents and feature words, so as to better meet the actual retrieval requirements. Finally, combining with the mail management requirements of ocean transportation business, the prototype system of ocean shipping mail management based on DBFS is constructed, and Uniwell (H.K.) Combining the actual business data of the company, the application research and analysis are carried out to verify the effectiveness of the improved DBFS model in the management of small files with small size and large order of magnitude. The effectiveness of the personalized research on inverted index and the improvement of the result sorting algorithm are verified by the experimental data. The research of this subject is similar to that of Uniwell (H. K.) Such ocean shipping companies provide a fast search and management method for the accumulated mass of ocean shipping business mail, and improve the efficiency of ocean shipping companies to quickly obtain voyage decision information from the accumulated mass information. It provides support for effective voyage decision making. To sum up, the research results have high theoretical significance and practical value.
【学位授予单位】：南京航空航天大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：F270.7;F550.6

【参考文献】