基于词序列拼积木模型的图像句子标注研究

发布时间：2018-04-22 03:17

本文选题：自然语言生成 + 词序列拼积木WSBB　；参考：《北京理工大学学报》2017年11期

【摘要】：用句子标注图像,建立图像与文本间的跨媒体关联,以提升信息检索准确率,改善用户检索交互体验.利用KDES模型抽取图像特征,在多核学习模型中融合出MK-KDES特征,准确刻画图像视觉特性;设计自然语言生成模型:词序列拼积木,评估单词与图像内容的相关性,优选单词,并根据单词间的语义相关性与句法模式约束,将单词组合成N元词序列;把N元词序列输入模板生成句子.结果表明:MK-KDES-1特征聚焦于图像的纹理及形状视觉特性,它是改善句子BLEU-1评分的关键;而单词间的语义相关性与句法模式约束是提升句子BLEU-2评分的重要前提.
[Abstract]:In order to improve the accuracy of information retrieval and improve the interactive experience of user retrieval, a cross-media association between image and text is established by tagging images with sentences. KDES model is used to extract image features, MK-KDES features are fused in the multi-core learning model to accurately depict the visual characteristics of images, and a natural language generation model is designed: word sequence building blocks, evaluating the correlation between words and image content, and selecting words. According to the semantic correlation between words and the constraints of syntactic patterns, the words are combined into N-meta word sequences, and the N meta-word sequences are input into templates to generate sentences. The results show that: MK-KDES-1 features focus on the texture and shape visual characteristics of the image, which is the key to improve the sentence BLEU-1 score, and the semantic correlation between words and syntactic pattern constraints are the important prerequisites to improve the sentence BLEU-2 score.
【作者单位】：华东交通大学软件学院;武汉大学计算机学院;
【基金】：国家自然科学基金资助项目(61373108,61173062) 教育部人文社科基金资助项目(16YJAZH029,17YJAZH117) 江西省科技厅科技攻关项目(20142BBG70011) 江西省社科规划基金资助项目(16TQ02) 江西省高校人文社科基金资助项目(XW1502、TQ1503) 江西省教育厅科技项目(GJJ160497,GJJ160509,GJJ160531) 江西省研究生创新基金(YC2016-S262)
【分类号】：TP391.41

【相似文献】