基于数据挖掘算法的shellcode检测研究与实现

发布时间：2018-10-12 19:42

【摘要】：二进制程序安全一直是安全领域的重中之重。Shellcode是二进制程序漏洞利用的核心代码,是取得目标机器控制权进而达到攻击者想要执行的操作必不可少的一小段二进制字节。因此对shellcode的检测是二进制程序安全防护中非常重要的一部分。Shellcode可能内嵌于暗藏恶意的本地的文件中,也可能出现在注入攻击的网络流量之中。对shellcode进行检测,是网络IDS、蜜罐系统等必不可少的核心功能。常规的shellcode检测技术依靠人工提取的静态字节特征进行检测,随着网络攻击和防御技术地协同进化,不断有新的编写技术、变形技术、编码技术和多态技术被用来规避针对shellcode的检测,当今的常规检测方案已经渐渐不能适应日益复杂多变的网络安全环境。本文针对shellcode的检测问题进行了分析研究,提出并实现了一种基于数据挖掘算法的shellcode检测模型。首先,本文针对各种shellcode技术进行了说明,论证了基于指令序列建模的可行性。其次,分析和比对了静态和动态方法获取指令序列的优势劣势,论证了选取静态反汇编的可行性和必然性。再次,本文将自然语言处理中的词袋模型应用到shellcode检测的具体场景,使得本系统不依赖于特定的特征而从数据中自动学习特征,因此能实现更广范围的检测和更高的准确率。接着,本文在模型的基础上设计和实现了一个流量shellcode检测系统,通过一个统一的模型对shellcode解码段和普通shellcode进行双层检测,保证了其对各种shellcode的检测能力。最后,通过可靠来源的数据集设计实验,验证了该基于数据挖掘算法shellcode检测模型的有效性,验证了流量shellcode检测系统的检测效果。本文设计和实现的系统和shellcode检测库libemu进行了对比实验。实验表明,本文的系统有更好的检测效果和更全面的检测范围。其对采用多种技术的普通、变形、编码和多态shellcode都具有较好的检测效果和较高的检测效率。
[Abstract]:Binary program security has always been the top priority in the field of security. Shellcode is the core code to exploit the vulnerability of binary programs, and it is a necessary bit of binary bytes to gain control of the target machine and to achieve the operations an attacker wants to perform. Therefore the detection of shellcode is a very important part of binary program security. Shellcode may be embedded in malicious local files or in the network traffic of injection attack. The detection of shellcode is the essential core function of network IDS, honeypot system. The conventional shellcode detection technology relies on the static byte features extracted by hand to detect. With the coevolution of network attack and defense technology, there are constantly new programming techniques and deformation techniques. Coding techniques and polymorphic techniques are used to avoid the detection of shellcode. Nowadays, the conventional detection schemes are gradually unable to adapt to the increasingly complex and changeable network security environment. In this paper, the detection problem of shellcode is analyzed, and a shellcode detection model based on data mining algorithm is proposed and implemented. Firstly, this paper explains various shellcode technologies and demonstrates the feasibility of modeling based on instruction sequence. Secondly, the advantages and disadvantages of obtaining instruction sequences by static and dynamic methods are analyzed and compared, and the feasibility and inevitability of selecting static disassembly are demonstrated. Thirdly, this paper applies the word bag model in natural language processing to the specific scene of shellcode detection, so that the system can automatically learn features from the data without relying on specific features, so that it can achieve a wider range of detection and higher accuracy. Then, a traffic shellcode detection system is designed and implemented on the basis of the model. A unified model is used to detect the shellcode decoding segment and the ordinary shellcode, which ensures the detection ability of the shellcode. Finally, the validity of the shellcode detection model based on the data mining algorithm is verified by the data set design experiment of reliable sources, and the detection effect of the traffic shellcode detection system is verified. The system designed and implemented in this paper is compared with shellcode detection library libemu. Experiments show that the system has better detection effect and more comprehensive detection range. It has better detection effect and higher detection efficiency for common, deformation, coding and polymorphic shellcode.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP309;TP311.13

【相似文献】