Abstract
Contemporary malware makes extensive use of different techniques such as packing, code obfuscation, polymorphism, and metamorphism, to evade signature-based detection. Traditional signature-based detection technique is hard to catch up with latest malware or unknown malware. Behavior-based detection models are being investigated as a new methodology to defeat malware. This kind of approaches typically relies on system call sequences/graphs to model a malicious specification/pattern. In this paper, we present a new class of attacks, namely “shadow attacks”, to evade current behavior-based malware detectors by partitioning one piece of malware into multiple “shadow processes”. None of the shadow processes contains a recognizable malicious behavior specification known to single-process-based malware detectors, yet those shadow processes as an ensemble can still fulfill the original malicious functionality. To demonstrate the feasibility of this attack, we have developed a compiler-level prototype tool, AutoShadow, to automatically generate shadow-process version of malware given the source code of original malware. Our preliminary result has demonstrated the effectiveness of shadow attacks in evading several behavior-based malware analysis/detection solutions in real world. With the increasing adoption of multi-core computers and multi-process programs, malware writers may exploit more such shadow attacks in the future. We hope our preliminary study can foster more discussion and research to improve current generation of behavior-based malware detectors to address this great potential threat before it becomes a security problem of the epidemic proportions.
Similar content being viewed by others
References
Martignoni, L., Stinson, E., Fredrikson, M., Jha, S., Mitchell, J. C.: A Layered Architecture for Detecting Malicious Behaviors. In: Proceedings of the 11th international Symposium on Recent Advances in intrusion Detection (RAID’08) (2008)
Lattner, C., Adve, V.: LLVM: A compilation framework for lifelong program analysis & transformation. In: Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO’04) (2004)
Christodorescu, M., Jha, S., Seshia, S., Song, D., Bryant, R.: Semantics-Aware Malware Detection. In: Proceedings of IEEE Symposium on Security and Privacy (2005)
Barford, P., Yagneswaran, V.: An Inside Look at Botnets. In: Advances in Information Security. Springer, Berlin (2006)
Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM conference on Computer and communications security (CCS’02) (2002)
Filiol E.: Formalisation and implementation aspects of k-ary (malicious) codes. J. Comput. Virol. 3(3), 75–86 (2007) (EICAR 2007 Best Academic Papers)
Harbour, N.: Stealth Secrets of the Malware Ninjas. https://www.blackhat.com/presentations/bh-usa-07/Harbour/Presentation/bh-usa-07-harbour.pdf.
Kolbitsch, C., Comparetti, P.M., Kruegel, C., Kirda, E., Zhou, X., Wang, X.: Effective and Efficient Malware Detection at the End Host. In: Proceedings of 18th USENIX Security Symposium (2009)
Nomenumbra: Counter Behavior Based Malware Analysis, Hacking at Random. HAR (2009)
Aciiçmez, O., Koç, Ç.K., Seifert, J.: On the power of simple branch prediction analysis. In: Proceedings of the 2nd ACM Symposium on information, Computer and Communications Security (ASIACCS’07) (2007)
Kernighan B.W., Lin S.: An efficient heuristic procedure for partition graphs. Bell Syst. Tech. J. 49, 291–307 (1970)
Christodorescu, M., Jha, S., Kruegel, C.: Mining specifications of malicious behavior. In: Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (2007)
Anubis. http://anubis.iseclab.org/
Lamport L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978)
Jiang, X., Walters, A., Buchholz, F., Xu, D., Wang, Y.M., Spafford, E.H.: Provenance-Aware Tracing of Worm Break-in and Contaminations: A Process Coloring Approach. In: Proceedings of 26th IEEE Int’l Conf. Distributed Computing Systems (ICDCS’06) (2006)
Fletcher, T.: Sharing a File Descriptor Between Processes. http://www.qnx.com/developers/articles/article_913_1.html
Yin, H., Song, D., Manuel, E., Kruegel, C., Kirda, E.: Panorama: Capturing system-wide information flow for malware detection and analysis. In: Proceedings of the 14th ACM Conferences on Computer and Communication Security (2007)
King, S.T., Chen, P.M.: Backtracking Intrusions. In: Proceedings of the 2003 Symposium on Operating Systems Principles, pp. 223–236 (2003)
Kirda, E., Kruegel, C., Banks, G., Vigna, G., Kemmerer, R.: Behavior-based Spyware Detection. In: Proceedings of the USENIX Security Symposium (2006)
Cohen F.: Computer viruses: theory and experiments. Comput. Secur. 6(1), 22–35 (1987)
Phoenix. https://connect.microsoft.com/Phoenix
Cavallaro, L., Saxena, P., Sekar, R.: On the limits of information flow techniques for malware analysis and containment. In: Proceedings of 5th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment (2008)
Szor P.: The Art of Computer Virus Research and Defense. Addison-Wesley Professional, Reading (2005)
Forrest, S., Perelson, A.S., Allen, L., Cherukuri, R., Self-Nonself Discrimination in a Computer. In: Proceedings of IEEE Symposium on Security & Privacy (1994)
Stinson, E., Mitchell, J.C.: Characterizing Bots’ Remote Control Behavior. In: Detection of Intrusions & Malware, and Vulnerability Assessment (2007)
Willems, C., Holz, T., Freiling, F.: Toward Automated Dynamic Malware Analysis Using CWSandbox. In: Proceedings of IEEE Security and Privacy (2007)
Kruegel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Automating mimicry attacks using static binary analysis. In: Proceedings of the 14th conference on USENIX Security Symposium (2005)
Norman Sandbox Whitepaper. http://www.norman.com
Srivastava, A., Lanzi, A., Giffin, J.: System Call API Obfuscation. In: Proceedings of the 11th International Symposium on Recent Advances in Intrusion Detection (2008)
Rieck, K., Holz, T., Willems, C., Düssel, P., Laskov, P.: Learning and Classification of Malware Behavior. In: Proceedings of Detection of Intrusions and Malware, and Vulnerability Assessment (2008)
Percival, C.: Cache missing for fun and profit. BSDCan (2005). http://www.daemonology.net/hyperthreading-considered-harmful/
Stevens R.: UNIX Network Programming, 2nd edn. Interprocess Communications, vol. 2. Prentice Hall, Englewood Cliffs (1999)
Dyshlevoi, K.V., Kamensky, V.E., Solovskaya, L.B.: Marshalling In Distributed Systems: Two Approaches (1997). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.9781
Borello J., Mé L.: Code obfuscation techniques for metamorphic viruses. J. Comput. Virol. 4, 211–220 (2008). doi:10.1007/s11416-008-0084-2
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ma, W., Duan, P., Liu, S. et al. Shadow attacks: automatically evading system-call-behavior based malware detection. J Comput Virol 8, 1–13 (2012). https://doi.org/10.1007/s11416-011-0157-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11416-011-0157-5