Machine learning has become one of the go-to methods for solving problems in the field of networking. This development is driven by data availability in large-scale networks and the commodification of machine learning frameworks. While this makes it easier for researchers to implement and deploy machine learning solutions on networks quickly, there are a number of vital factors to account for when using machine learning as an approach to a problem in networking and translate testing performance to real networks deployments successfully. This paper, rather than presenting a particular technical result, discusses the necessary considerations to obtain good results when using machine learning to analyze network-related data.
Detection of HTTPS Malware Traffic
In the last years there has been an increase in the amount of malware using HTTPS traffic for their communications. This situation pose a challenge for the security analysts because the traffic is encrypted and because it mostly looks like normal traffic. Therefore, there is a need to discover new features and methods to detect malware without decrypting the traffic. A detection method that does not need to unencrypt the traffic is cheaper (because no traffic interceptor is needed), faster and private, respecting the original idea of HTTPS. The goal of this thesis is to detect HTTPS malware connections by extracting new features and using data from the Bro IDS program. Since the data for the research is hard to come by, we used data from the Stratosphere project and we created, by hand, our own datasets. Our unit of analysis is an aggregation of all the information that is possible to obtain without decrypting the data. We group together flows, SSL data and X.509 certificates data as they are generated by Bro. To classify the HTTPS malware traffic we used several algorithms, such as Neural Networks, XGBoost and Random Forest. Our results show that the HTTPS malware behaviour is distinct from normal HTTPS behaviour and that our methods are able to separate them with an accuracy of at least 96.64%.
Observer effect: How Intercepting HTTPS traffic forces malware to change their behavior
During the last couple of years there has been an important surge on the use of HTTPs by malware. The reason for this increase is not completely understood yet, but it is hypothesized that it was forced by organizations only allowing web traffic to the Internet. Using HTTPs makes malware behavior similar to normal connections. Therefore, there has been a growing interest in understanding the usage of HTTPs by malware. This paper describes our research to obtain large quantities of real malware traffic using HTTPs, our use of man-in-the-middle HTTPs interceptor proxies to open and study the content, and our analysis of how the behavior of the malware changes after being intercepted. The research goal is to understand how malware uses HTTPs and the impact of intercepting its traffic. We conclude that the use of an interceptor proxy forces the malware to change its behavior and therefore should be carefully considered before being implemented.