Data Analytics-Based Detection At Scale

On this talk we plan to present a data analytics platform built in-house, including some key components that enable us to efficiently correlate events and detect more sophisticated attacks. In 2017, an average of 200,000 new malware samples have been captured, each day, increasing by 328% from the previous year. Cybercriminals have stepped up their game, and they already use advanced techniques to penetrate organization defenses. In order to tackle the tremendous increase in attacks, we built a next generation Data Analytics platform to extend and enhance the current systems in place. On this talk, we will explore some of the challenges of building in-house a data analytics platform. These components include a fully featured correlation engine that is highly scalable while maintaining low latencies and the use of machine learning algorithms to detect more sophisticated attacks. We plan to provide an overview of this detection platform and discuss in detail some components, leveraging Big Data Technologies - like Apache Flink and Spark, for complex correlation processes and data transformations on live streams of events. Moreover, we will discuss how we used a Deep Learning approach to identify malicious domains on Command and Control server traffic.