BSidesSLC 2017 -- James Dickenson & Chris Tilley -- Network Security Monitoring Product Evaluation

Selection of a network security monitoring (NSM) product can be a difficult process and proper instrumentation is critical to the success of a SOC. The security world is in no short supply of vendors or solutions. However, the challenge remains determining which of the handful data points can be used reliably to make a procurement decision. We will share hard earned lessons from our experiences analyzing product reviews, validating performance claims, and field testing to validate implementations and real world performance. We will explain the framework we developed for evaluating performance criteria and describe the lab we built to execute tests in a controlled repeatable manner. We will then discuss how to distill test results into a concise report which aids in selecting a product that satisfies your prioritized requirements. In short what it takes to build a holistic and comprehensive view of the strengths and weaknesses of any IDS, SIEM, and other device you might be trying to evaluate.