False positive security test results — in which errors are flagged that aren’t truly errors — are a bane to software delivery, throwing up unnecessary roadblocks to progress. False positives are a struggle for every single security product on the market, making them unavoidable. They are especially annoying in agile development environments, where they can be a serious bottleneck to continuous software delivery.
Beyond the delays false positives can introduce, they may also degrade the relationship between Development and Security as the former may see the latter as wasting their time. Obviously, this is counterproductive to the goal of integrating security into the delivery pipeline, making it, as DevOps Expert Gene Kim says, “part of everyone’s job, every day” (The DevOps Handbook, Chapter 22).
Large organizations such as aerospace and multinational tech companies that deal with highly sensitive information must be particularly vigilant about security, but, like so many other companies, they too feel pressure to deliver value to their users ever faster. Security failures that block production deployments are, therefore, not an option. But neither is skimping on measures that ensure compliance and make software secure for end users. For these companies, finding a way to do it all—agile development, integrated security and effective management of false positives is critical. Fortunately, it’s also completely doable.
Scoring Security Testing
We recently worked with a large, multinational technology company that was struggling with how to integrate and coordinate their security effort without diminishing the agility of their software development process. This company’s Security team overlays all of Development and they wanted to create standards for pushing out code in a secure fashion. It was imperative that false positives/security checks not delay the development and release of applications. This company also wanted the ability to provide visibility into the process so that business users and compliance officers would feel comfortable that their security requirements were being covered.
The key to managing false positives is to score security testing based on where you are in the pipeline. This capability to vary your security test thresholds is built into our XL Release orchestration software. XL Release allows you to move security left by managing thresholds earlier in the value stream. Using your security software of choice (which you integrate with XL Release), you specify the vulnerability types and assign security thresholds that makes sense for the phase you’re in.
Even though the DevSecOps movement is in its infancy, there are proven patterns that work and use cases to learn from.
For example, you could set a lower security threshold in your Development phase, say 70 or 80% for most tests and 100% on absolute failures. As long as the code doesn’t exceed the threshold, you can move it forward, and then raise the threshold to 100% before going to Production. We not only enable you to adjust thresholds depending on where you are in the pipeline, but also based on which products you’re using. The large technology company we worked with has 5 different security tools, each with a different threshold.
These capabilities let you get the code out of Development and into QA so you can keep the code secure and the process agile. Once you integrate XL Release with your security software, XL Release handles management of the thresholds automatically. By managing thresholds earlier, you get better, faster security scanning, which generally leads to far superior software than waiting until the end when fixing problems is much harder.
For the tech company above, XL Release mitigated the false positives, shifted security left, and fully automated the management of their security thresholds. It has also enabled Development and Security to work together to ensure the safest code possible.