Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
This is the third installment in this series on DevSecOps. Read the first installment, on static analysis, here and the second installment, on source composition analysis, here.
One weakness of static analysis is its failure to account for environment and use. Running static analysis on a code base as the only check before production deployment is akin to looking at the engine of a VW Golf before assembly and pronouncing the vehicle fit to use. The way a vehicle runs after assembly, and the way it is—or, more precisely for dynamic scans, could be—used, matter to any product. That is where dynamic scans come in.
Exercising the product as it runs is a big deal, and that is what dynamic scanning attempts to do. Automating the testing of applications by exercising inputs and watching the results, dynamic scans can detect a variety of issues that static analysis simply cannot. These tools are the source of a lot of the noise in DevSecOps because they’re testing a variety of scenarios with each run, and things that a dynamic scan sees as vulnerabilities might very well be known to not be a risk at all by the dev team. As with static analysis, this has been improving over time, and they’re well worth a look at this point.
The best use for these tools is as gateways—before going to test, in test and before going to production, set a policy for promotion requirements and then run dynamic scans to determine if the application meets that policy, needs remediation or needs approval to go live with possible issues known.
With all of that said, here is a starter list of things to look for when considering using dynamic scans:
- Production Safe: These tools run tests that try a documented vulnerability to see how the application responds. The thing is, if you’re not very careful, this can cause serious problems in production. Since long-term the trend is to eliminate a pure test environment for all but the most stability-conscious organizations, knowing that the product you choose can run in prod without causing outages is an important step.
- Vulnerabilities Tested: Most vendors offer a list of what they look for. Compare these lists to your organization’s needs and other vendor’s lists. Since choosing a product is unique to each org, I don’t have much in the way of buying advice, but here I will say if the product does not support at least the OWASP top 10, look elsewhere, simply because OWASP is both a recipe for you to prevent attacks and a handy list of easy vulnerabilities for attackers to look for. All of the vendors we looked at for our ASG work do support OWASP, so this might be non-advice.
- Integration: Dynamic scans need to be run after the product is fully assembled. This is not to say complete, but the various pieces need to be together—the scans generally look at an application, not pieces thereof. This means they can be spawned at any point after build—or, more specifically, after post-build deployment to dev/test/prod. Make certain there is a place in the toolchain to call the scanner from and proper gateways to suit your organization’s needs. Part of that evaluation is how the application is kicked off: Can your build tool call it directly? Is it deeply integrated into your chosen environment, or will you have to do work?
- Reporting and Gating: What level of reporting is available: Can dynamic reports and static reports be integrated to show that a potential problem reported by static scan is found to actually be a problem by the dynamic scan? Can an overall status report be found (à la C-level dashboards) to show the security posture of a given application? Can alerts be configured for apps that fail certain tests (so you can make informed decisions before going to production, for example)?
- RBAC: This is applicable to all scanning, but I’ve chosen to include it here because for dynamic (and interactive), it’s a bit more important. Can you limit what an individual can see/do? In the end, dynamic scans are reporting on actual vulnerabilities in your codebase—only those who need to know to fix the issue should be able to see it; others should just see pass/fail overall for the app, not a list of known vulnerabilities in the app. The ability to run these scans should likewise be limited. Who/when is a corporate decision that is as much policy and process as it is the chosen tool, but if the scanner doesn’t support that policy, enforcement becomes more difficult.
When you get to interactive scanning, it is time to consider how security issues traditionally have been handled versus how they should be moving forward. Most organizations hold security issues separate from other application development issues. It is time for that to end—and indeed, some organizations ended this process long ago, but too many are hanging on to the past. Security vulnerabilities that can result in a risk to the organization need to be run through the system as high-priority bugs. Because they are. We used to think that something making the UI look bad for 5% of customers was more important than a security issue impacting no one. A truly global internet and the rise of automation even among attackers make this a silly stance to take. No one is impacted by security issues at this second, but sometime in the next minute, everyone might be impacted. It’s like saying you don’t need scalability for Black Friday because the servers are handling traffic well right now (written in early November). The issues that dynamic scanners alert on are generally real (removal of false positives has come a long way in the dynamic space) and generally dangerous. If your scan tool can find it, assume attackers’ scan tools can, too.
As with the other installments, I’ll thank you all for keeping the lights on in a really topsy-turvy IT environment—you are doing us all a service—and mention that all of your hard work is at risk if security at your organization isn’t taken seriously. We quickly identified the threat from the sudden opening of millions of RDP ports on the public internet, and most of you took steps to secure your own. Keep it going. Make certain the apps you are developing and deploying are as safe as you can make them. And keep kicking rear—you are the rock that’s keeping the suddenly dispersed company clipping along.
