Industry analysts say that as much as 75 percent of all attacks are now targeted at the application layer.
For a long-time we have relied on penetration testing to address this threat. There are several ways to conduct penetration testing: black box testing assumes no prior knowledge of the system being tested and is often conducted as an outside hacker, white box provides the tester with complete knowledge of the infrastructure and therefore considers the internal threat or someone with inside knowledge. Grey box testing is variations between the two. Whilst the relative merits of these approaches are debated, there are a number of reasons why penetration testing, as it currently stands, is fundamentally flawed.
- It isn’t deterministic
Despite the increasing sophistication of the tools available, Penetration Testing will still come down to two key factors: the skill of the tester, and the time he has available. If you want to test this theory then the next time you commission a penetration test give the tester more time and he will find more issues! Alternatively, get two different testers to perform a penetration test on the same application and you will find that you get a different list of issues back.
“Penetration testers are not suddenly going to disappear off the face of the earth. Instead, we will see the practice undergo a transformation and be reborn as part of a tightly integrated approach to security.”
- David Harper
- Fortify Software
The reason for this is elementary. A penetration test only scratches the surface and doesn’t make a detailed examination of every entry point and any possible exploits.
- It provides the wrong information
Penetration test reports are despised by the development organization, lets face it no-one likes to have their hard work picked apart, but chiefly they report vulnerabilities based on the URL without any real advice on the underlying cause so it is then left for the developers to ponder the problem, pontificate the possibilities and often through a process of elimination discover how this relates to the code that they have developed. This, combined with the lack of security knowledge within the development organization, make vulnerabilities difficult to fix.
- It occurs at the wrong time
The nature of penetration testing means that it can only occur at the end of the development life-cycle. The problem is that this is really the worst possible time to fix an issue. As an order of magnitude, it is cheaper and quicker to fix an issue if it is discovered during development. Indeed frequently what happens is that the time to fix any vulnerability discovered is so great that the business will release the application into production with known security vulnerabilities and expose itself to the associated risk or worse, issue it with an ill-devised ‘patch’ that may actually introduce more problems than it fixes .
More than ever before, people understand the software security challenge, and penetration testing deserves credit for helping spread the word. But knowing a security problem exists is not the same as knowing how to fix it.
A better way
Organisations are starting to realise the error of their ways and are allocating larger budgets to get code right in the first place than proving it is wrong. They have realised the solution is to embed security activities through the software development life-cycle. At the requirements phase, security requirements need to be specified in the same way as other business targets. During the design phase, the potential threats an application is under need to be analyzed and the architecture needs to include compensating controls to mitigate those threats. As the code is developed it needs to be checked for common coding errors that lead to attacks like SQL Injection and Cross-site Scripting attacks. During testing the security controls need to be fully tested and, yes, you still need to perform penetration testing but now it’s role is a final QA check not as the primary means of defence.
These security activities can’t be left to an individual project team to define. Organizations need to embrace the culture of developing software securely. Typically this involves establishing a software security assurance (SSA) program that is responsible for ensuring all software is developed to an appropriate security standard and also provides resources to assist the development teams to meet this challenge.
- A given is that the organization needs to create a holistic program that fits its requirements as a generic approach is not likely to succeed – this is one area where one size most definitely does not fit all. Every organization has its own unique culture, technologies, and internal processes, and all of these determine the direction such a program must take.
- Then there are the people within the organisation – when securing the applications an organization uses is a key strategic priority, with buy-in from senior management, staff understand that this is not just a passing fad but something that is truly a major directive for the organization that will have tangible business benefits. It is important that the processes defined are not only effective but also efficient, so don’t add significant overhead to the development teams, budgets, and timelines.
- While tools and technology play a critical role in the success of an SSA program, they are by no means the only cog in this wheel – software security practitioners have a variety of tools available, ranging from static and dynamic analysis tools to binary analysis and fuzzing. That said, it is important not to ignore supporting risk management and governance tools, that ensure continuous learning across the organization, for instance, when new vulnerability types are discovered. In a large and diverse organization, with both internally and externally developed applications, when information about vulnerability categories and possible mitigation is shared across the board it can avoid the same vulnerability showing up elsewhere a few months later.
But where do you start to set-up an SSA program? What exactly are the appropriate security activities for your organization? What order should you implement these activities?
This may all sound like a lot of hard work, that’s aside from the problem of managing such a program, but there is help and advice, you just have to look and ask for it, and the rewards will speak for themselves. The Software Assurance Maturity Model (SAMM) is an open framework to help organisations formulate and implement a strategy for software security assurance that is tailored to the specific risks facing the organisation. It was defined with flexibility in mind so that it can be utilized by small, medium, and large organizations using any style of development As an open project, SAMM content will always remain vendor-neutral and freely available for all to use. Visit www.opensamm.org for more information.
Penetration testers are not suddenly going to disappear off the face of the earth. Instead, we will see the practice undergo a transformation and be reborn as part of a tightly integrated approach to security. Penetration testing as a stand alone solution is dead, long live penetration testing.
