The Top 4 Penetration Testing Methodologies
Penetration testing, also known as ethical hacking, is the practice of testing a computer system, network or web application to find security vulnerabilities that an attacker could exploit. Pen testing can be performed manually or using automated tools and follows a defined methodology.
There are several leading pen testing methodologies, each with their own approach, scope and areas of focus. In this comprehensive guide, we will explore the top 4 pen testing methodologies used by security professionals and organizations. Additionally, you can see how our own methodology, the Emagined Pentest Pathway, stacks up.
Open Source Security Testing Methodology Manual (OSSTMM)
The Open Source Security Testing Methodology Manual (OSSTMM) is a peer-reviewed methodology for performing penetration tests, security tests and metrics. Created by the Institute for Security and Open Methodologies (ISECOM), the OSSTMM focuses on testing the operational security of systems and applications from an attacker's perspective.
Some key aspects of the OSSTMM include:
Operational focus: Goes beyond just identifying technical vulnerabilities by also testing operational processes, physical security, human elements, wireless security, telecommunications, etc. Provides a holistic view of an organization's security posture.
Channel testing: Analyzing the communication channels into and out of an organization, such as Bluetooth, WiFi, telephone, VoIP, SMS, email, web, etc.
Metrics and measurements: The OSSTMM introduced the idea of using scientific measurements and metrics as part of the testing process. This enables quantitative analysis rather than just pass/fail assessment.
Trust analysis: Evaluation of how much the penetration test target can be trusted to maintain its security properties based on operational controls.
Attack surface: Identification of the different points where an attacker can try to enter data or extract data from a system.
The OSSTMM has evolved over the years and the latest version is OSSTMM 3.0 released in 2010. While initially focused on network security, the methodology has expanded to cover more aspects of operational security, human factors, wireless, telecommunications, mobile security, cloud, and IoT.
NIST Special Publication 800-115
The National Institute of Standards and Technology (NIST) is a government agency that promotes standards in many areas including computer security. NIST Special Publication 800-115 was written to provide technical guidelines for conducting penetration testing and vulnerability analysis.
Some highlights of NIST 800-115 include:
Planning: Provides guidance on planning activities like defining goals, scoping rules of engagement, identifying team roles & responsibilities, and developing penetration test plans.
Discovery: Techniques for information gathering, port/service identification, vulnerability detection, network sniffing, exploiting default credentials.
Attack: Methods for gaining access, escalating privileges, exploiting vulnerabilities, denial of service attacks, and pivoting through the network.
Reporting: Outlines the key elements that should be included in a penetration test report like findings, diagnoses, impact, and corrective actions.
Skills assessment: Evaluation of the testing team's capabilities across areas like computer networking, Windows/Linux, web apps, wireless, and telecom.
Legal considerations: Discussion of legal issues and restrictions that may apply to penetration testing engagements.
While OSSTMM takes a broad operational view, NIST 800-115 focuses purely on the technical aspects of executing a penetration test and vulnerability assessment. It provides guidelines applicable to IT professionals and pentesters.
Penetration Testing Execution Standard (PTES)
The Penetration Testing Execution Standard (PTES) is a framework designed to serve as a standard for performing penetration testing. It was developed by a group of security experts to provide a repeatable and consistent methodology for testing.
The key elements of PTES include:
Pre-engagement: Establishing rules of engagement, testing scope, communication mechanisms, and legal approval.
Intelligence gathering: Identifying the target organization's online presence, domain names, IP blocks, employee names/emails, and technologies used.
Threat modeling: Creating models describing how attackers could penetrate the system and cause damage. Used to guide and focus the testing.
Vulnerability analysis: Discovering and analyzing technical vulnerabilities like OS, network, and application weaknesses. Assessing vulnerability severity.
Exploitation: Attempting to gain access to systems through penetration techniques like password cracking, social engineering, and denial of service attacks.
Post exploitation: Extracting data from compromised systems, maintaining access, covering tracks, pivoting to other systems.
Reporting: Documenting discoveries, vulnerabilities, exploited systems, findings analysis, and recommended mitigation strategies.
The PTES methodology aims to cover the entire end-to-end penetration testing process in an organized and complete manner. It helps enable consistency across engagements.
OWASP Testing Guide
The Open Web Application Security Project (OWASP) is an open-source organization focused on improving web application security. OWASP maintains a comprehensive Testing Guide that outlines a methodology for testing the security of web apps.
Some key aspects of the OWASP Testing Guide:
Web-focused: Covers vulnerabilities and risks specific to web applications such as injection attacks, broken authentication, sensitive data exposure, cross-site scripting (XSS), broken access control, and security misconfigurations.
Technology agnostic: Applicable to web apps built on any technology or framework like Java, .NET, PHP, Node.js, Python, etc. Also covers APIs and web services.
Eight main principles: Define key principles including understanding the full scope of the app, proper staging & test data, appropriate access authorization, and reporting findings responsibly.
Four main phases: Information Gathering, threat assessment, Vulnerability Analysis, and Custom Code Review.
18 Test Types: Provides a methodology for specific test types like identity management, business logic, authentication, session management, input validation, and more.
The OWASP Testing Guide complements technology-focused standards like PTES and NIST by providing extensive guidelines tailored exclusively for penetration testing of web applications and APIs.
Comparative Analysis
While the discussed methodologies share some similarities in core test activities, they each have a different focus and approach. Some key differences:
OSSTMM takes the broadest view with its operational security and human element focus whereas OWASP is specialized for web apps and NIST covers technical testing methodology well. PTES aims to define the full end-to-end process.
OWASP and PTES provide very structured processes while OSSTMM and NIST offer more flexible guidelines.
OSSTMM pioneered test metrics and measurements whereas the others provide more pass/fail testing methodology.
NIST focuses in depth on reporting and legal aspects while the others cover reporting more generally.
PTES was designed by penetration testers for penetration testers while NIST and OSSTMM are more general security standards.
OWASP is highly web application focused compared to the others which cover more technology areas.
Choosing a Penetration Testing Methodology
With several standards and methodologies to choose from, here are some criteria to determine which approach may be most suitable:
Type of System: Web app vs network vs cloud vs operational security
Industry: Some methodologies like OWASP are tailored for specific industries
Scope: Size and complexity of the engagement
Compliance requirements: Regulations that must be satisfied
Team skills: Adopt methodology that aligns with tester capabilities
Required deliverables: Reports, findings, and metrics needed
Budget and timeline: Some methods require more time and cost
One methodology may not perfectly fit every scenario. Teams often combine elements from different methodologies to achieve test objectives within project constraints.
OSSTMM, NIST 800-115, PTES, and OWASP are the most widely adopted penetration testing methodologies.
Each standard has evolved over time with different areas of focus like technical testing, web apps, operational security, compliance, and end-to-end processes.
When selecting a methodology, consider the type of system, scope, skills, compliance needs, and more.
In practice, pentesters often blend aspects of different methodologies together.
Well-defined methodologies help improve the completeness, depth, and consistency of penetration testing engagements.
Keeping up to date with standards is critical as attack surfaces expand into new domains with cloud, mobile, IoT, and smart infrastructure.
With threats constantly evolving, following a structured methodology helps penetration testers systematically test for risks, provide meaningful insights, and deliver higher value to clients looking to enhance their cybersecurity posture.
Conducting Penetration Tests Internally vs Externally
Organizations have the option to conduct testing either using an internal team or by hiring an external firm. There are advantages and disadvantages to each approach.
Internal Penetration Testing
Having an internal pen testing team within the organization can provide these benefits:
Better understanding of systems: With insider knowledge of infrastructure and applications, tests may be more thorough.
Lower costs: No need to pay expensive consulting fees to external firms.
Faster testing: Internal teams can conduct a more agile, iterative testing methodology.
Security skills development: Testing builds skills that enhance in-house capabilities.
Potential downsides of internal testing:
Lack of expertise: Building a highly capable internal team takes time. Existing staff may lack pen testing experience.
Resource constraints: Fighting for priority within the organization as just another IT function.
Biased assessments: Insiders may consciously or unconsciously overlook certain vulnerabilities.
Limited perspectives: Evaluating from the inside without external context and cross-industry insights.
External Penetration Testing
Engaging an external pen testing firm offers these advantages:
Objective assessments: Unbiased tests from an independent third-party perspective.
Specialized expertise: Access to skilled test professionals who keep up with the latest methods and tools.
Best practices: Leverage knowledge and experience from testing many diverse organizations and industries.
Credibility: External testing may hold more weight and credibility with auditors and management.
No operational overhead: Testing conducted fully by the firm with no internal workload.
Flexible scaling: Can ramp testing up or down as needed.
The potential downsides of outsourced testing include:
Higher costs: Paying for consulting services has real costs even if vendors offer competitive rates.
Longer timelines: External penetration testing teams need time to understand the organization's environment and infrastructure.
Potential delays: Testing progress depends on external vendor availability and resource constraints.
Outdated methods: After some time, vendor testing practices may not evolve and improve.
One-off testing: Often not as integrated or iterative as internal testing.
Hybrid Approach
Given the pros and cons of each approach, some organizations take a hybrid approach by:
Having a small internal pen test team that handles regular testing but augments capacity with external vendors for larger, more specialized projects.
Beginning with external testing to establish a baseline, then conducting ongoing internal testing.
Using external penetration testing for compliance mandated audits, while internal team handles day-to-day assessments.
Conducting penetration testing internally first, then validating with an external test to identify any overlooked vulnerabilities.
The optimal approach depends on the organization - a large enterprise with deep security expertise may find an internal team more effective while a smaller company may be better served by fully outsourced penetration testing.
Evolving Penetration Testing Standards and Methods
Penetration testing methodologies continue to evolve as technology and attack methods change. Some trends shaping current standards and the future of testing include:
Cloud and Infrastructure as Code
Cloud adoption has exploded, which requires testing the unique security properties and risks of cloud infrastructure. Pen testing techniques for the cloud continue maturing. Meanwhile, infrastructure as code (IaC) means infra and apps are defined by code, requiring new processes for testing security before deployment.
DevSecOps
Integrating security earlier in DevOps pipelines is DevSecOps. This calls for new approaches to pen testing so assessments can happen continuously, versus traditional endpoint testing. Methodologies must adapt to integrate with rapid deployment.
Automation
Many activities like vulnerability scanning or basic penetration tasks can be automated. This increases efficiency and enables continuous testing. However, automation still lacks the creativity of skilled human testers. Methodologies must determine the right human-machine balance.
False Positives
The volume of inconsequential security alerts creates false positives. Pen testing methodologies should focus on validation, root causes, and exploitable, high-impact risks. Avoiding false positives improves signal value for security teams.
Reporting
Customers now expect testing reports to provide clear remediation guidance, not just data dumps of findings. Methodologies must continue improving reporting effectiveness, clarity, visualizations and integration with IT workflows.
Purple Teaming
Bringing red team pen testers and blue team defenders together is becoming more mainstream and provides a more rigorous test environment. Standards for effective purple teaming continue maturing.
By keeping methodologies aligned with the latest technologies, pen testing provides maximum defensive value against an ever-evolving threat landscape.
Conclusion
Pen testing has come a long way from early days of ethical hacking. Methodologies have evolved from loosely defined processes to robust frameworks like OSSTMM, NIST, PTES and OWASP which enable repeatable, consistent and effective testing for security.
Organizations can choose methodology flavors based on their environment, goals and constraints. Hybrid models combine multiple standards or internal and external testing.
With cyber threats constantly expanding into new domains, following an adaptable methodology ensures pen testers can find the most critical risks and actionable vulnerabilities. Mature standards also allow organizations to baseline security posture, quantify improvements over time and optimize application of pen testing resources.
Staying current with modern methodologies and techniques allows defenders to proactively identify and close security gaps before attackers can exploit them – delivering security, compliance and risk management value to the business.