.png)
It can be confusing to select the right type of penetration testing for your organization or
business when the need arises. Like Information Technology, Cybersecurity is chock full of acronyms and specialized dialects making it difficult to understand just what one is getting if one isn’t familiar with the vernacular. So, let’s get over that first hurdle together before we delve into why your organization can benefit from continuous penetration testing (CoPT).
A pen test by any other name…
Penetration testing is called by many names in the industry. You may hear it used interchangeably or in the same sentence as the following:
ethical hacking,
red teaming,
adversary emulation or simulation,
hacking,
security assessment,
vulnerability scanning,
vulnerability assessment,
and lastly vulnerability or security testing.
Without splitting hairs, the truth is, each of the preceding is slightly nuanced. While these terms are interrelated, they are not accurate in describing what one gets when purchasing a penetration test.
Now that we know what a penetration test isn’t by name, what can we expect to receive when we purchase a penetration test?
A penetration test will be divided into three phases, much like a story, it will have a beginning (this is the Assessment phase), a middle (this is the Reporting phase), and an end (this is the Remediation and follow-up phase). Reputable penetration tests will have administrative (think project management) and technical components (think pen testers testing). For this reason, a lot of automated tools and scans are passed off as penetration tests but do not contain any of that project management or post-technical analysis that helps ensure appropriate scope, time management, course adjustments, results validation, and so on, so they fail to live up to the quality standard and deliver subpar results, which in turn bogs down value.
As if that wasn’t enough to know already, penetration testing also comes in three flavors, or box colors to be exact: black, grey, and white.
Black box testing is probably what comes to mind first for everyone as this is the point where most penetration tests start – here’s a little bit of information about us (e.g., company name, IP address range) now go see what vulnerabilities you can find and exploit to get into our assets (e.g., applications, systems, networks, etc.)
Grey box testing where some information about the company is already known to the penetration tester. This is common with repeat customers as well as those who may share in advance their network configurations, applications programming interfaces, platforms, or other information germane to the penetration test that helps the penetration tester concentrate further.
White box testing is the last “bucket” or flavor of testing where the penetration tester is well-read into the security and configuration of the environment in which he/she/they are testing. In white box engagements, source code is often available for perusal and most information is directly accessible to the penetration tester.
Cost and time investments often increase when moving from black to grey to white box testing as more and more resources and time are needed to provide thorough coverage, often at great expense to the company. For this reason, most grey and all white box testing is considered too specialized for traditional penetration testing engagements and is out-of-scope, or more succinctly too pricey for all but enterprise-class businesses that can afford the time and investment. So where does that leave the rest of us?
Conventional pen testing and its challenges
Conventional penetration testing is still the predominant resource for most organizations looking to assess their security posture. But what do we mean when we say conventional? How is that defined? Conventional penetration testing generally follows this type of engagement:
It’s black box.
It’s unauthenticated.
It’s scheduled to last for one (1) week.
It’s a point-in-time test. That is, what’s up and available is what’s tested.
It’s generally targeted against a selected network, applications, systems, or mobile devices, though it can combine several of these – usually with more time assigned for the test.
Penetration testing provides organizations that employ it with vulnerability results for the testing target (e.g., network, application, systems, mobile devices, etc.) that are broad-based in their identification. Penetration tests don’t traditionally focus on a single vulnerability type or attack avenue but are instead focused on identifying all vulnerabilities present that can be identified within the testing timeframe specified. Unlike vulnerability assessments or scans, traditional penetration tests will also attempt to exploit those identified vulnerabilities and penetrate the environment to see to what lengths unauthorized access can be gained.
Conventional penetration testing is excellent at helping to identify technical vulnerabilities that can be exploited within an environment at a given point in time, but penetration tests are not without their own challenges and omissions.
Conventional pen testing challenges
There are environmental factors that contribute to the overall efficacy of a test and contribute to its overall success including:
differences present in the skills and understanding/knowledge levels of the individual penetration testers performing the engagement.
the number of systems or applications actively up and running at the time of testing,
whether the right environment was prepared and made available for testing that matches with what is planned for deployment in production,
if all the accounts were turned over and shared for testing,
if the smoke and QA (quality assurance) tests were passed prior to the penetration test beginning, and so on.
Conventional penetration can leave the report recipient with a few questions:
how accurate is a point-in-time test an hour after it concludes, a day, a week, six weeks, four months, or a year out from its occurrence?
What happens in that space in between?
Do more systems, applications, devices, or users come online?
If so, how does that affect the environment overall?
What happens with newly detected vulnerabilities (e.g., zero-days)?
How are these integrated into the program, if at all?
What does a penetration test in April do for security in August?
These and many other questions permeate the inquisitive business mind, especially for any of those of us who are responsible for budgets and sound positions on return on investment (ROI).
These and many other questions permeate the inquisitive business mind, especially for any of those of us who are responsible for budgets and sound positions on return on investment (ROI).
Enter continuous penetration testing (CoPT)
Like any good book, there is always a sequel or another tale to be told. In the case of conventional penetration testing, the next answer to the days-old question of what happens in the span of those 93 days above is continuous penetration testing.
We know what you’re thinking - is continuous penetration testing just another buzzword and contrived acronym for rehashed or resold conventional penetration testing, and is it truly continuous? The answer to both questions is a resounding “no”.
While it’s true continuous penetration testing follows a lot of the same methodology and uses a lot of the same tools and skills as conventional penetration testing, it has one emphatic difference – it attempts to be sentient where the environment is concerned. No, this isn’t another A.I. article or push for machine learning, rather, when we say sentient, continuous penetration testing attempts to provide an organization with a more constant view into its vulnerability posture and its exposures over time.
Conventional penetration testing is limited to being a point-in-time or a one-off occurrence and is therefore limited in its value over time. Continuous penetration testing on the other hand has the added benefit of testing the same environment over a given or specified interval – be it daily, weekly, monthly, semi-monthly, quarterly, or even yearly – thereby affording a more valuable assessment of the state of vulnerability within the targeted environment.
Besides identifying vulnerabilities at near real-time detection intervals, continuous penetration testing provides value at an analytical level far above what can occur with conventional penetration testing. CoPT provides the penetration tester and the organization employing it with a deeper and longer level of introspection and observance, making it possible to discern patterns and extrapolate vulnerability behavior from amongst the myriad of network traffic and metadata, allowing for the formulation of systemic and endemic issues within the targeted environment.
Sounds great, where do I sign up?
Not so fast. While continuous penetration testing puts more meat and potatoes on the plate than its predecessor, it’s not without the occasional brussel sprout as well. So, unless you’re an enthusiastic fan of brussel sprouts, you’ll want to be armed with what continuous penetration testing is not.
CoPT is not to be confused with PTaaS or penetration testing as a service. While CoPT is usually a PTaaS offering, the reverse is not always true. CoPT can exist in a non-as-a-service model as well.
PTaaS offerings generally swing to simply include the penetration testing as a service or the penetration testing and the infrastructure as a service pieces (if the infrastructure is hosted on infrastructure within the same cloud provider where the PTaaS resides.) It is rare to find a model where a third-party or other as-a-Service provider will cover all facets of the stack managed on the left. This is where CoPT shines and differentiates itself as it can be achieved both with and without dedicated service providers, or any vendors in this case. However, this model is often much more expensive as well as it equates to having a dedicated team of penetration testers available for all networks, applications, systems, and mobile devices actively being utilized or developed within the organization. Again, unless yours is an enterprise-class organization, it’s very unlikely these resources exist at all, let alone at the capacity needed to support a CI/CD (continuous integration (improvement) /continuous delivery (deployment)) model.
A continuous pen test by any other name…
More confusing still is the industry’s prolonged persistence in naming the same thing repeatedly using different labels and expecting their distinct nuances to be overlooked or ignored by those looking to employ or purchase them. Confused, yes, we are too. Welcome to the cybersecurity industry – complex solutions to simple problems. So it is with continuous penetration testing. Another term used interchangeably for CoPT is “on-demand” penetration testing.
This term is absolutely infuriating to those of us practitioners in the field as all penetration testing is “on demand”. Yet on-demand penetration testing has somehow morphed into another term to represent continuous pen testing. Reader beware, on-demand penetration testing is not the same thing, so please ask any vendors or providers to explain what they mean when they offer on-demand penetration testing.
In most cases, vendors and service providers are talking about their capabilities to run a penetration test at any hour/time of the day, or “as needed”. This is true of all vendors or providers offering penetration testing, and it’s the same for Emagined Security. Anyone with a penetration testing service will be happy to sell you a penetration test at any point in time that will run at any point in time – some just might charge a premium for off-hours and holidays. But it’s what happens after, before, and in between that needs a little more definition before you liken on-demand pen testing to a continuous penetration test.
CoPT is not on-demand penetration testing. Continuous penetration testing is carefully planned and thought out to ensure an organization receives the maximum value from the series of penetration tests that occur within the selected cycle. CoPT factors such things as location(s) of assets, staffing intervals, environments (e.g., testing, development, staging, quality assurance, production, etc.), asset diversity, time of day as well as day of week, and more. Each of these is factored into the project management equation and calculations performed to ensure that CoPT is right for an organization and employs the proper settings, methodology, and approach to yielding the best results possible. What works for a medium-sized business looks much different from what a small business might employ. Two medium-sized businesses in different industries/verticals may even apply slight tweaks or differences to account for the various threat landscapes each faces. Similarly, a micro-business will logically not have the same budget or breadth and depth of penetration testing scope as an enterprise-class business.
Choosing the CoPT model that’s best for you
Selecting the appropriate continuous penetration testing model for your organization and your needs can be a daunting task, especially considering all the confusing terminology and mixed services. Hopefully, we’ve cleared up some of what pen testing and CoPT aren’t earlier on in this article, but you still may be wondering where to start and how to go about selecting the right vendor or service provider to aid you. Let’s start with the easier of the two – start selecting CoPT based on the maturity of your organization.
Maturity
If you haven’t seen the maturity model below before, take a few moments to familiarize yourself with it, and then try to pinpoint where your organization falls on the scale. In most cases, smaller organizations will be more to the middle or left of the table, and larger organizations more to the middle or right. But this is not always the case, so again, bear in mind that this is more art (e.g., qualitative) than science (e.g., quantitative).
Regardless of where your organization falls on the scale, CoPT is a security component that can benefit your organization while increasing overall security posture and reducing the number of vulnerabilities and exposures present within the environment.
Use the table below to help you ascertain a likely starting point for your organization. Again, mileage may vary based on specific organizational needs and industry/ies of operation, so please factor this accordingly.
| Micro Business | Small Business | Medium Business | Large Business | Enterprise Business |
CoPT Frequency | Semi-Annually | Bi-Monthly | Monthly | Quarterly | Weekly |
CoPT Justification | Micro businesses are generally focused on keeping the lights on and keeping revenue/cash flowing. CoPT three to six times a year can help them stay abreast of vulnerabilities without impacting business and causing over-expenditure. This presumes the business assets (e.g., systems, applications, etc.) are dynamic but aren’t drastically changing in count month over month, and there is a single “production” network. | Smaller businesses are in a growth model and are busy standing up processes and procedures; to this end they often need more fluidity with penetration testing. CoPT six to eight times a year can help SBs recognize and mitigate threats. This presumes business assets are fairly dynamic on both the server and workstation side, but counts are reasonably managed, and there is a single “production” network and perhaps a handful of systems/servers in a test or other network. | Medium businesses are focused on maintaining the status quo and enhancing additional processes and procedures in addition to people and technology. Monthly CoPT can help medium businesses keep their focus more evenly distributed as they begin to move from reactive to proactive. Medium businesses tend to be the most chaotic environments as they juggle to balance between managing and maintaining an inclusive business feel while advancing the business to the next level. This presumes a very fluid environment where systems and server counts change frequently, and networks may be stood up or retired based on need/trial and error. | Large businesses are focused on achieving goals and meeting objectives and tend to have more mature processes and procedures. Quarterly CoPT can help large businesses achieve most vulnerability management goals as they operate in a proactive state. This presumes business assets are fairly static on both the server and workstation counts, and there are established and well-documented environments, including a production network and a “test/ development” network, in addition to staging or pre-production. | Enterprise businesses are focused on optimizing and increasing their levels of efficiency while maintaining productivity. Weekly CoPT can help enterprise businesses focus on the micro-level they prefer by providing as near constant and real-time vulnerability metrics as are possible for their environments. This presumes business assets are slightly static on the server side, and a little more dynamic on the workstation count. Network environments here are well-established and well-documented and do not change. Lab environments may be stood up on occasion but follow regimented change control and vulnerability management practices including penetration testing and vulnerability scanning prior to going live or environment “stand up”. |
ss
Choosing the right CoPT vendor
Now that you’ve determined continuous penetration testing is for you, how do you best go about selecting a reputable vendor?
Consider all the nuances we mentioned above and arm yourself with a checklist of questions you can ask each vendor you engage.
If they struggle to tell you the difference between a penetration test and a vulnerability assessment, you can more than likely be assured they haven’t operated in this space for very long or are passing vulnerability assessments off as penetration tests.
Similarly, if they are tossing out buzzwords like PTaaS and On-Demand testing as the latest and best thing since sliced bread, ask them to define succinctly what it is those services entail.
Are you receiving the same level of deeper inspection and analytics that you would come to expect from a provider that has extended visibility into your environment and one which can extrapolate patterns and behavioral trends from routine traffic or network “noise”?
Need some expert assistance?
Emagined Security will be glad to assist you without any high-pressure sales. Simply let us know what your business needs are concerning CoPT and we’ll be glad to chart out a roadmap or offer a solutions path for you.
Do you just want to bounce some additional ideas around for a second opinion, or do you have basic questions you’d like to ask about continuous penetration testing? We’d be glad to answer those too. Please contact us at the button below and let us know the best way to reach you and how you’d like us to respond – verbally, quote, service literature, email, etc.