As a cybersecurity leader, you’re responsible for a ton of moving parts in an evolving ecosystem. The operation and growth of your business is on the line.  

With cyber threats continuing to advance, it has become a common phrase in the industry to say, “It’s not if you get hacked, it’s when.”  

While this adage sounds dramatic, it is becoming increasingly common to see stories about corporate data breaches, ransomware attacks, and critical infrastructure incidents. All of these illustrate the point: you must find the weaknesses in your layered defense strategy before your adversaries do.

Penetration testing is one way to do just that.

Penetration testing (“pen testing”) is a practice that simulates threat actors targeting your environment. This practical approach to security testing combines automated and manual techniques to determine and demonstrate the impact of any potential gaps in your security strategy.  

While vulnerability scanning programs and routine risk assessments are examples of good cyber hygiene, pen testing can unveil weaknesses that aren’t normally picked up through proactive programs. A good pen tester will look holistically at your environment to exploit weaknesses between your security controls. These are the exact types of weaknesses a real-world attacker often relies on.

With that said, not all penetration tests (or testers) are created equal. In this article, we’ll explore some subjects and key points to consider that will help you to discern between highly effective pen tests and not-so-effective pen tests.

Why are pen tests important and when should I get one?

A good pen test relies on assessors with strong foundational knowledge in various technical competencies and experience across a wide array of systems, technologies, industries, and tools. Often, these testers may have previous experience in network/system administration, programming, and development, and/or security operations, prior to moving into the offensive security field. Knowing what constitutes “good” system architecture means understanding what makes an environment weak, and how to exploit it.

A successful pen test should result in a detailed report that outlines the weaknesses and vulnerabilities discovered in your environment and the "kill chains" used to link them. These kill chains demonstrate the stages of an attack from start to finish, and the goal will ultimately depend on the attackers' motivations, whether that be encrypting and/or selling your data for profit, stealing your trade secrets, or bringing awareness to a cause. A well-described kill chain will detail the findings noted at each link of the chain. More importantly, each of these findings will include thorough recommendations on what your next steps should be to increase the overall security posture of your infrastructure, policies, and systems.

Depending on the maturity of your security program, and the nature of the specific test, these reports can include a wealth of information to help you understand where and how to focus remediation efforts. For organizations undergoing their first pen test, the data contained in these reports can sometimes be overwhelming. A good testing partner will walk you through the results at your level and can help tailor recommendations to the specific capabilities and limitations of your organization.

Finally, because pen tests are point-in-time assessments, and conditions within your environment can change rapidly, even the best pen test can only ever offer a snapshot of your current security posture. Rather than considering these projects to be one-and-done engagements, they should instead be integrated into your overall security program. It’s best practice that this testing is performed at least annually, and after any significant environmental changes.

‍What separates a great vendor from a not-so-great vendor?

To achieve unbiased results from a pen test, it’s recommended that you work with a third party to conduct your security testing. Here are a few things to consider when evaluating potential pen test vendors:

1. There is an assumption that the best pen tests are associated with organizations that specialize in “red team” style work, but that may not be the best fit for your organization. While strictly offensive testers undoubtedly have extensive knowledge of their craft, the potential exists to come up short on practical experience with “blue team” (defensive) procedures and current security best practice. This limits their actions and the recommendations you’ll see in your final report make the pen test, generally, less useful than what it could be if conducted by a more experienced team.  

2. When it comes to test results and building the final report, you want to seek an organization that will align recommendations with your unique cybersecurity program maturity and suggest remediations that are realistic for your environment. This partnership begins in the sales process, where the vendor will work to understand not only your desired outcome for the project, but also your wider-ranging roadmap. Ultimately, how you choose to address any identified vulnerabilities is up to you, but finding a vendor who is interested in seeing you succeed with actionable advice, rather than selling you a report to file in a drawer as a compliance checkbox, is going to get you closer to a secure environment.

3. Many vendors, and an increasing number of start-ups, choose to sell vulnerability scans under the guise of “automated penetration testing,” or similar verbiage, often including buzz-terms like “AI.” Occasionally, these types of tests will have an experienced tester overseeing them to make sure that the results make sense in the context of the engagement. However, you should be cautious as many tests are automated processes that combine some off-the-shelf tools to improve the look and feel of their final deliverable. These types of tests miss out on some of the most important aspects of great pen testers – understanding, creativity, cunning, persistence and focus. Your attackers will certainly possess these qualities, so why shouldn’t your testers?

‍What makes a good tester?

Like the last section, let’s explore some of the aspects that separate great testers from the rest of their peers.

1. Cybersecurity is a broad domain and ties into so many industries, frameworks, platforms, and services. Ultimately, no one person can know everything, and that truism applies to pen testers as much as it does anyone else. An effective balance between depth and breadth of competencies means that while a tester may not know as much as they need to about a specific target they come across, a good tester will know how and where to learn about it. A great tester will do not only that, but also work to understand any pitfalls that come with that territory.  

2. While a good penetration tester is always learning, that process should always complement a solid knowledge base. For this, some vendors, Lyrical included, require that all their testers hold the Offensive Security Certified Professional (OSCP) certification, which ensures a baseline of competency in both evaluating systems for security weaknesses and effectively communicating those weaknesses in an approachable, understandable manner. While it certainly isn't a rule that “great testers must have a certification,” if you aren't sure what else to look for or are unsure of a prospective partner, don't be afraid to inquire about their testers' credentials and do your own research to determine whether they meet your expectations.  

3. Whether originating from the SOC, the DevOps team, the service desk, or another relevant position, many of the best penetration testers will understand your needs because they've worked from the perspective of the client (I.e., your perspective). They carry the experience - successes, failures, lessons learned - from past occupations with them into their latest trade. Those successes? They want to share those with you. Those failures? They weren't the only ones who made (or will make) them. Those lessons learned? They know others haven't learned them yet. But if the tester gets their way, they can share some of those lessons with your team as well - without the risk of them learning the hard way. When in doubt during the vendor selection process, don’t be afraid to inquire into the background and experience of the vendor’s testers. Wide expertise when paired with a specific job scope centered on offensive security is typically a winning combination.

Where pen tests are going: Operationalization and maximizing  impact.

A penetration test can be viewed as a one-time snapshot of your system, used to analyze and understand your organization’s overall security posture at a specific point-in-time. However, the industry’s best leaders are treating these exercises like an operationalized practice, integrating them into their security framework and proactive risk management policies on a consistent and repeated basis.  

Their outputs and recommendations should supplement your other security activities, helping to measure the real-world effectiveness of your controls and guiding the next steps of your security journey.  

Interested in learning more, one on one, with a pen test professional? We're always happy to chat.