The meat of the document includes the following main topics: Penetration Testing Components, Tester Qualifications, Methodologies, and Reporting. In addition to these there is a sort of introductory section which does a side-by-side comparison of Vulnerability Scans to Penetration Tests. I’ve read several similar comparisons in the past but the Penetration Testing Guidance document offers a nicely organized comparison table that many organizations may find helpful.
Scope is something that is covered ad nauseam throughout the document but not without just cause. In a PCI-based penetration test it is critical to understand exactly what encompasses the Cardholder Data Environment (CDE). Although this particular document focusses on scope and segmentation as it relates to the CDE, the same rules can be adjusted to most other penetration testing scenarios. This is why it is critical to understand what is important to your client and how they are protecting it. If it isn’t PCI data then perhaps it is Protected Health Information (PHI), or a production application cluster, or secret formulas (i.e. consider a pharmaceutical company). It is up to the penetration tester to properly understand not just the scope in terms of an IP address range but also what information or functionality is considered the “crown jewels” and what sort of boundary, such as network segmentation, exists around it.
Another area of interest in this document is the section on Social Engineering. According to the document, Social Engineering testing isn’t strictly required by PCI DSS. However it does enter a sort of gray area because it is recognized as a viable (and quite common) attack vector… which means it should actually be in scope. My take on it is that the Penetration Testing Guidance document strongly encourages Social Engineering testing even though it isn’t strictly required. It also seems to strongly advise that any company not including Social Engineering testing should document why it is not being included in the penetration test.
The section on Penetration Tester Qualifications is a little thin but there really isn’t much to say here especially since PCI DSS doesn’t require any specific certifications. The document does provide the common sense advice that the quality of the test will be significantly influenced by the level of experience of the tester and testing organization, and that having some certification indicates a common body of knowledge.
The Penetration Testing Methodologies section is perhaps the most useful section to any penetration testing company. It covers pre-engagement, engagement, and post-engagement activities. Included are lists of documentation the client should provide before the test begins, topics to be covered under the Rules of Engagement, and a description of Success Criteria. In addition this section covers a lot of detail on how past threats should be disclosed to the penetration testing team for consideration. This includes everything from previous vulnerability scan results, to last year’s penetration test, to actual threats from the wild. This information seems to be required under PCI but any company with the goal of reducing risk should provide this information to their penetration tester. I found the Penetration Testing Methodologies section very worthwhile. If you don’t read anything else in the document, do read this section.
The final section of the document (not including the use cases in the back) covers reporting guidelines. Included is quite a bit of advice on how to lay out a report, how to cover retesting, and what to do about evidence retention. I found it a little odd that one section of the report that is not mentioned at all is a conclusion, since most penetration testing reports I have seen do seem to include one. Aside from the report advice, the most interesting part of this section is the Report Evaluation Checklist. This guide has provided a checklist for entities hiring penetration testers, to ensure that the report they are receiving includes everything it should. This checklist is a good takeaway in that it has the potential of becoming a de facto report quality standard.
So that’s it. Although I don’t personally agree 100% with every detail, overall I’m impressed with what is covered and find the level of detail just right. I believe this is generally solid guidance for not just PCI-based penetration tests but guidance that can be adapted to virtually any kind of penetration test. The full document is available here:
https://www.pcisecuritystandards.org/documents/Penetration_Testing_Guidance_March_2015.pdf
Jason Gillam is a Senior Security Consultant with Secure Ideas. If you are in need of a penetration test or other security consulting services you can contact him at jgillam@secureideas.com, on Twitter @JGillam, or visit the Secure Ideas – ProfessionallyEvil site for services provided.