This course covers the skills measured in the Security Assessment and Testing domain of the Certified Information Systems Security Professional (CISSP) examination. This domain represents 12% of the total exam and is organized around two core disciplines: developing a security testing strategy, and conducting tests and remediation. While it is presented as a standalone domain, security assessment and testing is in practice woven through every other domain of the CISSP body of knowledge — it is the mechanism by which an organization proves (or disproves) that the controls it designed and implemented are actually working.
Table of Contents
- Module 1: Designing a Security Testing Strategy
- The Purpose and Principles of Security Testing
- Assessor Qualifications and the Timing of Assessments
- Enabling Assessment: Verification, Validation, and the IDOT Approach
- Planning the Assessment: Scope, Compliance, and Third-Party Testing
- Building a Test Strategy: Plans, Scripts, and Test Data
- Conducting Security Controls Testing
- Security Control Testing Techniques: Log Reviews, Synthetic Transactions, and Code Testing
- Vulnerability Assessments
- Penetration Testing Fundamentals
- Conducting a Penetration Test
- Module 2: Conducting Tests, Remediation, and Security Audits
- Testing Identity, Change Control, and Physical Security
- Testing Incident Response, Business Continuity, and Security Awareness
- Assessment Criteria: Vendor Management and Performance Indicators
- Conducting or Facilitating Security Audits
- The Audit Process: From Evidence to Final Report
- Format and Integrity of the Audit Report
- Self-Assessment Questions
- Summary
Module 1: Designing a Security Testing Strategy
The Purpose and Principles of Security Testing
Throughout the design and implementation of an information security management system, an organization examines risk, chooses safeguards and countermeasures, and selects secure architecture and network solutions. Security assessment and testing is the step that determines whether that effort actually paid off — no control can be trusted until it has been tested.
A useful way to frame the mindset of a tester: “The purpose of testing is not to see if the program works. The purpose of testing is to see if it fails.” Testing must be approached from the perspective of a user or an attacker who might enter data into the wrong field, leave a required field blank, or delete the wrong file — whether intentionally or by accident — and cause the system to fail.
Security assessment and testing breaks down into two broad efforts:
- Designing a security testing strategy — deciding what to test, how often, and by whom.
- Conducting tests and remediating what is found — executing the strategy and fixing identified gaps.
Testing must happen continuously throughout the information systems lifecycle, not only at the end of a project. The earlier a problem is discovered, the cheaper and more effective it is to fix.
flowchart LR
A[Requirements & Design] --> B[Development / Configuration]
B --> C[Ongoing Monitoring]
C --> D[Security Testing & Assessment]
D --> E{Controls Effective?}
E -- Yes --> F[Report Assurance to Management]
E -- No --> G[Remediate / Adjust Controls]
G --> D
F --> C
Security testing and assessment is explicitly encouraged by ISO/IEC 27001 as part of the continuous improvement cycle of an information security management system (ISMS). Every facet of the security program — technical and non-technical — is evaluated for its performance and its ability to achieve its intended purpose.
Testing is not limited to technical controls. It must also evaluate the “soft” or administrative areas:
| Testing Category | Examples |
|---|---|
| Technical | Firewalls, IDS/IPS, encryption, access controls, network configuration |
| Administrative | Policies and procedures, training programs, organizational culture |
| Process | Reporting to management, compliance workflows, escalation procedures |
| People | Staff awareness, adherence to procedures, susceptibility to social engineering |
Assessor Qualifications and the Timing of Assessments
A good security assessor must meet several criteria:
| Requirement | Meaning |
|---|---|
| Independent | Not influenced by the project team; free to give an accurate, thorough assessment |
| Qualified | Understands what is being tested well enough to test it meaningfully |
| Objective | Present to do the job well, not to validate someone else’s opinion |
| Knowledgeable | Understands both the business and the specific controls being tested |
| Diligent | Avoids “checkbox” testing; digs beneath the surface existence of a control |
When should assessment happen? The honest answer is always. Assessment activity should begin long before a system reaches production and should be triggered by:
- Every phase of the systems development lifecycle (SDLC), especially times of change
- Every security incident (each incident can usually be traced back to an inadequate or missing control)
- Reviews of business continuity and disaster recovery (BC/DR) plans
- Reviews of third-party service-level agreements (SLAs), covering maintenance, purchasing, and retirement of products
- Defensible destruction at end-of-life, ensuring equipment and paperwork are properly destroyed to avoid data breaches from discarded assets
Key Points: Assessment and testing of security controls is critical because it is the last chance to identify and resolve problems before a system enters production. Testing should be continuous, integrated with operations — not a single task performed at the edge of implementation.
Enabling Assessment: Verification, Validation, and the IDOT Approach
To properly test and assess controls, the organization must build the capability to monitor systems — this is called enabling assessment. Everything traces back to risk: residual risk (total risk minus the effect of control effectiveness) must be less than or equal to acceptable risk. Testing hunts for the gaps and weaknesses that could be exploited by an attacker — or by an accidental action from an employee.
There are two complementary concepts here:
| Concept | Definition | Example |
|---|---|---|
| Verification | Confirms the control was implemented as designed and operates correctly | The firewall was installed, is properly placed on the network, is correctly configured, and its logs show it allowing good traffic and dropping bad traffic |
| Validation | Confirms the control is actually the right control for the risk it was meant to mitigate | The firewall exists and works, but does it actually reduce the specific risk it was purchased to address? |
flowchart TD
A[Control Exists?] -->|Verification| B[Control Configured Correctly?]
B -->|Verification| C[Control Operates as Designed?]
C -->|Validation| D[Control Actually Mitigates the Targeted Risk?]
D -->|Yes| E[Risk Reduced to Acceptable Level]
D -->|No| F[Re-evaluate Control Selection]
Proper assessment goes three levels deep: existence → operation → desired result. This calls for a healthy dose of professional skepticism — a manager stating “everything is fine” is not sufficient evidence; the assessor must independently validate that statement.
During assessment, evidence is gathered into work papers: logs, test results, documentation reviews, and incident response records. A widely used interview/testing methodology is IDOT:
flowchart LR
I[Interview] --> D[Demonstrate]
D --> O[Observe]
O --> T[Test]
| Step | Description | Limitation |
|---|---|---|
| Interview | Ask staff — not just management — how the process works and what challenges exist | Answers may be self-serving or incomplete |
| Demonstrate | Have staff walk through the actual process | People often behave differently when they know they’re being watched (observer effect) |
| Observe | Watch the process happen naturally | Independence lets assessors notice things insiders no longer see |
| Test | Perform in-depth, hands-on testing | Most reliable, but resource-intensive |
Worked example — physical security at a building entrance: interview the security guard about their procedures, ask them to demonstrate handling a visitor or a forgotten badge, observe whether they actually follow the stated procedure (or have been socially engineered into skipping ID checks), and finally test by sending someone in with an expired or fake badge.
To make this kind of assessment possible, the organization must build in audit hooks (points where information can be captured mid-process), maintain logs, establish a clear reporting/escalation mechanism, and ensure audit staff are trained to correctly interpret what the tools show them — a tool is only as useful as the analyst’s ability to interpret its output.
Key Points: Testing supports governance by giving management insight into the organization’s risk status and demonstrating the ability to meet required levels of compliance.
Planning the Assessment: Scope, Compliance, and Third-Party Testing
Assessments must first confirm compliance with the organization’s own policies, procedures, and baselines — the internally required configurations and methods of operation. Assessments are frequently run against the organization’s own stated practices, not only against outside standards.
| Compliance Type | Examples |
|---|---|
| Internal | Organizational policies, standards, baselines, procedures |
| External / Industry | PCI-DSS, COBIT, ISO/IEC 27001 |
| Legal / Regulatory | HIPAA, NERC, and other jurisdiction-specific laws |
Where should testing occur? Anywhere there is substantial risk or a legal compliance requirement, across the entire test universe:
mindmap
root((Test Universe))
Applications
Web applications
Infrastructure
Operating systems
Networks
Physical Security
Incident Response Plans
BC/DR Plans
People
User training
Administrator competence
Business Procedures
Separation of duties
Change control
Identity and access management
Third-party testing is often used because:
- It provides confidence and assurance to external stakeholders (and is sometimes contractually required).
- It brings in technical skills the organization may not have internally.
- External parties are more independent of the internal organizational structure and can look at things “with new eyes.”
A key contractual consideration for third-party testing is defining escalation: who does the external tester report to if something goes wrong during the test?
Key Points: Audits and assessments support management by providing assurance of compliance and confirming that risk has been managed in a cost-effective way.
Building a Test Strategy: Plans, Scripts, and Test Data
The test strategy is the top-level document that decides what gets tested, driven primarily by legal/regulatory compliance requirements and areas of higher risk (e.g., finance, change control).
flowchart TD
Strategy[Test Strategy] --> Plan1[Test Plan: System A]
Strategy --> Plan2[Test Plan: System B]
Plan1 --> Script1[Test Scripts]
Plan1 --> Schedule1[Test Schedule]
Script1 --> Data1[Test Data]
Script1 --> Execute1[Execute Test]
Execute1 --> Report1[Report Findings]
Report1 --> Refine1[Refine Script & Retest]
| Element | Purpose |
|---|---|
| Test strategy | High-level, multi-year view of what areas to test and when (e.g., “this year vs. next year”) |
| Test schedule | When to test, balancing business convenience against staff availability |
| Test plan | Defines scope, objective, required skills, and duration for a specific system or area |
| Test scripts | The actual step-by-step test procedures and the data needed to execute them |
| Test data | Complete, representative data used to validate the test; must cover both normal and abnormal conditions |
The strategy defines the scope of each test (department, system, geographic location) and the method — testing should be structured, not “chaos,” so nothing important is missed. The depth of testing is often risk-driven and influenced by management: a mature, stable system might get only a high-level review, while a new system or major change warrants an in-depth test. If a superficial test uncovers a serious issue, the team must decide whether to expand the current test’s scope immediately or defer deeper testing to a future cycle — a decision that itself depends on risk.
How often should testing occur? Frequency may be mandated by a standard (e.g., quarterly, monthly, or “once every three years or at major change” for systems authorization), or it may be based on the organization’s own risk assessment.
Test plans define, for each test:
- Objective — what the test is meant to prove (e.g., “the change control system is being followed and manages risk correctly”)
- Required skills — internal staff vs. external experts
- Duration — a finite window so findings arrive in time to act on them
- Test scripts and test data, including a representative statistical sample when the full population (e.g., a million transactions) is too large to test exhaustively — techniques include stratified sampling and value-diversity sampling
- Success and failure criteria
- Scheduling and access — sometimes off-hours, to avoid disrupting production
Test data planning must account for sensitivity: test data may contain personally identifiable information (PII) or protected health information (PHI), so data sanitization/anonymization is often required — while recognizing the residual risk of deanonymization.
| Testing Technique | Purpose |
|---|---|
| Normal-load testing | Confirms correct behavior under everyday transaction volumes |
| Full/maximum-load testing | Confirms correct behavior at the highest expected daily volume |
| Stress testing | Doubles (or otherwise exceeds) expected load to evaluate scalability |
| Regression testing | Re-tests previously fixed issues to ensure a later change did not reintroduce or overwrite the fix |
Good testing principles balance structure (systematic coverage, nothing missed) with creativity (thinking like an attacker: “how would I try to break in?”). Tools are valuable but are only as effective as the expertise of the person using them.
Key Points: Good testing requires good planning, skilled assessors, and accurate reporting.
Conducting Security Controls Testing
Conducting the actual test requires business cooperation: access to people (for interviews), buildings, systems, logs, and documentation, as well as the ability to observe real processes and behaviors. Some testing — such as cloud-based penetration testing — requires special permission from the cloud provider.
For large, multi-location organizations, testing cannot cover every branch office; instead, a representative sample of locations is selected. Some testing can be conducted remotely; other testing (especially physical or cloud-environment testing) requires on-site presence.
Testing is frequently benchmarked against recognized standards:
| Standard / Body | Focus |
|---|---|
| ISO/IEC 27001 | Information security management system requirements |
| Cloud Security Alliance (CSA) — Cloud Controls Matrix (CCM) | Cloud-specific control good practices |
| Uptime Institute | Data center tier/availability standards |
A common informal technique is “rattling doorknobs” — trying things to see what is actually locked versus what merely appears locked. A sound security program is built around understanding the organization’s attack surface: defense is driven by an understanding of how offense (attackers) actually behaves, so testers deliberately adopt an adversarial mindset to find what a real attacker would target first.
Key Points: Many organizations have well-known, well-documented vulnerabilities they are simply unaware of. Testing against established standards and checklists is one of the most reliable ways to surface these before they become breaches.
Security Control Testing Techniques: Log Reviews, Synthetic Transactions, and Code Testing
The CISSP exam objectives for this domain call out a specific catalog of security-control-testing techniques beyond vulnerability assessment and penetration testing. Each targets a different layer of the environment:
mindmap
root((Security Control Testing Techniques))
Log Reviews
Manual or SIEM-assisted review of system, security, and application logs
Synthetic Transactions
Scripted, simulated user actions run on a schedule to verify availability
Code Review and Testing
Static analysis - SAST
Dynamic analysis - DAST
Peer review / walkthroughs
Misuse Case Testing
Deliberately test how the system responds to abuse and abnormal use
Coverage Analysis
Measures how much of the code or functionality was actually exercised by tests
Interface Testing
User interface, network interface, API testing
Breach and Attack Simulation
Continuous, automated adversary-emulation platforms
Compliance Checks
Configuration and control validation against a defined baseline or standard
| Technique | What It Does | Typical Use |
|---|---|---|
| Log reviews | Manual or automated (SIEM-assisted) inspection of system, security, application, and audit logs | Detecting anomalies, confirming controls fired as expected, supporting incident investigations |
| Synthetic transactions / benchmarks | Scripted actions that simulate real user behavior against a live system (e.g., an automated script that logs in and completes a purchase on a website) run at a set frequency | Continuously verifying that a website or service is functioning correctly, independent of real user traffic |
| Code review and testing | Static analysis (reviewing source code without executing it, often called SAST) and dynamic analysis (testing a running application, often called DAST), plus manual peer review | Finding insecure coding patterns, injection flaws, and logic errors before or after deployment |
| Misuse case testing | Deliberately testing what happens when a system is used incorrectly or abusively, not just how it behaves under valid (“use case”) conditions | Ensuring the system fails safely and does not expose data or functionality under abnormal input |
| Coverage analysis | Measures the percentage of code paths, functions, or requirements actually exercised by a test suite | Identifying untested code paths that could hide latent defects |
| Interface testing | Testing the user interface, network interfaces, and application programming interfaces (APIs) | Confirming that data crossing a boundary is validated, authorized, and handled correctly |
| Breach and attack simulation | Automated platforms that continuously emulate known adversary techniques against production or production-like environments | Ongoing validation of detective and preventive controls between formal penetration tests |
| Compliance checks | Direct validation of configuration settings and controls against a defined baseline, standard, or policy | Confirming hardening baselines, patch levels, and control settings remain in the approved state |
Two closely related exam terms are worth memorizing precisely:
- Synthetic transactions — tests that simulate the actions of a user on a website or application; the test runs automatically at a set frequency to confirm that the system is functioning properly.
- Regression testing — testing to ensure that “dead bugs remain dead,” i.e., that previously fixed issues (or previously implemented changes) have not been overwritten or re-introduced by a later change.
Vulnerability Assessments
A vulnerability assessment discovers, identifies, and prioritizes gaps, weaknesses, or areas of noncompliance. It involves analyzing systems and processes, generating reports, and — critically — prioritizing findings, since some results are meaningful and others are simply noise.
Vulnerability assessments can be a routine, scheduled scan (part of everyday operations) or a specialized, targeted effort against one system or area. Routine scans often surface unpatched or misconfigured systems; the goal is not only to fix the immediate issue but to determine why it occurred, so it doesn’t recur.
| Source of Vulnerability Knowledge | What It Provides |
|---|---|
| CVSS (Common Vulnerability Scoring System) | Standardized severity scoring for known vulnerabilities |
| CWE (Common Weakness Enumeration) | A catalog of common software/hardware weakness types |
| OWASP Top 10 | The ten most critical web application security risks |
A practical limitation: different scanning tools, run against the same target, can return different results, and tools must be kept current with the latest vulnerability data. Gathering scan data is not enough without the analytical skill to interpret it — and most tools only assess systems that are online and reachable at scan time.
flowchart TD
A[Gather Information<br/>Scanning & Footprinting] --> B[Identify Known Vulnerabilities/Weaknesses]
B --> C[Generate Report]
C --> D[Filter: Significant vs. Noise]
D --> E[Prioritize Remediation]
Attack vectors to cover: network, database, application, API, hardware, operating system, hypervisor — and always the human layer (users).
Key Points: Vulnerability assessments are valuable tools that help discover what could otherwise become a point of breach. A structured approach — examining web applications, the network, connected endpoints, and users — ensures nothing significant is missed.
Penetration Testing Fundamentals
A penetration test is a natural extension of a vulnerability assessment: where the vulnerability assessment identifies potential weaknesses across the entire attack surface, the pen test is narrower and attempts to exploit specific, often previously identified, vulnerabilities to determine how serious they really are.
flowchart LR
VA[Vulnerability Assessment<br/>Broad, identifies weaknesses] --> PT[Penetration Test<br/>Narrow, exploits weaknesses]
PT --> Q1{Can we get in?}
Q1 -->|Yes| Q2[How much access/impact?]
Q1 -->|No| Q3[Control confirmed resilient]
Q2 --> R[Report Risk of Compromise & Impact]
Q3 --> R
The aim of a pen test is to measure the resilience of the target: can it withstand an attack, log the attempt, and mitigate damage effectively?
Knowledge-level testing methodologies:
| Type | Knowledge Given to Tester | Typical Team | Notes |
|---|---|---|---|
| Full knowledge | Complete internal knowledge of systems and people | Internal “red team” | Most accurate/precise, since the team already understands the environment |
| Partial knowledge | Limited information (e.g., IP address range, network diagrams) | Internal or external | A middle ground; tester must work with what’s given |
| Zero knowledge | No knowledge at all — simulates an external attacker/hacker | Internal or external | Often includes open-source intelligence (OSINT) gathering, e.g., reviewing social media; typically performed after partial-knowledge issues are resolved |
“Hats” used to describe testing roles:
mindmap
root((Hacker "Hats"))
White Hat
Fully authorized, contracted, skilled professionals
Blue Hat
Pre-production QA-style testing before go-live
Purple Hat
Learning-focused, often a personal cyber range
Red Hat
Internal team simulating attackers to stop threat actors
Gray Hat
Unauthorized but discloses findings responsibly - may claim bug bounties
Black Hat
Malicious actor who exploits and causes damage
| Hat | Authorization | Intent |
|---|---|---|
| White | Fully authorized, contracted | Conduct legitimate, scoped penetration tests |
| Blue | Authorized (pre-production) | Quality-assurance style testing before deployment |
| Purple | Typically unaffiliated | Learning and experimentation (e.g., home cyber range) |
| Red | Authorized, internal | Simulate attackers to strengthen defenses |
| Gray | Unauthorized | Discloses findings responsibly rather than exploiting them; may collect bug bounties |
| Black | Unauthorized | Malicious; steals data or damages systems |
Bug bounty programs are a real-world mechanism for channeling gray-hat activity productively — rewards can range from roughly $5,000 up to $100,000 per finding, with some large organizations paying out several million dollars a year in aggregate bounties.
Blind vs. double-blind testing:
sequenceDiagram
participant RT as Red Team
participant SA as System/Network Admins
participant SEC as Security Team
Note over RT,SA: Blind Test
RT->>SA: Conducts test (unannounced)
SA-->>RT: "Was that you pinging our network?"
RT-->>SA: Confirms / rewards attentiveness
Note over RT,SEC: Double-Blind Test
RT->>SEC: Conducts test (neither admins nor security notified)
SEC-->>SEC: Must detect and respond independently
- Blind test — the test team knows they are testing; system/network administrators are not told in advance. If an admin notices and asks, the tester confirms, reinforcing good log-review habits.
- Double-blind test — neither administrators nor the security team are told, so the exercise also validates whether security personnel detect and respond to a live attack according to procedure.
Common pen test targets: networks, applications, websites, physical building security, cloud services, and people (social engineering).
Key Points: Penetration testing validates whether identified vulnerabilities represent a real, exploitable risk and measures the resilience of systems against an actual attack attempt.
Conducting a Penetration Test
flowchart TD
A[1. Management Approval<br/>Scope, methodology, rules of engagement] --> B[2. Independent, Skilled Test Team]
B --> C[3. Execute Test<br/>Network, apps, physical, social engineering]
C --> D[4. Confidential Reporting<br/>Scope, findings, recommendations]
D --> E[5. Management Response<br/>Accept risk OR remediate]
E --> F[6. Retest to Confirm Remediation]
F --> G[7. Update Risk Register]
- Management approval must always precede a penetration test, including a documented scope, methodology, and rules of engagement — certain tools or techniques may be explicitly prohibited because they are too dangerous.
- Independence and skill — the test team must be independent of the network/system owners and able to think creatively, beyond simply running a tool. Social engineering and physical-security-bypass attempts are frequently among the most effective tests.
- Required technical skill set includes network design analysis; techniques to get around firewalls, IDS, and IPS; router/switch administration and its weaknesses; Linux (command line, not just GUI); and both desktop and server variants of Windows — understanding the subtle differences between platforms is essential to finding a way in.
- Danger management — many of the same tools used by real attackers can cause outages or lasting damage (including hardware requiring replacement), which is exactly why management approval, a defined methodology, and contingency plans are required.
- Confidential reporting — reports disclose exploitable vulnerabilities and must be tightly controlled. Reports include the scope of the test, the findings, and prioritized, business-appropriate recommendations (a recommendation that would technically increase security but cripple the business is not the right recommendation).
- Response and remediation — every finding is reported, even if already fixed by the time the report is issued (so that the fix is not later “un-fixed” by mistake). Management may choose to accept the risk or apply remediation; once remediation is applied, the control is retested to confirm the risk was reduced to an acceptable level.
- Risk register — all findings from vulnerability assessments and security tests are logged in the risk register and updated as they are mitigated.
Key Points: Penetration testing provides valuable insight by measuring both the effectiveness of controls and the resilience of systems under an actual attack.
Module 2: Conducting Tests, Remediation, and Security Audits
Testing Identity, Change Control, and Physical Security
Beyond application, server, configuration, and endpoint testing, security assessment must also cover non-technical domains such as social engineering, and must directly assess whether identity and access management (IAM) is functioning:
- Are permissions reviewed on a scheduled basis, enforcing least privilege?
- Is there genuine separation of duties between mutually exclusive tasks, or is it possible for people to collude around the control?
A subtle but important distinction: a compliance test might show 100% compliance because the system enforces a required approval step — but that doesn’t confirm the approver actually reviewed the item rather than rubber-stamping it. Many real-world fraud cases occur precisely because a compliance control existed on paper but was not meaningfully exercised in practice.
flowchart TD
A[IAM Testing] --> B[Least Privilege Reviews]
A --> C[Separation of Duties]
C --> D{Real Review or Rubber-Stamp?}
D -->|Rubber-Stamp| E[Control Fails in Practice]
D -->|Genuine Review| F[Control Effective]
Change control testing confirms that changes go through the full pipeline: proper procedures were followed, testing was performed (not skipped because “it’s a small change”), documentation was kept current, and rollback/backup plans exist in case something goes wrong.
Physical security testing covers electronic door latches, alarms, perimeter defenses (cameras, lighting), and whether security guards are genuinely monitoring effectively. A key behavioral risk to test for is tailgating (also called piggybacking) — someone holding the door open for another person, bypassing individual badge authentication. Testing should also surface simple physical gaps such as doors left unlocked or propped open, so they can be corrected before an attacker finds them first.
Testing Incident Response, Business Continuity, and Security Awareness
Incident response testing confirms the organization meets regulatory requirements for response timing, system uptime, and the protection of employees, customers, and the community. Testing should emphasize events that are likely to happen (even with minimal individual impact), ensuring the organization is proactive about monitoring and alerting before small issues accumulate into large ones. This includes confirming the incident response team is properly trained, equipped, and following detailed, step-by-step checklists (alerting team members, specific response actions, etc.).
Business continuity plan (BCP) testing ensures the plans designed to keep the business functioning during major incidents are realistic, scheduled regularly, and updated with lessons learned from each test — the tests themselves also serve as valuable staff training.
Security awareness testing evaluates the effectiveness of training and awareness programs, starting with attendance (including by management), compliance with standards that mandate an awareness program (e.g., ISO), compliance with privacy laws, and — most importantly — whether the program has a genuine, lasting impact on behavior. Content must be kept current.
mindmap
root((Module 2 Testing Domains))
Identity and Access Management
Least privilege
Separation of duties
Change Control
Procedure adherence
Rollback readiness
Physical Security
Door latches / alarms
Tailgating and piggybacking
Incident Response
Team training
Checklists and escalation
Business Continuity / DR
Realistic, scheduled tests
Lessons learned feed back into plans
Security Awareness
Attendance and compliance
Measurable behavior change
Key Points: Configuration control is essential, and all patches must be tested for security impact before deployment. Vulnerability scanning helps surface software and configuration weaknesses. Incident response plans address issues that are likely to occur with typically minor impact; business continuity planning addresses issues that are unlikely but serious if they occur. Testing and assessment must span the entire security program.
Assessment Criteria: Vendor Management and Performance Indicators
Vendor and third-party assessment begins at the request-for-proposal (RFP) stage: confirm the vendor responded to every compliance requirement, and carry those requirements forward into service-level agreements (SLAs) and maintenance agreements. Contracts must address legal considerations such as data deletion, data restoration/backup, and data retention, and must preserve the organization’s right to audit and enforce contract conditions. Non-disclosure agreements (NDAs) are commonly required to protect sensitive customer data, along with contractual minimum encryption strength requirements.
Measuring performance relies on three related indicator types:
flowchart LR
KPI[Key Performance Indicator<br/>What we MUST do - the requirement]
KRI[Key Risk Indicator<br/>Early warning of an emerging risk]
KGI[Key Goal Indicator<br/>What we WANT to do - the aspirational target]
KGI -.tracks progress toward.-> KPI
KRI -.alerts before breaching.-> KPI
| Indicator | Definition | Worked Example |
|---|---|---|
| KPI (Key Performance Indicator) | A specific measure of performance against a stated standard or requirement | PCI DSS requires all security patches applied within 30 days |
| KRI (Key Risk Indicator) | Tracks trends to flag an emerging risk before a KPI is actually violated | An alert fires whenever patch deployment time exceeds 25 days |
| KGI (Key Goal Indicator) | Tracks progress toward an internally set, more ambitious goal | The organization’s internal goal is to patch within 5 days |
Worked trend example (patch deployment days):
| Month | Days to Patch | Status |
|---|---|---|
| January | 8 | Above 5-day goal, well within 30-day KPI |
| February | 4 | Meeting the 5-day goal |
| … | Rising trend | Concerning trajectory |
| July | 20 | Crosses the 25-day KRI alert threshold |
| September (projected) | 30+ | Risk of violating the 30-day KPI if the trend continues |
xychart-beta
title "Patch Deployment Trend vs. KGI / KRI / KPI Thresholds"
x-axis [Jan, Feb, Mar, Apr, May, Jun, Jul]
y-axis "Days to Patch" 0 --> 35
line [8, 4, 10, 14, 18, 19, 20]
line [5, 5, 5, 5, 5, 5, 5]
line [25, 25, 25, 25, 25, 25, 25]
line [30, 30, 30, 30, 30, 30, 30]
The KRI’s purpose is to give advance warning of an emerging problem, well before the actual KPI (compliance requirement) is breached.
Conducting or Facilitating Security Audits
Audit is a formal process that provides senior management with information about the organization’s current risk profile and assurance that policies and procedures are actually being followed and legally compliant. Because information reaching senior leadership is often heavily filtered by middle management, audit provides a channel that bypasses that filtering. Audits are formally mandated in many organizations (internal, external, or both) and often become legally discoverable documents.
flowchart TD
A[Security Audit] --> B[Internal Audit]
A --> C[External Audit]
A --> D[Regulatory / Third-Party Audit]
B --> B1[Based on annual audit plan<br/>Compared against internal policy/standards<br/>Demonstrates due diligence]
C --> C1[Independent of local management<br/>Validates compliance with accounting rules/good practices<br/>May lack deep business context]
D --> D1[Third-party attestation<br/>e.g., SSAE / SOC, FISMA]
| Audit Type | Comparison Basis | Strength | Limitation |
|---|---|---|---|
| Internal | Internal policies, standards, and practices | Deep knowledge of the business | Less independent |
| External | Accounting rules and industry good practices | More objective and independent; can supplement missing internal skills | Less familiarity with the specific business |
| Regulatory / Third-Party | External compliance standards (e.g., ISO, PCI) | Provides assurance to customers, suppliers, and regulators | Formal, often costly, and narrowly scoped to the standard |
The evolution of attestation standards (relevant to third-party/regulatory audits):
flowchart LR
SAS70[SAS-70<br/>Financial reporting only] --> SSAE16[SSAE-16, 2016<br/>Adds security] --> SSAE18[SSAE-18, 2018] --> SSAE20[SSAE-20, 2020] --> SSAE22[SSAE-22, current<br/>Continuous review common]
- SSAE stands for Statement on Standards for Attestation Engagements, published by the American Institute of Certified Public Accountants (AICPA). It replaced the original SAS-70, which was restricted purely to financial report accuracy. Beginning with SSAE-16, the standard expanded to also cover security, and is updated roughly every two years (16 → 18 → 20 → 22). A comparable European standard is ISAE-3402.
- Reports produced under SSAE are called SOC (Service Organization Control) reports.
| Report Type | Focus | Typical Distribution |
|---|---|---|
| SOC 1 | Financial reporting (equivalent to the old SAS-70) | Restricted |
| SOC 2 | Security-related trust services criteria: confidentiality, integrity, availability, privacy | Restricted (sensitive — can reveal exploitable weaknesses) |
| SOC 3 | High-level summary/opinion, no sensitive detail | Freely distributable (often published publicly, e.g., on a company website) |
Some organizations will share their full SOC 2 report with prospective customers under NDA for transparency, while relying on the SOC 3 as the public-facing summary — the trade-off being that SOC 3 lacks the operational detail needed to actually remediate specific issues.
| Report Duration | Description |
|---|---|
| Type 1 | Point-in-time review — controls were examined and found fine on this date |
| Type 2 | Review conducted over a period (commonly six months); many organizations now perform this on a continuous basis, making it far more valuable than a single snapshot |
The Audit Process: From Evidence to Final Report
Just like security testing, an audit gathers data and work papers, analyzes them, and produces findings, an auditor’s conclusions, and recommendations.
flowchart TD
A[Gather Evidence / Work Papers] --> B[Analyze]
B --> C[Findings - Facts]
C --> D[Auditor Conclusions]
D --> E[Recommendations]
E --> F[Draft Report]
F --> G[Present to Responsible Local Manager]
G --> H[Management Response]
H --> I[Final Report<br/>Includes Management Responses]
I --> J[Distribution per Audit Plan<br/>Board / Senior Management / Local Manager]
- Findings must be strictly factual — what was actually observed.
- The auditor’s conclusion, presented separately, states whether the controls are adequate.
- Recommendations flow from the conclusion — concrete suggestions to close identified gaps or vulnerabilities.
- A draft report goes first to the responsible local manager, who may correct misunderstandings and respond to the recommendations (agreeing, disagreeing, or proposing alternative remediation).
- The final report incorporates management’s responses and is distributed according to the pre-defined audit plan — typically an executive summary to the board of directors, and a fuller summary or complete report to senior management. Because this is a sensitive document, distribution must be strictly controlled.
Format and Integrity of the Audit Report
A well-formed audit report follows a consistent structure:
| Section | Content |
|---|---|
| Executive summary | High-level overview for senior leadership |
| Detailed findings | What was actually observed, described clearly |
| Work papers referenced | Supporting evidence for each finding |
| Recommendations | Realistic, business-appropriate remediation suggestions |
| Methodology | A defined, described approach so the reader understands how conclusions were reached |
Reports must be fair and balanced — the auditor’s interpretation belongs in the conclusion, not the factual findings, and language should reflect appropriate uncertainty (e.g., “this may lead to a breach” rather than an absolute assertion the auditor cannot fully substantiate). All material (significant) findings must be reported — even if management insists an issue was already fixed and asks that it be left out; it should still be documented, if only to help ensure the fix stays in place. Where issues have already been resolved, the report should give appropriate credit.
Key Points: Security control assessment must be objective and thorough across the whole organization, weighted toward areas of higher risk and aligned with the organization’s threat models. Audit is performed by a separate, independent team, but security managers should actively support its work. It is a formal process used to measure compliance with both external regulations and internal procedures — all in service of improving the organization’s overall security posture and resilience.
Self-Assessment Questions
The following questions (adapted from the course’s companion study guide) are representative of the type of scenario-based reasoning tested on the CISSP exam for this domain.
| # | Question | Answer | Rationale |
|---|---|---|---|
| 1 | What is the greatest risk associated with conducting a penetration test? (Disclosure of vulnerabilities / Interruption to business processes / Failure to comply with regulations / Unskilled test personnel) | Interruption to business processes | A pen test may cause systems to crash and impact operations. The test is supposed to find and disclose vulnerabilities, and regulations may actually require pen tests at a set frequency — so those aren’t “risks” of testing itself. Unskilled testers can contribute to business interruption, but the interruption is the core risk. |
| 2 | Who is accountable to remediate any issues found during a penetration test? (The security manager / The testing team / The business owner / The risk owner) | The risk owner | Accountability for remediation follows the risk owner, who may vary depending on where in the organization the risk resides. |
| 3 | What is an important first step when contracting an external team for a penetration test? (Defining the rules of engagement / Demonstration of technical competence / Clarity of reporting / Arranging access) | Defining the rules of engagement | Clear rules of engagement ensure the test is conducted with minimal business impact and within a defined scope. |
| 4 | When should an issue found during a test be reported — even if it has already been fixed? (Always / Never / Only if management agrees / Any time it is significant) | Always | All significant issues must be reported and documented, regardless of whether they were already remediated. |
| 5 | Should a security manager participate in or support an external audit? | The security manager should provide support when appropriate to increase the value of the audit | Support increases the audit’s value; it does not create a separation-of-duties conflict as long as the manager isn’t the one conducting the audit itself. |
| 6 | How can contractual obligations with an outside business partner be measured? (KGI / KRI / CMMI Level 4 / KPI) | KPI | A KPI measures a stated requirement. A KGI is the business’s own goal/objective; a KRI is the alert/alarm threshold indicating a need to act. |
| 7 | Which tool can be used to aggregate and analyze data? (SIEM / Syslog / Audit hooks / Risk response) | Security Information and Event Management (SIEM) system | A SIEM both collects and analyzes data, unlike raw syslog or audit hooks, which only collect. |
| 8 | When should the need to support testing capabilities first be addressed? (Testing phase / Operational phase / Design phase / Deployment phase) | Design phase | Testing capability — audit hooks, logging, and other security/audit features — needs to be designed into the system early in the SDLC. |
| 9 | What feature of some programming languages can prevent various erroneous inputs? (Encoding / Tokenization / Canonicalization / Type safety) | Type safety | Some languages prevent invalid input through type safety. Canonicalization instead refers to the ability to represent the same data in multiple ways (e.g., an IP address vs. its linked domain name). |
| 10 | Why should audit logs be protected? | They may contain sensitive information (such as passwords or PINs), they may be required by law enforcement, and they may overwrite/roll over quickly — so timely preservation matters. | Logs are a critical, sensitive, and often time-limited source of evidence. |
Summary
The Security Assessment and Testing domain exists to answer one question: are the controls the organization designed and implemented actually working, and are they actually mitigating risk? Every technique in this domain — from a routine vulnerability scan to a formal third-party SOC 2 audit — is ultimately a mechanism to detect the gap between the intended security posture and the actual security posture, and to close it.
flowchart TD
A[Design Test Strategy] --> B[Build Test Plans / Scripts / Data]
B --> C[Execute: Vulnerability Assessments, Pen Tests, Log Reviews, Synthetic Transactions, Code Testing, Compliance Checks]
C --> D[Analyze Results / Prioritize]
D --> E[Report to Management]
E --> F{Risk Acceptable?}
F -->|No| G[Remediate]
G --> H[Retest]
H --> D
F -->|Yes| I[Update Risk Register / Continue Monitoring]
C --> J[Formal Security Audits<br/>Internal / External / Third-Party]
J --> E
Exam-Relevant Quick-Reference Table
| Concept | One-Line Definition |
|---|---|
| Verification | Confirms a control was implemented and operates as designed |
| Validation | Confirms the control is actually the right one for the risk it addresses |
| Residual risk | Total risk minus the effect of implemented controls; must be ≤ acceptable risk |
| IDOT | Interview → Demonstrate → Observe → Test |
| Vulnerability assessment | Broad; identifies but does not exploit weaknesses |
| Penetration test | Narrow; attempts to exploit identified weaknesses to prove impact |
| Full / partial / zero knowledge | Levels of information given to a pen test team, from full internal insight to a true external-attacker simulation |
| Blind test | Testers know; system/network admins do not |
| Double-blind test | Neither admins nor security team know |
| Synthetic transaction | Scheduled, scripted simulation of user actions to verify availability |
| Regression testing | Confirms previously fixed issues have not been reintroduced |
| KPI | Measures compliance with a stated requirement |
| KRI | Early-warning trend indicator before a KPI is breached |
| KGI | Tracks progress toward an internal, aspirational goal |
| SOC 1 / 2 / 3 | Financial / detailed security (restricted) / public summary |
| Type 1 / Type 2 report | Point-in-time review / review over a period (commonly six months or continuous) |
| Materiality | Only significant findings must be reported — but all significant findings must be |
CISSP Domain 6 Study Checklist
- Explain the difference between verification and validation, with an example.
- Describe the IDOT interview/testing approach and the weakness of each step.
- Distinguish internal, external, and third-party assessments and when each is appropriate.
- List the components of a test strategy: strategy → plan → schedule → script → data.
- Explain stratified sampling, stress testing, and regression testing.
- Compare vulnerability assessments and penetration tests (scope, output, exploitation).
- Name the CVSS, CWE, and OWASP Top 10 and what each provides.
- Differentiate full-knowledge, partial-knowledge, and zero-knowledge penetration tests.
- Define white, black, gray, blue, purple, and red “hats.”
- Explain blind vs. double-blind testing and what each validates.
- Walk through the penetration test process from management approval to risk-register update.
- Define log reviews, synthetic transactions, code review/testing, misuse case testing, coverage analysis, interface testing, breach and attack simulation, and compliance checks.
- Explain tailgating/piggybacking and why it undermines physical access control.
- Differentiate KPI, KRI, and KGI with a worked numeric example.
- Trace the evolution from SAS-70 to SSAE-16/18/20/22 and the equivalent ISAE-3402.
- Compare SOC 1, SOC 2, and SOC 3 reports, and Type 1 vs. Type 2.
- Describe the audit lifecycle: evidence → findings → conclusions → recommendations → draft → management response → final report → distribution.
- Explain what makes an audit report objective, balanced, and materially complete.
- Explain why security assessment and testing, while its own domain, is really integrated across every other CISSP domain.
Search Terms
security · assessment · testing · cissp · governance · risk · compliance · networking · systems · conducting · assessments · audit · audits · control · penetration · report · strategy · test