DevOps Troubleshooting: Security and DevSecOps

Course: Configuring and Managing Kubernetes Security
Topics: RBAC, Supply Chain Security, Image Scanning, Policy Enforcement, Network Policies, Secrets Management, Defense-in-Depth

Table of Contents

• 1. DevSecOps Overview
  • 1.1 What is DevSecOps?
  • 1.2 Shift-Left Security
  • 1.3 Defense-in-Depth in Kubernetes
• 2. Module 1 – Securing Cluster Access
  • 2.1 Authentication and RBAC
  • 2.2 Service Accounts and Least Privilege
  • 2.3 End-User Authentication and MFA
  • 2.4 Demo: RBAC for Service Accounts
  • 2.5 Demo: MFA and RBAC for Users
• 3. Module 2 – Securing Application Deployment
  • 3.1 Supply Chain Complexity
  • 3.2 Image Scanning with Trivy Operator
  • 3.3 Vulnerability Reports and Policies
  • 3.4 Policy Enforcement with Kyverno
• 4. Module 3 – Securing Applications at Runtime
  • 4.1 Network Policies as Firewalls
  • 4.2 CSI Secrets Driver and Secrets Management
  • 4.3 Complete Security Model
• 5. DevSecOps Diagrams
  • 5.1 DevSecOps Pipeline – Shift-Left Security
  • 5.2 SAST / DAST / SCA Flow
  • 5.3 Threat Modeling
  • 5.4 Zero Trust Architecture
  • 5.5 Defense-in-Depth Kubernetes
• 6. YAML Snippets for Security Scans in CI/CD
  • 6.1 RBAC – Role and RoleBinding
  • 6.2 Trivy Operator – Vulnerability Report Policy
  • 6.3 Kyverno – Cluster Policies
  • 6.4 Network Policy – Default Deny and Allow Rules
  • 6.5 SecretProviderClass – CSI Driver with Azure Key Vault
  • 6.6 Pod Security Context
  • 6.7 GitHub Actions – Complete DevSecOps Pipeline
• 7. Reference Tables
  • 7.1 DevSecOps Security Tools
  • 7.2 OWASP Top 10 – 2021
  • 7.3 DevSecOps Practices by CI/CD Phase
  • 7.4 Kubernetes RBAC Resource Types
  • 7.5 Authentication Methods Comparison
• 8. Best Practices and Summary

1. DevSecOps Overview

1.1 What is DevSecOps?

DevSecOps is an evolution of DevOps that integrates security at every stage of the software development lifecycle (SDLC). Rather than treating security as a distinct phase at the end of the cycle, DevSecOps adopts a “Security as Code” approach where security controls are automated, measurable, and continuous.

Core Principles:

• Shared Responsibility: security is everyone’s responsibility (Dev, Ops, Sec)
• Automation: security scans run automatically in every pipeline
• Continuous Compliance: compliance is verified continuously, not annually
• Fail Fast: vulnerabilities are detected as early as possible in the development cycle
• Feedback Loops: developers receive immediate feedback on security issues

1.2 Shift-Left Security

The Shift-Left concept means moving security activities to the left on the project timeline — that is, as early as possible in the development cycle.

Why Shift-Left?

The cost of fixing a vulnerability grows exponentially the later it is detected in the cycle:

1.3 Defense-in-Depth in Kubernetes

Defense-in-Depth is a multi-layered security strategy where each layer limits potential damage if another layer is compromised. In Kubernetes, this translates to:

1. Cluster Access Layer: RBAC, MFA, certificates, identity providers
2. Deployment Layer: image scanning, admission control, policy enforcement
3. Runtime Layer: network policies, secrets management, pod security contexts
4. Observability Layer: audit logs, runtime security (Falco), metrics and alerts

Key principle: Each layer of defense forces an attacker to work harder, generating more noise and increasing the chances of detection.

2. Module 1 – Securing Cluster Access

2.1 Authentication and RBAC

Securing access to the Kubernetes API is the foundation of Defense-in-Depth. Without rigorous access control, all other security measures can be bypassed.

Kubernetes RBAC Resources

Kubernetes uses Role-Based Access Control (RBAC) to manage permissions. There are four types of RBAC resources:

RBAC Verbs (Actions)

RBAC rules combine verbs (actions) with resources:

verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
resources: ["pods", "services", "secrets", "configmaps", "deployments"]

Built-in ClusterRoles

2.2 Service Accounts and Least Privilege

Common issue: By default, every pod uses the default ServiceAccount of the namespace. If this account has broad permissions, any compromised pod becomes an attack vector for the entire infrastructure.

Real-world case – Wired Brain Coffee:
The team deployed a Kubernetes management application (kube-explorer) without configuring its ServiceAccount. The pod used default, which had a ClusterRoleBinding granting cluster-admin. Result: the application could list and delete pods in all namespaces, including kube-system.

Solution – Least Privilege:

1. Create a dedicated ServiceAccount per application
2. Define a Role with the minimum required permissions
3. Create a RoleBinding associating the ServiceAccount with the Role
4. For pods that do not need to access the Kubernetes API, disable token mounting:

spec:
  automountServiceAccountToken: false

2.3 End-User Authentication and MFA

Kubernetes does not manage users internally — there is no User object to create. Kubernetes delegates authentication to external systems via:

Authentication Methods

Risk of kubeconfig with static credentials:
The ~/.kube/config file contains base64-encoded tokens and certificates (not encrypted). If this file is compromised, the attacker has permanent access until the certificate expires (often years).

Azure Entra ID Integration (AKS)

For AKS clusters, integration with Azure Entra ID enables:

• Mandatory MFA authentication
• Temporary tokens via OAuth flow (kubelogin)
• Disabling local accounts (--disable-local-accounts)
• Enforcement of organizational policies

# Enable Entra ID integration and disable local accounts
az aks update \
  --enable-aad \
  --aad-admin-group-object-ids <GROUP_OBJECT_ID> \
  --disable-local-accounts \
  --resource-group <RG> \
  --name <CLUSTER>

2.4 Demo: RBAC for Service Accounts

Scenario: The kube-explorer application accidentally has cluster-admin via the default ServiceAccount.

Diagnosis:

# Check ClusterRoleBindings for the default ServiceAccount
kubectl get clusterrolebinding -o json | \
  jq '.items[] | select(.subjects[]?.name=="default" and .subjects[]?.kind=="ServiceAccount")'

# Check effective permissions
kubectl auth can-i --list --as=system:serviceaccount:default:default

Fix:

# Remove the excessive binding
kubectl delete clusterrolebinding <binding-name>

# Create a dedicated ServiceAccount with minimum permissions
kubectl create serviceaccount kube-explorer-sa -n default
kubectl apply -f rbac-pod-reader.yaml

2.5 Demo: MFA and RBAC for Users

Before (insecure):

# Retrieving credentials with static embedding
az aks get-credentials --resource-group <RG> --name <CLUSTER>
# → kubeconfig contains a static Bearer token
kubectl config view  # Shows the encoded but unencrypted token

After (secured with Entra + kubelogin):

# Fetch credentials with Entra ID
az aks get-credentials --resource-group <RG> --name <CLUSTER>
kubectl get nodes
# → Triggers an interactive OAuth flow via kubelogin
# → Temporary, revocable token with MFA enforced

3. Module 2 – Securing Application Deployment

3.1 Supply Chain Complexity

Every container image is a complete software supply chain. Example of a Python application:

Base Image (Debian/Alpine)
  └── Python Runtime
        └── Framework (Django/Flask/FastAPI)
              └── Application Dependencies
                    └── Transitive Dependencies (N levels deep)

Challenges:

• Independent schedules: each component follows its own release cycle
• Emerging CVEs: a clean image today may be vulnerable tomorrow
• Transitive dependencies: vulnerabilities can hide N levels deep
• Cascading compatibility: updating a lib may break the framework, which requires a new runtime version

Solution: Continuous Scanning
Point-in-time scanning at deployment is insufficient. You need a continuous solution that monitors all production images and alerts as soon as a new CVE is discovered in the database.

3.2 Image Scanning with Trivy Operator

Trivy Operator is an open source (CNCF) tool that integrates natively into Kubernetes to continuously scan images of all pods.

How It Works

1. Deployment via Helm into the cluster
2. Auto-discovery: Trivy Operator automatically scans all images from all pods, across all namespaces
3. VulnerabilityReport: creates a custom Kubernetes resource with scan results
4. Continuous update: the CVE database is regularly updated

# Installation via Helm
helm repo add aqua https://aquasecurity.github.io/helm-charts/
helm install trivy-operator aqua/trivy-operator \
  --namespace trivy-system \
  --create-namespace

# View vulnerability reports
kubectl get vulnerabilityreports -n <namespace>
kubectl get vulnerabilityreport <report-name> -o yaml | \
  yq '.report.summary'

VulnerabilityReport Structure

apiVersion: aquasecurity.github.io/v1alpha1
kind: VulnerabilityReport
metadata:
  name: replicaset-products-api-xxx-products-api
spec:
  scanner:
    name: Trivy
    version: "0.50.0"
report:
  summary:
    criticalCount: 4
    highCount: 12
    mediumCount: 28
    lowCount: 45
  vulnerabilities:
    - vulnerabilityID: CVE-2023-12345
      severity: CRITICAL
      resource: openssl
      installedVersion: "3.0.1"
      fixedVersion: "3.0.8"
      title: "OpenSSL: Buffer Overflow..."

3.3 Vulnerability Reports and Policies

VulnerabilityReports are standard Kubernetes resources, which allows automating responses:

3.4 Policy Enforcement with Kyverno

Kyverno is a native Kubernetes policy engine (CNCF) that allows defining rules in YAML to validate, mutate, or generate Kubernetes resources.

Validation Modes

Policy Types

1. Registry Restriction: Only images from approved registries are allowed
2. Non-Root Requirement: All containers must run as non-root
3. CVE Policy: Block or audit based on CVE severity level

4. Module 3 – Securing Applications at Runtime

4.1 Network Policies as Firewalls

Kubernetes networking is open by default: any pod can communicate with any other pod in any namespace. Once a container is compromised, it becomes a jump box allowing pivoting to any other resource in the cluster.

Network Policy is the standard Kubernetes API that acts as a label-based network firewall.

Key Concepts

• Ingress: incoming traffic to a pod
• Egress: outgoing traffic from a pod
• podSelector: selects the pods to which the policy applies
• Default Deny: rule with podSelector: {} that blocks all traffic by default

Zero Trust Architecture for Wired Brain Coffee

Internet → Web Frontend → Products API → PostgreSQL
                       → Stock API    ↗

Required rules:

• Web Frontend can only send to APIs (Egress)
• APIs can only receive from Web Frontend (Ingress) and send to DB (Egress)
• DB accepts only traffic from APIs (Ingress)
• DNS (port 53 UDP) allowed for name resolution

4.2 CSI Secrets Driver and Secrets Management

Problems with native Kubernetes Secrets:

1. Secrets are stored as base64 (encoding, NOT encryption)
2. Anyone with get secrets access can decode them immediately
3. etcd stores secrets in plaintext by default (unless explicitly configured otherwise)
4. Secrets injected as environment variables are visible via printenv inside the container

Solution: External Secrets with the Secret Store CSI Driver

The Secret Store CSI Driver enables loading secrets from external vaults (Azure Key Vault, AWS Secrets Manager, HashiCorp Vault) directly into pods as volumes, without storing credentials in Kubernetes.

Authentication Flow (Workload Identity)

Pod (with Workload Identity annotation)
  → Kubernetes Service Account
    → Azure Managed Identity (via OIDC federation)
      → Azure Key Vault (access with Azure RBAC permissions)
        → Secrets mounted as volumes in the Pod

Advantages:

• No Azure credentials stored in Kubernetes
• Seamless authentication via OIDC federation
• Each pod only sees the secrets it needs
• Automatic secrets rotation supported

4.3 Complete Security Model

By combining all layers, Wired Brain Coffee has a comprehensive security model:

Pod Security Context Hardening

The following measures can be applied in the container securityContext:

securityContext:
  runAsNonRoot: true
  runAsUser: 1000
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop: ["ALL"]
  seccompProfile:
    type: RuntimeDefault

Caution: These settings can break some applications. Apply and test them one at a time.

5. DevSecOps Diagrams

5.1 DevSecOps Pipeline – Shift-Left Security

flowchart LR
    A([Developer]) -->|commit| B[Source Code]
    
    subgraph IDE["IDE / Pre-commit"]
        B --> C[SAST Linting\nSecret Detection]
    end

    subgraph CI["CI Pipeline – Shift-Left"]
        C --> D[Unit Tests]
        D --> E[SAST Scan\nSonarQube / Semgrep]
        E --> F[SCA Scan\nDependency Check / Snyk]
        F --> G[Container Build]
        G --> H[Image Scan\nTrivy / Grype]
        H --> I{Security\nGate}
    end

    subgraph CD["CD Pipeline"]
        I -->|PASS| J[Staging Deploy]
        I -->|FAIL| K([Block & Alert])
        J --> L[DAST Scan\nOWASP ZAP]
        L --> M[Smoke Tests]
        M --> N[Production Deploy]
    end

    subgraph Runtime["Runtime Security"]
        N --> O[Continuous Monitoring\nFalco / Trivy Operator]
        O --> P[Audit Logs\nPrometheus + Grafana]
        P --> Q[Alerting\nPagerDuty / Slack]
    end

    style IDE fill:#e8f4f8,stroke:#2196F3
    style CI fill:#e8f8e8,stroke:#4CAF50
    style CD fill:#fff3e0,stroke:#FF9800
    style Runtime fill:#fce4ec,stroke:#E91E63
    style K fill:#ffcdd2,stroke:#f44336

5.2 SAST / DAST / SCA Flow

flowchart TD
    subgraph SAST["SAST – Static Application Security Testing"]
        S1[Source Code] --> S2[Static code analysis\nbefore execution]
        S2 --> S3[Detection: SQL Injection,\nXSS, Hard-coded Secrets,\nInsecure Functions]
        S3 --> S4[Report: line + severity]
        style SAST fill:#e3f2fd,stroke:#1976D2
    end

    subgraph SCA["SCA – Software Composition Analysis"]
        C1[dependencies.lock / pom.xml] --> C2[Open source dependency\nanalysis]
        C2 --> C3[CVE matching against\nNVD / OSV database]
        C3 --> C4[Report: CVE ID,\nCVSS score, Fix version]
        style SCA fill:#e8f5e9,stroke:#388E3C
    end

    subgraph DAST["DAST – Dynamic Application Security Testing"]
        D1[Application deployed\nin staging] --> D2[Application scan\nduring execution]
        D2 --> D3[Injection attacks,\nAuth bypass, CORS,\nSecurity Headers]
        D3 --> D4[Report: HTTP requests\n+ vulnerabilities found]
        style DAST fill:#fff8e1,stroke:#F57C00
    end

    SAST -->|findings integrated into| Pipeline[CI/CD Pipeline]
    SCA -->|findings integrated into| Pipeline
    DAST -->|findings integrated into| Pipeline
    Pipeline --> Gate{Security Gate}
    Gate -->|Critical / High| Block([BLOCK deployment])
    Gate -->|Acceptable| Deploy([DEPLOY])

5.3 Threat Modeling

flowchart LR
    subgraph STRIDE["STRIDE Threat Model"]
        direction TB
        S["S – Spoofing\nIdentity impersonation\n(auth bypass)"]
        T["T – Tampering\nData alteration\n(injection, MITM)"]
        R["R – Repudiation\nDenial of an action\n(missing audit logs)"]
        I["I – Information Disclosure\nData leakage\n(exposed secrets)"]
        D["D – Denial of Service\n(resource exhaustion)"]
        E["E – Elevation of Privilege\nPrivilege escalation\n(RBAC misconfiguration)"]
    end

    subgraph Controls["Kubernetes Mitigations"]
        direction TB
        CS["OIDC + MFA\nDedicated Service Accounts"]
        CT["TLS everywhere\nNetwork Policies"]
        CR["Kubernetes Audit Logs\nFalco runtime detection"]
        CI["CSI Secrets Driver\nExternal Vault"]
        CD2["Resource Quotas\nLimitRanges"]
        CE["RBAC Least Privilege\nKyverno Policies"]
    end

    S --> CS
    T --> CT
    R --> CR
    I --> CI
    D --> CD2
    E --> CE

5.4 Zero Trust Architecture

flowchart TD
    Internet((Internet)) --> LB[Load Balancer / Ingress]
    
    subgraph Cluster["Kubernetes Cluster – Zero Trust"]
        LB --> NP1{Network Policy\nIngress Rules}
        
        subgraph NS_Frontend["Namespace: frontend"]
            NP1 --> WEB[Web Frontend Pod\nServiceAccount: web-sa\nNon-root, read-only FS]
        end

        WEB --> NP2{Network Policy\nEgress Rules}
        NP2 --> |"Port 8080 only"| NP3{Network Policy\nIngress Rules}

        subgraph NS_Backend["Namespace: backend"]
            NP3 --> API1[Products API Pod\nServiceAccount: products-sa]
            NP3 --> API2[Stock API Pod\nServiceAccount: stock-sa]
        end

        API1 --> NP4{Network Policy\nEgress Rules}
        API2 --> NP4
        NP4 --> |"Port 5432 only"| NP5{Network Policy\nIngress Rules}

        subgraph NS_Data["Namespace: data"]
            NP5 --> DB[(PostgreSQL Pod\nServiceAccount: db-sa\nCSI Secrets Volume)]
        end

        DB -.->|"Secrets via CSI"| KV[(Azure Key Vault\n/ HashiCorp Vault)]
    end

    subgraph IdP["Identity Provider"]
        AAD[Azure Entra ID\nMFA Enforced]
    end

    Admin([Kubectl Admin]) -->|"OIDC + MFA"| AAD
    AAD -->|"Token + RBAC"| Cluster

    style NS_Frontend fill:#e3f2fd,stroke:#1976D2
    style NS_Backend fill:#e8f5e9,stroke:#388E3C
    style NS_Data fill:#fff3e0,stroke:#F57C00
    style IdP fill:#fce4ec,stroke:#E91E63

5.5 Defense-in-Depth Kubernetes

flowchart TB
    subgraph L1["Layer 1 – Cluster Access"]
        A1[OIDC / Azure Entra ID]
        A2[MFA Enforced]
        A3[RBAC Least Privilege]
        A4[kubeconfig expiration]
    end

    subgraph L2["Layer 2 – Deployment Security"]
        B1[Image Scanning - Trivy Operator]
        B2[Registry Restriction - Kyverno]
        B3[Non-Root Policy - Kyverno]
        B4[CVE Audit Policy]
    end

    subgraph L3["Layer 3 – Runtime Security"]
        C1[Network Policies - Zero Trust]
        C2[External Secrets - CSI Driver]
        C3[Pod Security Context]
        C4[seccompProfile]
    end

    subgraph L4["Layer 4 – Observability & Detection"]
        D1[Kubernetes Audit Logs]
        D2[Falco - Runtime Security]
        D3[Trivy Metrics → Prometheus]
        D4[Alerting - PagerDuty / Slack]
    end

    L1 --> L2
    L2 --> L3
    L3 --> L4

    Attacker([Attacker]) -.->|"Must bypass\neach layer"| L1

    style L1 fill:#e8eaf6,stroke:#3F51B5
    style L2 fill:#e8f5e9,stroke:#4CAF50
    style L3 fill:#fff3e0,stroke:#FF9800
    style L4 fill:#fce4ec,stroke:#F44336

6. YAML Snippets for Security Scans in CI/CD

6.1 RBAC – Role and RoleBinding

# Role with Least Privilege – read-only on pods
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

---
# Dedicated ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-explorer-sa
  namespace: default
automountServiceAccountToken: false  # Disable if no API access needed

---
# RoleBinding – bind the ServiceAccount to the Role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kube-explorer-pod-reader
  namespace: default
subjects:
  - kind: ServiceAccount
    name: kube-explorer-sa
    namespace: default
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

---
# ClusterRole for cluster-wide resources
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: namespace-reader
rules:
  - apiGroups: [""]
    resources: ["namespaces", "nodes"]
    verbs: ["get", "list", "watch"]

6.2 Trivy Operator – Vulnerability Report Policy

# Kyverno ClusterPolicy – block images with critical CVEs
# (Audit mode to avoid blocking production)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-vulnerabilities
spec:
  validationFailureAction: Audit  # Do not block in production
  background: false
  rules:
    - name: check-critical-vulnerabilities
      match:
        any:
          - resources:
              kinds: ["Pod"]
      context:
        - name: vulnerabilityreport
          apiCall:
            urlPath: "/apis/aquasecurity.github.io/v1alpha1/namespaces/{{request.namespace}}/vulnerabilityreports"
            jmesPath: "items[?contains(metadata.name, '{{request.object.spec.containers[0].name}}')]|[0].report.summary"
      validate:
        message: "Image has critical vulnerabilities: {{vulnerabilityreport.criticalCount}} critical CVEs found"
        deny:
          conditions:
            any:
              - key: "{{vulnerabilityreport.criticalCount}}"
                operator: GreaterThan
                value: 0

6.3 Kyverno – Cluster Policies

# Policy 1 – Image registry restriction (Enforce)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
spec:
  validationFailureAction: Enforce
  background: true
  rules:
    - name: validate-registries
      match:
        any:
          - resources:
              kinds: ["Pod"]
      validate:
        message: "Images must come from approved registry: ghcr.io/wiredbrain"
        pattern:
          spec:
            containers:
              - image: "ghcr.io/wiredbrain/*"

---
# Policy 2 – Non-Root Requirement (Enforce)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-non-root
spec:
  validationFailureAction: Enforce
  background: true
  rules:
    - name: check-runAsNonRoot
      match:
        any:
          - resources:
              kinds: ["Pod"]
      validate:
        message: "Containers must run as non-root. Set securityContext.runAsNonRoot: true"
        pattern:
          spec:
            containers:
              - securityContext:
                  runAsNonRoot: true

6.4 Network Policy – Default Deny and Allow Rules

# Default Deny – block all traffic by default in the namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: wiredbrain
spec:
  podSelector: {}  # Selects ALL pods in the namespace
  policyTypes:
    - Ingress
    - Egress
  # No rules = everything blocked

---
# Allow DNS – required for name resolution
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: wiredbrain
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - ports:
        - port: 53
          protocol: UDP
        - port: 53
          protocol: TCP

---
# Allow Frontend → APIs
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: wiredbrain
spec:
  podSelector:
    matchLabels:
      app: products-api
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: web-frontend
      ports:
        - port: 8080
          protocol: TCP

---
# Allow APIs → Database (PostgreSQL port 5432)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-to-db
  namespace: wiredbrain
spec:
  podSelector:
    matchLabels:
      app: postgres
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: products-api
        - podSelector:
            matchLabels:
              app: stock-api
      ports:
        - port: 5432
          protocol: TCP

6.5 SecretProviderClass – CSI Driver with Azure Key Vault

# CSI Driver installation (Helm)
# helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
# helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver \
#   --namespace kube-system

---
# SecretProviderClass – defines the secrets to retrieve from Key Vault
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: wiredbrain-key-vault
  namespace: wiredbrain
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    clientID: "<MANAGED_IDENTITY_CLIENT_ID>"  # Workload Identity
    keyvaultName: "wiredbrain-kv"
    cloudName: "AzurePublicCloud"
    objects: |
      array:
        - |
          objectName: db-password
          objectType: secret
          objectVersion: ""
        - |
          objectName: api-key
          objectType: secret
          objectVersion: ""
    tenantId: "<AZURE_TENANT_ID>"

---
# Deployment using the SecretProviderClass
apiVersion: apps/v1
kind: Deployment
metadata:
  name: products-api
  namespace: wiredbrain
spec:
  replicas: 2
  selector:
    matchLabels:
      app: products-api
  template:
    metadata:
      labels:
        app: products-api
      annotations:
        azure.workload.identity/client-id: "<MANAGED_IDENTITY_CLIENT_ID>"
    spec:
      serviceAccountName: products-api-sa
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
        - name: products-api
          image: ghcr.io/wiredbrain/products-api:1.2.3
          ports:
            - containerPort: 8080
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: secrets-store
              mountPath: "/mnt/secrets"
              readOnly: true
          env:
            - name: DB_PASSWORD_FILE
              value: "/mnt/secrets/db-password"
      volumes:
        - name: secrets-store
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "wiredbrain-key-vault"

6.6 Pod Security Context

# Pod spec with full Security Context hardening
apiVersion: v1
kind: Pod
metadata:
  name: secure-app
  namespace: production
spec:
  serviceAccountName: secure-app-sa
  automountServiceAccountToken: false  # Disable if no API access required
  
  # Pod-level Security Context
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    runAsGroup: 10001
    fsGroup: 10001
    seccompProfile:
      type: RuntimeDefault  # Restrictive syscall profile by default
  
  containers:
    - name: app
      image: ghcr.io/wiredbrain/app:1.0.0
      
      # Container-level Security Context
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true  # Read-only filesystem
        capabilities:
          drop: ["ALL"]  # Drop all Linux capabilities
          # add: ["NET_BIND_SERVICE"]  # Add only if necessary
      
      resources:
        requests:
          memory: "64Mi"
          cpu: "100m"
        limits:
          memory: "128Mi"
          cpu: "200m"
      
      # Temporary volume for files requiring write access
      volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: cache
          mountPath: /app/cache
  
  volumes:
    - name: tmp
      emptyDir: {}
    - name: cache
      emptyDir: {}

6.7 GitHub Actions – Complete DevSecOps Pipeline

# .github/workflows/devsecops-pipeline.yml
name: DevSecOps Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  IMAGE_REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # ──────────────────────────────────────────
  # SAST – Static Application Security Testing
  # ──────────────────────────────────────────
  sast:
    name: SAST Scan
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      # Secret detection in code
      - name: Secret Detection (Gitleaks)
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      
      # Static code analysis
      - name: SAST (Semgrep)
        uses: semgrep/semgrep-action@v1
        with:
          config: >-
            p/security-audit
            p/owasp-top-ten
            p/docker
            p/kubernetes
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

  # ──────────────────────────────────────────
  # SCA – Software Composition Analysis
  # ──────────────────────────────────────────
  sca:
    name: SCA / Dependency Scan
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: SCA Scan (Snyk)
        uses: snyk/actions/python@master
        continue-on-error: true
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: --severity-threshold=high --fail-on=all
      
      # Open source alternative with OWASP Dependency-Check
      - name: OWASP Dependency-Check
        uses: dependency-check/Dependency-Check_Action@main
        with:
          project: "my-app"
          path: "."
          format: "HTML"
          args: "--enableRetired --failOnCVSS 7"

  # ──────────────────────────────────────────
  # Build + Image Scan (Trivy)
  # ──────────────────────────────────────────
  build-and-scan:
    name: Build & Image Scan
    runs-on: ubuntu-latest
    needs: [sast, sca]
    permissions:
      contents: read
      packages: write
      security-events: write
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Login to Container Registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.IMAGE_REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Build Docker Image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: false
          tags: ${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
      
      # Image scan with Trivy
      - name: Image Vulnerability Scan (Trivy)
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: "${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}"
          format: "sarif"
          output: "trivy-results.sarif"
          severity: "CRITICAL,HIGH"
          exit-code: "1"  # Fail the pipeline if CRITICAL or HIGH
      
      - name: Upload Trivy Results to GitHub Security
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: "trivy-results.sarif"
      
      # Kubernetes / IaC configuration scan
      - name: IaC Security Scan (Checkov)
        uses: bridgecrewio/checkov-action@master
        with:
          directory: k8s/
          framework: kubernetes
          soft_fail: false
          check: CKV_K8S_*
      
      # Push only if all scans pass
      - name: Push Image to Registry
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
            ${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:latest

  # ──────────────────────────────────────────
  # Deploy to Staging + DAST
  # ──────────────────────────────────────────
  dast:
    name: DAST Scan (Staging)
    runs-on: ubuntu-latest
    needs: [build-and-scan]
    environment: staging
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Deploy to Staging
        run: |
          kubectl set image deployment/app \
            app=${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} \
            --namespace staging
          kubectl rollout status deployment/app --namespace staging --timeout=120s
      
      # DAST with OWASP ZAP
      - name: DAST Scan (OWASP ZAP)
        uses: zaproxy/action-full-scan@v0.10.0
        with:
          target: "https://staging.wiredbrain.example.com"
          rules_file_name: ".zap/rules.tsv"
          cmd_options: "-a -j"
        env:
          ZAP_AUTH_HEADER: "Authorization"
          ZAP_AUTH_HEADER_VALUE: ${{ secrets.STAGING_API_TOKEN }}

  # ──────────────────────────────────────────
  # Deploy to Production
  # ──────────────────────────────────────────
  deploy-production:
    name: Deploy to Production
    runs-on: ubuntu-latest
    needs: [dast]
    environment: production
    
    steps:
      - name: Deploy to Production
        run: |
          kubectl set image deployment/app \
            app=${{ env.IMAGE_REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} \
            --namespace production
          kubectl rollout status deployment/app --namespace production --timeout=300s
      
      - name: Verify Security Policies Post-Deploy
        run: |
          # Verify that Kyverno policies are compliant
          kubectl get policyreport -n production
          # Verify no critical CVEs are present
          kubectl get vulnerabilityreport -n production \
            -o jsonpath='{.items[*].report.summary.criticalCount}' | \
            tr ' ' '\n' | awk '{sum+=$1} END {if(sum>0) exit 1}'

7. Reference Tables

7.1 DevSecOps Security Tools

7.2 OWASP Top 10 – 2021

7.3 DevSecOps Practices by CI/CD Phase

7.4 Kubernetes RBAC Resource Types

Security Principals:

7.5 Authentication Methods Comparison

8. Best Practices and Summary

Core DevSecOps Principles

1. Least Privilege
Every component — pod, ServiceAccount, user — must have only the permissions strictly necessary for its operation. cluster-admin is reserved for break-glass scenarios.

2. Defense-in-Depth
Never rely on a single layer of security. Layer controls so that the compromise of one layer does not compromise the whole.

3. Shift-Left Security
Detect vulnerabilities as early as possible in the development cycle. Fixes in the development phase cost 100x less than in production.

4. Immutable Infrastructure
Containers should not be modified in production. A readOnlyRootFilesystem enforces this principle technically.

5. Zero Trust Networking
By default, all network traffic is blocked. Only explicitly allowed traffic may flow. Start with default-deny then add specific allow rules.

6. Secrets Never in Code
No secret (password, API key, certificate) should ever be committed in source code or stored in unencrypted native Kubernetes Secrets. Use an external vault.

7. Continuous Compliance
Security is not an annual audit. Compliance must be verified at every commit, every deployment, and continuously in production.

Kubernetes Security Checklist

Cluster Access

• OIDC/Entra ID authentication configured (no static certificates)
• MFA enforced for all users
• --disable-local-accounts enabled on AKS
• Dedicated Service Accounts per application
• automountServiceAccountToken: false on pods that do not need API access
• RBAC with Least Privilege verified via kubectl auth can-i --list

Deployment Security

• Trivy Operator installed and scanning enabled
• Kyverno policies deployed (registry restriction, non-root)
• Critical CVEs blocked or audited
• Images built from approved registries only
• Image signing with cosign (Sigstore)

Runtime Security

• default-deny Network Policy in all namespaces
• Minimal and documented Network Policy rules
• Secrets stored in Azure Key Vault / HashiCorp Vault
• CSI Secrets Driver configured
• securityContext.runAsNonRoot: true on all containers
• allowPrivilegeEscalation: false on all containers
• readOnlyRootFilesystem: true enabled and tested
• capabilities.drop: ["ALL"] applied

Observability

• Kubernetes Audit Logs enabled and archived
• Falco deployed for runtime detection
• Trivy metrics exported to Prometheus
• Alerts configured for new critical CVEs
• Grafana dashboard for vulnerability visibility

Key Quote

“Security shouldn’t be seen as a checkbox. New vulnerabilities emerge and attack techniques evolve. The principles of Defense-in-Depth — least privilege access controls for users and applications, ongoing vulnerability scanning enforced as one of many security policies, restricted access to networks and secrets for containment if there is a compromise — are the foundations for proactive defense.”

Course: Configuring and Managing Kubernetes Security — DevOps Foundations: Security and DevSecOps
Created: 2026-06-13

Search Terms

devops · troubleshooting · security · devsecops · ci/cd · git · rbac · authentication · kubernetes · policy · cluster · policies · securing · access · accounts · actions · architecture · azure · context · csi · defense-in-depth · deployment · driver · flow