Fix Kyverno Policy Not Enforcing

What's Actually Happening

Kyverno policies are created but not being enforced on Kubernetes resources. Resources bypass policy validation or mutation even when policies should apply.

The Error You'll See

```bash $ kubectl apply -f deployment.yaml // No policy validation, resource created

$ kubectl get deploy -n production NAME READY UP-TO-DATE AVAILABLE AGE myapp 1/1 1 1 10s

// Policy should have blocked this but didn't ```

Policy status shows inactive:

$ kubectl get clusterpolicy require-labels
NAME             READY   AGE
require-labels   false   5m

Admission webhook not firing:

$ kubectl describe deploy myapp
// No kyverno annotation, policy not applied

Why This Happens

1. 1.Policy not ready - Policy validation failed, status not ready
2. 2.Webhook not configured - Admission webhook missing or misconfigured
3. 3.Namespace mismatch - Policy doesn't match target namespace
4. 4.Selector mismatch - Resource selector doesn't match policy
5. 5.Webhook failure policy - Failure policy set to Ignore
6. 6.Kyverno pod not ready - Admission controller not running

Step 1: Check Kyverno Installation

```bash # Check Kyverno deployment: kubectl get deploy -n kyverno

# Check Kyverno pods: kubectl get pods -n kyverno

# Check Kyverno logs: kubectl logs -n kyverno deploy/kyverno -c kyverno

# Verify webhook service: kubectl get svc -n kyverno

# Check Kyverno admission webhook: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg

# Check mutating webhook: kubectl get mutatingwebhookconfiguration kyverno-mutating-webhook-cfg

# Verify webhook endpoints: kubectl get endpoints -n kyverno kyverno-svc ```

Step 2: Check Policy Status

```bash # List all policies: kubectl get clusterpolicy kubectl get policy -A

# Check policy details: kubectl get clusterpolicy require-labels -o yaml

# Check policy ready status: kubectl get clusterpolicy require-labels -o jsonpath='{.status.ready}'

# View policy validation errors: kubectl describe clusterpolicy require-labels

# Check policy background mode: kubectl get clusterpolicy require-labels -o jsonpath='{.spec.background}'

# Check policy validation failure action: kubectl get clusterpolicy require-labels -o jsonpath='{.spec.validationFailureAction}' # Should be "enforce" to block resources # "audit" only logs violations

# Check policy rules: kubectl get clusterpolicy require-labels -o yaml | grep -A20 "rules:" ```

Step 3: Verify Webhook Configuration

```bash # Check validating webhook: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg -o yaml

# Verify webhook service reference: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].clientConfig.service}'

# Check webhook namespace selector: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].namespaceSelector}'

# Check webhook object selector: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].objectSelector}'

# Check failure policy: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].failurePolicy}' # "Fail" blocks on webhook error # "Ignore" bypasses policy on error

# Check webhook rules (what resources it applies to): kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].rules}'

# Test webhook connectivity: kubectl run test --image=nginx --dry-run=client -o yaml | \ kubectl apply --dry-run=server -f - ```

Step 4: Check Policy Match Conditions

```yaml # Policy with match conditions: apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: require-labels spec: validationFailureAction: enforce rules: - name: check-labels match: any: - resources: kinds: - Deployment namespaces: - production selector: matchLabels: app: myapp validate: message: "Required label 'owner' is missing" pattern: metadata: labels: owner: "?*"

# Check if resource matches: # 1. Kind matches: Deployment # 2. Namespace matches: production # 3. Labels match: app=myapp

# Test with resource: kubectl get deploy myapp -n production -o yaml | grep -A5 metadata

# If selector doesn't match, policy won't apply ```

Step 5: Debug Policy Rules

```bash # Check rule match resources: kubectl get clusterpolicy require-labels -o yaml

# Look for: # match: # resources: # kinds: ["Deployment"] # namespaces: ["production"]

# Verify resource kind: kubectl api-resources | grep deployment

# Check if namespace exists: kubectl get namespace production

# Check if resource has matching labels: kubectl get deploy myapp -n production --show-labels

# Test policy with kyverno CLI: kyverno apply require-labels.yaml -r deployment.yaml

# Validate policy syntax: kyverno validate require-labels.yaml

# Check policy report: kubectl get policyreport -A kubectl describe policyreport -n production

# View violations: kubectl get policyreport -n production -o yaml | grep -A10 "results:" ```

Step 6: Fix Failure Policy

```yaml # Change failure policy from Ignore to Fail:

# Check current: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].failurePolicy}'

# If it's "Ignore", webhook errors bypass policy # Patch to "Fail": kubectl patch validatingwebhookconfiguration kyverno-validating-webhook-cfg \ --type='json' -p='[{"op": "replace", "path": "/webhooks/0/failurePolicy", "value": "Fail"}]'

# Also check mutating webhook: kubectl patch mutatingwebhookconfiguration kyverno-mutating-webhook-cfg \ --type='json' -p='[{"op": "replace", "path": "/webhooks/0/failurePolicy", "value": "Fail"}]'

# Verify webhook applies to correct operations: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].rules[0].operations}' # Should include CREATE, UPDATE ```

Step 7: Check Namespace Exclusions

```bash # Kyverno excludes certain namespaces by default: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].namespaceSelector.matchExpressions}'

# Default exclusions: # - kyverno # - kube-system # - kube-public

# If your namespace is excluded, policy won't apply

# Check if namespace has kyverno exclusion label: kubectl get namespace production --show-labels

# Label that excludes: # kyverno.io/skip: true

# Remove exclusion label: kubectl label namespace production kyverno.io/skip-

# Or configure webhook to include: kubectl patch validatingwebhookconfiguration kyverno-validating-webhook-cfg \ --type='json' -p='[{"op": "replace", "path": "/webhooks/0/namespaceSelector", "value": {}}]' ```

Step 8: Test Policy Enforcement

```bash # Create test deployment without required label: kubectl apply -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: test-deploy namespace: production spec: replicas: 1 selector: matchLabels: app: test template: metadata: labels: app: test spec: containers: - name: nginx image: nginx EOF

# If policy enforces, should be blocked: # Error from server: admission webhook "kyverno-validating-webhook-cfg" denied the request

# Check policy report for audit mode: kubectl get policyreport -n production

# View violation details: kubectl get policyreport -n production -o yaml | grep -A20 "test-deploy"

# Check Kyverno logs: kubectl logs -n kyverno deploy/kyverno | grep test-deploy

# Force apply to see rejection: kubectl apply -f deployment.yaml --dry-run=server ```

Step 9: Check Resource Exclusions

```bash # Kyverno may exclude certain resources:

# Check webhook object selector: kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].objectSelector}'

# If objectSelector exists, check labels: # Example exclusion: # matchLabels: # kyverno.io/exclude: "true"

# Check resource labels: kubectl get deploy myapp -n production --show-labels

# Remove exclusion label: kubectl label deploy myapp kyverno.io/exclude-

# Check policy match resources: kubectl get clusterpolicy require-labels -o yaml | grep -A10 "match"

# Ensure resource kind is in policy match: # kinds: ["Deployment", "Pod", "StatefulSet"] ```

Step 10: Kyverno Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-kyverno.sh #!/bin/bash

echo "=== Kyverno Pods ===" kubectl get pods -n kyverno

echo "" echo "=== Kyverno Webhooks ===" kubectl get validatingwebhookconfiguration | grep kyverno kubectl get mutatingwebhookconfiguration | grep kyverno

echo "" echo "=== Cluster Policies ===" kubectl get clusterpolicy

echo "" echo "=== Policy Status ===" for policy in $(kubectl get clusterpolicy -o name); do ready=$(kubectl get $policy -o jsonpath='{.status.ready}') echo "$policy: Ready=$ready" done

echo "" echo "=== Failure Policy ===" kubectl get validatingwebhookconfiguration kyverno-validating-webhook-cfg \ -o jsonpath='{.webhooks[0].failurePolicy}'

echo "" echo "=== Policy Reports ===" kubectl get policyreport -A

echo "" echo "=== Recent Violations ===" kubectl logs -n kyverno deploy/kyverno --tail=20 | grep -i "deny|violation"

echo "" echo "=== Recommendations ===" echo "1. Ensure Kyverno pods are running" echo "2. Check policy ready status is true" echo "3. Verify webhook failure policy is Fail" echo "4. Check policy match selectors" echo "5. Ensure namespace not excluded" echo "6. Verify validationFailureAction is enforce" EOF

chmod +x /usr/local/bin/check-kyverno.sh

# Usage: /usr/local/bin/check-kyverno.sh ```

Kyverno Policy Checklist

Verify the Fix

```bash # After fixing Kyverno policy enforcement

# 1. Check policy is ready kubectl get clusterpolicy require-labels // READY: true

# 2. Test with violating resource kubectl apply -f deployment.yaml // Error: admission webhook denied

# 3. Create compliant resource kubectl apply -f compliant-deployment.yaml // Created successfully

# 4. Check policy report kubectl get policyreport -n production // No violations for compliant resources

# 5. View Kyverno logs kubectl logs -n kyverno deploy/kyverno --tail=10 // Policy applied successfully

# 6. Verify webhook firing kubectl describe deploy compliant-deploy // kyverno.io/apply annotation present ```

Related Issues

• [Fix Kubernetes Admission Webhook Failed](/articles/fix-kubernetes-admission-webhook-failed)
• [Fix OPA Gatekeeper Violation False](/articles/fix-opa-gatekeeper-violation-false)
• [Fix Istio Traffic Not Routing](/articles/fix-istio-traffic-not-routing)