Fix KEDA Scaler Not Triggering

What's Actually Happening

KEDA (Kubernetes Event-driven Autoscaling) scaler is configured but not triggering scale events. Deployments are not scaling based on metrics.

The Error You'll See

```bash $ kubectl get scaledobject -n default

NAME SCALED READY ACTIVE AGE my-scaler Unknown False False 5m

$ kubectl logs -n keda deploy/keda-operator-manager

Error: trigger authentication reference not found ```

HPA not created:

```bash $ kubectl get hpa -n default

No resources found. ```

Metric server error:

Error: unable to get metric: the server could not find the requested resource

Why This Happens

1. 1.Trigger auth missing - TriggerAuthentication not created or wrong reference
2. 2.Scaler misconfigured - Wrong scaler type or parameters
3. 3.HPA creation failed - HorizontalPodAutoscaler not created
4. 4.Metrics server missing - Kubernetes metrics server not installed
5. 5.RBAC issues - KEDA lacks permissions
6. 6.Workload reference wrong - ScaledObject references wrong deployment

Step 1: Check ScaledObject Status

```bash # List ScaledObjects: kubectl get scaledobject -A

# Check specific ScaledObject: kubectl get scaledobject my-scaler -n default -o yaml

# Describe ScaledObject: kubectl describe scaledobject my-scaler -n default

# Check conditions: kubectl get scaledobject my-scaler -n default -o jsonpath='{.status.conditions}'

# Check HPA reference: kubectl get scaledobject my-scaler -n default -o jsonpath='{.status.hpaName}'

# Check ready status: kubectl get scaledobject my-scaler -n default -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

# View events: kubectl get events --field-selector involvedObject.name=my-scaler -n default

# Check scaler active status: kubectl get scaledobject my-scaler -n default -o jsonpath='{.status.conditions[?(@.type=="Active")].status}' ```

Step 2: Verify Trigger Authentication

```bash # List TriggerAuthentication: kubectl get triggerauth -A

# Check TriggerAuthentication: kubectl get triggerauth my-trigger-auth -n default -o yaml

# Describe TriggerAuthentication: kubectl describe triggerauth my-trigger-auth -n default

# Check secret reference in TriggerAuthentication: kubectl get triggerauth my-trigger-auth -n default -o jsonpath='{.spec.secretTargetRef}'

# Verify referenced secret exists: kubectl get secret -n default <secret-name>

# Create TriggerAuthentication: apiVersion: keda.sh/v1alpha1 kind: TriggerAuthentication metadata: name: my-trigger-auth namespace: default spec: secretTargetRef: - parameter: connection name: kafka-credentials key: connection-string

# Check ClusterTriggerAuthentication: kubectl get clustertriggerauth

# Verify TriggerAuthentication reference in ScaledObject: kubectl get scaledobject my-scaler -n default -o yaml | grep -A5 authenticationRef ```

Step 3: Check Scaler Configuration

```bash # Check triggers configuration: kubectl get scaledobject my-scaler -n default -o jsonpath='{.spec.triggers}'

# Common scaler types: # - kafka # - rabbitmq # - aws-sqs # - prometheus # - cpu # - memory

# Verify scaler type is correct: kubectl get scaledobject my-scaler -n default -o yaml | grep type

# Check scaler metadata: kubectl get scaledobject my-scaler -n default -o yaml | grep -A10 metadata

# Example Kafka scaler: triggers: - type: kafka metadata: bootstrapServers: kafka.svc:9092 consumerGroup: my-group topic: my-topic lagThreshold: "100"

# Example Prometheus scaler: triggers: - type: prometheus metadata: serverAddress: http://prometheus:9090 metricName: http_requests_total threshold: "100" query: rate(http_requests_total[5m])

# Validate scaler parameters: kubectl logs -n keda deploy/keda-operator-manager | grep -i "scaler|trigger" ```

Step 4: Check HPA Creation

```bash # Check if HPA was created: kubectl get hpa -n default

# Check HPA details: kubectl get hpa keda-hpa-my-scaler -n default -o yaml

# Describe HPA: kubectl describe hpa keda-hpa-my-scaler -n default

# Check HPA metrics: kubectl get hpa keda-hpa-my-scaler -n default -o jsonpath='{.spec.metrics}'

# Check HPA status: kubectl get hpa keda-hpa-my-scaler -n default -o jsonpath='{.status}'

# If HPA not created, check KEDA logs: kubectl logs -n keda deploy/keda-operator-manager | grep -i "hpa|horizontal"

# Check HPA reference in ScaledObject: kubectl get scaledobject my-scaler -n default -o jsonpath='{.status.hpaName}'

# Manual HPA check: kubectl get hpa -A | grep keda

# Verify HPA target deployment: kubectl get hpa keda-hpa-my-scaler -n default -o jsonpath='{.spec.scaleTargetRef}' ```

Step 5: Verify Workload Reference

```bash # Check scaleTargetRef in ScaledObject: kubectl get scaledobject my-scaler -n default -o yaml | grep -A5 scaleTargetRef

# Must match existing deployment: kubectl get deploy -n default

# Correct reference: # scaleTargetRef: # apiVersion: apps/v1 # kind: Deployment # name: my-deployment

# Verify deployment exists: kubectl get deploy my-deployment -n default

# Check deployment has replicas: kubectl get deploy my-deployment -n default -o jsonpath='{.spec.replicas}'

# If deployment missing, create or fix reference: kubectl patch scaledobject my-scaler -n default --type='merge' \ -p '{"spec":{"scaleTargetRef":{"name":"correct-deployment"}}}'

# Check StatefulSet reference: # scaleTargetRef: # apiVersion: apps/v1 # kind: StatefulSet # name: my-statefulset ```

Step 6: Check Metrics Server

```bash # Check if metrics server installed: kubectl get pods -n kube-system | grep metrics-server

# Check metrics server API: kubectl get --raw /apis/metrics.k8s.io/v1beta1/namespaces/default/pods

# Install metrics server if missing: kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# For EKS: kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Check metrics server logs: kubectl logs -n kube-system deploy/metrics-server

# Test metrics API: kubectl top pods -n default kubectl top nodes

# Check metrics server args: kubectl get deploy metrics-server -n kube-system -o yaml | grep -A10 args

# May need to add --kubelet-insecure-tls for development: args: - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP ```

Step 7: Check KEDA Controller

```bash # Check KEDA namespace: kubectl get all -n keda

# Check KEDA pods: kubectl get pods -n keda

# View KEDA logs: kubectl logs -n keda deploy/keda-operator-manager

# Check KEDA metrics server: kubectl logs -n keda deploy/keda-metrics-apiserver

# Check KEDA version: kubectl get deploy -n keda -o yaml | grep image

# Describe KEDA pods: kubectl describe pods -n keda

# Check KEDA resources: kubectl top pods -n keda

# Restart KEDA: kubectl rollout restart deploy/keda-operator-manager -n keda kubectl rollout restart deploy/keda-metrics-apiserver -n keda

# Check KEDA events: kubectl get events -n keda --sort-by='.lastTimestamp' ```

Step 8: Check RBAC Permissions

```bash # Check KEDA service account: kubectl get sa -n keda

# Check ClusterRole: kubectl get clusterrole | grep keda

# Check ClusterRoleBinding: kubectl get clusterrolebinding | grep keda

# Verify KEDA can read deployments: kubectl auth can-i get deployments \ --as=system:serviceaccount:keda:keda-operator \ -n default

# Check if KEDA can read secrets: kubectl auth can-i get secrets \ --as=system:serviceaccount:keda:keda-operator \ -n default

# Check HPA permissions: kubectl auth can-i create horizontalpodautoscalers \ --as=system:serviceaccount:keda:keda-operator \ -n default

# Create missing RBAC: apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: keda-operator rules: - apiGroups: [""] resources: ["pods", "secrets", "configmaps"] verbs: ["get", "list", "watch"] - apiGroups: ["apps"] resources: ["deployments", "statefulsets"] verbs: ["get", "list", "watch", "update", "patch"] - apiGroups: ["autoscaling"] resources: ["horizontalpodautoscalers"] verbs: ["create", "delete", "get", "list", "watch", "update", "patch"] ```

Step 9: Debug Scaler Metrics

```bash # Check external metrics API: kubectl get --raw /apis/external.metrics.k8s.io/v1beta1 | jq .

# Check specific metric: kubectl get --raw /apis/external.metrics.k8s.io/v1beta1/namespaces/default/kafka-lag | jq .

# Port-forward to KEDA metrics server: kubectl port-forward -n keda svc/keda-metrics-apiserver 443:443 &

# Query metrics: curl -k https://localhost:443/apis/external.metrics.k8s.io/v1beta1

# Check KEDA metrics server logs: kubectl logs -n keda deploy/keda-metrics-apiserver --tail=50

# Enable debug logging: kubectl set env deploy/keda-operator-manager -n keda LOG_LEVEL=debug

# Watch KEDA logs for scaler errors: kubectl logs -n keda deploy/keda-operator-manager -f | grep -i "error|scaler"

# Test scaler connection: # For Kafka: kubectl run test-kafka --image=confluentinc/cp-kafka:latest --rm -it --restart=Never \ -- kafka-broker-api-versions --bootstrap-server kafka:9092

# For Prometheus: kubectl run test-prom --image=curlimages/curl --rm -it --restart=Never \ -- curl http://prometheus:9090/api/v1/query?query=up ```

Step 10: KEDA Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-keda.sh #!/bin/bash

SCALER=${1:-"my-scaler"} NS=${2:-"default"}

echo "=== KEDA Pods ===" kubectl get pods -n keda

echo "" echo "=== ScaledObject Status ===" kubectl get scaledobject $SCALER -n $NS -o yaml | grep -A20 "status:"

echo "" echo "=== HPA Status ===" kubectl get hpa -n $NS

echo "" echo "=== TriggerAuthentication ===" kubectl get triggerauth -n $NS

echo "" echo "=== Target Deployment ===" DEPLOY=$(kubectl get scaledobject $SCALER -n $NS -o jsonpath='{.spec.scaleTargetRef.name}') kubectl get deploy $DEPLOY -n $NS 2>/dev/null || echo "Deployment not found: $DEPLOY"

echo "" echo "=== Metrics Server ===" kubectl get pods -n kube-system | grep metrics-server

echo "" echo "=== External Metrics ===" kubectl get --raw /apis/external.metrics.k8s.io/v1beta1/namespaces/$NS 2>/dev/null | jq '.resources[].name' 2>/dev/null || echo "No external metrics"

echo "" echo "=== KEDA Logs ===" kubectl logs -n keda deploy/keda-operator-manager --tail=20 | grep -i "scaler|error|warning"

echo "" echo "=== Recommendations ===" echo "1. Verify TriggerAuthentication references existing secret" echo "2. Check scaler type and metadata are correct" echo "3. Ensure target deployment/statefulset exists" echo "4. Verify metrics server is installed" echo "5. Check KEDA RBAC permissions" echo "6. Review HPA creation status" echo "7. Test scaler connectivity to external system" EOF

chmod +x /usr/local/bin/check-keda.sh

# Usage: /usr/local/bin/check-keda.sh my-scaler default ```

KEDA Scaler Checklist

Verify the Fix

```bash # After fixing KEDA scaler issues

# 1. Check ScaledObject kubectl get scaledobject my-scaler -n default // READY: true

# 2. Verify HPA created kubectl get hpa -n default // keda-hpa-my-scaler exists

# 3. Check metrics kubectl get hpa keda-hpa-my-scaler -n default // TARGETS shows metric values

# 4. Test scaling # Increase load/queue depth kubectl get deploy my-deployment -w // Replicas increase

# 5. Check KEDA logs kubectl logs -n keda deploy/keda-operator-manager --tail=10 // No errors

# 6. Verify external metric kubectl get --raw /apis/external.metrics.k8s.io/v1beta1/namespaces/default/kafka-lag // Returns metric value ```

Related Issues

• [Fix Kubernetes Deployment Not Working](/articles/fix-kubernetes-deployment-not-working)
• [Fix Prometheus Target Down](/articles/fix-prometheus-target-down)
• [Fix Kafka Consumer Lag High](/articles/fix-kafka-consumer-lag-high)