Fix ArgoCD Sync Timeout

What's Actually Happening

ArgoCD application sync times out before all resources are reconciled. The sync operation fails, leaving the application in a degraded state.

The Error You'll See

Sync timeout:

```bash $ argocd app get myapp

Status: Degraded Message: Operation has timed out after 5m0s Health: Progressing Sync Status: Synced ```

Operation failed:

```bash $ argocd app logs myapp

ERROR: sync operation timed out time="2024-01-01T00:00:00Z" level=error msg="Operation terminated with timeout" ```

UI shows:

Sync Status: Failed
Message: Operation timed out
Last Sync: 5 minutes ago

Why This Happens

1. 1.Slow resource creation - Resources take long to become ready
2. 2.Dependencies - Resources waiting on others
3. 3.Cluster overloaded - Kubernetes API slow
4. 4.Too many resources - Large application with many manifests
5. 5.Default timeout - 5-minute timeout insufficient
6. 6.Health checks slow - Custom health checks timing out

Step 1: Check Application Status

```bash # Get application details: argocd app get myapp

# Get sync status: argocd app get myapp --refresh

# Check sync operation: argocd app get myapp -o json | jq '.status.operationState'

# Check sync history: argocd app history myapp

# Check resources: argocd app resources myapp

# Check application in Kubernetes: kubectl get application myapp -n argocd -o yaml

# Check ArgoCD controller logs: kubectl logs -n argocd deployment/argocd-application-controller | grep myapp ```

Step 2: Check Timeout Configuration

```bash # Check application spec: kubectl get application myapp -n argocd -o yaml | grep -A 10 sync

# Default timeout is 5 minutes (300s)

# In Application manifest: spec: syncPolicy: syncOptions: - Timeout=600 # Increase to 10 minutes

# Or via CLI: argocd app set myapp --sync-option Timeout=600

# Check controller config: kubectl get configmap argocd-cm -n argocd -o yaml | grep timeout

# Global timeout in argocd-cmd-params: kubectl get configmap argocd-cmd-params-cm -n argocd -o yaml | grep timeout ```

Step 3: Increase Sync Timeout

```yaml # In Application manifest: apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: myapp spec: destination: namespace: default server: https://kubernetes.default.svc source: path: app repoURL: https://github.com/org/repo.git syncPolicy: syncOptions: - Timeout=900 # 15 minutes - ServerSideApply=true - CreateNamespace=true

# Or via argocd CLI: argocd app set myapp --sync-option Timeout=900

# For specific sync operation: argocd app sync myapp --timeout 900

# Restart controller after config changes: kubectl rollout restart deployment/argocd-application-controller -n argocd ```

Step 4: Configure Resource Health Checks

```bash # Check health status: argocd app get myapp --health

# Custom health checks in argocd-cm: apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: resource.customizations: | networking.k8s.io/Ingress: health.lua: | hs = {} hs.status = "Healthy" return hs

# For deployments with slow readiness: data: resource.customizations: | apps/Deployment: health.lua: | hs = {} if obj.status ~= nil then if obj.status.replicas ~= nil and obj.status.readyReplicas ~= nil then if obj.status.replicas == obj.status.readyReplicas then hs.status = "Healthy" return hs end end end hs.status = "Progressing" return hs

# Apply: kubectl apply -f argocd-cm.yaml ```

Step 5: Configure Sync Waves

```yaml # Order resource creation with sync waves: # In manifests, add annotations:

# First wave (secrets, configs): apiVersion: v1 kind: Secret metadata: name: db-secret annotations: argocd.argoproj.io/sync-wave: "0" --- # Second wave (databases): apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres annotations: argocd.argoproj.io/sync-wave: "1" --- # Third wave (applications): apiVersion: apps/v1 kind: Deployment metadata: name: app annotations: argocd.argoproj.io/sync-wave: "2"

# Waves are processed in order # Wait for each wave to complete before next

# In Application sync options: spec: syncPolicy: syncOptions: - Timeout=900 automated: selfHeal: true prune: true ```

Step 6: Configure Sync Hooks

```yaml # Use PreSync hooks for prerequisites: apiVersion: batch/v1 kind: Job metadata: generateName: presync-migration- annotations: argocd.argoproj.io/hook: PreSync argocd.argoproj.io/hook-delete-policy: HookSucceeded spec: template: spec: containers: - name: migration image: migration-image command: ["./migrate.sh"] restartPolicy: Never

# Hook types: # PreSync: Run before sync # Sync: Run during sync (with resources) # PostSync: Run after successful sync # SyncFail: Run if sync fails

# Hook delete policies: # HookSucceeded: Delete on success # HookFailed: Delete on failure # BeforeHookCreation: Delete previous before new ```

Step 7: Check Cluster Performance

```bash # Check Kubernetes API latency: kubectl get --raw /metrics | grep apiserver_request_duration_seconds

# Check ArgoCD controller resources: kubectl top pods -n argocd

# Check if cluster overloaded: kubectl describe nodes | grep -A 5 "Allocated resources"

# Check resource quotas: kubectl get resourcequota -A

# Check pending pods: kubectl get pods -A --field-selector=status.phase=Pending

# If cluster slow: # 1. Increase controller resources kubectl patch deployment argocd-application-controller -n argocd --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value":"2Gi"}]'

# 2. Increase API server resources # 3. Check for noisy neighbors ```

Step 8: Optimize Application Size

```bash # Check application size: argocd app get myapp -o json | jq '.status.resources | length'

# Large applications (100+ resources) can timeout

# Option 1: Split into multiple applications # app-components.yaml: apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: myapp-database spec: source: path: database --- apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: myapp-frontend spec: source: path: frontend

# Option 2: Use ApplicationSet apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: myapp spec: generators: - list: elements: - name: database path: database - name: frontend path: frontend

# Option 3: Increase parallelism # In argocd-application-controller: --status-processors 30 --operation-processors 20 ```

Step 9: Configure Retry Policy

```yaml # In Application sync policy: spec: syncPolicy: syncOptions: - Timeout=900 automated: selfHeal: true prune: true retry: limit: 5 # Max retries backoff: duration: 5s factor: 2 maxDuration: 3m

# Retry configuration: # limit: Maximum retry attempts # duration: Initial backoff duration # factor: Backoff multiplier # maxDuration: Maximum backoff duration

# Example: 5s, 10s, 20s, 40s, 80s with factor 2

# Check retry in status: kubectl get application myapp -n argocd -o json | jq '.status.operationState.retryCount' ```

Step 10: Monitor Sync Performance

```bash # Create monitoring script: cat << 'EOF' > /usr/local/bin/monitor-argocd.sh #!/bin/bash

echo "=== ArgoCD Applications ===" argocd app list

echo "" echo "=== Syncing Applications ===" argocd app list --status Progressing

echo "" echo "=== Failed Syncs ===" argocd app list --operation-failed

echo "" echo "=== Controller Metrics ===" kubectl exec -n argocd deployment/argocd-application-controller -- curl -s localhost:8082/metrics | grep argocd_sync

echo "" echo "=== Long-running Syncs ===" kubectl logs -n argocd deployment/argocd-application-controller | grep "sync.*took" | tail -10 EOF

chmod +x /usr/local/bin/monitor-argocd.sh

# ArgoCD Prometheus metrics: kubectl port-forward -n argocd svc/argocd-metrics 8082:8082 & curl http://localhost:8082/metrics | grep argocd_sync

# Key metrics: # argocd_app_sync_total # argocd_app_sync_failed_total # argocd_app_sync_duration_seconds

# Prometheus alerts: - alert: ArgoCDSyncTimeout expr: rate(argocd_app_sync_failed_total[10m]) > 0 for: 5m labels: severity: warning annotations: summary: "ArgoCD sync failures detected" ```

ArgoCD Sync Timeout Checklist

Verify the Fix

```bash # After adjusting sync settings

# 1. Trigger sync argocd app sync myapp // Sync completes successfully

# 2. Check application status argocd app get myapp // Status: Healthy, Synced

# 3. Verify resources argocd app resources myapp // All resources healthy

# 4. Check sync duration argocd app history myapp // Duration within timeout

# 5. Test with large change argocd app sync myapp --force // Still succeeds

# 6. Monitor controller kubectl logs -f -n argocd deployment/argocd-application-controller | grep myapp // No timeout errors ```

Related Issues

• [Fix ArgoCD App Out Of Sync](/articles/fix-argocd-app-out-of-sync)
• [Fix ArgoCD Health Check Failed](/articles/fix-argocd-health-check-failed)
• [Fix Helm Release Failed](/articles/fix-helm-release-failed)