What's Actually Happening
Kustomize fails to build Kubernetes manifests due to configuration errors, missing resources, or invalid transformations.
The Error You'll See
Build error:
```bash $ kustomize build ./overlay/production
Error: accumulating resources: accumulation err='accumulating resources from 'deployment.yaml': malformed resource' ```
Reference error:
Error: no resource matches the given path 'base/deployment.yaml'Transformer error:
Error: json: unknown field "replacements" in KustomizationValidation error:
Error: invalid Kustomization: missing metadata.nameWhy This Happens
- 1.Missing files - Referenced resources don't exist
- 2.Invalid YAML - Syntax errors in manifests
- 3.Wrong API version - Incompatible Kustomize API
- 4.Missing fields - Required metadata missing
- 5.Transformer error - Invalid transformation config
- 6.Circular dependency - Resources referencing each other
Step 1: Check Kustomize Version
```bash # Check Kustomize version: kustomize version
# Check Kubernetes version compatibility: kubectl version --client
# Common version issues: # Kustomization API v1beta1 deprecated in v3.8+ # Kustomization API v1 requires v4.0+
# Install specific version: curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
# Or use kubectl kustomize (embedded): kubectl kustomize ./overlay/production
# kubectl kustomize uses embedded version # May differ from standalone kustomize ```
Step 2: Check Directory Structure
```bash # Check directory structure: tree ./kustomize-project
# Expected structure: kustomize-project/ ├── base/ │ ├── kustomization.yaml │ ├── deployment.yaml │ └── service.yaml └── overlays/ ├── production/ │ ├── kustomization.yaml │ └── kustomizeconfig.yaml └── staging/ └── kustomization.yaml
# Check kustomization.yaml exists: ls -la base/kustomization.yaml ls -la overlays/production/kustomization.yaml
# Check file contents: cat base/kustomization.yaml ```
Step 3: Validate Kustomization YAML
```bash # Check kustomization.yaml syntax: cat base/kustomization.yaml | yamllint -
# Basic kustomization structure: apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization
resources: - deployment.yaml - service.yaml - configmap.yaml
commonLabels: app: myapp
# Check API version matches Kustomize version: # v1beta1 for Kustomize 3.x # v1 for Kustomize 4.x+
# Validate fields: kustomize build ./base --dry-run
# Check for typos in field names: # Common mistakes: # resources: -> resources: (correct) # resource: -> resources: (typo) # Resorces: -> resources: (wrong case) ```
Step 4: Check Resource References
```bash # Check referenced files exist: cat base/kustomization.yaml | grep -A 100 "resources:" | grep "\.yaml"
# Verify each file: for f in $(grep "\.yaml" base/kustomization.yaml); do ls -la base/$f done
# Common issues: # 1. Wrong path: resources: - ../wrong-path/deployment.yaml # Path doesn't exist
# 2. Missing file: resources: - deployment.yaml # File doesn't exist
# 3. Wrong extension: resources: - deployment.yml # But file is deployment.yaml
# Fix by creating missing files or correcting paths
# Check remote resources: resources: - https://github.com/org/repo/config/default?ref=v1.0.0
# Test remote resource: curl -I https://github.com/org/repo/config/default?ref=v1.0.0 ```
Step 5: Check Manifest Validity
```bash # Check each manifest file: for f in base/*.yaml; do echo "Checking $f..." kubectl apply --dry-run=client -f $f done
# Check YAML syntax: python -c "import yaml; yaml.safe_load(open('base/deployment.yaml'))"
# Use yamllint: yamllint base/deployment.yaml
# Common YAML errors: # 1. Wrong indentation: spec: containers: - name: app # Wrong: should be indented more
# 2. Missing quote in strings: command: ["echo", "test message] # Missing closing quote
# 3. Invalid characters: name: my-app@latest # @ not allowed in name
# Validate generated output: kustomize build ./base > output.yaml kubectl apply --dry-run=client -f output.yaml ```
Step 6: Check Transformers
```bash # Check transformer configuration: cat overlays/production/kustomization.yaml
# Common transformers: # commonLabels: # commonAnnotations: # images: # namePrefix: # nameSuffix: # namespace: # replacements: # patches: # configMapGenerator: # secretGenerator:
# Check images transformer: images: - name: myapp newTag: v2.0.0 newName: myapp-production
# Check patches: patches: - path: deployment-patch.yaml target: kind: Deployment name: myapp
# patchesStrategicMerge (deprecated): patchesStrategicMerge: - deployment-patch.yaml
# Use patches instead for Kustomize 4.x+ ```
Step 7: Check Replacements
```bash # Check replacements syntax: replacements: - source: kind: ConfigMap name: app-config fieldPath: data.APP_NAME targets: - select: kind: Deployment name: myapp fieldPaths: - spec.template.spec.containers.[name=app].env.[name=APP_NAME].value
# Common replacement errors: # 1. Wrong field path: fieldPath: data.APP_NAME # ConfigMap uses stringData?
# 2. Missing source resource: source: kind: ConfigMap name: missing-config # Doesn't exist
# 3. Wrong target path: fieldPaths: - spec.containers.env.value # Missing nested fields
# Test replacement: kustomize build ./overlay --enable-alpha-plugins
# For older Kustomize versions, use vars: vars: - name: APP_NAME objref: kind: ConfigMap name: app-config fieldref: fieldpath: data.APP_NAME ```
Step 8: Check ConfigMap/Secret Generators
```bash # Check configMapGenerator: configMapGenerator: - name: app-config files: - config.yaml literals: - DB_HOST=localhost - DB_PORT=5432
# Check referenced files: ls -la config.yaml
# Common errors: # 1. Missing file: files: - missing-config.yaml # File doesn't exist
# 2. Invalid literal format: literals: - INVALID CONFIG=value # No spaces in key
# Check secretGenerator: secretGenerator: - name: app-secret type: Opaque files: - password.txt literals: - username=admin
# Verify secret file: cat password.txt
# For env files: configMapGenerator: - name: env-config envs: - .env # Check .env file format ```
Step 9: Debug Build Output
```bash # Verbose output: kustomize build ./overlay --load-restrictor LoadRestrictionsNone
# Debug specific resource: kustomize build ./overlay 2>&1 | grep -A 10 "Error"
# Check intermediate output: kustomize cfg fmt ./base --dry-run
# Validate with Kubernetes: kustomize build ./overlay | kubectl apply --dry-run=client -f -
# Check for circular dependencies: # Resources that reference each other cause errors
# Trace resource loading: kustomize build ./overlay --trace
# Use kubectl kustomize for comparison: kubectl kustomize ./overlay
# Check order of transformations: # Later transformers override earlier ones ```
Step 10: Common Fixes
```bash # Fix 1: Update API version: # Old: apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization
# New (Kustomize 4.x+): apiVersion: kustomize.config.k8s.io/v1 kind: Kustomization
# Fix 2: Replace deprecated fields: # Old: patchesStrategicMerge: - patch.yaml
# New: patches: - path: patch.yaml
# Fix 3: Update generator syntax: # Old: configMapGenerator: - name: config data: # Not valid key: value
# New: configMapGenerator: - name: config literals: - key=value
# Fix 4: Remove null values: # Some fields can't be null commonLabels: app: myapp env: null # Remove or set value
# Fix 5: Escape special characters in paths: resources: - "file-with-dashes.yaml" # Quote if needed ```
Kustomize Build Error Checklist
| Check | Command | Expected |
|---|---|---|
| Kustomize version | kustomize version | Compatible |
| Directory structure | tree | Correct |
| kustomization.yaml | yamllint | Valid |
| Resource files | ls -la | All exist |
| Manifest syntax | kubectl dry-run | Valid |
| Transformer config | kustomize build | Success |
Verify the Fix
```bash # After fixing build errors
# 1. Run kustomize build kustomize build ./overlay/production // YAML output generated
# 2. Validate output kustomize build ./overlay/production | kubectl apply --dry-run=client -f - // No errors
# 3. Check specific resources kustomize build ./overlay/production | grep -A 20 "kind: Deployment" // Correct transformations applied
# 4. Test deployment kustomize build ./overlay/production | kubectl apply -f - // Resources created
# 5. Verify labels kubectl get deployment myapp -o jsonpath='{.metadata.labels}' // Labels from kustomization applied
# 6. Check with different overlay kustomize build ./overlay/staging // Staging overlay works too ```
Related Issues
- [Fix Helm Chart Template Render Failed](/articles/fix-helm-chart-template-render-failed)
- [Fix Kubernetes Deployment Not Progressing](/articles/fix-kubernetes-deployment-not-progressing)
- [Fix ArgoCD App Out Of Sync](/articles/fix-argocd-app-out-of-sync)