Fix GCP GKE Cluster Authentication Failed

What's Actually Happening

kubectl cannot authenticate with GKE cluster. Access to cluster is denied despite valid gcloud login.

The Error You'll See

```bash $ kubectl get pods

Unable to connect to the server: getting credentials: exec: executable gcloud not found ```

Authentication error:

Error from server (Forbidden): pods is forbidden: User "user@example.com" cannot list resource "pods" in API group ""

Token error:

```bash $ kubectl get nodes

The gcloud credential plugin does not support the provided API version ```

Config error:

Unable to connect to the server: getting credentials: exec: exit status 1

Why This Happens

1. 1.gcloud not installed - gcloud CLI missing or not in PATH
2. 2.Not logged in - gcloud auth login not completed
3. 3.Wrong project - gcloud configured for different project
4. 4.IAM permissions missing - User lacks GKE access permissions
5. 5.kubeconfig outdated - Credentials expired or cluster config stale
6. 6.Plugin not installed - gke-gcloud-auth-plugin missing

Step 1: Check gcloud Installation

```bash # Check gcloud installed: which gcloud gcloud --version

# Expected output: # Google Cloud SDK 400.0.0 # bq 2.0.85 # core 2023... # gke-gcloud-auth-plugin

# Install gcloud if missing: # Linux: curl https://sdk.cloud.google.com | bash exec -l $SHELL

# macOS: brew install google-cloud-sdk

# Or download from: # https://cloud.google.com/sdk/docs/install

# Install components: gcloud components install gke-gcloud-auth-plugin gcloud components install kubectl

# Update gcloud: gcloud components update

# Verify auth plugin: gke-gcloud-auth-plugin --version ```

Step 2: Check gcloud Authentication

```bash # Check current account: gcloud auth list

# Shows: # Credentialed Accounts: # - user@example.com (active)

# If no account, login: gcloud auth login

# For service account: gcloud auth activate-service-account --key-file=key.json

# Check current project: gcloud config get-value project

# Set correct project: gcloud config set project my-project

# Check authenticated status: gcloud auth print-access-token

# Verify token is valid: curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \ https://container.googleapis.com/v1/projects/my-project/locations/-/clusters

# Re-authenticate if token expired: gcloud auth login --force ```

Step 3: Get Cluster Credentials

```bash # Get cluster credentials: gcloud container clusters get-credentials CLUSTER_NAME \ --region REGION \ --project PROJECT_ID

# Example: gcloud container clusters get-credentials my-cluster \ --region us-central1 \ --project my-project

# Check kubeconfig: kubectl config current-context

# View kubeconfig: kubectl config view

# Check cluster in kubeconfig: kubectl config get-clusters

# Check contexts: kubectl config get-contexts

# If kubeconfig corrupted, regenerate: gcloud container clusters get-credentials my-cluster --region us-central1 ```

Step 4: Check IAM Permissions

```bash # Check user IAM policy: gcloud projects get-iam-policy my-project \ --flatten="bindings[].members" \ --filter="bindings.members:user:user@example.com"

# Required roles for GKE: # - roles/container.clusterViewer (minimum) # - roles/container.viewer # - roles/container.developer # - roles/container.admin

# Grant GKE viewer role: gcloud projects add-iam-policy-binding my-project \ --member="user:user@example.com" \ --role="roles/container.viewer"

# Grant developer role (for deploying): gcloud projects add-iam-policy-binding my-project \ --member="user:user@example.com" \ --role="roles/container.developer"

# Check cluster-specific IAM: gcloud container clusters describe my-cluster --region us-central1 | grep -A10 "masterAuthorizedNetworksConfig"

# If using master authorized networks, add IP: gcloud container clusters update my-cluster \ --region us-central1 \ --enable-master-authorized-networks \ --master-authorized-networks YOUR_IP/32 ```

Step 5: Check kubeconfig Configuration

```bash # View kubeconfig: kubectl config view --raw

# Check current context: kubectl config current-context

# Expected format: # gke_PROJECT_REGION_CLUSTER

# Check user in kubeconfig: kubectl config view -o jsonpath='{.users[].name}'

# Check auth provider: kubectl config view -o jsonpath='{.users[].user.auth-provider}'

# Should show: # name: gcp # config: # cmd-args: config config-helper --output=json # cmd-path: /path/to/gcloud

# If using exec: # user: # exec: # apiVersion: client.authentication.k8s.io/v1beta1 # command: gke-gcloud-auth-plugin # installHint: Install gke-gcloud-auth-plugin

# Regenerate config: gcloud container clusters get-credentials my-cluster --region us-central1 ```

Step 6: Fix Auth Plugin Issues

```bash # Check auth plugin: gke-gcloud-auth-plugin --version

# Install if missing: gcloud components install gke-gcloud-auth-plugin

# Check kubeconfig uses exec: kubectl config view | grep -A10 "exec:"

# Should see: # exec: # apiVersion: client.authentication.k8s.io/v1beta1 # command: gke-gcloud-auth-plugin # provideClusterInfo: true

# Update kubeconfig to use exec: gcloud container clusters get-credentials my-cluster --region us-central1

# Test auth plugin: gke-gcloud-auth-plugin

# Set environment for plugin: export USE_GKE_GCLOUD_AUTH_PLUGIN=True

# Verify: kubectl version --client ```

Step 7: Check Cluster Access

```bash # Test cluster connectivity: kubectl cluster-info

# Should show: # Kubernetes control plane is running at https://IP # GLBCDefaultBackend is running at https://IP

# Test with curl: CLUSTER_URL="https://$(gcloud container clusters describe my-cluster --region us-central1 --format='value(endpoint)')"

curl -k -H "Authorization: Bearer $(gcloud auth print-access-token)" \ $CLUSTER_URL/api

# Should return Kubernetes API response

# Check cluster endpoint: gcloud container clusters describe my-cluster \ --region us-central1 \ --format='value(endpoint)'

# Check cluster status: gcloud container clusters describe my-cluster \ --region us-central1 \ --format='value(status)'

# Should be: RUNNING ```

Step 8: Check Network Access

```bash # Check master authorized networks: gcloud container clusters describe my-cluster \ --region us-central1 \ --format='yaml(masterAuthorizedNetworksConfig)'

# If enabled, your IP must be in the list: # Get your IP: curl -s https://api.ipify.org

# Add your IP: gcloud container clusters update my-cluster \ --region us-central1 \ --enable-master-authorized-networks \ --master-authorized-networks YOUR_IP/32

# Or disable (not recommended for production): gcloud container clusters update my-cluster \ --region us-central1 \ --no-enable-master-authorized-networks

# Check private cluster: gcloud container clusters describe my-cluster \ --region us-central1 \ --format='value(privateClusterConfig.enablePrivateNodes)'

# If private cluster, need VPN/Cloud Interconnect ```

Step 9: Use Service Account Authentication

```bash # Create service account: gcloud iam service-accounts create gke-sa \ --display-name="GKE Service Account"

# Grant GKE role: gcloud projects add-iam-policy-binding my-project \ --member="serviceAccount:gke-sa@my-project.iam.gserviceaccount.com" \ --role="roles/container.developer"

# Create key: gcloud iam service-accounts keys create key.json \ --iam-account=gke-sa@my-project.iam.gserviceaccount.com

# Activate service account: gcloud auth activate-service-account \ --key-file=key.json

# Get credentials: gcloud container clusters get-credentials my-cluster \ --region us-central1

# Test: kubectl get pods

# Set GOOGLE_APPLICATION_CREDENTIALS: export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json ```

Step 10: GCP GKE Auth Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-gke-auth.sh #!/bin/bash

CLUSTER=$1 REGION=${2:-"us-central1"} PROJECT=$(gcloud config get-value project 2>/dev/null)

echo "=== gcloud Configuration ===" echo "Project: $PROJECT" gcloud config list

echo "" echo "=== Authenticated Accounts ===" gcloud auth list

echo "" echo "=== gcloud Version ===" gcloud version | head -5

echo "" echo "=== Auth Plugin ===" which gke-gcloud-auth-plugin 2>/dev/null && gke-gcloud-auth-plugin --version || echo "Not installed"

echo "" echo "=== Current kubectl Context ===" kubectl config current-context

if [ -n "$CLUSTER" ]; then echo "" echo "=== Cluster: $CLUSTER Status ===" gcloud container clusters describe $CLUSTER --region $REGION --format='yaml(status,endpoint,masterVersion)'

echo "" echo "=== Get Credentials ===" echo "Run: gcloud container clusters get-credentials $CLUSTER --region $REGION" fi

echo "" echo "=== IAM Policy (current user) ===" ACCOUNT=$(gcloud auth list --filter=status:ACTIVE --format="value(account)") gcloud projects get-iam-policy $PROJECT --flatten="bindings[].members" --filter="bindings.members:user:$ACCOUNT" --format="table(bindings.role)"

echo "" echo "=== Test Cluster Access ===" kubectl cluster-info 2>&1 | head -5

echo "" echo "=== Recommendations ===" if ! which gke-gcloud-auth-plugin &>/dev/null; then echo "Install auth plugin: gcloud components install gke-gcloud-auth-plugin" fi if ! kubectl config current-context &>/dev/null; then echo "Get cluster credentials: gcloud container clusters get-credentials CLUSTER --region REGION" fi EOF

chmod +x /usr/local/bin/check-gke-auth.sh

# Usage: /usr/local/bin/check-gke-auth.sh my-cluster us-central1

# Quick test: alias gke-auth='gcloud auth list && kubectl config current-context' ```

GCP GKE Auth Checklist

Verify the Fix

```bash # After fixing GKE authentication

# 1. Check gcloud auth gcloud auth list // Your account listed as active

# 2. Check project gcloud config get-value project // Correct project

# 3. Get cluster credentials gcloud container clusters get-credentials my-cluster --region us-central1 // Credentials updated

# 4. Check context kubectl config current-context // gke_PROJECT_REGION_CLUSTER

# 5. Test cluster access kubectl cluster-info // Kubernetes control plane running

# 6. List resources kubectl get nodes // Nodes listed successfully ```

Related Issues

• [Fix GCP GKE Cluster Unreachable](/articles/fix-gcp-gke-cluster-unreachable)
• [Fix Kubernetes Authentication Failed](/articles/fix-kubernetes-authentication-failed)
• [Fix GCP Permission Denied](/articles/fix-gcp-permission-denied)