Fix K9s Connection Refused

What's Actually Happening

K9s terminal UI cannot connect to Kubernetes cluster. Connection is refused when trying to access cluster resources.

The Error You'll See

```bash $ k9s

Error: dial tcp 127.0.0.1:6443: connect: connection refused ```

Kubeconfig error:

Error: unable to load kubeconfig: invalid configuration

Auth error:

Error: You must be logged in to the server (Unauthorized)

Context error:

Error: context "my-context" does not exist

Why This Happens

1. 1.Kubeconfig missing - KUBECONFIG not set correctly
2. 2.Cluster unreachable - API server not accessible
3. 3.Wrong context - Selected context not configured
4. 4.VPN disconnected - Network requires VPN
5. 5.API server down - Control plane not running
6. 6.Certificate expired - Client or server cert invalid

Step 1: Check Kubeconfig

```bash # Check KUBECONFIG environment variable: echo $KUBECONFIG

# Default locations: ls -la ~/.kube/config ls -la ~/.kube/

# Set KUBECONFIG: export KUBECONFIG=~/.kube/config

# Multiple kubeconfig files: export KUBECONFIG=~/.kube/config:~/.kube/config-aws

# View current config: kubectl config view

# View raw config: cat ~/.kube/config

# Check config syntax: kubectl config view --raw

# Validate config: kubectl config get-clusters

# If config missing, create: mkdir -p ~/.kube # Copy from cluster or cloud provider: # EKS: aws eks update-kubeconfig --name my-cluster # GKE: gcloud container clusters get-credentials my-cluster # AKS: az aks get-credentials --resource-group rg --name my-cluster ```

Step 2: Check Current Context

```bash # Get current context: kubectl config current-context

# List all contexts: kubectl config get-contexts

# Switch context: kubectl config use-context my-context

# Set default namespace for context: kubectl config set-context --current --namespace=mynamespace

# View context details: kubectl config view -o jsonpath='{.contexts[?(@.name=="my-context")]}'

# Check cluster association: kubectl config get-clusters

# Context structure in kubeconfig: contexts: - context: cluster: my-cluster user: my-user namespace: default name: my-context

# Create context: kubectl config set-context my-context \ --cluster=my-cluster \ --user=my-user \ --namespace=default ```

Step 3: Test Cluster Connectivity

```bash # Test cluster access: kubectl cluster-info

# Check API server: kubectl get --raw /healthz

# Get cluster details: kubectl cluster-info dump

# Test with verbose: kubectl get nodes -v=6

# Check API server URL: kubectl config view -o jsonpath='{.clusters[0].cluster.server}'

# Test API server connectivity: curl -k https://<api-server>:6443/healthz

# Check if API server is running: # For local clusters (minikube, kind, k3s): minikube status kind get clusters k3s kubectl get nodes

# For cloud clusters: # Check cloud provider console

# Test DNS resolution: nslookup my-cluster.eks.amazonaws.com dig my-cluster.eks.amazonaws.com ```

Step 4: Check Authentication

```bash # Check user configuration: kubectl config view -o jsonpath='{.users}'

# View current user: kubectl config view -o jsonpath='{.contexts[?(@.name=="'$(kubectl config current-context)'")].context.user}'

# Check for tokens: kubectl config view -o jsonpath='{.users[?(@.name=="my-user")].user.token}'

# For cloud clusters, refresh credentials:

# AWS EKS: aws eks update-kubeconfig --name my-cluster

# GKE: gcloud container clusters get-credentials my-cluster --zone us-central1-a

# Azure AKS: az aks get-credentials --resource-group my-rg --name my-cluster

# Check certificate: kubectl config view --raw -o jsonpath='{.users[?(@.name=="my-user")].user.client-certificate-data}' | base64 -d | openssl x509 -text -noout

# Check for exec authentication: kubectl config view -o jsonpath='{.users[?(@.name=="my-user")].user.exec}'

# Test exec command manually: aws eks get-token --cluster-name my-cluster ```

Step 5: Fix Network Issues

```bash # Check if VPN needed: # Some clusters require VPN for access

# Test with curl: curl -k https://<api-server>:6443

# Check firewall: # Ensure port 6443 is allowed

# Test specific port: telnet <api-server> 6443 nc -zv <api-server> 6443

# Check proxy settings: echo $HTTP_PROXY echo $HTTPS_PROXY echo $NO_PROXY

# Add cluster to NO_PROXY: export NO_PROXY=$NO_PROXY,<api-server-ip>

# For corporate networks: # May need to add Kubernetes API to allowed hosts

# Check DNS: nslookup <api-server-hostname>

# Add to /etc/hosts if needed: echo "<ip> <hostname>" | sudo tee -a /etc/hosts ```

Step 6: Check for Local Clusters

```bash # Minikube: minikube status minikube start

# Minikube kubeconfig: minikube update-context

# Kind: kind get clusters kind create cluster --name my-cluster

# Kind kubeconfig: kind export kubeconfig --name my-cluster

# K3s: k3s kubectl get nodes

# K3s kubeconfig: export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

# Docker Desktop: # Check Docker Desktop Kubernetes is enabled # Settings -> Kubernetes -> Enable Kubernetes

# MicroK8s: microk8s status microk8s config > ~/.kube/config

# Check if cluster is running: docker ps | grep kube ```

Step 7: Handle Certificate Issues

```bash # Check certificate expiration: kubectl config view --raw -o jsonpath='{.users[?(@.name=="my-user")].user.client-certificate-data}' | base64 -d | openssl x509 -enddate -noout

# Check certificate authority: kubectl config view --raw -o jsonpath='{.clusters[?(@.name=="my-cluster")].cluster.certificate-authority-data}' | base64 -d | openssl x509 -text -noout | head -5

# Certificate expired errors: # Error: x509: certificate has expired or is not yet valid

# Refresh credentials: # EKS: aws eks update-kubeconfig --name my-cluster --force

# GKE: gcloud container clusters get-credentials my-cluster

# Self-signed certificate bypass (not recommended): kubectl config set-cluster my-cluster --insecure-skip-tls-verify=true

# Add CA certificate: kubectl config set-cluster my-cluster --certificate-authority=/path/to/ca.crt ```

Step 8: Use Correct K9s Options

```bash # Specify kubeconfig: k9s --kubeconfig ~/.kube/config-aws

# Specify context: k9s --context my-context

# Specify cluster: k9s --cluster my-cluster

# Read-only mode: k9s --readonly

# Log level: k9s --log-level debug

# Specify namespace: k9s -n mynamespace

# Command mode: k9s info k9s version

# Config file location: cat ~/.config/k9s/config.yaml

# Reset K9s config: rm ~/.config/k9s/config.yaml

# Set default namespace in config: k9s: namespace: active: default ```

Step 9: Debug K9s Connection

```bash # Run K9s with debug: k9s --log-level debug

# Check K9s logs: cat ~/.local/state/k9s/k9s.log

# View K9s config: cat ~/.config/k9s/config.yaml

# K9s stores config: ls -la ~/.config/k9s/ ls -la ~/.local/state/k9s/

# Clear K9s cache: rm -rf ~/.local/state/k9s/cache/

# Test connection manually: kubectl get nodes kubectl get pods -A

# Check K9s can access config: K9SCONFIG=~/.kube/config k9s

# Force refresh: k9s --refresh ```

Step 10: K9s Connection Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-k9s-connection.sh #!/bin/bash

echo "=== Kubeconfig Location ===" echo "KUBECONFIG: $KUBECONFIG" ls -la ~/.kube/config 2>/dev/null || echo "Default config not found"

echo "" echo "=== Current Context ===" kubectl config current-context 2>/dev/null || echo "Cannot get current context"

echo "" echo "=== Available Contexts ===" kubectl config get-contexts 2>/dev/null || echo "Cannot list contexts"

echo "" echo "=== Cluster Connectivity ===" kubectl cluster-info 2>&1 | head -5

echo "" echo "=== Node Status ===" kubectl get nodes 2>&1 | head -5

echo "" echo "=== Auth Check ===" kubectl auth whoami 2>&1 || kubectl auth can-i --list 2>&1 | head -10

echo "" echo "=== Certificate Expiration ===" kubectl config view --raw -o jsonpath='{.users[0].user.client-certificate-data}' 2>/dev/null | base64 -d 2>/dev/null | openssl x509 -enddate -noout 2>/dev/null || echo "No client cert"

echo "" echo "=== K9s Version ===" k9s version 2>/dev/null || echo "K9s not installed or not in PATH"

echo "" echo "=== K9s Config ===" cat ~/.config/k9s/config.yaml 2>/dev/null || echo "K9s config not found"

echo "" echo "=== Recommendations ===" echo "1. Ensure KUBECONFIG is set correctly" echo "2. Switch to correct context" echo "3. Check cluster is running and accessible" echo "4. Refresh cloud credentials if expired" echo "5. Check VPN connection if required" echo "6. Verify certificate has not expired" echo "7. Try k9s with specific kubeconfig" EOF

chmod +x /usr/local/bin/check-k9s-connection.sh

# Usage: /usr/local/bin/check-k9s-connection.sh ```

K9s Connection Checklist

Verify the Fix

```bash # After fixing K9s connection issues

# 1. Test kubectl kubectl get nodes // Returns node list

# 2. Check context kubectl config current-context // Correct context

# 3. Start K9s k9s // Connects successfully

# 4. Navigate cluster # Use arrow keys and commands // Can view resources

# 5. Check logs cat ~/.local/state/k9s/k9s.log | tail // No connection errors

# 6. Switch contexts in K9s :ctx // Can switch contexts ```

Related Issues

• [Fix Kubernetes Deployment Not Working](/articles/fix-kubernetes-deployment-not-working)
• [Fix Kubernetes Authentication Failed](/articles/fix-kubernetes-authentication-failed)
• [Fix Kubectl Connection Refused](/articles/fix-kubectl-connection-refused)