Jenkins Agent Offline: Diagnosis and Resolution

# Jenkins Agent Offline: Diagnosis and Resolution

Your builds are queued, nothing is running, and the agent status shows a red "Offline" indicator. This is one of the most frustrating Jenkins issues because it can have many root causes.

Let me walk through systematic diagnosis and fixes for offline agents.

Quick Diagnosis: Check the Agent Status

First, understand why the agent is offline:

1. 1.Navigate to Manage Jenkins → Manage Nodes and Clouds
2. 2.Click on the offline agent
3. 3.Look at the status message next to the red indicator

The status message tells you a lot:

Fix 1: SSH Agent Connection Issues

For SSH-based agents, you'll see errors like:

Connection refused (Connection refused)

Or:

Permission denied (publickey,password)

Diagnosis steps:

```bash # 1. Can Jenkins reach the agent machine? ping agent-hostname

# 2. Is SSH running on the agent? ssh user@agent-hostname "echo SSH working"

# 3. Is the SSH port correct? (Default 22) telnet agent-hostname 22

# 4. Can the Jenkins user SSH without password? sudo -u jenkins ssh user@agent-hostname ```

Solution for permission denied:

1. 1.Generate an SSH key on the Jenkins master:

sudo -u jenkins ssh-keygen -t rsa -b 4096 -C "jenkins@master"

1. 1.Copy the public key to the agent:

sudo -u jenkins ssh-copy-id user@agent-hostname

1. 1.In Jenkins, go to Manage Jenkins → Manage Nodes and Clouds → [Agent Name] → Configure
2. 2.Under "Launch method", select "Launch agents via SSH"
3. 3.Set "Host" to the agent hostname/IP
4. 4.Add credentials: choose "SSH Username with private key"
5. 5.Paste the private key content from /var/lib/jenkins/.ssh/id_rsa

Solution for connection refused:

```bash # Check if SSH is running on the agent systemctl status sshd # RHEL/CentOS systemctl status ssh # Ubuntu/Debian

# If not running, start it sudo systemctl start sshd sudo systemctl enable sshd

# Check firewall sudo firewall-cmd --list-all # RHEL/CentOS sudo ufw status # Ubuntu ```

Open SSH port if needed:

```bash # RHEL/CentOS sudo firewall-cmd --add-service=ssh --permanent sudo firewall-cmd --reload

# Ubuntu sudo ufw allow ssh ```

Fix 2: JNLP Agent Connection Issues

JNLP (Java Web Start) agents connect outbound from the agent to Jenkins. You might see:

Agent didn't connect within the specified time

Or on the agent machine:

java.net.ConnectException: Connection refused

Diagnosis on the agent machine:

```bash # Can the agent reach Jenkins? curl http://jenkins-server:8080

# Is the TCP port for agent connections open? telnet jenkins-server 50000 ```

Solution:

1. 1.Ensure Jenkins TCP port is configured:
2. 2.- Go to Manage Jenkins → Configure Global Security
3. 3.- Under "Agents", set "TCP port for inbound agents" to "Fixed: 50000"
4. 4.Open firewall on Jenkins server:

# On Jenkins server
sudo firewall-cmd --add-port=50000/tcp --permanent
sudo firewall-cmd --reload

1. 1.Start the JNLP agent:

```bash # Download agent jar curl -o agent.jar http://jenkins-server:8080/jnlpJars/agent.jar

# Run with agent secret java -jar agent.jar -jnlpUrl http://jenkins-server:8080/computer/agent-name/slave-agent.jnlp -secret YOUR_SECRET_KEY -workDir /home/jenkins/agent-work ```

Fix 3: Agent Process Crashed

The agent was working, then suddenly went offline. The Jenkins logs show:

Agent agent-name was disconnected
Remoting connection to /192.168.1.100:12345 terminated

Diagnosis:

Check if the agent process is running:

```bash # On the agent machine ps aux | grep agent.jar

# Check for out of memory dmesg | grep -i "killed process"

# Check system logs journalctl -u jenkins-agent -n 100 # If running as service ```

Solution for memory issues:

The agent JVM might be running out of memory. Increase heap size:

# When starting the agent
java -Xmx1024m -Xms256m -jar agent.jar -jnlpUrl ...

Or in the agent startup script:

export JAVA_OPTS="-Xmx1024m -Xms256m"
java $JAVA_OPTS -jar agent.jar ...

Solution for agent service:

If the agent runs as a systemd service:

```bash # Check service status systemctl status jenkins-agent

# Restart if crashed systemctl restart jenkins-agent

# Check logs journalctl -u jenkins-agent -f ```

Fix 4: Agent Configuration Issues

Sometimes the agent is "connected" but Jenkins marks it offline due to configuration:

Offline: Label expression doesn't match

Or:

Offline: Agent computer is offline because it is disabled

Solution for disabled agent:

1. 1.Go to Manage Jenkins → Manage Nodes and Clouds
2. 2.Click the agent name
3. 3.Click "Configure"
4. 4.Uncheck "Mark this node temporarily offline"

Solution for label issues:

Jobs require specific labels that the agent doesn't have:

1. 1.Check job configuration → "Restrict where this project can be run"
2. 2.Note the label expression
3. 3.Go to agent configuration
4. 4.Add matching labels in the "Labels" field

Example: If job requires linux && docker: - Go to agent configuration - Add labels: linux docker - Save and reconnect

Fix 5: WebSocket Connection Issues (Jenkins 2.217+)

Newer Jenkins versions can use WebSocket for agent connections:

WebSocket connection failed

Solution:

1. 1.Go to Manage Jenkins → Configure Global Security
2. 2.Under "Agents", set "Protocol" to "WebSocket" (or both "TCP" and "WebSocket")
3. 3.If using a reverse proxy, ensure WebSocket support:

# Nginx configuration
location /ws/ {
    proxy_pass http://jenkins:8080;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

Fix 6: Cloud Agent Issues (Kubernetes, EC2, etc.)

If you're using cloud agents, the issue might be provisioning:

Waiting for agent to be provisioned

For Kubernetes plugin:

```bash # Check if pods can be created kubectl get pods -n jenkins

# Check events kubectl describe pod jenkins-agent-xxx -n jenkins

# Check Jenkins logs kubectl logs jenkins-pod -n jenkins | grep -i kubernetes ```

Common issues: - Insufficient cluster resources - Service account permissions - Image pull errors

Solution: Verify the Kubernetes cloud configuration:

1. 1.Go to Manage Jenkins → Manage Nodes and Clouds → Configure Clouds
2. 2.Check Kubernetes URL is correct
3. 3.Verify credentials (service account token or kubeconfig)
4. 4.Test connection with "Test Connection" button
5. 5.Check pod template has correct image and resources

Monitoring Agent Health

Prevent future outages with proactive monitoring:

// Add this to your Jenkinsfile
pipeline {
    agent any
    stages {
        stage('Check Agent') {
            steps {
                echo "Running on agent: ${env.NODE_NAME}"
                sh 'df -h /'  // Check disk
                sh 'free -m'  // Check memory
            }
        }
    }
}

Configure agent health checks:

1. 1.Go to agent configuration
2. 2.Under "Availability", select "Keep this agent online as much as possible"
3. 3.Set "In-demand delay" and "Idle delay" as needed
4. 4.Add "Retention strategy" for dynamic agents

Quick Reference: Agent Offline Troubleshooting