Fix Jenkins Agent Node Offline

What's Actually Happening

Jenkins agent nodes show as offline in the master. Jobs cannot run on the agent, builds queue indefinitely.

The Error You'll See

Node 'agent-1' is offline

Jenkins UI:

Build Queue: Waiting for executor
Nodes: agent-1 (offline)

Agent log:

```bash $ cat /var/log/jenkins/jenkins-agent.log

ERROR: Connection refused to master:50000 ```

Connection timeout:

Agent connection failed: java.net.ConnectException: Connection timed out

Why This Happens

1. 1.Network connectivity - Agent cannot reach master
2. 2.Firewall blocking - Port 50000 blocked
3. 3.Invalid credentials - Wrong agent secret
4. 4.JNLP issues - Java Web Start connection problems
5. 5.Master overloaded - Master cannot accept connections
6. 6.Agent service down - Agent process not running

Step 1: Check Agent Status in Jenkins

```bash # In Jenkins UI: # Manage Jenkins -> Manage Nodes -> <node-name>

# Check node details: # - Status: offline/online # - Offline cause reason # - Response time

# Via CLI: java -jar jenkins-cli.jar -s http://jenkins:8080/ -auth user:token node-provision --stats

# Via API: curl -u user:token http://jenkins:8080/computer/api/json?pretty=true

# Check specific node: curl -u user:token http://jenkins:8080/computer/agent-1/api/json?pretty=true

# Check offline cause: curl -u user:token http://jenkins:8080/computer/agent-1/api/json | jq '.offlineCause' ```

Step 2: Check Agent Process on Node

```bash # SSH to agent node:

# Check agent process: ps aux | grep jenkins ps aux | grep agent

# Check if agent jar running: pgrep -f agent.jar

# Check agent service: systemctl status jenkins-agent

# Check agent log: tail -f /var/log/jenkins/agent.log

# Start agent manually: java -jar agent.jar -jnlpUrl http://jenkins:8080/computer/agent-1/slave-agent.jnlp -secret <secret> -workDir /home/jenkins

# Check if work directory exists: ls -la /home/jenkins/ ```

Step 3: Verify Network Connectivity

```bash # From agent to master:

# Test TCP connectivity to port 50000: telnet jenkins-master 50000 nc -zv jenkins-master 50000

# Test HTTP: curl -I http://jenkins:8080

# Test HTTPS: curl -k https://jenkins:8443

# Check routing: traceroute jenkins-master

# Check DNS: dig jenkins-master nslookup jenkins-master

# Check firewall on agent: iptables -L -n ufw status

# Check firewall on master: # Ensure port 50000 allowed

# Test from master to agent: # SSH to master: telnet agent-node 22 ```

Step 4: Check Agent Credentials

```bash # Get agent secret: # In Jenkins UI: # Manage Jenkins -> Manage Nodes -> <node> -> Configure # Show "Secret" field

# Or via API: curl -u user:token http://jenkins:8080/computer/agent-1/slave-agent.jnlp

# Extract secret from JNLP: curl -u user:token http://jenkins:8080/computer/agent-1/slave-agent.jnlp | grep -oP 'secret=[^"]+'

# Regenerate secret if needed: # Delete node and recreate

# Verify credentials used: # In agent launch command, ensure secret matches

# Test connection with secret: java -jar agent.jar -jnlpUrl http://jenkins:8080/computer/agent-1/slave-agent.jnlp -secret YOUR_SECRET -workDir /home/jenkins ```

Step 5: Check Master Configuration

```bash # On Jenkins master:

# Check Jenkins URL: # Manage Jenkins -> Configure System -> Jenkins URL # Must be reachable from agents

# Check TCP port: # Manage Jenkins -> Configure Global Security -> TCP port for inbound agents # Default: 50000 # Fixed port recommended

# Check protocols enabled: # Manage Jenkins -> Configure Global Security # Enable: JNLP agent protocols

# Check agent protocols: # In Jenkins UI or via script console: Jenkins.instance.agentProtocols.each { println it }

# Check security realm: # Ensure agents can authenticate

# Check logs: tail -f /var/log/jenkins/jenkins.log | grep -i agent ```

Step 6: Fix Firewall Issues

```bash # On Jenkins master (allow inbound):

# Linux (ufw): ufw allow 50000/tcp ufw reload

# Linux (iptables): iptables -A INPUT -p tcp --dport 50000 -j ACCEPT

# Linux (firewalld): firewall-cmd --add-port=50000/tcp --permanent firewall-cmd --reload

# On agent (allow outbound): iptables -A OUTPUT -p tcp --dport 50000 -j ACCEPT

# Test connectivity: # From agent: nc -zv jenkins-master 50000

# Check if port listening on master: netstat -tlnp | grep 50000 ss -tlnp | grep 50000

# On master: lsof -i :50000 ```

Step 7: Configure SSH Agent (Alternative)

```bash # SSH agent configuration:

# In Jenkins UI: # Manage Jenkins -> Manage Nodes -> New Node # Type: Permanent Agent # Launch method: Launch agents via SSH

# Configure: # - Host: agent-hostname # - Credentials: SSH key # - Host Key Verification Strategy: Known hosts file

# Ensure SSH works: ssh jenkins@agent-hostname

# Test SSH connection: ssh -v jenkins@agent-hostname

# Check SSH key permissions: chmod 600 ~/.ssh/id_rsa chmod 644 ~/.ssh/id_rsa.pub

# Add SSH credentials in Jenkins: # Credentials -> System -> Global credentials -> Add Credentials # Kind: SSH Username with private key ```

Step 8: Restart Agent Connection

```bash # Via Jenkins UI: # Manage Jenkins -> Manage Nodes -> <node> # Click "Bring this node online" # Or "Disconnect" then "Bring online"

# Via CLI: java -jar jenkins-cli.jar -s http://jenkins:8080/ -auth user:token disconnect-node agent-1 java -jar jenkins-cli.jar -s http://jenkins:8080/ -auth user:token connect-node agent-1

# Via API: curl -X POST -u user:token http://jenkins:8080/computer/agent-1/toggleOffline?offlineMessage=restart

# Restart agent service: systemctl restart jenkins-agent

# Or kill and restart agent: pkill -f agent.jar java -jar agent.jar -jnlpUrl ... &

# Force reconnection: # Delete node and recreate with same name ```

Step 9: Check Resource Limits

```bash # Check agent resources:

# CPU: top -bn1 | head -20

# Memory: free -m

# Disk: df -h

# Check work directory space: df -h /home/jenkins

# Check if agent has enough resources: # Executor count set appropriately

# On master, check if too many agents: # Manage Jenkins -> Manage Nodes # Check master executors not overwhelmed

# Check thread count: cat /proc/$(pgrep java)/status | grep Threads

# Increase agent heap: # In agent launch: java -Xmx512m -jar agent.jar ... ```

Step 10: Jenkins Agent Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-jenkins-agent.sh #!/bin/bash

AGENT=$1 JENKINS_URL=${2:-"http://localhost:8080"}

echo "=== Agent Process ===" ps aux | grep -i jenkins-agent || echo "Agent process not running"

echo "" echo "=== Agent Service ===" systemctl status jenkins-agent --no-pager 2>/dev/null || echo "Service not configured"

echo "" echo "=== Network Connectivity ===" echo "Testing port 50000:" nc -zv $JENKINS_URL_HOST 50000 2>&1 || echo "Port 50000 not reachable" echo "Testing HTTP:" curl -s -o /dev/null -w "%{http_code}" $JENKINS_URL && echo " - HTTP OK" || echo "HTTP Failed"

echo "" echo "=== Agent Log (last 20) ===" if [ -f /var/log/jenkins/agent.log ]; then tail -20 /var/log/jenkins/agent.log elif [ -f /var/log/jenkins/jenkins-agent.log ]; then tail -20 /var/log/jenkins/jenkins-agent.log else echo "No agent log found" fi

echo "" echo "=== Work Directory ===" ls -la /home/jenkins/ 2>/dev/null || ls -la /var/lib/jenkins/ 2>/dev/null || echo "Work directory not found"

echo "" echo "=== Disk Space ===" df -h /home/jenkins 2>/dev/null || df -h /var/lib/jenkins 2>/dev/null

echo "" echo "=== Firewall Status ===" ufw status 2>/dev/null || iptables -L -n 2>/dev/null | head -10

echo "" echo "=== Recommendations ===" echo "1. Verify agent process running" echo "2. Check network connectivity to master:50000" echo "3. Verify agent secret matches" echo "4. Check firewall allows port 50000" echo "5. Review agent logs for errors" echo "6. Restart agent service if needed" EOF

chmod +x /usr/local/bin/check-jenkins-agent.sh

# Usage: /usr/local/bin/check-jenkins-agent.sh agent-1 http://jenkins:8080 ```

Jenkins Agent Checklist

Verify the Fix

```bash # After fixing agent connectivity

# 1. Check node status in Jenkins UI // Node shows "online"

# 2. Test connection from agent nc -zv jenkins-master 50000 // Connected

# 3. Run test job on agent // Job executes successfully

# 4. Check agent logs tail /var/log/jenkins/agent.log // Connected to master

# 5. Monitor agent in Jenkins # Build history shows agent used // Builds running

# 6. Check stability # Wait 30 minutes // Agent stays online ```

Related Issues

• [Fix Jenkins Build Stuck](/articles/fix-jenkins-build-stuck)
• [Fix GitLab Runner Not Picking Jobs](/articles/fix-gitlab-runner-not-picking-jobs)
• [Fix SonarQube Analysis Not Running](/articles/fix-sonarqube-analysis-not-running)