Introduction
CI/CD pipeline timeout errors occur when build, test, or deployment jobs exceed configured timeout limits, causing abrupt termination with errors like "The operation was canceled," "Job exceeded timeout," or "Build timeout." These failures waste compute resources, block deployments, and often indicate deeper issues with pipeline efficiency or resource constraints. Common causes include inefficient test suites running sequentially, slow dependency installation (npm install, pip install, mvn dependencies), resource-constrained runners (CPU/memory throttling), large artifact downloads/uploads, Docker image builds without layer caching, database migrations on large datasets, waiting for external services (environment spin-up, approvals), queue time counting toward job timeout, network latency for remote resources, and runaway processes (infinite loops, deadlocks). The fix requires identifying the bottleneck (tests, dependencies, builds), implementing caching strategies, parallelizing independent jobs, increasing timeout limits appropriately, and optimizing runner resources. This guide provides production-proven troubleshooting for CI/CD timeouts across GitHub Actions, GitLab CI, Jenkins, Azure DevOps, and CircleCI.
Symptoms
- "The operation was canceled" (GitHub Actions)
- "Job exceeded timeout" (GitLab CI)
- "Build timeout" (Jenkins)
- "Pipeline exceeded timeout" (Azure DevOps)
- Job killed at exact timeout threshold
- Tests passing locally but timing out in CI
- Intermittent timeouts during peak hours (runner contention)
- Specific steps consistently timeout (dependency install, build)
- Queue time + execution time exceeds total timeout
- Docker pull/build timeout on large images
Common Causes
- Sequential test execution when parallelization possible
- No dependency caching (npm, pip, Maven, Gradle downloading every run)
- Underpowered runner (shared runners with CPU throttling)
- Large monorepo builds without path filtering
- Docker builds without layer caching or multi-stage optimization
- Database test fixtures taking too long to populate
- Waiting for external services (environment provisioning, approvals)
- Network slowness downloading artifacts or dependencies
- Memory pressure causing swap/GC thrashing
- Timeout configured too low for actual workload
Step-by-Step Fix
### 1. Diagnose pipeline bottleneck
Identify slow steps:
```yaml # GitHub Actions - Add timing to each step name: Build on: push
jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4
# Add timing step - name: Record start time run: echo "START_TIME=$(date +%s)" >> $GITHUB_ENV
- name: Install dependencies
- run: npm ci
- # GitHub Actions shows step duration automatically
- name: Build
- run: npm run build
- name: Test
- run: npm test
# Calculate duration - name: Calculate duration run: | END_TIME=$(date +%s) DURATION=$((END_TIME - ${{ env.START_TIME }})) echo "Total job duration: ${DURATION} seconds" ```
GitHub Actions timing analysis:
```yaml # Use action timing reports name: Build with Timing on: push
jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4
# Install dependencies with timing - name: Install dependencies run: | time npm ci # Output shows real/user/sys time
# Use timing action - uses: benchmark-action/github-action-benchmark@v1 with: name: Build Performance tool: 'customSmallerIsBetter' output-file-path: benchmark-output.txt
# Generate step timing report - name: Step timing if: always() run: | echo "## Step Timing Report" >> $GITHUB_STEP_SUMMARY echo "| Step | Duration |" >> $GITHUB_STEP_SUMMARY echo "|------|----------|" >> $GITHUB_STEP_SUMMARY ```
GitLab CI timing:
```yaml # .gitlab-ci.yml - Enable job logs with timestamps stages: - install - build - test
# Show timestamps in job logs # Admin setting: Admin > Settings > CI/CD > Log timestamps
variables: GIT_DEPTH: "10" # Shallow clone for faster checkout
install_deps: stage: install script: - echo "Starting at $(date)" - time npm ci - echo "Finished at $(date)" artifacts: paths: - node_modules/ expire_in: 1 hour # Don't store too long
build: stage: build script: - time npm run build needs: - install_deps artifacts: paths: - dist/
test: stage: test script: - echo "Test suite starting at $(date)" - time npm test needs: - build ```
Jenkins pipeline timing:
```groovy // Jenkinsfile with timing pipeline { agent any
options { timestamps() // Add timestamps to console output timeout(time: 60, unit: 'MINUTES') // Overall timeout }
stages { stage('Checkout') { steps { script { def start = System.currentTimeMillis() checkout scm def duration = (System.currentTimeMillis() - start) / 1000 echo "Checkout took ${duration} seconds" } } }
stage('Install Dependencies') { steps { script { def start = System.currentTimeMillis() sh 'npm ci' def duration = (System.currentTimeMillis() - start) / 1000 echo "Install took ${duration} seconds" } } }
stage('Build') { steps { timeout(time: 15, unit: 'MINUTES') { // Step-level timeout sh 'npm run build' } } }
stage('Test') { steps { script { def start = System.currentTimeMillis() sh 'npm test' def duration = (System.currentTimeMillis() - start) / 1000 echo "Tests took ${duration} seconds" } } } }
post { always { // Print timing summary script { echo "=== Pipeline Timing Summary ===" } } } } ```
### 2. Increase timeout configuration
GitHub Actions timeout settings:
```yaml # GitHub Actions - Job-level timeout name: Build on: push
jobs: build: runs-on: ubuntu-latest timeout-minutes: 60 # Default is 360 minutes for GitHub-hosted
steps: - uses: actions/checkout@v4
# Step-level timeout - name: Long running step timeout-minutes: 15 # Override job timeout for this step run: ./long-running-script.sh
# Default timeout for other steps - name: Quick step run: npm test ```
GitLab CI timeout settings:
```yaml # .gitlab-ci.yml
# Global timeout (applies to all jobs) default: timeout: 1h # Default is 1 hour
# Job-specific timeout slow_job: script: - ./long-running-script.sh timeout: 2h # Override global timeout
# Timeout with custom error message build: script: - npm run build timeout: 30m allow_failure: true # Don't fail pipeline on timeout
# Timeout based on resource group deploy: script: - ./deploy.sh timeout: 1h resource_group: production # Shared resource timeout ```
Jenkins timeout configuration:
```groovy // Jenkinsfile - Timeout options pipeline { agent any
options { // Overall pipeline timeout timeout(time: 2, unit: 'HOURS')
// Or use activity timeout (fails if no output for X time) // timeout(activity: true, time: 30, unit: 'MINUTES') }
stages { stage('Build') { options { // Stage-level timeout timeout(time: 30, unit: 'MINUTES') } steps { sh 'npm run build' } }
stage('Test') { options { // Activity timeout - fails if no console output timeout(activity: true, time: 10, unit: 'MINUTES') } steps { sh 'npm test' } } } }
// Timeout with retry stage('Flaky Integration Test') { steps { retry(3) { timeout(time: 15, unit: 'MINUTES') { sh 'npm run integration-test' } } } } ```
Azure DevOps timeout:
```yaml # azure-pipelines.yml
jobs: - job: Build # Job timeout (default is 60 minutes) timeoutInMinutes: 120
steps: - script: npm ci displayName: 'Install Dependencies' timeoutInMinutes: 30 # Step-level timeout
- script: npm test
- displayName: 'Run Tests'
- timeoutInMinutes: 60
# Pool-specific timeout jobs: - job: Integration pool: name: Self-Hosted # Timeout for self-hosted agents timeoutInMinutes: 180 ```
### 3. Implement dependency caching
GitHub Actions caching:
```yaml # GitHub Actions - Cache dependencies name: Build on: push
jobs: build: runs-on: ubuntu-latest
steps: - uses: actions/checkout@v4
# Node.js caching - name: Cache node modules uses: actions/cache@v4 with: path: | node_modules ~/.npm key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-node-
# Python caching - name: Cache Python packages uses: actions/cache@v4 with: path: | ~/.cache/pip __pycache__ key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip-
# Maven caching - name: Cache Maven packages uses: actions/cache@v4 with: path: ~/.m2 key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven-
# Gradle caching - name: Cache Gradle packages uses: actions/cache@v4 with: path: | ~/.gradle/caches ~/.gradle/wrapper key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }} restore-keys: | ${{ runner.os }}-gradle-
- name: Install dependencies
- run: npm ci
- # With cache, this is much faster
- name: Build
- run: npm run build
`
GitLab CI caching:
```yaml # .gitlab-ci.yml - Global cache configuration default: cache: key: ${CI_COMMIT_REF_SLUG} paths: - node_modules/ - .npm/ policy: pull-push # pull for merge requests, push for branches
# Cache per dependency file cache-npm: script: - npm ci cache: key: files: - package-lock.json # Cache key based on lock file paths: - node_modules/ policy: pull-push
# Multiple caches build: script: - npm ci - pip install -r requirements.txt cache: - key: npm-${{ hashFiles('package-lock.json') }} paths: - node_modules/ - key: pip-${{ hashFiles('requirements.txt') }} paths: - .cache/pip/ policy: pull ```
Jenkins caching:
```groovy // Jenkinsfile - Directory caching pipeline { agent any
options { // Use workspace cleanup plugin disableConcurrentBuilds() }
stages { stage('Checkout') { steps { checkout scm } }
stage('Cache Dependencies') { steps { // Use directory caching dir('cached-deps') { cache pattern: '**/node_modules/**', includes: '**/*' } } }
stage('Install') { steps { script { // Check if cache exists if (fileExists('node_modules')) { echo 'Using cached node_modules' } else { sh 'npm ci' } } } } }
post { always { // Clean workspace but keep cache cleanWs( cleanWhenNotBuilt: false, deleteDirs: false, disableDeferredWipeout: true, patterns: [ [pattern: 'node_modules/**/*', type: 'INCLUDE'], [pattern: '**', type: 'EXCLUDE'] ] ) } } } ```
### 4. Parallelize test execution
GitHub Actions parallel tests:
```yaml # GitHub Actions - Parallel test execution name: Test on: push
jobs: # Split tests across multiple runners test-matrix: runs-on: ubuntu-latest timeout-minutes: 30
strategy: matrix: shard: [1, 2, 3, 4] # Split into 4 parallel jobs os: [ubuntu-latest, macos-latest, windows-latest] fail-fast: false # Continue other jobs if one fails
steps: - uses: actions/checkout@v4
- name: Setup Node.js
- uses: actions/setup-node@v4
- with:
- node-version: '20'
- name: Cache dependencies
- uses: actions/cache@v4
- with:
- path: node_modules
- key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
- run: npm ci
# Run tests in parallel shards - name: Run tests (shard ${{ matrix.shard }}) run: | # Jest shard example npx jest --shard=${{ matrix.shard }}/4
# Or Vitest shard # npx vitest --shard=${{ matrix.shard }}/4
# Upload test results - name: Upload test results if: always() uses: actions/upload-artifact@v4 with: name: test-results-${{ matrix.shard }}-${{ matrix.os }} path: junit.xml
# Aggregate results test-summary: runs-on: ubuntu-latest needs: test-matrix if: always()
steps: - name: Download all artifacts uses: actions/download-artifact@v4
- name: Combine results
- run: |
- echo "Test execution complete across all shards"
- # Parse and combine JUnit XML files
`
GitLab CI parallel tests:
```yaml # .gitlab-ci.yml - Parallel test execution stages: - test
# Parallel test jobs test: stage: test image: node:20 parallel: max: 4 # Run up to 4 parallel jobs
script: - npm ci # Run tests with parallel grouping - npx jest --maxWorkers=4
artifacts: when: always reports: junit: junit.xml
# Or manual test splitting test-unit: stage: test script: - npm run test:unit parallel: 4
test-integration: stage: test script: - npm run test:integration parallel: 2
# Combine with test stages test-quick: stage: test script: - npm run test:unit rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event"
test-full: stage: test script: - npm run test rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH ```
Jenkins parallel stages:
```groovy // Jenkinsfile - Parallel test execution pipeline { agent any
stages { stage('Install') { steps { sh 'npm ci' } }
stage('Parallel Tests') { parallel { stage('Unit Tests') { steps { timeout(time: 15, unit: 'MINUTES') { sh 'npm run test:unit' } } }
stage('Integration Tests') { steps { timeout(time: 30, unit: 'MINUTES') { sh 'npm run test:integration' } } }
stage('E2E Tests') { steps { timeout(time: 45, unit: 'MINUTES') { sh 'npm run test:e2e' } } }
stage('Lint') { steps { timeout(time: 10, unit: 'MINUTES') { sh 'npm run lint' } } } } } }
post { always { junit '**/junit*.xml' } } } ```
### 5. Optimize Docker builds
Multi-stage builds with caching:
```yaml # GitHub Actions - Optimized Docker build name: Docker Build on: push
jobs: build: runs-on: ubuntu-latest timeout-minutes: 60
steps: - uses: actions/checkout@v4
# Use Docker Buildx for better caching - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3
# Cache Docker layers - name: Cache Docker layers uses: actions/cache@v4 with: path: /tmp/.buildx-cache key: ${{ runner.os }}-buildx-${{ github.sha }} restore-keys: | ${{ runner.os }}-buildx-
# Build with cache - name: Build Docker image uses: docker/build-push-action@v5 with: context: . push: false cache-from: type=local,src=/tmp/.buildx-cache cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max load: true tags: myapp:${{ github.sha }}
# Fix cache growth - name: Move cache run: | rm -rf /tmp/.buildx-cache mv /tmp/.buildx-cache-new /tmp/.buildx-cache ```
```dockerfile # Dockerfile - Multi-stage with layer caching # Stage 1: Dependencies (cached separately) FROM node:20-alpine AS deps WORKDIR /app
# Copy only package files first (better layer caching) COPY package.json package-lock.json ./
# Install dependencies (cached if package files unchanged) RUN npm ci --only=production
# Stage 2: Build FROM node:20-alpine AS builder WORKDIR /app
COPY package.json package-lock.json ./ RUN npm ci
COPY . .
# Build application RUN npm run build
# Stage 3: Production (minimal image) FROM node:20-alpine AS production WORKDIR /app
# Copy only production dependencies COPY --from=deps /app/node_modules ./node_modules
# Copy built application COPY --from=builder /app/dist ./dist COPY --from=builder /app/package.json ./
# Non-root user for security RUN addgroup -g 1001 -S nodejs && \ adduser -S nodejs -u 1001
USER nodejs
EXPOSE 3000
CMD ["node", "dist/index.js"] ```
### 6. Fix resource contention
Runner resource optimization:
```yaml # GitHub Actions - Choose appropriate runner name: Build on: push
jobs: build: # Use larger runner for resource-intensive jobs runs-on: ubuntu-latest-8-cores # GitHub-hosted larger runners
# Or use self-hosted runners with known resources # runs-on: self-hosted # labels: high-memory
steps: - uses: actions/checkout@v4
# Monitor resource usage - name: Check resources run: | echo "CPU cores: $(nproc)" echo "Memory: $(free -h | grep Mem | awk '{print $7}')" echo "Disk: $(df -h / | tail -1 | awk '{print $4}')"
- name: Build with resource limits
- run: |
- # Limit npm to use available memory
- export NODE_OPTIONS="--max-old-space-size=4096"
- npm run build
`
Prevention
- Set appropriate timeouts based on historical job duration (p95 + buffer)
- Implement dependency caching for all package managers
- Parallelize independent test suites across multiple runners
- Use multi-stage Docker builds with layer caching
- Monitor queue time separately from execution time
- Implement test splitting for large test suites
- Use self-hosted runners for predictable performance
- Set up alerts for jobs approaching timeout threshold
- Document timeout configuration rationale
- Regularly review and optimize slow pipeline steps
Related Errors
- **Runner offline**: Self-hosted runner disconnected
- **Resource not found**: Artifact or dependency missing
- **Rate limit exceeded**: API rate limiting (npm, Docker Hub, GitHub)
- **Out of memory**: Runner memory exhausted
- **Disk space exhausted**: Runner disk full during build