Semaphore CI Error: Complete Troubleshooting Guide

# Semaphore CI Error: Complete Troubleshooting Guide

Semaphore CI is a cloud-based CI/CD platform with a focus on speed and simplicity. When Semaphore builds fail, it's usually due to pipeline YAML configuration, agent connectivity, environment setup, or Docker-related issues.

Let me walk through the most common Semaphore CI errors and how to fix each one.

Understanding Semaphore Pipeline Structure

Semaphore uses a hierarchical YAML structure:

• version - Pipeline format version
• agent - Execution environment configuration
• blocks - Groups of jobs that can run in parallel
• jobs - Individual tasks within blocks
• promotions - Conditional pipeline triggers

Fix 1: Pipeline YAML Syntax Errors

The .semaphore/semaphore.yml has validation errors.

Symptoms: - "Invalid pipeline YAML" - Pipeline doesn't load - Validation error message

Diagnosis:

```bash # Validate YAML locally sem validate pipeline.yml

# Or use yamllint pip install yamllint yamllint .semaphore/semaphore.yml ```

Solution A: Fix YAML structure:

```yaml # WRONG - missing required fields blocks: - name: Build jobs: - script: npm install

# CORRECT - complete structure version: v1.0 name: My Pipeline agent: machine: type: e1-standard-2 os_image: ubuntu2004

blocks: - name: Build task: jobs: - name: Install and Build commands: - npm install - npm run build

# Or with prologue/epilogue blocks: - name: Test prologue: commands: - checkout task: jobs: - name: Unit Tests commands: - npm test epilogue: always: commands: - artifact push workflow results/ ```

Solution B: Validate agent configuration:

agent:
  machine:
    type: e1-standard-2  # Valid machine types: e1-standard-2, e1-standard-4, e1-standard-8
    os_image: ubuntu2004  # Valid: ubuntu2004, ubuntu1804, macos-xcode14

Invalid machine types cause pipeline rejection.

Solution C: Check block dependencies:

```yaml blocks: - name: Build task: jobs: - name: Build commands: - make build

• name: Test
• dependencies:
• - Build # Must reference existing block name
• task:
• jobs:
• - name: Test
• commands:
• - make test
• `

Fix 2: Checkout Failures

Repository checkout fails.

Symptoms: - "checkout command failed" - "Repository not found" - "Authentication failed"

Solution A: Use checkout correctly:

blocks:
  - name: Setup
    prologue:
      commands:
        - checkout  # Semaphore built-in command
    task:
      jobs:
        - name: Build
          commands:
            - npm install

The checkout command clones the repository into $SEMAPHORE_GIT_DIR.

Solution B: Handle shallow clone:

prologue:
  commands:
    - checkout
    - git fetch --unshallow || true  # Get full history if needed

Solution C: Configure SSH for private repos:

Semaphore automatically handles authentication for connected repositories. For manual SSH setup:

prologue:
  commands:
    - checkout

If using custom SSH keys:

1. 1.Go to Project Settings → Secrets
2. 2.Add SSH key secret
3. 3.Reference in pipeline:

```yaml agent: machine: type: e1-standard-2

blocks: - name: Clone task: secrets: - name: ssh-key prologue: commands: - ssh-keyscan github.com >> ~/.ssh/known_hosts - eval ssh-agent -s - ssh-add ~/.ssh/id_rsa - git clone git@github.com:org/repo.git ```

Fix 3: Command Execution Failures

Commands in jobs fail.

Symptoms: - "command exited with non-zero status" - Script not found - Permission denied

Solution A: Fix command path:

jobs:
  - name: Build
    commands:
      - npm install
      - npm run build  # Commands run in checkout directory

For custom scripts:

jobs:
  - name: Run Script
    commands:
      - chmod +x scripts/build.sh
      - ./scripts/build.sh

Solution B: Handle missing dependencies:

blocks:
  - name: Setup
    prologue:
      commands:
        - checkout
        - sem-version node 20  # Set Node.js version
    task:
      jobs:
        - name: Build
          commands:
            - node --version
            - npm install

Use sem-version for language versions:

- sem-version ruby 3.2
- sem-version python 3.11
- sem-version node 20
- sem-version java 17

Solution C: Use cache for dependencies:

blocks:
  - name: Install
    prologue:
      commands:
        - checkout
        - cache restore npm-$SEMAPHORE_GIT_BRANCH-$(checksum package-lock.json)
    task:
      jobs:
        - name: Install
          commands:
            - npm ci
            - cache store npm-$SEMAPHORE_GIT_BRANCH-$(checksum package-lock.json) node_modules

Fix 4: Docker Build Failures

Docker image builds fail.

Symptoms: - "Docker build failed" - "Image pull failed" - "Cannot connect to Docker daemon"

Solution A: Configure Docker agent:

```yaml agent: machine: type: e1-standard-4 containers: - name: main image: 'docker:20.10'

blocks: - name: Build task: jobs: - name: Docker Build commands: - docker build -t my-image . ```

Solution B: Use Docker Compose:

```yaml agent: machine: type: e1-standard-4 containers: - name: main image: 'docker:20.10'

blocks: - name: Integration Test task: jobs: - name: Test commands: - checkout - docker-compose up -d - sleep 10 - npm test - docker-compose down ```

Solution C: Push to registry:

blocks:
  - name: Push
    task:
      secrets:
        - name: docker-hub
      jobs:
        - name: Push
          commands:
            - echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin
            - docker tag my-image $DOCKER_USERNAME/my-image:latest
            - docker push $DOCKER_USERNAME/my-image:latest

Add secrets in Semaphore UI:

1. 1.Project Settings → Secrets
2. 2.Add docker-hub secret with DOCKER_USERNAME and DOCKER_PASSWORD

Fix 5: Environment Variable Issues

Variables not available.

Symptoms: - $VARIABLE empty - Undefined variable errors

Solution A: Use Semaphore built-in variables:

jobs:
  - name: Info
    commands:
      - echo "Branch: $SEMAPHORE_GIT_BRANCH"
      - echo "Commit: $SEMAPHORE_GIT_SHA"
      - echo "Repo: $SEMAPHORE_GIT_REPO_SLUG"
      - echo "Job ID: $SEMAPHORE_JOB_ID"
      - echo "Pipeline ID: $SEMAPHORE_PIPELINE_ID"

Built-in variables: - SEMAPHORE_GIT_BRANCH - Branch name - SEMAPHORE_GIT_SHA - Commit SHA - SEMAPHORE_GIT_REPO_SLUG - org/repo - SEMAPHORE_JOB_ID - Job ID - SEMAPHORE_PIPELINE_ID - Pipeline ID

Solution B: Set environment variables:

blocks:
  - name: Build
    task:
      env_vars:
        - name: NODE_ENV
          value: test
        - name: APP_VERSION
          value: "1.0.0"
      jobs:
        - name: Build
          commands:
            - echo $NODE_ENV
            - npm run build

Solution C: Use secrets:

blocks:
  - name: Deploy
    task:
      secrets:
        - name: aws-credentials
        - name: api-keys
      jobs:
        - name: Deploy
          commands:
            - aws s3 sync dist/ s3://$S3_BUCKET/
          env_vars:
            - name: AWS_ACCESS_KEY_ID
              value: $AWS_ACCESS_KEY_ID  # From aws-credentials secret
            - name: AWS_SECRET_ACCESS_KEY
              value: $AWS_SECRET_ACCESS_KEY

Fix 6: Cache Issues

Cache not restoring or storing correctly.

Symptoms: - "cache restore failed" - Dependencies not cached - Cache invalidation issues

Solution A: Use correct cache key:

```yaml prologue: commands: - checkout - cache restore gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock)

jobs: - name: Install commands: - bundle install --path vendor/bundle - cache store gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock) vendor/bundle ```

Solution B: Handle cache misses:

prologue:
  commands:
    - checkout
    - |
      if cache restore npm-$(checksum package-lock.json); then
        echo "Cache restored"
      else
        echo "No cache, fresh install"
      fi

Solution C: Use cache for large directories:

```yaml # Cache node_modules cache store npm-modules-$SEMAPHORE_JOB_ID node_modules

# Restore in next block cache restore npm-modules-$SEMAPHORE_JOB_ID

# Or use branch-specific key cache store npm-$SEMAPHORE_GIT_BRANCH node_modules cache restore npm-$SEMAPHORE_GIT_BRANCH ```

Fix 7: Artifact Handling Issues

Artifacts not uploading or downloading.

Symptoms: - "artifact push failed" - "artifact pull failed" - Missing artifacts

Solution A: Push artifacts:

```yaml jobs: - name: Build commands: - npm run build - artifact push workflow dist/ # workflow-level artifact

# Or project-level - artifact push project dist/ --destination builds/$SEMAPHORE_PIPELINE_ID/ ```

Solution B: Pull artifacts:

jobs:
  - name: Deploy
    commands:
      - artifact pull workflow dist/
      - ./deploy.sh dist/

Solution C: Configure artifact lifetime:

# Artifacts expire after 30 days by default
# Specify expiration
artifact push workflow dist/ --expire-in 7d

Fix 8: Promotion Failures

Pipeline promotions don't trigger.

Symptoms: - Promotion not starting - "Promotion condition not met" - Manual approval issues

Solution A: Configure auto promotion:

promotions:
  - name: Deploy to Production
    pipeline_file: deploy-production.yml
    auto_promote:
      when:
        branch = 'main' AND
        result = 'passed'

Solution B: Configure manual promotion:

promotions:
  - name: Deploy to Staging
    pipeline_file: deploy-staging.yml
    # No auto_promote = manual trigger

Trigger manually from Semaphore UI or API:

sem promote pipeline-id "Deploy to Staging"

Solution C: Use promotion conditions:

promotions:
  - name: Deploy
    pipeline_file: deploy.yml
    auto_promote:
      when:
        branch = 'main' OR
        tag =~ '^v[0-9]'

Fix 9: Secret Access Issues

Secrets not accessible.

Symptoms: - "Secret not found" - Secret values empty - Permission denied for secrets

Solution A: Create secrets correctly:

1. 1.Go to Project Settings → Secrets
2. 2.Click "Add Secret"
3. 3.Add name and values
4. 4.Choose organization or project scope

Solution B: Reference secrets in pipeline:

blocks:
  - name: Deploy
    task:
      secrets:
        - name: deployment-keys  # Must exist
      jobs:
        - name: Deploy
          commands:
            - echo $DEPLOY_KEY  # Variable from secret

Solution C: Check secret permissions:

Organization secrets need to be shared with project:

1. 1.Go to Organization Settings → Secrets
2. 2.Find the secret
3. 3.Share with project

Fix 10: Parallel Job Resource Issues

Parallel jobs exhaust resources.

Symptoms: - Jobs timeout - Memory errors - Slow execution

Solution A: Limit parallelism:

blocks:
  - name: Test
    task:
      jobs:
        - name: Unit Tests
          commands:
            - npm run test:unit
        - name: Integration Tests
          commands:
            - npm run test:integration
      # Jobs run in parallel within block

For sequential execution:

```yaml blocks: - name: Unit Tests task: jobs: - name: Tests commands: - npm test

• name: Integration Tests # Separate block = sequential
• dependencies:
• - Unit Tests
• task:
• jobs:
• - name: Tests
• commands:
• - npm run integration
• `

Solution B: Use appropriate machine type:

```yaml agent: machine: type: e1-standard-8 # 8 CPU, 16 GB RAM for heavy jobs

# Available types: # e1-standard-2: 2 CPU, 4 GB RAM # e1-standard-4: 4 CPU, 8 GB RAM # e1-standard-8: 8 CPU, 16 GB RAM ```

Solution C: Split large jobs:

blocks:
  - name: Parallel Tests
    task:
      jobs:
        - name: Test Suite A
          commands:
            - npm run test:suite-a
        - name: Test Suite B
          commands:
            - npm run test:suite-b
        - name: Test Suite C
          commands:
            - npm run test:suite-c

Quick Reference: Semaphore Errors

Debugging Commands

```bash # Validate pipeline sem validate pipeline.yml

# Run pipeline manually sem run pipeline-name

# Get pipeline status sem get pipelines

# View build logs sem logs job-id

# Promote pipeline sem promote pipeline-id "promotion-name"

# Manage secrets sem create secret name --value KEY=VALUE sem get secrets

# Manage caches sem get caches sem delete cache cache-id

# Manage artifacts sem get artifacts sem artifact push workflow path/ sem artifact pull workflow path/ ```