Creating and Managing GitLab CI/CD Jobs and Stages

Details: Category: Master GitLab Workflows; By Saurabh Chase; 01.Dec; Hits: 62

GitLab CI/CD jobs and stages are the building blocks of your pipeline. Jobs define specific tasks like building, testing, or deploying code, while stages group these jobs into logical steps executed in sequence or parallel. Understanding how to create and manage jobs and stages is key to optimizing your CI/CD workflows.

Saurabh Chase

Innovator | Visionary | Influencer

tbd

Experience

tbd

More to Explore

Understanding Tailwind CSS and Its Advantages

12.Sep

In this article, we’ll explore how to define jobs and stages in a .gitlab-ci.yml file, set dependencies, configure parallel execution, and use advanced features like job rules and conditions. By the end, you’ll be able to create efficient and robust pipelines for your projects.

What Are GitLab CI/CD Jobs?

Jobs are tasks executed in the pipeline, such as running tests or deploying code. Each job has a unique name and belongs to a stage. A job can include:

Script: Commands to execute.
Dependencies: Files or artifacts required from previous stages.
Environment: The target deployment environment.
Rules: Conditions for when the job should run.

What Are GitLab CI/CD Stages?

Stages define the order in which jobs are executed. For example, a typical pipeline might include stages like build, test, and deploy. Jobs in the same stage run in parallel, while stages run sequentially.

Example Pipeline Stages:

build: Compile the application.
test: Run unit and integration tests.
deploy: Deploy the application to staging or production.

Defining Jobs and Stages in `.gitlab-ci.yml`

Here’s a basic example of a pipeline with multiple stages and jobs:

stages:
  - build
  - test
  - deploy

build-job:
  stage: build
  script:
    - echo "Building the application..."
    - dotnet build

unit-test-job:
  stage: test
  script:
    - echo "Running unit tests..."
    - dotnet test

integration-test-job:
  stage: test
  script:
    - echo "Running integration tests..."
    - dotnet test --filter IntegrationTests

deploy-job:
  stage: deploy
  script:
    - echo "Deploying application to production..."

Key Features for Managing Jobs

1. Setting Dependencies

Specify dependencies between jobs to share artifacts or outputs. For example:

build-job:
  stage: build
  script:
    - dotnet build
  artifacts:
    paths:
      - bin/

test-job:
  stage: test
  script:
    - dotnet test
  dependencies:
    - build-job

Here, test-job depends on build-job and uses its artifacts.

2. Running Jobs in Parallel

Run multiple instances of a job in parallel using the parallel keyword. For example:

test-job:
  stage: test
  script:
    - dotnet test
  parallel: 4

This splits the job into four parallel tasks, reducing execution time.

3. Conditional Job Execution

Use rules or only/except keywords to control when a job runs:

deploy-job:
  stage: deploy
  script:
    - echo "Deploying to production..."
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: always

This ensures the deploy-job runs only on the main branch.

4. Defining Job Timeouts

Set a timeout to prevent jobs from running indefinitely:

build-job:
  stage: build
  script:
    - dotnet build
  timeout: 10m

This limits the job to 10 minutes.

5. Using Environments

Define environments to specify deployment targets:

deploy-job:
  stage: deploy
  script:
    - echo "Deploying to production..."
  environment:
    name: production
    url: https://myapp.com

GitLab creates an environment dashboard for tracking deployments.

Best Practices for Managing Jobs and Stages

Follow these tips for efficient pipelines:

Keep Jobs Small: Break down large tasks into smaller jobs for better parallelism and error tracking.
Reuse Code: Use scripts or include reusable YAML files for consistency across jobs.
Optimize Execution: Cache dependencies to speed up repeated tasks.
Monitor Performance: Regularly review job execution times and logs to identify bottlenecks.

Example: CI/CD Pipeline for a .NET Project

Here’s a complete pipeline example for a .NET Framework project:

stages:
  - build
  - test
  - deploy

build-job:
  stage: build
  script:
    - dotnet restore
    - dotnet build
  artifacts:
    paths:
      - bin/

unit-test-job:
  stage: test
  script:
    - dotnet test

integration-test-job:
  stage: test
  script:
    - dotnet test --filter IntegrationTests

deploy-staging:
  stage: deploy
  script:
    - echo "Deploying to staging..."
  environment:
    name: staging

deploy-production:
  stage: deploy
  script:
    - echo "Deploying to production..."
  environment:
    name: production
    url: https://myapp.com
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'

Conclusion

Creating and managing jobs and stages in GitLab CI/CD allows you to build flexible and efficient pipelines. By defining stages, setting dependencies, and utilizing features like parallel execution and conditional rules, you can streamline your development and deployment workflows. Start building your custom pipeline today and unlock the full potential of GitLab CI/CD.

Contact Us