Configuring CI Using GitHub Actions and Nx
There are two general approaches to setting up CI with Nx - using a single job or distributing tasks across multiple jobs. For smaller repositories, a single job is faster and cheaper, but once a full CI run starts taking 10 to 15 minutes, using multiple jobs becomes the better option. Nx Cloud's distributed task execution allows you to keep the CI pipeline fast as you scale. As the repository grows, all you need to do is add more agents.
Process Only Affected Projects With One Job on GitHub Actions
Below is an example of an GitHub Actions setup that runs on a single job, building and testing only what is affected. This uses the nx affected command to run the tasks only for the projects that were affected by that PR.
1name: CI
2on:
3 push:
4 branches:
5 # Change this if your primary branch is not main
6 - main
7 pull_request:
8
9jobs:
10 main:
11 runs-on: ubuntu-latest
12 steps:
13 - uses: actions/checkout@v4
14 with:
15 fetch-depth: 0
16 # Cache node_modules
17 - uses: actions/setup-node@v3
18 with:
19 node-version: 20
20 cache: 'npm'
21 - run: npm ci
22 - uses: nrwl/nx-set-shas@v3
23 # This line is needed for nx affected to work when CI is running on a PR
24 - run: git branch --track main origin/main
25
26 - run: npx nx format:check
27 - run: npx nx affected -t lint,test,build --parallel=3
28Get the Commit of the Last Successful Build
GitHub can track the last successful run on the main branch and use this as a reference point for the BASE. The nrwl/nx-set-shas provides a convenient implementation of this functionality which you can drop into your existing CI config. To understand why knowing the last successful build is important for the affected command, check out the in-depth explanation in Actions's docs.
Distribute Tasks Across Agents on GitHub Actions
To set up Distributed Task Execution (DTE), you can run this generator:
❯
npx nx g ci-workflow --ci=github
Or you can copy and paste the workflow below:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8jobs:
9 main:
10 name: Nx Cloud - Main Job
11 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
12 with:
13 number-of-agents: 3
14 parallel-commands: |
15 npx nx-cloud record -- npx nx format:check
16 parallel-commands-on-agents: |
17 npx nx affected -t lint,test,build --parallel=2
18
19 agents:
20 name: Nx Cloud - Agents
21 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
22 with:
23 number-of-agents: 3
24This configuration is using two reusable workflows from the nrwl/ci repository. You can check out the full API for those workflows.
The first workflow is for the main job:
1 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
2The parallel-commands script will be run on the main job. The parallel-commands-on-agents script will be distributed across the available agents.
The second workflow is for the agents:
1 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
2The number-of-agents property controls how many agent jobs are created. Note that this property should be the same number for each workflow.
The number-of-agents property and the --parallel flag both parallelize tasks, but in different ways. The way this workflow is written, there will be 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.
Custom Distributed CI with Nx Cloud on GitHub Actions
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8env:
9 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
10 NX_CLOUD_DISTRIBUTED_EXECUTION_AGENT_COUNT: 3 # expected number of agents
11 NX_BRANCH: ${{ github.event.number || github.ref_name }}
12 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
13 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
14
15jobs:
16 main:
17 name: Nx Cloud - Main Job
18 runs-on: ubuntu-latest
19 steps:
20 - uses: actions/checkout@v4
21 name: Checkout [Pull Request]
22 if: ${{ github.event_name == 'pull_request' }}
23 with:
24 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
25 ref: ${{ github.event.pull_request.head.sha }}
26 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
27 fetch-depth: 0
28
29 - uses: actions/checkout@v4
30 name: Checkout [Default Branch]
31 if: ${{ github.event_name != 'pull_request' }}
32 with:
33 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
34 fetch-depth: 0
35
36 # Set node/npm/yarn versions using volta
37 - uses: volta-cli/action@v4
38 with:
39 package-json-path: '${{ github.workspace }}/package.json'
40
41 - name: Use the package manager cache if available
42 uses: actions/setup-node@v3
43 with:
44 node-version: 20
45 cache: 'npm'
46
47 - name: Install dependencies
48 run: npm ci
49
50 - name: Check out the default branch
51 run: git branch --track main origin/main
52
53 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done
54 run: npx nx-cloud start-ci-run --stop-agents-after=build
55
56 - name: Run commands in parallel
57 run: |
58 # initialize an array to store process IDs (PIDs)
59 pids=()
60
61 # function to run commands and store the PID
62 function run_command() {
63 local command=$1
64 $command & # run the command in the background
65 pids+=($!) # store the PID of the background process
66 }
67
68 # list of commands to be run on main has env flag NX_CLOUD_DISTRIBUTED_EXECUTION set to false
69 run_command "NX_CLOUD_DISTRIBUTED_EXECUTION=false npx nx-cloud record -- npx nx format:check"
70
71 # list of commands to be run on agents
72 run_command "npx nx affected -t lint,test,build --parallel=3"
73
74 # wait for all background processes to finish
75 for pid in ${pids[*]}; do
76 if ! wait $pid; then
77 exit 1 # exit with an error status if any process fails
78 fi
79 done
80
81 exit 0 # exits with success status if a all processes complete successfully
82
83 agents:
84 name: Agent ${{ matrix.agent }}
85 runs-on: ubuntu-latest
86 strategy:
87 matrix:
88 # Add more agents here as your repository expands
89 agent: [1, 2, 3]
90 steps:
91 - name: Checkout
92 uses: actions/checkout@v4
93
94 # Set node/npm/yarn versions using volta
95 - uses: volta-cli/action@v4
96 with:
97 package-json-path: '${{ github.workspace }}/package.json'
98
99 - name: Use the package manager cache if available
100 uses: actions/setup-node@v3
101 with:
102 node-version: 20
103 cache: 'npm'
104
105 - name: Install dependencies
106 run: npm ci
107
108 - name: Start Nx Agent ${{ matrix.agent }}
109 run: npx nx-cloud start-agent
110 env:
111 NX_AGENT_NAME: ${{ matrix.agent }}
112There are comments throughout the workflow to help you understand what is happening in each section.