Skip to content

Tasks

Tasks are the primary way that oneRepo manages multiple units of work for you and your team. Run at various lifecycles, tasks helps you run commands, checks, and other automation as fast as possible by parallelizing work as much as possible.

When running full task sets on a single machine, some tasks may need to use a lot of resources at a single time. For example, running the TypeScript plugin command will batch many processes across as many CPU cores as possible. For that reason, it should not be run in parallel to any other tasks.

onerepo.config.js
/** @type import('onerepo').TaskConfig */
export default {
'pre-commit': {
serial: ['$0 tsc'],
},
};

When you have multiple tasks that run quickly and consume very few system resources, it can be possible to run them at the same time to make things faster. All parallel tasks for a given lifecycle will be run first before serial tasks to hopefully provide faster feedback.

onerepo.config.js
/** @type import('onerepo').TaskConfig */
export default {
'pre-commit': {
parallel: ['$0 graph verify', '$0 other-fast-task'],
},
};

In some cases, we may find that tasks need to be run in a strictly sequential manner, or potentially not run at all. We can handle these needs with the demonstrably named sequential and conditional tasks.

In some cases, it may be necessary to run specific commands in sequential order, but still keep them as separate commands for different use-cases.

One example would be a build, publish to CDN, and deploy tasks for a web application.

Sequential tasks can be denoted by an array of strings within the list of tasks (instead of a single string or conditional task).

onerepo.config.js
/** @type import('onerepo').TaskConfig */
export default {
deploy: {
serial: [['$0 ws my-app build', '$0 ws my-app cdn-push', '$0 ws my-app deploy']],
},
};

Tasks may also be conditional based on glob patterns for modified files!

onerepo.config.js
/** @type import('onerepo').TaskConfig */
export default {
'pre-commit': {
serial: [{ match: '**/*.{ts,tsx}', cmd: '$0 tsc' }],
},
};

Conditional tasks can also be a sequential list of commands:

onerepo.config.js
/** @type import('onerepo').TaskConfig */
export default {
deploy: {
serial: [
{
match: './src/**/*',
cmd: ['$0 ws my-app build', '$0 ws my-app cdn-push', '$0 ws my-app deploy'],
},
],
},
};

Lastly, conditional tasks aren’t just conditional, they can also include a record of meta information. This information will only be available when listing tasks like in GitHub Actions.

type TaskDef = {
cmd: string | Array<string>;
match?: string;
meta?: Record<string, unknown>;
};

First, configure the extra available lifecycles that the task runner should have access to run:

onerepo.config.js
export default {
taskConfig: {
lifecycles: ['tacos', 'burritos'],
},
};

Now, in any of your onerepo.config.js files, you will have the ability to add tasks for tacos and burritos and run them using the lifecycle argument --lifecycle or its alias -c:

Terminal window
one tasks -c tacos

Some tokens in tasks can be used as special replacement values that the tasks command will determine for you. This is most useful when using self-referential commands that need to know how to access the oneRepo CLI to run commands, like $0 tsc will your your repo’s tsc command.

TokenDescription and replacementExample
$0Token for the repo’s oneRepo CLI. More specifically, process.argv[1]$0 tsc
${workspaces}The names of all affected workspaces will be spread comma-spaced. If you’re using withWorkspaces(), use as --workspaces ${workspaces}$0 build -w ${workspaces}

Run tasks against repo-defined lifecycles. This command will limit the tasks across the affected Workspace set based on the current state of the repository.

Terminal window
one tasks --lifecycle=<lifecycle> [options...]

You can fine-tune the determination of affected Workspaces by providing a --from-ref and/or through-ref. For more information, get help with --help --show-advanced.

OptionTypeDescriptionRequired
--affectedbooleanSelect all affected Workspaces. If no other inputs are chosen, this will default to true.
--all, -abooleanRun across all Workspaces
--ignore-unstagedbooleanForce staged-changes mode on or off. If true, task determination and runners will ignore unstaged changes.
--lifecycle, -c"pre-commit", "post-commit", "post-checkout", "pre-merge", "post-merge", "build", "pre-deploy", "pre-publish", "post-publish"Task lifecycle to run. All tasks for the given lifecycle will be run as merged parallel tasks, followed by the merged set of serial tasks.
--listbooleanList found tasks. Implies dry run and will not actually run any tasks.
--shardstringShard the lifecycle across multiple instances. Format as <shard-number>/<total-shards>
--stagedbooleanBackup unstaged files and use only those on the git stage to calculate affected files or Workspaces. Will re-apply the unstaged files upon exit.
--workspaces, -warrayList of Workspace names to run against
Advanced options
OptionTypeDescription
--from-refstringGit ref to start looking for affected files or Workspaces
--ignorearray, default: []List of filepath strings or globs to ignore when matching tasks to files.
--show-advancedbooleanPair with --help to show advanced options.
--through-refstringGit ref to start looking for affected files or Workspaces

Shard all tasks for the pre-merge lifecycle into 5 groups and runs the first shard.

Terminal window
one tasks --lifecycle=pre-merge --shard=1/5

Shard all tasks for the pre-merge lifecycle into 5 groups and runs the third shard.

Terminal window
one tasks --lifecycle=pre-merge --shard=3/5

While the tasks command does its best to split out parallel and serial tasks to run as fast as possible on a single machine, using GitHub Actions can save even more time by spreading out to separate runners using a matrix strategy. oneRepo offers a few options for this:

The following strategy will run all tasks on a single runner, the same way as if they were run on a developer’s machine.

.github/workflows/pull-request.yaml
name: Pull request
on: pull_request
jobs:
tasks:
runs-on: ubuntu-latest
name: oneRepo pre-merge tasks
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
- run: yarn
- run: yarn one tasks -c pre-merge

This strategy creates a known number of action runners and distributes tasks across them. If you have a limited number of action runners, sharding may be the best option.

.github/workflows/pull-request.yaml
name: Pull request
on: pull_request
jobs:
tasks:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
index: [1, 2, 3]
name: oneRepo ${{ matrix.index }}/3
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
cache: 'yarn'
- run: yarn
- run: yarn one tasks -c pre-merge --shard=${{ matrix.index }}/3 -vvvv

This strategy is the most distributed and best if you have a lot of capacity and available action runners. It also gives the clearest and fastest feedback.

To do this, we make use of the task --list argument to write a JSON-formatted list of tasks to standard output using a setup job, then read that in with a matrix strategy as a second job.

.github/workflows/pull-request.yaml
name: Pull request
on: pull_request
jobs:
setup:
runs-on: ubuntu-latest
# This job is the originator for determining the list of tasks to be farmed out to a matrix.
# This declares the output from the `tasks --list` step
outputs:
tasks: ${{ steps.tasks.outputs.tasks }}
steps:
- uses: actions/checkout@v3
with:
# Ensure you check out enough history to allow oneRepo to determine the
# merge-base and changed files. `0` will pull all history and should be sufficiently
# safe, unless your repo is gigabytes in size
fetch-depth: 0
- uses: actions/setup-node@v3
with:
node-version: 18
cache: 'yarn'
- run: yarn
# Determine the tasks for the given lifecycle and send them to the github output
- uses: paularmstrong/onerepo/actions/get-tasks@main
id: tasks # important!: this must match the ID used in the output
with:
packageManager: yarn
lifecycle: pre-merge
tasks:
runs-on: ubuntu-latest
needs: setup
# A conditional here prevents the job from failing unexpectedly in the rare case
# that there are no tasks to run at all.
if: ${{ fromJSON(needs.setup.outputs.tasks).parallel != '[]' && fromJSON(needs.setup.outputs.tasks).parallel != '[]' }}
strategy:
# Run all tasks, even if some fail
fail-fast: false
matrix:
# Because we run all tasks on separate runners, we do not need to worry about
# which tasks are parallel or serial – they can all be parallel
task:
- ${{ fromJSON(needs.setup.outputs.tasks).parallel }}
- ${{ fromJSON(needs.setup.outputs.tasks).serial }}
name: ${{ join(matrix.task.*.name, ', ') }}
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-node@v3
with:
node-version: 18
cache: 'yarn'
- run: yarn
- uses: paularmstrong/onerepo/actions/run-task@main
with:
task: |
${{ toJSON(matrix.task) }}