Tasks

Tasks are the primary way that oneRepo manages multiple units of work for you and your team. Run at various lifecycles, tasks helps you run commands, checks, and other automation as fast as possible by parallelizing work as much as possible.

Task types

Serial

When running full task sets on a single machine, some tasks may need to use a lot of resources at a single time. For example, running the TypeScript plugin command will batch many processes across as many CPU cores as possible. For that reason, it should not be run in parallel to any other tasks.

/** @type import('onerepo').TaskConfig */
export default {
  'pre-commit': {
    serial: ['$0 tsc'],
  },
};

Parallel

When you have multiple tasks that run quickly and consume very few system resources, it can be possible to run them at the same time to make things faster. All parallel tasks for a given lifecycle will be run first before serial tasks to hopefully provide faster feedback.

/** @type import('onerepo').TaskConfig */
export default {
  'pre-commit': {
    parallel: ['$0 graph verify', '$0 other-fast-task'],
  },
};

Advanced tasks

In some cases, we may find that tasks need to be run in a strictly sequential manner, or potentially not run at all. We can handle these needs with the demonstrably named sequential and conditional tasks.

Sequential

In some cases, it may be necessary to run specific commands in sequential order, but still keep them as separate commands for different use-cases.

One example would be a build, publish to CDN, and deploy tasks for a web application.

Sequential tasks can be denoted by an array of strings within the list of tasks (instead of a single string or conditional task).

/** @type import('onerepo').TaskConfig */
export default {
  deploy: {
    serial: [['$0 ws my-app build', '$0 ws my-app cdn-push', '$0 ws my-app deploy']],
  },
};

Conditional

Tasks may also be conditional based on glob patterns for modified files!

/** @type import('onerepo').TaskConfig */
export default {
  'pre-commit': {
    serial: [{ match: '**/*.{ts,tsx}', cmd: '$0 tsc' }],
  },
};

Conditional tasks can also be a sequential list of commands:

/** @type import('onerepo').TaskConfig */
export default {
  deploy: {
    serial: [
      {
        match: './src/**/*',
        cmd: ['$0 ws my-app build', '$0 ws my-app cdn-push', '$0 ws my-app deploy'],
      },
    ],
  },
};

Lastly, conditional tasks aren’t just conditional, they can also include a record of meta information. This information will only be available when listing tasks like in GitHub Actions.

type TaskDef = {
  cmd: string | Array<string>;
  match?: string;
  meta?: Record<string, unknown>;
};

Lifecycles

Adding more lifecycles

First, configure the extra available lifecycles that the task runner should have access to run:

export default {
  taskConfig: {
    lifecycles: ['tacos', 'burritos'],
  },
};

Now, in any of your onerepo.config.js files, you will have the ability to add tasks for tacos and burritos and run them using the lifecycle argument --lifecycle or its alias -c:

one tasks -c tacos

Special tokens

Some tokens in tasks can be used as special replacement values that the tasks command will determine for you. This is most useful when using self-referential commands that need to know how to access the oneRepo CLI to run commands, like $0 tsc will your your repo’s tsc command.

Token	Description and replacement	Example
`$0`	Token for the repo’s oneRepo CLI. More specifically, `process.argv[1]`	`$0 tsc`
`${workspaces}`	The names of all affected workspaces will be spread comma-spaced. If you’re using `withWorkspaces()`, use as `--workspaces ${workspaces}`	`$0 build -w ${workspaces}`

Commands

`one tasks`

Run tasks against repo-defined lifecycles. This command will limit the tasks across the affected Workspace set based on the current state of the repository.

one tasks --lifecycle=<lifecycle> [options...]

You can fine-tune the determination of affected Workspaces by providing a --from-ref and/or through-ref. For more information, get help with --help --show-advanced.

Option	Type	Description	Required
`--affected`	`boolean`	Select all affected Workspaces. If no other inputs are chosen, this will default to `true`.
`--all, -a`	`boolean`	Run across all Workspaces
`--ignore-unstaged`	`boolean`	Force staged-changes mode on or off. If `true`, task determination and runners will ignore unstaged changes.
`--lifecycle, -c`	`"pre-commit"`, `"post-commit"`, `"post-checkout"`, `"pre-merge"`, `"post-merge"`, `"build"`, `"pre-deploy"`, `"pre-publish"`, `"post-publish"`	Task lifecycle to run. All tasks for the given lifecycle will be run as merged parallel tasks, followed by the merged set of serial tasks.	✅
`--list`	`boolean`	List found tasks. Implies dry run and will not actually run any tasks.
`--shard`	`string`	Shard the lifecycle across multiple instances. Format as `<shard-number>/<total-shards>`
`--staged`	`boolean`	Backup unstaged files and use only those on the git stage to calculate affected files or Workspaces. Will re-apply the unstaged files upon exit.
`--workspaces, -w`	`array`	List of Workspace names to run against

Advanced options

Option	Type	Description
`--from-ref`	`string`	Git ref to start looking for affected files or Workspaces
`--ignore`	`array`, default: `[]`	List of filepath strings or globs to ignore when matching tasks to files.
`--show-advanced`	`boolean`	Pair with `--help` to show advanced options.
`--through-ref`	`string`	Git ref to start looking for affected files or Workspaces

Shard all tasks for the pre-merge lifecycle into 5 groups and runs the first shard.

one tasks --lifecycle=pre-merge --shard=1/5

Shard all tasks for the pre-merge lifecycle into 5 groups and runs the third shard.

one tasks --lifecycle=pre-merge --shard=3/5

Workflows

GitHub Actions

While the tasks command does its best to split out parallel and serial tasks to run as fast as possible on a single machine, using GitHub Actions can save even more time by spreading out to separate runners using a matrix strategy. oneRepo offers a few options for this:

1. Single runner

The following strategy will run all tasks on a single runner, the same way as if they were run on a developer’s machine.

name: Pull request
on: pull_request

jobs:
  tasks:
    runs-on: ubuntu-latest
    name: oneRepo pre-merge tasks
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node }}
          cache: 'yarn'

      - run: yarn

      - run: yarn one tasks -c pre-merge

2. Sharding

This strategy creates a known number of action runners and distributes tasks across them. If you have a limited number of action runners, sharding may be the best option.

name: Pull request
on: pull_request

jobs:
  tasks:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        index: [1, 2, 3]
    name: oneRepo ${{ matrix.index }}/3
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node }}
          cache: 'yarn'

      - run: yarn

      - run: yarn one tasks -c pre-merge --shard=${{ matrix.index }}/3 -vvvv

3. Task per runner

This strategy is the most distributed and best if you have a lot of capacity and available action runners. It also gives the clearest and fastest feedback.

To do this, we make use of the task --list argument to write a JSON-formatted list of tasks to standard output using a setup job, then read that in with a matrix strategy as a second job.

name: Pull request

on: pull_request

jobs:
  setup:
    runs-on: ubuntu-latest
    # This job is the originator for determining the list of tasks to be farmed out to a matrix.
    # This declares the output from the `tasks --list` step
    outputs:
      tasks: ${{ steps.tasks.outputs.tasks }}
    steps:
      - uses: actions/checkout@v3
        with:
          # Ensure you check out enough history to allow oneRepo to determine the
          # merge-base and changed files. `0` will pull all history and should be sufficiently
          # safe, unless your repo is gigabytes in size
          fetch-depth: 0

      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'yarn'
      - run: yarn

      # Determine the tasks for the given lifecycle and send them to the github output
      - uses: paularmstrong/onerepo/actions/get-tasks@main
        id: tasks # important!: this must match the ID used in the output
        with:
          packageManager: yarn
          lifecycle: pre-merge

  tasks:
    runs-on: ubuntu-latest
    needs: setup
    # A conditional here prevents the job from failing unexpectedly in the rare case
    # that there are no tasks to run at all.
    if: ${{ fromJSON(needs.setup.outputs.tasks).parallel != '[]' && fromJSON(needs.setup.outputs.tasks).parallel != '[]' }}
    strategy:
      # Run all tasks, even if some fail
      fail-fast: false
      matrix:
        # Because we run all tasks on separate runners, we do not need to worry about
        # which tasks are parallel or serial – they can all be parallel
        task:
          - ${{ fromJSON(needs.setup.outputs.tasks).parallel }}
          - ${{ fromJSON(needs.setup.outputs.tasks).serial }}
    name: ${{ join(matrix.task.*.name, ', ') }}
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'yarn'
      - run: yarn

      - uses: paularmstrong/onerepo/actions/run-task@main
        with:
          task: |
            ${{ toJSON(matrix.task) }}