A job with a `runs-on` that references matrix variables will not run with the expected labels. eg.
```
jobs:
matrix-runs-on:
strategy:
matrix:
os: [ubuntu-latest, ubuntu-20.04]
runs-on: ${{ matrix.os }}
steps:
...
```
Due to shared mutated state, both jobs that this generates will (w/ a race condition) either run with the `ubuntu-latest` or `ubuntu-20.04`, but rarely (never observed) with the expected outcome of running on both labels.
`EvaluateYamlNode` is used to evaluate expressions in the `runs-on` field in the context of the current running job, but mutating an object shared between multiple concurrent jobs (in matrix evaluation). This results in the evaluation results from one job spilling into another and corrupting their `runs-on` labels.
```
==================
WARNING: DATA RACE
Write at 0x00c00047e0b0 by goroutine 1739:
reflect.typedmemmove()
/.../go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/src/runtime/mbarrier.go:213 +0x0
reflect.Value.Set()
/.../go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/src/reflect/value.go:2062 +0x184
gopkg.in/yaml%2ev3.(*decoder).unmarshal()
/.../go/pkg/mod/gopkg.in/yaml.v3@v3.0.1/decode.go:493 +0x7b4
gopkg.in/yaml%2ev3.(*Node).Decode()
/.../go/pkg/mod/gopkg.in/yaml.v3@v3.0.1/yaml.go:149 +0x355
code.forgejo.org/forgejo/runner/v9/act/runner.expressionEvaluator.EvaluateYamlNode()
/.../forgejo-runner/act/runner/expression.go:372 +0x7a
code.forgejo.org/forgejo/runner/v9/act/runner.(*expressionEvaluator).EvaluateYamlNode()
<autogenerated>:1 +0x6b
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).runsOnPlatformNames()
/.../forgejo-runner/act/runner/run_context.go:1019 +0x2af
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).runsOnImage()
/.../forgejo-runner/act/runner/run_context.go:1002 +0x772
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).platformImage()
/.../forgejo-runner/act/runner/run_context.go:1032 +0x77
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).isEnabled()
/.../forgejo-runner/act/runner/run_context.go:1069 +0x3c7
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).Executor.func1()
/.../forgejo-runner/act/runner/run_context.go:964 +0x4b
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.1()
/.../forgejo-runner/act/runner/runner.go:223 +0x271
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.NewParallelExecutor.2.1()
/.../forgejo-runner/act/common/executor.go:107 +0x61
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.NewParallelExecutor.2.gowrap1()
/.../forgejo-runner/act/common/executor.go:109 +0x4f
Previous read at 0x00c00047e0b0 by goroutine 1742:
code.forgejo.org/forgejo/runner/v9/act/model.(*Job).RunsOn()
/.../forgejo-runner/act/model/workflow.go:361 +0x3c4
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).runsOnImage()
/.../forgejo-runner/act/runner/run_context.go:991 +0x57a
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).platformImage()
/.../forgejo-runner/act/runner/run_context.go:1032 +0x77
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).isEnabled()
/.../forgejo-runner/act/runner/run_context.go:1069 +0x3c7
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).Executor.func1()
/.../forgejo-runner/act/runner/run_context.go:964 +0x4b
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.1()
/.../forgejo-runner/act/runner/runner.go:223 +0x271
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.NewParallelExecutor.2.1()
/.../forgejo-runner/act/common/executor.go:107 +0x61
code.forgejo.org/forgejo/runner/v9/act/runner.(*runnerImpl).NewPlanExecutor.func1.NewParallelExecutor.2.gowrap1()
/.../forgejo-runner/act/common/executor.go:109 +0x4f
...
==================
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/871): <!--number 871 --><!--line 0 --><!--description Zml4OiBkYXRhIHJhY2UgaW4gJ3J1bnMtb24nIGV4cHJlc3Npb25zIGNhdXNlcyBpbmNvcnJlY3Qgam9iIGxhYmVscyBkdXJpbmcgZXhlY3V0aW9u-->fix: data race in 'runs-on' expressions causes incorrect job labels during execution<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/871
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
The `setupShell` function would update the shell stored on the `Step` object, setting it to either a default value from the job, an expression evaluated in the context of the job, a default from the workflow, or finally falling back to bash or powershell defaults. Typically this would be fine -- although it would trigger the data race detector because the `Step` is a shared object between multiple concurrent matrix evaluations for the job.
In the *really quite unlikely* case that the `shell` field on a step or job referenced a matrix variable, this data race would actually trigger the shared step's `Shell` value to end up as "whichever one was evaluated last", causing the wrong shell to be used. The new `matrix-shell` test triggers this behavior, and fails without the associated code fix.
As a fix, the `Shell` field in `Step` is never mutated; instead only the value on non-shared `stepRun` instance is updated from `setupShellCommand`. `Shell` was renamed to `RawShell` as part of verifying all references were updated and it seemed to make sense to keep that name since it is a pre-evaluator value.
```
==================
WARNING: DATA RACE
Write at 0x00c00013e9b0 by goroutine 1470:
code.forgejo.org/forgejo/runner/v9/act/runner.(*stepRun).setupShell()
/.../forgejo-runner/act/runner/step_run.go:210 +0x8f2
code.forgejo.org/forgejo/runner/v9/act/common/git.FindGitRevision()
/.../forgejo-runner/act/common/git/git.go:58 +0xc4
code.forgejo.org/forgejo/runner/v9/act/model.(*GithubContext).SetSha()
/.../forgejo-runner/act/model/github_context.go:161 +0x6b5
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).getGithubContext()
/.../forgejo-runner/act/runner/run_context.go:1228 +0x26ca
...
Previous write at 0x00c00013e9b0 by goroutine 1469:
code.forgejo.org/forgejo/runner/v9/act/runner.(*stepRun).setupShell()
/.../forgejo-runner/act/runner/step_run.go:210 +0x8f2
code.forgejo.org/forgejo/runner/v9/act/common/git.FindGitRevision()
/.../forgejo-runner/act/common/git/git.go:58 +0xc4
code.forgejo.org/forgejo/runner/v9/act/model.(*GithubContext).SetSha()
/.../forgejo-runner/act/model/github_context.go:161 +0x6b5
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).getGithubContext()
/.../forgejo-runner/act/runner/run_context.go:1228 +0x26ca
...
==================
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/865): <!--number 865 --><!--line 0 --><!--description Zml4OiBkYXRhIHJhY2UgY29uZGl0aW9uIGNhdXNpbmcgaW5jb3JyZWN0IGBzaGVsbGAgb24gYSB0YXNrIHN0ZXAgaWYgaXQgcmVmZXJlbmNlZCBhIG1hdHJpeCB2YXJpYWJsZQ==-->fix: data race condition causing incorrect `shell` on a task step if it referenced a matrix variable<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/865
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
In `setJobResult` there is no coordination between multiple jobs that are completing, leading to a possible condition where `jobResult` can be read from the matrix job as `"success"` by a job, marked as `"failed"` by another job, and then marked as `"success"` by other jobs.
To my knowledge, the race condition has not been observed in a real-world case, but has been reproduced in a unit test.
```
==================
WARNING: DATA RACE
Read at 0x00c0006d08a0 by goroutine 29232:
code.forgejo.org/forgejo/runner/v9/act/runner.setJobResult()
/.../forgejo-runner/act/runner/job_executor.go:173
+0x359
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.func6()
/.../forgejo-runner/act/runner/job_executor.go:118
+0x15d
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.Executor.Finally.func14()
/.../forgejo-runner/act/common/executor.go:183 +0x86
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.func7()
/.../forgejo-runner/act/runner/job_executor.go:161
+0x191
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.Executor.Finally.func16()
/.../forgejo-runner/act/common/executor.go:183 +0x86
...
Previous write at 0x00c0006d08a0 by goroutine 29234:
code.forgejo.org/forgejo/runner/v9/act/runner.(*RunContext).result()
/.../forgejo-runner/act/runner/run_context.go:897
+0x271
code.forgejo.org/forgejo/runner/v9/act/runner.setJobResult()
/.../forgejo-runner/act/runner/job_executor.go:181
+0x66e
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.func6()
/.../forgejo-runner/act/runner/job_executor.go:118
+0x15d
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.Executor.Finally.func14()
/.../forgejo-runner/act/common/executor.go:183 +0x86
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.func7()
/.../forgejo-runner/act/runner/job_executor.go:161
+0x191
code.forgejo.org/forgejo/runner/v9/act/runner.newJobExecutor.Executor.Finally.func16()
/.../forgejo-runner/act/common/executor.go:183 +0x86
...
==================
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/862): <!--number 862 --><!--line 0 --><!--description Zml4OiByYWNlIGNvbmRpdGlvbiBpbiBtYXRyaXggam9iIHJlc3VsdCBzdGF0ZSBtYXkgcmVzdWx0IGluIGZhaWxlZCBqb2JzIGJlaW5nIG1hcmtlZCBhcyBzdWNjZXNzZnVs-->fix: race condition in matrix job result state may result in failed jobs being marked as successful<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/862
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
It will be imported by Forgejo.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/777): <!--number 777 --><!--line 0 --><!--description Y2hvcmU6IHRvIGFsbG93IHRoZSBydW5uZXIgdG8gYmUgaW1wb3J0ZWQsIHY5IG5lZWRzIHRvIGJlIGluIHRoZSBnbyBtb2R1bGU=-->chore: to allow the runner to be imported, v9 needs to be in the go module<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/777
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
To prepare for a smooth merge in the runner codebase.
- run with --fix for gofumpt and golangci
- manual edits for
- disabling useless package naming warning
- rename variables that had underscore in their name
- remove trailing else at the end of a few functions
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/206
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
For instance, the volume name derived from the workflow name may exceed the file system limit when the container name it is derived from is too long.
Fixesforgejo/runner#152
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/191
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
This is a followup of https://code.forgejo.org/forgejo/act/pulls/170 so that it is possible to read a workflow without validation. It is not uncommon for Forgejo to read a workflow just to extract a few information from it, knowing it has been validated before. It would be a performance regression if schema validation happened in these cases.
This is a port of https://github.com/nektos/act/pull/2717/files
It is a breaking change in the context of Forgejo and Forgejo runner because it will need to add the new `validate` argument when reading workflows.
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/180
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
* fix: use non strict schema to allow some undefined behavior
* GitHub Actions doesn't use the newer strict schema in the service
* Tolerate more hallucinations
* Update workflow.go
* Update workflow.go
* Update pkg/model/workflow.go
Co-authored-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
---------
Co-authored-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
(cherry picked from commit b52da6190c971733cdf3ffcef97cdbdce4ba42e5)
- upgrade to golangci-lint@v1.62.2
- make it renovate friendly
- remove most frequent lint check that are not of consequence (unused
args, etc.)
- fix remaining lint errors
- add renovate custom manager to update the Makefile variable
* WorkflowDispatchConfig supports ScalarNode and SequenceNode yaml node kinds
* Avoid using log.Fatal
* package slices is not in golang 1.20
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* throw if `uses` is invalid
* update JobType to return error
* lint
* put //nolint:dupl on wrong test
* update error message to remove end punctuation
* lint
* update remote job type check
* move if statement
* rm nolint:dupl ... we'll see how that goes
---------
Co-authored-by: Casey Lee <cplee@nektos.com>
* Log incoming jobs.
Log the full contents of the job protobuf to make debugging jobs easier
* Ensure that the parallel executor always uses at least one thread.
The caller may mis-calculate the number of CPUs as zero, in which case
ensure that at least one thread is spawned.
* Use runtime.NumCPU for CPU counts.
For hosts without docker, GetHostInfo() returns a blank struct which
has zero CPUs and causes downstream trouble.
---------
Co-authored-by: Paul Armstrong <psa@users.noreply.gitea.com>
Co-authored-by: Jason Song <i@wolfogre.com>
This PR is to support overwriting the default `CMD` command of `services` containers.
This is a Gitea specific feature and GitHub Actions doesn't support this syntax.
Reviewed-on: https://gitea.com/gitea/act/pulls/50
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
Fix https://gitea.com/gitea/act_runner/issues/80
Fix https://gitea.com/gitea/act_runner/issues/85
To support reusable workflows, I made some improvements:
- read `yml` files from both `.gitea/workflows` and `.github/workflows`
- clone repository for local reusable workflows because the runner doesn't have the code in its local directory
- fix the incorrect clone url like `https://https://gitea.com`
Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/34
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
* fix: map job output for reusable workflows
This fixes the job outputs for reusable workflows. There is
a required indirection. Before this we took the outputs from
all jobs which is not what users express with the workflow
outputs.
* fix: remove double evaluation
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* feat: allow to spawn and run a local reusable workflow
This change contains the ability to parse/plan/run a local
reusable workflow.
There are still numerous things missing:
- inputs
- secrets
- outputs
* feat: add workflow_call inputs
* test: improve inputs test
* feat: add input defaults
* feat: allow expressions in inputs
* feat: use context specific expression evaluator
* refactor: prepare for better re-usability
* feat: add secrets for reusable workflows
* test: use secrets during test run
* feat: handle reusable workflow outputs
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This change does parse the different types of workflow jobs.
It is not much by itself but the start to implement reusable
workflows.
Relates to #826
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* test: check workflow_dispatch inputs
This implements a test to check for `workflow_dispatch` inputs.
This will be a prerequisite for implementing the inputs.
* feat: map workflow_dispatch input to expression evaluator
This changes adds the workflow_dispatch event inputs
to the `inputs` context and maintaining the boolean type
* fix: coerce boolean input types
* fix: use step env if available, rc env otherwise
* test: add test case for #1319
* fix: setup of composite inputs
This change fixes the composite action setup handling of inputs.
All inputs are taken from the env now. The env is composed of
the 'level above'.
For example:
- step env -> taken from run context
- action env -> taken from step env
- composite env -> taken from action env
Before this change the env setup for steps, actions and composite
run contexts was harder to understand as all parts looked into
one of these: parent run context, step, action, composite run context.
Now the 'data flow' is from higher levels to lower levels which should
make it more clean.
Fixes#1319
* test: add simple remote composite action test
Since we don't have a remote composite test at all
before this, we need at least the simplest case.
This does not check every feature, but ensures basic
availability of remote composite actions.
* refactor: move ActionRef and ActionRepository
Moving ActionRef and ActionRepository from RunContext into the
step, allows us to remove the - more or less - ugly copy operations
from the RunContext.
This is more clean, as each step does hold the data required anyway
and the RunContext shouldn't know about the action details.
* refactor: remove unused properties
