- remove go vet and build as they are now redundant
- replace ./pkg with ./act
- rename the workflow from test to act to distinguish it from the
runner workflow
cleanup and factorization with the runner workflow is left for later
Only changes test files.
---
- remove tests specific to running on a host with no container
they are the same as with the containers (TestRunEventHostEnvironment)
- prefix the name of the tests with a distinctive name to
more easily run them together
- use code.forgejo.org/oci images whereever possible
- remove some tests that are either
- difficult to understand (ancient bugs)
- not yet well understood (related to reusable workflows)
- depend on github (generation of workflows to be run on the fly
e.g. updateTestIfWorkflow)
- fix the TestSanitizeNetworkAlias tests that were not run
and make them easier to debug
- disable tests of options that are forbidden in Forgejo Actions
(testdata/container-hostname/push.yml)
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/202
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
- prefer the lowercase version of the DNS name which is more common they are case insensitive anyway
- fix the inverted information when sanitation happens
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/197
Reviewed-by: msrd0 <msrd0@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
- s/[^A-Z0-9-]/_/g
- add a log line in case the name is sanitized
Closesforgejo/runner#226
---
It is breaking because it will fail jobs that rely on service names that contain characters that are sanitized
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/190
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
This is a followup of https://code.forgejo.org/forgejo/act/pulls/170 so that it is possible to read a workflow without validation. It is not uncommon for Forgejo to read a workflow just to extract a few information from it, knowing it has been validated before. It would be a performance regression if schema validation happened in these cases.
This is a port of https://github.com/nektos/act/pull/2717/files
It is a breaking change in the context of Forgejo and Forgejo runner because it will need to add the new `validate` argument when reading workflows.
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/180
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
- do not override username and password when looping over services
- split prepareJobContainer out of startJobContainer
- split getNetworkName out as it is used by both
- add unit tests for prepareJobContainer
- make containre.NewContainer mockable
- add MockVariable helper
Closesforgejo/runner#575
---
Note to reviewers: do not show whitespace change, the refactor will show in a minimal way. When the fix is reverted the tests fail as follows:
```
Diff:
--- Expected
+++ Actual
@@ -81,4 +81,4 @@
Image: (string) (len=10) "some:image",
- Username: (string) (len=17) "containerusername",
- Password: (string) (len=17) "containerpassword",
+ Username: (string) (len=16) "service2username",
+ Password: (string) (len=16) "service2password",
Entrypoint: ([]string) (len=3) {
Test: TestStartJobContainer/Overlapping
```
Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/181
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
* Support array expressions in runs-on
* Simplify appproach to use EvaluateYamlNode, fix case-sensitivity bug
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: correct ref and ref_name
The ref in the GitHub context is always full qualified
(e.g. refs/heads/branch, refs/tags/v1).
The ref_name is the ref with the strippep prefix.
In case of pull_requests, this is the merge commit ref
(e.g. refs/pull/123/merge -> 123/merge).
* test: update test data
* fix: github.job property is empty, GITHUB_JOB should be job id
fix: github.job property is empty #1621
fix: GITHUB_JOB should be the id not the name #1473
* fix linter problem.
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: align github.ref to GitHub Action
The `github.ref` value should be `refs/heads/branch` in most cases.
There are a few exceptions handled by the code.
This change prefixes the default case with `refs/heads` and adds
tests for this and the excpetional cases.
* fix: correct existing assertions
* feat: use logger from context wherever possible
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add step/job id and results to json logs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: value to be masked should not be hard-coded in the action
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: replace values following ::add-mask:: in evaluated strings
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: [DEBUG] identifier for debug logs to distinguish them
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: replace logger with step logger
The container gets injected a job logger, but during the time that steps
are run, we want to use the step logger.
This commit wraps pre/main/post steps in an executor that replaces the
job logger with a step logger.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add pre/post stage identifier fields to json log output
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add job/step result status to skipped steps/jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* feat: add post step to actions and add state command
This commit includes requried changes for running post steps
for local and remote actions.
This allows general cleanup work to be done after executing
an action.
Communication is allowed between this steps, by using the
action state.
* feat: collect pre and post steps for composite actions
* refactor: move composite action logic into own file
* refactor: restructure composite handling
* feat: run composite post steps during post step lifecycle
* refactor: remove duplicate log output
* feat: run all composite post actions in a step
Since composite actions could have multiple pre/post steps inside,
we need to run all of them in a single top-level pre/post step.
This PR includes a test case for this and the correct order of steps
to be executed.
* refactor: remove unused lines of code
* refactor: simplify test expression
* fix: use composite job logger
* fix: make step output more readable
* fix: enforce running all post executor
To make sure every post executor/step is executed, it is chained
with it's own Finally executor.
* fix: do not run post step if no step result is available
Having no step result means we do not run any step (neither pre
nor main) and we do not need to run post.
* fix: setup defaults
If no pre-if or post-if is given, it should default to 'always()'.
This could be set even if there is no pre or post step.
In fact this is required for composite actions and included post
steps to run.
* fix: output step related if expression
* test: update expectation
* feat: run pre step from actions (#1110)
This PR implements running pre steps for remote actions.
This includes remote actions using inside local composite actions.
* fix: set correct expr default status checks
For post-if conditions the default status check should be
always(), while for all other if expression the default status
check is success()
References:
https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functionshttps://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runspost-if
* fix: remove code added during rebase
* refactor: remove composite action runcontext workaround
The RunContext is cloned to execute a composite action with all its
steps in a similar context. This required some workaround, since
the command handler has kept a reference to the original RunContext.
This is solved now, by replacing the docker LogWriter with a proper
scoped LogWriter.
This prepares for a simpler setup of composite actions to be able
to create and re-create the composite RunContext for pre/main/post
action steps.
* test: check env-vars for local js and docker actions
* test: test remote docker and js actions
* fix: merge github context into env when read and setup
* refacotr: simplify composite context setup
* test: use a map matcher to test input setup
* fix: restore composite log output
Since we create a new line writer, we need to log the raw_output as well.
Otherwise no output will be available from the log-writer
* fix: add RunContext JobName to fill GITHUB_JOBNAME
* test: use nektos/act-test-actions
* fix: allow masking values in composite actions
To allow masking of values from composite actions, we need
to use a custom job logger with a reference to the masked
values for the composite run context.
* refactor: keep existing logger for composite actions
To not introduce another new logger while still be able to use
the masking from the composite action, we add the masks to
the go context. To leverage that context, we also add the context
to the log entries where the valueMasker then could get the actual
mask values.
With this way to 'inject' the masked values into the logger, we do
- keep the logger
- keep the coloring
- stay away from inconsistencies due to parallel jobs
* fix: re-add removed color increase
This one should have never removed :-)
* fix: add missing ExtraPath attribute
* fix: merge run context env into composite run context env
This adds a test and fix for the parent environment. It should be
inherited by the composite environment.
* test: add missing test case
* fix: store github token next to secrets
We must not expose the secrets to composite actions, but the
`github.token` is available inside composite actions.
To provide this we store the token in the config and create it in
the GithubContext from there.
The token can be used with `github.token` but is not available as
`secrets.GITHUB_TOKEN`.
This implements the same behavior as on GitHub.
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Marcus Noll <markus.noll@new-work.se>
* fixup! fix: allow masking values in composite actions
* style: use tabs instead of spaces to fix linter errors
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Marcus Noll <markus.noll@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Prevents random failing of all other runs when qemu binfmt
is not set up
Signed-off-by: Ryan <me@hackerc.at>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* don't merge env var if it exists already
* remove test workflow
* add some tests
* change mergeMaps order instead of checking for existence
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* feat: implement expression evaluator
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se>
* feat: integrate exprparser into act
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se>
* Escape { and }, do not fail on missing properties
* Fix empty inputs context
* fix: contains() comparison for complex values
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se>
Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
This commit moves the githubContext, jobContext and stepResult structs
from the runner package to the model package in preparation for #908
because the expression.go file lives in the runner package and would
introduce cyclic dependencies with the exprparser package.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: continue jobs + steps after failure
To allow proper if expression handling on jobs and steps (like always,
success, failure, ...) we need to continue running all executors in
the prepared chain.
To keep the error handling intact we add an occurred error to the
go context and handle it later in the pipeline/chain.
Also we add the job result to the needs context to give expressions
access to it.
The needs object, failure and success functions are split between
run context (on jobs) and step context.
Closes#442
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* style: correct linter warnings
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* fix: job if value defaults to success()
As described in the documentation, a default value of "success()" is
applied when no "if" value is present on the job.
https://docs.github.com/en/actions/learn-github-actions/expressions#job-status-check-functions
Co-authored-by: Markus Wolf <mail@markus-wolf.de>
* fix: check job needs recursively
Ensure job result includes results of previous jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add runner test for job status check functions
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add unit tests for run context if evaluation
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* refactor: move if expression evaluation
Move if expression evaluation into own function (step context) to
better support unit testing.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* test: add unit tests for step context if evaluation
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: handle job error more resilient
The job error is not stored in a context map instead of a context
added value.
Since context values are immutable an added value requires to keep
the new context in all cases. This is fragile since it might slip
unnoticed to other parts of the code.
Storing the error of a job in the context map will make it more stable,
since the map is always there and the context of the pipeline is stable
for the whole run.
* feat: steps should use a default if expression of success()
* test: add integration test for if-expressions
* chore: disable editorconfig-checker for yaml multiline string
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* fix[workflow]: multiple fixes for workflow/matrix
fix[workflow]: default `max-parallel`
fix[workflow]: default `fail-fast`, it's `true`, not `false`
fix[workflow]: skipping over the job when `strategy:` is defined but `matrix:` isn't (fixes#625)
fix[workflow]: skip non-existing includes keys and hard fail on non-existing excludes keys
fix[workflow]: simplify Matrix decode (because I "think" I know how `yaml` works) (fixes#760)
fix[tests]: add test for planner and runner
* fix(workflow): use yaml node for env key
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: typo
* fix: remove debug for git references
it generates a massive amount of logs
* feat: read values from env
allows to test act on GHA when it's not a main repo
* fix: merge extrapath with PATH
* fix(tests): add additional shells for testing
* fix(image): update images
pin node to major version only, current node version: 12.22.1
replace most images with `node:12-buster-slim` to prevent
errors on macOS runner due to DockerHub pull limit
replace ocaml image
Co-authored-by: Casey Lee <cplee@nektos.com>
* Test fixups for Windows/Forks
* revert change for repo/actor/owner
this is fixed via https://github.com/nektos/act/pull/723
Co-authored-by: Ryan (hackercat) <me@hackerc.at>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* refactor: remove `gotest.tools`
* remove all references to `gotest.tools` and replace it with
`github.com/stretchr/testify` which was originally used for tests
* bump `golangci-lint` version
* add `depguard` and `importas` to prevent import of unwanted packages
* add custom schema and information about config since
schemastore.org has broken schema for `golangci-lint` config
* fix: handle more error cases
* define GITHUB_ACTION_PATH #603
* Add more environment variables
* Add job name
Note: the job name gets a suffix for matrix builds, but this is not part of the env var
* fix: remove unnecessary variables
* feat: add `RepositoryOwner`
credit: @KnisterPeter
* feat: add test for `getGithubContext()`
Co-authored-by: Ryan (hackercat) <me@hackerc.at>
* feat: bump `golangci-lint`, add `super-linter`, replace outdated linter
Bump `golangci-lint` version.
Add `super-linter` to lint other languages.
Go linter is disabled because it's currently broken:
https://github.com/github/super-linter/pull/370
Replacing `scopelint` with `exportloopref`: "[runner] The linter
'scopelint' is deprecated (since v1.39.0) due to: The repository of the
linter has been deprecated by the owner. Replaced by exportloopref."
Fixed formatting in `.golangci.yml`
Add addtional linters:
`misspell`: purely style, detects typos in comments
`whitespace`: detects leading and trailing whitespace
`goimports`: it's gofmt + checks unused imports
* fix: lint/fix `go` files
* fix: lint with `standardjs`
* fix: lint/fix with `markdownlint`, make template more verbose
* feat: add lint stuff to makefile
* fix: `UseGitIgnore` formatting
* fix: lint/fix `README.md`
Co-authored-by: Casey Lee <cplee@nektos.com>
* Comment for public function
* Add git describe fallback
* spelling: github
* Set initial branch to satisfy tests for modern git
* Clarify -even- if
* Go 1.16
* Support running commands in repositories without action.yaml
Support runnings commands with only a Docker file
Co-authored-by: Casey Lee <cplee@nektos.com>
* fix: give case insensitive secret more meanigful name
* refactor: use `string` in generating `env:` and `steps:` for workflows
Smaller text generation is much better to read with normal strings than
raw string literals.
* feat: sort keys for `env:` so it's always in specific order
* fix: update test workflows
* Test more if env variants
* The correct negation syntax is !=
* Make the Interpolate function support negated booleans from envs
* Move assert := a.New(t) into t.Run
This uncovered that some of the test premisses was wrong and the
Eval Bool function also had flaws
* Remove a stray logrus import
* Add an ACT env set to true
This can be used to skip certain steps that you don't want to run locally
when testing. E.g. steps that sends messages to Slack channels on successful
builds etc.
* Add a description about env.ACT to the readme
* A new attempt at Interpolation and EvalBool
* One small merge fix
* Remove some fmt.Printfs
* Fix some merge conflicts
* Test more if env variants
* The correct negation syntax is !=
* Make the Interpolate function support negated booleans from envs
* Move assert := a.New(t) into t.Run
This uncovered that some of the test premisses was wrong and the
Eval Bool function also had flaws
* Remove a stray logrus import
* Ignore .idea
* Add Env to the RunContext vm so we can Evaluate and Interpolate `env.xx`
* Make EvalBool support expressions more in line with the github runner
* Turns out Boolean(value) is what github is doing after all
* Add test for github context as well
