From 0597a1eed63468096f6efb778741d2fee11285c9 Mon Sep 17 00:00:00 2001 From: ScriptAutomate Date: Tue, 9 Apr 2024 17:12:48 -0500 Subject: [PATCH 1/2] Add details to docs on how to extend PR tests --- .github/PULL_REQUEST_TEMPLATE.md | 3 +- doc/topics/development/pull_requests.rst | 40 ++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 629117fe672..f06d37e9e14 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -19,6 +19,7 @@ Remove this section if not relevant ### Commits signed with GPG? Yes/No -Please review [Salt's Contributing Guide](https://docs.saltproject.io/en/master/topics/development/contributing.html) for best practices. +Please review [Salt's Contributing Guide](https://docs.saltproject.io/en/master/topics/development/contributing.html) for best practices, including the +[PR Guidelines](https://docs.saltproject.io/en/master/topics/development/pull_requests.html). See GitHub's [page on GPG signing](https://help.github.com/articles/signing-commits-using-gpg/) for more information about signing commits with GPG. diff --git a/doc/topics/development/pull_requests.rst b/doc/topics/development/pull_requests.rst index 15bcdd09cd2..5bb3efa4717 100644 --- a/doc/topics/development/pull_requests.rst +++ b/doc/topics/development/pull_requests.rst @@ -185,3 +185,43 @@ PR's that do not require tests include: * fixing tests * pylint * changes outside of the salt directory + +Pull Request CI/CD test suite +============================= + +By default, PRs run a limited subset of the test suite against the following +operating systems: + +* Linux: + - Latest ``Rocky Linux x86_64`` + - Latest ``Amazon Linux aarch64`` + - Latest ``Ubuntu LTS arm64`` + - Latest ``Arch Linux x86_64`` +* Latest ``Windows Server x86_64`` +* Latest ``MacOS arm64`` + +Optional OS additions +--------------------- + +There are times where a PR is specifically addressing a target OS, or a core component of +Salt is being updated that needs to be tested against all support operating systems. This +is often required to fix ``nightly`` builds, or introduce a new operating system into CI/CD. + +Labels can be applied to a PR, via those who have the appropriate permissions, using the +``test:os:`` format. + +* Example: ``test:os:debian-11-arm64`` would also ensure the Debian 11 arm64 OS is included. + +Optionally, ``test:os:all`` can be used as a way to target all operating systems +that nightlies, staging, release target without having to add them all individually. + +Optional test additions +----------------------- + +If wanting to also increase the scope of tests themselves, not just the scope of operating +systems, then additional labels can be used such as: + +* ``test:pkg``: Run all package-related tests that are otherwise not included by default + on the currently selected operating systems. +* ``test:full``: Run all tests in the Salt test suite that can be ran on the currently + selected operating systems. From 054269fbaa4a5ea585afa0eae5514e3f957ee961 Mon Sep 17 00:00:00 2001 From: Pedro Algarvio Date: Fri, 12 Apr 2024 12:46:32 +0100 Subject: [PATCH 2/2] Add information about the `test:random-cache-seed` and `test:coverage` labels --- doc/topics/development/pull_requests.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/topics/development/pull_requests.rst b/doc/topics/development/pull_requests.rst index 5bb3efa4717..4b6ffee9135 100644 --- a/doc/topics/development/pull_requests.rst +++ b/doc/topics/development/pull_requests.rst @@ -225,3 +225,7 @@ systems, then additional labels can be used such as: on the currently selected operating systems. * ``test:full``: Run all tests in the Salt test suite that can be ran on the currently selected operating systems. +* ``test:coverage``: Run all tests in the Salt test suite on all operating systems while + collecting code coverage. +* ``test:random-cache-seed``: Randomize the GH Actions cache seed to make sure no caches + are used during the build.