Commit graph

1580 commits

Author SHA1 Message Date
jeanluc
56ad1ec942 Fix saltext vault/pushover docs
* the correct organization is `salt-extensions`, not `saltstack`
* the release file is strict GitHub-flavoured Markdown only
2023-12-18 03:42:28 -07:00
Roald Nefs
60b890b800 Fix pre-commit
Signed-off-by: Roald Nefs <info@roaldnefs.com>
2023-12-16 18:26:29 -07:00
Wayne Werner
8dc4160a6a Fix potential bug, add changelog
In the source for redis, the default value for password is None, not "".
If the default is an empty string that may cause some unintended
consequences -- though it's entirely possible that Redis does not
support empty passwords, it's always safer to match the underlying
existing codebase.
2023-12-16 18:26:29 -07:00
vzhestkov
a5599d6c89 Add changelog entry 2023-12-16 14:48:10 -07:00
Michael Calmer
745e5f00f5 add changelog 2023-12-16 04:49:17 -07:00
Daniel A. Wozniak
37d4aa2615 Add changelog for #55687 2023-12-16 04:34:23 -07:00
jeanluc
c8c0633488 Bring code up to current standards 2023-12-16 04:32:10 -07:00
jeanluc
0ff2d2b7a8 Add signature verification to file.managed/archive.extracted 2023-12-16 04:32:10 -07:00
Michael Calmer
8a92dfca61 fixup! add changelog 2023-12-16 04:31:08 -07:00
Michael Calmer
cebe1e6cf4 add changelog 2023-12-16 04:31:08 -07:00
jeanluc
8da445b9f4 Make x509_v2 compound match detection use match runner 2023-12-16 01:27:15 -07:00
nicholasmhughes
a66c4fc174 fixes saltstack/salt#53982 add ability to use file.managed style check_cmd in file.serialize 2023-12-16 01:19:19 -07:00
Gareth J. Greenaway
5707801e8c Adding a changelog. 2023-12-16 01:17:47 -07:00
nicholasmhughes
a05205bb4f use sentence case instead of title case for headings 2023-12-15 21:46:44 -07:00
nicholasmhughes
93e906203d add changelog 2023-12-15 21:46:44 -07:00
jeanluc
f2121e5ade
Rewrite vault core, issue AppRoles to minions (#62684)
* Rewrite vault core, orchestrate AppRoles for minions

This commit represents a fundamental rewrite in how Salt interacts with
Vault. The master should still be compatible with minions running the
old code. There should be no breaking changes to public interfaces and
the old configuration format should still apply.

Core:
- Issue AppRoles to minions
- Manage entities with templatable metadata for minions
- Use inbuilt Salt cache
- Separate config cache from token cache
- Cache: introduce connection-scope vs global scope

Utility module:
- Support being imported (__utils__ deprecation)
- Raise exceptions on queries to simplify response handling
- Add classes to wrap complexity, especially regarding KV v2
- Lay some groundwork for renewing tokens

Execution module:
- Add patch_secret
- Add version support to delete_secret
- Allow returning listed keys only in list_secret
- Add policy_[fetch/write/delete] and policies_list
- Add query for arbitrary API queries

State module:
- Make use of execution module
- Change output format

Docs:
- Update for new configuration format
- Correct examples
- Add configuration examples
- Add required policies

* Fix linting for rewritten vault integration

* Add pytest unit tests for utils.vault, fix found issues

* Fix old vault runner tests

* Rewrite vault sdb tests, migrate to pytests

* Adapt vault ext_pillar tests

* Adapt vault execution module tests, migrate to pytests

* Add more vault execution module unit tests

* Support python <3.7 (vault util), time-independent tests

* Add/migrate vault runner unit tests (pytest)

* Add vault state module pytests

* Fix tests lint

* Refactor Vault container fixture, move to session scope

* Fix for existing vault execution/sdb module integration tests

* Improve existing vault runner integration tests

* Fix vault test support, add list policies

* Add more functional execution module tests, fix deprecated warning

* Refactor vault pytest support

* Add integration tests, improve/fix caching/issue_params

* Improve caching behavior, fix tests

* Always use session cache as well
* Also flush session cache when requested
* Make KV metadata caching behavior configurable
* Update tests to account for changes from prev commit

* Allow to autodiscover platform default CA bundle

* Remove runner approle param overrides

There is no simple way to ensure they are kept.

* Add clear_cache runner function

* Also manage token metadata for issued secret IDs

* Cleanup tests

* Cleanup code, pylint logging suggestions

* Do not always invalidate config when verify=default

* Ensure concatted metadata lists are sorted

* Add changelog (partly)

* Work with legacy peer_run configuration as well

* Consume a token use regardless of status code

* Correct verify semantics

* Refine token uses handling, add changelog/tests for old issues

* Add changelog for main features

* Add test for issue 58580

* Fix vault docs

* Provide all old make_request functionality, add tests

* Allow token use override, add docstrings to query funcs

* Simplify config_location merge

* Cleanup

* Fix make_request warning

* Attempt to fix memory issues during CI test run

* Increase documented version

* Improve lease handling

* Refine lease ttl handling/add token lifecycle management

* Fix docs build

* Adapt formatting

* assert what you get against what you expect
* drop empty parentheses after wrapper
* use `is` to compare against strictly boolean vars

* Fix issue param overrides

* during pillar rendering, they were always reset by the master (for
  AppRoles)
* overrides were only respected for some settings (AppRoles)
* old config syntax was using the old syntax internally (tech debt)

* Introduce session-scoped cache

* Tokens with a single use left are unrenewable

* Allow override of flushing of cached leases during lookup

* Refactor cache classes, save lease data

* Rename session token cache key

* Add lease management utility

* Fix runner integration tests

after renaming the token cache key

* Do not overwrite data of cached leases after renewal

* Pass token_lifecycle to minions

* Do not fail syncing multiple approles/entities with pillar templates

* Ensure config cache expiration can be disabled

* Rename changelog files (.md)

* Declare vaultpolicylexer as parallel read safe

* Correct meta[data] payload key

For tokens it is `meta`, but for secret IDs, `metadata`.

* Reuse TCP connection

* Refactor utils module

* Ensure client is recreated after clearing cache

* Always use unwrap_client config as expected server

This should fix the test failure in the runner integration test
TestAppRoleIssuance::test_server_switch_does_not_break_minion_auth

* Ensure client is recreated after clearing cache 2

* Simulate patch for KV v1 or missing `patch` capability

* Add `patch` option to Vault SDB driver

* Reduce lease validity when revocation fails

* Extract AppRole/Identity API from runner into utils

* Revoke tokens, fire events, improve cache/exception handling

* Tokens (and therefore associated leases) are revoked when cleared by default
* It's possible to disable clearing cache when a perfectly valid token
  is available, but a PermissionDeniedError is encountered.
* UnwrapExceptions always cause an event to be fired
* It's possible to enable sending of events when
    a) cache is cleared
    b) a lease is requested from cache, but it is/will be invalid
* A VaultAuthException does not immediately lead to clearing
  the connection cache
* get_authd_client and others: multiple small enhancements and fixes

* Allow updating cached config w/o closing session

* Homogenize funcs, update docs, cleanup

* Minor internal fixes

`is_valid_for` is present on all lease-like objects, while `is_valid`
specifically should account for more, e.g. the number of uses.

The Vault API does not return 404 when a lookup fails.

* Add release note

* Address review remarks

* Fix release notes

* Remove loading minion_mods from factory

* Address other review remarks

* Add inline specification of trusted CA root cert

* Small QoL additions

* Fix lint

* Fix lint for Python >=3.8 support

* Add missing fixes

* Fix unit tests

In some cases, the `spec` calls were failing because the underlying
object was already patched

---------

Co-authored-by: Thomas Phipps <tphipps@vmware.com>
2023-12-15 21:42:08 -07:00
Alan Somers
a11b4d5252 Add a changelog entry 2023-12-15 21:33:09 -07:00
Zhiwei Liang
17076c1a8f Implement salt cloud function set_backup_schedule for Linode VMs 2023-12-15 21:20:14 -07:00
Zhiwei Liang
e9cb14c91e Add backups_enabled option; improve docs 2023-12-15 01:17:27 -07:00
Daniel Wozniak
cd12ba1171
Merge pull request #65699 from s0undt3ch/hotfix/merge-forward
[master] Merge 3006.x into master
2023-12-15 01:16:58 -07:00
Pedro Algarvio
e3ba31dc7a
Merge 3006.x into 3007.x 2023-12-14 11:32:20 +00:00
Shane Lee
9501122807 Fix an issue with the openscap module
Fixes an issue with the openscap module where it would fail if the
command returned an exit code not found in the exit_codes_map
dictionary. Instead of failing, let's return false with the exit code
and error message in the return.
2023-12-12 15:26:32 -07:00
jeanluc
46e7afa3ef Fix salt-ssh with saltutil.cmd and in orchestration 2023-12-12 15:25:32 -07:00
nicholasmhughes
84f434cc76 fix extfs parameter and retcode handling 2023-12-12 15:22:55 -07:00
Salt Project Packaging
0472fd381e
Release v3006.5 2023-12-12 18:03:35 +00:00
jeanluc
caea12ed2d Allow accessing the regular mine/event bus from salt-ssh 2023-12-11 20:28:20 -07:00
David Murphy
a04d09ec10
Merge pull request #65351 from garethgreenaway/18907_lazy_unmount_when_fails
[master] Add fallback for when "remounting"  NFS or FUSE initially fails
2023-12-11 16:19:38 -07:00
jeanluc
3c2e1ba1fb Fixes after rebase 2023-12-11 14:05:22 -07:00
jeanluc
81b1236c5e Add changelog 2023-12-11 14:05:22 -07:00
David Murphy
a3f5ff9a7b
Merge branch 'master' into 18907_lazy_unmount_when_fails 2023-12-10 19:58:35 -07:00
Krzysztof Pawłowski
30690637b5 Return error if patch file passed to state file.patch is malformed
If patch file provided for file.patch state is malformed then state
returns `Patch was already applied` but patch is not applied.

          ID: patch_example
    Function: file.patch
        Name: /tmp/example
      Result: True
     Comment: Patch was already applied
     Started: 12:20:50.953163
    Duration: 61.558 ms
     Changes:

It is better to return error in such case.

          ID: patch_example
    Function: file.patch
        Name: /tmp/example
      Result: False
     Comment: /usr/bin/patch: **** malformed patch at line 7:
     Started: 12:33:44.915605
    Duration: 59.202 ms
     Changes:
2023-12-10 15:32:15 -07:00
Daniel A. Wozniak
9e95e3b2bf Fix changelog entry 2023-12-10 14:55:32 -07:00
piterpunk
913c0f7566 Removed an unused assignment in file.patch 2023-12-10 03:31:10 -07:00
David Murphy
5b23915205
Merge branch 'master' into 18907_lazy_unmount_when_fails 2023-12-08 10:43:24 -07:00
David Murphy
acbc56781b
Merge branch 'master' into 65080_require_included_sls 2023-12-08 08:40:43 -07:00
Daniel Wozniak
c9afe46ad9
Merge pull request #65553 from s0undt3ch/hotfix/merge-forward
[master] Merge 3006.x into master
2023-12-07 16:35:19 -07:00
Pedro Algarvio
967d3015f6
Merge 3006.x into master 2023-12-05 18:10:58 +00:00
Daniel A. Wozniak
21a3f6aa30 Add changelog for 65450 2023-12-04 14:20:01 -07:00
David Murphy
60a4fdb0af
Merge branch 'master' into 65080_require_included_sls 2023-12-04 13:42:31 -07:00
David Murphy
fd1cd1dcac
Merge branch 'master' into 18907_lazy_unmount_when_fails 2023-12-04 13:41:44 -07:00
David Murphy
cd279bd7d4 Fill out lsb_distrib_xxx grains (best guess) if lsb_release information unavailable 2023-11-29 21:40:06 +00:00
Pedro Algarvio
02c3b89e40
Merge 3006.x into master 2023-11-29 18:31:52 +00:00
Pedro Algarvio
28039bda69 Bump to cryptography==41.0.7 due to https://github.com/advisories/GHSA-jfhm-5ghh-2f97
Signed-off-by: Pedro Algarvio <palgarvio@vmware.com>
2023-11-29 10:57:19 -07:00
Pedro Algarvio
7cf72a7797
Merge 3006.x into master 2023-11-29 14:43:39 +00:00
butch12
8235409c51 maintain user-defined options in apt source definitions 2023-11-28 20:02:59 +00:00
Pedro Algarvio
f60c87aacb
Merge 3006.x into master 2023-11-28 17:46:50 +00:00
jeanluc
126cbc30fb Make SSH shell report exitcode to the best of its ability 2023-11-28 10:44:24 +00:00
jeanluc
92b46fb4ff Fix salt-ssh stacktrace when retcode is not an integer 2023-11-28 10:44:24 +00:00
vzhestkov
1e0c9d71c8 Add changelog entry 2023-11-26 15:27:39 -07:00
jeanluc
5adb7ec17b Fix salt-ssh master access during pillar rendering
This also ports #50489 into the present
2023-11-26 17:25:38 +00:00