Commit graph

21000 commits

Author SHA1 Message Date
Daniel A. Wozniak
d9b50659b2 Pyupgrade and drop six 2021-07-06 09:04:28 -04:00
Pedro Algarvio
de6f56ccaf No hacks when interacting with ansible. Access all available modules, not just internal. 2021-06-30 11:39:28 -07:00
Pedro Algarvio
d83dd0319f The `ansiblegate` module now correctly passes keyword arguments to Ansible module calls
Fixes #59792
2021-06-30 11:39:28 -07:00
Pedro Algarvio
cedc24249c Migrate unit.states.test_ansiblegate to PyTest 2021-06-30 11:39:28 -07:00
Pedro Algarvio
2d2c1c2005 Migrate the ansiblegate state integration tests to pytest 2021-06-30 11:39:28 -07:00
Pedro Algarvio
241009d4fa Fix calling ansible modules.
They cannot be called by passing their path, for example:
```
Traceback (most recent call last):
  File ".nox/pytest-parametrized-3-7-crypto-none-transport-zeromq-coverage-false/lib/python3.7/site-packages/ansible/modules/ping.py", line 63, in <module>
    from ansible.module_utils.basic import AnsibleModule
  File "/home/vampas/projects/SaltStack/salt/hotfix/ansiblegate-59792/.nox/pytest-parametrized-3-7-crypto-none-transport-zeromq-coverage-false/lib/python3.7/site-packages/ansible/module_utils/basic.py", line 53, in <module>
    import tempfile
  File "/home/vampas/projects/SaltStack/salt/hotfix/ansiblegate-59792/.nox/pytest-parametrized-3-7-crypto-none-transport-zeromq-coverage-false/lib/python3.7/site-packages/ansible/modules/tempfile.py", line 79, in <module>
    from tempfile import mkstemp, mkdtemp
ImportError: cannot import name 'mkstemp' from 'tempfile' (/home/vampas/projects/SaltStack/salt/hotfix/ansiblegate-59792/.nox/pytest-parametrized-3-7-crypto-none-transport-zeromq-coverage-false/lib/python3.7/site-packages/ansible/modules/tempfile.py)
```
2021-06-30 11:39:28 -07:00
Pedro Algarvio
e466589a6e Inject the ansible functions into Salt's ansiblegate module.
This was broken in 3001(92438882a9 (diff-519146c38c0ed7bc961ffc5f408d3dd380abb8760d48e839743dbd38dc28ba15R323)).

Fixes #60207
2021-06-30 11:39:28 -07:00
piterpunk
fc352b4b1e Updates hosts.rm_host tests 2021-06-30 13:37:43 -04:00
Gareth J. Greenaway
88769f2058 Removing extra newline from changelog. removing debugging messages from conftest.py. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
9c5b2e369a removing debugging lines. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
9b29bcf285 Updating all rabbitmq tests to force them to run. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
450afc1a18 using rabbitmqctl status as the indicator that the rabbitmq server is up and running. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
331760bf5a Additinal logging. Running rabbitmqctl status to see the status of rabbitmq instead of just a ping. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
1dcf7362dd Adding some additional debugging to see why container is not ready. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
6592ea0b6c Swapping out "from saltfactories.factories.daemons.container import ContainerFactory" for "from saltfactories.daemons.container import Container". Updating warning to use reason instead of warning. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
3f68173a9c moving common fixtures to tests/pytests/functional/states/rabbitmq/conftest.py, moving tests to tests/pytests/functional/states/rabbitmq 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
ce4fe0eb74 switching to using --formatter=json and working on some functional tests using docker containers. 2021-06-30 13:37:22 -04:00
Gareth J. Greenaway
caa72102c8 Fixing rabbitmq.list_user_permissions to ensure we are returning a permission list with three elements even when some values are empty. 2021-06-30 13:37:22 -04:00
Pedro Algarvio
03669b0161 Ensure proper access to the created temporary file
Fixes #60072
2021-06-28 11:47:56 -07:00
Pedro Algarvio
1c743afb48 Migrate unit.modules.test_gpg to PyTest 2021-06-28 13:53:56 -04:00
Pedro Algarvio
2c034cab47 Mirgate integration.states.test_zookeeper to Pytest functional tests 2021-06-28 13:36:47 -04:00
Pedro Algarvio
f0df76fcd6 Functional test modules can now provide `minion_config_{defaults,overrides}` 2021-06-28 13:36:47 -04:00
David Murphy
98e3a8e566 Added test keywords which should be ignored by IPv4 and IPv6 2021-06-28 13:26:28 -04:00
David Murphy
2f6e204fa2 Updated UIPv6 output for test to include default route 2021-06-28 13:26:28 -04:00
David Murphy
6202152494 Force test to pick _ip_route_linux for default_route_ipv6 2021-06-28 13:26:28 -04:00
David Murphy
790d49b5c2 Fixed typo in test for default_route_ipv6 2021-06-28 13:26:28 -04:00
David Murphy
6594784702 Additional tests for default_route with IPv6 and pass family to routes 2021-06-28 13:26:28 -04:00
David Murphy
5b8ca47bb0 Used pytest.fixtures as per reviewer's comments 2021-06-28 11:36:33 -04:00
David Murphy
123cb811ee Made test comparison of lists sorted, some platforms returned in different order then expected 2021-06-28 11:36:33 -04:00
David Murphy
83b7e1b9d1 Fix for runner network.wolmatch producing invalid arguements error 2021-06-28 11:36:33 -04:00
twangboy
15f2c879b7 Add tests 2021-06-25 13:50:33 -04:00
David Murphy
290f8b6fe1 Fix to allow netmiko_mod load unless proxy and deltaproxy active 2021-06-24 13:57:27 -04:00
Daniel A. Wozniak
d9d0a6806f Fix pre commit 2021-06-24 13:33:23 -04:00
Daniel A. Wozniak
d19de8d67f Fix linter 2021-06-24 13:33:23 -04:00
Daniel A. Wozniak
cbbee775c0 Set the name of engine processes #60259 2021-06-24 13:33:23 -04:00
Gareth J. Greenaway
0e4c7b02cb Removing unsupported version of Salt from tests.pytests.scenarios.compat.test_with_versions.py 2021-06-24 13:25:37 -04:00
David Murphy
91e12297a7 Reinstated Port option for Junos Proxy with tests 2021-06-24 11:44:33 -04:00
Gediminas Zlatkus
cb1917b0f4 Fix ValueError exception in state.show_state_usage 2021-06-23 13:00:23 -04:00
Megan Wilhite
90ed88638a
Merge freeze into master (#60396)
* Merge 3002.6 bugfix changes (#59822)

* Pass `CI_RUN` as an environment variable to the test run.

This allows us to know if we're running the test suite under a CI
environment or not and adapt/adjust if needed

* Migrate `unit.setup` to PyTest

* Backport ae36b15 just for test_install.py

* Only skip tests on CI runs

* Always store git sha in _version.py during installation

* Fix PEP440 compliance.

The wheel metadata version 1.2 states that the package version MUST be
PEP440 compliant.

This means that instead of `3002.2-511-g033c53eccb`, the salt version
string should look like `3002.2+511.g033c53eccb`, a post release of
`3002.2` ahead by 511 commits with the git sha `033c53eccb`

* Fix and migrate `tests/unit/test_version.py` to PyTest

* Skip test if `easy_install` is not available

* We also need to be PEP440 compliant when there's no git history

* Allow extra_filerefs as sanitized kwargs for SSH client

* Fix regression on cmd.run when passing tuples as cmd

Co-authored-by: Alexander Graul <agraul@suse.com>

* Add unit tests to ensure cmd.run accepts tuples

* Add unit test to check for extra_filerefs on SSH opts

* Add changelog file

* Fix comment for test case

* Fix unit test to avoid failing on Windows

* Skip failing test on windows

* Fix test to work on Windows

* Add all ssh kwargs to sanitize_kwargs method

* Run pre-commit

* Fix pylint

* Fix cmdmod loglevel and module_names tests

* Fix pre-commit

* Skip ssh tests if binary does not exist

* Use setup_loader for cmdmod test

* Prevent argument injection in restartcheck

* Add changelog for restartcheck fix

* docs_3002.6

* Add back tests removed in merge

Co-authored-by: Pedro Algarvio <pedro@algarvio.me>
Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com>
Co-authored-by: Bryce Larson <brycel@vmware.com>
Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com>
Co-authored-by: Alexander Graul <agraul@suse.com>
Co-authored-by: Frode Gundersen <fgundersen@saltstack.com>

* Remove glance state module in favor of glance_image

* update wording in changelog

* bump deprecation warning to Silicon.

* Updating warnutil version to Phosphorous.

* Update salt/modules/keystone.py

Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com>

* Check $HOMEBREW_PREFIX when linking against libcrypto

When loading `libcrypto`, Salt checks for a Homebrew installation of `openssl`
at Homebrew's default prefix of `/usr/local`. However, on Apple Silicon Macs,
Homebrew's default installation prefix is `/opt/homebrew`. On all platforms,
the prefix is configurable.  If Salt doesn't find one of those `libcrypto`s,
it will fall back on the un-versioned `/usr/lib/libcrypto.dylib`, which will
cause the following crash:

    Application Specific Information:
    /usr/lib/libcrypto.dylib
    abort() called
    Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

This commit checks $HOMEBREW_PREFIX instead of hard-coding `/usr/local`.

* Add test case

* Add changelog for 59808

* Add changelog entry

* Make _find_libcrypto fail on Big Sur if it can't find a library

Right now, if `_find_libcrypto` can't find any externally-managed versions of
libcrypto, it will fall back on the pre-Catalina un-versioned system libcrypto.
This does not exist on Big Sur and it would be better to raise an exception
here rather than crashing later when trying to open it.

* Update _find_libcrypto tests

This commit simplifies the unit tests for _find_libcrypto by mocking out the
host's filesystem and testing the common libcrypto installations (brew, ports,
etc.) on Big Sur. It simplifies the tests for falling back on system versions
of libcrypto on previous versions of macOS.

* Fix description of test_find_libcrypto_with_system_before_catalina

* Patch sys.platform for test_rsax931 tests

* modules/match: add missing "minion_id" in Pillar example

The documented Pillar example for `match.filter_by` lacks the `minion_id` parameter. Without it, the assignment won't work as expected.
- fix documentation
- add tests:
  - to prove the misbehavior of the documented example
  - to prove the proper behaviour when supplying `minion_id`
  - to ensure some misbehaviour observed with compound matchers doesn't occur

* Fix for issue #59773

- When instantiating the loader grab values of grains and pillars if
  they are NamedLoaderContext instances.
- The loader uses a copy of opts.
- Impliment deepcopy on NamedLoaderContext instances.

* Add changelog for #59773

* _get_initial_pillar function returns pillar

* Fix linter issues

* Clean up test

* Bump deprecation release for neutron

* Uncomment Sulfur release name

* Removing the _ext_nodes deprecation warning and alias.

* Adding changelog.

* Renaming changelog file.

* Update 59804.removed

* Initial pass at fips_mode config option

* Fix pre-commit

* Fix tests and add changelog

* update docs 3003

* update docs 3003 - newline

* Fix warts in changelog

* update releasenotes 3003

* add ubuntu-2004-amd64 m2crypto pycryptodome and tcp tests

* add distro_arch

* changing the cloud platforms file missed in 1a9b7be0e2

* Update __utils__ calls to import utils in azure

* Add changelog for 59744

* Fix azure unit tests and move to pytest

* Use contextvars from site-packages for thin

If a contextvars package exists one of the site-packages locations use
it for the generated thin tarball. This overrides python's builtin
contextvars and allows salt-ssh to work with python <=3.6 even when the
master's python is >3.6 (Fixes #59942)

* Add regression test for #59942

* Add changelog for #59942

* Update filemap to include test_py_versions

* Fix broken thin tests

* Always install the `contextvars` backport, even on Py3.7+

Without this change, salt-ssh cannot target systems with Python <= 3.6

* Use salt-factories to handle the container. Don't override default roster

* Fix thin tests on windows

* No need to use warn log level here

* Fix getsitepackages for old virtualenv versions

* Add explicit pyobjc reqs

* Add back the passthrough stuff

* Remove a line so pre-commit will run

* Bugfix release docs

* Bugfix release docs

* Removing pip-compile log files

* Bump requirements to address a few security issues

* Address traceback on macOS

```
Traceback (most recent call last):
  File "setup.py", line 1448, in <module>
    setup(distclass=SaltDistribution)
  File "/Users/jenkins/setup-tests/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/salt/lib/python3.7/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "setup.py", line 1068, in __init__
    self.update_metadata()
  File "setup.py", line 1074, in update_metadata
    attrvalue = getattr(self, attrname, None)
  File "setup.py", line 1182, in _property_install_requires
    install_requires += _parse_requirements_file(reqfile)
  File "setup.py", line 270, in _parse_requirements_file
    platform.python_version(), _parse_op(op), _parse_ver(ver)
  File "setup.py", line 247, in _check_ver
    return getattr(operator, "__{}__".format(op))(pyver, wanted)
  File "/opt/salt/lib/python3.7/distutils/version.py", line 46, in __eq__
    c = self._cmp(other)
  File "/opt/salt/lib/python3.7/distutils/version.py", line 337, in _cmp
    if self.version < other.version:
TypeError: '<' not supported between instances of 'str' and 'int'
```

* Replace `saltstack.com` with `saltproject.io` on URLs being tested

* Add back support to load old entrypoints by iterating instead of type checking

Fixes #59961

* Fix issue #59975

* Fix pillar serialization for jinja #60083

* Fix test

* Add changelog for #60083

* Update changelog and release for 3003.1

* Remove the changelog source refs

* Add connect to IPCMessageSubscriber's async_methods

Fixes #60049 by making sure an IPCMessageSubscriber that is wrapped by
SyncWrapper has a connect method that runs the coroutine rather than
returns a fugure.

* Add changelog for #60049

* Update 60049.fixed

* Fix coroutine spelling error

Co-authored-by: Wayne Werner <waynejwerner@gmail.com>

* IPC on windows cannot use socket paths

Fixes #60298

* Update Jinja2 and lxml due to security related bugfix releases

Jinja2
------

CVE-2020-28493
moderate severity
Vulnerable versions: < 2.11.3
Patched version: 2.11.3

This affects the package jinja2 from 0.0.0 and before 2.11.3. The ReDOS vulnerability of the regex is mainly due to the sub-pattern [a-zA-Z0-9.-]+.[a-zA-Z0-9.-]+ This issue can be mitigated by Markdown to format user content instead of the urlize filter, or by implementing request timeouts and limiting process memory.

lxml
----

CVE-2021-28957
moderate severity
Vulnerable versions: < 4.6.3
Patched version: 4.6.3

An XSS vulnerability was discovered in the python lxml clean module versions before 4.6.3. When disabling the safe_attrs_only and forms arguments, the Cleaner class does not remove the formaction attribute allowing for JS to bypass the sanitizer. A remote attacker could exploit this flaw to run arbitrary JS code on users who interact with incorrectly sanitized HTML. This issue is patched in lxml 4.6.3.

* fix github actions jobs on branch until bullseye comes out

* Upgrade to `six==1.16.0` to avoid problems on CI runs

```
13:59:02  nox > Session invoke-pre-commit was successful.
13:59:02  nox > Running session invoke-pre-commit
13:59:02  nox > pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt
13:59:02  Collecting blessings==1.7
13:59:02    Using cached blessings-1.7-py3-none-any.whl (18 kB)
13:59:02  Collecting invoke==1.4.1
13:59:02    Using cached invoke-1.4.1-py3-none-any.whl (210 kB)
13:59:02  Collecting pyyaml==5.3.1
13:59:02    Using cached PyYAML-5.3.1.tar.gz (269 kB)
13:59:02  Collecting six==1.15.0
13:59:02    Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
13:59:02  Building wheels for collected packages: pyyaml
13:59:02    Building wheel for pyyaml (setup.py) ... - \ | / - \ | done
13:59:02    Created wheel for pyyaml: filename=PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl size=546391 sha256=e42e1d66cc32087f4d33ceb81268c86b59f1a97029b19459f91b8d6ad1430167
13:59:02    Stored in directory: /var/jenkins/.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653
13:59:02  Successfully built pyyaml
13:59:02  Installing collected packages: six, pyyaml, invoke, blessings
13:59:02    Attempting uninstall: six
13:59:02      Found existing installation: six 1.16.0
13:59:02      Uninstalling six-1.16.0:
13:59:02  ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/var/jenkins/.cache/pre-commit/repomw8oee1s/py_env-python3/lib/python3.7/site-packages/__pycache__/six.cpython-37.pyc'
13:59:02
13:59:02  nox > Command pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt failed with exit code 1
13:59:02  nox > Session invoke-pre-commit failed.
```

* add changelog for https://github.com/saltstack/salt/issues/59982

* Regression test for #56273

* Fix race condition in batch. #56273

* Add changelog for #56273

* Update salt/client/__init__.py

Co-authored-by: Pedro Algarvio <pedro@algarvio.me>

* Update doc for salt/client

* Update changelog/56273.fixed

Thoreau said, "Simplify, Simplify"

* Update docs

* Update docs

* Update CHANGELOG.md

* Update 3003.1.rst

* Fix changelog

Co-authored-by: Daniel Wozniak <dwozniak@saltstack.com>
Co-authored-by: Pedro Algarvio <pedro@algarvio.me>
Co-authored-by: Bryce Larson <brycel@vmware.com>
Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com>
Co-authored-by: Alexander Graul <agraul@suse.com>
Co-authored-by: Frode Gundersen <fgundersen@saltstack.com>
Co-authored-by: Gareth J. Greenaway <gareth@saltstack.com>
Co-authored-by: Gareth J. Greenaway <gareth@wiked.org>
Co-authored-by: Hoa-Long Tam <hoalong@apple.com>
Co-authored-by: krionbsd <krion@freebsd.org>
Co-authored-by: Elias Probst <e.probst@ssc-services.de>
Co-authored-by: Daniel A. Wozniak <dwozniak@vmware.com>
Co-authored-by: Frode Gundersen <frogunder@gmail.com>
Co-authored-by: twangboy <slee@saltstack.com>
Co-authored-by: twangboy <leesh@vmware.com>
Co-authored-by: ScriptAutomate <derek@icanteven.io>
Co-authored-by: Wayne Werner <waynejwerner@gmail.com>
2021-06-23 12:46:52 -04:00
Daniel A. Wozniak
7cfa18ae01 Remove un-used import 2021-06-23 08:11:53 -04:00
Daniel A. Wozniak
d0141cc316 Address leaks in fileserver caused by git backends
At this time we do not have the ability to fix the upstream memory leaks
in the gitfs backend providers. Work around their limitations by
periodically restarting the file server update proccess. This will at
least partially address #50313
2021-06-23 08:11:53 -04:00
Pedro Algarvio
5ae3bc9821 Disable macOS Mojave log forwarding on the proxy minion tests
Fixes https://github.com/saltstack/salt/issues/60201
2021-06-22 13:40:42 -04:00
Pedro Algarvio
0c29595fb6 Migrate unit.proxy.test_netmiko_px to PyTest 2021-06-21 06:49:49 -04:00
Pedro Algarvio
679bbe9a86 Proxy tests only exists under pytests/. Delete unused proxy conf/code. 2021-06-21 06:49:49 -04:00
Pedro Algarvio
cc07d99222 Bump to pytest-salt-factories 0.906.x 2021-06-21 06:49:49 -04:00
Cédric Bosdonnat
e3c52e5fba Handle volumes on stopped pools in virt.vm_info
For VMs having at least a disk on a stopped volume, we don't want the
user to get an exception when running virt.vm_info. Instead just provide
less information.
2021-06-04 08:41:02 -04:00
Pedro Algarvio
4f49389e6d IPC on windows cannot use socket paths
Fixes #60298
2021-06-03 12:19:35 -04:00
twangboy
40d24844cb Remove unused import 2021-06-02 06:48:42 -04:00
twangboy
bcb9065853 pre-commit 2021-06-02 06:48:42 -04:00
twangboy
199f9ace07 Add some unit tests for install 2021-06-02 06:48:42 -04:00