saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-16 09:40:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge forward from 3004.1 (#61888)

Browse source 
* Redirect imports of ``salt.ext.six`` to ``six``

Fixes #60966

* Latest changelog update for 3004

* Handle signals and properly exit, instead of raising exceptions.

This was introduced in 26fcda5074

Fixes #60391
Fixes #60963

* Add test for #61003

* Fix #61003

Restored the previously shifted check for version_to_remove in
old[target]. This had been extracted along with the correctly extracted
double pkg_params[target] lookup, but that lost the `target in old`
guard.

Putting the check back here prevents KeyError when looking for a
non-existent target in `old`.

* Handle various architecture formats in aptpkg module

* Write file even if does not exist

* only run test on debian based platforms

* remove extra space for arch

* convert pathlib to string for pkgrepo test

* Use temporary files first then copy to sources files

* fixes saltstack/salt#59182 fix handling of duplicate keys in rest_cherrypy data

* added changelog

* remove log messages to prevent leaks of sensitive info

* Reverting changes in PR #60150. Updating installed and removed functions to return changes when test=True.

* Adding changelog.

* Add a test and fix for extra-filerefs

* Do not break master_tops for minion with version lower to 3003

* Add changelog file

* Add extra comment to clarify discussion

* Update changelog file

* Add deprecated changelog

* Assert that the command didn't finish

Refs https://github.com/saltstack/salt/pull/60972

* Always restore signals, even when exceptions occur

* Reset signal handlers before starting the process

* Make sure that the `ProcessManager` doesn't always ignore signals

* Provide valid default value for bootstrap_delay

* Update changelog for 3004

* Update changelog and release notes for 3004

* Add PR 61020 to changelog

* Change MD5 to SHA256 fingerprint for new github.com fingerprint

* Check only ssh-rsa encyption for set_known_host

* Use main branch for kitchen-docker project

* Add tests for validate_tgt

This function evolved over the years, but never had any tests. We're
adding tests now to cover the various cases:

- there are no valid minions (currently fails, should return False)
- there are target minions that aren't in valid minions (correctly
  fails)
- target minions are a subset of valid minions (i.e. all of the target
  minions are found in the valid minions -- there are no extras)
  (correctly passes)

* Refactor

minions should be a subset of v_minions - the extra code was just
getting in the way. Also, this function evolved over time but the
docstring never kept up. Updated the docstring to more accurately
describe the function's behavior.

* Fix #60413

When using a syndic and user auth, it was possible for v_minions and
minions to be two empty sets, which returned True. This allowed the user
to still publish the function. The Syndic would get the published event
and apply it, even though it should have been rejected.

However, if there are no valid minions, then it doesn't matter what the
targets are -- there are not valid targets, so there's no reason to do
any further checks.

* Rename changelog to security

* add cve# to changelog

* Sign pillar data

* Add regression tests for CVE-2022-22934

* Add changelog for cve-2022-22934

* Provide users with a nice warning when something goes wrong

* Rename changelog file

* Fix wart in tests

* Return bool when using m2crypo

* Limit the amount of empty space while searching ifconfig output

* Update changelog/cve-2020-22937.security

Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com>

* Prevent auth replays and sign replies

* Add tests for cve-2022-22935

* Add changelog for cve-2020-22935

* Fix typo

* Prevent replays of file server requests

* Add regresion tests for fileserver nonce

* Add changelog for cve-2022-22936

* Job replay mitigation

* Fix merge warts

* more test fixes

* Fix auth tests on windows

* Remove unwanted requirements change

* Clean up cruft

* update docs for 3004.1 release

* Fix warts in new minion auth

* Test fix

* Update release notes

* Remove cve from non cve worty issue

* Add serial to payload in publisher process

* Fix channel tests

Fix broken channel tests by populating an AES key and serial.

* Windows test fix

* windows tests plz work

Co-authored-by: Pedro Algarvio <pedro@algarvio.me>
Co-authored-by: ScriptAutomate <derek@icanteven.io>
Co-authored-by: Wayne Werner <wwerner@vmware.com>
Co-authored-by: Megan Wilhite <mwilhite@vmware.com>
Co-authored-by: nicholasmhughes <nicholasmhughes@gmail.com>
Co-authored-by: Gareth J. Greenaway <gareth@saltstack.com>
Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com>
Co-authored-by: Alyssa Rock <arock@saltstack.com>
Co-authored-by: krionbsd <krion@FreeBSD.org>
Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com>
Co-authored-by: Frode Gundersen <frogunder@gmail.com>
Co-authored-by: MKLeb <calebb@vmware.com>
This commit is contained in:
Daniel Wozniak 2022-04-18 04:14:51 -07:00 • committed by GitHub
parent 1568067961
commit 422302312a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
34 changed files with 1891 additions and 305 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
CHANGELOG.md
View file

@ -7,6 +7,19 @@ Versions are `MAJOR.PATCH`.
# Changelog
Salt 3004.1 (2022-02-16)
========================
Security
--------
- Sign authentication replies to prevent MiTM (cve-2022-22935)
- Prevent job and fileserver replays (cve-2022-22936)
- Sign pillar data to prevent MiTM attacks. (cve-2202-22934)
- Fixed targeting bug, especially visible when using syndic and user auth. (CVE-2022-22941) (#60413)
- Fix denial of service in junos ifconfig output parsing.
Salt 3004 (2021-10-11)
======================

2
doc/man/salt-api.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-API" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-API" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-api \- salt-api Command
.

2
doc/man/salt-call.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-CALL" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-CALL" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-call \- salt-call Documentation
.

2
doc/man/salt-cloud.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-CLOUD" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-CLOUD" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-cloud \- Salt Cloud Command
.

2
doc/man/salt-cp.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-CP" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-CP" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-cp \- salt-cp Documentation
.

2
doc/man/salt-key.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-KEY" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-KEY" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-key \- salt-key Documentation
.

2
doc/man/salt-master.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-MASTER" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-MASTER" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-master \- salt-master Documentation
.

2
doc/man/salt-minion.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-MINION" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-MINION" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-minion \- salt-minion Documentation
.

2
doc/man/salt-proxy.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-PROXY" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-PROXY" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-proxy \- salt-proxy Documentation
.

2
doc/man/salt-run.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-RUN" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-RUN" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-run \- salt-run Documentation
.

2
doc/man/salt-ssh.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-SSH" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-SSH" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-ssh \- salt-ssh Documentation
.

2
doc/man/salt-syndic.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-SYNDIC" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT-SYNDIC" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt-syndic \- salt-syndic Documentation
.

2
doc/man/salt.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SALT" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt \- salt
.

4
doc/man/salt.7
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT" "7" "Sep 27, 2021" "3004" "Salt"
.TH "SALT" "7" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
salt \- Salt Documentation
.
@ -193862,7 +193862,7 @@ Passes through all the parameters described in the
\fI\%utils.http.query function\fP:
.INDENT 7.0
.TP
.B salt.utils.http.query(url, method=\(aqGET\(aq, params=None, data=None, data_file=None, header_dict=None, header_list=None, header_file=None, username=None, password=None, auth=None, decode=False, decode_type=\(aqauto\(aq, status=False, headers=False, text=False, cookies=None, cookie_jar=None, cookie_format=\(aqlwp\(aq, persist_session=False, session_cookie_jar=None, data_render=False, data_renderer=None, header_render=False, header_renderer=None, template_dict=None, test=False, test_url=None, node=\(aqminion\(aq, port=80, opts=None, backend=None, ca_bundle=None, verify_ssl=None, cert=None, text_out=None, headers_out=None, decode_out=None, stream=False, streaming_callback=None, header_callback=None, handle=False, agent=\(aqSalt/3004rc0+315.g248e4e042c\(aq, hide_fields=None, raise_error=True, formdata=False, formdata_fieldname=None, formdata_filename=None, decode_body=True, **kwargs)
.B salt.utils.http.query(url, method=\(aqGET\(aq, params=None, data=None, data_file=None, header_dict=None, header_list=None, header_file=None, username=None, password=None, auth=None, decode=False, decode_type=\(aqauto\(aq, status=False, headers=False, text=False, cookies=None, cookie_jar=None, cookie_format=\(aqlwp\(aq, persist_session=False, session_cookie_jar=None, data_render=False, data_renderer=None, header_render=False, header_renderer=None, template_dict=None, test=False, test_url=None, node=\(aqminion\(aq, port=80, opts=None, backend=None, ca_bundle=None, verify_ssl=None, cert=None, text_out=None, headers_out=None, decode_out=None, stream=False, streaming_callback=None, header_callback=None, handle=False, agent=\(aqSalt/3004.1(aq, hide_fields=None, raise_error=True, formdata=False, formdata_fieldname=None, formdata_filename=None, decode_body=True, **kwargs)
Query a resource, and decode the return data
.UNINDENT
.INDENT 7.0

2
doc/man/spm.1
View file

@ -1,6 +1,6 @@
.\" Man page generated from reStructuredText.
.
.TH "SPM" "1" "Sep 27, 2021" "3004" "Salt"
.TH "SPM" "1" "Feb 16, 2022" "3004.1" "Salt"
.SH NAME
spm \- Salt Package Manager Command
.

33
doc/topics/releases/3004.1.rst Normal file
View file

@ -0,0 +1,33 @@
.. _release-3004-1:
=========================
Salt 3004.1 Release Notes
=========================
Version 3004.1 is a CVE security fix release for :ref:`3004 <release-3004>`.
Important notice about upgrading
--------------------------------
Version 3004.1 is a security release. 3004.1 minions are not able to
communicate with masters older than 3004.1. You must upgrade your masters
before upgrading minions.
Minion authentication security
------------------------------
Authentication between masters and minions rely on public/private key
encryption and message signing. To secure minion authentication before you must
pre-seed the master's public key on minions. To pre-seed the minions' master
key, place a copy of the master's public key in the minion's pki directory as
``minion_master.pub``.
Security
--------
- Sign authentication replies to prevent MiTM (cve-2022-22935)
- Prevent job and fileserver replays (cve-2022-22936)
- Sign pillar data to prevent MiTM attacks. (cve-2202-22934)
- Fixed targeting bug, especially visible when using syndic and user auth. (CVE-2022-22941) (#60413)
- Fix denial of service in junos ifconfig output parsing.

35
salt/channel/client.py
View file

@ -8,6 +8,7 @@ This includes client side transport, for the ReqServer and the Publisher
import logging
import os
import time
import uuid
import salt.crypt
import salt.exceptions
@ -150,6 +151,7 @@ class AsyncReqChannel:
return {
"enc": self.crypt,
"load": load,
"version": 2,
}
@salt.ext.tornado.gen.coroutine
@ -159,6 +161,8 @@ class AsyncReqChannel:
dictkey=None,
timeout=60,
):
nonce = uuid.uuid4().hex
load["nonce"] = nonce
if not self.auth.authenticated:
yield self.auth.authenticate()
ret = yield self.transport.send(
@ -178,10 +182,29 @@ class AsyncReqChannel:
else:
cipher = PKCS1_OAEP.new(key)
aes = cipher.decrypt(ret["key"])
# Decrypt using the public key.
pcrypt = salt.crypt.Crypticle(self.opts, aes)
data = pcrypt.loads(ret[dictkey])
data = salt.transport.frame.decode_embedded_strs(data)
raise salt.ext.tornado.gen.Return(data)
signed_msg = pcrypt.loads(ret[dictkey])
# Validate the master's signature.
master_pubkey_path = os.path.join(self.opts["pki_dir"], "minion_master.pub")
if not salt.crypt.verify_signature(
master_pubkey_path, signed_msg["data"], signed_msg["sig"]
):
raise salt.crypt.AuthenticationError(
"Pillar payload signature failed to validate."
)
# Make sure the signed key matches the key we used to decrypt the data.
data = salt.payload.loads(signed_msg["data"])
if data["key"] != ret["key"]:
raise salt.crypt.AuthenticationError("Key verification failed.")
# Validate the nonce.
if data["nonce"] != nonce:
raise salt.crypt.AuthenticationError("Pillar nonce verification failed.")
raise salt.ext.tornado.gen.Return(data["pillar"])
@salt.ext.tornado.gen.coroutine
def _crypted_transfer(self, load, timeout=60, raw=False):
@ -197,6 +220,9 @@ class AsyncReqChannel:
:param dict load: A load to send across the wire
:param int timeout: The number of seconds on a response before failing
"""
nonce = uuid.uuid4().hex
if load and isinstance(load, dict):
load["nonce"] = nonce
@salt.ext.tornado.gen.coroutine
def _do_transfer():
@ -210,7 +236,7 @@ class AsyncReqChannel:
# communication, we do not subscribe to return events, we just
# upload the results to the master
if data:
data = self.auth.crypticle.loads(data, raw)
data = self.auth.crypticle.loads(data, raw, nonce=nonce)
if not raw or self.ttype == "tcp": # XXX Why is this needed for tcp
data = salt.transport.frame.decode_embedded_strs(data)
raise salt.ext.tornado.gen.Return(data)
@ -404,6 +430,7 @@ class AsyncPubChannel:
return {
"enc": self.crypt,
"load": load,
"version": 2,
}
@salt.ext.tornado.gen.coroutine

162
salt/channel/server.py
View file

@ -137,10 +137,24 @@ class ReqServerChannel:
"bad load: id {} is not a string".format(id_)
)
version = 0
if "version" in payload:
version = payload["version"]
sign_messages = False
if version > 1:
sign_messages = True
# intercept the "_auth" commands, since the main daemon shouldn't know
# anything about our key auth
if payload["enc"] == "clear" and payload.get("load", {}).get("cmd") == "_auth":
raise salt.ext.tornado.gen.Return(self._auth(payload["load"]))
raise salt.ext.tornado.gen.Return(
self._auth(payload["load"], sign_messages)
)
nonce = None
if version > 1:
nonce = payload["load"].pop("nonce", None)
# TODO: test
try:
@ -156,20 +170,22 @@ class ReqServerChannel:
if req_fun == "send_clear":
raise salt.ext.tornado.gen.Return(ret)
elif req_fun == "send":
raise salt.ext.tornado.gen.Return(self.crypticle.dumps(ret))
raise salt.ext.tornado.gen.Return(self.crypticle.dumps(ret, nonce))
elif req_fun == "send_private":
raise salt.ext.tornado.gen.Return(
self._encrypt_private(
ret,
req_opts["key"],
req_opts["tgt"],
nonce,
sign_messages,
),
)
log.error("Unknown req_fun %s", req_fun)
# always attempt to return an error to the minion
salt.ext.tornado.Return("Server-side exception handling payload")
def _encrypt_private(self, ret, dictkey, target):
def _encrypt_private(self, ret, dictkey, target, nonce=None, sign_messages=True):
"""
The server equivalent of ReqChannel.crypted_transfer_decode_dictentry
"""
@ -184,7 +200,6 @@ class ReqServerChannel:
except OSError:
log.error("AES key not found")
return {"error": "AES key not found"}
pret = {}
key = salt.utils.stringutils.to_bytes(key)
if HAS_M2:
@ -192,9 +207,33 @@ class ReqServerChannel:
else:
cipher = PKCS1_OAEP.new(pub)
pret["key"] = cipher.encrypt(key)
pret[dictkey] = pcrypt.dumps(ret if ret is not False else {})
if ret is False:
ret = {}
if sign_messages:
if nonce is None:
return {"error": "Nonce not included in request"}
tosign = salt.payload.dumps(
{"key": pret["key"], "pillar": ret, "nonce": nonce}
)
master_pem_path = os.path.join(self.opts["pki_dir"], "master.pem")
signed_msg = {
"data": tosign,
"sig": salt.crypt.sign_message(master_pem_path, tosign),
}
pret[dictkey] = pcrypt.dumps(signed_msg)
else:
pret[dictkey] = pcrypt.dumps(ret)
return pret
def _clear_signed(self, load):
master_pem_path = os.path.join(self.opts["pki_dir"], "master.pem")
tosign = salt.payload.dumps(load)
return {
"enc": "clear",
"load": tosign,
"sig": salt.crypt.sign_message(master_pem_path, tosign),
}
def _update_aes(self):
"""
Check to see if a fresh AES key is available and update the components
@ -223,7 +262,7 @@ class ReqServerChannel:
payload["load"] = self.crypticle.loads(payload["load"])
return payload
def _auth(self, load):
def _auth(self, load, sign_messages=False):
"""
Authenticate the client, use the sent public key to encrypt the AES key
which was generated at start up.
@ -242,7 +281,10 @@ class ReqServerChannel:
if not salt.utils.verify.valid_id(self.opts, load["id"]):
log.info("Authentication request from invalid id %s", load["id"])
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
log.info("Authentication request from %s", load["id"])
# 0 is default which should be 'unlimited'
@ -280,7 +322,12 @@ class ReqServerChannel:
self.event.fire_event(
eload, salt.utils.event.tagify(prefix="auth")
)
return {"enc": "clear", "load": {"ret": "full"}}
if sign_messages:
return self._clear_signed(
{"ret": "full", "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": "full"}}
# Check if key is configured to be auto-rejected/signed
auto_reject = self.auto_key.check_autoreject(load["id"])
@ -307,8 +354,10 @@ class ReqServerChannel:
eload = {"result": False, "id": load["id"], "pub": load["pub"]}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
elif os.path.isfile(pubfn):
# The key has been accepted, check it
with salt.utils.files.fopen(pubfn, "r") as pubfn_handle:
@ -332,7 +381,12 @@ class ReqServerChannel:
self.event.fire_event(
eload, salt.utils.event.tagify(prefix="auth")
)
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed(
{"ret": False, "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": False}}
elif not os.path.isfile(pubfn_pend):
# The key has not been accepted, this is a new minion
@ -342,7 +396,10 @@ class ReqServerChannel:
eload = {"result": False, "id": load["id"], "pub": load["pub"]}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
if auto_reject:
key_path = pubfn_rejected
@ -365,7 +422,6 @@ class ReqServerChannel:
# Write the key to the appropriate location
with salt.utils.files.fopen(key_path, "w+") as fp_:
fp_.write(load["pub"])
ret = {"enc": "clear", "load": {"ret": key_result}}
eload = {
"result": key_result,
"act": key_act,
@ -374,7 +430,12 @@ class ReqServerChannel:
}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
return ret
if sign_messages:
return self._clear_signed(
{"ret": key_result, "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": key_result}}
elif os.path.isfile(pubfn_pend):
# This key is in the pending dir and is awaiting acceptance
@ -390,7 +451,6 @@ class ReqServerChannel:
"Pending public key for %s rejected via autoreject_file",
load["id"],
)
ret = {"enc": "clear", "load": {"ret": False}}
eload = {
"result": False,
"act": "reject",
@ -399,7 +459,10 @@ class ReqServerChannel:
}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
return ret
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
elif not auto_sign:
# This key is in the pending dir and is not being auto-signed.
@ -427,7 +490,12 @@ class ReqServerChannel:
self.event.fire_event(
eload, salt.utils.event.tagify(prefix="auth")
)
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed(
{"ret": False, "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": False}}
else:
log.info(
"Authentication failed from host %s, the key is in "
@ -446,7 +514,12 @@ class ReqServerChannel:
self.event.fire_event(
eload, salt.utils.event.tagify(prefix="auth")
)
return {"enc": "clear", "load": {"ret": True}}
if sign_messages:
return self._clear_signed(
{"ret": True, "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": True}}
else:
# This key is in pending and has been configured to be
# auto-signed. Check to see if it is the same key, and if
@ -468,7 +541,12 @@ class ReqServerChannel:
self.event.fire_event(
eload, salt.utils.event.tagify(prefix="auth")
)
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed(
{"ret": False, "nonce": load["nonce"]}
)
else:
return {"enc": "clear", "load": {"ret": False}}
else:
os.remove(pubfn_pend)
@ -478,7 +556,10 @@ class ReqServerChannel:
eload = {"result": False, "id": load["id"], "pub": load["pub"]}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
log.info("Authentication accepted from %s", load["id"])
# only write to disk if you are adding the file, and in open mode,
@ -497,7 +578,10 @@ class ReqServerChannel:
fp_.write(load["pub"])
elif not load["pub"]:
log.error("Public key is empty: %s", load["id"])
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
pub = None
@ -511,7 +595,10 @@ class ReqServerChannel:
pub = salt.crypt.get_rsa_pub_key(pubfn)
except salt.crypt.InvalidKeyError as err:
log.error('Corrupt public key "%s": %s', pubfn, err)
return {"enc": "clear", "load": {"ret": False}}
if sign_messages:
return self._clear_signed({"ret": False, "nonce": load["nonce"]})
else:
return {"enc": "clear", "load": {"ret": False}}
if not HAS_M2:
cipher = PKCS1_OAEP.new(pub)
@ -592,12 +679,16 @@ class ReqServerChannel:
ret["aes"] = pub.public_encrypt(aes, RSA.pkcs1_oaep_padding)
else:
ret["aes"] = cipher.encrypt(aes)
# Be aggressive about the signature
digest = salt.utils.stringutils.to_bytes(hashlib.sha256(aes).hexdigest())
ret["sig"] = salt.crypt.private_encrypt(self.master_key.key, digest)
eload = {"result": True, "act": "accept", "id": load["id"], "pub": load["pub"]}
if self.opts.get("auth_events") is True:
self.event.fire_event(eload, salt.utils.event.tagify(prefix="auth"))
if sign_messages:
ret["nonce"] = load["nonce"]
return self._clear_signed(ret)
return ret
def close(self):
@ -662,7 +753,7 @@ class PubServerChannel:
self.aes_funcs.destroy()
self.aes_funcs = None
def pre_fork(self, process_manager):
def pre_fork(self, process_manager, kwargs=None):
"""
Do anything necessary pre-fork. Since this is on the master side this will
primarily be used to create IPC channels and create our daemon process to
@ -671,15 +762,18 @@ class PubServerChannel:
:param func process_manager: A ProcessManager, from salt.utils.process.ProcessManager
"""
if hasattr(self.transport, "publish_daemon"):
process_manager.add_process(self._publish_daemon)
process_manager.add_process(self._publish_daemon, kwargs=kwargs)
def _publish_daemon(self):
def _publish_daemon(self, **kwargs):
if self.opts["pub_server_niceness"] and not salt.utils.platform.is_windows():
log.info(
"setting Publish daemon niceness to %i",
self.opts["pub_server_niceness"],
)
os.nice(self.opts["pub_server_niceness"])
secrets = kwargs.get("secrets", None)
if secrets is not None:
salt.master.SMaster.secrets = secrets
self.transport.publish_daemon(self.publish_payload, self.presence_callback)
def presence_callback(self, subscriber, msg):
@ -745,7 +839,8 @@ class PubServerChannel:
)
@salt.ext.tornado.gen.coroutine
def publish_payload(self, unpacked_package, *args):
def publish_payload(self, load, *args):
unpacked_package = self.wrap_payload(load)
try:
payload = salt.payload.loads(unpacked_package["payload"])
except KeyError:
@ -760,6 +855,7 @@ class PubServerChannel:
def wrap_payload(self, load):
payload = {"enc": "aes"}
load["serial"] = salt.master.SMaster.get_serial()
crypticle = salt.crypt.Crypticle(
self.opts, salt.master.SMaster.secrets["aes"]["secret"].value
)
@ -770,13 +866,12 @@ class PubServerChannel:
payload["sig"] = salt.crypt.sign_message(master_pem_path, payload["load"])
int_payload = {"payload": salt.payload.dumps(payload)}
# add some targeting stuff for lists only (for now)
if load["tgt_type"] == "list":
int_payload["topic_lst"] = load["tgt"]
# If topics are upported, target matching has to happen master side
match_targets = ["pcre", "glob", "list"]
if self.transport.topic_support and load["tgt_type"] in match_targets:
# add some targeting stuff for lists only (for now)
if load["tgt_type"] == "list":
int_payload["topic_lst"] = load["tgt"]
if isinstance(load["tgt"], str):
# Fetch a list of minions that match
_res = self.ckminions.check_minions(
@ -795,10 +890,9 @@ class PubServerChannel:
"""
Publish "load" to minions
"""
payload = self.wrap_payload(load)
log.debug(
"Sending payload to publish daemon. jid=%s load=%r",
"Sending payload to publish daemon. jid=%s load=%s",
load.get("jid", None),
load,
repr(load)[:40],
)
self.transport.publish(payload)
self.transport.publish(load)

277
salt/crypt.py
View file

@ -17,6 +17,7 @@ import stat
import sys
import time
import traceback
import uuid
import weakref
import salt.channel.client
@ -261,7 +262,11 @@ def verify_signature(pubkey_path, message, signature):
md = EVP.MessageDigest("sha1")
md.update(salt.utils.stringutils.to_bytes(message))
digest = md.final()
return pubkey.verify(digest, signature)
try:
return pubkey.verify(digest, signature)
except RSA.RSAError as exc:
log.debug("Signature verification failed: %s", exc.args[0])
return False
else:
verifier = PKCS1_v1_5.new(pubkey)
return verifier.verify(
@ -695,9 +700,17 @@ class AsyncAuth:
self._authenticate_future.set_exception(error)
else:
key = self.__key(self.opts)
AsyncAuth.creds_map[key] = creds
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
if key not in AsyncAuth.creds_map:
log.debug("%s Got new master aes key.", self)
AsyncAuth.creds_map[key] = creds
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
elif self._creds["aes"] != creds["aes"]:
log.debug("%s The master's aes key has changed.", self)
AsyncAuth.creds_map[key] = creds
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
self._authenticate_future.set_result(
True
) # mark the sign-in as complete
@ -728,7 +741,6 @@ class AsyncAuth:
with the publication port and the shared AES key.
"""
auth = {}
auth_timeout = self.opts.get("auth_timeout", None)
if auth_timeout is not None:
@ -740,10 +752,6 @@ class AsyncAuth:
if auth_tries is not None:
tries = auth_tries
m_pub_fn = os.path.join(self.opts["pki_dir"], self.mpub)
auth["master_uri"] = self.opts["master_uri"]
close_channel = False
if not channel:
close_channel = True
@ -768,59 +776,85 @@ class AsyncAuth:
finally:
if close_channel:
channel.close()
ret = self.handle_signin_response(sign_in_payload, payload)
raise salt.ext.tornado.gen.Return(ret)
if not isinstance(payload, dict):
def handle_signin_response(self, sign_in_payload, payload):
auth = {}
m_pub_fn = os.path.join(self.opts["pki_dir"], self.mpub)
auth["master_uri"] = self.opts["master_uri"]
if not isinstance(payload, dict) or "load" not in payload:
log.error("Sign-in attempt failed: %s", payload)
raise salt.ext.tornado.gen.Return(False)
if "load" in payload:
if "ret" in payload["load"]:
if not payload["load"]["ret"]:
if self.opts["rejected_retry"]:
log.error(
"The Salt Master has rejected this minion's public "
"key.\nTo repair this issue, delete the public key "
"for this minion on the Salt Master.\nThe Salt "
"Minion will attempt to to re-authenicate."
)
raise salt.ext.tornado.gen.Return("retry")
else:
log.critical(
"The Salt Master has rejected this minion's public "
"key!\nTo repair this issue, delete the public key "
"for this minion on the Salt Master and restart this "
"minion.\nOr restart the Salt Master in open mode to "
"clean out the keys. The Salt Minion will now exit."
)
# Add a random sleep here for systems that are using a
# a service manager to immediately restart the service
# to avoid overloading the system
time.sleep(random.randint(10, 20))
sys.exit(salt.defaults.exitcodes.EX_NOPERM)
# has the master returned that its maxed out with minions?
elif payload["load"]["ret"] == "full":
raise salt.ext.tornado.gen.Return("full")
else:
log.error(
"The Salt Master has cached the public key for this "
"node, this salt minion will wait for %s seconds "
"before attempting to re-authenticate",
self.opts["acceptance_wait_time"],
)
raise salt.ext.tornado.gen.Return("retry")
auth["aes"] = self.verify_master(payload, master_pub="token" in sign_in_payload)
if not auth["aes"]:
log.critical(
"The Salt Master server's public key did not authenticate!\n"
"The master may need to be updated if it is a version of Salt "
"lower than %s, or\n"
"If you are confident that you are connecting to a valid Salt "
"Master, then remove the master public key and restart the "
"Salt Minion.\nThe master public key can be found "
"at:\n%s",
salt.version.__version__,
m_pub_fn,
return False
clear_signed_data = payload["load"]
clear_signature = payload["sig"]
payload = salt.payload.loads(clear_signed_data)
if "pub_key" in payload:
auth["aes"] = self.verify_master(
payload, master_pub="token" in sign_in_payload
)
raise SaltClientError("Invalid master key")
if not auth["aes"]:
log.critical(
"The Salt Master server's public key did not authenticate!\n"
"The master may need to be updated if it is a version of Salt "
"lower than %s, or\n"
"If you are confident that you are connecting to a valid Salt "
"Master, then remove the master public key and restart the "
"Salt Minion.\nThe master public key can be found "
"at:\n%s",
salt.version.__version__,
m_pub_fn,
)
raise SaltClientError("Invalid master key")
master_pubkey_path = os.path.join(self.opts["pki_dir"], self.mpub)
if os.path.exists(master_pubkey_path) and not verify_signature(
master_pubkey_path, clear_signed_data, clear_signature
):
log.critical("The payload signature did not validate.")
raise SaltClientError("Invalid signature")
if payload["nonce"] != sign_in_payload["nonce"]:
log.critical("The payload nonce did not validate.")
raise SaltClientError("Invalid nonce")
if "ret" in payload:
if not payload["ret"]:
if self.opts["rejected_retry"]:
log.error(
"The Salt Master has rejected this minion's public "
"key.\nTo repair this issue, delete the public key "
"for this minion on the Salt Master.\nThe Salt "
"Minion will attempt to re-authenicate."
)
return "retry"
else:
log.critical(
"The Salt Master has rejected this minion's public "
"key!\nTo repair this issue, delete the public key "
"for this minion on the Salt Master and restart this "
"minion.\nOr restart the Salt Master in open mode to "
"clean out the keys. The Salt Minion will now exit."
)
# Add a random sleep here for systems that are using a
# a service manager to immediately restart the service
# to avoid overloading the system
time.sleep(random.randint(10, 20))
sys.exit(salt.defaults.exitcodes.EX_NOPERM)
# has the master returned that its maxed out with minions?
elif payload["ret"] == "full":
return "full"
else:
log.error(
"The Salt Master has cached the public key for this "
"node, this salt minion will wait for %s seconds "
"before attempting to re-authenticate",
self.opts["acceptance_wait_time"],
)
return "retry"
if self.opts.get("syndic_master", False): # Is syndic
syndic_finger = self.opts.get(
"syndic_finger", self.opts.get("master_finger", False)
@ -842,8 +876,9 @@ class AsyncAuth:
!= self.opts["master_finger"]
):
self._finger_fail(self.opts["master_finger"], m_pub_fn)
auth["publish_port"] = payload["publish_port"]
raise salt.ext.tornado.gen.Return(auth)
return auth
def get_keys(self):
"""
@ -891,6 +926,7 @@ class AsyncAuth:
payload = {}
payload["cmd"] = "_auth"
payload["id"] = self.opts["id"]
payload["nonce"] = uuid.uuid4().hex
if "autosign_grains" in self.opts:
autosign_grains = {}
for grain in self.opts["autosign_grains"]:
@ -1253,6 +1289,7 @@ class SAuth(AsyncAuth):
self.token = salt.utils.stringutils.to_bytes(Crypticle.generate_key_string())
self.pub_path = os.path.join(self.opts["pki_dir"], "minion.pub")
self.rsa_path = os.path.join(self.opts["pki_dir"], "minion.pem")
self._creds = None
if "syndic_master" in self.opts:
self.mpub = "syndic_master.pub"
elif "alert_master" in self.opts:
@ -1322,8 +1359,14 @@ class SAuth(AsyncAuth):
)
continue
break
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
if self._creds is None:
log.error("%s Got new master aes key.", self)
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
elif self._creds["aes"] != creds["aes"]:
log.error("%s The master's aes key has changed.", self)
self._creds = creds
self._crypticle = Crypticle(self.opts, creds["aes"])
def sign_in(self, timeout=60, safe=True, tries=1, channel=None):
"""
@ -1376,78 +1419,7 @@ class SAuth(AsyncAuth):
if close_channel:
channel.close()
if "load" in payload:
if "ret" in payload["load"]:
if not payload["load"]["ret"]:
if self.opts["rejected_retry"]:
log.error(
"The Salt Master has rejected this minion's public "
"key.\nTo repair this issue, delete the public key "
"for this minion on the Salt Master.\nThe Salt "
"Minion will attempt to to re-authenicate."
)
return "retry"
else:
log.critical(
"The Salt Master has rejected this minion's public "
"key!\nTo repair this issue, delete the public key "
"for this minion on the Salt Master and restart this "
"minion.\nOr restart the Salt Master in open mode to "
"clean out the keys. The Salt Minion will now exit."
)
sys.exit(salt.defaults.exitcodes.EX_NOPERM)
# has the master returned that its maxed out with minions?
elif payload["load"]["ret"] == "full":
return "full"
else:
log.error(
"The Salt Master has cached the public key for this "
"node. If this is the first time connecting to this "
"master then this key may need to be accepted using "
"'salt-key -a %s' on the salt master. This salt "
"minion will wait for %s seconds before attempting "
"to re-authenticate.",
self.opts["id"],
self.opts["acceptance_wait_time"],
)
return "retry"
auth["aes"] = self.verify_master(payload, master_pub="token" in sign_in_payload)
if not auth["aes"]:
log.critical(
"The Salt Master server's public key did not authenticate!\n"
"The master may need to be updated if it is a version of Salt "
"lower than %s, or\n"
"If you are confident that you are connecting to a valid Salt "
"Master, then remove the master public key and restart the "
"Salt Minion.\nThe master public key can be found "
"at:\n%s",
salt.version.__version__,
m_pub_fn,
)
sys.exit(42)
if self.opts.get("syndic_master", False): # Is syndic
syndic_finger = self.opts.get(
"syndic_finger", self.opts.get("master_finger", False)
)
if syndic_finger:
if (
salt.utils.crypt.pem_finger(
m_pub_fn, sum_type=self.opts["hash_type"]
)
!= syndic_finger
):
self._finger_fail(syndic_finger, m_pub_fn)
else:
if self.opts.get("master_finger", False):
if (
salt.utils.crypt.pem_finger(
m_pub_fn, sum_type=self.opts["hash_type"]
)
!= self.opts["master_finger"]
):
self._finger_fail(self.opts["master_finger"], m_pub_fn)
auth["publish_port"] = payload["publish_port"]
return auth
return self.handle_signin_response(sign_in_payload, payload)
class Crypticle:
@ -1462,10 +1434,11 @@ class Crypticle:
AES_BLOCK_SIZE = 16
SIG_SIZE = hashlib.sha256().digest_size
def __init__(self, opts, key_string, key_size=192):
def __init__(self, opts, key_string, key_size=192, serial=0):
self.key_string = key_string
self.keys = self.extract_keys(self.key_string, key_size)
self.key_size = key_size
self.serial = serial
@classmethod
def generate_key_string(cls, key_size=192):
@ -1535,13 +1508,17 @@ class Crypticle:
data = cypher.decrypt(data)
return data[: -data[-1]]
def dumps(self, obj):
def dumps(self, obj, nonce=None):
"""
Serialize and encrypt a python object
"""
return self.encrypt(self.PICKLE_PAD + salt.payload.dumps(obj))
if nonce:
toencrypt = self.PICKLE_PAD + nonce.encode() + salt.payload.dumps(obj)
else:
toencrypt = self.PICKLE_PAD + salt.payload.dumps(obj)
return self.encrypt(toencrypt)
def loads(self, data, raw=False):
def loads(self, data, raw=False, nonce=None):
"""
Decrypt and un-serialize a python object
"""
@ -1549,5 +1526,25 @@ class Crypticle:
# simple integrity check to verify that we got meaningful data
if not data.startswith(self.PICKLE_PAD):
return {}
load = salt.payload.loads(data[len(self.PICKLE_PAD) :], raw=raw)
return load
data = data[len(self.PICKLE_PAD) :]
if nonce:
ret_nonce = data[:32].decode()
data = data[32:]
if ret_nonce != nonce:
raise SaltClientError("Nonce verification error")
payload = salt.payload.loads(data, raw=raw)
if isinstance(payload, dict):
if "serial" in payload:
serial = payload.pop("serial")
if serial <= self.serial:
log.critical(
"A message with an invalid serial was received.\n"
"this serial: %d\n"
"last serial: %d\n"
"The minion will not honor this request.",
serial,
self.serial,
)
return {}
self.serial = serial
return payload

62
salt/master.py
View file

@ -128,6 +128,44 @@ class SMaster:
"""
return salt.daemons.masterapi.access_keys(self.opts)
@classmethod
def get_serial(cls, opts=None, event=None):
with cls.secrets["aes"]["secret"].get_lock():
if cls.secrets["aes"]["serial"].value == sys.maxsize:
cls.rotate_secrets(opts, event, use_lock=False)
else:
cls.secrets["aes"]["serial"].value += 1
return cls.secrets["aes"]["serial"].value
@classmethod
def rotate_secrets(cls, opts=None, event=None, use_lock=True):
log.info("Rotating master AES key")
if opts is None:
opts = {}
for secret_key, secret_map in cls.secrets.items():
# should be unnecessary-- since no one else should be modifying
if use_lock:
with secret_map["secret"].get_lock():
secret_map["secret"].value = salt.utils.stringutils.to_bytes(
secret_map["reload"]()
)
if "serial" in secret_map:
secret_map["serial"].value = 0
else:
secret_map["secret"].value = salt.utils.stringutils.to_bytes(
secret_map["reload"]()
)
if "serial" in secret_map:
secret_map["serial"].value = 0
if event:
event.fire_event({"rotate_{}_key".format(secret_key): True}, tag="key")
if opts.get("ping_on_rotate"):
# Ping all minions to get them to pick up the new key
log.debug("Pinging all connected minions due to key rotation")
salt.utils.master.ping_all_connected_minions(opts)
class Maintenance(salt.utils.process.SignalHandlingProcess):
"""
@ -140,6 +178,7 @@ class Maintenance(salt.utils.process.SignalHandlingProcess):
:param dict opts: The salt options
"""
self.master_secrets = kwargs.pop("master_secrets", None)
super().__init__(**kwargs)
self.opts = opts
# How often do we perform the maintenance tasks
@ -155,6 +194,8 @@ class Maintenance(salt.utils.process.SignalHandlingProcess):
in the parent process, then once the fork happens you'll start getting
errors like "WARNING: Mixing fork() and threads detected; memory leaked."
"""
if self.master_secrets is not None:
SMaster.secrets = self.master_secrets
# Load Runners
ropts = dict(self.opts)
ropts["quiet"] = True
@ -279,21 +320,8 @@ class Maintenance(salt.utils.process.SignalHandlingProcess):
to_rotate = True
if to_rotate:
log.info("Rotating master AES key")
for secret_key, secret_map in SMaster.secrets.items():
# should be unnecessary-- since no one else should be modifying
with secret_map["secret"].get_lock():
secret_map["secret"].value = salt.utils.stringutils.to_bytes(
secret_map["reload"]()
)
self.event.fire_event(
{"rotate_{}_key".format(secret_key): True}, tag="key"
)
SMaster.rotate_secrets(self.opts, self.event)
self.rotate = now
if self.opts.get("ping_on_rotate"):
# Ping all minions to get them to pick up the new key
log.debug("Pinging all connected minions due to key rotation")
salt.utils.master.ping_all_connected_minions(self.opts)
def handle_git_pillar(self):
"""
@ -667,8 +695,12 @@ class Master(SMaster):
salt.crypt.Crypticle.generate_key_string()
),
),
"serial": multiprocessing.Value(
ctypes.c_longlong, lock=False # We'll use the lock from 'secret'
),
"reload": salt.crypt.Crypticle.generate_key_string,
}
log.info("Creating master process manager")
# Since there are children having their own ProcessManager we should wait for kill more time.
self.process_manager = salt.utils.process.ProcessManager(wait_for_kill=5)
@ -676,7 +708,7 @@ class Master(SMaster):
log.info("Creating master publisher process")
for _, opts in iter_transport_opts(self.opts):
chan = salt.channel.server.PubServerChannel.factory(opts)
chan.pre_fork(self.process_manager)
chan.pre_fork(self.process_manager, kwargs={"secrets": SMaster.secrets})
pub_channels.append(chan)
log.info("Creating master event publisher process")

1
salt/minion.py
View file

@ -1703,6 +1703,7 @@ class Minion(MinionBase):
Override this method if you wish to handle the decoded data
differently.
"""
# Ensure payload is unicode. Disregard failure to decode binary blobs.
if "user" in data:
log.info(

4
salt/pillar/__init__.py
View file

@ -9,6 +9,7 @@ import logging
import os
import sys
import traceback
import uuid
import salt.channel.client
import salt.ext.tornado.gen
@ -262,6 +263,9 @@ class AsyncRemotePillar(RemotePillarMixin):
load,
dictkey="pillar",
)
except salt.crypt.AuthenticationError as exc:
log.error(exc.message)
raise SaltClientError("Exception getting pillar.")
except Exception: # pylint: disable=broad-except
log.exception("Exception getting pillar:")
raise SaltClientError("Exception getting pillar.")

19
salt/utils/minions.py
View file

@ -737,20 +737,27 @@ class CkMinions:
def validate_tgt(self, valid, expr, tgt_type, minions=None, expr_form=None):
"""
Return a Bool. This function returns if the expression sent in is
within the scope of the valid expression
Validate the target minions against the possible valid minions.
If ``minions`` is provided, they will be compared against the valid
minions. Otherwise, ``expr`` and ``tgt_type`` will be used to expand
to a list of target minions.
Return True if all of the requested minions are valid minions,
otherwise return False.
"""
v_minions = set(self.check_minions(valid, "compound").get("minions", []))
if not v_minions:
# There are no valid minions, so it doesn't matter what we are
# targeting - this is a fail.
return False
if minions is None:
_res = self.check_minions(expr, tgt_type)
minions = set(_res["minions"])
else:
minions = set(minions)
d_bool = not bool(minions.difference(v_minions))
if len(v_minions) == len(minions) and d_bool:
return True
return d_bool
return minions.issubset(v_minions)
def match_check(self, regex, fun):
"""

4
salt/utils/network.py
View file

@ -1006,10 +1006,10 @@ def _junos_interfaces_ifconfig(out):
pip = re.compile(
r".*?inet\s*(primary)*\s+mtu"
r" (\d+)\s+local=[^\d]*(.*?)\s+dest=[^\d]*(.*?)\/([\d]*)\s+bcast=((?:[0-9]{1,3}\.){3}[0-9]{1,3})"
r" (\d+)\s+local=[^\d]*(.*?)\s{0,40}dest=[^\d]*(.*?)\/([\d]*)\s{0,40}bcast=((?:[0-9]{1,3}\.){3}[0-9]{1,3})"
)
pip6 = re.compile(
r".*?inet6 mtu [^\d]+\s+local=([0-9a-f:]+)%([a-zA-Z0-9]*)/([\d]*)\s"
r".*?inet6 mtu [^\d]+\s{0,40}local=([0-9a-f:]+)%([a-zA-Z0-9]*)/([\d]*)\s"
)
pupdown = re.compile("UP")

26
tests/pytests/functional/channel/test_server.py
View file

@ -1,4 +1,6 @@
import ctypes
import logging
import multiprocessing
import os
import pathlib
import shutil
@ -10,8 +12,10 @@ import salt.channel.server
import salt.config
import salt.ext.tornado.gen
import salt.ext.tornado.ioloop
import salt.master
import salt.utils.platform
import salt.utils.process
import salt.utils.stringutils
from saltfactories.utils import random_string
from saltfactories.utils.ports import get_unused_localhost_port
@ -95,6 +99,21 @@ def process_manager():
process_manager.terminate()
@pytest.fixture
def master_secrets():
salt.master.SMaster.secrets["aes"] = {
"secret": multiprocessing.Array(
ctypes.c_char,
salt.utils.stringutils.to_bytes(salt.crypt.Crypticle.generate_key_string()),
),
"serial": multiprocessing.Value(
ctypes.c_longlong, lock=False # We'll use the lock from 'secret'
),
}
yield
salt.master.SMaster.secrets.pop("aes")
@salt.ext.tornado.gen.coroutine
def _connect_and_publish(
io_loop, channel_minion_id, channel, server, received, timeout=60
@ -117,7 +136,12 @@ def _connect_and_publish(
def test_pub_server_channel(
io_loop, channel_minion_id, master_config, minion_config, process_manager
io_loop,
channel_minion_id,
master_config,
minion_config,
process_manager,
master_secrets,
):
server_channel = salt.channel.server.PubServerChannel.factory(
master_config,

18
tests/pytests/functional/transport/server/test_req_channel.py
View file

@ -1,3 +1,4 @@
import ctypes
import logging
import multiprocessing
@ -7,6 +8,9 @@ import salt.channel.server
import salt.config
import salt.exceptions
import salt.ext.tornado.gen
import salt.master
import salt.transport.client
import salt.transport.server
import salt.utils.platform
import salt.utils.process
import salt.utils.stringutils
@ -32,6 +36,18 @@ class ReqServerChannelProcess(salt.utils.process.SignalHandlingProcess):
self.running = multiprocessing.Event()
def run(self):
salt.master.SMaster.secrets["aes"] = {
"secret": multiprocessing.Array(
ctypes.c_char,
salt.utils.stringutils.to_bytes(
salt.crypt.Crypticle.generate_key_string()
),
),
"serial": multiprocessing.Value(
ctypes.c_longlong, lock=False # We'll use the lock from 'secret'
),
}
self.io_loop = salt.ext.tornado.ioloop.IOLoop()
self.io_loop.make_current()
self.req_server_channel.post_fork(self._handle_payload, io_loop=self.io_loop)
@ -120,7 +136,7 @@ def test_basic(push_channel):
{"baz": "qux", "list": [1, 2, 3]},
]
for msg in msgs:
ret = push_channel.send(msg, timeout=5, tries=1)
ret = push_channel.send(dict(msg), timeout=5, tries=1)
assert ret["load"] == msg

66
tests/pytests/functional/transport/zeromq/test_pub_server_channel.py
View file

@ -1,9 +1,12 @@
import ctypes
import logging
import multiprocessing
import time
from concurrent.futures.thread import ThreadPoolExecutor
import pytest
import salt.channel.client
import salt.channel.server
import salt.config
import salt.exceptions
import salt.ext.tornado.gen
@ -21,11 +24,14 @@ log = logging.getLogger(__name__)
class Collector(salt.utils.process.SignalHandlingProcess):
def __init__(self, minion_config, pub_uri, timeout=30, zmq_filtering=False):
def __init__(
self, minion_config, pub_uri, aes_key, timeout=30, zmq_filtering=False
):
super().__init__()
self.minion_config = minion_config
self.pub_uri = pub_uri
self.timeout = timeout
self.aes_key = aes_key
self.hard_timeout = time.time() + timeout + 30
self.manager = multiprocessing.Manager()
self.results = self.manager.list()
@ -34,6 +40,21 @@ class Collector(salt.utils.process.SignalHandlingProcess):
self.started = multiprocessing.Event()
self.running = multiprocessing.Event()
def _rotate_secrets(self, now=None):
salt.master.SMaster.secrets["aes"] = {
"secret": multiprocessing.Array(
ctypes.c_char,
salt.utils.stringutils.to_bytes(
salt.crypt.Crypticle.generate_key_string()
),
),
"serial": multiprocessing.Value(
ctypes.c_longlong, lock=False # We'll use the lock from 'secret'
),
"reload": salt.crypt.Crypticle.generate_key_string,
"rotate_master_key": self._rotate_secrets,
}
def run(self):
"""
Gather results until then number of seconds specified by timeout passes
@ -45,6 +66,7 @@ class Collector(salt.utils.process.SignalHandlingProcess):
sock.setsockopt(zmq.SUBSCRIBE, b"")
sock.connect(self.pub_uri)
last_msg = time.time()
crypticle = salt.crypt.Crypticle(self.minion_config, self.aes_key)
self.started.set()
while True:
curr_time = time.time()
@ -58,7 +80,10 @@ class Collector(salt.utils.process.SignalHandlingProcess):
time.sleep(0.1)
else:
try:
payload = salt.payload.loads(payload)
serial_payload = salt.payload.loads(payload)
payload = crypticle.loads(serial_payload["load"])
if not payload:
continue
if "start" in payload:
self.running.set()
continue
@ -100,10 +125,20 @@ class PubServerChannelProcess(salt.utils.process.SignalHandlingProcess):
self.master_config = master_config
self.minion_config = minion_config
self.collector_kwargs = collector_kwargs
self.aes_key = salt.crypt.Crypticle.generate_key_string()
salt.master.SMaster.secrets["aes"] = {
"secret": multiprocessing.Array(
ctypes.c_char,
salt.utils.stringutils.to_bytes(self.aes_key),
),
"serial": multiprocessing.Value(
ctypes.c_longlong, lock=False # We'll use the lock from 'secret'
),
}
self.process_manager = salt.utils.process.ProcessManager(
name="ZMQ-PubServer-ProcessManager"
)
self.pub_server_channel = salt.transport.zeromq.PublishServer(
self.pub_server_channel = salt.channel.server.PubServerChannel.factory(
self.master_config
)
self.pub_server_channel.pre_fork(self.process_manager)
@ -111,7 +146,7 @@ class PubServerChannelProcess(salt.utils.process.SignalHandlingProcess):
self.queue = multiprocessing.Queue()
self.stopped = multiprocessing.Event()
self.collector = Collector(
self.minion_config, self.pub_uri, **self.collector_kwargs
self.minion_config, self.pub_uri, self.aes_key, **self.collector_kwargs
)
def run(self):
@ -138,7 +173,7 @@ class PubServerChannelProcess(salt.utils.process.SignalHandlingProcess):
if self.process_manager is None:
return
self.process_manager.terminate()
self.pub_server_channel.pub_close()
self.pub_server_channel.close()
# Really terminate any process still left behind
for pid in self.process_manager._process_map:
terminate_process(pid=pid, kill_children=True, slow_stop=False)
@ -208,12 +243,17 @@ def test_issue_36469_tcp(salt_master, salt_minion):
https://github.com/saltstack/salt/issues/36469
"""
def _send_small(server_channel, sid, num=10):
def _send_small(opts, sid, num=10):
server_channel = salt.channel.server.PubServerChannel.factory(opts)
for idx in range(num):
load = {"tgt_type": "glob", "tgt": "*", "jid": "{}-s{}".format(sid, idx)}
server_channel.publish(load)
time.sleep(0.3)
time.sleep(3)
server_channel.close_pub()
def _send_large(server_channel, sid, num=10, size=250000 * 3):
def _send_large(opts, sid, num=10, size=250000 * 3):
server_channel = salt.channel.server.PubServerChannel.factory(opts)
for idx in range(num):
load = {
"tgt_type": "glob",
@ -222,16 +262,20 @@ def test_issue_36469_tcp(salt_master, salt_minion):
"xdata": "0" * size,
}
server_channel.publish(load)
time.sleep(0.3)
time.sleep(3)
server_channel.close_pub()
opts = dict(salt_master.config.copy(), ipc_mode="tcp", pub_hwm=0)
send_num = 10 * 4
expect = []
with PubServerChannelProcess(opts, salt_minion.config.copy()) as server_channel:
assert "aes" in salt.master.SMaster.secrets
with ThreadPoolExecutor(max_workers=4) as executor:
executor.submit(_send_small, server_channel, 1)
executor.submit(_send_large, server_channel, 2)
executor.submit(_send_small, server_channel, 3)
executor.submit(_send_large, server_channel, 4)
executor.submit(_send_small, opts, 1)
executor.submit(_send_large, opts, 2)
executor.submit(_send_small, opts, 3)
executor.submit(_send_large, opts, 4)
expect.extend(["{}-s{}".format(a, b) for a in range(10) for b in (1, 3)])
expect.extend(["{}-l{}".format(a, b) for a in range(10) for b in (2, 4)])
results = server_channel.collector.results

151
tests/pytests/unit/test_crypt.py
View file

@ -4,10 +4,100 @@ tests.pytests.unit.test_crypt
Unit tests for salt's crypt module
"""
import uuid
import pytest
import salt.crypt
import salt.master
import salt.utils.files
PRIV_KEY = """
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAoAsMPt+4kuIG6vKyw9r3+OuZrVBee/2vDdVetW+Js5dTlgrJ
aghWWn3doGmKlEjqh7E4UTa+t2Jd6w8RSLnyHNJ/HpVhMG0M07MF6FMfILtDrrt8
ZX7eDVt8sx5gCEpYI+XG8Y07Ga9i3Hiczt+fu6HYwu96HggmG2pqkOrn3iGfqBvV
YVFJzSZYe7e4c1PeEs0xYcrA4k+apyGsMtpef8vRUrNicRLc7dAcvfhtgt2DXEZ2
d72t/CR4ygtUvPXzisaTPW0G7OWAheCloqvTIIPQIjR8htFxGTz02STVXfnhnJ0Z
k8KhqKF2v1SQvIYxsZU7jaDgl5i3zpeh58cYOwIDAQABAoIBABZUJEO7Y91+UnfC
H6XKrZEZkcnH7j6/UIaOD9YhdyVKxhsnax1zh1S9vceNIgv5NltzIsfV6vrb6v2K
Dx/F7Z0O0zR5o+MlO8ZncjoNKskex10gBEWG00Uqz/WPlddiQ/TSMJTv3uCBAzp+
S2Zjdb4wYPUlgzSgb2ygxrhsRahMcSMG9PoX6klxMXFKMD1JxiY8QfAHahPzQXy9
F7COZ0fCVo6BE+MqNuQ8tZeIxu8mOULQCCkLFwXmkz1FpfK/kNRmhIyhxwvCS+z4
JuErW3uXfE64RLERiLp1bSxlDdpvRO2R41HAoNELTsKXJOEt4JANRHm/CeyA5wsh
NpscufUCgYEAxhgPfcMDy2v3nL6KtkgYjdcOyRvsAF50QRbEa8ldO+87IoMDD/Oe
osFERJ5hhyyEO78QnaLVegnykiw5DWEF02RKMhD/4XU+1UYVhY0wJjKQIBadsufB
2dnaKjvwzUhPh5BrBqNHl/FXwNCRDiYqXa79eWCPC9OFbZcUWWq70s8CgYEAztOI
61zRfmXJ7f70GgYbHg+GA7IrsAcsGRITsFR82Ho0lqdFFCxz7oK8QfL6bwMCGKyk
nzk+twh6hhj5UNp18KN8wktlo02zTgzgemHwaLa2cd6xKgmAyuPiTgcgnzt5LVNG
FOjIWkLwSlpkDTl7ZzY2QSy7t+mq5d750fpIrtUCgYBWXZUbcpPL88WgDB7z/Bjg
dlvW6JqLSqMK4b8/cyp4AARbNp12LfQC55o5BIhm48y/M70tzRmfvIiKnEc/gwaE
NJx4mZrGFFURrR2i/Xx5mt/lbZbRsmN89JM+iKWjCpzJ8PgIi9Wh9DIbOZOUhKVB
9RJEAgo70LvCnPTdS0CaVwKBgDJW3BllAvw/rBFIH4OB/vGnF5gosmdqp3oGo1Ik
jipmPAx6895AH4tquIVYrUl9svHsezjhxvjnkGK5C115foEuWXw0u60uiTiy+6Pt
2IS0C93VNMulenpnUrppE7CN2iWFAiaura0CY9fE/lsVpYpucHAWgi32Kok+ZxGL
WEttAoGAN9Ehsz4LeQxEj3x8wVeEMHF6OsznpwYsI2oVh6VxpS4AjgKYqeLVcnNi
TlZFsuQcqgod8OgzA91tdB+Rp86NygmWD5WzeKXpCOg9uA+y/YL+0sgZZHsuvbK6
PllUgXdYxqClk/hdBFB7v9AQoaj7K9Ga22v32msftYDQRJ94xOI=
-----END RSA PRIVATE KEY-----
"""
PUB_KEY = """
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoAsMPt+4kuIG6vKyw9r3
+OuZrVBee/2vDdVetW+Js5dTlgrJaghWWn3doGmKlEjqh7E4UTa+t2Jd6w8RSLny
HNJ/HpVhMG0M07MF6FMfILtDrrt8ZX7eDVt8sx5gCEpYI+XG8Y07Ga9i3Hiczt+f
u6HYwu96HggmG2pqkOrn3iGfqBvVYVFJzSZYe7e4c1PeEs0xYcrA4k+apyGsMtpe
f8vRUrNicRLc7dAcvfhtgt2DXEZ2d72t/CR4ygtUvPXzisaTPW0G7OWAheCloqvT
IIPQIjR8htFxGTz02STVXfnhnJ0Zk8KhqKF2v1SQvIYxsZU7jaDgl5i3zpeh58cY
OwIDAQAB
-----END PUBLIC KEY-----
"""
PRIV_KEY2 = """
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAp+8cTxguO6Vg+YO92VfHgNld3Zy8aM3JbZvpJcjTnis+YFJ7
Zlkcc647yPRRwY9nYBNywahnt5kIeuT1rTvTsMBZWvmUoEVUj1Xg8XXQkBvb9Ozy
Gqy/G/p8KDDpzMP/U+XCnUeHiXTZrgnqgBIc2cKeCVvWFqDi0GRFGzyaXLaX3PPm
M7DJ0MIPL1qgmcDq6+7Ze0gJ9SrDYFAeLmbuT1OqDfufXWQl/82JXeiwU2cOpqWq
7n5fvPOWim7l1tzQ+dSiMRRm0xa6uNexCJww3oJSwvMbAmgzvOhqqhlqv+K7u0u7
FrFFojESsL36Gq4GBrISnvu2tk7u4GGNTYYQbQIDAQABAoIBAADrqWDQnd5DVZEA
lR+WINiWuHJAy/KaIC7K4kAMBgbxrz2ZbiY9Ok/zBk5fcnxIZDVtXd1sZicmPlro
GuWodIxdPZAnWpZ3UtOXUayZK/vCP1YsH1agmEqXuKsCu6Fc+K8VzReOHxLUkmXn
FYM+tixGahXcjEOi/aNNTWitEB6OemRM1UeLJFzRcfyXiqzHpHCIZwBpTUAsmzcG
QiVDkMTKubwo/m+PVXburX2CGibUydctgbrYIc7EJvyx/cpRiPZXo1PhHQWdu4Y1
SOaC66WLsP/wqvtHo58JQ6EN/gjSsbAgGGVkZ1xMo66nR+pLpR27coS7o03xCks6
DY/0mukCgYEAuLIGgBnqoh7YsOBLd/Bc1UTfDMxJhNseo+hZemtkSXz2Jn51322F
Zw/FVN4ArXgluH+XsOhvG/MFFpojwZSrb0Qq5b1MRdo9qycq8lGqNtlN1WHqosDQ
zW29kpL0tlRrSDpww3wRESsN9rH5XIrJ1b3ZXuO7asR+KBVQMy/+NcUCgYEA6MSC
c+fywltKPgmPl5j0DPoDe5SXE/6JQy7w/vVGrGfWGf/zEJmhzS2R+CcfTTEqaT0T
Yw8+XbFgKAqsxwtE9MUXLTVLI3sSUyE4g7blCYscOqhZ8ItCUKDXWkSpt++rG0Um
1+cEJP/0oCazG6MWqvBC4NpQ1nzh46QpjWqMwokCgYAKDLXJ1p8rvx3vUeUJW6zR
dfPlEGCXuAyMwqHLxXgpf4EtSwhC5gSyPOtx2LqUtcrnpRmt6JfTH4ARYMW9TMef
QEhNQ+WYj213mKP/l235mg1gJPnNbUxvQR9lkFV8bk+AGJ32JRQQqRUTbU+yN2MQ
HEptnVqfTp3GtJIultfwOQKBgG+RyYmu8wBP650izg33BXu21raEeYne5oIqXN+I
R5DZ0JjzwtkBGroTDrVoYyuH1nFNEh7YLqeQHqvyufBKKYo9cid8NQDTu+vWr5UK
tGvHnwdKrJmM1oN5JOAiq0r7+QMAOWchVy449VNSWWV03aeftB685iR5BXkstbIQ
EVopAoGAfcGBTAhmceK/4Q83H/FXBWy0PAa1kZGg/q8+Z0KY76AqyxOVl0/CU/rB
3tO3sKhaMTHPME/MiQjQQGoaK1JgPY6JHYvly2KomrJ8QTugqNGyMzdVJkXAK2AM
GAwC8ivAkHf8CHrHa1W7l8t2IqBjW1aRt7mOW92nfG88Hck0Mbo=
-----END RSA PRIVATE KEY-----
"""
PUB_KEY2 = """
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp+8cTxguO6Vg+YO92VfH
gNld3Zy8aM3JbZvpJcjTnis+YFJ7Zlkcc647yPRRwY9nYBNywahnt5kIeuT1rTvT
sMBZWvmUoEVUj1Xg8XXQkBvb9OzyGqy/G/p8KDDpzMP/U+XCnUeHiXTZrgnqgBIc
2cKeCVvWFqDi0GRFGzyaXLaX3PPmM7DJ0MIPL1qgmcDq6+7Ze0gJ9SrDYFAeLmbu
T1OqDfufXWQl/82JXeiwU2cOpqWq7n5fvPOWim7l1tzQ+dSiMRRm0xa6uNexCJww
3oJSwvMbAmgzvOhqqhlqv+K7u0u7FrFFojESsL36Gq4GBrISnvu2tk7u4GGNTYYQ
bQIDAQAB
-----END PUBLIC KEY-----
"""
def test_get_rsa_pub_key_bad_key(tmp_path):
"""
@ -18,3 +108,64 @@ def test_get_rsa_pub_key_bad_key(tmp_path):
fp.write("")
with pytest.raises(salt.crypt.InvalidKeyError):
salt.crypt.get_rsa_pub_key(key_path)
def test_cryptical_dumps_no_nonce():
master_crypt = salt.crypt.Crypticle({}, salt.crypt.Crypticle.generate_key_string())
data = {"foo": "bar"}
ret = master_crypt.dumps(data)
# Validate message structure
assert isinstance(ret, bytes)
une = master_crypt.decrypt(ret)
une.startswith(master_crypt.PICKLE_PAD)
assert salt.payload.loads(une[len(master_crypt.PICKLE_PAD) :]) == data
# Validate load back to orig data
assert master_crypt.loads(ret) == data
def test_cryptical_dumps_valid_nonce():
nonce = uuid.uuid4().hex
master_crypt = salt.crypt.Crypticle({}, salt.crypt.Crypticle.generate_key_string())
data = {"foo": "bar"}
ret = master_crypt.dumps(data, nonce=nonce)
assert isinstance(ret, bytes)
une = master_crypt.decrypt(ret)
une.startswith(master_crypt.PICKLE_PAD)
nonce_and_data = une[len(master_crypt.PICKLE_PAD) :]
assert nonce_and_data.startswith(nonce.encode())
assert salt.payload.loads(nonce_and_data[len(nonce) :]) == data
assert master_crypt.loads(ret, nonce=nonce) == data
def test_cryptical_dumps_invalid_nonce():
nonce = uuid.uuid4().hex
master_crypt = salt.crypt.Crypticle({}, salt.crypt.Crypticle.generate_key_string())
data = {"foo": "bar"}
ret = master_crypt.dumps(data, nonce=nonce)
assert isinstance(ret, bytes)
with pytest.raises(salt.crypt.SaltClientError, match="Nonce verification error"):
assert master_crypt.loads(ret, nonce="abcde")
def test_verify_signature(tmpdir):
tmpdir.join("foo.pem").write(PRIV_KEY.strip())
tmpdir.join("foo.pub").write(PUB_KEY.strip())
tmpdir.join("bar.pem").write(PRIV_KEY2.strip())
tmpdir.join("bar.pub").write(PUB_KEY2.strip())
msg = b"foo bar"
sig = salt.crypt.sign_message(str(tmpdir.join("foo.pem")), msg)
assert salt.crypt.verify_signature(str(tmpdir.join("foo.pub")), msg, sig)
def test_verify_signature_bad_sig(tmpdir):
tmpdir.join("foo.pem").write(PRIV_KEY.strip())
tmpdir.join("foo.pub").write(PUB_KEY.strip())
tmpdir.join("bar.pem").write(PRIV_KEY2.strip())
tmpdir.join("bar.pub").write(PUB_KEY2.strip())
msg = b"foo bar"
sig = salt.crypt.sign_message(str(tmpdir.join("foo.pem")), msg)
assert not salt.crypt.verify_signature(str(tmpdir.join("bar.pub")), msg, sig)

1
tests/pytests/unit/test_minion.py
View file

@ -10,6 +10,7 @@ import salt.minion
import salt.syspaths
import salt.utils.crypt
import salt.utils.event as event
import salt.utils.jid
import salt.utils.platform
import salt.utils.process
from salt._compat import ipaddress

187
tests/pytests/unit/transport/test_tcp.py
View file

@ -2,6 +2,7 @@ import socket
import attr
import pytest
import salt.channel.server
import salt.exceptions
import salt.ext.tornado
import salt.transport.tcp
@ -79,84 +80,118 @@ def test_message_client_cleanup_on_close(client_socket, temp_salt_master):
orig_loop.close(all_fds=True)
# XXX: Test channel for this
# def test_tcp_pub_server_channel_publish_filtering(temp_salt_master):
# opts = dict(
# temp_salt_master.config.copy(),
# sign_pub_messages=False,
# transport="tcp",
# acceptance_wait_time=5,
# acceptance_wait_time_max=5,
# )
# with patch("salt.master.SMaster.secrets") as secrets, patch(
# "salt.crypt.Crypticle"
# ) as crypticle, patch("salt.utils.asynchronous.SyncWrapper") as SyncWrapper:
# channel = salt.transport.tcp.TCPPubServerChannel(opts)
# wrap = MagicMock()
# crypt = MagicMock()
# crypt.dumps.return_value = {"test": "value"}
#
# secrets.return_value = {"aes": {"secret": None}}
# crypticle.return_value = crypt
# SyncWrapper.return_value = wrap
#
# # try simple publish with glob tgt_type
# channel.publish({"test": "value", "tgt_type": "glob", "tgt": "*"})
# payload = wrap.send.call_args[0][0]
#
# # verify we send it without any specific topic
# assert "topic_lst" not in payload
#
# # try simple publish with list tgt_type
# channel.publish({"test": "value", "tgt_type": "list", "tgt": ["minion01"]})
# payload = wrap.send.call_args[0][0]
#
# # verify we send it with correct topic
# assert "topic_lst" in payload
# assert payload["topic_lst"] == ["minion01"]
#
# # try with syndic settings
# opts["order_masters"] = True
# channel.publish({"test": "value", "tgt_type": "list", "tgt": ["minion01"]})
# payload = wrap.send.call_args[0][0]
#
# # verify we send it without topic for syndics
# assert "topic_lst" not in payload
async def test_async_tcp_pub_channel_connect_publish_port(
temp_salt_master, client_socket
):
"""
test when publish_port is not 4506
"""
opts = dict(
temp_salt_master.config.copy(),
master_uri="tcp://127.0.0.1:1234",
master_ip="127.0.0.1",
publish_port=1234,
transport="tcp",
acceptance_wait_time=5,
acceptance_wait_time_max=5,
)
patch_auth = MagicMock(return_value=True)
transport = MagicMock(spec=salt.transport.tcp.TCPPubClient)
with patch("salt.crypt.AsyncAuth.gen_token", patch_auth), patch(
"salt.crypt.AsyncAuth.authenticated", patch_auth
), patch("salt.transport.tcp.TCPPubClient", transport):
channel = salt.channel.client.AsyncPubChannel.factory(opts)
with channel:
# We won't be able to succeed the connection because we're not mocking the tornado coroutine
with pytest.raises(salt.exceptions.SaltClientError):
await channel.connect()
# The first call to the mock is the instance's __init__, and the first argument to those calls is the opts dict
assert channel.transport.connect.call_args[0][0] == opts["publish_port"]
# def test_tcp_pub_server_channel_publish_filtering_str_list(temp_salt_master):
# opts = dict(
# temp_salt_master.config.copy(),
# transport="tcp",
# sign_pub_messages=False,
# acceptance_wait_time=5,
# acceptance_wait_time_max=5,
# )
# with patch("salt.master.SMaster.secrets") as secrets, patch(
# "salt.crypt.Crypticle"
# ) as crypticle, patch("salt.utils.asynchronous.SyncWrapper") as SyncWrapper, patch(
# "salt.utils.minions.CkMinions.check_minions"
# ) as check_minions:
# channel = salt.transport.tcp.TCPPubServerChannel(opts)
# wrap = MagicMock()
# crypt = MagicMock()
# crypt.dumps.return_value = {"test": "value"}
#
# secrets.return_value = {"aes": {"secret": None}}
# crypticle.return_value = crypt
# SyncWrapper.return_value = wrap
# check_minions.return_value = {"minions": ["minion02"]}
#
# # try simple publish with list tgt_type
# channel.publish({"test": "value", "tgt_type": "list", "tgt": "minion02"})
# payload = wrap.send.call_args[0][0]
#
# # verify we send it with correct topic
# assert "topic_lst" in payload
# assert payload["topic_lst"] == ["minion02"]
#
# # verify it was correctly calling check_minions
# check_minions.assert_called_with("minion02", tgt_type="list")
def test_tcp_pub_server_channel_publish_filtering(temp_salt_master):
opts = dict(
temp_salt_master.config.copy(),
sign_pub_messages=False,
transport="tcp",
acceptance_wait_time=5,
acceptance_wait_time_max=5,
)
with patch("salt.master.SMaster.secrets") as secrets, patch(
"salt.crypt.Crypticle"
) as crypticle, patch("salt.utils.asynchronous.SyncWrapper") as SyncWrapper:
channel = salt.channel.server.PubServerChannel.factory(opts)
wrap = MagicMock()
crypt = MagicMock()
crypt.dumps.return_value = {"test": "value"}
secrets.return_value = {"aes": {"secret": None}}
crypticle.return_value = crypt
SyncWrapper.return_value = wrap
# try simple publish with glob tgt_type
payload = channel.wrap_payload(
{"test": "value", "tgt_type": "glob", "tgt": "*"}
)
# verify we send it without any specific topic
assert "topic_lst" in payload
assert payload["topic_lst"] == [] # "minion01"]
# try simple publish with list tgt_type
payload = channel.wrap_payload(
{"test": "value", "tgt_type": "list", "tgt": ["minion01"]}
)
# verify we send it with correct topic
assert "topic_lst" in payload
assert payload["topic_lst"] == ["minion01"]
# try with syndic settings
opts["order_masters"] = True
channel = salt.channel.server.PubServerChannel.factory(opts)
payload = channel.wrap_payload(
{"test": "value", "tgt_type": "list", "tgt": ["minion01"]}
)
# verify we send it without topic for syndics
assert "topic_lst" not in payload
def test_tcp_pub_server_channel_publish_filtering_str_list(temp_salt_master):
opts = dict(
temp_salt_master.config.copy(),
transport="tcp",
sign_pub_messages=False,
acceptance_wait_time=5,
acceptance_wait_time_max=5,
)
with patch("salt.master.SMaster.secrets") as secrets, patch(
"salt.crypt.Crypticle"
) as crypticle, patch("salt.utils.asynchronous.SyncWrapper") as SyncWrapper, patch(
"salt.utils.minions.CkMinions.check_minions"
) as check_minions:
channel = salt.channel.server.PubServerChannel.factory(opts)
wrap = MagicMock()
crypt = MagicMock()
crypt.dumps.return_value = {"test": "value"}
secrets.return_value = {"aes": {"secret": None}}
crypticle.return_value = crypt
SyncWrapper.return_value = wrap
check_minions.return_value = {"minions": ["minion02"]}
# try simple publish with list tgt_type
payload = channel.wrap_payload(
{"test": "value", "tgt_type": "list", "tgt": "minion02"}
)
# verify we send it with correct topic
assert "topic_lst" in payload
assert payload["topic_lst"] == ["minion02"]
# verify it was correctly calling check_minions
check_minions.assert_called_with("minion02", tgt_type="list")
@pytest.fixture(scope="function")

1038
tests/pytests/unit/transport/test_zeromq.py
View file

File diff suppressed because it is too large Load diff

59
tests/pytests/unit/utils/test_minions.py
View file

@ -1,3 +1,4 @@
import pytest
import salt.utils.minions
import salt.utils.network
from tests.support.mock import patch
@ -52,3 +53,61 @@ def test_connected_ids_remote_minions():
with patch_net, patch_list, patch_fetch, patch_remote_net:
ret = ckminions.connected_ids()
assert ret == {minion2, minion}
# These validate_tgt tests make the assumption that CkMinions.check_minions is
# correct. In other words, these tests are only worthwhile if check_minions is
# also correct.
def test_validate_tgt_should_return_false_when_no_valid_minions_have_been_found():
ckminions = salt.utils.minions.CkMinions(opts={})
with patch(
"salt.utils.minions.CkMinions.check_minions", autospec=True, return_value={}
):
result = ckminions.validate_tgt("fnord", "fnord", "fnord", minions=[])
assert result is False
@pytest.mark.parametrize(
"valid_minions, target_minions",
[
(["one", "two", "three"], ["one", "two", "five"]),
(["one"], ["one", "two"]),
(["one", "two", "three", "four"], ["five"]),
],
)
def test_validate_tgt_should_return_false_when_minions_have_minions_not_in_valid_minions(
valid_minions, target_minions
):
ckminions = salt.utils.minions.CkMinions(opts={})
with patch(
"salt.utils.minions.CkMinions.check_minions",
autospec=True,
return_value={"minions": valid_minions},
):
result = ckminions.validate_tgt(
"fnord", "fnord", "fnord", minions=target_minions
)
assert result is False
@pytest.mark.parametrize(
"valid_minions, target_minions",
[
(["one", "two", "three", "five"], ["one", "two", "five"]),
(["one"], ["one"]),
(["one", "two", "three", "four", "five"], ["five"]),
],
)
def test_validate_tgt_should_return_true_when_all_minions_are_found_in_valid_minions(
valid_minions, target_minions
):
ckminions = salt.utils.minions.CkMinions(opts={})
with patch(
"salt.utils.minions.CkMinions.check_minions",
autospec=True,
return_value={"minions": valid_minions},
):
result = ckminions.validate_tgt(
"fnord", "fnord", "fnord", minions=target_minions
)
assert result is True

8
tests/pytests/unit/utils/test_network.py Normal file
View file

@ -0,0 +1,8 @@
import salt.utils.network
def test_junos_ifconfig_output_parsing():
ret = salt.utils.network._junos_interfaces_ifconfig(
"inet mtu 0 local=" + " " * 3456
)
assert ret == {"inet": {"up": False}}

2
tests/unit/transport/test_ipc.py
View file

@ -38,6 +38,8 @@ class IPCMessagePubSubCase(salt.ext.tornado.testing.AsyncTestCase):
def setUp(self):
super().setUp()
self.opts = {"ipc_write_buffer": 0}
if not os.path.exists(RUNTIME_VARS.TMP):
os.mkdir(RUNTIME_VARS.TMP)
self.socket_path = os.path.join(RUNTIME_VARS.TMP, "ipc_test.ipc")
self.pub_channel = self._get_pub_channel()
self.sub_channel = self._get_sub_channel()