saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge remote-tracking branch 'upstream/2017.7' into SuperPommeDeTerre-patch-#26995

Browse source
This commit is contained in:
Jérémie Langlade 2017-08-25 16:45:24 +02:00
commit 30dd6f5d12
77 changed files with 2404 additions and 1659 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
conf/master
View file

@ -59,15 +59,14 @@
# Directory for custom modules. This directory can contain subdirectories for
# each of Salt's module types such as "runners", "output", "wheel", "modules",
# "states", "returners", etc.
#extension_modules: <no default>
# "states", "returners", "engines", "utils", etc.
#extension_modules: /var/cache/salt/master/extmods
# Directory for custom modules. This directory can contain subdirectories for
# each of Salt's module types such as "runners", "output", "wheel", "modules",
# "states", "returners", "engines", etc.
# "states", "returners", "engines", "utils", etc.
# Like 'extension_modules' but can take an array of paths
#module_dirs: <no default>
# - /var/cache/salt/minion/extmods
#module_dirs: []
# Verify and set permissions on configuration directories at startup:
#verify_env: True

16
doc/ref/cli/salt-cp.rst
View file

@ -39,6 +39,13 @@ specified target expression.
desitination will be assumed to be a directory. Finally, recursion is now
supported, allowing for entire directories to be copied.
.. versionchanged:: 2016.11.7,2017.7.2
Reverted back to the old copy mode to preserve backward compatibility. The
new functionality added in 2016.6.6 and 2017.7.0 is now available using the
``-C`` or ``--chunked`` CLI arguments. Note that compression, recursive
copying, and support for copying large files is only available in chunked
mode.
Options
=======
@ -56,9 +63,16 @@ Options
.. include:: _includes/target-selection.rst
.. option:: -C, --chunked
Use new chunked mode to copy files. This mode supports large files, recursive
directories copying and compression.
.. versionadded:: 2016.11.7,2017.7.2
.. option:: -n, --no-compression
Disable gzip compression.
Disable gzip compression in chunked mode.
.. versionadded:: 2016.3.7,2016.11.6,2017.7.0

4
doc/ref/configuration/master.rst
View file

@ -183,8 +183,8 @@ The directory to store the pki authentication keys.
Directory for custom modules. This directory can contain subdirectories for
each of Salt's module types such as ``runners``, ``output``, ``wheel``,
``modules``, ``states``, ``returners``, ``engines``, etc. This path is appended to
:conf_master:`root_dir`.
``modules``, ``states``, ``returners``, ``engines``, ``utils``, etc.
This path is appended to :conf_master:`root_dir`.
.. code-block:: yaml

6
doc/topics/cloud/action.rst
View file

@ -21,7 +21,7 @@ Or you may specify a map which includes all VMs to perform the action on:
$ salt-cloud -a reboot -m /path/to/mapfile
The following is a list of actions currently supported by salt-cloud:
The following is an example list of actions currently supported by ``salt-cloud``:
.. code-block:: yaml
@ -36,5 +36,5 @@ The following is a list of actions currently supported by salt-cloud:
- start
- stop
Another useful reference for viewing more salt-cloud actions is the
:ref:Salt Cloud Feature Matrix <salt-cloud-feature-matrix>
Another useful reference for viewing more ``salt-cloud`` actions is the
:ref:`Salt Cloud Feature Matrix <salt-cloud-feature-matrix>`.

2
doc/topics/cloud/function.rst
View file

@ -26,5 +26,5 @@ gathering information about instances on a provider basis:
$ salt-cloud -f list_nodes_full linode
$ salt-cloud -f list_nodes_select linode
Another useful reference for viewing salt-cloud functions is the
Another useful reference for viewing ``salt-cloud`` functions is the
:ref:`Salt Cloud Feature Matrix <salt-cloud-feature-matrix>`.

2
doc/topics/cloud/joyent.rst
View file

@ -49,7 +49,7 @@ Set up an initial profile at ``/etc/salt/cloud.profiles`` or in the
.. code-block:: yaml
joyent_512
joyent_512:
provider: my-joyent-config
size: g4-highcpu-512M
image: ubuntu-16.04

10
doc/topics/releases/2016.11.7.rst
View file

@ -3,3 +3,13 @@ Salt 2016.11.7 Release Notes
============================
Version 2016.11.7 is a bugfix release for :ref:`2016.11.0 <release-2016-11-0>`.
Changes for v2016.11.6..v2016.11.7
----------------------------------
Security Fix
============
CVE-2017-12791 Maliciously crafted minion IDs can cause unwanted directory traversals on the Salt-master
Correct a flaw in minion id validation which could allow certain minions to authenticate to a master despite not having the correct credentials. To exploit the vulnerability, an attacker must create a salt-minion with an ID containing characters that will cause a directory traversal. Credit for discovering the security flaw goes to: Vernhk@qq.com

23
doc/topics/releases/2016.3.7.rst
View file

@ -4,23 +4,12 @@ Salt 2016.3.7 Release Notes
Version 2016.3.7 is a bugfix release for :ref:`2016.3.0 <release-2016-3-0>`.
New master configuration option `allow_minion_key_revoke`, defaults to True. This option
controls whether a minion can request that the master revoke its key. When True, a minion
can request a key revocation and the master will comply. If it is False, the key will not
be revoked by the msater.
Changes for v2016.3.6..v2016.3.7
--------------------------------
New master configuration option `require_minion_sign_messages`
This requires that minions cryptographically sign the messages they
publish to the master. If minions are not signing, then log this information
at loglevel 'INFO' and drop the message without acting on it.
Security Fix
============
New master configuration option `drop_messages_signature_fail`
Drop messages from minions when their signatures do not validate.
Note that when this option is False but `require_minion_sign_messages` is True
minions MUST sign their messages but the validity of their signatures
is ignored.
CVE-2017-12791 Maliciously crafted minion IDs can cause unwanted directory traversals on the Salt-master
New minion configuration option `minion_sign_messages`
Causes the minion to cryptographically sign the payload of messages it places
on the event bus for the master. The payloads are signed with the minion's
private key so the master can verify the signature with its public key.
Correct a flaw in minion id validation which could allow certain minions to authenticate to a master despite not having the correct credentials. To exploit the vulnerability, an attacker must create a salt-minion with an ID containing characters that will cause a directory traversal. Credit for discovering the security flaw goes to: Vernhk@qq.com

29
doc/topics/releases/2016.3.8.rst Normal file
View file

@ -0,0 +1,29 @@
===========================
Salt 2016.3.8 Release Notes
===========================
Version 2016.3.8 is a bugfix release for :ref:`2016.3.0 <release-2016-3-0>`.
Changes for v2016.3.7..v2016.3.8
--------------------------------
New master configuration option `allow_minion_key_revoke`, defaults to True. This option
controls whether a minion can request that the master revoke its key. When True, a minion
can request a key revocation and the master will comply. If it is False, the key will not
be revoked by the msater.
New master configuration option `require_minion_sign_messages`
This requires that minions cryptographically sign the messages they
publish to the master. If minions are not signing, then log this information
at loglevel 'INFO' and drop the message without acting on it.
New master configuration option `drop_messages_signature_fail`
Drop messages from minions when their signatures do not validate.
Note that when this option is False but `require_minion_sign_messages` is True
minions MUST sign their messages but the validity of their signatures
is ignored.
New minion configuration option `minion_sign_messages`
Causes the minion to cryptographically sign the payload of messages it places
on the event bus for the master. The payloads are signed with the minion's
private key so the master can verify the signature with its public key.

4
doc/topics/utils/index.rst
View file

@ -87,8 +87,8 @@ Also you could even write your utility modules in object oriented fashion:
# -*- coding: utf-8 -*-
'''
My utils module
---------------
My OOP-style utils module
-------------------------
This module contains common functions for use in my other custom types.
'''

94
pkg/osx/pkg-scripts/postinstall
View file

@ -15,91 +15,119 @@
# This script is run as a part of the macOS Salt Installation
#
###############################################################################
echo "Post install started on:" > /tmp/postinstall.txt
date >> /tmp/postinstall.txt
###############################################################################
# Define Variables
###############################################################################
# Get Minor Version
OSX_VERSION=$(sw_vers | grep ProductVersion | cut -f 2 -d: | tr -d '[:space:]')
MINOR=$(echo ${OSX_VERSION} | cut -f 2 -d.)
# Path Variables
INSTALL_DIR="/opt/salt"
BIN_DIR="$INSTALL_DIR/bin"
CONFIG_DIR="/etc/salt"
TEMP_DIR="/tmp"
SBIN_DIR="/usr/local/sbin"
###############################################################################
# Set up logging and error handling
###############################################################################
echo "Post install script started on:" > "$TEMP_DIR/postinstall.txt"
date "+%Y/%m/%d %H:%m:%S" >> "$TEMP_DIR/postinstall.txt"
trap 'quit_on_error $LINENO $BASH_COMMAND' ERR
quit_on_error() {
echo "$(basename $0) caught error on line : $1 command was: $2" >> /tmp/postinstall.txt
echo "$(basename $0) caught error on line : $1 command was: $2" >> "$TEMP_DIR/postinstall.txt"
exit -1
}
###############################################################################
# Check for existing minion config, copy if it doesn't exist
###############################################################################
if [ ! -f /etc/salt/minion ]; then
echo "Config copy: Started..." >> /tmp/postinstall.txt
cp /etc/salt/minion.dist /etc/salt/minion
echo "Config copy: Successful" >> /tmp/postinstall.txt
if [ ! -f "$CONFIG_DIR/minion" ]; then
echo "Config: Copy Started..." >> "$TEMP_DIR/postinstall.txt"
cp "$CONFIG_DIR/minion.dist" "$CONFIG_DIR/minion"
echo "Config: Copied Successfully" >> "$TEMP_DIR/postinstall.txt"
fi
###############################################################################
# Create symlink to salt-config.sh
###############################################################################
# echo "Symlink: Creating symlink for salt-config..." >> /tmp/postinstall.txt
if [ ! -d "/usr/local/sbin" ]; then
mkdir /usr/local/sbin
if [ ! -d "$SBIN_DIR" ]; then
echo "Symlink: Creating $SBIN_DIR..." >> "$TEMP_DIR/postinstall.txt"
mkdir "$SBIN_DIR"
echo "Symlink: Created Successfully" >> "$TEMP_DIR/postinstall.txt"
fi
ln -sf /opt/salt/bin/salt-config.sh /usr/local/sbin/salt-config
echo "Symlink: Creating symlink for salt-config..." >> "$TEMP_DIR/postinstall.txt"
ln -sf "$BIN_DIR/salt-config.sh" "$SBIN_DIR/salt-config"
echo "Symlink: Created Successfully" >> "$TEMP_DIR/postinstall.txt"
###############################################################################
# Add salt to paths.d
###############################################################################
# echo "Path: Adding salt to the path..." >> /tmp/postinstall.txt
if [ ! -d "/etc/paths.d" ]; then
echo "Path: Creating paths.d directory..." >> "$TEMP_DIR/postinstall.txt"
mkdir /etc/paths.d
echo "Path: Created Successfully" >> "$TEMP_DIR/postinstall.txt"
fi
sh -c 'echo "/opt/salt/bin" > /etc/paths.d/salt'
sh -c 'echo "/usr/local/sbin" >> /etc/paths.d/salt'
echo "Path: Adding salt to the path..." >> "$TEMP_DIR/postinstall.txt"
sh -c "echo \"$BIN_DIR\" > /etc/paths.d/salt"
sh -c "echo \"$SBIN_DIR\" >> /etc/paths.d/salt"
echo "Path: Added Successfully" >> "$TEMP_DIR/postinstall.txt"
###############################################################################
# Register Salt as a service
###############################################################################
setup_services_maverick() {
echo "Using old (< 10.10) launchctl interface" >> /tmp/postinstall.txt
echo "Service: Using old (< 10.10) launchctl interface" >> "$TEMP_DIR/postinstall.txt"
if /bin/launchctl list "com.saltstack.salt.minion" &> /dev/null; then
echo "Stop running service..." >> /tmp/postinstall.txt
echo "Service: Stopping salt-minion..." >> "$TEMP_DIR/postinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.minion.plist
echo "Service: Stopped Successfully" >> "$TEMP_DIR/postinstall.txt"
fi;
echo "Service: Starting salt-minion..." >> "$TEMP_DIR/postinstall.txt"
launchctl load -w /Library/LaunchDaemons/com.saltstack.salt.minion.plist || return 1
echo "Service: Started Successfully" >> "$TEMP_DIR/postinstall.txt"
echo "Service start: Successful" >> /tmp/postinstall.txt
echo "Service disable: Disabling Master, Syndic, and API" >> /tmp/postinstall.txt
echo "Service: Disabling Master, Syndic, and API services..." >> "$TEMP_DIR/postinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.api.plist
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.master.plist
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.syndic.plist
echo "Service: Disabled Successfully" >> "$TEMP_DIR/postinstall.txt"
return 0
}
setup_services_yosemite_and_later() {
echo "Using new (>= 10.10) launchctl interface" >> /tmp/postinstall.txt
echo "Service: Using new (>= 10.10) launchctl interface" >> "$TEMP_DIR/postinstall.txt"
echo "Service: Enabling salt-minion..." >> "$TEMP_DIR/postinstall.txt"
launchctl enable system/com.saltstack.salt.minion
echo "Service start: Bootstrapping service..." >> /tmp/postinstall.txt
echo "Service: Enabled Successfully" >> "$TEMP_DIR/postinstall.txt"
echo "Service: Bootstrapping salt-minion..." >> "$TEMP_DIR/postinstall.txt"
launchctl bootstrap system /Library/LaunchDaemons/com.saltstack.salt.minion.plist
echo "Service: Bootstrapped Successfully" >> "$TEMP_DIR/postinstall.txt"
if /bin/launchctl list "com.saltstack.salt.minion" &> /dev/null; then
echo "Service is running" >> /tmp/postinstall.txt
echo "Service: Service Running" >> "$TEMP_DIR/postinstall.txt"
else
echo "Service start: Kickstarting service..." >> /tmp/postinstall.txt
echo "Service: Kickstarting Service..." >> "$TEMP_DIR/postinstall.txt"
launchctl kickstart -kp system/com.saltstack.salt.minion
echo "Service: Kickstarted Successfully" >> "$TEMP_DIR/postinstall.txt"
fi
echo "Service start: Successful" >> /tmp/postinstall.txt
echo "Service disable: Disabling Master, Syndic, and API" >> /tmp/postinstall.txt
echo "Service: Started Successfully" >> "$TEMP_DIR/postinstall.txt"
echo "Service: Disabling Master, Syndic, and API services" >> "$TEMP_DIR/postinstall.txt"
launchctl disable system/com.saltstack.salt.master
launchctl disable system/com.saltstack.salt.syndic
launchctl disable system/com.saltstack.salt.api
echo "Service: Disabled Successfully" >> "$TEMP_DIR/postinstall.txt"
return 0
}
OSX_VERSION=$(sw_vers | grep ProductVersion | cut -f 2 -d: | tr -d '[:space:]')
MINOR=$(echo ${OSX_VERSION} | cut -f 2 -d.)
echo "Service start: Enabling service..." >> /tmp/postinstall.txt
echo "Service: Configuring..." >> "$TEMP_DIR/postinstall.txt"
case $MINOR in
9 )
setup_services_maverick;
@ -108,7 +136,9 @@ case $MINOR in
setup_services_yosemite_and_later;
;;
esac
echo "Service: Configured Successfully" >> "$TEMP_DIR/postinstall.txt"
echo "Post install completed successfully" >> /tmp/postinstall.txt
echo "Post install completed successfully on:" >> "$TEMP_DIR/postinstall.txt"
date "+%Y/%m/%d %H:%m:%S" >> "$TEMP_DIR/postinstall.txt"
exit 0

104
pkg/osx/pkg-scripts/preinstall
View file

@ -6,7 +6,8 @@
# Date: December 2015
#
# Description: This script stops the salt minion service before attempting to
# install Salt on macOS
# install Salt on macOS. It also removes the /opt/salt/bin
# directory, symlink to salt-config, and salt from paths.d.
#
# Requirements:
# - None
@ -15,12 +16,29 @@
# This script is run as a part of the macOS Salt Installation
#
###############################################################################
echo "Preinstall started on:" > /tmp/preinstall.txt
date >> /tmp/preinstall.txt
###############################################################################
# Define Variables
###############################################################################
# Get Minor Version
OSX_VERSION=$(sw_vers | grep ProductVersion | cut -f 2 -d: | tr -d '[:space:]')
MINOR=$(echo ${OSX_VERSION} | cut -f 2 -d.)
# Path Variables
INSTALL_DIR="/opt/salt"
BIN_DIR="$INSTALL_DIR/bin"
CONFIG_DIR="/etc/salt"
TEMP_DIR="/tmp"
SBIN_DIR="/usr/local/sbin"
###############################################################################
# Set up logging and error handling
###############################################################################
echo "Preinstall started on:" > "$TEMP_DIR/preinstall.txt"
date "+%Y/%m/%d %H:%m:%S" >> "$TEMP_DIR/preinstall.txt"
trap 'quit_on_error $LINENO $BASH_COMMAND' ERR
quit_on_error() {
echo "$(basename $0) caught error on line : $1 command was: $2" >> /tmp/preinstall.txt
echo "$(basename $0) caught error on line : $1 command was: $2" >> "$TEMP_DIR/preinstall.txt"
exit -1
}
@ -31,24 +49,58 @@ MINOR=$(echo ${OSX_VERSION} | cut -f 2 -d.)
# Stop the service
###############################################################################
stop_service_maverick() {
echo "Using old (< 10.10) launchctl interface" >> /tmp/preinstall.txt
echo "Service: Using old (< 10.10) launchctl interface" >> "$TEMP_DIR/preinstall.txt"
if /bin/launchctl list "com.saltstack.salt.minion" &> /dev/null; then
echo "Stop service: Started..." >> /tmp/preinstall.txt
echo "Service: Unloading minion..." >> "$TEMP_DIR/preinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.minion.plist
echo "Stop service: Successful" >> /tmp/preinstall.txt
echo "Service: Unloaded Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.master" &> /dev/null; then
echo "Service: Unloading master..." >> "$TEMP_DIR/preinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.master.plist
echo "Service: Unloaded Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.syndic" &> /dev/null; then
echo "Service: Unloading syndic..." >> "$TEMP_DIR/preinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.syndic.plist
echo "Service: Unloaded Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.api" &> /dev/null; then
echo "Service: Unloading api..." >> "$TEMP_DIR/preinstall.txt"
launchctl unload -w /Library/LaunchDaemons/com.saltstack.salt.api.plist
echo "Service: Unloaded Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
}
stop_service_yosemite_and_later() {
echo "Using new (>= 10.10) launchctl interface" >> /tmp/preinstall.txt
echo "Service: Using new (>= 10.10) launchctl interface" >> "$TEMP_DIR/preinstall.txt"
if /bin/launchctl list "com.saltstack.salt.minion" &> /dev/null; then
echo "Stop service: Started..." >> /tmp/preinstall.txt
echo "Service: Stopping minion..." >> "$TEMP_DIR/preinstall.txt"
launchctl disable system/com.saltstack.salt.minion
launchctl bootout system /Library/LaunchDaemons/com.saltstack.salt.minion.plist
echo "Stop service: Successful" >> /tmp/preinstall.txt
echo "Service: Stopped Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.master" &> /dev/null; then
echo "Service: Stopping master..." >> "$TEMP_DIR/preinstall.txt"
launchctl disable system/com.saltstack.salt.master
launchctl bootout system /Library/LaunchDaemons/com.saltstack.salt.master.plist
echo "Service: Stopped Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.syndic" &> /dev/null; then
echo "Service: Stopping syndic..." >> "$TEMP_DIR/preinstall.txt"
launchctl disable system/com.saltstack.salt.syndic
launchctl bootout system /Library/LaunchDaemons/com.saltstack.salt.syndic.plist
echo "Service: Stopped Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
if /bin/launchctl list "com.saltstack.salt.api" &> /dev/null; then
echo "Service: Stopping api..." >> "$TEMP_DIR/preinstall.txt"
launchctl disable system/com.saltstack.salt.api
launchctl bootout system /Library/LaunchDaemons/com.saltstack.salt.api.plist
echo "Service: Stopped Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
}
echo "Service: Configuring..." >> "$TEMP_DIR/preinstall.txt"
case $MINOR in
9 )
stop_service_maverick;
@ -57,6 +109,36 @@ case $MINOR in
stop_service_yosemite_and_later;
;;
esac
echo "Preinstall Completed Successfully" >> /tmp/preinstall.txt
echo "Service: Configured Successfully" >> "$TEMP_DIR/preinstall.txt"
###############################################################################
# Remove the Symlink to salt-config.sh
###############################################################################
if [ -L "$SBIN_DIR/salt-config" ]; then
echo "Cleanup: Removing Symlink $BIN_DIR/salt-config" >> "$TEMP_DIR/preinstall.txt"
rm "$SBIN_DIR/salt-config"
echo "Cleanup: Removed Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
###############################################################################
# Remove the $INSTALL_DIR directory
###############################################################################
if [ -d "$INSTALL_DIR" ]; then
echo "Cleanup: Removing $INSTALL_DIR" >> "$TEMP_DIR/preinstall.txt"
rm -rf "$INSTALL_DIR"
echo "Cleanup: Removed Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
###############################################################################
# Remove the salt from the paths.d
###############################################################################
if [ ! -f "/etc/paths.d/salt" ]; then
echo "Path: Removing salt from the path..." >> "$TEMP_DIR/preinstall.txt"
rm "/etc/paths.d/salt"
echo "Path: Removed Successfully" >> "$TEMP_DIR/preinstall.txt"
fi
echo "Preinstall Completed Successfully on:" >> "$TEMP_DIR/preinstall.txt"
date "+%Y/%m/%d %H:%m:%S" >> "$TEMP_DIR/preinstall.txt"
exit 0

6
salt/cache/__init__.py vendored
View file

@ -224,7 +224,7 @@ class Cache(object):
fun = '{0}.flush'.format(self.driver)
return self.modules[fun](bank, key=key, **self._kwargs)
def ls(self, bank):
def list(self, bank):
'''
Lists entries stored in the specified bank.
@ -240,11 +240,9 @@ class Cache(object):
Raises an exception if cache driver detected an error accessing data
in the cache backend (auth, permissions, etc).
'''
fun = '{0}.ls'.format(self.driver)
fun = '{0}.list'.format(self.driver)
return self.modules[fun](bank, **self._kwargs)
list = ls
def contains(self, bank, key=None):
'''
Checks if the specified bank contains the specified key.

4
salt/cache/consul.py vendored
View file

@ -61,7 +61,7 @@ api = None
# Define the module's virtual name
__virtualname__ = 'consul'
__func_alias__ = {'list': 'ls'}
__func_alias__ = {'list_': 'list'}
def __virtual__():
@ -139,7 +139,7 @@ def flush(bank, key=None):
)
def ls(bank):
def list_(bank):
'''
Return an iterable object containing all entries stored in the specified bank.
'''

4
salt/cache/localfs.py vendored
View file

@ -23,7 +23,7 @@ import salt.utils.atomicfile
log = logging.getLogger(__name__)
__func_alias__ = {'list': 'ls'}
__func_alias__ = {'list_': 'list'}
def __cachedir(kwargs=None):
@ -143,7 +143,7 @@ def flush(bank, key=None, cachedir=None):
return True
def ls(bank, cachedir):
def list_(bank, cachedir):
'''
Return an iterable object containing all entries stored in the specified bank.
'''

6
salt/cache/redis_cache.py vendored
View file

@ -114,9 +114,7 @@ from salt.exceptions import SaltCacheError
# -----------------------------------------------------------------------------
__virtualname__ = 'redis'
__func_alias__ = {
'ls': 'list'
}
__func_alias__ = {'list_': 'list'}
log = logging.getLogger(__file__)
@ -418,7 +416,7 @@ def flush(bank, key=None):
return True
def ls(bank):
def list_(bank):
'''
Lists entries stored in the specified bank.
'''

70
salt/cli/cp.py
View file

@ -21,7 +21,7 @@ import salt.client
import salt.utils.gzip_util
import salt.utils.itertools
import salt.utils.minions
from salt.utils import parsers, to_bytes
from salt.utils import parsers, to_bytes, print_cli
from salt.utils.verify import verify_log
import salt.output
@ -101,10 +101,69 @@ class SaltCP(object):
empty_dirs.update(empty_dirs_)
return files, sorted(empty_dirs)
def _file_dict(self, fn_):
'''
Take a path and return the contents of the file as a string
'''
if not os.path.isfile(fn_):
err = 'The referenced file, {0} is not available.'.format(fn_)
sys.stderr.write(err + '\n')
sys.exit(42)
with salt.utils.fopen(fn_, 'r') as fp_:
data = fp_.read()
return {fn_: data}
def _load_files(self):
'''
Parse the files indicated in opts['src'] and load them into a python
object for transport
'''
files = {}
for fn_ in self.opts['src']:
if os.path.isfile(fn_):
files.update(self._file_dict(fn_))
elif os.path.isdir(fn_):
print_cli(fn_ + ' is a directory, only files are supported in non-chunked mode. '
'Use "--chunked" command line argument.')
sys.exit(1)
return files
def run(self):
'''
Make the salt client call
'''
if self.opts['chunked']:
ret = self.run_chunked()
else:
ret = self.run_oldstyle()
salt.output.display_output(
ret,
self.opts.get('output', 'nested'),
self.opts)
def run_oldstyle(self):
'''
Make the salt client call in old-style all-in-one call method
'''
arg = [self._load_files(), self.opts['dest']]
local = salt.client.get_local_client(self.opts['conf_file'])
args = [self.opts['tgt'],
'cp.recv',
arg,
self.opts['timeout'],
]
selected_target_option = self.opts.get('selected_target_option', None)
if selected_target_option is not None:
args.append(selected_target_option)
return local.cmd(*args)
def run_chunked(self):
'''
Make the salt client call in the new fasion chunked multi-call way
'''
files, empty_dirs = self._list_files()
dest = self.opts['dest']
gzip = self.opts['gzip']
@ -166,7 +225,7 @@ class SaltCP(object):
)
args = [
tgt,
'cp.recv',
'cp.recv_chunked',
[remote_path, chunk, append, gzip, mode],
timeout,
]
@ -212,14 +271,11 @@ class SaltCP(object):
else '',
tgt,
)
args = [tgt, 'cp.recv', [remote_path, None], timeout]
args = [tgt, 'cp.recv_chunked', [remote_path, None], timeout]
if selected_target_option is not None:
args.append(selected_target_option)
for minion_id, minion_ret in six.iteritems(local.cmd(*args)):
ret.setdefault(minion_id, {})[remote_path] = minion_ret
salt.output.display_output(
ret,
self.opts.get('output', 'nested'),
self.opts)
return ret

11
salt/cloud/__init__.py
View file

@ -730,18 +730,9 @@ class Cloud(object):
continue
for vm_name, details in six.iteritems(vms):
# If VM was created with use_fqdn with either of the softlayer drivers,
# we need to strip the VM name and only search for the short hostname.
if driver == 'softlayer' or driver == 'softlayer_hw':
ret = []
for name in names:
name = name.split('.')[0]
ret.append(name)
if vm_name not in ret:
continue
# XXX: The logic below can be removed once the aws driver
# is removed
elif vm_name not in names:
if vm_name not in names:
continue
elif driver == 'ec2' and 'aws' in handled_drivers and \

29
salt/cloud/clouds/ec2.py
View file

@ -3428,34 +3428,7 @@ def list_nodes_full(location=None, call=None):
'or --function.'
)
if not location:
ret = {}
locations = set(
get_location(vm_) for vm_ in six.itervalues(__opts__['profiles'])
if _vm_provider_driver(vm_)
)
# If there aren't any profiles defined for EC2, check
# the provider config file, or use the default location.
if not locations:
locations = [get_location()]
for loc in locations:
ret.update(_list_nodes_full(loc))
return ret
return _list_nodes_full(location)
def _vm_provider_driver(vm_):
alias, driver = vm_['driver'].split(':')
if alias not in __opts__['providers']:
return None
if driver not in __opts__['providers'][alias]:
return None
return driver == 'ec2'
return _list_nodes_full(location or get_location())
def _extract_name_tag(item):

16
salt/cloud/clouds/nova.py
View file

@ -728,12 +728,18 @@ def request_instance(vm_=None, call=None):
else:
pool = floating_ip_conf.get('pool', 'public')
for fl_ip, opts in six.iteritems(conn.floating_ip_list()):
if opts['fixed_ip'] is None and opts['pool'] == pool:
floating_ip = fl_ip
break
if floating_ip is None:
try:
floating_ip = conn.floating_ip_create(pool)['ip']
except Exception:
log.info('A new IP address was unable to be allocated. '
'An IP address will be pulled from the already allocated list, '
'This will cause a race condition when building in parallel.')
for fl_ip, opts in six.iteritems(conn.floating_ip_list()):
if opts['fixed_ip'] is None and opts['pool'] == pool:
floating_ip = fl_ip
break
if floating_ip is None:
log.error('No IP addresses available to allocate for this server: {0}'.format(vm_['name']))
def __query_node_data(vm_):
try:

5
salt/cloud/clouds/softlayer.py
View file

@ -508,7 +508,7 @@ def list_nodes_full(mask='mask[id]', call=None):
conn = get_conn(service='SoftLayer_Account')
response = conn.getVirtualGuests()
for node_id in response:
hostname = node_id['hostname'].split('.')[0]
hostname = node_id['hostname']
ret[hostname] = node_id
__utils__['cloud.cache_node_list'](ret, __active_provider_name__.split(':')[0], __opts__)
return ret
@ -594,9 +594,6 @@ def destroy(name, call=None):
transport=__opts__['transport']
)
# If the VM was created with use_fqdn, the short hostname will be used instead.
name = name.split('.')[0]
node = show_instance(name, call='action')
conn = get_conn()
response = conn.deleteObject(id=node['id'])

3
salt/cloud/clouds/softlayer_hw.py
View file

@ -526,9 +526,6 @@ def destroy(name, call=None):
transport=__opts__['transport']
)
# If the VM was created with use_fqdn, the short hostname will be used instead.
name = name.split('.')[0]
node = show_instance(name, call='action')
conn = get_conn(service='SoftLayer_Ticket')
response = conn.createCancelServerTicket(

1506
salt/cloud/deploy/bootstrap-salt.sh
View file

File diff suppressed because it is too large Load diff

17
salt/engines/slack.py
View file

@ -182,11 +182,20 @@ def start(token,
if 'aliases' in groups[group]:
aliases.update(groups[group]['aliases'])
if 'user' not in _m:
if 'message' in _m and 'user' in _m['message']:
log.debug('Message was edited, '
'so we look for user in '
'the original message.')
_user = _m['message']['user']
else:
_user = _m['user']
# Ensure the user is allowed to run commands
if valid_users:
log.debug('{0} {1}'.format(all_users, _m['user']))
if _m['user'] not in valid_users and all_users.get(_m['user'], None) not in valid_users:
channel.send_message('{0} not authorized to run Salt commands'.format(all_users[_m['user']]))
log.debug('{0} {1}'.format(all_users, _user))
if _user not in valid_users and all_users.get(_user, None) not in valid_users:
channel.send_message('{0} not authorized to run Salt commands'.format(all_users[_user]))
return
# Trim the ! from the front
@ -220,7 +229,7 @@ def start(token,
# Ensure the command is allowed
if valid_commands:
if cmd not in valid_commands:
channel.send_message('{0} is not allowed to use command {1}.'.format(all_users[_m['user']], cmd))
channel.send_message('{0} is not allowed to use command {1}.'.format(all_users[_user], cmd))
return
# Parse args and kwargs

7
salt/fileclient.py
View file

@ -621,6 +621,13 @@ class Client(object):
def on_header(hdr):
if write_body[1] is not False and write_body[2] is None:
if not hdr.strip() and 'Content-Type' not in write_body[1]:
# We've reached the end of the headers and not yet
# found the Content-Type. Reset the values we're
# tracking so that we properly follow the redirect.
write_body[0] = None
write_body[1] = False
return
# Try to find out what content type encoding is used if
# this is a text file
write_body[1].parse_line(hdr) # pylint: disable=no-member

2
salt/grains/core.py
View file

@ -792,6 +792,8 @@ def _virtual(osdata):
grains['virtual_subtype'] = 'ovirt'
elif 'Google' in output:
grains['virtual'] = 'gce'
elif 'BHYVE' in output:
grains['virtual'] = 'bhyve'
except IOError:
pass
elif osdata['kernel'] == 'FreeBSD':

4
salt/key.py
View file

@ -496,7 +496,7 @@ class Key(object):
if minion not in minions and minion not in preserve_minions:
shutil.rmtree(os.path.join(m_cache, minion))
cache = salt.cache.factory(self.opts)
clist = cache.ls(self.ACC)
clist = cache.list(self.ACC)
if clist:
for minion in clist:
if minion not in minions and minion not in preserve_minions:
@ -974,7 +974,7 @@ class RaetKey(Key):
if minion not in minions:
shutil.rmtree(os.path.join(m_cache, minion))
cache = salt.cache.factory(self.opts)
clist = cache.ls(self.ACC)
clist = cache.list(self.ACC)
if clist:
for minion in clist:
if minion not in minions and minion not in preserve_minions:

31
salt/modules/cp.py
View file

@ -58,7 +58,36 @@ def _gather_pillar(pillarenv, pillar_override):
return ret
def recv(dest, chunk, append=False, compressed=True, mode=None):
def recv(files, dest):
'''
Used with salt-cp, pass the files dict, and the destination.
This function receives small fast copy files from the master via salt-cp.
It does not work via the CLI.
'''
ret = {}
for path, data in six.iteritems(files):
if os.path.basename(path) == os.path.basename(dest) \
and not os.path.isdir(dest):
final = dest
elif os.path.isdir(dest):
final = os.path.join(dest, os.path.basename(path))
elif os.path.isdir(os.path.dirname(dest)):
final = dest
else:
return 'Destination unavailable'
try:
with salt.utils.fopen(final, 'w+') as fp_:
fp_.write(data)
ret[final] = True
except IOError:
ret[final] = False
return ret
def recv_chunked(dest, chunk, append=False, compressed=True, mode=None):
'''
This function receives files copied to the minion using ``salt-cp`` and is
not intended to be used directly on the CLI.

20
salt/modules/dockermod.py
View file

@ -798,7 +798,7 @@ def get_client_args():
salt myminion docker.get_client_args
'''
return salt.utils.docker.get_client_args()
return __utils__['docker.get_client_args']()
def _get_create_kwargs(image,
@ -902,6 +902,11 @@ def compare_container(first, second, ignore=None):
if item in ('OomKillDisable',):
if bool(val1) != bool(val2):
ret.setdefault(conf_dict, {})[item] = {'old': val1, 'new': val2}
elif item == 'Image':
image1 = inspect_image(val1)['Id']
image2 = inspect_image(val2)['Id']
if image1 != image2:
ret.setdefault(conf_dict, {})[item] = {'old': image1, 'new': image2}
else:
if item == 'Links':
val1 = _scrub_links(val1, first)
@ -920,6 +925,11 @@ def compare_container(first, second, ignore=None):
if item in ('OomKillDisable',):
if bool(val1) != bool(val2):
ret.setdefault(conf_dict, {})[item] = {'old': val1, 'new': val2}
elif item == 'Image':
image1 = inspect_image(val1)['Id']
image2 = inspect_image(val2)['Id']
if image1 != image2:
ret.setdefault(conf_dict, {})[item] = {'old': image1, 'new': image2}
else:
if item == 'Links':
val1 = _scrub_links(val1, first)
@ -3843,7 +3853,6 @@ def save(name,
if os.path.exists(path) and not overwrite:
raise CommandExecutionError('{0} already exists'.format(path))
compression = kwargs.get('compression')
if compression is None:
if path.endswith('.tar.gz') or path.endswith('.tgz'):
compression = 'gzip'
@ -3881,8 +3890,9 @@ def save(name,
saved_path = salt.utils.files.mkstemp()
else:
saved_path = path
cmd = ['docker', 'save', '-o', saved_path, inspect_image(name)['Id']]
# use the image name if its valid if not use the image id
image_to_save = name if name in inspect_image(name)['RepoTags'] else inspect_image(name)['Id']
cmd = ['docker', 'save', '-o', saved_path, image_to_save]
time_started = time.time()
result = __salt__['cmd.run_all'](cmd, python_shell=False)
if result['retcode'] != 0:
@ -3949,7 +3959,7 @@ def save(name,
ret['Size_Human'] = _size_fmt(ret['Size'])
# Process push
if kwargs.get(push, False):
if kwargs.get('push', False):
ret['Push'] = __salt__['cp.push'](path)
return ret

5
salt/modules/iptables.py
View file

@ -493,8 +493,11 @@ def build_rule(table='filter', chain=None, command=None, position='', full=None,
after_jump.append('--{0} {1}'.format(after_jump_argument, value))
del kwargs[after_jump_argument]
for key, value in kwargs.items():
for key in kwargs:
negation = maybe_add_negation(key)
# don't use .items() since maybe_add_negation removes the prefix from
# the value in the kwargs, thus we need to fetch it after that has run
value = kwargs[key]
flag = '-' if len(key) == 1 else '--'
value = '' if value in (None, '') else ' {0}'.format(value)
rule.append('{0}{1}{2}{3}'.format(negation, flag, key, value))

81
salt/modules/jenkins.py
View file

@ -37,7 +37,7 @@ import salt.utils
# Import 3rd-party libs
# pylint: disable=import-error,no-name-in-module,redefined-builtin
from salt.exceptions import SaltInvocationError
from salt.exceptions import CommandExecutionError, SaltInvocationError
# pylint: enable=import-error,no-name-in-module
log = logging.getLogger(__name__)
@ -89,9 +89,22 @@ def _connect():
password=jenkins_password)
def _retrieve_config_xml(config_xml, saltenv):
'''
Helper to cache the config XML and raise a CommandExecutionError if we fail
to do so. If we successfully cache the file, return the cached path.
'''
ret = __salt__['cp.cache_file'](config_xml, saltenv)
if not ret:
raise CommandExecutionError('Failed to retrieve {0}'.format(config_xml))
return ret
def run(script):
'''
.. versionadded:: Carbon
.. versionadded:: 2017.7.0
Execute a groovy script on the jenkins master
@ -166,7 +179,7 @@ def job_exists(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if server.job_exists(name):
@ -190,12 +203,12 @@ def get_job_info(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exist.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
job_info = server.get_job_info(name)
if job_info:
@ -219,17 +232,19 @@ def build_job(name=None, parameters=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exist.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist.'.format(name))
try:
server.build_job(name, parameters)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error building job \'{0}\': {1}'.format(name, err)
)
return True
@ -254,15 +269,15 @@ def create_job(name=None,
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
if job_exists(name):
raise SaltInvocationError('Job `{0}` already exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' already exists'.format(name))
if not config_xml:
config_xml = jenkins.EMPTY_CONFIG_XML
else:
config_xml_file = __salt__['cp.cache_file'](config_xml, saltenv)
config_xml_file = _retrieve_config_xml(config_xml, saltenv)
with salt.utils.fopen(config_xml_file) as _fp:
config_xml = _fp.read()
@ -271,7 +286,9 @@ def create_job(name=None,
try:
server.create_job(name, config_xml)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error creating job \'{0}\': {1}'.format(name, err)
)
return config_xml
@ -296,12 +313,12 @@ def update_job(name=None,
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
if not config_xml:
config_xml = jenkins.EMPTY_CONFIG_XML
else:
config_xml_file = __salt__['cp.cache_file'](config_xml, saltenv)
config_xml_file = _retrieve_config_xml(config_xml, saltenv)
with salt.utils.fopen(config_xml_file) as _fp:
config_xml = _fp.read()
@ -310,7 +327,9 @@ def update_job(name=None,
try:
server.reconfig_job(name, config_xml)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error updating job \'{0}\': {1}'.format(name, err)
)
return config_xml
@ -329,17 +348,19 @@ def delete_job(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
try:
server.delete_job(name)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error deleting job \'{0}\': {1}'.format(name, err)
)
return True
@ -358,17 +379,19 @@ def enable_job(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
try:
server.enable_job(name)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error enabling job \'{0}\': {1}'.format(name, err)
)
return True
@ -388,17 +411,19 @@ def disable_job(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
try:
server.disable_job(name)
except jenkins.JenkinsException as err:
raise SaltInvocationError('Something went wrong {0}.'.format(err))
raise CommandExecutionError(
'Encountered error disabling job \'{0}\': {1}'.format(name, err)
)
return True
@ -418,12 +443,12 @@ def job_status(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
return server.get_job_info('empty')['buildable']
@ -444,12 +469,12 @@ def get_job_config(name=None):
'''
if not name:
raise SaltInvocationError('Required parameter `name` is missing.')
raise SaltInvocationError('Required parameter \'name\' is missing')
server = _connect()
if not job_exists(name):
raise SaltInvocationError('Job `{0}` does not exists.'.format(name))
raise CommandExecutionError('Job \'{0}\' does not exist'.format(name))
job_info = server.get_job_config(name)
return job_info

164
salt/modules/kubernetes.py
View file

@ -9,9 +9,23 @@ Module for handling kubernetes calls.
kubernetes.user: admin
kubernetes.password: verybadpass
kubernetes.api_url: 'http://127.0.0.1:8080'
kubernetes.certificate-authority-data: '...'
kubernetes.client-certificate-data: '....n
kubernetes.client-key-data: '...'
kubernetes.certificate-authority-file: '/path/to/ca.crt'
kubernetes.client-certificate-file: '/path/to/client.crt'
kubernetes.client-key-file: '/path/to/client.key'
These settings can be also overrided by adding `api_url`, `api_user`,
or `api_password` parameters when calling a function:
`api_password`, `api_certificate_authority_file`, `api_client_certificate_file`
or `api_client_key_file` parameters when calling a function:
The data format for `kubernetes.*-data` values is the same as provided in `kubeconfig`.
It's base64 encoded certificates/keys in one line.
For an item only one field should be provided. Either a `data` or a `file` entry.
In case both are provided the `file` entry is prefered.
.. code-block:: bash
salt '*' kubernetes.nodes api_url=http://k8s-api-server:port api_user=myuser api_password=pass
@ -21,9 +35,11 @@ or `api_password` parameters when calling a function:
# Import Python Futures
from __future__ import absolute_import
import os.path
import base64
import logging
import yaml
import tempfile
from salt.exceptions import CommandExecutionError
from salt.ext.six import iteritems
@ -35,20 +51,18 @@ try:
import kubernetes.client
from kubernetes.client.rest import ApiException
from urllib3.exceptions import HTTPError
try:
# There is an API change in Kubernetes >= 2.0.0.
from kubernetes.client import V1beta1Deployment as AppsV1beta1Deployment
from kubernetes.client import V1beta1DeploymentSpec as AppsV1beta1DeploymentSpec
except ImportError:
from kubernetes.client import AppsV1beta1Deployment
from kubernetes.client import AppsV1beta1DeploymentSpec
HAS_LIBS = True
except ImportError:
HAS_LIBS = False
try:
# There is an API change in Kubernetes >= 2.0.0.
from kubernetes.client import V1beta1Deployment as AppsV1beta1Deployment
from kubernetes.client import V1beta1DeploymentSpec as AppsV1beta1DeploymentSpec
except ImportError:
from kubernetes.client import AppsV1beta1Deployment
from kubernetes.client import AppsV1beta1DeploymentSpec
log = logging.getLogger(__name__)
__virtualname__ = 'kubernetes'
@ -73,16 +87,31 @@ def _setup_conn(**kwargs):
'http://localhost:8080')
username = __salt__['config.option']('kubernetes.user')
password = __salt__['config.option']('kubernetes.password')
ca_cert = __salt__['config.option']('kubernetes.certificate-authority-data')
client_cert = __salt__['config.option']('kubernetes.client-certificate-data')
client_key = __salt__['config.option']('kubernetes.client-key-data')
ca_cert_file = __salt__['config.option']('kubernetes.certificate-authority-file')
client_cert_file = __salt__['config.option']('kubernetes.client-certificate-file')
client_key_file = __salt__['config.option']('kubernetes.client-key-file')
# Override default API settings when settings are provided
if kwargs.get('api_url'):
host = kwargs['api_url']
if 'api_url' in kwargs:
host = kwargs.get('api_url')
if kwargs.get('api_user'):
username = kwargs['api_user']
if 'api_user' in kwargs:
username = kwargs.get('api_user')
if kwargs.get('api_password'):
password = kwargs['api_password']
if 'api_password' in kwargs:
password = kwargs.get('api_password')
if 'api_certificate_authority_file' in kwargs:
ca_cert_file = kwargs.get('api_certificate_authority_file')
if 'api_client_certificate_file' in kwargs:
client_cert_file = kwargs.get('api_client_certificate_file')
if 'api_client_key_file' in kwargs:
client_key_file = kwargs.get('api_client_key_file')
if (
kubernetes.client.configuration.host != host or
@ -95,6 +124,45 @@ def _setup_conn(**kwargs):
kubernetes.client.configuration.user = username
kubernetes.client.configuration.passwd = password
if ca_cert_file:
kubernetes.client.configuration.ssl_ca_cert = ca_cert_file
elif ca_cert:
with tempfile.NamedTemporaryFile(prefix='salt-kube-', delete=False) as ca:
ca.write(base64.b64decode(ca_cert))
kubernetes.client.configuration.ssl_ca_cert = ca.name
else:
kubernetes.client.configuration.ssl_ca_cert = None
if client_cert_file:
kubernetes.client.configuration.cert_file = client_cert_file
elif client_cert:
with tempfile.NamedTemporaryFile(prefix='salt-kube-', delete=False) as c:
c.write(base64.b64decode(client_cert))
kubernetes.client.configuration.cert_file = c.name
else:
kubernetes.client.configuration.cert_file = None
if client_key_file:
kubernetes.client.configuration.key_file = client_key_file
if client_key:
with tempfile.NamedTemporaryFile(prefix='salt-kube-', delete=False) as k:
k.write(base64.b64decode(client_key))
kubernetes.client.configuration.key_file = k.name
else:
kubernetes.client.configuration.key_file = None
def _cleanup(**kwargs):
ca = kubernetes.client.configuration.ssl_ca_cert
cert = kubernetes.client.configuration.cert_file
key = kubernetes.client.configuration.key_file
if cert and os.path.exists(cert) and os.path.basename(cert).startswith('salt-kube-'):
salt.utils.safe_rm(cert)
if key and os.path.exists(key) and os.path.basename(key).startswith('salt-kube-'):
salt.utils.safe_rm(key)
if ca and os.path.exists(ca) and os.path.basename(ca).startswith('salt-kube-'):
salt.utils.safe_rm(ca)
def ping(**kwargs):
'''
@ -136,6 +204,8 @@ def nodes(**kwargs):
'Exception when calling CoreV1Api->list_node: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def node(name, **kwargs):
@ -158,6 +228,8 @@ def node(name, **kwargs):
'Exception when calling CoreV1Api->list_node: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
for k8s_node in api_response.items:
if k8s_node.metadata.name == name:
@ -212,6 +284,8 @@ def node_add_label(node_name, label_name, label_value, **kwargs):
'Exception when calling CoreV1Api->patch_node: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
return None
@ -245,6 +319,8 @@ def node_remove_label(node_name, label_name, **kwargs):
'Exception when calling CoreV1Api->patch_node: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
return None
@ -273,6 +349,8 @@ def namespaces(**kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def deployments(namespace='default', **kwargs):
@ -300,6 +378,8 @@ def deployments(namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def services(namespace='default', **kwargs):
@ -326,6 +406,8 @@ def services(namespace='default', **kwargs):
'CoreV1Api->list_namespaced_service: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def pods(namespace='default', **kwargs):
@ -352,6 +434,8 @@ def pods(namespace='default', **kwargs):
'CoreV1Api->list_namespaced_pod: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def secrets(namespace='default', **kwargs):
@ -378,6 +462,8 @@ def secrets(namespace='default', **kwargs):
'CoreV1Api->list_namespaced_secret: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def configmaps(namespace='default', **kwargs):
@ -404,6 +490,8 @@ def configmaps(namespace='default', **kwargs):
'CoreV1Api->list_namespaced_config_map: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_deployment(name, namespace='default', **kwargs):
@ -431,6 +519,8 @@ def show_deployment(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_service(name, namespace='default', **kwargs):
@ -457,6 +547,8 @@ def show_service(name, namespace='default', **kwargs):
'CoreV1Api->read_namespaced_service: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_pod(name, namespace='default', **kwargs):
@ -483,6 +575,8 @@ def show_pod(name, namespace='default', **kwargs):
'CoreV1Api->read_namespaced_pod: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_namespace(name, **kwargs):
@ -508,6 +602,8 @@ def show_namespace(name, **kwargs):
'CoreV1Api->read_namespace: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_secret(name, namespace='default', decode=False, **kwargs):
@ -543,6 +639,8 @@ def show_secret(name, namespace='default', decode=False, **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def show_configmap(name, namespace='default', **kwargs):
@ -572,6 +670,8 @@ def show_configmap(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_deployment(name, namespace='default', **kwargs):
@ -603,6 +703,8 @@ def delete_deployment(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_service(name, namespace='default', **kwargs):
@ -632,6 +734,8 @@ def delete_service(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_pod(name, namespace='default', **kwargs):
@ -663,6 +767,8 @@ def delete_pod(name, namespace='default', **kwargs):
'CoreV1Api->delete_namespaced_pod: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_namespace(name, **kwargs):
@ -691,6 +797,8 @@ def delete_namespace(name, **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_secret(name, namespace='default', **kwargs):
@ -722,6 +830,8 @@ def delete_secret(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def delete_configmap(name, namespace='default', **kwargs):
@ -754,6 +864,8 @@ def delete_configmap(name, namespace='default', **kwargs):
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_deployment(
@ -798,6 +910,8 @@ def create_deployment(
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_pod(
@ -842,6 +956,8 @@ def create_pod(
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_service(
@ -885,6 +1001,8 @@ def create_service(
'CoreV1Api->create_namespaced_service: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_secret(
@ -930,6 +1048,8 @@ def create_secret(
'CoreV1Api->create_namespaced_secret: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_configmap(
@ -971,6 +1091,8 @@ def create_configmap(
'CoreV1Api->create_namespaced_config_map: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def create_namespace(
@ -1005,6 +1127,8 @@ def create_namespace(
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def replace_deployment(name,
@ -1049,6 +1173,8 @@ def replace_deployment(name,
'{0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def replace_service(name,
@ -1098,6 +1224,8 @@ def replace_service(name,
'CoreV1Api->replace_namespaced_service: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def replace_secret(name,
@ -1143,6 +1271,8 @@ def replace_secret(name,
'CoreV1Api->replace_namespaced_secret: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def replace_configmap(name,
@ -1182,6 +1312,8 @@ def replace_configmap(name,
'CoreV1Api->replace_namespaced_configmap: {0}'.format(exc)
)
raise CommandExecutionError(exc)
finally:
_cleanup()
def __create_object_body(kind,

4
salt/modules/osquery.py
View file

@ -36,7 +36,7 @@ def _table_attrs(table):
'''
Helper function to find valid table attributes
'''
cmd = 'osqueryi --json "pragma table_info({0})"'.format(table)
cmd = ['osqueryi'] + ['--json'] + ['pragma table_info{0}'.format(table)]
res = __salt__['cmd.run_all'](cmd)
if res['retcode'] == 0:
attrs = []
@ -55,7 +55,7 @@ def _osquery(sql, format='json'):
'result': True,
}
cmd = 'osqueryi --json "{0}"'.format(sql)
cmd = ['osqueryi'] + ['--json'] + [sql]
res = __salt__['cmd.run_all'](cmd)
if res['retcode'] == 0:
ret['data'] = json.loads(res['stdout'])

9
salt/modules/runit.py
View file

@ -80,7 +80,8 @@ for service_dir in VALID_SERVICE_DIRS:
AVAIL_SVR_DIRS = []
# Define the module's virtual name
__virtualname__ = 'service'
__virtualname__ = 'runit'
__virtual_aliases__ = ('runit',)
def __virtual__():
@ -91,8 +92,12 @@ def __virtual__():
if __grains__.get('init') == 'runit':
if __grains__['os'] == 'Void':
add_svc_avail_path('/etc/sv')
global __virtualname__
__virtualname__ = 'service'
return __virtualname__
return False
if salt.utils.which('sv'):
return __virtualname__
return (False, 'Runit not available. Please install sv')
def _service_path(name):

4
salt/modules/selinux.py
View file

@ -403,7 +403,7 @@ def _context_string_to_dict(context):
return ret
def _filetype_id_to_string(filetype='a'):
def filetype_id_to_string(filetype='a'):
'''
Translates SELinux filetype single-letter representation
to a more human-readable version (which is also used in `semanage fcontext -l`).
@ -444,7 +444,7 @@ def fcontext_get_policy(name, filetype=None, sel_type=None, sel_user=None, sel_l
'sel_role': '[^:]+', # se_role for file context is always object_r
'sel_type': sel_type or '[^:]+',
'sel_level': sel_level or '[^:]+'}
cmd_kwargs['filetype'] = '[[:alpha:] ]+' if filetype is None else _filetype_id_to_string(filetype)
cmd_kwargs['filetype'] = '[[:alpha:] ]+' if filetype is None else filetype_id_to_string(filetype)
cmd = 'semanage fcontext -l | egrep ' + \
"'^{filespec}{spacer}{filetype}{spacer}{sel_user}:{sel_role}:{sel_type}:{sel_level}$'".format(**cmd_kwargs)
current_entry_text = __salt__['cmd.shell'](cmd)

23
salt/modules/testinframod.py
View file

@ -242,6 +242,8 @@ def _copy_function(module_name, name=None):
elif hasattr(mod, '__call__'):
mod_sig = inspect.getargspec(mod.__call__)
parameters = mod_sig.args
log.debug('Parameters accepted by module {0}: {1}'.format(module_name,
parameters))
additional_args = {}
for arg in set(parameters).intersection(set(methods)):
additional_args[arg] = methods.pop(arg)
@ -251,12 +253,15 @@ def _copy_function(module_name, name=None):
else:
modinstance = mod()
except TypeError:
modinstance = None
methods = {}
log.exception('Module failed to instantiate')
raise
valid_methods = {}
log.debug('Called methods are: {0}'.format(methods))
for meth_name in methods:
if not meth_name.startswith('_'):
methods[meth_name] = methods[meth_name]
for meth, arg in methods.items():
valid_methods[meth_name] = methods[meth_name]
log.debug('Valid methods are: {0}'.format(valid_methods))
for meth, arg in valid_methods.items():
result = _get_method_result(mod, modinstance, meth, arg)
assertion_result = _apply_assertion(arg, result)
if not assertion_result:
@ -285,13 +290,17 @@ def _register_functions():
functions, and then register them in the module namespace so that they
can be called via salt.
"""
for module_ in modules.__all__:
try:
modules_ = [_to_snake_case(module_) for module_ in modules.__all__]
except AttributeError:
modules_ = [module_ for module_ in modules.modules]
for mod_name in modules_:
mod_name = _to_snake_case(module_)
mod_func = _copy_function(mod_name, str(mod_name))
mod_func.__doc__ = _build_doc(module_)
mod_func.__doc__ = _build_doc(mod_name)
__all__.append(mod_name)
globals()[mod_name] = mod_func
if TESTINFRA_PRESENT:
_register_functions()

133
salt/modules/win_groupadd.py
View file

@ -12,6 +12,7 @@ from __future__ import absolute_import
# Import salt libs
import salt.utils
import salt.utils.win_functions
try:
@ -35,10 +36,18 @@ def __virtual__():
return (False, "Module win_groupadd: module only works on Windows systems")
def add(name, gid=None, system=False):
def add(name, **kwargs):
'''
Add the specified group
Args:
name (str):
The name of the group to add
Returns:
dict: A dictionary of results
CLI Example:
.. code-block:: bash
@ -57,29 +66,32 @@ def add(name, gid=None, system=False):
compObj = nt.GetObject('', 'WinNT://.,computer')
newGroup = compObj.Create('group', name)
newGroup.SetInfo()
ret['changes'].append((
'Successfully created group {0}'
).format(name))
ret['changes'].append('Successfully created group {0}'.format(name))
except pywintypes.com_error as com_err:
ret['result'] = False
if len(com_err.excepinfo) >= 2:
friendly_error = com_err.excepinfo[2].rstrip('\r\n')
ret['comment'] = (
'Failed to create group {0}. {1}'
).format(name, friendly_error)
ret['comment'] = 'Failed to create group {0}. {1}' \
''.format(name, friendly_error)
else:
ret['result'] = None
ret['comment'] = (
'The group {0} already exists.'
).format(name)
ret['comment'] = 'The group {0} already exists.'.format(name)
return ret
def delete(name):
def delete(name, **kwargs):
'''
Remove the named group
Args:
name (str):
The name of the group to remove
Returns:
dict: A dictionary of results
CLI Example:
.. code-block:: bash
@ -118,6 +130,14 @@ def info(name):
'''
Return information about a group
Args:
name (str):
The name of the group for which to get information
Returns:
dict: A dictionary of information about the group
CLI Example:
.. code-block:: bash
@ -151,6 +171,17 @@ def getent(refresh=False):
'''
Return info on all groups
Args:
refresh (bool):
Refresh the info for all groups in ``__context__``. If False only
the groups in ``__context__`` wil be returned. If True the
``__context__`` will be refreshed with current data and returned.
Default is False
Returns:
A list of groups and their information
CLI Example:
.. code-block:: bash
@ -182,16 +213,26 @@ def getent(refresh=False):
return ret
def adduser(name, username):
def adduser(name, username, **kwargs):
'''
add a user to a group
Add a user to a group
Args:
name (str):
The name of the group to modify
username (str):
The name of the user to add to the group
Returns:
dict: A dictionary of results
CLI Example:
.. code-block:: bash
salt '*' group.adduser foo username
'''
ret = {'name': name,
@ -209,7 +250,7 @@ def adduser(name, username):
'/', '\\').encode('ascii', 'backslashreplace').lower())
try:
if __fixlocaluser(username.lower()) not in existingMembers:
if salt.utils.win_functions.get_sam_name(username) not in existingMembers:
if not __opts__['test']:
groupObj.Add('WinNT://' + username.replace('\\', '/'))
@ -231,16 +272,26 @@ def adduser(name, username):
return ret
def deluser(name, username):
def deluser(name, username, **kwargs):
'''
remove a user from a group
Remove a user from a group
Args:
name (str):
The name of the group to modify
username (str):
The name of the user to remove from the group
Returns:
dict: A dictionary of results
CLI Example:
.. code-block:: bash
salt '*' group.deluser foo username
'''
ret = {'name': name,
@ -258,7 +309,7 @@ def deluser(name, username):
'/', '\\').encode('ascii', 'backslashreplace').lower())
try:
if __fixlocaluser(username.lower()) in existingMembers:
if salt.utils.win_functions.get_sam_name(username) in existingMembers:
if not __opts__['test']:
groupObj.Remove('WinNT://' + username.replace('\\', '/'))
@ -280,16 +331,27 @@ def deluser(name, username):
return ret
def members(name, members_list):
def members(name, members_list, **kwargs):
'''
remove a user from a group
Ensure a group contains only the members in the list
Args:
name (str):
The name of the group to modify
members_list (str):
A single user or a comma separated list of users. The group will
contain only the users specified in this list.
Returns:
dict: A dictionary of results
CLI Example:
.. code-block:: bash
salt '*' group.members foo 'user1,user2,user3'
'''
ret = {'name': name,
@ -297,7 +359,7 @@ def members(name, members_list):
'changes': {'Users Added': [], 'Users Removed': []},
'comment': []}
members_list = [__fixlocaluser(thisMember) for thisMember in members_list.lower().split(",")]
members_list = [salt.utils.win_functions.get_sam_name(m) for m in members_list.split(",")]
if not isinstance(members_list, list):
ret['result'] = False
ret['comment'].append('Members is not a list object')
@ -364,27 +426,26 @@ def members(name, members_list):
return ret
def __fixlocaluser(username):
'''
prefixes a username w/o a backslash with the computername
i.e. __fixlocaluser('Administrator') would return 'computername\administrator'
'''
if '\\' not in username:
username = ('{0}\\{1}').format(__salt__['grains.get']('host'), username)
return username.lower()
def list_groups(refresh=False):
'''
Return a list of groups
Args:
refresh (bool):
Refresh the info for all groups in ``__context__``. If False only
the groups in ``__context__`` wil be returned. If True, the
``__context__`` will be refreshed with current data and returned.
Default is False
Returns:
list: A list of groups on the machine
CLI Example:
.. code-block:: bash
salt '*' group.getent
salt '*' group.list_groups
'''
if 'group.list_groups' in __context__ and not refresh:
return __context__['group.getent']

821
salt/modules/win_lgpo.py
View file

File diff suppressed because it is too large Load diff

71
salt/modules/win_pkg.py
View file

@ -1281,21 +1281,22 @@ def install(name=None, refresh=False, pkgs=None, **kwargs):
#Compute msiexec string
use_msiexec, msiexec = _get_msiexec(pkginfo[version_num].get('msiexec', False))
# Build cmd and arguments
# cmd and arguments must be seperated for use with the task scheduler
if use_msiexec:
cmd = msiexec
arguments = ['/i', cached_pkg]
if pkginfo['version_num'].get('allusers', True):
arguments.append('ALLUSERS="1"')
arguments.extend(salt.utils.shlex_split(install_flags))
else:
cmd = cached_pkg
arguments = salt.utils.shlex_split(install_flags)
# Install the software
# Check Use Scheduler Option
if pkginfo[version_num].get('use_scheduler', False):
# Build Scheduled Task Parameters
if use_msiexec:
cmd = msiexec
arguments = ['/i', cached_pkg]
if pkginfo['version_num'].get('allusers', True):
arguments.append('ALLUSERS="1"')
arguments.extend(salt.utils.shlex_split(install_flags))
else:
cmd = cached_pkg
arguments = salt.utils.shlex_split(install_flags)
# Create Scheduled Task
__salt__['task.create_task'](name='update-salt-software',
user_name='System',
@ -1309,21 +1310,43 @@ def install(name=None, refresh=False, pkgs=None, **kwargs):
start_time='01:00',
ac_only=False,
stop_if_on_batteries=False)
# Run Scheduled Task
if not __salt__['task.run_wait'](name='update-salt-software'):
log.error('Failed to install {0}'.format(pkg_name))
log.error('Scheduled Task failed to run')
ret[pkg_name] = {'install status': 'failed'}
else:
# Build the install command
cmd = []
if use_msiexec:
cmd.extend([msiexec, '/i', cached_pkg])
if pkginfo[version_num].get('allusers', True):
cmd.append('ALLUSERS="1"')
# Special handling for installing salt
if pkg_name in ['salt-minion', 'salt-minion-py3']:
ret[pkg_name] = {'install status': 'task started'}
if not __salt__['task.run'](name='update-salt-software'):
log.error('Failed to install {0}'.format(pkg_name))
log.error('Scheduled Task failed to run')
ret[pkg_name] = {'install status': 'failed'}
else:
# Make sure the task is running, try for 5 secs
from time import time
t_end = time() + 5
while time() < t_end:
task_running = __salt__['task.status'](
'update-salt-software') == 'Running'
if task_running:
break
if not task_running:
log.error(
'Failed to install {0}'.format(pkg_name))
log.error('Scheduled Task failed to run')
ret[pkg_name] = {'install status': 'failed'}
# All other packages run with task scheduler
else:
cmd.append(cached_pkg)
cmd.extend(salt.utils.shlex_split(install_flags))
if not __salt__['task.run_wait'](name='update-salt-software'):
log.error('Failed to install {0}'.format(pkg_name))
log.error('Scheduled Task failed to run')
ret[pkg_name] = {'install status': 'failed'}
else:
# Combine cmd and arguments
cmd = [cmd].extend(arguments)
# Launch the command
result = __salt__['cmd.run_all'](cmd,
cache_path,

2
salt/modules/win_task.py
View file

@ -1259,7 +1259,7 @@ def status(name, location='\\'):
task_service = win32com.client.Dispatch("Schedule.Service")
task_service.Connect()
# get the folder to delete the folder from
# get the folder where the task is defined
task_folder = task_service.GetFolder(location)
task = task_folder.GetTask(name)

12
salt/modules/x509.py
View file

@ -330,10 +330,14 @@ def _parse_subject(subject):
for nid_name, nid_num in six.iteritems(subject.nid):
if nid_num in nids:
continue
val = getattr(subject, nid_name)
if val:
ret[nid_name] = val
nids.append(nid_num)
try:
val = getattr(subject, nid_name)
if val:
ret[nid_name] = val
nids.append(nid_num)
except TypeError as e:
if e.args and e.args[0] == 'No string argument provided':
pass
return ret

59
salt/modules/yumpkg.py
View file

@ -35,8 +35,10 @@ try:
import yum
HAS_YUM = True
except ImportError:
from salt.ext.six.moves import configparser
HAS_YUM = False
from salt.ext.six.moves import configparser
# pylint: enable=import-error,redefined-builtin
# Import salt libs
@ -2708,41 +2710,32 @@ def _parse_repo_file(filename):
'''
Turn a single repo file into a dict
'''
repos = {}
header = ''
repo = ''
with salt.utils.fopen(filename, 'r') as rfile:
for line in rfile:
if line.startswith('['):
repo = line.strip().replace('[', '').replace(']', '')
repos[repo] = {}
parsed = configparser.ConfigParser()
config = {}
# Even though these are essentially uselss, I want to allow the
# user to maintain their own comments, etc
if not line:
if not repo:
header += line
if line.startswith('#'):
if not repo:
header += line
else:
if 'comments' not in repos[repo]:
repos[repo]['comments'] = []
repos[repo]['comments'].append(line.strip())
continue
try:
parsed.read(filename)
except configparser.MissingSectionHeaderError as err:
log.error(
'Failed to parse file {0}, error: {1}'.format(filename, err.message)
)
return ('', {})
# These are the actual configuration lines that matter
if '=' in line:
try:
comps = line.strip().split('=')
repos[repo][comps[0].strip()] = '='.join(comps[1:])
except KeyError:
log.error(
'Failed to parse line in %s, offending line was '
'\'%s\'', filename, line.rstrip()
)
for section in parsed._sections:
section_dict = dict(parsed._sections[section])
section_dict.pop('__name__')
config[section] = section_dict
return (header, repos)
# Try to extract leading comments
headers = ''
with salt.utils.fopen(filename, 'r') as rawfile:
for line in rawfile:
if line.strip().startswith('#'):
headers += '{0}\n'.format(line.strip())
else:
break
return (headers, config)
def file_list(*packages):

2
salt/pillar/git_pillar.py
View file

@ -561,7 +561,7 @@ def ext_pillar(minion_id, repo, pillar_dirs):
)
for pillar_dir, env in six.iteritems(pillar.pillar_dirs):
# If pillarenv is set, only grab pillars with that match pillarenv
if opts['pillarenv'] and env != opts['pillarenv']:
if opts['pillarenv'] and env != opts['pillarenv'] and env != '__env__':
log.debug(
'env \'%s\' for pillar dir \'%s\' does not match '
'pillarenv \'%s\', skipping',

29
salt/state.py
View file

@ -2122,11 +2122,14 @@ class State(object):
reqs[r_state].append(chunk)
continue
try:
if (fnmatch.fnmatch(chunk['name'], req_val) or
fnmatch.fnmatch(chunk['__id__'], req_val)):
if req_key == 'id' or chunk['state'] == req_key:
found = True
reqs[r_state].append(chunk)
if isinstance(req_val, six.string_types):
if (fnmatch.fnmatch(chunk['name'], req_val) or
fnmatch.fnmatch(chunk['__id__'], req_val)):
if req_key == 'id' or chunk['state'] == req_key:
found = True
reqs[r_state].append(chunk)
else:
raise KeyError
except KeyError as exc:
raise SaltRenderError(
'Could not locate requisite of [{0}] present in state with name [{1}]'.format(
@ -2302,13 +2305,17 @@ class State(object):
req_val = lreq[req_key]
comment += \
'{0}{1}: {2}\n'.format(' ' * 23, req_key, req_val)
running[tag] = {'changes': {},
'result': False,
'comment': comment,
'__run_num__': self.__run_num,
'__sls__': low['__sls__']}
if low.get('__prereq__'):
run_dict = self.pre
else:
run_dict = running
run_dict[tag] = {'changes': {},
'result': False,
'comment': comment,
'__run_num__': self.__run_num,
'__sls__': low['__sls__']}
self.__run_num += 1
self.event(running[tag], len(chunks), fire_event=low.get('fire_event'))
self.event(run_dict[tag], len(chunks), fire_event=low.get('fire_event'))
return running
for chunk in reqs:
# Check to see if the chunk has been run, only run it if

6
salt/states/file.py
View file

@ -622,7 +622,11 @@ def _clean_dir(root, keep, exclude_pat):
while True:
fn_ = os.path.dirname(fn_)
real_keep.add(fn_)
if fn_ in ['/', ''.join([os.path.splitdrive(fn_)[0], '\\\\'])]:
if fn_ in [
os.sep,
''.join([os.path.splitdrive(fn_)[0], os.sep]),
''.join([os.path.splitdrive(fn_)[0], os.sep, os.sep])
]:
break
def _delete_not_kept(nfn):

4
salt/states/git.py
View file

@ -1456,8 +1456,6 @@ def latest(name,
user=user,
password=password,
ignore_retcode=True):
merge_rev = remote_rev if rev == 'HEAD' \
else desired_upstream
if git_ver >= _LooseVersion('1.8.1.6'):
# --ff-only added in version 1.8.1.6. It's not
@ -1474,7 +1472,7 @@ def latest(name,
__salt__['git.merge'](
target,
rev=merge_rev,
rev=remote_rev,
opts=merge_opts,
user=user,
password=password)

103
salt/states/group.py
View file

@ -1,10 +1,15 @@
# -*- coding: utf-8 -*-
'''
r'''
Management of user groups
=========================
The group module is used to create and manage unix group settings, groups
can be either present or absent:
The group module is used to create and manage group settings, groups can be
either present or absent. User/Group names can be passed to the ``adduser``,
``deluser``, and ``members`` parameters. ``adduser`` and ``deluser`` can be used
together but not with ``members``.
In Windows, if no domain is specified in the user or group name (ie:
`DOMAIN\username``) the module will assume a local user or group.
.. code-block:: yaml
@ -36,6 +41,10 @@ import sys
# Import 3rd-party libs
import salt.ext.six as six
# Import Salt libs
import salt.utils
import salt.utils.win_functions
def _changes(name,
gid=None,
@ -50,6 +59,18 @@ def _changes(name,
if not lgrp:
return False
# User and Domain names are not case sensitive in Windows. Let's make them
# all lower case so we can compare properly
if salt.utils.is_windows():
if lgrp['members']:
lgrp['members'] = [user.lower() for user in lgrp['members']]
if members:
members = [salt.utils.win_functions.get_sam_name(user) for user in members]
if addusers:
addusers = [salt.utils.win_functions.get_sam_name(user) for user in addusers]
if delusers:
delusers = [salt.utils.win_functions.get_sam_name(user) for user in delusers]
change = {}
if gid:
if lgrp['gid'] != gid:
@ -57,7 +78,7 @@ def _changes(name,
if members:
# -- if new member list if different than the current
if set(lgrp['members']) ^ set(members):
if set(lgrp['members']).symmetric_difference(members):
change['members'] = members
if addusers:
@ -79,31 +100,58 @@ def present(name,
addusers=None,
delusers=None,
members=None):
'''
r'''
Ensure that a group is present
name
The name of the group to manage
Args:
gid
The group id to assign to the named group; if left empty, then the next
available group id will be assigned
name (str):
The name of the group to manage
system
Whether or not the named group is a system group. This is essentially
the '-r' option of 'groupadd'.
gid (str):
The group id to assign to the named group; if left empty, then the
next available group id will be assigned. Ignored on Windows
addusers
List of additional users to be added as a group members.
system (bool):
Whether or not the named group is a system group. This is essentially
the '-r' option of 'groupadd'. Ignored on Windows
delusers
Ensure these user are removed from the group membership.
addusers (list):
List of additional users to be added as a group members. Cannot
conflict with names in delusers. Cannot be used in conjunction with
members.
members
Replace existing group members with a list of new members.
delusers (list):
Ensure these user are removed from the group membership. Cannot
conflict with names in addusers. Cannot be used in conjunction with
members.
Note: Options 'members' and 'addusers/delusers' are mutually exclusive and
can not be used together.
members (list):
Replace existing group members with a list of new members. Cannot be
used in conjunction with addusers or delusers.
Example:
.. code-block:: yaml
# Adds DOMAIN\db_admins and Administrators to the local db_admin group
# Removes Users
db_admin:
group.present:
- addusers:
- DOMAIN\db_admins
- Administrators
- delusers:
- Users
# Ensures only DOMAIN\domain_admins and the local Administrator are
# members of the local Administrators group. All other users are
# removed
Administrators:
group.present:
- members:
- DOMAIN\domain_admins
- Administrator
'''
ret = {'name': name,
'changes': {},
@ -233,8 +281,17 @@ def absent(name):
'''
Ensure that the named group is absent
name
The name of the group to remove
Args:
name (str):
The name of the group to remove
Example:
.. code-block:: yaml
# Removes the local group `db_admin`
db_admin:
group.absent
'''
ret = {'name': name,
'changes': {},

57
salt/states/jenkins.py
View file

@ -16,6 +16,7 @@ import logging
import salt.ext.six as six
from salt.ext.six.moves import zip
import salt.utils
from salt.exceptions import CommandExecutionError
# Import XML parser
import xml.etree.ElementTree as ET
@ -36,17 +37,23 @@ def _elements_equal(e1, e2):
return all(_elements_equal(c1, c2) for c1, c2 in zip(e1, e2))
def _fail(ret, msg):
ret['comment'] = msg
ret['result'] = False
return ret
def present(name,
config=None,
**kwargs):
'''
Ensure the job is present in the Jenkins
configured jobs
Ensure the job is present in the Jenkins configured jobs
name
The unique name for the Jenkins job
config
The Salt URL for the file to use for
configuring the job.
The Salt URL for the file to use for configuring the job
'''
ret = {'name': name,
@ -54,9 +61,7 @@ def present(name,
'changes': {},
'comment': ['Job {0} is up to date.'.format(name)]}
_job_exists = __salt__['jenkins.job_exists'](name)
if _job_exists:
if __salt__['jenkins.job_exists'](name):
_current_job_config = __salt__['jenkins.get_job_config'](name)
buf = six.moves.StringIO(_current_job_config)
oldXML = ET.fromstring(buf.read())
@ -68,21 +73,28 @@ def present(name,
diff = difflib.unified_diff(
ET.tostringlist(oldXML, encoding='utf8', method='xml'),
ET.tostringlist(newXML, encoding='utf8', method='xml'), lineterm='')
__salt__['jenkins.update_job'](name, config, __env__)
ret['changes'][name] = ''.join(diff)
ret['comment'].append('Job {0} updated.'.format(name))
try:
__salt__['jenkins.update_job'](name, config, __env__)
except CommandExecutionError as exc:
return _fail(ret, exc.strerror)
else:
ret['changes'] = ''.join(diff)
ret['comment'].append('Job \'{0}\' updated.'.format(name))
else:
cached_source_path = __salt__['cp.cache_file'](config, __env__)
with salt.utils.fopen(cached_source_path) as _fp:
new_config_xml = _fp.read()
__salt__['jenkins.create_job'](name, config, __env__)
try:
__salt__['jenkins.create_job'](name, config, __env__)
except CommandExecutionError as exc:
return _fail(ret, exc.strerror)
buf = six.moves.StringIO(new_config_xml)
diff = difflib.unified_diff('', buf.readlines(), lineterm='')
ret['changes'][name] = ''.join(diff)
ret['comment'].append('Job {0} added.'.format(name))
ret['comment'].append('Job \'{0}\' added.'.format(name))
ret['comment'] = '\n'.join(ret['comment'])
return ret
@ -91,24 +103,23 @@ def present(name,
def absent(name,
**kwargs):
'''
Ensure the job is present in the Jenkins
configured jobs
Ensure the job is absent from the Jenkins configured jobs
name
The name of the Jenkins job to remove.
The name of the Jenkins job to remove
'''
ret = {'name': name,
'result': True,
'changes': {},
'comment': []}
_job_exists = __salt__['jenkins.job_exists'](name)
if _job_exists:
__salt__['jenkins.delete_job'](name)
ret['comment'] = 'Job {0} deleted.'.format(name)
if __salt__['jenkins.job_exists'](name):
try:
__salt__['jenkins.delete_job'](name)
except CommandExecutionError as exc:
return _fail(ret, exc.strerror)
else:
ret['comment'] = 'Job \'{0}\' deleted.'.format(name)
else:
ret['comment'] = 'Job {0} already absent.'.format(name)
ret['comment'] = 'Job \'{0}\' already absent.'.format(name)
return ret

16
salt/states/netyang.py
View file

@ -92,6 +92,13 @@ def managed(name,
Use certain profiles to generate the config.
If not specified, will use the platform default profile(s).
compliance_report: ``False``
Return the compliance report in the comment.
The compliance report structured object can be found however
in the ``pchanges`` field of the output (not displayed on the CLI).
.. versionadded:: 2017.7.3
test: ``False``
Dry run? If set as ``True``, will apply the config, discard
and return the changes. Default: ``False`` and will commit
@ -140,6 +147,7 @@ def managed(name,
debug = kwargs.get('debug', False) or __opts__.get('debug', False)
commit = kwargs.get('commit', True) or __opts__.get('commit', True)
replace = kwargs.get('replace', False) or __opts__.get('replace', False)
return_compliance_report = kwargs.get('compliance_report', False) or __opts__.get('compliance_report', False)
profiles = kwargs.get('profiles', [])
temp_file = __salt__['temp.file']()
log.debug('Creating temp file: {0}'.format(temp_file))
@ -180,7 +188,13 @@ def managed(name,
log.debug('Loaded config result:')
log.debug(loaded_changes)
__salt__['file.remove'](temp_file)
return salt.utils.napalm.loaded_ret(ret, loaded_changes, test, debug)
loaded_changes['compliance_report'] = compliance_report
return salt.utils.napalm.loaded_ret(ret,
loaded_changes,
test,
debug,
opts=__opts__,
compliance_report=return_compliance_report)
def configured(name,

2
salt/states/service.py
View file

@ -893,7 +893,7 @@ def mod_watch(name,
try:
result = func(name, **func_kwargs)
except CommandExecutionError as exc:
ret['result'] = True
ret['result'] = False
ret['comment'] = exc.strerror
return ret

8
salt/states/testinframod.py
View file

@ -52,8 +52,12 @@ def _to_snake_case(pascal_case):
def _generate_functions():
for module in modules.__all__:
module_name = _to_snake_case(module)
try:
modules_ = [_to_snake_case(module_) for module_ in modules.__all__]
except AttributeError:
modules_ = [module_ for module_ in modules.modules]
for module_name in modules_:
func_name = 'testinfra.{0}'.format(module_name)
__all__.append(module_name)
log.debug('Generating state for module %s as function %s',

2
salt/thorium/__init__.py
View file

@ -69,7 +69,7 @@ class ThorState(salt.state.HighState):
cache = {'grains': {}, 'pillar': {}}
if self.grains or self.pillar:
if self.opts.get('minion_data_cache'):
minions = self.cache.ls('minions')
minions = self.cache.list('minions')
if not minions:
return cache
for minion in minions:

38
salt/utils/files.py
View file

@ -7,9 +7,11 @@ import contextlib
import errno
import logging
import os
import re
import shutil
import subprocess
import time
import urllib
# Import salt libs
import salt.utils
@ -258,3 +260,39 @@ def set_umask(mask):
yield
finally:
os.umask(orig_mask)
def safe_filename_leaf(file_basename):
'''
Input the basename of a file, without the directory tree, and returns a safe name to use
i.e. only the required characters are converted by urllib.quote
If the input is a PY2 String, output a PY2 String. If input is Unicode output Unicode.
For consistency all platforms are treated the same. Hard coded to utf8 as its ascii compatible
windows is \\ / : * ? " < > | posix is /
.. versionadded:: 2017.7.2
'''
def _replace(re_obj):
return urllib.quote(re_obj.group(0), safe=u'')
if not isinstance(file_basename, six.text_type):
# the following string is not prefixed with u
return re.sub('[\\\\:/*?"<>|]',
_replace,
six.text_type(file_basename, 'utf8').encode('ascii', 'backslashreplace'))
# the following string is prefixed with u
return re.sub(u'[\\\\:/*?"<>|]', _replace, file_basename, flags=re.UNICODE)
def safe_filepath(file_path_name):
'''
Input the full path and filename, splits on directory separator and calls safe_filename_leaf for
each part of the path.
.. versionadded:: 2017.7.2
'''
(drive, path) = os.path.splitdrive(file_path_name)
path = os.sep.join([safe_filename_leaf(file_section) for file_section in file_path_name.rsplit(os.sep)])
if drive:
return os.sep.join([drive, path])
else:
return path

42
salt/utils/gitfs.py
View file

@ -976,17 +976,17 @@ class GitPython(GitProvider):
else:
new = True
try:
ssl_verify = self.repo.git.config('--get', 'http.sslVerify')
except git.exc.GitCommandError:
ssl_verify = ''
desired_ssl_verify = str(self.ssl_verify).lower()
if ssl_verify != desired_ssl_verify:
self.repo.git.config('http.sslVerify', desired_ssl_verify)
try:
ssl_verify = self.repo.git.config('--get', 'http.sslVerify')
except git.exc.GitCommandError:
ssl_verify = ''
desired_ssl_verify = str(self.ssl_verify).lower()
if ssl_verify != desired_ssl_verify:
self.repo.git.config('http.sslVerify', desired_ssl_verify)
# Ensure that refspecs for the "origin" remote are set up as configured
if hasattr(self, 'refspecs'):
self.configure_refspecs()
# Ensure that refspecs for the "origin" remote are set up as configured
if hasattr(self, 'refspecs'):
self.configure_refspecs()
return new
@ -1454,18 +1454,18 @@ class Pygit2(GitProvider):
else:
new = True
try:
ssl_verify = self.repo.config.get_bool('http.sslVerify')
except KeyError:
ssl_verify = None
if ssl_verify != self.ssl_verify:
self.repo.config.set_multivar('http.sslVerify',
'',
str(self.ssl_verify).lower())
try:
ssl_verify = self.repo.config.get_bool('http.sslVerify')
except KeyError:
ssl_verify = None
if ssl_verify != self.ssl_verify:
self.repo.config.set_multivar('http.sslVerify',
'',
str(self.ssl_verify).lower())
# Ensure that refspecs for the "origin" remote are set up as configured
if hasattr(self, 'refspecs'):
self.configure_refspecs()
# Ensure that refspecs for the "origin" remote are set up as configured
if hasattr(self, 'refspecs'):
self.configure_refspecs()
return new

6
salt/utils/master.py
View file

@ -122,7 +122,7 @@ class MasterPillarUtil(object):
'and enfore_mine_cache are both disabled.')
return mine_data
if not minion_ids:
minion_ids = self.cache.ls('minions')
minion_ids = self.cache.list('minions')
for minion_id in minion_ids:
if not salt.utils.verify.valid_id(self.opts, minion_id):
continue
@ -141,7 +141,7 @@ class MasterPillarUtil(object):
'enabled.')
return grains, pillars
if not minion_ids:
minion_ids = self.cache.ls('minions')
minion_ids = self.cache.list('minions')
for minion_id in minion_ids:
if not salt.utils.verify.valid_id(self.opts, minion_id):
continue
@ -364,7 +364,7 @@ class MasterPillarUtil(object):
# in the same file, 'data.p'
grains, pillars = self._get_cached_minion_data(*minion_ids)
try:
c_minions = self.cache.ls('minions')
c_minions = self.cache.list('minions')
for minion_id in minion_ids:
if not salt.utils.verify.valid_id(self.opts, minion_id):
continue

10
salt/utils/minions.py
View file

@ -75,7 +75,7 @@ def get_minion_data(minion, opts):
if opts.get('minion_data_cache', False):
cache = salt.cache.factory(opts)
if minion is None:
for id_ in cache.ls('minions'):
for id_ in cache.list('minions'):
data = cache.fetch('minions/{0}'.format(id_), 'data')
if data is None:
continue
@ -342,13 +342,13 @@ class CkMinions(object):
if greedy:
minions = self._pki_minions()
elif cache_enabled:
minions = self.cache.ls('minions')
minions = self.cache.list('minions')
else:
return []
if cache_enabled:
if greedy:
cminions = self.cache.ls('minions')
cminions = self.cache.list('minions')
else:
cminions = minions
if cminions is None:
@ -412,7 +412,7 @@ class CkMinions(object):
mlist.append(fn_)
return mlist
elif cache_enabled:
return self.cache.ls('minions')
return self.cache.list('minions')
else:
return list()
@ -574,7 +574,7 @@ class CkMinions(object):
'''
minions = set()
if self.opts.get('minion_data_cache', False):
search = self.cache.ls('minions')
search = self.cache.list('minions')
if search is None:
return minions
addrs = salt.utils.network.local_port_tcp(int(self.opts['publish_port']))

61
salt/utils/napalm.py
View file

@ -24,6 +24,7 @@ from functools import wraps
log = logging.getLogger(__file__)
import salt.utils
import salt.output
# Import third party lib
try:
@ -432,58 +433,58 @@ def default_ret(name):
return ret
def loaded_ret(ret, loaded, test, debug):
def loaded_ret(ret, loaded, test, debug, compliance_report=False, opts=None):
'''
Return the final state output.
ret
The initial state output structure.
loaded
The loaded dictionary.
'''
# Always get the comment
ret.update({
'comment': loaded.get('comment', '')
})
changes = {}
pchanges = {}
ret['comment'] = loaded['comment']
if 'diff' in loaded:
changes['diff'] = loaded['diff']
pchanges['diff'] = loaded['diff']
if 'compliance_report' in loaded:
if compliance_report:
changes['compliance_report'] = loaded['compliance_report']
pchanges['compliance_report'] = loaded['compliance_report']
if debug and 'loaded_config' in loaded:
changes['loaded_config'] = loaded['loaded_config']
pchanges['loaded_config'] = loaded['loaded_config']
ret['pchanges'] = pchanges
if changes.get('diff'):
ret['comment'] = '{comment_base}\n\nConfiguration diff:\n\n{diff}'.format(comment_base=ret['comment'],
diff=changes['diff'])
if changes.get('loaded_config'):
ret['comment'] = '{comment_base}\n\nLoaded config:\n\n{loaded_cfg}'.format(
comment_base=ret['comment'],
loaded_cfg=changes['loaded_config'])
if changes.get('compliance_report'):
ret['comment'] = '{comment_base}\n\nCompliance report:\n\n{compliance}'.format(
comment_base=ret['comment'],
compliance=salt.output.string_format(changes['compliance_report'], 'nested', opts=opts))
if not loaded.get('result', False):
# Failure of some sort
return ret
if debug:
# Always check for debug
pchanges.update({
'loaded_config': loaded.get('loaded_config', '')
})
ret.update({
"pchanges": pchanges
})
if not loaded.get('already_configured', True):
# We're making changes
pchanges.update({
"diff": loaded.get('diff', '')
})
ret.update({
'pchanges': pchanges
})
if test:
for k, v in pchanges.items():
ret.update({
"comment": "{}:\n{}\n\n{}".format(k, v, ret.get("comment", ''))
})
ret.update({
'result': None,
})
ret['result'] = None
return ret
# Not test, changes were applied
ret.update({
'result': True,
'changes': pchanges,
'comment': "Configuration changed!\n{}".format(ret.get('comment', ''))
'changes': changes,
'comment': "Configuration changed!\n{}".format(loaded['comment'])
})
return ret
# No changes
ret.update({
'result': True
'result': True,
'changes': {}
})
return ret

11
salt/utils/parsers.py
View file

@ -2156,10 +2156,18 @@ class SaltCPOptionParser(six.with_metaclass(OptionParserMeta,
def _mixin_setup(self):
file_opts_group = optparse.OptionGroup(self, 'File Options')
file_opts_group.add_option(
'-C', '--chunked',
default=False,
dest='chunked',
action='store_true',
help='Use chunked files transfer. Supports big files, recursive '
'lookup and directories creation.'
)
file_opts_group.add_option(
'-n', '--no-compression',
default=True,
dest='compression',
dest='gzip',
action='store_false',
help='Disable gzip compression.'
)
@ -2180,7 +2188,6 @@ class SaltCPOptionParser(six.with_metaclass(OptionParserMeta,
self.config['tgt'] = self.args[0]
self.config['src'] = [os.path.realpath(x) for x in self.args[1:-1]]
self.config['dest'] = self.args[-1]
self.config['gzip'] = True
def setup_config(self):
return config.master_config(self.get_config_file_path())

16
salt/utils/process.py
View file

@ -15,6 +15,7 @@ import contextlib
import subprocess
import multiprocessing
import multiprocessing.util
import socket
# Import salt libs
@ -55,7 +56,20 @@ def notify_systemd():
import systemd.daemon
except ImportError:
if salt.utils.which('systemd-notify') and systemd_notify_call('--booted'):
return systemd_notify_call('--ready')
# Notify systemd synchronously
notify_socket = os.getenv('NOTIFY_SOCKET')
if notify_socket:
# Handle abstract namespace socket
if notify_socket.startswith('@'):
notify_socket = '\0{0}'.format(notify_socket[1:])
try:
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.connect(notify_socket)
sock.sendall('READY=1'.encode())
sock.close()
except socket.error:
return systemd_notify_call('--ready')
return True
return False
if systemd.daemon.booted():

3
salt/utils/reactor.py
View file

@ -283,6 +283,9 @@ class ReactWrap(object):
# Update called function's low data with event user to
# segregate events fired by reactor and avoid reaction loops
kwargs['__user__'] = self.event_user
# Replace ``state`` kwarg which comes from high data compiler.
# It breaks some runner functions and seems unnecessary.
kwargs['__state__'] = kwargs.pop('state')
l_fun(*f_call.get('args', ()), **kwargs)
except Exception:

12
salt/utils/schedule.py
View file

@ -846,6 +846,12 @@ class Schedule(object):
ret['return'] = self.functions[func](*args, **kwargs)
# runners do not provide retcode
if 'retcode' in self.functions.pack['__context__']:
ret['retcode'] = self.functions.pack['__context__']['retcode']
ret['success'] = True
data_returner = data.get('returner', None)
if data_returner or self.schedule_returner:
if 'return_config' in data:
@ -862,7 +868,6 @@ class Schedule(object):
for returner in OrderedDict.fromkeys(rets):
ret_str = '{0}.returner'.format(returner)
if ret_str in self.returners:
ret['success'] = True
self.returners[ret_str](ret)
else:
log.info(
@ -871,11 +876,6 @@ class Schedule(object):
)
)
# runners do not provide retcode
if 'retcode' in self.functions.pack['__context__']:
ret['retcode'] = self.functions.pack['__context__']['retcode']
ret['success'] = True
except Exception:
log.exception("Unhandled exception running {0}".format(ret['fun']))
# Although catch-all exception handlers are bad, the exception here

16
salt/utils/url.py
View file

@ -60,15 +60,15 @@ def is_escaped(url):
'''
test whether `url` is escaped with `|`
'''
if salt.utils.is_windows():
return False
scheme = urlparse(url).scheme
if not scheme:
return url.startswith('|')
elif scheme == 'salt':
path, saltenv = parse(url)
return path.startswith('|')
if salt.utils.is_windows() and '|' in url:
return path.startswith('_')
else:
return path.startswith('|')
else:
return False
@ -100,15 +100,15 @@ def unescape(url):
'''
remove escape character `|` from `url`
'''
if salt.utils.is_windows():
return url
scheme = urlparse(url).scheme
if not scheme:
return url.lstrip('|')
elif scheme == 'salt':
path, saltenv = parse(url)
return create(path.lstrip('|'), saltenv)
if salt.utils.is_windows() and '|' in url:
return create(path.lstrip('_'), saltenv)
else:
return create(path.lstrip('|'), saltenv)
else:
return url

4
salt/utils/versions.py
View file

@ -11,7 +11,7 @@
because on python 3 you can no longer compare strings against integers.
'''
# Import pytohn libs
# Import python libs
from __future__ import absolute_import
# pylint: disable=blacklisted-module
from distutils.version import StrictVersion as _StrictVersion
@ -19,7 +19,7 @@ from distutils.version import LooseVersion as _LooseVersion
# pylint: enable=blacklisted-module
# Import 3rd-party libs
import salt.ext.six as six
from salt.ext import six
class StrictVersion(_StrictVersion):

19
salt/utils/win_functions.py
View file

@ -4,6 +4,9 @@ Various functions to be used by windows during start up and to monkey patch
missing functions in other modules
'''
from __future__ import absolute_import
import platform
# Import Salt Libs
from salt.exceptions import CommandExecutionError
# Import 3rd Party Libs
@ -138,3 +141,19 @@ def get_current_user():
return False
return user_name
def get_sam_name(username):
'''
Gets the SAM name for a user. It basically prefixes a username without a
backslash with the computer name. If the username contains a backslash, it
is returned as is.
Everything is returned lower case
i.e. salt.utils.fix_local_user('Administrator') would return 'computername\administrator'
'''
if '\\' not in username:
username = '{0}\\{1}'.format(platform.node(), username)
return username.lower()

1
tests/integration/files/file/base/_modules/runtests_helpers.py
View file

@ -51,6 +51,7 @@ def get_invalid_docs():
allow_failure = (
'cmd.win_runas',
'cp.recv',
'cp.recv_chunked',
'glance.warn_until',
'ipset.long_range',
'libcloud_dns.get_driver',

13
tests/integration/modules/test_gem.py
View file

@ -19,8 +19,9 @@ from tornado.httpclient import HTTPClient
GEM = 'tidy'
GEM_VER = '1.1.2'
OLD_GEM = 'thor'
OLD_VERSION = '0.17.0'
OLD_GEM = 'brass'
OLD_VERSION = '1.0.0'
NEW_VERSION = '1.2.1'
GEM_LIST = [GEM, OLD_GEM]
@ -129,18 +130,18 @@ class GemModuleTest(ModuleCase):
self.run_function('gem.install', [OLD_GEM], version=OLD_VERSION)
gem_list = self.run_function('gem.list', [OLD_GEM])
self.assertEqual({'thor': ['0.17.0']}, gem_list)
self.assertEqual({OLD_GEM: [OLD_VERSION]}, gem_list)
self.run_function('gem.update', [OLD_GEM])
gem_list = self.run_function('gem.list', [OLD_GEM])
self.assertEqual({'thor': ['0.19.4', '0.17.0']}, gem_list)
self.assertEqual({OLD_GEM: [NEW_VERSION, OLD_VERSION]}, gem_list)
self.run_function('gem.uninstall', [OLD_GEM])
self.assertFalse(self.run_function('gem.list', [OLD_GEM]))
def test_udpate_system(self):
def test_update_system(self):
'''
gem.udpate_system
gem.update_system
'''
ret = self.run_function('gem.update_system')
self.assertTrue(ret)

14
tests/unit/cache/test_localfs.py vendored
View file

@ -209,26 +209,26 @@ class LocalFSTest(TestCase, LoaderModuleMockMixin):
with patch('os.remove', MagicMock(side_effect=OSError)):
self.assertRaises(SaltCacheError, localfs.flush, bank='', key='key', cachedir='/var/cache/salt')
# 'ls' function tests: 3
# 'list' function tests: 3
def test_ls_no_base_dir(self):
def test_list_no_base_dir(self):
'''
Tests that the ls function returns an empty list if the bank directory
doesn't exist.
'''
with patch('os.path.isdir', MagicMock(return_value=False)):
self.assertEqual(localfs.ls(bank='', cachedir=''), [])
self.assertEqual(localfs.list_(bank='', cachedir=''), [])
def test_ls_error_raised_no_bank_directory_access(self):
def test_list_error_raised_no_bank_directory_access(self):
'''
Tests that a SaltCacheError is raised when there is a problem accessing the
cache bank directory.
'''
with patch('os.path.isdir', MagicMock(return_value=True)):
with patch('os.listdir', MagicMock(side_effect=OSError)):
self.assertRaises(SaltCacheError, localfs.ls, bank='', cachedir='')
self.assertRaises(SaltCacheError, localfs.list_, bank='', cachedir='')
def test_ls_success(self):
def test_list_success(self):
'''
Tests the return of the ls function containing bank entries.
'''
@ -240,7 +240,7 @@ class LocalFSTest(TestCase, LoaderModuleMockMixin):
# Now test the return of the ls function
with patch.dict(localfs.__opts__, {'cachedir': tmp_dir}):
self.assertEqual(localfs.ls(bank='bank', cachedir=tmp_dir), ['key'])
self.assertEqual(localfs.list_(bank='bank', cachedir=tmp_dir), ['key'])
# 'contains' function tests: 1

1
tests/unit/modules/test_cmdmod.py
View file

@ -263,6 +263,7 @@ class CMDMODTestCase(TestCase, LoaderModuleMockMixin):
with patch('salt.utils.fopen', mock_open(read_data=MOCK_SHELL_FILE)):
self.assertFalse(cmdmod._is_valid_shell('foo'))
@skipIf(salt.utils.is_windows(), 'Do not run on Windows')
def test_os_environment_remains_intact(self):
'''
Make sure the OS environment is not tainted after running a command

36
tests/unit/modules/test_dockermod.py
View file

@ -136,10 +136,11 @@ class DockerTestCase(TestCase, LoaderModuleMockMixin):
with patch.dict(docker_mod.__salt__,
{'mine.send': mine_send,
'container_resource.run': MagicMock(),
'cp.cache_file': MagicMock(return_value=False),
'docker.get_client_args': client_args_mock}):
with patch.object(docker_mod, '_get_client', client):
command('container', *args)
'cp.cache_file': MagicMock(return_value=False)}):
with patch.dict(docker_mod.__utils__,
{'docker.get_client_args': client_args_mock}):
with patch.object(docker_mod, '_get_client', client):
command('container', *args)
mine_send.assert_called_with('docker.ps', verbose=True, all=True,
host=True)
@ -696,3 +697,30 @@ class DockerTestCase(TestCase, LoaderModuleMockMixin):
result = docker_mod.images()
self.assertEqual(result,
{'sha256:abcdefg': {'RepoTags': ['image:latest']}})
def test_compare_container_image_id_resolution(self):
'''
Test comparing two containers when one's inspect output is an ID and
not formatted in image:tag notation.
'''
def _inspect_container_effect(id_):
return {
'container1': {'Config': {'Image': 'realimage:latest'},
'HostConfig': {}},
'container2': {'Config': {'Image': 'image_id'},
'HostConfig': {}},
}[id_]
def _inspect_image_effect(id_):
return {
'realimage:latest': {'Id': 'image_id'},
'image_id': {'Id': 'image_id'},
}[id_]
inspect_container_mock = MagicMock(side_effect=_inspect_container_effect)
inspect_image_mock = MagicMock(side_effect=_inspect_image_effect)
with patch.object(docker_mod, 'inspect_container', inspect_container_mock):
with patch.object(docker_mod, 'inspect_image', inspect_image_mock):
ret = docker_mod.compare_container('container1', 'container2')
self.assertEqual(ret, {})

3
tests/unit/modules/test_iptables.py
View file

@ -60,6 +60,9 @@ class IptablesTestCase(TestCase, LoaderModuleMockMixin):
self.assertEqual(iptables.build_rule(**{'if': 'not eth0'}),
'! -i eth0')
self.assertEqual(iptables.build_rule(**{'proto': 'tcp', 'syn': '!'}),
'-p tcp ! --syn')
self.assertEqual(iptables.build_rule(dports=[80, 443], proto='tcp'),
'-p tcp -m multiport --dports 80,443')

4
tests/unit/states/test_boto_vpc.py
View file

@ -11,15 +11,11 @@ from tests.support.unit import skipIf, TestCase
from tests.support.mock import NO_MOCK, NO_MOCK_REASON, patch
# Import Salt libs
import salt.config
import salt.loader
import salt.utils.boto
from salt.ext import six
from salt.utils.versions import LooseVersion
import salt.states.boto_vpc as boto_vpc
# Import test suite libs
# pylint: disable=import-error,unused-import
from tests.unit.modules.test_boto_vpc import BotoVpcTestCaseMixin

34
tests/unit/test_doc.py
View file

@ -6,6 +6,7 @@
# Import Python libs
from __future__ import absolute_import
import os
# Import Salt Testing libs
from tests.support.unit import TestCase
@ -13,6 +14,7 @@ from tests.support.unit import TestCase
# Import Salt libs
import tests.integration as integration
import salt.modules.cmdmod
import salt.utils
class DocTestCase(TestCase):
@ -32,8 +34,15 @@ class DocTestCase(TestCase):
https://github.com/saltstack/salt/issues/12788
'''
salt_dir = integration.CODE_DIR
salt_dir += '/'
cmd = 'grep -r :doc: ' + salt_dir
if salt.utils.is_windows():
# No grep in Windows, use findstr
# findstr in windows doesn't prepend 'Binary` to binary files, so
# use the '/P' switch to skip files with unprintable characters
cmd = 'findstr /C:":doc:" /S /P {0}\\*'.format(salt_dir)
else:
salt_dir += '/'
cmd = 'grep -r :doc: ' + salt_dir
grep_call = salt.modules.cmdmod.run_stdout(cmd=cmd).split('\n')
@ -43,25 +52,30 @@ class DocTestCase(TestCase):
if line.startswith('Binary'):
continue
key, val = line.split(':', 1)
if salt.utils.is_windows():
# Need the space after the colon so it doesn't split the drive
# letter
key, val = line.split(': ', 1)
else:
key, val = line.split(':', 1)
# Don't test man pages, this file,
# the page that documents to not use ":doc:", or
# the doc/conf.py file
if 'man' in key \
or key.endswith('test_doc.py') \
or key.endswith('doc/conf.py') \
or key.endswith('/conventions/documentation.rst') \
or key.endswith('doc/topics/releases/2016.11.2.rst') \
or key.endswith('doc/topics/releases/2016.11.3.rst') \
or key.endswith('doc/topics/releases/2016.3.5.rst'):
or key.endswith(os.sep.join(['doc', 'conf.py'])) \
or key.endswith(os.sep.join(['conventions', 'documentation.rst'])) \
or key.endswith(os.sep.join(['doc', 'topics', 'releases', '2016.11.2.rst'])) \
or key.endswith(os.sep.join(['doc', 'topics', 'releases', '2016.11.3.rst'])) \
or key.endswith(os.sep.join(['doc', 'topics', 'releases', '2016.3.5.rst'])):
continue
# Set up test return dict
if test_ret.get(key) is None:
test_ret[key] = [val.lstrip()]
test_ret[key] = [val.strip()]
else:
test_ret[key].append(val.lstrip())
test_ret[key].append(val.strip())
# Allow test results to show files with :doc: ref, rather than truncating
self.maxDiff = None

5
tests/unit/test_stateconf.py
View file

@ -102,8 +102,9 @@ test:
- name: echo sls_dir={{sls_dir}}
- cwd: /
''', sls='path.to.sls')
self.assertEqual(result['test']['cmd.run'][0]['name'],
'echo sls_dir=path/to')
self.assertEqual(
result['test']['cmd.run'][0]['name'],
'echo sls_dir=path{0}to'.format(os.sep))
def test_states_declared_with_shorthand_no_args(self):
result = self._render_sls('''

27
tests/unit/utils/test_find.py
View file

@ -13,7 +13,6 @@ from tests.support.unit import skipIf, TestCase
from tests.support.paths import TMP
# Import salt libs
import salt.ext.six as six
import salt.utils
import salt.utils.find
@ -121,19 +120,11 @@ class TestFind(TestCase):
min_size, max_size = salt.utils.find._parse_size('+1m')
self.assertEqual(min_size, 1048576)
# sys.maxint has been removed in Python3. Use maxsize instead.
if six.PY3:
self.assertEqual(max_size, sys.maxsize)
else:
self.assertEqual(max_size, sys.maxint)
self.assertEqual(max_size, sys.maxsize)
min_size, max_size = salt.utils.find._parse_size('+1M')
self.assertEqual(min_size, 1048576)
# sys.maxint has been removed in Python3. Use maxsize instead.
if six.PY3:
self.assertEqual(max_size, sys.maxsize)
else:
self.assertEqual(max_size, sys.maxint)
self.assertEqual(max_size, sys.maxsize)
def test_option_requires(self):
option = salt.utils.find.Option()
@ -217,7 +208,7 @@ class TestFind(TestCase):
option = salt.utils.find.TypeOption('type', 's')
self.assertEqual(option.match('', '', [stat.S_IFSOCK]), True)
@skipIf(sys.platform.startswith('win'), 'No /dev/null on Windows')
@skipIf(sys.platform.startswith('win'), 'pwd not available on Windows')
def test_owner_option_requires(self):
self.assertRaises(
ValueError, salt.utils.find.OwnerOption, 'owner', 'notexist'
@ -226,7 +217,7 @@ class TestFind(TestCase):
option = salt.utils.find.OwnerOption('owner', 'root')
self.assertEqual(option.requires(), salt.utils.find._REQUIRES_STAT)
@skipIf(sys.platform.startswith('win'), 'No /dev/null on Windows')
@skipIf(sys.platform.startswith('win'), 'pwd not available on Windows')
def test_owner_option_match(self):
option = salt.utils.find.OwnerOption('owner', 'root')
self.assertEqual(option.match('', '', [0] * 5), True)
@ -234,7 +225,7 @@ class TestFind(TestCase):
option = salt.utils.find.OwnerOption('owner', '500')
self.assertEqual(option.match('', '', [500] * 5), True)
@skipIf(sys.platform.startswith('win'), 'No /dev/null on Windows')
@skipIf(sys.platform.startswith('win'), 'grp not available on Windows')
def test_group_option_requires(self):
self.assertRaises(
ValueError, salt.utils.find.GroupOption, 'group', 'notexist'
@ -247,7 +238,7 @@ class TestFind(TestCase):
option = salt.utils.find.GroupOption('group', group_name)
self.assertEqual(option.requires(), salt.utils.find._REQUIRES_STAT)
@skipIf(sys.platform.startswith('win'), 'No /dev/null on Windows')
@skipIf(sys.platform.startswith('win'), 'grp not available on Windows')
def test_group_option_match(self):
if sys.platform.startswith(('darwin', 'freebsd', 'openbsd')):
group_name = 'wheel'
@ -412,7 +403,7 @@ class TestPrintOption(TestCase):
option.execute('test_name', [0] * 9), [0, 'test_name']
)
@skipIf(sys.platform.startswith('Windows'), "no /dev/null on windows")
@skipIf(sys.platform.startswith('win'), "pwd not available on Windows")
def test_print_user(self):
option = salt.utils.find.PrintOption('print', 'user')
self.assertEqual(option.execute('', [0] * 10), 'root')
@ -420,7 +411,7 @@ class TestPrintOption(TestCase):
option = salt.utils.find.PrintOption('print', 'user')
self.assertEqual(option.execute('', [2 ** 31] * 10), 2 ** 31)
@skipIf(sys.platform.startswith('Windows'), "no /dev/null on windows")
@skipIf(sys.platform.startswith('win'), "grp not available on Windows")
def test_print_group(self):
option = salt.utils.find.PrintOption('print', 'group')
if sys.platform.startswith(('darwin', 'freebsd', 'openbsd')):
@ -433,7 +424,7 @@ class TestPrintOption(TestCase):
#option = salt.utils.find.PrintOption('print', 'group')
#self.assertEqual(option.execute('', [2 ** 31] * 10), 2 ** 31)
@skipIf(sys.platform.startswith('Windows'), "no /dev/null on windows")
@skipIf(sys.platform.startswith('win'), "no /dev/null on windows")
def test_print_md5(self):
option = salt.utils.find.PrintOption('print', 'md5')
self.assertEqual(option.execute('/dev/null', os.stat('/dev/null')), '')

2
tests/unit/utils/test_network.py
View file

@ -118,7 +118,7 @@ class NetworkTestCase(TestCase):
(2, 1, 6, '', ('192.30.255.113', 0)),
],
'ipv6host.foo': [
(10, 1, 6, '', ('2001:a71::1', 0, 0, 0)),
(socket.AF_INET6, 1, 6, '', ('2001:a71::1', 0, 0, 0)),
],
}[host]
except KeyError:

12
tests/unit/utils/test_url.py
View file

@ -38,6 +38,8 @@ class UrlTestCase(TestCase):
'''
path = '?funny/path with {interesting|chars}'
url = 'salt://' + path
if salt.utils.is_windows():
path = '_funny/path with {interesting_chars}'
self.assertEqual(salt.utils.url.parse(url), (path, None))
@ -48,6 +50,8 @@ class UrlTestCase(TestCase):
saltenv = 'ambience'
path = '?funny/path&with {interesting|chars}'
url = 'salt://' + path + '?saltenv=' + saltenv
if salt.utils.is_windows():
path = '_funny/path&with {interesting_chars}'
self.assertEqual(salt.utils.url.parse(url), (path, saltenv))
@ -59,6 +63,8 @@ class UrlTestCase(TestCase):
'''
path = '? interesting/&path.filetype'
url = 'salt://' + path
if salt.utils.is_windows():
url = 'salt://_ interesting/&path.filetype'
self.assertEqual(salt.utils.url.create(path), url)
@ -68,6 +74,8 @@ class UrlTestCase(TestCase):
'''
saltenv = 'raumklang'
path = '? interesting/&path.filetype'
if salt.utils.is_windows():
path = '_ interesting/&path.filetype'
url = 'salt://' + path + '?saltenv=' + saltenv
@ -149,6 +157,8 @@ class UrlTestCase(TestCase):
'''
path = 'dir/file.conf'
escaped_path = '|' + path
if salt.utils.is_windows():
escaped_path = path
self.assertEqual(salt.utils.url.escape(path), escaped_path)
@ -167,6 +177,8 @@ class UrlTestCase(TestCase):
path = 'dir/file.conf'
url = 'salt://' + path
escaped_url = 'salt://|' + path
if salt.utils.is_windows():
escaped_url = url
self.assertEqual(salt.utils.url.escape(url), escaped_url)

12
tests/unit/utils/test_which.py
View file

@ -44,18 +44,21 @@ class TestWhich(TestCase):
# The second, iterating through $PATH, should also return False,
# still checking for Linux
False,
# We will now also return False once so we get a .EXE back from
# the function, see PATHEXT below.
False,
# Lastly return True, this is the windows check.
True
]
# Let's patch os.environ to provide a custom PATH variable
with patch.dict(os.environ, {'PATH': '/bin'}):
with patch.dict(os.environ, {'PATH': '/bin',
'PATHEXT': '.COM;.EXE;.BAT;.CMD'}):
# Let's also patch is_windows to return True
with patch('salt.utils.is_windows', lambda: True):
with patch('os.path.isfile', lambda x: True):
self.assertEqual(
salt.utils.which('this-binary-exists-under-windows'),
# The returned path should return the .exe suffix
'/bin/this-binary-exists-under-windows.EXE'
os.path.join('/bin', 'this-binary-exists-under-windows.EXE')
)
def test_missing_binary_in_windows(self):
@ -106,6 +109,5 @@ class TestWhich(TestCase):
with patch('os.path.isfile', lambda x: True):
self.assertEqual(
salt.utils.which('this-binary-exists-under-windows'),
# The returned path should return the .exe suffix
'/bin/this-binary-exists-under-windows.CMD'
os.path.join('/bin', 'this-binary-exists-under-windows.CMD')
)