saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

[develop] Merge forward from 2016.3 to develop (#32494)

Browse source 
* fix sorting by latest version when called with an attribute

* remove reference to master_alive_check

* Fixes saltstack/salt#28262

* Resolve memory leak in authentication

* outputter virt_list does not exist anymore

* Update proxmox documentation

* Fix documentation on boto_asg and boto_elb modules and states

* modules.win_timezone: don't list all zones in debug log

* Correcty index glusterfs bricks

Fixes issue #32311

* Cleaner deprecation process with decorators

* Add deprecation decorator scaffold

* Capture type error and unhandled exceptions while function calls

* Aware of the current and future version of deprecation

* Implement initially is_deprecated decorator

* Add an alias for the capitalization

* Fix capitalization easier way

* Remove an extra line

* Add successor name to the deprecation decorator.

* Granulate logging and error messages.

* Implement function swapper

* Raise later the caught exception

* Clarify exception message

* Save function original name

* Remove an extra line

* Hide an alternative hidden function name in the error message, preserving the error itself

* Rename variable as private

* Add a method to detect if a function is using its previous version

* Message to the log and/or raise an exception accordingly to the status of used function

* Log an error along with the exception

* Add internal method documentation

* Add documentation and usage process for decorator "is_deprecated"

* Add documentation and process usage for the decorator "with_deprecated"

* Hide private method name

* Fix PEP8, re-word the error message

* Deprecate basic uptime function

* Add initial decorator unit test

* Rename old/new functions, mock versions

* Move frequent data to the test setup

* Add logging on EOL exception

* Rename and document high to low version test on is_deprecated

* Implement a test on low to high version of is_deprecated decorator

* Add a correction to the test description

* Remove a dead code

* Implement a test for high to low version on is_deprecated, using with_successor param

* Correct typso adn mistaeks

* Implement high to low version with successor param on is_deprecated

* Setup a virtual name for the module

* Implement test for with_deprecated should raise an exception if same deprecated function not found

* Implement test for with_deprecated an old function is picked up if configured

* Correct test description purpose

* Implement test with_deprecated when no deprecation is requested

* Add logging test to the configured deprecation request

* Add logging testing when deprecated version wasn't requested

* Implement test EOL for with_deprecated decorator

* Correct test explanation

* Rename the test

* Implement with_deprecated no EOL, deprecated other function name

* Implement with_deprecated, deprecated other function name, EOL reached

* Add test description for the with_deprecated + with_name + EOL

* Fix confusing test names

* Add logging test to the is_deprecated decorator when function as not found.

* Add more test point to each test, remove empty lines

* Bugfix: at certain conditions a wrong alias name is reported to the log

* Fix a typo in a comment

* Add test for the logging

* Disable a pylint: None will _never_ be raised

* Fix test for the deprecated "status.uptime" version

* Bugfix: Do not yank raised exceptions

* Remove unnecessary decorator

* Add test for the new uptime

* Add test for the new uptime fails when /proc/uptime does not exists

* Rename old test case

* Skip test for the UTC time, unless freeze time is used.

* Fix pylint

* Fix documentation

* Bugfix: proxy-pass the docstring of the decorated function

* Lint fix

* Fixes saltstack/salt#28262 for 2015.5 branch

* Update master config docs

* Improve git_pillar documentation/logging

* Add note about different behavior of top file in git_pillar

* Make log entry for a missing pillar SLS file more accurate for git_pillar

* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.

* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.

* Update glusterfs_test to be inline with #32312

* Fix salt-cloud paralell provisioning

Closes #31632

* Ignore Raspbian in service.py __virtual__ (#32421)

* Ignore Raspbian in service.py __virtual__

This prevents more than one execution module from trying to load as the
service virtual module.

Refs: #32413

* pack __salt__ before loading provider overrides

We can (and should) pack here since we're just packing a reference to the
object. __salt__ needs to be available when we're loading our provider
overrides

* Fix broken __salt__ dict in provider override

Using ret.items() here sets ``__salt__`` to its items (tuple containing
function name and reference), breaking usage of ``__salt__`` inside
overridden functions.

* Merge #32293 with test fixes (#32418)

* Fix issue #11497

* Remove check for working directory presence in tests

* Fix Domainname introspection

Default value needs to be extracted from the container itself,
because dockerd set Domainname value when network_mode=host.

* Add pgjsonb_queue to queue doc index

* Pylint fixes

* Pass parser options into batch mode

Resolves #31738

* Changed the target file in file.symlink test (#32443)

* Argument name in docs should match actual arg name (#32445)

Fixes #31851

* tests.integration: bypass MacOS TMPDIR, gettempdir (#32447)

Updates 0edd532, 8f558a5.

When logging in as root over `ssh root@host`, `$TMPDIR` and
`tempfile.gettempdir()` are both set to a variation of:
```
/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/
```
When logging in as root over `sudo -i`, `$TMPDIR` is unset and
`tempfile.gettempdir()` is set to `/tmp`.

My guess is that the second case is an unintended or uncorrected omision
by Apple as they have introduced the longer, randomized temp path in a
recent version of MacOS.

* Issue #28706: Fix state user.present behavior. (#32448)

- As mentionned in issue #28706, state user.present no longer remove
      user from groups if the keyword 'groups' with empty value '[]' is not
      explicitly set, salt will assume current groups are still wanted.

* tests.integration: fix 4230c8a

* Move the tables of virtual modules to individual documentation pages

* Add new doc pages to toctree

* Add external ref to windows package manager docs

* Improve docstrings

* Add documentation on virtual module provider overrides to the module docs

* Clarify the scope of the provider param in states.

* Add link to provider override docs to all package providers

* Add link to provider override docs to all service providers

* Add link to provider override docs to all user providers

* dd link to provider override docs to all shadow providers

* Add link to provider override docs to all group providers

* Backport 31164 and 31364 (#32474)

* Don't send REQ while another one is waiting for response.

The message has to be removed from the queue the only *after* it's
already processed to don't confuse send() functionality that expects
empty queue means: there's no active sendings.

* Fixed zeromq ReqMessageClient destroy

* Add link to provider override docs to opkg.py

This is a companion to https://github.com/saltstack/salt/pull/32458, but
this module was not added until the 2016.3 branch, so the documentation
is being updated there for this module.

* Add documentation for some master/minion configs (#32454)

Refs #32400

Adds docs for:

- cli_summary
- event_return_queue
- event_return_whitelist
- event_return_blacklist
- file_recv_max_size
- fileserver_followsymlinks
- fileserver_ignoresymlinks
- fileserver_limit_traversal

* Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440)

* Automatically detect MySQL password column

* Fix changing password in MySQL 5.7

* Fix lint test

* Fix unit tests (?)

They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that.

* Additional unit test fix

* Only unsub if we have a jid

Closes #32479
This commit is contained in:
Nicole Thomas 2016-04-11 17:07:15 -06:00 • committed by Mike Place
parent 679600eb67
commit 216d9fdc9a
81 changed files with 1536 additions and 293 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
conf/master
View file

@ -88,6 +88,12 @@
# (true by default).
# strip_colors: False
# To display a summary of the number of minions targeted, the number of
# minions returned, and the number of minions that did not return, set the
# cli_summary value to True. (False by default.)
#
#cli_summary: False
# Set the directory used to hold unix sockets:
#sock_dir: /var/run/salt/master
@ -118,12 +124,12 @@
# By default, events are not queued.
#event_return_queue: 0
# Only events returns matching tags in a whitelist
# Only return events matching tags in a whitelist,
# event_return_whitelist:
# - salt/master/a_tag
# - salt/master/another_tag
# Store all event returns _except_ the tags in a blacklist
# Store all event returns _except_ the tags in a blacklist.
# event_return_blacklist:
# - salt/master/not_this_tag
# - salt/master/or_this_one

16
conf/minion
View file

@ -468,6 +468,18 @@
# base:
# - /srv/salt
# Uncomment the line below if you do not want the file_server to follow
# symlinks when walking the filesystem tree. This is set to True
# by default. Currently this only applies to the default roots
# fileserver_backend.
#fileserver_followsymlinks: False
#
# Uncomment the line below if you do not want symlinks to be
# treated as the files they are pointing to. By default this is set to
# False. By uncommenting the line below, any detected symlink while listing
# files on the Master will not be returned to the Minion.
#fileserver_ignoresymlinks: True
#
# By default, the Salt fileserver recurses fully into all defined environments
# to attempt to find files. To limit this behavior so that the fileserver only
# traverses directories with SLS files and special Salt directories like _modules,
@ -495,6 +507,10 @@
#pillar_roots:
# base:
# - /srv/pillar
# Set a hard-limit on the size of the files that can be pushed to the master.
# It will be interpreted as megabytes. Default: 100
#file_recv_max_size: 100
#
#
###### Security settings #####

147
doc/ref/configuration/master.rst
View file

@ -332,6 +332,21 @@ to False.
color: False
.. conf_master:: cli_summary
``cli_summary``
---------------
Default: ``False``
When set to ``True``, displays a summary of the number of minions targeted,
the number of minions returned, and the number of minions that did not
return.
.. code-block:: yaml
cli_summary: False
.. conf_master:: sock_dir
``sock_dir``
@ -424,6 +439,58 @@ configuration requirements. Read the returner's documentation.
event_return: cassandra_cql
.. conf_master:: event_return_queue
``event_return_queue``
----------------------
.. versionadded:: 2015.5.0
Default: ``0``
On busy systems, enabling event_returns can cause a considerable load on
the storage system for returners. Events can be queued on the master and
stored in a batched fashion using a single transaction for multiple events.
By default, events are not queued.
.. code-block:: yaml
event_return_queue: 0
.. conf_master:: event_return_whitelist
``event_return_whitelist``
--------------------------
.. versionadded:: 2015.5.0
Default: ``[]``
Only return events matching tags in a whitelist.
.. code-block:: yaml
event_return_whitelist:
- salt/master/a_tag
- salt/master/another_tag
.. conf_master:: event_return_blacklist
``event_return_blacklist``
--------------------------
.. versionadded:: 2015.5.0
Default: ``[]``
Store all event returns _except_ the tags in a blacklist.
.. code-block:: yaml
event_return_blacklist:
- salt/master/not_this_tag
- salt/master/or_this_one
.. conf_master:: master_job_cache
``master_job_cache``
@ -726,6 +793,22 @@ security purposes.
file_recv: False
.. conf_master:: file_recv_max_size
``file_recv_max_size``
----------------------
.. versionadded:: 2014.7.0
Default: ``100``
Set a hard-limit on the size of the files that can be pushed to the master.
It will be interpreted as megabytes.
.. code-block:: yaml
file_recv_max_size: 100
.. conf_master:: master_sign_pubkey
``master_sign_pubkey``
@ -1031,6 +1114,60 @@ Example:
- roots
- git
.. conf_master:: fileserver_followsymlinks
``fileserver_followsymlinks``
-----------------------------
.. versionadded:: 2014.1.0
Default: ``True``
By default, the file_server follows symlinks when walking the filesystem tree.
Currently this only applies to the default roots fileserver_backend.
.. code-block:: yaml
fileserver_followsymlinks: True
.. conf_master:: fileserver_ignoresymlinks
``fileserver_ignoresymlinks``
-----------------------------
.. versionadded:: 2014.1.0
Default: ``False``
If you do not want symlinks to be treated as the files they are pointing to,
set ``fileserver_ignoresymlinks`` to ``True``. By default this is set to
False. When set to ``True``, any detected symlink while listing files on the
Master will not be returned to the Minion.
.. code-block:: yaml
fileserver_ignoresymlinks: False
.. conf_master:: fileserver_limit_traversal
``fileserver_limit_traversal``
------------------------------
.. versionadded:: 2014.1.0
Default: ``False``
By default, the Salt fileserver recurses fully into all defined environments
to attempt to find files. To limit this behavior so that the fileserver only
traverses directories with SLS files and special Salt directories like _modules,
set ``fileserver_limit_traversal`` to ``True``. This might be useful for
installations where a file root has a very large number of files and performance
is impacted.
.. code-block:: yaml
fileserver_limit_traversal: False
.. conf_master:: hash_type
``hash_type``
@ -1205,7 +1342,7 @@ compatible version installed will be the provider that is used.
Default: ``True``
Specifies whether or not to ignore SSL certificate errors when contacting the
remote repository. You might want to set this to ``False`` if you're using a
remote repository. The ``False`` setting is useful if you're using a
git repo that uses a self-signed certificate. However, keep in mind that
setting this to anything other ``True`` is a considered insecure, and using an
SSH-based transport (if available) may be a better option.
@ -2169,10 +2306,10 @@ files would be looked for in a subdirectory called ``pillar``.
.. versionadded:: 2015.8.0
.. versionchanged:: Carbon
Default: ``True``
Default: ``False``
Specifies whether or not to ignore SSL certificate errors when contacting the
remote repository. You might want to set this to ``False`` if you're using a
remote repository. The ``False`` setting is useful if you're using a
git repo that uses a self-signed certificate. However, keep in mind that
setting this to anything other ``True`` is a considered insecure, and using an
SSH-based transport (if available) may be a better option.
@ -2988,10 +3125,10 @@ branch/tag.
.. versionadded:: 2015.8.0
.. versionchanged:: Carbon
Default: ``True``
Default: ``False``
Specifies whether or not to ignore SSL certificate errors when contacting the
remote repository. You might want to set this to ``False`` if you're using a
remote repository. The ``False`` setting is useful if you're using a
git repo that uses a self-signed certificate. However, keep in mind that
setting this to anything other ``True`` is a considered insecure, and using an
SSH-based transport (if available) may be a better option.

70
doc/ref/configuration/minion.rst
View file

@ -979,6 +979,60 @@ the fileserver's environments. This parameter operates identically to the
- /srv/salt/prod/services
- /srv/salt/prod/states
.. conf_minion:: fileserver_followsymlinks
``fileserver_followsymlinks``
-----------------------------
.. versionadded:: 2014.1.0
Default: ``True``
By default, the file_server follows symlinks when walking the filesystem tree.
Currently this only applies to the default roots fileserver_backend.
.. code-block:: yaml
fileserver_followsymlinks: True
.. conf_minion:: fileserver_ignoresymlinks
``fileserver_ignoresymlinks``
-----------------------------
.. versionadded:: 2014.1.0
Default: ``False``
If you do not want symlinks to be treated as the files they are pointing to,
set ``fileserver_ignoresymlinks`` to ``True``. By default this is set to
False. When set to ``True``, any detected symlink while listing files on the
Master will not be returned to the Minion.
.. code-block:: yaml
fileserver_ignoresymlinks: False
.. conf_minion:: fileserver_limit_traversal
``fileserver_limit_traversal``
------------------------------
.. versionadded:: 2014.1.0
Default: ``False``
By default, the Salt fileserver recurses fully into all defined environments
to attempt to find files. To limit this behavior so that the fileserver only
traverses directories with SLS files and special Salt directories like _modules,
set ``fileserver_limit_traversal`` to ``True``. This might be useful for
installations where a file root has a very large number of files and performance
is impacted.
.. code-block:: yaml
fileserver_limit_traversal: False
.. conf_minion:: hash_type
``hash_type``
@ -1036,6 +1090,22 @@ the environment setting, but for pillar instead of states.
pillarenv: None
.. conf_minion:: file_recv_max_size
``file_recv_max_size``
----------------------
.. versionadded:: 2014.7.0
Default: ``100``
Set a hard-limit on the size of the files that can be pushed to the master.
It will be interpreted as megabytes.
.. code-block:: yaml
file_recv_max_size: 100
Security Settings
=================

2
doc/ref/modules/all/index.rst
View file

@ -127,6 +127,7 @@ Full list of builtin execution modules
gnomedesktop
gpg
grains
group
groupadd
grub_legacy
guestfs
@ -358,6 +359,7 @@ Full list of builtin execution modules
udev
upstart
uptime
user
useradd
uwsgi
varnish

24
doc/ref/modules/all/salt.modules.group.rst Normal file
View file

@ -0,0 +1,24 @@
.. _virtual-group:
==================
salt.modules.group
==================
.. py:module:: salt.modules.group
:synopsis: A virtual module for group management
``group`` is a virtual module that is fulfilled by one of the following
modules:
====================================== ========================================
Execution Module Used for
====================================== ========================================
:py:mod:`~salt.modules.groupadd` Linux, NetBSD, and OpenBSD systems using
``groupadd(8)``, ``groupdel(8)``, and
``groupmod(8)``
:py:mod:`~salt.modules.pw_group` FreeBSD-based OSes using ``pw(8)``
:py:mod:`~salt.modules.solaris_group` Solaris-based OSes using
``groupadd(1M)``, ``groupdel(1M)``, and
``groupmod(1M)``
:py:mod:`~salt.modules.win_groupadd` Windows
====================================== ========================================

45
doc/ref/modules/all/salt.modules.pkg.rst
View file

@ -1,3 +1,5 @@
.. _virtual-pkg:
================
salt.modules.pkg
================
@ -7,16 +9,33 @@ salt.modules.pkg
``pkg`` is a virtual module that is fulfilled by one of the following modules:
* :mod:`salt.modules.aptpkg`
* :mod:`salt.modules.brew`
* :mod:`salt.modules.ebuild`
* :mod:`salt.modules.freebsdpkg`
* :mod:`salt.modules.openbsdpkg`
* :mod:`salt.modules.pacman`
* :mod:`salt.modules.pkgin`
* :mod:`salt.modules.pkgng`
* :mod:`salt.modules.pkgutil`
* :mod:`salt.modules.solarispkg`
* :mod:`salt.modules.win_pkg`
* :mod:`salt.modules.yumpkg`
* :mod:`salt.modules.zypper`
====================================== ========================================
Execution Module Used for
====================================== ========================================
:py:mod:`~salt.modules.aptpkg` Debian/Ubuntu-based distros which use
``apt-get(8)`` for package management
:py:mod:`~salt.modules.brew` Mac OS software management using
`Homebrew`_
:py:mod:`~salt.modules.ebuild` Gentoo-based systems (utilizes the
``portage`` python module as well as
``emerge(1)``)
:py:mod:`~salt.modules.freebsdpkg` FreeBSD-based OSes using ``pkg_add(1)``
:py:mod:`~salt.modules.openbsdpkg` OpenBSD-based OSes using ``pkg_add(1)``
:py:mod:`~salt.modules.pacman` Arch Linux-based distros using
``pacman(8)``
:py:mod:`~salt.modules.pkgin` NetBSD-based OSes using ``pkgin(1)``
:py:mod:`~salt.modules.pkgng` FreeBSD-based OSes using ``pkg(8)``
:py:mod:`~salt.modules.pkgutil` Solaris-based OSes using `OpenCSW`_'s
``pkgutil(1)``
:py:mod:`~salt.modules.solarispkg` Solaris-based OSes using ``pkgadd(1M)``
:py:mod:`~salt.modules.solarisips` Solaris-based OSes using IPS ``pkg(1)``
:py:mod:`~salt.modules.win_pkg` Salt's :ref:`Windows Package Manager
<windows-package-manager`
:py:mod:`~salt.modules.yumpkg` RedHat-based distros and derivatives
using ``yum(8)`` or ``dnf(8)``
:py:mod:`~salt.modules.zypper` SUSE-based distros using ``zypper(8)``
====================================== ========================================
.. _Homebrew: http://brew.sh/
.. _OpenCSW: http://www.opencsw.org/

35
doc/ref/modules/all/salt.modules.service.rst
View file

@ -1,6 +1,39 @@
.. _virtual-service:
====================
salt.modules.service
====================
.. py:module:: salt.modules.service
:synopsis: A virtual module for service management
``service`` is a virtual module that is fulfilled by one of the following
modules:
====================================== ========================================
Execution Module Used for
====================================== ========================================
:py:mod:`~salt.modules.debian_service` Debian Wheezy and earlier
:py:mod:`~salt.modules.freebsdservice` FreeBSD-based OSes using ``service(8)``
:py:mod:`~salt.modules.gentoo_service` Gentoo Linux using ``sysvinit`` and
``rc-update(8)``
:py:mod:`~salt.modules.launchctl` Mac OS hosts using ``launchctl(1)``
:py:mod:`~salt.modules.netbsdservice` NetBSD-based OSes
:py:mod:`~salt.modules.openbsdservice` OpenBSD-based OSes
:py:mod:`~salt.modules.rh_service` RedHat-based distros and derivatives
using ``service(8)`` and
``chkconfig(8)``. Supports both pure
sysvinit and mixed sysvinit/upstart
systems.
:py:mod:`~salt.modules.service` Fallback which simply wraps sysvinit
scripts
:py:mod:`~salt.modules.smf` Solaris-based OSes which use SMF
:py:mod:`~salt.modules.systemd` Linux distros which use systemd
:py:mod:`~salt.modules.upstart` Ubuntu-based distros using upstart
:py:mod:`~salt.modules.win_service` Windows
====================================== ========================================
|
.. automodule:: salt.modules.service
:members:
:members:

22
doc/ref/modules/all/salt.modules.shadow.rst
View file

@ -1,6 +1,26 @@
.. _virtual-shadow:
===================
salt.modules.shadow
===================
.. py:module:: salt.modules.shadow
:synopsis: A virtual module for shadow file / password management
``shadow`` is a virtual module that is fulfilled by one of the following
modules:
====================================== ========================================
Execution Module Used for
====================================== ========================================
:py:mod:`~salt.modules.shadow` Linux
:py:mod:`~salt.modules.bsd_shadow` FreeBSD, OpenBSD, NetBSD
:py:mod:`~salt.modules.solaris_shadow` Solaris-based OSes
:py:mod:`~salt.modules.win_shadow` Windows
====================================== ========================================
|
.. automodule:: salt.modules.shadow
:members:
:members:

24
doc/ref/modules/all/salt.modules.user.rst Normal file
View file

@ -0,0 +1,24 @@
.. _virtual-user:
=================
salt.modules.user
=================
.. py:module:: salt.modules.user
:synopsis: A virtual module for user management
``user`` is a virtual module that is fulfilled by one of the following modules:
====================================== ========================================
Execution Module Used for
====================================== ========================================
:py:mod:`~salt.modules.useradd` Linux, NetBSD, and OpenBSD systems using
``useradd(8)``, ``userdel(8)``, and
``usermod(8)``
:py:mod:`~salt.modules.pw_user` FreeBSD-based OSes using ``pw(8)``
:py:mod:`~salt.modules.solaris_user` Solaris-based OSes using
``useradd(1M)``, ``userdel(1M)``, and
``usermod(1M)``
:py:mod:`~salt.modules.mac_user` MacOS
:py:mod:`~salt.modules.win_useradd` Windows
====================================== ========================================

60
doc/ref/modules/index.rst
View file

@ -202,8 +202,8 @@ Outputter.
.. code-block:: python
__outputter__ = {
'run': 'txt'
}
'run': 'txt'
}
This will ensure that the text outputter is used.
@ -242,8 +242,8 @@ Since ``__virtual__`` is called before the module is loaded, ``__salt__`` will b
unavailable as it will not have been packed into the module at this point in time.
.. note::
Modules which return a string from ``__virtual__`` that is already used by a module that
ships with Salt will _override_ the stock module.
Modules which return a string from ``__virtual__`` that is already used by
a module that ships with Salt will _override_ the stock module.
.. _modules-error-info:
@ -276,7 +276,7 @@ the case when the dependency is unavailable.
if HAS_ENZYMES:
return 'cheese'
else:
return (False, 'The cheese execution module cannot be loaded: enzymes unavailable.')
return False, 'The cheese execution module cannot be loaded: enzymes unavailable.'
.. code-block:: python
@ -292,18 +292,54 @@ the case when the dependency is unavailable.
if 'cheese.slice' in __salt__:
return 'cheese'
else:
return (False, 'The cheese state module cannot be loaded: enzymes unavailable.')
return False, 'The cheese state module cannot be loaded: enzymes unavailable.'
Examples
--------
The package manager modules are among the best examples of using the ``__virtual__``
function. Some examples:
The package manager modules are among the best examples of using the
``__virtual__`` function. A table of all the virtual ``pkg`` modules can be
found :ref:`here <virtual-pkg>`.
- :blob:`pacman.py <salt/modules/pacman.py>`
- :blob:`yumpkg.py <salt/modules/yumpkg.py>`
- :blob:`aptpkg.py <salt/modules/aptpkg.py>`
- :blob:`at.py <salt/modules/at.py>`
.. _module-provider-override:
Overriding Virtual Module Providers
-----------------------------------
Salt often uses OS grains (``os``, ``osrelease``, ``os_family``, etc.) to
determine which module should be loaded as the virtual module for ``pkg``,
``service``, etc. Sometimes this OS detection is incomplete, with new distros
popping up, existing distros changing init systems, etc. The virtual modules
likely to be affected by this are in the list below (click each item for more
information):
- :ref:`pkg <virtual-pkg>`
- :ref:`service <virtual-service>`
- :ref:`user <virtual-user>`
- :ref:`shadow <virtual-shadow>`
- :ref:`group <virtual-group>`
If Salt is using the wrong module for one of these, first of all, please
`report it on the issue tracker`__, so that this issue can be resolved for a
future release. To make it easier to troubleshoot, please also provide the
:py:mod:`grains.items <salt.modules.grains.items>` output, taking care to
redact any sensitive information.
Then, while waiting for the SaltStack development team to fix the issue, Salt
can be made to use the correct module using the :conf_minion:`providers` option
in the minion config file:
.. code-block:: yaml
providers:
service: systemd
pkg: aptpkg
The above example will force the minion to use the :py:mod:`systemd
<salt.modules.systemd>` module to provide service mangement, and the
:py:mod:`aptpkg <salt.modules.aptpkg>` module to provide package management.
.. __: https://github.com/saltstack/salt/issues/new
.. _modules-virtual-name:

1
doc/ref/queues/all/index.rst
View file

@ -10,4 +10,5 @@ Full list of builtin queues
:toctree:
:template: autosummary.rst.tmpl
pgjsonb_queue
sqlite_queue

125
doc/ref/states/providers.rst
View file

@ -9,9 +9,9 @@ properties of a system. These determinations are generally made for modules
that provide things like package and service management.
Sometimes in states, it may be necessary to use an alternative module to
provide the needed functionality. For instance, an older Arch Linux system may
not be running systemd, so instead of using the systemd service module, you can
revert to the default service module:
provide the needed functionality. For instance, an very old Arch Linux system
may not be running systemd, so instead of using the systemd service module, you
can revert to the default service module:
.. code-block:: yaml
@ -24,118 +24,13 @@ In this instance, the basic :py:mod:`~salt.modules.service` module (which
manages :program:`sysvinit`-based services) will replace the
:py:mod:`~salt.modules.systemd` module which is used by default on Arch Linux.
However, if it is necessary to make this override for most or every service,
it is better to just override the provider in the minion config file, as
described in the section below.
Setting a Provider in the Minion Config File
============================================
.. _`issue tracker`: https://github.com/saltstack/salt/issues
Sometimes, when running Salt on custom Linux spins, or distribution that are derived
from other distributions, Salt does not successfully detect providers. The providers
which are most likely to be affected by this are:
- pkg
- service
- user
- group
When something like this happens, rather than specifying the provider manually
in each state, it easier to use the :conf_minion:`providers` parameter in the
minion config file to set the provider.
If you end up needing to override a provider because it was not detected,
please let us know! File an issue on the `issue tracker`_, and provide the
output from the :mod:`grains.items <salt.modules.grains.items>` function,
taking care to sanitize any sensitive information.
Below are tables that should help with deciding which provider to use if one
needs to be overridden.
Provider: ``pkg``
*****************
======================= =======================================================
Execution Module Used for
======================= =======================================================
apt Debian/Ubuntu-based distros which use ``apt-get(8)``
for package management
brew Mac OS software management using `Homebrew`_
ebuild Gentoo-based systems (utilizes the ``portage`` python
module as well as ``emerge(1)``)
freebsdpkg FreeBSD-based OSes using ``pkg_add(1)``
openbsdpkg OpenBSD-based OSes using ``pkg_add(1)``
pacman Arch Linux-based distros using ``pacman(8)``
pkgin NetBSD-based OSes using ``pkgin(1)``
pkgng FreeBSD-based OSes using ``pkg(8)``
pkgutil Solaris-based OSes using `OpenCSW`_'s ``pkgutil(1)``
solarispkg Solaris-based OSes using ``pkgadd(1M)``
solarisips Solaris-based OSes using IPS ``pkg(1)``
win_pkg Windows
yumpkg RedHat-based distros and derivatives (wraps ``yum(8)``)
zypper SUSE-based distros using ``zypper(8)``
======================= =======================================================
.. _Homebrew: http://brew.sh/
.. _OpenCSW: http://www.opencsw.org/
Provider: ``service``
*********************
======================= =======================================================
Execution Module Used for
======================= =======================================================
debian_service Debian (non-systemd)
freebsdservice FreeBSD-based OSes using ``service(8)``
gentoo_service Gentoo Linux using :program:`sysvinit` and
``rc-update(8)``
launchctl Mac OS hosts using ``launchctl(1)``
netbsdservice NetBSD-based OSes
openbsdservice OpenBSD-based OSes
rh_service RedHat-based distros and derivatives using
``service(8)`` and ``chkconfig(8)``. Supports both
pure sysvinit and mixed sysvinit/upstart systems.
service Fallback which simply wraps sysvinit scripts
smf Solaris-based OSes which use SMF
systemd Linux distros which use systemd
upstart Ubuntu-based distros using upstart
win_service Windows
======================= =======================================================
Provider: ``user``
******************
======================= =======================================================
Execution Module Used for
======================= =======================================================
useradd Linux, NetBSD, and OpenBSD systems using
``useradd(8)``, ``userdel(8)``, and ``usermod(8)``
pw_user FreeBSD-based OSes using ``pw(8)``
solaris_user Solaris-based OSes using ``useradd(1M)``,
``userdel(1M)``, and ``usermod(1M)``
win_useradd Windows
======================= =======================================================
Provider: ``group``
*******************
======================= =======================================================
Execution Module Used for
======================= =======================================================
groupadd Linux, NetBSD, and OpenBSD systems using
``groupadd(8)``, ``groupdel(8)``, and ``groupmod(8)``
pw_group FreeBSD-based OSes using ``pw(8)``
solaris_group Solaris-based OSes using ``groupadd(1M)``,
``groupdel(1M)``, and ``groupmod(1M)``
win_groupadd Windows
======================= =======================================================
This change only affects this one state though. If it is necessary to make this
override for most or every service, it is better to just override the provider
in the minion config file, as described :ref:`here <module-provider-override>`.
Also, keep in mind that this only works for states with an identically-named
virtual module (:py:mod:`~salt.states.pkg`, :py:mod:`~salt.states.service`,
etc.).
Arbitrary Module Redirects
==========================
@ -155,4 +50,4 @@ In this example, the state is being instructed to use a custom module to invoke
commands.
Arbitrary module redirects can be used to dramatically change the behavior of a
given state.
given state.

2
doc/topics/cloud/proxmox.rst
View file

@ -3,7 +3,7 @@ Getting Started With Proxmox
============================
Proxmox Virtual Environment is a complete server virtualization management solution,
based on KVM virtualization and OpenVZ containers.
based on LXC and full virtualization with KVM.
Further information can be found at:
http://www.proxmox.org/

2
doc/topics/tutorials/states_pt1.rst
View file

@ -1,3 +1,5 @@
.. _states-tutorial:
=====================================
States tutorial, part 1 - Basic Usage
=====================================

9
salt/cli/batch.py
View file

@ -27,12 +27,13 @@ class Batch(object):
'''
Manage the execution of batch runs
'''
def __init__(self, opts, eauth=None, quiet=False):
def __init__(self, opts, eauth=None, quiet=False, parser=None):
self.opts = opts
self.eauth = eauth if eauth else {}
self.quiet = quiet
self.local = salt.client.get_local_client(opts['conf_file'])
self.minions, self.ping_gen = self.__gather_minions()
self.options = parser
def __gather_minions(self):
'''
@ -108,6 +109,11 @@ class Batch(object):
bwait = self.opts.get('batch_wait', 0)
wait = []
if self.options:
show_jid = self.options.show_jid
else:
show_jid = False
# the minion tracker keeps track of responses and iterators
# - it removes finished iterators from iters[]
# - if a previously detected minion does not respond, its
@ -145,6 +151,7 @@ class Batch(object):
*args,
raw=self.opts.get('raw', False),
ret=self.opts.get('return', ''),
show_jid=show_jid,
**self.eauth)
# add it to our iterators and to the minion_tracker
iters.append(new_iter)

2
salt/cli/salt.py
View file

@ -224,7 +224,7 @@ class SaltCMD(parsers.SaltCMDOptionParser):
else:
try:
batch = salt.cli.batch.Batch(self.config, eauth=eauth)
batch = salt.cli.batch.Batch(self.config, eauth=eauth, parser=self.options)
except salt.exceptions.SaltClientError as exc:
# We will print errors to the console further down the stack
sys.exit(1)

9
salt/client/__init__.py
View file

@ -682,8 +682,7 @@ class LocalClient(object):
if not fn_ret:
continue
yield fn_ret
self._clean_up_subscriptions(pub_data['jid'])
self._clean_up_subscriptions(pub_data['jid'])
def cmd_iter_no_block(
self,
@ -694,6 +693,7 @@ class LocalClient(object):
expr_form='glob',
ret='',
kwarg=None,
show_jid=False,
**kwargs):
'''
Yields the individual minion returns as they come in, or None
@ -737,9 +737,12 @@ class LocalClient(object):
tgt_type=expr_form,
block=False,
**kwargs):
if fn_ret and show_jid:
for minion in fn_ret.keys():
fn_ret[minion]['jid'] = pub_data['jid']
yield fn_ret
self._clean_up_subscriptions(pub_data['jid'])
self._clean_up_subscriptions(pub_data['jid'])
def cmd_full_return(
self,

3
salt/cloud/__init__.py
View file

@ -2223,8 +2223,7 @@ class Map(Cloud):
if self.opts['start_action']:
actionlist = []
grp = -1
for key, val in six.itervalues(groupby(iter(dmap['create'])),
lambda x: x['level']):
for key, val in groupby(six.itervalues(dmap['create']), lambda x: x['level']):
actionlist.append([])
grp += 1
for item in val:

18
salt/crypt.py
View file

@ -462,9 +462,12 @@ class AsyncAuth(object):
if not acceptance_wait_time_max:
acceptance_wait_time_max = acceptance_wait_time
creds = None
channel = salt.transport.client.AsyncReqChannel.factory(self.opts,
crypt='clear',
io_loop=self.io_loop)
while True:
try:
creds = yield self.sign_in()
creds = yield self.sign_in(channel=channel)
except SaltClientError:
break
if creds == 'retry':
@ -495,7 +498,7 @@ class AsyncAuth(object):
self._authenticate_future.set_result(True) # mark the sign-in as complete
@tornado.gen.coroutine
def sign_in(self, timeout=60, safe=True, tries=1):
def sign_in(self, timeout=60, safe=True, tries=1, channel=None):
'''
Send a sign in request to the master, sets the key information and
returns a dict containing the master publish interface to bind to
@ -527,7 +530,8 @@ class AsyncAuth(object):
auth['master_uri'] = self.opts['master_uri']
channel = salt.transport.client.AsyncReqChannel.factory(self.opts,
if not channel:
channel = salt.transport.client.AsyncReqChannel.factory(self.opts,
crypt='clear',
io_loop=self.io_loop)
@ -1002,10 +1006,11 @@ class SAuth(AsyncAuth):
'''
acceptance_wait_time = self.opts['acceptance_wait_time']
acceptance_wait_time_max = self.opts['acceptance_wait_time_max']
channel = salt.transport.client.ReqChannel.factory(self.opts, crypt='clear')
if not acceptance_wait_time_max:
acceptance_wait_time_max = acceptance_wait_time
while True:
creds = self.sign_in()
creds = self.sign_in(channel=channel)
if creds == 'retry':
if self.opts.get('caller'):
print('Minion failed to authenticate with the master, '
@ -1022,7 +1027,7 @@ class SAuth(AsyncAuth):
self._creds = creds
self._crypticle = Crypticle(self.opts, creds['aes'])
def sign_in(self, timeout=60, safe=True, tries=1):
def sign_in(self, timeout=60, safe=True, tries=1, channel=None):
'''
Send a sign in request to the master, sets the key information and
returns a dict containing the master publish interface to bind to
@ -1054,7 +1059,8 @@ class SAuth(AsyncAuth):
auth['master_uri'] = self.opts['master_uri']
channel = salt.transport.client.ReqChannel.factory(self.opts, crypt='clear')
if not channel:
channel = salt.transport.client.ReqChannel.factory(self.opts, crypt='clear')
sign_in_payload = self.minion_sign_in_payload()
try:

7
salt/modules/aptpkg.py
View file

@ -2,8 +2,13 @@
'''
Support for APT (Advanced Packaging Tool)
.. note::
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
.. note::
For virtual package support, either the ``python-apt`` or ``dctrl-tools``
package must be installed.

2
salt/modules/boto_elb.py
View file

@ -117,7 +117,7 @@ def exists(name, region=None, key=None, keyid=None, profile=None):
def get_elb_config(name, region=None, key=None, keyid=None, profile=None):
'''
Check to see if an ELB exists.
Get an ELB configuration.
CLI example:

6
salt/modules/bsd_shadow.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage the password database on BSD systems
.. important::
If you feel that Salt should be using this module to manage passwords on a
minion, and it is using a different module (or gives an error similar to
*'shadow.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/debian_service.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Service support for Debian systems (uses update-rc.d and /sbin/service)
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

2
salt/modules/dockerng.py
View file

@ -370,7 +370,7 @@ VALID_CREATE_OPTS = {
'domainname': {
'validator': 'string',
'path': 'Config:Domainname',
'default': '',
'get_default_from_container': True,
},
'interactive': {
'api_name': 'stdin_open',

6
salt/modules/ebuild.py
View file

@ -2,6 +2,12 @@
'''
Support for Portage
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
:optdepends: - portage Python adapter
For now all package names *MUST* include the package category,

6
salt/modules/freebsdpkg.py
View file

@ -2,6 +2,12 @@
'''
Remote package support using ``pkg_add(1)``
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
.. warning::
This module has been completely rewritten. Up to and including version

6
salt/modules/freebsdservice.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
The service module for FreeBSD
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/gentoo_service.py
View file

@ -2,6 +2,12 @@
'''
Top level package command wrapper, used to translate the os detected by grains
to the correct service manager
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import Python libs

6
salt/modules/groupadd.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage groups on Linux, OpenBSD and NetBSD
.. important::
If you feel that Salt should be using this module to manage groups on a
minion, and it is using a different module (or gives an error similar to
*'group.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/launchctl.py
View file

@ -2,6 +2,12 @@
'''
Module for the management of MacOS systems that use launchd/launchctl
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
:depends: - plistlib Python module
'''
from __future__ import absolute_import

6
salt/modules/mac_brew.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Homebrew for Mac OS X
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/mac_user.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage users on Mac OS 10.7+
.. important::
If you feel that Salt should be using this module to manage users on a
minion, and it is using a different module (or gives an error similar to
*'user.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

36
salt/modules/mysql.py
View file

@ -244,6 +244,26 @@ def __optimize_table(name, table, **connection_args):
return results
def __password_column(**connection_args):
dbc = _connect(**connection_args)
if dbc is None:
return 'Password'
cur = dbc.cursor()
qry = ('SELECT column_name from information_schema.COLUMNS '
'WHERE table_schema=%(schema)s and table_name=%(table)s '
'and column_name=%(column)s')
args = {
'schema': 'mysql',
'table': 'user',
'column': 'Password'
}
_execute(cur, qry, args)
if int(cur.rowcount) > 0:
return 'Password'
else:
return 'authentication_string'
def _connect(**kwargs):
'''
wrap authentication credentials here
@ -1082,7 +1102,7 @@ def user_exists(user,
password_hash=None,
passwordless=False,
unix_socket=False,
password_column='Password',
password_column=None,
**connection_args):
'''
Checks if a user exists on the MySQL server. A login can be checked to see
@ -1115,6 +1135,9 @@ def user_exists(user,
if dbc is None:
return False
if not password_column:
password_column = __password_column(**connection_args)
cur = dbc.cursor()
qry = ('SELECT User,Host FROM mysql.user WHERE User = %(user)s AND '
'Host = %(host)s')
@ -1185,7 +1208,7 @@ def user_create(user,
password_hash=None,
allow_passwordless=False,
unix_socket=False,
password_column='Password',
password_column=None,
**connection_args):
'''
Creates a MySQL user
@ -1236,6 +1259,9 @@ def user_create(user,
if dbc is None:
return False
if not password_column:
password_column = __password_column(**connection_args)
cur = dbc.cursor()
qry = 'CREATE USER %(user)s@%(host)s'
args = {}
@ -1285,6 +1311,7 @@ def user_chpass(user,
password_hash=None,
allow_passwordless=False,
unix_socket=None,
password_column=None,
**connection_args):
'''
Change password for a MySQL user
@ -1342,8 +1369,11 @@ def user_chpass(user,
if dbc is None:
return False
if not password_column:
password_column = __password_column(**connection_args)
cur = dbc.cursor()
qry = ('UPDATE mysql.user SET password='
qry = ('UPDATE mysql.user SET ' + password_column + '='
+ password_sql +
' WHERE User=%(user)s AND Host = %(host)s;')
args['user'] = user

6
salt/modules/netbsdservice.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
The service module for NetBSD
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/openbsdpkg.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Package support for OpenBSD
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/openbsdservice.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
The service module for OpenBSD
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/opkg.py
View file

@ -2,6 +2,12 @@
'''
Support for Opkg
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
.. versionadded: 2016.3.0
.. note::

6
salt/modules/pacman.py
View file

@ -2,6 +2,12 @@
'''
A module to wrap pacman calls, since Arch is the best
(https://wiki.archlinux.org/index.php/Arch_is_the_best)
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/pkgin.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Package support for pkgin based systems, inspired from freebsdpkg module
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

20
salt/modules/pkgng.py
View file

@ -2,6 +2,12 @@
'''
Support for ``pkgng``, the new package manager for FreeBSD
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
.. warning::
This module has been completely rewritten. Up to and including version
@ -279,17 +285,23 @@ def latest_version(*names, **kwargs):
cmd_prefix = _pkg(jail, chroot) + ['search']
for name in names:
cmd = [_pkg(jail, chroot), 'search']
# FreeBSD supports packages in format java/openjdk7
if '/' in name:
cmd = [_pkg(jail, chroot), 'search']
else:
cmd = [_pkg(jail, chroot), 'search', '-S', 'name', '-Q', 'version', '-e']
if quiet:
cmd.append('-q')
cmd.append(name)
pkgver = _get_version(
name,
__salt__['cmd.run'](cmd_prefix + [name],
output_loglevel='trace',
python_shell=False)
sorted(
__salt__['cmd.run'](cmd, python_shell=False, output_loglevel='trace').splitlines(),
reverse=True
).pop(0)
)
if pkgver is not None:
installed = pkgs.get(name, [])
if not installed:

6
salt/modules/pkgutil.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Pkgutil support for Solaris
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/pw_group.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage groups on FreeBSD
.. important::
If you feel that Salt should be using this module to manage groups on a
minion, and it is using a different module (or gives an error similar to
*'group.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/pw_user.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage users with the useradd command
.. important::
If you feel that Salt should be using this module to manage users on a
minion, and it is using a different module (or gives an error similar to
*'user.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Notes:

6
salt/modules/rh_service.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Service support for RHEL-based systems, including support for both upstart and sysvinit
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

7
salt/modules/rpm.py
View file

@ -475,6 +475,7 @@ def info(*packages, **attr):
"url": "%|URL?{url: %{URL}\\n}|",
"summary": "summary: %{SUMMARY}\\n",
"description": "description:\\n%{DESCRIPTION}\\n",
"edition": "edition: %|EPOCH?{%{EPOCH}:}|%{VERSION}-%{RELEASE}\\n",
}
attr = attr.get('attr', None) and attr['attr'].split(",") or None
@ -488,6 +489,9 @@ def info(*packages, **attr):
if 'name' not in attr:
attr.append('name')
query.append(attr_map['name'])
if 'edition' not in attr:
attr.append('edition')
query.append(attr_map['edition'])
else:
for attr_k, attr_v in attr_map.iteritems():
if attr_k != 'description':
@ -562,10 +566,11 @@ def info(*packages, **attr):
# pick only latest versions
# (in case multiple packages installed, e.g. kernel)
ret = dict()
for pkg_data in reversed(sorted(_ret, cmp=lambda a_vrs, b_vrs: version_cmp(a_vrs['version'], b_vrs['version']))):
for pkg_data in reversed(sorted(_ret, cmp=lambda a_vrs, b_vrs: version_cmp(a_vrs['edition'], b_vrs['edition']))):
pkg_name = pkg_data.pop('name')
if pkg_name not in ret:
ret[pkg_name] = pkg_data.copy()
del ret[pkg_name]['edition']
return ret

3
salt/modules/saltutil.py
View file

@ -992,8 +992,9 @@ def runner(_fun, **kwargs):
.. versionadded:: 2014.7.0
name
_fun
The name of the function to run
kwargs
Any keyword arguments to pass to the runner function

6
salt/modules/service.py
View file

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*-
'''
The default service module, if not otherwise specified salt will fall back
to this basic module
If Salt's OS detection does not identify a different virtual service module, the minion will fall back to using this basic module, which simply wraps sysvinit scripts.
'''
from __future__ import absolute_import
@ -44,7 +43,8 @@ def __virtual__():
'elementary OS',
'McAfee OS Server',
'Void',
'Mint'
'Mint',
'Raspbian'
))
if __grains__.get('os', '') in disable:
return (False, 'Your OS is on the disabled list')

8
salt/modules/shadow.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage the shadow file
Manage the shadow file on Linux systems
.. important::
If you feel that Salt should be using this module to manage passwords on a
minion, and it is using a different module (or gives an error similar to
*'shadow.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/smf.py
View file

@ -2,6 +2,12 @@
'''
Service support for Solaris 10 and 11, should work with other systems
that use SMF also. (e.g. SmartOS)
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import Python libs

6
salt/modules/solaris_group.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage groups on Solaris
.. important::
If you feel that Salt should be using this module to manage groups on a
minion, and it is using a different module (or gives an error similar to
*'group.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/solaris_shadow.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage the password database on Solaris systems
.. important::
If you feel that Salt should be using this module to manage passwords on a
minion, and it is using a different module (or gives an error similar to
*'shadow.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/solaris_user.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage users with the useradd command
.. important::
If you feel that Salt should be using this module to manage users on a
minion, and it is using a different module (or gives an error similar to
*'user.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/solarisips.py
View file

@ -2,6 +2,12 @@
'''
IPS pkg support for Solaris
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
This module provides support for Solaris 11 new package management - IPS (Image Packaging System).
This is the default pkg module for Solaris 11 (and later).

6
salt/modules/solarispkg.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Package support for Solaris
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

46
salt/modules/status.py
View file

@ -13,6 +13,7 @@ import re
import fnmatch
import collections
import copy
import time
# Import 3rd-party libs
import salt.ext.six as six
@ -26,14 +27,22 @@ import salt.utils.event
from salt.utils.network import host_to_ip as _host_to_ip
from salt.utils.network import remote_port_tcp as _remote_port_tcp
from salt.ext.six.moves import zip
from salt.utils.decorators import with_deprecated
from salt.exceptions import CommandExecutionError
__virtualname__ = 'status'
__opts__ = {}
# Don't shadow built-in's.
__func_alias__ = {
'time_': 'time'
}
def __virtual__():
if salt.utils.is_windows():
return (False, 'Cannot load status module on windows')
return False, 'Windows platform is not supported by this module'
return __virtualname__
@ -123,7 +132,38 @@ def custom():
return ret
def uptime(human_readable=True):
@with_deprecated(globals(), "Boron")
def uptime():
'''
Return the uptime for this system.
CLI Example:
.. code-block:: bash
salt '*' status.uptime
'''
ut_path = "/proc/uptime"
if not os.path.exists(ut_path):
raise CommandExecutionError("File {ut_path} was not found.".format(ut_path=ut_path))
ut_ret = {
'seconds': int(float(open(ut_path).read().strip().split()[0]))
}
utc_time = datetime.datetime.utcfromtimestamp(time.time() - ut_ret['seconds'])
ut_ret['since_iso'] = utc_time.isoformat()
ut_ret['since_t'] = time.mktime(utc_time.timetuple())
ut_ret['days'] = ut_ret['seconds'] / 60 / 60 / 24
hours = (ut_ret['seconds'] - (ut_ret['days'] * 24 * 60 * 60)) / 60 / 60
minutes = ((ut_ret['seconds'] - (ut_ret['days'] * 24 * 60 * 60)) / 60) - hours * 60
ut_ret['time'] = '{0}:{1}'.format(hours, minutes)
ut_ret['users'] = len(__salt__['cmd.run']("who -s").split(os.linesep))
return ut_ret
def _uptime(human_readable=True):
'''
Return the uptime for this minion
@ -876,7 +916,7 @@ def ping_master(master):
return result
def time(format='%A, %d. %B %Y %I:%M%p'):
def time_(format='%A, %d. %B %Y %I:%M%p'):
'''
.. versionadded:: 2016.3.0

6
salt/modules/systemd.py
View file

@ -3,6 +3,12 @@
Provide the service module for systemd
.. versionadded:: 0.10.0
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs
from __future__ import absolute_import

6
salt/modules/upstart.py
View file

@ -3,6 +3,12 @@
Module for the management of upstart systems. The Upstart system only supports
service starting, stopping and restarting.
.. important::
If you feel that Salt should be using this module to manage services on a
minion, and it is using a different module (or gives an error similar to
*'service.start' is not available*), see :ref:`here
<module-provider-override>`.
Currently (as of Ubuntu 12.04) there is no tool available to disable
Upstart services (like update-rc.d). This[1] is the recommended way to
disable an Upstart service. So we assume that all Upstart services

6
salt/modules/useradd.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage users with the useradd command
.. important::
If you feel that Salt should be using this module to manage users on a
minion, and it is using a different module (or gives an error similar to
*'user.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

6
salt/modules/win_groupadd.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage groups on Windows
.. important::
If you feel that Salt should be using this module to manage groups on a
minion, and it is using a different module (or gives an error similar to
*'group.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

7
salt/modules/win_pkg.py
View file

@ -1,6 +1,13 @@
# -*- coding: utf-8 -*-
'''
A module to manage software on Windows
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

6
salt/modules/win_shadow.py
View file

@ -1,6 +1,12 @@
# -*- coding: utf-8 -*-
'''
Manage the shadow file
.. important::
If you feel that Salt should be using this module to manage passwords on a
minion, and it is using a different module (or gives an error similar to
*'shadow.info' is not available*), see :ref:`here
<module-provider-override>`.
'''
from __future__ import absolute_import

5
salt/modules/win_timezone.py
View file

@ -493,7 +493,10 @@ def get_offset():
string = False
zone = __salt__['cmd.run'](['tzutil', '/g'], python_shell=False)
prev = ''
for line in __salt__['cmd.run'](['tzutil', '/l'], python_shell=False).splitlines():
zone_list = __salt__['cmd.run'](['tzutil', '/l'],
python_shell=False,
output_loglevel='trace').splitlines()
for line in zone_list:
if zone == line:
string = prev
break

6
salt/modules/win_useradd.py
View file

@ -2,6 +2,12 @@
'''
Module for managing Windows Users
.. important::
If you feel that Salt should be using this module to manage users on a
minion, and it is using a different module (or gives an error similar to
*'user.info' is not available*), see :ref:`here
<module-provider-override>`.
:depends:
- pywintypes
- win32api

6
salt/modules/yumpkg.py
View file

@ -2,6 +2,12 @@
'''
Support for YUM/DNF
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
.. note::
DNF is fully supported as of version 2015.5.10 and 2015.8.4 (partial
support for DNF was initially added in 2015.8.0), and DNF is used

7
salt/modules/zypper.py
View file

@ -3,6 +3,13 @@
Package support for openSUSE via the zypper package manager
:depends: - ``rpm`` Python module. Install with ``zypper install rpm-python``
.. important::
If you feel that Salt should be using this module to manage packages on a
minion, and it is using a different module (or gives an error similar to
*'pkg.install' is not available*), see :ref:`here
<module-provider-override>`.
'''
# Import python libs

27
salt/pillar/__init__.py
View file

@ -553,15 +553,24 @@ class Pillar(object):
log.error(msg)
errors.append(msg)
else:
log.debug(
'Specified SLS \'%s\' in environment \'%s\' was not '
'found. This could be because SLS \'%s\' is in an '
'environment other than \'%s\', but \'%s\' is included in '
'that environment\'s Pillar top file. It could also be '
'due to environment \'%s\' not being defined in '
'"pillar_roots"',
sls, saltenv, sls, saltenv, saltenv, saltenv
)
msg = ('Specified SLS \'{0}\' in environment \'{1}\' was not '
'found. '.format(sls, saltenv))
if self.opts.get('__git_pillar', False) is True:
msg += (
'This is likely caused by a git_pillar top file '
'containing an environment other than the one for the '
'branch in which it resides. Each git_pillar '
'branch/tag must have its own top file.'
)
else:
msg += (
'This could be because SLS \'{0}\' is in an '
'environment other than \'{1}\', but \'{1}\' is '
'included in that environment\'s Pillar top file. It '
'could also be due to environment \'{1}\' not being '
'defined in \'pillar_roots\'.'.format(sls, saltenv)
)
log.debug(msg)
# return state, mods, errors
return None, mods, errors
state = None

15
salt/pillar/git_pillar.py
View file

@ -26,6 +26,19 @@ the relevant environment, like so:
The branch/tag which maps to that environment must then be specified along with
the repo's URL. Configuration details can be found below.
.. important::
Each branch/tag used for git_pillar must have its own top file. This is
different from how the top file works when configuring :ref:`States
<states-tutorial>`. The reason for this is that each git_pillar branch/tag
is processed separately from the rest. Therefore, if the ``qa`` branch is
to be used for git_pillar, it would need to have its own top file, with the
``qa`` environment defined within it, like this:
.. code-block:: yaml
qa:
'dev-*':
- bar
.. _git-pillar-pre-2015-8-0:
@ -253,6 +266,7 @@ def ext_pillar(minion_id, repo, pillar_dirs):
else:
opts = copy.deepcopy(__opts__)
opts['pillar_roots'] = {}
opts['__git_pillar'] = True
pillar = salt.utils.gitfs.GitPillar(opts)
pillar.init_remotes(repo, PER_REMOTE_OVERRIDES)
pillar.checkout()
@ -456,6 +470,7 @@ def _legacy_git_pillar(minion_id, repo_string, pillar_dirs):
opts = copy.deepcopy(__opts__)
opts['pillar_roots'][environment] = [pillar_dir]
opts['__git_pillar'] = True
pil = Pillar(opts, __grains__, minion_id, branch)

53
salt/states/boto_asg.py
View file

@ -55,7 +55,6 @@ as a passed in dict, or as a string to pull from pillars or minion config:
- AddToLoadBalancer
- AlarmNotification
- scaling_policies
----------
- adjustment_type: ChangeInCapacity
- as_name: api-production-iad
- cooldown: 1800
@ -255,7 +254,7 @@ def present(
launch_config_name
Name of the launch config to use for the group. Or, if
launch_config is specified, this will be the launch config
``launch_config`` is specified, this will be the launch config
name's prefix. (see below)
launch_config
@ -263,8 +262,8 @@ def present(
launch config will be used or created, matching this set
of attributes, and the autoscale group will be set to use
that launch config. The launch config name will be the
launch_config_name followed by a hyphen followed by a hash
of the launch_config dict contents.
``launch_config_name`` followed by a hyphen followed by a hash
of the ``launch_config`` dict contents.
availability_zones
List of availability zones for the group.
@ -303,15 +302,23 @@ def present(
tags
A list of tags. Example:
.. code-block:: yaml
- key: 'key'
value: 'value'
propagate_at_launch: true
termination_policies
A list of termination policies. Valid values are: “OldestInstance”,
“NewestInstance”, “OldestLaunchConfiguration”,
“ClosestToNextInstanceHour”, “Default”. If no value is specified, the
“Default” value is used.
A list of termination policies. Valid values are:
* ``OldestInstance``
* ``NewestInstance``
* ``OldestLaunchConfiguration``
* ``ClosestToNextInstanceHour``
* ``Default``
If no value is specified, the ``Default`` value is used.
suspended_processes
List of processes to be suspended. see
@ -351,8 +358,11 @@ def present(
a dictionary of name->boto_cloudwatch_alarm sections to be associated with this ASG.
All attributes should be specified except for dimension which will be
automatically set to this ASG.
See the boto_cloudwatch_alarm state for information about these attributes.
if any alarm actions include ":self:" this will be replaced with the asg name.
See the :mod:`salt.states.boto_cloudwatch_alarm` state for information
about these attributes.
If any alarm actions include ":self:" this will be replaced with the asg name.
For example, alarm_actions reading "['scaling_policy:self:ScaleUp']" will
map to the arn for this asg's scaling policy named "ScaleUp".
In addition, any alarms that have only scaling_policy as actions will be ignored if
@ -376,24 +386,27 @@ def present(
that contains a dict with region, key and keyid.
notification_arn
The aws arn that notifications will be sent to
The AWS arn that notifications will be sent to
notification_arn_from_pillar
name of the pillar dict that contains notification_arn settings. A notification_arn
defined for this specific state will override the one from pillar.
name of the pillar dict that contains ``notifcation_arn`` settings. A
``notification_arn`` defined for this specific state will override the
one from pillar.
notification_types
A list of event names that will trigger a notification. The list of valid
notification types is:
- "autoscaling:EC2_INSTANCE_LAUNCH"
- "autoscaling:EC2_INSTANCE_LAUNCH_ERROR"
- "autoscaling:EC2_INSTANCE_TERMINATE"
- "autoscaling:EC2_INSTANCE_TERMINATE_ERROR"
- "autoscaling:TEST_NOTIFICATION"
* ``autoscaling:EC2_INSTANCE_LAUNCH``
* ``autoscaling:EC2_INSTANCE_LAUNCH_ERROR``
* ``autoscaling:EC2_INSTANCE_TERMINATE``
* ``autoscaling:EC2_INSTANCE_TERMINATE_ERROR``
* ``autoscaling:TEST_NOTIFICATION``
notification_types_from_pillar
name of the pillar dict that contains notifcation_types settings. Notification_types
defined for this specific state will override those from the pillar.
name of the pillar dict that contains ``notifcation_types`` settings.
``notification_types`` defined for this specific state will override those
from the pillar.
'''
ret = {'name': name, 'result': True, 'comment': '', 'changes': {}}
if vpc_zone_identifier:

10
salt/states/boto_elb.py
View file

@ -295,7 +295,8 @@ def present(
The security groups assigned to your LoadBalancer within your VPC.
scheme
The type of a LoadBalancer. internet-facing or internal. Once set, can not be modified.
The type of a LoadBalancer, ``internet-facing`` or ``internal``. Once
set, can not be modified.
health_check
A dict defining the health check for this ELB.
@ -314,11 +315,16 @@ def present(
Other DNS modules can be called by specifying the provider keyword.
the cnames dict will be passed to the state as kwargs.
See the :mod:`salt.states.boto_route53` state for information about
these attributes.
alarms:
a dictionary of name->boto_cloudwatch_alarm sections to be associated with this ELB.
All attributes should be specified except for dimension which will be
automatically set to this ELB.
See the boto_cloudwatch_alarm state for information about these attributes.
See the :mod:`salt.states.boto_cloudwatch_alarm` state for information
about these attributes.
alarms_from_pillar:
name of pillar dict that contains alarm settings. Alarms defined for this specific

28
salt/states/cmd.py
View file

@ -797,13 +797,6 @@ def run(name,
'result': False,
'comment': ''}
if cwd and not os.path.isdir(cwd):
ret['comment'] = (
'Desired working directory "{0}" '
'is not available'
).format(cwd)
return ret
# Need the check for None here, if env is not provided then it falls back
# to None and it is assumed that the environment is not being overridden.
if env is not None and not isinstance(env, (list, dict)):
@ -834,6 +827,13 @@ def run(name,
ret['comment'] = 'Command "{0}" would have been executed'.format(name)
return _reinterpreted_state(ret) if stateful else ret
if cwd and not os.path.isdir(cwd):
ret['comment'] = (
'Desired working directory "{0}" '
'is not available'
).format(cwd)
return ret
# Wow, we passed the test, run this sucker!
try:
cmd_all = __salt__['cmd.run_all'](
@ -1019,13 +1019,6 @@ def script(name,
'result': False,
'comment': ''}
if cwd and not os.path.isdir(cwd):
ret['comment'] = (
'Desired working directory "{0}" '
'is not available'
).format(cwd)
return ret
# Need the check for None here, if env is not provided then it falls back
# to None and it is assumed that the environment is not being overridden.
if env is not None and not isinstance(env, (list, dict)):
@ -1094,6 +1087,13 @@ def script(name,
'executed'.format(name)
return _reinterpreted_state(ret) if stateful else ret
if cwd and not os.path.isdir(cwd):
ret['comment'] = (
'Desired working directory "{0}" '
'is not available'
).format(cwd)
return ret
# Wow, we passed the test, run this sucker!
try:
cmd_all = __salt__['cmd.script'](source, python_shell=True, **cmd_kwargs)

2
salt/transport/tcp.py
View file

@ -891,6 +891,7 @@ class SaltMessageClient(object):
message_id, item = self.send_queue[0]
try:
yield self._stream.write(item)
del self.send_queue[0]
# if the connection is dead, lets fail this send, and make sure we
# attempt to reconnect
except tornado.iostream.StreamClosedError as e:
@ -906,7 +907,6 @@ class SaltMessageClient(object):
if self._connecting_future.done():
self._connecting_future = self.connect()
yield self._connecting_future
del self.send_queue[0]
def _message_id(self):
wrap = False

1
salt/transport/zeromq.py
View file

@ -83,6 +83,7 @@ class AsyncZeroMQReqChannel(salt.transport.client.ReqChannel):
new_obj = object.__new__(cls)
new_obj.__singleton_init__(opts, **kwargs)
loop_instance_map[key] = new_obj
log.trace('Inserted key into loop_instance_map id {0} for key {1} and process {2}'.format(id(loop_instance_map), key, os.getpid()))
else:
log.debug('Re-using AsyncZeroMQReqChannel for {0}'.format(key))
try:

345
salt/utils/decorators/__init__.py
View file

@ -13,7 +13,8 @@ from collections import defaultdict
# Import salt libs
import salt.utils
from salt.exceptions import CommandNotFoundError
from salt.exceptions import CommandNotFoundError, CommandExecutionError
from salt.version import SaltStackVersion, __saltstack_version__
from salt.log import LOG_LEVELS
# Import 3rd-party libs
@ -144,10 +145,7 @@ class Depends(object):
continue
class depends(Depends): # pylint: disable=C0103
'''
Wrapper of Depends for capitalization
'''
depends = Depends
def timing(function):
@ -248,3 +246,340 @@ def memoize(func):
cache[args] = func(*args)
return cache[args]
return _memoize
class _DeprecationDecorator(object):
'''
Base mix-in class for the deprecation decorator.
Takes care of a common functionality, used in its derivatives.
'''
def __init__(self, globals, version):
'''
Constructor.
:param globals: Module globals. Important for finding out replacement functions
:param version: Expiration version
:return:
'''
self._globals = globals
self._exp_version_name = version
self._exp_version = SaltStackVersion.from_name(self._exp_version_name)
self._curr_version = __saltstack_version__.info
self._options = self._globals['__opts__']
self._raise_later = None
self._function = None
self._orig_f_name = None
def _get_args(self, kwargs):
'''
Extract function-specific keywords from all of the kwargs.
:param kwargs:
:return:
'''
_args = list()
_kwargs = dict()
for arg_item in kwargs.get('__pub_arg', list()):
if type(arg_item) == dict:
_kwargs.update(arg_item.copy())
else:
_args.append(arg_item)
return _args, _kwargs
def _call_function(self, kwargs):
'''
Call target function that has been decorated.
:return:
'''
if self._raise_later:
raise self._raise_later # pylint: disable=E0702
if self._function:
args, kwargs = self._get_args(kwargs)
try:
return self._function(*args, **kwargs)
except TypeError as error:
error = str(error).replace(self._function.func_name, self._orig_f_name) # Hide hidden functions
log.error('Function "{f_name}" was not properly called: {error}'.format(f_name=self._orig_f_name,
error=error))
return self._function.__doc__
except Exception as error:
log.error('Unhandled exception occurred in '
'function "{f_name}: {error}'.format(f_name=self._function.func_name,
error=error))
raise error
else:
raise CommandExecutionError("Function is deprecated, but the successor function was not found.")
def __call__(self, function):
'''
Callable method of the decorator object when
the decorated function is gets called.
:param function:
:return:
'''
self._function = function
self._orig_f_name = self._function.func_name
class _IsDeprecated(_DeprecationDecorator):
'''
This decorator should be used only with the deprecated functions
to mark them as deprecated and alter its behavior a corresponding way.
The usage is only suitable if deprecation process is renaming
the function from one to another. In case function name or even function
signature stays the same, please use 'with_deprecated' decorator instead.
It has the following functionality:
1. Put a warning level message to the log, informing that
the deprecated function has been in use.
2. Raise an exception, if deprecated function is being called,
but the lifetime of it already expired.
3. Point to the successor of the deprecated function in the
log messages as well during the blocking it, once expired.
Usage of this decorator as follows. In this example no successor
is mentioned, hence the function "foo()" will be logged with the
warning each time is called and blocked completely, once EOF of
it is reached:
from salt.util.decorators import is_deprecated
@is_deprecated(globals(), "Beryllium")
def foo():
pass
In the following example a successor function is mentioned, hence
every time the function "bar()" is called, message will suggest
to use function "baz()" instead. Once EOF is reached of the function
"bar()", an exception will ask to use function "baz()", in order
to continue:
from salt.util.decorators import is_deprecated
@is_deprecated(globals(), "Beryllium", with_successor="baz")
def bar():
pass
def baz():
pass
'''
def __init__(self, globals, version, with_successor=None):
'''
Constructor of the decorator 'is_deprecated'.
:param globals: Module globals
:param version: Version to be deprecated
:param with_successor: Successor function (optional)
:return:
'''
_DeprecationDecorator.__init__(self, globals, version)
self._successor = with_successor
def __call__(self, function):
'''
Callable method of the decorator object when
the decorated function is gets called.
:param function:
:return:
'''
_DeprecationDecorator.__call__(self, function)
def _decorate(*args, **kwargs):
'''
Decorator function.
:param args:
:param kwargs:
:return:
'''
if self._curr_version < self._exp_version:
msg = ['The function "{f_name}" is deprecated and will '
'expire in version "{version_name}".'.format(f_name=self._function.func_name,
version_name=self._exp_version_name)]
if self._successor:
msg.append('Use successor "{successor}" instead.'.format(successor=self._successor))
log.warning(' '.join(msg))
else:
msg = ['The lifetime of the function "{f_name}" expired.'.format(f_name=self._function.func_name)]
if self._successor:
msg.append('Please use its successor "{successor}" instead.'.format(successor=self._successor))
log.warning(' '.join(msg))
raise CommandExecutionError(' '.join(msg))
return self._call_function(kwargs)
return _decorate
is_deprecated = _IsDeprecated
class _WithDeprecated(_DeprecationDecorator):
'''
This decorator should be used with the successor functions
to mark them as a new and alter its behavior in a corresponding way.
It is used alone if a function content or function signature
needs to be replaced, leaving the name of the function same.
In case function needs to be renamed or just dropped, it has
to be used in pair with 'is_deprecated' decorator.
It has the following functionality:
1. Put a warning level message to the log, in case a component
is using its deprecated version.
2. Switch between old and new function in case an older version
is configured for the desired use.
3. Raise an exception, if deprecated version reached EOL and
point out for the new version.
Usage of this decorator as follows. If 'with_name' is not specified,
then the name of the deprecated function is assumed with the "_" prefix.
In this case, in order to deprecate a function, it is required:
- Add a prefix "_" to an existing function. E.g.: "foo()" to "_foo()".
- Implement a new function with exactly the same name, just without
the prefix "_".
Example:
from salt.util.decorators import with_deprecated
@with_deprecated(globals(), "Beryllium")
def foo():
"This is a new function"
def _foo():
"This is a deprecated function"
In case there is a need to deprecate a function and rename it,
the decorator shuld be used with the 'with_name' parameter. This
parameter is pointing to the existing deprecated function. In this
case deprecation process as follows:
- Leave a deprecated function without changes, as is.
- Implement a new function and decorate it with this decorator.
- Set a parameter 'with_name' to the deprecated function.
- If a new function has a different name than a deprecated,
decorate a deprecated function with the 'is_deprecated' decorator
in order to let the function have a deprecated behavior.
Example:
from salt.util.decorators import with_deprecated
@with_deprecated(globals(), "Beryllium", with_name="an_old_function")
def a_new_function():
"This is a new function"
@is_deprecated(globals(), "Beryllium", with_successor="a_new_function")
def an_old_function():
"This is a deprecated function"
'''
MODULE_NAME = '__virtualname__'
CFG_KEY = 'use_deprecated'
def __init__(self, globals, version, with_name=None):
'''
Constructor of the decorator 'with_deprecated'
:param globals:
:param version:
:param with_name:
:return:
'''
_DeprecationDecorator.__init__(self, globals, version)
self._with_name = with_name
def _set_function(self, function):
'''
Based on the configuration, set to execute an old or a new function.
:return:
'''
full_name = "{m_name}.{f_name}".format(m_name=self._globals.get(self.MODULE_NAME, ''),
f_name=function.func_name)
if full_name.startswith("."):
self._raise_later = CommandExecutionError('Module not found for function "{f_name}"'.format(
f_name=function.func_name))
if full_name in self._options.get(self.CFG_KEY, list()):
self._function = self._globals.get(self._with_name or "_{0}".format(function.func_name))
def _is_used_deprecated(self):
'''
Returns True, if a component configuration explicitly is
asking to use an old version of the deprecated function.
:return:
'''
return "{m_name}.{f_name}".format(m_name=self._globals.get(self.MODULE_NAME, ''),
f_name=self._orig_f_name) in self._options.get(self.CFG_KEY, list())
def __call__(self, function):
'''
Callable method of the decorator object when
the decorated function is gets called.
:param function:
:return:
'''
_DeprecationDecorator.__call__(self, function)
def _decorate(*args, **kwargs):
'''
Decorator function.
:param args:
:param kwargs:
:return:
'''
self._set_function(function)
if self._is_used_deprecated():
if self._curr_version < self._exp_version:
msg = list()
if self._with_name:
msg.append('The function "{f_name}" is deprecated and will '
'expire in version "{version_name}".'.format(
f_name=self._with_name.startswith("_") and self._orig_f_name or self._with_name,
version_name=self._exp_version_name))
else:
msg.append('The function is using its deprecated version and will '
'expire in version "{version_name}".'.format(version_name=self._exp_version_name))
msg.append('Use its successor "{successor}" instead.'.format(successor=self._orig_f_name))
log.warning(' '.join(msg))
else:
msg_patt = 'The lifetime of the function "{f_name}" expired.'
if '_' + self._orig_f_name == self._function.func_name:
msg = [msg_patt.format(f_name=self._orig_f_name),
'Please turn off its deprecated version in the configuration']
else:
msg = ['Although function "{f_name}" is called, an alias "{f_alias}" '
'is configured as its deprecated version.'.format(
f_name=self._orig_f_name, f_alias=self._with_name or self._orig_f_name),
msg_patt.format(f_name=self._with_name or self._orig_f_name),
'Please use its successor "{successor}" instead.'.format(successor=self._orig_f_name)]
log.error(' '.join(msg))
raise CommandExecutionError(' '.join(msg))
return self._call_function(kwargs)
_decorate.__doc__ = self._function.__doc__
return _decorate
with_deprecated = _WithDeprecated

12
tests/integration/__init__.py
View file

@ -78,15 +78,11 @@ if salt.utils.is_windows():
SYS_TMP_DIR = os.path.realpath(
os.environ.get(
# Avoid MacOS ${TMPDIR} as it yields a base path too long for unix sockets:
# 'error: AF_UNIX path too long'
# Gentoo Portage prefers ebuild tests are rooted in ${TMPDIR}
'TMPDIR' if not salt.utils.is_darwin() else '',
tempfile.gettempdir()
)
# Avoid ${TMPDIR} and gettempdir() on MacOS as they yield a base path too long
# for unix sockets: ``error: AF_UNIX path too long``
# Gentoo Portage prefers ebuild tests are rooted in ${TMPDIR}
os.environ.get('TMPDIR', tempfile.gettempdir()) if salt.utils.is_darwin() else '/tmp'
)
TMP = os.path.join(SYS_TMP_DIR, 'salt-tests-tmpdir')
FILES = os.path.join(INTEGRATION_TEST_DIR, 'files')
PYEXEC = 'python{0}.{1}'.format(*sys.version_info)

12
tests/integration/files/file/base/_modules/runtests_helpers.py
View file

@ -17,15 +17,11 @@ import salt.utils
SYS_TMP_DIR = os.path.realpath(
os.environ.get(
# Avoid MacOS ${TMPDIR} as it yields a base path too long for unix sockets:
# 'error: AF_UNIX path too long'
# Gentoo Portage prefers ebuild tests are rooted in ${TMPDIR}
'TMPDIR' if not salt.utils.is_darwin() else '',
tempfile.gettempdir()
)
# Avoid ${TMPDIR} and gettempdir() on MacOS as they yield a base path too long
# for unix sockets: ``error: AF_UNIX path too long``
# Gentoo Portage prefers ebuild tests are rooted in ${TMPDIR}
os.environ.get('TMPDIR', tempfile.gettempdir()) if salt.utils.is_darwin() else '/tmp'
)
# This tempdir path is defined on tests.integration.__init__
TMP = os.path.join(SYS_TMP_DIR, 'salt-tests-tmpdir')

2
tests/unit/modules/mysql_test.py
View file

@ -88,7 +88,7 @@ class MySQLTestCase(TestCase):
mysql.user_chpass('testuser', password='BLUECOW')
calls = (
call().cursor().execute(
'UPDATE mysql.user SET password=PASSWORD(%(password)s) WHERE User=%(user)s AND Host = %(host)s;',
'UPDATE mysql.user SET Password=PASSWORD(%(password)s) WHERE User=%(user)s AND Host = %(host)s;',
{'password': 'BLUECOW',
'user': 'testuser',
'host': 'localhost',

48
tests/unit/modules/status_test.py
View file

@ -5,15 +5,14 @@ from __future__ import absolute_import
# Import Salt Libs
from salt.modules import status
from salt.exceptions import CommandExecutionError
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting import TestCase
from salttesting.helpers import ensure_in_syspath
from salttesting.mock import (
MagicMock,
patch,
NO_MOCK,
NO_MOCK_REASON
)
ensure_in_syspath('../../')
@ -22,36 +21,67 @@ ensure_in_syspath('../../')
status.__salt__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class StatusTestCase(TestCase):
'''
test modules.status functions
'''
def test_uptime(self):
'''
test modules.status.uptime function
Test modules.status.uptime function, new version
:return:
'''
class ProcUptime(object):
def __init__(self, *args, **kwargs):
self.data = "773865.18 1003405.46"
def read(self):
return self.data
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value="1\n2\n3")}):
with patch('os.path.exists', MagicMock(return_value=True)):
with patch('time.time', MagicMock(return_value=1458821523.72)):
status.open = ProcUptime
u_time = status.uptime()
self.assertEqual(u_time['users'], 3)
self.assertEqual(u_time['seconds'], 773865)
self.assertEqual(u_time['days'], 8)
self.assertEqual(u_time['time'], '22:57')
def test_uptime_failure(self):
'''
Test modules.status.uptime function should raise an exception if /proc/uptime does not exists.
:return:
'''
with patch('os.path.exists', MagicMock(return_value=False)):
with self.assertRaises(CommandExecutionError):
status.uptime()
def test_deprecated_uptime(self):
'''
test modules.status.uptime function, deprecated version
'''
mock_uptime = 'very often'
mock_run = MagicMock(return_value=mock_uptime)
with patch.dict(status.__salt__, {'cmd.run': mock_run}):
self.assertEqual(status.uptime(), mock_uptime)
self.assertEqual(status._uptime(), mock_uptime)
mock_uptime = 'very idle'
mock_run = MagicMock(return_value=mock_uptime)
with patch.dict(status.__salt__, {'cmd.run': mock_run}):
with patch('os.path.exists', MagicMock(return_value=True)):
self.assertEqual(status.uptime(human_readable=False), mock_uptime.split()[0])
self.assertEqual(status._uptime(human_readable=False), mock_uptime.split()[0])
mock_uptime = ''
mock_return = 'unexpected format in /proc/uptime'
mock_run = MagicMock(return_value=mock_uptime)
with patch.dict(status.__salt__, {'cmd.run': mock_run}):
with patch('os.path.exists', MagicMock(return_value=True)):
self.assertEqual(status.uptime(human_readable=False), mock_return)
self.assertEqual(status._uptime(human_readable=False), mock_return)
mock_return = 'cannot find /proc/uptime'
with patch('os.path.exists', MagicMock(return_value=False)):
self.assertEqual(status.uptime(human_readable=False), mock_return)
self.assertEqual(status._uptime(human_readable=False), mock_return)
if __name__ == '__main__':

12
tests/unit/states/cmd_test.py
View file

@ -131,12 +131,6 @@ class CmdTestCase(TestCase):
'changes': {},
'comment': ''}
with patch.object(os.path, 'isdir', MagicMock(return_value=False)):
with patch.dict(cmd.__opts__, {'test': False}):
comt = ('Desired working directory "/tmp/salt" is not available')
ret.update({'comment': comt})
self.assertDictEqual(cmd.run(name, cwd='/tmp/salt'), ret)
with patch.dict(cmd.__opts__, {'test': False}):
comt = ("Invalidly-formatted 'env' parameter. See documentation.")
ret.update({'comment': comt})
@ -180,12 +174,6 @@ class CmdTestCase(TestCase):
'changes': {},
'comment': ''}
with patch.object(os.path, 'isdir', MagicMock(return_value=False)):
with patch.dict(cmd.__opts__, {'test': False}):
comt = ('Desired working directory "/tmp/salt" is not available')
ret.update({'comment': comt})
self.assertDictEqual(cmd.script(name, cwd='/tmp/salt'), ret)
with patch.dict(cmd.__opts__, {'test': False}):
comt = ("Invalidly-formatted 'env' parameter. See documentation.")
ret.update({'comment': comt})

50
tests/unit/states/file_test.py
View file

@ -4,6 +4,7 @@
from __future__ import absolute_import
import json
import pprint
import tempfile
# Import Salt Testing libs
from salttesting import skipIf, TestCase
@ -138,13 +139,16 @@ class FileTestCase(TestCase):
'''
# 'symlink' function tests: 1
@destructiveTest
def test_symlink(self):
'''
Test to create a symlink.
'''
name = '/etc/grub.conf'
target = '/boot/grub/grub.conf'
name = '/tmp/testfile.txt'
target = tempfile.mkstemp()[1]
test_dir = '/tmp'
user = 'salt'
if salt.utils.is_windows():
group = 'salt'
else:
@ -200,10 +204,10 @@ class FileTestCase(TestCase):
with patch.dict(filestate.__opts__, {'test': False}):
with patch.object(os.path, 'isdir', mock_f):
with patch.object(os.path, 'exists', mock_f):
comt = ('Directory /etc for symlink is not present')
comt = ('Directory {0} for symlink is not present').format(test_dir)
ret.update({'comment': comt,
'result': False,
'pchanges': {'new': '/etc/grub.conf'}})
'pchanges': {'new': name}})
self.assertDictEqual(filestate.symlink(name, target,
user=user,
group=group), ret)
@ -232,16 +236,17 @@ class FileTestCase(TestCase):
'file.readlink': mock_target}):
with patch.dict(filestate.__opts__, {'test': False}):
with patch.object(os.path, 'isdir', mock_t):
with patch.object(os.path, 'lexists', mock_t):
comt = ('File exists where the backup target SALT'
' should go')
ret.update({'comment': comt,
'result': False,
'pchanges': {'new': name}})
self.assertDictEqual(filestate.symlink
(name, target, user=user,
group=group, backupname='SALT'),
ret)
with patch.object(os.path, 'exists', mock_f):
with patch.object(os.path, 'lexists', mock_t):
comt = ('File exists where the backup target SALT'
' should go')
ret.update({'comment': comt,
'result': False,
'pchanges': {'new': name}})
self.assertDictEqual(filestate.symlink
(name, target, user=user,
group=group, backupname='SALT'),
ret)
with patch.dict(filestate.__salt__, {'config.manage_mode': mock_t,
'file.user_to_uid': mock_uid,
@ -250,13 +255,16 @@ class FileTestCase(TestCase):
'file.readlink': mock_target}):
with patch.dict(filestate.__opts__, {'test': False}):
with patch.object(os.path, 'isdir', mock_t):
with patch.object(os.path, 'isfile', mock_t):
comt = ('File exists where the symlink {0} should be'
.format(name))
ret.update({'comment': comt, 'result': False})
self.assertDictEqual(filestate.symlink
(name, target, user=user,
group=group), ret)
with patch.object(os.path, 'exists', mock_f):
with patch.object(os.path, 'isfile', mock_t):
comt = ('File exists where the symlink {0} should be'
.format(name))
ret.update({'comment': comt,
'pchanges': {'new': name},
'result': False})
self.assertDictEqual(filestate.symlink
(name, target, user=user,
group=group), ret)
with patch.dict(filestate.__salt__, {'config.manage_mode': mock_t,
'file.user_to_uid': mock_uid,

232
tests/unit/utils/decorators_test.py Normal file
View file

@ -0,0 +1,232 @@
# -*- coding: utf-8 -*-
'''
:codeauthor: :email:`Bo Maryniuk (bo@suse.de)`
unit.utils.decorators_test
'''
# Import Python libs
from __future__ import absolute_import
# Import Salt Testing libs
from salttesting import TestCase
from salttesting.helpers import ensure_in_syspath
from salt.utils import decorators
from salt.version import SaltStackVersion
from salt.exceptions import CommandExecutionError
ensure_in_syspath('../../')
class DummyLogger(object):
'''
Dummy logger accepts everything and simply logs
'''
def __init__(self, messages):
self._messages = messages
def __getattr__(self, item):
return self._log
def _log(self, msg):
self._messages.append(msg)
class DecoratorsTest(TestCase):
'''
Testing decorators.
'''
def old_function(self):
return "old"
def new_function(self):
return "new"
def _mk_version(self, name):
'''
Make a version
:return:
'''
return name, SaltStackVersion.from_name(name)
def setUp(self):
'''
Setup a test
:return:
'''
self.globs = {
'__virtualname__': 'test',
'__opts__': {},
'old_function': self.old_function,
'new_function': self.new_function,
}
self.messages = list()
decorators.log = DummyLogger(self.messages)
def test_is_deprecated_version_eol(self):
'''
Use of is_deprecated will result to the exception,
if the expiration version is lower than the current version.
A successor function is not pointed out.
:return:
'''
depr = decorators.is_deprecated(self.globs, "Helium")
depr._curr_version = self._mk_version("Beryllium")[1]
with self.assertRaises(CommandExecutionError):
depr(self.old_function)()
self.assertEqual(self.messages,
['The lifetime of the function "old_function" expired.'])
def test_is_deprecated_with_successor_eol(self):
'''
Use of is_deprecated will result to the exception,
if the expiration version is lower than the current version.
A successor function is pointed out.
:return:
'''
depr = decorators.is_deprecated(self.globs, "Helium", with_successor="new_function")
depr._curr_version = self._mk_version("Beryllium")[1]
with self.assertRaises(CommandExecutionError):
depr(self.old_function)()
self.assertEqual(self.messages,
['The lifetime of the function "old_function" expired. '
'Please use its successor "new_function" instead.'])
def test_is_deprecated(self):
'''
Use of is_deprecated will result to the log message,
if the expiration version is higher than the current version.
A successor function is not pointed out.
:return:
'''
depr = decorators.is_deprecated(self.globs, "Beryllium")
depr._curr_version = self._mk_version("Helium")[1]
self.assertEqual(depr(self.old_function)(), self.old_function())
self.assertEqual(self.messages,
['The function "old_function" is deprecated '
'and will expire in version "Beryllium".'])
def test_is_deprecated_with_successor(self):
'''
Use of is_deprecated will result to the log message,
if the expiration version is higher than the current version.
A successor function is pointed out.
:return:
'''
depr = decorators.is_deprecated(self.globs, "Beryllium", with_successor="old_function")
depr._curr_version = self._mk_version("Helium")[1]
self.assertEqual(depr(self.old_function)(), self.old_function())
self.assertEqual(self.messages,
['The function "old_function" is deprecated '
'and will expire in version "Beryllium". '
'Use successor "old_function" instead.'])
def test_with_deprecated_notfound(self):
'''
Test with_deprecated should raise an exception, if a same name
function with the "_" prefix not implemented.
:return:
'''
self.globs['__opts__']['use_deprecated'] = ['test.new_function']
depr = decorators.with_deprecated(self.globs, "Beryllium")
depr._curr_version = self._mk_version("Helium")[1]
with self.assertRaises(CommandExecutionError):
depr(self.new_function)()
self.assertEqual(self.messages,
['The function is using its deprecated version and will expire in version "Beryllium". '
'Use its successor "new_function" instead.'])
def test_with_deprecated_found(self):
'''
Test with_deprecated should not raise an exception, if a same name
function with the "_" prefix is implemented, but should use
an old version instead, if "use_deprecated" is requested.
:return:
'''
self.globs['__opts__']['use_deprecated'] = ['test.new_function']
self.globs['_new_function'] = self.old_function
depr = decorators.with_deprecated(self.globs, "Beryllium")
depr._curr_version = self._mk_version("Helium")[1]
self.assertEqual(depr(self.new_function)(), self.old_function())
log_msg = ['The function is using its deprecated version and will expire in version "Beryllium". '
'Use its successor "new_function" instead.']
self.assertEqual(self.messages, log_msg)
def test_with_deprecated_found_eol(self):
'''
Test with_deprecated should raise an exception, if a same name
function with the "_" prefix is implemented, "use_deprecated" is requested
and EOL is reached.
:return:
'''
self.globs['__opts__']['use_deprecated'] = ['test.new_function']
self.globs['_new_function'] = self.old_function
depr = decorators.with_deprecated(self.globs, "Helium")
depr._curr_version = self._mk_version("Beryllium")[1]
with self.assertRaises(CommandExecutionError):
depr(self.new_function)()
self.assertEqual(self.messages,
['Although function "new_function" is called, an alias "new_function" '
'is configured as its deprecated version. The lifetime of the function '
'"new_function" expired. Please use its successor "new_function" instead.'])
def test_with_deprecated_no_conf(self):
'''
Test with_deprecated should not raise an exception, if a same name
function with the "_" prefix is implemented, but should use
a new version instead, if "use_deprecated" is not requested.
:return:
'''
self.globs['_new_function'] = self.old_function
depr = decorators.with_deprecated(self.globs, "Beryllium")
depr._curr_version = self._mk_version("Helium")[1]
self.assertEqual(depr(self.new_function)(), self.new_function())
self.assertFalse(self.messages)
def test_with_deprecated_with_name(self):
'''
Test with_deprecated should not raise an exception, if a different name
function is implemented and specified with the "with_name" parameter,
but should use an old version instead and log a warning log message.
:return:
'''
self.globs['__opts__']['use_deprecated'] = ['test.new_function']
depr = decorators.with_deprecated(self.globs, "Beryllium", with_name="old_function")
depr._curr_version = self._mk_version("Helium")[1]
self.assertEqual(depr(self.new_function)(), self.old_function())
self.assertEqual(self.messages,
['The function "old_function" is deprecated and will expire in version "Beryllium". '
'Use its successor "new_function" instead.'])
def test_with_deprecated_with_name_eol(self):
'''
Test with_deprecated should raise an exception, if a different name
function is implemented and specified with the "with_name" parameter
and EOL is reached.
:return:
'''
self.globs['__opts__']['use_deprecated'] = ['test.new_function']
depr = decorators.with_deprecated(self.globs, "Helium", with_name="old_function")
depr._curr_version = self._mk_version("Beryllium")[1]
with self.assertRaises(CommandExecutionError):
depr(self.new_function)()
self.assertEqual(self.messages,
['Although function "new_function" is called, '
'an alias "old_function" is configured as its deprecated version. '
'The lifetime of the function "old_function" expired. '
'Please use its successor "new_function" instead.'])
if __name__ == '__main__':
from integration import run_tests
run_tests(DecoratorsTest, needs_daemon=False)