saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge branch '2016.3' into '2016.11'

Browse source 
Conflicts:
  - salt/minion.py
This commit is contained in:
rallytime 2017-03-09 16:05:15 -07:00
commit c8b5d390b5
16 changed files with 185 additions and 63 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

43
doc/ref/configuration/minion.rst
View file

@ -40,6 +40,33 @@ Default: ``salt``
master: salt
master:port Syntax
~~~~~~~~~~~~~~~~~~
.. versionadded:: 2015.8.0
The ``master`` config option can also be set to use the master's IP in
conjunction with a port number by default.
.. code-block:: yaml
master: localhost:1234
For IPv6 formatting with a port, remember to add brackets around the IP address
before adding the port and enclose the line in single quotes to make it a string:
.. code-block:: yaml
master: '[2001:db8:85a3:8d3:1319:8a2e:370:7348]:1234'
.. note::
If a port is specified in the ``master`` as well as :conf_minion:`master_port`,
the ``master_port`` setting will be overridden by the ``master`` configuration.
List of Masters Syntax
~~~~~~~~~~~~~~~~~~~~~~
The option can can also be set to a list of masters, enabling
:ref:`multi-master <tutorial-multi-master>` mode.
@ -90,6 +117,22 @@ will try to automatically detect IPv6 connectivity to master.
ipv6: True
.. conf_minion:: master_uri_format
``master_uri_format``
---------------------
.. versionadded:: 2015.8.0
Specify the format in which the master address will be evaluated. Valid options
are ``default`` or ``ip_only``. If ``ip_only`` is specified, then the master
address will not be split into IP and PORT, so be sure that only an IP (or domain
name) is set in the :conf_minion:`master` configuration setting.
.. code-block:: yaml
master_uri_format: ip_only
.. conf_minion:: master_type
``master_type``

44
doc/ref/states/index.rst
View file

@ -147,16 +147,17 @@ watched file updated.
The Top File
````````````
The top file controls the mapping between minions and the states which should be
applied to them.
The top file controls the mapping between minions and the states which should
be applied to them.
The top file specifies which minions should have which SLS files applied and which
environments they should draw those SLS files from.
The top file specifies which minions should have which SLS files applied and
which environments they should draw those SLS files from.
The top file works by specifying environments on the top-level.
Each environment contains globs to match minions. Finally, each glob contains a list of
lists of Salt states to apply to matching minions:
Each environment contains :ref:`target expressions <targeting>` to match
minions. Finally, each target expression contains a list of Salt states to
apply to matching minions:
.. code-block:: yaml
@ -172,12 +173,33 @@ lists of Salt states to apply to matching minions:
This above example uses the base environment which is built into the default
Salt setup.
The base environment has two globs. First, the '*' glob contains a list of
SLS files to apply to all minions.
The base environment has target expressions. The first one matches all minions,
and the SLS files below it apply to all minions.
The second glob contains a regular expression that will match all minions with
an ID matching saltmaster.* and specifies that for those minions, the salt.master
state should be applied.
The second expression is a regular expression that will match all minions
with an ID matching ``saltmaster.*`` and specifies that for those minions, the
salt.master state should be applied. To
.. important::
Since version 2014.7.0, the default matcher (when one is not explicitly
defined as in the second expression in the above example) is the
:ref:`compound <targeting-compound>` matcher. Since this matcher parses
individual words in the expression, minion IDs containing spaces will not
match properly using this matcher. Therefore, if your target expression is
designed to match a minion ID containing spaces, it will be necessary to
specify a different match type (such as ``glob``). For example:
.. code-block:: yaml
base:
'test minion':
- match: glob
- foo
- bar
- baz
A full table of match types available in the top file can be found :ref:`here
<top-file-match-types>`.
.. _reloading-modules:

68
doc/ref/states/top.rst
View file

@ -158,8 +158,8 @@ Our top file references the environments:
- db
As seen above, the top file now declares the three environments and for each,
targets are defined to map globs of minion IDs to state files. For example,
all minions which have an ID beginning with the string ``webserver`` will have the
target expressions are defined to map minions to state files. For example, all
minions which have an ID beginning with the string ``webserver`` will have the
webserver state from the requested environment assigned to it.
In this manner, a proposed change to a state could first be made in a state
@ -219,12 +219,51 @@ also available:
Advanced Minion Targeting
=========================
In addition to globs, minions can be specified in top files a few other
ways. Some common ones are :ref:`compound matches <targeting-compound>`
and :ref:`node groups <targeting-nodegroups>`.
In the examples above, notice that all of the target expressions are globs. The
default match type in top files (since version 2014.7.0) is actually the
:ref:`compound matcher <targeting-compound>`, not the glob matcher as in the
CLI.
Below is a slightly more complex top file example, showing the different types
of matches you can perform:
A single glob, when passed through the compound matcher, acts the same way as
matching by glob, so in most cases the two are indistinguishable. However,
there is an edge case in which a minion ID contains whitespace. While it is not
recommended to include spaces in a minion ID, Salt will not stop you from doing
so. However, since compound expressions are parsed word-by-word, if a minion ID
contains spaces it will fail to match. In this edge case, it will be necessary
to explicitly use the ``glob`` matcher:
.. code-block:: yaml
base:
'minion 1':
- match: glob
- foo
.. _top-file-match-types:
The available match types which can be set for a target expression in the top
file are:
============ ================================================================================================================
Match Type Description
============ ================================================================================================================
glob Full minion ID or glob expression to match multiple minions (e.g. ``minion123`` or ``minion*``)
pcre Perl-compatible regular expression (PCRE) matching a minion ID (e.g. ``web[0-3].domain.com``)
grain Match a :ref:`grain <grain>`, optionally using globbing (e.g. ``kernel:Linux`` or ``kernel:*BSD``)
grain_pcre Match a :ref:`grain <grain>` using PCRE (e.g. ``kernel:(Free|Open)BSD``)
list Comma-separated list of minions (e.g. ``minion1,minion2,minion3``)
pillar :ref:`Pillar <pillar>` match, optionally using globbing (e.g. ``role:webserver`` or ``role:web*``)
pillar_pcre :ref:`Pillar <pillar>` match using PCRE (e.g. ``role:web(server|proxy)``
pillar_exact :ref:`Pillar <pillar>` match with no globbing or PCRE (e.g. ``role:webserver``)
ipcidr Subnet or IP address (e.g. ``172.17.0.0/16`` or ``10.2.9.80``)
data Match values kept in the minion's datastore (created using the :mod:`data <salt.modules.data>` execution module)
range :ref:`Range <targeting-range>` cluster
compound Complex expression combining multiple match types (see :ref:`here <targeting-compound>`)
nodegroup Pre-defined compound expressions in the master config file (see :ref:`here <targeting-nodegroups>`)
============ ================================================================================================================
Below is a slightly more complex top file example, showing some of the above
match types:
.. code-block:: yaml
@ -232,6 +271,13 @@ of matches you can perform:
# environment in the ``file_roots`` configuration value.
base:
# All minions which begin with the strings 'nag1' or any minion with
# a grain set called 'role' with the value of 'monitoring' will have
# the 'server.sls' state file applied from the 'nagios/' directory.
'nag1* or G@role:monitoring':
- nagios.server
# All minions get the following three state files applied
'*':
@ -296,14 +342,6 @@ of matches you can perform:
- match: pillar
- xyz
# All minions which begin with the strings 'nag1' or any minion with
# a grain set called 'role' with the value of 'monitoring' will have
# the 'server.sls' state file applied from the 'nagios/' directory.
'nag1* or G@role:monitoring':
- match: compound
- nagios.server
How Top Files Are Compiled
==========================

2
doc/topics/beacons/index.rst
View file

@ -221,7 +221,7 @@ Add the following to ``/srv/reactor/revert.sls``:
.. note::
The expression ``{{ data['data']['id] }}`` :ref:`is correct
The expression ``{{ data['data']['id'] }}`` :ref:`is correct
<beacons-and-reactors>` as it matches the event structure :ref:`shown above
<beacon-event-bus>`.

6
doc/topics/targeting/ipcidr.rst
View file

@ -10,7 +10,7 @@ notation).
.. code-block:: bash
salt -S 192.168.40.20 test.ping
salt -S 10.0.0.0/24 test.ping
salt -S 2001:db8::/64 test.ping
Ipcidr matching can also be used in compound matches
@ -27,7 +27,3 @@ It is also possible to use in both pillar and state-matching
- internal
.. _CIDR: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
.. note::
Only IPv4 matching is supported at this time.

2
doc/topics/targeting/range.rst
View file

@ -1,4 +1,4 @@
.. _targeting_range:
.. _targeting-range:
==========
SECO Range

9
salt/auth/yubico.py
View file

@ -78,15 +78,16 @@ def auth(username, password):
client = Yubico(_cred['id'], _cred['key'])
try:
if client.verify(password):
return True
else:
return False
return client.verify(password)
except yubico_exceptions.StatusCodeError as e:
log.info('Unable to verify YubiKey `{0}`'.format(e))
return False
def groups(username, *args, **kwargs):
return False
if __name__ == '__main__':
__opts__ = {'yubico_users': {'damian': {'id': '12345', 'key': 'ABC123'}}}

11
salt/cloud/clouds/ec2.py
View file

@ -2719,11 +2719,7 @@ def queue_instances(instances):
'''
for instance_id in instances:
node = _get_node(instance_id=instance_id)
for name in node:
if instance_id == node[name]['instanceId']:
__utils__['cloud.cache_node'](node[name],
__active_provider_name__,
__opts__)
__utils__['cloud.cache_node'](node, __active_provider_name__, __opts__)
def create_attach_volumes(name, kwargs, call=None, wait_to_finish=True):
@ -3274,10 +3270,7 @@ def show_instance(name=None, instance_id=None, call=None, kwargs=None):
)
node = _get_node(name=name, instance_id=instance_id)
for name in node:
__utils__['cloud.cache_node'](node[name],
__active_provider_name__,
__opts__)
__utils__['cloud.cache_node'](node, __active_provider_name__, __opts__)
return node

2
salt/cloud/clouds/msazure.py
View file

@ -1600,7 +1600,7 @@ def cleanup_unattached_disks(kwargs=None, conn=None, call=None):
for disk in disks:
if disks[disk]['attached_to'] is None:
del_kwargs = {
'name': disks[disk]['name'][0],
'name': disks[disk]['name'],
'delete_vhd': kwargs.get('delete_vhd', False)
}
log.info('Deleting disk {name}, deleting VHD: {delete_vhd}'.format(**del_kwargs))

22
salt/minion.py
View file

@ -88,6 +88,7 @@ import salt.utils.jid
import salt.pillar
import salt.utils.args
import salt.utils.event
import salt.utils.network
import salt.utils.minion
import salt.utils.minions
import salt.utils.schedule
@ -179,9 +180,11 @@ def resolve_dns(opts, fallback=True, connect=True):
if master == '':
master = unknown_str
if opts.get('__role') == 'syndic':
err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. Set \'syndic_master\' value in minion config.'.format(master)
err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. ' \
'Set \'syndic_master\' value in minion config.'.format(master)
else:
err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. Set \'master\' value in minion config.'.format(master)
err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. ' \
'Set \'master\' value in minion config.'.format(master)
log.error(err)
raise SaltSystemExit(code=42, msg=err)
else:
@ -199,7 +202,10 @@ def resolve_dns(opts, fallback=True, connect=True):
def prep_ip_port(opts):
ret = {}
if opts['master_uri_format'] == 'ip_only':
# Use given master IP if "ip_only" is set or if master_ip is an ipv6 address without
# a port specified. The is_ipv6 check returns False if brackets are used in the IP
# definition such as master: '[::1]:1234'.
if opts['master_uri_format'] == 'ip_only' or salt.utils.network.is_ipv6(opts['master']):
ret['master'] = opts['master']
else:
ip_port = opts['master'].rsplit(":", 1)
@ -209,9 +215,13 @@ def prep_ip_port(opts):
else:
# e.g. master: localhost:1234
# e.g. master: 127.0.0.1:1234
# e.g. master: ::1:1234
ret['master'] = ip_port[0]
ret['master_port'] = ip_port[1]
# e.g. master: [::1]:1234
# Strip off brackets for ipv6 support
ret['master'] = ip_port[0].strip('[]')
# Cast port back to an int! Otherwise a TypeError is thrown
# on some of the socket calls elsewhere in the minion and utils code.
ret['master_port'] = int(ip_port[1])
return ret

8
salt/modules/boto_vpc.py
View file

@ -2599,7 +2599,13 @@ def _maybe_set_name_tag(name, obj):
def _maybe_set_tags(tags, obj):
if tags:
obj.add_tags(tags)
# Not all objects in Boto have an 'add_tags()' method.
try:
obj.add_tags(tags)
except AttributeError:
for tag, value in tags.items():
obj.add_tag(tag, value)
log.debug('The following tags: {0} were added to {1}'.format(', '.join(tags), obj))

5
salt/modules/grains.py
View file

@ -415,7 +415,7 @@ def remove(key, val, delimiter=DEFAULT_TARGET_DELIM):
return setval(key, grains)
def delval(key, destructive=False, refresh=True):
def delval(key, destructive=False):
'''
.. versionadded:: 0.17.0
@ -427,9 +427,6 @@ def delval(key, destructive=False, refresh=True):
destructive
Delete the key, too. Defaults to False.
refresh
Refresh modules and pillar after removing the grain.
CLI Example:
.. code-block:: bash

12
salt/transport/zeromq.py
View file

@ -28,6 +28,7 @@ import salt.transport.mixins.auth
from salt.exceptions import SaltReqTimeoutError
import zmq
import zmq.error
import zmq.eventloop.ioloop
# support pyzmq 13.0.x, TODO: remove once we force people to 14.0.x
if not hasattr(zmq.eventloop.ioloop, 'ZMQIOLoop'):
@ -1064,9 +1065,14 @@ class ZeroMQSocketMonitor(object):
def start_poll(self):
log.trace("Event monitor start!")
while self._monitor_socket is not None and self._monitor_socket.poll():
msg = self._monitor_socket.recv_multipart()
self.monitor_callback(msg)
try:
while self._monitor_socket is not None and self._monitor_socket.poll():
msg = self._monitor_socket.recv_multipart()
self.monitor_callback(msg)
except (AttributeError, zmq.error.ContextTerminated):
# We cannot log here because we'll get an interrupted system call in trying
# to flush the logging buffer as we terminate
pass
@property
def event_map(self):

6
salt/utils/gitfs.py
View file

@ -82,7 +82,11 @@ except ImportError:
HAS_GITPYTHON = False
try:
import pygit2
# Squelch warning on cent7 due to them upgrading cffi
import warnings
with warnings.catch_warnings():
warnings.simplefilter('ignore')
import pygit2
HAS_PYGIT2 = True
try:
GitError = pygit2.errors.GitError

3
salt/utils/process.py
View file

@ -399,6 +399,8 @@ class ProcessManager(object):
yield gen.sleep(10)
else:
time.sleep(10)
if len(self._process_map) == 0:
break
# OSError is raised if a signal handler is called (SIGTERM) during os.wait
except OSError:
break
@ -416,6 +418,7 @@ class ProcessManager(object):
if self._restart_processes is True:
for pid, mapping in six.iteritems(self._process_map):
if not mapping['Process'].is_alive():
log.trace('Process restart of {0}'.format(pid))
self.restart_process(pid)
def kill_children(self, *args, **kwargs):

5
salt/utils/schedule.py
View file

@ -745,7 +745,10 @@ class Schedule(object):
# This also needed for ZeroMQ transport to reset all functions
# context data that could keep paretns connections. ZeroMQ will
# hang on polling parents connections from the child process.
self.functions = salt.loader.minion_mods(self.opts)
if self.opts['__role'] == 'master':
self.functions = salt.loader.runner(self.opts)
else:
self.functions = salt.loader.minion_mods(self.opts)
self.returners = salt.loader.returners(self.opts, self.functions)
ret = {'id': self.opts.get('id', 'master'),
'fun': func,