saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Adding some dictupdate merge tests

Browse source 
Add new option for top_file_merging_strategy to config

Add master config file option

Adding additional tests

Lexicographical test works

Make default strategy

Simplify to just an option about whether or not to merge

Changes

Implement ordered top files

Implement merge strategies and default top files

Lint

Doc rewrite and start on ordering

Remove debugging

Moving to env_order

Adding env_order

Still writing tests

More tests and docs
This commit is contained in:
Mike Place 2015-06-15 11:30:14 -06:00 • committed by rallytime
parent 1c31722b49
commit cfb6982227
10 changed files with 444 additions and 249 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
conf/master
View file

@ -418,6 +418,18 @@
#file_roots:
# base:
# - /srv/salt
#
# When using multiple environments, each with their own top file, the
# default behaviour is an unordered merge. To prevent top files from
# being merged together and instead to only use the top file from the
# requested environment, set this value to False
# top_file_merge: True
# To specify the order in which environments are merged, set the ordering
# in the env_order option. Given a conflict, the last matching value will
# win.
# env_order: ['base', 'dev', 'prod']
# The hash_type is the hash to use when discovering the hash of a file on
# the master server. The default is md5, but sha1, sha224, sha256, sha384

466
doc/ref/states/top.rst
View file

@ -4,33 +4,87 @@
The Top File
============
The top file (top.sls) is used to map what SLS modules get loaded onto what
minions via the state system. The top file creates a few general abstractions.
First it maps what nodes should pull from which environments, next it defines
which matches systems should draw from.
Introduction
============
Most infrastructures are made up of groups of machines, each machine in the
group performing a role similar to others. Those groups of machines work
in concert with each other to create an application stack.
To effectively manage those groups of machines, an administrator needs to
be able to create roles for those groups. For example, a group of machines
that serve front-end web traffic might have a roles which indicate that
those machines should all have the Apache webserver package installed and
that the Apache service should always be running.
In Salt, the file which contains a mapping between groups of machines on a
network and the configuration roles that should be applied to them is
called a `top file`.
Top files are always named `top.sls` and they are so-named because they
always exist in the "top" of a directory hiearchy that contains state files.
That directory hiearchy is called a `state tree`.
A Basic Example
===============
Top files have three components:
- Environment: A state tree directory containing a set of state files to
configure systems.
- Target: A grouping of machines which will have a set of states applied
to them.
- State files: A list of state files to apply to a target. Each state
file describes one or more states to be configured and enforced
on the targeted machines.
The relationship between these three components is nested as follows:
- Environments contain targets
- Targets contain states
Putting these concepts together, we can describe a scenario in which all
minions with an ID that begins with `web` have an `apache` state applied
to them:
.. code-block:: yaml
base: # Apply SLS files from the directory root for the 'base' environment
'web*': # All minions with a minion_id that begins with 'web'
- apache # Apply the state file named 'apache.sls'
.. _states-top-environments:
Environments
============
Environments allow conceptually organizing state tree directories. Environments
can be made to be self-contained or state trees can be made to bleed through
environments.
Environments are directory hiearchies which contain a top files and a set
of state files.
.. note::
Environments can be used in many ways, however there is no requirement that
they be used at all. In fact, the most common way to deploy Salt is with
a single environment, called `base`. It is recommended that users only
create multiple environments if they have a use case which specifically
calls for multiple versions of state trees.
Environments in Salt are very flexible. This section defines how the top
file can be used to define what states from what environments are to be
used for specific minions.
If the intent is to bind minions to specific environments, then the
`environment` option can be set in the minion configuration file.
Getting Started with Top Files
==============================
Each environment is defined inside a salt master configuration variable
called, :conf_master:`file_roots` .
In the most common single-environment setup, only the ``base`` environment is
defined in :conf_master:`file_roots` along with only one directory path for
the state tree.
The environments in the top file corresponds with the environments defined in
the :conf_master:`file_roots` variable. In a simple, single environment setup
you only have the ``base`` environment, and therefore only one state tree. Here
is a simple example of :conf_master:`file_roots` in the master configuration:
.. code-block:: yaml
@ -38,8 +92,13 @@ is a simple example of :conf_master:`file_roots` in the master configuration:
base:
- /srv/salt
This means that the top file will only have one environment to pull from,
here is a simple, single environment top file:
In the above example, the top file will only have a single environment to pull
from.
Next is a simple single-environment top file placed in `/srv/salt/top.sls`,
illustrating that for the environment called `base`, all minions will have the
state files named `core.sls` and `edit.sls` applied to them.
.. code-block:: yaml
@ -48,15 +107,26 @@ here is a simple, single environment top file:
- core
- edit
This also means that :file:`/srv/salt` has a state tree. But if you want to use
multiple environments, or partition the file server to serve more than
just the state tree, then the :conf_master:`file_roots` option can be expanded:
Assuming the `file_roots` configuration from above, Salt will look in the
`/srv/salt` directory for `core.sls` and `edit.sls`.
Multiple Environments
=====================
In some cases, teams may wish to create versioned state trees which can be
used to test Salt configurations in isolated sets of systems such as a staging
environment before deploying states into production.
For this case, multiple environments can be used to accomplish this task.
To create multiple environments, the :conf_master:`file_roots` option can be
expanded:
.. code-block:: yaml
file_roots:
base:
- /srv/salt/base
dev:
- /srv/salt/dev
qa:
@ -64,304 +134,224 @@ just the state tree, then the :conf_master:`file_roots` option can be expanded:
prod:
- /srv/salt/prod
Then our top file could reference the environments:
In the above, we declare three environments: `dev`, `qa` and `prod`.
Each environment has a single directory assigned to it.
Our top file references the environments:
.. code-block:: yaml
dev:
'webserver*dev*':
'webserver*':
- webserver
'db*dev*':
'db*':
- db
qa:
'webserver*qa*':
'webserver*':
- webserver
'db*qa*':
'db*':
- db
prod:
'webserver*prod*':
'webserver*':
- webserver
'db*prod*':
'db*':
- db
In this setup we have state trees in three of the four environments, and no
state tree in the ``base`` environment. Notice that the targets for the minions
specify environment data. In Salt the master determines who is in what
environment, and many environments can be crossed together. For instance, a
separate global state tree could be added to the ``base`` environment if it
suits your deployment:
As seen above, the top file now declares the three enviornments and for each,
targets are defined to map globs of minion IDs to state files. For example,
all minions which have an ID beginning with the string `webserver` will have the
webserver state from the requested environment assigned to it.
.. code-block:: yaml
In this manner, a proposed change to a state could first be made in a state
file in `/srv/salt/dev` and the applied to development webservers before moving
the state into QA by copying the state file into `/srv/salt/qa`.
base:
'*':
- global
dev:
'webserver*dev*':
- webserver
'db*dev*':
- db
qa:
'webserver*qa*':
- webserver
'db*qa*':
- db
prod:
'webserver*prod*':
- webserver
'db*prod*':
- db
In this setup all systems will pull the global SLS from the base environment,
as well as pull from their respective environments. If you assign only one SLS
to a system, as in this example, a shorthand is also available:
Choosing an Environment to Target
=================================
Minions may be pinned to a particular environment by setting the `environment`
value in the minion configuration file. In doing so, a minion will only
request files from the environment to which it is assigned.
The environment to use may also be dynamically selected at the time that
a `salt`, `salt-call` or `salt-ssh` by passing passing a flag to the
execution module being called. This is most commonly done with
functions in the `state` module by using the `saltenv=` argument. For
example, to run a `highstate` on all minions, using the state files in
the `prod` state tree, run: `salt '*' state.highstate saltenv=prod`.
.. note::
Not all functions accept `saltenv` as an argument See individual
function documentation to verify.
Shorthand
=========
If you assign only one SLS to a system, as in this example, a shorthand is
also available:
.. code-block:: yaml
base:
'*': global
dev:
'webserver*dev*': webserver
'db*dev*': db
'webserver*': webserver
'db*': db
qa:
'webserver*qa*': webserver
'db*qa*': db
'webserver*': webserver
'db*': db
prod:
'webserver*prod*': webserver
'db*prod*': db
'webserver*': webserver
'db*': db
.. note::
The top files from all defined environments will be compiled into a single
top file for all states. Top files are environment agnostic.
Remember, that since everything is a file in Salt, the environments are
primarily file server environments, this means that environments that have
nothing to do with states can be defined and used to distribute other files.
.. _states-top-file_roots:
A clean and recommended setup for multiple environments would look like this:
.. code-block:: yaml
# Master file_roots configuration:
file_roots:
base:
- /srv/salt/base
dev:
- /srv/salt/dev
qa:
- /srv/salt/qa
prod:
- /srv/salt/prod
Then only place state trees in the dev, qa, and prod environments, leaving
the base environment open for generic file transfers. Then the top.sls file
would look something like this:
.. code-block:: yaml
dev:
'webserver*dev*':
- webserver
'db*dev*':
- db
qa:
'webserver*qa*':
- webserver
'db*qa*':
- db
prod:
'webserver*prod*':
- webserver
'db*prod*':
- db
Other Ways of Targeting Minions
===============================
Advanced Minion Targeting
=========================
In addition to globs, minions can be specified in top files a few other
ways. Some common ones are :doc:`compound matches </topics/targeting/compound>`
and :doc:`node groups </topics/targeting/nodegroups>`.
Here is a slightly more complex top file example, showing the different types
Below is a slightly more complex top file example, showing the different types
of matches you can perform:
.. code-block:: yaml
# All files will be taken from the file path specified in the base
# environment in the `file_roots` configuration value.
base:
# All minions get the following three state files applied
'*':
- ldap-client
- networking
- salt.minion
# All minions which have an ID that begins with the phrase
# 'salt-master' will have an SLS file applied that is named
# 'master.sls' and is in the 'salt' directory, underneath
# the root specified in the `base` environment in the
# configuration value for `file_roots`.
'salt-master*':
- salt.master
# Minions that have an ID matching the following regular
# expression will have the state file called 'web.sls' in the
# nagios/mon directory applied. Additionally, minions matching
# the regular expression will also have the 'server.sls' file
# in the apache/ directory applied.
# NOTE!
#
# Take note of the 'match' directive here, which tells Salt
# to treat the target string as a regex to be matched!
'^(memcache|web).(qa|prod).loc$':
- match: pcre
- nagios.mon.web
- apache.server
# Minions that have a grain set indicating that they are running
# the Ubuntu operating system will have the state file called
# 'ubuntu.sls' in the 'repos' directory applied.
#
# Again take note of the 'match' directive here which tells
# Salt to match against a grain instead of a minion ID.
'os:Ubuntu':
- match: grain
- repos.ubuntu
# Minions that are either RedHat or CentOS should have the 'epel.sls'
# state applied, from the 'repos/' directory.
'os:(RedHat|CentOS)':
- match: grain_pcre
- repos.epel
# The three minions with the IDs of 'foo', 'bar' and 'baz' should
# have 'database.sls' applied.
'foo,bar,baz':
- match: list
- database
# Any minion for which the pillar key 'somekey' is set and has a value
# of that key matching 'abc' will have the 'xyz.sls' state applied.
'somekey:abc':
- match: pillar
- xyz
# All minions which begin with the strings 'nag1' or any minion with
# a grain set called 'role' with the value of 'monitoring' will have
# the 'server.sls' state file applied from the 'nagios/' directory.
'nag1* or G@role:monitoring':
- match: compound
- nagios.server
In this example ``top.sls``, all minions get the ldap-client, networking, and
salt.minion states. Any minion with an id matching the ``salt-master*`` glob
will get the salt.master state. Any minion with ids matching the regular
expression ``^(memcache|web).(qa|prod).loc$`` will get the nagios.mon.web and
apache.server states. All Ubuntu minions will receive the repos.ubuntu state,
while all RHEL and CentOS minions will receive the repos.epel state. The
minions ``foo``, ``bar``, and ``baz`` will receive the database state. Any
minion with a pillar named ``somekey``, having a value of ``abc`` will receive
the xyz state. Finally, minions with ids matching the nag1* glob or with a
grain named ``role`` equal to ``monitoring`` will receive the nagios.server
state.
How Top Files Are Compiled
==========================
.. warning::
When using multiple enviornments, it is not necessary to create a top file for
each environment. The most common approach, and the easiest to maintain, is
to use a single top file placed in only one environment.
There is currently a known issue with the topfile compilation. The below
may not be completely valid until
https://github.com/saltstack/salt/issues/12483#issuecomment-64181598
is closed.
However, some workflows do call for multiple top files. In this case, top
files may be merged together to create `high data` for the state compiler
to use as a source to compile states on a minion.
As mentioned earlier, the top files in the different environments are compiled
into a single set of data. The way in which this is done follows a few rules,
which are important to understand when arranging top files in different
environments. The examples below all assume that the :conf_master:`file_roots`
are set as in the :ref:`above multi-environment example
<states-top-file_roots>`.
For the following discussion of top file compliation, assume the following
configuration:
1. The ``base`` environment's top file is processed first. Any environment which
is defined in the ``base`` top.sls as well as another environment's top file,
will use the instance of the environment configured in ``base`` and ignore
all other instances. In other words, the ``base`` top file is
authoritative when defining environments. Therefore, in the example below,
the ``dev`` section in ``/srv/salt/dev/top.sls`` would be completely
ignored.
``/srv/salt/base/top.sls:``
``/etc/salt/master``
.. code-block:: yaml
<snip>
file_roots:
first_env:
- /srv/salt/first
second_env:
- /srv/salt/second
base:
``/srv/salt/first/top.sls:``
.. code-block:: yaml
first_env:
'*':
- common
dev:
'webserver*dev*':
- webserver
'db*dev*':
- db
- first
second_env:
'*':
- second
``/srv/salt/dev/top.sls:``
The astute reader will ask how the state compiler resolves which should be
an obvious conflict if a minion is not pinned to a particular environment
and if no environment argument is passed into a state function.
.. code-block:: yaml
Given the above, it is initally unclear whether `first.sls` will be applied
or whether `second.sls` will be applied in a `salt '*' state.highstate` command.
dev:
'10.10.100.0/24':
- match: ipcidr
- deployments.dev.site1
'10.10.101.0/24':
- match: ipcidr
- deployments.dev.site2
When conflicting keys arise, there are several configuration options which
control the behaviour of salt:
.. note::
The rules below assume that the environments being discussed were not
defined in the ``base`` top file.
- `env_order`
Setting `env_order` will set the order in which environments are processed
by the state compiler.
2. If, for some reason, the ``base`` environment is not configured in the
``base`` environment's top file, then the other environments will be checked
in alphabetical order. The first top file found to contain a section for the
``base`` environment wins, and the other top files' ``base`` sections are
ignored. So, provided there is no ``base`` section in the ``base`` top file,
with the below two top files the ``dev`` environment would win out, and the
``common.centos`` SLS would not be applied to CentOS hosts.
- `top_file_merging_strategy`
Can be set to `same`, which will process only the top file from the environment
that the minion belongs to via the `environment` configuration setting or
the environment that is requested via the `saltenv` argument supported
by some functions in the `state` module.
``/srv/salt/dev/top.sls:``
Can also be set to `merge`. This is the default. When set to `merge`,
top files will be merged together. The order in which top files are
merged together can be controlled with `env_order`.
.. code-block:: yaml
base:
'os:Ubuntu':
- common.ubuntu
dev:
'webserver*dev*':
- webserver
'db*dev*':
- db
``/srv/salt/qa/top.sls:``
.. code-block:: yaml
base:
'os:Ubuntu':
- common.ubuntu
'os:CentOS':
- common.centos
qa:
'webserver*qa*':
- webserver
'db*qa*':
- db
3. For environments other than ``base``, the top file in a given environment
will be checked for a section matching the environment's name. If one is
found, then it is used. Otherwise, the remaining (non-``base``) environments
will be checked in alphabetical order. In the below example, the ``qa``
section in ``/srv/salt/dev/top.sls`` will be ignored, but if
``/srv/salt/qa/top.sls`` were cleared or removed, then the states configured
for the ``qa`` environment in ``/srv/salt/dev/top.sls`` will be applied.
``/srv/salt/dev/top.sls:``
.. code-block:: yaml
dev:
'webserver*dev*':
- webserver
'db*dev*':
- db
qa:
'10.10.200.0/24':
- match: ipcidr
- deployments.qa.site1
'10.10.201.0/24':
- match: ipcidr
- deployments.qa.site2
``/srv/salt/qa/top.sls:``
.. code-block:: yaml
qa:
'webserver*qa*':
- webserver
'db*qa*':
- db
.. note::
When in doubt, the simplest way to configure your states is with a single
top.sls in the ``base`` environment.

29
salt/config.py
View file

@ -480,9 +480,21 @@ VALID_OPTS = {
# Whether or not a copy of the master opts dict should be rendered into minion pillars
'pillar_opts': bool,
'pillar_safe_render_error': bool,
# When creating a pillar, there are several stratigies to choose from when
# encountering duplicate values
'pillar_source_merging_strategy': str,
# The ordering for environment merging
'env_order': list,
'default_top': str,
# Can be 'merge', 'same', 'preferred'
'top_file_merging_strategy': str,
'top_file_base': str,
'ping_on_rotate': bool,
'peer': dict,
'preserve_minion_cache': bool,
@ -758,6 +770,9 @@ DEFAULT_MINION_OPTS = {
'file_roots': {
'base': [salt.syspaths.BASE_FILE_ROOTS_DIR],
},
'default_top': 'base',
'env_order': [],
'top_file_merging_strategy': 'merge',
'fileserver_limit_traversal': False,
'file_recv': False,
'file_recv_max_size': 100,
@ -920,6 +935,9 @@ DEFAULT_MASTER_OPTS = {
'pillar_roots': {
'base': [salt.syspaths.BASE_PILLAR_ROOTS_DIR],
},
'default_top': 'base',
'env_order': [],
'top_file_merging_strategy': 'merge',
'file_client': 'local',
'git_pillar_base': 'master',
'git_pillar_branch': 'master',
@ -1195,9 +1213,12 @@ def _expand_glob_path(file_roots):
'''
unglobbed_path = []
for path in file_roots:
if glob.has_magic(path):
unglobbed_path.extend(glob.glob(path))
else:
try:
if glob.has_magic(path):
unglobbed_path.extend(glob.glob(path))
else:
unglobbed_path.append(path)
except Exception:
unglobbed_path.append(path)
return unglobbed_path

1
salt/daemons/masterapi.py
View file

@ -475,6 +475,7 @@ class RemoteFuncs(object):
if saltenv not in file_roots:
file_roots[saltenv] = []
mopts['file_roots'] = file_roots
mopts['env_order'] = self.opts['env_order']
if load.get('env_only'):
return mopts
mopts['renderer'] = self.opts['renderer']

1
salt/master.py
View file

@ -964,6 +964,7 @@ class AESFuncs(object):
if saltenv not in file_roots:
file_roots[saltenv] = []
mopts['file_roots'] = file_roots
mopts['env_order'] = self.opts['env_order']
if load.get('env_only'):
return mopts
mopts['renderer'] = self.opts['renderer']

2
salt/minion.py
View file

@ -1098,7 +1098,7 @@ class Minion(MinionBase):
)
ret['out'] = 'nested'
except TypeError as exc:
msg = 'Passed invalid arguments: {0}\n{1}'.format(exc, func.__doc__)
msg = 'Passed invalid arguments to {0}: {1}\n{2}'.format(function_name, exc, func.__doc__, )
log.warning(msg, exc_info_on_loglevel=logging.DEBUG)
ret['return'] = msg
ret['out'] = 'nested'

12
salt/modules/state.py
View file

@ -1027,10 +1027,20 @@ def show_top(queue=False, **kwargs):
salt '*' state.show_top
'''
opts = copy.deepcopy(__opts__)
if 'env' in kwargs:
salt.utils.warn_until(
'Boron',
'Passing a salt environment should be done using \'saltenv\' '
'not \'env\'. This functionality will be removed in Salt Boron.'
)
opts['environment'] = kwargs['env']
elif 'saltenv' in kwargs:
opts['environment'] = kwargs['saltenv']
conflict = _check_queue(queue, kwargs)
if conflict is not None:
return conflict
st_ = salt.state.HighState(__opts__)
st_ = salt.state.HighState(opts)
errors = []
top_ = st_.get_top()
errors += st_.verify_tops(top_)

23
salt/state.py
View file

@ -2305,6 +2305,7 @@ class BaseHighState(object):
'state_auto_order',
opts['state_auto_order'])
opts['file_roots'] = mopts['file_roots']
opts['env_order'] = mopts['env_order']
opts['state_events'] = mopts.get('state_events')
opts['state_aggregate'] = mopts.get('state_aggregate', opts.get('state_aggregate', False))
opts['jinja_lstrip_blocks'] = mopts.get('jinja_lstrip_blocks', False)
@ -2315,10 +2316,17 @@ class BaseHighState(object):
'''
Pull the file server environments out of the master options
'''
envs = set(['base'])
envs = ['base']
if 'file_roots' in self.opts:
envs.update(list(self.opts['file_roots']))
return envs.union(set(self.client.envs()))
envs.extend(list(self.opts['file_roots']))
client_envs = self.client.envs()
if self.opts.get('env_order', []):
return self.opts['env_order']
else:
for cenv in client_envs:
if cenv not in envs:
envs.append(cenv)
return envs
def get_tops(self):
'''
@ -2329,6 +2337,13 @@ class BaseHighState(object):
done = DefaultOrderedDict(list)
found = 0 # did we find any contents in the top files?
# Gather initial top files
if self.opts['top_file_merging_strategy'] == 'same' and \
not self.opts['environment']:
if not self.opts['default_top']:
raise SaltRenderError('Top file merge strategy set to same, but no default_top '
'configuration option was set')
self.opts['environment'] = self.opts['default_top']
if self.opts['environment']:
contents = self.client.cache_file(
self.opts['state_top'],
@ -2344,7 +2359,7 @@ class BaseHighState(object):
saltenv=self.opts['environment']
)
]
else:
elif self.opts['top_file_merging_strategy'] == 'merge':
found = 0
if self.opts.get('state_top_saltenv', False):
saltenv = self.opts['state_top_saltenv']

88
tests/unit/highstateconf_test.py
View file

@ -5,6 +5,7 @@ from __future__ import absolute_import
import os
import os.path
import tempfile
import copy
# Import Salt Testing libs
from salttesting import TestCase
@ -16,7 +17,8 @@ ensure_in_syspath('../')
import integration
import salt.config
from salt.state import HighState
from salt.utils.odict import OrderedDict, DefaultOrderedDict
from salttesting.mock import NO_MOCK, NO_MOCK_REASON, Mock, patch
class HighStateTestCase(TestCase):
def setUp(self):
@ -66,6 +68,90 @@ class HighStateTestCase(TestCase):
'state2,state3')
self.assertEqual(matches, {'env': ['state2', 'state3']})
class TopFileMergeTestCase(TestCase):
'''
Test various merge strategies for multiple tops files collected from
multiple environments. Various options correspond to merge strategies
which can be set by the user with the top_file_merging_strategy config
option.
Refs #12483
'''
def setUp(self):
'''
Create multiple top files for use in each test
'''
self.env1 = {'base': {'*': ['e1_a', 'e1_b', 'e1_c']}}
self.env2 = {'base': {'*': ['e2_a', 'e2_b', 'e2_c']}}
self.env3 = {'base': {'*': ['e3_a', 'e3_b', 'e3_c']}}
self.config = self._make_default_config()
self.highstate = HighState(self.config)
def _make_default_config(self):
config = salt.config.minion_config(None)
root_dir = tempfile.mkdtemp(dir=integration.TMP)
state_tree_dir = os.path.join(root_dir, 'state_tree')
cache_dir = os.path.join(root_dir, 'cachedir')
config['root_dir'] = root_dir
config['state_events'] = False
config['id'] = 'match'
config['file_client'] = 'local'
config['file_roots'] = dict(base=[state_tree_dir])
config['cachedir'] = cache_dir
config['test'] = False
return config
def _get_tops(self):
'''
A test helper to emulate HighState.get_tops() but just to construct
an appropriate data structure for top files from multiple environments
'''
tops = DefaultOrderedDict(list)
tops['a'].append(self.env1)
tops['b'].append(self.env2)
tops['c'].append(self.env3)
return tops
def test_basic_merge(self):
'''
This is the default approach for Salt. Merge the top files with the
earlier appends taking precendence. Since Salt does the appends
lexecographically, this is effectively a test against the default
lexecographical behaviour.
'''
merged_tops = self.highstate.merge_tops(self._get_tops())
expected_merge = DefaultOrderedDict(OrderedDict)
expected_merge['base']['*'] = ['e1_c', 'e1_b', 'e1_a']
self.assertEqual(merged_tops, expected_merge)
def test_merge_strategy_same(self):
'''
Test to see if the top file that corresponds
to the requested env is the one that is used
by the state system
'''
config = self._make_default_config()
config['top_file_merging_strategy'] = 'same'
config['environment'] = 'b'
highstate = HighState(config)
ret = highstate.get_tops()
self.assertEqual(ret, OrderedDict([('b', [{}])]) )
def test_ordered_merge(self):
'''
Test to see if the merger respects environment
ordering
'''
config = self._make_default_config()
config['top_file_merging_strategy'] = 'merge'
config['env_order'] = ['b', 'a', 'c']
highstate = HighState(config)
ret = highstate.get_tops()
self.assertEqual(ret, OrderedDict([('b', [{}]), ('a', [{}]), ('c', [{}])]))
if __name__ == '__main__':
from integration import run_tests

59
tests/unit/utils/dictupdate_test.py
View file

@ -68,6 +68,65 @@ class UtilDictupdateTestCase(TestCase):
)
self.assertEqual(res, mdict)
class UtilDictMergeTestCase(TestCase):
dict1 = {'A': 'B', 'C': {'D': 'E', 'F': {'G': 'H', 'I': 'J'}}}
def test_merge_overwrite_traditional(self):
'''
Test traditional overwrite, wherein a key in the second dict overwrites a key in the first
'''
mdict = copy.deepcopy(self.dict1)
mdict['A'] = 'b'
ret = dictupdate.merge_overwrite(copy.deepcopy(self.dict1), {'A': 'b'})
self.assertEqual(mdict, ret)
def test_merge_overwrite_missing_source_key(self):
'''
Test case wherein the overwrite strategy is used but a key in the second dict is
not present in the first
'''
mdict = copy.deepcopy(self.dict1)
mdict['D'] = 'new'
ret = dictupdate.merge_overwrite(copy.deepcopy(self.dict1), {'D': 'new'})
self.assertEqual(mdict, ret)
def test_merge_aggregate_traditional(self):
'''
Test traditional aggregation, where a val from dict2 overwrites one
present in dict1
'''
mdict = copy.deepcopy(self.dict1)
mdict['A'] = 'b'
ret = dictupdate.merge_overwrite(copy.deepcopy(self.dict1), {'A': 'b'})
self.assertEqual(mdict, ret)
def test_merge_list_traditional(self):
'''
Test traditional list merge, where a key present in dict2 will be converted
to a list
'''
mdict = copy.deepcopy(self.dict1)
mdict['A'] = ['B', 'b']
ret = dictupdate.merge_list(copy.deepcopy(self.dict1), {'A': 'b'})
self.assertEqual(mdict, ret)
def test_merge_list_append(self):
'''
This codifies the intended behaviour that items merged into a dict val that is already
a list that those items will *appended* to the list, and not magically merged in
'''
mdict = copy.deepcopy(self.dict1)
mdict['A'] = ['B', 'b', 'c']
# Prepare a modified copy of dict1 that has a list as a val for the key of 'A'
mdict1 = copy.deepcopy(self.dict1)
mdict1['A'] = ['B']
ret = dictupdate.merge_list(mdict1, {'A': ['b', 'c']})
self.assertEqual({'A': [['B'], ['b', 'c']], 'C': {'D': 'E', 'F': {'I': 'J', 'G': 'H'}}}, ret)
if __name__ == '__main__':
from integration import run_tests
run_tests(UtilDictupdateTestCase, needs_daemon=False)