saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-16 17:50:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge pull request #43829 from rallytime/merge-develop

Browse source 
[develop] Merge forward from 2017.7 to develop
This commit is contained in:
Nicole Thomas 2017-10-05 16:09:23 -04:00 • committed by GitHub
commit 7785919437
26 changed files with 3547 additions and 246 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

383
doc/topics/development/contributing.rst
View file

@ -31,7 +31,7 @@ documentation for more information.
.. _github-pull-request:
Sending a GitHub pull request
=============================
-----------------------------
Sending pull requests on GitHub is the preferred method for receiving
contributions. The workflow advice below mirrors `GitHub's own guide <GitHub
@ -66,7 +66,7 @@ Fork a Repo Guide_>`_ and is well worth reading.
.. code-block:: bash
git fetch upstream
git checkout -b fix-broken-thing upstream/2016.3
git checkout -b fix-broken-thing upstream/2016.11
If you're working on a feature, create your branch from the develop branch.
@ -130,7 +130,7 @@ Fork a Repo Guide_>`_ and is well worth reading.
.. code-block:: bash
git fetch upstream
git rebase upstream/2016.3 fix-broken-thing
git rebase upstream/2016.11 fix-broken-thing
git push -u origin fix-broken-thing
or
@ -170,9 +170,9 @@ Fork a Repo Guide_>`_ and is well worth reading.
https://github.com/my-account/salt/pull/new/fix-broken-thing
#. If your branch is a fix for a release branch, choose that as the base
branch (e.g. ``2016.3``),
branch (e.g. ``2016.11``),
https://github.com/my-account/salt/compare/saltstack:2016.3...fix-broken-thing
https://github.com/my-account/salt/compare/saltstack:2016.11...fix-broken-thing
If your branch is a feature, choose ``develop`` as the base branch,
@ -205,80 +205,206 @@ Fork a Repo Guide_>`_ and is well worth reading.
.. _which-salt-branch:
Which Salt branch?
==================
GitHub will open pull requests against Salt's main branch, ``develop``, by
default. Ideally, features should go into ``develop`` and bug fixes and
documentation changes should go into the oldest supported release branch
affected by the bug or documentation update. See
:ref:`Sending a GitHub pull request <github-pull-request>`.
If you have a bug fix or doc change and have already forked your working
branch from ``develop`` and do not know how to rebase your commits against
another branch, then submit it to ``develop`` anyway and we'll be sure to
back-port it to the correct place.
The current release branch
--------------------------
The current release branch is the most recent stable release. Pull requests
containing bug fixes should be made against the release branch.
The branch name will be a date-based name such as ``2016.3``.
Bug fixes are made on this branch so that minor releases can be cut from this
branch without introducing surprises and new features. This approach maximizes
stability.
The Salt development team will "merge-forward" any fixes made on the release
branch to the ``develop`` branch once the pull request has been accepted. This
keeps the fix in isolation on the release branch and also keeps the ``develop``
branch up-to-date.
.. note:: Closing GitHub issues from commits
This "merge-forward" strategy requires that `the magic keywords to close a
GitHub issue <Closing issues via commit message_>`_ appear in the commit
message text directly. Only including the text in a pull request will not
close the issue.
GitHub will close the referenced issue once the *commit* containing the
magic text is merged into the default branch (``develop``). Any magic text
input only into the pull request description will not be seen at the
Git-level when those commits are merged-forward. In other words, only the
commits are merged-forward and not the pull request.
The ``develop`` branch
Salt's Branch Topology
----------------------
There are three different kinds of branches in use: develop, main release
branches, and dot release branches.
- All feature work should go into the ``develop`` branch.
- Bug fixes and documentation changes should go into the oldest supported
**main** release branch affected by the the bug or documentation change.
Main release branches are named after a year and month, such as
``2016.11`` and ``2017.7``.
- Hot fixes, as determined by SaltStack's release team, should be submitted
against **dot** release branches. Dot release branches are named after a
year, month, and version. Examples include ``2016.11.8`` and ``2017.7.2``.
.. note::
GitHub will open pull requests against Salt's main branch, ``develop``,
byndefault. Be sure to check which branch is selected when creating the
pull request.
The Develop Branch
==================
The ``develop`` branch is unstable and bleeding-edge. Pull requests containing
feature additions or non-bug-fix changes should be made against the ``develop``
branch.
The Salt development team will back-port bug fixes made to ``develop`` to the
current release branch if the contributor cannot create the pull request
against that branch.
.. note::
Release Branches
----------------
If you have a bug fix or documentation change and have already forked your
working branch from ``develop`` and do not know how to rebase your commits
against another branch, then submit it to ``develop`` anyway. SaltStack's
development team will be happy to back-port it to the correct branch.
For each release, a branch will be created when the SaltStack release team is
ready to tag. The release branch is created from the parent branch and will be
the same name as the tag minus the ``v``. For example, the ``2017.7.1`` release
branch was created from the ``2017.7`` parent branch and the ``v2017.7.1``
release was tagged at the ``HEAD`` of the ``2017.7.1`` branch. This branching
strategy will allow for more stability when there is a need for a re-tag during
the testing phase of the release process.
**Please make sure you let the maintainers know that the pull request needs
to be back-ported.**
Once the release branch is created, the fixes required for a given release, as
determined by the SaltStack release team, will be added to this branch. All
commits in this branch will be merged forward into the parent branch as well.
Main Release Branches
=====================
The current release branch is the most recent stable release. Pull requests
containing bug fixes or documentation changes should be made against the main
release branch that is affected.
The branch name will be a date-based name such as ``2016.11``.
Bug fixes are made on this branch so that dot release branches can be cut from
the main release branch without introducing surprises and new features. This
approach maximizes stability.
Dot Release Branches
====================
Prior to tagging an official release, a branch will be created when the SaltStack
release team is ready to tag. The dot release branch is created from a main release
branch. The dot release branch will be the same name as the tag minus the ``v``.
For example, the ``2017.7.1`` dot release branch was created from the ``2017.7``
main release branch. The ``v2017.7.1`` release was tagged at the ``HEAD`` of the
``2017.7.1`` branch.
This branching strategy will allow for more stability when there is a need for
a re-tag during the testing phase of the release process and further increases
stability.
Once the dot release branch is created, the fixes required for a given release,
as determined by the SaltStack release team, will be added to this branch. All
commits in this branch will be merged forward into the main release branch as
well.
Merge Forward Process
=====================
The Salt repository follows a "Merge Forward" policy. The merge-forward
behavior means that changes submitted to older main release branches will
automatically be "merged-forward" into the newer branches.
For example, a pull request is merged into ``2016.11``. Then, the entire
``2016.11`` branch is merged-forward into the ``2017.7`` branch, and the
``2017.7`` branch is merged-forward into the ``develop`` branch.
This process makes is easy for contributors to make only one pull-request
against an older branch, but allows the change to propagate to all **main**
release branches.
The merge-forward work-flow applies to all main release branches and the
operation runs continuously.
Merge-Forwards for Dot Release Branches
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The merge-forward policy applies to dot release branches as well, but has a
slightly different behavior. If a change is submitted to a **dot** release
branch, the dot release branch will be merged into its parent **main**
release branch.
For example, a pull request is merged into the ``2017.7.2`` release branch.
Then, the entire ``2017.7.2`` branch is merged-forward into the ``2017.7``
branch. From there, the merge forward process continues as normal.
The only way in which dot release branches differ from main release branches
in regard to merge-forwards, is that once a dot release branch is created
from the main release branch, the dot release branch does not receive merge
forwards.
.. note::
The merge forward process for dot release branches is one-way:
dot release branch --> main release branch.
Closing GitHub issues from commits
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This "merge-forward" strategy requires that `the magic keywords to close a
GitHub issue <Closing issues via commit message_>`_ appear in the commit
message text directly. Only including the text in a pull request will not
close the issue.
GitHub will close the referenced issue once the *commit* containing the
magic text is merged into the default branch (``develop``). Any magic text
input only into the pull request description will not be seen at the
Git-level when those commits are merged-forward. In other words, only the
commits are merged-forward and not the pull request text.
.. _backporting-pull-requests:
Backporting Pull Requests
=========================
If a bug is fixed on ``develop`` and the bug is also present on a
currently-supported release branch, it will need to be back-ported to an
applicable branch.
.. note:: Most Salt contributors can skip these instructions
These instructions do not need to be read in order to contribute to the
Salt project! The SaltStack team will back-port fixes on behalf of
contributors in order to keep the contribution process easy.
These instructions are intended for frequent Salt contributors, advanced
Git users, SaltStack employees, or independent souls who wish to back-port
changes themselves.
It is often easiest to fix a bug on the oldest supported release branch and
then merge that branch forward into ``develop`` (as described earlier in this
document). When that is not possible the fix must be back-ported, or copied,
into any other affected branches.
These steps assume a pull request ``#1234`` has been merged into ``develop``.
And ``upstream`` is the name of the remote pointing to the main Salt repo.
#. Identify the oldest supported release branch that is affected by the bug.
#. Create a new branch for the back-port by reusing the same branch from the
original pull request.
Name the branch ``bp-<NNNN>`` and use the number of the original pull
request.
.. code-block:: bash
git fetch upstream refs/pull/1234/head:bp-1234
git checkout bp-1234
#. Find the parent commit of the original pull request.
The parent commit of the original pull request must be known in order to
rebase onto a release branch. The easiest way to find this is on GitHub.
Open the original pull request on GitHub and find the first commit in the
list of commits. Select and copy the SHA for that commit. The parent of
that commit can be specified by appending ``~1`` to the end.
#. Rebase the new branch on top of the release branch.
* ``<release-branch>`` is the branch identified in step #1.
* ``<orig-base>`` is the SHA identified in step #3 -- don't forget to add
``~1`` to the end!
.. code-block:: bash
git rebase --onto <release-branch> <orig-base> bp-1234
Note, release branches prior to ``2016.11`` will not be able to make use of
rebase and must use cherry-picking instead.
#. Push the back-port branch to GitHub and open a new pull request.
Opening a pull request for the back-port allows for the test suite and
normal code-review process.
.. code-block:: bash
git push -u origin bp-1234
Keeping Salt Forks in Sync
==========================
--------------------------
Salt is advancing quickly. It is therefore critical to pull upstream changes
Salt advances quickly. It is therefore critical to pull upstream changes
from upstream into your fork on a regular basis. Nothing is worse than putting
hard work into a pull request only to see bunches of merge conflicts because it
has diverged too far from upstream.
@ -340,92 +466,53 @@ the name of the main `saltstack/salt`_ repository.
the current release branch.
Posting patches to the mailing list
===================================
-----------------------------------
Patches will also be accepted by email. Format patches using `git
format-patch`_ and send them to the `salt-users`_ mailing list. The contributor
will then get credit for the patch, and the Salt community will have an archive
of the patch and a place for discussion.
.. _backporting-pull-requests:
Backporting Pull Requests
=========================
If a bug is fixed on ``develop`` and the bug is also present on a
currently-supported release branch it will need to be back-ported to all
applicable branches.
.. note:: Most Salt contributors can skip these instructions
These instructions do not need to be read in order to contribute to the
Salt project! The SaltStack team will back-port fixes on behalf of
contributors in order to keep the contribution process easy.
These instructions are intended for frequent Salt contributors, advanced
Git users, SaltStack employees, or independent souls who wish to back-port
changes themselves.
It is often easiest to fix a bug on the oldest supported release branch and
then merge that branch forward into ``develop`` (as described earlier in this
document). When that is not possible the fix must be back-ported, or copied,
into any other affected branches.
These steps assume a pull request ``#1234`` has been merged into ``develop``.
And ``upstream`` is the name of the remote pointing to the main Salt repo.
1. Identify the oldest supported release branch that is affected by the bug.
2. Create a new branch for the back-port by reusing the same branch from the
original pull request.
Name the branch ``bp-<NNNN>`` and use the number of the original pull
request.
.. code-block:: bash
git fetch upstream refs/pull/1234/head:bp-1234
git checkout bp-1234
3. Find the parent commit of the original pull request.
The parent commit of the original pull request must be known in order to
rebase onto a release branch. The easiest way to find this is on GitHub.
Open the original pull request on GitHub and find the first commit in the
list of commits. Select and copy the SHA for that commit. The parent of
that commit can be specified by appending ``~1`` to the end.
4. Rebase the new branch on top of the release branch.
* ``<release-branch>`` is the branch identified in step #1.
* ``<orig-base>`` is the SHA identified in step #3 -- don't forget to add
``~1`` to the end!
.. code-block:: bash
git rebase --onto <release-branch> <orig-base> bp-1234
Note, release branches prior to ``2016.3`` will not be able to make use of
rebase and must use cherry-picking instead.
5. Push the back-port branch to GitHub and open a new pull request.
Opening a pull request for the back-port allows for the test suite and
normal code-review process.
.. code-block:: bash
git push -u origin bp-1234
Issue and Pull Request Labeling System
======================================
--------------------------------------
SaltStack uses several labeling schemes to help facilitate code contributions
and bug resolution. See the :ref:`Labels and Milestones
<labels-and-milestones>` documentation for more information.
Mentionbot
----------
SaltStack runs a mention-bot which notifies contributors who might be able
to help review incoming pull-requests based on their past contribution to
files which are being changed.
If you do not wish to receive these notifications, please add your GitHub
handle to the blacklist line in the ``.mention-bot`` file located in the
root of the Salt repository.
.. _probot-gpg-verification:
GPG Verification
----------------
SaltStack has enabled `GPG Probot`_ to enforce GPG signatures for all
commits included in a Pull Request.
In order for the GPG verification status check to pass, *every* contributor in
the pull request must:
- Set up a GPG key on local machine
- Sign all commits in the pull request with key
- Link key with GitHub account
This applies to all commits in the pull request.
GitHub hosts a number of `help articles`_ for creating a GPG key, using the
GPG key with ``git`` locally, and linking the GPG key to your GitHub account.
Once these steps are completed, the commit signing verification will look like
the example in GitHub's `GPG Signature Verification feature announcement`_.
.. _`saltstack/salt`: https://github.com/saltstack/salt
.. _`GitHub Fork a Repo Guide`: https://help.github.com/articles/fork-a-repo
.. _`GitHub issue tracker`: https://github.com/saltstack/salt/issues
@ -434,14 +521,6 @@ and bug resolution. See the :ref:`Labels and Milestones
.. _`Closing issues via commit message`: https://help.github.com/articles/closing-issues-via-commit-messages
.. _`git format-patch`: https://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html
.. _salt-users: https://groups.google.com/forum/#!forum/salt-users
Mentionbot
==========
SaltStack runs a mention-bot which notifies contributors who might be able
to help review incoming pull-requests based on their past contribution to
files which are being changed.
If you do not wish to receive these notifications, please add your GitHub
handle to the blacklist line in the `.mention-bot` file located in the
root of the Salt repository.
.. _GPG Probot: https://probot.github.io/apps/gpg/
.. _help articles: https://help.github.com/articles/signing-commits-with-gpg/
.. _GPG Signature Verification feature announcement: https://github.com/blog/2144-gpg-signature-verification

3
doc/topics/releases/2017.7.0.rst
View file

@ -21,6 +21,9 @@ Salt will no longer support Python 2.6. We will provide python2.7 packages on ou
.. _repo: https://repo.saltstack.com/
As this will impact the installation of additional dependencies for salt modules please use pip packages if there is not a package available in a repository. You will need to install the python27-pip package to get access to the correct pip27 executable: ``yum install python27-pip``
============
Known Issues
============

3160
doc/topics/releases/2017.7.2.rst Normal file
View file

File diff suppressed because it is too large Load diff

9
salt/cli/batch.py
View file

@ -143,10 +143,11 @@ class Batch(object):
# sure that the main while loop finishes even with unresp minions
minion_tracker = {}
# We already know some minions didn't respond to the ping, so inform
# the user we won't be attempting to run a job on them
for down_minion in self.down_minions:
salt.utils.print_cli('Minion {0} did not respond. No job will be sent.'.format(down_minion))
if not self.quiet:
# We already know some minions didn't respond to the ping, so inform
# the user we won't be attempting to run a job on them
for down_minion in self.down_minions:
salt.utils.print_cli('Minion {0} did not respond. No job will be sent.'.format(down_minion))
# Iterate while we still have things to execute
while len(ret) < len(self.minions):

4
salt/daemons/masterapi.py
View file

@ -236,12 +236,12 @@ def access_keys(opts):
# Check other users matching ACL patterns
if opts['client_acl_verify'] and HAS_PWD:
log.profile('Beginning pwd.getpwall() call in masterarpi access_keys function')
log.profile('Beginning pwd.getpwall() call in masterapi access_keys function')
for user in pwd.getpwall():
user = user.pw_name
if user not in keys and salt.utils.check_whitelist_blacklist(user, whitelist=acl_users):
keys[user] = mk_key(opts, user)
log.profile('End pwd.getpwall() call in masterarpi access_keys function')
log.profile('End pwd.getpwall() call in masterapi access_keys function')
return keys

8
salt/engines/docker_events.py
View file

@ -74,8 +74,12 @@ def start(docker_url='unix://var/run/docker.sock',
else:
__salt__['event.send'](tag, msg)
client = docker.Client(base_url=docker_url,
timeout=timeout)
try:
# docker-py 2.0 renamed this client attribute
client = docker.APIClient(base_url=docker_url, timeout=timeout)
except AttributeError:
client = docker.Client(base_url=docker_url, timeout=timeout)
try:
events = client.events()
for event in events:

10
salt/fileserver/roots.py
View file

@ -351,6 +351,16 @@ def _file_lists(load, form):
'roots: %s symlink destination is %s',
abs_path, link_dest
)
if salt.utils.is_windows() \
and link_dest.startswith('\\\\'):
# Symlink points to a network path. Since you can't
# join UNC and non-UNC paths, just assume the original
# path.
log.trace(
'roots: %s is a UNCH path, using %s instead',
link_dest, abs_path
)
link_dest = abs_path
if link_dest.startswith('..'):
joined = os.path.join(abs_path, link_dest)
else:

19
salt/master.py
View file

@ -59,7 +59,6 @@ import salt.minion
import salt.key
import salt.acl
import salt.engines
import salt.fileserver
import salt.daemons.masterapi
import salt.defaults.exitcodes
import salt.transport.server
@ -183,7 +182,8 @@ class Maintenance(salt.utils.process.SignalHandlingMultiprocessingProcess):
in the parent process, then once the fork happens you'll start getting
errors like "WARNING: Mixing fork() and threads detected; memory leaked."
'''
# Init fileserver manager
# Avoid circular import
import salt.fileserver
self.fileserver = salt.fileserver.Fileserver(self.opts)
# Load Runners
ropts = dict(self.opts)
@ -444,6 +444,8 @@ class Master(SMaster):
)
if self.opts.get(u'fileserver_verify_config', True):
# Avoid circular import
import salt.fileserver
fileserver = salt.fileserver.Fileserver(self.opts)
if not fileserver.servers:
errors.append(
@ -479,16 +481,15 @@ class Master(SMaster):
if git_pillars:
try:
new_opts = copy.deepcopy(self.opts)
from salt.pillar.git_pillar \
import PER_REMOTE_OVERRIDES as per_remote_overrides, \
PER_REMOTE_ONLY as per_remote_only
import salt.pillar.git_pillar
for repo in git_pillars:
new_opts[u'ext_pillar'] = [repo]
try:
git_pillar = salt.utils.gitfs.GitPillar(new_opts)
git_pillar.init_remotes(repo[u'git'],
per_remote_overrides,
per_remote_only)
git_pillar.init_remotes(
repo[u'git'],
salt.pillar.git_pillar.PER_REMOTE_OVERRIDES,
salt.pillar.git_pillar.PER_REMOTE_ONLY)
except FileserverConfigError as exc:
critical_errors.append(exc.strerror)
finally:
@ -956,6 +957,8 @@ class AESFuncs(object):
'''
Set the local file objects from the file server interface
'''
# Avoid circular import
import salt.fileserver
self.fs_ = salt.fileserver.Fileserver(self.opts)
self._serve_file = self.fs_.serve_file
self._file_find = self.fs_._find_file

6
salt/modules/boto_kinesis.py
View file

@ -66,15 +66,17 @@ except ImportError:
log = logging.getLogger(__name__)
__virtualname__ = 'boto_kinesis'
def __virtual__():
'''
Only load if boto3 libraries exist.
'''
if not HAS_BOTO:
return False
return False, 'The boto_kinesis module could not be loaded: boto libraries not found.'
__utils__['boto3.assign_funcs'](__name__, 'kinesis')
return True
return __virtualname__
def _get_basic_stream(stream_name, conn):

9
salt/modules/ini_manage.py
View file

@ -373,10 +373,11 @@ class _Ini(_Section):
with salt.utils.files.fopen(self.name) as rfh:
inicontents = rfh.read()
except (OSError, IOError) as exc:
raise CommandExecutionError(
"Unable to open file '{0}'. "
"Exception: {1}".format(self.name, exc)
)
if __opts__['test'] is False:
raise CommandExecutionError(
"Unable to open file '{0}'. "
"Exception: {1}".format(self.name, exc)
)
if not inicontents:
return
# Remove anything left behind from a previous run.

3
salt/modules/win_repo.py
View file

@ -27,7 +27,8 @@ from salt.exceptions import CommandExecutionError, SaltRenderError
from salt.runners.winrepo import (
genrepo as _genrepo,
update_git_repos as _update_git_repos,
PER_REMOTE_OVERRIDES
PER_REMOTE_OVERRIDES,
PER_REMOTE_ONLY
)
from salt.ext import six
try:

7
salt/pillar/__init__.py
View file

@ -886,9 +886,12 @@ class Pillar(object):
and self.opts.get('__role') != 'minion':
# Avoid circular import
import salt.utils.gitfs
from salt.pillar.git_pillar import PER_REMOTE_OVERRIDES
import salt.pillar.git_pillar
git_pillar = salt.utils.gitfs.GitPillar(self.opts)
git_pillar.init_remotes(self.ext['git'], PER_REMOTE_OVERRIDES)
git_pillar.init_remotes(
self.ext['git'],
salt.pillar.git_pillar.PER_REMOTE_OVERRIDES,
salt.pillar.git_pillar.PER_REMOTE_ONLY)
git_pillar.fetch_remotes()
except TypeError:
# Handle malformed ext_pillar

6
salt/pillar/git_pillar.py
View file

@ -348,6 +348,12 @@ from salt.ext import six
PER_REMOTE_OVERRIDES = ('env', 'root', 'ssl_verify', 'refspecs')
PER_REMOTE_ONLY = ('name', 'mountpoint')
# Fall back to default per-remote-only. This isn't technically needed since
# salt.utils.gitfs.GitBase.init_remotes() will default to
# salt.utils.gitfs.PER_REMOTE_ONLY for this value, so this is mainly for
# runners and other modules that import salt.pillar.git_pillar.
PER_REMOTE_ONLY = salt.utils.gitfs.PER_REMOTE_ONLY
# Set up logging
log = logging.getLogger(__name__)

23
salt/runners/cache.py
View file

@ -18,12 +18,11 @@ import salt.utils.master
import salt.utils.versions
import salt.payload
import salt.cache
import salt.fileserver.gitfs
import salt.pillar.git_pillar
import salt.runners.winrepo
from salt.exceptions import SaltInvocationError
from salt.fileserver import clear_lock as _clear_lock
from salt.fileserver.gitfs import PER_REMOTE_OVERRIDES as __GITFS_OVERRIDES
from salt.pillar.git_pillar \
import PER_REMOTE_OVERRIDES as __GIT_PILLAR_OVERRIDES
from salt.runners.winrepo import PER_REMOTE_OVERRIDES as __WINREPO_OVERRIDES
log = logging.getLogger(__name__)
@ -330,8 +329,10 @@ def clear_git_lock(role, remote=None, **kwargs):
if role == 'gitfs':
git_objects = [salt.utils.gitfs.GitFS(__opts__)]
git_objects[0].init_remotes(__opts__['gitfs_remotes'],
__GITFS_OVERRIDES)
git_objects[0].init_remotes(
__opts__['gitfs_remotes'],
salt.fileserver.gitfs.PER_REMOTE_OVERRIDES,
salt.fileserver.gitfs.PER_REMOTE_ONLY)
elif role == 'git_pillar':
git_objects = []
for ext_pillar in __opts__['ext_pillar']:
@ -340,7 +341,10 @@ def clear_git_lock(role, remote=None, **kwargs):
if not isinstance(ext_pillar['git'], list):
continue
obj = salt.utils.gitfs.GitPillar(__opts__)
obj.init_remotes(ext_pillar['git'], __GIT_PILLAR_OVERRIDES)
obj.init_remotes(
ext_pillar['git'],
salt.pillar.git_pillar.PER_REMOTE_OVERRIDES,
salt.pillar.git_pillar.PER_REMOTE_ONLY)
git_objects.append(obj)
elif role == 'winrepo':
winrepo_dir = __opts__['winrepo_dir']
@ -352,7 +356,10 @@ def clear_git_lock(role, remote=None, **kwargs):
(__opts__['winrepo_remotes_ng'], __opts__['winrepo_dir_ng'])
):
obj = salt.utils.gitfs.WinRepo(__opts__, base_dir)
obj.init_remotes(remotes, __WINREPO_OVERRIDES)
obj.init_remotes(
remotes,
salt.runners.winrepo.PER_REMOTE_OVERRIDES,
salt.runners.winrepo.PER_REMOTE_ONLY)
git_objects.append(obj)
else:
raise SaltInvocationError('Invalid role \'{0}\''.format(role))

9
salt/runners/winrepo.py
View file

@ -31,6 +31,12 @@ log = logging.getLogger(__name__)
# Global parameters which can be overridden on a per-remote basis
PER_REMOTE_OVERRIDES = ('ssl_verify', 'refspecs')
# Fall back to default per-remote-only. This isn't technically needed since
# salt.utils.gitfs.GitBase.init_remotes() will default to
# salt.utils.gitfs.PER_REMOTE_ONLY for this value, so this is mainly for
# runners and other modules that import salt.runners.winrepo.
PER_REMOTE_ONLY = salt.utils.gitfs.PER_REMOTE_ONLY
def genrepo(opts=None, fire_event=True):
'''
@ -211,7 +217,8 @@ def update_git_repos(opts=None, clean=False, masterless=False):
# New winrepo code utilizing salt.utils.gitfs
try:
winrepo = salt.utils.gitfs.WinRepo(opts, base_dir)
winrepo.init_remotes(remotes, PER_REMOTE_OVERRIDES)
winrepo.init_remotes(
remotes, PER_REMOTE_OVERRIDES, PER_REMOTE_ONLY)
winrepo.fetch_remotes()
# Since we're not running update(), we need to manually call
# clear_old_remotes() to remove directories from remotes that

7
salt/states/boto_kinesis.py
View file

@ -63,13 +63,16 @@ import logging
log = logging.getLogger(__name__)
__virtualname__ = 'boto_kinesis'
def __virtual__():
'''
Only load if boto_kinesis is available.
'''
ret = 'boto_kinesis' if 'boto_kinesis.exists' in __salt__ else False
return ret
if 'boto_kinesis.exists' in __salt__:
return __virtualname__
return False, 'The boto_kinesis module could not be loaded: boto libraries not found.'
def present(name,

12
salt/states/ini_manage.py
View file

@ -93,7 +93,7 @@ def options_present(name, sections=None, separator='=', strict=False):
del changes[section_name]
else:
changes = __salt__['ini.set_option'](name, sections, separator)
except IOError as err:
except (IOError, KeyError) as err:
ret['comment'] = "{0}".format(err)
ret['result'] = False
return ret
@ -102,12 +102,10 @@ def options_present(name, sections=None, separator='=', strict=False):
ret['comment'] = 'Errors encountered. {0}'.format(changes['error'])
ret['changes'] = {}
else:
if changes:
ret['changes'] = changes
ret['comment'] = 'Changes take effect'
else:
ret['changes'] = {}
ret['comment'] = 'No changes take effect'
for name, body in changes.items():
if body:
ret['comment'] = 'Changes take effect'
ret['changes'].update({name: changes[name]})
return ret

5
salt/utils/path.py
View file

@ -120,6 +120,11 @@ def readlink(path):
# comes out in 8.3 form; convert it to LFN to make it look nicer
target = win32file.GetLongPathName(target)
except pywinerror as exc:
# If target is on a UNC share, the decoded target will be in the format
# "UNC\hostanme\sharename\additional\subdirs\under\share". So, in
# these cases, return the target path in the proper UNC path format.
if target.startswith('UNC\\'):
return re.sub(r'^UNC\\+', r'\\\\', target)
# if file is not found (i.e. bad symlink), return it anyway like on *nix
if exc.winerror == 2:
return target

1
tests/unit/modules/test_snapper.py
View file

@ -141,6 +141,7 @@ MODULE_RET = {
}
@skipIf(sys.platform.startswith('win'), 'Snapper not available on Windows')
@skipIf(NO_MOCK, NO_MOCK_REASON)
class SnapperTestCase(TestCase, LoaderModuleMockMixin):

24
tests/unit/modules/test_state.py
View file

@ -283,7 +283,7 @@ class MockTarFile(object):
'''
Mock tarfile class
'''
path = "/tmp"
path = os.sep + "tmp"
def __init__(self):
pass
@ -961,36 +961,32 @@ class StateTestCase(TestCase, LoaderModuleMockMixin):
'''
Test to execute a packaged state run
'''
tar_file = os.sep + os.path.join('tmp', 'state_pkg.tgz')
mock = MagicMock(side_effect=[False, True, True, True, True, True, True, True])
with patch.object(os.path, 'isfile', mock), \
patch('salt.modules.state.tarfile', MockTarFile), \
patch('salt.modules.state.json', MockJson()):
self.assertEqual(state.pkg("/tmp/state_pkg.tgz", "", "md5"), {})
self.assertEqual(state.pkg(tar_file, "", "md5"), {})
mock = MagicMock(side_effect=[False, 0, 0, 0, 0])
with patch.object(salt.utils, 'get_hash', mock):
self.assertDictEqual(state.pkg("/tmp/state_pkg.tgz", "", "md5"),
{})
# Verify hash
self.assertDictEqual(state.pkg(tar_file, "", "md5"), {})
self.assertDictEqual(state.pkg("/tmp/state_pkg.tgz", 0, "md5"),
{})
# Verify file outside intended root
self.assertDictEqual(state.pkg(tar_file, 0, "md5"), {})
MockTarFile.path = ""
MockJson.flag = True
with patch('salt.utils.files.fopen', mock_open()):
self.assertListEqual(state.pkg("/tmp/state_pkg.tgz",
0,
"md5"),
[True])
self.assertListEqual(state.pkg(tar_file, 0, "md5"), [True])
MockTarFile.path = ""
MockJson.flag = False
if six.PY2:
with patch('salt.utils.files.fopen', mock_open()), \
patch.dict(state.__utils__, {'state.check_result': MagicMock(return_value=True)}):
self.assertTrue(state.pkg("/tmp/state_pkg.tgz",
0, "md5"))
self.assertTrue(state.pkg(tar_file, 0, "md5"))
else:
with patch('salt.utils.files.fopen', mock_open()):
self.assertTrue(state.pkg("/tmp/state_pkg.tgz",
0, "md5"))
self.assertTrue(state.pkg(tar_file, 0, "md5"))

52
tests/unit/modules/test_status.py
View file

@ -2,6 +2,7 @@
# Import Python libs
from __future__ import absolute_import
import os
# Import Salt Libs
import salt.utils.platform
@ -78,18 +79,19 @@ class StatusTestCase(TestCase, LoaderModuleMockMixin):
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value="1\n2\n3")}):
with patch('time.time', MagicMock(return_value=m.now)):
with patch('os.path.exists', MagicMock(return_value=True)):
proc_uptime = '{0} {1}'.format(m.ut, m.idle)
with patch('salt.utils.files.fopen', mock_open(read_data=proc_uptime)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3']))}), \
patch('time.time', MagicMock(return_value=m.now)), \
patch('os.path.exists', MagicMock(return_value=True)):
proc_uptime = '{0} {1}'.format(m.ut, m.idle)
with patch('os.path.exists', MagicMock(return_value=False)):
with self.assertRaises(CommandExecutionError):
status.uptime()
with patch('salt.utils.files.fopen', mock_open(read_data=proc_uptime)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
with patch('os.path.exists', MagicMock(return_value=False)):
with self.assertRaises(CommandExecutionError):
status.uptime()
def test_uptime_sunos(self):
'''
@ -103,13 +105,13 @@ class StatusTestCase(TestCase, LoaderModuleMockMixin):
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value="1\n2\n3"),
'cmd.run_all': MagicMock(return_value=m2.ret)}):
with patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3'])),
'cmd.run_all': MagicMock(return_value=m2.ret)}), \
patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
def test_uptime_macos(self):
'''
@ -125,12 +127,14 @@ class StatusTestCase(TestCase, LoaderModuleMockMixin):
is_darwin=MagicMock(return_value=True),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value="1\n2\n3"),
'sysctl.get': MagicMock(return_value=kern_boottime)}):
with patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3'])),
'sysctl.get': MagicMock(return_value=kern_boottime)}), \
patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
with patch.dict(status.__salt__, {'sysctl.get': MagicMock(return_value='')}):
with self.assertRaises(CommandExecutionError):

9
tests/unit/modules/test_virt.py
View file

@ -3,6 +3,7 @@
# Import python libs
from __future__ import absolute_import
import re
import os
# Import Salt Testing libs
from tests.support.mixins import LoaderModuleMockMixin
@ -13,6 +14,7 @@ from tests.support.mock import NO_MOCK, NO_MOCK_REASON, MagicMock, patch
import salt.modules.virt as virt
import salt.modules.config as config
from salt._compat import ElementTree as ET
import salt.config
import salt.utils
# Import third party libs
@ -245,8 +247,9 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
disks = root.findall('.//disk')
self.assertEqual(len(disks), 1)
disk = disks[0]
self.assertTrue(disk.find('source').attrib['file'].startswith('/'))
self.assertTrue('hello/system' in disk.find('source').attrib['file'])
root_dir = salt.config.DEFAULT_MINION_OPTS.get('root_dir')
self.assertTrue(disk.find('source').attrib['file'].startswith(root_dir))
self.assertTrue(os.path.join('hello', 'system') in disk.find('source').attrib['file'])
self.assertEqual(disk.find('target').attrib['dev'], 'vda')
self.assertEqual(disk.find('target').attrib['bus'], 'virtio')
self.assertEqual(disk.find('driver').attrib['name'], 'qemu')
@ -284,7 +287,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
self.assertEqual(len(disks), 1)
disk = disks[0]
self.assertTrue('[0]' in disk.find('source').attrib['file'])
self.assertTrue('hello/system' in disk.find('source').attrib['file'])
self.assertTrue(os.path.join('hello', 'system') in disk.find('source').attrib['file'])
self.assertEqual(disk.find('target').attrib['dev'], 'sda')
self.assertEqual(disk.find('target').attrib['bus'], 'scsi')
self.assertEqual(disk.find('address').attrib['unit'], '0')

12
tests/unit/modules/test_win_service.py
View file

@ -143,14 +143,14 @@ class WinServiceTestCase(TestCase, LoaderModuleMockMixin):
{'Status': 'Stop Pending'},
{'Status': 'Stopped'}])
with patch.object(win_service, 'status', mock_false):
with patch.dict(win_service.__salt__, {'cmd.run': MagicMock(return_value="service was stopped")}):
self.assertTrue(win_service.stop('spongebob'))
with patch.object(win_service, 'status', mock_true):
with patch.object(win32serviceutil, 'StopService', mock_true):
with patch.object(win_service, 'info', mock_info):
with patch.object(win_service, 'status', mock_false):
self.assertTrue(win_service.stop('spongebob'))
with patch.dict(win_service.__salt__, {'cmd.run': MagicMock(return_value="service was stopped")}), \
patch.object(win32serviceutil, 'StopService', mock_true), \
patch.object(win_service, 'info', mock_info), \
patch.object(win_service, 'status', mock_false):
self.assertTrue(win_service.stop('spongebob'))
def test_restart(self):
'''

6
tests/unit/modules/test_znc.py
View file

@ -54,7 +54,8 @@ class ZncTestCase(TestCase, LoaderModuleMockMixin):
Tests write the active configuration state to config file
'''
mock = MagicMock(return_value='SALT')
with patch.dict(znc.__salt__, {'ps.pkill': mock}):
with patch.dict(znc.__salt__, {'ps.pkill': mock}), \
patch.object(znc, 'signal', MagicMock()):
self.assertEqual(znc.dumpconf(), 'SALT')
# 'rehashconf' function tests: 1
@ -64,7 +65,8 @@ class ZncTestCase(TestCase, LoaderModuleMockMixin):
Tests rehash the active configuration state from config file
'''
mock = MagicMock(return_value='SALT')
with patch.dict(znc.__salt__, {'ps.pkill': mock}):
with patch.dict(znc.__salt__, {'ps.pkill': mock}), \
patch.object(znc, 'signal', MagicMock()):
self.assertEqual(znc.rehashconf(), 'SALT')
# 'version' function tests: 1

2
tests/unit/modules/test_zypper.py
View file

@ -190,7 +190,7 @@ class ZypperTestCase(TestCase, LoaderModuleMockMixin):
}
with patch.dict('salt.modules.zypper.__salt__', {'cmd.run_all': MagicMock(return_value=ref_out)}):
with self.assertRaisesRegex(CommandExecutionError,
"^Zypper command failure: Some handled zypper internal error\nAnother zypper internal error$"):
"^Zypper command failure: Some handled zypper internal error{0}Another zypper internal error$".format(os.linesep)):
zypper.list_upgrades(refresh=False)
# Test unhandled error

4
tests/unit/states/test_mac_package.py
View file

@ -2,19 +2,21 @@
# Import Python libs
from __future__ import absolute_import
import sys
# Import Salt Libs
import salt.states.mac_package as macpackage
# Import Salt Testing Libs
from tests.support.mixins import LoaderModuleMockMixin
from tests.support.unit import TestCase
from tests.support.unit import TestCase, skipIf
from tests.support.mock import (
MagicMock,
patch
)
@skipIf(sys.platform.startswith('win'), "Not a Windows test")
class MacPackageTestCase(TestCase, LoaderModuleMockMixin):
def setup_loader_modules(self):
return {macpackage: {}}