saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge branch 'develop' of https://github.com/saltstack/salt into develop

Browse source 
Conflicts:
	salt/states/network.py
	setup.py
	tests/integration/__init__.py
This commit is contained in:
Intchanter 2012-07-08 22:50:12 -06:00
commit f663dd2c46
33 changed files with 878 additions and 554 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

170
doc/topics/installation/debian.rst
View file

@ -1,168 +1,10 @@
===============
Debian & Ubuntu
===============
Ubuntu
======
Debian
======
Installation
============
Salt is currently available in in the Debian package tree:
To install Salt on Ubuntu, use the following command:
http://packages.debian.org/source/salt
.. code-block:: bash
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:saltstack/salt
sudo apt-get update
sudo apt-get install salt-master
sudo apt-get install salt-minion
.. admonition:: Installing on Ubuntu 11.04
There is a conflict with `msgpack-python` on Ubuntu 11.04 and the current
saltstack PPA. You can work around the conflict by installing
`msgpack-python` from Oneiric:
.. code-block:: bash
sudo add-apt-repository 'deb http://us.archive.ubuntu.com/ubuntu/ oneiric universe'
sudo add-apt-repository ppa:saltstack/salt
sudo apt-get update
sudo apt-get install msgpack-python
sudo apt-get install salt-master
sudo apt-get install salt-minion
sudo add-apt-repository --remove 'deb http://us.archive.ubuntu.com/ubuntu/ oneiric universe'
After installation you'll need to make a few changes to the configuration files.
Configuration
=============
To configure your Salt files we must modify both master and minion
configuration files. We need to set where the master binds, by default salt
listens on all interfaces. If you have a need to bind to a specific local IP,
make the change as needed. To edit the master type in the following command:
.. code-block:: bash
sudo vim /etc/salt/master
From here make the following changes:
.. code-block:: diff
- interface: 0.0.0.0
+ interface: 192.168.0.10
To configure the minion type in the following command:
.. code-block:: bash
sudo vim /etc/salt/minion
Once inside the editor make the following changes:
.. code-block:: diff
- master: salt
+ master: 192.168.0.10
After making the following changes you need to restart both the master and the
minion. To do so type in the following commands:
.. code-block:: bash
sudo /etc/init.d/salt-master restart
sudo /etc/init.d/salt-minion restart
Test
====
To test Salt we must first sign the key of the minion to the master. To see the
pending keys type in the following command:
.. code-block:: bash
sudo salt-key -L
From here you will should see a key name underneath the Unaccepted Keys
portion. To sign the minion key to the master type in the following command:
.. code-block:: bash
sudo salt-key -a $minion
Where ``$minion`` is the unaccepted key.
Now that you have signed the key we need to see if the key was accepted and
that we can ping the minion and get a response. To do this you can type in one
of the previous commands ``sudo salt-key -L`` and see if the key has been
accepted, then also ping the minion to see if it's working by typing in the
following command:
.. code-block:: bash
sudo salt \* test.ping
If it is working properly you should see this result:
.. code-block:: bash
{'$minion': True}
Troubleshooting
===============
To see if the Salt master is running properly type in the following command:
.. code-block:: bash
netstat -natp | grep 450
This should return ``192.168.0.10:4505`` and ``192.168.0.10:4506`` if the master was
configured properly. If this does not return those values recheck your master
and minion config files for mistakes.
To see if both master and minion are running properly type in the following
command:
.. code-block:: bash
ps -efH | grep sal[t]
This should return 8 Salt masters and 1 Salt minion if both are configured
properly. If you are still having issues with your Salt configuration please
reference the trouble shooting page :doc:`Troubleshooting</topics/troubleshooting/index>`.
What Now?
=========
Congratulations you have just successfully installed Salt on your Ubuntu machine
and configured both the master and the minion. From this point you are now
able to send remote commands. Depending on the primary way you want to
manage your machines you may either want to visit the section regarding Salt
States, or the section on Modules.
Debian
------
`A deb package is currently in testing`__ for inclusion in apt. Until that is
accepted you can install Salt by downloading the latest ``.deb`` in the
`downloads section on GitHub`__ and installing that manually using ``dpkg -i``.
.. __: http://mentors.debian.net/package/salt
.. __: https://github.com/saltstack/salt/downloads
.. admonition:: Installing ZeroMQ on Squeeze (Debian 6)
There is a `python-zmq`__ package available in Debian \"wheezy (testing)\".
If you don't have that repo enabled the best way to install Salt and pyzmq
is by using ``pip`` (or ``easy_install``):
.. code-block:: bash
pip install pyzmq salt
.. __: http://packages.debian.org/search?keywords=python-zmq
If the desired Debian release is not supported rebuilding the source package
on your target platform or installing from source is recommended.

1
doc/topics/installation/index.rst
View file

@ -40,6 +40,7 @@ Platform-specific installation instructions
arch
debian
ubuntu
fedora
freebsd
gentoo

144
doc/topics/installation/ubuntu.rst Normal file
View file

@ -0,0 +1,144 @@
======
Ubuntu
======
Installation
============
To install Salt on Ubuntu, use the following command:
.. code-block:: bash
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:saltstack/salt
sudo apt-get update
sudo apt-get install salt-master
sudo apt-get install salt-minion
.. admonition:: Installing on Ubuntu 11.04
There is a conflict with `msgpack-python` on Ubuntu 11.04 and the current
saltstack PPA. You can work around the conflict by installing
`msgpack-python` from Oneiric:
.. code-block:: bash
sudo add-apt-repository 'deb http://us.archive.ubuntu.com/ubuntu/ oneiric universe'
sudo add-apt-repository ppa:saltstack/salt
sudo apt-get update
sudo apt-get install msgpack-python
sudo apt-get install salt-master
sudo apt-get install salt-minion
sudo add-apt-repository --remove 'deb http://us.archive.ubuntu.com/ubuntu/ oneiric universe'
After installation you'll need to make a few changes to the configuration files.
Configuration
=============
To configure your Salt files we must modify both master and minion
configuration files. We need to set where the master binds, by default salt
listens on all interfaces. If you have a need to bind to a specific local IP,
make the change as needed. To edit the master type in the following command:
.. code-block:: bash
sudo vim /etc/salt/master
From here make the following changes:
.. code-block:: diff
- interface: 0.0.0.0
+ interface: 192.168.0.10
To configure the minion type in the following command:
.. code-block:: bash
sudo vim /etc/salt/minion
Once inside the editor make the following changes:
.. code-block:: diff
- master: salt
+ master: 192.168.0.10
After making the following changes you need to restart both the master and the
minion. To do so type in the following commands:
.. code-block:: bash
sudo /etc/init.d/salt-master restart
sudo /etc/init.d/salt-minion restart
Test
====
To test Salt we must first sign the key of the minion to the master. To see the
pending keys type in the following command:
.. code-block:: bash
sudo salt-key -L
From here you will should see a key name underneath the Unaccepted Keys
portion. To sign the minion key to the master type in the following command:
.. code-block:: bash
sudo salt-key -a $minion
Where ``$minion`` is the unaccepted key.
Now that you have signed the key we need to see if the key was accepted and
that we can ping the minion and get a response. To do this you can type in one
of the previous commands ``sudo salt-key -L`` and see if the key has been
accepted, then also ping the minion to see if it's working by typing in the
following command:
.. code-block:: bash
sudo salt \* test.ping
If it is working properly you should see this result:
.. code-block:: bash
{'$minion': True}
Troubleshooting
===============
To see if the Salt master is running properly type in the following command:
.. code-block:: bash
netstat -natp | grep 450
This should return ``192.168.0.10:4505`` and ``192.168.0.10:4506`` if the master was
configured properly. If this does not return those values recheck your master
and minion config files for mistakes.
To see if both master and minion are running properly type in the following
command:
.. code-block:: bash
ps -efH | grep sal[t]
This should return 8 Salt masters and 1 Salt minion if both are configured
properly. If you are still having issues with your Salt configuration please
reference the trouble shooting page :doc:`Troubleshooting</topics/troubleshooting/index>`.
What Now?
=========
Congratulations you have just successfully installed Salt on your Ubuntu machine
and configured both the master and the minion. From this point you are now
able to send remote commands. Depending on the primary way you want to
manage your machines you may either want to visit the section regarding Salt
States, or the section on Modules.

1
salt/__init__.py
View file

@ -209,6 +209,7 @@ class Minion(object):
verify_env([
self.opts['pki_dir'],
self.opts['cachedir'],
self.opts['sock_dir'],
self.opts['extension_modules'],
os.path.dirname(self.opts['log_file']),
], self.opts['user'])

15
salt/client.py
View file

@ -860,19 +860,10 @@ class LocalClient(object):
if self.opts['order_masters']:
payload_kwargs['to'] = timeout
package = salt.payload.format_payload('clear', **payload_kwargs)
# Prep zmq
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.linger = 0
socket.connect(
'tcp://{0[interface]}:{0[ret_port]}'.format(
self.opts
)
sreq = salt.payload.SREQ(
'tcp://{0[interface]}:{0[ret_port]}'.format(self.opts),
)
socket.send(package)
payload = self.serial.loads(socket.recv())
payload = sreq.send('clear', payload_kwargs)
return {'jid': payload['load']['jid'],
'minions': minions}

18
salt/crypt.py
View file

@ -24,7 +24,7 @@ import zmq
import salt.utils
import salt.payload
import salt.utils.verify
from salt.exceptions import AuthenticationError, SaltClientError
from salt.exceptions import AuthenticationError, SaltClientError, SaltReqTimeoutError
log = logging.getLogger(__name__)
@ -241,15 +241,17 @@ class Auth(object):
'''
auth = {}
try:
self.opts['master_ip'] = salt.utils.dns_check(self.opts['master'], True)
self.opts['master_ip'] = salt.utils.dns_check(
self.opts['master'],
True
)
except SaltClientError:
return 'retry'
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(self.opts['master_uri'])
payload = self.serial.dumps(self.minion_sign_in_payload())
socket.send(payload)
payload = self.serial.loads(socket.recv())
sreq = salt.payload.SREQ(self.opts['master_uri'])
try:
payload = sreq.send_auto(self.minion_sign_in_payload())
except SaltReqTimeoutError:
return 'retry'
if 'load' in payload:
if 'ret' in payload['load']:
if not payload['load']['ret']:

5
salt/exceptions.py
View file

@ -70,3 +70,8 @@ class SaltRenderError(SaltException):
'''
Used when a renderer needs to raise an explicit error
'''
class SaltReqTimeoutError(SaltException):
'''
Thrown when a salt master request call fails to return within the timeout
'''

111
salt/fileclient.py
View file

@ -15,7 +15,7 @@ import yaml
import zmq
# Import salt libs
from salt.exceptions import MinionError
from salt.exceptions import MinionError, SaltReqTimeoutError
import salt.client
import salt.crypt
import salt.loader
@ -533,16 +533,7 @@ class RemoteClient(Client):
def __init__(self, opts):
Client.__init__(self, opts)
self.auth = salt.crypt.SAuth(opts)
self.socket = self.__get_socket()
def __get_socket(self):
'''
Return the ZeroMQ socket to use
'''
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(self.opts['master_uri'])
return socket
self.sreq = salt.payload.SREQ(self.opts['master_uri'])
def get_file(self, path, dest='', makedirs=False, env='base'):
'''
@ -552,7 +543,6 @@ class RemoteClient(Client):
cache
'''
path = self._check_proto(path)
payload = {'enc': 'aes'}
load = {'path': path,
'env': env,
'cmd': '_serve_file'}
@ -570,9 +560,16 @@ class RemoteClient(Client):
load['loc'] = 0
else:
load['loc'] = fn_.tell()
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
data = self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
data = self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
if not data['data']:
if not fn_ and data['dest']:
# This is a 0 byte file on the master
@ -595,23 +592,35 @@ class RemoteClient(Client):
'''
List the files on the master
'''
payload = {'enc': 'aes'}
load = {'env': env,
'cmd': '_file_list'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
def file_list_emptydirs(self, env='base'):
'''
List the empty dirs on the master
'''
payload = {'enc': 'aes'}
load = {'env': env,
'cmd': '_file_list_emptydirs'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
def hash_file(self, path, env='base'):
'''
@ -633,43 +642,67 @@ class RemoteClient(Client):
ret['hsum'] = hashlib.md5(f.read()).hexdigest()
ret['hash_type'] = 'md5'
return ret
payload = {'enc': 'aes'}
load = {'path': path,
'env': env,
'cmd': '_file_hash'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
def list_env(self, path, env='base'):
'''
Return a list of the files in the file server's specified environment
'''
payload = {'enc': 'aes'}
load = {'env': env,
'cmd': '_file_list'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
def master_opts(self):
'''
Return the master opts data
'''
payload = {'enc': 'aes'}
load = {'cmd': '_master_opts'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''
def ext_nodes(self):
'''
Return the metadata derived from the external nodes system on the
master.
'''
payload = {'enc': 'aes'}
load = {'cmd': '_ext_nodes',
'id': self.opts['id']}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(
self.sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
60)
)
except SaltReqTimeoutError:
return ''

9
salt/grains/core.py
View file

@ -177,6 +177,8 @@ def _virtual(osdata):
if 'Vendor: QEMU' in output:
# FIXME: Make this detect between kvm or qemu
grains['virtual'] = 'kvm'
if 'Vendor: Bochs' in output:
grains['virtual'] = 'kvm'
elif 'VirtualBox' in output:
grains['virtual'] = 'VirtualBox'
# Product Name: VMware Virtual Platform
@ -196,6 +198,8 @@ def _virtual(osdata):
grains['virtual'] = 'VirtualBox'
elif 'qemu' in model:
grains['virtual'] = 'kvm'
elif 'virtio' in model:
grains['virtual'] = 'kvm'
choices = ('Linux', 'OpenBSD', 'SunOS', 'HP-UX')
isdir = os.path.isdir
if osdata['kernel'] in choices:
@ -435,6 +439,11 @@ def os_data():
grains['os'] = 'Scientific'
else:
grains['os'] = 'RedHat'
elif os.path.isfile('/etc/system-release'):
grains['os_family'] = 'RedHat'
data = open('/etc/system-release', 'r').read()
if 'amazon' in data.lower():
grains['os'] = 'Amazon'
elif os.path.isfile('/etc/SuSE-release'):
grains['os_family'] = 'Suse'
data = open('/etc/SuSE-release', 'r').read()

12
salt/loader.py
View file

@ -12,6 +12,8 @@ import imp
import salt
import logging
import tempfile
# Import Salt libs
from salt.exceptions import LoaderError
log = logging.getLogger(__name__)
@ -79,6 +81,16 @@ def returners(opts):
return load.filter_func('returner')
def pillars(opts, functions):
'''
Returns the returner modules
'''
load = _create_loader(opts, 'pillar', 'pillar')
pack = {'name': '__salt__',
'value': functions}
return load.filter_func('ext_pillar', pack)
def states(opts, functions):
'''
Returns the state modules

131
salt/minion.py
View file

@ -20,7 +20,7 @@ import zmq
# Import salt libs
from salt.exceptions import AuthenticationError, \
CommandExecutionError, CommandNotFoundError, SaltInvocationError, \
SaltClientError
SaltClientError, SaltReqTimeoutError
import salt.client
import salt.crypt
import salt.loader
@ -374,10 +374,7 @@ class Minion(object):
# The file is gone already
pass
log.info('Returning information for job: {0}'.format(ret['jid']))
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(self.opts['master_uri'])
payload = {'enc': 'aes'}
sreq = salt.payload.SREQ(self.opts['master_uri'])
if ret_cmd == '_syndic_return':
load = {'cmd': ret_cmd,
'jid': ret['jid'],
@ -399,10 +396,10 @@ class Minion(object):
load['out'] = oput
except KeyError:
pass
payload['load'] = self.crypticle.dumps(load)
data = self.serial.dumps(payload)
socket.send(data)
ret_val = self.serial.loads(socket.recv())
try:
ret_val = sreq.send('aes', self.crypticle.dumps(load))
except SaltReqTimeoutError:
ret_val = ''
if isinstance(ret_val, string_types) and not ret_val:
# The master AES key has changed, reauth
self.authenticate()
@ -475,13 +472,43 @@ class Minion(object):
Lock onto the publisher. This is the main event loop for the minion
'''
context = zmq.Context()
# Prepare the minion event system
#
# Start with the publish socket
epub_sock = context.socket(zmq.PUB)
epub_uri = 'ipc://{0}'.format(
os.path.join(self.opts['sock_dir'], 'minion_event_pub.ipc')
)
# Create the pull socket
epull_sock = context.socket(zmq.PULL)
epull_uri = 'ipc://{0}'.format(
os.path.join(self.opts['sock_dir'], 'minion_event_pull.ipc')
)
# Bind the event sockets
epub_sock.bind(epub_uri)
epull_sock.bind(epull_uri)
# Restrict access to the sockets
os.chmod(
os.path.join(self.opts['sock_dir'],
'minion_event_pub.ipc'),
448
)
os.chmod(
os.path.join(self.opts['sock_dir'],
'minion_event_pull.ipc'),
448
)
poller = zmq.Poller()
epoller = zmq.Poller()
socket = context.socket(zmq.SUB)
socket.setsockopt(zmq.SUBSCRIBE, '')
if self.opts['sub_timeout']:
socket.setsockopt(zmq.IDENTITY, self.opts['id'])
socket.connect(self.master_pub)
poller.register(socket, zmq.POLLIN)
epoller.register(epull_sock, zmq.POLLIN)
# Make sure to gracefully handle SIGUSR1
enable_sigusr1_handler()
@ -489,43 +516,63 @@ class Minion(object):
if self.opts['sub_timeout']:
last = time.time()
while True:
socks = dict(poller.poll(self.opts['sub_timeout']))
if socket in socks and socks[socket] == zmq.POLLIN:
payload = self.serial.loads(socket.recv())
self._handle_payload(payload)
last = time.time()
if time.time() - last > self.opts['sub_timeout']:
# It has been a while since the last command, make sure
# the connection is fresh by reconnecting
if self.opts['dns_check']:
try:
socks = dict(poller.poll(self.opts['sub_timeout']))
if socket in socks and socks[socket] == zmq.POLLIN:
payload = self.serial.loads(socket.recv())
self._handle_payload(payload)
last = time.time()
if time.time() - last > self.opts['sub_timeout']:
# It has been a while since the last command, make sure
# the connection is fresh by reconnecting
if self.opts['dns_check']:
try:
# Verify that the dns entry has not changed
self.opts['master_ip'] = salt.utils.dns_check(
self.opts['master'], safe=True)
except SaltClientError:
# Failed to update the dns, keep the old addr
pass
poller.unregister(socket)
socket.close()
socket = context.socket(zmq.SUB)
socket.setsockopt(zmq.SUBSCRIBE, '')
socket.setsockopt(zmq.IDENTITY, self.opts['id'])
socket.connect(self.master_pub)
poller.register(socket, zmq.POLLIN)
last = time.time()
time.sleep(0.05)
multiprocessing.active_children()
self.passive_refresh()
# Check the event system
if epoller.poll(1):
try:
# Verify that the dns entry has not changed
self.opts['master_ip'] = salt.utils.dns_check(
self.opts['master'], safe=True)
except SaltClientError:
# Failed to update the dns, keep the old addr
package = epull_sock.recv(zmq.NOBLOCK)
epub_sock.send(package)
except Exception:
pass
poller.unregister(socket)
socket.close()
socket = context.socket(zmq.SUB)
socket.setsockopt(zmq.SUBSCRIBE, '')
socket.setsockopt(zmq.IDENTITY, self.opts['id'])
socket.connect(self.master_pub)
poller.register(socket, zmq.POLLIN)
last = time.time()
time.sleep(0.05)
multiprocessing.active_children()
self.passive_refresh()
except Exception as exc:
log.critical('A fault occured in the main minion loop {0}'.format(exc))
else:
while True:
socks = dict(poller.poll(60))
if socket in socks and socks[socket] == zmq.POLLIN:
payload = self.serial.loads(socket.recv())
self._handle_payload(payload)
last = time.time()
time.sleep(0.05)
multiprocessing.active_children()
self.passive_refresh()
try:
socks = dict(poller.poll(60))
if socket in socks and socks[socket] == zmq.POLLIN:
payload = self.serial.loads(socket.recv())
self._handle_payload(payload)
last = time.time()
time.sleep(0.05)
multiprocessing.active_children()
self.passive_refresh()
# Check the event system
if epoller.poll(1):
try:
package = epull_sock.recv(zmq.NOBLOCK)
epub_sock.send(package)
except Exception:
pass
except Exception as exc:
log.critical('A fault occured in the main minion loop {0}'.format(exc))
class Syndic(salt.client.LocalClient, Minion):

34
salt/modules/event.py
View file

@ -3,27 +3,37 @@ Fire events on the minion, events can be fired up to the master
'''
# Import Salt libs
import salt.crypt
import salt.utils.event
import salt.payload
# Import third party libs
import zmq
def fire_master(data, tag):
'''
Fire an event off on the master server
CLI Example::
salt '*' event.fire_master 'stuff to be in the event' 'tag'
'''
serial = salt.payload.Serial(__opts__)
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(__opts__['master_uri'])
load = {'id': __opts__['id'],
'tag': tag,
'data': data,
'cmd': '_minion_event'}
auth = salt.crypt.SAuth(__opts__)
payload = {'enc': 'aes'}
payload['load'] = auth.crypticle.dumps(load)
auth = salt.crypt.SAuth(__opts__)
socket.send(serial.dumps(payload))
socket.recv()
sreq = salt.payload.SREQ(__opts__['master_uri'])
try:
sreq.send('aes', auth.crypticle.dumps(load))
except:
pass
return True
def fire(data, tag):
'''
Fire an event on the local minion event bus
CLI Example::
salt '*' event.fire 'stuff to be in the event' 'tag'
'''
esock = salt.utils.event.MinionEvent(__opts__['sock_dir'])
return esock.fire_event(data, tag)

23
salt/modules/git.py
View file

@ -312,3 +312,26 @@ def init(cwd, opts=None, user=None):
cmd = 'git init {0} {1}'.format(cwd, opts)
return __salt__['cmd.run'](cmd, runas=user)
def submodule(cwd, init=True, opts=None, user=None):
'''
Initialize git submodules
cwd
The path to the Git repository
init : True
Ensure that new submodules are initialized
opts : None
Any additional options to add to the command line
user : None
Run git as a user other than what the minion runs as
'''
_check_git()
if not opts:
opts = ''
cmd = 'git submodule update {0} {1}'.format('--init' if init else '', opts)
return __salt__['cmd.run'](cmd, cwd=cwd, runas=user)

105
salt/modules/mysql.py
View file

@ -21,8 +21,11 @@ REQUIREMENT 2:
Required python modules: MySQLdb
'''
# Import Python libs
import time
import logging
# Import third party libs
try:
import MySQLdb
import MySQLdb.cursors
@ -53,6 +56,7 @@ def __check_table(name, table):
log.debug( results )
return results
def __repair_table(name, table):
db = connect()
cur = db.cursor(MySQLdb.cursors.DictCursor)
@ -63,6 +67,7 @@ def __repair_table(name, table):
log.debug( results )
return results
def __optimize_table(name, table):
db = connect()
cur = db.cursor(MySQLdb.cursors.DictCursor)
@ -73,6 +78,7 @@ def __optimize_table(name, table):
log.debug( results )
return results
def connect(**kwargs):
'''
wrap authentication credentials here
@ -102,6 +108,62 @@ def connect(**kwargs):
return db
def query(database, query):
'''
Run an arbitrary SQL query and return the results or
the number of affected rows.
CLI Example::
salt '*' mysql.query mydb "UPDATE mytable set myfield=1 limit 1"
returns: {'query time': {'human': '39.0ms', 'raw': '0.03899'},
'rows affected': 1L}
salt '*' mysql.query mydb "SELECT id,name,cash from users limit 3"
returns: {'columns': ('id', 'name', 'cash'),
'query time': {'human': '1.0ms', 'raw': '0.001'},
'results': ((1L, 'User 1', Decimal('110.000000')),
(2L, 'User 2', Decimal('215.636756')),
(3L, 'User 3', Decimal('0.040000'))),
'rows returned': 3L}
salt '*' mysql.query mydb "insert into users values (null,'user 4', 5)"
returns: {'query time': {'human': '25.6ms', 'raw': '0.02563'},
'rows affected': 1L}
salt '*' mysql.query mydb "delete from users where id = 4 limit 1""
returns: {'query time': {'human': '39.0ms', 'raw': '0.03899'},
'rows affected': 1L}
'''
#Doesn't do anything about sql warnings, e.g. empty values on an insert.
#I don't think it handles multiple queries at once, so adding "commit" might not work.
#This should be accessible via {{ salt[mysql.query mydb "myquery"]}} but there's too much extra info here.
ret = {}
db = connect(**{'db': database})
cur = db.cursor()
start = time.time()
affected = cur.execute(query)
log.debug('Using db: ' + database + ' to run query: ' + query)
results = cur.fetchall()
elapsed = (time.time() - start)
if elapsed < 0.200:
elapsed_h = str(round(elapsed * 1000, 1)) + 'ms'
else:
elapsed_h = str(round(elapsed, 2)) + 's'
ret['query time'] = {'human': elapsed_h, 'raw': str(round(elapsed, 5))}
if len(results) == 0:
ret['rows affected'] = affected
return ret
else:
ret['rows returned'] = affected
columns = ()
for column in cur.description:
columns += (column[0],)
ret['columns'] = columns
ret['results'] = results
return ret
def status():
'''
Return the status of a MySQL server using the output
@ -136,6 +198,7 @@ def version():
row = cur.fetchone()
return row
def slave_lag():
'''
Return the number of seconds that a slave SQL server is lagging behind the
@ -199,9 +262,8 @@ def free_slave():
else:
return 'failed'
'''
Database related actions
'''
#Database related actions
def db_list():
'''
Return a list of databases of a MySQL server using the output
@ -222,6 +284,7 @@ def db_list():
log.debug(ret)
return ret
def db_tables(name):
'''
Shows the tables in the given MySQL database (if exists)
@ -247,6 +310,7 @@ def db_tables(name):
log.debug( ret )
return ret
def db_exists(name):
'''
Checks if a database exists on the MySQL server.
@ -287,6 +351,7 @@ def db_create(name):
return True
return False
def db_remove(name):
'''
Removes a databases from the MySQL server.
@ -318,9 +383,8 @@ def db_remove(name):
log.info("Database '{0}' has not been removed".format(name,))
return False
'''
User related actions
'''
# User related actions
def user_list():
'''
Return a list of users on a MySQL server
@ -336,8 +400,8 @@ def user_list():
log.debug( results )
return results
def user_exists(user,
host='localhost'):
def user_exists(user, host='localhost'):
'''
Checks if a user exists on the MySQL server.
@ -352,8 +416,8 @@ def user_exists(user,
cur.execute( query )
return cur.rowcount == 1
def user_info(user,
host='localhost'):
def user_info(user, host='localhost'):
'''
Get full info on a MySQL user
@ -370,6 +434,7 @@ def user_info(user,
log.debug( result )
return result
def user_create(user,
host='localhost',
password=None,
@ -405,6 +470,7 @@ def user_create(user,
log.info("User '{0}'@'{1}' is not created".format(user,host,))
return False
def user_chpass(user,
host='localhost',
password=None,
@ -437,6 +503,7 @@ def user_chpass(user,
log.info("Password for user '{0}'@'{1}' is not changed".format(user,host,))
return False
def user_remove(user,
host='localhost'):
'''
@ -458,9 +525,8 @@ def user_remove(user,
log.info("User '{0}'@'{1}' has NOT been removed".format(user,host,))
return False
'''
Maintenance
'''
# Maintenance
def db_check(name,
table=None):
'''
@ -482,6 +548,7 @@ def db_check(name,
ret = __check_table(name, table)
return ret
def db_repair(name,
table=None):
'''
@ -503,6 +570,7 @@ def db_repair(name,
ret = __repair_table(name, table)
return ret
def db_optimize(name,
table=None):
'''
@ -524,9 +592,8 @@ def db_optimize(name,
ret = __optimize_table(name, table)
return ret
'''
Grants
'''
# Grants
def __grant_generate(grant,
database,
user,
@ -551,6 +618,7 @@ def __grant_generate(grant,
log.debug("Query generated: {0}".format(query,))
return query
def user_grants(user,
host='localhost'):
'''
@ -577,6 +645,7 @@ def user_grants(user,
log.debug(ret)
return ret
def grant_exists(grant,
database,
user,
@ -594,6 +663,7 @@ def grant_exists(grant,
log.debug("Grant does not exist, or is perhaps not ordered properly?")
return False
def grant_add(grant,
database,
user,
@ -623,6 +693,7 @@ def grant_add(grant,
log.info("Grant '{0}' on '{1}' for user '{2}' has NOT been added".format(grant,database,user,))
return False
def grant_revoke(grant,
database,
user,

34
salt/modules/publish.py
View file

@ -2,23 +2,17 @@
Publish a command from a minion to a target
'''
import zmq
# Import python libs
import ast
# Import third party libs
import zmq
# Import salt libs
import salt.crypt
import salt.payload
from salt._compat import string_types
def _get_socket():
'''
Return the ZeroMQ socket to use
'''
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(__opts__['master_uri'])
return socket
from salt.exceptions import SaltReqTimeoutError
def _publish(
tgt,
@ -60,9 +54,9 @@ def _publish(
if isinstance(arg, string_types):
arg = arg.split(',')
sreq = salt.payload.SREQ(__opts__['master_uri'])
auth = salt.crypt.SAuth(__opts__)
tok = auth.gen_token('salt')
payload = {'enc': 'aes'}
load = {
'cmd': 'minion_publish',
'fun': fun,
@ -74,10 +68,8 @@ def _publish(
'tmo': timeout,
'form': form,
'id': __opts__['id']}
payload['load'] = auth.crypticle.dumps(load)
socket = _get_socket()
socket.send(serial.dumps(payload))
return auth.crypticle.loads(serial.loads(socket.recv()))
return auth.crypticle.loads(
sreq.send('aes', auth.crypticle.dumps(load), 1))
def publish(tgt, fun, arg=None, expr_form='glob', returner='', timeout=5):
@ -133,16 +125,14 @@ def runner(fun, arg=None):
if isinstance(arg, string_types):
arg = arg.split(',')
sreq = salt.payload(__opts__['master_uri'])
auth = salt.crypt.SAuth(__opts__)
tok = auth.gen_token('salt')
payload = {'enc': 'aes'}
load = {
'cmd': 'minion_runner',
'fun': fun,
'arg': arg,
'tok': tok,
'id': __opts__['id']}
payload['load'] = auth.crypticle.dumps(load)
socket = _get_socket()
socket.send(serial.dumps(payload))
return auth.crypticle.loads(serial.loads(socket.recv()))
return auth.crypticle.loads(
sreq.send('aes', auth.crypticle.dumps(load), 1))

119
salt/modules/rh_ip.py
View file

@ -38,7 +38,11 @@ _RH_CONFIG_OPTS = [
_RH_CONFIG_BONDING_OPTS = [
'mode', 'miimon', 'arp_interval',
'arp_ip_target', 'downdelay', 'updelay',
'use_carrier', 'lacp_rate', 'hashing-algorithm'
'use_carrier', 'lacp_rate', 'hashing-algorithm',
'max_bonds', 'tx_queues', 'num_grat_arp',
'num_unsol_na', 'primary', 'primary_reselect',
'ad_select', 'xmit_hash_policy', 'arp_validate',
'fail_over_mac', 'all_slaves_active', 'resend_igmp'
]
_RH_NETWORK_SCRIPT_DIR = '/etc/sysconfig/network-scripts'
_RH_NETWORK_FILE = '/etc/sysconfig/network'
@ -57,25 +61,25 @@ def _error_msg_iface(iface, option, expected):
Build an appropriate error message from a given option and
a list of expected values.
'''
msg = 'Invalid option -- Interface: %s, Option: %s, Expected: [%s]'
return msg % (iface, option, '|'.join(expected))
msg = 'Invalid option -- Interface: {0}, Option: {1}, Expected: [{2}]'
return msg.format(iface, option, '|'.join(expected))
def _log_default_iface(iface, opt, value):
msg = 'Using default option -- Interface: %s Option: %s Value: %s'
log.info(msg % (iface, opt, value))
msg = 'Using default option -- Interface: {0} Option: {1} Value: {2}'
log.info(msg.format(iface, opt, value))
def _error_msg_network(option, expected):
'''
Build an appropriate error message from a given option and
a list of expected values.
'''
msg = 'Invalid network setting -- Setting: %s, Expected: [%s]'
return msg % (option, '|'.join(expected))
msg = 'Invalid network setting -- Setting: {0}, Expected: [{1}]'
return msg.format(option, '|'.join(expected))
def _log_default_network(opt, value):
msg = 'Using existing setting -- Setting: %s Value: %s'
log.info(msg % (opt, value))
msg = 'Using existing setting -- Setting: {0} Value: {1}'
log.info(msg.format(opt, value))
def _parse_rh_config(path):
rh_config = _read_file(path)
@ -153,10 +157,18 @@ def _parse_settings_bond(opts, iface):
'''
bond_def = {
# 803.ad aggregation selection logic
# 0 for stable (default)
# 1 for bandwidth
# 2 for count
'ad_select' : '0',
# Max number of transmit queues (default = 16)
'tx_queues' : '16',
# Link monitoring in milliseconds. Most NICs support this
'miimon': '100',
# arp interval in milliseconds
'arp_interval': '250',
# miimon * 2
# Delay before considering link down in milliseconds (miimon * 2)
'downdelay': '200',
# lacp_rate 0: Slow - every 30 seconds
# lacp_rate 1: Fast - every 1 second
@ -176,18 +188,25 @@ def _parse_settings_bond(opts, iface):
}
if opts['mode'] in ['balance-rr', '0']:
log.info('Device: {0} Bonding Mode: load balancing (round-robin)'.format(iface))
return _parse_settings_bond_0(opts, iface, bond_def)
elif opts['mode'] in ['active-backup', '1']:
log.info('Device: {0} Bonding Mode: fault-tolerance (active-backup)'.format(iface))
return _parse_settings_bond_1(opts, iface, bond_def)
elif opts['mode'] in ['balance-xor', '2']:
log.info('Device: {0} Bonding Mode: load balancing (xor)'.format(iface))
return _parse_settings_bond_2(opts, iface, bond_def)
elif opts['mode'] in ['broadcast', '3']:
log.info('Device: {0} Bonding Mode: fault-tolerance (broadcast)'.format(iface))
return _parse_settings_bond_3(opts, iface, bond_def)
elif opts['mode'] in ['802.3ad', '4']:
log.info('Device: {0} Bonding Mode: IEEE 802.3ad Dynamic link aggregation'.format(iface))
return _parse_settings_bond_4(opts, iface, bond_def)
elif opts['mode'] in ['balance-tlb', '5']:
log.info('Device: {0} Bonding Mode: transmit load balancing'.format(iface))
return _parse_settings_bond_5(opts, iface, bond_def)
elif opts['mode'] in ['balance-alb', '6']:
log.info('Device: {0} Bonding Mode: adaptive load balancing'.format(iface))
return _parse_settings_bond_6(opts, iface, bond_def)
else:
valid = [
@ -207,6 +226,7 @@ def _parse_settings_bond_0(opts, iface, bond_def):
'''
bond = {'mode': '0'}
# arp targets in n.n.n.n form
valid = ['list of ips (up to 16)']
if 'arp_ip_target' in opts:
if isinstance(opts['arp_ip_target'], list):
@ -364,7 +384,7 @@ def _parse_settings_bond_4(opts, iface, bond_def):
bond = {'mode': '4'}
for bo in ['miimon', 'downdelay', 'updelay', 'lacp_rate']:
for bo in ['miimon', 'downdelay', 'updelay', 'lacp_rate', 'ad_select']:
if bo in opts:
if bo == 'lacp_rate':
if opts[bo] == 'fast':
@ -477,7 +497,7 @@ def _parse_settings_bond_6(opts, iface, bond_def):
return bond
def _parse_settings_eth(opts, iface_type, iface):
def _parse_settings_eth(opts, iface_type, enabled, iface):
'''
Fiters given options and outputs valid settings for a
network interface.
@ -522,7 +542,7 @@ def _parse_settings_eth(opts, iface_type, iface):
result[opt] = opts[opt]
valid = _CONFIG_TRUE + _CONFIG_FALSE
for opt in ['onboot', 'peerdns', 'slave', 'userctl', 'vlan']:
for opt in ['peerdns', 'slave', 'vlan']:
if opt in opts:
if opts[opt] in _CONFIG_TRUE:
result[opt] = 'yes'
@ -531,6 +551,27 @@ def _parse_settings_eth(opts, iface_type, iface):
else:
_raise_error_iface(iface, opts[opt], valid)
if 'onboot' in opts:
log.warning('''The 'onboot' option is controlled by the 'enabled' option. Interface: {0} Enabled: {1}'''.format(iface, enabled))
if enabled:
result['onboot'] = 'yes'
else:
result['onboot'] = 'no'
# If the interface is defined then we want to always take
# control away from non-root users; unless the administrator
# wants to allow non-root users to control the device.
if 'userctl' in opts:
if opts['userctl'] in _CONFIG_TRUE:
result['userctl'] = 'yes'
elif opts['userctl'] in _CONFIG_FALSE:
result['userctl'] = 'no'
else:
_raise_error_iface(iface, opts['userctl'], valid)
else:
result['userctl'] = 'no'
return result
def _parse_network_settings(opts, current):
@ -610,10 +651,10 @@ def _write_file_iface(iface, data, folder, pattern):
'''
Writes a file to disk
'''
filename = join(folder, pattern % iface)
filename = join(folder, pattern.format(iface))
if not exists(folder):
msg = '%s cannot be written. %s does not exists'
msg = msg % (filename, folder)
msg = '{0} cannot be written. {1} does not exists'
msg = msg.format(filename, folder)
log.error(msg)
raise AttributeError(msg)
fout = open(filename, 'w')
@ -636,19 +677,26 @@ def build_bond(iface, settings):
CLI Example::
salt '*' ip.build_bond br0 mode=balance-alb
salt '*' ip.build_bond bond0 mode=balance-alb
'''
rh_major = __grains__['osrelease'][:1]
rh_minor = __grains__['osrelease'][2:]
opts = _parse_settings_bond(settings, iface)
template = env.get_template('conf.jinja')
data = template.render({'name': iface, 'bonding': opts})
_write_file_iface(iface, data, _RH_NETWORK_CONF_FILES, '%s.conf')
path = join(_RH_NETWORK_CONF_FILES, '%s.conf' % iface)
_write_file_iface(iface, data, _RH_NETWORK_CONF_FILES, '{0}.conf')
path = join(_RH_NETWORK_CONF_FILES, '{0}.conf'.format(iface))
if rh_major == '5':
__salt__['cmd.run']('sed -i -e "/^alias\s{0}.*/d" /etc/modprobe.conf'.format(iface))
__salt__['cmd.run']('sed -i -e "/^options\s{0}.*/d" /etc/modprobe.conf'.format(iface))
__salt__['cmd.run']('cat {0} >> /etc/modprobe.conf'.format(path))
__salt__['kmod.load']('bonding')
return _read_file(path)
def build_interface(iface, iface_type, settings):
def build_interface(iface, iface_type, enabled, settings):
'''
Build an interface script for a network interface.
@ -656,6 +704,9 @@ def build_interface(iface, iface_type, settings):
salt '*' ip.build_interface eth0 eth <settings>
'''
rh_major = __grains__['osrelease'][:1]
rh_minor = __grains__['osrelease'][2:]
if iface_type not in _IFACE_TYPES:
_raise_error_iface(iface, iface_type, _IFACE_TYPES)
@ -670,16 +721,16 @@ def build_interface(iface, iface_type, settings):
settings['vlan'] = 'yes'
if iface_type in ['eth', 'bond', 'slave', 'vlan']:
opts = _parse_settings_eth(settings, iface_type, iface)
template = env.get_template('eth.jinja')
opts = _parse_settings_eth(settings, iface_type, enabled, iface)
template = env.get_template('rh{0}_eth.jinja'.format(rh_major))
ifcfg = template.render(opts)
_write_file_iface(iface, ifcfg, _RH_NETWORK_SCRIPT_DIR, 'ifcfg-%s')
path = join(_RH_NETWORK_SCRIPT_DIR, 'ifcfg-%s' % iface)
_write_file_iface(iface, ifcfg, _RH_NETWORK_SCRIPT_DIR, 'ifcfg-{0}')
path = join(_RH_NETWORK_SCRIPT_DIR, 'ifcfg-{0}'.format(iface))
return _read_file(path)
def down(iface):
def down(iface, iface_type, opts):
'''
Shutdown a network interface
@ -687,7 +738,10 @@ def down(iface):
salt '*' ip.down eth0
'''
__salt__['cmd.run']('ifdown %s' % iface)
# Slave devices are controlled by the master.
if iface_type not in ['slave']:
return __salt__['cmd.run']('ifdown {0}'.format(iface))
return None
def get_bond(iface):
@ -696,9 +750,9 @@ def get_bond(iface):
CLI Example::
salt '*' ip.get_bond br0
salt '*' ip.get_bond bond0
'''
path = join(_RH_NETWORK_CONF_FILES, '%s.conf' % iface)
path = join(_RH_NETWORK_CONF_FILES, '{0}.conf'.format(iface))
return _read_file(path)
@ -710,11 +764,11 @@ def get_interface(iface):
salt '*' ip.get_interface eth0
'''
path = join(_RH_NETWORK_SCRIPT_DIR, 'ifcfg-%s' % iface)
path = join(_RH_NETWORK_SCRIPT_DIR, 'ifcfg-{0}'.format(iface))
return _read_file(path)
def up(iface):
def up(iface, iface_type, opts):
'''
Start up a network interface
@ -722,7 +776,10 @@ def up(iface):
salt '*' ip.up eth0
'''
__salt__['cmd.run']('ifup %s' % iface)
# Slave devices are controlled by the master.
if iface_type not in ['slave']:
return __salt__['cmd.run']('ifup {0}'.format(iface))
return None
def get_network_settings():

19
salt/modules/rh_ip/rh5_eth.jinja Normal file
View file

@ -0,0 +1,19 @@
DEVICE={{name}}
{% if addr %}HWADDR={{addr}}
{%endif%}{% if userctl %}USERCTL={{userctl}}
{%endif%}{% if master %}MASTER={{master}}
{%endif%}{% if slave %}SLAVE={{slave}}
{%endif%}{% if vlan %}VLAN={{vlan}}
{%endif%}{% if proto %}BOOTPROTO={{proto}}
{%endif%}{% if onboot %}ONBOOT={{onboot}}
{%endif%}{% if ipaddr %}IPADDR={{ipaddr}}
{%endif%}{% if netmask %}NETMASK={{netmask}}
{%endif%}{% if gateway %}GATEWAY={{gateway}}
{%endif%}{% if srcaddr %}SRCADDR={{srcaddr}}
{%endif%}{% if peerdns %}PEERDNS={{peerdns}}
{%endif%}{%if bonding %}BONDING_OPTS="{%for item in bonding %}{{item}}={{bonding[item]}} {%endfor%}"
{%endif%}{% if ethtool %}ETHTOOL_OPTS="{%for item in ethtool %}{{item}} {{ethtool[item]}} {%endfor%}"
{%endif%}
{% for server in dns -%}
DNS{{loop.index}}={{server}}
{% endfor -%}

0
salt/modules/rh_ip/eth.jinja → salt/modules/rh_ip/rh6_eth.jinja
View file

1
salt/modules/service.py
View file

@ -13,6 +13,7 @@ grainmap = {
'Ubuntu': '/etc/init.d',
'Gentoo': '/etc/init.d',
'CentOS': '/etc/init.d',
'Amazon': '/etc/init.d',
'SunOS': '/etc/init.d',
}

12
salt/modules/upstart.py
View file

@ -211,6 +211,18 @@ def restart(name):
return not __salt__['cmd.retcode'](cmd)
def reload(name):
'''
Reload the named service
CLI Example::
salt '*' service.reload <service name>
'''
cmd = 'service {0} reload'.format(name)
return not __salt__['cmd.retcode'](cmd)
def status(name, sig=None):
'''
Return the status for a service, returns a bool whether the service is

4
salt/modules/yumpkg.py
View file

@ -30,6 +30,8 @@ def __virtual__():
return 'pkg'
else:
return False
elif __grains__['os'] == 'Amazon':
return 'pkg'
else:
if __grains__['os'] in dists:
if int(__grains__['osrelease'].split('.')[0]) >= 6:
@ -237,7 +239,7 @@ def install(pkgs, refresh=False, repo='', skip_verify=False, **kwargs):
for pkg in pkgs:
try:
yb.install(name=pkg)
except yum.Errors.InstallError:
except Exception:
log.error('Package {0} failed to install'.format(pkg))
# Resolve Deps before attempting install. This needs to be improved
# by also tracking any deps that may get upgraded/installed during this

58
salt/payload.py
View file

@ -4,10 +4,18 @@ encrypted keys to general payload dynamics and packaging, these happen
in here
'''
# Import python libs
import sys
# Import salt libs
import salt.log
import salt.crypt
from salt.exceptions import SaltReqTimeoutError
from salt._compat import pickle
# Import zeromq
import zmq
log = salt.log.logging.getLogger(__name__)
try:
@ -64,8 +72,12 @@ class Serial(object):
serialization in Salt
'''
def __init__(self, opts):
self.opts = opts
self.serial = self.opts.get('serial', 'msgpack')
if isinstance(opts, dict):
self.serial = opts.get('serial', 'msgpack')
elif isinstance(opts, str):
self.serial = opts
else:
self.serial = 'msgpack'
def loads(self, msg):
'''
@ -102,3 +114,45 @@ class Serial(object):
'''
fn_.write(self.dumps(msg))
fn_.close()
class SREQ(object):
'''
Create a generic interface to wrap salt zeromq req calls.
'''
def __init__(self, master, serial='msgpack', linger=0):
self.master = master
self.serial = Serial(serial)
context = zmq.Context()
self.socket = context.socket(zmq.REQ)
self.socket.linger = linger
self.socket.connect(master)
def send(self, enc, load, tries=1, timeout=60):
'''
Takes two arguments, the encryption type and the base payload
'''
payload = {'enc': enc}
payload['load'] = load
package = self.serial.dumps(payload)
self.socket.send(package)
poller = zmq.Poller()
poller.register(self.socket, zmq.POLLIN)
tried = 0
while True:
if not poller.poll(timeout*1000) and tried >= tries:
raise SaltReqTimeoutError('Waited {0} seconds'.format(timeout))
else:
break
tried += 1
ret = self.serial.loads(self.socket.recv())
poller.unregister(self.socket)
return ret
def send_auto(self, payload):
'''
Detect the encryption type based on the payload
'''
enc = payload.get('enc', 'clear')
load = payload.get('load', {})
return self.send(enc, load)

71
salt/pillar.py → salt/pillar/__init__.py
View file

@ -24,40 +24,6 @@ import yaml
log = logging.getLogger(__name__)
def hiera(conf, grains=None):
'''
Execute hiera and return the data
'''
if not isinstance(grains, dict):
grains = {}
cmd = 'hiera {0}'.format(conf)
for key, val in grains.items():
if isinstance(val, string_types):
cmd += ' {0}={1}'.format(key, val)
out = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
shell=True
).communicate()[0]
return yaml.safe_load(out)
def cmd_yaml(command, grains=None):
'''
Execute a command and read the output as YAML
'''
out = subprocess.Popen(
command,
stdout=subprocess.PIPE,
shell=True
).communicate()[0]
return yaml.safe_load(out)
ext_pillar = {'hiera': hiera,
'cmd_yaml': cmd_yaml}
def get_pillar(opts, grains, id_, env=None):
'''
Return the correct pillar driver based on the file_client option
@ -81,30 +47,21 @@ class RemotePillar(object):
self.grains = grains
self.id_ = id_
self.serial = salt.payload.Serial(self.opts)
self.sreq = salt.payload.SREQ(self.opts['master_uri'])
self.auth = salt.crypt.SAuth(opts)
self.socket = self.__get_socket()
def __get_socket(self):
'''
Return the zeromq socket to use
'''
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(self.opts['master_uri'])
return socket
def compile_pillar(self):
'''
Return the pillar data from the master
'''
payload = {'enc': 'aes'}
load = {'id': self.id_,
'grains': self.grains,
'env': self.opts['environment'],
'cmd': '_pillar'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
return self.auth.crypticle.loads(
self.sreq.send('aes', self.auth.crypticle.dumps(load), 3, 7200)
)
class Pillar(object):
@ -116,7 +73,9 @@ class Pillar(object):
self.opts = self.__gen_opts(opts, grains, id_, env)
self.client = salt.fileclient.get_file_client(self.opts)
self.matcher = salt.minion.Matcher(self.opts)
self.rend = salt.loader.render(self.opts, {})
self.functions = salt.loader.minion_mods(self.opts)
self.rend = salt.loader.render(self.opts, self.functions)
self.ext_pillars = salt.loader.pillars(self.opts, self.functions)
def __gen_opts(self, opts, grains, id_, env=None):
'''
@ -354,17 +313,19 @@ class Pillar(object):
if not isinstance(run, dict):
log.critical('The "ext_pillar" option is malformed')
return {}
if len(run) != 1:
log.critical('The "ext_pillar" option is malformed')
return {}
for key, val in run.items():
if key not in ext_pillar:
if key not in self.ext_pillars:
err = ('Specified ext_pillar interface {0} is '
'unavailable').format(key)
log.critical(err)
return {}
continue
try:
ext.update(ext_pillar[key](val, self.opts['grains']))
if isinstance(val, dict):
ext.update(self.ext_pillars[key](**val))
elif isinstance(val, list):
ext.update(self.ext_pillars[key](*val))
else:
ext.update(self.ext_pillars[key](val))
except Exception as e:
log.critical('Failed to load ext_pillar {0}'.format(key))
return ext

14
salt/pillar/cmd_yaml.py Normal file
View file

@ -0,0 +1,14 @@
'''
Execute a command and read the output as YAML. The YAML data is then directly
overlaid onto the minion's pillar data
'''
# Import third party libs
import yaml
def ext_pillar(command):
'''
Execute a command and read the output as YAML
'''
return yaml.safe_load(__salt__['cmd.run'](command))

17
salt/pillar/hiera.py Normal file
View file

@ -0,0 +1,17 @@
'''
Take in a hiera configuration file location and execute it.
Adds the hiera data to pillar
'''
# Import third party libs
import yaml
def ext_pillar(conf):
'''
Execute hiera and return the data
'''
cmd = 'hiera {0}'.format(conf)
for key, val in __grains__.items():
if isinstance(val, string_types):
cmd += ' {0}={1}'.format(key, val)
return yaml.safe_load(__salt__['cmd.run'](cmd))

24
salt/state.py
View file

@ -32,6 +32,7 @@ import salt.fileclient
from salt._compat import string_types, callable
from salt.template import compile_template, compile_template_str
from salt.exceptions import SaltReqTimeoutError
log = logging.getLogger(__name__)
@ -1585,25 +1586,20 @@ class RemoteHighState(object):
self.grains = grains
self.serial = salt.payload.Serial(self.opts)
self.auth = salt.crypt.SAuth(opts)
self.socket = self.__get_socket()
def __get_socket(self):
'''
Return the zeromq socket to use
'''
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect(self.opts['master_uri'])
return socket
def compile_master(self):
'''
Return the state data from the master
'''
payload = {'enc': 'aes'}
load = {'grains': self.grains,
'opts': self.opts,
'cmd': '_master_state'}
payload['load'] = self.auth.crypticle.dumps(load)
self.socket.send(self.serial.dumps(payload))
return self.auth.crypticle.loads(self.serial.loads(self.socket.recv()))
try:
return self.auth.crypticle.loads(sreq.send(
'aes',
self.auth.crypticle.dumps(load),
3,
72000))
except SaltReqTimeoutError:
return {}

2
salt/states/cmd.py
View file

@ -45,6 +45,8 @@ def _run_check(cmd_kwargs, onlyif, unless, cwd, user, group, shell):
'''
Execute the onlyif logic and return data if the onlyif fails
'''
ret = {}
if group:
try:
egid = grp.getgrnam(group).gr_gid

32
salt/states/git.py
View file

@ -30,7 +30,9 @@ def latest(name,
rev=None,
target=None,
runas=None,
force=None):
force=None,
submodules=False,
):
'''
Make sure the repository is cloned to the given directory and is up to date
@ -45,6 +47,8 @@ def latest(name,
Name of the user performing repository management operations
force
Force git to clone into pre-existing directories (deletes contents)
submodules
Update submodules on clone or branch change (Default: False)
'''
ret = {'name': name, 'result': True, 'comment': '', 'changes': {}}
if not target:
@ -67,8 +71,12 @@ def latest(name,
('Repository {0} update is probably required (current '
'revision is {1})').format(target, current_rev))
if rev:
__salt__['git.checkout'](target, rev)
__salt__['git.checkout'](target, rev, user=runas)
__salt__['git.pull'](target, user=runas)
if submodules:
__salt__['git.submodule'](target, user=runas)
new_rev = __salt__['git.revision'](cwd=target, user=runas)
if current_rev != new_rev:
log.info('Repository {0} updated: {1} => {2}'.format(target,
@ -101,13 +109,21 @@ def latest(name,
result = __salt__['git.clone'](target, name, user=runas)
if not os.path.isdir(target):
return _fail(ret, result)
if rev:
__salt__['git.checkout'](target, rev)
else:
message = 'Repository {0} cloned to {1}'.format(name, target)
log.info(message)
ret['comment'] = message
ret['changes']['new'] = name
__salt__['git.checkout'](target, rev, user=runas)
if submodules:
__salt__['git.submodule'](target, user=runas)
new_rev = __salt__['git.revision'](cwd=target, user=runas)
message = 'Repository {0} cloned to {1}'.format(name, target)
log.info(message)
ret['comment'] = message
ret['changes']['new'] = name
ret['changes']['revision'] = new_rev
return ret

35
salt/states/network.py
View file

@ -116,12 +116,7 @@ supported. This module will therefore only work on RH/CentOS/Fedora.
import difflib
def managed(
name,
type,
enabled=True,
**kwargs
):
def managed(name, type, enabled=True, **kwargs):
'''
Ensure that the named interface is configured properly.
@ -153,7 +148,7 @@ def managed(
# Build interface
try:
old = __salt__['ip.get_interface'](name)
new = __salt__['ip.build_interface'](name, type, kwargs)
new = __salt__['ip.build_interface'](name, type, enabled, kwargs)
if __opts__['test']:
if old == new:
return ret
@ -197,9 +192,9 @@ def managed(
#Bring up/shutdown interface
try:
if enabled:
__salt__['ip.up'](name)
__salt__['ip.up'](name, type, kwargs)
else:
__salt__['ip.down'](name)
__salt__['ip.down'](name, type, kwargs)
except Exception as error:
ret['result'] = False
ret['comment'] = error.message
@ -208,10 +203,7 @@ def managed(
return ret
def system(
name,
**kwargs
):
def system(name, **kwargs):
'''
Ensure that global network settings are configured properly.
@ -229,7 +221,7 @@ def system(
'result': True,
'comment': 'Global network settings are up to date.'
}
apply_net_settings = False
# Build global network settings
try:
old = __salt__['ip.get_network_settings']()
@ -247,10 +239,12 @@ def system(
'Global network settings are set to be updated.'
return ret
if not old and new:
apply_net_settings = True
ret['changes']['network_settings'] = \
'Added global network settings.'
elif old != new:
diff = difflib.unified_diff(old, new)
apply_net_settings = True
ret['changes']['network_settings'] = ''.join(diff)
except AttributeError as error:
ret['result'] = False
@ -258,11 +252,12 @@ def system(
return ret
# Apply global network settings
try:
__salt__['ip.apply_network_settings'](kwargs)
except AttributeError as error:
ret['result'] = False
ret['comment'] = error.message
return ret
if apply_net_settings:
try:
__salt__['ip.apply_network_settings'](kwargs)
except AttributeError as error:
ret['result'] = False
ret['comment'] = error.message
return ret
return ret

106
setup.py
View file

@ -22,7 +22,7 @@ if 'SETUPTOOLS' in os.environ:
except:
with_setuptools = False
if with_setuptools == False:
if with_setuptools is False:
from distutils.core import setup
exec(compile(open("salt/version.py").read(), "salt/version.py", 'exec'))
@ -74,63 +74,59 @@ with open('requirements.txt') as f:
requirements = f.read()
setup_kwargs = {
'name': NAME,
'version': VER,
'description': DESC,
'author': 'Thomas S Hatch',
'author_email': 'thatch45@gmail.com',
'url': 'http://saltstack.org',
'cmdclass': {'test': TestCommand},
'classifiers': [
'Programming Language :: Python',
'Programming Language :: Cython',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: Developers',
'Intended Audience :: Information Technology',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: Apache Software License',
'Operating System :: POSIX :: Linux',
'Topic :: System :: Clustering',
'Topic :: System :: Distributed Computing',
],
'packages': [
'salt',
'salt.cli',
'salt.ext',
'salt.grains',
'salt.modules',
'salt.renderers',
'salt.returners',
'salt.runners',
'salt.states',
'salt.utils',
],
'package_data': {
'salt.modules': ['rh_ip/*.jinja'],
},
'data_files': [('share/man/man1',
['doc/man/salt-master.1',
'doc/man/salt-key.1',
'doc/man/salt.1',
'doc/man/salt-cp.1',
'doc/man/salt-call.1',
'doc/man/salt-syndic.1',
'doc/man/salt-run.1',
'doc/man/salt-minion.1',
]),
('share/man/man7', ['doc/man/salt.7']),
],
'install_requires': requirements,
}
setup_kwargs = {'name': NAME,
'version': VER,
'description': DESC,
'author': 'Thomas S Hatch',
'author_email': 'thatch45@gmail.com',
'url': 'http://saltstack.org',
'cmdclass': {'test': TestCommand},
'classifiers': ['Programming Language :: Python',
'Programming Language :: Cython',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: Developers',
'Intended Audience :: Information Technology',
'Intended Audience :: System Administrators',
('License :: OSI Approved ::'
' Apache Software License'),
'Operating System :: POSIX :: Linux',
'Topic :: System :: Clustering',
'Topic :: System :: Distributed Computing',
],
'packages': ['salt',
'salt.cli',
'salt.ext',
'salt.grains',
'salt.modules',
'salt.pillar',
'salt.renderers',
'salt.returners',
'salt.runners',
'salt.states',
'salt.utils',
],
'package_data': {'salt.modules': ['rh_ip/*.jinja']},
'data_files': [('share/man/man1',
['doc/man/salt-master.1',
'doc/man/salt-key.1',
'doc/man/salt.1',
'doc/man/salt-cp.1',
'doc/man/salt-call.1',
'doc/man/salt-syndic.1',
'doc/man/salt-run.1',
'doc/man/salt-minion.1',
]),
('share/man/man7', ['doc/man/salt.7']),
],
'install_requires': requirements,
}
if with_setuptools:
setup_kwargs['entry_points'] = {
"console_scripts": [
"salt-master = salt.scripts:salt_master",
"console_scripts": ["salt-master = salt.scripts:salt_master",
"salt-minion = salt.scripts:salt_minion",
"salt-syndic = salt.scripts:salt_syndic",
"salt-key = salt.scripts:salt_key",

103
tests/integration/__init__.py
View file

@ -82,30 +82,26 @@ class TestDaemon(object):
self._clean()
self.master_opts['hosts.file'] = os.path.join(TMP, 'hosts')
self.minion_opts['hosts.file'] = os.path.join(TMP, 'hosts')
verify_env([
os.path.join(self.master_opts['pki_dir'], 'minions'),
os.path.join(self.master_opts['pki_dir'], 'minions_pre'),
os.path.join(
self.master_opts['pki_dir'], 'minions_rejected'
),
os.path.join(self.master_opts['cachedir'], 'jobs'),
os.path.join(self.smaster_opts['pki_dir'], 'minions'),
os.path.join(
self.smaster_opts['pki_dir'], 'minions_pre'
),
os.path.join(
self.smaster_opts['pki_dir'], 'minions_rejected'
),
os.path.join(self.smaster_opts['cachedir'], 'jobs'),
os.path.dirname(self.master_opts['log_file']),
self.minion_opts['extension_modules'],
self.sub_minion_opts['extension_modules'],
self.sub_minion_opts['pki_dir'],
self.master_opts['sock_dir'],
self.smaster_opts['sock_dir'],
],
pwd.getpwuid(os.getuid())[0]
)
verify_env([os.path.join(self.master_opts['pki_dir'], 'minions'),
os.path.join(self.master_opts['pki_dir'], 'minions_pre'),
os.path.join(self.master_opts['pki_dir'],
'minions_rejected'),
os.path.join(self.master_opts['cachedir'], 'jobs'),
os.path.join(self.smaster_opts['pki_dir'], 'minions'),
os.path.join(self.smaster_opts['pki_dir'], 'minions_pre'),
os.path.join(self.smaster_opts['pki_dir'],
'minions_rejected'),
os.path.join(self.smaster_opts['cachedir'], 'jobs'),
os.path.dirname(self.master_opts['log_file']),
self.minion_opts['extension_modules'],
self.sub_minion_opts['extension_modules'],
self.sub_minion_opts['pki_dir'],
self.master_opts['sock_dir'],
self.smaster_opts['sock_dir'],
self.sub_minion_opts['sock_dir'],
self.minion_opts['sock_dir'],
],
pwd.getpwuid(os.getuid())[0])
master = salt.master.Master(self.master_opts)
self.master_process = multiprocessing.Process(target=master.start)
@ -173,18 +169,22 @@ class ModuleCase(TestCase):
Generate the tools to test a module
'''
self.client = salt.client.LocalClient(
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'master'
)
)
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'master'
)
)
def run_function(self, function, arg=(), **kwargs):
'''
Run a single salt function and condition the return down to match the
behavior of the raw function call
'''
orig = self.client.cmd('minion', function, arg, timeout=5, kwarg=kwargs)
orig = self.client.cmd('minion',
function,
arg,
timeout=100,
kwarg=kwargs)
return orig['minion']
def state_result(self, ret):
@ -205,11 +205,11 @@ class ModuleCase(TestCase):
Return the options used for the minion
'''
return salt.config.minion_config(
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'minion'
)
)
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'minion'
)
)
@property
def master_opts(self):
@ -217,11 +217,11 @@ class ModuleCase(TestCase):
Return the options used for the minion
'''
return salt.config.minion_config(
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'master'
)
)
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'master'
)
)
class SyndicCase(TestCase):
@ -233,11 +233,11 @@ class SyndicCase(TestCase):
Generate the tools to test a module
'''
self.client = salt.client.LocalClient(
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'syndic_master'
)
)
os.path.join(
INTEGRATION_TEST_DIR,
'files', 'conf', 'syndic_master'
)
)
def run_function(self, function, arg=()):
'''
@ -261,11 +261,10 @@ class ShellCase(TestCase):
return False
ppath = 'PYTHONPATH={0}:{1}'.format(CODE_DIR, ':'.join(sys.path[1:]))
cmd = '{0} {1} {2} {3}'.format(ppath, PYEXEC, path, arg_str)
data = subprocess.Popen(
cmd,
shell=True,
stdout=subprocess.PIPE
).communicate()[0].split('\n')
data = subprocess.Popen(cmd,
shell=True,
stdout=subprocess.PIPE
).communicate()[0].split('\n')
return data
def run_salt(self, arg_str):
@ -291,8 +290,8 @@ class ShellCase(TestCase):
'''
ret = {}
ret['out'] = self.run_run(
'{0} {1} {2}'.format(options, fun, ' '.join(arg))
)
'{0} {1} {2}'.format(options, fun, ' '.join(arg))
)
opts = salt.config.master_config(
os.path.join(INTEGRATION_TEST_DIR, 'files', 'conf', 'master'))
opts.update({'doc': False,

1
tests/integration/files/conf/minion
View file

@ -5,6 +5,7 @@ root_dir: /tmp/salttest
pki_dir: pki
id: minion
cachedir: cachedir
sock_dir: minion_sock
#acceptance_wait_time: = 1
open_mode: True
log_file: minion

1
tests/integration/files/conf/sub_minion
View file

@ -5,6 +5,7 @@ root_dir: /tmp/subsalttest
pki_dir: pki
id: sub_minion
cachedir: cachedir
sock_dir: sub_minion_sock
#acceptance_wait_time: 1
open_mode: True
log_file: minion