mirror of
https://github.com/saltstack/salt.git
synced 2025-04-17 10:10:20 +00:00
Add detail for writing a returner
This commit is contained in:
Corin Kochenower
parent
cb89231372
commit
f44e25715a
3 changed files with 266 additions and 7 deletions
|
|
@ -374,6 +374,25 @@ local job cache on the master.
|
|||
|
||||
ext_job_cache: redis
|
||||
|
||||
.. conf_master:: event_return
|
||||
|
||||
``event_return``
|
||||
-----------------
|
||||
|
||||
Default: ``''``
|
||||
|
||||
Specify the returner to use to log events. A returner may have installation and
|
||||
configuration requirements. Read the returner's documentation.
|
||||
|
||||
.. note::
|
||||
|
||||
Not all returners support event returns. Verify that a returner has an
|
||||
``event_return()`` function before configuring this option with a returner.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
event_return: cassandra_cql
|
||||
|
||||
.. conf_master:: master_job_cache
|
||||
|
||||
``master_job_cache``
|
||||
|
|
@ -1049,6 +1068,8 @@ Walkthrough <gitfs-per-remote-config>`.
|
|||
Specify the provider to be used for gitfs. More information can be found in the
|
||||
:ref:`GitFS Walkthrough <gitfs-dependencies>`.
|
||||
|
||||
Specify one value among valid values: ``gitpython``, ``pygit2``, ``dulwich``
|
||||
|
||||
.. _pygit2: https://github.com/libgit2/pygit2
|
||||
.. _GitPython: https://github.com/gitpython-developers/GitPython
|
||||
.. _dulwich: https://www.samba.org/~jelmer/dulwich/
|
||||
|
|
|
|||
|
|
@ -45,13 +45,20 @@ test.ping command will be sent out to the three named returners.
|
|||
Writing a Returner
|
||||
==================
|
||||
|
||||
A returner is a Python module which contains a function called ``returner``.
|
||||
A returner is a Python module containing at minimum a ``returner`` function.
|
||||
Other optional functions can be included to add support for
|
||||
:ref:`master_job_cache`, :ref:`external_job_cache`, and `Event Returners`_.
|
||||
|
||||
The ``returner`` function must accept a single argument for the return data from
|
||||
the called minion function. So if the minion function ``test.ping`` is called
|
||||
the value of the argument will be ``True``.
|
||||
``returner``
|
||||
The ``returner`` function must accept a single argument. The argument
|
||||
contains return data from the called minion function. If the minion
|
||||
function ``test.ping`` is called, the value of the argument will be a
|
||||
dictionary. Run the following command from a Salt master to get a sample
|
||||
of the dictionary:
|
||||
|
||||
A simple returner is implemented below:
|
||||
.. code-block:: bash
|
||||
|
||||
salt-call --local --metadata test.ping --out=pprint
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -75,6 +82,209 @@ A simple returner is implemented below:
|
|||
The above example of a returner set to send the data to a Redis server
|
||||
serializes the data as JSON and sets it in redis.
|
||||
|
||||
Master Job Cache Support
|
||||
------------------------
|
||||
|
||||
:ref:`master_job_cache`, :ref:`external_job_cache`, and `Event Returners`_.
|
||||
Salt's :ref:`master_job_cache` allows returners to be used as a pluggable
|
||||
replacement for the :ref:`default_job_cache`. In order to do so, a returner
|
||||
must implement the following functions:
|
||||
|
||||
.. note::
|
||||
|
||||
The code samples contained in this section were taken from the cassandra_cql
|
||||
returner.
|
||||
|
||||
``prep_jid``
|
||||
Ensures that job ids (jid) don't collide, unless passed_jid is provided.
|
||||
|
||||
``nochache`` is an optional boolean that indicates if return data
|
||||
should be cached. ``passed_jid`` is a caller provided jid which should be
|
||||
returned unconditionally.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def prep_jid(nocache, passed_jid=None): # pylint: disable=unused-argument
|
||||
'''
|
||||
Do any work necessary to prepare a JID, including sending a custom id
|
||||
'''
|
||||
return passed_jid if passed_jid is not None else salt.utils.jid.gen_jid()
|
||||
|
||||
``save_load``
|
||||
Save job information. The ``jid`` is generated by ``prep_jid`` and should
|
||||
be considered a unique identifier for the job. The jid, for example, could
|
||||
be used as the primary/unique key in a database. The ``load`` is what is
|
||||
returned to a Salt master by a minion. The following code example stores
|
||||
the load as a JSON string in the salt.jids table.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def save_load(jid, load):
|
||||
'''
|
||||
Save the load to the specified jid id
|
||||
'''
|
||||
query = '''INSERT INTO salt.jids (
|
||||
jid, load
|
||||
) VALUES (
|
||||
'{0}', '{1}'
|
||||
);'''.format(jid, json.dumps(load))
|
||||
|
||||
# cassandra_cql.cql_query may raise a CommandExecutionError
|
||||
try:
|
||||
__salt__['cassandra_cql.cql_query'](query)
|
||||
except CommandExecutionError:
|
||||
log.critical('Could not save load in jids table.')
|
||||
raise
|
||||
except Exception as e:
|
||||
log.critical('''Unexpected error while inserting into
|
||||
jids: {0}'''.format(str(e)))
|
||||
raise
|
||||
|
||||
|
||||
``get_load``
|
||||
must accept a job id (jid) and return the job load stored by ``save_load``,
|
||||
or an empty dictionary when not found.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def get_load(jid):
|
||||
'''
|
||||
Return the load data that marks a specified jid
|
||||
'''
|
||||
query = '''SELECT load FROM salt.jids WHERE jid = '{0}';'''.format(jid)
|
||||
|
||||
ret = {}
|
||||
|
||||
# cassandra_cql.cql_query may raise a CommandExecutionError
|
||||
try:
|
||||
data = __salt__['cassandra_cql.cql_query'](query)
|
||||
if data:
|
||||
load = data[0].get('load')
|
||||
if load:
|
||||
ret = json.loads(load)
|
||||
except CommandExecutionError:
|
||||
log.critical('Could not get load from jids table.')
|
||||
raise
|
||||
except Exception as e:
|
||||
log.critical('''Unexpected error while getting load from
|
||||
jids: {0}'''.format(str(e)))
|
||||
raise
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
External Job Cache Support
|
||||
--------------------------
|
||||
|
||||
Salt's :ref:`external_job_cache` extends the :ref:`master_job_cache`. External
|
||||
Job Cache support requires the following functions in addition to what is
|
||||
required for Master Job Cache support:
|
||||
|
||||
``get_jid``
|
||||
Return a dictionary containing the information (load) returned by each
|
||||
minion when the specified job id was executed.
|
||||
|
||||
Sample:
|
||||
|
||||
.. code-block:: JSON
|
||||
|
||||
{
|
||||
"local": {
|
||||
"master_minion": {
|
||||
"fun_args": [],
|
||||
"jid": "20150330121011408195",
|
||||
"return": true,
|
||||
"retcode": 0,
|
||||
"success": true,
|
||||
"cmd": "_return",
|
||||
"_stamp": "2015-03-30T12:10:12.708663",
|
||||
"fun": "test.ping",
|
||||
"id": "master_minion"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
``get_fun``
|
||||
Return a dictionary of minions that called a given Salt function as their
|
||||
last function call.
|
||||
|
||||
Sample:
|
||||
|
||||
.. code-block:: JSON
|
||||
|
||||
{
|
||||
"local": {
|
||||
"minion1": "test.ping",
|
||||
"minion3": "test.ping",
|
||||
"minion2": "test.ping"
|
||||
}
|
||||
}
|
||||
|
||||
``get_jids``
|
||||
Return a list of all job ids.
|
||||
|
||||
Sample:
|
||||
|
||||
.. code-block:: JSON
|
||||
|
||||
{
|
||||
"local": [
|
||||
"20150330121011408195",
|
||||
"20150330195922139916"
|
||||
]
|
||||
}
|
||||
|
||||
``get_minions``
|
||||
Returns a list of minions
|
||||
|
||||
Sample:
|
||||
|
||||
.. code-block:: JSON
|
||||
|
||||
{
|
||||
"local": [
|
||||
"minion3",
|
||||
"minion2",
|
||||
"minion1",
|
||||
"master_minion"
|
||||
]
|
||||
}
|
||||
|
||||
Please refer to one or more of the existing returners (i.e. mysql,
|
||||
cassandra_cql) if you need further clarification.
|
||||
|
||||
|
||||
Event Support
|
||||
-------------
|
||||
|
||||
An ``event_return`` function must be added to the returner module to allow
|
||||
events to be logged from a master via the returner. A list of events are passed
|
||||
to the function by the master.
|
||||
|
||||
The following example was taken from the MySQL returner. In this example, each
|
||||
event is inserted into the salt_events table keyed on the event tag. The tag
|
||||
contains the jid and therefore is guaranteed to be unique.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def event_return(events):
|
||||
'''
|
||||
Return event to mysql server
|
||||
|
||||
Requires that configuration be enabled via 'event_return'
|
||||
option in master config.
|
||||
'''
|
||||
with _get_serv(events, commit=True) as cur:
|
||||
for event in events:
|
||||
tag = event.get('tag', '')
|
||||
data = event.get('data', '')
|
||||
sql = '''INSERT INTO `salt_events` (`tag`, `data`, `master_id` )
|
||||
VALUES (%s, %s, %s)'''
|
||||
cur.execute(sql, (tag, json.dumps(data), __opts__['id']))
|
||||
|
||||
Custom Returners
|
||||
----------------
|
||||
|
||||
Place custom returners in a ``_returners`` directory within the
|
||||
:conf_master:`file_roots` specified by the master config file.
|
||||
|
||||
|
|
@ -89,6 +299,9 @@ Any custom returners which have been synced to a minion that are named the
|
|||
same as one of Salt's default set of returners will take the place of the
|
||||
default returner with the same name.
|
||||
|
||||
Naming the Returner
|
||||
-------------------
|
||||
|
||||
Note that a returner's default name is its filename (i.e. ``foo.py`` becomes
|
||||
returner ``foo``), but that its name can be overridden by using a
|
||||
:ref:`__virtual__ function <virtual-modules>`. A good example of this can be
|
||||
|
|
@ -110,6 +323,25 @@ loaded as simply ``redis``:
|
|||
return False
|
||||
return __virtualname__
|
||||
|
||||
|
||||
Testing the Returner
|
||||
--------------------
|
||||
|
||||
The ``returner``, ``prep_jid``, ``save_load``, ``get_load``, and
|
||||
``event_return`` functions can be tested by configuring the
|
||||
:ref:`master_job_cache` and `Event Returners`_ in the master config
|
||||
file and submitting a job to ``test.ping`` each minion from the master.
|
||||
|
||||
Once you have successfully exercised the Master Job Cache functions, test the
|
||||
External Job Cache functions using the ``ret`` execution module.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
salt-call ret.get_jids cassandra_cql --output=json
|
||||
salt-call ret.get_fun cassandra_cql test.ping --output=json
|
||||
salt-call ret.get_minions cassandra_cql --output=json
|
||||
salt-call ret.get_jid cassandra_cql 20150330121011408195 --output=json
|
||||
|
||||
Event Returners
|
||||
===============
|
||||
|
||||
|
|
@ -136,4 +368,4 @@ Full List of Returners
|
|||
.. toctree::
|
||||
all/index
|
||||
|
||||
.. _`redis`: https://github.com/saltstack/salt/tree/develop/salt/returners/redis_return.py
|
||||
.. _`redis`: https://github.com/saltstack/salt/tree/develop/salt/returners/redis_return.py
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ The Salt Master maintains a job cache of all job executions which can be
|
|||
queried via the jobs runner. The way this job cache is managed is very
|
||||
pluggable via Salt's underlying returner interface.
|
||||
|
||||
.. _default_job_cache:
|
||||
|
||||
Default Job Cache
|
||||
=================
|
||||
|
||||
|
|
@ -34,6 +36,8 @@ register of executed jobs. Salt comes with two main mechanisms to do this, the
|
|||
master job cache and the external job cache. The difference is how the external
|
||||
data store is accessed.
|
||||
|
||||
.. _master_job_cache:
|
||||
|
||||
Master Job Cache
|
||||
================
|
||||
|
||||
|
|
@ -49,6 +53,8 @@ add the following configuration to the master configuration file:
|
|||
|
||||
master_job_cache: mysql
|
||||
|
||||
.. _external_job_cache:
|
||||
|
||||
External Job Cache
|
||||
==================
|
||||
|
||||
|
|
@ -65,4 +71,4 @@ in the master configuration file:
|
|||
|
||||
.. code-block:: yaml
|
||||
|
||||
ext_job_cache: redis
|
||||
ext_job_cache: redis
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue