Additional docs on external and master job cache, assorted doc fixes

Refs #25277

doc/contents.rst

@@ -17,6 +17,7 @@ Salt Table of Contents
     topics/eauth/access_control
     topics/jobs/index
     topics/jobs/job_cache
+    topics/jobs/external_cache
     topics/sdb/index
     topics/event/index
     topics/beacons/index

doc/topics/jobs/external_cache.rst

@@ -0,0 +1,163 @@
+.. _external-master-cache:
+
+=========================================
+Storing Job Results in an External System
+=========================================
+
+After a job executes, job results are returned
+to the Salt Master by each Salt Minion. These results are stored in the
+:ref:`Default Job Cache <default_job_cache>`.
+
+In addition to the Default Job Cache, Salt provides two additional
+mechanisms to send job results to other systems (databases, local syslog,
+and others):
+
+* External Job Cache
+* Master Job Cache 
+
+The major different between these two
+mechanism is from where results are returned (from the Salt Master or Salt
+Minion).
+
+External Job Cache - Minion-Side Returner
+-----------------------------------------
+
+When an External Job Cache is configured, data is returned to the Default Job
+Cache on the Salt Master like usual, and then results are also sent to an
+External Job Cache using a Salt returner module running on the Salt Minion.
+
+.. image:: /_static/external-job-cache.png
+    :align: center
+
+* Advantages: Data is stored without placing additional load on the Salt Master.
+* Disadvantages: Each Salt Minion connects to the external job cache, which can
+  result in a large number of connections.  Also requires additional configuration to
+  get returner module settings on all Salt Minions.
+
+Master Job Cache - Master-Side Returner
+---------------------------------------
+
+.. versionadded:: 2014.7
+
+Instead of configuring an External Job Cache on each Salt Minion, you can
+configure the Master Job Cache to send job results from the Salt Master
+instead. In this configuration, Salt Minions send data to the Default Job Cache
+as usual, and then the Salt Master sends the data to the external system using
+a Salt returner module running on the Salt Master.
+
+.. image:: /_static/master-job-cache.png
+    :align: center
+
+* Advantages: A single connection is required to the external system. This is
+  preferred for databases and similar systems.
+* Disadvantages: Places additional load on your Salt Master.
+
+Configure an External or Master Job Cache
+-----------------------------------------
+
+Step 1: Understand Salt Returners
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before you configure a job cache, it is essential to understand Salt returner
+modules ("returners"). Returners are pluggable Salt Modules that take the data
+returned by jobs, and then perform any necessary steps to send the data to an
+external system. For example, a returner might establish a connection,
+authenticate, and then format and transfer data. 
+
+The Salt Returner system provides the core functionality used by the External
+and Master Job Cache systems, and the same returners are used by both systems.
+
+Salt currently provides many different returners that let you connect to a
+wide variety of systems. A complete list is available at
+:ref:`all Salt returners <all-salt.returners>`.
+Each returner is configured differently, so make sure you read and follow the
+instructions linked from that page.
+
+For example, the MySQL returner requires:
+
+* A database created using provided schema (structure is available at
+  :mod:`MySQL returner <salt.returners.mysql>`)
+* A user created with with privileges to the database
+* Optional SSL configuration
+
+A simpler returner, such as Slack or HipChat, requires:
+
+* An API key/version
+* The target channel/room
+* The username that should be used to send the message
+
+Step 2: Configure the Returner
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+After you understand the configuration and have the external system ready, add
+the returner configuration settings to the Salt Minion configuration file for
+the External Job Cache, or to the Salt Master configuration file for the Master
+Job Cache. 
+
+For example, MySQL requires:
+
+.. code-block:: yaml
+
+    mysql.host: 'salt'
+    mysql.user: 'salt'
+    mysql.pass: 'salt'
+    mysql.db: 'salt'
+    mysql.port: 3306
+
+Slack requires:
+
+.. code-block:: yaml
+
+    slack.channel: 'channel' 
+    slack.api_key: 'key'
+    slack.from_name: 'name'
+
+After you have configured the returner and added settings to the configuration
+file, you can enable the External or Master Job Cache.
+
+Step 3: Enable the External or Master Job Cache
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Configuration is a single line that specifies an
+already-configured returner to use to send all job data to an external system.
+
+External Job Cache
+""""""""""""""""""
+
+To enable a returner as the External Job Cache (Minion-side), add the following
+line to the Salt Master configuration file:
+
+.. code-block:: yaml
+
+    ext_job_cache: <returner>
+
+For example:
+
+.. code-block:: yaml
+
+    ext_job_cache: mysql
+
+.. note::
+    When configuring an External Job Cache (Minion-side), the returner settings are
+    added to the Minion configuration file, but the External Job Cache setting
+    is configured in the Master configuration file.
+
+Master Job Cache
+""""""""""""""""
+
+To enable a returner as a Master Job Cache (Master-side), add the following
+line to the Salt Master configuration file:
+
+.. code-block:: yaml
+
+    master_job_cache: <returner>
+
+For example:
+
+.. code-block:: yaml
+
+    master_job_cache: mysql
+
+Verify that the returner configuration settings are in the Master configuration
+file, and be sure to restart the salt-master service after you make
+configuration changes. (``service salt-master restart``).

doc/topics/jobs/job_cache.rst

@@ -1,10 +1,9 @@
-=======================
+======================
 Managing the Job Cache
-=======================
+======================
 
 The Salt Master maintains a job cache of all job executions which can be
-queried via the jobs runner. The way this job cache is managed is very
-pluggable via Salt's underlying returner interface.
+queried via the jobs runner. This job cache is called the Default Job Cache.
 
 .. _default_job_cache:
 
@@ -14,7 +13,7 @@ Default Job Cache
 A number of options are available when configuring the job cache. The default
 caching system uses local storage on the Salt Master and can be found in the
 job cache directory (on Linux systems this is typically
-/var/cache/salt/master/jobs). The default caching system is suitable for most
+``/var/cache/salt/master/jobs``). The default caching system is suitable for most
 deployments as it does not typically require any further configuration or
 management.
 
@@ -28,47 +27,13 @@ Salt Master configuration file. The value passed in is measured via hours:
 
     keep_jobs: 24
 
-External Job Cache Options
-==========================
+Additional Job Cache Options
+============================
 
 Many deployments may wish to use an external database to maintain a long term
 register of executed jobs. Salt comes with two main mechanisms to do this, the
-master job cache and the external job cache. The difference is how the external
-data store is accessed.
+master job cache and the external job cache.
 
-.. _master_job_cache:
+See :ref:`Storing Job Results in an External System <external-master-cache>`.
 
-Master Job Cache
-================
 
-.. versionadded:: 2014.7
-
-The master job cache setting makes the built in job cache on the master
-modular. This system allows for the default cache to be swapped out by the Salt
-returner system. To configure the master job cache, set up an external returner
-database based on the instructions included with each returner and then simply
-add the following configuration to the master configuration file:
-
-.. code-block:: yaml
-
-    master_job_cache: mysql
-
-.. _external_job_cache:
-
-External Job Cache
-==================
-
-The external job cache setting instructs the minions to directly contact the
-data store. This scenario is helpful when the data store needs to be made
-available to the minions. This can be an effective way to share historic data
-across an infrastructure as data can be retrieved from the external job cache
-via the ``ret`` execution module.
-
-To configure the external job cache, set up a returner database in the manner
-described in the specific returner documentation. Ensure that the returner
-database is accessible from the minions, and set the `ext_job_cache` setting
-in the master configuration file:
-
-.. code-block:: yaml
-
-    ext_job_cache: redis

salt/modules/tls.py

@@ -14,16 +14,16 @@ or use Self-Signed certificates.
 CLI Example #1
 Creating a CA, a server request and its signed certificate:
 
-    .. code-block:: bash
+.. code-block:: bash
 
     # salt-call tls.create_ca my_little \
-      days=5 \
-      CN='My Little CA' \
-      C=US \
-      ST=Utah \
-      L=Salt Lake City \
-      O=Saltstack \
-      emailAddress=pleasedontemail@thisisnot.coms
+    days=5 \
+    CN='My Little CA' \
+    C=US \
+    ST=Utah \
+    L=Salt Lake City \
+    O=Saltstack \
+    emailAddress=pleasedontemail@thisisnot.coms
 
     Created Private Key: "/etc/pki/my_little/my_little_ca_cert.key"
     Created CA "my_little_ca": "/etc/pki/my_little_ca/my_little_ca_cert.crt"
@@ -38,7 +38,7 @@ Creating a CA, a server request and its signed certificate:
 CLI Example #2:
 Creating a client request and its signed certificate
 
-    .. code-block:: bash
+.. code-block:: bash
 
     # salt-call tls.create_csr my_little CN=DBReplica_No.1 cert_type=client
     Created Private Key: "/etc/pki/my_little/certs//DBReplica_No.1.key."
@@ -50,7 +50,8 @@ Creating a client request and its signed certificate
 CLI Example #3:
 Creating both a server and client req + cert for the same CN
 
-    .. code-block:: bash
+.. code-block:: bash
+
     # salt-call tls.create_csr my_little CN=MasterDBReplica_No.2  \
         cert_type=client
     Created Private Key: "/etc/pki/my_little/certs/MasterDBReplica_No.2.key."
@@ -83,7 +84,7 @@ Creating both a server and client req + cert for the same CN
 CLI Example #4:
 Create a server req + cert with non-CN filename for the cert
 
-    .. code-block:: bash
+.. code-block:: bash
 
     # salt-call tls.create_csr my_little CN=www.anothersometh.ing \
         cert_type=server type_ext=True
@@ -93,7 +94,6 @@ Create a server req + cert with non-CN filename for the cert
     # salt-call tls_create_ca_signed_cert my_little CN=www.anothersometh.ing \
         cert_type=server cert_filename="something_completely_different"
     Created Certificate for "www.anothersometh.ing": /etc/pki/my_little/certs/something_completely_different.crt
-
 '''
 from __future__ import absolute_import
 # pylint: disable=C0103

salt/netapi/rest_cherrypy/app.py

@@ -8,9 +8,11 @@ A REST API for Salt
 .. py:currentmodule:: salt.netapi.rest_cherrypy.app
 
 :depends:
-    - CherryPy Python module. Versions 3.2.{2,3,4} are strongly
-      recommended due to a known `SSL error
-      <https://bitbucket.org/cherrypy/cherrypy/issue/1298/ssl-not-working>`_
+    - CherryPy Python module. Version 3.2.3 is currently recommended when
+      SSL is enabled, since this version worked the best with SSL in
+      internal testing. Versions 3.2.3 - 4.x can be used if SSL is not enabled.
+      Be aware that there is a known
+      `SSL error <https://bitbucket.org/cherrypy/cherrypy/issue/1298/ssl-not-working>`_
       introduced in version 3.2.5. The issue was reportedly resolved with
       CherryPy milestone 3.3, but the patch was committed for version 3.6.1.
 :optdepends:    - ws4py Python module for websockets support.