Address docs and hard coded strings

doc/topics/channels/index.rst

@@ -0,0 +1,32 @@
+.. _channels:
+
+=============
+Salt Channels
+=============
+
+One of fundamental features of Salt is remote execution. Salt has two basic
+"channels" for communicating with minions. Each channel requires a
+client (minion) and a server (master) implementation to work within Salt. These
+pairs of channels will work together to implement the specific message passing
+required by the channel interface. Channels use :ref:`Transports <transports>` for sending and
+receiving messages.
+
+
+Pub Channel
+===========
+The pub channel, or publish channel, is how a master sends a job (payload) to a
+minion. This is a basic pub/sub paradigm, which has specific targeting semantics.
+All data which goes across the publish system should be encrypted such that only
+members of the Salt cluster can decrypt the publishes.
+
+
+Req Channel
+===========
+The req channel is how the minions send data to the master. This interface is
+primarily used for fetching files and returning job returns. The req channels
+have two basic interfaces when talking to the master. ``send`` is the basic
+method that guarantees the message is encrypted at least so that only minions
+attached to the same master can read it-- but no guarantee of minion-master
+confidentiality, whereas the ``crypted_transfer_decode_dictentry`` method does
+guarantee minion-master confidentiality. The req channel is also used by the
+salt cli to publish jobs to the master.

doc/topics/configuration/index.rst

@@ -30,6 +30,7 @@ secure and troubleshoot, and how to perform many other administrative tasks.
     ../tutorials/cron
     ../hardening
     ../../security/index
+    ../channels/index
     ../transports/index
     ../master_tops/index
     ../../ref/returners/index

doc/topics/transports/index.rst

@@ -1,33 +1,37 @@
- .. _transports:
+.. _transports:
 
 ==============
 Salt Transport
 ==============
 
-One of fundamental features of Salt is remote execution. Salt has two basic
-"channels" for communicating with minions. Each channel requires a
-client (minion) and a server (master) implementation to work within Salt. These
-pairs of channels will work together to implement the specific message passing
-required by the channel interface.
+
+Transports in Salt are used by :ref:`Channels <channels>` to send messages between Masters Minions
+and the Salt cli. Transports can be brokerless or brokered. There are two types
+of server / client implimentations needed to impliment a channel.
 
 
-Pub Channel
-===========
-The pub channel, or publish channel, is how a master sends a job (payload) to a
-minion. This is a basic pub/sub paradigm, which has specific targeting semantics.
-All data which goes across the publish system should be encrypted such that only
-members of the Salt cluster can decrypt the publishes.
+Publish Server
+==============
+
+The publish server impliments a publish / subscribe paradigm and is used by
+Minions to receive jobs from Masters.
+
+Publish Client
+==============
+
+The publish client subscribes and receives messages from a Publish Server.
 
 
-Req Channel
-===========
-The req channel is how the minions send data to the master. This interface is
-primarily used for fetching files and returning job returns. The req channels
-have two basic interfaces when talking to the master. ``send`` is the basic
-method that guarantees the message is encrypted at least so that only minions
-attached to the same master can read it-- but no guarantee of minion-master
-confidentiality, whereas the ``crypted_transfer_decode_dictentry`` method does
-guarantee minion-master confidentiality.
+Request Server
+==============
+
+The request server impliments a request / reply paradigm. Every request sent by
+the client must recieve exactly one reply.
+
+Request Client
+==============
+
+The request client sends requests to a Request Server and recieves a reply message.
 
 
 .. toctree::

doc/topics/transports/tcp.rst

@@ -2,7 +2,7 @@
 TCP Transport
 =============
 
-The tcp transport is an implementation of Salt's channels using raw tcp sockets.
+The tcp transport is an implementation of Salt's transport using raw tcp sockets.
 Since this isn't using a pre-defined messaging library we will describe the wire
 protocol, message semantics, etc. in this document.
 
@@ -83,17 +83,17 @@ Crypto
 The current implementation uses the same crypto as the ``zeromq`` transport.
 
 
-Pub Channel
-===========
-For the pub channel we send messages without "message ids" which the remote end
-interprets as a one-way send.
+Publish Server and Client
+=========================
+For the publish server and client we send messages without "message ids" which
+the remote end interprets as a one-way send.
 
 .. note::
 
     As of today we send all publishes to all minions and rely on minion-side filtering.
 
 
-Req Channel
-===========
-For the req channel we send messages with a "message id". This "message id" allows
-us to multiplex messages across the socket.
+Request Server and Client
+=========================
+For the request server and client we send messages with a "message id". This
+"message id" allows us to multiplex messages across the socket.

doc/topics/transports/zeromq.rst

@@ -10,17 +10,18 @@ Zeromq is a messaging library with bindings into many languages. Zeromq implemen
 a socket interface for message passing, with specific semantics for the socket type.
 
 
-Pub Channel
-===========
-The pub channel is implemented using zeromq's pub/sub sockets. By default we don't
-use zeromq's filtering, which means that all publish jobs are sent to all minions
-and filtered minion side. Zeromq does have publisher side filtering which can be
-enabled in salt using :conf_master:`zmq_filtering`.
+Publish Server and Client
+=========================
+The publish server and client are implemented using zeromq's pub/sub sockets. By
+default we don't use zeromq's filtering, which means that all publish jobs are
+sent to all minions and filtered minion side. Zeromq does have publisher side
+filtering which can be enabled in salt using :conf_master:`zmq_filtering`.
 
 
-Req Channel
-===========
-The req channel is implemented using zeromq's req/rep sockets. These sockets
-enforce a send/recv pattern, which forces salt to serialize messages through these
-socket pairs. This means that although the interface is asynchronous on the minion
-we cannot send a second message until we have received the reply of the first message.
+Request Server and Client
+=========================
+The request server and client are implemented using zeromq's req/rep sockets.
+These sockets enforce a send/recv pattern, which forces salt to serialize
+messages through these socket pairs. This means that although the interface is
+asynchronous on the minion we cannot send a second message until we have
+received the reply of the first message.

salt/cli/caller.py

@@ -41,21 +41,7 @@ class Caller:
 
     @staticmethod
     def factory(opts, **kwargs):
-        # Default to ZeroMQ for now
-        ttype = "zeromq"
-
-        # determine the ttype
-        if "transport" in opts:
-            ttype = opts["transport"]
-        elif "transport" in opts.get("pillar", {}).get("master", {}):
-            ttype = opts["pillar"]["master"]["transport"]
-
-        # switch on available ttypes
-        if ttype in ("zeromq", "tcp", "detect"):
-            return ZeroMQCaller(opts, **kwargs)
-        else:
-            raise Exception("Callers are only defined for ZeroMQ and TCP")
-            # return NewKindOfCaller(opts, **kwargs)
+        return ZeroMQCaller(opts, **kwargs)
 
 
 class BaseCaller:

salt/cli/daemons.py

@@ -300,8 +300,7 @@ class Minion(
 
         transport = self.config.get("transport").lower()
 
-        # TODO: AIO core is separate from transport
-        if transport in ("zeromq", "tcp", "detect"):
+        try:
             # Late import so logging works correctly
             import salt.minion
 
@@ -314,11 +313,9 @@ class Minion(
             if self.config.get("master_type") == "func":
                 salt.minion.eval_master_func(self.config)
             self.minion = salt.minion.MinionManager(self.config)
-        else:
+        except Exception:  # pylint: disable=broad-except
             log.error(
-                "The transport '%s' is not supported. Please use one of "
-                "the following: tcp, zeromq, or detect.",
-                transport,
+                "An error occured while setting up the minion manager", exc_info=True
             )
             self.shutdown(1)
 

salt/master.py

@@ -58,6 +58,7 @@ import salt.wheel
 from salt.config import DEFAULT_INTERVAL
 from salt.defaults import DEFAULT_TARGET_DELIM
 from salt.ext.tornado.stack_context import StackContext
+from salt.transport import TRANSPORTS
 from salt.utils.channel import iter_transport_opts
 from salt.utils.ctx import RequestContext
 from salt.utils.debug import (
@@ -234,7 +235,7 @@ class Maintenance(salt.utils.process.SignalHandlingProcess):
         if self.opts["key_cache"] == "sched":
             keys = []
             # TODO DRY from CKMinions
-            if self.opts["transport"] in ("zeromq", "tcp"):
+            if self.opts["transport"] in TRANSPORTS:
                 acc = "minions"
             else:
                 acc = "accepted"

salt/minion.py

@@ -35,6 +35,7 @@ import salt.payload
 import salt.pillar
 import salt.serializers.msgpack
 import salt.syspaths
+import salt.transport
 import salt.transport.client
 import salt.utils.args
 import salt.utils.context
@@ -805,7 +806,7 @@ class MinionBase:
                 try:
                     if self.opts["transport"] == "detect":
                         self.opts["detect_mode"] = True
-                        for trans in ("zeromq", "tcp"):
+                        for trans in salt.transport.TRANSPORTS:
                             if trans == "zeromq" and not zmq:
                                 continue
                             self.opts["transport"] = trans

salt/modules/mine.py

@@ -8,6 +8,7 @@ import traceback
 
 import salt.crypt
 import salt.payload
+import salt.transport
 import salt.transport.client
 import salt.utils.args
 import salt.utils.dictupdate
@@ -66,7 +67,7 @@ def _mine_send(load, opts):
 
 
 def _mine_get(load, opts):
-    if opts.get("transport", "") in ("zeromq", "tcp"):
+    if opts.get("transport", "") in salt.transport.TRANSPORTS:
         try:
             load["tok"] = _auth().gen_token(b"salt")
         except AttributeError:

salt/modules/publish.py

@@ -7,6 +7,7 @@ import time
 
 import salt.crypt
 import salt.payload
+import salt.transport
 import salt.transport.client
 import salt.utils.args
 from salt.exceptions import SaltInvocationError, SaltReqTimeoutError
@@ -18,7 +19,9 @@ __virtualname__ = "publish"
 
 def __virtual__():
     return (
-        __virtualname__ if __opts__.get("transport", "") in ("zeromq", "tcp") else False
+        __virtualname__
+        if __opts__.get("transport", "") in salt.transport.TRANSPORTS
+        else False
     )
 
 

salt/output/key.py

@@ -6,6 +6,7 @@ The ``salt-key`` command makes use of this outputter to format its output.
 """
 
 import salt.output
+import salt.transports
 import salt.utils.color
 import salt.utils.data
 
@@ -22,7 +23,7 @@ def output(data, **kwargs):  # pylint: disable=unused-argument
     ident = 0
     if __opts__.get("__multi_key"):
         ident = 4
-    if __opts__["transport"] in ("zeromq", "tcp"):
+    if __opts__["transport"] in salt.transport.TRANSPORTS:
         acc = "minions"
         pend = "minions_pre"
         den = "minions_denied"

salt/transport/__init__.py

@@ -7,6 +7,7 @@ import logging
 import warnings
 
 from salt.transport.base import (
+    TRANSPORTS,
     publish_client,
     publish_server,
     request_client,

salt/transport/base.py

@@ -1,5 +1,11 @@
 import salt.ext.tornado.gen
 
+TRANSPORTS = (
+    "zeromq",
+    "tcp",
+    "rabbitmq",
+)
+
 
 def request_server(opts, **kwargs):
     # Default to ZeroMQ for now
@@ -23,7 +29,7 @@ def request_server(opts, **kwargs):
 
         return salt.transport.tcp.TCPReqServer(opts)
     elif ttype == "rabbitmq":
-        import salt.transport.tcp
+        import salt.transport.rabbitmq
 
         return salt.transport.rabbitmq.RabbitMQReqServer(opts)
     elif ttype == "local":
@@ -80,7 +86,7 @@ def publish_server(opts, **kwargs):
 
         return salt.transport.tcp.TCPPublishServer(opts)
     elif ttype == "rabbitmq":
-        import salt.transport.tcp
+        import salt.transport.rabbitmq
 
         return salt.transport.rabbitmq.RabbitMQPubServer(opts, **kwargs)
     elif ttype == "local":  # TODO:
@@ -108,7 +114,7 @@ def publish_client(opts, io_loop):
 
         return salt.transport.tcp.TCPPubClient(opts, io_loop)
     elif ttype == "rabbitmq":
-        import salt.transport.tcp
+        import salt.transport.rabbitmq
 
         return salt.transport.rabbitmq.RabbitMQPubClient(opts, io_loop)
     raise Exception("Transport type not found: {}".format(ttype))

salt/transport/rabbitmq.py

@@ -792,6 +792,7 @@ class AsyncRabbitMQReqClient(salt.transport.base.RequestClient):
 
     # an init for the singleton instance to call
     def __init__(self, opts, master_uri, io_loop, **kwargs):
+        super().__init__(opts, master_uri, io_loop, **kwargs)
         self.opts = dict(opts)
         if "master_uri" in kwargs:
             self.opts["master_uri"] = kwargs["master_uri"]
@@ -989,6 +990,7 @@ class AsyncRabbitMQPubChannel(salt.transport.base.PublishClient):
     ttype = "rabbitmq"
 
     def __init__(self, opts, **kwargs):
+        super().__init__(opts, **kwargs)
         self.opts = opts
         self.io_loop = (
             kwargs.get("io_loop") or salt.ext.tornado.ioloop.IOLoop.instance()
@@ -1019,7 +1021,7 @@ class AsyncRabbitMQPubChannel(salt.transport.base.PublishClient):
     def __enter__(self):
         return self
 
-    def __exit__(self, *args):
+    def __exit__(self, exc_type, exc_val, exc_tb):
         self.close()
 
     # TODO: this is the time to see if we are connected, maybe use the req channel to guess?

salt/utils/event.py

@@ -147,11 +147,9 @@ def get_master_event(opts, sock_dir, listen=True, io_loop=None, raise_errors=Fal
     """
     Return an event object suitable for the named transport
     """
-    # TODO: AIO core is separate from transport
-    if opts["transport"] in ("zeromq", "tcp", "detect"):
-        return MasterEvent(
-            sock_dir, opts, listen=listen, io_loop=io_loop, raise_errors=raise_errors
-        )
+    return MasterEvent(
+        sock_dir, opts, listen=listen, io_loop=io_loop, raise_errors=raise_errors
+    )
 
 
 def fire_args(opts, jid, tag_data, prefix=""):

salt/utils/minions.py

@@ -13,6 +13,7 @@ import salt.auth.ldap
 import salt.cache
 import salt.payload
 import salt.roster
+import salt.transport
 import salt.utils.data
 import salt.utils.files
 import salt.utils.network
@@ -211,7 +212,7 @@ class CkMinions:
         self.opts = opts
         self.cache = salt.cache.factory(opts)
         # TODO: this is actually an *auth* check
-        if self.opts.get("transport", "zeromq") in ("zeromq", "tcp"):
+        if self.opts.get("transport", "zeromq") in salt.transport.TRANSPORTS:
             self.acc = "minions"
         else:
             self.acc = "accepted"