Add auto_detect feature to ssh_ext_alternatives

doc/topics/releases/sodium.rst

@@ -54,11 +54,11 @@ Salt-SSH updates
 ssh_pre_flight
 --------------
 
-A new Salt-SSH roster option `ssh_pre_flight` has been added. This enables you to run a
+A new Salt-SSH roster option ``ssh_pre_flight`` has been added. This enables you to run a
 script before Salt-SSH tries to run any commands. You can set this option in the roster
-for a specific minion or use the `roster_defaults` to set it for all minions.
+for a specific minion or use the ``roster_defaults`` to set it for all minions.
 
-Example for setting `ssh_pre_flight` for specific host in roster file
+Example for setting ``ssh_pre_flight`` for specific host in roster file
 
 .. code-block:: yaml
 
@@ -68,7 +68,7 @@ Example for setting `ssh_pre_flight` for specific host in roster file
     passwd: P@ssword
     ssh_pre_flight: /srv/salt/pre_flight.sh
 
-Example for setting `ssh_pre_flight` using roster_defaults, so all minions
+Example for setting ``ssh_pre_flight`` using roster_defaults, so all minions
 run this script.
 
 .. code-block:: yaml
@@ -76,7 +76,7 @@ run this script.
   roster_defaults:
     ssh_pre_flight: /srv/salt/pre_flight.sh
 
-The `ssh_pre_flight` script will only run if the thin dir is not currently on the
+The ``ssh_pre_flight`` script will only run if the thin dir is not currently on the
 minion. If you want to force the script to run you have the following options:
 
 * Wipe the thin dir on the targeted minion using the -w arg.
@@ -99,6 +99,31 @@ You can set this setting in your roster file like so:
     set_path: '$PATH:/usr/local/bin/'
 
 
+auto_detect
+-----------
+
+You can now auto detect the dependencies to be packed into the salt thin when using
+the ``ssh_ext_alternatives`` feature.
+
+.. code-block:: yaml
+
+       ssh_ext_alternatives:
+           2019.2:                     # Namespace, can be anything.
+               py-version: [2, 7]      # Constraint to specific interpreter version
+               path: /opt/2019.2/salt  # Main Salt installation directory.
+               auto_detect: True       # Auto detect dependencies
+               py_bin: /usr/bin/python2.7 # Python binary path used to auto detect dependencies
+
+This new ``auto_detect`` option needs to be set to True in your ``ssh_ext_alternatives`` configuration.
+Salt-ssh will attempt to auto detect the file paths required for the default dependencies to include
+in the thin. If you have a dependency already set in your configuration, it will not attempt to auto
+detect for that dependency.
+
+You can also set the ``py_bin`` option to set the python binary to be used to auto detect the
+dependencies. If ``py_bin`` is not set, it will attempt to use the major Python version set in
+``py-version``. For example, if you set ``py-version`` to be ``[2, 7]`` it will attempt to find and
+use the ``python2`` binary.
+
 State Changes
 =============
 - Adding a new option for the State compiler, ``disabled_requisites`` will allow

doc/topics/ssh/index.rst

@@ -311,3 +311,13 @@ It is recommended that one modify this command a bit by removing the ``-l quiet`
 .. toctree::
 
     roster
+    ssh_ext_alternatives
+
+Different Python Versions
+=========================
+The Sodium release removed python 2 support in Salt. Even though this python 2 support
+is being dropped we have provided multiple ways to work around this with Salt-SSH. You
+can use the following options:
+
+  * :ref:`ssh_pre_flight <ssh_pre_flight>`
+  * :ref:`SSH ext alternatives <ssh-ext-alternatives>`

doc/topics/ssh/ssh_ext_alternatives.rst

@@ -0,0 +1,70 @@
+.. _ssh-ext-alternatives:
+
+====================
+SSH Ext Alternatives
+====================
+
+In the 2019.2.0 release the ``ssh_ext_alternatives`` feature was added.
+This allows salt-ssh to work across different python versions. You will
+need to ensure you have the following:
+
+  - Salt is installed, with all required dependnecies for both Python2 and Python3
+  - Everything needs to be importable from the respective Python environment.
+
+To enable using this feature you will need to edit the master configuration similar
+to below:
+
+.. code-block:: yaml
+
+       ssh_ext_alternatives:
+           2019.2:                     # Namespace, can be anything.
+               py-version: [2, 7]      # Constraint to specific interpreter version
+               path: /opt/2019.2/salt  # Main Salt installation directory.
+               dependencies:           # List of dependencies and their installation paths
+                 jinja2: /opt/jinja2
+                 yaml: /opt/yaml
+                 tornado: /opt/tornado
+                 msgpack: /opt/msgpack
+                 certifi: /opt/certifi
+                 singledispatch: /opt/singledispatch.py
+                 singledispatch_helpers: /opt/singledispatch_helpers.py
+                 markupsafe: /opt/markupsafe
+                 backports_abc: /opt/backports_abc.py
+
+auto_detect
+-----------
+
+In the Sodium release the ``auto_detect`` feature was added for ``ssh_ext_alternatives``.
+This allows salt-ssh to automatically detect the path to all of your dependencies and
+does not require you to define them under ``dependencies``.
+
+.. code-block:: yaml
+
+       ssh_ext_alternatives:
+           2019.2:                     # Namespace, can be anything.
+               py-version: [2, 7]      # Constraint to specific interpreter version
+               path: /opt/2019.2/salt  # Main Salt installation directory.
+               auto_detect: True       # Auto detect dependencies
+               py_bin: /usr/bin/python2.7 # Python binary path used to auto detect dependencies
+
+If ``py_bin`` is not set alongside ``auto_detect``, it will attempt to auto detect
+the dependnecies using the major version set in ``py-version``. For example if you
+have ``[2, 7]`` set as your ``py-version``, it will attempt to use the binary ``python2``.
+
+You can also use ``auto_detect`` and ``dependencies`` together.
+
+.. code-block:: yaml
+
+       ssh_ext_alternatives:
+           2019.2:                     # Namespace, can be anything.
+               py-version: [2, 7]      # Constraint to specific interpreter version
+               path: /opt/2019.2/salt  # Main Salt installation directory.
+               auto_detect: True       # Auto detect dependencies
+               py_bin: /usr/bin/python2.7 # Python binary path to auto detect dependencies
+               dependencies:           # List of dependencies and their installation paths
+                 jinja2: /opt/jinja2
+
+If a dependency is defined in the ``dependecies`` list ``ssh_ext_alternatives`` will use
+this dependency, instead of the path that ``auto_detect`` finds. For example, if you define
+``/opt/jinja2`` under your ``dependencies`` for jinja2, it will not try to autodetect the
+file path to the jinja2 module, and will favor ``/opt/jinja2``.

salt/utils/thin.py

@@ -181,6 +181,62 @@ def gte():
     return salt.utils.json.dumps(tops, ensure_ascii=False)
 
 
+def get_tops_python(py_ver, exclude=None):
+    """
+    Get top directories for the ssh_ext_alternatives dependencies
+    automatically for the given python version. This allows
+    the user to add the dependency paths automatically.
+
+    :param py_ver:
+        python binary to use to detect binaries
+
+    :param exclude:
+        list of modules not to auto detect
+    """
+    files = {}
+    for mod in [
+        "jinja2",
+        "yaml",
+        "tornado",
+        "msgpack",
+        "certifi",
+        "singledispatch",
+        "concurrent",
+        "singledispatch_helpers",
+        "ssl_match_hostname",
+        "markupsafe",
+        "backports_abc",
+    ]:
+        if exclude and mod in exclude:
+            continue
+
+        if not salt.utils.path.which(py_ver):
+            log.error(
+                "{} does not exist. Could not auto detect dependencies".format(py_ver)
+            )
+            return {}
+        py_shell_cmd = "{0} -c 'import {1}; print({1}.__file__)'".format(py_ver, mod)
+        cmd = subprocess.Popen(py_shell_cmd, stdout=subprocess.PIPE, shell=True)
+        stdout, _ = cmd.communicate()
+        mod_file = os.path.abspath(salt.utils.data.decode(stdout).rstrip("\n"))
+
+        if not stdout or not os.path.exists(mod_file):
+            log.error(
+                "Could not auto detect file location for module {} for python version {}".format(
+                    mod, py_ver
+                )
+            )
+            continue
+
+        if os.path.basename(mod_file).split(".")[0] == "__init__":
+            mod_file = os.path.dirname(mod_file)
+        else:
+            mod_file = mod_file.replace("pyc", "py")
+
+        files[mod] = mod_file
+    return files
+
+
 def get_ext_tops(config):
     """
     Get top directories for the dependencies, based on external configuration.
@@ -363,6 +419,76 @@ def _get_thintar_prefix(tarname):
     return tmp_tarname
 
 
+def _pack_alternative(extended_cfg, digest_collector, tfp):
+    # Pack alternative data
+    config = copy.deepcopy(extended_cfg)
+    # Check if auto_detect is enabled and update dependencies
+    for ns, cfg in _six.iteritems(config):
+        if cfg.get("auto_detect"):
+            py_ver = "python" + str(cfg.get("py-version", [""])[0])
+            if cfg.get("py_bin"):
+                py_ver = cfg["py_bin"]
+
+            exclude = []
+            # get any manually set deps
+            deps = config[ns].get("dependencies")
+            if deps:
+                for dep in deps.keys():
+                    exclude.append(dep)
+            else:
+                config[ns]["dependencies"] = {}
+
+            # get auto deps
+            auto_deps = get_tops_python(py_ver, exclude=exclude)
+            for dep in auto_deps:
+                config[ns]["dependencies"][dep] = auto_deps[dep]
+
+    for ns, cfg in _six.iteritems(get_ext_tops(config)):
+        tops = [cfg.get("path")] + cfg.get("dependencies")
+        py_ver_major, py_ver_minor = cfg.get("py-version")
+
+        for top in tops:
+            top = os.path.normpath(top)
+            base, top_dirname = os.path.basename(top), os.path.dirname(top)
+            os.chdir(top_dirname)
+            site_pkg_dir = (
+                _is_shareable(base) and "pyall" or "py{0}".format(py_ver_major)
+            )
+            log.debug(
+                'Packing alternative "%s" to "%s/%s" destination',
+                base,
+                ns,
+                site_pkg_dir,
+            )
+            if not os.path.exists(top):
+                log.error(
+                    "File path {} does not exist. Unable to add to salt-ssh thin".format(
+                        top
+                    )
+                )
+                continue
+            if not os.path.isdir(top):
+                # top is a single file module
+                if os.path.exists(os.path.join(top_dirname, base)):
+                    tfp.add(base, arcname=os.path.join(ns, site_pkg_dir, base))
+                continue
+            for root, dirs, files in salt.utils.path.os_walk(base, followlinks=True):
+                for name in files:
+                    if not name.endswith((".pyc", ".pyo")):
+                        digest_collector.add(os.path.join(root, name))
+                        arcname = os.path.join(ns, site_pkg_dir, root, name)
+                        if hasattr(tfp, "getinfo"):
+                            try:
+                                tfp.getinfo(os.path.join(site_pkg_dir, root, name))
+                                arcname = None
+                            except KeyError:
+                                log.debug(
+                                    'ZIP: Unable to add "%s" with "getinfo"', arcname
+                                )
+                        if arcname:
+                            tfp.add(os.path.join(root, name), arcname=arcname)
+
+
 def gen_thin(
     cachedir,
     extra_mods="",
@@ -597,44 +723,9 @@ def gen_thin(
                 shutil.rmtree(tempdir)
                 tempdir = None
 
-    # Pack alternative data
     if extended_cfg:
         log.debug("Packing libraries based on alternative Salt versions")
-    for ns, cfg in _six.iteritems(get_ext_tops(extended_cfg)):
-        tops = [cfg.get("path")] + cfg.get("dependencies")
-        py_ver_major, py_ver_minor = cfg.get("py-version")
-        for top in tops:
-            base, top_dirname = os.path.basename(top), os.path.dirname(top)
-            os.chdir(top_dirname)
-            site_pkg_dir = (
-                _is_shareable(base) and "pyall" or "py{0}".format(py_ver_major)
-            )
-            log.debug(
-                'Packing alternative "%s" to "%s/%s" destination',
-                base,
-                ns,
-                site_pkg_dir,
-            )
-            if not os.path.isdir(top):
-                # top is a single file module
-                if os.path.exists(os.path.join(top_dirname, base)):
-                    tfp.add(base, arcname=os.path.join(ns, site_pkg_dir, base))
-                continue
-            for root, dirs, files in salt.utils.path.os_walk(base, followlinks=True):
-                for name in files:
-                    if not name.endswith((".pyc", ".pyo")):
-                        digest_collector.add(os.path.join(root, name))
-                        arcname = os.path.join(ns, site_pkg_dir, root, name)
-                        if hasattr(tfp, "getinfo"):
-                            try:
-                                tfp.getinfo(os.path.join(site_pkg_dir, root, name))
-                                arcname = None
-                            except KeyError:
-                                log.debug(
-                                    'ZIP: Unable to add "%s" with "getinfo"', arcname
-                                )
-                        if arcname:
-                            tfp.add(os.path.join(root, name), arcname=arcname)
+        _pack_alternative(extended_cfg, digest_collector, tfp)
 
     os.chdir(thindir)
     with salt.utils.files.fopen(thinver, "w+") as fp_:

tests/unit/utils/test_thin.py

@@ -4,11 +4,15 @@
 """
 from __future__ import absolute_import, print_function, unicode_literals
 
+import copy
 import os
+import shutil
 import sys
 
+import jinja2
 import salt.exceptions
 import salt.ext.six
+import salt.utils.hashutils
 import salt.utils.json
 import salt.utils.platform
 import salt.utils.stringutils
@@ -17,6 +21,7 @@ from salt.utils import thin
 from salt.utils.stringutils import to_bytes as bts
 from tests.support.helpers import TstSuiteLoggingHandler
 from tests.support.mock import MagicMock, patch
+from tests.support.runtests import RUNTIME_VARS
 from tests.support.unit import TestCase, skipIf
 
 try:
@@ -31,6 +36,34 @@ class SSHThinTestCase(TestCase):
     TestCase for SaltSSH-related parts.
     """
 
+    def setUp(self):
+        self.jinja_fp = os.path.dirname(jinja2.__file__)
+
+        self.ext_conf = {
+            "test": {
+                "py-version": [2, 7],
+                "path": os.path.join(RUNTIME_VARS.CODE_DIR, "salt"),
+                "dependencies": {"jinja2": self.jinja_fp},
+            }
+        }
+        self.tops = copy.deepcopy(self.ext_conf)
+        self.tops["test"]["dependencies"] = [self.jinja_fp]
+        self.tar = self._tarfile(None).open()
+        self.digest = salt.utils.hashutils.DigestCollector()
+        self.exp_files = ["salt/payload.py", "jinja2/__init__.py"]
+        lib_root = os.path.join(RUNTIME_VARS.TMP, "fake-libs")
+        self.fake_libs = {
+            "jinja2": os.path.join(lib_root, "jinja2"),
+            "yaml": os.path.join(lib_root, "yaml"),
+            "tornado": os.path.join(lib_root, "tornado"),
+            "msgpack": os.path.join(lib_root, "msgpack"),
+        }
+
+    def tearDown(self):
+        for lib, fp in self.fake_libs.items():
+            if os.path.exists(fp):
+                shutil.rmtree(fp)
+
     def _popen(self, return_value=None, side_effect=None, returncode=0):
         """
         Fake subprocess.Popen
@@ -890,7 +923,18 @@ class SSHThinTestCase(TestCase):
 
         :return:
         """
-        thin.gen_thin("")
+        ext_conf = {
+            "namespace": {
+                "py-version": [2, 7],
+                "path": "/opt/2015.8/salt",
+                "dependencies": {
+                    "certifi": "/opt/certifi",
+                    "whatever": "/opt/whatever",
+                },
+            }
+        }
+
+        thin.gen_thin("", extended_cfg=ext_conf)
         files = []
         for py in ("pyall", "pyall", "py2"):
             for i in range(1, 4):
@@ -951,3 +995,243 @@ class SSHThinTestCase(TestCase):
         )
         for t_line in ["second-system-effect:2:7", "solar-interference:2:6"]:
             self.assertIn(t_line, out)
+
+    def test_get_tops_python(self):
+        """
+        test get_tops_python
+        """
+        patch_proc = patch(
+            "salt.utils.thin.subprocess.Popen",
+            self._popen(
+                None,
+                side_effect=[
+                    (bts("jinja2/__init__.py"), bts("")),
+                    (bts("yaml/__init__.py"), bts("")),
+                    (bts("tornado/__init__.py"), bts("")),
+                    (bts("msgpack/__init__.py"), bts("")),
+                    (bts("certifi/__init__.py"), bts("")),
+                    (bts("singledispatch.py"), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                ],
+            ),
+        )
+
+        exp_ret = {
+            "jinja2": os.path.join(RUNTIME_VARS.CODE_DIR, "jinja2"),
+            "yaml": os.path.join(RUNTIME_VARS.CODE_DIR, "yaml"),
+            "tornado": os.path.join(RUNTIME_VARS.CODE_DIR, "tornado"),
+            "msgpack": os.path.join(RUNTIME_VARS.CODE_DIR, "msgpack"),
+            "certifi": os.path.join(RUNTIME_VARS.CODE_DIR, "certifi"),
+            "singledispatch": os.path.join(RUNTIME_VARS.CODE_DIR, "singledispatch.py"),
+        }
+
+        patch_os = patch("os.path.exists", return_value=True)
+        with patch_proc, patch_os:
+            with TstSuiteLoggingHandler() as log_handler:
+                ret = thin.get_tops_python("python2.7")
+                assert ret == exp_ret
+                assert (
+                    "ERROR:Could not auto detect file location for module concurrent for python version python2.7"
+                    in log_handler.messages
+                )
+
+    def test_get_tops_python_exclude(self):
+        """
+        test get_tops_python when excluding modules
+        """
+        patch_proc = patch(
+            "salt.utils.thin.subprocess.Popen",
+            self._popen(
+                None,
+                side_effect=[
+                    (bts("tornado/__init__.py"), bts("")),
+                    (bts("msgpack/__init__.py"), bts("")),
+                    (bts("certifi/__init__.py"), bts("")),
+                    (bts("singledispatch.py"), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                ],
+            ),
+        )
+
+        exp_ret = {
+            "tornado": os.path.join(RUNTIME_VARS.CODE_DIR, "tornado"),
+            "msgpack": os.path.join(RUNTIME_VARS.CODE_DIR, "msgpack"),
+            "certifi": os.path.join(RUNTIME_VARS.CODE_DIR, "certifi"),
+            "singledispatch": os.path.join(RUNTIME_VARS.CODE_DIR, "singledispatch.py"),
+        }
+
+        patch_os = patch("os.path.exists", return_value=True)
+        with patch_proc, patch_os:
+            ret = thin.get_tops_python("python2.7", exclude=["jinja2", "yaml"])
+            assert ret == exp_ret
+
+    def test_pack_alternatives_exclude(self):
+        """
+        test pack_alternatives when mixing
+        manually set dependencies and auto
+        detecting other modules.
+        """
+        patch_proc = patch(
+            "salt.utils.thin.subprocess.Popen",
+            self._popen(
+                None,
+                side_effect=[
+                    (bts(self.fake_libs["yaml"]), bts("")),
+                    (bts(self.fake_libs["tornado"]), bts("")),
+                    (bts(self.fake_libs["msgpack"]), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                    (bts(""), bts("")),
+                ],
+            ),
+        )
+
+        patch_os = patch("os.path.exists", return_value=True)
+        ext_conf = copy.deepcopy(self.ext_conf)
+        ext_conf["test"]["auto_detect"] = True
+
+        for lib in self.fake_libs.values():
+            os.makedirs(lib)
+            with salt.utils.files.fopen(os.path.join(lib, "__init__.py"), "w+") as fp_:
+                fp_.write("test")
+
+        exp_files = self.exp_files.copy()
+        exp_files.extend(
+            ["yaml/__init__.py", "tornado/__init__.py", "msgpack/__init__.py"]
+        )
+        with patch_os, patch_proc:
+            thin._pack_alternative(ext_conf, self.digest, self.tar)
+            calls = self.tar.mock_calls
+            for _file in exp_files:
+                assert [x for x in calls if "{}".format(_file) in x.args]
+
+    def test_pack_alternatives(self):
+        """
+        test thin._pack_alternatives
+        """
+        with patch("salt.utils.thin.get_ext_tops", MagicMock(return_value=self.tops)):
+            thin._pack_alternative(self.ext_conf, self.digest, self.tar)
+            calls = self.tar.mock_calls
+            for _file in self.exp_files:
+                assert [x for x in calls if "{}".format(_file) in x.args]
+                assert [
+                    x
+                    for x in calls
+                    if "test/pyall/{}".format(_file) in x.kwargs["arcname"]
+                ]
+
+    def test_pack_alternatives_not_normalized(self):
+        """
+        test thin._pack_alternatives when the path
+        is not normalized
+        """
+        tops = copy.deepcopy(self.tops)
+        tops["test"]["dependencies"] = [self.jinja_fp + "/"]
+        with patch("salt.utils.thin.get_ext_tops", MagicMock(return_value=tops)):
+            thin._pack_alternative(self.ext_conf, self.digest, self.tar)
+            calls = self.tar.mock_calls
+            for _file in self.exp_files:
+                assert [x for x in calls if "{}".format(_file) in x.args]
+                assert [
+                    x
+                    for x in calls
+                    if "test/pyall/{}".format(_file) in x.kwargs["arcname"]
+                ]
+
+    def test_pack_alternatives_path_doesnot_exist(self):
+        """
+        test thin._pack_alternatives when the path
+        doesnt exist. Check error log message
+        and expect that because the directory
+        does not exist jinja2 does not get
+        added to the tar
+        """
+        bad_path = "/tmp/doesnotexisthere"
+        tops = copy.deepcopy(self.tops)
+        tops["test"]["dependencies"] = [bad_path]
+        with patch("salt.utils.thin.get_ext_tops", MagicMock(return_value=tops)):
+            with TstSuiteLoggingHandler() as log_handler:
+                thin._pack_alternative(self.ext_conf, self.digest, self.tar)
+                msg = "ERROR:File path {} does not exist. Unable to add to salt-ssh thin".format(
+                    bad_path
+                )
+                assert msg in log_handler.messages
+        calls = self.tar.mock_calls
+        for _file in self.exp_files:
+            arg = [x for x in calls if "{}".format(_file) in x.args]
+            kwargs = [
+                x for x in calls if "test/pyall/{}".format(_file) in x.kwargs["arcname"]
+            ]
+            if "jinja2" in _file:
+                assert not arg
+                assert not kwargs
+            else:
+                assert arg
+                assert kwargs
+
+    def test_pack_alternatives_auto_detect(self):
+        """
+        test thin._pack_alternatives when auto_detect
+        is enabled
+        """
+        ext_conf = copy.deepcopy(self.ext_conf)
+        ext_conf["test"]["auto_detect"] = True
+
+        for lib in self.fake_libs.values():
+            os.makedirs(lib)
+            with salt.utils.files.fopen(os.path.join(lib, "__init__.py"), "w+") as fp_:
+                fp_.write("test")
+
+        patch_tops_py = patch(
+            "salt.utils.thin.get_tops_python", return_value=self.fake_libs
+        )
+
+        exp_files = self.exp_files.copy()
+        exp_files.extend(
+            ["yaml/__init__.py", "tornado/__init__.py", "msgpack/__init__.py"]
+        )
+        with patch_tops_py:
+            thin._pack_alternative(ext_conf, self.digest, self.tar)
+            calls = self.tar.mock_calls
+            for _file in exp_files:
+                assert [x for x in calls if "{}".format(_file) in x.args]
+
+    def test_pack_alternatives_empty_dependencies(self):
+        """
+        test _pack_alternatives when dependencies is not
+        set in the config.
+        """
+        ext_conf = copy.deepcopy(self.ext_conf)
+        ext_conf["test"]["auto_detect"] = True
+        ext_conf["test"].pop("dependencies")
+
+        for lib in self.fake_libs.values():
+            os.makedirs(lib)
+            with salt.utils.files.fopen(os.path.join(lib, "__init__.py"), "w+") as fp_:
+                fp_.write("test")
+
+        patch_tops_py = patch(
+            "salt.utils.thin.get_tops_python", return_value=self.fake_libs
+        )
+
+        exp_files = self.exp_files.copy()
+        exp_files.extend(
+            ["yaml/__init__.py", "tornado/__init__.py", "msgpack/__init__.py"]
+        )
+        with patch_tops_py:
+            thin._pack_alternative(ext_conf, self.digest, self.tar)
+            calls = self.tar.mock_calls
+            for _file in exp_files:
+                assert [x for x in calls if "{}".format(_file) in x.args]