saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-16 01:30:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Fix OS grain inconsistencies if lsb-release is installed or not

Browse source 
Most Linux distributions ship an os-release file by default. Some do not
ship lsb-release information, but they can be installed afterwards.
Installing/Removing lsb-release can lead to different OS grain values.

| OS               | grain      | without lsb-release              | with lsb-release |
|------------------|------------|----------------------------------|------------------|
| AlmaLinux 8      | oscodename | AlmaLinux 8.5 (Arctic Sphynx)    | ArcticSphynx     |
| Astra CE         | os         | Astra (Orel)                     | AstraLinuxCE     |
| Astra CE         | os_family  | Astra (Orel)                     | Debian           |
| Astra CE         | osfullname | Astra Linux (Orel)               | AstraLinuxCE     |
| Astra CE 2.12.40 | osfinger   | Astra Linux (Orel)-2             | AstraLinuxCE-2   |
| Debian           | osfullname | Debian GNU/Linux                 | Debian           |
| Mendel           | osfullname | Mendel GNU/Linux                 | Mendel           |
| Mendel 10        | osfinger   | Mendel GNU/Linux-10              | Mendel-10        |
| Mint             | osfullname | Linux Mint                       | Linuxmint        |
| Mint 20.3        | osfinger   | Linuxmint-20                     | Linux Mint-20    |
| Pop              | osfullname | Pop!_OS                          | Pop              |
| Pop 20.04        | osfinger   | Pop!_OS-20                       | Pop-20           |
| Rocky            | osfullname | Rocky Linux                      | Rocky            |
| Rocky 8          | osfinger   | Rocky Linux-8                    | Rocky-8          |
| Rocky 8          | oscodename | Rocky Linux 8.5 (Green Obsidian) | GreenObsidian    |

The current code that determines the OS grains on Linux is a mess: First
lsb-release is queried. If that fails, fall back to read os-release and
parse some `/etc/*-release` files. Then query `_linux_distribution` and
use a mixtures of those for the OS grains. `_linux_distribution` queries
the Python `distro` library. `distro` queries the os-release file,
lsb-release, and then `/etc/*-release`.

Rewrite the code that determines the OS grains on Linux. Solely rely on
the data provided by the os-release file. To not cause regressions, only
switch the distribution that has been tested. All other distributions
will use the legacy code (which was moved to
`_legacy_linux_distribution_data`).

The new code derives the `os_family` grain from the `ID_LIKE` field from
os-release (see https://github.com/saltstack/salt/issues/59061 for this
feature request). To enable this feature, the new code needs to be used
by default (and not just for selected distributions).

This commit introduces a few changes to the OS grains:

* AlmaLinux and Rocky Linux extract the codename from the `VERSION` field
  now instead of using the full `PRETTY_NAME`.
* Mendel uses now `Mendel GNU/Linux` as `osfullname` and correctly
  extracts the `osrelease` from `PRETTY_NAME`.
* Pop!_OS changes the `osfullname` from `Pop` to `Pop!_OS`.
* Astra Linux changes the `osfullname` from `AstraLinuxCE` to
  `Astra Linux (Orel)` and `AstraLinuxSE` to `Astra Linux (Smolensk)`
  respectively.

Fixes https://github.com/saltstack/salt/issues/61618
Signed-off-by: Benjamin Drung <benjamin.drung@ionos.com>
This commit is contained in:
Benjamin Drung 2022-02-11 15:12:04 +01:00 • committed by Megan Wilhite
parent 733c68646b
commit ca20bb320d
3 changed files with 282 additions and 39 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
changelog/61618.fixed Normal file
View file

@ -0,0 +1,4 @@
Some Linux distributions (like AlmaLinux, Astra Linux, Debian, Mendel, Linux
Mint, Pop!_OS, Rocky Linux) report different `oscodename`, `osfullname`,
`osfinger` grains if lsb-release is installed or not. They have been changed to
only derive these OS grains from `/etc/os-release`.

184
salt/grains/core.py
View file

@ -1685,6 +1685,11 @@ def id_():
return {"id": __opts__.get("id", "")}
# Pattern for os-release PRETTY_NAME containing "name version (codename)"
_PRETTY_NAME_RE = re.compile(r"[^\d]+ (?P<version>\d[\d.+\-a-z]*) \((?P<codename>.+)\)")
# Pattern for os-release VERSION containing "version (codename)"
_VERSION_RE = re.compile(r"\d[\d.+\-a-z]* \((?P<codename>.+)\)")
_REPLACE_LINUX_RE = re.compile(r"\W(?:gnu/)?linux", re.IGNORECASE)
# This maps (at most) the first ten characters (no spaces, lowercased) of
@ -1731,14 +1736,31 @@ _OS_NAME_MAP = {
"mendel": "Mendel",
}
# This dictionary maps the pair of os-release ID and NAME to the 'os' grain
# that Salt traditionally uses, and is used by the os_data() function to
# create the "os" grain.
#
# Add entries to this dictionary to retain historic values of the "os" grain.
_ID_AND_NAME_TO_OS_NAME_MAP = {
("astra", "Astra Linux (Orel)"): "AstraLinuxCE",
("astra", "Astra Linux (Smolensk)"): "AstraLinuxSE",
("pop", "Pop!_OS"): "Pop",
}
def _derive_os_grain(osfullname):
def _derive_os_grain(osfullname, os_id=None):
"""
Derive the 'os' grain from the 'osfullname' grain
For deriving the 'os' grain from the os-release data,
pass NAME as 'osfullname' and ID as 'os_id'.
The 'os' grain that Salt traditionally uses is a shortened
version of the 'osfullname' grain.
"""
if (os_id, osfullname) in _ID_AND_NAME_TO_OS_NAME_MAP:
return _ID_AND_NAME_TO_OS_NAME_MAP[(os_id, osfullname)]
distroname = _REPLACE_LINUX_RE.sub("", osfullname).strip()
# return the first ten characters with no spaces, lowercased
shortname = distroname.replace(" ", "").lower()[:10]
@ -1820,6 +1842,25 @@ _OS_FAMILY_MAP = {
"OSMC": "Debian",
}
# Map the 'family_id' (from os-release) to the 'os_family' grain. If your
# system is having trouble with detection, please make sure that the
# 'family_id' is determined correctly first (in case multiple ID_LIKE entries
# are specified).
_OS_FAMILY_ID_MAP = {
# Red Hat Enterprise Linux (RHEL) is based on Fedora
# and Fedora is the successor of Red Hat Linux (RHL).
"fedora": "RedHat"
}
def _prettify_os_family(family_id):
if family_id in _OS_FAMILY_ID_MAP:
return _OS_FAMILY_ID_MAP[family_id]
# Fall back to use the os_id with an capital starting letter.
return family_id.capitalize()
# Matches any possible format:
# DISTRIB_ID="Ubuntu"
# DISTRIB_ID='Mageia'
@ -2020,6 +2061,91 @@ def _linux_lsb_distrib_data():
return grains, has_error
def _family_id(os_id, id_like):
"""
Return the family ID which is the oldest distribution ancestor.
"""
if not id_like:
# If ID_LIKE is not specified, the distribution has no derivative.
return os_id
ids_like = [os_id] + id_like.split()
# Linux Mint 20.3 does not declare to be a derivative of Debian.
if "debian" in ids_like or "ubuntu" in ids_like:
return "debian"
# The IDs are ordered from closest to farthest.
return ids_like[-1]
def _os_release_quirks_for_oscodename(os_release):
"""
Apply quirks for 'oscodename' grain for faulty os-release files
Some distributions do not (fully) follow the os-release
specification. This function bundles all required quirks
for the 'oscodename' grain. To be on the safe side, only
apply the quirks for allow-listed distributions. Better
not set the codename instead of setting it wrong.
"""
if os_release["ID"] in ("astra",):
# Astra Linux has no version codename, but Salt used
# to report the variant ID as oscodename.
return os_release.get("VARIANT_ID")
if os_release["ID"] in ("almalinux", "rocky"):
# VERSION_CODENAME is not set, but the codename is
# mentioned in PRETTY_NAME and VERSION.
match = _VERSION_RE.match(os_release.get("VERSION", ""))
if match:
return match.group("codename")
return None
def _os_release_quirks_for_osrelease(os_release):
"""
Apply quirks for 'osrelease' grain for faulty os-release files
Some distributions do not (fully) follow the os-release
specification. This function bundles all required quirks
for the 'osrelease' grain. To be on the safe side, only
apply the quirks for allow-listed distributions. Better
not set the release instead of setting it wrong.
"""
if os_release["ID"] in ("mendel",):
# Mendel sets VERSION_CODENAME but not VERSION_ID.
# Only PRETTY_NAME mentions the version number.
match = _PRETTY_NAME_RE.match(os_release["PRETTY_NAME"])
if match:
return match.group("version")
return None
def _os_release_to_grains(os_release):
"""
Transform the given os-release data to grains.
The os-release file is a freedesktop.org standard:
https://www.freedesktop.org/software/systemd/man/os-release.html
The keys NAME, ID, and PRETTY_NAME are expected to exist. All
other keys are optional.
"""
family_id = _family_id(os_release["ID"], os_release.get("ID_LIKE"))
grains = {
"os": _derive_os_grain(os_release["NAME"], os_release["ID"]),
"os_family": _prettify_os_family(family_id),
"oscodename": os_release.get("VERSION_CODENAME")
or _os_release_quirks_for_oscodename(os_release),
"osfullname": os_release["NAME"].strip(),
"osrelease": os_release.get("VERSION_ID")
or _os_release_quirks_for_osrelease(os_release),
}
# oscodename and osrelease could be empty or None. Remove those.
return {key: value for key, value in grains.items() if key}
def _linux_distribution_data():
"""
Determine distribution information like OS name and version.
@ -2033,9 +2159,53 @@ def _linux_distribution_data():
This function might also return lsb_distrib_* grains
from _linux_lsb_distrib_data().
Most Linux distributions should ship a os-release file
and this file should be the sole source for deriving the
OS grains. To not cause regressions, only switch the
distribution that has been tested.
"""
grains, lsb_has_error = _linux_lsb_distrib_data()
log.trace("Getting OS name, release, and codename from freedesktop_os_release")
try:
os_release = _freedesktop_os_release()
grains.update(_os_release_to_grains(os_release))
# To prevent regressions, only let distributions solely
# use os-release after testing.
if os_release["ID"] in (
"almalinux",
"astra",
"debian",
"linuxmint",
"mendel",
"pop",
"rocky",
"ubuntu",
):
# Solely use os-release data. See description of the function.
return grains
except OSError:
os_release = {}
# Warning: The remaining code is legacy code. Please solely rely
# on os-release data. See description of the function.
if "osrelease" in grains:
# Let the legacy code define osrelease to avoid discrepancies.
del grains["osrelease"]
return _legacy_linux_distribution_data(grains, os_release, lsb_has_error)
def _legacy_linux_distribution_data(grains, os_release, lsb_has_error):
"""
Legacy heuristics to determine distribution information.
Most Linux distributions should ship a os-release file
and this file should be the sole source for deriving the
OS grains. See _linux_distribution_data.
"""
if lsb_has_error:
if grains.get("lsb_distrib_description", "").lower().startswith("antergos"):
# Antergos incorrectly configures their /etc/lsb-release,
@ -2044,10 +2214,6 @@ def _linux_distribution_data():
grains["osfullname"] = "Antergos Linux"
elif "lsb_distrib_id" not in grains:
log.trace("Failed to get lsb_distrib_id, trying to parse os-release")
try:
os_release = _freedesktop_os_release()
except OSError:
os_release = {}
if os_release:
if "NAME" in os_release:
grains["lsb_distrib_id"] = os_release["NAME"].strip()
@ -2056,13 +2222,7 @@ def _linux_distribution_data():
if "VERSION_CODENAME" in os_release:
grains["lsb_distrib_codename"] = os_release["VERSION_CODENAME"]
elif "PRETTY_NAME" in os_release:
codename = os_release["PRETTY_NAME"]
# https://github.com/saltstack/salt/issues/44108
if os_release["ID"] == "debian":
codename_match = re.search(r"\((\w+)\)$", codename)
if codename_match:
codename = codename_match.group(1)
grains["lsb_distrib_codename"] = codename
grains["lsb_distrib_codename"] = os_release["PRETTY_NAME"]
if "CPE_NAME" in os_release:
cpe = _parse_cpe_name(os_release["CPE_NAME"])
if not cpe:

133
tests/pytests/unit/grains/test_core.py
View file

@ -1017,7 +1017,7 @@ def test_rocky_8_os_grains():
expectation = {
"os": "Rocky",
"os_family": "RedHat",
"oscodename": "Rocky Linux 8.5 (Green Obsidian)",
"oscodename": "Green Obsidian",
"osfullname": "Rocky Linux",
"osrelease": "8.5",
"osrelease_info": (8, 5),
@ -1052,8 +1052,22 @@ def test_osmc_os_grains():
@pytest.mark.skip_unless_on_linux
def test_mendel_os_grains():
"""
Test if OS grains are parsed correctly in Mendel Linux
Test if OS grains are parsed correctly in Mendel Linux 5.3 Eagle (Nov 2021)
"""
# From https://coral.ai/software/
# downloaded enterprise-eagle-flashcard-20211117215217.zip
# -> flashcard_arm64.img -> rootfs.img -> /etc/os-release
_os_release_data = {
"PRETTY_NAME": "Mendel GNU/Linux 5 (Eagle)",
"NAME": "Mendel GNU/Linux",
"ID": "mendel",
"ID_LIKE": "debian",
"HOME_URL": "https://coral.ai/",
"SUPPORT_URL": "https://coral.ai/",
"BUG_REPORT_URL": "https://coral.ai/",
"VERSION_CODENAME": "eagle",
}
# Note: "lsb_release -a" falsely reports the version to be 10.0
_os_release_map = {
"_linux_distribution": ("Mendel", "10.0", "eagle"),
}
@ -1062,13 +1076,13 @@ def test_mendel_os_grains():
"os": "Mendel",
"os_family": "Debian",
"oscodename": "eagle",
"osfullname": "Mendel",
"osrelease": "10.0",
"osrelease_info": (10, 0),
"osmajorrelease": 10,
"osfinger": "Mendel-10",
"osfullname": "Mendel GNU/Linux",
"osrelease": "5",
"osrelease_info": (5,),
"osmajorrelease": 5,
"osfinger": "Mendel GNU/Linux-5",
}
_run_os_grains_tests(None, _os_release_map, expectation)
_run_os_grains_tests(_os_release_data, _os_release_map, expectation)
@pytest.mark.skip_unless_on_linux
@ -1100,7 +1114,7 @@ def test_almalinux_8_os_grains():
expectation = {
"os": "AlmaLinux",
"os_family": "RedHat",
"oscodename": "AlmaLinux 8.5 (Arctic Sphynx)",
"oscodename": "Arctic Sphynx",
"osfullname": "AlmaLinux",
"osrelease": "8.5",
"osrelease_info": (8, 5),
@ -1273,20 +1287,37 @@ def test_pop_focal_os_grains():
"""
Test if OS grains are parsed correctly in Pop!_OS 20.04 "Focal Fossa"
"""
# /etc/pop-os/os-release data taken from
# pop-default-settings 4.0.6~1642047816~20.04~932caee
_os_release_data = {
"NAME": "Pop!_OS",
"VERSION": "20.04 LTS",
"ID": "pop",
"ID_LIKE": "ubuntu debian",
"PRETTY_NAME": "Pop!_OS 20.04 LTS",
"VERSION_ID": "20.04",
"HOME_URL": "https://pop.system76.com",
"SUPPORT_URL": "https://support.system76.com",
"BUG_REPORT_URL": "https://github.com/pop-os/pop/issues",
"PRIVACY_POLICY_URL": "https://system76.com/privacy",
"VERSION_CODENAME": "focal",
"UBUNTU_CODENAME": "focal",
"LOGO": "distributor-logo-pop-os",
}
_os_release_map = {
"_linux_distribution": ("Pop", "20.04", "focal"),
"_linux_distribution": ("pop", "20.04", "focal"),
}
expectation = {
"os": "Pop",
"os_family": "Debian",
"oscodename": "focal",
"osfullname": "Pop",
"osfullname": "Pop!_OS",
"osrelease": "20.04",
"osrelease_info": (20, 4),
"osmajorrelease": 20,
"osfinger": "Pop-20.04",
"osfinger": "Pop!_OS-20.04",
}
_run_os_grains_tests(None, _os_release_map, expectation)
_run_os_grains_tests(_os_release_data, _os_release_map, expectation)
@pytest.mark.skip_unless_on_linux
@ -1294,20 +1325,37 @@ def test_pop_impish_os_grains():
"""
Test if OS grains are parsed correctly in Pop!_OS 21.10 "Impish Indri"
"""
# /etc/pop-os/os-release data taken from
# pop-default-settings 5.1.0~1640204937~21.10~3f0be51
_os_release_data = {
"NAME": "Pop!_OS",
"VERSION": "21.10",
"ID": "pop",
"ID_LIKE": "ubuntu debian",
"PRETTY_NAME": "Pop!_OS 21.10",
"VERSION_ID": "21.10",
"HOME_URL": "https://pop.system76.com",
"SUPPORT_URL": "https://support.system76.com",
"BUG_REPORT_URL": "https://github.com/pop-os/pop/issues",
"PRIVACY_POLICY_URL": "https://system76.com/privacy",
"VERSION_CODENAME": "impish",
"UBUNTU_CODENAME": "impish",
"LOGO": "distributor-logo-pop-os",
}
_os_release_map = {
"_linux_distribution": ("Pop", "21.10", "impish"),
"_linux_distribution": ("pop", "21.10", "impish"),
}
expectation = {
"os": "Pop",
"os_family": "Debian",
"oscodename": "impish",
"osfullname": "Pop",
"osfullname": "Pop!_OS",
"osrelease": "21.10",
"osrelease_info": (21, 10),
"osmajorrelease": 21,
"osfinger": "Pop-21.10",
"osfinger": "Pop!_OS-21.10",
}
_run_os_grains_tests(None, _os_release_map, expectation)
_run_os_grains_tests(_os_release_data, _os_release_map, expectation)
@pytest.mark.skip_unless_on_linux
@ -1315,20 +1363,37 @@ def test_astralinuxce_os_grains():
"""
Test that OS grains are parsed correctly for Astra Linux Orel
"""
# os-release data taken from astra-version 8.1.24+v2.12.43.6
# found in pool on installer ISO downloaded from
# https://mirrors.edge.kernel.org/astra/stable/orel/iso/orel-current.iso
_os_release_data = {
"PRETTY_NAME": "Astra Linux (Orel 2.12.43)",
"NAME": "Astra Linux (Orel)",
"ID": "astra",
"ID_LIKE": "debian",
"ANSI_COLOR": "1;31",
"HOME_URL": "http://astralinux.ru",
"SUPPORT_URL": "http://astralinux.ru/support",
"VARIANT_ID": "orel",
"VARIANT": "Orel",
"LOGO": "astra",
"VERSION_ID": "2.12.43",
"VERSION_CODENAME": "orel",
}
_os_release_map = {
"_linux_distribution": ("AstraLinuxCE", "2.12.40", "orel"),
"_linux_distribution": ("astra", "2.12.43", "orel"),
}
expectation = {
"os": "AstraLinuxCE",
"os_family": "Debian",
"oscodename": "orel",
"osfullname": "AstraLinuxCE",
"osrelease": "2.12.40",
"osrelease_info": (2, 12, 40),
"osfullname": "Astra Linux (Orel)",
"osrelease": "2.12.43",
"osrelease_info": (2, 12, 43),
"osmajorrelease": 2,
"osfinger": "AstraLinuxCE-2",
"osfinger": "Astra Linux (Orel)-2",
}
_run_os_grains_tests(None, _os_release_map, expectation)
_run_os_grains_tests(_os_release_data, _os_release_map, expectation)
@pytest.mark.skip_unless_on_linux
@ -1336,20 +1401,34 @@ def test_astralinuxse_os_grains():
"""
Test that OS grains are parsed correctly for Astra Linux Smolensk
"""
# /etc/os-release data taken from base-files 7.2astra2
# from Docker image crbrka/astra16se:latest
_os_release_data = {
"PRETTY_NAME": "Astra Linux (Smolensk 1.6)",
"NAME": "Astra Linux (Smolensk)",
"ID": "astra",
"ID_LIKE": "debian",
"ANSI_COLOR": "1;31",
"HOME_URL": "http://astralinux.ru",
"SUPPORT_URL": "http://astralinux.ru/support",
"VARIANT_ID": "smolensk",
"VARIANT": "Smolensk",
"VERSION_ID": "1.6",
}
_os_release_map = {
"_linux_distribution": ("AstraLinuxSE", "1.6", "smolensk"),
"_linux_distribution": ("astra", "1.6", "smolensk"),
}
expectation = {
"os": "AstraLinuxSE",
"os_family": "Debian",
"oscodename": "smolensk",
"osfullname": "AstraLinuxSE",
"osfullname": "Astra Linux (Smolensk)",
"osrelease": "1.6",
"osrelease_info": (1, 6),
"osmajorrelease": 1,
"osfinger": "AstraLinuxSE-1",
"osfinger": "Astra Linux (Smolensk)-1",
}
_run_os_grains_tests(None, _os_release_map, expectation)
_run_os_grains_tests(_os_release_data, _os_release_map, expectation)
@pytest.mark.skip_unless_on_windows