Fix false failure events sent when using syndic

To properly report on nodegroup minions which failed to return, the
master reports minions which don't match the target expression as
"missing". Later, when aggregating returns and writing them to the CLI,
failure events are fired for minions which were reported as missing.

The problem with this approach is that the lower-level minions do not
subscribe directly to the master-of-masters, and thus they are always
marked as missing.

To keep from falsely reporting them as missing, this commit filters out
any minion IDs for which a return was received, so that we only report
on minions which actually failed to return.
This commit is contained in:
Erik Johnson 2018-04-24 09:49:26 -05:00
parent 2ed4b38b02
commit 8012ad12f8
No known key found for this signature in database
GPG key ID: 5E5583C437808F3F

View file

@ -1159,7 +1159,7 @@ class LocalClient(object):
minion_timeouts = {}
found = set()
missing = []
missing = set()
# Check to see if the jid is real, if not return the empty dict
try:
if self.returners['{0}.get_load'.format(self.opts['master_job_cache'])](jid) == {}:
@ -1199,7 +1199,7 @@ class LocalClient(object):
if 'minions' in raw.get('data', {}):
minions.update(raw['data']['minions'])
if 'missing' in raw.get('data', {}):
missing.extend(raw['data']['missing'])
missing.update(raw['data']['missing'])
continue
if 'return' not in raw['data']:
continue
@ -1341,6 +1341,12 @@ class LocalClient(object):
for minion in list((minions - found)):
yield {minion: {'failed': True}}
# Filter out any minions marked as missing for which we received
# returns (prevents false events sent due to higher-level masters not
# knowing about lower-level minions).
missing -= found
# Report on missing minions
if missing:
for minion in missing:
yield {minion: {'failed': True}}