saltstack/salt
1
0
Fork 0
mirror of https://github.com/saltstack/salt.git synced 2025-04-17 10:10:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

saltnado: don't use gather_job_timeout as a timeout signal in job_not_running

Browse source 
because the future was generated from tornado.gen.sleep, its eventual timer
still fires regardless of if the future is manually set with set_result. When
set_result is called twice (once manually by saltnado, once by tornado call_later),
a stacktrace like so is thrown:

[ERROR   ] Uncaught exception POST / (::1)
Traceback (most recent call last):
  File "/home/mphillips81/.pyenv/versions/3.6.5/lib/python3.6/site-packages/tornado/web.py", line 1369, in _stack_context_handle_exception
    raise_exc_info((type, value, traceback))
  File "<string>", line 3, in raise_exc_info
  File "/home/mphillips81/.pyenv/versions/3.6.5/lib/python3.6/site-packages/tornado/stack_context.py", line 314, in wrapped
    ret = fn(*args, **kwargs)
  File "/home/mphillips81/.pyenv/versions/3.6.5/lib/python3.6/site-packages/tornado/gen.py", line 771, in later
    f.set_result(None)
  File "/home/mphillips81/.pyenv/versions/3.6.5/lib/python3.6/site-packages/tornado/concurrent.py", line 254, in set_result
    self._set_done()
  File "/home/mphillips81/.pyenv/versions/3.6.5/lib/python3.6/site-packages/tornado/concurrent.py", line 298, in _set_done
    for cb in self._callbacks:
TypeError: 'NoneType' object is not iterable

This patch attempts to fix that.
This commit is contained in:
Matt Phillips 2018-09-07 15:42:48 -04:00 • committed by Daniel Wozniak
parent 4bcf5d1c81
commit 1f640fc69a
2 changed files with 6 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
changelog/49572.fixed Normal file
View file

@ -0,0 +1 @@
fix frequent rest_tornado non-fatal tracebacks

15
salt/netapi/rest_tornado/saltnado.py
View file

@ -1048,9 +1048,7 @@ class SaltAPIHandler(BaseSaltAPIHandler): # pylint: disable=W0223
)
# To ensure job_not_running and all_return are terminated by each other, communicate using a future
is_finished = salt.ext.tornado.gen.sleep(
self.application.opts["gather_job_timeout"]
)
is_finished = Future()
# ping until the job is not running, while doing so, if we see new minions returning
# that they are running the job, add them to the list
@ -1135,14 +1133,11 @@ class SaltAPIHandler(BaseSaltAPIHandler): # pylint: disable=W0223
tag=ping_tag,
timeout=self.application.opts["gather_job_timeout"],
)
f = yield Any([event, is_finished])
# When finished entire routine, cleanup other futures and return result
if f is is_finished:
if not event.done():
event.set_result(None)
raise salt.ext.tornado.gen.Return(True)
event = f.result()
event = yield event
except TimeoutException:
if not event.done():
event.set_result(None)
if not minion_running:
raise salt.ext.tornado.gen.Return(True)
else: