Normal As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:
for key in self._hosts:
queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key)
if not queue:
continue
now = time.time()
last = self._last_request_times.get(key)
request_delay = REQUEST_DELAY[key]
last_ms = (now - last) * 1000 if last is not None else request_delay
if last_ms >= request_delay:
self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
d = request_delay
queue.popleft()()
else:
d = request_delay - last_ms
self.log.debug("Waiting %d ms before starting another request to %s", d, key)
if d < delay:
delay = d
Should this code be more like:
for priority_queues in (self._high_priority_queues,self._low_priority_queues)
for key in self._hosts:
queue = priority_queues.get(key)
if not queue:
continue
now = time.time()
last = self._last_request_times.get(key)
request_delay = REQUEST_DELAY[key]
last_ms = (now - last) * 1000 if last is not None else request_delay
if last_ms >= request_delay:
self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
d = request_delay
queue.popleft()()
else:
d = request_delay - last_ms
self.log.debug("Waiting %d ms before starting another request to %s", d, key)
if d < delay:
delay = d
If experts agree with this I will code, test and create a pull request for it.
S
High priority webservice queue not actually prioritised
As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:
for key in self._hosts:
queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key)
if not queue:
continue
now = time.time()
last = self._last_request_times.get(key)
request_delay = REQUEST_DELAY[key]
last_ms = (now - last) * 1000 if last is not None else request_delay
if last_ms >= request_delay:
self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
d = request_delay
queue.popleft()()
else:
d = request_delay - last_ms
self.log.debug("Waiting %d ms before starting another request to %s", d, key)
if d < delay:
delay = d
Should this code be more like:
for priority_queues in (self._high_priority_queues,self._low_priority_queues)
for key in self._hosts:
queue = priority_queues.get(key)
if not queue:
continue
now = time.time()
last = self._last_request_times.get(key)
request_delay = REQUEST_DELAY[key]
last_ms = (now - last) * 1000 if last is not None else request_delay
if last_ms >= request_delay:
self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
d = request_delay
queue.popleft()()
else:
d = request_delay - last_ms
self.log.debug("Waiting %d ms before starting another request to %s", d, key)
if d < delay:
delay = d
If experts agree with this I will code, test and create a pull request for it.
S
-
Sophist
-
- Votes:
-
0 Vote for this issue
- Watchers:
-
1 Start watching this issue
- Created:
- Updated:
- Resolved:
| Version | Package |
|---|---|
| 1.3 |