As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:
for key in self._hosts: queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key) if not queue: continue now = time.time() last = self._last_request_times.get(key) request_delay = REQUEST_DELAY[key] last_ms = (now - last) * 1000 if last is not None else request_delay if last_ms >= request_delay: self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms) d = request_delay queue.popleft()() else: d = request_delay - last_ms self.log.debug("Waiting %d ms before starting another request to %s", d, key) if d < delay: delay = d
Should this code be more like:
for priority_queues in (self._high_priority_queues,self._low_priority_queues) for key in self._hosts: queue = priority_queues.get(key) if not queue: continue now = time.time() last = self._last_request_times.get(key) request_delay = REQUEST_DELAY[key] last_ms = (now - last) * 1000 if last is not None else request_delay if last_ms >= request_delay: self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms) d = request_delay queue.popleft()() else: d = request_delay - last_ms self.log.debug("Waiting %d ms before starting another request to %s", d, key) if d < delay: delay = d
If experts agree with this I will code, test and create a pull request for it.
S
High priority webservice queue not actually prioritised
As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:
for key in self._hosts: queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key) if not queue: continue now = time.time() last = self._last_request_times.get(key) request_delay = REQUEST_DELAY[key] last_ms = (now - last) * 1000 if last is not None else request_delay if last_ms >= request_delay: self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms) d = request_delay queue.popleft()() else: d = request_delay - last_ms self.log.debug("Waiting %d ms before starting another request to %s", d, key) if d < delay: delay = d
Should this code be more like:
for priority_queues in (self._high_priority_queues,self._low_priority_queues) for key in self._hosts: queue = priority_queues.get(key) if not queue: continue now = time.time() last = self._last_request_times.get(key) request_delay = REQUEST_DELAY[key] last_ms = (now - last) * 1000 if last is not None else request_delay if last_ms >= request_delay: self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms) d = request_delay queue.popleft()() else: d = request_delay - last_ms self.log.debug("Waiting %d ms before starting another request to %s", d, key) if d < delay: delay = d
If experts agree with this I will code, test and create a pull request for it.
S