Uploaded image for project: 'Picard'
  1. Picard
  2. PICARD-441

High priority webservice queue not actually prioritised

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Normal Normal
    • 1.3
    • 1.2
    • Other
    • None
    • Win8 64-bit

      As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:

              for key in self._hosts:
                  queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key)
                  if not queue:
                      continue
                  now = time.time()
                  last = self._last_request_times.get(key)
                  request_delay = REQUEST_DELAY[key]
                  last_ms = (now - last) * 1000 if last is not None else request_delay
                  if last_ms >= request_delay:
                      self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
                      d = request_delay
                      queue.popleft()()
                  else:
                      d = request_delay - last_ms
                      self.log.debug("Waiting %d ms before starting another request to %s", d, key)
                  if d < delay:
                      delay = d
      

      Should this code be more like:

      for priority_queues in (self._high_priority_queues,self._low_priority_queues)
              for key in self._hosts:
                  queue = priority_queues.get(key)
                  if not queue:
                      continue
                  now = time.time()
                  last = self._last_request_times.get(key)
                  request_delay = REQUEST_DELAY[key]
                  last_ms = (now - last) * 1000 if last is not None else request_delay
                  if last_ms >= request_delay:
                      self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
                      d = request_delay
                      queue.popleft()()
                  else:
                      d = request_delay - last_ms
                      self.log.debug("Waiting %d ms before starting another request to %s", d, key)
                  if d < delay:
                      delay = d
      

      If experts agree with this I will code, test and create a pull request for it.

      S

          Loading...
          Uploaded image for project: 'Picard'
          1. Picard
          2. PICARD-441

          High priority webservice queue not actually prioritised

            • Icon: Bug Bug
            • Resolution: Fixed
            • Icon: Normal Normal
            • 1.3
            • 1.2
            • Other
            • None
            • Win8 64-bit

              As I understand it, Picard should dispatch work from the high_priority_queues before the low-priority_queues, however reading the code in file: webservice.py function: _run_next_task it seems to me that it handles these with equal priority because for each key in _hosts it checks both queues rather than checking the high_priority queue first and only then looking in the low_priority_queue:

                      for key in self._hosts:
                          queue = self._high_priority_queues.get(key) or self._low_priority_queues.get(key)
                          if not queue:
                              continue
                          now = time.time()
                          last = self._last_request_times.get(key)
                          request_delay = REQUEST_DELAY[key]
                          last_ms = (now - last) * 1000 if last is not None else request_delay
                          if last_ms >= request_delay:
                              self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
                              d = request_delay
                              queue.popleft()()
                          else:
                              d = request_delay - last_ms
                              self.log.debug("Waiting %d ms before starting another request to %s", d, key)
                          if d < delay:
                              delay = d
              

              Should this code be more like:

              for priority_queues in (self._high_priority_queues,self._low_priority_queues)
                      for key in self._hosts:
                          queue = priority_queues.get(key)
                          if not queue:
                              continue
                          now = time.time()
                          last = self._last_request_times.get(key)
                          request_delay = REQUEST_DELAY[key]
                          last_ms = (now - last) * 1000 if last is not None else request_delay
                          if last_ms >= request_delay:
                              self.log.debug("Last request to %s was %d ms ago, starting another one", key, last_ms)
                              d = request_delay
                              queue.popleft()()
                          else:
                              d = request_delay - last_ms
                              self.log.debug("Waiting %d ms before starting another request to %s", d, key)
                          if d < delay:
                              delay = d
              

              If experts agree with this I will code, test and create a pull request for it.

              S

                    sophist Sophist
                    sophist Sophist
                    Votes:
                    0 Vote for this issue
                    Watchers:
                    1 Start watching this issue

                      Created:
                      Updated:
                      Resolved:

                        Version Package
                        1.3

                          sophist Sophist
                          sophist Sophist
                          Votes:
                          0 Vote for this issue
                          Watchers:
                          1 Start watching this issue

                            Created:
                            Updated:
                            Resolved:

                              Version Package
                              1.3