ExportXMLWordPrintable

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Normal
    • None
    • Affects Version/s: None
    • Component/s: Indexer
    • None

      As an operator of a Musicbrainz mirror server on docker desktop on a personal machine, 
      I want `sir reindex` to generate far more concise log messages on errors, 
      Because solr container failures are common, and the present flood of tracebacks hides the relevant error messages.

      Using the `musicbrainz-docker` compose on my local machine, my experience is that the Solr software on the `search` container fails routinely when I am running the `sir reindex` script. The usual failure is that something kills the solr process

      For example,  `sir reindex` for all entity types generates 29 lines of log message when the other containers stay up, but 27,029 lines when Solr fails on the indexer and the database stops responding. Most of those lines are Python backtraces. Most of the backtraces are from chains of exceptions, for instance three backtraces from three exceptions followed by two backtraces from further exceptions raised while handling the first exceptions.

      Here is an example:

       

      2026-06-27 07:36:34,969: Error while submitting data to Solr:
      Traceback (most recent call last):
        File "/usr/local/lib/python3.13/site-packages/urllib3/connection.py", line 204, in _new_conn
          sock = connection.create_connection(
              (self._dns_host, self.port),
      ...[113 lines elided]...
        File "/usr/local/lib/python3.13/site-packages/pysolr.py", line 349, in _send_request
          raise SolrError(error_message % (url, err))
      pysolr.SolrError: Failed to connect to server at http://search:8983/solr/recording/update/: HTTPConnectionPool(host='search', port=8983): Max retries exceeded with url: /solr/recording/update/ (Caused by NameResolutionError("HTTPConnection(host='search', port=8983): Failed to resolve 'search' ([Errno -2] Name or service not known)")) 

       

      This flood of backtraces buries the actual error diagnostics. When running a job at INFO log level, that final line is all I really need to see. 

      I have submitted Pull Request #177 Reduce exception noise in indexing.py logs. Show backtraces less often. This makes changes to sir's `{}main{}.py` and `indexer.py` modules to reduce the volume of log messages greatly. The full backtraces are perhaps of interest to someone diagnosing the problem. When the run `sir` with the `--debug` option, the backtraces appear in the log.  

      [2026-07-13 Edited to add actual PR number, and fix a typo.]

            Assignee:
            Unassigned
            Reporter:
            Jim DeLaHunt
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:

                Version Package