Uploaded image for project: 'Other'
  1. Other
  2. OTHER-311

BrainzBot crashes on pg server restart

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Normal Normal
    • BrainzBot
    • None

      botbot-web logs the following error whenever the postgres server has been restarted:

      ERROR Line Dispatch Failed                                                                               
      Traceback (most recent call last):                                                                       
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 218, in listen                                                                                                        
          self.dispatch(line)                                                                                  
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 228, in dispatch                                                                                                      
          active_firehose_plugins = line._active_plugin_slugs.intersection(                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 104, in _active_plugin_slugs                                                                                          
          self._active_plugin_slugs_cache = self._channel.active_plugin_slugs                                  
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/bots/models.py", line 207, in active_plugin_slugs                                                                                              
          slug_set = set([actv.plugin.slug for actv in plugins])                                               
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__                                                                                   
          self._fetch_all()                                                                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all                                                                                 
          self._result_cache = list(self.iterator())                                                           
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 238, in iterator                                                                                   
          results = compiler.execute_sql()                                                                     
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql                                                                         
          cursor.execute(sql, params)                                                                          
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django_statsd/patches/utils.py", line 12, in wrapper                                                                                
          return patch_function(original_function, *args, **kw)                                                
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django_statsd/patches/db.py", line 34, in patched_execute                                                                           
          return orig_execute(self, query, *args, **kwargs)                                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute                                                                                   
          return self.cursor.execute(sql, params)                                                              
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__                                                                                           
          six.reraise(dj_exc_type, dj_exc_value, traceback)                                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute                                                                                   
          return self.cursor.execute(sql, params)                                                              
      OperationalError: terminating connection due to administrator command                                    
      SSL connection has been closed unexpectedly                                                              
                                                                                                               
      ERROR Line Dispatch Failed                                                                               
      Traceback (most recent call last):                                                                       
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 218, in listen                                                                                                        
          self.dispatch(line)                                                                                  
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 228, in dispatch                                                                                                      
          active_firehose_plugins = line._active_plugin_slugs.intersection(                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/plugins/runner.py", line 104, in _active_plugin_slugs                                                                                          
          self._active_plugin_slugs_cache = self._channel.active_plugin_slugs                                  
        File "/mnt/disk-1/var/lib/botbotme/botbot/src/botbot/botbot/apps/bots/models.py", line 207, in active_plugin_slugs                                                                                              
          slug_set = set([actv.plugin.slug for actv in plugins])                                               
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__                                                                                   
          self._fetch_all()                                                                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all                                                                                 
          self._result_cache = list(self.iterator())                                                           
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/query.py", line 238, in iterator                                                                                   
          results = compiler.execute_sql()                                                                     
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 838, in execute_sql                                                                         
          cursor = self.connection.cursor()                                                                    
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 164, in cursor                                                                               
          cursor = self.make_cursor(self._cursor())                                                            
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 137, in _cursor                                                                              
          return self.create_cursor()
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
          six.reraise(dj_exc_type, dj_exc_value, traceback)
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 137, in _cursor
          return self.create_cursor()
        File "/mnt/disk-1/var/lib/botbotme/botbot/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 212, in create_cursor
          cursor = self.connection.cursor()
      InterfaceError: connection already closed
      

      InterfacError keeps repeating until botbot-web and botbot-plugins have been restarted.
      Why postgres restarts in the first place is a mystery.

            leo verto Leo Verto
            leo verto Leo Verto
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:

                Version Package