Back to index

apport  2.3
Public Member Functions | Public Attributes | Private Member Functions
apport.crashdb.CrashDatabase Class Reference
Inheritance diagram for apport.crashdb.CrashDatabase:
Inheritance graph
[legend]

List of all members.

Public Member Functions

def __init__
def get_bugpattern_baseurl
def accepts
def init_duplicate_db
def check_duplicate
def known
def duplicate_db_fixed
def duplicate_db_remove
def duplicate_db_change_master_id
def duplicate_db_publish
def duplicate_sig_hash
def upload
def get_comment_url
def get_id_url
def download
def update
def update_traces
def set_credentials
def get_distro_release
def get_unretraced
def get_dup_unchecked
def get_unfixed
def get_fixed_version
def get_affected_packages
def is_reporter
def can_update
def duplicate_of
def close_duplicate
def mark_regression
def mark_retraced
def mark_retrace_failed

Public Attributes

 auth_file
 options
 duplicate_db
 format_version

Private Member Functions

def _duplicate_db_upgrade
def _duplicate_search_signature
def _duplicate_search_address_signature
def _duplicate_db_dump
def _duplicate_db_sync_status
def _duplicate_db_add_address_signature
def _duplicate_db_merge_id
def _mark_dup_checked

Detailed Description

Definition at line 37 of file crashdb.py.


Constructor & Destructor Documentation

def apport.crashdb.CrashDatabase.__init__ (   self,
  auth_file,
  options 
)
Initialize crash database connection.

You need to specify an implementation specific file with the
authentication credentials for retracing access for download() and
update(). For upload() and get_comment_url() you can use None.

options is a dictionary with additional settings from crashdb.conf; see
get_crashdb() for details.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 38 of file crashdb.py.

00038 
00039     def __init__(self, auth_file, options):
00040         '''Initialize crash database connection.
00041 
00042         You need to specify an implementation specific file with the
00043         authentication credentials for retracing access for download() and
00044         update(). For upload() and get_comment_url() you can use None.
00045 
00046         options is a dictionary with additional settings from crashdb.conf; see
00047         get_crashdb() for details.
00048         '''
00049         self.auth_file = auth_file
00050         self.options = options
00051         self.duplicate_db = None

Here is the caller graph for this function:


Member Function Documentation

def apport.crashdb.CrashDatabase._duplicate_db_add_address_signature (   self,
  sig,
  id 
) [private]

Definition at line 559 of file crashdb.py.

00559 
00560     def _duplicate_db_add_address_signature(self, sig, id):
00561         # sanity check
00562         existing = self._duplicate_search_address_signature(sig)
00563         if existing:
00564             if existing != id:
00565                 raise SystemError('ID %i has signature %s, but database already has that signature for ID %i' % (
00566                     id, sig, existing))
00567         else:
00568             cur = self.duplicate_db.cursor()
00569             cur.execute('INSERT INTO address_signatures VALUES (?, ?)', (_u(sig), id))
00570             self.duplicate_db.commit()

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase._duplicate_db_dump (   self,
  with_timestamps = False 
) [private]
Return the entire duplicate database as a dictionary.

The returned dictionary maps "signature" to (crash_id, fixed_version)
pairs.

If with_timestamps is True, then the map will contain triples
(crash_id, fixed_version, last_change) instead.

This is mainly useful for debugging and test suites.

Definition at line 500 of file crashdb.py.

00500 
00501     def _duplicate_db_dump(self, with_timestamps=False):
00502         '''Return the entire duplicate database as a dictionary.
00503 
00504         The returned dictionary maps "signature" to (crash_id, fixed_version)
00505         pairs.
00506 
00507         If with_timestamps is True, then the map will contain triples
00508         (crash_id, fixed_version, last_change) instead.
00509 
00510         This is mainly useful for debugging and test suites.
00511         '''
00512         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00513 
00514         dump = {}
00515         cur = self.duplicate_db.cursor()
00516         cur.execute('SELECT * FROM crashes')
00517         for (sig, id, ver, last_change) in cur:
00518             if with_timestamps:
00519                 dump[sig] = (id, ver, last_change)
00520             else:
00521                 dump[sig] = (id, ver)
00522         return dump

def apport.crashdb.CrashDatabase._duplicate_db_merge_id (   self,
  dup,
  master 
) [private]
Merge two crash IDs.

This is necessary when having to mark a bug as a duplicate if it
already is in the duplicate DB.

Definition at line 571 of file crashdb.py.

00571 
00572     def _duplicate_db_merge_id(self, dup, master):
00573         '''Merge two crash IDs.
00574 
00575         This is necessary when having to mark a bug as a duplicate if it
00576         already is in the duplicate DB.
00577         '''
00578         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00579 
00580         cur = self.duplicate_db.cursor()
00581         cur.execute('DELETE FROM crashes WHERE crash_id = ?', [dup])
00582         cur.execute('UPDATE address_signatures SET crash_id = ? WHERE crash_id = ?',
00583                     [master, dup])
00584         self.duplicate_db.commit()

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase._duplicate_db_sync_status (   self,
  id 
) [private]
Update the duplicate db to the reality of the report in the crash db.

This uses get_fixed_version() to get the status of the given crash.
An invalid ID gets removed from the duplicate db, and a crash which got
fixed is marked as such in the database.

Definition at line 523 of file crashdb.py.

00523 
00524     def _duplicate_db_sync_status(self, id):
00525         '''Update the duplicate db to the reality of the report in the crash db.
00526 
00527         This uses get_fixed_version() to get the status of the given crash.
00528         An invalid ID gets removed from the duplicate db, and a crash which got
00529         fixed is marked as such in the database.
00530         '''
00531         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00532 
00533         cur = self.duplicate_db.cursor()
00534         cur.execute('SELECT fixed_version FROM crashes WHERE crash_id = ?', [id])
00535         db_fixed_version = cur.fetchone()
00536         if not db_fixed_version:
00537             return
00538         db_fixed_version = db_fixed_version[0]
00539 
00540         real_fixed_version = self.get_fixed_version(id)
00541 
00542         # crash got rejected
00543         if real_fixed_version == 'invalid':
00544             print('DEBUG: bug %i was invalidated, removing from database' % id)
00545             self.duplicate_db_remove(id)
00546             return
00547 
00548         # crash got fixed
00549         if not db_fixed_version and real_fixed_version:
00550             print('DEBUG: bug %i got fixed in version %s, updating database' % (id, real_fixed_version))
00551             self.duplicate_db_fixed(id, real_fixed_version)
00552             return
00553 
00554         # crash got reopened
00555         if db_fixed_version and not real_fixed_version:
00556             print('DEBUG: bug %i got reopened, dropping fixed version %s from database' % (id, db_fixed_version))
00557             self.duplicate_db_fixed(id, real_fixed_version)
00558             return

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase._duplicate_db_upgrade (   self,
  cur_format 
) [private]
Upgrade database to current format

Definition at line 422 of file crashdb.py.

00422 
00423     def _duplicate_db_upgrade(self, cur_format):
00424         '''Upgrade database to current format'''
00425 
00426         # Format 3 added a primary key which can't be done as an upgrade in
00427         # SQLite
00428         if cur_format < 3:
00429             raise SystemError('Cannot upgrade database from format earlier than 3')
00430 
00431         cur = self.duplicate_db.cursor()
00432 
00433         cur.execute('UPDATE version SET format = ?', (cur_format,))
00434         self.duplicate_db.commit()
00435 
00436         assert cur_format == self.format_version

Return ID for crash address signature.

Return None if signature is unknown.

Definition at line 482 of file crashdb.py.

00482 
00483     def _duplicate_search_address_signature(self, sig):
00484         '''Return ID for crash address signature.
00485 
00486         Return None if signature is unknown.
00487         '''
00488         if not sig:
00489             return None
00490 
00491         cur = self.duplicate_db.cursor()
00492 
00493         cur.execute('SELECT crash_id FROM address_signatures WHERE signature == ?', [sig])
00494         existing_ids = cur.fetchall()
00495         assert len(existing_ids) <= 1
00496         if existing_ids:
00497             return existing_ids[0][0]
00498         else:
00499             return None

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase._duplicate_search_signature (   self,
  sig,
  id 
) [private]
Look up signature in the duplicate db.

Return [(id, fixed_version)] tuple list.

There might be several matches if a crash has been reintroduced in a
later version. The results are sorted so that the highest fixed version
comes first, and "unfixed" being the last result.

id is the bug we are looking to find a duplicate for. The result will
never contain id, to avoid marking a bug as a duplicate of itself if a
bug is reprocessed more than once.

Definition at line 437 of file crashdb.py.

00437 
00438     def _duplicate_search_signature(self, sig, id):
00439         '''Look up signature in the duplicate db.
00440 
00441         Return [(id, fixed_version)] tuple list.
00442 
00443         There might be several matches if a crash has been reintroduced in a
00444         later version. The results are sorted so that the highest fixed version
00445         comes first, and "unfixed" being the last result.
00446 
00447         id is the bug we are looking to find a duplicate for. The result will
00448         never contain id, to avoid marking a bug as a duplicate of itself if a
00449         bug is reprocessed more than once.
00450         '''
00451         cur = self.duplicate_db.cursor()
00452         cur.execute('SELECT crash_id, fixed_version FROM crashes WHERE signature = ? AND crash_id <> ?', [_u(sig), id])
00453         existing = cur.fetchall()
00454 
00455         def cmp(x, y):
00456             x = x[1]
00457             y = y[1]
00458             if x == y:
00459                 return 0
00460             if x == '':
00461                 if y is None:
00462                     return -1
00463                 else:
00464                     return 1
00465             if y == '':
00466                 if x is None:
00467                     return 1
00468                 else:
00469                     return -1
00470             if x is None:
00471                 return 1
00472             if y is None:
00473                 return -1
00474             return apport.packaging.compare_versions(x, y)
00475 
00476         if sys.version[0] >= '3':
00477             existing.sort(key=cmp_to_key(cmp))
00478         else:
00479             existing.sort(cmp=cmp)
00480 
00481         return existing

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase._mark_dup_checked (   self,
  id,
  report 
) [private]
Mark crash id as checked for being a duplicate

This is an internal method that should not be called from outside.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 787 of file crashdb.py.

00787 
00788     def _mark_dup_checked(self, id, report):
00789         '''Mark crash id as checked for being a duplicate
00790 
00791         This is an internal method that should not be called from outside.
00792         '''
00793         raise NotImplementedError('this method must be implemented by a concrete subclass')
00794 
00795 #
00796 # factory
00797 #
00798 

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.accepts (   self,
  report 
)
Check if this report can be uploaded to this database.

Crash databases might limit the types of reports they get with e. g.
the "problem_types" option.

Definition at line 60 of file crashdb.py.

00060 
00061     def accepts(self, report):
00062         '''Check if this report can be uploaded to this database.
00063 
00064         Crash databases might limit the types of reports they get with e. g.
00065         the "problem_types" option.
00066         '''
00067         if 'problem_types' in self.options:
00068             return report.get('ProblemType') in self.options['problem_types']
00069 
00070         return True

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.can_update (   self,
  id 
)
Check whether the user is eligible to update a report.

A user should add additional information to an existing ID if (s)he is
the reporter or subscribed, the bug is open, not a duplicate, etc. The
exact policy and checks should be done according to  the particular
implementation.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 742 of file crashdb.py.

00742 
00743     def can_update(self, id):
00744         '''Check whether the user is eligible to update a report.
00745 
00746         A user should add additional information to an existing ID if (s)he is
00747         the reporter or subscribed, the bug is open, not a duplicate, etc. The
00748         exact policy and checks should be done according to  the particular
00749         implementation.
00750         '''
00751         raise NotImplementedError('this method must be implemented by a concrete subclass')

def apport.crashdb.CrashDatabase.check_duplicate (   self,
  id,
  report = None 
)
Check whether a crash is already known.

If the crash is new, it will be added to the duplicate database and the
function returns None. If the crash is already known, the function
returns a pair (crash_id, fixed_version), where fixed_version might be
None if the crash is not fixed in the latest version yet. Depending on
whether the version in report is smaller than/equal to the fixed
version or larger, this calls close_duplicate() or mark_regression().

If the report does not have a valid crash signature, this function does
nothing and just returns None.

By default, the report gets download()ed, but for performance reasons
it can be explicitly passed to this function if it is already available.

Definition at line 133 of file crashdb.py.

00133 
00134     def check_duplicate(self, id, report=None):
00135         '''Check whether a crash is already known.
00136 
00137         If the crash is new, it will be added to the duplicate database and the
00138         function returns None. If the crash is already known, the function
00139         returns a pair (crash_id, fixed_version), where fixed_version might be
00140         None if the crash is not fixed in the latest version yet. Depending on
00141         whether the version in report is smaller than/equal to the fixed
00142         version or larger, this calls close_duplicate() or mark_regression().
00143 
00144         If the report does not have a valid crash signature, this function does
00145         nothing and just returns None.
00146 
00147         By default, the report gets download()ed, but for performance reasons
00148         it can be explicitly passed to this function if it is already available.
00149         '''
00150         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00151 
00152         if not report:
00153             report = self.download(id)
00154 
00155         self._mark_dup_checked(id, report)
00156 
00157         if 'DuplicateSignature' in report:
00158             sig = report['DuplicateSignature']
00159         else:
00160             sig = report.crash_signature()
00161         existing = []
00162         if sig:
00163             # use real duplicate signature
00164             existing = self._duplicate_search_signature(sig, id)
00165 
00166             if existing:
00167                 # update status of existing master bugs
00168                 for (ex_id, _) in existing:
00169                     self._duplicate_db_sync_status(ex_id)
00170                 existing = self._duplicate_search_signature(sig, id)
00171 
00172         try:
00173             report_package_version = report['Package'].split()[1]
00174         except (KeyError, IndexError):
00175             report_package_version = None
00176 
00177         # check the existing IDs whether there is one that is unfixed or not
00178         # older than the report's package version; if so, we have a duplicate.
00179         master_id = None
00180         master_ver = None
00181         for (ex_id, ex_ver) in existing:
00182             if not ex_ver or not report_package_version or apport.packaging.compare_versions(report_package_version, ex_ver) < 0:
00183                 master_id = ex_id
00184                 master_ver = ex_ver
00185                 break
00186         else:
00187             # if we did not find a new enough open master report,
00188             # we have a regression of the latest fix. Mark it so, and create a
00189             # new unfixed ID for it later on
00190             if existing:
00191                 self.mark_regression(id, existing[-1][0])
00192 
00193         # now query address signatures, they might turn up another duplicate
00194         # (not necessarily the same, due to Stacktraces sometimes being
00195         # slightly different)
00196         addr_sig = report.crash_signature_addresses()
00197         if addr_sig:
00198             addr_match = self._duplicate_search_address_signature(addr_sig)
00199             if addr_match and addr_match != master_id:
00200                 if master_id is None:
00201                     # we have a duplicate only identified by address sig, close it
00202                     master_id = addr_match
00203                 else:
00204                     # our bug is a dupe of two different masters, one from
00205                     # symbolic, the other from addr matching (see LP#943117);
00206                     # make them all duplicates of each other, using the lower
00207                     # number as master
00208                     if master_id < addr_match:
00209                         self.close_duplicate(report, addr_match, master_id)
00210                         self._duplicate_db_merge_id(addr_match, master_id)
00211                     else:
00212                         self.close_duplicate(report, master_id, addr_match)
00213                         self._duplicate_db_merge_id(master_id, addr_match)
00214                         master_id = addr_match
00215                         master_ver = None  # no version tracking for address signatures yet
00216 
00217         if master_id is not None:
00218             if addr_sig:
00219                 self._duplicate_db_add_address_signature(addr_sig, master_id)
00220             self.close_duplicate(report, id, master_id)
00221             return (master_id, master_ver)
00222 
00223         # no duplicate detected; create a new record for the ID if we don't have one already
00224         if sig:
00225             cur = self.duplicate_db.cursor()
00226             cur.execute('SELECT count(*) FROM crashes WHERE crash_id == ?', [id])
00227             count_id = cur.fetchone()[0]
00228             if count_id == 0:
00229                 cur.execute('INSERT INTO crashes VALUES (?, ?, ?, CURRENT_TIMESTAMP)', (_u(sig), id, None))
00230                 self.duplicate_db.commit()
00231         if addr_sig:
00232             self._duplicate_db_add_address_signature(addr_sig, id)
00233 
00234         return None

Here is the call graph for this function:

def apport.crashdb.CrashDatabase.close_duplicate (   self,
  report,
  id,
  master 
)
Mark a crash id as duplicate of given master ID.

If master is None, id gets un-duplicated.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 759 of file crashdb.py.

00759 
00760     def close_duplicate(self, report, id, master):
00761         '''Mark a crash id as duplicate of given master ID.
00762 
00763         If master is None, id gets un-duplicated.
00764         '''
00765         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.download (   self,
  id 
)
Download the problem report from given ID and return a Report.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 647 of file crashdb.py.

00647 
00648     def download(self, id):
00649         '''Download the problem report from given ID and return a Report.'''
00650 
00651         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.duplicate_db_change_master_id (   self,
  old_id,
  new_id 
)
Change a crash ID.

Definition at line 332 of file crashdb.py.

00332 
00333     def duplicate_db_change_master_id(self, old_id, new_id):
00334         '''Change a crash ID.'''
00335 
00336         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00337 
00338         cur = self.duplicate_db.cursor()
00339         cur.execute('UPDATE crashes SET crash_id = ?, last_change = CURRENT_TIMESTAMP WHERE crash_id = ?',
00340                     [new_id, old_id])
00341         cur.execute('UPDATE address_signatures SET crash_id = ? WHERE crash_id = ?',
00342                     [new_id, old_id])
00343         self.duplicate_db.commit()

def apport.crashdb.CrashDatabase.duplicate_db_fixed (   self,
  id,
  version 
)
Mark given crash ID as fixed in the duplicate database.

version specifies the package version the crash was fixed in (None for
'still unfixed').

Definition at line 306 of file crashdb.py.

00306 
00307     def duplicate_db_fixed(self, id, version):
00308         '''Mark given crash ID as fixed in the duplicate database.
00309 
00310         version specifies the package version the crash was fixed in (None for
00311         'still unfixed').
00312         '''
00313         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00314 
00315         cur = self.duplicate_db.cursor()
00316         n = cur.execute('UPDATE crashes SET fixed_version = ?, last_change = CURRENT_TIMESTAMP WHERE crash_id = ?',
00317                         (version, id))
00318         assert n.rowcount == 1
00319         self.duplicate_db.commit()

Here is the caller graph for this function:

Create text files suitable for www publishing.

Create a number of text files in the given directory which Apport
clients can use to determine whether a problem is already reported to
the database, through the known() method. This directory is suitable
for publishing to the web.

The database is indexed by the first two fields of the duplicate or
crash signature, to avoid having to download the entire database every
time.

If the directory already exists, it will be updated. The new content is
built in a new directory which is the given one with ".new" appended,
then moved to the given name in an almost atomic way.

Definition at line 344 of file crashdb.py.

00344 
00345     def duplicate_db_publish(self, dir):
00346         '''Create text files suitable for www publishing.
00347 
00348         Create a number of text files in the given directory which Apport
00349         clients can use to determine whether a problem is already reported to
00350         the database, through the known() method. This directory is suitable
00351         for publishing to the web.
00352 
00353         The database is indexed by the first two fields of the duplicate or
00354         crash signature, to avoid having to download the entire database every
00355         time.
00356 
00357         If the directory already exists, it will be updated. The new content is
00358         built in a new directory which is the given one with ".new" appended,
00359         then moved to the given name in an almost atomic way.
00360         '''
00361         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00362 
00363         # first create the temporary new dir; if that fails, nothing has been
00364         # changed and we fail early
00365         out = dir + '.new'
00366         os.mkdir(out)
00367 
00368         # crash addresses
00369         addr_base = os.path.join(out, 'address')
00370         os.mkdir(addr_base)
00371         cur_hash = None
00372         cur_file = None
00373 
00374         cur = self.duplicate_db.cursor()
00375 
00376         cur.execute('SELECT * from address_signatures ORDER BY signature')
00377         for (sig, id) in cur.fetchall():
00378             h = self.duplicate_sig_hash(sig)
00379             if h is None:
00380                 # some entries can't be represented in a single line
00381                 continue
00382             if h != cur_hash:
00383                 cur_hash = h
00384                 if cur_file:
00385                     cur_file.close()
00386                 cur_file = open(os.path.join(addr_base, cur_hash), 'w')
00387 
00388             cur_file.write('%i %s\n' % (id, sig))
00389 
00390         if cur_file:
00391             cur_file.close()
00392 
00393         # duplicate signatures
00394         sig_base = os.path.join(out, 'sig')
00395         os.mkdir(sig_base)
00396         cur_hash = None
00397         cur_file = None
00398 
00399         cur.execute('SELECT signature, crash_id from crashes ORDER BY signature')
00400         for (sig, id) in cur.fetchall():
00401             h = self.duplicate_sig_hash(sig)
00402             if h is None:
00403                 # some entries can't be represented in a single line
00404                 continue
00405             if h != cur_hash:
00406                 cur_hash = h
00407                 if cur_file:
00408                     cur_file.close()
00409                 cur_file = open(os.path.join(sig_base, cur_hash), 'wb')
00410 
00411             cur_file.write(('%i %s\n' % (id, sig)).encode('UTF-8'))
00412 
00413         if cur_file:
00414             cur_file.close()
00415 
00416         # switch over tree; this is as atomic as we can be with directories
00417         if os.path.exists(dir):
00418             os.rename(dir, dir + '.old')
00419         os.rename(out, dir)
00420         if os.path.exists(dir + '.old'):
00421             shutil.rmtree(dir + '.old')

Here is the call graph for this function:

Remove crash from the duplicate database.

This happens when a report got rejected or manually duplicated.

Definition at line 320 of file crashdb.py.

00320 
00321     def duplicate_db_remove(self, id):
00322         '''Remove crash from the duplicate database.
00323 
00324         This happens when a report got rejected or manually duplicated.
00325         '''
00326         assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
00327 
00328         cur = self.duplicate_db.cursor()
00329         cur.execute('DELETE FROM crashes WHERE crash_id = ?', [id])
00330         cur.execute('DELETE FROM address_signatures WHERE crash_id = ?', [id])
00331         self.duplicate_db.commit()

Here is the caller graph for this function:

Return master ID for a duplicate bug.

If the bug is not a duplicate, return None.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 752 of file crashdb.py.

00752 
00753     def duplicate_of(self, id):
00754         '''Return master ID for a duplicate bug.
00755 
00756         If the bug is not a duplicate, return None.
00757         '''
00758         raise NotImplementedError('this method must be implemented by a concrete subclass')

Create a www/URL proof hash for a duplicate signature

Definition at line 586 of file crashdb.py.

00586 
00587     def duplicate_sig_hash(klass, sig):
00588         '''Create a www/URL proof hash for a duplicate signature'''
00589 
00590         # cannot hash multi-line custom duplicate signatures
00591         if '\n' in sig:
00592             return None
00593 
00594         # custom DuplicateSignatures have a free format, split off first word
00595         i = sig.split(' ', 1)[0]
00596         # standard crash/address signatures use ':' as field separator, usually
00597         # for ExecutableName:Signal
00598         i = '_'.join(i.split(':', 2)[:2])
00599         # we manually quote '/' to make them nicer to read
00600         i = i.replace('/', '_')
00601         i = quote_plus(i.encode('UTF-8'))
00602         # avoid too long file names
00603         i = i[:200]
00604         return i

Here is the caller graph for this function:

Return list of affected source packages for given ID.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 732 of file crashdb.py.

00732 
00733     def get_affected_packages(self, id):
00734         '''Return list of affected source packages for given ID.'''
00735 
00736         raise NotImplementedError('this method must be implemented by a concrete subclass')

Return the base URL for bug patterns.

See apport.report.Report.search_bug_patterns() for details. If this
function returns None, bug patterns are disabled.

Definition at line 52 of file crashdb.py.

00052 
00053     def get_bugpattern_baseurl(self):
00054         '''Return the base URL for bug patterns.
00055 
00056         See apport.report.Report.search_bug_patterns() for details. If this
00057         function returns None, bug patterns are disabled.
00058         '''
00059         return self.options.get('bug_pattern_url')

def apport.crashdb.CrashDatabase.get_comment_url (   self,
  report,
  handle 
)
Return an URL that should be opened after report has been uploaded
and upload() returned handle.

Should return None if no URL should be opened (anonymous filing without
user comments); in that case this function should do whichever
interactive steps it wants to perform.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 627 of file crashdb.py.

00627 
00628     def get_comment_url(self, report, handle):
00629         '''Return an URL that should be opened after report has been uploaded
00630         and upload() returned handle.
00631 
00632         Should return None if no URL should be opened (anonymous filing without
00633         user comments); in that case this function should do whichever
00634         interactive steps it wants to perform.
00635         '''
00636         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

Get 'DistroRelease: <release>' from the report ID.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 685 of file crashdb.py.

00685 
00686     def get_distro_release(self, id):
00687         '''Get 'DistroRelease: <release>' from the report ID.'''
00688 
00689         raise NotImplementedError('this method must be implemented by a concrete subclass')

Return set of crash IDs which need duplicate checking.

This is mainly useful for crashes of scripting languages such as
Python, since they do not need to be retraced. It should not return
bugs that are covered by get_unretraced().

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 698 of file crashdb.py.

00698 
00699     def get_dup_unchecked(self):
00700         '''Return set of crash IDs which need duplicate checking.
00701 
00702         This is mainly useful for crashes of scripting languages such as
00703         Python, since they do not need to be retraced. It should not return
00704         bugs that are covered by get_unretraced().
00705         '''
00706         raise NotImplementedError('this method must be implemented by a concrete subclass')

Return the package version that fixes a given crash.

Return None if the crash is not yet fixed, or an empty string if the
crash is fixed, but it cannot be determined by which version. Return
'invalid' if the crash report got invalidated, such as closed a
duplicate or rejected.

This function should make sure that the returned result is correct. If
there are any errors with connecting to the crash database, it should
raise an exception (preferably IOError).

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 718 of file crashdb.py.

00718 
00719     def get_fixed_version(self, id):
00720         '''Return the package version that fixes a given crash.
00721 
00722         Return None if the crash is not yet fixed, or an empty string if the
00723         crash is fixed, but it cannot be determined by which version. Return
00724         'invalid' if the crash report got invalidated, such as closed a
00725         duplicate or rejected.
00726 
00727         This function should make sure that the returned result is correct. If
00728         there are any errors with connecting to the crash database, it should
00729         raise an exception (preferably IOError).
00730         '''
00731         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.get_id_url (   self,
  report,
  id 
)
Return URL for a given report ID.

The report is passed in case building the URL needs additional
information from it, such as the SourcePackage name.

Return None if URL is not available or cannot be determined.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 637 of file crashdb.py.

00637 
00638     def get_id_url(self, report, id):
00639         '''Return URL for a given report ID.
00640 
00641         The report is passed in case building the URL needs additional
00642         information from it, such as the SourcePackage name.
00643 
00644         Return None if URL is not available or cannot be determined.
00645         '''
00646         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

Return an ID set of all crashes which are not yet fixed.

The list must not contain bugs which were rejected or duplicate.

This function should make sure that the returned list is correct. If
there are any errors with connecting to the crash database, it should
raise an exception (preferably IOError).

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 707 of file crashdb.py.

00707 
00708     def get_unfixed(self):
00709         '''Return an ID set of all crashes which are not yet fixed.
00710 
00711         The list must not contain bugs which were rejected or duplicate.
00712 
00713         This function should make sure that the returned list is correct. If
00714         there are any errors with connecting to the crash database, it should
00715         raise an exception (preferably IOError).
00716         '''
00717         raise NotImplementedError('this method must be implemented by a concrete subclass')

Return set of crash IDs which have not been retraced yet.

This should only include crashes which match the current host
architecture.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 690 of file crashdb.py.

00690 
00691     def get_unretraced(self):
00692         '''Return set of crash IDs which have not been retraced yet.
00693 
00694         This should only include crashes which match the current host
00695         architecture.
00696         '''
00697         raise NotImplementedError('this method must be implemented by a concrete subclass')

Initialize duplicate database.

path specifies an SQLite database. It will be created if it does not
exist yet.

Definition at line 76 of file crashdb.py.

00076 
00077     def init_duplicate_db(self, path):
00078         '''Initialize duplicate database.
00079 
00080         path specifies an SQLite database. It will be created if it does not
00081         exist yet.
00082         '''
00083         import sqlite3 as dbapi2
00084 
00085         assert dbapi2.paramstyle == 'qmark', \
00086             'this module assumes qmark dbapi parameter style'
00087 
00088         self.format_version = 3
00089 
00090         init = not os.path.exists(path) or path == ':memory:' or \
00091             os.path.getsize(path) == 0
00092         self.duplicate_db = dbapi2.connect(path, timeout=7200)
00093 
00094         if init:
00095             cur = self.duplicate_db.cursor()
00096             cur.execute('CREATE TABLE version (format INTEGER NOT NULL)')
00097             cur.execute('INSERT INTO version VALUES (?)', [self.format_version])
00098 
00099             cur.execute('''CREATE TABLE crashes (
00100                 signature VARCHAR(255) NOT NULL,
00101                 crash_id INTEGER NOT NULL,
00102                 fixed_version VARCHAR(50),
00103                 last_change TIMESTAMP,
00104                 CONSTRAINT crashes_pk PRIMARY KEY (crash_id))''')
00105 
00106             cur.execute('''CREATE TABLE address_signatures (
00107                 signature VARCHAR(1000) NOT NULL,
00108                 crash_id INTEGER NOT NULL,
00109                 CONSTRAINT address_signatures_pk PRIMARY KEY (signature))''')
00110 
00111             self.duplicate_db.commit()
00112 
00113         # verify integrity
00114         cur = self.duplicate_db.cursor()
00115         cur.execute('PRAGMA integrity_check')
00116         result = cur.fetchall()
00117         if result != [('ok',)]:
00118             raise SystemError('Corrupt duplicate db:' + str(result))
00119 
00120         try:
00121             cur.execute('SELECT format FROM version')
00122             result = cur.fetchone()
00123         except self.duplicate_db.OperationalError as e:
00124             if 'no such table' in str(e):
00125                 # first db format did not have version table yet
00126                 result = [0]
00127         if result[0] > self.format_version:
00128             raise SystemError('duplicate DB has unknown format %i' % result[0])
00129         if result[0] < self.format_version:
00130             print('duplicate db has format %i, upgrading to %i' %
00131                   (result[0], self.format_version))
00132             self._duplicate_db_upgrade(result[0])

def apport.crashdb.CrashDatabase.is_reporter (   self,
  id 
)
Check whether the user is the reporter of given ID.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 737 of file crashdb.py.

00737 
00738     def is_reporter(self, id):
00739         '''Check whether the user is the reporter of given ID.'''
00740 
00741         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.known (   self,
  report 
)
Check if the crash db already knows about the crash signature.

Check if the report has a DuplicateSignature, crash_signature(), or
StacktraceAddressSignature, and ask the database whether the problem is
already known. If so, return an URL where the user can check the status
or subscribe (if available), or just return True if the report is known
but there is no public URL. In that case the report will not be
uploaded (i. e. upload() will not be called).

Return None if the report does not have any signature or the crash
database does not support checking for duplicates on the client side.

The default implementation uses a text file format generated by
duplicate_db_publish() at an URL specified by the "dupdb_url" option.
Subclasses are free to override this with a custom implementation, such
as a real database lookup.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase.

Definition at line 235 of file crashdb.py.

00235 
00236     def known(self, report):
00237         '''Check if the crash db already knows about the crash signature.
00238 
00239         Check if the report has a DuplicateSignature, crash_signature(), or
00240         StacktraceAddressSignature, and ask the database whether the problem is
00241         already known. If so, return an URL where the user can check the status
00242         or subscribe (if available), or just return True if the report is known
00243         but there is no public URL. In that case the report will not be
00244         uploaded (i. e. upload() will not be called).
00245 
00246         Return None if the report does not have any signature or the crash
00247         database does not support checking for duplicates on the client side.
00248 
00249         The default implementation uses a text file format generated by
00250         duplicate_db_publish() at an URL specified by the "dupdb_url" option.
00251         Subclasses are free to override this with a custom implementation, such
00252         as a real database lookup.
00253         '''
00254         if not self.options.get('dupdb_url'):
00255             return None
00256 
00257         for kind in ('sig', 'address'):
00258             # get signature
00259             if kind == 'sig':
00260                 if 'DuplicateSignature' in report:
00261                     sig = report['DuplicateSignature']
00262                 else:
00263                     sig = report.crash_signature()
00264             else:
00265                 sig = report.crash_signature_addresses()
00266 
00267             if not sig:
00268                 continue
00269 
00270             # build URL where the data should be
00271             h = self.duplicate_sig_hash(sig)
00272             if not h:
00273                 return None
00274 
00275             # the hash is already quoted, but we really want to open the quoted
00276             # file names; as urlopen() unquotes, we need to double-quote here
00277             # again so that urlopen() sees the single-quoted file names
00278             url = os.path.join(self.options['dupdb_url'], kind, quote_plus(h))
00279 
00280             # read data file
00281             try:
00282                 f = urlopen(url)
00283                 contents = f.read().decode('UTF-8')
00284                 f.close()
00285                 if '<title>404 Not Found' in contents:
00286                     continue
00287             except (IOError, URLError):
00288                 # does not exist, failed to load, etc.
00289                 continue
00290 
00291             # now check if we find our signature
00292             for line in contents.splitlines():
00293                 try:
00294                     id, s = line.split(None, 1)
00295                     id = int(id)
00296                 except ValueError:
00297                     continue
00298                 if s == sig:
00299                     result = self.get_id_url(report, id)
00300                     if not result:
00301                         # if we can't have an URL, just report as "known"
00302                         result = '1'
00303                     return result
00304 
00305         return None

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.mark_regression (   self,
  id,
  master 
)
Mark a crash id as reintroducing an earlier crash which is
already marked as fixed (having ID 'master').

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 766 of file crashdb.py.

00766 
00767     def mark_regression(self, id, master):
00768         '''Mark a crash id as reintroducing an earlier crash which is
00769         already marked as fixed (having ID 'master').'''
00770 
00771         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.mark_retrace_failed (   self,
  id,
  invalid_msg = None 
)
Mark crash id as 'failed to retrace'.

If invalid_msg is given, the bug should be closed as invalid with given
message, otherwise just marked as a failed retrace.

This can be a no-op if you are not interested in this.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase.

Definition at line 777 of file crashdb.py.

00777 
00778     def mark_retrace_failed(self, id, invalid_msg=None):
00779         '''Mark crash id as 'failed to retrace'.
00780 
00781         If invalid_msg is given, the bug should be closed as invalid with given
00782         message, otherwise just marked as a failed retrace.
00783 
00784         This can be a no-op if you are not interested in this.
00785         '''
00786         raise NotImplementedError('this method must be implemented by a concrete subclass')

Mark crash id as retraced.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 772 of file crashdb.py.

00772 
00773     def mark_retraced(self, id):
00774         '''Mark crash id as retraced.'''
00775 
00776         raise NotImplementedError('this method must be implemented by a concrete subclass')

def apport.crashdb.CrashDatabase.set_credentials (   self,
  username,
  password 
)
Set username and password.

Definition at line 680 of file crashdb.py.

00680 
00681     def set_credentials(self, username, password):
00682         '''Set username and password.'''
00683 
00684         raise NotImplementedError('this method must be implemented by a concrete subclass')

def apport.crashdb.CrashDatabase.update (   self,
  id,
  report,
  comment,
  change_description = False,
  attachment_comment = None,
  key_filter = None 
)
Update the given report ID with all data from report.

This creates a text comment with the "short" data (see
ProblemReport.write_mime()), and creates attachments for all the
bulk/binary data.

If change_description is True, and the crash db implementation supports
it, the short data will be put into the description instead (like in a
new bug).

comment will be added to the "short" data. If attachment_comment is
given, it will be added to the attachment uploads.

If key_filter is a list or set, then only those keys will be added.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 653 of file crashdb.py.

00653 
00654                attachment_comment=None, key_filter=None):
00655         '''Update the given report ID with all data from report.
00656 
00657         This creates a text comment with the "short" data (see
00658         ProblemReport.write_mime()), and creates attachments for all the
00659         bulk/binary data.
00660 
00661         If change_description is True, and the crash db implementation supports
00662         it, the short data will be put into the description instead (like in a
00663         new bug).
00664 
00665         comment will be added to the "short" data. If attachment_comment is
00666         given, it will be added to the attachment uploads.
00667 
00668         If key_filter is a list or set, then only those keys will be added.
00669         '''
00670         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.update_traces (   self,
  id,
  report,
  comment = '' 
)
Update the given report ID for retracing results.

This updates Stacktrace, ThreadStacktrace, StacktraceTop,
and StacktraceSource. You can also supply an additional comment.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase.

Definition at line 671 of file crashdb.py.

00671 
00672     def update_traces(self, id, report, comment=''):
00673         '''Update the given report ID for retracing results.
00674 
00675         This updates Stacktrace, ThreadStacktrace, StacktraceTop,
00676         and StacktraceSource. You can also supply an additional comment.
00677         '''
00678         self.update(id, report, comment, key_filter=[
00679             'Stacktrace', 'ThreadStacktrace', 'StacktraceSource', 'StacktraceTop'])

Here is the call graph for this function:

Here is the caller graph for this function:

def apport.crashdb.CrashDatabase.upload (   self,
  report,
  progress_callback = None 
)
Upload given problem report return a handle for it.

This should happen noninteractively.

If the implementation supports it, and a function progress_callback is
passed, that is called repeatedly with two arguments: the number of
bytes already sent, and the total number of bytes to send. This can be
used to provide a proper upload progress indication on frontends.

Implementations ought to "assert self.accepts(report)". The UI logic
already prevents uploading a report to a database which does not accept
it, but for third-party users of the API this should still be checked.

This method can raise a NeedsCredentials exception in case of failure.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase, and apport.crashdb_impl.memory.CrashDatabase.

Definition at line 609 of file crashdb.py.

00609 
00610     def upload(self, report, progress_callback=None):
00611         '''Upload given problem report return a handle for it.
00612 
00613         This should happen noninteractively.
00614 
00615         If the implementation supports it, and a function progress_callback is
00616         passed, that is called repeatedly with two arguments: the number of
00617         bytes already sent, and the total number of bytes to send. This can be
00618         used to provide a proper upload progress indication on frontends.
00619 
00620         Implementations ought to "assert self.accepts(report)". The UI logic
00621         already prevents uploading a report to a database which does not accept
00622         it, but for third-party users of the API this should still be checked.
00623 
00624         This method can raise a NeedsCredentials exception in case of failure.
00625         '''
00626         raise NotImplementedError('this method must be implemented by a concrete subclass')

Here is the caller graph for this function:


Member Data Documentation

Definition at line 48 of file crashdb.py.

Definition at line 50 of file crashdb.py.

Definition at line 87 of file crashdb.py.

Reimplemented in apport.crashdb_impl.launchpad.CrashDatabase.

Definition at line 49 of file crashdb.py.


The documentation for this class was generated from the following file: