Back to index

python-cliapp  1.20120630
Public Member Functions | Public Attributes | Private Member Functions | Private Attributes
cliapp.app.Application Class Reference

List of all members.

Public Member Functions

def __init__
def add_settings
def run
def add_subcommand
def setup_logging
def log_config
def app_directory
def setup_plugin_manager
def enable_plugins
def disable_plugins
def parse_args
def setup
def cleanup
def process_args
def process_inputs
def open_input
def process_input
def process_input_line
def runcmd
def runcmd_unchecked
def dump_memory_profile

Public Attributes

 fileno
 global_lineno
 lineno
 arg_synopsis
 cmd_synopsis
 subcommands
 settings
 plugin_subdir
 output
 pluginmgr

Private Member Functions

def _envname
def _run
def _subcommand_methodnames
def _normalize_cmd
def _unnormalize_cmd
def _format_usage
def _format_description
def _format_subcommand_description
def _vmrss

Private Attributes

 _description

Detailed Description

A framework for Unix-like command line programs.

The user should subclass this base class for each application.
The subclass does not need code for the mundane, boilerplate
parts that are the same in every utility, and can concentrate on the 
interesting part that is unique to it.

To start the application, call the `run` method.

The ``progname`` argument sets tne name of the program, which is
used for various purposes, such as determining the name of the
configuration file.

Similarly, ``version`` sets the version number of the program.

``description`` and ``epilog`` are included in the output of
``--help``. They are formatted to fit the screen. Unlike the
default behavior of ``optparse``, empty lines separate
paragraphs.

Definition at line 65 of file app.py.


Constructor & Destructor Documentation

def cliapp.app.Application.__init__ (   self,
  progname = None,
  version = '0.0.0',
  description = None,
  epilog = None 
)

Definition at line 90 of file app.py.

00090 
00091                  epilog=None):
00092         self.fileno = 0
00093         self.global_lineno = 0
00094         self.lineno = 0
00095         self._description = description
00096         if not hasattr(self, 'arg_synopsis'):
00097             self.arg_synopsis = '[FILE]...'
00098         if not hasattr(self, 'cmd_synopsis'):
00099             self.cmd_synopsis = {}
00100 
00101         self.subcommands = {}
00102         for method_name in self._subcommand_methodnames():
00103             cmd = self._unnormalize_cmd(method_name)
00104             self.subcommands[cmd] = getattr(self, method_name)
00105         
00106         self.settings = cliapp.Settings(progname, version, 
00107                                         usage=self._format_usage,
00108                                         description=self._format_description,
00109                                         epilog=epilog)
00110 
00111         self.plugin_subdir = 'plugins'
        

Member Function Documentation

def cliapp.app.Application._envname (   self,
  progname 
) [private]
Create an environment variable name of the name of a program.

Definition at line 132 of file app.py.

00132 
00133     def _envname(self, progname):
00134         '''Create an environment variable name of the name of a program.'''
00135         
00136         basename = os.path.basename(progname)
00137         if '.' in basename:
00138             basename = basename.split('.')[0]
00139         
00140         ok = 'abcdefghijklmnopqrstuvwxyz0123456789'
00141         ok += ok.upper()
00142         
00143         return ''.join(x.upper() if x in ok else '_' for x in basename)

Here is the caller graph for this function:

Format OptionParser description, with subcommand support.

Definition at line 248 of file app.py.

00248 
00249     def _format_description(self):
00250         '''Format OptionParser description, with subcommand support.'''
00251         if self.subcommands:
00252             paras = []
00253             for cmd in sorted(self.subcommands.keys()):
00254                 paras.append(self._format_subcommand_description(cmd))
00255             cmd_desc = '\n\n'.join(paras)
00256             return '%s\n\n%s' % (self._description or '', cmd_desc)
00257         else:
00258             return self._description

Here is the call graph for this function:

def cliapp.app.Application._format_subcommand_description (   self,
  cmd 
) [private]

Definition at line 259 of file app.py.

00259 
00260     def _format_subcommand_description(self, cmd): # pragma: no cover
00261 
00262         def remove_empties(lines):
00263             while lines and not lines[0].strip():
00264                 del lines[0]
00265 
00266         def split_para(lines):
00267             para = []
00268             while lines and lines[0].strip():
00269                 para.append(lines[0].strip())
00270                 del lines[0]
00271             return para
00272 
00273         indent = ' ' * 4
00274         method = self.subcommands[cmd]
00275         doc = method.__doc__ or ''
00276         lines = doc.splitlines()
00277         remove_empties(lines)
00278         if lines:
00279             heading = '* %s -- %s' % (cmd, lines[0])
00280             result = [heading]
00281             del lines[0]
00282             remove_empties(lines)
00283             while lines:
00284                 result.append('')
00285                 para_lines = split_para(lines)
00286                 para_text = ' '.join(para_lines)
00287                 result.append(para_text)
00288                 remove_empties(lines)
00289             return '\n'.join(result)
00290         else:
00291             return '* %s' % cmd
        

Here is the caller graph for this function:

def cliapp.app.Application._format_usage (   self) [private]
Format usage, possibly also subcommands, if any.

Definition at line 235 of file app.py.

00235 
00236     def _format_usage(self):
00237         '''Format usage, possibly also subcommands, if any.'''
00238         if self.subcommands:
00239             lines = []
00240             prefix = 'Usage:'
00241             for cmd in sorted(self.subcommands.keys()):
00242                 args = self.cmd_synopsis.get(cmd, '')
00243                 lines.append('%s %%prog [options] %s %s' % (prefix, cmd, args))
00244                 prefix = ' ' * len(prefix)
00245             return '\n'.join(lines)
00246         else:
00247             return None

def cliapp.app.Application._normalize_cmd (   self,
  cmd 
) [private]

Definition at line 228 of file app.py.

00228 
00229     def _normalize_cmd(self, cmd):
00230         return 'cmd_%s' % cmd.replace('-', '_')

def cliapp.app.Application._run (   self,
  args = None,
  stderr = sys.stderr,
  log = logging.critical 
) [private]

Definition at line 144 of file app.py.

00144 
00145     def _run(self, args=None, stderr=sys.stderr, log=logging.critical):
00146         try:
00147             self.add_settings()
00148             self.setup_plugin_manager()
00149             
00150             # A little bit of trickery here to make --no-default-configs and
00151             # --config=foo work right: we first parse the command line once,
00152             # and pick up any config files. Then we read configs. Finally,
00153             # we re-parse the command line to allow any options to override
00154             # config file settings.
00155             self.setup()
00156             self.enable_plugins()
00157             args = sys.argv[1:] if args is None else args
00158             self.parse_args(args, configs_only=True)
00159             self.settings.load_configs()
00160             args = self.parse_args(args)
00161 
00162             self.setup_logging()
00163             self.log_config()
00164             
00165             if self.settings['output']:
00166                 self.output = open(self.settings['output'], 'w')
00167             else:
00168                 self.output = sys.stdout
00169 
00170             self.process_args(args)
00171             self.cleanup()
00172             self.disable_plugins()
00173         except AppException, e:
00174             log(traceback.format_exc())
00175             stderr.write('ERROR: %s\n' % str(e))
00176             sys.exit(1)
00177         except SystemExit, e:
00178             sys.exit(e.code if type(e.code) == int else 1)
00179         except KeyboardInterrupt, e:
00180             sys.exit(255)
00181         except IOError, e: # pragma: no cover
00182             if e.errno == errno.EPIPE and e.filename is None:
00183                 # We're writing to stdout, and it broke. This almost always
00184                 # happens when we're being piped to less, and the user quits
00185                 # less before we finish writing everything out. So we ignore
00186                 # the error in that case.
00187                 sys.exit(1)
00188             log(traceback.format_exc())
00189             stderr.write('ERROR: %s\n' % str(e))
00190             sys.exit(1)
00191         except OSError, e: # pragma: no cover
00192             log(traceback.format_exc())
00193             if hasattr(e, 'filename') and e.filename:
00194                 stderr.write('ERROR: %s: %s\n' % (e.filename, e.strerror))
00195             else:
00196                 stderr.write('ERROR: %s\n' % e.strerror)
00197             sys.exit(1)
00198         except BaseException, e: # pragma: no cover
00199             log(traceback.format_exc())
00200             stderr.write(traceback.format_exc())
00201             sys.exit(1)
00202 
00203         logging.info('%s version %s ends normally' % 
00204                      (self.settings.progname, self.settings.version))
    

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 222 of file app.py.

00222 
00223     def _subcommand_methodnames(self):
00224         return [x 
00225                  for x in dir(self) 
00226                  if x.startswith('cmd_') and 
00227                     inspect.ismethod(getattr(self, x))]

def cliapp.app.Application._unnormalize_cmd (   self,
  method 
) [private]

Definition at line 231 of file app.py.

00231 
00232     def _unnormalize_cmd(self, method):
00233         assert method.startswith('cmd_')
00234         return method[len('cmd_'):].replace('_', '-')

def cliapp.app.Application._vmrss (   self) [private]
Return current resident memory use, in KiB.

Definition at line 489 of file app.py.

00489 
00490     def _vmrss(self): # pragma: no cover
00491         '''Return current resident memory use, in KiB.'''
00492         f = open('/proc/self/status')
00493         rss = 0
00494         for line in f:
00495             if line.startswith('VmRSS'):
00496                 rss = line.split()[1]
00497         f.close()
00498         return rss

Here is the caller graph for this function:

Add application specific settings.

Definition at line 112 of file app.py.

00112 
00113     def add_settings(self):
00114         '''Add application specific settings.'''

Here is the caller graph for this function:

def cliapp.app.Application.add_subcommand (   self,
  name,
  func,
  arg_synopsis = None 
)
Add a subcommand.

Normally, subcommands are defined by add ``cmd_foo`` methods
to the application class. However, sometimes it is more convenient
to have them elsewhere (e.g., in plugins). This method allows
doing that.

The callback function must accept a list of command line
non-option arguments.

Definition at line 205 of file app.py.

00205 
00206     def add_subcommand(self, name, func, arg_synopsis=None):
00207         '''Add a subcommand.
00208         
00209         Normally, subcommands are defined by add ``cmd_foo`` methods
00210         to the application class. However, sometimes it is more convenient
00211         to have them elsewhere (e.g., in plugins). This method allows
00212         doing that.
00213         
00214         The callback function must accept a list of command line
00215         non-option arguments.
00216         
00217         '''
00218         
00219         if name not in self.subcommands:
00220             self.subcommands[name] = func
00221             self.cmd_synopsis[name] = arg_synopsis
    
Return the directory where the application class is defined.

Plugins are searched relative to this directory, in the subdirectory
specified by self.plugin_subdir.

Definition at line 341 of file app.py.

00341 
00342     def app_directory(self):
00343         '''Return the directory where the application class is defined.
00344         
00345         Plugins are searched relative to this directory, in the subdirectory
00346         specified by self.plugin_subdir.
00347         
00348         '''
00349         
00350         module_name = self.__class__.__module__
00351         module = sys.modules[module_name]
00352         dirname = os.path.dirname(module.__file__) or '.'
00353         return dirname

Clean up after process_args.

This method is called just after process_args. By default it
does nothing, but subclasses may override it with a suitable
implementation. This is easier than overriding process_args
itself.

Definition at line 391 of file app.py.

00391 
00392     def cleanup(self):
00393         '''Clean up after process_args.
00394         
00395         This method is called just after process_args. By default it
00396         does nothing, but subclasses may override it with a suitable
00397         implementation. This is easier than overriding process_args
00398         itself.
00399         
00400         '''

Definition at line 367 of file app.py.

00367 
00368     def disable_plugins(self):
00369         self.pluginmgr.disable_plugins()

def cliapp.app.Application.dump_memory_profile (   self,
  msg 
)
Log memory profiling information.

Get the memory profiling method from the dump-memory-profile
setting, and log the results at DEBUG level. ``msg`` is a
message the caller provides to identify at what point the profiling
happens.

Definition at line 499 of file app.py.

00499 
00500     def dump_memory_profile(self, msg): # pragma: no cover
00501         '''Log memory profiling information.
00502         
00503         Get the memory profiling method from the dump-memory-profile
00504         setting, and log the results at DEBUG level. ``msg`` is a
00505         message the caller provides to identify at what point the profiling
00506         happens.
00507         
00508         '''
00509 
00510         kind = self.settings['dump-memory-profile']
00511 
00512         if kind == 'none':
00513             return
00514 
00515         logging.debug('dumping memory profiling data: %s' % msg)
00516         logging.debug('VmRSS: %s KiB' % self._vmrss())
00517         
00518         if kind == 'simple':
00519             return
00520 
00521         # These are fairly expensive operations, so we only log them
00522         # if we're doing expensive stuff anyway.
00523         logging.debug('# objects: %d' % len(gc.get_objects()))
00524         logging.debug('# garbage: %d' % len(gc.garbage))
00525 
00526         if kind == 'heapy':
00527             from guppy import hpy
00528             h = hpy()
00529             logging.debug('memory profile:\n%s' % h.heap())
00530         elif kind == 'meliae':
00531             filename = 'obnam-%d.meliae' % self.memory_dump_counter
00532             logging.debug('memory profile: see %s' % filename)
00533             from meliae import scanner
00534             scanner.dump_all_objects(filename)
00535             self.memory_dump_counter += 1
00536 

Here is the call graph for this function:

Load plugins.

Definition at line 360 of file app.py.

00360 
00361     def enable_plugins(self): # pragma: no cover
00362         '''Load plugins.'''
00363         for plugin in self.pluginmgr.plugins:
00364             plugin.app = self
00365             plugin.setup()
00366         self.pluginmgr.enable_plugins()

Here is the caller graph for this function:

Definition at line 329 of file app.py.

00329 
00330     def log_config(self):
00331         logging.info('%s version %s starts' % 
00332                      (self.settings.progname, self.settings.version))
00333         logging.debug('sys.argv: %s' % sys.argv)
00334         logging.debug('environment variables:')
00335         for name in os.environ:
00336             logging.debug('environment: %s=%s' % (name, os.environ[name]))
00337         cp = self.settings.as_cp()
00338         f = StringIO.StringIO()
00339         cp.write(f)
00340         logging.debug('Config:\n%s' % f.getvalue())

Here is the caller graph for this function:

def cliapp.app.Application.open_input (   self,
  name,
  mode = 'r' 
)
Open an input file for reading.

The default behaviour is to open a file named on the local
filesystem. A subclass might override this behavior for URLs,
for example.

The optional mode argument speficies the mode in which the file
gets opened. It should allow reading. Some files should perhaps
be opened in binary mode ('rb') instead of the default text mode.

Definition at line 440 of file app.py.

00440 
00441     def open_input(self, name, mode='r'):
00442         '''Open an input file for reading.
00443         
00444         The default behaviour is to open a file named on the local
00445         filesystem. A subclass might override this behavior for URLs,
00446         for example.
00447         
00448         The optional mode argument speficies the mode in which the file
00449         gets opened. It should allow reading. Some files should perhaps
00450         be opened in binary mode ('rb') instead of the default text mode.
00451         
00452         '''
00453         
00454         if name == '-':
00455             return sys.stdin
00456         else:
00457             return open(name, mode)

Here is the caller graph for this function:

def cliapp.app.Application.parse_args (   self,
  args,
  configs_only = False 
)
Parse the command line.

Return list of non-option arguments.

Definition at line 370 of file app.py.

00370 
00371     def parse_args(self, args, configs_only=False):
00372         '''Parse the command line.
00373         
00374         Return list of non-option arguments.
00375         
00376         '''
00377 
00378         return self.settings.parse_args(args, configs_only=configs_only,
00379                                          arg_synopsis=self.arg_synopsis,
00380                                          cmd_synopsis=self.cmd_synopsis)

Here is the caller graph for this function:

def cliapp.app.Application.process_args (   self,
  args 
)
Process command line non-option arguments.

The default is to call process_inputs with the argument list,
or to invoke the requested subcommand, if subcommands have
been defined.

Definition at line 401 of file app.py.

00401 
00402     def process_args(self, args):
00403         '''Process command line non-option arguments.
00404         
00405         The default is to call process_inputs with the argument list,
00406         or to invoke the requested subcommand, if subcommands have
00407         been defined.
00408         
00409         '''
00410         
00411             
00412         if self.subcommands:
00413             if not args:
00414                 raise SystemExit('must give subcommand')
00415             if args[0] in self.subcommands:
00416                 method = self.subcommands[args[0]]
00417                 method(args[1:])
00418             else:
00419                 raise SystemExit('unknown subcommand %s' % args[0])
00420         else:
00421             self.process_inputs(args)

Here is the call graph for this function:

def cliapp.app.Application.process_input (   self,
  name,
  stdin = sys.stdin 
)
Process a particular input file.

The ``stdin`` argument is meant for unit test only.

Definition at line 458 of file app.py.

00458 
00459     def process_input(self, name, stdin=sys.stdin):
00460         '''Process a particular input file.
00461         
00462         The ``stdin`` argument is meant for unit test only.
00463         
00464         '''
00465 
00466         self.fileno += 1
00467         self.lineno = 0
00468         f = self.open_input(name)
00469         for line in f:
00470             self.global_lineno += 1
00471             self.lineno += 1
00472             self.process_input_line(name, line)
00473         if f != stdin:
00474             f.close()

Here is the call graph for this function:

Here is the caller graph for this function:

def cliapp.app.Application.process_input_line (   self,
  filename,
  line 
)
Process one line of the input file.

Applications that are line-oriented can redefine only this method in
a subclass, and should not need to care about the other methods.

Definition at line 475 of file app.py.

00475 
00476     def process_input_line(self, filename, line):
00477         '''Process one line of the input file.
00478         
00479         Applications that are line-oriented can redefine only this method in
00480         a subclass, and should not need to care about the other methods.
00481         
00482         '''
        

Here is the caller graph for this function:

def cliapp.app.Application.process_inputs (   self,
  args 
)
Process all arguments as input filenames.

The default implementation calls process_input for each
input filename. If no filenames were given, then 
process_input is called with ``-`` as the argument name.
This implements the usual Unix command line practice of
reading from stdin if no inputs are named.

The attributes ``fileno``, ``global_lineno``, and ``lineno`` are set,
and count files and lines. The global line number is the
line number as if all input files were one.

Definition at line 422 of file app.py.

00422 
00423     def process_inputs(self, args):
00424         '''Process all arguments as input filenames.
00425         
00426         The default implementation calls process_input for each
00427         input filename. If no filenames were given, then 
00428         process_input is called with ``-`` as the argument name.
00429         This implements the usual Unix command line practice of
00430         reading from stdin if no inputs are named.
00431         
00432         The attributes ``fileno``, ``global_lineno``, and ``lineno`` are set,
00433         and count files and lines. The global line number is the
00434         line number as if all input files were one.
00435         
00436         '''
00437 
00438         for arg in args or ['-']:
00439             self.process_input(arg)

Here is the call graph for this function:

Here is the caller graph for this function:

def cliapp.app.Application.run (   self,
  args = None,
  stderr = sys.stderr,
  sysargv = sys.argv,
  log = logging.critical 
)
Run the application.

Definition at line 116 of file app.py.

00116 
00117             log=logging.critical):
00118         '''Run the application.'''
00119         
00120         def run_it():
00121             self._run(args=args, stderr=stderr, log=log)
00122 
00123         if self.settings.progname is None and sysargv:
00124             self.settings.progname = os.path.basename(sysargv[0])
00125         envname = '%s_PROFILE' % self._envname(self.settings.progname)
00126         profname = os.environ.get(envname, '')
00127         if profname: # pragma: no cover
00128             import cProfile
00129             cProfile.runctx('run_it()', globals(), locals(), profname)
00130         else:
00131             run_it()

Here is the call graph for this function:

def cliapp.app.Application.runcmd (   self,
  args,
  kwargs 
)

Definition at line 483 of file app.py.

00483 
00484     def runcmd(self, *args, **kwargs): # pragma: no cover
00485         return cliapp.runcmd(*args, **kwargs)
        
def cliapp.app.Application.runcmd_unchecked (   self,
  args,
  kwargs 
)

Definition at line 486 of file app.py.

00486 
00487     def runcmd_unchecked(self, *args, **kwargs): # pragma: no cover
00488         return cliapp.runcmd_unchecked(*args, **kwargs)

Prepare for process_args.

This method is called just before process_args. By default it
does nothing, but subclasses may override it with a suitable
implementation. This is easier than overriding process_args
itself.

Definition at line 381 of file app.py.

00381 
00382     def setup(self):
00383         '''Prepare for process_args.
00384         
00385         This method is called just before process_args. By default it
00386         does nothing, but subclasses may override it with a suitable
00387         implementation. This is easier than overriding process_args
00388         itself.
00389         
00390         '''

Here is the caller graph for this function:

Set up logging.

Definition at line 292 of file app.py.

00292 
00293     def setup_logging(self): # pragma: no cover
00294         '''Set up logging.'''
00295         
00296         level_name = self.settings['log-level']
00297         levels = {
00298             'debug': logging.DEBUG,
00299             'info': logging.INFO,
00300             'warning': logging.WARNING,
00301             'error': logging.ERROR,
00302             'critical': logging.CRITICAL,
00303             'fatal': logging.FATAL,
00304         }
00305         level = levels.get(level_name, logging.INFO)
00306 
00307         if self.settings['log'] == 'syslog':
00308             handler = logging.handlers.SysLogHandler(address='/dev/log')
00309         elif self.settings['log'] and self.settings['log'] != 'none':
00310             handler = LogHandler(
00311                             self.settings['log'],
00312                             perms=int(self.settings['log-mode'], 8),
00313                             maxBytes=self.settings['log-max'], 
00314                             backupCount=self.settings['log-keep'],
00315                             delay=False)
00316         else:
00317             handler = logging.FileHandler('/dev/null')
00318             # reduce amount of pointless I/O
00319             level = logging.FATAL
00320 
00321         fmt = '%(asctime)s %(levelname)s %(message)s'
00322         datefmt = '%Y-%m-%d %H:%M:%S'
00323         formatter = logging.Formatter(fmt, datefmt)
00324         handler.setFormatter(formatter)
00325 
00326         logger = logging.getLogger()
00327         logger.addHandler(handler)
00328         logger.setLevel(level)

Here is the caller graph for this function:

Create a plugin manager.

Definition at line 354 of file app.py.

00354 
00355     def setup_plugin_manager(self):
00356         '''Create a plugin manager.'''
00357         self.pluginmgr = cliapp.PluginManager()
00358         dirname = os.path.join(self.app_directory(), self.plugin_subdir)
00359         self.pluginmgr.locations = [dirname]

Here is the caller graph for this function:


Member Data Documentation

Definition at line 94 of file app.py.

Definition at line 96 of file app.py.

Definition at line 98 of file app.py.

Definition at line 91 of file app.py.

Definition at line 92 of file app.py.

Definition at line 93 of file app.py.

Definition at line 165 of file app.py.

Definition at line 110 of file app.py.

Definition at line 356 of file app.py.

Definition at line 105 of file app.py.

Definition at line 100 of file app.py.


The documentation for this class was generated from the following file: