Back to index

python3.2  3.2.2
Classes | Functions | Variables
pipes Namespace Reference

Classes

class  Template

Functions

def makepipeline
def quote

Variables

list __all__ = ["Template"]
string FILEIN_FILEOUT = 'ff'
string STDIN_FILEOUT = '-f'
string FILEIN_STDOUT = 'f-'
string STDIN_STDOUT = '--'
string SOURCE = '.-'
string SINK = '-.'
list stepkinds
tuple _safechars = frozenset(string.ascii_letters + string.digits + '@%_-+=:,./')

Detailed Description

Conversion pipeline templates.

The problem:
------------

Suppose you have some data that you want to convert to another format,
such as from GIF image format to PPM image format.  Maybe the
conversion involves several steps (e.g. piping it through compress or
uuencode).  Some of the conversion steps may require that their input
is a disk file, others may be able to read standard input; similar for
their output.  The input to the entire conversion may also be read
from a disk file or from an open file, and similar for its output.

The module lets you construct a pipeline template by sticking one or
more conversion steps together.  It will take care of creating and
removing temporary files if they are necessary to hold intermediate
data.  You can then use the template to do conversions from many
different sources to many different destinations.  The temporary
file names used are different each time the template is used.

The templates are objects so you can create templates for many
different conversion steps and store them in a dictionary, for
instance.


Directions:
-----------

To create a template:
    t = Template()

To add a conversion step to a template:
   t.append(command, kind)
where kind is a string of two characters: the first is '-' if the
command reads its standard input or 'f' if it requires a file; the
second likewise for the output. The command must be valid /bin/sh
syntax.  If input or output files are required, they are passed as
$IN and $OUT; otherwise, it must be  possible to use the command in
a pipeline.

To add a conversion step at the beginning:
   t.prepend(command, kind)

To convert a file to another file using a template:
  sts = t.copy(infile, outfile)
If infile or outfile are the empty string, standard input is read or
standard output is written, respectively.  The return value is the
exit status of the conversion pipeline.

To open a file for reading or writing through a conversion pipeline:
   fp = t.open(file, mode)
where mode is 'r' to read the file, or 'w' to write it -- just like
for the built-in function open() or for os.popen().

To create a new template object initialized to a given one:
   t2 = t.clone()

For an example, see the function test() at the end of the file.

Function Documentation

def pipes.makepipeline (   infile,
  steps,
  outfile 
)

Definition at line 181 of file pipes.py.

00181 
00182 def makepipeline(infile, steps, outfile):
00183     # Build a list with for each command:
00184     # [input filename or '', command string, kind, output filename or '']
00185 
00186     list = []
00187     for cmd, kind in steps:
00188         list.append(['', cmd, kind, ''])
00189     #
00190     # Make sure there is at least one step
00191     #
00192     if not list:
00193         list.append(['', 'cat', '--', ''])
00194     #
00195     # Take care of the input and output ends
00196     #
00197     [cmd, kind] = list[0][1:3]
00198     if kind[0] == 'f' and not infile:
00199         list.insert(0, ['', 'cat', '--', ''])
00200     list[0][0] = infile
00201     #
00202     [cmd, kind] = list[-1][1:3]
00203     if kind[1] == 'f' and not outfile:
00204         list.append(['', 'cat', '--', ''])
00205     list[-1][-1] = outfile
00206     #
00207     # Invent temporary files to connect stages that need files
00208     #
00209     garbage = []
00210     for i in range(1, len(list)):
00211         lkind = list[i-1][2]
00212         rkind = list[i][2]
00213         if lkind[1] == 'f' or rkind[0] == 'f':
00214             (fd, temp) = tempfile.mkstemp()
00215             os.close(fd)
00216             garbage.append(temp)
00217             list[i-1][-1] = list[i][0] = temp
00218     #
00219     for item in list:
00220         [inf, cmd, kind, outf] = item
00221         if kind[1] == 'f':
00222             cmd = 'OUT=' + quote(outf) + '; ' + cmd
00223         if kind[0] == 'f':
00224             cmd = 'IN=' + quote(inf) + '; ' + cmd
00225         if kind[0] == '-' and inf:
00226             cmd = cmd + ' <' + quote(inf)
00227         if kind[1] == '-' and outf:
00228             cmd = cmd + ' >' + quote(outf)
00229         item[1] = cmd
00230     #
00231     cmdlist = list[0][1]
00232     for item in list[1:]:
00233         [cmd, kind] = item[1:3]
00234         if item[0] == '':
00235             if 'f' in kind:
00236                 cmd = '{ ' + cmd + '; }'
00237             cmdlist = cmdlist + ' |\n' + cmd
00238         else:
00239             cmdlist = cmdlist + '\n' + cmd
00240     #
00241     if garbage:
00242         rmcmd = 'rm -f'
00243         for file in garbage:
00244             rmcmd = rmcmd + ' ' + quote(file)
00245         trapcmd = 'trap ' + quote(rmcmd + '; exit') + ' 1 2 3 13 14 15'
00246         cmdlist = trapcmd + '\n' + cmdlist + '\n' + rmcmd
00247     #
00248     return cmdlist
00249 
00250 
00251 # Reliably quote a string as a single argument for /bin/sh
00252 
# Safe unquoted

Here is the call graph for this function:

def pipes.quote (   file)
Return a shell-escaped version of the file string.

Definition at line 255 of file pipes.py.

00255 
00256 def quote(file):
00257     """Return a shell-escaped version of the file string."""
00258     for c in file:
00259         if c not in _safechars:
00260             break
00261     else:
00262         if not file:
00263             return "''"
00264         return file
00265     # use single quotes, and put single quotes into double quotes
00266     # the string $'b is then quoted as '$'"'"'b'
00267     return "'" + file.replace("'", "'\"'\"'") + "'"

Here is the caller graph for this function:


Variable Documentation

list pipes.__all__ = ["Template"]

Definition at line 67 of file pipes.py.

tuple pipes._safechars = frozenset(string.ascii_letters + string.digits + '@%_-+=:,./')

Definition at line 253 of file pipes.py.

Definition at line 71 of file pipes.py.

Definition at line 73 of file pipes.py.

Definition at line 76 of file pipes.py.

Definition at line 75 of file pipes.py.

Definition at line 72 of file pipes.py.

Definition at line 74 of file pipes.py.

Initial value:
00001 [FILEIN_FILEOUT, STDIN_FILEOUT, FILEIN_STDOUT, STDIN_STDOUT, \
00002              SOURCE, SINK]

Definition at line 78 of file pipes.py.