Back to index

obnam  1.1
Public Member Functions | Public Attributes
obnamlib.vfs.VirtualFileSystem Class Reference

List of all members.

Public Member Functions

def __init__
def log_stats
def connect
def close
def reinit
def abspath
def getcwd
def chdir
def listdir
def listdir2
def lock
def unlock
def exists
def isdir
def mkdir
def makedirs
def rmdir
def rmtree
def remove
def rename
def lstat
def get_username
def get_groupname
def llistxattr
def lgetxattr
def lsetxattr
def lchown
def chmod
def lutimes
def link
def readlink
def symlink
def open
def cat
def write_file
def overwrite_file
def scan_tree

Public Attributes

 baseurl
 bytes_read
 bytes_written

Detailed Description

A virtual filesystem interface.

The backup program needs to access both local and remote files.
To make it easier to support all kinds of files both locally and
remotely, we use a custom virtual filesystem interface so that
all filesystem access is done the same way. This way, we can
easily support user data and backup repositories in any combination of
local and remote filesystems.

This class defines the interface for such virtual filesystems.
Sub-classes will actually implement the interface.

When a VFS is instantiated, it is bound to a base URL. When
accessing the virtual filesystem, all paths are then given
relative to the base URL. The Unix syntax for files is used
for the relative paths: directory components separated by
slashes, and an initial slash indicating the root of the
filesystem (in this case, the base URL).

Definition at line 27 of file vfs.py.


Constructor & Destructor Documentation

def obnamlib.vfs.VirtualFileSystem.__init__ (   self,
  baseurl 
)

Definition at line 50 of file vfs.py.

00050 
00051     def __init__(self, baseurl):
00052         self.baseurl = baseurl
00053         self.bytes_read = 0
00054         self.bytes_written = 0
00055         logging.info('VFS: __init__: baseurl=%s' % self.baseurl)


Member Function Documentation

def obnamlib.vfs.VirtualFileSystem.abspath (   self,
  pathname 
)
Return absolute version of pathname.

Definition at line 76 of file vfs.py.

00076 
00077     def abspath(self, pathname):
00078         '''Return absolute version of pathname.'''
00079         return os.path.abspath(os.path.join(self.getcwd(), pathname))

Here is the call graph for this function:

def obnamlib.vfs.VirtualFileSystem.cat (   self,
  pathname 
)
Return the contents of a file.

Definition at line 182 of file vfs.py.

00182 
00183     def cat(self, pathname):
00184         '''Return the contents of a file.'''

def obnamlib.vfs.VirtualFileSystem.chdir (   self,
  pathname 
)
Change current working directory to pathname.

Definition at line 83 of file vfs.py.

00083 
00084     def chdir(self, pathname):
00085         '''Change current working directory to pathname.'''

def obnamlib.vfs.VirtualFileSystem.chmod (   self,
  pathname,
  mode 
)
Like os.chmod.

Definition at line 154 of file vfs.py.

00154 
00155     def chmod(self, pathname, mode):
00156         '''Like os.chmod.'''

Close connection to filesystem.

Definition at line 63 of file vfs.py.

00063 
00064     def close(self):
00065         '''Close connection to filesystem.'''
00066         self.log_stats()

Here is the call graph for this function:

Connect to filesystem.

Definition at line 60 of file vfs.py.

00060 
00061     def connect(self):
00062         '''Connect to filesystem.'''
        
def obnamlib.vfs.VirtualFileSystem.exists (   self,
  pathname 
)
Does the file or directory exist?

Definition at line 98 of file vfs.py.

00098 
00099     def exists(self, pathname):
00100         '''Does the file or directory exist?'''

Here is the caller graph for this function:

Return name for group, or None if not known.

Definition at line 138 of file vfs.py.

00138 
00139     def get_groupname(self, gid):
00140         '''Return name for group, or None if not known.'''

Return name for user, or None if not known.

Definition at line 135 of file vfs.py.

00135 
00136     def get_username(self, uid):
00137         '''Return name for user, or None if not known.'''

Return current working directory as absolute pathname.

Definition at line 80 of file vfs.py.

00080 
00081     def getcwd(self):
00082         '''Return current working directory as absolute pathname.'''
        

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.isdir (   self,
  pathname 
)
Is it a directory?

Definition at line 101 of file vfs.py.

00101 
00102     def isdir(self, pathname):
00103         '''Is it a directory?'''

def obnamlib.vfs.VirtualFileSystem.lchown (   self,
  pathname,
  uid,
  gid 
)
Like os.lchown.

Definition at line 151 of file vfs.py.

00151 
00152     def lchown(self, pathname, uid, gid):
00153         '''Like os.lchown.'''

def obnamlib.vfs.VirtualFileSystem.lgetxattr (   self,
  pathname,
  attrname 
)
Return value of an extended attribute.

Definition at line 145 of file vfs.py.

00145 
00146     def lgetxattr(self, pathname, attrname):
00147         '''Return value of an extended attribute.'''
        
def obnamlib.vfs.VirtualFileSystem.link (   self,
  existing_path,
  new_path 
)
Like os.link.

Definition at line 165 of file vfs.py.

00165 
00166     def link(self, existing_path, new_path):
00167         '''Like os.link.'''

def obnamlib.vfs.VirtualFileSystem.listdir (   self,
  pathname 
)
Return list of basenames of entities at pathname.

Definition at line 86 of file vfs.py.

00086 
00087     def listdir(self, pathname):
00088         '''Return list of basenames of entities at pathname.'''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.listdir2 (   self,
  pathname 
)
Return list of basenames and stats of entities at pathname.

Definition at line 89 of file vfs.py.

00089 
00090     def listdir2(self, pathname):
00091         '''Return list of basenames and stats of entities at pathname.'''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.llistxattr (   self,
  pathname 
)
Return list of names of extended attributes for file.

Definition at line 141 of file vfs.py.

00141 
00142     def llistxattr(self, pathname):
00143         '''Return list of names of extended attributes for file.'''
00144         return []
        
def obnamlib.vfs.VirtualFileSystem.lock (   self,
  lockname,
  data 
)
Create a lock file with the given name.

Definition at line 92 of file vfs.py.

00092 
00093     def lock(self, lockname, data):
00094         '''Create a lock file with the given name.'''

Definition at line 56 of file vfs.py.

00056 
00057     def log_stats(self):
00058         logging.info('VFS: baseurl=%s read=%d written=%d' %
00059                      (self.baseurl, self.bytes_read, self.bytes_written))

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.lsetxattr (   self,
  pathname,
  attrname,
  attrvalue 
)
Set value of an extended attribute.

Definition at line 148 of file vfs.py.

00148 
00149     def lsetxattr(self, pathname, attrname, attrvalue):
00150         '''Set value of an extended attribute.'''

def obnamlib.vfs.VirtualFileSystem.lstat (   self,
  pathname 
)
Like os.lstat.

Definition at line 132 of file vfs.py.

00132 
00133     def lstat(self, pathname):
00134         '''Like os.lstat.'''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.lutimes (   self,
  pathname,
  atime_sec,
  atime_nsec,
  mtime_sec,
  mtime_nsec 
)
Like lutimes(2).

This isn't quite like lutimes, actually. Most importantly, it uses
nanosecond timestamps rather than microsecond. This is important.

Definition at line 157 of file vfs.py.

00157 
00158     def lutimes(self, pathname, atime_sec, atime_nsec, mtime_sec, mtime_nsec):
00159         '''Like lutimes(2).
00160         
00161         This isn't quite like lutimes, actually. Most importantly, it uses
00162         nanosecond timestamps rather than microsecond. This is important.
00163         
00164         '''

def obnamlib.vfs.VirtualFileSystem.makedirs (   self,
  pathname 
)
Create a directory, and missing parents.

Definition at line 111 of file vfs.py.

00111 
00112     def makedirs(self, pathname):
00113         '''Create a directory, and missing parents.'''

def obnamlib.vfs.VirtualFileSystem.mkdir (   self,
  pathname 
)
Create a directory.

Parent directories must already exist.

Definition at line 104 of file vfs.py.

00104 
00105     def mkdir(self, pathname):
00106         '''Create a directory.
00107         
00108         Parent directories must already exist.
00109         
00110         '''
        
def obnamlib.vfs.VirtualFileSystem.open (   self,
  pathname,
  mode 
)
Open a file, like the builtin open() or file() function.

The return value is a file object like the ones returned
by the builtin open() function.

Definition at line 174 of file vfs.py.

00174 
00175     def open(self, pathname, mode):
00176         '''Open a file, like the builtin open() or file() function.
00177 
00178         The return value is a file object like the ones returned
00179         by the builtin open() function.
00180 
00181         '''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.overwrite_file (   self,
  pathname,
  contents 
)
Like write_file, but overwrites existing file.

Definition at line 197 of file vfs.py.

00197 
00198     def overwrite_file(self, pathname, contents):
00199         '''Like write_file, but overwrites existing file.'''

def obnamlib.vfs.VirtualFileSystem.readlink (   self,
  symlink 
)
Like os.readlink.

Definition at line 168 of file vfs.py.

00168 
00169     def readlink(self, symlink):
00170         '''Like os.readlink.'''

def obnamlib.vfs.VirtualFileSystem.reinit (   self,
  new_baseurl,
  create = False 
)
Go back to the beginning.

This behaves like instantiating a new instance, but possibly
faster for things like SftpFS. If there is a network
connection already open, it will be reused.

Definition at line 67 of file vfs.py.

00067 
00068     def reinit(self, new_baseurl, create=False):
00069         '''Go back to the beginning.
00070         
00071         This behaves like instantiating a new instance, but possibly
00072         faster for things like SftpFS. If there is a network
00073         connection already open, it will be reused.
00074         
00075         '''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.remove (   self,
  pathname 
)
Remove a file.

Definition at line 126 of file vfs.py.

00126 
00127     def remove(self, pathname):
00128         '''Remove a file.'''

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.rename (   self,
  old,
  new 
)
Rename a file.

Definition at line 129 of file vfs.py.

00129 
00130     def rename(self, old, new):
00131         '''Rename a file.'''

def obnamlib.vfs.VirtualFileSystem.rmdir (   self,
  pathname 
)
Remove an empty directory.

Definition at line 114 of file vfs.py.

00114 
00115     def rmdir(self, pathname):
00116         '''Remove an empty directory.'''

def obnamlib.vfs.VirtualFileSystem.rmtree (   self,
  dirname 
)
Remove a directory tree, including its contents.

Definition at line 117 of file vfs.py.

00117 
00118     def rmtree(self, dirname):
00119         '''Remove a directory tree, including its contents.'''
00120         if self.isdir(dirname):
00121             for pathname, st in self.scan_tree(dirname):
00122                 if stat.S_ISDIR(st.st_mode):
00123                     self.rmdir(pathname)
00124                 else:
00125                     self.remove(pathname)

Here is the call graph for this function:

def obnamlib.vfs.VirtualFileSystem.scan_tree (   self,
  dirname,
  ok = None,
  dirst = None,
  log = logging.error,
  error_handler = None 
)
Scan a tree for files.

Return a generator that returns ``(pathname, stat_result)``
pairs for each file and directory in the tree, in 
depth-first order.

If ``ok`` is not None, it must be a function that determines
if a particular file or directory should be returned.
It gets the pathname and stat result as arguments, and
should return True or False. If it returns False on a
directory, ``scan_tree`` will not recurse into the
directory.

``dirst`` is for internal optimization, and should not
be used by the caller. ``log`` is used by unit tests and
should not be used by the caller.

Errors from calling ``listdir`` or ``lstat`` are logged,
but do not stop the scanning. Such files or directories are
not returned, however. If `error_handler` is defined, it is
called once for every problem, giving the name and exception
as arguments.

Definition at line 201 of file vfs.py.

00201 
00202                   error_handler=None):
00203         '''Scan a tree for files.
00204         
00205         Return a generator that returns ``(pathname, stat_result)``
00206         pairs for each file and directory in the tree, in 
00207         depth-first order.
00208         
00209         If ``ok`` is not None, it must be a function that determines
00210         if a particular file or directory should be returned.
00211         It gets the pathname and stat result as arguments, and
00212         should return True or False. If it returns False on a
00213         directory, ``scan_tree`` will not recurse into the
00214         directory.
00215         
00216         ``dirst`` is for internal optimization, and should not
00217         be used by the caller. ``log`` is used by unit tests and
00218         should not be used by the caller.
00219         
00220         Errors from calling ``listdir`` or ``lstat`` are logged,
00221         but do not stop the scanning. Such files or directories are
00222         not returned, however. If `error_handler` is defined, it is
00223         called once for every problem, giving the name and exception
00224         as arguments.
00225         
00226         '''
00227         
00228         error_handler = error_handler or (lambda name, e: None)
00229 
00230         try:
00231             pairs = self.listdir2(dirname)
00232         except OSError, e:
00233             log('listdir failed: %s: %s' % (e.filename, e.strerror))
00234             error_handler(dirname, e)
00235             pairs = []
00236             
00237         queue = []
00238         for name, st in pairs:
00239             pathname = os.path.join(dirname, name)
00240             if isinstance(st, BaseException):
00241                 error_handler(pathname, st)
00242             elif ok is None or ok(pathname, st):
00243                 if stat.S_ISDIR(st.st_mode):
00244                     for t in self.scan_tree(pathname, ok=ok, dirst=st):
00245                         yield t
00246                 else:
00247                     queue.append((pathname, st))
00248 
00249         for pathname, st in queue:
00250             yield pathname, st
00251 
00252         if dirst is None:
00253             try:
00254                 dirst = self.lstat(dirname)
00255             except OSError, e:
00256                 log('lstat for dir failed: %s: %s' % (e.filename, e.strerror))
00257                 return
00258 
00259         yield dirname, dirst
00260 
        

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.vfs.VirtualFileSystem.symlink (   self,
  source,
  destination 
)
Like os.symlink.

Definition at line 171 of file vfs.py.

00171 
00172     def symlink(self, source, destination):
00173         '''Like os.symlink.'''

def obnamlib.vfs.VirtualFileSystem.unlock (   self,
  lockname 
)
Remove a lock file.

Definition at line 95 of file vfs.py.

00095 
00096     def unlock(self, lockname):
00097         '''Remove a lock file.'''

def obnamlib.vfs.VirtualFileSystem.write_file (   self,
  pathname,
  contents 
)
Write a new file.

The file must not yet exist. The file is not necessarily written 
atomically, meaning that if the writing fails (connection to
server drops, for example), the file might exist in a partial
form. The callers need to deal with this.

Any directories in pathname will be created if necessary.

Definition at line 185 of file vfs.py.

00185 
00186     def write_file(self, pathname, contents):
00187         '''Write a new file.
00188 
00189         The file must not yet exist. The file is not necessarily written 
00190         atomically, meaning that if the writing fails (connection to
00191         server drops, for example), the file might exist in a partial
00192         form. The callers need to deal with this.
00193         
00194         Any directories in pathname will be created if necessary.
00195 
00196         '''

Here is the caller graph for this function:


Member Data Documentation

Definition at line 51 of file vfs.py.

Definition at line 52 of file vfs.py.

Definition at line 53 of file vfs.py.


The documentation for this class was generated from the following file: