Back to index

obnam  1.1
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions
obnamlib.clientmetadatatree.ClientMetadataTree Class Reference
Inheritance diagram for obnamlib.clientmetadatatree.ClientMetadataTree:
Inheritance graph
[legend]
Collaboration diagram for obnamlib.clientmetadatatree.ClientMetadataTree:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def init_caches
def default_file_id
def hashkey
def fskey
def fs_unkey
def genkey
def int2bin
def chunk_key
def chunk_unkey
def get_file_id
def set_file_id
def commit
def init_forest
def start_changes
def find_generation
def list_generations
def start_generation
def set_current_generation_is_checkpoint
def get_is_checkpoint
def remove_generation
def get_generation_id
def get_generation_times
def get_generation_data
def get_generation_file_count
def create
def get_metadata
def set_metadata
def remove
def listdir
def get_file_chunks
def set_file_chunks
def append_file_chunks
def chunk_in_use
def list_chunks_in_generation
def set_file_data
def get_file_data

Public Attributes

 current_time
 genhash
 chunkids_per_key
 known_generations
 file_ids
 file_count
 total_data
 tree

Static Public Attributes

int PREFIX_FS_META = 0
int FILE_NAME = 0
int FILE_CHUNKS = 1
int FILE_METADATA = 3
int DIR_CONTENTS = 4
int FILE_DATA = 5
int FILE_METADATA_ENCODED = 0
int PREFIX_CHUNK_REF = 1
int PREFIX_GEN_META = 2
int GEN_ID = 0
int GEN_STARTED = 1
int GEN_ENDED = 2
int GEN_IS_CHECKPOINT = 3
int GEN_FILE_COUNT = 4
int GEN_TOTAL_DATA = 5
int TYPE_MAX = 255
tuple SUBKEY_MAX = struct.pack('!Q', obnamlib.MAX_ID)

Private Member Functions

def _bad_default_file_id
def _lookup_int
def _insert_int
def _get_generation_id_or_None
def _lookup_time
def _lookup_count
def _insert_count
def _encode_chunks
def _decode_chunks
def _insert_chunks

Detailed Description

Store per-client metadata about files.

Actual file contents is stored elsewhere, this stores just the 
metadata about files: names, inode info, and what chunks of
data they use.

See http://braawi.org/obnam/ondisk/ for a description of how
this works.

Definition at line 27 of file clientmetadatatree.py.


Constructor & Destructor Documentation

def obnamlib.clientmetadatatree.ClientMetadataTree.__init__ (   self,
  fs,
  client_dir,
  node_size,
  upload_queue_size,
  lru_size,
  repo 
)

Definition at line 72 of file clientmetadatatree.py.

00072 
00073                  repo):
00074         tracing.trace('new ClientMetadataTree, client_dir=%s' % client_dir)
00075         self.current_time = repo.current_time
00076         key_bytes = len(self.hashkey(0, self.default_file_id(''), 0, 0))
00077         obnamlib.RepositoryTree.__init__(self, fs, client_dir, key_bytes, 
00078                                          node_size, upload_queue_size, 
00079                                          lru_size, repo)
00080         self.genhash = self.default_file_id('generation')
00081         self.chunkids_per_key = max(1,
00082                                     int(node_size / 4 / struct.calcsize('Q')))
00083         self.init_caches()


Member Function Documentation

For use by unit tests.

Definition at line 97 of file clientmetadatatree.py.

00097 
00098     def _bad_default_file_id(self, filename):
00099         '''For use by unit tests.'''
00100         return struct.pack('!Q', 0)

def obnamlib.clientmetadatatree.ClientMetadataTree._decode_chunks (   self,
  encoded 
) [private]

Definition at line 465 of file clientmetadatatree.py.

00465 
00466     def _decode_chunks(self, encoded):
00467         size = struct.calcsize('Q')
00468         count = len(encoded) / size
00469         fmt = '!' + ('Q' * count)
00470         return struct.unpack(fmt, encoded)

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._encode_chunks (   self,
  chunkids 
) [private]

Definition at line 461 of file clientmetadatatree.py.

00461 
00462     def _encode_chunks(self, chunkids):
00463         fmt = '!' + ('Q' * len(chunkids))
00464         return struct.pack(fmt, *chunkids)

Here is the caller graph for this function:

Definition at line 297 of file clientmetadatatree.py.

00297 
00298     def _get_generation_id_or_None(self, tree):
00299         try:
00300             return self.get_generation_id(tree)
00301         except KeyError: # pragma: no cover
00302             return None

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._insert_chunks (   self,
  tree,
  file_id,
  i,
  chunkids 
) [private]

Definition at line 471 of file clientmetadatatree.py.

00471 
00472     def _insert_chunks(self, tree, file_id, i, chunkids):
00473         key = self.fskey(file_id, self.FILE_CHUNKS, i)
00474         encoded = self._encode_chunks(chunkids)
00475         tree.insert(key, encoded)

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._insert_count (   self,
  genid,
  count_type,
  count 
) [private]

Definition at line 325 of file clientmetadatatree.py.

00325 
00326     def _insert_count(self, genid, count_type, count):
00327         tree = self.find_generation(genid)
00328         key = self.genkey(count_type)
00329         return self._insert_int(tree, key, count)

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._insert_int (   self,
  tree,
  key,
  value 
) [private]

Definition at line 209 of file clientmetadatatree.py.

00209 
00210     def _insert_int(self, tree, key, value):
00211         return tree.insert(key, struct.pack('!Q', value))

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._lookup_count (   self,
  genid,
  count_type 
) [private]

Definition at line 317 of file clientmetadatatree.py.

00317 
00318     def _lookup_count(self, genid, count_type):
00319         tree = self.find_generation(genid)
00320         key = self.genkey(count_type)
00321         try:
00322             return self._lookup_int(tree, key)
00323         except KeyError:
00324             return None

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._lookup_int (   self,
  tree,
  key 
) [private]

Definition at line 206 of file clientmetadatatree.py.

00206 
00207     def _lookup_int(self, tree, key):
00208         return struct.unpack('!Q', tree.lookup(key))[0]

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree._lookup_time (   self,
  tree,
  what 
) [private]

Definition at line 303 of file clientmetadatatree.py.

00303 
00304     def _lookup_time(self, tree, what):
00305         try:
00306             return self._lookup_int(tree, self.genkey(what))
00307         except KeyError:
00308             return None

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.append_file_chunks (   self,
  filename,
  chunkids 
)

Definition at line 493 of file clientmetadatatree.py.

00493 
00494     def append_file_chunks(self, filename, chunkids):
00495         tracing.trace('filename=%s', filename)
00496         tracing.trace('chunkids=%s', repr(chunkids))
00497 
00498         file_id = self.set_file_id(filename)
00499 
00500         minkey = self.fskey(file_id, self.FILE_CHUNKS, 0)
00501         maxkey = self.fskey(file_id, self.FILE_CHUNKS, self.SUBKEY_MAX)
00502         i = self.tree.count_range(minkey, maxkey)
00503 
00504         while chunkids:
00505             some = chunkids[:self.chunkids_per_key]
00506             self._insert_chunks(self.tree, file_id, i, some)
00507             for chunkid in some:
00508                 self.tree.insert(self.chunk_key(chunkid, file_id), '')
00509             i += 1
00510             chunkids = chunkids[self.chunkids_per_key:]

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.chunk_in_use (   self,
  gen_id,
  chunk_id 
)
Is a chunk used by a generation?

Definition at line 511 of file clientmetadatatree.py.

00511 
00512     def chunk_in_use(self, gen_id, chunk_id):
00513         '''Is a chunk used by a generation?'''
00514         
00515         minkey = self.chunk_key(chunk_id, 0)
00516         maxkey = self.chunk_key(chunk_id, obnamlib.MAX_ID)
00517         t = self.find_generation(gen_id)
00518         return not t.range_is_empty(minkey, maxkey)

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.chunk_key (   self,
  chunk_id,
  file_id 
)
Generate a key for a chunk reference.

Definition at line 150 of file clientmetadatatree.py.

00150 
00151     def chunk_key(self, chunk_id, file_id):
00152         '''Generate a key for a chunk reference.'''
00153         return self.hashkey(self.PREFIX_CHUNK_REF, self.int2bin(chunk_id),
00154                             0, file_id)

Here is the call graph for this function:

Here is the caller graph for this function:

Return the chunk and file ids in a chunk key.

Definition at line 155 of file clientmetadatatree.py.

00155 
00156     def chunk_unkey(self, key):
00157         '''Return the chunk and file ids in a chunk key.'''
00158         parts = struct.unpack('!BQBQ', key)
00159         return parts[1], parts[3]

Here is the caller graph for this function:

Definition at line 212 of file clientmetadatatree.py.

00212 
00213     def commit(self):
00214         tracing.trace('committing ClientMetadataTree')
00215         if self.tree:
00216             now = int(self.current_time())
00217             self._insert_int(self.tree, self.genkey(self.GEN_ENDED), now)
00218             genid = self._get_generation_id_or_None(self.tree)
00219             if genid is not None:
00220                 t = [(self.GEN_FILE_COUNT, 'file_count'),
00221                      (self.GEN_TOTAL_DATA, 'total_data')]
00222                 for subkey, attr in t:
00223                     if hasattr(self, attr):
00224                         self._insert_count(genid, subkey, getattr(self, attr))
00225         obnamlib.RepositoryTree.commit(self)

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.create (   self,
  filename,
  encoded_metadata 
)

Definition at line 333 of file clientmetadatatree.py.

00333 
00334     def create(self, filename, encoded_metadata):
00335         tracing.trace('filename=%s', filename)
00336         file_id = self.set_file_id(filename)
00337         gen_id = self.get_generation_id(self.tree)
00338         try:
00339             old_metadata = self.get_metadata(gen_id, filename)
00340         except KeyError:
00341             old_metadata = None
00342             self.file_count += 1
00343         else:
00344             old = obnamlib.decode_metadata(old_metadata)
00345             if old.isfile():
00346                 self.total_data -= old.st_size or 0
00347 
00348         metadata = obnamlib.decode_metadata(encoded_metadata)
00349         if metadata.isfile():
00350             self.total_data += metadata.st_size or 0
00351 
00352         if encoded_metadata != old_metadata:
00353             tracing.trace('new or changed metadata')
00354             self.set_metadata(filename, encoded_metadata)
00355 
00356         # Add to parent's contents, unless already there.
00357         parent = os.path.dirname(filename)
00358         tracing.trace('parent=%s', parent)
00359         if parent != filename: # root dir is its own parent
00360             basename = os.path.basename(filename)
00361             parent_id = self.set_file_id(parent)
00362             key = self.fskey(parent_id, self.DIR_CONTENTS, file_id)
00363             # We could just insert, but that would cause unnecessary
00364             # churn in the tree if nothing changes.
00365             try:
00366                 self.tree.lookup(key)
00367                 tracing.trace('was already in parent') # pragma: no cover
00368             except KeyError:
00369                 self.tree.insert(key, basename)
00370                 tracing.trace('added to parent')

Here is the call graph for this function:

Return hash of filename suitable for use as main key.

Definition at line 88 of file clientmetadatatree.py.

00088 
00089     def default_file_id(self, filename):
00090         '''Return hash of filename suitable for use as main key.'''
00091         tracing.trace(repr(filename))
00092         def hash(s):
00093             return hashlib.md5(s).digest()[:4]
00094         dirname = os.path.dirname(filename)
00095         basename = os.path.basename(filename)
00096         return hash(dirname) + hash(basename)

Here is the caller graph for this function:

Definition at line 234 of file clientmetadatatree.py.

00234 
00235     def find_generation(self, genid):
00236     
00237         def fill_cache():
00238             key = self.genkey(self.GEN_ID)
00239             for t in self.forest.trees:
00240                 t_genid = self._lookup_int(t, key)
00241                 if t_genid == genid:
00242                     self.known_generations[genid] = t
00243                     return t
00244     
00245         if self.forest:
00246             if genid in self.known_generations:
00247                 return self.known_generations[genid]
00248             t = fill_cache()
00249             if t is not None:
00250                 return t
00251         raise KeyError('Unknown generation %s' % genid)

Here is the call graph for this function:

Here is the caller graph for this function:

Inverse of fskey.

Definition at line 137 of file clientmetadatatree.py.

00137 
00138     def fs_unkey(self, key):
00139         '''Inverse of fskey.'''
00140         parts = struct.unpack('!B8sB8s', key)
00141         return parts[1], parts[3]

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.fskey (   self,
  mainhash,
  subtype,
  subkey 
)
'Generate key for filesystem metadata.

Definition at line 133 of file clientmetadatatree.py.

00133 
00134     def fskey(self, mainhash, subtype, subkey):
00135         ''''Generate key for filesystem metadata.'''
00136         return self.hashkey(self.PREFIX_FS_META, mainhash, subtype, subkey)
        

Here is the call graph for this function:

Here is the caller graph for this function:

Generate key for generation metadata.

Definition at line 142 of file clientmetadatatree.py.

00142 
00143     def genkey(self, subkey):
00144         '''Generate key for generation metadata.'''
00145         return self.hashkey(self.PREFIX_GEN_META, self.genhash, 0, subkey)

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.get_file_chunks (   self,
  genid,
  filename 
)

Definition at line 447 of file clientmetadatatree.py.

00447 
00448     def get_file_chunks(self, genid, filename):
00449         tree = self.find_generation(genid)
00450         try:
00451             file_id = self.get_file_id(tree, filename)
00452         except KeyError:
00453             return []
00454         minkey = self.fskey(file_id, self.FILE_CHUNKS, 0)
00455         maxkey = self.fskey(file_id, self.FILE_CHUNKS, self.SUBKEY_MAX)
00456         pairs = tree.lookup_range(minkey, maxkey)
00457         chunkids = []
00458         for key, value in pairs:
00459             chunkids.extend(self._decode_chunks(value))
00460         return chunkids

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.get_file_data (   self,
  gen_id,
  filename 
)
Return contents of file, if set, or None.

Definition at line 542 of file clientmetadatatree.py.

00542 
00543     def get_file_data(self, gen_id, filename): # pragma: no cover
00544         '''Return contents of file, if set, or None.'''
00545         tree = self.find_generation(gen_id)
00546         file_id = self.get_file_id(tree, filename)
00547         key = self.fskey(file_id, self.FILE_DATA, 0)
00548         try:
00549             return tree.lookup(key)
00550         except KeyError:
00551             return None
00552 

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.get_file_id (   self,
  tree,
  pathname 
)
Return id for file in a given generation.

Definition at line 160 of file clientmetadatatree.py.

00160 
00161     def get_file_id(self, tree, pathname):
00162         '''Return id for file in a given generation.'''
00163         
00164         if tree in self.file_ids:
00165             if pathname in self.file_ids[tree]:
00166                 return self.file_ids[tree][pathname]
00167         else:
00168             self.file_ids[tree] = {}
00169         
00170         default_file_id = self.default_file_id(pathname)
00171         minkey = self.fskey(default_file_id, self.FILE_NAME, 0)
00172         maxkey = self.fskey(default_file_id, self.FILE_NAME, obnamlib.MAX_ID)
00173         for key, value in tree.lookup_range(minkey, maxkey):
00174             def_id, file_id = self.fs_unkey(key)
00175             assert def_id == default_file_id, \
00176                 'def=%s other=%s' % (repr(def_id), repr(default_file_id))
00177             self.file_ids[tree][value] = file_id
00178             if value == pathname:
00179                 return file_id
00180 
00181         raise KeyError('%s does not yet have a file-id' % pathname)

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 314 of file clientmetadatatree.py.

00314 
00315     def get_generation_data(self, genid):
00316         return self._lookup_count(genid, self.GEN_TOTAL_DATA)

Here is the call graph for this function:

Definition at line 330 of file clientmetadatatree.py.

00330 
00331     def get_generation_file_count(self, genid):
00332         return self._lookup_count(genid, self.GEN_FILE_COUNT)

Here is the call graph for this function:

Definition at line 294 of file clientmetadatatree.py.

00294 
00295     def get_generation_id(self, tree):
00296         return self._lookup_int(tree, self.genkey(self.GEN_ID))

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 309 of file clientmetadatatree.py.

00309 
00310     def get_generation_times(self, genid):
00311         tree = self.find_generation(genid)
00312         return (self._lookup_time(tree, self.GEN_STARTED),
00313                 self._lookup_time(tree, self.GEN_ENDED))

Here is the call graph for this function:

Definition at line 279 of file clientmetadatatree.py.

00279 
00280     def get_is_checkpoint(self, genid):
00281         tree = self.find_generation(genid)
00282         key = self.genkey(self.GEN_IS_CHECKPOINT)
00283         try:
00284             return self._lookup_int(tree, key)
00285         except KeyError:
00286             return 0

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.get_metadata (   self,
  genid,
  filename 
)

Definition at line 371 of file clientmetadatatree.py.

00371 
00372     def get_metadata(self, genid, filename):
00373         tree = self.find_generation(genid)
00374         file_id = self.get_file_id(tree, filename)
00375         key = self.fskey(file_id, self.FILE_METADATA, 
00376                          self.FILE_METADATA_ENCODED)
00377         return tree.lookup(key)

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.hashkey (   self,
  prefix,
  mainhash,
  subtype,
  subkey 
)
Compute a full key.

The full key consists of three parts:

* prefix (0 for filesystem metadata, 1 for chunk refs)
* a hash of mainkey (64 bits)
* the subkey type (8 bits)
* type subkey (64 bits)

These are catenated.

mainhash must be a string of 8 bytes.

subtype must be an integer in the range 0.255, inclusive.

subkey must be either a string or an integer. If it is a string,
it will be padded with NUL bytes at the end, if it is less than
8 bytes, and truncated, if longer. If it is an integer, it will
be converted as a string, and the value must fit into 64 bits.

Definition at line 101 of file clientmetadatatree.py.

00101 
00102     def hashkey(self, prefix, mainhash, subtype, subkey):
00103         '''Compute a full key.
00104 
00105         The full key consists of three parts:
00106 
00107         * prefix (0 for filesystem metadata, 1 for chunk refs)
00108         * a hash of mainkey (64 bits)
00109         * the subkey type (8 bits)
00110         * type subkey (64 bits)
00111 
00112         These are catenated.
00113 
00114         mainhash must be a string of 8 bytes.
00115 
00116         subtype must be an integer in the range 0.255, inclusive.
00117 
00118         subkey must be either a string or an integer. If it is a string,
00119         it will be padded with NUL bytes at the end, if it is less than
00120         8 bytes, and truncated, if longer. If it is an integer, it will
00121         be converted as a string, and the value must fit into 64 bits.
00122 
00123         '''
00124         
00125         if type(subkey) == str:
00126             subkey = (subkey + '\0' * 8)[:8]
00127             fmt = '!B8sB8s'
00128         else:
00129             assert type(subkey) in [int, long]
00130             fmt = '!B8sBQ'
00131 
00132         return struct.pack(fmt, prefix, mainhash, subtype, subkey)

Here is the caller graph for this function:

Definition at line 84 of file clientmetadatatree.py.

00084 
00085     def init_caches(self):
00086         self.known_generations = {}
00087         self.file_ids = {}

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.init_forest (   self,
  args,
  kwargs 
)

Definition at line 226 of file clientmetadatatree.py.

00226 
00227     def init_forest(self, *args, **kwargs):
00228         self.init_caches()
00229         return obnamlib.RepositoryTree.init_forest(self, *args, **kwargs)

Here is the call graph for this function:

Here is the caller graph for this function:

Convert an integer to a binary string representation.

Definition at line 146 of file clientmetadatatree.py.

00146 
00147     def int2bin(self, integer):
00148         '''Convert an integer to a binary string representation.'''
00149         return struct.pack('!Q', integer)

Here is the caller graph for this function:

Return list of chunk ids used in a given generation.

Definition at line 519 of file clientmetadatatree.py.

00519 
00520     def list_chunks_in_generation(self, gen_id):
00521         '''Return list of chunk ids used in a given generation.'''
00522         
00523         minkey = self.chunk_key(0, 0)
00524         maxkey = self.chunk_key(obnamlib.MAX_ID, obnamlib.MAX_ID)
00525         t = self.find_generation(gen_id)
00526         return list(set(self.chunk_unkey(key)[0]
00527                         for key, value in t.lookup_range(minkey, maxkey)))

Here is the call graph for this function:

Definition at line 252 of file clientmetadatatree.py.

00252 
00253     def list_generations(self):
00254         if self.forest:
00255             genids = []
00256             for t in self.forest.trees:
00257                 genid = self._get_generation_id_or_None(t)
00258                 if genid is not None:
00259                     genids.append(genid)
00260             return genids
00261         else:
00262             return []

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.listdir (   self,
  genid,
  dirname 
)

Definition at line 434 of file clientmetadatatree.py.

00434 
00435     def listdir(self, genid, dirname):
00436         tree = self.find_generation(genid)
00437         try:
00438             dir_id = self.get_file_id(tree, dirname)
00439         except KeyError:
00440             return []
00441         minkey = self.fskey(dir_id, self.DIR_CONTENTS, 0)
00442         maxkey = self.fskey(dir_id, self.DIR_CONTENTS, self.SUBKEY_MAX)
00443         basenames = []
00444         for key, value in tree.lookup_range(minkey, maxkey):
00445             basenames.append(value)
00446         return basenames

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 389 of file clientmetadatatree.py.

00389 
00390     def remove(self, filename):
00391         tracing.trace('filename=%s', filename)
00392 
00393         file_id = self.get_file_id(self.tree, filename)
00394         genid = self.get_generation_id(self.tree)
00395         self.file_count -= 1
00396         
00397         try:
00398             encoded_metadata = self.get_metadata(genid, filename)
00399         except KeyError:
00400             pass
00401         else:
00402             metadata = obnamlib.decode_metadata(encoded_metadata)
00403             if metadata.isfile():
00404                 self.total_data -= metadata.st_size or 0
00405 
00406         # Remove any children.
00407         minkey = self.fskey(file_id, self.DIR_CONTENTS, 0)
00408         maxkey = self.fskey(file_id, self.DIR_CONTENTS, obnamlib.MAX_ID)
00409         for key, basename in self.tree.lookup_range(minkey, maxkey):
00410             self.remove(os.path.join(filename, basename))
00411 
00412         # Remove chunk refs.
00413         for chunkid in self.get_file_chunks(genid, filename):
00414             key = self.chunk_key(chunkid, file_id)
00415             self.tree.remove_range(key, key)
00416             
00417         # Remove this file's metadata.
00418         minkey = self.fskey(file_id, 0, 0)
00419         maxkey = self.fskey(file_id, self.TYPE_MAX, self.SUBKEY_MAX)
00420         self.tree.remove_range(minkey, maxkey)
00421         
00422         # Remove filename.
00423         default_file_id = self.default_file_id(filename)
00424         key = self.fskey(default_file_id, self.FILE_NAME, file_id)
00425         self.tree.remove_range(key, key)
00426         
00427         # Also remove from parent's contents.
00428         parent = os.path.dirname(filename)
00429         if parent != filename: # root dir is its own parent
00430             parent_id = self.set_file_id(parent)
00431             key = self.fskey(parent_id, self.DIR_CONTENTS, file_id)
00432             # The range removal will work even if the key does not exist.
00433             self.tree.remove_range(key, key)
            

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 287 of file clientmetadatatree.py.

00287 
00288     def remove_generation(self, genid):
00289         tracing.trace('genid=%s', genid)
00290         tree = self.find_generation(genid)
00291         if tree == self.tree:
00292             self.tree = None
00293         self.forest.remove_tree(tree)

Here is the call graph for this function:

Definition at line 273 of file clientmetadatatree.py.

00273 
00274     def set_current_generation_is_checkpoint(self, is_checkpoint):
00275         tracing.trace('is_checkpoint=%s', is_checkpoint)
00276         value = 1 if is_checkpoint else 0
00277         key = self.genkey(self.GEN_IS_CHECKPOINT)
00278         self._insert_int(self.tree, key, value)

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.set_file_chunks (   self,
  filename,
  chunkids 
)

Definition at line 476 of file clientmetadatatree.py.

00476 
00477     def set_file_chunks(self, filename, chunkids):
00478         tracing.trace('filename=%s', filename)
00479         tracing.trace('chunkids=%s', repr(chunkids))
00480     
00481         file_id = self.set_file_id(filename)
00482         minkey = self.fskey(file_id, self.FILE_CHUNKS, 0)
00483         maxkey = self.fskey(file_id, self.FILE_CHUNKS, self.SUBKEY_MAX)
00484 
00485         for key, value in self.tree.lookup_range(minkey, maxkey):
00486             for chunkid in self._decode_chunks(value):
00487                 k = self.chunk_key(chunkid, file_id)
00488                 self.tree.remove_range(k, k)
00489 
00490         self.tree.remove_range(minkey, maxkey)
00491 
00492         self.append_file_chunks(filename, chunkids)

Here is the call graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.set_file_data (   self,
  filename,
  contents 
)
Store contents of file, if small, in B-tree instead of chunk.

The length of the contents should be small enough to fit in a
B-tree leaf.

Definition at line 528 of file clientmetadatatree.py.

00528 
00529     def set_file_data(self, filename, contents): # pragma: no cover
00530         '''Store contents of file, if small, in B-tree instead of chunk.
00531         
00532         The length of the contents should be small enough to fit in a
00533         B-tree leaf.
00534         
00535         '''
00536         tracing.trace('filename=%s' % filename)
00537         tracing.trace('contents=%s' % repr(contents))
00538         
00539         file_id = self.set_file_id(filename)
00540         key = self.fskey(file_id, self.FILE_DATA, 0)
00541         self.tree.insert(key, contents)

Here is the call graph for this function:

Set and return the file-id for a file in current generation.

Definition at line 182 of file clientmetadatatree.py.

00182 
00183     def set_file_id(self, pathname):
00184         '''Set and return the file-id for a file in current generation.'''
00185 
00186         default_file_id = self.default_file_id(pathname)
00187         minkey = self.fskey(default_file_id, self.FILE_NAME, 0)
00188         maxkey = self.fskey(default_file_id, self.FILE_NAME, obnamlib.MAX_ID)
00189         file_ids = set()
00190         for key, value in self.tree.lookup_range(minkey, maxkey):
00191             def_id, file_id = self.fs_unkey(key)
00192             assert def_id == default_file_id
00193             if value == pathname:
00194                 return file_id
00195             file_ids.add(file_id)
00196         
00197         while True:
00198             n = random.randint(0, obnamlib.MAX_ID)
00199             file_id = struct.pack('!Q', n)
00200             if file_id not in file_ids:
00201                 break
00202         
00203         key = self.fskey(default_file_id, self.FILE_NAME, file_id)
00204         self.tree.insert(key, pathname)
00205         return file_id

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.set_metadata (   self,
  filename,
  encoded_metadata 
)

Definition at line 378 of file clientmetadatatree.py.

00378 
00379     def set_metadata(self, filename, encoded_metadata):
00380         tracing.trace('filename=%s', filename)
00381 
00382         file_id = self.set_file_id(filename)
00383         key1 = self.fskey(file_id, self.FILE_NAME, file_id)
00384         self.tree.insert(key1, filename)
00385         
00386         key2 = self.fskey(file_id, self.FILE_METADATA, 
00387                           self.FILE_METADATA_ENCODED)
00388         self.tree.insert(key2, encoded_metadata)

Here is the call graph for this function:

Here is the caller graph for this function:

def obnamlib.clientmetadatatree.ClientMetadataTree.start_changes (   self,
  args,
  kwargs 
)

Definition at line 230 of file clientmetadatatree.py.

00230 
00231     def start_changes(self, *args, **kwargs):
00232         self.init_caches()
00233         return obnamlib.RepositoryTree.start_changes(self, *args, **kwargs)

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 263 of file clientmetadatatree.py.

00263 
00264     def start_generation(self):
00265         tracing.trace('start new generation')
00266         self.start_changes()
00267         gen_id = self.forest.new_id()
00268         now = int(self.current_time())
00269         self._insert_int(self.tree, self.genkey(self.GEN_ID), gen_id)
00270         self._insert_int(self.tree, self.genkey(self.GEN_STARTED), now)
00271         self.file_count = self.get_generation_file_count(gen_id) or 0
00272         self.total_data = self.get_generation_data(gen_id) or 0

Here is the call graph for this function:


Member Data Documentation

Definition at line 80 of file clientmetadatatree.py.

Definition at line 74 of file clientmetadatatree.py.

Definition at line 45 of file clientmetadatatree.py.

Definition at line 43 of file clientmetadatatree.py.

Definition at line 270 of file clientmetadatatree.py.

Definition at line 46 of file clientmetadatatree.py.

Definition at line 86 of file clientmetadatatree.py.

Definition at line 44 of file clientmetadatatree.py.

Definition at line 48 of file clientmetadatatree.py.

Definition at line 42 of file clientmetadatatree.py.

Definition at line 60 of file clientmetadatatree.py.

Definition at line 62 of file clientmetadatatree.py.

Definition at line 58 of file clientmetadatatree.py.

Definition at line 61 of file clientmetadatatree.py.

Definition at line 59 of file clientmetadatatree.py.

Definition at line 63 of file clientmetadatatree.py.

Definition at line 79 of file clientmetadatatree.py.

Definition at line 85 of file clientmetadatatree.py.

Definition at line 53 of file clientmetadatatree.py.

Definition at line 41 of file clientmetadatatree.py.

Definition at line 57 of file clientmetadatatree.py.

Definition at line 69 of file clientmetadatatree.py.

Definition at line 271 of file clientmetadatatree.py.

Definition at line 291 of file clientmetadatatree.py.

Definition at line 68 of file clientmetadatatree.py.


The documentation for this class was generated from the following file: