Back to index

nordugrid-arc-nox  1.1.0~rc6
Public Member Functions | Public Attributes | Private Member Functions
arcom.store.picklestore.PickleStore Class Reference
Inheritance diagram for arcom.store.picklestore.PickleStore:
Inheritance graph
[legend]
Collaboration diagram for arcom.store.picklestore.PickleStore:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def list
def get
def set
def lock
def unlock

Public Attributes

 datadir
 non_existent_object
 llock

Private Member Functions

def _filename
def _tmpfilename
def _list

Detailed Description

Class for storing object in a serialized python format. 

Definition at line 10 of file picklestore.py.


Constructor & Destructor Documentation

def arcom.store.picklestore.PickleStore.__init__ (   self,
  storecfg,
  non_existent_object = {} 
)
Constructor of PickleStore.

PickleStore(storecfg)

'storecfg' is an XMLNode with a 'DataDir'
'non_existent_object' will be returned if an object not found

Reimplemented from arcom.store.basestore.BaseStore.

Reimplemented in arcom.store.cachedpicklestore.CachedPickleStore, arcom.store.cachedstringstore.CachedStringStore, and arcom.store.stringstore.StringStore.

Definition at line 13 of file picklestore.py.

00013 
00014     def __init__(self, storecfg, non_existent_object = {}):
00015         """ Constructor of PickleStore.
00016 
00017         PickleStore(storecfg)
00018 
00019         'storecfg' is an XMLNode with a 'DataDir'
00020         'non_existent_object' will be returned if an object not found
00021         """
00022         BaseStore.__init__(self, storecfg, non_existent_object)
00023         log.msg(arc.VERBOSE, "PickleStore constructor called")
00024         log.msg(arc.VERBOSE, "datadir:", self.datadir)


Member Function Documentation

def arcom.store.picklestore.PickleStore._filename (   self,
  ID 
) [private]
Creates a filename from an ID.

_filename(ID)

'ID' is the ID of the given object.
The filename will be 
datadir/<two first letters of base64 encoded form of the ID>/<base64 encoded form of the ID>.

Definition at line 25 of file picklestore.py.

00025 
00026     def _filename(self, ID):
00027         """ Creates a filename from an ID.
00028 
00029         _filename(ID)
00030 
00031         'ID' is the ID of the given object.
00032         The filename will be 
00033         datadir/<two first letters of base64 encoded form of the ID>/<base64 encoded form of the ID>.
00034         """
00035         name = base64.b64encode(ID)
00036         return os.path.join(self.datadir, name[:2], name)

Here is the caller graph for this function:

List all the existing files.

_list()

Returns a list of filenames.

Definition at line 49 of file picklestore.py.

00049 
00050     def _list(self):
00051         """ List all the existing files.
00052         
00053         _list()
00054         
00055         Returns a list of filenames.
00056         """
00057         names = []
00058         # list the contects of each subdirectory withtin the data directory
00059         for subdir in os.listdir(self.datadir):
00060             if os.path.isdir(os.path.join(self.datadir,subdir)):
00061                 names.extend(os.listdir(os.path.join(self.datadir, subdir)))
00062         return names

Here is the caller graph for this function:

def arcom.store.picklestore.PickleStore._tmpfilename (   self,
  ID 
) [private]
Creates a tmpfilename from an ID.

_tmpfilename(ID)

'ID' is the ID of the given object.
The filename will be datadir/<base64 encoded form of the ID>.
The corresponding file to _tmpfilename should always be moved to _filename(ID)

Definition at line 37 of file picklestore.py.

00037 
00038     def _tmpfilename(self, ID):
00039         """ Creates a tmpfilename from an ID.
00040 
00041         _tmpfilename(ID)
00042 
00043         'ID' is the ID of the given object.
00044         The filename will be datadir/<base64 encoded form of the ID>.
00045         The corresponding file to _tmpfilename should always be moved to _filename(ID)
00046         """
00047         name = base64.b64encode(ID)
00048         return os.path.join(self.datadir, name)

Here is the caller graph for this function:

def arcom.store.picklestore.PickleStore.get (   self,
  ID 
)
Returns the object with the given ID.

get(ID)

'ID' is the ID of the requested object.
If there is no object with this ID, returns the given non_existent_object value.

Reimplemented in arcom.store.cachedpicklestore.CachedPickleStore, and arcom.store.stringstore.StringStore.

Definition at line 80 of file picklestore.py.

00080 
00081     def get(self, ID):
00082         """ Returns the object with the given ID.
00083 
00084         get(ID)
00085 
00086         'ID' is the ID of the requested object.
00087         If there is no object with this ID, returns the given non_existent_object value.
00088         """
00089         try:
00090             # generates a filename from the ID
00091             # then use pickle to load the previously serialized data
00092             return pickle.load(file(self._filename(ID), 'rb'))
00093         except IOError:
00094             # don't print 'file not found' if there is no such ID
00095             pass
00096         except EOFError:
00097             # TODO: find out what causes this problem
00098             pass
00099         except:
00100             # print whatever exception happened
00101             log.msg()
00102             log.msg(arc.ERROR, "filename:", self._filename(ID))
00103         # if there was an exception, return the given non_existent_object
00104         return copy.deepcopy(self.non_existent_object)

Here is the call graph for this function:

Here is the caller graph for this function:

List the IDs of the existing entries.

list()

Returns a list of IDs.

Reimplemented in arcom.store.cachedpicklestore.CachedPickleStore.

Definition at line 63 of file picklestore.py.

00063 
00064     def list(self):
00065         """ List the IDs of the existing entries.
00066         
00067         list()
00068         
00069         Returns a list of IDs.
00070         """
00071         IDs = []
00072         # get all the filenames
00073         for name in self._list():
00074             try: # decode the filename
00075                 ID = base64.b64decode(name)
00076                 IDs.append(ID)
00077             except:
00078                 log.msg()
00079         return IDs

Here is the call graph for this function:

def arcom.store.basestore.BaseStore.lock (   self,
  blocking = True 
) [inherited]
Acquire the lock.

lock(blocking = True)

'blocking': if blocking is True, then this only returns when the lock is acquired.
If it is False, then it returns immediately with False if the lock is not available,
or with True if it could be acquired.

Reimplemented in storage.ahash.replicatedahash.ReplicationStore.

Definition at line 27 of file basestore.py.

00027 
00028     def lock(self, blocking = True):
00029         """ Acquire the lock.
00030 
00031         lock(blocking = True)
00032 
00033         'blocking': if blocking is True, then this only returns when the lock is acquired.
00034         If it is False, then it returns immediately with False if the lock is not available,
00035         or with True if it could be acquired.
00036         """
00037         return self.llock.acquire(blocking)

Here is the caller graph for this function:

def arcom.store.picklestore.PickleStore.set (   self,
  ID,
  object 
)
Stores an object with the given ID..

set(ID, object)

'ID' is the ID of the object
'object' is the object itself
If there is already an object with this ID it will be overwritten completely.

Reimplemented in arcom.store.cachedpicklestore.CachedPickleStore, arcom.store.stringstore.StringStore, and arcom.store.cachedstringstore.CachedStringStore.

Definition at line 105 of file picklestore.py.

00105 
00106     def set(self, ID, object):
00107         """ Stores an object with the given ID..
00108 
00109         set(ID, object)
00110 
00111         'ID' is the ID of the object
00112         'object' is the object itself
00113         If there is already an object with this ID it will be overwritten completely.
00114         """
00115         if not ID:
00116             raise Exception, 'ID is empty'
00117         try:
00118             # generates a filename from the ID
00119             fn = self._filename(ID)
00120             tmp_fn = self._tmpfilename(ID)
00121             # if 'object' is empty, don't make file
00122             if object:
00123                 # serialize the given list into tmp_fn
00124                 pickle.dump(object, file(tmp_fn,'wb'))
00125                 # try to rename the file
00126                 try:
00127                     os.rename(tmp_fn,fn)
00128                 except:
00129                     # try to create parent dir first, then rename the file
00130                     os.mkdir(os.path.dirname(fn))
00131                     os.rename(tmp_fn,fn)
00132             elif os.path.isfile(fn):
00133                 # object empty, file is not needed anymore
00134                 os.remove(fn)
00135         except:
00136             log.msg()

Here is the call graph for this function:

Here is the caller graph for this function:

def arcom.store.basestore.BaseStore.unlock (   self) [inherited]
Release the lock.

unlock()

Reimplemented in storage.ahash.replicatedahash.ReplicationStore.

Definition at line 38 of file basestore.py.

00038 
00039     def unlock(self):
00040         """ Release the lock.
00041 
00042         unlock()
00043         """
00044         self.llock.release()
        

Here is the caller graph for this function:


Member Data Documentation

Definition at line 18 of file basestore.py.

Definition at line 25 of file basestore.py.

Definition at line 20 of file basestore.py.


The documentation for this class was generated from the following file: