Back to index

plone3  3.1.7
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions | Private Attributes
ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex Class Reference
Inheritance diagram for ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex:
Inheritance graph
[legend]
Collaboration diagram for ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def clear
def insertEntry
def index_object
def unindex_object
def search
def getIndexSourceNames

Public Attributes

 indexed_attrs

Static Public Attributes

string meta_type = "ExtendedPathIndex"
tuple manage_options
tuple query_options
tuple index_html = DTMLFile('dtml/index', globals())
tuple manage_workspace = DTMLFile('dtml/manageExtendedPathIndex', globals())

Private Member Functions

def _apply_index

Private Attributes

 _index_parents
 _index_items

Detailed Description

A path index stores all path components of the physical
path of an object:

Internal datastructure (regular pathindex):

- a physical path of an object is split into its components

- every component is kept as a  key of a OOBTree in self._indexes

- the value is a mapping 'level of the path component' to
  'all docids with this path component on this level'

In addition

- there is a terminator (None) signifying the last component in the path

Definition at line 19 of file ExtendedPathIndex.py.


Constructor & Destructor Documentation

def ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.__init__ (   self,
  id,
  extra = None,
  caller = None 
)
ExtendedPathIndex supports indexed_attrs 

Definition at line 49 of file ExtendedPathIndex.py.

00049 
00050     def __init__(self, id, extra=None, caller=None):
00051         """ ExtendedPathIndex supports indexed_attrs """
00052         PathIndex.__init__(self, id, caller)
00053 
00054         def get(o, k, default):
00055             if isinstance(o, dict):
00056                 return o.get(k, default)
00057             else:
00058                 return getattr(o, k, default)
00059 
00060         attrs = get(extra, 'indexed_attrs', None)
00061         if attrs is None:
00062             return
00063         if isinstance(attrs, str):
00064             attrs = attrs.split(',')
00065         attrs = filter(None, [a.strip() for a in attrs])
00066 
00067         if attrs:
00068             # We only index the first attribute so snip off the rest
00069             self.indexed_attrs = tuple(attrs[:1])

Here is the call graph for this function:


Member Function Documentation

def ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex._apply_index (   self,
  request,
  cid = '' 
) [private]
hook for (Z)Catalog
    'request' --  mapping type (usually {"path": "..." }
     additionaly a parameter "path_level" might be passed
     to specify the level (see search())

    'cid' -- ???

Definition at line 352 of file ExtendedPathIndex.py.

00352 
00353     def _apply_index(self, request, cid=''):
00354         """ hook for (Z)Catalog
00355             'request' --  mapping type (usually {"path": "..." }
00356              additionaly a parameter "path_level" might be passed
00357              to specify the level (see search())
00358 
00359             'cid' -- ???
00360         """
00361 
00362         record = parseIndexRequest(request,self.id,self.query_options)
00363         if record.keys==None: return None
00364 
00365         level    = record.get("level", 0)
00366         operator = record.get('operator', self.useOperator).lower()
00367         depth    = getattr(record, 'depth', -1) # Set to 0 or navtree later
00368                                                 # use getattr to get 0 value
00369         navtree  = record.get('navtree', 0)
00370         navtree_start  = record.get('navtree_start', 0)
00371 
00372         # depending on the operator we use intersection of union
00373         if operator == "or":  set_func = union
00374         else: set_func = intersection
00375 
00376         res = None
00377         for k in record.keys:
00378             rows = self.search(k,level, depth, navtree, navtree_start)
00379             res = set_func(res,rows)
00380 
00381         if res:
00382             return res, (self.id,)
00383         else:
00384             return IISet(), (self.id,)

Here is the call graph for this function:

Definition at line 70 of file ExtendedPathIndex.py.

00070 
00071     def clear(self):
00072         PathIndex.clear(self)
00073         self._index_parents = OOBTree()
00074         self._index_items = OIBTree()

Here is the caller graph for this function:

return names of indexed attributes 

Definition at line 385 of file ExtendedPathIndex.py.

00385 
00386     def getIndexSourceNames(self):
00387         """ return names of indexed attributes """
00388 
00389         # By default PathIndex advertises getPhysicalPath even
00390         # though the logic in index_object is different.
00391 
00392         try:
00393             return tuple(self.indexed_attrs)
00394         except AttributeError:
00395             return ('getPhysicalPath',)

def ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.index_object (   self,
  docid,
  obj,
  threshold = 100 
)
hook for (Z)Catalog 

Definition at line 101 of file ExtendedPathIndex.py.

00101 
00102     def index_object(self, docid, obj ,threshold=100):
00103         """ hook for (Z)Catalog """
00104 
00105         # PathIndex first checks for an attribute matching its id and
00106         # falls back to getPhysicalPath only when failing to get one.
00107         # The presence of 'indexed_attrs' overrides this behavior and
00108         # causes indexing of the custom attribute.
00109 
00110         attrs = getattr(self, 'indexed_attrs', None)
00111         if attrs:
00112             index = attrs[0]
00113         else:
00114             index = self.id
00115 
00116         f = getattr(obj, index, None)
00117         if f is not None:
00118             if safe_callable(f):
00119                 try:
00120                     path = f()
00121                 except AttributeError:
00122                     return 0
00123             else:
00124                 path = f
00125 
00126             if not isinstance(path, (str, tuple)):
00127                 raise TypeError('path value must be string or tuple '
00128                                 'of strings: (%r, %s)' % (index, repr(path)))
00129         else:
00130             try:
00131                 path = obj.getPhysicalPath()
00132             except AttributeError:
00133                 return 0
00134 
00135         if isinstance(path, (list, tuple)):
00136             path = '/'+ '/'.join(path[1:])
00137         comps = filter(None, path.split('/'))
00138         parent_path = '/' + '/'.join(comps[:-1])
00139 
00140         # Make sure we reindex properly when path change
00141         if self._unindex.has_key(docid) and self._unindex.get(docid) != path:
00142             self.unindex_object(docid)
00143 
00144         if not self._unindex.has_key(docid):
00145             self._length.change(1)
00146 
00147         for i in range(len(comps)):
00148             self.insertEntry(comps[i], docid, i)
00149 
00150         # Add terminator
00151         self.insertEntry(None, docid, len(comps)-1, parent_path, path)
00152 
00153         self._unindex[docid] = path
00154         return 1

Here is the call graph for this function:

def ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.insertEntry (   self,
  comp,
  id,
  level,
  parent_path = None,
  object_path = None 
)
Insert an entry.

   parent_path is the path of the parent object

   path is the object path, it is assumed to be unique, i.e. there
   is a one to one mapping between physical paths and docids.  This
   will be large, and is only used for breadcrumbs.

   id is the docid

Definition at line 75 of file ExtendedPathIndex.py.

00075 
00076     def insertEntry(self, comp, id, level, parent_path=None, object_path=None):
00077         """Insert an entry.
00078 
00079            parent_path is the path of the parent object
00080 
00081            path is the object path, it is assumed to be unique, i.e. there
00082            is a one to one mapping between physical paths and docids.  This
00083            will be large, and is only used for breadcrumbs.
00084 
00085            id is the docid
00086         """
00087 
00088         PathIndex.insertEntry(self, comp, id, level)
00089 
00090         if parent_path is not None:
00091             if not self._index_parents.has_key(parent_path):
00092                 self._index_parents[parent_path] = IISet()
00093 
00094             self._index_parents[parent_path].insert(id)
00095 
00096         # We make the assumption that a full path corresponds one and only
00097         # one object.
00098 
00099         if object_path is not None:
00100             self._index_items[object_path] = id

Here is the caller graph for this function:

def ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.search (   self,
  path,
  default_level = 0,
  depth = -1,
  navtree = 0,
  navtree_start = 0 
)
path is either a string representing a
relative URL or a part of a relative URL or
a tuple (path,level).

level >= 0  starts searching at the given level
level <  0  not implemented yet

Definition at line 206 of file ExtendedPathIndex.py.

00206 
00207                                                              navtree_start=0):
00208         """
00209         path is either a string representing a
00210         relative URL or a part of a relative URL or
00211         a tuple (path,level).
00212 
00213         level >= 0  starts searching at the given level
00214         level <  0  not implemented yet
00215         """
00216 
00217         if isinstance(path, basestring):
00218             startlevel = default_level
00219         else:
00220             startlevel = int(path[1])
00221             path = path[0]
00222 
00223         absolute_path = isinstance(path, basestring) and path.startswith('/')
00224 
00225         comps = filter(None, path.split('/'))
00226 
00227         orig_comps = [''] + comps[:]
00228         # Optimization - avoid using the root set
00229         # as it is common for all objects anyway and add overhead
00230         # There is an assumption about catalog/index having
00231         # the same container as content
00232         if default_level == 0:
00233             indexpath = list(filter(None, self.getPhysicalPath()))
00234             while min(len(indexpath), len(comps)):
00235                 if indexpath[0] == comps[0]:
00236                     del indexpath[0]
00237                     del comps[0]
00238                     startlevel += 1
00239                 else:
00240                     break
00241 
00242         if len(comps) == 0:
00243             if depth == -1 and not navtree:
00244                 return IISet(self._unindex.keys())
00245 
00246         # Make sure that we get depth = 1 if in navtree mode
00247         # unless specified otherwise
00248 
00249         orig_depth = depth
00250         if depth == -1:
00251             depth = 0 or navtree
00252 
00253         # Optimized navtree starting with absolute path
00254         if absolute_path and navtree and depth == 1 and default_level==0:
00255             set_list = []
00256             # Insert root element
00257             if navtree_start >= len(orig_comps):
00258                 navtree_start = 0
00259             # create a set of parent paths to search
00260             for i in range(len(orig_comps), navtree_start, -1):
00261                 parent_path = '/'.join(orig_comps[:i])
00262                 parent_path = parent_path and parent_path or '/'
00263                 try:
00264                     set_list.append(self._index_parents[parent_path])
00265                 except KeyError:
00266                     pass
00267             return multiunion(set_list)
00268         # Optimized breadcrumbs
00269         elif absolute_path and navtree and depth == 0 and default_level==0:
00270             item_list = IISet()
00271             # Insert root element
00272             if navtree_start >= len(orig_comps):
00273                 navtree_start = 0
00274             # create a set of parent paths to search
00275             for i in range(len(orig_comps), navtree_start, -1):
00276                 parent_path = '/'.join(orig_comps[:i])
00277                 parent_path = parent_path and parent_path or '/'
00278                 try:
00279                     item_list.insert(self._index_items[parent_path])
00280                 except KeyError:
00281                     pass
00282             return item_list
00283         # Specific object search
00284         elif absolute_path and orig_depth == 0 and default_level == 0:
00285             try:
00286                 return IISet([self._index_items[path]])
00287             except KeyError:
00288                 return IISet()
00289         # Single depth search
00290         elif absolute_path and orig_depth == 1 and default_level == 0:
00291             # only get objects contained in requested folder
00292             try:
00293                 return self._index_parents[path]
00294             except KeyError:
00295                 return IISet()
00296         # Sitemaps, relative paths, and depth queries
00297         elif startlevel >= 0:
00298 
00299             pathset = None # Same as pathindex
00300             navset  = None # For collecting siblings along the way
00301             depthset = None # For limiting depth
00302 
00303             if navtree and depth and \
00304                    self._index.has_key(None) and \
00305                    self._index[None].has_key(startlevel):
00306                 navset = self._index[None][startlevel]
00307 
00308             for level in range(startlevel, startlevel+len(comps) + depth):
00309                 if level-startlevel < len(comps):
00310                     comp = comps[level-startlevel]
00311                     if not self._index.has_key(comp) or not self._index[comp].has_key(level): 
00312                         # Navtree is inverse, keep going even for
00313                         # nonexisting paths
00314                         if navtree:
00315                             pathset = IISet()
00316                         else:
00317                             return IISet()
00318                     else:
00319                         pathset = intersection(pathset,
00320                                                      self._index[comp][level])
00321                     if navtree and depth and \
00322                            self._index.has_key(None) and \
00323                            self._index[None].has_key(level+depth):
00324                         navset  = union(navset, intersection(pathset,
00325                                               self._index[None][level+depth]))
00326                 if level-startlevel >= len(comps) or navtree:
00327                     if self._index.has_key(None) and self._index[None].has_key(level):
00328                         depthset = union(depthset, intersection(pathset,
00329                                                     self._index[None][level]))
00330 
00331             if navtree:
00332                 return union(depthset, navset) or IISet()
00333             elif depth:
00334                 return depthset or IISet()
00335             else:
00336                 return pathset or IISet()
00337 
00338         else:
00339             results = IISet()
00340             for level in range(0,self._depth + 1):
00341                 ids = None
00342                 error = 0
00343                 for cn in range(0,len(comps)):
00344                     comp = comps[cn]
00345                     try:
00346                         ids = intersection(ids,self._index[comp][level+cn])
00347                     except KeyError:
00348                         error = 1
00349                 if error==0:
00350                     results = union(results,ids)
00351             return results

Here is the call graph for this function:

Here is the caller graph for this function:

hook for (Z)Catalog 

Definition at line 155 of file ExtendedPathIndex.py.

00155 
00156     def unindex_object(self, docid):
00157         """ hook for (Z)Catalog """
00158 
00159         if not self._unindex.has_key(docid):
00160             logger.log(logging.INFO,
00161                        'Attempt to unindex nonexistent object with id '
00162                        '%s' % docid)
00163             return
00164 
00165         # There is an assumption that paths start with /
00166         path = self._unindex[docid]
00167         if not path.startswith('/'):
00168             path = '/'+path
00169         comps =  path.split('/')
00170         parent_path = '/'.join(comps[:-1])
00171 
00172         def unindex(comp, level, docid=docid, parent_path=None,
00173                                                             object_path=None):
00174             try:
00175                 self._index[comp][level].remove(docid)
00176 
00177                 if not self._index[comp][level]:
00178                     del self._index[comp][level]
00179 
00180                 if not self._index[comp]:
00181                     del self._index[comp]
00182                 # Remove parent_path and object path elements
00183                 if parent_path is not None:
00184                     self._index_parents[parent_path].remove(docid)
00185                     if not self._index_parents[parent_path]:
00186                         del self._index_parents[parent_path]
00187                 if object_path is not None:
00188                     del self._index_items[object_path]
00189             except KeyError:
00190                 logger.log(logging.INFO,
00191                            'Attempt to unindex object with id '
00192                            '%s failed' % docid)
00193 
00194         for level in range(len(comps[1:])):
00195             comp = comps[level+1]
00196             unindex(comp, level)
00197 
00198         # Remove the terminator
00199         level = len(comps[1:])
00200         comp = None
00201         unindex(comp, level-1, parent_path=parent_path, object_path=path)
00202 
00203         self._length.change(-1)
00204         del self._unindex[docid]

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Definition at line 73 of file ExtendedPathIndex.py.

Definition at line 72 of file ExtendedPathIndex.py.

tuple ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.index_html = DTMLFile('dtml/index', globals()) [static]

Definition at line 396 of file ExtendedPathIndex.py.

Definition at line 68 of file ExtendedPathIndex.py.

Initial value:
(
        {'label': 'Settings',
         'action': 'manage_main',
         'help': ('ExtendedPathIndex','ExtendedPathIndex_Settings.stx')},
    )

Definition at line 40 of file ExtendedPathIndex.py.

tuple ExtendedPathIndex.ExtendedPathIndex.ExtendedPathIndex.manage_workspace = DTMLFile('dtml/manageExtendedPathIndex', globals()) [static]

Definition at line 397 of file ExtendedPathIndex.py.

Definition at line 38 of file ExtendedPathIndex.py.

Initial value:
("query", "level", "operator", "depth", "navtree",
                                                              "navtree_start")

Definition at line 46 of file ExtendedPathIndex.py.


The documentation for this class was generated from the following file: