Back to index

plone3  3.1.7
Public Member Functions | Public Attributes
ResourceRegistries.tools.packer.KeywordMapper Class Reference
Inheritance diagram for ResourceRegistries.tools.packer.KeywordMapper:
Inheritance graph
[legend]

List of all members.

Public Member Functions

def __init__
def analyseKeywords
def analyse
def getKeywords
def sub

Public Attributes

 regexp
 encoder
 mapping

Detailed Description

Definition at line 7 of file packer.py.


Constructor & Destructor Documentation

def ResourceRegistries.tools.packer.KeywordMapper.__init__ (   self,
  regexp,
  encoder 
)

Reimplemented in ResourceRegistries.tools.packer.JavascriptKeywordMapper.

Definition at line 8 of file packer.py.

00008 
00009     def __init__(self, regexp, encoder):
00010         if isinstance(regexp, (str, unicode)):
00011             self.regexp = re.compile(regexp)
00012         else:
00013             self.regexp = regexp
00014         self.encoder = encoder
00015         self.mapping = {}


Member Function Documentation

Definition at line 75 of file packer.py.

00075 
00076     def analyse(self, input):
00077         self.mapping = self.analyseKeywords(input)

Here is the call graph for this function:

Definition at line 16 of file packer.py.

00016 
00017     def analyseKeywords(self, input):
00018         matches = self.regexp.findall(input)
00019 
00020         protected = {}
00021         keyword_count = {}
00022         index = 0
00023         for match in matches:
00024             if match not in keyword_count:
00025                 keyword_count[match] = 0
00026                 protected[self.encoder(index)] = index
00027                 index = index + 1
00028             keyword_count[match] = keyword_count[match] + 1
00029 
00030         for match in matches:
00031             if match in protected and keyword_count[match]:
00032                 keyword_count[match] = 0
00033 
00034         protected = {}
00035         for match in keyword_count:
00036             if not keyword_count[match]:
00037                 protected[match] = None
00038 
00039         ## sorted_matches = [(c,len(v),v) for v,c in keyword_count.iteritems()]
00040         # the above line implements the original behaviour, the code below
00041         # removes keywords which have not enough weight to be encoded, in total
00042         # this saves some bytes, because the total length of the generated
00043         # codes is a bit smaller. This needs corresponding code in the
00044         # fast_decode javascript function of the decoder, see comment there
00045         sorted_matches = []
00046         for value, count in keyword_count.iteritems():
00047             weight = count * len(value)
00048             if len(value) >= weight:
00049                 keyword_count[value] = 0
00050                 sorted_matches.append((0, value))
00051             else:
00052                 sorted_matches.append((weight, value))
00053         sorted_matches.sort()
00054         sorted_matches.reverse()
00055         sorted_matches = [x[-1] for x in sorted_matches]
00056 
00057         index = 0
00058         mapping = {}
00059         for match in sorted_matches:
00060             if not keyword_count[match]:
00061                 if match not in protected:
00062                     mapping[match] = (-1, match)
00063                 continue
00064             while 1:
00065                 encoded = self.encoder(index)
00066                 index = index + 1
00067                 if encoded in protected:
00068                     mapping[encoded] = (index-1, encoded)
00069                     continue
00070                 else:
00071                     break
00072             mapping[match] = (index-1, encoded)
00073 
00074         return mapping

Here is the caller graph for this function:

Definition at line 78 of file packer.py.

00078 
00079     def getKeywords(self):
00080         sorted = zip(self.mapping.itervalues(), self.mapping.iterkeys())
00081         sorted.sort()
00082         keywords = []
00083         for (index, encoded), value in sorted:
00084             if index >= 0:
00085                 if encoded != value:
00086                     keywords.append(value)
00087                 else:
00088                     keywords.append('')
00089         return keywords

Here is the caller graph for this function:

Definition at line 90 of file packer.py.

00090 
00091     def sub(self, input):
00092         def repl(m):
00093             return self.mapping.get(m.group(0), ('', m.group(0)))[1]
00094         return self.regexp.sub(repl, input)
00095 

Here is the caller graph for this function:


Member Data Documentation

Reimplemented in ResourceRegistries.tools.packer.JavascriptKeywordMapper.

Definition at line 13 of file packer.py.

Reimplemented in ResourceRegistries.tools.packer.JavascriptKeywordMapper.

Definition at line 14 of file packer.py.

Reimplemented in ResourceRegistries.tools.packer.JavascriptKeywordMapper.

Definition at line 10 of file packer.py.


The documentation for this class was generated from the following file: