Back to index

moin  1.9.0~rc2
Public Member Functions | Static Public Attributes | Private Attributes | Static Private Attributes
MoinMoin.support.xappy.fieldactions.FieldActions Class Reference

List of all members.

Public Member Functions

def __init__
def add
def perform

Static Public Attributes

int STORE_CONTENT = 1
int INDEX_EXACT = 2
int INDEX_FREETEXT = 3
int SORTABLE = 4
int COLLAPSE = 5
int TAG = 6
int FACET = 7
int SORT_AND_COLLAPSE = 1

Private Attributes

 _actions
 _fieldname

Static Private Attributes

list _unsupported_actions = []
dictionary _action_info

Detailed Description

An object describing the actions to be performed on a field.

The supported actions are:

- `STORE_CONTENT`: store the unprocessed content of the field in the search
  engine database.  All fields which need to be displayed or used when
  displaying the search results need to be given this action.

- `INDEX_EXACT`: index the exact content of the field as a single search
  term.  Fields whose contents need to be searchable as an "exact match"
  need to be given this action.

- `INDEX_FREETEXT`: index the content of this field as text.  The content
  will be split into terms, allowing free text searching of the field.  Four
  optional parameters may be supplied:

  - 'weight' is a multiplier to apply to the importance of the field.  This
    must be an integer, and the default value is 1.
  - 'language' is the language to use when processing the field.  This can
    be expressed as an ISO 2-letter language code.  The supported languages
    are those supported by the xapian core in use.
  - 'stop' is an iterable of stopwords to filter out of the generated
    terms.  Note that due to Xapian design, only non-positional terms are
    affected, so this is of limited use.
  - 'spell' is a boolean flag - if true, the contents of the field will be
    used for spelling correction.
  - 'nopos' is a boolean flag - if true, positional information is not
    stored.
  - 'allow_field_specific' is a boolean flag - if False, prevents terms with the field
    prefix being generated.  This means that searches specific to this
    field will not work, and thus should only be used when only non-field
    specific searches are desired.  Defaults to True.
  - 'search_by_default' is a boolean flag - if False, the field will not be
    searched by non-field specific searches.  If True, or omitted, the
    field will be included in searches for non field-specific searches.

- `SORTABLE`: index the content of the field such that it can be used to
  sort result sets.  It also allows result sets to be restricted to those
  documents with a field values in a given range.  One optional parameter
  may be supplied:

  - 'type' is a value indicating how to sort the field.  It has several
    possible values:

    - 'string' - sort in lexicographic (ie, alphabetical) order.
      This is the default, used if no type is set.
    - 'float' - treat the values as (decimal representations of) floating
      point numbers, and sort in numerical order.  The values in the field
      must be valid floating point numbers (according to Python's float()
      function).
    - 'date' - sort in date order.  The values must be valid dates (either
      Python datetime.date objects, or ISO 8601 format (ie, YYYYMMDD or
      YYYY-MM-DD).

- `COLLAPSE`: index the content of the field such that it can be used to
  "collapse" result sets, such that only the highest result with each value
  of the field will be returned.

- `TAG`: the field contains tags; these are strings, which will be matched
  in a case insensitive way, but otherwise must be exact matches.  Tag
  fields can be searched for by making an explict query (ie, using
  query_field(), but not with query_parse()).  A list of the most frequent
  tags in a result set can also be accessed easily.

- `FACET`: the field represents a classification facet; these are strings
  which will be matched exactly, but a list of all the facets present in
  the result set can also be accessed easily - in addition, a suitable
  subset of the facets, and a selection of the facet values, present in the
  result set can be calculated.  One optional parameter may be supplied:

  - 'type' is a value indicating the type of facet contained in the field:

    - 'string' - the facet values are exact binary strings.
    - 'float' - the facet values are floating point numbers.

Definition at line 199 of file fieldactions.py.


Constructor & Destructor Documentation

Definition at line 298 of file fieldactions.py.

00298 
00299     def __init__(self, fieldname):
00300         # Dictionary of actions, keyed by type.
00301         self._actions = {}
00302         self._fieldname = fieldname


Member Function Documentation

def MoinMoin.support.xappy.fieldactions.FieldActions.add (   self,
  field_mappings,
  action,
  kwargs 
)
Add an action to perform on a field.

Definition at line 303 of file fieldactions.py.

00303 
00304     def add(self, field_mappings, action, **kwargs):
00305         """Add an action to perform on a field.
00306 
00307         """
00308         if action in self._unsupported_actions:
00309             raise errors.IndexerError("Action unsupported with this release of xapian")
00310 
00311         if action not in (FieldActions.STORE_CONTENT,
00312                           FieldActions.INDEX_EXACT,
00313                           FieldActions.INDEX_FREETEXT,
00314                           FieldActions.SORTABLE,
00315                           FieldActions.COLLAPSE,
00316                           FieldActions.TAG,
00317                           FieldActions.FACET,
00318                          ):
00319             raise errors.IndexerError("Unknown field action: %r" % action)
00320 
00321         info = self._action_info[action]
00322 
00323         # Check parameter names
00324         for key in kwargs.keys():
00325             if key not in info[1]:
00326                 raise errors.IndexerError("Unknown parameter name for action %r: %r" % (info[0], key))
00327 
00328         # Fields cannot be indexed both with "EXACT" and "FREETEXT": whilst we
00329         # could implement this, the query parser wouldn't know what to do with
00330         # searches.
00331         if action == FieldActions.INDEX_EXACT:
00332             if FieldActions.INDEX_FREETEXT in self._actions:
00333                 raise errors.IndexerError("Field %r is already marked for indexing "
00334                                    "as free text: cannot mark for indexing "
00335                                    "as exact text as well" % self._fieldname)
00336         if action == FieldActions.INDEX_FREETEXT:
00337             if FieldActions.INDEX_EXACT in self._actions:
00338                 raise errors.IndexerError("Field %r is already marked for indexing "
00339                                    "as exact text: cannot mark for indexing "
00340                                    "as free text as well" % self._fieldname)
00341 
00342         # Fields cannot be indexed as more than one type for "SORTABLE": to
00343         # implement this, we'd need to use a different prefix for each sortable
00344         # type, but even then the search end wouldn't know what to sort on when
00345         # searching.  Also, if they're indexed as "COLLAPSE", the value must be
00346         # stored in the right format for the type "SORTABLE".
00347         if action == FieldActions.SORTABLE or action == FieldActions.COLLAPSE:
00348             if action == FieldActions.COLLAPSE:
00349                 sorttype = None
00350             else:
00351                 try:
00352                     sorttype = kwargs['type']
00353                 except KeyError:
00354                     sorttype = 'string'
00355             kwargs['type'] = sorttype
00356             action = FieldActions.SORT_AND_COLLAPSE
00357 
00358             try:
00359                 oldsortactions = self._actions[FieldActions.SORT_AND_COLLAPSE]
00360             except KeyError:
00361                 oldsortactions = ()
00362 
00363             if len(oldsortactions) > 0:
00364                 for oldsortaction in oldsortactions:
00365                     oldsorttype = oldsortaction['type']
00366 
00367                 if sorttype == oldsorttype or oldsorttype is None:
00368                     # Use new type
00369                     self._actions[action] = []
00370                 elif sorttype is None:
00371                     # Use old type
00372                     return
00373                 else:
00374                     raise errors.IndexerError("Field %r is already marked for "
00375                                                "sorting, with a different "
00376                                                "sort type" % self._fieldname)
00377 
00378         if 'prefix' in info[3]:
00379             field_mappings.add_prefix(self._fieldname)
00380         if 'slot' in info[3]:
00381             purposes = info[3]['slot']
00382             if isinstance(purposes, basestring):
00383                 field_mappings.add_slot(self._fieldname, purposes)
00384             else:
00385                 slotnum = None
00386                 for purpose in purposes:
00387                     slotnum = field_mappings.get_slot(self._fieldname, purpose)
00388                     if slotnum is not None:
00389                         break
00390                 for purpose in purposes:
00391                     field_mappings.add_slot(self._fieldname, purpose, slotnum=slotnum)
00392 
00393         # Make an entry for the action
00394         if action not in self._actions:
00395             self._actions[action] = []
00396 
00397         # Check for repetitions of actions
00398         for old_action in self._actions[action]:
00399             if old_action == kwargs:
00400                 return
00401 
00402         # Append the action to the list of actions
00403         self._actions[action].append(kwargs)

def MoinMoin.support.xappy.fieldactions.FieldActions.perform (   self,
  doc,
  value,
  context 
)
Perform the actions on the field.

- `doc` is a ProcessedDocument to store the result of the actions in.
- `value` is a string holding the value of the field.
- `context` is an ActionContext object used to keep state in.

Definition at line 404 of file fieldactions.py.

00404 
00405     def perform(self, doc, value, context):
00406         """Perform the actions on the field.
00407 
00408         - `doc` is a ProcessedDocument to store the result of the actions in.
00409         - `value` is a string holding the value of the field.
00410         - `context` is an ActionContext object used to keep state in.
00411 
00412         """
00413         for type, actionlist in self._actions.iteritems():
00414             info = self._action_info[type]            
00415             for kwargs in actionlist:
00416                 info[2](self._fieldname, doc, value, context, **kwargs)


Member Data Documentation

Initial value:
{
        STORE_CONTENT: ('STORE_CONTENT', (), _act_store_content, {}, ),
        INDEX_EXACT: ('INDEX_EXACT', (), _act_index_exact, {'prefix': True}, ),
        INDEX_FREETEXT: ('INDEX_FREETEXT', ('weight', 'language', 'stop', 'spell', 'nopos', 'allow_field_specific', 'search_by_default', ), 
            _act_index_freetext, {'prefix': True, }, ),
        SORTABLE: ('SORTABLE', ('type', ), None, {'slot': 'collsort',}, ),
        COLLAPSE: ('COLLAPSE', (), None, {'slot': 'collsort',}, ),
        TAG: ('TAG', (), _act_tag, {'prefix': True,}, ),
        FACET: ('FACET', ('type', ), _act_facet, {'prefix': True, 'slot': 'facet',}, ),

        SORT_AND_COLLAPSE: ('SORT_AND_COLLAPSE', ('type', ), _act_sort_and_collapse, {'slot': 'collsort',}, ),
    }

Definition at line 417 of file fieldactions.py.

Definition at line 300 of file fieldactions.py.

Definition at line 301 of file fieldactions.py.

Definition at line 291 of file fieldactions.py.

Definition at line 282 of file fieldactions.py.

Definition at line 284 of file fieldactions.py.

Definition at line 279 of file fieldactions.py.

Definition at line 280 of file fieldactions.py.

Definition at line 289 of file fieldactions.py.

Definition at line 281 of file fieldactions.py.

Definition at line 278 of file fieldactions.py.

Definition at line 283 of file fieldactions.py.


The documentation for this class was generated from the following file: