Description

Normaliser objects are chained after Extracters in order to transform the data from the record or query. A Normaliser might standardise case, perform stemming or transform a date into ISO8601 format. Normalisers are also needed to transform the terms in a request into the same format as the term in the Index. For example a date index might be searched using a free text date and that would need to be parsed into the normalised form in order to compare it with the stored data.

Implementations

• normaliser.SimpleNormliser
  Simply returns the data with no modifications
• normaliser.CaseNormliser
  Returns the data in all lower case
• normaliser.ArticleNormliser
  Strips leading english articles (the, a, an)
• normaliser.PossessiveNormliser
  Strips trailing possessive 's, eg truck's -> truck
• normaliser.IntNormliser
  Returns a long int from the numeric data, or None if there is no mapping possible
• normaliser.StringIntNormliser
  Returns a 0 padded length 12 string for an integer
• normaliser.StoplistNormliser
  Strip stoplisted words from the data
• normaliser.StemNormliser
  Apply a stemming algorithm to the data. Implementation can be configured for 11+ different languages.
• normaliser.DiacriticNormliser
  Map characters with diacritics to their simple form. eg é -> e

API

Function	Parameters	Returns	Description
__init__	config, parent
process_string	session, string	data	Process a string into an alternative form