Description

An Index is an object which defines an access point into records and is responsable for extracting that information from them. It can then store the information extracted in an IndexStore. The entry point can be defined using one or more XPath expressions, and the extraction process can be defined using a workflow chain of standard objects. These chains must start with an Extracter, but from there might then include PreParsers, Parsers, Transformers, Normalisers and even other Indexes.

A simple chain for a stemmed keyword index might be:

• KeywordExtracter
• CaseNormaliser
• StemNormaliser

Whereas a chain which takes a URL from an attribute, fetches it and then extract s title keywords from it might look like:

• ExactExtracter
• URLNormaliser
• URLFetchPreParser
• HtmlPreParser
• DefaultParser
• TitleIndex
• KeywordExtracter
• CaseNormaliser

Implementations

• index.SimpleIndex
  An index implementation that handles process chains for indexing and searchClauses for querying. Stores total matching records, total occurences, and record id plus occurences per record.
• index.ProximityIndex
  An index which also stores proximity information. Currently this is word based or element based, but further types could be added with appropriate Extracters.

API

Function	Parameters	Returns	Description
__init__	domNode, parentObject		The constructer takes a DOM tree containing the configuration of the index and the object which the database should consider as its parent, normally a database.
indexRecord	session, record		Index a record according to this index's rules
removeRecord	session, record		Remove the terms of a record from our index
updateRecord	session, newRecord, oldRecord		Remove the terms from the old record and add the new
search	session, searchClause, database	resultSet	Process search query against self and return resultSet
scan	session, searchClause, numberOfTerms, direction	list of term information	Handle a scan/browse request
sort	session, resultSets	resultSet	Sort resultsets according to the extraction rules of this index.
begin_indexing	session		Set up for batch mode indexing (many records will be indexed, optimize and save data on commit)
commit_indexing	session		Save batch mode indexing