Process files as streams of records, each with tags with values.
Many files in biology are structured as multiple records, each of which can be broken down into lines composed from some tag and an associated value. For example, EMBL files have two letter tags and values extend from column 5 to the end of the line. There are a vast array of files all of which have broadly similar structures, and this package aims to provide a framework within which parsing strategies and data consumers can be reused as much as possible.
The data associated with each record is represented by a stream of events
encapsulated by callbacks on the
TagValueListener interface. It is up to the
user to provide implementations of this interface that build static
representations of the data if they so wish.
Parser and Pushing Sub-Documents
Often file formats have embedded sub-documents. For example, in EMBL format files the feature table area is identical to that in GENBANK files if the first five columns are ignored. In ACeDB files, every time an ace-tag is found, it causes a new sub-document to be induced with its own structure and set of allowed tags and values. Python code uses indent depth to represent code blocks.
TagValueListener objects to request
that all of the values associated with the current tag should be handled by a
TagValueListener pair. The
Parser instance will use the original
to process the line as before, and then take the value that would have been
handed to the listener's value method, and present it to the newly registered
TagValueParser to tokenize into tag and value portions. That tag
and value will then be passed onto the new
TagValueListener can itself choose to push a new pair of parser
and listener to start a new sub-sub-document. This can be repeated to arbitrary
depth. As soon as a parser and listener pair are registered, the pushed listener
receives a startRecord() message. Once the entire containing record ends (due to
a record separator line such as "//", or because the end of file has been
reached), or if the tag that caused the delegation ends, the pushed listener
will receive the appropriate endTag() message and also endRecord().
TagDelegator is a useful helper class that always delegates to
a given parser and listener pair on a given tag.
Rewriting the Event Stream
Often while parsing, you will need to change tag names or modify values. In the simple case, all the tags and values will be String instances. You will probably want different types, such as the numeric objects (Double, Int and their friends), or to instantiate your own objects from these Strings. Additionaly, some values are themselves better represented as lists of more fundamental items. There are several TagValueListener helper classes that extend TagValueWrapper that allow you to configure a chain of event transducors while writing the minimal amount of code.
TagMapper remaps a sub-set of the tags it sees. For example, it
could be configured to replace all "FOO_ID" tags with "accession.number".
ValueChanger intercepts the value() calls for specific tags and
uses either a
ValueChanger.Splitter instance to replace or sub-devide the value
before passing it onto another listener.
Interface Summary Interface Description BoundaryFinder ChangeTable.ChangerCallback used to produce a new value from an old one. ChangeTable.SplitterCallback used to produce a list of values from a single old one. PropertyChangerInterface for objects that change tag names or properties systematically. StateMachine.ExitNotificationInterface implemented by State listeners that want notification when a transition leaves the State. StateMachine.StateInterface for a State within this StateMachine TagValueContextCommunication interface between Parser and a TagValueListener that allows listeners to request that a parser/listener pair be pushed onto the stack to handle the current tag. TagValueListenerAn object that wishes to be informed of events during the parsing of a file. TagValueParserTokenize single records (lines of text, objects) into a tag and a value. TagValueWrapperInterface for TagValueListeners that wrap other TagValueListeners
Class Summary Class Description AbstractWrapperAn abstract TagValueWrapper that does nothing! AggregatorJoins multipel values into single values. AnnotationBuilderBuilds an Annotation tree from TagValue events using an AnnotationType to work out which fields are of what type. ChangeTableA mapping between keys and actions to turn old values into new values. ChangeTable.ChainedChangerAn implementation of Changer that applies a list of Changer instances to the value in turn. EchoA simple listener that just echoes events back to the console. FormatsThis is intended as a repository for tag-value and AnnotationType information about common file formats. Index2Model IndexerListens to tag-value events and passes on indexing events to an IndexStore. Indexer2Listens to tag-value events and passes on indexing events to an IndexStore. LineSplitParserA parser that splits a line into tag/value at a given column number. MultiTaggerPartician multiple values for a tag into their own tag groups. ParserEncapsulate the parsing of lines from a buffered reader into tag-value events. ParserListener
ParserListeneris an immutable pairing of a parser and listener.
RegexChangerA ValueChanger.Changer that returns a specific match value using a regex Pattern. RegexFieldFinder RegexParserA TagValueParser that splits a line based upon a regular expression. RegexSplitterA ValueChanger.Splitter that splits a line of text using a regular expression, returning one value per match. SimpleTagValueWrapperHelper class to wrap one TagValueListener inside another one. StateMachineThis class implements a state machine for parsing events from the Parser class. TagDelegatorPushes a new parser and listener, or delegate to a listener depending on the tag. TagDropperSilently drop all tags except those specified, and pass the rest onto a delegate. TagMapper
TagMappermaps arbitrary object keys to new keys.
TagRenamerRename tags using a TagMapper. TagValueUtility class for representing tag-value pairs for TagValueParser implementors. ValueChangerIntercept the values associated with some tags and change them systematically.