Class CompactedDataStoreFactory

  • All Implemented Interfaces:
    DataStoreFactory

    public class CompactedDataStoreFactory
    extends Object
    implements DataStoreFactory

    Builder for a data store that is backed by a java.nio.MappedByteBuffer. This has a limitation that the total size of the mapped buffer and therefore the hash table can not exceed 2 gigs.

    The data store file has the following structure.

     file: header, hash table, nameArray, nameTable, hitTable
    
     header:
       int hashTablePos, // byte offset in file
       int hitTablePos,  // byte offset in file
       int nameArrayPos, // byte offset in file
       int nameTablePos, // byte offset in file
       int wordLength,
       int serializedPackingLength,
       byte[] serializedPacking
    
       hash table:
         int hashTableLength,
         int[hashTableLength] hits // index into hitTable
    
      nameArray:
        int nameArrayLength,
        int[nameArrayLength] nameRecord // byte offset into nameTable
    
      nameRecord:
        int nameTableOffset
        int sequenceStartOffset
     
      nameTable:
        int nameTableSize, // size in bytes
        (short nameLength, char[nameLength] name)[nameTableSize] names
    
      hitTable:
        int hitTableSize, // size in bytes
        hitTableRecord[hitTableSize] hits
    
      hitTableRecord:
        int hitCount,
        hitRecord[hitCount] hit
    
      hit:
        int offset    // offset into the sequence
     

    Author:
    Matthew Pocock, Thomas Down