Package loci.formats

Class Memoizer

java.lang.Object
loci.formats.ReaderWrapper
loci.formats.Memoizer
All Implemented Interfaces:
Closeable, AutoCloseable, loci.formats.ICompressedTileReader, loci.formats.IFormatHandler, loci.formats.IFormatReader, loci.formats.IMetadataConfigurable, loci.formats.IPyramidHandler
public class Memoizer extends loci.formats.ReaderWrapper
ReaderWrapper implementation which caches the state of the delegate (including and other ReaderWrapper instances) after setId(String) has been called. Initializing a Bio-Formats reader can consume substantial time and memory. Most of the initialization time is spent in the setId(String) call. Various factors can impact the performance of this step including the file size, the amount of metadata in the image and also the file format itself. With the Memoizer reader wrapper, if the time required to call the setId(String) method is larger than minimumElapsed, the initialized reader including all reader wrappers will be cached in a memo file via saveMemo(). Any subsequent call to setId(String) with a reader decorated by the Memoizer on the same input file will load the reader from the memo file using loadMemo() instead of performing a full reader initialization. In essence, the speed-up gained from memoization will happen only after the first initialization of the reader for a particular file.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    Memoizer.Deser
     
    static class 
    Memoizer.KryoDeser
     

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final long
    DEFAULT_MINIMUM_ELAPSED
    Default value for minimumElapsed if none is provided in the constructor.
    protected Memoizer.Deser
    ser
     
    static final Integer
    VERSION
    Default file version.

    Fields inherited from class loci.formats.ReaderWrapper

    reader

    Fields inherited from interface loci.formats.IFormatReader

    CAN_GROUP, CANNOT_GROUP, MUST_GROUP

  • Constructor Summary

    Constructors
    Constructor
    Description
    Memoizer()
    Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
    Memoizer(long minimumElapsed)
    Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
    Memoizer(long minimumElapsed, File directory)
    Constructs a memoizer around a new ImageReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
    Memoizer(loci.formats.IFormatReader r)
    Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
    Memoizer(loci.formats.IFormatReader r, long minimumElapsed)
    Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
    Memoizer(loci.formats.IFormatReader r, long minimumElapsed, File directory)
    Constructs a memoizer around the given IFormatReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
    Memoizer(loci.formats.IFormatReader r, File directory)
    Constructs a memoizer around the given IFormatReader creating memo file under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.

  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    cleanup()
     
    void
    close()
     
    void
    close(boolean fileOnly)
     
    boolean
    deleteMemo()
    Force the current memo file to be deleted, if it exists.
    protected boolean
    deleteQuietly(File file)
    Attempts to delete an existing file, logging at warn if the deletion returns false or at error if an exception is thrown.
    boolean
    generateMemo(String file)
    Convenience method to generate (or regenerate) the memo file for a given file.
    protected Memoizer.Deser
    getDeser()
    Returns a configured Kryo instance.
    File
    getMemoFile()
     
    File
    getMemoFile(String id)
    Constructs a File object from id string.
    loci.formats.meta.MetadataStore
    getMetadataStore()
     
    protected loci.formats.services.OMEXMLService
    getService()
     
    protected loci.formats.IFormatReader
    handleMetadataStore(loci.formats.IFormatReader memo)
    Return the IFormatReader instance that is passed in or null if it has been invalidated, which will include the instance being closed.
    boolean
    isLoadedFromMemo()
    Returns whether the ReaderWrapper.reader instance currently active was loaded from the memo file during setId(String).
    boolean
    isSavedToMemo()
    Returns whether the ReaderWrapper.reader instance currently active was saved to the memo file during setId(String).
    boolean
    isVersionChecking()
    Returns whether or not version checking is done based upon major/minor version numbers.
    protected boolean
    isWritableDirectory(File writeDirectory)
    Test if the given File instance is a writable directory.
    loci.formats.IFormatReader
    loadMemo()
    Load a memo file if possible, returning a null if not.
    static void
    main(String[] args)
     
    boolean
    saveMemo()
    Save a reader including all reader wrappers inside a memo file.
    void
    setId(String id)
     
    void
    setMetadataStore(loci.formats.meta.MetadataStore store)
     
    void
    setVersionChecking(boolean version)
    Set whether version checking is done based upon major/minor version numbers.
    void
    skipSave(boolean skip)
    Set whether a memo file should be saved if a valid file is not present.
    protected Slf4JStopWatch
    stopWatch()
     
    boolean
    versionMismatch()
    Returns true if the version of the memo file as returned by Memoizer.Deser.loadReleaseVersion() and Memoizer.Deser.loadRevision() do not match the current version as specified by FormatTools.VERSION.

    Methods inherited from class loci.formats.ReaderWrapper

    coreIndexToSeries, duplicate, fileGroupOption, get16BitLookupTable, get8BitLookupTable, getAdvancedSeriesUsedFiles, getAdvancedUsedFiles, getBitsPerPixel, getCoreIndex, getCoreMetadataList, getCurrentFile, getDatasetStructureDescription, getDimensionOrder, getDomains, getEffectiveSizeC, getFillColor, getFormat, getGlobalMetadata, getImageCount, getIndex, getIndex, getMetadataOptions, getMetadataStoreRoot, getMetadataValue, getModuloC, getModuloT, getModuloZ, getNativeDataType, getOptimalTileHeight, getOptimalTileWidth, getPixelType, getPossibleDomains, getReader, getRequiredDirectories, getResolution, getResolutionCount, getRGBChannelCount, getSeries, getSeriesCount, getSeriesMetadata, getSeriesMetadataValue, getSeriesUsedFiles, getSeriesUsedFiles, getSizeC, getSizeT, getSizeX, getSizeY, getSizeZ, getSuffixes, getSupportedMetadataLevels, getThumbSizeX, getThumbSizeY, getTileCodec, getTileCodecOptions, getTileColumns, getTileRows, getUnderlyingReaders, getUsedFiles, getUsedFiles, getZCTCoords, getZCTModuloCoords, hasCompanionFiles, hasFlattenedResolutions, isFalseColor, isGroupFiles, isIndexed, isInterleaved, isInterleaved, isLittleEndian, isMetadataComplete, isMetadataFiltered, isNormalized, isOrderCertain, isOriginalMetadataPopulated, isRGB, isSingleFile, isThisType, isThisType, isThisType, isThisType, isThumbnailSeries, openBytes, openBytes, openBytes, openBytes, openCompressedBytes, openCompressedBytes, openPlane, openThumbBytes, reopenFile, seriesToCoreIndex, setCoreIndex, setFillColor, setFlattenedResolutions, setGroupFiles, setMetadataFiltered, setMetadataOptions, setNormalized, setOriginalMetadataPopulated, setResolution, setSeries, unwrap, unwrap, unwrap

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • VERSION

      public static final Integer VERSION
      Default file version. Bumping this number will invalidate all other cached items. This should happen when the order and type of objects stored in the memo file changes.

    • DEFAULT_MINIMUM_ELAPSED

      public static final long DEFAULT_MINIMUM_ELAPSED
      Default value for minimumElapsed if none is provided in the constructor.
      See Also:

    • ser

      protected transient Memoizer.Deser ser

  • Constructor Details

    • Memoizer

      public Memoizer()
      Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.

    • Memoizer

      public Memoizer(long minimumElapsed)
      Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.

    • Memoizer

      public Memoizer(long minimumElapsed, File directory)
      Constructs a memoizer around a new ImageReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

    • Memoizer

      public Memoizer(loci.formats.IFormatReader r)
      Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
      Parameters:
      r - an IFormatReader instance. If null, a new ImageReader is created.

    • Memoizer

      public Memoizer(loci.formats.IFormatReader r, long minimumElapsed)
      Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.

    • Memoizer

      public Memoizer(loci.formats.IFormatReader r, File directory)
      Constructs a memoizer around the given IFormatReader creating memo file under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

    • Memoizer

      public Memoizer(loci.formats.IFormatReader r, long minimumElapsed, File directory)
      Constructs a memoizer around the given IFormatReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

  • Method Details

    • isLoadedFromMemo

      public boolean isLoadedFromMemo()
      Returns whether the ReaderWrapper.reader instance currently active was loaded from the memo file during setId(String).
      Returns:
      true if the reader was loaded from the memo file, false otherwise.

    • isSavedToMemo

      public boolean isSavedToMemo()
      Returns whether the ReaderWrapper.reader instance currently active was saved to the memo file during setId(String).
      Returns:
      true if the reader was saved to the memo file, false otherwise.

    • isVersionChecking

      public boolean isVersionChecking()
      Returns whether or not version checking is done based upon major/minor version numbers.
      Returns:
      true if version checking is done based upon major/minor version numbers, false otherwise.

    • versionMismatch

      public boolean versionMismatch() throws IOException
      Returns true if the version of the memo file as returned by Memoizer.Deser.loadReleaseVersion() and Memoizer.Deser.loadRevision() do not match the current version as specified by FormatTools.VERSION.
      Throws:
      IOException

    • setVersionChecking

      public void setVersionChecking(boolean version)
      Set whether version checking is done based upon major/minor version numbers. If true, then a mismatch between the major/minor version of the calling code (e.g. 4.4) and the major/minor version saved in the memo file (e.g. 5.0) will result in the memo file being invalidated. If false (default), a mismatch in the Git commit hashes will invalidate the memo file. This method allows for less strict version checking.
      Parameters:
      version - a boolean specifying whether version checking is done based upon major/minor version numbers to invalidate the memo file

    • skipSave

      public void skipSave(boolean skip)
      Set whether a memo file should be saved if a valid file is not present. If false (default), then a memo file will be saved. This may cause an existing memo file to be overwritten. If true, then a memo file will not be saved. This effectively makes the current Memoizer read-only.

    • cleanup

      protected void cleanup()

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class loci.formats.ReaderWrapper
      Throws:
      IOException

    • close

      public void close(boolean fileOnly) throws IOException
      Specified by:
      close in interface loci.formats.IFormatReader
      Overrides:
      close in class loci.formats.ReaderWrapper
      Throws:
      IOException

    • setId

      public void setId(String id) throws loci.formats.FormatException, IOException
      Specified by:
      setId in interface loci.formats.IFormatHandler
      Overrides:
      setId in class loci.formats.ReaderWrapper
      Throws:
      loci.formats.FormatException
      IOException

    • setMetadataStore

      public void setMetadataStore(loci.formats.meta.MetadataStore store)
      Specified by:
      setMetadataStore in interface loci.formats.IFormatReader
      Overrides:
      setMetadataStore in class loci.formats.ReaderWrapper

    • getMetadataStore

      public loci.formats.meta.MetadataStore getMetadataStore()
      Specified by:
      getMetadataStore in interface loci.formats.IFormatReader
      Overrides:
      getMetadataStore in class loci.formats.ReaderWrapper

    • deleteQuietly

      protected boolean deleteQuietly(File file)
      Attempts to delete an existing file, logging at warn if the deletion returns false or at error if an exception is thrown.
      Returns:
      the result from File.delete() or false if an exception is thrown.

    • getDeser

      protected Memoizer.Deser getDeser()
      Returns a configured Kryo instance. This method can be modified by consumers. The returned instance is not thread-safe.
      Returns:
      a non-null Kryo instance.

    • getService

      protected loci.formats.services.OMEXMLService getService() throws loci.formats.MissingLibraryException
      Throws:
      loci.formats.MissingLibraryException

    • stopWatch

      protected Slf4JStopWatch stopWatch()

    • getMemoFile

      public File getMemoFile(String id)
      Constructs a File object from id string. This method can be modified by consumers, but then existing memo files will not be found.
      Parameters:
      id - the path passed to setId(java.lang.String)
      Returns:
      a File object pointing at the location of the memo file

    • isWritableDirectory

      protected boolean isWritableDirectory(File writeDirectory)
      Test if the given File instance is a writable directory.
      Parameters:
      writeDirectory - a possible writable directory
      Returns:
      if the given File is indeed a writable directory

    • loadMemo

      public loci.formats.IFormatReader loadMemo() throws IOException, loci.formats.FormatException
      Load a memo file if possible, returning a null if not. Corrupt memo files will be deleted if possible. Kryo exceptions should never propagate to the caller. Only the regular Bio-Formats exceptions should be thrown.
      Throws:
      IOException
      loci.formats.FormatException

    • saveMemo

      public boolean saveMemo()
      Save a reader including all reader wrappers inside a memo file.

    • deleteMemo

      public boolean deleteMemo()
      Force the current memo file to be deleted, if it exists.
      Returns:
      true if the delete succeeded

    • getMemoFile

      public File getMemoFile()
      Returns:
      current memo file (may be null)

    • handleMetadataStore

      protected loci.formats.IFormatReader handleMetadataStore(loci.formats.IFormatReader memo) throws loci.formats.MissingLibraryException
      Return the IFormatReader instance that is passed in or null if it has been invalidated, which will include the instance being closed.
      • Serialization: If an unknown MetadataStore implementation is passed in when no memo file exists, then a replacement MetadataStore will be created and set on the ReaderWrapper.reader delegate before calling ReaderWrapper.setId(String). This stack will then be serialized, before any values are copied into userMetadataStore.
      • Deserialization: If an unknown MetadataStore implementation is set before calling setId(String) then ...
      Throws:
      loci.formats.MissingLibraryException

    • generateMemo

      public boolean generateMemo(String file) throws IOException
      Convenience method to generate (or regenerate) the memo file for a given file. Uses the cache directory and timing settings passed via Memoizer's constructor.
      Parameters:
      file - the file for which to generate a memo file
      Returns:
      true if a memo file was saved
      Throws:
      IOException

    • main

      public static void main(String[] args) throws Exception
      Throws:
      Exception