org.janelia.saalfeldlab.n5.zarr

Class ZarrKeyValueReader

java.lang.Object

org.janelia.saalfeldlab.n5.zarr.ZarrKeyValueReader

All Implemented Interfaces:

AutoCloseable, org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer, org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader, org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader, org.janelia.saalfeldlab.n5.GsonN5Reader, org.janelia.saalfeldlab.n5.N5Reader

Direct Known Subclasses:

N5ZarrReader, ZarrKeyValueWriter

public class ZarrKeyValueReader
extends Object
implements org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader, org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

N5Reader implementation through KeyValueAccess with JSON attributes parsed with Gson.

Author:

Stephan Saalfeld, John Bogovic

Nested Class Summary

Nested classes/interfaces inherited from interface org.janelia.saalfeldlab.n5.N5Reader

org.janelia.saalfeldlab.n5.N5Reader.Version

Field Summary

Fields

Modifier and Type	Field and Description
protected ZarrJsonCache	cache
protected boolean	cacheMeta
protected HashMap<org.janelia.saalfeldlab.n5.DatasetAttributes,ZarrDatasetAttributes>	datasetAttributesMap
protected String	dimensionSeparator
protected com.google.gson.Gson	gson
protected org.janelia.saalfeldlab.n5.KeyValueAccess	keyValueAccess
protected boolean	mapN5DatasetAttributes
protected boolean	mergeAttributes
protected URI	uri
static org.janelia.saalfeldlab.n5.N5Reader.Version	VERSION
static org.janelia.saalfeldlab.n5.N5Reader.Version	VERSION_ZERO
static String	ZARR_FORMAT_KEY
static String	ZARRAY_FILE
static String	ZATTRS_FILE
static String	ZGROUP_FILE

Fields inherited from interface org.janelia.saalfeldlab.n5.N5Reader

NO_VERSION, VERSION_KEY

Constructor Summary

Constructors

Modifier	Constructor and Description
	ZarrKeyValueReader(boolean checkVersion, org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess, String basePath, com.google.gson.GsonBuilder gsonBuilder, boolean mapN5DatasetAttributes, boolean mergeAttributes, boolean cacheMeta) Opens an ZarrKeyValueReader at a given base path with a custom GsonBuilder to support custom attributes.
protected	ZarrKeyValueReader(boolean checkVersion, org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess, String basePath, com.google.gson.GsonBuilder gsonBuilder, boolean mapN5DatasetAttributes, boolean mergeAttributes, boolean cacheMeta, boolean checkRootExists)
	ZarrKeyValueReader(org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess, String basePath, com.google.gson.GsonBuilder gsonBuilder, boolean mapN5DatasetAttributes, boolean mergeAttributes, boolean cacheMeta) Opens an ZarrKeyValueReader at a given base path with a custom GsonBuilder to support custom attributes.

Method Summary

Modifier and Type	Method and Description
protected static com.google.gson.GsonBuilder	addTypeAdapters(com.google.gson.GsonBuilder gsonBuilder)
boolean	cacheMeta()
protected static com.google.gson.JsonElement	combine(com.google.gson.JsonElement base, com.google.gson.JsonElement add) Returns one JsonElement that combines two others.
protected static com.google.gson.JsonElement	combineAll(com.google.gson.JsonElement... elements) Returns one JsonElement that (attempts to) combine all passed json elements.
org.janelia.saalfeldlab.n5.DatasetAttributes	createDatasetAttributes(com.google.gson.JsonElement attributes)
boolean	datasetExists(String pathName)
boolean	exists(String pathName)
boolean	existsFromContainer(String normalPathName) Tests whether the key at normalPathName exists for the backend.
<T> T	getAttribute(String pathName, String key, Class<T> clazz)
<T> T	getAttribute(String pathName, String key, Type type)
com.google.gson.JsonElement	getAttributes(String path) Returns the attributes at the given path.
com.google.gson.JsonElement	getAttributesFromContainer(String normalResourceParent, String normalResourcePath)
String	getAttributesKey()
protected com.google.gson.JsonElement	getAttributesUnmapped(String path)
ZarrJsonCache	getCache()
ZarrDatasetAttributes	getConvertedDatasetAttributes(org.janelia.saalfeldlab.n5.DatasetAttributes attributes)
org.janelia.saalfeldlab.n5.DatasetAttributes	getDatasetAttributes(String pathName) Returns the ZarrDatasetAttributes located at the given path, if present.
com.google.gson.Gson	getGson()
protected com.google.gson.JsonElement	getJsonResource(String normalPath, String jsonName) Returns a JsonElement representing the contents of a JSON file located at the given path if present, and null if not.
org.janelia.saalfeldlab.n5.KeyValueAccess	getKeyValueAccess()
URI	getURI()
org.janelia.saalfeldlab.n5.N5Reader.Version	getVersion() Returns the version of this zarr container.
protected static org.janelia.saalfeldlab.n5.N5Reader.Version	getVersion(com.google.gson.JsonElement json)
protected com.google.gson.JsonElement	getZArray(String path) Returns a JsonElement representing the contents of .zarray located at the given path if present, and null if not.
protected ZArrayAttributes	getZArrayAttributes(com.google.gson.JsonElement attributes) Constructs ZArrayAttributes from a JsonElement.
ZArrayAttributes	getZArrayAttributes(String pathName) Returns the ZArrayAttributes located at the given path, if present.
com.google.gson.JsonElement	getZAttributes(String path) Returns a JsonElement representing the contents of .zattrs located at the given path if present, and null if not. public static JsonObject serialize(JsonSerializationContext context, final DefaultChunkKeyEncoding cke) { final JsonObject out = new JsonObject(); out.addProperty("name", "default"); final JsonObject config = new JsonObject(); config.addProperty("separator", cke.separator); out.add("configuration", config); return out; }
protected com.google.gson.JsonElement	getZGroup(String path) Returns a JsonElement representing the contents of .zgroup located at the given path if present, and null if not.
boolean	groupExists(String pathName)
boolean	isDatasetFromAttributes(String normalCacheKey, com.google.gson.JsonElement attributes)
boolean	isDatasetFromContainer(String normalPathName)
boolean	isGroupFromAttributes(String normalCacheKey, com.google.gson.JsonElement attributes)
boolean	isGroupFromContainer(String normalPath)
protected com.google.gson.JsonElement	n5ToZarrDatasetAttributes(com.google.gson.JsonElement elem)
ZarrJsonCache	newCache()
protected static com.google.gson.JsonElement	reverseAttrsWhenCOrder(com.google.gson.JsonElement elem)
String	toString()
protected com.google.gson.JsonElement	zarrToN5DatasetAttributes(com.google.gson.JsonElement elem)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

existsFromContainer, hasDatasetAttributes, list, listFromContainer, normalGetDatasetAttributes

Methods inherited from interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

absoluteAttributesPath, absoluteGroupPath, blockExists, readBlock, readChunk, readChunks

Methods inherited from interface org.janelia.saalfeldlab.n5.GsonN5Reader

listAttributes

Methods inherited from interface org.janelia.saalfeldlab.n5.N5Reader

close, deepList, deepList, deepList, deepList, deepList, deepListDatasets, deepListDatasets, deepListDatasets, deepListDatasets, deepListHelper, getGroupSeparator, groupPath, readSerializedBlock

Field Detail

VERSION_ZERO

public static final org.janelia.saalfeldlab.n5.N5Reader.Version VERSION_ZERO

VERSION

public static final org.janelia.saalfeldlab.n5.N5Reader.Version VERSION

ZARR_FORMAT_KEY

public static final String ZARR_FORMAT_KEY

See Also:

Constant Field Values

ZARRAY_FILE

public static final String ZARRAY_FILE

See Also:

Constant Field Values

ZATTRS_FILE

public static final String ZATTRS_FILE

See Also:

Constant Field Values

ZGROUP_FILE

public static final String ZGROUP_FILE

See Also:

Constant Field Values

keyValueAccess

protected final org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess

gson

protected final com.google.gson.Gson gson

cache

protected final ZarrJsonCache cache

cacheMeta

protected final boolean cacheMeta

uri

protected URI uri

datasetAttributesMap

protected final HashMap<org.janelia.saalfeldlab.n5.DatasetAttributes,ZarrDatasetAttributes> datasetAttributesMap

mapN5DatasetAttributes

protected final boolean mapN5DatasetAttributes

mergeAttributes

protected final boolean mergeAttributes

dimensionSeparator

protected String dimensionSeparator

Constructor Detail

ZarrKeyValueReader

public ZarrKeyValueReader(boolean checkVersion,
                          org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess,
                          String basePath,
                          com.google.gson.GsonBuilder gsonBuilder,
                          boolean mapN5DatasetAttributes,
                          boolean mergeAttributes,
                          boolean cacheMeta)
                   throws org.janelia.saalfeldlab.n5.N5Exception

Opens an ZarrKeyValueReader at a given base path with a custom GsonBuilder to support custom attributes.

Parameters:

checkVersion - perform version check

keyValueAccess -

basePath - N5 base path

gsonBuilder - the gson builder

mapN5DatasetAttributes - If true, getAttributes and variants of getAttribute methods will contain keys used by n5 datasets, and whose values are those for their corresponding zarr fields. For example, if true, the key "dimensions" (from n5) may be used to obtain the value of the key "shape" (from zarr).

mergeAttributes - If true, fields from .zgroup, .zarray, and .zattrs will be merged when calling getAttributes, and variants of getAttribute

cacheMeta - cache attributes and meta data Setting this to true avoids frequent reading and parsing of JSON encoded attributes and other meta data that requires accessing the store. This is most interesting for high latency backends. Changes of cached attributes and meta data by an independent writer will not be tracked.

Throws:

org.janelia.saalfeldlab.n5.N5Exception - if the base path cannot be read or does not exist, if the N5 version of the container is not compatible with this implementation.

ZarrKeyValueReader

protected ZarrKeyValueReader(boolean checkVersion,
                             org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess,
                             String basePath,
                             com.google.gson.GsonBuilder gsonBuilder,
                             boolean mapN5DatasetAttributes,
                             boolean mergeAttributes,
                             boolean cacheMeta,
                             boolean checkRootExists)

ZarrKeyValueReader

public ZarrKeyValueReader(org.janelia.saalfeldlab.n5.KeyValueAccess keyValueAccess,
                          String basePath,
                          com.google.gson.GsonBuilder gsonBuilder,
                          boolean mapN5DatasetAttributes,
                          boolean mergeAttributes,
                          boolean cacheMeta)
                   throws org.janelia.saalfeldlab.n5.N5Exception

Opens an ZarrKeyValueReader at a given base path with a custom GsonBuilder to support custom attributes.

Parameters:

keyValueAccess -

basePath - N5 base path

gsonBuilder -

cacheMeta - cache attributes and meta data Setting this to true avoids frequent reading and parsing of JSON encoded attributes and other meta data that requires accessing the store. This is most interesting for high latency backends. Changes of cached attributes and meta data by an independent writer will not be tracked.

Throws:

org.janelia.saalfeldlab.n5.N5Exception - if the base path cannot be read or does not exist, if the N5 version of the container is not compatible with this implementation.

Method Detail

getAttributesKey

public String getAttributesKey()

Specified by:

getAttributesKey in interface org.janelia.saalfeldlab.n5.GsonN5Reader

getGson

public com.google.gson.Gson getGson()

Specified by:

getGson in interface org.janelia.saalfeldlab.n5.GsonN5Reader

newCache

public ZarrJsonCache newCache()

Specified by:

newCache in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

cacheMeta

public boolean cacheMeta()

Specified by:

cacheMeta in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

getCache

public ZarrJsonCache getCache()

Specified by:

getCache in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

getKeyValueAccess

public org.janelia.saalfeldlab.n5.KeyValueAccess getKeyValueAccess()

Specified by:

getKeyValueAccess in interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

getURI

public URI getURI()

Specified by:

getURI in interface org.janelia.saalfeldlab.n5.N5Reader

getVersion

public org.janelia.saalfeldlab.n5.N5Reader.Version getVersion()
                                                       throws org.janelia.saalfeldlab.n5.N5Exception

Returns the version of this zarr container.

Specified by:

getVersion in interface org.janelia.saalfeldlab.n5.N5Reader

Returns:

the Version

Throws:

org.janelia.saalfeldlab.n5.N5Exception

exists

public boolean exists(String pathName)

Specified by:

exists in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

exists in interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

Specified by:

exists in interface org.janelia.saalfeldlab.n5.N5Reader

existsFromContainer

public boolean existsFromContainer(String normalPathName)

Tests whether the key at normalPathName exists for the backend. Such a key may exist for paths that are neither zarr groups nor zarr datasets.

Parameters:

normalPathName - normalized path name

Returns:

true if the key exists

groupExists

public boolean groupExists(String pathName)

Specified by:

groupExists in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

groupExists in interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

isGroupFromContainer

public boolean isGroupFromContainer(String normalPath)

Specified by:

isGroupFromContainer in interface org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

Specified by:

isGroupFromContainer in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

isGroupFromAttributes

public boolean isGroupFromAttributes(String normalCacheKey,
                                     com.google.gson.JsonElement attributes)

Specified by:

isGroupFromAttributes in interface org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

Specified by:

isGroupFromAttributes in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

datasetExists

public boolean datasetExists(String pathName)
                      throws org.janelia.saalfeldlab.n5.N5Exception.N5IOException

Specified by:

datasetExists in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

datasetExists in interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

Specified by:

datasetExists in interface org.janelia.saalfeldlab.n5.N5Reader

Throws:

org.janelia.saalfeldlab.n5.N5Exception.N5IOException

isDatasetFromContainer

public boolean isDatasetFromContainer(String normalPathName)
                               throws org.janelia.saalfeldlab.n5.N5Exception

Specified by:

isDatasetFromContainer in interface org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

Specified by:

isDatasetFromContainer in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Throws:

org.janelia.saalfeldlab.n5.N5Exception

isDatasetFromAttributes

public boolean isDatasetFromAttributes(String normalCacheKey,
                                       com.google.gson.JsonElement attributes)

Specified by:

isDatasetFromAttributes in interface org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

Specified by:

isDatasetFromAttributes in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

getDatasetAttributes

public org.janelia.saalfeldlab.n5.DatasetAttributes getDatasetAttributes(String pathName)
                                                                  throws org.janelia.saalfeldlab.n5.N5Exception

Returns the ZarrDatasetAttributes located at the given path, if present.

Specified by:

getDatasetAttributes in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

getDatasetAttributes in interface org.janelia.saalfeldlab.n5.GsonN5Reader

Specified by:

getDatasetAttributes in interface org.janelia.saalfeldlab.n5.N5Reader

Parameters:

pathName - the path relative to the container's root

Returns:

the zarr array attributes

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

getConvertedDatasetAttributes

public ZarrDatasetAttributes getConvertedDatasetAttributes(org.janelia.saalfeldlab.n5.DatasetAttributes attributes)

Specified by:

getConvertedDatasetAttributes in interface org.janelia.saalfeldlab.n5.N5Reader

getZArrayAttributes

public ZArrayAttributes getZArrayAttributes(String pathName)
                                     throws org.janelia.saalfeldlab.n5.N5Exception

Returns the ZArrayAttributes located at the given path, if present.

Parameters:

pathName - the path relative to the container's root

Returns:

the zarr array attributes

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

getZArrayAttributes

protected ZArrayAttributes getZArrayAttributes(com.google.gson.JsonElement attributes)

Constructs ZArrayAttributes from a JsonElement.

Parameters:

attributes - the json element

Returns:

the zarr array attributse

createDatasetAttributes

public org.janelia.saalfeldlab.n5.DatasetAttributes createDatasetAttributes(com.google.gson.JsonElement attributes)

Specified by:

createDatasetAttributes in interface org.janelia.saalfeldlab.n5.GsonN5Reader

getAttribute

public <T> T getAttribute(String pathName,
                          String key,
                          Class<T> clazz)
                   throws org.janelia.saalfeldlab.n5.N5Exception

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.GsonN5Reader

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.N5Reader

Throws:

org.janelia.saalfeldlab.n5.N5Exception

getAttribute

public <T> T getAttribute(String pathName,
                          String key,
                          Type type)
                   throws org.janelia.saalfeldlab.n5.N5Exception

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.GsonN5Reader

Specified by:

getAttribute in interface org.janelia.saalfeldlab.n5.N5Reader

Throws:

org.janelia.saalfeldlab.n5.N5Exception

getJsonResource

protected com.google.gson.JsonElement getJsonResource(String normalPath,
                                                      String jsonName)
                                               throws org.janelia.saalfeldlab.n5.N5Exception

Returns a JsonElement representing the contents of a JSON file located at the given path if present, and null if not. Also updates the cache.

Parameters:

normalPath - the normalized path

jsonName - the name of the JSON file

Returns:

the JSON element

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

reverseAttrsWhenCOrder

protected static com.google.gson.JsonElement reverseAttrsWhenCOrder(com.google.gson.JsonElement elem)

getZGroup

protected com.google.gson.JsonElement getZGroup(String path)
                                         throws org.janelia.saalfeldlab.n5.N5Exception

Returns a JsonElement representing the contents of .zgroup located at the given path if present, and null if not.

Parameters:

path - the path

Returns:

the JSON element

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

getZArray

protected com.google.gson.JsonElement getZArray(String path)
                                         throws org.janelia.saalfeldlab.n5.N5Exception

Returns a JsonElement representing the contents of .zarray located at the given path if present, and null if not.

Parameters:

path - the path

Returns:

the JSON element

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

zarrToN5DatasetAttributes

protected com.google.gson.JsonElement zarrToN5DatasetAttributes(com.google.gson.JsonElement elem)

n5ToZarrDatasetAttributes

protected com.google.gson.JsonElement n5ToZarrDatasetAttributes(com.google.gson.JsonElement elem)

getZAttributes

public com.google.gson.JsonElement getZAttributes(String path)
                                           throws org.janelia.saalfeldlab.n5.N5Exception

Returns a JsonElement representing the contents of .zattrs located at the given path if present, and null if not. public static JsonObject serialize(JsonSerializationContext context, final DefaultChunkKeyEncoding cke) { final JsonObject out = new JsonObject(); out.addProperty("name", "default"); final JsonObject config = new JsonObject(); config.addProperty("separator", cke.separator); out.add("configuration", config); return out; }

Parameters:

path - the path

Returns:

the JSON element

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

getAttributes

public com.google.gson.JsonElement getAttributes(String path)
                                          throws org.janelia.saalfeldlab.n5.N5Exception

Returns the attributes at the given path.

If this reader was constructed with mergeAttributes as true, this method will attempt to combine the contents of .zgroup, .zarray, and .zattrs, when present using ZarrUtils#combineAll(JsonElement...). Otherwise, this method will return only the contents of .zattrs, as getZAttributes.

Specified by:

getAttributes in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Specified by:

getAttributes in interface org.janelia.saalfeldlab.n5.GsonKeyValueN5Reader

Specified by:

getAttributes in interface org.janelia.saalfeldlab.n5.GsonN5Reader

Parameters:

path - the path

Returns:

the json element

Throws:

org.janelia.saalfeldlab.n5.N5Exception - the exception

getAttributesUnmapped

protected com.google.gson.JsonElement getAttributesUnmapped(String path)
                                                     throws org.janelia.saalfeldlab.n5.N5Exception

Throws:

org.janelia.saalfeldlab.n5.N5Exception

getAttributesFromContainer

public com.google.gson.JsonElement getAttributesFromContainer(String normalResourceParent,
                                                              String normalResourcePath)
                                                       throws org.janelia.saalfeldlab.n5.N5Exception

Specified by:

getAttributesFromContainer in interface org.janelia.saalfeldlab.n5.cache.N5JsonCacheableContainer

Specified by:

getAttributesFromContainer in interface org.janelia.saalfeldlab.n5.CachedGsonKeyValueN5Reader

Throws:

org.janelia.saalfeldlab.n5.N5Exception

toString

public String toString()

Overrides:

toString in class Object

getVersion

protected static org.janelia.saalfeldlab.n5.N5Reader.Version getVersion(com.google.gson.JsonElement json)

combineAll

protected static com.google.gson.JsonElement combineAll(com.google.gson.JsonElement... elements)

Returns one JsonElement that (attempts to) combine all passed json elements. Reduces the list by repeatedly calling the two-argument combine method.

Parameters:

elements - an array of json elements

Returns:

a new, combined element

combine

protected static com.google.gson.JsonElement combine(com.google.gson.JsonElement base,
                                                     com.google.gson.JsonElement add)

Returns one JsonElement that combines two others. The returned instance is a deep copy, and the arguments are not modified. A copy of base element is returned if the two arguments can not be combined. The two arguments may be combined if they are both JsonObjects or both JsonArrays.

If both arguments are JsonObjects, every key-value pair in the add argument is added to the (copy of) the base argument, overwriting any duplicate keys. If both arguments are JsonArrays, the add argument is concatenated to the (copy of) the base argument.

Parameters:

base - the base element

add - the element to add

Returns:

the new element

addTypeAdapters

protected static com.google.gson.GsonBuilder addTypeAdapters(com.google.gson.GsonBuilder gsonBuilder)