org.janelia.saalfeldlab.n5.universe

Class N5DatasetDiscoverer

java.lang.Object

org.janelia.saalfeldlab.n5.universe.N5DatasetDiscoverer

public class N5DatasetDiscoverer
extends Object

This class aids in detecting and parsing datasets in an N5 container.

An N5DatasetDiscoverer specifies the types of N5MetadataParsers to attempt, and an ExecutorService that enables parsing in parallel. The parsers are passed to the constructor in a list. Group parsers are called after all others are called, and should be used when a parsers result depends on its children.

The discoverAndParseRecursive method returns a N5TreeNode containing all child nodes, each of which contains parsed metadata. For each group/dataset, the parsers will be called in order, and will return the first non-empty result. As such parsers should be ordered from most- to least-strict.

Author:

Caleb Hulbert, John Bogovic

Field Summary

Fields

Modifier and Type	Field and Description
static N5MetadataParser<?>[]	DEFAULT_PARSERS

Constructor Summary

Constructors

Constructor and Description
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor)
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers)
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor, Optional<Comparator<? super String>> comparator, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers)
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor, Optional<Comparator<? super String>> comparator, Predicate<N5TreeNode> filter, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers) Creates an N5 discoverer.
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor, Optional<Comparator<? super String>> comparator, Predicate<N5TreeNode> filter, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers, List<N5MetadataParser<?>> shallowGroupParsers) Creates an N5 discoverer.
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, ExecutorService executor, Predicate<N5TreeNode> filter, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers)
N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers) Creates an N5 discoverer.

Method Summary

Modifier and Type	Method and Description
static List<N5MetadataParser<?>>	defaultGroupParsers(org.janelia.saalfeldlab.n5.N5Reader n5) Parser configuration depends on the n5 reader.
static List<N5MetadataParser<?>>	defaultShallowGroupParsers(org.janelia.saalfeldlab.n5.N5Reader n5) Parser configuration depends on the n5 reader.
static N5TreeNode	discover(org.janelia.saalfeldlab.n5.N5Reader n5)
static N5TreeNode	discover(org.janelia.saalfeldlab.n5.N5Reader n5, List<N5MetadataParser<?>> parsers)
static N5TreeNode	discover(org.janelia.saalfeldlab.n5.N5Reader n5, List<N5MetadataParser<?>> parsers, List<N5MetadataParser<?>> groupParsers)
static N5TreeNode	discover(org.janelia.saalfeldlab.n5.N5Reader n5, String basePath)
static N5TreeNode	discover(org.janelia.saalfeldlab.n5.N5Reader n5, String basePath, List<N5MetadataParser<?>> parsers, List<N5MetadataParser<?>> groupParsers)
N5TreeNode	discoverAndParseRecursive(N5TreeNode root)
N5TreeNode	discoverAndParseRecursive(N5TreeNode root, Consumer<N5TreeNode> callback)
N5TreeNode	discoverAndParseRecursive(N5TreeNode root, Predicate<String> filter, Consumer<N5TreeNode> callback)
N5TreeNode	discoverAndParseRecursive(String base) Recursively discovers and parses metadata for datasets that are children of the given base path using N5Reader.deepList(java.lang.String, java.util.function.Predicate<java.lang.String>).
N5TreeNode	discoverAndParseRecursive(String base, Consumer<N5TreeNode> callback)
static N5TreeNode	discoverShallow(org.janelia.saalfeldlab.n5.N5Reader n5) Discovers metadata present at the root without listing
static N5TreeNode	discoverShallow(org.janelia.saalfeldlab.n5.N5Reader n5, String dataset) Discovers metadata present at the provided dataset without listing.
N5TreeNode	discoverShallow(N5TreeNode base)
N5TreeNode	discoverShallow(N5TreeNode base, Consumer<N5TreeNode> callback)
N5TreeNode	discoverShallow(String base)
N5TreeNode	discoverShallow(String base, Consumer<N5TreeNode> callback)
void	filterRecursive(N5TreeNode node)
static List<N5MetadataParser<?>>	fromParsers(N5MetadataParser<?>[] parsers)
N5TreeNode	parse(N5TreeNode node)
N5TreeNode	parse(String dataset)
static void	parseMetadata(org.janelia.saalfeldlab.n5.N5Reader n5, N5TreeNode node, List<N5MetadataParser<?>> metadataParsers)
static void	parseMetadata(org.janelia.saalfeldlab.n5.N5Reader n5, N5TreeNode node, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers) Parses metadata for a node using the given parsers, stopping after the first success.
void	parseMetadataRecursive(N5TreeNode rootNode) Parses metadata for the given node and all children in parallel using this object's executor.
void	parseMetadataRecursive(N5TreeNode rootNode, Consumer<N5TreeNode> callback) Parses metadata for the given node and all children in parallel using this object's executor.
void	parseMetadataRecursive(N5TreeNode rootNode, Consumer<N5TreeNode> callback, boolean skipParsingIfPresent) Parses metadata for the given node and all children in parallel using this object's executor.
static void	parseMetadataShallow(org.janelia.saalfeldlab.n5.N5Reader n5, N5TreeNode node, List<N5MetadataParser<?>> metadataParsers, List<N5MetadataParser<?>> groupParsers) Parses metadata for a node using the given parsers, stopping after the first success.
void	sort(N5TreeNode node)
static void	sort(N5TreeNode node, Comparator<? super String> comparator, Consumer<N5TreeNode> callback)
void	sort(N5TreeNode node, Consumer<N5TreeNode> callback)
void	sortAndTrimRecursive(N5TreeNode node)
void	sortAndTrimRecursive(N5TreeNode node, Consumer<N5TreeNode> callback)
static boolean	trim(N5TreeNode node)
static boolean	trim(N5TreeNode node, Consumer<N5TreeNode> callback) Removes branches of the N5 container tree that do not contain any nodes that can be opened (nodes with metadata).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_PARSERS

public static final N5MetadataParser<?>[] DEFAULT_PARSERS

Constructor Detail

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor)

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers)

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers)

Creates an N5 discoverer.

Parameters:

n5 - n5 reader

metadataParsers - metadata parsers

groupParsers - group parsers

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor,
                           Predicate<N5TreeNode> filter,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers)

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor,
                           Optional<Comparator<? super String>> comparator,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers)

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor,
                           Optional<Comparator<? super String>> comparator,
                           Predicate<N5TreeNode> filter,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers)

Creates an N5 discoverer.

If the optional parameter comparator is specified, the groups and datasets will be listed in the order determined by this comparator.

Parameters:

n5 - the n5 reader

executor - the executor

comparator - optional string comparator

filter - the dataset filter

metadataParsers - metadata parsers

groupParsers - group parsers

N5DatasetDiscoverer

public N5DatasetDiscoverer(org.janelia.saalfeldlab.n5.N5Reader n5,
                           ExecutorService executor,
                           Optional<Comparator<? super String>> comparator,
                           Predicate<N5TreeNode> filter,
                           List<N5MetadataParser<?>> metadataParsers,
                           List<N5MetadataParser<?>> groupParsers,
                           List<N5MetadataParser<?>> shallowGroupParsers)

Creates an N5 discoverer.

If the optional parameter comparator is specified, the groups and datasets will be listed in the order determined by this comparator.

Parameters:

n5 - the n5 reader

executor - the executor

comparator - optional string comparator

filter - the dataset filter

metadataParsers - metadata parsers

groupParsers - group parsers

shallowGroupParsers - shallow group parsers

Method Detail

parseMetadata

public static void parseMetadata(org.janelia.saalfeldlab.n5.N5Reader n5,
                                 N5TreeNode node,
                                 List<N5MetadataParser<?>> metadataParsers)
                          throws IOException

Throws:

IOException

parseMetadata

public static void parseMetadata(org.janelia.saalfeldlab.n5.N5Reader n5,
                                 N5TreeNode node,
                                 List<N5MetadataParser<?>> metadataParsers,
                                 List<N5MetadataParser<?>> groupParsers)
                          throws IOException

Parses metadata for a node using the given parsers, stopping after the first success.

Parameters:

n5 - the N5Reader

node - the tree node

metadataParsers - list of metadata parsers

groupParsers - list of group parsers

Throws:

IOException - the exception

parseMetadataShallow

public static void parseMetadataShallow(org.janelia.saalfeldlab.n5.N5Reader n5,
                                        N5TreeNode node,
                                        List<N5MetadataParser<?>> metadataParsers,
                                        List<N5MetadataParser<?>> groupParsers)

Parses metadata for a node using the given parsers, stopping after the first success.

The group parsers given are assumed to be shallow.

Parameters:

n5 - the N5Reader

node - the tree node

metadataParsers - list of metadata parsers

groupParsers - list of shallow group parsers

trim

public static boolean trim(N5TreeNode node)

trim

public static boolean trim(N5TreeNode node,
                           Consumer<N5TreeNode> callback)

Removes branches of the N5 container tree that do not contain any nodes that can be opened (nodes with metadata).

Parameters:

node - the node

callback - the callback function

Returns:

true if the branch contains a node that can be opened, false otherwise

sort

public static void sort(N5TreeNode node,
                        Comparator<? super String> comparator,
                        Consumer<N5TreeNode> callback)

sort

public void sort(N5TreeNode node,
                 Consumer<N5TreeNode> callback)

sort

public void sort(N5TreeNode node)

discoverShallow

public N5TreeNode discoverShallow(String base)

discoverShallow

public N5TreeNode discoverShallow(String base,
                                  Consumer<N5TreeNode> callback)

discoverShallow

public N5TreeNode discoverShallow(N5TreeNode base)

discoverShallow

public N5TreeNode discoverShallow(N5TreeNode base,
                                  Consumer<N5TreeNode> callback)

discoverAndParseRecursive

public N5TreeNode discoverAndParseRecursive(String base)
                                     throws IOException

Recursively discovers and parses metadata for datasets that are children of the given base path using N5Reader.deepList(java.lang.String, java.util.function.Predicate<java.lang.String>). Returns an N5TreeNode that can be displayed as a JTree.

Parameters:

base - the base path

Returns:

the n5 tree node

Throws:

IOException - the io exception

discoverAndParseRecursive

public N5TreeNode discoverAndParseRecursive(String base,
                                            Consumer<N5TreeNode> callback)
                                     throws IOException

Throws:

IOException

discoverAndParseRecursive

public N5TreeNode discoverAndParseRecursive(N5TreeNode root)
                                     throws IOException

Throws:

IOException

discoverAndParseRecursive

public N5TreeNode discoverAndParseRecursive(N5TreeNode root,
                                            Consumer<N5TreeNode> callback)
                                     throws IOException

Throws:

IOException

discoverAndParseRecursive

public N5TreeNode discoverAndParseRecursive(N5TreeNode root,
                                            Predicate<String> filter,
                                            Consumer<N5TreeNode> callback)
                                     throws IOException

Throws:

IOException

parse

public N5TreeNode parse(String dataset)

parse

public N5TreeNode parse(N5TreeNode node)

sortAndTrimRecursive

public void sortAndTrimRecursive(N5TreeNode node)

sortAndTrimRecursive

public void sortAndTrimRecursive(N5TreeNode node,
                                 Consumer<N5TreeNode> callback)

filterRecursive

public void filterRecursive(N5TreeNode node)

parseMetadataRecursive

public void parseMetadataRecursive(N5TreeNode rootNode)

Parses metadata for the given node and all children in parallel using this object's executor.

Parameters:

rootNode - the root node

parseMetadataRecursive

public void parseMetadataRecursive(N5TreeNode rootNode,
                                   Consumer<N5TreeNode> callback)

Parses metadata for the given node and all children in parallel using this object's executor. The given function is called for every node after parsing is completed, successful or not.

Parameters:

rootNode - the root node

callback - the callback function

parseMetadataRecursive

public void parseMetadataRecursive(N5TreeNode rootNode,
                                   Consumer<N5TreeNode> callback,
                                   boolean skipParsingIfPresent)

Parses metadata for the given node and all children in parallel using this object's executor. The given function is called for every node after parsing is completed, successful or not.

The skipParsingIfPresent argument allows parsing to be skipped if metadata for this node are already present. This should generally be true.

Parameters:

rootNode - the root node

callback - the callback function

skipParsingIfPresent - skip parsing the root node if metadata are already present

fromParsers

public static final List<N5MetadataParser<?>> fromParsers(N5MetadataParser<?>[] parsers)

discover

public static N5TreeNode discover(org.janelia.saalfeldlab.n5.N5Reader n5,
                                  String basePath,
                                  List<N5MetadataParser<?>> parsers,
                                  List<N5MetadataParser<?>> groupParsers)

discover

public static N5TreeNode discover(org.janelia.saalfeldlab.n5.N5Reader n5,
                                  List<N5MetadataParser<?>> parsers,
                                  List<N5MetadataParser<?>> groupParsers)

discover

public static N5TreeNode discover(org.janelia.saalfeldlab.n5.N5Reader n5,
                                  List<N5MetadataParser<?>> parsers)

discover

public static N5TreeNode discover(org.janelia.saalfeldlab.n5.N5Reader n5,
                                  String basePath)

discover

public static N5TreeNode discover(org.janelia.saalfeldlab.n5.N5Reader n5)

discoverShallow

public static N5TreeNode discoverShallow(org.janelia.saalfeldlab.n5.N5Reader n5,
                                         String dataset)

Discovers metadata present at the provided dataset without listing.

Parameters:

n5 - the reader

dataset - the dataset

Returns:

a node with shallow-parsed metadata

discoverShallow

public static N5TreeNode discoverShallow(org.janelia.saalfeldlab.n5.N5Reader n5)

Discovers metadata present at the root without listing

Parameters:

n5 - the reader

Returns:

the root tree node

defaultGroupParsers

public static List<N5MetadataParser<?>> defaultGroupParsers(org.janelia.saalfeldlab.n5.N5Reader n5)

Parser configuration depends on the n5 reader. OME-Zarr parsers reverse axis parmeters for zarr, but not for n5.

Parameters:

n5 - the n5 reader

Returns:

the group parsers

defaultShallowGroupParsers

public static List<N5MetadataParser<?>> defaultShallowGroupParsers(org.janelia.saalfeldlab.n5.N5Reader n5)

Parser configuration depends on the n5 reader. OME-Zarr parsers reverse axis parmeters for zarr, but not for n5.

Parameters:

n5 - the n5 reader

Returns:

the shallow group parsers