SioxSegmentator (Fiji Javadocs 0-SNAPSHOT API)

java.lang.Object
- org.siox.SioxSegmentator

```
public class SioxSegmentator
extends Object
```
Image segmentator based on SIOX: Simple Interactive Object Extraction.
To segmentate an image one has to perform the following steps.
1. Construct an instance of SioxSegmentator.
2. Create a confidence matrix, where each entry marks its corresponding image pixel to belong to the foreground, to the background, or being of unknown type.
3. Call segmentate on the image with the confidence matrix. This stores the result as new foreground confidence into the confidence matrix, with each entry being either zero (CERTAIN_BACKGROUND_CONFIDENCE) or one (CERTAIN_FOREGROUND_CONFIDENCE).
4. Optionally call subpixelRefine to areas where pixels contain both foreground and background (e.g. object borders or highly detailed features like flowing hairs). The pixel are then assigned confidence values between zero and one to give them a measure of "foregroundness". This step may be repeated as often as needed.
For algorithm documentation refer to G. Friedland, K. Jantz, L. Knipping, R. Rojas: Image Segmentation by Uniform Color Clustering -- Approach and Benchmark Results, Technical Report B-05-07, Department of Computer Science, Freie Universitaet Berlin, June 2005.

See http://www.siox.org for more information.

Algorithm idea by Gerald Friedland.
Version:

1.13

Author:

Gerald Friedland, Kristian Jantz, Lars Knipping

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`ADD_EDGE` Add mode for the subpixel refinement.
`static float`	`BACKGROUND_CONFIDENCE` Confidence for a region likely being background.
`static float`	`CERTAIN_BACKGROUND_CONFIDENCE` Confidence corresponding to a certain background region (equals zero).
`static float`	`CERTAIN_FOREGROUND_CONFIDENCE` Confidence corresponding to a certain foreground region (equals one).
`static float`	`FOREGROUND_CONFIDENCE` Confidence for a region likely being foreground.
`static String`	`SUB_EDGE` Subtract mode for the subpixel refinement.
`static float`	`UNKNOWN_REGION_CONFIDENCE` Confidence for foreground or background type being equally likely.

Constructor Summary

Constructors
Constructor and Description
`SioxSegmentator(int w, int h, float[] limits)` Constructs a SioxSegmentator Object to be used for image segmentation.
`SioxSegmentator(int w, int h, float[] limits, float[][] bgSignature, float[][] fgSignature)` Constructs a SioxSegmentator Object to be used for image segmentation.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`boolean`	`applyPrecomputedSignatures(int[] image, float[] cm, int smoothness, double sizeFactorToKeep)` Segments the given image with the previously calculated color signatures.
`float[][]`	`getBgSignature()` Get background signature
`float[][]`	`getFgSignature()` Get foreground signature
`protected void`	`keepOnlyLargeComponents(float[] cm, float threshold, double sizeFactorToKeep)` Clears given confidence matrix except entries for the largest connected component and every component with `size*sizeFactorToKeep >= sizeOfLargestComponent`.
`protected void`	`keepOnlyLargeComponents(float[] cm, float threshold, double sizeFactorToKeep, IntArrayList predefinedFgPixels)` Clears given confidence matrix except entries for the user-defined connected component and every component with `size*sizeFactorToKeep >= sizeOfLargestComponent`.
`boolean`	`segmentate(int[] image, float[] cm, int smoothness, double sizeFactorToKeep)` Segments the given image with information from the confidence matrix.
`boolean`	`segmentatevideo_firstframe(int[] image, float[] cm, double sizeFactorToKeep)` Segments the first frame of a scene in a video.
`boolean`	`segmentatevideo_nextframe(int[] image, float[] cm, double sizeFactorToKeep)` Segment the further frames of a scene in a video.
`void`	`subpixelRefine(Area area, String brushmode, float threshold, float[] cf)` Refines the classification stored in the confidence matrix by modifying the confidences for regions which have characteristics to both foreground and background if they fall into the specified area.
`void`	`subpixelRefine(int x, int y, String brushmode, float threshold, float[] cf, int brushsize)` Refines the classification stored in the confidence matrix by modifying the confidences for regions which have characteristics to both foreground and background if they fall into the specified square.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - ADD_EDGE
```
public static final String ADD_EDGE
```
    Add mode for the subpixel refinement.
    
    See Also:
    
    Constant Field Values
  - SUB_EDGE
```
public static final String SUB_EDGE
```
    Subtract mode for the subpixel refinement.
    
    See Also:
    
    Constant Field Values
  - CERTAIN_FOREGROUND_CONFIDENCE
```
public static final float CERTAIN_FOREGROUND_CONFIDENCE
```
    Confidence corresponding to a certain foreground region (equals one).
    
    See Also:
    
    Constant Field Values
  - FOREGROUND_CONFIDENCE
```
public static final float FOREGROUND_CONFIDENCE
```
    Confidence for a region likely being foreground.
    
    See Also:
    
    Constant Field Values
  - UNKNOWN_REGION_CONFIDENCE
```
public static final float UNKNOWN_REGION_CONFIDENCE
```
    Confidence for foreground or background type being equally likely.
    
    See Also:
    
    Constant Field Values
  - BACKGROUND_CONFIDENCE
```
public static final float BACKGROUND_CONFIDENCE
```
    Confidence for a region likely being background.
    
    See Also:
    
    Constant Field Values
  - CERTAIN_BACKGROUND_CONFIDENCE
```
public static final float CERTAIN_BACKGROUND_CONFIDENCE
```
    Confidence corresponding to a certain background region (equals zero).
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - SioxSegmentator
```
public SioxSegmentator(int w,
                       int h,
                       float[] limits)
```
    Constructs a SioxSegmentator Object to be used for image segmentation.
    
    Parameters:
    
    w - X resolution of the image to be segmented.
    
    h - Y resolution of the image to be segmented.
    
    limits - Size of the cluster on LAB axises. If null, the default value {0.64f,1.28f,2.56f} is used.
  - SioxSegmentator
```
public SioxSegmentator(int w,
                       int h,
                       float[] limits,
                       float[][] bgSignature,
                       float[][] fgSignature)
```
    Constructs a SioxSegmentator Object to be used for image segmentation.
    
    Parameters:
    
    w - X resolution of the image to be segmented.
    
    h - Y resolution of the image to be segmented.
    
    limits - Size of the cluster on LAB axises. If null, the default value {0.64f,1.28f,2.56f} is used.
    
    bgSignature - background color signature
    
    fgSignature - foreground color signature
- Method Detail
  - segmentate
```
public boolean segmentate(int[] image,
                          float[] cm,
                          int smoothness,
                          double sizeFactorToKeep)
```
    Segments the given image with information from the confidence matrix. For faster segmentation in videos, use the methods segmentatevideo_firstframe() and segmentatevideo_nextframe().
    The confidence entries of BACKGROUND_CONFIDENCE or less are mark known background pixel for the segmentation, those of at least FOREGROUND_CONFIDENCE mark known foreground pixel for the segmentation. Any other entry is treated as region of unknown affiliation.
    As result, each pixel is classified either as foreground or background, stored back into its cm entry as confidence CERTAIN_FOREGROUND_CONFIDENCE or CERTAIN_BACKGROUND_CONFIDENCE.
    
    Parameters:
    
    image - Pixel data of the image to be segmented. Every integer represents one ARGB-value.
    
    cm - Confidence matrix specifying the probability of an image belonging to the foreground before and after the segmentation.
    
    smoothness - Number of smoothing steps in the post processing.
    
    sizeFactorToKeep - Segmentation retains the largest connected foreground component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
    
    Returns:
    
    true if the segmentation algorithm succeeded, false if segmentation is impossible
    
    Throws:
    
    IllegalStateException - if the confidence matrix defines no image foreground.
  - applyPrecomputedSignatures
```
public boolean applyPrecomputedSignatures(int[] image,
                                          float[] cm,
                                          int smoothness,
                                          double sizeFactorToKeep)
```
    Segments the given image with the previously calculated color signatures.
    
    Parameters:
    
    image - Pixel data of the image to be segmented. Every integer represents one ARGB-value.
    
    cm - Confidence matrix specifying the probability of an image belonging to the foreground before and after the segmentation.
    
    smoothness - Number of smoothing steps in the post processing.
    
    sizeFactorToKeep - Segmentation retains the largest connected foreground component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
    
    Returns:
    
    true if the segmentation algorithm succeeded, false if segmentation is impossible
    
    Throws:
    
    IllegalStateException - if the confidence matrix defines no image foreground.
  - segmentatevideo_firstframe
```
public boolean segmentatevideo_firstframe(int[] image,
                                          float[] cm,
                                          double sizeFactorToKeep)
```
    Segments the first frame of a scene in a video. Only a reduced postprocessing is applied. Still image segmentation should be performed with segmentate(). This methods also rebuilds the color signatures.
    The confidence entries of BACKGROUND_CONFIDENCE or less are mark known background pixel for the segmentation, those of at least FOREGROUND_CONFIDENCE mark known foreground pixel for the segmentation. Any other entry is treated as region of unknown affiliation.
    As result, each pixel is classified either as foreground or background, stored back into its cm entry as confidence CERTAIN_FOREGROUND_CONFIDENCE or CERTAIN_BACKGROUND_CONFIDENCE.
    
    Parameters:
    
    image - Pixel data of the image to be segmented. Every integer represents one ARGB-value.
    
    cm - Confidence matrix specifying the probability of an image belonging to the foreground before and after the segmentation.
    
    sizeFactorToKeep - Segmentation retains the largest connected foreground component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
    
    Returns:
    
    true if the segmentation algorithm succeeded, false if segmentation is impossible
    
    Throws:
    
    IllegalStateException - if the confidence matrix defines no image foreground.
  - segmentatevideo_nextframe
```
public boolean segmentatevideo_nextframe(int[] image,
                                         float[] cm,
                                         double sizeFactorToKeep)
```
    Segment the further frames of a scene in a video. Only a reduced postprocessing is applied. The methods DOES NOT rebuild the color signatures. This method can only be used after a call to segmentate() or segmentatevideo_firstframe().
    The confidence entries of BACKGROUND_CONFIDENCE or less are mark known background pixel for the segmentation, those of at least FOREGROUND_CONFIDENCE mark known foreground pixel for the segmentation. Any other entry is treated as region of unknown affiliation. This method does not require any known confidences in a frame.
    As result, each pixel is classified either as foreground or background, stored back into its cm entry as confidence CERTAIN_FOREGROUND_CONFIDENCE or CERTAIN_BACKGROUND_CONFIDENCE.
    
    Parameters:
    
    image - Pixel data of the image to be segmented. Every integer represents one ARGB-value.
    
    cm - Confidence matrix specifying the probability of an image belonging to the foreground before and after the segmentation.
    
    sizeFactorToKeep - Segmentation retains the largest connected foreground component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
    
    Returns:
    
    true if the segmentation algorithm succeeded, false if segmentation is impossible
    
    Throws:
    
    IllegalStateException - if no color signature has been defined before calling this method.
  - keepOnlyLargeComponents
```
protected void keepOnlyLargeComponents(float[] cm,
                                       float threshold,
                                       double sizeFactorToKeep)
```
    Clears given confidence matrix except entries for the largest connected component and every component with size*sizeFactorToKeep >= sizeOfLargestComponent.
    
    Parameters:
    
    cm - Confidence matrix to be analyzed
    
    threshold - Pixel visibility threshold. Exactly those cm entries larger than threshold are considered to be a "visible" foreground pixel.
    
    sizeFactorToKeep - This method keeps the largest connected component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
  - keepOnlyLargeComponents
```
protected void keepOnlyLargeComponents(float[] cm,
                                       float threshold,
                                       double sizeFactorToKeep,
                                       IntArrayList predefinedFgPixels)
```
    Clears given confidence matrix except entries for the user-defined connected component and every component with size*sizeFactorToKeep >= sizeOfLargestComponent.
    
    Parameters:
    
    cm - Confidence matrix to be analyzed
    
    threshold - Pixel visibility threshold. Exactly those cm entries larger than threshold are considered to be a "visible" foreground pixel.
    
    sizeFactorToKeep - This method keeps the largest connected component plus any component with size at least sizeOfLargestComponent/sizeFactorToKeep.
    
    predefinedFgPixels - list of pixels defined for the user as foreground.
  - subpixelRefine
```
public void subpixelRefine(int x,
                           int y,
                           String brushmode,
                           float threshold,
                           float[] cf,
                           int brushsize)
```
    Refines the classification stored in the confidence matrix by modifying the confidences for regions which have characteristics to both foreground and background if they fall into the specified square.
    The can be used in displaying the image by assigning the alpha values of the pixels according to the confidence entries.
    In the algorithm descriptions and examples GUIs this step is referred to as Detail Refinement (Brush).
    
    Parameters:
    
    x - Horizontal coordinate of the squares center.
    
    y - Vertical coordinate of the squares center.
    
    brushmode - Mode of the refinement applied, ADD_EDGE or SUB_EDGE. Add mode only modifies pixels formerly classified as background, sub mode only those formerly classified as foreground.
    
    threshold - Threshold for the add and sub refinement, deciding at the confidence level to stop at.
    
    cf - The confidence matrix to modify, generated by segmentate, possibly already refined by previous calls to subpixelRefine.
    
    brushsize - Halfed diameter of the square shaped brush.
    
    Throws:
    
    IllegalStateException - if there is no segmentation calculated yet to be refined
    
    See Also:
    
    segmentate(int[], float[], int, double)
  - subpixelRefine
```
public void subpixelRefine(Area area,
                           String brushmode,
                           float threshold,
                           float[] cf)
```
    Refines the classification stored in the confidence matrix by modifying the confidences for regions which have characteristics to both foreground and background if they fall into the specified area.
    The can be used in displaying the image by assigning the alpha values of the pixels according to the confidence entries.
    In the algorithm descriptions and examples GUIs this step is referrered to as Detail Refinement (Brush).
    
    Parameters:
    
    area - Area in which the reworking of the segmentation is applied to.
    
    brushmode - Mode of the refinement applied, ADD_EDGE or SUB_EDGE. Add mode only modifies pixels formerly classified as background, sub mode only those formerly classified as foreground.
    
    threshold - Threshold for the add and sub refinement, deciding at the confidence level to stop at.
    
    cf - The confidence matrix to modify, generated by segmentate, possibly already refined by privious calls to subpixelRefine.
    
    Throws:
    
    IllegalStateException - if there is no segmentation calculated yet to be refined
    
    See Also:
    
    segmentate(int[], float[], int, double)
  - getBgSignature
```
public float[][] getBgSignature()
```
    Get background signature
  - getFgSignature
```
public float[][] getFgSignature()
```
    Get foreground signature

Class SioxSegmentator

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

ADD_EDGE

SUB_EDGE

CERTAIN_FOREGROUND_CONFIDENCE

FOREGROUND_CONFIDENCE

UNKNOWN_REGION_CONFIDENCE

BACKGROUND_CONFIDENCE

CERTAIN_BACKGROUND_CONFIDENCE

Constructor Detail

SioxSegmentator

SioxSegmentator

Method Detail

segmentate

applyPrecomputedSignatures

segmentatevideo_firstframe

segmentatevideo_nextframe

keepOnlyLargeComponents

keepOnlyLargeComponents

subpixelRefine

subpixelRefine

getBgSignature

getFgSignature