weka.classifiers.evaluation.Evaluation

All Implemented Interfaces:: Serializable, RevisionHandler, Summarizable

Direct Known Subclasses:: AggregateableEvaluation

public class Evaluation extends Object implements Summarizable, RevisionHandler, Serializable

Class for evaluating machine learning models.

-------------------------------------------------------------------

General options when evaluating a learning scheme from the command-line:

-t filename
Name of the file with the training data. (required)

-T filename
Name of the file with the test data. If missing a cross-validation is performed.

-c index
Index of the class attribute (1, 2, ...; default: last).

-x number
The number of folds for the cross-validation (default: 10).

-no-cv
No cross validation. If no test file is provided, no evaluation is done.

-split-percentage percentage
Sets the percentage for the train/test set split, e.g., 66.

-preserve-order
Preserves the order in the percentage split instead of randomizing the data first with the seed value ('-s').

-s seed
Random number seed for the cross-validation and percentage split (default: 1).

-m filename
The name of a file containing a cost matrix.

-disable list
A comma separated list of metric names not to include in the output.

-l filename
Loads classifier from the given file. In case the filename ends with ".xml", a PMML file is loaded or, if that fails, options are loaded from XML.

-d filename
Saves classifier built from the training data into the given file. In case the filename ends with ".xml" the options are saved XML, not the model.

-v
Outputs no statistics for the training data.

-o
Outputs statistics only, not the classifier.

-output-models-for-training-splits
Output models for training splits if cross-validation or percentage-split evaluation is used.

-do-not-output-per-class-statistics
Do not output statistics per class.

-k
Outputs information-theoretic statistics.

-classifications "weka.classifiers.evaluation.output.prediction.AbstractOutput + options"
Uses the specified class for generating the classification output. E.g.: weka.classifiers.evaluation.output.prediction.PlainText or : weka.classifiers.evaluation.output.prediction.CSV -p range
Outputs predictions for test instances (or the train instances if no test instances provided and -no-cv is used), along with the attributes in the specified range (and nothing else). Use '-p 0' if no attributes are desired.

Deprecated: use "-classifications ..." instead.

-distribution
Outputs the distribution instead of only the prediction in conjunction with the '-p' option (only nominal classes).

Deprecated: use "-classifications ..." instead.

-no-predictions
Turns off the collection of predictions in order to conserve memory.

-r
Outputs cumulative margin distribution (and nothing else).

-g
Only for classifiers that implement "Graphable." Outputs the graph representation of the classifier (and nothing else).

-xml filename | xml-string
Retrieves the options from the XML-data instead of the command line.

-threshold-file file
The file to save the threshold data to. The format is determined by the extensions, e.g., '.arff' for ARFF format or '.csv' for CSV.

-threshold-label label
The class label to determine the threshold data for (default is the first label)

-------------------------------------------------------------------

Example usage as the main of a classifier (called FunkyClassifier):

 public static void main(String [] args) {
   runClassifier(new FunkyClassifier(), args);
 }

------------------------------------------------------------------

Example usage from within an application:

 Instances trainInstances = ... instances got from somewhere
 Instances testInstances = ... instances got from somewhere
 Classifier scheme = ... scheme got from somewhere

 Evaluation evaluation = new Evaluation(trainInstances);
 evaluation.evaluateModel(scheme, testInstances);
 System.out.println(evaluation.toSummaryString());

Version:

$Revision: 15278 $

Author:

Eibe Frank (eibe@cs.waikato.ac.nz), Len Trigg (trigg@cs.waikato.ac.nz)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type

Field

Description

static final String[]

BUILT_IN_EVAL_METRICS
Constructor Summary

Constructors

Constructor

Description

Evaluation(Instances data)

Initializes all the counters for the evaluation.

Evaluation(Instances data, CostMatrix costMatrix)

Initializes all the counters for the evaluation and also takes a cost matrix as parameter.
Method Summary

Modifier and Type

Method

Description

double

areaUnderPRC(int classIndex)

Returns the area under precision-recall curve (AUPRC) for those predictions that have been collected in the evaluateClassifier(Classifier, Instances) method.

double

areaUnderROC(int classIndex)

Returns the area under ROC for those predictions that have been collected in the evaluateClassifier(Classifier, Instances) method.

final double

avgCost()

Gets the average cost, that is, total cost of misclassifications (incorrect plus unclassified) over the total number of instances.

double[][]

confusionMatrix()

Returns a copy of the confusion matrix.

final double

correct()

Gets the number of instances correctly classified (that is, for which a correct prediction was made).

final double

correlationCoefficient()

Returns the correlation coefficient if the class is numeric.

final double

coverageOfTestCasesByPredictedRegions()

Gets the coverage of the test cases by the predicted regions at the confidence level specified when evaluation was performed.

void

crossValidateModel(String classifierString, Instances data, int numFolds, String[] options, Random random)

Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances.

void

crossValidateModel(Classifier classifier, Instances data, int numFolds, Random random)

Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances.

void

crossValidateModel(Classifier classifier, Instances data, int numFolds, Random random, Object... forPrinting)

Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances.

boolean

equals(Object obj)

Tests whether the current evaluation object is equal to another evaluation object.

final double

errorRate()

Returns the estimated error rate or the root mean squared error (if the class is numeric).

static String

evaluateModel(String classifierString, String[] options)

Evaluates a classifier with the options given in an array of strings.

static String

evaluateModel(Classifier classifier, String[] options)

Evaluates a classifier with the options given in an array of strings.

double[]

evaluateModel(Classifier classifier, Instances data, Object... forPredictionsPrinting)

Evaluates the classifier on a given set of instances.

double

evaluateModelOnce(double[] dist, Instance instance)

Evaluates the supplied distribution on a single instance.

void

evaluateModelOnce(double prediction, Instance instance)

Evaluates the supplied prediction on a single instance.

double

evaluateModelOnce(Classifier classifier, Instance instance)

Evaluates the classifier on a single instance.

double

evaluateModelOnceAndRecordPrediction(double[] dist, Instance instance)

Evaluates the supplied distribution on a single instance.

double

evaluateModelOnceAndRecordPrediction(Classifier classifier, Instance instance)

Evaluates the classifier on a single instance and records the prediction.

double

evaluationForSingleInstance(double[] dist, Instance instance, boolean storePredictions)

Evaluates the supplied distribution on a single instance.

double

falseNegativeRate(int classIndex)

Calculate the false negative rate with respect to a particular class.

double

falseNegativeRate(int classIndex, double numPositives)

Calculate the false negative rate with respect to a particular class.

double

falsePositiveRate(int classIndex)

Calculate the false positive rate with respect to a particular class.

double

falsePositiveRate(int classIndex, double numNegatives)

Calculate the false positive rate with respect to a particular class.

double

fMeasure(double precision, double recall)

Calculate the F-Measure with respect to a particular class.

double

fMeasure(int classIndex)

Calculate the F-Measure with respect to a particular class.

static List<String>

getAllEvaluationMetricNames()

Utility method to get a list of the names of all built-in and plugin evaluation metrics

double[]

getClassPriors()

Get the current weighted class counts.

boolean

getDiscardPredictions()

Returns whether predictions are not recorded at all, in order to conserve memory.

Instances

getHeader()

Returns the header of the underlying dataset.

List<String>

getMetricsToDisplay()

Get a list of the names of metrics to have appear in the output The default is to display all built in metrics and plugin metrics that haven't been globally disabled.

AbstractEvaluationMetric

getPluginMetric(String name)

Get the named plugin evaluation metric

List<AbstractEvaluationMetric>

getPluginMetrics()

Returns the list of plugin metrics in use (or null if there are none)

String

getRevision()

Returns the revision string.

final double

incorrect()

Gets the number of instances incorrectly classified (that is, for which an incorrect prediction was made).

final double

kappa()

Returns value of kappa statistic if class is nominal.

final double

KBInformation()

Return the total Kononenko & Bratko Information score in bits.

final double

KBMeanInformation()

Return the Kononenko & Bratko Information score in bits per instance.

final double

KBRelativeInformation()

Return the Kononenko & Bratko Relative Information score.

static void

main(String[] args)

A test method for this class.

double

matthewsCorrelationCoefficient(int classIndex)

Calculates the matthews correlation coefficient (sometimes called phi coefficient) for the supplied class.

double

matthewsCorrelationCoefficient(int classIndex, double numTP, double numTN, double numFP, double numFN)

Calculates the matthews correlation coefficient (sometimes called phi coefficient) for the supplied class.

final double

meanAbsoluteError()

Returns the mean absolute error.

final double

meanPriorAbsoluteError()

Returns the mean absolute error of the prior.

final double

missingClass()

Gets the weight of the instances that had missing class values

double

numClassified()

Returns the number (really, weight) of instances that have been classified.

double

numFalseNegatives(int classIndex)

Calculate number (really, weight) of false negatives with respect to a particular class.

double

numFalsePositives(int classIndex)

Calculate the number (really, weight) of false positives with respect to a particular class.

final double

numInstances()

Gets the number of test instances that had a known class value (actually the sum of the weights of test instances with known class value).

double

numNegatives(int classIndex)

Calculate the number (really, weight) of instances not in the given class.

double

numPositives(int classIndex)

Calculate the number (really, weight) of instances of the given class.

double

numPredictedNegatives(int classIndex)

Calculate the number (really, weight) of instances predicted not to be of the given class.

double

numPredictedPositives(int classIndex)

Calculate the number (really, weight) of instances predicted to be of the given class.

double

numTrueNegatives(int classIndex)

Calculate the number (really, weight) of true negatives with respect to a particular class.

double

numTruePositives(int classIndex)

Calculate the number (really, weight) of true positives with respect to a particular class.

final double

pctCorrect()

Gets the percentage of instances correctly classified (that is, for which a correct prediction was made).

final double

pctIncorrect()

Gets the percentage of instances incorrectly classified (that is, for which an incorrect prediction was made).

final double

pctUnclassified()

Gets the percentage of instances not classified (that is, for which no prediction was made by the classifier).

double

precision(int classIndex)

Calculate the precision with respect to a particular class.

double[]

predictedClassCounts()

Returns the predicted number (really, weight) of instances in each class.

ArrayList<Prediction>

predictions()

Returns the predictions that have been collected.

final double

priorEntropy()

Returns the mean base-2 log loss wrt the null model.

double

recall(int classIndex)

Calculate the recall with respect to a particular class.

final double

relativeAbsoluteError()

Returns the relative absolute error.

final double

rootMeanPriorSquaredError()

Returns the root mean prior squared error.

final double

rootMeanSquaredError()

Returns the root mean squared error.

final double

rootRelativeSquaredError()

Returns the root relative squared error if the class is numeric.

void

setDiscardPredictions(boolean value)

Sets whether to discard predictions, ie, not storing them for future reference via predictions() method in order to conserve memory.

void

setMetricsToDisplay(List<String> display)

Set a list of the names of metrics to have appear in the output.

void

setPriors(Instances train)

Sets the class prior probabilities.

final double

SFEntropyGain()

Returns the difference in base-2 log loss between null model and scheme.

final double

SFMeanEntropyGain()

Returns the mean difference in base-2 log loss between null model and scheme.

final double

SFMeanPriorEntropy()

Returns the mean base-2 log loss wrt the null model.

final double

SFMeanSchemeEntropy()

Returns the mean base-2 log loss wrt the scheme.

final double

SFPriorEntropy()

Returns the base-2 log loss wrt the null model.

final double

SFSchemeEntropy()

Returns the base-2 log loss wrt the scheme.

final double

sizeOfPredictedRegions()

Gets the average size of the predicted regions, relative to the range of the target in the training data, at the confidence level specified when evaluation was performed.

String

toClassDetailsString()

Generates a breakdown of the accuracy for each class (with default title), incorporating various information-retrieval statistics, such as true/false positive rate, precision/recall/F-Measure.

String

toClassDetailsString(String title)

Generates a breakdown of the accuracy for each class, incorporating various information-retrieval statistics, such as true/false positive rate, precision/recall/F-Measure.

String

toCumulativeMarginDistributionString()

Output the cumulative margin distribution as a string suitable for input for gnuplot or similar package.

void

toggleEvalMetrics(List<String> metricsToToggle)

Toggle the output of the metrics specified in the supplied list.

String

toMatrixString()

Calls toMatrixString() with a default title.

String

toMatrixString(String title)

Outputs the performance statistics as a classification confusion matrix.

String

toSummaryString()

Calls toSummaryString() with no title and no complexity stats.

String

toSummaryString(boolean printComplexityStatistics)

Calls toSummaryString() with a default title.

String

toSummaryString(String title, boolean printComplexityStatistics)

Outputs the performance statistics in summary form.

final double

totalCost()

Gets the total cost, that is, the cost of each prediction times the weight of the instance, summed over all instances.

double[]

trueClassCounts()

Returns the number (really, weight) of instances in each class.

double

trueNegativeRate(int classIndex)

Calculate the true negative rate with respect to a particular class.

double

trueNegativeRate(int classIndex, double numNegatives)

Calculate the true negative rate with respect to a particular class.

double

truePositiveRate(int classIndex)

Calculate the true positive rate with respect to a particular class.

double

truePositiveRate(int classIndex, double numPositives)

Calculate the true positive rate with respect to a particular class.

final double

unclassified()

Gets the number of instances not classified (that is, for which no prediction was made by the classifier).

double

unweightedMacroFmeasure()

Unweighted macro-averaged F-measure.

double

unweightedMicroFmeasure()

Unweighted micro-averaged F-measure.

void

updatePriors(Instance instance)

Updates the class prior probabilities or the mean respectively (when incrementally training).

void

useNoPriors()

disables the use of priors, e.g., in case of de-serialized schemes that have no access to the original training set, but are evaluated on a set set.

double

weightedAreaUnderPRC()

Calculates the weighted (by class size) AUPRC.

double

weightedAreaUnderROC()

Calculates the weighted (by class size) AUROC.

double

weightedFalseNegativeRate()

Calculates the weighted (by class size) false negative rate.

double

weightedFalsePositiveRate()

Calculates the weighted (by class size) false positive rate.

double

weightedFMeasure()

Calculates the macro weighted (by class size) average F-Measure.

double

weightedMatthewsCorrelation()

Calculates the weighted (by class size) matthews correlation coefficient.

double

weightedPrecision()

Calculates the weighted (by class size) precision.

double

weightedRecall()

Calculates the weighted (by class size) recall.

double

weightedTrueNegativeRate()

Calculates the weighted (by class size) true negative rate.

double

weightedTruePositiveRate()

Calculates the weighted (by class size) true positive rate.

static String

wekaStaticWrapper(Sourcable classifier, String className)

Wraps a static classifier in enough source to test using the weka class libraries.

final double

withClass()

Gets the weight of the instances that had a non-missing class value

Methods inherited from class java.lang.Object
getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- BUILT_IN_EVAL_METRICS
  
  public static final String[] BUILT_IN_EVAL_METRICS
Constructor Details
- Evaluation
  
  public Evaluation(Instances data) throws Exception
  
  Initializes all the counters for the evaluation. Use useNoPriors() if the dataset is the test set and you can't initialize with the priors from the training set via setPriors(Instances).
  Parameters:
  
  data - set of training instances, to get some header information and prior class distribution information
  
  Throws:
  
  Exception - if the class is not defined
  
  See Also:
  
  useNoPriors()
  
  setPriors(Instances)
- Evaluation
  
  public Evaluation(Instances data, CostMatrix costMatrix) throws Exception
  
  Initializes all the counters for the evaluation and also takes a cost matrix as parameter. Use useNoPriors() if the dataset is the test set and you can't initialize with the priors from the training set via setPriors(Instances).
  Parameters:
  
  data - set of training instances, to get some header information and prior class distribution information
  
  costMatrix - the cost matrix---if null, default costs will be used
  
  Throws:
  
  Exception - if cost matrix is not compatible with data, the class is not defined or the class is numeric
  
  See Also:
  
  useNoPriors()
  
  setPriors(Instances)
Method Details
- getAllEvaluationMetricNames
  
  public static List<String> getAllEvaluationMetricNames()
  
  Utility method to get a list of the names of all built-in and plugin evaluation metrics
  
  Returns:
  
  the complete list of available evaluation metrics
- getHeader
  
  public Instances getHeader()
  
  Returns the header of the underlying dataset.
  
  Returns:
  
  the header information
- setDiscardPredictions
  
  public void setDiscardPredictions(boolean value)
  
  Sets whether to discard predictions, ie, not storing them for future reference via predictions() method in order to conserve memory.
  Parameters:
  
  value - true if to discard the predictions
  
  See Also:
  
  predictions()
- getDiscardPredictions
  
  public boolean getDiscardPredictions()
  
  Returns whether predictions are not recorded at all, in order to conserve memory.
  Returns:
  
  true if predictions are not recorded
  
  See Also:
  
  predictions()
- getPluginMetrics
  
  public List<AbstractEvaluationMetric> getPluginMetrics()
  
  Returns the list of plugin metrics in use (or null if there are none)
  
  Returns:
  
  the list of plugin metrics
- setMetricsToDisplay
  
  public void setMetricsToDisplay(List<String> display)
  
  Set a list of the names of metrics to have appear in the output. The default is to display all built in metrics and plugin metrics that haven't been globally disabled.
  
  Parameters:
  
  display - a list of metric names to have appear in the output
- getMetricsToDisplay
  
  public List<String> getMetricsToDisplay()
  
  Get a list of the names of metrics to have appear in the output The default is to display all built in metrics and plugin metrics that haven't been globally disabled.
  
  Returns:
  
  a list of metric names to have appear in the output
- toggleEvalMetrics
  
  public void toggleEvalMetrics(List<String> metricsToToggle)
  
  Toggle the output of the metrics specified in the supplied list.
  
  Parameters:
  
  metricsToToggle - a list of metrics to toggle
- getPluginMetric
  
  public AbstractEvaluationMetric getPluginMetric(String name)
  
  Get the named plugin evaluation metric
  
  Parameters:
  
  name - the name of the metric (as returned by AbstractEvaluationMetric.getName()) or the fully qualified class name of the metric to find
  
  Returns:
  
  the metric or null if the metric is not in the list of plugin metrics
- areaUnderROC
  
  public double areaUnderROC(int classIndex)
  
  Returns the area under ROC for those predictions that have been collected in the evaluateClassifier(Classifier, Instances) method. Returns Utils.missingValue() if the area is not available.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the area under the ROC curve or not a number
- weightedAreaUnderROC
  
  public double weightedAreaUnderROC()
  
  Calculates the weighted (by class size) AUROC. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted AUC.
- areaUnderPRC
  
  public double areaUnderPRC(int classIndex)
  
  Returns the area under precision-recall curve (AUPRC) for those predictions that have been collected in the evaluateClassifier(Classifier, Instances) method. Returns Utils.missingValue() if the area is not available.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the area under the precision-recall curve or not a number
- weightedAreaUnderPRC
  
  public double weightedAreaUnderPRC()
  
  Calculates the weighted (by class size) AUPRC. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted AUPRC.
- confusionMatrix
  
  public double[][] confusionMatrix()
  
  Returns a copy of the confusion matrix.
  
  Returns:
  
  a copy of the confusion matrix as a two-dimensional array
- crossValidateModel
  
  public void crossValidateModel(Classifier classifier, Instances data, int numFolds, Random random) throws Exception
  
  Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances. Now performs a deep copy of the classifier before each call to buildClassifier() (just in case the classifier is not initialized properly).
  
  Parameters:
  
  classifier - the classifier with any options set.
  
  data - the data on which the cross-validation is to be performed
  
  numFolds - the number of folds for the cross-validation
  
  random - random number generator for randomization
  
  Throws:
  
  Exception - if a classifier could not be generated successfully or the class is not defined
- crossValidateModel
  
  public void crossValidateModel(Classifier classifier, Instances data, int numFolds, Random random, Object... forPrinting) throws Exception
  
  Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances. Performs a deep copy of the classifier before each call to buildClassifier() (just in case the classifier is not initialized properly).
  
  Parameters:
  
  classifier - the classifier with any options set.
  
  data - the data on which the cross-validation is to be performed
  
  numFolds - the number of folds for the cross-validation
  
  random - random number generator for randomization
  
  forPrinting - varargs parameter that, if supplied, is expected to hold a weka.classifiers.evaluation.output.prediction.AbstractOutput object or a StringBuffer for model output
  
  Throws:
  
  Exception - if a classifier could not be generated successfully or the class is not defined
- crossValidateModel
  
  public void crossValidateModel(String classifierString, Instances data, int numFolds, String[] options, Random random) throws Exception
  
  Performs a (stratified if class is nominal) cross-validation for a classifier on a set of instances.
  
  Parameters:
  
  classifierString - a string naming the class of the classifier
  
  data - the data on which the cross-validation is to be performed
  
  numFolds - the number of folds for the cross-validation
  
  options - the options to the classifier. Any options
  
  random - the random number generator for randomizing the data accepted by the classifier will be removed from this array.
  
  Throws:
  
  Exception - if a classifier could not be generated successfully or the class is not defined
- evaluateModel
  
  public static String evaluateModel(String classifierString, String[] options) throws Exception
  
  Evaluates a classifier with the options given in an array of strings.
  
  Valid options are:
  
  -t filename
  Name of the file with the training data. (required)
  
  -T filename
  Name of the file with the test data. If missing a cross-validation is performed.
  
  -c index
  Index of the class attribute (1, 2, ...; default: last).
  
  -x number
  The number of folds for the cross-validation (default: 10).
  
  -no-cv
  No cross validation. If no test file is provided, no evaluation is done.
  
  -split-percentage percentage
  Sets the percentage for the train/test set split, e.g., 66.
  
  -preserve-order
  Preserves the order in the percentage split instead of randomizing the data first with the seed value ('-s').
  
  -s seed
  Random number seed for the cross-validation and percentage split (default: 1).
  
  -m filename
  The name of a file containing a cost matrix.
  
  -l filename
  Loads classifier from the given file. In case the filename ends with ".xml",a PMML file is loaded or, if that fails, options are loaded from XML.
  
  -d filename
  Saves classifier built from the training data into the given file. In case the filename ends with ".xml" the options are saved XML, not the model.
  
  -v
  Outputs no statistics for the training data.
  
  -o
  Outputs statistics only, not the classifier.
  
  -output-models-for-training-splits
  Output models for training splits if cross-validation or percentage-split evaluation is used.
  
  -do-not-output-per-class-statistics
  Do not output statistics per class.
  
  -k
  Outputs information-theoretic statistics.
  
  -classifications "weka.classifiers.evaluation.output.prediction.AbstractOutput + options"
  Uses the specified class for generating the classification output. E.g.: weka.classifiers.evaluation.output.prediction.PlainText or : weka.classifiers.evaluation.output.prediction.CSV
  -p range
  Outputs predictions for test instances (or the train instances if no test instances provided and -no-cv is used), along with the attributes in the specified range (and nothing else). Use '-p 0' if no attributes are desired.
  Deprecated: use "-classifications ..." instead.
  
  -distribution
  Outputs the distribution instead of only the prediction in conjunction with the '-p' option (only nominal classes).
  Deprecated: use "-classifications ..." instead.
  
  -no-predictions
  Turns off the collection of predictions in order to conserve memory.
  
  -r
  Outputs cumulative margin distribution (and nothing else).
  
  -g
  Only for classifiers that implement "Graphable." Outputs the graph representation of the classifier (and nothing else).
  
  -xml filename | xml-string
  Retrieves the options from the XML-data instead of the command line.
  
  -threshold-file file
  The file to save the threshold data to. The format is determined by the extensions, e.g., '.arff' for ARFF format or '.csv' for CSV.
  
  -threshold-label label
  The class label to determine the threshold data for (default is the first label)
  
  Parameters:
  
  classifierString - class of machine learning classifier as a string
  
  options - the array of string containing the options
  
  Returns:
  
  a string describing the results
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- main
  
  public static void main(String[] args)
  
  A test method for this class. Just extracts the first command line argument as a classifier class name and calls evaluateModel.
  
  Parameters:
  
  args - an array of command line arguments, the first of which must be the class name of a classifier.
- evaluateModel
  
  public static String evaluateModel(Classifier classifier, String[] options) throws Exception
  
  Evaluates a classifier with the options given in an array of strings.
  Valid options are:
  -t name of training file
  Name of the file with the training data. (required)
  -T name of test file
  Name of the file with the test data. If missing a cross-validation is performed.
  -c class index
  Index of the class attribute (1, 2, ...; default: last).
  -x number of folds
  The number of folds for the cross-validation (default: 10).
  -no-cv
  No cross validation. If no test file is provided, no evaluation is done.
  -split-percentage percentage
  Sets the percentage for the train/test set split, e.g., 66.
  -preserve-order
  Preserves the order in the percentage split instead of randomizing the data first with the seed value ('-s').
  -s seed
  Random number seed for the cross-validation and percentage split (default: 1).
  -m file with cost matrix
  The name of a file containing a cost matrix.
  -l filename
  Loads classifier from the given file. In case the filename ends with ".xml",a PMML file is loaded or, if that fails, options are loaded from XML.
  -d filename
  Saves classifier built from the training data into the given file. In case the filename ends with ".xml" the options are saved XML, not the model.
  -v
  Outputs no statistics for the training data.
  -o
  Outputs statistics only, not the classifier.
  -output-models-for-training-splits
  Output models for training splits if cross-validation or percentage-split evaluation is used.
  -do-not-output-per-class-statistics
  Do not output statistics per class.
  -k
  Outputs information-theoretic statistics.
  -classifications "weka.classifiers.evaluation.output.prediction.AbstractOutput + options"
  Uses the specified class for generating the classification output. E.g.: weka.classifiers.evaluation.output.prediction.PlainText or : weka.classifiers.evaluation.output.prediction.CSV -p range
  Outputs predictions for test instances (or the train instances if no test instances provided and -no-cv is used), along with the attributes in the specified range (and nothing else). Use '-p 0' if no attributes are desired.
  Deprecated: use "-classifications ..." instead.
  -distribution
  Outputs the distribution instead of only the prediction in conjunction with the '-p' option (only nominal classes).
  Deprecated: use "-classifications ..." instead.
  -no-predictions
  Turns off the collection of predictions in order to conserve memory.
  -r
  Outputs cumulative margin distribution (and nothing else).
  -g
  Only for classifiers that implement "Graphable." Outputs the graph representation of the classifier (and nothing else).
  -xml filename | xml-string
  Retrieves the options from the XML-data instead of the command line.
  
  Parameters:
  
  classifier - machine learning classifier
  
  options - the array of string containing the options
  
  Returns:
  
  a string describing the results
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- evaluateModel
  
  public double[] evaluateModel(Classifier classifier, Instances data, Object... forPredictionsPrinting) throws Exception
  
  Evaluates the classifier on a given set of instances. Note that the data must have exactly the same format (e.g. order of attributes) as the data used to train the classifier! Otherwise the results will generally be meaningless.
  
  Parameters:
  
  classifier - machine learning classifier
  
  data - set of test instances for evaluation
  
  forPredictionsPrinting - varargs parameter that, if supplied, is expected to hold a weka.classifiers.evaluation.output.prediction.AbstractOutput object
  
  Returns:
  
  the predictions
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- evaluationForSingleInstance
  
  public double evaluationForSingleInstance(double[] dist, Instance instance, boolean storePredictions) throws Exception
  
  Evaluates the supplied distribution on a single instance.
  
  Parameters:
  
  dist - the supplied distribution
  
  instance - the test instance to be classified
  
  storePredictions - whether to store predictions for nominal classifier
  
  Returns:
  
  the prediction
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- evaluateModelOnceAndRecordPrediction
  
  public double evaluateModelOnceAndRecordPrediction(Classifier classifier, Instance instance) throws Exception
  
  Evaluates the classifier on a single instance and records the prediction.
  
  Parameters:
  
  classifier - machine learning classifier
  
  instance - the test instance to be classified
  
  Returns:
  
  the prediction made by the clasifier
  
  Throws:
  
  Exception - if model could not be evaluated successfully or the data contains string attributes
- evaluateModelOnce
  
  public double evaluateModelOnce(Classifier classifier, Instance instance) throws Exception
  
  Evaluates the classifier on a single instance.
  
  Parameters:
  
  classifier - machine learning classifier
  
  instance - the test instance to be classified
  
  Returns:
  
  the prediction made by the clasifier
  
  Throws:
  
  Exception - if model could not be evaluated successfully or the data contains string attributes
- evaluateModelOnce
  
  public double evaluateModelOnce(double[] dist, Instance instance) throws Exception
  
  Evaluates the supplied distribution on a single instance.
  
  Parameters:
  
  dist - the supplied distribution
  
  instance - the test instance to be classified
  
  Returns:
  
  the prediction
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- evaluateModelOnceAndRecordPrediction
  
  public double evaluateModelOnceAndRecordPrediction(double[] dist, Instance instance) throws Exception
  
  Evaluates the supplied distribution on a single instance.
  
  Parameters:
  
  dist - the supplied distribution
  
  instance - the test instance to be classified
  
  Returns:
  
  the prediction
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- evaluateModelOnce
  
  public void evaluateModelOnce(double prediction, Instance instance) throws Exception
  
  Evaluates the supplied prediction on a single instance.
  
  Parameters:
  
  prediction - the supplied prediction
  
  instance - the test instance to be classified
  
  Throws:
  
  Exception - if model could not be evaluated successfully
- predictions
  
  public ArrayList<Prediction> predictions()
  
  Returns the predictions that have been collected.
  
  Returns:
  
  a reference to the FastVector containing the predictions that have been collected. This should be null if no predictions have been collected.
- wekaStaticWrapper
  
  public static String wekaStaticWrapper(Sourcable classifier, String className) throws Exception
  
  Wraps a static classifier in enough source to test using the weka class libraries.
  
  Parameters:
  
  classifier - a Sourcable Classifier
  
  className - the name to give to the source code class
  
  Returns:
  
  the source for a static classifier that can be tested with weka libraries.
  
  Throws:
  
  Exception - if code-generation fails
- numInstances
  
  public final double numInstances()
  
  Gets the number of test instances that had a known class value (actually the sum of the weights of test instances with known class value).
  
  Returns:
  
  the number of test instances with known class
- coverageOfTestCasesByPredictedRegions
  
  public final double coverageOfTestCasesByPredictedRegions()
  
  Gets the coverage of the test cases by the predicted regions at the confidence level specified when evaluation was performed.
  
  Returns:
  
  the coverage of the test cases by the predicted regions
- sizeOfPredictedRegions
  
  public final double sizeOfPredictedRegions()
  
  Gets the average size of the predicted regions, relative to the range of the target in the training data, at the confidence level specified when evaluation was performed.
  
  Returns:
  
  the average size of the predicted regions
- withClass
  
  public final double withClass()
  
  Gets the weight of the instances that had a non-missing class value
  
  Returns:
  
  the weight of the instances that had a non-missing class value
- missingClass
  
  public final double missingClass()
  
  Gets the weight of the instances that had missing class values
  
  Returns:
  
  the weight of the instances that had missing class values
- incorrect
  
  public final double incorrect()
  
  Gets the number of instances incorrectly classified (that is, for which an incorrect prediction was made). (Actually the sum of the weights of these instances)
  
  Returns:
  
  the number of incorrectly classified instances
- pctIncorrect
  
  public final double pctIncorrect()
  
  Gets the percentage of instances incorrectly classified (that is, for which an incorrect prediction was made).
  
  Returns:
  
  the percent of incorrectly classified instances (between 0 and 100)
- totalCost
  
  public final double totalCost()
  
  Gets the total cost, that is, the cost of each prediction times the weight of the instance, summed over all instances.
  
  Returns:
  
  the total cost
- avgCost
  
  public final double avgCost()
  
  Gets the average cost, that is, total cost of misclassifications (incorrect plus unclassified) over the total number of instances.
  
  Returns:
  
  the average cost.
- correct
  
  public final double correct()
  
  Gets the number of instances correctly classified (that is, for which a correct prediction was made). (Actually the sum of the weights of these instances)
  
  Returns:
  
  the number of correctly classified instances
- pctCorrect
  
  public final double pctCorrect()
  
  Gets the percentage of instances correctly classified (that is, for which a correct prediction was made).
  
  Returns:
  
  the percent of correctly classified instances (between 0 and 100)
- unclassified
  
  public final double unclassified()
  
  Gets the number of instances not classified (that is, for which no prediction was made by the classifier). (Actually the sum of the weights of these instances)
  
  Returns:
  
  the number of unclassified instances
- pctUnclassified
  
  public final double pctUnclassified()
  
  Gets the percentage of instances not classified (that is, for which no prediction was made by the classifier).
  
  Returns:
  
  the percent of unclassified instances (between 0 and 100)
- errorRate
  
  public final double errorRate()
  
  Returns the estimated error rate or the root mean squared error (if the class is numeric). If a cost matrix was given this error rate gives the average cost.
  
  Returns:
  
  the estimated error rate (between 0 and 1, or between 0 and maximum cost)
- kappa
  
  public final double kappa()
  
  Returns value of kappa statistic if class is nominal.
  
  Returns:
  
  the value of the kappa statistic
- correlationCoefficient
  
  public final double correlationCoefficient() throws Exception
  
  Returns the correlation coefficient if the class is numeric.
  
  Returns:
  
  the correlation coefficient
  
  Throws:
  
  Exception - if class is not numeric
- meanAbsoluteError
  
  public final double meanAbsoluteError()
  
  Returns the mean absolute error. Refers to the error of the predicted values for numeric classes, and the error of the predicted probability distribution for nominal classes.
  
  Returns:
  
  the mean absolute error
- meanPriorAbsoluteError
  
  public final double meanPriorAbsoluteError()
  
  Returns the mean absolute error of the prior.
  
  Returns:
  
  the mean absolute error
- relativeAbsoluteError
  
  public final double relativeAbsoluteError() throws Exception
  
  Returns the relative absolute error.
  
  Returns:
  
  the relative absolute error
  
  Throws:
  
  Exception - if it can't be computed
- rootMeanSquaredError
  
  public final double rootMeanSquaredError()
  
  Returns the root mean squared error.
  
  Returns:
  
  the root mean squared error
- rootMeanPriorSquaredError
  
  public final double rootMeanPriorSquaredError()
  
  Returns the root mean prior squared error.
  
  Returns:
  
  the root mean prior squared error
- rootRelativeSquaredError
  
  public final double rootRelativeSquaredError()
  
  Returns the root relative squared error if the class is numeric.
  
  Returns:
  
  the root relative squared error
- priorEntropy
  
  public final double priorEntropy()
  
  Returns the mean base-2 log loss wrt the null model. Just calls SFMeanPriorEntropy.
  
  Returns:
  
  the null model entropy per instance
- KBInformation
  
  public final double KBInformation() throws Exception
  
  Return the total Kononenko & Bratko Information score in bits.
  
  Returns:
  
  the K&B information score
  
  Throws:
  
  Exception - if the class is not nominal
- KBMeanInformation
  
  public final double KBMeanInformation() throws Exception
  
  Return the Kononenko & Bratko Information score in bits per instance.
  
  Returns:
  
  the K&B information score
  
  Throws:
  
  Exception - if the class is not nominal
- KBRelativeInformation
  
  public final double KBRelativeInformation() throws Exception
  
  Return the Kononenko & Bratko Relative Information score. Differs slightly from the expression used in KB's paper because it uses the mean log-loss of the TEST instances wrt to the null model for normalization.
  
  Returns:
  
  the K&B relative information score
  
  Throws:
  
  Exception - if the class is not nominal
- SFPriorEntropy
  
  public final double SFPriorEntropy()
  
  Returns the base-2 log loss wrt the null model.
  
  Returns:
  
  the total null model entropy
- SFMeanPriorEntropy
  
  public final double SFMeanPriorEntropy()
  
  Returns the mean base-2 log loss wrt the null model.
  
  Returns:
  
  the null model entropy per instance
- SFSchemeEntropy
  
  public final double SFSchemeEntropy()
  
  Returns the base-2 log loss wrt the scheme.
  
  Returns:
  
  the total scheme entropy
- SFMeanSchemeEntropy
  
  public final double SFMeanSchemeEntropy()
  
  Returns the mean base-2 log loss wrt the scheme.
  
  Returns:
  
  the scheme entropy per instance
- SFEntropyGain
  
  public final double SFEntropyGain()
  
  Returns the difference in base-2 log loss between null model and scheme.
  
  Returns:
  
  the total "SF score"
- SFMeanEntropyGain
  
  public final double SFMeanEntropyGain()
  
  Returns the mean difference in base-2 log loss between null model and scheme.
  
  Returns:
  
  the "SF score" per instance
- toCumulativeMarginDistributionString
  
  public String toCumulativeMarginDistributionString() throws Exception
  
  Output the cumulative margin distribution as a string suitable for input for gnuplot or similar package.
  
  Returns:
  
  the cumulative margin distribution
  
  Throws:
  
  Exception - if the class attribute is nominal
- toSummaryString
  
  public String toSummaryString()
  
  Calls toSummaryString() with no title and no complexity stats.
  
  Specified by:
  
  toSummaryString in interface Summarizable
  
  Returns:
  
  a summary description of the classifier evaluation
- toSummaryString
  
  public String toSummaryString(boolean printComplexityStatistics)
  
  Calls toSummaryString() with a default title.
  
  Parameters:
  
  printComplexityStatistics - if true, complexity statistics are returned as well
  
  Returns:
  
  the summary string
- toSummaryString
  
  public String toSummaryString(String title, boolean printComplexityStatistics)
  
  Outputs the performance statistics in summary form. Lists number (and percentage) of instances classified correctly, incorrectly and unclassified. Outputs the total number of instances classified, and the number of instances (if any) that had no class value provided.
  
  Parameters:
  
  title - the title for the statistics
  
  printComplexityStatistics - if true, complexity statistics are returned as well
  
  Returns:
  
  the summary as a String
- toMatrixString
  
  public String toMatrixString() throws Exception
  
  Calls toMatrixString() with a default title.
  
  Returns:
  
  the confusion matrix as a string
  
  Throws:
  
  Exception - if the class is numeric
- toMatrixString
  
  public String toMatrixString(String title) throws Exception
  
  Outputs the performance statistics as a classification confusion matrix. For each class value, shows the distribution of predicted class values.
  
  Parameters:
  
  title - the title for the confusion matrix
  
  Returns:
  
  the confusion matrix as a String
  
  Throws:
  
  Exception - if the class is numeric
- toClassDetailsString
  
  public String toClassDetailsString() throws Exception
  
  Generates a breakdown of the accuracy for each class (with default title), incorporating various information-retrieval statistics, such as true/false positive rate, precision/recall/F-Measure. Should be useful for ROC curves, recall/precision curves.
  
  Returns:
  
  the statistics presented as a string
  
  Throws:
  
  Exception - if class is not nominal
- toClassDetailsString
  
  public String toClassDetailsString(String title) throws Exception
  
  Generates a breakdown of the accuracy for each class, incorporating various information-retrieval statistics, such as true/false positive rate, precision/recall/F-Measure. Should be useful for ROC curves, recall/precision curves.
  
  Parameters:
  
  title - the title to prepend the stats string with
  
  Returns:
  
  the statistics presented as a string
  
  Throws:
  
  Exception - if class is not nominal
- numClassified
  
  public double numClassified()
  
  Returns the number (really, weight) of instances that have been classified. This is equal to the sum of all entries in the confusion matrix.
  
  Returns:
  
  the number (really, weight) of instances that have been classified
- numPositives
  
  public double numPositives(int classIndex)
  
  Calculate the number (really, weight) of instances of the given class. This only includes instances for which a prediction is made. Unclassified instances are not included even if they have a class value.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the number of positive instances
- numPredictedPositives
  
  public double numPredictedPositives(int classIndex)
  
  Calculate the number (really, weight) of instances predicted to be of the given class. Unclassified instances are not included in the calculation.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the number of instances that are predicted to be positive
- numNegatives
  
  public double numNegatives(int classIndex)
  
  Calculate the number (really, weight) of instances not in the given class. This only includes instances for which a prediction is made. Unclassified instances are not included even if they have a class value.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the number of negative instances
- numPredictedNegatives
  
  public double numPredictedNegatives(int classIndex)
  
  Calculate the number (really, weight) of instances predicted not to be of the given class. Unclassified instances are not included in the calculation.
  
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the number of instances that are predicted to be negative
- trueClassCounts
  
  public double[] trueClassCounts()
  
  Returns the number (really, weight) of instances in each class. Unclassified instances are not included.
  
  Returns:
  
  an array with the number (really, weight) of instances in each class
- predictedClassCounts
  
  public double[] predictedClassCounts()
  
  Returns the predicted number (really, weight) of instances in each class. Unclassified instances are not included in the calculation.
  
  Returns:
  
  an array with the predicted number (really, weight) of instances in each class
- numTruePositives
  
  public double numTruePositives(int classIndex)
  Calculate the number (really, weight) of true positives with respect to a particular class. This is defined as
  
  correctly classified positives
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the true positive rate
- numFalseNegatives
  
  public double numFalseNegatives(int classIndex)
  Calculate number (really, weight) of false negatives with respect to a particular class. This is defined as
  
  incorrectly classified positives
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the false positive rate
- numTrueNegatives
  
  public double numTrueNegatives(int classIndex)
  Calculate the number (really, weight) of true negatives with respect to a particular class. This is defined as
  
  correctly classified negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the true positive rate
- numFalsePositives
  
  public double numFalsePositives(int classIndex)
  Calculate the number (really, weight) of false positives with respect to a particular class. This is defined as
  
  incorrectly classified negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the false positive rate
- truePositiveRate
  
  public double truePositiveRate(int classIndex)
  Calculate the true positive rate with respect to a particular class. This is defined as
  
  correctly classified positives ------------------------------ total positives
  This does not include any instances that are left unclassified by the classifier.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the true positive rate
- truePositiveRate
  
  public double truePositiveRate(int classIndex, double numPositives)
  Calculate the true positive rate with respect to a particular class. This is defined as
  
  correctly classified positives ------------------------------ total positives
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  numPositives - the total number (really, weight) of positives for that class
  
  Returns:
  
  the true positive rate
- weightedTruePositiveRate
  
  public double weightedTruePositiveRate()
  
  Calculates the weighted (by class size) true positive rate. Unclassified instances are not included in the calculation. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted true positive rate.
- trueNegativeRate
  
  public double trueNegativeRate(int classIndex)
  Calculate the true negative rate with respect to a particular class. This is defined as
  
  correctly classified negatives ------------------------------ total negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the true positive rate
- trueNegativeRate
  
  public double trueNegativeRate(int classIndex, double numNegatives)
  Calculate the true negative rate with respect to a particular class. This is defined as
  
  correctly classified negatives ------------------------------ total negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  numNegatives - the number (really, weight) of the negative instances
  
  Returns:
  
  the true positive rate
- weightedTrueNegativeRate
  
  public double weightedTrueNegativeRate()
  
  Calculates the weighted (by class size) true negative rate. Unclassified instances are not included in the calculation. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted true negative rate.
- falsePositiveRate
  
  public double falsePositiveRate(int classIndex)
  Calculate the false positive rate with respect to a particular class. This is defined as
  
  incorrectly classified negatives -------------------------------- total negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the false positive rate
- falsePositiveRate
  
  public double falsePositiveRate(int classIndex, double numNegatives)
  Calculate the false positive rate with respect to a particular class. This is defined as
  
  incorrectly classified negatives -------------------------------- total negatives
  Unclassified instances are not included in the calculation
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  numNegatives - the number (really, weight) of the negative instances
  
  Returns:
  
  the false positive rate
- weightedFalsePositiveRate
  
  public double weightedFalsePositiveRate()
  
  Calculates the weighted (by class size) false positive rate. Unclassified instances are not included in the calculation. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted false positive rate.
- falseNegativeRate
  
  public double falseNegativeRate(int classIndex)
  Calculate the false negative rate with respect to a particular class. This is defined as
  
  incorrectly classified positives -------------------------------- total positives
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the false positive rate
- falseNegativeRate
  
  public double falseNegativeRate(int classIndex, double numPositives)
  Calculate the false negative rate with respect to a particular class. This is defined as
  
  incorrectly classified positives -------------------------------- total positives
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  numPositives - the number (really, weight) of the instances in the positive class
  
  Returns:
  
  the false positive rate
- weightedFalseNegativeRate
  
  public double weightedFalseNegativeRate()
  
  Calculates the weighted (by class size) false negative rate. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted false negative rate.
- matthewsCorrelationCoefficient
  
  public double matthewsCorrelationCoefficient(int classIndex)
  
  Calculates the matthews correlation coefficient (sometimes called phi coefficient) for the supplied class. Unclassified instances are not included in the calculation.
  
  Parameters:
  
  classIndex - the index of the class to compute the matthews correlation coefficient for Unclassified instances are not included in the calculation.
  
  Returns:
  
  the mathews correlation coefficient
- matthewsCorrelationCoefficient
  
  public double matthewsCorrelationCoefficient(int classIndex, double numTP, double numTN, double numFP, double numFN)
  
  Calculates the matthews correlation coefficient (sometimes called phi coefficient) for the supplied class. Unclassified instances are not included in the calculation.
  
  Parameters:
  
  classIndex - the index of the class to compute the matthews correlation coefficient for
  
  numTP - the number (really, weight) of the true positive instances
  
  numTN - the number (really, weight) of the true negative instances
  
  numFP - the number (really, weight) of the false positive instances
  
  numFN - the number (really, weight) of the false negative instances
  
  Returns:
  
  the mathews correlation coefficient
- weightedMatthewsCorrelation
  
  public double weightedMatthewsCorrelation()
  
  Calculates the weighted (by class size) matthews correlation coefficient. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted matthews correlation coefficient.
- recall
  
  public double recall(int classIndex)
  Calculate the recall with respect to a particular class. This is defined as
  
  correctly classified positives ------------------------------ total positives
  
  This is the same as the true positive rate. Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the recall
- weightedRecall
  
  public double weightedRecall()
  
  Calculates the weighted (by class size) recall. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted recall.
- precision
  
  public double precision(int classIndex)
  Calculate the precision with respect to a particular class. This is defined as
  
  correctly classified positives ------------------------------ total predicted as positive
  Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the precision
- weightedPrecision
  
  public double weightedPrecision()
  
  Calculates the weighted (by class size) precision. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted precision.
- fMeasure
  
  public double fMeasure(int classIndex)
  Calculate the F-Measure with respect to a particular class. This is defined as
  
  2 * recall * precision ---------------------- recall + precision
  Returns zero when both precision and recall are zero. Unclassified instances are not included in the calculation.
  Parameters:
  
  classIndex - the index of the class to consider as "positive"
  
  Returns:
  
  the F-Measure
- fMeasure
  
  public double fMeasure(double precision, double recall)
  Calculate the F-Measure with respect to a particular class. This is defined as
  
  2 * recall * precision ---------------------- recall + precision
  Returns zero when both precision and recall are zero. Unclassified instances are not included in the calculation.
  Parameters:
  
  precision - the precision
  
  recall - the recall
  
  Returns:
  
  the F-Measure
- weightedFMeasure
  
  public double weightedFMeasure()
  
  Calculates the macro weighted (by class size) average F-Measure. Unclassified instances are not included in the calculation.
  
  Returns:
  
  the weighted F-Measure.
- unweightedMacroFmeasure
  
  public double unweightedMacroFmeasure()
  
  Unweighted macro-averaged F-measure. If some classes not present in the test set, they're just skipped (since recall is undefined there anyway) . Unclassified instances are not included in the calculation.
  
  Returns:
  
  unweighted macro-averaged F-measure.
- unweightedMicroFmeasure
  
  public double unweightedMicroFmeasure()
  
  Unweighted micro-averaged F-measure. If some classes not present in the test set, they have no effect. Note: if the test set is *single-label*, then this is the same as accuracy. Unclassified instances are not included in the calculation.
  
  Returns:
  
  unweighted micro-averaged F-measure.
- setPriors
  
  public void setPriors(Instances train) throws Exception
  
  Sets the class prior probabilities.
  
  Parameters:
  
  train - the training instances used to determine the prior probabilities
  
  Throws:
  
  Exception - if the class attribute of the instances is not set
- getClassPriors
  
  public double[] getClassPriors()
  
  Get the current weighted class counts.
  
  Returns:
  
  the weighted class counts
- updatePriors
  
  public void updatePriors(Instance instance) throws Exception
  
  Updates the class prior probabilities or the mean respectively (when incrementally training).
  
  Parameters:
  
  instance - the new training instance seen
  
  Throws:
  
  Exception - if the class of the instance is not set
- useNoPriors
  
  public void useNoPriors()
  
  disables the use of priors, e.g., in case of de-serialized schemes that have no access to the original training set, but are evaluated on a set set.
- equals
  
  public boolean equals(Object obj)
  
  Tests whether the current evaluation object is equal to another evaluation object.
  
  Overrides:
  
  equals in class Object
  
  Parameters:
  
  obj - the object to compare against
  
  Returns:
  
  true if the two objects are equal
- getRevision
  
  public String getRevision()
  
  Returns the revision string.
  
  Specified by:
  
  getRevision in interface RevisionHandler
  
  Returns:
  
  the revision

Class Evaluation

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Details

BUILT_IN_EVAL_METRICS

Constructor Details

Evaluation

Evaluation

Method Details

getAllEvaluationMetricNames

getHeader

setDiscardPredictions

getDiscardPredictions

getPluginMetrics

setMetricsToDisplay

getMetricsToDisplay

toggleEvalMetrics

getPluginMetric

areaUnderROC

weightedAreaUnderROC

areaUnderPRC

weightedAreaUnderPRC

confusionMatrix

crossValidateModel

crossValidateModel

crossValidateModel

evaluateModel

main

evaluateModel

evaluateModel

evaluationForSingleInstance

evaluateModelOnceAndRecordPrediction

evaluateModelOnce

evaluateModelOnce

evaluateModelOnceAndRecordPrediction

evaluateModelOnce

predictions

wekaStaticWrapper

numInstances

coverageOfTestCasesByPredictedRegions

sizeOfPredictedRegions

withClass

missingClass

incorrect

pctIncorrect

totalCost

avgCost

correct

pctCorrect

unclassified

pctUnclassified

errorRate

kappa

correlationCoefficient

meanAbsoluteError

meanPriorAbsoluteError

relativeAbsoluteError

rootMeanSquaredError

rootMeanPriorSquaredError

rootRelativeSquaredError

priorEntropy

KBInformation

KBMeanInformation

KBRelativeInformation

SFPriorEntropy

SFMeanPriorEntropy

SFSchemeEntropy

SFMeanSchemeEntropy

SFEntropyGain

SFMeanEntropyGain

toCumulativeMarginDistributionString

toSummaryString

toSummaryString

toSummaryString

toMatrixString

toMatrixString

toClassDetailsString

toClassDetailsString