Package nltk_lite :: Module evaluate
[hide private]
[frames] | no frames]

Module evaluate

source code

Utility functions for evaluating processing modules.

Classes [hide private]
  ConfusionMatrix
The confusion matrix between a list of reference values and a corresponding list of test values.
Functions [hide private]
 
accuracy(reference, test)
Given a list of reference values and a corresponding list of test values, return the percentage of corresponding values that are equal.
source code
float or None
precision(reference, test)
Given a set of reference values and a set of test values, return the percentage of test values that appear in the reference set.
source code
float or None
recall(reference, test)
Given a set of reference values and a set of test values, return the percentage of reference values that appear in the test set.
source code
float or None
f_measure(reference, test, alpha=0.5)
Given a set of reference values and a set of test values, return the f-measure of the test values, when compared against the reference values.
source code
 
log_likelihood(reference, test)
Given a list of reference values and a corresponding list of test probability distributions, return the average log likelihood of the reference values, given the probability distributions.
source code
 
demo() source code
Function Details [hide private]

accuracy(reference, test)

source code 

Given a list of reference values and a corresponding list of test values, return the percentage of corresponding values that are equal. In particular, return the percentage of indices 0<i<=len(test) such that test[i] == reference[i].

Parameters:
  • reference (list) - An ordered list of reference values.
  • test (list) - A list of values to compare against the corresponding reference values.
Raises:
  • ValueError - If reference and length do not have the same length.

precision(reference, test)

source code 

Given a set of reference values and a set of test values, return the percentage of test values that appear in the reference set. In particular, return |referencetest|/|test|. If test is empty, then return None.

Parameters:
  • reference (Set) - A set of reference values.
  • test (Set) - A set of values to compare against the reference set.
Returns: float or None

recall(reference, test)

source code 

Given a set of reference values and a set of test values, return the percentage of reference values that appear in the test set. In particular, return |referencetest|/|reference|. If reference is empty, then return None.

Parameters:
  • reference (Set) - A set of reference values.
  • test (Set) - A set of values to compare against the reference set.
Returns: float or None

f_measure(reference, test, alpha=0.5)

source code 

Given a set of reference values and a set of test values, return the f-measure of the test values, when compared against the reference values. The f-measure is the harmonic mean of the precision and recall, weighted by alpha. In particular, given the precision p and recall r defined by:

  • p = |referencetest|/|test|
  • r = |referencetest|/|reference|

The f-measure is:

  • 1/(alpha/p + (1-alpha)/r)

If either reference or test is empty, then f_measure returns None.

Parameters:
  • reference (Set) - A set of reference values.
  • test (Set) - A set of values to compare against the reference set.
Returns: float or None

log_likelihood(reference, test)

source code 

Given a list of reference values and a corresponding list of test probability distributions, return the average log likelihood of the reference values, given the probability distributions.

Parameters:
  • reference (list) - A list of reference values
  • test (list of ProbDist) - A list of probability distributions over values to compare against the corresponding reference values.