Online construction of classifiers is a challenging problem. We have adapted the online Kernel Density Estimation (oKDE) framework, which allows online adaptation of generative models from new data-points. We extend the oKDE to take into account the interclass discrimination through a new distance function between the classifiers. The result is an online discriminative Kernel Density Estimator (odKDE).
From a classification point of view, two classifiers are equivalent as long as they classify the relevant part fo the feature space equally. Therefore, we can accept compressions which do not change significantly the posterior distribution over the classes. We have proposed a measure that evaluates how much the posterior changes during compression of the positive class model in the classifier. This measure is used with the odKDE to determine the extent of the allowed compression during online adaptation.
Discriminative density estimation examples
These two examples show how relation between the two classes affects the compression. The complexity of the estimated pdfs is reduced while the classification accuracy remains high.
Online resources
Matlab online KDE and online discriminative KDE code:
- Version 3.5: link