Knowledge transfer among classifiers in machine learning

Mihec Pezdirc (2011) Knowledge transfer among classifiers in machine learning. EngD thesis.

	PDF Download (1382Kb)
	PDF Download (1382Kb)

Abstract

The aim of the thesis is to propose and test three methods for knowledge transfer between various machine learning models and evaluate the success of such transfers in terms of classification accuracy. The first among the proposed methods performs the knowledge transfer by re-labeling the learning set by the teacher-model and passing this newly labeled learning set to the student-model for learning. The second proposed method filters out a chosen ratio of examples that have the lowest probability of the most probable class (with respect to the predicted class probability distribution). The third method uses filtering of examples based on a classification reliability estimate (proposed in the related work in progress). We evaluate knowledge transfer using five different machine learning models: SVM, ANN, kNN, naive Bayes and decision trees. The tests for all three developed knowledge transfer methods and five models were performed on ten different benchmark domains (UCI). The results indicate that the knowledge transfer resulted in a statistically unchanged classification accuracy in 61% of experiments. In 33% of experiments classification accuracy decreased and in 6% of experiments the classification accuracy increased. In terms of gaining classification accuracy, the biggest advantage was shown using the naive Bayes model that after having transferred knowledge from another model, it achieved the highest percentage of classification accuracy increases.

Actions (login required)