Estimating the quality of arguments in argument-based machine learning

Matevž Pavlič (2015) Estimating the quality of arguments in argument-based machine learning. MSc thesis.

Preview

PDF Download (973Kb)

Abstract

Argument-based machine learning (ABML) knowledge refinement loop enables an interaction between a machine learning algorithm and a domain expert. It represents a powerful knowledge elicitation tool, suitable for obtaining expert knowledge in difficult domains. The loop enables the expert to focus on the most critical parts of the current knowledge base, and helps him or her to argue about automatically chosen relevant examples. The expert only needs to explain a single example at the time, which facilitates articulating arguments. It also helps the expert to improve the explanations by providing (automatically chosen) relevant counter examples. It has been shown recently that ABML knowledge refinement loop also enables design of argumentation-based interactive teaching tool. However, so far the machine was not able to provide neither the teachers (that designed such a tool) nor the students (that used it for learning) with concrete estimations about the quality of their arguments. In this thesis, we have designed three approaches for giving immediate feedback about the quality of arguments used in the ABML knowledge refinement loop. The chosen experimental domain was financial statement analysis, more concretely estimating credit scores of companies (enterprises). Our goal was twofold: to obtain a successful classification model for predicting the credit scores, and to enable the students to learn about this rather difficult domain. In the experimental sessions, both the teacher and the students were involved in the process of knowledge elicitation with the ABML knowledge refinement loop, receiving the feedback about their arguments. The goal of the learning session with the teacher was in particular to obtain advanced concepts (attributes) that describe the domain well, are suitable for teaching, and also enable successful predictions. This was done with the help of a financial expert. In the “tutoring" sessions, the students learned about the intricacies of the domain and strived for the best predictive model as possible, also by using the teacher's advanced concepts in their arguments. The main contributions of this work are: - the design of three approaches for estimating the quality of arguments used in the argument-based machine learning (ABML) knowledge refinement loop, - implementation of argumentation-based interactive teaching tool for estimating credit scores of companies (enterprises), using real data, - a detailed description of the learning session, where the student received three types of feedback about the arguments used.

Actions (login required)