ePrints.FRI - University of Ljubljana, Faculty of Computer and Information Science

Attribute Interactions in Machine Learning

Aleks Jakulin (2003) Attribute Interactions in Machine Learning. MSc thesis.

Download (1284Kb)
    [img] Postscript
    Download (466Kb)


      To make decisions, multiple data are used. It is preferred to decide on the basis of each datum separately, afterwards joining these decisions to take all data into consideration, for example by averaging. This approach is effective, but only correct when each datum is independent from all others. When this is not the case, there is an interaction between data. An interaction is true when there is synergism among the data: when the sum of all individual effects is smaller than the total effect. When the sum of individual effects is lower than the total effect, the interaction is false. The concept of an interaction is opposite to the concept of independence. An interaction is atomic and irreducible: it cannot be simplified or collapsed into a set of mutually independent simpler interactions. In this text we present a survey of interactions through a variety of fields, from game theory to machine learning. We propose a method of automatic search for interactions, and demonstrate that results of such analysis can be presented visually to a human analyst. We suggest that instead of special tests for interactions, a pragmatic test of quality improvement of a classifier is sufficient and preferable. Using the framework of probabilistic classifier learning, we investigate how awareness of interactions improves the classification performance of machine learning algorithms. We provide preliminary evidence that resolving true and false interactions improves classification results obtained with the naive Bayesian classifier, logistic regression, and support vector machines.

      Item Type: Thesis (MSc thesis)
      Keywords: interaction, dependence, dependency, independence assumption, constructive induction, feature construction, feature selection, attribute selection, myopic, information gain, naive Bayes, simple Bayes, naive Bayesian classifier, simple Bayesian classifier, information theory, entropy, relative entropy, mutual information
      Language of Content: English
      Related URLs:
      URLURL Type
      http://ai.fri.uni-lj.si/~aleks/Int/interactions_full.pdfAlternative location
      http://ai.fri.uni-lj.si/~aleks/Int/interactions_full.ps.gzAlternative location
      Link to COBISS: http://www.cobiss.si/scripts/cobiss?command=search&base=50070&select=(ID=3335508)
      Institution: University of Ljubljana
      Department: Faculty of Computer and Information Science
      Divisions: Faculty of Computer and Information Science > Artificial Intelligence Laboratory
      Item ID: 77
      Date Deposited: 28 Apr 2003
      Last Modified: 13 Aug 2011 00:32
      URI: http://eprints.fri.uni-lj.si/id/eprint/77

      Actions (login required)

      View Item