ePrints.FRI - University of Ljubljana, Faculty of Computer and Information Science

Widget for measuring classifier performance based on curves

Miha Biček (2009) Widget for measuring classifier performance based on curves. EngD thesis.

[img] PDF
Download (1374Kb)


    In machine learning we are often faced with a task of evaluating and comparing classifiers based on their performance. Alongside scalar measures various graphical methods are being used. Graphical methods such as Receiver Operating Characheristic Curve (ROC) or Precision-Recall (PR) are already well known and used in the machine learning community. This thesis explores the options of using alternative graphical methods to help us explore and compare classifiers. The first chapter describes two of most commonly used graphical methods along with the algorithm to produce the set of points for this methods. Later on it depicts examples of two less conventional methods that could be used along side ROC and PR curve. End of this chapter covers methods that help us estimate the measure of variance. The second chapter begins with a description of a generic algorithm that can be used to define methods to evaluate and compare classifiers. Afterwards it describes an implementation of this algorithm as a widget in a data mining software called Orange. It also describes the user interface of the widget and some guidelines how to use it. The third chapter gives some examples how the widget could be used. The fourth chapter is the conclusion of the thesis. It consists of assessing if the goals of the thesis were met and pointing out some flaws of the widget.

    Item Type: Thesis (EngD thesis)
    Keywords: data mining, ROC, classifier evaluation, clasification, visualization
    Number of Pages: 45
    Language of Content: Slovenian
    Mentor / Comentors:
    Name and SurnameIDFunction
    doc. dr. Janez Demšar257Mentor
    Link to COBISS: http://www.cobiss.si/scripts/cobiss?command=search&base=50070&select=(ID=7148884)
    Institution: University of Ljubljana
    Department: Faculty of Computer and Information Science
    Item ID: 861
    Date Deposited: 08 Jun 2009 09:01
    Last Modified: 13 Aug 2011 00:35
    URI: http://eprints.fri.uni-lj.si/id/eprint/861

    Actions (login required)

    View Item