ePrints.FRI - University of Ljubljana, Faculty of Computer and Information Science

Reliability estimation of ensemble model predictions

Tomaž Kariž (2015) Reliability estimation of ensemble model predictions. MSc thesis.

Download (610Kb)


    In today's world, the reliability of a prediction is very important, especially in areas such as health and finance, where we do not want to make predictions that are not sufficiently reliable. To solve these problems in the context of machine learning, methods are being researched that assess the reliability of predictions. There are two types of methods: those specialized for a specific model and those who do not presume in advance the model type. The first may take into account additional information in determining the reliability, because they can use the parameters that are specific to the model as additional information. Others, however, are applicable to all models. In this work, we present some methods that operate on ensemble models, therefore, they are among those that are specific to a particular model. Methods operate on both the classification as well as regression datasets. Performance of methods is evaluated by Pearson correlation coefficient in the case of regression problems and Wilcoxon-Mann-Whitney statistics in the case of classification. The developed methods are compared with existing ones. We also show the results using critical distance diagrams.

    Item Type: Thesis (MSc thesis)
    Keywords: machine learning, reliability assessment, prediction reliability, ensemble models
    Number of Pages: 57
    Language of Content: Slovenian
    Mentor / Comentors:
    Name and SurnameIDFunction
    izr. prof. dr. Janez Demšar257Mentor
    Link to COBISS: http://www.cobiss.si/scripts/cobiss?command=search&base=51012&select=(ID=1536493251)
    Institution: University of Ljubljana
    Department: Faculty of Computer and Information Science
    Item ID: 3041
    Date Deposited: 03 Sep 2015 14:45
    Last Modified: 22 Sep 2015 11:08
    URI: http://eprints.fri.uni-lj.si/id/eprint/3041

    Actions (login required)

    View Item