Home

About

Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network
FAQHELPSITE MAPCONTACT US


  Advanced Search  

 
NOTE Communicated by Thomas Dietterich Combined 5 2 cv F Test for Comparing Supervised
 

Summary: NOTE Communicated by Thomas Dietterich
Combined 5 × 2 cv F Test for Comparing Supervised
Classification Learning Algorithms
Ethem Alpaydin
IDIAP, CP 592 CH-1920 Martigny, Switzerland
and
Department of Computer Engineering, Bogazi¸ci University, TR-80815 Istanbul, Turkey
Dietterich (1998) reviews five statistical tests and proposes the 5 × 2 cv t
test for determining whether there is a significant difference between the
error rates of two classifiers. In our experiments, we noticed that the 5 × 2
cv t test result may vary depending on factors that should not affect the
test, and we propose a variant, the combined 5×2 cv F test, that combines
multiple statistics to get a more robust test. Simulation results show that
this combined version of the test has lower type I error and higher power
than 5 × 2 cv proper.
1 Introduction
Given two learning algorithms and a training set, we want to test if the
two algorithms construct classifiers that have the same error rate on a test
example. The way we proceed is as follows: Given a labeled sample, we
divide it into a training set and a test set (or many such pairs), train the

  

Source: Alpaydın, Ethem - Department of Computer Engineering, Bogaziçi University

 

Collections: Computer Technologies and Information Sciences