Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Conference Reviewing Considered Harmful Thomas Anderson

Summary: Conference Reviewing Considered Harmful
Thomas Anderson
Department of Computer Science & Engineering
University of Washington
This paper develops a model of computer systems research
to help prospective authors understand the often obscure work-
ings of conference program committees. We present data
to show that the variability between reviewers is often the
dominant factor as to whether a paper is accepted. We ar-
gue that paper merit is likely to be zipf distributed, making
it inherently difficult for program committees to distinguish
between most papers. We use game theory to show that with
noisy reviews and zipf merit, authors have an incentive to
submit papers too early and too often. These factors make
conference reviewing, and systems research as a whole, less
efficient and less effective. We describe some recent changes
in conference design to address these issues, and we suggest
some further potential improvements.


Source: Anderson, Tom - Department of Computer Science and Engineering, University of Washington at Seattle


Collections: Computer Technologies and Information Sciences