Summary: Implementing I/OEfficient Data Structures Using TPIE
Lars Arge ? , Octavian Procopiuc ?? , and Jeffrey Scott Vitter ? ? ?
Center for Geometric and Biological Computing,
Department of Computer Science, Duke University, Durham, NC 27708, USA.
Abstract. In recent years, many theoretically I/Oefficient algorithms and data
structures have been developed. The TPIE project at Duke University was started
to investigate the practical importance of these theoretical results. The goal of
this ongoing project is to provide a portable, extensible, flexible, and easy to use
C++ programming environment for efficiently implementing I/Oalgorithms and
data structures. The TPIE library has been developed in two phases. The first
phase focused on supporting algorithms with a sequential I/O pattern, while the
recently developed second phase has focused on supporting online I/Oefficient
data structures, which exhibit a more random I/O pattern. This paper describes
the design and implementation of the second phase of TPIE.
In many modern massive dataset applications I/Ocommunication between fast internal
memory and slow disks, rather than actual internal computation time, is the bottle
neck in the computation. Examples of such applications can be found in a wide range
of domains such as scientific computing, geographic information systems, computer