Summary: Efficient Unsupervised Recursive Word Segmentation Using Minimum
Description Length
Shlomo ARGAMONa
Navot AKIVAb
Amihood AMIRb
Oren KAPAHb
a
Illinois Institute of Technology, Dept. of Computer Science, Chicago, IL 60616, USA
argamon@iit.edu
b
Bar-Ilan University, Dept. of Computer Science, Ramat Gan 52900, ISRAEL
{navot,amir,kapaho}@cs.biu.ac.il
Abstract
Automatic word segmentation is a basic re-
quirement for unsupervised learning in morpho-
logical analysis. In this paper, we formulate a
novel recursive method for minimum descrip-
tion length (MDL) word segmentation, whose
basic operation is resegmenting the corpus on
a prefix (equivalently, a suffix). We derive a

Source: Argamon, Shlomo - Department of Computer Science, Illinois Institute of Technology
Association for Computational Linguistics (ACL) Anthology

Collections: Computer Technologies and Information Sciences