| | |
Summary: A Scalable Decoder for Parsing-based Machine Translation
with Equivalent Language Model State Maintenance
Zhifei Li and Sanjeev Khudanpur
Department of Computer Science and Center for Language and Speech Processing
Johns Hopkins University, Baltimore, MD 21218, USA
zhifei.work@gmail.com and khudanpur@jhu.edu
Abstract
We describe a scalable decoder for parsing-
based machine translation. The decoder is
written in JAVA and implements all the es-
sential algorithms described in Chiang (2007):
chart-parsing, m-gram language model inte-
gration, beam- and cube-pruning, and unique
k-best extraction. Additionally, parallel
and distributed computing techniques are ex-
ploited to make it scalable. We also propose
an algorithm to maintain equivalent language
model states that exploits the back-off prop-
erty of m-gram language models: instead of
maintaining a separate state for each distin-
|
