Summary: A System for Large Vocabulary Sign Search
Haijing Wang 1 , Alexandra Stefan 1 , Sajjad Moradi 1 , Vassilis Athitsos 1 ,
Carol Neidle 2 , and Farhad Kamangar 1
1 Computer Science and Engineering Department, University of Texas at Arlington
Arlington, Texas 76019, USA
2 Linguistics Program, Boston University
Boston, Massachusetts 02215, USA
Abstract. A method is presented to help users look up the meaning of
an unknown sign from American Sign Language (ASL). The user sub­
mits a video of the unknown sign as a query, and the system retrieves
the most similar signs from a database of sign videos. The user then
reviews the retrieved videos to identify the video displaying the sign of
interest. Hands are detected in a semi­automatic way: the system per­
forms some hand detection and tracking, and the user has the option to
verify and correct the detected hand locations. Features are extracted
based on hand motion and hand appearance. Similarity between signs is
measured by combining dynamic time warping (DTW) scores, which are
based on hand motion, with a simple similarity measure based on hand
appearance. In user­independent experiments, with a system vocabulary
of 1,113 signs, the correct sign was included in the top 10 matches for

Source: Athitsos, Vassilis - Department of Computer Science and Engineering, University of Texas at Arlington

Collections: Computer Technologies and Information Sciences