ROPE: Recoverable Order-Preserving Embedding of Natural Language
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
We present a novel Recoverable Order-Preserving Embedding (ROPE) of natural language. ROPE maps natural language passages from sparse concatenated one-hot representations to distributed vector representations of predetermined fixed length. We use Euclidean distance to return search results that are both grammatically and semantically similar. ROPE is based on a series of random projections of distributed word embeddings. We show that our technique typically forms a dictionary with sufficient incoherence such that sparse recovery of the original text is possible. We then show how our embedding allows for efficient and meaningful natural search and retrieval on Microsoft’s COCO dataset and the IMDB Movie Review dataset.
- Research Organization:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-07NA27344
- OSTI ID:
- 1239214
- Report Number(s):
- LLNL-TR-682663
- Country of Publication:
- United States
- Language:
- English
Similar Records
Computationally Efficient Learning of Quality Controlled Word Embeddings for Natural Language Processing
Left-corner unification-based natural language processing