TREC 2010 legal track: method and results of the ELK collaboration
- Los Alamos National Laboratory
- KAYVIUM
- EWA-IIT
The ELK team ([E]WA-IIT, [L]os Alamos National laboratory (LANL), and [K]ayvium Corporation (ELK)) used the legal Track task 302 as an opportunity to compare and integrate advanced semantic-automation strategies. The team members believe that enabling parties to discover, consume, analyze, and make decisions in a noisy and information-overloaded environment requires new tools. Together, as well as independently, they are actively developing these tools and view the TREC exercise as an opportunity to test, compare, and complement tools and approaches. Our collaboration is new to TREC, brought together by a shared interest in document relevance, concept-in-context identification and annotation, and the recognition that words out-of-context do not a match make. The team's intent was to lay the foundation for automating the mining and analysis of large volumes of electronic information by litigants and their lawyers, not only in the context of document discovery, but also to support litigation strategy, motion practice, deposition, trial tactics, etc. The premise was that a Subject Matter Expert- (SME-) built model can be automatically mapped onto various search engines for document retrieval, organization, relevance scoring, analysis and decision support. In the end, we ran nearly a dozen models, mostly, but not exclusively, with Kayvium Corporation's knowledge automation technology. The Sal Database Search Engine we used had a bug in its proximity feature, requiring that we develop a workaround. While the work-around was successful, it left us with insufficient time to converge the models to achieve expected quality. However, with optimized proximity processing in place, we would be able to run the model many more times, and believe repeatable quality would be a matter of working through a few requests to get the approach right. We believe that with more time, the results we would achieve might point towards a new way of processing documents for litigation support, so litigators can be confident that results are complete but not overly inclusive.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- OSTI ID:
- 1038891
- Report Number(s):
- LA-UR-10-07194; LA-UR-10-7194; TRN: US201209%%43
- Resource Relation:
- Conference: Text Retrieval Conference 2010 ; November 16, 2010 ; Gaithersburg, MD
- Country of Publication:
- United States
- Language:
- English
Similar Records
Demand Activated Manufacturing Architecture
Human Factors for Situation Assessment in Grid Operations