MarFS, a Near-POSIX Interface to Cloud Objects

Journal Article · Sun Jan 01 00:00:00 EST 2017 · ;Login

OSTI ID:1351232

Inman, Jeffrey Thornton ^[1]; Vining, William Flynn ^[1]; Ransom, Garrett Wilson ^[1]; Grider, Gary Alan ^[1]

---

1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

The engineering forces driving development of “cloud” storage have produced resilient, cost-effective storage systems that can scale to 100s of petabytes, with good parallel access and bandwidth. These features would make a good match for the vast storage needs of High-Performance Computing datacenters, but cloud storage gains some of its capability from its use of HTTP-style Representational State Transfer (REST) semantics, whereas most large datacenters have legacy applications that rely on POSIX file-system semantics. MarFS is an open-source project at Los Alamos National Laboratory that allows us to present cloud-style object-storage as a scalable near-POSIX file system. We have also developed a new storage architecture to improve bandwidth and scalability beyond what’s available in commodity object stores, while retaining their resilience and economy. Additionally, we present a scheme for scaling the POSIX interface to allow billions of files in a single directory and trillions of files in total.

• Research Organization: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
• Sponsoring Organization: USDOE
• Grant/Contract Number: AC52-06NA25396; FC02-06ER25750
• OSTI ID: 1351232
• Report Number(s): LA-UR-16-28720; LA-UR-16-28952
• Journal Information: Login, Vol. 42, Issue 1; Related Information: Spring 2017 issue; ISSN 1044-6397
• Publisher: USENIX Association
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Computer Hardware
Computer Science
Information Science
MarFS
Campaign
scalability
storage
objects
POSIX