Abstract
DataMover-Lite (DML) is a client program whose purpose is to automatically manage multi-file transfers to clients. It expects as its input a transfer-instruction file, and a local target directory where the files will be moved to. The input file contains the Logical File Names (LFNs) and the transfer URLs (TURLs) of files that need to be transferred. The TURLs can have various protocols depending on the server that manages the files, including gridftp, ftp, https, and srm. Given a multi-file request for a large number of files, the DML can issue concurrently a number of transfers in order to get the files. The number of transfers it can issue is set as a parameter and is usually less than 10. For example, if the multi-file is for 100 files, the DML may keep active only 5 transfers at any one time. When any of the 5 transfers completes it will initiate another transfer. It is not necessary for a single request to contain files whose TURLs have same protocol. If the request to the DML contains TURLs with various protocols, the DML will contact the various servers on different threads according to each TURL's protocol. The DMK take care if
More>>
- Developers:
- Release Date:
- 2011-01-03
- Project Type:
- Open Source, No Publicly Available Repository
- Software Type:
- Scientific
- Programming Languages:
-
java 1.5.0{sub minus}x or above
- Licenses:
-
Other (Commercial or Open-Source): https://sdm.lbl.gov/dml/DMLLicense.html
- Sponsoring Org.:
-
USDOEPrimary Award/Contract Number:AC02-05CH11231
- Code ID:
- 54833
- Site Accession Number:
- 5175; CR-3004
- Research Org.:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
- Country of Origin:
- United States
Citation Formats
Natarajan, Vijaya, Sim, Alex, and Shoshani, Arie.
DataMover-Lite.
Computer Software.
USDOE.
03 Jan. 2011.
Web.
doi:10.11578/dc.20210416.53.
Natarajan, Vijaya, Sim, Alex, & Shoshani, Arie.
(2011, January 03).
DataMover-Lite.
[Computer software].
https://doi.org/10.11578/dc.20210416.53.
Natarajan, Vijaya, Sim, Alex, and Shoshani, Arie.
"DataMover-Lite." Computer software.
January 03, 2011.
https://doi.org/10.11578/dc.20210416.53.
@misc{
doecode_54833,
title = {DataMover-Lite},
author = {Natarajan, Vijaya and Sim, Alex and Shoshani, Arie},
abstractNote = {DataMover-Lite (DML) is a client program whose purpose is to automatically manage multi-file transfers to clients. It expects as its input a transfer-instruction file, and a local target directory where the files will be moved to. The input file contains the Logical File Names (LFNs) and the transfer URLs (TURLs) of files that need to be transferred. The TURLs can have various protocols depending on the server that manages the files, including gridftp, ftp, https, and srm. Given a multi-file request for a large number of files, the DML can issue concurrently a number of transfers in order to get the files. The number of transfers it can issue is set as a parameter and is usually less than 10. For example, if the multi-file is for 100 files, the DML may keep active only 5 transfers at any one time. When any of the 5 transfers completes it will initiate another transfer. It is not necessary for a single request to contain files whose TURLs have same protocol. If the request to the DML contains TURLs with various protocols, the DML will contact the various servers on different threads according to each TURL's protocol. The DMK take care if scheduling the transfers, monitoring their progress, releasing the files, and getting the authorization for each file to be transferred.},
doi = {10.11578/dc.20210416.53},
url = {https://doi.org/10.11578/dc.20210416.53},
howpublished = {[Computer Software] \url{https://doi.org/10.11578/dc.20210416.53}},
year = {2011},
month = {jan}
}