Abstract
The DeepLynx DAG repository will contain several Airflow DAGs (Directed Acyclic Graphs) which will be used in the context of DeepLynx's deployed Apache Airflow instance. These DAGs will be used for multiple data management tasks for DeepLynx data, including but not limited to:
- bringing data from various sources and tools into DeepLynx
- managing sequential data workflows, such as running Python scripts on data to perform analysis and returning the results to DeepLynx
- performing any necessary transformation or pre-processing on data coming into DeepLynx from external sources or out of DeepLynx to go to external applications
- Developers:
-
Brownlee, Jaren [1] ; Cavaluzzi, Jack [1] ;
[1]
;
[1]
;
Zohner, Porter
[1]
;
Hergesheimer, Natalie
[1]
;
Swinford, Jake
[1]
- Idaho National Laboratory (INL), Idaho Falls, ID (United States)
- Release Date:
- 2024-09-26
- Project Type:
- Open Source, Publicly Available Repository
- Software Type:
- Scientific
- Programming Languages:
-
Python
- Licenses:
-
MIT License
- Sponsoring Org.:
-
USDOE Office of Nuclear Energy (NE)Primary Award/Contract Number:AC07-05ID14517
- Code ID:
- 145281
- Research Org.:
- Idaho National Laboratory (INL), Idaho Falls, ID (United States)
- Country of Origin:
- United States
- Keywords:
- DeepLynx; Digital Twin; Workflow; ETL; Extract Transform Load; Digital Engineering; Digital Thread; Airflow
Citation Formats
Brownlee, Jaren M., Cavaluzzi, Jack M., Browning, Jeren M., Houck, Edward K., Zohner, Porter J., Hergesheimer, Natalie A., and Swinford, Jake O.
Deeplynx Dag Repository.
Computer Software.
https://github.com/idaholab/DLDAGS.
USDOE Office of Nuclear Energy (NE).
26 Sep. 2024.
Web.
doi:10.11578/dc.20241007.2.
Brownlee, Jaren M., Cavaluzzi, Jack M., Browning, Jeren M., Houck, Edward K., Zohner, Porter J., Hergesheimer, Natalie A., & Swinford, Jake O.
(2024, September 26).
Deeplynx Dag Repository.
[Computer software].
https://github.com/idaholab/DLDAGS.
https://doi.org/10.11578/dc.20241007.2.
Brownlee, Jaren M., Cavaluzzi, Jack M., Browning, Jeren M., Houck, Edward K., Zohner, Porter J., Hergesheimer, Natalie A., and Swinford, Jake O.
"Deeplynx Dag Repository." Computer software.
September 26, 2024.
https://github.com/idaholab/DLDAGS.
https://doi.org/10.11578/dc.20241007.2.
@misc{
doecode_145281,
title = {Deeplynx Dag Repository},
author = {Brownlee, Jaren M. and Cavaluzzi, Jack M. and Browning, Jeren M. and Houck, Edward K. and Zohner, Porter J. and Hergesheimer, Natalie A. and Swinford, Jake O.},
abstractNote = {The DeepLynx DAG repository will contain several Airflow DAGs (Directed Acyclic Graphs) which will be used in the context of DeepLynx's deployed Apache Airflow instance. These DAGs will be used for multiple data management tasks for DeepLynx data, including but not limited to:
- bringing data from various sources and tools into DeepLynx
- managing sequential data workflows, such as running Python scripts on data to perform analysis and returning the results to DeepLynx
- performing any necessary transformation or pre-processing on data coming into DeepLynx from external sources or out of DeepLynx to go to external applications},
doi = {10.11578/dc.20241007.2},
url = {https://doi.org/10.11578/dc.20241007.2},
howpublished = {[Computer Software] \url{https://doi.org/10.11578/dc.20241007.2}},
year = {2024},
month = {sep}
}