ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
- ORNL
- Fujitsu Research America
- AMD
- AIST, Japan
Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 65,536 GPUs, achieving up to 4.1 ExaFLOPS sustained throughput and 74–98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with R2 scores in range of 0.98–0.99 against observation data.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 3007902
- Resource Relation:
- SC '25: The International Conference for High Performance Computing, Networking, Storage, and Analysis - St. Louis, Missouri, United States of America - 11/16/2025-11/21/2025
- Country of Publication:
- United States
- Language:
- English
Similar Records
ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling