FV3: Finite-Volume Cubed-Sphere Dynamical Core
Performance
The National Weather Service has performed a technical evaluation of high-performance computing suitability and readiness of all candidate model dycores running on the NERSC Edison system, for the Next Generation Global Prediction System initiative. A detailed report can be found from the Advanced Computing Evaluation Committee.
Here, we highlight the FV3 performance and scaling tests for 13-km and 3-km simulations shown in the report. Based on Tables 3 and 4 in the report, we replot the charts with better labels to indicate the numerical precision (32-bit or 64-bit) used by each model.
The NGGPS test plan specified 8.5 minutes per forecast day as the operational forecast speed requirement. Assuming dynamics would comprise half the model run time, the speed requirement of the dycore is to run a 2-hour simulation in 21.25 seconds. The figure below shows model speed as a function of number of cores, normalized to the speed requirement of 8.5 min/day, for 13-km simulations.
For 3-km simulations, the performance is plotted as elapsed time to run a 30-minute simulation. FV3 with single (double) precision takes 265 (554) seconds when using 6411 processor cores, while scaling from 6144 to 110592 processor cores with 78 (88) percent efficiency.
GFDL is working with our partners at the Allen Institute for Artificial Intelligence, NASA Global Modeling and Assimilation Office, and the Swiss National Supercomputing Centre to port FV3 into the GT4py framework, allowing performance portability to new supercomputing architectures including GPUs. An initial port shows great promise for significant speedups on the new systems.