A Scalable Implementation of a Finite-Volume Dynamical Core in the Community Atmosphere Model

Abstract
We present a distributed memory message passing parallel implementation of a finite-volume discretization of the primitive equations in the Community Atmosphere Model. Due to the data dependences resulting from the polar singularity of the latitude-longitude coordinate system, we employ two separate domain decompositions within the dynamical core: one in latitude/level space and the other in longitude/latitude space. This requires that the data be periodically redistributed between these two decompositions. In addition, the domains contain halo regions that cover the nearest-neighbor data dependences. A combination of several techniques, such as one-sided communication and multithreading, are presented to optimize data movements. The resulting algorithm is shown to scale to very large machine configurations, even for relatively coarse resolutions.