Increasing the Interoperability of an Earth System Model:

Atmospheric-Ocean Dynamics and Tracer Transports

NCC4-624

Milestone K: Customer delivery accomplished.

Demonstrate ESMF functionality by analyzing the El Niño prediction capability of the AGCM coupled to POP and the MIT OGCM, in combination with NASA/JPL ocean data and optimization products. Complete User’s Guide/Maintenance manual. Documented source code made publicly available via the Web.

Carlos R. Mechoso⁽¹⁾, Gabriel Cazes Boezio^(1,3), Joseph A. Spahr⁽¹⁾, and Dimitris Menemenlis⁽²)

(1) Department of Atmospheric Sciences, University of California Los Angeles (UCLA)

(2) NASA/Caltech Jet Propulsion Laboratory (JPL), Pasadena, California

(3) On leave from IMFIA, Universidad de la Republica, Uruguay.

1. Introduction

Our work in the current round of NASA’s Computational Technologies (CT) Project addresses three major thrusts of the project: 1) to further our understanding of and ability to predict the dynamic interaction of physical and chemical processes affecting Earth, 2) to incorporate the use of NASA data and highlight its importance, and 3) to demonstrate interoperability of codes used in the community of Earth Science. We are following a three-tiered approach to accomplish our goals.

Tier I consists of upgrading coupled atmosphere-ocean part of the UCLA Earth System Model (ESM). This part comprises the UCLA atmospheric general circulation model (UCLA AGCM) and the LANL oceanic general circulation model (OGCM) also known as the Parallel Ocean Program (POP). The principal model science upgrade is in the formulation of processes in the planetary boundary layer of the atmosphere (PBL).

Tier II addresses the issues of code interoperability by using the ESMF services to couple the AGCM with either POP or the OGCM developed at the Massachusetts Institute of Technology (MIT), and by performing forecasts of El Niño/Southern Oscillation (ENSO).

Tier III focuses on the impact of NASA data and consists of comparing ENSO forecasts made from initial conditions corresponding to the quasi-operational analysis of the time-evolving ocean circulation produced by the consortium for Estimating the Circulation and Climate of the Ocean (ECCO). This analysis is maintained at JPL (http://ecco.jpl.nasa.gov), it is freely available, and it is being used for a variety of science applications (e. g., Lee and Fukumori 2003; Fukumori et al. 2004). The MIT OGCM is a component in ECCO’s data assimilation system, while POP is not.

2. The upgrade in PBL and entrainment processes in the AGCM

The parameterization of PBL processes in an AGCM provides fields that are of crucial importance in coupled air-sea interaction processes: 1) the exchanges of momentum, heat and mass between the atmosphere and the underlying surface, and 2) boundary layer cloudiness, which strongly influences the surface radiative fluxes and hence sea surface temperatures (SSTs) (Ma et al. 1996, Mechoso et al. 2000). As in other versions of the UCLA AGCM, the sigma-type vertical coordinate system defines a coordinate surface at the PBL-top (Suarez et al. 1983). This framework facilitates the explicit representation of processes concentrated near the PBL top, which is crucial for predicting PBL clouds. In the present work, the PBL parameterization predicts the layer’s bulk (vertically integrated) turbulent kinetic energy (TKE) (Randall and Schubert 2004). TKE is used in the computation of surface fluxes of moisture, sensible heat and momentum at the earth surface, and mass entrainment rate at the PBL top. The reader is referred to Konor et al. (2004) and Konor and Arakawa (2005) for details on the schemes and their implementation in the UCLA AGCM.

The surface fluxes of momentum, temperature and moisture are determined from an aerodynamic formula (Deardorff 1972) in which the velocity scale is determined by both the square root of the bulk TKE and the mean large-scale PBL velocity. This formulation is expected to provide better estimates of the surface fluxes than the traditional methods, since the mean wind can be weak while the convective mixing is strong. The fluxes of momentum, temperature and moisture are computed as follows:

(1)

where r_s is air density at the Earth’s surface, C_U and C_T are coefficients that depend on the bulk Richardson number, the PBL thickness and the surface roughness length, and are computed as in Deardorff (1972). u_M e_M, v_M, q_M and q_M are respectively the module of the velocity, TKE, vector velocity, potential temperature, and moisture of the PBL. q_G is potential temperature at the earth surface, and q_G is saturation moisture at the temperature and pressure of the earth surface. k is a coefficient representative of water availability of the terrain. This coefficient is one in water surfaces, and close to zero in arid terrains. a₁, a₂, b₁ and b₂ are empirical scale coefficients. Entrainment formulas (Randall and Schubert 2004) consider separately the cloud topped and cloud free cases. The expressions for entrainment that we are presently using are given by equations (8.12) and (8.14) in Konor and Arakawa (2005).

3. The MIT OGCM and POP

In our implementation, the MIT OGCM and POP have the same resolution and bathymetry. The models domain spans the latitudes 80S to 79N. In the zonal direction the resolution is 1 degree. In the meridional direction, the resolution is 0.3 degrees within 10 degrees of the equator, increasing to 1 degree outside the tropics. There are 46 levels in the vertical, with thicknesses ranging from 10 to 400 m down to a maximum depth of 5815 m. The integration time step is 1 hour. At the top, a free surface condition is applied. More details on the MIT OGCM and POP are given in Marshall et al. (1997) and Smith et al. (1992).

4. Results of the coupled GCM

Before the upgrade of the PBL parameterization the AGCM coupled to either of the OGCMs produced a severe climate drift at midlatitudes. Figure 1 shows the mean sea surface temperature (SST) distribution for December 1997 simulated by the UCLA AGCM with low resolution (5 longitude by 4 latitude) and upgraded PBL parameterization coupled to the quasi-global version of POP. The initial conditions corresponding to June 1, 1997 in the ECCO analysis. The results shown in Fig. 1 show that the coupled model produces very realistic results in integration of the length used for ENSO predictions. The performance of the quasi-global POP, therefore, satisfies the requirements for use in ENSO forecasts.

Figure 1. SST distribution for December by the UCLA AGCM coupled to POP (left) and Observation (right).

5. ENSO forecasts by the coupled atmosphere-ocean systems

The oceanic 3-D states produced by ECCO for July 5 of the years 1993 to 2002 were used to initialize the coupled system using either MIT or POP near global models. Figure 2 shows the SST anomalies for December-February (computed respect to the average of the 10 forecasts) obtained with the MIT global OGCM (left columns) and the respective observed SST anomalies (right column), for the years 10 years selected. The results obtained with the UCLA AGCM/MIT OGCM combination are very encouraging with respect to forecast skill of this coupled model and initialization analysis. The forecasts capture the strong ENSO episodes that occurred in 1997, 1998 and 1999. In most cases, the results reproduce details of the SST anomaly pattern (e.g., 1995, 1996, 1997, 1998 and 2001). Figure 3 shows similar forecast produced with the POP OGCM. Forecasted anomalies are significantly weaker and less realistic than those with MIT OGCM. A possible reason is that the version of MIT OGCM used is in a more advanced state of its implementation than the version used for POP OGCM. In addition, the MIT OGCM went through careful tunning process in uncoupled (Menemenlis et al. 2004) and coupled simulations (Cazes Boezio et al. 2005).

6. Integration into the ESMF

The ESMF is a structured collection of software building blocks to assist in the development of model components, and assemble them into an ESM (www.esmf.ucar.edu). We have integrated into the ESMF the UCLA AGCM, POP, and the MIT OGCM. These are designated as “gridded components”, while the exchange of information is performed by the “coupling components”. The integration process (see Fig. 2) started by designing an ESM Driver Program (EDP) in order to control and define the ESMF environment and execution sequence of components. The EDP consists of five routines, three of which are drivers of the gridded components while the two remainders are drivers of the coupling components (Atmosphere to Ocean coupling, and Ocean to Atmosphere coupling). The transfer of execution between EDP routines is done with an ESMF entry point registration and an ESMF call to branch to the appropriate routine. Entry points are registered for each of the drivers and their initialize, run and finalize methods.

Figure 2. SST anomalies for December-February SST of 1993 to 1997 and of 1998 to 2002 (Fig. 3 continuation), predicted by the UCLA AGCM coupled to the MIT OGCM from early June initial conditions provided by ECCO analysis (left column), and the corresponding observed SST anomalies (right column). Contour interval is 1 K; +/- 0.5 intervals are also shown.

Figure 2 (continuation).

Figure 3. Same as Fig. 3 except predicted by the UCLA AGCM coupled to the POP OGCM.

Figure 3. continuation.

In the EDP, the ESM driver subroutine initializes the ESMF and specifies the coupled system, i.e. the gridded components to be used and the execution sequence. The initialize method creates and registers each of the gridded and coupling components, and the import/export states. The run method has the time integration loop for the application and transfers control, using ESMF calls, to other components in the appropriate sequence. The finalize method terminates the execution of components. In the AGCM initialization, the model defines constants, decomposition/layout, geometry, and communications, and obtains the initial and boundary conditions and executes the first physics time step. In the initialization of an OGCM, the model selects the initial and boundary conditions. After initialization is completed, the gridded components advance the simulation time until the next instance of coupling. The variables to be exchanged are inserted into the export state so they are ready to be transferred when the coupling driver is called. The finalize method termination routines close files, write restarts and print out statistics. The coupling drivers routines transfer variables between the import to export state by utilizing the list of symbolic variable names that was generated during the initialization. They also take care of any necessary regridding.

Figure 4. Superstructure of the coupled model integrated into the ESMF

Extensive work was performed with the UCLA AGCM itself to make it ESMF compliant, as is described in Milestone J. The model’s own control code and coupling routines were removed, since these functions are now performed by the EDP routines. To conform to ESMF methodology, entry points were defined in the model to allow for discrete calls to the initialize, run, and finalize methods. The coupling interface, which was based on the methodology of concurrent execution, was rewritten due to limitations of the ESMF version available at the time when the work was started (2.1.0) and that only allowed for sequential execution. (More recent ESMF versions allow for concurrent execution.) In addition, routines were created so the EDP can extract grid, geometry and variable information from the model. The UCLA AGCM was tested using the EDP. No functional or performance degradation was found. The code can still run stand-alone (without ESMF) by setting a flag. Chris Hill and Phil Jones modified the MIT OGCM and POP, respectively, to be ESMF compliant.

7. ENSO forecasts using the ESMF

The forecasts presented in section 5 of this paper were performed with model versions that are not integrated in the ESMF. In order to demonstrate that the results are reproducible with the model versions integrated in the ESMF, we repeat the case of the strong El Nino event of 1997. Figure 5 shows the observed SST anomalies for December-February (top) and the corresponding predictions (computed respect to the average of the 10 forecasts) obtained with the MIT global OGCM (middle panel) and the POP OGCM.

We note that the OGCM versions integrated in the ESMF that are available correspond to more recent versions than those used to obtain the results shown in Fgs. 2 and 3. The agreement between the appropriate panels in Fgs. 2, 3 and 5 show that the ESMF has not affected the results.

To provide information on the computational overhead refined by the ESMF we computed the CPU time required by this application and compared it with the times required by both our Distributed and Conventional Data Brokers. For 16 processors, these timings are 0.1014s, 0.1013s, and 0.1001s.

8. Summary

Under our NASA Computational Technologies Project we have achieved three important goals. First, the PBL parameterization used in the UCLA AGCM was upgraded. The upgrade improved the simulations in several respects, particularly in the surface fluxes that play key roles in the coupling between atmosphere and ocean. Second, the UCLA AGCM code was integrated in the ESMF and framework services were used for its coupling to quasi-global versions of POP and MIT OGCM. Third, ENSO predictions were made with the coupled system. A preliminary assessment of the ENSO forecast skill of this model combined with initial conditions provided by ECCO analysis has provided very encouraging results.

9. Deliverables and documentation

Two tar files have been created that contain the two coupled models used in this milestone. http://esm-a.atmos.ucla.edu/~esm/UCLA_POP.tar.gz tar file contains the coupled UCLA AGCM and LANL POP model. The UCLA AGCM coupled with MITogcm can be retrieved from http://esm-a.atmos.ucla.edu/~esm/UCLA_MIT.tar.gz. The user guide for the UCLA AGCM that has been integrated with the ESMF can be found in http://www.atmos.ucla.edu/~mechoso/esm/agcmug.html.

For the LANL POP and the MITogcm access their web sites directly for their documentation, http://climate.acl.lanl.gov/models/pop/documentation and http://mitgcm.org/docs.html.

For additional information of the UCLA AGCM coupled system contact Joseph Spahr at (spahr@atmos.ucla.edu, 310-825-1555), Phil Jones (pwjones@lanl.gov, 505-667-6386) for the LANL POP OGCM and Chris Hill (cnh@mit.edu, 617-253-6430) for the MITogcm.

Acknowledgements: Dr. C. S. Konor and Professor A. Arakawa lead the implementation of the PBL parameterization used in the current version of the UCLA AGCM.

References

Deardorff, J. W., 1972: Parameterization of the planetary boundary layer use in general circulation models. Mon. Wea. Rev, 93-106. Deardorff, J. W., 1976: On the entrainment rate of a stratocumulus-capped mixed layer. Quart. J. Roy. Meteor. Soc., 102, 563-582.

Fukumori, I., T. Lee, B. Cheng, and D. Menemenlis, 2004: The origins, pathway, and destination of Niño-3 water estimated by a simulated passive tracer and its adjoint. J. Phys. Oceanogr., 34(3), 582–604.

Lee, T., and I. Fukumori, 2003: Interannual-to-decadal variations of tropical-subtropical exchange in the Pacific Ocean: Boundary versus interior pycnocline transports. J. Climate, 16(24), 4022–4042.

Konor C. S., G. Cazes-Boezio, C. R. Mechoso and A. Arakawa, (2004): Evaluation of a new PBL parameterization with emphasis in surface fluxes, Proceedings of the 13^th AMS Conference on Air-Sea interactions, Portland, Maine.

Cazes-Boezio G., Konor C. S., C. R. Mechoso, Menemenlis D. and A. Arakawa, (2005): Coupled simulations obtained with a the UCLA AGCM with a new PBL Parameterization and the MIT global OGCM. 17^th AMS Conference on Climate Variability and Change, Cambridge, Massachusetts.

Konor C. S. and A. Arakawa, 2005: Incorporation of moist processes and a PBL parameterization into the generalized vertical coordinate model. Technical Report No. 102, Department of Atmospheric Sciences, University of California, Los Angeles, 405 Hilgard Ave. 90035, Los Angeles, CA, USA. Available at www.atmos.ucla.edu/~csk.

Ma, C. -C., C. R. Mechoso, A. W. Robertson, and A. Arakawa, 1996: Peruvian Stratus Clouds and the Tropical Pacific Circulation: A Coupled Ocean-Atmosphere GCM Study, J. Climate, 9, 1635-1645.

Marshall, J., A. Adcroft, C. Hill, L. Perelman, and C. Heisey, 1997: A finite-volume, incompressible Navier-Stokes model for studies of the ocean on parallel computers. J. Geophys. Res., 102(C3), 5753–5766.

Mechoso, C.R., J-Y. Yu and A. Arakawa, 2000: A coupled GCM pilgrimage: from climate catastrophe to ENSO simulations. General Circulation Model Development: Past, Present and Future. Proceedings of a Symposium in Honor of Professor Akio Arakawa, D. A. Randall, Ed., Academic Press, pp. 539-575.

Menemenlis D., I. Fukumori and T. Lee, 2004: Using Green functions to calibrate an ocean general circulation model, Mon. Weath. Rev., in press.

Randall D. A. and W. H. Schubert, 2004: Dreams of a stratocumulus sleeper. In Atmospheric Turbulence and Mesoscale Meteorology, Scientific Research inspired by Douglas Lilly, E. Fedorovich, R. Rotuno and B. Stevens (eds.), Cambridge University Press.

Smith, R. D., J. K. Dukowicz, and R. C. Malone, 1992: Parallel ocean general circulation modeling, Physica D, 60, 38-61.

Suarez, M. J., A. Arakawa and D. A. Randall, 1983: The parameterization of the planetary boundary layer in the UCLA general circulation model: Formulation and results. Mon. Wea. Rev., 111, 2224-2243