Title: An Evaluation of the Large‐Scale Atmospheric Circulation and Its Variability in CESM2 and Other CMIP Models
Abstract: Journal of Geophysical Research: AtmospheresVolume 125, Issue 13 e2020JD032835 Research ArticleOpen Access An Evaluation of the Large-Scale Atmospheric Circulation and Its Variability in CESM2 and Other CMIP Models Isla R. Simpson, Corresponding Author Isla R. Simpson [email protected] orcid.org/0000-0002-2915-1377 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USA Correspondence to: I. R. Simpson, [email protected] for more papers by this authorJulio Bacmeister, Julio Bacmeister orcid.org/0000-0001-8848-975X Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorRichard B. Neale, Richard B. Neale orcid.org/0000-0003-4222-3918 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorCecile Hannay, Cecile Hannay orcid.org/0000-0001-6363-6151 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorAndrew Gettelman, Andrew Gettelman orcid.org/0000-0002-8284-2599 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorRolando R. Garcia, Rolando R. Garcia orcid.org/0000-0002-6963-4592 Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorPeter H. Lauritzen, Peter H. Lauritzen orcid.org/0000-0001-7066-138X Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorDaniel R. Marsh, Daniel R. Marsh orcid.org/0000-0001-6699-494X Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorMichael J. Mills, Michael J. Mills orcid.org/0000-0002-8054-1346 Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorBrian Medeiros, Brian Medeiros orcid.org/0000-0003-2188-4784 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorJadwiga H. Richter, Jadwiga H. Richter orcid.org/0000-0001-7048-0781 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this author Isla R. Simpson, Corresponding Author Isla R. Simpson [email protected] orcid.org/0000-0002-2915-1377 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USA Correspondence to: I. R. Simpson, [email protected] for more papers by this authorJulio Bacmeister, Julio Bacmeister orcid.org/0000-0001-8848-975X Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorRichard B. Neale, Richard B. Neale orcid.org/0000-0003-4222-3918 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorCecile Hannay, Cecile Hannay orcid.org/0000-0001-6363-6151 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorAndrew Gettelman, Andrew Gettelman orcid.org/0000-0002-8284-2599 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorRolando R. Garcia, Rolando R. Garcia orcid.org/0000-0002-6963-4592 Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorPeter H. Lauritzen, Peter H. Lauritzen orcid.org/0000-0001-7066-138X Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorDaniel R. Marsh, Daniel R. Marsh orcid.org/0000-0001-6699-494X Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorMichael J. Mills, Michael J. Mills orcid.org/0000-0002-8054-1346 Atmospheric Chemistry Observations and Modelling Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorBrian Medeiros, Brian Medeiros orcid.org/0000-0003-2188-4784 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this authorJadwiga H. Richter, Jadwiga H. Richter orcid.org/0000-0001-7048-0781 Climate and Global Dynamics Laboratory, National Center for Atmospheric Research, Boulder, CO, USASearch for more papers by this author First published: 02 June 2020 https://doi.org/10.1029/2020JD032835Citations: 13AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract The Community Earth System Model 2 (CESM2) is the latest Earth System Model developed by the National Center for Atmospheric Research in collaboration with the university community and is significantly advanced in most components compared to its predecessor (CESM1). Here, CESM2's representation of the large-scale atmospheric circulation and its variability is assessed. Further context is providedthrough comparison to the CESM1 large ensemble and other models from the Coupled Model Intercomparison Project (CMIP5 and CMIP6). This includes an assessment of the representation of jet streams and storm tracks, stationary waves, the global divergent circulation, the annular modes, the North Atlantic Oscillation, and blocking. Compared to CESM1, CESM2 is substantially improved in the representation of the storm tracks, Northern Hemisphere (NH) stationary waves, NH winter blocking and the global divergent circulation. It ranks within the top 10% of CMIP class models in many of these features. Some features of the Southern Hemisphere (SH) circulation have degraded, such as the SH jet strength, stationary waves, and blocking, although the SH jet stream is placed at approximately the correct location. This analysis also highlights systematic deficiencies in these features across the new CMIP6 archive, such as the continued tendency for the SH jet stream to be placed too far equatorward, the North Atlantic westerlies to be too strong over Europe, the storm tracks as measured by low-level meridional wind variance to be too weak and a lack of blocking in the North Atlantic sector. 1 Introduction The Community Earth System Model, Version 2 (CESM2), is the second generation Earth System Model developed by the U.S.'s National Center for Atmospheric Research (NCAR), in collaboration with university researchers (Hurrell et al., 2013). Prior to the first incarnation of CESM (CESM1), the history of development of this model can be traced through the Community Climate System Model, Versions 4 (CCSM4, Gent et al., 2011), 3 (CCSM3, Collins et al., 2006), 2 (CCSM2 Kiehl & Gent, 2004), the Climate System Model 1 (CSM1 Boville & Gent, 1998), and, before that, the Community Climate Model, versions 3 (CCM3 Kiehl et al., 1998), 2 (CCM2 Hack et al., 1993), 1 (CCM1 Williamson et al., 1987), and 0 (CCM0 Washington, 1982; Williamson, 1983). As such, CESM2 represents the current state-of-the-art in Earth System Modelling from this center, incorporating model development contributions from over four decades of research and the efforts of countless individuals. Over this development history, the array of complex atmospheric, oceanic, hydrologic, cryospheric, and biogeophysical processes represented by this model has made CESM2 one of the most comprehensive and complex Earth System Models (ESMs) available. Given its fundamental role in the Earth System, the large-scale atmospheric circulation has been represented with some realism, relatively speaking, since the earliest days of climate modeling. Nevertheless, persistent biases remain in certain aspects and, as our models increase in complexity, we must continue to strive for the greatest accuracy possible in the representation of this underpinning feature of the Earth System. In this study we present an evaluation of basic features of the large-scale atmospheric circulation and its variability in CESM2. We provide context by assessing changes compared to its predecessor (CESM1) and by placing it within the wider distribution of Earth System Models as represented by those participating in the Coupled Model Intercomparison Project, Phases 5 and 6 (CMIP5 Taylor et al., 2012 and CMIP6 Eyring et al., 2016). The range of atmospheric circulation features presented here is not exhaustive, and the primary focus is on the global climatology of the divergent circulation and stationary waves, midlatitude jet streams and storm tracks, and aspects of extratropical variability. Separate studies in this special issue provide an assessment of tropical intraseasonal variability, monsoons (Meehl et al., 2020), and El Niño–Southern Oscillation (ENSO) variability and its teleconnections (Capotondi et al., 2020). Rather than taking the traditional approach of providing an overall introduction, methodological description and summary of the results, we instead provide a self-contained introduction and methodology within each results section for each feature considered, such that a reader can easily find all the relevant information in one place for their feature of interest. This diagnostic analysis is intended primarily as a resource for CESM2 users but also serves as a concise summary of the representation of these features in CMIP5 and CMIP6 models. We begin by describing the model simulations and observational data sets in section 2, followed by a description of the error metrics used and the uncertainty assessments performed in section 3. In section 4we discuss the representation of jet streams and storm tracks, in section 5 we discuss stationary waves and the global divergent circulation and in section 6 we assess the annular modes, North Atlantic Oscillation (NAO) and blocking. Summary and conclusions are provided in section 7. 2 Model Simulations and Observation-Based Data Sets For each of the model historical simulations and reanalyses described below, our primary focus is on the period from 1979 to 2014 and on monthly and daily averaged fields of zonal wind (ua), meridional wind (va), geopotential height (zg), and sea level pressure (slp). Note that here we are using variable names as specified by CMIP as opposed those used in CESM2. Each of these fields is first regridded to a common 2° horizontal grid using bilinear interpolation before any other fields or metrics are derived. Only the summer and winter seasons are considered in the main text, but equivalent figures are shown for the spring and autumn in the supporting information. 2.1 CESM2 In its default configuration, CESM2 simulates the global coupled Earth System at approximately 1° horizontal resolution. It contains interactive components for the atmosphere, land, ocean, sea ice, river transport, and land ice. CESM2 represents a significant advance over CESM1 in many ways (see Danabasoglu et al., 2019 for more details). As the updates within the atmosphere component (Community Atmosphere Model 6, CAM6) are likely to be the most relevant, we summarize some of those major changes here, but readers are referred to Bogenschutz et al. (2018) and Gettelman et al. (2019) for a more detailed description of CAM6 and the high-top atmospheric component (Whole Atmosphere Community Climate Model, WACCM6), respectively. In the transition from CAM5 to CAM6, almost every physical parameterization within the atmosphere has been updated, with the exception of radiation. A major change is that the boundary layer, shallow convection, and cloud macrophysics are combined within the new Cloud Layers Unified By Binormals (CLUBB) scheme (Golaz et al., 2002), resulting in a more consistent representation of boundary layer turbulence (Bogenschutz et al., 2013). The prognostic cloud microphysics scheme (MG2, Gettelman & Morrison, 2015) has been updated from its predecessor (MG1) with a major change being the inclusion of prognostic precipitation. Finally, and of relevance to some of the following results, there have been major updates to the representation of orographic drag. The orographic gravity wave drag scheme now includes a representation of the orientation of subgrid orography (ridges) and the effects of mesoscale orographic blocking (MOB). Furthermore, the turbulent orographic form drag (TOFD) scheme has been updated from the Turbulent Mountain Stress (TMS, Richter et al., 2010) parameterization to that of Beljaars et al. (2004). Our primary focus will be on four CESM2 historical ensembles that differ in the vertical extent of the atmospheric component and in the presence or absence of coupling to the fully dynamic ocean model. These ensembles are summarized in the lower left of Table 1 and a more detailed description is provided in Table 2. Eleven members make up the ensemble BCAM6 in which the low-top atmosphere model (CAM6), with 32 layers in the vertical extending to ∼40 km, is coupled to the ocean model. A three-member coupled ensemble with the high-top WACCM6 (Gettelman et al., 2019), which has 70 levels in the vertical extending to ∼130 km, will be referred to as BWACCM6. Spatial maps in the main text are only shown for BCAM6, but equivalent figures for BWACCM6 are shown in supporting information Figures S16 and S17. In addition, there are three member ensembles, with prescribed historical sea surface temperatures (SSTs) (Hurrell et al., 2008), referred to as FCAM6 and FWACCM6, for CAM6 and WACCM6, respectively. In this naming convention, B refers to the CESM B-component set, which includes coupling to the ocean model, while F refers to the CESM F-component set where SSTs and sea ice are prescribed (i.e., AMIP-type simulations). Table 1. Summary of Simulations Used CMIP5 CMIP6 # Name Members # Name Members 1 ACCESS1-0 1 1 ACCESS-CM2*+ 2 2 ACCESS1-3 1 2 ACCESS-ESM1-5* 3 3 bcc-csm1-1 1 3 AWI-CM-1-1-MR 5 4 bcc-csm1-1-m 1 4 BCC-CSM2-MR*+ 3 5 BNU-ESM*+ 1 5 BCC-ESM1*+ 3 6 CanESM2*+ 5 6 CAMS-CSM2-0 1 7 CCSM4*+ 6 7 CanESM5*+ 25 8 CESM1-CAM5 3 8 CNRM-CM6-1*+ 15 9 CESM1-WACCM 1 9 CNRM-CM6-1-HR*+ 1 10 CMCC-CM 1 10 CNRM-ESM2-1*+ 5 11 CMCC-CMS 1 11 E3SM-1-0 5 12 CNRM-CM5*+ 5 12 E3SM-1-1 1 13 CSIRO-Mk3-6-0* 10 13 EC-Earth3*+ 10 14 FGOALS-g2 1 14 EC-Earth3-Veg* 4 15 FIO-ESM 1 15 FGOALS-f3-L 3 16 GFDL-CM3*+ 1 16 FGOALS-g3 3 17 GFDL-ESM2G*+ 1 17 FIO-ESM-2-0 3 18 GFDL-ESM2M*+ 1 18 GFDL-CM4*+ 1 19 GISS-E2-H 1 19 GFDL-ESM4 1 20 GISS-E2-R 1 20 GISS-E2-1-G*+ 10 21 HadGEM2-AO 1 21 GISS-E2-1-G-CC 1 22 HadGEM2-CC 1 22 GISS-E2-1-H 10 23 HadGEM2-ES 1 23 HadGEM3-GC31-LL*+ 4 24 inmcm4 1 24 HadGEM3-GC31-MM*+ 2 25 IPSL-CM5A-LR*+ 4 25 INM-CM4-8* 1 26 IPSL-CM5A-MR 1 26 INM-CM5-0* 10 27 IPSL-CM5B-LR 1 27 IPSL-CM6A-LR*+ 32 28 MIROC5*+ 3 28 KACE-1-0-G 3 29 MIROC-ESM*+ 1 29 MCM-UA-1-0 1 30 MIROC-ESM-CHEM*+ 1 30 MIROC6*+ 10 31 MPI-ESM-LR*+ 3 31 MIROC-ES2L* 3 32 MPI-ESM-MR*+ 1 32 MPI-ESM-1-2-HAM*+ 2 33 MRI-CGCM3*+ 1 33 MPI-ESM1-2-HR*+ 10 34 NorESM1-M*+ 1 34 MPI-ESM1-2-LR*+ 10 35 NorESM1-ME 1 35 MRI-ESM2-0*+ 5 CESM1 36 NESM3 5 LENS 40 37 NorCPM1 30 CESM2 38 NorESM2-LM*+ 3 BCAM6 11 39 NorESM2-MM* 1 BWACCM6 3 40 SAM0-UNICON* 1 FCAM6 3 41 TaiESM1 1 FWACCM6 3 42 UKESM1-0-LL*+ 4 Note. The top portion of the left three columns depict the model number (used to depict the model in each plot), model name, and number of members of each CMIP5 model. The right-hand columns show the same for CMIP6. An “*” at the end of the CMIP model name depicts whether that model is used in the analyses requiring daily ua or va, and a “+” depicts whether a model is used in analysis requiring daily zg. The lower portion of the left columns summarizes the CESM1 and CESM2 simulations. The period from 1979 to 2014 is used for all simulations. Table 2. Summary of CESM2 Simulations Name Res (lon×lat) # levels/lid p Description BCAM6 1.9° × 2.5° 32/2.26 hPa historical, coupled ocean BWACCM6 1.9° × 2.5° 70/4.5e−6 hPa historical, coupled ocean FCAM6 1.9° × 2.5° 32/2.26 hPa historical, prescribed observed SSTs (Hurrell et al., 2008) FWACCM6 1.9° × 2.5° 70/4.5e−6 hPa historical, prescribed observed SSTs (Hurrell et al., 2008) FCAM6MOD 1.9° × 2.5° 32/2.26 hPa historical, prescribed SSTs from member 11 of BCAM6 FCAM6* 1.9° × 2.5° 32/2.26 hPa as FCAM6 but with land biogeochemistry turned off FCAM5 1.9° × 2.5° 32/2.26 hPa as FCAM6* but with CAM5 physics FCAM6_TMS 1.9° × 2.5° 32/2.26 hPa as FCAM6* but with the Beljaars scheme replaced by TMS FCAM6_NOMOB 1.9° × 2.5° 32/2.26 hPa as FCAM6* but with mesoscale orographic blocking turned off Note. From left to right: simulation name; approximate horizontal resolution in degrees longitude × latitude format; number of vertical (mid)levels and the model lid pressure; and a description of external forcings and the form of the SSTs or parameterization changes. For FCAM6*, FCAM5, FCAM6_TMS, and FCAM6_NOMOB, we use 1979–2005, while for the other simulations, we use 1979–2014. These simulations are run under historical forcings (Hoesly et al., 2018; van Marle et al., 2017) until 2014. The coupled simulations are each initialized from different years from a preindustrial (i.e., perpetual year 1850 forcing) control that has been spun-up for over 1,000 years (Danabasoglu et al., 2019), while the prescribed SST simulations begin in 1950. For each ensemble we will only consider the period from 1979 to 2014 for comparison with modern reanalyses over the satellite era. In addition to these four ensembles of simulations which are contributed to the CMIP6 archive, we will make use of the following simulations that are designed to isolate the underlying cause of some of the changes found in CESM2. FCAM6MOD is an historical simulation with CAM6 with prescribed SSTs but with the SSTs taken from one of the coupled BCAM6 members, as opposed to observations. We will use 1979–2014 of this simulation to explore the role of SST differences versus the lack of coupling in explaining differences between BCAM6 and FCAM6. To explore the influence of changes in orographic parameterizations schemes, four single-member experiments, under historical forcings from 1979–2005, with SSTs prescribed to observations will be considered. FCAM6* is an uncoupled simulation with prescribed observed SSTs, very similar to FCAM6 described above, but with biogeochemistry in the land turned off. While the issue of land biogeochemistry is not important for our purposes, we use this rather than FCAM6 for like-with-like comparison with each of the following experiments that also have biogeochemistry turned off. FCAM5 is a simulation performed in the same way as FCAM6* (same forcings, boundary conditions, and land model) but with CAM5 physics used instead of CAM6. This allows for an assessment of the overall influence of the atmospheric physics package in isolation, which can then be compared with the following two experiments to isolate the orographic influence. FCAM6_TMS is as FCAM6* but with the new TOFD scheme of Beljaars et al. (2004) replaced by the older Turbulent Mountain Stress (TMS) parameterization of CAM5. A comparison of FCAM6* with FCAM6_TMS demonstrates the influence of this change in TOFD. FCAM6_NOMOB is as FCAM6* but without the new MOB parameterization included. A comparison between FCAM6* and FCAM6_NOMOB indicates the influence of the new MOB scheme. 2.2 CESM1 To examine the changes that have arisen as a result of the developments in advancing from CESM1 to CESM2 and to provide an indication of the sampling uncertainty in each metric as a result of internal variability, we will compare with the CESM1 large ensemble (Kay et al., 2014). This 40-member ensemble of simulations is initialized in 1920 from a single state, with ensemble spread introduced through a round-off level noise perturbation added to the temperature field at initialization. The initial state is that of a single realization that was branched from an 1850s control simulation and run until 1920 under historical forcings (Lamarque et al., 2010). The 40-member ensemble is then run under historical forcings to 2005 and RCP8.5 forcings, thereafter (Lamarque et al., 2011; Meinshausen et al., 2011). We will assess the 1979–2014 period using the historical and RCP8.5 simulations combined and this will be referred to as LENS. 2.3 CMIP5 As with LENS, we will combine years 1979–2005 of the historical simulations with years 2006–2014 of the RCP8.5 simulations for the 35 CMIP5 models listed in Table 1. For monthly data we make use of all available ensemble members that have both historical and RCP8.5 components, resulting in ensemble sizes ranging from 1 to 10 members (third column of Table 1). We will always show the ensemble mean of a metric for each model when multiple members are available. Error metrics are first calculated for individual members before the ensemble averaging is performed so as to avoid comparing smoother ensemble mean spatial fields with the noisier fields of individual members. For metrics that involve daily ua and va data, we use one member from the 16 models (highlighted with an ∗ in Table 1), and for daily zg data, we use one member from the 15 models (highlighted with a + in Table 1). For models that have more than one member available, we use the member with the lowest realization number. Daily fields are obtained by averaging 6-hourly pressure level fields. In each figure, a CMIP5 model can be identified by the model number given in the left column of Table 1. 2.4 CMIP6 For CMIP6, we make use of 1979–2014 of the historical simulations, run under the same forcings as the CESM2 simulations described above. At the time of writing, 42 models are available with ensemble sizes ranging from 1 to 32 (Table 1, right three columns). While the BCAM6 and BWACCM6 ensembles are contributed to the CMIP6 archive, we consider them separately here. Only a subset of 27/20 models, highlighted with a ∗/+ in Table 1, have daily averaged (ua,va)/zg data available and for each of these we only use one member (the member with the lowest realization number). In each figure, a CMIP6 model can be identified by the model number given in the third from right column of Table 1. We only show error metric summaries for the CMIP6 models in the main text, but ensemble mean spatial bias maps along with indications of model consensus are provided in the Appendix A. 2.5 Observation-Based Data Sets Our primary observational comparison will be with atmospheric reanalyses. The new ERA5 reanalysis (C3S, 2019) will be taken as the observational baseline and all simulations and other reanalysis products will be compared to that. Three other modern reanalyses that assimilate a wide array of observations will also be shown: ERA-Interim (Dee et al., 2011), MERRA2 (Gelaro et al., 2017), and JRA55 (Kobayashi et al., 2015). Two twentieth century reanalysis products: ECMWF's ERA20C (Poli et al., 2016) and NOAA's twentieth century reanalysis, 20CR (Compo et al., 2011), are considered, partly for the purpose of assessing those reanalysis products compared to others and also for the purpose of providing a longer-term context for the observational record in certain metrics. These twentieth century reanalyses are only constrained by surface pressure observations (and marine surface winds in the case of ERA20C) and, therefore, lack the additional constraint arising from the multitude of other observations that are assimilated in the other products. For the most part, we only use 1979 to 2014 for these reanalysis products for direct comparison with the model simulations. MERRA2 only starts in 1980, so for that product we use 1980 to 2014 and ERA20C only extends to 2010, so for that product we use 1979 to 2010. For the North Atlantic, where a relatively large number of surface pressure observations constrain the twentieth century reanalyses back to 1900, we provide an assessment in the variability of metrics using all overlapping 36-year segments between 1900 and 2014 (2010 for ERA20C). For metrics involving daily data, we do not make use of the twentieth century reanalyses. 3 Error Metric and Uncertainty Assessments 3.1 Normalized Mean Square Error Metric When assessing the error in a spatial field (X), we will use the Normalized Mean Square Error (NMSE) metric proposed by Williamson (1995). This metric has been applied to the geopotential height field in evaluations of previous NCAR models (Collins et al., 2006; Kiehl et al., 1998; Neale et al., 2013), but here we apply it to all spatial fields considered. The NMSE of the model field Xm is given by (1)where Xo refers to the “observed” field (in our case ERA5); the overbar refers to the area weighted spatial average and the prime refers to the deviation therefrom. To give some indication of where the errors are coming from, the NMSE can be further decomposed into three components as follows: (2)with (3)where σo and σm refer to the spatial standard deviation of the observations and model, respectively, and rmo refers to the spatial correlation between the model and observations. For a derivation of this, see Murphy (1988), their Equation 10, which is essentially the same as this but without normalization of the mean squared error. The first term, U, is the unconditional bias, which is a nondimensional measure of the overall bias in the spatial mean of a field. The second term, C, is the conditional bias, which is nondimensional and arises through both amplitude and phase errors. It is only nonzero if the regression of Xm onto Xo yields a slope of 1, that is, if Xm is perfectly correlated with Xo and their spatial variances are equal. The third term is the phase error P, which arises only from errors in the phasing of the spatial variations. If rmo = 1, that is, the phase error is 0, the interpretation of any conditional bias is straightforward; it arises if the amplitude of the spatial variations are too large or too small. When the phase error is nonzero, the interpretation of the conditional bias is less straightforward as it arises through both amplitude and phase errors. Furthermore, the conditional bias can be artificially reduced through a lack of spatial variance. Interpretation can, therefore, be aided by consideration of the scaled variance ratio (SVR) given by (4)which indicates whether conditional bias arises from too much (SVR > NMSE) or too little (SVR < NMSE) spatial variance. When SVR < NMSE, it also provides caution that the conditional bias component may be artificially reduced through the lack of spatial variance. Altogether, U, C, P, and SVR provide an indication of the roles of an overall mean bias, biases that arise due to errors in the amplitude of spatial variations and biases that arise due to phasing errors. Figure 1 provides an explanatory key for how this will be represented in each figure. The NMSE will be depicted in each plot with a vertical bar composed of three different colored components for U, C, and P, while the SVR will be depicted by a circular symbol. When the SVR symbol lies above/below the bar the SVR is greater/less than 1. For models with a spatial mean (unconditional) bias with a magnitude greater than 10% of the ERA5 spatial mean, we depict whether that bias is positive or negative by shading the SVR symbol red or blue, respectively. Figure 1Open in figure viewerPowerPoint An explanation of the representation of the NMSE, SVR, and unconditional bias in each figure. The NMSE is depicted by vertical bars with three shaded components (where relevant). The three components are, from bottom/light to top/dark, the Unconditional Bias, Conditional Bias, and the Phase Error (see Equation 3). The SVR is depicted by the circle. Where the SVR lies above the bar as in the first, third, and fifth bars here, the SVR is greater than 1 and vice versa. When the magnitude of the spatial mean bias is more than 10% of the ERA5 spatial mean, we consider that to be a “large” unconditional bias and shade the SVR circle red for positive (third and fourth bar examples) and blue for negative (fifth and sixth bar examples) biases. Otherwise, the SVR circle is left open. 3.2 Assessment of Uncertainty due to Internal Variability The 36-year observational record that the models are being compared to will be subject to uncertainty due to the sampling of internal variability. To provide some indication of the magnitude of this effect, or the significance of differences between the model and the reanalysis, we take a number of approaches: When assessing the bias of BCAM6 or LENS relative to ERA5 in map form, we provide an assessment of whether ERA5 lies within the distribution of the 11(40) ensemble members for BCAM6(LENS) and assume that where this is not the case, there is a significant difference between the real world and th