Climate models are routinely ranked for regional applications by skill scores that combine mean-state bias, variability amplitude, and temporal correlation with a reference. Whether such a ranking is a property of the models or of the evaluation period is rarely tested. We rank 39 free-running CMIP6 models against the ERA5 reanalysis over the 46 IPCC AR6 land regions, on two non-overlapping subsets of the annual 1980–2014 record, and correlate the two rankings for a Kling–Gupta-type total error and for each of its components, as displayed by the Aras diagram. Under this ratio-based score the total-error ranking is weakly reproducible (Spearman ρ = 0.40; 95% interval 0.12–0.59). The mean-state ranking is highly reproducible (ρ = 0.97), the variability ranking intermediate (ρ = 0.68), and the temporal-correlation ranking weak (ρ = 0.19, −0.05 to 0.38; 0.07–0.40 across variables, where the highest value, for temperature, falls to 0.09 after detrending). A second realisation of 17 models reproduces the bias ranking (ρ = 0.96) but not the correlation ranking (ρ = −0.25; −0.73 to 0.35), and the bias ranking persists into the observed 2015–2025 period (ρ = 0.93). A synthetic control recovers large injected signals and shows that stable between-model differences of the measured size would reproduce about as the real ones do, so the ordering reflects the between-model spread relative to sampling noise. The reproducibility of a single score depends on how its components are defined and weighted (0.99 under a standardised additive form), and all rankings are relative to ERA5. The results concern annual regional evaluation of free-running models, not the seasonal cycle, spatial patterns, or reanalysis-driven simulations.
Are regional rankings of CMIP6 climate models reproducible? Only the mean state survives a change of reference period / Izzaddin, A., Langousis, A., Yaseen, M., Gioia, A., Pagano, A., Iacobellis, V.. - In: STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT. - ISSN 1436-3240. - ELETTRONICO. - 40:12(2026). [10.1007/s00477-026-03408-7]
Are regional rankings of CMIP6 climate models reproducible? Only the mean state survives a change of reference period
Izzaddin, Aras
;Yaseen, Marwah;Gioia, Andrea;Pagano, Alessandro;Iacobellis, Vito
2026
Abstract
Climate models are routinely ranked for regional applications by skill scores that combine mean-state bias, variability amplitude, and temporal correlation with a reference. Whether such a ranking is a property of the models or of the evaluation period is rarely tested. We rank 39 free-running CMIP6 models against the ERA5 reanalysis over the 46 IPCC AR6 land regions, on two non-overlapping subsets of the annual 1980–2014 record, and correlate the two rankings for a Kling–Gupta-type total error and for each of its components, as displayed by the Aras diagram. Under this ratio-based score the total-error ranking is weakly reproducible (Spearman ρ = 0.40; 95% interval 0.12–0.59). The mean-state ranking is highly reproducible (ρ = 0.97), the variability ranking intermediate (ρ = 0.68), and the temporal-correlation ranking weak (ρ = 0.19, −0.05 to 0.38; 0.07–0.40 across variables, where the highest value, for temperature, falls to 0.09 after detrending). A second realisation of 17 models reproduces the bias ranking (ρ = 0.96) but not the correlation ranking (ρ = −0.25; −0.73 to 0.35), and the bias ranking persists into the observed 2015–2025 period (ρ = 0.93). A synthetic control recovers large injected signals and shows that stable between-model differences of the measured size would reproduce about as the real ones do, so the ordering reflects the between-model spread relative to sampling noise. The reproducibility of a single score depends on how its components are defined and weighted (0.99 under a standardised additive form), and all rankings are relative to ERA5. The results concern annual regional evaluation of free-running models, not the seasonal cycle, spatial patterns, or reanalysis-driven simulations.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


