GMAO Observing System Simulation Experiments (OSSE)
What is an OSSE?
Observing System Simulation Experiments (OSSEs) are designed to mimic the process of data assimilation. In applications of an atmospheric data assimilation system with regard to real observations, real imperfect observations are drawn from the real atmosphere to produce estimates of global atmospheric states at sequences of time. In an atmospheric OSSE, simulated observations with simulated errors are drawn from a simulated atmosphere (termed a "nature run" or "NR") and provided to a data assimilation system to produce estimates of those NR states. Since the OSSE deals entirely with simulations, it is not restricted to using only observations that actually exist. Also, the underlying "true" atmospheric state is known precisely. These two properties of the OSSE facilitate many types of informative experiments relevant to NASA's missions.
Why do an OSSE?
OSSEs may be used to help guide development of new instruments, as OSSEs may be performed during their planning stages. By estimating impacts of the envisioned observations on data assimilation and forecast system performance, project requirements and design implications can be better determined. The OSSE setup may also be used to investigate more theoretical questions, as even impractical observations can be generated with relative ease; e.g., because the quality of the observations can be strictly controlled in the OSSE context, how the accuracy of data assimilation products depend on instrument error characteristics can be easily examined. Unlike when assimilating real observations, the NR states from which the OSSE observations are drawn are known perfectly and analysis errors therefore can be explicitly and precisely determined. This OSSE property can therefore be used to greatly expedite development and testing of new data assimilation algorithms.
OSSE Development at GMAO
An OSSE has recently been developed at the GMAO. This OSSE uses a Nature Run generated by the European Centre for Medium-Range Weather Forecasts (ECMWF) as part of a multi-agency Joint OSSE project using a 2005 version of the ECMWF operational numerical weather prediction model. This 13-month Nature Run has been evaluated and found to be satisfactory for use in OSSEs. A full suite of synthetic observations with calibrated observational error has been created from the Nature Run to replicate the entire observational network. The Gridpoint Statistical Interpolation (GSI) data assimilation and GEOS-5 forecast model are used to ingest the synthetic observations and generate experimental forecasts.
Performance and validation of the GMAO OSSE
It is imperative that a baseline OSSE be well validated. The OSSE should faithfully reproduce many metrics used to assess observations and data assimilation systems when similar existing observation types are considered. For past OSSEs, when validations have been performed, usually the best included a few data denial comparisons (i.e., corresponding observing system experiments) performed in OSSE and real contexts. The GMAO OSSE has been extensively calibrated using an iterative process in which the synthetic observations and their errors were tuned to behave realistically.
The images below show a comparison of real AIRS data (channel 295, 18 UTC 12 July 2005) in the top panel with synthetic observations from the GMAO OSSE at the same time in the lower panel. The observation locations are not expected to be identical due to the difference in cloud fields in the Nature Run versus reality, but the overall count and distribution of observations is similar.
AIRS Observations, Real Data
AIRS Observations, OSSE Data
The impact of the observations on the analyses and forecasts should be realistic in the OSSE in order to obtain meaningful experimental results. The analysis increment, or analysis minus background, is a measure of how much work the observations do in changing the analysis state. The figures below show the July 2005 square root of the zonal mean of temporal variances of analysis increments for zonal wind and temperature. On the left are the Real data analysis increments, on the right are the OSSE analysis increments.
Zonal Wind (m/s)
In general, the agreement between OSSE and real results is remarkable, especially since the use of a specific month comes only through the specified data coverage and sea surface temperature.