Xionia · Forecast evaluation against observations
Ensemble backtesting in Agios Stefanos and Athens 850 hPa
Measured ensemble forecast skill for Agios Stefanos surface temperature and rain, with Athens/Hellinikon radiosonde verification at 850 hPa. GEFS, AIGEFS, IFS ENS, AIFS ENS and GEPS.
Errors and probabilities across five model families
Each family is a separate group of forecasts: its members start with small differences and give a range of possible outcomes. Here we check that range against real observations.
Checking for an available report…
| Model | Unique valid times | Forecast cases | CRPS | Mean error | Brier |
|---|
Lower CRPS and Brier mean better performance on this sample. CRPS and mean error use °C or mm; Brier is unitless. Brier refers to temperature ≤0°C or rain ≥1 mm in six hours. Positive mean error means the model predicted a higher value.
How backtesting and local calibration work
We archive the first available forecast from each run before its verification time. An old forecast downloaded after the event does not enter the scores. Observations and forecasts retain their own timestamps and sources.
Surface temperature is compared with the Agios Stefanos station, allowing at most five minutes from forecast valid time. Rain must cover exactly the preceding six hours. Missing data, uncertain rain-interval boundaries and counter resets are excluded.
The default common sample compares the same run and valid time across all five models. Unique observation times are counted separately from forecast cases: several runs can verify against the same observation. Small samples do not establish lasting model superiority.
Probability assessment keeps later observations out of training. Once enough observations exist, an adjustment is checked against those later cases. This report does not automatically activate new calibration or change the Agios Stefanos forecast formula.
850 hPa temperature: Athens / Hellinikon radiosonde
The regional reference comes from radiosonde station 16716 at Hellinikon. We use the 850 hPa temperature reported in the sounding profile, without adding our own interpolation between neighbouring heights. Comparison with the Agios Stefanos forecast retains the distance to each model grid point, capped at 50 km.
Hellinikon is about 30 km from Agios Stefanos. The measurement describes the air mass over the wider Attica region, rather than a direct observation above Agios Stefanos. Time refers to the nominal UTC cycle; balloon drift and the exact time of crossing 850 hPa are not resolved in this assessment.
REA · Athens, Thessaloniki and Heraklion soundings · University of Wyoming · Radiosonde archive