SR/SSD 98-21


Technical Attachment



Ed Agre, Senior Forecaster

Mark Cunningham, Forecaster


As an offshoot of the Winter Olympics in Nagano, Japan, we decided to hold a numerical model challenge of our own during the months of February and March. The challenge was placed before five of the most accessible models via AFOS or the Internet to our local field offices. These five models were: the USA - Medium Range Forecast (MRF - NWS) and Navy Operational Global Atmospheric Prediction System (NOGAPS - Navy), the European - ECMWF, the Canadian - SEF, and Great Britain - UKMET. The goal was to test the abilities of these models in forecasting the development of central/southeastern United States storm systems at 120 hours of the model run.

Each model was compared in five forecast features as determined by the local committee: 1&2) the position and depth of the 500mb low center/vorticity maximum, 3) position of the 564 dm height contour across the southern US, 4&5) the position and intensity of the surface low pressure center.

Gold, silver, and bronze medals were awarded in each round of the Games that began on February 6, 1998 and concluded on March 20, 1998. Since the MRF and SEF models run out to 120 hours only at 0000 GMT, and the ECMWF runs only at 1200 GMT, 5 of the 6 competitions were initiated on the 0000 GMT cycle runs. This factor placed the ECMWF at a slight disadvantage as we had to average its 120 hour and 144 hour progs to verify at an appropriate 0000 GMT. The initialization of the AVN/MRF model at the verifying time was used in determining the medal awards.


The first round of competition began with the 0000 GMT model run of February 6 when all 5 participants were developing a storm system in the central plains by 0000 GMT of February 11. The NOGAPS won the gold as it accurately outperformed the other models in all five categories, most notably the surface low position near Texarkana being only 4 mb too high. The silver went to the SEF and bronze to UKMET.

The second round of competition began with the 1200 GMT model run of February 18 verifying at 1200 GMT on February 23. The previous 0000 GMT run of the MRF and SEF were used in this unique case. Once again the NOGAPS took the gold with the an accurate 120 hour forecast of a surface low on the South Carolina coast being only 5 mb too high. The silver medal went to the ECMWF with a tie for the bronze between MRF and UKMET.

The third round of competition began with the 0000 GMT model run of February 23 verifying at 0000 GMT on February 28. For the third consecutive time, the NOGAPS walked away with gold easily winning both surface and 500 mb progs of a 992 mb(only 3 mb too deep) low near Minneapolis and a 534 dm (only 4 dm too high) low near Des Moines, respectively. The silver went to the UKMET followed by a close contest for the bronze medal that went to the MRF.

The fourth round of competition began with the 0000 GMT model run of March 3 verifying at 0000 GMT on March 8. A new winner emerged in this round as the MRF took home gold with a 500mb prog of a 542 dm low near Childress, TX, verifying as a 543 dm low near Roswell, NM. The silver went to the NOGAPS and bronze to the ECMWF.

The fifth round of competition began with the 0000 GMT model run of March 4 verifying at 0000 GMT on March 9. The gold performance went to the UKMET with a 500 mb prog of 546 dm low/vorticity center near Memphis that verified as a 536 dm low near Harrison, AR. Overall this cycle was the worst for all of the models as the silver went to the SEF and bronze to the NOGAPS.

The sixth round of competition began with the 0000 GMT model run of March 15 verifying at 0000 GMT on March 20. The gold went to the MRF with an excellent 500 mb prog of a 544 dm low near Wichita verifying at 544 dm near Fort Smith, AR, and a surface prog of a 1004 mb low near Cape Girardeau, MO, verifying at 1000 mb near Hopkinsville, KY. The silver medal went to the NOGAPS and bronze to the ECMWF.

Finally, the medal count was in (Table 1) with only the NOGAPS model gaining a top three position in every round. It should also be noted that the one bronze medal won by the NOGAPS was only the result of being 12 hours too fast in developing a mostly correct scenario in the Ohio valley during round five of the competition.

Table 1.

Model Gold Silver Bronze
NOGAPS 3 2 1
MRF 2 0 2
UKMET 1 1 2
SEF 0 2 0
ECMWF 0 1 2


A short two-month Olympic style competition between medium range numerical forecasting models revealed that the Navy's NOGAPS model was very reliable in making 120-hour predictions of the surface and 500 mb features across the central and southeastern United States. Only one notable bias of the NOGAPS was a tendency to move southern stream systems too far south across the Gulf of Mexico (may be related to El Nino year activity). Forecasters at the field office level should be utilizing this model via the Internet when feasible. The incredible accuracy of several forecasts has led to increased usage of this tool at NWSFO Jackson. We also believe that consideration should be given to continuing/improving access to the NOGAPS data suite during and following the AWIPS modernization phase of the National Weather Service.