SR/SSD 97-8 2-15-97

Technical Attachment


Brian Korty and Mike Eckert
Forecast Operations Branch
NCEP Hydrometeorological Prediction Center

Last year was an exceptional one, in terms of QPF performance by meteorologists in the Forecast Operations Branch of the HPC. Verification records were set in eight out of nine categories for manually-produced 24-hour QPF. This success is in large part attributable to improved skill in the models. Here is a summary of threat scores:

Category New Annual Record Old Record/Year
Day 1 0.50" 0.387 0.351/1995
1.00" 0.271 0.265/1995
2.00" 0.170 0.169/1995
3.00" 0.131 0.095/1991
Update 0.50" 0.342 0.312/1995
1.00" 0.232 0.215/1995
2.00" 0.146 0.126/1995
Day 2 0.50" 0.304 0.283/1995

The only annual record that survived the performance of 1996 was the Day 2 1.00" category, with a threat score of 0.196, set in 1995. The 1996 threat score (0.193) was close to the record.

Monthly records were also set in six of the nine categories for the month of December 1996. Overall in the various categories, 41 new monthly records were set in 1996 (38% of the total possible).

As was the case for 24-hour QPFs, forecasters also set many record threat scores for 6-hour QPFs in 1996. Every previous annual record and 40% of the monthly records were broken. The issuance of 0-6 hour QPFs began in 1992, while 6-12 hour and 12-18 hour QPFs were begun in 1981. Here is a summary of new record threat scores:

Category New Annual Record Old Record/Year
0-6 hour0.25"0.33019960.3221995
6-12 hour0.25"0.27419960.2611995
12-18 hour0.25"0.23919960.2281995
0.50"0.1621996 (tie)0.1621995
0-6 hour0.25"0.439Jan*0.389Jan 1995
0.404Feb*0.365Feb 1994
0.363Apr*0.328Apr 1995
0.315Sep*0.267Sep 1993
0-6 hour0.50"0.356Jan*0.309Jan 1995
0.291Feb0.276Feb 1995
0.286Apr*0.252Apr 1995
0.178Jul*0.132Jul 1993
0.236Sep*0.201Sep 1993
0.330Dec*0.296Dec 1994
6-12 hour0.25"0.393Jan0.378Jan 1987
0.346Feb0.326Feb 1989
0.310Apr*0.273Apr 1983
0.254Sep0.251Sep 1988
0.353Dec0.350Dec 1994
6-12 hour0.50"0.309Jan*0.266Jan 1987/91
0.247Feb0.244Feb 1989
0.120Jul0.118Jul 1990
0.181Sep0.178Sep 1988
0.275Dec*0.240Dec 1988
12-18 hour0.25"0.347Jan0.328Jan 1987
0.318Feb*0.287Feb 1989
0.258Apr*0.232Apr 1987
0.232Sep0.212Sep 1989
0.323Dec0.318Dec 1993
12-18 hour0.50"0.114Jul*0.089Jul 1990
0.151Sep0.138Sep 1988

An improvement of at least 10% over the previous record is marked with an asterisk (*)

Ed. Note: The threat score reflects how well the forecast matches the area which actually reaceived a given amout of precipitation. It accounts for size, shape, and location of the forecast, relative to what occurs. For example, imagine an isohyet which encloses the area where more than 2 inches of rain is forecast (call it area "F"). Now imagine an isohyet which shows where 2 inches or more actually fell ("O"). The threat score compares the area correctly forecast (the overlap of "F" and "O"--call it "X") with the area forecast AND observed ("F+O"). In other words:

TS = X / (F + O - X)

If the forecast and observed areas overlap perfectly, then TS = 1, the maximum possible score. If they fail to overlap, TS = 0; there is no "credit" for a misplaced forecast, even if the shape and size of the area is the same as what is observed.