SR SSD 2001-07
3/2001

Technical Attachment

A Comparison of the WFO Amarillo Warning Verification Statistics
and Population Density

John M. Wolters
National Weather Service
Amarillo, TX

Introduction

The National Weather Service (NWS) continually strives to improve its services to the public. A critical component of these services is the issuance of severe weather warnings. The purpose of this paper is to come to a better understanding of how the population density of the Texas and Oklahoma panhandles affects verification of severe thunderstorm and tornado warnings. This understanding will allow the warning meteorologist to issue more reliable products to the public.

Data

Warning verification statistics were gathered for the individual counties that make up the county warning area (CWA) of the NWS office in Amarillo. The CWA includes the top (northernmost) 20 counties of the Texas Panhandle and the three counties of the Oklahoma Panhandle. These statistics consisted of the number of warnings issued, the number of verified warnings, and the number of unwarned events. There was no distinction made between a severe thunderstorm warning and a tornado warning. For this study, the number of warnings for a county equals the sum of the severe and tornado warnings issued between January 1, 1993 and November 16, 2000. The data were obtained from the NWS Storm Data and Verification home page (http://www.nws.noaa.gov/verification).

The population information for the individual counties was taken from the U.S. Census Bureau's 1999 estimates. This study should be revisited once the official 2000 Census figures are published, sometime in the summer of 2001.

Analysis

The population data were converted into people per square mile (PPSM) for each county. The population density is broken down for counties instead of individual cities because the NWS warnings are structured for counties or parts of counties and not cities. The number of warnings which verified were divided by the number of warnings issued for each county to get a percentage of the warnings which verified. The population density of a county was then compared to the percentage of warnings verified for that county, as well as the number of warnings issued and the number of missed events in that county. (An event is defined to be the occurrence of awind gust of 58 mph or greater, hail 3/4 inch in diameter or larger, or a tornado.)

Results

Looking at the population density of the Texas and Oklahoma panhandles, it quickly becomes apparent the greatest concentration of people is in the central Texas Panhandle. Potter and Randall Counties alone make up over half of the population of the Amarillo CWA. This is to be expected because Amarillo, the largest city in the area, is located in these two counties. Furthermore, if Potter and Randall are included with Gray (Pampa), Moore (Dumas) and Hutchinson (Borger) counties, more than 70% of the population in the CWA is accounted for.

Issuance of warnings vs. population of counties. Table 1 shows that the majority of the counties have fewer than 10 people per square mile (PPSM). It can be seen in Fig. 1 that between January 1, 1993 and November 16, 2000, the number of warnings issued for counties with fewer than 10 PPSM ranged from 34 to 120, but 61% of the counties had between 50 and 80 warnings issued. The five counties with more than 20 PPSM all had 77 or more warnings issued in the same period. This would seem to suggest that in general and for some reason, fewer warnings were issued for counties with low populations, relative to the more populous counties.

On the other hand, there are a few counties with less than 10 PPSM which had more than 90 warnings issued. These counties were Carson, Wheeler, Texas, and Beaver, all of which are in the eastern half of the CWA, with the exception of Texas County in the center of the CWA. This could be at least partly attributed to the likelihood of thunderstorms generally increasing from west to east across the CWA. The eastern portion of the CWA experiences significantly more thunderstorms, as that area generally remains east of the dryline where deep low-level moisture is considerably more abundant. Another factor could be that Interstate 40 lies in Carson and Wheeler Counties, and there are a greater number of very small communities in the eastern counties. Both factors might increase the chances of a severe storm (or potentially severe storm) being reported to the WFO. It is also possible a warning meteorologist might be more sensitive to the potential for thunderstorms affecting such highly traveled highways and small communities that are spread out across the eastern portion of the CWA.

Warning accuracy vs. population. How does population affect whether or not the warning is verified? One would suspect the larger the population, the more opportunities for observing a severe event and thereby verifying the warning. For WFO Amarillo, on average 67% of the warnings issued for a county are verified. Roberts County has by far the fewest residents with less than a half-percent of the total population in Amarillo's CWA, but more than 70% of the warnings issued for Roberts County are verified. Conversely, Potter County has the largest population with 120 PPSM, but only 60.5% of the warnings issued for Potter County verified. Why? Table 1 indicates more than twice as many warnings were issued for Potter County than for Roberts County. It would be extremely unlikely for a forecaster to double the number of warnings without a concurrent increase in the number of false alarms, but it may also be that some of the false alarms were attributable to increased concern on the part of forecasters for marginal severe events over populated areas. Marginal events such as dime-size hail or 60 mph thunderstorm wind gusts are much more likely to be verified. In addition, half of Amarillo lies in Potter and half in Randall County. To cover the entire city of Amarillo, a warning has to be issued for both counties and independently verified for each.

In general, there does not appear to be a trend in the percentage of warnings verified when compared to population density. Figure 2 shows that in most of the counties in the Amarillo CWA, between 60 and 75% of the warnings issued verified, with just as many counties above and below the county average of 67%. The reason may be the active chasing of storms in rural areas by several groups, including televison personnel, community spotter groups, researchers and academic groups. Proactive verification efforts on the part of the staff at the NWS office in Amarillo may also contribute to this.

Missed events vs. population. When examining warning verification, it is a good idea to look at how many events were missed, that is, how many events occurred without warnings? Twenty of the 23 counties in the WFO Amarillo CWA had 20 missed events or fewer from January 1, 1993 through November 16, 2000. Over half of the counties had less than two missed events per year. Looking at Fig. 3, there appears to be a general trend of increasing missed events as the PPSM increases. Potter and Randall, the two most populous counties, have some of the largest number of missed events. Since there are more people in these counties, it is more likely for someone to observe a marginally severe event, despite the fact that other evidence hints forecasters may have a tendency to warn more populated areas more often. Sherman County, one of the least populated counties, had only one missed event in the same time period.

One county stands out - Texas County. It had the most number of missed events for WFO Amarillo's CWA with 37 (9.0 PPSM). At the same time, however, almost 75% of the 117 warnings issued for that county were verified. The problem is not so much warning accuracy as too few warnings issued. This implies the warning meteorologist may need to be more aggressive in issuing warnings for Texas County. Another possibility is that the under-warning resulted from just one - or at most a few - severe weather outbreaks which caused an unusual number of events in that county, compared to surrounding counties.

Conclusions

The population density of the CWA does not appear to have an effect on the percentage of warnings which are verified. Considering all of the warnings issued for the entire CWA, 67.5% of the warnings are verified, which is very comparable to the county average of 67%. Most of the counties verify between 60 and 75% of the warnings issued, with just as many counties above the county average as below it.

There does seem to be a trend for increased missed events as the population density increases. Counties with 10 PPSM or less generally had a very low number of missed events. For example Roberts County, which has the lowest population density, had only six missed events. Counties with more than 10 PPSM tended to have more missed events. Potter and Randall counties have the highest PPSM and combined for more than 50 missed events.

While examining the number of warnings issued, more warnings were generally issued for populous counties and for those counties in the eastern CWA. The exception was Oldham County for which 76 warnings were issued. It also had the worst percentage of warnings verified. This could be attributed to the fact that Interstate 40 runs through the county. All of the counties through which I-40 runs have significantly more warnings issued for them when compared to counties with similar population densities. However, statistics suggest more discretion in warning decision making for Oldham County may be advised.

Observations

Looking at individual counties, over 84% of the warnings for Hutchinson County verified with only 12 unwarned events. Texas County had more than 74% of the warnings verify, but had the most number of unwarned events with 37. Finally, 53% of the warnings for Oldham County verified, the lowest percentage in the CWA, and there were only nine unwarned events.

Acknowledgments

I would like to thank Steve Cobb and Steve Drillette for their editing as well as their help in interpreting the data. I would also like to thank John Holsenbeck for his help with creating the County Warning Area map and the graphs. I would also like to acknowledge Mark Tew, from where the idea for this paper originated, based on his project The Effect of Population Density on Severe Warning Verification while at the NWS office in San Angelo, Texas.

Table 1