On the Weather Forecasts made by the Old Farmer’s Almanac
One way to annoy many meteorologists is to suggest that the weather forecasts from the Old Farmer’s Almanac (OFA) are more skillful than those from the National Weather Service. I certainly share this peeve, and so on a lark some years ago, I decided to determine for myself whether the OFA forecasts for the Pacific Northwest had merit. The results of this analysis are summarized here.
The OFA does not fully explain how they make their forecasts. A secret formula devised by the founder of OFA is supposedly locked up in a black box in an office in Dublin, NH, with some refinements and enhancements to the method over the years. The OFA is not forthcoming with details but claim to use sunspots and solar activity in particular to project how the weather will differ from seasonal norms. These projections are made in the late summer/early fall for the next year for 18 different regions of the US. They include qualitative forecasts for the entire year for blocks of days ranging from a few days to about a week in length. Because they are qualitative, these forecasts cannot be verified quantitatively. Moreover, the chaotic nature of the atmosphere is such that there are strict limits to the predictability of the occurrence and timing of weather events such as cold snaps or rainy periods. It is simply impossible to predict specific events beyond a couple of weeks, at best. On the other hand, the OFA also provides numerical forecasts of monthly temperature and precipitation anomalies, and in principle, these forecasts could be skillful. The OFA claims “80% accuracy” in their weather forecasts (it is unclear whether they are referring to the qualitative forecasts for the day-to-day weather within each month, or to the temperature and precipitation anomaly forecasts for months on the whole), and so it seems reasonable to check to see if this is actually the case.
This analysis considers the monthly temperature and precipitation anomaly forecasts from OFA for the Pacific Northwest (their Region 15) during winter (November through March) over the period of 1990 to 2002. These forecasts were compared with the observed temperature and precipitation anomalies based on averages from the appropriate NOAA/NCDC Climate Divisions (Div. 1-4 for WA, 1-4 for OR, and 1 for CA). An unfavorable correspondence was found between the OFA forecasts and the observed monthly mean temperature and precipitation anomalies. The temperature forecasts were wrong as often as not; the precipitation forecasts were actually a bit worse than by chance. In more quantitative and statistical terms, the correlation coefficient between the forecasts and verifying observations was essentially zero for temperature, and about -0.2 for precipitation. This analysis was carried in 2003, and of course more data is now available, but there is no reason to think that an updated analysis would yield much different results.
The present analysis is by no means the only independent assessment of OFA weather forecasts. In particular, John Walsh and David Allen of the University of Illinois published an article (Weatherwise, October 1981) with an evaluation of the accuracy of 60 monthly temperature and precipitation forecasts for 32 stations across the U.S. They found that the OFA forecast the sign of the temperature anomalies correctly 50.7% of the time. The corresponding accuracy for precipitation was 51.9%. These compare with a rate of 50% by flipping a coin, over the long haul. Jan Null, a certified consulting meteorologist with Golden Gate Weather Services, has done his own OFA forecast verifications for a number of different periods across the country. While there are some occasional hits, his analysis has revealed in general, quite poor performance. Jeff Masters, of Weather Underground, a reputable private forecasting outfit, has also found OFA forecasts to be no better than those made by chance and sometimes significantly worse.
Evidence is lacking that the OFA’s weather forecasts have skill, despite their claims to the contrary. Nevertheless, they seem to have an enduring popularity. I can only speculate on the reason(s), but I suspect it has something to do with the appeal of simple methods, i.e., the superiority of Yankee ingenuity over complicated statistical and computer models. I also expect the definitiveness of the OFA product, especially the day-to-day projections, makes them attractive relative to the probabalistic forecasts from NOAA’s Climate Prediction Center and other climate forecast groups. The readers of this newsletter are advised that it is the latter-type of seasonal predictions that should be taken seriously.