One has to have thick skin to work as a statistician at USDA’s National Agriculture Statistics Service. Regardless of the numbers one calculates and develops as a crop estimate, there will be instant criticism, challenges to your sanity, and possibly even questions about your parentage. Although the NASS staff is politically neutered and prohibited from engaging in any commodity market speculation, they are accused from all quarters of nearly everything including high crimes and misdemeanors. Poor devils, they cannot get a break.
Until now.
IL ag economists were hired by USDA’s Chief Economist Joe Glauberto evaluate the accuracy of NASS crop and WASDE supply-demand reports. While they had some recommendations for improvement, they generally reported that the market had more trouble anticipating the numbers in the reports, instead of the crop reports being incorrect.
The researchers reporte, “With a few exceptions, WAOB (World Agricultural Outlook Board) corn and soybean forecast errors since 2006 generally are within the historical range of errors. While there are instances of large forecast errors, these are readily explained by an unusual weather condition or insect problem that occurred after the forecasts were released.”
The economists also found, “NASS yield forecasts for corn reveal no evidence of bias in any month over 1990-2012 and forecast errors since 2006 are well within the historical range of errors. There is some evidence of improvement in the accuracy of NASS corn yield forecasts over time. Soybean forecast errors since 2006 are also within the historical range, except for Sept. & Oct. 2012…there is also a general tendency for soybean forecasts to be conservative.”
Consequently, they report, “The statistical evidence indicates there has been a sharp decline in market analysts’ ability to anticipate actual quarterly corn usage as implied by NASS Grain Stocks reports since the start of the 2006 marketing year. The standard deviation of market surprises for implied corn usage in 2006-2012, 6.2%, was twice the standard deviation over 1990-2005, 3.1%. In contrast, there was no compelling evidence of deterioration in market analysts’ ability to anticipate actual quarterly soybean usage as implied by NASS Grain Stocks reports over 1990-2012.”
Why is there so much difference in USDA projects and market analysts’ estimates? The economists said, “The explanation with the most merit is that unresolved errors in production estimates for corn led to the large surprises. NASS stocks estimates undoubtedly encompass sampling errors for both production and stocks estimates and it is likely that unresolved sampling errors for corn production estimates are large enough to explain the surprises.”
So what can be done to disseminate better information with increased credibility? The researchers came up with 10 recommendations which they believe will provide for better USDA reports, and help the market better understand the process of developing crop estimates:
1) Provide monthly information about the exact process used to determine yield forecasts.
2) Review soybean yield forecasting procedures to eliminate the source of any bias.
3) Provide more openness about the formulas used for estimating crop yields.
4) Initiate a study about incorporating yield monitor data into crop estimation procedures.
5) For stocks reports, a specified weight per bushel should be used for on & off farm storage.
6) Review corn stocks estimation procedures to determine any methodological problems.
7) Add a question about on-farm grain stocks to the 5-year agricultural census.
8) NASS and the market need to better understand each other about stocks estimates.
9) USDA should evaluate the cost and benefit of adding a survey about ethanol use of corn.
10) USDA should evaluate the cost and benefit of adding a survey about corn feed use. (The economists said this is the single most important change that USDA should consider.)
Summary:
After many years of criticism about the accuracy of USDA crop reports, recommendations have been made to not only improve the reports, but enhance the market’s understanding of the estimation process. Researchers found little to criticise USDA about, however, believe that some changes could be made for the better, among them a better way of calculating feed use of corn and the volume of corn used for ethanol production.