Media & News

News Releases
University of Wisconsin Study Based on Shaky Foundation of Faulty Data and Conclusions

April 7, 2015

           

A recent study published in Environmental Research Letters by authors at the University of Wisconsin uses error-prone satellite data to suggest that growth in U.S. corn and soybean production from 2008 to 2012 drove massive conversion of grassland, forest, and other native lands to cropland.[1]

 

The authors attribute these purported land conversion events, in part, to increased demand for biofuels. However, contrary to the study's results, there is no empirical evidence to support the argument that U.S. cropland has expanded since 2008, let alone that large tracts of native grassland and forest have been converted to crops. In fact, USDA data clearly show that the area planted to field crops was more than 1 million acres smaller in 2012 than in 2008.

 

Indeed, the natural conclusion suggested by USDA acreage data is that increases in corn and soybean production have been accommodated through crop switching and higher yields per acre not through conversion of non-agricultural lands. Increased corn and soybean acreage has been largely offset by reductions in the amount of acres planted to wheat, sorghum, hay, and other crops.

 

Further, some previous cropland enrolled in the Conservation Reserve Program (CRP) has returned to crop production.

 

USDA Data Dispute Study Findings 

 

The study's authors suggest that net U.S. cropland expanded by 2.98 million acres from 2008 to 2012. Yet, USDA data show a net reduction in area planted to field crops (i.e., planted cropland) between 2008 and 2012.[2] Planted cropland totaled 325.6 million acres in 2008, fell in successive years through 2011, then rebounded to 324.3 million acres in 2012still more than 1 million acres below 2008 levels.[3] Thus, while the amount of cropland planted to corn and soybeans increased by 12.8 million acres from 2008 to 2012, cropland planted to all other field crops fell by 13.5 million acres more than offsetting the increase in corn and soybeans .

 

The paper further asserts that [o]ur magnitude of net expansion (2.98 million acres) is consistent with the higher level of expansion found in the most recent five-year census of agriculture. This statement is puzzling and ultimately misleading, as the most recent Census of Agriculture shows a significant reduction in cropland from 2008 and 2012.[4]

 

Key findings from the most recent Census of Agriculture include:

  • Total cropland fell 4% (16.7 million acres) from 406.4 million acres in 2007 to 389.7 million acres in 2012.
  • Total woodland increased 2.5% (1.9 million acres) from 75.1 million acres in 2007 to 77.0 million acres in 2012.
  • Permanent pasture and rangeland increased 1.6% (6.5 million acres) from 408.8 million acres in 2007 to 415.3 million acres in 2012.
  • Irrigated land fell by 1.4%, calling into question the paper's argument that cropland expansion raises substantial concerns about water use and sustainability.

Despite the contentions of the Wisconsin paper, USDA data provide strong evidence to support the fact that biofuels expansion in the 2008-2012 timeframe did not lead to net expansion of cropland in the United States.

 

The Gross vs. Net Dilemma and Misuse of USDA and USGS Satellite Data

 

The authors suggest that examining net cropland changes in aggregate (as in the charts above) masks gross conversions of grassland and forest to cropland. They argue that viewing cropland changes on a net basis and in the aggregate obscures local and regional land-use changes and emissions impacts (i.e., because conversions of non-cropland to cropland in one area could presumably be offset by abandonment or idling of previous cropland in another area). While this may make sense in theory, it ignores the economic reality that farmers would not respond to increased demand for certain crops by retiring or abandoning existing cropland in one area only to turn around and convert non-agricultural land to cropland in another area. Still, to address this net vs. gross concern, the authors rely on USDA's Cropland Data Layer (CDL) satellite tool to estimate gross land conversions.

 

While a useful tool for analyzing potential cropland trends, the CDL system is not intended for analyzing non-agricultural land classes and has a spotty record with regard to identifying certain land types. In particular, the tool has demonstrated a high level of error in differentiating between grasslands and land planted to hay, other forage crops, and close-row or broadcast-planted crops. In other words, the CDL satellite tool is highly inaccurate in distinguishing among native grassland, hay, managed pasture, barley, oats, CRP, and idle/fallow cropland. This is important because these land types can have very different carbon stocks. USDA itself acknowledges that, Unfortunately, the grassland-related categories have traditionally had very low classification accuracy in the CDL.

 

Further, USDA states that We continue to search for program enhancements and ancillary datasets that may help improve the identification of grassland and pasture categories within the CDL. The table below shows the frequency of commission errors in the CDL tool for key corn-producing states for certain cropland types in 2012. A commission error occurs when a satellite image pixel is included in the incorrect land category (e.g., if a pixel is labeled as fallow/idle in the CDL, but ground-truthing reveals that the land is actually planted to alfalfa, this would represent a commission error). As an example, the table below shows that 44% of the pixels classified in CDL as alfalfa in Missouri were incorrectly included in the alfalfa category and actually belonged in another land category. Many land categories in corn-producing states had commission error rates of more than 50%, meaning the CDL was wrong more often than it was right regarding those land types. In particular, the fallow/idle land and other hay categories generally demonstrate an extremely high rate of error. Given the high degree of misclassification within CDL, it is highly likely that the authors incorrectly identified some land cover types, calling into question the validity of their land cover change analysis.

 

Frequency of Commission Errors for Certain Land Cover Types in CDL (2012)

 

[This table shows the frequency with which certain land cover types were incorrectly identified in the CDL tool for 2012]

 

Alfalfa

Other Hay

Barley

Oats

Fallow/Idle

Iowa

32%

74%

75%

37%

92%

Illinois

40%

65%

 

43%

85%

Kansas

11%

 

43%

40%

11%

Minnesota

31%

70%

11%

56%

85%

Missouri

44%

44%

 

96%

39%

Nebraska

11%

16%

48%

27%

9%

N. Dakota

36%

48%

16%

53%

28%

S. Dakota

35%

66%

43%

41%

26%

Wisconsin

14%

42%

62%

39%

56%

Source: USDA

 

The above table clearly demonstrates why the CDL tool should not be relied upon to identify certain land cover categories for the type of land cover change analyses performed by the Wisconsin authors. Current satellite tools are simply unable to reliably tell the difference between certain land cover types, and the authors did little or nothing to characterize the uncertainty of their results or place their findings in proper context. Instead, they presented their results as absolute, definitive conclusions.

 

Because the CDL tool is known to perform poorly in identifying some crop categories and all non-crop categories, USDA explicitly states that the accuracy of the non-agricultural land cover classes within the Cropland Data Layer are entirely dependent upon the [U.S. Geological Surveys National Land Cover Database, or NLDC]. However, the NLCD itself has limited data and capabilities for the type of analysis undertaken by the Wisconsin authors. Certain land classes are treated inconsistently or misidentified in the NLCD and the tool provides only a snapshot of land cover once every five years (the most recent data available is a snapshot of 2011).

 

The problems inherent in taking a snapshot approach to land cover are brushed under the rug in the study's estimate of gross conversion of non-cropland. As an example, the authors claim that [o]f the gross change, 7.34 million acres of land uncultivated since at least 2001 were converted to crop production 20082012. Apparently, the authors are comparing the 2001 NLDC snapshot to the 2008-2012 CDL data that is rife with classification errors regarding grassland, pasture, hay, CRP, idle/fallow, etc. But even if the 2008-2012 data were correct, this method omits or ignores the uses of the land for the four years spanning 2002 to 2005, as well as 2007. It is entirely possible that certain land was uncultivated/fallow/idle in 2001 when the NLDC snapshot was taken, then entered crop production at some point in the 2002-2005 period, then returned to fallow/idle in 2006-2007, and then re-entered crop production at some point in the 2008-2012 period.

 

But because the study's methodology relies on point-in-time snapshots prior to the 2008-2012 period, it ignores the potential for these types of long-term rotations of cropland with fallow/idle land. USGS has recognized the limitations of providing only snapshot data in the NLCD database and discourages comparison of snapshots from different time periods. Yet, the Wisconsin authors did exactly that. They apparently compared past datasets to one another to estimate longer-term land cover changes even though USGS warns that it is not possible to compare all three NLCD products (1992, 2001 & 2006) at this time. As such, there is no legitimacy to the Wisconsin studys conclusion that 1.6 million acres of long-term (20 + year) unimproved grasslands were transformed to cropland during our recent four-year study period.

 

Other Concerns with Study's Methodology

 

A careful review of the Wisconsin paper reveals a number of other questionable assumptions and methodological choices, including:

  • The paper treated CRP land, pasture, and hay ground as non-cropland, despite evidence that these land types are often interchanged with cropland in long rotations. Thus, when the error-prone satellite data appeared to show conversion of CRP, hay ground, or pasture to cropland, the authors recorded this as a conversion of non-cropland to cropland and treat it no differently than conversion of native grassland to cropland. In reality, CRP, pasture and hay ground are actively managed agricultural lands that can rotate with cropland over long time periods depending on market conditions. The carbon stocks associated with these land types are substantially lower than carbon stocks associated with native grassland or prairie, yet the authors treated them as one in the same.
  • Further, the papers abstract states definitively that corn was the most common crop planted directly on new land, as if the authors had somehow specifically tracked and validated exactly what crops were planted on every acre of cropland every year. Only later in the paper do we discover that the authors used a questionable accounting trick to arrive at this conclusion. The authors write that [w]e allocated land conversion responsibility to specific crops by assuming that new conversion was proportional to a crops change in area. Only crops that experienced a net increase in area were assigned responsibility for cropland expansion. Crops unchanged in area were not considered responsible for conversion, and crops that experienced a net decline were assigned responsibility for abandonment. This approach results in exactly the same time type of net vs. gross dilemma that appears to concern the authors with regard to total cropland area.
  • As stated above, the authors did not include any discussion of uncertainty in their analysis. Clearly, the high degree of error associated with the satellite data is an important factor of which readers of the study should be made aware. In fact, the uncertainty associated with the satellite data is such that the results presented in the study are not even directionally consistent with other data maintained by USDA (i.e., NASS acreage data shows a net reduction in cropland during the study period, while the authors suggest there was a net expansion). The reasons for this important discrepancy are not discussed.

Conclusion: Proceed with Caution

 

In the end, the authors of the Wisconsin paper fail to explain why their land use results derived from highly suspect satellite data analysis differ dramatically from official data reported by USDA's National Agriculture Statistics Service. The authors suggest cropland has expanded on both a gross and net basis at the expense of grassland, forest and other native lands. Yet, there is little or no evidence in USDA's widely used data sets that this has in fact occurred, nor is even sufficient anecdotal evidence to substantiate the study's claims. Instead, USDA data show that the long-term national trend toward less planted cropland persisted following passage of the expanded Renewable Fuel Standard in 2007, and that increases in corn and soybean acres were accommodated through crop switching.  

 


[1] T.J. Lark, J.M. Salmon, and H.K. Gibbs. Cropland expansion outpaces agricultural and biofuel policies in the United States. Environ. Res Lett. 10 (2015) 044003. Doi:10.1088/1748-9326/10/4/044003.

[2] U.S. Dept. of Agriculture, National Agriculture Statistics Service. Field crops include all principal planted crops (barley, beans, canola, corn, cotton, flax, hay, haylage, hops, lentils, millet, mustard, oats, peanuts, peas, potatoes, rapeseed, rice, rye, safflower, sorghum, soybeans, sugarbeets, sugarcane, sunflower, taro, tobacco, and wheat.

[3] See NASS, Statistics by Subject: Crops: Field Crops. http://www.nass.usda.gov/Statistics_by_Subject/index.php?sector=CROPS 

[4] See: http://www.agcensus.usda.gov/Publications/2012/Full_Report/Volume_1,_Chapter_1_US/st99_1_008_008.pdf