Technical Review of West et al. 2020 and 2023, Guizar-Coutiño 2022, and Coverage in Britain’s Guardian

The authors used satellite imagery to assess the physical characteristics that they considered. In contrast, Verra requires extensive on-the-ground analysis, as noted in section 3 above, which includes a detailed consideration of local circumstances and socio-economic processes. Satellite imagery can be helpful, but overreliance on it, in contrast to the extensive body of information gathered through on-the-ground data collection for REDD projects, can be problematic.

In the case of these two works, the limitations of using satellite imagery are exacerbated by the authors’ choice of which satellite imagery to use.

West et al. 2020 used the land-cover MapBiomas dataset as the basis for their analyses. However, to estimate deforestation, they first resampled the original high-resolution data (30 meters x 30 meters) to lower-resolution data (250 meters x 25 meters). As a general rule, using higher-resolution data is better than lower-resolution data:

• With higher-resolution 30-meter data, a feature would have to be smaller than 30 meters by 30 meters to go unnoticed; and
• With lower-resolution 250-meter data, any feature smaller than 250 meters by 250 meters (i.e., 6.5 hectares) would go unnoticed.

The use of lower-resolution data can, of course, go in both directions: it could overcount forest cover by only detecting a continuous tract of forest rather than one with small patches of deforestation, or it could undercount forest cover by only detecting a vast expanse of agriculture with only small remaining fragments of forest remaining. What is clear, however, is that West et al.’s use of 250-meter resolution does not meet the minimum mapping unit that Verra requires for REDD projects (100 meters by 100 meters, or finer), and this likely led to a significant underestimation of deforestation in their synthetic controls, as Figure S1 in West et al. 2020 clearly shows.

West et al. 2023 used Hansen et al. (2013)’s tree cover loss data taken from the Global Forest Watch database, developed by the World Resources Institute, to estimate annual deforestation over the period 2001 to 2020 for the REDD projects and their synthetic controls. The use of these data for this purpose is highly questionable.

First, as the Global Forest Watch portal itself states, its dataset shows tree cover extent and its changes over time, with tree cover being defined as: “all vegetation taller than 5 meters in height. “Tree cover” is the biophysical presence of trees and may take the form of natural forests or plantations existing over a range of canopy densities”. For this reason, this dataset bears key limitations, presented here verbatim from the Global Forest Review webpage:

• Not all tree cover is a forest. Satellite data are effective for monitoring changes in tree cover, but forests are typically defined as a combination of tree cover and land use. For example, agricultural tree cover, such as an oil palm plantation, is not usually considered to be forest. As such, satellite-based monitoring systems may overestimate forest area unless combined with additional land-use data sets. No land-use data set currently exists at an adequate resolution or updated frequency to enable this analysis at global scale.
• Not all tree cover loss is deforestation. Defined as permanent conversion of forested land to other land uses, deforestation can only be identified at the moment trees are removed if it is known how the land will be used afterward. In the absence of a global data set on land use, it is not possible to accurately classify tree cover loss as permanent (i.e., deforestation) or temporary (e.g., where it is associated with wildfire, timber harvesting rotations, or shifting cultivation) at the time it occurs. However, new models analyzing spatial and temporal trends in tree cover loss are enabling better insights into the drivers of loss.
• Tree cover is a one-dimensional measure of a forest. Many qualities of a forest cannot be measured as a function of tree cover and are difficult, if not impossible, to detect from space using existing technologies. Forests that are vastly different in terms of form and function—such as an intact primary forest and a planted forest managed for timber production—are nearly indistinguishable in satellite imagery based on tree cover. Detecting forest degradation through remote sensing is also challenging because degradation often entails small changes occurring beneath the forest canopy.

Second, Hansen et al. (2013) improved their methodology by using finer resolution data and improved analytical method, such that:

• these changes lead to a different and improved detection of global forest loss. However, the years preceding 2011 have not yet been reprocessed in this manner, and users will notice inconsistencies as a result… The integrated use of version 1.0 2000–2012 data and updated version 1.7 2011–2019 data should be performed with caution”.

Notwithstanding this limitation, the authors used data from 2001 to 2020 apparently with no adjustment to make allowance for such changes, which makes their analysis and results questionable.

Third, in many forest types, Hansen et al. (2013)’s data yield inconsistent results, either overestimating or underestimating forest cover change. As the figure below indicates, a stand of open forest existed in February 2013 (top panel), with the remaining forest in February 2020 (middle panel); however, the forest loss detected by the Global Forest Watch dataset (black squares, lower panel) is much smaller than the amount of forest that was actually lost. This inability to detect real forest loss in a control area calls into question the usefulness of the exercise. Further, as the accuracy of this dataset has not been assessed at the local level, and it might vary significantly between localities, deforestation estimates using this data set are unreliable.

Figure 1. Area in Tanzania showing a stand of open forest in February 2013 (top panel), forest remaining in the same area as of February 2018 (middle panel), and forest loss (black squares) up to October 2019 (bottom panel). Top and middle panels © Google Earth. Bottom panel © Global Forest Watch dataset.

For these and other reasons, several studies, including some by the World Resources Institute itself (e.g., Harris et al., 2018; Bos et al. 2019; Chen et al. 2020), have explicitly stated that Hansen et al. (2013) data should not be used off-the-shelf to estimate deforestation or for REDD purposes, although they might be used for those purposes provided suitable adjustments are made first. Such elementary but crucial caution was neglected by West et al. 2023, rendering their results doubtful and their conclusions questionable.

The authors briefly acknowledged this shortcoming: “Many remote sensing studies highlight the differences in deforestation rates between [the Global Forest Watch] and the numbers officially recognized by governments. Such differences emerge from different mapping methodologies and definitions of deforestation and forest degradation”. However, the authors ignored the implications of this shortcoming.

Verra finds that the authors’ use of unsuitable data – whether being too crude (West et al. 2020) or not accommodating data uncertainty (West et al. 2023) – means that their estimates of deforestation in their synthetic controls are highly questionable.

Verra further finds that the questionable nature of their estimates of deforestation in their synthetic controls seriously undermines the authors’ conclusions about the REDD projects in question.