Objectives
Digital soil mapping (DSM) increasingly uses machine-learning approaches to predict the spatial distribution of soil properties from relationships between soil observations and environmental predictors. However, spatial dependence among observations can affect model evaluation and estimates of spatial generalizability. Under random data splitting, training and test observations may be geographically close or share similar spatial characteristics, resulting in performance estimates that may not represent predictions at spatially separated locations. Spatial cross-validation (SCV) can reduce this potential overlap by increasing spatial separation between training and test observations. Because soil properties can also vary at different spatial scales, the influence of the validation strategy may differ among properties. This study therefore investigated the effect of spatial structure on the evaluation of a Random Forest (RF) model for soil micronutrients by comparing random holdout and SCV strategies in the Maru’ak Dam area, Lorestan Province, Iran.
Methodology
The study area covers approximately 1804 ha along the Tireh River, northwestern Dorud County. A total of 166 surface-soil samples were collected at an approximate spacing of 250 m, and available Fe, Mn, Zn, and Cu were determined. Environmental predictors derived from a digital elevation model, remote-sensing indices, geological data, and land-use information were used to develop the RF model. Model performance was first evaluated using a random train-test split. The spatial autocorrelation range of each micronutrient was then estimated, and five spatial folds were used for SCV. Spatial block sizes were selected according to the spatial structure of the target variables while considering the number and spatial distribution of observations. Permutation-based variable importance was also used to assess the relative contribution of environmental predictors.
Results
Random train-test split validation produced different predictive performances among the micronutrients, with test-set R² values of 0.58, 0.38, 0.26, and 0.13 for Fe, Mn, Cu, and Zn, respectively. Their spatial autocorrelation ranges also differed markedly, reaching 7462 m for Mn, 263 m for Fe, 70 m for Cu, and 57 m for Zn.
Under SCV, mean R² values for Fe, Mn, Zn, and Cu were 0.65, 0.08, 0.18, and 0.21, respectively. The largest change occurred for Mn, with R² declining from 0.38 under random train-test split to 0.08 under SCV. The very large spatial autocorrelation range of Mn relative to the sampling interval may partly explain this difference, as random splitting increases the likelihood that training and test observations share similar spatial structures. In contrast, Fe showed an increase in R² from 0.58 to 0.65, while its autocorrelation range (263 m) was relatively close to the sampling interval. For Zn and Cu, whose autocorrelation ranges were shorter than the sampling interval, changes were comparatively limited: R² increased from 0.13 to 0.18 for Zn and decreased from 0.26 to 0.21 for Cu. These results suggest that the sensitivity of model evaluation to the validation strategy varies among micronutrients and may be related to their spatial scales of variability. Nevertheless, performance differences cannot be attributed solely to autocorrelation range, since the composition of training and test samples and the configuration of spatial folds may also affect validation results.
Permutation-based variable importance indicated that remote-sensing indices were among the most important predictors for the studied micronutrients. The leading predictors for Fe, Mn, Cu, and Zn were the normalized difference sand index (NDSI), salinity index (SI), carbonate index (),CaI and bare soil index (BSI), respectively. Their importance should not necessarily be interpreted as evidence of direct linear relationships with micronutrient concentrations; instead, these variables may provide complementary information on surface conditions, parent material, mineralogical characteristics, and other factors associated with micronutrient spatial variability.
Conclusion
The results demonstrate that the spatial structure of soil micronutrients can substantially affect the evaluation of machine-learning models in DSM. Differences between random train-test split and SCV, particularly for Mn, show that ignoring spatial dependence can produce different estimates of predictive performance. The smaller changes observed for properties with shorter autocorrelation ranges further suggest that the effect of validation strategy depends on the spatial characteristics of the target property. Therefore, validation in DSM should consider the spatial structure and scale of variability of the target soil property. Integrating machine-learning models with spatially informed validation can provide a more representative assessment of their spatial generalizability. |