Predictive performance of silt content of tidal flats by using encoded features from a deep learning model

Sediment properties play an important role in determining the habitats for macrozoobenthic communities (e.g., bivalves, polychaetes and crustaceans) which are dominant secondary producers on tidal flats. Many studies found a positive relationship between benthic fauna density and silt content. Monitoring and mapping silt content through high-frequency and high-resolution remote-sensing images aids the sustainable management of coastal ecosystems. We present a novel approach to enhance the information available for mapping silt content from Sentinel-2 spectral bands (blue, green, red and near-infrared) in the Dutch Wadden Sea. We trained a Variational Auto Encoder (VAE) deep-learning model with image patches of tidal flats to reproduce the input images. The representative image information captured in the encoded layers of the VAE model is extracted as features. Together with the spectral bands, we used this additional information for the mapping of silt content on two tidal flats (Pinkegat and Zoutkamperlaag) of the Dutch Wadden Sea for three years (2018, 2019, 2020). The use of encoded features and spectral bands together, as opposed to the use of only the spectral bands, consistently increased the prediction results (coefficient of variation, R2) of a random forest model by 14% points on average. For Pinkegat, the silt content is predicted with an R2 of 0.6 while for Zoutkamperlaag an R2 of 0.5 was achieved, where variation in prediction results reflects field-data distribution. The approach of extracting features from a VAE model to act as complementary information in spectrally poor regions can be utilised in various remote-sensing applications.