HSA-CNN: A Hybrid Spectral-Attention Multi-Agent Framework for Explainable Cloud Detection in Multispectral Remote Sensing Imagery

Abstract

Cloud detection is an essential preprocessing step in a remote sensing application. However, the presence of clouds and their shadows severely hamper the accuracy of the surface observations and the subsequent analysis. Reliable identification of thin clouds and cloud shadows is still a problem to which even state-of-the-art deep learning-based cloud detection methods have not provided a solution, because of spectral ambiguity, spatial variability, and the lack of awareness of uncertainty of the models. This work presents HSA-CNN, a hybrid spectral, attention, multi-agent deep learning framework that accurately and explainably identifies pixel-wise clouds in multispectral satellite imagery. The proposed architecture is a U-Net-based encoder-decoder architecture that is complemented by SpectralDirectionalKernel (SDK) blocks for multi-scale feature extraction, and integrates a set of specialized agents, including a transformer-based spectral attention agent, a MobileNet-based spatial context agent, a bidirectional LSTM-based temporal sequence agent, and a Bayesian uncertainty agent. This meta-agent orchestration mechanism performs confidence-aware, per-pixel expert selection and ensemble fusion, enabling robust predictions and reliable uncertainty estimation. The experimental results show that HSA and CNN can accurately classify four cloud categories: clear sky, thick cloud, thin cloud, and cloud shadow. Moreover, it significantly improves thin cloud discrimination and prediction stability. Furthermore, the framework can provide interpretable outputs via attention maps, agent-weight visualizations, and pixel-level uncertainty maps, which improve transparency and operational trust. The proposed method is a powerful and interpretable tool in remote sensing that can be employed for atmospheric correction, environmental monitoring, and climate analysis.