Vision transformer-based automatic weather detection with adaptive transfer learning using Bayesian optimization and genetic algorithm

¹ LABGED Laboratory, Computer Science Department, Badji Mokhtar-Annaba University, P.O. Box 12, Annaba 23000, Algeria
² Computer Science Department, Badji Mokhtar-Annaba University, P.O. Box 12, Annaba 23000, Algeria

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Received: 30 January 2025
Accepted: 8 May 2026

Abstract

Recognizing weather patterns plays a vital role in numerous areas of daily life, including weather forecasting, transportation, agriculture, and forest management. Machine learning approaches, particularly Convolutional Neural Networks (CNNs), offer improved weather pattern analysis compared to traditional radar systems. However, CNNs struggle to capture multi-level dependencies without increasing model complexity. Recently, Vision Transformers (ViT), adapted from Natural Language Processing, have demonstrated superior performance in capturing global image relationships. This study presents a deep learning approach to accurately detect and classify weather conditions into multiple categories through transfer learning. To tackle the challenge of selecting the optimal number of trainable layers in CNNs and blocks in ViTs, an Adaptive Layer Freezing (ALF) technique is introduced, dynamically modifying the trainable layers to maximize efficiency during transfer learning with the help of Bayesian Optimization (BO). BO, widely recognized for its capability in hyperparameter tuning, ensures the ALF process operates with optimal configurations, enhancing both training efficiency and model accuracy. We compare BO with Genetic Algorithms (GA), a robust metaheuristic inspired by natural selection. Experimental results show that GA achieves slightly lower accuracy than BO. A new architecture based on ViT-L32 is evaluated against two CNN models – EfficientNetB0 and MobileNetV2 – using two weather imaging datasets (WEAPD and WCD), with ViT-L32 achieving the best classification results, attaining 98.28% accuracy for binary and 96.45% for multi-class classifications.