Indonesian J our nal of Electrical Engineering and Computer Science V ol. 37, No. 1, January 2025, pp. 241 ∼ 249 ISSN: 2502-4752, DOI: 10.11591/ijeecs.v37.i1.pp241-249 ❒ 241 A comparati v e study of pr e-trained models f or image featur e extraction in weather image classication using orange data mining P afan Doungpaisan 1 , P eerapol Khunarsa 2 1 Department of Information T echnology , F aculty of Industrial T echnology and Management, King Mongkut’ s Uni v ersity of T echnology North Bangk ok, Bangsue, Thailand 2 Department of Data Science, F aculty of Science and T echnology , Uttaradit Rajabhat Uni v ersity , Uttaradit, Thailand Article Inf o Article history: Recei v ed Jan 28, 2024 Re vised Aug 20, 2024 Accepted Aug 31, 2024 K eyw ords: Deep learning Image classication Image embeddings Inception-V3 Machine learning Neural netw orks W eather classication ABSTRA CT This paper presents a detailed comparati v e analysis of pre-trained models for feature e xtraction in the domain of weather image classication. Utilizing the orange data mining toolkit, we in v estig ated the ef fecti v eness of six prominent pre-trained models-InceptionV3, SqueezeNet, V GG-16, V GG-19, painter , and DeepLoc-in accurately classifying weather phenomena images. Among these models, InceptionV3, in conjunction with neural netw orks, emer ged as the most ef fecti v e, achie vi ng a classication accurac y (CA) of 96.1%. P ainter and SqueezeNet also sho wed strong performance, with accuracies of 95.1% and 86.7%, respecti v ely , although the y wer e surpassed by InceptionV3. V GG-16 and V GG-19 pro vided moderate accurac y , while DeepLoc underperformed sig- nicantly with a maximum accurac y of 56%. Neural ne tw orks consistently out- performed other classiers across all models. This study highlights the critical importance of sel ecting appropriate pre-trained models to enhance the accurac y and reliability of weather image classication systems. This is an open access article under the CC BY -SA license . Corresponding A uthor: Peerapol Khunarsa Department of Data Science, F aculty of Science and T echnology , Uttaradit Rajabhat Uni v ersity Uttaradit, 53000 Thailand Email: peerapol@uru.ac.th 1. INTR ODUCTION W eather image classication in v olv es automatically cate gorizing images based on depicted weather conditions, such as sunn y , cloudy , rain y , sno wy , foggy , and stormy [1]. This task necessitates the de v elopment of sophisticated machine learning models, particularly con v olutional neural netw orks (CNNs), that can accu- rately recognize and classify di v erse weather patterns. The challenge lies in designing algorithms that reliably analyze weather -related imagery , pro viding precise labels essential for i mpro ving weather forecasting accurac y , which is crucial for disaster preparedness and response [2]-[4]. Accurate weather image classication is vital across v arious sectors. In transportation, it enhances road safety by enabling v ehicles to detect and adapt to changing weather conditions [5], [6]. In agriculture, it impro v es crop management and irrig ation through ef fecti v e monitoring of weather conditions [7], [8]. En viron- mental monitoring agencies rely on weather classication for tracking air quality and other ecological f actors, contrib uting to en vironmental protection [9], [10]. Additionally , it is crucial for assessing the impact of weather on critical infrastructure, ensuring safety and durability [11], [12]. J ournal homepage: http://ijeecs.iaescor e .com Evaluation Warning : The document was created with Spire.PDF for Python.
242 ❒ ISSN: 2502-4752 Adv anced techniques are emplo yed to address the challenges of weather image classication. CNNs are widely adopted due to their procienc y in learning hierarchical features from images [2], [3]. T ransfer learning, where pre-trained CNN models are ne-tuned [3], [4], and ensemble learning methods, which com- bine multiple models, further enhance classication performance [5], [6]. Recurrent neural netw orks (RNNs) are pi v otal in capturing temporal dependencies in sequences of weather images [13], [14]. Data augmenta- tion techniques, such as rotations and translations, help address limited training data by generating additional samples [7], [8]. Spatial-temporal models, attention mechanisms, and the fusion of data from di v erse sources, including satellite imagery and radar da ta, contrib ute to more ac curate predictions [15]-[19]. Semi-supervised and weakly supervised learning approaches le v erage both labeled and unlabeled data, especially i n scenarios with limited labeled data [20]-[22]. The selection of suit able techniques depends on spe cic problem require- ments, dataset characteristics, computational resources, and desired accurac y . Continuous e xploration of no v el methodologies and eng agement with the latest research de v elopments are essential for enhancing the ef cac y of weather image classication systems, dri ving inno v ation, and adv ancing the eld. 2. RELA TED W ORK The classication and prediction of weather conditions based on image data has become an increas- ingly important eld of research, dri v en by adv ancements in deep learning and computer vision. Accurate weather classication not only impro v es fore casting accurac y b ut also enhances v arious applications, such as autonomous dri ving, en vironmental monitoring, and disaster management. T raditional methods of weather pre- diction rely on comple x meteorological models, satellite data, and human e xpertise, which are often resource- intensi v e and not al w ays practical for real-time applications [23]. In contras t, recent de v elopments in machine learning, particularly CNNs, ha v e demonstrated signicant potential for automating and impro ving weather classication tasks [23], [24]. Deep learning techniques, especially CNNs, ha v e been widely used in image classication tasks, and their application in weather image classication has g ained substantial attention. F or instance, Cao and Y ang [23] de v eloped a CNN-based model for weather classication that outperformed se v eral state-of-the- art methods, achie ving a rem arkable 98% accurac y . Th e ir approach utilized data augmentation techniques to o v ercome the challenges posed by imbalanced datasets and v arying weather conditions. Similarly , Li et al. [24] introduced a vision transformer (V iT)-based model for multi-class weather image classication, demonstrating the adv antages of using transformer architectures o v er traditional CNNs. Their model achie v ed an impressi v e accurac y of 92.83% across 14 dif ferent weather classes [25]. Moreo v er , h ybrid models that combine CNNs with other ma chine learning techniques, such as support v ector machines (S VMs), ha v e been e xplored to enhance classication performance. F or e xample, a study by T ri v a et al. [26] de v eloped a CNN-SVM h ybrid model that impro v ed classication accurac y (CA) for v arious weather conditions, particularly in adv erse en vironments lik e fog and rain. This h ybrid approach pro v ed ef fecti v e in applications such as autonomous dri ving, where precise weather condition recognition is critical for safety [26]. Be yond CNNs, the use of cloud image classication for weather prediction has also emer ged as a promising approach. Cao and Y ang [23] de v eloped a mobile application that utilizes CNNs to classify cloud images into four cate gories: rain clouds, f air -weather clouds, tornado clouds, and fog. Their system demon- strated high accurac y in identifying rain and fog, though further impro v ements were needed for tornado pre- diction [23]. This de v elopment reects the gro wing interest in portable, real-time weather prediction tools that le v erage modern smartphone capabilities. Other researchers ha v e also focused on the challenges of weather classication in specic domains, such as v ehicle en vironments. In one such study , a CNN model w as designed to classify weather conditions from front-vie w camera images in autonomous v ehicles. The m od e l achie v ed high performance, with an F1 measure of 98.3%, demonstrating its potential for real-w orld applications in autonomous dri ving [26]. Ov erall, the inte gration of deep learning techniques, particularly CNNs, with weather image data has re v olutionized the eld of weather classication. This study focuses on e v aluating pre-trained models for feature e xtraction in weather image classi- cation using orange data mining. Models such as InceptionV3, V GG16, V GG19, SqueezeNet, painter , and DeepLoc were assessed, with InceptionV3 consistently outperforming others across v arious classiers, under - scoring its ef fecti v eness for this task. Indonesian J Elec Eng & Comp Sci, V ol. 37, No. 1, January 2025: 241–249 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 243 3. MA TERIAL AND METHODS This study uses the weather phenomenon image database (WEAPD), de v eloped by Xiao et al. [27], as the main dataset. WEAPD consists of 6,877 images, which are or g anized into 11 dif ferent cate gories as sho wn in Figure 1, including (a) de w , (b) fog/smok e, (c) frost, (d) glaze, (e) hail, (f) lightning, (g) rain, (h) rainbo w , (i) frost, (j) sandstorm, and (k) sno w , which are classied based on their visual shapes and color attrib utes. All e xample images are presented in Figure 1. T o f acilitate comprehensi v e model e v aluation, K-fold cross v alidation is applied, which di vides the dataset into se v eral subsets for training and v alidation. W eather image classication w as performed using the orange data mining toolkit , an open-source platform de v eloped in Python at the Uni v ersity of Ljubljana, Slo v enia [28]. Orange supports v arious machine learning tasks, including re gression, classication, and clustering, making it ideal for educational use and prototyping. (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) Figure 1. The representati v e e xamples of 11 WEAPD; (a) de w , (b) fog/smog, (c) frost, (d) glaze, (e) hail, (f) lightning, (g) rain, (h) rainbo w , (i) rime, (j) sandstorm, and (k) sno w In this study , pre-trained models such as InceptionV3, V GG16, V GG19, SqueezeNet, painter , and DeepLoc were emplo yed for image embedding to generate v ector representations of each image. These em- beddings were then used to train classiers including neural netw orks, K-ne arest neighbors (KNN), and SVM. T ransfer l earning techniques were applied, le v eraging limited data from fruits and v e getables to enhance model accurac y and reduce data requirements [29]-[34]. Model performance w as assessed using orange’ s test and score widget, which calculated metrics lik e accurac y , precision, recall, and F1-score. The use of K-fold cross v alidation enhanced the rob ustness of model e v aluation by pro viding comprehensi v e metrics such as CA, A UC, and F1-score across dif ferent test folds. Detailed e xperiment parameters are presented in T able 1. T able 1. Ev aluating the classication of weather phenomenon image using InceptionV3 as a feature Model Algorithm parameter Neural netw ork Neural in hidden layer = 100 Acti v ation = ReLu Solv er = Adam Re gularization, α = 0.0001 Maximal number of iterations = 200 KNN Euclidean distance, Manhattan distance, Chebyshe v distance Number of neighbors = 5 SVM (Linear) K ernel = Linear , Polynomial, RBF , Sigmoid Optimization parameters (Numerical tolerance = 0.0010) Optimization parameters (Iteration limit = 100) A compar ative study of pr e-tr ained models for ima g e featur e ... (P afan Doungpaisan) Evaluation Warning : The document was created with Spire.PDF for Python.
244 ❒ ISSN: 2502-4752 4. RESUL TS AND DISCUSSION A systematic empirical e v aluation of multiple classication approaches w as conducted for weather image cate gorization using the WEAPD [27]. The pre-trained InceptionV3 model played a pi v ota l role in our approach, serving as a feature e xtractor within the Orange toolkit, generating v ector representations for each image, and enhancing the feature space for classication. Among the v arious machine learning classiers applied, neural netw orks demonstrated superior performance in weather image classication. The inte gration of InceptionV3 embeddings with neural netw orks resulted in high accur ac y and precision, with the neural netw ork model e xcelling in capturing comple x patterns within the dataset. The e xperimental results, summarized in T able 2, indicate that the combination of InceptionV3 for feature e xtraction and classiers such as neural netw orks pro vides a successful strate gy for high-performance weather image classication. This approach enabled ef cient feature e xtraction and demonstrated the potential of adv anced classiers to achie v e accurate cate gorization of weather phenomena with a high le v el of precision. Specically , the neural netw ork model with InceptionV3 embeddings achie v ed an A UC of 0.971, a CA of 96.1%, an F1-score of 0.778, and a matthe ws correlation coef cient (MCC) of 0.757, indicating a strong and balanced classication performance. Other classiers, such as KNN with v arious distance met rics, and SVM with dif ferent k ernels, sho wed v arying le v els of performance. KNN models, while achie ving respectable results, e xhibited lo wer precision and recall balance compared to neural netw orks. The SVM models, particularly with Polynomial and RBF k ernels, performed well, b ut still fell short of the neural netw ork model in terms of o v erall accurac y and precision. Ov erall, the neural netw ork model with InceptionV3 embeddings pro v ed to be the most ef fecti v e approach for weather image classication, surpassing other classiers in terms of both accurac y and precision. This study emphasizes the ef fecti v eness of combining pre-trained models for feature e xtraction with adv anced classiers, highlighting the c ritical import ance of selecting appropriate models and metrics for achie ving high- performance image classication T able 2. Performance metrics for weather classication using InceptionV3 embeddings as a feature Model Are a under Classication F1-score Precision Recall MCC an R OC curv e accurac y Neural netw ork 0.971 0.961 0.778 0.792 0.765 0.757 KNN (Euclidean distance) 0.889 0.940 0.603 0.757 0.501 0.586 KNN (Manhattan distance) 0.886 0.941 0.603 0.773 0.494 0.590 KNN (Chebyshe v distance) 0.841 0.930 0.503 0.702 0.391 0.491 SVM (Linear) 0.962 0.952 0.723 0.762 0.688 0.698 SVM (Polynomial) 0.966 0.958 0.769 0.761 0.778 0.746 SVM (RBF) 0.968 0.959 0.773 0.773 0.773 0.750 SVM (Sigmoid) 0.949 0.934 0.685 0.600 0.797 0.657 4.1. Comparison with other image embeddings In our e xperiments, Incept ionV3 w a s used for image embedding feature e xtraction. W e performed a detailed comparati v e analysis with other methods, including SqueezeNet, V GG-16, V GG-19, painters, and DeepLoc, to e v aluate the ef fecti v eness of InceptionV3 relati v e to these alternati v e techniques. This comparison aimed to assess the reliability and rob ustness of InceptionV3 in the conte xt of image feature e xtraction. By e xamining these dif ferent methods, we aimed to better understand the performance and adv antages of Incep- tionV3 compared to other feature e xtraction methodologies. The architecture is illustrated in Figure 2. 4.1.1. Experimental r esults using SqueezeNet embeddings The e xperimental results, as presented in T able 3, pro vide a thorough e v aluation of v arious clas siers utilizing SqueezeNet e mbeddings for weather classication. Among the tested models, the neural netw ork classier emer ged as the optimal choice, achie ving the highest accurac y of 86.7% across k e y e v aluation metrics, including A UC, F1-score, precision, recall, and MCC. This highlights the strong syner gy betwee n the neural netw ork architecture and the features e xtracted by SqueezeNet. KNN classiers, despite using dif ferent distance metrics, achie v ed similar performance, with accura- cies around 77-78%. Ho we v er , these models were outperformed by the neural netw ork classier , indicating that while KNNs performed reasonably well, the y were less ef fecti v e in le v eraging SqueezeNet embeddings. SVM classiers demonstrated v aried performance depending on the k ernel used. Polynomial and RBF k ernels were the most competiti v e, achie ving an accurac y of 78%, comparable to the best KNN models. Indonesian J Elec Eng & Comp Sci, V ol. 37, No. 1, January 2025: 241–249 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 245 In contrast, the linear k ernel lagged behind with an accurac y of 71.4%, and the Sigmoid k ernel performed poorly , with a signicantly lo wer accurac y of 37.4%. This suggests that the Sigmoid k ernel is not well-suited for SqueezeNet features in the conte xt of weather classication. Ov erall, the neural netw ork classier w as the most ef fecti v e model when paired with SqueezeNet embeddings, with SVM classiers using polynomial or RBF k ernels being the closest competitors. KNN models, while performing adequately , did not match the accurac y of neural netw orks or the top-performing SVM k ernels. The Sigmoid SVM k ernel notably underperformed, reinforcing the importance of selecting appropriate k ernels for optimal classication results. Figure 2. Structure for utilizing orange data mining in image classication T able 3. Ev aluating the classication of weather phenomenon image using SqueezeNet embeddings as features Model Area under Classication F1-score Precision Recall MCC an R OC curv e accurac y Neural netw ork 0.983 0.847 0.846 0.846 0.847 0.829 KNN (Euclidean distance ) 0.947 0.777 0.771 0.776 0.777 0.752 KNN (Manhattan distance ) 0.946 0.778 0.772 0.777 0.778 0.752 KNN (Chebyshe v distance) 0.938 0.752 0.743 0.751 0.752 0.724 SVM (Linear) 0.967 0.714 0.714 0.719 0.714 0.683 SVM (Polynomial) 0.977 0.783 0.780 0.781 0.783 0.759 SVM (RBF) 0.977 0.785 0.782 0.786 0.785 0.761 SVM (Sigmoid) 0.870 0.374 0.397 0.500 0.374 0.319 4.1.2. Experimental r esults using V GG-16 embeddings The e xperimental results in T able 4 present a comprehensi v e analysis of classier performance when paired with V GG-16 embeddings for weather classication. Among the models tested, the neural netw ork clas- sier emer ged as the top performer , achie ving the highest accurac y of 84% across k e y m etrics, including A UC, F1-score, precision, recall, and MCC. This indicates a strong syner gy between the neural netw ork architecture and V GG-16 features, demonstrating the ef fecti v eness of this combination for accurate weather classication. KNN classiers, while performing reasonably well with accuracies in the range of 74-75%, were outperformed by the neural netw ork model. The v ariations in distance metrics had a mar gina l impact on KNN performance, b ut these models did not match the accurac y le v els of neural netw orks when utilizing V GG-16 embeddings. SVM classiers e xhibited v aried performance based on the k ernel used. Polynomial and RBF k ernels were the most competiti v e among SVM s, achie ving an accurac y of around 76%, comparable to the best KNN models. Ho we v er , the linear k ernel sho wed lo wer accurac y at 71.2%, and the Sigmoid k ernel signicantly underperformed, with an accurac y of 57.8%, indicating its unsuitability for V GG-16 features in this task. Ov erall, the neural netw ork classier pro v ed to be the most ef fecti v e model when paired with V GG- 16 embeddings, with SVMs using polynomial or RBF k ernels as the closest competitors. KNN classiers, al- though performing adequately , did not achie v e the same le v el of accurac y as neural netw orks or top-performing SVM k ernels. The Sigmoid SVM k ernel once ag ain demonstrated signicant weaknesses in this conte xt. A compar ative study of pr e-tr ained models for ima g e featur e ... (P afan Doungpaisan) Evaluation Warning : The document was created with Spire.PDF for Python.
246 ❒ ISSN: 2502-4752 T able 4. Ev aluating the classication of weather phenomenon image using V GG-16 embeddings as features Model Area under Classication F1-score Precision Recall MCC an R OC curv e accurac y Neural netw ork 0.982 0.840 0.840 0.841 0.840 0.822 KNN (Euclidean distance ) 0.942 0.754 0.749 0.763 0.754 0.727 KNN (Manhattan distance ) 0.941 0.747 0.741 0.756 0.747 0.719 KNN (Chebyshe v distance) 0.939 0.734 0.728 0.745 0.734 0.705 SVM (Linear) 0.969 0.712 0.712 0.718 0.712 0.680 SVM (Polynomial) 0.975 0.764 0.763 0.776 0.764 0.739 SVM (RBF) 0.974 0.760 0.761 0.771 0.760 0.734 SVM (Sigmoid) 0.939 0.578 0.573 0.649 0.578 0.545 4.1.3. Experimental r esults using V GG-19 embeddings The e xperimental results in T able 5 sho w that the neural netw ork classier performed best with V GG- 19 embeddings, making it the most ef fecti v e choice for weather classication. The neural netw ork achie v ed an accurac y of 82% and e xcelled across all e v aluation metrics, including A UC, F1-score, precision, recall, and MCC. KNN classiers, re g ardles s of the distance metric used, sho wed moderate performance with ac curacies ranging from 71% to 74%. The choice of distance metric had little impact on the KNN results. SVM classiers with polynomial, RBF , and linear k ernels achie v ed similar accuracies around 71%, close to those of the KNN classiers. Ho we v er , the Sigmoid k ernel underperformed signicantly , with an accurac y of 55%, indicating it w as not well-suited for use with V GG-19 features. In summary , the neural netw ork w as the top-performing model with V GG-19 embeddings, achie ving an accurac y of 82%. SVM and KNN classiers had lo wer performance, with accuracies in the lo w 70% range, while the Sigmoid k ernel w as notably less ef fecti v e in this conte xt. T able 5. Ev aluating the classication of weather phenomenon image using V GG-19 embeddings as features Model Area under Classication F1-score Precision Recall MCC an R OC curv e accurac y Neural netw ork 0.978 0.820 0.820 0.822 0.820 0.799 KNN Euclidean 0.939 0.744 0.738 0.750 0.744 0.715 KNN Manhattan 0.928 0.788 0.732 0.746 0.738 0.708 KNN Chebyshe v 0.929 0.714 0.708 0.722 0.714 0.682 SVM linear 0.967 0.714 0.712 0.717 0.714 0.682 SVM Polynomial 0.968 0.711 0.703 0.750 0.711 0.686 SVM RBF 0.968 0.712 0.705 0.743 0.712 0.686 SVM Sigmoid 0.924 0.550 0.547 0.635 0.550 0.514 4.1.4. Experimental r esults using painter embeddings The e xperimental results in T able 6 demonstrate that the neural netw ork classier achie v ed the best performance with painter embeddings for weather classication, with an accurac y of 95.1% and high scores across all e v aluation metrics, including A UC, F1-score, precision, recall, and MCC. This suggests a strong compatibility between the neural netw ork and painter embeddings for accurate weather classication. KNN classiers also perform ed well, with the Manhattan distance metric achie ving an accurac y of 93.8%, slightly higher than the Euclidean metric at 93.6%. The Chebyshe v metric w as less ef fecti v e, resulting in an accurac y of 92.4. SVM classiers performed competiti v ely with painter embeddings, with polynomial, RBF , and line ar k ernels achie ving accuracies between 94% and 95%. The polynomial k ernel achie v ed the highest accurac y among the SVM classiers at 94.6%, while the Sigmoid k ernel had a lo wer accurac y of 90%, indicating it w as less suitable for this task. In summary , neural netw orks, SVM k ernels, and KNN classiers all achie v ed st rong performance abo v e 90% accurac y with painter embeddings. The neural netw ork classier w as the top performer with an accurac y of 95.1%, highlighting the importance of selecting suitable classiers and distance metrics based on the data’ s characteristics. 4.1.5. Experimental r esults using DeepLoc embeddings The e xperimental res ults in T able 7 s ho w that DeepLoc embeddings were less ef fecti v e for weather classication compared to other embedding methods. The neural netw ork classier performed the best with Indonesian J Elec Eng & Comp Sci, V ol. 37, No. 1, January 2025: 241–249 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 247 T able 6. Ev aluating the classication of weather phenomenon image using painter embeddings as features Model Area under Classication F1-score Precision Recall MCC an R OC curv e accur ac y Neural netw ork 0.961 0.951 0.724 0.744 0.704 0.697 KNN Euclidean 0.887 0.936 0.579 0.720 0.485 0.559 KNN Manhattan 0.886 0.938 0.590 0.741 0.494 0.570 KNN Chebyshe v 0.829 0.924 0.450 0.647 0.345 0.436 SVM linear 0.949 0.943 0.675 0.690 0.660 0.644 SVM Polynomial 0.951 0.946 0.694 0.715 0.675 0.665 SVM RBF 0.953 0.944 0.692 0.687 0.697 0.661 SVM Sigmoid 0.930 0.900 0.585 0.468 0.781 0.555 DeepLoc embeddings, achie ving an accurac y of 56%, b ut this w as still signicantly lo wer than accuracies obtained with other embeddings, which ranged from 86% to 95%. KNN classiers sho wed e v en lo wer accurac y , around 47%, re g ardless of the distance metric used. SVM classiers performed poorly as well, with accuracies ranging from 22.8% to 47.4%, depending on the k ernel. These resul ts suggest that DeepLoc embeddings are not well-suited for weather classication tasks. T able 7. Ev aluating the classication of weather phenomenon image using DeepLoc embeddings as features Model Area under Classication F1-score Precision Recall MCC an R OC curv e accur ac y Neural netw ork 0.886 0.560 0.559 0.559 0.560 0.508 KNN Euclidean 0.816 0.474 0.468 0.473 0.474 0.413 KNN Manhattan 0.817 0.477 0.470 0.477 0.477 0.417 KNN Chebyshe v 0.800 0.439 0.432 0.437 0.439 0.374 SVM Linear 0.816 0.474 0.468 0.473 0.474 0.413 SVM Polynomial 0.789 0.272 0.231 0.360 0.272 0.222 SVM RBF 0.843 0.379 0.367 0.413 0.379 0.319 SVM Sigmoid 0.746 0.228 0.183 0.262 0.228 0.154 4.2. Comparati v e analysis of experimental ndings acr oss v arious image embeddings The e xperimental outcomes, as detailed in T ables 2-7, indicate that the neural netw ork classier con- sistently surpassed other classiers across all tested image embeddings, establishing it as the most ef fecti v e approach for weather image classication. The highest accurac y w as observ ed with the InceptionV3 embed- ding, achie ving 96.1%, closely follo wed by the painter embedding at 95.1%. The SqueezeNet, V GG-16, and V GG-19 models yielded accuracies of 86.7%, 84%, and 82%, respecti v ely . SVM with polynomial or RBF k ernels emer ged as the most competiti v e alternati v es to neural netw orks, outperforming the KNN classiers. Although KNN achie v ed reasonably good accurac y , its performance w as inferior to that of SVM and neural netw orks. Notably , the DeepLoc embedding w as an outli er , with the neural netw ork classier attaining only 56% accurac y , signifying that this embedding w as less suitable for weather classication across all classiers. W ithin the KNN v ariants, the choice of distance metric generally did not e x ert a signicant inuence on performance, although in certain instances, the Manhattan distance pro vided a mar ginal impro v ement o v er Euclidean distance. The Sigmoid k ernel for SVM consistently underperformed across all embeddings, with accurac y dropping as lo w as 22.8%. In summary , neural netw orks emer ged as the most ef fecti v e classier across all image embeddings, with InceptionV3 achie ving the highest accurac y at 96 .1%. While SVM and KNN classiers produced competiti v e results , the y were generally outperformed by neural netw orks. The DeepLoc embedding w as found to be unsuitable for this classication task across all e v aluated classiers. 5. CONCLUSION In this study , we compared se v eral pre-trained models for feature e xtraction in weather image classi - cation using the o r ange data mining toolkit. The models e v aluated were InceptionV3, SqueezeNet, V GG-16, V GG-19, painter , and DeepLoc, which were tested with neural netw orks, KNN, and SVM using dif ferent k er - nels. Our ndings sho w that InceptionV3 w as the most ef fecti v e model, achie ving the highest CA of 96.1% when paired with neural netw orks. It also e xcelled in other e v aluation metrics such as A UC, F1-score, pre- cision, recall, and MCC. P ainter and SqueezeNet also performed well, with accuracies of 95.1% and 86.7% A compar ative study of pr e-tr ained models for ima g e featur e ... (P afan Doungpaisan) Evaluation Warning : The document was created with Spire.PDF for Python.
248 ❒ ISSN: 2502-4752 respecti v ely , while V GG-16 and V GG-19 were less ef fecti v e wi th accuracies of 84% and 82%. DeepLoc sig- nicantly lagged behind, with only a 56% accurac y . Additionally , neural netw orks were cons istently superior to KNN and SVM cl assiers across all mod- els. Among SVM k ernels, Polynomial and RBF k ernels were competiti v e, b ut still not as ef fecti v e as neural netw orks. The Sigmoid k ernel underperformed, emphasizing its unsuitability for this task. This analysis high- lights the importance of choosing the right pre-trained model for enhancing CA in weather image classication tasks. The optimal performance achie v ed by InceptionV3 and neural netw orks suggests this combination as the best approach for similar machine learning applications. REFERENCES [1] N. A. Mashudi, N. Ahmad, S. M. Sam, N. Mohamed, and R. Ahmad, “Deep learning approaches for weather image recognition in agriculture, ” in Pr oceedings of the 2022 IEEE Symposium on Futur e T elecommunication T ec hnolo gies, SOFTT 2022 , No v . 2022, pp. 72–77, doi: 10.1109/SOFTT56880.2022.10010161. [2] Z. Dong, R. Zhang, and X. Shao, “ A CNN-RNN h ybrid model with 2D w a v ele t transform layer for image classication, ” in Pr o- ceedings - International Confer ence on T ools with Articial Intellig ence , ICT AI , No v . 2019, v ol. 2019-No v em, pp. 1050–1056, doi: 10.1109/ICT AI.2019.00147. [3] B . Gupta, R. Sharma, R. Bansal, G. K. Soni, P . Ne gi, and P . Purdhani, “ An adapti v e system for predicting student attenti v eness in online classrooms, ” Indonesian J ournal of Electrical Engineering and Computer Science (IJEECS) , v ol. 31, no. 2, p. 1136, Aug. 2023, doi: 10.11591/ijeecs.v31.i2.pp1136-1146. [4] E. Camporeale, “The challenge of machine learning in space weather: no wcasting and forecasting, ” Space W eather , v ol. 17, no. 8, pp. 1166–1207, Aug. 2019, doi: 10.1029/2018SW002061. [5] V . K ukreja, R. Sharma, and R. Y ada v , “Multi-weather classication using deep learning: a CNN-SVM amalg amated approach, ” in 2023 W orld Confer ence on Communication and Computing , WCONF 2023 , Jul. 2023, pp. 1–5, doi: 10.1109/WCONF58270.2023.10235097. [6] B. Kim and D. Suh, “ A h ybrid spatio-temporal prediction model for solar photo v oltaic generation using numerical weather data and satellite images, ” Remote Sensing , v ol. 12, no. 22, pp. 1–21, No v . 2020, doi: 10.3390/rs12223706. [7] T . Selz and G. C. Craig, “Can articial intelligence-based weather prediction models simulate the b uttery ef fect?, ” Geophysical Resear c h Letter s , v ol. 50, no. 20, Oct. 2023, doi: 10.1029/2023GL105747. [8] W . F ang, Q. Xue, L. Shen, and V . S. Sheng, “Surv e y on the application of deep learning in e xtreme weather prediction, ” Atmospher e , v ol. 12, no. 6, p. 661, May 2021, doi: 10.3390/atmos12060661. [9] A. McGo v ern et al. , “ A re vie w of machine learning for con v ecti v e weather , ” Articial Intellig ence for the Earth Systems , v ol. 2, no. 3, Jul. 2023, doi: 10.1175/aies-d-22-0077.1. [10] M. Ahmadi, M . Khashei, and N. Bakhtiarv and, “Enhancing air quality classication using a no v el discrete learning-based multilayer perceptron model (DMLP), ” International J ournal of En vir onmental Science and T ec hnolo gy , Aug. 2024, doi: 10.1007/s13762-024- 06017-5. [11] J. Li et al. , “Deep discriminati v e representation learning with attention map for scene classication, ” Remote Sensing , v ol. 12, no. 9, p. 1366, Apr . 2020, doi: 10.3390/RS12091366. [12] M. T orbicki, “Longtime prediction of climate-weather change inuence on critical infrastructure safety and resilience, ” in IEEE International Confer ence on Industri al Engineering and Engineering Mana g ement , Dec. 2018, v ol. 2019-Decem, pp. 996–1000, doi: 10.1109/IEEM.2018.8607308. [13] Y . E. Cebeci, “ A recurrent neural netw ork model for weather forecasting, ” in UBMK 2019 - Pr oceedings, 4th International Confer - ence on Computer Science and Engineering , Sep. 2019, pp. 591–595, doi: 10.1109/UBMK.2019.8907196. [14] B. Zhao, X. Li, X. Lu, and Z. W ang, “ A CNN–RNN architecture for multi-label weather recognition, ” Neur ocomputing , v ol. 322, pp. 47–57, Dec. 2018, doi: 10.1016/j.neucom.2018.09.048. [15] F . O’Donncha, Y . Hu, P . P almes, M. Burk e, R. Filgueira, and J. Grant, “ A spatio-temporal LSTM model to forecast across multiple temporal and spatial scales, ” Ecolo gical Informatics , v ol. 69, p. 101687, Jul. 2022, doi: 10.1016/j.ecoinf.2022.101687. [16] L. Xiang, J. Xiang, J. Guan, L. Zhang, Z. Cao, and J. Xia, “Spatiotemporal forecasting model based on h ybrid con v olution for local weather prediction post-processing, ” F r ontier s in Earth Science , v ol. 10, Sep. 2022, doi: 10.3389/feart.2022.978942. [17] T . K. Ga w ali and S. S. Deore, “Spatio-tempor al transportation images classication based on light and weather conditions, ” Inter - national J ournal of Intellig ent Systems and Applications in Engineering , v ol. 12, no. 13s, pp. 150–154, 2024. [18] D. P . Nik ezi ´ c, U. R. Ramadani, D. S. Radi v oje vi ´ c, I. M. Lazo vi ´ c, and N. S. Mirk o v , “Deep learning model for global spatio-temporal image prediction, ” Mathematics , v ol. 10, no. 18, p. 3392, Sep. 2022, doi: 10.3390/math10183392. [19] S. Shan, C. Li, Y . W ang, S. F ang, K. Zhang, and H. W ei, “ A deep learning model for multi-modal spatio-temporal irradiance forecast, ” Expert Systems with Applications , v ol. 244, p. 122925, Jun. 2024, doi: 10.1016/j.esw a.2023.122925. [20] W . Jiang, K. Huang, J. Geng, and X. Deng, “Multi-scale metric learning for fe w-shot learning, ” IEEE T r ansactions on Cir cuits and Systems for V ideo T ec hnolo gy , v ol. 31, no. 3, pp. 1091–1102, Mar . 2021, doi: 10.1109/TCSVT .2020.2995754. [21] X. Zheng et al. , “Hyperspectral image classication with imbalanced data based on semi-supervised learning, ” Applied Sciences (Switzerland) , v ol. 12, no. 8, 2022, doi: 10.3390/app12083943. [22] Y . C. Hao and Y . Zhu, “W eather classication for multi-class weather image based on CNN, ” in Pr oceedings - 2022 Inter - national Confer ence on Mac hine Learning and Intellig ent Systems Engineering , MLISE 2022 , Aug. 2022, pp. 363–366, doi: 10.1109/MLISE57402.2022.00079. [23] Y . Cao and H. Y ang, “W eather prediction using cloud’ s images, ” in Pr oceedings - 2022 International Confer ence on Big Data, Information and Computer Network, BDICN 2022 , Jan. 2022, pp. 820–823, doi: 10.1109/BDICN55575.2022.00157. [24] S. Li, W . T ian, X. W u, C. T an, and L. Cui, “Classication of multi-class weather image data, ” in Pr oceedings - 2023 9th International Symposium on System Security , Safety , and Reliability , ISSSR 2023 , Jun. 2023, pp. 203–207, doi: 10.1109/ISSSR58837.2023.00038. Indonesian J Elec Eng & Comp Sci, V ol. 37, No. 1, January 2025: 241–249 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 249 [25] J. Deng, Z. Liu, J. Zhang, and H. Bi, “CNN based tar get classication frame w ork based on comple x sparse sar image: initial result, ” in International Geoscience and Remote Sensing Symposium (IGARSS) , Jul. 2022, v ol. 2022-July , pp. 2259–2262, doi: 10.1109/IGARSS46834.2022.9883371. [26] J. T ri v a, R. Grbic, M. Vranjes, and N. T eslic, “W eather condition classication in v ehicle en vironment based on front-vie w cam- era images, ” in 2022 21st International Sympos ium INFO TEH-J AHORIN A (INFO TEH) , Mar . 2022, pp. 1–4, doi: 10.1109/IN- FO TEH53737.2022.9751279. [27] H. Xiao, F . Zhang, Z. Shen, K. W u, and J. Zhang, “Classication of weather phenomenon from images by using deep con v olutional neural netw ork, ” Earth and Space Science , v ol. 8, no. 5, May 2021, doi: 10.1029/2020EA001604. [28] J. Dem ˇ sar et al. , “Orange: data mining toolbox in p ython, ” J ournal of Mac hine Learning Resear c h , v ol. 14, pp. 2349–2353, 2013. [29] S. Gupta and M. K. Gupta, “ A comparati v e analysis of deep learning approaches for predicting breast cancer survi v ability , ” Ar c hives of Computational Methods in Engineering , v ol. 29, no. 5, pp. 2959–2975, Aug. 2022, doi: 10.1007/s11831-021-09679-3. [30] F . Shahidi, S. M. Daud, H. Abas, N. A. Ahmad, and N. Maarop, “Breast cancer classication using deep learning approaches and histopathology image: a comparison study , ” IEEE Access , v ol. 8, pp. 187531–187552, 2020, doi: 10.1109/A CCESS.2020.3029881. [31] S. Mascarenhas and M. Ag arw al, “ A comparison between V GG16, V GG19 and ResNet50 architecture frame w orks for Image Classication, ” in Pr oceedings of IEEE International Confer ence on Disruptive T ec hnolo gies for Multi-Disciplinary Resear c h and Applications, CENTCON 2021 , No v . 2021, pp. 96–99, doi: 10.1109/CENTCON52345.2021.9687944. [32] S. R. Shah, S. Qadri , H. Bibi, S. M. W . Shah, M. I. Sharif, and F . Marinello, “Comparing InceptionV3, V GG 16, V GG 19, CNN, a nd ResNet 50: a case study on early detection of a rice disease, ” Agr onomy , v ol. 13, no. 6, p. 1633, Jun. 2023, doi: 10.3390/agronomy13061633. [33] K. Srini v as, R. Gag ana Sri, K. Pra v allika, K. Nishitha, and S. R. Polamuri, “CO VID-19 prediction based on h ybrid Inception V3 with V GG16 using chest X-ray images, ” Multimedia T ools and Applications , v ol. 83, no. 12, pp. 36665–36682, J un. 2024, doi: 10.1007/s11042-023-15903-y . [34] C. Sze gedy , V . V anhouck e, S. Iof fe, J. Shlens, and Z. W ojna, “Rethinking the inception architecture for computer vision, ” i n Pr o- ceedings of the IEEE Computer Society Confer ence on Computer V ision and P attern Reco gnition, J un. 2016 , v ol. 2016-Decem, pp. 2818–2826, doi: 10.1109/CVPR.2016.308. BIOGRAPHIES OF A UTHORS P afan Doungpaisan is assistant professor at F aculty of Industrial T echnology and Man- agement, King Mongkut’ s Uni v ersity of T echnology North Bangk ok, Thailand. She holds a Ph.D. de gree in (Information T echnology , King Mongkut’ s Uni v ersity of T echnology North Bangk ok with specialization in machine learning, image analysis. She research areas are image/signal process- ing, image analysis, machine learning, and pattern recognition. She can be contacted at email: paf an.d@itm.kmutnb .ac.th. P eerapol Khunarsa recei v ed the B.Sc. de gree in computer science (1999) from the Uttaradit Rajabhat Uni v ersity , Thailand, the M.Sc. de gree in Information T echnology (2002) from the King Mongkut’ s Uni v ersity of T echnology North Bangk ok, Thailand and the Ph.D. de gree in soft computing from Chulalongk orn Uni v ersity , Bangk ok, Thailand. He research interests include pattern recognition, machine learning, and intelligent systems. He can be contacted at email: peer - apol@uru.ac.th. A compar ative study of pr e-tr ained models for ima g e featur e ... (P afan Doungpaisan) Evaluation Warning : The document was created with Spire.PDF for Python.