Indonesian J our nal of Electrical Engineering and Computer Science V ol. 36, No. 3, December 2024, pp. 1683 ∼ 1693 ISSN: 2502-4752, DOI: 10.11591/ijeecs.v36.i3.pp1683-1693 ❒ 1683 Stacking-based ensemble lear ning f or identifying artist signatur es on paintings Shintami Chusnul Hidayati, Agustinus Aldi Irawan Rahardja, Nanik Suciati Department of Informatics, Institut T eknologi Sepuluh Nopember , Surabaya, Indonesia Article Inf o Article history: Recei v ed Jul 4, 2024 Re vised Aug 7, 2024 Accepted Aug 25, 2024 K eyw ords: Artist signature identication Con v olutional neural netw orks Cultural heritage Ensemble learning Image classication ABSTRA CT Identifying artist signatures on paintings is essential for authenticating artw orks and adv ancing digital humanities. An artist’ s signature is a consistent element included in each painting that the artist creates, pro viding a unique identier for their w ork. T raditional methods that rely on e xpert analysis and manual compar - ison are time-consuming and are prone to human error . Although con v olutional neural netw orks (CNNs) ha v e sho wn promise in automating this process , e x- isting single-model approaches struggle with the di v ersity and comple xity of artistic styles, leading to limitations in their performance and generalizability . Therefore, this study proposes an ensemble learning approach that inte grates the predicti v e po wer of multiple CNN-based models. The proposed frame w ork le v erages the strengths of three state-of-the-art CNNs: Ef cientNetB4, ResNet- 50, and Xception. These models were independently trained, and the predictions were combined using a meta -learning strate gy . T o address class imbalance, data augmentation techniques and weighted loss functions were emplo yed. The e x- perimental results obtained on a da taset of more than 8,000 paintings from 50 artists demonstrate s ignicant impro v ements o v er indi vidual CNN architectures and other ensemble methods, thereby ef fecti v ely capt uring comple x features and impro ving generalizability . This is an open access article under the CC BY -SA license . Corresponding A uthor: Shintami Chusnul Hidayati Department of Informatics, Institut T eknologi Sepuluh Nopember ITS Suk olilo Campus, Surabaya 60111, Indonesia Email: shintami@its.ac.id 1. INTR ODUCTION Identifying artist signatures on paintings is crucial for art authentication, origin research, and digital humanities [1]. Artist signatures pro vide essential information about the or igin and authenticity of art- w orks, making their accura te identication indispensable for art historians, curators, and collectors. T raditional signature identication methods primarily rely on e xpert analysis and manual comparison, which are time-consuming and prone to human error [2], [3]. In recent years, adv ances in machine learning, par - ticularly con v olutional neural netw orks (CNNs), ha v e demonst rated signicant potential for automating this process [4]. CNNs e xcel at image recognition and classication tasks due to their ability to learn hierarchical feature representations [5]-[7]. Ho we v er , current state-of-the-a rt methods, which typically rely on single CNN architectures, often f ace challenges due to the di v ersity and comple xity of artistic styles, which limits their performance and generalizability [8]-[10]. Early studies ha v e e xplored using handcrafted features for artist identication with a limited num- ber of painter classess. F or e xample, the study by Shamir et al. [11] assessed the use of sher scores. J ournal homepage: http://ijeecs.iaescor e .com Evaluation Warning : The document was created with Spire.PDF for Python.
1684 ❒ ISSN: 2502-4752 Condoro vici et al. [12] inte grated lo w-le v el features (i.e., 3D RGB Histograms and Gabor Ener gy Features) with classi cal machine learning techniques. Similarly , Alyannezhadi [13] e xtracted 11 statistical features, in- cluding edge detection and color histograms, and used hierarchical support v ector machines (SVM) to enhance artist identication. Sheng and Jiang [14] applied handcrafted features to classify Chinese ink paintings. Ho w- e v er , handcrafted features often require domain e xpertise and may not capture the full range of v ariability in art istic styles [15]-[17]. Handcrafted features are less rob ust to lighting, resolution, and noise v ariations, which are common in real-w orld datasets. In addition, the y are typically less e xible and scalable than features learned by deep learning models; thus, the y are less applicable to di v erse and comple x datasets. Se v eral studies ha v e e xplored v arious approaches to impro v e the accurac y of signature ident ication. F or e xample, T an et al. [18] utilized end-to-end deep con v olutional models to classify artis ts based on their painting styles and demonstrated impro v ed classication accurac y compared to traditional methods that em- plo yed lo w-le v el descriptors. Saleh and Elg ammal [19] combined handcrafted and CNN-based features to emphasize the importance of learning the correct feature e xtraction metric. Another study by Hong and Kim [20] de v eloped a h ybrid model for artist identication that combined hand-crafted local features with learned features. V inaya v ekhin et al. [21], emplo yed Siamese neural netw orks and a self-supervised learning approach to identify relationships paintings of Japanese and W estern artists. Recently , some studies [4], [22], [23] further adv anced this task by ne-tuning CNNs specically for artist attrib ution. Their w ork highlights the ef fecti v e- ness of transfer learning in adapting pretrained models to the artistic domain. Despite these adv ancements, se v eral challenges still need to be addressed re g arding the use of con- v entional deep-learning methods for artist identication. Art datasets often suf fer from class imbalance, where some artists represent signicantly more w ork than others, leading to biased models. The wide range of styles and techniques used by dif ferent artists further complicates the task, as single-model approaches may not fully capture the intri cate details needed for accurate identication. In addition, the limited a v ailability of high- quality annotated data poses challenges, which mak es it dif cult to train deep learning models ef fecti v ely . Ov ertting remains a concern because models trained on s mall datasets may perform well on training data b ut must be generalized to unseen artw orks. Moti v ated by the limitations of e xisting methods, we propose a stacking ensemble approach to en- hance the performance of artist signature identication. Stacking ensembles combine the strengths of multiple models, le v eraging their di v erse predictions to impro v e o v erall accurac y and rob ustnes s [24], [25]. By inte grat- ing se v eral base learners with unique adv antages, this approach can better capture subtle and comple x features of artist si gn a tures, mitig ating indi vidual model weaknesses and enhancing their strengths. Specically , our method inte grates the predicti v e po wer of multiple CNN-based models: Ef cientNetB4 [26], ResNet-50 [27], and Xception [28]. These models are independently ne-tuned on a dataset of o v er 8,000 paintings from 50 dif- ferent painters and their predictions are combined through a meta-learning strate gy . T o address class imbalance in the dataset, we emplo y data augmentation techniques such as random rotations, ips, and color adjustments, along with weighted loss functions during training to assign greater importance to less represented classes, thereby reducing bias to w ard more pre v alent classes. In summary , this study mak es se v eral k e y contrib utions. First, we de v eloped an ef fecti v e preprocessing pipeline to standardize and augment input images. Second, we i nte grated di v erse CNN architectures to capture a wide range of artistic features. Third, we implemented a meta-learner to combine the predictions of the base models, thereby enhancing o v erall performance. These contrib utions address the limitations of e xisting methods and impro v e the automated identication of artist signatures. The remainder of this paper is or g anized as follo ws. Section 2 details the methodology , including the preprocessing pipeline, base learner architectures, and ensemble learning approach. Section 3 describes the e xperimental setup, e v aluation metrics, and e xperimental results. Finally , section 4 concludes the paper , summarizing the k e y contrib utions and outcomes of this study . 2. METHOD This section details the proposed methodology , which inte grates the predicti v e po wer of multiple CNN-based model s to impro v e o v erall predicti v e performance. Gi v en an input image, the proposed method automatically identies the corresponding painter artists. Figure 1 illustrates the frame w ork, which comprises three main steps: image preprocessing, base learning, and ensemble learning. Indonesian J Elec Eng & Comp Sci, V ol. 36, No. 3, December 2024: 1683–1693 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 1685 Figure 1. The proposed frame w ork 2.1. Image pr epr ocessing In this step, the input images are standardized and augmented to ensure consistenc y for training and inference. Initially , the images are resized to specic dimensions to meet the unique architectural requirements of each base model. F or ResNet-50, images are resized to 224 × 224 pix els. Xception, which requires lar ger input dimensions, utilizes images resized to 299 × 299 pix els. Ef cientNetB4 benets from e v en higher res- olution inputs, with images resized to 380 × 380 pix els. In the f o l lo wing resizing step, color normalization is performed to standardize the input images. Each pix el v a lue is scaled to f all within the range of 0 to 1 by di viding by the maximum pix el v alue, 255. Furthermore, pix el v alues are normalized to ha v e zero mean and unit v ariance based on the dataset’ s mean and standard de viation. This normalization step accelerates the con v er gence of the training process by ensuring that the input data ha v e a uniform distrib ution. Se v eral data augmentation techniques are applied to augment the dataset to pre v ent o v ertting [29], [30]. These techniques increase the di v ersity of the training d a taset. The random transformations used in this study include random rotations, horizontal and v ertic al ipping, and brightness and contrast adjustments. Ran- dom rotations within the range of -15° to +15° introduce v ariations in image ori entation, thereby impro ving model in v ariance under slight rotations. Horizontal and v ertical ipping simulate dif ferent perspecti v es, ensur - ing that the model recognizes artist signatures re g ardless of ori entation. Additionally , random brightness and contrast adjustments are applied to simulate v ariations in the lighting conditions, thereby making the model more rob ust to dif ferent lighting en vironments. 2.2. Base lear ning As mentioned pre viously , the ensemble learning approach harnesses the strengths of three stat e-of- the-art CNNs: Ef cientNetB4 [26], ResNet-50 [27], and Xception [28]. Each base learner of fers unique ad- v antages, enabling the model to capture and analyze subtle and comple x characteristics of an artist’ s paintings. Independently trai ned on a dataset of paintings, these base learners generate class predictions that collecti v ely enhance the ensemble’ s o v erall performance. The moti v ations for empl o yi ng these three base learners are as follo ws. Ef cientNetB4 is kno wn for its optimized balance between accurac y and computational ef cienc y . This model emplo ys a compound scaling method that uniformly scales the netw ork depth, width, and resolution. This scaling approach enabled the model to achie v e higher accurac y with fe wer computational resources than traditional scaling methods. The ability to handle high-resolution inputs and its ef cient use of parameters mak es Ef cientNetB4 a v aluable component in our ensemble. ResNet-50, the second base learner in our ensemble, is a deep residual netw ork reno wned for its ability to address the v anishing gradient problem, which is common in deep neural netw orks. By using residual connections, ResNet-50 f acilitates the training of v ery deep netw orks, allo wing it to learn intricate patterns and features in i mages. Xception, which stands for “e xtreme inception”, is another rob ust CNN architecture emplo yed as a base learner . It b uilds upon the inception model using depth-wise separable con v olutions. This architectural choice signicantly r educes the number of parameters while maintaining high performance, enabling the model to learn spatial hierarchies in the input data ef ciently . Xception’ s design allo ws i t to capture detailed and comple x features, making it well-suited for tasks in v olving ne-grained visual distinctions, such as identifying artist paintings. 2.3. Ensemble lear ning This step inte grates the classication results from the base learners-Ef cientNetB4, ResNet-50, and Xception. In this inte gration, we rst t ransform the classication results obtained from each base learner to Stac king-based ensemble learning for identifying artist signatur es ... (Shintami Chusnul Hidayati) Evaluation Warning : The document was created with Spire.PDF for Python.
1686 ❒ ISSN: 2502-4752 one-hot encoding. In other w ords, each predicted class is represented by a v ector whose length is equal to the number of classes in the dataset (50 classes of artists A in this study). The predicted class is mark ed as 1 while all other classes are mark ed as 0. Once the one-hot encodings of the base learners are obtained, we concatenate them into a single feature v ector . Let the feature v ectors obtained from Ef cientNetB4, ResNet-50, and Xception for image I be represented as x e i , x r i , and x c i , respecti v ely , where each v ector has 50 dimensions. Then, the concatenated feature v ector for image I , x i , is gi v en by (1): x i = [ x e i , x r i , x c i ] . (1) Therefore, the combined v ector encapsulates the predicti v e information from the three base models, pro viding a rich and detailed representation of the input data. The feature v ector is then input to a decision-making process that emplo ys ensemble methods. Suppose we ha v e the dataset dened in (2): D = { ( x i , y i ) } N i =1 , (2) where x i represents the feature v ector , y i is the tar get v ariable, and N is the number of paintings in the dataset. The meta-learner b uilds M decision trees, dened in (3): { T m } M m =1 , (3) using bootstrap samples from the original dataset. A bootstrap sample is created by randomly selecting N instances from the original dataset D with replacement. Thus, some instances may be selected multiple times, while others may not be selected at all. F ormally , let D ∗ denote a bootstrap sample, as written in (4). D ∗ = { ( x ∗ i , y ∗ i ) } N i =1 , where ( x ∗ i , y ∗ i ) ∈ D . (4) Bootstrap sampling creates di v erse training sets for each decision tree in the ensemble [31], [32]. This di v ersity helps reduce o v ertting by ensuring that each tree is trained on a slightly dif ferent subset of the dataset, which captures v arious aspects of the dataset’ s underlying structure. Moreo v er , bootstrap sampling allo ws the model to estimate the prediction v ariance and impro v es the rob ustness and reliability of the nal model. Each tree T m in (2.3.) is constructed using a random subset of features at each split to enhance the di v ersity of the trees. F or a gi v en input x , each tree T m pro vides a class prediction h m ( x ) . The nal prediction of the classier is determined by aggre g ating the predictions of all the trees. Specically , the output ˆ y is gi v en by the mode of the predicted classes, as formulated in (5): ˆ y = mode { h 1 ( x ) , h 2 ( x ) , . . . , h M ( x ) } . (5) this majority v oting mechanism ensures that the meta-learner classier reduces o v ertting and impro v es generalizability compared to indi vidual decision trees. 3. EXPERIMENTS This section presents the dataset, e xperimental setup, e v aluation metrics, and results obtained using the proposed ensemble learning approach. W e e v aluated the performance of the base learners and ensemble method on a painting dataset. The results highlight the ef fecti v eness of the proposed method. 3.1. Dataset The dataset used in this study w as collected from art challenge under the cate gory of f amous paint ers. It comprises a total of 8,446 paintings by 50 dif ferent paint ers [33]. Each image w as associated with a specic artist, pro viding a labeled dataset suitable for supervised learning to train and e v aluate the artist identication models. As sho wn in Figure 2, which sho ws the image distrib ution across 50 painters, the dataset e xhibits an imbalance in the number of paintings painted by each artist. This i mbalance challenges learning algorithms because models trained on imbalanced data may be biased to w ard more represented classes. Indonesian J Elec Eng & Comp Sci, V ol. 36, No. 3, December 2024: 1683–1693 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 1687 Figure 2. Number of paintings for each artist in the dataset (Better vie w in electronic v ersion) 3.2. Baseline models T o e v aluate the ef fecti v eness of the proposed ensemble learning approach, we compared its perfor - mance to that of se v eral baseline models. These baselines include indi vidual CNN architectures and other ensemble methods. The indi vidual CNN architectures considered are Ef cientNetB4 [26], ResNet-50 [27], and Xception [28]. The ensemble methods include majority v oting, weighted v oting [34], and XGBoost [35]. As mentioned pre viously , one of the main reasons behind the success of ensemble methods is i ncreas- ing the di v ersity among the base classiers [36]. Specically , in the majority v oting approach, the majority class determines the nal prediction that the base learners predict. Each base l earner equally contrib utes to the nal decision, and the cl ass with the most v ot es is selected. The weighted v oting method assigns dif ferent weights to predictions of each base learner based on indi vidual accurac y performance. The nal prediction is a weighted sum of the base learners’ predictions, with higher -performing models signicantly inuencing the outcome. XGBoost is a more adv anced ensemble method that uses gradient boosting to combine the predic- tions of multiple weak learners. XGBoost is kno wn for its high accurac y and ef cienc y , making it a strong baseline for comparison. 3.3. Experimental setup This subsection outlines the e xperimental setup. It includes the hardw are and softw are congurat ions as well as the training procedures for the base learners. Additionally , it species the h yperparameters used for each model. 3.3.1. Hard war e and softwar e congurations The e xperiments were conducted on Google Collaboratory Pro, equipped with T4 graphics pr o c essing units (GPUs). The softw are en vironment included Python, T ensorFlo w , and K eras. These frame w orks pro vide rob ust tools for implementing and training CNN architectures. 3.3.2. T raining pr ocedur es Each base learner (Ef cientNetB4, ResNet-50, and Xcepti o n) w as independently trained on the p a int- ings dataset. The dataset w as di vided into training, v alidation, and test sets with an 80-10-10 ratio. The training process in v olv ed ne-tuning the pretrained models on the ImageNet dataset to adapt t h e m to the painter iden- tication task. Each model w as trained for 50 epochs using a batch size of 32, with early stopping based on v alidation loss to pre v ent o v ertting. The Adam optimizer w as used with an initial learning rate of 0.0001, which w as reduced by a f actor of 10 if v alidation loss did not impro v e o v er v e consecuti v e epochs. F or train- ing the ensemble model, the meta-learner emplo yed an ensemble of decision trees, with 100 trees constructed using bootstrap samples from the original dataset. Each tree w as trained using a random subset of features at each split to enhance di v ersity . 3.4. Ev aluation metrics The model s’ performance w as assessed using se v eral metrics to pro vide a comprehensi v e understand- ing of their ef fecti v eness in identifying painter signatures. The metrics included accurac y , precision, recall, and F1-score. In addition, accurac y and loss were e v aluated during the training period. Accurac y measures the o v erall correctness of the model by calculating the proportion of corr ectly classied instances out of the t otal number of instances. Precision, also kno wn as positi v e predicti v e v alue, measures the accurac y of the positi v e predictions made by the model. High precision indicates a lo w f alse Stac king-based ensemble learning for identifying artist signatur es ... (Shintami Chusnul Hidayati) Evaluation Warning : The document was created with Spire.PDF for Python.
1688 ❒ ISSN: 2502-4752 positi v e rate. Recall, or sensiti vity , measures the model’ s ability to identify all rel e v ant instances. High recall indicates that the model correctly identied the most positi v e instances; thus, high recall is necessary when the cost of f alse ne g ati v es is high. The F1-score is the harmonic mean of precision and recall, pro viding a metric that balances both concerns. W e plotted the accurac y and loss graphs for both the training and v alidation sets o v er epochs to visu- alize the training process and monitor o v ertting and undertting. The accurac y graph sho ws ho w the model’ s accurac y e v olv es during training, while the loss graph depicts the change in the loss function. These graphs help visualize the model’ s learning beha vior and adjust the h yperparameters to impro v e performance. 3.5. Results and discussion T able 1 summarizes the performances of dif ferent models in identifying artist signatures on paintings. The results highlight the ef fecti v eness of the proposed ensemble learning approach compared to indi vidual CNN architectures and other ensemble methods. This section discusses the implications and insights deri v ed from these ndings. T able 1. Performance report (in %) Method Accurac y Precision Recall F1-score Ef cientNetB4 [26] 41.52 32.80 33.87 33.33 Resnet-50 [27] 54.67 40.71 39.41 40.05 Xception [28] 66.67 54.89 65.40 52.03 Majority v oting 67.82 60.37 52.46 56.13 W eighted v oting [33] 67.94 59.59 53.77 56.53 XGBoost [34] 87.20 93.24 89.46 91.31 Ours 88.12 93.47 90.06 91.73 Among the indi vidual base learners, Xception achie v ed the highest accurac y of 66.67%, with a pre- cision of 54.89%, recall of 65.40%, and an F1-score of 52.03%. ResNet-50 and Ef cientNetB4, while still performing reasonably well, lagged with accurac y v alues of 54.67% and 41.52%, respecti v ely . These results highlight the strengths and limitations of using single-model approaches for this task. Xception’ s higher perfor - mance can be attrib uted to its depthwise separable con v olutions, which ef fecti v ely capture detailed and comple x painting features. Ho we v er , the best-performing single model (Xception) w as belo w the desired performance le v els, which indicates the need for a more rob ust solution. Figure 3 presents the training and v alidation accurac y and loss for each base learner , which pro- vides insights into the learning beha vior and performance of each model o v er training epochs. Ov erall, Ef cientNetB4 demonstrated signs of o v ertting, with a lar ge g ap betwee n the training and v alidation metrics (Figures 3(a) and 3(d)), whereas ResNet-50 e xhibited uctuations in performance despite its ef cient learning (Figures 3(b) and 3(e)), which suggests its sensiti vity to training dynamics. In contrast, Xception demonstrated strong generalizability , maintaining closely aligned training and v alidation metr ics (Figures 3(c) and 3(f)). These ndings underscore the strengths and weaknesses of each base learner and underscore the importance of combining their predictions into an ensemble model. By le v eraging the di v erse learning beha viors and feature e xtraction capabilities of Ef cientNetB4, ResNet-50, and Xcepti on, the proposed ensemble approach aims to mitig ate indi vidual model weaknesses and enhance o v erall performance in identifying artist signatures. Moreo v er , from T able 1, it is e vident that majority v oting and weighted v oting outperformed indi vidual base learners with accurac y v alues of 67.82% and 67.94%, respecti v ely . These methods combine the strengths of mul tiple models, leading to more balanced and accurate predictions. Ho we v er , their performance metrics indicate that although the y enhance rob ustness, the y still need to reach the highest accurac y and reliability required for precise artist identication. XGBoost, a po werful gradient boosting algorithm, signicantly outperformed indi vidual models and basic ensemble methods, achie ving accurac y of 87.20%, precision of 93.24%, recall of 89.46%, and F1-score of 91.31%. Compared to Xception, XGBoost demonstrated an increase in accurac y of o v er 20 percentage points, rising from 66.67% to 87.20%. The precision notably impro v ed from 54.89% to 93.24%, while the recall increased from 65.40% to 89.46%, and the F1-score rose from 52.03% to 91.31%. Thes e results demonstrate the potential of adv anced ensemble techniques to im pro v e classication performance. XGBoost’ s ability to handle imbalanced data and capture comple x patterns via boosting contrib uted to its superior performance. Indonesian J Elec Eng & Comp Sci, V ol. 36, No. 3, December 2024: 1683–1693 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 1689 (a) (b) (c) (d) (e) (f) Figure 3. T raining and v alidation accurac y; (a) Ef cientNet accurac y , (b) ResNet-50 accurac y , and (c) Xception accurac y and loss; (d) Ef cientNet loss, (e) ResNet-50 loss, and (f) Xception loss for each base learner The proposed ensemble model, which inte grate s ResNet-50, Xception, and Ef cientNetB4 via a st ack- ing approach, achie v ed superior performance across all metrics. W ith an accurac y of 8 8.12%, precision of 93.47%, recall of 90.06%, and F1-score of 91.73%, our model not only outperformed indi vidual base learn- ers and simple ensemble methods b ut also impro v ed upon the results of XGBoost, can be seen in Figure 4. As sho wn in Figures 4(a) and 4(b), both majority v oting and weighted v oting ha v e limitations, particularly in identifying paintings belonging to minor artist classes, such as those by Cara v aggio, Monet, and Die go Ri v era. In contrast, XGBoost e xcelled in this task (Figure 4(c)), and the proposed method signicantly enhanced the recognition of these minority classes (F igure 4(d)). Although the impro v ement o v er XGBoost w as modest, it highlights the added v alue of our sophisticated ensemble method. These impro v ements underscore the sub- stantial g ains in performance achie v ed through the proposed ensemble approach, which ef fecti v el y le v erages the unique strengths of each base learner to capture di v erse features and patterns that indi vidual models may o v erlook. Figure 5 sho ws e xamples of paintings used in this study , along with their ground truth and pre dicted artist identications by v arious models. In Figure 5(a), a painting by Andrei Ruble v , most models incorrectly predicted the artist as V incent v an Gogh, Andy W arhol, or Giotto di Bondone, whereas the proposed model correctly ident ied Andrei Ruble v . Figures 5(b) and 5(c) feature paintings by Albrecht D ¨ urer . In Figure 5(b), predictions v aried, with models s uggesting artists lik e Rembrandt, Salv ador Dal ´ ı, and Albrecht D ¨ urer; ho we v er , the proposed model correctly identied Albrecht D ¨ urer . Figure 5(c) sho wed higher accurac y , where most models correctly predicted Albrecht D ¨ urer . This demonstrates the proposed model’ s ef fecti v eness in accurately identifying artists, highlighting its rob ustness in handling di v erse artistic styles. Despite the superior performance of the proposed ensemble model, se v eral challenges remain. Clas s imbalance in the dataset can lead to biased models f a v oring more represented classes. T echniques such as data augmentation and weighted loss functions were emplo yed to mitig ate this issue, further renem ent is required. Additionally , the comple xity of artistic styles requires models that learn a broad spectrum of features, which can be computationally intensi v e. Future w ork will address these challenges by e xploring more adv anced data augmentation techniques and incorporating additional base learners to capture more di v erse feat ures. W e also plan to in v estig ate other adv anced ensemble methods and meta-learning strate gies to enhance the model’ s generalizability and per - formance. By continuing to rene and e xpand our approach, we aim to de v elop more accurate and reliable automated artist signature identication models. Stac king-based ensemble learning for identifying artist signatur es ... (Shintami Chusnul Hidayati) Evaluation Warning : The document was created with Spire.PDF for Python.
1690 ❒ ISSN: 2502-4752 (a) (b) (c) (d) Figure 4. Performance results of the proposed approach compared to other ensemble methods; (a) ensemble by majority v oting, (b) ensemble by weighted v oting, (c) ensemble by XGBoost, and (d) pur proposed (Better vie w in electronic v ersion) Indonesian J Elec Eng & Comp Sci, V ol. 36, No. 3, December 2024: 1683–1693 Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ❒ 1691 (a) (b) (c) Figure 5. Examples of paintings in the study and their predicted artists; (a) a painting by Andrei Ruble v , (b) a painting by Albrecht D ¨ urer , and (c) a painting by Albrecht D ¨ urer 4. CONCLUSION This paper presented a no v el ensemble learning approach for identifying artist signat ures on paint ings, le v eraging the strengths of three state-of-the-art CNNs: Ef cientNetB4, ResNet-50, and Xception. The pro- posed methodology inte grated these di v erse models through a sophisticated meta-lear ning strate gy that com- bined predictions to enhance o v erall performance. Through comprehensi v e image preprocessing, including resizing, normalization, and data augmentation, we ensured that the input data w as consistent for training and inference. The use of a meta-learner with multiple decision trees further rened the predictions, reducing o v ertting and impro ving generalizability across the test data. Experimental results on a dataset of o v er 8,000 paintings from 50 artists demonstrated that the proposed model outperformed indi vidual CNN architectures and other ensemble methods, such as majority v oting, weighted v oting, and XGBoost. A CKNO WLEDGEMENTS The authors gratefully ackno wledge nancial support from the Institut T eknologi Sepuluh Nopember for this w ork, under project scheme of the Publication Writing and IPR Incenti v e Program (PPHKI) 2024. REFERENCES [1] J. W ilmerding, “Signs of the artist: signatures and sel f-e xpre ssion in American paintings, ” Choice Re vie ws Online , v ol. 41, no. 07, pp. 41-3861-41–3861, Mar . 2004, doi: 10.5860/CHOICE.41-3861. [2] S. L yu, D. Rockmore, and H. F arid, “ A digital technique for art aut hentication, ” Proceedings of the National Academy of Sciences of the United States of America , v ol. 101, no. 49, pp. 17006–17010, Dec. 2004, doi: 10.1073/pnas.0406398101. [3] G. Polatkan, S. Jaf arpour , A. Braso v eanu, S. Hughes, and I. Daubechies, “Detection of for gery in paintings using su- pervised learning, ” i n Proceedings - International Conference on Image Processing, ICIP , No v . 2009, pp. 2921–2924, doi: 10.1109/ICIP .2009.5413338. [4] C. Comert, M. Ozbayoglu, and C. Kasnak oglu, “P ainter prediction from artw orks wi th transfer learning, ” in 2021 7th International C onference on Mechatronics and Robotics Engineering, ICMRE 2021 , Feb . 2021, pp. 204–208, doi: 10.1109/ICMRE51691.2021.9384828. [5] Z. M. Liu, Y . Y . Chen, S. Hidayati, S. C. Chien, F . C. Chang, and K.-L. Hua, “3D model retrie v al based on deep Au- toencoder neural netw orks, ” in 2017 International Conference on Signals and Systems (ICSigSys) , May 2017, pp. 290–296, doi: 10.1109/ICSIGSYS.2017.7967059. [6] S. C. Hidayati and Y . Anistyasari, “Body shape calculator: understanding the type of body shapes from anthropometric measure- ments, ” in ICMR 2021 - Proceedings of the 2021 International Conf erence on Multimedia Retrie v al , Aug. 2021, pp. 461–465, doi: 10.1145/3460426.3463582. [7] Q. Qi, A. Ahmad, and W . K e, “Image classication based on fe w-shot learning algorithms: a re vie w , ” Indonesian Journal of Electrical Engineering and Computer Science , v ol. 35, no. 2, pp. 933–943, Aug. 2024, doi: 10.11591/ijeecs.v35.i2.pp933-943. [8] L. Alzubaidi et al., “Re vie w of deep learning: concepts, CNN architectures, challenges, applications , future directions, ” Journal of Big Data , v ol. 8, no. 1, p. 53, Mar . 2021, doi: 10.1186/s40537-021-00444-8. [9] J. K. Lee et al. , “Resource-ef cient con v olutional netw orks: a surv e y on model-, arithmetic-, and implementation-le v el techniques, ” A CM Computing Surv e ys , v ol. 55, no. 13 s, pp. 1–36, Dec. 2023, doi: 10.1145/3587095. [10] J. V alencia, G. G. Pineda, V . G. Pineda, A. V alencia-Arias, J. Arcila-Diaz, and R. T . de la Puente, “Using machine learning to predict artistic styles: an analysis of trends and the research agenda, ” Articial Intelligence Re vie w , v ol. 57, no. 5, p. 118, Apr . 2024, doi: 10.1007/s10462-024-10727-0. [11] L. Shamir , T . Macura, N. Orlo v , D. M. Eckle y , and I. G. Goldber g, “Impress ionism, e xpressionism, surrealism, ” A CM T ransactions on Applied Perception , v ol. 7, no. 2, pp. 1–17, Feb . 2010, doi: 10.1145/1670671.1670672. [12] R . Condoro vici, C. Florea, and C. V ertan, “ Author identication for digitized paintings collections, ” in ISSCS 2013 - International Symposium on Signals, Circuits and Systems , Jul. 2013, pp. 1–4, doi: 10.1109/ISSCS.2013.6651197. Stac king-based ensemble learning for identifying artist signatur es ... (Shintami Chusnul Hidayati) Evaluation Warning : The document was created with Spire.PDF for Python.
1692 ❒ ISSN: 2502-4752 [13] M . M. Alyannezhadi, H. Dabbaghan, S. Moghani, and M. F or ghani, “ A painting artist recognition system based on image processing and hierarchical SVM, ” in 2019 IEEE 5th Conference on Kno wledge Based Engineering and Inno v ation, KBEI 2019 , Feb . 2019, pp. 537–541, doi: 10.1109/KBEI.2019.8734911. [14] J . Sheng and J . Jiang, “Style-based classication of Chinese ink and w ash paintings, ” Optical Engineering , v ol. 52, no. 9, p. 093101, Sep. 2013, doi: 10.1117/1.oe.52.9.093101. [15] T . Geor giou, Y . Liu, W . Chen, and M. Le w , “ A surv e y of traditional and deep learning-based feature descriptors for high dimen- sional data in computer vision, ” International Journal of Multimedia Information Retrie v al , v ol. 9, no. 3, pp. 135–170, Sep. 2020, doi: 10.1007/s13735-019-00183-w . [16] J . Ma, X. Jiang, A. F an, J. Jiang, and J. Y an, “Image matching from handcrafted to deep features: a surv e y , ” International Journal of Computer V ision , v ol. 129, no. 1, pp. 23–79, Jan. 2021, doi: 10.1007/s11263-020-01359-2. [17] R . Y elchuri, J. K. Dash, P . Singh, A. Mahapatro, and S. P anigrahi, “Exploiting deep and hand-crafted features for te xture image re- trie v al using class membership, ” P attern Recognition Letters , v ol. 160, pp. 163–171, Aug. 2022, doi: 10.1016/j.patrec.2022.06.017. [18] W . R. T an, C. S. Chan, H. E. Aguirre, and K. T anaka, “Ceci n’est pas une pipe: a deep con v olutional netw ork for ne-art paintings classication, ” in Proceedings - International Conference on Image Processing, ICIP , Sep. 2016, v ol. 2016-August, pp. 3703–3707, doi: 10.1109/ICIP .2016.7533051. [19] B. Saleh and A. Elg ammal, “Lar ge-scale class ication of ne-art paintings: learning the right metric on t he right feature, ” arXi v preprint , 2015, doi: 10.11588/dah.2016.2.23376. [20] Y . Hong and J. Kim, “ Art painting detection and identication bas ed on deep learning and image local features, ” Multimedia T ools and Applications , v ol. 78, no. 6, pp. 6513–6528, Mar . 2019, doi: 10.1007/s11042-018-6387-5. [21] P . V inaya v ekhin, B. Khomkham, V . Suppakitpaisarn, P . Codognet, T . T erada, and A. Miura, “Identifying relationships and clas- sifying western-style paintings: machine learning approaches for artw orks by western artists and Meiji-era Japanese artists, ” Journal on Computing and Cultural Heritage , v ol. 17, no. 1, pp. 1–18, Feb . 2024, doi: 10.1145/3631136. [22] E. Cetinic, T . Li pic, and S. Gr gic, “Fine-tuning con v olutional neural netw orks for ne art classicat ion, ” Expert Systems with Applications , v ol. 114, pp. 107–118, Dec. 2018, doi: 10.1016/j.esw a.2018.07.026. [23] H. Ug ail, D. G. Stork, H. Edw ards, S. C. Se w ard, and C. Brook e, “Deep transfer learning for visual analysis and attrib ution of paintings by Raphael, ” Heritage Science, v ol. 11, no. 1, p. 268, Dec. 2023, doi: 10.1186/s40494-023-01094-0. [24] N. R. Octa vianto and A. W ibo w o, “Stacking classier method for predicti on of human body performance, ” Indone- sian Journal of Electrical Engineering and Computer Science (IJEECS) , v ol. 34, no. 3, pp. 1832–1839, Jun. 2024, doi: 10.11591/ijeecs.v34.i3.pp1832-1839. [25] L. Bagdadi and B. Messabih, “Distrib uted denial of service at tacks classication system using features selection and ensemble techniques, ” Indonesian J ournal of Electri cal Engineering and Computer Sci ence (IJEECS) , v ol. 34, no. 3, pp. 1868–1878, Jun. 2024, doi: 10.11591/ijeecs.v34.i3.pp1868-1878. [26] M. T an and Q. V . Le, “Ef cientNet: Rethinking model scaling for con v olutional neural netw orks, ” 36th International Conference on Machine Learning, ICML 2019 , v ol. 2019-June, pp. 10691–10700, 2019. [27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition, ” in 2016 IEEE Conference on Computer V ision and P attern Recognition (CVPR) , Jun. 2016, v ol. 2016-Decem, pp. 770–778, doi: 10.1109/CVPR.2016.90. [28] F . Chollet, “Xception: deep learning with depthwise separable con v olutions, ” in Proceedings - 30th IEEE Conference on Computer V ision and P attern Recognition, CVPR 2017 , Jul. 2017, v ol. 2017-January , pp. 1800–1807, doi: 10.1109/CVPR.2017.195. [29] C. Shorten and T . M. Khoshgoftaar , “ A surv e y on image data augmentation for deep learning, ” Journal of Big Data , v ol. 6, no. 1, p. 60, Dec. 2019, doi: 10.1186/s40537-019-0197-0. [30] M. Xu, S. Y oon, A. Fuentes, and D. S. P ark, “ A comprehensi v e surv e y of image augmentation techniques for deep learning, ” P attern Recognition , v ol. 137, p. 109347, May 2023, doi: 10.1016/j.patcog.2023.109347. [31] R. Duangsoithong and T . W indeatt, “Bootstrap feature selection for ensemble classiers, ” in Lecture Notes in Computer Science (including subseries Lecture Notes in Articial Intelligence and Lecture Notes in Bioinfor matics) , v ol. 6171 LN AI, P . Perner , Ed. Berlin, Heidelber g: Springer Berlin Heidelber g, 2010, pp. 28–41. [32] A. Mohammed and R. K ora, “ A comprehensi v e re vie w on ensemble deep learning: opportunities and challenges, ” Journal of King Saud Uni v ersity - Computer and Information Sciences , v ol. 35, no. 2, pp. 757–774, Feb . 2023, doi: 10.1016/j.jksuci.2023.01.014. [33] “ Art challenge. ” https://artchallenge.ru/ (accessed: 2023-06-01). [34] A. Dog an and D. Birant, “ A weighted majority v oting ensemble approach for classication, ” in 2019 4th International Conference on Computer Science and Engineering (UBMK) , Sep. 2019, pp. 1–6, doi: 10.1109/UBMK.2019.8907028. [35] T . Chen and C. Guestrin, “XGBoost: a scalable tree boosting system, ” in Proceedings of the A CM SIGKDD International Confer - ence on Kno wledge Disco v ery and Data Mining , Aug. 2016, v ol. 13-17-August-2016, pp. 785–794, doi: 10.1145/2939672.2939785. [36] M. A. Ganaie, M. Hu, A. K. Malik, M. T an v eer , and P . N. Sug anthan, “Ensemble deep learning: a re vie w , ” Engineering Applications of Articial Intelligence , v ol. 115, p. 105151, Oct. 2022, doi: 10.1016/j.eng appai.2022.105151. Indonesian J Elec Eng & Comp Sci, V ol. 36, No. 3, December 2024: 1683–1693 Evaluation Warning : The document was created with Spire.PDF for Python.