IAES
Inter
national
J
our
nal
of
Articial
Intelligence
(IJ-AI)
V
ol.
13,
No.
4,
December
2024,
pp.
4722
∼
4735
ISSN:
2252-8938,
DOI:
10.11591/ijai.v13.i4.pp4722-4735
❒
4722
Image
classication
in
cultural
heritage
Muath
Sabha
1
,
Muhammed
Saffarini
1,2
,
Rami
Y
ousef
2
1
Department
of
Multimedia
T
echnology
,
F
aculty
of
Information
T
echnology
,
Arab
American
Uni
v
ersity
,
Jenin,
P
alestine
2
Department
of
Computer
Systems
Engineering,
F
aculty
of
Engineering
and
T
echnology
,
P
alestine
T
echnical
Uni
v
ersity—Kadoorie,
T
ulkarm,
P
alestine
Article
Inf
o
Article
history:
Recei
v
ed
Jan
28,
2024
Re
vised
May
29,
2024
Accepted
Jun
3,
2024
K
eyw
ords:
Con
v
olutional
neural
netw
ork
Cultural
heritage
Image
classication
Image
se
gmentation
K-nearest
neighbors
ABSTRA
CT
In
this
paper
,
an
automated
supervised
image
classication
technique,
speci-
cally
for
classifying
images
in
the
cultural
heritage
domain,
is
de
v
eloped.
The
de
v
eloped
technique
classies
images
according
to
a
particular
date,
culture,
people
and
historical
age.
The
proposed
technique
consists
of
tw
o
stages,
fea-
ture
e
xtraction
us
ing
the
unsupervised
se
gmentation
technique,
and
the
classi-
cation
stage
using
supervised
classication
techniques.
Common
features
are
e
xtracted,
and
their
histograms
are
applied
t
o
three
classiers:
k-nearest
neigh-
bor
(KNN),
logi
stic
re
gression
(LR),
and
decision
tree
(DT).
When
our
tech-
nique
w
as
applied
to
a
repository
of
images
from
cultural
heritage,
it
sho
wed
reduced
comple
xity
and
impro
v
ed
classication
accurac
y
.
DT
has
achie
v
ed
a
higher
weighted
a
v
erage
recall.
This
is
also
represented
by
the
weighted
a
v-
erage
f-measure
where
DT
has
obtained
0.81.
DT
has
outperformed
the
other
classiers
in
terms
of
classifying
heritage
images.
This
is
an
open
access
article
under
the
CC
BY
-SA
license
.
Corresponding
A
uthor:
Muath
Sabha
Department
of
Multimedia
T
echnology
,
F
aculty
of
Information
T
echnology
,
Arab
American
Uni
v
ersity
Jenin,
P
alestine
Email:
muath.sabha@aaup.edu
1.
INTR
ODUCTION
There
are
tw
o
main
cate
gories
of
image
classication
algorithms:
unsupervised
and
supervised.
In
unsupervised
classication,
the
system
analyzes
the
images
without
the
need
of
pro
viding
an
y
sample.
While
the
supervised
learning
is
based
on
selecting
samples
and
putting
labels
for
images
by
the
user
and
applying
the
training
using
training
set
(group
of
images
that
ha
v
e
labels).
After
that,
the
system
predicts
the
class
that
an
image
belongs
to.
F
or
humans,
the
classication
method
consists
of
man
y
stages,
starting
with
image
se
gmentation,
e
xtracting
features
from
the
image,
and
then
recognition
and
classication
of
the
image,
that
is,
assigning
t
he
testing
i
mage
to
the
right
cate
gory
.
In
machine
learning,
it
depicts
the
human
w
ay
of
classication.
In
general,
the
supervised
image
classication
consists
of
the
follo
wing
stages:
–
Building
the
training
set:
the
input
will
be
N
images;
each
image
is
labeled
with
one
of
the
K
distinguished
classes.
–
Learning:
using
the
dataset
to
identify
the
characteristics
of
each
class.
This
step
is
called
training
the
classier
or
learning.
–
Ev
aluation:
estimating
the
quality
of
the
classier
by
pro
viding
prediction
labels
from
the
data
set
for
a
ne
w
set
of
images
that
it
has
ne
v
er
seen
before.
The
true
labels
of
these
images
are
compared
to
the
ones
predicted
by
the
classier
.
Man
y
of
the
image
classications
are
used
for
kno
wn
and
semi-visible
objects
and
shapes.
This
research
is
to
classify
images
related
to
a
particular
date,
culture,
peoples,
and
historical
ages
by
e
xtracting
the
J
ournal
homepage:
http://ijai.iaescor
e
.com
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Artif
Intell
ISSN:
2252-8938
❒
4723
common
features
(clothes,
tools,
and
b
uildings)
of
the
image
and
then
classify
the
fea
tures
for
which
culture
it
belongs
to.
The
problem
is
that
each
culture
is
dened
by
man
y
symbols
and
each
age
is
identied
by
dif
ferent
colors
and
shapes.
Since
there
are
man
y
cultures
that
ha
v
e
things
in
common,
and
also
often
cultural
images
ha
v
e
dif
ferent
object
sizes,
the
process
of
object
identication
needs
more
time.
In
the
follo
wing
sections,
we
re
vie
w
some
of
the
related
techniques
from
the
literature
in
section
2.
W
e
then
describe
our
methodology
in
an
e
xtensi
v
e
w
ay
at
section
3.
W
e
also
e
xplain
the
algorithms
tha
t
we
use
in
our
technique
and
the
reasons
behind
using
these
algorithms.
The
results
are
sho
wn
and
discussed
in
section
4.
Finally
the
conclusions
and
future
w
ork
are
discussed
in
section
5.
2.
LITERA
TURE
REVIEW
Image
clas
sication
could
be
cate
gorized
in
tw
o
major
research
areas,
computer
vision
and
deep
lea
rn-
ing.
In
computer
vision,
image
processing
is
used
to
se
gment
the
image
and
then
to
automatically
describe
by
using
other
AI
techniques.
Deep
learning
is
using
multilayer
decision
making
algorithms
in
art
icial
intelli-
gence
to
understand
the
image.
It
is
part
of
machine
learning.
This
section
tackles
these
tw
o
elds.
2.1.
Segmentation
In
image
se
gmentation,
P
aschos
[1]
compare
the
performance
of
three
color
spaces
(Lab,
HSV
,
and
RGB).
Because
RGB
data
are
clearly
ready
as
the
e
xplicitly
data
pro
vided
by
the
camera,
the
RGB
color
space
is
practically
uni
v
ersally
recognized
by
the
image
processing
research
society
for
displaying
images.
Ho
we
v
er
,
there
are
more
re
gular
cogniti
v
e
spaces,
such
as
Lab
and
Luv
,
where
the
dif
ference
in
hue
is
quantied
in
analogy
to
ho
w
a
human
percei
v
es
such
a
dif
ference.
In
the
research
done
by
Delignon
et
al.
[2],
a
comparison
of
dif
ferent
color
spaces
is
made.
The
Lab
color
space
has
the
potential
to
sho
w
results
in
a
more
natural
style
to
human
vision.
W
ang
and
Suter
[3]
proposed
a
ne
w
color
image
se
gmentation
method
that
incorporates
both
global
and
local
homogeneity
.
In
the
hue
and
density
subspace
of
HSV
,
the
method
applied
the
a
v
erage
shift
algorithm.
The
suggested
technique
additionally
tak
es
into
account
the
color
component’
s
c
yclic
characteristic.
Experiments
with
natural-color
photographs
ha
v
e
yielded
interesting
results.
Ohashi
et
al.
[4]
uses
the
hill-climbing
algorithm
for
image
se
g-
mentation.
It
does
se
gmentation
by
rst
determining
the
histogram
for
the
image’
s
three
layers
,
then
calculating
the
local
maxim
um
v
alues
in
the
hi
stogram
for
the
three-layer
H,
S,
and
V
in
the
image,
then
the
algorithm
connects
image
pix
els
with
their
respecti
v
e
local
maximum
v
alues.
Ho
we
v
er
,
there
are
se
v
eral
disadv
antages
to
t
his
method,
including
a
at
part
of
the
search
space
in
which
all
surrounding
states
ha
v
e
the
same
v
alue
and
the
necessity
for
a
lar
ge
jump
to
establish
a
ne
w
section.
Most
thresholding
approaches
in
v
olv
e
establishing
boundaries
based
on
gray
v
alues
or
pix
el
density
of
the
image
[5].
The
color
thresholding
procedure
is
carried
out
based
on
acclimation
and
minor
changes
to
the
gre
y-scale
thresholding
approaches.
The
RGB
data
from
the
surrounding
area
and
another
item
ha
v
e
been
coupled
to
multile
v
el
thresholding.
The
object
w
as
able
to
be
separated
from
the
background
using
image
fragmentation
technology
.
Man
y
researches
mak
e
use
of
machine
learning
techniques
to
se
gments
the
images
lik
e
[6],
where
the
K-Means
algorithm
is
applied
to
2,
3
and
k-cluster
colored
images
for
(k¿3).
Silhouette
analysis
helps
to
assign
the
peaks
for
set
k-cluster
.
Ho
we
v
er
,
there
are
some
dra
wbacks,
such
as
the
dif
culty
in
selecting
the
optimal
number
of
clusters
and
the
f
act
that
the
selection
of
the
initial
centroids
is
random.
More
techniques
depend
on
the
te
xture
lik
e
[7]–[9].
According
to
V
adi
v
el
et
al.
[10],
images
are
analyzed
and
se
gmented
using
the
HSV
color
space.
In
this
technique,
features
are
e
xtracted
by
choosing
either
the
hue
or
the
density
as
the
dominating
characteristic,
which
is
determined
by
the
pix
el’
s
saturation
v
alue.
Image
se
gmentation
is
done
using
statistical
re
gion
mer
ging
(SRM)
[11].
The
algorithm
looks
at
the
statistical
underpinnings
of
a
process
that
is
frequently
described
in
computer
vision.
The
algorithm
is
used
to
estimate
the
v
alues
in
the
re
gional
e
xtension
and
classify
them
together
based
on
a
smaller
list
standard.
JSEG
used
color
quantication
to
re
p
r
esent
numerous
classes
that
can
be
used
to
dif
ferentiate
sections
in
the
image.
Color
class
labels
are
used
to
e
xchange
pix
el
colors,
forming
a
class-map
of
the
image
[12].
Singh
and
Khare
[13]
creates
an
autonomous
threshold-based
information
parameter
for
image
se
gmentation
using
a
genetic
algorithm
to
nd
the
best
number
of
se
gmentation
areas.
Only
gray-scale
photos
are
used
in
this
technique.
According
to
Sun
et
al.
[14],
multilayer
thresholding
is
used
to
re
visit
gray-scale
Ima
g
e
classication
in
cultur
al
herita
g
e
(Muath
Sabha)
Evaluation Warning : The document was created with Spire.PDF for Python.
4724
❒
ISSN:
2252-8938
image
se
gmentation.
The
authors
used
nature-inspired
a
lgorithms
to
try
to
solv
e
the
challenge
of
multilayer
thresholding
(N
As).
T
o
produce
f
acial
picture
histograms,
a
256-le
v
el
histogram
must
rst
be
created,
and
then
e
v
ery
eight
contiguous
le
v
els
are
represented
as
one
v
alue
to
simplify
computations
and
comparisons,
as
well
as
to
speed
up
image
processing
without
sacricing
image
quality
.
Instead
of
256
pins,
there
will
be
just
32
pins
after
this
process
[15].
This
algorithm
is
commonly
used
in
f
acial
recognition,
b
ut
we
will
adapt
it
and
apply
it
to
our
method.
Man
y
techniques
are
pro
vided
in
[16],
[17]
based
on
the
scale-in
v
ariant
features
transform
(SIFT).
Another
algorithm
di
vides
the
operation
into
tw
o
steps:
recognition
and
se
gment
ation,
using
a
top-do
wn
technique
for
recognition
and
a
bottom-up
approach
for
se
gmentation.
The
rst
is
the
h
ypothesis
and
the
second
is
the
v
erication
of
the
h
ypothesis,
which
results
in
the
creation
of
a
set
of
h
ypotheses
such
as
object
location
and
ground
mask.
It
computes
the
list
of
probable
se
gments
created
by
the
h
ypothesis
step
in
the
v
erication
step
[18].
2.2.
Cultural
heritage
image
classication
using
deep
lear
ning
Image
classication
in
cultural
heritage
is
an
important
task
in
the
process
of
digitalization.
It
can
be
particularly
challenging
due
to
the
lar
ge
number
of
dif
ferent
image
cate
gories,
the
v
aria
b
i
lity
of
the
character
-
istics,
and
t
h
e
need
for
high
reliability
.
Deep
learning
techniques
ha
v
e
been
emplo
yed
for
architectural
heritage
images
classication
using
pre-trained
con
v
olutional
neural
netw
orks
(CNN).
Deep
learning
algorithms
ha
v
e
demonstrated
remarkable
performance
in
image
classication
tasks.
Deep
learning
is
a
subdiscipline
of
the
area
of
articial
intelligence
that
e
v
olv
ed
from
classical
ma-
chine
learning.
T
o
create
a
learning
model,
deep
learning
approaches
mimic
the
natural
processing
capability
of
the
human
brain
[19].
T
o
comprehend
deep
learning,
it
is
required
to
rst
dene
neural
netw
ork,
which
is
one
of
the
machine
learning
strate
gies
gi
v
en
for
creating
a
self-learning
learning
model.
Support
v
ector
machine
(SVM),
na
¨
ıv
e
Bayes
(NB),
decision
tree
(DT),
k-nearest
neighbor
(KNN),
and
other
machine
learning
tech-
niques
are
e
xamples.
In
terms
of
cl
assication,
neural
netw
ork
has
performed
admirably
.
Dif
ferent
adaptations
of
articial
neural
netw
orks
(ANN),
such
as
perceptron,
multilayer
perceptron,
and
backpropag
ation,
ha
v
e
been
presented;
ho
we
v
er
,
there
is
a
serious
dra
wback
still
obstructs
the
precise
learning
process.
Classic
ANN
has
a
high
le
v
el
of
ability
to
learn
and
classify
a
certain
dataset
e
xact
itude
(i.e.
99%).
When
the
ANN
is
tested
on
a
ne
w
dataset,
ho
we
v
er
,
it
does
not
reach
the
same
precision
[20].
Ov
ertting
or
v
anishing
ingredients
are
terms
used
to
describe
this
problem.
When
an
ANN
learns
the
characteristics
and
features
of
certain
data
in
such
a
w
ay
that
an
y
change
in
the
data
af
fects
the
ANN’
s
capacity
to
nd
signicant
patterns,
a
problem
arises.
Deep
learning
algorithms
are
one
of
the
neural
netw
ork
architectures
that
mak
e
use
of
se
v
eral
hidden
layers
[21].
Figure
1
sho
ws
the
basic
architecture
for
deep
learning
netw
orks.
Figure
1.
Basic
architecture
of
a
deep
neural
netw
ork
Recurrent
neural
netw
ork,
long
short-term
memory
(LSTM),
deep
belief
netw
ork
(DBN),
and
CNN
are
e
xamples
of
deep
learning
architectures
[22].
The
sub-sections
that
follo
w
will
go
through
these
archi-
Int
J
Artif
Intell,
V
ol.
13,
No.
4,
December
2024:
4722–4735
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Artif
Intell
ISSN:
2252-8938
❒
4725
tectures
in
greater
depth.
The
CNN
for
image
clas
sication
w
as
used
by
[23]
to
classify
architectural
history
images
using
a
deep
learning
neural
netw
ork.
The
results
demonstrate
a
le
v
el
of
precision
of
up
to
90%.
De-
spite
the
f
act
that
man
y
researchers
are
tackling
the
problem
of
properly
classifying
and
annotating
the
digital
heritage,
Belhi
et
al.
[24]
use
a
specic
type
of
deep
learning
classier
,
each
of
which
is
assigned
the
task
of
classifying
a
combination
of
asset
classes.
According
to
Kam
bau
et
al.
[25]
to
classify
images,
audio,
video,
and
te
xt,
the
authors
use
the
CNN
and
recurrent
neural
netw
orks
approaches.
The
y
ha
v
e
applied
this
by
propos-
ing
a
cro
wdsourcing
platform
to
acquire
data
from
the
Indian
Digital
Heritage
Space
(IDHS)
monuments,
and
designing
a
transfer
learning-based
image
classier
.
K
ulkarni
et
al.
[26]
perform
image
classication
and
query-based
image
label
retrie
v
al.
Using
t
h
e
pre-trained
MobileNet
V2
architecture
and
the
ImageNet
dataset,
the
proposed
transfer
learning
technique
achie
v
es
an
inference
accurac
y
of
98.75%.
Jank
o
vi
´
c
[27]
consider
image
classication
in
terms
of
cultural
heritage
as
one
of
the
most
essential
jobs
in
the
digital
era.
It
is
crucial
to
b
uild
classication
methods
that
are
accurate;
thus,
their
research
com-
pares
four
classication
algorithms
with
a
CNN.
The
y
use:
i)
the
multilayer
perception,
ii)
a
v
eraged
depen-
dence
estimators,
ii
i)
forest
by
penalizing
attrib
utes,
and
i
v)
the
KNN
rough
sets
and
analogy-based
reasoning.
Belhi
et
al.
[28]
tackle
the
challenge
of
automatically
classifying
and
annotating
cultural
heritage
assets
using
their
visual
features
as
well
as
the
metadata
a
v
ailable
at
hand
by
using
multitask
neural
netw
ork,
where
a
CNN
is
designed
for
visual
feature
learning,
and
a
re
gular
neural
netw
ork
is
used
for
te
xtual
feature
learning.
According
to
A
w
ad
et
al.
[29],
an
older
classication
frame
w
ork
w
as
presented.
It
depends
on
ho
w
the
image
is
annotated
and
ho
w
human
feedback
is
recei
v
ed.
The
classier
will
use
these
truth-ground
annotations
as
a
training
set
once
the
human
gi
v
es
a
li
st
of
image
annotations.
The
classier
is
presented
with
a
ne
w
test
image
and
is
ask
ed
to
predict
the
annotations
on
it.
F
or
each
ne
w
image
added
to
the
training
set,
a
human
pro
vides
additional
input.
The
block
diagram
of
image
classication
is
sho
wn
in
Figure
2.
Figure
2.
Block
diagram
for
image
classication
3.
METHODOLOGY
In
this
research,
RGB
and
Lab
are
tested
in
the
e
xpression
of
their
performance
in
the
ideal
i
mage
processing
function,
such
as
the
se
gmentation
of
images.
There
has
been
an
only
restricted
act
on
the
color
sides
of
te
xtured
images.
Our
methodology
is
summarized
in
Figure
3.
W
e
rst
con
v
ert
the
image
color
space
from
RGB
to
CIELab
.
In
Lab
color
space,
there
are
tw
o
color
channels;
the
rst
channel
is
blue
to
w
ard
yello
w
(opponents)
and
the
second
is
green
to
w
ards
red
channel,
which
is
almost
closer
to
ho
w
humans
percei
v
e
images.
In
addition
to
that,
we
ha
v
e
the
lightness
L.
Using
ab
from
Lab
,
we
can
e
xtract
the
lightness
from
the
image
to
reduce
the
ef
fect
of
shado
ws,
darkness,
and
lightness
inside
the
images.
W
e
intend
to
use
pure
color
and
to
use
the
mix
ed
colors
inside
each
layer
of
the
image
to
mimic
the
human
visual
system,
which
may
produce
more
accurate
se
gmentation.
Moreo
v
er
,
the
Lab
g
amut
is
lar
ger
than
RGB
g
amut,
which
separates
colors
by
dif
ferent
absolute
relati
v
e
distances
than
it
in
the
RGB
space
[30].
Ima
g
e
classication
in
cultur
al
herita
g
e
(Muath
Sabha)
Evaluation Warning : The document was created with Spire.PDF for Python.
4726
❒
ISSN:
2252-8938
There
are
man
y
algorithms
for
image
se
gmentation
and
there
are
also
man
y
algorithms
for
image
clas-
sication.
In
this
paper
,
some
of
these
se
gmentation
and
classication
algorithms
are
discussed
and
the
results
of
the
proposed
algori
thm
are
compared
with
the
results
of
the
most
well-kno
wn
algorithms.
W
e
will
st
art
with
se
gmentation
and
we
will
discuss
three
main
algorithms
which
are
gray
thresholding,
color
thresholding,
and
max
v
alue
thresholding.
Figure
3.
Our
methodology
steps
3.1.
Segmentation
In
this
step,
we
se
gment
the
images
and
tak
e
the
common
features
inside
them,
and
then
calculate
the
histogram
for
each
Lab
channel
of
the
image.
The
Lab
color
space
has
three
layers
L
,
a
,
and
b
.
The
range
v
alues
for
L
are
[0
to
100],
the
range
v
alues
for
a
are
[-86.185
to
98.254]
and
the
range
v
alues
for
b
are
[-107.863
to
94.482].
Each
v
alue
in
one
layer
may
ha
v
e
a
set
of
v
alues
in
other
layers.
F
or
e
xample,
we
may
ha
v
e
30
pix
els
that
ha
v
e
the
v
alue
10
on
the
b
channel
b
ut
dif
ferent
v
alues
in
the
a
channel.
W
e
assumed
that
we
ha
v
e
201
dif
ferent
sampled
v
alues
in
the
a
channel,
which
are
the
inte
ger
v
alues
in
the
range
of
the
a
channel
mentioned
earlier
.
This
is
to
reduce
the
number
of
histogram
columns.
After
this
step,
we
mer
ge
e
v
ery
3
adjacent
columns
for
e
v
ery
layer
,
which
reduces
the
number
of
columns
in
each
channel.
That
will
reduce
the
time
needed
to
recognize
the
features
as
sho
wn
in
the
ne
xt
stage
from
N
×
M
to
a
constant
v
alue.
W
e
repeat
the
mer
ging
step
until
we
reach
the
total
number
of
columns
of
29
columns
for
each
layer
.
These
29
v
alues
in
calculation
do
not
tak
e
too
much
processing
time.
The
mer
ging
process
is
performed
by
taking
the
a
v
erage
of
adjacent
columns.
After
that,
we
create
a
v
ector
of
29
v
alues,
which
results
in
a
ne
w
histogram
of
29
columns.
This
will
reduce
the
time
comple
xity
in
the
ne
xt
step
as
these
v
ectors
will
be
the
ne
w
representation
for
each
layer
instead
of
the
pre
vious
histogram
for
the
a
and
b
layers.
After
that,
we
enter
these
ne
w
v
ectors
(the
ne
w
histogram)
into
equations
that
calculate
the
local
minimum
and
maximum
v
alues
in
these
histograms
by
nding
the
centers
of
the
pre
vious
histograms
to
con
v
ert
the
columns
to
groups
of
points.
That
means,
rst
we
con
v
ert
the
histogram
fr
om
the
columns
to
the
linear
chart
that
contains
continuous
v
alues,
then
we
apply
the
loca
l
min-max
v
alue
so
that
we
can
calculate
the
local
min-max
v
alues.
The
aim
of
calculating
the
local
min-max
v
alues
is
to
see
when
the
color
inside
Lab
layers
changes
from
one
color
to
another
.
After
calculating
the
min-max
v
alues,
we
store
these
v
alues
in
v
ectors,
thus
producing
these
v
ectors
of
points,
and
we
call
them
clusters
(
C
)
.
The
y
consist
of
local
min-max
v
alues
for
three
layers
in
the
Lab
color
histogram
of
the
image.
W
e
assume
that
e
v
ery
cluster
will
ha
v
e
the
v
alues
between
e
v
ery
contiguous
local
min
and
max
v
al
ues
in
the
histogram.
At
this
stage,
we
found
that
the
time
for
computation
will
be
short
and
constant,
with
time
comple
xity
O
(
C
)
.
W
e
apply
the
equation
of
mer
ging
columns
that
gi
v
e
20
columns,
(not
for
all
the
columns
in
the
rst
histogram,)
then
the
number
of
the
local
min-max
v
alues
equals
the
number
of
clusters
that
we
will
use
to
se
gment
the
images,
into
se
gments
to
use
these
se
gments
for
recognition
and
classication
as
we
will
see
in
the
ne
xt
steps.
After
nding
min-max
v
ectors,
we
use
these
v
ectors
as
the
clusters
by
applying
these
v
ectors
into
k-means
clustering
(an
unsupervised
technique
for
clustering
images
without
the
user
pro
viding
sample
classes
or
labels,
and
uses
an
equation
to
get
results
such
that
Euclidean
distance).
After
that,
we
apply
the
k-means
to
the
whole
original
image,
which
is
a
tw
o-dimensional
array
.
W
e
then
tak
e
the
number
of
local
min-max
v
alues
that
will
be
the
number
of
groups
and
the
number
of
se
gments
in
the
image
which
we
w
ant
to
apply
this
Int
J
Artif
Intell,
V
ol.
13,
No.
4,
December
2024:
4722–4735
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Artif
Intell
ISSN:
2252-8938
❒
4727
algorithm
on,
and
the
v
alues
for
the
v
ectors
that
are
the
outcomes
of
local
min-max
v
alues
as
initial
v
alues
for
centroid
in
each
cluster
.
W
e
nally
produce
indices
for
clusters,
which
will
enhance
the
nal
outcomes
of
the
k-means
that
will
represent
the
se
gments
in
the
image
we
are
w
orking
on.
The
L
2
norm
is
used
in
the
k-means
algorithm
to
group
the
Lab
colors,
considering
the
Lab
as
a
Euclidean
space.
d
(
p
1
,
p
2
)
=
p
(
x
1
−
x
2
)
2
+
(
y
1
−
y
2
)
2
W
e
nd
the
distance
from
e
v
ery
color
to
e
v
ery
centroid
to
get
the
minimum
distance,
and
so
to
wh
i
ch
cluster
this
color
is
related.
If
the
color
distance
is
shorter
to
another
centroid,
this
means
that
this
color
does
not
belong
to
this
centroid.
This
color
is
transferred
to
the
cluster
that
has
the
minimum
distance
from
its
centroid.
After
that,
we
check
if
there
is
an
y
ne
w
pix
el
to
be
transferred
to
another
cluster
,
update
the
centers
of
clusters
(as
the
rst
steps),
and
see
if
there
are
no
transfers
between
pix
els
from
an
y
cluster
to
another
.
The
result
will
be
the
v
ector
containing
the
cluster
indices
that
will
r
epresent
the
se
gments
in
the
image
on
which
we
apply
this
algorithm.
That
means,
this
step
will
produce
a
ne
w
image
with
a
group
of
se
gments
where
each
se
gment
is
represented
inside
the
image
with
a
limited
number
of
colors
where
each
color
represents
the
se
gment
from
the
original
image.
As
sho
wn
in
Figure
4,
the
ne
w
image
will
ha
v
e
a
limited
number
of
colors
as
labels,
each
color
represents
a
se
gment
inside
the
image.
These
se
gments
will
be
used
in
the
recognition
and
classication
steps.
Then
we
trim
the
background
area
adjacent
to
this
object
by
trimming
the
area
of
the
same
color
(so
that
the
image
contains
a
nite
number
of
colors
to
represent
each
color
se
gment).
W
e
apply
the
trimming
process
in
the
original
image
in
the
Lab
color
space
by
comparing
each
area
of
the
ne
w
image
(containing
a
nite
number
of
colors)
with
the
original
image
containing
all
colors.
Then
we
get
all
pix
els
that
ha
v
e
a
color
with
v
alue
C
that
found
in
the
inde
x
(
i
s
,
j
s
)
and
get
all
pix
els
from
the
original
image
with
the
same
indices,
b
ut
there
will
be
a
group
of
colors
found
within
the
inde
x
es
(
i
s
,
j
s
)
in
the
original
image.
This
operation
will
be
applied
for
all
three
layers
L
,
a
and
b
in
the
(
i
s
,
j
s
)
re
gions.
Figure
4.
The
result
from
applying
K-means
and
taking
local
min-max
v
alues
as
the
clusters
The
se
gmented
re
gions
of
the
k-means
output
are
transferred
to
the
original
image
in
the
same
pla
ce
for
all
layers.
The
areas
that
were
trimmed
in
the
pre
vious
stages
will
be
used
as
images
and
the
transfer
of
these
ne
w
images
to
the
recognition
and
classication
of
calls
in
the
ne
xt
stage.
After
se
gmenting
the
images,
each
se
gment
is
tak
en
separately
(these
se
gments
should
represent
a
meaningful
se
gment;
otherwis
e,
it
will
be
rejected).
3.2.
Recognition
step
At
this
stage,
the
se
gments
e
xtracted
from
the
original
image
must
be
recognized
as
sho
wn
in
the
pre
vious
stage.
As
discussed
in
the
pre
vious
sections,
all
se
gments
will
be
treated
as
a
ne
w
image.
The
Luminance
histogram
of
this
se
gment
is
calculated
using
a
v
ector
of
256
columns.
Ev
ery
3
adjacent
columns
are
mer
ged
by
taking
the
a
v
erage
for
these
three
column
v
alues,
as
sho
wn
in
Figure
5.
The
resulting
86
columns
are
mer
ged
ag
ain
(each
3
columns)
to
produce
29
columns.
This
operation
will
be
applied
to
all
images
in
the
data
set.
Ev
ery
image
will
be
represented
by
a
v
ector
of
29
columns,
which
means
a
v
ector
of
29
v
alues
instead
of
se
gments
of
N
×
M
pix
els.
W
e
just
need
a
v
ector
of
limited
size
of
29
v
alues.
The
operation
of
this
se
gment
will
con
v
ert
from
comple
xity
of
O
(
N
×
M
)
to
O
(
C
)
.
This
means
con
v
erting
from
the
quadratic
system
to
the
constant
system,
which
will
increase
the
speed
and
the
performance
and
mak
e
it
more
po
werful.
This
v
ector
will
be
used
in
the
classication
step
instead
of
the
original
se
gment.
Ima
g
e
classication
in
cultur
al
herita
g
e
(Muath
Sabha)
Evaluation Warning : The document was created with Spire.PDF for Python.
4728
❒
ISSN:
2252-8938
Figure
5.
Mer
ging
each
3
columns
of
the
histogram
by
taking
their
a
v
erage
3.3.
K-near
est
neighbor
calculations
The
KNN
algorithm
will
be
used
to
classify
these
se
gments
to
determine
to
which
class
each
se
gment
belongs,
of
which
cultural
heritage
is
stored
in
the
datas
et.
The
v
ector
that
is
produced
in
the
pre
vious
step
will
be
considered
and
all
the
v
ectors
that
are
stored
inside
the
dataset
that
hold
labels
to
see
which
of
these
v
ectors
that
are
nearest
to
the
ne
w
v
ector
by
using
this
formula:
d
1
(
I
1
,
I
2
)
=
P
P
|
I
p
1
−
I
p
2
|
Where
P
is
the
cell
number
or
the
inde
x
inside
the
v
ector
,
I
1
,
2
represents
the
ima
g
e
s
or
se
gments,
and
d
is
the
result
that
has
the
label
of
specic
cultural
heritage
that
the
ne
w
image
belongs
to.
W
e
subtract
the
v
ector
from
the
29
v
alues
that
represent
the
se
gment
produced
in
the
last
step
with
some
v
ectors
from
a
set
of
v
ectors
in
the
complete
data
set.
3.4.
Classication
After
that,
the
last
step
produces
a
group
of
v
alues,
each
v
alue
representing
the
sum
of
all
29
v
alues
produced
from
the
subtracting
operation.
Finding
the
m
inimum
v
alue
of
these
v
alues
and
looking
at
the
image
that
will
get
the
min
v
alue
from
these
operations
is
ne
xt.
This
image
will
hold
the
goal
cultural
heritage
for
this
se
gment
b
ut
not
the
whole
original
image
because
the
whole
original
image
has
man
y
se
gments,
some
of
them
not
cultural
features.
Instead
of
applying
the
KNN
algorithm
and
subtracting
the
entire
image
of
size
(
x
×
y
×
3)
(three
layers),
we
only
need
to
subtract
29
v
alues,
then
the
time
comple
xity
will
be
O(C)
instead
of
O
(
N
×
M
)
as
mentioned
earlier
,
which
has
a
signicant
ef
fect
for
a
lar
ge
number
of
pix
els
that
will
be
calculated.
Ev
ery
culture
has
common
characteristics,
and
the
classication
is
to
tak
e
e
v
ery
se
gment
of
the
image
tested
and
nd
its
match
from
the
v
ectors.
It
is
labeled
accordingly
and
then
decided
that
this
se
gment
belongs
to
a
group
that
represents
a
certain
cultural
heritage.
The
image
is
added
to
the
data
set,
and
the
data
set
is
gro
wn
by
one.
After
lling
in
the
v
ector
,
determine
to
which
cultural
heritage
group
the
image
belongs.
Ho
we
v
er
,
the
time
comple
xity
will
be
O
(
K
×
N
)
where
K
is
the
number
of
se
gments
in
each
image
in
data
set
and
N
is
the
number
of
images
in
data
set.
It
will
be
huge
for
huge
data
sets.
F
or
e
xample,
if
we
ha
v
e
100
,
000
images
in
the
data
set
a
n
d
each
image
has
50
se
gments,
then
the
number
of
iterations
will
be
100
,
000
×
50
that
will
produce
5
,
000
,
000
which
is
huge
in
real
time
system.
T
o
o
v
ercome
this
problem,
we
use
another
methodology
to
store
data
which
is
a
binary
search
tree.
This
means
that
each
se
gment
inside
specic
cultural
heritage
images
will
be
stored
in
a
binary
search
tree
that
represents
this
cultural
heritage.
The
aim
of
using
the
binary
search
tree
is
to
apply
its
concept
that
the
v
alue
of
the
root
is
lar
ger
than
the
left
child
and
less
than
the
right
child
that
gi
v
es
e
xibility
for
our
search,
Int
J
Artif
Intell,
V
ol.
13,
No.
4,
December
2024:
4722–4735
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Artif
Intell
ISSN:
2252-8938
❒
4729
instead
of
taking
O
(
k
)
by
us
ing
linear
search,
or
using
search
item
by
item
where
the
comple
xity
is
O
(
Log
(
k
))
then
it
will
tak
e
l
og
(
K
)
for
the
iteration,
instead
of
K
iterations.
This
means
that
we
do
not
need
to
test
the
whole
tree,
just
l
og
(
k
)
of
elements
from
this
tree
(that
the
binary
search
tree
is
just
testing
the
left
or
right
branch).
Then,
the
test
will
be
by
calculating
the
dif
ference
between
the
v
ector
and
the
v
ectors
in
the
data
set,
getting
the
summation,
and
testing
it
with
the
tree.
That
will
help
to
mak
e
the
decision
to
go
to
the
left
or
right
or
to
stop
i
f
the
v
alue
is
lar
ger
than
the
pre
vious
le
v
el.
Ev
ery
time
we
go
deep
in
the
left
branches,
the
v
alue
should
be
lo
wer
than
pre
vious
left
le
v
el
and
vice
v
ersa
for
the
right
side;
then
in
e
v
ery
test
we
will
discard
half
of
the
v
ectors
from
v
ectors
inside
the
tree
that
represent
specic
image;
then
its
v
alue
of
search
will
be
smaller
than
the
le
v
el
that
stands
in
it
before.
W
e
als
o
applied
three
classiers
including
KNN,
logistic
re
gression
(LR),
and
DT
.
The
splitting
rate
of
training
and
testing
has
been
set
to
%75
and
%25
respecti
v
ely
.
The
e
v
aluation
of
classication
results
will
be
based
on
the
common
information
retrie
v
al
metrics
including
precision,
recall,
and
f-measure.
4.
RESUL
TS
AND
DISCUSSION
After
around
150
e
xperiment
s
on
3200
images
of
cultural
heritage
from
the
Pinterest
data
set
of
20
cultures,
it
is
noticed
that
the
se
gmentation
step
has
se
v
eral
attrib
utes
to
consider
,
such
as
the
dependence
on
the
color
dif
ference
within
the
image.
Another
thing
is
that
se
gmentation
depends
on
image
r
esolution,
where
higher
resolution
results
in
better
classication.
There
is
another
attrib
ute
that
determines
the
resol
ution
of
the
color
-space
se
gmentation
which
is
the
channels
created
by
the
color
space
in
the
laboratory
,
which
made
the
selection
of
the
section
more
accurate
and
solv
ed
the
problem
created
by
the
lightness.
F
or
se
gmenting
certain
re
gions,
as
sho
wn
in
our
methodology
,
we
se
gment
the
images
by
taking
the
closed
re
gions.
As
sho
wn
in
Figure
6,
the
number
of
se
gments
is
3,
representing
the
columns,
the
sk
y
,
and
the
area
between
the
columns.
The
rst
se
gme
n
t
will
recognize
the
column
as
roman
culture,
the
other
se
gments
will
get
unkno
wn
label,
the
result
will
be
a
roman
cultural
image
depending
on
the
labeled
se
gment.
Figure
6.
Recognition
result
from
testing
a
roman
culture
image
4.1.
T
ime
complexity
In
Figure
7,
a
comparison
is
made
between
the
time
comple
xity
of
the
proposed
method
compared
to
the
def
ault
technique
(linear
binary
search
tree).
The
time
comple
xity
is
signicantly
reduced
in
the
proposed
technique,
where
the
comple
xity
of
the
proposed
technique
is
O
(
N
×
Log
(
K
))
,
while
it
is
O
(
N
×
K
)
in
the
linear
binary
tree.
As
sho
wn
in
Figure
7,
the
g
ap
between
the
tw
o
curv
es
increases
e
xponentially
as
the
x-axis
progresses
(number
of
se
gments).
Ima
g
e
classication
in
cultur
al
herita
g
e
(Muath
Sabha)
Evaluation Warning : The document was created with Spire.PDF for Python.
4730
❒
ISSN:
2252-8938
0
20
40
60
80
10
0
12
0
0
2
4
6
8
10
12
Ti
m
e
#i
m
a
g
es
*
1
0
Ti
m
e
C
o
m
p
l
e
x
i
t
y
O
(
N
*
L
o
g
K
)
O(
N
*
K
)
Figure
7.
T
ime
comple
xity
for
our
method
vs.
the
BST
Figure
8
compares
the
time
comple
xity
in
the
proposed
methodology
and
that
in
the
KNN
algorithm.
The
KNN
algorithm
subtracts
the
entire
image
tested
with
images
from
the
dataset
that
has
N
ro
ws
and
M
columns.
Ho
we
v
er
,
the
proposed
method
needs
only
to
subtract
a
v
ector
of
29
v
alues
that
is
static
for
all
images
inside
the
dataset.
The
g
ap
between
the
tw
o
curv
es
increases
as
the
number
of
images
increases,
and
this
is
another
bonus
point
for
the
proposed
methodology
.
0
20
00
0
40
00
0
60
00
0
80
00
0
10
00
00
12
00
00
0
50
10
0
15
0
20
0
25
0
30
0
35
0
Ti
m
e
#
im
ag
e
s
Ti
m
e
C
o
m
p
l
e
x
i
t
y
f
o
r
s
u
b
t
r
a
c
t
i
o
n
O(
N
*
M
)
29
v
a
l
u
e
Figure
8.
T
ime
comple
xity
for
our
subtraction
method
ag
ainst
subtraction
in
KNN
Figure
9
compares
the
accurac
y
of
the
results
by
using
three
dif
ferent
color
spaces,
CIELab,
HSV
,
and
RGB.
It
is
clear
that
the
curv
es
are
almost
parallel
to
each
other
,
and
the
results
are
almost
the
same
with
a
pri
vile
ge
to
the
CIE-Lab
color
space.
W
e
think
that
the
results
are
close
and
parallel
because
the
original
images
are
captured
in
the
RGB
color
space.
The
pri
vile
ge
of
Lab
and
HSV
comes
from
the
f
act
that
these
color
spaces
are
closer
to
human
perception.
The
CIE-Lab
color
space
has
a
bigger
g
amut
and
it
gi
v
es
better
results
in
the
k-means
algorithm
[30].
This
pro
v
es
that
our
method
in
using
tw
o
dimensions
out
of
the
Lab
color
space
(
ab
)
impro
v
ed
the
results.
Int
J
Artif
Intell,
V
ol.
13,
No.
4,
December
2024:
4722–4735
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Artif
Intell
ISSN:
2252-8938
❒
4731
0%
10
%
20
%
30
%
40
%
50
%
60
%
70
%
80
%
0
20
40
60
80
10
0
12
0
14
0
16
0
Ac
c
u
r
a
c
y
#
I
m
a
g
es
Ac
c
u
r
a
c
y
CI
E
L
A
B
c
o
l
o
r
s
p
a
c
e
HS
V
c
o
l
o
r
s
p
a
c
e
RG
B
c
o
l
o
r
s
p
a
c
e
Figure
9.
Accurac
y
using
CIELab,
RGB
and
HSV
color
spaces
Figure
10
compares
the
classication
accurac
y
of
the
proposed
technique
with
v
e
of
the
most
related
algorithms
and
techniques
from
pre
vious
research,
namely
hill
climbing,
gray
multile
v
el
thresholding
(GMT),
SRM,
te
xture
features,
and
k-means
color
clustering.
The
classication
accurac
y
is
measured
relati
v
e
to
the
human
classication.
It
is
noticed
that
the
closest
algorithms
are
the
te
xture
and
K-means
color
clustering.
The
y
depend
on
the
number
of
clusters
to
gi
v
e
a
nite
number
of
se
gments.
On
the
other
hand,
the
y
used
the
L
-layer
of
the
Lab
color
space
in
their
solution,
while
the
proposed
algorithm
concentrated
on
pure
color
to
mimic
human
perception.
0%
10
%
20
%
30
%
40
%
50
%
60
%
70
%
80
%
0
20
40
60
80
10
0
12
0
14
0
16
0
Ac
c
u
r
a
c
y
#
i
m
a
g
es
Co
m
p
a
r
i
s
o
n
mi
n
-
m
a
x
w
i
t
h
L
A
B
hi
l
l
c
l
i
m
b
i
n
g
Gr
a
y
M
u
l
t
i
l
e
v
e
l
T
h
r
e
s
h
o
l
d
i
n
g
SR
M
Te
x
t
u
r
e
F
e
a
t
u
r
e
s
K-
m
e
a
n
s
c
l
u
s
t
e
r
a
n
a
l
y
s
i
s
f
o
r
i
m
a
g
e
s
e
g
m
e
n
t
a
t
i
o
n
Figure
10.
Comparing
our
algorithm
(min-max
with
Lab)
to
v
e
other
well-kno
wn
most
related
algorithms
Hill
climbing
algorithm
produces
less
accurate
results,
and
that
is
because
it
depends
on
max
v
a
lue,
where
the
number
of
clusters
is
small,
and
it
remo
v
es
too
much
features
in
the
image.
Therefore,
the
number
of
se
gments
will
be
less
than
our
proposed
algorithm.
The
number
of
se
gments
in
our
algorithm
is
about
double.
The
gray
thresholding
uses
one
layer
while
our
algorithm
uses
tw
o
layers
from
a
perceptual
color
space,
which
uses
more
useful
information.
Finally
,
as
sho
wn
in
Figure
10,
our
technique
gi
v
es
the
best
result
in
cultural
heritage
features
among
all
the
compared
techniques.
Ima
g
e
classication
in
cultur
al
herita
g
e
(Muath
Sabha)
Evaluation Warning : The document was created with Spire.PDF for Python.