Indonesi
an
Journa
l
of El
ect
ri
cal Engineer
ing
an
d
Comp
ut
er
Scie
nce
Vo
l.
24
,
No.
1
,
Octo
be
r
20
21
,
pp.
48
4
~
490
IS
S
N:
25
02
-
4752, DO
I: 10
.11
591/ijeecs
.v
2
4
.i
1
.
pp
48
4
-
49
0
484
Journ
al h
om
e
page
:
http:
//
ij
eecs.i
aesc
or
e.c
om
Multi
-
lab
el c
l
assification
approa
ch
for Q
u
ra
ni
c verses l
abeli
ng
Ad
el
eke
Abdu
ll
ah
i
1
, No
or A
z
ah
Sa
m
sudin
2
, Mohd His
yam A
b
dul
Rahim
3
,
Sha
m
sul K
am
al Ahma
d
Kh
alid
4
, Risw
an
Efendi
5
1
,2,3,4
De
p
a
rt
m
ent
of
So
ftwa
re
Eng
ine
e
r
ing,
Univer
siti
Tun
Hus
sein O
nn
Malay
sia
,
P
ar
it Ra
j
a,
Bat
u
P
aha
t
,
Ma
lay
sia
5
De
p
a
rt
m
ent
of
Mathe
m
at
i
cs,
U
nive
rsit
as
Islam
Nege
ri
Sul
ta
n
Sy
ar
if
Kasim
R
iau,
R
i
au
,
Indone
s
ia
Art
ic
le
In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
Ma
r 5
,
2021
Re
vised
A
ug 6,
20
21
Accepte
d
Aug
11
,
2021
Mac
hine
l
ea
rn
in
g
invol
ves
the
t
ask
of
tra
ini
ng
s
y
stems
to
be
a
ble
to
m
ake
dec
isions
witho
ut
bei
ng
exp
li
c
i
tly
p
rogra
m
m
ed.
Im
porta
nt
among
m
ac
hine
le
arn
ing
t
asks
is
cl
assifi
cation
in
volvi
ng
th
e
proc
ess
of
tra
in
ing
m
ac
hine
s
t
o
m
ake
pre
dic
t
io
ns
from
pre
def
ine
d
l
abe
ls
.
Cla
ss
ifi
ca
t
ion
is
broa
dl
y
ca
t
egor
ized
in
to
thre
e
disti
n
ct
groups:
single
-
l
abe
l
(SL)
,
m
ulti
-
class
,
and
m
ult
i
-
la
b
el
(ML)
cl
assificat
ion
.
T
his
rese
arc
h
wor
k
pre
sents
an
ap
pli
c
at
ion
of
a
m
ult
i
-
la
b
el
c
las
sifi
ca
ti
on
(ML
C)
te
chni
qu
e
in
aut
om
at
ing
Qurani
c
ve
rses
la
be
li
ng.
MLC
h
as
bee
n
gai
n
ing
at
t
ent
ion
in
r
ec
e
nt
y
ea
rs.
Th
is
is
due
to
th
e
inc
re
asing
amount
of
works
bas
ed
on
rea
l
-
worl
d
cl
assificat
ion
proble
m
s
of
m
ult
i
-
la
b
el
data.
In
tra
dit
ion
al
class
ifi
cation
prob
l
ems
,
pat
te
rns
ar
e
associa
t
ed
with
a
singl
e
-
label
from
a
set
of
disjoi
nt
l
abel
s.
How
eve
r,
in
MLC,
a
n
insta
nc
e
of
dat
a
is
associa
te
d
wi
t
h
a
set
of
la
be
ls.
In
thi
s
pape
r,
thr
ee
stand
ard
MLC
m
et
hods:
bina
r
y
r
el
ev
an
ce
(BR)
,
c
la
ss
i
fie
r
ch
ai
n
(CC)
,
and
la
b
el
powerset
(LP)
al
gorit
hm
s
ar
e
i
m
ple
m
ent
ed
wit
h
four
base
li
ne
cl
assifi
ers
:
support
vec
tor
m
ac
hine
(
SVM
)
,
naï
v
e
B
a
y
es
(
NB)
,
k
-
nea
rest
nei
ghbors
(
k
-
NN
)
,
and
J48.
The
rese
arc
h
m
et
hodolog
y
ado
pts
the
m
ult
i
-
l
a
bel
probl
em
tra
nsform
at
ion
(PT)
appr
oa
ch.
The
result
s
are
v
al
id
ated
using
six
conve
nt
iona
l
pe
rform
anc
e
m
et
r
i
cs.
Th
ese
inc
lud
e:
h
amm
ing
loss,
a
cc
ur
a
c
y
,
one
err
or,
m
ic
r
o
-
F1,
m
ac
ro
-
F1,
and
avg.
pre
c
i
sion.
From
the
result
s,
th
e
cl
assifi
ers
eff
ec
t
ive
l
y
ac
h
ie
ved
above
70%
acc
ura
c
y
m
ark
.
Ov
era
l
l,
SV
M
ac
hi
eve
d
the be
s
t
resul
ts wi
th
CC a
nd
LP
a
lgori
th
m
s.
Ke
yw
or
d
s
:
Ho
ly
Qura
n
Ma
chine
le
a
rn
i
ng
Mult
i
-
la
bel cla
ssific
at
ion
Mult
i
-
la
bel ev
a
luati
on
m
et
rics
Text cla
ssific
at
ion
This
is an
open
acc
ess arti
cl
e
un
der
the
CC
B
Y
-
SA
l
ic
ense
.
Corres
pond
in
g
Aut
h
or
:
Ad
el
e
ke Abd
ullahi
Faculty
of Com
pu
te
r
Scie
nc
e an
d Inform
ation
Tech
nolo
gy
Un
i
ver
sit
i T
un
Hu
s
sei
n O
nn
Ma
la
ysi
a
Jo
ho
r,
8640
0
P
arit
Raja,
Bat
u Pahat,
Ma
la
ysi
a
Em
a
il
:
abd
ul20
40@yah
oo.c
om
1.
INTROD
U
CTION
The
fiel
d
of
m
achine
le
ar
ning
focuses
on
t
he
stu
dy
that
giv
es
a
rtific
ia
l
i
ntell
igence
(
AI
)
syst
e
m
s
the
capab
il
it
y
to
i
m
pr
ov
e
it
s
perform
ance
ov
er
a
tim
e
per
iod
throu
gh
ac
qu
i
rin
g
ne
w
knowle
dge
an
d
ski
ll
s
[1
]
.
Con
ce
ptu
al
ly
, ma
chine
le
arn
i
ng
is base
d
on trainin
g
m
achi
nes
to b
e
able
t
o
detect
patte
r
ns
an
d
a
dap
t
t
o
a
new
ci
rcu
m
sta
nce
[
2].
Im
portant
t
o
m
achine
le
ar
ning
is
t
he
pro
blem
of
cl
assif
ic
at
ion
,
t
he
ta
s
k
of
ide
ntifyi
ng
to
wh
ic
h
cat
egor
y/
cl
ass
an
ob
se
rv
at
io
n
(in
sta
nc
e)
belo
ngs
[
3].
Trad
it
io
nally
,
in
a
ty
pical
classificat
ion
pr
ob
le
m
,
the goal i
s t
o p
red
ic
t a
uto
m
at
i
cal
ly
o
ne of
th
e prede
fine
d
cl
asses eac
h
to
a
set
o
f
sam
ples.
Give
n
a
n
i
nput
,
the
goal
of
cl
assifi
cat
ion
is
to
le
ar
n
a
m
app
ing
from
input
to
ou
t
pu
t
wh
e
re
∈
{
,
…
,
}
,
rep
re
sentin
g
nu
m
ber
of
cl
as
ses.
This
is
re
f
err
e
d
to
as
a
si
ng
le
-
la
bel
cl
assifi
cat
ion
(
S
L
C)
pro
blem
.
Howe
ver,
in
s
om
e
real
-
w
or
l
d
cl
assifi
cat
io
n
pro
blem
s,
su
c
h
as
in
t
he
Q
uranic
te
xt
cl
ass
ific
at
ion
ta
s
k,
a
data
instance m
ay
b
e cat
egorized
into
m
ulti
ple cla
sses at t
he
sa
m
e tim
e.
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Multi
-
lab
el
cla
ssif
ic
ation
approac
h f
or
Q
ur
anic
verses
la
be
li
ng
(
Adeleke
Ab
du
ll
ahi
)
485
Fo
r
exam
ple,
a
verse
in
t
he
Q
ur
a
n
m
ay
be
ta
rg
et
e
d
to
wards
seve
ral
issues
(or
to
pics)
su
c
h
as
relat
ed
to
fai
th,
fam
ily,
wo
r
sh
i
p,
go
od
deeds,
pa
ra
dise,
hell
am
o
ng
oth
e
rs.
T
his
kind
of
cl
as
sific
at
ion
pro
bl
e
m
is
te
rm
ed
m
ult
i
-
l
abel
cl
assifi
cat
ion
(MLC
)
[
4].
In
MLC
,
w
hich
is
an
exte
nsi
on
of
the
c
onven
ti
onal
SLC
,
dat
a
instances a
re a
sso
ci
at
ed wit
h a set
of labels
Y
⊆
L
.
Pr
im
arily,
the
co
ncep
t
of
MLC
or
i
gin
at
ed
from
te
xt
[5
]
wh
e
re
of
te
n
do
c
um
ents
are
ass
ociat
ed
si
m
ultaneou
sly
with
m
ulti
pl
e
top
ic
s
su
c
h
as
new
s,
s
ports,
ed
ucati
on,
econom
y
et
c
.
The
te
ch
niques
of
MLC
hav
e
be
en
f
ur
t
her
a
ppli
ed
to
oth
e
r
cl
assifi
cat
ion
pro
blem
s
includi
ng
m
ark
et
ing
[6
]
,
im
aging
[7
]
,
m
ul
tim
edia
[8
]
,
an
d
genom
ic
s
[9
]
.
Alth
ough,
there
hav
e
bee
n
inc
reasin
g
a
m
ou
nt
of
resea
rch
w
orks
on
m
ul
ti
-
la
bel
cl
assifi
cat
ion
m
e
tho
ds
pro
posed
in
li
te
ratur
e
s,
ho
wever
in
the
Qura
nic
te
xt
cl
assifi
cat
ion
pro
blem
,
the
a
pp
li
cat
io
n
of
MLC
is
rela
ti
vely
new
.
H
ence,
this
pa
per
prese
nts
the
i
m
ple
m
entat
ion
of
m
ult
i
-
la
bel
cl
assifi
cat
ion
m
et
ho
ds an
d
al
gorithm
s ap
pli
cable i
n aut
oma
ti
ng
Qura
nic
ver
se
s labeli
ng
task.
In
this
w
ork,
s
ta
nd
a
rd
m
achine
le
ar
ning
al
gorithm
s
(classi
fiers)
are
a
pp
li
ed
for
t
he
m
ulti
-
la
bel
ta
sk.
The
e
xp
e
rim
ental
wo
r
k
in
volves
the
us
e
of
b
ina
ry
releva
nc
e
(BR),
cl
assi
fier
chai
n
(CC
),
a
nd
la
bel
po
wer
set
(LP)
al
gorithm
s.
These
MLC
m
et
ho
ds
will
be
us
e
d
to
cl
a
ssify
Qura
nic
ver
se
s
sim
ultan
eo
us
ly
into
one
or
m
or
e
pr
ed
e
fin
ed
cat
eg
or
ie
s
(or
cl
ass
la
bels)
nam
el
y:
fait
h
(“
im
an
”
),
wors
hip
(“
i
badah
”
),
an
d
et
iqu
et
te
s
(“
ak
hla
k
”
).
T
he
sel
ect
ed
cat
egories
are
fro
m
the
m
os
t
fund
am
ental
asp
ect
s
of
Islam
as
rec
ognized
by
th
e
Qura
n
e
xp
e
rts
[2
]
.
Gen
e
rall
y,
a
classificat
ion
ta
sk
is
th
e
pr
oble
m
of
pr
e
dicti
ng
cl
ass
la
bels
fo
r
an
in
sta
nce
descr
i
bed
by
a
finite
set
of
featur
e
s.
Give
n
a
set
of
at
tr
ibu
te
s
=
{
1
,
…
,
}
,
a
set
of
cl
ass
la
bels
=
{
1
,
…
,
}
,
a
trai
ning
datase
t
co
m
pr
isi
ng
of
instances
:
{
(
1
,
1
)
,
(
2
,
2
)
,
…
,
(
,
)
}
,
each
corr
esp
onds
to
a
n
at
tribu
te
vecto
r
(
1
,
…
,
)
that
stores
va
lues
(i
nfor
m
at
ion
)
f
or
t
he
s
et
of
at
trib
utes
in
,
an
d
eac
h
∈
corres
ponds to
a sin
gle cla
ss l
abel.
Fr
om
the
work
[
10
]
,
t
her
e
are
t
wo
cl
assic
al
appr
oac
hes
(or
m
et
ho
ds)
em
plo
ye
d
to
so
l
ve
cl
assifi
cat
ion
pro
blem
s
inv
ol
vin
g
m
ulti
-
lab
el
data:
p
robl
e
m
transf
orm
a
ti
on
(P
T
)
an
d
a
lgorit
hm
ada
ptati
on
(AA)
m
et
ho
ds
.
Pr
oble
m
transf
orm
ation
ap
proach
is
a
si
m
plifie
d
way
to
ad
dr
ess
MLC
pr
ob
le
m
s.
It
wo
r
ks
by
sel
ect
ing
fo
r
ea
ch
m
ulti
-
la
bel d
at
a insta
nce a si
ng
le
la
bel fro
m
it
s
m
ulti
-
la
bel subset
⊆
.
PT
m
et
ho
ds
ar
e
al
go
rithm
ind
epe
ndent
an
d
hav
e
bee
n
su
c
cessf
ully
e
m
pl
oyed
to
so
l
ve
cl
assifi
cat
io
n
pro
blem
s
[1
1
]
,
[
12]
.
I
n
oth
er
words,
the
m
eth
ods
wor
k
by
t
ran
s
f
or
m
ing
m
ulti
-
la
bel
cl
assifi
cat
ion
pro
blem
to
on
e
or
m
or
e
sing
le
-
la
bel
cl
assifi
cat
ion
pr
oble
m
s.
Ther
eaf
te
r,
any
of
the
avail
able
SL
C
al
go
rithm
s
su
ch
as
su
pp
or
t
vect
or
m
achine
s
(
S
V
Ms
)
,
naï
ve
B
ay
es,
k
-
near
est
neig
hbors
(
k
-
NN
)
,
neural
ne
tworks
,
an
d
de
ci
sion
trees can
b
e
im
plem
ented
dire
ct
ly
as b
asel
ine
classi
fiers.
On
t
he other
hand, alg
or
it
hm
ad
aptat
io
n
(als
o
re
ferre
d
to as
algorit
hm
d
ep
end
e
nt)
i
nvolve
s ex
te
ndi
ng
the
sin
gle
-
la
be
l
cl
assifi
ers
to
a
dap
t
a
nd
be
im
ple
m
e
nted
directl
y
in
m
ulti
-
la
bel
pro
blem
s
[5
]
,
[
10
]
.
AA
al
gorithm
s
are
sp
eci
fical
ly
dev
el
ope
d
to
so
lve
a
giv
e
n
m
ul
ti
-
la
bel
pro
blem
.
Hen
ce,
t
hey
la
ck
fle
xibi
li
t
y
and
sim
plici
t
y
[5
]
.
The
se
set
ba
cks
are
the
m
ai
n
reasons
w
hy
AA
m
et
ho
ds
hav
e
bee
n
le
ss
popu
la
r
com
par
e
d
to
the
PT
m
et
hods
.
E
xisti
ng
w
orks
base
d
on
AA
a
pp
ro
ac
h
incl
ud
e
probabil
ist
ic
m
et
ho
ds
[13],
neural
netw
orks [
14
]
,
[
15]
, supp
or
t
ve
ct
or
m
achines
[16
]
,
[
17
]
, a
nd
d
eci
sio
n
tre
es
[18
]
,
[
19]
.
This
stu
dy
em
plo
ye
d
t
he
PT
appr
oach
for
the
Q
ur
a
nic
te
xt
m
ulti
-
la
bel
classificat
ion
pro
blem
du
e
to
it
s
po
pula
rity
and
sim
plici
ty
.
Ther
e
a
re
sev
eral
al
gorithm
s
avail
able
for
i
m
ple
m
entati
on
base
d
on
the
P
T
appr
oach.
The
stud
y
em
plo
ye
d
three o
f
t
he
m
os
t
con
ven
ti
on
al
MLC
al
go
rithm
s:
BR
[2
0
]
,
[
21]
,
CC
[22
]
,
[
23]
,
and LP
[24
]
.
Review
of these
al
gorithm
s ar
e d
oc
um
ented
in
the
nex
t
sect
io
n.
2.
METHO
DS
A
ND M
ATERI
ALS
This
wor
k
in
volves
the
m
ul
ti
-
la
bel
cl
assif
ic
at
ion
of
Q
uranic
ver
ses
usi
ng
t
hr
ee
sta
ndar
d
ML
C
m
et
ho
ds
:
bin
a
ry
releva
nce
(
BR
),
cl
assifi
er
chain
(CC)
,
and
la
bel
pow
erset
(L
P)
al
gorithm
s.
The
MLC
al
gorithm
s
will
be
us
e
d
t
o
cl
a
ssify
the
i
nput
ver
se
s
int
o
on
e
or
m
or
e
of
t
he
prede
fine
d
la
be
ls:
fait
h
(“
i
man
”
)
,
wors
hip
(“
ib
adah
”
),
an
d
et
iqu
et
te
s
(“
akh
l
ak
”
)
.
Tra
diti
onal
sing
le
-
la
bel
al
go
rithm
s
su
ch
as
SV
Ms
a
re
no
t
capab
le
of
ha
nd
li
ng
the
cl
a
ssifi
cat
ion
of
m
ul
ti
ple
la
bels
si
m
ultaneou
sl
y.
In
this
paper,
f
our
sin
gle
-
la
bel
cl
assifi
cat
ion
a
lgorit
hm
s:
SV
Ms,
naï
ve
B
ay
es,
k
-
NN,
a
nd
decisi
on
trees
(J48)
are
im
pl
e
m
ented
as
ba
sel
ine
cl
assifi
ers
al
ong
with
the
MLC
m
et
ho
ds.
The
resea
rc
h
m
e
tho
dolo
gy
fo
ll
ows
the
pro
blem
transf
orm
ation
appr
oach
previ
ou
sly
e
xp
la
i
ne
d.
T
he
e
xperim
ental
w
orkf
l
ow
(as
s
how
n
i
n
Figure
1)
com
pr
ise
s
of
f
our
phases:
input data,
pr
e
-
processi
ng, pre
dicti
on
,
and
ou
tpu
t
resu
lt
s.
2.1.
E
xp
eri
m
ent
al
d
ataset
The
dataset
ex
per
im
ented
in
this
w
ork
as
gi
ve
n
in
Ta
ble
1
c
on
sist
s o
f
1098
ver
ses (
data
in
sta
nces)
o
f
the
Q
ur
a
nic
te
xt.
F
ro
m
the
c
la
ss
weig
ht
distribu
ti
on,
fait
h
(“
im
an
”
)
cl
as
s
la
bel
ha
s
the
m
os
t
cl
ass
m
e
m
ber
s
(in
pu
t ve
rses). Th
is i
s as ex
pe
ct
ed
since
m
os
t of
the
ayaat
(
ver
se
s)
of the Q
ura
n
a
re co
nnect
ed
to f
ai
th
(
im
an
).
The
pri
m
ary
s
ources
of
the
Qura
nic
te
xtu
a
l
data
are
the
En
glish
tran
sla
ti
on
of
the
Q
ur
an
by
A
bdulla
h
Y
usuf
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
2
4
, N
o.
1
,
Oct
o
ber
20
21
:
48
4
-
490
486
Ali
(obtai
ned
from
ww
w.quran
database
.or
g
)
a
nd
the
E
ngli
sh
com
m
ent
ary
by
I
bn
K
at
hir
(
ob
ta
ine
d
fr
om
www.
al
la
hsw
ord
.co
m
).
To
th
e
best
of
our
knowle
dge,
th
ere
is
no
a
vaila
bili
ty
of
sta
ndar
d
E
ng
li
sh
Qura
n
dataset
for m
ac
hin
e lea
rn
i
ng c
la
ssific
at
ion
ta
sk
s.
Figure
1. Ex
pe
rim
ental
w
orkfl
ow
Table
1
.
P
erce
ntage
c
om
po
sit
ion
of cla
ss la
be
ls
Dataset
No
of
Ins
tan
ces
Clas
s W
eig
h
t
Faith
(
ima
n
)
W
o
rsh
ip
(
ib
a
d
a
h
)
Etiqu
ettes
(
a
kh
la
k)
Faith
-
W
o
rsh
ip
(
ima
n
-
ib
a
d
a
h
)
Faith
-
Etiqu
ettes
(
ima
n
-
a
kh
la
k)
W
o
rsh
ip
-
Etiqu
ettes
(
ib
a
d
a
h
-
a
kh
l
a
k)
Faith
-
W
o
rsh
ip
-
Etiqu
ettes
(
i
ma
n
-
ib
a
d
a
h
-
a
kh
la
k)
Qu
ran
1098
1
0
5
1
.0
1
1
5
.0
2
4
9
.0
9
5
.0
6
4
.0
4
4
.0
5
1
.0
2.2.
Te
xt
p
re
-
pr
ocessin
g
Pr
e
processin
g
is
the
pr
oces
s
of
extracti
ng
featur
es,
nor
m
al
iz
ing
,
and
transfor
m
ing
te
xtu
al
data
su
it
able
f
or
a
na
ly
sis
and
im
pl
e
m
entat
ion
.
T
he
Quran
ic
te
xt
is
first
co
nv
e
rted
to
the
sta
ndard
at
trib
ute
-
re
la
ti
on
Fil
e
fo
rm
at
(A
RFF),
wh
ic
h
i
s
the
fo
rm
at
fo
r
m
achine
le
arn
i
ng
in
Me
ka
(an
exte
ns
io
n
of
Wek
a
m
achine
le
arn
in
g
s
oft
w
are).
T
her
ea
fter,
featu
res
a
re
gen
e
rated
f
rom
the
tran
sf
orm
ed
te
xt
us
in
g
Strin
gT
oWo
r
dV
ect
or
[25]
an
d
te
rm
fr
e
qu
e
ncy
-
i
nv
e
rse
do
c
um
ent
fr
eq
ue
ncy
(
TF
-
I
DF
)
[26
]
,
[
27
]
.
These
are
sta
ndar
d
filt
er
to
ols
f
or
at
tribu
te
s
(f
eat
ur
es
) ge
ner
at
io
n
a
nd ex
t
racti
on.
TF
-
ID
F
is
one
of
the
m
os
t
widely
-
us
ed
m
et
hod
for
acce
ssing
a
nd
m
ea
su
ri
ng
the
sig
ni
ficance
of
a
word
to
a
docu
m
ent.
TF
-
I
DF
is
a
co
m
bin
at
i
on
of
tw
o
sta
tisti
cal
weigh
ti
ng
m
e
tho
ds
:
te
r
m
fr
equ
e
ncy
(
TF)
an
d
inv
e
rse
docum
ent
fr
e
qu
e
ncy
(
ID
F
).
T
he
te
r
m
fr
equ
e
ncy
(
,
)
of
a
par
ti
cular
word
as
ex
pre
ssed
in
(
1)
is
def
i
ned
as
t
he
nu
m
ber
of
ti
m
es
a
wor
d
ap
pe
ars
in
a
do
c
um
ent
.
In
a
dd
it
ion
,
i
nv
e
rse
-
do
cum
ent
fr
eq
ue
ncy
(expr
e
ssed
in
(
2
)
)
is a
m
et
ho
d use
d
t
o
f
urt
her ve
rify if a
ter
m
is com
m
on
/rare ac
r
os
s all
do
c
um
ents.
(
,
)
=
0
.
5
+
0
.
5
×
(
,
)
(1)
(
,
)
=
⎹
{
∈
:
∈
}
⎸
(
2
)
wh
e
re
is
the
total
nu
m
ber
of
d
oc
um
ents
in
,
⎹
{
∈
:
∈
}
⎸
is
the
num
ber
of
doc
um
ents
wh
e
re
featur
e
d.
2.3.
Multi
-
l
abel
cl
as
sific
at
i
on
(
MLC) m
odel
s
Mult
i
-
la
bel
cl
assifi
cat
ion
is
th
e
ta
sk
of
cat
e
gorizi
ng
(
or
pr
e
dicti
ng)
a
set
of
data
instan
ce
s
into
one
or
m
or
e
pr
ede
fin
ed
la
bels
us
in
g
m
ult
i
-
la
bel
cl
assifi
cat
ion
al
gorithm
s.
In
this
ex
per
im
ent
al
wo
r
k,
the
pro
blem
trans
form
ation
(P
T)
a
ppro
ac
h
is
ado
pte
d
f
o
r
the
cl
assifi
cat
i
on
ta
s
k.
The
st
ud
y
im
ple
m
ent
ed
three
of
the
m
os
t
app
li
ed
PT
m
et
hods
:
b
ina
ry
releva
nce
(B
R),
c
la
ssifie
r
chain
(CC),
a
nd
l
abel
pow
erset
(L
P).
I
n
the
cl
assifi
cat
ion
/p
red
ic
ti
on
phas
e,
strat
ifie
d
10
-
f
old
c
ro
s
s
val
idati
on
m
et
ho
d
[28
]
,
[
29]
is
u
se
d
f
or
t
he
tr
ai
ning
and
te
sti
ng
process.
Fou
r
tr
aditi
on
al
s
up
e
rv
ise
d
le
ar
ning
al
gorithm
s:
SV
Ms,
NB,
k
NN,
an
d
J
48,
wer
e
i
m
ple
m
ented
as
baseli
ne
sin
gl
e
-
la
bel
cl
assifi
ers
with
def
a
ul
t
par
am
et
er
val
ues
as
sp
eci
fied
in
Me
ka
To
ol
box
for
m
achine
l
earn
i
ng
pro
j
ec
t
s
(o
btain
ed
f
ro
m
https:/
/sou
rce
for
ge.net/
pro
j
ect
s/m
eka/
)
.
The
input
to
the
cl
assifi
er
is
a
Quran
ic
ver
s
e
represente
d
by
a
vecto
r
of
te
rm
cou
nt,
wh
il
e
the
ou
t
pu
ts
from
the
MLC
cl
assifi
ers
are
the
pr
e
de
fine
d
cl
ass
la
bels:
fai
th
‘
iman
’,
w
orship
‘
i
badah
’,
e
tiq
uette
s
‘
ak
hl
ak
’.
The
m
ult
i
-
la
bel
cl
assifi
cat
ion
m
et
ho
ds are e
xpla
ined
as
fo
ll
ow
s:
1)
Bi
nar
y
r
el
eva
nc
e
(BR)
is
the
m
os
t
widely
-
a
pp
li
ed
pro
blem
transf
orm
at
i
on
m
et
ho
d.
T
he
MLC
al
go
rithm
works
by
trai
ni
ng
m
ulti
ple
sing
le
-
la
bel
bin
a
r
y
cl
assifi
ers.
It
buil
ds
M
bi
nar
y
cl
assifi
ers,
one
f
or
e
ach
la
bel
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Multi
-
lab
el
cla
ssif
ic
ation
approac
h f
or
Q
ur
anic
verses
la
be
li
ng
(
Adeleke
Ab
du
ll
ahi
)
487
L
(wher
e
M
=
L
).
I
n
tu
rn,
each
cl
assif
ie
r
pr
e
dicts
a
ye
s/no
(i.e.,
0/1)
per
cl
ass.
F
or
a
new
in
sta
nce
,
the
BR
m
et
ho
d o
utput
s all
the
po
sit
iv
el
y pr
e
dicte
d
l
abels
l
i
by the
M
cl
assifi
ers.
2)
C
la
ssifie
r
c
hain
(CC
)
is
al
so
on
e
of
the
m
os
t
po
pula
r
m
ulti
-
la
bel
cl
assifi
cat
ion
m
e
tho
ds
base
d
on
pro
ble
m
trans
form
ation
ap
proac
h.
CC
is
a
direct
exte
ns
io
n
of
bi
nar
y
releva
nce
(BR
)
m
et
ho
d.
Th
e
MLC
al
gorithm
ta
kes
int
o
co
nsi
der
at
io
n
la
be
l
dep
e
ndency
wh
il
e
retai
ning
the
sim
plicit
y
and
e
ff
ic
ie
nc
y
of
t
he
bin
a
ry
releva
nce
m
eth
od.
CC
wor
ks
si
m
il
ar
to
B
R
by
trai
ning
f
irst
a
cl
assifi
er
for
each
la
bel
L
(wher
e
M
=
L
).
Howe
ver
di
ff
e
ren
t
f
ro
m
BR
,
the
al
gorithm
m
akes
pr
e
dicti
on
s
base
d
on
the
chain
order
seq
uen
ce
of
la
bels
rand
om
l
y
init
i
at
ed.
T
he
va
lu
e
of
t
he
fi
rst
la
bel
in
the
seq
ue
nce
is
pre
dicte
d,
th
en
t
he
pr
edict
ed
value
a
long
with
it
s
instan
ce
will
be
us
e
d
as
in
pu
t
t
o
pr
e
dict
the
val
ue
of
the
ne
xt
la
bel.
This
process
c
on
ti
nue
s
fo
ll
owin
g
t
he
r
andom
ly
o
rd
er
ed
c
hain seq
ue
nce
un
ti
l t
he
last
la
bel is pre
dicte
d.
3)
Label
p
ow
e
rse
t
(LP)
m
ult
i
-
la
bel
cl
assifi
cat
ion
al
gorithm
is
a
si
m
ple
but
le
ss
po
pula
r
of
the
pro
bl
em
trans
form
ation
m
et
ho
ds
[
3
0
]
.
The
MLC
al
gorithm
ta
kes
into
co
ns
ide
rat
ion
la
bel
c
orre
la
ti
on
s
that
m
a
y
exist
a
m
on
g
the
cl
ass
la
bels.
It
con
si
der
s
each
set
of
la
bels
in
the
m
ulti
-
la
bel
trai
nin
g
data
as
on
e
of
the
la
bels
of
a
new
si
ng
le
-
la
bel
cl
assifi
cat
ion
pro
blem
.
F
or
a
new
in
sta
nce,
the
si
ng
le
-
la
bel
cl
assifi
e
r
pr
e
dicts
the
m
os
t
li
kely
la
bel
(w
hic
h
in
ret
urn
is
a
set
of
la
bels).
Th
e
m
ajo
r
set
back
with
LP
is
high
com
plexity
[
3
0
]
as a r
es
ult o
f l
arg
e
num
ber
of
po
ssi
ble label
subsets c
om
bin
at
ions t
hat c
ould e
xist.
2.4.
E
va
lu
at
i
on
m
et
ri
cs
In
m
ulti
-
la
bel
cl
assifi
cat
ion
ta
sk
,
there
a
r
e
sta
nd
a
rd
perform
ance
m
ea
su
res
dif
fer
e
nt
fr
om
tho
se
conve
ntion
al
ly
us
ed
i
n
sin
gle
-
la
bel
cl
assifi
cat
ion
pr
ob
le
m
s
.
Am
on
g
t
hese
include
ham
m
ing
loss,
one
error,
rankin
g
los
s,
recall
,
pr
eci
si
on,
accu
racy,
and
a
v
g.
P
r
eci
sion
.
In
t
he
exp
e
rim
enta
l
wo
r
k,
six
s
ta
nd
a
rd
perform
ance
m
et
rics
wer
e
e
m
plo
ye
d
for
evaluati
ng
t
he
m
ult
i
-
la
bel
cl
assifi
cat
ion
al
gorithm
s.
Given
an
evaluati
on
dat
aset
:
(
,
)
;
=
1
,
…
,
denotes
a
m
ulti
-
la
bel
data
sam
ple
,
⊆
denotes
s
et
of
tr
ue
la
bels,
=
{
λj
∶
=
1
,
…
,
}
de
note
s
set
of
al
l
la
bels
,
de
note
s
set
of
pre
dicte
d
la
be
ls,
a
nd
(
)
denote
s
ra
nk
pr
e
dicte
d for a
la
bel
λ,
t
he per
form
ance
m
easur
es
are
explai
ned as
fo
ll
ows:
1)
Ham
m
ing
loss
[
3
1
]
is
a
sta
ndar
d
perf
or
m
ance
m
et
ric
that
ta
kes
i
nto
c
on
siderati
on
pr
e
di
ct
ion
er
r
or
s
(i.
e.,
inco
rr
ect
la
bel
s),
an
d
al
so
m
issi
ng
er
rors
(i
.e.,
la
bels
not
pr
e
dicte
d).
Th
e
m
et
ric
is
us
ed
to
evaluate
the
fr
e
qu
e
ncy
of
a
m
isc
la
ssifie
d
la
bel.
T
he
be
st
perform
ance
i
s
at
ta
ined
w
he
n
ham
m
ing
loss
value
is
e
qual
to
zero i
.e., t
he
s
m
al
le
r
the h
am
m
ing
loss
, th
e
bette
r
the
p
e
rfo
rm
ance o
f
t
he M
LC m
e
tho
d.
=
1
∑
|
△
|
=
1
(
3
)
2)
Accuracy
is
use
d
to
sym
m
et
ri
cal
ly
m
easur
e h
ow cl
os
e a
set
o
f
tr
ue
la
bels
(
)
is t
o a set
of
pr
e
dicte
d
la
bel
s
(
)
[
3
2
]
. T
hus,
the
higher
the a
ccur
acy
value
, t
he
bette
r
the
pe
rfor
m
ance of
the MLC
m
et
ho
d.
=
1
∑
|
∩
∪
|
=
1
(
4
)
3)
On
e
er
ror
e
val
uation
m
et
ric
[3
3
]
is
us
e
d
t
o
m
easur
e
the
f
r
equ
e
ncy
of
the
top
-
ra
nked
la
be
l
that
was
not
in
the set
of
t
ru
e
labels.
As
it
s
va
lue ten
ds
t
ow
a
r
ds
ze
ro, the
b
e
st perf
or
m
ance is r
eac
hed.
=
1
∑
(
∈
(
)
)
=
1
(
5
)
4)
Avg.
pr
eci
si
on
m
easur
es
the
aver
a
ge
f
racti
on
of
la
bels
ra
nked
a
bove
a
pa
rtic
ular
la
bel
∈
,
w
hich
is
act
ually
in
.
It
is
the
ave
ra
ge
of
pr
eci
sio
n
ta
ken
for
al
l
pos
sible
la
bels
.
T
he
best
res
ult
i
s
ac
hieve
d
w
he
n
avg.
pr
eci
sio
n
i
s 1
[
3
4
]
. T
hus
, a
larg
e
r
a
vg. pr
eci
si
on
’
s
value
sig
nifies a
bette
r per
form
ance.
.
=
1
∑
|
△
|
|
|
=
1
(
6
)
5)
Mi
cro
-
a
ver
a
ge
d
F
-
m
easur
e
(
Mi
cro
-
F1)
[3
4
]
rep
rese
nts
ha
rm
on
ic
m
ean
of
m
ic
ro
-
preci
sion
(Mic
-
P)
a
nd
m
ic
ro
-
recall
(
Mi
c
-
R).
−
1
=
2
×
(
−
)
×
(
−
)
(
−
)
+
(
−
)
(
7
)
6)
Ma
cro
-
ave
rag
e
d
F
-
m
easur
e
(
Ma
cro
-
F
1)
[3
4
]
represents
ha
rm
on
ic
m
ean
of
m
acro
-
preci
sion
an
d
m
acro
-
recall
.
−
1
=
2
×
(
−
)
×
(
−
)
|
|
(
8
)
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
2
4
, N
o.
1
,
Oct
o
ber
20
21
:
48
4
-
490
488
3.
E
X
PERI
MEN
TAL
RES
UL
TS A
ND AN
A
LYSIS
This
sect
ion
r
eports
the
ex
pe
rim
ental
resu
l
ts
of
the
stu
dy
.
Im
ple
m
entat
i
on
was
car
ried
out
us
in
g
three
sta
ndar
d
m
ul
ti
-
la
bel
cl
a
ssific
at
ion
m
eth
ods:
BR
,
CC
,
and
LP
.
In
a
ddit
ion
,
four
tra
diti
on
al
sin
gle
-
la
bel
cl
assifi
ers:
S
V
Ms,
NB
,
k
-
NN,
an
d
J48
we
re
us
e
d
as
baseli
ne
cl
assifi
ers.
A
lso,
si
x
sta
nd
a
r
d
e
valuat
io
n
m
et
rics
wer
e
appli
ed
t
o vali
date the
e
ff
ect
ive
ness o
f t
he
cl
assifi
cat
ion al
gorithm
s.
The
res
ults
obta
ined
us
in
g
the
MLC
m
et
hods
al
ong
with
the
SLC
baseli
ne
cl
assifi
ers
we
re
exh
a
us
ti
vely
c
om
par
ed.
Ta
bl
es 2
to 4 s
how
ed
the
resu
lt
s c
om
par
ison
i
n
te
rm
s o
f
ham
m
i
ng
l
os
s,
acc
ura
cy
, o
ne
error,
av
g.
pr
e
ci
sion
,
m
ic
ro
-
F1
,
a
nd
m
acro
-
F1.
I
n
the
bold
are
t
he
best
r
esults
achiev
ed
by
the
ba
sel
in
e
SLC
al
gorithm
s w
it
h resp
ect
t
o
ea
ch of
the e
valu
at
ion
m
et
rics and MLC
m
et
ho
ds.
Table
2
.
M
ulti
-
la
bel cla
ssific
at
ion
resu
l
ts
us
i
ng
bin
a
ry r
el
ev
an
ce al
gorithm
Evalu
atio
n
m
e
tri
cs
BR
NB
SVM
k
-
NN
J4
8
Accurac
y
↑
0
.77
8
0
.85
2
0
.82
3
0
.83
8
Ha
m
m
i
n
g
los
s↓
0
.18
6
0
.10
7
0
.12
9
0
.11
4
On
e er
ror↓
0
.07
1
0
.03
7
0
.05
1
0
.10
1
Micr
o
-
F1
↑
0
.80
7
0
.87
5
0
.84
3
0
.86
6
Macr
o
-
F1
↑
0
.83
9
0
.89
3
0
.86
6
0
.88
Av
g
.
Precisio
n
↑
0
.58
9
0
.57
8
0
.60
2
0
.56
5
Table
3
.
M
ulti
-
la
bel cla
ssific
at
ion
resu
lt
s
us
i
ng
cl
assifi
er chai
n al
gorithm
Evalu
atio
n
m
e
tri
cs
CC
NB
SVM
k
-
NN
J4
8
Accurac
y
↑
0
.77
7
0
.86
0
.81
8
0
.84
1
Ha
m
m
i
n
g
los
s↓
0
.18
7
0
.10
6
0
.13
3
0
.11
5
On
e er
ror↓
0
.04
7
0
.03
5
0
.06
0
.04
5
Micr
o
-
F1
↑
0
.80
6
0
.88
0
.83
6
0
.86
5
Macr
o
-
F1
↑
0
.83
6
0
.87
8
0
.86
0
.88
2
Av
g
.
Precisio
n
↑
0
.65
2
0
.58
1
0
.60
8
0
.59
5
Table
4
.
M
ulti
-
la
bel cla
ssific
at
ion
resu
lt
s
us
i
ng label
powe
r
set
algorit
hm
Evalu
atio
n
m
e
tri
cs
LP
NB
SVM
k
-
NN
J4
8
Accurac
y
↑
0
.79
7
0
.86
0
.81
7
0
.82
9
Ha
m
m
i
n
g
los
s↓
0
.16
3
0
.10
3
0
.13
4
0
.12
5
On
e er
ror↓
0
.03
4
0
.03
9
0
.06
0
.05
1
Micr
o
-
F1
↑
0
.82
7
0
.88
0
.83
7
0
.85
4
Macr
o
-
F1
↑
0
.85
5
0
.89
8
0
.85
9
0
.87
3
Av
g
.
Precisio
n
↑
0
.61
4
0
.58
3
0
.60
6
0
.60
6
In
Ta
ble
2,
i
m
plem
entat
ion
with
m
ulti
-
la
bel
BR
m
e
tho
d
sh
owe
d
va
ryi
ng
res
ults
acr
oss
the
baseli
ne
SLC
al
gorithm
s.
S
VM
cl
assif
ie
r
achie
ve
d
th
e
best
res
ults
i
n
m
os
t
of
the
m
et
rics
evalua
te
d,
wh
il
e
deci
sion
trees
(J48)
al
gorithm
fo
ll
ow
e
d
cl
os
el
y.
The
NB
al
gori
thm
had
the
le
ast
resu
lt
s
acro
ss
t
he
evaluati
on
m
et
rics.
This
co
uld
be
du
e
to
the
natur
e
of
the
ex
pe
rim
ental
dataset
since
m
os
t
le
arn
in
g
al
gor
it
h
m
s
are
sensiti
ve
to
data.
I
n
ad
diti
on,
the
com
bin
at
ion
of
th
e
bi
nar
y
releva
nc
e
MLC
m
e
thod
with
the
le
ar
ning
al
g
or
it
hms
cou
l
d
hav
e
a si
gn
ific
ant in
flue
nce
on the
classi
fica
ti
on
perform
ance.
Assessi
ng
t
he
perform
ance
of
the
CC
m
ulti
-
la
bel
cl
assifi
cat
ion
m
et
ho
d
li
ke
wise
sho
wed
com
petit
ive
resu
lt
s.
It
c
ould
be
see
n
that
SV
M
cl
assifi
e
r
agai
n
ac
hieve
d
th
e
best
re
su
l
ts
a
cr
os
s
al
l
e
va
luati
on
m
et
ric
s
us
e
d
excep
t
for
av
g.
pr
eci
sio
n
wh
e
re
the
naï
ve
B
ay
es
al
go
rithm
disp
la
ce
d
the
cl
assifi
er
to
to
p
posit
ion
ac
hi
evin
g
65.2%
avg.
preci
sion
val
ue.
Fu
rthe
rm
or
e,
NB
cl
assifi
cat
ion
al
gorithm
had
the
le
ast
resu
lt
s
with
the
CC
m
et
ho
d
cl
o
sel
y
si
m
i
la
r
to
th
e
bin
a
ry
rele
va
nce
m
et
ho
d.
Since
cl
assifi
e
r
chai
n
MLC
al
gorithm
ta
kes
into
consi
der
at
io
n
l
abels
co
rr
el
at
i
on,
this
had
im
pr
ov
em
ent
ov
er
t
he
bi
nar
y
releva
nce
m
eth
od.
C
onsist
en
tl
y,
the
com
bin
at
ion
of CC
and the
b
a
sel
ine SLC al
gorithm
s p
er
for
m
ed
bette
r
ac
r
os
s t
he per
for
m
ance m
easur
es.
Table
4
repor
ts
the
cl
assifi
cat
ion
p
er
f
or
m
ance
with
LP
m
ult
i
-
la
bel
cl
assifi
cat
ion
al
gorith
m
.
Fr
o
m
the
ta
ble,
SV
M
cl
assifi
cat
ion
m
od
el
consi
ste
ntly
pr
ove
d
to
be
an
eff
ic
ie
nt
an
d
powe
rful
le
arn
i
ng
al
gorith
m
.
The
b
asel
ine
cl
assi
fier
had
t
he
overall
hi
gh
e
st
scor
e
s
of
86
%
accuracy,
88
%
m
ic
ro
-
F
1,
89.
8%
m
acro
-
F
1,
a
nd
10.3%
ham
m
i
ng
l
os
s.
I
n
te
r
m
s
of
error
rate
and
av
g.
pre
ci
sion
,
naïve
B
ay
es
cl
assifi
er
ha
d
bette
r
re
su
lt
s
of
0.034
a
nd
61.4%
res
pecti
vely
.
As
pr
e
viousl
y
est
ablished
,
t
he
natu
re
of
e
xp
e
rim
ental
dataset
s
as
well
as
the
MLC
m
e
tho
ds
app
li
ed
on t
he l
earn
in
g
al
gorithm
s
m
ay
sign
ific
antly
inf
l
uence t
he
cl
assifi
c
at
ion
perform
a
nce.
In
ge
ner
al
,
a
na
ly
sis
of
the
cl
a
ssific
at
ion
perf
or
m
ance
of
B
R,
CC
,
an
d
L
P
m
ult
i
-
la
bel
c
la
ssific
at
ion
m
et
ho
ds
sho
w
ed
c
om
petit
iv
e
res
ults
with
the
baseli
ne
cl
assifi
ers:
S
V
M,
NB,
k
-
N
N
,
an
d
J
48
le
a
rn
i
ng
al
gorithm
s.
This is d
ue
t
o
the
fact t
hat ev
e
ry cla
ssifie
r
has
it
s strength a
nd
weakness
. I
t i
s
diff
ic
ult t
o
co
nc
lud
e
on
on
e
ulti
m
ate
best
cl
assi
fie
r.
H
ow
e
ve
r,
t
he
SV
M
cl
assif
ic
at
ion
al
gorithm
pr
ov
e
d
to
be
a
con
sist
en
t
and
eff
ic
ie
nt
cl
assi
fier.
It
ac
hieve
d
with
BR
,
CC
an
d
L
P
m
ulti
-
la
bel
cl
assifi
cat
ion
m
et
ho
ds,
the
be
st
acc
ur
acy
,
m
ic
ro
-
F
1,
a
nd
m
acro
-
F
1
re
su
l
ts
of
86%,
88
%,
89.
8%
res
pe
ct
ively
.
Fo
ll
ow
ed
cl
os
el
y
is
the
decisi
on
tre
e
(J48)
with
the
sec
ond
hi
gh
est
acc
uracy
,
m
ic
ro
-
F
1,
an
d
m
acro
-
F
1
res
ults
of
84.
1%,
86.6%,
88.2%
res
pecti
vely
.
Con
se
quently
,
SV
M
cl
assifi
e
r
pe
rfo
rm
ed
be
st
with
the
MLC
m
et
ho
ds
achievi
ng
the
best
lo
west
ham
m
ing
loss
(
0.1
03),
w
hi
le
naïv
e
B
ay
es
cl
assifi
er
ha
d
the
lo
west
er
ror
rate
(
0.034
)
an
d
highest
a
vg.
preci
sion
va
lue
of
65.2%.
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Multi
-
lab
el
cla
ssif
ic
ation
approac
h f
or
Q
ur
anic
verses
la
be
li
ng
(
Adeleke
Ab
du
ll
ahi
)
489
4.
CONCL
US
I
O
N
This
researc
h
s
tud
y
is
based
on
the
app
li
cat
ion
of
m
ulti
-
la
bel
cl
assifi
cat
io
n
m
e
tho
ds
in
Qura
nic
te
xt
(v
e
rses
)
la
beli
ng
pr
ob
le
m
.
I
n
the
e
xp
e
rim
ental
w
ork,
three
MLC
al
gorithm
s:
BR
,
CC
,
an
d
LP
we
re
i
m
ple
m
ented
with
four
tra
diti
on
al
sin
gle
-
la
bel
cl
assifi
ers:
NB,
SV
M
,
k
-
NN,
a
nd
J
48.
The
im
ple
m
entat
ion
fo
ll
owe
d
the
PT
strat
e
gy,
wh
e
re
t
he
sta
ndar
d
SLC
al
gorithm
s
functi
on
e
d
as
t
he
ba
sel
ine
cl
assifi
ers.
The
cl
assifi
cat
ion
pe
rfor
m
ance
wa
s
validat
ed
e
xhaustivel
y
us
in
g
six
sta
nd
a
r
d
evaluati
on
m
et
r
ic
s
of
te
n
em
plo
ye
d
i
n
MLC
pr
ob
le
m
s.
Con
sist
ently
,
the
S
VM
cl
assifi
er
in
com
bin
at
io
n
with
t
he
MLC
m
et
ho
ds
achie
ve
d
the
to
p
ranke
d
po
sit
io
n.
The
SLC
al
gorithm
achieved
the
overal
l
best
resu
lt
s
a
cro
ss
the
pe
rfor
m
ance
m
et
ri
cs.
We
cou
l
d
infe
r
fro
m
the
cl
assifi
c
at
ion
res
ults
that
S
VM
le
arn
ing
al
go
rithm
i
s
ver
y
eff
ic
ie
nt
with
relat
ively
la
rge
dataset
.
I
n
the
fu
tu
re
w
orks
,
we
lookin
g
f
orward
to
e
xp
l
or
i
ng
a
nd
im
ple
m
enting
ML
C
te
chn
iq
ues
t
o
ot
her
relat
ed
te
xt
cl
assifi
cat
ion
prob
le
m
s.
Also,
the
stu
dy
will
fo
c
us
on
the
de
velo
pm
ent
of
a
co
m
plete
Eng
li
s
h
Qura
n datase
t t
hat could
b
e
st
and
a
r
dized fo
r m
achine lear
nin
g t
asks
.
ACKN
OWLE
DGE
MENTS
The
a
uthors
w
ou
l
d
li
ke
to
t
ha
nk
t
he
Mi
nist
ry
of
Highe
r
Ed
ucati
on,
Ma
la
ysi
a
fo
r
s
up
portin
g
this
researc
h
un
der
Fu
ndam
ental
Re
search
G
r
ant
Schem
e
Vo
t
K21
3
(F
R
GS
/1/
2019/IC
T0
2/UT
HM/0
2/
2)
an
d
Un
i
ver
sit
i T
un
Hu
s
sei
n O
nn
Ma
la
ysi
a fo
r M
ulti
discipli
na
ry Rese
arc
h,
V
ot H5
11.
REFERE
NCE
S
[1]
A.
O
Adele
ke
,
N.
A.
Sam
sudin,
A.
Mus
ta
pha
,
and
N.
M.
Naw
i,
“
Com
par
at
iv
e
Anal
y
s
is
of
Text
Cla
ss
ifica
t
ion
Algorit
hm
s
for
Autom
at
ed
L
abelli
ng
of
Qurani
c
Verses,”
In
t.
J.
on
Adv
an
ce
Scie
nce
,
Engi
n
ee
rin
g
and
Info
.
Te
ch
,
vol.
7
,
no
.
4
,
pp
.
1419
-
1427,
201
7
,
doi
:
10
.
18517
/i
ja
se
it.7.
4
.
2198
.
[2]
A.
Adele
ke
,
N.
Sam
sudi
n,
A.
M
ustapha
,
and
S.
A.
Khali
d,
“
Autom
at
ing
Qurani
c
Verses
La
bel
in
g
Us
ing
Mac
hine
Le
arn
ing
Approac
h”
Indone
sian
Journal
of
El
ectric
al
Engi
n
ee
ri
ng
and
Computer
Sci
enc
e
,
vol.
1
6,
no.
2,
pp.
925
-
931,
2019
,
doi
:
10.
11591/i
j
eecs.v
16.
i2.
pp925
-
931
.
[3]
J.
Hart
m
aa
n,
J.
Huppert
z,
C
.
Schamp,
and
M.
H
ei
tmann,
“
Com
par
ing
aut
om
ated
te
xt
class
ifi
c
at
io
n
m
et
hod,
”
Int
J
.
of
R
ese
arch
in Mar
ke
ti
ng
,
vo
l.
36,
no
.
1
,
pp
.
20
-
38,
2019
,
doi
:
10.
1016/j.ij
r
esm
ar.
2018.
09
.
009
.
[4]
N.
K.
Mishra
and
P.
K.
Singh,
“FS
-
MLC:
Feat
ure
Sele
c
ti
on
for
Multi
-
la
b
el
c
la
ss
ifi
c
at
ion
using
c
l
usteri
ng
in
fe
at
u
re
spac
e,”
In
formation
Proc
essing
&
M
anageme
nt,
vol.
5
7
,
m
o.
4,
2
020
,
doi
:
10
.
101
6/j
.
ipm.2020
.
10
2240
.
[5]
S.
M.
Garc
ia,
C
.
J.
Manta
s,
J.
G.
Caste
llano,
a
nd
J.
Abell
an
,
“
Non
-
par
ametr
i
c
pre
dictive
i
nfe
re
nce
for
solving
m
ult
i
-
la
b
el
class
ifi
c
at
ion
,
”
Appli
ed
Sof
t
Computi
ng,
vol
.
88
,
2020
,
doi
:
10
.
1016/j.a
soc.
2019.
10601
1
.
[6]
M.
Bogae
rt,
J.
L
oote
ns,
D.
V.
Poel,
and
M.
Balli
ngs,
“
Eva
lua
ti
ng
Multi
-
la
be
l
class
ifi
ers
and
rec
o
m
m
ende
r
sy
ste
m
s
in
the
fin
anc
i
al
servic
e
se
ct
or,”
European
J.
of
Operational
Re
s
earc
h,
vol
.
279,
no.
2,
pp.
620
-
634,
2019
,
doi:
10.
1016/j.e
jor
.
2
019.
05.
037
.
[7]
Y.
Zha
ng
,
Y
.
W
ang,
X.
-
Y.
Liu,
S.
Mi
,
and
M
.
-
L.
Zh
ang,
“
Large
-
sca
l
e
m
ulti
-
l
abe
l
class
ifi
c
at
io
n
using
unknow
n
strea
m
ing
imag
e
s,”
Pa
ttern R
e
co
gnit
ion,
vol
.
99
,
2020
,
doi
:
10
.
10
16/j
.
p
at
cog
.
2019
.
107100
.
[8]
L.
A.
C.
-
Diego
,
N.
Bessis,
and
I.
Korkontze
los,
“
Cla
ss
if
y
ing
emotions
in
stac
k
over
flow
and
JIRA
using
a
m
ult
i
-
la
be
l
appr
o
ac
h
,
”
Knowle
dge
-
Base
d
Syste
ms
,
vol. 1
95,
2020
,
doi
:
10.
1016/j.knos
y
s.2
020.
105633
.
[9]
S
.
Khan
and
A.
R.
Bai
g
,
“
Ant
co
lon
y
op
ti
m
izati
o
n
base
d
hi
era
rch
ic
a
l
m
ult
i
-
l
abel
cl
assifi
ca
t
ion
a
lg
orit
hm
,
”
App
lied
Soft
Computing
,
vol.
55
,
pp
.
462
-
479,
2017
,
doi
:
1
0.
1016/j.a
soc
.
20
17.
02.
021
.
[10]
R.
B.
Per
ei
ra
,
A.
Plasti
no,
B
.
Z
ad
roz
n
y
,
and
L
.
H.
C.
Merschm
ann
,
“
Corre
lation
an
aly
s
is
of
per
for
m
anc
e
m
ea
sures
for
m
ult
i
-
l
abe
l
c
la
ss
ifi
c
at
ion
,
”
In
formation
Proces
sing
and
Manage
ment,
vol
.
54,
no.
3
,
pp.
359
-
369,
2018
,
doi
:
10.
1016/j.ipm.2
018.
01.
002
.
[11]
T.
Gong,
B.
Li
u,
Q.
Chu,
and
N.
Yu,
“Using
m
ult
i
-
la
bel
cl
assifi
cation
to
improve
o
bje
c
t
det
e
ct
ion
,
”
Neurocomputi
ng
,
vol
.
370
,
pp
.
17
4
-
185,
2019
,
doi
:
10.
1016
/j.ne
uc
om
.
2019.
08.
089
.
[12]
V.
Kum
ar,
A.
K
.
Pujar
i,
V.
Padm
ana
bhan,
and
V.
R.
Kagit
a,
“
Group
pre
serving
la
bel
embeddi
ng
for
m
ult
i
-
la
b
el
cl
assifi
ca
t
ion,”
Pat
te
rn
Recogni
t
ion
,
vo
l. 90, pp.
23
-
34,
2019
,
doi
:
10.
1016
/j.pa
t
co
g.
2019.
01
.
009
.
[13]
A.
Akbarne
j
ad
a
nd
M.S.
Baghsh
ah,
“
A
Probabil
i
stic
m
ult
i
-
la
be
l
cl
assifi
er
with
m
issing
and
nois
y
l
abe
ls
h
andl
i
ng
ca
pab
il
i
t
y
,
”
Pa
ttern R
e
cogni
t
ion L
et
te
rs
,
vo
l. 89,
pp.
18
-
24
,
2017
,
doi:
10.
1016
/j
.
p
at
re
c.
2017
.
01.
02
2
.
[14]
H.
Cevi
k
al
p
,
B.
Benl
igi
r
a
y
,
and
O.
N.
Ger
ek,
“
Sem
i
-
supervise
d
r
obust
dee
p
neur
al
n
et
works
for
m
ult
i
-
la
b
el
image
cl
assifi
ca
t
ion,”
Pat
te
rn
Recogni
t
ion
,
vo
l. 100, 20
19
,
doi
:
10
.
1016
/j
.
p
at
cog
.
2019.
1
07164
.
[15]
F.
Gargi
ulo,
S.
Silve
stri,
M
.
Ciam
p
i,
and
G.
D.
Pict
ro,
“
Dee
p
ne
ura
l
net
work
for
hie
rar
chi
c
al
ex
t
reme
m
ult
i
-
la
b
e
l
te
xt
class
ifi
c
at
io
n,
”
Appl
i
ed
So
ft
Computing
,
vo
l.
79,
pp
.
125
-
138
,
2019
,
doi:
10.
10
16/j
.
asoc
.
2019
.
0
3.
041
.
[16]
J.
Cao,
S.
W
ang
,
R.
W
ang
,
X.
Zh
ang,
and
S.
Kw
ong,
“
Conte
nt
-
or
i
ent
ed
im
age
qu
a
li
t
y
assess
m
ent
with
m
ult
i
-
l
abel
SV
M
cl
assifier,”
S
ignal
Pr
oce
ss
ing:
Ima
ge
Comm
unication
,
vol.
7
8,
pp.
388
-
3
97,
2019
,
do
i:
10.
1016/j.image
.
2019.
07.
018
.
[17]
B.
Buddhaha
i
,
W
.
W
ongsere
e,
and
P.
Rakkwa
m
suk,
“A
non
-
int
rusive
loa
d
m
onit
oring
s
y
st
e
m
using
m
ult
i
-
l
abe
l
cl
assifi
ca
t
ion
appr
oac
h
,
”
Su
stainabl
e
Cit
i
es
and
Soci
et
y
,
vol.
39,
pp.
621
-
630,
2018
,
doi
:
10.
1016/j.scs.20
18.
02.
002
.
[18]
R.
Sous
a
and
J.
Gam
a,
“
Multi
-
l
abe
l
class
ifi
cati
on
from
high
-
spee
d
d
at
a
strea
m
s
with
ada
p
ti
ve
m
odel
rule
s
and
ran
dom
rule
s,
”
P
rogr
ess i
n
Arti
f
i
ci
al
Intelli
g
ent
,
vol.
7
,
pp
.
177
-
1
87,
2018
,
doi
:
10
.
1007
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
2
4
, N
o.
1
,
Oct
o
ber
20
21
:
48
4
-
490
490
[19]
R.
Cerr
i,
M.
P.
Basgal
upp,
R
.
C.
Barr
os,
an
d
A.
C.
P.
L.
F.
Carva
lho,
“
I
nduci
ng
Hier
ar
c
h
ic
a
l
Multi
-
l
abel
cl
assifi
ca
t
ion
ru
le
s
with
G
ene
t
ic
Algor
it
hm
s,”
Appl
i
ed
So
ft
Computing
,
vo
l
.
77,
pp.
584
-
604,
2019
,
doi
:
10.
1016/j.a
soc
.
2
019.
01.
017
.
[20]
M.
R.
Boutell,
J.
Luo,
X.
Shen
,
a
nd
C.
M.
Brown,
“
Le
arn
ing
m
ulti
-
la
b
el
sce
n
e
c
las
sific
at
ion
,
”
Pat
t
ern
Re
cog
n
it
ion
,
vol.
37
,
no
.
9
,
pp
.
1757
-
1771
,
20
04
,
doi
:
10
.
1016
/j
.
p
at
cog
.
2004.
0
3.
009
.
[21]
G.
Tsoum
aka
s
and
K.
Io
ann
is,
“
Multi
-
la
b
el
cl
assifi
ca
t
ion:
An
over
vie
w,
”
Inte
rnat
ional
Journal
of
D
ata
Warehousing
an
d
Mini
ng
(
IJDWM)
,
vol.
3
,
no
.
3
,
pp.
1
-
13,
2007
,
doi:
10.
4018/
jdwm
.
2007070101
.
[22]
J.
Rea
d,
B
.
Pfahringe
r,
G.
Hol
m
es,
and
E.
Frank,
“
Cla
ss
ifi
er
cha
ins
for
m
ult
i
-
la
b
el
c
la
ss
ific
at
ion
,
”
Mac
h
in
e
le
arning
,
vol
.
85
,
pp
.
333
-
359
,
2
011
,
doi
:
10
.
100
7/s10994
-
011
-
5256
-
5
.
[23]
J.
Rea
d
,
B.
Pfah
ringe
r,
G.
Holm
es,
and
E.
Frank
,
“
Cl
assifie
r
chai
ns
for
m
ult
i
-
la
b
e
l
class
ifi
c
at
ion
,
”
Joi
nt
European
Confe
renc
e
on
Mac
hine
Learni
ng
and
Knowle
d
ge
Discov
ery
in
Databases
,
Spri
nger
,
vo
l.
5782,
pp.
254
-
269,
20
09
,
doi:
10
.
1007/97
8
-
3
-
642
-
04174
-
7_17
.
[24]
K.
Trohi
dis,
G.
Tsoum
aka
s,
G
.
Kall
ir
is,
and
I.
V
la
hav
as,
“
Multi
-
la
b
el
c
la
s
sific
a
ti
on
of
m
usic
b
y
emotio
n,
”
EURA
SIP
Journ
al
on
Audi
o,
Sp
ee
ch
,
and
Music
Proce
ss
ing
,
vol
.
1,
pp.
1
-
9,
201
1
,
doi:
10.
1186/
1687
-
4722
-
2011
-
426793
.
[25]
A.
Pere
z
,
R.
Bas
agoi
ti,
R.
A
.
Cor
te
z
,
F.
L
arr
ina
g
a
,
E.
B
arr
asa
,
and
A.
Urru
ti
a
,
“
A
ca
se
stud
y
on
the
use
of
m
ac
hine
le
arn
ing
te
chn
iq
ues
for
supporting
te
chnol
og
y
watc
h,
”
Data
&
Knowle
dge
En
gine
ering
,
vol.
117,
pp.
239
-
251,
2018
,
doi
:
10
.
10
16/j
.
d
at
ak
.
2018.
08.
001
.
[26]
D.
Kim
,
D.
Seo
,
S.
Cho,
and
P
.
Kang,
“
Multi
-
co
-
training
for
document
class
ifi
cation
using
v
ari
ous
document
rep
rese
nt
at
ions:
TF
-
IDF
,
LDA,
and
Doc2
Vec
,
”
In
formation
Sci
en
ce
s
,
vol.
477,
pp
.
15
-
29,
2019
,
doi:
10.
1016/j.ins.
20
18.
10.
006
[27]
M.
Ta
n,
J.
Zhou
,
Z.
Peng,
J.
Yu,
and
F.
Ta
ng,
“
Fine
-
gra
in
ed
image
class
ifi
c
at
io
n
with
fac
tori
zed
dee
p
user
cl
i
c
k
fea
tur
e,”
In
formation Proce
ss
ing
&
Manage
ment
,
vol. 57,
no.
3,
2
020
,
doi
:
10.
101
6/j
.
ipm.2019
.
10
2186Get
.
[28]
N.
K.
Mishra
a
n
d
P.
K.
Singh,
“
Feat
ure
construction
and
sm
ote
-
base
d
imbal
anc
e
handl
ing
for
m
ult
i
-
la
b
el
learni
ng,
”
Information
Sc
ienc
es
,
vo
l. 563, p
p.
342
-
357
,
202
1
,
doi
:
10
.
1016/j.ins.
2021.
03
.
001
.
[29]
K.
Sechi
d
is,
G.
Tsoum
aka
s,
and
I.
Vlah
ava
s,
“
On
the
stratificat
ion
of
m
ult
i
-
l
abel
d
at
a
,
”
Joi
n
t
Euro
pean
Conf
ere
nce
on
Mac
hine
Lea
rning
and
Knowle
dge
Discov
ery
in
Databases
,
2011,
pp.
145
-
15
8
,
doi:
10.
1007
/
978
-
3
-
642
-
23808
-
6_10
.
[30]
A.
A.
Gonza
l
ez,
J.
F.
D.
-
Pastor,
J.
J.
Rodriguez,
and
C.
G.
Os
orio,
“
Stud
y
of
da
ta
tr
ansform
at
io
n
te
chn
ique
s
for
ada
pt
ing
single
-
l
abe
l
proto
t
y
pe
s
el
e
ct
ion
al
gori
th
m
s
to
m
ult
i
-
la
b
e
l
le
arn
ing,”
Ex
p
e
rt
Syste
ms
wit
h
Appl
ic
a
t
ions
,
vo
l.
109,
pp
.
114
-
13
0,
2018
,
doi
:
10
.
1016/j
.
eswa.
201
8.
05.
017
.
[31]
G.
Nan,
Q
.
L
i,
R.
Don,
and
J
.
Li
u
,
“
Local
po
siti
ve
and
n
ega
t
ive
cor
relati
on
-
base
d
k
-
l
abe
lse
t
s
for
m
ult
i
-
l
abel
cl
assifi
ca
t
ion,”
Neurocomputi
ng
,
vol
.
318
,
pp
.
90
-
101,
2018
,
doi
:
10.
1016/j.ne
u
co
m
.
2018.
08.
035
.
[32]
S.
Vlu
y
m
ans,
C.
Cornelis,
F.
He
rre
ra
,
and
Y.
Sa
e
y
s
,
“
Multi
-
l
abel
c
la
ss
ifi
c
at
ion
u
sing
a
fu
zzy
rou
gh
nei
ghborhoo
d
conse
nsus
,
”
In
fo
rm
ati
on
Scienc
e
s
,
vol. 433
-
434
,
pp.
96
-
114
,
201
8
,
doi
:
10
.
1016/j.ins.
2017.
12
.
034
.
[33]
Z.
Zha
ng
and
C.
Sun,
“
Multi
-
site
struc
tural
d
amage
id
ent
if
ica
ti
on
using
a
m
ult
i
-
l
abel
c
la
ss
ifica
t
ion
sche
m
e
o
f
m
ac
hine
learni
n
g,
”
M
easuremen
t
,
vo
l. 154, 2020
,
doi:
10.
1016
/j
.
m
ea
surem
ent
.
20
20.
107473
.
[34]
A.
La
w
and
A.
Ghos
h,
“
Multi
-
la
bel
cl
assifi
cati
on
using
a
ca
sc
ade
of
sta
cke
d
a
utoe
nco
d
er
and
ext
reme
l
ea
rn
in
g
m
ac
hine
s,”
Neur
ocomputi
ng
,
vol
.
358,
pp.
222
-
23
4,
2019
,
doi
:
10
.
1016/j
.
n
euc
om
.
2
019.
05.
051
.
Evaluation Warning : The document was created with Spire.PDF for Python.