I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
14
,
No
.
5
,
Octo
b
er
20
25
,
p
p
.
3
9
8
2
~
3
9
9
3
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
14
.i
5
.
p
p
3
9
8
2
-
3
9
9
3
3982
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
A hybrid mo
del
f
o
r handli
ng
t
he
i
mba
la
nced multi
c
la
ss
cla
ss
ificatio
n pro
blem
E
s
ra
'
a
Als
h
da
if
a
t
1
,
F
a
iro
uz
H
us
s
ei
n
2
,
Ala
'
a
Al
-
s
hd
a
if
a
t
1
,
M
a
la
k
Al
-
H
a
s
s
a
n
3
,
E
ns
hira
h Alt
a
ra
wneh
4
1
D
e
p
a
r
t
m
e
n
t
o
f
I
n
f
o
r
m
a
t
i
o
n
T
e
c
h
n
o
l
o
g
y
,
F
a
c
u
l
t
y
o
f
P
r
i
n
c
e
A
l
-
H
u
s
s
e
i
n
B
i
n
A
b
d
a
l
l
a
h
I
I
f
o
r
I
n
f
o
r
m
a
t
i
o
n
T
e
c
h
n
o
l
o
g
y
,
T
h
e
H
a
s
h
e
m
i
t
e
U
n
i
v
e
r
s
i
t
y
,
Z
a
r
q
a
,
J
o
r
d
a
n
2
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
I
n
f
o
r
m
a
t
i
o
n
S
y
st
e
m
s
,
F
a
c
u
l
t
y
o
f
P
r
i
n
c
e
A
l
-
H
u
ss
e
i
n
B
i
n
A
b
d
a
l
l
a
h
I
I
f
o
r
I
n
f
o
r
mat
i
o
n
T
e
c
h
n
o
l
o
g
y
,
Th
e
H
a
s
h
e
m
i
t
e
U
n
i
v
e
r
si
t
y
,
Z
a
r
q
a
,
J
o
r
d
a
n
3
K
i
n
g
A
b
d
u
l
l
a
h
I
I
S
c
h
o
o
l
o
f
I
n
f
o
r
ma
t
i
o
n
Te
c
h
n
o
l
o
g
y
,
T
h
e
U
n
i
v
e
r
si
t
y
o
f
J
o
r
d
a
n
,
A
m
ma
n
,
J
o
r
d
a
n
4
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
En
g
i
n
e
e
r
i
n
g
,
F
a
c
u
l
t
y
o
f
En
g
i
n
e
e
r
i
n
g
,
Th
e
H
a
s
h
e
mi
t
e
U
n
i
v
e
r
si
t
y
,
Z
a
r
q
a
,
J
o
r
d
a
n
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Oct
31
,
2
0
2
4
R
ev
is
ed
J
u
l
5
,
2
0
2
5
Acc
ep
ted
Au
g
6
,
2
0
2
5
Da
ta
in
m
a
n
y
a
p
p
li
c
a
ti
o
n
d
o
m
a
in
s
is
imb
a
lan
c
e
d
.
I
n
m
a
c
h
in
e
lea
rn
in
g
,
a
d
d
re
ss
in
g
imb
a
lan
c
e
d
d
a
ta
is
c
r
u
c
ial
to
p
re
v
e
n
t
b
ias
to
wa
rd
s
th
e
d
o
m
in
a
n
t
c
las
s
lab
e
l
a
n
d
e
n
s
u
re
th
a
t
p
re
d
ictio
n
m
o
d
e
ls
c
a
n
lea
rn
a
n
d
p
re
d
ict
th
e
m
in
o
rit
y
c
las
s
p
ro
ficie
n
tl
y
.
Th
is
p
a
p
e
r
p
r
o
p
o
se
s
a
h
y
b
ri
d
i
m
b
a
lan
c
e
d
c
las
sifica
ti
o
n
m
o
d
e
l
(HICD
)
t
o
a
d
d
re
ss
th
e
m
u
l
ti
c
las
s
imb
a
lan
c
e
d
d
a
ta
p
ro
b
lem
.
Th
e
p
rima
ry
i
d
e
a
is
to
c
o
m
b
in
e
e
ffe
c
ti
v
e
m
e
th
o
d
s
t
o
c
o
n
stru
c
t
a
c
las
sifica
ti
o
n
m
o
d
e
l
t
h
a
t
c
a
n
h
a
n
d
le
m
u
l
ti
c
las
s
imb
a
lan
c
e
d
d
a
ta
e
ffe
c
ti
v
e
ly
.
F
o
u
r
m
e
th
o
d
s
a
re
e
m
p
lo
y
e
d
:
a
n
o
v
e
rsa
m
p
li
n
g
m
e
th
o
d
t
o
b
a
lan
c
e
th
e
d
a
ta,
a
d
e
c
o
m
p
o
siti
o
n
m
e
th
o
d
t
o
c
o
n
v
e
rt
th
e
m
u
lt
icla
ss
p
ro
b
lem
in
t
o
a
se
t
o
f
b
in
a
ry
p
ro
b
lem
s,
e
n
se
m
b
le
c
las
sifica
ti
o
n
to
in
teg
ra
te
b
a
se
c
las
sifiers
to
imp
ro
v
e
p
re
d
ictio
n
,
a
n
d
a
b
o
o
sti
n
g
m
e
th
o
d
t
o
e
n
c
o
u
ra
g
e
th
e
c
las
sifier
to
p
a
y
m
o
re
a
tt
e
n
ti
o
n
t
o
m
isc
las
sified
sa
m
p
les
.
To
e
v
a
lu
a
te
t
h
e
p
r
o
p
o
se
d
m
o
d
e
l
,
se
v
e
n
tee
n
imb
a
lan
c
e
d
d
a
tas
e
ts
fro
m
v
a
rio
u
s
a
p
p
li
c
a
ti
o
n
d
o
m
a
in
s,
fe
a
tu
rin
g
d
iffere
n
t
n
u
m
b
e
rs
o
f
c
las
se
s,
in
s
tan
c
e
s,
fe
a
tu
re
s
,
a
n
d
imb
a
lan
c
e
r
a
ti
o
s
,
a
re
as
se
ss
e
d
.
Th
e
e
x
p
e
rime
n
tal
r
e
su
lt
s
a
n
d
sta
ti
stica
l
sig
n
ifi
c
a
n
c
e
tes
ts
d
e
m
o
n
stra
te
th
a
t
th
e
p
r
o
p
o
se
d
h
y
b
ri
d
m
o
d
e
l
sig
n
ifi
c
a
n
tl
y
o
u
t
p
e
rfo
rm
s
th
e
sta
n
d
a
rd
o
n
e
-
vs
-
o
n
e
(OV
O)
a
p
p
r
o
a
c
h
a
n
d
th
e
OV
O
c
o
m
b
i
n
e
d
wi
t
h
o
v
e
rsa
m
p
li
n
g
tec
h
n
i
q
u
e
(
S
M
O
TE
)
,
b
o
t
h
c
o
n
si
d
e
re
d
sta
te
-
of
-
t
h
e
-
a
rt
fo
r
a
d
d
re
ss
in
g
imb
a
lan
c
e
d
m
u
lt
icla
ss
d
a
tas
e
ts,
in
term
s o
f
F
1
-
sc
o
re
.
K
ey
w
o
r
d
s
:
B
o
o
s
tin
g
B
o
o
ts
tr
ap
I
m
b
alan
ce
d
d
ata
Mu
lticlas
s
cla
s
s
if
icatio
n
O
ne
-
vs
-
one
SMOT
E
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
E
s
r
a
'
a
Al
s
h
d
aif
at
Dep
ar
tm
en
t o
f
I
n
f
o
r
m
atio
n
T
e
ch
n
o
lo
g
y
Facu
lty
o
f
Prin
ce
Al
-
Hu
s
s
ein
B
in
Ab
d
allah
I
I
f
o
r
I
n
f
o
r
m
atio
n
T
ec
h
n
o
lo
g
y
,
T
h
e
Hash
em
ite
Un
iv
er
s
ity
P.O.
B
o
x
3
3
0
1
2
7
,
Z
ar
q
a
1
3
1
3
3
,
J
o
r
d
an
E
m
ail: e
s
r
aa
@
h
u
.
ed
u
.
jo
1.
I
NT
RO
D
UCT
I
O
N
I
n
s
ev
er
al
r
ea
l
-
wo
r
ld
p
r
o
b
lem
s
,
s
u
ch
as
d
is
ea
s
e
id
en
tific
ati
o
n
,
tex
t
class
if
icatio
n
,
n
etwo
r
k
in
tr
u
s
io
n
d
etec
tio
n
,
an
d
s
p
am
f
ilter
in
g
,
im
b
alan
ce
d
d
ata
is
co
m
m
o
n
.
W
h
er
e
th
e
f
r
e
q
u
en
c
y
o
f
class
lab
els
in
th
e
d
ataset
is
u
n
eq
u
al,
in
o
th
er
wo
r
d
s
,
o
n
e
o
r
m
o
r
e
class
es
ar
e
u
n
d
er
r
ep
r
esen
ted
,
in
co
n
t
r
ast,
th
e
r
em
ain
in
g
class
es
ar
e
h
ig
h
ly
r
ep
r
esen
ted
in
th
e
d
ataset.
T
h
e
clas
s
r
ep
r
esen
ted
b
y
a
s
ig
n
if
ican
tly
lar
g
er
n
u
m
b
er
o
f
o
b
s
er
v
atio
n
s
r
elativ
e
to
o
th
er
class
es,
in
th
e
d
ataset,
is
r
ef
er
r
e
d
to
as
th
e
“
m
ajo
r
ity
class
”.
W
h
ile
th
e
class
th
at
is
r
ep
r
esen
ted
b
y
a
n
o
ticea
b
ly
s
m
aller
n
u
m
b
er
o
f
o
b
s
er
v
atio
n
s
r
elativ
e
to
o
th
er
class
es
is
r
ef
er
r
ed
to
as
th
e
“m
in
o
r
ity
class
”.
T
wo
m
ai
n
i
m
b
alan
ce
p
r
o
b
lem
s
ca
n
b
e
id
en
tifie
d
:
b
in
a
r
y
im
b
alan
ce
d
p
r
o
b
lem
,
wh
er
e
th
e
d
ataset
co
n
tain
s
o
n
ly
two
cl
ass
es
(
th
e
m
ajo
r
ity
an
d
m
in
o
r
ity
class
)
,
an
d
m
u
lticlas
s
im
b
alan
ce
d
p
r
o
b
lem
,
wh
ich
in
clu
d
es m
o
r
e
th
an
two
class
es,
with
o
n
e
o
r
m
o
r
e
class
es r
ep
r
esen
ted
b
y
f
ewe
r
in
s
tan
ce
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
h
a
n
d
lin
g
t
h
e
imb
a
la
n
ce
d
mu
lticla
s
s
cla
s
s
ifica
tio
n
p
r
o
b
lem
(
E
s
r
a
’
a
A
ls
h
d
a
ifa
t)
3983
Usi
n
g
th
e
s
tan
d
a
r
d
m
ac
h
in
e
l
ea
r
n
in
g
alg
o
r
ith
m
s
as
th
e
y
a
r
e
on
an
im
b
ala
n
ce
d
d
ataset
w
ill
r
esu
lt
in
m
ajo
r
ity
class
lab
el
b
ias,
an
d
th
e
ac
cu
r
ac
y
o
f
th
e
p
r
o
d
u
ce
d
m
o
d
el
will
n
o
t
b
e
r
ep
r
esen
tativ
e
o
f
its
ac
tu
al
u
s
ef
u
ln
ess
.
T
o
m
ak
e
th
is
p
r
o
b
lem
clea
r
,
im
ag
in
e
a
m
ed
ical
d
iag
n
o
s
is
d
ata
s
et
h
av
in
g
two
class
es
:
i)
m
ajo
r
ity
class
(
n
eg
ativ
e)
,
wh
ich
f
o
r
m
s
9
5
%
o
f
s
am
p
les
,
a
n
d
ii)
m
i
n
o
r
ity
(
p
o
s
itiv
e)
class
,
wh
ich
f
o
r
m
s
5
%
o
f
s
am
p
les.
C
r
ea
tin
g
a
class
if
icatio
n
m
o
d
el
th
at
co
n
s
tan
tly
o
u
tp
u
ts
t
h
e
m
ajo
r
ity
class
,
g
i
v
es
an
ac
cu
r
ac
y
r
ate
o
f
9
5
%.
I
n
th
is
s
ce
n
ar
io
,
th
e
s
am
p
les
o
f
th
e
m
in
o
r
ity
class
wer
e
n
e
g
lecte
d
b
y
th
e
class
if
icatio
n
alg
o
r
ith
m
,
an
d
th
e
o
b
tain
ed
ac
cu
r
ac
y
s
co
r
e
is
co
n
s
id
er
ed
m
is
lead
in
g
.
No
te
h
er
e
th
at
g
r
ea
ter
im
p
o
r
tan
ce
is
o
f
ten
g
iv
en
to
th
e
u
n
d
er
r
ep
r
esen
ted
class
.
Fo
r
i
n
s
tan
ce
,
in
th
e
p
r
e
v
io
u
s
m
e
d
i
ca
l
d
iag
n
o
s
is
p
r
o
b
lem
,
th
e
m
in
o
r
ity
class
is
th
e
“p
o
s
itiv
e”
s
am
p
les,
wh
ich
ar
e
r
ar
e
b
u
t
ess
en
tial
to
b
e
d
etec
t
ed
p
r
ec
is
ely
.
T
h
e
s
am
e
is
s
u
e
o
cc
u
r
s
in
m
u
lticlas
s
im
b
alan
ce
d
p
r
o
b
lem
s
;
h
o
wev
er
,
it
is
m
o
r
e
ch
allen
g
in
g
.
C
o
n
s
id
er
in
g
a
h
ea
r
t
d
is
ea
s
e
d
ata
s
et,
wh
er
e
p
atien
ts
ar
e
ca
teg
o
r
ized
in
to
f
iv
e
class
es
b
ased
o
n
th
e
s
ev
er
ity
o
f
h
e
ar
t
d
is
ea
s
e,
wh
ich
r
an
g
e
f
r
o
m
class
0
(
n
o
d
is
ea
s
e)
to
class
es
o
n
e
t
o
f
o
u
r
(
s
ev
er
e
d
is
ea
s
es).
T
h
e
n
o
d
is
ea
s
e
a
n
d
n
o
n
-
s
ev
er
e
d
is
ea
s
e
class
e
s
ar
e
th
e
d
o
m
in
an
t
class
es,
wh
er
ea
s
clas
s
es
th
at
r
ep
r
esen
t
m
o
r
e
s
ev
er
e
ca
s
es
ar
e
r
ep
r
esen
ted
b
y
f
ewe
r
s
a
m
p
les.
T
r
ain
in
g
a
class
if
ier
o
n
th
is
d
ataset
will
b
e
ef
f
ec
tiv
e
in
p
r
e
d
ictin
g
n
o
o
r
m
ild
d
is
ea
s
e
class
es,
b
u
t
it
m
ig
h
t
n
o
t
b
e
ab
le
to
d
etec
t p
atien
ts
b
elo
n
g
i
n
g
to
m
o
r
e
s
ev
er
e
class
es.
Fro
m
th
e
f
o
r
e
g
o
in
g
,
h
an
d
lin
g
im
b
alan
ce
d
d
atasets
is
co
n
s
id
er
ed
a
c
h
allen
g
in
g
an
d
w
ell
-
k
n
o
wn
p
r
o
b
lem
in
th
e
m
ac
h
in
e
lear
n
i
n
g
f
ield
.
C
o
n
s
eq
u
en
tly
,
m
u
ch
r
esear
ch
wo
r
k
h
as
b
ee
n
co
n
d
u
cted
b
y
n
u
m
e
r
o
u
s
r
esear
ch
er
s
to
tack
le
th
is
p
r
o
b
lem
.
T
h
e
wo
r
k
in
a
d
d
r
ess
in
g
th
e
im
b
alan
ce
d
d
ata
p
r
o
b
lem
ca
n
b
e
ca
teg
o
r
ize
d
in
to
th
r
ee
m
ain
ca
teg
o
r
ies
[
1
]
:
i)
d
ata
-
lev
el
m
eth
o
d
s
,
ii)
al
g
o
r
ith
m
ic
-
lev
el
m
eth
o
d
s
,
an
d
ii
i)
h
y
b
r
id
m
eth
o
d
s
.
I
n
d
ata
-
lev
el
m
et
h
o
d
s
,
b
ala
n
c
in
g
th
e
d
ata
is
p
er
f
o
r
m
ed
b
y
au
g
m
en
tin
g
th
e
m
in
o
r
ity
cla
s
s
o
b
s
er
v
atio
n
s
o
r
r
ed
u
cin
g
th
e
m
aj
o
r
ity
class
o
b
s
er
v
atio
n
s
,
wh
ich
ar
e
k
n
o
w
n
as
o
v
er
-
s
am
p
lin
g
a
n
d
u
n
d
e
r
s
am
p
lin
g
m
eth
o
d
s
.
C
o
n
ce
r
n
in
g
t
h
e
alg
o
r
ith
m
ic
-
le
v
el
m
eth
o
d
s
,
s
u
ch
m
eth
o
d
s
in
v
o
lv
e
m
o
d
if
y
i
n
g
e
x
is
tin
g
alg
o
r
ith
m
s
o
r
p
r
o
p
o
s
in
g
a
s
tr
u
ctu
r
e
f
o
r
n
ew
alg
o
r
ith
m
s
to
ad
d
r
ess
th
e
im
b
alan
ce
d
d
ata
p
r
o
b
lem
.
W
ith
r
esp
ec
t
to
h
y
b
r
id
m
eth
o
d
s
,
a
co
m
b
in
atio
n
o
f
d
ata
-
lev
el
an
d
alg
o
r
ith
m
ic
-
lev
el
m
et
h
o
d
s
is
e
m
p
lo
y
ed
to
h
a
n
d
le
th
e
im
b
alan
ce
d
d
ata.
T
h
e
s
o
lu
tio
n
p
r
o
p
o
s
ed
in
th
is
p
ap
er
f
o
r
h
an
d
lin
g
t
h
e
im
b
ala
n
ce
d
d
ata
p
r
o
b
lem
b
elo
n
g
s
to
th
e
h
y
b
r
i
d
m
eth
o
d
s
ca
teg
o
r
y
.
Mo
r
e
s
p
ec
if
ically
,
f
o
u
r
m
et
h
o
d
s
a
r
e
co
m
b
in
ed
to
tack
le
th
e
im
b
alan
ce
d
d
ata
a
n
d
o
b
tain
a
n
ef
f
ec
tiv
e
class
if
icatio
n
m
o
d
el.
T
h
e
f
ir
s
t
m
eth
o
d
is
a
d
ata
-
lev
el
m
eth
o
d
:
th
e
well
-
k
n
o
wn
s
y
n
th
etic
m
in
o
r
ity
o
v
er
s
am
p
lin
g
tech
n
i
q
u
e
(
SMOT
E
)
is
u
tili
ze
d
[
2
]
.
T
h
e
s
e
co
n
d
m
eth
o
d
is
a
n
e
n
s
em
b
le
m
eth
o
d
,
w
h
er
e
a
co
llectio
n
o
f
class
if
ier
s
is
u
tili
ze
d
to
en
h
an
ce
class
if
icatio
n
ef
f
ec
tiv
e
n
ess
.
T
h
e
t
h
ir
d
m
eth
o
d
is
a
d
ec
o
m
p
o
s
itio
n
m
eth
o
d
,
i
n
wh
ich
a
m
u
lticlas
s
clas
s
if
icatio
n
p
r
o
b
lem
is
d
ec
o
m
p
o
s
ed
in
to
a
n
u
m
b
er
o
f
b
i
n
ar
y
s
u
b
-
p
r
o
b
lem
s
,
an
d
ea
c
h
class
if
ier
f
o
cu
s
es
o
n
l
y
o
n
two
class
es;
th
u
s
,
b
etter
class
if
icatio
n
ef
f
ec
tiv
en
ess
ca
n
b
e
o
b
tain
ed
.
T
h
e
f
o
u
r
th
m
eth
o
d
i
s
a
b
o
o
s
tin
g
m
eth
o
d
,
wh
ich
i
d
en
tifie
s
th
e
lo
w
-
p
e
r
f
o
r
m
an
ce
b
ase
class
if
ier
s
an
d
f
o
r
ce
s
th
em
to
f
o
c
u
s
o
n
m
is
class
if
ied
in
s
tan
ce
s
u
s
in
g
a
b
o
o
ts
tr
ap
tech
n
iq
u
e.
T
h
e
id
ea
is
t
h
at
in
teg
r
atin
g
f
o
u
r
ef
f
ec
tiv
e
m
eth
o
d
s
f
o
r
h
a
n
d
lin
g
im
b
alan
ce
d
m
u
lticlas
s
class
if
icatio
n
ca
n
r
esu
lt
in
a
h
ig
h
-
p
er
f
o
r
m
an
ce
h
y
b
r
i
d
m
o
d
el.
Fu
r
th
e
r
in
f
o
r
m
atio
n
ab
o
u
t th
e
p
r
o
p
o
s
ed
m
o
d
el
is
p
r
o
v
id
ed
in
s
ec
tio
n
3
.
T
h
e
r
est
o
f
th
is
p
ap
er
is
s
tr
u
ct
u
r
ed
in
th
e
f
o
llo
win
g
s
ec
tio
n
s
:
s
ec
tio
n
2
p
r
o
v
i
d
es
an
o
v
e
r
v
i
ew
o
f
th
e
m
eth
o
d
s
u
s
ed
to
h
an
d
le
im
b
alan
ce
d
d
atasets
.
Sectio
n
3
e
x
p
lain
s
th
e
g
en
er
atio
n
a
n
d
u
s
e
o
f
th
e
s
u
g
g
ested
h
y
b
r
id
im
b
ala
n
ce
d
m
u
lticlas
s
class
if
icatio
n
m
o
d
el.
Sectio
n
4
p
r
esen
ts
a
g
e
n
er
al
d
escr
ip
tio
n
o
f
th
e
ev
alu
atio
n
d
atasets
.
Sectio
n
5
co
v
er
s
th
e
ex
p
er
im
en
tal
s
etu
p
an
d
r
ep
o
r
ts
th
e
p
r
o
d
u
ce
d
r
esu
lts
.
Sectio
n
6
s
u
m
m
ar
izes
th
e
p
ap
er
an
d
p
r
o
v
id
es so
m
e
d
ir
ec
tio
n
s
f
o
r
f
u
tu
r
e
wo
r
k
.
2.
L
I
T
E
R
AT
U
RE
R
E
VI
E
W
I
n
th
is
s
ec
tio
n
,
an
o
v
er
v
iew
o
f
th
e
m
eth
o
d
s
u
s
ed
to
h
an
d
l
e
im
b
alan
ce
d
d
atasets
is
p
r
es
en
ted
.
As
m
en
tio
n
ed
ea
r
lier
,
i
m
b
alan
ce
d
d
atasets
ca
n
b
e
h
an
d
led
u
s
in
g
th
r
ee
p
r
im
ar
y
m
eth
o
d
s
[
1
]
:
i
)
d
ata
-
lev
el
m
eth
o
d
s
,
ii)
a
lg
o
r
ith
m
ic
-
le
v
el
m
eth
o
d
s
,
an
d
iii)
h
y
b
r
id
m
et
h
o
d
s
.
C
o
m
m
en
cin
g
with
th
e
d
ata
-
lev
el
m
eth
o
d
s
,
wh
ich
ar
e
u
s
ed
to
b
alan
ce
th
e
d
ata
d
u
r
in
g
th
e
p
r
ep
r
o
ce
s
s
in
g
p
h
ase.
T
h
ese
m
eth
o
d
s
ca
n
b
e
d
iv
id
e
d
in
to
two
g
r
o
u
p
s
:
o
v
er
s
am
p
lin
g
an
d
u
n
d
er
s
am
p
lin
g
m
eth
o
d
s
.
I
n
o
v
e
r
s
am
p
lin
g
,
th
e
class
im
b
alan
ce
is
ad
d
r
ess
ed
b
y
in
cr
ea
s
in
g
th
e
n
u
m
b
er
o
f
m
in
o
r
ity
class
s
am
p
les.
T
h
is
ca
n
b
e
ac
h
iev
ed
b
y
eith
er
d
u
p
licatin
g
ex
is
tin
g
m
in
o
r
ity
class
in
s
tan
ce
s
r
an
d
o
m
ly
o
r
b
y
g
en
e
r
atin
g
n
ew
s
y
n
th
etic
s
a
m
p
les.
T
h
e
f
ir
s
t
ap
p
r
o
ac
h
in
v
o
lv
es
r
ep
ea
tin
g
s
o
m
e
in
s
tan
ce
s
,
wh
ich
is
s
tr
aig
h
tf
o
r
war
d
b
u
t
m
a
y
ca
u
s
e
o
v
er
f
itti
n
g
.
T
h
e
s
ec
o
n
d
ap
p
r
o
ac
h
a
p
p
lies
in
ter
p
o
latio
n
b
etwe
en
m
in
o
r
ity
class
o
b
s
er
v
atio
n
s
to
g
en
er
ate
n
ew
o
b
s
er
v
atio
n
s
,
s
u
ch
as
u
s
in
g
th
e
SM
OT
E
[
2
]
,
th
e
r
esu
lt
h
er
e
is
m
o
r
e
d
iv
er
s
e
s
y
n
th
etic
s
am
p
les
[
3
]
.
SMOT
E
is
co
n
s
i
d
er
ed
th
e
m
o
s
t w
id
ely
u
s
ed
o
v
er
s
am
p
lin
g
m
eth
o
d
an
d
h
as b
r
o
a
d
ap
p
licatio
n
s
[
4
]
.
M
an
y
r
esear
ch
er
s
ap
p
lied
it to
im
b
alan
ce
d
d
ata
p
r
o
b
lem
s
an
d
r
ep
o
r
ted
th
at
th
e
m
o
d
el
p
e
r
f
o
r
m
an
ce
im
p
r
o
v
e
d
s
ig
n
if
ican
tly
[
5
]
,
[
6
]
.
On
t
h
e
o
th
e
r
h
a
n
d
,
s
o
m
e
r
esear
c
h
er
s
ar
g
u
e
th
at
t
h
e
r
esu
ltin
g
s
y
n
th
etic
s
am
p
les
m
ay
n
o
t
ac
cu
r
ately
r
ef
lect
th
e
o
r
ig
in
al
d
ata,
an
d
t
h
ey
r
ef
er
r
ed
to
th
e
n
ew
s
am
p
les
as
“u
n
r
ea
lis
tic
s
am
p
les”,
ar
g
u
i
n
g
th
at
th
is
ca
n
d
e
g
r
ad
e
class
if
ier
ac
cu
r
ac
y
[
7
]
.
Ad
ap
tiv
e
s
y
n
th
etic
(
ADASYN
)
s
am
p
lin
g
ap
p
r
o
ac
h
f
o
r
im
b
al
an
ce
d
lear
n
in
g
also
cr
ea
tes
s
y
n
th
etic
ex
am
p
les,
b
u
t
it
ad
o
p
ts
a
m
o
r
e
ad
ap
tiv
e
way
co
m
p
ar
e
d
to
tr
a
d
itio
n
al
S
MO
T
E
[
3
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2252
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
20
25
:
3
9
8
2
-
3
9
9
3
3984
R
eg
ar
d
in
g
th
e
u
n
d
e
r
s
am
p
lin
g
m
eth
o
d
s
,
s
am
p
les
ar
e
r
em
o
v
ed
f
r
o
m
t
h
e
m
ajo
r
ity
class
u
n
til
th
e
d
ataset
b
ec
o
m
es
b
alan
ce
d
.
T
h
is
is
d
o
n
e
to
av
o
id
b
ias
in
cla
s
s
if
icatio
n
m
o
d
els
to
war
d
th
e
m
ajo
r
ity
class
[
8
]
.
R
an
d
o
m
u
n
d
e
r
s
am
p
lin
g
(
R
US)
,
is
co
n
s
id
er
ed
o
n
e
o
f
th
e
s
im
p
lest
an
d
m
o
s
t
co
m
m
o
n
u
n
d
er
s
am
p
lin
g
m
eth
o
d
s
,
in
wh
ich
s
am
p
les
f
r
o
m
m
ajo
r
ity
class
e
s
ar
e
r
em
o
v
ed
r
an
d
o
m
ly
.
Ho
wev
er
,
th
is
lead
s
to
a
lo
s
s
o
f
v
alu
ab
le
in
f
o
r
m
atio
n
th
at
co
u
l
d
im
p
ac
t
th
e
p
e
r
f
o
r
m
an
ce
o
f
th
e
r
esu
ltin
g
m
o
d
el
[
9
]
.
C
o
n
s
eq
u
e
n
tly
,
o
th
e
r
m
eth
o
d
s
em
er
g
ed
an
d
attem
p
ted
to
r
e
m
o
v
e
s
am
p
les
f
r
o
m
th
e
m
ajo
r
ity
class
e
s
b
ased
o
n
s
o
m
e
d
ef
in
ed
cr
iter
ia
,
s
u
ch
as
th
e
r
ad
ial
-
b
ased
u
n
d
e
r
s
am
p
lin
g
alg
o
r
ith
m
[
1
0
]
.
W
ith
r
esp
ec
t
to
th
e
alg
o
r
ith
m
ic
-
lev
el
m
eth
o
d
s
,
wh
ich
ar
e
a
ls
o
k
n
o
wn
as
“in
ter
n
al
ap
p
r
o
a
ch
es”,
th
e
d
ata
im
b
alan
ce
p
r
o
b
lem
is
h
an
d
led
b
y
cr
ea
tin
g
o
r
im
p
r
o
v
in
g
ex
is
tin
g
class
if
icatio
n
alg
o
r
ith
m
s
[
4
]
.
T
h
ese
m
eth
o
d
s
in
clu
d
e
th
r
esh
o
ld
a
d
ju
s
tm
en
ts
,
o
n
e
-
class
lear
n
in
g
,
co
s
t
-
s
en
s
itiv
e
lear
n
in
g
,
a
n
d
en
s
em
b
le
-
b
ased
tech
n
iq
u
es
[
4
]
,
[
1
1
]
–
[
1
3
]
.
I
n
t
h
e
th
r
esh
o
ld
a
d
ju
s
tm
en
t m
eth
o
d
,
c
lass
if
ier
s
o
f
ten
p
r
o
v
id
e
p
r
o
b
ab
ilit
ies th
at
r
ef
er
to
wh
ich
class
an
o
b
s
er
v
atio
n
b
elo
n
g
s
,
wh
ich
ca
n
b
e
u
s
ed
t
o
ad
ju
s
t
th
r
esh
o
ld
s
an
d
r
ef
in
e
class
ass
ig
n
m
en
ts
[
1
1
]
.
C
o
s
t
-
s
en
s
itiv
e
lear
n
in
g
ass
ig
n
s
g
r
ea
ter
m
is
class
if
icati
o
n
co
s
ts
to
m
in
o
r
ity
class
s
am
p
les
to
en
co
u
r
ag
e
th
e
class
if
ier
to
p
ay
m
o
r
e
atte
n
tio
n
to
u
n
d
er
r
ep
r
esen
ted
s
a
m
p
les
[
4
]
.
On
e
class
class
if
icatio
n
f
o
cu
s
es
o
n
th
e
m
in
o
r
ity
class
an
d
lear
n
in
g
its
ch
ar
ac
ter
is
tics
to
d
if
f
er
e
n
tiate
it
f
r
o
m
th
e
o
th
er
d
ata
[
1
1
]
.
E
n
s
em
b
le
class
if
ier
s
aim
to
en
h
a
n
ce
th
e
p
er
f
o
r
m
an
ce
o
f
class
if
icatio
n
task
s
b
y
co
m
b
in
in
g
p
r
ed
ictio
n
s
f
r
o
m
a
s
et
o
f
b
ase
class
if
ier
s
[
1
4
]
.
C
o
m
m
o
n
e
n
s
em
b
le
m
et
h
o
d
s
in
clu
d
e
b
ag
g
i
n
g
an
d
b
o
o
s
tin
g
[
1
4
]
.
Usi
n
g
en
s
em
b
les
o
f
class
if
ier
s
h
as
b
ec
o
m
e
a
p
o
p
u
lar
m
eth
o
d
f
o
r
ad
d
r
ess
in
g
class
im
b
alan
ce
in
m
ac
h
in
e
lear
n
in
g
[
1
1
]
,
[
1
2
]
.
So
m
e
r
esear
ch
wo
r
k
s
f
o
cu
s
ed
o
n
s
im
p
lify
i
n
g
an
d
co
n
v
er
tin
g
th
e
s
in
g
le
m
u
lticlas
s
p
r
o
b
lem
in
to
m
an
y
b
in
a
r
y
p
r
o
b
lem
s
u
s
in
g
s
p
ec
if
ic
d
ec
o
m
p
o
s
itio
n
tech
n
i
q
u
es,
s
u
ch
as
o
n
e
-
vs
-
o
n
e
(
OV
O)
,
o
n
e
-
vs
-
all
(
OVA)
,
an
d
th
e
b
in
ar
y
tr
ee
m
eth
o
d
[
1
5
]
.
T
h
e
id
ea
h
er
e
is
to
f
o
cu
s
o
n
o
n
e
o
r
two
class
es
in
s
tea
d
o
f
cr
ea
tin
g
a
m
o
d
el
th
at
d
if
f
er
en
tiates
b
etwe
en
s
ev
er
al
class
es.
So
m
e
r
esear
ch
er
s
f
o
cu
s
ed
th
eir
r
esear
ch
o
n
co
m
b
in
i
n
g
d
ata
-
lev
el
m
eth
o
d
s
an
d
alg
o
r
it
h
m
ic
-
lev
el
m
eth
o
d
s
to
g
e
n
er
ate
m
o
r
e
p
o
wer
f
u
l
m
o
d
els
to
h
an
d
le
th
e
im
b
alan
ce
class
p
r
o
b
lem
,
th
ese
m
eth
o
d
s
ar
e
r
ef
er
r
ed
to
as
h
y
b
r
id
m
eth
o
d
s
[
1
6
]
.
I
t
is
im
p
o
r
tan
t
to
n
o
te
th
at
h
y
b
r
id
m
o
d
els
ca
n
b
e
d
if
f
er
en
tiated
ac
co
r
d
in
g
to
:
i)
th
e
ad
o
p
ted
d
ata
a
n
d
al
g
o
r
ith
m
m
et
h
o
d
s
,
an
d
ii)
wh
eth
er
th
e
a
d
d
r
ess
ed
class
if
icatio
n
p
r
o
b
lem
is
b
in
ar
y
o
r
m
u
lticlas
s
.
Mo
s
t
r
esear
ch
wo
r
k
r
elate
d
to
th
e
g
en
er
atio
n
o
f
h
y
b
r
id
im
b
alan
ce
d
m
o
d
els
h
as
b
ee
n
co
n
d
u
cted
o
n
b
i
n
ar
y
im
b
alan
ce
d
p
r
o
b
le
m
s
.
C
o
m
m
en
cin
g
with
th
e
b
i
n
ar
y
h
y
b
r
id
m
o
d
el
p
r
o
p
o
s
ed
b
y
Su
n
et
a
l
.
[
1
7
]
,
in
wh
ich
th
e
b
ag
g
in
g
en
s
em
b
le
m
eth
o
d
is
co
m
b
in
ed
with
SMOT
E
.
Sh
i
et
a
l
.
[
1
8
]
in
teg
r
ate
d
a
n
o
v
el
d
en
s
ity
-
b
ased
s
am
p
lin
g
tech
n
i
q
u
e
with
th
e
en
s
em
b
le
ap
p
r
o
ac
h
to
c
o
n
s
tr
u
ct
a
b
in
ar
y
h
y
b
r
id
im
b
al
an
ce
d
class
if
icatio
n
m
o
d
el
(
HI
C
D)
.
HI
C
D
p
ar
titi
o
n
s
th
e
d
ata
s
p
ac
e
in
t
o
f
iv
e
ar
e
as
ac
co
r
d
in
g
to
d
ata
d
en
s
ity
,
an
d
th
en
th
e
d
ata
is
s
am
p
led
f
r
o
m
th
ese
ar
ea
s
.
On
ce
th
e
d
ata
is
s
am
p
led
,
th
e
en
s
em
b
le
m
o
d
el
is
g
en
er
ated
.
W
h
ile
th
e
m
o
d
el
p
r
o
p
o
s
ed
b
y
T
h
ee
p
h
o
o
wian
g
an
d
Han
s
k
u
n
atai
[
1
9
]
s
p
lits
th
e
d
ata
i
n
to
f
o
u
r
d
if
f
er
en
t
g
r
o
u
p
s
ac
co
r
d
i
n
g
t
o
th
e
o
v
er
lap
p
i
n
g
an
d
n
o
n
-
o
v
er
lap
p
in
g
co
n
ce
p
t
b
etwe
en
th
e
m
ajo
r
ity
an
d
m
in
o
r
ity
class
es
in
s
tan
ce
s
,
th
e
d
ata
ca
teg
o
r
ies
ar
e
th
en
u
s
ed
t
o
f
o
r
m
f
iv
e
d
atasets
,
wh
ich
ar
e
r
e
s
am
p
led
u
s
in
g
d
if
f
e
r
en
t
SMO
T
E
s
.
T
h
e
s
am
p
led
d
atasets
ar
e
th
en
u
s
ed
to
g
en
er
ate
th
e
class
if
icatio
n
m
o
d
els
u
s
in
g
d
if
f
er
en
t
s
in
g
le
an
d
e
n
s
em
b
le
alg
o
r
ith
m
s
.
Sh
an
an
d
C
h
u
n
g
[
2
0
]
co
u
p
led
d
ata
-
lev
el
tech
n
iq
u
es
an
d
lo
s
s
f
u
n
ctio
n
t
o
g
en
e
r
ate
th
e
d
esire
d
h
y
b
r
id
m
o
d
el.
T
h
e
s
u
g
g
ested
m
o
d
el
b
e
g
in
s
with
d
iv
id
in
g
s
am
p
les
b
ased
o
n
th
eir
ef
f
ec
t
o
n
im
b
alan
ce
d
d
ata
class
if
icatio
n
in
to
s
ev
er
al
ca
teg
o
r
ies,
th
u
s
ap
p
r
o
p
r
iate
s
am
p
les
ca
n
b
e
s
elec
ted
f
o
r
s
am
p
lin
g
.
A
l
o
s
s
f
u
n
ctio
n
is
th
e
n
p
r
o
p
o
s
ed
,
r
ely
in
g
o
n
s
am
p
le
d
if
f
icu
lty
.
Mu
l
tic
lass
im
b
a
la
n
c
ed
cl
ass
i
f
ic
ati
o
n
p
r
o
b
l
em
is
co
n
s
i
d
e
r
e
d
ch
all
en
g
i
n
g
r
es
ea
r
c
h
d
u
e
t
o
t
h
e
co
m
p
le
x
it
ies
c
a
u
s
e
d
b
y
m
u
lti
p
le
cl
ass
es
[
2
1
]
.
S
e
v
e
r
al
r
es
ea
r
c
h
e
r
s
t
r
ie
d
t
o
c
o
m
b
in
e
t
h
e
en
s
e
m
b
le
m
et
h
o
d
s
,
s
u
c
h
as
b
ag
g
i
n
g
o
r
b
o
o
s
ti
n
g
,
wit
h
o
v
e
r
s
a
m
p
li
n
g
o
r
u
n
d
e
r
s
a
m
p
li
n
g
tec
h
n
i
q
u
es
t
o
a
d
d
r
ess
t
h
e
m
u
l
ticl
ass
i
m
b
ala
n
ce
d
p
r
o
b
l
em
[
2
2
]
.
M
o
r
e
r
ec
en
t
w
o
r
k
o
n
m
u
l
ticl
ass
i
m
b
al
an
ce
d
h
y
b
r
i
d
m
o
d
e
l
g
e
n
e
r
at
io
n
is
f
o
cu
s
ed
o
n
p
r
o
p
o
s
in
g
u
n
i
q
u
e
d
a
ta
-
le
v
el
m
e
th
o
d
s
an
d
co
m
b
i
n
i
n
g
t
h
e
m
wit
h
th
e
en
s
em
b
le
m
et
h
o
d
s
o
r
i
n
te
g
r
a
ti
n
g
th
e
s
tat
e
-
of
-
th
e
-
a
r
t
s
am
p
li
n
g
m
et
h
o
d
s
w
it
h
a
n
o
v
el
a
lg
o
r
it
h
m
ic
-
l
e
v
el
m
et
h
o
d
.
T
h
e
w
o
r
k
p
r
o
p
o
s
e
d
b
y
Har
to
n
o
et
a
l
.
[
2
3
]
in
tr
o
d
u
ce
d
a
g
en
er
aliza
tio
n
p
o
ten
tial
an
d
lear
n
in
g
d
if
f
icu
lt
y
-
b
ased
h
y
b
r
id
s
am
p
lin
g
(
G
DHS)
m
eth
o
d
as
a
d
ata
-
lev
el
m
eth
o
d
an
d
co
m
b
i
n
ed
it
with
th
e
g
r
ad
ien
t
b
o
o
s
tin
g
d
ec
is
io
n
tr
ee
(
DT
)
en
s
em
b
le
m
o
d
el.
I
n
GDHS
,
m
in
o
r
ity
class
r
ep
r
esen
tatio
n
is
im
p
r
o
v
ed
b
y
ap
p
ly
in
g
in
t
ellig
en
t
o
v
er
s
am
p
lin
g
,
an
d
th
e
m
ajo
r
ity
class
es
ar
e
clea
n
ed
to
m
in
im
ize
n
o
is
e
an
d
o
v
er
lap
.
So
m
e
r
ese
ar
ch
er
s
tr
ied
to
co
m
b
i
n
e
O
VO
o
r
OVA
with
o
v
er
s
am
p
lin
g
m
eth
o
d
s
an
d
en
s
em
b
le
class
if
icatio
n
o
r
d
ee
p
lear
n
in
g
,
s
u
ch
as
th
e
wo
r
k
p
r
o
p
o
s
ed
in
[
2
4
]
,
[
2
5
]
.
Saleh
i
an
d
Kh
ed
m
ati
[
2
1
]
s
u
g
g
ested
a
h
y
b
r
i
d
clu
s
ter
-
b
a
s
ed
o
v
er
s
am
p
lin
g
an
d
u
n
d
er
s
am
p
lin
g
(
HC
B
OU)
tech
n
iq
u
e,
w
h
ich
clu
s
ter
s
class
es
in
to
m
ajo
r
ity
an
d
m
in
o
r
it
y
g
r
o
u
p
s
to
g
u
id
e
th
e
s
am
p
lin
g
p
r
o
ce
s
s
.
HC
B
OU
p
r
eser
v
es
th
e
class
s
tr
u
ctu
r
e
an
d
p
r
o
d
u
ce
s
co
n
v
e
n
ien
t
s
y
n
th
etic
s
am
p
les.
T
h
e
n
o
v
el
HC
B
OU
i
s
in
teg
r
ated
with
OVO
an
d
OVA
class
if
ica
tio
n
d
ec
o
m
p
o
s
itio
n
m
eth
o
d
s
.
T
h
e
wo
r
k
p
r
esen
ted
in
th
is
p
ap
er
is
d
ir
ec
ted
at
g
en
er
atin
g
a
h
y
b
r
id
im
b
alan
ce
d
m
u
lticlas
s
class
if
icatio
n
m
o
d
el.
T
h
e
c
o
r
e
id
ea
is
to
in
te
g
r
ate
f
o
u
r
well
-
k
n
o
wn
p
o
wer
f
u
l
m
et
h
o
d
s
f
o
r
h
an
d
lin
g
im
b
alan
ce
d
d
ata
p
r
o
b
lem
,
t
o
co
n
s
tr
u
ct
a
h
ig
h
-
p
er
f
o
r
m
an
c
e
h
y
b
r
id
m
o
d
el.
Mo
r
e
s
p
ec
if
ically
,
th
e
u
tili
ze
d
m
eth
o
d
s
ar
e:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
h
a
n
d
lin
g
t
h
e
imb
a
la
n
ce
d
mu
lticla
s
s
cla
s
s
ifica
tio
n
p
r
o
b
lem
(
E
s
r
a
’
a
A
ls
h
d
a
ifa
t)
3985
‒
SMOT
E
m
eth
o
d
,
in
wh
ich
th
e
m
in
o
r
ity
class
is
o
v
er
s
am
p
led
to
b
alan
ce
th
e
d
ata
an
d
im
p
r
o
v
e
g
en
er
aliza
tio
n
.
‒
OVO
m
eth
o
d
,
in
wh
ich
a
m
u
l
ticlass
d
ata
s
et
is
m
ap
p
ed
in
to
a
n
u
m
b
er
o
f
b
in
a
r
y
d
atasets
,
an
d
a
class
if
ier
is
g
en
er
ated
f
o
r
ea
ch
.
T
h
is
s
im
p
lific
atio
n
ca
n
p
r
o
d
u
ce
b
etter
class
if
icatio
n
ef
f
ec
tiv
en
ess
.
‒
E
n
s
em
b
le
m
eth
o
d
,
in
wh
ic
h
s
ev
er
al
class
if
ier
s
ar
e
jo
in
e
d
t
o
en
h
an
ce
class
if
icatio
n
ef
f
ec
tiv
en
ess
.
No
te
h
er
e
th
at
th
e
b
in
ar
y
class
if
ier
s
g
en
er
ated
u
s
in
g
OVO
d
ec
o
m
p
o
s
itio
n
ar
e
co
n
s
id
er
ed
a
f
o
r
m
o
f
en
s
em
b
le.
Mo
r
eo
v
er
,
an
en
s
em
b
le
o
f
class
if
ier
s
th
at
ca
n
b
e
u
s
ed
as
a
b
ase
clas
s
if
ier
f
o
r
ea
ch
class
p
air
is
a
f
o
r
m
o
f
en
s
em
b
le,
an
d
b
o
th
f
o
r
m
s
ar
e
co
n
s
id
er
ed
in
th
e
wo
r
k
p
r
esen
t
ed
in
th
is
p
ap
e
r
.
‒
B
o
o
s
tin
g
m
eth
o
d
,
in
wh
ich
ea
ch
b
ase
class
if
ier
with
in
th
e
e
n
s
em
b
le
is
ev
alu
ated
,
an
d
th
o
s
e
with
lo
wer
p
er
f
o
r
m
an
ce
ar
e
b
o
o
s
ted
to
f
o
cu
s
m
o
r
e
o
n
th
e
s
am
p
les th
ey
m
is
class
if
ied
.
3.
T
H
E
H
YB
RID
I
M
B
A
L
ANC
E
D
M
U
L
T
I
C
L
AS
S CL
ASS
I
F
I
CAT
I
O
N
M
O
D
E
L
T
h
is
s
ec
tio
n
illu
s
tr
ates
th
e
co
n
s
tr
u
ctio
n
an
d
u
s
e
o
f
th
e
h
y
b
r
id
i
m
b
alan
ce
d
m
u
lticlas
s
class
if
icatio
n
m
o
d
el.
A
g
ain
,
th
e
f
u
n
d
am
e
n
tal
id
ea
is
to
m
er
g
e:
i)
o
v
er
s
am
p
lin
g
,
ii)
en
s
em
b
le,
iii)
d
ec
o
m
p
o
s
itio
n
,
an
d
iv
)
b
o
o
s
tin
g
m
eth
o
d
s
to
co
n
s
t
r
u
ct
an
ef
f
ec
tiv
e
class
if
icatio
n
m
o
d
el
f
o
r
im
b
alan
ce
d
m
u
lticlas
s
clas
s
if
icatio
n
p
r
o
b
lem
s
.
Fig
u
r
e
1
p
r
esen
ts
a
n
ex
am
p
le
o
f
th
e
d
esire
d
m
o
d
el
g
en
er
atio
n
p
r
o
ce
s
s
f
o
r
a
d
at
aset
in
clu
d
in
g
f
o
u
r
class
lab
els.
T
h
e
p
r
o
ce
s
s
b
eg
in
s
with
ap
p
ly
in
g
th
e
SMOT
E
to
b
alan
ce
th
e
d
ata.
Ne
x
t,
th
e
m
u
lticlas
s
d
ataset
i
s
d
ec
o
m
p
o
s
ed
in
to
m
u
ltip
le
b
in
ar
y
d
atasets
u
s
in
g
th
e
OVO
a
p
p
r
o
ac
h
.
An
in
itial
s
et
o
f
b
ase
class
if
ier
s
is
th
en
tr
ain
ed
an
d
ev
alu
ated
.
B
ased
o
n
th
e
e
v
alu
atio
n
r
esu
lts
,
ea
ch
b
ase
class
if
ier
is
eith
er
b
o
o
s
ted
o
r
n
o
t
,
an
d
af
ter
war
d
r
etr
ain
e
d
o
n
th
e
en
tire
co
r
r
esp
o
n
d
in
g
b
in
ar
y
d
at
aset
to
av
o
id
an
y
d
ata
lo
s
s
.
As
a
r
esu
lt,
a
s
et
o
f
b
alan
ce
d
an
d
b
o
o
s
ted
b
ase
c
lass
if
ier
s
i
s
g
en
er
ated
,
co
llec
tiv
ely
f
o
r
m
i
n
g
th
e
f
in
al
d
esire
d
h
y
b
r
id
m
o
d
el.
Alth
o
u
g
h
th
e
m
o
d
el
g
en
er
atio
n
p
r
o
ce
s
s
in
v
o
lv
es sev
er
al
s
tag
es,
it is
p
er
f
o
r
m
e
d
o
n
l
y
o
n
ce
.
Fig
u
r
e
1
.
T
h
e
g
e
n
er
atio
n
p
r
o
c
ess
o
f
th
e
h
y
b
r
id
im
b
ala
n
ce
d
m
u
lticlas
s
clas
s
if
icatio
n
m
o
d
e
l
T
h
e
d
etailed
p
r
o
ce
s
s
o
f
m
o
d
el
co
n
s
tr
u
ctio
n
is
ex
p
lain
ed
in
Alg
o
r
ith
m
1
.
T
h
e
alg
o
r
it
h
m
h
as
f
iv
e
in
p
u
ts
:
i)
th
e
i
n
p
u
t
d
ataset
D
,
ii)
th
e
s
et
o
f
class
es
C
,
iii)
th
e
SMOT
E
th
at
will
b
e
u
s
ed
t
o
b
alan
ce
th
e
d
ata
O
,
iv
)
th
e
class
if
icatio
n
alg
o
r
it
h
m
A
lg
o
th
at
will
b
e
u
tili
ze
d
to
c
o
n
s
tr
u
ct
th
e
b
ase
clas
s
if
ier
s
,
an
d
v
)
th
e
p
er
f
o
r
m
an
ce
th
r
esh
o
l
d
a
cc
_
th
r
esh
o
ld
th
at
will
b
e
ad
o
p
ted
to
s
p
o
t
th
e
class
if
ier
s
th
at
n
ee
d
to
b
e
b
o
o
s
ted
.
T
h
e
alg
o
r
ith
m
b
eg
in
s
b
y
a
p
p
ly
i
n
g
th
e
SMOT
E
to
th
e
d
at
aset
to
p
r
o
d
u
ce
t
h
e
b
alan
c
ed
r
esam
p
led
d
ata
R
esa
mp
led
_
D
(
lin
e
9
)
.
T
h
en
,
all
p
o
s
s
ib
le
c
o
m
b
in
atio
n
s
o
f
s
ize
two
class
es
f
ea
tu
r
ed
in
th
e
d
ataset
will
b
e
f
o
u
n
d
(
lin
e
1
0
)
.
T
h
e
alg
o
r
ith
m
th
en
lo
o
p
s
th
r
o
u
g
h
th
e
s
et
o
f
class
co
m
b
in
atio
n
s
,
an
d
o
n
ea
ch
iter
atio
n
,
it f
in
d
s
a
s
et
o
f
ex
am
p
les
D
i
in
D
th
at
f
ea
tu
r
e
s
C
i
(
lin
e
1
1
)
.
T
h
e
n
it
d
iv
id
es
D
i
in
to
tr
ain
in
g
an
d
v
al
id
atio
n
s
ets,
th
u
s
a
class
if
ier
ca
n
b
e
b
u
ilt
an
d
e
v
a
lu
ated
to
g
en
e
r
ate
an
ac
c
u
r
ac
y
s
co
r
e
a
cc
i
(
lin
es
14
an
d
1
5
)
.
T
h
e
n
ex
t
s
tep
is
to
id
en
tify
wea
k
class
if
ier
s
b
y
c
o
m
p
ar
in
g
th
e
ev
alu
ated
ac
cu
r
ac
y
s
co
r
e
with
th
e
ac
cu
r
ac
y
th
r
esh
o
ld
(
lin
e
1
6
)
.
I
f
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2252
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
20
25
:
3
9
8
2
-
3
9
9
3
3986
th
e
ac
cu
r
ac
y
s
co
r
e
is
u
n
d
e
r
t
h
e
p
r
e
d
ef
in
ed
th
r
esh
o
l
d
,
th
e
b
o
o
ts
tr
ap
m
et
h
o
d
is
ap
p
lied
to
th
e
m
is
class
if
ied
d
ata
,
an
d
th
e
r
esu
lt
is
ad
d
ed
to
th
e
D
i
d
ata
an
d
u
s
ed
to
r
eb
u
ild
th
e
b
o
o
s
ted
b
ase
class
if
ier
b
o
o
s
ted
_
cla
s
s
ifier
i
,
wh
ich
is
th
en
a
d
d
ed
to
t
h
e
s
et
o
f
b
ase
class
if
ier
s
f
o
r
m
in
g
th
e
h
y
b
r
id
m
o
d
el
(
lin
es
16
t
o
2
0
)
.
W
h
ile
if
t
h
e
ac
cu
r
ac
y
s
co
r
e
is
ab
o
v
e
th
e
p
r
ed
ef
in
ed
th
r
esh
o
ld
,
th
e
n
th
e
b
ase
class
if
ier
i
s
r
ec
o
n
s
tr
u
cted
u
s
in
g
th
e
tr
ain
in
g
d
ata
D
i
with
o
u
t
ap
p
ly
in
g
b
o
o
s
tin
g
an
d
th
en
ad
d
ed
to
th
e
s
et
o
f
b
ase
class
if
ier
s
f
o
r
m
in
g
th
e
h
y
b
r
id
m
o
d
e
l
(
l
in
es
2
2
a
n
d
2
3
)
.
T
h
e
h
y
b
r
id
class
if
icatio
n
m
o
d
el
is
th
e
o
u
tp
u
t
o
f
th
e
alg
o
r
ith
m
,
wh
ic
h
co
n
s
is
ts
o
f
a
s
et
o
f
b
in
ar
y
b
alan
ce
d
b
ase
class
if
ier
s
.
Alg
o
r
ith
m
1
.
Hy
b
r
id
im
b
alan
c
ed
m
u
lticlas
s
class
if
icat
io
n
m
o
d
el
co
n
s
tr
u
ctio
n
1: INPUT
2:
D:
the input dataset
3.
C:
the unique classes in
D
4.
O:
the oversampling technique
5:
Algo:
the classification algorithm
6:
acc_threshold:
accuracy threshold
7: OUTPUT
8: The generated hybrid classification model
9:
Resampled_D
=
Apply
O
on
D
10:
C_combinations
=
Find all sets of size 2 combinations in
C
11: for
i
=1 to
j
=|
C_combinations
| do
12:
D
i
= Find set of examples in
D
that feature
s
C
i
13
:
T
i
, V
i
= divide D
i
into training and validation sets
14:
classifier
i
= Use
Algo
to construct base classifier
classifier
i
using training set
T
i
15:
acc
i
= use
V
i
to evaluate
classifier
i
16:
if (
acc
i
<
acc_threshold)
17:
boosted_misclassified
i
= apply bootstrap on misclassified data
18:
boosted_D
i
=
D
i
∪
boosted_misclassified
i
19:
boosted_classifier
i
= Use
Algo
to construct base classifier using
boosted_D
i
20:
hybrid_model
=
hybrid_model
∪
boosted_classifier
i
21:
else
22:
classifier
i
= Use
Algo
to construct base classifier
C
i
using training set
D
i
23.
hybrid_model
=
hybrid_model
∪
classifier
i
24:
end if
25: end for
W
h
en
u
s
in
g
th
e
g
en
er
ate
d
h
y
b
r
id
m
o
d
el
f
o
r
p
r
ed
ictio
n
,
a
m
ajo
r
ity
v
o
tin
g
ap
p
r
o
ac
h
is
a
d
o
p
ted
to
ag
g
r
eg
ate
th
e
p
r
ed
ictio
n
s
f
r
o
m
th
e
m
em
b
er
b
in
a
r
y
class
if
ier
s
.
Mo
r
e
p
ar
ticu
lar
ly
,
to
cla
s
s
if
y
a
n
ew
u
n
s
ee
n
s
am
p
le,
all
th
e
in
d
iv
id
u
al
b
in
ar
y
class
if
ier
s
in
th
e
g
en
er
ated
h
y
b
r
id
class
if
icatio
n
m
o
d
el
ar
e
u
tili
ze
d
to
class
if
y
th
e
s
am
p
le,
an
d
th
e
cl
ass
lab
el
th
at
r
ec
eiv
es
th
e
m
ajo
r
ity
o
f
v
o
tes
is
co
n
s
id
er
ed
t
h
e
f
in
al
o
u
tp
u
t
an
d
is
ass
ig
n
ed
to
th
e
u
n
s
ee
n
s
am
p
le.
Hen
ce
,
th
e
well
-
k
n
o
wn
SMOT
E
m
eth
o
d
is
u
tili
ze
d
,
an
d
th
e
ad
o
p
ted
d
ec
o
m
p
o
s
itio
n
m
eth
o
d
is
th
e
OVO
;
we
will
r
ef
er
to
th
e
h
y
b
r
id
m
o
d
el
as
B
o
o
s
ted
-
OVO&
SMOT
E
th
r
o
u
g
h
o
u
t
th
e
r
est o
f
th
e
p
ap
er
.
Fo
r
ev
alu
atin
g
t
h
e
r
esu
ltin
g
m
o
d
el,
th
e
ac
c
u
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all
,
an
d
F1
-
s
co
r
e
a
r
e
co
n
s
id
er
ed
:
‒
Acc
u
r
ac
y
:
th
e
r
atio
o
f
c
o
r
r
ec
t
ly
p
r
ed
ict
ed
o
b
s
er
v
atio
n
s
to
all
o
b
s
er
v
atio
n
s
in
a
g
iv
en
test
s
et
[
2
6
]
.
=
+
+
+
+
(
1
)
‒
Pre
cisi
o
n
:
th
e
r
atio
o
f
o
b
s
er
v
atio
n
s
co
r
r
ec
tly
p
r
ed
icted
a
s
p
o
s
itiv
e
to
all
o
b
s
er
v
atio
n
s
p
r
ed
icted
as
p
o
s
itiv
e
[
2
6
]
.
=
+
(
2
)
‒
R
ec
all:
th
e
r
atio
o
f
o
b
s
er
v
atio
n
s
co
r
r
ec
tly
p
r
ed
icted
as p
o
s
itiv
e
to
all
ac
tu
al
p
o
s
itiv
e
o
b
s
er
v
atio
n
s
[
2
6
]
.
=
+
(
3
)
‒
F1
-
s
co
r
e:
it
r
ep
r
esen
ts
a
co
m
b
in
atio
n
o
f
t
h
e
p
r
ec
is
io
n
a
n
d
r
e
ca
ll sco
r
es
[
2
6
]
.
1
−
=
2
∗
∗
+
(
4
)
Her
e
,
TP
d
en
o
te
s
th
e
tr
u
e
p
o
s
itiv
e
,
T
N
d
en
o
te
s
th
e
tr
u
e
n
eg
ativ
e
,
FP
d
en
o
te
s
th
e
f
als
e
p
o
s
itiv
e
,
an
d
FN
d
en
o
te
s
th
e
f
alse
n
eg
ativ
e
r
ec
o
r
d
s
.
B
ec
au
s
e
th
e
d
atasets
tak
en
in
to
co
n
s
id
er
atio
n
i
n
th
is
s
tu
d
y
ar
e
m
u
lticlas
s
d
atasets
,
m
ac
r
o
s
co
r
es a
r
e
u
tili
ze
d
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
h
a
n
d
lin
g
t
h
e
imb
a
la
n
ce
d
mu
lticla
s
s
cla
s
s
ifica
tio
n
p
r
o
b
lem
(
E
s
r
a
’
a
A
ls
h
d
a
ifa
t)
3987
4.
DATAS
E
T
S
T
h
is
s
ec
tio
n
p
r
o
v
id
es
a
s
u
m
m
ar
y
o
f
th
e
m
ain
attr
ib
u
tes
o
f
t
h
e
d
atasets
u
s
ed
to
ass
es
s
th
e
p
r
o
p
o
s
ed
h
y
b
r
id
m
o
d
el.
Sev
en
teen
im
b
alan
ce
d
d
atasets
f
r
o
m
v
ar
io
u
s
d
is
cip
lin
es,
ea
ch
with
a
d
if
f
er
en
t
n
u
m
b
er
o
f
o
b
s
er
v
atio
n
s
,
class
es
an
d
attr
ib
u
tes
,
all
s
o
u
r
ce
d
f
r
o
m
th
e
Un
iv
er
s
ity
o
f
C
alif
o
r
n
ia
I
r
v
i
n
e
(
UC
I
)
Ma
ch
in
e
L
ea
r
n
in
g
R
ep
o
s
ito
r
y
[
2
7
]
.
T
a
b
le
1
o
u
tlin
es
th
e
k
ey
f
ea
tu
r
es
o
f
th
ese
d
atasets
.
B
ec
au
s
e
th
e
r
esear
ch
p
r
esen
ted
in
th
is
p
ap
er
f
o
cu
s
es
o
n
im
b
a
lan
ce
d
m
u
lticlas
s
class
if
icatio
n
p
r
o
b
lem
s
,
th
e
d
atasets
in
clu
d
e
a
r
an
g
e
o
f
class
d
is
tr
ib
u
tio
n
r
ates.
T
ab
le
1
.
T
h
e
d
escr
ip
tio
n
o
f
th
e
ex
p
er
im
en
tal
d
atasets
D
o
ma
i
n
D
i
st
r
i
b
u
t
i
o
n
o
f
c
l
a
ss
e
s
r
a
t
i
o
#
o
f
i
n
s
t
a
n
c
e
s
#
o
f
f
e
a
t
u
r
e
s
#
o
f
c
l
a
ss
e
s
D
a
t
a
s
e
t
B
i
o
l
o
g
y
1
4
0
7
/
2
4
0
6
/
3
6
4
(
R
a
t
i
o
=
3
3
.
7
:
5
7
.
6
:
8
.
7
)
4
1
7
7
8
3
A
b
a
l
o
n
e
H
e
a
l
t
h
a
n
d
m
e
d
i
c
i
n
e
4
1
5
/
2
2
7
/
8
3
1
(
R
a
t
i
o
=
2
8
.
1
7
:
1
5
.
4
1
:
5
6
.
4
2
)
1
4
7
3
9
3
C
o
n
t
r
a
c
e
p
t
i
v
e
m
e
t
h
o
d
S
o
c
i
a
l
s
c
i
e
n
c
e
6
5
/
6
4
/
3
1
(
R
a
t
i
o
=
4
0
.
6
3
:
4
0
.
0
0
:
1
9
.
3
8
)
1
6
0
4
3
H
a
y
e
s
-
R
o
t
h
H
e
a
l
t
h
a
n
d
m
e
d
i
c
i
n
e
2
/
2
4
/
6
4
(
R
a
t
i
o
=
2
.
2
2
:
2
6
.
6
7
:
7
1
.
1
1
)
90
8
3
P
o
st
-
o
p
e
r
a
t
i
v
e
H
e
a
l
t
h
a
n
d
m
e
d
i
c
i
n
e
1
5
0
/
3
5
/
3
0
(
R
a
t
i
o
=
6
9
.
7
:
1
6
.
3
:
1
4
.
0
)
2
1
5
5
3
Th
y
r
o
i
d
H
e
a
l
t
h
a
n
d
m
e
d
i
c
i
n
e
6
0
/
1
5
0
/
1
0
0
(
R
a
t
i
o
=
1
9
.
3
5
:
4
8
.
3
9
:
3
2
.
2
6
)
3
1
0
6
3
V
e
r
t
e
b
r
a
l
A
u
t
o
mo
t
i
v
e
1
9
9
/
2
1
7
/
2
1
8
/
2
1
2
(
R
a
t
i
o
=
2
3
.
5
2
:
2
5
.
6
6
:
2
5
.
7
9
:
2
5
.
0
3
)
8
4
6
18
4
V
e
h
i
c
l
e
A
u
t
o
mo
t
i
v
e
1
2
1
0
/
3
8
4
/
6
5
/
6
9
(
R
a
t
i
o
=
7
0
.
0
:
2
2
.
2
:
3
.
8
:
4
.
0
)
1
7
2
8
6
4
Car
H
e
a
l
t
h
a
n
d
m
e
d
i
c
i
n
e
1
6
0
/
5
4
/
3
5
/
3
5
/
1
3
(
R
a
t
i
o
=
5
3
.
9
:
1
8
.
2
:
1
1
.
8
:
1
1
.
8
:
4
.
4
)
2
9
7
13
5
H
e
a
r
t
(
C
l
e
v
e
l
a
n
d
)
S
o
c
i
a
l
s
c
i
e
n
c
e
4
3
2
0
/
2
/
3
2
8
/
4
2
6
6
/
4
0
4
4
(
R
a
t
i
o
=
3
3
.
3
:
0
.
0
1
5
:
2
.
5
:
3
2
.
9
:
3
1
.
2
)
1
2
9
6
0
8
5
N
u
r
sery
C
o
m
p
u
t
e
r
s
c
i
e
n
c
e
4
9
1
3
/
3
2
9
/
2
8
/
8
8
/
1
1
5
(
R
a
t
i
o
=
8
9
.
8
:
6
.
0
:
0
.
5
:
1
.
6
:
2
.
1
)
5
4
7
3
10
5
P
a
g
e
b
l
o
c
k
s
H
e
a
l
t
h
a
n
d
M
e
d
i
c
i
n
e
1
1
2
/
6
1
/
7
2
/
4
9
/
5
2
/
2
0
(
R
a
t
i
o
=
3
0
.
6
:
1
6
.
7
:
1
9
.
7
:
1
3
.
4
:
1
4
.
2
:
5
.
5
)
3
6
6
34
6
D
e
r
mat
o
l
o
g
y
B
i
o
l
o
g
y
2
0
2
7
/
1
3
2
2
/
5
2
2
/
1
6
3
0
/
1
9
2
8
/
2
6
3
6
/
3
5
4
6
(
R
a
t
i
o
=
1
4
.
9
:
9
.
7
:
3
.
8
:
1
2
.
0
:
1
4
.
2
:
1
9
.
3
:
2
6
.
0
)
1
3
6
1
1
16
7
D
r
y
b
e
a
n
P
h
y
s
i
c
s
a
n
d
c
h
e
mi
s
t
r
y
7
0
/
1
7
/
0
/
7
6
/
1
3
/
9
/
2
9
(
R
a
t
i
o
=
3
2
.
7
:
7
.
9
:
0
.
0
:
3
5
.
5
:
6
.
1
:
4
.
2
:
1
3
.
6
)
2
1
4
9
7
G
l
a
ss
B
i
o
l
o
g
y
1
4
3
/
7
7
/
5
2
/
3
5
/
2
0
/
5
/
2
/
2
(
R
a
t
i
o
=
4
2
.
5
:
2
2
.
9
:
1
5
.
4
:
1
0
.
4
:
5
.
9
:
1
.
5
:
0
.
6
:
0
.
6
)
3
3
6
7
8
E.
c
o
l
i
C
o
m
p
u
t
e
r
s
c
i
e
n
c
e
1
1
4
3
/
1
1
4
3
/
1
1
4
4
/
1
0
5
5
/
1
1
4
4
/
1
0
5
5
/
1
0
5
6
/
1
1
4
2
/
1
0
5
5
/
1
0
5
5
(
R
a
t
i
o
=
1
0
.
4
:
1
0
.
4
:
1
0
.
4
:
9
.
6
:
1
0
.
4
:
9
.
6
:
9
.
6
:
1
0
.
4
:
9
.
6
:
9
.
6
)
1
0
9
9
2
16
10
P
e
n
d
i
g
i
t
s
B
i
o
l
o
g
y
2
4
4
/
4
2
9
/
4
6
3
/
4
4
/
3
5
/
5
1
/
1
6
3
/
3
0
/
2
0
/
5
(
R
a
t
i
o
=
1
6
.
4
:
2
8
.
9
:
3
1
.
2
:
3
.
0
:
2
.
4
:
3
.
4
:
1
1
.
0
:
2
.
0
:
1
.
3
:
0
.
3
)
1
4
8
4
8
10
Y
e
a
st
5.
E
XP
E
R
I
M
E
N
T
S
AN
D
ANA
L
YS
I
S
T
h
is
s
ec
tio
n
d
is
cu
s
s
es
th
e
ex
p
er
im
en
tal
s
etu
p
an
d
r
ep
o
r
ts
th
e
o
b
tain
ed
r
esu
lts
.
Fo
r
b
u
ild
in
g
th
e
in
d
iv
id
u
al
class
if
ier
s
,
th
r
ee
alg
o
r
ith
m
s
wer
e
em
p
lo
y
ed
:
i
)
DT
,
ii)
s
u
p
p
o
r
t
v
ec
to
r
m
a
ch
in
e
(
SVM)
,
an
d
iii)
r
an
d
o
m
f
o
r
est
(
R
F).
T
h
ese
alg
o
r
ith
m
s
wer
e
ch
o
s
en
b
ec
a
u
s
e
o
f
:
i)
t
h
eir
d
if
f
er
en
t
lea
r
n
i
n
g
b
e
h
av
io
r
s
,
wh
ich
en
ab
le
co
m
p
r
eh
en
s
iv
e
ev
al
u
a
tio
n
o
f
th
e
ef
f
ec
tiv
en
ess
o
f
t
h
e
s
u
g
g
ested
h
y
b
r
id
m
o
d
el
to
b
e
co
n
d
u
cted
,
an
d
ii)
th
eir
p
o
p
u
lar
ity
an
d
r
e
p
o
r
ted
p
e
r
f
o
r
m
an
ce
in
p
r
ed
ic
tio
n
.
DT
is
well
-
k
n
o
wn
f
o
r
its
s
im
p
licity
an
d
in
ter
p
r
etab
ilit
y
,
SVM
is
ef
f
ec
t
iv
e
in
h
ig
h
-
d
im
e
n
s
io
n
al
s
p
ac
es
,
an
d
R
F,
a
s
an
en
s
em
b
le
cla
s
s
if
icatio
n
m
eth
o
d
,
is
r
ec
o
g
n
ized
f
o
r
im
p
r
o
v
in
g
c
lass
if
icatio
n
ef
f
ec
tiv
en
ess
.
T
o
en
s
u
r
e
p
r
ec
is
e
r
esu
lts
,
ten
-
f
o
ld
cr
o
s
s
v
alid
atio
n
(
T
C
V)
was
em
p
lo
y
ed
f
o
r
all
th
e
ex
p
er
im
en
ts
r
ep
o
r
te
d
in
th
is
p
ap
er
.
T
h
e
ev
alu
atio
n
m
ea
s
u
r
es
in
clu
d
ed
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
e
ca
ll
,
an
d
F
1
-
s
co
r
e.
T
o
s
im
p
lify
t
h
e
a
n
aly
s
is
,
th
e
r
esu
lts
will
b
e
d
is
cu
s
s
ed
b
ased
o
n
t
h
e
F1
-
s
co
r
e
b
ec
au
s
e:
i)
it c
o
m
b
in
es two
m
ea
s
u
r
es; p
r
ec
is
io
n
an
d
r
ec
all
,
an
d
ii)
it r
ef
lects p
r
ec
is
e
p
er
f
o
r
m
an
ce
f
o
r
im
b
alan
ce
d
d
atasets
.
W
ith
r
esp
ec
t
to
SMOT
E
,
th
e
k
-
n
e
ig
h
b
o
r
s
p
a
r
am
eter
is
s
et
to
o
n
e
b
ec
au
s
e
s
o
m
e
ev
alu
atio
n
d
atasets
in
clu
d
e
o
n
ly
two
s
am
p
les
f
o
r
th
e
m
in
o
r
ity
class
.
T
h
e
S
VM
cla
s
s
if
ier
e
m
p
lo
y
ed
th
e
r
a
d
ial
b
asis
f
u
n
ctio
n
(
R
B
F)
k
er
n
el.
Fifty
class
if
ier
s
wer
e
co
n
s
tr
u
cted
as
b
ase
cla
s
s
if
ier
s
f
o
r
th
e
R
F
cla
s
s
if
ier
.
E
ac
h
d
ataset
is
ev
alu
ated
u
s
in
g
th
r
e
e
m
eth
o
d
s
co
u
p
led
with
th
r
ee
class
if
icatio
n
alg
o
r
ith
m
s
.
Mo
r
e
s
p
ec
if
ically
,
f
o
r
ea
ch
class
if
icatio
n
alg
o
r
ith
m
,
th
e
m
eth
o
d
s
ar
e:
i)
OVO
with
o
n
e
o
f
th
e
b
ase
class
if
ier
s
(
OVO)
,
ii)
OVO
a
n
d
SMOT
E
(
OVO&
SMOT
E
)
,
an
d
iii)
OVO
co
u
p
led
with
SMOT
E
an
d
b
o
o
ts
tr
ap
b
o
o
s
tin
g
(
B
o
o
s
ted
-
OVO&
SMOT
E
)
.
As
n
o
ted
ea
r
lier
,
a
th
r
esh
o
ld
v
alu
e
is
u
tili
ze
d
to
s
p
o
t
th
e
class
if
ier
s
th
at
s
h
o
u
ld
b
e
b
o
o
s
ted
;
s
ev
er
al
ex
p
er
im
en
ts
wer
e
co
n
d
u
cted
to
id
en
tify
th
e
b
est
th
r
esh
o
ld
v
alu
e
f
o
r
ea
ch
d
ataset
an
d
class
if
icatio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2252
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
20
25
:
3
9
8
2
-
3
9
9
3
3988
alg
o
r
ith
m
.
T
a
b
le
2
p
r
esen
ts
th
e
ad
o
p
ted
t
h
r
esh
o
ld
v
alu
es f
o
r
ea
ch
co
n
s
id
er
e
d
ev
alu
atio
n
d
a
taset
an
d
class
if
ier
.
T
h
e
p
r
o
d
u
ce
d
r
esu
lts
ar
e
p
r
ese
n
ted
an
d
d
is
cu
s
s
ed
in
th
e
n
e
x
t
s
ub
-
s
ec
tio
n
s
.
T
ab
le
2
.
T
h
e
ad
o
p
ted
b
o
o
s
tin
g
th
r
esh
o
ld
v
alu
es
B
e
st
b
o
o
st
i
n
g
t
h
r
e
s
h
o
l
d
v
a
l
u
e
D
a
t
a
s
e
t
DT
b
o
o
st
i
n
g
t
h
r
e
sh
o
l
d
S
V
M
b
o
o
st
i
n
g
t
h
r
e
s
h
o
l
d
RF
B
o
o
st
i
n
g
t
h
r
e
sh
o
l
d
A
b
a
l
o
n
e
0
.
7
5
0
.
8
0
0
.
9
5
C
o
n
t
r
a
c
e
p
t
i
v
e
0
.
7
0
0
.
7
0
0
.
7
5
H
a
y
e
s
R
o
t
h
0
.
8
5
0
.
9
0
0
.
7
0
P
o
st
-
o
p
e
r
a
t
i
v
e
0
.
8
5
0
.
7
2
0
.
6
5
Th
y
r
o
i
d
0
.
9
9
0
.
9
5
0
.
9
5
V
e
r
t
e
b
r
a
l
0
.
9
5
0
.
7
0
0
.
9
5
V
e
h
i
c
l
e
0
.
9
0
0
.
9
9
0
.
7
5
Car
0
.
9
5
0
.
9
9
0
.
9
9
H
e
a
r
t
0
.
7
5
0
.
9
9
0
.
8
0
N
u
r
sery
0
.
9
5
0
.
9
5
0
.
9
5
P
a
g
e
b
l
o
c
k
s
0
.
9
5
0
.
9
0
0
.
9
5
D
e
r
mat
o
l
o
g
y
0
.
9
4
0
.
9
5
0
.
9
5
D
r
y
b
e
a
n
0
.
9
0
0
.
9
5
0
.
9
9
G
l
a
ss
0
.
8
9
0
.
7
0
0
.
9
5
E
.
c
o
l
i
0
.
9
9
0
.
9
0
0
.
9
0
P
e
n
d
i
g
i
t
s
0
.
9
9
0
.
9
9
0
.
9
9
Y
e
a
st
0
.
9
0
0
.
6
0
0
.
8
0
5
.
1
.
Resul
t
s
o
bta
ined f
ro
m
u
s
ing
t
he
DT
cla
s
s
if
ier
t
o
co
ns
t
ruct
t
he
hy
brid m
o
del
I
n
th
is
s
ec
tio
n
,
th
e
r
esu
lts
p
r
o
d
u
ce
d
f
r
o
m
u
s
in
g
th
e
DT
c
lass
if
ier
to
g
en
er
ate
th
e
d
esir
ed
h
y
b
r
id
m
o
d
el
ar
e
p
r
esen
ted
a
n
d
d
is
cu
s
s
ed
.
T
h
e
r
esu
lts
ar
e
tab
u
lated
in
T
ab
le
3
,
an
d
th
e
b
est r
esu
lts
ar
e
h
ig
h
lig
h
ted
in
b
o
ld
f
o
n
t.
C
o
m
m
e
n
cin
g
with
co
m
p
ar
in
g
th
e
p
er
f
o
r
m
an
ce
o
f
OVO
an
d
OVO
SMOT
E
m
o
d
els
,
f
r
o
m
t
h
e
tab
le
,
it
is
clea
r
th
at
co
m
b
in
in
g
SMOT
E
an
d
OVO
o
u
tp
er
f
o
r
m
s
u
s
in
g
OVO
alo
n
e.
T
h
e
s
am
e
o
b
s
er
v
atio
n
is
n
o
ticed
wh
en
co
m
p
a
r
in
g
th
e
r
esu
lts
o
b
tain
ed
f
r
o
m
u
s
in
g
th
e
B
o
o
s
ted
-
OVO&
SMOT
E
h
y
b
r
id
m
o
d
el
an
d
th
e
OV
O
m
o
d
el.
T
h
u
s
,
co
m
b
in
in
g
O
VO
an
d
SMOT
E
to
g
en
e
r
a
te
a
h
y
b
r
i
d
m
o
d
el
im
p
r
o
v
e
d
th
e
class
if
icatio
n
ef
f
ec
tiv
en
ess
.
R
eg
ar
d
in
g
c
o
m
p
ar
in
g
th
e
p
r
o
p
o
s
ed
h
y
b
r
id
m
o
d
el
(
B
o
o
s
ted
-
OVO&
SMOT
E
)
with
OVO&
SMOT
E
,
it
i
s
o
b
v
io
u
s
th
at
th
e
h
y
b
r
id
m
o
d
el
o
u
tp
er
f
o
r
m
s
th
e
OVO&
SMOT
E
m
o
d
el.
Mo
r
e
s
p
ec
if
ically
,
B
o
o
s
ted
-
OVO&
S
MO
T
E
g
en
er
ated
th
e
b
est
r
esu
lts
f
o
r
all
th
e
co
n
s
id
er
e
d
d
at
asets
.
Ho
wev
er
,
f
o
r
s
ix
d
atasets
,
th
e
s
am
e
r
esu
lts
wer
e
o
b
tain
e
d
f
r
o
m
u
s
in
g
OVO&
SMOT
E
.
C
o
n
s
eq
u
e
n
tly
,
b
o
o
s
tin
g
th
e
r
elativ
ely
lo
w
-
p
er
f
o
r
m
a
n
ce
cl
ass
if
ier
s
r
esu
lted
in
im
p
r
o
v
in
g
th
e
class
if
icatio
n
ef
f
ec
tiv
en
ess
.
T
ab
le
3
.
R
esu
lts
o
b
tain
ed
f
r
o
m
u
s
in
g
th
e
DT
c
lass
if
ier
as th
e
b
ase
c
lass
if
ier
D
a
t
a
s
e
t
OVO
O
V
O
&S
M
O
TE
B
o
o
st
e
d
-
O
V
O
&S
M
O
TE
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
b
a
l
o
n
e
0
.
6
6
8
0
.
5
7
1
0
.
5
8
9
0
.
5
7
5
0
.
8
0
0
0
.
7
9
9
0
.
8
0
0
0
.
7
9
9
0
.
8
0
0
0
.
7
9
9
0
.
8
0
0
0
.
7
9
9
C
o
n
t
r
a
c
e
p
t
i
v
e
0
.
4
6
4
0
.
4
4
9
0
.
4
4
1
0
.
4
3
9
0
.
5
6
9
0
.
5
6
9
0
.
5
7
0
0
.
5
6
5
0
.
5
7
4
0
.
5
7
3
0
.
5
7
4
0
.
5
7
1
H
a
y
e
s
R
o
t
h
0
.
8
2
5
0
.
8
6
4
0
.
8
5
5
0
.
8
5
3
0
.
8
3
7
0
.
8
4
4
0
.
8
2
7
0
.
8
1
4
0
.
8
4
1
0
.
8
5
2
0
.
8
4
3
0
.
8
2
6
P
o
st
-
o
p
e
r
a
t
i
v
e
0
.
6
4
3
0
.
5
3
6
0
.
5
2
7
0
.
5
2
1
0
.
7
9
7
0
.
8
0
4
0
.
7
9
5
0
.
7
8
9
0
.
8
2
3
0
.
8
4
2
0
.
8
2
5
0
.
8
1
6
Th
y
r
o
i
d
0
.
9
4
4
0
.
9
4
2
0
.
9
2
3
0
.
9
2
2
0
.
9
6
4
0
.
9
6
4
0
.
9
6
3
0
.
9
6
3
0
.
9
7
6
0
.
9
7
7
0
.
9
7
4
0
.
9
7
5
V
e
r
t
e
b
r
a
l
0
.
8
0
6
0
.
7
7
0
0
.
7
5
6
0
.
7
5
2
0
.
8
3
3
0
.
8
4
1
0
.
8
3
0
0
.
8
2
8
0
.
8
4
9
0
.
8
4
6
0
.
8
4
6
0
.
8
4
3
V
e
h
i
c
l
e
0
.
7
0
4
0
.
7
2
3
0
.
7
0
6
0
.
7
0
9
0
.
7
1
6
0
.
7
2
0
0
.
7
1
4
0
.
7
1
1
0
.
7
4
6
0
.
7
5
0
0
.
7
4
6
0
.
7
4
3
Car
0
.
8
5
9
0
.
8
6
3
0
.
8
1
6
0
.
7
9
0
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
H
e
a
r
t
0
.
4
5
9
0
.
2
7
9
0
.
2
7
0
0
.
2
6
8
0
.
7
9
1
0
.
7
9
8
0
.
7
8
8
0
.
7
8
6
0
.
7
9
1
0
.
7
9
8
0
.
7
8
8
0
.
7
8
6
N
u
r
sery
0
.
8
4
8
0
.
8
6
6
0
.
8
3
5
0
.
8
1
2
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
0
.
9
9
9
P
a
g
e
b
l
o
c
k
s
0
.
9
5
9
0
.
8
1
7
0
.
8
0
9
0
.
7
9
9
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
0
.
9
9
1
D
e
r
mat
o
l
o
g
y
0
.
9
4
4
0
.
9
4
8
0
.
9
4
8
0
.
9
4
2
0
.
9
7
4
0
.
9
7
4
0
.
9
6
9
0
.
9
7
0
0
.
9
7
6
0
.
9
7
5
0
.
9
7
1
0
.
9
7
2
D
r
y
b
e
a
n
0
.
7
0
3
0
.
7
8
3
0
.
7
6
4
0
.
7
3
3
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
0
.
9
4
4
G
l
a
ss
0
.
7
1
1
0
.
6
9
1
0
.
7
1
6
0
.
6
8
3
0
.
8
6
6
0
.
8
6
5
0
.
8
7
5
0
.
8
5
8
0
.
8
9
0
0
.
8
9
4
0
.
8
9
5
0
.
8
8
5
E.
c
o
l
i
0
.
8
0
9
0
.
7
0
4
0
.
6
8
9
0
.
6
7
2
0
.
9
4
0
0
.
9
3
9
0
.
9
4
0
0
.
9
3
7
0
.
9
4
8
0
.
9
4
7
0
.
9
5
1
0
.
9
4
7
P
e
n
d
i
g
i
t
s
0
.
9
6
5
0
.
9
6
6
0
.
9
6
5
0
.
9
6
5
0
.
9
6
9
0
.
9
6
9
0
.
9
6
9
0
.
9
6
9
0
.
9
7
1
0
.
9
7
1
0
.
9
7
1
0
.
9
7
1
Y
e
a
st
0
.
4
8
7
0
.
4
4
9
0
.
4
5
8
0
.
4
4
2
0
.
8
4
3
0
.
8
4
8
0
.
8
4
2
0
.
8
4
3
0
.
8
4
9
0
.
8
5
3
0
.
8
4
9
0
.
8
4
9
5
.
2
.
Resul
t
s
o
bta
ined f
ro
m
u
s
ing
t
he
s
u
pp
o
rt
v
ec
t
o
r
m
a
c
hin
e
cla
s
s
if
ier
t
o
co
ns
t
ruct
t
he
hy
brid m
o
del
I
n
th
is
s
ec
tio
n
,
th
e
r
esu
lts
ac
h
iev
ed
f
r
o
m
u
s
in
g
th
e
SVM
class
if
ier
to
g
en
er
ate
th
e
d
esire
d
h
y
b
r
id
m
o
d
el
ar
e
p
r
esen
ted
an
d
d
is
c
u
s
s
ed
.
T
h
e
r
esu
lts
ar
e
tab
u
lat
ed
in
T
a
b
le
4
.
Ag
ain
,
co
m
b
in
in
g
SMOT
E
with
OVO
o
u
tp
er
f
o
r
m
s
u
s
in
g
OVO
alo
n
e.
R
eg
ar
d
i
n
g
co
m
p
ar
in
g
t
h
e
B
o
o
s
ted
-
OVO&
SMOT
E
an
d
OVO&
SMOT
E
,
th
e
b
o
o
s
ted
ap
p
r
o
ac
h
g
e
n
er
at
ed
th
e
s
am
e
o
r
b
etter
r
esu
lts
f
o
r
all
th
e
co
n
s
id
er
ed
d
atasets
.
Mo
r
e
s
p
ec
if
ically
,
B
o
o
s
ted
-
OVO&
SMOT
E
p
r
o
d
u
ce
d
b
etter
r
esu
lts
f
o
r
eig
h
t
o
f
th
e
s
ev
en
teen
co
n
s
id
er
e
d
d
atasets
(
ab
alo
n
e,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
h
a
n
d
lin
g
t
h
e
imb
a
la
n
ce
d
mu
lticla
s
s
cla
s
s
ifica
tio
n
p
r
o
b
lem
(
E
s
r
a
’
a
A
ls
h
d
a
ifa
t)
3989
co
n
tr
ac
ep
tiv
e
,
Hay
es
R
o
th
,
th
y
r
o
id
,
ca
r
,
h
ea
r
t,
g
lass
,
an
d
E
.
co
li
)
,
f
o
r
t
h
e
n
in
e
r
em
ain
in
g
d
atasets
th
e
s
am
e
r
esu
lts
wer
e
p
r
o
d
u
ce
d
b
y
th
e
OVO&
SMOT
E
m
o
d
el.
No
te
h
er
e
th
at
th
e
o
b
s
er
v
atio
n
s
f
r
o
m
u
s
in
g
th
e
DT
an
d
SVM
class
if
ier
s
ar
e
h
ar
m
o
n
ic.
T
ab
le
4
.
R
esu
lts
o
b
tain
ed
f
r
o
m
u
s
in
g
th
e
SVM
c
lass
if
ier
as th
e
b
ase
c
lass
if
ier
D
a
t
a
s
e
t
OVO
O
V
O
&S
M
O
TE
B
o
o
st
e
d
-
O
V
O
&S
M
O
TE
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
b
a
l
o
n
e
0
.
7
4
8
0
.
5
1
6
0
.
5
2
2
0
.
5
1
0
0
.
7
1
6
0
.
7
2
2
0
.
7
1
6
0
.
7
1
8
0
.
7
2
3
0
.
7
2
2
0
.
7
2
3
0
.
7
2
0
C
o
n
t
r
a
c
e
p
t
i
v
e
0
.
4
8
9
0
.
4
7
4
0
.
4
7
2
0
.
4
7
0
0
.
5
1
7
0
.
5
3
7
0
.
5
1
8
0
.
5
0
9
0
.
5
1
6
0
.
5
1
7
0
.
5
1
5
0
.
5
1
0
H
a
y
e
s
R
o
t
h
0
.
5
5
0
0
.
6
3
2
0
.
5
7
3
0
.
5
6
9
0
.
6
5
2
0
.
6
5
7
0
.
6
6
5
0
.
6
3
7
0
.
7
3
8
0
.
7
6
6
0
.
7
4
9
0
.
7
2
8
P
o
st
-
o
p
e
r
a
t
i
v
e
0
.
6
8
8
0
.
2
8
7
0
.
4
1
7
0
.
3
3
9
0
.
7
9
8
0
.
8
1
5
0
.
8
0
2
0
.
7
9
1
0
.
7
9
8
0
.
8
1
5
0
.
8
0
2
0
.
7
9
1
Th
y
r
o
i
d
0
.
9
5
3
0
.
9
4
7
0
.
8
9
2
0
.
8
9
2
0
.
9
7
3
0
.
9
7
5
0
.
9
7
3
0
.
9
7
3
0
.
9
8
0
0
.
9
8
2
0
.
9
7
9
0
.
9
7
9
V
e
r
t
e
b
r
a
l
0
.
8
1
6
0
.
8
0
1
0
.
7
8
1
0
.
7
8
1
0
.
7
9
3
0
.
7
9
5
0
.
7
9
4
0
.
7
8
7
0
.
7
9
3
0
.
7
9
5
0
.
7
9
4
0
.
7
8
7
V
e
h
i
c
l
e
0
.
7
5
1
0
.
7
3
9
0
.
7
5
3
0
.
7
3
8
0
.
7
6
7
0
.
7
5
5
0
.
7
6
9
0
.
7
5
4
0
.
7
6
3
0
.
7
6
3
0
.
7
6
3
0
.
7
5
4
Car
0
.
9
2
9
0
.
9
2
9
0
.
8
5
8
0
.
8
5
9
0
.
9
8
9
0
.
9
9
0
0
.
9
8
9
0
.
9
8
9
0
.
9
9
0
0
.
9
9
0
0
.
9
9
0
0
.
9
9
0
H
e
a
r
t
0
.
5
7
2
0
.
2
3
3
0
.
2
7
6
0
.
2
5
1
0
.
7
2
0
0
.
7
2
3
0
.
7
1
9
0
.
7
0
5
0
.
7
4
6
0
.
7
5
5
0
.
7
4
7
0
.
7
4
3
N
u
r
sery
0
.
9
0
7
0
.
8
8
0
0
.
8
4
8
0
.
8
4
7
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
0
.
9
8
9
P
a
g
e
b
l
o
c
k
s
0
.
9
3
9
0
.
6
8
8
0
.
5
2
6
0
.
5
7
0
0
.
9
3
7
0
.
9
3
8
0
.
9
3
7
0
.
9
3
7
0
.
9
3
7
0
.
9
3
8
0
.
9
3
7
0
.
9
3
7
D
e
r
mat
o
l
o
g
y
0
.
9
7
5
0
.
9
7
5
0
.
9
7
1
0
.
9
7
2
0
.
9
8
6
0
.
9
8
5
0
.
9
8
8
0
.
9
8
6
0
.
9
8
6
0
.
9
8
5
0
.
9
8
8
0
.
9
8
6
D
r
y
b
e
a
n
0
.
8
9
5
0
.
9
2
5
0
.
9
1
1
0
.
9
0
6
0
.
9
4
0
0
.
9
4
1
0
.
9
4
0
0
.
9
4
0
0
.
9
4
0
0
.
9
4
1
0
.
9
4
0
0
.
9
4
0
G
l
a
ss
0
.
6
8
2
0
.
5
1
7
0
.
5
4
0
0
.
5
1
4
0
.
7
6
5
0
.
7
7
4
0
.
7
7
1
0
.
7
5
6
0
.
7
8
5
0
.
7
9
8
0
.
7
8
6
0
.
7
7
8
E.
c
o
l
i
0
.
8
6
0
0
.
8
0
7
0
.
7
6
7
0
.
7
7
0
0
.
8
9
9
0
.
9
0
4
0
.
9
0
0
0
.
8
9
7
0
.
9
0
9
0
.
9
1
4
0
.
9
1
0
0
.
9
0
7
P
e
n
d
i
g
i
t
s
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
0
.
9
9
5
Y
e
a
st
0
.
5
9
8
0
.
5
7
9
0
.
5
3
8
0
.
5
4
1
0
.
6
6
8
0
.
6
9
9
0
.
6
6
8
0
.
6
7
1
0
.
6
6
8
0
.
6
9
9
0
.
6
6
8
0
.
6
7
1
5
.
3
.
Resul
t
s
o
bta
ined f
ro
m
u
s
ing
t
he
ra
nd
o
m
f
o
re
s
t
ens
em
ble
t
o
co
ns
t
ruct
t
he
hy
brid m
o
del
T
h
is
s
ec
tio
n
illu
s
tr
ates
an
d
d
escr
ib
es
th
e
ex
p
er
im
en
tal
r
esu
lts
p
r
o
d
u
ce
d
wh
en
th
e
R
F
en
s
em
b
le
class
if
ier
was
u
til
ized
as
th
e
b
ase
class
if
ier
to
g
en
er
ate
th
e
s
u
g
g
ested
h
y
b
r
id
m
o
d
el.
T
h
e
r
esu
lts
ar
e
tab
u
lated
in
T
ab
le
5
.
L
ik
e
th
e
ca
s
e
o
f
DT
an
d
SVM
class
if
ier
s
,
B
o
o
s
ted
-
OVO&
SMOT
E
u
s
in
g
R
F
as
b
ase
cla
s
s
if
ier
s
p
r
o
d
u
ce
d
th
e
b
est
F1
-
s
co
r
e
f
o
r
m
o
s
t
d
atasets
.
Mo
r
eo
v
e
r
,
em
p
lo
y
in
g
R
F
as
th
e
b
ase
class
if
ier
r
esu
lted
in
f
u
r
th
er
p
er
f
o
r
m
a
n
ce
im
p
r
o
v
e
m
en
ts
.
Mo
r
e
s
p
ec
if
ically
,
it
a
ch
iev
ed
t
h
e
h
ig
h
est
F1
-
s
co
r
e
f
o
r
s
ix
teen
o
u
t
o
f
t
h
e
s
ev
en
teen
d
atasets
co
n
s
id
er
ed
in
th
e
in
v
esti
g
atio
n
,
alth
o
u
g
h
f
o
r
f
o
u
r
o
f
th
o
s
e
d
atasets
,
th
e
OVO&
SMOT
E
m
o
d
el
ac
h
iev
e
d
th
e
s
am
e
s
co
r
e.
No
w,
to
s
h
o
w
th
at
B
o
o
s
ted
-
OVO&
SMOT
E
s
ig
n
if
ican
tly
o
u
tp
er
f
o
r
m
s
OVO
an
d
OVO
&
SMOT
E
h
y
b
r
id
m
o
d
els
th
e
Frie
d
m
an
s
tati
s
tical
s
ig
n
if
ican
ce
test
[
2
8
]
was
ap
p
lied
.
Acc
o
r
d
in
g
to
Frie
d
m
an
tes
t
s
tatis
t
ics,
th
er
e
is
a
s
ig
n
if
ic
an
t
d
if
f
e
r
en
ce
i
n
p
e
r
f
o
r
m
an
c
e
am
o
n
g
th
e
h
y
b
r
id
m
o
d
els
(
X
2
(
2
)
=
4
.
5
0
0
0
,
p
=
0
.
0
0
0
0
0
)
.
Acc
o
r
d
in
g
ly
,
t
h
e
Nem
en
y
i
p
o
s
t
-
h
o
c
test
[
2
9
]
was
em
p
lo
y
e
d
to
id
e
n
tify
th
e
s
u
p
er
io
r
h
y
b
r
id
m
o
d
el.
Fig
u
r
e
2
d
is
p
lay
s
th
e
o
u
tp
u
t
o
f
t
h
e
Nem
en
y
i
p
o
s
t
-
h
o
c
test
.
No
te
h
er
e
th
at
to
c
o
n
s
id
er
o
n
e
m
o
d
el
s
ig
n
if
ican
tly
ex
ce
ed
s
an
o
th
er
,
th
e
d
if
f
er
e
n
ce
b
etwe
en
th
eir
c
alcu
lated
av
er
ag
e
r
an
k
s
s
h
o
u
l
d
b
e
g
r
ea
ter
th
a
n
o
r
eq
u
al
to
a
cr
itical
d
if
f
e
r
en
ce
(
CD
)
.
Fro
m
th
e
f
ig
u
r
e,
b
o
th
B
o
o
s
ted
-
OVO&
SMOT
E
an
d
OVO&
SMOT
E
m
o
d
els
p
er
f
o
r
m
b
etter
th
a
n
th
e
OVO
m
o
d
el,
f
u
r
th
e
r
,
th
e
B
o
o
s
ted
-
OVO&
SMOT
E
h
y
b
r
id
m
o
d
el
s
ig
n
if
ican
tly
o
u
tp
er
f
o
r
m
s
th
e
OVO&
SMOT
E
h
y
b
r
id
m
o
d
el.
T
ab
le
5
.
R
esu
lts
o
b
tain
ed
f
r
o
m
u
s
in
g
th
e
R
F e
n
s
em
b
le
as th
e
b
ase
c
lass
if
ier
D
a
t
a
s
e
t
OVO
O
V
O
&S
M
O
TE
B
o
o
st
e
d
-
O
V
O
&S
M
O
TE
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
c
c
.
P
r
e
c
.
R
e
c
.
F1
A
b
a
l
o
n
e
0
.
7
5
7
0
.
5
9
8
0
.
7
2
1
0
.
6
1
4
0
.
8
6
9
0
.
8
6
8
0
.
8
6
9
0
.
8
6
8
0
.
8
7
1
0
.
8
7
0
0
.
8
7
1
0
.
8
7
0
C
o
n
t
r
a
c
e
p
t
i
v
e
0
.
5
1
1
0
.
4
8
4
0
.
4
9
6
0
.
4
8
5
0
.
6
2
1
0
.
6
1
9
0
.
6
2
1
0
.
6
1
8
0
.
6
2
9
0
.
6
2
7
0
.
6
2
8
0
.
6
2
6
H
a
y
e
s
R
o
t
h
0
.
8
0
6
0
.
8
5
3
0
.
8
3
9
0
.
8
3
6
0
.
8
3
7
0
.
8
4
4
0
.
8
1
9
0
.
8
1
0
0
.
8
4
7
0
.
8
6
4
0
.
8
4
7
0
.
8
2
7
P
o
st
-
o
p
e
r
a
t
i
v
e
0
.
6
7
3
0
.
4
8
0
0
.
5
5
6
0
.
5
0
6
0
.
8
1
8
0
.
8
3
0
0
.
8
2
4
0
.
8
1
5
0
.
8
2
4
0
.
8
4
3
0
.
8
3
1
0
.
8
2
1
Th
y
r
o
i
d
0
.
9
6
3
0
.
9
7
5
0
.
9
3
2
0
.
9
4
2
0
.
9
8
9
0
.
9
9
0
0
.
9
8
7
0
.
9
8
8
0
.
9
8
9
0
.
9
9
0
0
.
9
8
7
0
.
9
8
8
V
e
r
t
e
b
r
a
l
0
.
8
4
2
0
.
8
1
2
0
.
7
9
0
0
.
7
8
9
0
.
9
0
7
0
.
9
0
8
0
.
9
0
9
0
.
9
0
6
0
.
9
1
1
0
.
9
1
3
0
.
9
1
2
0
.
9
0
9
V
e
h
i
c
l
e
0
.
7
6
6
0
.
7
6
4
0
.
7
6
8
0
.
7
6
2
0
.
7
5
2
0
.
7
4
3
0
.
7
5
4
0
.
7
4
1
0
.
7
7
2
0
.
7
6
3
0
.
7
7
2
0
.
7
6
3
Car
0
.
8
5
8
0
.
7
9
2
0
.
7
6
6
0
.
7
4
3
0
.
9
9
6
0
.
9
9
6
0
.
9
9
6
0
.
9
9
6
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
H
e
a
r
t
0
.
5
5
6
0
.
2
4
1
0
.
2
6
2
0
.
2
4
2
0
.
8
8
3
0
.
8
8
6
0
.
8
8
8
0
.
8
8
3
0
.
8
8
3
0
.
8
8
6
0
.
8
8
8
0
.
8
8
3
N
u
r
sery
0
.
8
1
8
0
.
8
4
6
0
.
7
8
6
0
.
7
7
4
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
0
.
9
9
8
P
a
g
e
b
l
o
c
k
s
0
.
9
6
8
0
.
8
9
0
0
.
8
2
5
0
.
8
3
7
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
D
e
r
mat
o
l
o
g
y
0
.
9
7
8
0
.
9
7
9
0
.
9
7
4
0
.
9
7
5
0
.
9
8
9
0
.
9
8
9
0
.
9
9
1
0
.
9
8
9
0
.
9
9
1
0
.
9
9
2
0
.
9
9
1
0
.
9
9
1
D
r
y
b
e
a
n
0
.
7
3
0
0
.
7
8
3
0
.
7
8
2
0
.
7
5
1
0
.
9
5
7
0
.
9
5
7
0
.
9
5
7
0
.
9
5
7
0
.
9
5
8
0
.
9
5
8
0
.
9
5
8
0
.
9
5
8
G
l
a
ss
0
.
7
9
1
0
.
7
0
0
0
.
7
3
5
0
.
7
0
6
0
.
9
2
3
0
.
9
2
7
0
.
9
3
0
0
.
9
2
2
0
.
9
2
5
0
.
9
2
6
0
.
9
3
0
0
.
9
2
3
E.
c
o
l
i
0
.
8
6
6
0
.
7
7
6
0
.
7
5
9
0
.
7
5
3
0
.
9
6
0
0
.
9
5
9
0
.
9
6
0
0
.
9
5
8
0
.
9
6
2
0
.
9
6
2
0
.
9
6
3
0
.
9
6
0
P
e
n
d
i
g
i
t
s
0
.
9
8
8
0
.
9
8
8
0
.
9
8
8
0
.
9
8
8
0
.
9
9
0
0
.
9
9
0
0
.
9
9
0
0
.
9
9
0
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
0
.
9
9
4
Y
e
a
st
0
.
5
8
8
0
.
5
2
9
0
.
4
9
9
0
.
5
0
0
0
.
8
8
9
0
.
8
9
0
0
.
8
8
9
0
.
8
8
8
0
.
8
9
0
0
.
8
9
1
0
.
8
9
0
0
.
8
8
9
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2252
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
14
,
No
.
5
,
Octo
b
er
20
25
:
3
9
8
2
-
3
9
9
3
3990
Fig
u
r
e
2
.
T
h
e
r
esu
lt o
f
th
e
Ne
m
en
y
i p
o
s
t
-
h
o
c
test
f
o
r
c
o
m
p
a
r
in
g
OVO
m
o
d
el,
OVO
&
SMOT
E
m
o
d
el
,
a
n
d
B
o
o
s
ted
-
OVO&
SMOT
E
m
o
d
el
5
.
4
.
Co
m
pa
ring
t
he
perf
o
r
ma
nce
o
f
t
he
cla
s
s
if
iers ut
ilize
d t
o
g
ener
a
t
e
t
he
hy
brid m
o
del
T
h
is
s
ec
tio
n
p
r
esen
ts
a
co
m
p
ar
is
o
n
am
o
n
g
th
e
b
ase
class
if
ier
s
u
s
ed
to
g
en
er
ate
th
e
d
esire
d
h
y
b
r
id
m
o
d
el.
Fig
u
r
e
3
d
is
p
lay
s
t
h
e
p
er
f
o
r
m
an
ce
co
m
p
ar
is
o
n
in
ter
m
s
o
f
F1
-
s
co
r
e
f
o
r
t
h
e
th
r
ee
c
o
n
s
id
er
ed
class
if
ier
s
:
i)
DT
,
ii)
SVM
,
an
d
iii)
R
F
class
if
ier
s
.
Fro
m
th
e
f
ig
u
r
e
,
it
is
o
b
v
io
u
s
th
at
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
R
F
h
y
b
r
id
m
o
d
el
o
u
t
p
er
f
o
r
m
s
DT
an
d
SVM
h
y
b
r
id
m
o
d
els
f
o
r
t
h
e
m
o
s
t
co
n
s
id
er
ed
d
atasets
.
Mo
r
e
s
p
ec
if
ically
,
th
e
R
F
h
y
b
r
id
m
o
d
el
ac
h
iev
e
d
th
e
b
est
F1
-
s
co
r
e
f
o
r
f
if
teen
d
atasets
,
wh
ile
th
e
DT
h
y
b
r
id
m
o
d
el
g
en
er
ated
th
e
b
est
F1
-
s
co
r
e
f
o
r
two
d
atasets
(
ca
r
an
d
n
u
r
s
er
y
)
.
No
te
th
at
th
e
s
am
e
r
esu
lt
was
o
b
tain
ed
f
o
r
th
e
c
ar
d
ataset
u
s
in
g
th
e
DT
an
d
R
F
h
y
b
r
id
m
o
d
els.
Fo
r
o
n
ly
o
n
e
d
ataset
(
p
en
d
ig
its
)
,
th
e
SVM
h
y
b
r
id
m
o
d
el
p
r
o
d
u
ce
d
th
e
b
est
F1
-
s
co
r
e.
T
h
er
ef
o
r
e,
th
e
ad
o
p
ted
cla
s
s
if
ier
ca
n
s
ig
n
if
ican
tly
in
f
lu
en
ce
th
e
o
v
er
all
ef
f
ec
tiv
en
ess
o
f
t
h
e
h
y
b
r
id
m
o
d
el.
T
o
co
n
d
u
ct
a
p
r
ec
is
e
co
m
p
ar
is
o
n
,
th
e
Frie
d
m
an
test
was
ad
o
p
ted
an
d
r
ep
o
r
ted
a
s
ig
n
if
ican
t d
i
f
f
er
en
ce
am
o
n
g
th
e
co
n
s
id
er
ed
m
o
d
els (
X
2
(
2
)
=
2
1
.
5
2
2
4
,
p
=
0
.
0
0
0
0
2
)
.
T
h
er
ef
o
r
e,
th
e
Nem
en
y
i
p
o
s
t
-
h
o
c
test
was
e
m
p
lo
y
ed
to
h
ig
h
lig
h
t
t
h
e
s
u
p
e
r
io
r
h
y
b
r
id
m
o
d
el.
Fig
u
r
e
4
s
u
m
m
ar
izes
th
e
o
u
tp
u
t
o
f
th
e
Nem
e
n
y
i
p
o
s
t
-
h
o
c
test
.
Fro
m
th
e
f
ig
u
r
e,
th
e
R
F
h
y
b
r
id
m
o
d
el
s
ig
n
if
ican
tly
o
u
tp
er
f
o
r
m
s
b
o
t
h
DT
an
d
SVM
h
y
b
r
id
m
o
d
els.
I
n
ad
d
i
tio
n
,
n
o
s
ig
n
if
ica
n
t
d
if
f
er
en
c
e
b
etwe
en
t
h
e
DT
h
y
b
r
id
m
o
d
el
an
d
th
e
SVM
h
y
b
r
id
m
o
d
el
(
co
n
n
ec
ted
m
o
d
els
in
th
e
f
ig
u
r
e
in
d
icate
n
o
s
ig
n
if
ican
t
d
if
f
er
e
n
ce
)
.
I
n
s
u
m
m
ar
y
,
u
tili
zin
g
R
F
t
o
g
en
er
ate
th
e
d
esire
d
h
y
b
r
id
m
o
d
el
p
r
o
v
id
es
clea
r
e
v
id
en
ce
t
h
at
ad
o
p
tin
g
en
s
em
b
le
class
if
icatio
n
im
p
r
o
v
es
th
e
class
if
icatio
n
ef
f
ec
tiv
en
ess
f
o
r
th
e
h
y
b
r
id
m
o
d
el.
Fig
u
r
e
3
.
C
o
m
p
a
r
in
g
t
h
e
p
er
f
o
r
m
an
ce
o
f
th
e
th
r
ee
co
n
s
id
er
e
d
class
if
ier
s
u
tili
ze
d
to
g
en
er
at
e
th
e
h
y
b
r
id
m
o
d
el
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
F1
-
sc
o
r
e
E
v
al
u
ati
o
n
D
atase
ts
DT
H
yb
ri
d
M
o
d
e
l
S
V
M Hy
b
ri
d
Mo
d
e
l
R
F
H
yb
ri
d
Mo
d
e
l
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
h
a
n
d
lin
g
t
h
e
imb
a
la
n
ce
d
mu
lticla
s
s
cla
s
s
ifica
tio
n
p
r
o
b
lem
(
E
s
r
a
’
a
A
ls
h
d
a
ifa
t)
3991
Fig
u
r
e
4
.
T
h
e
r
esu
lt o
f
th
e
Ne
m
en
y
i p
o
s
t
-
h
o
c
test
f
o
r
c
o
m
p
a
r
in
g
: RF
h
y
b
r
id
m
o
d
el,
DT
h
y
b
r
id
m
o
d
el
,
an
d
SVM
h
y
b
r
id
m
o
d
el
6.
CO
NCLU
SI
O
N
I
n
th
is
p
ap
er
,
a
n
o
v
el
s
o
lu
ti
o
n
to
th
e
well
-
k
n
o
wn
im
b
al
an
ce
d
m
u
lticlas
s
clas
s
if
icatio
n
p
r
o
b
lem
b
elo
n
g
in
g
to
th
e
h
y
b
r
i
d
m
eth
o
d
s
ca
teg
o
r
y
is
p
r
esen
te
d
an
d
illu
s
tr
ated
.
T
h
e
p
r
im
ar
y
id
ea
is
to
co
m
b
in
e
f
o
u
r
p
o
wer
f
u
l
m
et
h
o
d
s
f
o
r
h
a
n
d
lin
g
im
b
alan
ce
d
m
u
lticlas
s
class
if
icatio
n
to
co
n
s
tr
u
ct
a
h
ig
h
-
p
er
f
o
r
m
a
n
ce
h
y
b
r
id
m
o
d
el.
T
h
e
ex
am
in
e
d
m
eth
o
d
s
in
clu
d
e:
th
e
well
-
k
n
o
wn
SMOT
E
d
ata
-
lev
el
m
eth
o
d
,
th
e
d
ec
o
m
p
o
s
itio
n
m
eth
o
d
,
th
e
e
n
s
em
b
le
m
et
h
o
d
a
n
d
t
h
e
b
o
o
s
tin
g
m
eth
o
d
.
R
eg
ar
d
i
n
g
t
h
e
en
s
em
b
le
an
d
d
ec
o
m
p
o
s
itio
n
m
eth
o
d
s
,
th
ese
wer
e
ac
h
iev
e
d
th
r
o
u
g
h
th
e
OVO
ap
p
r
o
ac
h
,
wh
ich
in
v
o
lv
es
d
ec
o
m
p
o
s
in
g
th
e
m
u
lticlas
s
p
r
o
b
lem
in
to
m
u
ltip
le
b
in
a
r
y
p
r
o
b
lem
s
an
d
co
n
s
tr
u
ctin
g
a
tailo
r
ed
class
if
ier
f
o
r
ea
c
h
b
in
ar
y
p
r
o
b
lem
.
C
o
n
ce
r
n
in
g
th
e
b
o
o
s
tin
g
m
eth
o
d
,
th
e
id
ea
was
to
id
en
tify
th
e
less
ef
f
ec
tiv
e
cla
s
s
if
ier
s
an
d
b
o
o
s
t
th
em
u
s
in
g
th
e
b
o
o
ts
tr
ap
m
eth
o
d
.
C
o
n
s
eq
u
en
tly
,
o
u
r
h
y
b
r
id
m
o
d
el
is
r
e
f
er
r
ed
t
o
as
B
o
o
s
ted
-
OVO&
SMOT
E
.
Acc
o
r
d
in
g
to
th
e
f
in
d
in
g
s
,
th
e
B
o
o
s
ted
-
OVO&
SMOT
E
h
y
b
r
id
m
o
d
el
s
ig
n
if
ican
tly
o
u
tp
er
f
o
r
m
s
th
e
co
n
v
en
tio
n
al
OVO
m
o
d
el.
Mo
r
e
o
v
er
,
th
e
s
u
g
g
ested
m
o
d
el
im
p
r
o
v
ed
class
if
icatio
n
ef
f
ec
tiv
en
ess
o
r
p
r
eser
v
ed
th
e
s
am
e
p
er
f
o
r
m
an
ce
wh
en
co
m
p
a
r
ed
with
th
e
OVO&
SMOT
E
m
o
d
el,
th
is
in
d
icate
s
th
at
th
e
m
o
d
el
is
ef
f
ec
tiv
e
in
s
p
o
ttin
g
th
e
class
if
ier
s
th
at
r
e
q
u
ir
e
b
o
o
s
tin
g
.
I
n
o
th
er
wo
r
d
s
,
th
e
s
u
g
g
ested
m
o
d
el
will
p
r
o
d
u
ce
b
etter
r
esu
lts
wh
en
th
e
b
i
n
ar
y
class
if
ier
s
with
in
th
e
OVO
in
clu
d
e
r
ela
tiv
ely
“lo
w
-
p
er
f
o
r
m
a
n
ce
”
cla
s
s
if
ier
s
.
Mo
r
eo
v
er
,
u
tili
zin
g
th
e
R
F
en
s
em
b
le
class
if
ier
as
a
b
ase
cla
s
s
if
ier
s
ig
n
if
ican
tly
en
h
an
ce
s
th
e
o
v
er
all
p
er
f
o
r
m
an
ce
co
m
p
ar
ed
to
u
s
in
g
s
in
g
le
clas
s
if
ier
s
.
N
o
te
th
at
th
e
r
esu
ltin
g
m
o
d
el
is
a
f
o
r
m
o
f
a
n
en
s
e
m
b
le
o
f
en
s
em
b
les.
T
h
r
ee
k
ey
d
ir
ec
tio
n
s
ca
n
b
e
co
n
s
id
er
e
d
f
o
r
f
u
tu
r
e
wo
r
k
.
T
h
e
f
ir
s
t
d
ir
ec
tio
n
f
o
cu
s
es
o
n
en
h
an
cin
g
th
e
s
ca
lab
ilit
y
o
f
th
e
s
u
g
g
ested
h
y
b
r
id
m
o
d
el
to
ad
d
r
ess
b
ig
d
atasets
in
ter
m
s
o
f
th
e
n
u
m
b
er
o
f
in
s
tan
ce
s
an
d
class
es.
T
h
e
s
ec
o
n
d
d
ir
ec
tio
n
co
n
ce
n
tr
ates
o
n
in
v
esti
g
atin
g
m
o
r
e
ap
p
licatio
n
d
o
m
ai
n
s
to
ev
alu
ate
th
e
g
en
er
aliza
tio
n
o
f
t
h
e
h
y
b
r
i
d
m
o
d
el.
T
h
e
th
ir
d
d
ir
ec
tio
n
f
o
cu
s
es
o
n
r
e
d
u
cin
g
th
e
m
o
d
el
co
m
p
lex
ity
b
y
ex
p
lo
r
in
g
th
e
in
teg
r
atio
n
o
f
p
r
u
n
in
g
tec
h
n
iq
u
es.
F
UNDING
I
NF
O
R
M
A
T
I
O
N
Au
th
o
r
s
s
tate
n
o
f
u
n
d
in
g
in
v
o
lv
ed
.
AUTHO
R
CO
NT
RI
B
UT
I
O
NS ST
A
T
E
M
E
N
T
T
h
is
jo
u
r
n
al
u
s
es
th
e
C
o
n
tr
ib
u
to
r
R
o
les
T
ax
o
n
o
m
y
(
C
R
ed
iT
)
to
r
ec
o
g
n
ize
in
d
iv
id
u
al
au
th
o
r
co
n
tr
ib
u
tio
n
s
,
r
ed
u
ce
au
th
o
r
s
h
ip
d
is
p
u
tes,
an
d
f
ac
ilit
ate
co
llab
o
r
atio
n
.
Na
m
e
o
f
Aut
ho
r
C
M
So
Va
Fo
I
R
D
O
E
Vi
Su
P
Fu
E
s
r
a
'
a
Al
s
h
d
aif
at
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Fair
o
u
z
Hu
s
s
ein
✓
✓
✓
✓
✓
✓
Ala
'
a
A
l
-
s
h
d
aif
at
✓
✓
✓
✓
✓
✓
Ma
lak
Al
-
Hass
an
✓
✓
✓
✓
✓
E
n
s
h
ir
ah
Altar
awn
eh
✓
✓
✓
✓
✓
C
:
C
o
n
c
e
p
t
u
a
l
i
z
a
t
i
o
n
M
:
M
e
t
h
o
d
o
l
o
g
y
So
:
So
f
t
w
a
r
e
Va
:
Va
l
i
d
a
t
i
o
n
Fo
:
Fo
r
mal
a
n
a
l
y
s
i
s
I
:
I
n
v
e
s
t
i
g
a
t
i
o
n
R
:
R
e
so
u
r
c
e
s
D
:
D
a
t
a
C
u
r
a
t
i
o
n
O
:
W
r
i
t
i
n
g
-
O
r
i
g
i
n
a
l
D
r
a
f
t
E
:
W
r
i
t
i
n
g
-
R
e
v
i
e
w
&
E
d
i
t
i
n
g
Vi
:
Vi
su
a
l
i
z
a
t
i
o
n
Su
:
Su
p
e
r
v
i
s
i
o
n
P
:
P
r
o
j
e
c
t
a
d
mi
n
i
st
r
a
t
i
o
n
Fu
:
Fu
n
d
i
n
g
a
c
q
u
i
si
t
i
o
n
Evaluation Warning : The document was created with Spire.PDF for Python.