I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
1
5
,
No
.
3
,
J
u
n
e
2
0
2
6
,
p
p
.
2920
~
2
9
3
4
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
3
.
p
p
2
9
2
0
-
2
9
3
4
2920
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
Cla
ss
imba
la
nce r
eso
lution in Io
T n
etworks us
ing
ad
v
a
nced
elk
herd
o
ptimi
za
tion
wit
h
SMOTE
and iterativ
ely
fine
-
t
uned de
ep
BiL
STM
Srik
a
nth
M
ud
iy
a
nu
r
Srira
m
a
pp
a
1
,
Ana
nd
a
B
a
bu
J
a
y
a
cha
nd
ra
1
,
Va
s
a
ntha
K
um
a
ra
M
a
ha
dev
a
cha
r
2
,
Ash
wi
ni K
a
ila
s
3
,
T
.
G
.
K
ee
rt
ha
n K
um
a
r
4
1
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
o
n
S
c
i
e
n
c
e
a
n
d
E
n
g
i
n
e
e
r
i
n
g
,
M
a
l
n
a
d
C
o
l
l
e
g
e
o
f
E
n
g
i
n
e
e
r
i
n
g
a
f
f
i
l
i
a
t
e
d
t
o
V
i
sv
e
s
v
a
r
a
y
a
Te
c
h
n
o
l
o
g
i
c
a
l
U
n
i
v
e
r
si
t
y
,
B
e
l
a
g
a
v
i
,
I
n
d
i
a
2
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
a
n
d
E
n
g
i
n
e
e
r
i
n
g
,
G
o
v
e
r
n
me
n
t
E
n
g
i
n
e
e
r
i
n
g
C
o
l
l
e
g
e
a
f
f
i
l
i
a
t
e
d
t
o
V
i
s
v
e
s
v
a
r
a
y
a
Te
c
h
n
o
l
o
g
i
c
a
l
U
n
i
v
e
r
si
t
y
,
B
e
l
a
g
a
v
i
,
I
n
d
i
a
3
D
e
p
a
r
t
me
n
t
o
f
B
i
o
M
e
d
i
c
a
l
En
g
i
n
e
e
r
i
n
g
,
S
r
i
S
i
d
d
h
a
r
t
h
a
I
n
st
i
t
u
t
e
o
f
T
e
c
h
n
o
l
o
g
y
a
f
f
i
l
i
a
t
e
d
t
o
S
r
i
S
i
d
d
h
a
r
t
h
a
A
c
a
d
e
m
y
o
f
H
i
g
h
e
r
Ed
u
c
a
t
i
o
n
,
Tu
m
k
u
r
,
I
n
d
i
a
4
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
o
n
S
c
i
e
n
c
e
a
n
d
E
n
g
i
n
e
e
r
i
n
g
,
S
i
d
d
a
g
a
n
g
a
I
n
st
i
t
u
t
e
o
f
Te
c
h
n
o
l
o
g
y
a
f
f
i
l
i
a
t
e
d
t
o
V
i
s
v
e
s
v
a
r
a
y
a
Te
c
h
n
o
l
o
g
i
c
a
l
U
n
i
v
e
r
si
t
y
,
B
e
l
a
g
a
v
i
,
I
n
d
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Oct
1
,
2
0
2
4
R
ev
is
ed
Feb
2
1
,
2
0
2
6
Acc
ep
ted
Ap
r
2
0
,
2
0
2
6
Th
e
"
i
n
tern
e
t
o
f
th
i
n
g
s
(I
o
T)
"
m
e
n
ti
o
n
s
t
o
a
sy
ste
m
in
w
h
ich
m
u
lt
ip
le
n
e
two
rk
p
r
o
t
o
c
o
ls
a
re
u
se
d
to
li
n
k
to
g
e
t
h
e
r
d
isp
a
ra
te
d
e
v
ice
s
th
a
t
a
re
a
lwa
y
s
sh
a
rin
g
i
n
fo
rm
a
ti
o
n
wi
th
o
n
e
a
n
o
th
e
r.
Th
e
h
o
p
e
is
t
h
a
t
t
h
is
st
u
d
y
will
a
d
d
to
th
e
e
x
isti
n
g
b
o
d
y
o
f
k
n
o
wle
d
g
e
b
y
su
g
g
e
sti
n
g
wa
y
s
to
m
a
k
e
in
tr
u
sio
n
d
e
tec
ti
o
n
sy
ste
m
s
(IDS)
m
o
re
e
f
fe
c
ti
v
e
.
Train
i
n
g
d
a
tas
e
ts
fo
r
th
e
p
ro
p
o
se
d
m
o
d
e
l
we
re
c
o
ll
e
c
ted
fro
m
tele
m
e
try
o
f
n
e
tw
o
rk
(T
o
N)
-
Io
T
n
e
two
rk
traffic.
Afte
r
c
lea
n
in
g
a
n
d
n
o
rm
a
li
z
in
g
th
e
d
a
tas
e
ts,
th
e
sy
n
t
h
e
ti
c
m
in
o
rit
y
o
v
e
r
-
sa
m
p
li
n
g
tec
h
n
iq
u
e
(S
M
O
TE
)
i
s
u
se
d
to
b
a
lan
c
e
t
h
e
d
a
tas
e
ts
th
a
t
a
re
imb
a
lan
c
e
d
.
Op
t
ima
l
sa
m
p
li
n
g
ra
tes
a
re
c
rit
ica
l
fo
r
re
so
l
v
in
g
c
las
s
imb
a
lan
c
e
,
a
s
S
M
OTE'
s
e
fficie
n
c
y
is
d
e
p
e
n
d
e
n
t
o
n
it
fo
r
i
n
sta
n
c
e
s
in
v
o
lv
i
n
g
m
in
o
rit
y
c
las
se
s.
Im
p
ro
v
in
g
c
las
sifica
ti
o
n
a
c
c
u
ra
c
y
th
ro
u
g
h
fin
d
in
g
a
p
p
ro
p
riate
sa
m
p
le
ra
tes
fo
r
in
p
u
t
d
a
tas
e
ts
is
th
e
g
o
a
l
o
f
th
i
s
p
a
p
e
r'
s
in
tro
d
u
c
ti
o
n
o
f
a
d
v
a
n
c
e
d
e
lk
h
e
rd
o
p
t
imiz
a
ti
o
n
(AEHO
)
wit
h
S
M
OTE.
F
in
a
ll
y
,
a
d
e
e
p
b
id
irec
ti
o
n
a
l
l
o
n
g
sh
o
rt
-
term
m
e
m
o
ry
(
d
e
e
p
Bi
LS
T
M
)
m
o
d
e
l
b
a
se
d
o
n
d
e
e
p
lea
rn
i
n
g
is
u
se
d
to
c
las
sify
a
tt
a
c
k
s.
Th
e
fi
n
e
-
tu
n
in
g
tec
h
n
iq
u
e
is
u
se
d
d
u
ri
n
g
tes
ti
n
g
t
o
u
p
d
a
te
t
h
e
h
ig
h
li
m
it
s
a
n
d
w
h
e
n
c
o
m
b
i
n
e
d
with
t
h
e
d
a
ta
b
a
lan
c
in
g
m
e
c
h
a
n
ism
a
n
d
AEHO
-
b
a
se
d
-
S
M
OTE,
t
h
e
re
su
lt
s
g
re
a
tl
y
e
n
h
a
n
c
e
th
e
c
las
sifica
ti
o
n
tec
h
n
i
q
u
e
s.
De
e
p
B
iL
S
T
M
p
e
rfo
rm
s
b
e
tt
e
r
th
a
n
9
0
%
in
e
v
e
ry
c
a
teg
o
ry
:
c
las
sifica
ti
o
n
,
re
c
a
ll
,
p
re
c
isio
n
,
a
n
d
F
1
-
sc
o
r
e
.
K
ey
w
o
r
d
s
:
Ad
v
an
ce
d
el
k
h
er
d
o
p
tim
izati
o
n
Dee
p
B
iLST
M
I
n
f
o
r
m
atio
n
s
h
ar
in
g
I
n
ter
n
et
o
f
th
in
g
s
I
n
tr
u
s
io
n
d
etec
tio
n
s
y
s
tem
SMOT
E
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Srik
an
th
Mu
d
iy
a
n
u
r
Srir
am
ap
p
a
Dep
ar
tm
en
t o
f
I
n
f
o
r
m
atio
n
Scien
ce
an
d
E
n
g
in
ee
r
i
n
g
Ma
ln
ad
C
o
lleg
e
o
f
E
n
g
in
ee
r
i
n
g
af
f
iliated
to
Vis
v
esv
ar
ay
a
T
ec
h
n
o
lo
g
ical
Un
iv
er
s
ity
B
elag
av
i,
I
n
d
ia
E
m
ail:
s
r
ik
an
th
is
e@
g
m
ail.
co
m
1.
I
NT
RO
D
UCT
I
O
N
I
n
tellig
en
t
tech
n
o
lo
g
ies
th
at
en
ab
le
m
ac
h
in
e
-
to
-
m
ac
h
in
e
i
n
ter
ac
tio
n
ar
e
b
ac
k
b
o
n
e
o
f
I
n
d
u
s
tr
y
4
.
0
,
wh
ich
aim
s
to
d
ig
italize
tr
a
d
itio
n
al
in
d
u
s
tr
ial
p
r
o
d
u
ctio
n
an
d
wo
r
k
in
g
p
r
o
ce
d
u
r
es.
Ad
v
an
ce
d
in
ter
n
et
o
f
th
in
g
s
(
I
o
T
)
tech
n
iq
u
es
an
d
i
n
s
tr
u
m
en
ts
au
to
m
ate
all
p
r
o
d
u
ctio
n
,
j
o
b
p
r
o
c
ess
in
g
,
an
d
o
p
er
atio
n
p
r
o
ce
s
s
es
in
in
d
u
s
tr
ial
au
to
m
atio
n
[
1
]
.
T
h
e
ter
m
"I
I
o
T
"
d
escr
ib
es
a
m
e
th
o
d
o
f
u
s
in
g
th
e
I
o
T
f
o
r
in
d
u
s
tr
ial
au
to
m
atio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
la
s
s
imb
a
la
n
ce
r
eso
lu
tio
n
in
I
o
T n
etw
o
r
ks u
s
in
g
a
d
va
n
ce
d
elk
h
erd
…
(
S
r
ika
n
th
Mu
d
iya
n
u
r
S
r
ir
a
ma
p
p
a
)
2921
B
o
th
m
ac
h
in
e
-
to
-
m
ac
h
in
e
(
M
-
M
)
an
d
h
u
m
a
n
-
m
ac
h
in
e
(
H
-
M
)
co
m
m
u
n
icatio
n
s
ar
e
e
n
co
m
p
ass
ed
b
y
th
is
n
o
tio
n
o
f
in
d
u
s
tr
ial
au
to
m
atio
n
.
H
ea
lth
ca
r
e,
m
an
u
f
ac
tu
r
i
n
g
,
th
e
s
u
p
p
ly
ch
ain
,
an
d
m
an
u
f
ac
tu
r
er
s
ar
e
am
o
n
g
th
e
m
an
y
in
d
u
s
tr
ies
th
at
h
a
v
e
r
ec
en
tly
s
h
o
wn
a
m
a
r
k
ed
i
n
cr
ea
s
e
in
t
h
eir
a
d
o
p
tio
n
o
f
I
o
T
tech
n
o
lo
g
y
[
2
]
.
Vir
tu
aliza
tio
n
,
au
to
m
atio
n
,
le
ar
n
in
g
m
et
h
o
d
s
,
a
r
tific
ial
in
tellig
en
ce
ap
p
r
o
ac
h
es,
an
d
th
e
I
o
T
ar
e
s
o
m
e
o
f
th
e
m
o
s
t
u
tili
ze
d
to
o
ls
in
i
n
d
u
s
tr
y
4
.
0
in
f
r
astru
ctu
r
es.
T
h
e
n
e
west
in
n
o
v
atio
n
s
ar
e
in
teg
r
at
ed
in
to
s
ev
er
al
ar
ea
s
o
f
s
m
ar
t
m
a
n
u
f
ac
tu
r
in
g
a
n
d
co
m
m
u
n
iq
u
é
i
n
i
n
d
u
s
tr
y
4
.
0
[
3
]
.
I
n
f
o
s
ec
u
r
ity
is
co
n
s
t
an
tly
d
em
a
n
d
in
g
in
i
n
d
u
s
tr
y
4
.
0
s
in
ce
d
ata
in
all
o
f
th
ese
s
ec
to
r
s
is
av
ailab
le
o
n
d
is
tan
t
p
latf
o
r
m
s
o
r
clo
u
d
s
er
v
er
s
,
wh
ic
h
ca
n
b
e
tar
g
eted
b
y
i
n
tr
u
s
io
n
ass
au
lts
.
T
h
e
in
cr
ea
s
ed
s
ec
u
r
ity
r
is
k
is
a
r
esu
lt
o
f
I
n
d
u
s
tr
y
4
.
0
'
s
r
elia
n
ce
o
n
d
e
v
ices
an
d
n
etwo
r
k
s
f
o
r
co
m
m
u
n
iq
u
é
an
d
d
ata
tr
an
s
m
is
s
io
n
[
4
]
.
An
y
s
m
ar
t d
ev
ice
lin
k
ed
to
n
etwo
r
k
p
r
o
v
id
es a
p
o
ten
tial
en
tr
y
p
o
in
t
f
o
r
cy
b
er
cr
im
in
als
an
d
in
tr
u
d
er
s
to
ca
r
r
y
o
u
t
m
al
icio
u
s
o
p
er
atio
n
s
.
T
h
is
co
u
ld
r
esu
lt
in
s
ig
n
if
ican
t
d
ata
lo
s
s
an
d
f
in
an
cial
c
o
n
s
eq
u
en
ce
s
f
o
r
t
h
e
o
r
g
an
izatio
n
.
No
n
etwo
r
k
,
n
o
m
atter
h
o
w
i
t
is
s
et
u
p
,
ca
n
b
e
co
n
s
id
er
e
d
s
ec
u
r
e
with
o
u
t
I
DS
[
5
]
.
T
h
is
s
y
s
tem
is
d
esig
n
ed
t
o
id
e
n
tify
an
o
m
alie
s
an
d
p
r
ev
e
n
t
in
tr
u
s
io
n
s
o
r
h
o
s
ts
f
r
o
m
p
o
s
in
g
d
an
g
er
s
to
th
e
n
etwo
r
k
,
p
r
o
v
id
in
g
h
ig
h
lev
els
o
f
s
ec
u
r
ity
.
Ad
ap
t
ab
ilit
y
an
d
d
etec
tio
n
o
f
c
y
b
er
attac
k
s
an
d
th
r
ea
ts
ar
e
th
e
m
ai
n
g
o
als
o
f
in
t
r
u
s
io
n
d
etec
tio
n
s
y
s
tem
s
(
I
DS)
.
T
w
o
p
r
im
ar
y
ca
teg
o
r
ies
o
f
in
t
r
u
s
io
n
d
etec
tio
n
s
ch
em
es
ar
e
a
n
o
m
aly
a
n
d
m
is
u
s
e.
I
n
c
o
m
i
n
g
t
r
a
f
f
i
c'
s
s
i
g
n
a
t
u
r
es
a
r
e
e
x
a
m
i
n
e
d
a
n
d
m
a
t
c
h
e
d
a
g
ai
n
s
t
a
d
a
t
a
b
as
e
o
f
k
n
o
w
n
a
s
s
a
u
lts
b
y
t
h
e
f
o
r
m
e
r
[
6
]
.
T
h
e
s
e
t
ec
h
n
o
l
o
g
i
e
s
a
r
e
v
e
r
y
ac
c
u
r
a
t
e
i
n
t
h
e
i
r
d
et
e
c
ti
o
n
,
b
u
t
t
h
e
y
s
t
il
l
c
a
n
'
t
f
i
n
d
m
o
d
i
f
i
e
d
o
r
ze
r
o
-
d
a
y
t
h
r
e
a
t
s
.
T
o
a
d
d
i
n
s
u
l
t
t
o
i
n
j
u
r
y
,
a
t
t
ac
k
e
r
s
ar
e
a
l
w
a
y
s
c
o
m
i
n
g
u
p
w
i
t
h
n
ew
m
e
t
h
o
d
s
a
n
d
p
l
a
n
s
,
t
h
u
s
I
DS
n
e
e
d
t
o
b
e
f
le
x
i
b
l
e
t
o
k
e
e
p
u
p
.
T
h
e
p
r
e
s
e
n
t
a
p
p
r
o
a
c
h
t
o
t
u
n
i
n
g
s
i
g
n
a
t
u
r
es
m
i
g
h
t
b
e
m
o
r
e
t
r
u
s
t
w
o
r
t
h
y
,
b
u
t
a
la
s
,
i
t
is
n
o
t
[
7
]
.
B
y
u
s
in
g
s
o
p
h
is
ticated
s
tatis
ti
ca
l
m
eth
o
d
s
to
th
e
s
tu
d
y
o
f
n
etwo
r
k
tr
af
f
ic
p
atter
n
s
,
an
o
m
aly
-
b
ased
I
DS
aim
to
cir
cu
m
v
en
t
th
e
s
h
o
r
tco
m
in
g
s
o
f
ab
u
s
e
I
DSs
.
I
t
ca
n
b
e
u
tili
ze
d
to
id
en
tify
s
u
s
p
icio
u
s
o
cc
u
r
r
e
n
ce
s
as
th
ey
h
ap
p
en
.
A
n
u
m
b
er
o
f
s
tatis
tical
m
eth
o
d
s
,
in
clu
d
in
g
m
ac
h
in
e
lear
n
in
g
,
ar
e
a
v
ailab
le
f
o
r
u
s
e
in
an
o
m
aly
d
etec
tio
n
[
8
]
.
An
o
m
aly
I
DSs
ar
e
s
u
p
er
io
r
to
s
ig
n
atu
r
e
I
DSs
at
d
etec
tin
g
ze
r
o
-
d
ay
ass
au
lts
.
B
u
t
b
ec
au
s
e
th
ey
m
ay
s
p
o
t
u
n
u
s
u
al
tr
af
f
ic,
th
ey
ca
n
also
h
av
e
a
h
ig
h
p
r
o
p
o
r
tio
n
o
f
f
alse
alar
m
s
[
9
]
.
T
h
at
way
,
th
ey
ca
n
id
en
tify
m
alicio
u
s
attem
p
ts
th
at
d
is
r
u
p
t
th
e
n
o
r
m
ally
f
u
n
ctio
n
in
g
o
f
th
e
n
etwo
r
k
.
At
th
is
tim
e,
attac
k
s
ig
n
atu
r
es
tr
an
s
m
itted
ac
r
o
s
s
th
e
I
o
T
n
etwo
r
k
ca
n
n
o
t
b
e
d
e
tecte
d
b
y
ab
u
s
ed
I
DS
[
1
0
]
.
D
u
e
to
th
e
lack
o
f
a
u
n
iv
er
s
al
s
ig
n
atu
r
e
f
o
r
n
ew
a
s
s
au
lts
,
m
an
y
m
eth
o
d
s
h
av
e
b
ee
n
d
ev
is
ed
to
f
o
r
estall
th
eir
r
ep
etitio
n
.
T
h
is
ca
n
b
e
ac
co
m
p
lis
h
ed
b
y
a
n
aly
zin
g
th
e
d
ata
ac
q
u
ir
ed
b
y
n
etw
o
r
k
tr
af
f
ic
u
s
in
g
m
ac
h
in
e
le
ar
n
in
g
tec
h
n
iq
u
es.
Fin
d
in
g
s
ec
u
r
ity
r
is
k
s
to
I
o
T
n
etwo
r
k
s
ca
n
th
u
s
b
e
aid
ed
b
y
m
ac
h
in
e
lear
n
in
g
[
1
1
]
.
T
h
e
d
etec
tio
n
an
d
m
o
n
ito
r
in
g
ca
p
ab
ilit
ies
o
f
d
ata
g
ath
er
ed
b
y
I
o
T
n
etwo
r
k
s
h
av
e
b
ee
n
en
h
an
ce
d
b
y
th
e
d
ev
elo
p
m
e
n
t
an
d
im
p
le
m
en
tatio
n
o
f
n
u
m
e
r
o
u
s
m
a
ch
in
e
lear
n
in
g
m
o
d
els
with
f
ea
tu
r
e
r
e
d
u
ctio
n
alg
o
r
ith
m
s
[
1
2
]
.
Desp
ite
th
e
s
e
m
o
d
els'
o
u
tp
u
t,
th
eir
r
elia
b
ilit
y
f
o
r
I
o
T
n
etwo
r
k
s
is
s
till
in
q
u
esti
o
n
.
I
n
ad
d
itio
n
,
th
e
r
elativ
e
m
er
its
o
f
v
ar
io
u
s
m
ac
h
i
n
e
lear
n
in
g
m
eth
o
d
s
f
o
r
c
o
n
s
tr
u
ctin
g
a
n
ad
ap
tiv
e
I
DS
r
em
ain
d
eb
atab
le.
Ma
n
y
d
ee
p
lear
n
in
g
p
r
o
ce
d
u
r
es
h
a
v
e
b
ee
n
test
ed
o
n
d
if
f
er
e
n
t
d
atasets
to
f
in
d
o
u
t
h
o
w
ef
f
ec
tiv
e
th
ey
ar
e
at
d
etec
tin
g
I
o
T
I
DS
[
1
3
]
.
B
ec
au
s
e
o
f
h
o
w
im
p
o
r
t
an
t
tim
e
is
to
a
s
u
cc
ess
f
u
l
I
o
T
attac
k
p
r
e
v
en
tio
n
s
y
s
tem
,
it
is
cr
u
cial
to
th
in
k
ab
o
u
t
way
s
to
co
n
s
tr
u
ct
a
n
d
tr
a
in
th
e
s
y
s
tem
f
aster
.
R
ed
u
cin
g
th
e
co
m
p
u
tatio
n
al
tim
e
n
ee
d
ed
b
y
I
DS
ca
n
ac
c
o
m
p
lis
h
th
is
.
W
h
en
m
ac
h
in
e
lear
n
in
g
class
if
ier
s
'
p
er
f
o
r
m
an
ce
is
p
o
o
r
,
r
e
s
ea
r
ch
er
s
u
s
e
th
e
en
s
em
b
le
ap
p
r
o
ac
h
to
b
u
ild
a
s
tr
o
n
g
er
p
r
ed
ictio
n
m
o
d
el
b
y
co
m
b
in
in
g
th
eir
wea
k
e
r
class
if
ier
s
[
1
4
]
.
Fu
r
th
e
r
m
o
r
e
,
th
e
ca
p
ac
ity
o
f
th
e
m
ac
h
in
e
lear
n
in
g
d
etec
tio
n
s
y
s
tem
is
d
atab
ase
d
ep
en
d
en
t;
th
u
s
,
it
is
ess
en
tial
to
g
ath
er
o
r
c
r
ea
te
a
tr
u
s
two
r
th
y
d
ataset
f
r
o
m
th
e
I
o
T
in
f
r
astru
ctu
r
es
s
ettin
g
at
s
ev
er
al
lev
els,
in
clu
d
in
g
ac
tu
al
tr
af
f
ic
[
1
5
]
.
Data
s
ets
telem
etr
y
o
f
n
etwo
r
k
(
T
o
N
)
-
I
o
T
,
c
o
m
p
i
led
f
r
o
m
a
wid
e
v
ar
iety
o
f
I
o
T
d
ev
ices,
h
a
v
e
r
ec
e
n
tly
u
n
d
er
g
o
n
e
test
in
g
an
d
au
th
en
ticatio
n
in
a
cy
b
er
lab
t
o
en
s
u
r
e
th
eir
r
eliab
ilit
y
f
o
r
u
s
e
in
m
ac
h
in
e
lea
r
n
in
g
p
r
o
ce
d
u
r
es.
T
h
e
f
o
llo
win
g
ar
e
a
f
ew
o
f
th
e
m
o
s
t
s
ig
n
if
ican
t
d
if
f
icu
lties
:
d
ata
im
b
alan
ce
in
th
e
I
o
T
en
v
ir
o
n
m
en
t
a
n
d
u
n
k
n
o
wn
th
r
ea
ts
,
s
u
ch
ze
r
o
-
d
ay
attac
k
s
,
ar
e
th
e
m
o
s
t
im
p
o
r
tan
t
is
s
u
es
d
is
cu
s
s
ed
in
th
is
ar
ticle.
A
m
ajo
r
h
u
r
d
le
to
ef
f
ec
tiv
e
in
tr
u
s
io
n
d
etec
tio
n
in
s
u
c
h
s
itu
atio
n
s
is
th
e
is
s
u
e
r
esear
ch
er
s
co
n
f
r
o
n
t
wh
e
n
d
ea
lin
g
with
d
i
v
er
s
e
an
d
im
b
alan
ce
d
d
ata
ac
q
u
ir
e
d
f
r
o
m
v
ar
io
u
s
a
n
d
d
is
tin
ct
d
ev
ices
.
W
ith
in
th
e
I
o
T
'
s
I
DS
,
th
er
e
is
an
o
th
e
r
p
r
o
b
lem
th
at
n
ee
d
s
f
i
x
in
g
.
T
h
is
d
if
f
ic
u
lty
is
r
elate
d
to
a
class
o
f
ass
au
lts
th
at
co
u
ld
d
am
ag
e
th
e
n
etwo
r
k
b
u
t
wh
o
s
e
n
atu
r
e
is
u
n
clea
r
an
d
ca
n
n
o
t
b
e
p
r
ed
icted
in
a
d
v
an
ce
.
L
ik
e
ze
r
o
-
d
ay
attac
k
s
,
th
e
ex
a
ct
tim
e
o
f
th
ese
a
s
s
au
lt
s
is
als
o
im
p
o
s
s
ib
le
to
f
o
r
esee.
As
a
r
esu
lt,
co
n
v
en
tio
n
al
m
eth
o
d
s
o
f
d
etec
tio
n
g
r
o
w
in
c
r
ea
s
in
g
ly
in
ef
f
ec
tiv
e
,
m
ak
in
g
t
h
em
an
ess
en
tial su
b
ject
o
f
c
o
n
tin
u
in
g
r
esear
ch
.
T
h
e
m
ain
co
n
tr
ib
u
tio
n
o
f
th
e
r
esear
ch
wo
r
k
is
m
e
n
tio
n
e
d
a
s
f
o
llo
ws:
o
u
r
r
esear
ch
s
u
g
g
e
s
ts
a
n
ew
I
DS
th
at
u
s
es
d
ee
p
lear
n
in
g
p
r
o
ce
d
u
r
es
to
m
o
n
ito
r
I
o
T
n
e
two
r
k
d
ata
f
o
r
s
ig
n
s
o
f
in
f
iltr
atio
n
.
T
h
e
p
r
im
ar
y
o
b
jectiv
e
o
f
o
u
r
m
eth
o
d
is
to
im
p
r
o
v
e
th
e
p
r
ec
is
io
n
o
f
a
s
s
au
lt
d
etec
tio
n
.
I
n
o
r
d
e
r
to
d
is
tin
g
u
is
h
b
etwe
en
ty
p
ical
an
d
n
o
n
-
s
tan
d
ar
d
I
o
T
n
etwo
r
k
tr
af
f
ic,
we
b
u
ilt
an
d
test
ed
f
o
u
r
s
u
p
er
v
is
ed
m
ac
h
i
n
e
lear
n
in
g
m
o
d
els.
C
o
n
s
eq
u
en
tly
,
th
e
f
o
llo
win
g
a
r
e
th
e
co
n
t
r
ib
u
tio
n
s
o
f
th
e
p
ap
er
:
i)
Su
g
g
esti
n
g
a
s
im
p
le
way
to
la
b
el
n
etwo
r
k
tr
a
f
f
ic
f
r
o
m
I
o
T
d
ev
ices
as
eith
er
n
o
r
m
al
o
r
p
r
o
b
lem
atic.
T
h
e
I
o
T
n
etwo
r
k
d
ev
ices'
I
DS
p
er
f
o
r
m
an
ce
ca
n
b
e
im
p
r
o
v
e
d
b
y
u
s
in
g
an
o
p
tim
izatio
n
s
tr
ateg
y
to
in
cr
ea
s
e
th
e
s
y
n
th
etic
m
in
o
r
ity
o
v
e
r
-
s
am
p
lin
g
tech
n
iq
u
e
(
SMOT
E
)
s
am
p
lin
g
r
ate.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
3
,
J
u
n
e
2
0
2
6
:
2
9
2
0
-
2
9
3
4
2922
ii)
B
u
ild
in
g
an
d
test
in
g
a
m
o
d
el
f
o
r
attac
k
d
etec
tio
n
ca
lled
d
ee
p
b
id
ir
ec
tio
n
al
lo
n
g
s
h
o
r
t
-
t
er
m
m
em
o
r
y
(
d
ee
p
B
iLST
M)
.
Me
tr
ics
lik
e
as
p
ar
t
o
f
th
e
p
r
esen
tatio
n
e
v
alu
atio
n
.
T
h
e
b
u
ilt
m
o
d
el
o
u
t
p
er
f
o
r
m
ed
th
e
m
o
s
t r
ec
en
t stu
d
y
u
s
in
g
th
e
s
a
m
e
d
ataset
in
ter
m
s
o
f
d
etec
tio
n
ap
p
r
o
ac
h
p
er
f
o
r
m
an
ce
.
T
h
e
r
est
o
f
t
h
e
p
a
p
er
is
ar
r
a
n
g
ed
as
f
o
llo
ws
.
Sectio
n
1
i
n
tr
o
d
u
ce
s
I
o
T
an
d
I
DS
alo
n
g
with
th
e
co
n
tr
ib
u
tio
n
s
o
f
t
h
is
wo
r
k
.
Se
ctio
n
2
r
ev
iews
r
elate
d
liter
at
u
r
e
.
Sectio
n
3
d
escr
ib
es
p
r
o
p
o
s
ed
m
eth
o
d
o
lo
g
y
.
Sectio
n
4
p
r
esen
ts
ex
p
e
r
im
en
t
al
r
esu
lts
an
d
d
is
cu
s
s
io
n
.
L
astl
y
,
s
ec
tio
n
5
c
o
n
clu
d
es th
e
p
ap
er
.
2.
RE
L
AT
E
D
WO
RK
S
Kau
s
h
ik
et
a
l.
[
1
6
]
s
tu
d
ied
a
d
ec
is
io
n
tr
ee
m
o
d
el
with
u
n
iq
u
e
f
e
atu
r
e
s
elec
tio
n
alg
o
r
ith
m
f
o
r
d
etec
tio
n
o
f
in
tr
u
s
io
n
in
th
e
e
n
v
ir
o
n
m
en
t.
T
h
e
p
er
f
o
r
m
a
n
ce
was
im
p
r
o
v
ed
b
y
th
e
d
esig
n
e
d
m
o
d
el
an
d
n
ea
r
l
y
27
-
6
3
%
o
f
tr
ain
in
g
tim
e
wa
s
m
in
im
ized
b
y
class
if
ier
s
.
T
h
e
d
etec
tio
n
ac
cu
r
ac
y
was
im
p
r
o
v
ed
an
d
th
e
co
m
p
u
tatio
n
al
o
v
er
h
ea
d
was
m
in
im
ized
b
y
u
s
in
g
th
e
wid
el
y
u
s
ed
d
is
cr
im
in
ativ
e
f
ea
tu
r
es.
W
h
ile
co
m
p
ar
in
g
with
NSL
-
KDD
d
atase
t,
th
e
p
r
o
p
o
s
ed
m
o
d
el
ac
h
iev
e
d
o
v
er
9
9
.
9
%
o
f
r
ec
all,
p
r
ec
is
io
n
,
F1
-
s
co
r
e
,
an
d
ac
cu
r
ac
y
o
n
I
o
T
-
I
D2
0
d
ataset
.
Fo
r
th
e
p
u
r
p
o
s
e
o
f
in
t
r
u
s
io
n
d
etec
tio
n
in
I
o
T
n
etwo
r
k
s
u
s
in
g
a
m
ac
h
in
e
lear
n
i
n
g
-
b
a
s
ed
attac
k
ca
teg
o
r
izatio
n
f
r
am
ewo
r
k
,
L
i
et
a
l.
[
1
7
]
h
as
co
n
d
u
cted
a
c
o
m
p
r
eh
en
s
iv
e
c
o
m
p
ar
is
o
n
o
f
f
e
atu
r
e
ex
tr
ac
tio
n
a
n
d
s
elec
tio
n
.
Usi
n
g
class
if
icatio
n
,
it
ex
am
in
es
p
er
f
o
r
m
an
ce
p
ar
am
eter
s
s
u
ch
as
ac
cu
r
ac
y
,
F
1
-
s
co
r
e,
an
d
r
u
n
tim
e
o
n
th
e
n
etwo
r
k
T
o
N
-
I
o
T
d
ata
s
et,
wh
ich
is
a
h
eter
o
g
en
e
o
u
s
I
o
T
d
ataset.
Sin
ce
th
e
r
e
ar
e
f
e
wer
ch
ar
ac
ter
is
tics
to
wo
r
k
with
in
f
ea
tu
r
e
ex
tr
a
ctio
n
,
it
o
u
tp
er
f
o
r
m
s
f
ea
tu
r
e
s
elec
tio
n
in
ter
m
s
o
f
d
etec
tio
n
p
er
f
o
r
m
an
ce
.
No
t
o
n
ly
th
at,
e
x
tr
ac
tio
n
is
less
af
f
ec
ted
b
y
ch
a
n
g
es
in
th
e
q
u
an
ti
ty
o
f
f
ea
tu
r
es
an
d
e
x
h
ib
its
less
f
ea
tu
r
e
r
ed
u
ctio
n
th
an
s
elec
tio
n
.
Nev
er
th
eless
,
wh
en
co
n
tr
asted
with
its
alter
n
ativ
e,
f
ea
tu
r
e
s
elec
tio
n
r
esu
l
ts
in
s
h
o
r
ter
m
o
d
el
tr
ain
in
g
an
d
in
f
e
r
en
ce
tim
es.
Ad
d
itio
n
ally
,
with
a
lar
g
er
f
ea
tu
r
e
s
et,
th
er
e
is
m
o
r
e
r
o
o
m
to
r
ef
in
e
th
e
s
elec
tio
n
p
r
o
ce
s
s
r
ath
er
th
an
th
e
ex
tr
ac
t
io
n
p
h
ase.
B
o
th
m
u
lticlas
s
an
d
b
in
ar
y
class
if
icatio
n
s
ad
h
er
e
to
th
is
.
Fin
d
in
g
th
e
r
ig
h
t
in
tr
u
s
io
n
d
etec
tio
n
m
et
h
o
d
s
f
o
r
d
if
f
er
en
t
s
itu
atio
n
s
is
m
ad
e
ea
s
ier
with
th
e
h
elp
o
f
th
e
s
tu
d
y
'
s
r
ec
o
m
m
en
d
atio
n
s
.
C
o
m
p
ar
is
o
n
an
d
r
ec
o
m
m
en
d
atio
n
s
b
ased
o
n
th
e
T
o
N
-
I
o
T
h
eter
o
g
e
n
eo
u
s
I
o
T
d
ataset
wer
e
p
r
ev
io
u
s
ly
d
is
r
eg
a
r
d
ed
.
I
n
s
u
m
,
th
e
s
tu
d
y
p
r
o
v
id
es
a
co
m
p
r
eh
en
s
iv
e
an
aly
s
is
o
f
f
ea
tu
r
e
r
e
d
u
ctio
n
ap
p
r
o
ac
h
es
f
o
r
in
tr
u
s
io
n
d
etec
tio
n
in
I
o
T
n
etwo
r
k
s
p
o
wer
e
d
b
y
lear
n
in
g
.
Usi
n
g
a
d
ee
p
lear
n
in
g
s
y
s
tem
,
Yar
as
an
d
Den
er
[
1
8
]
we
r
e
ab
le
to
an
aly
ze
th
e
c
o
llected
d
ataset
o
f
n
etwo
r
k
t
r
af
f
ic
i
n
a
lar
g
e
d
at
a
s
ettin
g
an
d
id
e
n
tify
n
etwo
r
k
th
r
ea
ts
.
T
h
e
r
esear
ch
is
ca
r
r
ied
o
u
t
in
a
Go
o
g
le
C
o
lab
o
r
ato
r
y
s
ettin
g
u
tili
zin
g
Py
Sp
ar
k
an
d
Ap
ac
h
e
s
p
ar
k
.
T
h
e
r
esear
ch
m
ak
es
u
s
e
o
f
th
e
Scik
i
t
-
lear
n
an
d
Ker
as
lib
r
ar
ies.
T
h
e
m
o
d
el
is
tr
ain
ed
an
d
test
ed
u
s
in
g
th
e
'
C
I
C
I
o
T
2
0
2
3
'
an
d
'
T
o
N_
I
o
T
'
d
atasets
.
B
y
r
ed
u
cin
g
th
e
n
u
m
b
er
o
f
f
ea
tu
r
es
in
th
e
d
atasets
b
y
co
r
r
elatio
n
,
we
ca
n
b
e
s
u
r
e
th
at
th
e
test
s
will
o
n
ly
in
clu
d
e
th
e
m
o
s
t
im
p
o
r
tan
t
f
ea
t
u
r
es.
T
h
e
g
o
al
was
to
cr
ea
te
a
d
ee
p
lear
n
in
g
al
g
o
r
ith
m
th
at
co
m
b
in
es
L
STM
with
one
-
d
im
e
n
s
io
n
al
co
n
v
o
l
u
tio
n
a
l
n
eu
r
al
n
etwo
r
k
(
C
NN)
.
T
en
d
if
f
e
r
en
t
alg
o
r
ith
m
s
f
r
o
m
th
e
f
ield
s
o
f
m
ac
h
in
e
lear
n
in
g
an
d
d
ee
p
lear
n
in
g
w
er
e
test
ed
alo
n
g
s
id
e
th
e
n
ew
tech
n
iq
u
e.
F1
p
ar
am
eter
s
,
r
ec
all,
ac
cu
r
ac
y
,
a
n
d
p
r
ec
is
io
n
wer
e
u
s
ed
to
ass
es
s
th
e
m
o
d
el'
s
p
er
f
o
r
m
an
ce
.
T
h
e
r
esear
c
h
s
h
o
ws
t
h
at
u
s
in
g
th
e
'
C
I
C
I
o
T
2
0
2
3
'
d
ataset,
th
e
ac
cu
r
ac
y
class
if
icatio
n
is
9
9
.
9
5
%
an
d
f
o
r
m
u
lticlas
s
if
icatio
n
it
is
9
9
.
9
6
%.
A
b
in
ar
y
class
if
icatio
n
s
u
cc
ess
r
ate
o
f
9
8
.
7
5
% is
ac
h
iev
ed
in
th
e
'
T
o
N
-
I
o
T
'
d
ataset.
T
o
im
p
r
o
v
e
I
DS
ac
cu
r
ac
y
an
d
r
ed
u
ce
attac
k
r
is
k
,
So
ltan
i
et
a
l.
[
1
9
]
p
r
o
p
o
s
ed
a
h
y
b
r
id
m
ac
h
in
e
lear
n
in
g
m
eth
o
d
th
at
u
s
es
r
an
d
o
m
f
o
r
ests
an
d
k
-
n
ea
r
e
s
t
n
eig
h
b
o
r
s
as
s
u
p
er
v
is
ed
class
if
ier
s
.
L
in
ea
r
d
is
cr
im
in
an
t
an
aly
s
is
an
d
b
ac
k
war
d
elim
in
atio
n
also
u
s
e
d
t
o
r
ed
u
ce
co
m
p
u
tatio
n
al
c
o
s
ts
an
d
r
e
d
u
ce
f
ea
t
u
r
es.
W
h
en
d
is
ag
r
ee
m
en
ts
em
e
r
g
ed
b
etwe
en
th
e
class
if
ier
s
'
co
n
clu
s
io
n
s
af
ter
tr
ain
i
n
g
,
t
h
e
f
in
al
v
er
d
ict
was
b
ac
k
e
d
b
y
I
SO/IE
C
2
7
0
0
1
s
tan
d
ar
d
s
.
T
h
e
s
u
g
g
ested
m
o
d
el'
s
C
I
C
I
DS
2
0
1
7
,
NSL
-
KDD,
b
esid
es
T
o
N
-
I
o
T
d
atasets
with
in
a
Py
th
o
n
p
r
o
g
r
a
m
m
in
g
en
v
ir
o
n
m
en
t.
T
h
e
r
esu
lts
s
h
o
wed
th
at
th
e
s
u
g
g
ested
m
eth
o
d
ac
h
iev
ed
an
im
p
r
ess
iv
e
p
r
esen
tatio
n
with
a
m
u
lti
-
class
cla
s
s
if
icatio
n
ac
cu
r
ac
y
o
f
9
9
.
9
6
%
o
n
th
e
C
I
C
I
DS
2
0
1
7
d
ataset,
a
b
in
ar
y
class
if
icatio
n
ac
cu
r
ac
y
o
f
9
9
.
3
7
%
o
n
th
e
NSL
-
KDD
d
ataset,
an
d
a
m
u
lti
-
class
class
if
icatio
n
ac
cu
r
ac
y
o
f
9
9
.
9
6
%
o
n
th
e
T
o
N
-
I
o
T
d
ataset.
Attack
s
u
cc
es
s
r
ates
f
o
r
ea
ch
d
ataset
ar
e
0
.
0
5
%,
0
.
2
4
%,
an
d
0
%,
in
d
iv
id
u
ally
,
s
h
o
win
g
a
c
o
n
s
id
er
ab
le
d
ec
r
ea
s
e
wh
en
r
elate
d
t
o
o
th
er
a
p
p
r
o
ac
h
es.
An
in
n
o
v
ativ
e
an
d
ef
f
icien
t
f
r
am
ewo
r
k
b
ased
o
n
d
ee
p
lear
n
in
g
h
as
b
ee
n
c
r
ea
ted
to
id
en
tify
in
tr
u
s
io
n
s
in
s
m
ar
t
a
g
r
icu
ltu
r
a
l
s
y
s
tem
s
b
y
Keth
in
en
i
a
n
d
Pr
ad
ee
p
in
i
[
2
0
]
.
T
h
e
f
ir
s
t
o
f
t
h
e
th
r
ee
lev
els
o
f
th
e
d
esig
n
is
th
e
s
en
s
o
r
lay
er
,
wh
ich
co
m
p
r
is
es
th
e
ac
tu
al
in
s
tallatio
n
o
f
s
en
s
o
r
s
in
f
ield
s
an
d
o
th
er
ag
r
ic
u
ltu
r
al
s
ettin
g
s
.
T
h
e
s
u
g
g
ested
I
DS
i
s
in
s
talled
in
ev
er
y
f
o
g
n
o
d
e
t
h
at
m
ak
es
u
p
t
h
e
s
ec
o
n
d
tier
,
th
e
f
o
g
co
m
p
u
tin
g
lay
er
(
FC
L
)
.
Fo
r
ad
d
itio
n
al
d
ata
p
r
o
ce
s
s
in
g
,
th
e
co
llected
in
f
o
r
m
atio
n
is
s
en
t
to
th
is
f
o
g
lay
er
.
Data
s
to
r
ag
e
an
d
en
d
-
to
-
e
n
d
s
er
v
ices
ar
e
p
r
o
v
id
ed
b
y
th
e
th
ir
d
tier
,
t
h
e
cl
o
u
d
c
o
m
p
u
tin
g
lay
e
r
.
T
o
id
en
tify
an
d
ca
teg
o
r
iz
e
in
tr
u
s
io
n
s
,
th
e
s
u
g
g
ested
m
o
d
el
co
m
b
in
es
a
C
NN
m
o
d
el
with
a
b
id
ir
ec
tio
n
al
g
ated
r
ec
u
r
r
en
t
u
n
it
(
B
iGR
U)
m
o
d
el.
T
h
e
B
iGR
U
m
o
d
el
in
co
r
p
o
r
ates
an
atten
tio
n
m
eth
o
d
to
d
is
co
v
er
th
e
cr
itical
asp
ec
ts
th
at
tr
ig
g
er
th
e
d
is
tr
ib
u
ted
d
en
ial
o
f
s
er
v
ice
(
DDo
S
)
attac
k
.
T
h
e
wild
h
o
r
s
e
alg
o
r
ith
m
,
a
m
eta
-
h
eu
r
is
tic
o
p
tim
izatio
n
tech
n
iq
u
e
in
s
p
ir
ed
b
y
n
atu
r
e,
f
u
r
th
er
im
p
r
o
v
es
th
e
class
if
icatio
n
m
o
d
el'
s
ac
cu
r
ac
y
.
Fin
ally
,
d
ata
is
co
llected
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
la
s
s
imb
a
la
n
ce
r
eso
lu
tio
n
in
I
o
T n
etw
o
r
ks u
s
in
g
a
d
va
n
ce
d
elk
h
erd
…
(
S
r
ika
n
th
Mu
d
iya
n
u
r
S
r
ir
a
ma
p
p
a
)
2923
f
r
o
m
f
o
g
n
o
d
es
a
n
d
s
to
r
ag
e
s
er
v
ices
ar
e
p
r
o
v
id
ed
b
y
clo
u
d
lay
er
,
last
lay
er
.
T
h
e
T
o
N
-
I
o
T
an
d
APA
-
DDo
S
attac
k
d
atasets
wi
ll
b
e
u
s
ed
to
ev
alu
ate
th
e
p
r
o
p
o
s
ed
s
y
s
tem
af
ter
it
is
b
u
ilt
in
th
e
P
y
th
o
n
p
latf
o
r
m
.
I
n
co
m
p
ar
is
o
n
t
o
th
e
c
u
r
r
en
t
ap
p
r
o
ac
h
es,
th
e
s
u
g
g
ested
s
y
s
tem
ac
h
iev
es
b
etter
r
esu
lts
in
ter
m
s
o
f
r
ate
(
9
9
.
0
2
%)
,
p
r
ec
is
io
n
(
9
9
.
8
9
%),
an
d
F
1
-
s
co
r
e
(
9
9
.
0
5
%)
o
n
APA
DDo
S a
ttack
d
ataset,
an
d
o
n
T
o
N
-
I
o
T
d
ataset
(
9
9
.
7
1
%).
T
o
im
p
lem
en
t
s
em
i
-
s
u
p
er
v
is
e
d
lear
n
in
g
-
b
ased
(
I
DS)
with
o
u
t
th
e
n
ee
d
f
o
r
attac
k
tr
ain
in
g
s
am
p
les,
Ud
d
in
et
a
l
.
[
2
1
]
p
r
o
p
o
s
ed
tw
o
m
eth
o
d
s
:
i
)
c
o
n
s
tr
u
ctin
g
a
n
o
n
e
class
class
if
icatio
n
(
OC
C
)
m
o
d
el
tr
ain
e
d
ju
s
t
o
n
h
ar
m
less
n
etwo
r
k
tr
af
f
ic
an
d
ii
)
tr
ain
in
g
a
s
u
p
e
r
v
is
ed
m
ac
h
in
e
lear
n
in
g
m
o
d
el
wi
th
s
y
n
th
etic
attac
k
s
am
p
les
d
is
tr
ib
u
ted
u
n
if
o
r
m
ly
an
d
r
a
n
d
o
m
l
y
.
W
e
p
u
t
b
o
th
m
eth
o
d
s
to
th
e
test
o
n
ten
u
p
-
to
-
d
ate
b
en
c
h
m
ar
k
I
DS
d
atasets
an
d
co
m
p
ar
ed
t
h
eir
r
esu
lts
.
I
n
ter
m
s
o
f
d
etec
tin
g
p
r
ev
i
o
u
s
ly
u
n
k
n
o
wn
attac
k
s
,
in
p
ar
ticu
lar
,
th
e
r
esu
lts
s
h
o
w
th
at
th
e
O
C
C
m
o
d
el
tr
ain
ed
an
d
ev
alu
ated
u
s
in
g
r
ea
l
-
life
s
ce
n
ar
io
s
wo
r
k
s
f
ar
b
etter
th
an
co
n
v
en
tio
n
al
s
u
p
er
v
is
ed
class
if
icatio
n
an
d
o
t
h
er
OC
C
b
ased
ap
p
r
o
ac
h
es.
T
h
is
m
o
d
el
is
b
ased
o
n
t
h
e
s
tate
-
of
-
th
e
-
ar
t
an
o
m
aly
d
ete
ctio
n
p
r
o
ce
d
u
r
e.
An
ad
a
p
tiv
e
b
o
o
s
tin
g
(
A
d
aBo
o
s
t
)
-
b
ased
in
tr
u
s
io
n
d
etec
tio
n
f
r
am
ewo
r
k
h
as
b
ee
n
p
r
o
p
o
s
ed
[
2
2
]
.
T
o
b
r
in
g
o
u
r
m
o
d
el
to
li
f
e,
th
e
ch
i
-
s
q
u
a
r
ed
m
eth
o
d
is
em
p
lo
y
ed
t
o
tack
le
th
e
class
im
b
alan
ce
p
r
o
b
lem
.
T
wo
d
atasets
—
UNS
W
-
NB
1
5
an
d
T
o
N
-
I
o
T
—
ar
e
u
s
ed
t
o
e
v
alu
ate
th
e
m
o
d
el.
F1
-
s
co
r
e,
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
an
d
r
ec
all
ar
e
u
s
ed
to
ass
ess
th
e
m
o
d
el.
Kr
is
h
n
an
an
d
Sh
r
in
ath
[
2
3
]
w
as
u
s
ed
a
m
eth
o
d
ca
lled
lo
c
alize
d
g
en
er
alize
d
m
atr
ix
lear
n
i
n
g
v
ec
to
r
q
u
an
tizatio
n
(
L
GM
L
VQ)
to
d
eter
m
in
e
wh
ich
f
ea
tu
r
es
we
r
e
m
o
s
t
im
p
o
r
ta
n
t
f
o
r
ea
ch
c
lass
.
Ap
p
ly
in
g
o
u
r
m
o
d
el
to
th
e
well
-
k
n
o
wn
NF
-
B
o
T
-
I
o
T
d
ataset,
it
p
er
f
o
r
m
s
ad
m
ir
ab
ly
,
with
an
ac
cu
r
ac
y
s
co
r
e
o
f
9
9
.
9
9
5
2
%.
Un
s
ee
n
attac
k
d
etec
tio
n
u
s
in
g
a
s
h
allo
w
au
to
en
co
d
er
with
r
e
co
n
s
tr
u
ctio
n
er
r
o
r
th
r
esh
o
ld
in
g
is
an
o
th
er
ar
ea
o
f
in
ter
est
in
th
e
s
u
g
g
ested
r
esear
ch
.
T
h
e
m
eth
o
d
is
test
ed
o
n
two
b
en
ch
m
a
r
k
d
atasets
,
NF
-
T
o
N
-
I
o
T
b
esid
es
NF
-
C
SE
-
C
I
C
-
I
DS
2
0
1
8
,
to
s
ee
h
o
w
well
it
wo
r
k
s
.
T
h
e
m
o
d
el'
s
p
r
esen
tatio
n
o
n
n
ew
s
am
p
les
is
im
p
r
es
s
iv
e,
with
an
av
er
a
g
e
F1
-
s
co
r
e
o
f
9
5
.
1
4
5
%,
an
ac
cu
r
ac
y
o
f
9
3
.
7
1
5
%
,
a
p
r
ec
is
io
n
o
f
9
9
.
9
5
5
%
,
an
d
a
r
ec
all
o
f
9
0
.
8
6
5
%
.
I
n
th
e
s
ettin
g
o
f
im
b
alan
ce
d
d
ata,
th
e
p
r
o
p
o
s
ed
wo
r
k
o
f
f
e
r
s
a
co
m
p
lete
s
o
lu
tio
n
th
at
h
as
p
r
o
v
en
p
er
f
o
r
m
an
ce
in
i
d
en
tify
in
g
b
o
th
k
n
o
w
n
an
d
n
ew
attac
k
s
,
m
a
k
in
g
s
u
b
s
tan
tial a
d
d
itio
n
to
I
o
T
s
ec
u
r
ity
.
3.
M
E
T
H
O
D
I
n
th
is
s
ec
tio
n
,
th
e
b
r
ief
ex
p
la
n
atio
n
o
f
th
e
p
r
o
jecte
d
class
ical
m
eth
o
d
is
m
en
tio
n
ed
,
wh
er
e
Fig
u
r
e
1
p
r
esen
ts
th
e
r
esear
ch
wo
r
k
f
lo
w
o
f
wo
r
k
.
I
n
itially
,
th
e
d
atas
et
th
at
is
g
o
in
g
to
v
alid
ate
th
e
p
r
o
p
o
s
ed
m
o
d
els’
an
aly
s
is
is
d
escr
ib
ed
,
th
en
p
r
e
-
p
r
o
ce
s
s
in
g
tech
n
i
q
u
es
th
at
is
u
s
ed
to
n
o
r
m
alize
th
e
in
p
u
t
d
a
taset
is
m
en
tio
n
ed
.
T
h
en
,
th
e
d
etailed
d
escr
ip
tio
n
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
f
o
r
attac
k
d
etec
tio
n
is
m
ath
em
atica
lly
ex
p
lain
ed
.
Fig
u
r
e
1
.
W
o
r
k
f
lo
w
o
f
th
e
r
es
ea
r
ch
wo
r
k
3
.
1
.
Descript
io
n o
f
T
o
N
-
I
o
T
net
wo
rk
da
t
a
s
et
Data
f
o
r
th
e
T
o
N
I
o
T
n
etwo
r
k
tr
af
f
ic
s
et
co
m
es
f
r
o
m
ac
tu
al
h
o
m
e
-
b
ased
I
o
T
d
e
v
ice
s
ce
n
ar
io
s
[
2
4
]
.
I
n
C
an
b
er
r
a,
at
th
e
Un
iv
er
s
ity
o
f
New
So
u
t
h
a
n
d
I
n
f
o
r
m
a
tio
n
T
ec
h
n
o
lo
g
y
(
SEI
T
)
,
th
e
Au
s
tr
alian
Def
en
s
e
Fo
r
ce
Aca
d
em
y
(
ADFA)
is
lo
ca
ted
,
wh
ich
is
also
th
e
s
o
u
r
ce
o
f
th
e
T
o
N
-
I
o
T
d
ataset
n
etw
o
r
k
[
2
5
]
.
T
o
en
s
u
r
e
th
e
T
o
N
-
I
o
T
d
atasets
,
g
ath
er
e
d
f
r
o
m
v
a
r
io
u
s
I
o
T
d
e
v
ices
u
s
in
g
I
o
T
co
n
s
tr
u
ctio
n
,
ar
e
m
o
r
e
tr
u
s
two
r
th
y
wh
en
ap
p
lied
an
d
im
p
lem
en
ted
u
s
in
g
m
ac
h
in
e
lear
n
in
g
m
et
h
o
d
s
,
t
h
ey
h
av
e
b
ee
n
e
v
alu
ated
an
d
g
en
u
in
e
in
a
cy
b
e
r
lab
.
No
n
et
h
eless
,
we
p
r
o
v
id
e
a
b
i
n
ar
y
ca
teg
o
r
izatio
n
o
f
ty
p
ical
an
d
n
o
n
-
s
tan
d
ar
d
n
etwo
r
k
tr
af
f
ic
f
r
o
m
I
o
T
d
ev
ices
in
th
is
s
tu
d
y
.
B
y
b
u
il
d
in
g
a
n
ef
f
ec
tiv
e
e
n
s
em
b
le
f
r
am
ewo
r
k
o
f
s
u
p
er
v
is
ed
m
ac
h
i
n
e
lear
n
in
g
m
o
d
els,
th
e
s
u
g
g
ested
m
o
d
el
o
f
f
er
s
a
way
to
en
h
a
n
ce
I
DS p
er
f
o
r
m
a
n
ce
.
Data
f
r
o
m
th
e
I
I
o
T
s
en
s
o
r
s
,
s
y
s
tem
lo
g
s
,
an
d
d
etails o
f
n
etwo
r
k
tr
af
f
ic
ar
e
ju
s
t a
f
ew
ex
am
p
les o
f
th
e
d
iv
er
s
e
ty
p
es
o
f
ev
id
e
n
ce
t
h
at
ca
n
b
e
g
ath
er
e
d
a
n
d
an
aly
ze
d
u
s
in
g
th
is
d
ataset.
I
I
o
T
is
b
o
r
n
f
r
o
m
a
wo
r
k
ab
le
n
etwo
r
k
.
A
n
u
m
b
er
o
f
p
h
y
s
ical
d
ev
ices,
clo
u
d
n
o
d
es,
a
n
d
v
ir
tu
al
s
er
v
er
s
ar
e
all
in
ter
co
n
n
ec
ted
in
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
3
,
J
u
n
e
2
0
2
6
:
2
9
2
0
-
2
9
3
4
2924
T
o
N
-
I
o
T
d
ataset.
As
p
ar
t
o
f
th
is
p
r
o
ce
s
s
,
we
m
u
s
t
ev
alu
at
e
th
e
ef
f
icac
y
an
d
ac
cu
r
ac
y
o
f
v
ar
io
u
s
AI
-
b
ase
d
in
f
o
r
m
atio
n
s
ec
u
r
ity
in
itiativ
e
s
.
T
h
er
e
ar
e
c
y
b
er
attac
k
s
o
n
b
u
s
in
ess
es
in
th
is
d
ataset.
A
n
u
m
b
er
o
f
f
ac
to
r
s
wer
e
co
n
s
id
er
ed
in
th
e
test
in
g
,
s
u
ch
as
W
in
d
o
ws
7
an
d
1
0
,
n
etwo
r
k
s
am
p
les,
tr
an
s
m
is
s
io
n
d
etails,
an
d
I
o
T
tr
ain
in
g
m
o
d
els,
b
esid
es
a
ttack
s
o
n
ea
c
h
.
R
eg
u
lar
,
d
en
ial
o
f
s
er
v
ice
(
Do
S)
,
b
ac
k
d
o
o
r
,
DD
OS,
m
an
-
in
-
th
e
-
m
id
d
le
(
MI
T
M)
,
i
n
jectio
n
,
r
an
s
o
m
war
e,
p
ass
wo
r
d
en
t
r
ies
ar
e
all
p
ar
t
o
f
th
e
T
o
N
-
I
o
T
d
ataset
[
2
6
]
.
A
co
m
p
r
eh
e
n
s
iv
e
o
v
e
r
v
iew
o
f
th
e
T
o
N
-
I
o
T
d
ataset
is
p
r
o
v
i
d
ed
in
T
ab
le
1
.
T
ab
le
1
.
T
o
N
-
I
o
T
d
ataset
im
ag
er
ies
To
t
a
l
r
e
c
o
r
d
s
D
a
t
a
b
a
se
d
o
n
e
v
e
n
t
s
7
1
,
4
0
1
S
c
a
n
n
i
n
g
2
1
,
0
8
9
C
r
o
ss
-
si
t
e
scri
p
t
i
n
g
(
X
S
S
)
1
7
,
1
8
5
P
a
ssw
o
r
d
7
9
,
6
3
8
N
o
r
mal
3
3
,
7
5
3
D
o
S
4
5
,
2
6
5
I
n
j
e
c
t
i
o
n
s
7
,
2
8
0
R
a
n
so
mw
a
r
e
5
0
,
8
1
1
B
a
c
k
-
d
o
o
r
6
1
,
6
5
0
D
D
o
S
1
0
5
M
I
TM
3
.
2
.
Da
t
a
pre
-
pro
ce
s
s
ing
Miss
in
g
v
alu
es,
to
o
m
an
y
f
ea
tu
r
es,
d
u
p
licatio
n
o
f
b
eh
av
io
r
,
a
n
d
o
th
er
is
s
u
es
d
im
in
is
h
th
e
p
er
f
o
r
m
an
ce
o
f
t
h
e
m
o
d
els
u
s
ed
in
th
ese
d
ata
s
am
p
les.
F
o
r
th
e
p
u
r
p
o
s
e
o
f
d
ata
b
alan
cin
g
an
d
class
if
ier
p
er
f
o
r
m
an
ce
im
p
r
o
v
em
e
n
t,
we
em
p
lo
y
ed
class
weig
h
ts
.
A
r
an
k
in
g
c
o
lu
m
n
is
in
clu
d
ed
in
th
e
T
o
N
-
I
o
T
s
am
p
les
f
o
r
ea
ch
attac
k
s
u
b
cl
ass
ty
p
e,
tak
in
g
i
n
to
ac
c
o
u
n
t
its
g
en
er
al
b
eh
av
io
r
a
n
d
v
u
ln
er
ab
ilit
y
s
tate.
T
h
e
d
ata
was sh
o
wn
as a
C
SV f
i
le.
T
h
is
d
ataset
m
o
s
tly
in
clu
d
es n
in
e
im
p
o
r
ta
n
t ty
p
es o
f
attac
k
s
.
T
h
e
n
o
t a
n
u
m
b
er
(
Na
N)
class
,
o
r
r
ec
o
r
d
s
m
is
s
in
g
k
ey
i
n
f
o
r
m
atio
n
,
was c
u
lled
f
r
o
m
th
e
d
atab
ase.
Data
s
ets
co
n
tain
in
g
ca
teg
o
r
ic
al
in
f
o
r
m
atio
n
will
n
ee
d
t
o
h
av
e
it
c
o
n
v
e
r
ted
t
o
n
u
m
er
i
ca
l
v
alu
es
b
ef
o
r
e
d
ee
p
lear
n
i
n
g
tec
h
n
iq
u
es
m
ay
b
e
ap
p
lied
.
T
h
e
s
cik
it
-
lear
n
p
ac
k
ag
e'
s
lab
el
en
co
d
er
class
was
u
s
ed
f
o
r
th
is
co
n
v
er
s
io
n
.
A
n
u
m
er
ical
n
u
m
b
er
was
g
iv
e
n
to
ea
ch
attr
ib
u
te
v
alu
e.
Var
io
u
s
lev
els
o
f
th
e
s
p
ec
im
e
n
'
s
attr
ib
u
tes
ar
e
m
ea
s
u
r
ed
in
t
h
e
d
ataset.
T
o
p
r
o
p
er
ly
e
v
alu
ate
t
h
ese
s
p
ec
im
en
s
u
s
in
g
d
ee
p
le
ar
n
in
g
a
p
p
r
o
ac
h
es,
th
e
tast
er
d
ata
n
ee
d
s
to
b
e
s
p
o
k
en
in
a
c
o
n
s
is
ten
t
p
ar
ticu
lar
m
ea
s
u
r
e.
T
h
is
was
ac
h
iev
ed
b
y
a
p
p
ly
in
g
th
e
Z
-
s
co
r
e
n
o
r
m
aliza
tio
n
tech
n
i
q
u
e
to
th
e
d
ataset
u
s
in
g
th
e
s
tan
d
ar
d
s
ca
ler
class
in
th
e
li
b
r
ar
y
.
T
h
e
d
ataset's
o
b
s
er
v
atio
n
s
wer
e
th
e
n
n
o
r
m
alize
d
with
a
m
ea
n
a
v
er
ag
e
o
f
ze
r
o
an
d
a
s
tan
d
ar
d
d
ev
ia
tio
n
o
f
o
n
e.
F
ix
e
d
p
r
o
p
er
ties
lik
e
d
s
p
o
r
t,
s
r
cip
,
d
s
tip
,
an
d
s
p
o
r
t
th
at
wer
e
n
o
t
n
ee
d
ed
wer
e
also
r
em
o
v
ed
,
in
tr
o
d
u
ce
d
r
e
g
u
lar
v
ec
to
r
s
to
war
d
s
th
e
"Na
Ncla
s
s
,
"
an
d
s
et
th
e
"Na
Ncla
s
s
"
v
ar
iab
les
to
ze
r
o
.
E
ac
h
win
d
o
w
is
s
en
t
in
a
d
is
t
in
ct
p
ac
k
et
s
o
th
at
th
e
c
u
r
r
en
t
d
ata
g
r
am
ca
n
i
d
en
tify
if
an
attac
k
was tr
ig
g
er
ed
b
y
p
r
io
r
tr
an
s
m
i
s
s
io
n
s
.
3
.
2
.
1
.
Da
t
a
s
pli
t
t
ing
Par
titi
o
n
in
g
th
e
d
ata
ac
co
r
d
i
n
g
to
th
e
f
ea
tu
r
e
m
ap
p
in
g
is
an
im
p
o
r
tan
t
th
in
g
to
k
ee
p
in
m
i
n
d
.
Usi
n
g
r
an
d
o
m
izatio
n
an
d
s
tr
atif
icatio
n
,
th
e
d
ataset
was
d
iv
id
ed
in
to
tr
ain
in
g
b
esid
es
test
in
g
s
u
b
s
ets
with
th
e
s
am
e
8
0
:2
0
class
ty
p
e
s
p
lit
as
th
e
o
r
ig
in
al
d
ataset.
T
h
e
tr
ain
i
n
g
d
at
a
wer
e
u
s
ed
to
b
u
ild
p
r
ed
ictiv
e
m
o
d
els,
an
d
th
en
we
test
ed
th
em
o
n
a
s
ep
a
r
ate
s
et
to
s
ee
wh
ich
i
n
tr
u
s
io
n
d
etec
tio
n
m
o
d
el
p
er
f
o
r
m
e
d
th
e
b
est.
Ov
er
f
itti
n
g
,
h
o
wev
er
,
ca
n
o
cc
u
r
if
a
lar
g
e
am
o
u
n
t
o
f
tr
ai
n
in
g
d
ata
is
ch
o
s
en
.
T
h
is
h
ap
p
en
s
wh
en
class
i
f
ier
'
s
p
er
f
o
r
m
an
ce
o
n
u
n
s
ee
n
test
d
ata
s
u
f
f
er
s
b
ec
au
s
e
it h
as g
r
o
wn
o
v
er
ly
b
iass
ed
an
d
s
p
ec
ialized
to
th
e
tr
ain
i
n
g
d
ata.
3
.
3
.
So
lv
ing
i
m
ba
la
nced
da
t
a
A
ca
teg
o
r
ical
v
ar
iab
le
s
er
v
es
as
th
e
o
u
tp
u
t
in
a
class
if
icati
o
n
task
,
an
d
th
e
g
o
al
is
to
esti
m
ate
th
e
class
g
r
o
u
p
f
r
o
m
th
e
in
p
u
t
d
at
a.
W
h
en
o
n
e
g
r
o
u
p
o
f
p
eo
p
le
h
as
m
o
r
e
in
f
o
r
m
atio
n
th
an
an
o
th
er
,
t
h
at
g
r
o
u
p
is
co
n
s
id
er
ed
th
e
m
ajo
r
ity
,
an
d
wh
en
o
n
e
g
r
o
u
p
h
as
less
in
f
o
r
m
atio
n
,
th
at
g
r
o
u
p
is
co
n
s
id
er
ed
th
e
m
in
o
r
ity
.
W
h
en
s
o
m
e
d
ataset
class
es
ar
e
o
v
er
r
ep
r
esen
ted
co
m
p
ar
ed
t
o
o
th
e
r
s
,
th
e
d
ata
is
co
n
s
id
er
e
d
im
b
alan
ce
d
[
2
5
]
.
R
eso
lv
in
g
im
b
alan
ce
s
in
d
atas
ets
is
cr
u
cial
wh
en
u
tili
zin
g
th
em
to
b
u
ild
a
d
etec
tio
n
m
o
d
el
.
Hen
ce
,
to
b
u
ild
a
f
in
al
m
o
d
el
t
h
at
is
n
'
t
b
iass
ed
i
n
id
en
tify
in
g
s
p
ec
if
ic
k
i
n
d
s
o
f
attac
k
s
,
th
e
tr
ain
ed
with
a
s
u
itab
le
p
r
o
p
o
r
tio
n
o
f
d
ata
f
r
o
m
m
u
ltip
le
class
es.
T
h
e
im
b
alan
ce
d
d
ataset
is
s
u
e
h
as
m
u
ltip
le
s
o
lu
tio
n
s
.
Fo
r
th
e
p
u
r
p
o
s
e
o
f
o
v
er
s
am
p
lin
g
m
in
o
r
ities
,
th
e
SMOT
E
m
eth
o
d
is
em
p
lo
y
ed
in
th
is
wo
r
k
.
As
c
h
o
o
s
in
g
th
e
r
ig
h
t
s
am
p
le
r
ate
is
a
to
u
g
h
task
,
b
u
t
a
f
r
eq
u
en
t
m
ac
h
in
e
lear
n
i
n
g
m
eth
o
d
is
to
a
d
d
r
ess
class
im
b
alan
ce
p
r
o
ce
d
u
r
e.
Var
io
u
s
asp
ec
ts
,
s
u
ch
as
d
ataset
p
r
o
p
er
ties
,
alg
o
r
ith
m
s
,
an
d
p
r
o
b
lem
-
s
p
ec
if
ic
n
u
an
ce
s
,
in
f
lu
en
ce
th
e
ap
p
r
o
p
r
iate
s
am
p
le
r
ate,
s
o
th
e
r
e
ar
e
n
o
g
e
n
er
al
r
u
les
f
o
r
t
h
is
.
W
h
en
tr
ai
n
in
g
m
o
d
els,
it
is
ess
en
tial
to
m
ain
tain
a
b
alan
ce
d
d
is
tr
ib
u
tio
n
o
f
class
es;
SMOT
E
aim
s
to
d
o
th
is
.
Ov
er
f
itti
n
g
,
u
n
d
e
r
f
itti
n
g
,
o
r
p
o
o
r
m
o
d
el
p
er
f
o
r
m
a
n
ce
m
ig
h
t
r
esu
lt f
r
o
m
c
h
o
o
s
in
g
t
h
e
in
co
r
r
ec
t sam
p
le
r
ate.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
la
s
s
imb
a
la
n
ce
r
eso
lu
tio
n
in
I
o
T n
etw
o
r
ks u
s
in
g
a
d
va
n
ce
d
elk
h
erd
…
(
S
r
ika
n
th
Mu
d
iya
n
u
r
S
r
ir
a
ma
p
p
a
)
2925
B
ef
o
r
e
b
e
g
in
n
in
g
wo
r
k
o
n
cl
u
s
ter
p
r
o
b
lem
s
,
r
esear
c
h
er
s
f
r
eq
u
en
tly
em
p
lo
y
SMOT
E
tec
h
n
iq
u
es
t
o
b
alan
ce
th
eir
d
atasets
.
T
h
is
all
o
ws
th
em
to
av
o
i
d
r
ate
s
elec
tio
n
f
o
r
m
in
o
r
ity
class
es.
Gr
id
s
ea
r
ch
is
a
m
eth
o
d
f
o
r
o
p
tim
izin
g
ass
ess
m
en
t
m
ea
s
u
r
es
lik
e
ar
ea
u
n
d
er
th
e
c
u
r
v
e
(
A
UC
)
,
F1
-
s
co
r
e,
r
ec
all,
an
d
p
r
ec
is
io
n
b
y
test
in
g
o
u
t
d
i
f
f
er
en
t
p
r
e
d
eter
m
in
ed
s
am
p
le
r
ates.
An
im
p
r
o
v
ed
e
v
alu
atio
n
ac
r
o
s
s
n
u
m
e
r
o
u
s
d
ata
s
u
b
s
ets
is
p
r
o
v
id
e
d
b
y
cr
o
s
s
-
v
alid
atio
n
,
wh
ich
im
p
r
o
v
es
th
is
p
r
o
ce
d
u
r
e.
R
esear
ch
er
s
o
f
ten
u
s
e
an
iter
ativ
e
r
ef
in
e
m
en
t
m
eth
o
d
t
o
p
r
o
g
r
ess
iv
ely
r
ed
u
ce
th
e
id
ea
l
s
am
p
le
r
ate
t
h
r
o
u
g
h
s
tu
d
y
a
n
d
test
in
g
.
Kn
o
w
in
g
h
o
w
m
ac
h
i
n
e
lear
n
in
g
alg
o
r
ith
m
s
r
ea
ct
to
v
a
r
io
u
s
s
am
p
le
r
ates
is
also
v
er
y
im
p
o
r
tan
t.
Ov
er
all,
s
elec
tin
g
th
e
o
p
tim
al
s
am
p
le
d
eg
r
ee
i
n
SMOT
E
is
a
co
m
p
le
x
p
r
o
ce
s
s
th
at
ca
lls
f
o
r
em
p
ir
i
ca
l
ap
p
r
o
ac
h
es,
d
o
m
ain
k
n
o
wl
ed
g
e,
a
n
d
r
ep
ea
ted
ex
p
lo
r
atio
n
to
f
in
d
th
e
s
wee
t
s
p
o
t
th
at
wo
r
k
s
f
o
r
th
e
g
iv
e
n
d
ataset
an
d
p
r
o
b
lem
a
r
ea
.
T
o
en
s
u
r
e
d
ata
s
et
b
alan
ce
,
s
am
p
les
f
r
o
m
m
in
o
r
it
y
g
r
o
u
p
s
will
b
e
g
en
er
ated
u
s
i
n
g
th
e
p
r
o
p
o
s
ed
m
et
h
o
d
f
o
r
id
en
tify
in
g
th
e
m
o
s
t
ac
cu
r
ate
s
am
p
lin
g
r
ate
.
An
a
d
v
an
ce
d
o
p
tim
izatio
n
m
eth
o
d
,
th
e
ad
v
a
n
ce
d
elk
h
er
d
o
p
t
im
izatio
n
(
AE
HO)
alg
o
r
ith
m
,
is
u
tili
ze
d
b
y
th
is
s
o
lu
tio
n
.
3
.
3
.
1
.
Descript
io
n o
f
a
dv
a
nce
d e
lk
herd
o
ptim
iza
t
io
n
First,
th
e
co
n
s
tr
ain
ts
d
is
co
v
e
r
ed
d
u
r
in
g
len
g
th
y
ex
p
er
im
e
n
tatio
n
with
b
en
ch
m
ar
k
f
u
n
ctio
n
s
ar
e
d
is
cu
s
s
ed
,
f
o
llo
wed
b
y
a
n
ex
p
lan
atio
n
o
f
th
e
b
aseli
n
e
v
ar
iatio
n
o
f
elk
h
e
r
d
o
p
tim
izatio
n
(
E
HO
)
m
etah
eu
r
is
tics
.
T
h
en
,
a
n
u
p
g
r
ad
ed
v
e
r
s
io
n
o
f
E
HO
is
r
elea
s
ed
,
with
th
e
in
ten
tio
n
o
f
f
u
r
t
h
er
m
ax
im
izin
g
th
e
p
er
f
o
r
m
an
ce
le
v
el
o
f
o
r
ig
in
al
E
HO.
T
h
e
E
HO
m
etah
e
u
r
is
tics
ar
e
co
n
s
id
er
e
d
to
b
e
am
o
n
g
th
e
m
o
s
t
cu
r
r
en
t
p
r
o
ce
d
u
r
es
b
ec
au
s
e
E
HO
was
p
r
o
p
o
s
ed
in
ea
r
l
y
2
0
2
4
[
2
7
]
.
I
t
is
a
p
ar
t
o
f
th
e
p
o
p
u
latio
n
-
b
ased
s
tr
ateg
ies
th
at
d
r
aw
in
s
p
ir
atio
n
f
r
o
m
n
atu
r
e,
n
am
ely
th
e
m
atin
g
an
d
b
r
ee
d
i
n
g
b
eh
av
io
r
s
o
f
elk
.
T
h
er
e
ar
e
two
p
r
im
ar
y
p
h
ases
to
th
ese
ac
tiv
itie
s
:
ca
lv
in
g
.
I
n
th
e
f
ir
s
t
s
tag
e,
th
e
h
er
d
is
d
iv
id
ed
in
to
f
am
ilies
o
f
v
ar
y
i
n
g
s
izes.
T
h
e
b
u
lls
'
s
tr
u
g
g
le
f
o
r
s
u
p
r
e
m
ac
y
d
ir
ec
ts
th
e
p
r
o
ce
s
s
o
f
s
ep
ar
atio
n
,
with
th
e
s
tr
o
n
g
est
b
u
lls
m
atin
g
with
m
u
ltip
le
f
em
ales
to
estab
lis
h
f
a
m
ilies
.
I
n
th
e
s
ec
o
n
d
s
tag
e,
th
e
d
o
m
i
n
an
t m
ale
an
d
ass
o
ciate
d
f
em
ales in
ea
ch
f
a
m
ily
g
iv
e
b
ir
t
h
to
n
ew
ca
lv
es.
A
s
in
g
le
co
n
tr
o
l v
a
r
iab
le,
B
r
,
r
ep
r
esen
tin
g
th
e
f
ir
s
t
b
u
ll r
atio
in
th
e
p
o
p
u
lace
,
c
h
ar
ac
ter
izes
th
e
b
aselin
e
f
o
r
m
o
f
E
HO.
T
h
e
in
itial
p
o
p
u
latio
n
,
an
elk
h
er
d
with
b
u
lls
an
d
h
a
r
em
s
,
is
g
en
er
ated
b
e
f
o
r
e
ea
c
h
m
e
tah
eu
r
is
tic
ex
ec
u
tio
n
.
I
n
(
1
)
s
h
o
ws a
m
atr
ix
r
ep
r
esen
tin
g
th
e
h
e
r
d
E
H,
w
ith
d
im
en
s
io
n
s
n
×N
,
wh
e
r
e
N
is
th
e
h
er
d
s
ize.
=
[
1
1
2
1
⋯
1
1
2
2
2
⋯
2
⋮
1
⋮
2
⋯
⋯
⋮
]
(
1
)
E
v
er
y
s
in
g
le
elk
j
is
p
r
o
d
u
ce
d
r
en
d
er
in
g
to
(
2
)
.
=
+
(
−
)
×
(
0
,
1
)
(
2
)
I
n
wh
ich
u
b
b
esid
es
ar
e
th
e
b
o
u
n
d
s
o
f
t
h
e
k
ey
d
o
m
ain
,
r
esp
ec
tiv
ely
.
T
h
e
elk
s
in
th
e
g
r
o
u
p
ar
e
th
en
r
a
n
k
ed
f
r
o
m
b
est to
wo
r
s
t a
cc
o
r
d
i
n
g
t
o
th
eir
f
itn
ess
lev
els.
Du
r
in
g
t
h
e
m
atin
g
s
ea
s
o
n
,
th
e
m
ale
r
ate
B
r
is
u
s
ed
to
estab
lis
h
th
e
f
a
m
ily
u
n
it.
I
t
is
p
o
s
s
i
b
le
to
f
ir
s
t
ascer
tain
th
e
to
tal
n
u
m
b
er
o
f
f
am
ilies
b
y
=
|
×
|
.
T
h
e
f
itn
ess
v
alu
es
o
f
th
e
m
ales
in
th
e
p
o
p
u
latio
n
ar
e
u
s
ed
f
o
r
s
elec
tio
n
p
u
r
p
o
s
es.
T
h
e
s
tr
o
n
g
est
m
ales
in
a
h
er
d
f
i
g
h
t
f
o
r
d
o
m
in
an
ce
,
lead
in
g
t
o
f
am
ilies
.
T
h
e
to
p
B
elk
s
in
ter
m
s
o
f
f
itn
ess
ar
e
ca
ll
ed
b
u
lls
,
r
ep
r
esen
tin
g
th
is
elite
g
r
o
u
p
.
T
h
er
ef
o
r
e,
in
o
r
d
er
to
s
tar
t
f
a
m
ilies
,
th
e
b
u
lls
in
B
f
ig
h
t.
Usi
n
g
th
eir
f
itn
ess
s
co
r
es
r
el
ativ
e
to
th
e
cu
m
u
lativ
e
f
itn
ess
,
th
e
b
u
lls
in
B
ar
e
ass
ig
n
ed
f
em
ales
u
s
i
n
g
th
e
r
o
u
lette
wh
ee
l
tech
n
iq
u
e.
I
t
is
s
p
ec
if
ically
ass
ig
n
ed
a
p
r
o
b
ab
ilit
y
f
o
r
ea
c
h
g
u
y
in
B
.
,
wh
ich
is
s
tr
o
n
g
-
m
in
d
e
d
b
y
its
co
m
p
lete
f
itn
ess
m
ar
k
ed
b
y
(
)
,
ex
clu
d
e
d
f
r
o
m
co
n
s
id
er
atio
n
d
u
e
to
th
e
to
tal
f
itn
ess
o
f
all
m
ales,
as seen
in
(
3
)
.
=
(
)
∑
(
)
=
1
(
3
)
I
n
p
h
ase,
o
f
ev
er
y
f
am
ily
,
la
b
eled
(
+
1
)
,
is
f
ash
io
n
ed
b
ased
ch
ief
ly
o
n
f
r
o
m
th
e
p
ater
n
al
b
u
ll
ℎ
b
esid
es
elk
,
d
e
n
o
ted
as
(
)
.
I
n
ca
s
es
wh
er
e
th
e
ca
lf
(
+
1
)
u
s
es
th
e
s
am
e
in
d
ex
(
i)
as
th
e
f
am
ily
'
s
p
atr
ilin
ea
l b
u
ll a
n
d
is
cr
ea
ted
u
s
in
g
(
4
)
.
(
+
1
)
=
(
)
+
.
(
(
)
−
(
)
)
(
4
)
T
h
e
r
ate
o
f
p
ass
in
g
o
n
q
u
alities
f
r
o
m
th
e
ar
b
itra
r
ily
p
ick
ed
elk
in
th
e
p
o
p
u
lace
ca
n
b
e
f
o
u
n
d
u
s
in
g
(
4
)
,
wh
e
r
e
α
ca
n
b
e
an
y
v
alu
e
b
etwe
en
0
an
d
1
(
)
.
Gr
ea
ter
v
alu
es
o
f
α
lead
to
a
g
r
ea
ter
lik
elih
o
o
d
o
f
ar
b
itra
r
y
au
g
m
en
ts
d
iv
e
r
s
if
icatio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
3
,
J
u
n
e
2
0
2
6
:
2
9
2
0
-
2
9
3
4
2926
An
o
th
er
p
o
s
s
ib
ilit
y
is
th
at
th
e
m
o
th
er
an
d
ca
lf
s
h
ar
e
th
e
s
am
e
in
d
ex
.
T
h
en
(
+
1
)
is
g
o
in
g
to
b
e
d
er
iv
ed
f
r
o
m
m
o
th
er
an
d
th
e
f
ath
er
ℎ
b
ased
o
n
th
e
(
5
)
.
(
+
1
)
=
(
)
+
(
ℎ
(
)
−
(
)
)
+
(
(
)
−
(
)
)
(
5
)
W
h
er
e
(
+
1
)
in
th
e
+
1
-
th
r
estatem
en
t,
ℎ
is
th
e
b
u
ll
o
f
th
e
j
-
th
h
ar
em
b
esid
es
r
em
ain
s
th
e
in
d
ex
o
f
a
r
an
d
o
m
l
y
ch
o
s
en
b
u
ll;
th
is
is
b
ec
au
s
e,
in
th
e
wild
,
th
er
e'
s
a
ce
r
tain
ch
an
ce
th
at
th
e
b
u
lls
in
th
e
h
er
d
if
th
e
b
u
ll
d
id
n
'
t p
r
o
tect
h
er
p
r
o
p
er
ly
.
Fin
ally
,
th
e
v
ar
iab
les
an
d
,
wh
ich
ca
n
tak
e
o
n
v
alu
es b
etwe
en
0
an
d
2
,
ar
e
u
s
ed
to
r
an
d
o
m
ly
ch
o
o
s
e
th
e
r
atio
o
f
tr
aits
p
ass
ed
d
o
wn
f
r
o
m
p
r
ev
io
u
s
ly
p
r
o
d
u
ce
d
ca
lv
es.
B
u
lls
,
f
em
ales,
an
d
ca
lv
es
f
r
o
m
all
th
e
d
if
f
er
en
t
f
am
ilies
ar
e
g
ath
er
ed
to
g
eth
er
in
th
e
n
e
x
t
s
tag
e.
I
n
th
is
s
y
s
tem
,
th
e
elk
s
ar
e
r
an
k
ed
ac
co
r
d
in
g
to
th
eir
f
itn
ess
lev
el,
with
th
e
b
est
elk
s
b
ein
g
k
ep
t
f
o
r
t
h
e
n
ex
t
g
en
er
atio
n
.
I
m
p
r
o
v
ed
el
k
h
er
d
o
p
tim
izatio
n
alg
o
r
ith
m
:
a
lo
t
o
f
test
in
g
with
b
en
ch
m
a
r
k
f
u
n
ctio
n
s
h
as
r
ev
ea
led
th
at
th
e
b
aselin
e
E
HO,
wh
ich
is
an
in
n
o
v
ativ
e
m
eth
o
d
,
m
ig
h
t
b
e
ev
en
b
etter
.
I
n
o
r
d
er
to
im
p
r
o
v
e
t
h
e
s
ea
r
ch
s
p
ac
e
co
v
e
r
ag
e,
th
is
p
u
b
licatio
n
r
ec
o
m
m
e
n
d
s
in
cl
u
d
in
g
a
le
ar
n
in
g
tech
n
iq
u
e
(
q
u
asi
-
r
e
f
lex
iv
e
lear
n
in
g
(
QR
L
)
)
d
u
r
in
g
th
e
p
r
o
ce
d
u
r
e'
s
in
itial
izatio
n
s
tag
e
[
2
8
]
.
R
eg
ar
d
in
g
ea
ch
v
ar
iab
le
(
)
,
a
q
u
asi
-
r
ef
lex
iv
e
-
o
p
p
o
s
ite
lim
it (
)
will b
e
r
en
d
e
r
in
g
to
(
6
)
.
=
(
+
2
,
)
(
6
)
W
h
er
e
ag
r
ee
s
to
th
e
r
a
n
d
o
m
s
u
m
f
r
o
m
[
+
2
,
]
.
W
h
en
s
tar
tin
g
an
E
HO
alg
o
r
ith
m
,
th
e
f
ir
s
t
s
tep
is
to
g
en
er
ate
NP/2
s
o
lu
tio
n
s
u
s
in
g
th
e
QR
L
tech
n
iq
u
e
,
all
with
o
u
t
ad
d
in
g
co
m
p
lex
ity
to
th
e
al
g
o
r
ith
m
i
n
ter
m
s
o
f
f
itn
ess
f
u
n
ctio
n
ev
alu
atio
n
s
(
FF
E
s
)
.
T
h
is
is
a
ty
p
ical
w
ay
to
ass
ess
th
e
co
m
p
lex
ity
o
f
m
etah
eu
r
is
tic
alg
o
r
ith
m
s
,
s
in
ce
co
m
p
u
tin
g
th
e
f
itn
ess
f
u
n
ctio
n
is
th
e
co
s
tlies
t
p
r
o
ce
s
s
th
at
o
cc
u
r
s
d
u
r
in
g
al
g
o
r
ith
m
ex
ec
u
tio
n
.
Her
e
is
th
e
r
ec
o
m
m
en
d
ed
m
eth
o
d
f
o
r
in
itializatio
n
.
Alg
o
r
ith
m
1
ex
p
lain
s
th
e
s
tag
es
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el.
Alg
o
r
ith
m
1
.
QR
L
in
itializatio
n
p
r
ac
tice
Stag
e
1
:
co
n
s
tr
u
ct
s
tar
tin
g
p
o
p
u
lace
Pin
it h
av
in
g
NP/2
in
d
iv
i
d
u
als th
r
o
u
g
h
tr
a
d
itio
n
al
E
HO
in
itializatio
n
s
ch
em
e
p
r
o
v
id
ed
in
(
2
)
Stag
e
2
:
co
n
s
tr
u
ct
QR
L
p
o
p
u
l
ac
e
Pq
r
u
p
o
n
Pin
it b
y
em
p
lo
y
in
g
th
e
(
6
)
Stag
e
3
:
co
n
s
tr
u
ct
f
in
al
i
n
itial p
o
p
u
lace
P a
s
u
n
io
n
o
f
Pin
it a
n
d
Pq
r
(
P
∪
Pq
r
)
Stag
e
4
:
co
m
p
u
te
th
e
f
itn
ess
s
co
r
e
f
o
r
ev
er
y
s
o
lu
tio
n
i
n
P
Stag
e
5
:
ar
r
an
g
e
all
s
o
lu
tio
n
s
b
elo
n
g
in
g
to
P with
r
esp
ec
t to
th
eir
f
itn
ess
s
co
r
es
Star
tin
g
f
r
o
m
th
e
in
itializatio
n
s
tag
e
an
d
co
n
tin
u
in
g
th
r
o
u
g
h
o
u
t
th
e
m
etah
e
u
r
is
tic'
s
ex
ec
u
tio
n
,
th
e
wo
r
s
t
s
o
lu
tio
n
is
r
em
o
v
ed
an
d
s
u
b
s
titu
ted
with
th
e
QR
L
in
v
er
s
e
o
f
t
h
e
b
est d
is
tin
ct
(
d
ir
ec
t
ed
b
est
m
eth
o
d
)
f
o
r
ea
ch
iter
atio
n
.
T
h
e
co
m
p
lica
tio
n
o
f
th
e
b
aselin
e
tech
n
iq
u
e
ca
lcu
lated
in
FF
E
s
i
s
n
o
t
in
cr
ea
s
ed
b
y
th
is
in
tr
o
d
u
ce
d
a
d
ju
s
tm
en
t
b
ec
au
s
e
th
e
f
itn
ess
s
co
r
es
ar
e
n
o
t
r
e
v
iewe
d
.
T
h
e
E
HO
also
h
a
d
a
n
o
th
er
m
o
d
if
icatio
n
th
at
was
p
r
o
m
p
ted
b
y
a
g
en
eti
c
alg
o
r
ith
m
[
2
9
]
.
T
h
e
s
ea
r
ch
c
en
ter
o
n
th
e
t
o
p
s
o
lu
tio
n
s
d
is
c
o
v
er
ed
th
u
s
f
a
r
as
th
e
iter
atio
n
s
p
r
o
g
r
ess
an
d
th
e
p
r
o
ce
d
u
r
e
s
tar
ts
to
co
n
v
e
r
g
e.
At
th
e
en
d
o
f
th
e
m
ax
_
iter
/2
r
o
u
n
d
s
—
wh
er
e
m
ax
_
iter
is
th
e
m
ax
i
m
u
m
n
u
m
b
er
o
f
iter
atio
n
s
—
th
e
alg
o
r
i
th
m
is
s
p
ed
u
p
b
y
s
wap
p
in
g
o
u
t
th
e
s
o
lu
ti
o
n
wit
h
th
e
s
ec
o
n
d
-
s
co
r
e
f
o
r
a
n
ew
in
d
iv
id
u
al.
T
h
is
n
ew
in
d
iv
i
d
u
al
is
a
cr
o
s
s
o
f
th
e
two
b
est
s
o
lu
tio
n
s
o
b
tain
ed
af
ter
p
u
t
o
n
an
o
p
er
ato
r
,
wh
e
r
e
p
c
is
th
e
p
ar
am
eter
o
f
th
e
in
d
iv
i
d
u
al
an
d
th
e
p
r
o
b
ab
ilit
y
o
f
cr
o
s
s
o
v
er
p
er
g
en
e
is
0
.
1
.
T
h
is
u
p
d
ate
s
p
ee
d
s
u
p
th
e
alg
o
r
ith
m
b
y
r
ein
f
o
r
cin
g
th
e
ex
p
lo
itatio
n
.
On
ce
a
g
ain
,
th
is
ch
an
g
e
d
o
esn
'
t
ad
d
an
y
e
x
tr
a
w
o
r
k
f
o
r
th
e
FF
E
s
b
ec
au
s
e
it
d
o
esn
'
t
ad
d
an
y
f
u
r
th
er
f
itn
ess
v
alu
e
ca
lcu
latio
n
s
.
As
a
r
esu
lt,
th
e
m
o
d
if
ied
E
HO
an
d
th
e
b
aselin
e
E
HO
ar
e
eq
u
ally
co
m
p
le
x
.
T
h
e
m
o
d
if
ied
v
e
r
s
io
n
o
f
E
HO
is
n
o
w
k
n
o
wn
as
ac
ce
ler
ated
g
u
id
e
d
Alg
o
r
ith
m
2
s
h
o
ws th
e
p
s
eu
d
o
co
d
e
f
o
r
t
h
is
o
p
tim
izatio
n
iter
atio
n
,
wh
e
r
e
t is a
v
ar
iab
le.
Alg
o
r
ith
m
2
.
AE
HO
p
s
eu
d
o
c
o
d
e
Synthesize initial population P with QRL as explained in Algorithm 1
t = 0
while (t < max_iter) do
Sort solutions within P regarding their fitness scores
for each solution X in P do
Employ EHO search to conduct optimization
Synthesize fresh solution as QRL opposite of the current best
Replace the worst solution within P with this fresh solution
if (t > max_iter 2) then
Produce new individual as the hybrid of the best pair of individuals by applying
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
la
s
s
imb
a
la
n
ce
r
eso
lu
tio
n
in
I
o
T n
etw
o
r
ks u
s
in
g
a
d
va
n
ce
d
elk
h
erd
…
(
S
r
ika
n
th
Mu
d
iya
n
u
r
S
r
ir
a
ma
p
p
a
)
2927
uniform crossover operator having pc = 0.1
Replace the second
-
worst solution with this novel hybrid individual
end if
end for
t = t + 1
end while
return the best solution found within P
3
.
4
.
F
e
a
t
ure
s
elec
t
io
n
Featu
r
e
s
elec
tio
n
is
a
m
eth
o
d
f
o
r
d
ec
r
ea
s
in
g
a
d
ataset'
s
d
i
m
en
s
io
n
ality
o
r
th
e
am
o
u
n
t
o
f
f
ea
tu
r
es
it
co
n
tain
s
.
I
t a
ch
iev
es th
is
b
y
s
if
tin
g
th
r
o
u
g
h
th
e
r
aw
d
ata
in
s
ea
r
ch
o
f
u
s
ef
u
l a
n
d
p
er
tin
e
n
t in
f
o
r
m
atio
n
.
I
t ta
k
es
th
ese
f
ea
tu
r
es
an
d
u
s
es
th
em
to
g
en
er
ate
a
s
m
aller
s
et
o
f
f
ea
tu
r
es.
C
o
n
v
en
tio
n
al
wis
d
o
m
h
o
ld
s
th
at
ea
ch
class
in
a
t
r
ain
in
g
s
et
is
eq
u
ally
im
p
o
r
tan
t;
n
ev
er
th
eless
,
th
is
ig
n
o
r
es
th
e
p
o
s
s
ib
ilit
y
o
f
an
u
n
e
v
en
d
is
tr
ib
u
tio
n
o
f
tr
ain
in
g
d
ata.
T
h
is
m
eth
o
d
u
n
d
er
s
to
o
d
th
e
im
p
o
r
tan
ce
o
f
th
e
ch
ar
ac
ter
is
tics
in
th
e
b
alan
ce
d
d
ata
b
y
u
s
in
g
a
weig
h
t
ad
ju
s
tm
en
t
m
ec
h
a
n
is
m
in
r
an
d
o
m
f
o
r
est
.
T
h
e
m
eth
o
d
id
en
tifie
d
t
h
e
ef
f
icac
y
o
f
a
s
p
lit
in
d
iv
id
in
g
th
e
to
tal
s
am
p
les o
f
a
p
ar
ticu
lar
class
in
a
p
ar
ticu
lar
n
o
d
e
af
ter
e
x
am
in
in
g
t
h
e
Gin
i im
p
u
r
ity
.
3
.
5
.
Cla
s
s
if
ica
t
io
n wit
h dee
p B
iL
ST
M
E
v
en
wh
ile
r
ec
u
r
r
en
t
n
eu
r
al
n
etwo
r
k
s
(
R
NNs)
ar
e
g
r
ea
t
at
r
etr
iev
in
g
h
is
to
r
ical
d
ata,
th
ey
ca
n
'
t
h
an
d
le
e
x
p
lo
d
i
n
g
g
r
ad
ien
ts
v
er
y
well,
th
er
e
f
o
r
e
th
ey
'
r
e
n
o
t
v
er
y
u
s
ef
u
l
f
o
r
l
o
n
g
-
te
r
m
task
s
.
T
h
e
s
o
lu
tio
n
t
o
th
is
p
r
o
b
lem
was
to
u
s
e
d
ee
p
B
iLST
M,
an
ex
ten
s
io
n
o
f
B
iLST
M,
a
s
p
ec
ialized
L
STM
v
er
s
io
n
.
B
y
co
llectin
g
p
er
s
is
ten
t
p
atter
n
s
in
s
p
atial
-
tem
p
o
r
al
in
f
o
r
m
atio
n
,
th
is
ar
c
h
itectu
r
e
is
g
r
ea
t
at
an
aly
zin
g
s
eq
u
en
tial
s
p
atial
f
ea
tu
r
es
in
th
e
im
m
ed
iate
v
icin
ity
.
Fig
u
r
e
2
s
h
o
ws
th
e
d
ee
p
B
iLST
M
m
o
d
el'
s
ar
ch
itectu
r
e
in
d
etail,
h
ig
h
lig
h
tin
g
its
ca
p
ac
ity
to
th
o
r
o
u
g
h
ly
an
aly
ze
an
d
ca
te
g
o
r
iz
e
th
e
T
o
N
-
I
o
T
d
ataset'
s
m
u
lti
-
class
as
s
au
lts
.
Fig
u
r
e
2
.
Ar
c
h
itectu
r
es o
f
L
S
T
M
[
3
0
]
T
h
e
th
r
ee
g
ates
—
g
ate,
th
e
f
o
r
g
et
g
ate,
an
d
th
e
o
u
tp
u
t
g
ate
—
ar
e
wh
at
m
ak
e
L
STM
th
e
b
asis
o
f
th
e
d
ee
p
B
iLST
M
m
o
d
el.
I
n
(
7
)
is
a
d
etailed
d
escr
ip
tio
n
o
f
a
n
L
STM
u
n
it
.
=
(
Υ
⨁
+
⊕
ℎ
−
1
+
)
(
7
)
T
h
is
is
wh
er
e
b
esid
es
s
tan
d
f
o
r
th
e
weig
h
t
m
atr
ix
,
f
o
r
th
e
b
ias
ter
m
,
f
o
r
th
e
in
p
u
t
d
at
a
at
a
g
iv
en
tim
e,
⨁
f
o
r
m
u
ltip
licatio
n
p
r
o
ce
d
u
r
e,
an
d
σ
f
o
r
th
e
f
u
n
ctio
n
.
(
t)
,
b
esid
es
ℎ
t
−
1
r
ep
r
esen
ts
th
e
r
esu
lt
o
f
th
e
L
STM
u
n
it
th
at
ca
m
e
b
ef
o
r
e
i
t.
T
h
e
in
p
u
t
g
ate
p
la
y
s
a
p
iv
o
t
al
r
o
le
in
d
eter
m
in
in
g
wh
ich
d
ata
p
o
in
ts
f
r
o
m
th
e
p
r
ev
io
u
s
u
n
it r
eq
u
ir
e
ed
itin
g
.
T
h
e
f
o
r
g
et
g
ate
is
f
o
r
m
u
lated
i
n
(
8
)
.
=
(
Υ
.
+
.
ℎ
−
1
+
)
(
8
)
W
h
er
e
s
tan
d
s
f
o
r
g
ate,
wh
ich
is
in
ch
ar
g
e
o
f
ca
lcu
latin
g
th
e
im
p
o
r
tan
ce
o
f
d
ata
an
d
er
asi
n
g
p
r
ev
io
u
s
d
ata.
T
h
e
ca
n
d
id
ate
c
ell
s
tate
is
g
iv
en
in
(
9
)
an
d
th
e
ce
ll st
ate
u
p
d
ate
is
d
ef
in
ed
in
(
1
0
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
3
,
J
u
n
e
2
0
2
6
:
2
9
2
0
-
2
9
3
4
2928
̃
=
ℎ
(
Υ
.
+
.
ℎ
−
1
+
)
(
9
)
=
⨀
−
1
+
⨀
̃
(
1
0
)
W
h
er
e
th
e
p
u
b
lic
o
f
th
e
ap
p
lican
t
is
u
n
wav
er
in
g
th
r
o
u
g
h
th
e
o
p
er
atio
n
o
f
th
e
tan
g
en
t
f
u
n
ctio
n
,
as
d
em
o
n
s
tr
ated
i
n
(
10
)
.
T
h
e
n
,
th
e
s
tate
is
ass
ess
ed
an
d
p
r
o
v
id
i
n
g
i
n
eq
u
atio
n
,
wh
er
e
⊙
ch
a
r
ac
ter
izes
m
u
ltip
licatio
n
.
T
h
e
o
u
tp
u
t g
at
e
is
co
m
p
u
ted
in
(
1
1
)
an
d
th
e
h
id
d
en
s
tate
o
u
t
p
u
t is g
iv
en
in
(
1
2
)
.
=
(
Υ
+
ℎ
−
1
+
)
(
1
1
)
ℎ
=
⨀
ta
n
h
(
)
(
1
2
)
Giv
en
th
e
o
u
tp
u
t
o
f
th
e
L
ST
M
u
n
it,
d
e
n
o
ted
as
h
t,
as
s
h
o
wn
in
(
1
2
)
,
th
e
o
u
t
p
u
t
g
ate
g
t
is
co
m
p
u
ted
u
s
in
g
(
1
1
)
.
T
h
e
b
aselin
e
L
STM
m
o
d
el
u
s
es
h
is
to
r
ical
d
a
ta
ex
clu
s
iv
ely
to
f
o
r
ec
ast
h
u
m
an
ac
tio
n
s
in
th
e
p
r
esen
t.
T
h
e
r
is
k
o
f
m
is
s
in
g
i
m
p
o
r
tan
t
d
etails
wh
en
d
ata
is
o
n
ly
an
aly
ze
d
in
o
n
e
way
is
o
b
v
i
o
u
s
.
Fig
u
r
e
3
d
em
o
n
s
tr
atio
n
s
th
at
t
h
e
B
iLST
M
co
n
s
is
ts
o
f
two
L
STM
l
ay
er
s
th
at
ca
n
f
u
n
ctio
n
in
b
o
th
d
ir
ec
tio
n
s
.
T
h
e
f
o
llo
win
g
is
th
e
f
o
r
m
u
latio
n
o
f
,
th
e
o
u
tp
u
t in
th
e
d
ee
p
B
iLST
M
,
is
g
iv
en
in
(
1
3
)
[
3
1
]
.
=
[
ℎ
⃗
⃗
⃗
ℎ
⃖
⃗
⃗
⃗
⃗
]
(
1
3
)
T
h
e
f
o
r
war
d
b
esid
es
b
ac
k
war
d
co
n
s
eq
u
en
ce
s
L
STM
u
n
its
is
ch
ar
ac
ter
ized
b
y
th
e
s
y
m
b
o
l
s
ℎ
⃖
⃗
⃗
⃗
an
d
ℎ
⃗
⃗
⃗
.
I
n
o
r
d
er
to
g
et
th
e
o
u
t
p
u
t
,
th
ese
two
L
STM
u
n
its
wo
r
k
to
g
eth
er
.
T
h
e
c
o
r
e
id
ea
b
eh
in
d
R
NN
is
th
at
wh
en
d
ata
is
f
ed
in
to
th
e
n
eu
r
al
n
etwo
r
k
,
t
h
e
wh
o
le
d
ataset
is
n
o
t
p
r
o
v
i
d
ed
at
o
n
ce
,
b
u
t
r
at
h
er
in
a
s
eq
u
en
tial
f
ash
io
n
,
ess
en
tially
ad
d
in
g
th
e
tem
p
o
r
al
v
ar
iab
le
to
g
eth
e
r
.
I
f
th
er
e
is
a
s
et
o
f
in
p
u
t
v
alu
es,
th
e
p
r
o
ce
s
s
m
ay
s
tar
t
b
y
f
ee
d
in
g
th
e
n
etwo
r
k
t
h
e
b
e
g
in
n
in
g
v
al
u
e
a
n
d
o
b
tai
n
in
g
t
h
e
ap
p
r
o
p
r
iate
o
u
tp
u
t.
T
h
en
,
t
h
e
f
o
llo
win
g
in
p
u
t is f
ed
alo
n
g
s
id
e
th
e
p
r
ev
io
u
s
o
u
tp
u
t t
o
g
et
th
e
f
o
llo
win
g
o
u
tco
m
e.
Fig
u
r
e
3
.
Stru
ctu
r
e
d
iag
r
am
o
f
d
ee
p
B
iLST
M
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
W
in
d
o
ws
1
1
o
p
er
atin
g
s
y
s
tem
h
av
in
g
3
2
GB
o
f
DDR4
R
AM
p
o
wer
e
d
b
y
a
C
o
r
e
i7
-
1
2
7
0
0
p
r
o
ce
s
s
o
r
is
u
s
ed
f
o
r
s
im
u
latio
n
en
v
ir
o
n
m
en
t
.
T
h
e
d
esk
to
p
c
o
m
p
u
ter
'
s
h
ar
d
d
is
c
ca
p
ac
ity
is
2
ter
ab
y
tes.
T
h
e
NVI
DI
A
R
T
X
3
0
8
0
is
th
e
GPU
s
u
p
p
o
r
t
in
g
th
e
g
r
ap
h
ics ca
r
d
.
An
in
d
icato
r
o
f
h
o
w
well
a
m
ac
h
in
e
lear
n
in
g
class
if
icatio
n
s
y
s
tem
is
d
o
in
g
is
th
e
co
n
f
u
s
io
n
m
atr
ix
[
3
2
]
.
T
h
e
m
o
d
el'
s
p
er
f
o
r
m
an
ce
ca
n
b
e
ev
alu
ated
in
v
ar
io
u
s
class
if
icatio
n
o
u
tco
m
es
u
s
in
g
th
is
m
eth
o
d
,
wh
ich
is
b
ased
o
n
th
e
ass
o
cia
tio
n
am
o
n
g
r
ea
l
class
es
b
esid
es
th
e
clas
s
es
f
o
r
eto
ld
b
y
th
e
class
ical
(
Fig
u
r
e
4
)
.
T
h
e
x
-
ax
is
s
h
o
ws
th
e
p
r
e
d
ictab
le
o
u
tco
m
es,
b
esid
es
th
e
y
-
a
x
is
ex
p
r
ess
io
n
s
th
e
o
b
s
er
v
ed
o
u
tco
m
es.
I
n
a
t
r
u
e
p
o
s
itiv
e
(
T
P)
s
itu
atio
n
,
th
e
a
ctu
al
an
d
f
o
r
ec
ast
class
if
icatio
n
s
ar
e
i
n
ag
r
ee
m
en
t;
in
a
t
r
u
e
n
e
g
ativ
e
(
T
N)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
la
s
s
imb
a
la
n
ce
r
eso
lu
tio
n
in
I
o
T n
etw
o
r
ks u
s
in
g
a
d
va
n
ce
d
elk
h
erd
…
(
S
r
ika
n
th
Mu
d
iya
n
u
r
S
r
ir
a
ma
p
p
a
)
2929
s
itu
atio
n
,
th
e
ac
tu
al
b
esid
es
p
r
ed
icted
class
if
icatio
n
s
ar
e
at
o
d
d
s
with
o
n
e
an
o
th
e
r
.
I
t
is
ca
l
led
a
f
alse
p
o
s
itiv
e
(
FP
)
wh
en
th
e
ac
t
u
al
class
if
icatio
n
is
wr
o
n
g
b
u
t
th
e
an
ticip
ated
class
if
icatio
n
is
r
ig
h
t.
W
h
en
th
e
an
ticip
ate
d
class
if
icatio
n
is
wr
o
n
g
b
u
t th
e
class
if
icatio
n
is
r
ig
h
t,
it's
ca
lle
d
a
f
alse n
eg
ativ
e
(
FN)
.
4
.
1
.
E
v
a
lua
t
i
o
n m
et
rics
I
n
o
r
d
er
to
d
eter
m
in
e
h
o
w
wel
l
an
d
ac
cu
r
ately
th
e
s
u
g
g
ested
m
o
d
el
id
e
n
tifie
s
attac
k
s
,
th
e
e
v
alu
atio
n
p
r
o
ce
s
s
is
o
f
th
e
h
ig
h
est
im
p
o
r
tan
ce
.
A
co
n
f
u
s
io
n
m
atr
i
x
is
u
s
ed
to
m
ea
s
u
r
e
th
e
s
u
g
g
ested
class
ical
in
ter
m
s
o
f
ac
cu
r
ac
y
,
r
ec
all,
p
r
ec
is
io
n
,
b
esid
es
F1
-
s
co
r
e.
R
esear
ch
es
ca
n
o
b
jectiv
ely
e
v
al
u
ate
th
e
s
y
s
tem
'
s
p
er
f
o
r
m
an
ce
o
n
th
ese
im
p
o
r
tan
t
m
etr
ics
b
y
u
s
in
g
th
is
m
atr
ix
.
R
esu
lts
f
r
o
m
ac
cu
r
ately
an
d
e
f
f
icien
tly
d
etec
tin
g
an
d
id
en
tif
y
in
g
ass
au
lts
u
s
in
g
th
e
s
u
g
g
ested
m
eth
o
d
o
lo
g
y
wer
e
en
co
u
r
ag
in
g
.
On
b
in
ar
y
attac
k
s
,
Fig
u
r
e
5
s
h
o
ws th
e
s
u
g
g
ested
B
iLST
M
'
s
co
n
f
u
s
io
n
m
atr
ix
.
Fig
u
r
e
4
.
C
o
n
f
u
s
io
n
m
atr
i
x
Fig
u
r
e
5
.
C
o
n
f
u
s
io
n
m
atr
i
x
o
f
th
e
p
r
o
p
o
s
ed
B
iLST
M
4
.
2
.
Va
lid
a
t
i
o
n
a
na
ly
s
is
o
f
pro
po
s
ed
deep
lea
rning
m
o
del f
o
r
bin
a
ry
cla
s
s
T
h
e
ef
f
ec
tiv
en
ess
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
is
v
alid
ated
with
ex
is
tin
g
tech
n
iq
u
es
s
u
ch
as
Ma
r
k
o
v
r
an
d
o
m
f
ield
(
MRF
)
[
1
6
]
,
1
D
-
C
NN
an
d
L
STM
[
1
8
]
,
B
iGR
U
[
2
0
]
,
an
d
Ad
aBo
o
s
t
[
2
2
]
.
T
h
e
ex
is
tin
g
tech
n
iq
u
es
u
s
e
v
ar
io
u
s
d
atasets
an
d
th
er
ef
o
r
e
t
h
e
r
esear
ch
wo
r
k
im
p
lem
en
ts
th
ese
m
o
d
e
ls
o
n
d
ataset.
T
h
e
r
esu
lts
ar
e
av
er
ag
ed
in
Fig
u
r
es
6
an
d
7
.
Fig
u
r
e
6
d
em
o
n
s
tr
atio
n
s
th
e
co
m
p
ar
ativ
e
in
v
esti
g
atio
n
o
f
th
e
s
u
g
g
ested
p
er
f
ec
t.
T
h
e
MRF
[
1
6
]
an
aly
s
is
y
ield
ed
a
cla
s
s
if
ier
ac
cu
r
ac
y
o
f
9
4
.
9
8
a
n
d
a
co
r
r
esp
o
n
d
in
g
F1
-
s
co
r
e
o
f
8
9
.
0
2
.
Nex
t,
th
e
F1
-
s
co
r
e
was
9
0
.
4
1
b
esid
es
th
e
class
if
ier
ac
cu
r
ac
y
was
9
6
.
2
4
ac
c
o
r
d
in
g
to
Ad
aBo
o
s
t
[
2
2
]
.
T
h
e
class
if
ier
ac
cu
r
ac
y
f
o
r
th
e
C
NN
an
d
L
S
T
M
[
1
8
]
was
th
en
9
7
.
5
2
an
d
9
3
.
5
7
,
r
esp
ec
tiv
ely
,
f
o
r
th
e
F1
-
s
co
r
e.
T
h
e
n
,
th
e
co
r
r
esp
o
n
d
in
g
B
iGR
U
[
2
0
]
class
if
ier
ac
cu
r
ac
y
was
9
8
.
4
8
~9
5
.
8
2
.
T
h
e
d
ee
p
B
iLST
M
clas
s
if
ier
ac
cu
r
ac
y
was th
en
r
ep
o
r
ted
as 9
9
.
0
6
with
a
co
r
r
esp
o
n
d
in
g
F1
-
s
co
r
e
o
f
9
7
.
4
8
.
I
n
F
ig
u
r
e
7
s
ig
n
if
i
e
s
th
e
v
i
s
u
a
l
r
ep
r
e
s
en
ta
tio
n
o
f
d
i
f
f
er
en
t
m
o
d
el
s
in
t
er
m
s
o
f
s
ev
er
a
l
m
etr
ic
s
.
I
n
th
e
in
v
e
s
tig
at
io
n
o
f
M
R
F
[
1
6
]
t
ec
h
n
iq
u
e
p
r
e
ci
s
io
n
a
s
9
3
.
5
2
an
d
r
e
ca
ll
o
f
9
1
.
5
7
co
r
r
e
s
p
o
n
d
in
g
ly
.
T
h
en
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.