I
nte
rna
t
io
na
l J
o
urna
l o
f
I
nfo
rm
a
t
ics a
nd
Co
m
m
un
ica
t
io
n T
ec
hn
o
lo
g
y
(
I
J
-
I
CT
)
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
,
p
p
.
565
~
5
7
7
I
SS
N:
2252
-
8
7
7
6
,
DOI
:
1
0
.
1
1
5
9
1
/iji
ct
.
v
1
5
i
2
.
pp
565
-
5
7
7
565
J
o
ur
na
l
ho
m
ep
a
g
e
:
h
ttp
:
//ij
ict.
ia
esco
r
e.
co
m
Ens
emble
win
do
ws
intrusio
n
det
e
ction
sy
stem
usi
n
g
XG
B
o
o
st
a
nd
deep
l
ea
rning
P
ra
nitha
K
eda
m
ba
dy
Sh
iv
a
,
P
us
hp
a
ra
j
D
.
Sh
et
t
y
D
e
p
a
r
t
me
n
t
of
M
a
t
h
e
m
a
t
i
c
a
l
a
n
d
C
o
mp
u
t
a
t
i
o
n
a
l
S
c
i
e
n
c
e
s,
N
a
t
i
o
n
a
l
I
n
st
i
t
u
t
e
of
T
e
c
h
n
o
l
o
g
y
K
a
r
n
a
t
a
k
a
(
N
I
TK
)
,
S
u
r
a
t
h
k
a
l
,
I
n
d
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
7
,
2
0
2
5
R
ev
is
ed
J
an
7
,
2
0
2
6
Acc
ep
ted
Ma
r
3
0
,
2
0
2
6
In
tru
si
o
n
d
e
tec
ti
o
n
sy
ste
m
s
(IDS
)
a
re
c
rit
ica
l
fo
r
p
re
se
rv
in
g
th
e
Wi
n
d
o
ws
e
n
v
iro
n
m
e
n
t
fro
m
an
e
v
e
r
-
c
h
a
n
g
i
n
g
c
o
ll
e
c
ti
o
n
of
c
y
b
e
r
t
h
re
a
ts.
Cu
rre
n
t
IDS
u
se
s
d
e
e
p
lea
rn
in
g
(DL),
wh
ich
a
re
h
e
a
v
y
m
o
d
e
ls
if
u
se
d
fo
r
d
e
tec
ti
o
n
,
wh
il
e
o
th
e
rs
u
se
m
a
c
h
i
n
e
lea
rn
in
g
(M
L
)
tec
h
n
i
q
u
e
s,
wh
ich
re
q
u
ire
e
x
ter
n
a
l
fe
a
tu
re
e
x
trac
ti
o
n
.
To
re
so
lv
e
th
is
c
h
a
ll
e
n
g
e
,
th
is
p
a
p
e
r
i
n
tr
o
d
u
c
e
s
XG
B
NN
,
a
n
e
w
e
n
se
m
b
le
m
o
d
e
l
th
a
t
c
o
m
b
in
e
s
t
h
e
b
e
n
e
fit
s
of
ML
a
n
d
DL
to
id
e
n
ti
fy
a
n
d
m
it
ig
a
te
a
tt
a
c
k
s
a
g
a
in
st
Wi
n
d
o
ws
m
a
c
h
in
e
s
e
ffe
c
ti
v
e
ly
.
T
h
e
v
a
rio
u
s
ML
m
e
th
o
d
s
a
re
train
e
d
on
th
e
p
u
b
li
c
ly
a
v
a
il
a
b
le
d
a
tas
e
t
to
c
las
sify
e
ig
h
t
t
y
p
e
s
of
a
tt
a
c
k
s
in
a
Wi
n
d
o
ws
e
n
v
iro
n
m
e
n
t.
Ad
d
it
i
o
n
a
ll
y
,
d
e
e
p
n
e
u
ra
l
n
e
two
r
k
s
(DN
Ns
)
a
re
p
ro
p
o
se
d
by
o
p
ti
m
izin
g
th
e
lay
e
rs
a
n
d
h
y
p
e
r
p
a
ra
m
e
ters
to
a
c
h
iev
e
th
e
b
e
st
a
c
c
u
ra
c
y
.
Th
e
n
,
th
e
DNN
m
o
d
e
l
a
n
d
XG
Bo
o
st
m
o
d
e
l
a
re
in
teg
ra
ted
to
d
e
tec
t
i
n
tr
u
sio
n
s
by
u
ti
li
z
i
n
g
th
e
fe
a
t
u
re
e
x
trac
ti
o
n
a
b
il
it
y
of
DNN
a
n
d
p
ro
v
id
i
n
g
th
e
in
term
e
d
iate
fe
a
tu
re
s
e
x
trac
ted
fro
m
th
e
l
a
st
se
c
o
n
d
lay
e
r
of
th
e
DNN
to
t
h
e
XG
B
fo
r
c
las
sifica
ti
o
n
.
T
h
e
En
se
m
b
le
m
o
d
e
l
XG
B
NN
o
p
ti
m
ize
s
fe
a
tu
re
s
a
n
d
o
ffe
rs
b
e
tt
e
r
d
e
c
isio
n
s.
T
h
e
p
ro
p
o
se
d
m
o
d
e
l
a
c
h
iev
e
s
an
e
x
c
e
p
ti
o
n
a
l
a
c
c
u
ra
c
y
of
1
0
0
%
,
as
d
e
m
o
n
stra
ted
by
th
e
e
m
p
iri
c
a
l
re
su
lt
s,
a
n
d
o
u
t
p
e
rfo
rm
s
t
h
e
b
e
n
c
h
m
a
rk
m
o
d
e
ls
wit
h
an
imp
r
o
v
e
m
e
n
t
of
0
.
0
0
4
%
.
Th
e
p
u
r
p
o
se
of
t
h
is
st
u
d
y
is
to
h
i
g
h
li
g
h
t
t
h
e
e
ffe
c
ti
v
e
n
e
ss
of
h
y
b
ri
d
a
rc
h
it
e
c
tu
re
s
in
i
n
tru
si
o
n
d
e
tec
ti
o
n
.
T
h
e
se
ar
c
h
it
e
c
tu
re
s
o
ffe
r
a
m
o
re
ro
b
u
st,
sc
a
lab
le,
a
n
d
e
ffe
c
ti
v
e
m
e
th
o
d
to
imp
ro
v
e
th
e
se
c
u
rit
y
of
t
h
e
Wi
n
d
o
ws
sy
ste
m
a
g
a
in
st
m
o
re
so
p
h
isti
c
a
ted
a
tt
a
c
k
s.
K
ey
w
o
r
d
s
:
C
y
b
er
-
attac
k
s
Deep
n
eu
r
al
n
etwo
r
k
s
I
n
tr
u
s
io
n
d
etec
tio
n
R
an
d
o
m
f
o
r
est
XGBo
o
s
t
T
h
is
is
an
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r
th
e
CC
BY
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Pra
n
ith
a
Ked
am
b
ad
y
Sh
iv
p
p
a
Dep
ar
tm
en
t
of
Ma
th
em
atica
l
a
nd
C
o
m
p
u
tatio
n
al
Scien
ce
s
Natio
n
al
I
n
s
titu
te
of
T
ec
h
n
o
lo
g
y
Kar
n
atak
a
(
NI
T
K)
Su
r
ath
k
al,
I
n
d
ia
E
m
ail
:
p
r
an
ith
ak
s
1
2
@
g
m
ail.
c
o
m
or
p
r
an
ith
a
k
s
.
2
1
7
m
a5
0
1
@
n
itk
.
ed
u
.
i
n
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
n
u
m
b
e
r
of
lin
k
e
d
co
m
p
u
ter
s
an
d
th
e
v
o
lu
m
e
of
d
ata
ex
ch
an
g
e
d
b
etwe
en
th
em
ar
e
g
r
o
win
g
ex
p
o
n
e
n
tially
in
tan
d
em
with
o
n
g
o
in
g
tec
h
n
o
lo
g
ical
b
r
ea
k
th
r
o
u
g
h
s
.
T
h
is
c
o
n
d
itio
n
en
a
b
le
s
cy
b
er
cr
im
i
n
als
to
f
o
cu
s
th
eir
atten
tio
n
an
d
s
atis
f
y
th
eir
n
ee
d
s
.
C
y
b
er
attac
k
s
,
s
u
ch
as
m
alwa
r
e
an
d
o
th
er
n
etwo
r
k
attac
k
s
,
s
ee
k
to
u
n
d
er
m
i
n
e
th
e
s
ec
u
r
ity
,
d
e
p
e
n
d
ab
ilit
y
,
an
d
ac
ce
s
s
ib
ilit
y
of
a
s
y
s
tem
or
a
lin
k
ed
d
ev
ice
[
1
]
.
T
h
u
s
,
d
etec
tin
g
cy
b
er
attac
k
s
is
ess
en
tia
l
to
c
o
m
p
u
ter
n
etwo
r
k
an
d
co
m
p
o
n
en
t
s
ec
u
r
ity
.
C
o
n
s
eq
u
en
tly
,
n
u
m
er
o
u
s
I
n
tr
u
s
io
n
d
etec
tio
n
s
y
s
tem
s
(
I
DS
)
h
av
e
b
ee
n
p
r
o
p
o
s
ed
to
i
d
en
tify
a
n
d
d
etec
t
in
tr
u
s
io
n
s
with
in
th
e
s
y
s
tem
.
I
DS
d
etec
t,
ev
alu
ate,
an
d
r
e
p
o
r
t
an
y
s
u
s
p
icio
u
s
ac
tiv
ity
th
ey
f
in
d
in
o
r
d
er
to
p
r
e
v
en
t
u
n
a
u
th
o
r
ize
d
a
cc
ess
to
co
m
p
u
ter
n
etwo
r
k
s
an
d
s
y
s
tem
s
[
2
]
.
I
DS
m
o
n
ito
r
s
co
m
p
u
ter
an
d
n
etwo
r
k
ac
tiv
ities
,
co
llects
d
ata
ab
o
u
t
th
em
,
an
d
ch
ec
k
s
th
is
in
f
o
r
m
atio
n
f
o
r
h
ar
m
f
u
l
ac
tio
n
s
[
3
]
.
If
th
e
y
d
et
ec
t
a
s
u
s
p
icio
u
s
at
tack
or
ac
tiv
ity
,
th
ey
will
aler
t
th
e
s
y
s
tem
ad
m
in
is
tr
ato
r
[
4
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
5
6
5
-
5
7
7
566
Netwo
r
k
-
b
ased
I
DS
(
NI
DS)
a
n
d
Ho
s
t
-
b
ased
I
DS
(
HI
DS)
ar
e
two
ty
p
es
of
I
DS
an
d
ar
e
c
ateg
o
r
ized
b
ased
on
wh
er
e
th
ey
g
et
th
ei
r
d
ata.
I
DS
u
s
es
d
etec
tio
n
in
v
ar
io
u
s
way
s
,
b
u
t
two
of
th
e
m
o
s
t
co
m
m
o
n
o
n
es
ar
e
s
ig
n
atu
r
e
d
etec
tio
n
an
d
an
o
m
aly
d
etec
tio
n
[
5
]
.
By
m
o
n
ito
r
in
g
an
d
an
aly
zi
n
g
n
etwo
r
k
tr
af
f
ic,
NI
DS
d
etec
t
an
o
m
alo
u
s
ac
tiv
ity
or
n
etwo
r
k
-
b
ased
attac
k
s
.
NI
DS
ar
e
ty
p
ically
im
p
lem
en
ted
on
n
etwo
r
k
in
f
r
astru
ctu
r
e
to
s
u
p
er
v
is
e
all
n
etwo
r
k
ac
tiv
ities
.
T
h
e
NI
DS
h
as
a
m
o
n
ito
r
in
g
p
h
ase
w
h
er
e
it
co
llects
d
ata
ab
o
u
t
n
etwo
r
k
tr
af
f
ic,
s
u
ch
as
p
ac
k
et
s
ize,
n
u
m
b
er
of
p
ac
k
ets,
p
o
r
t
n
u
m
b
er
,
an
d
IP
a
d
d
r
ess
.
In
o
r
d
er
to
id
en
tif
y
m
al
icio
u
s
n
etwo
r
k
ac
tiv
ity
,
N
I
DS
em
p
lo
y
s
ig
n
atu
r
e
a
n
d
an
o
m
al
y
d
etec
tio
n
m
eth
o
d
s
to
ex
a
m
in
e
co
ll
ec
ted
d
ata.
Du
r
in
g
th
e
r
ea
ctio
n
p
h
ase,
an
y
id
en
tif
ied
attac
k
or
s
tr
an
g
e
IP
ad
d
r
e
s
s
in
th
e
n
etwo
r
k
or
ac
tiv
ity
is
co
m
m
u
n
icate
d
to
th
e
n
etwo
r
k
a
d
m
in
is
tr
ato
r
f
o
r
s
u
itab
le
in
ter
v
en
tio
n
[
6
]
.
In
co
n
tr
ast
to
NI
DS,
HI
DS
o
b
s
er
v
es
an
in
d
i
v
id
u
al
h
o
s
t
or
d
ev
ice
to
id
e
n
tify
a
n
y
illeg
al
ac
tiv
ities
.
HI
DS
s
y
s
tem
s
ar
e
d
esig
n
ed
to
in
clu
d
e
t
h
e
ab
ilit
y
to
m
o
n
ito
r
,
id
en
tify
,
an
d
r
esp
o
n
d
to
u
n
u
s
u
al
b
eh
av
io
r
s
th
at
ar
e
r
elev
an
t
to
t
h
e
h
o
s
t
d
e
v
ice
[
7
]
.
Similar
to
NI
DS,
it
also
h
as
Sig
n
atu
r
e
-
b
ased
d
etec
ti
o
n
tech
n
iq
u
es
th
at
id
en
tify
ass
au
lts
by
co
r
r
elatin
g
ce
r
tain
b
e
h
av
io
r
s
with
estab
l
is
h
ed
attac
k
s
ig
n
atu
r
es
[
8
]
.
T
h
e
i
d
e
n
t
i
f
i
c
at
i
o
n
of
t
h
e
p
r
o
f
i
l
e
t
h
a
t
o
u
t
l
i
n
es
a
b
n
o
r
m
a
l
p
r
o
c
e
s
s
a
ct
i
v
i
ti
e
s
is
es
t
a
b
li
s
h
e
d
t
h
r
o
u
g
h
t
h
e
u
s
e
of
ML
in
I
DS
[
9
]
.
T
h
e
s
e
a
p
p
r
o
a
c
h
e
s
a
r
e
t
r
a
i
n
e
d
on
s
t
a
n
d
a
r
d
d
a
t
a
s
a
m
p
l
e
s
.
An
a
n
o
m
a
l
y
-
b
as
ed
H
I
D
S
w
i
ll
n
o
t
i
f
y
t
h
e
s
y
s
t
e
m
a
d
m
i
n
is
t
r
at
o
r
of
a
n
y
u
n
u
s
u
a
l
or
h
a
r
m
f
u
l
p
r
o
c
e
s
s
i
n
g
a
c
t
i
v
i
t
y
if
it
d
e
v
ia
t
es
f
r
o
m
t
h
is
p
r
o
f
i
l
e
.
I
DS
o
f
t
e
n
u
s
e
s
ML
a
n
d
DL
m
e
t
h
o
d
s
t
h
a
t
a
u
t
o
n
o
m
o
u
s
l
y
i
d
e
n
t
i
f
y
u
n
e
x
p
e
c
t
e
d
a
n
d
u
n
p
r
e
d
i
c
t
a
b
l
e
a
tt
a
c
k
s
[
1
0
]
,
[
1
1
]
.
T
h
is
p
ap
er
p
r
o
p
o
s
es
an
in
n
o
v
ativ
e
h
y
b
r
id
s
tr
ateg
y
t
h
at
u
s
es
an
ML
ap
p
r
o
ac
h
to
d
etec
t
W
in
d
o
ws
10
attac
k
s
.
As
ML
an
d
DL
tech
n
iq
u
es
can
id
en
tify
in
tr
icate
p
a
tter
n
s
an
d
b
eh
av
io
u
r
s
in
s
y
s
tem
d
ata,
th
ey
h
av
e
b
o
th
d
em
o
n
s
tr
ated
s
ig
n
if
ican
t
ef
f
ec
tiv
en
ess
in
HI
DS.
HI
DS
is
ca
p
ab
le
of
co
m
p
ilin
g
a
v
ast
an
d
d
iv
er
s
e
a
r
r
a
y
of
in
f
o
r
m
atio
n
by
m
o
n
ito
r
in
g
a
wid
e
r
an
g
e
of
ev
e
n
ts
,
in
clu
d
in
g
s
y
s
tem
ca
lls
,
f
ile
m
o
d
if
ica
tio
n
s
,
an
d
n
etwo
r
k
tr
af
f
ic.
R
u
le
-
b
ased
s
y
s
tem
s
ty
p
ically
h
a
v
e
d
if
f
icu
lty
r
ec
o
g
n
i
zin
g
c
o
m
p
lex
an
d
e
v
er
-
ch
a
n
g
i
n
g
attac
k
p
atter
n
s
,
m
ain
ly
wh
en
th
e
en
v
i
r
o
n
m
e
n
t
is
in
a
s
tate
of
co
n
s
tan
t
ch
an
g
e.
No
n
eth
eless
,
ML
an
d
DL
alg
o
r
ith
m
s
h
av
e
d
em
o
n
s
tr
ated
s
u
p
er
i
o
r
ity
in
th
is
d
o
m
ain
o
win
g
to
th
eir
ab
ilit
y
to
au
to
n
o
m
o
u
s
ly
lear
n
f
r
o
m
d
ata
an
d
d
etec
t
an
o
m
alies
or
h
a
r
m
f
u
l
ac
tiv
itie
s
th
at
d
iv
er
g
e
f
r
o
m
s
tan
d
a
r
d
p
atter
n
s
.
In
o
r
d
er
to
f
i
n
d
lik
ely
in
c
u
r
s
io
n
s
,
ML
tech
n
iq
u
es
can
ex
am
in
e
d
ata
f
r
o
m
m
u
ltid
im
en
s
io
n
al
s
y
s
tem
s
an
d
r
ev
ea
l
s
u
b
tle
r
elatio
n
s
h
ip
s
b
etwe
en
v
ar
io
u
s
f
ea
t
u
r
es.
T
h
is
id
ea
is
ad
v
an
ce
d
by
d
e
ep
lear
n
in
g
m
o
d
els,
p
ar
ticu
lar
ly
n
eu
r
al
n
etwo
r
k
s
,
wh
ich
m
ak
e
u
s
e
of
m
u
ltip
le
lay
er
s
of
r
ep
r
esen
tatio
n
.
T
h
is
en
ab
les
th
em
to
d
if
f
er
en
tiate
b
etwe
en
m
o
r
e
co
m
p
lex
an
d
n
o
n
-
lin
ea
r
co
r
r
elatio
n
s
with
in
th
e
d
ata.
Deep
lear
n
in
g
alg
o
r
ith
m
s
ar
e
v
er
y
g
o
o
d
at
f
i
n
d
in
g
ad
v
a
n
ce
d
r
is
k
s
,
lik
e
ze
r
o
-
d
ay
attac
k
s
,
w
h
ich
o
f
te
n
leav
e
m
in
o
r
s
ig
n
s
in
th
e
ac
tiv
ities
of
a
s
y
s
tem
[
1
2
]
.
Fu
r
th
er
m
o
r
e,
ML
an
d
DL
ap
p
r
o
ac
h
es'
ab
ilit
y
to
co
n
ti
n
u
o
u
s
ly
im
p
r
o
v
e
an
d
a
d
ap
t
t
h
r
o
u
g
h
lear
n
in
g
allo
ws
th
em
to
ad
v
a
n
ce
in
tan
d
em
with
e
m
er
g
in
g
th
r
ea
ts
.
A
lo
t
of
r
es
ea
r
c
h
h
as
b
e
en
c
o
n
d
u
ct
ed
to
d
ev
el
o
p
v
ar
io
u
s
t
ec
h
n
i
q
u
es
f
o
r
c
lass
i
f
y
i
n
g
at
tac
k
s
in
m
u
l
ti
p
le
en
v
i
r
o
n
m
e
n
ts
.
F
ig
u
ei
r
e
d
o
et
a
l.
[
1
3
]
p
r
es
en
te
d
th
e
DL
-
b
ase
d
I
DS
[
1
3
]
t
h
at
u
s
es
th
e
s
t
ac
k
in
g
l
o
n
g
-
s
h
o
r
t
-
te
r
m
m
e
m
o
r
y
(
L
ST
M)
m
o
d
el
a
n
d
t
h
e
im
p
r
o
v
e
d
p
r
e
-
p
r
o
c
ess
i
n
g
a
p
p
r
o
ac
h
,
r
e
tu
r
n
s
c
o
n
te
x
t
-
f
r
e
e
in
f
o
r
m
a
ti
o
n
,
a
n
d
ac
h
ie
v
es
o
v
e
r
9
9
%
ac
c
u
r
ac
y
on
th
e
C
I
C
I
DS
2
0
1
7
d
a
tase
t.
K
h
a
n
et
a
l
.
[
1
4
]
u
til
izes
a
r
e
cu
r
r
en
t
n
e
u
r
al
n
et
wo
r
k
co
m
b
i
n
e
d
wit
h
g
ate
d
r
ec
u
r
r
en
t
u
n
its
(
R
NN
-
GR
U
)
to
class
if
y
att
ac
k
s
.
T
h
e
d
ata
u
s
e
d
is
th
e
T
o
N
-
I
o
T
d
at
aset
,
wh
i
ch
w
as
i
n
te
g
r
at
e
d
wit
h
t
h
e
t
h
r
ee
-
t
ie
r
e
d
I
o
T
s
y
s
t
em
an
d
co
n
s
is
ts
of
n
o
v
el
att
ac
k
ty
p
es
n
o
t
f
o
u
n
d
in
o
th
er
o
p
e
n
-
s
o
u
r
ce
d
at
asets
.
T
h
e
r
ec
o
m
m
en
d
ed
m
o
d
e
l
att
ai
n
s
an
a
cc
u
r
ac
y
of
8
8
%
f
o
r
th
e
W
i
n
d
o
ws
10
d
at
aset
.
Similar
ly
,
L
ilh
o
r
e
et
a
l.
[
1
5
]
in
tr
o
d
u
ce
d
th
e
c
o
m
b
in
ati
o
n
of
OC
NN
-
L
STM
an
d
a
tr
a
n
s
f
er
lear
n
in
g
ap
p
r
o
ac
h
.
T
h
e
s
u
g
g
ested
m
o
d
el
em
p
lo
y
s
an
im
p
r
o
v
ed
co
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
(
C
N
N)
by
en
h
a
n
cin
g
its
p
ar
am
eter
s
th
r
o
u
g
h
t
h
e
g
r
ey
wo
lf
o
p
tim
izatio
n
tech
n
i
q
u
e
,
wh
ich
r
ef
in
es
th
e
C
NN
h
y
p
er
p
ar
am
eter
s
an
d
im
p
r
o
v
es
th
e
C
NN's
ac
cu
r
ac
y
.
T
h
e
r
ec
o
m
m
e
n
d
ed
m
o
d
el
at
tain
ed
a
p
r
ec
is
io
n
of
9
2
.
7
%
on
th
e
W
in
d
o
ws
10
d
ataset.
Awo
tu
n
d
e
et
a
l.
[
1
6
]
also
u
s
ed
M
L
tech
n
iq
u
es
on
T
ON
-
I
o
T
d
atasets
an
d
ac
h
iev
ed
th
e
h
ig
h
est
ac
cu
r
ac
y
of
9
9
.
7
3
%.
L
ik
ewise,
C
h
alich
alam
ala
et
a
l.
[
1
7
]
r
ec
o
m
m
en
d
an
e
n
s
em
b
le
class
if
ier
b
ased
on
lo
g
is
tic
r
eg
r
ess
io
n
(
LR
)
f
o
r
th
e
ef
f
icien
t
d
ev
elo
p
m
en
t
of
I
DS.
T
h
e
s
u
g
g
ested
m
o
d
el
also
in
teg
r
ates
Ad
aBo
o
s
t
an
d
r
an
d
o
m
f
o
r
est
(
RF
)
to
cr
ea
te
a
p
r
o
f
icien
t
class
if
ier
em
p
lo
y
in
g
th
e
iter
ativ
e
en
s
em
b
le
m
eth
o
d
o
lo
g
y
.
T
h
e
ad
ap
tiv
e
s
y
n
th
eti
c
s
am
p
lin
g
m
eth
o
d
m
itig
ates
th
e
p
r
o
b
lem
of
d
ata
im
b
alan
ce
.
Mo
r
eo
v
er
,
u
n
s
u
itab
le
f
ea
tu
r
e
s
ar
e
r
em
o
v
ed
by
r
ec
u
r
r
en
t
f
ea
tu
r
e
elim
in
atio
n
.
T
h
e
s
u
g
g
ested
m
o
d
el
attain
s
an
ac
cu
r
ac
y
of
9
9
.
9
9
%
f
o
r
th
e
B
o
T
-
I
o
T
d
ataset,
s
u
r
p
ass
in
g
th
at
of
TL
-
I
DS
an
d
L
STM
.
Sin
g
h
et
a
l.
[
1
2
]
s
elec
t
attr
ib
u
tes
u
s
in
g
a
m
ix
of
th
e
in
f
o
r
m
atio
n
g
ain
ca
lcu
latio
n
an
d
th
e
s
im
u
late
d
an
n
ea
lin
g
tech
n
iq
u
e.
T
h
e
s
u
g
g
ested
DL
-
b
ased
m
o
d
el
a
ch
iev
es
r
em
ar
k
ab
le
ac
cu
r
ac
ies
of
9
9
.
9
9
0
%
an
d
9
9
.
9
9
6
%
on
th
e
T
ON
-
I
OT
win
7
a
n
d
T
ON
-
I
OT
win
1
0
d
atasets
[
1
8
]
,
[
1
9
]
.
Similar
to
th
e
ab
o
v
e
ap
p
r
o
ac
h
,
C
h
er
f
i
et
a
l.
[
2
0
]
s
u
g
g
est
ed
a
DL
-
I
DS
m
o
d
el
th
at
in
t
eg
r
ates
an
au
to
en
co
d
er
with
a
d
ee
p
n
e
u
r
al
n
etwo
r
k
(
DNN
)
.
T
h
e
i
n
f
o
r
m
atio
n
g
ain
ass
ess
m
en
t
an
d
th
e
s
im
u
lated
an
n
ea
lin
g
ap
p
r
o
ac
h
ar
e
u
s
ed
f
o
r
attr
ib
u
te
s
elec
tio
n
.
T
h
e
h
ig
h
est
ac
cu
r
ac
y
of
9
9
.
9
0
%
an
d
9
9
.
8
6
%
is
ac
h
iev
ed
on
d
atasets
T
ON
-
I
OT
win
7
a
n
d
T
ON
-
I
OT
win
1
0
,
r
esp
ec
tiv
ely
.
Alter
n
ativ
ely
,
Din
g
et
a
l
.
[
2
1
]
in
tr
o
d
u
ce
an
alter
n
ativ
e
DL
m
o
d
el
f
o
r
th
e
d
etec
tio
n
of
c
y
b
er
attac
k
s
tar
g
etin
g
I
o
T
d
e
v
ices.
Dee
p
AK
-
I
o
T
is
f
o
u
n
d
e
d
on
th
r
ee
co
m
p
o
n
e
n
ts
:
th
e
b
lo
ck
s
f
o
r
s
p
atial
m
o
d
elin
g
b
ase
d
on
r
esid
u
als
(
R
SR
)
,
tem
p
o
r
al
m
o
d
elin
g
,
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
E
n
s
emb
le
w
in
d
o
w
s
in
tr
u
s
io
n
d
etec
tio
n
s
ystem
u
s
in
g
xg
b
o
o
s
t
a
n
d
d
e
ep
…
(
P
r
a
n
ith
a
K
ed
a
m
b
a
d
y
S
h
iva
)
567
d
etec
tio
n
.
A
s
k
ip
lin
k
is
in
co
r
p
o
r
ated
in
t
o
each
R
SR
b
lo
ck
of
th
e
f
o
u
r
co
n
v
o
l
u
tio
n
al
lay
e
r
s
,
wh
ich
is
done
in
o
r
d
er
to
p
r
ev
e
n
t
v
an
is
h
in
g
or
ex
p
a
n
d
in
g
g
r
ad
ien
ts
.
T
h
ese
lay
er
s
ar
e
c
o
n
n
e
cted
in
p
a
r
al
lel.
T
h
e
last
b
lo
ck
class
if
ies
th
e
in
p
u
t.
T
h
e
r
ec
o
m
m
en
d
ed
m
o
d
el'
s
ac
cu
r
ac
y
is
9
0
.
5
7
%
f
o
r
T
ON
I
o
T
.
T
h
e
an
aly
ze
d
wo
r
k
s
em
p
h
asize
th
e
in
cr
ea
s
in
g
ap
p
licatio
n
of
s
o
p
h
is
ticated
ML
an
d
DL
m
eth
o
d
o
l
o
g
ies
in
th
e
cr
ea
tio
n
of
ef
f
icien
t
I
DS,
p
ar
ticu
la
r
ly
in
m
an
ag
in
g
in
tr
icate
,
lar
g
e
-
s
ca
le
d
atasets
with
in
d
iv
er
s
e
n
etwo
r
k
s
ettin
g
s
.
T
ec
h
n
iq
u
es
s
u
ch
as
L
STM
,
GR
U
,
C
NN,
an
d
h
y
b
r
id
m
o
d
els
h
av
e
b
ee
n
ef
f
ec
tiv
ely
u
tili
ze
d
,
d
em
o
n
s
tr
atin
g
ex
ce
ll
en
t
ac
cu
r
ac
y
in
i
n
tr
u
s
io
n
d
etec
tio
n
,
p
ar
ticu
lar
ly
in
I
o
T
a
n
d
in
d
u
s
tr
ial
n
etwo
r
k
s
.
T
h
ese
m
o
d
els
h
av
e
b
ee
n
a
u
g
m
en
ted
with
m
an
y
o
p
tim
izatio
n
tech
n
iq
u
es,
in
clu
d
in
g
g
r
ey
wo
lf
o
p
tim
izatio
n
an
d
s
o
p
h
is
ticated
o
p
tim
izer
s
lik
e
Ad
am
,
to
b
o
o
s
t
p
er
f
o
r
m
an
ce
.
Pre
v
alen
t
is
s
u
es
s
u
ch
as
d
ata
im
b
alan
ce
,
f
ea
tu
r
e
s
elec
tio
n
,
an
d
th
e
cla
s
s
if
icatio
n
of
n
ew
attac
k
s
ar
e
s
till
p
r
esen
t.
T
h
u
s
,
th
is
p
ap
er
p
r
o
p
o
s
es
a
n
o
v
el
h
y
b
r
id
ap
p
r
o
ac
h
by
u
tili
zin
g
t
he
DNN
an
d
XGBo
o
s
t
m
o
d
el
s
to
d
ev
elo
p
an
I
DS
th
at
ca
n
d
etec
t
attac
k
s
an
d
class
if
y
th
em
in
to
v
ar
io
u
s
class
es.
Hen
ce
,
th
e
s
ig
n
if
ican
t
co
n
tr
ib
u
tio
n
of
th
is
r
esear
ch
is
g
iv
en
b
elo
w.
T
h
is
p
ap
er
in
tr
o
d
u
ce
s
a
n
o
v
el
h
y
b
r
i
d
m
o
d
el
th
at
co
m
b
in
es
DNN
f
o
r
f
ea
tu
r
e
e
x
tr
ac
tio
n
an
d
ML
f
o
r
class
if
icatio
n
,
r
esu
ltin
g
in
im
p
r
o
v
ed
ac
c
u
r
ac
y
an
d
r
o
b
u
s
tn
ess
.
−
T
h
e
v
ar
io
u
s
ML
alg
o
r
ith
m
s
,
s
u
ch
as
Gau
s
s
ian
n
aïv
e
B
ay
es
(
GNB),
L
R
,
K
n
ea
r
est
n
eig
h
b
o
r
s
(
KNN)
,
d
ec
is
io
n
tr
ee
(
DT
)
,
R
F,
an
d
XGB,
ar
e
tr
ain
ed
on
th
e
o
p
en
-
s
o
u
r
ce
d
atasets
.
−
A
cu
s
to
m
ized
DNN
ar
ch
itectu
r
e
h
as
b
ee
n
d
ev
elo
p
ed
an
d
tr
ai
n
ed
u
s
in
g
t
h
e
s
am
e
d
ataset.
−
I
n
ter
m
ed
iate
Featu
r
e
E
x
tr
ac
ti
o
n
:
T
h
e
m
o
d
el
ex
tr
ac
ts
d
etailed
f
ea
tu
r
es
f
r
o
m
th
e
s
ec
o
n
d
-
to
-
last
lay
er
of
th
e
C
u
s
to
m
ized
DNN
an
d
u
s
es
th
ese
f
ea
tu
r
es
as
in
p
u
t
f
o
r
th
e
XGB
clas
s
if
ier
,
wh
ich
s
ig
n
if
ican
tly
im
p
r
o
v
es
p
e
r
f
o
r
m
an
ce
co
m
p
ar
ed
to
u
s
in
g
r
aw
d
ata.
−
T
h
e
s
tu
d
y
also
em
p
h
asizes
d
ata
s
ca
lin
g
th
r
o
u
g
h
s
tan
d
ar
d
izatio
n
to
n
o
r
m
alize
in
p
u
t
ch
ar
a
cter
is
tics
an
d
one
-
h
o
t
lab
el
en
co
d
in
g
f
o
r
o
p
t
im
al
in
p
u
t
f
o
r
th
e
DNN
an
d
XGB
clas
s
if
ier
s
.
−
T
h
e
h
y
b
r
i
d
m
o
d
el
is
p
r
o
p
o
s
ed
by
co
m
b
i
n
in
g
th
e
DNN'
s
r
e
p
r
esen
tatio
n
al
p
o
wer
with
XGB
's
ab
il
ity
to
r
ef
in
e
d
ec
is
io
n
b
o
u
n
d
s
.
2.
RE
S
E
ARCH
M
E
T
H
O
D
T
h
e
p
r
o
p
o
s
ed
m
eth
o
d
aim
s
to
in
teg
r
ate
t
h
e
DNN
an
d
th
e
XGB
m
o
d
el
to
d
ev
el
o
p
n
o
v
el
en
s
em
b
le
tech
n
iq
u
es.
Dif
f
e
r
en
t
ML
tec
h
n
iq
u
es
a
r
e
tr
ain
e
d
on
t
h
e
o
p
en
-
s
o
u
r
ce
d
ataset
to
id
en
tify
a
n
d
th
e
n
class
if
y
th
e
attac
k
in
to
eig
h
t
d
is
tin
ct
class
es.
T
h
e
DNN
m
o
d
el
is
f
in
e
-
tu
n
ed
an
d
o
p
tim
ized
on
th
e
s
am
e
d
ataset
to
ac
h
iev
e
th
e
o
p
tim
u
m
p
er
f
o
r
m
an
ce
.
Fi
n
ally
,
to
tak
e
ad
v
an
tag
e
of
t
h
e
DNN
an
d
b
o
o
s
tin
g
m
eth
o
d
,
th
e
n
ew
m
o
d
el
XGBNN
was
d
ev
elo
p
ed
,
wh
ich
is
th
e
in
ter
ac
tio
n
of
DNN
an
d
XGB
.
T
h
e
tr
ain
ed
DNN
m
o
d
el’
s
in
ter
m
ed
iate
f
ea
tu
r
es
f
r
o
m
th
e
last
s
ec
o
n
d
lay
er
ar
e
u
s
ed
as
in
p
u
t
to
th
e
XGB
m
o
d
el.
Fig
u
re
1
p
r
o
v
i
d
es
a
v
is
u
al
r
ep
r
esen
tatio
n
of
th
e
p
r
o
p
o
s
ed
ap
p
r
o
ac
h
.
T
h
e
p
r
o
p
o
s
ed
A
lg
o
r
ith
m
1
f
o
r
th
e
H
y
b
r
id
I
DS
is
g
iv
en
b
el
o
w.
Alg
o
r
ith
m
1
: X
GB
NN
Alg
o
r
ith
m
Input:
Training data
∈
ℝ
×
and labels
∈
ℝ
, where n is the number of samples and d
is the number of features.
Output:
Classify the instances into eight classes of attacks
Step 1:
Data Pre
-
processing
St
ep
1.
1.
:
No
rm
al
iz
e
t
he
in
pu
t
da
ta
us
in
g
th
e
fo
rm
ul
a
as
=
−
and
=
−
,
where
is
th
e
me
an
of
th
e
tr
ai
ni
ng
in
st
an
ce
s,
an
d
is
th
e
st
an
da
rd
de
vi
at
io
n
of the training instances.
Step 1.2: Apply label encoding as
(
)
=
{
1
,
(
)
=
0
,
ℎ
,
i
=
1
,
.
.
,
n
,
g
=
1
,
.
.
,
G
, where
is the number
of classes, and
is the label feature matrix.
Step 2:
Develop a customized DNN model
Input:
with shape (n x d), where d is the dimensionality of the features.
DNN architecture is defined as:
Input layer:
ℎ
0
=
Hi
dd
en
La
ye
r
1:
ℎ
1
=
(
1
ℎ
0
+
1
)
,
wh
er
e
f
is
th
e
RE
LU
ac
ti
va
ti
on
f
un
ct
io
n
(
)
=
max
(
0
,
)
,
1
∈
ℝ
1
0
2
4
×
, and
1
=
ℝ
1
0
2
4
.
Hidden layer 2:
ℎ
2
=
(
2
ℎ
1
+
2
)
, where
2
∈
ℝ
5
1
2
×
, and
2
=
ℝ
512
.
Hidden layer 3:
ℎ
3
=
(
3
ℎ
2
+
3
)
, where
3
∈
ℝ
256
×
, and
3
=
ℝ
256
.
Output layer:
̂
=
(
4
ℎ
3
+
4
)
, where
4
∈
ℝ
×
256
, and
4
=
ℝ
Loss Function (Categorical Cross Entropy) is calculated as
(
,
̂
)
=
−
∑
∑
,
̂
,
=
1
=
1
Step 3:
Training the DNN model
Step 3.1: Use ADAM optimizer with learning rate
and Train for E epochs
Step 3.2: Early Stopping: If the validation loss does not improve for p consecutive epochs,
training should be terminated.
Step 4:
Extract the intermediate features from DNN
Step 4.1
: After training DNN, use the last second layer dense layer
ℎ
3
as the feature map.
Step 5:
Train different ML techniques to find the best among them
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
5
6
5
-
5
7
7
568
Step 6:
Train the highest
-
performing ML model with the intermediate features from
ℎ
3
layer.
Step 7:
Final evaluation of the proposed Hybrid model
St
ep
7.
1:
Co
mb
in
e
DN
N
so
ft
ma
x
pr
ed
ic
ti
on
s
an
d
th
e
hi
gh
es
t
-
performing
ML
model’s
predictions
to get the final classification result.
̂
=
.
̂
+
(
1
−
)
.
̂
, where
is the weighting factor
Ste
p 7.2: Measure accuracy, recall, precision, F1 score, and Area under curve
Fig
u
re
1
.
T
h
e
f
lo
w
c
h
ar
t
of
th
e
p
r
o
p
o
s
ed
h
y
b
r
id
DNN+
ML
I
DS
2
.
1
.
Da
t
a
s
et
des
cr
iptio
n
T
h
e
d
ataset
u
s
ed
f
o
r
t
h
e
ex
p
er
im
en
tatio
n
s
is
th
e
o
p
en
-
s
o
u
r
ce
d
ataset
“Wi
n
d
o
ws
10”
f
r
o
m
th
e
T
ON_
I
o
T
d
ataset
UNSW
,
Sy
d
n
ey
19
.
In
W
in
d
o
ws
10,
n
ew
f
ea
tu
r
es
ar
e
ex
tr
ac
ted
f
r
o
m
th
e
p
r
o
ce
s
s
o
r
,
m
em
o
r
y
,
p
r
o
ce
s
s
es,
an
d
h
ar
d
d
is
k
au
d
it
tr
ails
u
s
in
g
an
in
n
o
v
ativ
e
I
o
T
n
etwo
r
k
d
esig
n
.
T
h
r
ee
tier
s
-
ed
g
e,
f
o
g
,
an
d
clo
u
d
-
wer
e
u
s
ed
to
im
p
le
m
en
t
th
e
test
b
ed
.
T
h
e
r
e
ar
e
t
h
r
ee
lev
els
to
a
n
etwo
r
k
:
t
h
e
ed
g
e,
wh
ic
h
h
o
u
s
es
n
etwo
r
k
d
ev
ices
an
d
th
e
I
o
T
;
th
e
f
o
g
,
wh
ich
c
o
n
tain
s
v
ir
tu
al
m
ac
h
in
es
an
d
g
atew
ay
s
;
an
d
th
e
cl
o
u
d
,
wh
ich
in
clu
d
es
s
er
v
ices
o
f
f
er
e
d
by
th
e
clo
u
d
,
lik
e
s
tatis
tical
an
aly
s
i
s
an
d
v
is
u
aliza
tio
n
.
So
f
twar
e
-
d
ef
in
e
d
n
etwo
r
k
i
n
g
an
d
n
etwo
r
k
f
u
n
ctio
n
v
ir
tu
aliza
tio
n
d
y
n
am
ically
ad
m
in
i
s
ter
ed
th
e
lev
els
u
s
in
g
th
e
VM
war
e
NS
X
an
d
v
C
lo
u
d
NFV
p
latf
o
r
m
s
.
T
h
r
o
u
g
h
o
u
t
t
h
e
co
n
s
tr
u
ctio
n
an
d
d
ep
lo
y
m
en
t
of
th
e
test
b
ed
,
b
o
th
s
tan
d
ar
d
an
d
m
alicio
u
s
ac
tiv
ities
wer
e
ex
e
cu
ted
to
g
ath
er
a
n
n
o
tated
s
am
p
les
of
d
ata
b
ased
on
an
a
u
th
en
tic
g
r
o
u
n
d
tr
u
t
h
t
ab
le
f
o
r
e
v
alu
atin
g
th
e
ef
f
ec
ti
v
en
ess
of
in
n
o
v
ativ
e
c
y
b
er
s
ec
u
r
ity
s
o
lu
tio
n
s
[1
8
]
.
T
h
er
e
ar
e
21
,
104
in
s
tan
ce
s
an
d
1
2
6
f
ea
tu
r
es
in
th
e
W
in
d
o
w
s
10
d
ataset,
co
m
p
r
is
in
g
1
0
,
0
0
0
n
o
r
m
al
r
ec
o
r
d
s
an
d
1
1
,
1
0
4
m
alicio
u
s
r
ec
o
r
d
s
class
if
ied
in
to
eig
h
t
ca
teg
o
r
ies:
DDo
S,
Do
S,
in
jectio
n
,
n
o
r
m
al,
x
s
s
,
p
ass
wo
r
d
,
s
ca
n
n
in
g
,
an
d
MI
T
M
[
1
9
]
.
T
h
e
n
u
m
b
er
of
in
s
tan
ce
s
in
each
ca
teg
o
r
y
of
th
e
d
ataset
is
g
iv
en
in
Fig
u
re
2.
T
h
e
h
ig
h
est
n
u
m
b
er
of
attac
k
s
is
f
o
r
DDo
S,
wh
ich
h
as
4
,
6
0
8
s
am
p
les.
2
.
2
.
Da
t
a
pre
-
pro
ce
s
s
ing
T
h
e
d
ata
is
p
r
o
ce
s
s
ed
to
n
o
r
m
alize
it
f
o
r
b
etter
p
er
f
o
r
m
an
ce
.
No
r
m
aliza
tio
n
is
an
ess
en
tial
p
r
ep
r
o
ce
s
s
in
g
p
r
o
ce
d
u
r
e
f
o
r
d
atasets
,
p
ar
ticu
lar
ly
th
o
s
e
in
cl
u
d
in
g
m
an
y
c
h
ar
ac
ter
is
tics
with
d
is
p
ar
ate
s
ca
les.
No
r
m
aliza
tio
n
[
2
2
]
m
ea
n
s
s
ta
n
d
ar
d
izin
g
all
f
ea
tu
r
es
to
a
u
n
if
o
r
m
s
ca
le,
h
en
ce
e
n
h
an
ci
n
g
th
e
ef
f
icac
y
of
ML
alg
o
r
ith
m
s
.
In
th
e
ab
s
en
ce
of
n
o
r
m
aliza
tio
n
,
f
ea
tu
r
es
with
g
r
ea
ter
n
u
m
er
ical
r
an
g
es
m
ay
o
v
er
s
h
a
d
o
w
th
e
lear
n
in
g
p
r
o
ce
s
s
,
r
esu
ltin
g
in
b
iased
or
in
f
e
r
io
r
m
o
d
el
p
e
r
f
o
r
m
an
ce
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
E
n
s
emb
le
w
in
d
o
w
s
in
tr
u
s
io
n
d
etec
tio
n
s
ystem
u
s
in
g
xg
b
o
o
s
t
a
n
d
d
e
ep
…
(
P
r
a
n
ith
a
K
ed
a
m
b
a
d
y
S
h
iva
)
569
A
p
r
ev
alen
t
m
eth
o
d
of
n
o
r
m
aliza
tio
n
is
s
tan
d
ar
d
izatio
n
,
co
m
m
o
n
ly
r
e
f
er
r
e
d
to
as
Z
-
s
co
r
e
n
o
r
m
aliza
tio
n
.
T
h
is
ap
p
r
o
ac
h
n
o
r
m
alize
s
each
f
ea
tu
r
e
to
h
av
e
a
m
ea
n
of
0
a
n
d
a
s
tan
d
ar
d
d
ev
iatio
n
of
1.
W
h
en
th
e
d
ata
is
c
lo
s
e
to
b
ei
n
g
n
o
r
m
ally
d
is
tr
ib
u
ted
,
s
tan
d
ar
d
izatio
n
b
ec
o
m
es
ev
en
m
o
r
e
u
s
ef
u
l
b
ec
au
s
e
it
en
s
u
r
es
th
at
ea
c
h
f
ea
tu
r
e
p
r
o
v
i
d
es
an
eq
u
al
co
n
tr
i
b
u
tio
n
to
th
e
m
o
d
el.
Fo
r
a
f
ea
tu
r
e
,
t
h
e
n
o
r
m
alize
d
v
alu
e
′
is
ca
lcu
lated
as
g
iv
en
in
(
1
)
.
x
′
=
−
(
1
)
wh
er
e
is
th
e
o
r
ig
in
al
f
ea
tu
r
e,
is
th
e
m
ea
n
,
an
d
is
th
e
s
tan
d
ar
d
d
ev
iatio
n
.
L
ab
el
en
co
d
i
n
g
[
2
3
]
t
r
an
s
f
o
r
m
s
each
d
is
tin
ct
ca
teg
o
r
y
in
to
an
in
teg
er
n
u
m
b
er
.
T
h
is
m
eth
o
d
is
s
tr
aig
h
tf
o
r
war
d
a
n
d
d
ec
r
ea
s
es
th
e
d
ata'
s
d
im
en
s
io
n
ality
as
it
d
o
es
not
g
e
n
er
ate
ad
d
itio
n
al
b
i
n
ar
y
f
ea
t
u
r
es.
T
h
e
b
en
ef
its
of
la
b
el
en
co
d
i
n
g
en
co
m
p
ass
its
s
im
p
licity
an
d
ef
f
icien
cy
,
p
ar
ticu
lar
ly
in
th
e
p
r
esen
ce
of
n
u
m
er
o
u
s
u
n
iq
u
e
ca
te
g
o
r
ie
s
.
No
n
eth
eless
,
th
e
d
r
awb
ac
k
is
th
at
it
m
ay
cr
ea
te
u
n
wan
te
d
ass
o
ciatio
n
s
ac
r
o
s
s
ca
teg
o
r
ies,
r
esu
ltin
g
in
er
r
o
n
eo
u
s
p
r
ed
ictio
n
s
.
Fo
r
ex
am
p
l
e,
if
an
ML
m
o
d
el
p
er
ce
iv
e
s
h
ig
h
er
v
alu
es
as
"su
p
er
io
r
"
or
"e
n
h
a
n
ce
d
,
"
it
m
ay
er
r
o
n
eo
u
s
ly
in
f
er
a
h
ier
ar
ch
y
am
o
n
g
th
e
e
n
co
d
e
d
ca
teg
o
r
i
es.
L
ab
el
en
c
o
d
in
g
is
s
u
itab
le
w
h
en
th
e
r
e
e
x
is
ts
an
o
r
d
in
al
r
el
atio
n
s
h
ip
am
o
n
g
th
e
ca
teg
o
r
i
es,
s
u
ch
as
"L
o
w,
"
"M
ed
iu
m
,
"
an
d
"Hig
h
,
"
or
wh
e
n
em
p
l
o
y
in
g
alg
o
r
it
h
m
s
th
at
do
not
r
e
g
ar
d
th
e
e
n
co
d
ed
n
u
m
b
er
s
as
p
o
s
s
ess
in
g
an
o
r
d
er
,
s
u
ch
as
d
ec
is
io
n
tr
ee
s
.
T
h
e
eq
u
atio
n
f
o
r
ca
lcu
latin
g
th
e
lab
el
e
n
co
d
e
,
(
)
=
{
1
,
(
)
=
0
,
ℎ
(
2
)
f
o
r
,
=
1
,
.
.
,
n
,
a
n
d
g
=
1
,
.
.
,
G
.
wh
er
e
G
is
th
e
n
u
m
b
e
r
of
class
es,
an
d
Y
is
th
e
lab
el
f
ea
tu
r
e
m
atr
ix
.
Fig
u
re
2
.
Nu
m
b
er
of
s
am
p
les
in
each
lab
el
2
.
3
.
Deep
lea
rning
a
rc
hite
ct
ure
T
h
e
cu
s
to
m
ized
DNN,
wh
ich
co
n
s
is
ts
of
an
ar
ch
itectu
r
e,
is
p
r
o
p
o
s
ed
,
as
s
h
o
w
n
in
Fig
u
re
3.
T
h
e
d
iag
r
am
d
ep
icts
a
s
eq
u
en
ce
of
f
o
u
r
c
o
m
p
letely
lin
k
ed
(
d
en
s
e)
lay
er
s
with
in
a
n
eu
r
al
n
etw
o
r
k
.
E
v
er
y
la
y
er
is
d
esig
n
ated
with
its
n
am
e,
in
p
u
t
d
im
en
s
io
n
s
,
an
d
o
u
tp
u
t
d
im
e
n
s
io
n
s
.
L
ay
er
d
en
s
e_
1
6
:
T
h
is
co
n
s
tit
u
tes
th
e
in
au
g
u
r
al
d
en
s
e
lay
er
with
in
th
e
n
etwo
r
k
.
T
h
e
in
p
u
t
h
as
a
s
h
ap
e
of
(
No
n
e,
1
2
5
)
,
wh
e
r
e
"No
n
e"
s
ig
n
if
ies
a
v
ar
iab
le
b
atch
s
ize,
an
d
125
d
en
o
tes
th
e
d
im
en
s
io
n
ality
of
th
e
in
p
u
t
f
ea
tu
r
es.
T
h
e
o
u
tp
u
t
d
im
en
s
io
n
of
t
h
is
lay
er
is
(
No
n
e,
1
0
2
4
)
,
in
d
icatin
g
it
c
o
n
v
er
ts
th
e
in
p
u
t
in
to
a
1024
-
d
im
en
s
io
n
al
s
p
ac
e.
T
h
is
s
u
g
g
ests
th
at
th
e
lay
er
co
m
p
r
is
es
1024
n
eu
r
o
n
s
,
with
each
n
eu
r
o
n
lin
k
ed
to
ev
er
y
in
p
u
t
ch
a
r
ac
ter
is
tic.
L
ay
er
d
en
s
e_
1
7
:
T
h
e
o
u
t
p
u
t
f
r
o
m
th
e
p
r
ec
ed
in
g
lay
e
r
,
a
1
,
024
-
d
im
en
s
io
n
al
v
ec
to
r
,
is
u
tili
ze
d
as
in
p
u
t
f
o
r
th
is
lay
e
r
.
T
h
e
in
p
u
t
f
o
r
m
is
(
No
n
e,
1
,
0
2
4
)
,
w
h
ile
th
e
o
u
tp
u
t
is
d
im
in
is
h
e
d
to
(
N
o
n
e,
5
1
2
)
,
s
ig
n
if
y
in
g
th
at
th
is
lay
er
co
n
d
en
s
es
th
e
f
ea
tu
r
e
s
p
ac
e
to
512
d
im
e
n
s
io
n
s
.
T
h
is
is
ac
c
o
m
p
lis
h
ed
v
ia
5
1
2
n
eu
r
o
n
s
in
th
is
lay
er
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
5
6
5
-
5
7
7
570
L
ay
er
d
e
n
s
e_
1
8
:
T
h
e
th
ir
d
l
ay
er
p
r
o
ce
s
s
es
a
512
-
d
im
en
s
io
n
al
in
p
u
t
an
d
r
e
d
u
ce
s
it
to
a
2
5
6
-
d
im
en
s
io
n
al
o
u
tp
u
t,
in
d
icate
d
by
its
o
u
tp
u
t
s
h
ap
e
of
(
No
n
e
,
2
5
6
)
.
T
h
is
co
m
p
r
ess
io
n
p
er
p
etu
ates
th
e
p
r
o
ce
s
s
of
f
ea
tu
r
e
ab
s
tr
ac
tio
n
an
d
r
e
d
u
ctio
n
v
ia
a
f
u
lly
co
n
n
ec
ted
lay
er
.
L
ay
er
d
e
n
s
e_
1
9
:
T
h
e
co
n
cl
u
d
in
g
d
e
n
s
e
lay
er
c
o
m
p
r
ess
es
th
e
in
p
u
t
f
r
o
m
2
5
6
d
im
e
n
s
io
n
s
to
8
d
im
en
s
io
n
s
,
as
ev
id
en
ce
d
by
t
h
e
o
u
tp
u
t
s
h
ap
e
(
No
n
e,
8
)
.
T
h
is
lay
er
p
r
o
b
a
b
ly
f
u
n
ctio
n
s
as
th
e
o
u
tp
u
t
la
y
er
or
a
co
m
p
o
n
en
t
of
th
e
u
ltima
te
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
in
t
h
e
m
o
d
el,
co
n
tin
g
en
t
u
p
o
n
th
e
s
p
ec
if
ic
task
(
e.
g
.
,
class
if
icatio
n
with
eig
h
t
p
o
ten
tial
class
e
s
or
o
th
er
task
ty
p
es).
T
h
is
ar
ch
itectu
r
e
ex
em
p
lifie
s
a
co
n
v
en
tio
n
al
f
r
am
ewo
r
k
th
at
s
y
s
tem
atica
l
ly
d
im
in
is
h
es
t
h
e
f
ea
tu
r
e
s
p
ac
e
f
r
o
m
a
h
ig
h
e
r
-
d
im
e
n
s
io
n
al
in
p
u
t,
1
2
5
f
ea
tu
r
es,
to
a
s
ig
n
if
ican
tly
l
o
wer
-
d
im
e
n
s
io
n
a
l
o
u
tp
u
t,
8
class
es,
v
ia
a
s
er
ies
of
d
en
s
e
lay
er
s
.
E
v
er
y
lay
er
f
u
n
ctio
n
to
ca
p
tu
r
e
an
d
ab
s
tr
ac
t
v
ar
y
in
g
lev
els
of
f
ea
tu
r
e
r
ep
r
esen
tatio
n
s
[
2
4
]
.
Fig
u
re
3
.
Pro
p
o
s
ed
c
u
s
to
m
ized
DNN
2
.
4
.
M
a
chine
lea
rning
a
lg
o
ri
t
hm
T
h
e
v
ar
io
u
s
ML
alg
o
r
ith
m
s
[
2
5
]
tr
ain
ed
a
r
e
d
escr
ib
ed
b
elo
w.
GNB
is
a
p
r
o
b
ab
ilis
tic
cla
s
s
if
icatio
n
tech
n
iq
u
e
d
e
r
iv
ed
f
r
o
m
B
ay
e
s
'
T
h
eo
r
em
,
p
r
ed
icate
d
on
th
e
p
r
em
is
e
th
at
all
f
ea
tu
r
es
ar
e
in
d
ep
en
d
en
t
an
d
co
n
f
o
r
m
to
a
Gau
s
s
ian
d
is
tr
ib
u
tio
n
[
2
6
]
.
LR
is
a
lin
ea
r
m
o
d
el
em
p
lo
y
ed
f
o
r
b
in
ar
y
class
if
icatio
n
ap
p
licatio
n
s
.
It
ca
lcu
lates
th
e
lik
elih
o
o
d
of
a
d
ata
p
o
in
t
b
ein
g
class
if
ied
i
n
to
o
n
e
of
two
ca
teg
o
r
ies
by
a
lo
g
is
tic
f
u
n
ctio
n
,
y
ield
in
g
a
v
alu
e
r
a
n
g
in
g
f
r
o
m
0
to
1
[
2
7
]
.
KNN
is
a
s
tr
aig
h
tf
o
r
war
d
an
d
in
tu
itiv
e
in
s
t
an
ce
-
b
ased
lear
n
in
g
tech
n
iq
u
e.
R
ath
er
th
a
n
c
o
n
s
tr
u
ctin
g
an
e
x
p
licit
m
o
d
el,
it
ca
teg
o
r
izes
n
ew
d
ata
p
o
in
ts
by
ag
g
r
eg
atin
g
a
m
ajo
r
ity
v
o
te
f
r
o
m
th
e
"k
"
n
e
ar
est
d
ata
p
o
in
ts
in
th
e
tr
ain
in
g
s
et
[
2
8
]
.
DT
ar
e
non
-
lin
ea
r
m
o
d
els
th
at
p
ar
titi
o
n
d
ata
in
to
s
u
b
s
ets
ac
co
r
d
in
g
to
f
ea
tu
r
e
v
alu
es,
f
o
r
m
in
g
a
tr
ee
-
lik
e
s
tr
u
ctu
r
e
in
wh
ich
each
in
ter
n
al
node
s
ig
n
if
ies
a
d
ec
is
io
n
r
u
le,
an
d
each
leaf
n
o
d
e
d
en
o
tes
a
p
r
ed
icted
class
lab
el
[
2
9
]
.
RF
is
an
en
s
em
b
le
lear
n
in
g
m
eth
o
d
th
at
m
itig
ates
th
e
o
v
e
r
f
itti
n
g
p
r
o
b
lem
of
d
ec
is
io
n
t
r
e
es
th
r
o
u
g
h
th
e
a
g
g
r
e
g
atio
n
of
s
ev
er
al
tr
ee
s
.
E
ac
h
tr
ee
is
tr
ain
ed
on
a
d
is
tin
ct
r
a
n
d
o
m
s
u
b
s
et
of
th
e
d
ata
an
d
ch
ar
ac
ter
is
tics
,
with
th
e
f
in
al
p
r
ed
ictio
n
d
er
iv
e
d
f
r
o
m
a
g
g
r
e
g
atin
g
th
e
p
r
ed
icti
o
n
s
of
all
tr
ee
s
(
b
ag
g
in
g
)
[
3
0
]
.
Dec
is
io
n
tr
ee
s
s
er
v
e
as
th
e
e
s
s
en
tial
lear
n
er
s
in
(
XGB),
an
im
p
r
o
v
ed
v
er
s
io
n
of
g
r
ad
ien
t
b
o
o
s
tin
g
.
R
eg
u
lar
izatio
n
,
p
ar
allel
p
r
o
ce
s
s
in
g
,
an
d
th
e
h
an
d
lin
g
of
m
is
s
in
g
d
ata
ar
e
s
o
m
e
of
th
e
i
m
p
r
o
v
e
m
en
ts
th
at
g
iv
e
it
its
s
p
ee
d
an
d
ef
f
ec
tiv
e
n
ess
[
3
1
]
.
2
.
5
.
P
r
o
po
s
ed
hy
brid
m
o
del
T
h
e
DNN
is
tr
ain
ed
to
ex
tr
ac
t
th
e
f
ea
tu
r
es
f
r
o
m
th
e
last
s
ec
o
n
d
lay
er
,
a
n
d
th
ese
f
ea
tu
r
es
ar
e
p
r
o
v
id
e
d
to
th
e
XGB
f
o
r
class
if
icatio
n
.
DNNs
ar
e
d
ev
elo
p
ed
to
ac
q
u
ir
e
h
ier
ar
ch
ical
r
e
p
r
esen
tatio
n
s
of
d
ata.
As
in
p
u
t
d
ata
tr
av
er
s
es
th
e
s
ev
er
al
lay
e
r
s
of
a
DNN,
th
e
m
o
d
el
ac
q
u
i
r
es
p
r
o
g
r
ess
iv
ely
in
tr
icate
f
ea
t
u
r
es
at
each
lay
er
.
T
h
e
lo
wer
lay
er
s
id
en
tify
th
e
m
o
s
t
n
o
tab
le
f
ea
tu
r
es
lik
e
b
o
r
d
er
s
an
d
b
ac
k
g
r
o
u
n
d
.
T
h
e
h
ig
h
er
lay
er
s
ar
e
r
esp
o
n
s
ib
le
f
o
r
more
in
s
ig
h
tf
u
l
in
f
o
r
m
atio
n
,
s
u
c
h
as
th
e
s
h
ap
e
an
d
th
e
k
in
d
of
o
b
jects.
E
x
tr
ac
tin
g
f
ea
tu
r
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
E
n
s
emb
le
w
in
d
o
w
s
in
tr
u
s
io
n
d
etec
tio
n
s
ystem
u
s
in
g
xg
b
o
o
s
t
a
n
d
d
e
ep
…
(
P
r
a
n
ith
a
K
ed
a
m
b
a
d
y
S
h
iva
)
571
m
ap
s
f
r
o
m
in
ter
m
ed
iate
lay
er
s
y
ield
s
r
ich
an
d
v
al
u
ab
le
d
ata
r
ep
r
esen
tatio
n
s
t
h
at
e
n
ca
p
s
u
late
ess
en
tial
in
f
o
r
m
atio
n
,
ad
v
an
ta
g
eo
u
s
f
o
r
s
u
b
s
eq
u
en
t
task
s
.
Utilizin
g
th
ese
attr
ib
u
tes
allo
ws
th
e
f
u
tu
r
e
ML
m
o
d
el
to
ca
p
italize
on
ex
is
tin
g
k
n
o
wle
d
g
e,
h
en
ce
d
im
in
is
h
in
g
th
e
n
ee
d
f
o
r
tr
ain
in
g
d
ata
an
d
f
r
e
q
u
en
tly
en
h
an
cin
g
p
er
f
o
r
m
an
ce
.
I
n
ter
m
e
d
iate
f
ea
tu
r
e
m
ap
s
o
f
ten
f
u
n
ctio
n
as
a
m
eth
o
d
of
d
im
en
s
io
n
ality
r
ed
u
ctio
n
.
R
ath
er
t
h
an
u
tili
zin
g
r
aw
d
ata
with
p
o
s
s
ib
ly
lar
g
e
d
im
e
n
s
io
n
ality
,
th
e
f
ea
tu
r
e
m
a
p
s
d
is
till
p
er
tin
e
n
t
in
f
o
r
m
atio
n
in
a
m
o
r
e
ac
ce
s
s
ib
le
m
an
n
er
.
2
.
6
.
P
er
f
o
r
m
a
nce
m
e
a
s
ures
T
h
e
v
a
r
i
o
u
s
s
t
a
n
d
ar
d
p
er
f
o
r
m
a
n
c
e
m
et
r
i
cs
ar
e
u
s
e
d
to
e
v
al
u
ate
t
h
e
p
er
f
o
r
m
an
ce
of
t
h
e
p
r
o
p
o
s
ed
h
y
b
r
i
d
m
o
d
el
.
T
h
e
e
q
u
at
io
n
s
of
ac
cu
r
a
c
y
,
r
ec
all
,
p
r
e
cisi
o
n
,
a
n
d
F1
s
c
o
r
e
ar
e
g
i
v
en
in
(
3
)
-
(
6
)
,
r
es
p
e
cti
v
el
y
[
3
2
]
.
Acc
u
r
ac
y
:
T
h
e
r
atio
of
co
r
r
ec
t
ly
p
r
ed
icted
ca
s
es
to
th
e
to
tal
n
u
m
b
er
of
in
s
tan
ce
s
.
An
u
n
ev
en
d
ataset
m
ay
y
ield
d
ec
ep
tiv
e
r
esu
lts
.
=
+
(
3
)
R
ec
all
in
d
icate
s
th
e
p
r
o
p
o
r
tio
n
of
a
ctu
al
p
o
s
itiv
e
ca
s
es
th
at
th
e
m
o
d
el
s
u
cc
ess
f
u
lly
id
en
ti
f
ied
.
It
is
ad
v
an
tag
e
o
u
s
wh
en
th
e
p
o
ten
t
ial
f
o
r
f
alse
n
eg
ativ
es
ex
is
ts
.
=
+
(
4
)
Pre
cisi
o
n
:
It
d
eter
m
in
es
th
e
p
r
o
p
o
r
tio
n
of
c
o
r
r
ec
tly
a
n
ticip
at
ed
p
o
s
itiv
e
ca
s
es
r
elativ
e
to
al
l
in
s
tan
ce
s
p
r
ed
icted
as
p
o
s
itiv
e.
Fals
e
p
o
s
itiv
es
ar
e
b
en
ef
icial
wh
en
th
e
y
ar
e
n
ec
ess
ar
y
.
Pr
e
c
isio
n
=
+
Po
s
i
t
i
v
es
(
5
)
F1
s
co
r
e:
It
is
th
e
h
ar
m
o
n
ic
m
ea
n
of
Pre
cisi
o
n
a
n
d
R
ec
all.
It
is
u
s
ed
in
th
e
ca
s
e
of
class
im
b
alan
ce
.
1
=
2
×
×
+
(
6
)
3.
RE
SU
L
T
S
AND
D
I
SCU
SS
I
O
N
T
h
e
ex
p
er
im
e
n
ts
ar
e
p
er
f
o
r
m
e
d
on
Go
o
g
le
C
o
llab
u
s
in
g
th
e
P1
0
0
g
r
ap
h
ical
p
r
o
ce
s
s
in
g
u
n
i
ts
(
GPU
s
)
f
o
r
f
aster
tr
ain
in
g
of
d
ee
p
lear
n
in
g
alg
o
r
ith
m
s
.
T
h
is
p
ap
e
r
id
en
tifie
s
th
e
b
est
ML
al
g
o
r
ith
m
by
im
p
lem
en
tin
g
v
ar
io
u
s
ML
an
d
DL
tech
n
iq
u
es.
T
h
e
h
y
p
er
p
ar
am
eter
s
u
s
ed
f
o
r
th
e
C
u
s
to
m
ized
DNN
ar
e
g
iv
en
in
T
a
b
le
1.
T
h
e
ev
alu
atio
n
of
m
o
d
els
ac
co
r
d
in
g
to
p
r
ec
is
io
n
,
ac
c
u
r
ac
y
,
r
ec
all,
an
d
F1
-
s
co
r
e
r
e
v
ea
ls
d
if
f
er
in
g
p
e
r
f
o
r
m
an
ce
lev
els,
as
s
h
o
wn
in
T
ab
le
2.
T
ab
le
1
.
Fin
etu
n
e
d
Hy
p
e
r
p
ar
a
m
eter
s
f
o
r
DNN
H
y
p
e
r
p
a
r
a
me
t
e
r
s
V
a
l
u
e
N
u
mb
e
r
of
D
e
n
se
l
a
y
e
r
s
3
Ep
o
c
h
s
1
0
0
B
a
t
c
h
si
z
e
1
2
8
Lo
ss
C
r
o
ss
En
t
r
o
p
y
T
ab
le
2
.
C
o
m
p
a
r
ativ
e
an
aly
s
is
of
all
th
e
ML,
DL
,
an
d
p
r
o
p
o
s
ed
h
y
b
r
id
alg
o
r
ith
m
M
o
d
e
l
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F1
sc
o
r
e
GNB
8
9
.
9
1
%
9
0
.
9
0
%
8
9
.
9
1
%
8
9
.
0
9
%
LR
9
9
.
9
0
%
9
9
.
8
5
%
9
9
.
9
0
%
9
9
.
8
8
%
K
N
N
9
9
.
7
6
%
9
9
.
7
7
%
9
9
.
7
6
%
9
9
.
7
6
%
DT
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
RF
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
XGB
9
9
.
9
1
%
9
9
.
8
5
%
9
9
.
9
1
%
9
9
.
8
8
%
DNN
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
9
9
.
9
5
%
P
r
o
p
o
se
d
X
G
B
N
N
1
0
0
%
1
0
0
%
1
0
0
%
1
0
0
%
As
g
iv
en
in
T
a
b
le
2,
GNB
h
as
th
e
least
ef
f
ec
tiv
e
p
er
f
o
r
m
an
c
e,
ac
h
iev
in
g
an
ac
cu
r
ac
y
of
8
9
.
9
1
%
an
d
an
F1
-
s
co
r
e
of
8
9
.
0
9
%,
attr
ib
u
tab
le
to
its
b
asic
ass
u
m
p
tio
n
s
th
at
co
n
s
tr
ain
its
ca
p
ac
ity
to
m
an
ag
e
in
tr
icate
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
5
6
5
-
5
7
7
572
d
ata
p
atter
n
s
.
LR
ac
h
iev
es
a
n
o
tab
le
ac
cu
r
ac
y
of
9
9
.
9
0
%
an
d
an
F1
-
s
co
r
e
of
9
9
.
8
8
%,
in
d
icatin
g
r
eliab
ilit
y
.
Ho
wev
er
,
it
r
em
ain
s
m
ar
g
in
a
lly
in
f
er
io
r
to
ad
v
an
ce
d
m
o
d
els.
KNN
atta
in
s
an
ac
cu
r
ac
y
an
d
F1
-
s
co
r
e
of
9
9
.
7
6
%,
s
lig
h
tly
in
f
er
io
r
to
L
R
,
p
er
h
ap
s
b
ec
au
s
e
of
its
s
u
s
ce
p
tib
ilit
y
to
th
e
s
elec
tio
n
of
th
e
"k
"
p
ar
am
ete
r
.
T
h
e
DT
an
d
RF
b
o
th
attain
9
9
.
9
5
%
ac
r
o
s
s
all
m
etr
ics,
wh
ich
in
d
icate
s
th
eir
ef
f
ec
tiv
en
es
s
an
d
h
ig
h
d
ec
is
io
n
r
ate.
Ho
wev
er
,
XGB
attain
s
9
9
.
8
6
%
ac
r
o
s
s
all
m
etr
ics,
w
h
ich
m
ak
es
it
th
e
s
ec
o
n
d
-
b
est
-
p
er
f
o
r
m
in
g
m
o
d
el.
DNN
also
ac
h
iev
ed
9
9
.
9
5
%,
wh
ich
in
d
icate
s
th
at
more
p
r
o
m
in
en
t
f
ea
t
u
r
es
ar
e
ex
tr
ac
t
ed
th
an
with
XGB.
Fin
ally
,
th
e
h
ig
h
est
ac
cu
r
ac
y
is
o
b
s
er
v
ed
by
th
e
p
r
o
p
o
s
ed
XGBNN
m
o
d
el,
wh
ich
ex
ce
ed
s
all
th
e
b
en
ch
m
ar
k
ed
m
eth
o
d
s
by
ac
h
iev
in
g
1
0
0
%
ac
cu
r
ac
y
alo
n
g
with
o
th
er
p
er
f
o
r
m
an
ce
m
etr
ic
s
,
h
en
ce
p
r
o
v
in
g
its
ex
ce
llen
ce
in
p
r
ec
is
io
n
,
ac
cu
r
a
cy
,
r
ec
all,
an
d
F1
-
s
co
r
e.
W
ith
th
e
co
m
b
in
atio
n
of
DN
N
f
ea
tu
r
e
e
x
tr
ac
tio
n
an
d
XGB
,
it
is
p
o
s
s
ib
le
to
tak
e
a
d
v
a
n
tag
e
of
th
e
b
en
ef
its
th
at
b
o
th
m
o
d
els
o
f
f
e
r
,
wh
ich
u
ltima
tely
r
esu
lts
in
g
r
ea
ter
p
er
f
o
r
m
a
n
ce
in
co
m
p
a
r
is
o
n
to
each
m
o
d
el
wh
en
u
s
ed
alo
n
e
.
T
h
e
lay
e
r
ed
ar
ch
itectu
r
e
of
DNNs
en
ab
les
th
em
to
e
f
f
ec
tiv
ely
ac
q
u
ir
e
c
o
m
p
lex
,
h
ig
h
-
lev
el,
an
d
ab
s
tr
ac
t
f
ea
tu
r
es
f
r
o
m
d
at
a,
th
er
eb
y
id
en
tify
i
n
g
n
u
an
ce
d
p
atter
n
s
th
at
m
ay
be
o
b
s
cu
r
ed
in
th
e
r
aw
d
ata.
Ho
wev
er
,
ev
en
th
o
u
g
h
DNNs
ar
e
q
u
ite
p
o
wer
f
u
l,
th
ey
c
an
s
o
m
etim
es
ex
p
er
ien
ce
o
v
er
f
itti
n
g
or
r
eq
u
ir
e
a
lo
t
of
a
d
ju
s
tm
en
t
to
im
p
r
o
v
e
p
e
r
f
o
r
m
an
ce
.
W
h
en
it
co
m
es
to
m
an
ag
in
g
tab
u
lar
d
ata
with
m
in
im
al
b
ias
an
d
v
o
latilit
y
,
XGB
,
an
i
n
cr
ed
ib
ly
ef
f
icien
t
g
r
a
d
ien
t
-
b
o
o
s
tin
g
a
p
p
r
o
ac
h
,
is
s
ec
o
n
d
to
n
o
n
e.
By
m
ea
n
s
of
XGB
's
ca
p
ac
ity
f
o
r
p
r
o
d
u
cin
g
th
o
r
o
u
g
h
,
h
ig
h
-
q
u
ality
d
ata
r
ep
r
esen
tatio
n
s
,
u
s
in
g
f
ea
tu
r
es
o
b
tain
ed
f
r
o
m
th
e
DNN
as
in
p
u
t
h
elp
s
th
e
s
y
s
tem
to
u
s
e
th
is
ca
p
ab
ilit
y
,
h
e
n
c
e
im
p
r
o
v
i
n
g
XGB
's
ef
f
icac
y
in
class
if
icatio
n
or
r
eg
r
ess
io
n
task
s
.
T
h
is
h
y
b
r
id
tech
n
iq
u
e
im
p
r
o
v
es
p
er
f
o
r
m
an
ce
by
co
m
b
in
in
g
d
ee
p
lear
n
in
g
m
o
d
els'
r
ep
r
esen
tatio
n
al
ca
p
a
b
ilit
ies
w
ith
XGB
's
p
r
ec
is
io
n
an
d
ef
f
ici
en
cy
.
T
h
e
C
o
n
f
u
s
io
n
m
atr
i
x
f
o
r
G
B
N,
L
R
,
KNN,
DT
,
R
F,
XGB,
an
d
th
e
p
r
o
p
o
s
ed
m
o
d
el
is
g
iv
en
in
Fig
u
re
4.
T
h
e
h
ea
tm
ap
of
th
e
co
n
f
u
s
io
n
m
atr
ix
illu
s
tr
ates
th
e
ef
f
icac
y
of
a
m
u
lti
-
class
class
if
icatio
n
m
o
d
el
o
v
er
ei
g
h
t
ca
te
g
o
r
ies
(
d
esig
n
a
ted
0
to
7
)
,
w
h
er
e
0
r
ep
r
esen
ts
DDo
S,
1
r
ep
r
esen
ts
Do
S,
2
r
ep
r
esen
ts
in
jectio
n
,
3
r
ep
r
esen
ts
MI
T
M,
4
r
e
p
r
es
en
ts
n
o
r
m
al,
5
r
e
p
r
esen
ts
a
p
ass
wo
r
d
,
6
r
ep
r
esen
ts
s
ca
n
n
in
g
,
an
d
7
r
ep
r
esen
ts
x
s
s
.
E
ac
h
ce
ll
d
en
o
tes
t
h
e
c
o
u
n
t
of
ca
s
es
ca
teg
o
r
ize
d
as
a
s
p
ec
if
ic
class
,
with
r
o
ws
r
e
f
lectin
g
th
e
ac
tu
al
lab
els
an
d
c
o
lu
m
n
s
r
ef
lectin
g
th
e
an
ticip
ated
lab
els.
T
h
e
d
ia
g
o
n
al
ce
lls
,
ex
ten
d
i
n
g
f
r
o
m
th
e
to
p
lef
t
to
th
e
b
o
tto
m
r
ig
h
t,
s
ig
n
if
y
ac
cu
r
ate
class
i
f
icatio
n
s
.
T
h
e
h
ig
h
est
m
is
class
if
icatio
n
is
o
b
s
er
v
ed
in
GB
N,
as
s
h
o
wn
in
Fig
u
re
4
(
a)
.
C
lass
4,
i.e
.
,
n
o
r
m
al
class
,
h
as
th
e
m
o
s
t
ac
cu
r
a
cy
,
with
1
,
0
1
5
o
cc
u
r
r
en
ce
s
ac
cu
r
ately
ca
teg
o
r
ized
,
an
d
class
2,
i.e
.
,
in
jectio
n
,
d
e
m
o
n
s
tr
ates
th
e
lo
west,
with
m
er
ely
35
c
o
r
r
ec
t
p
r
ed
ictio
n
s
.
Misclass
if
icatio
n
s
o
cc
u
r
in
th
e
o
f
f
-
d
iag
o
n
al
ce
lls
,
wh
er
e,
f
o
r
in
s
tan
ce
,
37
in
s
tan
ce
s
of
class
0,
i.e
.
,
DDo
S,
wer
e
er
r
o
n
e
o
u
s
ly
ca
teg
o
r
ized
as
class
7,
i.e
.
,
x
s
s
,
an
d
75
in
s
tan
ce
s
of
class
0,
i.e
.
,
DDo
S,
wer
e
m
is
class
if
i
ed
as
class
5,
i.e
.
,
p
ass
wo
r
d
.
C
lass
6,
i.e
.
,
s
ca
n
n
i
n
g
,
ex
h
i
b
its
m
in
im
al
m
is
class
if
icatio
n
s
,
with
o
n
ly
one
o
cc
u
r
r
en
ce
er
r
o
n
eo
u
s
ly
ca
teg
o
r
ized
as
class
7,
i.e
.
,
x
s
s
.
T
h
is
m
atr
ix
d
elin
ea
tes
th
e
m
o
d
el'
s
s
tr
en
g
th
s
an
d
wea
k
n
ess
es
ac
r
o
s
s
m
an
y
class
es,
o
f
f
er
in
g
in
s
ig
h
ts
in
to
p
o
ten
tial
ar
ea
s
f
o
r
en
h
a
n
ce
m
e
n
t.
T
h
e
co
n
f
u
s
io
n
m
atr
i
x
f
o
r
LR
is
g
iv
en
in
Fig
u
re
4
(
b
)
,
w
h
ich
s
h
o
ws
2
m
is
s
class
if
icatio
n
s
;
f
o
r
KNN
is
g
iv
en
in
Fig
u
re
4
(
c)
,
wh
ich
s
h
o
ws
3
m
is
s
cla
s
s
if
icatio
n
s
;
f
o
r
DT
is
g
iv
en
in
Fig
u
re
4
(
d
)
wh
ich
s
h
o
ws
1
m
is
s
clas
s
if
icatio
n
;
f
o
r
RF
is
g
iv
en
in
Fig
u
re
4
(
e)
w
h
ich
s
h
o
ws
1
m
is
s
class
if
icatio
n
s
,
f
o
r
XGB
is
g
iv
en
in
Fig
u
re
4
(f)
wh
ich
s
h
o
ws
2
m
is
s
class
if
icat
io
n
s
,
f
o
r
D
NN
is
g
iv
en
in
f
ig
u
re
4
(
g
)
wh
ich
s
h
o
ws
2
m
is
s
clas
s
if
icatio
n
s
,
an
d
f
o
r
p
r
o
p
o
s
ed
XGBNN
is
g
iv
en
in
Fig
u
re
4
(
h
)
,
wh
ich
s
h
o
ws
no
m
i
s
s
clas
s
if
icatio
n
s
.
Ho
wev
er
,
all
th
e
o
th
er
alg
o
r
ith
m
s
h
av
e
v
er
y
f
ew
m
is
class
if
icatio
n
s
,
an
d
th
e
Pro
p
o
s
ed
X
GB
NN
h
as
no
m
is
class
if
icatio
n
s
,
th
u
s
h
av
in
g
1
0
0
%
ac
cu
r
ac
y
.
T
h
e
ac
cu
r
ac
y
g
r
ap
h
s
h
o
wn
in
Fig
u
re
5
illu
s
tr
ates
a
r
ap
id
en
h
an
ce
m
e
n
t
in
th
e
DNN
m
o
d
el'
s
p
er
f
o
r
m
an
ce
on
b
o
th
t
h
e
tr
ain
in
g
an
d
v
alid
atio
n
d
a
tasets
.
T
h
e
tr
ain
in
g
ac
cu
r
ac
y
,
d
e
n
o
ted
by
th
e
b
l
u
e
lin
e,
b
eg
in
s
at
a
r
ea
s
o
n
ab
ly
elev
ated
lev
el
an
d
ap
p
r
o
ac
h
es
9
9
.
9
5
%
d
u
r
in
g
th
e
in
itial
ep
o
ch
s
,
s
ig
n
if
y
in
g
th
at
t
h
e
m
o
d
el
r
a
p
id
ly
ass
im
ilates
t
h
e
p
atter
n
s
in
th
e
tr
ain
in
g
d
ata
.
C
o
r
r
esp
o
n
d
in
g
ly
,
th
e
v
alid
atio
n
ac
cu
r
ac
y
(
s
h
o
wn
by
th
e
o
r
an
g
e
lin
e)
i
n
cr
ea
s
es
s
ig
n
if
ican
tly
in
th
e
ea
r
ly
ep
o
ch
s
an
d
s
u
b
s
eq
u
en
tly
s
tab
ilizes
n
ea
r
9
9
.
9
5
%,
im
p
ly
in
g
th
at
th
e
m
o
d
el
g
en
er
alize
s
ef
f
ec
tiv
ely
to
u
n
s
ee
n
d
ata.
T
h
e
elev
ated
,
c
o
n
s
is
ten
t
ac
cu
r
ac
y
f
o
r
b
o
th
tr
ain
in
g
a
n
d
v
alid
atio
n
d
atasets
in
d
icate
s
th
at
th
e
m
o
d
el
is
f
u
n
ctio
n
i
n
g
ef
f
ec
tiv
ely
with
o
u
t
c
o
n
s
id
er
ab
le
o
v
er
f
itti
n
g
.
T
h
e
tr
ain
in
g
an
d
v
alid
atio
n
l
o
s
s
es,
wh
ich
m
ea
s
u
r
e
p
r
ed
ictio
n
er
r
o
r
ac
r
o
s
s
th
e
s
am
e
ep
o
ch
s
,
ar
e
d
ep
icted
in
t
h
e
lo
s
s
g
r
ap
h
in
F
ig
u
re
6.
As
th
e
b
lu
e
lin
e
s
h
o
w
s
,
th
e
tr
ain
in
g
l
o
s
s
s
tar
ts
h
ig
h
er
but
q
u
ick
ly
d
r
o
p
s
d
o
wn
in
t
h
e
ea
r
ly
e
p
o
ch
s
,
g
et
tin
g
clo
s
e
to
ze
r
o
,
wh
ich
m
ea
n
s
th
e
m
o
d
el
is
d
o
in
g
a
g
o
o
d
jo
b
of
elim
in
atin
g
er
r
o
r
s
in
th
e
tr
ain
i
n
g
d
ata.
T
h
e
v
alid
atio
n
lo
s
s
r
ep
r
esen
ted
by
th
e
o
r
a
n
g
e
lin
e
r
e
d
u
ce
s
s
ig
n
if
ican
tly
o
v
er
th
e
in
itial
ep
o
ch
s
an
d
s
tab
ilizes
n
ea
r
ze
r
o
,
s
h
o
win
g
th
at
th
e
m
o
d
el'
s
p
r
ed
ictio
n
s
r
em
ain
ac
c
u
r
ate
on
p
r
ev
io
u
s
ly
u
n
k
n
o
wn
d
ata.
T
h
e
p
r
o
x
im
ity
of
th
e
tr
ain
in
g
lo
s
s
lev
el
an
d
th
e
v
alid
atio
n
lo
s
s
lev
el,
b
o
th
of
wh
ich
ar
e
clo
s
e
to
ze
r
o
,
len
d
s
co
n
f
i
d
en
ce
to
t
h
e
id
ea
th
at
th
e
m
o
d
el
h
as
n
o
t
o
v
er
f
it
an
d
is
ef
f
ec
tiv
ely
g
e
n
er
a
lizin
g
k
n
o
wled
g
e.
Fig
u
re
5
in
d
icate
s
th
at
th
e
DNN
m
o
d
el
is
ef
f
ec
tiv
ely
o
p
ti
m
ized
.
It
attain
s
elev
ated
ac
cu
r
ac
y
an
d
r
ed
u
ce
d
l
o
s
s
on
b
o
th
th
e
tr
ain
i
n
g
an
d
v
alid
atio
n
d
atasets
,
ex
h
ib
itin
g
n
eg
li
g
ib
le
d
iv
er
g
en
ce
b
etwe
en
th
em
.
T
h
e
eq
u
ilib
r
iu
m
of
h
ig
h
ac
cu
r
ac
y
an
d
m
in
im
al
lo
s
s
on
u
n
k
n
o
wn
d
ata
s
ig
n
if
ies
th
at
th
e
m
o
d
el
h
as
ac
q
u
ir
e
d
k
n
o
wled
g
e
ef
f
icien
tly
with
o
u
t
o
v
er
f
itti
n
g
,
r
en
d
er
in
g
it
d
ep
en
d
ab
le
f
o
r
p
r
ac
tical
ap
p
licatio
n
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
E
n
s
emb
le
w
in
d
o
w
s
in
tr
u
s
io
n
d
etec
tio
n
s
ystem
u
s
in
g
xg
b
o
o
s
t
a
n
d
d
e
ep
…
(
P
r
a
n
ith
a
K
ed
a
m
b
a
d
y
S
h
iva
)
573
(
a)
GNB
(
b
)
LR
(
c)
KNN
(
d
)
DT
(
e)
R
F
(f)
XGB
(
g
)
DNN
(
h
)
Pro
p
o
s
ed
h
y
b
r
id
m
o
d
el
Fig
u
r
e
4
.
T
h
e
co
n
f
u
s
io
n
m
atr
ix
wh
er
e
0
r
ep
r
esen
ts
DDo
S,
1
r
ep
r
esen
ts
d
o
s
,
2
r
ep
r
esen
ts
in
jectio
n
,
3
r
ep
r
esen
ts
m
itm
,
4
r
ep
r
esen
ts
n
o
r
m
al,
5
r
ep
r
esen
ts
a
p
ass
wo
r
d
,
6
r
ep
r
esen
ts
s
ca
n
n
in
g
,
an
d
7
r
ep
r
esen
ts
x
s
s
.
a)
GB
N;
b
)
L
R
; c
)
KNN;
d
)
DT
; e
)
R
F; f
)
XGB;
g
)
DNN;
h
)
Pro
p
o
s
ed
XGBNN
Fig
u
re
7
illu
s
tr
ates
th
e
r
ec
eiv
er
o
p
er
atin
g
c
h
ar
ac
ter
is
tic
(
R
O
C
)
cu
r
v
es
f
o
r
a
m
u
lti
-
class
cla
s
s
if
icatio
n
m
o
d
el
GNB
en
co
m
p
ass
in
g
eig
h
t
class
es,
d
esig
n
ated
0
to
7.
T
h
is
g
r
ap
h
illu
s
tr
ates
th
at
class
es
4
an
d
7
ex
h
ib
it
th
e
m
o
s
t
s
ig
n
if
ican
t
AUC
v
alu
es
of
1
.
0
0
,
in
d
icatin
g
n
ea
r
-
o
p
ti
m
al
ef
f
icac
y
in
d
if
f
er
e
n
tiatin
g
th
ese
g
r
o
u
p
s
f
r
o
m
th
e
r
em
ain
d
er
.
C
lass
es
0,
2,
a
n
d
6
ex
h
ib
it
ex
ce
llen
t
p
er
f
o
r
m
an
ce
,
ev
id
en
ce
d
by
h
ig
h
AUC
v
alu
es
of
0
.
9
9
an
d
0
.
9
8
,
s
ig
n
if
y
in
g
th
e
m
o
d
el'
s
r
o
b
u
s
t
ca
p
ac
ity
to
d
is
tin
g
u
is
h
t
h
ese
class
es.
C
la
s
s
es
1
an
d
5
ex
h
ib
it
m
ar
g
in
ally
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
5
6
5
-
5
7
7
574
lo
wer
AUC
v
alu
es
of
0
.
9
5
;
h
o
wev
er
,
th
ese
a
r
e
s
till
s
u
g
g
esti
v
e
of
c
o
m
m
en
d
ab
le
p
e
r
f
o
r
m
an
ce
.
T
h
e
elev
ated
AUC
v
alu
es
f
o
r
all
class
es
d
em
o
n
s
tr
ate
th
at
th
e
m
o
d
el
ef
f
ec
tiv
ely
d
is
tin
g
u
is
h
es
each
cl
ass
f
r
o
m
th
e
o
th
er
s
with
n
eg
lig
ib
le
co
n
f
u
s
io
n
.
T
h
is
in
d
icate
s
r
o
b
u
s
t
m
o
d
el
p
er
f
o
r
m
a
n
ce
an
d
ef
f
icien
t
ca
t
eg
o
r
izatio
n
ac
r
o
s
s
s
ev
er
al
class
es.
Fig
u
re
8
s
h
o
ws
th
e
R
OC
cu
r
v
e
f
o
r
L
R
,
R
F,
XGB,
DNN,
an
d
Pro
p
o
s
ed
XGBNN,
wh
ich
in
d
icate
s
th
at
all
th
e
class
es
h
av
e
an
AUC
of
1,
wh
ile
Fig
u
re
9
s
h
o
ws
th
e
R
OC
cu
r
v
e
of
th
e
KNN
an
d
DT
class
if
ier
s
,
in
d
icatin
g
th
at
all
th
e
class
es
h
av
e
an
AUC
of
1,
ex
ce
p
t
class
6,
i.e
.
,
s
ca
n
n
in
g
.
T
h
e
p
r
o
p
o
s
e
d
XGBNN
alg
o
r
ith
m
is
co
m
p
ar
ed
with
th
e
b
e
n
ch
m
ar
k
ed
tech
n
iq
u
es,
as
s
h
o
wn
in
T
a
b
le
3.
Fig
u
r
e
5
.
Acc
u
r
ac
y
o
v
er
th
e
e
p
o
ch
s
f
o
r
DNN
Fig
u
r
e
6
.
L
o
s
s
o
v
er
th
e
e
p
o
ch
s
f
o
r
DNN
Fig
u
re
7
.
R
OC
of
GNB
Fig
u
re
8
.
R
OC
of
L
R
,
R
F,
XGB,
DNN,
an
d
Pro
p
o
s
ed
XGBNN
Evaluation Warning : The document was created with Spire.PDF for Python.