I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
15
,
No
.
4
,
A
u
g
u
s
t
20
25
,
p
p
.
4
1
6
0
~
4
1
7
1
I
SS
N:
2088
-
8
7
0
8
,
DOI
:
1
0
.
1
1
5
9
1
/ijece.
v
15
i
4
.
pp
4
1
6
0
-
4
1
7
1
4160
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
Exploring
t
h
e re
c
urrent
and sequ
e
ntial securi
ty pa
tc
h data
using
deep
l
ea
rni
ng
appro
a
ches
F
a
la
h M
uh
a
m
m
a
d Ala
m
,
Dev
i F
it
ria
na
h
D
e
p
a
r
t
me
n
t
o
f
Te
c
h
n
o
l
o
g
y
I
n
f
o
r
m
a
t
i
o
n
,
F
a
c
u
l
t
y
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
B
i
n
u
s U
n
i
v
e
r
si
t
y
,
J
a
k
a
r
t
a
,
I
n
d
o
n
e
s
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
1
8
,
2
0
2
4
R
ev
is
ed
Ap
r
1
0
,
2
0
2
5
Acc
ep
ted
Ma
y
2
4
,
2
0
2
5
Th
e
e
v
e
r
-
c
h
a
n
g
i
n
g
n
a
tu
re
o
f
v
u
ln
e
ra
b
il
it
ies
a
n
d
t
h
e
in
tri
c
a
c
y
o
f
tem
p
o
ra
l
c
o
n
n
e
c
ti
o
n
s
m
a
k
e
th
e
c
las
sifica
ti
o
n
o
f
se
c
u
rit
y
p
a
tch
d
a
ta,
b
o
th
s
e
q
u
e
n
ti
a
l
a
n
d
re
c
u
rre
n
t,
a
fo
rm
id
a
b
le
c
h
a
ll
e
n
g
e
in
c
y
b
e
rse
c
u
rit
y
.
T
h
e
g
o
a
l
o
f
th
i
s
re
se
a
rc
h
is
to
imp
ro
v
e
t
h
e
e
ffica
c
y
a
n
d
p
re
c
isio
n
o
f
se
c
u
ri
ty
p
a
tch
m
a
n
a
g
e
m
e
n
t
b
y
o
p
ti
m
izin
g
d
e
e
p
lea
rn
in
g
m
o
d
e
ls
to
d
e
a
l
with
th
e
se
issu
e
s.
In
o
rd
e
r
t
o
a
ss
e
ss
th
e
ir
p
e
rfo
rm
a
n
c
e
o
n
th
e
P
a
tch
DB
d
a
tas
e
t,
fo
u
r
m
o
d
e
ls
we
re
u
se
d
:
re
c
u
rre
n
t
n
e
u
ra
l
n
e
two
rk
s
(RNN
),
lo
n
g
sh
o
rt
-
term
m
e
m
o
ry
(LS
TM
),
g
a
ted
re
c
u
rre
n
t
u
n
it
(G
RU),
a
n
d
b
id
irec
ti
o
n
a
l
LS
T
M
(B
i
-
LS
TM
).
M
e
tri
c
s
li
k
e
F1
-
sc
o
re
,
a
re
a
u
n
d
e
r
th
e
re
c
e
iv
e
r
o
p
e
ra
ti
n
g
c
h
a
ra
c
teristic
c
u
rv
e
(AU
C
-
ROC)
,
re
c
a
ll
,
a
c
c
u
ra
c
y
,
a
n
d
p
re
c
isio
n
we
re
u
se
d
to
e
v
a
lu
a
te
p
e
rfo
rm
a
n
c
e
.
Wh
e
n
it
c
a
m
e
to
p
ro
c
e
ss
in
g
se
q
u
e
n
ti
a
l
d
a
ta,
t
h
e
G
RU
m
o
d
e
l
wa
s th
e
m
o
st efficie
n
t,
wit
h
t
h
e
b
e
st ac
c
u
ra
c
y
(7
7
.
3
9
%
)
,
re
c
a
ll
(6
5
.
6
3
%
),
a
n
d
AUC
-
ROC
sc
o
re
(0
.
8
1
2
7
).
Wi
t
h
a
7
5
.
1
7
%
a
c
c
u
ra
c
y
ra
te
a
n
d
a
n
A
UC
-
ROC
sc
o
re
o
f
0
.
7
7
5
2
,
th
e
RNN
m
o
d
e
l
su
c
c
e
ss
fu
ll
y
re
d
u
c
e
d
fa
lse
n
e
g
a
ti
v
e
s.
Wi
t
h
AUC
-
ROC
sc
o
re
s
o
f
0
.
7
7
9
2
a
n
d
0
.
8
0
5
5
,
re
sp
e
c
ti
v
e
ly
,
LS
TM
a
n
d
Bi
-
LS
TM
h
a
d
b
e
tt
e
r
sp
e
c
ifi
c
it
y
b
u
t
m
o
re
fa
lse
n
e
g
a
ti
v
e
s.
T
o
imp
r
o
v
e
c
y
b
e
rse
c
u
rit
y
o
p
e
ra
ti
o
n
s,
d
e
c
re
a
se
m
it
ig
a
ti
o
n
ti
m
e
,
a
n
d
a
u
to
m
a
te
th
e
c
las
sifi
c
a
ti
o
n
o
f
se
c
u
rit
y
u
p
d
a
tes
,
th
is
stu
d
y
p
re
se
n
ts
a
m
e
th
o
d
o
lo
g
y
.
T
o
imp
r
o
v
e
t
h
e
m
o
d
e
ls'
p
ra
c
ti
c
a
li
ty
,
f
u
tu
re
e
ff
o
rts
will
c
e
n
ter
o
n
i
n
c
re
a
sin
g
d
a
tas
e
ts
a
n
d
te
stin
g
th
e
m
in
re
a
l
-
wo
rl
d
se
tt
in
g
s.
K
ey
w
o
r
d
s
:
B
id
ir
ec
tio
n
al
l
o
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
Dee
p
l
ea
r
n
in
g
Gate
d
r
ec
u
r
r
e
n
t u
n
it
L
o
n
g
s
h
o
r
t
-
ter
m
m
e
m
o
r
y
R
ec
u
r
r
en
t n
eu
r
al
n
etwo
r
k
s
Secu
r
ity
p
atch
es
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Falah
Mu
h
am
m
ad
Alam
Dep
ar
tm
en
t o
f
T
ec
h
n
o
lo
g
y
I
n
f
o
r
m
atio
n
,
Facu
lty
o
f
C
o
m
p
u
te
r
Scien
ce
,
B
in
u
s
Un
iv
er
s
ity
Keb
o
n
J
er
u
k
St.
No
.
2
7
,
W
est J
ak
ar
ta
1
1
5
3
0
,
DKI
J
ak
ar
ta,
I
n
d
o
n
esia
E
m
ail: f
alah
.
alam
@
b
in
u
s
.
ac
.
id
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
f
ast
d
e
v
elo
p
m
en
t
o
f
d
ig
it
al
tech
n
o
lo
g
y
an
d
th
e
r
is
in
g
d
ep
en
d
en
ce
o
n
s
o
f
twar
e
in
f
r
ast
r
u
ctu
r
e
in
m
an
y
s
ec
to
r
s
h
av
e
m
a
d
e
cy
b
er
s
ec
u
r
ity
a
p
ar
am
o
u
n
t
co
n
ce
r
n
in
th
e
r
ec
e
n
t
d
ec
ad
e
.
B
y
f
ix
in
g
p
r
ev
io
u
s
ly
d
is
co
v
er
ed
s
o
f
twar
e
v
u
l
n
er
ab
i
liti
es,
s
ec
u
r
ity
p
atch
es
p
r
o
tect
s
y
s
tem
s
f
r
o
m
a
b
r
o
ad
v
ar
iety
o
f
n
ew
th
r
ea
ts
[
1
]
.
T
h
e
d
an
g
er
s
o
f
c
y
b
er
attac
k
s
wh
ich
ca
n
r
e
s
u
lt
in
s
u
b
s
t
an
tial
f
in
an
cial
lo
s
s
es,
r
ep
u
tatio
n
al
h
ar
m
,
an
d
in
ter
r
u
p
tio
n
s
to
o
p
er
atio
n
s
m
u
s
t
b
e
m
itig
ated
b
y
a
p
p
ly
in
g
th
ese
f
ix
es
in
a
tim
ely
an
d
ef
f
ec
tiv
e
m
an
n
e
r
.
B
u
t
th
er
e
ar
e
a
l
o
t
o
f
o
b
s
tacle
s
to
o
v
er
co
m
e
wh
en
an
aly
zin
g
a
n
d
d
ep
lo
y
i
n
g
s
ec
u
r
ity
p
atch
es.
T
h
is
is
esp
ec
iall
y
tr
u
e
b
ec
au
s
e
p
atc
h
d
ata
is
s
eq
u
en
tial a
n
d
r
ec
u
r
r
in
g
,
wh
ich
m
ak
es a
n
aly
s
is
an
d
p
r
e
d
ictio
n
m
o
r
e
d
if
f
icu
lt
[
2
]
.
I
t
is
n
ec
ess
ar
y
to
ap
p
ly
u
p
d
at
es
in
a
p
r
ec
is
e
s
eq
u
en
ce
to
f
ix
v
u
ln
er
a
b
ilit
ies
as
th
ey
o
cc
u
r
,
an
d
th
is
s
eq
u
en
tial
p
atter
n
is
ty
p
ically
s
ee
n
in
s
ec
u
r
ity
p
atch
d
ata.
T
h
e
f
ac
t
t
h
at
u
p
d
ates
ar
e
o
f
ten
ap
p
lied
to
m
u
ltip
le
s
o
f
twar
e
v
er
s
io
n
s
to
ad
d
r
ess
th
e
s
am
e
o
r
s
im
ilar
v
u
ln
er
ab
ilit
ies
f
u
r
th
er
c
o
m
p
licates
m
atter
s
[
3
]
.
Static
an
aly
s
is
an
d
r
u
le
-
b
ased
ap
p
r
o
ac
h
es
ar
e
e
x
am
p
les
o
f
t
r
ad
itio
n
al
m
eth
o
d
s
th
at
u
s
e
h
eu
r
is
t
ics
an
d
p
r
ed
ef
i
n
ed
r
u
les
to
f
in
d
v
u
ln
er
a
b
ilit
ies.
A
lth
o
u
g
h
th
ese
m
eth
o
d
s
wo
r
k
well
in
s
ta
tic
s
i
tu
atio
n
s
,
th
ey
ca
n
'
t
h
an
d
le
d
ata
th
at
i
s
in
tr
in
s
ically
s
eq
u
en
tial
an
d
tim
e
-
s
en
s
itiv
e,
wh
ich
m
ak
es
it
d
if
f
icu
lt
to
ca
p
tu
r
e
h
o
w
s
o
f
t
war
e
v
u
ln
er
ab
ilit
ies
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
E
xp
lo
r
in
g
th
e
r
ec
u
r
r
en
t a
n
d
s
eq
u
en
tia
l secu
r
ity
p
a
tch
d
a
ta
u
s
in
g
…
(
F
a
la
h
Mu
h
a
mma
d
A
la
m
)
4161
ch
an
g
e
o
v
e
r
tim
e
[
4
]
.
T
h
is
s
h
o
r
tco
m
in
g
h
as
p
r
o
m
p
ted
r
esear
ch
in
to
m
o
r
e
s
o
p
h
is
ticated
m
eth
o
d
s
th
at
ca
n
h
an
d
le
th
e
i
n
tr
icac
ies o
f
s
ec
u
r
i
ty
p
atch
d
ata.
B
ec
au
s
e
o
f
its
ca
p
ac
ity
to
h
a
n
d
le
m
ass
iv
e
am
o
u
n
ts
o
f
co
m
p
licated
d
ata
an
d
d
is
co
v
er
p
atter
n
s
th
at
co
n
v
en
tio
n
al
ap
p
r
o
ac
h
es f
r
eq
u
en
tly
f
ail
to
n
o
tice,
d
ee
p
lear
n
in
g
h
as a
r
is
en
as a
p
o
ten
tial
o
p
tio
n
f
o
r
ev
alu
atin
g
s
eq
u
en
tial
an
d
tem
p
o
r
al
d
ata
[
5
]
.
I
n
tr
u
s
io
n
d
etec
tio
n
,
m
alwa
r
e
ca
teg
o
r
izatio
n
,
an
d
v
u
ln
er
ab
ilit
y
ass
ess
m
en
t
ar
e
ju
s
t
a
f
ew
o
f
th
e
cy
b
er
s
ec
u
r
ity
ap
p
licatio
n
s
wh
er
e
d
ee
p
lear
n
in
g
m
o
d
els
h
av
e
b
ee
n
s
h
o
wn
to
b
e
ef
f
ec
tiv
e
in
r
ec
en
t
s
tu
d
ies
[
6
]
.
Secu
r
i
ty
p
atch
class
if
icatio
n
task
s
a
r
e
well
-
s
u
ited
f
o
r
m
o
d
els
lik
e
r
ec
u
r
r
en
t
n
eu
r
al
n
etwo
r
k
s
(
R
NN)
,
lo
n
g
s
h
o
r
t
-
t
er
m
m
em
o
r
y
(
L
STM
)
,
g
ated
r
ec
u
r
r
en
t
u
n
its
(
GR
U)
,
an
d
b
id
ir
ec
tio
n
al
L
STM
(
B
i
-
L
STM
)
b
ec
au
s
e
o
f
th
eir
ef
f
ec
tiv
en
ess
in
lear
n
in
g
lo
n
g
-
ter
m
d
e
p
en
d
e
n
cies
wi
th
in
s
eq
u
en
ce
s
[
7
]
,
[
8
]
.
W
h
en
it
co
m
es
to
th
ese,
B
i
-
L
STM
s
tan
d
s
o
u
t
s
in
ce
it
tak
es
in
to
ac
co
u
n
t
b
o
th
th
e
p
ast
an
d
th
e
f
u
tu
r
e
,
wh
ich
im
p
r
o
v
es its
p
r
ed
ictio
n
ab
ilit
ies in
s
itu
atio
n
s
wh
er
e
th
e
s
eq
u
e
n
ce
o
f
e
v
en
ts
is
cr
u
cial
[
7
]
.
E
x
am
in
in
g
an
d
co
n
tr
asti
n
g
th
e
p
er
f
o
r
m
an
ce
o
f
R
NN,
L
STM
,
GR
U,
an
d
B
i
-
L
STM
m
o
d
els
i
n
d
ea
lin
g
with
s
eq
u
en
tial
an
d
r
ec
u
r
r
e
n
t
s
ec
u
r
ity
p
atch
d
a
ta
is
th
e
m
ain
g
o
al
o
f
th
is
r
esear
ch
.
T
h
i
s
wo
r
k
th
o
r
o
u
g
h
ly
ev
alu
ates
th
ese
m
o
d
els
u
s
in
g
th
e
Patch
DB
d
ataset.
I
t
em
p
lo
y
s
r
ig
o
r
o
u
s
ex
p
er
im
en
tal
ap
p
r
o
ac
h
es
s
u
ch
as
d
ata
p
r
ep
ar
atio
n
,
h
y
p
e
r
p
ar
am
eter
t
u
n
in
g
,
tr
ain
in
g
,
v
alid
atio
n
,
a
n
d
test
in
g
.
T
o
f
in
d
t
h
e
b
est
m
e
th
o
d
f
o
r
s
ec
u
r
ity
p
atch
class
if
icatio
n
,
s
cien
tis
ts
em
p
lo
y
m
etr
ics
in
clu
d
in
g
r
ec
a
ll,
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
F1
-
s
co
r
e,
an
d
ar
ea
u
n
d
er
th
e
r
ec
eiv
er
o
p
er
atin
g
ch
ar
ac
t
er
is
tic
cu
r
v
e
(
AUC
-
R
OC
)
to
ev
alu
ate
m
o
d
el
p
er
f
o
r
m
an
ce
[
9
]
–
[
1
1
]
.
T
h
is
s
tu
d
y
d
o
es
m
o
r
e
th
an
ju
s
t
co
m
p
ar
e
m
o
d
el
p
e
r
f
o
r
m
an
ce
;
it
al
s
o
lo
o
k
s
in
to
h
o
w
h
y
p
er
p
ar
am
eter
ad
ju
s
tm
en
t
ca
n
im
p
r
o
v
e
d
ee
p
lear
n
in
g
m
o
d
els'
ef
f
icac
y
[
1
2
]
.
T
h
is
p
r
o
ject
s
ee
k
s
to
d
ev
elo
p
a
s
t
r
o
n
g
f
r
am
ewo
r
k
f
o
r
an
al
y
zin
g
s
ec
u
r
ity
p
atch
d
ata
b
y
d
eter
m
in
i
n
g
o
p
tim
al
co
n
f
ig
u
r
atio
n
s
.
B
ec
au
s
e
th
ey
p
r
o
v
i
d
e
u
s
ef
u
l
in
f
o
r
m
atio
n
ab
o
u
t
h
o
w
to
u
s
e
d
ee
p
lear
n
in
g
to
au
to
m
ate
p
atch
m
an
ag
em
e
n
t
p
r
o
ce
s
s
es,
th
e
r
esu
lt
s
s
h
o
u
ld
h
av
e
a
m
ajo
r
im
p
ac
t o
n
cy
b
er
s
ec
u
r
ity
.
I
n
th
e
en
d
,
th
i
s
r
esear
ch
s
e
t
s
th
e
s
tag
e
f
o
r
cr
ea
tin
g
s
o
p
h
is
ticated
s
y
s
tem
s
th
at
en
h
an
ce
th
e
d
ep
e
n
d
ab
ilit
y
a
n
d
e
f
f
ec
tiv
en
ess
o
f
cy
b
er
s
ec
u
r
ity
p
r
o
ce
s
s
es,
d
ec
r
e
asin
g
th
e
n
ee
d
f
o
r
h
u
m
an
i
n
ter
v
en
tio
n
an
d
lo
wer
in
g
th
e
r
is
k
s
lin
k
ed
to
s
o
f
twar
e
v
u
ln
er
a
b
ilit
ies.
2.
T
H
E
CO
M
P
RE
H
E
NS
I
VE
T
H
E
O
RE
T
I
C
A
L
B
ASI
S
Gr
asp
in
g
h
o
w
d
ee
p
lear
n
i
n
g
a
r
ch
itectu
r
es
ad
d
r
ess
th
e
s
eq
u
en
tial
an
d
r
ep
etitiv
e
elem
en
ts
in
tr
in
s
ic
to
s
ec
u
r
ity
p
atch
d
ata
is
p
ar
am
o
u
n
t
f
o
r
en
s
u
r
in
g
ac
c
u
r
ate
p
atch
class
if
icatio
n
.
R
NN,
L
S
T
M,
GR
U,
an
d
B
i
-
L
STM
ar
e
s
p
ec
if
ically
d
esig
n
ed
to
ef
f
ec
tiv
el
y
ca
p
tu
r
e
te
m
p
o
r
al
d
ep
en
d
en
cies
in
s
eq
u
en
tial
d
atasets
.
B
y
lev
er
ag
in
g
th
eir
d
is
tin
ctiv
e
ab
ilit
y
to
r
etain
h
is
to
r
ical
co
n
tex
t
an
d
m
an
a
g
e
d
ata
s
eq
u
en
ce
s
with
v
ar
y
i
n
g
co
m
p
lex
ities
,
th
ese
ar
ch
itectu
r
es si
g
n
if
ican
tly
im
p
r
o
v
e
th
e
cl
ass
if
icatio
n
ac
cu
r
ac
y
o
f
s
ec
u
r
it
y
p
atch
es.
2
.
1
.
Rec
urre
nt
a
nd
s
eq
uentia
l d
a
t
a
in cy
bersecurit
y
Patch
es
ar
e
ty
p
ically
is
s
u
ed
i
n
a
s
eq
u
en
tial
f
ash
io
n
to
ad
d
r
ess
d
if
f
er
en
t
s
o
f
twar
e
v
u
ln
e
r
ab
ilit
ies
a
s
th
ey
ar
e
f
o
u
n
d
,
b
ec
a
u
s
e
s
ec
u
r
ity
p
atch
d
ata
is
n
atu
r
ally
r
ec
u
r
r
en
t
a
n
d
s
eq
u
e
n
tial.
B
ec
au
s
e
o
f
th
is
s
eq
u
en
tial
ch
ar
ac
ter
,
th
e
d
ata
tak
es
o
n
a
tim
e
d
im
en
s
io
n
;
th
e
s
eq
u
en
c
e
an
d
tim
in
g
o
f
p
atch
r
elea
s
e
s
ca
n
h
av
e
a
m
ajo
r
ef
f
ec
t
o
n
a
s
y
s
tem
'
s
o
v
er
all
s
ec
u
r
ity
.
T
h
ese
tem
p
o
r
al
d
ep
en
d
en
cies
ar
e
cr
u
cial
f
o
r
g
o
o
d
v
u
ln
e
r
ab
ilit
y
m
an
ag
em
en
t,
y
et
tr
ad
itio
n
al
a
p
p
r
o
ac
h
es
to
v
u
ln
e
r
ab
ilit
y
d
et
ec
tio
n
,
s
u
ch
as
s
tatic
an
aly
s
is
m
eth
o
d
s
,
g
en
e
r
ally
f
ail
to
ca
p
tu
r
e
th
em
[
1
]
.
R
es
ea
r
ch
h
as
d
em
o
n
s
tr
ated
th
at
s
tatic
an
aly
s
i
s
m
et
h
o
d
s
h
av
e
th
eir
u
s
es,
b
u
t
th
ey
f
r
eq
u
e
n
tly
o
v
er
lo
o
k
s
o
f
twar
e
v
u
ln
er
ab
ilit
ies'
ev
er
-
ch
an
g
in
g
n
atu
r
e
,
esp
ec
ially
wh
en
th
ey
h
ap
p
en
r
ep
ea
ted
l
y
an
d
s
eq
u
en
tially
[
5
]
.
R
NNs
an
d
L
STM
n
etwo
r
k
s
,
wh
ich
ar
e
s
p
ec
if
ically
d
esig
n
ed
f
o
r
s
eq
u
en
tial
d
ata,
p
r
o
v
i
d
e
a
p
o
ten
tial
s
o
lu
tio
n
to
th
ese
p
r
o
b
lem
s
b
y
ac
cu
r
ately
m
o
d
ell
in
g
th
e
tem
p
o
r
al
r
elatio
n
s
h
ip
s
in
th
e
d
ata
,
wh
ic
h
im
p
r
o
v
es th
e
r
eliab
ilit
y
an
d
ac
cu
r
ac
y
o
f
v
u
ln
er
ab
ilit
y
d
etec
ti
o
n
[
2
]
.
2
.
2
.
Rec
urre
nt
n
eura
l net
wo
rk
s
(
RNN)
T
im
e
s
er
ies
an
aly
s
is
is
o
n
e
ar
ea
wh
er
e
th
e
s
eq
u
en
ce
o
f
d
ata
p
o
in
ts
is
v
er
y
im
p
o
r
tan
t,
an
d
o
n
e
o
f
th
e
f
u
n
d
am
e
n
tal
d
esig
n
s
f
o
r
h
a
n
d
lin
g
s
eq
u
e
n
tial
d
ata
is
t
h
e
R
NN
[
8
]
.
R
NNs
lear
n
d
e
p
en
d
e
n
cies
o
v
er
tim
e
b
y
p
r
eser
v
in
g
in
f
o
r
m
atio
n
f
r
o
m
p
ast
tim
e
s
tep
s
in
a
h
id
d
en
s
tate.
T
h
e
v
an
is
h
in
g
g
r
a
d
ien
t
p
r
o
b
lem
is
a
well
-
k
n
o
wn
is
s
u
e
with
R
NNs;
i
t
h
ap
p
en
s
wh
en
th
e
g
r
ad
ien
ts
u
tili
ze
d
in
b
ac
k
p
r
o
p
a
g
atio
n
g
et
to
o
s
m
all,
p
r
ev
en
tin
g
th
e
n
etwo
r
k
f
r
o
m
lear
n
in
g
d
at
a
d
ep
en
d
en
cies
o
v
er
th
e
lo
n
g
t
er
m
.
W
h
en
w
o
r
k
in
g
with
le
n
g
th
y
s
eq
u
e
n
ce
s
,
th
is
is
s
u
e
ca
n
s
ev
er
ely
im
p
ai
r
R
NN
p
er
f
o
r
m
an
ce
,
an
d
it
is
esp
ec
ially
n
o
ticea
b
le
in
d
ee
p
n
etwo
r
k
s
[
1
3
]
.
I
n
s
p
ite
o
f
th
ese
o
b
s
tacle
s
,
R
NN
s
h
av
e
e
s
tab
lis
h
ed
a
f
o
u
n
d
atio
n
f
o
r
co
m
p
r
eh
e
n
d
in
g
s
eq
u
en
tial
in
p
u
t,
an
d
th
eir
d
esig
n
h
as
p
r
o
m
p
ted
m
o
r
e
s
o
p
h
is
ti
ca
ted
m
o
d
els
s
u
ch
as
L
STM
n
etwo
r
k
s
,
wh
ich
o
v
er
c
o
m
e
s
o
m
e
o
f
th
ese
s
h
o
r
tco
m
in
g
s
b
y
in
co
r
p
o
r
atin
g
tech
n
iq
u
es to
m
ai
n
tain
g
r
a
d
ie
n
ts
th
r
o
u
g
h
o
u
t tim
e
[
8
]
.
2
.
3
.
L
o
ng
s
ho
rt
-
t
er
m
m
em
o
r
y
n
et
wo
rk
s
(
L
ST
M
)
T
o
o
v
er
co
m
e
th
e
is
s
u
es
with
r
eg
u
lar
R
NNs,
s
u
ch
as
th
e
v
a
n
is
h
in
g
g
r
ad
ien
t
p
r
o
b
lem
th
at
p
r
e
v
en
ts
R
NNs
f
r
o
m
lear
n
in
g
lo
n
g
-
ter
m
d
ep
en
d
en
cies
in
s
eq
u
en
tial
d
ata,
L
STM
n
etwo
r
k
s
wer
e
cr
ea
ted
[
8
]
.
L
STM
s
g
et
ar
o
u
n
d
th
is
p
r
o
b
lem
b
y
in
co
r
p
o
r
atin
g
m
em
o
r
y
ce
lls
an
d
g
atin
g
m
ec
h
a
n
is
m
s
th
at
let
th
e
n
etwo
r
k
k
ee
p
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
25
:
4
1
6
0
-
4171
4162
u
p
d
ate
d
ata
f
o
r
lo
n
g
p
er
i
o
d
s
o
f
tim
e,
th
er
e
b
y
ca
p
tu
r
in
g
lo
n
g
-
ter
m
d
ep
en
d
en
cies
[
1
4
]
.
Natu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
,
tim
e
s
er
ies
f
o
r
ec
a
s
tin
g
,
an
d
th
e
an
aly
s
is
o
f
s
eq
u
en
tial
d
ata
(
s
u
ch
as
s
ec
u
r
ity
p
atch
es)
ar
e
s
o
m
e
ex
am
p
les
o
f
jo
b
s
th
at
L
STM
s
ex
ce
l
at
b
ec
a
u
s
e
o
f
th
ese
ca
p
ab
ilit
ies.
W
ith
its
ab
ilit
y
to
s
i
m
u
late
th
e
tem
p
o
r
al
co
n
n
ec
tio
n
s
b
etwe
en
s
o
f
twar
e
u
p
d
ates
an
d
im
p
r
o
v
e
th
e
s
p
ee
d
an
d
ac
cu
r
ac
y
o
f
v
u
ln
er
ab
ilit
y
id
en
tific
atio
n
th
r
o
u
g
h
th
e
s
tu
d
y
o
f
s
y
s
tem
lo
g
s
,
L
STM
s
h
av
e
d
em
o
n
s
tr
at
ed
g
r
ea
t
p
r
o
m
is
e
in
th
e
d
o
m
a
in
o
f
cy
b
er
s
ec
u
r
ity
.
T
h
ey
also
im
p
r
o
v
e
th
e
p
r
ed
ict
io
n
o
f
th
e
ef
f
ec
tiv
en
ess
o
f
s
ec
u
r
ity
p
atch
es.
B
ec
au
s
e
o
f
th
es
e
f
ea
tu
r
es,
L
STM
s
a
r
e
an
ef
f
ec
tiv
e
to
o
l
f
o
r
p
r
o
te
ctin
g
s
o
f
twar
e
s
y
s
tem
s
f
r
o
m
n
ew
th
r
ea
ts
,
esp
ec
ially
in
s
itu
at
io
n
s
wer
e
ap
p
ly
in
g
s
ec
u
r
ity
f
ix
es a
cc
u
r
ately
a
n
d
i
n
a
tim
ely
m
an
n
er
is
cr
itical.
2
.
4
.
G
a
t
ed
re
curr
ent
un
it
s
(
G
RU)
T
o
s
im
p
lify
L
STM
n
etwo
r
k
ar
ch
itectu
r
e
wh
ile
k
ee
p
in
g
th
eir
ef
f
icien
cy
in
ca
p
tu
r
in
g
tem
p
o
r
al
d
ep
en
d
e
n
cies,
a
r
elativ
ely
r
ec
en
t
d
ev
elo
p
m
en
t
in
r
ec
u
r
r
en
t n
eu
r
al
n
etwo
r
k
d
esig
n
s
ar
e
G
R
Us
[
1
5
]
.
I
n
o
r
d
er
to
r
ed
u
ce
co
m
p
u
tatio
n
al
c
o
m
p
le
x
ity
with
o
u
t
s
ac
r
if
icin
g
p
e
r
f
o
r
m
an
ce
,
GR
Us
m
er
g
e
th
e
in
p
u
t
an
d
f
o
r
g
et
g
ates
in
to
o
n
e
s
in
g
le
g
ate.
Fo
r
s
itu
atio
n
s
with
co
n
s
tr
ain
e
d
co
m
p
u
tin
g
r
eso
u
r
ce
s
,
th
is
m
a
k
es
GR
U
s
s
u
p
er
io
r
to
L
STM
s
[
1
6
]
.
B
ec
au
s
e
o
f
th
ei
r
ad
v
an
tag
eo
u
s
tr
ad
e
-
o
f
f
b
etwe
en
co
m
p
u
tatio
n
al
co
m
p
lex
it
y
an
d
p
er
f
o
r
m
an
ce
,
GR
Us h
av
e
f
o
u
n
d
u
s
ef
u
l u
s
e
i
n
r
ea
l
-
tim
e
v
u
ln
e
r
ab
ilit
y
d
etec
tio
n
s
y
s
tem
s
wi
th
in
th
e
cy
b
er
s
ec
u
r
ity
d
o
m
ai
n
[
6
]
.
B
ec
au
s
e
o
f
th
ese
f
ea
tu
r
es,
G
R
Us
ar
e
id
ea
l
f
o
r
r
ea
l
-
tim
e
cy
b
er
s
ec
u
r
ity
ap
p
licatio
n
s
b
e
ca
u
s
e
o
f
th
e
s
o
lid
p
er
f
o
r
m
an
ce
th
e
y
p
r
o
v
id
e
w
h
ile
p
r
o
ce
s
s
in
g
s
eq
u
en
tial in
p
u
t
q
u
ick
ly
a
n
d
ac
cu
r
atel
y.
2
.
5
.
B
idi
re
ct
io
na
l L
ST
M
(
B
i
-
L
S
T
M
)
B
y
co
m
b
in
in
g
th
e
b
est
f
ea
t
u
r
es
o
f
f
o
r
wa
r
d
an
d
b
ac
k
war
d
p
r
o
ce
s
s
in
g
,
b
id
ir
ec
tio
n
al
L
ST
M
n
etwo
r
k
s
im
p
r
o
v
e
u
p
o
n
tr
ad
itio
n
al
L
STM
s
an
d
en
ab
le
m
o
d
els
to
d
etec
t
d
ep
en
d
en
cies
th
at
wo
u
ld
o
th
er
wis
e
g
o
u
n
n
o
ticed
.
W
ith
th
is
two
-
wa
y
ap
p
r
o
ac
h
,
B
i
-
L
STM
s
ca
n
th
i
n
k
a
b
o
u
t
th
e
p
ast
an
d
th
e
f
u
t
u
r
e
at
th
e
s
am
e
tim
e,
wh
ich
h
elp
s
th
em
u
n
d
er
s
tan
d
th
e
d
ata'
s
tem
p
o
r
al
lin
k
ag
es
b
etter
.
Usi
n
g
its
ca
p
ac
ity
to
an
aly
ze
s
eq
u
en
ce
s
m
o
r
e
d
ee
p
ly
,
B
i
-
L
STM
s
h
av
e
b
ee
n
s
u
cc
ess
f
u
lly
u
s
ed
to
im
p
r
o
v
e
th
e
d
etec
tio
n
o
f
v
u
ln
er
ab
ilit
ies
in
s
ec
u
r
ity
p
atch
es.
T
h
is
h
as
led
to
d
ee
p
er
in
s
ig
h
t
in
to
p
o
ten
tial
s
ec
u
r
ity
th
r
ea
ts
an
d
b
etter
p
r
ed
i
ctio
n
ac
cu
r
ac
y
in
cy
b
er
s
ec
u
r
ity
a
p
p
licatio
n
s
[
7
]
.
2
.
6
.
Su
m
m
a
ry
a
nd
a
pp
lica
t
io
n o
f
deep
lea
rning
in v
uln
er
a
b
ili
t
y
det
ec
t
io
n
A
n
u
m
b
er
o
f
r
ec
e
n
t
s
tu
d
ies
h
av
e
co
n
ce
n
tr
ated
o
n
th
e
u
s
e
o
f
R
NN,
L
STM
,
GR
U,
an
d
B
i
-
L
STM
d
ee
p
lear
n
in
g
m
o
d
els
in
cy
b
er
s
ec
u
r
ity
,
n
am
ely
in
th
e
ar
ea
s
o
f
v
u
ln
er
ab
ilit
y
tr
ac
k
in
g
an
d
p
a
tch
ad
m
in
is
tr
atio
n
.
Secu
r
ity
p
atch
d
ata
p
r
esen
ts
u
n
iq
u
e
is
s
u
es,
b
u
t
ea
c
h
m
o
d
e
l
h
as
its
o
wn
b
en
ef
its
wh
e
n
it
co
m
es
to
h
an
d
lin
g
s
eq
u
en
tial
d
ata.
R
esear
ch
er
s
h
o
p
e
t
o
s
tr
en
g
t
h
en
s
o
f
twar
e
s
y
s
tem
s
'
s
ec
u
r
ity
b
y
m
a
k
in
g
b
etter
u
s
e
o
f
th
ese
m
o
d
els to
d
is
co
v
er
v
u
ln
e
r
ab
ili
ties
m
o
r
e
q
u
ick
ly
an
d
ac
c
u
r
ate
ly
.
3.
M
E
T
H
O
D
3
.
1
.
O
v
er
v
iew
o
f
m
et
ho
do
lo
g
y
I
n
o
r
d
er
to
class
if
y
s
ec
u
r
ity
p
atch
es
e
f
f
ec
tiv
ely
,
th
is
s
tu
d
y
'
s
tech
n
iq
u
e
tak
es
in
to
ac
co
u
n
t
t
h
e
Patch
DB
d
ataset
's
r
ec
u
r
r
en
t
an
d
s
eq
u
en
tial
f
ea
tu
r
es.
T
h
is
s
ec
tio
n
g
iv
es
a
d
etailed
ex
p
lan
atio
n
o
f
th
e
d
ataset,
in
clu
d
in
g
its
s
tatis
tics
an
d
it
s
u
n
iq
u
e
f
ea
tu
r
es
in
clu
d
in
g
th
e
d
ep
lo
y
m
en
t
o
f
p
atch
es
s
eq
u
en
tially
th
r
o
u
g
h
o
u
t
tim
e.
I
n
o
r
d
er
to
d
ea
l
with
th
e
tim
e
-
s
en
s
itiv
e
d
ata,
we
o
f
f
e
r
co
m
p
r
eh
en
s
iv
e
p
r
ep
a
r
atio
n
p
r
o
ce
d
u
r
es,
s
u
ch
as
to
k
en
izatio
n
an
d
n
o
r
m
aliza
tio
n
.
B
y
v
is
u
alizin
g
a
n
d
u
n
c
o
v
e
r
in
g
d
is
tr
ib
u
tio
n
s
a
n
d
p
atter
n
s
with
in
th
e
d
ataset
ac
r
o
s
s
tim
e,
ex
p
lo
r
at
o
r
y
d
ata
an
aly
s
is
(
E
DA)
p
r
o
v
i
d
es
u
s
ef
u
l
in
s
ig
h
ts
f
o
r
d
ev
elo
p
in
g
m
o
d
els.
Ad
d
itio
n
ally
,
th
e
tech
n
iq
u
e
p
r
o
v
i
d
es
an
ex
p
lan
atio
n
f
o
r
w
h
y
s
p
ec
if
ic
d
ee
p
lear
n
in
g
m
o
d
els
wer
e
ch
o
s
e
n
,
in
clu
d
in
g
R
NN,
L
STM
,
GR
U,
an
d
B
i
-
L
STM
,
all
o
f
wh
ic
h
e
x
ce
l
at
ca
p
tu
r
in
g
d
e
p
en
d
e
n
cies
in
s
eq
u
e
n
tial
d
atasets
.
W
e
also
g
o
o
v
er
th
e
ass
ess
m
en
t
m
ea
s
u
r
es
u
s
ed
,
in
clu
d
in
g
ac
cu
r
ac
y
,
p
r
e
cisi
o
n
,
r
ec
all,
an
d
AUC
-
R
O
C
,
with
an
em
p
h
asis
o
n
h
o
w
th
ey
p
e
r
tain
to
s
eq
u
e
n
tial
d
ata
p
r
o
ce
s
s
in
g
,
an
d
h
o
w
to
o
p
tim
ize
t
h
e
m
o
d
el'
s
p
e
r
f
o
r
m
a
n
ce
th
r
o
u
g
h
h
y
p
er
p
ar
am
eter
tu
n
in
g
an
d
ad
v
an
ce
d
tr
ain
in
g
p
r
o
ce
d
u
r
es.
T
h
e
d
if
f
icu
lties
o
f
s
ec
u
r
ity
p
atc
h
ca
teg
o
r
izatio
n
ca
n
b
e
p
r
o
p
er
ly
a
d
d
r
ess
ed
with
th
e
h
elp
o
f
th
is
o
r
g
a
n
ized
m
eth
o
d
.
3
.
2
.
Resea
rc
h
s
t
a
g
es
As
p
r
esen
ted
in
Fig
u
r
e
1
,
th
e
p
r
o
p
o
s
ed
s
tu
d
y
em
p
lo
y
s
a
s
t
r
u
ctu
r
ed
,
m
u
lti
-
p
h
ased
m
eth
o
d
o
lo
g
y
to
class
if
y
s
ec
u
r
ity
p
atch
es
u
s
i
n
g
d
ee
p
lear
n
in
g
m
o
d
els.
T
h
e
p
r
o
ce
s
s
b
eg
in
s
with
d
ata
ac
q
u
is
itio
n
an
d
p
r
elim
in
ar
y
a
n
aly
s
is
,
wh
ich
aim
to
ex
p
l
o
r
e
th
e
d
ataset’
s
ch
ar
ac
ter
is
tics
th
o
r
o
u
g
h
ly
.
Nex
t,
ess
en
ti
al
p
r
ep
r
o
ce
s
s
in
g
s
tep
s
ar
e
p
er
f
o
r
m
ed
:
co
m
m
it
m
ess
ag
es
ar
e
co
m
b
in
ed
with
th
e
ass
o
ciate
d
co
d
e
d
if
f
er
e
n
ce
s
,
an
d
tex
t
d
ata
ar
e
tr
an
s
f
o
r
m
ed
u
s
in
g
t
er
m
f
r
eq
u
e
n
cy
-
i
n
v
er
s
e
d
o
c
u
m
en
t
f
r
eq
u
e
n
cy
(
T
F
-
I
DF)
to
k
en
izatio
n
.
O
n
ce
th
e
tex
tu
al
d
ata
h
as
b
ee
n
p
r
ep
ar
e
d
,
th
e
d
atas
et
is
d
iv
id
e
d
in
t
o
tr
ain
in
g
a
n
d
test
in
g
s
u
b
s
ets
to
en
s
u
r
e
a
r
eliab
le
ass
es
s
m
en
t o
f
m
o
d
el
p
er
f
o
r
m
a
n
ce
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
E
xp
lo
r
in
g
th
e
r
ec
u
r
r
en
t a
n
d
s
eq
u
en
tia
l secu
r
ity
p
a
tch
d
a
ta
u
s
in
g
…
(
F
a
la
h
Mu
h
a
mma
d
A
la
m
)
4163
Fo
llo
win
g
d
ata
p
r
ep
ar
atio
n
,
v
ar
io
u
s
d
ee
p
lear
n
in
g
a
r
ch
ite
ctu
r
es
n
am
ely
R
NN,
L
STM
,
GR
U,
an
d
Bi
-
L
STM
ar
e
d
ev
elo
p
ed
an
d
f
in
e
-
tu
n
e
d
th
r
o
u
g
h
h
y
p
e
r
p
ar
a
m
eter
o
p
t
im
izatio
n
to
ac
h
iev
e
s
u
p
er
io
r
ac
cu
r
ac
y
.
Af
ter
id
en
tify
in
g
th
e
o
p
tim
al
c
o
n
f
ig
u
r
atio
n
s
,
ea
ch
m
o
d
el
is
tr
ain
ed
an
d
ev
alu
ated
u
s
in
g
a
p
p
r
o
p
r
iate
m
etr
ics
t
o
g
au
g
e
its
p
r
ed
ictiv
e
ca
p
ab
ilit
ies.
T
h
e
ev
alu
atio
n
p
h
ase
h
ig
h
lig
h
ts
th
e
s
tr
en
g
th
s
an
d
wea
k
n
ess
es
o
f
ev
er
y
m
o
d
el,
s
h
ed
d
in
g
lig
h
t
o
n
th
eir
r
esp
ec
tiv
e
ef
f
ec
tiv
en
ess
in
h
an
d
lin
g
s
eq
u
en
tial
an
d
r
ep
etitiv
e
p
atch
d
ata.
T
h
is
co
m
p
r
eh
e
n
s
iv
e
ap
p
r
o
ac
h
u
ltima
tely
en
h
an
ce
s
s
o
f
twar
e
s
ec
u
r
ity
m
an
a
g
em
en
t
b
y
en
a
b
lin
g
s
wif
t
an
d
p
r
ec
is
e
d
etec
tio
n
o
f
s
ec
u
r
ity
p
atch
es.
Fig
u
r
e
1
.
T
h
e
p
r
o
p
o
s
ed
m
u
lti
-
s
tag
e
r
esear
ch
f
r
am
ewo
r
k
f
o
r
s
ec
u
r
ity
p
atch
class
if
icatio
n
3
.
3
.
Da
t
a
c
o
llect
io
n
I
n
th
is
p
ar
t,
th
e
d
ataset
th
at
was
u
tili
ze
d
f
o
r
tr
ain
in
g
an
d
ass
ess
in
g
d
ee
p
lear
n
in
g
m
o
d
els
f
o
r
s
eq
u
en
tial
s
ec
u
r
ity
p
atch
ca
t
eg
o
r
izatio
n
is
d
escr
ib
ed
,
with
an
em
p
h
asis
o
n
its
im
p
o
r
t
an
ce
.
T
h
e
Su
n
L
a
b
-
cr
ea
ted
"Patch
DB
:
A
lar
g
e
-
s
ca
le
s
ec
u
r
ity
p
atch
d
ataset"
[
1
7
]
,
co
n
tain
s
b
o
th
s
ec
u
r
ity
an
d
n
o
n
-
s
ec
u
r
ity
p
atch
es
cu
lled
f
r
o
m
well
-
k
n
o
wn
GitH
u
b
p
r
o
jects
an
d
th
e
Natio
n
al
Vu
ln
er
ab
ilit
y
Data
b
ase
(
NVD)
.
A
p
er
f
ec
t
f
i
t
f
o
r
s
eq
u
en
tial
an
d
tem
p
o
r
al
d
ata
an
aly
s
is
,
it
co
llects
co
m
m
it
an
d
d
if
f
attr
ib
u
tes
to
g
eth
er
with
m
u
lti
-
lab
els
(
"Sec
u
r
ity
"
an
d
"No
n
-
s
ec
u
r
it
y
")
.
T
o
s
tar
t
g
at
h
er
in
g
d
ata,
w
e
cr
awle
d
th
e
NVD
an
d
co
m
m
o
n
v
u
ln
er
a
b
ilit
ies
an
d
ex
p
o
s
u
r
es
(
C
VE
)
d
atab
a
s
es
f
o
r
d
o
cu
m
en
te
d
s
ec
u
r
ity
p
atch
es,
m
ak
in
g
s
u
r
e
to
i
n
clu
d
e
o
n
ly
v
alid
ated
p
atch
es.
B
y
ad
d
in
g
r
ea
l
-
wo
r
ld
p
atch
es
to
GitHu
b
co
m
m
its
an
d
u
s
in
g
o
v
er
s
am
p
lin
g
tec
h
n
i
q
u
es
to
b
alan
ce
th
e
q
u
an
tity
o
f
s
ec
u
r
ity
an
d
n
o
n
-
s
ec
u
r
ity
ch
a
n
g
es,
we
wer
e
ab
le
to
ad
d
r
ess
class
im
b
a
lan
ce
an
d
p
r
o
m
o
te
d
iv
er
s
ity
.
A
b
alan
ce
d
,
d
iv
er
s
i
f
ied
,
an
d
h
ig
h
-
q
u
ality
d
ataset
th
at
is
wel
l
-
s
u
ited
f
o
r
d
ee
p
lear
n
in
g
ap
p
licatio
n
s
was
ac
h
iev
ed
u
s
in
g
th
is
all
-
en
co
m
p
ass
in
g
m
eth
o
d
o
lo
g
y
.
I
n
o
r
d
er
to
ac
h
iev
e
r
eliab
le
a
n
d
p
r
ec
is
e
s
ec
u
r
ity
p
atch
ca
teg
o
r
izatio
n
,
t
h
e
s
tu
d
y
m
ak
es
u
s
e
o
f
Patch
DB
an
d
t
h
e
ca
p
ab
ilit
ies
o
f
d
ee
p
lea
r
n
in
g
m
o
d
els
in
clu
d
in
g
R
NN,
L
STM
,
G
R
U,
an
d
B
i
-
L
STM
.
T
h
ese
m
o
d
els
ex
ce
l
at
d
etec
tin
g
p
atter
n
s
an
d
d
e
p
en
d
en
cies
in
s
eq
u
en
tial
d
ata
o
v
er
t
h
e
lo
n
g
ter
m
.
3
.
4
.
P
re
pro
ce
s
s
ing
d
a
t
a
Data
q
u
ality
,
co
n
s
is
ten
cy
,
a
n
d
s
u
itab
ilit
y
f
o
r
s
eq
u
en
tial
d
ee
p
lear
n
in
g
m
o
d
els
wer
e
g
u
ar
an
teed
th
r
o
u
g
h
a
th
o
r
o
u
g
h
s
eq
u
e
n
ce
o
f
p
r
ep
a
r
atio
n
p
r
o
ce
s
s
es
th
at
wer
e
p
er
f
o
r
m
e
d
o
n
th
e
d
atase
t
to
g
et
it
r
ea
d
y
f
o
r
m
o
d
ellin
g
.
T
h
is
p
r
o
ce
s
s
b
eg
a
n
with
ex
p
lo
r
ato
r
y
d
ata
an
al
y
s
is
(
E
DA)
to
id
en
tify
d
is
tr
i
b
u
tio
n
s
,
tr
en
d
s
,
an
d
p
o
ten
tial
an
o
m
alies
in
th
e
p
atch
d
ataset.
T
h
e
E
DA
in
clu
d
ed
class
im
b
alan
ce
in
s
p
ec
tio
n
,
to
k
en
f
r
e
q
u
en
c
y
d
is
tr
ib
u
tio
n
,
a
n
d
s
eq
u
en
ce
le
n
g
th
an
aly
s
is
to
e
n
s
u
r
e
c
o
m
p
ati
b
ilit
y
with
s
eq
u
en
tial
m
o
d
els.
Su
b
s
eq
u
en
tly
,
d
ata
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
25
:
4
1
6
0
-
4171
4164
clea
n
in
g
a
n
d
p
r
e
p
ar
atio
n
s
tep
s
s
u
ch
as
h
an
d
lin
g
m
is
s
in
g
v
al
u
es
an
d
f
o
r
m
attin
g
d
ata
s
tr
u
ctu
r
es
wer
e
ap
p
lied
t
o
en
s
u
r
e
th
e
d
ataset
m
et
th
e
i
n
p
u
t r
eq
u
ir
e
m
en
ts
f
o
r
tim
e
-
s
er
ies d
ee
p
lear
n
in
g
alg
o
r
ith
m
s
.
3
.
4
.
1
.
E
x
plo
ra
t
o
ry
da
t
a
a
na
l
y
s
is
(
E
DA)
T
h
e
E
DA
o
f
th
e
d
ataset
r
ev
e
als
a
s
u
b
s
tan
tial
class
im
b
ala
n
ce
b
etwe
en
s
ec
u
r
ity
an
d
n
o
n
-
s
ec
u
r
ity
p
atch
es,
wh
ich
was
ad
d
r
ess
ed
b
y
o
v
er
s
am
p
lin
g
d
u
r
in
g
d
ata
p
r
ep
r
o
ce
s
s
in
g
.
Sp
ec
if
ically
,
th
e
n
u
m
b
er
o
f
n
o
n
-
s
ec
u
r
ity
p
atch
es
s
ig
n
if
ican
tly
o
u
tn
u
m
b
er
ed
t
h
e
s
ec
u
r
ity
p
atch
es,
wh
ich
co
u
ld
lea
d
to
m
o
d
el
b
ias
if
lef
t
u
n
h
an
d
led
.
T
h
is
im
b
alan
ce
ca
n
b
e
s
ee
n
clea
r
ly
in
Fig
u
r
e
2
,
wh
er
e
th
e
b
ar
c
h
ar
t h
ig
h
lig
h
ts
th
e
d
if
f
e
r
in
g
c
o
u
n
ts
b
etwe
en
th
ese
two
p
atch
ty
p
es.
Fig
u
r
e
2
also
in
clu
d
es
a
p
ie
ch
ar
t
th
at
illu
s
tr
ates
h
o
w
8
8
.
6
%
o
f
th
e
p
atch
es
ar
e
co
n
s
id
er
ed
wild
,
wh
ile
1
1
.
4
%
co
m
e
f
r
o
m
t
h
e
c
o
m
m
o
n
v
u
ln
er
ab
ilit
ies
an
d
ex
p
o
s
u
r
es
d
atab
ase.
T
h
ese
v
is
u
al
r
ep
r
esen
tatio
n
s
n
o
t
o
n
ly
s
h
o
wca
s
e
th
e
d
ataset's
d
iv
er
s
ity
b
u
t
also
u
n
d
er
s
co
r
e
th
e
im
p
o
r
ta
n
ce
o
f
b
alan
cin
g
s
tr
a
teg
ies
f
o
r
im
p
r
o
v
ed
class
if
icatio
n
p
er
f
o
r
m
a
n
ce
.
B
y
r
ec
o
g
n
izin
g
an
d
co
r
r
ec
tin
g
t
h
e
d
o
m
in
an
ce
o
f
wild
p
atc
h
es
at
th
is
ea
r
ly
s
tag
e,
th
e
s
u
b
s
eq
u
e
n
t
s
tep
s
o
f
d
ata
p
r
ep
ar
atio
n
an
d
d
ee
p
lear
n
in
g
m
o
d
el
d
e
v
elo
p
m
e
n
t
wer
e
b
etter
p
o
s
itio
n
ed
to
ac
h
iev
e
r
o
b
u
s
t a
n
d
ac
c
u
r
ate
s
e
cu
r
ity
p
atch
class
if
icatio
n
.
Fig
u
r
e
2
.
Par
t o
f
th
e
E
DA,
wit
h
a
p
ie
ch
a
r
t sh
o
win
g
t
h
e
p
atc
h
ty
p
e
s
p
r
ea
d
an
d
a
b
ar
ch
ar
t c
o
m
p
ar
in
g
s
ec
u
r
ity
an
d
n
o
n
-
s
ec
u
r
ity
p
atch
es
3
.
4
.
2
.
Da
t
a
c
lea
nin
g
A
th
o
r
o
u
g
h
d
ata
clea
n
i
n
g
p
r
o
ce
s
s
was
ca
r
r
ied
o
u
t
t
o
p
r
eser
v
e
th
e
d
ataset'
s
co
n
s
is
ten
cy
,
r
eliab
ilit
y
,
an
d
in
te
g
r
ity
,
m
ak
in
g
it
a
p
p
r
o
p
r
iate
f
o
r
d
ee
p
lear
n
i
n
g
m
o
d
els.
T
o
elim
in
ate
u
n
n
ec
ess
ar
y
r
e
p
etitio
n
,
we
elim
in
ated
d
u
p
licate
en
tr
ies
a
n
d
c
o
m
b
in
e
d
tex
t
f
ield
s
lik
e
c
o
m
m
it
m
ess
ag
es
an
d
co
d
e
d
if
f
s
in
to
o
n
e
s
tan
d
ar
d
s
ty
le
[
4
]
.
u
p
o
r
d
er
to
f
ill
u
p
a
n
y
g
a
p
s
o
r
m
is
s
in
g
d
ata,
we
ch
ec
k
ed
with
o
th
er
r
eso
u
r
ce
s
,
s
u
ch
as
th
e
NVD
an
d
GitHu
b
[
1
7
]
.
A
h
ig
h
-
q
u
ality
d
ataset
th
at
r
ed
u
ce
d
b
iases
an
d
en
h
an
ce
d
th
e
m
o
d
els'
r
eliab
ilit
y
d
u
r
in
g
tr
ain
in
g
an
d
ev
al
u
atio
n
was
p
r
o
d
u
c
ed
b
y
m
eticu
lo
u
s
ly
r
ev
iew
in
g
ea
ch
item
to
g
u
ar
a
n
te
e
u
n
iq
u
en
ess
an
d
co
m
p
leten
ess
.
3
.
4
.
3
.
L
a
bel
e
nco
din
g
I
t
was
cr
itical
to
tr
an
s
f
o
r
m
t
h
e
ca
teg
o
r
ical
la
b
els
in
to
a
n
u
m
er
ical
f
o
r
m
at
th
at
th
e
alg
o
r
ith
m
s
co
u
ld
in
ter
p
r
et
s
u
cc
ess
f
u
lly
in
o
r
d
er
to
p
r
ep
ar
e
th
e
d
ataset
f
o
r
d
ee
p
lear
n
in
g
m
o
d
els.
"Sec
u
r
ity
"
a
n
d
"No
n
-
s
ec
u
r
ity
"
ar
e
ex
am
p
les
o
f
te
x
tu
al
lab
els
th
at
wer
e
co
n
v
e
r
ted
in
to
n
u
m
e
r
ical
v
alu
es
(
e.
g
.
,
1
an
d
0
)
b
y
l
ab
el
en
co
d
in
g
[
4
]
.
B
ec
au
s
e
o
f
th
is
ch
an
g
e,
th
e
m
o
d
els
wer
e
ab
le
to
tr
ain
a
p
p
r
o
p
r
iately
o
n
th
e
ca
teg
o
r
ies,
wh
i
ch
al
lo
wed
th
em
t
o
d
is
co
v
er
u
s
ef
u
l
lin
k
s
an
d
p
atte
r
n
s
in
th
e
d
ata.
Ach
iev
in
g
r
eli
ab
le
p
r
ed
ictio
n
s
an
d
a
s
m
o
o
th
d
ataset
in
teg
r
atio
n
with
d
ee
p
lear
n
in
g
f
r
a
m
ewo
r
k
s
r
elied
h
ea
v
ily
o
n
th
e
lab
el
e
n
co
d
in
g
p
r
o
ce
s
s
.
3
.
4
.
4
.
T
o
k
eniza
t
io
n a
nd
v
ec
t
o
riz
a
t
io
n
T
h
e
tr
an
s
f
o
r
m
atio
n
o
f
r
aw
tex
t
in
to
n
u
m
e
r
ical
r
e
p
r
esen
tatio
n
s
was
cr
u
cial
f
o
r
th
e
d
ee
p
lear
n
in
g
m
o
d
els
to
ef
f
icien
tly
p
r
o
ce
s
s
tex
tu
al
d
ata.
T
er
m
f
r
eq
u
en
cy
-
in
v
er
s
e
d
o
c
u
m
en
t
f
r
e
q
u
en
cy
(TF
-
I
DF)
v
ec
to
r
izatio
n
an
d
to
k
e
n
izatio
n
allo
wed
u
s
to
ac
h
iev
e
th
is
.
Fi
r
s
t,
th
e
c
o
m
b
in
e
d
tex
t
f
i
eld
s
,
i
n
clu
d
in
g
co
d
e
d
if
f
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
E
xp
lo
r
in
g
th
e
r
ec
u
r
r
en
t a
n
d
s
eq
u
en
tia
l secu
r
ity
p
a
tch
d
a
ta
u
s
in
g
…
(
F
a
la
h
Mu
h
a
mma
d
A
la
m
)
4165
an
d
co
m
m
it m
ess
ag
es,
wer
e
to
k
en
ized
to
m
a
k
e
th
em
ea
s
ier
t
o
wo
r
k
with
[
4
]
.
T
o
m
ak
e
th
e
m
u
s
ab
le
as in
p
u
t to
th
e
m
o
d
el,
th
ese
to
k
en
s
wer
e
co
n
v
er
ted
in
t
o
n
u
m
e
r
ical
v
ec
to
r
s
with
a
p
r
ed
eter
m
in
ed
a
m
o
u
n
t
o
f
f
ea
tu
r
es.
T
o
k
en
izatio
n
a
n
d
v
ec
to
r
izatio
n
p
r
eser
v
ed
th
e
r
ele
v
an
ce
o
f
ter
m
s
b
ased
o
n
th
eir
f
r
eq
u
en
cy
an
d
u
n
i
q
u
en
ess
with
in
th
e
d
ataset,
im
p
r
o
v
in
g
th
e
tex
t
d
ata
f
o
r
m
at
f
o
r
d
ee
p
l
ea
r
n
in
g
ap
p
licatio
n
s
.
T
h
is
allo
wed
th
e
m
o
d
els
to
tr
ain
an
d
p
r
ed
ict
well
[
1
8
]
.
3
.
4
.
5
.
Da
t
a
Sp
litt
ing
Sep
ar
atin
g
th
e
d
ataset
in
to
s
u
b
s
ets
f
o
r
tr
ain
in
g
,
v
alid
atio
n
,
an
d
test
in
g
was
cr
u
cial
to
p
r
o
v
id
e
a
s
tr
o
n
g
an
d
im
p
ar
tial
ass
ess
m
e
n
t
o
f
th
e
m
o
d
els.
Stra
tifie
d
s
am
p
lin
g
was
u
s
ed
to
en
s
u
r
e
th
at
th
e
"Sec
u
r
ity
"
an
d
"No
n
-
s
ec
u
r
ity
"
ca
teg
o
r
ies
wer
e
e
v
en
ly
d
is
tr
ib
u
ted
t
h
r
o
u
g
h
o
u
t
all
s
u
b
s
ets
o
f
th
e
d
ataset,
wh
ich
was
th
e
n
d
iv
id
ed
in
to
th
r
ee
p
ar
ts
: 7
0
% f
o
r
tr
ain
in
g
,
1
5
% f
o
r
v
alid
atio
n
,
an
d
1
5
% f
o
r
test
in
g
[
3
]
.
T
h
is
m
eth
o
d
p
r
e
v
en
ted
o
v
er
f
itti
n
g
d
u
r
in
g
tr
ain
in
g
wh
ile
s
till
p
r
o
v
id
in
g
th
e
m
o
d
els
with
en
o
u
g
h
d
ata
f
o
r
lear
n
in
g
.
A
s
ep
ar
ate
m
etr
ic
f
o
r
th
e
m
o
d
el'
s
ef
f
icac
y
wa
s
s
u
p
p
lied
b
y
th
e
test
s
et,
wh
ich
allo
wed
f
o
r
h
y
p
er
p
ar
am
eter
tu
n
in
g
an
d
p
er
f
o
r
m
an
ce
m
o
n
ito
r
in
g
in
t
h
e
v
alid
atio
n
s
et.
T
h
e
e
v
alu
atio
n
p
r
o
ce
s
s
co
n
s
is
ten
tly
ev
alu
ated
th
e
m
o
d
els'
ca
p
ac
ity
to
g
e
n
er
alize
to
u
n
k
n
o
wn
d
ata
b
y
u
tili
zin
g
th
is
s
tr
u
ctu
r
ed
s
p
litt
in
g
s
tr
ateg
y
[
1
2
]
.
3
.
4
.
6
.
Def
ine
m
o
del (
RNN,
L
ST
M
,
G
RU,
B
I
-
L
S
T
M
)
W
e
d
ev
elo
p
ed
an
d
d
ep
l
o
y
ed
f
o
u
r
d
ee
p
lear
n
in
g
ar
ch
itectu
r
e
s
R
NN,
L
STM
,
G
R
U,
an
d
B
i
-
L
STM
to
ef
f
icien
tly
h
an
d
le
an
d
ca
teg
o
r
ize
s
eq
u
en
tial
s
ec
u
r
ity
p
atch
d
ata.
I
n
o
r
d
e
r
to
ca
p
tu
r
e
t
h
e
s
em
an
tic
lin
k
s
b
etwe
en
wo
r
d
s
,
ea
ch
m
o
d
el
d
esig
n
s
tar
ted
with
a
n
em
b
e
d
d
in
g
lay
er
t
h
at
tu
r
n
ed
in
p
u
t
to
k
en
s
in
t
o
d
e
n
s
e
v
ec
to
r
s
[
1
8
]
.
Ad
a
p
ted
to
th
e
s
p
ec
if
ics
o
f
ea
ch
m
o
d
el,
th
e
f
o
llo
win
g
r
ec
u
r
r
e
n
t
lay
er
s
we
r
e
b
u
ilt
u
p
o
n
th
ese
em
b
ed
d
in
g
s
.
T
o
h
a
n
d
le
lo
n
g
-
ter
m
d
e
p
en
d
e
n
cies,
th
e
L
ST
M
m
o
d
el
m
ad
e
u
s
e
o
f
its
g
atin
g
m
ec
h
an
is
m
s
,
wh
er
ea
s
th
e
R
NN
m
o
d
el
lear
n
t
s
eq
u
en
tial
d
ep
en
d
en
cie
s
u
s
in
g
a
Simp
leR
NN
lay
er
[
8
]
.
A
GR
U
lay
er
was
u
s
ed
b
y
th
e
c
o
m
p
u
tatio
n
ally
ef
f
icien
t
GR
U
m
o
d
el
[
1
5
]
,
wh
ile
a
B
id
ir
ec
tio
n
al
L
STM
lay
er
was
u
s
ed
b
y
th
e
b
id
ir
ec
tio
n
ally
e
f
f
icien
t
B
i
-
L
STM
m
o
d
el
[
7
]
to
p
r
o
ce
s
s
d
ata
in
b
o
th
d
ir
ec
tio
n
s
.
Pro
b
a
b
ilit
y
s
co
r
es
f
o
r
b
in
a
r
y
class
if
icatio
n
wer
e
g
en
er
ated
u
s
in
g
a
d
e
n
s
e
o
u
t
p
u
t
lay
e
r
u
s
in
g
a
s
ig
m
o
i
d
ac
tiv
atio
n
f
u
n
c
tio
n
.
Fo
r
ac
c
u
r
ate
p
r
ed
ictio
n
s
an
d
ef
f
icie
n
t
lear
n
in
g
,
all
m
o
d
els
wer
e
f
in
e
-
tu
n
ed
with
th
e
Ad
am
o
p
tim
izer
an
d
b
u
ilt
with
th
e
b
in
ar
y
cr
o
s
s
-
en
tr
o
p
y
lo
s
s
f
u
n
c
tio
n
.
T
h
e
m
o
d
els we
r
e
ab
le
to
ac
cu
r
ately
class
if
y
s
ec
u
r
ity
p
atch
d
ata
b
ec
au
s
e
o
f
th
e
ar
ch
itectu
r
al
d
esig
n
'
s
em
p
h
asi
s
o
n
s
eq
u
en
tial a
n
d
tem
p
o
r
al
p
atter
n
s
.
3
.
5
.
H
y
perpa
ra
m
et
er
t
uning
T
o
m
ax
im
ize
th
e
ef
f
icien
c
y
o
f
a
d
ee
p
lear
n
in
g
m
o
d
el,
h
y
p
er
p
ar
am
eter
tu
n
in
g
is
an
ess
en
ti
al
p
r
o
ce
s
s
f
o
r
d
eter
m
in
in
g
t
h
e
b
est
p
o
s
s
ib
le
p
ar
am
eter
s
ettin
g
s
.
Usi
n
g
an
Op
t
u
n
a
-
b
ased
ap
p
r
o
ac
h
a
s
tate
-
of
-
th
e
-
a
r
t
o
p
tim
izatio
n
f
r
am
ewo
r
k
,
we
m
eth
o
d
ically
i
n
v
esti
g
ated
s
ev
er
al
h
y
p
er
p
ar
a
m
eter
c
o
m
b
in
at
io
n
s
in
th
is
s
tu
d
y
.
Op
tu
n
a
f
in
d
s
th
e
b
est
s
ettin
g
s
b
y
i
n
tellig
en
tly
s
am
p
lin
g
s
etu
p
s
,
as
o
p
p
o
s
ed
to
g
r
i
d
s
ea
r
c
h
,
wh
ich
ev
alu
ates
ev
er
y
p
o
ten
tial
p
ar
am
eter
v
alu
e
ex
h
au
s
tiv
ely
[
1
2
]
.
T
h
e
s
tu
d
y
'
s
cr
itical
h
y
p
er
p
ar
am
eter
s
wer
e
th
e
f
o
llo
win
g
:
lear
n
in
g
r
ate,
b
atc
h
s
ize,
ep
o
c
h
s
,
n
u
m
b
er
o
f
lay
e
r
s
,
n
u
m
b
er
o
f
u
n
its
in
r
ec
u
r
r
en
t
la
y
er
s
,
an
d
n
u
m
b
er
o
f
la
y
er
s
o
v
er
all.
Mo
d
el
ac
cu
r
ac
y
,
r
e
ca
ll,
p
r
ec
is
io
n
,
F1
-
s
co
r
e,
a
n
d
AUC
-
R
OC
wer
e
all
m
ax
im
ized
u
s
in
g
t
h
is
p
r
o
ce
d
u
r
e.
R
esear
ch
h
as
s
h
o
wn
th
at
h
y
p
e
r
p
ar
am
eter
twea
k
in
g
h
as
a
m
ajo
r
ef
f
ec
t
o
n
th
e
ac
cu
r
ac
y
a
n
d
co
m
p
u
tatio
n
al
ef
f
icien
c
y
o
f
m
o
d
els,
am
o
n
g
o
th
er
th
in
g
s
[
1
9
]
,
[
2
0
]
.
T
o
d
eter
m
i
n
e
th
e
ef
f
icac
y
o
f
ea
ch
co
llectio
n
o
f
h
y
p
e
r
p
ar
am
eter
s
,
m
o
d
els
wer
e
test
ed
with
a
v
alid
atio
n
s
u
b
s
et.
C
o
n
s
is
ten
t
an
d
v
er
y
ac
cu
r
ate
m
o
d
el
class
if
icatio
n
o
f
s
eq
u
e
n
tial
s
ec
u
r
ity
p
atch
d
ata
was
ac
h
iev
ed
as
a
c
o
n
s
eq
u
e
n
ce
o
f
th
is
r
ig
o
r
o
u
s
tu
n
in
g
p
r
o
ce
d
u
r
e.
3
.
6
.
T
ra
ini
ng
m
o
dels
I
n
d
ee
p
lear
n
in
g
ap
p
licatio
n
s
with
s
eq
u
en
tial
d
ata,
p
r
o
p
e
r
m
o
d
el
tr
ain
in
g
is
cr
u
cial
f
o
r
ac
h
iev
in
g
o
p
tim
al
p
er
f
o
r
m
a
n
ce
wh
ile
m
in
im
izin
g
o
v
e
r
f
itti
n
g
.
Her
e
,
w
e
o
p
tim
ized
R
NN,
L
STM
,
G
R
U,
an
d
B
i
-
L
ST
M
m
o
d
els
u
s
in
g
s
tate
-
of
-
th
e
-
ar
t
tr
ain
in
g
m
eth
o
d
s
.
B
ec
au
s
e
o
f
its
ca
p
ac
ity
to
d
y
n
am
ically
a
d
ju
s
t
lear
n
in
g
r
ates
an
d
ex
p
ed
ite
co
n
v
er
g
e
n
ce
,
th
e
Ad
am
o
p
tim
izer
is
well
-
k
n
o
wn
f
o
r
its
ef
f
icien
c
y
in
d
ee
p
l
ea
r
n
in
g
task
s
,
an
d
it
was
u
s
ed
to
tr
ain
all
o
f
th
e
m
o
d
els
[
2
1
]
.
T
h
r
o
u
g
h
o
u
t
th
e
tr
ain
in
g
p
r
o
ce
s
s
,
th
e
m
o
d
el's
p
er
f
o
r
m
a
n
ce
was
co
n
s
tan
tly
tr
ac
k
ed
o
n
a
v
alid
atio
n
s
et
u
s
in
g
a
d
ef
in
ed
n
u
m
b
er
o
f
ep
o
c
h
s
.
E
ar
ly
s
to
p
p
i
n
g
was
u
s
ed
to
en
d
tr
ain
in
g
wh
e
n
v
alid
atio
n
p
er
f
o
r
m
an
ce
d
id
n
o
t
s
h
o
w
a
n
y
a
d
d
itio
n
al
im
p
r
o
v
em
en
t;
th
is
was
d
o
n
e
to
f
u
r
th
er
im
p
r
o
v
e
g
en
er
aliza
b
ilit
y
an
d
p
r
ev
en
t
o
v
er
f
itti
n
g
[
2
2
]
.
B
y
u
s
in
g
th
is
ap
p
r
o
ac
h
,
we
co
u
ld
b
e
ce
r
tain
th
at
o
u
r
m
o
d
els
wo
u
ld
r
eliab
ly
d
etec
t
i
m
p
o
r
tan
t
p
atter
n
s
in
o
u
r
tr
ain
i
n
g
d
ata
an
d
co
n
tin
u
e
to
p
er
f
o
r
m
ad
m
ir
a
b
ly
wh
e
n
p
r
esen
ted
with
n
ew,
u
n
k
n
o
wn
test
d
ata
[
2
3
]
.
Ou
r
m
o
d
els
f
o
r
ca
teg
o
r
izin
g
s
ec
u
r
ity
p
atch
d
ata
h
av
e
p
r
o
v
en
to
b
e
r
eliab
le
an
d
ef
f
ec
tiv
e
b
y
u
s
in
g
th
ese
tr
ain
in
g
p
r
o
ce
d
u
r
es.
3
.
7
.
E
v
a
lua
t
io
n
m
et
rics
T
o
p
r
o
p
e
r
ly
ev
alu
ate
d
ee
p
le
ar
n
in
g
m
o
d
els'
s
u
cc
ess
in
id
en
tify
in
g
s
ec
u
r
ity
f
ix
es,
it
is
cr
u
cial
to
ch
o
o
s
e
ap
p
r
o
p
r
iate
ass
ess
m
en
t
m
e
asu
r
es.
T
h
is
s
tu
d
y
th
o
r
o
u
g
h
ly
ev
alu
ate
d
th
e
ef
f
icac
y
o
f
th
e
m
o
d
el
b
y
u
s
in
g
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
25
:
4
1
6
0
-
4171
4166
a
co
m
b
in
atio
n
o
f
g
en
e
r
ally
r
ec
o
g
n
ized
ca
te
g
o
r
izatio
n
m
e
asu
r
es.
Pre
cisi
o
n
m
ea
s
u
r
es
th
e
d
ep
e
n
d
ab
ilit
y
o
f
p
o
s
itiv
e
p
r
ed
ictio
n
s
,
r
e
ca
ll
ev
alu
ates
th
e
ab
ilit
y
to
id
en
tify
tr
u
e
p
o
s
itiv
es,
F1
-
s
co
r
e
s
tr
ik
es
a
b
alan
ce
b
etwe
en
r
ec
all
an
d
p
r
ec
is
io
n
,
a
n
d
AU
C
-
R
O
C
m
ea
s
u
r
es
th
e
m
o
d
el's
ca
p
ac
ity
to
d
is
tin
g
u
is
h
b
et
wee
n
class
es
ac
r
o
s
s
d
if
f
er
en
t
th
r
esh
o
ld
s
[
2
4
]
.
Ac
cu
r
ac
y
is
u
s
ed
f
o
r
g
en
er
al
p
er
f
o
r
m
a
n
ce
ass
ess
m
en
t.
All
o
f
th
ese
m
ea
s
u
r
es
wo
r
k
ed
to
g
eth
e
r
to
p
r
o
v
id
e
a
t
h
o
r
o
u
g
h
ass
ess
m
en
t,
s
h
o
win
g
h
o
w
ef
f
ec
tiv
ely
ea
c
h
m
o
d
el
d
ea
lt
with
s
eq
u
en
tial
s
ec
u
r
ity
p
atc
h
d
ata
a
n
d
wh
e
r
e
it f
ell
s
h
o
r
t.
3
.
7
.
1
.
Co
nfusi
o
n
m
a
t
rix
C
las
s
if
icatio
n
m
o
d
el
p
er
f
o
r
m
an
ce
ca
n
b
e
b
etter
u
n
d
er
s
to
o
d
with
th
e
h
elp
o
f
to
o
ls
th
at
r
ev
ea
l
th
eir
p
r
ed
ictio
n
s
in
g
r
ea
t
d
etail.
Fo
r
ex
am
p
le,
th
e
co
n
f
u
s
io
n
m
atr
i
x
class
if
ies
f
o
r
ec
asts
a
s
eith
er
tr
u
e
p
o
s
itiv
e
(
T
P),
tr
u
e
n
eg
ativ
e
(
T
N)
,
f
alse
p
o
s
itiv
e
(
FP
)
,
o
r
f
alse
n
e
g
ativ
e
(
FN)
.
T
h
e
m
o
d
el'
s
p
r
ed
icte
d
ac
cu
r
ac
y
ca
n
b
e
th
o
r
o
u
g
h
ly
ev
al
u
ated
th
an
k
s
t
o
th
is
b
r
ea
k
d
o
wn
[
9
]
.
T
h
e
co
n
f
u
s
io
n
m
atr
ix
is
a
u
s
ef
u
l
to
o
l
f
o
r
id
en
tify
in
g
th
e
m
o
d
el'
s
s
tr
en
g
th
s
an
d
wea
k
n
e
s
s
es,
s
u
ch
a
s
it
s
p
o
s
itiv
e
an
d
n
eg
ativ
e
in
s
tan
ce
class
if
icatio
n
ac
cu
r
ac
y
a
n
d
th
e
ex
ten
t
to
wh
ich
it
m
is
class
if
ies
d
ata
[
1
0
]
.
An
o
v
er
v
iew
o
f
th
e
co
n
f
u
s
io
n
m
atr
ix
s
tr
u
ct
u
r
e
is
p
r
o
v
id
ed
in
T
ab
le
1
,
wh
ich
s
h
o
ws
h
o
w
th
e
m
o
d
el'
s
p
er
f
o
r
m
a
n
ce
is
ev
alu
ated
b
y
co
m
p
ar
in
g
p
r
e
d
ictio
n
s
with
ac
tu
al
o
u
tco
m
es.
T
ab
le
1
.
C
o
n
f
u
s
io
n
m
atr
ix
P
r
e
d
i
c
t
e
d
p
o
si
t
i
v
e
P
r
e
d
i
c
t
e
d
n
e
g
a
t
i
v
e
A
c
t
u
a
l
p
o
si
t
i
v
e
Tr
u
e
p
o
si
t
i
v
e
(
TP)
F
a
l
se
n
e
g
a
t
i
v
e
(
F
N
)
A
c
t
u
a
l
n
e
g
a
t
i
v
e
F
a
l
se
p
o
si
t
i
v
e
(
F
P
)
Tr
u
e
n
e
g
a
t
i
v
e
(
TN
)
3
.
7
.
2
.
Acc
ura
cy
A
p
o
p
u
lar
m
etr
ic,
ac
cu
r
ac
y
s
h
o
ws
h
o
w
m
an
y
in
s
tan
ce
s
o
u
t
o
f
th
e
to
tal
n
u
m
b
e
r
o
f
in
s
ta
n
ce
s
in
th
e
d
ataset
wer
e
p
r
o
p
er
ly
p
r
ed
icte
d
.
An
ea
s
y
-
to
-
u
n
d
er
s
tan
d
o
v
er
v
iew
o
f
th
e
m
o
d
el'
s
p
er
f
o
r
m
a
n
ce
is
p
r
o
v
id
ed
b
y
it
[
1
6
]
.
B
u
t
it
m
ig
h
t
b
e
d
ec
ei
v
in
g
in
d
atasets
wh
er
e
th
er
e
is
a
lar
g
e
d
is
p
ar
ity
ac
r
o
s
s
clas
s
es
[
2
5
]
.
Acc
u
r
ac
y
r
is
k
s
g
iv
in
g
a
n
ex
a
g
g
er
ate
d
p
ictu
r
e
o
f
th
e
m
o
d
el'
s
ef
f
icac
y
wh
en
o
n
e
class
is
s
u
b
s
tan
tially
m
o
r
e
n
u
m
er
o
u
s
th
an
th
e
o
th
er
[
2
6
]
.
E
q
u
atio
n
(
1
)
s
h
o
ws
th
e
f
o
r
m
u
la
f
o
r
c
alcu
latin
g
ac
cu
r
ac
y
,
wh
e
r
e
T
P
an
d
T
N
ar
e
th
e
n
u
m
b
er
o
f
c
o
r
r
ec
tly
ca
te
g
o
r
iz
ed
in
s
tan
ce
s
an
d
to
tal
in
s
tan
ce
s
ar
e
th
e
to
tal
d
ata
s
ize:
A
c
c
ura
c
y =
+
T
o
t
a
l
I
n
st
a
n
c
e
s
(
1
)
3
.
7
.
3
.
P
re
cisi
o
n
An
im
p
o
r
ta
n
t
m
ea
s
u
r
e
f
o
r
ass
ess
in
g
th
e
ac
cu
r
ac
y
o
f
a
m
o
d
e
l
is
its
p
r
ec
is
io
n
,
wh
ich
is
d
e
f
i
n
ed
as
th
e
p
er
ce
n
tag
e
o
f
co
r
r
ec
t
p
r
ed
icti
o
n
s
r
elativ
e
to
t
h
e
to
tal
n
u
m
b
er
o
f
co
r
r
ec
t
p
r
ed
ictio
n
s
.
I
n
c
o
n
tex
ts
wh
er
e
f
alse
p
o
s
itiv
es
h
av
e
s
u
b
s
tan
tial
im
p
licatio
n
s
,
lik
e
m
ed
ical
d
iag
n
o
s
is
o
r
f
r
au
d
d
etec
tio
n
,
a
h
ig
h
p
r
ec
is
io
n
r
ep
r
esen
ts
a
lo
w
f
alse p
o
s
itiv
e
r
ate,
wh
ic
h
is
esp
ec
ially
cr
u
cial
[
2
7
]
.
I
n
u
n
b
alan
ce
d
d
ata
s
ets,
wh
er
e
th
e
f
alse p
o
s
itiv
e
co
s
t
co
u
ld
e
x
ce
ed
t
h
e
f
alse
n
eg
ativ
e
co
s
t,
th
is
s
tatis
tic
b
ec
o
m
es
e
v
en
m
o
r
e
im
p
o
r
tan
t
[
1
0
]
.
Pre
cisen
ess
g
u
ar
an
tees
m
ea
n
in
g
f
u
l
an
d
d
e
p
en
d
a
b
le
m
o
d
el
p
r
e
d
ictio
n
s
,
esp
ec
ially
in
h
ig
h
-
s
tak
es
ap
p
licatio
n
s
,
b
y
f
o
cu
s
in
g
on
r
ed
u
cin
g
th
e
n
u
m
b
er
o
f
f
alse
p
o
s
itiv
es.
T
h
e
f
o
r
m
u
la
f
o
r
ca
l
c
u
latin
g
p
r
ec
is
io
n
is
g
iv
en
b
y
(
2
)
,
wh
ic
h
d
iv
id
es
th
e
to
tal
n
u
m
b
er
o
f
tr
u
e
p
o
s
itiv
es (
T
P)
b
y
th
e
s
u
m
o
f
all
tr
u
e
p
o
s
itiv
es a
n
d
f
alse p
o
s
itiv
es (
FP
)
:
=
+
(
2
)
3
.
7
.
4
.
Rec
a
ll
T
h
e
s
en
s
itiv
ity
o
r
r
ec
all
o
f
a
m
o
d
el
is
d
e
f
in
ed
as
th
e
p
er
ce
n
tag
e
o
f
co
r
r
ec
t
p
r
ed
ictio
n
s
r
e
lativ
e
to
th
e
to
tal
n
u
m
b
er
o
f
co
r
r
ec
t
p
r
ed
ic
tio
n
s
.
I
f
th
e
m
o
d
el
h
as
a
h
ig
h
r
ec
all,
it
m
ea
n
s
i
t
s
u
cc
es
s
f
u
lly
id
en
tifie
s
a
lar
g
e
p
er
ce
n
tag
e
o
f
t
r
u
e
p
o
s
itiv
es.
T
h
is
is
esp
ec
ially
im
p
o
r
ta
n
t
in
m
ed
ical
s
cr
ee
n
in
g
an
d
s
ec
u
r
ity
ap
p
licatio
n
s
wh
er
e
f
alse
p
o
s
itiv
es
ca
n
h
av
e
s
er
io
u
s
im
p
licatio
n
s
[
2
8
]
.
I
n
illn
ess
d
iag
n
o
s
is
,
f
o
r
in
s
tan
c
e,
a
h
ig
h
r
ec
all
r
ate
g
u
ar
an
tees
th
e
d
etec
tio
n
o
f
th
e
m
ajo
r
ity
o
f
ca
s
es,
n
o
twith
s
ta
n
d
i
n
g
th
e
p
o
s
s
ib
ilit
y
o
f
s
o
m
e
f
alse
p
o
s
itiv
es
[
1
1
]
.
R
ec
all,
wh
ich
s
h
o
ws
h
o
w
well
th
e
m
o
d
el
id
en
tifie
s
th
e
m
in
o
r
ity
class
wh
ich
is
f
r
eq
u
e
n
tly
m
o
r
e
im
p
o
r
tan
t
is
a
cr
u
cial
p
ar
am
eter
to
co
n
s
id
er
wh
en
wo
r
k
in
g
with
ex
tr
em
ely
im
b
alan
ce
d
d
atasets
[
2
6
]
.
T
o
f
in
d
th
e
r
ec
all,
we
d
iv
id
e
th
e
to
tal
n
u
m
b
er
o
f
tr
u
e
p
o
s
itiv
es
(
T
P)
b
y
th
e
to
tal
n
u
m
b
er
o
f
tr
u
e
n
eg
ativ
es
(
FN)
u
s
in
g
th
e
f
o
r
m
u
la
g
iv
en
in
(
3
)
:
=
+
(
3
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
E
xp
lo
r
in
g
th
e
r
ec
u
r
r
en
t a
n
d
s
eq
u
en
tia
l secu
r
ity
p
a
tch
d
a
ta
u
s
in
g
…
(
F
a
la
h
Mu
h
a
mma
d
A
la
m
)
4167
3
.
7
.
5
.
F1
-
s
co
re
An
im
p
o
r
ta
n
t
m
etr
ic
f
o
r
b
alan
cin
g
r
ec
all
a
n
d
p
r
ec
is
io
n
is
th
e
F1
-
s
co
r
e,
wh
ic
h
is
th
e
h
ar
m
o
n
ic
m
ea
n
o
f
th
e
two
.
I
t
o
f
f
e
r
s
a
s
in
g
le
m
etr
ic
th
at
ca
p
tu
r
es
th
e
ac
cu
r
ac
y
o
f
p
o
s
itiv
e
p
r
ed
ictio
n
s
(
p
r
ec
is
io
n
)
an
d
th
e
co
m
p
leten
ess
o
f
th
e
m
o
d
el
’
s
p
o
s
itiv
e
d
etec
tio
n
s
(
r
ec
all)
two
im
p
o
r
tan
t
m
etr
ics
in
a
d
ata
s
et
with
an
u
n
ev
en
class
d
is
tr
ib
u
tio
n
[
2
9
]
.
I
n
ca
s
e
s
wh
en
o
n
e
g
r
o
u
p
is
g
r
o
s
s
ly
u
n
d
er
-
r
e
p
r
esen
ted
,
th
is
s
tatis
tic
b
ec
o
m
es
cr
u
cial
in
p
r
ev
en
tin
g
an
o
v
er
em
p
h
asis
o
n
eith
er
r
ec
all
o
r
p
r
ec
is
io
n
[
3
0
]
.
Fo
r
a
th
o
r
o
u
g
h
e
v
alu
ati
o
n
o
f
th
e
m
o
d
el'
s
p
er
f
o
r
m
an
ce
,
th
e
F1
-
s
co
r
e
is
wid
ely
em
p
lo
y
ed
i
n
class
if
icat
io
n
an
d
in
f
o
r
m
atio
n
r
etr
iev
al
t
ask
s
to
m
ea
s
u
r
e
th
e
tr
ad
e
-
o
f
f
b
etwe
en
r
ec
all
an
d
p
r
ec
is
io
n
.
Usi
n
g
a
co
m
b
in
atio
n
o
f
r
ec
all
an
d
p
r
ec
is
io
n
,
t
h
e
F1
-
s
co
r
e
is
ca
lcu
late
d
ac
co
r
d
in
g
to
(
4
)
:
1
−
=
2
×
×
+
(
4
)
3
.
7
.
6
.
Are
a
un
der
t
he
RO
C
c
urv
e
(
AUC
-
RO
C)
On
e
im
p
o
r
tan
t
m
ea
s
u
r
e
f
o
r
a
s
s
es
s
in
g
a
m
o
d
el'
s
c
lass
d
i
s
cr
im
in
atio
n
ca
p
ab
ilit
ies
is
th
e
AUC
-
R
O
C
.
T
h
e
R
OC
cu
r
v
e
g
iv
es
a
co
m
p
lete
p
ictu
r
e
o
f
th
e
m
o
d
el'
s
p
er
f
o
r
m
a
n
ce
ac
r
o
s
s
all
class
if
i
ca
tio
n
th
r
esh
o
ld
s
b
y
p
lo
ttin
g
th
e
tr
u
e
p
o
s
itiv
e
r
ate
(
T
PR
)
v
er
s
u
s
th
e
f
alse
p
o
s
itiv
e
r
ate
(
FP
R
)
at
d
if
f
er
en
t
th
r
esh
o
ld
s
ettin
g
s
[
9
]
.
B
y
r
ef
lectin
g
th
e
p
r
o
b
ab
ilit
y
th
at
th
e
m
o
d
el
r
ates
a
r
an
d
o
m
ly
s
elec
ted
p
o
s
itiv
e
in
s
tan
ce
h
ig
h
er
th
an
a
r
an
d
o
m
ly
ch
o
s
en
n
eg
ativ
e
o
n
e,
a
b
ig
g
e
r
ar
ea
u
n
d
e
r
th
e
cu
r
v
e
(
AUC)
s
ig
n
if
ies
b
etter
p
e
r
f
o
r
m
an
ce
,
wh
ich
is
a
m
ea
s
u
r
e
o
f
s
ep
ar
ab
ilit
y
[
3
1
]
.
I
n
ca
s
es
wh
er
e
th
e
d
ataset
is
n
o
t
e
v
en
l
y
d
is
tr
ib
u
ted
,
t
h
is
s
tatis
tic
b
ec
o
m
es
in
v
alu
ab
le,
as
a
cc
u
r
ac
y
alo
n
e
ca
n
d
is
to
r
t th
e
p
ictu
r
e
o
f
h
o
w
well
a
m
o
d
el
is
d
o
in
g
[
3
2
]
.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
Ou
r
s
tu
d
y
'
s
f
in
d
in
g
s
b
r
ea
k
d
o
wn
ea
ch
m
o
d
el'
s
tr
ain
in
g
an
d
v
alid
atio
n
p
r
o
ce
d
u
r
es
in
g
r
ea
t
d
etail.
Alo
n
g
with
h
y
p
er
p
ar
am
eter
a
d
ju
s
tm
en
t
an
d
p
er
f
o
r
m
an
ce
g
r
ap
h
s
,
we
also
p
r
o
v
id
e
d
is
cu
s
s
i
o
n
an
d
co
m
p
ar
ativ
e
an
aly
s
is
u
s
in
g
ev
al
u
atio
n
m
e
asu
r
es.
Fu
r
th
er
m
o
r
e,
we
p
r
esen
t
a
ca
s
e
s
tu
d
y
th
at
illu
s
tr
a
tes
h
o
w
th
e
test
ed
m
o
d
els we
r
e
ap
p
lied
in
p
r
ac
tic
al
s
itu
atio
n
s
an
d
g
o
o
v
er
t
h
e
r
e
s
u
lts
.
4
.
1
.
H
y
perpa
ra
m
et
er
t
un
ing
re
s
u
lt
T
ab
le
2
s
h
o
ws
th
e
o
u
tco
m
es
o
f
h
y
p
er
p
ar
a
m
eter
tu
n
in
g
f
o
r
o
u
r
R
NN,
L
STM
,
GR
U,
an
d
B
i
-
L
STM
d
ee
p
lear
n
in
g
m
o
d
els
ass
ess
e
d
ac
r
o
s
s
s
ev
er
al
co
n
f
ig
u
r
atio
n
s
.
Du
r
in
g
th
e
m
o
d
el
cr
ea
tio
n
p
r
o
ce
s
s
,
th
e
ch
o
s
en
h
y
p
er
p
ar
am
eter
s
,
s
u
ch
as
em
b
ed
d
in
g
s
ize,
n
u
m
b
er
o
f
u
n
it
s
,
an
d
lear
n
in
g
r
ate,
ar
e
d
is
p
lay
ed
in
ea
ch
r
o
w.
T
h
ese
s
ettin
g
s
will
b
e
u
s
ed
t
o
ac
h
iev
e
o
p
tim
al
p
er
f
o
r
m
an
ce
.
T
h
e
L
STM
m
o
d
el
attain
e
d
a
s
co
r
e
o
f
0
.
6
6
9
4
with
an
e
m
b
ed
d
in
g
s
ize
o
f
1
2
5
,
2
4
6
u
n
its
,
an
d
a
lea
r
n
in
g
r
a
te
o
f
0
.
0
0
2
4
3
6
0
0
,
in
co
n
tr
ast
t
o
th
e
R
NN
m
o
d
el'
s
0
.
6
3
0
7
,
wh
ich
was
p
r
o
d
u
ce
d
b
y
an
em
b
ed
d
in
g
s
ize
o
f
1
4
6
,
7
4
u
n
its
an
d
a
lear
n
in
g
r
ate
o
f
0
.
0
0
0
1
5
7
3
0
.
W
ith
1
2
0
em
b
ed
d
in
g
s
,
7
5
u
n
its
,
a
n
d
a
lear
n
in
g
r
ate
o
f
0
.
0
0
2
8
2
0
0
0
,
t
h
e
GR
U
m
o
d
el
ac
h
iev
e
d
an
im
p
r
ess
iv
e
s
co
r
e
o
f
0
.
6
7
6
2
,
s
u
r
p
ass
in
g
all
o
f
it
s
co
m
p
etito
r
s
.
W
ith
2
8
9
,
2
5
4
u
n
its
o
f
em
b
e
d
d
in
g
s
ize
an
d
0
.
0
0
0
1
8
6
2
0
u
n
its
o
f
lear
n
in
g
r
ate,
th
e
B
i
-
L
STM
m
o
d
el
ac
h
iev
ed
a
s
co
r
e
o
f
0
.
6
6
9
7
.
T
h
ese
r
esu
lts
h
ig
h
lig
h
t
th
e
n
ee
d
o
f
cu
s
to
m
izin
g
th
e
h
y
p
er
p
a
r
am
eter
s
ettin
g
s
f
o
r
ev
er
y
m
o
d
el
to
g
et
th
e
b
est
o
u
tco
m
e
s
.
Hy
p
er
p
ar
am
eter
co
m
b
in
atio
n
s
in
clu
d
in
g
em
b
e
d
d
in
g
s
ize
an
d
lear
n
in
g
r
ate
ca
n
d
r
am
atica
lly
af
f
ec
t th
e
ef
f
ica
cy
an
d
p
r
ec
is
io
n
o
f
d
ee
p
lear
n
in
g
m
o
d
els,
as
d
e
m
o
n
s
tr
ated
b
y
th
e
GR
U
m
o
d
el'
s
o
u
tp
er
f
o
r
m
an
ce
.
Fo
r
ta
s
k
s
in
v
o
lv
in
g
t
h
e
class
if
icatio
n
o
f
s
ec
u
r
ity
p
atc
h
es
in
s
eq
u
e
n
tial
an
d
r
ec
u
r
r
e
n
t
d
ata,
it
is
p
ar
ticu
lar
l
y
im
p
o
r
tan
t
to
tu
n
e
an
d
ca
r
ef
u
lly
im
p
lem
e
n
t th
ese
p
ar
am
eter
s
d
u
r
in
g
th
e
m
o
d
el
-
b
u
ild
in
g
p
r
o
ce
s
s
.
T
ab
le
2
.
Hy
p
er
p
ar
a
m
eter
tu
n
i
n
g
b
est
r
esu
lt
M
o
d
e
l
Tr
i
a
l
Emb
e
d
d
i
n
g
U
n
i
t
s
Le
a
r
n
i
n
g
r
a
t
e
V
a
l
u
e
(
S
c
o
r
e
)
R
N
N
6
1
4
6
74
0
.
0
0
0
1
5
7
3
0
0
.
6
3
0
7
LSTM
2
1
2
5
2
4
6
0
.
0
0
2
4
3
6
0
0
0
.
6
6
9
4
G
R
U
15
1
2
0
75
0
.
0
0
2
8
2
0
0
0
0
.
6
7
6
2
Bi
-
LST
M
8
2
8
9
2
5
4
0
.
0
0
0
1
8
6
2
0
0
.
6
6
9
7
4
.
2
.
E
v
a
lua
t
io
n
m
et
rics
T
ab
le
3
d
em
o
n
s
tr
ates
th
at
w
h
ile
o
th
er
m
o
d
els
h
av
e
lo
wer
f
alse
n
eg
ativ
e
r
ates,
th
e
R
NN
m
o
d
el
(
D
ef
au
lt
)
h
as
a
m
o
r
e
ev
en
p
er
f
o
r
m
an
ce
.
W
ith
th
e
h
elp
o
f
a
d
ju
s
tm
en
t,
th
e
R
NN
m
o
d
el
b
ec
o
m
es
m
o
r
e
ef
f
icien
t,
lead
in
g
to
f
ewe
r
f
al
s
e
n
eg
ativ
es.
W
h
ile
th
e
L
STM
m
o
d
el
(
D
ef
a
u
lt
)
p
r
o
d
u
ce
s
m
o
r
e
f
alse
p
o
s
itiv
es
o
v
er
all,
it
m
ain
tain
s
a
r
ea
s
o
n
ab
le
r
atio
o
f
f
alse
n
eg
ativ
es
to
tr
u
e
p
o
s
itiv
es.
T
h
e
tu
n
ed
L
STM
m
o
d
el
d
em
o
n
s
tr
ates
an
im
p
r
o
v
em
e
n
t
in
r
ec
all
b
y
m
ar
g
in
ally
r
e
d
u
c
in
g
th
e
am
o
u
n
t
o
f
f
alse
n
eg
ati
v
es.
T
h
e
m
o
d
if
ied
v
er
s
io
n
o
f
th
e
GR
U
m
o
d
el
s
ig
n
if
ican
tly
r
ed
u
ce
s
f
alse
n
eg
ati
v
es,
m
ak
in
g
it
o
n
e
o
f
th
e
m
o
r
e
b
alan
ce
d
m
o
d
els,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
25
:
4
1
6
0
-
4171
4168
an
d
th
e
m
o
d
el
as
a
wh
o
le
s
h
o
ws
g
r
ea
t
p
er
f
o
r
m
an
ce
.
As
a
r
esu
lt
o
f
b
alan
cin
g
s
p
ec
if
icity
an
d
s
en
s
itiv
ity
,
th
e
Bi
-
L
STM
m
o
d
el
(
D
ef
a
u
lt
)
p
r
o
d
u
ce
s
th
e
m
o
s
t
f
alse
p
o
s
itiv
es.
Alth
o
u
g
h
th
e
B
i
-
L
STM
m
o
d
el
h
as
i
m
p
r
o
v
ed
ac
cu
r
ac
y
af
ter
tu
n
in
g
,
it st
ill p
r
o
d
u
ce
s
a
s
ig
n
if
ica
n
t a
m
o
u
n
t o
f
f
alse n
eg
ativ
es.
T
ab
le
3
.
Mo
d
el
p
er
f
o
r
m
a
n
ce
m
etr
ics
M
o
d
e
l
Tr
u
e
p
o
si
t
i
v
e
F
a
l
se
p
o
si
t
i
v
e
Tr
u
e
n
e
g
a
t
i
v
e
F
a
l
se
n
e
g
a
t
i
v
e
A
c
c
u
r
a
c
y
R
e
c
a
l
l
P
r
e
c
i
s
i
o
n
F1
-
s
c
o
r
e
R
N
N
(
D
e
f
a
u
l
t
)
8
9
2
6
2
0
2
9
5
9
9
0
2
0
.
7
1
6
7
3
2
0
.
4
9
7
2
1
3
0
.
5
8
9
9
4
7
0
.
5
3
9
6
2
5
R
N
N
(
Tu
n
e
d
)
1
0
9
0
6
3
0
2
9
4
9
7
0
4
0
.
7
5
1
7
2
2
0
.
6
0
7
5
8
1
0
.
6
3
3
7
2
1
0
.
6
2
0
3
7
6
LSTM
(
D
e
f
a
u
l
t
)
1
1
0
4
6
9
6
2
8
8
3
6
9
0
0
.
7
4
2
0
4
4
0
.
6
1
5
3
8
5
0
.
6
1
3
3
3
3
0
.
6
1
4
3
5
7
LSTM
(
Tu
n
e
d
)
1
1
1
6
6
6
2
2
9
1
7
6
7
8
0
.
7
5
0
6
0
5
0
.
6
2
2
0
7
4
0
.
6
2
7
6
7
2
0
.
6
2
4
8
6
G
R
U
(
D
e
f
a
u
l
t
)
1
1
2
9
5
7
7
3
0
0
2
6
6
5
0
.
7
6
8
8
4
4
0
.
6
2
9
3
2
0
.
6
6
1
7
8
2
0
.
6
4
5
1
4
3
G
R
U
(
Tu
n
e
d
)
1
1
8
1
6
0
2
2
9
7
7
6
1
3
0
.
7
7
3
8
6
9
0
.
6
5
8
3
0
5
0
.
6
6
2
3
6
7
0
.
6
6
0
3
3
Bi
-
LST
M
(
D
e
f
a
u
l
t
)
1
1
4
9
7
1
8
2
8
6
1
6
4
5
0
.
7
4
6
3
2
4
0
.
6
4
0
4
6
8
0
.
6
1
5
4
2
6
0
.
6
2
7
6
9
7
Bi
-
LST
M
(
T
u
n
e
d
)
1
0
9
3
5
4
1
3
0
3
8
7
0
1
0
.
7
6
8
8
4
4
0
.
6
0
9
2
5
3
0
.
6
6
8
9
1
1
0
.
6
3
7
6
9
L
o
o
k
in
g
at
it
an
aly
tically
,
th
e
R
NN
m
o
d
el
m
ay
m
is
s
r
ea
l
s
ec
u
r
ity
co
n
c
er
n
s
b
ec
a
u
s
e
to
i
ts
d
ef
au
lt
co
n
f
ig
u
r
atio
n
'
s
in
cr
ea
s
ed
f
r
e
q
u
en
cy
o
f
f
alse
n
eg
ativ
es,
wh
i
ch
m
ak
es
it
less
s
u
cc
ess
f
u
l
in
s
itu
atio
n
s
wh
er
e
r
ec
all
is
cr
itical.
W
h
en
it
i
s
cr
u
cial
to
m
in
im
ize
b
o
th
f
alse
p
o
s
itiv
es
an
d
f
alse
n
eg
ativ
es,
t
h
e
L
STM
an
d
GR
U
m
o
d
els
esp
ec
ially
in
th
eir
tu
n
e
d
v
er
s
io
n
s
s
tr
ik
e
a
s
u
p
er
io
r
b
a
lan
ce
b
etwe
en
r
ec
all
an
d
p
r
ec
i
s
io
n
.
Alth
o
u
g
h
th
e
Bi
-
L
STM
m
o
d
el's
s
p
ec
if
icity
is
en
h
an
ce
d
af
ter
tu
n
i
n
g
,
it
m
ay
n
o
t
b
e
ab
le
to
d
etec
t
al
l
p
o
s
itiv
e
ca
s
es
b
ec
au
s
e
to
its
r
elativ
ely
h
ig
h
f
alse
n
eg
ativ
e
r
ate.
I
f
r
e
d
u
cin
g
f
alse
n
e
g
ativ
es
is
m
o
r
e
im
p
o
r
tan
t
th
a
n
m
in
im
izin
g
f
alse
p
o
s
itiv
es,
as
is
th
e
ca
s
e
with
th
e
B
i
-
L
STM
,
th
en
th
e
b
est
m
o
d
el
to
u
s
e
wo
u
ld
b
e
th
e
o
n
e
th
at
b
est
s
u
its
th
e
ap
p
licatio
n
'
s
d
em
an
d
s
.
T
o
m
a
k
e
s
u
r
e
all
p
o
s
s
ib
le
th
r
ea
ts
ar
e
id
en
tifie
d
an
d
h
a
n
d
led
p
r
o
p
er
ly
,
f
o
r
ex
am
p
le,
it
m
ay
b
e
m
o
r
e
im
p
o
r
tan
t to
m
in
im
ize
f
alse n
eg
ativ
es in
s
o
f
twar
e
s
ec
u
r
ity
.
4
.
3
.
RO
C
–
AUC
r
esu
lt
T
u
n
in
g
clea
r
ly
im
p
r
o
v
es
ea
ch
m
o
d
el'
s
AUC
-
R
OC
p
er
f
o
r
m
an
ce
,
as
s
ee
n
in
Fig
u
r
e
3
.
I
n
t
h
e
ca
s
e
o
f
s
eq
u
en
tial
s
ec
u
r
ity
p
atch
d
ata,
th
e
GR
U
m
o
d
el
co
n
s
is
ten
tly
ea
r
n
s
th
e
g
r
ea
test
AUC
-
R
OC
s
co
r
es,
wh
eth
er
in
its
d
ef
au
lt
o
r
c
u
s
to
m
ized
v
er
s
io
n
.
T
h
is
s
u
g
g
ests
th
at
it
is
v
er
y
ca
p
a
b
le
o
f
ef
f
icien
tly
d
is
tin
g
u
is
h
in
g
b
etwe
en
class
es.
T
h
e
R
NN
m
o
d
el's
p
er
f
o
r
m
a
n
ce
is
g
r
ea
tly
im
p
r
o
v
e
d
b
y
tu
n
in
g
,
s
h
o
wca
s
in
g
its
i
m
p
r
o
v
e
d
ca
p
ac
ity
to
d
etec
t
im
p
o
r
tan
t
p
atter
n
s
in
th
e
d
ata
an
d
d
ec
r
ea
s
e
class
if
icatio
n
m
is
tak
es.
T
u
n
in
g
also
i
m
p
r
o
v
es
A
UC
-
R
OC
f
o
r
L
STM
an
d
B
i
-
L
STM
m
o
d
els,
th
o
u
g
h
to
a
less
er
e
x
ten
t
th
an
f
o
r
GR
U
an
d
R
NN.
E
s
p
ec
ially
f
o
r
co
m
p
licated
task
s
with
r
ec
u
r
r
en
t
an
d
s
eq
u
en
tial
d
ata,
th
ese
f
in
d
in
g
s
s
h
o
w
th
at
h
y
p
e
r
p
ar
a
m
eter
ad
ju
s
tm
en
t
is
cr
u
cial
f
o
r
im
p
r
o
v
in
g
m
o
d
el
p
er
f
o
r
m
a
n
ce
.
T
u
n
i
n
g
y
ield
s
v
ar
y
in
g
r
esu
lts
f
o
r
d
if
f
er
en
t m
o
d
e
ls
,
h
ig
h
lig
h
tin
g
th
e
n
ee
d
f
o
r
a
d
ataset
-
s
p
ec
if
ic,
in
d
iv
id
u
alize
d
s
tr
ateg
y
.
Fig
u
r
e
3
.
B
ar
ch
a
r
t o
f
R
OC
-
AUC r
esu
lt
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
E
xp
lo
r
in
g
th
e
r
ec
u
r
r
en
t a
n
d
s
eq
u
en
tia
l secu
r
ity
p
a
tch
d
a
ta
u
s
in
g
…
(
F
a
la
h
Mu
h
a
mma
d
A
la
m
)
4169
4
.
4
.
Dis
cu
s
s
io
n
W
ith
h
y
p
er
p
ar
a
m
eter
ad
ju
s
tm
en
t
in
p
ar
ticu
la
r
,
th
e
e
x
p
er
im
e
n
tal
f
in
d
in
g
s
s
h
o
w
th
at
th
e
G
R
U
m
o
d
el
attain
ed
th
e
b
est
ac
cu
r
ac
y
a
n
d
s
h
o
wed
b
alan
ce
d
p
er
f
o
r
m
an
ce
ac
r
o
s
s
all
ev
alu
atio
n
m
ea
s
u
r
es,
in
clu
d
in
g
F1
-
s
co
r
e,
p
r
ec
is
io
n
,
r
ec
all,
an
d
ac
cu
r
ac
y
.
T
h
is
in
d
icate
s
th
at
GR
U
m
o
d
el
is
well
s
u
ited
f
o
r
th
is
class
if
icatio
n
jo
b
s
in
ce
it
is
v
er
y
g
o
o
d
at
ca
p
tu
r
in
g
th
e
p
atter
n
s
an
d
tem
p
o
r
al
d
ep
en
d
en
cies
th
at
ar
e
in
h
er
en
t
in
s
eq
u
en
tial
an
d
r
ec
u
r
r
en
t
s
ec
u
r
ity
p
atch
d
ata.
T
o
en
s
u
r
e
th
at
m
ajo
r
s
ec
u
r
ity
v
u
ln
er
a
b
ilit
ies
ar
e
d
is
co
v
er
ed
im
m
ed
iately
,
GR
Us ac
cu
r
ately
d
etec
t tr
en
d
s
ac
r
o
s
s
m
an
y
p
atch
es d
ep
lo
y
e
d
o
v
er
tim
e
b
y
m
o
d
ellin
g
s
u
ch
tem
p
o
r
al
lin
k
ag
es.
I
t
is
cr
itical
to
n
o
t
m
is
s
an
y
s
ec
u
r
ity
r
is
k
s
,
p
ar
ticu
la
r
ly
r
e
cu
r
r
en
t
o
n
es,
a
n
d
th
e
tu
n
ed
R
NN
m
o
d
el
s
h
o
wn
co
n
s
id
er
ab
le
im
p
r
o
v
em
e
n
t
in
th
is
ar
ea
,
p
ar
ticu
lar
ly
in
lo
wer
in
g
f
alse
n
eg
ativ
es.
On
th
e
o
t
h
er
s
id
e,
th
e
L
STM
an
d
B
i
-
L
STM
m
o
d
els
p
r
o
d
u
ce
d
m
o
r
e
f
alse
n
eg
ativ
es d
esp
ite
k
ee
p
in
g
s
p
ec
if
icity
h
i
g
h
.
T
h
is
s
u
g
g
ests
th
at
th
ey
ar
e
m
o
r
e
ca
u
tio
u
s
an
d
m
a
y
o
v
er
lo
o
k
s
o
m
e
s
ec
u
r
ity
u
p
d
ate
s
,
esp
ec
ially
th
o
s
e
t
h
at
ar
e
p
a
r
t
o
f
a
c
o
n
s
ec
u
tiv
e
r
elea
s
e.
T
h
is
co
m
p
r
o
m
is
e
em
p
h
asizes
th
e
s
ig
n
if
ica
n
ce
o
f
s
elec
tin
g
a
m
o
d
el
ac
co
r
d
in
g
to
t
h
e
u
n
iq
u
e
p
r
o
p
e
r
ties
o
f
th
e
s
ec
u
r
ity
p
atch
d
ata
an
d
th
e
r
eq
u
ir
e
m
en
ts
o
f
t
h
e
ap
p
licatio
n
.
A
m
o
d
el
lik
e
as
th
e
twe
ak
ed
GR
U,
wh
ich
p
r
o
v
id
es
a
b
alan
ce
d
way
to
ca
p
tu
r
e
b
o
th
s
h
o
r
t
-
ter
m
an
d
l
o
n
g
-
ter
m
d
ata
d
e
p
en
d
e
n
cies,
co
u
ld
b
e
b
etter
in
s
itu
atio
n
s
wh
er
e
th
e
r
ep
er
c
u
s
s
io
n
s
o
f
m
is
s
in
g
a
p
o
s
s
ib
le
d
an
g
er
ar
e
h
ig
h
.
T
h
e
r
esu
lts
s
h
o
w
th
at
twea
k
in
g
h
y
p
er
p
ar
am
eter
s
is
cr
u
cial
f
o
r
im
p
r
o
v
in
g
m
o
d
el
p
e
r
f
o
r
m
an
c
e,
esp
ec
ially
with
r
ec
u
r
r
e
n
t
a
n
d
s
eq
u
e
n
tial
d
ata,
b
ec
au
s
e
all
m
o
d
els
im
p
r
o
v
ed
s
i
g
n
if
ican
tly
.
T
h
er
e
ar
e
a
lo
t
o
f
r
ea
l
-
wo
r
ld
ap
p
licatio
n
s
f
o
r
th
ese
f
in
e
-
tu
n
ed
m
o
d
els
wh
en
it
c
o
m
es
to
id
e
n
tify
in
g
s
ec
u
r
ity
p
atch
es.
On
e
ar
ea
w
h
er
e
t
h
ey
co
u
ld
b
e
u
s
ef
u
l
is
in
s
o
f
twar
e
s
y
s
tem
s
,
wh
er
e
th
e
r
elea
s
e
o
r
d
er
an
d
tim
in
g
o
f
p
atch
es a
r
e
v
er
y
im
p
o
r
tan
t
f
o
r
v
u
ln
er
ab
ilit
y
m
an
ag
em
e
n
t.
5.
CO
NCLU
SI
O
N
Fin
ally
,
th
is
wo
r
k
s
et
o
u
t
t
o
u
s
e
d
ee
p
lear
n
in
g
tech
n
iq
u
es
to
in
v
esti
g
ate
a
n
d
r
eso
lv
e
is
s
u
es
r
elate
d
to
s
eq
u
en
tial
an
d
r
ec
u
r
r
e
n
t
s
ec
u
r
ity
p
atch
d
ata.
T
h
e
r
esear
ch
s
u
cc
ess
f
u
lly
f
o
u
n
d
th
e
b
est
s
u
ited
d
ee
p
lear
n
in
g
m
o
d
els
f
o
r
th
is
co
m
p
licated
task
b
y
ex
ec
u
tin
g
a
s
er
ies
o
f
ca
r
ef
u
lly
p
r
ep
ar
ed
e
x
p
er
i
m
en
ts
.
T
h
ese
tr
ials
in
clu
d
ed
d
etailed
h
y
p
er
p
ar
a
m
eter
tu
n
in
g
an
d
m
o
d
el
ev
al
u
atio
n
.
I
n
ter
m
s
o
f
ca
p
tu
r
i
n
g
co
m
p
lex
tem
p
o
r
al
co
r
r
elatio
n
s
with
in
s
ec
u
r
ity
p
atch
d
ata
a
n
d
m
ain
tai
n
in
g
b
alan
ce
d
p
e
r
f
o
r
m
an
ce
ac
r
o
s
s
im
p
o
r
tan
t
m
e
tr
ics
in
clu
d
in
g
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
an
d
F1
-
s
co
r
e,
th
e
GR
U
m
o
d
el
p
r
o
v
e
d
to
b
e
th
e
m
o
s
t
ef
f
ec
tiv
e
am
o
n
g
th
o
s
e
th
at
wer
e
ev
alu
ated
.
B
ec
au
s
e
it
g
u
ar
an
tees
b
o
th
h
ig
h
s
en
s
itiv
ity
an
d
s
p
ec
if
icity
tw
o
q
u
alities
th
at
ar
e
cr
itical
f
o
r
co
r
r
ec
tly
d
etec
tin
g
an
d
c
ateg
o
r
izin
g
s
ec
u
r
ity
p
atch
es
th
e
GR
U
m
o
d
el
is
e
s
p
ec
ially
u
s
ef
u
l
in
h
an
d
lin
g
s
eq
u
en
tial
an
d
r
ec
u
r
r
en
t
d
ata
d
u
e
to
its
b
alan
ce
d
p
er
f
o
r
m
an
ce
.
T
h
e
tr
ad
e
-
o
f
f
b
etwe
en
m
in
im
izin
g
f
alse
p
o
s
itiv
es
an
d
av
o
id
in
g
t
h
e
o
m
is
s
io
n
o
f
ac
tu
al
th
r
ea
ts
was
h
ig
h
lig
h
ted
b
y
m
o
d
els
l
ik
e
L
STM
an
d
B
i
-
L
STM
,
wh
ich
s
h
o
wed
h
ig
h
er
s
p
ec
if
icity
b
u
t
p
r
o
d
u
ce
d
m
o
r
e
f
alse
n
eg
ativ
es,
ev
en
th
o
u
g
h
th
ey
co
u
ld
p
r
o
ce
s
s
ex
ten
s
iv
e
tem
p
o
r
al
s
eq
u
en
ce
s
.
B
ec
au
s
e
v
ar
io
u
s
s
ce
n
ar
io
s
m
ay
p
lace
a
p
r
em
iu
m
o
n
d
if
f
er
en
t
asp
ec
ts
o
f
p
er
f
o
r
m
an
ce
,
o
u
r
r
esu
lts
h
ig
h
l
ig
h
t
th
e
n
ee
d
o
f
ap
p
licatio
n
-
s
p
ec
if
ic
m
o
d
el
s
elec
tio
n
a
n
d
h
y
p
er
p
ar
am
ete
r
ad
ju
s
tm
en
t.
T
h
is
s
tu
d
y
s
h
ed
s
lig
h
t
o
n
d
ee
p
lear
n
in
g
'
s
p
o
ten
tial
in
cy
b
er
s
ec
u
r
ity
b
y
s
o
lv
in
g
th
e
f
u
n
d
a
m
en
tal
p
r
o
b
lem
o
f
ca
teg
o
r
izin
g
co
m
p
licated
,
s
eq
u
en
tial
s
ec
u
r
ity
p
atch
d
ata.
Dep
lo
y
in
g
well
-
o
p
tim
ized
d
ee
p
lear
n
in
g
m
o
d
els
ca
n
g
r
ea
tly
im
p
r
o
v
e
s
o
f
twar
e
s
y
s
tem
s
ec
u
r
ity
b
y
m
ak
i
n
g
v
u
ln
er
a
b
ilit
y
id
e
n
tific
atio
n
an
d
p
atc
h
m
an
ag
em
en
t
p
r
o
ce
s
s
es
m
o
r
e
ef
f
icien
t
an
d
less
r
eq
u
ir
in
g
h
u
m
an
in
ter
ac
tio
n
.
T
h
is
h
as
f
a
r
-
r
ea
ch
in
g
p
r
ac
tical
r
am
if
icatio
n
s
.
A
d
d
in
g
m
o
r
e
t
y
p
es
o
f
d
ata
to
th
e
d
ataset,
p
u
ttin
g
th
ese
m
o
d
els
th
r
o
u
g
h
t
h
eir
p
ac
es
in
r
ea
l
-
wo
r
ld
s
ce
n
ar
io
s
,
an
d
e
x
p
lo
r
in
g
h
y
b
r
id
m
o
d
els
th
at
d
r
aw
f
r
o
m
d
if
f
er
en
t
d
ee
p
lear
n
in
g
ar
ch
itectu
r
es
s
h
o
u
ld
all
b
e
g
o
als
o
f
f
u
tu
r
e
s
tu
d
ies.
I
n
ad
d
itio
n
,
b
y
u
tili
zin
g
ex
t
er
n
al
d
ata
s
o
u
r
ce
s
o
r
in
co
r
p
o
r
atin
g
c
o
n
tex
tu
al
in
f
o
r
m
atio
n
,
a
d
v
an
ce
d
f
ea
t
u
r
e
en
g
i
n
ee
r
in
g
tech
n
i
q
u
es
ca
n
en
h
an
ce
t
h
ese
m
o
d
els'
p
er
f
o
r
m
an
ce
an
d
ad
ap
tab
ilit
y
.
T
h
is
m
ea
n
s
th
e
y
ca
n
b
e
ap
p
lied
to
a
wid
e
r
r
an
g
e
o
f
cy
b
er
s
ec
u
r
ity
c
h
allen
g
es.
ACK
NO
WL
E
DG
M
E
N
T
S
T
h
e
a
u
th
o
r
s
wo
u
l
d
lik
e
to
e
x
p
r
ess
th
eir
s
in
ce
r
e
g
r
atitu
d
e
to
all
in
d
i
v
id
u
als
wh
o
p
r
o
v
id
e
d
tech
n
ical
an
d
ed
ito
r
ial
s
u
p
p
o
r
t t
h
r
o
u
g
h
o
u
t th
is
r
esear
ch
.
F
UNDING
I
NF
O
R
M
A
T
I
O
N
Au
th
o
r
s
s
tate
n
o
f
u
n
d
in
g
in
v
o
lv
ed
.
AUTHO
R
CO
NT
RI
B
UT
I
O
NS
T
h
is
jo
u
r
n
al
u
s
es
th
e
C
o
n
tr
ib
u
to
r
R
o
les
T
ax
o
n
o
m
y
(
C
R
ed
iT)
to
r
ec
o
g
n
ize
in
d
iv
id
u
al
au
th
o
r
co
n
tr
ib
u
tio
n
s
,
r
ed
u
ce
au
th
o
r
s
h
ip
d
is
p
u
tes,
an
d
f
ac
ilit
ate
co
llab
o
r
atio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.