I
A
E
S
I
n
t
e
r
n
at
io
n
al
Jou
r
n
al
of
A
r
t
if
ic
ia
l
I
n
t
e
ll
ig
e
n
c
e
(
I
J
-
AI
)
V
ol
. 14, No. 5, O
c
to
be
r
2025
, pp.
4061
~
4073
I
S
S
N
:
2252
-
8938
,
D
O
I
:
10.11591/
ij
a
i.
v
14
.i
5
.pp
4061
-
4073
4061
Jou
r
n
al
h
om
e
page
:
ht
tp
:
//
ij
ai
.
ia
e
s
c
or
e
.c
om
D
e
si
gn
an
d
an
a
l
ysi
s of
r
e
i
n
f
or
c
e
m
e
n
t
l
e
ar
n
i
n
g m
o
d
e
l
s f
or
au
t
om
at
e
d
p
e
n
e
t
r
at
i
on
t
e
st
i
n
g
S
u
r
e
s
h
Jagan
at
h
an
1
, M
r
it
h
u
la
K
e
s
avan
L
at
h
a
2
, K
r
it
h
ik
a D
h
ar
an
ik
ot
a
3
1
D
e
pa
r
t
m
e
nt
of
C
om
put
e
r
S
c
i
e
nc
e
a
nd E
ngi
ne
e
r
i
ng
, S
r
i
S
i
va
s
ubr
a
m
a
ni
ya
N
a
da
r
C
ol
l
e
ge
of
E
ngi
ne
e
r
i
ng, C
he
nna
i
, I
ndi
a
2
S
of
t
w
a
r
e
D
e
ve
l
opm
e
nt
A
na
l
ys
t
,
C
i
t
i
c
or
p S
e
r
vi
c
e
s
I
ndi
a
L
t
d, C
he
nn
a
i
, I
ndi
a
3
D
e
pa
r
t
m
e
nt
of
C
om
put
e
r
S
c
i
e
nc
e
, U
ni
ve
r
s
i
t
y of
S
out
he
r
n C
a
l
i
f
or
ni
a
,
L
os
A
nge
l
e
s
,
U
ni
t
e
d S
t
a
t
e
s
A
r
t
ic
le
I
n
f
o
A
B
S
T
R
A
C
T
A
r
ti
c
le
h
is
to
r
y
:
R
e
c
e
iv
e
d
J
un 13, 2024
R
e
vi
s
e
d
J
ul
26, 2025
A
c
c
e
pt
e
d
A
ug 6, 2025
Our
paper
proposes
a
framework
to
automate
penetration
testing
by
utilizing
reinforcement
learning
(RL)
capabiliti
es.
The
framework
aims
to
i
dentify
and
prioritize
vulnerable
paths
within
a
network
by
dynamically
le
arning
and
adapting
strategies
for
vulnerability
assessment
by
acquiri
ng
the
network
data
obtained
from
a
comprehensive
network
scanner.
The
study
evaluates
three
RL
algorit
hms
:
deep
Q
-
network
(DQN),
deep
deter
ministic
policy
gradient
(DDPG),
and
asynchrono
us
episodi
c
deep
determin
istic
policy
gradient
(AE
-
DDPG)
in
order
to
compare
their
effectiveness
f
or
this
task. DQN
uses a lea
rned mode
l
of the e
nvironment to
make dec
isions
and is
hence
called
model
-
based
RL,
while
DDPG
and
AE
-
DDPG
learn
d
irectly
from
interactions
with
the
network
environment
and
are
called
model
-
free
RL.
By
dynamically
adapting
its
strategies,
the
framework
can
identi
fy
and
focus
on
the
most
critical
vulnerabilities
within
the
network
infrastr
ucture.
Our
work
is
to
check
how
wel
l
the
RL
technique
picked
s
ecurity
vulnerabilities.
The
identified
vulnerable
paths
are
tested
using
Meta
sploit,
which
also
confirmed
the
accuracy
of
the
RL
approach
'
s
result
s.
The
tabulated
findings
show
that
RL
promises
to
automate
penetratio
n
testing
tasks
.
K
e
y
w
o
r
d
s
:
A
s
ync
hr
onous
e
pi
s
odi
c
de
e
p
de
te
r
m
in
is
ti
c
pol
ic
y gr
a
di
e
nt
A
ut
om
a
te
d pe
ne
tr
a
ti
on t
e
s
ti
ng
D
e
e
p de
te
r
m
in
is
ti
c
pol
ic
y
gr
a
di
e
nt
D
e
e
p Q
-
ne
twor
k
R
e
in
f
or
c
e
m
e
nt
l
e
a
r
ni
ng
This is an
open
acce
ss artic
le unde
r the
CC BY
-
SA
license.
C
or
r
e
s
pon
di
n
g A
u
th
or
:
S
ur
e
s
h J
a
ga
na
th
a
n
D
e
pa
r
tm
e
nt
of
C
om
put
e
r
S
c
ie
nc
e
a
nd E
ngi
ne
e
r
in
g
,
S
r
i
S
iv
a
s
ubr
a
m
a
ni
ya
N
a
da
r
C
ol
le
ge
of
E
ngi
ne
e
r
in
g
C
he
nna
i,
I
ndi
a
E
m
a
il
:
s
ur
e
s
hj
@
s
s
n.e
du.i
n
1.
I
N
T
R
O
D
U
C
T
I
O
N
P
e
ne
tr
a
ti
on
te
s
ti
ng,
or
pe
n
-
te
s
ti
ng,
is
c
r
uc
ia
l
f
or
e
va
lu
a
ti
ng
in
f
or
m
a
ti
on
te
c
hnol
ogy
(
IT
)
in
f
r
a
s
tr
uc
tu
r
e
r
e
s
il
ie
nc
e
a
ga
in
s
t
c
ybe
r
th
r
e
a
ts
by
p
r
oa
c
ti
ve
ly
i
de
nt
if
yi
ng
vul
ne
r
a
bi
li
ti
e
s
.
T
hi
s
pr
oc
e
s
s
he
lp
s
or
ga
ni
z
a
ti
ons
s
tr
e
ngt
he
n
th
e
ir
s
e
c
ur
it
y
pos
tu
r
e
a
nd
pr
e
ve
nt
po
te
nt
ia
l
da
ta
br
e
a
c
he
s
a
nd
f
in
a
nc
ia
l
de
tr
im
e
nt
.
P
e
n
-
te
s
ti
ng
a
ls
o
a
id
s
in
r
e
gul
a
to
r
y
c
om
pl
ia
nc
e
a
nd
pr
ovi
de
s
in
s
ig
ht
s
f
or
in
f
or
m
e
d
s
e
c
ur
it
y
in
ve
s
tm
e
nt
s
w
hi
c
h
m
a
nda
te
r
e
gul
a
r
s
e
c
ur
it
y
e
va
lu
a
ti
ons
.
W
it
h
th
e
e
vol
ut
io
n
o
f
m
a
c
hi
ne
le
a
r
ni
ng,
pa
r
ti
c
ul
a
r
ly
r
e
in
f
or
c
e
m
e
nt
le
a
r
ni
ng
(
R
L
)
,
pe
n
-
te
s
ti
ng
te
c
hni
que
s
a
r
e
be
c
om
in
g
m
or
e
a
ut
om
a
te
d
a
nd
e
f
f
e
c
ti
ve
,
im
pr
ovi
ng
s
e
c
ur
it
y
a
s
s
e
s
s
m
e
nt
s
f
or
or
ga
ni
z
a
ti
ons
.
W
e
w
il
l
e
xa
m
in
e
th
e
e
xi
s
ti
ng
la
nds
c
a
pe
of
m
ode
l
-
ba
s
e
d
pe
n
e
tr
a
ti
on
te
s
ti
ng
a
nd
in
tr
oduc
e
a
n
in
nova
ti
v
e
f
r
a
m
e
w
or
k
[
1]
th
a
t
ha
r
ne
s
s
e
s
R
L
to
s
tr
e
a
m
li
ne
a
nd
a
ugm
e
nt
th
e
p
e
ne
tr
a
ti
on
te
s
ti
ng pr
oc
e
s
s
.
C
om
b
in
in
g
pe
ne
t
r
a
ti
o
n
te
s
ti
ng
w
i
th
R
L
p
r
e
s
e
nt
s
a
p
r
om
is
in
g
s
t
r
a
te
gy
to
e
nha
n
c
e
c
o
nve
nt
io
n
a
l
m
e
t
hod
ol
og
ie
s
.
R
L
a
lg
o
r
it
h
m
s
c
a
n
a
u
to
m
a
t
ic
a
l
ly
pr
io
r
it
iz
e
vul
ne
r
a
b
il
it
ie
s
ba
s
e
d
o
n
th
e
i
r
l
ik
e
li
hoo
d
a
nd
p
ot
e
nt
ia
l
i
m
pa
c
t,
m
i
ni
m
iz
in
g
t
he
n
e
e
d
f
o
r
m
a
n
ua
l
in
te
r
ve
nt
io
n.
T
h
e
s
e
a
l
go
r
it
h
m
s
a
da
p
t
e
f
f
e
c
t
iv
e
ly
to
d
yna
m
ic
e
nv
ir
on
m
e
n
ts
a
n
d
e
v
ol
v
in
g
t
hr
e
a
t
la
nds
c
a
pe
s
,
e
ns
u
r
i
ng
t
ha
t
pe
ne
t
r
a
t
io
n
te
s
ti
ng
t
e
c
h
ni
q
ue
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 5, O
c
to
be
r
2025
:
4061
-
4073
4062
r
e
m
a
in
e
f
f
e
c
ti
ve
o
ve
r
t
im
e
.
F
ur
th
e
r
m
o
r
e
,
R
L
e
na
b
le
s
p
a
r
a
l
le
l
e
xp
lo
r
a
t
io
n
o
f
va
r
io
us
a
t
ta
c
k
s
tr
a
te
gi
e
s
,
le
a
d
in
g
to
m
o
r
e
c
o
m
p
r
e
h
e
ns
i
ve
v
ul
n
e
r
a
bi
li
ty
a
s
s
e
s
s
m
e
n
ts
.
T
h
is
s
c
a
la
bi
li
ty
m
a
ke
s
R
L
w
e
ll
-
s
u
it
e
d
f
or
ha
nd
li
ng
e
x
te
ns
iv
e
a
n
d
in
tr
ic
a
te
ne
t
w
o
r
ks
,
pr
ov
id
in
g
o
r
ga
ni
z
a
ti
ons
w
i
th
a
c
o
m
p
r
e
-
he
ns
iv
e
a
pp
r
oa
c
h
to
s
e
c
ur
it
y e
va
l
ua
ti
o
ns
.
P
e
ne
tr
a
ti
on
te
s
ti
ng
pr
im
a
r
il
y
r
e
li
e
s
on
m
ode
l
-
ba
s
e
d
te
c
hni
qu
e
s
,
w
he
r
e
e
xpe
r
ts
c
r
e
a
t
e
a
de
ta
il
e
d
m
ode
l
of
th
e
ne
twor
k
us
in
g
da
ta
f
r
om
s
c
a
ns
a
nd
a
n
a
ly
s
e
s
to
id
e
nt
if
y
vul
ne
r
a
bi
li
ti
e
s
.
W
hi
le
e
f
f
e
c
ti
ve
in
c
ont
r
ol
le
d
e
nvi
r
onm
e
nt
s
,
th
is
a
ppr
oa
c
h
s
tr
uggl
e
s
w
it
h
th
e
dyna
m
ic
na
tu
r
e
of
m
ode
r
n
ne
twor
ks
.
T
h
e
e
m
e
r
ge
nc
e
of
ne
w
vul
ne
r
a
bi
li
ti
e
s
a
nd
ne
twor
k
c
ha
ng
e
s
m
a
ke
m
a
in
ta
in
in
g
a
n
a
c
c
ur
a
te
m
ode
l
c
ha
ll
e
ngi
ng.
T
o
a
ddr
e
s
s
th
is
,
in
te
gr
a
ti
ng
R
L
in
to
m
ode
l
-
ba
s
e
d
m
e
th
odol
ogi
e
s
c
a
n
pr
ovi
de
a
d
a
pt
iv
e
c
a
pa
bi
li
ti
e
s
.
B
y
doi
ng
s
o,
or
ga
ni
z
a
ti
ons
c
a
n
im
pr
ove
th
e
a
c
c
ur
a
c
y
a
nd
e
f
f
e
c
ti
ve
ne
s
s
of
th
e
ir
vu
ln
e
r
a
bi
li
ty
a
s
s
e
s
s
m
e
nt
s
,
le
a
di
ng
to
r
obus
t
s
e
c
ur
it
y m
e
a
s
ur
e
s
.
P
e
ne
tr
a
ti
on
te
s
ti
ng
m
e
th
ods
ha
ve
e
vol
ve
d,
ye
t
obs
ta
c
le
s
pe
r
s
is
t
in
e
f
f
ic
ie
nt
ly
de
te
c
ti
ng
a
nd
pr
io
r
it
iz
in
g
vul
ne
r
a
bi
l
it
ie
s
,
e
s
pe
c
ia
ll
y
a
c
r
os
s
e
xt
e
n
s
iv
e
,
in
tr
ic
a
te
ne
twor
ks
.
M
a
nua
l
pr
oc
e
s
s
e
s
de
m
a
nd
s
ubs
ta
nt
ia
l
ti
m
e
a
nd
e
f
f
or
t,
pot
e
nt
ia
ll
y
r
e
s
ul
ti
ng
in
ove
r
s
ig
ht
or
in
c
or
r
e
c
t
pr
io
r
i
ti
z
a
ti
on
of
s
e
c
ur
it
y
vul
ne
r
a
bi
li
ti
e
s
due
to
hum
a
n
f
a
ll
ib
il
it
y.
A
ddi
ti
ona
ll
y,
th
e
c
om
pl
e
xi
ty
of
m
ode
r
n
ne
twor
ks
c
a
n
m
a
ke
it
di
f
f
ic
ul
t
to
id
e
nt
if
y
th
e
m
os
t
c
r
it
ic
a
l
vul
ne
r
a
bi
li
ti
e
s
th
a
t
pos
e
th
e
gr
e
a
te
s
t
r
is
k
to
a
n
or
ga
ni
z
a
ti
on.
T
r
a
di
ti
ona
l
m
ode
l
-
ba
s
e
d
a
ppr
oa
c
he
s
s
tr
uggl
e
to
a
d
a
pt
to
ne
w
pr
obl
e
m
s
,
r
e
s
ul
ti
ng
in
l
e
s
s
e
f
f
e
c
ti
ve
vul
ne
r
a
bi
li
ty
a
s
s
e
s
s
m
e
nt
s
.
T
he
pr
opos
e
d
s
ol
ut
io
n
a
ddr
e
s
s
e
s
c
ha
ll
e
nge
s
in
pe
ne
tr
a
ti
on
te
s
ti
ng
by
de
ve
lo
pi
ng
a
n
a
ut
om
a
te
d
f
r
a
m
e
w
or
k
us
in
g
R
L
to
id
e
nt
if
y
a
nd
pr
io
r
it
iz
e
vul
ne
r
a
bi
li
ti
e
s
in
a
n
e
twor
k.
T
he
f
r
a
m
e
w
or
k
dyna
m
ic
a
ll
y
le
a
r
ns
a
nd
a
da
pt
s
it
s
s
tr
a
te
gi
e
s
b
a
s
e
d
on
ne
twor
k
d
a
ta
f
r
om
a
c
om
pr
e
he
ns
iv
e
s
c
a
nne
r
.
I
t
c
on
s
is
ts
of
th
r
e
e
m
odul
e
s
:
ne
twor
k
a
na
ly
z
e
r
,
RL
e
ngi
n
e
us
in
g
de
e
p
Q
-
ne
twor
k
(
D
Q
N
)
,
de
e
p
de
te
r
m
in
is
ti
c
pol
ic
y
gr
a
di
e
nt
(
D
D
P
G
)
,
a
nd
a
s
ync
hr
onous
e
pi
s
odi
c
de
e
p
de
te
r
m
in
is
ti
c
pol
ic
y
gr
a
di
e
nt
(
A
E
-
D
D
P
G
)
a
lg
or
i
th
m
s
,
a
nd
a
pe
n
-
te
s
ti
ng
m
odul
e
.
T
he
ne
twor
k
a
na
ly
z
e
r
c
ol
le
c
ts
a
nd
a
na
ly
s
e
s
ne
twor
k
da
ta
,
id
e
nt
if
yi
ng
vul
ne
r
a
bi
li
ti
e
s
a
nd
a
tt
a
c
k
pa
th
s
.
T
he
R
L
e
ngi
ne
pr
io
r
it
iz
e
s
vul
ne
r
a
bi
li
ti
e
s
a
nd
de
t
e
r
m
in
e
s
opt
im
a
l
a
tt
a
c
k
pa
th
s
,
w
hi
le
th
e
pe
n
-
te
s
ti
ng
m
odul
e
ve
r
if
ie
s
th
e
s
e
pa
th
s
us
in
g
in
dus
tr
y
-
s
ta
nda
r
d
t
ool
s
.
T
hi
s
in
te
gr
a
ti
on
a
im
s
to
a
ut
om
a
te
a
nd
opt
im
iz
e
pe
ne
tr
a
ti
on
te
s
ti
ng,
a
da
pt
in
g
to
ne
twor
k
c
ha
nge
s
a
nd
r
e
duc
in
g
c
ybe
r
a
tt
a
c
k
r
is
ks
.
U
lt
im
a
te
ly
,
th
e
f
r
a
m
e
w
or
k
s
e
e
ks
to
r
e
vol
ut
io
ni
z
e
c
ybe
r
s
e
c
ur
it
y
pr
a
c
ti
c
e
s
,
pr
o
vi
di
ng
de
f
e
nde
r
s
w
it
h
a
da
pt
iv
e
a
nd
in
te
ll
ig
e
nt
to
ol
s
t
o c
om
ba
t
c
ybe
r
t
hr
e
a
ts
e
f
f
e
c
ti
ve
ly
.
2.
R
E
L
A
T
E
D
WORKS
T
he
c
onc
e
pt
of
a
ut
om
a
ti
ng
pe
ne
tr
a
ti
on
te
s
ti
ng
[
2]
ha
s
be
e
n
a
lo
ngs
ta
ndi
ng
pur
s
ui
t,
in
i
ti
a
ll
y
m
a
ni
f
e
s
ti
ng
in
th
e
f
or
m
of
a
tt
a
c
k
gr
a
phs
[
3
]
.
A
tt
a
c
k
gr
a
phs
[
4
]
s
e
r
ve
a
s
m
ode
ls
to
de
pi
c
t
s
ys
te
m
s
a
nd
th
e
ir
s
us
c
e
pt
ib
il
it
y
to
pa
r
ti
c
ul
a
r
e
xpl
oi
ts
. T
r
a
di
ti
ona
ll
y,
id
e
nt
if
yi
ng
th
e
s
e
a
tt
a
c
k
pa
th
s
in
vol
ve
d e
m
pl
oyi
ng
c
la
s
s
ic
a
l
pl
a
nni
ng
m
e
th
odol
ogi
e
s
.
H
ow
e
ve
r
,
a
s
ub
s
ta
nt
ia
l
di
s
a
dva
n
ta
ge
of
th
is
a
ppr
oa
c
h
is
it
s
r
e
li
a
nc
e
on
c
om
pr
e
he
ns
iv
e
knowle
dge
of
th
e
ne
twor
k
to
pol
ogy
a
nd
th
e
c
onf
ig
ur
a
ti
on
of
e
a
c
h
m
a
c
hi
ne
,
r
e
nde
r
in
g
it
im
pr
a
c
ti
c
a
l
f
r
om
t
he
pe
r
s
pe
c
ti
ve
of
a
n a
tt
a
c
ke
r
.
A
n
a
lt
e
r
na
t
iv
e
a
pp
r
oa
c
h
to
m
ode
ll
in
g
a
nd
s
t
r
a
te
g
iz
i
ng
a
tt
a
c
ks
a
ga
in
s
t
a
s
ys
te
m
i
nv
ol
ve
s
e
m
p
lo
y
in
g
a
M
a
r
ko
v
de
c
is
io
n
p
r
oc
e
s
s
(
M
D
P
)
to
s
im
u
la
te
t
he
ope
r
a
t
io
na
l
e
nv
ir
on
m
e
n
t.
A
n M
D
P
[
5
]
s
e
r
ve
s
a
s
a
ve
r
s
a
ti
le
f
r
a
m
e
w
o
r
k
f
o
r
r
e
p
r
e
s
e
nt
in
g
d
is
c
r
e
te
de
c
is
i
on
-
m
a
ki
ng
s
c
e
na
r
i
os
a
m
id
un
c
e
r
ta
in
ty
.
W
he
n
a
ppl
ie
d
to
p
e
ne
tr
a
ti
on
te
s
t
in
g
,
t
he
s
ta
te
s
pa
c
e
w
it
hi
n
th
e
M
D
P
e
nc
om
pa
s
s
e
s
t
he
po
te
n
ti
a
l
c
o
n
f
ig
u
r
a
t
io
ns
of
ta
r
ge
t
m
a
c
hi
ne
s
o
r
th
e
ne
two
r
k
,
w
it
h
a
c
ti
ons
r
e
p
r
e
s
e
nt
in
g
a
va
il
a
bl
e
e
x
pl
oi
ts
o
r
s
c
a
ns
a
n
d
r
e
w
a
r
ds
c
ont
in
ge
n
t
u
pon
th
e
c
os
ts
a
s
s
oc
i
a
te
d
w
i
th
a
c
t
io
ns
a
nd
th
e
va
l
u
e
a
c
c
r
ue
d
upo
n
s
uc
c
e
s
s
f
ul
ly
c
om
p
r
om
is
in
g
a
s
ys
te
m
.
R
L
e
m
e
r
ge
s
a
s
a
te
c
h
ni
que
c
a
p
a
b
le
of
d
e
r
iv
in
g
op
ti
m
a
l
pol
ic
ie
s
f
or
M
D
P
s
.
I
t
le
ve
r
a
ge
s
in
t
e
r
a
c
t
io
ns
w
it
h
t
he
e
nv
i
r
on
m
e
n
t
t
o
ge
ne
r
a
te
s
a
m
pl
e
s
,
t
he
r
e
by
op
ti
m
iz
in
g
pe
r
f
or
m
a
n
c
e
.
D
is
t
in
c
t
a
dva
nt
a
ge
s
ove
r
c
la
s
s
ic
a
l
p
la
n
ni
ng
m
e
th
od
ol
o
gi
e
s
i
nc
lu
de
it
s
a
de
pt
ne
s
s
a
t
ha
n
dl
in
g
e
xpa
ns
iv
e
e
nvi
r
on
m
e
n
ts
a
nd
i
ts
a
ppl
ic
a
bi
li
ty
i
n
s
c
e
na
r
io
s
w
he
r
e
e
it
he
r
a
m
ode
l
of
t
he
e
nv
i
r
on
m
e
nt
is
a
bs
e
nt
o
r
ut
i
li
z
i
ng
th
e
m
ode
l
pr
ove
s
c
o
m
pu
ta
ti
ona
ll
y i
nt
r
a
c
ta
b
le
.
A
p
a
r
ti
a
ll
y
obs
e
r
va
b
le
M
a
r
ko
v
de
c
is
io
n
p
r
oc
e
s
s
(
P
O
M
D
P
)
is
a
va
r
ia
nt
o
f
t
he
M
D
P
th
a
t
in
c
o
r
po
r
a
t
e
s
unc
e
r
ta
i
nt
y
a
b
ou
t
th
e
p
r
e
c
is
e
s
ta
te
o
f
t
he
s
ys
te
m
.
I
n
a
P
O
M
D
P
,
t
he
c
u
r
r
e
n
t
s
ta
te
is
m
o
de
l
le
d
a
s
a
pr
oba
bi
li
ty
d
is
t
r
i
bu
ti
on
e
nc
om
p
a
s
s
in
g
a
l
l
p
ot
e
nt
ia
l
s
ta
t
e
s
.
W
he
n
a
p
pl
ie
d
t
o
p
e
ne
tr
a
t
io
n
t
e
s
t
in
g
,
th
e
s
ta
te
s
pa
c
e
w
it
hi
n
t
he
P
O
M
D
P
e
nc
om
pa
s
s
e
s
th
e
po
te
n
ti
a
l
c
on
f
i
gu
r
a
t
io
ns
o
f
th
e
t
a
r
ge
t
m
a
c
h
in
e
o
r
ne
tw
or
k.
A
c
ti
ons
r
e
p
r
e
s
e
nt
a
va
i
la
b
le
e
x
pl
oi
ts
or
s
c
a
ns
,
w
hi
le
t
he
o
bs
e
r
va
t
io
n
s
pa
c
e
c
om
p
r
is
e
s
th
e
in
f
o
r
m
a
t
io
n
ga
th
e
r
e
d
w
h
e
n
a
n
e
xp
lo
it
o
r
s
c
a
n
is
e
x
e
c
u
te
d
(
e
.
g.
,
ope
n
p
or
t
s
,
s
uc
c
e
s
s
,
or
f
a
il
ur
e
of
th
e
e
xp
lo
it
)
.
R
e
w
a
r
ds
a
r
e
de
t
e
r
m
i
ne
d
ba
s
e
d
o
n
t
he
c
os
t
o
f
a
n
a
c
ti
on
a
nd
th
e
va
lu
e
ga
in
e
d
f
r
om
s
uc
c
e
s
s
f
u
ll
y
c
om
p
r
om
is
in
g
a
s
ys
te
m
.
P
O
M
D
P
l
e
ve
r
a
g
e
s
th
e
c
o
nc
e
p
t
o
f
"
b
e
l
ie
f
"
t
o
r
e
pr
e
s
e
nt
un
c
e
r
ta
in
ty
in
d
e
c
is
io
n
-
m
a
k
in
g
du
r
i
ng
a
tt
a
c
ks
.
P
O
M
D
P
is
us
e
d
to
s
im
ul
a
te
[
6]
t
he
"
P
e
n
te
s
t
"
ta
s
k,
a
im
in
g
t
o
f
i
nd
t
he
s
h
o
r
te
s
t
pa
t
h
to
th
e
ta
r
ge
t
node
.
H
ow
e
ve
r
,
d
ue
t
o
th
e
in
he
r
e
nt
c
om
pl
e
xi
ty
o
f
P
O
M
D
P
a
lg
o
r
it
hm
s
,
th
e
i
r
a
pp
li
c
a
bi
l
it
y
is
c
on
f
in
e
d
to
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
D
e
s
ig
n and analys
i
s
of
r
e
in
fo
r
c
e
m
e
nt
l
e
a
r
ni
ng m
ode
ls
f
o
r
aut
om
at
e
d
pe
ne
tr
at
io
n
…
(
Sur
e
s
h
J
aganathan
)
4063
e
nvi
r
on
m
e
n
ts
i
nv
ol
v
in
g
o
nl
y
two
hos
ts
.
R
e
c
o
gni
z
i
ng
th
e
li
m
i
ta
ti
o
ns
p
os
e
d
by
P
O
M
D
P
a
l
go
r
it
h
m
s
,
in
t
e
l
li
ge
n
t
a
u
to
m
a
te
d
pe
ne
t
r
a
ti
on
te
s
t
in
g
s
ys
t
e
m
(
I
A
P
T
S
)
[
7]
is
de
ve
l
ope
d
to
a
dd
r
e
s
s
th
e
c
ha
ll
e
nge
of
a
ut
o
m
a
ti
ng
pe
n
-
te
s
t
in
g i
n
la
r
ge
ne
two
r
k
e
n
vi
r
o
nm
e
nt
s
.
D
e
s
pi
te
it
s
ut
il
it
y,
P
O
M
D
P
-
ba
s
e
d
s
ol
ve
r
s
e
nc
ount
e
r
a
c
r
it
ic
a
l
li
m
it
a
ti
on
in
s
c
a
li
ng e
f
f
ic
ie
nt
ly
.
A
s
th
e
s
ta
te
s
pa
c
e
in
c
r
e
a
s
e
s
in
s
iz
e
,
P
O
M
D
P
-
ba
s
e
d
s
ol
ve
r
s
of
te
n
b
e
c
om
e
c
om
put
a
ti
ona
ll
y
in
f
e
a
s
ib
le
,
ha
m
pe
r
in
g
th
e
ir
pr
a
c
ti
c
a
l
a
ppl
ic
a
ti
on
in
la
r
ge
-
s
c
a
le
s
c
e
na
r
io
s
.
C
ons
e
qu
e
nt
ly
,
a
n
in
c
r
e
a
s
in
g
num
be
r
of
r
e
s
e
a
r
c
he
r
s
a
r
e
opt
in
g
to
m
ode
l
th
e
pe
n
-
te
s
t
t
a
s
k
u
s
in
g
th
e
M
D
P
,
w
he
r
e
a
c
ti
on
out
c
om
e
s
a
r
e
de
te
r
m
in
is
ti
c
,
th
e
r
e
by
c
ir
c
um
ve
nt
in
g t
he
c
om
put
a
ti
ona
l
c
om
pl
e
xi
ti
e
s
a
s
s
oc
ia
te
d
w
it
h P
O
M
D
P
-
ba
s
e
d a
ppr
oa
c
he
s
.
W
hi
le
m
ode
l
-
ba
s
e
d
m
e
th
ods
ha
v
e
pr
ove
n
e
f
f
e
c
ti
ve
,
th
e
y
a
r
e
in
he
r
e
nt
ly
c
ons
tr
a
in
e
d
by
th
e
r
e
qui
r
e
m
e
nt
f
or
hum
a
n
e
xpe
r
ts
to
de
f
in
e
th
e
dyna
m
ic
s
of
th
e
m
ode
ls
.
I
n
r
e
c
e
nt
ti
m
e
s
,
th
e
r
e
ha
s
be
e
n
a
s
hi
f
t
to
w
a
r
ds
e
m
pl
oyi
ng
m
ode
l
-
f
r
e
e
R
L
a
lg
or
it
hm
s
to
a
ddr
e
s
s
pe
ne
t
r
a
ti
on
te
s
ti
ng
c
ha
ll
e
nge
s
.
U
nl
ik
e
m
ode
l
-
ba
s
e
d
a
ppr
oa
c
he
s
r
e
li
a
nt
on
e
xpe
r
t
-
de
s
ig
ne
d
m
ode
l
s
,
m
ode
l
-
f
r
e
e
a
ge
nt
s
a
ut
onomous
ly
in
te
r
a
c
t
w
it
h
th
e
e
nvi
r
onm
e
nt
t
o de
r
iv
e
opt
im
a
l
s
tr
a
te
gi
e
s
.
O
ur
r
e
s
e
a
r
c
h
a
dopt
s
a
s
im
il
a
r
m
ode
l
-
f
r
e
e
R
L
a
ppr
oa
c
h,
a
lb
e
it
w
it
h
a
pr
i
m
a
r
y
f
oc
us
on
c
r
i
ti
c
a
ll
y
a
s
s
e
s
s
in
g
bot
h
m
ode
l
-
ba
s
e
d
a
nd
m
ode
l
-
f
r
e
e
R
L
te
c
hni
que
s
a
nd
e
va
lu
a
ti
ng
th
e
ir
e
f
f
ic
a
c
y
in
a
ut
om
a
ti
ng
pe
ne
tr
a
ti
on
te
s
ti
ng
pr
oc
e
s
s
e
s
.
A
f
r
a
m
e
w
or
k
[
8]
is
pr
opos
e
d
f
or
us
in
g
R
L
to
le
a
r
n
a
tt
a
c
k
pa
th
s
,
w
hi
c
h
is
e
va
lu
a
te
d
on
a
s
im
ul
a
te
d
e
nvi
r
onm
e
nt
a
nd
s
how
e
d
th
a
t
it
c
a
n
le
a
r
n
to
f
in
d
e
f
f
e
c
ti
ve
a
tt
a
c
k
pa
th
s
m
or
e
e
f
f
ic
ie
nt
ly
th
a
n
tr
a
di
ti
ona
l
pe
ne
tr
a
ti
on
te
s
ti
ng
m
e
th
ods
.
T
h
e
n
c
om
e
s
th
e
tr
a
di
ti
ona
l
D
Q
N
a
lg
or
it
hm
.
A
lt
hough
it
in
c
or
por
a
te
s
R
L
,
it
e
a
s
il
y
ove
r
e
s
ti
m
a
te
s
th
e
Q
va
lu
e
,
w
hi
c
h
l
e
a
ds
to
in
e
f
f
e
c
ti
ve
pol
ic
y
upda
te
s
a
nd
uns
ta
bl
e
be
ha
vi
our
[
9]
.
T
he
D
D
P
G
a
lg
or
it
hm
,
a
s
pr
e
s
e
nt
e
d
in
[
10
]
,
of
f
e
r
s
a
m
ode
l
-
f
r
e
e
,
of
f
-
pol
ic
y
a
c
to
r
-
c
r
it
ic
a
ppr
oa
c
h
e
m
pl
oyi
ng
de
e
p
f
unc
ti
on
a
ppr
oxi
m
a
to
r
s
c
a
pa
bl
e
of
le
a
r
ni
ng
pol
ic
ie
s
in
hi
gh
-
di
m
e
ns
io
na
l,
c
ont
in
uous
a
c
ti
on s
pa
c
e
s
.
B
y
in
te
gr
a
ti
ng
in
s
ig
ht
s
f
r
om
th
e
s
uc
c
e
s
s
of
D
D
P
G
,
A
E
-
D
D
P
G
[
11]
a
ddr
e
s
s
e
s
th
e
c
ha
ll
e
nge
of
s
a
m
pl
e
im
ba
la
nc
e
.
A
ddi
ti
ona
ll
y,
th
e
A
E
-
D
D
P
G
a
lg
or
it
hm
in
tr
oduc
e
s
e
pi
s
odi
c
m
e
m
or
y
in
to
de
e
p
R
L
te
c
hni
que
s
f
or
c
ont
in
uous
pr
obl
e
m
s
,
le
ve
r
a
gi
ng
e
pi
s
odi
c
c
ont
r
ol
(
E
M
)
th
in
ki
ng
to
r
e
de
s
ig
n
th
e
e
xpe
r
ie
nc
e
r
e
pl
a
y
of
D
D
P
G
,
th
e
r
e
by
f
a
c
il
it
a
ti
ng
r
a
pi
d
a
c
qui
s
it
io
n
of
hi
g
h
-
r
e
w
a
r
d
pol
ic
ie
s
.
N
ot
a
bl
y,
A
E
-
D
D
P
G
is
th
e
f
ir
s
t
m
ode
l
to
in
c
or
por
a
te
e
pi
s
odi
c
m
e
m
or
y
in
to
d
eep
-
r
e
i
nf
or
c
e
m
e
nt
le
a
r
ni
ng
(
D
R
L
)
te
c
hni
que
s
f
or
c
ont
in
uous
pr
obl
e
m
s
.
I
t
a
ls
o
in
c
or
por
a
te
s
m
ul
ti
pl
e
a
ge
nt
s
[
12]
w
hi
c
h
in
te
r
a
c
t
w
it
h
th
e
e
nvi
r
onm
e
nt
a
s
ync
hr
onous
ly
in
[
13]
,
a
c
om
pr
e
he
ns
iv
e
e
xa
m
in
a
ti
on
of
D
R
L
c
ha
ll
e
nge
s
a
nd
c
or
r
e
s
ponding
s
ol
ut
io
ns
is
pr
e
s
e
nt
e
d,
pa
r
ti
c
ul
a
r
ly
f
oc
us
in
g
on
th
e
r
e
w
a
r
d
de
s
ig
n
is
s
ue
w
it
hi
n
hum
a
n
-
r
obot
c
ol
la
bor
a
ti
on
c
ont
e
xt
s
.
F
ur
th
e
r
m
or
e
, t
he
s
tu
dy e
xpl
or
e
s
pot
e
nt
ia
l
a
ve
nue
s
f
or
f
ut
ur
e
r
e
s
e
a
r
c
h w
it
hi
n t
hi
s
doma
in
.
T
he
m
ul
ti
-
di
m
e
ns
io
na
l
de
e
p
Q
-
ne
twor
k
(
M
D
D
Q
N
)
a
lg
or
it
hm
,
in
tr
oduc
e
d
by
C
he
n
[
14]
,
in
te
gr
a
te
s
a
tt
a
c
k
gr
a
phs
f
r
om
m
ul
ti
hos
t,
m
ul
ti
s
ta
ge
vul
n
e
r
a
bi
li
ty
a
na
ly
s
is
la
ngua
ge
(
M
ul
V
A
L
)
w
it
h
double
d
e
e
p
Q
-
ne
twor
k
(
D
D
Q
N
)
to
e
nha
nc
e
a
tt
a
c
k
p
a
th
pl
a
nni
ng.
F
ol
lo
w
in
g
th
is
,
M
ul
V
A
L
[
15]
a
nd
de
pt
h
-
f
ir
s
t
s
e
a
r
c
h
(
D
F
S
)
w
e
r
e
ut
il
iz
e
d
to
c
ons
tr
uc
t
a
n
a
tt
a
c
k
m
a
tr
ix
,
w
it
h
D
Q
N
e
m
pl
oye
d
to
a
na
ly
z
e
th
e
m
a
tr
ix
a
nd
id
e
nt
if
y
th
e
m
os
t
vul
ne
r
a
bl
e
a
tt
a
c
k
pa
th
f
or
th
e
ne
twor
k.
T
hi
s
id
e
nt
if
ie
d
a
tt
a
c
k
pa
th
c
oul
d
th
e
n
unde
r
go
te
s
ti
ng
us
in
g
in
dus
tr
y
-
s
ta
nda
r
d
to
ol
s
s
uc
h
a
s
M
e
ta
s
pl
oi
t
a
nd
W
ir
e
s
h
a
r
k,
a
s
out
li
ne
d
in
[
16]
.
T
o
e
lu
c
id
a
te
th
e
us
a
g
e
of
th
e
M
e
ta
s
pl
oi
t
f
r
a
m
e
w
or
k
to
ol
in
a
de
ta
il
e
d
m
a
nne
r
,
out
li
ni
ng
pr
o
c
e
dur
e
s
f
or
e
a
c
h
te
s
ti
ng
pha
s
e
a
lo
ng
w
it
h
th
e
r
e
qui
s
it
e
c
om
m
a
nds
(
s
ynt
a
x)
c
onduc
te
d
w
it
hi
n
a
K
a
li
L
in
ux
e
nvi
r
onm
e
nt
[
17]
.
A
ddi
ti
ona
ll
y,
a
n
in
nova
ti
ve
a
ppr
oa
c
h
to
a
ut
om
a
te
d
pe
ne
tr
a
ti
on
te
s
ti
ng
e
m
pl
oyi
ng
D
R
L
is
pr
e
s
e
nt
e
d
in
a
s
tu
dy
by
[
18]
.
T
hi
s
f
r
a
m
e
w
or
k
ut
il
iz
e
s
D
R
L
te
c
hni
que
s
to
id
e
nt
if
y
opt
im
a
l
a
tt
a
c
k
pa
th
s
w
it
hi
n
s
im
ul
a
te
d
ne
twor
k
e
nvi
r
onm
e
nt
s
,
le
ve
r
a
gi
ng
to
ol
s
s
uc
h
a
s
ne
twor
k
m
a
ppe
r
(
N
m
a
p
)
,
M
ul
V
A
L
,
a
nd
th
e
na
ti
ona
l
vul
ne
r
a
bi
li
ty
da
ta
ba
s
e
(
N
V
D
)
to
a
na
ly
z
e
a
tt
a
c
k
gr
a
phs
a
nd
de
te
r
m
in
e
th
e
m
os
t
e
f
f
e
c
ti
ve
pa
th
ba
s
e
d
on
c
om
m
on
vul
ne
r
a
bi
li
ty
s
c
or
in
g
s
ys
te
m
(
C
V
S
S
)
s
c
or
e
s
[
19]
.
T
hi
s
pa
pe
r
s
ig
ni
f
ic
a
nt
ly
e
nha
nc
e
s
our
und
e
r
s
ta
ndi
ng
of
ne
twor
k
a
na
ly
s
is
to
ol
s
a
nd
m
e
th
odol
ogi
e
s
.
G
iv
e
n
th
e
c
a
pa
bi
li
ti
e
s
of
th
e
s
e
a
lg
or
it
hm
s
,
o
ur
r
e
s
e
a
r
c
h
f
oc
u
s
e
s
on
c
om
pa
r
in
g
D
Q
N
a
s
a
r
e
pr
e
s
e
nt
a
ti
ve
of
m
ode
l
-
ba
s
e
d
le
a
r
ni
ng
w
it
h
D
D
P
G
a
nd
A
E
-
D
D
P
G
f
o
r
m
ode
l
-
f
r
e
e
le
a
r
ni
ng
in
th
e
c
ont
e
xt
of
a
ut
om
a
ti
ng pe
ne
tr
a
ti
on t
e
s
ti
ng
[
20]
.
3.
M
E
T
H
O
D
F
i
gu
r
e
1
s
ho
w
s
th
e
p
r
o
pos
e
d
a
r
c
hi
te
c
tu
r
a
l
f
r
a
m
e
w
o
r
k
w
hi
c
h
is
di
vi
d
e
d
i
nt
o
th
r
e
e
d
is
t
in
c
t
m
od
ul
e
s
:
i)
ne
two
r
k
s
c
a
n
ni
ng
a
nd
in
f
or
m
a
t
io
n
g
a
th
e
r
in
g
,
ii
)
RL
,
a
n
d
i
i
i)
pe
n
-
te
s
t
in
g
.
A
de
ta
i
le
d
e
xa
m
in
a
ti
on
o
f
e
a
c
h
m
od
ul
e
,
in
c
lu
di
ng
t
he
i
r
r
e
s
pe
c
ti
ve
in
pu
t
s
a
n
d
ou
tp
u
ts
,
is
p
r
o
vi
de
d
b
e
lo
w
in
o
r
d
e
r
t
o
im
p
a
r
t
a
c
o
m
p
r
e
he
ns
i
ve
un
de
r
s
ta
nd
in
g
of
th
e
ir
r
o
le
s
w
it
h
i
n
th
e
o
ve
r
a
l
l
a
r
c
h
it
e
c
tu
r
e
.
T
he
f
r
a
m
e
w
or
k
c
ol
le
c
ts
us
e
r
i
npu
t
o
n
t
he
lo
g
ic
a
l
t
a
r
ge
t
ne
t
w
o
r
k
,
i
nc
l
ud
in
g
vul
ne
r
a
b
il
it
y
i
nf
or
m
a
ti
on.
N
e
x
t,
pr
os
pe
c
ti
ve
a
tt
a
c
k
t
r
e
e
s
a
r
e
i
de
n
ti
f
i
e
d
us
in
g
t
he
M
u
lVA
L
a
t
ta
c
k
-
g
r
a
ph
ge
ne
r
a
t
o
r
a
n
d
f
e
d
in
to
th
e
R
L
e
ng
in
e
in
a
r
e
d
uc
e
d
f
o
r
m
a
t.
T
he
us
e
r
c
a
n
th
e
n
e
xa
m
in
e
how
t
he
a
t
ta
c
k
c
oul
d
be
e
xe
c
ut
e
d
on
a
n
a
c
tu
a
l
ta
r
ge
t
ne
two
r
k
by
us
i
ng
pe
ne
t
r
a
t
io
n
te
s
t
in
g t
oo
ls
li
ke
M
e
ta
s
p
lo
it
t
o
le
ve
r
a
ge
t
he
a
tt
a
c
k
pa
t
hs
pr
odu
c
e
d
by
t
he
R
L
e
n
gi
ne
in
th
e
f
r
a
m
e
w
o
r
k.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 5, O
c
to
be
r
2025
:
4061
-
4073
4064
F
ig
ur
e
1.
A
r
c
hi
te
c
tu
r
e
di
a
gr
a
m
of
t
he
pr
opos
e
d f
r
a
m
e
w
or
k
3.1.
N
e
t
w
or
k
s
c
an
n
in
g an
d
i
n
f
or
m
at
io
n
gat
h
e
r
in
g
O
ur
r
e
s
e
a
r
c
h
e
m
pl
oys
M
ul
V
A
L
[
21]
a
s
th
e
ne
twor
k
a
na
ly
z
e
r
.
M
ul
V
A
L
is
a
lo
gi
c
-
ba
s
e
d
ne
twor
k
s
e
c
ur
it
y
a
na
ly
z
e
r
th
a
t
is
a
c
c
e
s
s
ib
le
a
s
a
n
ope
n
-
s
our
c
e
to
ol
de
s
ig
ne
d
to
c
ons
tr
uc
t
a
n
a
tt
a
c
k
gr
a
ph
f
or
a
s
pe
c
if
ie
d
ne
twor
k
a
r
c
hi
te
c
tu
r
e
.
A
ll
e
xp
e
r
im
e
nt
s
in
our
s
tu
d
y
a
r
e
c
onduc
te
d
w
it
hi
n
a
s
im
ul
a
te
d
ne
twor
k
e
nvi
r
onm
e
nt
.
T
he
in
put
,
c
om
pr
is
in
g
ne
twor
k
in
f
or
m
a
ti
on,
is
e
xpr
e
s
s
e
d
in
D
a
ta
lo
g.
M
ul
V
A
L
pr
oc
e
s
s
e
s
th
is
ne
twor
k
da
ta
,
pr
oduc
in
g
out
put
s
th
a
t
in
c
lu
de
th
e
id
e
nt
i
f
ic
a
ti
on
of
vul
ne
r
a
bi
li
ti
e
s
a
nd
m
a
c
hi
ne
c
onf
ig
ur
a
ti
o
n
de
ta
il
s
pr
e
s
e
nt
e
d
in
th
e
f
or
m
of
pr
e
di
c
a
te
s
.
T
h
e
vul
ne
r
a
bi
li
ti
e
s
a
r
e
c
om
pa
r
e
d
w
it
h
th
e
c
om
m
on
vul
ne
r
a
bi
li
ti
e
s
a
nd
e
xpl
oi
ts
(
C
V
E
)
in
th
e
N
V
D
,
w
hi
c
h
is
th
e
U
.S
.
gove
r
nm
e
nt
'
s
r
e
pos
it
or
y
of
vul
ne
r
a
bi
li
ty
m
a
na
ge
m
e
nt
da
ta
ba
s
e
d
on
N
a
ti
ona
l
I
ns
ti
tu
te
of
S
ta
nda
r
ds
a
nd
T
e
c
hnol
ogy
(
N
I
S
T
)
s
ta
nda
r
ds
.
W
e
a
r
e
a
ls
o
u
s
in
g
th
e
M
e
ta
s
pl
oi
t
E
xpl
oi
t
D
a
ta
ba
s
e
,
w
hi
c
h
c
ont
a
in
s
a
li
s
t
of
vul
ne
r
a
bi
li
ti
e
s
th
a
t
h
a
ve
be
e
n
di
s
c
ov
e
r
e
d.
T
he
s
e
vul
ne
r
a
bi
li
ti
e
s
a
r
e
id
e
nt
if
ie
d
a
nd
a
na
ly
z
e
d
by
M
ul
V
A
L
th
r
ough
th
e
in
te
gr
a
ti
on
of
f
or
m
a
l
vul
ne
r
a
bi
li
ty
s
pe
c
if
ic
a
ti
ons
f
r
om
bug
-
r
e
por
ti
ng
c
om
m
uni
ti
e
s
,
hos
t
a
nd
ne
twor
k
c
onf
ig
ur
a
ti
on
in
f
or
m
a
ti
on,
a
nd
ot
he
r
r
e
le
v
a
nt
da
ta
e
nc
od
e
d
a
s
D
a
ta
lo
g
f
a
c
ts
.
M
ul
V
A
L
'
s
r
e
a
s
oni
ng
e
ngi
n
e
is
m
e
a
nt
to
s
c
a
le
e
f
f
ic
ie
nt
ly
w
it
h
ne
twor
k
s
iz
e
,
e
na
bl
in
g
e
f
f
ic
ie
nt
a
na
ly
s
is
of
ne
twor
ks
c
ont
a
in
in
g t
hous
a
nds
of
m
a
c
hi
n
e
s
.
3.2.
R
e
in
f
or
c
e
m
e
n
t
l
e
ar
n
in
g
e
n
gi
n
e
F
ol
lo
w
in
g
th
e
a
na
ly
s
is
c
onduc
te
d
by
M
ul
V
A
L
,
th
e
out
put
s
,
e
nc
om
pa
s
s
in
g
id
e
nt
if
ie
d
vul
ne
r
a
bi
li
ti
e
s
a
nd ma
c
hi
ne
c
onf
ig
ur
a
ti
on de
ta
il
s
i
n t
he
f
or
m
of
p
r
e
di
c
a
te
s
, s
e
r
ve
a
s
c
r
uc
ia
l
in
put
s
f
or
our
R
L
m
ode
l.
T
he
R
L
m
ode
l
is
ta
s
ke
d
w
it
h
de
te
r
m
in
in
g
th
e
m
os
t
vul
ne
r
a
bl
e
pa
t
hs
w
it
hi
n
th
e
ne
twor
k
to
pol
ogy,
e
m
pl
oyi
n
g
a
dva
nc
e
d
a
lg
or
it
hm
s
s
uc
h
a
s
D
Q
N
,
D
D
P
G
a
nd
A
E
-
D
D
P
G
.
T
he
s
e
a
lg
or
it
hm
s
unde
r
go
a
c
om
pr
e
he
n
s
iv
e
e
va
lu
a
ti
on
to
di
s
c
e
r
n
th
e
ir
e
f
f
e
c
ti
ve
ne
s
s
in
pr
io
r
it
i
z
in
g
a
nd
na
vi
ga
ti
ng
pot
e
nt
ia
l
vul
ne
r
a
bi
li
ti
e
s
w
it
hi
n
th
e
ne
twor
k
a
nd
e
nha
n
c
e
th
e
e
f
f
ic
ie
nc
y
of
a
ut
om
a
te
d
pe
n
e
tr
a
ti
on
te
s
ti
ng
[
22]
.
F
ig
ur
e
2
de
pi
c
t
s
th
e
out
li
ne
of
th
e
R
L
e
ngi
ne
.
F
ig
ur
e
2.
O
ut
li
ne
of
t
he
R
L
e
ngi
ne
3
.2.1
.
D
e
e
p
Q
-
n
e
t
w
or
k
D
Q
N
is
a
s
ophi
s
ti
c
a
te
d
de
e
p
R
L
m
ode
l
w
hi
c
h
a
c
hi
e
ve
d
e
x
tr
a
or
di
na
r
y
pe
r
f
or
m
a
nc
e
in
le
a
r
ni
ng
c
ont
r
ol
pol
ic
ie
s
f
r
om
hi
gh
-
di
m
e
ns
io
na
l
s
e
ns
or
y
in
put
.
T
he
tr
a
in
in
g
D
Q
N
im
pl
e
m
e
nt
s
a
va
r
ia
nt
of
th
e
Q
-
le
a
r
ni
ng
a
lg
or
it
hm
[
23]
,
w
hi
c
h
in
vol
ve
s
a
n
it
e
r
a
ti
ve
upda
te
of
ne
twor
k
w
e
ig
ht
s
us
in
g
s
to
c
ha
s
ti
c
gr
a
di
e
nt
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
D
e
s
ig
n and analys
i
s
of
r
e
in
fo
r
c
e
m
e
nt
l
e
a
r
ni
ng m
ode
ls
f
o
r
aut
om
at
e
d
pe
ne
tr
at
io
n
…
(
Sur
e
s
h
J
aganathan
)
4065
de
s
c
e
nt
.
D
Q
N
,
a
n
R
L
m
ode
l,
pos
e
s
s
om
e
di
f
f
ic
ul
ti
e
s
dur
in
g
tr
a
in
in
g
due
to
s
pa
r
s
e
a
nd
d
e
la
ye
d
r
e
w
a
r
ds
,
c
or
r
e
la
te
d
da
ta
,
a
nd
non
-
s
ta
ti
ona
r
y
di
s
tr
ib
ut
io
ns
.
T
o
ove
r
c
om
e
th
is
,
D
Q
N
a
dopt
s
a
te
c
hni
que
c
a
ll
e
d
e
xpe
r
ie
nc
e
r
e
pl
a
y,
w
he
r
e
th
e
a
ge
nt
'
s
e
xpe
r
ie
nc
e
s
,
c
on
s
is
ti
ng
of
tu
pl
e
s
of
(
s
ta
te
,
a
c
ti
on,
r
e
w
a
r
d,
a
nd
ne
xt
s
ta
te
)
,
a
r
e
s
to
r
e
d
in
a
r
e
pl
a
y
m
e
m
or
y.
R
a
ndom
uni
f
or
m
s
a
m
pl
i
ng
of
e
xpe
r
ie
nc
e
s
f
r
om
th
is
m
e
m
or
y
e
na
bl
e
s
a
m
or
e
e
f
f
ic
ie
nt
us
e
of
pa
s
t
e
xpe
r
ie
nc
e
s
,
th
e
r
e
by
s
m
oot
hi
ng
out
th
e
tr
a
in
in
g
di
s
tr
ib
ut
io
n
a
nd
im
pr
ovi
ng
da
ta
e
f
f
ic
ie
nc
y.
A
ddi
ti
ona
ll
y,
it
s
ta
bi
li
z
e
s
th
e
le
a
r
ni
ng
pr
oc
e
s
s
by
r
e
duc
in
g
th
e
c
or
r
e
la
ti
on
be
twe
e
n
c
ons
e
c
ut
iv
e
s
a
m
pl
e
s
.
A
ddi
ti
ona
ll
y,
D
Q
N
e
m
pl
oys
a
f
r
a
m
e
-
s
ki
ppi
ng
te
c
hni
q
ue
,
w
hi
c
h
r
e
duc
e
s
th
e
c
om
put
a
ti
ona
l
de
m
a
nds
by
r
e
pe
a
ti
ng
th
e
a
ge
nt
'
s
c
hos
e
n
a
c
ti
on
f
or
a
c
e
r
ta
in
num
be
r
of
f
r
a
m
e
s
.
T
hi
s
te
c
hni
que
e
na
bl
e
s
th
e
a
ge
nt
to
pr
oc
e
s
s
m
or
e
ga
m
e
f
r
a
m
e
s
w
it
hout
s
ig
ni
f
ic
a
nt
ly
in
c
r
e
a
s
in
g
th
e
c
om
put
a
ti
ona
l
c
os
t.
F
ur
th
e
r
m
or
e
,
dur
in
g
tr
a
in
in
g, D
Q
N
e
m
pl
oys
a
n e
ps
il
on
-
gr
e
e
dy poli
c
y t
o s
tr
ik
e
a
ba
l
a
nc
e
be
twe
e
n
e
xpl
or
a
ti
on a
nd e
xpl
oi
ta
ti
on.
3.2.2
.
D
e
e
p
d
e
t
e
r
m
in
is
t
ic
p
ol
ic
y gr
ad
ie
n
t
T
he
D
D
P
G
a
lg
or
it
hm
is
a
m
o
de
l
-
f
r
e
e
a
l
go
r
i
th
m
th
a
t
u
ti
li
z
e
s
th
e
de
t
e
r
m
i
ni
s
ti
c
po
li
c
y
g
r
a
di
e
nt
to
ope
r
a
te
i
n
c
o
nt
in
uo
us
a
c
ti
o
n
s
pa
c
e
s
.
T
he
a
r
c
hi
te
c
tu
r
e
is
ba
s
e
d
on
a
n
a
c
t
or
-
c
r
it
ic
f
r
a
m
e
w
o
r
k
w
i
th
a
r
e
p
la
y
bu
f
f
e
r
a
nd
ut
il
iz
e
s
a
ta
r
ge
t
ne
t
w
o
r
k
to
s
ta
b
il
iz
e
th
e
le
a
r
n
in
g
p
r
oc
e
s
s
.
D
D
P
G
e
m
pl
oys
f
o
u
r
ne
ur
a
l
ne
t
w
o
r
ks
:
a
Q
-
ne
t
w
o
r
k,
a
de
te
r
m
in
is
ti
c
po
li
c
y
ne
tw
or
k,
a
ta
r
ge
t
Q
-
ne
tw
or
k,
a
nd
a
ta
r
ge
t
p
ol
ic
y
ne
t
w
o
r
k.
T
he
in
t
e
r
a
c
t
io
n
b
e
tw
e
e
n
t
he
Q
-
ne
two
r
k
a
nd
po
l
ic
y
ne
two
r
k
is
ve
r
y
s
im
il
a
r
t
o
a
s
i
m
p
le
A
d
va
n
ta
g
e
A
c
t
or
-
C
r
i
ti
c
.
H
ow
e
ve
r
,
in
D
D
P
G
,
t
he
a
c
t
o
r
di
r
e
c
tl
y
m
a
ps
s
ta
te
s
to
a
c
ti
ons
r
a
th
e
r
th
a
n
pr
od
uc
i
ng
a
p
r
o
ba
b
il
it
y
di
s
t
r
ib
u
ti
on
a
c
r
os
s
a
d
is
c
r
e
te
a
c
t
io
n
s
pa
c
e
.
T
he
ta
r
ge
t
ne
t
w
or
ks
a
r
e
t
im
e
-
de
la
ye
d
du
pl
ic
a
t
e
s
o
f
t
he
i
r
or
ig
in
a
l
ne
two
r
ks
t
ha
t
gr
a
du
a
l
ly
f
o
ll
ow
t
he
ta
u
gh
t
ne
two
r
ks
.
T
he
us
e
o
f
ta
r
ge
t
va
lu
e
ne
two
r
ks
s
ig
ni
f
ic
a
n
tl
y
in
c
r
e
a
s
e
s
l
e
a
r
ni
n
g s
ta
b
il
it
y.
D
D
P
G
ut
il
iz
e
s
a
r
e
pl
a
y
buf
f
e
r
to
s
a
m
pl
e
e
xpe
r
ie
nc
e
s
a
nd
upda
te
th
e
ne
ur
a
l
ne
twor
k
pa
r
a
m
e
te
r
s
.
T
hr
oughout
e
a
c
h
tr
a
je
c
to
r
y
r
ol
l
-
out
,
a
ll
e
xp
e
r
ie
nc
e
tu
pl
e
s
(
s
t
a
te
,
a
c
ti
on,
r
e
w
a
r
d,
a
nd
n
e
xt
_s
ta
te
)
a
r
e
s
a
ve
d a
nd
m
a
in
ta
in
e
d
in
a
f
in
it
e
c
a
c
he
c
a
ll
e
d
a
"
r
e
pl
a
y
buf
f
e
r
."
R
a
ndom
m
in
i
-
ba
tc
he
s
of
e
xpe
r
ie
nc
e
f
r
om
th
e
r
e
pl
a
y
buf
f
e
r
a
r
e
th
e
n
s
a
m
pl
e
d
w
hi
le
th
e
va
lu
e
a
nd
pol
ic
y
ne
twor
ks
a
r
e
upda
te
d.
T
he
va
lu
e
ne
twor
k
i
s
upda
te
d
in
a
m
a
nne
r
s
im
il
a
r
to
Q
-
le
a
r
ni
ng.
T
he
ta
r
ge
t
v
a
lu
e
ne
twor
k
a
n
d
ta
r
ge
t
pol
ic
y
n
e
twor
k,
on
th
e
ot
he
r
h
a
nd,
ge
ne
r
a
te
th
e
ne
xt
-
s
ta
te
Q
-
va
lu
e
s
,
w
hi
c
h
a
r
e
th
e
n
ut
il
iz
e
d
to
m
in
im
iz
e
th
e
m
e
a
n
-
s
qua
r
e
d
lo
s
s
be
twe
e
n
th
e
upda
te
d
Q
-
va
lu
e
a
nd
th
e
or
ig
in
a
l
Q
-
va
lu
e
.
T
he
pol
ic
y
f
unc
ti
o
n
s
e
e
ks
to
m
a
xi
m
iz
e
th
e
e
xpe
c
te
d
r
e
tu
r
n
a
nd
c
om
put
e
th
e
pol
ic
y
lo
s
s
,
s
o
th
e
de
r
iv
a
ti
ve
of
th
e
obj
e
c
ti
ve
f
unc
ti
on
w
it
h
r
e
s
pe
c
t
to
th
e
pol
ic
y
pa
r
a
m
e
te
r
is
us
e
d.
E
xpl
or
a
ti
on
in
c
ont
in
uous
a
c
ti
on
s
pa
c
e
s
in
vol
ve
s
a
ddi
ng
noi
s
e
to
bot
h
th
e
a
c
ti
on
a
nd
th
e
pa
r
a
m
e
te
r
s
pa
c
e
.
T
he
noi
s
e
is
a
dd
e
d
to
th
e
a
c
to
r
pol
ic
y
to
a
ll
ow
f
or
e
xpl
or
a
ti
on
in
de
pe
nde
nt
of
th
e
le
a
r
ni
ng
pr
oc
e
dur
e
,
a
nd i
t
is
c
r
e
a
te
d us
in
g t
he
O
r
n
s
te
in
-
U
hl
e
nbe
c
k pr
oc
e
s
s
t
o of
f
e
r
t
e
m
por
a
ll
y c
or
r
e
la
te
d e
xpl
or
a
ti
on
.
3.2.3
.
A
s
yn
c
h
r
on
ou
s
e
p
is
od
ic
d
e
e
p
d
e
t
e
r
m
in
is
t
ic
p
ol
ic
y gr
ad
ie
n
t
T
he
A
E
-
D
D
P
G
a
lg
or
it
hm
is
de
v
e
lo
pe
d
f
or
c
ont
in
uous
c
ont
r
ol
in
c
om
put
a
ti
ona
ll
y
c
om
pl
e
x
e
nvi
r
onm
e
nt
s
.
T
he
a
lg
or
it
hm
c
om
pr
is
e
s
a
n
a
c
to
r
-
c
r
it
ic
duo,
w
h
e
r
e
th
e
a
c
to
r
in
te
r
a
c
ts
w
it
h
m
ul
ti
pl
e
s
to
c
ha
s
ti
c
e
nvi
r
onm
e
nt
s
s
im
ul
ta
ne
ous
ly
to
c
ol
le
c
t
da
ta
a
s
ync
hr
onous
ly
.
T
he
c
ol
le
c
te
d
da
ta
is
s
to
r
e
d
in
m
e
m
or
y
buf
f
e
r
s
f
or
e
xpe
r
ie
nc
e
r
e
pl
a
y,
w
it
h
a
f
oc
us
on
ba
l
a
nc
in
g
da
ta
g
e
ne
r
a
ti
on
a
nd
ut
il
iz
a
ti
on
to
im
pr
ove
s
a
m
pl
e
e
f
f
ic
ie
nc
y
a
nd
di
ve
r
s
it
y.
A
E
-
D
D
P
G
di
s
ti
ngui
s
he
s
it
s
e
lf
f
r
om
D
D
P
G
b
y
a
ddr
e
s
s
in
g
da
ta
in
s
uf
f
ic
ie
nc
y
a
nd
tr
a
in
in
g
in
e
f
f
ic
ie
nc
y
th
r
ough
a
n
a
s
ync
hr
onous
f
r
a
m
e
w
or
k,
e
pi
s
odi
c
c
ont
r
ol
,
a
nd
th
e
in
je
c
ti
on
of
ne
w
noi
s
e
in
th
e
a
c
ti
on
s
pa
c
e
,
w
hi
c
h
ul
ti
m
a
te
ly
le
a
ds
to
im
pr
ove
d
le
a
r
ni
ng
e
f
f
ic
ie
nc
y
a
nd
pe
r
f
or
m
a
nc
e
in
c
om
pl
e
x
e
nvi
r
onm
e
nt
s
.
T
he
a
r
c
hi
te
c
tu
r
e
of
A
E
-
D
D
P
G
in
c
lu
de
s
m
e
m
or
y
buf
f
e
r
s
f
or
e
xpe
r
ie
nc
e
r
e
pl
a
y,
w
it
h
s
e
pa
r
a
te
c
a
c
he
buf
f
e
r
s
f
or
in
di
vi
dua
l
in
te
r
a
c
ti
on
th
r
e
a
d
s
a
nd
two
m
e
m
or
y
buf
f
e
r
s
f
or
e
xpe
r
ie
nc
e
r
e
pl
a
y.
T
hi
s
a
lg
or
it
hm
in
c
or
por
a
te
s
th
e
c
on
c
e
pt
of
e
pi
s
odi
c
c
ont
r
ol
to
s
w
i
f
tl
y
a
c
qui
r
e
a
dva
nc
e
d
knowle
dge
f
r
om
hi
gh
-
r
e
w
a
r
d e
xpe
r
ie
nc
e
s
w
hi
le
a
ls
o i
nc
r
e
a
s
in
g t
he
di
ve
r
s
it
y of
s
a
m
pl
in
g pa
th
s
f
or
e
xpe
r
ie
nc
e
r
e
pl
a
y.
I
n
te
r
m
s
of
a
lg
or
it
hm
f
unc
ti
ona
li
ty
,
a
s
ync
hr
onous
in
te
r
a
c
ti
on
e
na
bl
e
s
th
e
a
c
to
r
to
c
ol
le
c
t
m
or
e
da
ta
f
or
pol
ic
y
le
a
r
ni
ng,
pa
r
ti
c
ul
a
r
ly
in
c
om
put
a
ti
ona
ll
y
c
om
pl
e
x
e
nvi
r
onm
e
nt
s
.
T
he
m
e
m
or
y
buf
f
e
r
s
s
to
r
e
tr
a
je
c
to
r
ie
s
f
or
e
xpe
r
ie
nc
e
r
e
pl
a
y,
ut
il
iz
in
g
a
nove
l
bi
o
-
in
s
pi
r
e
d
e
pi
s
odi
c
e
xpe
r
ie
n
c
e
r
e
pl
a
y
a
ppr
oa
c
h
th
a
t
a
im
s
to
ba
la
nc
e
da
ta
ge
n
e
r
a
ti
on
a
nd
ut
il
iz
a
ti
on
s
p
e
e
ds
to
pr
e
ve
nt
s
a
m
pl
e
im
ba
l
a
nc
e
a
nd
e
nha
nc
e
s
a
m
pl
e
di
ve
r
s
it
y.
I
n
a
ddi
ti
on,
a
nove
l
ty
pe
of
noi
s
e
c
a
ll
e
d
r
a
ndom
w
a
lk
noi
s
e
ha
s
b
e
e
n
de
ve
lo
pe
d
to
e
nh
a
nc
e
e
xpl
or
a
ti
on e
f
f
ic
ie
nc
y a
nd s
a
m
pl
e
di
ve
r
s
it
y i
n R
L
.
3.
3
.
P
e
n
t
e
s
t
in
g
S
ubs
e
que
nt
ly
,
to
va
li
da
te
th
e
id
e
nt
if
ie
d
vul
ne
r
a
bl
e
pa
th
s
pr
io
r
it
i
z
e
d
by
R
L
,
w
e
ut
il
i
z
e
e
s
ta
bl
is
he
d
pe
ne
tr
a
ti
on
te
s
ti
ng
to
ol
s
,
li
ke
M
e
ta
s
pl
oi
t.
M
e
ta
s
pl
oi
t
a
id
s
i
n
a
s
s
e
s
s
in
g
th
e
r
e
a
l
-
w
or
ld
e
xpl
oi
ta
bi
li
ty
of
vul
ne
r
a
bi
li
ti
e
s
[
24]
,
[
25]
.
T
hi
s
c
om
pr
e
he
ns
iv
e
va
li
da
ti
on
a
ppr
oa
c
h
e
ns
ur
e
s
th
e
pr
a
c
ti
c
a
li
ty
a
nd
r
e
li
a
bi
li
ty
of
th
e
R
L
-
dr
iv
e
n
r
e
s
ul
ts
,
of
f
e
r
in
g
a
th
or
ough
a
s
s
e
s
s
m
e
nt
of
pot
e
nt
ia
l
s
e
c
ur
it
y
w
e
a
kne
s
s
e
s
w
it
hi
n
th
e
n
e
twor
k.
B
y i
nt
e
gr
a
ti
ng t
he
s
e
r
obus
t
pe
ne
tr
a
ti
on t
e
s
ti
ng t
ool
s
i
nt
o our
va
l
id
a
ti
on pr
oc
e
s
s
, w
e
a
im
t
o r
ig
or
ous
ly
e
va
lu
a
te
th
e
e
f
f
e
c
ti
ve
ne
s
s
a
nd r
e
li
a
bi
li
ty
of
t
he
R
L
-
dr
iv
e
n r
e
s
ul
ts
, e
ns
ur
in
g t
ha
t
th
e
i
de
nt
if
ie
d vulne
r
a
bi
li
ti
e
s
a
li
gn w
it
h
r
e
a
l
-
w
or
ld
s
c
e
na
r
io
s
a
nd e
nha
n
c
in
g t
he
ove
r
a
ll
s
e
c
ur
it
y pos
tu
r
e
of
t
he
ne
twor
k.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 5, O
c
to
be
r
2025
:
4061
-
4073
4066
4.
R
E
S
U
L
T
S
A
N
D
D
I
S
C
U
S
S
I
O
N
T
he
in
it
ia
l
pha
s
e
e
nc
om
pa
s
s
e
s
th
e
de
ve
lo
pm
e
nt
of
th
e
ne
twor
k
s
c
a
nni
ng
a
nd
in
f
or
m
a
ti
on
-
ga
th
e
r
in
g
m
odul
e
,
w
hi
c
h
us
e
s
M
ul
V
A
L
to
ge
ne
r
a
te
a
tt
a
c
k
gr
a
phs
pr
e
di
c
a
te
d
on
s
im
ul
a
te
d
ne
twor
k
c
ondi
ti
on
s
.
T
h
e
s
e
gr
a
phs
s
ubs
e
qu
e
nt
ly
s
e
r
ve
a
s
i
nput
s
f
or
our
R
L
e
ngi
ne
.
4
.1.
M
u
lt
is
t
age
vu
ln
e
r
ab
il
it
y an
a
ly
s
is
l
an
gu
age
T
he
in
it
ia
l
pha
s
e
e
nc
om
pa
s
s
e
s
th
e
de
ve
lo
pm
e
nt
of
th
e
ne
twor
k
s
c
a
nni
ng
a
nd
in
f
or
m
a
ti
on
-
ga
th
e
r
in
g
m
odul
e
.
T
he
c
e
nt
r
a
l
a
im
of
th
is
pha
s
e
w
a
s
th
e
s
e
a
m
le
s
s
in
te
gr
a
ti
on
of
M
ul
V
A
L
in
to
our
p
r
oj
e
c
t
f
r
a
m
e
w
or
k.
T
hi
s
i
nt
e
gr
a
ti
on f
a
c
il
it
a
te
d t
he
ge
ne
r
a
ti
on of
a
tt
a
c
k gr
a
phs
pr
e
di
c
a
te
d on s
im
ul
a
te
d ne
twor
k c
ondi
ti
ons
, w
hi
c
h
s
ubs
e
que
nt
ly
s
e
r
ve
a
s
in
put
s
f
or
our
R
L
e
ngi
ne
.
P
r
e
r
e
qui
s
it
e
s
f
or
th
e
in
s
ta
ll
a
ti
on
of
M
ul
V
A
L
in
c
lu
de
X
S
B
,
G
r
a
phV
iz
,
a
nd
M
a
r
ia
D
B
,
a
n
ope
n
-
s
our
c
e
-
c
om
pa
ti
bl
e
ve
r
s
io
n
of
M
yS
Q
L
.
T
he
s
e
c
om
pone
nt
s
c
ons
ti
tu
te
e
s
s
e
nt
ia
l
de
pe
nd
e
nc
ie
s
f
or
M
ul
V
A
L
'
s
pr
ope
r
f
unc
ti
oni
ng.
F
ig
u
r
e
3
pr
e
s
e
nt
s
th
e
in
put
D
a
ta
lo
g
c
ode
f
ur
n
is
he
d
to
M
ul
V
A
L
f
or
ne
twor
k s
c
a
nni
ng a
nd i
nf
or
m
a
ti
on ga
th
e
r
in
g
a
nd F
ig
ur
e
4 s
how
s
i
m
pl
e
m
e
nt
a
ti
on of
M
ul
V
a
l.
F
ig
ur
e
3.
D
a
ta
lo
g c
ode
f
or
a
3
-
hos
t
ne
twor
k t
o be
f
e
d t
o M
ul
V
A
L
F
ig
ur
e
4.
I
m
pl
e
m
e
nt
a
ti
on of
M
ul
V
a
l
F
ir
s
tl
y,
w
e
e
s
ta
bl
is
h
th
e
lo
c
a
ti
on
of
th
e
a
tt
a
c
ke
r
,
id
e
nt
if
yi
ng
th
e
m
a
s
s
it
ua
te
d
w
it
hi
n
th
e
"
in
te
r
ne
t"
e
nt
it
y.
A
ddi
ti
ona
ll
y,
w
e
s
pe
c
if
y
th
e
a
tt
a
c
k
goa
l
of
th
e
a
tt
a
c
ke
r
,
in
di
c
a
ti
ng
th
e
ir
in
te
nt
to
e
xe
c
ut
e
c
ode
on
th
e
"
w
or
kS
ta
ti
on"
e
nt
i
ty
. N
e
xt
, w
e
de
f
in
e
r
ul
e
s
r
e
ga
r
di
ng
ne
twor
k
a
c
c
e
s
s
, de
not
e
d by the
"
ha
c
k
"
pr
e
di
c
a
te
. T
he
s
e
r
ul
e
s
pe
r
m
it
a
c
c
e
s
s
be
twe
e
n
va
r
io
us
e
nt
it
ie
s
w
it
hi
n
th
e
ne
twor
k,
s
uc
h
a
s
th
e
"
w
e
bS
e
r
ve
r
,"
"
f
il
e
S
e
r
ve
r
,"
a
nd
"
w
or
k
s
ta
ti
on."
T
he
c
onf
ig
ur
a
ti
on
in
f
o
r
m
a
ti
on
f
o
r
ne
twor
k
e
nt
i
ti
e
s
,
in
c
lu
di
ng
th
e
"
f
il
e
S
e
r
ve
r
,
"
"
w
e
bS
e
r
ve
r
,
"
a
nd
"
w
or
kS
ta
ti
on,"
is
th
e
n
de
ta
il
e
d.
T
hi
s
in
f
or
m
a
ti
on
e
nc
om
pa
s
s
e
s
ne
twor
k
s
e
r
vi
c
e
de
ta
il
s
,
vul
ne
r
a
bi
li
ty
id
e
nt
if
ie
r
s
,
a
nd
ne
twor
k
f
il
e
s
ys
te
m
(
N
F
S
)
c
onf
ig
ur
a
ti
ons
.
F
o
r
i
ns
ta
nc
e
,
th
e
c
onf
ig
ur
a
ti
on
in
f
or
m
a
ti
on
f
o
r
th
e
"
f
il
e
S
e
r
ve
r
"
e
nt
it
y
s
pe
c
if
ie
s
th
a
t
it
r
uns
th
e
"
m
ount
d"
s
e
r
vi
c
e
o
ve
r
th
e
r
e
m
ot
e
pr
oc
e
dur
e
c
a
ll
(
R
P
C
)
p
r
ot
oc
ol
on
por
t
100005,
g
r
a
nt
in
g
r
oo
t
pr
iv
i
le
ge
s
.
F
ur
th
e
r
m
or
e
,
N
F
S
e
xpor
t
in
f
o
r
m
a
ti
on
in
di
c
a
te
s
th
a
t
th
e
"
/e
xpor
t"
di
r
e
c
to
r
y on the
f
il
e
s
e
r
ve
r
i
s
a
c
c
e
s
s
ib
le
w
it
h a
ny pe
r
m
is
s
io
ns
b
y t
he
"
w
or
kS
ta
ti
on."
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
D
e
s
ig
n and analys
i
s
of
r
e
in
fo
r
c
e
m
e
nt
l
e
a
r
ni
ng m
ode
ls
f
o
r
aut
om
at
e
d
pe
ne
tr
at
io
n
…
(
Sur
e
s
h
J
aganathan
)
4067
V
ul
ne
r
a
bi
li
ty
in
f
or
m
a
ti
on
is
a
ls
o
pr
ovi
de
d,
id
e
nt
if
yi
ng
vul
ne
r
a
bi
li
ti
e
s
a
s
s
oc
i
a
te
d
w
it
h
th
e
"
f
il
e
S
e
r
ve
r
"
a
nd
"
w
e
bS
e
r
ve
r
"
e
nt
it
ie
s
.
P
r
ope
r
ti
e
s
of
th
e
s
e
vu
ln
e
r
a
bi
li
ti
e
s
,
s
uc
h
a
s
th
e
ir
e
xpl
oi
ta
bi
li
ty
a
nd
pot
e
nt
ia
l
im
pa
c
t,
a
r
e
out
li
ne
d
w
it
hi
n
th
e
c
od
e
.
T
hi
s
c
od
e
is
no
w
f
ur
ni
s
he
d
a
s
in
put
to
M
ul
V
A
L
.
B
y
de
f
a
ul
t,
M
ul
V
A
L
out
put
s
th
e
r
e
s
ul
ti
ng
a
tt
a
c
k
gr
a
ph
in
bot
h
t
e
xt
ua
l
(
A
tt
a
c
kG
r
a
ph.t
xt
)
a
nd
X
M
L
(
A
tt
a
c
kG
r
a
ph.xml
)
f
or
m
a
ts
,
w
it
h
th
e
in
te
nde
d
s
e
m
a
nt
ic
s
be
in
g
s
e
lf
-
e
vi
de
nt
.
F
ur
th
e
r
m
or
e
,
th
e
in
voc
a
ti
on
o
f
th
e
-
v
opt
io
n
f
a
c
il
it
a
te
s
th
e
ge
n
e
r
a
ti
on
of
a
vi
s
ua
l
r
e
pr
e
s
e
nt
a
ti
on
of
th
e
a
tt
a
c
k
gr
a
ph
in
P
D
F
f
or
m
a
t
(
A
tt
a
c
kG
r
a
ph.pdf
)
th
r
ough Gr
a
phV
iz
.
U
pon
s
pe
c
if
ic
a
ti
on
of
th
e
a
ppr
opr
ia
te
opt
io
ns
,
M
ul
V
A
L
e
xt
e
nds
it
s
out
put
c
a
p
a
bi
li
ti
e
s
to
in
c
lu
de
a
tt
a
c
k
-
gr
a
ph
in
f
or
m
a
ti
on
in
C
S
V
f
or
m
a
t,
c
om
pr
is
in
g
V
E
R
T
I
C
E
S
.C
S
V
a
nd
A
R
C
S
.C
S
V
f
il
e
s
.
T
he
s
e
C
S
V
f
il
e
s
a
r
e
in
s
tr
um
e
nt
a
l
f
or
s
ubs
e
que
nt
r
e
nde
r
in
g
pr
ogr
a
m
s
to
g
e
ne
r
a
te
di
ve
r
s
e
vi
e
w
s
of
th
e
a
tt
a
c
k
gr
a
ph
a
t
a
la
te
r
s
ta
ge
.
W
e
ha
ve
a
l
s
o
ge
ne
r
a
te
d
th
e
a
tt
a
c
k
gr
a
ph
f
or
a
not
he
r
ne
twor
k
w
he
r
e
th
e
r
e
a
r
e
two
m
or
e
w
e
b
s
e
r
ve
r
s
a
s
s
oc
ia
t
e
d
w
it
h
th
e
"
ht
tp
d"
s
e
r
vi
c
e
,
m
a
r
ke
d
a
s
a
lo
c
a
l
e
xpl
oi
t
w
it
h
th
e
pot
e
nt
ia
l
im
pa
c
t
of
a
de
ni
a
l
of
s
e
r
vi
c
e
(
D
o
S
)
a
nd
"
w
e
bS
e
r
ve
r
3"
ha
s
a
vul
ne
r
a
bi
li
ty
w
it
h
th
e
I
D
'
V
U
L
N
-
ID
-
3'
a
s
s
oc
ia
te
d
w
it
h
th
e
"
f
tp
d"
s
e
r
vi
c
e
,
m
a
r
ke
d
a
s
a
r
e
m
ot
e
e
xpl
oi
t
w
it
h
th
e
pot
e
nt
ia
l
im
pa
c
t
of
una
ut
hor
i
z
e
d
a
c
c
e
s
s
r
e
s
pe
c
ti
v
e
ly
.
F
ig
ur
e
5
s
how
s
t
he
out
put
of
M
ul
V
A
L
a
nd t
he
ge
n
e
r
a
te
d a
tt
a
c
k gr
a
ph
.
F
ig
ur
e
5. E
xe
c
ut
io
n of
M
ul
V
A
L
on 3
-
hos
t
in
put
a
nd t
e
r
m
in
a
l
o
ut
put
4
.2.
R
e
in
f
or
c
e
m
e
n
t
l
e
ar
n
in
g
e
n
gi
n
e
T
he
a
tt
a
c
k
gr
a
phs
ge
ne
r
a
te
d
by
M
ul
V
A
L
s
e
r
ve
a
s
c
r
uc
ia
l
in
put
s
f
or
th
r
e
e
R
L
e
ngi
ne
m
ode
ls
:
i)
DQN
,
ii
)
D
D
P
G
,
a
nd
ii
i)
A
E
-
D
E
P
G
.
T
he
s
e
gr
a
phs
,
e
nc
a
p
s
ul
a
ti
ng
pot
e
nt
ia
l
a
tt
a
c
k
ve
c
to
r
s
a
nd
n
e
twor
k
vul
ne
r
a
bi
li
ti
e
s
,
gui
de
th
e
m
ode
l
’
s
e
xpl
or
a
ti
on
pr
oc
e
s
s
.
B
y
a
na
ly
z
in
g
th
e
a
tt
a
c
k
gr
a
ph,
th
e
m
ode
ls
c
a
n
l
e
a
r
n
th
e
r
e
la
ti
ons
hi
ps
b
e
twe
e
n
n
e
twor
k
e
nt
it
ie
s
,
vul
ne
r
a
bi
li
ti
e
s
,
a
nd
pot
e
nt
ia
l
e
xpl
oi
ts
.
U
s
in
g
th
e
le
a
r
ne
d
de
ta
il
s
,
th
e
m
ode
l
c
a
n
id
e
nt
if
y
a
nd
pr
io
r
it
iz
e
th
e
m
os
t
vul
ne
r
a
bl
e
pa
th
w
it
hi
n
th
e
ne
twor
k,
ul
ti
m
a
te
ly
le
a
di
ng
to
th
e
m
os
t
e
f
f
e
c
ti
ve
a
tt
a
c
k
s
tr
a
te
gy
f
or
th
e
s
im
ul
a
te
d
pe
ne
tr
a
ti
on
te
s
t
in
g
s
c
e
na
r
io
.
T
h
e
m
os
t
v
ul
ne
r
a
bl
e
pa
th
in
th
e
a
tt
a
c
k gr
a
ph i
s
pr
oduc
e
d a
s
out
put
r
e
f
e
r
F
ig
ur
e
s
6
to
8.
4.3. P
e
r
f
or
m
an
c
e
m
e
t
r
ic
s
T
he
s
e
a
r
e
th
e
pe
r
f
or
m
a
nc
e
m
e
tr
ic
s
us
e
d
to
e
va
lu
a
te
a
nd
c
om
pa
r
e
th
e
pe
r
f
or
m
a
nc
e
of
th
e
th
r
e
e
a
lg
or
it
hm
s
:
‒
A
ve
r
a
ge
r
e
w
a
r
d
pe
r
e
pi
s
ode
:
th
is
m
e
tr
ic
m
e
a
s
ur
e
s
th
e
a
ve
r
a
ge
r
e
w
a
r
d
obt
a
in
e
d
by
th
e
RL
a
lg
or
i
th
m
i
n
e
a
c
h
e
pi
s
od
e
of
tr
a
in
in
g.
A
hi
ghe
r
a
ve
r
a
ge
r
e
w
a
r
d
in
di
c
a
te
s
t
ha
t
th
e
a
lg
or
it
hm
is
pe
r
f
or
m
in
g
be
tt
e
r
a
t
a
c
hi
e
vi
ng i
ts
obj
e
c
ti
ve
s
.
‒
T
r
a
in
in
g
ti
m
e
(
in
s
e
c
onds
)
:
th
is
m
e
tr
ic
m
e
a
s
ur
e
s
th
e
ti
m
e
ta
k
e
n
f
or
th
e
a
lg
or
it
hm
to
c
om
pl
e
te
tr
a
in
in
g.
A
s
hor
te
r
tr
a
in
in
g
ti
m
e
is
de
s
ir
a
bl
e
a
s
it
in
di
c
a
t
e
s
th
a
t
th
e
a
lg
or
it
hm
c
a
n
le
a
r
n
a
nd
a
d
a
pt
m
or
e
qui
c
kl
y
to
th
e
e
nvi
r
onm
e
nt
.
‒
C
onve
r
ge
nc
e
s
pe
e
d
(
in
e
pi
s
ode
s
)
:
c
onve
r
ge
nc
e
s
pe
e
d
r
e
f
e
r
s
to
th
e
num
be
r
of
e
pi
s
ode
s
it
ta
ke
s
f
or
th
e
a
lg
or
it
hm
to
c
onve
r
ge
to
a
s
ta
bl
e
pol
ic
y.
A
lo
w
e
r
c
onve
r
ge
nc
e
s
pe
e
d
in
di
c
a
te
s
th
a
t
th
e
a
lg
or
it
hm
le
a
r
ns
m
or
e
qui
c
kl
y a
nd e
f
f
ic
ie
nt
ly
.
‒
S
a
m
pl
e
e
f
f
ic
ie
nc
y:
s
a
m
pl
e
e
f
f
ic
ie
nc
y
m
e
a
s
ur
e
s
how
w
e
ll
th
e
a
lg
or
it
hm
ut
il
iz
e
s
th
e
a
va
il
a
bl
e
tr
a
in
in
g
da
ta
.
A
hi
ghe
r
s
a
m
pl
e
e
f
f
ic
ie
nc
y
in
di
c
a
te
s
th
a
t
th
e
a
lg
or
it
hm
c
a
n
a
c
hi
e
ve
good
pe
r
f
or
m
a
nc
e
w
it
h
f
e
w
e
r
tr
a
in
in
g s
a
m
pl
e
s
.
‒
A
ve
r
a
ge
pa
th
le
ngt
h
(
A
P
L
)
:
in
th
e
c
ont
e
xt
of
pe
ne
tr
a
ti
on
te
s
ti
ng,
A
P
L
r
e
f
e
r
s
to
th
e
a
ve
r
a
ge
num
be
r
of
s
te
ps
or
a
c
ti
on
s
ta
ke
n
by
th
e
a
lg
or
it
hm
to
r
e
a
c
h
th
e
ta
r
ge
t
(
e
.g.,
id
e
nt
if
yi
ng
a
nd
pr
io
r
it
iz
in
g
vul
ne
r
a
bi
li
ti
e
s
)
. A
s
hor
te
r
A
P
L
in
di
c
a
te
s
t
ha
t
th
e
a
lg
or
it
hm
i
s
a
bl
e
t
o f
in
d m
or
e
e
f
f
ic
ie
nt
s
ol
ut
io
ns
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 5, O
c
to
be
r
2025
:
4061
-
4073
4068
F
ig
ur
e
6
.
E
xe
c
ut
io
n a
nd t
e
r
m
in
a
l
out
put
of
DQN
a
lg
or
it
hm
F
ig
ur
e
7. E
xe
c
ut
io
n a
nd t
e
r
m
in
a
l
out
put
f
o
r
D
D
P
G
a
lg
o
r
it
hm
F
ig
ur
e
8. E
xe
c
ut
io
n a
nd t
e
r
m
in
a
l
out
put
f
o
r
A
E
-
D
D
P
G
a
lg
or
it
h
m
4.3.1
.
E
p
s
il
on
d
e
la
y
A
n
e
ps
il
on
de
la
y
gr
a
ph
(ε
-
de
la
y
gr
a
ph)
F
ig
ur
e
9
c
a
n
pr
o
vi
de
in
s
ig
ht
s
in
to
th
e
e
xpl
or
a
ti
on
-
e
xpl
oi
ta
ti
on
tr
a
de
-
of
f
dur
in
g
th
e
tr
a
in
in
g
p
r
oc
e
s
s
.
A
lo
g
-
li
ke
de
c
r
e
a
s
in
g
ε
-
de
la
y
gr
a
ph
of
D
Q
N
a
nd
D
D
P
G
in
di
c
a
te
s
th
a
t
th
e
R
L
m
ode
l
m
ig
ht
be
pr
io
r
it
iz
in
g
e
xpl
oi
ta
ti
on
ove
r
e
xpl
or
a
ti
on
to
o
he
a
vi
ly
.
A
s
tr
a
ig
ht
-
li
ne
dow
nw
a
r
d
tr
e
nd
in
th
e
ε
-
de
la
y
gr
a
ph
s
ugge
s
ts
th
e
R
L
m
ode
l
is
pr
io
r
it
iz
in
g
e
xpl
oi
ta
ti
on
ove
r
e
xpl
o
r
a
ti
on
to
o
he
a
vi
ly
.
O
ve
r
a
ll
,
th
e
s
tr
a
ig
ht
-
li
ne
dow
nw
a
r
d
gr
a
ph
s
ugg
e
s
ts
t
ha
t
th
e
A
E
-
D
D
P
G
m
ode
l
s
uf
f
e
r
s
f
r
om
s
e
ve
r
e
ove
r
-
e
xpl
oi
ta
ti
on,
pot
e
nt
ia
ll
y
m
is
s
in
g
be
tt
e
r
a
tt
a
c
k
pa
th
s
.
W
hi
l
e
lo
g
-
li
ke
gr
a
phs
f
or
D
Q
N
a
nd
D
D
P
G
m
ode
ls
in
di
c
a
te
a
pot
e
nt
ia
l
e
xpl
or
a
ti
on
-
e
xpl
oi
ta
ti
on
im
ba
la
nc
e
,
th
e
y
m
ig
ht
ha
ve
e
xpl
or
e
d
m
or
e
th
a
n
A
E
-
D
D
P
G
,
w
hi
c
h m
a
ke
s
t
he
m
pot
e
nt
ia
ll
y be
tt
e
r
c
a
ndi
da
te
s
f
or
f
in
di
ng
vul
ne
r
a
bi
li
ti
e
s
due
t
o s
om
e
l
e
ve
l
of
e
xpl
or
a
ti
on.
I
t
is
di
f
f
ic
ul
t
to
de
f
in
i
ti
ve
ly
s
a
y
w
hi
c
h
m
ode
l
is
th
e
be
s
t
pe
r
f
or
m
e
r
w
it
hout
c
ons
id
e
r
in
g
ot
he
r
f
a
c
to
r
s
a
lo
ngs
id
e
th
e
ε
-
de
la
y gr
a
phs
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
D
e
s
ig
n and analys
i
s
of
r
e
in
fo
r
c
e
m
e
nt
l
e
a
r
ni
ng m
ode
ls
f
o
r
aut
om
at
e
d
pe
ne
tr
at
io
n
…
(
Sur
e
s
h
J
aganathan
)
4069
F
ig
ur
e
9. E
xpl
or
a
ti
on vs
e
xpl
oi
ta
ti
on c
ur
ve
f
or
D
Q
N
, D
D
P
G
,
a
nd A
E
-
D
D
P
G
4.3.2
.
A
ve
r
age
p
at
h
l
e
n
gt
h
A
n
A
P
L
gr
a
ph
in
th
e
c
ont
e
xt
of
R
L
m
ode
ls
p
e
r
f
or
m
in
g
pe
ne
tr
a
ti
on
te
s
ti
ng
pr
ovi
de
s
in
s
ig
ht
s
in
to
th
e
e
f
f
ic
ie
nc
y
of
th
e
m
ode
l
in
f
in
di
ng
vul
ne
r
a
bi
li
ti
e
s
w
it
hi
n
th
e
a
tt
a
c
k
gr
a
ph.
I
t
i
s
th
e
num
be
r
of
s
t
e
ps
th
e
m
ode
l
ta
ke
s
pe
r
e
pi
s
ode
,
in
ot
he
r
w
or
ds
,
it
is
th
e
num
be
r
of
a
c
ti
ons
of
th
e
a
ge
nt
in
R
L
.
I
t
doe
s
not
de
pe
nd
on
th
e
le
ngt
h of
t
he
pa
th
t
ha
t
c
ont
a
in
s
vul
ne
r
a
bi
li
ti
e
s
, w
hi
c
h w
e
r
e
f
e
r
t
o a
s
t
he
'
vul
ne
r
a
bl
e
pa
th
'
.
A
m
ode
l
th
a
t
c
on
s
is
te
nt
ly
d
e
m
ons
tr
a
te
s
a
lo
w
e
r
A
P
L
th
r
ou
ghout
tr
a
in
in
g
e
pi
s
ode
s
hi
nt
s
a
t
it
s
s
upe
r
io
r
e
f
f
ic
ie
nc
y
in
na
vi
ga
ti
ng
th
e
a
tt
a
c
k
gr
a
ph.
S
uc
h
a
m
od
e
l
r
e
qui
r
e
s
f
e
w
e
r
s
t
e
ps
(
a
c
ti
on
s
)
on
a
ve
r
a
ge
to
de
te
c
t
a
nd
e
xpl
oi
t
vul
ne
r
a
bi
li
ti
e
s
,
in
di
c
a
ti
ng
a
m
or
e
di
r
e
c
t
a
nd
e
f
f
e
c
ti
ve
a
tt
a
c
k
s
tr
a
te
gy.
C
onve
r
s
e
ly
,
a
m
ode
l
w
it
h
a
c
on
s
is
te
nt
ly
hi
ghe
r
A
P
L
m
a
y
be
le
s
s
e
f
f
ic
ie
nt
,
a
s
it
n
e
c
e
s
s
it
a
t
e
s
m
or
e
s
te
p
s
(
a
c
ti
ons
)
on
a
ve
r
a
g
e
to
r
e
a
c
h vulne
r
a
bi
li
ti
e
s
.
T
he
gr
a
ph
F
ig
ur
e
10,
w
it
h
th
e
i
r
di
s
ti
nc
ti
ve
pe
a
ks
a
nd
tr
oughs
,
pr
ovi
de
a
vi
s
ua
l
de
pi
c
ti
on
of
th
e
pa
t
h
le
ngt
h
f
or
e
a
c
h
tr
a
in
in
g
e
pi
s
ode
.
T
he
pe
a
k
s
s
ig
ni
f
y
e
pi
s
ode
s
w
he
r
e
th
e
m
ode
l
to
ok
a
lo
nge
r
pa
th
to
id
e
nt
if
y
a
vul
ne
r
a
bi
li
ty
,
w
hi
le
th
e
tr
oughs
r
e
pr
e
s
e
nt
e
pi
s
ode
s
w
it
h
s
hor
te
r
pa
th
s
.
T
he
dot
te
d
li
ne
in
th
e
gr
a
ph
r
e
pr
e
s
e
nt
s
th
e
A
P
L
.
F
r
om
th
e
s
e
gr
a
phs
,
w
e
c
a
n
de
duc
e
th
a
t
D
Q
N
b
oa
s
ts
th
e
lo
w
e
s
t
A
P
L
,
a
ppr
oxi
m
a
te
ly
9.610,
in
di
c
a
ti
ng
it
s
s
up
e
r
io
r
e
f
f
ic
ie
nc
y
in
na
vi
ga
ti
ng
th
e
a
tt
a
c
k
gr
a
p
h.
O
n
th
e
ot
he
r
h
a
nd,
A
E
-
D
D
P
G
e
xhi
bi
ts
th
e
hi
ghe
s
t
A
P
L
of
54.864, s
ugge
s
ti
ng i
t
is
t
he
l
e
a
s
t
e
f
f
ic
ie
nt
m
ode
l,
f
ol
lo
w
e
d by DD
P
G
w
it
h a
n
A
P
L
of
34.735.
4.3.3.
A
ve
r
age
r
e
w
a
r
d
T
he
a
ve
r
a
ge
r
e
w
a
r
d
pe
r
e
pi
s
ode
gr
a
ph
in
th
e
c
ont
e
xt
of
R
L
m
ode
ls
pe
r
f
or
m
in
g
pe
ne
tr
a
ti
on
te
s
ti
ng
pr
ovi
de
s
in
s
ig
ht
s
in
to
th
e
e
f
f
e
c
ti
ve
ne
s
s
of
th
e
m
ode
l
in
id
e
nt
if
yi
ng
a
nd
pr
io
r
it
iz
in
g
vul
ne
r
a
bi
li
ti
e
s
.
A
m
ode
l
w
it
h
a
c
ons
is
te
nt
ly
hi
ghe
r
a
ve
r
a
ge
r
e
w
a
r
d
a
c
r
os
s
tr
a
in
in
g
e
pi
s
ode
s
s
ugge
s
t
s
it
is
m
or
e
e
f
f
e
c
ti
ve
a
t
a
c
hi
e
vi
ng
th
e
obj
e
c
ti
ve
of
th
e
pe
ne
tr
a
ti
on
te
s
ti
ng
s
c
e
na
r
io
.
T
hi
s
m
e
a
ns
th
e
m
ode
l
is
s
uc
c
e
s
s
f
ul
ly
id
e
nt
if
yi
ng
a
nd
e
xpl
oi
ti
ng
vul
ne
r
a
bi
li
ti
e
s
th
a
t
le
a
d
to
hi
gh
e
r
r
e
w
a
r
ds
.
A
m
od
e
l
w
it
h
a
c
on
s
is
te
nt
ly
lo
w
e
r
a
v
e
r
a
ge
r
e
w
a
r
d
m
ig
ht
be
l
e
s
s
e
f
f
e
c
ti
ve
.
T
he
gr
a
ph
F
ig
ur
e
11
w
i
th
to
ta
l
r
e
w
a
r
d
on
th
e
Y
-
a
xi
s
a
nd
e
pi
s
ode
on
th
e
X
-
a
xi
s
s
how
s
th
e
c
um
ul
a
ti
ve
r
e
w
a
r
d
a
c
hi
e
ve
d
by
th
e
R
L
m
ode
l
th
r
oughout
t
r
a
in
in
g.
H
ow
e
ve
r
,
it
doe
s
not
di
r
e
c
tl
y
di
s
pl
a
y
th
e
a
ve
r
a
ge
r
e
w
a
r
d
pe
r
e
pi
s
od
e
.
I
n
th
is
e
xpe
r
im
e
nt
,
w
e
m
e
ti
c
ul
ous
ly
c
a
lc
ul
a
te
d
th
e
a
ve
r
a
ge
r
e
w
a
r
d
f
r
om
th
is
gr
a
ph
by
di
vi
di
ng
th
e
X
-
a
xi
s
(
num
be
r
of
e
pi
s
ode
s
)
in
to
e
qua
l
in
te
r
va
ls
.
F
or
e
a
c
h
in
te
r
va
l,
w
e
e
s
ti
m
a
te
d
th
e
a
ve
r
a
ge
s
lo
pe
of
t
he
c
um
ul
a
ti
ve
r
e
w
a
r
d l
in
e
, a
pr
oc
e
s
s
t
ha
t
r
e
qui
r
e
s
c
a
r
e
f
ul
a
tt
e
nt
io
n t
o de
ta
il
. A
s
te
e
pe
r
s
lo
pe
in
di
c
a
te
s
a
hi
ghe
r
a
ve
r
a
ge
r
e
w
a
r
d
f
or
e
pi
s
ode
s
w
it
hi
n
th
a
t
in
t
e
r
va
l.
A
f
te
r
c
a
lc
ul
a
ti
ng
a
n
e
s
ti
m
a
te
d
a
ve
r
a
ge
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
,
V
ol
. 14, No. 5, O
c
to
be
r
2025
:
4061
-
4073
4070
r
e
w
a
r
d
f
or
e
a
c
h
in
te
r
va
l,
w
e
s
um
it
a
nd
di
vi
de
it
by
th
e
to
ta
l
num
be
r
of
in
te
r
va
ls
to
c
a
lc
ul
a
te
th
e
ove
r
a
l
l
a
ve
r
a
ge
r
e
w
a
r
d, e
ns
ur
in
g t
he
pr
e
c
is
io
n
a
nd a
c
c
ur
a
c
y of
our
a
na
ly
s
is
.
F
ig
ur
e
10
.
A
P
L
pe
r
e
pi
s
ode
f
or
D
Q
N
, D
D
P
G
,
a
nd
AE
-
D
D
P
G
F
ig
ur
e
11. Ave
r
a
ge
r
e
w
a
r
d pe
r
e
pi
s
ode
f
or
D
Q
N
, D
D
P
G
, a
nd
AE
-
D
D
P
G
Evaluation Warning : The document was created with Spire.PDF for Python.