I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
15
,
No
.
2
,
A
p
r
il
20
25
,
p
p
.
2
0
4
2
~
2
0
5
4
I
SS
N:
2088
-
8
7
0
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijece.
v
15
i
2
.
pp
2
0
4
2
-
2
0
5
4
2042
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
Ta
rg
et
ima
g
e va
lida
tion mo
deling
using
deep
n
eura
l
net
wo
rk
a
lg
o
rithm
Na
em
a
h M
ub
a
ra
k
a
h
1,
2
,
P
o
lt
a
k
Sih
o
m
bin
g
3
,
Sy
a
hril E
f
endi
3
,
F
a
hm
i
2
1
D
o
c
t
o
r
a
l
P
r
o
g
r
a
m i
n
C
o
mp
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
a
n
d
I
n
f
o
r
mat
i
o
n
T
e
c
h
n
o
l
o
g
y
,
U
n
i
v
e
r
si
t
a
s
S
u
ma
t
e
r
a
U
t
a
r
a
,
M
e
d
a
n
,
I
n
d
o
n
e
s
i
a
2
D
e
p
a
r
t
me
n
t
o
f
El
e
c
t
r
i
c
a
l
En
g
i
n
e
e
r
i
n
g
,
F
a
c
u
l
t
y
o
f
E
n
g
i
n
e
e
r
i
n
g
,
U
n
i
v
e
r
si
t
a
s
S
u
ma
t
e
r
a
U
t
a
r
a
,
M
e
d
a
n
,
I
n
d
o
n
e
si
a
3
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
a
n
d
I
n
f
o
r
mat
i
o
n
T
e
c
h
n
o
l
o
g
y
,
U
n
i
v
e
r
s
i
t
a
s S
u
m
a
t
e
r
a
U
t
a
r
a
,
M
e
d
a
n
,
I
n
d
o
n
e
s
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
u
l 1
,
2
0
2
4
R
ev
is
ed
No
v
2
4
,
2
0
2
4
Acc
ep
ted
Dec
2
,
2
0
2
4
Re
se
a
rc
h
o
n
ima
g
e
v
a
li
d
a
ti
o
n
m
o
d
e
ls
is
a
n
in
tere
stin
g
t
o
p
ic.
Th
e
a
p
p
li
c
a
ti
o
n
o
f
d
e
e
p
lea
rn
in
g
(DL)
f
o
r
o
b
j
e
c
t
d
e
tec
ti
o
n
h
a
s
b
e
e
n
d
e
m
o
n
stra
ted
to
e
ffe
c
ti
v
e
ly
a
n
d
e
fficie
n
tl
y
a
d
d
re
s
s
th
e
c
h
a
ll
e
n
g
e
s
in
t
h
is
field
.
De
e
p
n
e
u
ra
l
n
e
two
rk
s
(DN
N)
a
re
d
e
e
p
lea
rn
in
g
a
lg
o
rit
h
m
s
c
a
p
a
b
le
o
f
h
a
n
d
l
in
g
larg
e
d
a
tas
e
ts
a
n
d
e
ffe
c
ti
v
e
l
y
so
lv
i
n
g
c
o
m
p
lex
p
r
o
b
lem
s
d
u
e
t
o
th
e
ir
ro
b
u
st
lea
rn
in
g
c
a
p
a
c
it
y
.
De
sp
it
e
th
e
ir
a
b
il
it
y
to
a
d
d
re
ss
c
o
m
p
lex
p
ro
b
le
m
s,
DN
N
e
n
c
o
u
n
ter
c
h
a
ll
e
n
g
e
s
re
late
d
to
t
h
e
n
e
c
e
ss
it
y
fo
r
i
n
tri
c
a
te
a
rc
h
it
e
c
tu
re
s
a
n
d
a
larg
e
n
u
m
b
e
r
o
f
h
i
d
d
e
n
lay
e
rs.
T
h
e
o
b
jec
ti
v
e
o
f
t
h
is
re
se
a
rc
h
is
t
o
i
d
e
n
ti
f
y
th
e
m
o
st
e
ffe
c
ti
v
e
m
o
d
e
l
fo
r
a
c
h
iev
in
g
o
p
ti
m
a
l
p
e
rfo
rm
a
n
c
e
in
ima
g
e
v
a
li
d
a
ti
o
n
.
T
h
is
stu
d
y
i
n
v
e
stig
a
tes
targ
e
t
ima
g
e
v
a
li
d
a
ti
o
n
u
s
in
g
DN
N
a
lg
o
rit
h
m
s,
e
x
a
m
in
i
n
g
a
rc
h
i
tec
tu
re
s
with
3
,
4
,
5
,
a
n
d
6
h
i
d
d
e
n
lay
e
rs.
Th
is
stu
d
y
a
lso
e
v
a
lu
a
tes
th
e
p
e
rfo
r
m
a
n
c
e
o
f
ima
g
e
v
a
li
d
a
ti
o
n
a
c
ro
ss
v
a
rio
u
s
a
c
ti
v
a
ti
o
n
f
u
n
c
ti
o
n
s,
b
a
tc
h
siz
e
s,
a
n
d
n
u
m
b
e
rs
o
f
n
e
u
ro
n
s.
T
h
e
re
su
lt
s
o
f
th
e
stu
d
y
s
h
o
w
t
h
a
t
th
e
b
e
st
p
e
rfo
rm
a
n
c
e
fo
r
ima
g
e
v
a
li
d
a
ti
o
n
is
a
c
h
i
e
v
e
d
u
sin
g
th
e
Lea
k
y
-
Re
LU
a
n
d
S
ig
m
o
id
a
c
ti
v
a
ti
o
n
f
u
n
c
ti
o
n
s,
with
a
b
a
tch
s
ize
o
f
6
4
,
a
n
d
a
n
a
rc
h
i
tec
tu
re
c
o
n
sistin
g
o
f
3
h
id
d
e
n
lay
e
rs
wit
h
n
e
u
r
o
n
siz
e
s
o
f
2
5
6
,
1
2
8
,
a
n
d
6
4
.
T
h
is
m
o
d
e
l
is
c
a
p
a
b
le
o
f
p
r
o
v
i
d
in
g
re
a
l
-
ti
m
e
targ
e
t
ima
g
e
v
a
li
d
a
ti
o
n
wit
h
a
n
a
c
c
u
ra
c
y
o
f
u
p
to
9
4
.
3
1
%
.
K
ey
w
o
r
d
s
:
Activ
atio
n
f
u
n
ctio
n
s
B
atch
s
izes
Dee
p
n
eu
r
al
n
etwo
r
k
I
m
ag
e
v
alid
atio
n
Neu
r
o
n
s
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Nae
m
ah
Mu
b
ar
a
k
ah
Do
cto
r
al
Pro
g
r
a
m
in
C
o
m
p
u
te
r
Scien
ce
,
Facu
lty
o
f
C
o
m
p
u
te
r
Scien
ce
an
d
I
n
f
o
r
m
atio
n
T
ec
h
n
o
lo
g
y
,
Un
iv
er
s
itas
Su
m
ater
a
Utar
a
Me
d
an
,
Su
m
ater
a
Utar
a,
I
n
d
o
n
esia
E
m
ail: n
ae
m
ah
@
u
s
u
.
ac
.
id
1.
I
NT
RO
D
UCT
I
O
N
Valid
atio
n
is
s
till
a
f
ascin
atin
g
ar
ea
o
f
s
tu
d
y
.
R
esear
ch
er
s
ar
e
ac
tiv
ely
l
o
o
k
in
g
f
o
r
t
h
e
b
est
way
s
to
p
r
o
v
id
e
r
o
b
u
s
t
v
alid
atio
n
in
o
r
d
er
to
m
in
im
ize
c
o
r
r
ec
tiv
e
e
r
r
o
r
s
,
th
an
k
s
to
d
e
v
elo
p
m
e
n
ts
in
m
ac
h
in
e
lear
n
in
g
an
d
in
ter
n
et
o
f
th
in
g
s
(
I
o
T
)
s
e
n
s
o
r
s
.
No
n
-
lin
ea
r
an
d
n
o
n
-
s
tat
io
n
ar
y
d
ata
ar
e
an
aly
ze
d
u
s
in
g
a
v
ar
iety
o
f
d
ata
-
d
r
iv
en
tec
h
n
iq
u
es,
i
n
clu
d
in
g
m
ac
h
in
e
lear
n
i
n
g
a
n
d
s
ig
n
al
p
r
o
ce
s
s
in
g
.
Ho
wev
e
r
,
in
ad
eq
u
ate
in
f
o
r
m
atio
n
f
o
r
r
ea
l
-
tim
e
ap
p
licatio
n
s
f
r
eq
u
en
tly
r
esu
lts
in
a
r
ed
u
ctio
n
i
n
p
er
f
o
r
m
a
n
ce
.
Nu
m
er
o
u
s
s
tu
d
ie
s
in
th
e
v
alid
atio
n
s
ec
to
r
h
av
e
u
s
ed
a
v
ar
iety
o
f
tech
n
iq
u
es,
s
u
ch
as
ar
c
h
itectu
r
al
f
r
am
ewo
r
k
alter
atio
n
s
[
1
]
,
g
en
etic
alg
o
r
ith
m
(
GA)
[
2
]
,
s
u
p
er
lear
n
er
alg
o
r
i
th
m
[
3
]
,
an
d
d
if
f
er
en
tial
e
v
o
l
u
tio
n
(
DE
)
[
4
]
.
R
ea
l
-
tim
e
m
o
d
el
ap
p
licab
ilit
y
is
s
till
q
u
ite
lim
ited
d
esp
ite
a
lar
g
e
n
u
m
b
er
o
f
s
tu
d
ies.
Dee
p
lear
n
in
g
m
eth
o
d
s
f
o
r
o
b
ject
d
etec
tio
n
a
r
e
ac
k
n
o
wle
d
g
ed
f
o
r
th
eir
ef
f
icien
cy
,
o
wi
n
g
to
th
eir
ca
p
ac
ity
to
u
tili
ze
d
iv
er
s
e
lear
n
in
g
s
tr
ateg
ies
an
d
tr
ain
o
n
ex
ten
s
iv
e
d
atasets
[
5
]
,
[
6
]
.
T
h
is
ef
f
icien
cy
is
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Ta
r
g
et
ima
g
e
va
lid
a
tio
n
mo
d
elin
g
u
s
in
g
d
ee
p
n
e
u
r
a
l n
etw
o
r
k
a
lg
o
r
ith
m
(
N
a
ema
h
Mu
b
a
r
a
ka
h
)
2043
ev
id
en
ce
d
b
y
s
ig
n
if
ica
n
t
ad
v
a
n
ce
m
en
ts
in
s
eg
m
en
tatio
n
[
7
]
,
d
etec
tio
n
[
8
]
,
an
d
class
if
icatio
n
[
9
]
.
His
to
r
ically
,
im
ag
e
o
b
ject
ex
p
lo
r
atio
n
tech
n
iq
u
es
r
elied
o
n
co
lo
r
d
escr
ip
to
r
s
[
1
0
]
an
d
im
a
g
e
d
escr
ip
to
r
s
[
1
1
]
,
p
r
o
ce
s
s
ed
th
r
o
u
g
h
u
n
s
u
p
e
r
v
is
ed
alg
o
r
ith
m
s
s
u
ch
as
K
-
m
ea
n
s
clu
s
ter
in
g
[
1
2
]
,
s
p
ec
tr
al
clu
s
ter
in
g
[
1
3
]
,
p
o
o
lin
g
clu
s
ter
s
[
1
4
]
,
a
n
d
s
p
an
n
in
g
tr
ee
s
[
1
5
]
,
as
well
as
le
s
s
r
o
b
u
s
t
s
u
p
er
v
is
ed
alg
o
r
ith
m
s
lik
e
d
ee
p
m
etr
i
c
lear
n
in
g
[
1
6
]
an
d
s
u
b
s
p
ac
e
lear
n
in
g
[
1
7
]
.
Dee
p
n
eu
r
al
n
etwo
r
k
s
(
DNNs)
,
wh
ich
f
all
u
n
d
er
th
e
d
ee
p
lear
n
in
g
ca
teg
o
r
y
,
ar
e
ch
ar
ac
ter
ized
b
y
th
eir
m
u
lti
-
lay
er
ed
s
tr
u
ctu
r
es
-
ty
p
ically
c
o
m
p
r
is
in
g
th
r
ee
o
r
m
o
r
e
in
t
er
co
n
n
ec
ted
lay
er
s
.
DNNs
ex
ce
l
in
ad
d
r
ess
in
g
co
m
p
lex
p
r
o
b
le
m
s
an
d
h
av
e
b
ee
n
in
s
tr
u
m
e
n
tal
in
d
r
iv
in
g
s
ig
n
if
ican
t
in
n
o
v
atio
n
s
ac
r
o
s
s
v
ar
io
u
s
s
o
cieta
l
[
1
8
]
an
d
in
d
u
s
tr
ial
d
o
m
ai
n
s
[
1
9
]
–
[
2
2
]
.
Ho
wev
er
,
d
esp
ite
th
eir
ef
f
icac
y
in
s
o
lv
in
g
co
m
p
lex
ch
allen
g
es,
DNNs
n
ec
ess
itate
s
o
p
h
is
ticated
ar
c
h
itectu
r
es
with
n
u
m
er
o
u
s
h
id
d
en
lay
er
s
,
wh
ich
r
esu
lts
in
p
r
o
lo
n
g
ed
tr
ain
i
n
g
d
u
r
atio
n
s
[
2
3
]
.
T
h
e
p
r
i
m
ar
y
ch
allen
g
e
in
d
ee
p
n
eu
r
al
n
etwo
r
k
s
(
DNN)
alg
o
r
ith
m
s
is
d
eter
m
in
in
g
th
e
o
p
ti
m
al
m
o
d
el
to
ac
h
iev
e
th
e
b
est
p
er
f
o
r
m
a
n
ce
in
tar
g
et
im
ag
e
v
alid
atio
n
.
T
h
is
s
tu
d
y
f
o
cu
s
es
o
n
tar
g
et
im
ag
e
v
alid
atio
n
u
s
in
g
DNN
alg
o
r
ith
m
s
in
r
ea
l
-
tim
e
s
en
s
o
r
s
.
DNNs,
wh
ich
f
ea
tu
r
e
n
u
m
er
o
u
s
h
id
d
e
n
lay
er
s
,
ar
e
ev
alu
ated
b
y
co
m
p
ar
in
g
co
n
f
ig
u
r
atio
n
s
with
3
to
6
h
i
d
d
en
lay
er
s
.
T
h
e
ch
o
ice
o
f
ac
tiv
atio
n
f
u
n
ctio
n
is
cr
itical
to
DNN
p
er
f
o
r
m
an
ce
,
m
a
k
in
g
t
h
e
s
elec
tio
n
o
f
th
e
a
p
p
r
o
p
r
iate
a
ctiv
atio
n
f
u
n
ctio
n
ess
en
tial.
Ad
d
itio
n
ally
,
th
is
r
esear
ch
ass
ess
e
s
th
e
im
p
ac
t
o
f
b
atch
s
ize
an
d
th
e
n
u
m
b
er
o
f
n
eu
r
o
n
s
o
n
m
o
d
el
p
e
r
f
o
r
m
a
n
ce
.
T
h
e
g
o
al
is
to
id
en
tify
th
e
o
p
tim
al
m
o
d
el
ar
ch
itectu
r
e
f
o
r
tar
g
et
im
a
g
e
v
alid
atio
n
.
T
h
e
m
o
d
el
will
b
e
test
ed
in
r
ea
l
-
tim
e
co
n
tex
ts
to
e
v
alu
ate
its
ef
f
ec
t
iv
en
ess
.
T
h
e
f
i
n
d
in
g
s
o
f
t
h
is
s
tu
d
y
ca
n
m
ak
e
a
s
ig
n
if
ican
t
co
n
tr
ib
u
tio
n
to
th
e
ad
v
an
ce
m
e
n
t o
f
d
ata
m
in
in
g
t
ec
h
n
iq
u
es.
2.
M
AT
E
R
I
AL
A
ND
M
E
T
H
O
D
2
.
1
.
Dee
p neura
l net
wo
rk
A
n
etwo
r
k
m
ad
e
u
p
o
f
lay
er
s
o
f
n
e
u
r
o
n
s
is
ca
lled
a
DNN,
a
n
d
ea
ch
n
e
u
r
o
n
is
co
n
n
ec
ted
to
th
e
o
th
er
s
b
y
r
an
d
o
m
n
u
m
b
e
r
b
iases
[
2
2
]
.
T
h
r
o
u
g
h
ch
a
n
n
els
with
v
alu
es
ca
lled
weig
h
ts
,
n
eu
r
o
n
s
in
o
n
e
la
y
er
co
m
m
u
n
icate
with
n
eu
r
o
n
s
in
th
e
n
ex
t
lay
er
.
T
h
e
in
f
o
r
m
ati
o
n
th
at
is
s
h
ar
e
d
b
etwe
en
n
e
u
r
o
n
s
is
d
eter
m
in
ed
b
y
th
ese
weig
h
ts
an
d
b
iases
.
T
h
e
n
etwo
r
k
g
en
e
r
ates
an
o
u
tp
u
t
th
at
r
ef
lects
th
e
p
r
ed
ictio
n
o
f
th
e
p
r
o
ce
s
s
ed
in
p
u
t in
th
e
last
lay
er
,
r
ef
e
r
r
ed
to
as th
e
o
u
tp
u
t la
y
er
[
2
4
]
.
An
ar
tific
ial
n
eu
r
o
n
is
s
ee
n
in
Fig
u
r
e
1
.
Fig
u
r
e
1
.
An
ar
tific
ial
n
eu
r
o
n
T
h
e
in
p
u
t
is
co
n
n
ec
ted
t
o
th
e
n
eu
r
o
n
th
r
o
u
g
h
weig
h
ted
c
o
n
n
ec
tio
n
s
,
an
d
th
e
s
u
m
o
f
al
l
in
p
u
ts
,
ea
ch
m
u
ltip
lied
b
y
its
co
r
r
esp
o
n
d
in
g
weig
h
t Wi
,
is
co
m
p
u
te
d
.
T
h
is
s
u
m
m
atio
n
is
th
en
ad
d
ed
to
a
b
ias (
b
)
,
an
d
th
e
r
esu
lt
is
s
u
b
s
eq
u
en
tly
p
r
o
ce
s
s
ed
u
s
in
g
an
ac
tiv
atio
n
f
u
n
ctio
n
(
)
.
Ma
th
em
atica
lly
,
th
e
o
u
tp
u
t
o
f
a
p
er
ce
p
tr
o
n
u
n
it c
an
b
e
f
o
r
m
u
l
ated
as
(
1
)
[
2
3
]
.
=
(
∑
=
+
)
(
1
)
T
h
e
f
o
llo
win
g
is
an
o
th
er
way
to
m
o
d
el
it with
m
atr
i
x
n
o
tatio
n
:
=
(
.
+
)
(
2
)
W
h
er
e
=
[
1
2
…
]
d
an
=
[
1
2
⋮
]
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
25
:
2
0
4
2
-
2
0
5
4
2044
T
o
p
u
t
it
s
im
p
ly
,
a
n
eu
r
al
n
et
wo
r
k
is
a
g
r
o
u
p
o
f
lay
er
s
m
a
d
e
u
p
o
f
m
a
n
y
n
eu
r
o
n
s
.
E
ac
h
l
ay
er
h
as
an
o
u
tp
u
t
v
ec
to
r
,
a
b
ias
v
ec
to
r
,
a
n
d
a
weig
h
t
m
atr
ix
,
g
en
e
r
atin
g
co
lu
m
n
s
o
f
n
e
u
r
o
n
s
th
at
wo
r
k
in
p
ar
allel.
W
h
en
p
r
o
ce
s
s
in
g
an
i
n
p
u
t
v
ec
to
r
in
a
lay
er
o
f
n
eu
r
o
n
s
,
is
th
e
weig
h
t
o
f
th
e
co
n
n
ec
tio
n
b
et
wee
n
th
e
-
th
in
p
u
t
an
d
t
h
e
-
th
n
eu
r
o
n
in
t
h
at
lay
er
,
an
d
an
d
ar
e
th
e
-
th
n
eu
r
o
n
'
s
o
u
tp
u
t
an
d
b
ias,
r
esp
ec
tiv
ely
[
2
5
]
.
As
a
r
esu
lt,
th
e
f
o
llo
win
g
m
atr
ix
n
o
tatio
n
ca
n
b
e
u
s
ed
t
o
r
ep
r
esen
t a
n
eu
r
o
n
lay
e
r
:
=
[
11
⋮
1
.
.
.
⋮
.
.
.
⋮
]
(
3
)
I
n
m
atr
ix
n
o
tatio
n
,
th
e
co
l
u
m
n
in
d
ex
in
d
icate
s
th
e
co
n
n
ec
ti
o
n
'
s
in
p
u
t
s
o
u
r
ce
.
W
h
ile
th
e
r
o
w
in
d
ex
in
d
icate
s
th
e
d
esti
n
atio
n
n
eu
r
o
n
f
o
r
th
e
co
r
r
esp
o
n
d
in
g
co
n
n
ec
tio
n
.
As
a
r
esu
lt,
th
e
lay
e
r
'
s
o
u
tp
u
t
ca
n
b
e
wr
itten
lik
e
th
is
:
[
⋮
⋮
]
=
[
(
∑
1
=
+
)
⋮
(
∑
=
+
)
⋮
(
∑
=
+
)
]
=
(
.
+
)
(
4
)
W
h
er
e
b
=
[
⋮
]
.
I
n
n
eu
r
o
n
la
y
er
s
,
s
u
p
e
r
s
cr
ip
t
i
n
d
ices
ar
e
also
u
s
ed
.
Fo
r
ex
a
m
p
le,
in
d
icate
s
th
e
weig
h
t
b
etwe
en
th
e
-
th
n
eu
r
o
n
in
lay
er
an
d
th
e
-
th
n
eu
r
o
n
in
lay
er
(
−
1
)
,
wh
ile
in
d
icate
s
th
e
o
u
tp
u
t o
f
th
e
-
th
n
eu
r
o
n
in
lay
er
.
Fu
r
th
er
m
o
r
e,
is
in
ten
d
ed
to
s
y
m
b
o
lize
th
e
q
u
an
tity
o
f
b
u
r
ied
n
eu
r
o
n
s
in
lay
er
k
.
T
h
er
ef
o
r
e
,
th
e
f
u
n
ctio
n
t
h
at
ca
n
b
e
d
er
iv
e
d
f
r
o
m
th
is
n
etwo
r
k
is
as (
5
)
:
3
=
[
3
⋮
3
⋮
3
3
]
=
(
3
2
+
3
)
=
(
3
(
2
1
+
2
)
+
3
)
=
(
3
(
2
(
(
1
+
)
)
+
2
)
3
)
(
5
)
On
e
k
in
d
o
f
ar
tific
ial
n
eu
r
al
n
etwo
r
k
with
s
ev
er
al
la
y
er
s
i
s
ca
lled
a
DNN.
An
in
p
u
t
la
y
er
,
N>
2
h
id
d
en
lay
er
s
,
an
d
an
o
u
tp
u
t
lay
er
ar
e
th
e
th
r
ee
lay
er
s
th
at
ar
e
ty
p
ically
p
r
esen
t
in
a
DNN.
'
Dee
p
'
d
escr
ib
es
th
e
co
m
p
ar
ativ
ely
h
ig
h
n
u
m
b
er
o
f
lay
e
r
s
.
Dee
p
lear
n
i
n
g
is
th
e
ter
m
f
o
r
th
e
lear
n
in
g
p
r
o
c
ess
th
at
tak
es
p
lace
in
s
id
e
a
DNN.
A
d
ee
p
n
eu
r
al
n
etwo
r
k
is
th
e
n
am
e
g
i
v
en
to
th
e
n
eu
r
al
n
etwo
r
k
in
a
DNN
[
2
4
]
.
Fo
r
m
u
la
(
6
)
i
s
u
s
ed
to
ca
lc
u
late
th
e
f
in
al
o
u
t
p
u
t
o
f
a
DNN
with
f
o
u
r
lay
er
s
,
wh
er
e
σ
is
th
e
ac
tiv
atio
n
f
u
n
ctio
n
an
d
β,
γ
,
a
n
d
λ
s
tan
d
f
o
r
n
o
is
e
o
r
b
ias.
=
(
∑
,
(
∑
.
(
∑
,
+
=
1
)
+
1
=
1
+
)
2
=
1
)
(
6
)
Dee
p
n
eu
r
al
n
etwo
r
k
s
ca
n
b
e
tr
ain
ed
u
s
in
g
th
e
b
ac
k
-
p
r
o
p
ag
atio
n
p
r
o
ce
s
s
.
I
t
ca
n
b
e
d
if
f
icu
lt
to
esti
m
ate
p
ar
am
eter
s
in
d
ee
p
n
eu
r
al
n
etwo
r
k
s
b
ec
au
s
e
o
f
t
h
eir
co
m
p
lex
ity
,
wh
ich
in
cl
u
d
es sev
er
al
lay
er
s
an
d
a
lar
g
e
n
u
m
b
er
o
f
s
y
n
ap
tic
we
ig
h
ts
.
Neu
r
al
n
etwo
r
k
s
ca
lab
ilit
y
is
s
tr
o
n
g
ly
r
elate
d
to
t
h
e
b
ac
k
-
p
r
o
p
a
g
atio
n
tech
n
iq
u
e,
w
h
ich
is
f
r
eq
u
en
tl
y
u
s
ed
f
o
r
tr
ain
i
n
g
n
e
u
r
al
n
et
wo
r
k
s
.
T
h
e
m
eth
o
d
m
ak
es
ite
r
ativ
e
m
o
d
if
icatio
n
s
to
g
et
o
p
tim
al
weig
h
t c
o
n
f
i
g
u
r
atio
n
s
.
T
h
r
ee
-
lay
e
r
DNN
is
s
ee
n
in
Fig
u
r
e
2
.
2
.
2
.
DNN
a
lg
o
rit
hm
perf
o
r
m
a
nce
m
e
a
s
urem
ent
Per
f
o
r
m
an
ce
m
ea
s
u
r
em
e
n
t
is
cr
itical
in
th
e
f
ield
o
f
m
ac
h
in
e
lear
n
in
g
.
T
h
e
ar
ea
u
n
d
er
th
e
c
u
r
v
e
(
AUC)
o
f
th
e
r
ec
eiv
er
o
p
er
ati
n
g
ch
ar
ac
ter
is
tic
(
R
OC
)
cu
r
v
e
is
a
co
m
m
o
n
ly
u
s
ed
p
er
f
o
r
m
an
ce
s
tatis
tic.
A
n
im
p
o
r
tan
t
m
etr
ic
f
o
r
ev
alu
atin
g
th
e
ef
f
ec
tiv
en
ess
o
f
class
if
icatio
n
m
o
d
els
is
th
e
AUC
[
2
6
]
.
I
n
p
a
r
ticu
lar
,
th
e
ar
ea
u
n
d
er
th
e
R
OC
cu
r
v
e
is
q
u
an
tifie
d
b
y
th
e
AUC.
Plo
ttin
g
th
e
tr
u
e
p
o
s
itiv
e
r
ate
(
T
P
R
)
v
er
s
u
s
th
e
f
alse
p
o
s
itiv
e
r
ate
(
FP
R
)
ac
r
o
s
s
v
ar
io
u
s
class
if
icatio
n
th
r
esh
o
ld
s
allo
ws th
e
R
OC
cu
r
v
e,
a
g
r
ap
h
i
ca
l to
o
l,
to
ass
ess
a
class
if
icatio
n
m
o
d
el'
s
p
er
f
o
r
m
an
ce
.
T
h
e
f
o
llo
win
g
a
r
e
th
e
m
ain
elem
en
ts
o
f
th
e
R
OC
cu
r
v
e:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Ta
r
g
et
ima
g
e
va
lid
a
tio
n
mo
d
elin
g
u
s
in
g
d
ee
p
n
e
u
r
a
l n
etw
o
r
k
a
lg
o
r
ith
m
(
N
a
ema
h
Mu
b
a
r
a
ka
h
)
2045
a.
T
h
e
r
atio
o
f
ac
c
u
r
ately
a
n
ticip
ated
p
o
s
itiv
e
ca
s
es
to
all
ac
tu
al
p
o
s
itiv
es
is
k
n
o
w
n
as
th
e
T
PR
,
an
d
it
is
d
ef
in
ed
as (
7
)
:
=
+
(
7
)
b.
T
h
e
r
atio
o
f
f
alsely
p
r
o
jecte
d
p
o
s
itiv
e
ca
s
es
to
all
ac
tu
al
n
e
g
ativ
es
is
k
n
o
wn
as
th
e
FP
R
,
an
d
it
is
d
ef
in
e
d
as (
8
)
:
=
+
(
8
)
wh
er
e
is
tr
u
e
p
o
s
itiv
e,
is
tr
u
e
n
eg
ativ
es,
is
f
al
s
e
p
o
s
itiv
e
an
d
is
f
alse
n
eg
ativ
es.
T
h
e
is
r
ep
r
esen
ted
b
y
t
h
e
Y
-
ax
is
in
th
e
R
OC
cu
r
v
e,
wh
e
r
ea
s
th
e
is
r
ep
r
esen
ted
b
y
th
e
X
-
a
x
is
.
T
h
e
AUC
v
alu
es
ca
n
b
e
in
ter
p
r
eted
as f
o
llo
ws:
a.
AUC =
1
: T
h
e
m
o
d
el
e
x
h
ib
its
f
lawless
ca
teg
o
r
izatio
n
ca
p
ab
i
liti
es.
b.
0
.
5
<
AUC
<
1
:
T
h
e
m
o
d
el
o
u
tp
e
r
f
o
r
m
s
r
an
d
o
m
g
u
ess
in
g
;
th
e
clo
s
er
th
e
AUC
v
alu
e
is
to
1
,
th
e
b
etter
th
e
m
o
d
el
p
er
f
o
r
m
s
.
c.
AUC =
0
.
5
: T
h
e
m
o
d
el'
s
p
er
f
o
r
m
an
ce
is
o
n
p
ar
with
g
u
ess
wo
r
k
.
d.
AUC
<
0
.
5
:
T
h
e
m
o
d
el
p
e
r
f
o
r
m
s
wo
r
s
e
th
an
r
an
d
o
m
g
u
ess
in
g
,
wh
ic
h
m
a
y
im
p
l
y
th
at
th
e
m
o
d
el
is
in
v
er
ted
Fig
u
r
e
2
.
T
h
r
ee
-
lay
e
r
DNN
2
.
3
.
Arc
hite
ct
ure
re
s
ea
rc
h
Var
io
u
s
s
ch
em
es
wer
e
ap
p
lie
d
with
d
if
f
e
r
en
t
co
n
f
ig
u
r
atio
n
s
o
f
h
id
d
e
n
lay
er
s
,
ac
tiv
ati
o
n
f
u
n
ctio
n
s
,
b
atch
s
izes,
an
d
n
u
m
b
er
s
o
f
n
eu
r
o
n
s
.
Af
ter
d
eter
m
i
n
in
g
t
h
e
o
p
tim
al
m
o
d
el
f
o
r
o
b
ject
v
alid
atio
n
u
s
in
g
t
h
e
DNN
alg
o
r
ith
m
,
th
is
m
o
d
el
was
d
ir
ec
tly
ap
p
lied
to
o
b
jec
ts
ca
p
tu
r
ed
b
y
a
ca
m
er
a
f
o
r
r
ea
l
-
tim
e
an
aly
s
is
.
Fig
u
r
e
3
s
h
o
ws
th
e
DNN
-
b
as
ed
m
ac
h
in
e
lear
n
in
g
m
o
d
elin
g
p
r
o
ce
d
u
r
e.
T
h
e
s
tu
d
y
f
lo
wc
h
ar
t
is
d
is
p
lay
ed
in
Fig
u
r
e
3
(
a)
.
Sev
e
r
al
m
ac
h
in
e
l
ea
r
n
in
g
m
o
d
els
ar
e
v
iewe
d
in
th
e
cr
ate,
tr
ain
,
an
d
ass
ess
m
en
t
p
h
ase
in
o
r
d
er
to
s
elec
t
th
e
b
est
m
o
d
el,
as
illu
s
tr
ated
in
Fig
u
r
e
3
(
b
)
.
T
ab
le
1
lis
ts
th
e
m
an
y
p
ar
am
eter
s
u
s
ed
in
th
e
s
ea
r
ch
f
o
r
th
e
o
p
tim
al
m
o
d
el.
T
o
id
en
tify
th
e
b
est
m
o
d
el,
r
e
f
er
en
ce
p
ar
am
eter
s
will
b
e
ch
o
s
en
b
ased
o
n
th
e
o
u
tco
m
es
o
f
p
ar
am
eter
m
o
d
if
icatio
n
s
.
T
h
e
R
OC
cu
r
v
e'
s
AU
C
s
er
v
es
a
s
th
e
f
o
u
n
d
a
tio
n
f
o
r
th
is
ev
alu
atio
n
.
T
h
e
b
est
m
o
d
el
will
b
e
ch
o
s
en
f
o
r
tar
g
et
p
ictu
r
e
v
alid
atio
n
af
te
r
it
h
as
b
ee
n
d
ete
r
m
in
ed
a
n
d
ev
alu
ated
in
r
ea
l
-
tim
e
to
ass
ess
its
v
alid
atio
n
p
er
f
o
r
m
a
n
ce
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
25
:
2
0
4
2
-
2
0
5
4
2046
(
a
)
(
b
)
Fig
u
r
e
3
.
R
esear
ch
f
lo
wc
h
ar
t
:
(
a)
m
ain
p
r
o
ce
s
s
an
d
(
b
)
m
o
d
e
l
d
ev
elo
p
m
e
n
t p
r
o
ce
s
s
in
m
ac
h
in
e
lear
n
in
g
with
DNN
T
ab
le
1
.
T
h
e
r
esear
ch
'
s
p
ar
am
eter
s
N
o
.
P
a
r
a
me
t
e
r
s
u
se
d
i
n
t
h
e
r
e
se
a
r
c
h
1.
N
u
mb
e
r
o
f
h
i
d
d
e
n
l
a
y
e
r
s
:
3
,
4
,
5
,
6
2.
A
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
:
R
e
c
t
i
f
i
e
d
l
i
n
e
a
r
u
n
i
t
(
R
e
LU
)
,
S
i
g
m
o
i
d
,
L
e
a
k
y
R
e
LU
,
T
a
n
h
,
Li
n
i
e
r
,
S
c
a
l
e
d
e
x
p
o
n
e
n
t
i
a
l
l
i
n
e
a
r
u
n
i
t
(
S
ELU
)
a
n
d
S
o
f
t
M
a
x
.
3.
N
u
mb
e
r
o
f
b
a
t
c
h
si
z
e
s:
1
6
,
3
2
,
6
4
,
1
2
8
,
2
5
6
4.
N
u
mb
e
r
o
f
n
e
u
r
o
n
s:
6
4
,
1
2
8
,
2
5
6
,
5
1
2
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
3
.
1
.
Act
iv
a
t
io
n
f
un
ct
io
ns
v
a
ria
t
io
n
Dee
p
lear
n
in
g
m
eth
o
d
s
r
ely
o
n
ac
tiv
atio
n
f
u
n
ctio
n
s
to
g
en
er
ate
ef
f
icien
t
s
y
s
tem
p
e
r
f
o
r
m
a
n
ce
.
Kn
o
win
g
w
h
ich
ac
tiv
atio
n
f
u
n
ctio
n
is
ap
p
r
o
p
r
iate
to
u
s
e
wit
h
th
e
DNN
m
eth
o
d
is
th
e
r
ef
o
r
e
ess
en
tial.
Sev
er
al
ac
tiv
atio
n
f
u
n
ctio
n
m
o
d
if
icati
o
n
s
,
s
u
ch
as
r
ec
tifie
d
lin
ea
r
u
n
it
(
R
eL
U)
,
L
u
ck
y
R
eL
U,
SE
L
U,
Sig
m
o
id
,
T
an
h
,
L
in
ea
r
,
an
d
So
f
t
m
ax
,
ar
e
u
s
ed
in
th
is
wo
r
k
.
C
h
an
g
es
in
t
h
e
n
u
m
b
e
r
o
f
h
id
d
en
la
y
er
s
ar
e
u
s
ed
to
test
th
e
R
eL
U,
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
.
I
n
t
h
is
in
s
tan
ce
,
t
h
r
ee
,
f
o
u
r
,
f
iv
e,
an
d
s
ix
h
id
d
en
lev
el
s
ar
e
im
p
lem
en
ted
.
Fig
u
r
e
4
d
is
p
lay
s
th
e
co
n
f
u
s
io
n
m
atr
ix
r
esu
lts
o
f
th
e
R
eL
U
an
d
Sig
m
o
i
d
ac
tiv
atio
n
f
u
n
cti
o
n
o
n
3
,
4
,
5
,
a
n
d
6
h
id
d
en
lay
e
r
s
,
wh
ile
Fig
u
r
e
5
d
is
p
lay
s
th
e
tr
ain
in
g
an
d
v
ali
d
atio
n
lo
s
s
g
r
ap
h
s
.
Fig
u
r
es
4
an
d
5
d
em
o
n
s
tr
ate
h
o
w
th
e
DNN
m
eth
o
d
p
er
f
o
r
m
s
p
o
o
r
ly
wh
e
n
R
eL
U
an
d
Si
g
m
o
id
ar
e
u
s
ed
as
ac
tiv
atio
n
f
u
n
ctio
n
s
.
T
h
e
AUC
f
in
d
in
g
s
f
r
o
m
th
e
R
OC
f
o
r
th
e
ap
p
licatio
n
o
f
th
e
R
eL
U
ar
e
d
er
iv
ed
f
r
o
m
th
e
r
esu
lts
o
f
Fig
u
r
es
4
an
d
5
,
as
s
h
o
wn
in
Fig
u
r
e
6
.
T
h
e
R
OC
o
f
th
e
o
t
h
er
ac
tiv
atio
n
f
u
n
ctio
n
'
s
AU
C
is
d
is
p
lay
ed
in
Fig
u
r
e
7
.
T
h
e
p
er
f
o
r
m
a
n
ce
o
f
th
e
s
u
g
g
ested
DNN
m
o
d
el
with
v
a
r
io
u
s
ac
tiv
atio
n
f
u
n
ctio
n
s
is
d
is
p
lay
ed
in
Fig
u
r
e
7
.
T
h
e
R
OC
o
f
th
e
DNN
m
o
d
el
with
th
r
ee
to
s
ix
h
id
d
en
lay
er
s
th
at
u
s
es
Sig
m
o
id
in
th
e
o
u
tp
u
t
lay
er
an
d
L
ea
k
y
-
R
eL
U
ac
tiv
atio
n
lay
er
s
in
th
e
h
id
d
en
lay
er
s
is
d
is
p
lay
ed
in
Fig
u
r
e
7
(
a)
.
T
h
e
DNN
m
o
d
el's
p
er
f
o
r
m
an
ce
with
T
a
n
h
a
n
d
Sig
m
o
id
ac
tiv
atio
n
lay
er
s
is
s
h
o
wn
in
Fig
u
r
e
7
(
b
)
.
T
h
e
DNN
m
o
d
el'
s
p
er
f
o
r
m
an
ce
u
tili
zin
g
th
e
lin
e
ar
an
d
s
ig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
s
is
d
is
p
lay
ed
in
Fig
u
r
e
7
(
c)
.
T
h
e
AUC
f
o
r
th
e
DNN
m
o
d
el
with
s
ig
m
o
id
in
th
e
o
u
tp
u
t
lay
e
r
an
d
SEL
U
in
th
e
h
id
d
en
la
y
er
is
th
en
s
h
o
wn
in
Fig
u
r
e
7
(
d
)
.
Ad
d
itio
n
ally
,
Fig
u
r
e
7
(
e)
illu
s
tr
ates
h
o
w
Sig
m
o
id
is
u
s
ed
in
all
lay
er
s
,
in
clu
d
i
n
g
t
h
e
o
u
tp
u
t
an
d
h
id
d
e
n
lay
e
r
s
.
L
astl
y
,
th
e
m
o
d
el'
s
p
er
f
o
r
m
a
n
ce
em
p
lo
y
in
g
th
e
SEL
U
an
d
So
f
tMa
x
as
th
e
ac
tiv
atio
n
f
u
n
ctio
n
is
d
is
p
lay
ed
in
Fig
u
r
e
7
(
f
)
.
Acc
o
r
d
in
g
t
o
th
e
o
u
tco
m
e,
t
h
e
m
o
d
el'
s
AU
C
r
is
es
f
r
o
m
0
.
5
to
0
.
8
2
wh
e
n
Sig
m
o
id
is
u
s
ed
in
th
e
o
u
tp
u
t
lay
er
.
T
h
is
is
b
ec
au
s
e
th
e
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
is
ap
p
r
o
p
r
iate
f
o
r
b
in
a
r
y
cla
s
s
if
icatio
n
p
r
o
b
lem
s
b
ec
au
s
e
it
p
r
o
d
u
ce
s
v
al
u
es
b
etwe
en
0
an
d
1
.
Fu
r
th
er
m
o
r
e,
th
e
SEL
U,
Sig
m
o
id
with
f
o
u
r
h
id
d
en
lay
er
s
,
y
ield
s
th
e
b
est
AUC
o
f
an
y
s
tu
d
ied
a
ctiv
atio
n
f
u
n
ctio
n
,
at
0
.
8
2
.
H
o
wev
er
,
T
ab
le
2
an
d
Fig
u
r
e
8
d
em
o
n
s
tr
ate
th
at
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Ta
r
g
et
ima
g
e
va
lid
a
tio
n
mo
d
elin
g
u
s
in
g
d
ee
p
n
e
u
r
a
l n
etw
o
r
k
a
lg
o
r
ith
m
(
N
a
ema
h
Mu
b
a
r
a
ka
h
)
2047
Sig
m
o
id
m
o
d
el
p
e
r
f
o
r
m
s
b
e
s
t
wh
en
ev
alu
atin
g
th
e
s
tab
ilit
y
o
f
t
h
e
m
o
d
el'
s
p
er
f
o
r
m
a
n
ce
ac
r
o
s
s
v
a
r
y
in
g
n
u
m
b
er
s
o
f
h
id
d
en
lay
e
r
s
,
with
an
av
e
r
ag
e
AUC o
f
0
.
8
0
5
f
o
r
th
e
L
ea
k
y
-
R
eL
U.
Fig
u
r
e
8
s
h
o
ws
th
e
av
er
a
g
e
AUC
s
co
r
es
f
o
r
v
ar
io
u
s
ac
tiv
atio
n
f
u
n
ctio
n
s
.
B
ased
o
n
t
h
e
g
r
ap
h
,
th
e
av
er
ag
e
AUC
v
ar
ies
d
ep
en
d
in
g
o
n
t
h
e
ac
tiv
atio
n
f
u
n
ctio
n
co
m
b
in
atio
n
s
u
s
ed
.
T
h
e
co
m
b
in
atio
n
o
f
L
ea
k
y
-
R
eL
U
with
Sig
m
o
id
an
d
L
in
ier
with
Sig
m
o
id
d
em
o
n
s
tr
ates
th
e
b
est
p
er
f
o
r
m
an
ce
,
with
an
av
er
ag
e
AUC
0
.
8
0
5
an
d
0
.
7
9
,
i
n
d
icatin
g
b
etter
m
o
d
el
class
if
icatio
n
ca
p
ab
ilit
ies.
Ov
er
all,
ac
tiv
atio
n
f
u
n
ctio
n
s
s
u
ch
as
L
ea
k
y
-
R
eL
U,
L
in
ier
an
d
Sig
m
o
id
ten
d
t
o
p
r
o
d
u
ce
g
o
o
d
p
er
f
o
r
m
an
ce
wh
en
p
air
ed
wi
th
th
e
ap
p
r
o
p
r
iate
ac
tiv
atio
n
f
u
n
ctio
n
.
T
h
e
r
ef
o
r
e
,
th
e
c
o
m
b
in
atio
n
o
f
L
ea
k
y
-
R
e
L
U
with
Sig
m
o
id
is
r
ec
o
m
m
e
n
d
ed
to
ac
h
iev
e
t
h
e
b
est p
er
f
o
r
m
an
ce
.
Fig
u
r
e
4
.
C
o
n
f
u
s
io
n
m
atr
i
x
o
f
R
eL
U,
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
s
at
3
,
4
,
5
,
an
d
6
h
id
d
e
n
lay
er
s
Fig
u
r
e
5
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
lo
s
s
g
r
ap
h
with
R
eL
U
an
d
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
25
:
2
0
4
2
-
2
0
5
4
2048
Fig
u
r
e
6
.
R
OC
g
r
ap
h
o
f
th
e
R
eL
U
-
Sig
m
o
id
ac
tiv
atio
n
f
u
n
cti
o
n
(
a)
(
b
)
(
c)
(
d
)
(
e)
(f)
Fig
u
r
e
7
.
R
OC
g
r
ap
h
o
f
: (
a
)
L
ea
k
y
R
eL
U,
Sig
m
o
id
,
(
b
)
T
a
n
h
,
Sig
m
o
id
,
(
c)
L
in
ie
r
,
Sig
m
o
i
d
,
(
d
)
SEL
U,
Sig
m
o
id
,
(
e)
Sig
m
o
id
,
Sig
m
o
i
d
,
an
d
(
f
)
SEL
U,
So
f
tMa
x
ac
ti
v
atio
n
f
u
n
ctio
n
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Ta
r
g
et
ima
g
e
va
lid
a
tio
n
mo
d
elin
g
u
s
in
g
d
ee
p
n
e
u
r
a
l n
etw
o
r
k
a
lg
o
r
ith
m
(
N
a
ema
h
Mu
b
a
r
a
ka
h
)
2049
T
ab
le
2
.
Ar
ea
u
n
d
e
r
th
e
cu
r
v
e
f
r
o
m
v
ar
io
u
s
ac
tiv
atio
n
f
u
n
cti
o
n
s
N
o
.
A
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
s
A
U
C
f
r
o
m
d
i
f
f
e
r
e
n
t
n
u
mb
e
r
o
f
h
i
d
d
e
n
l
a
y
e
r
s
A
v
e
r
a
g
e
A
U
C
s
c
o
r
e
3
4
5
6
1
R
e
LU
,
S
i
g
m
o
i
d
0
.
6
1
0
.
5
9
0
.
5
6
0
.
7
1
0
.
6
1
7
5
2
Le
a
k
y
R
e
LU
,
S
i
g
mo
i
d
0
.
8
0
0
.
8
1
0
.
8
1
0
.
8
0
0
.
8
0
5
3
Ta
n
h
,
S
i
g
m
o
i
d
0
.
6
2
0
.
4
7
0
.
5
2
0
.
5
1
0
.
5
3
4
Li
n
i
e
r
,
S
i
g
m
o
i
d
0
.
7
9
0
.
7
9
0
.
7
9
0
.
7
9
0
.
7
9
5
S
ELU
,
S
i
g
m
o
i
d
0
,
7
7
0
.
8
2
0
.
4
8
0
.
5
4
0
.
6
5
2
5
6
S
i
g
m
o
i
d
,
S
i
g
m
o
i
d
0
.
7
9
0
.
6
6
0
.
7
2
0
.
5
5
0
.
6
8
7
S
ELU
,
S
o
f
t
M
a
x
0
.
5
0
.
5
0
.
5
0
.
5
0
.
5
Fig
u
r
e
8
.
Gr
a
p
h
o
f
th
e
a
v
er
ag
e
AUC v
alu
e
f
r
o
m
v
ar
io
u
s
ac
ti
v
atio
n
f
u
n
ctio
n
s
3
.
2
.
B
a
t
ch
s
izes v
a
ria
t
i
o
n
Fro
m
s
ec
tio
n
3
.
1
it
is
o
b
tain
e
d
th
at
th
e
u
s
ag
e
o
f
L
ea
k
y
-
R
eL
U,
s
ig
m
o
id
ac
tiv
ati
o
n
f
u
n
cti
o
n
r
et
u
r
n
s
th
e
b
est
p
er
f
o
r
m
a
n
ce
.
T
h
e
r
ef
o
r
e,
in
th
e
b
atc
h
s
ize
v
ar
iatio
n
test
s
,
th
e
L
ea
k
y
-
R
eL
U,
s
ig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
is
ch
o
s
en
.
I
n
t
h
is
test
,
v
ar
y
in
g
b
atch
s
ize
1
6
,
3
2
,
6
4
,
1
2
8
,
an
d
2
5
6
a
r
e
u
s
ed
f
o
r
th
e
DNN
m
o
d
el
with
v
ar
io
u
s
n
u
m
b
er
o
f
h
id
d
en
lay
er
s
.
Fig
u
r
e
9
d
is
p
lay
s
th
e
m
o
d
el'
s
tr
ain
in
g
an
d
v
alid
atio
n
l
o
s
s
es
f
o
r
b
atch
s
ize
s
o
f
1
6
.
Ad
d
itio
n
ally
,
Fig
u
r
e
1
0
d
is
p
lay
s
th
e
m
o
d
el'
s
AU
C
s
c
o
r
e
with
a
b
atch
s
ize
o
f
1
6
,
with
an
av
e
r
ag
e
AUC
s
co
r
e
o
f
0
.
8
2
.
Usi
n
g
f
iv
e
h
id
d
en
lay
er
s
y
ield
s
th
e
g
r
ea
test
r
esu
lts
,
with
an
AUC
s
co
r
e
o
f
0
.
8
2
.
T
h
e
m
o
d
el'
s
AUC
s
co
r
e
is
al
s
o
d
is
p
lay
ed
in
Fig
u
r
e
1
1
f
o
r
b
atch
s
izes
o
f
3
2
,
6
4
,
1
2
8
a
n
d
2
5
6
.
Fig
u
r
e
1
1
(
a)
d
is
p
lay
s
th
e
AUC
o
f
th
e
R
OC
wh
en
u
s
in
g
a
b
atc
h
s
ize
o
f
3
2
,
Fig
u
r
e
1
1
(
b
)
d
is
p
lay
s
th
e
AUC
wh
en
u
s
in
g
a
b
atc
h
s
ize
o
f
6
4
,
Fig
u
r
e
1
1
(
c)
d
is
p
lay
s
th
e
AUC
o
f
th
e
m
o
d
el
u
s
in
g
a
b
atch
s
ize
o
f
1
2
8
an
d
Fig
u
r
e
1
1
(
d
)
d
is
p
lay
s
th
e
AUC
wh
en
u
s
in
g
a
b
atch
s
ize
o
f
2
5
6
.
Fig
u
r
e
9
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
lo
s
s
u
s
in
g
L
ea
k
y
-
R
eL
U
an
d
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
f
o
r
b
atch
s
izes o
f
1
6
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
15
,
No
.
2
,
Ap
r
il
20
25
:
2
0
4
2
-
2
0
5
4
2050
Fig
u
r
e
1
0
.
R
OC
g
r
ap
h
with
L
e
ak
y
-
R
eL
U,
Sig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
f
o
r
b
atch
s
izes o
f
1
6
(
a)
(
b
)
(
c)
(
d
)
Fig
u
r
e
1
1
.
Gr
ap
h
o
f
R
OC
c
u
r
v
e
with
b
atch
s
izes o
f
: (
a)
3
2
,
(
b
)
6
4
,
(
c)
1
2
8
,
an
d
(
d
)
2
5
6
B
ased
o
n
th
e
b
atch
s
ize
v
a
r
iatio
n
test
,
all
th
e
b
atch
s
izes
p
r
o
d
u
ce
th
e
AUC
s
co
r
e
n
o
lo
w
er
th
an
0
.
8
with
th
e
h
i
g
h
est
AUC
s
co
r
e
o
f
0
.
8
2
.
Ho
wev
e
r
,
u
s
in
g
d
if
f
er
en
t
b
atc
h
s
izes
f
o
r
tr
ain
i
n
g
th
e
p
r
o
p
o
s
ed
DNN
m
o
d
el
d
o
es
n
o
t
s
ig
n
if
ican
tly
im
p
r
o
v
e
th
e
m
o
d
el’
s
p
er
f
o
r
m
an
ce
.
T
h
e
o
v
er
all
p
er
f
o
r
m
a
n
c
e
o
f
th
e
b
atch
s
iz
e
v
ar
iatio
n
test
is
s
h
o
wn
in
T
ab
l
e
3
.
Fro
m
T
ab
le
3
,
it
ca
n
b
e
s
ee
n
th
at
th
e
h
ig
h
est
av
er
ag
e
A
UC
s
co
r
e
o
f
0
.
8
1
5
is
ac
h
iev
ed
wh
en
u
s
in
g
b
atch
s
ize
is
6
4
.
T
h
er
ef
o
r
e,
it
is
r
ec
o
m
m
en
d
e
d
to
u
s
e
b
atch
s
ize
o
f
6
4
wh
en
tr
ai
n
in
g
th
e
DNN
m
o
d
el.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Ta
r
g
et
ima
g
e
va
lid
a
tio
n
mo
d
elin
g
u
s
in
g
d
ee
p
n
e
u
r
a
l n
etw
o
r
k
a
lg
o
r
ith
m
(
N
a
ema
h
Mu
b
a
r
a
ka
h
)
2051
T
ab
le
3
.
AUC s
co
r
e
f
r
o
m
R
OC
with
v
ar
io
u
s
b
atch
s
ize
N
o
.
B
a
t
c
h
si
z
e
A
U
C
f
r
o
m
d
i
f
f
e
r
e
n
t
n
u
mb
e
r
o
f
h
i
d
d
e
n
l
a
y
e
r
s
A
v
e
r
a
g
e
A
U
C
s
c
o
r
e
3
4
5
6
1
16
0
.
8
0
0
.
8
1
0
.
8
2
0
.
8
0
0
.
8
0
7
5
2
32
0
.
8
0
0
.
8
1
0
.
8
1
0
.
8
0
0
.
8
0
5
3
64
0
.
8
1
0
.
8
2
0
.
8
0
0
.
8
3
0
.
8
1
5
4
1
2
8
0
.
8
2
0
.
8
0
0
.
8
2
0
.
8
0
0
.
8
1
5
2
5
6
0
.
8
1
0
.
8
0
0
.
8
0
0
.
8
2
0
.
8
0
7
5
3
.
3
.
Num
ber
o
f
neuro
ns
v
a
r
ia
t
io
n
Af
ter
o
b
tain
in
g
th
e
b
est
ac
tiv
atio
n
f
u
n
ctio
n
an
d
b
atch
s
ize,
f
u
r
th
er
in
v
esti
g
atio
n
is
m
ad
e
t
o
f
in
d
th
e
b
est
n
u
m
b
er
o
f
n
eu
r
o
n
s
f
o
r
th
e
DNN
m
o
d
el.
T
h
e
n
u
m
b
e
r
o
f
n
eu
r
o
n
s
in
th
e
DNN
lay
er
s
h
as
a
b
ig
im
p
ac
t
o
n
h
o
w
well
th
e
DNN
m
o
d
el
p
er
f
o
r
m
s
.
T
h
e
m
o
d
el'
s
ac
cu
r
ac
y
an
d
lik
elih
o
o
d
o
f
g
en
er
alizin
g
s
u
cc
ess
f
u
lly
wo
u
ld
b
o
th
b
e
en
h
an
ce
d
b
y
a
d
d
in
g
ad
d
itio
n
al
n
e
u
r
o
n
s
,
wh
ich
wo
u
ld
let
it
to
lear
n
m
o
r
e
ab
o
u
t
th
e
in
tr
icate
u
n
d
er
ly
i
n
g
p
atter
n
s
in
th
e
d
at
a.
Un
d
er
f
itti
n
g
,
in
w
h
ich
th
e
m
o
d
el
is
to
o
b
asic
to
ca
p
tu
r
e
t
h
e
lin
k
b
etwe
en
th
e
in
p
u
t
a
n
d
o
u
tp
u
t
d
ata,
r
esu
lts
f
r
o
m
u
s
in
g
to
o
f
ew
n
e
u
r
o
n
s
,
w
h
ich
p
r
ev
en
ts
th
e
m
o
d
el
f
r
o
m
co
m
p
r
eh
e
n
d
in
g
th
e
p
atter
n
s
in
th
e
d
ata.
T
h
er
e
f
o
r
e,
d
eter
m
in
i
n
g
t
h
e
b
est
n
u
m
b
er
o
f
n
eu
r
o
n
s
in
ea
ch
lay
er
o
f
th
e
DNN
m
o
d
el
is
ess
en
tial f
o
r
im
p
r
o
v
i
n
g
th
e
m
o
d
el’
s
ac
cu
r
ac
y
in
d
ata
v
alid
ati
o
n
.
Fig
u
r
e
1
2
s
h
o
ws
th
e
g
r
ap
h
f
o
r
th
e
tr
ain
in
g
an
d
v
alid
atio
n
l
o
s
s
wh
en
ap
p
ly
in
g
6
4
n
eu
r
o
n
s
f
o
r
ea
ch
lay
er
o
f
th
e
DNN
m
o
d
el.
I
t
is
o
b
s
er
v
ed
th
at
th
e
v
alid
atio
n
lo
s
s
f
o
r
ea
ch
la
y
er
is
b
elo
w
0
.
6
wh
er
e
th
e
tr
ai
n
in
g
lo
s
s
is
ar
o
u
n
d
0
.
5
.
B
ased
o
n
t
h
e
g
r
a
p
h
,
h
id
d
e
n
lay
er
4
d
em
o
n
s
tr
ates
b
etter
p
er
f
o
r
m
an
ce
co
m
p
ar
ed
t
o
th
e
o
th
er
lay
er
s
.
T
h
is
is
ev
id
en
t f
r
o
m
tr
a
in
in
g
lo
s
s
an
d
v
alid
atio
n
lo
s
s
,
wh
ich
co
n
s
is
ten
tly
d
ec
r
ea
s
e
at
th
e
b
eg
in
n
in
g
a
n
d
s
tab
ilized
with
o
u
t sig
n
if
ican
t f
lu
ctu
atio
n
s
,
in
d
icatin
g
th
at
th
e
m
o
d
el
n
eith
e
r
o
v
e
r
f
its
n
o
r
u
n
d
er
f
its
.
F
i
g
u
r
e
1
2
.
T
r
a
in
i
n
g
an
d
v
a
l
id
a
t
i
o
n
l
o
s
s
f
o
r
L
e
ak
y
-
R
eL
U
,
Si
g
m
o
id
ac
t
i
v
a
t
io
n
l
a
y
er
w
i
th
6
4
n
e
u
r
o
n
s
p
e
r
la
y
er
T
h
e
m
o
d
el'
s
AU
C
s
co
r
e
i
s
d
i
s
p
lay
ed
in
Fig
u
r
e
1
3
f
o
r
d
if
f
e
r
en
t
n
u
m
b
e
r
s
o
f
n
eu
r
o
n
s
in
ea
ch
h
id
d
e
n
lay
er
.
Fig
u
r
e
1
3
(
a
)
d
ep
icts
th
e
AUC
s
co
r
e
wh
en
th
e
m
o
d
el
e
m
p
lo
y
ed
6
4
n
eu
r
o
n
s
in
ea
c
h
l
ay
er
f
o
r
3
,
4
,
5
,
an
d
6
h
id
d
e
n
lay
er
a
r
ch
itectu
r
e.
Nex
t,
Fig
u
r
e
1
3
(
b
)
s
h
o
ws
th
e
m
o
d
el
p
e
r
f
o
r
m
an
ce
wh
e
n
1
2
8
n
eu
r
o
n
s
ar
e
u
s
ed
,
Fig
u
r
e
1
3
(
c)
s
h
o
ws
th
e
p
er
f
o
r
m
an
ce
wh
en
2
5
6
n
e
u
r
o
n
s
ar
e
u
s
ed
an
d
f
in
ally
Fig
u
r
e
1
3
(
d
)
s
h
o
ws
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
m
o
d
el
wh
en
5
1
2
n
eu
r
o
n
s
ar
e
u
s
ed
in
ea
ch
h
id
d
e
n
lay
er
.
T
h
e
h
ig
h
est
av
er
ag
e
AUC
s
co
r
e
is
o
b
tain
ed
wh
en
5
1
2
n
eu
r
o
n
s
ar
e
em
p
lo
y
e
d
in
ea
c
h
lay
er
as
s
h
o
wn
in
Fig
u
r
e
1
3
(
d
)
an
d
T
a
b
le
4
.
T
ab
le
4
d
ep
icts
th
e
av
e
r
ag
e
AUC
s
co
r
e
f
o
r
v
a
r
io
u
s
n
u
m
b
e
r
o
f
n
eu
r
o
n
s
wh
er
e
th
e
h
ig
h
est
av
er
ag
e
is
ac
h
iev
ed
wh
en
u
s
in
g
5
1
2
n
e
u
r
o
n
s
in
ea
ch
h
i
d
d
en
lay
e
r
.
Ho
wev
er
,
th
e
AUC
s
co
r
e
o
b
tain
ed
wh
en
u
s
in
g
3
-
h
id
d
e
n
lay
er
ar
ch
itectu
r
e
with
2
5
6
n
e
u
r
o
n
s
in
ea
ch
lay
e
r
was
also
n
o
tab
ly
h
ig
h
,
r
ea
ch
i
n
g
a
v
alu
e
o
f
0
.
8
2
.
T
h
is
m
ak
es
it
a
g
o
o
d
co
n
te
n
d
er
to
b
e
em
p
lo
y
ed
in
t
h
e
DNN
m
o
d
el
as
lo
wer
n
u
m
b
er
o
f
n
eu
r
o
n
s
m
ay
p
o
ten
tially
r
ed
u
ce
th
e
c
o
m
p
u
t
atio
n
al
co
s
t a
n
d
th
e
t
r
ain
in
g
ti
m
e
o
f
th
e
m
o
d
el.
Evaluation Warning : The document was created with Spire.PDF for Python.