I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
,
p
p
.
1213
~
1
2
2
6
I
SS
N:
2088
-
8
7
0
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijece.
v
1
6
i
3
.
pp
1
2
1
3
-
1
2
2
6
1213
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
O
ptimi
za
tion o
f
t
ra
nsfer learning
f
o
r f
a
cia
l emo
tion
cla
ss
ificatio
n on t
he F
ER
-
2
0
1
3
data
set
Nida
M
u
hli
y
a
B
a
rk
a
h,
Sh
o
f
wa
t
ul ‘
Uy
un
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
m
a
t
i
c
s,
U
n
i
v
e
r
si
t
a
s I
sl
a
m N
e
g
e
r
i
S
u
n
a
n
K
a
l
i
j
a
g
a
Y
o
g
y
a
k
a
r
t
a
,
I
n
d
o
n
e
s
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Oct
1
5
,
2
0
2
5
R
ev
is
ed
J
an
3
0
,
2
0
2
6
Acc
ep
ted
Ma
r
1
6
,
2
0
2
6
F
a
c
ial
e
x
p
re
ss
io
n
s
p
lay
a
k
e
y
ro
le
in
n
o
n
-
v
e
rb
a
l
c
o
m
m
u
n
ica
ti
o
n
b
y
n
a
tu
ra
ll
y
re
flec
ti
n
g
h
u
m
a
n
e
m
o
ti
o
n
s.
F
a
c
ial
e
m
o
ti
o
n
re
c
o
g
n
i
ti
o
n
(F
E
R)
u
sin
g
c
o
m
p
u
ter
v
isi
o
n
h
a
s
g
a
in
e
d
a
t
ten
ti
o
n
with
a
d
v
a
n
c
e
s
in
d
e
e
p
lea
rn
in
g
.
Ho
we
v
e
r,
d
e
e
p
lea
rn
in
g
m
o
d
e
ls
re
q
u
ire
larg
e
d
a
tas
e
ts
to
p
e
rfo
rm
we
ll
,
p
o
sin
g
a
c
h
a
ll
e
n
g
e
fo
r
F
ER
tas
k
s
with
li
m
it
e
d
d
a
ta.
Tra
n
sfe
r
lea
rn
in
g
is
a
p
ro
m
isin
g
a
p
p
ro
a
c
h
t
o
a
d
d
re
ss
th
i
s
issu
e
,
b
u
t
a
sta
n
d
a
rd
ize
d
m
e
th
o
d
fo
r
F
ER
is
y
e
t
to
b
e
e
sta
b
li
sh
e
d
.
T
h
is
stu
d
y
o
p
ti
m
ize
s
th
re
e
tran
sfe
r
lea
rn
in
g
m
o
d
e
ls
Re
sN
e
t
-
5
0
,
In
c
e
p
ti
o
n
V3
,
a
n
d
Xc
e
p
ti
o
n
o
n
th
e
F
ER
-
2
0
1
3
d
a
tas
e
t.
Ex
p
e
rime
n
ts
in
c
l
u
d
e
tes
ti
n
g
i
n
p
u
t
ima
g
e
siz
e
s,
h
y
p
e
rp
a
ra
m
e
ter
tu
n
in
g
,
d
a
ta
a
u
g
m
e
n
tatio
n
,
lay
e
r
a
d
d
it
i
o
n
,
a
n
d
trai
n
in
g
m
e
th
o
d
s.
Re
su
lt
s
s
h
o
w
e
a
c
h
m
o
d
e
l
re
q
u
ires
d
i
ffe
re
n
t
i
n
p
u
t
siz
e
s
fo
r
b
e
st
a
c
c
u
ra
c
y
.
H
y
p
e
r
p
a
ra
m
e
ter
tu
n
i
n
g
imp
ro
v
e
s
a
c
c
u
ra
c
y
b
y
6
.
3
5
%
,
4
.
6
9
%
,
a
n
d
1
.
0
4
%
fo
r
Re
sN
e
t
-
5
0
,
In
c
e
p
ti
o
n
V3
,
a
n
d
Xc
e
p
ti
o
n
,
re
sp
e
c
ti
v
e
ly
.
Au
g
m
e
n
ti
n
g
o
n
ly
t
h
e
d
i
sg
u
st
c
las
s
y
ield
s
b
e
tt
e
r
a
c
c
u
ra
c
y
th
a
n
a
u
g
m
e
n
ti
n
g
a
ll
c
las
se
s.
Th
e
fre
e
z
e
fin
e
-
tu
n
in
g
m
e
th
o
d
is
les
s
e
ffe
c
ti
v
e
th
a
n
fin
e
-
tu
n
i
n
g
a
lo
n
e
o
n
d
a
tas
e
ts
with
th
o
u
sa
n
d
s
o
f
sa
m
p
les
b
u
t
o
u
t
p
e
rfo
rm
s
th
e
fr
e
e
z
e
lay
e
r
m
e
th
o
d
.
Th
e
b
e
st
a
c
c
u
ra
c
ies
a
c
h
iev
e
d
a
re
6
4
.
8
9
%
(Re
sN
e
t
-
5
0
),
6
5
.
8
3
%
(Xc
e
p
ti
o
n
),
a
n
d
6
6
.
4
0
%
(In
c
e
p
ti
o
n
V3
)
.
T
h
e
se
fin
d
in
g
s
p
ro
v
i
d
e
in
si
g
h
ts
in
t
o
fre
e
z
e
fin
e
-
tu
n
in
g
li
m
it
a
ti
o
n
s a
n
d
g
u
i
d
a
n
c
e
fo
r
o
p
ti
m
izin
g
tran
sfe
r
lea
rn
i
n
g
in
F
ER
with
li
m
it
e
d
d
a
ta.
K
ey
w
o
r
d
s
:
Dee
p
lear
n
in
g
E
m
o
tio
n
E
m
o
tio
n
r
ec
o
g
n
itio
n
I
m
ag
e
class
if
icatio
n
T
r
an
s
f
er
lear
n
i
n
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Nid
a
Mu
h
liy
a
B
ar
k
ah
Dep
ar
tm
en
t o
f
I
n
f
o
r
m
atics,
U
n
iv
er
s
itas
I
s
lam
Neg
er
i Su
n
an
Kalijag
a
Yo
g
y
ak
ar
ta
J
l.
L
ak
s
d
a
Ad
is
u
cip
to
,
Dep
o
k
,
Slem
an
,
5
5
2
8
1
,
Dae
r
ah
I
s
tim
ewa
Yo
g
y
ak
ar
ta
E
m
ail:
n
id
am
u
h
liy
a
3
5
@
g
m
ail.
co
m
1.
I
NT
RO
D
UCT
I
O
N
E
m
o
tio
n
s
o
r
f
ac
ial
e
x
p
r
ess
io
n
s
ar
e
cr
u
cial
in
n
o
n
-
v
er
b
al
co
m
m
u
n
icatio
n
an
d
ar
e
o
n
e
o
f
th
e
m
o
s
t
n
atu
r
al
way
s
to
co
n
v
e
y
in
ter
n
al
h
u
m
an
f
ee
lin
g
s
in
p
er
s
o
n
al
in
ter
ac
tio
n
s
[
1
]
.
E
m
o
tio
n
s
ar
e
g
en
er
ated
f
r
o
m
th
e
m
o
v
em
en
ts
o
f
f
ac
ial
m
u
s
cles
an
d
ca
n
b
e
u
s
ed
to
co
n
v
ey
a
r
an
g
e
o
f
s
ig
n
als
i
n
c
o
m
m
u
n
icatio
n
,
f
r
o
m
in
d
icato
r
s
o
f
d
an
g
e
r
to
s
u
b
tle
co
m
m
u
n
icativ
e
s
ig
n
als
[
2
]
.
Mo
s
t
p
s
y
ch
o
lo
g
ical
s
tu
d
ies
s
h
o
w
th
at
h
alf
o
f
th
e
in
f
o
r
m
atio
n
co
n
v
ey
e
d
in
a
co
n
v
er
s
atio
n
c
o
m
es
f
r
o
m
th
e
em
o
tio
n
s
d
is
p
la
y
ed
[
1
]
.
Pau
l
E
k
m
an
,
a
lead
in
g
p
s
y
ch
o
lo
g
is
t
an
d
r
esear
ch
er
in
th
e
f
ield
o
f
em
o
tio
n
p
s
y
ch
o
lo
g
y
a
n
d
f
ac
ial
ex
p
r
ess
io
n
s
,
d
is
co
v
er
ed
th
at
h
u
m
an
s
u
n
i
v
er
s
ally
ex
p
r
ess
th
eir
e
m
o
tio
n
s
t
h
r
o
u
g
h
s
ev
en
d
is
tin
ct
f
ac
ial
ex
p
r
ess
io
n
p
atter
n
s
:
h
ap
p
in
ess
,
s
ad
n
ess
,
an
g
er
,
f
ea
r
,
s
u
r
p
r
is
e,
d
is
g
u
s
t,
an
d
n
eu
tr
alit
y
[
3
]
.
Facial
em
o
tio
n
r
ec
o
g
n
itio
n
is
o
n
e
o
f
t
h
e
r
esear
c
h
b
r
an
ch
es
o
f
f
ac
e
r
ec
o
g
n
itio
n
.
Facial
r
ec
o
g
n
itio
n
is
a
b
io
m
etr
ics
tech
n
o
lo
g
y
th
at
u
s
es
f
ac
e
r
ec
o
g
n
itio
n
alg
o
r
ith
m
s
to
r
ec
o
g
n
ize
an
in
d
i
v
id
u
a
l
b
y
co
m
p
ar
i
n
g
an
in
p
u
t
f
ac
ial
im
ag
e
an
d
a
d
atab
ase
o
f
f
ac
es
to
f
in
d
th
e
f
a
ce
th
at
b
est
m
atch
es
th
e
in
p
u
t
im
a
g
e
[
4
]
.
I
n
f
o
r
m
atio
n
ab
o
u
t
em
o
tio
n
s
o
r
f
ac
ial
ex
p
r
ess
io
n
s
o
b
tain
ed
th
r
o
u
g
h
i
m
ag
e
p
r
o
ce
s
s
in
g
allo
ws
th
e
s
y
s
tem
to
o
p
er
ate
s
im
ilar
ly
to
h
o
w
th
e
h
u
m
a
n
b
r
ain
p
r
o
ce
s
s
es
an
d
r
ec
o
g
n
izes
in
f
o
r
m
atio
n
.
T
h
e
av
aila
b
ilit
y
o
f
th
is
r
ich
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
2
1
3
-
1
2
2
6
1214
in
f
o
r
m
atio
n
p
r
o
v
id
es
cr
u
cial
c
lu
es
f
o
r
th
e
s
y
s
tem
to
in
ter
p
r
e
t
h
u
m
an
in
ten
tio
n
s
.
T
h
er
ef
o
r
e
,
f
ac
ial
ex
p
r
ess
io
n
r
ec
o
g
n
itio
n
h
as
b
r
o
ad
ap
p
licatio
n
s
in
v
ar
io
u
s
f
ield
s
an
d
is
b
ec
o
m
in
g
a
cr
u
cial
co
m
p
o
n
e
n
t
in
th
e
d
ev
elo
p
m
e
n
t
o
f
af
f
ec
tiv
e
co
m
p
u
tin
g
[
1
]
.
E
m
o
tio
n
r
ec
o
g
n
itio
n
is
in
f
lu
en
c
ed
b
y
f
ac
to
r
s
s
u
ch
as
lig
h
tin
g
,
p
o
s
e,
b
ac
k
g
r
o
u
n
d
,
v
iewin
g
an
g
le,
th
e
ca
m
er
a
u
s
ed
to
ca
p
tu
r
e
th
e
im
a
g
e,
an
d
o
cc
lu
s
io
n
,
wh
ich
o
cc
u
r
s
wh
e
n
th
e
f
ac
e
is
b
lo
c
k
ed
b
y
an
o
b
ject
[
5
]
.
E
m
o
tio
n
r
ec
o
g
n
itio
n
ca
n
b
e
p
er
f
o
r
m
e
d
u
s
i
n
g
f
ac
ial
im
a
g
e
d
atasets
,
wh
i
ch
th
en
g
o
th
r
o
u
g
h
p
r
e
-
p
r
o
ce
s
s
in
g
an
d
a
n
aly
s
is
s
tag
es
u
s
in
g
p
atter
n
r
ec
o
g
n
it
io
n
an
d
m
ac
h
in
e
lear
n
in
g
te
ch
n
iq
u
es
s
u
c
h
as
ar
tific
ial
in
tellig
en
ce
(
AI
)
,
c
o
m
p
u
ter
v
is
io
n
,
an
d
d
ee
p
lear
n
i
n
g
[
6
]
.
D
e
e
p
l
e
a
r
n
i
n
g
is
a
b
r
a
n
c
h
o
f
m
a
c
h
i
n
e
l
e
a
r
n
i
n
g
w
h
e
r
e
al
g
o
r
it
h
m
s
a
r
e
i
n
s
p
i
r
e
d
b
y
t
h
e
s
t
r
u
ct
u
r
e
o
f
t
h
e
h
u
m
a
n
b
r
a
i
n
,
l
e
a
d
i
n
g
t
o
t
h
e
c
r
e
a
t
i
o
n
o
f
a
r
t
i
f
i
ci
a
l
n
e
u
r
a
l
n
et
wo
r
k
s
(
A
NN
)
.
D
ee
p
l
ea
r
n
i
n
g
e
n
c
o
m
p
a
s
s
es
s
e
v
e
r
a
l
a
l
g
o
r
i
t
h
m
s
,
i
n
c
l
u
d
i
n
g
l
o
n
g
s
h
o
r
t
t
e
r
m
m
e
m
o
r
y
n
e
t
w
o
r
k
s
(
L
S
T
M
)
,
r
e
c
u
r
r
e
n
t
n
e
u
r
a
l
n
e
t
w
o
r
k
s
(
R
NN
)
,
s
e
l
f
-
o
r
g
a
n
i
z
i
n
g
m
a
p
s
(
S
O
M
)
,
an
d
c
o
n
v
o
l
u
t
i
o
n
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
s
(
C
N
Ns
)
[
7
]
.
C
NN
s
a
r
e
a
t
y
p
e
o
f
d
e
e
p
l
ea
r
n
i
n
g
a
r
t
i
f
i
c
ia
l
n
e
u
r
a
l
n
e
t
w
o
r
k
wi
d
e
ly
u
s
e
d
i
n
d
i
g
i
t
a
l
i
m
a
g
e
a
n
al
y
s
is
,
p
r
o
v
i
n
g
t
o
b
e
h
i
g
h
l
y
e
f
f
e
c
t
i
v
e
i
n
v
a
r
i
o
u
s
p
a
t
t
e
r
n
r
e
c
o
g
n
i
t
i
o
n
a
n
d
i
m
a
g
e
c
l
a
s
s
i
f
ic
a
t
i
o
n
t
a
s
k
s
[
8
]
.
T
r
a
n
s
f
e
r
l
e
a
r
n
i
n
g
i
s
a
n
a
p
p
r
o
a
c
h
i
n
d
e
e
p
l
e
a
r
n
i
n
g
t
h
a
t
u
t
i
l
i
z
es
e
x
i
s
ti
n
g
k
n
o
w
l
e
d
g
e
f
r
o
m
o
n
e
ta
s
k
t
o
as
s
i
s
t
i
n
m
o
d
e
l
t
r
a
i
n
i
n
g
o
n
a
d
i
f
f
e
r
e
n
t
t
as
k
.
I
t
c
a
n
r
e
d
u
c
e
t
h
e
r
e
l
ia
n
c
e
o
n
l
a
r
g
e
a
m
o
u
n
t
s
o
f
l
a
b
el
e
d
d
a
t
a
an
d
t
r
a
i
n
i
n
g
c
o
s
ts
,
a
ll
o
w
i
n
g
t
h
e
a
d
a
p
t
a
t
i
o
n
o
f
t
r
a
i
n
e
d
m
o
d
e
ls
t
o
n
e
w
t
a
s
k
s
[
9
]
.
Mo
s
t
ea
r
ly
r
esear
ch
o
n
f
ac
i
al
em
o
tio
n
r
ec
o
g
n
itio
n
u
s
ed
lab
o
r
ato
r
y
-
d
ev
elo
p
e
d
d
atase
ts
,
s
u
ch
as
J
AFFE
[
1
0
]
an
d
C
K+
[
1
1
]
.
Su
ch
lab
o
r
ato
r
y
d
atasets
h
av
e
th
e
d
is
ad
v
an
tag
e
o
f
b
ein
g
t
o
o
u
n
if
o
r
m
,
as
th
ey
ty
p
ically
in
clu
d
e
o
n
l
y
p
o
s
itiv
e
ex
p
r
ess
io
n
s
with
o
u
t
o
cc
lu
s
io
n
,
wh
ich
m
ak
es
th
em
less
ap
p
licab
le
to
co
m
p
lex
r
ea
l
-
life
s
itu
atio
n
s
.
T
o
o
v
er
c
o
m
e
th
is
p
r
o
b
lem
,
m
an
y
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
s
tu
d
i
es
h
av
e
d
e
v
elo
p
e
d
d
atasets
co
llected
in
u
n
r
estricte
d
,
r
ea
l
-
wo
r
ld
e
n
v
ir
o
n
m
en
t
s
[
1
2
]
,
[
1
3
]
.
Am
o
n
g
all
d
atasets
,
th
e
FER
-
2
0
1
3
d
ataset
is
o
n
e
o
f
th
e
m
o
s
t
co
m
m
o
n
ly
u
s
ed
o
n
es,
as
it
co
n
tain
s
a
lar
g
e
n
u
m
b
er
o
f
f
ac
ial
im
ag
es
ca
p
tu
r
ed
in
an
u
n
co
n
s
tr
ain
e
d
s
ettin
g
b
u
t
s
till
h
as
d
r
awb
ac
k
s
s
u
ch
as
lo
w
i
m
ag
e
r
eso
lu
tio
n
(
o
n
ly
4
8
×
4
8
p
ix
els),
im
b
alan
ce
d
class
d
is
tr
ib
u
tio
n
,
an
d
ex
p
r
ess
io
n
s
ca
n
v
ar
y
g
r
ea
tly
b
etwe
en
in
d
iv
id
u
als.
T
h
is
m
ak
es
th
e
em
o
tio
n
class
if
icatio
n
p
r
o
ce
s
s
m
o
r
e
d
if
f
icu
lt.
T
r
an
s
f
er
lear
n
in
g
is
v
er
y
ef
f
ec
tiv
e
in
a
d
d
r
ess
in
g
t
h
is
ch
allen
g
e,
as
it
allo
ws
m
o
d
els
p
r
e
-
tr
ain
ed
o
n
lar
g
e
d
atasets
to
b
e
ap
p
lied
to
s
m
aller
d
ataset
s
lik
e
FE
R
-
2
0
1
3
,
im
p
r
o
v
in
g
p
er
f
o
r
m
an
ce
d
esp
ite
lim
ited
d
ata.
Sev
er
al
p
r
ev
io
u
s
s
tu
d
ies
h
av
e
s
h
o
wn
th
at
tr
an
s
f
er
lear
n
in
g
is
s
u
p
er
io
r
to
b
u
ild
in
g
a
m
o
d
el
f
r
o
m
s
cr
atch
in
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
.
A
s
tu
d
y
b
y
Hu
n
g
et
a
l
.
[
1
4
]
u
s
in
g
Den
s
e_
Face
L
iv
e
Net
with
a
two
-
s
tag
e
tr
an
s
f
er
lear
n
in
g
ap
p
r
o
ac
h
o
n
th
e
J
AFFE,
KDE
F,
an
d
FER
-
2
0
1
3
d
atasets
s
u
cc
ess
f
u
lly
in
cr
ea
s
ed
th
e
ac
cu
r
ac
y
to
9
1
.
9
3
%,
o
r
1
2
.
9
%
h
i
g
h
er
th
an
with
o
u
t
tr
an
s
f
e
r
lear
n
in
g
.
An
o
th
er
s
tu
d
y
co
m
p
a
r
in
g
two
ap
p
r
o
ac
h
es
o
n
C
NN
Alex
Net
an
d
VGG1
6
with
th
e
R
aFD
d
ataset
also
s
h
o
wed
th
at
tr
a
n
s
f
er
lear
n
in
g
n
o
t
o
n
ly
p
r
o
d
u
ce
d
h
ig
h
er
ac
cu
r
ac
y
(
9
8
.
3
3
%)
b
u
t
also
s
ig
n
if
ican
tly
ac
ce
ler
ated
th
e
tr
ain
i
n
g
tim
e
[
1
5
]
.
T
h
e
u
s
e
o
f
I
n
ce
p
tio
n
V3
an
d
Mo
b
ileNet
-
V2
p
r
etr
ai
n
e
d
m
o
d
els
o
n
th
e
E
m
o
g
n
itio
n
d
ataset
also
p
r
o
d
u
ce
d
b
etter
p
er
f
o
r
m
an
ce
th
an
m
o
d
els
b
u
ilt
f
r
o
m
s
cr
atch
,
with
an
ac
c
u
r
ac
y
o
f
9
6
%
a
n
d
an
F1
-
s
co
r
e
o
f
0
.
9
5
[
1
6
]
.
I
n
ad
d
itio
n
,
t
h
e
d
ev
elo
p
m
e
n
t
o
f
th
e
lig
h
tweig
h
t
R
S
-
Xce
p
tio
n
m
o
d
el
s
h
o
wed
th
at
tr
an
s
f
er
lear
n
in
g
im
p
r
o
v
ed
ef
f
icien
c
y
an
d
ac
cu
r
ac
y
o
n
v
ar
io
u
s
d
atasets
,
in
clu
d
in
g
FER2
0
1
3
a
n
d
R
AF
-
DB
[
1
7
]
.
T
h
ese
f
in
d
in
g
s
c
o
n
f
ir
m
th
at
tr
an
s
f
e
r
lear
n
in
g
is
a
m
o
r
e
ef
f
ec
tiv
e
a
p
p
r
o
ac
h
f
o
r
f
ac
ial
em
o
tio
n
r
e
co
g
n
itio
n
task
s
,
esp
ec
ially
u
n
d
er
lim
ited
d
ata
an
d
co
m
p
lex
e
n
v
ir
o
n
m
en
ts
.
So
m
e
r
esear
ch
o
n
em
o
tio
n
r
ec
o
g
n
itio
n
in
th
e
FER
-
2
0
1
3
d
ataset
u
s
in
g
h
as
b
ee
n
co
n
d
u
cted
p
r
ev
io
u
s
ly
.
T
h
is
in
clu
d
es
a
s
tu
d
y
b
y
Yen
a
n
d
L
i
[
1
]
,
wh
ich
aim
s
to
d
eter
m
in
e
th
e
i
m
p
o
r
tan
ce
o
f
u
s
in
g
tr
an
s
f
er
lear
n
in
g
f
o
r
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
(
FER)
an
d
t
h
e
ef
f
ec
t o
f
tr
ain
in
g
d
atasets
an
d
tr
ain
in
g
ty
p
es o
n
tr
an
s
f
er
lear
n
in
g
.
T
h
e
r
esu
lts
o
f
r
esear
ch
o
n
f
iv
e
tr
a
n
s
f
er
lear
n
in
g
m
o
d
els
s
h
o
w
th
at
class
weig
h
t
is
th
e
o
p
tim
al
tech
n
iq
u
e
f
o
r
b
alan
c
in
g
d
ata,
th
e
f
r
ee
ze
+f
in
e
-
tu
n
i
n
g
tr
ain
in
g
m
eth
o
d
p
r
o
p
o
s
ed
in
th
is
s
tu
d
y
ca
n
im
p
r
o
v
e
ac
cu
r
ac
y
with
o
u
t
b
ein
g
af
f
ec
ted
b
y
d
ata
s
ize,
an
d
m
u
lti
-
s
tag
e
tr
ain
in
g
s
ig
n
if
ican
tly
im
p
r
o
v
es
m
o
d
el
ac
cu
r
ac
y
o
n
th
e
FER
-
2
0
1
3
d
ataset.
R
esear
ch
co
n
d
u
cted
b
y
Sh
ah
za
d
et
a
l.
[
1
8
]
aim
s
to
im
p
r
o
v
e
FER
p
er
f
o
r
m
an
ce
with
a
z
o
n
in
g
-
b
a
s
ed
m
eth
o
d
(
Z
FER)
th
at
ex
tr
a
cts
an
d
d
iv
id
es
f
ac
ial
r
ef
e
r
en
c
e
p
o
in
ts
in
to
f
o
u
r
r
eg
io
n
s
,
u
s
in
g
VGG
-
1
6
an
d
f
u
lly
co
n
n
ec
ted
n
e
u
r
al
n
etwo
r
k
(
FC
NN)
m
o
d
els
f
o
r
em
o
tio
n
class
if
icatio
n
.
As
a
r
esu
lt,
th
e
m
eth
o
d
ac
h
iev
e
d
9
8
.
4
%
ac
cu
r
ac
y
o
n
th
e
C
K+
d
ataset
an
d
6
5
%
o
n
FER
-
2
0
1
3
,
with
zo
n
in
g
in
cr
ea
s
in
g
th
e
ac
cu
r
ac
y
f
r
o
m
9
8
.
4
7
%
to
9
8
.
7
4
%
o
n
C
K+
.
R
esear
ch
co
n
d
u
cted
b
y
Ur
n
is
h
a
et
a
l.
[
1
9
]
f
o
cu
s
es
o
n
im
p
r
o
v
in
g
FER
u
s
in
g
th
e
t
r
an
s
f
er
lear
n
in
g
m
eth
o
d
with
Mo
b
ileNetV2
ar
ch
itectu
r
e
.
T
h
e
r
esu
lts
s
h
o
wed
an
ac
cu
r
ac
y
o
f
9
9
%
o
n
r
an
d
o
m
i
m
ag
es
an
d
v
id
e
o
clip
s
,
an
d
an
ac
cu
r
ac
y
v
al
u
e
o
f
6
1
%
o
n
t
h
e
FER
-
2
0
1
3
d
ataset,
s
h
o
win
g
p
r
o
g
r
ess
in
r
ea
l
-
tim
e
f
ac
ial
ex
p
r
ess
io
n
r
ec
o
g
n
itio
n
.
Sev
er
al
p
r
ev
io
u
s
s
tu
d
ies
h
av
e
p
r
o
v
e
n
th
at
tr
an
s
f
er
lear
n
in
g
an
d
th
e
u
s
e
o
f
m
u
ltip
le
tr
ai
n
in
g
m
o
d
els
c
an
im
p
r
o
v
e
ac
c
u
r
ac
y
o
n
th
e
FER
2
0
1
3
d
ataset.
Su
m
m
ar
y
o
f
p
r
e
v
io
u
s
r
esear
ch
r
elate
d
to
f
ac
ial
r
ec
o
g
n
itio
n
c
an
b
e
s
ee
n
in
T
ab
le
1
.
T
h
is
s
tu
d
y
p
r
o
p
o
s
es
an
o
p
ti
m
izatio
n
o
f
th
r
ee
tr
an
s
f
er
l
ea
r
n
in
g
:
R
esNet
-
5
0
,
I
n
ce
p
tio
n
V3
an
d
Xce
p
tio
n
f
o
r
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
o
n
FER
2
0
1
3
d
ata
s
et
b
y
test
in
g
v
ar
i
o
u
s
s
tr
ateg
i
es,
s
u
ch
as
im
ag
e
r
esizin
g
,
h
y
p
e
r
p
ar
am
ete
r
tu
n
in
g
,
d
ata
a
u
g
m
en
tatio
n
,
a
n
d
th
e
ad
d
itio
n
o
f
d
r
o
p
o
u
t
lay
er
s
an
d
b
atch
n
o
r
m
aliza
tio
n
.
I
n
ad
d
itio
n
,
tr
a
in
in
g
m
eth
o
d
s
s
u
ch
as
f
in
e
-
tu
n
in
g
,
f
r
ee
zin
g
,
a
n
d
f
r
ee
ze
f
in
e
-
tu
n
in
g
a
r
e
ap
p
lied
to
th
e
p
r
e
-
tr
ain
ed
m
o
d
els to
id
en
tify
th
e
b
est co
m
b
in
atio
n
f
o
r
ac
h
iev
in
g
th
e
h
ig
h
est ac
cu
r
a
cy
.
Un
lik
e
p
r
ev
io
u
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
fo
r
fa
cia
l e
mo
tio
n
cla
s
s
ific
a
tio
n
o
n
…
(
N
id
a
Mu
h
liya
B
a
r
ka
h
)
1215
s
tu
d
ies
th
at
co
n
s
id
er
ed
f
r
ee
ze
f
in
e
-
tu
n
in
g
to
b
e
s
u
p
er
io
r
,
th
e
f
in
d
in
g
s
r
ev
ea
l
th
at
th
is
m
eth
o
d
is
less
ef
f
ec
tiv
e
o
n
s
m
all
d
atasets
lik
e
FER
-
2
0
1
3
.
T
h
ese
r
esu
lts
p
r
o
v
i
d
e
n
ew
in
s
ig
h
ts
in
to
th
e
lim
itatio
n
s
o
f
f
r
ee
ze
f
in
e
-
tu
n
in
g
an
d
o
f
f
er
p
r
ac
tical
g
u
id
a
n
ce
f
o
r
o
p
tim
izin
g
t
r
an
s
f
er
lear
n
in
g
i
n
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
ta
s
k
s
.
T
ab
le
1
.
Su
m
m
a
r
y
o
f
p
r
ev
io
u
s
r
esear
ch
No
A
u
t
h
o
r
s
(
Y
e
a
r
)
M
e
t
h
o
d
D
a
t
a
s
e
t
I
n
p
u
t
t
y
p
e
A
c
c
u
r
a
c
y
/
N
o
t
e
s
1
[
1
4
]
D
e
n
se
_
F
a
c
e
L
i
v
e
N
e
t
w
i
t
h
t
w
o
-
st
a
g
e
t
r
a
n
sf
e
r
l
e
a
r
n
i
n
g
JA
F
F
E,
K
D
EF,
F
ER
-
2
0
1
3
I
mag
e
,
V
i
d
e
o
A
c
c
u
r
a
c
y
9
1
.
9
3
%
,
1
2
.
9
%
h
i
g
h
e
r
t
h
a
n
w
i
t
h
o
u
t
t
r
a
n
sf
e
r
l
e
a
r
n
i
n
g
2
[
1
5
]
C
N
N
(
A
l
e
x
N
e
t
,
V
G
G
1
6
)
w
i
t
h
t
r
a
n
sf
e
r
l
e
a
r
n
i
n
g
R
a
F
D
I
mag
e
Tr
a
n
sf
e
r
l
e
a
r
n
i
n
g
:
A
c
c
u
r
a
c
y
9
8
.
3
3
%
,
f
a
st
e
r
t
r
a
i
n
i
n
g
t
i
me
t
h
a
n
f
r
o
m s
c
r
a
t
c
h
3
[
1
6
]
I
n
c
e
p
t
i
o
n
V
3
,
M
o
b
i
l
e
N
e
t
V
2
(
p
r
e
t
r
a
i
n
e
d
)
Emo
g
n
i
t
i
o
n
V
i
d
e
o
t
o
I
mag
e
A
c
c
u
r
a
c
y
9
6
%,
F
1
-
sc
o
r
e
0
.
9
5
,
o
u
t
p
e
r
f
o
r
me
d
o
p
t
i
m
i
z
e
d
m
o
d
e
l
s fr
o
m s
c
r
a
t
c
h
4
[
1
7
]
Li
g
h
t
w
e
i
g
h
t
R
S
-
X
c
e
p
t
i
o
n
(
t
r
a
n
sf
e
r
l
e
a
r
n
i
n
g
-
b
a
se
d
)
F
ER
-
2
0
1
3
,
C
K
+
,
B
i
g
f
e
r
2
0
1
3
,
R
A
F
-
DB
I
mag
e
Tr
a
n
sf
e
r
l
e
a
r
n
i
n
g
i
m
p
r
o
v
e
d
a
c
c
u
r
a
c
y
a
n
d
e
f
f
i
c
i
e
n
c
y
a
c
r
o
ss
d
a
t
a
set
s
5
[
1
]
X
c
e
p
t
i
o
n
,
E
f
f
i
c
i
e
n
t
N
e
t
-
B
0
,
I
n
c
e
p
t
i
o
n
,
R
e
sN
e
t
-
5
0
,
D
e
n
seN
e
t
-
1
2
1
F
ER
-
2
0
1
3
I
mag
e
F
r
e
e
z
e
+
f
i
n
e
-
t
u
n
i
n
g
i
m
p
r
o
v
e
s
a
c
c
u
r
a
c
y
;
c
l
a
ss
w
e
i
g
h
t
s e
f
f
e
c
t
i
v
e
f
o
r
b
a
l
a
n
c
i
n
g
6
[
1
8
]
ZFER
w
i
t
h
V
G
G
-
1
6
a
n
d
F
C
N
N
C
K
+
,
F
ER
-
2
0
1
3
I
mag
e
9
8
.
4
%
(
C
K
+
)
,
6
5
%
(
F
E
R
-
2
0
1
3
)
;
z
o
n
i
n
g
i
mp
r
o
v
e
d
C
K
+
a
c
c
u
r
a
c
y
t
o
9
8
.
7
4
%
7
[
1
9
]
M
o
b
i
l
e
N
e
t
V
2
(
t
r
a
n
sf
e
r
l
e
a
r
n
i
n
g
)
F
ER
-
2
0
1
3
I
mag
e
,
V
i
d
e
o
9
9
%
(
r
a
n
d
o
m
i
m
a
g
e
s/
v
i
d
e
o
)
,
6
1
%
(
F
ER
-
2
0
1
3
)
;
e
f
f
e
c
t
i
v
e
f
o
r
r
e
a
l
-
t
i
me
r
e
c
o
g
n
i
t
i
o
n
8
Th
i
s
st
u
d
y
(
P
r
o
p
o
s
e
d
)
R
e
sN
e
t
-
5
0
,
I
n
c
e
p
t
i
o
n
V
3
,
X
c
e
p
t
i
o
n
F
ER
-
2
0
1
3
I
mag
e
O
p
t
i
m
i
z
a
t
i
o
n
t
h
r
o
u
g
h
m
u
l
t
i
p
l
e
st
r
a
t
e
g
i
e
s
:
i
)
i
n
p
u
t
si
z
e
t
e
st
i
n
g
,
ii
)
h
y
p
e
r
p
a
r
a
me
t
e
r
t
u
n
i
n
g
,
i
i
i
)
a
u
g
me
n
t
a
t
i
o
n
sch
e
mes,
iv
)
a
d
d
i
t
i
o
n
a
l
a
r
c
h
i
t
e
c
t
u
r
a
l
l
a
y
e
r
s
,
a
n
d
v
)
e
v
a
l
u
a
t
i
o
n
o
f
t
h
r
e
e
t
r
a
i
n
i
n
g
m
e
t
h
o
d
s
2.
B
ACK
G
RO
UND
S
T
UD
Y
2
.
1
.
E
m
o
t
io
n r
ec
o
g
nitio
n
Facial
em
o
tio
n
s
an
d
ex
p
r
ess
io
n
s
ar
e
cr
u
cial
in
n
o
n
-
v
er
b
al
co
m
m
u
n
icatio
n
an
d
s
er
v
e
as
a
n
atu
r
al
way
to
co
n
v
ey
h
u
m
an
in
ter
n
al
f
ee
lin
g
s
in
p
er
s
o
n
al
in
ter
ac
tio
n
s
[
1
]
.
E
m
o
tio
n
s
ar
is
e
f
r
o
m
th
e
m
o
v
em
en
ts
o
f
f
ac
ial
m
u
s
cles
an
d
ca
n
c
o
m
m
u
n
icate
v
ar
io
u
s
s
ig
n
als
in
co
m
m
u
n
ica
tio
n
,
f
r
o
m
war
n
in
g
s
to
s
u
b
tle
cu
es.
Fo
r
in
s
tan
ce
,
r
aisi
n
g
ey
eb
r
o
ws
o
r
f
u
r
r
o
win
g
th
e
b
r
o
w
d
u
r
in
g
a
c
o
n
v
er
s
ati
o
n
ca
n
co
n
v
ey
m
ess
ag
es
with
o
u
t
wo
r
d
s
[
2
]
,
Su
ch
ex
p
r
ess
io
n
s
h
elp
clar
if
y
em
o
tio
n
s
,
in
ten
tio
n
s
,
o
r
f
ee
lin
g
s
an
d
ca
n
s
tr
en
g
th
en
o
r
co
m
p
lem
en
t
v
er
b
al
co
m
m
u
n
icatio
n
.
Stu
d
ies
in
p
s
y
ch
o
lo
g
y
in
d
icate
th
at
a
b
o
u
t
h
alf
o
f
th
e
in
f
o
r
m
atio
n
ex
c
h
an
g
ed
in
c
o
n
v
e
r
s
atio
n
s
co
m
es
f
r
o
m
d
is
p
lay
e
d
em
o
tio
n
s
[
1
]
.
Fo
r
ex
a
m
p
le,
a
co
n
v
er
s
atio
n
ac
co
m
p
an
ied
b
y
a
h
a
p
p
y
o
r
s
ad
f
ac
e
ca
n
s
ig
n
if
ican
tly
in
f
lu
en
ce
h
o
w
th
e
lis
ten
er
r
ec
eiv
es
th
e
m
ess
ag
e.
R
en
o
wn
ed
p
s
y
ch
o
lo
g
is
t
Pau
l
E
k
m
an
d
is
co
v
er
ed
th
at
h
u
m
an
s
u
n
iv
er
s
ally
ex
p
r
ess
em
o
tio
n
s
th
r
o
u
g
h
s
ev
en
s
im
ilar
f
ac
ial
ex
p
r
ess
io
n
s
:
h
ap
p
in
ess
,
s
ad
n
ess
,
an
g
er
,
f
ea
r
,
s
u
r
p
r
is
e,
d
is
g
u
s
t,
an
d
n
e
u
tr
ality
[
3
]
.
E
m
o
tio
n
r
ec
o
g
n
itio
n
is
in
f
lu
e
n
ce
d
b
y
f
ac
to
r
s
lik
e
lig
h
tin
g
,
p
o
s
e,
b
ac
k
g
r
o
u
n
d
,
p
e
r
s
p
ec
tiv
e,
ca
m
er
a
q
u
ality
,
an
d
o
cc
lu
s
io
n
,
w
h
er
e
a
f
ac
e
is
p
ar
tially
o
b
s
tr
u
ct
ed
b
y
an
o
th
er
o
b
ject.
T
h
e
a
cc
u
r
ac
y
o
f
e
m
o
tio
n
r
ec
o
g
n
itio
n
is
lar
g
ely
d
ep
en
d
e
n
t
o
n
t
h
e
p
r
o
ce
s
s
in
g
ca
p
a
b
ilit
ies
o
f
th
e
v
is
u
al
r
ec
o
g
n
itio
n
s
y
s
tem
,
s
u
p
p
o
r
ted
b
y
h
o
w
in
f
o
r
m
atio
n
is
u
n
d
e
r
s
to
o
d
an
d
p
r
o
ce
s
s
ed
[
5
]
.
E
m
o
tio
n
r
ec
o
g
n
itio
n
ca
n
b
e
p
e
r
f
o
r
m
ed
u
s
in
g
f
ac
ial
im
ag
e
d
atasets
,
wh
ich
u
n
d
er
g
o
p
r
e
p
r
o
ce
s
s
in
g
an
d
an
aly
s
is
th
r
o
u
g
h
p
atter
n
r
ec
o
g
n
itio
n
tech
n
iq
u
es
an
d
m
ac
h
in
e
lear
n
in
g
m
eth
o
d
s
lik
e
co
m
p
u
t
er
v
is
io
n
,
ar
tific
ial
in
tellig
en
c
e,
an
d
d
ee
p
lea
r
n
in
g
[
6
]
.
R
ec
o
g
n
izin
g
em
o
tio
n
s
is
ess
en
tial
as
it
en
h
an
ce
s
th
e
q
u
ality
o
f
in
ter
ac
tio
n
s
b
etwe
en
h
u
m
an
s
an
d
co
m
p
u
ter
s
,
a
p
p
licab
le
in
v
ar
io
u
s
f
ield
s
.
I
n
to
u
r
is
m
,
AI
-
b
ased
f
a
cial
em
o
tio
n
r
ec
o
g
n
itio
n
ca
n
h
elp
ass
ess
to
u
r
is
ts
'
s
atis
f
ac
tio
n
o
r
d
is
s
atis
f
ac
tio
n
in
r
ea
l
-
tim
e
b
y
an
aly
zin
g
f
a
cial
ex
p
r
ess
io
n
s
,
allo
win
g
f
o
r
m
o
r
e
ac
cu
r
ate
r
esp
o
n
s
e
ad
ju
s
tm
en
ts
,
s
u
ch
as
o
f
f
er
in
g
ad
d
itio
n
al
s
o
lu
tio
n
s
wh
en
n
eg
ativ
e
em
o
tio
n
s
ar
e
d
etec
ted
to
en
h
an
ce
c
u
s
to
m
er
ex
p
er
ien
ce
[
2
0
]
.
I
n
h
ea
lth
ca
r
e,
em
o
tio
n
r
ec
o
g
n
iti
o
n
tech
n
o
lo
g
y
aid
s
in
m
o
n
it
o
r
in
g
p
atien
ts
'
em
o
tio
n
al
co
n
d
itio
n
s
,
esp
ec
ially
th
o
s
e
with
f
ac
ial
p
ar
al
y
s
is
,
f
ac
ilit
atin
g
d
iag
n
o
s
is
an
d
tr
ea
tm
e
n
t
ad
ju
s
tm
en
ts
[
2
1
]
.
I
n
e
d
u
ca
ti
o
n
,
f
ac
ial
e
m
o
tio
n
r
ec
o
g
n
itio
n
h
el
p
s
ass
ess
s
tu
d
en
t
en
g
a
g
em
en
t
in
r
ea
l
-
tim
e,
e
n
ab
lin
g
ed
u
ca
to
r
s
to
ad
ju
s
t
te
ac
h
in
g
m
eth
o
d
s
an
d
m
ater
ials
to
im
p
r
o
v
e
ef
f
ec
tiv
en
ess
an
d
ac
ad
em
ic
r
esu
lts
[
2
2
]
.
Ov
er
all,
th
is
tech
n
o
lo
g
y
en
ab
les
co
m
p
u
te
r
s
y
s
tem
s
to
r
esp
o
n
d
to
u
s
er
em
o
tio
n
s
m
o
r
e
h
u
m
a
n
ely
.
2
.
2
.
T
ra
ns
f
er
l
ea
rning
T
r
an
s
f
er
lear
n
in
g
is
a
m
ac
h
in
e
lear
n
in
g
ap
p
r
o
ac
h
th
at
en
a
b
les
th
e
u
s
e
o
f
a
m
o
d
el
tr
ain
ed
f
o
r
o
n
e
s
p
ec
if
ic
task
as
a
s
tar
tin
g
p
o
in
t
f
o
r
ad
d
r
ess
in
g
an
o
th
e
r
r
ela
ted
task
.
T
h
is
m
eth
o
d
is
esp
e
cially
ad
v
an
tag
e
o
u
s
wh
en
ex
is
tin
g
p
r
o
ce
d
u
r
es
th
at
tack
le
th
e
m
ain
is
s
u
e
ar
e
av
a
ilab
le,
an
d
th
e
n
ew
task
d
em
a
n
d
s
a
lar
g
e
am
o
u
n
t
o
f
d
ata
[
2
3
]
,
I
t
is
f
r
eq
u
e
n
tly
u
tili
ze
d
ac
r
o
s
s
v
ar
io
u
s
m
o
d
el
ty
p
es,
p
ar
ticu
lar
ly
C
NNs
in
im
ag
e
r
ec
o
g
n
itio
n
task
s
.
I
n
tr
an
s
f
er
lear
n
in
g
,
th
e
f
ea
tu
r
es
f
r
o
m
th
e
p
r
e
-
tr
ain
e
d
m
o
d
el
ar
e
ex
tr
ac
ted
,
wh
ich
m
ea
n
s
th
at
tr
ain
in
g
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
2
1
3
-
1
2
2
6
1216
ca
n
co
m
m
en
ce
with
o
u
t
s
tar
ti
n
g
f
r
o
m
s
cr
atch
.
T
y
p
ically
,
tr
an
s
f
er
lear
n
i
n
g
m
o
d
els
ar
e
tr
ain
ed
o
n
e
x
ten
s
iv
e
d
atasets
,
an
d
th
e
p
a
r
am
eter
s
d
er
iv
ed
f
r
o
m
th
ese
m
o
d
els
ca
n
b
e
u
tili
ze
d
in
s
p
ec
ialized
n
eu
r
al
n
etwo
r
k
s
f
o
r
o
th
er
s
im
ilar
task
s
.
T
h
is
allo
w
s
th
e
m
o
d
el
to
b
e
d
ir
ec
tly
em
p
lo
y
ed
f
o
r
m
ak
in
g
p
r
ed
ictio
n
s
in
n
ew
task
s
o
r
f
o
r
tr
ain
in
g
in
r
elate
d
ap
p
licatio
n
s
.
2
.
2
.
1
.
ResNet
-
50
R
esNet
-
50
[
2
4
]
is
a
p
o
p
u
lar
C
NN
m
o
d
el
d
esig
n
ed
to
ad
d
r
ess
n
etwo
r
k
d
eg
r
a
d
atio
n
in
d
ee
p
ar
ch
itectu
r
es.
I
ts
s
k
ip
co
n
n
ec
t
io
n
s
allo
w
in
p
u
ts
to
s
k
ip
ce
r
tain
lay
er
s
,
h
elp
in
g
to
r
ed
u
ce
v
an
is
h
in
g
g
r
ad
ien
ts
an
d
o
v
er
f
itti
n
g
.
T
h
is
d
esig
n
co
n
tr
ib
u
ted
to
its
s
u
cc
ess
in
win
n
in
g
th
e
2
0
1
5
I
m
a
g
eNe
t
ch
allen
g
e
.
2
.
2
.
2
.
Xce
ptio
n
Xce
p
tio
n
[
2
5
]
is
a
m
o
d
e
r
n
C
N
N
m
o
d
el
th
at
c
o
m
b
in
es
c
o
n
ce
p
ts
f
r
o
m
Go
o
g
leNe
t
an
d
R
esNet.
Un
lik
e
th
o
s
e
two
m
o
d
els,
Xce
p
tio
n
u
s
es
a
s
ep
ar
ab
le
co
n
v
o
lu
tio
n
lay
er
th
at
s
ep
ar
ates
s
p
atial
an
d
cr
o
s
s
-
ch
an
n
el
p
atter
n
s
.
T
h
is
r
e
d
u
ce
s
co
m
p
u
t
atio
n
al
co
m
p
le
x
ity
,
n
u
m
b
e
r
o
f
p
ar
am
eter
s
,
a
n
d
m
em
o
r
y
u
s
a
g
e
wh
ile
im
p
r
o
v
in
g
m
o
d
el
p
er
f
o
r
m
a
n
ce
.
2
.
2
.
3
.
I
ncept
io
n
I
n
ce
p
tio
n
[
2
6
]
o
r
Go
o
g
leNe
t,
was
d
ev
elo
p
e
d
b
y
Go
o
g
le
to
o
p
tim
ize
p
a
r
am
eter
u
s
ag
e
in
n
etwo
r
k
s
.
T
h
e
m
o
d
el
u
tili
ze
s
an
ef
f
icie
n
t
in
ce
p
tio
n
m
o
d
u
le,
m
i
n
im
izin
g
th
e
n
u
m
b
er
o
f
p
ar
am
ete
r
s
with
o
u
t
s
ac
r
if
icin
g
ac
cu
r
ac
y
.
I
n
ce
p
tio
n
V
3
is
o
n
e
o
f
th
e
m
o
s
t f
r
eq
u
e
n
tly
u
s
ed
v
a
r
ian
ts
in
f
ac
ial
em
o
tio
n
r
ec
o
g
n
itio
n
.
3.
M
E
T
H
O
D
T
h
is
s
tu
d
y
aim
s
to
o
p
tim
ize
tr
an
s
f
er
lear
n
in
g
f
o
r
f
ac
ial
e
m
o
tio
n
class
if
icatio
n
o
n
th
e
FER
-
2
0
1
3
d
ataset
b
y
u
tili
zin
g
th
r
ee
p
r
e
-
t
r
ain
ed
m
o
d
els:
R
es
Net
-
5
0
,
I
n
ce
p
tio
n
V3
,
an
d
Xce
p
tio
n
.
T
h
e
ex
p
er
im
en
ts
wer
e
co
n
d
u
cte
d
d
ir
ec
tly
o
n
Kag
g
le
n
o
teb
o
o
k
s
with
P1
0
0
G
PU
ac
ce
ler
ato
r
.
T
h
e
lo
ca
l
m
ac
h
in
e
u
s
ed
f
o
r
p
r
ep
r
o
ce
s
s
in
g
an
d
m
in
o
r
c
o
m
p
u
tatio
n
s
h
ad
t
h
e
f
o
llo
win
g
s
p
ec
if
icatio
n
s
:
I
n
tel
(R)
C
o
r
e
(T
M)
i5
-
8
2
6
5
U,
4
GB
R
AM
an
d
r
u
n
n
in
g
in
W
in
d
o
ws
1
1
.
All
m
o
d
els
wer
e
im
p
lem
en
ted
u
s
in
g
T
en
s
o
r
Flo
w
with
Ker
as
API
,
an
d
ad
d
itio
n
al
Py
th
o
n
lib
r
ar
ies
in
clu
d
ed
Nu
m
Py
,
Pan
d
as
an
d
Scik
it
-
L
ea
r
n
.
T
h
e
o
p
tim
izatio
n
p
r
o
ce
s
s
is
ca
r
r
ied
o
u
t
in
s
ev
er
al
s
tag
es
an
d
ap
p
lied
to
ea
ch
m
o
d
el.
T
h
e
f
i
r
s
t
s
tag
e
in
v
o
lv
es
test
in
g
th
r
ee
v
ar
iatio
n
s
o
f
in
p
u
t
im
ag
e
s
izes
to
d
eter
m
in
e
th
e
m
o
s
t
s
u
itab
le
r
eso
lu
tio
n
f
o
r
ea
ch
m
o
d
el.
T
h
e
s
ec
o
n
d
s
tag
e
f
o
cu
s
es
o
n
tu
n
in
g
f
o
u
r
k
ey
h
y
p
e
r
p
ar
a
m
eter
s
:
lear
n
in
g
r
ate,
b
atc
h
s
ize,
o
p
tim
izer
,
an
d
e
p
o
ch
s
,
aim
in
g
to
f
in
d
th
e
b
est
co
m
b
in
atio
n
f
o
r
e
a
c
h
a
r
c
h
i
t
e
c
t
u
r
e
.
T
h
e
t
h
i
r
d
s
t
a
g
e
a
p
p
l
i
e
s
d
a
t
a
a
u
g
m
e
n
t
a
t
i
o
n
t
o
a
d
d
r
e
s
s
t
h
e
i
m
b
a
l
a
n
c
e
o
f
t
h
e
d
a
t
a
s
et
w
it
h
t
w
o
t
y
p
e
s
o
f
s
c
h
em
e
s
,
a
p
p
l
ie
d
o
n
l
y
t
o
t
h
e
m
i
n
o
r
i
t
y
c
l
as
s
a
n
d
a
p
p
l
i
e
d
t
o
a
l
l
c
la
s
s
e
s
.
I
n
t
h
e
f
o
u
r
t
h
s
t
a
g
e
,
a
r
c
h
i
te
c
t
u
r
a
l
m
o
d
i
f
i
ca
t
i
o
n
s
a
r
e
m
a
d
e
b
y
a
d
d
i
n
g
d
r
o
p
o
u
t
l
a
y
e
r
s
a
n
d
a
c
o
m
b
i
n
a
t
i
o
n
o
f
d
r
o
p
o
u
t
l
a
y
e
r
s
a
n
d
b
a
t
c
h
n
o
r
m
a
l
i
z
at
i
o
n
t
o
i
m
p
r
o
v
e
m
o
d
e
l
g
e
n
e
r
a
li
z
a
ti
o
n
a
n
d
s
t
ab
i
l
i
t
y
.
T
h
e
f
i
n
a
l
s
ta
g
e
t
es
ts
t
h
r
e
e
d
i
f
f
e
r
e
n
t
t
r
a
i
n
i
n
g
m
e
t
h
o
d
s
t
o
e
v
a
l
u
a
te
t
h
ei
r
i
m
p
ac
t
o
n
m
o
d
e
l
p
e
r
f
o
r
m
a
n
c
e
.
T
h
e
b
e
s
t
r
es
u
lt
s
f
r
o
m
e
a
c
h
s
ta
g
e
s
er
v
e
a
s
t
h
e
b
as
is
f
o
r
t
h
e
n
e
x
t
s
ta
g
e
,
e
n
s
u
r
i
n
g
t
h
a
t
t
h
e
o
p
t
i
m
a
l
c
o
m
b
i
n
a
ti
o
n
o
f
s
e
tt
i
n
g
s
a
n
d
a
r
c
h
i
t
e
ct
u
r
e
s
is
a
c
h
ie
v
ed
f
o
r
e
a
c
h
m
o
d
e
l
.
All
e
x
p
e
r
im
e
n
ts
we
r
e
im
p
l
e
m
e
n
te
d
u
s
i
n
g
th
e
T
en
s
o
r
Fl
o
w
a
n
d
Ke
r
as
f
r
a
m
ew
o
r
k
s
in
Py
t
h
o
n
.
T
h
e
ex
p
e
r
i
m
e
n
ts
w
er
e
co
n
d
u
ct
ed
u
s
in
g
Ka
g
g
le
N
o
te
b
o
o
k
s
wi
th
t
h
e
GPU
P
1
0
0
a
cc
e
le
r
a
to
r
,
s
u
p
p
o
r
t
e
d
b
y
a
p
e
r
s
o
n
al
lap
to
p
wi
th
a
n
I
n
t
el
(
R)
C
o
r
e
(TM
)
i5
-
8
2
6
5
U
p
r
o
ce
s
s
o
r
,
4
GB
R
AM
,
a
n
d
W
i
n
d
o
ws
1
1
o
p
e
r
ati
n
g
s
y
s
t
em
.
I
d
e
n
ti
ca
l
d
at
a
s
p
l
its
an
d
p
r
e
p
r
o
ce
s
s
in
g
s
tep
s
w
er
e
a
p
p
li
e
d
ac
r
o
s
s
al
l
m
o
d
els
t
o
e
n
s
u
r
e
f
ai
r
c
o
m
p
a
r
i
s
o
n
o
f
p
e
r
f
o
r
m
an
ce
.
Fig
u
r
e
1
i
ll
u
s
tr
at
es
t
h
e
o
v
e
r
a
l
l
r
esea
r
c
h
s
ta
g
es
a
n
d
e
x
p
er
im
en
t
al
s
et
u
p
u
s
e
d
i
n
t
h
is
s
tu
d
y
,
p
r
o
v
i
d
i
n
g
a
cle
ar
d
e
p
i
cti
o
n
o
f
t
h
e
w
o
r
k
f
lo
w
i
m
p
lem
en
te
d
f
o
r
o
p
ti
m
iz
in
g
t
r
a
n
s
f
er
le
ar
n
i
n
g
o
n
t
h
e
FER
-
2
0
1
3
d
atas
et.
Fig
u
r
e
1
.
E
x
p
er
im
e
n
tal
s
etu
p
an
d
r
esear
ch
s
tag
es o
f
p
r
o
p
o
s
e
d
m
eth
o
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
fo
r
fa
cia
l e
mo
tio
n
cla
s
s
ific
a
tio
n
o
n
…
(
N
id
a
Mu
h
liya
B
a
r
ka
h
)
1217
3
.
1
.
Da
t
a
s
et
FER
-
2
0
1
3
is
a
d
ataset
in
tr
o
d
u
ce
d
at
a
n
i
n
ter
n
atio
n
al
co
n
f
er
en
ce
o
n
m
ac
h
in
e
lear
n
in
g
in
2
0
1
3
[
2
7
]
.
I
t
ca
n
b
e
ac
ce
s
s
ed
f
o
r
f
r
ee
th
r
o
u
g
h
th
e
Kag
g
le
web
s
ite.
T
h
e
FER
-
2
0
1
3
d
ataset
co
n
s
is
ts
o
f
3
5
,
8
8
7
im
ag
es
with
g
r
ay
s
ca
le
m
o
d
e
an
d
an
i
m
ag
e
s
ize
o
f
4
8
×
4
8
p
ix
els.
T
h
e
FER
-
2
0
1
3
d
ataset
is
d
iv
id
ed
in
to
tr
ai
n
in
g
d
ata
(
2
7
,
8
0
9
im
ag
es)
an
d
test
in
g
d
ata
(
7
,
1
7
8
im
a
g
es),
with
ea
ch
s
et
f
u
r
th
er
ca
teg
o
r
ized
in
to
s
ev
en
em
o
tio
n
class
es
:
an
g
r
y
,
d
is
g
u
s
t,
f
ea
r
,
h
ap
p
y
,
n
e
u
tr
al,
s
ad
,
an
d
s
u
r
p
r
is
e.
A
s
am
p
le
o
f
FER
-
2
0
1
3
f
ac
ial
em
o
tio
n
im
ag
es
is
s
h
o
wn
in
Fig
u
r
e
2
an
d
th
e
d
is
tr
ib
u
tio
n
o
f
im
ag
e
s
in
th
e
tr
ain
in
g
an
d
test
in
g
d
ata
ca
n
b
e
s
ee
n
in
T
ab
le
2
.
An
g
r
y
Dis
g
u
s
t
Fear
Hap
p
y
Neu
tr
al
Sad
Su
r
p
r
is
e
Fig
u
r
e
2
.
Sam
p
le
f
ac
ial
em
o
ti
o
n
im
ag
es
f
r
o
m
th
e
FER
-
2
0
1
3
T
ab
le
2
.
Data
s
et
d
escr
ip
tio
n
C
l
a
s
s
Tr
a
i
n
i
n
g
Te
st
i
n
g
A
n
g
r
y
3
.
9
9
5
9
5
8
D
i
sg
u
st
4
3
6
1
1
1
F
e
a
r
4
.
0
9
7
1
.
0
2
4
H
a
p
p
y
7
.
2
1
5
1
.
7
7
4
N
e
u
t
r
a
l
4
.
9
6
5
1
.
2
3
3
S
a
d
4
.
8
3
0
1
.
2
4
7
S
u
r
p
r
i
s
e
3
.
1
7
1
8
3
1
To
t
a
l
2
7
.
8
0
9
7
.
1
7
8
3
.
2
.
Da
t
a
pre
-
pro
ce
s
s
ing
T
h
e
d
ata
p
r
e
-
p
r
o
ce
s
s
in
g
s
tag
e
in
clu
d
es
ch
an
g
in
g
t
h
e
im
ag
e
m
o
d
e
to
R
GB
an
d
r
esizin
g
th
e
im
ag
e
to
150
×
1
5
0
p
i
x
els,
2
0
0
×
2
0
0
p
ix
e
ls
,
an
d
2
2
4
×
2
2
4
p
ix
els.
T
h
is
r
esizin
g
aim
s
to
d
eter
m
in
e
th
e
m
o
s
t
s
u
itab
le
in
p
u
t
im
ag
e
s
ize
f
o
r
ea
ch
tr
a
n
s
f
er
l
ea
r
n
in
g
m
o
d
el
(
R
esNet
-
5
0
,
I
n
ce
p
tio
n
V3
,
an
d
Xce
p
tio
n
)
to
ac
h
iev
e
th
e
h
ig
h
est
ac
cu
r
ac
y
.
All
p
ix
el
v
alu
es
ar
e
n
o
r
m
alize
d
b
y
s
ca
lin
g
th
em
to
th
e
r
an
g
e
[
0
,
1
]
to
s
tan
d
ar
d
ize
th
e
in
p
u
t
d
is
tr
ib
u
tio
n
.
At
th
is
s
tag
e,
test
in
g
ex
p
e
r
im
en
ts
ar
e
ca
r
r
ied
o
u
t
o
n
th
e
th
r
ee
tr
an
s
f
er
lear
n
in
g
m
o
d
els
to
f
in
d
th
e
m
o
s
t
s
u
itab
le
in
p
u
t
im
ag
e
s
ize
f
o
r
ea
ch
tr
an
s
f
er
lear
n
i
n
g
m
o
d
el
s
o
th
at
it
p
r
o
d
u
ce
s
th
e
h
ig
h
est
ac
cu
r
ac
y
v
alu
e
.
T
h
is
test
wa
s
co
n
d
u
cted
u
s
in
g
Ad
am
o
p
tim
izer
,
lear
n
in
g
r
ate
0
.
0
0
1
,
b
atch
s
ize
6
4
,
an
d
ep
o
ch
1
0
f
o
r
ea
ch
m
o
d
el.
I
t
is
im
p
o
r
tan
t
to
n
o
te
th
at
n
o
d
ata
au
g
m
e
n
tatio
n
is
ap
p
lied
at
th
is
s
tag
e,
au
g
m
en
tat
io
n
is
im
p
lem
en
ted
later
in
a
s
ep
ar
ate
ex
p
er
im
en
t
al
p
h
ase
to
ass
ess
its
s
p
ec
if
ic
co
n
tr
ib
u
tio
n
to
p
er
f
o
r
m
a
n
ce
a
n
d
class
im
b
alan
ce
m
itig
atio
n
.
3
.
3
.
F
e
a
t
ure
e
x
t
r
a
ct
io
n
Featu
r
e
ex
tr
ac
tio
n
in
th
is
s
tu
d
y
ap
p
lies
th
r
ee
tr
a
n
s
f
er
lear
n
in
g
m
o
d
els
th
at
h
av
e
p
r
o
v
en
t
o
ex
ce
l
in
th
e
im
ag
e
r
ec
o
g
n
itio
n
p
r
o
ce
s
s
:
R
esNe
t
-
5
0
,
I
n
ce
p
tio
n
V3
,
an
d
Xce
p
tio
n
.
E
ac
h
tr
a
n
s
f
er
lear
n
in
g
m
o
d
el
u
tili
ze
s
a
d
ee
p
co
n
v
o
lu
tio
n
al
ar
ch
itect
u
r
e
to
ex
tr
ac
t
v
is
u
al
p
atter
n
s
f
r
o
m
f
ac
ial
im
ag
es
in
th
e
FER
-
2
0
1
3
d
ataset.
T
h
e
f
ea
tu
r
e
e
x
tr
ac
tio
n
p
r
o
ce
s
s
in
v
o
lv
es
u
s
in
g
th
e
in
itial
lay
er
s
o
f
th
ese
m
o
d
els
to
o
b
tain
r
elev
an
t
r
ep
r
esen
tatio
n
s
,
wh
ich
ar
e
th
en
u
s
ed
i
n
th
e
cl
ass
if
icatio
n
s
tag
e.
T
h
e
ar
ch
itectu
r
al
s
tr
u
ctu
r
e
o
f
R
esNet
-
5
0
,
I
n
ce
p
tio
n
V3
,
an
d
Xce
p
tio
n
u
s
ed
as th
e
b
aselin
e
m
o
d
els in
th
is
s
tu
d
y
is
illu
s
tr
ated
in
Fig
u
r
e
3
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
2
1
3
-
1
2
2
6
1218
R
esNet5
0
I
n
ce
p
tio
n
V
3
X
c
e
p
t
i
o
n
Fig
u
r
e
3
.
T
r
an
s
f
er
lear
n
in
g
a
r
c
h
itectu
r
e
3
.
4
.
H
y
perpa
ra
m
e
t
er
t
uning
At
th
is
s
tag
e,
th
e
h
y
p
er
p
ar
am
eter
tu
n
in
g
test
s
ce
n
ar
io
will
b
e
co
n
d
u
cted
u
s
in
g
th
e
in
p
u
t
im
ag
e
th
at
ac
h
iev
ed
th
e
b
est
ac
cu
r
ac
y
in
p
r
ev
io
u
s
test
s
f
o
r
ea
ch
m
o
d
el.
R
esNe
t
-
5
0
an
d
Xce
p
tio
n
will
u
s
e
an
in
p
u
t
im
ag
e
s
ize
o
f
1
5
0
×
1
5
0
p
ix
els,
wh
ile
I
n
ce
p
tio
n
v
3
will
u
s
e
an
i
n
p
u
t
im
ag
e
s
ize
o
f
2
2
4
×
2
2
4
p
ix
el
s
.
Hy
p
er
p
ar
a
m
eter
tu
n
in
g
test
in
g
is
co
n
d
u
cted
to
id
en
tify
th
e
o
p
tim
al
co
m
b
in
a
tio
n
o
f
o
p
tim
izer
,
b
atch
s
ize,
lear
n
in
g
r
ate,
an
d
ep
o
ch
f
o
r
th
e
th
r
ee
m
o
d
els
th
at
p
r
o
v
id
e
th
e
b
est
p
er
f
o
r
m
an
ce
ch
ar
ac
ter
ized
b
y
th
e
h
i
g
h
e
s
t
ac
cu
r
ac
y
v
alu
e.
T
h
e
o
p
tim
izer
s
test
ed
ar
e
Ad
a
m
,
R
MSPr
o
p
,
a
n
d
SGD.
T
h
e
b
atch
s
izes
to
b
e
test
ed
ar
e
3
2
,
6
4
,
a
n
d
1
2
8
.
T
h
e
lear
n
in
g
r
ates
to
b
e
e
v
alu
ated
ar
e
0
.
0
1
,
0
.
0
0
1
,
an
d
0
.
0
0
0
1
.
T
h
e
n
u
m
b
er
o
f
ep
o
ch
s
to
b
e
test
ed
ar
e
1
0
,
5
0
,
a
n
d
1
0
0
.
T
h
e
d
ef
a
u
lt
in
itial
p
ar
am
eter
s
u
s
ed
f
o
r
all
m
o
d
els
ar
e
Ad
am
o
p
tim
izer
,
lear
n
in
g
r
ate
o
f
0
.
0
0
1
,
b
atch
s
ize
o
f
6
4
,
an
d
1
0
e
p
o
ch
s
.
3
.
5
.
Aug
m
ent
a
t
io
n
T
o
ad
d
r
ess
th
e
is
s
u
e
o
f
cla
s
s
i
m
b
alan
ce
,
p
ar
ticu
lar
ly
th
e
lim
ited
n
u
m
b
er
o
f
d
is
g
u
s
t
class
i
m
ag
es,
two
au
g
m
en
tatio
n
s
tr
ateg
ies
ar
e
ev
alu
ated
in
a
s
ep
ar
ate
ex
p
er
im
en
tal
p
h
ase.
T
h
e
f
ir
s
t
in
v
o
lv
es
tar
g
eted
au
g
m
en
tatio
n
ap
p
lied
o
n
ly
to
t
h
e
d
is
g
u
s
t
class
u
s
in
g
Op
en
C
V
tr
an
s
f
o
r
m
atio
n
s
(
e.
g
.
,
r
o
tati
o
n
,
zo
o
m
,
f
lip
p
in
g
)
.
T
h
e
s
ec
o
n
d
a
p
p
r
o
ac
h
ap
p
lies
au
g
m
en
tatio
n
to
all
class
es
u
s
in
g
th
e
Au
g
m
en
to
r
lib
r
ar
y
to
m
ain
tain
b
alan
ce
d
r
ep
r
esen
tatio
n
.
T
h
ese
m
eth
o
d
s
aim
to
en
h
a
n
ce
tr
ain
in
g
d
iv
er
s
ity
an
d
ass
ess
wh
ich
au
g
m
en
tatio
n
tech
n
i
q
u
e
y
ield
s
th
e
b
est ac
cu
r
ac
y
ac
r
o
s
s
d
if
f
er
en
t m
o
d
els.
Ov
er
s
am
p
l
in
g
o
r
weig
h
ted
lo
s
s
f
u
n
ctio
n
s
ar
e
n
o
t u
s
ed
in
th
i
s
s
tu
d
y
;
au
g
m
en
tatio
n
is
th
e
p
r
im
ar
y
s
tr
ateg
y
f
o
r
h
an
d
lin
g
im
b
alan
ce
.
T
h
is
test
in
g
s
tag
e
u
s
es
th
e
o
p
tim
al
p
ar
am
eter
s
o
b
tain
ed
f
r
o
m
t
h
e
p
r
ev
io
u
s
ly
co
n
d
u
cted
h
y
p
er
p
a
r
am
eter
t
u
n
in
g
s
ce
n
ar
io
s
.
Au
g
m
en
tatio
n
p
ar
am
eter
s
f
o
r
th
e
d
is
g
u
s
t c
lass
alo
n
e,
as we
ll a
s
f
o
r
all
class
es,
ar
e
s
h
o
wn
in
T
a
b
le
3
.
T
ab
le
3
.
Au
g
m
en
tatio
n
p
ar
am
eter
s
o
n
ly
f
o
r
d
is
g
u
s
t c
lass
an
d
en
tire
class
A
u
g
m
e
n
t
a
t
i
o
n
Li
b
r
a
r
y
P
a
r
a
me
t
e
r
O
n
l
y
d
i
s
g
u
st
c
l
a
ss
O
p
e
n
C
V
(
c
v
2
)
R
o
t
a
t
e
9
0
C
l
o
c
k
w
i
se
R
o
t
a
t
e
9
0
C
o
u
n
t
e
r
c
l
o
c
k
w
i
s
e
R
o
t
a
t
e
1
8
0
F
l
i
p
H
o
r
i
z
o
n
t
a
l
F
l
i
p
H
o
r
i
z
o
n
t
a
l
+
R
o
t
a
t
e
9
0
D
e
g
r
e
e
s
C
l
o
c
k
w
i
s
e
F
l
i
p
H
o
r
i
z
o
n
t
a
l
+
R
o
t
a
t
e
9
0
D
e
g
r
e
e
s
C
o
u
n
t
e
r
c
l
o
c
k
w
i
s
e
F
l
i
p
V
e
r
t
i
c
a
l
F
l
i
p
V
e
r
t
i
c
a
l
+
R
o
t
a
t
e
9
0
D
e
g
r
e
e
s
C
l
o
c
k
w
i
s
e
F
l
i
p
V
e
r
t
i
c
a
l
+
R
o
t
a
t
e
9
0
D
e
g
r
e
e
s
C
o
u
n
t
e
r
c
l
o
c
k
w
i
se
En
t
i
r
e
c
l
a
ss
(
I
n
c
l
u
d
i
n
g
d
i
sg
u
s
t
c
l
a
ss)
A
u
g
m
e
n
t
o
r
R
o
t
a
t
e
(
P
r
o
b
a
b
i
l
i
t
y
=
0
.
5
,
M
a
x
_
L
e
f
t
_
R
o
t
a
t
i
o
n
=
2
0
,
M
a
x
_
R
i
g
h
t
_
R
o
t
a
t
i
o
n
=
2
0
)
R
o
t
a
t
e
_
R
a
n
d
o
m_
9
0
(
P
r
o
b
a
b
i
l
i
t
y
=
0
.
5
)
W
i
d
t
h
_
S
h
i
f
t
_
R
a
n
g
e
=
0
.
2
H
e
i
g
h
t
_
S
h
i
f
t
_
R
a
n
g
e
=
0
.
2
F
l
i
p
_
L
e
f
t
_
R
i
g
h
t
(
P
r
o
b
a
b
i
l
i
t
y
=
0
.
5
)
F
l
i
p
_
T
o
p
_
B
o
t
t
o
m(
P
r
o
b
a
b
i
l
i
t
y
=
0
.
5
)
F
i
l
l
_
M
o
d
e
=
'N
e
a
r
e
s
t
'
S
a
mp
l
e
(
7
2
1
5
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
fo
r
fa
cia
l e
mo
tio
n
cla
s
s
ific
a
tio
n
o
n
…
(
N
id
a
Mu
h
liya
B
a
r
ka
h
)
1219
3
.
6
.
Addi
t
io
n o
f
l
a
y
er
s
Dr
o
p
o
u
t
an
d
b
atch
n
o
r
m
ali
za
tio
n
ar
e
cr
u
cial
tech
n
i
q
u
es
in
n
e
u
r
al
n
etwo
r
k
s
th
at
en
h
a
n
ce
p
er
f
o
r
m
an
ce
an
d
tr
ain
in
g
s
tab
ilit
y
.
Dr
o
p
o
u
t
r
ed
u
ce
s
o
v
er
f
itt
in
g
b
y
r
an
d
o
m
ly
d
is
ab
lin
g
s
o
m
e
n
eu
r
o
n
s
d
u
r
in
g
tr
ain
in
g
,
wh
ile
b
atch
n
o
r
m
ali
za
tio
n
n
o
r
m
alize
s
ea
ch
m
in
i
-
b
atch
'
s
in
p
u
t
to
ac
ce
ler
ate
tr
a
in
in
g
an
d
m
itig
ate
v
an
is
h
in
g
g
r
ad
ien
t
is
s
u
es.
B
o
th
tech
n
iq
u
es
im
p
r
o
v
e
th
e
g
e
n
er
aliza
tio
n
an
d
ef
f
icien
c
y
o
f
n
eu
r
al
n
etwo
r
k
s
.
E
x
p
er
im
en
ts
with
ad
d
in
g
d
r
o
p
o
u
t
an
d
b
atch
n
o
r
m
aliza
tio
n
lay
er
s
aim
to
ass
es
s
th
eir
i
m
p
ac
t
o
n
ac
cu
r
ac
y
wh
en
au
g
m
e
n
tin
g
th
e
d
is
g
u
s
t
class
in
ea
ch
tr
an
s
f
er
lear
n
in
g
m
o
d
el
.
T
h
e
ar
c
h
itectu
r
e
o
f
ea
ch
m
o
d
el
wh
e
n
ad
d
in
g
d
r
o
p
o
u
t la
y
e
r
s
an
d
b
atch
n
o
r
m
aliza
tio
n
is
p
r
esen
ted
i
n
Fig
u
r
e
s
4
an
d
5
.
Fig
u
r
e
4
.
Dr
o
p
o
u
t la
y
er
ar
ch
it
ec
tu
r
e
Fig
u
r
e
5
.
Dr
o
p
o
u
t a
n
d
b
atch
n
o
r
m
aliza
tio
n
lay
er
ar
ch
itectu
r
e
3
.
7
.
T
ra
ini
ng
m
et
ho
d
B
ased
o
n
s
ev
er
al
p
r
e
v
io
u
s
s
t
u
d
ies,
co
m
m
o
n
tr
ain
in
g
m
et
h
o
d
s
in
co
m
p
u
ter
v
is
io
n
r
esear
ch
in
clu
d
e
f
in
e
-
tu
n
in
g
,
f
r
ee
ze
lay
e
r
,
an
d
f
r
ee
ze
+f
in
e
-
t
u
n
in
g
.
Fin
e
-
tu
n
in
g
in
v
o
lv
es
tak
in
g
a
p
r
e
-
tr
ain
ed
m
o
d
el
an
d
ad
ap
tin
g
it
t
o
a
n
ew,
m
o
r
e
s
p
ec
if
ic
d
ataset.
T
h
is
p
r
o
ce
s
s
u
p
d
ates
th
e
weig
h
ts
o
f
s
o
m
e
o
r
all
n
etwo
r
k
lay
er
s
,
allo
win
g
th
e
m
o
d
el
to
lear
n
f
e
atu
r
es th
at
ar
e
m
o
r
e
r
elev
a
n
t to
th
e
n
ew
d
ata.
Fre
ez
e
lay
er
is
a
s
tr
ateg
y
wh
er
e
ce
r
tain
lay
er
s
in
a
n
eu
r
al
n
etwo
r
k
ar
e
k
ep
t
f
r
o
ze
n
a
n
d
n
o
t
u
p
d
ated
d
u
r
in
g
th
e
f
i
n
e
-
tu
n
in
g
p
r
o
ce
s
s
.
T
h
is
m
ea
n
s
th
at
th
e
weig
h
ts
an
d
p
a
r
am
eter
s
in
th
ese
lay
e
r
s
r
em
ain
u
n
ch
an
g
ed
wh
ile
o
n
ly
t
h
e
o
th
e
r
lay
e
r
s
ar
e
allo
wed
to
ad
ap
t
to
n
ew
d
ata.
T
h
is
tech
n
i
q
u
e
h
elp
s
p
r
ev
en
t
th
e
waste
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
2
1
3
-
1
2
2
6
1220
co
m
p
u
tin
g
r
eso
u
r
ce
s
an
d
tr
ai
n
in
g
tim
e
d
u
r
i
n
g
th
e
f
in
e
-
t
u
n
i
n
g
p
r
o
ce
s
s
.
I
t
is
co
m
m
o
n
ly
u
s
ed
wh
en
ap
p
ly
in
g
tr
an
s
f
er
lear
n
in
g
to
ex
tr
ac
t
f
e
atu
r
es,
as
f
r
ee
zin
g
th
e
f
ea
t
u
r
e
-
ex
tr
ac
tin
g
la
y
er
s
allo
ws
Fin
e
-
T
u
n
in
g
to
f
o
cu
s
o
n
ad
ap
tin
g
m
o
r
e
co
m
p
lex
f
ea
tu
r
es r
elev
an
t to
n
ew
task
s
.
Fre
ez
e
+f
in
e
-
tu
n
in
g
is
a
c
o
m
b
in
atio
n
o
f
th
e
two
p
r
ev
io
u
s
s
tr
ateg
ies
an
d
r
ep
r
esen
ts
a
n
e
w
m
eth
o
d
in
tr
o
d
u
ce
d
b
y
Ye
n
an
d
L
i
[
1
]
.
I
n
th
e
i
n
itial
s
tag
e,
s
o
m
e
o
r
all
lay
er
s
(
b
ase
lay
er
s
/p
r
e
-
tr
ain
ed
m
eth
o
d
)
ar
e
f
r
o
ze
n
t
o
p
r
e
v
en
t
u
p
d
ates
d
u
r
in
g
s
ev
er
al
tr
ain
in
g
iter
atio
n
s
.
On
ce
th
ese
f
r
o
ze
n
lay
er
s
s
tab
ilize
o
r
co
n
v
er
g
e
,
th
ey
ar
e
th
en
u
n
f
r
o
ze
n
,
an
d
th
e
en
tire
n
etwo
r
k
is
r
etr
ain
ed
(
f
in
e
-
tu
n
ed
)
to
b
etter
ad
ap
t to
th
e
n
ew
d
ataset.
T
h
is
ap
p
r
o
ac
h
c
o
m
b
in
es
t
h
e
a
d
v
an
tag
es
o
f
tr
an
s
f
er
lear
n
in
g
with
s
p
ec
if
ic
ad
j
u
s
tm
en
ts
to
n
ew
d
ata,
en
h
an
cin
g
th
e
m
eth
o
d
'
s
ad
ap
tab
ilit
y
to
ch
a
n
g
in
g
co
n
d
itio
n
s
o
r
d
ata
ch
ar
ac
te
r
is
tics
.
T
h
e
tr
ain
in
g
m
eth
o
d
test
was
co
n
d
u
cte
d
to
d
eter
m
i
n
e
wh
ic
h
m
eth
o
d
p
er
f
o
r
m
ed
b
est,
as
i
n
d
icate
d
b
y
th
e
h
ig
h
est
ac
cu
r
ac
y
v
alu
e.
E
ac
h
m
eth
o
d
was
test
ed
with
o
u
t
au
g
m
e
n
tatio
n
an
d
with
a
u
g
m
en
tatio
n
a
p
p
lied
to
th
e
d
is
g
u
s
t
class
,
in
co
r
p
o
r
ati
n
g
d
r
o
p
o
u
t
o
r
b
atch
n
o
r
m
ali
za
tio
n
lay
er
s
b
ased
o
n
th
e
b
est
ac
cu
r
ac
y
v
alu
es
o
b
tain
ed
f
r
o
m
p
r
ev
io
u
s
test
s
.
3
.
8
.
E
v
a
lua
t
i
o
n
T
h
e
ev
alu
atio
n
o
f
th
e
s
tu
d
y
was
co
n
d
u
cted
b
y
ap
p
ly
in
g
a
m
u
lti
-
class
co
n
f
u
s
io
n
m
atr
ix
to
ca
lcu
late
th
e
ac
cu
r
ac
y
p
ar
a
m
eter
s
f
o
r
e
ac
h
tr
an
s
f
er
lear
n
in
g
m
o
d
el.
T
h
e
ev
alu
atio
n
u
s
ed
7
,
1
7
8
f
a
cial
em
o
tio
n
im
ag
es
as test d
ata.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
4
.
1
.
Resul
t
s
4
.
1
.
1
.
Da
t
a
pre
-
pro
ce
s
s
ing
T
h
e
r
esu
lts
o
f
th
e
in
p
u
t
im
ag
e
s
ize
te
s
tin
g
s
h
o
w
th
at
th
e
b
est
ac
cu
r
ac
y
f
o
r
th
e
R
esNet
-
5
0
m
o
d
el
is
ac
h
iev
ed
at
a
n
im
ag
e
s
ize
o
f
1
5
0
×
1
5
0
p
ix
els,
with
an
ac
c
u
r
ac
y
v
alu
e
o
f
5
8
.
5
4
0
0
%.
Fo
r
th
e
I
n
ce
p
tio
n
V3
m
o
d
el,
th
e
b
est
ac
cu
r
ac
y
o
b
t
ain
ed
is
6
1
.
3
4
0
2
%
at
an
im
ag
e
s
ize
o
f
2
2
4
×
2
2
4
p
ix
els.
T
h
e
Xce
p
tio
n
m
o
d
el
d
em
o
n
s
tr
ates
th
e
h
ig
h
est
ac
cu
r
ac
y
at
an
im
a
g
e
s
ize
o
f
1
5
0
×
1
5
0
p
i
x
els,
r
ea
ch
in
g
an
ac
c
u
r
ac
y
o
f
6
4
.
8
2
3
1
%.
B
ased
o
n
th
e
im
ag
e
s
ize
tes
ti
n
g
r
esu
lts
,
th
e
Xce
p
tio
n
m
o
d
e
l
o
u
tp
er
f
o
r
m
s
th
e
o
th
er
two
m
o
d
els
as
it
h
as
th
e
h
ig
h
est ac
cu
r
ac
y
.
T
h
e
r
esu
lts
o
f
th
e
d
ata
p
r
e
-
p
r
o
ce
s
s
in
g
test
in
g
ca
n
b
e
s
ee
n
in
T
a
b
le
4
.
T
ab
le
4
.
R
esu
lts
o
f
in
p
u
t im
a
g
e
s
ize
ex
p
er
im
en
ts
f
o
r
ea
ch
m
o
d
el
M
o
d
e
l
I
mag
e
s
i
z
e
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
1
5
0
×
1
5
0
5
8
.
5
4
00
%
1
.
4
2
9
1
2
0
0
×
2
0
0
5
6
.
7
1
5
0
%
1
.
7
9
1
9
2
2
4
×
2
2
4
5
6
.
3
9
4
5
%
1
.
5
9
7
9
I
n
c
e
p
t
i
o
n
V
3
1
5
0
×
1
5
0
5
9
.
0
2
7
6
%
1
.
1
6
9
8
2
0
0
×
2
0
0
6
0
.
4
0
6
8
%
1
.
2
9
5
0
2
2
4
×
2
2
4
6
1
.
3
4
0
2
%
1
.
1
1
4
7
X
c
e
p
t
i
o
n
1
5
0
×
1
5
0
6
4
.
8
2
3
1
%
1
.
3
9
4
2
2
0
0
×
2
0
0
6
2
.
3
4
3
3
%
1
.
7
2
3
3
2
2
4
×
2
2
4
5
9
.
0
1
3
7
%
1
.
9
3
1
3
4
.
1
.
2
.
H
y
perpa
ra
m
et
er
t
uning
I
n
h
y
p
er
p
ar
am
eter
tu
n
in
g
,
f
o
u
r
s
ce
n
ar
i
o
s
wer
e
co
n
d
u
cted
.
I
n
th
e
f
ir
s
t
s
ce
n
a
r
io
,
th
r
ee
ty
p
es
o
f
o
p
t
i
m
i
z
e
r
s
:
A
d
a
m
,
S
GD
,
a
n
d
R
M
S
P
r
o
p
,
we
r
e
t
es
t
e
d
u
s
i
n
g
t
h
e
f
o
l
l
o
w
i
n
g
f
i
x
e
d
p
a
r
a
m
e
te
r
s
:
l
e
ar
n
i
n
g
r
a
t
e
=
0
.
0
0
1
,
b
atch
s
ize
=
6
4
,
an
d
ep
o
c
h
s
1
0
.
T
h
e
test
r
esu
lts
s
h
o
wed
th
at
th
e
R
esNet
-
5
0
m
o
d
el
ex
p
er
ie
n
ce
d
an
in
cr
ea
s
e
in
ac
cu
r
ac
y
wh
en
u
s
in
g
th
e
R
MSPr
o
p
o
p
tim
izer
,
wh
ile
th
e
h
ig
h
est
ac
cu
r
ac
y
f
o
r
th
e
o
th
er
two
m
o
d
els
was
ac
h
iev
ed
with
th
e
Ad
am
o
p
tim
izer
.
T
h
e
lo
west a
cc
u
r
ac
y
ac
r
o
s
s
all
m
o
d
els wa
s
o
b
s
er
v
ed
wh
en
u
s
in
g
th
e
SGD
o
p
tim
izer
.
I
n
th
e
s
ec
o
n
d
s
ce
n
ar
io
,
th
r
ee
lear
n
in
g
r
ate
v
alu
e
s
wer
e
test
ed
:
0
.
0
1
,
0
.
0
0
1
,
an
d
0
.
0
0
0
1
,
with
ea
ch
m
o
d
el
u
s
in
g
th
e
o
p
tim
izer
th
at
p
r
o
d
u
ce
d
th
e
b
est
ac
cu
r
ac
y
in
th
e
f
ir
s
t
s
ce
n
ar
io
.
T
h
e
r
em
ain
in
g
p
ar
am
ete
r
s
wer
e
s
et
as
f
o
llo
ws:
b
atch
s
ize
=
6
4
an
d
ep
o
c
h
s
1
0
.
T
h
e
test
r
esu
lts
s
h
o
wed
th
at
R
esNe
t
-
5
0
an
d
I
n
ce
p
tio
n
V3
ac
h
iev
ed
in
c
r
ea
s
ed
ac
cu
r
ac
y
with
a
lear
n
in
g
r
ate
=
0
.
0
0
0
1
,
wh
ile
Xce
p
tio
n
r
ea
c
h
ed
th
e
h
ig
h
est
ac
cu
r
ac
y
wit
h
a
lear
n
in
g
r
ate
=
0
.
0
0
1
.
All m
o
d
els ex
h
ib
ited
th
e
lo
west a
cc
u
r
ac
y
with
a
lear
n
in
g
r
ate
o
f
0
.
0
1
.
I
n
th
e
th
ir
d
s
ce
n
ar
i
o
,
th
r
ee
b
at
ch
s
izes:
3
2
,
6
4
,
an
d
1
2
8
,
we
r
e
test
ed
with
ea
ch
m
o
d
el
u
s
in
g
th
e
b
est
o
p
tim
izer
an
d
lear
n
in
g
r
ate
f
r
o
m
th
e
p
r
e
v
io
u
s
test
s
ce
n
ar
io
,
an
d
th
e
r
em
ain
in
g
p
ar
am
eter
was
s
et
to
ep
o
ch
s
=
1
0
.
T
h
e
test
r
esu
lts
s
h
o
wed
th
at
th
e
R
esNet
-
5
0
m
o
d
el
ac
h
iev
ed
h
ig
h
e
r
ac
cu
r
ac
y
with
a
b
atch
s
ize
o
f
1
2
8
,
wh
ile
th
e
h
ig
h
est
ac
c
u
r
a
cy
f
o
r
th
e
o
th
er
two
m
o
d
els
was
o
b
tain
ed
with
a
b
atc
h
s
ize
o
f
6
4
.
T
h
e
lo
west
ac
cu
r
ac
y
f
o
r
R
esNet
-
5
0
an
d
Xce
p
tio
n
was
o
b
s
er
v
ed
with
a
b
atch
s
ize
o
f
3
2
,
wh
ile
I
n
ce
p
t
io
n
V3
ac
h
iev
ed
its
lo
west
ac
cu
r
ac
y
with
a
b
atch
s
ize
o
f
1
2
8
.
T
h
e
test
r
esu
lts
i
n
d
icate
th
at
I
n
ce
p
tio
n
V3
an
d
Xce
p
tio
n
ac
h
iev
ed
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
fo
r
fa
cia
l e
mo
tio
n
cla
s
s
ific
a
tio
n
o
n
…
(
N
id
a
Mu
h
liya
B
a
r
ka
h
)
1221
in
cr
ea
s
ed
ac
cu
r
ac
y
with
5
0
ep
o
ch
s
,
wh
er
ea
s
R
esNet
-
5
0
s
h
o
wed
im
p
r
o
v
e
d
ac
cu
r
ac
y
with
1
0
0
ep
o
c
h
s
.
I
n
th
is
s
ce
n
ar
io
,
all
m
o
d
els
ex
p
er
ie
n
ce
d
en
h
an
ce
d
p
er
f
o
r
m
a
n
ce
.
T
h
e
r
esu
lts
o
f
th
e
h
y
p
er
p
a
r
am
eter
tu
n
i
n
g
ar
e
p
r
esen
ted
in
T
ab
le
5
to
T
ab
le
8
.
T
h
e
o
p
tim
al
h
y
p
er
p
a
r
am
ete
r
tu
n
in
g
s
ce
n
ar
io
f
o
r
ea
ch
m
o
d
el,
wh
ich
p
r
o
d
u
ce
d
th
e
b
est ac
cu
r
ac
y
,
is
p
r
esen
ted
in
T
ab
le
9
.
T
ab
le
5
.
Op
tim
izer
test
r
esu
lts
M
o
d
e
l
S
c
e
n
a
r
i
o
1
:
O
p
t
i
m
i
z
e
r
O
p
t
i
mi
z
e
r
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
A
d
a
m
5
8
.
5
4
0
0
%
1
.
4
2
9
1
S
G
D
5
8
.
3
7
2
8
%
1
.
5
1
2
1
R
M
S
p
r
o
p
5
9
.
4
4
5
5
%
1
.
6
2
0
6
I
n
c
e
p
t
i
o
n
V
3
A
d
a
m
6
1
.
3
4
0
2
%
1
.
1
1
4
7
S
G
D
5
4
.
1
6
5
5
%
1
.
4
5
2
4
R
M
S
p
r
o
p
5
9
.
0
6
9
4
%
1
.
3
6
1
6
X
c
e
p
t
i
o
n
A
d
a
m
6
4
.
8
2
3
1
%
1
.
3
9
4
2
S
G
D
5
3
.
3
1
5
7
%
1
.
2
7
6
8
R
M
S
p
r
o
p
6
0
.
2
2
5
7
%
1
.
6
8
2
6
T
ab
le
6
.
L
ea
r
n
in
g
r
ate
test
r
esu
lts
M
o
d
e
l
S
c
e
n
a
r
i
o
2
:
Le
a
r
n
i
n
g
r
a
t
e
Le
a
r
n
i
n
g
r
a
t
e
Le
a
r
n
i
n
g
r
a
t
e
Le
a
r
n
i
n
g
r
a
t
e
R
e
sN
e
t
-
50
0
.
0
1
0
.
0
1
0
.
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
I
n
c
e
p
t
i
o
n
V
3
0
.
0
1
0
.
0
1
0
.
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
X
c
e
p
t
i
o
n
0
.
0
1
0
.
0
1
0
.
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
0
.
0
0
0
1
T
ab
le
7
.
B
atch
s
ize
test
r
esu
lts
M
o
d
e
l
S
c
e
n
a
r
i
o
3
:
B
a
t
c
h
s
i
z
e
B
a
t
c
h
si
z
e
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
32
5
8
.
8
0
4
7
%
3
.
1
5
7
3
64
6
0
.
9
0
8
3
%
2
.
8
2
4
5
1
2
8
6
4
.
3
9
1
2
%
2
.
9
2
6
2
I
n
c
e
p
t
i
o
n
V
3
32
6
4
.
3
7
7
3
%
1
.
6
3
2
9
64
6
4
.
8
0
9
1
%
1
.
7
9
7
9
1
2
8
6
2
.
2
5
9
7
%
1
.
8
8
3
0
X
c
e
p
t
i
o
n
32
5
9
.
6
9
6
3
%
1
.
4
5
4
3
64
6
4
.
8
2
3
1
%
1
.
3
9
4
2
1
2
8
6
1
.
5
4
9
2
%
1
.
4
6
4
2
T
ab
le
8
.
E
p
o
ch
test
r
esu
lts
M
o
d
e
l
S
c
e
n
a
r
i
o
4
:
Ep
o
c
h
Ep
o
c
h
Ep
o
c
h
Ep
o
c
h
R
e
sN
e
t
-
50
10
10
10
50
50
50
1
0
0
1
0
0
1
0
0
I
n
c
e
p
t
i
o
n
V
3
10
10
10
50
50
50
1
0
0
1
0
0
1
0
0
X
c
e
p
t
i
o
n
10
10
10
50
50
50
1
0
0
1
0
0
1
0
0
T
ab
le
9
.
Op
tim
al
h
y
p
er
p
ar
am
e
ter
tu
n
in
g
s
ce
n
a
r
io
s
f
o
r
ea
ch
m
o
d
el
M
o
d
e
l
I
n
p
u
t
s
h
a
p
e
O
p
t
i
mi
z
e
r
Le
a
r
n
i
n
g
r
a
t
e
B
a
t
c
h
s
i
z
e
Ep
o
c
h
R
e
sN
e
t
-
50
1
5
0
×
1
5
0
R
M
S
p
r
o
p
0
.
0
0
0
1
1
2
8
1
0
0
I
n
c
e
p
t
i
o
n
V
3
2
2
4
×
2
2
4
A
d
a
m
0
.
0
0
0
1
64
50
X
c
e
p
t
i
o
n
1
5
0
×
1
5
0
A
d
a
m
0
.
0
0
1
64
50
4
.
1
.
3
.
Aug
m
ent
a
t
io
n
T
h
e
r
esu
lts
o
f
d
ata
a
u
g
m
e
n
tatio
n
ac
r
o
s
s
all
tr
an
s
f
er
lear
n
i
n
g
m
o
d
els
in
d
icate
th
at
au
g
m
en
tin
g
th
e
d
is
g
u
s
t
class
in
th
e
FER
-
2
0
1
3
d
ataset
y
ield
s
b
etter
ac
cu
r
a
cy
co
m
p
a
r
ed
to
ap
p
ly
i
n
g
au
g
m
en
tatio
n
ac
r
o
s
s
all
class
es
in
th
e
d
ataset.
T
h
e
a
u
g
m
en
tatio
n
ap
p
lied
to
th
e
d
is
g
u
s
t
class
,
wh
ich
is
th
e
m
i
n
o
r
ity
class
in
th
e
FER
-
2
0
1
3
d
ataset,
en
ab
les th
e
tr
an
s
f
er
lear
n
in
g
m
o
d
el
to
b
etter
r
ec
o
g
n
ize
an
d
d
ee
p
ly
lear
n
th
e
im
ag
e
p
atter
n
s
with
in
th
at
class
.
Ho
wev
er
,
th
e
ap
p
licatio
n
o
f
d
ata
a
u
g
m
en
tatio
n
t
o
th
e
FER
-
2
0
1
3
d
ataset
u
s
in
g
t
h
e
R
esNet
-
5
0
,
I
n
ce
p
tio
n
V3
,
a
n
d
Xce
p
tio
n
tr
an
s
f
er
lear
n
in
g
m
o
d
els
d
id
n
o
t
s
ig
n
if
ica
n
tly
im
p
ac
t
th
e
o
v
er
all
m
o
d
el
p
er
f
o
r
m
a
n
ce
.
I
n
th
e
R
esNet
-
5
0
an
d
Xce
p
tio
n
m
o
d
els,
th
e
im
p
lem
en
tatio
n
o
f
au
g
m
en
tatio
n
led
t
o
ac
cu
r
ac
y
d
ec
r
ea
s
es
o
f
1
.
5
1
8
5
%
an
d
1
.
3
2
3
5
%,
r
esp
ec
tiv
ely
.
Me
an
wh
ile,
in
th
e
I
n
ce
p
tio
n
V3
m
o
d
el,
au
g
m
en
tatio
n
s
lig
h
tly
in
cr
ea
s
e
d
ac
cu
r
ac
y
b
y
0
.
1
5
3
3
%.
T
h
e
r
esu
lts
o
f
au
g
m
en
tatio
n
f
o
r
th
e
d
is
g
u
s
t c
lass
alo
n
e
an
d
f
o
r
all
class
es in
th
e
FER
-
2
0
1
3
d
ataset
ar
e
p
r
esen
ted
in
T
ab
le
1
0
.
T
ab
le
1
0
.
A
u
g
m
en
tatio
n
test
in
g
r
esu
lts
o
n
ly
f
o
r
th
e
d
is
g
u
s
t c
lass
an
d
en
tire
class
es
M
o
d
e
l
A
u
g
m
e
n
t
a
t
i
o
n
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
N
o
n
e
6
4
.
8
9
2
7
%
3
.
1
5
7
0
D
i
sg
u
st
6
3
.
3
7
4
2
%
3
.
3
1
6
6
A
l
l
C
l
a
ss
5
0
.
2
6
4
7
%
4
.
2
4
7
6
I
n
c
e
p
t
i
o
n
V
3
N
o
n
e
6
6
.
0
3
5
1
%
1
.
9
7
0
7
D
i
sg
u
st
6
6
.
1
8
8
4
%
2
.
0
1
1
9
A
l
l
C
l
a
ss
5
7
.
8
4
3
4
%
2
.
5
6
7
1
X
c
e
p
t
i
o
n
N
o
n
e
6
5
.
8
2
6
1
%
2
.
4
1
0
0
D
i
sg
u
st
6
4
.
5
0
2
6
%
2
.
1
0
1
8
A
l
l
C
l
a
ss
5
2
.
8
0
0
2
%
2
.
4
8
9
0
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
2
1
3
-
1
2
2
6
1222
4
.
1
.
4
.
Addi
t
io
n o
f
l
a
y
er
s
B
ased
o
n
th
e
ex
p
e
r
im
en
tal
r
e
s
u
lts
o
f
ad
d
in
g
d
r
o
p
o
u
t
a
n
d
b
atch
n
o
r
m
aliza
tio
n
lay
er
s
to
t
h
e
d
is
g
u
s
t
class
au
g
m
en
tatio
n
,
it
ca
n
b
e
s
ee
n
th
at
f
o
r
th
e
R
e
s
Net
-
5
0
m
o
d
el,
th
e
h
ig
h
est
ac
cu
r
ac
y
o
f
6
3
.
3
7
4
2
%
was
ac
h
iev
ed
with
o
u
t
ad
d
in
g
an
y
lay
er
s
.
I
n
co
n
tr
ast,
th
e
I
n
ce
p
t
io
n
V3
m
o
d
el
a
ch
iev
ed
th
e
h
ig
h
est
ac
cu
r
ac
y
o
f
6
6
.
3
9
7
3
%
wh
en
th
e
d
r
o
p
o
u
t
lay
er
was
in
clu
d
ed
.
I
n
th
e
Xc
ep
tio
n
m
o
d
el,
th
e
h
ig
h
est
ac
c
u
r
ac
y
o
f
6
5
.
4
7
7
8
%
was
ac
h
iev
ed
with
t
h
e
ad
d
itio
n
o
f
b
o
t
h
th
e
d
r
o
p
o
u
t
an
d
b
at
ch
n
o
r
m
aliza
tio
n
la
y
er
s
.
Fo
r
t
h
e
R
esNet
-
5
0
an
d
Xce
p
tio
n
m
o
d
els,
ap
p
ly
in
g
au
g
m
en
tatio
n
to
t
h
e
d
is
g
u
s
t
clas
s
alo
n
g
with
ad
d
in
g
d
r
o
p
o
u
t
o
r
b
atch
n
o
r
m
aliza
tio
n
lay
er
s
d
id
n
o
t
i
m
p
r
o
v
e
ac
cu
r
ac
y
c
o
m
p
a
r
ed
t
o
th
e
m
o
d
els
with
o
u
t
a
u
g
m
en
tatio
n
.
Ho
wev
er
,
i
n
th
e
I
n
ce
p
tio
n
V3
m
o
d
el,
t
h
e
in
clu
s
io
n
o
f
th
e
d
r
o
p
o
u
t
la
y
er
en
h
an
ce
d
ac
c
u
r
ac
y
,
s
u
r
p
ass
in
g
th
e
ac
cu
r
ac
y
ac
h
iev
ed
with
o
u
t
au
g
m
e
n
tatio
n
o
n
t
h
e
d
is
g
u
s
t
class
.
T
h
e
r
es
u
lts
o
f
in
co
r
p
o
r
atin
g
th
ese
lay
er
s
in
to
ea
ch
m
o
d
el
ar
e
p
r
esen
ted
in
T
a
b
le
1
1
.
A
co
m
p
ar
is
o
n
o
f
ac
c
u
r
ac
y
v
alu
e
s
with
an
d
with
o
u
t
au
g
m
en
tat
io
n
f
o
r
th
e
d
is
g
u
s
t
class
is
s
h
o
wn
in
T
ab
le
1
2
.
T
ab
le
1
1
.
E
x
p
er
im
e
n
tal
r
esu
lts
o
f
ad
d
in
g
d
r
o
p
o
u
t la
y
e
r
s
an
d
b
atch
n
o
r
m
aliza
tio
n
to
ea
ch
m
o
d
el
M
o
d
e
l
La
y
e
r
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
N
o
n
e
6
3
.
3
7
4
2
%
3
.
3
1
6
6
La
y
e
r
D
r
o
p
o
u
t
6
1
.
2
5
6
6
%
4
.
9
1
1
7
La
y
e
r
D
r
o
p
o
u
t
+
B
a
t
c
h
N
o
r
ma
l
i
z
a
t
i
o
n
5
7
.
8
4
3
4
%
3
.
4
9
5
0
I
n
c
e
p
t
i
o
n
V
3
N
o
n
e
6
6
.
1
8
8
4
%
2
.
0
1
1
9
La
y
e
r
D
r
o
p
o
u
t
6
6
.
3
9
7
3
%
2
.
1
8
8
0
La
y
e
r
D
r
o
p
o
u
t
+
B
a
t
c
h
N
o
r
ma
l
i
z
a
t
i
o
n
6
6
.
1
8
8
4
%
1
.
8
4
9
9
X
c
e
p
t
i
o
n
N
o
n
e
6
4
.
5
0
2
6
%
2
.
1
0
1
8
La
y
e
r
D
r
o
p
o
u
t
2
4
.
7
1
4
4
%
1
.
9
0
5
6
La
y
e
r
D
r
o
p
o
u
t
+
B
a
t
c
h
N
o
r
ma
l
i
z
a
t
i
o
n
6
5
.
4
7
7
8
%
2
.
1
3
5
3
T
ab
le
1
2
.
C
o
m
p
ar
is
o
n
o
f
ac
cu
r
ac
y
v
alu
es with
o
u
t a
p
p
ly
i
n
g
a
u
g
m
en
tatio
n
,
ap
p
ly
in
g
au
g
m
e
n
tatio
n
,
a
n
d
a
p
p
ly
in
g
ad
d
itio
n
al
la
y
er
s
to
ea
ch
m
o
d
el
M
o
d
e
l
A
u
g
m
e
n
t
a
t
i
o
n
La
y
e
r
A
c
c
u
r
a
c
y
Lo
ss
R
e
sN
e
t
-
50
N
o
n
e
N
o
n
e
6
4
.
8
9
2
7
%
3
.
1
5
7
0
D
i
sg
u
st
N
o
n
e
6
3
.
3
7
4
2
%
3
.
3
1
6
6
D
i
sg
u
st
D
r
o
p
o
u
t
6
1
.
2
5
6
6
%
4
.
9
1
1
7
I
n
c
e
p
t
i
o
n
V
3
N
o
n
e
N
o
n
e
6
6
.
0
3
5
1
%
1
.
9
7
0
7
D
i
sg
u
st
N
o
n
e
6
6
.
1
8
8
4
%
2
.
0
1
1
9
D
i
sg
u
st
D
r
o
p
o
u
t
6
6
.
3
9
7
3
%
2
.
1
8
8
0
X
c
e
p
t
i
o
n
N
o
n
e
N
o
n
e
6
5
.
8
2
6
1
%
2
.
4
1
0
0
D
i
sg
u
st
N
o
n
e
6
4
.
5
0
2
6
%
2
.
1
0
1
8
D
i
sg
u
st
D
r
o
p
o
u
t
+
B
a
t
c
h
N
o
r
m
a
l
i
z
a
t
i
o
n
6
5
.
4
7
7
8
%
2
.
1
3
5
3
T
h
e
im
p
lem
en
tatio
n
o
f
d
r
o
p
o
u
t
an
d
b
atch
n
o
r
m
aliza
tio
n
la
y
er
s
r
esu
lted
in
a
s
ig
n
if
ican
t
d
ec
r
ea
s
e
in
ac
cu
r
ac
y
ac
r
o
s
s
all
m
o
d
els.
T
h
e
ap
p
licatio
n
o
f
d
r
o
p
o
u
t
alo
n
e
o
n
th
e
Xce
p
tio
n
m
o
d
el
led
to
a
s
u
b
s
tan
tial
ac
cu
r
ac
y
d
r
o
p
o
f
4
1
.
1
1
1
7
%,
wh
ile
R
esNet
-
5
0
ex
p
e
r
ien
ce
d
a
d
ec
r
ea
s
e
o
f
3
.
6
3
6
1
%.
I
n
co
n
tr
ast,
I
n
ce
p
tio
n
V3
s
h
o
wed
a
s
lig
h
t
ac
cu
r
ac
y
in
c
r
ea
s
e
o
f
0
.
3
6
2
2
%,
alth
o
u
g
h
th
i
s
ch
an
g
e
was
n
o
t
s
ig
n
if
ican
t.
T
h
is
in
d
icate
s
th
at
th
e
ef
f
ec
ts
o
f
u
s
in
g
d
r
o
p
o
u
t
an
d
b
atch
n
o
r
m
aliza
tio
n
lay
er
s
ca
n
v
ar
y
g
r
ea
tly
d
e
p
en
d
i
n
g
o
n
t
h
e
tr
an
s
f
er
lear
n
in
g
m
o
d
el
em
p
lo
y
ed
.
Ad
ju
s
tm
en
ts
b
ased
o
n
d
ee
p
er
o
b
s
er
v
atio
n
s
ar
e
n
ec
ess
ar
y
if
th
es
e
tech
n
iq
u
es
ar
e
to
b
e
ap
p
lied
t
o
tr
an
s
f
er
lear
n
in
g
m
o
d
els.
4
.
1
.
5
.
T
ra
ini
ng
m
et
ho
d
B
ased
o
n
th
e
test
r
esu
lts
,
it
ca
n
b
e
co
n
cl
u
d
ed
th
at
all
th
r
ee
t
r
an
s
f
er
lear
n
i
n
g
m
o
d
els
ac
h
iev
ed
th
e
b
est
ac
cu
r
ac
y
b
y
ap
p
ly
i
n
g
th
e
f
i
n
e
-
tu
n
in
g
tr
ain
in
g
m
eth
o
d
.
Fo
r
th
e
R
esNet
-
5
0
an
d
Xce
p
tio
n
m
o
d
els,
th
e
h
ig
h
est
ac
cu
r
ac
y
was
o
b
tain
e
d
with
o
u
t
ap
p
ly
in
g
an
y
a
u
g
m
e
n
tatio
n
o
r
ad
d
itio
n
al
lay
er
s
,
with
ac
cu
r
ac
y
v
al
u
es
o
f
6
4
.
8
9
2
7
%
an
d
6
5
.
8
2
6
1
%,
r
e
s
p
ec
tiv
ely
.
I
n
th
e
ca
s
e
o
f
t
h
e
I
n
ce
p
tio
n
V3
m
o
d
el,
th
e
b
est
ac
cu
r
ac
y
was
ac
h
iev
ed
b
y
a
p
p
ly
in
g
au
g
m
en
tatio
n
an
d
ad
d
in
g
a
d
r
o
p
o
u
t
l
ay
er
,
r
esu
ltin
g
in
a
n
ac
cu
r
ac
y
o
f
6
6
.
3
9
7
3
%.
T
h
e
tr
ain
in
g
m
eth
o
d
test
r
esu
lts
a
r
e
p
r
esen
ted
in
T
ab
le
1
3
.
T
h
e
co
m
b
in
atio
n
o
f
au
g
m
en
tatio
n
an
d
lay
e
r
s
th
at
p
r
o
d
u
ce
d
th
e
h
ig
h
est
ac
cu
r
a
cy
f
o
r
ea
ch
m
o
d
el
ca
n
b
e
s
ee
n
in
T
ab
le
1
4
,
wh
ile
th
e
co
m
b
in
atio
n
s
o
f
au
g
m
en
tatio
n
an
d
ad
d
itio
n
al
lay
er
s
f
o
r
ea
ch
tr
a
n
s
f
er
le
ar
n
in
g
m
o
d
el
d
u
r
in
g
tr
ain
i
n
g
ar
e
d
is
p
lay
e
d
in
T
ab
le
1
5
.
Evaluation Warning : The document was created with Spire.PDF for Python.