I
A
E
S
I
n
t
e
r
n
at
io
n
al
Jou
r
n
al
of
A
r
t
if
ic
ia
l
I
n
t
e
ll
ig
e
n
c
e
(
I
J
-
A
I
)
V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024, pp.
4435
~
444
4
I
S
S
N
:
2252
-
8938
,
D
O
I
:
10.11591/
ij
a
i.
v13.i4.pp
4435
-
444
4
4435
Jou
r
n
al
h
om
e
page
:
ht
tp
:
//
ij
ai
.
ia
e
s
c
or
e
.c
om
O
p
t
i
m
i
z
e
d
m
u
l
t
i
-
l
aye
r
se
l
f
-
at
t
e
n
t
i
on
n
e
t
w
or
k
f
or
f
e
at
u
r
e
-
l
e
v
e
l
d
at
a f
u
si
on
i
n
e
m
ot
i
on
r
e
c
ogn
i
t
i
on
B
as
am
m
a U
m
e
s
h
P
at
il
1,
2
, A
s
h
ok
a D
avan
age
r
i
V
ir
u
p
ak
s
h
ap
p
a
2
, A
j
ay P
r
ak
as
h
B
as
ap
p
a V
ij
aya
3
1
D
e
pa
r
t
m
e
nt
of
C
om
put
e
r
S
c
i
e
nc
e
a
nd E
ngi
ne
e
r
i
ng, J
S
S
A
c
a
de
m
y of
T
e
c
hni
c
a
l
E
duc
a
t
i
on B
e
nga
l
ur
u, V
i
s
ve
s
va
r
a
ya
T
e
c
hnol
ogi
c
a
l
U
ni
ve
r
s
i
t
y, B
e
l
a
ga
vi
, I
ndi
a
2
D
e
pa
r
t
m
e
nt
of
I
nf
or
m
a
t
i
on
S
c
i
e
nc
e
a
nd E
ngi
ne
e
r
i
ng, J
S
S
A
c
a
de
m
y of
T
e
c
hni
c
a
l
E
duc
a
t
i
on, B
e
nga
l
ur
u, V
i
s
ve
s
va
r
a
y
a
T
e
c
hnol
ogi
c
a
l
U
ni
ve
r
s
i
t
y, B
e
l
a
ga
vi
, I
ndi
a
3
D
e
pa
r
t
m
e
nt
of
A
r
t
i
f
i
c
i
a
l
I
nt
e
l
l
i
ge
nc
e
a
nd M
a
c
hi
ne
L
e
a
r
ni
ng, D
r
. A
m
be
dka
r
I
ns
t
i
t
ut
e
of
T
e
c
hnol
ogy, B
e
nga
l
ur
u, V
i
s
ve
s
va
r
a
ya
T
e
c
hnol
ogi
c
a
l
U
ni
ve
r
s
i
t
y, B
e
l
a
ga
vi
, I
ndi
a
A
r
t
ic
le
I
n
f
o
A
B
S
T
R
A
C
T
A
r
ti
c
le
h
is
to
r
y
:
R
e
c
e
iv
e
d
D
e
c
1, 2023
R
e
vi
s
e
d
M
a
y 2, 2024
A
c
c
e
pt
e
d
J
un 1,
2024
Understanding
human
emotions
across
diverse
data
sources
presents
challenges
in
various
applicati
ons
includi
ng
healthcare,
human
-
machine
interactio
n,
security,
marketing,
and
gaming.
Prior
resear
ch
has
ex
plored
fusion
techniques
to
address
multimodal
data
heterogeneity,
yet
often
overlooks
the
importance
of
discriminative
unimodal
informatio
n
and
potential
complementarity
among
fusion
strategies.
Recognizing
em
otions
from
video
and
audio
data
poses
challenges
such
as
non
-
verba
l
cues
interpreta
tion,
varying
expressio
n,
ambigui
ty
in
context,
and
the
ne
ed
for
nuanced
feature
extraction
to
capture
subtle
emotional
nuances
accurat
ely.
To
tackle
these
issues,
it
is
imperative
to
employ
efficient
emotion
representation
and mul
timodal
fusion
techniq
ues, as t
hese tas
ks have significant impo
rtance
within
the
realm
of
multifaceted
recognizing
study.
This
study
introd
uced
a
novel
approach,
optimized
multi
-
layer
self
-
attenti
on
network
for
e
motion
recognition
(OMSN
-
ER),
focusing
on
feature
-
level
data
fusion.
OM
SN
-
ER
precis
ely
assesses
emotional
states
by
merging
facial
and
voice
data,
ut
ilizing
a
multi
-
layer
progressive
dense
residual
fusion
network
and
a
self
-
att
ention
mountain
gazelle
convolution
neural
network.
Implemented
in
Pytho
n
with
the
RAVDESS
dataset,
the
methodol
ogy
achieves
exceptional
accuracy
(0.9908),
surpassing
benchmarks
and
demonstrati
ng
efficacy
in
multi
modal
emotion
recognit
ion. Th
is research
represents
promis
ing
advancement
s
in th
e
intricate
field of e
motion reco
gnition.
K
e
y
w
o
r
ds
:
E
m
ot
io
n
r
e
c
ogni
ti
on
M
ul
ti
m
oda
li
ti
e
s
F
e
a
tu
r
e
l
e
ve
l
f
us
io
n
M
ul
ti
-
la
ye
r
pr
ogr
e
s
s
iv
e
de
ns
e
r
e
s
id
ua
l
f
us
io
n ne
twor
k
S
e
lf
-
a
tt
e
nt
io
n m
ount
a
in
ga
z
e
ll
e
c
onvolut
io
n ne
ur
a
l
ne
twor
k
This is an
open
acce
ss artic
le unde
r the
CC BY
-
SA
license.
C
or
r
e
s
pon
di
n
g A
u
th
or
:
B
a
s
a
m
m
a
U
m
e
s
h P
a
ti
l
D
e
pa
r
tm
e
nt
of
I
nf
or
m
a
ti
on S
c
ie
nc
e
a
nd E
ngi
ne
e
r
in
g, J
S
S
A
c
a
de
m
y of
T
e
c
hni
c
a
l
E
duc
a
ti
on
V
is
ve
s
va
r
a
y
a
T
e
c
hnol
ogi
c
a
l
U
ni
ve
r
s
it
y
Dr
. V
is
hnuva
r
dha
na
R
oa
d,
B
e
nga
lu
r
u
-
560060, Ka
r
na
ta
ka
, I
ndi
a
E
m
a
il
:
bupa
ti
l2
5@
gm
a
il
.c
om
1.
I
N
T
R
O
D
U
C
T
I
O
N
E
m
ot
io
ns
,
in
ta
ngi
bl
e
ye
t
pr
o
f
ound
m
e
nt
a
l
s
ta
te
s
e
xpr
e
s
s
e
d
th
r
ough
w
r
it
te
n,
vi
s
ua
l,
a
nd
ve
r
ba
l
c
ue
s
,
la
c
k
c
le
a
r
phys
io
lo
gi
c
a
l
in
di
c
a
to
r
s
[
1]
,
[
2]
.
U
nde
r
s
ta
ndi
ng
th
e
s
e
e
m
ot
io
ns
pr
of
oundly
in
f
lu
e
nc
e
s
c
om
m
uni
c
a
ti
on
a
c
r
os
s
dom
a
in
s
li
ke
li
e
de
te
c
ti
on
a
nd
s
ur
ve
il
la
nc
e
,
unde
r
s
c
or
in
g
th
e
ne
e
d
to
e
xpl
or
e
e
m
ot
io
na
l
va
r
ia
bi
li
ty
f
or
in
s
ig
ht
s
in
to
ps
yc
hol
ogi
c
a
l
w
e
ll
-
be
in
g.
I
n
to
da
y’
s
s
oc
i
e
ty
,
e
m
ot
io
ns
pl
a
y
a
n
in
te
gr
a
l
r
ol
e
in
our
da
il
y
li
ve
s
,
pr
om
pt
in
g
th
e
c
r
uc
ia
l
ne
e
d
f
or
m
a
c
hi
ne
s
to
c
om
pr
e
he
nd
a
nd
r
e
s
pond
a
ppr
opr
ia
te
ly
.
F
or
be
tt
e
r
out
c
om
e
s
,
it
is
a
ls
o c
r
uc
ia
l
f
or
m
a
c
hi
ne
s
to
de
te
c
t
hum
a
n
e
m
ot
i
ons
[
3]
,
[
4]
.
A
dva
nc
in
g
e
m
ot
io
n
c
la
s
s
if
ic
a
ti
on
te
c
hni
que
s
is
k
e
y
f
or
s
oc
ie
ta
l
pr
ogr
e
s
s
,
in
te
gr
a
ti
ng
in
to
da
il
y
li
f
e
[
5]
,
[
6]
. E
m
ot
io
n
r
e
c
ogni
ti
on
in
c
onve
r
s
a
ti
ons
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
, V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024:
4435
-
444
4
4436
(
E
R
C
)
,
e
nc
om
pa
s
s
in
g
te
xt
ua
l,
vi
s
ua
l,
a
nd
a
udi
o
c
ue
s
,
s
ta
nds
pi
vot
a
l
in
a
na
ly
z
in
g
m
ul
t
im
e
di
a
in
f
o
r
m
a
ti
on,
e
s
s
e
nt
ia
l
f
or
a
s
s
e
s
s
in
g
u
s
e
r
-
c
ont
e
nt
in
te
r
a
c
ti
ons
[
7]
–
[
9]
.
M
or
e
ove
r
,
a
s
he
a
lt
hc
a
r
e
c
onc
e
r
ns
,
s
uc
h
a
s
a
ir
pol
lu
ti
on,
e
s
c
a
la
te
,
le
v
e
r
a
gi
ng
m
a
c
hi
ne
le
a
r
ni
ng
in
s
pi
r
e
d
by
hum
a
n
e
m
ot
io
n
r
e
c
ogni
ti
on
be
c
om
e
s
vi
ta
l
[
10]
–
[
12]
.
A
ddi
ti
ona
ll
y, pr
e
c
is
e
r
e
c
ogni
ti
on o
f
huma
n a
c
ti
vi
ti
e
s
vi
a
m
ul
ti
s
e
ns
or
y m
oda
li
ti
e
s
f
ur
th
e
r
m
ot
iv
a
te
s
a
dva
nc
e
m
e
nt
s
in
e
m
ot
io
n
r
e
c
ogni
ti
on
s
ys
te
m
s
[
13]
,
[
14]
.
P
s
yc
hol
ogi
s
ts
ha
r
ne
s
s
th
e
s
tr
e
ngt
h
s
of
te
xt
ua
l,
vi
s
ua
l,
a
nd
a
udi
o
m
ode
ls
in
a
c
om
bi
ne
d
a
ppr
oa
c
h
to
a
c
hi
e
ve
a
m
or
e
c
o
m
pr
e
he
ns
iv
e
unde
r
s
ta
ndi
ng
of
hum
a
n
e
m
ot
io
ns
[
15]
–
[
17]
.
N
ot
a
bl
y,
s
tu
di
e
s
hi
ghl
ig
ht
th
e
in
a
de
qua
c
y
of
r
e
ly
in
g
s
ol
e
ly
on
one
m
oda
li
ty
f
or
e
m
ot
io
n
r
e
c
ogni
ti
on,
e
m
pha
s
iz
in
g t
he
s
up
e
r
io
r
it
y of
a
ppr
oa
c
he
s
i
nt
e
gr
a
ti
ng mul
ti
pl
e
m
oda
li
ti
e
s
.
F
or
a
c
ons
id
e
r
a
bl
e
pe
r
io
d,
s
ta
te
-
of
-
th
e
-
a
r
t
s
ys
te
m
s
in
s
pe
e
c
h
e
m
ot
io
n
r
e
c
ogni
ti
on
(
S
E
R
)
f
a
c
e
d
c
ha
ll
e
nge
s
of
lo
w
a
c
c
ur
a
c
y
a
nd
hi
gh
c
om
put
a
ti
ona
l
de
m
a
nds
[
18]
,
[
19]
.
H
ow
e
ve
r
,
r
e
c
e
nt
a
dva
nc
e
m
e
nt
s
ha
ve
in
tr
oduc
e
d
r
e
a
l
-
ti
m
e
s
ys
te
m
s
e
xhi
bi
ti
ng
not
a
bl
e
pe
r
f
or
m
a
nc
e
s
in
e
m
ot
io
n
r
e
c
ogni
ti
on.
A
nva
r
jo
n
e
t
al
.
[
20]
in
tr
oduc
e
d
a
li
ght
w
e
ig
ht
c
onvolut
io
na
l
ne
ur
a
l
ne
twor
k
(
C
N
N
)
m
ode
l
w
it
h
pl
a
in
r
e
c
ta
ngul
a
r
ke
r
ne
ls
a
nd
c
us
to
m
iz
e
d pooli
ng l
a
ye
r
s
,
a
c
hi
e
vi
ng r
e
m
a
r
ka
bl
e
a
c
c
ur
a
c
y on
t
he
in
te
r
a
c
ti
ve
e
m
ot
io
na
l
dya
di
c
m
ot
io
n c
a
pt
ur
e
(
I
E
M
O
C
A
P
)
(
77.01
%
)
a
nd
e
m
ot
io
na
l
s
pe
e
c
h
da
ta
ba
s
e
(
E
M
O
-
D
B
)
da
ta
s
e
ts
(
92.02%
)
.
C
ha
ll
e
nge
s
pe
r
s
is
t,
pa
r
ti
c
ul
a
r
ly
i
n t
he
i
nt
e
gr
a
ti
on of
a
ddi
ti
ona
l
m
ul
ti
m
oda
l
in
put
s
our
c
e
s
l
ik
e
f
a
c
ia
l
e
xpr
e
s
s
io
n
s
, a
s
s
e
e
n
i
n s
tu
di
e
s
s
uc
h
a
s
F
r
a
nz
oni
e
t
al
.
[
21]
,
w
he
r
e
pa
r
ti
a
l
f
a
c
ia
l
im
a
ge
s
f
oc
u
s
in
g
s
ol
e
ly
on
m
out
h
m
ove
m
e
nt
s
m
a
in
ta
in
e
d
a
5%
a
c
c
ur
a
c
y l
os
s
c
om
pa
r
e
d t
o f
ul
l
-
f
a
c
e
i
m
a
ge
s
a
c
r
os
s
f
our
e
m
ot
io
ns
:
ne
ut
r
a
l,
ha
ppy, s
ur
pr
is
e
d, a
nd a
ngr
y.
A
not
he
r
e
m
e
r
gi
ng
s
our
c
e
of
in
f
or
m
a
ti
on
f
or
r
e
a
l
-
ti
m
e
s
ys
te
m
s
is
te
xt
u
a
l
da
ta
.
W
it
h
th
e
a
dv
e
nt
of
tr
a
ns
f
or
m
e
r
s
[
22]
a
nd
bi
di
r
e
c
ti
ona
l
e
nc
ode
r
r
e
pr
e
s
e
nt
a
ti
ons
f
r
om
tr
a
ns
f
or
m
e
r
s
(
B
E
R
T
)
m
ode
ls
[
23]
,
nu
m
e
r
ous
publ
ic
a
ti
ons
ha
ve
hi
ghl
ig
ht
e
d
th
e
a
dva
nt
a
ge
s
of
us
in
g
na
tu
r
a
l
la
ngua
ge
due
to
s
m
a
ll
e
r
f
il
e
s
iz
e
s
a
nd
th
e
a
bunda
nc
e
of
a
va
il
a
bl
e
da
ta
s
e
ts
.
T
hi
s
ha
s
le
d
to
it
is
a
ppl
ic
a
t
io
n
in
va
r
io
us
ta
s
ks
,
in
c
lu
di
ng
s
e
nt
im
e
nt
a
nd
e
m
ot
io
n
r
e
c
ogni
ti
on
[
24]
,
[
25]
,
s
how
c
a
s
in
g
it
is
ve
r
s
a
ti
li
ty
a
c
r
os
s
dom
a
in
s
.
J
im
é
n
e
z
e
t
al
.
[2
6]
pr
opos
e
d
a
n
e
m
ot
io
n
r
e
c
ogni
ti
on
s
y
s
te
m
ut
il
iz
in
g
tr
a
ns
f
e
r
l
e
a
r
ni
ng
te
c
hni
que
s
.
T
h
e
f
r
a
m
e
w
or
k
in
vol
ve
d
a
pr
e
-
tr
a
in
e
d
s
pa
ti
a
l
tr
a
ns
f
or
m
e
r
ne
twor
k on s
a
li
e
nc
y m
a
ps
a
nd f
a
c
ia
l
im
a
ge
s
, f
ol
lo
w
e
d by a
bi
di
r
e
c
ti
ona
l
lo
ng s
hor
t
-
te
r
m
m
e
m
or
y
(
B
iL
S
T
M
)
w
it
h
a
n
a
tt
e
n
ti
on
m
e
c
ha
ni
s
m
.
E
m
pl
oyi
ng
a
la
te
f
us
i
on
s
tr
a
te
gy
yi
e
ld
e
d
a
n
a
c
c
ur
a
c
y
of
80.08%
on
th
e
r
ye
r
s
on
a
udi
o
-
vi
s
ua
l
da
ta
ba
s
e
of
e
m
ot
io
na
l
s
pe
e
c
h
a
nd
s
ong
(
R
A
V
D
E
S
S
)
da
ta
s
e
t.
H
ow
e
ve
r
,
di
r
e
c
t
a
ppl
ic
a
ti
on t
o vi
de
o
-
ba
s
e
d t
a
s
k
s
e
nc
ount
e
r
e
d doma
in
a
da
pt
a
ti
o
n i
s
s
ue
s
.
J
im
é
ne
z
e
t
al
.
[
13
]
pr
opos
e
d
a
n e
m
ot
io
n r
e
c
ogni
ti
on s
ys
t
e
m
e
m
pl
oyi
ng a
ur
a
l
tr
a
ns
f
or
m
e
r
s
a
nd a
c
ti
on
uni
ts
f
or
th
e
R
A
V
D
E
S
S
da
ta
s
e
t.
T
hi
s
c
om
pr
is
e
d
a
s
pe
e
c
h
e
m
ot
io
n
r
e
c
ogni
z
e
r
(
S
E
R
)
a
nd
a
f
a
c
ia
l
e
m
ot
io
n
r
e
c
ogni
z
e
r
(
F
E
R
)
.
T
he
S
E
R
u
ti
li
z
e
d
a
pr
e
-
tr
a
in
e
d
xl
s
r
-
W
a
v2V
e
c
2.0
tr
a
ns
f
or
m
e
r
w
it
h
tr
a
ns
f
e
r
-
le
a
r
ni
n
g
te
c
hni
que
s
s
uc
h
a
s
e
m
be
ddi
ng
e
xt
r
a
c
ti
on
a
nd
f
in
e
-
tu
ni
ng.
T
he
F
E
R
in
vol
ve
d
th
e
e
xt
r
a
c
ti
on
o
f
a
c
ti
on
uni
ts
f
r
o
m
vi
de
os
.
A
la
te
f
us
io
n
te
c
hni
que
c
om
bi
ni
ng
th
e
s
e
m
oda
li
ti
e
s
a
c
hi
e
ve
d
a
n
a
c
c
ur
a
c
y
of
86.70%
.
L
im
it
a
ti
ons
in
c
lu
de
d
th
e
in
c
a
pa
c
it
y
of
a
c
ti
on
uni
ts
to
c
a
pt
ur
e
c
r
uc
ia
l
in
f
o
r
m
a
ti
on
f
r
om
vi
de
o
f
r
a
m
e
s
.
R
a
doi
e
t
al
.
[2
7
]
pr
opos
e
d
a
n
e
nd
-
to
-
e
nd
e
m
ot
io
n
r
e
c
ogni
ti
on
f
r
a
m
e
w
or
k
ba
s
e
d
on
th
e
te
m
por
a
l
a
ggr
e
ga
ti
on
of
m
ul
ti
m
oda
l
in
f
or
m
a
ti
on.
T
e
m
por
a
l
a
udi
o
-
vi
s
ua
l
in
f
or
m
a
ti
on
w
a
s
e
xt
r
a
c
te
d
us
in
g
th
e
te
m
por
a
ll
y
a
ggr
e
ga
te
d
a
udi
o
-
vi
s
ua
l
ne
twor
k
(
T
A
-
A
V
N
)
,
a
c
hi
e
vi
ng
a
n
ove
r
a
ll
a
c
c
ur
a
c
y
of
84%
f
or
th
e
C
R
E
M
A
-
D
da
t
a
s
e
t
a
nd
78.7%
f
or
th
e
R
A
V
D
E
S
S
da
ta
s
e
t.
H
ow
e
ve
r
,
us
in
g
li
m
it
e
d
a
nnot
a
te
d
da
ta
,
obt
a
in
e
d
th
r
ough
a
r
a
ndom
s
e
le
c
ti
on
of
te
m
p
or
a
l
w
in
dow
s
w
it
hi
n i
ndi
vi
dua
l
vi
de
o f
r
a
gm
e
nt
s
, pr
ove
d i
ns
uf
f
ic
ie
nt
f
or
e
m
ot
io
n r
e
c
ogni
ti
on.
E
xi
s
ti
ng
m
e
th
odol
ogi
e
s
in
e
m
ot
io
n
r
e
c
ogni
ti
on
of
te
n
gr
a
ppl
e
w
it
h
c
ha
ll
e
nge
s
r
e
la
te
d
to
th
e
in
te
gr
a
ti
on
of
di
ve
r
s
e
m
oda
li
ti
e
s
,
le
a
di
ng
to
in
e
f
f
ic
ie
nc
ie
s
in
in
f
or
m
a
ti
on
pr
oc
e
s
s
in
g
a
nd
in
a
c
c
ur
a
c
ie
s
in
e
m
ot
io
n
de
t
e
c
ti
on.
T
r
a
di
ti
ona
l
a
ppr
oa
c
he
s
m
a
y
la
c
k
pr
e
c
is
e
c
ont
r
ol
ove
r
in
f
or
m
a
ti
on
f
lo
w
,
r
e
s
ul
ti
ng
in
th
e
lo
s
s
of
va
lu
a
bl
e
da
t
a
a
nd
une
xpe
c
te
d
noi
s
e
f
r
om
in
di
vi
dua
l
m
oda
li
ti
e
s
.
A
ddi
ti
ona
ll
y,
th
e
s
e
m
e
th
ods
te
nd
to
c
ons
um
e
s
ubs
ta
nt
ia
l
pr
oc
e
s
s
in
g
ti
m
e
w
it
hout
a
c
hi
e
vi
ng
opt
im
a
l
a
c
c
ur
a
c
y.
T
o
a
ddr
e
s
s
th
e
s
e
li
m
it
a
ti
ons
,
th
is
r
e
s
e
a
r
c
h
pr
opo
s
e
s
th
e
a
dopt
io
n of
f
e
a
tu
r
e
-
le
ve
l
da
ta
f
us
io
n t
e
c
hni
que
s
. B
y i
nt
e
gr
a
ti
ng
f
e
a
tu
r
e
s
e
xt
r
a
c
te
d f
r
om
m
ul
ti
pl
e
m
oda
li
ti
e
s
a
t
a
f
e
a
tu
r
e
-
le
ve
l
r
e
pr
e
s
e
nt
a
ti
on,
th
is
a
ppr
oa
c
h
of
f
e
r
s
s
e
ve
r
a
l
a
dva
nt
a
ge
s
.
F
e
a
tu
r
e
-
le
ve
l
f
us
io
n
f
a
c
il
it
a
te
s
a
m
or
e
ef
f
e
c
ti
ve
c
om
bi
na
ti
on
of
in
f
or
m
a
ti
on
f
r
om
di
f
f
e
r
e
nt
s
our
c
e
s
,
e
na
bl
in
g
e
nha
nc
e
d
c
la
s
s
if
ic
a
ti
on
a
c
c
ur
a
c
y
a
nd
r
e
duc
e
d
noi
s
e
.
M
or
e
ove
r
,
it
a
ll
ow
s
f
or
a
m
or
e
c
om
pr
e
he
ns
iv
e
ut
il
iz
a
ti
on
of
a
va
il
a
bl
e
da
ta
w
hi
le
m
in
im
iz
in
g
pr
oc
e
s
s
in
g
ti
m
e
.
T
he
ut
il
iz
a
ti
on
of
f
e
a
tu
r
e
-
le
ve
l
da
t
a
f
us
io
n
in
th
is
s
tu
dy
a
im
s
to
ov
e
r
c
om
e
th
e
s
hor
tc
om
in
gs
of
e
xi
s
ti
ng
m
e
th
ods
,
s
tr
iv
in
g
f
or
e
nha
nc
e
d
m
ul
ti
m
oda
l
f
us
io
n
a
nd
c
la
s
s
if
ic
a
ti
on
e
f
f
ic
ie
nc
y.
B
y
le
ve
r
a
gi
ng
f
e
a
tu
r
e
-
le
ve
l
f
us
io
n
te
c
hni
que
s
,
th
e
pr
opos
e
d
a
ppr
oa
c
h
e
nde
a
vor
s
to
a
c
hi
e
ve
s
upe
r
io
r
a
c
c
ur
a
c
y,
c
om
pr
e
he
ns
iv
e
in
f
or
m
a
ti
on
ut
i
li
z
a
ti
on,
r
e
duc
e
d
noi
s
e
,
m
in
im
iz
e
d
pr
oc
e
s
s
in
g
ti
m
e
,
a
nd
r
obus
t
f
us
io
n.
U
lt
im
a
te
ly
,
th
is
in
nova
ti
ve
m
e
th
odol
ogy
s
e
e
ks
to
r
e
vol
ut
io
ni
z
e
e
m
ot
io
n
r
e
c
og
ni
ti
on
s
ys
te
m
s
,
of
f
e
r
in
g
nua
nc
e
d
in
s
ig
ht
s
a
nd
a
c
c
ur
a
te
a
s
s
e
s
s
m
e
nt
s
of
e
m
ot
io
na
l
s
ta
te
s
a
c
r
os
s
di
ve
r
s
e
d
om
a
in
s
,
th
e
r
e
by
r
e
s
ha
pi
ng
hum
a
n
-
c
om
put
e
r
in
te
r
a
c
ti
ons
a
nd e
m
ot
io
na
l
unde
r
s
ta
ndi
ng.
E
f
f
e
c
ti
ve
e
m
ot
io
n
r
e
c
ogni
ti
on
a
c
r
os
s
m
ul
ti
pl
e
m
oda
li
ti
e
s
r
e
m
a
in
s
a
f
or
m
id
a
bl
e
c
ha
ll
e
nge
due
t
o
li
m
it
a
ti
ons
i
n t
r
a
di
ti
ona
l
f
e
a
tu
r
e
e
xt
r
a
c
ti
on m
e
th
ods
a
nd
t
he
n
e
e
d f
or
e
nha
nc
e
d
ge
ne
r
a
li
z
a
ti
on
a
nd c
r
os
s
-
m
oda
l
in
f
or
m
a
ti
on
in
te
gr
a
ti
on.
C
ur
r
e
nt
a
ppr
oa
c
he
s
of
te
n
e
nc
ount
e
r
c
om
pl
e
xi
ty
li
m
it
a
ti
ons
a
nd
in
e
f
f
ic
ie
nc
ie
s
in
c
a
pt
ur
in
g
te
m
por
a
l
de
pe
nde
nc
i
e
s
.
T
hi
s
l
ead
s
to
s
ubopti
m
a
l
p
e
r
f
or
m
a
nc
e
in
r
e
c
ogni
z
in
g
nua
nc
e
d
e
m
ot
io
na
l
s
ta
te
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
O
pt
imi
z
e
d m
ul
ti
-
la
y
e
r
s
e
lf
-
at
te
nt
io
n ne
tw
or
k
f
or
f
e
at
u
r
e
-
le
v
e
l
d
at
a f
us
io
n i
n …
(
B
as
am
m
a U
m
e
s
h P
at
il
)
4437
I
n
th
is
s
tu
dy,
a
nove
l
a
ppr
oa
c
h
ha
s
be
e
n
pr
opos
e
d
to
a
ddr
e
s
s
k
e
y
c
ha
ll
e
nge
s
in
e
m
ot
io
n
r
e
c
ogni
ti
on
a
c
r
os
s
va
r
io
us
m
oda
li
ti
e
s
.
T
he
pr
opos
e
d
c
ont
r
ib
ut
io
ns
e
nc
om
pa
s
s
s
e
ve
r
a
l
in
nova
ti
ve
m
e
th
odol
ogi
e
s
:
i)
ha
ndc
r
a
f
te
d
f
e
a
tu
r
e
e
xt
r
a
c
ti
on
is
ti
m
e
-
c
ons
um
in
g.
B
iL
S
T
M
a
ll
ow
s
e
nd
-
to
-
e
nd
le
a
r
ni
ng
f
or
be
tt
e
r
pe
r
f
or
m
a
nc
e
,
a
voi
di
ng
c
om
pl
e
xi
ty
li
m
it
a
ti
ons
;
ii
)
to
boos
t
ge
ne
r
a
li
z
a
ti
on,
th
e
m
ode
l
e
m
pl
oys
B
iL
S
T
M
f
or
a
udi
o
-
vi
s
ua
l
ta
s
ks
,
c
a
pt
ur
in
g
te
m
por
a
l
de
pe
nde
nc
ie
s
e
f
f
e
c
ti
ve
ly
.
I
t
e
nha
nc
e
s
e
f
f
ic
ie
nc
y
by
pr
io
r
it
iz
in
g
ke
yf
r
a
m
e
s
;
ii
i)
th
e
m
ul
ti
-
la
ye
r
pr
ogr
e
s
s
iv
e
de
ns
e
r
e
s
id
u
a
l
f
us
i
on
ne
twor
k
(
M
P
D
R
F
-
ne
t)
e
nha
nc
e
s
e
m
ot
io
n
c
la
s
s
if
ic
a
ti
on
by
in
te
gr
a
ti
ng
m
ul
ti
-
m
oda
l
in
f
or
m
a
ti
on
us
in
g
f
e
a
tu
r
e
-
le
ve
l
f
us
io
n,
a
ddr
e
s
s
in
g
uni
m
oda
l
s
hor
tc
om
in
gs
f
or
im
pr
ove
d
c
r
os
s
-
m
oda
l
in
f
or
m
a
ti
on
e
xc
ha
nge
a
nd
r
e
c
ogni
ti
on
pe
r
f
or
m
a
nc
e
;
a
nd
iv
)
in
e
xi
s
ti
ng
m
e
th
ods
,
a
to
ta
l
8 c
la
s
s
e
s
in
th
e
R
A
V
D
E
S
S
da
ta
s
e
t
a
r
e
not
c
la
s
s
if
ie
d,
but
th
e
s
e
lf
-
a
tt
e
nt
io
n
M
ount
a
in
G
a
z
e
ll
e
C
N
N
(
S
M
G
C
N
N
)
a
c
c
ur
a
te
ly
c
a
te
gor
iz
e
s
ha
ppy,
a
ngr
y,
di
s
gus
te
d,
f
e
a
r
f
ul
,
c
a
lm
,
s
a
d,
s
ur
pr
is
e
d,
a
nd
ne
ut
r
a
l
s
ta
te
s
.
2.
M
E
T
H
O
D
I
n
th
e
r
e
a
lm
of
e
m
ot
io
n
id
e
nt
if
ic
a
ti
on
th
r
ough
m
ul
ti
m
oda
l
da
ta
li
ke
vi
de
o
a
nd
a
udi
o,
f
e
a
tu
r
e
r
e
pr
e
s
e
nt
a
ti
on
a
nd
f
us
io
n
s
ta
nd
out
a
s
c
r
uc
ia
l
ye
t
c
ha
ll
e
ngi
ng
ta
s
ks
.
T
hi
s
r
e
s
e
a
r
c
h
in
tr
oduc
e
s
a
n
e
nh
a
nc
e
d
opt
im
iz
e
d
m
ul
ti
-
la
ye
r
s
e
lf
-
a
tt
e
nt
io
n
ne
twor
k
f
o
r
e
m
ot
io
n
r
e
c
ogni
ti
on
(
O
M
S
N
-
E
R
)
a
im
e
d
a
t
a
c
c
ur
a
te
ly
a
s
s
e
s
s
in
g
th
e
pa
r
ti
c
ip
a
nt
’
s
e
m
ot
io
na
l
s
ta
te
s
a
c
r
os
s
di
f
f
e
r
e
nt
m
oda
li
ti
e
s
.
T
hi
s
te
c
hni
que
a
m
a
lg
a
m
a
te
s
in
f
or
m
a
ti
on
f
r
o
m
a
udi
o
a
nd
vi
de
o
s
our
c
e
s
,
e
na
bl
in
g
a
c
om
pr
e
he
ns
iv
e
a
ppr
oa
c
h
to
a
udi
o
-
vi
de
o
e
m
ot
io
n
id
e
nt
if
ic
a
ti
o
n.
T
he
w
or
kf
lo
w
of
th
is
r
e
s
e
a
r
c
h
is
de
pi
c
te
d
in
F
ig
u
r
e
1.
I
ni
ti
a
l
da
ta
a
c
qui
s
it
io
n
in
vol
ve
s
ga
th
e
r
in
g
vi
s
ua
l
a
nd
a
udi
o
da
ta
f
r
om
th
e
R
A
V
D
E
S
S
da
ta
s
e
t.
P
r
e
-
pr
o
c
e
s
s
in
g
te
c
hni
que
s
a
r
e
a
ppl
ie
d
to
m
in
im
iz
e
unde
s
ir
a
bl
e
e
f
f
e
c
ts
,
e
nha
nc
in
g
th
e
qua
li
ty
of
vi
de
o
da
ta
a
nd
r
e
f
in
in
g
s
pe
e
d
s
ig
na
ls
.
T
he
pr
e
-
pr
oc
e
s
s
e
d
out
put
f
e
e
ds
i
nt
o t
he
B
iL
S
T
M
-
ba
s
e
d f
e
a
tu
r
e
e
xt
r
a
c
ti
on
pr
oc
e
s
s
. A
udi
o
f
e
a
tu
r
e
e
xt
r
a
c
ti
on i
nvol
ve
s
c
a
pt
ur
in
g t
im
e
a
nd
f
r
e
que
nc
y
dom
a
in
f
e
a
tu
r
e
s
a
lo
ng
w
it
h
M
e
l
-
ba
s
e
d
f
e
a
tu
r
e
s
.
F
or
v
id
e
o,
c
ont
e
xt
ua
l
in
f
or
m
a
ti
on
w
i
th
in
w
or
d
-
le
ve
l
vi
s
ua
l
f
e
a
tu
r
e
s
is
e
xt
r
a
c
te
d.
T
he
e
xt
r
a
c
te
d
f
e
a
tu
r
e
s
unde
r
go
f
us
io
n
us
in
g
th
e
M
P
D
R
F
-
N
e
t.
T
he
s
e
f
us
e
d
f
e
a
tu
r
e
s
a
r
e
th
e
n
in
put
te
d
in
to
th
e
S
M
G
C
N
N
to
a
c
c
ur
a
te
ly
c
la
s
s
if
y
th
e
e
m
ot
io
n
s
ta
te
s
s
uc
h
a
s
ne
ut
r
a
l,
c
a
lm
,
ha
ppy,
s
a
d,
a
ngr
y,
f
e
a
r
f
ul
,
di
s
gus
te
d,
a
nd
s
ur
pr
is
e
d.
T
he
e
f
f
ic
a
c
y
of
th
is
pr
op
os
e
d
m
e
th
od
is
a
s
s
e
s
s
e
d
a
nd
c
om
pa
r
e
d
a
ga
in
s
t
e
xi
s
ti
ng
m
e
th
ods
us
in
g
va
r
io
us
e
va
lu
a
ti
on
m
e
tr
ic
s
.
T
he
a
udi
o
a
nd
vi
de
o
da
ta
a
r
e
pr
e
pr
oc
e
s
s
e
d
to
r
e
m
ove
noi
s
e
a
nd pr
e
pa
r
e
t
he
m
f
or
f
ur
th
e
r
a
na
ly
s
is
.
F
ig
ur
e
1. T
he
w
or
kf
lo
w
of
O
M
S
N
-
D
E
R
f
or
e
m
ot
io
n de
te
c
ti
on
us
in
g t
he
R
A
V
D
E
S
S
da
ta
s
e
t
2.1. Dat
as
e
t
d
e
s
c
r
ip
t
io
n
T
he
R
A
V
D
E
S
S
da
ta
ba
s
e
[
2
8
]
is
a
va
li
da
te
d
m
ul
ti
m
oda
l
da
ta
s
e
t
c
ont
a
in
in
g
e
m
ot
io
na
l
s
pe
e
c
h
a
nd
s
ong
r
e
c
or
di
ngs
.
I
t
is
d
e
s
ig
ne
d
s
pe
c
if
ic
a
ll
y
f
or
r
e
c
ogni
z
in
g
e
m
ot
io
ns
a
nd
is
u
s
e
d
e
xt
e
ns
iv
e
ly
in
th
e
f
ie
ld
of
c
om
put
in
g
a
nd
a
udi
o
a
na
ly
s
i
s
.
I
t
c
ons
is
t
s
of
24
pr
of
e
s
s
io
na
l
a
c
to
r
s
(
12
f
e
m
a
le
a
nd
12
m
a
le
)
pe
r
f
or
m
in
g
s
c
r
ip
te
d
s
pe
e
c
h
a
nd
s
ong
in
8
di
f
f
e
r
e
nt
e
m
ot
io
na
l
s
ta
te
s
s
uc
h
a
s
ha
ppy,
f
e
a
r
f
ul
,
c
a
lm
,
a
ngr
y,
s
a
d,
di
s
gus
te
d,
ne
ut
r
a
l
a
nd
s
ur
pr
is
e
d.
T
h
e
r
e
a
r
e
7,356
f
il
e
s
in
th
e
R
A
V
D
E
S
S
.
T
hr
e
e
m
oda
li
ty
f
or
m
s
,
in
c
lu
di
ng
a
udi
o
-
onl
y
(
16
bi
t,
48
kH
z
.w
a
v)
,
a
udi
o
-
vi
de
o
(
720
p
H
.264,
A
A
C
48
kH
z
,
.m
p4)
,
a
nd
vi
de
o
-
onl
y
(
no
s
ound)
,
a
r
e
p
r
e
s
e
nt
in
th
e
da
ta
s
e
t.
T
h
e
a
udi
o
f
il
e
s
a
r
e
ty
pi
c
a
ll
y
s
to
r
e
d
in
th
e
W
A
V
f
or
m
a
t,
w
hi
c
h
is
a
s
ta
nda
r
d
unc
om
pr
e
s
s
e
d
a
udi
o
f
or
m
a
t.
A
c
to
r
s
e
xpr
e
s
s
e
d e
m
ot
io
ns
a
t
nor
m
a
l
a
nd
s
tr
ong int
e
ns
i
ty
l
e
ve
ls
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
, V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024:
4435
-
444
4
4438
2.2. P
r
e
-
p
r
oc
e
s
s
in
g t
e
c
h
n
iq
u
e
s
T
he
in
it
ia
l
s
te
p
in
pr
e
p
a
r
in
g
th
e
a
udi
o
a
nd
vi
de
o
da
ta
f
r
o
m
th
e
R
A
V
D
E
S
S
da
ta
s
e
t
in
vol
ve
s
c
om
pr
e
he
ns
iv
e
pr
e
-
pr
oc
e
s
s
in
g
[
29
]
.
T
hi
s
s
ta
ge
is
c
r
uc
ia
l
to
m
it
i
ga
te
is
s
ue
s
li
ke
in
c
om
pl
e
te
ne
s
s
,
in
c
ons
i
s
te
nc
y,
a
nd nois
e
t
ha
t
c
om
m
onl
y a
f
f
li
c
t
r
a
w
da
ta
.
T
hr
ough te
c
hni
que
s
s
uc
h a
s
bi
la
te
r
a
l
f
il
te
r
in
g f
or
vi
de
o a
nd s
pe
c
tr
a
l
noi
s
e
ga
te
(
S
N
G
)
f
or
a
udi
o,
unw
a
nt
e
d
a
r
ti
f
a
c
ts
a
r
e
r
e
m
ove
d
to
e
nha
nc
e
th
e
qua
li
ty
of
th
e
vi
s
u
a
l
f
r
a
m
e
s
a
nd
s
pe
e
c
h
s
ig
na
ls
.
N
oi
s
e
ha
ndl
in
g:
m
a
na
gi
ng
noi
s
e
in
a
udi
o
a
nd
vi
de
o
da
ta
is
pi
vot
a
l
to
e
ns
ur
in
g
th
e
r
obus
tn
e
s
s
of
s
ubs
e
que
nt
m
ode
ls
a
ga
in
s
t
ove
r
f
it
ti
ng.
A
ddr
e
s
s
in
g
s
e
ns
or
noi
s
e
,
c
om
pr
e
s
s
io
n
a
r
ti
f
a
c
ts
,
a
nd
e
nvi
r
onm
e
nt
a
l
c
ondi
ti
ons
,
a
de
noi
s
in
g
th
r
e
s
hol
d
of
0.1
is
a
ppl
ie
d
to
s
tr
ik
e
a
ba
la
nc
e
be
twe
e
n
pr
e
s
e
r
vi
ng
da
ta
in
te
gr
it
y
a
nd
noi
s
e
r
e
m
ova
l.
S
to
r
a
ge
e
f
f
ic
ie
nc
y
a
nd
tr
a
c
ki
ng:
vi
de
o
s
in
he
r
e
nt
ly
de
m
a
nd
e
xt
e
ns
iv
e
s
to
r
a
ge
due
to
th
e
ir
hi
gh
-
di
m
e
ns
io
na
l
na
tu
r
e
.
E
m
pl
oyi
ng
pr
e
-
pr
oc
e
s
s
in
g
te
c
hni
qu
e
s
a
id
s
in
r
e
duc
in
g
vi
d
e
o
f
il
e
s
iz
e
s
w
it
hout
c
om
pr
om
is
in
g
qua
li
ty
.
F
ur
th
e
r
m
or
e
,
obj
e
c
t
de
te
c
ti
on
a
nd
tr
a
c
k
in
g
m
e
c
ha
ni
s
m
s
e
na
bl
e
th
e
id
e
nt
if
ic
a
ti
on
a
nd
m
oni
to
r
in
g
of
s
pe
c
if
ic
obj
e
c
t
s
or
r
e
gi
on
s
of
in
te
r
e
s
t
w
it
hi
n
th
e
f
r
a
m
e
s
.
A
udi
o
s
e
pa
r
a
ti
on
a
nd
l
a
ngua
ge
de
te
c
ti
on:
th
e
pr
oc
e
s
s
in
vol
ve
s
tr
a
n
s
c
r
i
bi
ng
s
pe
e
c
h
c
ont
e
nt
,
id
e
nt
if
yi
ng
s
poke
n
la
ngua
ge
s
,
a
nd
d
e
lv
in
g
in
to
th
e
e
m
ot
io
na
l
or
s
e
nt
im
e
nt
a
l
a
s
pe
c
ts
of
t
he
a
udi
o. S
im
ul
ta
ne
ous
ly
, s
e
gr
e
ga
ti
ng s
pe
e
c
h f
r
om
ba
c
kgr
ound nois
e
i
s
e
s
s
e
nt
ia
l
to
c
a
pt
ur
e
t
he
i
nt
e
nde
d c
ont
e
nt
e
f
f
e
c
ti
ve
ly
. F
e
a
tu
r
e
s
a
r
e
e
xt
r
a
c
te
d
f
r
om
t
he
pr
e
pr
oc
e
s
s
e
d a
udi
o a
nd vide
o da
ta
.
2.3. M
u
lt
im
od
al
f
e
at
u
r
e
e
xt
r
ac
t
io
n
w
it
h
b
id
ir
e
c
t
io
n
al
l
on
g s
h
or
t
-
t
e
r
m
m
e
m
or
y
F
e
a
tu
r
e
e
xt
r
a
c
ti
on
is
a
c
r
uc
ia
l
s
t
e
p
in
da
ta
r
e
c
ove
r
y,
a
im
in
g
to
e
xt
r
a
c
t
pe
r
ti
ne
nt
a
udi
o
a
nd
vi
de
o
f
e
a
tu
r
e
s
f
or
im
p
r
ove
d
c
la
s
s
if
ic
a
ti
on
out
c
om
e
s
.
P
a
tt
e
r
ns
a
r
e
ty
pi
c
a
ll
y
e
xt
r
a
c
te
d
f
r
om
r
a
w
a
udi
o
da
ta
th
r
ough
ha
nd
-
c
r
a
f
te
d
f
e
a
tu
r
e
s
,
e
s
s
e
nt
ia
l
a
s
r
a
w
a
udi
o
da
t
a
is
in
he
r
e
nt
ly
in
th
e
t
im
e
dom
a
in
,
w
it
h
vi
ta
l
in
f
or
m
a
ti
o
n
unve
il
e
d
in
th
e
f
r
e
que
nc
y
dom
a
in
.
T
hi
s
pa
pe
r
in
tr
oduc
e
s
a
f
e
a
tu
r
e
ve
c
to
r
e
nc
om
pa
s
s
in
g
s
e
lf
-
e
ngi
ne
e
r
e
d
f
e
a
tu
r
e
s
f
r
om
bot
h doma
in
s
. T
he
B
iL
S
T
M
-
ba
s
e
d e
nc
od
e
r
i
s
e
m
pl
oye
d f
or
e
xt
r
a
c
ti
on.
A
udi
o
f
e
a
tu
r
e
e
xt
r
a
c
ti
on
[
30
]
:
th
is
pha
s
e
in
vol
ve
s
e
xt
r
a
c
ti
ng
va
r
io
us
f
e
a
tu
r
e
s
li
ke
M
e
l
-
f
r
e
que
nc
y
c
e
ps
tr
a
l
c
oe
f
f
ic
ie
nt
s
,
e
ne
r
gy,
pi
tc
h,
a
nd
f
a
s
t
f
our
ie
r
tr
a
ns
f
or
m
(
F
F
T
)
pa
r
a
m
e
te
r
s
.
T
r
a
ns
f
or
m
in
g
a
udi
o
s
ig
na
ls
f
r
om
th
e
ti
m
e
to
f
r
e
que
nc
y
dom
a
in
r
e
ve
a
ls
s
ubs
ta
nt
ia
l
in
s
ig
ht
s
.
F
e
a
tu
r
e
s
s
uc
h
a
s
a
m
pl
it
ude
e
nve
lo
pe
,
z
e
r
o,
a
nd
m
e
a
n c
r
os
s
in
g r
a
te
a
r
e
c
om
put
e
d t
o
e
nr
ic
h t
he
t
im
e
doma
in
c
ha
r
a
c
te
r
is
ti
c
s
.
V
id
e
o
f
e
a
tu
r
e
e
xt
r
a
c
ti
on:
th
e
ut
il
iz
a
ti
on
of
hi
s
to
gr
a
m
of
or
ie
nt
e
d
gr
a
di
e
nt
m
ode
ls
f
a
c
il
it
a
te
s
th
e
e
xt
r
a
c
ti
on
of
lo
c
a
l
s
ha
pe
a
nd
te
xt
ur
e
in
f
or
m
a
ti
on
f
r
om
vi
de
o
c
ont
e
nt
.
B
iL
S
T
M
a
id
s
in
a
c
qui
r
in
g
c
ont
e
xt
ua
l
in
f
or
m
a
ti
on
f
r
om
th
e
f
r
a
m
e
s
,
c
ont
r
ib
ut
in
g
to
a
d
e
e
p
e
r
unde
r
s
t
a
ndi
ng
of
th
e
vi
s
u
a
l
da
ta
.
T
he
pr
oc
e
s
s
in
vol
ve
s
pow
e
r
s
pe
c
tr
a
c
om
put
a
ti
on,
m
e
l
-
f
il
te
r
ba
nk
a
ppl
ic
a
ti
on,
lo
ga
r
it
hm
ic
tr
a
ns
f
or
m
a
ti
on,
a
nd
F
F
T
.
T
he
e
xt
r
a
c
te
d
a
udi
o a
nd vide
o f
e
a
tu
r
e
s
a
r
e
t
he
n f
us
e
d t
o c
r
e
a
te
a
m
ul
ti
m
oda
l
r
e
pr
e
s
e
nt
a
ti
on of
t
he
e
m
ot
io
na
l
s
ta
te
.
2.4.
F
e
at
u
r
e
-
l
e
ve
l
m
u
lt
i
m
od
al
f
e
at
u
r
e
f
u
s
io
n
w
it
h
m
u
lt
i
-
la
ye
r
p
r
og
r
e
s
s
iv
e
d
e
n
s
e
r
e
s
id
u
al
f
u
s
io
n
n
e
t
w
or
k
F
ol
lo
w
in
g
th
e
e
xt
r
a
c
ti
on
of
f
e
a
tu
r
e
s
f
r
om
bot
h
a
udi
o
a
nd
vi
de
o
s
our
c
e
s
,
th
e
in
te
gr
a
ti
on
of
th
e
s
e
di
ve
r
s
e
f
e
a
tu
r
e
s
a
s
s
um
e
s
pa
r
a
m
ount
im
por
ta
nc
e
.
F
e
a
tu
r
e
-
le
ve
l
da
ta
f
u
s
io
n,
f
a
c
il
it
a
te
d
by
th
e
M
P
D
R
F
-
N
e
t,
be
c
om
e
s
pi
vot
a
l
in
ha
r
m
oni
z
in
g
th
e
e
xt
r
a
c
te
d f
e
a
tu
r
e
s
t
o m
in
i
m
iz
e
i
nf
or
m
a
ti
on l
os
s
[
3
1
]
‒
[3
3
]
.
T
hr
ough the
i
nt
e
gr
a
ti
o
n
of
f
e
a
tu
r
e
s
us
in
g
dow
n
-
s
a
m
pl
in
g
a
ggr
e
ga
te
d
r
e
s
id
ua
l
(
D
A
R
)
m
odul
e
s
a
nd
a
r
e
s
id
ua
l
tr
a
ns
f
or
m
e
r
,
th
e
M
P
D
R
F
-
N
e
t
e
f
f
e
c
ti
ve
ly
c
om
bi
ne
s
f
e
a
tu
r
e
-
le
ve
l
r
e
pr
e
s
e
nt
a
ti
ons
to
e
xt
r
a
c
t
lo
ng
-
di
s
ta
nc
e
in
f
or
m
a
ti
on.
T
he
r
e
s
ul
ti
ng
f
us
e
d
f
e
a
tu
r
e
r
e
c
ons
tr
uc
ti
on
e
nha
nc
e
s
s
e
m
a
nt
ic
in
f
or
m
a
ti
on,
c
ont
r
ib
ut
in
g
to
im
pr
ove
d
e
m
ot
io
n
c
la
s
s
if
ic
a
ti
on.
2.4.1. P
r
ogr
e
s
s
iv
e
f
e
at
u
r
e
f
u
s
io
n
M
P
D
R
F
-
N
e
t’
s
a
r
c
hi
te
c
tu
r
e
is
s
tr
uc
tu
r
e
d
to
a
de
pt
ly
c
om
bi
ne
th
e
r
ic
h
a
udi
o
a
nd
vi
de
o
f
e
a
tu
r
e
s
.
T
hr
ough
pr
ogr
e
s
s
iv
e
f
e
a
tu
r
e
f
us
io
n,
it
s
tr
a
te
gi
c
a
ll
y
a
m
a
lg
a
m
a
te
s
th
e
nua
nc
e
d
d
e
ta
il
s
c
a
pt
ur
e
d
by
bot
h
m
oda
li
ti
e
s
.
T
he
in
te
gr
a
ti
on
pr
oc
e
s
s
in
vol
ve
s
a
s
e
r
ie
s
of
tr
a
n
s
f
or
m
a
ti
ons
th
a
t
r
e
f
in
e
th
e
f
us
e
d
f
e
a
tu
r
e
s
.
T
he
a
r
c
hi
te
c
tu
r
e
of
th
e
M
P
D
R
F
-
N
e
t
m
ode
l,
de
pi
c
te
d
in
F
ig
ur
e
2,
e
nc
om
pa
s
s
e
s
b
il
a
t
e
r
a
l
c
ha
nne
l
pi
xe
l
w
e
ig
ht
e
d
(
B
C
P
W
)
m
odul
e
s
,
s
pe
c
if
ic
a
ll
y
de
s
ig
ne
d
to
le
ve
r
a
ge
th
e
r
ic
h
a
r
r
a
y
of
f
in
e
-
g
r
a
in
e
d
f
e
a
tu
r
e
s
pr
e
s
e
nt
in
bot
h
a
udi
o
a
nd
vi
s
ua
l
da
ta
.
B
y
in
c
or
por
a
ti
ng
D
A
R
m
odul
e
s
a
lo
ng
s
id
e
r
e
s
i
dua
l
tr
a
ns
f
or
m
e
r
s
,
th
e
m
ode
l
a
de
pt
ly
c
a
pt
ur
e
s
lo
ng
-
di
s
ta
nc
e
in
f
or
m
a
ti
on,
f
a
c
il
it
a
ti
ng
th
e
ge
ne
r
a
ti
on
of
f
us
e
d
f
e
a
tu
r
e
s
th
a
t
e
nc
a
ps
ul
a
te
th
e
e
s
s
e
nc
e
of
bot
h
m
oda
li
ti
e
s
.
T
h
e
s
e
f
us
e
d
f
e
a
tu
r
e
s
a
r
e
s
ubs
e
qu
e
nt
ly
in
put
te
d
in
to
a
c
la
s
s
if
ic
a
ti
on
m
ode
l,
s
uc
h
a
s
a
s
e
lf
-
a
tt
e
nt
io
n
m
ount
a
in
ga
z
e
ll
e
C
N
N
,
to
pr
e
di
c
t
th
e
e
m
ot
io
na
l
s
ta
te
of
th
e
in
di
vi
dua
l
in
th
e
da
ta
s
e
t.
T
hr
ough
th
e
M
P
D
R
F
-
N
e
t,
th
e
e
xt
r
a
c
te
d
f
e
a
tu
r
e
s
f
r
om
a
udi
o
a
nd
vi
de
o
s
our
c
e
s
a
r
e
f
us
e
d,
u
ti
li
z
in
g
pr
ogr
e
s
s
iv
e
f
e
a
tu
r
e
f
us
io
n
te
c
hni
que
s
to
m
it
ig
a
te
in
f
or
m
a
ti
on
lo
s
s
e
f
f
e
c
ti
ve
ly
.
T
he
a
udi
o
a
nd
vi
de
o
f
e
a
tu
r
e
s
,
e
a
c
h
r
e
pr
e
s
e
nt
e
d
by
a
(
10,10
)
m
a
tr
ix
,
a
r
e
f
e
d
in
to
in
put
la
ye
r
s
1
a
nd
2,
r
e
s
p
e
c
ti
ve
ly
,
w
he
r
e
th
e
P
M
C
-
N
e
t
in
th
e
f
e
a
tu
r
e
f
us
io
n
la
ye
r
pr
og
r
e
s
s
iv
e
ly
in
te
gr
a
te
s
m
ul
ti
m
oda
l
f
e
a
tu
r
e
s
w
hi
le
le
a
r
ni
ng
th
e
in
tr
ic
a
c
ie
s
of
e
a
c
h
la
ye
r
.
T
he
f
us
io
n
pr
oc
e
s
s
in
te
gr
a
te
s
c
ol
le
c
te
d
f
e
a
tu
r
e
s
us
in
g
a
c
om
bi
na
ti
on
of
D
A
R
m
od
ul
e
s
a
nd
r
e
s
id
ua
l
tr
a
n
s
f
or
m
e
r
s
,
f
a
c
il
it
a
ti
ng
th
e
e
xt
r
a
c
ti
on
of
c
r
uc
ia
l
lo
ng
-
di
s
ta
nc
e
in
f
or
m
a
ti
on.
D
ur
in
g
de
c
odi
ng
,
th
e
f
us
e
d
f
e
a
tu
r
e
in
f
or
m
a
ti
on
is
r
e
c
on
s
tr
uc
te
d
us
in
g
th
e
e
xt
r
a
c
te
d
s
e
m
a
nt
ic
in
f
or
m
a
ti
on,
w
it
h
a
ggr
e
ga
te
d
r
e
s
id
ua
l
ope
r
a
ti
ons
ups
a
m
pl
in
g
f
e
a
tu
r
e
s
a
c
r
os
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
O
pt
imi
z
e
d m
ul
ti
-
la
y
e
r
s
e
lf
-
at
te
nt
io
n ne
tw
or
k
f
or
f
e
at
u
r
e
-
le
v
e
l
d
at
a f
us
io
n i
n …
(
B
as
am
m
a U
m
e
s
h P
at
il
)
4439
s
ta
ge
s
1
to
3.
E
m
pl
oyi
ng
de
ns
e
s
ki
p
c
onne
c
ti
ons
f
or
in
put
f
e
a
tu
r
e
s
,
node
s
in
s
ta
ge
s
1
to
4
e
nha
n
c
e
th
e
m
ove
m
e
nt
of
f
e
a
tu
r
e
in
f
or
m
a
ti
on
a
t
e
a
c
h
la
ye
r
,
ul
ti
m
a
te
ly
a
m
a
lg
a
m
a
ti
ng
f
e
a
tu
r
e
s
f
r
om
a
ll
m
oda
li
ti
e
s
to
ge
ne
r
a
te
f
us
e
d
f
e
a
tu
r
e
s
.
F
ig
ur
e
2. A
r
c
hi
te
c
tu
r
e
of
t
he
pr
opos
e
d f
us
io
n m
ode
l
2.5.
S
e
lf
-
at
t
e
n
t
io
n
m
ou
n
t
ai
n
g
az
e
ll
e
c
on
vol
u
t
i
on
al
n
e
u
r
al
n
e
t
w
or
k
-
b
as
e
d
c
la
s
s
i
f
i
e
r
f
o
r
8
e
m
ot
io
n
al
s
t
at
e
s
T
he
s
ub
s
e
que
nt
ph
a
s
e
in
vol
ve
s
de
pl
oyi
ng
th
e
S
M
G
C
N
N
,
a
n
a
tt
e
nt
io
n
m
e
c
ha
ni
s
m
-
in
te
gr
a
te
d
de
e
p
C
N
N
,
s
pe
c
if
ic
a
ll
y
ta
il
or
e
d
f
or
e
m
ot
io
n
r
e
c
ogni
ti
on
f
r
om
f
a
c
ia
l
e
xpr
e
s
s
io
ns
a
nd
a
udi
o
c
ue
s
.
D
e
e
p
le
a
r
ni
ng
pe
r
f
or
m
a
nc
e
s
c
a
le
s
up
by
in
it
ia
ll
y
dow
ns
a
m
pl
in
g
a
nd
e
xt
r
a
c
ti
ng
e
xpr
e
s
s
io
n
f
e
a
tu
r
e
s
,
a
ppl
yi
ng
th
e
m
to
a
s
e
lf
-
a
tt
e
nt
io
n r
e
s
id
ua
l
m
odul
e
. T
he
f
in
a
l
e
xpr
e
s
s
io
n i
s
c
la
s
s
if
ie
d
t
hr
ough a
f
ul
ly
c
onne
c
te
d l
a
ye
r
.
2.5.1. Re
s
id
u
al
n
e
t
w
or
k
an
d
s
e
lf
-
at
t
e
n
t
io
n
m
od
u
le
T
he
a
dopt
io
n
of
a
r
e
s
id
ua
l
ne
twor
k
a
ddr
e
s
s
e
s
th
e
c
h
a
ll
e
nge
of
th
e
gr
a
di
e
nt
va
ni
s
hi
ng
pr
obl
e
m
e
nc
ount
e
r
e
d
w
he
n
de
e
pe
ni
ng
th
e
ne
twor
k
la
ye
r
s
,
m
a
ki
ng
it
di
f
f
ic
ul
t
to
e
f
f
e
c
ti
ve
ly
tr
a
in
a
m
ode
l.
B
y
e
m
pl
oyi
ng
a
r
e
s
id
ua
l
ne
twor
k,
th
e
c
onvolut
io
na
l
la
ye
r
le
a
r
ns
th
e
r
e
s
id
ua
l
be
twe
e
n
in
put
a
nd
out
put
,
f
a
c
il
it
a
ti
ng
th
e
id
e
nt
if
ic
a
ti
on
of
th
e
m
a
ppi
ng
be
tw
e
e
n
th
e
in
put
a
nd
out
put
by
e
s
ta
bl
is
hi
ng
c
onne
c
ti
on
s
be
tw
e
e
n
th
e
m
.
T
he
out
put
of
t
he
r
e
s
id
ua
l
ne
twor
k i
s
de
f
in
e
d m
a
th
e
m
a
ti
c
a
ll
y, r
e
pr
e
s
e
nt
in
g t
he
out
put
of
t
he
r
e
s
id
u
a
l
c
onne
c
ti
on
a
s
a
f
unc
ti
on
of
th
e
in
put
ve
c
to
r
a
nd
th
e
m
a
ppi
ng
be
tw
e
e
n
in
p
ut
a
nd
out
put
.
A
ddi
ti
ona
ll
y,
th
e
s
e
lf
-
a
tt
e
nt
io
n
m
odul
e
is
in
tr
oduc
e
d
to
ove
r
c
om
e
th
e
li
m
it
a
ti
ons
im
pos
e
d
by
t
he
s
m
a
ll
s
iz
e
of
c
onvolut
io
na
l
ke
r
ne
l
s
iz
e
s
in
C
N
N
s
,
w
hi
c
h
r
e
s
tr
ic
ts
e
a
c
h
c
onvolut
io
na
l
ope
r
a
ti
on
to
c
ove
r
in
g
a
s
m
a
ll
a
r
e
a
a
r
ound
th
e
pi
xe
l.
T
hi
s
m
odul
e
le
ve
r
a
ge
s
non
-
lo
c
a
l
ope
r
a
ti
ons
to
c
a
lc
ul
a
t
e
th
e
c
or
r
e
la
ti
on
w
e
ig
h
t
be
twe
e
n
e
a
c
h
po
s
it
io
n
of
th
e
f
e
a
tu
r
e
m
a
p
a
nd
th
e
im
a
ge
s
ig
na
l
r
e
pr
e
s
e
nt
a
ti
on,
e
na
bl
in
g
th
e
c
a
pt
ur
e
of
lo
ng
-
di
s
ta
nc
e
f
e
a
tu
r
e
s
.
T
hr
ough
va
r
io
us
tr
a
ns
f
or
m
a
ti
ons
a
nd
nor
m
a
li
z
a
ti
on
f
unc
ti
ons
,
th
e
s
e
lf
-
a
tt
e
nt
io
n
la
ye
r
tr
a
ns
f
or
m
s
in
put
f
e
a
tu
r
e
m
a
ps
in
to
two
e
m
be
ddi
ng
s
pa
c
e
s
,
r
e
duc
in
g
bot
h
th
e
c
om
put
a
ti
ona
l
c
om
pl
e
xi
ty
a
nd
th
e
num
be
r
of
c
ha
nne
ls
.
T
he
out
put
of
th
e
a
tt
e
nt
io
n
la
ye
r
is
obt
a
in
e
d
th
r
ough
m
a
tr
ix
m
ul
ti
pl
ic
a
ti
on
w
it
h
le
a
r
ni
ng
pa
r
a
m
e
te
r
s
,
c
ont
r
ib
ut
in
g
to
th
e
pr
opa
ga
ti
on
of
gr
a
di
e
nt
s
th
r
ough
th
e
r
e
s
id
ua
l
c
onne
c
ti
on.
D
ur
i
ng
m
ode
l
tr
a
in
in
g,
bi
na
r
y
c
r
os
s
-
e
nt
r
opy
lo
s
s
is
e
m
pl
oye
d
to
m
e
a
s
ur
e
th
e
di
s
s
im
il
a
r
it
y
be
twe
e
n
pr
e
d
ic
te
d
pr
o
ba
bi
li
ti
e
s
a
nd
tr
ue
bi
na
r
y
la
be
l
s
,
pr
ovi
di
ng
a
m
e
c
ha
ni
s
m
f
or
opt
im
iz
in
g m
ode
l
pe
r
f
or
m
a
nc
e
.
2.5.2. L
e
ar
n
in
g p
ar
am
e
t
e
r
op
t
im
iz
at
io
n
u
s
in
g m
ou
n
t
ai
n
ga
z
e
ll
e
op
t
im
iz
e
r
I
ns
pi
r
e
d
by
th
e
hi
e
r
a
r
c
hi
c
a
l
s
tr
uc
tu
r
e
a
nd
s
oc
ia
l
dyna
m
ic
s
obs
e
r
ve
d
in
w
il
d
m
ount
a
in
ga
z
e
ll
e
s
,
th
e
m
ount
a
in
ga
z
e
ll
e
opt
im
iz
e
r
(
M
G
O
)
opt
im
iz
e
s
le
a
r
ni
ng
r
a
te
pa
r
a
m
e
te
r
s
w
it
hi
n
th
e
s
e
lf
-
a
tt
e
nt
io
n
m
e
c
ha
ni
s
m
-
in
te
gr
a
te
d
de
e
p c
onvolut
io
na
l
ne
ur
a
l
ne
twor
k
(
S
M
G
C
N
N
)
.
L
e
ve
r
a
gi
ng
te
r
r
it
o
r
ia
l
be
ha
vi
or
s
,
he
r
d
dyna
m
ic
s
,
a
nd
m
ig
r
a
ti
on
pa
tt
e
r
ns
of
ga
z
e
ll
e
s
,
th
is
opt
im
iz
e
r
or
c
he
s
tr
a
te
s
a
n
e
f
f
e
c
ti
ve
le
a
r
ni
ng
r
a
te
f
or
th
e
c
la
s
s
if
ie
r
.
T
he
a
lg
or
it
hm
of
th
e
M
G
O
[
3
4
]
is
e
num
e
r
a
te
d
be
l
ow
.
T
he
A
lg
or
it
hm
1,
de
s
c
r
ib
e
d
is
a
he
ur
is
ti
c
opt
im
iz
a
ti
on
a
ppr
oa
c
h
in
s
pi
r
e
d
by
th
e
s
oc
ia
l
be
ha
vi
or
s
of
g
a
z
e
ll
e
s
in
na
tu
r
e
.
I
t
b
e
gi
ns
by
in
it
ia
li
z
in
g
a
popula
ti
on
of
ga
z
e
ll
e
s
w
it
h
50
a
nd
to
ta
l
it
e
r
a
ti
ons
to
100.
E
a
c
h
ga
z
e
ll
e
’
s
f
it
ne
s
s
is
th
e
n
e
va
lu
a
te
d
ba
s
e
d
on
m
a
th
e
m
a
ti
c
a
l
r
e
pr
e
s
e
nt
a
ti
ons
of
be
h
a
vi
or
s
s
uc
h
a
s
te
r
r
it
or
ia
l
s
ol
it
a
r
y
m
a
le
s
(
T
S
M
)
,
m
a
te
r
ni
ty
he
r
ds
(
M
H
)
,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
, V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024:
4435
-
444
4
4440
ba
c
he
lo
r
m
a
le
he
r
ds
(
B
M
H
)
,
a
nd
m
ig
r
a
ti
on
to
s
e
a
r
c
h
f
or
f
ood
(
M
S
F
)
.
T
he
s
e
be
ha
vi
or
s
a
r
e
f
or
m
ul
a
te
d
a
s
e
qua
ti
ons
th
a
t
a
s
s
e
s
s
a
s
p
e
c
ts
li
ke
te
r
r
it
or
y
e
s
ta
bl
is
hm
e
nt
,
r
e
pr
oduc
ti
on,
a
nd
r
e
s
our
c
e
s
e
a
r
c
hi
ng.
T
he
f
it
ne
s
s
f
unc
ti
on
is
de
f
in
e
d
a
s
th
e
m
in
im
iz
a
ti
on
of
th
e
lo
s
s
f
unc
ti
on
of
th
e
S
M
G
C
N
N
,
c
ons
id
e
r
in
g
pr
e
di
c
te
d
pr
oba
bi
li
ti
e
s
a
nd
tr
ue
bi
na
r
y
la
be
ls
.
T
h
e
popula
ti
on
is
th
e
n
a
r
r
a
nge
d
ba
s
e
d
on
f
it
ne
s
s
,
a
nd
th
e
po
s
it
io
n
of
th
e
be
s
t
-
pe
r
f
or
m
in
g
ga
z
e
ll
e
is
upda
t
e
d
a
c
c
or
di
ngl
y.
T
hi
s
pr
oc
e
s
s
it
e
r
a
te
s
unt
il
a
t
e
r
m
in
a
ti
on
c
ondi
ti
on
is
m
e
t,
ty
pi
c
a
ll
y
in
vol
vi
ng
a
m
a
xi
m
um
num
be
r
of
it
e
r
a
ti
ons
or
s
a
ti
s
f
a
c
to
r
y
f
it
ne
s
s
le
ve
ls
.
U
lt
im
a
te
ly
,
th
e
a
lg
or
it
hm
a
im
s
to
opt
im
iz
e
le
a
r
ni
ng
r
a
te
pa
r
a
m
e
te
r
s
w
it
hi
n
th
e
S
M
G
C
N
N
m
ode
l,
e
nha
nc
in
g
it
s
pr
e
di
c
ti
ve
c
a
pa
bi
li
ti
e
s
by
le
ve
r
a
gi
ng pr
in
c
ip
le
s
in
s
pi
r
e
d by ga
z
e
ll
e
s
oc
ia
l
dyna
m
ic
s
.
A
lg
or
it
hm
1. M
ount
a
in
ga
z
e
ll
e
opt
im
iz
e
r
1
I
nput
T
ot
a
lI
te
r
a
ti
on=
100, P
opul
a
ti
onS
iz
e
=
50
2
O
ut
put
s
O
pt
im
a
lP
os
it
io
n, B
e
s
tF
it
ne
s
s
3
I
ni
ti
a
li
z
e
P
opul
a
ti
on r
a
ndoml
y a
s
X
_i
(
i=
1, 2, ..., P
opul
a
ti
onS
iz
e
)
3
C
a
lc
ul
a
te
f
it
ne
s
s
l
e
ve
ls
f
or
e
a
c
h i
ndi
vi
dua
l
in
t
he
popula
ti
on
4
W
hi
le
c
ondi
ti
on not m
e
t
do:
5
F
or
e
a
c
h G
a
z
e
ll
e
(
X
_i
)
i
n P
opul
a
ti
on do:
6
C
a
lc
ul
a
te
T
S
M
, M
H
,
B
M
H
, a
nd
M
S
F
be
ha
vi
or
s
7
C
a
lc
ul
a
te
f
it
ne
s
s
va
lu
e
s
f
or
e
a
c
h b
e
ha
vi
or
:
8
T
S
M
F
it
ne
s
s
=
|m
a
le
-
|(
r
i_
1*B
H
-
r
i_
2*X
(
t)
)
*
F
|*C
of
_r
|
9
M
H
F
it
ne
s
s
=
|(
B
H
+
C
of
_(
1,r
)
)
+
(
r
i_
3* ma
le
-
r
i_
4*X
_r
a
nd)
*C
of
_
(
1, r
)
|
10
B
M
H
F
it
ne
s
s
=
|(
X
(
t)
-
D
)
+
(
r
i_
5*±*ma
le
-
r
i_
6*B
H
)
*C
of
_
r
|
11
M
S
F
F
it
ne
s
s
=
|(
ub
-
lb
)
*r
i_
7+
lb
|
12
C
a
lc
ul
a
te
f
it
ne
s
s
va
lu
e
a
s
t
he
m
in
im
iz
a
ti
on of
t
he
l
os
s
f
unc
ti
on
of
S
M
G
C
N
N
13
=
{
∑
(
∗
(
_
)
+
(
1
−
)
∗
(
1
−
_
)
)
}
14
A
r
r
a
nge
t
he
ga
z
e
ll
e
popula
ti
on ba
s
e
d on f
it
ne
s
s
va
lu
e
s
U
pda
te
t
he
pos
it
io
n of
t
he
be
s
t
ga
z
e
ll
e
15
E
nd w
hi
le
16
R
e
tu
r
n t
he
be
s
t
ga
z
e
ll
e
'
s
pos
it
io
n a
nd i
ts
f
it
ne
s
s
va
lu
e
17
E
nd P
r
oc
e
dur
e
3.
E
X
P
E
R
I
M
E
N
T
A
L
R
E
S
U
L
T
S
F
or
e
xpe
r
im
e
nt
a
ti
on,
W
in
dow
s
10
P
r
o,
s
pe
c
if
ic
a
ll
y
on
bui
ld
19045.2965,
w
a
s
e
m
pl
oye
d
a
s
th
e
ope
r
a
ti
ng
s
ys
te
m
.
T
he
pr
ogr
a
m
m
in
g
la
ngua
ge
P
yt
hon
ve
r
s
io
n
3.10.9
a
nd
li
br
a
r
ie
s
s
uc
h
a
s
N
um
P
y,
L
ib
r
os
a
,
M
a
tp
lo
tl
ib
, O
pe
nC
V
, a
nd T
e
ns
or
F
lo
w
a
r
e
ut
il
iz
e
d t
o i
m
pl
e
m
e
n
t
t
he
t
a
s
ks
w
it
hi
n t
he
e
xpe
r
im
e
nt
. T
he
da
ta
s
e
ts
us
e
d
f
or
a
na
ly
s
is
a
r
e
s
our
c
e
d
f
r
om
R
A
V
D
E
S
S
.
T
he
pr
opos
e
d
O
M
S
N
-
D
E
R
m
ode
l
is
bui
lt
a
nd
ut
il
iz
e
d
f
or
e
m
ot
io
n
r
e
c
ogni
ti
on
a
c
ti
vi
ty
.
T
he
s
y
s
te
m
is
e
qui
ppe
d
w
it
h
16.
0
G
B
of
R
A
M
,
w
it
h
15.8
G
B
be
in
g
us
a
bl
e
f
or
pr
oc
e
s
s
in
g
ta
s
k
s
th
r
oughout
th
e
e
xpe
r
im
e
nt
s
.
T
o
m
a
ke
a
c
om
pl
e
te
de
s
c
r
ip
ti
on
of
m
ul
ti
m
oda
l
e
m
ot
io
n
r
e
c
ogni
ti
on,
th
is
s
e
c
ti
on
di
s
c
us
s
e
s
th
e
R
A
V
D
E
S
S
da
ta
s
e
t
-
ba
s
e
d
di
f
f
e
r
e
nt
s
c
or
e
s
to
c
om
put
e
th
e
s
im
il
a
r
it
ie
s
a
nd
di
f
f
e
r
e
nc
e
s
be
twe
e
n pr
opos
e
d a
nd t
r
a
di
ti
ona
l
m
e
th
ods
. T
h
is
s
e
c
ti
on a
ls
o t
a
ke
s
a
qua
nt
it
a
ti
ve
a
na
ly
s
is
i
n t
e
r
m
s
of
s
om
e
e
va
lu
a
ti
on
m
e
tr
ic
s
im
pl
e
m
e
nt
e
d
in
P
Y
T
H
O
N
.
T
he
e
va
lu
a
ti
on
m
e
tr
ic
s
s
uc
h
a
s
a
c
c
ur
a
c
y,
r
e
c
a
ll
,
F1
-
s
c
or
e
,
s
pe
c
if
ic
it
y,
pr
e
c
is
io
n,
a
nd
r
e
c
e
iv
e
r
ope
r
a
ti
ng
c
ha
r
a
c
t
e
r
is
ti
c
s
(
R
O
C
)
c
ur
ve
a
r
e
us
e
d
to
c
om
p
a
r
e
th
e
pr
opos
e
d m
ode
l
w
it
h t
he
r
e
la
te
d m
e
th
ods
.
3.1. Con
f
u
s
io
n
m
at
r
ix
f
or
m
u
lt
im
od
al
f
e
at
u
r
e
s
T
he
pe
r
f
or
m
a
nc
e
a
s
s
e
s
s
m
e
nt
,
a
s
de
pi
c
te
d
by
th
e
c
onf
us
io
n
m
a
tr
ix
,
hi
ghl
ig
ht
s
th
e
m
ode
l’
s
a
c
c
ur
a
c
y
in
pr
e
di
c
ti
ng
e
m
ot
io
na
l
s
ta
te
s
.
I
t
a
ls
o
r
e
v
e
a
ls
pot
e
nt
ia
l
a
r
e
a
s
of
m
is
c
la
s
s
if
ic
a
ti
on
a
nd
c
onf
us
io
n
be
twe
e
n
c
e
r
ta
in
e
m
ot
io
ns
. A
c
c
or
di
ng t
o
F
ig
ur
e
3, t
he
m
ode
l
d
e
m
on
s
tr
a
te
d e
x
c
e
l
le
nt
pe
r
f
or
m
a
nc
e
f
or
m
os
t
s
a
m
pl
e
s
,
w
it
h a
f
e
w
e
xc
e
pt
io
ns
.
A
t
‘
ne
ut
r
a
l’
,
s
om
e
two
c
la
s
s
e
s
a
r
e
pr
e
di
c
te
d
a
s
‘
s
a
d’
a
nd
‘
f
e
a
r
f
ul
’
.
M
or
e
ove
r
,
th
e
two
‘
ha
ppy’
c
la
s
s
e
s
a
r
e
pr
e
di
c
te
d
a
s
‘
ne
ut
r
a
l’
a
nd ‘
s
ur
pr
is
e
d’
a
nd t
he
40 c
la
s
s
e
s
a
r
e
a
c
c
ur
a
te
ly
p
r
e
di
c
ti
ng
t
he
‘
ha
ppy’
c
l
a
s
s
.
T
he
two
c
la
s
s
e
s
in
‘
s
ur
pr
is
e
d’
a
r
e
c
onf
us
e
d
a
nd
pr
e
di
c
te
d
a
s
‘
c
a
lm
’
a
nd
‘
s
a
d’
.
T
hi
s
c
onf
us
io
n
be
twe
e
n
‘
s
a
d’
a
nd ‘
f
e
a
r
f
ul
’
c
a
n be
a
tt
r
ib
ut
e
d t
o t
he
l
ow
a
r
ous
a
l
le
ve
ls
pr
e
s
e
nt
i
n both e
m
ot
io
ns
.
3.2. An
al
ys
is
of
r
u
n
t
im
e
a
c
c
u
r
ac
y an
d
l
os
s
T
he
m
e
th
od
unde
r
w
e
nt
t
r
a
in
in
g
a
nd e
v
a
lu
a
ti
on ove
r
200
e
poc
h
s
, di
vi
di
ng t
he
R
A
V
D
E
S
S
d
a
ta
s
e
t
i
nt
o
80%
tr
a
in
in
g
a
nd
20%
va
li
da
ti
on
s
ubs
e
ts
.
T
h
e
O
M
S
N
-
D
E
R
a
ppr
oa
c
h
is
a
n
a
ly
z
e
d
in
F
ig
ur
e
4,
w
he
r
e
F
ig
ur
e
4(
a
)
s
how
s
th
e
hi
gh
tr
a
in
in
g
a
c
c
ur
a
c
y a
tt
a
in
e
d,
a
nd
F
ig
u
r
e
4(
b)
de
m
ons
tr
a
te
s
a
de
c
r
e
a
s
e
d
tr
a
in
in
g
lo
s
s
f
or
m
ul
ti
m
oda
l
da
ta
.
T
he
r
e
s
ul
ts
s
p
e
c
if
y
th
a
t
th
e
pr
opos
e
d
m
ode
l
e
xhi
bi
ts
hi
ghe
r
a
c
c
ur
a
c
y
in
e
m
ot
io
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
O
pt
imi
z
e
d m
ul
ti
-
la
y
e
r
s
e
lf
-
at
te
nt
io
n ne
tw
or
k
f
or
f
e
at
u
r
e
-
le
v
e
l
d
at
a f
us
io
n i
n …
(
B
as
am
m
a U
m
e
s
h P
at
il
)
4441
id
e
nt
if
ic
a
ti
on
f
or
m
ul
ti
m
oda
l
da
ta
(
a
udi
o
a
nd
vi
de
o)
.
F
ur
th
e
r
m
or
e
,
it
de
m
ons
tr
a
te
d
s
upe
r
io
r
pe
r
f
or
m
a
nc
e
in
bot
h
th
e
tr
a
in
in
g
a
nd
te
s
ti
ng
s
e
ts
.
B
e
tt
e
r
a
c
c
ur
a
c
y
a
nd
lo
w
lo
s
s
va
lu
e
a
r
e
a
c
hi
e
ve
d
a
f
te
r
c
r
os
s
in
g
100
it
e
r
a
ti
ons
.
F
ig
ur
e
3. C
onf
us
io
n m
a
tr
ix
of
R
A
V
D
E
S
S
da
ta
s
e
t
(
a
)
(
b)
F
ig
ur
e
4. A
na
ly
s
is
f
or
di
f
f
e
r
e
nt
e
poc
hs
of
(
a
)
a
c
c
ur
a
c
y a
nd (
b)
l
os
s
3.3. R
e
c
e
iv
e
r
op
e
r
at
in
g c
h
ar
ac
t
e
r
i
s
t
ic
s
c
u
r
v
e
F
ig
ur
e
5
s
how
s
th
e
R
O
C
c
ur
ve
f
or
th
e
m
in
im
um
num
be
r
of
s
a
m
pl
e
s
ne
e
de
d
to
tr
a
in
a
n
S
M
G
C
N
N
,
w
hi
c
h i
s
hi
ghl
y
de
pe
nde
nt
on t
he
s
p
e
c
if
ic
c
la
s
s
if
ic
a
ti
on t
a
s
k.
T
h
e
R
O
C
c
ur
ve
us
e
s
t
he
c
a
lc
ul
a
t
e
d f
a
ls
e
pos
it
iv
e
r
a
te
(
F
P
R
)
a
nd
tr
ue
pos
it
iv
e
r
a
te
(
T
P
R
)
va
lu
e
s
.
I
t
is
a
pl
ot
of
T
P
R
a
ga
in
s
t
F
P
R
,
w
it
h
di
f
f
e
r
e
nt
poi
nt
s
on
th
e
c
ur
ve
c
or
r
e
s
ponding
to
di
f
f
e
r
e
nt
c
la
s
s
if
ic
a
ti
on
th
r
e
s
hol
ds
.
T
h
e
R
O
C
c
ur
ve
s
ta
r
ts
a
t
(
0.0,
0.0)
a
nd
e
nds
a
t
(
1.0, 1.0)
, i
ndi
c
a
ti
ng t
ha
t
it
s
uc
c
e
s
s
f
ul
ly
c
la
s
s
if
ie
s
a
ll
po
s
it
iv
e
s
a
m
pl
e
s
.
F
ig
ur
e
5. R
O
C
c
ur
ve
of
S
M
G
C
N
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
, V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024:
4435
-
444
4
4442
3.4. Com
p
ar
is
on
o
f
O
M
S
N
-
D
E
R
w
it
h
e
xi
s
t
in
g e
m
ot
io
n
r
e
c
o
gn
it
io
n
m
e
t
h
od
s
T
a
bl
e
1
s
how
s
th
e
c
om
pa
r
is
on
of
O
M
S
N
-
D
E
R
w
it
h
e
xi
s
ti
ng
e
m
ot
io
na
l
r
e
c
ogni
ti
on
m
e
th
ods
w
h
e
r
e
“
-
”
i
ndi
c
a
te
s
t
ha
t
th
e
c
or
r
e
s
ponding e
va
lu
a
ti
on me
tr
ic
s
a
r
e
not
a
va
il
a
bl
e
i
n t
ha
t
r
e
s
e
a
r
c
h. A
ll
e
va
lu
a
ti
on
s
how
s
th
a
t
th
e
pr
opos
e
d
m
e
th
od
a
c
hi
e
ve
s
hi
gh
a
c
c
ur
a
c
y
a
nd
ot
he
r
s
c
or
e
s
w
hi
c
h
c
onc
lu
de
s
th
a
t
th
e
O
M
S
N
-
D
E
R
a
tt
a
in
s
be
tt
e
r
pe
r
f
or
m
a
nc
e
in
th
e
c
la
s
s
if
ic
a
ti
on
of
a
ll
c
la
s
s
e
s
. T
he
f
us
io
n
a
ppr
oa
c
h
pr
ovi
de
s
th
e
be
s
t
a
nd
m
os
t
r
e
le
va
nt
f
e
a
tu
r
e
s
of
a
ll
c
la
s
s
e
s
w
he
r
e
a
ll
th
e
in
di
vi
dua
l
f
e
a
tu
r
e
s
a
r
e
c
onc
a
te
na
te
d
a
nd
gi
ve
n
to
c
la
s
s
if
ic
a
ti
on.
T
hi
s
r
e
duc
e
s
t
he
c
om
put
a
ti
ona
l
c
om
pl
e
xi
ty
a
nd t
im
e
.
T
a
bl
e
1. C
om
pa
r
is
on of
O
M
S
N
-
D
E
R
w
it
h e
xi
s
ti
ng me
th
ods
T
e
c
hni
que
s
P
r
e
c
i
s
i
on
F1
-
s
c
or
e
A
c
c
ur
a
c
y
R
e
c
a
l
l
S
pe
c
i
f
i
c
i
t
y
S
E
R
a
nd F
E
R
[
13]
0.9399
-
0.9805
0.7237
-
O
pt
i
m
a
l
m
ul
t
i
m
oda
l
e
m
ot
i
on r
e
c
ogni
t
i
on
[
14]
0.86
0.86
0.86
0.86
0.98
B
i
di
r
e
c
t
i
ona
l
pa
r
a
l
l
e
l
E
S
N
[
15]
0.8656
0.8519
0.7539
0.8854
-
M
F
A
[
16]
0.97
0.97
0.9507
0.98
-
M
C
C
A
[
17
]
0.9098
0.9014
0.9525
0.9061
-
S
e
l
f
-
s
upe
r
vi
s
e
d l
e
a
r
ni
ng
a
l
gor
i
t
hm
[2
8
]
0.8950
0.8168
0.876
0.8550
-
3D
C
N
N
-
L
S
T
M
[
29
]
-
0.9609
0.90
-
99.45
O
M
S
N
-
D
E
R
(
P
r
opos
e
d)
0.9633
0.9633
0. 9908
0.9633
0.9947
3.5. Com
p
ar
is
on
o
f
O
M
S
N
-
D
E
R
w
it
h
e
xi
s
t
in
g e
m
ot
io
n
r
e
c
o
gn
it
io
n
m
e
t
h
od
s
T
a
bl
e
2
il
lu
s
tr
a
te
s
th
e
c
om
pa
r
is
on
of
th
e
a
c
c
ur
a
c
y
of
O
M
S
N
-
D
E
R
w
it
h
e
xi
s
ti
ng
m
e
th
ods
ba
s
e
d
on
8
e
m
ot
io
na
l
c
la
s
s
e
s
(
ha
ppy,
f
e
a
r
f
ul
,
c
a
lm
,
a
ngr
y,
s
a
d,
di
s
gu
s
te
d, ne
ut
r
a
l
,
a
nd
s
ur
pr
is
e
d)
w
he
r
e
“
-
”
in
di
c
a
te
s
th
a
t
th
e
c
or
r
e
s
ponding
c
la
s
s
w
a
s
not
c
la
s
s
if
ie
d
by
th
e
e
xi
s
ti
ng
m
e
th
ods
.
F
r
om
a
ll
th
e
obs
e
r
va
ti
ons
,
it
is
obvi
ous
th
a
t
th
e
O
M
S
N
-
D
E
R
a
tt
a
in
s
hi
gh
a
c
c
ur
a
c
y
in
a
ll
8
c
la
s
s
e
s
.
I
t
is
c
onc
lu
de
d
th
a
t
th
e
O
M
S
N
-
D
E
R
a
tt
a
in
s
be
tt
e
r
pe
r
f
or
m
a
nc
e
in
th
e
c
la
s
s
i
f
ic
a
ti
on
of
a
ll
c
la
s
s
e
s
.
C
l
a
s
s
if
ic
a
ti
on
r
e
s
ul
ts
a
r
e
ba
s
e
d
on
f
a
c
to
r
s
s
u
c
h
a
s
s
iz
e
,
a
nd
qua
li
ty
of
t
he
da
ta
s
e
t,
t
he
a
r
c
hi
te
c
tu
r
e
a
nd pa
r
a
m
e
te
r
s
of
t
he
m
ode
l,
t
he
pr
e
pr
oc
e
s
s
in
g t
e
c
hni
que
s
a
ppl
ie
d, a
nd
th
e
f
us
io
n t
a
s
k a
t
ha
nd.
T
a
bl
e
2. C
om
pa
r
is
on of
O
M
S
N
-
D
E
R
a
c
c
ur
a
c
y w
it
h e
xi
s
ti
ng me
th
ods
ba
s
e
d on 8
e
m
ot
io
na
l
c
la
s
s
e
s
T
e
c
hni
que
s
H
a
ppy
F
e
a
r
f
ul
C
a
l
m
A
ngr
y
S
a
d
D
i
s
gus
t
e
d
N
e
ut
r
a
l
S
ur
pr
i
s
e
d
B
i
di
r
e
c
t
i
ona
l
pa
r
a
l
l
e
l
E
S
N
[
11]
0.73
0.82
0.89
0.84
0.58
0.89
0.57
0.72
M
F
A
[
12]
0.97
0.98
-
0.96
0.99
0.99
0.96
-
S
e
l
f
-
s
upe
r
vi
s
e
d l
e
a
r
ni
ng
a
l
gor
i
t
hm
[
13]
0.79
0.806
0.816
0.830
0.782
0.851
0.773
0.883
M
ul
t
i
-
he
a
d
s
e
l
f
-
a
t
t
e
nt
i
on m
e
c
ha
ni
s
m
[
14]
0.87
0.85
-
0.97
0.98
0.97
-
0.95
D
e
e
p E
S
N
m
ode
l
[
16]
0.75
0.97
0.91
0.95
0.87
0.89
0.94
-
O
M
S
N
-
D
E
R
(
P
r
opos
e
d)
0.993
0.996
0.99
0.99
0.983
0.996
0.99
0.986
3.6. Dis
c
u
s
s
io
n
s
T
he
O
M
S
N
-
D
E
R
m
ode
l
m
a
r
ks
a
s
ig
ni
f
ic
a
nt
a
dva
nc
e
m
e
nt
in
th
e
f
ie
ld
o
f
m
ul
ti
m
oda
l
e
m
ot
io
n
r
e
c
ogni
ti
on,
s
how
c
a
s
in
g
r
e
m
a
r
ka
bl
e
a
c
c
ur
a
c
y
a
nd
e
f
f
ic
ie
nc
y
i
n
c
la
s
s
if
yi
ng
a
w
id
e
r
a
ng
e
of
e
m
ot
io
na
l
s
ta
te
s
f
r
om
a
udi
o
-
vi
s
ua
l
da
ta
.
I
t
is
in
nova
ti
ve
us
e
of
opt
im
iz
e
d
m
ul
ti
-
la
ye
r
s
e
lf
-
a
tt
e
nt
io
n
m
e
c
ha
ni
s
m
s
a
nd
dyna
m
ic
e
m
ot
io
n
r
e
c
ogni
ti
on
te
c
hni
que
s
s
e
t
s
a
ne
w
be
n
c
hm
a
r
k
a
nd
out
pe
r
f
or
m
s
e
xi
s
ti
ng
m
ode
ls
in
pr
e
c
is
io
n,
r
e
c
a
ll
,
a
nd
s
pe
c
if
ic
it
y.
T
hi
s
m
ode
l’
s
a
bi
li
ty
to
a
c
c
ur
a
te
ly
in
te
r
pr
e
t
c
o
m
pl
e
x
e
m
ot
io
na
l
e
xpr
e
s
s
io
ns
hol
ds
s
ubs
ta
nt
ia
l
pr
om
is
e
f
or
e
nha
nc
in
g
hum
a
n
-
c
om
put
e
r
i
nt
e
r
a
c
ti
on a
nd
e
m
ot
io
na
l
a
na
ly
s
i
s
a
ppl
ic
a
ti
on
s
. T
h
e
r
e
s
e
a
r
c
h, t
hough
a
dva
nc
e
d
in
m
ul
ti
m
oda
l
e
m
ot
io
n
r
e
c
ogni
ti
on,
ha
s
li
m
it
a
ti
on
s
.
I
t
pr
im
a
r
il
y
f
oc
us
e
s
on
a
udi
o
a
nd
vi
d
e
o,
ne
gl
e
c
ti
ng
th
e
r
ic
hne
s
s
of
te
xt
ua
l
a
nd
phys
io
lo
gi
c
a
l
da
ta
.
F
ut
u
r
e
r
e
s
e
a
r
c
h
s
houl
d
a
im
to
in
te
gr
a
te
t
e
xt
m
oda
li
t
y
to
e
nr
ic
h e
m
ot
io
n r
e
c
ogni
ti
on c
a
pa
bi
li
ti
e
s
.
4.
C
O
N
C
L
U
S
I
O
N
T
hi
s
r
e
s
e
a
r
c
h
e
m
pha
s
iz
e
s
e
m
ot
io
n
r
e
pr
e
s
e
nt
a
ti
on
a
nd
f
e
a
tu
r
e
f
us
io
n
us
in
g
O
M
S
N
-
D
E
R
f
or
a
c
c
ur
a
te
e
m
ot
io
na
l
a
s
s
e
s
s
m
e
nt
f
r
om
a
udi
o
a
nd
v
id
e
o
m
oda
li
ti
e
s
.
E
m
pl
o
yi
ng
M
P
D
R
F
-
N
e
t
a
nd
S
M
G
C
N
N
,
th
e
m
e
th
od
a
c
hi
e
ve
s
s
upe
r
io
r
e
m
ot
io
n
c
la
s
s
if
ic
a
ti
on
(
f
e
a
r
f
ul
,
a
ngr
y,
ne
ut
r
a
l,
s
ur
pr
is
e
d,
ha
ppy,
c
a
lm
,
di
s
gu
s
te
d,
a
nd
s
a
d)
on
th
e
R
A
V
D
E
S
S
da
ta
s
e
t.
E
va
lu
a
ti
on
a
ga
in
s
t
e
xi
s
ti
ng
m
e
th
ods
r
e
ve
a
ls
out
s
t
a
ndi
ng
pe
r
f
or
m
a
nc
e
w
it
h
a
n
a
c
c
ur
a
c
y
of
0.9908,
pr
e
c
is
io
n
of
0.9633,
r
e
c
a
ll
of
0.9633,
F
1
-
s
c
or
e
of
0.9633,
a
nd
s
pe
c
if
ic
it
y
of
0.9947.
T
he
s
e
r
e
s
ul
t
s
de
m
ons
tr
a
te
th
e
e
f
f
e
c
ti
ve
ne
s
s
of
O
M
S
N
-
D
E
R
in
m
ul
ti
m
oda
l
e
m
ot
io
n
r
e
c
ogni
ti
on,
c
ont
r
ib
ut
in
g
a
r
obus
t
a
ppr
oa
c
h
to
a
ddr
e
s
s
c
om
pl
e
xi
ty
in
e
m
ot
io
n
a
s
s
e
s
s
m
e
nt
.
T
he
r
e
s
e
a
r
c
h
pr
im
a
r
il
y
f
oc
us
e
s
on
a
udi
o
a
nd
vi
de
o
m
oda
li
ti
e
s
, pot
e
nt
ia
ll
y ove
r
lo
oki
ng t
he
be
ne
f
it
s
o
f
i
nc
or
por
a
ti
n
g ot
he
r
m
oda
li
ti
e
s
. T
hi
s
i
nc
lu
de
s
t
e
xt
, w
hi
c
h i
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
nt
J
A
r
ti
f
I
nt
e
ll
I
S
S
N
:
2252
-
8938
O
pt
imi
z
e
d m
ul
ti
-
la
y
e
r
s
e
lf
-
at
te
nt
io
n ne
tw
or
k
f
or
f
e
at
u
r
e
-
le
v
e
l
d
at
a f
us
io
n i
n …
(
B
as
am
m
a U
m
e
s
h P
at
il
)
4443
c
r
uc
ia
l
f
or
c
om
pr
e
he
ns
iv
e
e
m
ot
io
n
r
e
c
ogni
ti
on
s
y
s
te
m
s
.
A
ddi
ti
ona
ll
y,
th
e
s
tu
dy’
s
r
e
li
a
nc
e
on
s
p
e
c
if
ic
da
ta
s
e
ts
,
li
ke
R
A
V
D
E
S
S
,
r
a
is
e
s
c
on
c
e
r
ns
a
bout
ge
ne
r
a
li
z
a
bi
li
ty
,
w
a
r
r
a
nt
in
g
e
xpl
or
a
ti
on
a
c
r
os
s
di
ve
r
s
e
da
ta
s
e
ts
f
or
a
m
or
e
r
obus
t
e
va
lu
a
ti
on. I
n
t
he
f
ut
u
r
e
, m
or
e
e
va
lu
a
ti
on pa
r
a
m
e
te
r
s
w
il
l
be
f
oc
us
e
d on f
in
d
in
g t
he
pe
r
f
or
m
a
nc
e
of
t
he
r
e
s
e
a
r
c
h w
or
k a
nd w
il
l
im
pr
ove
t
he
m
ul
ti
m
oda
l
e
va
lu
a
ti
on w
it
h t
e
xt
m
oda
li
ty
t
oo. T
he
c
ur
r
e
nt
m
ode
l
is
va
li
da
te
d
by
e
m
pl
oyi
ng
va
r
io
us
a
dva
nc
e
d
f
us
io
n
m
ode
ls
.
A
ddi
ti
ona
ll
y,
opt
im
i
z
a
ti
on
of
c
r
uc
ia
l
hype
r
pa
r
a
m
e
te
r
s
is
pe
r
f
or
m
e
d, pote
nt
ia
ll
y i
nf
lu
e
nc
in
g t
he
c
la
s
s
if
ic
a
ti
on a
ppr
oa
c
h
s
ig
ni
f
ic
a
nt
ly
.
R
E
F
E
R
E
N
C
E
S
[
1]
M
.
L
i
e
t
al
.
,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
a
nd
s
t
a
t
e
a
na
l
ys
i
s
of
c
l
a
s
s
r
oo
m
vi
de
o
a
nd
a
udi
o
ba
s
e
d
on
de
e
p
ne
ur
a
l
ne
t
w
or
k,”
J
our
nal
of
I
nt
e
r
c
onne
c
t
i
on N
e
t
w
or
k
s
, vol
. 22, J
un. 2022, doi
:
10.1142/
S
0219265921460117.
[
2]
K
.
Z
ha
ng,
Y
.
L
i
,
J
.
W
a
ng,
Z
.
W
a
ng,
a
nd
X
.
L
i
,
“
F
e
a
t
ur
e
f
us
i
on
f
or
m
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
ba
s
e
d
on
de
e
p
c
a
noni
c
a
l
c
or
r
e
l
a
t
i
on a
na
l
ys
i
s
,”
I
E
E
E
Si
gnal
P
r
oc
e
s
s
i
ng L
e
t
t
e
r
s
, vol
. 28, pp. 1898
–
1902,
2021, doi
:
10.1109/
L
S
P
.2021.3112314.
[
3]
L.
-
N
.
D
o,
H
.
-
J
.
Y
a
ng,
H
.
-
D
.
N
guye
n,
S
.
-
H
.
K
i
m
,
G
.
-
S
.
L
e
e
,
a
nd
I
.
-
S
.
N
a
,
“
D
e
e
p
ne
ur
a
l
ne
t
w
or
k
-
ba
s
e
d
f
us
i
on
m
ode
l
f
or
e
m
ot
i
on
r
e
c
ogni
t
i
on us
i
ng vi
s
ua
l
da
t
a
,”
T
he
J
our
nal
of
Supe
r
c
om
put
i
ng
, vol
. 77, no.
10,
pp. 10773
–
10790, O
c
t
. 2021, doi
:
10.1007/
s
11227
-
021
-
03690
-
y.
[
4]
C
.
A
.
D
e
vi
a
nd
D
.
K
.
R
e
nuka
,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
f
r
a
m
e
w
or
k
us
i
ng
a
de
c
i
s
i
on
-
l
e
ve
l
f
us
i
on
a
nd
f
e
a
t
ur
e
-
l
e
ve
l
f
us
i
on
a
ppr
oa
c
h,”
I
E
T
E
J
our
nal
of
R
e
s
e
ar
c
h
, vol
. 69, no. 12, pp. 8909
–
8920, D
e
c
. 202
3, doi
:
10.1080/
03772063.2023.2173668.
[
5]
D
.
D
r
e
s
vya
ns
ki
y,
E
.
R
yum
i
na
,
H
.
K
a
ya
,
M
.
M
a
r
ki
t
a
nt
ov,
A
.
K
a
r
pov,
a
nd
W
. M
i
nke
r
,
“
E
nd
-
to
-
e
nd m
ode
l
i
ng
a
nd
t
r
a
ns
f
e
r
l
e
a
r
ni
ng
f
or
a
udi
ovi
s
ua
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
i
n
-
t
he
-
w
i
l
d,”
M
ul
t
i
m
odal
T
e
c
hnol
ogi
e
s
and
I
nt
e
r
ac
t
i
on
,
vol
.
6,
no.
2,
J
a
n.
2022,
doi
:
10.3390/
m
t
i
6020011.
[
6]
S
.
L
i
u,
P
.
G
a
o,
Y
.
L
i
,
W
.
F
u,
a
nd
W
.
D
i
ng,
“
M
ul
t
i
-
m
oda
l
f
us
i
on
ne
t
w
or
k
w
i
t
h
c
om
pl
e
m
e
nt
a
r
i
t
y
a
nd
i
m
por
t
a
nc
e
f
or
e
m
ot
i
on
r
e
c
ogni
t
i
on,”
I
nf
or
m
at
i
on Sc
i
e
nc
e
s
, vol
. 619, pp. 679
–
694, J
a
n. 2023, doi
:
10.1016/
j
.i
ns
.2022.11.076.
[
7]
S
.
L
e
e
,
D
.
K
.
H
a
n,
a
nd
H
.
K
o,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
f
us
i
on
a
n
a
l
ys
i
s
a
da
pt
i
ng
B
E
R
T
w
i
t
h
he
t
e
r
oge
ne
ou
s
f
e
a
t
ur
e
uni
f
i
c
a
t
i
on,”
I
E
E
E
A
c
c
e
s
s
, vol
. 9, pp. 94557
–
94572, 2021, doi
:
10.1109/
A
C
C
E
S
S
.2021.3092735.
[
8]
J
.
Z
he
ng,
S
.
Z
ha
ng,
Z
.
W
a
ng,
X
.
W
a
ng,
a
nd
Z
.
Z
e
ng,
“
M
ul
t
i
-
c
ha
nne
l
w
e
i
ght
-
s
ha
r
i
ng
a
ut
oe
nc
ode
r
ba
s
e
d
on
c
a
s
c
a
de
m
ul
t
i
-
he
a
d
a
t
t
e
nt
i
on
f
or
m
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on,”
I
E
E
E
T
r
ans
ac
t
i
ons
on
M
ul
t
i
m
e
di
a
,
vol
.
25,
pp.
2213
–
2225,
2023,
doi
:
10.1109/
T
M
M
.2022.3144885.
[
9]
M
.
L
i
e
t
al
.
,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on m
ode
l
ba
s
e
d
on
a
de
e
p
ne
ur
a
l
n
e
t
w
or
k
w
i
t
h
m
ul
t
i
obj
e
c
t
i
ve
opt
i
m
i
z
a
t
i
on,”
W
i
r
e
l
e
s
s
C
om
m
uni
c
at
i
ons
and M
obi
l
e
C
om
put
i
ng
, pp. 1
–
10, A
ug. 2021, doi
:
10.1155/
20
21/
6971100.
[
10]
D
.
W
e
n,
S
.
Z
he
ng,
J
.
C
he
n,
Z
.
Z
he
ng,
C
.
D
i
ng,
a
nd
L
.
Z
ha
ng,
“
H
ype
r
pa
r
a
m
e
t
e
r
-
opt
i
m
i
z
a
t
i
on
-
i
ns
pi
r
e
d
l
ong
s
hor
t
-
t
e
r
m
m
e
m
o
r
y
ne
t
w
or
k f
or
a
i
r
qua
l
i
t
y g
r
a
de
pr
e
di
c
t
i
on,”
I
nf
or
m
at
i
on
, vol
. 14, no. 4, A
pr
. 2023
, doi
:
10.3390/
i
nf
o14040243.
[
11]
Y
. R
. P
a
nd
e
ya
a
nd
J
. L
e
e
, “
D
e
e
p l
e
a
r
ni
ng
-
ba
s
e
d l
a
t
e
f
us
i
on of
m
ul
t
i
m
oda
l
i
nf
o
r
m
a
t
i
on f
or
e
m
ot
i
on c
l
a
s
s
i
f
i
c
a
t
i
on of
m
u
s
i
c
vi
de
o,
”
M
ul
t
i
m
e
di
a T
ool
s
and A
ppl
i
c
at
i
ons
, vol
. 80, no. 2, pp. 2887
–
2905, J
a
n. 2021, d
oi
:
10.1007/
s
11042
-
020
-
08836
-
3.
[
12]
P
.
S
i
ngh,
R
.
S
r
i
va
s
t
a
va
,
K
.
P
.
S
.
R
a
na
,
a
nd
V
.
K
um
a
r
,
“
A
m
ul
t
i
m
oda
l
hi
e
r
a
r
c
hi
c
a
l
a
ppr
oa
c
h
t
o
s
pe
e
c
h
e
m
ot
i
on
r
e
c
ogni
t
i
on
f
r
om
a
udi
o a
nd t
e
xt
,”
K
now
l
e
dge
-
B
as
e
d S
y
s
t
e
m
s
, vol
. 229, O
c
t
. 2021, doi
:
10.1016/
j
.knos
ys
.2021.107316.
[
13]
C
.
L
.
-
J
i
m
é
ne
z
,
R
.
K
l
e
i
nl
e
i
n,
D
.
G
r
i
ol
,
Z
.
C
a
l
l
e
j
a
s
,
J
.
M
.
M
ont
e
r
o,
a
nd
F
.
F
.
-
M
a
r
t
í
ne
z
,
“
A
pr
opos
a
l
f
or
m
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
us
i
ng
a
ur
a
l
t
r
a
ns
f
or
m
e
r
s
a
nd
a
c
t
i
on
uni
t
s
on
R
A
V
D
E
S
S
da
t
a
s
e
t
,”
A
ppl
i
e
d
Sc
i
e
nc
e
s
,
vol
.
12,
no.
1,
D
e
c
.
2021,
doi
:
10.3390/
a
pp12010327.
[
14]
A
. I
. M
i
ddya
, B
. N
a
g, a
nd S
. R
oy,
“
D
e
e
p l
e
a
r
ni
ng ba
s
e
d m
ul
t
i
m
oda
l
e
m
ot
i
on r
e
c
ogni
t
i
on us
i
ng m
ode
l
-
l
e
ve
l
f
us
i
on of
a
udi
o
-
vi
s
ua
l
m
oda
l
i
t
i
e
s
,”
K
now
l
e
dg
e
-
B
as
e
d Sy
s
t
e
m
s
, vol
. 244, M
a
y 2022, doi
:
10.1016/
j
.kn
os
ys
.2022.108580.
[
15]
H
.
I
br
a
hi
m
,
C
.
K
.
L
oo,
a
nd
F
.
A
l
na
j
j
a
r
,
“
B
i
di
r
e
c
t
i
ona
l
pa
r
a
l
l
e
l
e
c
ho
s
t
a
t
e
ne
t
w
or
k
f
or
s
pe
e
c
h
e
m
ot
i
on
r
e
c
ogni
t
i
on,”
N
e
u
r
al
C
om
put
i
ng and A
ppl
i
c
at
i
ons
, vol
. 34, no. 20, pp. 17581
–
17599, O
c
t
. 2022, doi
:
10.1007/
s
00521
-
022
-
07410
-
2.
[
16]
P
.
V
.
V
.
S
.
S
r
i
ni
va
s
a
nd
P
.
M
i
s
hr
a
,
“
H
um
a
n
e
m
ot
i
on
r
e
c
ogni
t
i
on
by
i
nt
e
g
r
a
t
i
n
g
f
a
c
i
a
l
a
nd
s
pe
e
c
h
f
e
a
t
ur
e
s
:
a
n
i
m
pl
e
m
e
nt
a
t
i
on
o
f
m
ul
t
i
m
oda
l
f
r
a
m
e
w
or
k
us
i
ng
C
N
N
,”
I
nt
e
r
nat
i
onal
J
our
nal
of
A
dv
anc
e
d
C
om
put
e
r
Sc
i
e
nc
e
and
A
ppl
i
c
at
i
ons
,
vol
.
13,
no.
1,
2022,
doi
:
10.14569/
I
J
A
C
S
A
.2022.0130172.
[
17]
F
.
M
.
A
l
a
m
gi
r
a
nd
M
.
S
.
A
l
a
m
,
“
H
ybr
i
d
m
ul
t
i
-
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
f
r
a
m
e
w
or
k
ba
s
e
d
on
I
nc
e
pt
i
onV
3D
e
ns
e
N
e
t
,”
M
ul
t
i
m
e
di
a
T
ool
s
and A
ppl
i
c
at
i
ons
, vol
. 82, no. 26, pp. 40375
–
40402, N
ov. 2023, doi
:
10.1007/
s
11042
-
023
-
15066
-
w.
[
18]
M
.
M
c
T
e
a
r
,
Z
.
C
a
l
l
e
j
a
s
,
a
nd
D
.
G
r
i
ol
,
“
T
he
c
onve
r
s
a
t
i
ona
l
i
nt
e
r
f
a
c
e
:
t
a
l
ki
ng
t
o
s
m
a
r
t
de
vi
c
e
s
,”
T
he
C
onv
e
r
s
at
i
onal
I
nt
e
r
f
ac
e
:
T
al
k
i
ng t
o Sm
ar
t
D
e
v
i
c
e
s
, pp. 1
–
422, 2016, doi
:
10.1007/
978
-
3
-
319
-
32967
-
3.
[
19]
B
.
W
.
S
c
hul
l
e
r
a
nd
A
.
M
.
B
a
t
l
i
ne
r
,
C
om
put
at
i
onal
par
al
i
ngui
s
t
i
c
s
:
e
m
ot
i
o
n,
af
f
e
c
t
and
pe
r
s
onal
i
t
y
i
n
s
pe
e
c
h
and
l
anguage
pr
oc
e
s
s
i
ng
.
H
oboke
n, U
ni
t
e
d S
t
a
t
e
s
:
J
ohn W
i
l
e
y a
nd S
ons
, 2013. doi
:
10.1002/
9781118706664.
[
20]
T
.
A
nva
r
j
on,
M
us
t
a
qe
e
m
,
a
nd
S
.
K
w
on,
“
D
e
e
p
-
N
e
t
:
a
l
i
ght
w
e
i
ght
C
N
N
-
ba
s
e
d
s
pe
e
c
h
e
m
ot
i
on
r
e
c
ogni
t
i
on
s
ys
t
e
m
us
i
ng
d
e
e
p
f
r
e
que
nc
y f
e
a
t
ur
e
s
,”
Se
ns
o
r
s
, vol
. 20, no. 18, S
e
p. 2020, doi
:
10.3390/
s
2018521
2.
[
21]
V
.
F
r
a
nz
oni
,
G
.
B
i
ondi
,
D
.
P
e
r
r
i
,
a
nd
O
.
G
e
r
va
s
i
,
“
E
nha
nc
i
ng
m
out
h
-
ba
s
e
d
e
m
ot
i
on
r
e
c
ogni
t
i
on
us
i
ng
t
r
a
ns
f
e
r
l
e
a
r
ni
ng,”
Se
ns
or
s
,
vol
. 20, no. 18, S
e
p. 2020, doi
:
10.3390/
s
20185222.
[
22]
A
. V
a
s
w
a
ni
e
t
al
.
, “
A
t
t
e
nt
i
on i
s
a
l
l
you ne
e
d,”
A
dv
anc
e
s
i
n ne
ur
al
i
nf
or
m
at
i
on
pr
oc
e
s
s
i
ng s
y
s
t
e
m
s
, vol
. 30, 2017.
[
23]
J
.
D
e
vl
i
n,
M
.
-
W
.
C
ha
ng,
K
.
L
e
e
,
a
nd
K
.
T
out
a
nova
,
“
B
E
R
T
:
p
r
e
-
t
r
a
i
ni
ng
of
de
e
p
bi
di
r
e
c
t
i
ona
l
t
r
a
ns
f
or
m
e
r
s
f
o
r
l
a
ngua
ge
unde
r
s
t
a
ndi
ng,”
i
n
P
r
oc
e
e
di
ngs
of
t
he
2019
C
onf
e
r
e
nc
e
of
t
he
N
or
t
h
,
S
t
r
ouds
bur
g,
U
S
A
:
A
s
s
oc
i
a
t
i
on
f
or
C
om
put
a
t
i
ona
l
L
i
ngui
s
t
i
c
s
,
2019, pp. 4171
–
4186. doi
:
10.18653/
v1/
N
19
-
1423.
[
24]
V
.
S
a
nt
uc
c
i
,
S
.
S
pi
na
,
A
.
M
i
l
a
ni
,
G
.
B
i
ondi
,
a
nd
G
.
D
.
B
a
r
i
,
“
D
e
t
e
c
t
i
ng
ha
t
e
s
p
e
e
c
h
f
or
i
t
a
l
i
a
n
l
a
ngua
ge
i
n
s
oc
i
a
l
m
e
di
a
,”
i
n
E
V
A
L
I
T
A
E
v
al
uat
i
on
of
N
L
P
and
Spe
e
c
h
T
ool
s
f
or
I
t
al
i
an
,
T
or
i
no:
A
c
c
a
de
m
i
a
U
ni
ve
r
s
i
t
y
P
r
e
s
s
,
2018,
pp.
239
–
243.
doi
:
10.4000/
books
.a
a
c
c
a
d
e
m
i
a
.4799.
[
25]
V
.
F
r
a
nz
oni
,
A
.
M
i
l
a
ni
,
a
nd
G
.
B
i
ondi
,
“
S
E
M
O
:
a
s
e
m
a
nt
i
c
m
ode
l
f
or
e
m
ot
i
on
r
e
c
ogni
t
i
on
i
nw
e
b
obj
e
c
t
s
,”
i
n
P
r
oc
e
e
di
ng
s
of
t
h
e
I
nt
e
r
nat
i
onal
C
onf
e
r
e
nc
e
on W
e
b I
nt
e
l
l
i
ge
nc
e
, N
e
w
Y
or
k, U
S
A
:
A
C
M
, A
ug. 2
017, pp. 953
–
958. doi
:
10.1145/
3106426.3109417.
[
26]
C
.
L
.
-
J
i
m
é
ne
z
,
D
.
G
r
i
ol
,
Z
.
C
a
l
l
e
j
a
s
,
R
.
K
l
e
i
nl
e
i
n,
J
.
M
.
M
ont
e
r
o,
a
nd
F
.
F
.
-
M
a
r
t
í
ne
z
,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
o
n
R
A
V
D
E
S
S
da
t
a
s
e
t
us
i
ng t
r
a
ns
f
e
r
l
e
a
r
ni
ng,”
Se
ns
o
r
s
, vol
. 21, no. 22, N
ov. 2021
, doi
:
10.3390/
s
21227665.
[
27]
A
.
R
a
doi
, A
.
B
i
r
ha
l
a
,
N
.
-
C
.
R
i
s
t
e
a
, a
nd L
.
-
C
.
D
ut
u, “
A
n
e
nd
-
to
-
e
nd
e
m
ot
i
on
r
e
c
ogni
t
i
on
f
r
a
m
e
w
or
k
ba
s
e
d
on
t
e
m
por
a
l
a
ggr
e
ga
t
i
on
of
m
ul
t
i
m
oda
l
i
nf
o
r
m
a
t
i
on,”
I
E
E
E
A
c
c
e
s
s
, vol
. 9, pp. 135559
–
135570, 2021, d
oi
:
10.1109/
A
C
C
E
S
S
.2021.3116530.
[
28]
S
.
R
.
L
i
vi
ngs
t
one
a
nd
F
.
A
.
R
us
s
o,
“
T
he
r
ye
r
s
on
a
udi
o
-
vi
s
ua
l
da
t
a
b
a
s
e
of
e
m
ot
i
ona
l
s
pe
e
c
h
a
nd
s
ong
(
R
A
V
D
E
S
S
)
:
A
dyna
m
i
c
,
m
ul
t
i
m
oda
l
s
e
t
of
f
a
c
i
a
l
a
nd
voc
a
l
e
xpr
e
s
s
i
ons
i
n
N
or
t
h
A
m
e
r
i
c
a
n
E
ngl
i
s
h,”
P
L
O
S
O
N
E
,
vol
.
13,
no.
5,
M
a
y
2018,
doi
:
10.1371/
j
our
na
l
.pone
.0196391.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
2252
-
8938
I
nt
J
A
r
ti
f
I
nt
e
ll
, V
ol
. 13, No. 4, D
e
c
e
m
be
r
2024:
4435
-
444
4
4444
[
29]
A
.
C
ha
udha
r
i
,
C
.
B
ha
t
t
,
A
.
K
r
i
s
hna
,
a
nd
C
.
M
.
T
.
-
G
onz
á
l
e
z
,
“
F
a
c
i
a
l
e
m
ot
i
on
r
e
c
ogni
t
i
on
w
i
t
h
i
nt
e
r
-
m
oda
l
i
t
y
-
a
t
t
e
nt
i
on
-
t
r
a
ns
f
or
m
e
r
-
ba
s
e
d s
e
l
f
-
s
upe
r
vi
s
e
d l
e
a
r
ni
ng,”
E
l
e
c
t
r
oni
c
s
, vol
. 12, no. 2, J
a
n. 2023, doi
:
10.3
390/
e
l
e
c
t
r
oni
c
s
12020288.
[
30]
O
.
A
t
i
l
a
a
nd
A
.
Ş
e
ngür
,
“
A
t
t
e
nt
i
on
gui
de
d
3D
C
N
N
-
L
S
T
M
m
ode
l
f
or
a
c
c
u
r
a
t
e
s
pe
e
c
h
ba
s
e
d
e
m
ot
i
on
r
e
c
ogni
t
i
on,”
A
ppl
i
e
d
A
c
ous
t
i
c
s
, vol
. 182, N
ov. 2021, doi
:
10.1016/
j
.a
pa
c
ous
t
.2021.108260.
[
31]
M
.
K
ha
n,
A
.
E
l
S
a
ddi
k,
F
.
S
.
A
l
ot
a
i
bi
,
a
nd
N
.
T
.
P
ha
m
,
“
A
A
D
-
N
e
t
:
A
dva
nc
e
d
e
nd
-
to
-
e
nd
s
i
gna
l
pr
oc
e
s
s
i
ng
s
y
s
t
e
m
f
or
hum
a
n
e
m
ot
i
on
de
t
e
c
t
i
on
&
a
m
p;
r
e
c
ogni
t
i
on
u
s
i
ng
a
t
t
e
nt
i
on
-
ba
s
e
d
de
e
p
e
c
ho
s
t
a
t
e
n
e
t
w
or
k,”
K
now
l
e
dge
-
B
as
e
d
Sy
s
t
e
m
s
,
vol
.
270,
J
un.
2023, doi
:
10.1016/
j
.knos
ys
.2023.110525.
[
32]
D
.
M
a
m
i
e
va
,
A
.
B
.
A
bdus
a
l
om
ov,
A
.
K
ut
l
i
m
ur
a
t
ov,
B
.
M
um
i
nov,
a
nd
T
.
K
.
W
ha
ngbo,
“
M
ul
t
i
m
oda
l
e
m
ot
i
on
de
t
e
c
t
i
on
vi
a
a
t
t
e
nt
i
on
-
ba
s
e
d f
us
i
on of
e
xt
r
a
c
t
e
d f
a
c
i
a
l
a
nd s
p
e
e
c
h f
e
a
t
ur
e
s
,
”
Se
ns
o
r
s
, vol
. 23, no. 12, J
un. 2023, doi
:
10.3390/
s
23125475.
[
33]
S
. S
i
ngkul
a
nd K
. W
or
a
r
a
t
pa
nya
, “
V
e
c
t
or
l
e
a
r
ni
ng r
e
pr
e
s
e
nt
a
t
i
on f
or
ge
ne
r
a
l
i
z
e
d s
pe
e
c
h e
m
ot
i
on r
e
c
ogni
t
i
on,”
H
e
l
i
y
on
, vol
. 8, no.
3, M
a
r
. 2022, doi
:
10.1016/
j
.he
l
i
yon.2022.e
09196.
[
34]
B
.
A
bdol
l
a
hz
a
de
h,
F
.
S
.
G
ha
r
e
hc
hopogh,
N
.
K
hoda
da
di
,
a
nd
S
.
M
i
r
j
a
l
i
l
i
,
“
M
ount
a
i
n
ga
z
e
l
l
e
opt
i
m
i
z
e
r
:
a
ne
w
na
t
ur
e
-
i
ns
pi
r
e
d
m
e
t
a
he
ur
i
s
t
i
c
a
l
gor
i
t
hm
f
or
gl
oba
l
opt
i
m
i
z
a
t
i
on
pr
obl
e
m
s
,”
A
dv
anc
e
s
i
n
E
ngi
ne
e
r
i
ng
Sof
t
w
ar
e
,
vol
.
174,
D
e
c
.
2022,
doi
:
10.1016/
j
.a
dve
ngs
of
t
.2022.103282.
B
I
O
G
R
A
P
H
I
E
S
O
F
A
U
T
H
O
R
S
Basamma
Umesh
Patil
working
as
an
Assistant
Professor
in
the
Department
of
Information
Science
and
Engineering,
JSS
Academy
of
Technical
Ed
ucation,
Bengaluru.
She
received her M.Tech
.
from VTU in
the year
2014 and
her B.E
.
from V
TU in the year
2009. Sh
e
is
pursuing
her
Ph.D.
in
Computer
Science
and
Engineering
in
the
Department
of
Computer
Scienc
e
and
Engine
ering,
JSSATE,
Bengal
uru.
She
has
12
years
of
academic
teaching
experience
and
years
of
research
experience.
Her
research
field
of
interest
s
includes
data
science,
artificial
intelligence,
and
machine
learning.
She
has
publis
hed
around
16
researc
h
papers
in
internationa
l
journals
and
conferences
like
IEEE,
Springer
,
and
Elsevier.
She
can
be
contacted
at email
: bupat
il25@
gmail.co
m or pat
ilbu@
jssateb.
ac.in.
Ashoka
Davanageri
Virupakshappa
working
as
a
Professor
(ISE)
at
JSS
Academy
of
Technical
Education
Bengaluru.
He
served
as
Professor
a
nd
Head,
Dean
at
various
reputed
engineering
colleges
of
Karnataka.
He
received
his
Ph.D.
in
Computer
Science
and
Engineering
from
Dr.
MGR
University
in
2009
-
10.
He
is
a
member
of
IEEE,
MCSI,
MISTE,
Fellow
Institute
of
Engineers.
Six
students
have
been
awarded
Ph.
D
.
degrees
.
in
Computer
Scienc
e
and
Engine
ering
under
his
guidanc
e.
He
was
one
of
the
Nat
ional
Awar
d
winner
s
in
“
Rashtriya
Ekta
Samman
-
2013”.
He
is
an
editorial
board
member
of
reputed
internatio
nal
journals.
His
biography
is
included
in
Who’s
Who
in
the
World
20
11
-
12.
He
has
over
100
publications
in
different
internationa
l
and
national
journals
and
confere
nces
.
His
areas
of
e
xpertise
include
software
engineering,
software
architecture,
requir
ement
engineering,
data
science,
knowledge
engineering
,
and
operating
system
virtualization.
He
can
be
contacted
at
email:
dr.dvasho
ka@
gmail.co
m
.
Ajay
Prakash
Basappa
Vijaya
currently
working
as
a
Research
Professo
r
in
the
Department
of
Artificial
Intelligence
and
Machine
Learning
at
Ambedkar
Institute
of
Technology,
Bengaluru.
He
worked
as
an
Associate
Professor
in
the
Department
of
Computer
Scienc
e
and
Engine
ering,
SJBIT
Bengal
uru.
He
rece
ived
his
Ph.D.
in
Computer
Scienc
e
and
Engineering from VTU in 2018. He received his M.Tech
.
from VTU i
n the year 200
8 and B.E
.
from
VTU
in
the
year
2005.
He
has
15
years
of
academic
teaching
experience
and
8
years
of
research
experience
.
He
is
a
Wipro
Certified
Faculty
(WCF)
for
project
based
learning
f
ramework
in
Java
-
J2EE
and
imparting
the
learning
to
students.
His
research
field
of
interest
includes know
ledge eng
ineering,
software
archite
cture, da
ta scienc
e, and ma
chine lea
rning. He
has
published
around
22
research
papers
in
international
journals
an
d
conference
like
IEEE,
Springe
r,
Elsevie
r
,
and
Inderscience.
Recently
received
Rs.
5,00,000/
-
L
ahks
grants
from
VGST,
the gover
nment of Ka
rnataka
for
the data
science
project
titled “intellig
ent and inte
ractive
plant
disease
supervision
using
deep
learning”.
He
can
be
contacted
at
email:
ajayprakas@gmail.com.
Evaluation Warning : The document was created with Spire.PDF for Python.