Int
ern
at
i
onal
Journ
al of Ele
ctrical
an
d
Co
mput
er
En
gin
eeri
ng
(IJ
E
C
E)
Vo
l.
10
,
No.
3
,
June
2020
,
pp. 3
307
~
3314
IS
S
N: 20
88
-
8708
,
DOI: 10
.11
591/
ijece
.
v10
i
3
.
pp3307
-
33
14
3307
Journ
al h
om
e
page
:
http:
//
ij
ece.i
aesc
or
e.c
om/i
nd
ex
.ph
p/IJ
ECE
Analysis
on te
chniq
ues used to
recogniz
e and id
entifying
the
Hum
an emoti
ons
Pra
veen Kul
k
arni
1
,
R
aj
es
h
T M
2
1
Resea
rch
Schol
ar
a
t
Da
y
a
nand
a S
aga
r
Univer
si
t
y,
Indi
a
1
Depa
rtment
of C
om
pute
r
Scie
n
ce
and
Engi
ne
ering,
Facu
lty
of E
ngine
er
ing, CHRIST (
Dee
m
ed t
o
be
un
ive
rsi
t
y
)
,
India
2
Depa
rtment of
Com
pute
r
Scie
n
ce
and
Engi
ne
ering,
Da
y
a
nanda Saga
r
Univ
ersity
,
India
Art
ic
le
In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
J
ul
26
, 2
019
Re
vised
Dec
17
,
20
19
Accepte
d
J
an
8
, 2020
Faci
a
l
expr
ession
is
a
m
aj
or
area
for
non
-
ver
ba
l
la
ngu
age
in
d
a
y
to
d
a
y
li
f
e
comm
unic
at
ion.
As
the
stat
isti
c
al
a
na
l
y
s
is
show
s
only
7
per
ce
nt
o
f
the
m
essage
in
comm
unic
at
ion
was
cove
red
in
ver
bal
comm
unic
ation
whil
e
55
per
ce
n
t
tr
ansm
it
te
d
b
y
fa
cial
expr
ession.
Em
oti
onal
expr
essi
on
has
bee
n
a
rese
ar
ch
subject
of
ph
y
siol
og
y
sinc
e
Dar
win’s
work
on
emotiona
l
expr
ession
in
the
19th
c
ent
u
r
y
.
Acc
ording
to
Ps
y
chol
og
ic
a
l
theo
r
y
the
class
ifi
c
at
io
n
of
hum
an
em
oti
on
is
class
ifi
e
d
m
aj
orl
y
int
o
s
ix
emotions:
happi
ness,
f
ea
r,
ange
r,
surprise
,
disgust,
and
sadne
ss
.
Facial
expr
essions
which
invol
v
e
t
he
emotions
and
the
na
ture
of
sp
ee
ch
p
lay
a
fore
m
ost
role
in
expr
essing
th
ese
emotions.
The
r
ea
ft
er,
rese
arc
h
e
rs
deve
lop
ed
a
s
y
stem
b
ase
d
on
Anatomic
of
fac
e
n
amed
Facial
Act
ion
Codin
g
S
y
stem
(FA
CS
)
in
1970.
Eve
r
sin
ce
the
d
eve
lopment
of
F
ACS
the
re
is
a
r
api
d
progr
ess
in
the
dom
ai
n
of
emotion
re
cogni
ti
on
.
Thi
s
work
is
int
ende
d
to
give
a
thorough
compara
ti
v
e
an
aly
s
is
of
the
v
ari
ous
t
ec
hniqu
es
and
m
et
hod
s
tha
t
wer
e
appl
i
ed
to
re
co
gniz
e
and
ide
n
t
if
y
hum
an
emotions.
T
h
is
ana
l
y
sis
r
esult
s
will
h
el
p
to
i
dent
if
y
prope
r
and
sui
ta
bl
e
te
chn
ique
s,
al
g
orit
hm
s
an
d
the
m
et
hodologies
for
future
re
sea
rch
dir
ec
t
ion
s.
In
thi
s
pape
r
ext
ensiv
e
ana
l
y
sis
on
var
i
ous
rec
ognit
ion
t
ec
hniqu
es
used
to
ide
nti
f
y
the
co
m
ple
xity
i
n
rec
ogni
zi
ng
the f
ac
i
al
expr
ession
is pr
ese
nt
ed.
Ke
yw
or
d
s
:
Cl
assifi
cat
ion
Face detect
io
n
Feat
ur
e
ex
tr
act
ion
Copyright
©
202
0
Instit
ut
e
o
f Ad
vanc
ed
Engi
n
ee
r
ing
and
S
cienc
e
.
Al
l
rights re
serv
ed
.
Corres
pond
in
g
Aut
h
or
:
Pr
a
veen K
ulk
a
rn
i
,
Dep
a
rtm
ent o
f C
om
pu
te
r
Scie
nce a
nd
E
ng
i
ne
erin
g,
Faculty
of E
ng
i
neer
i
ng,
CHRIST
(Dee
m
ed
to
be Un
i
ver
sit
y),
Ba
ng
al
or
e,
In
di
a
.
Em
a
il
:
pr
avee
nc
s024@gm
ai
l.
com
1.
INTROD
U
CTION
Em
otion
s
pla
y
a
ve
ry
im
portant
f
un
ct
i
on
in
m
any
fiel
ds
li
kes
f
or
e
ns
ic
c
rim
e
detect
ion,
ps
yc
holo
gi
cal
ly
aff
ect
ed
pat
ie
nts,
stu
de
nts
m
entor
in
g
i
n
academ
ic
s
and
victi
m
ob
se
rv
at
io
n
in
co
urt
,
et
c.
Howe
ver,
s
o
m
uch
of
t
he
grow
t
h
happe
ned
in
this
te
c
hnol
og
ic
al
do
m
ai
n,
sti
ll
,
we
ca
n
fi
nd
lots
of
draw
backs
and
lo
ophole
s
in
t
erm
s
of
accuracy
in
va
riou
s
resu
lt
s
we
fo
un
d
in
our
su
r
vey
w
ork.
Ther
e
a
re
a
num
ber
of
j
obs
pe
rfor
m
ed
by
in
div
id
ua
ls
and
gr
oups
.
The
goal
of
fa
ci
al
expressio
n
is
to
ide
ntify
on
e
sel
f
by
obs
erv
i
ng
a
sin
gle
im
age
or
m
ulti
ple
im
ages,
wh
ic
h
is
the
em
otion
that
the
im
age
sh
ows
.
T
he
hu
m
an
face
has
s
ever
al
com
po
ne
nts
s
uc
h
as
ey
e,
nose
,
m
ou
th,
Br
ow
an
d
a
fe
w
ot
he
rs.
Ba
s
ed
on
t
he
m
ov
em
ent
of
th
os
e
c
om
po
ne
nts
and
c
ha
ng
e
of
sh
ap
e
an
d
siz
es
em
otion
s
m
ay
be
extracte
d
in
var
i
ou
s
w
ay
s.
Re
orga
nizat
ion
of
em
oti
on
i
n
hu
m
an
h
as
bec
om
e
a
gr
eat
est
chall
eng
e
face
d
in
the
i
nterac
ti
on
of
c
om
pu
te
r
an
d
hum
ans.
Most
of
the
e
ffor
t
on
em
otion
rec
ogniti
on
f
ocus
es
on
in
form
at
i
on
e
xtracte
d
f
r
om
visu
al
or
a
ud
i
o
se
par
at
el
y.
N
um
ero
us
s
urveys
hav
e
bee
n
publ
ished
in
dif
fer
e
nt
areas
,
a
naly
zi
ng
a
nd
recog
nizing
the
gest
ur
es
an
d
m
ov
e
m
ents
of
hum
an
face
and
trac
king
of
ey
e
is
still
an
un
s
olv
e
d
pro
bl
e
m
with
resp
e
ct
to
accuracy
rate.
M
any
res
earche
rs
in
thi
s
area
are
bee
n
set
as
a
ben
c
hm
ark
and
b
asem
ent
for
m
any
of
the
current
res
earc
h
top
ic
s.
Re
sear
ch
er
in h
is
w
ork
[
1]
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
o
m
p
En
g,
V
ol.
10
, No
.
3
,
J
une
2020
:
33
0
7
-
33
1
4
3308
has
ded
ic
at
e
d
his
caree
r
to
the
purs
uit
of
e
m
otion
s,
f
oc
us
in
g
m
ai
nly
on
six
dif
fer
e
nt
em
otion
s
s
urpr
ise
,
sad
ness,
fear
,
ang
e
r,
disgust,
happine
ss.
H
ow
e
ve
r,
in
t
he
ye
ar
1990s,
he
dilat
ed
his
li
st
of
basic
em
otions,
tog
et
he
r
with
a
var
ie
ty
of
posi
ti
ve
and
neg
at
i
ve
em
otion
s
in
cl
ud
in
g
am
us
em
ent,
Disli
ke,
awkwa
rdness
,
reli
ef,
pleasu
re, sat
isf
act
ion
, g
uilt
and
pr
i
de
Unde
rstan
ding
facial
ex
pr
e
ss
ion
ca
n
be
see
n
as
a
com
m
on
ph
e
nom
eno
n
but
has
bec
om
e
the
m
os
t
essenti
al
ind
ic
at
ion
in
the
his
tory
of
em
otion
al
ps
yc
holo
gy.
Mo
re
rece
ntl
y,
resea
rch
on
facial
ex
pr
essi
on
s
ha
s
le
d
to
the
e
m
e
rg
e
nce
of
new
con
ce
pts,
ne
w
te
chn
i
qu
es
,
and
in
novative
resu
lt
s.
In
thi
s
case,
sci
entist
s
are
form
ulati
ng
al
t
ern
at
ive
acc
ounts,
w
hile
pr
e
vious
ve
rsions
are
acqu
iri
ng
new
inte
rests.
Darwin
’s
ar
gum
ents
su
ggest
that
e
m
ot
ion
s
ha
ve
evo
l
ved
t
o
the
po
int
of
act
ing
as
a
n
instr
um
ent
of
com
m
un
ic
at
ion
be
tween
ind
ivi
du
al
s
.
In
the
ye
ar
2003
auth
or
Ga
o
et
L
us
ed
dL
HD
a
nd
ID3
decisi
on
tree
cl
assifi
cat
ion
m
et
ho
d
(Noh
et
al
in
2007).
Zha
o
and
Piet
ikaine
n
in
2009
us
e
d
SV
M
an
d
S
ong
et
al
in
20
10
us
e
d
S
V
M
fo
r
the
cl
assifi
cat
ion
m
et
ho
d.
Th
e
researc
her
in
his
wor
k
sp
ec
ifie
d
that
SV
M
cl
assifi
er
giv
e
s
hig
est
accu
ra
cy
for
sever
al
facial
exp
re
ssio
ns
[
2
].
The
unsupe
rv
ise
d
le
arn
i
ng
al
gori
thm
cal
le
d
L
VQ
-
Lea
rn
i
ng
Vecto
r
Qu
a
ntiza
ti
on
(
B
ashyal
in
2008)
a
nd
MLP
is
al
so
us
ed
f
or
cl
assifi
cat
ion
[3
]
.
K
NN
al
gorithm
(P
our
sa
ber
i
i
n
2012)
is
a
m
eth
od
of
cl
assi
ficat
ion
th
rou
gh
wh
ic
h
relat
io
ns
hi
p
am
on
g
t
he
assessm
ent
m
od
el
s
are
est
i
m
at
ed
durin
g
trai
ni
ng.
HMM
cl
ass
ifie
r
be
the
one
of
the
sta
ti
sti
cal
rep
rese
ntat
ion
f
or
cat
e
gorize
expressi
on
into
var
i
ou
s
ty
pes
in
face
detect
ion
(Tayl
or
in
2014)
.
Cl
assifi
cat
ion
a
nd
r
egr
es
sio
n
tree
is
m
achine
le
arn
i
ng
al
gorithm
(
S
alma
n
in
2016)
us
ed
for
cl
as
sific
at
ion
us
i
ng
distance
vec
tors.
C
NN
(20
16).
MFF
N
N,
DNN
(20
15),
MDC
(
Islam
in
2018)
are
so
m
e
of
th
e
cl
assifi
ers
use
d
rece
ntly
. W
it
h
resp
ect
to m
any
cl
assifi
ers
SV
M
giv
es
im
pr
ov
e
d
reorga
nizat
ion
acc
ur
at
ene
s
s
an
d
al
s
o
bette
r
cl
assifi
cat
io
n.
C
om
par
ed
to
th
e
past
al
gorithm
us
e
d
f
or
cl
as
sif
ic
at
ion
prese
ntly
the
SV
M
cl
assifi
er
is
gr
eat
ly
us
e
cl
assifi
e
r.
Othe
r
cl
assif
ie
rs
li
ke
CAR
T,
pai
r
wise,
Ne
ur
al
N
et
work b
ase
d
a
re
us
ed
in
m
odern y
ears c
om
par
ed
to past
de
cades
Em
otion
rec
ogniti
on
syst
em
s
reali
ze
a
pp
li
cat
ion
s
i
n
se
ve
ral
fasci
natin
g
a
reas.
W
it
h
the
recent
adv
a
nces
in
ar
ti
fici
al
intel
li
g
ence,
sig
nifica
ntly
autom
a
ton
rob
ots,
the
r
equ
i
rem
ent
fo
r
a
stron
g
e
xpr
ession
recog
niti
on
sys
tem
is ob
vi
ou
s
.
Th
e recog
niti
on of em
otion
s p
la
ys a ver
y i
m
po
rtant r
ole
within the
rec
ogniti
on
of
on
e
’s
own
f
ondnes
s
a
nd
i
n
turn,
hel
ps
to
form
a
sensiti
ve
and
se
ns
it
ive
hu
m
an
-
m
achine
inter
face
(
HCI).
Additi
on
al
ly
,
t
he
2
m
ai
n
ap
plica
ti
on
s,
pa
r
ti
cularly
Ar
ti
fi
ci
al
In
te
ll
igen
ce
an
d
se
ns
it
ive
hu
m
an
m
a
chine
interface
,
Em
otion
rec
ogniti
on
syst
em
s
area
un
it
util
iz
ed
i
n
an
ou
tsi
zed
var
ie
ty
of
al
te
rn
at
ive
do
m
ai
ns
li
ke
te
le
com
m
un
ic
a
ti
on
s
fiel
d,
vi
de
o
gam
ing
,
vid
eo
a
nim
ation
,
ps
yc
hiatry
,
in
autom
otive
secur
it
y,
ed
ucat
ion
al
com
pu
te
r
syst
e
m
and
m
any
oth
e
rs.
Ma
ny
al
gorithm
s
hav
e
been
s
uggest
ed
to
de
velo
p
syst
e
m
s/a
pp
li
cat
ion
s
that
ca
n
detect
e
m
otion
s
ver
y
well
.
Com
pu
te
r
ap
plica
ti
on
s
cou
l
d
com
m
un
ic
at
e
bette
r
by
changin
g
res
pons
e
s
base
d on the e
m
ot
ion
al
stat
e
of hum
an
us
e
rs
in vari
ou
s
inte
racti
on
s
Em
otion
detect
ion
us
i
ng
the
facial
com
po
ne
nts
sug
gested
befor
e
is
sti
ll
bein
g
use
d.
It
was
m
ai
nly
ob
s
er
ved
that
f
or
em
otion
det
ect
ion
we
nee
d
to
do
it
by
sta
ge
by
sta
ge
.
P
r
eprocessi
ng
is
the
fir
st
sta
ge
,
and
then
c
om
es
the
featur
e
e
xtract
ion
a
nd
the
n
th
e
cl
assifi
cat
ion.
As
the
ye
ar
pa
ssed
the
re
has
been
lot
of
grow
t
h
and
a
dv
a
ncem
ent
in
these
th
r
ee
sta
ges
with
diff
e
re
nt
var
ie
ty
of
al
gorithm
s.
In
the
prese
nt
ob
se
r
vations
m
or
e
nu
m
ber
of
fea
tures
are
e
xtra
ct
ed
from
the
faces
to
ve
rify
the
e
m
otion
s.
Faci
al
e
m
otion
recogn
it
io
n
fiel
d
is
gaining
a
to
n
of
at
te
ntio
n
a
nd
in
past
tw
o
decad
e
s
with
app
li
cat
io
n
s
a
nd
m
od
e
rn
iz
at
ion
no
t
s
olely
within
the
sen
sory
ac
ti
vity
and
ps
yc
ho
l
og
ic
al
featur
e
sci
ences
,
howe
ve
r
ad
diti
onal
ly
in
em
oti
on
al
c
om
pu
ti
ng
a
nd
com
pu
te
r
ani
m
at
ion
s.
A
m
uch
m
or
e
m
od
er
n
stu
dy
sugg
e
sts
that
there
are
fa
r
m
or
e
basic
em
oti
on
s
t
han
antece
de
ntly
belie
ved.
W
it
hin
the
stu
dy
rev
eal
e
d
in
Pr
oc
eedi
ng
s
of
Nati
onal
A
cadem
y
of
Scie
nces
,
researc
hers
kn
own
27
t
otall
y
diff
e
ren
t
cl
a
sses
of
em
oti
on.
I
ns
te
ad
of
bein
g
e
ntirel
y
disti
nct,
ho
wev
e
r,
the
resea
rc
her
s
fou
nd
that
in
div
id
ual’s
ex
pe
rtise
these
e
m
ot
ion
s
on
a
gr
a
dient.
Re
co
gn
it
io
n
in
the
form
of
facial
e
m
otion
and
e
xpressi
on
has
a
ddit
iona
ll
y
been
incr
eased.
Re
ce
ntly
becau
se
of
fast
de
velo
pme
nt
of
m
achine
le
arni
ng
an
d
arti
fic
ia
l
intelli
gen
t
(AI)
te
ch
nique
s,
as
well
as
the
hu
m
an
co
m
pu
te
r
interact
ion
,
a
com
pu
t
er
ga
m
e
(V
R),
au
gm
ent
reali
ty
(
AR).
Detect
io
n
of
facial
ex
pr
essi
on
in
ad
van
ce
d
dri
ve
r
assist
ant
syst
e
m
s an
d re
creati
on is also
foun
d.
Althou
gh
th
e
te
chnolo
gy
f
or
recog
nizing
e
m
ot
ion
s
is
i
m
po
rta
nt
and
h
as
evo
l
ved
i
n
dif
f
eren
t
fiel
ds,
it
is
sti
ll
the
un
an
swe
red
pro
blem
.
Detect
ing
the
feeli
ng
of
bei
ng
hum
an
can
be
ac
hiev
ed
th
rou
gh
the
us
e
of
facial
i
m
ages,
vo
ic
e,
a
nd
bo
dy
sh
ape.
With
this
detect
ion,
the
i
m
age
of
t
he
face
is
the
m
os
t
fr
equ
e
nt
so
urce
and
to
detect
e
m
otion
s
ab
ov
e
the
e
ntire
fr
onta
l
facial
i
m
age
are
c
omm
on
ly
us
ed
to
detect
em
ot
ion
s
.
The
pr
ocedu
re
for
rec
ognizing
em
otion
s
i
s
no
t
sim
ple
bu
t
c
om
plex
because
it
extracts
the
ap
propriat
e
char
act
e
risti
cs
an
d
Detect
in
g
em
otion
s
re
qu
i
res
c
om
plex
ste
ps
.T
he
F
aci
al
Expressi
on
Re
co
gnit
io
n
(F
ER)
consi
sts
of
fiv
e
ph
a
ses
as
s
how
n
in
Fi
g
ur
e
1.
No
ise
el
i
m
inati
on
/im
pr
ov
em
ent
is
pe
rfor
m
ed
in
th
e
pr
e
-
processi
ng
phase
ta
ke
an
im
age
or
a
sequ
e
nce
of
im
a
ges
(a
tim
e
s
eries
of
im
ages
fr
om
neu
tral
to
an
expressi
on)
as
an
in
pu
t
fac
e
for
f
ur
t
her
processi
ng.
D
et
ect
ion
of
fa
ci
al
co
m
po
ne
nts
detect
s
ret
urn
on
inv
est
m
ent
in
ey
es,
nose,
ch
eeks,
m
ou
th,
e
ye
s,
f
or
e
hea
d,
ear,
fro
nt
hea
d,
et
c.
T
he
c
ha
r
act
erist
ic
extra
ct
ion
ph
a
se
c
on
ce
r
ns
the
e
xtracti
on
of
the
c
ha
racteri
sti
cs
from
the
ROI
.
Her
e
,
w
e
disc
us
s
t
he
work
n
ot
i
nclu
ded
i
n
pr
e
vious
s
urve
ys,
w
hich
has
change
d
in
t
he
la
st
two
de
cades
in
t
he
detect
ion
of
e
m
ot
ion
s.
The
top
ic
s
discusse
d
in
t
he
f
ollow
i
ng
s
ect
ion
s
are
on
the
com
par
ison
betwee
n
pr
evio
us
ly
done
work
a
nd
the
pr
ese
nt
researc
h bein
g ca
rr
ie
d o
ut.
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
An
alysis o
n
te
c
hn
i
qu
e
s
us
ed
to
rec
ognize
an
d
ide
ntif
yi
ng th
e Hum
an e
mo
ti
on
s
(
Pravee
n K
ulkar
ni
)
3309
Figure
1. P
has
es of
facial
exp
ressio
n reco
gnit
ion
2.
RESEA
R
CH MET
HO
D
Pr
e
-
t
reatm
ent
will
be
the
first
ste
p
in
detect
ing
a
face.
To
get
purely
facial
i
m
ages
with
good
In
te
ns
it
y,
pro
pe
r
siz
e,
a
nd
id
entic
al
norm
al
i
zed
s
ha
pe
w
e
hav
e
to
c
ondu
ct
pr
e
-
pr
ocessi
ng.
T
o
c
onve
rt
i
m
age
into
a
cl
ean
im
age
of
the
norm
al
iz
e
face
for
ta
king
out
the
char
act
e
risti
c
is
the
detect
ion
of
chara
ct
erist
ic
po
i
nts,
w
hic
h
ro
ta
te
to
get
in
li
ne,
ide
ntif
y
and
c
ut
the
facial
reg
io
n
by
m
eans
of
a
four
-
side
d
fig
ur
e,
accor
ding
to
th
e
cop
y
of
th
e
f
ace.
A
sin
gle
im
age
in
the
D
et
ect
or
faces
c
an
be
cat
e
goriz
ed
into
fou
r
m
et
hods
base
d on Feat
ure
-
base
d,
A
pp
e
aran
ce
-
ba
sed
, Kn
owle
dge
-
ba
sed
a
nd tem
pla
te
b
ase
d
as
sho
wn in Fi
g
ur
e
2.
Figure
2. Face
detect
ion m
et
h
od
s
2
.
1.
Fe
at
ure
b
as
ed
Feat
ur
e
base
d:
This
te
ch
niqu
e
is
us
ed
t
o
fi
nd
faces
by
e
xtr
act
ing
str
uctu
r
al
op
ti
ons
with
in
the
face
.
In
it
ia
ll
y
i
t
will
be
trai
ned
as
cl
assifi
er
an
d
so
to
dif
fer
e
ntiat
e
between
whic
h
is
facial
and
non
-
facial
re
gion
.
Con
ce
pt
is
our
sel
f
-
gen
e
rated
inf
or
m
at
io
n
of
faces
is
to
be
at
the
bonds.
This
ap
proac
h
di
vid
e
d
into
m
any
ste
ps
a
nd ev
e
n photo
s w
it
h
se
ver
al
faces t
he
y rep
or
t a
s
ucc
ess r
at
e
of 94%
.
2
.
2
.
Te
xture
-
ba
sed
Textu
re
-
Ba
se
d:
The
recogn
it
ion
of
the
em
otion
is
done
on
the
plo
t
char
act
erist
ic
extract
from
the
gr
ay
-
le
vel
coex
ist
e
nce
m
at
rix,
GLCM
c
har
act
erist
ic
s
a
re
extracte
d
an
d
Form
at
with
the
support
m
a
chine
carriers
by
div
e
rse
co
res.
Stat
ist
ic
al
a
nd
featu
res
that
are
ta
ke
n
f
ro
m
GLPM
are
gi
ven
as
the
input
f
or
cl
assifi
cat
ion
to
SV
M
.
The
detect
ion
r
at
e
i
s
identifie
d
a
s
90
%
[4
]
.
T
he
al
gorithm
detect
s
the
cha
racteri
s
ti
c
po
ints
of
a
victim
iz
ation
abstracti
on
filt
er
te
chn
i
qu
e
by
the
data
giv
e
n
by
aut
hor
[
5].
The
gr
oup
in
dicat
es
the
vi
ct
i
m
iz
at
ion
of
the
facial
cand
i
date.
I
niti
al
ly
it
sh
ould
be
trai
ned
s
uc
h
t
hat
the cla
ssifie
r
c
an
ide
ntify t
he faci
al
r
egi
on a
nd no
n
-
facial
re
gion.
T
his tec
hn
i
qu
e
is b
a
se
d on
a
n 8
5% d
et
ect
ion
rate wit
h o
ne h
undred
im
ages w
it
h diff
e
re
nt s
cal
es an
d direc
ti
on
of purp
os
e
.
2
.
3
.
Skin
col
or
ba
s
ed
Sk
in
col
or
B
ased:
A
s
e
xpr
essed
by
a
uthor
i
n
[6
]
,
al
gorith
m
s
are
c
om
par
ed
us
in
g
3
c
olo
r
s.
This
m
ixtur
e
of
c
olor
le
ads
to
face
detect
ion
al
gorit
hm
with
col
or
r
ep
la
ce
m
ent.
On
c
e
the
sk
in
regi
on
is
ob
ta
ine
d,
t
he
expressi
on
of
the
ey
es
is
el
i
m
inate
d
by
ob
scur
i
ng
t
he
ba
ckgr
ound
col
ors
an
d
the
im
a
ge
is
resh
a
pe
d
in
gra
ysc
al
e and the
bin
a
ry im
age as
an
accepta
ble lim
i
t. Th
e
rate
of
detect
ion
is
accu
rate t
o 9
5.18.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
o
m
p
En
g,
V
ol.
10
, No
.
3
,
J
une
2020
:
33
0
7
-
33
1
4
3310
2
.
4
.
M
ultipl
e
feature
Mult
iple
Feat
ure:
Ad
a
boost
has
sel
ect
ed
t
he
m
ulti
ple
featur
es
of
the
face
re
gion
al
gorithm
[7
]
.
The
face
is
di
vi
ded
i
nto
dif
fe
r
ent
re
gions
by
diff
e
re
nt
ort
ho
gonal
reg
i
ons.
Com
po
ne
nts
li
ke
nose,
ey
es
,
m
ou
th
are
ide
ntifie
d
for
the
pur
pos
e
of
an
al
ysi
s.
Ar
ea
w
her
e
th
e
com
bin
at
ion
was
us
e
d
a
nd
giv
e
n
a
s
in
pu
t
for
the
cl
assifi
er.
In
this
phase
it
will
cho
o
s
e
on
e
of
the
be
st
arr
an
gem
e
nts
and
the
n
m
od
ify
the
weigh
t
in
the
com
ing
pr
ocess.
Detect
io
n
r
ou
ti
ne
in
m
any
cha
racters
is
gr
eat
tha
n
th
e
m
ai
n
com
po
nen
t
of
t
he
co
m
m
on
or
t
hogonal
par
t
an
al
ysi
s m
et
ho
d.
2.5.
Te
mpla
te
matchin
g
me
t
ho
ds
Tem
plate
m
at
c
hing
m
et
ho
ds
:
wh
e
n
t
he
im
a
ge
is
gi
ven
as
an
in
pu
t
t
his
m
et
ho
d
will
m
at
ch
it
with
the
de
fine
d
te
m
pla
te
and
i
de
ntify
the
face.
Fo
r
ex
am
ple,
we
ca
n
div
ide
the
face
into
m
any
par
ts,
li
ke
ey
es,
no
s
e,
m
ou
th
.
Also
,
we
ca
n
desig
n
a
m
od
el
in
w
hich
can
be
help
f
ul
f
or
the
e
dg
e
dete
ct
ion
.
This
a
pp
ro
ac
h
i
s
easy
to
i
m
ple
m
ent,
however
,
it
is
no
t
su
ff
ic
ie
nt
fo
r
detect
ing
of
face
.
H
oweve
r,
s
hap
e
of
tem
plate
s
is
plan
ned
and m
anag
ed
i
n
the
se iss
ues.
2.6.
A
ppe
arance
-
b
as
ed
met
ho
ds
Appea
ran
ce
-
ba
sed
m
et
ho
ds
:
Desig
ne
d
m
ain
ly
for
face
detect
ion.
The
app
ea
ra
nce
-
ba
sed
te
ch
niqu
e
dep
e
nds
on
th
e
num
ber
of
pe
op
le
a
utho
rized
c
oach
i
ng
fa
ce
pictu
res
to
search
out
fac
e
m
od
el
s.
This
ty
pe
of
appr
oach
is
be
st
com
par
ed
to
al
te
rn
at
ive
wh
e
n
we
thi
nk
of
perf
or
m
ance.
Ge
ne
rall
y,
the
ap
pear
a
nc
e
-
base
d
te
chn
iq
ue
has
fait
h
in
te
c
hn
i
qu
e
s
f
ro
m
appl
ie
d
m
a
the
m
atics
analy
sis
an
d
m
achine
le
arn
i
ng
t
o
sea
rc
h
out
the
releva
nt
char
act
erist
ic
s
of
face
picture
s.
This
te
ch
niq
ue
c
onjointl
y
e
m
plo
ye
d
in
featur
e
e
xtract
ion
f
or
face r
ec
ogniti
on.
3.
RESU
LT
S
A
ND
DI
SCUS
S
ION
S
Eff
ect
of
li
gh
t
sh
ould
be
el
im
inate
d
to
scal
e
fo
r
a
fi
xed
siz
e
i
m
age
[8
]
.
To
ide
ntifie
s
the
face
on
the
im
age
autom
a
ti
cal
l
y
facial
po
ints
s
hould
be
det
ect
ed.
Algorit
hm
ic
sche
m
e
pro
posed
[
9]
with
the
te
chn
i
qu
e
[10]
is
com
m
on
ly
us
ed
f
or
fa
ce
detec
ti
on
.
Al
gorith
m
ic
ru
le
has
been
te
ste
d
within
the
Cohn
-
Ka
na
de
inf
o,
al
s
o
as
in
te
chnolo
gy
&
Ma
them
at
ic
al
Mod
el
ing
[
11
]
in
fo.
D
et
ect
ion
of
four
te
e
n
po
i
nts
of
facial
exp
re
ssio
n
with
a
m
edian
exa
ct
ness
of
86%
in
Cohn
-
Kan
a
de
inf
o
an
d
83
%
within
the
inf
o
of
m
at
he
m
at
ic
a
l
m
od
el
ing
.In
E
igen
face
base
d
al
gorit
hm
of
Y
og
es
h
T
ay
al
et
.al
[12]
a
ppli
es
to
a
m
ixt
ur
e
of
i
m
ages
ta
ken
with
s
pecial
il
lum
inati
on
s
a
nd
backg
rou
nd,
the
siz
e
of
th
e
i
m
age
an
d
t
he
ti
m
e
need
ed
f
or
t
he
al
gorithm
is
4.
54
56
se
conds
.
He
re
we
can
ta
ke
Eucli
dea
n
wei
gh
t
&
distanc
e
of
t
he
in
pu
t
i
m
age.
W
it
h
the
help
of
data
ba
se
com
par
ing,
c
har
act
erist
ic
s
and
cal
c
ulati
on
of
r
eco
gniz
ing
of
face
is
done.
Kno
wled
ge
ba
sed
m
et
ho
ds
-
this
m
et
ho
d
m
ai
nly
dep
en
ds
on
r
u
le
s
that
are
set
and
it
suppo
rt
hu
m
an
inf
or
m
at
ion
in
disc
ov
e
rin
g
t
he
face.
F
or
exa
m
ple,
face
has
a
nose
an
d
m
ou
t
h
at
a
bo
und
distance
a
nd
sam
e
with
the
nose
with
on
e
an
othe
r.
As
t
he
la
r
ge
dr
a
w
bac
k
wi
th
this
strat
e
gy
is
that
the
pro
blem
in
con
str
uctin
g
asso
ci
at
e
de
gree
acce
ptable
set
of
ru
le
.
Se
ver
al
false
pos
it
ive
foundati
ons
w
ere
ver
y
gen
e
ral
an
d
ca
reful.
This
ty
pe
of
appr
oach
al
on
e
is
sh
or
t
a
ndno
t
ca
pab
le
to
search
ou
t
se
ver
al
faces
i
n
nu
m
ero
us
pic
tures.
The
face,
c
olor
and
s
hape
of
the
sk
in
ada
pt
to
th
e
siz
e
of
the
wind
ow
an
d
to
the
colo
r
sign
at
ur
e
to
ca
lc
ulate
the
col
or
dista
nce
[
13]
.
A
fac
e
norm
al
ly
com
pr
ise
s
no
se
,
ey
es
&
m
ou
th
with
e
xact
dist
ance
a
nd
their
po
sit
io
n
with
eac
h
oth
e
r.
T
he
m
ajor
pro
blem
in
the
m
et
ho
d
c
om
es
w
hen
r
ules
ha
ve
to
be
de
fi
ned.
Tact
ic
s
re
ached
93.4%
detect
ion
with
false p
osi
ti
ves
up to
7.
1%.
3.1.
Te
mpla
te
matchin
g
Tem
plate
Ma
t
chin
g:
A
uthor
in
his
pa
per
[14]
say
s
it
is
based
on
l
ocal,
sta
ti
sti
ca
l
and
local
char
act
e
risti
cs
Bi
nar
y
m
od
el
s
(LBP)
for
the
recogn
it
io
n
of
the
in
de
pende
nt
expre
ssion
of
the
per
s
on
.
The det
ect
ion
with
help o
f
M
MI is e
qu
i
valent to
86.
97% a
nd it
is 85.
6%
i
n
the
J
AF
FE
dat
abase.
3.2.
Ac
tive
sh
ap
e
mode
l
Acti
ve
Sh
a
pe
Mod
el
:
A
uthor
in
his
pap
e
r
[
15
]
,
say
s
the
s
equ
e
nce
of
im
ages
is
perfor
m
ed
fr
an
kly
Mod
el
of
w
i
re
str
uctu
re
and
al
gorithm
for
s
uppo
rt
of
vect
or
and
ta
ck
o
f
act
ive
ap
pe
aran
ce
.
The
m
achine
(S
VM
)
is
us
e
d
for
cl
assifi
c
at
ion
.
T
he
m
o
del
defor
m
s
t
he
sh
a
pe
f
or
t
he
final
f
ram
e
wh
e
n
the exp
ressio
n chan
ge
d.
3.3.
Dis
tribu
t
ion
features
Distrib
ution
F
eat
ur
es:
A
utho
r
in
his
pa
per
[16]
pap
e
r,
the
i
m
ages
wer
e
ta
ken
an
d
fi
ve
sign
ific
ant
par
ts
w
ere
c
ut
ou
t
the
im
age
that
is
m
ade
fo
r
e
xtracti
on
and,
there
for
e,
stores
t
he
sp
eci
fic
ei
ge
nvect
ors
for
ex
pressi
on
s.
The
ei
ge
nvect
or
s
a
re
cal
culat
ed
a
nd
i
nput
facial
im
age
is
acce
pted.
SV
M
is
use
d
for
cl
assifi
cat
ion.
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
An
alysis o
n
te
c
hn
i
qu
e
s
us
ed
to
rec
ognize
an
d
ide
ntif
yi
ng th
e Hum
an e
mo
ti
on
s
(
Pravee
n K
ulkar
ni
)
3311
3.
4
.
Fe
at
ure
e
xt
r
act
i
on
Feat
ur
e
Extra
c
ti
on
:
It
is
a
wa
y
of
recog
nizing
face.
Se
veral
ste
ps
are
i
nvol
ved
su
c
h
as
Re
du
ct
io
n,
extracti
on
of
f
aci
al
app
ea
rance
an
d
c
ollec
tio
n
of
c
har
act
er
ist
ic
s.
Dim
ensi
on
al
decre
ase
i
s
a
sig
nificant
ta
sk
i
n
the
f
or
m
of
re
cogniti
on
syst
em
.
Fig
ur
e
3
s
hows
t
he
diff
e
r
ent
featu
re
e
xtracti
on
with
th
ei
r
recog
niti
on
rate
.
Discrete
Cosi
ne
Tran
s
f
or
m
-
Way
in
of
im
a
ge
DCT
is
a
ppli
ed
and
featu
r
es
su
c
h
as
m
ou
th,
ey
es
are
ext
racted
from
i
m
age [1
7]
an
d
cl
assifi
e
r used i
s Ada
boo
st
w
it
h t
he r
ecognit
ion rate
of
75.93%.
Figure
3. Feat
ure e
xtracti
on a
nd r
ec
ogniti
on
rate
3.
5
.
G
abor
filter
Gabo
r
Fil
te
r:
To
se
parat
e
di
ff
e
ren
t
e
xpress
ion
Ga
bor
filt
ers
are
use
d.
T
he
database
of
ex
pr
essi
ons
us
e
d
to
r
ec
ogni
ze the
recog
niti
on
syst
e
m
w
as
JA
F
FE a
nd re
cogniti
on r
at
e i
s abo
ve 93
%
.
3.
6
.
P
CA
PCA:
Extracti
on
of
facial
fe
at
ur
es
by
anal
yz
ing
the
m
ai
n
co
m
po
ne
nts
accor
ding
to
th
e
auth
or
[18]
An
al
ysi
s
of
th
e
m
a
in
weig
ht
ed
c
om
po
ne
nts
(
WPCA)
t
he
m
et
ho
d
is
us
e
d
in
m
erg
ing
m
ul
ti
ple
functi
on
s
f
or
the
cl
assifi
cat
ion.
T
he
E
uclid
ean
distance
is
cal
culat
ed
t
o
ob
ta
in
the
rese
m
blance
involvin
g
the
m
od
e
ls
an
d
therefo
r
e
the
r
ecognit
ion
of
t
he
facial
ex
pr
e
ssion
is
done
with
the
ne
are
st
al
gorithm
.
The
recog
niti
on
rate
i
s
88.25%
L
DA
:
Linear
discri
m
inant
m
et
ho
d:
was
pro
pos
ed
f
or
cal
culat
ing
disc
rim
inati
ng
vecto
rs
w
it
h
two
sta
ge
proce
dur
e
[
19
]
.
Vect
ors
are
com
bin
e
d
to
get
her
us
i
ng
the
K
-
m
ea
ns
gro
up
i
ng
m
et
hod
with
ea
ch
one
change
sam
ple
and
91% r
ec
ogniti
on
was pr
opos
e
d
i
n
this
cl
assifi
cat
ion
s
:
3.
6
.
1.
Hidden
mark
ov m
od
e
l
Hidden
Ma
rko
v
Mo
del:
Hidd
en
Ma
r
kov
m
od
el
was
dev
el
op
e
d,
cat
e
gori
zi
ng
the
highe
r
em
otion
al
sta
te
s
as
inv
ol
ved
,
insec
ure
,
disag
reein
g,
hopeful
and
discoura
ging,
sta
rting
fro
m
the
lowest
le
vel.
Au
t
hor
vie
w
is
Em
otion
al
cat
egorizat
ion
i
s
well
form
ed
to
know
t
he
sta
te
of
em
oti
on
s
,
s
o
it
wor
ks
as
inf
or
m
at
ion
,
s
o
a
c
orres
pondin
g
assi
gn
m
ent
is
assig
ne
d
betwee
n
facial
featur
es
.
E
xpert
R
ule
is
use
d
for
segm
enting
&
for
rec
ognizin
g
the
em
otion
sta
te
of
nu
m
ber
of
vid
e
o
seq
ue
nces.
I
n
this
the
pro
ba
bili
stic
fr
am
e
of
m
od
el
in
g
va
riable
ti
m
e
s
equ
e
nce
a
nd
t
he
un
i
on
of
de
te
ct
ion
cal
cul
at
ion
is
perf
orm
ed
in
act
ual
tim
e.
Un
sa
fe em
otion
reorg
a
nizat
io
n
is
87 %.
3.
6
.
2.
Neur
al
network
s
Neural
Netw
or
ks
:
The
re
is
an
arr
a
ng
em
ent
of
2
Me
th
od
s
,
the
extracti
on
of
feature
s
an
d
the
ne
ur
al
netw
ork.
T
here
are
two
ph
a
ses
in
face
det
ect
ion
&
cat
al
og
i
ng.
Pr
e
proc
essing
of
im
age
is
do
ne
t
o
r
edu
c
e
the
ti
m
e
ta
ken
an
d
i
ncr
ease
the
ti
m
e
I
m
age
qual
it
y.
He
re
neural
netw
ork
trai
ns
with
fa
ce
an
d
not
the
fac
e
pictures
as
of
t
he
Yale
Face
inf
or
m
at
ion
.
Pic
tures
withi
n
t
he
knowle
dge
set
area
unit
27x1
8
ei
ghte
e
n
pix
el
s
the p
ic
tu
res a
re
a unit
in Grays
cal
e in wra
ng
le
for
m
a and
it
s
sp
ee
d of
c
om
e is 8
4.4 perce
nt.
3.
6
.
3
.
Sup
po
r
t
v
ec
to
r
ma
c
hine
Suppor
t
Vecto
r
Ma
chine:
Sc
ann
i
ng
of
each
vid
e
o
f
ram
es
in
real
tim
e
fo
r
the
fir
st
tim
e
will
detect
the
f
ront
face
,
so
the
faces
a
r
e
resized
i
n
pa
tc
hes
of
im
ages
of
the
sam
e
siz
e
tog
et
he
r
by
m
eans
of
a
ben
c
h
Gabo
r
ene
rg
y
filt
ers.
Finall
y,
auth
or
de
ve
lop
e
d
a
syst
e
m
that
giv
es
the
sty
le
of
com
plete
ly
d
iffer
e
nt
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
o
m
p
En
g,
V
ol.
10
, No
.
3
,
J
une
2020
:
33
0
7
-
33
1
4
3312
countena
nce
c
od
e
s
at
twenty
-
f
our
f
ram
es
per
seco
nd
an
d
anim
a
te
s
the
facial
loo
k
ge
ne
r
at
ed
by
the
co
m
pu
te
r;
it
’s
ad
diti
on
al
ly
the
secret
wr
it
in
g
of
a
bsolutel
y
com
pute
rized
facial
act
ion
.
The
refor
e
,
the
rec
ogniti
on
rate i
s 93%
.
3.
6
.
4.
AdaBo
ost
Ad
aB
oost:
The
fr
ont
face
m
ay
be
a
sequ
en
ce
of
pictu
res
cl
assifi
ed
in
seven
cat
eg
ori
es
as
su
rprise
,
ang
e
r,
fear
,
di
sg
ust
ne
utral,
j
oy,
sa
dness.
Her
e
t
he
po
pula
r
te
ch
nique
is
dead
wh
il
e
no
t
f
eat
ur
e
blo
c
k.
The
col
or
of
the
sk
i
n
is
detect
ed
durin
g
thi
s
do
c
um
e
nt.
The
facial
expre
ssion
ar
ea
unit
extracte
d
an
d
at
la
s
t
the
facial
e
xpr
essions
are
a
unit
disti
nct
fro
m
the
sh
ifti
ng
the
c
ha
racteri
sti
cs
of
the
w
or
l
d
m
ark
on
the
fac
e
from
the
fo
rm
ula
pro
j
ect
ed
by
the
author
victim
iz
at
ion
Classifier
suppo
r
te
d
Ad
aB
oost.
Th
ere
f
or
e,
the
rate
of
accuracy
is
90
%.
As
sho
wn
in
Fig
ure
4
auth
or
s
hav
e
diff
e
re
nt
op
i
nio
n
for
cl
assifi
cat
ion
s
an
d
ac
cur
acy
rate
[20].
Tabl
e
1
sh
ows
the
analy
sis
of
diff
e
ren
t
m
et
ho
ds
and
te
ch
niqu
e
us
ed
f
or
em
otion
detect
ion
with
their acc
ur
acy
.
Figure
4. Cl
assifi
cat
ion
a
nd a
ccur
acy
rate
Table
1.
A
naly
sis of d
i
ff
e
ren
t
m
et
ho
ds an
d
it
s accu
racy
Metho
d
s
Reco
g
n
itio
n
Accurac
y
(%
)
No
.
Of
E
x
p
ressio
n
s
Reco
g
n
ized
Ad
v
an
tag
es An
d
M
ajo
r
Co
n
tribu
tio
n
Ref
.
an
d
Year
CNN
No
t r
ep
o
rted
No
t r
ep
o
rted
-
Multis
cale
f
eatu
re
extractor
-
I
n
plan
e po
se v
ar
iatio
n
[
2
1
]
2
0
0
2
d
LHD
an
d
L
EM
8
6
.6
3
Oriented
structu
ral
f
eatu
res
[
2
2
]
2
0
0
3
RVM
-
relevan
ce
v
ecto
r
m
a
ch
in
e
9
0
.84
-
Reco
g
n
itio
n
in Sta
tic i
m
ag
es
[
2
3
]
2
0
0
5
Multi strea
m
HMM
s
-
-
Reco
g
n
itio
n
er
ror red
u
ced to
44
% com
p
ared
to
Sin
g
le strea
m
[
2
4
]2
0
0
6
ID3 d
ecisio
n
tr
ee
75
6
-
C
o
st ef
f
ectiv
e with respect to
acc
u
r
a
cy
and
with
sp
eed
[
2
5
]
2
0
0
7
LVQ
an
d
GF
8
8
.86
No
t r
ep
o
rted
Ef
f
icien
t e
m
o
tio
n
d
etectio
n
[
2
6
]2
0
0
8
SVM
an
d
G
ASM
9
3
.85
6
Lear
n
in
g
us
in
g
Ad
ab
o
o
st
-
S
electio
n
was
f
lex
ib
le
[
2
7
]2
0
0
9
5
parallel ba
y
esian
class
if
iers
-
-
-
Multi class
if
icatio
n
is achiev
ed
-
S
tatic
an
d
r
eal ti
m
e vid
eo
[
2
8
]
2
0
1
0
SVM
8
2
.5
6
-
B
ased
on
dis
tan
ce featu
res
-
E
f
f
ectiv
e r
ecog
n
itio
n
hig
h
est
CRR
[
2
9
]2
0
1
1
LBP,SV
M
9
5
.84
6
-
I
m
ag
e bas
ed
r
e
co
g
n
itio
n
-
S
p
atial
te
m
p
o
ral
f
eatu
re
[
3
0
]2
0
1
2
Lines
of
con
n
ectiv
ity
9
3
.8
3
Tr
ian
g
u
lar
f
ace
u
si
n
g
L
C an
d
g
eo
m
et
ric
ap
p
roach
[
3
1
]2
0
1
3
GF,M
F
FNN
9
4
.16
7
Co
m
p
u
tatio
n
al cos
t is ver
y
less
[
3
2
]2
0
1
4
SVM
an
d
DC
T
9
8
.63
6
Fast an
d
hig
h
acc
u
racy
[
3
3
]2
0
1
5
OSEL
M
-
SC
9
5
.17
6
-
O
n
lin
e
seq
u
en
tial extre
m
e
lear
n
in
g
m
a
ch
in
e
-
G
o
o
d
r
ecog
n
itio
n
acc
u
rac
y
and
r
o
b
u
stn
ess
[
3
4
]2
0
1
7
DCT,GF
,SV
M
More than
90
p
erce
n
t
8
Go
o
d
Detec
tio
n
r
at
e in v
ariou
s lig
h
t so
u
rce
[
3
5
]2
0
1
8
CNN,SV
M
-
-
Better classif
icatio
n
co
m
p
a
red to
stat
e of
ar
t
CNN
-
D
eep lea
rnin
g
Fea
tu
res
[
3
6
]2
0
1
9
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
An
alysis o
n
te
c
hn
i
qu
e
s
us
ed
to
rec
ognize
an
d
ide
ntif
yi
ng th
e Hum
an e
mo
ti
on
s
(
Pravee
n K
ulkar
ni
)
3313
4.
CONCL
US
I
O
N
The
pur
po
se
of
t
his
pa
per
is
to
identify
face
detect
io
n
pro
blem
s
and
chall
e
ng
es
and
c
om
par
e
nu
m
erous
way
s
fo
r
face
detect
ion
.
T
her
e
’
s
a
m
ajo
r
ad
van
cem
ent
in
this
area
beca
us
e
it
is
helpf
ul
in
real
-
w
orl
d
a
ppli
cat
ion
pro
du
c
t.
Var
io
us
face
detect
ion
te
ch
niques
are
s
um
m
arized,
and
even
t
ually
m
e
thod
s
are
m
entioned
fo
r
face
detec
ti
on
,
their
opti
on
s
,
ad
va
ntage
s
and
acc
ur
ac
y.
This
pa
per
com
par
es
al
gorithm
s
wh
ic
h
ar
e
hel
pful
for
em
otion
detect
ion
ba
sed
on
their
a
ccur
acy
a
nd
re
cent
de
v
el
opm
ent.
T
her
e
is
s
ti
ll
an
honest
sc
ope
f
or
w
ork
to
urg
e
eco
no
m
ic
al
resu
lt
s
by
c
om
bin
ing
or
raisi
ng
the
c
hoic
e
of
op
ti
ons
f
or
de
te
ct
ion
of
face
pictur
es
in
s
pite
of
inten
sit
y
of
bac
kgrou
nd
colo
r
or
a
ny
occlusi
on.
T
he
i
m
po
rtant
f
eat
ur
e
enh
a
ncem
ents
wh
ic
h
ar
e
discuss
e
d
f
ro
m
recent
pap
e
rs
are
em
otion
de
te
ct
ion
from
the
side
vie
w
s
and
dif
fe
re
nt p
a
ra
m
et
ers
for real
tim
e
app
li
cat
ion
s s
uc
h
as m
ed
ic
al
, r
ob
otics, f
or
e
ns
ic
secti
on
and m
any
m
ore.
REFERE
NCE
S
[1]
P.
Ekman,
and
W
.
V
Friesen,
“
Faci
a
l
ac
t
ion
co
ding
s
y
stem:
Manua
l
,”
Pal
o
Al
t
o,
Cali
f:
Consul
ti
ng
Psyc
ho
logi
sts
Press,
1978.
[2]
I.
M.
Revi
n
a,
an
d
W
.
R.
Sam
Em
m
anue
l,
“
A
Surve
y
on
Hum
an
Fa
ce
Expr
ession
Rec
ogni
ti
on
T
ec
h
nique
s
,
”
Journal
of
King
Saud
Un
iv
ersit
y
-
Comput
er
and
Informati
on
Scienc
es,
201
8
.
[3]
Ming
-
Hs
uan
Yang,
D
.
J
.
Kri
eg
m
an,
and
N
.
Ah
uja
,
“
Det
ec
t
ing
Face
s
in
Im
age
s
:
A
surve
y
,”
IE
EE
Tr
ansacti
ons
on
Pat
te
rn
Analysis
and
Mac
h
ine In
te
lligen
ce
,
vol
.
2
4,
no
.
1
,
pp
.
34
-
58,
2002
.
[4]
A.
Punitha
,
and
M.
K
.
Gee
tha
,
“
Te
xtur
e
Based
E
m
oti
on
Rec
ognition
from
Faci
al
Expre
ss
ions
using
Support
Vec
tor
Mac
hine,”
Int
.
J
.
Comput.
Appl,
v
ol.
80
,
no
.
5
,
pp
.
1
-
5,
2013
.
[5]
K.
C.
Yow
,
and
R.
Cipo
ll
a
,
“
Fea
ture
-
Based
Hum
an
Fac
e
De
te
c
tion
,
”
Imag
e
and
Vi
sion
Computin
g
,
vol
.
15
,
no
.
9
,
pp
.
713
-
735
,
19
97
.
[6]
S.
Kr.
Singh,
D.
S.
Chauha
n,
and
M.
Vatsa
,
and
R
.
Singh,
“
A
Robust
Skin
Color
Based
Face
De
tect
ion
Algorit
hm
,
”
Tam
kang
Journal
of
Scienc
e
and
Engi
ne
ering,
vol
6,
no.
4,
pp
.
227
-
234,
2003
[7]
J.
Li,
G.
Poulto
n,
Y.
Guo
,
and
Rong
-
Yu
Qiao,
“
Face
Rec
ogn
it
i
on
Based
on
Multi
pl
e
Reg
ion
F
ea
tur
es
,
”
DICT
A,
pp
.
69
-
78
,
2003
.
[8]
R
.
Pati
l
,
and
V
.
Sahula
,
and
A
S
Manda
l,
“
Autom
at
ic
Det
ec
t
io
n
of
Faci
al
Fea
t
ure
Points
in
Im
age
Sequenc
es
,
”
Inte
rnational
Co
nfe
renc
e
on
Image
Information
Proce
ss
ing,
pp
.
1
-
5,
2011
.
[9]
W
.
Kienz
l
e,
et
al.
,
“
Fac
e
Det
ection
-
Eff
ic
i
ent
an
d
Rank
Defi
c
ient
,
”
Adv
an
ce
s
i
n
Neural
Information
Proc
essing
Syste
ms
,
pp
.
673
-
680,
2005
.
[10]
P.
Viola
,
M
.
Jones
,
“
Robust
Rea
l
-
Ti
m
e
Obje
ct
Dete
c
ti
on
,
”
Int
ernati
onal
Journal
of
Compute
r
Vi
sion
,
vol
.
4
,
pp
.
34
-
47
,
2001
.
[11]
G
Hem
al
at
ha,
a
nd
CP
Sum
at
hi,
“
A
Study
of
T
ec
hniqu
es
for
F
ac
i
al
Detect
ion
and
Expre
ss
ion
Cla
ss
ifi
c
at
ion
,
”
Inte
rnational
Jo
urnal
of
Comput
er
Scienc
e
and
E
ngine
ering
Surv
ey
,
vol
.
5,
no
.
2
,
2014
.
[12]
Y.
Tay
al
,
P.
K
.
Pande
y
and
D.
B.
V.
Singh,
“
Face
Rec
ogn
it
io
n
using
Ei
genf
a
ce
,
”
Inte
rnation
al
Associat
ion
o
f
Sci
en
ti
fic
Inno
va
ti
on
and
Re
s
earc
h
(
IASI
R)
,
vol. 3
,
no
.
1
,
pp
.
50
-
5
5,
201
3.
[13]
Y
-
Buhee
and
s
ukhanl
e
e,
“
Robust
Face
Det
ec
t
ion
Based
on
Know
le
dge‐
Dire
ct
ed
Speci
f
ication
of
Bot
tom‐U
p
Sali
en
c
y
,
”
ETRI
journal
,
vo
l
.
33,
no
.
4,
2011
.
[14]
C.
Shan,
S
.
Gong
,
and
P.
W.
McOw
an,
“
Fa
ci
a
l
Expre
ss
ion
Rec
ognit
ion
B
ase
d
on
Loc
al
Bina
r
y
Pa
tt
e
rns
:
A
Com
pre
hensive
Stud
y
,
”
Image
and
vi
sion
Computing,
vol.
27
,
n
o.
6
.
pp
.
803
-
81
6,
2009
.
[15]
R
.
A
.
Pati
l
,
V
.
Sahula
and
AS
Manda
l,
“
Facial
Expre
ss
ion
Recogniti
on
in
Im
a
ge
Sequences
using
Acti
ve
Shap
e
Model
and
Support
Vec
tor
Ma
chi
ne
,
”
2011
U
KSIM
5th
European
Symposiu
m
on
Compute
r
Mode
li
ng
and
Simulat
ion
,
201
1.
[16]
J
.
Kali
t
a,
and
K
.
Das,
“
Rec
og
nit
ion
of
Fa
ci
a
l
Expre
ss
ion
usi
ng
Ei
genv
ec
to
r
Based
Distribu
te
d
Fea
ture
s
an
d
Euc
li
d
ea
n
Dista
nce
B
ase
d
De
ci
s
ion
Making
T
echnique
,
”
I
JA
CS
A,
vo
l
.
4,
no.
2,
2013
.
[17]
Z
.
Zha
ng
,
X
.
Mu,
L.
Gao,
“
Rec
ogni
zi
ng
Fa
ci
a
l
Expr
essions
base
d
on
G
a
bor
Filt
e
r
Sel
e
ct
ion
,
”
2011
4
th
Inte
rnational
Co
ngress
on
Image
and
Signa
l Proc
essing
,
vol
.
3,
pp
.
1544
-
1548
,
20
11.
[18]
Z
.
Niu
,
and
X
.
Qiu,
“
Faci
a
l
E
xpre
ss
ion
Rec
o
gnit
ion
b
ase
d
o
n
W
ei
ghte
d
Pri
nci
pa
l
Com
ponent
Anal
y
sis
an
d
Support
Vec
tor
Mac
hine
s
,
”
201
0
3rd
Inte
rnational
Confe
renc
e
on
Adv
ance
d
C
omputer
Theory
and
Engi
nee
rin
g
(
ICACTE
)
,
2010.
[19]
Li
-
Fen
Chen,
et
al.
,
“
A
New
LDA
-
Based
Face
Rec
ognition
Sy
stem
which
c
a
n
Solve
the
Smal
l
Sam
ple
Size
Problem
,
”
Pat
te
r
n
Rec
ogn
it
ion
,
v
ol
.
33
,
no
.
3,
pp
.
1713
-
1726,
200
0
.
[20]
G
Hem
al
at
ha,
a
nd
CP
Sum
at
hi,
“
A
Study
of
T
ec
hniqu
e
s
for
F
ac
i
al
Detect
ion
and
Expre
ss
ion
Cla
ss
ifi
c
at
ion
,
”
Inte
rnational
Jo
urnal
of
Comput
er
Scienc
e
and
E
ngine
ering
Surv
ey
,
vol
.
5,
no
.
2,
pp
.
27
-
37
,
2014
.
[21]
B
.
Fasel
,
“
Hea
d
-
Pos
e
Inva
ria
n
t
F
ac
i
al
Expre
ss
ion
Rec
ogn
it
io
n
usi
ng
Convolut
ion
a
l
Neura
l
Networ
ks
,
”
Proc
ee
d
ings
of
th
e
4th
IE
EE I
nte
rnational
Co
nfe
renc
e
on
Mul
ti
modal
In
te
rfac
es,
p
p
.
529
,
200
2
[22]
D
.
Datc
u
and
L.
J.
M.
Rothkra
ntz
,
“
Faci
al
Exp
re
ss
ion
Rec
ognit
io
n
with
Rel
eva
n
c
e
Vec
tor
Mac
hi
nes
,
2005
IEE
E
Inte
rnational
Co
nfe
renc
e
on
Mul
ti
media
and
E
xp
o
,
pp
.
193
-
196
,
2005
.
[23]
P
.
S
.
Katsa
ggel
os
,
and
A
.
K
Aggel
os
,
“
Autom
at
ic
Faci
a
l
Ex
pre
ss
io
n
Rec
ognition
using
Faci
a
l
Anim
at
ion
Para
m
eter
s
and
Mult
istre
am
HMMs,
”
IEEE T
rans
act
ions o
n
Information
For
ensic
s and
S
ec
ur
it
y
,
vol
.
1
,
no
.
1
,
pp
.
3
-
11
,
2006
.
[24]
P
.
Ekman
,
and
W
.
V.
Friesen,
“
Un
m
asking
the
Face
:
A
Guide
to
Re
cogni
z
ing
Emotions
From
Faci
al
C
lue
s
,
”
Malor
Books,
20
03
.
[25]
S
.
Noh,
H.
Pa
rk,
Y.
Jin
,
Jong
-
Il
.
Park
,
“
Feat
ure
-
Adap
ti
ve
Motion
Ene
rg
y
Anal
y
sis
for
Faci
a
l
Expre
ss
i
on
Rec
ognition
,
”
In
te
rnational
Sym
posium on
Vi
sua
l
Computing
,
pp
.
452
-
463,
2007.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
o
m
p
En
g,
V
ol.
10
, No
.
3
,
J
une
2020
:
33
0
7
-
33
1
4
3314
[26]
Y
.
Sun,
L
.
Yin,
“
Faci
al
Expre
s
sion
Rec
og
nit
io
n
base
d
on
3D
Dy
n
amic
Rang
e
Model
Sequenc
es
,
”
Europea
n
Confe
renc
e
on
C
omputer
Vi
sion
,
pp
.
58
–
71
,
2008
.
[27]
G
.
Zha
o,
and
M
.
Piet
ik,
“
Boosted
Multi
-
Resolutio
n
Spati
ote
m
pora
l
Descri
ptors
for
Faci
a
l
Expre
ss
io
n
Rec
ognition
,
”
Pat
te
rn
Recogni
t
ion
Letters
,
vo
l
.
30,
no
.
12
,
pp
.
1
117
-
1127,
2009
.
[28]
S
.
B.
Ka
zmi
,
Qu
rat
-
ul
-
Ain
,
M.
Ishtia
q
,
M
.
A
.
Jaff
ar,
“
T
ext
ure
Ana
l
y
sis
B
ase
d
Fa
cial
Exp
ression
R
ec
ogni
ti
on
using
A
Bank
Of
Ba
y
esi
an
Cla
ss
ifie
rs
,
”
2010
Inte
rn
ati
onal
Confe
r
e
nce
on
Information
and
Eme
rging
Technol
ogie
s
,
pp
.
1
-
6,
2010
.
[29]
L
.
Zh
ang,
and
D
.
Tj
ondrone
go
ro,
“
Faci
al
Exp
ression
Rec
ogni
ti
on
using
Facial
Movem
ent
Feat
ur
es
,
”
I
EEE
Tr
ansacti
ons on Aff
e
ctive Compu
ti
ng,
vol
.
2
,
no
.
4,
pp
.
219
-
229
,
2011
.
[30]
A
.
Pours
abe
ri,
H
.
A.
Noubari
,
M.
Gavri
lova,
and
S.
N.
Yanushke
vic
h,
“
Gauss
–
Laguerre
W
avele
t
Te
xtur
al
Fea
ture
Fus
ion
with
Geo
m
et
ric
a
l
Inform
at
ion
for
Faci
a
l
E
xpre
ss
ion
Ide
nti
f
ic
a
ti
on,
”
EURA
S
IP
Journal
on
Image
and
Vi
deo
Proce
ss
ing,
vo
l
.
2012,
no
.
1
,
pp
.
17,
2012
.
[31]
M
.
R
.
Dil
ee
p
,
and
A
.
Dant
i,
“
Li
nes
Of
Connec
t
ivi
t
y
-
Fa
ce
Model
for
Re
cogni
ti
on
of
th
e
Hum
an
Faci
a
l
Expre
ss
ions,”
In
t.
J. A
rti
f. Intell. Mec
h,
vol
.
2,
no
.
2,
2013
[32]
E
.
Ow
usu,
Y.
Zha
n,
and
Q.
R.
Mao
,
“
A
Neura
l
-
Adaboost
Based
Faci
al
Expre
ss
ion
Rec
ognition
Sy
stem
,
”
Ex
p
ert
Syste
ms
wit
h
Ap
pli
cations,
vo
l
.
4
1,
no
.
7,
pp
.
338
3
-
3390,
2014
.
[33]
S
.
Biswas,
and
J
.
Sil,
“
An
Eff
ic
i
ent
Expre
ss
ion
Rec
ognition
Met
hod
using
Contourl
et
Tr
ansform
,
”
Proce
ed
ings
of
the
2nd
Int
ernational
Conf
ere
nce on
P
erc
ep
ti
on
a
nd
Mac
hin
e
In
telli
gen
ce,
pp
.
167
-
174,
2015
.
[34]
A.
Uca
r
,
Y.
De
m
ir
and
C.
Guz
el
is.
“
A
New
Facial
Expre
ss
ion
Rec
ogn
it
ion
ba
sed
on
Curve
le
t
Tra
nsform
and
Online
Sequ
ent
i
al
Ext
reme
Lea
rning
Mac
h
ine
Init
iali
ze
d
wi
th
Spherical
Cluste
ring
,
”
Neural
Computing
an
d
Appl
ic
a
ti
ons,
vo
l
.
27
,
no
.
1
,
pp
.
1
31
-
142,
2016
.
[35]
Hung
-
Hs
u
Tsai
,
Yi
-
Cheng
Chang
,
“
Faci
al
Expre
ss
ion
Re
cogni
t
ion
using
A Com
bina
ti
on
of
Mult
iple
Facial
Feat
ur
es
and
Support
V
ector
Mac
h
ine
,
”
So
ft
Computing
,
vo
l
.
22
,
no
.
13
,
pp
.
4389
-
4405,
201
8
.
[36]
Feng
y
uanWang
,
Jianhua
Lv,
Guode
Ying,
Shenghui
Chen,
Chi
Z
hang,
“
Faci
a
l
Ex
pre
ss
ion
Rec
ogn
it
ion
from
Im
age
base
d
on
H
y
brid
Feat
ur
es
Under
standi
ng
,
”
Journal
of
Vi
sual
Co
mm
unic
ati
on
an
d
Image
R
epre
s
ent
ati
on
,
v
ol
.
59
,
pp
.
84
-
88
,
2019
.
Evaluation Warning : The document was created with Spire.PDF for Python.