Inter
national
J
our
nal
of
Electrical
and
Computer
Engineering
(IJECE)
V
ol.
10,
No.
4,
August
2020,
pp.
4080
4092
ISSN:
2088-8708,
DOI:
10.11591/ijece.v10i4.pp4080-4092
r
4080
Local
featur
e
extraction
based
facial
emotion
r
ecognition:
A
sur
v
ey
Khadija
Slimani
1
,
Mohamed
Kas
2
,
Y
oussef
El
merabet
3
,
Y
assine
Ruichek
4
,
Rochdi
Messoussi
5
1,2,3,5
Laboratory
Systems
of
T
elecommunication
and
Decision
Engineering
(LASTID),
F
aculty
of
Sciences,
Ibn
T
of
ail
Uni
v
ersity
,
Morocco
2,4
CIAD
UMR
7533,
Uni
v
ersit
´
e
Bour
gogne
Franche-Comt
´
e,
UTBM,
France
Article
Inf
o
Article
history:
Recei
v
ed
Oct
2,
2019
Re
vised
Feb
25,
2020
Accepted
Mar
4,
2020
K
eyw
ords:
Basic
emotion
Features
e
xtraction
Image
processing
Machine
learning
ABSTRA
CT
Notwithstanding
the
recent
technological
adv
ancement,
the
identification
of
f
acial
and
emotional
e
xpressions
is
s
till
one
of
the
greatest
challenges
scientists
ha
v
e
e
v
er
f
aced.
Generally
,
the
human
f
ace
is
identified
as
a
composition
made
up
of
te
xture
s
arranged
in
micro-patterns.
Currently
,
there
has
bee
n
a
tremendous
increase
in
the
us
e
of
Lo-
cal
Binary
P
attern
based
te
xture
algorithms
which
ha
v
e
in
v
ariably
been
identified
to
being
essential
in
the
completion
of
a
v
ariety
of
tasks
and
in
the
e
xtraction
of
essen-
tial
attrib
utes
from
an
image.
Ov
er
the
years,
lots
of
LBP
v
ariants
ha
v
e
been
literally
re
vie
wed.
Ho
we
v
er
,
what
is
left
is
a
thorough
and
comprehensi
v
e
analysis
of
their
independent
performance.
This
research
w
ork
aims
at
filling
this
g
ap
by
performing
a
lar
ge-scale
performance
e
v
aluation
of
46
recent
state-of-the
-art
LBP
v
ariants
for
f
acial
e
xpression
r
ecognition.
Extensi
v
e
e
xperim
ental
results
on
the
well-kno
wn
challenging
and
benchmark
KDEF
,
J
AFFE,
CK
and
MUG
databases
tak
en
under
dif
ferent
f
acial
e
xpression
conditions,
indicate
that
a
number
of
e
v
aluated
state-of-the-art
LBP-lik
e
methods
achie
v
e
promising
results,
which
are
better
or
competiti
v
e
than
se
v
eral
re-
cent
state-of-the-art
f
acial
recognition
systems.
Recognition
rates
of
100%,
98.57%,
95.92%
and
100%
ha
v
e
been
reache
d
for
CK,
J
AFFE,
KDEF
and
MUG
databases,
respecti
v
ely
.
Copyright
©
2020
Insitute
of
Advanced
Engineeering
and
Science
.
All
rights
r
eserved.
Corresponding
A
uthor:
Slimani
Khadija,
Laboratory
Systems
of
T
elecommunication
and
Decision
Engineering
(LASTID),
F
aculty
of
Sciences,
Ibn
T
of
ail
Uni
v
ersity
,
K
enitra,
Morocco.
Email:
slimani.khadija@uit.ac.ma
1.
INTR
ODUCTION
W
ith
the
de
v
elopment
of
artificial
intelligence
and
pattern
recognition,
computer
based
f
acial
e
xpres-
sion
recognition
has
attracted
man
y
researchers
in
the
domain
of
computer
vision.
Se
v
eral
studies
ha
v
e
sho
wn
that
the
f
acial
e
xpression
contrib
utes
to
better
understand
the
con
v
ersations
[1,
2],
and
it
helps
to
e
xpress
the
indi
vidual’
s
internal
emotions,
also,
it
is
considered
as
the
main
modality
for
human
communication.
Recent
progresses
in
psychology
and
neuroscience
fields
gi
v
e
a
more
positi
v
e
interpretation
of
the
emotions
role
in
human
beha
vior
[3].
The
f
acial
emotion
recognition
system
resides
of
three
i
mportant
steps;
f
ace
de-
tection,
feature
e
xtraction
and
classification.
By
taking
image
or
series
of
images
as
input,
the
most
important
step
is
feature
e
xtract
ion
that
all
o
ws
to
descri
be
the
input
images
and
calculate
their
characteristic
v
ector
using
a
gi
v
en
operator
.
Indeed,
e
xtracting
poor
features
in
v
olv
es
producing
poor
recognition
quality
e
v
en
with
the
use
of
best
classifiers.
Because
of
the
e
xceptional
e
xhibition
of
LBP
based
techniques,
the
y
ha
v
e
de
v
eloped
as
one
of
the
most
unmistakable
local
image
descriptors.
Although
initially
intended
for
te
xture
analysis
[4],
the
LBP
descriptor
has
gi
v
en
e
xcellent
outcomes
in
dif
ferent
applications
because
of
its
in
v
ariance
to
monotonic
global
grayle
v
el
changes,
furthermore,
its
better
resistance
ag
ainst
brightening
changes
property
in
real-w
orld
J
ournal
homepage:
http://ijece
.iaescor
e
.com/inde
x.php/IJECE
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
4081
applications
including
f
ace
recognition.
Another
equally
important
property
is
its
computational
ef
fortlessness
and
the
lo
w
length
of
its
histogram
v
ector
,
which
mak
e
it
ready
to
e
xamine
images
in
challenging
real-time
settings.
The
achie
v
ement
of
the
LBP
in
numerous
applications
concei
v
ed
an
of
fspring
of
an
immense
number
of
LBP
v
a
riations,
which
ha
v
e
been
proposed
and
k
eep
on
being
proposed.
W
ithout
a
doubt,
since
Ojala’
s
w
ork
[4]
and
because
of
its
adaptability
and
ef
fecti
v
eness,
the
general
LBP-lik
e
w
ay
of
thinking
has
demonstrated
e
xtremely
well
kno
wn,
and
an
e
xtraordinary
assortment
of
LBP
v
ariations
ha
v
e
been
proposed
in
the
writing
to
impro
v
e
discriminati
v
e
po
wer
,
rob
ustness,
and
appropriateness
of
LBP
.
The
main
o
bj
ecti
v
e
of
this
study
is
to
perform
a
lar
ge
scale
performance
e
v
aluation
for
f
acial
emotion
recognition,
assessing
46
recent
state-of-the-
art
te
xture
features,
on
four
widely-used
benchmark
databases.
Performance
of
the
adopted
f
acial
e
xpression
recognition
system
coupled
with
the
best
e
v
aluated
te
xture
descriptor
on
each
dataset
is
compared
ag
ainst
those
of
state-of-the-art
approaches.
W
e
disclose
in
the
e
xperimental
section
the
f
act
that
some
descriptors
originally
proposed
for
applications
other
than
f
acial
emotional
recognition
allo
w
outperforming
se
v
eral
recent
state-of-
the-art
systems.
The
remaining
sections
of
this
research
w
ork
are
arranged
in
the
follo
wing
w
ay:
Section
2.
re
vie
ws
the
traditional
LBP
operator
as
well
as
some
of
its
recent
and
popular
v
ariants.
Section
3.
re
vie
ws
the
fe
w
e
xisting
surv
e
ys
on
te
xture
descriptor
based
classification
a
nd
recognition
as
well
as
the
e
v
aluated
state-
of-the
art
LBP-lik
e
methods.
Section
4.
pro
vides
detailed
e
xplanation
on
the
results
of
t
he
e
xperiments
while
comparing
the
performances
of
the
best
performing
descriptors
on
each
tested
datasets
with
those
of
recent
state-of-the-art
f
acial
emotional
recognition
systems.
Finally
,
section
5.
dra
w
this
paper
to
a
close
by
proposing
some
future
research
perspecti
v
es.
2.
BRIEF
REVIEW
OF
EXISTING
METHODS
The
original
Local
Binary
P
atter
n
(LBP)
operator
proposed
by
Ojala
et
al
[4]
,
which
consists
in
coding
the
pix
el-wi
se
information
in
an
image,
is
a
po
werful
te
xture
analysis
descriptor
.
It
aims
to
search
micro-te
xtons
in
local
re
gions.
The
v
alue
I
p
of
the
pix
els
in
a
3
3
grayscale
image
patch
around
the
central
pix
el
I
c
are
turned
into
binary
v
alues
(0
or
1)
by
comparing
them
with
I
c
(v
alue
of
the
central
pix
el).
The
obtained
binary
numbers
are
encoded
to
characterize
a
local
structure
pattern
and
then
the
code
is
transformed
into
decimal
number
.
Once
a
LBP
code
of
each
pix
el
is
obtained,
a
histogram
is
b
uilt
to
represent
the
te
xture
image.
F
or
a
3
3
neighborhood,
the
definition
of
the
k
ernel
function
of
LBP
operator
is
gi
v
en
in
(cf.
Eq
(1)),
where
I
p
(p
2
f
1,
2,
...,
P
g
)
signifies
the
gray
le
v
els
of
the
peripheral
pix
els,
P
corresponds
to
the
number
of
neighboring
pix
els
(P=8)
and
'
(
)
is
the
Hea
viside
step
function
(cf.
Eq
(1)).
LBP
(
I
c
)
=
P
=
8
X
p
=
1
'
(
I
p
I
c
)
2
p
1
;
'
(
z
)
=
1
;
z
0
0
;
z
<
0
(1)
Local
binary
patterns
by
neighborhoods
(nLBPd)
operator
[5]
consists
in
encoding
the
relationship
between
each
pair
of
the
peripheral
pix
els
I
0
,
I
1
,
I
2
,
...,
I
7
around
the
central
pix
el
I
c
in
a
3
3
square
neighborhood.
The
pairs
of
pix
els
are
compared
with
sequential
neighbors
or
within
neighbors
possesing
a
distance
length
d.
The
k
ernel
function
of
nLBPd
code
is
defined
by
(cf.
Eq.
(2)).
When
d=1,
the
binary
code
of
the
central
pix
el
I
c
is
gotten
as
belo
w
(Eq.
(3)):
nLBP
d
(
I
c
)
=
P
1
X
p
=0
'
(
I
p
;
I
(
p
+
d
mod
P
)
)
2
p
(2)
I
c
=
'
(
I
0
>
I
1
)
;
'
(
I
1
>
I
2
)
;
'
(
I
2
>
I
3
)
;
'
(
I
3
>
I
4
)
;
'
(
I
4
>
I
5
)
;
'
(
I
5
>
I
6
)
;
'
(
I
6
>
I
7
)
;
'
(
I
7
>
I
8
)
(3)
The
pr
ocedur
e
of
Local
Gr
aph
Structur
e
(LGS)
descriptor
introduced
by
Ab
usham
et
al.
[6]
is
to
e
xploit
the
d
om
inant
graph
process
in
order
to
encode
the
spatial
data
for
an
y
pix
el
in
the
image.
LGS
is
based
on
local
graph
structures
in
local
graph
neighborhood.
The
graph
structure
of
LGS
represents
more
left-handed
neighbor
pix
els
than
right-handed
ones.
T
o
o
v
ercome
this
defect,
Extended
Local
Graph
Structure
(ELGS)
operator
is
proposed
[7].
The
procedure
for
ELGS
is
based
on
using
the
LGS
te
xture
descriptor
to
b
uild
tw
o
descriptions
(horizontally
and
v
ertically)
and
then
combine
them
into
a
global
description.
Local
featur
e
e
xtr
action
based
facial...
(Slimani
khadija)
Evaluation Warning : The document was created with Spire.PDF for Python.
4082
r
ISSN:
2088-8708
3.
EV
ALU
A
TED
ST
A
TE-OF-THE-AR
T
LBP
V
ARIANTS
The
pioneering
LBP
w
ork
[4]
and
its
success
in
numerous
computer
vision
problems
and
a
p
pl
ications
has
inspired
the
de
v
elopment
of
great
number
of
ne
w
po
werful
LBP
v
ariants.
LBP
descriptor
is
adaptable
to
suit
in
man
y
dif
ferent
applications
requirements.
Indeed,
after
Ojala’
s
w
ork,
e.g.,
Heikkila
et
al
[8],
se
v
eral
modifications
and
e
x
t
ensions
of
LBP
ha
v
e
been
de
v
eloped
with
the
aim
to
increase
its
rob
ustness
and
discrimi-
nati
v
e
po
wer
.
These
e
xtensions
and
modifications
of
LBP
,
de
v
eloped
usually
i
n
conjunction
with
their
intended
applications
(see
T
able
1),
focus
on
se
v
eral
aspects
of
the
LBP
method
such
as,
Quantization
to
multiple
le
v
el
via
thresholding;
sampeling
local
feature
v
ectors
and
pix
el
patterns
with
some
neighborhood
topology;
com-
bining
multiple
complementary
features
within
LBP-lik
e
and
with
non-LBP
descriptors
for
both
images
and
videos
and
finally
,
re
grouping
and
mer
ging
patterns
to
increase
distincti
v
eness.
T
able
1.
Summary
of
te
xture
descriptors
tested.
Ref
Y
ear
Complete
name
Abbre
viation
Application
[4]
2002
Local
Binary
P
attern
LBP
T
e
xture
classification
[9]
2003
Simplified
T
e
xture
Unit
+
STU+
T
e
xture
classification
[10]
2004
Gradient
te
xture
unit
coding
GTUC
T
e
xture
classification
[11]
2005
Dif
ference
Symmetric
Local
Graph
Structure
DSLGS
Finger
v
ein
recognition
[8]
2006
Center
-Symmetric
Local
Binary
P
atterns
CSLBP
T
e
xture
classification
[12]
2008
Centralized
Binary
P
attern
CBP
F
acial
e
xpression
recognition
[13]
2010
Local
T
ernary
P
atterns
L
TP
F
ace
recognition
[14]
2010
Directional
Binary
Code
DBC
F
ace
recognition
[15]
2010
Impro
v
ed
Local
T
ernary
P
atterns
IL
TP
Medical
image
analysis
[16]
2010
Local
Directional
P
attern
LDP
F
ace
recognition
[17]
2011
Binary
Gradient
Contours
(1)
BGC1
T
e
xture
classification
[17]
2011
Binary
Gradient
Contours
(2)
BGC2
T
e
xture
classification
[17]
2011
Binary
Gradient
Contours
(3)
BGC3
T
e
xture
classification
[18]
2011
Center
-Symmetric
Local
T
ernary
P
atterns
CSL
TP
Feature
description
[18]
2011
Extended
Center
-Symmetric
Local
T
ernary
P
atterns
eCSL
TP
Image
retrie
v
al
[19]
2011
Impro
v
ed
Local
Binary
P
atterns
ILBP
F
ace
detection
[6]
2011
Local
Graph
Structure
LGS
F
ace
recognition
[20]
2012
Local
Maximum
Edge
Binary
P
atterns
LMEBP
Image
retrie
v
al
[16]
2013
Impro
v
ed
binary
gradient
contours
(1)
IBGC1
T
e
xture
classification
[21]
2013
Local
Directional
Number
P
attern
LDN
F
ace
e
xpression
analysis
[22]
2013
Local
Gray
Code
P
attern
LGCP
F
ace
e
xpression
analysis
[23]
2013
Rotated
Local
Binary
P
attern
RLBP
T
e
xture
classification
[24]
2015
Adapti
v
e
Extended
Local
T
ernary
P
attern
AEL
TP
T
e
xture
classification
[5]
2015
Directional
Local
Binary
P
atterns
dLBP
T
e
xture
classification
[5]
2015
Local
Binary
P
atterns
by
neighborhoods
nLBPd
T
e
xture
classification
[25]
2015
Maximum
Edge
Position
Octal
P
attern
MMEPOP
Image
retrie
v
al
[26]
2015
Multi-Orientation
W
eighted
Symmetric
Local
Graph
Structure
MO
W
-SLGS
Finger
v
ein
recognition
[27]
2015
Orthogonal
Symmetric
Local
T
ernary
P
attern
OSL
TP
Image
re
gion
description
[26]
2015
Symmetric
Local
Graph
Structure
SLGS
Finger
v
ein
recognition
[28]
2015
eXtended
Center
-Symmetric
Local
Binary
P
attern
XCS
LBP
T
e
xture
classification
[29]
2016
Adapti
v
e
Local
T
ernary
P
atterns
AL
TP
F
ace
recognition
[29]
2016
Center
-Symmetric
AL
TP
CSAL
TP
F
ace
recognition
[30]
2016
Diagonal
Direction
Binary
P
attern
DDBP
F
ace
recognition
[7]
2016
Extended
Local
Graph
Structure
ELGS
T
e
xture
classification
[31]
2016
Local
Extreme
Sign
T
rio
P
atterns
LESTP
Image
retrie
v
al
[32]
2016
Quad
Binary
P
attern
QBP
T
ar
get
tracking
[31]
2016
Sign
Maximum
Edge
Position
Octal
P
attern
SMEPOP
Image
retrie
v
al
[33]
2016
Complete
Eight
Local
Directional
P
atterns
CELDP
F
ace
recognition
[34]
2017
Centre
Symmetric
Quadruple
P
attern
CSQP
F
acial
image
recognition
and
retrie
v
al
[35]
2017
Local
Directional
Binary
P
atterns
LDBP
T
e
xture
classification
[36]
2017
Local
neighborhood
dif
ference
pattern
LNDP
Natural
and
te
xture
image
retrie
v
al
[37]
2017
Local
Quadruple
P
attern
LQP
A
T
F
acial
image
recognition
and
retrie
v
al
[38]
2018
Local
Diagonal
Extrema
Number
P
attern
LDENP
F
ace
recognition
[39]
2018
Local
Conca
v
e-and-Con
v
e
x
Micro-Structure
P
atterns
LCCMSP
T
e
xture
classification
[40]
2018
Local
Directional
T
ernary
P
attern
LDTP
T
e
xture
classification
[41]
2018
Repulsi
v
e-and-Attracti
v
e
Local
Binary
Gradient
Contours
RALBGC
T
e
xture
classification
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
4080
–
4092
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
4083
There
are
se
v
eral
researches
reported
in
the
literature
that
are
de
v
oted
to
surv
e
ying
LBP
and
its
v
ari
-
ants.
One
can
cite:
(a)
Hadid
et
al
.
[42]
re
vie
wed
13
LBP
v
ariants
and
pro
vided
a
comparati
v
e
analysis
on
tw
o
dif
ferent
problems
which
are
gender
and
te
xture
classification.
(b)
The
w
ork
of
Fernandez
et
al.
[43]
attempted
to
b
uild
a
general
frame
w
ork
for
te
xture
e
xamination
that
the
authors
refer
to
as
histograms
of
equi
v
alent
patterns
(HEP).
A
set
of
38
LBP
v
ariants
and
non
LBP
strate
gies
are
e
x
ecuted
and
e
xperimentally
assessed
on
ele
v
en
te
xture
datasets.
(c)
Huang
et
al.
[44]
displayed
a
surv
e
y
of
LBP
v
ariants
in
the
application
re
gion
of
f
acial
image
processing.
Ho
we
v
er
,
there
is
no
e
xperimental
study
of
the
LBP
strate
gies
themselv
es.
(d)
Nanni
et
al.
[45]
e
xamined
the
performance
of
LBP
based
te
xture
descriptors
in
a
f
airly
specific
and
narro
w
application,
which
consists
in
classifying
cell
and
tissue
images
of
fi
v
e
datasets.
(e)
Michael
Bereta
et
al.
[46]
highlighted
man
y
types
of
local
descriptors
including
local
binary
patterns
and
their
combination
with
Gabor
filters.
The
y
e
xamined
only
14
LBP
v
ariants
on
FERET
database.
(f)
Lumini
et
al.
[47]
e
v
aluated
the
ef
fecti
v
eness
of
LBP
,
HOG,
POEM,
MBC,
HASC,
GOLD,
RICLBP
,
and
CLBP
descriptors.
Each
of
these
feature
e
xtraction
methods
is
carried
out
only
on
tw
o
datasets:
FERET
and
the
Labeled
F
aces
in
the
W
ild
(LFW).
(g)
Liu
et
al.
[48]
pro
vided
a
systematic
re
vie
w
of
LBP
v
ariants
while
re
grouping
them
into
dif
ferent
cat-
e
gories.
40
te
xture
features
including
thirty
tw
o
LBP-lik
e
descriptors
and
eight
non-LBP
methods
are
e
v
aluated
and
compared
on
thirteen
te
xture
datasets.
(h)
Slimani
et
al.
[49]
re
vie
wed
the
performance
of
22
state-of-the-art
LBP-lik
e
descriptors
and
some
of
its
recent
v
ariations
and
pro
vides
a
comparati
v
e
analysis
on
f
acial
e
x
pr
ession
recognition
problem
using
tw
o
benchmark
databases.
It
can
be
infer
red
that
there
is
a
limited
number
of
state-of-the-art
published
w
orks
which
are
de
v
oted
to
surv
e
y
LBP-lik
e
methods
in
te
xture
and
f
ace
recognition
and
in
particular
f
acial
emotion
recognition
which
is
practically
none
xistent.
Note
that,
most
of
these
w
orks
remain
limited
in
terms
of
num
b
e
r
of
LBP-lik
e
de-
scriptors
re
vie
wed
and
tested
datasets,
suf
fer
from
lack
of
recent
LBP
v
ariants
and
some
of
them
do
not
include
e
xperimental
e
v
aluation.
Since
no
broad
assessment
has
been
performed
on
an
incredible
number
of
LBP
v
ari-
ations,
and
considering
recent
rapid
increase
in
the
number
of
publications
on
LBP-lik
e
descriptors,
this
paper
aims
to
pro
vide
such
a
comparati
v
e
study
in
f
acial
emotion
recognition
problem
and
of
fers
a
more
up-to-date
introduction
to
the
area.
F
or
that,
46
recent
state-of-the-art
LBP
v
ariants
are
e
v
aluated
and
compared
o
v
er
four
challenging
representati
v
e
widely-used
f
acial
e
xpression
databases.
The
performance
of
the
best
te
xture
de-
scriptor
on
each
dataset
is
also
composed
to
those
of
state-of-the-art
f
acial
emotion
recognition
systems.
Note
that
for
the
descriptors,
we
utilized
the
original
source
code
if
it
is
freely
accessible;
otherwise
we
ha
v
e
b
uilt
up
our
o
wn
implementation.
The
e
v
aluated
state-of-the-art
te
xture
descriptors
and
their
intended
applications
are
summarized
in
T
able
1.
4.
EXPERIMENT
AL
RESUL
TS
AND
DISCUSSION
In
t
his
section,
the
state-of-the-art
LBP
v
ariants
summarized
in
T
able
1
are
e
xtensi
v
ely
e
v
aluated
and
compared
o
v
er
four
publicly
a
v
ailable
f
acia
l
e
xpression
datasets
(see
section
4.2.).
In
addition,
performance
of
the
best
performing
method
on
each
dataset
has
been
compared
ag
ainst
those
of
recent
state-of-the-art
f
acial
emotion
recognition
systems.
The
follo
wing
subsections
describe:
1)
the
e
xperimental
configuration;
2)
the
datasets
considered
in
the
e
xperiments,
3)
the
obtained
results
and
4)
comparisons
with
other
e
xisting
approaches.
4.1.
Experimental
configuration
In
order
to
systematical
ly
e
v
aluate
the
performance
of
the
tested
m
ethods,
we
setup
a
comparati
v
e
analysis
through
a
supervised
image
classification
task.
Similar
to
most
state-of-the-art
f
acial
e
xpression
recog-
nition
systems,
the
adopted
system,
sho
wn
in
Figure
1,
in
v
olv
es
se
v
eral
s
teps
including
1)
image
processing
to
alter
and
resize
f
aces
to
ha
v
e
a
common
resolution;
2)
feature
e
xtraction
using
the
e
v
aluated
LBP
v
ari-
ants;
3)
histogram
v
ector
calculation.
In
this
step,
in
order
to
incorporate
more
spatial
information
into
the
final
feature
v
ectors,
the
obtained
feature
images
were
spatially
di
vided
into
multiple
non-o
v
erlapping
re
gions
and
histograms
were
e
xtracted
from
each
re
gion.
F
or
e
xample,
the
LBP
code
map
is
di
vided
into
m
n
non-
o
v
erlapping
sub-re
gions,
from
each
of
which
a
sub-histogram
feature
is
e
xtracted
and
is
normalized
to
sum
Local
featur
e
e
xtr
action
based
facial...
(Slimani
khadija)
Evaluation Warning : The document was created with Spire.PDF for Python.
4084
r
ISSN:
2088-8708
one.
By
concatenating
these
re
gional
sub-histograms
into
a
single
v
ector
,
a
final
LBP
based
f
ac
ial
emotion
representation
is
obtained;
and
4)
image
class
ification
using
the
SVM
classifier
.
In
this
step,
the
images
of
each
dataset
are
preliminarily
di
vided
into
a
random
split
containing
tw
o
sub-sets,
one
for
the
training
and
the
other
for
testing.
In
the
e
xperiments,
we
tackled
the
7-e
xpression
classification
problems
and
o
v
erall
results
are
computed
as
the
a
v
erage
of
the
per
-class
accuracies
and
not
the
a
v
erage
accurac
y
of
all
samples,
which
a
v
oids
biasing
to
w
ard
e
xpressions
with
more
samples
in
the
databases.
Figure
1.
Outline
of
the
adopted
f
acial
emotion
recognition
system.
4.2.
T
ested
datasets
In
our
e
xperiments,
we
used
four
benchmark
databases;
the
Cohn
Kanade
(CK),
the
Japanese
Fe
male
F
acial
Expression
(J
AFFE),
the
Karolinska
Directed
Emotional
F
aces
(KDEF)
and
the
Multimedia
Understand-
ing
Group
(MUG)
databases.
The
main
characteristics
of
each
database
are
described
herein
belo
w
.
The
four
datasets
include
f
acial
e
xpressions
of
six
basic
emotions;
Anger
,
Disgust,
Fear
,
Happiness,
Sadness,
Surprise
and
the
neutral
f
acial
e
xpression.
(a)
The
J
AFFE
database
[50]
contains
213
f
acial
e
xpression
images
from
10
Japanese
females
where
e
v
ery
subject
e
xpres
ses
three
times
the
se
v
en
f
acial
e
xpressions.
The
images
ha
v
e
a
resoluti
on
of
256x256
pix
els.
(b)
The
CK
database
[51]
includes
2105
digitized
image
sequences
(video)
from
182
adults
ranging
from
18
to
30
years
old.
Each
image
has
a
resolution
of
640x490
pix
els
with
eight-bit
accurac
y
for
gray
scale
v
alues.
(c)
The
KDEF
dataset
[52]
contains
tw
o
sessions
of
multi-vi
e
w
posed
f
acial
e
xpression
images
from
70
am-
ateur
actors,
with
age
ranging
from
20
to
30
years
old.
The
database
has
totally
4900
2D
images
of
se
v
en
human
f
acial
e
xpressions
of
emotions.
The
images
ha
v
e
a
resolution
of
562x762
pix
els,
and
each
of
the
se
v
en
f
acial
e
xpressions
is
acquired
from
fi
v
e
dif
ferent
angles
-90
,
-45
,
0
,
45
,
90
.
(d)
The
MUG
Database
[53]
contains
86
subjects,
where
51
are
males
and
35
are
females.
All
subjects
are
between
20
and
35
years
old.
Only
52
subject
images
are
a
v
ailable
for
usage
with
this
database.
F
or
each
e
xpression,
a
total
of
50
to
160
images
are
e
xisting.
The
images
ha
v
e
a
resolution
of
896x896
pix
els.
4.3.
Results
and
analysis
T
ables
2
and
3
report
the
a
v
erage
accurac
y
of
each
tested
descript
or
obtained
on
CK,
J
AFFE,
KDEF
and
MUG
Databases.
The
first
column
consists
of
the
name
of
the
descriptor
along
with
the
parameter
used
if
that
concerns
a
parametric
descriptor
.
The
other
columns
c
o
nc
ern
the
abbre
viation
of
em
otion
cate
gories
that
we
tested
and
the
accurac
y
obtained;
NE:
NEUTRAL,
HA
:
HAPPY
,
FE
:
FEAR,
SA:
SAD,
AN:
ANGR
Y
,
DI:
DISGUST
,
SU:
SURPRISE,
Acc:
Accurac
y
.
4.3.1.
P
erf
ormance
analysis
on
Cohn-Kanade
(CK)
Database
F
or
this
database,
we
used
a
subset
of
10
sequences
that
reflect
only
the
samples
e
xpressing
the
se
v
en
cate
gories
of
emotions,
and
then
we
selected
the
four
latest
frames
of
each
sequence
that
ha
v
e
the
highest
e
xpression
intensity
.
The
optimal
number
of
non-o
v
erlapping
sub-re
gions
to
compute
the
histogram
features
is
14x14
for
al
l
the
tested
descriptors.
F
or
each
emotion
e
xpression,
tw
o
images
are
used
as
training
set
and
the
tw
o
others
are
used
as
test
set.
T
able
2
illustrates
the
obtained
e
xperimental
results
for
the
basic
emotion
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
4080
–
4092
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
4085
recognition
recorded
on
CK
dataset
using
the
46
e
v
aluated
state-of-the-art
te
xture
descriptors.
It
can
be
inferred
that
almost
all
the
tested
descriptors
produce
good
results
on
CK
dataset
where
their
a
v
erage
accurac
y
is
abo
v
e
96%.
T
weenty-se
v
en
LBP-met
hods
lik
e
RALBGC,
BGC1,
BGC2,
BGC3,
dLBP
,
ELGS
manage
successfully
to
dif
ferentiate
all
classes
perfectly
(a
v
erage
accurac
y
equal
to
100%),
lea
ving
then,
essentially
,
no
room
for
impro
v
ement.
Note
that,
all
the
e
v
aluated
descriptors
reached
a
score
of
100%
for
”Happ
y”
and
”Surprise”
classes.
4.3.2.
P
erf
ormance
analysis
on
J
AFFE
Database
In
this
second
e
xperiment,
each
emotion
in
J
AFFE
database
is
desi
gnated
i
nto
10
femal
es
with
three
samples.
One
image
is
tak
en
for
each
person
and
for
each
emotion
e
xpression
in
the
test,
making
a
total
of
70
samples
in
the
testing
phase
while
the
remaining
140
samples
depict
the
training
set.
All
f
aces
are
preprocessed
to
align
them
into
a
canonical
images
with
a
resolution
of
128x128.
The
histograms
are
produced
on
the
feature
images
spatially
di
vided
into
12x12
non-o
v
erlapping
sub-re
gions.
It
is
apparent
from
T
able
2
that
DSLGS,
ELGS
and
SLGS
operators
yield
the
highest
a
v
erage
rate
as
the
y
reached
a
score
of
98.57%.
Then,
come
the
eight
descriptors:
BGC2,
CSLBP
,
dLBP
,
ILBP
,
LCCMSP
,
LDENP
,
LGCP
and
OS
L
TP
which
reached
a
recognition
rate
of
97.14%.
It
can
be
noticed
that
se
v
eral
tested
LBP-lik
e
descriptors
ha
v
e
perf
ectly
recognized
some
classes
by
getting
the
accurac
y
of
100%.
Note
that
there
is
a
significant
performance
drop
for
all
the
tested
descriptors
on
the
class
of
”sadness”
where
the
reached
accurac
y
is
in
the
range
[50%,
90%].
It
also
emer
ges
from
T
able
2
that
some
methods
lik
e
CSAL
TP
,
GTUC
and
LMEBP
produce
the
w
orst
performance
on
almost
all
the
classes
where
their
accurac
y
is
sometime
belo
w
70%.
W
e
w
ould
also
point
out
that
although
parametric
methods
lik
e
eCS
L
TP
,
IL
TP
,
GTUC,
AEL
TP
are
re
g
arded
as
”optimized”
since
their
parameter
v
alues
are
tuned
during
the
e
xperiment,
their
performance
is
mark
edly
weak
er
than
the
non-parametric
ones.
4.3.3.
P
erf
ormance
analysis
on
KDEF
database
W
e
choose
the
images
of
both
sessions
for
each
subject
and
only
the
vie
w
angle
0
is
considered.
The
subset
contains
70
subjects,
each
one
e
xpresses
tw
o
times
the
se
v
en
emotion
cate
gories.
Thus
,
in
total
we
use
980
images.
W
e
altered
the
sizes
of
all
the
f
aces
of
KDEF
database
into
a
steady
sized
template,
which
ha
v
e
the
same
resolution
of
256x256
and
the
f
aces
were
then
split
into
14x14
blocks
for
re
gion-based
feature
e
xtraction.
Each
subject
e
xpress
tw
o
times
the
se
v
en
cate
gories,
so
we
selected
one
f
acial
image
per
subject
for
training
phase
and
the
other
one
for
test
phase.
It
is
apparent
from
T
able
3
that
the
LGS
operator
is
rank
ed
as
the
top
1
descriptor
in
KDEF
database
as
it
achie
v
es
a
recognition
rate
of
95.92%,
with
perfect
recognition
(100%)
of
happ
y
and
neutral
cate
gories,
follo
wed
by
DSLGS,
SLGS
and
LBP
descriptors
which
reached
a
score
of
95.31%.
Then,
come
se
v
en
de-
scriptors
lik
e
BGC2,
BGC3,
CSLBP
,
dLBP
,
ELGS,
ILBP
and
LQP
A
T
which
allo
wed
to
achie
v
e
accuracies
between
[94.08%
-
94.90%].
Then
tweenty-six
LBP-methods
attained
accuracies
between
[90.20%
-
93.88%]
where
three
descriptors
RLBP
,
BGC1
and
SMEPOP
reached
93.88%
and
tw
o
descriptors
MMEPOP
and
DBC
attained
90.20%
and
90.41%,
respecti
v
ely
.
Accuracies
between
[80.61%
-
86.53%]
were
achie
v
ed
by
eight
LBP-lik
e
me
thods
in
which
80.61%
w
as
achie
v
ed
by
AL
TP
and
86.53%
by
XCS
LBP
.
W
e
can
o
bs
erv
e
from
T
able
3
that
the
w
orst
performance
of
59.39%
w
as
attained
by
CSAL
TP
descriptor
.
4.3.4.
P
erf
ormance
analysis
on
Multimedia
Understanding
Gr
oup
(MUG)
Database
W
e
ha
v
e
used
924
f
acial
e
xpression
im
ages,
i.e.,
132
images
for
each
f
acial
e
xpression.
All
f
a
ces
were
altered
and
resized
to
ha
v
e
a
common
resolution
of
256x256.
Then,
the
y
were
split
into
18x18
blocks
for
re
gion-based
feature
e
xtraction.
F
or
this
e
xperiment,
in
each
emotion
cate
gory
,
we
used
four
images
per
subject,
tw
o
for
training
phase
and
tw
o
for
test
phase.
T
able
3
g
athers
the
obtained
e
xperimental
results.
Clearly
,
it
can
be
observ
ed
that
eight
of
the
test
ed
descriptors
ELGS,
LDTP
,
LDENP
,
LGCP
,
LNDP
,
L
TP
,
LQP
A
T
and
SMEPOP
manage
to
dif
ferentiate
all
classes
perfectly
100%
in
accurac
y
lea
ving
then,
no
room
for
impro
v
ement.
In
addition,
thirty-one
LBP-lik
e
methods
gi
v
e
accuracies
between
[99.03%
-
99.68%],
LBP
attained
98.73%,
DBC
reached
98.05%,
XCS
LBP
got
97.40%
and
finally
,
GTUC
attained
an
accurac
y
of
97.08%.
As
we
can
observ
e,
all
tested
methods
obtain
v
ery
promising
results
on
the
MUG
dataset,
e
xcpect
three
state-of-the-art
methods
AEL
TP
,
LMEBP
and
CSAL
TP
attained
the
lo
west
accuracies
comparing
with
the
other
methods
t
ested.
The
undermost
accurac
y
of
71.43%
w
as
achie
v
ed
by
CSAL
TP
.
Then
an
accurac
y
of
84.09%
w
as
attained
by
AEL
TP
and
finally
89.94%
w
as
obtained
when
testing
LMEBP
method.
Local
featur
e
e
xtr
action
based
facial...
(Slimani
khadija)
Evaluation Warning : The document was created with Spire.PDF for Python.
4086
r
ISSN:
2088-8708
T
able
2.
Experiments
Results
on
CK
and
J
AFFE
Databases
Cohn
Canade
Database
J
AFFE
Database
NE
HA
FE
DI
AN
SA
SU
Acc
NE
HA
FE
DI
AN
SA
SU
Acc
LDTP
100
100
100
95
95
100
100
98.57
90
90
80
90
60
70
100
82.86
RALBGC
100
100
100
100
100
100
100
100
90
100
80
90
80
80
100
88.57
RLBP
100
100
100
100
100
100
100
100
90
90
90
80
90
80
100
88.57
CELDP
100
100
100
100
100
100
100
100
90
80
80
80
100
80
100
87.14
AEL
TP
f
1
g
95
100
100
100
95
95
100
97.86
80
80
90
80
90
80
100
85.71
AL
TP
f
0.006
g
90
100
100
100
95
95
100
97.14
100
100
90
90
100
80
100
94.29
BGC1
100
100
100
100
100
100
100
100
90
90
80
100
100
80
100
91.43
BGC2
100
100
100
100
100
100
100
100
100
100
100
100
100
80
100
97.14
BGC3
100
100
100
100
100
100
100
100
100
90
90
100
100
80
100
94.29
CBP
1
100
100
100
90
100
90
100
97.14
100
90
90
100
100
90
100
95.71
CSAL
TP
f
0.006
g
100
100
100
100
100
95
100
99.29
70
90
80
80
50
60
100
75.71
CSLBP
f
1
g
100
100
100
100
100
100
100
100
100
100
100
100
100
80
100
97.14
CSL
TP
f
1
g
100
100
100
100
100
100
100
100
100
100
90
100
90
80
100
94.29
CSQP
100
100
100
100
100
100
100
100
100
90
100
90
100
80
100
94.29
DBC
f
45
g
100
100
100
95
95
100
100
98.57
100
100
90
90
90
90
100
94.29
DDBP
100
100
100
100
95
100
100
99.29
90
90
100
100
100
80
100
94.29
dLBP
f
45
g
100
100
100
100
100
100
100
100
100
100
100
90
100
90
100
97.14
DSLGS
100
100
100
100
100
100
100
100
100
100
100
100
100
90
100
98.57
eCS
L
TP
f
1
g
100
100
100
90
95
100
100
97.86
100
100
80
90
90
90
100
92.86
ELGS
100
100
100
100
100
100
100
100
100
100
100
100
100
90
100
98.57
GTUC
f
2
g
100
100
95
95
100
100
100
98.57
100
90
60
70
80
50
80
75.71
IBGC1
100
100
100
100
100
100
100
100
90
90
70
90
90
70
100
85.71
ILBP
f
1
g
100
100
100
100
100
100
100
100
100
90
100
100
100
90
100
97.14
IL
TP
f
1
g
95
100
100
100
95
95
100
97.86
90
100
80
90
80
80
100
88.57
LBP
100
100
100
100
95
100
100
99.29
100
100
100
90
90
80
100
94.29
nLBPd
f
1
g
100
100
100
100
100
100
100
100
100
90
80
100
100
80
100
92.86
LCCMSP
100
100
95
90
95
95
100
96.43
100
90
100
100
100
90
100
97.14
LDBP
100
100
100
100
100
100
100
100
100
90
80
100
100
80
100
92.86
LDENP
100
100
100
100
100
100
100
100
100
100
100
100
100
80
100
97.14
LDN
100
100
100
100
100
100
100
100
100
100
90
100
100
80
100
95.71
LDP
f
1
g
100
100
100
100
100
100
100
100
100
100
90
100
100
70
100
94.29
LESTP
10
100
100
100
100
95
100
100
99.29
90
100
90
90
100
80
100
92.86
LGCP
100
100
100
100
100
100
100
100
100
100
100
100
100
80
100
97.14
LGS
100
100
100
100
100
100
100
100
100
100
90
100
100
80
100
95.71
LMEBP
100
100
100
90
95
100
100
97.86
60
90
70
90
50
60
80
71.43
LNDP
100
100
100
100
95
100
100
99.29
90
100
100
100
100
80
100
95.71
L
TP
f
1
g
90
100
100
100
95
95
100
97.14
90
100
90
90
100
80
100
92.86
LQP
A
T
100
100
100
100
100
100
100
100
90
100
100
100
100
80
100
95.71
MMEPOP
100
100
100
100
100
100
100
100
100
100
90
90
100
80
100
94.29
MO
W
SLGS
100
100
100
100
100
100
100
100
100
90
90
100
100
80
100
94.29
OS
L
TP
f
1
g
100
100
100
100
100
100
100
100
100
100
100
100
100
80
100
97.14
QBP
f
1
g
100
100
100
95
100
100
100
99.29
100
100
90
100
100
70
100
94.29
SLGS
100
100
100
100
100
100
100
100
100
100
100
100
100
90
100
98.57
SMEPOP
100
100
100
100
100
100
100
100
90
100
100
90
100
80
100
94.29
STU+
f
1
g
100
100
100
95
100
100
100
99.29
100
100
80
100
100
70
100
92.86
XCS
LBP
100
100
100
100
95
100
100
99.29
90
100
90
70
90
80
100
88.57
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
4080
–
4092
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
4087
T
able
3.
Experiments
Results
on
KDEF
and
MUG
Databases
KDEF
Database
MUG
Database
AN
DI
FE
HA
NE
SA
SU
Acc
AN
DI
FE
HA
NE
SA
SU
Acc
LDTP
82.86
91.43
90
95.71
95.71
91.43
95.71
91.84
100
100
100
100
100
100
100
100
RALBGC
84.29
87.14
91.43
100
97.14
91.43
98.57
92.86
100
100
100
100
100
100
97.73
99.68
RLBP
88.57
92.86
91.43
97.14
98.57
92.86
95.71
93.88
100
100
100
100
97.73
100
100
99.68
CELDP
87.14
87.14
87.14
97.14
94.29
92.86
95.71
91.63
100
100
100
100
97.73
100
100
99.68
AEL
TP
f
10
g
68.57
80
80
95.71
94.29
82.86
87.14
84.08
93.18
65.91
68.18
68.18
100
93.18
100
84.09
AL
TP
f
0.006
g
60
74.29
74.29
97.14
91.43
75.71
91.43
80.61
100
100
97.73
100
100
100
100
99.68
BGC1
85.71
92.86
90
100
98.57
92.86
97.14
93.88
100
97.73
100
100
100
100
97.73
99.35
BGC2
90
91.43
90
100
100
94.29
95.71
94.49
100
95.45
100
100
97.73
100
100
99.03
BGC3
94.29
92.86
90
98.57
97.14
91.43
94.29
94.08
100
100
97.73
100
100
100
100
99.68
CBP
f
10
g
87.14
88.57
88.57
97.14
95.71
88.57
92.86
91.22
100
97.73
97.73
97.73
100
100
100
99.03
CSAL
TP
f
5
g
40
41.43
34.29
65.71
82.86
55.71
95.71
59.39
65.91
68.18
68.18
63.64
65.91
68.18
100
71.43
CSLBP
f
1
g
92.86
91.43
88.57
100
98.57
92.86
94.29
94.08
100
97.73
100
100
100
100
100
99.68
CSL
TP
f
1
g
91.43
88.57
87.14
100
98.57
92.86
94.29
93.27
100
95.45
100
100
100
100
100
99.35
CSQP
90
91.43
88.57
98.57
95.71
92.86
94.29
93.06
100
100
97.73
100
97.73
100
100
99.35
DBC
f
45
g
85.71
87.14
87.14
100
94.29
87.14
91.43
90.41
100
95.45
97.73
97.73
97.73
100
97.73
98.05
DDBP
85.71
90
91.43
98.57
97.14
91.43
95.71
92.86
100
95.45
100
100
97.73
100
100
99.03
dLBP
f
135
g
87.14
90
91.43
98.57
98.57
95.71
97.14
94.08
100
100
100
100
100
100
97.73
99.68
DSLGS
87.14
94.29
92.86
100
98.57
95.71
98.57
95.31
100
97.73
97.73
100
97.73
100
100
99.03
eCS
L
TP
f
1
g
91.43
91.43
91.43
94.29
97.14
88.57
92.86
92.45
100
97.73
100
97.73
100
100
100
99.35
ELGS
85.71
94.29
92.86
100
100
92.86
98.57
94.90
100
100
100
100
100
100
100
100
GTUC
f
1
g
78.57
81.43
87.14
95.71
90
85.71
85.71
86.33
97.73
95.45
95.45
93.18
100
100
97.73
97.08
IBGC1
82.86
88.57
90
100
95.71
91.43
97.14
92.24
100
97.73
97.73
100
100
100
97.73
99.03
ILBP
87.14
94.29
90
100
100
92.86
95.71
94.29
100
95.45
100
100
97.73
100
100
99.03
IL
TP
f
1
g
62.86
75.71
75.71
97.14
90
80
91.43
81.84
100
100
100
100
97.73
100
100
99.68
LBP
88.57
94.29
91.43
100
100
94.29
98.57
95.31
97.73
95.45
97.73
100
97.73
100
100
98.73
nLBP
d
f
1
g
81.43
90
91.43
100
98.57
92.86
95.71
92.86
100
100
100
100
97.73
100
100
99.68
LCCMSP
82.86
87.14
87.14
98.57
97.14
92.86
98.57
92.04
100
100
100
100
97.73
100
100
99.68
LDBP
81.43
88.57
87.14
100
98.57
91.43
98.57
92.24
100
97.73
100
97.73
100
100
97.73
99.03
LDENP
90
90
87.14
100
100
91.43
97.14
93.67
100
100
100
100
100
100
100
100
LDN
87.14
88.57
90
98.57
97.14
92.86
95.71
92.86
97.73
95.45
100
100
100
100
100
99.03
LDP
f
1
g
88.57
90
91.43
97.14
97.14
91.43
94.29
92.86
100
97.73
100
100
100
100
100
99.68
LESTP
f
10
g
64.29
78.57
77.14
97.14
91.43
80
91.43
82.86
100
100
100
100
100
100
97.73
99.68
LGCP
88.57
92.86
84.29
100
100
92.86
95.71
93.47
100
100
100
100
100
100
100
100
LGS
90
95.71
92.86
100
100
94.29
98.57
95.92
100
95.45
100
100
97.73
100
100
99.03
LMEBP
75.71
77.14
90
94.29
84.29
81.43
91.43
84.90
81.82
95.45
88.64
93.18
90.91
90.91
88.64
89.94
LNDP
77.14
87.14
90
100
97.14
91.43
97.14
91.43
100
100
100
100
100
100
100
100
L
TP
f
10
g
65.71
80
77.14
95.71
94.29
81.43
90
83.47
100
100
100
100
100
100
100
100
LQP
A
T
84.29
88.57
95.71
100
98.57
94.29
97.14
94.08
100
100
100
100
100
100
100
100
MMEPOP
74.29
87.14
90
98.57
95.71
91.43
94.29
90.20
100
100
100
100
97.73
100
100
99.68
MO
W
SLGS
84.29
94.29
87.14
100
95.71
94.29
95.71
93.06
100
97.73
97.73
100
100
100
100
99.35
OS
L
TP
f
1
g
91.43
91.43
87.14
100
98.57
91.43
94.29
93.47
100
97.73
100
100
100
100
100
99.68
QBP
f
1
g
91.43
90
88.57
97.14
97.14
92.86
87.14
92.04
100
95.45
100
100
100
100
100
99.35
SLGS
87.14
94.29
92.86
100
98.57
95.71
98.57
95.31
100
97.73
97.73
100
97.73
100
100
99.03
SMEPOP
87.14
94.29
90
100
97.14
94.29
94.29
93.88
100
100
100
100
100
100
100
100
STU+
f
1
g
88.57
88.57
92.86
98.57
98.57
94.29
90
93.06
100
100
100
93.18
100
100
100
99.03
XCS
LBP
82.86
84.29
80
98.57
94.29
78.57
87.14
86.53
100
93.18
95.45
93.18
100
100
100
97.40
Local
featur
e
e
xtr
action
based
facial...
(Slimani
khadija)
Evaluation Warning : The document was created with Spire.PDF for Python.
4088
r
ISSN:
2088-8708
4.4.
Comparison
with
state-of-the-art
methods
In
this
section,
we
compare
the
performance
of
the
best
performi
ng
descriptors
on
each
database
with
those
of
e
xisting
state-of-the-art
methods.
W
e
should
note
that
the
performance
e
v
aluation
with
other
state-
of-the-art
approaches
may
not
be
directly
comparable
due
to
the
dif
ferences
in
partitioning
the
dataset
into
training
and
testing
sets,
number
of
classes,
number
of
subjects
and
features
used.
Ho
we
v
er
,
distincti
v
e
results
of
e
v
ery
approach
still
can
be
indicated.
The
e
xtracted
results
from
the
re
vie
we
d
state-of-the-art
papers
as
well
as
the
recognition
rates
reached
by
the
best
performing
e
v
aluated
LBP-v
ariants
on
each
database
are
arranged
in
T
able
4.
It
can
be
observ
ed
from
T
able
4
that,
e
xcept
for
both
J
AFFE
and
KDEF
databases,
where
t
he
number
of
the
used
samples
is
relati
v
el
y
the
same
for
almost
all
the
e
xisting
systems,
the
used
number
of
samples
on
CK
and
MUG
databases
v
aries
from
one
e
xisti
ng
approach
to
another
.
Gi
v
en
tw
o
dif
ferent
systems
to
compare
on
a
gi
v
en
database,
tw
o
cases
are
possible
to
pro
vide
a
f
air
and
accurate
comparison
of
their
results.
In
the
first
one,
the
used
number
of
samples
and
the
configuration
into
train/test
sets
should
be
the
same,
whereas
in
the
second
case,
the
system
using
a
less
number
of
samples,
must
at
least
be
tested
with
a
delicate
configuration
into
train/test
sets
compared
to
the
other
which
uses
a
higher
number
of
samples.
W
e
used
the
second
case
in
our
e
v
aluation
for
comparing
the
state-of-the-art
methods
with
the
adopted
syst
em,
which
uses
the
most
dif
ficult
configuration
in
terms
of
train/test
sets.
Indeed,
almost
all
the
e
xisting
state-of-the-art
systems
us
e
a
partition
where
the
number
of
training
images
is
superior
to
that
of
test
images
(e.g.,
10-fold),
while
in
this
study
,
the
half-half
configuration
is
adopted.
T
able
4.
Comparison
with
state-of-the-art
methods
Database
Ref
(Y
ear)
Method
Samples
Classifier
(Measure
train-test)
Classes
Accuracy
KDEF
[54]
(2016)
Local
dominant
binary
pattern
1168
SVM
(10-fold)
7
class
83.51
[55]
(2017)
F
acial
landmarks
+
Center
of
Gra
vity
(COG)
980
SVM
(70%-30%)
6
class
90.82
[56]
(2017)
LBP
+
HOG
-
K-means
+
self-or
g
anizing
map
6
class
85.8
[57]
(2017)
Lo
w-Rank
Sparse
Error
dictionary
(LRSE)
980
CRC
(lea
v
e
one-subject-out
10-fold)
7
class
79.39
[58]
(2017)
L
TP+HOG
280
SVM
(10-fold)
7
class
93.34
This
paper
LGS
980
SVM
(half-half)
7
class
95.92
MUG
[59]
(2013)
Local
Fisher
Discriminant
Analysis
567
1NN
(lea
v
e-one-out)
7
class
95.24
[60]
(2014)
ASM
1260
LD
A
(2/3-1/3)
7
class
99.71
[61]
(2015)
Geometric
features
324
SVM
(fi
v
e-fold)
6
class
95.50
[62]
(2017)
MRDTP+GSDRS
567
ELM
(10-fold)
7
class
95.7
[63]
(2017)
GLBP
-
Random
F
orest
(10-fold)
7
class
92.60
This
paper
Se
v
eral
LBP
v
ariants
including
ELGS,
LDTP
,
LDENP
924
SVM
(half-half)
7
class
100
J
AFFE
[59]
(2013)
Local
Fisher
Discriminant
Analysis
213
1NN
(lea
v
e-one-out)
7
class
94.37
[64]
(2016)
Curv
elet
transform
213
OSELM-SC
7
class
94.65
[65]
(2017)
HOG
182
SVM
(70%-30%)
7
class
92.75
[66]
(2017)
DDL
+
CRC
LBP
213
CRC
(10-fold)
7
class
97.3
[62]
(2017)
MRDTP+GSDRS
213
ELM
(10-fold)
7
class
94.3
[67]
(2017)
HOG
+
U-L
TP
213
SVM
(64%-36%)
7
class
97.14
This
paper
DSLGS,
ELGS
and
SLGS
213
SVM
(half-half)
7
class
98.57
Ck
[68]
(2015)
IMF1
+
KLFD
A
404
SVM
(10-fold)
7
class
99.75
[69]
(2015)
LGBP
150
SVM
(57.2%-42.8%)
7
class
97.4
[65]
(2017)
HOG
1478
SVM
(70%-30%)
7
class
98.37
[58]
(2017)
L
TP+HOG
610
SVM
(10-fold)
7
class
96.06
[66]
(2017)
DDL
+
CRC
LBP
-
CRC
(10-fold)
7
class
98.8
This
paper
27
LBP
v
ariants
including
RALBGC,
ELGS,
DSLGS
280
SVM
(half-half)
7
class
100%
Examining
T
able
4,
we
could
mak
e
the
follo
wing
findings
:
(a)
KDEF
database:
It
can
be
easil
y
observ
ed
that
the
LGS
operator
is
the
best
performing
method
which
achie
v
ed
the
higher
performance
o
v
er
the
recent
state-of-the-art
systems
with
a
recognition
rate
reaching
95.92%.
(b)
J
AFFE
database:
It
is
easily
found
that
the
accurac
y
recorded
by
three
LBP-lik
e
v
ariants
outperformed
those
obtained
by
the
state-of-the-art
approaches.
Indeed,
it
emer
ges
from
T
able
4that
the
top
rank
ed
method
on
J
AFFE
database
is
that
presented
in
[66]
as
it
reached
a
score
of
97.3%
which
is
lo
wer
than
that
obtained
by
DSLGS,
ELGS
and
SLGS
operators
(98.57%).
(c)
CK
database:
It
is
apparent
from
T
able
4
that
the
highest
score
achie
v
ed
on
CK
database
is
99.75%
obtained
by
the
method
presented
in
[68]
while
T
able
2indicates
that
27
LBP
v
ariants
reached
a
score
of
100%.
(d)
MUG
database:
As
for
CK
database,
se
v
eral
e
v
aluated
LBP
v
ariants
lik
e
ELGS,
LDTP
,
LDENP
,
LGCP
,
LNDP
LQP
and
SMEPOP
descriptors
reached
a
score
of
100%
outperforming
the
best
performing
state-
of-the-art
approach
presented
in
[60]
which
reached
a
score
of
99.71%.
The
LGS,
DSLGS
and
ELGS
descriptors,
which
are
based
on
the
graph
concept,
manage
to
achie
v
e
remarkable
accuracies
o
v
er
all
the
tested
benchmarks.
This
f
act
is
clearly
highlighted
on
KDEF
e
xperiment
where
we
find
that
fe
w
descriptors
succeeded
to
record
abo
v
e
94%
a
v
erage
accurac
y
.
Then,
the
dominant
graph
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
4080
–
4092
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
4089
encoding
process
justifies
the
rob
ustness
and
ef
fecti
v
eness
of
LGS,
DSLGS
and
ELGS
des
criptors.
On
the
other
hand,
we
remark
that
CSAL
TP
descriptor
suf
fers
on
KEDF
e
xperiment
reaching
just
59.39%
also
on
J
AFFE
and
MUG
e
xperiments,
on
which
the
results
were
v
ery
high
by
the
majority
of
the
tested
descriptors,
the
reason
behind
is
the
user
specified
threshold
used
in
this
operator
,
which
needs
to
be
identified
on
each
e
xperiment
based
on
testing
man
y
v
alues
requiring
man
y
computations.
Rather
than
this,
all
the
other
descriptors
record
good
performances
pro
ving
the
discriminati
v
e
po
wer
of
the
local
description
concept.
5.
CONCLUSION
AND
FUTURE
W
ORKS
W
e
reported
in
this
present
w
ork
a
comprehensi
v
e
comparati
v
e
e
xperimental
analysis
of
a
great
num-
ber
of
recent
state-of-the-art
LBP-lik
e
descriptors
on
f
acial
e
xpression
recognition.
It
is
note
w
orth
y
that
the
choice
of
an
appropriate
descriptor
is
crucial
and
genera
lly
depends
on
the
intended
application
and
man
y
f
ac-
tors,
such
as
computational
ef
ficienc
y
,
discriminati
v
e
po
wer
,
rob
ustness
to
illumination
and
imaging
system
used.
The
e
xperiments
presented
herein
significantly
constitute
a
good
reference
model
when
trying
to
find
an
appropriate
method
for
a
gi
v
en
application.
Our
e
xperiments
on
f
acial
e
xpression
recognition
included
a
detailed
and
comprehensi
v
e
performance
study
of
46
te
xture
descriptors
of
the
literature
co
v
ering
numerous
application
areas
lik
e
te
xture
classifica
tion,
image
retrie
v
al,
finger
v
ein
recognition,
medical
image
analysis,
f
ace
recognition,
f
ace
e
xpression
analysis,
etc.
T
o
sho
w
descriptors
performance
o
v
er
se
v
eral
challenging
situ-
ations,
the
test
ed
descriptors
were
applied
on
four
f
a
mous
and
widely
used
datasets
such
as
J
AFFE,
CK,
KDEF
and
MUG
databases.
The
main
finding
that
can
be
dra
wn
from
the
analysis
of
the
o
v
erall
performance
from
the
e
xperiments
is
that
although
some
LBP-lik
e
features
ha
v
e
been
originally
concei
v
ed
and
proposed
for
te
xture
classification,
the
y
sho
w
considerable
performance
in
f
acial
e
xpression
recognition.
Indeed,
e
v
en
though
the
y
were
not
specifically
designed
for
f
acial
e
xpression
recognition,
some
LBP
v
ariants
outperform
all
state-of-
the-art
approaches
o
v
er
the
tested
databases.
It
is
of
great
importance
to
note
that
the
descriptors
based
on
dominating
set
and
graph
present
a
significant
performance
stability
ag
ainst
the
other
e
v
aluated
state-of-the-art
descriptors
as
the
y
are
often
foun
d
among
the
best
performing
LBP
v
ariants
on
the
four
tested
databases.
F
or
KDEF
database,
LGS
opera
tor
,
which
is
based
on
dominating
set
and
graph
theory
,
is
the
best
performing
de-
scriptor
reaching
a
score
of
95.92%
outperforming
the
recent
state-of-the-art
systems.
F
or
J
AFEE
database,
the
better
recognition
rate
which
w
as
98.57%
has
been
achie
v
ed
by
three
descriptors
based
also
on
dominating
set
and
graph
theory
such
as
DSLGS,
ELGS
and
SLGS.
27
LBP
v
ariants
including
ag
ain
those
based
on
dominat-
ing
set
and
graph
theory
reached
a
score
of
100%
on
CK
database.
Finally
,
man
y
e
v
aluated
LBP
v
ariants
lik
e
LDTP
,
LDENP
,
LGCP
,
LNDP
LQP
and
SMEPOP
descriptors
as
well
as
the
ELGS
operator
reached
a
score
of
100%
o
v
er
MUG
database.
As
future
w
orks,
we
look
forw
ard
to
e
xtend
this
study
to
include
the
e
v
aluation
of
deep
features
and
deep
classifiers.
Furthermore,
we
wish
to
further
e
xplore
the
po
wer
of
te
xture
descriptors
in
other
applications
such
as
compound
emotion
recognit
ion,
gender
classification,
f
ace
recognition,
te
xture
classification,
etc.,
in
order
to
assess
their
ability
to
w
ork
with
v
arious
classification
problems.
A
CKNO
WLEDGMENTS
The
authors
are
thankful
to
the
National
Center
for
Scientific
and
T
echnological
Research
for
funding
this
research
through
the
scholarship
of
e
xcellence
No
757UIT
and
No
7UIT2017
for
the
first
and
second
authors.
Our
w
ork
w
as
also
part
of
the
V
olubilis
AI
33/SI/14
program.
REFERENCES
[1]
S.-J.
W
ang,
W
.-J.
Y
an
et
al.
,
“Micro-e
xpression
recognition
using
rob
ust
principal
component
analysis
and
local
spatiotemporal
directional
features,
”
in
W
orkshop
at
the
Eur
opean
confer
ence
on
computer
vision
.
Springer
,
2014,
pp.
325–338.
[2]
K.
Slimani,
R.
Messoussi
et
al.
,
“
An
i
n
t
elligent
system
solution
for
impro
ving
the
distance
collaborati
v
e
w
ork,
”
in
Intellig
ent
Systems
and
Computer
V
ision
(ISCV)
.
IEEE,
2017,
pp.
1–4.
[3]
S.
Bourekkadi,
S.
Khoulji
et
al.
,
“The
design
of
a
psychotherap
y
remote
intelligent
system,
”
J
ournal
of
Theor
etical
and
Applied
Information
T
ec
hnolo
gy
,
v
ol.
93,
no.
1,
p.
116,
2016.
[4]
T
.
Ojala,
M.
Pietik
¨
ainen
et
al.
,
“Multiresolution
gray-scale
and
rotation
in
v
ariant
t
e
xt
ure
classification
with
local
binary
patterns,
”
IEEE
T
r
ansactions
on
pattern
analysis
and
mac
hine
intellig
ence
,
v
ol.
24,
no.
7,
pp.
971–987,
2002.
Local
featur
e
e
xtr
action
based
facial...
(Slimani
khadija)
Evaluation Warning : The document was created with Spire.PDF for Python.