Internati
o
nal
Journal of Ele
c
trical
and Computer
Engineering
(IJE
CE)
Vol.
5, No. 6, Decem
ber
2015, pp. 1227~
1
233
I
S
SN
: 208
8-8
7
0
8
1
227
Jo
urn
a
l
h
o
me
pa
ge
: h
ttp
://iaesjo
u
r
na
l.com/
o
n
lin
e/ind
e
x.ph
p
/
IJECE
An Efficient Face Recognition Us
ing SIFT Descriptor in RGB-D
Images
M.
I.
Ouloul*,
Z
.
Moutakki
*, K.
Afdel
*
*,
A.
Am
gh
ar*
* Department of
Ph
y
s
ics, LMTI
,
Ibn Zohr Un
iver
sity
, Morocco
** Departmen
t
o
f
Computer Science,
LabSIV, Ib
n Zohr Univ
ersity
, Morocco
Article Info
A
B
STRAC
T
Article histo
r
y:
Received J
u
l
3, 2015
Rev
i
sed
Au
g
25
, 20
15
Accepted
Sep 13, 2015
Automatic face r
ecognition has known a ve
r
y
important evo
l
utio
n in the las
t
decad
e, du
e to i
t
s huge usage i
n
the secur
i
t
y
s
y
stem
s. Th
e m
o
st of faci
a
l
recognition appr
oaches use 2D image, but
the p
r
oblem is that this ty
p
e
o
f
im
age is v
e
r
y
s
e
nsible
to t
h
e
il
lum
i
nation
and
lighting
ch
ange
s. Another
approach us
es
th
e 3D cam
era an
d s
t
ereo cam
er
as
as
well, but i
t
’s
rarel
y
us
e
d
becaus
e
it requ
ir
es
a rela
tiv
el
y lo
ng proces
s
i
ng duration
.
A new ap
proach ris
e
in this field, which is based on RGB-D
im
ages
produced b
y
Kine
ct, th
is
t
y
p
e
of cameras cost less and it can be us
ed in an
y
environment an
d under an
y
circumstances. I
n
this work we pr
opose a new algorithm that co
mbines the
RGB im
age with Depth m
a
p which is less sensible to i
llum
i
na
t
i
on changes
.
We got
a r
ecogn
ition
rate of
96,
63% in r
a
nk 2
.
Keyword:
Face Recognition
Id
en
tificatio
n
Keypo
in
ts
RGB-Dep
t
h
SIFT
Copyright ©
201
5 Institut
e
o
f
Ad
vanced
Engin
eer
ing and S
c
i
e
nce.
All rights re
se
rve
d
.
Co
rresp
ond
i
ng
Autho
r
:
M
oham
e
d
Im
ad Oul
oul
,
Depa
rt
m
e
nt
of
Phy
s
i
c
s,
Ibn
Zoh
r
Un
iversity,
B.P
8106
, I
b
n
Zoh
r
Un
iv
er
sity,
Ag
ad
i
r
,
M
o
ro
cco.
Em
a
il: m
d
1
.
o
u
lo
u
l
@g
m
a
il.co
m
1.
INTRODUCTION
Faci
al
reco
gni
t
i
on
becom
e
s n
o
wa
day
s
one
o
f
t
h
e m
o
st
use
f
ul
m
e
t
hods
of
i
ndi
vi
d
u
al
i
d
e
n
t
i
f
i
cat
i
on,
d
u
e
to
its ab
ility to
id
en
tify
fro
m
d
i
stan
ce. Th
is techn
o
l
o
g
y
h
a
s a l
o
t of app
licatio
n
s
i
n
sev
e
ral
d
o
m
ain
s
, su
ch
as sen
s
itiv
e lo
catio
n
s
su
rv
eillan
ce (ai
r
po
rt,
ban
k
s...),
access-con
t
ro
l, E-
commerce. Th
ere ar
e two
ap
proach
es:
one
t
h
at
uses
2D
i
m
ages,
an
d
t
h
e
ot
her
t
h
at
uses 3
D
. T
h
e fi
rst
one
gi
ves g
o
o
d
res
u
l
t
s
,
b
u
t
t
h
ey
a
r
e
t
o
o
sen
s
itiv
e to
illu
m
i
n
a
tio
n
ch
an
g
e
s and
to
po
se v
a
riatio
n.
On
t
h
e o
t
h
e
r
h
a
nd
th
e
3D ap
pro
ach
is less u
s
efu
l
because it requires s
o
m
e
sort of s
p
ecial se
nsors a
n
d a
rel
a
tively
long processing du
ration, which lim
i
ts this
app
r
oach
f
r
om
bei
n
g
fu
nct
i
o
n
e
d i
n
real
t
i
m
e’s ap
pl
i
cat
i
ons
.
A new
approa
ch has recentl
y
appea
r
e
d
which
uses
R
G
B-D im
ages (see
figure
1); the
s
e types
of
im
ages were
produced by a
Kinect cam
era, which
was
pr
i
n
ci
pal
l
y
de
vel
ope
d
fo
r sam
p
l
e
usa
g
e i
n
a c
o
m
put
er
gam
e
envi
ro
n
m
ent
[1]
.
The
R
G
B
-
D i
m
age com
b
ines two types of data;
a 2D col
o
r i
m
age capture
d with a
RGB cam
era
and a
de
pth
map captured
by an i
n
frare
d
ca
m
e
ra th
at fu
n
c
tion
i
n
p
a
rallel with
an
in
frared
e
m
it
ter. Du
e t
o
its in
v
a
rian
ce
to
illu
m
i
n
a
tio
n
an
d
ligh
t
n
i
ng
ch
ang
e
s, d
e
p
t
h
m
a
p
can
b
e
u
s
ed
ro
bu
stly in
a n
o
n
-
cont
rol
l
e
d e
n
vi
ro
nm
ent
.
Ho
w
e
ver t
h
e ki
n
ect
cam
e
ra has a
hi
g
h
spee
d
of i
m
age capt
u
ri
n
g
an
d c
o
st
l
e
ss,
whi
c
h
m
a
kes ki
nect
a
n
al
t
e
r
n
at
i
v
e t
o
3
D
se
ns
ors
[
2
]
,
[
3
]
,
see
Tabl
e
1.
In
rece
nt years, num
e
rous m
e
th
ods
on face
recognition
ha
ve a
ppea
r
e
d
. The
work prese
n
ted i
n
[4]
u
s
es R
G
B-D i
m
ag
e produ
ced b
y
Ki
n
ect for
facial recogn
itio
n. Th
e algorith
m
p
r
op
osed in
th
is
wo
rk
cal
cu
lates
ent
r
opy
m
a
p and
vi
sual
sal
i
e
ncy
fr
om
R
G
B-D i
m
age, and
uses H
O
G
[
5
]
(Hi
s
t
o
g
r
am
of Ori
e
nt
al
Gra
d
i
e
nt
s) t
o
extract the
features
from
these im
ages
; the diffe
re
nt features obtained a
r
e
concate
n
ated i
n
a single desc
riptor
wh
ich
is
u
s
ed
t
o
trai
n
a R
D
F
(Ran
dom
Decision
Forest) classifier.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJECE
Vol. 5, No. 6, D
ecem
ber
2015 :
1227 –
1233
1
228
In
[6
], ano
t
h
e
r work
th
at co
nsists o
n
u
s
i
n
g
o
n
l
y th
e d
e
p
t
h
m
a
p
s
fo
r
face recog
n
ition
.
Th
e propo
sed
alg
o
rith
m
is b
a
sed
on
three st
ep
s, it starts
with
th
e seg
m
ent
a
t
i
on
of
t
h
e
fac
e
fr
om
t
h
e de
pt
h m
a
p, aft
e
r t
h
at
t
h
e
face is divide
d into 64 re
gions. The
n
the application of 3DLBP (3D L
o
ca
l
Binary Patterns) allows ext
r
acting
th
e features. Fi
n
a
lly in
th
e th
ird
step
, th
e
face d
e
scri
p
t
or resu
lted
is u
s
ed
to
train
th
e SVM (Supp
ort Vecto
r
Machines
) clas
sifier.
Th
e adv
a
n
t
ag
e th
at lies in
[4] is th
e sim
u
lt
an
eou
s
u
tilizatio
n
o
f
RGB i
m
ag
es and
d
e
p
t
h
m
a
p
fo
r
facial reco
gn
itio
n.
Howev
e
r th
e
HOG (His
to
gram
o
f
Orien
t
al Grad
ien
t
s) d
e
scrip
t
o
r
u
s
ed
i
n
th
is wo
rk
is no
n-
robu
st to
ro
tatio
n
an
d
scale ch
ang
e
s. Th
is
p
r
ob
lem
li
mite
d
th
e ap
p
licatio
n
o
f
t
h
is wo
rk
in
n
o
n
-
con
t
ro
lled
en
v
i
ron
m
en
ts. In
[6
], th
e
u
tilizatio
n
of
d
e
p
t
h
m
a
p
cou
l
d
be in
sufficien
t to
acco
m
p
lish
th
e id
en
tificatio
n in
case of large si
milarity betwe
e
n
faces.
(a)
(b
)
Fi
gu
re
1.
R
G
B
-
D i
m
ages p
r
o
duce
d
by
Ki
ne
ct
(a):
R
G
B
(
b
)
:
Dept
h m
a
p
Tabl
e 1.
C
o
m
p
ari
s
o
n
Of
3
d
S
e
ns
ors [
3
]
Device
Speed (sec)
Charge (s)
Size (inch3)
Price
(USD
)
Acc (
m
m
)
3dM
D
0.
002
10 sec
N/A
> $50k
< 0.
2
M
i
nolta
2.
5
no
1408
> $50k
~ 0.
1
Ar
tec
E
v
a
0.
063
no
160.
3
> $20k
~ 0.
5
3D3 HDI
R1
1.
3
no
N/A
> $10k
> 0.
3
SwissRanger
0.
02
no
17.
53
> $5k
~ 10
DAVID SLS
2.4
no
N/A
> $2k
~ 0.5
Kinect
0.
033
no
41.
25
< $200
1.
5-
50
To
d
e
v
e
lop
a p
e
rform
i
n
g
facial reco
gn
itio
n syste
m
th
at
is ro
bu
st to
illu
min
a
tio
n
an
d
scale ch
an
g
e
s,
we
pr
o
pos
e, i
n
t
h
i
s
pape
r,
a ne
w al
go
ri
t
h
m
based
o
n
t
h
e a
ppl
i
cat
i
o
n
of
S
I
FT
(S
cal
e In
va
ri
ant
Feat
u
r
e
Tran
sf
orm
)
de
scri
pt
o
r
on
R
G
B
-
D i
m
ages pr
od
uce
d
by
Ki
n
ect
. The
rest
of
t
h
e pa
pe
r i
s
o
r
gani
ze
d as
fol
l
ows:
a
prese
n
t
a
t
i
on
o
f
t
h
e pr
o
pose
d
m
e
t
hod i
n
sect
i
on 2, t
h
e se
ct
i
on 3 co
nt
ai
ns an ex
pl
an
at
i
on o
f
t
h
e R
e
sear
c
h
m
e
t
hod,
t
h
e
ex
peri
m
e
nt
resul
t
s an
d a
n
al
y
s
i
s
are i
n
sect
i
o
n
4
,
an
d
we
fi
ni
s
h
wi
t
h
t
h
e
co
ncl
u
si
o
n
i
n
sect
i
o
n
5.
2.
PROP
OSE
D
METHO
D
Th
e
p
r
o
cedure in
vo
lv
ed
in
ou
r al
g
o
rith
m
is
g
e
n
e
rally p
r
esen
ted
in
fiv
e
st
ep
s (Figu
r
e
2
)
:
at first we
b
e
g
i
n
with
face d
e
tection
in
RGB-D im
ag
es. Th
en
co
m
p
u
tin
g th
e
salien
c
y m
a
p
and LTP (Local T
e
rna
r
y
Pat
t
e
rns
)
cor
r
e
s
po
n
d
i
n
g res
p
e
c
t
i
v
el
y
t
o
R
G
B and De
pt
h m
a
p.
After th
at we u
s
e SIFT
d
e
scrip
t
or to
ex
tract th
e
feat
ure
s
det
ect
ed i
n
R
G
B
,
sal
i
e
ncy
M
a
p, an
d de
pt
h m
a
p. T
h
en
we a
ppl
y
K-m
eans al
go
r
i
t
h
m
t
o
norm
a
l
i
ze t
h
e
feature
s
retained i
n
eac
h im
a
g
e. At last
we
concate
n
at
e all features
in a
s
i
ngle
vector
na
med Face
desc
ript
or
wh
ich
will b
e
u
s
ed
in th
e classificatio
n
.
3.
R
E
SEARC
H M
ETHOD
In
t
h
is section
,
we
p
r
esen
t t
h
e essen
tial elemen
ts
t
h
at
are
use
d
t
o
c
o
m
pose
ou
r
pr
o
p
o
s
ed m
e
t
hod
in
trodu
ced in
sectio
n
2
.
3.
1. Scal
e In
v
a
ri
an
tFe
a
t
u
re
T
r
ans
f
o
rm (S
IFT
)
Scal
e In
va
ri
an
t
Feat
ure
Tra
n
sfo
r
m
Descri
pt
or
, p
r
op
ose
d
b
y
Davi
d L
o
we
i
n
[
7
]
,
perm
i
t
s t
h
e l
o
cal
m
a
t
c
hi
ng
bet
w
een di
ffe
rent
i
m
ages by
u
s
i
n
g t
h
e i
nva
ri
ant
s
Key
p
o
i
n
t
s
w
h
i
c
h a
r
e r
o
bust
t
o
scal
e an
d r
o
t
a
t
i
o
n
changes
[8]. T
h
e SIFT
De
scri
ptor’s
calcula
t
i
o
n
co
ul
d
be
ac
com
p
l
i
s
hed i
n
fo
ur
st
eps:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
An
Efficien
t
Face Recogn
itio
n Using
SIFT Descrip
t
o
r
in
RGB-D
Imag
es (
M
. I.
Ou
l
o
u
l
)
1
229
1)
Detectin
g
t
h
e
p
o
t
en
tial Keypo
in
ts i
n
th
e i
m
ag
e
by
usi
n
g t
h
e
di
f
f
e
r
en
ce o
f
Ga
ussi
a
n
(
D
o
G
)
fu
nct
i
o
n prese
n
t
e
d by
(Eq
u
at
i
on (
1
)
)
,
,
,
,
,
,
∗
,
(
1
)
,
,
(
2
)
Where
G (E
quation (2)) is the Gaussian
kernel, k is the scale factor, and
,
is the source im
a
g
e.
2)
Th
e Keypo
in
ts th
at p
r
es
ent a
maxim
u
m
or minimu
m
are stab
le so we
keep them
. The other
points are instable and they’re
rejected.
3)
An
ori
e
nt
at
i
on
and m
a
gni
t
ude i
s
assi
gned t
o
e
ach key
poi
nt
.
4)
Each key
poi
nt
i
s
coded i
n
t
o
a vect
or wi
t
h
a 12
8 di
m
e
nsi
ons whi
c
h i
s
i
nvari
ant
t
o
scal
e, r
o
t
a
t
i
o
n
an
d
illu
m
i
n
a
t
i
o
n
ch
ang
e
s.
Fi
gu
re 2.
The
pr
o
pose
d
al
go
r
i
t
h
m
3.
2.
Face
De
te
ction
in
RGB-
D Im
a
g
es
The face
detec
tion is the first
step of t
h
e Fac
i
al r
ecognition
process. It permits
to localize the face in
an im
age that
can contain se
veral obj
ects. In our
proposed algorit
hm
we
start by detecting t
h
e face in
a RGB
im
age (Figure
3(a
))
using the
Viola & J
one
s
m
e
thod
[9].
After t
h
at we t
h
en
get to
loc
a
lize the face in the
dept
h m
a
p (Fi
g
ure
3
(
b
)
)
by
us
i
ng t
h
e c
o
r
r
es
p
o
n
d
e
n
ce
of
co
o
r
di
nat
e
s bet
w
e
e
n R
G
B
i
m
age an
d
dept
h m
a
p.
(a)
(b
)
Figure
3. Face
detection in
(a): RGB (b):
De
pth m
a
p
3.
3. Sal
i
e
nc
y Ma
p
Sal
i
e
ncy
i
s
a very
im
port
a
nt
t
echni
que i
n
t
h
e com
put
er vi
s
i
on
dom
ai
n. It
i
s
gene
ral
l
y
used i
n
case
o
f
segm
ent
a
t
i
on and
det
ect
i
on p
r
obl
em
s. It
per
m
i
t
s
t
o
l
o
cal
i
z
e t
h
e
m
o
st
im
port
a
nt
regi
on
s i
n
an im
age due t
o
t
h
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJECE
Vol. 5, No. 6, D
ecem
ber
2015 :
1227 –
1233
1
230
co
m
b
in
atio
n
o
f
certain
ch
aracteristics su
ch as co
l
o
r, in
t
e
n
s
i
t
y
and ori
e
nt
at
i
on [
10]
, [1
1]
. The vi
sual
Sal
i
e
ncy
m
a
p (Fi
g
u
r
e
4(
c)),
p
r
o
d
u
ce
d d
u
ri
ng t
h
e a
ppl
i
cat
i
on
of t
h
e sa
liency technique on the R
G
B i
m
age (Fi
g
ure
4(a)),
prese
n
ts a
ne
w and e
fficient
source
of
data that
help
s t
o
i
n
crease
the
differe
n
t
inter-cla
ss bet
w
een images
.
Sal
i
e
ncy
o
r
i
e
nt
SIF
T
descri
pt
or
t
o
det
ect
ne
w
key
p
o
i
n
t
s
(F
i
g
u
r
e
4(
b)
an
d
4(
d)
) i
n
t
h
e
i
m
po
rt
ant
re
gi
o
n
s
o
f
t
h
e
face (m
outh,
nose, eyes).
(a)
(b
)
(c)
(d
)
Figu
re
4.
(a
) R
G
B im
age, (b
)
key
p
o
ints
dete
cted in R
G
B i
m
age, (c
) Salie
ncy
m
a
p,
(d
) new
key
poi
nt
s det
ect
ed
i
n
Sal
i
e
ncy
m
a
p
3.
4. L
o
c
a
l
T
ernar
y P
a
t
t
erns
Ima
g
es
The de
pt
h m
a
p pr
od
uce
d
by
Ki
nect
i
s
com
pose
d
of a
num
ber
of
pi
xel
s
w
h
ere eac
h pi
xel
prese
n
t
s
t
h
e
distance
betwe
e
n the cam
era and a c
o
rres
pondi
ng
point in
the face [1]. T
h
e problem
we confront duri
ng the
appl
i
cat
i
o
n
of
SIFT
de
scri
pt
o
r
on
de
pt
h m
a
p i
s
t
h
at
t
h
e
n
u
m
b
er o
f
key
p
o
i
n
t
s
det
ect
ed
i
s
i
n
su
ffi
ci
e
n
t
fo
r t
h
e
matching bet
w
een im
ages (Figure 5(a)
). T
o
ove
r
com
e
this problem
, we use a pretre
atm
e
nt by the LT
P
d
e
scri
p
t
or pro
p
o
s
ed
b
y
X.Tan an
d B.Tri
g
g
s
in
[12
]
.
By applyin
g
th
e LTP
d
e
scri
p
t
or, th
e
d
e
p
t
h
im
ag
e will b
e
tran
sform
e
d
in
to
two
im
ag
es n
a
m
e
d
th
e p
o
sitiv
e (Fig
u
r
e
5
(
b
)
) and
th
e
n
e
g
a
tiv
e im
ag
e (Figu
r
e
5
(
c)). The
num
ber
of
Ke
y
poi
nt
s
det
ect
ed by
S
I
FT
d
e
scri
pt
o
r
i
n
ea
ch o
f
t
h
e t
w
o
im
ages i
s
m
u
ch i
m
port
a
nt
t
h
an t
h
e
num
ber det
ect
ed
i
n
t
h
e de
pt
h
m
a
p.
(a)
(b
)
(c)
Fi
gu
re 5.
Key
p
oi
nt
s det
ect
ed
i
n
(a):
Dept
h
m
a
p, (
b
):
LT
P+, (c):
LT
P-
3.
5. SIFT
M
a
t
r
i
x
Whe
n
we a
ppl
y the SIFT des
c
riptor on t
h
e image I(
x,y), we detect a certa
in
nu
m
b
er
of
K
e
ypo
in
ts
N
that describe
s the im
age. On
one
hand the num
b
er of
key
p
oi
nt
s de
pe
nds
on t
h
e S
I
FT
pa
ram
e
t
e
rs (num
ber
of
oct
a
ves
,
ed
ge t
h
res
h
ol
d,
ker
n
el
Gaussi
a
n
), a
nd
o
n
t
h
e ot
he
r ha
nd
o
n
t
h
e i
m
age t
y
pe (R
GB
, g
r
ay
-scal
e
,
de
pt
h
m
a
p, bi
na
ry
…
)
. Al
l
key
poi
nt
s are gat
h
ere
d
i
n
a m
a
t
r
i
x
na
m
e
d SIFT m
a
tri
x
, i
n
w
h
i
c
h t
h
e n
u
m
b
er o
f
c
o
l
u
m
n
s
i
s
set
on 12
8, a
nd t
h
e
num
ber of l
i
n
es eq
ual
s
N. A
f
t
e
r
th
at the K-m
ean
s alg
o
r
ith
m
tran
sform
s
th
e SIFT
matrix
of R
G
B
,
Sal
i
e
ncy
m
a
p and
LTB
im
ages i
n
t
o
vect
o
r
s. T
h
e
s
e vectors are
then concat
e
n
a
t
ed in a single
vector
wh
ich
will b
e
u
s
ed
in th
e classificatio
n
.
4.
EX
PER
I
M
E
NTA
L
R
E
SU
LTS AN
D ANALY
S
IS
4.
1. D
a
t
a
b
a
se
To e
v
aluate t
h
e perform
ance
of
our algorithm
we us
e t
h
e
E
U
R
E
C
O
M
dat
a
base.
It
i
s
c
o
m
posed o
f
5
2
sub
j
ect
s:
3
8
m
a
l
e
s and
1
4
fem
a
l
e
s from
di
f
f
ere
n
t
et
h
n
i
c
gr
ou
ps
.
D
a
t
a
base i
m
ages are
t
a
ke
n
fr
om
t
w
o
separate
d sessions. Each se
ssion c
ontains
faces of different expr
essi
ons and positioning (see Fi
gure 6)
(Ne
u
t
r
al
, l
i
g
ht
ni
n
g
c
h
an
ges,
sm
i
l
i
ng , o
p
e
n
ed m
out
h,
occl
usi
o
n ey
es,
oc
cl
usi
o
n m
out
h,
occl
usi
o
n
pa
p
e
r, l
e
ft
pr
ofi
l
e
,
ri
g
h
t
pr
ofi
l
e
)
[
13]
.
To
ove
rc
om
e t
h
e pr
o
b
l
e
m
of fac
e
det
ect
i
o
n,
we
o
n
l
y
f
o
cus
o
n
t
h
e fi
r
s
t
f
o
u
r
i
m
ages.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
An
Efficien
t
Face Recogn
itio
n Using
SIFT Descrip
t
o
r
in
RGB-D
Imag
es (
M
. I.
Ou
l
o
u
l
)
1
231
Fig
u
re
6
.
Variatio
n
in expressi
o
n
illu
min
a
tio
n and
p
o
se
4.
2. Resul
t
s a
nd An
al
ysi
s
A faci
al
rec
o
g
n
i
t
i
on sy
st
em
can ha
ve t
w
o
di
ffe
re
nt
o
p
era
t
i
ons:
w
h
et
he
r
i
n
veri
fi
cat
i
on m
ode, i
n
wh
ich
th
e syst
e
m
d
ecid
e
s if
th
e id
en
tity p
r
o
c
lai
m
ed
b
y
th
e p
e
rso
n
tru
e
o
r
false, or in id
en
tificatio
n
m
o
d
e
wh
ere t
h
e syst
e
m
h
a
v
e
t
o
find
th
e p
e
rso
n
’s
id
en
tity with
in th
e ex
isted
i
d
en
tities in
th
e
Datab
a
se [1
4
]
, [15
]
.
Fo
r our system can
b
e
fu
n
c
tion
e
d in
no
n-co
ntro
lled
env
i
ro
nmen
ts, we u
s
e
it in
th
e i
d
en
tificatio
n
m
o
d
e
.
Th
e
pr
opo
sed
alg
o
r
ith
m
in
ou
r wo
rk is
b
a
sed
on
t
h
e
f
e
atu
r
e ex
tr
action f
r
o
m
d
i
f
f
e
r
e
nt typ
e
s
o
f
i
m
ag
es (RGB, Salien
c
y m
a
p
,
LTP im
ag
e). To
ev
alu
a
te t
h
e u
tility o
f
the ex
tracted featu
r
es,
we tested
our
sy
stem
in three diffe
re
nt cases: (1) R
G
B o
n
ly
, (2
) RGB
+ Saliency
m
a
p, (
3
) RGB +
Saliency
m
a
p+ LT
P
im
ages. Each c
a
se prese
n
t
s
t
h
e fu
nct
i
oni
ng
o
f
o
u
r sy
st
em
w
i
t
h
one
or se
ve
ral
t
y
pes of i
m
ages. F
r
om
t
h
e cur
v
e
prese
n
t
e
d
i
n
Fi
gu
re
7,
we
ca
n
anal
y
ze t
h
e
ev
ol
ut
i
o
n
of
ou
r
sy
st
em
depe
nd
i
ng
o
n
t
h
e t
y
pe
s o
f
t
h
e
use
d
i
m
ages.
Th
us, t
h
e
f
unct
i
oni
n
g
of t
h
e s
y
st
em
i
n
case (2) R
G
B
+
Salien
c
y m
a
p
is effectiv
e co
m
p
ared
to
th
e
fu
n
c
ti
o
n
i
n
g
of the sam
e
syste
m
in case (1) RGB only.
Howe
ver,
th
e fu
n
c
tion
i
ng
in
case (3) RGB + Salien
c
y
m
a
p
+
LTP
is the m
o
st effi
cient.
Th
e im
p
r
ov
emen
t of our reco
gn
itio
n system is co
m
p
o
s
ed
of two
step
s. Th
e
first step
con
s
ists i
n
u
s
ing
Salien
c
y
m
a
p
to
d
e
tect a n
e
w k
e
ypo
i
n
ts, th
e in
crease o
f
th
e d
e
tected
k
e
ypo
in
ts
b
y
SIFT invo
lv
es
t
h
e
increase
of inte
r-class di
ffe
rences betwee
n images of
pe
rs
o
n
s d
u
ri
ng t
h
e l
earni
ng
pha
se.
Thi
s
fi
rst
st
ep
al
l
o
ws
us t
o
i
n
cre
a
se
t
h
e rec
o
g
n
i
t
i
o
n
rat
e
by
6% t
o
ran
k
2.
I
n
t
h
e
seco
nd
st
ep,
we a
dd
de
pt
h
im
age. Thi
s
t
y
pe
of
i
m
ag
e is ch
aracterized
b
y
its resistan
ce ag
ain
s
t th
e illu
min
a
tio
n
s
chang
e
s [16
]
. Th
is
featu
r
e
redu
ces th
e in
tra-
class d
i
fferen
c
es b
e
tween
th
e i
m
ag
es o
f
th
e
sam
e
p
e
rson
du
e to
th
e illu
min
a
tio
n
ch
ang
e
s. Th
e add
itio
n o
f
th
e
dept
h im
age in the system
has allowe
d
reachi
n
g
a recognition rate
equal
to 96.63%
to rank
2.
To
v
a
lid
ate th
i
s
work, we
h
a
v
e
co
m
p
ared
t
h
e resu
lts ob
tain
ed
u
s
ing
ou
r ap
pro
a
ch
to
oth
e
r ex
isting
m
e
t
hods:
L
o
c
a
l
bi
nary
pat
t
erns
(LB
P
), L
i
near Di
sc
ri
m
i
nant
Anal
y
s
i
s
(LD
A
) a
nd
Pri
n
ci
pal
C
o
m
p
o
n
e
n
t
Analysis (PC
A
). Based on
the com
p
ar
i
s
o
n
re
sul
t
s
prese
n
t
e
d i
n
Ta
bl
e
2 a
n
d
Fi
g
u
re
8,
we
not
i
ce t
h
at
t
h
e
reco
g
n
i
t
i
on rat
e
i
n
t
h
e
m
e
t
hods (
L
B
P
, L
D
A an
d PC
A)
has n
o
t
excee
ded
90% t
o
r
a
nk
2. H
o
we
v
e
r, t
h
e
reco
g
n
i
t
i
on
ra
t
e
i
n
ou
r m
e
tho
d
has rea
c
h
e
d 9
6
.
6
3% t
o
ran
k
2.
We
can de
d
u
ce t
h
at
ou
r ap
p
r
oa
ch i
s
significa
ntly more
efficient.
12
34
0,
6
0,
7
0,
8
0,
9
1,
0
Recogn
it
i
o
n
Rat
e
Ra
n
k
RG
B
RG
B+ Sa
l
i
e
n
c
y
m
a
p
P
r
op
os
ed
12
3
4
0,
5
0,
6
0,
7
0,
8
0,
9
1,
0
Recogn
it
i
o
n
Rat
e
R
ank
PC
A
LD
A
LB
P
P
r
oposed
Fig
u
re
7
.
Cu
mu
lativ
e Match
Ch
aracteristics curv
e
illu
stratin
g
p
e
rform
a
n
ce o
f
the propo
sed algo
rith
m
wi
t
h
eac
h dat
a
t
y
pe
use
d
Fi
gu
re
8.
C
o
m
p
ari
n
g
t
h
e
res
u
l
t
s
obt
ai
ne
d
usi
n
g
o
u
r
pr
o
pose
d
al
go
r
i
t
h
m
t
o
ot
her e
x
i
s
t
e
d m
e
t
hods
i
n
literatu
re
Tab
l
e
2
.
C
o
m
p
aring
th
e Recog
n
ition
Rate
(%)
o
f
ou
r Meth
od
t
o
o
t
h
e
r existin
g
Metho
d
s
M
e
thod
Rank1
Rank2
Rank3
Rank4
L
B
P
78.
84%
87.
01%
90.
38%
91.
82%
L
D
A
52.
88%
59.
61%
80.
28%
79.
80%
PCA
53.
36%
57.
21%
63.
46%
64.
90%
P
r
oposed
83.
65%
96.
63%
96.
15%
98.
55%
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJECE
Vol. 5, No. 6, D
ecem
ber
2015 :
1227 –
1233
1
232
5.
CO
NCL
USI
O
N
The a
p
proac
h
t
h
at uses t
h
e R
G
B-D im
ages,
produce
d
by
kinect, a
r
e s
u
itable for t
h
e
real tim
e
facial
reco
g
n
i
t
i
on sy
s
t
em
s i
n
n
o
n
-
co
nt
r
o
l
l
e
d e
nvi
ro
nm
ent
s
;
where
l
i
ght
ni
n
g
a
n
d i
l
l
u
m
i
nat
i
on a
r
e
vari
a
n
t
s
.
In this
work we have
propose
d
a ne
w facial
rec
ogn
itio
n
algo
rith
m
th
at u
s
es th
e RGB-D imag
es, It is
base
d on the
extraction and the concatenation of th
e SIFT desc
ript
ors from
these
data sources (RGB,
Saliency m
a
p, LTP im
ages).The
perform
a
nce of
our al
go
rith
m
h
a
s b
e
en
v
a
li
d
a
ted
by testin
g
it with
th
e
EUR
E
C
O
M
da
t
a
base. T
h
e al
g
o
ri
t
h
m
we ha
v
e
pr
o
pose
d
i
n
t
h
i
s
pa
per ca
n
be de
vel
o
pe
d i
n
a f
u
t
u
re w
o
r
k
an
d
mayb
e will b
e
u
s
ed
in case
o
f
i
m
ag
es with occlu
s
ion
an
d po
se
v
a
riation
s
.
ACKNOWLE
DGE
M
ENTS
Th
is work
was su
ppo
rted
b
y
th
e Natio
n
a
l
Cen
t
er
for
Scien
tific and
Techn
i
cal Research (CNRST).
REFERE
NC
ES
[1]
K. Khoshelham, S. Elber
i
nk, "
A
ccuracy
and r
e
solution of k
i
n
ect d
e
pth d
a
ta f
o
r indoor mapping applications,"
Sensors
, vol. 12, no. 2, pp. 1437–
1454, 2012
.
[2]
Anapu
et a
l
., "Fusion of RGB and Depth Images for Robust
Face R
ecognition
using Clos
e-Ran
g
e 3D Camera,"
International Jo
urnal of Inno
vative
Research
in
C
o
mputer
and Co
mmunication
En
gineering
, vol. 2
,
no
. 7
,
2014
.
[3]
Li, B
.
Y
et a
l
., "
A
. Using kinect
for face
recognit
i
on under var
y
i
n
g poses, expr
essi
ons, illum
i
nation
and disguise
," in
IEEE Workshop
Applica
tions o
f
Computer Vision
,
pp
. 186
– 192
.
[4]
G.
Goswami,
et al
.
,
"On RGB-D F
ace Reco
gnition using
Kinect
,"
in IEEE Sixth Interna
tional Conferen
ce
Biometrics: Theory, App
l
ication
s
and Systems 20
13
, pp
. 1
- 6
.
[5]
Dalal N. and
Triggs B., "Histograms of
oriented gradients for human detection
,"
in IEEE C
o
mputer Society
Computer Vision
and Pa
ttern
Recognition C
V
P
R
,
2005, pp
. 886
–
893.
[6]
Neto J. B. C. and Marana, A. N.
, "Face Recogni
t
i
on Using 3DLBP Method A
pplied to Depth Maps Obtained from
Kinect Sensors",
in
x workshop
computer vision
WVC,
2014
. pp
.
168 – 172
.
[7]
David G.
Lowe,
"Dis
tinctiv
e
im
age f
eatur
es
fro
m
s
cale-inv
a
ri
an
t ke
ypoin
t
s
,
"
International
Journal of Computer
Vision,
Vol. 60,
n. 2
,
pp
. 91-110
, 2004.
[8]
Luo J., Ma
Y.,
Takik
a
wa E
.,
La
o S., Kawad
e
M.,
Lu B.
L. "Pers
on-specifi
c SIFT featur
es for fa
c
e
recogn
ition
,
"
in
IEEE Internatio
nal Conf
erence Acoustics,
Sp
eec
h and Signa
l
Processing 2007
, pp
. 1520-6149
.
[9]
Viola P., and Jo
nes M. "Rapid object detection
using a boosted cascade of simple features
,"
in
Pr
oceed
ings
of the
2001 IEEE Com
puter Society C
onference Comp
ut
er Vision
and
Pattern
Recognition CVPR 2001
.
Vol.
1 pp
.
511-
518.
[10]
L. I
tti
, C.
Koch,
E. Ni
ebur,
et
al
., "A M
odel
of
S
a
lien
c
y
B
as
ed
Vis
u
al Atten
tion
for Rapid
S
cen
e Anal
ys
is
",
IEE
E
Transactions on
Pattern
Analys
is and Mach
ine In
telligen
ce
, Vol.
20, n
.
11
, pp
.125
4–1259, 1998
.
[11]
Liu, R.
, Cao, J
.
,
Lin, Z.
, S
h
an, S
., "A
daptive pa
rtial diff
eren
tia
l equat
i
on learn
i
n
g
for visual saliency
d
e
tection", in
IEEE
conf
erence Com
puter Visio
n
and Patt
ern Recognition
2014.
pp. 3866
– 3873
.
[12]
Tan X.,
and Tr
ig
gs B., "Enh
anced local t
e
xtur
e f
eatur
e se
ts for
face r
ecogni
tion u
nder diff
icul
t l
i
g
h
ting cond
itions
,"
IEEE Transactio
ns on Image
Processing
, vo
l. 19,
no. 6
,
pp
. 1635–
1650, 2010
.
[13]
Rui M
i
n,
Nes
l
i
h
an Kos
e
,
J
ean-
L
uc Duge
la
y,
"
K
inectF
a
ceDB:
A Kinect
Dat
a
b
a
s
e
for
F
ace
Re
cognition"
,
IE
EE
Transactions on
system
s man and cyb
ernetics
, vol. 44
, no
.11, pp.1
534-1548, 2014
.
[14]
Anouar Mellak
h
, "Reconnaissance des vi
sages
en conditions
dégradées," Ph
.D. Thesis, Insti
t
ut Nation
a
l des
Télécommunications, France, 20
09.
[15]
S. G. ABABSA,
"Authentificatio
n d’i
ndividus par reconnaissance de car
actéris
tiq
ues biométriques liées aux visag
e
s
2D/3D," Ph.D.
Thesis, Université
Evr
y
V
a
l d
'
Essonne, Fran
ce, 200
8.
[16]
HAN Jungong,
SHAO
Ling, XU Dong,
et al
., “
E
nhanced computer vision with
Mi
crosoft kinect sensor: A review.”
Cybernetics, IEEE
Transactions
on, 2013
, vol. 4
3
, no
5, p. 1318-
1334.
BIOGRAP
HI
ES OF
AUTH
ORS
Ouloul M
o
hame
d Imad
was born in
Taza, Mo
rocco, in
30/07/
1989, he got th
e Master
thesis
Instrumentation
and Telecommunication in
20
13
from
the U
n
ivers
i
t
y
o
f
Ibn
Zohr,
Agadir
,
Morocco. Sin
ce
November 2013, he prep
ared
his
Ph.D thesis on computer vision and embedded
s
y
stems. His main scien
tific in
terests are
face detection/r
ec
ognition and complex
s
y
stems based
on FPGA board. He is curren
t
inter
e
sts lie
i
n
face re
cogni
ti
on using RGB-
Depth im
ages
produced b
y
Kin
ect
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
An
Efficien
t
Face Recogn
itio
n Using
SIFT Descrip
t
o
r
in
RGB-D
Imag
es (
M
. I.
Ou
l
o
u
l
)
1
233
M
o
utakki Zaka
r
i
a
was born in casablan
c
a, Morocco,
in
25/07/1988, he got the master thesis
Instrumentation
and Telecommunication in
20
11
from
the U
n
ivers
i
t
y
o
f
Ibn
Zohr,
Agadir
,
Morocco. Since
October 2011, h
e
prepared his
Ph. D thesis on c
o
mputer vision and embedded
s
y
ste
m
s.
His ma
in inte
re
sts a
r
e
the v
i
deo
s
u
rveill
anc
e
s
y
s
t
em
s
a
pplied on
road safety
, tr
affic
management an
d the
embedded
s
y
stems using FPGA boards. h
e
is
curr
entl
y
in
terest
ed on th
e
optim
izat
ion of
t
h
e de
te
ction
a
c
c
u
rac
y
in ro
ad
tra
ffic surve
ill
anc
e
s
y
stem
s.
Afdel Karim
is
a P
r
ofes
s
o
r in the Com
puter s
c
ie
nce Depar
t
m
e
nt,
F
acult
y of S
c
ien
ce, Univ
ers
i
t
y
Ibn Zohr , Mor
o
cco. He receiv
e
d the Doctorat
(French Ph.D)
in Computer En
gineer
ing and
M
e
dica
l Im
age
P
r
oces
s
i
ng from
the Univ
ers
i
t
y
of
Aix P
r
oven
ce F
r
ance
.
His
areas
of r
e
s
e
arch
inte
res
t
s
in
c
l
ude Im
age
Processing and An
aly
s
is,
computer
Vision,
and
M
e
dica
l Im
age
Amghar Abdellah
is
a P
r
ofes
s
o
r in the
P
h
y
s
i
c
s
Depar
t
m
e
nt,
F
acult
y
of S
c
i
e
nce,
Univers
i
t
y
Ibn Zohr ,Morocco. He received
his DEA and
DE
S degree in 1994 from
Department of Phy
s
ics ,
Faculty
of Scien
ce, University
H
a
ssan II
, Moro
cco. In Janu
ar
y
2002, he has
Ph.D degr
ee
in
microelectronic
from Department of
Phy
s
ics, Faculty
of Scien
c
e, University
Ibn
Zohr ,Morocco
.
His
areas
of
res
earch in
ter
e
s
t
s
include Cr
yptograph
y
, DNT, em
bedded
s
y
s
t
em
s
and
m
i
croele
ctroni
c.
Evaluation Warning : The document was created with Spire.PDF for Python.