Indonesi
an
Journa
l
of El
ect
ri
cal Engineer
ing
an
d
Comp
ut
er
Scie
nce
Vo
l.
1
3
,
No.
2
,
Febr
uar
y
201
9
, pp.
6
65
~
670
IS
S
N: 25
02
-
4752, DO
I: 10
.11
591/ijeecs
.v1
3
.i
2
.pp
665
-
670
665
Journ
al h
om
e
page
:
http:
//
ia
es
core.c
om/j
ourn
als/i
ndex.
ph
p/ij
eecs
Multi
mo
dal ve
rge for s
ca
l
e and p
ose v
ar
i
an
t real ti
me face
tracking
and re
cogniti
on
G. Ram
kum
ar
,
E. L
ogasha
nmugam
Depa
rtment
o
f
E
le
c
troni
cs
and
C
om
m
unic
at
ion
E
ngine
er
ing, Sat
h
y
ab
ama
Insti
tute
of
Sci
ence and Tec
hno
log
y
,
I
nd
ia
Art
ic
le
In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
Oct
28
, 201
8
Re
vised
N
ov
2
1
, 2
018
Accepte
d
Dec
3
, 2
018
In
r
ec
en
t
ti
m
es
fa
ce
tracki
ng
a
nd
fa
ce
r
ec
ogn
i
ti
on
have
turned
out
to
be
inc
re
asingly
d
y
n
amic
r
ese
ar
ch
fi
el
d
in
image
pro
ce
ss
ing.
Th
is
wo
rk
proposed
the
fra
m
ework
DEt
ec
t
ing
Conti
guous
O
utl
ie
rs
in
th
e
LOw
-
ran
k
Repre
sent
at
ion
f
or
fa
ce
tracki
ng
,
in
th
is
a
lgor
it
hm
the
ba
ckgr
ound
is
assess
ed
b
y
a
low
-
ran
k
net
work
and
fo
reg
round
art
i
cle
s
ca
n
be
dist
in
guished
as
anomali
es.
Thi
s
is
suita
b
le
for
non
-
rig
id
fore
ground
m
oti
on
and
m
oving
ca
m
era.
Th
e
f
ace
of
a
fore
groun
d
per
son
is
c
aug
ht
from
the
fra
m
e
and
the
n
i
t
is
con
tra
sted
an
d
th
e
spec
ul
ated
pi
ct
ure
s
stored
in
the
da
ta
set
.
Here
we
used
Viola
-
Jones
a
lg
orit
hm
for
fa
ce
rec
ogni
ti
on.
This
appr
oac
h
outp
erf
orm
s
the
tra
ditiona
l
al
gor
it
hm
s
on
m
ulti
m
odal
vid
eo
m
et
hodologies
an
d
i
t
works
ade
qua
te
l
y
on
ex
te
nsive
var
ie
t
y
o
f s
ec
ur
ity
and
surveil
l
ance purpo
ses. Result
s
on
the
continuo
us
demons
tra
te
tha
t
the
propose
d
calc
ul
at
ion
c
a
n
cor
re
ctl
y
obta
in
fa
ci
a
l
f
e
at
ure
s
po
int
s.
T
he
a
lgorithm
is
relega
te
on
the
continuous
ca
m
era i
nput
an
d
under
ong
oing ec
olog
ic
a
l condi
ti
ons.
Ke
yw
or
d
s
:
Bi
om
e
tric
s
Face Re
co
gnit
ion
Face Trac
ki
ng
Im
age A
naly
sis
Si
m
ulati
on
Copyright
©
201
9
Instit
ut
e
o
f Ad
vanc
ed
Engi
n
ee
r
ing
and
S
cienc
e
.
Al
l
rights re
serv
ed
.
Corres
pond
in
g
Aut
h
or
:
G.
Ram
ku
m
ar,
Re
search
Sc
hola
r,
Dep
a
rtm
ent o
f
Elec
tro
nics
and
C
omm
un
i
cat
ion
E
nginee
rin
g,
Sathya
bam
a Insti
tute of
Scie
nc
e an
d
Tec
hnol
og
y,
Jep
piaar
Nag
a
r
, Raj
i
v Gand
hi
Sala
i, Che
nn
ai
–
600119,
Tam
il
nad
u,
India
Em
a
il
: pg
r
vlsi@gm
ai
l.co
m
1.
INTROD
U
CTION
The
m
ajo
r
ta
s
k
is
to
trac
k
and
detect
the
face,
as
c
orr
ect
as
co
nceiv
able
in
e
ve
ry
on
e
of
the
fr
am
es
of
a
vid
eo
.
A
vis
ual
per
s
on
trac
king
ha
s
bee
n
a
s
ta
ndou
t
am
ongst
the
m
os
t
m
a
instream
loo
k
in
the
com
pu
te
r
pe
rc
eption
a
rea
[
1].
Es
pecial
ly
,
hum
an
face
trac
king
over
vid
e
o
receive
m
or
e
co
ns
i
der
at
io
n,
w
hich
would
pe
rm
it
help
fu
l
us
ef
ul
app
li
cat
io
ns
.
I
n
oth
er
case,
f
ace
tracki
ng
is
as
ye
t
a
tric
ky
wh
ic
h
ca
n'
t
be
viewe
d
as
fath
om
ed.
This
is
beca
use
du
e
t
o
the
f
eat
ur
e
of
fac
e
app
ea
ra
nce
c
hange
wh
ic
h
increase
t
he
t
r
ackin
g
com
plexity
[2
]
.
Face
trac
king
is
not
strai
gh
t
fo
r
ward
beca
us
e
m
or
e
va
riat
ion
s
in
t
he
im
age
ap
pea
ran
c
e,
li
ke
po
s
e
var
ia
ti
on
(frontal
a
nd
non
f
r
on
ta
l)
,
im
ped
im
ent,
pict
ur
e
intr
oductio
n,
li
gh
ti
ng
up
conditi
on
a
nd
facial
expressi
on
[
3].
Face
rec
ognit
i
on
play
a
vital
r
ole
in
different
fiel
ds
li
ke
busines
s,
r
est
or
at
ive
or
m
ilit
ary
fr
am
ewo
r
ks
.
F
ace
rec
ogniti
on
a
re
util
iz
ed
as
a
par
t
of
re
al
secur
it
y
or
reli
gious
s
pots
an
d
reg
i
on
s
li
ke
a
i
r
te
rm
inals
and
oth
e
r
ve
ry
deli
cat
e.
Eve
n
th
ough
recog
niti
on
pa
rts
fa
ci
ng
s
om
e
issue
du
e
to
so
m
e
factor
s
,
in
that
po
s
e
var
ia
ti
on
is
one
of
t
he
m
ajor
nuisa
nce
f
act
or
[
4].
How
ever,
t
o
ce
rtai
nl
y
and
ef
fectua
ll
y
util
iz
e
the
m
ul
ti
-
visio
n
vi
deo
in
form
ation
,
we
regularly
nee
d
to
ap
pr
ai
se
t
he
posture
of
the
ind
i
vidual'
s
head
.
Wh
il
e
the
re
ar
e
nu
m
erous
stra
te
gies
f
or
m
ul
ti
view
pose
est
i
m
ation
bu
t
fin
ding
t
he
head
posit
io
n
is
sig
nificant
issue
,
par
ti
cula
rly
when
the
qual
it
y
of
t
he
im
ages
is
poor
a
nd
the
sta
nd
a
rd
iz
at
io
n
of
cam
eras
isn
'
t
adeq
uatel
y
exact
t
o
per
m
it
ro
bust
m
ul
ti
-
vision
f
u
sion
[
5].
S
uch
a
sit
uation
is
pa
rtic
ularly
vali
d
with
re
ga
rd
s
of
sur
veill
ance.
W
e
pro
po
se
the
fac
e tracki
ng and
recog
niti
on
of
per
s
on
from
m
ulti
-
visio
n vide
os
.
The e
ff
ect
ive
a
lgorit
hm
fo
r fa
ce
trackin
g i
s
done
by
DECO
LOR
w
hic
h ap
pro
xim
a
te
s
the
bac
kgr
ou
nd
and
the
fore
groun
d
obj
ect
s
s
i
m
ultaneou
sly
.
Ga
bor
featur
e
extracti
on
is
carried
out
as
the
ne
xt
ste
p
pers
on
identific
at
ion
a
nd
rec
ogniti
on
is
done
by
Viol
a
Jones
Algorit
hm
.
Fo
r
a
gi
ve
n
m
ulti
-
vision
vid
e
o
a
rr
a
ng
e
m
ents,
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
1
3
, N
o.
2
,
Fe
bru
ary
201
9
:
665
–
670
666
we
util
iz
e
a
com
po
sit
e
tem
plate
to
trac
k
t
he
3D
a
rea
of
the
hea
d
util
iz
ing
m
ulti
-
vision
inf
orm
ation.
The
pro
po
se
d
m
et
h
od p
er
form
s
be
tt
er
tha
n
t
he
e
xi
sti
ng
h
ig
hlig
hts
an
d
al
gorith
m
s
on
a
m
ulti
-
visio
n
vid
e
o
da
ta
bas
e
com
po
sed
util
i
zi
ng
a
cam
era
.
Face
trac
king
is
a
c
r
ucial
prec
edin
g
ste
p
t
hat
lim
it
s
the
re
gion
of
the
face
i
n
vide
o
fr
am
es,
from
wh
ic
h
a
a
ppropr
ia
te
featu
re
s
et
can
be
e
xtra
ct
ed
a
nd
c
on
se
qu
e
ntly
sup
plied
as
in
put
to
t
he
fac
e
recog
nizer. As
su
c
h,
t
he
ef
fici
ency o
f
trac
ki
ng
directl
y co
ntr
ols the
am
biti
o
us
t
o
ide
ntify s
ubj
ect
s i
n vide
o.
F
ace
tracki
ng
has
rec
ei
ve
d
s
pecial
at
te
ntion
in
t
he
visi
on
com
m
un
it
y
[6
]
.
Exact
trac
kin
g
is
not
easy
because
of
c
ha
ng
i
ng
ap
peara
nce
of
ta
r
gets
du
e
to
thei
r
non
ri
gid
st
ru
ct
ure,
3D
m
otion
,
interface
with
oth
e
r
obj
ect
s
an
d
ch
ang
e
s
i
n
t
he
e
nv
i
ronm
ent
li
ke
li
gh
ti
ng
va
ri
at
ion
s.
T
his
m
et
hod
detect
s
hu
m
an
face
by
the
geo
m
et
ric
connecti
ons
betwe
en'
s
area
of
fac
e
an
d
hairs
i
n
each
e
dg
e
of
a
vid
e
o
do
c
um
e
nt
[
7].
Desira
bl
e
face
reg
i
on
s
ca
n
be
fig
ur
e
ou
t
us
in
g
the
ra
nge
of
s
kin
c
olo
r
i
n
ord
er
to
at
first
co
nf
i
ne
the
face
,
besides
,
the
plausib
l
e
sq
ua
res
i
n
a
n
im
age
outl
ine
a
re
dicta
te
d
by
m
et
ho
ds
f
or
s
pe
ct
ru
m
s.
Con
s
olidate
d
s
kin
a
nd
squa
res
c
oncl
ud
e
app
li
cant
face
areas
in
li
gh
t
of
the
geo
m
et
ric
connecti
on.
The
sta
ge
connecti
on
m
ov
e
m
ent
est
i
m
at
ion
cal
culat
ion
for
the
m
os
t
par
t
use
d
to
l
ooks
at
the
su
cce
ssive
edg
e
s
in
a
vid
e
o
ar
ra
ng
em
ent
to
gro
up
face
s
that
are
in
m
ov
em
e
nt
an
d
t
rack
the
hum
an
appear
ances
from
the
vid
e
o
record
.
W
it
h
10f
ps
fr
a
m
e
rate,
the
e
ffi
ci
ency
of
sin
gle
-
face
t
rack
i
ng
is
a
ppr
ox
im
at
el
y
cl
os
er
to
10
0%.
Vi
de
o
incl
ud
es
m
or
e
num
ber
of
datas
tha
n
im
a
ges
[8
]
.
An
im
m
ediat
e
m
et
ho
d
to
deal
with
si
ng
le
vie
w
rec
ordin
gs
is
to
e
xp
l
oit
the
inf
or
m
at
ion
e
xc
ess
a
nd
perfor
m
see
determ
inati
on
.
On
e
hypotheti
cal
ly
con
cei
va
ble
a
rr
a
ngem
e
nt
is
to
a
pp
ly
a
bri
ghte
ni
ng
st
and
a
r
dizat
io
n
s
trat
egy
to
restrict
the
l
igh
t
va
riet
ie
s
im
pact
bef
ore
t
rack
i
ng.
In
a
ny
case,
this
isn'
t
a
powe
rful
a
rrang
em
ent
in
li
gh
t
of
the
fact
that
th
e
br
i
gh
te
ning
s
ta
nd
a
rd
iz
at
io
n
cal
culat
ion
s
not
per
f
orm
well
in
low
res
olu
ti
on
face
im
ages.
Mor
e
cal
culat
ion
s
were
c
reated
for
diff
e
re
nt a
pp
li
c
at
ion
s a
nd
util
iz
ed u
ns
ucce
ssf
ully
. In
a
ny cas
e, these
calc
ula
ti
on
s
are
ver
y
t
rou
bl
esom
e
and
di
ff
ic
ult
t
o
face
the
c
onti
nuou
s
prere
quisi
te
s
of
s
pecific
fra
m
e
-
rate.
T
hu
s,
the
pro
po
se
d
ca
n
be
m
os
t
li
kely
trans
planted
t
o
an
em
bedde
d
f
ram
ewo
r
k,
li
ke
the
em
erg
ing
li
tt
le
ro
bot
to
do
dynam
ic
f
ace d
et
ect
ion
a
nd tr
ackin
g.
2.
PROP
OSE
D MET
HO
DOL
OGY
In
t
his
w
ork
w
e
en
deavor
e
d
t
o
co
nsoli
date
t
he
face
t
rack
i
ng
an
d
recog
niti
on.
First
t
he
de
colo
rin
g
is
done
in
that
ba
ckgr
ound
le
a
rni
ng
a
nd
obj
ect
d
et
ect
ion
is
do
ne
sim
ultaneousl
y.
I
n
ge
nuine
recordi
ngs,
the
f
ront
obj
ect
s
ge
ne
rall
y
are
sm
all
pa
ckag
e
s.
I
n
this
m
ann
er
,
neig
hbori
ng
areas
ought
t
o
be
c
hosen
to
be
recog
nized
.
An
im
age
or
a
fr
am
e
is
ca
ptu
r
ed
f
ro
m
a
real
-
tim
e
vid
eo
s
ou
rce
[
9]
.T
he
n
the
face
re
gion
is
detect
ed
a
nd
afte
r
that t
he dete
ct
ed face i
s se
nt for rec
ogniti
on
us
in
g Viola
-
Jones al
gorithm
,
as sho
wn in Fi
gure
1
.
Figure
1
.
Bl
oc
k Diag
ram
o
f
P
rop
os
ed
Syste
m
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Multi
mod
al ve
rg
e f
or
sc
ale
and p
os
e
va
ri
ant real ti
me
f
ace
trackin
g a
nd r
ecognit
ion
(
G.
Ra
mkum
ar
)
667
Ther
e
are
3
ke
y
strides
f
or
m
achine
-
c
on
t
r
olled
vid
e
o
e
xam
inati
on
s
uc
h
as
i
den
ti
fica
ti
on
,
trac
king
and
r
eco
gnit
ion
.
I
n
t
he
i
niti
al
ste
p,
obj
ect
id
entifi
cat
ion
ta
r
gets
t
o
trace
a
nd
sect
i
on
no
te
worthy
quest
io
ns
i
n
a
vid
e
o.
At
that
point,
that
it
em
s
can
be
f
ol
lowe
d
from
edg
e
to
e
dg
e
,
a
nd
the
trac
ks
can
be
prom
pted
t
o
per
cei
ve
obj
ect
Be
ha
vior. A
cc
ordin
gly, o
bjec
t
assum
es
an
i
ndispe
ns
a
ble p
a
rt
in
pract
ic
al
usa
nces
[
10
]
.
O
bj
ect
detect
ion
i
n
a
vid
e
o
is
ge
neral
ly
acco
m
plis
hed
by
ob
j
ect
identifie
r
or
ba
ckg
r
ound
subtr
act
ion
pr
ocedu
res.
I
n
this,
we
te
nd
t
o
dem
on
strat
e
that
the
ab
ove
diff
ic
ulti
es
can
be
te
nd
e
d
t
o
in
a
unifie
d
fr
am
ewo
r
k
w
hi
ch
is
cal
l
it
as
sle
ut
hin
g
Co
ntig
uous
O
utli
ers
i
nto
t
he
lo
w
-
r
ank
il
lustrati
on
(D
EC
OL
OR
).
T
his
c
onstr
uction
coor
din
at
es
obj
ect
identific
at
ion
an
d
backg
round
le
ar
ning
i
nt
o
a
s
olit
ary
procedu
re
of
in
c
orp
or
at
io
n,
a
nd
it
can
natu
rall
y
m
od
e
l com
plica
te
d
backg
rou
nd and a
vo
i
d
the
c
om
pl
ic
at
ed
com
pu
ta
ti
on
of for
egro
und
m
otion
[11].
An
obj
ect
locat
or
is
r
ou
ti
nely
a
cl
assifi
er
that
outp
uts
the
pic
ture
by
a
sli
ding
wind
ow
a
nd
nam
es
each
su
b
pictu
re
por
tray
ed
by
t
he
window
as
ei
th
er
quest
io
n
or
s
ta
rt
of
a
vi
deo.
The
n
agai
n,
ba
ckgr
ound
s
ub
tr
act
ion
syst
e
m
co
ntras
ts t
he
im
ages
with a
fo
undation m
od
el
a
nd fi
nd
s
the
pr
ogre
ssion
s
as
artic
le
s. It
re
gula
rly
exp
ec
t
that
no
obj
ect
s
hows
up
in
im
ages
w
hen
assem
bling
the
bac
kgr
ound
dis
play
.
S
uc
h
a
rr
a
ngem
e
nts
of
fun
dam
ental
s
cases
f
or
obj
e
ct
or
backg
rou
nd
m
od
el
ing
a
bs
ol
utely
re
duce
the
ap
plica
ti
on
of
a
bove
-
noti
ced
te
chn
iq
ues
in
m
echan
iz
ed
vi
deo
exam
inat
i
on.
F
reque
ntly
,
an
ob
j
ect
de
te
ct
or
hav
e
to
nee
d
to
phys
ic
al
ly
qu
al
ifie
d
case
s
f
or
pre
par
e
a
bin
a
ry
cl
assifi
er,
w
hen
backgro
und
s
ubtrac
ti
on
requires
a
prepa
rati
on
groupin
g
that
ha
s
no
obje
ct
s
to
de
velo
p
a
bac
kgr
ound
m
od
el
[12].
To
preset
t
he
e
xplorati
on,
o
bject
identific
at
ion
without
an
in
div
i
du
al
prepa
rin
g
sta
ge
trans
form
into
a
sig
nificant
e
r
rand.
I
nd
i
vidu
al
s
ha
ve
e
nd
ea
vore
d
to
ha
ndle
this
op
e
rati
on
by
ut
il
iz
ing
m
ov
em
ent
data.
E
xam
inati
on
s
on
bo
t
h
sim
ulate
d
in
form
at
ion
a
nd
real
a
rr
a
ng
e
m
ents
dem
on
strat
es
t
hat
D
ECOL
O
R
excee
d
the
be
st
in
cl
ass
te
c
hn
i
qu
e
s
a
nd
it
can
wor
k
producti
vely
on
a
n
ou
t
sco
pe
of com
plex
sit
uations
, as
s
hown in Fi
gure
2
.
Figure
2
.
Dec
ol
or
in
g process
from
the input
vid
e
o
The
ne
xt
ste
p
-
highli
ght
extracti
on
-
i
nclu
des
reali
zi
ng
appr
opriat
e
fa
ci
al
hig
hlig
hts
from
the
inf
or
m
at
ion
.
T
hese h
ig
hlig
hts
co
uld be
s
peci
fic
face
re
gion
s,
var
ie
ti
es,
e
dges,
w
hich
ca
n
be
hum
an
sig
ni
ficant
or
non
si
gn
ific
ant.
T
his
area
ha
s
so
m
e
diff
ere
nt
u
sa
nces
li
ke
facial
featur
e
t
r
ackin
g
or
feeli
ng
rec
ogniti
on
[13].
Finall
y,
the
f
ra
m
ewo
r
k
disti
nguis
hes
the
fac
e.
I
n
an
ac
knowle
dgm
ent
under
ta
king,
the
f
ram
ewo
rk
wou
ld
report
a
integ
rity
fro
m
a
database.
Gabo
r
highli
ghts
extricat
e
l
oc
al
bits of
d
at
a w
hic
h
a
r
e fin
al
ly
m
erg
e
to
ide
ntify
a
n
obj
ect
or local
e of intri
gu
e
[1
4].
The
pri
m
ary
find
i
ng
was
the
dynam
ic
conne
ct
ion
en
gin
ee
ri
ng
wh
ic
h
prese
nted
Gabor
j
et
idea.
A
set
of
Gabor
te
m
plate
s
with
var
i
ous
f
reque
ncies
and
i
ntrod
uctio
ns
m
igh
t
be
us
e
fu
l
f
or
e
xtracts
t
he
es
sentia
l
fe
at
ur
es
from
an
pictur
e.
Util
iz
at
ion
of
Ga
bor
filt
ers
in
feat
ur
e
e
xtr
act
ion
ca
n
be
def
e
nded
by
orga
nic
disc
ove
ries
in
visio
n
fr
am
ew
orks,
c
omm
on
picture
sta
ti
sti
cs
and
ac
hieve
m
ent
in
pr
e
vaili
ng
a
pp
li
cat
io
ns
[15].
Re
fine
m
ent
of
their
dete
rm
in
at
ion
an
d
c
onve
nience
a
dvan
ces
their
use
a
dd
it
io
nally
in
up
c
om
ing
ap
pl
ic
at
ion
s.
Gabo
r
f
il
te
rs
le
ads
a
n
im
po
r
ta
nt
r
ole
i
n
th
e
ap
plica
ti
on
of
com
pu
te
r
visi
on,
m
or
e
pract
ic
al
is
due
t
o
t
heir
su
cce
ss
i
n
face
detect
ion, rec
ogniti
on, a
nd all
the
bio
m
et
ric t
echn
i
qu
e
s. Fea
ture
e
xtracti
on
us
in
g ga
bor
te
m
pla
te
is g
ive
n by
Ψ
(x,
y
)
= f
2
π β
α
e − (
f
2
β
2
x
02
+ f
2
α
2
y
02
)
e
j
2π
f
x
X
0
=
x
c
os
θ +
y si
n
θ
Y
0
= −
x
si
n
θ
+
y cos
θ
wh
e
re
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
1
3
, N
o.
2
,
Fe
bru
ary
201
9
:
665
–
670
668
f
--
ce
ntral fre
quency
of the
tem
pla
te
,
θ
–
De
gr
ee
of t
he rotat
io
n
a
ngle
,
β
–
Ma
jor
ax
is
(b
a
ndwi
dth
)
and
α
–
Mi
nor
ax
is
(sh
a
r
pn
ess
)
Asp
e
ct
rati
o
of
the
ga
us
sia
n
f
un
ct
io
n
is
gi
ven
by
α
/γ
.
F
r
equ
e
ncy
do
m
ai
n
f
unct
ion
f
or
the
giv
e
n
form
is
Ψ(a,
b) = e −
π
2 f
2
(β
2
(aꞌ−
f)
2+
α
2
bꞌ
2
)
aꞌ = a c
os
θ
+
b si
nθ
bꞌ=
−a
sin θ
+
b
c
os
θ
U
ti
li
zi
ng
a
cl
assifi
er,
as
basi
c
as
Ga
us
sia
n
m
ixtur
e
m
od
el
s
in
the
facial
featur
e
m
od
el
s
in
the
facial
com
po
ne
nt
us
e
d
t
o
disti
nguis
h
a
nd
per
cei
ve
com
plex
ge
nuine
or
ol
d
st
ru
c
tures
in
im
ages
.
Face
rec
ognit
ion
is
a
quic
kly
gro
w
ing
up
in
novat
ion
,
gen
e
rall
y
uti
li
zed
as
a
pa
rt
of
crim
inal
reco
gniz
able
pr
oof,
sec
ur
e
d
ac
cess,
and
ja
il
secu
rity
[16].
T
he
m
achine
le
arn
i
ng
and
PC
desi
gns
gro
ups
a
re
li
kew
ise
co
ntin
uous
ly
as
so
ci
at
e
d
with
face
recog
niti
on
.
M
or
e
over
,
t
her
e
are
a
m
ore
num
ber
of
business
,
sec
uri
ti
es
re
qu
iri
ng
t
he
util
iz
at
ion
of
fa
ce
recog
niti
on
te
chnolo
gies.
Fac
e
rec
ogniti
on
has
intri
gu
e
d
m
uch
co
ns
ide
r
at
ion
an
d
it
s
e
xp
l
or
at
io
n
ha
s
rap
i
dly
sp
rea
d o
ut b
y e
ng
i
neer
s
as
we
ll
as n
e
uroscie
ntist
s
The
sam
ple
vi
deo
is
gi
ve
n
as
input.
T
he
m
ajor
ste
p
in
pre
-
processi
ng
is
the
in
put
vi
deo
is
conve
rted
into
fr
am
es.
L
ikewise
t
o
e
nhance
th
e
pictu
re
to
gua
ran
te
e
the
acc
om
pli
sh
m
ent
of
f
ur
t
her
proce
dures
.
(i.e
)
enh
a
ncin
g
c
ontrast
,
e
vacu
at
in
g
no
ise
,
ide
ntif
yi
ng
the
data
richar
eas
[
17
]
.
F
ro
m
the
input
vid
e
o,
at
eac
h
fr
am
e
the
bac
kgrou
nd
va
ries
sli
gh
tl
y.
T
hese
bac
kg
rou
nd
s
a
re
co
nsi
der
e
d
as
n,
n+
1.
.
....s
uch
generate
d
ba
ck
gro
unds
in
each
fr
am
e
is
note
d
a
nd
recor
ded
he
re.T
he
i
m
ages
of
the
pe
rsons
we
are
t
rack
i
ng
will
be
store
d
i
n
the
da
ta
base
.
So
t
hat
the
te
s
t
i
m
age
is
c
om
par
ed
with
t
he
ref
e
ren
c
e
i
m
ages
sto
red
and
the
trac
kin
g
is
done
.
In
Ad
a
ptive
backg
rou
nd
s
ubtract
io
n,
the
backg
rou
nd
of
each
f
ram
es
(i.e)
n,
n+1,...
in
a
vid
e
o
is
su
bt
racted
on
l
y
the
backg
rou
nd
s
a
lon
e
a
re
sep
ar
at
ed
so
t
hat
th
e
per
s
on/o
b
j
e
c
t
can
tracke
d
dow
n
easi
ly
.
It
com
par
es
the
i
m
ages
with
a
bac
kground
dem
on
str
a
te
an
d
ide
ntifie
s
the
a
dju
stm
ents
i
n
obj
ect
[18].
M
orp
ho
l
og
ic
al
filt
erin
g
is
for
enh
a
ncin
g
the
im
age
su
c
h
as
s
m
oo
thing
or
sim
pl
ific
at
ion
,
noise
s
uppressi
on.
Ma
jorly
it
c
ontrib
u
te
s
i
n
re
m
ov
ing
the
a
rtifact
s
(noise)
that
are
i
ntr
oduce
d
w
hile
pr
ocessin
g
th
e
im
age.
T
he
a
ct
ual
im
age
wa
s
init
ia
ll
y
cha
nged
t
o
RGB
-
CbC
rCg
colo
r
s
pace.
At
that
po
i
nt
the
s
kin
el
em
e
nts
we
re
div
i
de
d
in
vie
w
of
the
pros
pecti
ve
s
ki
n
detect
ion
syst
e
m
po
rtrayed
be
foreh
a
nd.
T
he
refor
e
m
orpholo
gical
sifti
ng
w
as
c
onnect
ed
to
dec
rease
false
po
sit
ives
. Atl
ast
the f
ace
det
ect
ion
recog
nized uti
li
zi
ng
Viol
a
-
Jones
f
ace
de
te
ct
or
.
The
propose
d
wor
k
f
or
t
he
face
detect
io
n
are
im
ple
m
ented
by
Ma
tl
ab
s
of
twa
re.
No
te
that
t
he
m
or
p
holo
gical
operat
or
s
we
re
act
ualiz
e
d
util
iz
ing
the
c
apacit
ie
s
(im
e
rode
a
nd
im
fill
)
w
orke
d
in
Im
age
Pr
oc
essin
g
T
oolb
ox,
w
hile
V
iola
-
Jones
al
gorithm
was
giv
e
n
by
C
om
pu
te
r
Visio
n
Syst
em
Too
lb
ox.
F
r
om
the
al
gorithm
we
us
e,
DECOL
O
R
w
her
e
t
he
backg
rou
nd
is
ap
pro
x
im
at
e
by
th
e
l
ow
ra
nk
m
at
rix
[
19
]
.
Th
e
per
s
on/o
bj
ect
f
ro
m
the
i
m
age
is
segm
ented
to
track.
Faci
al
im
age
extracti
on
pr
ov
i
des
the
f
eat
ur
es
of
t
he
tr
acked
per
s
on.
I
f
t
he
i
nput
data
is
to
o
la
r
ge,
the
n
it
c
an
be
it
can
be
change
d
i
nto
a
dim
inished
ar
r
ang
em
ent
o
f
fe
at
ur
e
s
.
Gen
e
rall
y
the
e
xtracted
featu
r
es
c
on
ta
in
a
ppli
cable
in
form
ation
f
ro
m
the
in
pu
t;
with
the
goal
t
hat
the
favor
e
d
un
der
t
akin
g
sho
uld
be
possi
ble
by
util
iz
ing
t
his
r
edu
ce
d
re
prese
ntati
on
rat
her
t
han
the
e
ntire
init
ia
l
data.
T
he
te
st
im
age
is
c
onve
r
te
d
int
o
gr
ay
sc
al
e
in
orde
r
to
process
it
in
thi
s
ste
p.
In
this,
t
he
data
base
is
trai
ned
to
identify
the
per
s
on
we
are
trackin
g
by
pro
vid
i
ng
t
he
im
ages
that
are
store
d
[
20]
.
I
n
kn
ow
le
dg
e
ba
se
th
e
wh
e
re
im
ages
are
st
or
e
d
an
d
w
he
re
we
c
om
par
e
the
te
st
im
a
ge
an
d
t
he
n
t
hey
pro
vide
the
re
su
lt
w
hethe
r
it
m
at
ches
with
t
he
database
im
age
or
not.
Fin
al
ly
the
face
of
the
trac
ke
d
pe
rson
is
rec
ogni
sed
a
nd
prov
i
de
s
the
authe
ntica
ti
on
if
the
im
age
m
at
ches
with
th
e
im
age
puta
w
ay
in
dataset
.
I
f
both
im
ages
are
c
oor
din
at
e
d,
th
e
acce
ss is c
on
ce
ded. Else t
he
a
ccess wil
l be
denied t
o
th
e s
pe
ci
fic p
e
rson.
3.
SIMULATI
O
N RESULTS
AND
PE
RF
O
RMA
NC
E
A
NAYSI
S
The
im
pr
oved
form
i
m
ple
ments
an
ar
rang
e
m
ent
of
c
ha
nnel
s
sel
f
-
com
par
at
ive,
i.e
.
m
easur
e
d
a
nd
tur
ned v
a
riants
o
f
each
o
t
her,
insp
i
te
of
t
he
r
ecurrence
f an
d or
ie
ntati
on
θ
.
G
a
bor feat
ure
s,
al
lu
ded
t
o
as
G
a
bor
j
et
,
m
ulti
-
reso
l
ution
Gabo
r
fe
at
ur
e,
are
de
ve
lop
e
d
from
rea
ct
ion
s
of
Ga
bor
filt
ers
in
th
e
above
co
ndit
io
ns
by
util
iz
ing
num
erous
c
ha
nn
el
s
on
fe
w
fr
e
qu
e
ncies
f
m
an
d
or
ie
ntati
on
s
θ
n.
Crude
featu
re
s
are
t
he
c
omple
x
est
ee
m
ed
react
ion
s
of a
n
a
rr
a
ng
em
ent of m
ulti
d
et
erm
inati
on
Gabo
r
c
hann
el
s as lit
u
p i
n
Figure
3.
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Multi
mod
al ve
rg
e f
or
sc
ale
and p
os
e
va
ri
ant real ti
me
f
ace
trackin
g a
nd r
ecognit
ion
(
G.
Ra
mkum
ar
)
669
Figure
3
.
Ga
bo
r
Feat
ur
e
Extr
a
ct
ion
4.
CONCL
US
I
O
N
In
ou
r
wor
k,
we
pro
posed
a
n
a
ppr
oach
f
or
face
rec
ogniti
on
by
integ
rati
ng
gabor
feat
ure
e
xtracti
on
te
chn
iq
ue
with
vi
ola
jo
nes
al
gorithm
.
W
e
a
dd
it
io
nally
pro
po
s
ed
an
al
go
rithm
fo
r
trac
king
t
o
c
on
t
rol
th
e
highli
gh
t
ca
pturin
g
i
n
a
cam
era arran
gem
ent
set
ti
ng
.
T
her
e
i
s m
os
t l
ikely
a
sing
le
f
ace
, or
if the
re
are
different
faces,
the
bi
ggest
will
be
t
he
pr
i
nciple
us
e
r
of
the
c
om
pu
te
r
a
nd
t
he
one
of
i
ntrigue.
T
herefo
re,
we
ca
n
li
m
it
our
detect
ion
proc
ess
t
o
a
si
ng
le
face
a
nd
quit
prepa
rin
g
once
a
sin
gle
face
is
f
oun
d.
We
s
howe
d
the
e
xec
ution
of
our
w
ork
on
a
m
od
eratel
y
uncon
t
ro
ll
ed
m
ult
i
-
visio
n
vid
e
o
database
.
In
Ta
ble
1
,
t
his
exec
ution
ou
t
perfor
m
s
the
tradit
ion
al
alg
ori
thm
s o
n m
ult
i
m
od
al
v
i
deo
m
et
ho
dolo
gies
interm
s o
f
a
cc
ur
acy
,
s
peed, e
ff
ic
ie
ncy a
nd it
works
adequat
el
y o
n extensi
ve varie
ty
o
f
sec
uri
ty
an
d su
r
veill
ance
purp
os
es
.
Table
1.
O
ver
a
ll
Perfor
m
ance
Metho
d
Accurac
y
Sp
eed
Ef
f
icien
cy
LPP
5
6
.1%
5
8
.8%
6
5
.9%
LDA
3
7
.3%
4
0
.6%
4
7
.4%
SH
-
PCA
4
0
.7%
3
9
.3%
5
2
.2%
Prop
o
sed
6
5
.3%
7
9
.2%
8
7
.3%
REFERE
NCE
S
[1]
Feife
i
Zha
ng
et
al
.
P
o
s
e
-
r
o
b
u
s
t
f
e
a
t
u
r
e
l
e
a
r
n
i
n
g
f
o
r
f
a
c
i
a
l
e
x
p
r
e
s
s
i
o
n
r
e
c
o
g
n
i
t
i
o
n
,
F
r
o
n
t
i
e
r
s
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
2016,
Vol
10,
Iss
ue
5,
pp
832
-
84
4.
[2]
G.Ra
m
kum
ar,
E
.
Loga
shanm
uga
m
“
Hy
br
id
Fra
m
ework
for
detec
t
ion
of
hum
a
n
face
b
ase
d
on
haa
r
-
li
ke
fe
at
ur
e
”
Inte
rnati
onal
Jo
urnal
of Engi
ne
e
ring
&
Technol
ogy
,
7
(3)
2018,
1
786
-
1790
[3]
X.
Li,
D.
L
i,
Z.
Yang
and
W
.
C
hen,
"A
Pa
tc
h
-
B
ase
d
Sal
ie
nc
y
D
et
e
ct
ion
Method
for
As
sess
ing
t
he
Visual
Priva
c
y
Le
ve
ls of
Obje
cts
in
Photos
,
"
in
I
EE
E
A
ccess
,
201
7,
vo
l. 5, pp
.
243
32
-
24343,
.
doi
:
10.
1109/ACCESS
.
2017.
2767622
.
[4]
Chani
ntorn
Jit
taw
iriy
anukoon
“
Propos
ed
a
lgori
th
m
for
imag
e
class
ifi
cation
using
r
egr
ession
-
base
d
pre
-
proc
essing
a
nd
rec
ogni
ti
on
m
od
el
s”
In
te
rnationa
l
journal
of El
ec
t
rical
and
Computer
Eng
ine
ering
(
IJE
CE)
Vol
9
No.
2
2018.
[5]
C.
A
.
Cornea
nu
,
M.
O.
Sim
ón,
J.
F.
Cohn
and
S.
E.
Guerrero,
"S
urve
y
on
RGB,
3D,
The
rm
al,
and
Mul
ti
m
oda
l
Approac
hes
for
Fac
ial
Expre
ss
ion
R
ec
ogni
ti
on
:
Histor
y
,
Tr
en
ds,
and
Affect
-
Rel
ated
Appli
cations,
"
in
IE
EE
Tr
ansacti
ons on Patt
ern
Analysis and
Mac
h
ine In
te
ll
ige
n
ce
,
2016
,
vol. 38, no. 8, p
p.
1548
-
1568
.
[6]
Agung
Nugroho
et
a
l
”
Com
par
is
on
Anal
y
sis
of
Gait
Cla
ss
ifica
t
ion
For
Hum
an
Mot
ion
Ide
n
ti
fi
catio
n
using
Embedd
ed
Com
pute
r
”
Int
ernati
onal journal of Electric
al
and
Computer
Eng
i
nee
ring (
IJE
C
E)
Vol
8
No.
6
201
8.
[7]
M.
Du,
A.
C
.
Sankar
an
aray
a
nan
and
R
.
Chellap
pa,
"Robus
t
Fa
c
e
R
ec
ogni
ti
on
F
rom
Multi
-
vi
ew
Videos,
"
in
IEEE
Tr
ansacti
ons on Im
age
Proc
essing
,
2014
,
vol
.
23
,
no.
3
,
pp
.
1105
-
1117.
[8]
G.
Ramkum
ar,
E.
Loga
shanm
u
gam,
"A
n
eff
ec
t
ual
f
ace
tr
ac
ki
n
g
base
d
on
tr
an
sform
ed
al
gorit
h
m
using
compos
it
e
m
ask
,"
2016
IE
EE
In
te
rnationa
l
Conf
ere
nce
on
Computati
ona
l
Intelli
g
ence
an
d
Computing
R
ese
arch
(
ICCIC)
,
Chenna
i
,
2016
,
pp.
1
-
5.
.
[9]
Eng
-
jon
et
a
l,
Ro
bust
Facial
Fea
t
ure
Tracki
ng
Us
ing
Shap
e
-
Constrai
ned
Mult
ire
so
lut
ion
-
Sel
ecte
d
Li
ne
ar
Pred
ic
to
r
s
Patt
ern
Anal
y
sis
and
Ma
chi
n
e
In
t
el
li
g
ence,
IEEE T
rans
act
ions
on
2011,
(Volum
e:33 ,
Iss
ue: 9
)
[10]
X.
Chai,
S.
Shan
,
X.
Chen
,
W
.
Ga
o,
“
Loc
a
lly
li
ne
a
r
reg
ression
forp
ose
-
inva
ri
ant
f
ace
re
cogni
t
ion,
”
I
EE
E
Tr
ans.
Ima
ge
Proce
ss
.
,
2007,
vol.
16
,
no
.
7
,
pp
.
1716
–
1725
.
[11]
Fabi
an
Parsia
G
eorge
et
al
“
Recogniti
on
of
emot
iona
l
st
at
es
usin
g
EE
G
signa
ls
ba
sed
on
ti
m
e
-
fre
q
uency
an
aly
sis
a
nd
SV
M c
la
ss
ifi
er”
Inte
rnational
jou
rnal
of El
e
ct
ri
ca
l
and
Comput
er
Engi
ne
ering
(
IJ
ECE
)
Vol
9
No.
2
2018
.
[12]
S.
Li,
X.
Li
u
,
X
.
Ch
ai
,
H.
Zh
an
g,
S.
L
ao,
S.
Shan,
“
Morphable
displa
c
ement
f
ield
base
d
image
m
at
chi
ng
for
f
ace
rec
ogni
ti
on
ac
ro
ss
pose,
” in
Pro
c
.
Eur
.
Conf
.
Co
mput.
V
is.
,
2012
,
pp.
102
–
115.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vo
l.
1
3
, N
o.
2
,
Fe
bru
ary
201
9
:
665
–
670
670
[13]
R.
W
ang,
S.
Sha
n,
X.
Chen
,
G
.
W
en,
“
Manifol
d
-
m
ani
fold
dista
n
ce
w
it
h
app
li
c
ation
to
fa
ce
re
cogni
t
i
on
base
d
on
image
set,
”
in
Proc
.
I
E
EE
Con
f. Comput.
Vi
s.
Pattern
R
ec
ogni
t.
,
2008
,
p
p.
1
–
8.
[14]
G.
Aggarwal
,
A.
K.
Ro
y
-
Cho
wdhur
y
,
R
.
Ch
el
l
appa
,
“
A
s
y
stem
ide
nt
ifi
c
ati
on
appr
oa
ch
fo
r
vide
o
-
bas
ed
f
ac
e
rec
ogni
ti
on,
” in
Proc.
In
t. C
onf
.
Pat
te
rn
Recogni
t
.
,
2004
,
pp.
175
–
178.
[15]
Fan
Ou
e
t
al.,
Evalua
ti
on
an
d
Select
ion
of
Discriminat
ing
Gabor
Fea
ture
s
for
Fa
ce
Re
co
gnit
ion
Autom
at
ic
Ide
nti
f
ic
a
ti
on
Advanc
ed
T
ec
hno
logi
es,
IE
EE
Wo
rkshop
2007
[16]
Kam
ara
ine
n
,
Ma
chi
ne
Vision
&
Patt
ern
Rec
ogn
ition
L
ab
.
,
(
LUT
Kouvola)
,
La
pp
ee
m
ran
ta,
Fin
la
n
d
Gbor
fe
at
ure
i
n
image
an
aly
sis 2
012,
15
-
18
IEEE
2154
-
5111
[17]
G.
Ramkum
ar,
E
.
Loga
sh
anmugam
,
"S
tud
y
on
im
pulsive
assess
m
ent
of
chr
onic
p
a
in
cor
re
lated
ex
p
ressions
in
facia
l
images”
B
iomed
ic
al
Re
search
-
2
018,
Volum
e
29
,
Iss
ue
16
[18]
Y
a
n
m
e
i
W
a
n
g
e
t
a
l
.
,
“
F
a
c
i
a
l
r
e
c
o
g
n
t
i
o
n
b
a
s
e
d
o
n
k
e
r
n
a
l
P
A
C
“
T
h
i
r
d
i
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
r
e
n
c
e
on
Inte
lligent
Ne
two
rks
and
Intelli
g
e
nt
Syst
ems 2010
,
978
-
0
-
7695
-
424
9
-
2/10
I
EEE
[19]
Guan
Chun
Luh.,
Fa
ce
detec
t
ion
using
combinati
on
of
sk
in
co
lor
pixe
l
detec
ti
on
a
nd
viola
-
jone
s
fa
ce
d
etec
tor
,
201
4
978
-
1
-
4799
-
4215
-
2/14/
$31.
00
,
I
EE
E
[20]
Ming
Du,
As
win
C.
S
ank
ara
n
ar
a
y
ana
n
Rama
ch
el
l
appa
.
Robust
fac
e
re
cogni
t
ion
from
Multi
vie
w
Videos
©
2014
,
P
No.
1105
–
1117
Doi
10
.
1109/T
I
P.2014.
2300812
IEE
E
.
BIOGR
AP
HI
ES OF
A
UTH
ORS
G.
R
amkum
ar
is
cur
r
emtl
y
worki
ng
as
an
As
sista
nt
Profess
or
in
J
eppi
a
ar
Maa
m
alan
Engi
n
ee
ring
Coll
ege,
Ch
enn
ai
.
He
h
as
com
ple
t
ed
his
B
ac
h
el
or
Degr
ee
in
El
e
ct
roni
cs
and
Com
m
unic
at
ion
Engi
ne
eri
ng
fro
m
Sath
y
ab
ama
Instit
ute
of
Sci
enc
e
and
Te
ch
nolog
y
in
th
e
y
e
ar
2009
and
Master
’s
in
VLS
I D
esign
from
S
at
h
y
ab
ama Ins
t
itute
of
Scie
n
ce
a
nd
T
ec
hnolog
y
i
n
th
e
y
ea
r
2012
.
He
is
pursuing
h
is
Ph.D
in
Sath
yaba
m
a
Instit
u
te
of
Sci
ence
and
T
ec
hnolog
y
.
H
is
r
ese
arc
h
area
is
image
pr
oc
essin
g.
He
has
Sev
eral
publica
ti
ons i
n
Nati
ona
l
/
Int
ern
at
ion
al
Journa
ls
/Confe
ren
ce
s
Dr.
E
.
Loga
shan
m
ugam,
Dire
c
tor
Adm
ini
strat
i
on,
Depa
rtment
of
E
le
c
troni
cs
and
C
om
m
unic
at
ion,
has
completed
hi
s
Bac
hel
o
r
Degr
ee
in
Elec
troni
cs
and
Com
m
unic
at
ion
Eng
ine
e
ring
from
Madura
i
Kam
ara
ja
r
Univ
ersity
in
th
e
y
e
ar
1991
and
Mast
e
r’s
in
E
lectr
oni
c
s
Engi
n
ee
r
ing
fr
om
MIT
Anna
Univer
sit
y
in
th
e
y
e
ar
2002
.
He
re
ceive
d
his
Ph
.
D
from
Sath
y
a
bama
Insti
tute
o
f
Sci
ence
and
Te
chno
log
y
in
t
he
y
e
ar
2009.
H
e
jo
ine
d
as
a
L
ec
tur
er
in
Sa
th
yaba
m
a
Inst
it
ut
e
of
Scie
n
ce
and
Te
chno
log
y
in
t
he
y
ea
r
1995.
His
rese
arc
h
intere
sts
are
vide
o
signal
proc
essing,
Embedde
d
S
y
stems
,
Im
age
Proce
ss
ing
e
tc.,
He
has
Sever
al
r
ese
arc
h
publ
ications
in
N
at
ion
al
/
Int
ern
ational
Journals
/Confe
r
enc
es
Evaluation Warning : The document was created with Spire.PDF for Python.