Indonesi
an
Journa
l
of El
ect
ri
cal Engineer
ing
an
d
Comp
ut
er
Scie
nce
Vo
l.
9
, No
.
2
,
Febr
ua
ry
201
8
,
pp.
474
~
480
IS
S
N:
25
02
-
4752
,
DOI: 10
.11
591/
ijeecs
.
v9.i
2
.
pp
474
-
480
474
Journ
al h
om
e
page
:
http:
//
ia
es
core.c
om/j
ourn
als/i
ndex.
ph
p/ij
eecs
Visu
al
-
based Fing
erti
p Detec
tio
n fo
r
Hand
Re
ha
bil
itati
on
Day
ang Q
urr
at
u
’
aini
1
, Ali
So
p
hian
2*
,
W
ah
j
u S
e
dion
o
3
, Haz
li
na
M
d
Yu
s
of
4
,
Sud S
udirma
n
5
1, 2, 3, 4
Depa
rtmen
t
of
Me
chatroni
c
s E
ngineeri
ng
,
K
ull
i
yy
ah
of Engi
nee
ring
,
In
te
rn
ational Isla
m
ic Unive
rsit
y
M
al
a
y
si
a,
Jala
n
Gom
bak, 5
3100
Kuala L
um
pur,
Mal
a
y
s
ia
5
School
of
Com
puti
ng
and
Math
emati
c
al Sci
en
ces
,
Li
v
erp
ool
John Moores Univers
ity
,
Li
ve
rpool,
UK
Art
ic
l
e In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
S
ep
11
, 201
7
Re
vised
N
ov
5
, 201
7
Accepte
d
Dec
11
, 201
7
Thi
s
pape
r
pre
se
nts
a
visual
det
e
ct
ion
of
finge
r
tips
by
using
a
class
ifi
catio
n
te
chn
ique
b
ase
d
on
the
b
ag
-
of
-
w
ords
m
et
hod.
In
thi
s
work,
th
e
fi
nger
ti
ps
ar
e
spec
ifica
l
l
y
of
p
eopl
e
who
ar
e
h
oldi
ng
a
th
era
p
y
bal
l
,
as
it
is
intended
to
b
e
used
in
a
hand
reh
abilitati
on
pr
oje
c
t.
Speed
ed
Up
Robust
Feat
ure
s
(SU
RF
)
desc
ript
ors
are
used
to
gen
erate
feature
v
ec
to
rs
and
the
n
th
e
ba
g
-
of
-
fea
tu
r
e
m
odel
is
construc
te
d
b
y
K
-
m
ean
cl
usteri
ng
wh
ic
h
red
u
ce
s
the
num
ber
of
fea
tur
es.
Finall
y,
a
Support
Vec
tor
Mac
hin
e
(SV
M)
is
tra
ine
d
t
o
produc
e
a
cl
assifi
er
that
di
stingui
shes
whet
her
the
f
eature
v
ec
tor
b
el
ongs
to
a
finge
r
ti
p
or
not.
A
tot
al
of
4200
images,
2
100
finge
rti
p
images
a
nd
2100
non
-
finge
rt
ip
images,
were
us
ed
in
th
e
experi
m
ent
.
Our
results
show
tha
t
the
s
ucc
ess
rates
for
the
finge
r
ti
p
det
ection
a
re
hi
gher
tha
n
94%
which
demons
tra
te
s
that
th
e
proposed
m
et
ho
d
produc
es
a
p
rom
ising
result
for
finge
r
ti
p
de
te
c
ti
on
fo
r
the
rap
y
-
b
all
-
ho
l
ding
hands.
Ke
yw
or
d
s
:
Ba
g of
Wor
ds
Fing
e
rtip
detec
ti
on
K
-
m
ean cluster
ing
SU
RF
Copyright
©
201
8
Instit
ut
e
o
f Ad
vanc
ed
Engi
n
ee
r
ing
and
S
cienc
e
.
Al
l
rights re
serv
ed
.
Corres
pond
in
g
Aut
h
or
:
Ali So
phia
n
,
Dep
a
rtm
ent o
f M
echatronic
s
En
gin
eeri
ng,
K
ulli
yy
ah
of E
nginee
rin
g,
In
te
rn
at
io
nal Isl
am
ic
U
ni
ver
sit
y
Ma
la
ysi
a, Jalan
Go
m
bak
,
53
100 K
uala L
um
pu
r,
Mal
ay
sia
Em
a
il
:
ali_s
op
hian
@ii
um
.ed
u.
m
y
1.
INTROD
U
CTION
Me
dical
app
li
cat
ion
s,
incl
ud
i
ng
hand
re
ha
bi
li
ta
t
ion
for
str
ok
e
s
urviv
ors,
hav
e
be
ne
fite
d
from
the
adv
a
nces
i
n
te
chnolo
gy
f
or
m
any
ye
ars.
The
ex
plo
it
at
ion
of
c
om
pu
te
r
vi
sion
in
this
a
ppli
cat
ion
fiel
d
has
no
t
been
s
pa
red
a
nd
has
bee
n
t
he
sub
j
ect
of
m
any
research
work
s
.
Alth
ough
c
om
pu
te
r
visio
n
te
chnol
og
y
ha
s
been
a
dvanci
ng
ra
pid
ly
thr
ough
ou
t
the
ye
a
r
s,
the
re
are
st
il
l
so
m
e
diff
ic
ult
chall
eng
e
s
that
relat
e
to
vi
sion
-
base
d
ap
proac
h
f
or
f
in
ger
ti
p
detect
ion
that need
t
o
be
ove
rco
m
e.
The
ch
al
le
ng
es
that n
eed
to d
eal
with
are (
1)
the
no
n
-
rigid
natu
re
of
hand
s
possessi
ng
a
high
degree
of
f
reedom
that
m
akes
i
t
diff
i
cult
to
m
at
ch
var
i
ou
s
sh
a
pes
of
fi
ng
e
rs
with
a
set
of
i
m
ages,
(2)
t
he
re
is
a
va
riet
y
of
or
ie
ntati
on
and
ap
pea
ran
c
e
of
fin
ger
;
t
hus
it
is
diff
ic
ult
to
det
ect
the
s
hap
e
a
nd
post
ur
e
of
t
he
fin
ger
s
accu
ratel
y
and
r
obust
ly
,
and
(
3)
sli
gh
t
dif
fer
e
nces
m
a
y
le
ad
to
su
bs
ta
ntial
err
or
in
the
case
of
fi
ngerti
ps
that
be
longs
to
the
sam
e
per
son
[1
]
.
These
chall
en
ges
get
even m
or
e sig
ni
ficant w
he
n
c
omm
ercial
v
isi
on syst
em
s ar
e u
se
d,
i
ns
te
ad o
f
th
os
e
of in
du
stria
l gr
a
de.
In
t
his
pa
pe
r,
a
pote
ntial
so
lut
ion
us
in
g
m
a
chine
le
a
rn
i
ng
i
s
to
be
use
d
i
n
hand
reh
a
bili
ta
ti
on
.
O
ne
of
the
wi
dely
pr
a
ct
ic
ed
re
hab
il
it
at
ion
e
xer
ci
se
is
by
aski
ng
the
patie
nt
to
s
qu
eeze
a
fle
xib
le
exe
rcise
ba
ll
in
his/her
ha
nd
s
r
epeti
ti
vely
[2
]
.
The
balls
ha
ve
var
io
us
le
vel
s
of
resist
a
nce
to
accom
m
od
at
e
the
var
io
us
le
vels
of
li
m
i
ta
ti
on
of
t
he
patie
nt
s’
ha
nds.
Howev
e
r,
one
of
the
c
halle
nges
is
to
m
e
asur
e
obj
ect
i
ve
ly
or
qu
a
ntit
at
ively
t
he
pro
gress tha
t has b
ee
n
m
ade if
any. Machi
ne
-
visio
n
-
base
d
syst
em
m
ay
offer
a
non
-
intr
us
iv
e
way
of
m
easurem
ent
of
fin
g
e
rtip
posit
ion.
S
om
e
pr
esent
re
hab
il
it
at
ion
is
assist
ed
by
m
a
chine
vision
ba
sed
syst
e
m
involv
es
the
interact
ion
betwee
n
hu
m
an
and
virtua
l
wo
rl
d.
Dete
ct
ion
an
d
trac
king
of
fin
ger
t
ip
are
essenti
al
in
a
re
cogniti
on
of
f
ing
e
rtip in
a c
onta
ct
le
ss positi
on m
easu
rem
e
nt.
Ther
e
hav
e
be
en
wor
ks
on
fing
e
rtip
detect
ion
usi
ng
m
achine
vision
by
ot
her
resea
rch
e
r
s.
An
en
gi
ne
dev
el
op
m
ent
fo
r
fi
ng
e
rtip
de
te
ct
ion
in
rea
l
-
tim
e
that
is
ta
rg
et
ed
at
m
ob
il
e
de
vices
for
the
Nat
ur
a
l
Use
r
In
te
r
faces
(
N
UI
s
)
[3
]
;
syst
e
m
dev
el
op
m
ent
tha
t
is
ca
pabl
e
of
detect
in
g
fin
ger
ti
p
in
a
reli
able
m
anner
in
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Visua
l
-
base
d
F
ing
ert
ip
D
et
ect
ion
f
or
H
and
R
ehabili
tati
on
(
Da
y
ang Q
ur
r
at
u’aini
)
475
com
plex
env
ir
on
m
ent
under
diff
e
re
nt
li
gh
t
conditi
ons,
dif
fer
e
nt
scenes
without
any
m
ark
e
rs
[
4];
Feng
et
al
.
(20
12)
us
e
d
Kinect
se
ns
or
for
fin
ge
rtip
detect
ion
for
wr
it
in
g
-
in
-
the
-
ai
r
char
act
e
r
r
ecog
niti
on
syst
e
m
;
an
appr
oach
th
at
al
lows
the
detect
ion
of
ha
nd
a
nd
fin
ger
ti
p
wi
th
or
with
ou
t
il
lum
inati
on
in
c
lutt
ered
backgroun
d
[5
]
.
It
s
houl
d
be
note
d
that
no
t
al
l
hand
gest
ure
recog
niti
on
would
re
quire
t
he
determ
inatio
n
of
t
he
posit
ion
of
the f
i
ng
e
rtips.
They m
ay
j
us
t
rely
on th
e
ove
rall
sh
a
pe of
th
e h
a
nd [6].
This
pa
per
is
orga
nized
as
f
ol
lows
.
T
he
relat
ed
w
ork
on
fin
ger
ti
p
detect
ion
s
is
rev
ie
wed
in
Sect
ion
2.
I
n
sect
io
n
3,
t
he
pro
po
s
e
d
al
go
rithm
for
fi
ng
e
rtip
det
ect
ion
t
he
e
xperim
ental
resu
lt
s
are
pr
e
se
nte
d
a
nd
discusse
d. Fi
na
ll
y, the su
m
m
a
ry of the
wo
rk
is pr
e
sente
d
in
the concl
udi
ng secti
on.
2.
FIN
GERTI
P D
ET
ECTI
O
N ALGO
RIT
HM
2.1.
B
ag
of
W
ords
Ba
g
of
wor
ds
(BoW)
m
od
el
has
been
us
e
d
in
m
achine
vis
ion
for
a
rou
nd
a
deca
de.
The
m
od
el
was
or
i
gin
al
ly
app
l
ie
d
in
nat
ur
al
la
ngua
ge
analy
sis
wh
e
re
a
te
xt
do
c
um
ent
is
r
epr
ese
nted
i
n
a
histogram
of
words
without
co
ns
id
erin
g
the
gr
am
m
ar
and
the
orde
r
or
the
loc
at
ion
of
the
w
ords
in
the
te
xt
.
The
m
od
el
would
bu
il
d
a
dicti
on
ary
con
sist
in
g
the
vo
ca
bula
ry
of
w
ords
it
has
found
in
the
te
xts
that
are
fed
into
the
m
od
el
as
the
input.
When
it
com
es
t
o
the
ap
plica
ti
on
in
m
achine
vision,
the
m
od
el
has
bee
n
popu
la
r
due
to
it
s
si
m
plici
t
y
and
eff
ect
ive
ness
[
7]
and
it
is
al
s
o
widely
know
n
as
bag
of
vis
ual
words
and
bag
of
featu
res
.
The
sam
e
researchers
sta
te
d
that
tradit
ion
al
ly
Bo
W
em
plo
ys
scal
e
-
inv
a
riant
f
eat
ur
e
tra
nsfo
rm
(S
IF
T)
de
s
cripto
r
s
that re
du
ces
th
e d
im
ension
al
it
y of t
he feat
ure
sp
ace
.
To
buil
d
the
di
ct
ion
ary,
al
so
known
as
c
odeboo
k,
that
co
ns
ist
s
of
the
vi
su
al
words,
th
e
te
chn
iq
ue
extracts
th
ese
visu
al
w
ords
from
the
trai
ning
im
ages
–
as
il
lustrate
d
by
th
e
flo
wc
har
t
i
n
Figure
1.
D
ur
i
ng
th
e
le
arn
in
g
sta
ge,
a
la
rg
e
set
of
im
ages
of
dif
fere
nt
cl
asses
a
re
us
e
d.
From
each
im
age,
ext
ra
ct
ion
of
key
points
is
init
ia
ll
y
carried
out.
S
ubseq
ue
ntly
,
for
each
keyp
oin
t,
f
eat
ur
e
desc
riptors
are
est
ablishe
d
w
hich
re
pres
ent
th
e
featur
e
s
of
the
neig
hborh
ood
of
the
keyp
oi
nt.
I
n
the
next
ste
p,
f
or
dim
ension
reduc
ti
on
pu
rposes,
thes
e
descr
i
ptors
a
re
cl
us
te
re
d
int
o
gro
up
s
,
wh
ic
h
are
cal
le
d
visua
l
wor
ds
.
All
the
gen
e
rated
vi
su
al
w
ords
fro
m
the
trai
ning
i
m
ages
are
colle
ct
ed
as
the
cod
e
book,
w
hich
is
eq
uiv
al
ent
to
a
di
ct
ion
ary
co
ntainin
g
the
vocab
ulary
of wor
ds
.
Figure
1. Ext
ra
ct
ion
of
Feat
ur
es
an
d
Ge
ner
at
ion
of the
Co
de
book
Durin
g
an
im
a
ge
rec
ogniti
on
sta
ge,
extract
ion
of
key
po
i
nts,
de
fini
ng
f
eat
ur
e
de
script
or
s
a
nd
t
he
cl
us
te
rin
g
of
th
e
descr
i
pto
r
s
are
carrie
d
out
in
ge
ner
at
in
g
th
e
bag
of
wor
ds
for
the
im
age,
wh
ic
h
is
basic
al
ly
a
histo
gr
am
o
f
t
he
v
is
ual wor
ds
that are
present
in
the
im
age,
su
ch
as s
how
n
i
n
Fi
gure
2.
Figure
2 Hist
ogram
o
f
Vis
ual
w
ord
Occ
urre
nces
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vol
.
9
,
No.
2
,
Fe
br
uary
201
8
:
4
7
4
–
4
8
0
476
2.2
.
Speede
d
Up
R
ob
us
t
Fe
at
u
res
(S
U
RF
)
SU
RF
was
intr
oduce
d
by
Ba
y
et
al
[
8]
,
wh
ic
h
has
pro
ven
t
o
be
e
ff
ect
ive
and
po
pu
la
r,
t
hanks
to
it
s
rep
eat
a
bili
ty
,
disti
nctiveness
and
relat
ively
fast
sp
ee
d.
I
n
var
i
ou
s
c
om
par
at
ive
w
orks,
s
uch
as
[9]
,
al
thou
gh
SU
RF
has
lo
w
er
num
ber
of
i
den
ti
fie
d
featu
res
an
d
sli
ghtl
y
lower
nu
m
ber
of
co
rr
ect
m
a
tc
hes
com
par
e
d
to
it
s
pr
e
decess
or,
S
cal
e
-
Inva
riant
Feat
ur
e
T
ra
ns
f
or
m
(S
IF
T
),
it
perform
s
a
hig
her
num
ber
of
correct
m
at
ch
es
pe
r
giv
e
n
ti
m
e
[10]
.
Bot
h
of
th
ese
m
et
ho
ds
a
re
scal
e
i
nv
a
ri
ant
an
d
im
ple
m
entable
in
re
al
-
tim
e
syst
e
ms
[
11
]
.
SU
RF co
ns
ist
s
o
f
fou
r
sta
ges, wh
ic
h
are inte
gr
al
i
m
age g
en
erati
on, approxim
at
ed
Hessian
detect
or, d
es
cripto
r
or
ie
ntati
on
as
sign
m
ent
an
d
de
scripto
r
gen
e
ra
ti
on
[
12]
.
F
or achievin
g
high
sp
ee
d,
f
ollo
wi
ng
it
s
popula
ri
zat
ion
by
Viola
an
d
Jo
nes
[13]
,
this
d
et
ect
ion
us
es
inte
gr
al
im
ages
that
red
uce
t
he
num
ber
of
m
at
hem
at
ic
al
op
e
rati
ons. Th
e integ
ral im
ag
e
Σ
is de
fine
d
m
at
hem
a
ti
cal
l
y as t
he follo
wing
:
Σ
(
,
)
=
∑
∑
(
,
)
=
0
=
0
(1)
Hessian
m
at
rix
is
us
e
d
on
the
integr
al
im
age
for
the
local
iz
a
ti
on
a
nd
scal
in
g
of
i
nterest p
oi
nts,
w
hic
h
par
ti
cula
rly
loo
ks
f
or
blob
-
li
ke
str
uctu
res
w
her
e
t
he
high
de
te
rm
inants
of
the
m
a
trix
are
pr
ese
nt.
T
he
H
essia
n
m
at
rix
H(X, σ)
in an im
age’
s point
X
at
scale
σ is d
e
fine
d
as
fo
ll
ows:
(
,
)
=
[
(
,
)
(
,
)
(
,
)
(
,
)
]
(2)
wh
e
re
(
,
)
is
t
he
conv
olu
ti
on
of
the
Gaussi
an
sec
ond
order
de
rivati
ve
2
2
(
)
w
it
h
th
e
i
m
age I
at
po
i
nt
X
, a
nd sim
il
a
rly
f
or
(
,
)
,
(
,
)
an
d
(
,
)
.
Fo
ll
owin
g
inte
rest
point
de
te
ct
ion
,
S
URF
i
d
entifi
es
an
i
nterest
point
de
sc
riptor
ar
ound
e
ach
intere
st
po
i
nt,
wh
ic
h
i
nc
lud
es
the
dom
inant
or
ie
ntati
on
.
Each
re
gion
ar
ound
the
int
erest
point
is
s
plit
into
s
ubre
gi
on
s
.
Fo
r
eac
h
s
ub
-
r
egio
n,
a
v
ect
or is de
fine
d by
us
in
g Haar
w
a
velet
r
es
pons
es
. T
hese
vecto
rs
for
m
the d
esc
r
iptor.
2.3. K
-
Me
an
s
Clust
eri
n
g
K
-
m
ean
cl
us
te
rin
g
is
one
of
the
m
e
tho
ds
f
or
im
age
seg
m
e
ntati
on
,
w
hich
is
the
cl
assifi
cat
ion
of
a
n
i
m
age
into
dist
inct
gro
up
s
.
Be
fore
ap
plyi
ng
this
unsuper
vi
sed
le
ar
ning
te
chn
i
qu
e
,
an
i
niti
al
enh
ancem
ent
is
a
pp
li
ed
to
the
i
m
age
for
im
a
ge
im
pr
ovem
e
nt.
A
s
ubtract
ive
cl
ust
erin
g
m
et
ho
d
ge
ner
a
te
s
centr
oid
s
wh
ic
h
i
s
base
d
on
the
pote
ntial
value
of
data
points.
In
oth
e
r
w
ord
s,
subtract
ive
c
luster
is
us
e
d
t
o
ge
ne
rate
the
init
ia
l
centers
w
hich
i
s u
se
d
i
n K
-
m
ean alg
ori
t
hm
f
or the
data
po
i
nts [1
4]
In
t
his
w
ork,
it
is
us
ed
to
ass
oc
ia
te
the
gen
e
r
at
ed
desc
ript
or
to
the
ri
ght
cl
us
te
r,
w
hich
is
al
so
kn
own
as
vis
ual
worl
d
i
n
the
bag
-
of
-
w
ords
te
ch
nique.
By
us
i
ng
t
his
cl
ust
erin
g,
the
cl
assifi
cat
ion
sta
ge
,
w
hich
is
t
he
nex
t
ste
p,
will
deal with
lo
we
r data
d
im
ension that, i
n
t
urn, help
s in
g
ai
ning a
highe
r proc
essing sp
eed
.
2.4. Sup
po
r
t Vect
or M
achi
ne (
S
V
M)
SV
M i
s a supe
rv
ise
d
le
ar
ning
m
et
ho
d
that i
s
u
sed for r
e
gre
ssion
a
nd classi
ficat
ion
[15
]
. I
t carries out
cl
assifi
cat
ion
by
creati
ng
a
m
ulti
-
dim
ensional
hyperplane
wh
ic
h
di
vid
es
the
data
into
t
wo
gro
ups
opti
m
al
l
y.
This
m
akes
S
VM
cl
assifi
er
m
od
el
cl
os
el
y
associat
ed
wit
h
neural
netw
orks
.
T
he
SV
M
cl
assifi
er
m
od
e
l
us
es
a
sigm
oid
k
er
nel
fun
ct
io
n, w
hic
h
is si
m
il
ar to
the tw
o
-
la
ye
r
pe
rcep
tr
on
of n
e
ur
al
netw
ork.
3.
E
X
PE
RIME
NT AN
D D
ATA GAT
HE
RING
3.1.
E
xp
eri
me
nt
al
Setup
In
this
work,
a
com
m
ercial
hig
h
-
de
ns
it
y
(HD)
L
ogit
ech
C615
w
ebcam
with
a
res
olu
ti
on
of
1920
x
1080 p
ixels
ha
s b
een used
. An
exam
ple o
f
im
age cap
ture
d by the w
ebcam
is as sh
own
in
Figu
r
e
1
.
Fig
ure
1
is
an
exam
ple
of
an
im
age
of
a
hand
ho
l
ding
a
thera
py
ball.
The
im
ages
are
captu
red
wh
i
le
the
web
cam
fa
ci
ng
upwa
rd
s
wh
ic
h
is
facing
a
li
gh
t
-
em
itti
ng
sou
rce
in
the
cei
lin
g.
The
glare
f
ro
m
the
li
gh
t
s
ource
co
ntribut
es
to
the v
a
riat
ion o
f
intensit
y i
n
ea
ch
ca
ptured
im
age.
The
set
up
f
or
the
data
im
age
gathe
rin
g
is
il
l
us
trat
ed
i
n
Fig
ur
e
4.
T
he
bl
ue
ci
rcles
deno
te
the
posit
ion
of
the
hands
w
her
e
the
distan
ce
betwee
n
ad
ja
cent
blu
e
ci
rc
le
s
is
app
r
ox
i
m
at
ely
10
cm
.
The
distance
Yh
a
nd
denotes
th
e
pe
r
pendicula
r
distance
of
t
he
po
s
it
ion
of
hands
t
o
the
we
bcam
.
The
web
cam
captu
red
t
wo
im
ages
of the
ha
nd at
each
posit
ion
.
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Visua
l
-
base
d
F
ing
ert
ip
D
et
ect
ion
f
or
H
and
R
ehabili
tati
on
(
Da
y
ang Q
ur
r
at
u’aini
)
477
Figure
1
E
xam
ple of
a
Ca
pt
ured
Im
age
of
a
Ther
a
py
-
Ba
ll
-
Ho
l
ding
Hand
Figure
4
.
Ex
pe
rim
ental
Setup
(Blue
Ci
rcles
Denotes
t
he
P
osi
ti
on
of
t
he
H
ands
in
the
E
xperim
ent
)
3.2.
I
ma
ge D
ata
G
at
heri
n
g
Fo
r
im
age
data
gat
her
in
g,
a
fe
w
set
s
of
im
ages
we
re
ca
ptur
ed.
Dif
fer
e
nt
ha
nd
siz
es,
s
kin
col
or
s
,
a
nd
or
ie
ntati
on
s
f
r
om
10
dif
fer
e
nt
ind
i
viduals
(
5
m
al
e
and
5
f
e
m
al
e)
wer
e
in
cl
ud
e
d
i
n
the
c
aptu
red
im
age
data.
Exam
ples o
f
hands
of
dif
fer
e
nt orientat
io
ns
are s
how
n
in
E
rror!
R
e
feren
ce so
urce n
ot foun
d.
.
Figure
5. Im
ages of
Hand
of
Diff
e
re
nt
O
rientat
ion
s
The
n,
the
im
ages
of
the
fin
ge
rtips
an
d
no
n
-
f
ing
e
rtips
we
re
crop
ped
from
hand
ho
l
ding
ba
ll
i
m
ages.
The
siz
e
of
t
he
crop
ped
im
ages
for
bo
t
h
fi
ngerti
p
a
nd
non
-
fin
ge
rtip
im
ages
is
50x5
0
pi
xels.
Ba
sic
al
ly
,
non
-
fin
ger
ti
p
im
ages
are
im
ages
t
hat
do
not
co
nt
ai
n
any
fin
gert
ip,
instea
d
t
he
y
con
ta
in
the
ba
ckgr
ound,
the
ball,
the
hand
w
rist,
et
c.
All
the
cr
oppe
d
i
m
ages
are
store
d
in
two
se
par
at
e
f
ol
der
s,
on
e
of
wh
ic
h
is
fo
r
fing
e
rtip
i
m
ages
and
the
oth
e
r
is
f
or
no
n
-
fin
ger
ti
p
one
s.
Exam
ples
f
r
om
bo
th
group
s
of
im
ages
are
show
n
in
Fig
ure
2
.
A
total
of 42
00 i
m
ages h
a
ve b
een
ob
ta
in
ed
th
at
w
il
l be
us
e
d for
bo
t
h
cl
assif
ic
at
ion
traini
ng and
validat
io
n.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vol
.
9
,
No.
2
,
Fe
br
uary
201
8
:
4
7
4
–
4
8
0
478
3.3.
De
tection
Va
li
d
at
i
on
Te
sting
By
us
ing
the
im
age
data
gath
ered,
the
cl
assifi
cat
ion
m
achi
ne
was
the
n
tra
ined
an
d
the
n
the
detect
io
n
su
ccess
r
at
e
w
as eval
uated. F
igure
7
ca
ptu
r
e
s how the
exp
e
rim
ent an
d
e
va
luati
on
we
re
done
step
-
by
-
ste
p.
An
ar
ray
of
im
age
set
s
is
c
on
structed
ba
sed
on
tw
o
m
ai
n
cat
egories;
fin
ge
rtip
a
nd
no
n
-
f
ing
e
rtip.
T
he
nu
m
ber
of
im
a
ges
pe
r
cat
eg
ory
as
well
as
categ
ory
la
bels
w
as
insp
ect
e
d.
I
f
the
num
ber
of
i
m
ages
are
un
equ
a
l
per
cat
e
gory,
then
it
can
be
a
dju
ste
d
so
t
hat
there
will
be
equ
al
nu
m
ber
of
im
ages
per
cat
egory.
The
set
s
are
then
sepa
rated
into
t
rainin
g
an
d
validat
io
n
set
s.
T
he
s
plit
ti
ng
w
as
r
andom
iz
ed
to
preve
nt
the
resu
lt
s
to b
e
b
ia
se
d.
Figure
7.
Ca
te
gory
Cl
assifi
ca
ti
on
T
raini
ng
The
bag
of
w
ord
te
ch
nique
is
from
the
natur
al
la
ngua
ge
processi
ng
a
da
pted
to
c
om
pu
te
r
vision.
Im
ages
do
not
con
ta
in
discret
e
word
s
,
there
f
or
e
,
SU
RF
featur
es
from
each
i
m
age
cat
ego
r
y
m
us
t
be
colle
ct
ed
into
a
vis
ual
‘
vo
ca
b
ulary’.
T
he
visu
al
voca
bu
la
ry
is
co
ns
t
ru
ct
e
d
by
re
du
ci
ng
t
he
num
ber
of
featu
res
t
hro
ugh
qu
a
ntiza
ti
on
of
featur
e
sp
ace
us
in
g
K
-
m
ean
cl
us
te
rin
g.
F
ur
t
her
m
or
e,
the
vi
su
al
wor
d
occ
urre
nces
in
an
im
age
wer
e
c
ounted
by
const
ru
ct
i
ng
a
histo
gr
am
to
reduce
the
rep
re
sentat
io
n
of
an
im
age
as
sh
ow
n
in
Er
ror!
Ref
ere
nce
so
u
rce
no
t
f
ound
.
.
The
e
ncode
d
trai
ning
im
a
ges
f
ro
m
bo
th
cat
ego
ries
are
fed
into
a
cl
assifi
e
r
trai
ning
proces
s.
Durin
g
the
ev
al
uation
cl
assi
fier’
s
pe
rfor
m
ance,
t
he
trai
ning
set
was
te
ste
d
an
d
a
near
pe
rf
ect
conf
us
io
n
m
at
rix
was
pr
oduce
d.
T
he
cl
assifi
er
e
valuati
on
s
te
p
was
al
so
pe
rfor
m
ed
with
validat
io
n
set
,
wh
ic
h
was
not
us
e
d
durin
g
the
trai
ning.
T
he
co
nfusi
on
m
at
rix
pro
du
ce
d
is
a
good
in
dicat
or
of
ho
w
we
ll
the
cl
assifi
er is
perform
ing
.
(a)
(b)
Figure
2
E
xam
ples
of
t
he
Im
a
ges
: (a
) Fi
ngert
ip and
(
b)
N
on
-
fin
ge
rtip
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Visua
l
-
base
d
F
ing
ert
ip
D
et
ect
ion
f
or
H
and
R
ehabili
tati
on
(
Da
y
ang Q
ur
r
at
u’aini
)
479
4.
E
xp
eri
men
t
al R
e
sults
a
n
d
A
n
aly
sis
In
this
sect
io
n,
we
assess
the
su
ccess
rate
of
the
detect
io
n
al
go
rithm
.
In
the
ex
per
im
ent,
a
total
of
4200
im
ages
was
use
d.
The
i
m
age
data
set
s
con
sist
of
2
m
ai
n
su
bs
et
s
su
c
h
as
fin
ge
rtip,
an
d
non
-
fi
ng
e
rtip
i
m
ages,
each
with
a
reso
l
ution
of
50
x
50
pix
el
s.
T
he
set
i
m
ages
are
div
ide
d
into
th
re
e
cat
ego
ries:
tr
ai
nin
g,
validat
io
n,
a
nd
unu
s
ed
sets. T
he
s
plit
ti
ng
of the
data set
s
wa
s r
a
ndom
iz
ed
to a
vo
i
d biasi
ng
the
resu
lt
s.
Table
1
sho
ws
the
ave
ra
ged
su
ccess
rate
f
or
the
detect
ion
of
fin
ge
rtip
a
nd
non
-
fi
ngerti
p
w
he
n
th
e
nu
m
ber
of
vali
dation
im
ages
var
ie
s
f
r
om
10
0
to
20
00
im
a
ges.
Ba
se
d
on
Figure
8
t
hat
sh
ows
the
gr
a
phic
al
represe
ntati
on
of
t
he
da
ta
in
Table
1,
we
ob
serv
e
d
t
hat
the
highest
s
ucces
s
rate
f
or
t
he
fi
ng
e
rtip
is
95.6%
an
d
for
no
n
-
fin
ger
t
ip
is
92.4%,
w
hich
is
acce
ptably
high.
T
he
tren
d
al
so
s
how
s
that
if
the
nu
m
ber
of
trai
ni
ng
dat
a
is i
ncr
ease
d, a
higher
su
cce
ss
rate can
b
e
obt
ai
ned
,
es
pecial
ly
f
or the
non
-
f
ing
e
rtip
detect
ion.
Table
1.
A
ver
a
ged
S
uccess R
at
e
from
Vali
dation
Set
No
.
o
f
tr
ain
in
g
i
m
ag
es
Av
eraged
su
ccess
rate
(%)
Fin
g
ertip
No
n
-
f
in
g
ertip
100
9
3
.8
83
500
9
4
.2
8
7
.4
1000
9
5
.6
92
2000
9
4
.4
9
2
.4
Figure
3
.
G
raph
of
No
. o
f
T
ra
ining
Im
ages
vs
A
ver
a
ge
d
S
uc
cess
Ra
te
of t
he
Detect
ion
of
Fi
ng
e
rtip
a
nd
N
on
-
fin
ger
ti
p usi
ng
Vali
dation
Set
A
hist
ogram
of
vis
ual
w
ord
occurre
nces
w
as
ge
ner
at
e
d
duri
ng
cl
assi
ficat
ion
trai
ni
ng
as
show
n
i
n
Er
ror!
Ref
ere
nce
sourc
e
n
ot
f
ou
n
d.
.
The
histo
gr
am
fo
r
m
s
a
basis
fo
r
trai
ning
a
cl
ass
ifie
r
an
d
f
or
th
e
act
ual
i
m
age
cl
assifi
cat
ion
.
I
n
oth
e
r
words,
it
encodes
an
im
age
into
a
feat
ur
e
ve
ct
or
.
Eac
h
e
nc
od
e
d
trai
ning
i
m
ages
in
each
cat
eg
ory
are
fed
into
a
cl
assifi
er
tr
ai
nin
g.
In
t
he
recog
niti
on
sta
ge,
t
he
im
age
is
represe
nted
by
th
e
visu
al
w
ords
t
ha
t wil
l be
disti
nguis
hed b
y t
he
classi
fier.
Figure
4
s
how
s
ty
pical
res
ults
of
the
detect
ion
al
gorithm
wh
e
n
t
he
al
go
rithm
is
app
li
ed
sca
nnin
g
ov
e
r
a
f
ull
i
m
age.
T
he
gr
ee
n
detect
ion
box
sign
ifie
s
par
t
of
t
he
im
age
wh
e
re
fi
ng
e
rtips
are
detect
ed.
T
he
y
sh
ow
ho
w
the
im
pr
ov
em
ent
has
been
achie
ve
d
w
hen
a
high
er
nu
m
ber
of
trai
nin
g
im
ages
is
us
ed.
T
he
ou
tpu
ts
sh
ows
a
prom
i
sing res
ult i
n
t
he dete
ct
ion.
(a)
(b)
(c)
Figure
4
Re
s
ul
ts of the
Detect
ion
Algo
rithm
;
(
a)
N
o.
of
T
rainin
g
Im
ages
=10
0,
(b) No
. of
Trainin
g
Im
ages
=1
000, (c
) N
o.
of
T
rainin
g
I
m
ages
=2
000
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci,
Vol
.
9
,
No.
2
,
Fe
br
uary
201
8
:
4
7
4
–
4
8
0
480
5.
CON
CLUSIONS
In
this
work,
it
has
bee
n
sho
wn
that
the
m
e
thod
base
d
on
SU
RF
a
nd
bag
of
w
ords
has
been
s
how
n
a
good
pe
rfor
m
ance
in
detect
in
g
fin
ge
rtips
in
i
m
ages
wh
e
re
a
hand
is
holdi
ng
a
the
ra
py
ba
ll
that
is
no
r
m
al
l
y
us
e
d
in
a
post
-
stroke
hand
th
erap
y.
T
he
s
uc
cess
rate
was
ge
ner
al
ly
f
ound
to
be
inc
rease
d
w
hen
the
num
ber
of
trai
ning
im
age
s
wer
e
i
ncr
eas
ed,
es
pecial
ly
in
the
co
rr
ect
i
den
ti
ficat
io
n
of
the
non
-
fin
ge
rtip,
i.e.
l
ow
e
r
fals
e
po
sit
ive
detect
ion
r
at
es.
T
he
su
ccess
rate
for
the
fi
ngert
ip
detect
ion
r
eached
highe
r
tha
n
94
%
wi
th
the
al
gorithm
,
whic
h
is
reas
ona
bly
high
for
t
he
the
rap
y
a
ppli
cat
ion
s,
des
pite
the
use
of
com
m
ercial
-
gr
a
de
ca
m
eras.
ACKN
OWLE
DGE
MENTS
This
researc
h h
as b
ee
n
s
uppo
r
te
d
by
I
ntern
at
i
on
al
Islam
ic
U
niv
e
rsity
Mal
ay
sia
thr
ou
gh th
e resea
rch
gr
a
nt
RI
GS1
5
-
151
-
01
51.
REFERE
NCE
S
[1]
G.
W
u
and
W
.
Kang,
“
Robust
Fingert
ip
De
tect
ion
in
a
Com
plex
Envi
ronm
ent,”
IEE
E
Tr
ansactions
on
Mult
imed
ia
,
vol.
18
,
no
.
6
.
pp
.
978
–
987
,
2016
.
[2]
D.
Jabe
r;
R.
;
He
ws
on;
F.,
D.;
J.,
“
Design
and
va
li
dation
of
the
Grip
-
bal
l
for
m
ea
sure
m
ent
of
hand
grip
strengt
h
,
”
Med.
Eng
.
Ph
y
s.
,
vol
.
34
,
no
.
9
,
p
p.
1356
–
1361
,
2
012.
[3]
M.
Bal
dauf
,
S.
Za
m
bani
ni
,
P.
F
röhli
ch
,
and
P.
Rei
chl,
“
Marke
rless
vi
sual
fi
nge
rtip
det
ection
fo
r
natural
mobil
e
dev
i
ce i
nt
erac
ti
o
n
,
”
Proc
.
13
th
In
t.
Conf
.
Hum
.
C
om
put.
Inter
act.
with
Mob.
D
evi
c
es
Serv.
,
pp
.
539
–
544,
2011
.
[4]
D.
-
D.
Y.
D.
-
D.
Yang,
L.
-
W
.
J.
L.
-
W
.
Jin,
and
J.
-
X.
Y.
J.
-
X.
Yin,
“An
ef
fe
c
tive
robus
t
fi
ngerti
p
de
te
c
ti
on
method
f
or
fi
nger
writi
ng
ch
aracte
r
rec
ognition
system,
”
2005
Int.
Conf.
Mac
h.
L
ea
rn
.
C
y
b
ern
.
,
vol
.
8
,
no.
Augus
t,
pp.
18
–
21,
2005.
[5]
N.
Branc
a
ti
,
G.
Caggi
an
ese
,
M.
Frucc
i,
L
.
Gall
o
,
and
P.
Neroni
,
“Robust
fi
ngert
ip
det
e
ct
ion
in
e
goce
ntri
c
vi
sion
under
vary
ing
i
llum
inat
ion condi
t
ions,”
2015
IE
E
E
Int
.
Conf
.
Mul
ti
m
ed.
Expo
W
o
rk.
ICMEW
201
5,
2015
.
[6]
L.
L
i
and
L.
Zh
ang,
“
Corner
De
te
c
ti
on
of
Hand
Gesture
,
”
Telk
o
mnika
Indone
s.
J.
E
lectr.
Eng.,
vol.
10,
no.
8
,
p
p.
2088
–
2094,
201
2.
[7]
W
.
Li
,
P.
Dong,
B.
Xiao,
and
L
.
Zhou,
“
Obje
ct
rec
ognition
bas
ed
on
the
Reg
io
n
of
Inte
rest
an
d
opti
m
al
Bag
o
f
W
ords m
odel
,
”
Neurocomputi
ng
,
vol
.
172
,
pp
.
27
1
–
280,
2016
.
[8]
H.
Ba
y
,
A
.
Ess,
T.
Tu
y
tela
ars,
and
L
.
Van
Go
ol,
“
Speede
d
-
U
p
Robust
Feat
ur
es
(SU
RF
),
”
Comput.
Vi
s
.
Image
Unders
t
.
,
vo
l. 11
0,
no
.
3
,
pp
.
346
–
359,
2008
.
[9]
J.
Baue
r
,
N.
Sünderha
uf,
and
P.
Protz
el,
“
COMPAR
ING
S
EV
E
RA
L
IMP
LEM
E
NT
ATIONS
OF
TWO
RE
CENTL
Y
PUBLISHED FEATURE
DETE
CT
ORS
,
”
IFA
C
Proc.
Vol
.
,
vol.
40,
no
.
15
,
pp
.
1
43
–
148,
2007
.
[10]
F.
Qi,
X.
W
ei
ho
ng,
and
L.
Qi
ang
,
“
Resea
rch
of
I
m
age
Matc
hin
g
Based
on
Im
prove
d
SU
RF
Algorit
hm
,
”
Tel
komni
ka
Indone
s. J.
Elec
t
r.
Eng
.
,
vol. 12,
no.
2
,
pp
.
1395
–
1402,
2014
.
[11]
M.
Khale
di
an
a
nd
M.
B.
Menha
j,
“
Real
-
ti
m
e
Vision
-
base
d
Hand
Gesture
Rec
ognition
Us
ing
Sift
Feat
ure
s,
”
Telk
omnika
Indo
nes.
J
.
Elec
tr.
E
ng.
,
vol. 15, no.
1,
pp
.
162
–
170
,
2015.
[12]
V.
A,
D.
Hebb
ar,
V.
S
.
Shekh
ar,
K.
N
.
B.
M
urth
y
,
and
S.
N
at
ar
aj
an
,
“
T
wo
Nove
l
De
tector
-
Descriptor
Base
d
Approache
s for Face Recognitio
n
Us
ing
SIFT
an
d
SURF,
”
Proce
dia
Com
put. Sci.
,
vol
.
70
,
pp
.
185
–
197,
2015
.
[13]
P.
Viola
and
M.
Jones,
“
Rapi
d
obje
c
t
det
e
ction
using
a
boosted
casc
ade
of
simple
fe
atures,
”
in
Proce
ed
ings
of
the
20
01
IEEE
Com
pute
r
Soc
ie
t
y
Co
nfe
ren
c
e
on
Co
m
pute
r
Vision
a
nd
Pattern
R
ec
o
gnit
ion, 2001
.
[14]
N.
Dhana
cha
ndr
a,
K.
Mangle
m
,
and
Y.
J.
Chanu,
“
Image
Segme
ntat
ion
Us
ing
K
-
means
Cluste
ri
ng
Al
gorithm
and
Subtract
ive
C
luste
ring
Al
gorithm
,
”
Proce
di
a
Com
put.
Sc
i.,
vo
l. 5
4
,
pp
.
764
–
771
,
2
015.
[15]
N.
H.
Darda
s
and
N.
D.
Georga
nas,
“
Rea
l
-
t
ime
Hand
Gesture
Det
ec
t
ion
and
Rec
o
gnit
ion
Us
ing
Bag
-
of
-
Feat
ur
es
a
nd
Support
Vec
tor
Mac
hine T
ec
hni
ques.
pdf,”
IE
EE
Tr
ans.
Instrum
.
Me
as.,
vol. 60, n
o.
11
,
pp
.
3592
–
3607,
2011
.
Evaluation Warning : The document was created with Spire.PDF for Python.