TELKOM
NIKA
, Vol. 11, No. 5, May 2013, pp. 2605 ~ 2611
ISSN: 2302-4
046
2605
Re
cei
v
ed
Jan
uary 14, 201
3
;
Revi
sed Ma
rch 1
3
, 2013;
Acce
pted Ma
rch 2
3
, 2013
A Robot Control System Based on Gesture Recognition
Using Kinect
Biao MA
*
1,a
, Wen
s
heng
X
U
2,b
Songlin WANG
2,c
1
School of Aut
o
matio
n
, Beiji
n
g
Institute
of
T
e
chn
o
lo
g
y
, Beij
ing 1
0
0
081, Ch
ina
2
chool of Mec
han
ical, Electr
onic a
nd C
ontr
o
l Eng
i
ne
eri
ng,
Beiji
ng Jia
o
ton
g
Univ
ersit
y
,
Beiji
ng 1
0
0
044
, China
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: mabia
oed
d
y
@gmai
l
.com
a
, shxu@bjtu.edu.cn
b
, 11214
21
@bjtu.e
du.cn
c
A
b
st
r
a
ct
T
he Kinect c
a
mera is w
i
d
e
ly use
d
for capturi
ng hu
man bo
dy i
m
ag
es and h
u
m
a
n
moti
o
n
recog
n
itio
n i
n
vide
o g
a
m
e
pl
ayin
g, an
d th
e
r
e are
a
l
re
a
d
y
so
me r
e
se
arc
h
w
o
rks o
n
g
e
s
ture rec
ogn
iti
on.
How
e
ver, to
a
c
hiev
e th
e a
n
t
i-interfere
n
ce
perfor
m
a
n
ce, t
he c
u
rrent
rec
ogn
ition
a
l
gor
ithms
ar
e ofte
n
compl
e
x an
d t
a
rdi
ness, a
nd
most
of
the a
p
p
licati
ons
are
base
d
o
n
the
i
n
co
mp
lete
ges
ture li
brary a
n
d
not
all
ha
nd
gestur
e
s can
b
e
rec
o
gni
z
e
d. T
h
is
pa
per ex
pl
ores
a
new
metho
d
a
nd a
l
g
o
rith
m w
h
ich c
an
descr
i
b
e
all five fin
gertip
s
for each han
d in any ti
me f
o
r han
d
gestur
e
recog
n
itio
n w
i
th the Kinect s
ystem. T
he ha
n
d
imag
es ar
e pr
ocesse
d to
bu
i
l
d th
e h
a
n
d
mode
ls w
h
ich
ar
e the
n
co
mpar
ed w
i
th th
e g
e
s
ture li
brary f
o
r
gesture
reco
gn
ition. After
han
d g
e
stures
are
recog
n
i
z
e
d
w
i
t
h hi
gh
accura
cy an
d l
e
ss co
mp
utin
g, contr
o
l
commands cor
r
esponding t
o
hand
gestu
res are s
ent from
the hand gesture rec
o
gnition system
to
a
hexa
g
o
n
rob
o
t
control
l
er w
i
re
lessly, the
hex
ago
n ro
bot
ca
n then
be
cont
rolle
d w
i
rel
e
ssl
y and
cha
n
g
e
its
shap
e accor
d
i
n
g to the h
and
g
e
sture co
mma
nd. T
hus t
he r
o
bot can i
n
terac
t
w
i
th huma
n
s
pro
m
ptly thr
o
u
gh
the gesture r
e
c
ognition system
.
Ke
y
w
ords
: rob
o
t control, gest
u
re reco
gniti
on
, Kinect, depth
imag
es
Copy
right
©
2013 Un
ive
r
sita
s Ah
mad
Dah
l
an
. All rig
h
t
s r
ese
rved
.
1. Introduc
tion
The stu
d
ie
s of gestu
re reco
gnition te
chni
que
s sta
r
ted from 1
9
77 wh
en El
ectro
n
ic
Visuali
z
ation
Labo
rato
ry created the fi
rst
data glov
e
which i
s
calle
d
the Sayre
Gl
ove [1]. For t
h
e
past thirty-fiv
e years, re
sear
che
r
s gra
dually ado
pt the cam
e
ra
to impleme
n
t the huma
n
-
comp
uter inte
ractio
n (HCI
). Unlike the da
ta glov
e, gest
u
re recognitio
n
throu
gh ca
mera
s ma
ke
s it
more natu
r
al and
di
re
ct
to reali
z
e HCI.
There
a
r
e
th
ree tasks for the ge
stu
r
e
re
cog
n
ition
syst
em:
segm
entation
,
feature extraction a
nd reco
gnition.
F
o
r the se
gm
entation task, the camera
is
usu
a
lly u
s
ed
for two diffe
rent tasks:
ca
pturin
g
RGB
image
s a
n
d
captu
r
ing
de
pth ima
ges.
For
captu
r
ing
RG
B image
s, th
e sy
stem util
ize
s
t
he
ch
a
r
acte
ri
stics of
the hu
man
compl
e
xion t
o
sep
a
rate
the
gestu
re
s
fro
m
the ba
ckground. Lin det
ects the
skin
can
d
idate
re
g
i
ons on
the
color
image
with G
aussia
n
Mixture M
odel
(G
MM) skin m
o
del [2]. And in Kramb
e
rg
e
r
’s
re
sea
r
ch, to
improve dete
c
tion a
ccu
ra
cy of pixel-based
skin
co
l
o
r se
gmentat
ion, a param
etric skin
col
o
r
model i
s
u
s
e
d
[3]. For
ca
pturing
depth
image
s,
the
depth info
rm
ation is
empl
oyed to foun
d a
colum
n
di
agram to di
sting
u
ish th
e h
u
m
an bo
dy
(the
nea
re
st obje
c
t) a
nd th
e b
a
ckgroun
d (t
he
furthes
t object).
For the featu
r
e extractio
n
a
nd re
cog
n
itio
n task,
recent
research
es
are n
o
rmally i
n
three
categories: (1) probability grap
h model based methods, which
involve Hidden Markov Model
(HMM
) [4] and Dynami
c
Bayesian
Ne
twork [5, 6]; (2) template
based m
e
th
ods, which a
r
e
related to the
template matchin
g
algo
rith
m [7, 8]; and (3) rule ba
se
d method
s, which
contain t
h
e
finite-state m
a
chi
ne (FSM) [9] and the stocha
stic
co
n
t
ext-free gra
mmar [10, 1
1
]. By employing
these alg
o
rit
h
ms, so
me rese
arche
r
s impleme
n
t the recognitio
n
task ba
sed
on the skelet
on.
Their
system
s ca
n captur
e the main p
o
ints of the h
u
man
sk
eleto
n
and recogn
ize the motio
n
s.
Mean
while,
others achie
v
e the intera
ction ba
se
d on the static hand ge
st
ure, such as t
he
“victory”
or “ok”
ge
sture
s
.
However, th
ese g
e
stu
r
e
s
are re
co
gni
zed by the e
x
tended fing
ers,
whi
c
h mea
n
the bent finge
rs’ inform
atio
n
is lost in re
co
gnition proce
ss.
In 201
0, Mi
crosoft lau
n
ch
ed a
n
inf
r
are
d
range
-sen
sing
came
ra
– Kine
ct, whi
c
h
ca
n
provide
users with raw
d
epth dat
a.
Wan et al. [
12] pro
p
o
s
e
to set a thre
shol
d value
and
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NIKA
Vol. 11, No
. 5, May 2013 : 2606 – 261
1
2606
sep
a
rate
the
han
d from
the b
a
ckg
r
ou
nd
combi
n
ing
the d
epth d
a
ta which is
provide
d
by t
h
e
Kinect. Comp
ared
with
oth
e
rs’
algo
rithm
s
, this
app
ro
ach i
s
m
o
re
dire
ct and
efficient. Li
ke
Wan
et al., many
rese
arche
r
s’
works a
r
e
ba
sed
on
the
O
pen
Com
pute
r
Visi
on li
bra
r
y (Op
enCV
)
[
13]
and O
pen
Natural Inte
ra
ction (O
pen
NI) [14] whil
e th
ere
are
le
ss
studie
s
b
a
se
d on th
e Kin
e
ct
Software
Dev
e
lopme
n
t Kit
(SDK)
whi
c
h
wa
s rele
ased
in 2011.
In this pape
r, utilizing the depth data from
the Kinect, a gesture reco
gnition
system i
s
prop
osed
and
develo
ped
which
can
effectively re
c
o
gniz
e
h
a
n
d
ge
s
t
u
r
es
w
i
th
less c
o
mp
u
t
in
g bu
t
high accu
ra
cy. The gesture recognitio
n
system is
de
veloped with
Microsoft Visual Studio 20
10
and the Kine
ct Wind
ows
SDK. A new algorith
m
–
th
e slot algo
rit
h
m is propo
sed in our
system
and it can ca
pture the info
rmation of bot
h t
he bent fingers and exte
nded fi
nge
rs. And the system
is ba
se
d on a
gestu
re lib
ra
ry whi
c
h i
s
constructe
d on
an alg
o
rithm
with mu
ch lo
wer
co
mplexi
ty.
Usi
ng this
re
al-time recog
n
ition syst
em
, a
hexagon
robot d
e
vel
oped by u
s
can
re
ceive t
h
e
gestu
re
i
n
formation wirel
e
ssly
a
nd can
cha
nge
it
s sh
ape
a
c
co
rdin
gly,
so
the ro
bot
ca
n
inte
ract
with peo
ple p
r
omptly throu
gh huma
n
ha
nd ge
sture
re
cog
n
ition.
2. Sy
stem Structur
e
The ro
bot co
ntrol sy
stem inclu
d
e
s
four
parts a
s
sho
w
n in Figu
re
1:
(1)
a gestu
re recognition
syste
m
runni
ng on
a laptop com
puter;
(2)
a Kinect ca
m
e
ra conn
ecte
d with the lap
t
op comp
uter;
(3)
a hexago
n ro
bot and the robot co
ntrolle
r;
(4)
a pair of wi
re
less co
mmun
i
cation mo
dul
es c
onn
ecte
d
with the gesture re
co
gniti
on syste
m
and the ro
bot
controll
er respectively.
Figure 1. Th
e system
stru
ct
ure of the robot co
ntrol system ba
sed
on ge
sture
re
cog
n
ition
The Kin
e
ct
camera i
s
u
s
e
d
to o
b
tain t
he ima
ge
dat
a of the
hum
an p
a
lm a
n
d
fingers.
Then the dat
a are processed to re
cog
n
ize ho
w
ma
ny fingers a
r
e extended a
nd strai
ght. Each
gestu
re i
s
correspon
ding
to a diffe
re
nt rob
o
t cont
rol
comm
and
. Then th
e
APC220
wi
re
less
module i
s
used to se
nd
these different robot
co
ntrol com
m
a
nds to the robot cont
roll
er.
Acco
rdi
ngly, the hexag
on robot will d
o
a
c
tion
s ac
co
rdi
ng to differen
t
human ha
n
d
gestu
re
s, th
us
human
-robot
intera
ction ca
n be achieve
d
.
The system is
devel
ope
d
with C#
of Micr
osoft Visual Studio
2
010, u
s
ing
the SDK
instea
d of the
Ope
n
NI. As
the ne
we
st ki
t, the SD
K
p
r
ovides a set of
po
werful
al
gorithm
s whi
c
h
enabl
es the
system
to ex
tract th
e de
p
t
h data
and
transfo
rm th
e
scen
e ima
g
es i
n
to bi
nary
image
s, there
a
fter to build a body skelet
on model.
Ho
wever, the 20
-point
-skel
e
to
n model cann
ot
satisfy the
n
eed
s for ha
n
d
ge
sture re
cog
n
ition. Th
us, the
dept
h data
of th
e two
palm
s
are
utilized for hand detail
s
pr
ocessi
ng and recognition.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
A Robot Co
ntrol System
Base
d on Ge
st
ur
e Recogniti
on Using Kin
e
ct (Biao MA
)
2607
3. Recog
n
ition of Ha
nd
Gestures
3.1. The Cap
t
ure an
d Process o
f
Stati
c
Hand Imag
es
In orde
r to e
x
tract the fe
ature
s
of ha
nd ge
stures,
first we n
e
e
d
to divide the han
d
image
s from t
he b
a
ckg
r
ou
n
d
. A lot of
re
searche
r
s
h
a
ve propo
se
d di
fferent alg
o
rit
h
ms to u
s
e
th
e
compl
e
xion feature
s
. Ou
r method co
mbine
s
t
hese compl
e
xio
n
algorithm
s with the depth
informatio
n a
nd palm
s
skeleton inform
ation to accomplish the
gestu
re mo
d
e
l. The pal
ms
skeleton i
n
formation i
s
pro
v
ided by the
SDK. Thu
s
o
u
r mai
n
work
is extra
c
ting t
he ha
nd im
ag
es
from the
ba
ckgrou
nd. T
he
main m
e
thod
to se
parate t
he h
and
imag
es i
s
usi
ng
a
depth th
re
sh
old
whi
c
h
stan
ds for t
he thi
ckness
of the
h
and
and
com
parin
g th
e
cu
rre
nt poi
nt’s
depth val
u
e
with
the previo
us
one’
s. Then
we g
e
t the contour
of
the
hand
ge
sture
s
. Let P den
o
t
es the p
r
evi
ous
point, C de
no
tes the curre
n
t point and
T denote
s
th
e thickne
ss
o
f
the hand. T
he han
d extracting
algorith
m
is shown belo
w
.
(1)
P and
C
are
set i
n
a
3
×
3 pixel
are
a
. Assume
P i
s
the
center of the
are
a
, C m
o
ves
clo
c
k
w
i
s
e.
(2)
At the beginn
ing, utilizing t
he SDK, we
gain
the p
a
lm
s skel
eton inf
o
rmatio
n of the hum
an
s.
We si
mply use the palm po
int as the first
P.
And the u
pper pixel of
P is the first C.
(3)
If C.depthVal
ue >= P.dept
hValue + T,
we beli
e
ve that C is the b
a
ckgroun
d p
o
int, then C
moves to the
next place
wh
ile P’s positio
n is re
co
rde
d
in the conto
u
r set K.
(4)
If C.depthVal
ue < P.depth
V
alue + T, we believe
that
C is in
side t
he han
d cont
our, then
C
become
s
a n
e
w P while th
e previou
s
P start
s
to move as a ne
w C.
After we
get the contou
r of
the ha
nd, we
use the
k-mean
algo
rith
m to divide th
e point
s
into two ha
n
d
gro
u
p
s
an
d
extract the f
eature
s
of th
e han
d [15]. Some re
se
arche
r
s
propo
se a
method
which de
scri
bes t
he h
and
with
extended
fi
ng
ers a
nd
a p
a
l
m
center.
Th
ey mainly
use
the
binary i
m
age
s to i
dentify the ge
stures.
Ho
weve
r,
th
e bin
a
ry ima
ges ju
st sim
p
ly provid
e t
h
e
conto
u
r
of th
e ha
nd
and
a lot of i
n
formation i
s
lo
st. For in
stan
ce,
whe
n
we
re
cog
n
ize t
he
“victory” g
e
st
ure, the bina
ry image will
lose t
he info
rmation of th
e thumb, ring
finger and lit
tle
finger. To solve this pro
b
lem, we p
r
opo
se an al
gorithm
calle
d slot algo
rithm. In the slot
algorith
m
, ea
ch h
and
is
d
e
scrib
ed
by the po
sition
o
f
five fingerti
ps a
nd
a pal
m ce
nter
even
though
some
fingers are be
nt. The slot al
gor
ithm i
s
divided into five step
s:
(1)
Traverse the
conto
u
r set K and find the deep
est poi
nt D.
(2)
Find th
e top
point, the
bottom p
o
int, the
point f
r
om th
e very l
e
ft an
d the
point f
r
om the
very
right. Use
th
ese
point
s to
build
a
recta
ngle. Put the
point
s which
are
in
side th
e re
ctan
gle
into a s
e
t R. Find the c
l
oses
t point A within this
s
e
t.
(3)
Define the thi
c
kne
ss T
2
of the ge
sture. T
2
= D.depthV
alue - A.dept
hValue.
(4)
Define
th
e sl
ot
width W which ha
s
the half
value of
T
2
. Use
the
checkin
g
alg
o
rithm to build
a s
e
t R’, let T =
T
2
and A be the starting
point.
(5)
Use the
k-cu
rvature
algo
ri
thm to cal
c
ul
ate the set K and R’, th
en we have
the set P
k
(finge
rtips of
extended fing
ers) an
d P
r
(fingertip
s
of b
ent fingers).
Therefore, we can g
e
t the positio
n of the five fingertips.
The way of usin
g the k-curvature algo
rithm
to find the fingertip
s
is sho
w
n a
s
follows
[16]: When
we have g
o
t the han
d conto
u
r
set K, we
start to u
s
e t
he elem
ents
of it. For ea
ch K
(i), we g
e
t the neigh
bori
n
g
points, and
u
t
ilize thes
e p
o
ints to gen
erate two vecto
r
s a
nd calcul
ate
the minimum angle which they form. The distan
ce
be
tween the poi
nts is marke
d
by m.
And the
vectors a
r
e
fo
rmed
by K(i
)
---K(i
-m) an
d
K(i)---K
(i+m
).
If the angl
e is less th
an a
specifi
c
value
α
,
we believe th
at it is a fingertip. The
more approp
riate
values foun
d
are m=22 an
d
α
= 4
0
deg
ree.
Based o
n
the
k-curvatu
r
e a
l
gorithm,
all fingertip
s
can
be identified.
3.2. Finger G
estur
e Re
co
gnition
Figure 2 sho
w
s the mi
rror image of a n
o
rmal
left ha
nd take
n by the Kine
ct ca
mera a
n
d
the corre
s
po
nding
re
co
gn
ized
ha
nd
co
ntour whic
h i
s
m
a
rked
by
the
name
of
finge
rtips.
T
he
whol
e expe
ri
ment situ
atio
n is
sho
w
n
in Figu
re 3.
Take
the left
hand fo
r in
stance
,
the h
and
gestu
re recog
n
ition pro
c
e
s
s is de
sig
ned
as thre
e step
s as follo
ws:
(1)
Thumb
an
d li
ttle finger sta
t
e judg
ement
: Gestu
r
e
s
are cl
assified
b
y
wheth
e
r th
e thum
b o
r
the little finger is extende
d or not. Get the positio
n of point B from the very left fr
om set P
k
.
If there is n
o
point (reg
ardless they a
r
e ext
ende
d o
r
bent
) on th
e left of B, then the little
finger is exte
nded. Othe
rwise, the little finger i
s
bent.
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NIKA
Vol. 11, No
. 5, May 2013 : 2606 – 261
1
2608
Among the
fingers, the th
umb i
s
the
on
ly one that i
s
far a
w
ay from
the othe
rs. If
there i
s
an
element of set P
k
which i
s
far a
w
ay from the other
element
s, then the thumb i
s
extende
d.
Otherwise, it is
bent.
(2)
Finge
r
counti
ng: Ge
stures ar
e
cl
assifie
d
by the
num
ber
of
extend
ed finge
rs. If the sy
stem
still can
not i
dentify the g
e
stur
e, the
e
x
tended fin
g
e
r
n
u
mbe
r
an
d the
state
of
thumb
an
d
little finger wil
l
be sent to the next step.
(3)
Index finger
state judg
em
ent: Ignore t
he ge
stures
whi
c
h a
r
e h
a
rd to ma
ke
for normal
peopl
e, we list every possible ge
stu
r
e
in Tabl
e 1. For “1
” den
otes the finger is extended
while “0” de
n
o
tes the finge
r is bent. Among the
s
e 19
kind
s of gest
u
re
s, there a
r
e 4 pairs of
gestu
re
s that
are
ha
rd to
disting
u
ish
with
ea
ch
ot
her i
n
the
p
a
ir. However,
they have
different
state
s
of
the i
ndex
finge
r. Comb
ine the
set P
k
and
P
r
. If the
thumb
is extende
d, find
out the
point
that has three poin
ts on
the left. Otherwis
e, find
out the point that has
four
points o
n
the left.
Figure 2. Re
cognition of th
e left hand fingers
Then if it
co
mes f
r
om P
k
,
the ind
e
x fin
ger i
s
extend
ed. Othe
rwi
s
e, it is b
ent.
These
gestu
re
s are marked by bl
ack ba
ckgro
u
nd in Table 1.
With the
me
thod a
bove,
the h
and
g
e
st
ures can
be re
co
gni
ze
d
conveni
ently and
effic
i
ently.
Table 1. The
cla
ssifi
cation
of finger state
s
in the ge
stu
r
e library
E-N
u
m
L/T
0
1 2 3
4
5
1/0
10000
10010
11100/10110
11110
1/1
10001
10011/10101
11101/10111
11111
0/1
00001 00011 00111
0/0
00000
00010/00100
00110 01110
E-Num: nu
m
ber of stretch
ed-o
u
t fingers;
T/L: states of Thumb/Little finger
xxxxx: states of thumb, index, middle, ring, little finger
4. Robot
Co
ntrol
w
i
th G
e
sture
Rec
o
g
n
ition
After gestu
re
re
cog
n
ition, each ge
stu
r
e is
e
n
code
d and th
e
coding i
s
sen
t
to the
hexago
n ro
b
o
t cont
rolle
r
wirel
e
ssly. T
he hexa
gon
robot devel
op
ed by u
s
can
cha
nge it
s form
into triangle, rectan
gle, or h
e
xagon etc. a
nd can
ro
ll on
the groun
d from left to right or from right
to left. Each
shape
or acti
on i
s
corresponding
to a specific
ha
nd gesture. Th
us the hexagon
robot can intera
ct with hu
man han
d ge
sture
s
p
r
omp
t
ly. The gesture re
cog
n
ition
system and the
hexago
n rob
o
t are
sho
w
n
in Fig
u
re
3.
Several
exa
m
ples of the
ge
sture
re
cognition
and
the
sha
pe chan
g
e
s of the hex
agon robot in
our ex
pe
rime
nts are
sho
w
n in Figure 4 to 6.
The exp
e
rim
ents
sh
ow th
at the h
and
g
e
stur
e
re
cog
n
ition meth
od
pro
p
o
s
ed
ab
ove ha
s
high a
c
curacy and sta
b
ility in captu
r
in
g
the inform
at
ion of the
wh
ole ha
nd, an
d it simplifie
s the
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
A Robot Co
ntrol System
Base
d on Ge
st
ur
e Recogniti
on Using Kin
e
ct (Biao MA
)
2609
algorith
m
of
gestu
re
ident
ifying with
th
e hel
p of
th
e ge
stu
r
e li
b
r
ary. Previou
s
re
sea
r
che
r
s
prop
osed several compl
e
x algorithm
s for ge
sture id
entification, such a
s
Rashi
d
’s sy
stem with
the Hu-Mome
n
ts algo
rithm
[17].
Figure 3. The
gesture re
co
gnition sy
ste
m
and the he
xagon ro
bot
Figure 4. Re
cognition of ge
sture “three” and
the
shap
e
cha
nge of the hexag
on robot
Figure 5. Re
cognition of ge
sture
“fou
r”
a
nd the sh
ape
cha
nge of the
hexagon rob
o
t
Figure 6. Re
cognition of ge
sture
“all-ex
tended
” and th
e sha
pe chan
ge of the hexagon
robot
Hexag
on
robot
Huma
n
hand
Kinect
came
ra
Gestu
r
e
recognitio
n
comp
uter
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NIKA
Vol. 11, No
. 5, May 2013 : 2606 – 261
1
2610
But their al
g
o
rithm
s
a
r
e
compl
e
x an
d
need
mo
re
pro
c
e
ssi
ng ti
me, and
the
y
cann
ot
recogni
ze
all
gestu
re
s in
al
l situatio
ns a
c
curately
. F
o
r insta
n
ce, Ra
shid’
s
system
ha
s difficulties
in di
stingui
shi
ng
wheth
e
r a
n
extend
ed fi
nger i
s
a
n
in
d
e
x finge
r o
r
a
middl
e fing
er. Our sy
stem
is
su
ccessfully built with the use
of the interg
rated inf
o
rmatio
n of all five fingertips, and it can
recogni
ze th
e
status
of every fi
nge
r an
d it can
en
su
re that the
sy
stem h
a
s
a stronge
r a
b
ility in
disting
u
ishing
similar g
eatu
r
es.
5. Conclusio
n
Based o
n
the
slot algo
rithm prop
osed i
n
this pap
er,
the conto
u
r o
f
both the extended
fingers an
d b
ent fingers
ca
n be id
entifie
d and th
e a
c
tual po
sition
s
of the fingers
can
be g
o
t, thus
hand g
e
stu
r
e
s
ca
n be reco
gnized a
c
curately and e
fficiently. By sending the g
e
sture informati
on
to the
rob
o
t co
ntrolle
r
wirele
ssly, the
hexag
on
ro
bot can
ch
a
nge it
s
sha
p
e
into t
r
ian
g
le,
recta
ngle
or
hexago
n, etc. accordi
ngly
,
thus
hum
a
n
-robot inte
raction
s
can
be a
c
hieved.
In
future, by id
entifying more han
d ge
st
ure
s
a
nd by
adoptin
g a
more
versa
t
ile rob
o
t, the
intera
ction
s
b
e
twee
n the h
u
man
s
and th
e robot
can b
e
more a
bun
dant and flexi
b
le.
Ackn
o
w
l
e
dg
ement
This work is f
unde
d by Beijing Jia
o
tong
University Student Innovat
ion Prog
ram.
Referen
ces
[1]
Sturman
DJ, Z
e
ltzer
D. A sur
v
e
y
of
glov
e-b
a
sed
in
put.
IE
EE Co
mputer
Graphics
an
d
Appl
icatio
ns
.
199
4; 14(1): 30
–39.
[2]
Lin
La
n, Co
ng
Yang, T
ang Y
and
on
g. Ha
nd
gesture r
e
co
gn
ition
usin
g RG
B-D cues.
Info
rmati
on an
d
Autom
a
tion (ICIA)
, 2012
Internatio
nal C
onfer
ence o
n
. Shen
yang
. 20
12: 31
1-31
6.
[3]
Kramber
ger I. Reflectanc
e Measur
ement
S
y
stem
for Skin Col
o
r Mod
e
lin
g in Chr
o
maticit
y
C
o
lor
Space. 20
09 1
6
th
Internati
o
n
a
l Co
nferenc
e
on System
s,
Sign
als an
d Ima
ge Proc
essi
ng
. C
h
al
ki
da
.
200
9: 1-4.
[4]
Rizkha
Emi
lli
a
N
y
om
an, Su
yanto
NF
N, Ma
hara
n
i W
a
r
i
h. Isolated
w
o
r
d
recog
n
itio
n usi
ng ergo
di
c
hid
den mark
ov
models a
nd g
enetic a
l
g
o
rith
m.
Te
lko
m
nika
.
2012; 1
0
(1): 1
29-1
36.
[5]
Budi
harto W
,
S
antoso
A, Pur
w
anto
D, Jazi
die
A.
Multip
le
m
o
ving obstac
l
es avoi
danc
e of
s
e
rvice
r
o
b
o
t
usin
g stereo vi
sion.
Te
lkomn
i
ka
. 2011; 9(
3): 433-
444.
[6]
Z
hu Ch
un, Sh
eng W
e
ihu
a
.
Realtim
e
rec
o
gniti
on of c
o
mple
x
dai
l
y
activ
i
ties usi
ng
d
y
n
a
mic Ba
ye
sia
n
net
w
o
rk.
IEEE
Internati
ona
l
Confer
ence
o
n
Intelli
ge
nt Ro
bots a
nd Syst
ems
. S
an F
r
a
n
cisco.
201
1:
339
5-34
00.
[7]
Celik IB, Kunta
l
p M. Deve
lop
m
ent of a rob
o
t
ic-arm
control
l
e
r b
y
usi
ng h
a
nd g
e
sture rec
ogn
ition. 2
0
1
2
Internatio
na
l Symp
osi
u
m
on I
nnov
atio
ns in I
n
telli
ge
nt Systems a
nd Ap
plic
ations (INIST
A)
. Piscata
w
a
y.
201
2: 5.
[8]
Lai K, Konr
ad
J, Ish
w
ar P.
A gesture-
d
rive
n
computer i
n
terface usi
ng Ki
ne
ct
. Proceedi
ng
s of the 201
2
IEEE South
w
e
s
t Sy
m
posi
u
m on Image An
al
y
s
is & In
terpr
e
tation (SSIAI 2012). Santa Fe
. 2012: 185
-
188.
[9]
Ding
li A
l
e
x
iei,
Attard Da
nie
l
, Mamo
Rub
e
n
.
T
u
rning hom
e
s int
o
lo
w
-
cost ambient as
sisted livin
g
envir
onme
n
ts.
Internatio
na
l Journ
a
l of Ambi
ent Co
mp
utin
g and Intel
l
i
genc
e
. 2012; 4(
2): 1-23.
[10]
Mitomi H, Fujiw
a
ra F, Yama
moto M, Sato T
.
Bayesi
an cl
assificati
on of
a huma
n
cust
om base
d
on
stochastic cont
ext-fre
e
gramm
a
r.
Systems an
d Co
mp
uters in
Japan
. 2
007;
38(9): 71
6-7
2
6
.
[11]
Qing Ch
en, G
eorg
anas
ND,
Petriu EM. H
and
ges
ture r
e
cog
n
itio
n usi
ng Ha
ar-lik
e features
and
a
stochastic c
o
n
t
ext-free
gram
mar.
IEEE Transactions on I
n
strumentation and Meas
urem
ent
. 20
08;
57(8): 15
62-
15
71.
[12]
W
an T
i
ng, W
ang Y
u
ju
n, Li
J
unke. H
a
n
d
g
e
sture rec
o
g
n
i
t
ion s
y
stem
u
s
ing
de
pth d
a
ta. 20
12
2
nd
Internatio
na
l Confer
ence
o
n
Cons
u
m
er
Electr
on
ics, Communic
a
tio
n
s
and N
e
tw
orks (CECNet).
Yichang
. 20
12:
1063-
10
66.
[13]
Z
hang B
a
o
y
u
n
,
Yun Ru
w
e
i,
Qiu Hu
aqi
ng.
Han
d
Gesture
Reco
gniti
on
i
n
Natur
a
l Stat
e Base
d o
n
Rotatio
n
Invari
ance
and O
p
e
n
CV Re
al
izatio
n.
Entertain
m
e
n
t for Educati
o
n
.
D
i
g
i
t
a
l
Te
chn
i
qu
e
s
and
Systems. 5th
I
n
ternati
o
n
a
l
C
onfere
n
ce
on
E-lear
nin
g
and
Games, Ed
ut
ain
m
ent 2
0
1
0
. Berlin. 2010:
486-
496.
[14]
Cho Ok-Hue,
Lee W
on-H
y
u
ng.
Gesture re
cogn
ition us
in
g simple-O
p
e
n
N
I for impl
e
m
ent interactiv
e
contents
. Lectu
re Notes in El
e
c
trical Eng
i
ne
e
r
ing. Vanc
ouv
e
r
. 2012; 2: 141-
146.
[15] Li
Y.
Hand Ges
t
ure Reco
gniti
o
n
Using Ki
nect
. Proceed
ings o
f
2012 IEEE 3rd Internatio
nal
Confer
enc
e
on Soft
w
a
re E
ngi
neer
in
g and
Service Scie
n
c
e. Beiji
ng. 20
12: 196-
19
9.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
A Robot Co
ntrol System
Base
d on Ge
st
ur
e Recogniti
on Using Kin
e
ct (Biao MA
)
2611
[16] Mühe
ndis
lig
i Bölüm
ü
, Orta Dogu.
3D o
b
j
e
ct recogn
ition
by
geo
metric h
a
s
h
in
g
. 2009 IEEE 17th Signal
Processi
ng an
d Commu
nicati
ons App
licati
o
n
s
Confere
n
ce. Antal
y
a. 20
09;
932-
935.
[17]
Omer Rash
id,
A
y
o
ub A
l
-Ham
adi, Ber
nd M
i
c
hae
lis.
Ro
bust
Han
d
Postur
e
Reco
gniti
on
w
i
th Micro a
n
d
Macro
Leve
l
F
eatures
usi
n
g
Kinect.
2011 I
EEE International Conference
on Intelligent
Com
p
uting
and
Intelli
gent Systems (ICIS 20
1
1
)
. Guangzh
o
u
.
2011; 3: 391-
341.
Evaluation Warning : The document was created with Spire.PDF for Python.