Int
ern
at
i
onal
Journ
al of Ele
ctrical
an
d
Co
mput
er
En
gin
eeri
ng
(IJ
E
C
E)
Vo
l.
9
, No
.
5
,
Octo
ber
201
9
, pp.
3739
~3
750
IS
S
N: 20
88
-
8708
,
DOI: 10
.11
591/
ijece
.
v9
i
5
.
pp3739
-
37
50
373
9
Journ
al h
om
e
page
:
http:
//
ia
es
core
.c
om/
journa
ls
/i
ndex.
ph
p/IJECE
System i
ntegr
ation o
f a fl
uoros
copic ima
ge calibr
ation usi
ng
ro
b
ot assisted su
rg
i
cal gu
idance f
or distal
l
oc
kin
g
process i
n
closed in
tramedu
llary n
ailing o
f
fe
mu
r
Sa
k
ol
Nakdh
am
ab
h
orn
,
Jac
krit Suth
akor
n
Depa
rtment
o
f
B
iomedic
a
l
Eng
in
ee
ring
,
Fa
cul
t
y
o
f
Engi
n
ee
ring
,
Mahidol
Univ
ersity
,
Thail
and
Art
ic
le
In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
Dec
12
, 201
8
Re
vised
A
pr 4
,
201
9
Accepte
d
Apr
11
, 201
9
Distal
lo
cki
ng
proc
edur
e
is
o
ne
of
th
e
m
ost
complex
ta
sk
s
in
cl
ose
int
ramedul
la
r
y
nai
li
ng
op
erati
o
n
which
r
equi
r
es
fluorosc
op
ic
image
to
int
erp
r
et
2
-
D
di
stal
lo
cki
ng
posi
ti
on
on
image
r
el
a
te
d
to
3
-
D
di
stal
lo
cki
ng
positi
on
on
th
e
p
at
i
ent
site.
H
enc
e
the
surgeon
has
t
o
per
form
the
dis
ta
l
lo
c
king
proc
ess
b
y
usin
g
m
ult
ip
le
fluor
oscopic
images
which
ca
uses
a
lot
of
x
-
r
a
y
exposure
to
the
pat
i
ent
and
surg
eon
and
is
a
ti
m
e
consum
ing
t
ask.
Thi
s
p
ape
r
pre
sents
the
s
y
s
t
em
integra
ti
on
o
f
a
f
luoroscopi
c
i
m
age
c
alibra
t
ion
using
robot
assisted
surgic
al
gu
ida
nc
e.
The
s
ystem
int
egr
at
ion
consists
of
thr
ee
par
ts;
d
ista
l
loc
king
re
cove
r
y
,
f
luoroscopi
c
ca
l
ibra
t
ion
and
tra
ck
ing,
and
ro
bot
assiste
d
surgica
l
gu
ida
nc
e.
Th
e
distal
lo
cki
ng
-
hol
e
rec
o
ver
y
al
gor
it
hm
i
s
base
d
on
cha
ra
cteri
sti
c
inf
orm
at
ion
of
th
e
m
aj
or
and
m
inor
axe
s
of
dist
al
lo
cki
ng
hol
e.
The
f
luoroscopic
c
al
ibr
ation
an
d
tracki
ng
is
m
odel
ed
as
p
in
-
hol
e
proj
ec
t
ion
m
odel
to
esti
m
ate a
proj
ec
t
ion eq
uat
ion
base
d
on
opti
c
al
tracki
ng
s
y
stem.
Th
e
robot
-
assisted
su
rgic
a
l
guid
ance
i
s
deve
lop
ed
to
o
ver
lay
a
traject
or
y
p
at
h
using
a
l
ase
r
bea
m
for
red
u
ci
ng
the
pr
oble
m
of
h
and
–
e
y
e
coor
dinati
on
on
m
ost
surgica
l
nav
iga
t
i
on
s
y
st
em.
W
e
int
egr
at
e
e
ac
h
p
art
to
complete
a
surgi
ca
l
navi
ga
ti
on
s
y
s
t
em
for
distal
lo
cki
ng
proc
ess.
The
expe
riment
of
s
y
stem i
n
te
gr
at
ion is c
onduc
te
d
to
va
li
da
te
the
ac
cur
acy
o
f
dista
l
lo
cki
ng
a
xis
positi
on
an
d
orie
nt
at
ion
.
The
resul
ts
of
the
s
y
st
em
int
egr
at
ion
show
s
a
m
ea
n
angu
lar
err
or
of
1
.
10
a
nd
m
ea
n
Eu
cl
id
e
an
d
ista
nc
e
in
X
-
Y pl
an
e err
or
of
3
.
65
m
m
.
Ke
yw
or
d
s
:
F
luor
os
co
pic i
m
age cali
br
at
ion
cl
os
e
intram
ed
ullary
n
ai
li
ng
R
obot assist
ed
su
r
gical
gu
i
dan
ce
Su
r
gical
nav
i
ga
ti
on
syst
em
O
ptica
l t
rac
king
dev
ic
e
Copyright
©
201
9
Instit
ut
e
o
f Ad
vanc
ed
Engi
n
ee
r
ing
and
S
cienc
e
.
Al
l
rights re
serv
ed
.
Corres
pond
in
g
Aut
h
or
:
Jackr
it
S
uth
a
kor
n
,
Depa
rtment
o
f
B
iomedic
a
l
Eng
in
ee
ring
,
Facul
t
y
of Engin
ee
ring
,
Mah
idol
Univer
sit
y
,
Th
ailand
.
Em
a
il
:
j
ackr
it
.s
ut@m
ahido
l.ac
.th
j
ack
rit@
bart
la
b.
org
1.
INTROD
U
CTION
En
gin
eeri
ng
a
nd
healt
hcar
e
ar
e
an
d
ha
ve
bee
n a
n i
m
po
rtant
par
t
of
hum
an l
ife.
Re
ce
ntly
, e
ng
i
neer
i
ng
has
pai
d
m
or
e
at
te
nti
on
t
o
an
d
ha
s
ha
d
an
in
flue
nce
on
se
veral
healt
hca
re
pr
ocedu
res
[1,
2]
.
Com
pu
te
r
-
I
nte
gr
at
e
d
Surg
e
ry
(C
IS
)
a
nd
R
obot
-
Assiste
d
S
urger
y
(
R
AS
)
[
3
]
play
im
po
rtant
ro
le
s
in
nu
m
ero
us
m
edical
op
erat
ion
s
a
nd
proce
dures.
A
n i
m
p
or
ta
nt
sect
ion
of
CI
S/R
AS
is
“
m
edical
nav
ig
at
ion
”
w
hich i
nvolve
s
su
r
gical
plan
nin
g
a
nd
guidi
ng
i
n
pr
e
-
a
nd
i
ntra
-
operati
ve
proce
dures.
O
r
thopedic
sur
ge
ry
is
on
e
of
th
e
m
os
t
com
m
on
op
e
ra
ti
on
s
c
onduct
ed
in
hosp
it
al
s.
In
w
hich
on
e
of
the
m
os
t
com
m
on
injur
y
cas
es
are
of
l
ong
bone
fr
act
ures.
The
r
e
are
m
any
cas
es
of
or
t
hope
di
c
su
r
ger
ie
s
th
at
are
co
nduct
ed
in
the
hos
pital
s
on
a
daily
basis.
Or
t
hope
dic
s
urger
ie
s
re
quire
m
uch
m
or
e
ac
cur
acy
in
the
p
ro
ce
dure
tha
n
the
act
ual
ti
m
e
ta
ken
in
the
s
urgical
process
it
sel
f.
Ther
e
f
or
e,
the
s
e
sur
ger
ie
s
re
quire
highly
e
xperience
d
s
urge
on
s
to
perform
these
ta
s
ks
.
H
oweve
r,
the
sca
rcit
y
of
exp
e
rience
d
surge
ons
a
vaila
bl
e
at
a
ny
gi
ve
n
m
ay
no
t
be
e
noug
h
as
the
num
ber
of
the
pa
ti
ents
wait
ing
f
or
t
re
atm
ents.
Cl
os
e
d
int
ram
edu
ll
ary
naili
ng
is
a
fr
e
qu
e
nt
proce
dure
a
nd
ef
fici
ent
te
ch
nique
t
o
treat
long
bone
fr
act
ur
es
. T
his
tech
nique
r
e
qu
i
res
the s
urge
on t
o i
ns
ert
an i
nt
ra
m
edu
ll
ary
nail
into
the
bo
ne
c
anal
of
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
om
p
En
g,
V
ol.
9
, N
o.
5
,
Oct
ober
20
19
:
3739
-
3750
3740
the
fr
act
ur
e
d
lo
ng
bone
,
su
c
h
as,
fem
ur
,
ti
bia
and
hum
or
ous
,
after
bone
-
fix
ing
process.
Th
e
intram
edu
ll
ary
nail
is use
d
as
an i
nt
ern
al
str
uctu
re
to ho
l
d
the
fra
ct
ur
es t
og
et
her in thei
r prope
r
sh
a
pes.
Fig
ure
1
s
how
s
the
intram
ed
ullary
nail
afte
r
in
se
rted
i
nto
the
f
ractu
re
d
bone.
Cl
os
e
d
Nail
ing
is
a
m
ini
m
ally
inv
asi
ve
sur
ger
y
(
MIS)
w
hich
re
qu
i
res
only
a
f
ew
sm
al
l
incisi
on
s
durin
g
t
he
process
.
H
oweve
r,
m
edical
i
m
age
-
gu
i
ding
te
ch
ni
qu
e
s
are
re
qu
ir
ed.
Fl
uor
os
c
opic
syst
e
m
(C
-
arm
)
is
fr
equ
e
ntly
us
ed
i
n
this
proces
s
.
At
the
en
d
of
Cl
os
ed
Nail
in
g
proces
s,
t
he
s
urge
on
is
re
quired
to
pe
rfor
m
nail
loc
king
process
w
hich
i
nc
lud
es
pro
xim
a
l
locking
proce
ss
as
s
how
n
in
Fig
ur
e
1
(a)
a
nd
distal
locki
ng
proce
s
ses
as
s
how
n
in
Fig
ur
e
1
(b).
T
hese
locking
proce
s
ses
are
to
i
ns
ert
two
scr
ews
int
o
the
prox
im
al
locking
an
d
dis
ta
l
locking
hole
s
on
t
he
nail
to
keep
the
f
ractu
res
in
place
.
Howe
ve
r,
duri
ng
the
na
il
insertio
n
pr
ocess,
the
nail
m
igh
t
get
de
f
orm
ed a
nd
be
nt,
w
hic
h
can
be
cau
sed
by
exter
nal
f
or
ces
an
d
to
rques
[4]
.
The
dif
ficult
y
is
to
identi
fy
the
po
sit
io
n
and
ori
entat
io
n
of
t
he
distal
hole
s’
ax
is,
an
d
to
guid
e
the
surge
ons
to
perform
their
ta
s
ks
pro
per
l
y.
I
n
c
onve
ntio
nal
m
et
ho
d,
s
urge
on
us
es
flu
orosc
opic
im
ages
to
locat
e
a
nd
i
de
ntify
the
pos
it
ion
a
nd
ori
entat
ion
of
a
di
sta
l
locki
ng
hole
of
intram
edu
ll
ary
nail.
The
s
urg
eon
us
es
t
rial
and
er
r
or
m
et
h
od
to
al
ign
the
flu
orosc
opic
s
yst
e
m
into
the
rig
ht
directi
on
unti
l
the
distal
loc
kin
g
hole
ap
pea
r
s
in
a
perfect
c
ircular
s
ha
pe
on
the
dis
play
.
Ther
e
f
or
e,
the
axis
of
i
m
aging
c
om
po
ne
nts
on
flu
orosc
opic
syst
e
m
is
the
sam
e
as
t
he
distal
ho
le
s
’
a
xis,
w
hich
hel
ps
/a
ssi
sts
the
su
r
ge
on
t
o
us
e
as
a
guidin
g
pa
rt
for
sc
rew
i
nse
rting
i
n
dis
ta
l
locking
proces
s.
Th
us,
sur
geon
nee
ds
t
o
reali
ze
a
three
-
dim
ension
al
sc
rew
i
ns
er
ti
ng
base
d
on
two
-
dim
ensional
flu
orosc
opic
im
age.
The
refor
e
,
la
r
ge
nu
m
be
r
of
flu
orosc
op
ic
i
m
ages
can
be
r
equ
i
red
for
ta
r
geting
t
he
scre
w
dri
ll
ing
tra
j
e
ct
or
y.
T
hese
re
qu
i
re
a
lot
of
s
urgical
tim
e
and
pro
du
ce
a
l
ot
of
x
-
ra
y
ra
diati
on
ex
posures
durin
g
t
he
operati
on
w
hich
cau
se
ha
r
m
fu
l
eff
ect
of
X
-
ray
rad
ia
ti
on
e
xposure
to
bo
t
h,
t
he
surge
on
an
d
t
he
patie
nt
in
t
heir
lo
ng
te
r
m
healt
h
an
d
well
-
bei
ng.
Th
e
ove
rall
rad
ia
ti
on
e
xposure
tim
e
in
this
pa
rtic
ular
pro
cess
can v
a
ry
f
ro
m
3
-
30
m
in
utes
an
d
ca
n
ta
ke
up
t
o
a few
h
ou
rs
base
d upo
n
the
surg
e
on’s
sk
il
l
and e
xp
e
rienc
e [
5].
Fig
ure
1. The
nail l
ock
i
ng
process i
n
int
ram
edu
ll
ary
of f
e
m
ur
(
a) T
he p
r
ox
im
al
lock
in
g (
b)
The
d
ist
al
locking
Seve
ral
te
chn
i
qu
e
s
an
d
syst
em
s
hav
e
bee
n
dev
el
op
e
d
to
o
ver
c
om
e
these
diff
ic
ulti
es.
Fo
r
e
xam
ple,
a
nail
m
ou
nte
d
ta
rg
et
in
g
dev
ic
e,
im
age
intens
ifie
r
m
ou
nte
d
t
arg
et
in
g
de
vice,
sel
f
-
loc
king
na
il
ing
syst
em
,
ste
reo
-
flu
orosc
opy
and
c
om
pu
te
r
nav
i
gation
syst
e
m
[
6
,
7]
.
H
oweve
r,
t
hese
de
vices
an
d
te
c
hn
i
qu
e
s
ha
ve
s
ever
al
disad
va
ntages,
f
or
exam
ple
the
la
c
k
of
ve
rsati
li
ty
of
thes
e
dev
ic
es
,
a
nd
pract
ic
al
us
er
-
inter
face
.
They
are
al
s
o
no
t
easy
to
us
e.
Ma
ny
Com
pu
te
r
In
te
gr
at
e
d
Surg
e
ry
Syst
e
m
s
hav
e
been
pro
pose
d
to
a
ssist
ed
op
e
rati
ons,
es
pe
ci
al
ly
in
distal
interl
oc
king
hole
s
ta
r
geting.
The
a
ppr
oach
f
or
rec
ov
e
r
a
posit
ion
an
d
or
ie
ntati
on
of
distal
-
hole
a
xis
us
es
on
ly
th
e
area
of
pro
j
ec
te
d
im
age
to
re
cov
e
r
the
nails
ro
ta
ti
on
a
ngle
with
res
pect
t
o
fitt
ing
curve
te
ch
niqu
e
[
8
,
9]
.
T
hier
r
y
Lel
oup
et
.
al
,
[
10,
11]
de
ve
lop
e
d
a
te
ch
ni
qu
e
of
rec
over
y
of
a
distal
l
oc
king
ho
le
w
hich
use
d
on
ly
tw
o
flu
orosc
opic
i
m
ages
with
no
need
t
o
captu
re
t
he
axis
of
the
hole
.
A
flu
orosc
op
y
is
cal
ibrate
d
i
n
pr
e
-
ope
rati
on
process
us
i
ng
tw
o
pa
rall
el
plate
s.
The
lo
wer
plate
c
on
t
ai
nin
g
m
et
al
l
ic
balls
el
e
m
ent
is
us
e
d
to
cal
ibrate
im
age
distor
ti
on
a
nd
the
uppe
r
pl
at
e
con
ta
i
ning
m
et
al
l
ic
ring
s
is
us
e
d
to
determ
ine
the
X
-
ray
source
posi
ti
on.
T
he
extrinsic
of
t
he
up
per
cal
ibrati
on
plate
is
at
ta
ched
with
a
n
opti
cal
trackin
g
m
ark
e
r
the
n
m
et
a
ll
ic
balls
an
d
m
etall
ic
ring
s
in
both
cal
ib
rati
on
plate
s
can
determ
ine
the
locat
ion
s
relat
ive to t
his
op
ti
cal
m
ark
er.
Guoyan
Z
he
ng
et
al
,
[1
2,
13]
dev
e
l
op
e
d
a
n
a
uto
m
at
ic
reco
ve
ry
of
d
ist
al
interl
oc
king
hole
.
This
w
ork
was
base
d
on
a
sin
gle
cal
i
br
at
ed
a
nd
re
gister
ed
flu
orosc
opic
im
age.
T
hey
def
i
ned
the
rec
ov
e
ry
pro
blem
into
a
seq
uen
ti
al
tw
o
-
sta
ge
m
od
el
-
ba
se
op
ti
m
al
fit
ti
ng
proces
s.
T
he
flu
orosc
opic
im
ages
wer
e
cal
ibrate
d
by
us
i
ng
weak
-
per
s
pecti
ve
pi
n
-
ho
le
ca
m
era
m
od
el
co
m
bin
ed
with
opti
cal
trackin
g
to
relat
e
a
2D
pi
xel
into
3D
po
int
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
System
integr
at
ion
of a fl
uorosc
opic
im
ag
e
c
alibr
ation u
si
ng r
obot
as
sist
e
d.
.
.
(
Sa
k
ol Na
kdha
mab
horn
)
3741
al
so
c
orrect
a
c
-
arm
disto
rtion.
Yan
i
v
a
nd
Jo
s
kowicz
[
14,
15]
de
velo
ped
a
pr
eci
se
rob
ot
assist
ed
guide
po
sit
io
ning
f
or
th
e
distal
loc
ki
ng
int
ram
edu
ll
ary
nail.
This
rob
ot
was
a
bo
ne
-
m
ounted
m
iniat
ur
e
rob
ot
fitt
ed
with
a
dri
ll
gu
i
de.
T
he
r
obot
r
igidly
at
ta
ched
to
the
nail
or
t
he
bo
ne.
T
her
e
fore,
the
syst
e
m
do
es
not
ha
ve
any
serio
us
e
ff
ect
on
t
he
im
m
ob
il
i
zat
ion
of
the
le
g.
The
cal
ib
rati
on
m
et
ho
d,
the
y
m
od
el
ed
the
f
luorosc
opic
as
a
pi
n
-
ho
le
cam
era
a
nd
cal
ibrate
fluor
os
co
pic
im
age
distor
ti
on.
A
no
t
her
syst
e
m
cal
le
d
“HIT
-
RA
OS
”,
a
s
yst
e
m
dev
el
op
e
d
by
Du
Z
hi
-
j
ia
ng
[
16
]
,
assist
s
t
he
su
r
ge
on
s
th
r
u
m
any
proces
s
duri
ng
the
cl
os
e
intram
edu
ll
ary
naili
ng
.
Su
c
h
a
s,
t
o
re
posit
ion
brok
e
n
bones
,
t
o
guide
the
s
urgeon
lo
ckin
g
nail,
t
o
r
edu
ce
the
s
urg
eon’s
w
orki
ng
unde
r
flu
orosc
op
ic
s
yst
e
m
by
te
le
-
op
e
rati
on
syst
e
m
.
The
au
gme
nted
reali
ty
na
vig
at
io
n
syst
e
m
was
de
velo
ped
to
assist
ed
sur
ge
on
s
duri
ng
di
s
ta
l
interl
ock
i
ng.
T
he
au
gm
ented
ra
dio
l
ucen
t
dr
il
l
co
m
bin
ed
m
ulti
-
m
od
a
l
visu
al
iz
at
ion
t
o
t
rack
a
nd
ov
e
rlay
a
dri
ll
ing
t
raj
ect
or
y
un
der
a
vi
de
o
-
guide
d
i
n
real
-
ti
m
e
[1
7,
1
8
]
.
The
co
nce
pt
of
s
urgical
nav
i
gation
for
cl
ose
intram
edu
ll
ary
naili
ng
of
f
e
m
ur
us
in
g
an
overlay
la
se
r
gu
i
ding
was pr
opose
d
by N
a
kdham
abh
or
n
a
nd J.
T
. L
ia
ng
[
1
9
,
20
].
A
syst
e
m
integrati
on
of
Flu
orosc
op
ic
im
a
ge
na
vig
at
io
n
syst
em
based
Ro
bot
-
A
ssist
ed
Sur
gical
Gu
i
dan
ce
syst
e
m
fo
r
distal
locki
ng
process
in
Cl
os
ed
I
ntr
a
m
edu
ll
ary
Na
il
ing
of
Fem
ur
is
pr
op
os
e
d
in
thi
s
pap
e
r.
The
pr
opose
d
ap
proac
h
is
t
o
inte
gr
at
e
a
syst
em
,
wh
ic
h
is
a
ble
to
ge
ne
rate
a
nav
i
gation
tra
j
ect
or
y
pa
th
of
distal
locking
ho
le
s
dr
il
li
ng
a
xis,
a
nd
a
r
obot
ic
gu
ida
nce
sy
stem
to
interfa
ce
with
the
sur
geon
t
o
pe
rform
the
ta
sk
s.
The
s
yst
e
m
con
sist
s
of
4
m
ajo
r
pa
rts:
(
a)
T
he
recove
r
y
distal
locki
ng
-
hol
e
a
xis
wit
h
flu
orosc
op
ic
i
m
ages;
(b)
Fl
uor
os
co
pi
c
i
m
age
cal
ibr
at
ion
;
(
c
)
Flu
orosc
opic
trac
kin
g
f
or
m
at
ching
a
sp
at
ia
l
im
a
ge
c
oor
din
at
e
(
two
-
dim
ension
al
)
into
world
ref
e
r
ence
co
ordi
nate
(
th
ree
-
dim
en
sion
al
)
us
i
ng
opti
cal
trackin
g
syst
e
m
;
(d
)
A
novel
Robot assi
ste
d
su
r
gical
guid
an
ce by
us
in
g
a
laser
for navi
ga
ti
ng
a
dri
ll
ing
t
raj
ect
or
y
path
.
2.
RESEA
R
CH MET
HO
D
This
pa
pe
r
pre
sents
a
no
vel
appr
oach
of
i
ntegr
at
in
g
fl
uor
osc
opic
i
m
age
cal
ibrati
on
a
nd
trackin
g
syst
em
with
R
obot
-
As
sist
ed
S
urgical
Gu
i
dan
ce
syst
e
m
fo
r
distal
loc
king
process
in
Cl
os
e
d
In
tram
edu
ll
ary
Nail
in
g
of
Fem
ur
.
T
he
al
gorithm
s
co
ns
ist
of
Dista
l
l
oc
ki
ng
hole
’s
posit
ion
a
nd
ori
entat
ion
rec
overy
al
gorithm
,
the
flu
orosc
op
ic
im
age
cal
ib
rati
on,
t
he
fl
uoros
cop
ic
trac
king,
a
nd
the
r
ob
ot
-
assist
ing
s
urgical
guida
nce.
Th
ese
ar
e
descr
i
bed as s
ubsect
io
n belo
w
.
2.1.
Sy
s
tem
over
vi
ew
The
over
view
of
t
he
syst
em
is
show
n
i
n
Fi
g
ure
2.
T
he
syst
e
m
is
separ
at
ed
into
pr
e
-
op
e
rati
ve
a
nd
intra
-
op
e
rati
ve.
I
n
pre
-
op
e
rati
ve,
th
e
flu
orosc
opic
i
m
age
cal
ibrati
on
a
nd
the
dist
al
locki
ng
-
ho
le
rec
ov
e
ry
al
gor
it
h
m
are
perform
ed
to
ge
ne
rate
a
s
et
of
cal
ibrati
on
par
am
et
ers
wh
ic
h
a
re
requ
ired
t
o
im
ple
m
ent
in
intra
-
operati
ve
.
The
fl
uor
os
co
pi
c
trackin
g
in
volves
i
n
the
intr
a
-
operati
ve
to
trans
f
or
m
two
-
dim
ension
al
po
sit
ion
an
d
or
ie
nt
at
ion
of
distal
loc
king
hole
t
o
t
hr
ee
-
dim
ension
al
w
or
l
d
c
oor
din
at
e
by
usi
ng
cal
ib
rated
pa
ram
et
e
rs.
At
that
poin
t,
the
three
-
dim
ension
al
distal
loc
ki
ng
hole
tra
j
ect
or
y
se
nds
to
Robot
-
Assisted
gu
i
dan
ce
f
or
ge
ner
at
in
g
a
sur
gical
gu
i
ding
path.
Figure
2
.
The
ov
e
r
view of
th
e syst
e
m
integra
ti
on
2.2.
Dist
al
lockin
g hole’s
axis rec
ov
er
y
The
distal
locking
ho
le
’s
reco
ver
y
al
go
r
it
hm
is
app
li
ed t
o
identify
a
po
sit
ion
and
o
rientat
ion
of
distal
locking
ho
le
’s
axis b
ased
on
a flu
or
os
copi
c
im
age.
To si
m
plify
a
ref
eren
ce
coo
rd
inate
,
the intram
edu
ll
ary nail
is
def
ined
as
a
ref
eren
ce
coo
rd
inate
as
sh
ow
n
in
Fig
ure
3(
a).
The
X
axis
places
al
on
g
with
the
nail
axis
.
The
Z
axis
places
al
on
g
with
distal
locking
-
hole
’s
axis.
In
the
op
erati
on
,
the
translat
ion
and
ro
ta
ti
on
ar
e
occu
rr
ed
on
ly
in
X
and
Y
directi
on
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
om
p
En
g,
V
ol.
9
, N
o.
5
,
Oct
ober
20
19
:
3739
-
3750
3742
Fig
ure
3. (
a
) A refe
ren
ce
coor
d
inate
of an
i
ntram
edu
ll
ary
nail
(b)
T
he
c
har
a
ct
erist
ic
s o
f
the
project
io
n
im
age
of
distal
locki
ng
hole
The
con
ven
ti
onal
m
et
ho
d
of
locat
ing
the
distal
locking
ho
le
was
com
plete
ly
reli
ant
on
the
sk
il
ls
of
a
par
ti
cular
su
rg
eon
.
The
su
rg
eon
adj
us
ts
the
ang
le
of
the
fluo
ro
sco
pi
c
syst
em
to
m
ake
a
pr
oj
ect
ion
of
distal
locking
ho
le
im
age
wh
ic
h
app
ears
as
a
ci
rcu
la
r
sh
ape.
This
al
go
rithm
us
es
on
ly
two
fluo
ro
scopi
c
im
ages
t
o
reali
ze
the
distal
locking
-
hole
’s
axis.
The
al
go
rithm
util
iz
es
char
act
erist
ic
info
rm
at
ion
of
the
intram
edu
ll
ary
nail,
su
ch
as,
nail
rad
ius,
m
ajo
r
and
m
ino
r
axes
of
distal
locking
ho
le
,
and
the
area
of
distal
locking
ho
le
as
sh
ow
n
in Fig
ure
3(b).
The
al
go
rithm
is
separ
at
ed
into
two
it
erati
on
s.
The
first
it
erati
on
is
the
sim
ulati
on
pr
ocess.
In
thi
s
pr
ocess,
the
intram
edu
ll
ary
nail
is
ro
ta
te
d
in
al
l
po
ssible
pr
oj
ect
ion
s.
The
visible
pr
oj
ect
ion
of
im
age
app
ea
r
s
at
0
to
30
deg
ree
of
ro
ta
ti
on
.
Then
,
the
im
age
is
captur
ed
at
0,
2,
4,
6,
8,
10
…,
and
30
deg
rees
of
ro
ta
ti
on
t
o
create
sim
ulate
d
im
ages.
All
the
sim
ulate
d
nail
im
ages
are
then
extra
ct
ed
in
a
rati
o
between
the
m
ajo
r
a
nd
m
ino
r
axes
of
the
distal
locking
ho
le
by
us
ing
fitt
ing
el
li
p
se
im
age
pr
ocessing
al
go
rithm
[
21
]
.
So
,
the
rati
o
between
the
m
ajo
r
and
m
ino
r
axis
in
each
ro
ta
ti
on
ang
le
,
wh
ic
h
is
stored
tog
et
her
with
the
X
-
Y
-
Z
Eule
r
ro
ta
t
ion
al
ang
le
in
a
database.
This
par
t
is
do
ne
in
the
pr
e
–
op
erati
ve
pr
ocess.
The
secon
d
it
erati
on
is
f
or
extracti
ng
info
rm
at
ion
fr
om
the
real
-
ti
m
e
distal
locking
-
hole
im
ages.
The
edg
e
detect
ion
and
fitt
ing
el
li
ps
e
te
chn
iqu
es
are
app
li
ed
to
find
a
po
sit
ion
an
d
the
char
act
erist
ic
of
distal
locking
ho
le
on
an
im
age.
The
rati
o
between
m
ajo
r
and
m
ino
r
axis
of
distal
locking
ho
le
is
then
cal
culat
ed
in
real
-
ti
m
e
m
at
ching
pr
ocess
to
recov
e
r
a
distal
locking
-
hole
’s
axis
ro
ta
ti
on
.
The
cub
ic
cur
ve
fitt
ing
is
then
app
li
e
d
to
find
the
best
fit
to
the
data
in
th
e
database.
The
too
l
cur
ve
is
us
ed
to
rev
ersely
reco
ver
the
ro
ta
ti
on
al
ang
le
of
distal
locking
-
hole
’s
axis.
This
ste
p
is per
fo
rm
ed
du
ring
the intra
-
op
erati
ve
pr
ocess
.
2.3.
Fluor
os
c
op
ic
im
ag
e
calibr
ati
on
The
pu
rp
ose
of
fluo
ro
scopi
c
im
age
cal
ibrati
on
is
to
determ
ine
the
im
age
distor
ti
on
par
am
et
ers
and
th
e
im
age
pr
oj
ect
ion
par
am
et
ers.
The
fluo
ro
sco
pi
c
im
age
cal
ibrati
on
m
et
ho
d
req
uires
a
cal
ibrati
on
ph
antom
t
o
create
a
set
of
cal
ibrati
on
po
ints
wh
ic
h
is
us
ed
to
cal
culat
e
th
e
im
age
distor
ti
on
par
am
et
ers
and
the
im
age
pr
oj
ect
ion
par
am
et
ers.
The
cal
ibrati
on
ph
antom
is m
ade o
f
sq
uar
ed
acryl
ic
m
at
erial
wh
ic
h
is em
bed
ded
with
a
do
tt
ed
patte
rn
of
sm
al
l
m
et
al
balls
as
sh
ow
n
in
Fig
ure
4.
The
sp
at
ia
l
distance
within
each
do
t
patte
rn
i
s
2
m
il
li
m
et
ers
in b
oth
h
or
iz
on
ta
l and
v
erti
cal
d
irect
ion
.
Fig
ure
4. The
c
al
ibrati
on pha
nt
om
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
System
integr
at
ion
of a fl
uorosc
opic
im
ag
e
c
alibr
ation u
si
ng r
obot
as
sist
e
d.
.
.
(
Sa
k
ol Na
kdha
mab
horn
)
3743
The
fluo
ro
scopi
c
im
age
has
so
m
e
distor
ti
on
and
the
dig
it
al
im
age
fr
om
fluo
ro
sco
pic
syst
em
is
distor
te
d
by
the
le
ns
of
fluo
ro
sco
pe
and
x
-
ray
so
ur
ce.
The
distor
ti
on
par
am
et
ers
are
then
add
ed
into
th
e
fluo
ro
sco
pic
im
age
to
r
e
-
cor
rect
the
distor
te
d
im
age
[2
2
]
.
In
sect
ion
2.
2,
the
po
sit
ion
and
or
ie
ntati
on
of
distal
locking
ho
le
is
based
on
the
info
rm
at
ion
of
pix
el
im
age
coo
rd
inate
wh
ic
h
is
a
two
-
dim
ension
al
im
age.
In
app
ropri
at
el
y,
the
dr
il
li
ng
trajector
y
is
rep
resen
te
d
in
three
-
dim
ension
al
coo
rd
inate
(w
or
ld
ref
ere
nc
e
coo
rd
inate
).
Then
,
the
fluo
ro
sco
pic
im
age
cal
ibrati
on
is
app
li
ed
to
transf
or
m
and
track
the
relat
ion
sh
ip
betwe
e
n
two
-
dim
ension
al
im
age co
or
din
at
e
and
three
-
dim
ension
al
wo
rld
ref
eren
ce coo
rd
inate
.
The
ob
j
ect
ive
of
the
proj
ect
i
on
cal
ib
rati
on
is
to
dete
rm
ine
a
pro
j
ect
io
n
m
atr
ix
that
tra
nsfo
rm
s
a
three
-
dim
ension
al
point
i
nto
a
t
wo
-
dim
ension
al
pi
xel
point.
T
he
pe
rs
pecti
ve
proj
ect
io
n
(
pinh
ole
cam
era
m
od
e
l)
is
def
i
ned
as
a
ge
om
et
ric
m
od
el
.
Th
e
m
od
el
i
s
ba
sed
on
pr
oject
ion
c
har
act
erist
ic
s
w
hich
are
t
he
fo
cal
le
ng
t
h,
sk
ewi
ng
par
a
m
et
er
an
d
ce
nt
er
of
the
im
a
ge
pro
j
ect
io
n.
The
pro
j
ect
io
n
pa
ram
et
ers
ar
e
co
ns
tr
ucte
d
i
nto
the
m
at
rix
cal
le
d
a
n
“i
nt
rinsic
m
a
trix”.
T
he
t
ransform
at
ion
of
im
age
plane
co
ordinate
t
o
w
orl
d
c
oor
din
at
e
wh
i
c
h
is descri
be
d
in
the tra
ns
f
or
m
ation
m
at
rix
cal
le
d
a
n
“e
xtrin
si
c m
a
trix”. T
he pin
-
hole
im
age p
r
oject
io
n
is s
how
n
in (1).
[
p
p
1
]
=
[
f
x
u
0
0
0
f
y
v
0
0
0
0
1
0
]
[
11
12
13
x
21
22
23
y
31
32
33
z
0
0
0
1
]
[
w
w
w
1
]
(1)
W
her
e
f
x
is
a
fo
cal
le
ng
th o
n
x
axis,
f
y
is
a
fo
cal
le
ng
th o
n
y
axis, w
hich
are
scal
ed
in
m
il
li
m
et
ers
resp
ect
ively
,
is
a
sk
ewing
par
am
et
er,
u
0
and
v
0
is t
he c
enter
of
the
im
age in
pix
el
,
r
and
t
are
translat
ion
and
or
ie
ntati
on
of
the
im
age
plane.
The
intrinsic
m
at
rix
and
extrinsic
m
at
rix
in
(
1)
are
com
bin
ed
into
a
pr
oj
ect
ion
m
at
rix
P
a
nd
rew
ritt
en
as (
2)
.
The
pro
j
ect
ion
m
at
rix
P
is
the
trans
form
ation
m
at
rix,
f
or
t
ransform
ing
a
w
or
ld
co
ordi
nate
po
int
(
X,
Y,
Z)
into
a
pix
el
coor
din
at
e
poi
nt
(
u,
v)
.
The
m
at
rix
P
can be
est
i
m
at
ed
by
us
in
g
the d
irec
t
li
near
trans
f
orm
ation
(D
L
T)
al
gorith
m
[
23
]
.
The
ext
rinsic
m
at
rix
is
the
posit
io
n
a
nd
or
ie
ntati
on
of
the
fl
uorosco
pi
c
i
m
age
coordi
nate
fr
am
e
relat
ive
t
o
t
h
e
w
or
l
d
c
oor
din
at
e.
T
he
intrinsic
pa
ram
et
er
c
on
sist
s
of
a
f
ocal
le
ngth
of
the
cam
era,
scal
ing
factors i
n t
he x
and y
directi
on a
nd the ce
nte
r of
t
he
im
age p
la
ne
i
n pixel.
[
1
1
1
1
0
0
0
0
–
1
1
–
1
1
–
1
1
−
1
0
0
0
0
1
1
1
1
–
1
1
–
1
1
–
1
1
−
1
2
2
2
1
0
0
0
0
–
2
2
–
2
2
–
2
2
–
2
0
0
0
0
2
2
2
1
–
2
2
–
2
2
–
2
2
−
2
.
.
.
n
n
n
1
0
0
0
0
–
n
n
–
n
n
–
n
n
–
n
0
0
0
0
n
n
n
1
–
n
n
–
n
n
–
n
n
−
n
]
[
11
12
.
.
.
34
]
=
0
(2)
2.4.
Fluor
os
c
op
ic
im
ag
e
regis
tration
a
n
d
tra
c
k
ing
In
this
syst
em
,
the
opti
cal
trac
king
syst
em
is
def
i
ned
as
a
re
f
eren
ce
wo
rl
d
c
oor
din
at
e.
The
syst
e
m
us
es
a
com
m
ercial
op
ti
cal
trac
king
“
Po
la
ris
Vic
ra”
from
Northern
Digital
Inc.
T
he
opti
cal
Track
i
ng
syst
e
m
is
a
syst
e
m
fo
r
m
easur
em
ent
of
th
e
posit
io
n
a
nd
or
ie
ntati
on
in
a
3
-
dim
ension
al
act
ive
or
pas
sive
m
ark
e
r.
The
act
ive
m
ark
er
is
m
ade
up
of
in
fr
a
re
d
li
ght
-
em
it
t
ing
a
nd
t
he
wirel
ess p
assive
m
ark
e
r
is
m
ade
up of
a
r
eflect
iv
e
s
ph
e
re
and
there
f
or
e,
the
ste
re
o
cam
era
recei
ves
li
gh
t
from
the
m
ark
er
ref
le
ct
ion
or
m
ark
er
e
m
issi
on
.
T
he
op
ti
cal
trackin
g
syst
e
m
can
identify
an
d
rec
ognize
the
m
ark
ers
by
the
dif
fer
e
nt
patte
rn
of
m
ark
ers
sh
a
pe.
T
he
n,
t
he
op
ti
cal
tracki
ng
syst
em
pr
ovides
a
th
ree
-
di
m
ension
al
pos
it
ion
an
d
or
ie
nt
at
ion
of
the
m
ark
er
relat
ed
to
the
op
ti
cal
trac
king
c
oor
din
at
e.
The
posit
ion
is
co
ns
tr
ucted
i
n
3x
1
po
sit
io
n
ve
ct
or
a
nd
the
ori
entat
ion
is
s
pe
ci
fied
as
3x
3
ro
ta
ti
on
m
at
rix.
Th
e
posit
ion
vect
or
is
com
bin
e
d
with
ro
ta
ti
on
m
at
rix
to
cr
eat
e
the
hom
og
en
ous
trans
form
ation
m
at
rix
in
(3)
w
hich
is
re
pr
e
se
nted
a
s
a
t
ran
s
f
or
m
at
ion
bet
w
een
m
ark
er
co
ordi
nate
an
d
t
he
op
ti
cal
trackin
g refe
re
nce c
oor
din
at
e.
=
[
3
3
3
1
0
0
0
1
]
(3)
H
is ho
m
og
eno
us
tran
sfo
rm
at
ion
;
R
is
3x3
ro
ta
ti
on
m
at
rix;
T
is
3x1
po
sit
on
v
ect
or
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
om
p
En
g,
V
ol.
9
, N
o.
5
,
Oct
ober
20
19
:
3739
-
3750
3744
Con
sequ
ently
,
the
ho
m
og
eno
us
transf
or
m
at
ion
can
be
determ
ined
by
at
ta
ched
m
ark
ers
to
the
ob
j
ec
t
.
In
the
syst
em
,
the
m
ark
er
is
at
ta
ched
with
fluo
ro
sco
pic
syst
em
and
Rob
ot
-
assist
ed
su
rg
ic
al
gu
idance.
Th
e
ho
m
og
eno
us
transf
or
m
at
ion
relat
ion
sh
ip
between
each
ob
j
ect
can
be
determ
ined
with
the
ho
m
og
e
no
us
transf
or
m
at
ion
r
el
at
ion
sh
ip
[24]
as il
lustrate
d
in
Figu
re 5
.
Fig
ure
5. The
ho
m
og
e
nous
tr
ansfo
rm
ation
re
la
ti
on
sh
i
p of t
he
cal
ib
rati
on
and trac
king
The
flu
orosc
opic
i
m
age
co
ord
ina
te
is
de
fine
d
as
c
oord
i
nate
{C
}.
The
hom
og
e
nous
tra
ns
f
or
m
at
ion
of
i
m
age
co
ordina
te
to
opti
cal
tra
ckin
g
ref
e
re
nce
co
ordi
nate
is
.
The
co
ordin
at
e
{M
}
represe
nts
the
fl
uor
os
co
pic
syst
e
m
coord
i
nate
wh
ic
h
is
the
ho
m
og
e
no
us
tran
sf
or
m
ation
of
opti
cal
trackin
g
re
fer
e
nce
c
oor
din
at
e
to
flu
orosc
op
ic
syst
e
m
coord
i
nat
e.
T
he
is
def
i
ne
d
a
s
t
he
sta
ti
c
hom
og
en
ous
tra
ns
f
or
m
at
ion
of
coor
din
at
e
{M
}
to
c
oord
i
nate
{
C}
w
hen
e
ve
r,
the
fl
uor
os
co
pi
c
syst
em
m
ov
es,
the
sta
ti
c
ho
m
og
enous
t
ransform
at
ion
will
be
c
onsta
nt.
On
ce
the
cal
ib
r
at
ion
proces
s
i
n
sect
i
on
2.
1
is
com
plete
d,
the
intrinsic
a
nd
e
xt
rinsic
m
at
rix
is
cal
culat
ed
.
The
intrin
sic
pa
ram
et
er
is
a
s
pecific
pro
j
ect
ion
par
am
et
er
of
flu
orosc
op
ic
i
m
age
to
pro
j
e
ct
an
ob
j
ect
int
o
a
n
i
m
age
plan
e.
T
he
e
xtri
ns
ic
m
a
trix
is
a
hom
og
enous
tra
ns
f
or
m
at
ion
of
the
im
age
co
ordi
na
te
relat
ed
to
the
opti
cal
trackin
g
c
oor
di
nate w
hich
is e
qu
al
t
o
. Th
e
n, the
is cal
culat
ed
by
(
4).
=
−
1
−
1
(4)
In
intra
-
op
erati
ve,
the
fluo
ro
scopi
c
syst
em
is
m
ov
ed
to
captur
e
a
distal
locking
-
hole
im
age
so
th
e
fluo
ro
sco
pic
im
age
coo
rd
inate
is
chan
ged
as
sh
ow
n
in
Fig
ure
6(
b)
.
In
or
der
to
track
the
transf
or
m
at
ion
of
th
e
fluo
ro
sco
pic
im
age
coo
rd
inate
{C
},
the
sta
ti
c
ho
m
og
eno
us
transf
or
m
at
ion
is
us
ed
to
cal
culat
e
the
new
or
n
ew
extrinsic m
at
rix
by (
5)
=
−
1
−
1
(5)
Hen
ce,
the
pin
-
ho
le
im
age
pr
oj
ect
ion
in
(
1)
can
be
com
plet
ed
with
a
new
extrinsic
m
at
rix.
The
tracking
pr
ocess
up
dates
a
new
extrinsic
m
at
rix
to
create
a
new
pr
oj
ect
ion
equ
at
ion
.
Ther
efo
r
e,
the
syst
em
can calculat
e
a
three
-
dim
ension
al
(X, Y, Z
)
po
sit
ion
in
ref
eren
ce
coo
rd
inate
by inp
utti
ng
the
two
-
dim
ensiona
l
im
age
pix
el
(u
,
v)
coo
rd
inate
.
The
equ
at
ion
of
the
relat
ion
between
two
-
dim
ension
al
po
sit
ion
and
the
three
-
dim
ensi
on
al
p
os
it
ion
are
sh
ow
n
in (
6)
an
d
(7
).
=
x
11
+
12
+
13
+
1
31
+
32
+
33
+
3
+
x
(6)
=
y
21
+
22
+
23
+
2
31
+
32
+
33
+
3
+
y
(7)
Wh
e
re
u, v
ar
e
the in
pu
t
of the
p
ixel
posit
ion
on the im
age
,
f
x
is a f
ocal le
ng
th on
x
axis
,
f
y
is a f
ocal le
ngt
h on
y
axis,
c
x
a
nd
c
y
is t
he
ce
nter o
f
the
im
age in
p
ixel,
R
a
nd
t
a
re tra
ns
la
ti
on a
nd orienta
ti
on
of the im
age p
l
ane.
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
System
integr
at
ion
of a fl
uorosc
opic
im
ag
e
c
alibr
ation u
si
ng r
obot
as
sist
e
d.
.
.
(
Sa
k
ol Na
kdha
mab
horn
)
3745
2.5.
Robot
gu
id
ance
sy
s
tem
In
t
he
s
urgical
nav
i
gation
syst
e
m
,
our
rob
otics
guidi
ng
syst
e
m
is
central
t
o
the
pr
ocess.
This
syst
em
interact
s
with
the
sur
geon
by
us
in
g
la
ser
beam
to
ov
e
rlay
the
distal
locki
ng
dri
ll
ing
path
.
T
he
gu
idin
g
inf
or
m
at
ion
of
the
se
ver
al
na
vig
at
io
n
syst
e
m
s
are
disp
la
ye
d
via
the
m
on
it
or
by
dem
on
s
trat
ing
t
he
a
nim
at
ion
gr
a
phic
s
on
the
scree
n.
C
onve
rsely
,
a surg
e
on
w
ould
prefe
r
durin
g
t
he
op
erati
on,
to
p
ay
m
or
e
at
te
ntion
to
th
e
su
r
gical
sit
e
than
on
the
sc
reen.
T
he
pr
opose
d
syst
em
el
i
m
inate
s
the
ha
nd
-
ey
e
co
ordinati
on
pro
blem
.
The
r
obot
c
ons
ist
s
of
f
our
de
gr
ees
of
f
reedo
m
,
wh
ic
h
a
re
two
tra
ns
la
ti
on
j
oi
nts
a
nd
two
ro
ta
ti
on
joints.
T
he
rob
ot
is
at
ta
ched
at
the
Fl
uor
os
c
op
ic
syst
e
m
(C
-
Ar
m
)
as
sh
ow
n
in
Fig
ure
6
(a).
A
fter
t
he
na
vi
gation
syst
e
m
gen
e
rates
th
e
t
raj
ect
or
y
path
,
the
r
obot
is
pointi
ng
a
la
ser
be
a
m
at
the
entr
y
locat
ion
w
hile
path
ori
entat
ion
i
s
al
so
der
ive
d by
co
m
par
in
g
the
dril
li
ng
to
ol a
xis a
nd the lase
r beam
ax
is
.
(a)
(b)
Figure
6
.
(
a
)
C
AD
desi
gn
m
od
el
of t
he
r
obot
-
assist
ed (
b)
T
he ro
bot
-
assist
ed
at
ta
ches
w
it
h
the
C
-
Arm
The
plana
r
li
ne
ar
tra
ns
la
ti
on
m
ov
e
m
ent
is
ge
ner
at
e
d
by
two
tra
ns
la
ti
on
jo
ints,
w
hic
h
is
i
m
ple
m
ented
by
a
li
near
guidance
m
ov
em
e
nt
d
rive
n
by
a
belt
m
echan
ism
.
The
two
r
ot
at
ion
m
ov
em
e
nts
are
us
e
d
to
adjust
the
ori
entat
io
n
of
la
ser
beam
al
ign
ed
wit
h
distal
lo
ckin
g
a
xi
s.
The
e
nd
-
ef
f
ect
or
of
the
r
obot
is
at
ta
ched
with
a
la
ser
pointe
r
in
o
r
de
r
to
overla
y t
he
sur
gical
dril
li
ng
path. Th
e rob
ot is
portable an
d
ca
n
eas
il
y
m
ou
nted o
n t
he
flu
orosc
op
ic
sy
stem
.
The
low
-
le
vel
con
t
ro
ll
er
us
es
the
P
IC
m
ic
ro
co
ntro
ll
e
r
with
a
po
sit
io
n
co
ntr
oller
to
con
t
ro
l
the D
C
m
oto
rs and t
he se
r
vo
m
oto
rs
[
25
]
.
3.
RESU
LT
S
A
ND AN
ALYSIS
The
fo
ll
owin
g
exp
e
rim
ent
is
separ
at
e
d
in
tw
o
sect
io
ns
.
Firs
tl
y,
the
distal
locki
ng
-
ho
le
a
xi
s
recove
ry
al
gorithm
wh
ic
h
is
im
ple
m
ented
t
o
recove
r
t
he
ro
ta
ti
on
of
t
he
dr
il
li
ng
tra
je
ct
or
y
a
nd
the
seco
nd
ex
pe
rim
ent
is
integrate
d
with
the
first
e
xperi
m
ent
so
as
t
o
c
om
plete
the
sys
tem
integrati
on
of
rob
ot
-
assist
ed
s
urgical
gu
i
dan
ce
.
3.1.
Experim
en
t o
n
dist
al loc
kin
g
h
ole’s
axis r
otat
i
on
rec
ove
ry
In
t
he
e
xp
e
rim
ent, the
al
go
rithm
o
f
distal
lo
ckin
g ho
le
a
xis
’s
r
otati
on r
ec
overy
was
im
ple
m
ented
in a
flu
orosc
op
ic
syst
e
m
.
The
m
odel
of
flu
orosc
opic
syst
em
is
a
Ph
il
ips
BV
P
ulsera.
T
he
flu
orosc
op
ic
im
ages
we
re
captu
red
via
B
NC
port
at
the
back
pan
el
of
the
flu
orosc
opic
syst
em
.
Then,
the
US
B
vi
de
o
fr
am
e
grab
be
r
was
us
e
d
t
o
ca
pture
the
sig
nal
a
nd
the
n
sent
to
th
e
com
pu
te
r
f
or
pro
cessi
ng.
T
he
te
sti
ng
im
a
ge
was
ca
ptu
re
d
at
5,
10,
15,
20
an
d
25
de
gr
ees
of
ro
ta
ti
on
of
the
intram
edu
ll
ary
nail
as
show
n
in
Fig
ur
e
7
.
T
he
im
age
pr
oc
essin
g
te
chn
iq
ue
ext
ra
ct
ed
in
f
or
m
at
ion
o
f
th
os
e
fl
uo
ro
sc
opic
im
ages
to
m
ake
a d
at
aset
of
“
To
ol
c
urve”
w
hich
is t
hen
store
d
in
the
da
ta
base.
T
his
process
was d
on
e duri
ng the
pre
-
operati
ve
sta
ge.
The
fl
uorosco
pic
i
m
age
was
captur
e
d
10
ti
m
es
in
each
de
gr
ee
ro
ta
t
io
n
duri
ng
the
int
r
a
-
operati
on.
The
ac
quire
d
i
m
age
was
ext
r
act
ed
a
featu
re
of
distal
locki
ng
hole
by
im
a
ge
processi
ng
te
chn
i
qu
e
wh
ic
h
the
n
cal
culat
ed
t
he
degree
of
r
otati
on
of
C
-
a
rm
by
us
i
ng
c
urve
fit
ti
ng
e
quat
io
n.
The
cal
culat
io
n
proces
s
us
e
d
a
cu
r
ve
fitt
ing
e
qu
a
ti
on.
The
res
ult
of
C
-
a
rm
deg
ree
of
ro
ta
t
ion
was
s
hown
on
the
gr
a
ph
ic
us
e
r
i
nterf
ace.
The
Fig
ur
e
8
il
lustrate
s
the
i
m
age
res
ult,
w
hich
sho
wn
on
t
he
gr
a
ph
ic
us
er
inte
rf
a
ce
of
the
pro
gr
am
.
The
ex
pe
rim
en
t
was
c
onduct
e
d
fifty
ti
m
es
w
it
h
fi
ve
dif
fe
re
n
t
de
gr
ee
s
of
r
ot
a
t
ion
(
5
o
,
10
o
,
15
o
,
20
o
,
an
d
25
o
).
The
s
umm
ary
of
the
res
ult
is
sho
wn
in
T
ab
le
1.
The
ave
r
age
a
ngular
er
ror
is
1.1
5
degree
of
i
ntram
e
du
ll
a
r
y
nail r
otati
on
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
om
p
En
g,
V
ol.
9
, N
o.
5
,
Oct
ober
20
19
:
3739
-
3750
3746
Figure
7
.
Th
e
distal
lock
i
ng
ho
le
im
ages in
each
degree
of
ro
ta
ti
on
Fig
ure
8.
The
Gr
a
phic
u
se
r
i
nterf
ace
of t
he dist
al
lock
i
ng
ho
le
rec
ov
e
ry
pro
gr
am
Table
1
.
T
he
s
umm
ary of
a
ng
ular
e
rror o
f
i
ntram
edu
ll
ary na
il
r
otati
on
Deg
ree
o
f
intra
m
e
d
u
lary
n
ail Ro
tatio
n
The p
redictio
n
ang
u
lar
er
ror in d
eg
re
e of
in
tr
m
ed
u
llar
y
nail
rotatio
n
Max
Min
Av
erage
SD
5
1
.81
1
0
.02
6
0
.84
2
6
0
.70
7
3
10
1
.49
6
0
.04
0
.58
1
9
0
.46
4
4
15
1
.22
5
0
.27
5
0
.64
0
6
0
.47
1
1
20
2
.66
3
0
.87
1
1
.63
2
0
.59
8
8
25
3
.94
5
0
.08
6
2
.07
5
4
1
.36
8
8
to
tal
3
.94
5
0
.02
6
1
.15
4
5
0
.97
0
8
3.2.
Experim
en
t o
n f
lu
or
os
co
py
calibra
tion a
n
d system
inte
gr
at
i
on
of ro
bot
-
as
siste
d
gu
id
an
ce
The
cal
ibrati
on
m
et
ho
d
was
s
epar
at
e
d
i
nto
2
ope
rati
on
s
duri
ng
the
e
xp
e
rim
ent
into
p
re
-
op
e
rati
ve
a
nd
intra
-
op
e
rati
ve
op
e
rati
on.
T
he
intra
-
op
e
rati
ve
ex
per
im
ent
pr
e
form
s
2
ste
ps
of
fl
uorosco
pic
i
m
age
cal
i
br
at
io
n.
The
first
step
is fl
uorosco
pic
i
m
age d
ist
ort
io
n
cal
ib
rati
on a
nd the
sec
ond s
te
p
is
flu
orosc
op
ic
im
age p
r
oject
io
n
cal
ibrati
on.
T
he
ex
per
im
ent
set
up
is
s
how
n
i
n
Fi
gure
9
.
T
he
opti
cal
trackin
g
syst
em
is
pla
ced
dow
n
be
hi
nd
the
flu
orosc
op
ic
s
yst
e
m
fo
r
set
ti
ng
up
th
ree
-
di
m
ension
al
wor
ld
ref
e
re
nce
c
oor
din
at
e.
The
op
ti
cal
m
ark
er
and
t
he
rob
ot
-
assist
ed
su
r
gical
guid
an
ce sy
stem
are
a
tt
ached
t
o
the
fl
uoro
sc
opic
sy
stem
.
Evaluation Warning : The document was created with Spire.PDF for Python.
In
t J
Elec
&
C
om
p
En
g
IS
S
N: 20
88
-
8708
System
integr
at
ion
of a fl
uorosc
opic
im
ag
e
c
alibr
ation u
si
ng r
obot
as
sist
e
d.
.
.
(
Sa
k
ol Na
kdha
mab
horn
)
3747
Figure
9
.
Inte
grat
ion o
f
the
ro
bo
t
-
assist
ed su
rg
ic
al
gu
i
dan
ce
syst
e
m
3.2.1.
Pre
-
op
e
rat
i
ve
a.
STE
P I:
Fluor
os
c
op
ic
ima
ge
dist
ort
i
on
c
al
ibrati
on
The
im
age
distor
ti
on
cal
ibrat
ion
was
im
ple
m
ented
to
dete
rm
ine
distor
ti
on
co
-
ef
fici
ent
par
am
et
ers.
The
distor
ti
on
co
-
e
ff
ic
ie
nt
para
m
et
ers were
use
d t
o c
orrect
t
he dist
or
ti
on
of
the
flu
orosc
op
ic
i
m
age. The
s
et
of
ca
li
br
at
ion
im
a
ges
we
re
c
orre
ct
ed
in
di
ff
e
re
nt
posit
ion
a
nd
or
ie
ntati
on.
T
he
im
age
pr
oc
essing
te
c
hn
i
que
was
app
li
ed
to
seg
m
ent
a
posit
io
n
of
t
he
cal
ib
ra
ti
on
po
i
nt
in
a
n
i
m
age
w
hich
c
orres
ponds
with
the
sp
at
ia
l
posit
ion
of
cal
ib
rati
on
po
i
nts
on
the
a
cryl
ic
ph
a
nto
m
.
Twel
ve
cal
ib
rati
on
phant
om
i
m
ages
were
acqu
i
red
t
o
ge
ner
at
e
300
c
orres
pondin
g
cal
ibrati
on
points
.
The
c
orres
pondin
g
c
al
ibrati
on
point
s
wer
e
us
e
d
to
est
i
m
at
e
the
distor
ti
on
par
am
et
ers.
T
he
r
es
ult o
f
c
orr
ect
ed
im
age
is
sh
ow
n
in
Fi
gur
e
1
0
.
Fig
ure
10
. (a)
Distorte
d
cal
ib
rati
on im
age (
b)
un
-
dist
or
te
d c
al
ibrati
on im
ag
e
b.
STE
P II: Fl
u
or
os
c
op
ic
ima
ge
projecti
on
ge
omet
r
y
c
alib
rat
i
on
The
distor
ti
on
par
am
et
er
in
ste
p
I
was
a
pp
li
e
d
to
c
orrect
the
flu
orosc
op
ic
c
al
ibrati
on
im
age,
then
t
he
un
-
distor
ti
on
i
m
age
was
se
gm
ented
to
the
cal
ibrati
on
po
int
posit
ion
on
the
im
age
to
create
a
set
of
t
wo
dim
ension
al
co
rr
es
pondin
g p
oi
nts. At
the
sa
m
e
tim
e,
the
op
ti
cal
trackin
g
syst
e
m
pr
ov
i
de
d a
three
dim
e
ns
io
nal
po
sit
io
n
of
cal
i
br
at
io
n
po
i
nt
on
the
cal
ib
rati
on
pha
nto
m
to
gen
e
rate
a
set
of
t
hr
ee
dim
ension
al
c
orres
pond
i
ng
po
i
nts.
The
sys
tem
acqu
ire
d
12
cal
ib
rati
on
i
m
ages
to
creat
e
30
0
cal
ib
rati
on
po
i
nts
for
es
tim
a
ti
ng
the
proj
ect
i
on
m
at
rix.
T
he
pro
j
ect
io
n
m
at
rix
was
dec
ompo
s
ed
to
intri
ns
ic
par
am
et
ers
an
d
ext
rinsi
c
pa
ram
et
ers
wh
ic
h
descr
i
bed
a
s
pr
oj
ect
io
n
(
1). T
hen, the
sta
ti
c h
om
og
e
neous
t
ran
s
f
or
m
at
ion
(
)
can b
e
determ
ined
by
(
4)
. T
he
com
plete
d
so
luti
o
n
res
ult
of
pro
j
ect
ion
e
quat
ion
was
s
how
n
in
(
8)
a
nd
the
resu
lt
of
(
)
ho
m
og
e
ne
ous
trans
form
ation
was rep
rese
nted
in
(
9).
[
1
]
=
[
1720
0
216
.
75
0
0
1556
.
7
958
.
15
0
0
0
1
0
]
[
0
.
292
−
0
.
774
0
.
561
695
.
366
−
0
.
640
0
.
277
0
.
7158
970
.
973
−
0
.
7099
−
0
.
568
−
0
.
415
−
734
.
348
0
0
0
1
]
[
w
w
w
1
]
(8)
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2088
-
8708
In
t J
Elec
&
C
om
p
En
g,
V
ol.
9
, N
o.
5
,
Oct
ober
20
19
:
3739
-
3750
3748
=
[
−
0
.
021
0
.
428
−
0
.
903
−
274
.
522
0
.
993
−
0
.
092
−
0
.
067
−
33
.
893
−
0
.
112
−
0
.
898
−
0
.
423
109
.
1282
0
0
0
1
]
(9)
3.2.2.
I
nt
r
a
-
oper
at
i
ve
Af
te
r
the
cal
ib
rati
on
pro
cess
in
pre
-
operati
ve
sta
ge
was
do
ne,
t
he
disto
rtion
pa
ram
et
ers,
pro
j
ect
io
n
geo
m
et
ry
par
a
m
et
ers
and
t
he
sta
ti
c
ho
m
og
e
nous
t
ran
s
f
or
m
at
ion
(
)
we
re
im
plem
ented
in
intra
-
operati
on
.
The
fl
uor
os
c
opic
syst
e
m
was
m
ov
ed
to
ca
ptu
re
a d
ist
al
loc
king
hole
im
age
then
t
he
dist
ort
ion
p
a
ram
et
e
rs
wa
s
us
e
d t
o
co
rr
e
ct
the
distor
ti
on
of
a
n i
m
age. T
he
distal
loc
king
-
hole
recovery
al
gorithm
recovere
d a
n
ori
ent
at
ion
of
distal
locki
ng
hole
’
a
xis
then
e
xtracte
d
distal
locki
ng
-
hole
posit
io
n
on
a
n
im
age.
The
trac
king
s
yst
e
m
pro
vid
e
d
a
real
-
tim
e
ho
m
og
e
nous
tra
nsfo
rm
a
ti
on
of
fl
uor
os
c
op
ic
syst
em
to
cal
culat
e
a
new
extri
ns
ic
m
at
ri
x
for
const
ru
ct
in
g
a
pro
j
ect
ion
e
quat
ion
t
he
n
t
he
pr
eci
se
th
ree
-
dim
ension
al
pos
it
ion
a
nd
or
ie
nt
at
ion
of
distal
l
ock
i
ng
ho
le
was
cal
culat
ed
by
s
olvi
ng
t
he
pro
j
ec
ti
on
e
qu
at
i
on.
The
r
obot
-
as
s
ist
ed
sur
gical
gu
i
dan
ce
recei
ved
a
three
-
dim
ension
al
posit
io
n
a
n
or
ie
ntati
on
to g
ene
rate
a dril
li
ng
t
raj
ect
or
y p
at
h
f
or
guidi
ng
a
sur
geon
a
s
s
how
n
in Figu
re
9
.
The
syst
em
was
te
ste
d
with
5
diff
e
re
nt
intr
a
m
edu
ll
ary
nai
l
ro
ta
ti
ons
(5,
10,
15,
20
a
nd
25
de
gr
e
e
)
.
The
ex
pe
rim
en
ts
we
re
co
nduc
te
d
with
the
5
diff
e
re
nt
degre
es
of
r
otati
on
t
en
tim
es
each
r
especti
vely
,
m
akin
g
a
total
of
fifty
e
xp
e
rim
ents
tha
t
we
re
c
onduct
ed.
T
he
tra
ns
la
ti
on
e
rro
r
i
n
ea
ch
a
xis
is
s
hown
in
Fi
g
ure
s
1
1
-
13.
The
s
patia
l dist
ance e
rror i
n
X
-
Y
plane
is
s
umm
arized in Ta
bl
e
2.
Figure
11. T
he
r
es
ult o
f po
sit
ion er
ror
i
n X
directi
on
Figure
12.
T
he
r
es
ult o
f po
sit
ion er
ror
i
n Y
di
recti
on
Figure
13. T
he
r
es
ult o
f po
sit
ion er
ror
i
n
Z
di
recti
on
Table
2.
T
he
s
umm
ary of
sp
a
ti
al
d
ist
ance erro
r
in
x
-
y
plan
e.
Sp
atial dis
tan
ce e
r
r
o
r
in
X
-
Y
plan
e (
m
m
)
X
Y
Euclid
ean Dis
tan
c
e(
m
m
)
Min
0
.07
3
0
.05
2
0
.09
0
Max
4
.21
1
9
.59
3
1
0
.47
Mean
1
.60
7
3
.28
0
3
.65
2
Evaluation Warning : The document was created with Spire.PDF for Python.