TELKOM
NIKA
, Vol. 13, No. 4, Dece
mb
er 201
5, pp. 1289
~1
297
ISSN: 1693-6
930,
accredited
A
by DIKTI, De
cree No: 58/DIK
T
I/Kep/2013
DOI
:
10.12928/TELKOMNIKA.v13i4.3103
1289
Re
cei
v
ed Se
ptem
ber 16, 2015; Revi
se
d No
vem
ber
2, 2015; Acce
pted No
vem
b
er 14, 201
5
An Automatic Calibration Method for Near-infrared
Camera in Optical Surgical Navigation
Rong Qia
n
Yang
1
, Xuan Si
1
, Qin Yong
Lin
1
, Ken Cai
*
2
1
Departme
n
t of Biomed
ical En
gin
eeri
ng, Sout
h Chi
na Un
iver
sit
y
of T
e
chnol
og
y, Guan
gzho
u, 5100
06,
Chin
a
2
School of Infor
m
ation Sci
enc
e and T
e
chno
l
o
g
y
, Z
hongk
ai
Univers
i
t
y
of Agricult
ure an
d Engi
neer
in
g,
Guangz
ho
u, 5102
25, Ch
ina
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: icken@
126.c
o
m
A
b
st
r
a
ct
Optical surgic
al navi
gati
on
system (SNS)
w
i
th near-infrared trackin
g
system is beco
m
i
n
g
extensiv
ely us
ed in c
lin
ics, and th
e accur
a
cy of SNS
is
influe
nce
d
by
the cali
bratio
n of near-
i
nfra
re
d
cameras (
N
IR
Cs). W
e
pro
p
o
s
e a
n
a
u
to
mat
i
c cal
i
brati
o
n
meth
od
for NI
RCs. T
he
method
is
base
d
on
a
desi
gne
d cal
i
b
r
ation b
oar
d. In our
exp
e
ri
me
nts, corner
s are auto
m
a
t
ically extracte
d to obtai
n the
para
m
eters of NIRCs. T
h
is meth
od h
a
s the adva
n
tag
e
s
of saving ti
me,
efficiency in c
o
mputati
on, hi
gh
accuracy,
and
reli
abi
lity. In
our
exper
i
m
en
ts, an NI
RC
c
an
be c
a
li
brat
ed i
n
only
5 s
.
Meanw
hi
le, t
h
e
avera
ge rel
a
tiv
e
errors of the focal l
engt
h and
princi
pal p
o
i
n
t are 0.87
% an
d 1.39%, resp
ect
i
vely.
Ke
y
w
ords
: Ca
mer
a
cali
brati
o
n, Cali
bratio
n b
oard,
Cor
ner e
x
traction, Near
-infrare
d ca
mer
a
Copy
right
©
2015 Un
ive
r
sita
s Ah
mad
Dah
l
an
. All rig
h
t
s r
ese
rved
.
1. Introduc
tion
Came
ra
calib
ration i
s
an i
m
porta
nt issue in
bino
cul
a
r visio
n
syst
em. The pu
rpose of
came
ra
calib
ration i
s
to
determi
ne th
e mappi
ng t
r
an
sform
a
tion
betwe
en im
age a
nd world
coo
r
din
a
tes
of object
s
[1]. In surgical
navi
gation system, the bino
cula
r vision system i
s
comp
osed of
two ne
ar-inf
rared
came
ras
(NI
R
Cs
) [
2
-5]. The
preci
s
ion
of NI
RC
cali
bratio
n
determi
ne
s th
e perfo
rma
n
ce of the entire navigati
on sy
st
em.
A
s
s
u
ch,
NI
R
C
s must
be cali
b
r
at
ed
acc
u
rately.
Came
ra
cali
bration
meth
ods fo
r visi
ble came
ra
s are
gen
era
lly divided i
n
to two
categ
o
rie
s
, n
a
mely, traditional calibration method a
nd self-cali
b
ration method
[6]. The traditional
calib
ration
m
e
thod
ba
sed
on a
pattern
has hig
h
p
r
e
c
isi
on
and
usually ha
s two
stag
es [7–1
1],
namely, direct linear trans
f
ormation and nonlin
ear optimiz
a
tion. The method proposed by Zhang
is m
o
re
flexibl
e
be
ca
use it
use
s
a
plana
r pattern [
12].
A limitation
of the tradition
a
l
metho
d
i
s
th
at
it
need
s
a
calibratio
n
p
a
ttern with
a kno
w
n struct
ure.
T
he self
-cali
b
ration method ha
s low
pre
c
isi
on, b
u
t it is
extensi
v
ely use
d
b
e
c
au
se
it
dire
ctly extra
c
ts
environ
menta
l
inform
ation
as
calib
ration inf
o
rmatio
n [13].
The NIRC is
comp
osed of a visible ca
m
e
ra
an
d nea
r-infrared filter, so the pattern use
d
in
the
traditio
nal calib
ratio
n
meth
od ca
n
not
be
sen
s
e
d
by NIRCs. Several studi
es have
u
s
ed
the
calib
ration
re
sult of a cam
e
ra
without filter as t
he cali
bration re
sult
of
NIRCs. Other research
ers
use
d
an external light sou
r
ce to en
su
re
that
the NIRCs
sen
s
e the
calibration b
oard, such a
s
a
che
c
kerboa
rd
. Ho
wever, th
e traditio
nal
calibrati
o
n
met
hod can
not meet
the nee
d
for accu
ra
cy,
and the self-calibratio
n
met
hod is influ
e
n
c
ed by li
ght from the su
rro
undin
g
s.
Thi
s
study pro
poses
a dire
ct NI
RC calib
ration m
e
thod u
s
ing
a
n
NIRC
calibration boa
rd,
whi
c
h con
s
ist
s
of 64 (8 ×
8)
near-infrared surfa
c
e
-
mo
un
ted diode
s (NIR-SMDs).
In our
previous
wo
rk [1
4], we
de
signe
d
a calibration
boa
rd, which
has 64
(8
×
8)
NIR-
SMDs o
n
a bread
boa
rd. In the present study, the
cal
i
bration b
oard is improve
d
.
We desig
n the
calib
ration
bo
ard
by con
s
tructing
a
print
ed
circ
uit b
o
a
rd
(PCB
) u
s
ing a
5 V
direct
cu
rre
nt (DC)
power a
dapt
er a
s
the
su
pply voltage,
whi
c
h e
n
su
res th
at the
curre
n
t flowing thro
ugh
the
calib
ration
b
oard
is m
o
re
stable to
a
c
hieve m
o
re
accu
rate
ca
libration
re
su
lts. If the exact
geomet
ric inf
o
rmatio
n of the boa
rd i
s
u
n
kn
own,
the board cannot
be used dire
ctly for calib
rating
NIRCs.
We u
s
e a
cali
brate
d
bino
cul
a
r vi
sion
syst
e
m
compo
s
ed
of two visi
ble
ca
mera
s to o
b
tain
the geom
etri
c information
of the boa
rd
. Our p
r
eviou
s
work can u
s
e the
de
sig
ned b
oard a
n
d
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
9
30
TELKOM
NIKA
Vol. 13, No
. 4, Decem
b
e
r
2015 : 128
9 – 1297
1290
existing calib
ration meth
o
d
to calib
rate
the NI
RC
s
.
H
o
w
e
ve
r
,
huma
n
–c
o
m
p
u
te
r
in
te
ra
c
t
ion i
s
need
ed du
rin
g
the pro
c
ed
ure. The o
p
e
r
ator h
a
s to
sele
ct the co
rne
r
s in the
calib
ration b
o
a
rd
image m
anu
ally, which
mean
s that
the entire p
r
oce
s
s is tire
some
an
d time con
s
umi
ng.
More
over, the method can
not implem
e
n
t
automation for cali
bration.
This
study m
a
inly co
nsi
d
e
r
s two a
s
pe
cts. Fi
rst, we d
e
s
ign
a
calib
ra
tion boa
rd
usi
ng PCB
and obtain its structu
r
al inf
o
rmatio
n usin
g the ex
isting
method for NIRCs. Seco
nd, we propo
se
an a
u
tomati
c cali
bration
method
for
NIRCs, i
n
wh
ic
h th
e h
u
m
a
n
–c
o
m
pu
te
r in
te
ra
c
t
ion
is
no
t
requi
re
d. Re
sults sh
ow that
the propo
se
d
met
hod sig
n
i
f
icantly redu
ces the calibra
tion time.
2. Design of
the Calibrati
on Boar
d for
NIRC
s and
Extrac
tion of Geometric I
n
forma
t
ion
Given that NI
RCs
can
sen
s
e n
ear
-infrared light only,
the texture of
the calib
ratio
n
boa
r
d
for
visibl
e ca
mera
ca
nnot be sen
s
ed. T
hus, de
signi
n
g
a b
o
ard tha
t
can
meet th
e nee
d of
NI
RC
calib
ration i
s
necessa
ry.
In
this study,
we de
sign a
cali
bration boar
d b
a
sed
on
NIR-SM
Ds. T
he
circuit of the
calib
ration
bo
ard i
s
de
sign
ed and
co
nst
r
ucte
d u
s
ing
a PCB. On this bo
ard, 64
NIR-SM
Ds a
r
e
arrang
ed in
a
n
array
of (8
× 8
)
with a
re
ctang
ular
net
sha
pe. Th
e
si
ze
of ea
ch
di
ode i
s
1.6 m
m
×
0.8 mm
×
0.3
mm. The
si
ze of the
light-emitting poi
nt is
0.3 mm
×
0.3 mm,
with
a wavelength
of
940 nm. A 5
V DC
po
wer adapte
r
is u
s
ed
as
su
ppl
y voltage for
the boa
rd to
ensure th
at the
curre
n
t flowin
g throu
gh
NIR-SM
Ds i
s
stable. T
he l
u
minan
ce of t
he light-emitting poi
nts
ca
n be
modified by adjustin
g
the variabl
e re
sist
or on t
he boa
rd to achieve
Gaussian di
stributio
n for the
grayscal
e
of l
i
ght spots in
the im
age
se
nse
d
by
NI
RCs an
d
extra
c
t the
subpix
e
l coo
r
dinate
s
of
the light-emi
tting points.
The d
e
si
g
ned
calib
rati
on bo
ard fo
r NI
RCs h
a
s
the foll
owing
advantag
es.
First, comp
ared with th
e common
cali
bration bo
ard,
near-inf
rared light
from NIR-
SMDs
can
be
sen
s
ed
by NIRCs. Seco
n
d
, the lumi
na
nce of light
-e
mitting points can b
e
adju
sted
to meet the
n
eed of
NIRC
calib
ration. T
h
ird, 6
4
light
-emitting poi
nts a
r
e
suffici
e
n
t to co
mpute
the
para
m
eters o
f
NIRCs.
We
sh
ould
o
b
tain a
c
curate ge
ometri
c i
n
formatio
n of
the b
oard to
cali
brate
the
NIRCs
with the de
si
gned b
o
a
r
d. A binocular
vision sy
ste
m
com
posed
of two visibl
e cam
e
ra
s
(MV-
130
UM) i
s
de
sign
ed to obt
ain this info
rmation. After calib
rating th
e system a
ccordin
g to Zha
ng’s
work, it can
be u
s
ed to
o
b
tain the
cali
bration
pa
ttern for
NIRCs
.
Several images
in different
positio
ns
are
captu
r
ed
in
a da
rkroo
m
to avoi
d int
e
rferen
ce fro
m
enviro
n
me
ntal light wh
en
extracting
the
sub
p
ixel coo
r
dinate
s
. Th
e
new
calibration patte
rn
ca
n be
obtaine
d acco
rdi
n
g t
o
our previou
s
work
[1
5].
Th
e
3
D
coo
r
din
a
tes of
ea
ch
point
o
n
the
pattern ca
n
a
l
so be acquired.
The cali
brati
on pattern
with kn
own
geomet
ric inf
o
rmatio
n is then obtain
e
d
and sho
w
n
in
Figure 2, whi
c
h can be u
s
ed to calib
rat
e
NIRCs di
re
ctly.
Figure 1. De
signed
calib
rat
i
on boa
rd (a)
captu
r
ed by the visible
ca
mera a
n
d
(b) captu
r
ed by
the
NIRC
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
9
30
An Autom
a
tic Calibration
Method for
Near-infra
re
d Cam
e
ra in O
p
tical Surgica
l
… (Ken Cai
)
1291
Figure 2. Cali
bration p
a
ttern
3. Automa
tic
Calibration
Metho
d
for
NIRCs Base
d on a Patte
r
n
The purpo
se
of came
ra cal
i
bration i
s
to comp
ute the internal a
nd e
x
ternal param
eters of
came
ra
s. Th
e automatic
calibratio
n
method for NI
RCs pro
p
o
s
ed i
n
this study is ba
sed on a
nd
improve
s
Zh
a
ng’s two-stag
e cam
e
ra
cali
bration m
e
th
od [16]. First,
classi
cal pin
hole ima
g
ing
is
adopte
d
a
s
th
e mod
e
l of
NIRCs. Th
e di
st
ortion m
odel
i
s
the
n
introdu
ced
[17]. The
paramete
rs
of
NIRCs a
r
e ca
lculate
d
ba
se
d on the disto
r
tion mod
e
l.
Con
s
id
erin
g that our re
se
arch is mainl
y
used for bi
nocular visi
o
n
system in surgical
navigation,
o
n
ly the
radi
al
and
tan
gent
ial di
stor
tion
s sh
ould
be
consi
dered,
a
s
exp
r
e
s
sed
in
expre
ssi
on (1), in whi
c
h
22
2
xy
r
and
12
1
2
[,
,
,
,
0
]
T
kk
k
p
p
is the lens di
stortion
coefficie
n
t of
NIR
C
s:
24
2
2
12
1
2
24
2
2
12
2
2
(1
)
2
(
2
)
.
(1
)
2
(
2
)
x
y
xk
r
k
r
p
x
y
p
x
r
yk
r
k
r
p
x
y
p
y
r
(
1
)
As
sho
w
n
in
Figure 3, th
e
pro
c
e
s
s of
NI
RC ca
libratio
n
is mai
n
ly di
vided into
three
step
s.
First, the p
o
si
tions of the
NIRCs relative to the
cali
brat
ion bo
ard
are
adju
sted to
keep the li
ght-
emitting point
s from spread
ing aro
und th
e cente
r
of
the image a
n
d
captu
r
e a
seri
es of imag
es
of
the calib
ratio
n
boa
rd in di
fferent po
sitio
n
s. Se
cond, the su
bpixel coordi
nate
s
of
feature poi
nts
are
com
pute
d
usi
ng the
method of g
r
ay-wei
ghted
averag
e. The
positio
n and
orde
r of the
four
corne
r
s ba
se
d on
the
tria
n
gular me
sh
method
ar
e a
l
so dete
r
mine
d. The 2D informatio
n in each
image of th
e
calib
ration
board is the
n
acquired.
Finally,
the point-to
-
poi
nt
co
rre
sp
ond
e
n
ce
betwe
en the sub
p
ixel coo
r
dinate
s
of each light-e
mitting point and
the 3D coo
r
dinate
s
of each
point on the
calib
ration
pattern mu
st
be esta
b
lished to obtai
n the param
eters. T
he i
n
itial
para
m
eters
are e
s
timate
d usin
g a linear e
s
timati
on method,
and the di
stortion mo
del
is
introdu
ce
d. T
he p
a
ra
mete
rs are o
p
timi
zed
u
s
ing
th
e meth
od
of nonli
nea
r e
s
timation. In t
h
is
pro
c
e
ss,
the i
nnovative p
o
i
n
t is t
he auto
m
atic dete
c
tion
of
the co
rn
ers,
which i
s
t
h
e fou
ndatio
n
of
automatic cali
bration.
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No
. 4, Decem
b
e
r
2015 : 128
9 – 1297
1292
Figure 3. Pro
c
e
ss of NI
RC calibration
In step 2, fou
r
initial value
s
of light
spot
s a
r
e obtai
ne
d usi
ng a
u
to
matic
corner
detectio
n
in ea
ch ima
ge. Co
rne
r
detectio
n
is
alway
s
a dif
f
icult pro
b
le
m in cam
e
ra calib
ratio
n
[18].
Operators ha
ve to use th
e artificial o
r
human
–c
om
puter inte
ra
ction by cli
cki
n
g
the mou
s
e
to
extract and
o
b
tain more preci
s
e informa
t
ion, which is tiresom
e
an
d time con
s
u
m
ing [19–2
1]. In
this study, an automatic
corne
r
detect
i
on me
thod
based on th
e triangul
ar
mesh meth
o
d
is
prop
osed u
s
i
ng the NIRC
calib
ration b
o
a
rd de
sig
ned
previou
s
ly.
A flag point
is set at the beginnin
g
o
f
t
he (8 × 8) NIR-SM
D a
rray in the d
e
sig
ned
calib
ration b
o
a
rd. The flag
point is nea
r the point at
the first ro
w of the first col
u
m
n
, as sh
own i
n
Figure 1.
Th
e ima
g
e
s
ca
ptured
by th
e
NIRC ar
e saved.
Th
e
lig
ht-emitting p
o
ints have a high
contrast
with
the b
a
ckg
r
o
und. Th
us, th
e imag
es
do
not n
eed filt
ering,
whi
c
h
can
imp
r
ove
the
spe
ed of sea
r
chi
ng featu
r
e points. We
use the s
u
b
p
ixel coo
r
din
a
tes to ch
ara
c
teri
ze the li
ght-
emitting point
s in the
imag
es to im
prove
pre
c
i
s
ion. Th
e metho
d
pro
posed in
Ref.
[16] is u
s
e
d
to
extract the
su
bpixel coo
r
di
nates
of the f
eature
poi
nts and
obtain
2
D
info
rmation
.
The extra
c
ti
on
result of the subpixel coord
i
nates i
s
sh
o
w
n in Figu
re
4(b
)
.
After the su
bpixel coordi
nates
of the
f
eature
poi
nts a
r
e o
b
ta
ined, the im
age i
s
pro
c
e
s
sed
using the t
r
ian
g
u
lar
me
sh m
e
thod.
Usi
ng
the subpixel
points a
s
the
triangle
verti
c
es,
the image is
divided into a
few triangle
area
s to
dete
r
mine the po
sition and orde
r of corn
ers. An
example of the results is
sh
own in Fig
u
re
5(a).
The su
m of the vertex
angle of the
triangle
s
in
every feature point is calcul
ated.
Assu
ming th
at the value
of the su
m is
M
, as
sho
w
n in Figu
re
5
(
a), the val
u
e
of
M
in the f
l
ag
point is sig
n
ificantly less than 90
°. Afte
r determi
ning
the position
of the flag point, its triangu
lar
mesh
is
delet
ed, as
sh
own
in Figu
re 5
(
b). We then
contin
ue to
calcul
ate
M
. T
he value
of
M
is
clo
s
e
to 90° whe
n
the co
rners
a
r
e se
l
e
cted a
s
th
e v
e
rtex. The va
lue of
M
i
s
cl
ose to
18
0°
or
360° when th
e other point
s are
sele
cte
d
as the ve
rtex. The positi
ons of
the flag point and four
corne
r
s can
be d
e
termi
n
e
d
through
the
value
of
M
. The
first
a
nd third co
rne
r
s are
dete
r
min
e
d
usin
g the
di
stance b
e
twe
e
n
the
co
rne
r
s and
flag
poi
nt. Whe
n
cal
c
ulatin
g the
d
i
stan
ce
between
the flag
point
and
ea
ch
co
rner, the
ne
arest i
s
the
fi
rst co
rne
r
and
the farth
e
st
is
the third
corn
er.
The
se
cond
and fou
r
th
corne
r
s are d
e
termin
ed by
cal
c
ulatin
g the an
gle
s
. T
he flag p
o
int
is
denote
d
as
P
f
. The first an
d third corn
ers are denote
d
as
P
1
and
P
3
, respe
c
tively. The othe
r two
unsure
corne
r
s
are denot
ed
a
s
P
m
and
P
n
. We
s
e
t
α
a
s
the
an
gle b
e
twe
en
vector
P
f
P
1
an
d
vec
t
or
P
1
P
m
and
β
a
s
th
e
angle
bet
we
en vecto
r
P
f
P
1
and ve
cto
r
P
1
P
n
. If angle
α
i
s
l
e
s
s
t
h
a
n
angle
β
, then
P
m
is the se
con
d
corner
and
P
n
is the fourth
c
o
rner. Otherwis
e,
P
m
is the fourth
corne
r
and
P
n
is the seco
nd co
rne
r
. Base
d on the
aforem
ention
ed pro
c
e
s
s, the co
rne
r
s a
r
e
automatically dete
c
ted
and
their o
r
de
rs i
n
ea
ch
imag
e a
r
e
th
e
s
a
me
. F
i
n
a
lly, the 2D information
in each imag
e of the calibration boa
rd is acqui
red.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
An Autom
a
tic Calibration
Method for
Near-infra
re
d Cam
e
ra in O
p
tical Surgica
l
… (Ken Cai
)
1293
Figure 4. (a)
Origin
al imag
e of the
calibration boa
rd capture
d
by NIRC a
n
d
(b)
sub
p
ixel e
x
traction resu
lts, whe
r
e “+”
rep
r
e
s
ent
s the sub
p
ixel co
ordin
a
tes
Figure 5. (a)
Trian
gula
r
m
e
sh result of the su
bpixel p
o
ints an
d
(b) tria
ngul
ar
mesh
re
sult o
f
the subpi
xel
points, exce
pt for the flag point
Assu
ming th
at
P
is a point in global
spa
c
e a
nd
p(
x
p
, y
p
)
i
s
its came
ra coo
r
dinate
captu
r
ed
by the NI
RCs, we de
ri
ve expression
(2
), which
ca
n be
expre
s
sed in
matrix form
i
n
expre
ssi
on (3
) as follo
ws:
0
0
xx
y
p
p
yy
f
u
x
y
fv
(
2
)
0
0
0
10
0
1
1
p
xx
x
p
yy
xf
f
u
yf
v
(
3
)
The matrix of the NIRC intri
n
si
c par
amet
ers
can b
e
expre
s
sed a
s
follows:
0
0
0
00
1
xx
y
f
fu
A
fv
(
4
)
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No
. 4, Decem
b
e
r
2015 : 128
9 – 1297
1294
In expre
s
sion
(4),
f
x
,
f
y
are
the focal len
g
ths of th
e camera, (
u
0
, v
0
) are the
co
ordin
a
te
s
of the main point of the ca
mera, an
d
α
is the tilt factor of the two-i
m
age coo
r
din
a
te axis.
First, the NI
RCs a
r
e
con
s
i
dere
d
an a
p
p
r
oxim
ately id
eal pinh
ole m
odel when
estimating
the initial parameters of
NI
RC
s, whi
c
h
mean
s
k
= 0.
We set:
11
12
13
14
21
22
23
24
31
32
33
34
41
42
43
44
mm
m
m
mm
m
m
MA
mm
R
mm
mm
m
T
m
(
5
)
For the
i
th fe
ature
point
se
lected i
n
the
calib
ra
tion proce
s
s,
with coordi
nate
s
of (
u
i
, v
i
, 1
)
in the image
coo
r
din
a
te sy
stem an
d (
X
Wi
, Y
Wi
, Z
Wi
, 1
)
in the wo
rld
coordi
nate
system, we de
rive
expre
ssi
on
(6
),
which can
be conve
r
ted
into
the
linea
r equ
ation
s
ex
pre
s
sed
in
expre
ssi
on
(7
)
as
follows
:
11
12
1
3
1
4
21
2
2
23
24
31
32
33
3
4
41
4
2
43
44
1
1
Wi
i
Wi
Ci
i
Wi
mm
m
m
X
u
mm
m
m
Y
Zv
mm
m
m
Z
mm
m
m
(
6
)
1
1
12
13
14
31
32
33
34
2
1
22
23
24
31
32
33
34
Wi
Wi
Wi
i
W
i
i
Wi
i
W
i
i
Wi
Wi
Wi
i
W
i
i
Wi
i
W
i
i
m
X
m
Y
m
Z
m
m
u
X
mu
Y
m
u
Z
mu
m
X
m
Y
m
Z
m
m
v
X
mv
Y
m
v
Z
mv
(7)
For a
n
y feature p
o
int in t
he calibratio
n
pro
c
e
s
s, we derive t
he
two line
a
r e
q
uation
s
expre
s
sed in
expre
ssi
on (7). Thu
s
, for
N
feature poi
nts, 2
N
lin
ea
r equation
s
can be o
b
tain
ed.
The m
a
trix
can b
e
com
p
u
t
ed by
solvin
g the li
nea
r
e
quation
s
. F
o
r a total
of 1
2
un
kno
w
n
s
i
n
M
,
the value of
N
must b
e
greater than 6.
The result ob
tained u
s
ing t
he aforeme
n
tioned m
e
thod
is not hi
ghly accurate be
cause it
doe
s n
o
t co
n
s
ide
r
di
sto
r
tion. Di
stortion
i
s
intr
odu
ce
d t
o
en
han
ce
accuracy. A
s
su
ming that
a to
tal
of
N
imag
es are
captu
r
e
d
and fo
r 6
4
featur
e
poi
nts in eve
r
y image, we
can
obtain
6
4
N
coo
r
din
a
tes
o
f
pixels wh
en
the feature
p
o
ints a
r
e extracted.
We
se
t the real pix
e
l co
ordi
nate
of
the ith feature p
o
int a
s
p
i
(
x
pi
, y
pi
) and the id
eal pixel co
ordin
a
te obt
ained u
s
in
g
the
aforem
ention
ed metho
d
as
p
i
′
(
u
i
, v
i
).
Based
on t
he no
nlinea
r model
with
the influen
ce of
distortio
n
,
p
i
,
p
i
′
are fitted accordi
ng to the obje
c
tive function, a
s
sh
own in exp
r
e
ssi
on (8
):
64
22
1
1
mi
n
(
)
(
)
64
N
ip
i
i
p
i
i
Eu
x
v
y
N
(
8
)
We set the value obtai
ne
d usin
g the afor
em
ention
ed method a
s
the optimized initial
value of the nonlin
ear ite
r
ations.
We th
en use it
erati
v
e optimizati
on by the lea
s
t sq
uare me
thod
to derive the
global
optimu
m
solutio
n
. T
hus, the
i
n
tri
n
si
c pa
ram
e
ters
are obtai
ned. The
rel
a
tive
positio
n relati
onship b
e
twe
en the
NIRCs an
d
calib
rat
i
on bo
ard in
every po
sitio
n
, whi
c
h
are
the
extrinsi
c pa
ra
meters, are
si
multaneo
usly
obtained.
4. Experimental Re
sults
In this
se
ctio
n, we
co
nce
n
trate o
n
the
com
p
a
r
iso
n
between th
e man
ual
ca
libration
method an
d a
u
tomatic
calib
ration meth
od
propo
se
d in this stu
d
y.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
An Autom
a
tic Calibration
Method for
Near-infra
re
d Cam
e
ra in O
p
tical Surgica
l
… (Ken Cai
)
1295
The NI
RC
ca
pture
s
80 im
age
s of the calibratio
n
boa
rd, whi
c
h a
r
e
evenly divided int
o
four g
r
oup
s.
For e
a
ch gro
up, the re
sult
s of
the ma
n
ual and
auto
m
atic calibration metho
d
s
are
sho
w
n in Ta
b
l
e 1. Notably, the result
s of t
he manual
and autom
atic cali
bratio
n method
s are the
same, the
r
eb
y indicating th
at the automa
t
ic ca
lib
ration
method can reliably calib
ra
te NIRCs.
The time
s
co
nsum
ed
by th
e man
ual a
n
d
automati
c
ca
libration
meth
ods in
calib
ra
ting the
four group
s a
r
e sho
w
n in
Table 2. Th
e mean ti
me
co
nsum
ed by th
e manu
al cali
bration m
e
th
od
is 1
24
s, a
nd
the mea
n
tim
e
con
s
ume
d
by the a
u
tom
a
tic
calib
ratio
n
metho
d
i
s
5
.
57 s.
Thi
s
re
sult
sho
w
s that th
e propo
sed
a
u
tomatic
cali
bration
me
th
od si
gnificantl
y
redu
ce
s the
calib
ration
time
of the NIRCs.
As sh
own in
Table 1, the
mean relative
erro
rs of the focal length
s
and prin
cip
a
l
points
for the four group
s are 0.8
4
% and 1.46
%, 0.
76% and 0.88%, 1.03% and
1.74
%, and 0.86% and
1.48%, re
spe
c
tively. The t
o
tal mea
n
rel
a
tive errors o
f
the focal
le
ngths
and
pri
n
cip
a
l poi
nts
are
0.87% and 1.
39%, respe
c
tively. Table 3 sho
w
s the m
ean relative e
rro
rs
of the focal len
g
ths a
nd
prin
cipal
poin
t
s for the vi
sible came
ra
s
use
d
in
Refs.
[8], [22], and [23]. These
results in
dica
te
that the prop
ose
d
automat
ic cali
bratio
n method is a
c
curate and
sa
tisfacto
ry.
For gro
up 1, 20
ima
g
e
s
of the
calibratio
n
bo
ar
d
at diff
erent
po
sition
s a
r
e
ca
pture
d
by the
NIRC. The
s
e image
s
co
ntain 128
0 reproj
ectio
n
coordi
nate
s
. Mean
while,
a total of 1280
resi
dual
s a
r
e
acq
u
ire
d
and
sho
w
n in Fi
g
u
re 6. Th
e re
con
s
tru
c
tion
result
s of the 3D coo
r
dinat
es
are sho
w
n in
Figure 7. T
he re
sidu
als
of the
X
-dire
c
tion mainly
con
c
e
n
trate i
n
the interval
o
f
−
0.15
pixel
s
t
o
0.15
pixel
s
, in
whi
c
h th
e
numbe
r
of fe
ature
point
s i
s
1
270,
acco
unting fo
r
99.
2%
of
the
total. T
he re
sidual
s of
the
Y
-direction mai
n
ly concentra
te
in
the inte
rval
of
−
0.1
pixel
s
to
0.1 pixels, in
whi
c
h the n
u
m
ber
of featu
r
e poi
nts
i
s
1
268, a
c
counti
ng for 9
9
.1% of the total. The
results of o
u
r previou
s
work
rep
o
rted
in
Ref. [14]
are
−
0.6
pixels to
0.6
pixels (9
5.7%) in
the
X
-
dire
ction and
−
0.5
pixels to
0.5 pixel
s
(9
2.2%) in
the
Y
-dire
c
tion.
T
hese results i
ndicate that t
he
new cali
brati
on
bo
ard ha
s
hig
her precision,
a
nd
it
can
de
scribe
the NIRC p
r
ojectio
n
process
corre
c
tly usin
g the imaging
geometry mo
del esta
blishe
d by the calib
ration results.
Table 1. Re
sults of the ma
nual an
d auto
m
atic calibration method
s
Grou
p 1
Grou
p 2
Grou
p 3
Grou
p 4
Manual (fc
:
mm)
[2115.84, 21
15.2
0
] ±
[17.30, 18.19]
[2101.81, 21
03.7
9
] ±
[15.83, 16.29]
[2046.79, 20
43.7
5
] ±
[220.48, 21.6
5
]
[2100.44, 21
01.0
5
] ±
[018.27, 17.8
6
]
Au
t
o
m
a
t
i
c
(fc: mm
)
[2115.84, 21
15.2
0
] ±
[2115.8, 18.1
9
]
[2101.81, 21
03.7
9
] ±
[215.83, 16.2
9
]
[2046.79, 20
43.7
5
] ±
[220.48, 21.6
5
]
[2100.44, 21
01.0
5
] ±
[218.27, 17.8
6
]
Manual (cc:
pixels)
[631.42, 618.
77]
±
[311.53, 6.78]
[692.42, 609.
50]
±
[97.13, 4.65]
[645.41, 565.
05]
±
[414.53, 6.91]
[614.98, 571.
18]
±
[111.50,6.25]
Au
t
o
m
a
t
i
c
(cc: pixel)
[631.42, 618.
77]
±
[311.53, 6.78]
[692.42, 609.
50]
±
[97.13, 4.65]
[645.41, 565.
05]
±
[414.53, 6.91]
[614.98, 571.
18]
±
[111.50,6.25]
Table 2. Time
con
s
ume
d
b
y
the manual and autom
atic cali
bratio
n method
s
Grou
p 1
Grou
p 2
Grou
p 3
Grou
p 4
Manual
125 s
122 s
129 s
120 s
Automatic
5.28 s
5.34 s
5.87 s
5.78 s
Table 3. Mea
n
relative erro
rs
Ref. [8]
Ref. [22]
Ref. [23]
This paper
Focal
length
6.82%
0.57%
6.22%
0.87%
Principal
point
4.44%
1.20%
0.56%
1.39%
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
9
30
TELKOM
NIKA
Vol. 13, No
. 4, Decem
b
e
r
2015 : 128
9 – 1297
1296
Figure 6. Rep
r
oje
c
tion e
rro
r analysi
s
Figure 7. Re
constructio
n
re
sults of the 3
D
co
ordi
nate
s
at different
positio
ns
5. Conclusio
n
s
This pa
pe
r propo
se
s an a
u
tomatic cali
bration m
e
th
od to calib
rat
e
NIRCs. A calibratio
n
board with
NI
R-SM
Ds
is d
e
sig
ned be
ca
use
th
e
com
m
on
calib
rati
on bo
ard can
n
ot be
ca
ptured
by NIRCs. A
c
curate ge
om
etric info
rmati
on is
me
a
s
u
r
ed u
s
ing
a visible
bino
cula
r vision
sy
ste
m
.
The
calib
ratio
n
pattern i
s
o
b
tained,
whi
c
h can
be
us
e
d
to
calib
rate
the NI
RCs
directly. The
init
ial
para
m
eter va
lues a
r
e obta
i
ned usi
ng di
rect line
a
r
tra
n
sformation t
o
calib
rate th
e NIRCs. Mo
re
accurate p
a
rameters
are
then
achieved
by the
nonlin
ear
optimi
zati
on meth
od. I
n
the
pro
c
e
ss of
algorith
m
implementatio
n, the 2D informatio
n of the image of the calibration b
oard is
automatically acce
ssed.
The re
sult
s o
f
our experim
ents sho
w
that the
averag
e relative errors of the focal length
and p
r
in
cipal
point of
NIRC
are 0.87
% and 1.3
9
%
, resp
ectiv
e
ly, which in
dicate th
at the
prop
osed me
thod ha
s hig
h
accu
ra
cy. The time co
n
s
ume
d
by the automati
c
calibratio
n
pro
c
e
s
s
is 5.57
s, an
d the time
co
nsum
ed by t
he man
ual
calibratio
n
met
hod i
s
12
4 s.
The
s
e findin
gs
indicate that the pro
p
o
s
ed
method si
gnif
i
cantly red
u
ces the calibra
tion time.
In summ
ary,
the propo
sed
NIRC auto
m
atic
calibratio
n
metho
d
ha
s the
advant
age
s
of
saving time, efficien
cy in computat
ion, h
i
gh accu
ra
cy, and relia
bility.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
An Autom
a
tic Calibration
Method for
Near-infra
re
d Cam
e
ra in O
p
tical Surgica
l
… (Ken Cai
)
1297
Ackn
o
w
l
e
dg
ement
This
re
se
arch
wa
s fun
ded
by the Gu
an
gdon
g Natura
l Scien
c
e
Fo
undatio
n un
d
e
r G
r
a
n
t
No. S201
304
0014
993, the
State Schol
arship F
und
unde
r G
r
ant
CSC
NO. 20
1408
4403
26,
the
Pearl Rive
r S&T Nova Program of Guan
gzhou
unde
r Grant
No. 2014
J2
2000
49 and
No.
2015
0601
003
5, the Guang
dong Provincial Scien
c
e a
nd Te
chnol
o
g
y Progra
m
unde
r Grant No
.
2013B0
906
0
0057
and
No.
2014A0
202
1
5006, the F
u
ndame
n
tal Rese
arch F
u
n
d
s for th
e Ce
ntral
Universitie
s
u
nder G
r
a
n
t No. 2014Z
G00
3
D.
Referen
ces
[1]
Sun JH,
Ch
en
X, Go
ng Z
,
L
i
u
Z
.
and Z
h
ao
YT
. “Accurate
camera c
a
li
bra
t
ion
w
i
t
h
distor
tion mo
de
ls
usin
g sph
e
re i
m
ages”.
Optics
& Laser T
e
chn
o
lo
gy
. 201
5; 65: 83-87.
[2]
Lab
udzki
R, L
egutko
S, Ra
o
s
P. T
he e
ssence a
nd
ap
plic
ations
of mach
ine v
i
sio
n
.
Te
hn
i
cki
Vj
e
s
ni
k
.
201
4; 21(4): 90
3-90
9.
[3]
Kalomir
o
s JA,
L
y
go
uras J. H
a
rd
w
a
r
e
pr
inci
ples for t
he
de
sign
of a ster
e
o
-matchi
ng sta
t
e machi
n
e
base
d
on
d
y
n
a
mic pro
g
ram
m
ing.
Jo
urna
l
of Engi
neer
in
g
Scienc
e an
d
T
e
chno
logy
R
e
view
. 200
8;
1(1): 19-2
4
.
[4]
Cai K, Yan
g
R
,
Ning H, Ou
S, Z
eng Z
.
An aut
omatic a
l
g
o
rithm for disti
ngu
ishi
ng o
p
tic
a
l nav
ig
atio
n
markers use
d
duri
ng surg
er
y
.
DYNA
. 2015;
90(2): 20
3-2
0
9
.
[5]
Z
hang
XD, Ho
u ML, H
u
YY, Z
hang
XQ, W
u
YH.
Stud
y on th
e 3
D
i
n
formatio
n
rec
o
nstruction
an
d
mana
geme
n
t o
f
cultural
rel
i
cs
base
d
o
n
th
e
articul
a
ted
ar
m scann
er.
Jo
urna
l of D
i
gita
l
Informatio
n
Mana
ge
me
nt
. 201
5; 13(1): 31
-38.
[6]
Li
X
C
, Wang YH. Auto
matic
Selecti
on for
Optimal Ca
lib
ration M
ode
l
of Camer
a
.
TEL
K
OMNIKA
Indon
esi
an Jou
r
nal of Electric
al Eng
i
ne
eri
ng.
2014; 1
2
(3):46
48-4
653.
[7]
Samper D, S
antol
aria J
o
rg
e, Majar
ena
AC, A
gui
lar J
J
. Compre
hen
sive simu
lati
o
n
soft
w
a
re for
teachi
ng c
a
m
e
ra ca
libr
a
tio
n
b
y
a c
onstr
uctivist metho
dol
og
y.
Me
as
ure
m
e
n
t: Jour
nal
of the
Internatio
na
l Measur
e
m
ent C
onfed
erati
o
n
. 2
010; 43(
5): 618
-630.
[8]
W
ang RY, Gu
a
ng J, Qua
n
L,
W
u
CK. Camer
a
cal
i
brati
on
us
ing
ide
n
tica
l o
b
j
ects.
Machi
ne Visio
n
an
d
Appl
icatio
ns
. 2
012; 23(
3): 579
-587.
[9]
W
an YW
, Hu
ang Y, B
u
ckle
s
B. Camera
calibr
a
tio
n
a
n
d
vehic
l
e tracki
ng: Hi
gh
w
a
y traffic vide
o
analy
t
i
cs.
T
r
ansportatio
n
Res
earch Part C: Emer
gi
ng T
e
ch
nol
ogi
es
. 201
4; 44: 202-2
13.
[10]
John
K. A cam
e
ra ca
libr
a
tio
n
method
for a
h
a
mmer thro
w
a
nal
ysis
too
l
.
Proced
ia E
ngi
ne
erin
g
. 20
14
;
72: 74-7
9
.
[11]
Gao JC, Liu MY. Camera Sel
f
Calibr
a
tio
n
in
the AUV Mono
cular Visi
on N
a
vig
a
tion a
nd
Positio
n
in
g
.
T
E
LKOMNIKA Indon
esi
an Jou
r
nal of Electric
al Eng
i
ne
eri
n
g
.
2013; 1
1
(12):
715
1-71
58.
[12]
Z
hang Z
Y
. A flexibl
e
ne
w
tec
hni
que for ca
mera cali
brati
o
n.
IEEE Transactions on Pattern Analys
is
and Mac
h
in
e Intelli
ge
nce
. 20
00; 22: 13
30-1
334
[13]
Yang
XF
, Hu
ang
YM, Gao
F
.
A simpl
e
camera c
a
l
i
br
ation
metho
d
base
d
o
n
s
u
b
-
pi
xel
cor
n
e
r
extracti
on of th
e chessb
oar
d i
m
age.
Intell
ig
e
n
ce Co
mputin
g
and Intell
ig
ent System.
20
10; 29-3
1
.
[14]
W
en XY, Liu
SJ, Yang RQ, W
ang Z
G. Pa
ttern desig
n a
nd rea
lizati
on for calibr
a
tio
n
near i
n
frare
d
camera in surgic
a
l
na
vi
ga
ti
on
.
Optoelectron
ic
s Letters
. 2012
; 8(6): 409-41
3
.
[15]
Z
heng BS, Ji
JP, Yang R
Q. Calibr
a
tion
st
ud
y of hi
gh
-precisi
on
nea
r infrare
d
cam
e
ra.
Ch
ine
s
e
Me
d
i
ca
l
Eq
ui
pm
en
t Jo
u
r
na
l
. 201
1; 32(1
2
): 15-17.
[16]
W
ang Z
G. Stud
y of tech
ni
que for o
p
tica
l tra
cking s
u
r
g
ical
instrume
nt in surg
er
y
navi
gatio
n.
A
Dissertati
on Su
bmitted for the Degr
ee of Mas
t
er: South Chi
n
a Univ
ersity of T
e
chno
logy
. 2
012.
[17]
William T
R
, Mege
ath ST
. Ma
rtin C. Absol
u
t
e
cali
brati
on of
the infrar
ed ar
ra
y
c
a
mera
on
the spritze
r
space tel
e
sco
p
e
.
Public
atio
ns of the Astrono
mi
c
a
l Society of the Pacific
. 2005; 11
7(8
35): 978-
990.
[18]
X
i
a JX
, X
i
ong
JL, X
u
X
Q, Qin HY. A multiscale
su
b-pi
xel
detector for co
rners in c
a
mer
a
cali
bratio
n
targets.
20
10 I
n
ternati
o
n
a
l C
o
nferenc
e o
n
Intelli
ge
nt
Co
mp
u
t
ation T
e
ch
no
l
ogy a
nd A
u
to
mation
. 20
10
;
196-
199.
[19]
Che
n
DZ
, Z
h
a
ng GJ. A n
e
w
sub-p
i
xel
detec
to
r for
X
-
corners in c
a
mera
ca
l
i
b
ra
ti
on
ta
rgets.
T
he 13t
h
Internatio
na
l C
onfere
n
ce
in
Centra
l Eur
o
p
e
o
n
C
o
mput
er
Grap
hics, Visua
l
i
z
a
t
i
on a
nd
C
o
mp
uter
Visio
n
200
5 in
co-op
e
ra
tion with EUROGRAPHICS
. 2005;
97-1
00.
[20]
Li LL, Z
hao W
C
, W
u
F, Liu Y, Gu
W
.
Exper
i
m
ental a
nal
ys
i
s
and improv
e
m
ent on camer
a
calibr
a
tio
n
pattern.
Optical
Engin
eeri
n
g
. 2
014; 53 (1): 0
1
310
4.1-7.
[21]
Arturo E, Jose MA. Automa
tic chessbo
a
rd
detectio
n
for intrinsic a
nd e
x
trins
i
c camer
a
param
ete
r
calibration.
Sensors
. 201
0; 1
0
: 2027-
20
44.
[22]
Jamil D, Se
ba
stien R, Peter
S.
Plane-
base
d
cali
brati
on fo
r line
a
r camer
a
s.
Internatio
n
a
l Jour
na
l o
f
Co
mp
uter Visi
on
. 201
1; 91: 1
46-1
56.
[23]
Luis
P, Yalin
B, Peter S.
C
a
libr
a
tio
n
of c
entral
cata
dio
p
t
ric camer
a
s u
s
ing
a
DLT
-
like a
ppro
a
ch.
Internatio
na
l Journ
a
l of Co
mputer Visi
on
. 2
011; 93: 1
01-1
14.
Evaluation Warning : The document was created with Spire.PDF for Python.