TELKOM
NIKA Indonesia
n
Journal of
Electrical En
gineering
Vol. 15, No. 2, August 201
5, pp. 336 ~
345
DOI: 10.115
9
1
/telkomni
ka.
v
15i2.812
4
336
Re
cei
v
ed Ap
ril 28, 2015; Revi
sed
Jul
y
3, 2015; Accept
ed Jul
y
17, 2
015
Image Mosaic Method Based on Gaussian Second-
order Difference Featu
r
e Operator
Yong Chen
*, Yu-bin Hao,
Di Zhan
Ke
y
Lab
orator
y of Industrial In
ternet
of T
h
ings& Net
w
o
r
k Co
ntrol, MOE,
Cho
ngq
in
g Uni
v
ersit
y
of Posts
and T
e
lecom
m
unic
a
tions, C
hon
gqi
ng, Ch
in
a, 4000
65
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: chen
yo
ng@c
qupt.ed
u
.cn
A
b
st
r
a
ct
T
o
co
mpos
e th
e w
i
de v
i
sua
l
a
ngl
e a
nd
hig
h
r
e
sol
u
tion
i
m
a
g
e
fro
m
the s
e
q
uenc
e of i
m
ag
es w
h
ic
h
have
overl
app
i
ng reg
i
o
n
in th
e sa
me sce
ne
quickly
and c
o
rrectly, an i
m
prove
d
SIF
T
algorit
hm w
h
ic
h
is
base
d
on D
2
o
G
interest poi
n
t
detector w
a
s prop
osed.
It extracted the i
m
a
ge f
eatur
e po
in
ts and ge
ner
ate
d
corresp
ond
in
g
feature d
e
scr
iptors by i
m
pr
oved SIF
T
alg
o
rith
m. T
hen,
usin
g the ra
n
d
o
m
co
nsisten
cy
(RANSAC) alg
o
rith
m
p
u
rifi
ed
feature po
int match
i
n
g
p
a
irs
and
calc
ul
atin
g the tra
n
sfor
mati
on
matrix
H.
Last, co
mp
lete
the s
e
a
m
l
e
ss
mos
a
ic
of i
m
a
ges
by
usin
g t
he
i
m
ag
e fus
i
o
n
a
l
gor
ith
m
of
slipp
i
n
g
i
n
to
a
n
d
out. It respecti
vely proc
ess th
e i
m
ag
es w
h
ic
h ha
d the fo
ur typical tra
n
sfor
mati
ons w
i
th th
e traditi
ona
l SIF
T
and the
prop
o
s
ed metho
d
. T
he result in
di
cated that
the
number of fe
ature pa
irs is few
e
r than SIF
T
alg
o
rith
m
an
d
the
mos
a
ic ti
me is s
horter,
a
nd th
en
t
he
matchin
g
effici
en
cy is h
i
g
her th
an th
e l
a
ter. T
h
i
s
prop
osed
met
hod r
e
d
u
ces
the co
mp
lexit
y
of op
er
atio
n an
d i
m
prov
es rea
l
-ti
m
e
of imag
e
mo
sai
c
simulta
neo
usly
.
Ke
y
w
ords
:
sequ
enc
e i
m
a
ge, i
m
a
ge
ma
tching, SIF
T
algorit
hm, first-
order d
i
fferenc
e pyra
mi
d, se
cond-
order d
i
fferenc
e pyra
mi
d
Copy
right
©
2015 In
stitu
t
e o
f
Ad
van
ced
En
g
i
n
eerin
g and
Scien
ce. All
rig
h
t
s reser
ve
d
.
1. Introduc
tion
The pe
rspe
cti
v
e of a singl
e
image o
b
tai
ned by
ima
g
e
acquisitio
n
device i
s
limit
ed. It
is
need
ed to compo
s
e the
wide visual a
ngle and hi
g
h
resolution i
m
age from t
he se
quen
ce
o
f
image
s which have overl
appin
g
regi
o
n
in som
e
a
pplication
s
[1]. With the developm
ent
of
comp
uter
an
d image
processing te
ch
n
o
logy, image
st
itchin
g tech
nology i
s
wi
d
e
ly use
d
in
space
exploratio
n, remote
sen
s
in
g imag
e p
r
o
c
essing, m
edi
cal im
age
an
alysis an
d vid
eo
retrieval,
etc.
Image regi
stration is the
key techn
o
log
y
of im
age m
o
sai
c
, the me
thod main i
n
clude
s area-ba
sed
approa
ch a
n
d
feature-b
a
sed ap
pro
a
ch. In the area
-b
ase
d
ap
pro
a
ch, there i
s
a
classic
algo
rith
m
whi
c
h i
s
called re
gist
ratio
n
algo
rithm
based o
n
a
template. Th
e method
of based o
n
g
r
ay
correl
ation is prop
osed b
y
literature [
2
], howev
e
r
, due to the
limitations of
time-con
su
ming
a
ffe
c
t
e
d
,
an
d th
e
a
l
go
r
i
thm’s
pr
ac
tic
a
l is
c
o
ns
tr
ain
ed, this
meth
od’s
erro
rs i
s
larg
e. Featu
r
e-
based a
ppro
a
ch i
s
mat
c
h
ed by the
cha
r
acte
ri
stics
which derive
d
from
the pixel values.
Su
ch as
SUSAN co
rn
er dete
c
tion algorith
m
whi
c
h doe
sn’t
n
eed gradient
operator, so
the efficiency
of
algorith
m
is raise
d
. And t
h
is
algo
rithm
ha
s inte
g
r
al
ch
ara
c
te
risti
c
s which ma
ke it
have b
een
greatly imp
r
o
v
ed in term
s of com
putin
g sp
eed
and
noise immu
nity. Howeve
r, an ex
ce
ssi
ve
numbe
r
of corne
r
s of thi
s
al
gorith
m
i
s
n
o
t cond
u
c
ive to
co
rn
er m
a
tchi
ng.
The
metho
d
of
Morave
c
co
rner
dete
c
tion
is p
r
op
osed
by literat
ure
[3]. This me
thod’s
cal
c
ul
ation is
sm
all
but
sen
s
itive to the influen
ce
of the noise.
When
Ha
rri
s
ope
rato
r is used in filtering and the first
orde
r differen
c
e of gray scale,
the co
rn
er features
e
x
tracted by
it are unifo
rm
and rea
s
ona
ble.
Although, th
e algo
rithm
can
dete
c
t corne
r
s in a
singl
e scal
e, the po
sition
ing a
c
cura
cy
of
detectio
n
is
p
oor [4]. Acco
rding to differe
nt scena
ri
o
s
,
Mikolaj
c
zy
k e
t
c t
a
ke
a t
e
st
f
o
r a v
a
ri
et
y
of
the mo
st rep
r
esentative d
e
scripto
r
s [5]
.
The
SIFT
descri
p
tor pe
rforma
nce i
s
goo
d, but t
he
compl
e
xity of
the algorith
m
is high, an
d t
he cal
c
ul
ation
of the algorithm is large.
To compo
s
e
the wi
de visual an
gle a
n
d
high
re
sol
u
tion imag
e from the
seq
u
ence of
image
s whi
c
h have overl
appin
g
regi
o
n
in the
sam
e
scene qui
ckly and co
rre
c
tly, an improved
SIFT algo
rith
m which i
s
b
a
se
d o
n
D2o
G
inte
re
st poi
nt dete
c
tor was
propo
sed.
It extracte
d t
h
e
image fe
ature point
s a
n
d
gene
rate
d
corre
s
p
ondin
g
feature de
scripto
r
s by improve
d
SIFT
algorith
m
[6]. Then, u
s
ing
the ran
dom
con
s
i
s
te
n
c
y (RANSAC) al
gorithm p
u
rifi
ed feature p
o
int
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
Im
age Mosai
c
Method Ba
sed on Ga
ussi
an Seco
nd
-order Diffe
ren
c
e Feature… (Yong Ch
en
)
337
matchin
g
pairs and calculat
ing the transf
o
rmatio
n matrix
H
. The result indicated t
hat the numb
e
r
of feature p
a
i
rs i
s
fewer t
han SIFT al
gorithm a
nd
the mosaic ti
me is
sho
r
te
r, and the
n
the
matchin
g
efficien
cy is hig
h
e
r than the lat
e
r.
And the rest
part of this i
s
orga
nized a
s
fo
llows, part
2 introd
uces t
he pri
n
ci
ple o
f
Multi-
scale featu
r
e
extraction, a
n
d
in se
ction 3
we give
a ve
ry clea
r elab
o
r
ation of the
algorith
m
whi
c
h
we propo
se
d
.
In part 4, we evaluat
ed the e
ffect of different algorithm
s
and come to a
con
c
lu
sio
n
.
2. Multi-scal
e Feature Ex
trac
tion
The fe
ature
point i
s
a
n
area
which i
s
diffe
rent i
n
color an
d
gray
level
with the
surro
undi
ng.
SIFT feature
point i
s
to fin
d
the
extr
em
e poi
nt in th
e
sp
atial
scale
,
and
extra
c
ts its
positio
n, scal
e and rotatio
n
invariant [7].
SIFT feature extraction i
s
mainly divide
d into three st
eps:
(1) Build
scal
e-inva
riant sp
ace;
(2) F
eature p
o
int positio
nin
g
;
(3) G
ene
ratin
g
feature poi
n
t
descripto
r [8].
SIFT feature
point l
o
catio
n
is sho
w
n i
n
Fig
u
re
1. I
n
DoG
sp
ace.Then
take
a furth
e
r
descri
p
tion
o
f
the featu
r
e
point
s
and
form a
1
28-dimen
s
ion
a
l
feature
vecto
r
. Finally, ta
ke
sub
s
e
que
nt matchin
g
to the feature d
e
scripto
r
and
get the feature
s
co
rresp
ondin
g
matching
point pair of the image [9
-1
0].
In three ste
p
s
of SIFT feature extra
c
ti
on,
there a
r
e more l
a
yers of LoG a
n
d DoG
pyramid
s
ne
e
d
to be con
s
t
r
ucte
d. And there i
s
a nee
d to use the l
o
cal info
rmati
on of the thre
e
layers ima
ge
in DoG Pyra
mid to l
o
cate
the featu
r
e p
o
ints.
T
herefo
r
e,
featu
r
e de
tection occu
pi
es
about 80% of
the time of SIFT algorith
m
[11]. And sp
licin
g take
s
too much time, it is difficult to
reali
z
e real-ti
m
e perfo
rma
n
ce. Thi
s
pa
per redu
ce
d the co
mputati
onal complex
i
ty by simplifying
the
pyra
mid structu
r
e and chang
ed
the
m
e
thod of
featu
r
e p
o
int p
o
siti
oning. T
h
u
s
p
r
opo
se
d SIFT
feature extra
c
tion algorith
m
is based on
D2o
G
.
3
.
The Principle of SIFT Featur
e Extr
a
c
tion
w
i
th
D2oG
3.1. D2oG Fe
ature
De
tec
t
ion Opera
t
or
In this pap
er,
to get D2o
G
function by ta
king a
differe
ntial ope
ratio
n
to DoG fu
n
c
tion
s is
sho
w
n in Equ
a
tion (1). And
repla
c
e extre
m
e point
differential of Do
G function wit
h
zero-crossi
ng
detectio
n
of D2o
G
functio
n
. Then dete
r
mine the
ch
aracteri
stic
scal
e and dete
c
t feature p
o
ints.
(1)
Whe
r
e
is
D2
oG fun
c
tion,
is
Do
G fu
n
c
tion,
is spati
a
l coordinate
s
,
is
scale coordin
a
tes,
is scal
e factor a
s
Equ
a
tion (2
) sh
o
w
n.
(2)
(3)
Acco
rdi
ng to Equation (3), whe
r
e:
,
(4)
By shows of the derivation
of
Equation (2), (3
) and
(4) that the ze
ro cro
ssi
ng o
f
D2oG
function
is th
e zero
-p
oint
of the first de
rivative
of Do
G fun
c
tion,
which
is the l
o
cal extrem
e p
o
i
nts
2
,,
,,
,,
D
x
y
D
xy
k
D
xy
,
,
2
y
x
D
,
,
y
x
D
y
x
,
k
2
,,
,,
,,
Dx
y
k
Dx
y
D
x
y
D
kk
2
,,
,,
,,
D
D
x
y
D
xy
k
D
xy
k
0
k
2
,,
0
0
D
Dx
y
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NI
KA
Vol. 15, No. 2, August 2015 : 336 –
345
338
of DoG
fun
c
tion. So the
ze
ro d
e
tectio
n
of D2o
G
pyra
mid can b
e
u
s
ed to
re
pla
c
e local extre
m
e
point dete
c
tio
n
of DoG pyramid to
achi
e
v
e feature poi
nt extraction.
3.2. D2oG Fe
ature
De
tec
t
ion Steps
(1) E
s
tablish D2o
G
pyrami
d
First, get the
Gauss ima
g
e
at different
scal
e
s by
Gau
ss
ke
rne
l
function for image
convol
ution.
Then, get the
DoG pyra
mi
d by doing su
btractio
n bet
wee
n
adja
c
e
n
t two layers
within the
same
ord
e
r
of Gaussia
n
pyramid. Fi
nally, get
the first layer of D2og p
y
ramid by d
o
ing
subtractio
n b
e
twee
n a
d
ja
cent t
w
o l
a
yers
with
in th
e same
ord
e
r
of Dog
p
y
ramid. In t
he
establi
s
hm
en
t of the first and se
co
nd o
r
der G
a
u
ssi
an
second
-o
rde
r
are
differen
c
e pyramid
s
for
su
ch a
s
sho
w
n in
Figu
re
2. There a
r
e 5 l
a
yers
of Gau
ssi
an
pyramid
wh
ich h
a
ve be
en
establi
s
h
ed. Get
Ga
ussian
differen
c
e pyramid by
doin
g
su
btra
ction
betwe
en a
d
ja
cent two laye
rs
of Gaussia
n
pyramid. And
finally, get 3
layers
Ga
ussian
second
-orde
r
differe
n
c
e pyra
mids
by
doing
subtraction betwe
en
adja
c
ent two l
a
yers of
4 lay
e
rs
Gau
s
sian
difference pyramid.
(2) Dete
ction zero-point
of
eac
h layer of D2o
G
pyrami
d.
By setting t
he
zero poi
nt dete
c
tion
threshold
T,
dete
c
ting pi
xels which
Gau
ssi
an
s
e
c
o
nd
-
o
r
d
e
r
d
i
ffe
r
e
n
c
e
ab
s
o
lu
te
va
lue c
l
os
e to
z
e
ro in
ea
ch
layer
of D2oG
p
y
ramid. Pixel
s
i
s
con
s
id
ere
d
a
s
feature poi
nts when thei
r Gau
s
si
an
seco
nd-order
differen
c
e a
b
s
olute valu
es are
less than or
equal to T. And then re
co
rd t
he locatio
n
and scale
of the point
.Throu
gh
experim
ental
statistics, comprehe
nsiv
e con
s
id
erati
on of both accuracy an
d spe
ed of the
algorith
m
, this pap
er set the threshold T
=0.00
009
9.
(3) Preci
s
e p
o
sitioni
ng fea
t
ure point
s.
Mappin
g
the feature
point
s of D2oG
sp
a
c
e ba
ck
to DoG space, that is: the feature p
o
int
locate in
order
i
layer
j
o
f
D2oG py
ra
mid is
corre
s
pondi
ng to the pixe
l
locate
in o
r
d
e
r
1 l
a
yer
2 of
D2o
G
pyrami
d which ha
s t
he
same
pa
rameter with
the featu
r
e
po
int.
So the p
r
e
c
ise po
sitionin
g
of feature p
o
ints in
DoG
sp
ace
can
b
e
tran
sfo
r
me
d to the
pre
c
ise
positio
ning of
the in DoG p
y
ramid.
(4) Extract
e
dge fe
ature
points. th
e fe
atur
e
point
s i
n
the
edg
e a
r
e
determine
d by the
cha
r
a
c
teri
stics of th
e the
He
ssi
an m
a
trix’s Eigen
va
lues ,whi
ch
are
propo
rtio
nal to the
m
a
in
curvatu
r
e
values of the
G
aussia
n
diffe
ren
c
e fu
nctio
n
D. T
he
He
ssi
an m
a
trix
is a
s
sho
w
n
in
Equation (5).
(5)
Functi
on
on
the
x
dire
ction.
Ta
ke
derivative on the x and
y direction a
nd take
se
co
nd
derivative o
n
wh
ere
re
sp
ectively the
se
co
n
d
de
riv
a
tive of Gau
ssi
an
differe
nce
y
dire
ction.
(6)
And,
(7)
Then,
(8)
,,
ii
i
xy
,,
mm
m
xy
,,
mm
m
xy
yy
xy
xy
xx
D
D
D
D
H
yy
xy
xx
D
D
D
,
,
2
2
2
2
)
(
y
D
x
D
D
D
H
Tr
yy
xx
2
2
2
2
2
2
2
)
(
y
x
D
y
D
x
D
D
D
D
H
Det
xy
yy
xx
k
k
H
Det
H
Tr
2
2
2
1
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
Im
age Mosai
c
Method Ba
sed on Ga
ussi
an Seco
nd
-order Diffe
ren
c
e Feature… (Yong Ch
en
)
339
Whe
r
e
is the
trace of the
H matrix,
is the dete
r
min
a
n
t of H matri
x
,
is the
large
r
eig
en value,
is the smaller ei
gen
value.
k
To dete
r
min
e
wheth
e
r t
he
point is t
he e
dge p
o
int by
setting the
si
ze of
k
. If Eq
uation
(8)
is bigg
er than
this thre
shol
d, then this p
o
int
is extract
ed as the e
d
g
e
point. In this pap
er,
.
Matching accuracy will drop be
cause the dimension
of feature
descriptor i
s
too l
o
w. And
the time-consuming of matching cal
c
ulation w
ill increase because the dimension i
s
too high.
The
origi
nal
128-dime
nsio
nal d
e
scri
pto
r
i
s
a
comp
romise
value.
In the
case
of less
accu
racy
and less time-con
sumi
ng,
the dimensi
on of descr
ip
tor can b
e
re
duced to increase the of the
spe
ed of stitching. After extracting th
e featur
e
points by the propo
sed m
e
thod,
128-dime
nsio
nal
feature descri
ptors are
still used
for its expression and subse
quent regist
rati
on operation.
4. Featur
e Points Ma
tchi
ng and Fusi
on
4.1. Dete
rmining of the T
r
ansformatio
n
Matrix H
(9)
Whe
r
e
are the co
ordinate
s
of the refe
renc
e imag
e,
are the
coo
r
d
i
nates of the
image
whi
c
h is to be
spliced.
are para
m
eters o
f
persp
ective
matrix.
There are ei
g
h
t indepen
de
nt linear equ
a
t
ions c
an be
obtaine
d by four pai
rs of
matchin
g
points. And t
he pe
rspe
ctive tran
sform
a
tion matr
ix fro
m
Equation
(
9
)
ca
n be d
e
te
rmine
d
by
whi
c
h
are
o
b
tained
by e
q
u
a
tion. In o
r
d
e
r
to
av
oid
th
e
equ
ation ha
s
n
o
solutio
n
becau
se of
the
four p
a
irs of randomly
sele
cted
featu
r
e
points
whi
c
h
con
s
titute the
equatio
n lo
cated in the
sa
me
plane, it is necessa
ry to take
the m
e
thod of sol
v
ing initial value of mod
e
l transfo
rma
t
ion
para
m
eters b
y
selectin
g four pai
rs of fe
ature poi
nts randomly. After the initial model pa
ram
e
ters
are
cal
c
ulate
d
, it should
b
e
used to che
ck
other m
a
tching poi
nt. And then all th
e matchi
ng p
a
irs
whi
c
h
meet t
he m
odelin
g
a certai
n tole
ran
c
e
can
be
obtain
ed. Fi
nally, to o
b
tain pa
ram
e
ters of
the image tra
n
sformation
model by
usi
ng these matche
d pair.
4.2. The Extr
action o
f
Fe
ature M
a
tc
hing Points
Usi
ng the 12
8-dim
e
n
s
iona
l feature vect
or of
feature descri
p
tor in
the image. M
a
tchin
g
the SIFT detector
by the Euclide
an di
stance
sim
ila
ri
ty judgment method a
s
shown in Equ
a
tion
(10
)
[12-13]. And finding
o
u
t the minimu
m dist
an
ce
a
nd the
secon
d
small
e
st di
stance
by
Equation (1
0). Then cal
c
ul
ating the ratio
.
T
he corre
s
p
ondin
g
feature points will b
e
the
matchin
g
p
o
i
n
t wh
en
R i
s
le
ss than
a thre
sh
old
value. The
thre
shol
d i
s
set to
0.75
by
comp
ari
s
o
n
. Find the n
earest neig
hbo
r
and next ne
a
r
est n
e
igh
bor by the Best-Bin-First (BB
F
)
algorith
m
.
(10)
Whe
r
e
is the
eigenve
c
tors
of feature
poi
nts in
the
ref
e
ren
c
e
imag
e
.
is the i-th
feature
point ve
ctors in ima
ge
wh
ich i
s
to
be
registe
r
ed.
D i
s
the
Eucli
d
e
an di
stan
ce.
Excluding
fal
s
e
matchin
g
poi
nts by the RANSAC algo
rithm [14]
and calculating
the transformation matri
x
H.
Steps are as
follows
To cal
c
ulate
matrix H by selectin
g four gr
ou
ps of mat
c
hin
g
points randomly. And
then to
cal
c
ulate
the
distan
ce
of all the
rem
a
inin
g mat
c
hin
g
p
o
ints. T
he
m
a
tchin
g
p
o
int
s
a
r
e
inte
rior
point of H if the value of
is in
the sco
pe
of the erro
r th
reshold.
H
Tr
H
Det
10
k
1
1
8
5
2
7
4
1
6
3
0
i
i
i
i
y
x
h
h
h
h
h
h
h
h
h
y
x
i
i
y
x
,
i
i
y
x
,
8
0
h
h
8
0
h
h
mi
n
D
s
cn
D
mi
n
s
cn
RD
D
128
2
21
1
()
ii
j
D
Xj
X
j
2
i
X
j
1
i
X
j
i
d
i
d
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NI
KA
Vol. 15, No. 2, August 2015 : 336 –
345
340
(1)
To recalculate H
by the set of p
o
in
ts
whi
c
h i
n
cl
ude the l
a
rg
e
s
t num
ber
of interio
r
point. To minimize erro
rs
by using the
least
squ
a
re
s method. T
hen to cal
c
ul
ate the average
error of a ne
w set of point
s after re
movi
ng a few “o
utside p
o
ints”.
(2) Repe
at the above two step
s
until achieve satisfie
d effect. To calcul
ate the final homo
grap
h H with
the point
set
whi
c
h h
a
s th
e
smalle
st
ave
r
age
erro
r. Th
e tran
sform relation of Ima
ge
and
can b
e
expre
s
sed a
s
:
(1
1)
4.3. Image Fusion
To reali
z
e t
he se
amle
ss Mosai
c
wit
h
Gra
dual f
ade out fu
si
on metho
d
[15] after
regi
stratio
n
. The pixel values of the non-ove
r
la
ppin
g
area keep
R, G, B
value of their pixels
remai
n
s the
same in the two ima
ges. A
nd the pixe
l value
s
of the overlap
p
ing a
r
ea to obtai
n R,
G, B value of
the new pixel
s
value
s
with
t
he weig
ht value. As sh
own
in Equation (12).
(12
)
Whe
r
e
a
re
th
e fused i
m
ag
e, the
referen
c
e i
m
age
a
n
d
the
ima
g
e
stitchi
n
g
.
are wei
ght value
s
.
is the pixel coo
r
din
a
te.
Assu
ming
th
at the a
b
sci
s
sa
of current
pixel i
s
,
and
the a
b
sci
s
sa of left a
n
d
right
boun
dary of the overla
p re
gion are
,
.
So
,
are cal
c
ul
ate
d
as follo
wed:
(13)
5. Results a
nd Discu
ssi
on
The lat form
for experi
m
ents a
r
e VS
2010 a
nd O
pen
CV. The
size of ima
ge is 3
40
280.In this p
aper,
we cho
o
se the n
u
m
ber of
imag
e
matching
po
ints, spli
cing
time, matchin
g
accuracy an
d
matching efficien
cy
as the evaluati
on. And the
matchin
g
effi
cien
cy [16
-
1
7
] is
s
h
ow
n
in
Eq
ua
tio
n
(
1
4)
.
(14)
Whe
r
e,
Our expe
rim
ent cho
o
ses
four kin
d
s of
typical
test cha
r
t for testi
ng. Figure 3(a) is the
image of the vertical tran
slation. Figu
re 3(b)
i
s
an
image with a sub
s
tantial
chang
e in the
brightn
e
ss a
nd contrast. Figure
3
(
c)
i
s
the
imag
e whi
c
h cam
e
ra
viewi
ng
a
n
g
le cha
nge
s 40
degree. Figu
re 3(d
)
is th
e image
whi
c
h i
s
rotate
d by 45 deg
re
e an
d red
u
ced by
half. Espe
cia
lly,
the image
s o
n
the left of the 4 g
r
ou
ps
are the
refe
re
nce im
age
s a
nd on th
e rig
h
t of the 4 g
r
oup
s
are the
stitchi
ng imag
es.
Choo
sing th
e
module
-
b
a
se
d image
mo
saic a
nd ima
g
e mosaic
ba
sed
on SIFT feature p
o
ints
co
mpare with t
he imag
e mo
sai
c
metho
d
based on
ze
ro detectio
n
we
prop
osed.
1
I
2
I
1
2
I
H
I
y
x
I
d
y
x
I
d
y
x
I
,
,
,
2
2
1
1
y
x
I
,
y
x
I
,
1
y
x
I
,
2
1
d
2
d
y
x
,
i
x
l
x
r
x
1
d
2
d
l
r
l
i
l
r
i
r
x
x
x
x
d
d
x
x
x
x
d
1
2
1
1
,
()
()
()
M
at
c
h
in
g
a
c
c
urac
y
M
a
tc
hing
ef
fic
ienc
y
C
o
mputat
ion
t
ti
m
e
C
o
nsiste
nt
fo
cu
s
o
n
m
a
t
ch
poin
t
pa
irs
A
ll
of
the
m
atc
h
ing
p
oint
pairs
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
Im
age Mosai
c
Method Ba
sed on Ga
ussi
an Seco
nd
-order Diffe
ren
c
e Feature… (Yong Ch
en
)
341
(a) Ve
rtical transl
a
tion
(b) T
he contrast su
bsta
ntial cha
nge
d
(c
) 40° p
a
rall
ax
(d) 4
5
° rot
a
tion and re
du
ce
d by half
Figure 3. Test chart
5.1. The Experimental Re
sults
(1) Mo
dule
-
b
a
se
d image
mosai
c
The rend
erin
g after
stitchi
ng of 3
(
a
)
whi
c
h
is
pr
oc
es
se
d
b
y
mod
u
l
e-
b
a
s
e
d ima
ge mo
sa
ic
[18-20] a
s
sh
own in Fig
u
re
4.
Figure
4. The
rende
rin
g
of module
-b
ase
d
The p
hen
om
enon
of di
slo
c
ation
of te
mplate-ba
s
ed
regi
stration
method
ca
n
be
see
n
form Figu
re
4. It means that the imag
e can’t fi
nd t
he be
st match positio
n well whe
n
it is in
rotation.
(2) T
he SIFT algorith
m
(a) Ve
rtical transl
a
tion
(b) T
he contrast sub
s
tanti
a
l cha
nge
d
(c) 40° p
a
rall
ax
(d) 4
5
°r
otatio
n and re
du
ce
d by half
Figure 5. SIFT feature de
scripto
r
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NI
KA
Vol. 15, No. 2, August 2015 : 336 –
345
342
We u
s
e the
SIFT algorith
m
to do the feature ex
traction of the four photo
s
in the Figu
re
3. And to
d
e
t
ermine
the
numbe
r
of S
I
FT featur
e
points in
Fig
u
re
5(a) a
r
e
137
6
and
1
0
71
respe
c
tively. The num
be
r of SIFT feature poi
nts
in Fi
gure
5(b
)
a
r
e
350 an
d 246
. The numb
e
r of
SIFT feature
points in
Figu
re 5
(
c) are 69
8 and 8
39.
T
he num
b
er
of SIFT feature
points in
Fig
u
re
5(d
)
are 7
73
and 60
7.
Then we cal
c
ulate
Figu
re 5
feature poi
nts
by
kd-tre
e algo
rithm to obtain
crud
e matchi
ng
point pai
rs:
5
66 pai
rs (Fi
g
ure
6(a
)),
12
1 pai
rs
(F
ig
u
r
e 6
(
b
)), 1
0
1
pairs
(Figu
r
e 6(c)), 23
0p
airs
(
F
ig
ur
e
6(
d
)
)
.
(a) Ve
rtical
transl
a
tion
(b) Cha
nge
s of
brightn
e
ss an
d
contrast
(c) 40° p
a
rall
ax
(d) 4
5
° rot
a
tion and
redu
ce
d by half
Figure 6. Fea
t
ure point
s pa
irs
To eliminate false mat
c
hin
g
points by RAN
SAC algo
rithm and cal
c
ulate the perspective
transfo
rmatio
n matrix H. To tran
sform
the sp
licin
g
image into the coo
r
di
nat
e system of the
referen
c
e ima
ge. And then
get the mosai
c
im
age by i
m
age fusi
on
as sho
w
n in
Figure 7.
(a) Ve
rtical
transl
a
tion
(b) Chan
ge
s of
brightn
e
ss an
d
contrast
(c) 4
0
° pa
rall
ax
(d) 4
5
° rot
a
tion and
redu
ce
d by half
Figure 7. Image of SIFT fusion meth
od
The a
bove
calcul
ation
sh
ows that the
numb
e
r
of feature
poi
n
ts in Figu
re
5, Figure 6
detectio
n
are
large. Ho
we
ver, we nee
d only 4 pai
rs of feature
points pair
to calcul
ate the
transmissio
n
transfo
rmati
on matrix. It will
lead to long co
m
putation time and mem
o
ry
con
s
um
ption.
(3) SIFT alg
o
r
ithm of D2o
G
ze
ro dete
c
t
i
on
We u
s
e the
SIFT algorith
m
of D2oG
zero
dete
c
tio
n
to extract i
m
age featu
r
e
points,
image
regi
stration an
d im
age fu
sion. T
hen o
b
tain th
e crude
match point
s after coa
r
se mat
c
h:
199 pai
rs
(Fig
ure 8
(
a)), 15
pairs (Fig
ure 8(b
)),
18 pai
rs (Fig
ure 8
(
c)), 18pai
rs (Fi
gure 8
(
d
)
)
(a) Ve
rtical
transl
a
tion
(b) Cha
nge
s of
brightn
e
ss an
d
contrast
(c) 40° p
a
rall
ax
(d) 4
5
° rot
a
tion and
redu
ce
d by half
Figure 8. Fea
t
ure point
s pa
irs
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
Im
age Mosai
c
Method Ba
sed on Ga
ussi
an Seco
nd
-order Diffe
ren
c
e Feature… (Yong Ch
en
)
343
To eliminate
false matchin
g
points by RA
NSAC algo
rithm and cal
c
ulate the parameter
of coordinate
transfo
rmatio
n matrix H of stit
chin
g imag
e in Figure 3 by Equation (9).
After the paramete
r
of transfo
rmatio
n
matr
ix H, r is determin
ed, we ca
n get the
transmissio
n diagram of Figure 3.
(a) Ve
rtical
transl
a
tion
(b) Cha
nge
s of
brightn
e
ss an
d
contrast
(c) 40° p
a
rall
ax
(d) 4
5
° rot
a
tion and
redu
ce
d by half
Figure 9. Tra
n
smi
ssi
on dia
g
ram
To fuse the
transmi
ssion
di
agra
m
sho
w
n
in Fig
u
re
9 a
nd the
refe
re
nce
imag
e
sh
own
in
Figure 3 by the grad
ual fad
e
out method,
we ca
n get the fuse
d ima
ge as
sho
w
n i
n
Figure 10.
(a) Ve
rtical
transl
a
tion
(b) Cha
nge
s of
brightn
e
ss an
d
contrast
(c) 40° p
a
rall
ax
(d) 4
5
°rotatio
n and
redu
ce
d by half
Figure 10. Th
e D2o
G
meth
od image
To verify the effectiveness
of the proposed
method, this
article will analy
s
e and
discu
ssi
on bo
th subje
c
tive and obj
ective
evaluation.
5.2. Analy
s
is
and Discu
s
s
ion
It can be see
n
from Figure 10 that the mosai
c
effect of the meth
od we p
r
op
o
s
ed i
s
a
sup
e
rio
r
mod
u
le-b
ased re
gistratio
n
m
e
thod
u
s
e
d
in
Figu
re
4. It
better
solve
s
the
pro
b
lem
of
image disl
ocation.
The
m
odule
-
ba
se
d mosai
c
meth
od
ha
s gre
a
t limitations wh
en
de
al with the
rotation
scale
d
image. So
the method b
a
se
d on feat
ure p
o
ints i
s
more p
r
a
c
tical. By observ
i
ng
Figure 7 an
d
10, we
can fi
nd that the di
fference
of splicin
g effect
betwe
en the
method
whi
c
h is
based o
n
the
SIFT algorith
m
and the
propo
sed m
e
th
od is
not obvi
ous. To ve
rify the effect of the
prop
osed
me
thod, we
com
pare
the t
w
o
method
s by
t
he follo
wing
obje
ctive eval
uation. Fe
atu
r
e
-
0
.
99907
0.
000040
132.
001421
0.
000
040
132.
00
1421
-0.
000017
132.
001421
-0.
000017
-
0
.
999
958
H
991628
.
0
000472
.
0
302306
.
1
000472
.
0
302306
.
1
017415
.
0
302306
.
1
017415
.
0
996046
.
0
H
086682
.
1
871204
.
0
183310
.
108
871204
.
0
183310
.
108
858201
.
0
183310
.
108
858201
.
0
051323
.
1
H
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 23
02-4
046
TELKOM
NI
KA
Vol. 15, No. 2, August 2015 : 336 –
345
344
points extra
c
t
i
on, matchin
g
accuracy an
d efficien
cy
of image stitch
ing is the key to real-time
of
spli
cing. We use the n
u
m
ber of all the
matching p
o
i
nts pairs (i
e coa
r
se match
i
ng point
s), the
numbe
r of consi
s
tent focus ma
tching
points, stitching time,
matchin
g
accu
ra
cy and matching
efficien
cy as the evaluatio
n
to compa
r
e the two metho
d
s.
Table
1 sho
w
s the
experi
m
ental calculati
on dat
a
of
spl
i
cing. It can
b
e
se
en from
Table 1
that the numb
e
r of match p
o
ints extra
c
te
d by the
prop
ose
d
method,
stitching time
is signifi
cantl
y
lowe
r than th
e data obtai
n
ed by SIFT algorithm
wh
il
e matchi
ng a
c
cura
cy, efficiency is
high
er
than the latter. It proves the feasibility and effe
ctiven
ess of the propo
sed meth
od is better t
han
SIFT algorith
m
.
Table 1
Stitching Images
Fig3(a)
Fig3(b)
Fig3(c)
Fig3(d)
Evaluation Index
SIFT
Our M
e
thod
SIFT
Our
M
e
thod
SIFT
Our
M
e
thod
SIFT
OurMe
t
hod
Rough
Matching
566
199
121 15 101
18 230
18
Fine
Matching
536
196
96 12 65
15
199
14
Matching Accuracy
（
γ
/%
）
93.9%
99.0%
79.4%
93.3%
77.5%
65.4%
85.8%
77.8%
Splicing
Time(t/s)
7.003
3.422
2.057
1.384
4.572
2.431
4.298
2.554
5. Conclusio
n
Based
on
st
udy of the S
I
FT feature
matchin
g
al
g
o
rithm, the
i
m
prove
d
SIF
T
features
stitchin
g alg
o
r
ithm ba
se
d
on D2oG fe
a
t
ure dete
c
tio
n
ope
rato
r is prop
osed in
this pa
per.
The
improve
d
alg
o
rithm im
pro
v
e the spee
d
by simp
lifyin
g
the
stru
ctu
r
e of
pyrami
d whi
c
h
mea
n
s
redu
ce t
he n
u
mbe
r
of lay
e
rs of Ga
ussi
an pyra
mi
d a
nd repla
c
ing
the extremu
m
of pixels
of the
three
-
dime
nsi
onal pl
ane i
n
DoG
sp
ace with the ze
ro
pi
xels in a
sin
g
l
e
layer of
a two-dime
nsio
nal
plane
of D2
o
G
sp
ace. And
the D2
oG py
ramid
still
ret
a
ins th
e DoG
pyramid
effective layer du
ri
ng
con
s
tru
c
tion
whi
c
h i
s
to e
n
su
re the
effectivene
ss of
image
stitchi
ng. Experime
n
tal sh
ows th
at
the propo
sed
method
ha
s
a better
effect of stitchin
g
to col
o
r, g
r
ay
scale ima
ge.
It improves t
he
effectivene
ss of stitchin
g.
And it
ha
s a
better effe
ct of stitchin
g
to
image with
t
r
an
slation, small
angle differen
c
e
s
and
small
deformatio
n
. So it has a ce
rtain refe
ren
c
e value.
Ackn
o
w
l
e
dg
ements
Authors
woul
d like to
than
k the
Chon
g
q
ing
E
ducation Committe
e
Sci
e
n
c
e of Chin
a
fo
r
sup
portin
g
the Found
ation
of prog
ram, No KJ140
043
4
.
Referen
ces
[1]
Lin J
i
nme
i
, Zhou Fug
en, Ji
n
T
i
ng. Stabilizat
ion
alg
o
rithm
b
a
sed
on SIFT
feature for
d
y
n
a
mic air
bor
n
e
vide
o.
Infrared
and L
a
ster Eng
i
ne
erin
g.
2011; 40(12): 2552-2557.
[2]
Yang
Z
han
lo
ng
. Rese
arch
on
i
m
age
reg
i
strati
on
and
mos
a
ic
base
d
o
n
fe
atu
r
e p
o
int. Sh
an
xi: Gradu
ate
XI
DI
A
N
.
[3]
Z
hang Y
u
ji
e.
Stud
y on t
he
Detectin
g a
n
d
Matchi
n
g
T
e
chni
que
of Loc
al Invar
i
ant F
e
ature a
nd Its
Appl
icatio
ns. Jiangs
u: Gradua
te Nanj
ing U
n
iv
ersit
y
of Sci
enc
e and T
e
chno
l
o
g
y
. 201
0.
[4]
W
ang W
e
i, T
ang Yi
pi
ng, R
e
ng Ju
ali. A
im
pr
ove
d
a
l
gor
ith
m
for Harris c
o
rner
detecti
o
n
.
Optics and
Precisio
n Eng
i
neer
ing.
2
008;
16(1
0
): 199
5-2
001.
[5]
JIANG DG, Y
I
JK.
Comp
aris
on an
d study
of classic
feat
ure po
int dete
c
tion al
gorith
m
. Compute
r
Scienc
e & Service S
y
st
em. Nanji
ng,
P.R. Ch
ina. 20
12: 23
0
7
-23
09.
[6]
JB R
y
u, HH P
a
rk. Log-lo
g scal
ed Harris c
o
rn
er detector.
Ele
c
tronics Letters
.
2010; 46(
24): 160
2-16
04.
[7]
LOW
E
DG.
Distinctive
im
age fe
atures
from scale-i
nv
aria
nt ke
ypo
i
n
t
s.
Internation
a
l Jo
urna
l o
f
Co
mp
uter Visi
on.
200
4; 60(2
0
): 91-11
0.
[8]
W
ang M
e
i, T
u
Da
xi
on
g, Z
h
o
u
Xuc
h
a
o
. Mo
ving
ob
ject
de
tection
b
y
c
o
mbin
g SIF
T
and
differe
ntia
l
multipl
i
cati
on.
Optics and Pre
c
ision E
ngi
ne
e
r
ing.
20
11; 19(
4): 1391-
13
97.
[9]
Z
eng Lu
an, W
ang Yu
an
qin,
T
A
N Jiubin. Improve
d
alg
o
rith
m for SIFT
feature e
x
tractio
n
and match
i
ng
.
Optics and Pre
c
ision E
ngi
ne
e
r
ing.
20
11; 19(
6): 1391-
13
97.
[10]
Liu
Xian
gze
ng,
T
i
an Z
heng, S
h
i Z
h
e
n
g
uan
g,
CHEN Z
H
SH.
SAR ima
ge m
u
lti-scal
e
re
gist
ration
bas
ed
on KF
ICA-SIF
T
features.
Op
tics and Precis
i
on Eng
i
n
eeri
n
g
.
2011; 19(
9): 2186-
219
5.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
2302-4
046
Im
age Mosai
c
Method Ba
sed on Ga
ussi
an Seco
nd
-order Diffe
ren
c
e Feature… (Yong Ch
en
)
345
[1
1
]
ZH
U
ZH
W, SH
EN
ZH
F, LUO JC
H
.
Pa
ra
l
l
e
l
re
mo
te se
nsi
n
g i
m
ag
e reg
i
stra
ti
on
b
a
s
ed
on
i
m
p
r
o
v
ed
SIFT
point feature.
Journ
a
l of
Re
mote Se
nsi
n
g
. 201
1; 15(5)
: 1024-1
0
3
1
.
[12]
Z
hang H, Li
ng
J, Lian Y.
F
a
st mosaic d
e
t
e
ction for rea
l
-time vi
de
o ba
sed on te
mp
la
te matchi
ng
strategy
. Image Anal
y
s
is an
d Sign
al Process
i
ng, In
ternati
o
n
a
l Co
nferenc
e
on IEEE. 2012:
1-4.
[13]
WAN LZH, CHONG-WAH NGO. Flip-Invaria
nt SIF
T
for copy
and object
detection.
IEEE Transaction
on Image Proc
essin
g
. 201
3; 22(3): 980-
99
1.
[14]
Z
HU XF
, MA CW
, LIU B.
T
a
rget classi
fication us
in
g SIFT
sequenc
e scale i
n
vari
ants.
System
s
Engi
neer
in
g an
d Electron
ics
. 201
2; 23(5): 63
3-63
9.
[15]
YANG BLST
. Multifocus i
m
age fusi
on
and restrati
on
w
i
th sp
a
r
se repres
ent
ation.
IEEE
TRANSACTIONSON INSTRUMENTATION AND MEASUREMENT.
201
0; 59(4): 88
4-8
92.
[16]
Liu
Li, Pe
ng F
u
yu
an, Z
h
a
o
Ku
n. Simpl
i
fied SI
FT
algorithm fo
r fast image m
a
tchin
g
.
Infrare
d
an
d L
a
s
e
r
Engi
neer
in
g.
2008; 37(
1): 181
-184.
[17]
Z
hang
W
e
izh
ong, Y
ang
Houj
un, Z
h
an
g Li
ya
n. A
subp
i
x
el
matc
hin
g
a
l
gor
ith
m
for curv
e
corresp
ond
enc
es from im
ag
e
s
.
Journ
a
l
of B
e
iji
ng
Un
iversit
y
of Posts
and
T
e
leco
mmunic
a
tions
. 20
08;
31(4): 66-
69.
[18]
Ding
Xiao
li. Stud
y on im
ag
e
stitching tech
no
l
o
g
y
. Nan
jin
g: Graduate S
out
heast Un
iversit
y
. 20
09.
[19]
Yong C
h
e
n
, Jie
Xio
ng, H
u
a
n
-lin
Liu, Qia
n
g
F
an. Comb
i
ne T
a
rget Ext
r
action a
nd E
nha
nceme
n
t
Methods
to F
u
se Infrare
d
a
n
d
L
LL Ima
ges.
T
E
LKOMNIKA Indon
esi
an J
o
urna
l of T
e
leco
mmu
n
icati
o
n
,
Co
mp
uting, El
ectronics a
nd
Contro
l.
201
4; 12(3): 60
5-6
1
2
.
[20]
Jian
hua
L, Jia
ngu
o Y.
Multif
ocus im
age fu
sion
b
y
SML i
n
the sh
earl
e
t
subb
an
ds.
TEL
K
OMNIKA
Indon
esi
an Jou
r
nal of Electric
al Eng
i
ne
eri
ng.
2014; 1
2
(1): 6
18-6
26.
Evaluation Warning : The document was created with Spire.PDF for Python.