TELKOM
NIKA
, Vol.12, No
.2, June 20
14
, pp. 379~3
8
8
ISSN: 1693-6
930,
accredited
A
by DIKTI, De
cree No: 58/DIK
T
I/Kep/2013
DOI
:
10.12928/TELKOMNIKA.v12i2.2053
379
Re
cei
v
ed Ma
rch 1
6
, 2014;
Re
vised April
28, 2014; Accepte
d
May 1
2
, 2014
A Novel Multi-focus Image Fusion Method Based on
Non-ne
gative Matrix Factorization
Yongxin Zha
ng
1,2
, Li Chen*
1
, Zhihua Zhao
1
, Jian Jia
3
, Jie Chen
4
1
School of Info
rmation Sci
enc
e and T
e
chno
l
o
g
y
,North
w
e
st Univers
i
t
y
,
Xi’
a
n 710
12
7, Sha
a
n
x
i,C
h
in
a
2
Luo
yan
g
Nor
m
al Univ
ersit
y
,
Luo
ya
ng 4
7
1
0
22, He’
n
a
n
, Ch
ina
3
Department o
f
Mathematics, North
w
e
s
t Un
iv
ersit
y
,
Xi’
an 7
1
012
7, Shaa
n
x
i,
Chin
a
4
Xi'
an C
o
mmu
nit
y
Inform
atio
n Service & Ma
nag
em
ent C
e
n
t
er, Xi’
an 71
00
05, Shaa
n
x
i, C
h
in
a
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: tabo12
6@
12
6.com
A
b
st
r
a
ct
In ord
e
r to effi
ciently
extract the focus
ed r
e
g
i
ons
fro
m
th
e s
ource
imag
es
and
i
m
prov
e th
e qu
ality
of the fuse
d
imag
e, this p
aper
prese
n
ts
a nov
el
i
m
a
ge fusi
on sc
h
e
me w
i
th n
o
n
-
neg
ative
matr
ix
factori
z
a
t
io
n (
N
MF
). T
he source i
m
ages
a
r
e fused
by
N
M
F
to constru
c
t temp
orary f
u
sed
i
m
ag
e, w
hose
regi
on h
o
m
og
e
neityis us
ed to
split the so
urce i
m
ag
es
into r
egi
ons.T
he foc
u
sed re
gi
ons a
r
e detecte
d an
d
integr
ated
to c
onstruct the
fin
a
l fus
ed
i
m
a
ge.
Exper
imenta
l
r
e
sults
de
mo
nstrat
e that th
e
pr
opos
edsc
h
e
m
e
i
s
capa
ble
ofeffici
ently extracti
ng
the focus
ed r
e
gio
n
s a
nd si
gn
i
f
icantly i
m
provi
ng the
fusio
n
q
uality c
o
mpar
e
d
to other existing fusion
m
e
thods,in term
s
of visual
an
d qu
anti
t
ative eval
uati
o
ns.
Ke
y
w
ords
: image fusi
on, no
n-ne
gative
mat
r
ix factori
z
at
i
o
n
,
quad tree d
e
c
o
mpos
ition, re
gio
n
ho
mo
ge
n
e
ity
1. Introduc
tion
Multi-focus i
m
age fu
sion
has b
een p
r
o
v
en to be an
effective way
to extend the
depth of
the field [1]. Image fusio
n
aims to produ
ce a sin
g
le sha
r
p
e
r i
m
age by co
mbining a set of
image
scaptu
r
ed from the
same
scene
with differe
nt focus p
o
i
n
ts. In gene
ral, image fusion
method
s
ca
n
be
categ
o
ri
ze
d into t
w
o
gro
ups:
sp
atial d
o
main
fusio
n
and t
r
an
sform dom
ain fu
sion
[2]. The spatial domain fu
sion meth
od
s are e
a
sy to implement
and have lo
w com
putatio
nal
compl
e
xity, while the
spati
a
l dom
ain m
e
thod
s
may prod
uce
blo
c
king artifa
cts and com
p
ro
mise
the qu
ality of the final
fused ima
ge.
Di
fferent
fro
m
t
he
spatial
do
main fu
sion,
the tran
sfo
r
m
domain fu
sio
n
method
s may achieve i
m
prove
d
con
t
rast, as well
as better si
g
nal-to
-
noi
se ratio
and better fu
sion quality, but the transfo
rm domai
n fusion meth
od
s are time/sp
a
c
e-co
nsumin
g to
implement[3],[4]. This paper parti
cularly focuse
s on the spatial dom
a
in fusion methods.
Lee a
nd Su
n
g
[5]develop
e
d
non
-n
egative matrix
fa
cto
r
izatio
n (NMF
) in 1
999.A
s
a novel
techni
que, it can de
com
pose multivariate data
into a smalle
r numbe
r of basi
s
vectors an
d
encodin
g
und
er no
n-n
egati
v
e con
s
trai
nts, and
can al
so reveal the
latent
stru
cture, feature a
n
d
pattern
of the
input d
a
ta [6
]. Zhang et
al
. [7] have firstly applied
NMF in ima
ge
fusion
by u
s
i
n
g
sha
r
pn
ess co
nstrai
nts an
d
achieved
bet
ter fu
sed
result. So far,
m
any multi-fo
cus i
m
ag
e fu
si
on
method
s b
a
sed o
n
NMF
have be
en
d
e
velope
d [8]-[
13]. But mo
st of them
suffer from various
probl
em
s. Xu et al.
[8] ha
ve propo
se
d a fusion met
hod usi
ng NMF coeffici
en
ts to detect the
focu
sed im
a
ge blo
ck. It works bette
r in pre
s
e
r
vi
n
g
the sali
ent
information,
but suffers
from
contrast re
du
ction and al
gorithm
com
p
lexity.
Zhan
g et al. [9]
have p
r
op
osed a fa
st fu
sion
method b
a
se
d on weighte
d
non
-ne
gative matrix fa
cto
r
izatio
n (WNMF) an
d re
gi
on segme
n
tation
in spatial d
o
m
ain.Visibl
e and infra
r
e
d
image
s are
en
dowed with di
fferent wei
ght
. It works
well
for
multi-focus i
m
age, visibl
e
and infrare
d
image, but
suffers fro
m
the influen
ce
of para
m
ete
r
s
setting
and
the
compl
e
xity of regi
on
segmentatio
n
algorith
m
. Ye
et al. [10]
h
a
ve develo
p
e
d
a
fusion meth
o
d
based on
local no
n-n
egative matr
i
x
factorizatio
n (LNMF). It improve
s
the
obje
c
tive fun
c
tion of the
stand
ard
NM
F to enh
an
ce localization
con
s
traint a
nd works wel
l
for
SAR and vi
si
ble imag
e. But it does
not
wo
rk
well in
the glob
al fe
ature extracti
on an
d the d
e
tail
feature
rep
r
e
s
entatio
n. Li
u et al. [11]
hav
e p
r
op
o
s
ed
a fu
sion
schem
e ba
sed
on
dyna
mic
WNMF. This
scheme e
nha
nce
s
the abili
ty of feat
ure extraction a
n
d
improve
s
the visual qu
a
lity
of the fused
image, b
u
t con
s
um
es m
o
re time.
W
a
ng et al. [12
]
have devel
oped th
e fusion
method
ba
se
d on
accel
e
ra
ted NM
F in
n
on-sub
s
a
m
pl
ed
contou
rlet
tran
sform
(NSCT)
domai
n
.
It
pre
s
e
r
ves m
o
re e
dge d
e
tails info
rmati
on of the
sou
r
ce i
m
age
s a
nd improves
the quality of the
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 12, No. 2, June 20
14: 379 – 38
8
380
fused ima
ge,
but it doesn
’
t approp
riate
for lar
ge
scale data sets. Most of the existing fusi
on
method
s
ba
sed o
n
NMF
u
s
e
matrix fa
ct
orization
to
e
x
tract
salie
nt
feature
of the
so
urce
imag
es.
But NMF i
s
confronted
with two mai
n
pro
b
le
m
s
which are
un
satisfacto
ry
a
c
curacy and b
a
d
gene
rality. The processe
d
object
s
of NMF are i
n
trin
sically vectors and
ne
ce
ssaryve
ctori
z
a
t
ion
for eve
r
y ma
trix in the
proce
s
sed
mat
r
ix-se
t, whi
c
h
often ma
ke
the
corre
s
p
ondin
g
NMF
b
e
atypical
kind
of small
-
sam
p
le learni
ng and
comp
romi
ses the
ability of NM
F in
generali
z
ation and
feature repre
s
entatio
n [13].
Different fro
m
the metho
d
s me
ntioned
above,
this
pape
r presen
ts a novel NMF based
image fu
sion
scheme.
NMF is u
s
e
d
to co
nstruct t
he tempo
r
a
r
y fused im
age
and extra
c
t
the
unde
rlying sa
lient informati
on fromthe source imag
e
s
. To inhibit blockin
g
artifacts, the sou
r
ce
image
s
a
r
e split
based on the
re
gion ho
mogen
eity
of
the tempo
r
a
r
y fused im
ag
e. The o
b
je
ctive
of this pa
pe
r i
s
to imp
r
ove t
heefficie
n
cy a
nd
pe
rforman
c
e of th
e fusi
on meth
od. T
he con
s
tru
c
ti
on
of temporary
fused im
age
with NMF fo
r effectiv
e im
age fu
sion i
s
the main co
ntribution
ofthis
pape
r. The propo
sed meth
od can effici
e
n
tly extract
the focu
sed re
gion
s details
from the sou
r
ce
image
s and i
m
prove the vi
sual q
uality of the fused im
age.
The
rest of the paper is org
anized
as follows. In Section 2,
the basi
c idea
of NM
F will
be
briefly descri
bed,follo
wed
by the new method with
NMF for image
fusion in Section 3. In Section
4,
extensive
simulatio
n
s
a
r
e perfo
rme
d
to
evaluat
e
th
e pe
rforman
c
e of th
e p
r
op
ose
d
m
e
thod.
In
addition, sev
e
ral expe
rime
ntal results are pre
s
ente
d
and discu
s
se
d. Finally, conclu
ding rem
a
rks
are d
r
a
w
n in
Section 5.
2. Non-n
e
ga
tiv
e
Matrix Factori
z
ation
NMF i
n
corp
o
r
ates the
no
n-ne
gativity con
s
tr
ai
nt an
d thu
s
o
b
tains th
e p
a
rts-ba
se
d
rep
r
e
s
entatio
n as
well
as e
nhan
ce
s the i
n
terp
retab
ility
of the issu
e
corre
s
p
ondin
g
ly [14],which
is
a low-ran
k
a
pproxim
ation techni
que for un-supe
rvise
d
multivariate
data analysi
s
and produ
ces
non-neg
ative
matrix to p
r
o
c
ess a
n
im
age
[15]. NM
F fa
ctori
z
e
s
a
nm
ori
g
inal m
a
trix
V
into two
factor mat
r
ices. One i
s
a
nr
non-n
egativ
e basi
s
matrix
W
and the
other is a
rm
non-
negative weig
ht matrix
H
.
()
()
ij
ia
ia
a
j
j
ij
ia
ia
ja
j
ij
aj
aj
ia
i
ij
V
WW
H
WH
W
W
W
V
HH
W
WH
(1)
The two facto
r
matri
c
e
s
ca
n app
roximat
e
the o
r
igin
al
matrix
V
ac
cor
d
ing to some c
o
s
t
functions. The conver
gence of the algori
t
hm has been
proved by Lee and Seung [5],[6].
Re
cent years, several vari
ant
s of NMF
su
ch a
s
LNM
F
[16
], sparse non-neg
ative matrix
factori
z
ation
(SNMF
)
[17]
and n
o
n
-
ne
gative matr
ix
factori
z
atio
n
with spa
r
se
ness
con
s
traints
(NM
F
sc) [18
]
have b
een
pro
p
o
s
ed
to imp
r
ove
NMF f
r
om v
a
riou
s
pe
rsp
e
ctives.
The
s
e
extensio
ns a
r
e mainly perf
o
rme
d
on m
odified mo
d
e
l
s
, modified constraints a
n
d modified cost
function
s. O
ne impo
rtant
variable in
the redu
ction
of dimensio
ns in ea
ch
NMF metho
d
is
comm
only ca
lled the varia
b
le
r
(number of the basis vector).
How
large a data
matrix will be
redu
ce
d is
d
e
termin
ed by
r
. The larg
er
value of
r
the
smalle
r dim
ensi
on is
red
u
ce
d and the
smaller value of
r
the large
r
dimensi
on is redu
ced [19]
.Due to NMF
is a method t
o
find a part-
based repre
s
entatio
n of
origin
al
dat
a, the so
urce ima
g
e
s
are lin
ear
a
nd no
n-n
ega
tive
combi
nation
s
of the
r
ba
sis ima
g
e
s
derived from
NMF on th
e sou
r
ce im
age
s wh
en the
para
m
eter
of NMF i
s
set to
r
(
1
r
). Similarly, the sou
r
ce i
m
age
s are lin
ear a
nd no
n-negative
combi
nation
s
of the only one pa
rts d
e
riv
ed from NMF on the source ima
g
e
s
when
1
r
. The
only part d
e
ri
ved from
NM
F with
1
r
on the so
urce ima
ges i
s
t
he
su
bstantial fe
ature of th
e
sou
r
ce ima
g
e
s
. The
su
bsta
ntial feature
of the
so
urce
image
s
can
be seen
as t
he glo
bal feat
ure
of the sou
r
ce
image
s. Th
e
fuse
d ima
g
e
ca
n be
obtai
ned from the
ob
serve
d
im
age
s by u
s
in
g
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A Novel M
u
lti-focus Im
age Fusion Method Ba
sed on Non-Negative .
... (Yongxin Z
hang)
381
NMF with
1
r
. The ob
se
rved
image i
s
ca
st as th
e sou
r
ce imag
es i
n
i
m
age fu
sio
n
. The m
u
lti-
focu
s imag
e fusio
n
by usin
g NMF i
s
sh
o
w
n in Fig
u
re
1
.
The sou
r
ce
image
s an
d the fuse
d ima
ge
obtaine
d by
NMF
are
sho
w
n i
n
Fi
gure
s
1
(a
), (b) an
d (c),
re
sp
ect
i
vely. It is ob
viously that t
h
e
fused
imag
e
is extra
c
ted
the
sub
s
tantia
l featur
e
of
source i
m
age
s. The
sha
r
p
regi
on
s of t
h
e
fused im
age
in Figu
re1
(c) are corresp
ondin
g
to
the sh
arp
regi
ons
of the source im
age
s i
n
Figures1
(a) and (b
),
re
sp
ectively.This pape
r
u
s
e
s
t
he
NMF to
construct th
e t
e
mpo
r
ary fu
sed
image.
Figure1. Multi-focu
s ima
ge
fusion b
a
sed
on NMF
3. Multi-foc
u
s Image Fusion Bas
e
d on
NMF
3.1. Fusion Algorithm
In this se
ctio
n, a novel fusion al
gorith
m
using
NM
F is pro
p
o
s
e
d
. The pro
p
o
s
ed fu
sion
frame
w
ork i
s
depi
cted in Fi
gure
2.
0
I
is the
temporary fu
sed im
age of
A
I
and
B
I
. For the
sa
ke of
simpli
city, this p
ape
r a
s
su
mes th
at there ar
e o
n
ly two re
giste
r
ed
sou
r
ce ima
g
e
s
, nam
ely
A
I
and
B
I
, respe
c
tively. The
rational
e be
hind th
e
prop
osed
sch
e
me a
pplie
s t
o
the fu
sion
of more than
two multi-fo
cus imag
es. T
he so
urce im
age
s are a
ssumed to be
pre
-re
giste
r
e
d
and the im
age
regi
stratio
n
is not includ
ed
in the frame
w
ork. Th
e fusio
n
algorith
m
consi
s
ts of the
following th
ree
st
ep
s:
Figure 2. Block di
agram of
propo
se
d mu
lti-focu
s imag
e fusion fra
m
ewo
r
k
Step 1: Perform NMF o
n
the input matri
x
V
con
s
i
s
ting
of the source
image
s {
A
I
,
B
I
} to
con
s
tru
c
t the
temporary fu
sed i
m
ag
e
0
kl
I
¡
. The
sou
r
ce i
m
age
s {
A
I
,
B
I
},
,
kl
AB
II
¡
are
firs
t trans
f
ormed to
c
o
lumn vec
t
ors
A
V
and
B
V
, res
pec
tively.
A
V
and
B
V
are th
en
co
mbine
d
together to
repre
s
e
n
t the
input
matrix
V
. For two
g
r
ayscale im
ag
es
, the input matrix
V
is
defined a
s
:
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 12, No. 2, June 20
14: 379 – 38
8
382
[]
AB
VV
V
(2)
whe
r
e
2
K
V
¡
(
Kk
l
) is the input mat
r
ix for th
e
N
M
F
mo
de
l. In
pu
t ma
tr
ix
V
is
factori
z
ed
into the ba
si
s matrix
kl
r
W
¡
and
weig
ht matrix
2
r
H
¡
. The parameter
r
is
set to
1. Then,
the size of the basi
s
mat
r
i
x
W
is reset to the sa
me si
ze
as the source image
s, an
d is u
s
ed a
s
the temporary fused imag
e.
For mo
re tha
n
two source
image
s, the sou
r
ce imag
e
s
are tran
sfo
r
med to the
colum
n
v
e
ct
or
s
1
V
,
2
V
,…,
1
N
V
and
N
V
, and
combine
d
tog
e
ther to
re
prese
n
t the inp
u
t matrix
V
. T
he
input matrix
V
is defined as f
o
llows:
12
1
[]
(
2
)
NN
VV
V
V
V
N
(3)
whe
r
e
K
N
V
¡
(
Kk
l
) is
the input matrix for th
e N
M
F
mo
de
l. In
pu
t ma
tr
ix
V
is
fa
c
t
o
r
ized
into the bas
is matrix
kl
r
W
¡
and weig
ht matrix
rN
H
¡
. The param
eter
r
is
s
e
t to 1.
Step 2: Part
ition the te
mporary fused imag
e
0
I
into blocks by
usin
g the
regio
n
homog
eneity of
0
I
.
A
I
and
B
I
are split base
d
on
the split re
sul
t
s of
0
I
, res
p
ec
t
i
vely.
Step 3: Acco
rding
to th
e f
u
sio
n
rule
s, t
he
fo
cu
sed
region
s
of the
so
urce
imag
es
whi
c
h
corre
s
p
ondin
g
to the salie
nt region
s of
0
I
are integ
r
ate
d
to con
s
tru
c
t
the final fuse
d image.
3.2. Fusion Rules
There are two key i
s
sue
s
[20] involved
with t
he fu
sio
n
rule
s. Th
e first is
ho
w to
measure
the activity le
vel of the
so
urce ima
g
e
s
, whi
c
h
re
cog
n
ize
s
the
sha
r
pne
ss of th
e
sou
r
ce im
ag
es.
We
use the
e
nergy
of imag
e gradie
n
t (E
OG) to
me
a
s
ure th
e a
c
tivity level of the
sou
r
ce ima
g
e
s
.
The EOG of each image b
l
ock ca
n be d
e
fined a
s
:
22
()
(1
,
)
(
,
)
(,
1
)
(,
)
ij
ij
i
j
EO
G
I
I
II
i
j
I
i
j
II
i
j
I
i
j
(4)
whe
r
e
(,
)
I
ij
indicat
e
s the value
of the elemen
t location
(,
)
ij
in the image bl
o
ck.
The other i
s
how to integrate the focused pixe
ls or b
l
ocks of the source imag
es into the
cou
n
terp
art
s
of the fused i
m
age. Th
us,
the blo
c
k wit
h
a la
rge
r
E
OG i
s
cho
s
e
n
to con
s
tru
c
t the
fused image.
However,
the fixed block
size
will
l
ead to non-smooth tr
ansitions between blocks.
In ord
e
r to
redu
ce the
bl
ocking
artifa
cts, qu
ad tre
e
de
com
posi
t
ion [21] is
applie
d to bl
ock
division. Th
e division i
s
first perform
ed o
n
the low
re
solution ima
g
e
,
and then the
subdivi
sion i
s
perfo
rmed
on
the hi
gh
re
so
lution ima
ge
based
on th
e
division
of th
e lo
w
re
soluti
on ima
ge.
Qu
ad
tree de
comp
osition can a
daptively con
t
rol the bl
ock size of the subdivisi
on of the image ba
sed
on the regi
o
n
homog
enei
ty of
the block. Figu
re
3in
d
icate
s
the
decompo
sitio
n
of the image
“Len
a”. It i
s
obviou
s
that
the salient fe
ature
s
su
ch as edge
s an
d
texture
s
of
Figu
re3 (a
) are
corre
s
p
ondin
g
to the salie
nt feature of
Figure3 (b
). T
o
overcome t
he di
sadvant
age
s of the small
block in trad
itional block-based image
fusion
meth
od, the minimum block
size is set for
terminatin
g
the
furth
e
r di
vision whe
n
the
re
gion
h
o
moge
neity
of the blo
c
k
doe
sn’t me
et the
threshold
con
d
ition. The re
gion hom
oge
neity is define
d
as:
(,
)
(
,
)
|m
a
x
(
)
m
i
n
(
)
|
RR
ij
ij
BB
T
(5)
whe
r
e
(,
)
R
ij
B
is the
value of the
element lo
cation
(,
)
ij
in the image bl
ock.
T
is the threshol
d
con
d
ition. In
this p
ape
r, q
uad t
r
ee
de
compo
s
ition
is pe
rform
ed
o
n
the
tempo
r
ary fu
sed
im
age
and the thre
shold conditio
n
is set a
s
0.005, and the
minimum blo
ck
size is set as
88
.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A Novel M
u
lti-focus Im
age Fusion Method Ba
sed on Non-Negative .
... (Yongxin Z
hang)
383
Figure 3. Qua
d
tree de
com
positio
n of image ‘Le
na’
. (a
) sou
r
ce imag
e, (b) de
com
positio
n re
sul
t
Figure1in
d
ica
t
es the salie
nt feature
s
of tempora
r
y fuse
d image
0
I
agre
e
well
wi
th the
local f
eatures of the fo
cu
se
d obje
c
t
s
in t
he
sou
r
ce im
age
s. The
so
urce im
age
s
A
I
and
B
I
can
be
divided into b
l
ocks b
a
sed
on the split
result ofthe te
mporary fuse
d image
0
I
, respec
tively. Let
()
k
A
B
and
()
k
B
B
den
ote
the
k
th blo
c
ks
of the
so
urce
imag
es
A
I
and
B
I
, re
sp
ectively. Let
A
B
k
EO
G
and
B
B
k
EO
G
be the EOG of
()
k
A
B
an
d
()
k
B
B
, respec
tively.
A
B
k
EO
G
and
B
B
k
EO
G
are
compa
r
ed to
determi
ne which pixel
of the co
rresp
o
nding
bl
ock i
s
in focus. A
deci
s
ion
ma
trix
M
N
H
¡
is
con
s
tru
c
ted f
o
r re
co
rdin
g the com
p
a
r
iso
n
results a
c
cordin
g to the sele
ction rule
as follows:
1,
(,
)
0,
AB
B
B
kk
EOG
E
OG
Hi
j
oth
e
rwise
=
(6)
w
h
er
e“
1
”
in
H
indicates the
pixel lo
cation
(,
)
ij
in th
e
sou
r
ce
imag
e
A
I
is in f
o
cus,
while
“0”
in
H
indic
a
tes
the pixel loc
a
tion
(,
)
ij
in the source image
B
I
is in focus.
Ho
wever, ju
d
g
ing by EOG
alone i
s
not
sufficie
n
t to d
e
tect all the f
o
cu
se
d blo
cks. The
r
e
are thi
n
protrusi
on
s, na
rrow b
r
e
a
ks, thin gulf
s
an
d sm
all hol
e
s
in
H
. T
o
o
v
er
c
o
me
th
ese
disa
dvantag
e
s
, morp
holo
g
i
cal ope
ratio
n
s [22] are
perfo
rmed o
n
H
. Opening,
denoted a
s
H
Z
o
, is simply erosio
n of
H
by
the stru
ctu
r
e element
Z
, followe
d
by dilation of the result by
Z
. This pro
c
e
s
s ca
n remov
e
thin gulfs and thin prot
rusi
on
s. Clo
s
ing, denoted
as
H
Z
, is
dilation follo
wed by ero
s
io
n
.
It can join n
a
rrow b
r
e
a
ks
and thin g
u
lfs. To corre
c
tly judge the
sm
all
hole
s
, a thre
shol
d is set to remove
th
e hole
s
smal
ler tha
n
the
threshold.In t
h
is p
ape
r, th
e
stru
cture ele
m
ent
Z
of the propo
se
d meth
od is a
88
matrix
with logical 1
’
s and the thresh
old is
set to 1000. T
hus, the final
fused ima
ge i
s
co
nst
r
u
c
ted
accordi
ng to the rule a
s
follows:
(,
)
,
(,
)
1
(,
)
(,
)
,
(,
)
0
A
B
Ii
j
H
i
j
Fi
j
Ii
j
H
i
j
=
(7)
whe
r
e the
(,
)
A
I
ij
and
(,
)
B
I
ij
are the values of the pix
e
ls at the
(,
)
ij
in the so
urce im
age
s
A
I
and
B
I
, res
p
ec
tively.
4. Experimental Re
sults
In orde
r to ev
aluate the p
e
r
forma
n
ce of
t
he pro
p
o
s
ed
method, several exp
e
rim
e
nts are
perfo
rmed o
n
two pairs of
multi-focus i
m
age
s [23]
differing in co
ntent and texture, as sho
w
n
in
Figure 4. Th
e two pai
rs
are g
r
ay
scale image
s
with si
ze of
5
1
2
384
and
64
0
4
80
pixels,
respe
c
tively. In this pa
p
e
r, all
the
source
imag
e
s
a
r
e
a
s
sum
ed to
have
been
re
giste
r
ed.
Experiment
s
are
cond
ucte
d with Matlab
in Windo
ws environ
ment
on a co
mpute
r
with Intel
Xeon
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 12, No. 2, June 20
14: 379 – 38
8
384
X5570 an
d 48G memo
ry. For compa
r
ison, besid
e
the prop
osed method, som
e
existing multi
-
focu
s ima
ge
fusion
metho
d
s
are
al
so i
m
pleme
n
ted
on the
sa
me
set of
sou
r
ce
image
s. Th
e
s
e
method
s are spatial fre
que
ncy (SF) (Li’s method
[24]), NMF (Zha
n
g
’s metho
d
[7]), LNMF (Ye’
s
method
[10]),
SNMF
[17]
and
NMF
s
c [
18]. Du
e to
the la
ck
of o
r
i
g
inal
so
urce
cod
e
, this p
a
per
use
s
the Ed
u
a
rdo
Fernan
d
e
z
Can
ga’
sM
atlab imag
e
fusio
n
toolb
o
x [25] as the
referen
c
e fo
r
SF.
The NMF to
olbox [26] is use
d
a
s
th
e refe
ren
c
e f
o
r NMF, LNMF, SNMF
and
NMF
sc.
The
para
m
eters o
f
LNMF
a
r
e
set a
s
1
r
,
1.0
an
d
1.
0
; those of
SNMF
are
set as
1
r
,
0.01
, a
n
d
th
os
e
of N
M
Fsc
ar
e
s
e
t as
1
r
,
0.1
sW
.In order to q
uantit
atively com
p
are
the
perfo
rman
ce
of the prop
osed method
with that
of th
e method
s m
entione
d abo
ve, two metrics
are u
s
e
d
to evaluate th
e fusion
performan
ce:
(i
) Mutual info
rmation
(MI) [27,28], wh
ich
determi
ne
s th
e deg
re
e of d
epen
den
ce
of the source
i
m
age
s a
nd th
e fuse
d ima
g
e
, and
(ii)
/
AB
F
Q
[29], whi
c
h
m
easure
s
the
a
m
ount
of ed
g
e
info
rmat
ion
transfe
rred
from the
sou
r
ce ima
g
e
s
to t
h
e
fused ima
ge. In these metri
cs, a la
rge
r
value indi
cate
s a better fusio
n
result.
Figure 4.Multi-focu
s
sou
r
ce
images: (a) n
ear
focused i
m
age ’Rose’; (b) fa
r focu
se
d image
’Ro
s
e’; (c) far focused ima
ge ’Book’; (d) near fo
cused
image ’Boo
k’
4.1. Qualitati
v
e
Analy
s
is
For q
ualitative com
pari
s
o
n
,
the fused im
age
s ‘R
o
s
e’
a
nd ‘Boo
k’ of d
i
fferent metho
d
s a
r
e
sho
w
n i
n
Fig
u
re
s 5
(a-f
)
and 6
(a
-f), resp
ectively. The differen
c
e image
s b
e
t
ween th
e ri
ght
focu
sed
source i
m
age
‘Bo
o
k’
and
its
co
rre
sp
ondi
ng f
u
se
d ima
ge
o
b
tained
by dif
f
erent m
e
tho
d
s
are sho
w
n in
Figures 7 (a-f
).
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A Novel M
u
lti-focus Im
age Fusion Method Ba
sed on Non-Negative .
... (Yongxin Z
hang)
385
Figure 5. Fused image
s for ‘Ro
s
e’ obtai
ned by
differe
nt fusion mth
ods: (a
)SF; (b
)NM
F
;
(c)L
NMF; (d
)SNMF; (e
)NMF
sc; (f)the
prop
osed met
hod.
Figure 6. Fused image
s for ‘Book’ obtain
ed by di
fferen
t
fusion mtho
ds: (a
) SF; (b
) NMF; (
c
)
LNMF; (d) S
N
MF; (e
) NM
Fsc; (f) the propo
sed meth
od.
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 12, No. 2, June 20
14: 379 – 38
8
386
Figure 7. Differren
ce imag
esb
e
twe
en th
e far
focu
sed
sou
r
ce imag
e
‘Book’ and
correspon
ding
fused ima
g
e
s
obtaine
d by different fusion
mthods: (a
)
SF; (b) NMF;
(c) LNM
F
; (d) SNMF; (e)
NMF
sc; (f) th
e prop
osed m
e
thod.
The fu
se
d im
age
s o
b
taine
d
by th
e oth
e
r fusio
n
m
e
tho
d
s
dem
on
stra
te obviou
s
bl
ur,
su
ch
as th
e do
or f
r
ame in
Figu
re5 (a), the
up
per
edg
e of t
he rose an
d t
he ri
ght pla
n
t in the flo
w
e
r
pot
in Figu
re
s 5
(b-e
). The
co
n
t
rast of Fi
gure 5 (e) i
s
bett
e
r tha
n
that o
f
Figure
s
5
(a
-d). T
h
e
s
e bl
urs
also a
ppe
ar i
n
the fuse
d image in Fi
gures 6
(
a-e),
respectively, su
ch as the
cove
r of the left boo
k
in Figu
re
s 6
(a),
(c) an
d (e), the e
dge
betwe
en the
two bo
oks in
Figure 6
(b),
and the
upp
e
r
edge of the l
e
ft book in Fi
gure
s
6 (d).
The obviou
s
blockin
g
artifacts a
ppe
ar i
n
the fused i
m
age
obtaine
d by SF, such as t
he upp
er e
d
g
e
of the clo
ck in the left door fram
e in Fi
gure
5 (a
), a
nd
the cover of the left book i
n
Figures 6 (a). In
additio
n
, the blocki
n
g
artifacts al
so appea
r in the
differen
c
e im
age
s in
Figu
re 7
(a). T
h
e
r
e are
some
obviou
s
resi
d
uals i
n
the
di
fference ima
ges
obtaine
d by t
he exten
s
io
n
of the
NMF
-
b
a
se
d fu
sion
method
s i
n
Fi
gure
s
7
(b
-e),
su
ch
a
s
th
e
right
regio
n
s in Fi
g
u
re
s 7
(a
), (c) and
(d
), the
cente
r
r
egio
n
in Fig
u
re
7
(b). It sh
ould
b
e
note
d
that t
h
e
resi
dual
s i
n
Figure 7
(e) i
s
so
mu
ch th
at it c
an
se
e
the content
o
f
the ri
ght b
o
o
k’
s
cove
r. T
he
right regio
n
in
Figure 7
(f) i
s
smooth
and
flat.
Upon in
spe
c
ting th
e fuse
d imag
es
in Figu
re
s 5
(a-
f) an
d 6
(a-f),
it is ea
sy to
see
that th
e
cont
rast
of th
e fused
imag
e obtai
ned
b
y
the p
r
op
osed
method
is bet
ter tha
n
that
of the fu
sed
i
m
age
s o
b
tain
ed by th
e oth
e
r fu
sion
met
hod
s. Th
eref
ore,
the fused i
m
a
ges of the
propo
sed m
e
th
od achieve
superi
o
r visual
perfo
rman
ce
by containi
n
g
all
of the focuse
d conte
n
ts fro
m
the sou
r
ce image
s witho
u
t introdu
cing
artifacts.
4.2. Quantati
v
e
Analy
s
is
For q
uantitati
v
e analysi
s
, the qu
antitative re
sult
s of th
e tw
o quality measures are
sho
w
n
in Tabl
e 1. T
he runnin
g
ti
mes
are
also
sho
w
n i
n
Ta
b
l
e 1. The
pro
posed m
e
tho
d
gain
s
hi
ghe
r MI
and
/
AB
F
Q
values than the othe
r methods. Th
e MI and
/
AB
F
Q
va
lues of NM
F, LNMF a
nd SNM
F
are
almo
st th
e sa
me a
nd
h
i
gher than th
at of NM
F
s
c.
The runni
ng t
i
mes
of NMF
s
c is l
ong
er t
han
that of the other meth
od
s, which lies in
the com
puta
t
ional co
st of the s
parse
n
e
ss of the ba
si
s
matrix.It can
be seen that the pr
oposed method
requires longer
com
putational time than
the
other meth
od
s, except for
LNMF, SNM
F
and NMF
sc. The dra
w
ba
ck of hi
gh co
mputational
cost
lies i
n
that te
mporary fu
sio
n
of the
source imag
es a
c
counts for the
majority of th
e computatio
na
l
load.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A Novel M
u
lti-focus Im
age Fusion Method Ba
sed on Non-Negative .
... (Yongxin Z
hang)
387
Table 1.Perfo
r
man
c
e
s
of di
fferent fusion
method
s for
multi-focus i
m
age
s
Method
Rose Book
MI
/
AB
F
Q
Run-time(s)
MI
/
AB
F
Q
Run-time(s)
SF
6.78
0.71 0.66 8.41
0.70
1.04
NMF
5.40
0.70 0.96 7.65
0.62
1.51
LNMF
5.40
0.70 9.60 7.65
0.63
14.14
SNMF
5.40
0.70
10.12
7.65
0.62
15.91
NMFsc 5.50
0.68
21.86
7.53
0.47
32.97
Proposed
8.34 0.74
1.11
9.40 0.73
1.73
5. Conclusio
n
This pap
er propo
se
s a
nov
el multi-fo
cu
s image
fu
sion
method
with
NMF to
en
ha
nce
the
validity of focu
sed
re
gion
s extra
c
tion
and
blo
ckin
g
artifact
s in
h
i
bition. The
qualitative a
nd
quantitative e
v
aluation
s
ha
ve demon
stra
ted that
the p
r
opo
se
d meth
od can p
r
o
d
u
c
e b
e
tter fu
se
d
image an
d si
gnifica
ntly inhibit the bl
ocki
ng artifa
cts. In the future, we will
con
s
i
der o
p
timizin
g
the
prop
osed met
hod to red
u
ce
time con
s
um
ption and imp
r
ove the meth
od’s a
dapta
b
i
lity.
Ackn
o
w
l
e
dg
ements
The
wo
rk wa
s
sup
porte
d
by Natio
nal K
e
y Te
chnol
og
y Scien
c
e
an
d Te
ch
nique
Suppo
rt
Program
(No
.
2013BA
H
4
9
F03
)
, Key
Tech
nolo
g
ie
s R&
D Program of
He
na
n Provin
ce
(No.
1321
0221
051
5), the
Nation
al Natu
re S
c
i
ence Fou
nda
tion of Chi
na
(No. 6
137
901
0), the
Natural
Scien
c
e Ba
si
c Re
se
arch P
l
an in Shaanx
iP
rovince of Chin
a (No. 2012
JQ1
012
).
Referen
ces
[1]
HJ Z
hao, Z
W
Shan
g, YY T
a
ng, B F
ang. Multi-
focus im
ag
e fusion
base
d
on the ne
ig
hb
or distanc
e.
Pattern Reco
g
n
itio
n
. 201
3; 46(3): 100
2-1
0
1
1
.
[2]
ST
Li, XD K
a
n
g
, JW
Hu, B Y
ang. Ima
ge m
a
tting fo
r fus
i
on
of multi-foc
u
s i
m
ages
in
d
y
n
a
mic scen
e
s.
Information Fusion
. 20
13; 14(
2): 147-1
62.
[3]
H Hari
hara
n
. Exte
ndi
ng D
e
p
t
h of F
i
eld vi
a
Multi-focus F
u
sion.
Ph
D The
s
is
.
Kno
x
vil
l
e. Univers
i
t
y
of
T
ennessee. 20
11.
[4]
W
F
eng, W Bao. A Ne
w
T
e
c
hno
lo
g
y
of Remote
Sensin
g Image F
u
si
on.
TELKOMNIKA
T
e
leco
mmunic
a
tion, Co
mputi
ng, Electron
ics
and Co
ntrol
. 2
012; 10(
3): 551
-556.
[5]
DD Le
e, HS S
eun
g. Le
arni
ng
the parts of
o
b
jects b
y
n
on-
neg
ative
matri
x
factorizatio
n.
Nature
. 1
999
;
401(
675
5): 788
-791.
[6]
DD Le
e, HS Seun
g. Algor
ith
m
fo
r non-n
e
g
a
tive matri
x
fa
ctorizatio
n.
Adv
ances i
n
ne
ur
al infor
m
ation
process
i
ng sys
tems
. 20
01; 13
: 556-56
2.
[7]
JY Z
hang, L
W
e
i, QG Miao, Y W
ang.
Image fus
i
on
ba
sed o
n
no
n-n
egativ
e
matrix
factori
z
at
io
n
.
Procee
din
g
s of
IEEE Internationa
l Conf
erenc
e on Image Pr
ocessi
ng. 20
04
: 973-97
6.
[8]
L Xu, JY Dong,
CB Cai, Z
Chen.
Multi-focus i
m
a
ge fusi
ng b
a
sed o
n
no
n-n
egativ
e matrix factori
z
a
t
i
o
n
.
Proceedings of IEEE
International Confer
ence
on Mec
h
atronics and Machi
ne Vision
in Practice.
200
7: 108-
111.
[9]
SW
Z
hang, J Che
n
, DD Mia
o
.
An Imag
e F
u
sio
n
Metho
d
Based o
n
W
N
MF
and Reg
i
o
n
Seg
m
e
n
tatio
n
.
Procee
din
g
s
of IEEE Internatio
nal
Conf
erenc
e
on C
o
mputati
o
n
a
l Intelli
ge
nce a
n
d
Industria
l
Appl
icatio
n. 20
08: 282-
28
5.
[10]
YS Ye, BJ Z
hao, LB
T
ang.
SAR and
vi
sible
i
m
a
ge fu
sion
base
d
o
n
loca
l no
n-n
e
gative
matrix
factori
z
ation
. Procee
din
g
s of ICEMI. 2009: 2
63-2
66.
[11]
SP Liu, Q Ha
o, Y Song. D
y
n
a
mic W
e
i
g
h
t
ed N
on-
Ne
ga
tive Matrix F
a
ctorizatio
n an
d Its Using
Rese
arch in Im
age F
u
si
on.
Ch
ines
e Journ
a
l o
f
Sensors an
d Actuators
. 201
0; 23(9): 12
66-
127
1.
[12]
J W
ang, S
La
i, M Li. Impr
o
v
ed Ima
ge F
u
sion M
e
tho
d
Based
on
NS
CT
and Acce
l
e
rated
NMF
.
Sensors (Bas
el
).
2012; 12 (5):
5872-5
8
8
7
.
[13]
L Li, YJ Z
hang
. A surve
y
on
alg
o
rithms
of n
on-n
egativ
e m
a
tr
ix factor
izati
on.
Acta Electronic
a
sinic
a
.
200
8; 36(4): 73
7-74
7.
[14]
Y W
ang, Y Z
h
ang. No
n-ne
ga
tive matrix
fact
orizati
on: a compreh
ensiv
e revie
w
.
IEEE Transactions on
Kn
o
w
l
e
dg
e
and
D
a
ta
En
gi
nee
ri
ng
. 201
3; 25
(6): 1336-
13
53
.
[15]
F
A
Lu
w
i
nd
a, D
D
Sa
ntika.Ra
n
k
Matri
x
Optimi
zati
on
on
NMF
,
LNMF
a
n
d
ns
NMF
for F
eatur
e E
x
tractio
n
on Mammogram Classification.
Procedi
a En
gin
eeri
n
g
. 20
1
2
; 50: 606-
612.
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 12, No. 2, June 20
14: 379 – 38
8
388
[16]
SZ
Li, XW
H
ou, HJ Z
h
a
n
g
,
and et a
l
.
L
earn
i
ng s
pati
a
l
l
y loca
li
z
e
d
,
p
a
rts-base
d
rep
r
esentati
o
n
.
Procee
din
g
s o
f
IEEE Compu
t
er Societ
y
Co
nferenc
e on C
o
mputer Vis
i
o
n
and P
a
ttern
Reco
gniti
on.
200
1: 1-6.
[17]
W
Liu, N Z
h
e
ng, X
Lu.
N
o
n
-
neg
ative
matri
x
factori
z
at
io
n
for visual c
o
d
i
ng
. Proc
eed
in
gs of IEEE
Internatio
na
l C
onfere
n
ce o
n
Acoustics, Spee
ch and Si
gn
al. 200
3: 293-
296.
[18]
PO Ho
yer. N
o
n-ne
gative
mat
r
ix fa
ctorizati
o
n
w
i
th
sp
arsen
e
ss co
nstraint
s.
T
he Jo
urna
l
of Mac
h
in
e
Lear
nin
g
Res
e
arch
. 200
4; 5(9
)
: 1457-1
4
6
9
.
[19]
D Guillam
e
t. J Vitria.
Col
o
r his
t
ogra
m
class
i
fi
cation us
in
g NMF
.CVC.
T
e
ch. Report numb
e
r: 57. 200
1.
[20]
Y Jian
g, M W
ang. Imag
e fu
sion
w
i
th mor
p
hol
ogic
a
l c
o
mp
one
nt an
al
ysis.
Information F
u
sion
. 20
14;
18: 107-
11
8.
[21]
P Jag
a
d
eesh,
P N
aga
bh
us
han,
RP K
u
m
a
r. A N
o
vel
Image
Scramb
l
i
ng
T
e
chniq
u
e
Base
d On
Information
En
trop
y An
d Qu
ad T
r
ee Dec
o
mpositi
on.
Inte
rnatio
nal
Jour
nal
of Co
mput
er Scie
nc
e
Issues
. 201
3; 10: 285-2
94.
[22]
XZ
Bai, F
G
Z
hou, BD
Xu
e.
Image
enh
anc
em
ent usi
ng mu
lti-
scale im
ag
e fe
atures e
x
tracte
d b
y
top-
hat
transform.
Optics & Laser T
e
chno
logy
. 2
012;
44: 328-3
36.
[23]
Image sets: http://
w
w
w
.
ec
e.le
hig
h
.ed
u
/spcrl. 200
5.
[24]
S Li, JT
K
w
ok
, Y W
ang. Combin
ation of i
m
ages
w
i
t
h
di
verse focuses
usin
g the spati
a
l frequ
enc
y,
Information fusion
. 20
01; 2(3):
169-1
76.
[25]
Image fusio
n
tool
bo
x: http://
w
w
w
.
i
m
ag
efusi
o
n.org/.
[26]
NMF toolbox
:
http://cs
.
uw
inds
or.ca/~li11112c
/
doc/nmfv1_4.z
i
p.
[27]
DJC MacKa
y
.
Information theor
y, infer
e
n
c
e and l
earn
i
n
g
alg
o
rithms.
Ca
mbri
dg
e uni
versity press
.
200
3.
[28]
Y San
g
,Y
F
an. An I
m
prove
d
Image
Co
ntrast Assessment
Metho
d
.
TEL
K
OMNIKA
T
e
leco
mmunic
a
tion, Co
mputi
ng, Electron
ics
and Co
ntrol
. 2
013; 11(
2): 305
-312.
[29]
CS Xy
deas, V Petrovic. Object
ive imag
e fus
i
on
performa
nc
e meas
ure.
Electronics Letters
. 2000; 3
6
:
308-
309.
Evaluation Warning : The document was created with Spire.PDF for Python.