TELKOM
NIKA
, Vol.13, No
.3, Septembe
r 2015, pp. 9
76~984
ISSN: 1693-6
930,
accredited
A
by DIKTI, De
cree No: 58/DIK
T
I/Kep/2013
DOI
:
10.12928/TELKOMNIKA.v13i3.1805
976
Re
cei
v
ed Ma
rch 2
5
, 2015;
Re
vised June
14, 2015; Accepte
d
Ju
ne
30, 2015
RVM Classification of Hyperspectral Images Based on
Wavelet
Kernel Non-negative Matrix Fractorization
Lin Bai
1
, Def
a
Hu*
2
, Meng Hui
3
, Yanbo Li
4
1,3,
4
School of El
ectronics a
nd
Contro
l Engi
ne
erin
g, Chan
g'
A
n
Univ
ersit
y
,
Xi
’An 7
1
0
064,
Shaa
n
x
i, Chi
n
a
2
School of Co
mputer an
d Informatio
n
Engi
n
eeri
ng, Hun
an
Univers
i
t
y
of Commerce,
Cha
ngsh
a
41
0
205, Hu
na
n, Chin
a
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: hdf666
@1
63.
com
A
b
st
r
a
ct
A novel k
e
rne
l
framew
ork for hypers
pectr
al i
m
a
ge cl
as
sificatio
n
bas
e
d
on re
leva
nc
e vector
mac
h
i
ne (RVM
) is prese
n
ted
in this p
a
p
e
r. T
he new
featu
r
e extractio
n
a
l
gorit
hm
bas
ed
on Mexic
an
h
a
t
w
a
velet ker
nel
non-
ne
gative
matrix f
a
ctori
z
ation (W
KN
MF
) for hyp
e
rspe
ctral re
mote s
ensi
ng i
m
ag
es
is
prop
osed. By
usin
g th
e fe
ature
of
multi-
resol
u
tion
a
n
a
l
ysis, the
new
metho
d
of
n
onli
n
e
a
r
ma
pp
in
g
capa
bil
i
ty b
a
s
ed
on k
e
rn
el
NMF
can
be
i
m
pr
ove
d
. T
h
e
new
cl
assific
a
tion
fra
m
ew
o
r
k of hy
persp
e
c
tral
imag
e d
a
ta c
o
mb
in
ed w
i
th th
e n
o
vel
W
K
N
M
F
and
RVM.
T
he si
mu
lati
on
exp
e
ri
me
ntal r
e
sults
on
HYD
I
CE
and
AVIRIS da
ta sets are
b
o
th sh
ow
that th
e class
i
ficati
on
accuracy
of p
r
opos
ed
metho
d
co
mp
are
d
w
i
t
h
other ex
peri
m
e
n
t meth
ods
ev
en ca
n be
i
m
pr
oved
over 1
0
%
in so
me c
a
ses
and th
e class
i
ficatio
n
prec
isio
n
of smal
l sa
mp
l
e
data are
a
ca
n be i
m
pr
oved
effectively.
Ke
y
w
ords
:
hypers
pectral
classificati
on, non-
neg
ative matrix
f
a
ctori
z
ation, re
lev
a
n
c
e vector
ma
chin
e,
kernel method
Copy
right
©
2015 Un
ive
r
sita
s Ah
mad
Dah
l
an
. All rig
h
t
s r
ese
rved
.
1. Introduc
tion
It is well kn
ow th
at ea
ch mate
rial h
a
s it
s o
w
n
spe
c
ific ele
c
t
r
oma
gneti
c
radiation
spe
c
tru
m
ch
a
r
acte
ri
stic. Using hypersp
e
c
tral ima
g
e
r
y (HSI) sen
s
o
r
s, it is possib
l
e to recogni
ze
material
s a
n
d
their physi
cal
state
s
by
mea
s
ur
i
ng t
he
spe
c
trum
of the ele
c
tro
m
agneti
c
e
n
e
rgy
they reflect o
r
emit. The spectral data
whi
c
h
co
nsi
s
t
of hundre
d
s
of bands a
r
e
usu
a
lly acqui
red
by a rem
o
te
platform, such as
a satellite or an
ai
rcraft, and all bands are av
ail
a
ble at increasing
spatial
and
spectral resol
u
tions. After
30 ye
a
r
s
of
developm
ent, HSI technol
ogy ha
s n
o
t
only
been
widely
use
d
in milita
r
y, but also h
a
s be
en
su
ccessfully appli
ed in o
c
ea
n remote sen
s
in
g,
vegetation su
rveys,
geol
og
ical
ma
ppin
g
, environ
ment
al monito
ring
and oth
e
r
ci
vilian are
a
s [
1
,
2].
Due to
the
state of art
of sensor te
ch
nol
ogy develo
p
e
d
re
ce
ntly, an incre
a
si
ng
numbe
r
of spe
c
tral b
and
s have b
e
com
e
avail
able. Hu
ge
volumes
of remote sen
s
i
ng imag
es a
r
e
contin
uou
sly being a
c
q
u
ire
d
and a
r
chived. This tr
em
endo
us am
ou
nt of high sp
ectral
re
soluti
on
image
ry ha
s
dram
atically i
n
crea
se
d
the
informatio
n
source
and
in
crea
sed
the vo
lume of
imag
ery
s
t
ored [2, 3].
Ho
wever, th
e exce
ssive
HSI data increa
se
the dif
f
iculty of image p
r
o
c
essi
ng and
analysi
s
. Such as
sup
e
rvi
s
ed cl
assificati
on of HSI im
age
s is a ve
ry challen
g
ing
task
due to t
he
gene
rally unf
avorabl
e ratio
betwe
en the
large
num
ber
of spe
c
tral b
and
s an
d the
limited num
b
e
r
of training
sa
mples avail
a
ble a pri
o
ri, whi
c
h re
sult
s in the ‘Hugh
es ph
enom
e
non’. Witho
u
t the
sup
port
s
of n
e
w scie
ntific
con
c
e
p
ts an
d
novel te
ch
no
logical metho
d
s, the existin
g
larg
e volum
e
s
of data prohi
bit any syste
m
atic
exploit
a
tion. This h
a
s led to gre
a
t demand
s
to develop n
e
w
con
c
e
p
ts an
d
methods to d
eal
with large
data sets [2
-4].
Over the last years, ma
ny feature e
x
traction techniqu
es hav
e been inte
grated in
pro
c
e
ssi
ng
chain
s
inte
nd
ed fo
r
red
u
ce the
dime
n
s
ion
a
lity of t
he d
a
ta, thu
s
mitig
a
ting
the
Hug
h
e
s
ph
e
nomen
on. T
hese meth
o
d
s
can
be
unsupe
rvised o
r
supe
rvised.
Cla
s
sic
unsupe
rvise
d
techni
que
s inclu
de p
r
i
n
cip
a
l comp
onent a
naly
s
is
(PCA
), or ind
epe
nd
ent
comp
one
nt analysi
s
(ICA).
Supervise
d approa
che
s
comp
ri
se discrimi
nate an
alysis for fea
t
ure
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
RVM Cla
s
sification of Hyp
e
rspe
ctral Im
age
s Base
d
o
n
Wa
velet Ke
rnel Non
-
neg
ative…
(Lin B
a
i)
977
extraction
(DAFE), deci
s
io
n boun
da
ry feature
ex
traction (DBFE
)
, and no
n-para
m
etric
weig
hted
feature extra
c
tion (NWFE),
among m
any others [4-7].
Re
cently, it wa
s sho
w
n b
y
Lee an
d S
eung th
at
po
sitivity or non
-neg
ativity of a linea
r
expan
sion is
a very powe
r
ful const
r
aint
that also
se
e
m
s to yield sparse re
presentation
s
[8,
9].
Their te
ch
niq
ue, call
ed n
on-n
egative
matrix fa
ctori
z
ation
(NMF), was
sh
own
to be a u
s
eful
techni
que
in
approximatin
g hig
h
dim
e
n
s
ion
a
l d
a
ta
where
the
data
are
comp
rise
d of n
onn
ega
tive
comp
one
nts.
Ho
wever,
NMF and
man
y
of its va
ria
n
ts a
r
e e
s
se
ntially linear,
and thu
s
ca
n’t
disclo
se nonli
near
stru
ctures hid
den in t
he HSI dat
a. Beside
s, they can only de
a
l
with data wit
h
attribute valu
es, while in
many appli
c
a
t
ions
we do
not kn
ow th
e
detailed attri
bute value
s
and
only relation
ships a
r
e avai
lable. The NMF can
not
b
e
dire
ctly applied to su
ch
relation dat
a.
Furthe
rmo
r
e, one re
quirem
ent of NMF is that t
he values of data should be n
o
n
-
neg
ative, wh
ile
in many real
worl
d proble
m
s t
he no
n-n
egative co
nst
r
aints
can
not
be satisfie
d. Since the m
i
d-
1990
s, n
u
cl
e
a
r m
e
thod
h
a
s
bee
n
su
cce
ssfully
app
lied in
the
future, th
ere
a
r
e ma
ny sch
o
l
ars
have pro
p
o
s
e
d
Nonli
nea
r feature extract
i
on method b
a
se
d on kern
el method [10
-
13].
Suppo
rt vect
or ma
chi
ne
(SVM) h
a
ve been fo
und
to be pa
rti
c
ula
r
ly promi
s
ing fo
r
cla
ssifi
cation
of HSI data
becau
se of
their lo
we
r
sen
s
itivity to the cu
rse o
f
dimensi
ona
lity.
De
spite its widespre
ad su
ccess in
HSI
c
l
ass
i
fic
a
tion, the SVM
s
u
ffers
from
s
o
me import
ant
limitations, o
ne of the m
o
st si
gnifican
t
being
that
it mak
e
s
point predic
tions
rather than
gene
rating p
r
edi
ctive dist
ribution
s
. Rece
ntly
the Relevan
c
e
Vector Ma
chin
e (RVM
), a
prob
abili
stic
model who
s
e
functional f
o
rm is
e
quiv
a
lent to the SVM has be
en used in
HSI
cla
ssifi
cation.
RVM may require fe
wer training cases
than a SVM in orde
r to cla
ssify a data set.
It has been
sug
g
e
s
ted th
at the useful
training
cases for
cla
ssif
i
cation by a
RVM are a
n
ti-
boun
dary i
n
nature
while t
hose for u
s
e
in cla
s
sificati
on by a
SVM tend to li
e n
ear th
e b
oun
dary
betwe
en
cla
s
se
s. It achiev
es
comp
arabl
e re
cog
n
itio
n
accuracy to
the SVM, yet provide
s
a f
u
ll
predi
ctive dist
ribution, an
d al
so requi
re
s
sub
s
tantially fewe
r ke
rn
el functio
n
s [14].
The novel
method
whi
c
h propo
se
d i
n
this
pap
er
use
s
ke
rnel f
unctio
n
into t
he cl
assi
c
NMF an
d im
proved it by repla
c
e
d
trad
itional ke
rnel
function wit
h
Mexican
h
a
t wavelet kernel
function
(WK
NMF
)
. By the featur
e of m
u
lti-re
sol
u
tion
analysi
s
, the
nonline
a
r m
appin
g
ca
pab
ility
of WKNM
F
method
can
be imp
r
ove
d
. The
clas
sificatio
n
fra
m
ewo
r
k for
HSI image
data
combi
ned
wit
h
the n
o
vel
WKNM
F an
d
RVM. Th
e si
mulation
s results sho
w
tha
t, the method
of
WKNM
F refle
c
t the nonlin
e
a
r ch
aracte
ri
stics of the hyperspe
c
tral image.
The
pro
p
o
s
e
d
meth
od i
s
applie
d to
HY
DICE
data
an
d AVIRIS dat
a sets
co
mpa
r
ed
with
the othe
r
algo
rithms, th
e
cl
assificatio
n
a
c
cura
cy
ca
n be
in
crea
sed
even
ove
r
10
%
in some
ca
se
s
and the
cla
s
si
fication p
r
e
c
ision of small
sample d
a
ta a
r
ea can b
e
im
proved
effecti
v
ely. Section
2
pre
s
ent
s the
prop
osed fe
a
t
ure extractio
n
ba
se
d
o
n
W
K
N
M
F
a
nd R
V
M
c
l
as
s
i
fic
a
tion framwork
.
Experimental
results a
r
e re
ported in
se
ction 3.
Finally, con
c
lu
sion
s
are given in
section 4.
2. Methodol
og
y
2.1. Non-negativ
e
Matrix Factoriz
ation
NMF im
po
se
s the
non
-n
e
gativity con
s
traints in l
earning the
ba
si
s ima
g
e
s
. Both the
values of the
ba
sis imag
e
s
a
nd th
e
co
efficients for
recon
s
tru
c
tio
n
are all
no
n
-
neg
ative. Th
e
additive pro
p
e
rty ensure
s
that the co
mpone
nt
s a
r
e com
b
ined
to form a whole in the
non-
negative
way
,
which ha
s b
een
sho
w
n to
be the p
a
rt
based rep
r
e
s
entation of th
e origi
nal d
a
ta.
Ho
wever, the
additive part
s
learned by
NM
F a
r
e not
necessa
rily locali
zed [8, 9].
Given
the
non-n
egative
m
n
matrix V and the con
s
tant
r, the non-n
e
gative matrix
factori
z
ation
algorith
m
finds a non
-n
e
gative
r
n
matrix W and an
other no
n-n
e
g
a
tive
m
r
matrix H su
ch
that they minimize the follo
wing o
p
timality problem:
)
,
(
min
H
W
f
(1)
Subject to
0
,
0
H
W
This can
be interpreted as follows
: each colu
mn
of
matrix W
con
t
ains
a b
a
si
s vecto
r
while
ea
ch
column
of H contain
s
the
weights
ne
ede
d to a
pproximate the
co
rresp
ondi
ng
co
lum
n
in V usi
ng th
e ba
sis from
W. So the
produ
ct WH
ca
n be
reg
a
rde
d
as a
com
p
ressed fo
rm
o
f
the
data in V. The ran
k
r is u
s
ually ch
ose
n
)
,
min(
m
n
r
.
)
,
(
H
W
f
is a loss function. In this
pape
r, we set loss fun
c
tion
as follo
w:
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No. 3, September 20
15 : 976 – 984
978
n
i
m
j
ij
ij
WH
V
H
W
f
11
2
)
)
(
(
2
1
)
,
(
(2)
Solving the multiplicative iteration
rule fu
nction a
s
follo
ws:
bj
T
bj
T
bj
WH
W
V
W
H
)
(
)
(
ib
T
ib
T
ib
ib
WHH
VH
W
W
)
(
)
(
(3)
The
conve
r
g
ence of the
pro
c
e
s
s is
ensure
d
. The initialization is perform
ed using
positive random initial condit
ions
for matric
es
W and
H.
2.2. Kernel Non-Nega
tiv
e
Matrix Fa
cto
r
ization
Given m obj
ects
12
3
,
,
,
...
,
,
m
with attri
bute value
s
repre
s
e
n
ted a
s
an
n
by
m
matrix
12
[
,
,
...,
]
m
,each column of whi
c
h represent
one of the m object
s
. Define the
nonlin
ear ma
p from
o
r
igin
al inp
u
t spa
c
e
to a hi
ghe
r or infinite
dimensi
onal
fe
ature
sp
ace
as
follows
：
:(
)
xx
(4)
From the m o
b
ject
s, denot
e:
12
(
)
[
(
),
(
)
,
.
.
.
,
(
)]
m
(5)
Similar as
NMF, KNMF finds
two non
-negative matrix factors
W
and
H
s
u
c
h
that:
()
WH
(6)
W
is the
ba
se
s i
n
feature
spa
c
e
and
H
is its combining
coefficients, each column of
whi
c
h
den
ote
s
n
o
w the
di
mensi
o
n
-re
du
ced
re
presen
tation for the
co
rrespon
di
ng o
b
je
ct. It is
worth
noting
that sin
c
e
()
is unknown. It is
imprac
tical to direc
t
ly fac
t
oriz
e
()
. From
Equation (6), we obtai
n:
()
()
()
TT
WH
(7)
A kernel i
s
a function in th
e input spa
c
e
and
at the same time the inner produ
ct in the
feature
sp
ace thro
ugh th
e ke
rn
el-in
d
u
c
ed
nonli
nea
r
map
p
ing.
More
sp
ecifi
c
ally, a ke
rnel
is
defined a
s
:
(,
)
(
)
,
(
)
(
)
(
)
T
kx
y
x
y
x
y
(8)
From Equ
a
tio
n
(8), the left side of Equati
on (7
) ca
n be
rewritten as:
,1
,1
()
(
)
(
(
)
)
(
)
(
,
)
m
m
T
T
ij
i
j
ij
ij
kK
(9)
Den
o
te
()
T
YW
(10)
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
RVM Cla
s
sification of Hyp
e
rspe
ctral Im
age
s Base
d
o
n
Wa
velet Ke
rnel Non
-
neg
ative…
(Lin B
a
i)
979
From Equ
a
tio
n
(9) a
nd (1
0), Equation (7) can be
cha
n
ged a
s
:
KY
H
(11)
Comp
ari
ng
Equation
(11
)
with
Equat
ion (6),
it
can be
foun
d
that the
co
mbining
coeffici
ent
H
is the sa
me. Since
W
is
le
ar
n
ed b
a
s
es
o
f
()
, similarly we
call
Y
in Equation
(11
)
as the b
a
se
s of the kernel m
a
trix
K
. Equation (11) provide
s
a p
r
acti
cal way for obtaini
ng
the dimen
s
io
n-redu
ce
d re
pre
s
entatio
n
H
by performin
g NMF on
ke
rn
els.
For a ne
w dat
a point, the dimensi
o
n
-re
du
ced rep
r
e
s
ent
ation is comp
uted as follo
ws:
ne
w
n
e
w
HW
+
()
(
)
TT
ne
w
W
+
=
ne
w
YK
(12)
Her
e
A
dona
tes the gen
erali
z
ed
(Mo
o
re
-Pen
ro
se) inverse of
matrix
A
, a
nd
()
T
new
n
ew
K
is the
ke
rnel
matrix between t
he
m training i
n
sta
n
c
e a
nd the
new
instan
ce. Eq
uation (1
1)
and (1
2) co
nstru
c
t t
he key compo
n
e
n
ts of KNMF when u
s
e
d
for
cla
ssifi
cation,
it is easy to see that, the com
puting
of KNMF ne
ed not to kn
ow the attrib
ute
values of obj
ects, an
d onl
y the kernel
matrix
K
and
ne
w
K
are requi
re
d.
Obviou
sly, KNMF i
s
m
o
re
gene
ral th
an
NMF
be
cau
s
e the fo
rme
r
can
de
al with
not o
n
ly
attribute valu
e data b
u
t also relation
al
data. A
nothe
r advantag
e o
f
KNMF is th
at it is appli
c
able
to data
with
negative val
u
es
sin
c
e
the
kernel
matr
ix
in K
N
MF i
s
alway
s
n
on-n
egative fo
r
some
spe
c
if
ic ker
n
e
l
s.
2.3. Wav
e
let Kern
el Non
-
Nega
tiv
e
Ma
trix Factorization
The pu
rp
ose
of buildin
g ke
rnel fun
c
tion i
s
p
r
oje
c
t hyp
e
rspe
ctral
ob
serve
d
data f
r
om lo
w
dimen
s
ion
a
l spa
c
e to an
o
t
her high
dim
ensi
onal
spa
c
e. Thi
s
WK
NMF meth
od
use
s
the kernel
function
into t
he
NMF
and
i
m
prove
d
it by
re
pla
c
ed
the
traditional
kernel fun
c
tion
with the
wavele
t
kernel fu
nctio
n
. By the fea
t
ure of m
u
lti-reso
l
u
tion a
n
a
l
ysis, the
non
linear map
p
in
g ca
pability o
f
kernel no
n-ne
gative matrix factori
z
at
ion
method can b
e
improve
d
[15, 16].
A
ssumi
ng
()
hx
is a
wavelet
fu
nction,
pa
ra
meter
re
pre
s
ent st
retch a
n
d
represent
pan. If there
,'
N
x
xR
,
then we g
e
t dot produ
ct form of wavelet kernel fun
c
tio
n
:
1
''
(,
'
)
(
)
(
)
N
ii
i
i
i
xx
Kx
x
h
h
(13)
Meet the rea
s
on
able exp
r
ession p
r
od
u
c
t
approved
unde
r the co
ndition of tra
n
slatio
n
invarian
ce, th
e Equation (1
3) ca
n be rewritten as:
1
'
(,
'
)
(
)
N
ii
i
x
x
Kx
x
h
(14)
In this pa
pe
r Mexican h
a
t wavelet f
unctio
n
was sele
cted
as gene
rating
function,
according to
the theory of
transl
a
tion i
n
varian
ce
wa
velet function
, kern
el funct
i
on co
nst
r
u
c
ted
as:
2
2(
/
2
)
()
(
1
)
x
hx
x
e
(15)
From E
quatio
n (1
3),
(14
)
a
nd (15) a
wa
velet
ke
rnel
functio
n
me
ets the
re
quire
ments
of
Mercer
ke
rnel
function buil
d
as:
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No. 3, September 20
15 : 976 – 984
980
'2
'
2
22
1
()
()
(
,
'
)
[
1
]
e
xp[
]
2
N
ii
i
i
i
x
xx
x
Kx
x
aa
(16)
Use E
quati
on
(16
)
in
kernel
no
n-ne
gative
matrix
facto
r
ization
,
we
can
get
Wavel
e
t
kernel no
n-ne
gative matrix factori
z
ation.
2.4. Relev
a
n
ce Vecto
r
Machine Cla
s
s
i
er Introduc
tion
The RVM is
a possibili
stic counte
r
pa
rt to
the SVM, b
a
se
d on a Ba
yesian formul
ation of
a linear m
o
d
e
l with an a
ppro
p
ri
ate prior that
re
sul
t
s in a sp
arser
rep
r
e
s
en
tation than that
achi
eved by SVM. The ke
y advantage
s of the RVM
over the SVM inclu
de a redu
ced
sen
s
i
t
ivity
to the hyper-parameter
settings
, an ability to use non-Mercer
kernel
s, the provision of
a
prob
abili
stic
output, no
n
eed to
defin
e the
pa
ram
e
ter, a
nd
often a
requi
re
ment for fe
wer
relevan
c
e ve
ctors than
suppo
rt vecto
r
s for a
particular a
naly
s
is [15]. Usi
ng a Bern
o
u
lli
distrib
u
tion th
e likelih
ood fu
nction for the
analysi
s
woul
d be:
1
1
(
|
)
{
(
(
))}
[
1
{
(
(
)
)}
]
ii
n
yy
ii
i
py
g
y
x
y
x
(17)
Whe
r
e
g i
s
a
set
of adj
ust
able
weig
hts,
for m
u
lticla
ss
cla
ssifi
catio
n
(1
7) can
b
e
written
as:
11
(|
)
{
(
(
)
)
}
ij
q
n
y
ji
ij
py
g
y
x
(18)
1
((
)
)
1e
x
p
(
(
)
)
x
x
(19)
Duri
ng t
r
ainin
g
, the hyp
e
r-para
m
eter for a la
rge
nu
m
ber
of trai
nin
g
cases will
a
ttain very
large
value
and th
e a
ssociate
d
weig
hts
will b
e
redu
ced
to
zero. T
h
u
s
, th
e traini
ng
proce
s
s
applied to a
typical traini
ng set acquired follo
wing standard met
hods
w
ill m
a
ke most of t
h
e
training
case
s ’irrel
e
vant’
and l
eave o
n
l
y the u
s
ef
ul
trainin
g
ca
ses. A
s
a
re
sult only a
small
numbe
r of training cases
are requi
red
for final
cla
s
sification. The
assi
gnme
n
t of an individual
hyper-pa
r
am
eter to ea
ch
weig
ht is the
ultimate
rea
s
on for the spa
r
se
pro
p
e
r
ty of RVM. For
more
informatio
n a
bout RVM se
e referen
c
e [14]
，
[17-18].
3. Experiment Re
sults a
nd Analy
s
is
3.1. Experim
e
ntal on HY
DICE Da
ta S
e
t
The Fi
gure 1
sho
w
s a
sim
u
lated
colo
r IR view
of
an
ai
rbo
r
ne
HSI d
a
ta flight lin
e
over the
Wa
shin
gton
DC Mall
provided
with
the
permi
ssion
of Sp
e
c
tral
Inform
a
t
ion Te
ch
nol
ogy
Applicatio
n Center of Virgi
n
ia wh
o wa
s
respon
sibl
e for its colle
ctio
n. The se
nso
r
system
use
d
in
this ca
se
m
e
asu
r
ed
pixel respon
se
in 210 ban
ds
in
the 0.4
to 2.
4 µm
regi
on
of the visi
ble
and
infrared spe
c
trum. Band
s i
n
the 0.9 an
d
1.4 µm
regi
o
n
whe
r
e the
atmosp
he
re i
s
opa
que
ha
ve
been
omitted
from the
dat
a set, leaving
191
ban
ds.
The d
a
ta
set
contai
ns
120
8 sca
n
line
s
with
307 pixel
s
in
each scan li
n
e
. It totals approximately 1
50 Meg
abyte
s. The ima
ge
at left was m
ade
usin
g b
and
s
60, 27,
an
d 1
7
for the
re
d, green,
and
b
l
ue
colo
rs
respectively. Th
e HY
DICE
da
t
a
set incl
ude
Roofs, Street, Path (gravele
d paths d
o
wn
the mall cent
er), G
r
a
ss, Trees, Water, a
nd
Shado
w.
For ve
rificatio
n
the featu
r
e
extraction
alg
o
rith
m effe
ct to hypersp
ect
r
al data
cla
ssif
i
cation
appli
c
ation,
RVM cla
ssifie
r
use
d
in thi
s
p
aper.
Given a
set of trainin
g
exampl
es,
each ma
rked
a
s
belon
ging
to one of
two
ca
tegorie
s, an RVM
traini
ng
algo
rithm
bui
lds
a m
odel
that a
ssi
gn
s n
e
w
example
s
int
o
one
categ
o
r
y or the othe
r. An RVM
m
odel is
a rep
r
ese
n
tation of
the example
s
as
points in
spa
c
e, mapp
ed so that the example
s
of
the sepa
rate
cat
egori
e
s a
r
e di
vided by a cl
ear
gap that is a
s
wid
e
a
s
po
ssi
ble. Ne
w
exampl
e
s
are then map
p
ed into that same
sp
ace an
d
predi
cted to b
e
long to a category ba
se
d on whi
c
h
side
of the gap they fall on.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
9
30
RVM Cla
s
sification of Hyp
e
rspe
ctral Im
age
s Base
d
on Wavelet Kernel Non-negative…
(Lin B
a
i)
981
Figure 1. False colo
r imag
e
s
of HYDI
CE
Classificatio
n
experi
m
ent
s
on
hypersp
ectr
al data w
i
th
RVM,
PC
A+RVM, NM
F+
RVM
and KPCA
+
RVM (G
au
ss kernel, wi
dth co
efficient
is 0.5)
re
spectively, co
mpared
with
the
WKNM
F+RV
M method
propo
sed i
n
thi
s
pa
per. T
h
e
Overall A
c
cu
racy
(OA)
used a
s
evalu
a
tion
stand in th
e
experime
n
t results. Experime
n
t
ra
ndomly sele
ct 1%, 3% and 5% sa
mples
respe
c
tively as traini
ng da
ta sets on ori
g
inal
hypersp
ectral d
a
ta and other sam
p
les a
s
test data
sets.
T
he cla
ssifi
cation experim
ents we
re
repe
at
ed
10 time
s, taki
ng the
statisti
cal ave
r
ag
e for
final re
sult
s. The
RVM
kernel fu
ncti
ons ar
e used RBF
(Ra
d
ial
Ba
sis Functio
n
)
kernel
function, the
width coeffi
ci
ent of 0.5.
Experiment
with featu
r
e
extra
c
tion
algorith
m
, fe
ature
dime
nsions taken
b
e
fore
15
feature
com
p
onent
s a
s
inp
u
t, the ene
rg
y of the
total energy acco
u
n
ted for m
o
re
than 96%. T
he
cla
ssifi
cation
result wa
s
sho
w
n a
s
T
able 1,
Tabl
e 2 and T
a
ble 3. An impact of fe
ature
dimen
s
ion
a
lity to the RVM cla
ssi
er fo
r hype
rs
pe
ctral rem
o
te se
nsin
g imag
e
wa
s shown
as
Figure 2 (10
%
training sa
mple data
)
.
Table 1. Cla
s
sificatio
n
re
su
lts use 1% tra
i
ning sample
data
No.
Class name
Classification methods
RVM PCA+RVM
NMF+RVM
KPCA+RVM
WKNMF
+RVM
1
2
3
4
5
6
7
Roofs
Street
Path
Gras
s
Trees
Water
Shado
w
51.7%
90.3%
88.5%
87.2%
77.8%
91.5%
74.6%
53.8%
91.1%
88.9%
87.3%
78.1%
89.9%
75.3%
61.1%
91.6%
89.8%
85.7%
79.3%
91.9%
78.9%
60.6%
90.4%
88.5%
89.1%
81.4%
91.8%
77.5%
66.4%
91.1%
89.9%
87.8%
85.8%
92.8%
79.5%
（）
OA
71.8%
72.8%
74.1%
75.3%
81.9%
Table 2. Cla
s
sificatio
n
re
su
lts use 3% tra
i
ning sample
data
No.
Class name
Classification methods
RVM PCA+RVM
NMF+RVM
KPCA+RVM
WKNMF
+RVM
1
2
3
4
5
6
7
Roofs
Street
Path
Gras
s
Trees
Water
Shado
w
55.1%
91.5%
89.5%
87.5%
78.8%
91.2%
78.6%
56.7%
91.2%
88.9%
88.3%
80.1%
91.9%
79.3%
62.3%
92.3%
94.6%
89.7%
85.1%
93.3%
81.9%
62.2%
92.4%
95.5%
92.5%
86.4%
93.8%
81.3%
68.6%
93.1%
95.9%
94.8%
86.8%
95.8%
83.2%
（）
OA
74.8%
75.8%
77.8%
78.8%
82.3%
Table 3. Cla
s
sificatio
n
re
su
lts use 5% tra
i
ning sample
data
No.
Class name
Classification methods
RVM PCA+RVM
NMF+RVM
KPCA+RVM
WKNMF
+RVM
1 Roofs
61.1%
63.8%
66.5%
70.1%
76.7%
2 Street
97.5%
100%
93.4%
96.4%
95.5%
3 Path
99.5%
99.9%
100%
98.5%
99.9%
4 Grass
97.2%
97.3%
96.7%
100%
97.2%
5 Trees
97.8%
97.1%
98.3%
93.4%
93.4%
6 Water
100%
98.9%
96.9%
95.8%
96.8%
7 Shado
w
81.6%
78.3%
84.9%
87.5%
84.8%
（）
OA
77.8%
79.8%
81.1%
82.8%
88.2%
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No. 3, September 20
15 : 976 – 984
982
Figure 2. Cla
ssifi
cation OA
with re
spe
c
t to redu
ced di
mensi
onality in HYDICE
(10% traini
ng
sample d
a
ta)
3.2. Experimental on AVI
R
IS Data Set
The expe
rim
ents were
ca
rrie
d
out on
HSI im
age
s prod
uced by the AVIRIS. In orde
r to
simplify the l
o
gistics
of ma
rking
this exa
m
ple a
nal
ysi
s
available
to
others, only
a
small
po
rtion
of
data
set was
cho
s
e
n
for thi
s
expe
rime
nt. It contain
s
1
45 line
s
by 1
45 pixel
s
(21
025 pixel
s
) a
n
d
190 sp
ect
r
al band
s sel
e
ct
ed from a Ju
ne 1992 AVIRI
S data set of a mixed agriculture/fore
stry
land
scape in
the Indian Pin
e
Test Site in North
w
e
s
te
rn
Indiana.
We sele
ct corn
-min,
corn
-notil, soybea
n-min,
soybe
an-n
o
til an
d
woo
d
s f
r
om
AVIRIS
image
s for cl
assificatio
n
e
x
perime
n
t. The 3-
band
s (20, 80, 140
band
) false
colo
r synth
e
sis
image u
s
ed i
n
experim
ent and the grou
nd
truth are shown in Figu
re 3.
Figure 3. False colo
r imag
e
s
and g
r
ou
nd
truth of AVIRIS
Cla
ssifi
cation
expe
riment
s on
hype
rsp
e
ct
ral
d
a
ta with RVM,
P
C
A+
RVM, N
M
F+
RVM
and KPCA
+
RVM (G
au
ss kernel, wi
dth co
efficient
is 0.5)
re
spectively, co
mpared
with
the
KNMF
+RVM
method p
r
op
ose
d
in this
pape
r. The
Overall A
c
cu
racy
(OA)
used a
s
evalua
tion
stand in th
e
experime
n
t results. Expe
riment
rand
o
m
ly sele
ct 0.5%, 2% and 5% sam
p
l
e
s
respe
c
tively as traini
ng da
ta sets on ori
g
inal
hypersp
ectral d
a
ta and other sam
p
les a
s
test data
sets.
The
cl
a
ssifi
cation
ex
perim
ents we
re
rep
eated
1
0
time
s a
s
HYDICE exp
e
ri
ment, taki
ng t
h
e
statistical av
erag
e for fin
a
l re
sults. T
he
RVM
kern
el functio
n
s
are u
s
e
d
RB
F
(Ra
d
ial Ba
sis
Functio
n
)
kernel functio
n
, t
he width
coef
ficient of 0.5.
Experiment
with featu
r
e
extra
c
tion
algorith
m
, fe
ature
dime
nsions taken
b
e
fore
20
feature
com
p
onent
s a
s
inp
u
t, the ene
rg
y of the
total energy acco
u
n
ted for m
o
re
than 97%. T
he
cla
ssifi
cation
result wa
s sh
own a
s
Tabl
e
4, Table 5 an
d Table 6.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
RVM Cla
s
sification of Hyp
e
rspe
ctral Im
age
s Base
d
o
n
Wa
velet Ke
rnel Non
-
neg
ative…
(Lin B
a
i)
983
Table 4. Cla
s
sificatio
n
re
su
lts use 0.5% t
r
ainin
g
sa
mpl
e
data
No.
Class name
Classification methods
RVM
PCA+RVM
NMF+RVM
KPCA+RVM WKNMF
+RVM
1 corn-min
73.3%
74.3%
75.1%
76.6%
78.4%
2 corn-notil
70.9%
71.7%
72.4%
75.4%
81.1%
3 soybean-
min
77.6%
78.3%
80.6%
83.5%
89.9%
4 soybean-
notil
52.5%
54.3%
56.0%
59.1%
63.8%
5 Woods
85.6%
87.1%
89.6%
89.4%
90.8%
（）
OA
68.6%
70.0%
71.5%
73.3%
76.7%
Table 5. Cla
s
sificatio
n
re
su
lts use 2% tra
i
ning sample
data
No.
Class name
Classification methods
RVM
PCA+RVM
NMF+RVM
KPCA+RVM WKNMF
+RVM
1 corn-min
77.4%
77.6%
77.7%
80.6%
83.4%
2 corn-notil
76.3%
75.5%
77.8%
79.4%
82.1%
3 soybean-
min
83.5%
83.7%
83.8%
88.5%
90.1%
4 soybean-
notil
57.9%
58.8%
60.7%
65.1%
67.8%
5 Woods
91.1%
91.2%
92.3%
93.4%
95.8%
（）
OA
73.2%
74.2%
75.1%
77.3%
80.5%
Table 6. Cla
s
sificatio
n
re
su
lts use 5% tra
i
ning sample
data
No.
Class name
Classification methods
RVM
PCA+RVM
NMF+RVM
KPCA+RVM WKNMF
+RVM
1 corn-min
77.9%
79.8%
80.1%
82.6%
86.1%
2 corn-notil
77.3%
78.1%
79.9%
81.4%
85.6%
3 soybean-
min
84.5%
85.9%
87.2%
89.5%
93.9%
4 soybean-
notil
58.2%
60.3%
62.4%
69.1%
73.8%
5 Woods
92.8%
93.4%
94.0%
94.4%
95.8%
（）
OA
74.6%
75.9%
77.4%
79.3%
84.8%
From
the
cl
a
ssifi
cation
ex
perim
ental
re
sults,
it
can
be
see
n
that
the a
ppli
c
ati
on of th
e
prop
osed me
thod
is better than the
other
algo
rithm
s
,
and the perf
o
rma
n
ce
of wavelet ke
rn
el
function i
s
superi
o
r
t
o
the traditional
kern
el
fun
c
tion. The
cl
assificatio
n
accuracy u
s
in
g
RVM cla
s
sifie
r
can
ach
i
eve
high
er with
fe
wer
sa
m
p
les,
hype
rsp
e
ctral i
m
age
cla
ssifi
cation
probl
em
s
so i
t
is suitabl
e for small sam
p
le,
high dim
ensi
on an
d l
a
rge
amou
nt of
data.
4. Conclusio
n
A novel kern
el frame
w
o
r
k for hypersp
ectr
al im
age
classificatio
n
based on
RVM is
pre
s
ente
d
in
this
pap
er.
This WK
NM
F metho
d
u
s
es th
e
ke
rne
l
functio
n
int
o
the
NMF
and
improve
d
it by Mexican ha
t wavelet ke
rnel functi
o
n
. By the feature of multi-re
solution an
alysis,
the nonline
a
r mapping
ca
pability of WKNMF metho
d
can b
e
improved. Be
ca
use of RVM
ha
s
good
gen
eral
ization
ability,
difficult affected by t
he
classifier p
a
ra
meters sele
ction and i
n
the
choi
ce
of reg
u
lari
zation
co
efficient app
ropriate,
RV
M has app
rox
i
m
a
t
e
cla
s
sif
i
cat
i
on
a
ccu
ra
cy
as
SVM. So we combi
ne WK
NMF an
d RV
M as ne
w cla
ssifi
cation fra
m
ewo
r
k for HSI data.
The
expe
rime
nt on HY
DICE and AVIRIS data
set
s
sho
w
that
the
WKNM
F me
thod a
s
feature
extra
c
tion
ha
s mo
re ability
than
the co
mp
ared
alg
o
rith
ms,
and
the
pe
rform
ance
of
wavelet
ke
rnel
functio
n
ha
s
b
e
tter
perfo
rm
ance tha
n
gene
ral
kernel
fun
c
tion.
The
final
processed d
a
ta i
s
appli
ed to
HSI im
age classificatio
n
b
a
se
d
on RVM
cla
ssifie
r
.
In
some
c
a
s
e
s,
t
he
cla
ssif
i
cat
i
on
a
c
c
u
ra
cy
ca
n
be in
c
r
e
a
se
d ov
e
r
10%
a
n
d
t
h
e
cla
ssifi
cation
pre
c
isi
on ca
n effectively
improv
e in
small sampl
e
area. Exp
e
rime
nt resu
lts
proved the eff
e
ctivene
ss of t
he cla
ssif
i
cat
i
on f
r
ame
w
o
r
k.
Ackn
o
w
l
e
dg
ements
The work wa
s suppo
rted
by national n
a
tu
ral
sci
en
ce foundatio
n
(No.4
110
13
57) a
nd
Nation
al Natu
ral Scie
nce Found
ation of Chin
a (No. 61202
464
).
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 13, No. 3, September 20
15 : 976 – 984
984
Referen
ces
[1]
J Muñoz-Marí,
D T
u
ia, G Camps-Valls. Se
misup
e
rvise
d
c
l
assificati
on
of remote se
nsin
g imag
es
w
i
t
h
active qu
eries.
IEEE Trans.on Geosci.
Rem
o
t
e
Sensing.
20
1
2
; 50(10): 3
751
-376
3.
[2]
Biouc
as-Di
a
s JM, Plaza A, Camps-Va
lls,
G Scheund
ers P, Nasra
bad
i NM, Ch
anuss
o
t J.
H
y
pers
pectral
Remote S
ensi
ng Dat
a
Ana
l
ysis and F
u
tur
e
Chal
le
nges.
IE
EE Geoscie
n
c
e
an
d Re
mot
e
Sensi
ng Ma
ga
z
i
n
e
.
20
13; 1(2
)
: 6-36.
[3]
Yushi C
hen,
Xi
ng Z
h
a
o
, Z
hou
ha
n Lin.
Optimizin
g
Su
bspac
e SVM Ensemb
le for
H
y
p
e
rsp
e
ctra
l
Imager
y Cl
assi
fication.
IEEE Journ
a
l of Sel
e
cted Topics i
n
A
ppl
ied E
a
rth Observatio
n
s
and Re
mot
e
Sensi
ng.
201
4
;
7(4): 1295-1
3
05.
[4]
Dopi
do I, Vill
a A, Plaza A, Ga
mba P. A Quantitative a
nd C
o
mpar
ative As
sessment of U
n
mi
xin
g
-Bas
ed
F
eature E
x
trac
tion T
e
chniqu
e
s
for H
y
pers
p
e
c
tral Image Cla
ssificatio
n
.
IEEE Journal of selected topics
in ap
pli
ed e
a
rth observ
a
tio
n
s and re
mote se
nsin
g
. 201
2; 5(2): 421-4
35.
[5]
YL He, DZ
Liu, SH Yi. A
ne
w
b
and se
l
e
ction meth
od
forh
y
p
ersp
ectral imag
es.
Opto-Electronic
Engi
neer
in
g
. 2010; 37(
9): 122
-126.
[6]
F
Liu, JY Gong. A classificati
on meth
od for
hig
h
spati
a
l re
soluti
on rem
o
tel
y
sens
ed im
age
base
d
o
n
multi-featur
e.
Geogra
phy a
n
d
Geo-Infor
m
at
ion Sci
ence
. 2
009; 25(
3): 19-
41.
[7]
H Su, H
Ya
ng
, Q Du, YH S
hen
g. Semis
u
pervis
ed
ba
nd
clusteri
ng
for
dime
nsi
ona
lit
y red
u
ction
of
h
y
pers
pectral i
m
ager
y.
IEEE Geoscience and Re
m
o
te Sensing Letters
. 2011; 8(6): 1
135
-113
9.
[8]
Sen Ji
a, Yunt
ao Qia
n
. Co
n
s
traine
d No
nn
egativ
e Ma
tri
x
F
a
ctorizatio
n
for H
y
pers
pect
r
al U
n
mi
xi
ng.
IEEE Transactions on Geosci
ence and Remote Sensing.
2
009; 47(
1): 161
-173.
[9]
Xu
eso
ng Li
u, W
e
i Xi
a, Bin W
ang, Limin
g
Z
hang. An ap
proac
h base
d
on constra
i
ne
d
nonn
egativ
e
matrix factor
iz
ation to
unmi
x
h
y
pers
pectral
data.
IEEE Transacti
ons o
n
Geoscie
n
ce
and
Re
mot
e
Sensi
ng.
20
11;
49(2): 757-
77
2.
[10]
Song Xia
ngfa,
Jiao Lich
en
g.
Classificti
on o
f
h
y
p
e
rsp
e
ctral
remote se
nsi
ng im
age
bas
ed o
n
spars
e
repres
entati
on
and
spectra
l
i
n
formatio
n
.
Jour
nal
of El
ectroni
cs & Infor
m
ati
o
n T
e
ch
nol
ogy.
201
2;
34(
2)
:
268-
272.
[11]
F
auvel M, Ch
anuss
o
t J, Bened
iktsson JA
. A
Spatial s
pectral Ker
n
e
l
based A
ppro
a
ch for the
Classific
a
tio
n
o
f
Remote sensi
ng Images.
Pat
t
ern Reco
gniti
o
n
. 2012; 4
5
(1): 381-
392.
[12]
Che
n
Yi,
Nass
er M, T
r
ac DT
. H
y
p
e
rspectra
l
Image
Cl
assifi
cation
via
Ker
nel
Spars
e
R
e
prese
n
tatio
n
.
IEEE Transactions on Geoscienc
e and Remote Sensing.
2
013; 51(
1): 217
-231.
[13]
W
ang J. J
a
b
b
a
r MA. Mu
ltipl
e
Ker
nel
L
ear
nin
g
for
ad
apti
v
e gr
aph
re
gu
larize
d
no
nne
g
a
tive m
a
tri
x
factorizatio
n.
Nation
al ass
o
cia
t
ion of scienc
e
and tech
no
logy
for develo
p
m
e
n
t.
2012; 41(
3): 115-1
22.
[14]
Mahes
h p
a
l, G
iles M
F
ood
y.
Evalu
a
tion
of
SVM, RVM an
d SMLR
for A
ccurate Ima
g
e
Class
ificatio
n
w
i
t
h
Limited Ground data.
IEEE Jour
nal
of
selected topics
in
applied ear
th
obs
erva
tions and rem
o
t
e
sensi
ng.
20
12; 5(5): 134
4-1
3
5
5
.
[15]
Yunj
ie Xu,
Sh
udo
ng Xi
u
.
Di
agn
ostic Stud
y Based o
n
W
a
vel
e
t Packet Entrop
y an
d
W
ear Loss of
Supp
ort Vecto
r
Machi
ne.
T
E
LKOMNIKA T
e
leco
mmunic
a
tion
Co
m
puti
ng El
ectron
ics
and
Co
ntrol.
201
4; 12(4): 84
7-85
4.
[16]
W
e
i F
e
n
g
, W
e
n
x
i
ng B
ao.
A
New
T
e
ch
no
l
ogy
of R
e
mot
e
Se
nsin
g I
m
age
F
u
sio
n
. T
E
LKOMNIKA
T
e
leco
mmunic
a
tion C
o
mputi
n
g Electron
ics a
nd Co
ntrol.
20
12; 10(3): 5
51-
556.
[17]
F
A
Mianj
i, Y Z
han
g. Ro
bust
h
y
pers
pectral
classificati
on
u
s
ing r
e
l
e
vanc
e
vector mac
h
in
e.
IEEE Trans.
Geosci. Re
mot
e
Sens.
20
11; 49(6): 21
00-
21
12.
[18]
Z
hao C
h
u
nhu
i, Qi Bin, Z
h
a
n
g
Yi. H
y
pers
pec
tral
ima
ge c
l
as
sificatio
n
b
a
se
d on
vari
ation
a
l re
leva
n
c
e
vector machine.
ACTA Optica
Sinica.
201
2; 32(8): 256-
26
1.
Evaluation Warning : The document was created with Spire.PDF for Python.