TELKOM
NIKA
, Vol.14, No
.1, March 2
0
1
6
, pp. 245~2
5
3
ISSN: 1693-6
930,
accredited
A
by DIKTI, De
cree No: 58/DIK
T
I/Kep/2013
DOI
:
10.12928/TELKOMNIKA.v14i1.2812
245
Re
cei
v
ed O
c
t
ober 7, 20
15;
Revi
se
d De
cem
ber 18, 20
15; Accepted
Jan
uary 7, 20
16
Hybridizing PSO with SA for Optimizing SVR Applied to
Software Effort Estimation
Dinda Nov
i
ta
sari, Imam Cholissodin, Wa
y
a
n Firdaus Mahmud
y
Dept. of Informatics/ Compute
r
Science, Bra
w
i
j
a
y
a
Un
iversi
t
y
, Mala
ng 65
1
4
5
e-mail: id.
d
in
da
novitas
ari@
gm
ail.com,
imamc
s
@ub.ac.i
d, w
a
yanfm@
ub.ac
.id
A
b
st
r
a
ct
T
h
is study
in
vestigates
Pa
rticle Sw
ar
m
Opti
mi
z
a
t
i
o
n
(
PSO) hybrid
i
z
ation w
i
th S
i
mu
late
d
Anne
ali
ng (SA)
to opti
m
i
z
e
S
u
pport V
e
ctor M
a
chi
ne (S
V
R
). T
he opti
m
i
z
e
d
SVR is
used
fo
r softw
are effort
estimatio
n
. T
he opti
m
i
z
a
t
i
on
of SVR co
nsist
s
of tw
o
sub-pr
obl
e
m
s that
must be s
o
lve
d
s
i
multa
neo
usly;
the
first is inp
u
t fe
ature se
lecti
o
n
that infl
uenc
e
s
me
t
hod
accu
racy an
d co
mp
uting ti
me. T
h
e
next su
b-pro
b
l
e
m
is find
in
g o
p
ti
mal SVR
p
a
ra
meter that
eac
h
para
m
eter
g
i
ve
s sign
ifica
n
t i
m
pact to
metho
d
perfor
m
ance.
T
o
dea
l w
i
th a hu
ge nu
mber of
cand
idat
e solu
tions of
the pr
obl
e
m
s, a pow
erful ap
pro
a
ch
is requir
ed. T
h
e
prop
osed
ap
pr
oach tak
e
s a
d
v
antag
es of g
o
od so
lu
ti
on q
u
a
lity fro
m
PSO
and
SA. W
e
i
n
troduc
e SA b
a
sed
accepta
n
ce ru
l
e
to accept n
e
w
position i
n
P
S
O. T
he SA para
m
eter se
lec
t
ion is i
n
troduc
ed to i
m
pr
ove the
qua
lity as st
oc
hastic a
l
g
o
rith
m
is se
nsitive
to its
par
a
m
et
er. T
he co
mpa
r
ative w
o
rks h
a
ve b
e
e
n
b
e
tw
een
PSO in
qu
ality
of so
luti
on
an
d co
mputi
ng ti
me.
Accord
in
g
to the
resu
lts, the
pro
pose
d
mode
l o
u
tperfor
m
s
PSO SVR in quality of solution.
Ke
y
w
ords
:
Particle sw
arm optimi
z
a
t
i
on, simulat
ed an
ne
alin
g,
sup
port vector regress
i
o
n
, feature sel
e
ction,
para
m
eter opti
m
i
z
at
io
n
Copy
right
©
2016 Un
ive
r
sita
s Ah
mad
Dah
l
an
. All rig
h
t
s r
ese
rved
.
1. Introduc
tion
The m
o
st
im
portant
pa
rt o
f
softwa
r
e
p
r
oject i
s
software
effort
esti
mation. It det
ermin
e
s
how
many
re
sou
r
ces that
proje
c
t n
eede
d and
mu
st b
e
don
e a
c
curately. If we h
a
ve big
error
rate
in estimation,
it will lead into big loss su
ch
a
s
unp
redi
ctable d
e
lay time and un
expecte
d bud
ge
t.
To preve
n
t many losse
s
in the future
, some
ap
proache
s are
d
e
velope
d to estimate
software
effort. One of
them is m
a
chine lea
r
ni
ng.
Suppo
rt vect
or ma
chi
ne is machi
ne le
a
r
ning
algo
rith
m
introdu
ce
d b
y
Vapnik to solve cla
s
sification p
r
obl
e
m
. Due to so
lve real wo
rl
d probl
ems,
SVM
wa
s develop
ed to solve regre
s
sion a
n
d
time se
rie
s
predi
ction called SVM base
d
reg
r
e
s
sion
(SVR). In ord
e
r to solve n
online
a
r regres
sion p
r
obl
e
m
, SVR mapped data to h
i
gh dimen
s
io
nal
feature
sp
ace u
s
ing
ke
rn
el fun
c
tion. T
h
is
ke
rnel
m
u
st
satisfy M
e
rcer conditi
on [1] an
d o
ne of
kernel
s is rad
i
al basi
s
fun
c
tion (RBF
).
In ma
chin
e l
earni
ng, feat
ure
sele
ction
intro
d
u
c
ed
a
s
a
p
r
o
c
e
s
s
of sel
e
ctin
g
a sub
s
et
feature fo
r u
s
e in m
odel
con
s
tru
c
tion.
This p
r
o
c
e
ss is nee
ded f
o
r SVR
sin
c
e it can
sim
p
lify
comp
uting p
r
ocess and
red
u
ci
ng computing
ti
me, espe
cia
lly whe
n
co
mputing i
n
high
dimen
s
ion
a
l spa
c
e. Besi
d
e
s that, prop
er pa
ramete
r setting
s ca
n influen
ce SVR accu
ra
cy. SVR-
RBF ha
s pa
rameters influ
enced its pe
rforman
c
e i.
e.
erro
r pe
nalt
y
, insensitive
loss, an
d ra
dial
basi
s
[2]. Th
o
s
e m
ention
e
d
above
are
crucial i
n
SVR-RBF b
e
ca
use
feature
sele
ction influen
ce
s
SVR pa
ramet
e
r a
nd vi
ce v
e
rsa [3]. In th
e pa
st research, Olivie
ra i
n
vestigated
th
e u
s
e of SV
R in
orde
r to
do
software
effort
estimatio
n
[
4
]. It gives p
r
omi
s
ing
re
sult but
cann
o
t
guarantee
give
good result since u
s
in
g predefine
d
nu
mber of f
eatu
r
es a
nd pa
ra
meter mea
n
s cann
ot disco
v
er
other
option
s
that can le
a
d
into hi
ghe
r
accuracy
rate
.
Nume
ro
us can
d
idate of solutio
n
can be
gene
rated
in
order to
ha
ve great n
u
m
ber of
su
b
s
et featu
r
e
combinatio
n a
nd va
ry ra
ng
e of
para
m
eter, if
we u
s
e en
u
m
eratio
n. Ho
wever, it
doe
s not utilize a fitne
ss fu
n
c
tion, and i
s
thus
ungui
ded, often failing to find goo
d sol
u
tion. Due to
the compl
e
xity of the prob
lem, a powerful
approa
ch is
required to get
a good soluti
on.
Some sto
c
ha
stic optimi
z
ati
on metho
d
s
become
alte
rnative to sele
ct su
bset feat
ure a
n
d
optimize
pa
ra
meter. It gen
erate
s
can
d
id
ate sol
u
tion
s, involves
obje
c
tive functio
n
to evaluate t
h
e
quality
of solu
tion
so solutio
n
searchi
ng could be
l
ead
i
n
to a
goo
d
so
lution. Bra
ga
et al p
r
op
ose
d
geneti
c
alg
o
ri
thm (GA) to o
p
timize SV
R i
n
software effort e
s
timation
[5]. Our p
r
e
v
ious
resea
r
ch
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 1, March 2
016 : 245 – 2
5
3
246
prop
osed
particle
swa
r
m
o
p
timization
(P
SO) to
optimi
z
e SV
R in th
e same
pro
b
l
e
m do
main [
6
].
Basically, PSO is in
spi
r
ed by flocki
ng bird moti
on empl
oyed
parall
e
l se
arch techniq
ues,
exploitation
and explo
r
ati
on. Ho
weve
r, PSO
has
disa
dvantag
e
,
trapped in
local mini
m
u
m
becau
se pa
rticle
s move in high veloci
ty and
gain prem
ature
co
nverge
nce [7]. On the oth
e
r
hand,
simulat
ed an
nealin
g
inspi
r
ed
by pro
c
e
ss
of a
nneali
ng in
metallurgy, is good in fin
d
i
ng
local o
p
timu
m [8]. Theref
ore, this
stu
d
y invest
igat
es hyb
r
idi
z
ati
on PSO with
SA in orde
r to
enha
nce se
a
r
chi
ng
cap
a
ci
ty. This prop
ose
d
mo
d
e
l is u
s
ed to o
p
t
imize SVR
para
m
eter
a
nd
sele
ct su
bset feature ap
plie
d to softwa
r
e
effort estimati
on.
Several rese
arche
s
inve
stigated SVR optimizatio
n
have bee
n
done
and
gaine
d
promi
s
in
g re
sult. Braga et.al [5] investigated GA
appl
ication to sel
e
ct su
bset feature an
d SVR
para
m
eter a
pplied to sof
t
ware effo
rt estimation. T
hey use
d
binary co
ded
chromo
som
e
as
solutio
n
repre
s
entatio
n for
sub
s
et fe
ature an
d
SVR p
a
ram
e
ter.
T
h
eir re
sea
r
ch
reporte
d su
ccess
to improve S
V
R pe
rforma
nce.
Our p
r
e
v
ious
re
sea
r
ch [6] inve
stig
ated PSO
ap
plicatio
n to
select
sub
s
et featu
r
e and SV
R p
a
ram
e
ter
app
lied to softwa
r
e effort
esti
mation. We u
s
ed
co
ntinuo
us
value type to
optimize
SVR paramete
r
a
nd di
screte
v
a
lue type to
select
sub
s
et f
eature. An
oth
e
r
effort has been done by
Adhani
[9], who optimized
SVR
wi
th
GAPSO. They are report
ed
su
ccess to b
u
ild SVR mo
del for predi
cting rainfa
ll i
n
dry se
ason
. Howeve
r, o
t
her re
se
arch
es
have bee
n do
ne to investig
ate on ho
w to
improve PSO perfo
rma
n
ce. Xue [10] introdu
ce
d Qo
S-
based hyb
r
id
particl
e swa
r
m optimizatio
n (G
HPSO)
t
o
sched
ule workflo
w
in
clo
ud co
mputin
g
.
It
gaine
d bette
r perfo
rma
n
ce
than PSO. A
re
sea
r
ch
co
ndu
cted
by Shieh
et.al [11
]
in modifi
cati
on
PSO with SA. Their research proposed
SAPSO to
enhance searching capa
city algorithm. They
repo
rted
pro
posed m
odel
co
uld h
a
ve
highe
r effi
ci
e
n
cy, better q
uality and fa
ster
co
nverg
ence
than PSO. T
herefore,
based on past
researches
, this
study proposed
SAPSO SVR applied to
softwa
r
e effo
rt estimation.
By using SA
PSO,
can b
e
gene
rated
m
o
re o
p
timize
SVR paramet
er
,
better selecte
d
feature, an
d low cost val
ue.
2. Support V
ector
Regr
es
sion
Given trai
nin
g
data
{
x
i
,y
i
},
i
=
1,...,
l
;
x
i
∈
R
d
;
y
i
∈
R
d
where
x
i
,
y
i
is i
nput
(vector)
and
output (scala
r value a
s
target). Othe
r fo
rm
s
of altern
ative for bia
s
to calculatio
n
f
(
x
)
is can be
build solution
lik
e bias
as
follows
[1]:
l
i
k
i
i
i
k
l
i
k
i
i
i
k
k
T
k
x
x
k
y
x
x
y
x
w
y
b
1
*
1
*
,
,
(
1
)
x
i
is supp
ort vector
whe
r
e |
α
i
-
α
i
*
| isn’t zero. Equation
f
(
x
) ca
n be wri
tten as follows:
b
x
x
x
f
l
i
i
i
i
1
*
(
2
)
Lambd
a (
λ
) is s
c
a
lar
c
o
ns
tant, with it’s
an augme
n
ted
factor d
e
fined
as follows [1
2]:
.
)
)
,
(
)(
(
)
(
1
2
*
l
i
i
i
i
x
x
K
x
f
(
3
)
2.1. Sequential Algorithm for SVR
Vijayakum
a
r
has m
ade ta
ctical
step
s throu
gh the p
r
ocess of ite
r
ation to obt
ain the
solutio
n
of o
p
timization
p
r
oble
m
s
of a
n
y natur
e
by way of a trade-off on th
e value
s
of the
weig
hts
x
i
, or
calle
d
α
i
to m
a
ke th
e results of the
reg
r
e
ssi
on b
e
com
e
s
clo
s
er to a
c
tual valu
e. T
he
step by step
as follo
ws [12
]
:
1. Initialize
0
,
0
*
i
i
. Compute
2
)
,
(
]
[
j
i
ij
x
x
K
R
(
4
)
for
i,j
= 1,
…,
n
2.
For ea
ch training poi
nt (
x
i
),
i=1
to
n
, compute:
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
Hybridiz
ing PSO with SA f
o
r Optimiz
i
ng SVR A
pplied to Software Effort
… (Din
d
a
No
vitas
a
ri
)
247
n
j
ij
i
i
i
i
R
y
E
1
*
)
(
(
5
)
}.
],
),
(
min{max[
*
*
*
i
i
i
i
C
E
(
6
)
}.
],
),
(
min{max[
i
i
i
i
C
E
(
7
)
.
*
*
*
i
i
i
(
8
)
.
i
i
i
(
9
)
3.
Rep
eat step
2 until meet stop con
d
ition.
Whe
r
e lea
r
ni
ng rate
γ
is computed fro
m
:
matrice
kernel
of
diagonal
max
constant
rate
learning
(
1
0
)
3. Particle Sw
arm Optimi
z
a
tion
Particle
swarm optimizatio
n wa
s intro
d
u
c
ed
by Ken
n
edy and Ebe
nhart [13], as a nature
inspi
r
ed al
go
rithm. Particl
e
s a
r
e defin
ed as
soluti
on for p
r
obl
em. Develo
p
i
ng by Shi and
Ebenha
rt [14
], PSO is ad
ded
by ine
r
ti
a weight
to
i
m
prove
s
pe
rforma
nce. Ea
ch
pa
rticle
h
a
s
positio
n and
velocity, and update
s
that in ever
y iterati
ng. The velocity is updated
by:
v
ij
(t+
1
)
=
wv
ij
(t)
+
c
1
r
1j
(t)[y
ij
(t)-
x
ij
(t)]+
c
2
r
2j
(t)[
ŷ
(t
)-
x
ij
(t)]
(
1
5
)
And its positi
on upd
ated b
y
:
x
i
(
t
+1)=
x
i
(
t
)+
v
i
(
t
+
1
)
(
1
6
)
Whe
r
e
v
ij
(
t
) i
s
velo
city of particl
e
i
in d
i
mensi
on
j
=
1
,
...
n
at time
t
,
x
ij
(
t
) i
s
po
si
tion of
particl
e
i
in dimensi
on
j
at time
t
,
c
1
and
c
2
are a
c
cele
ration con
s
ta
nts used to scale
contri
but
ion
of the co
gniti
ve and
so
cial
comp
one
nts,
r
1j
and
r
2j
are ran
dom val
ues i
n
the ra
nge [0,1].
W
is
inertia weight
obtained by:
max
max
max
min
max
iter
iter
iter
(
1
7
)
Whe
r
e
w
ma
x
and
w
mi
n
are maximal and minimum inertia weight
,
iter
ma
x
is m
a
ximum
numbe
r of it
eration
s
,
iter
is current
ite
r
ation
nu
mbe
r
.
Y
i
is pe
rso
nal be
st p
o
si
tion of pa
rticl
e
i
obtaine
d by:
t
i
y
f
t
i
x
f
t
i
x
t
i
y
f
t
i
x
f
t
i
y
t
i
y
1
if
1
1
if
1
(
1
8
)
And
ŷ
repre
s
ents glo
bal b
e
st po
sition o
f
particle
i
obt
ained by:
Ŷ
(t)
∈
{y
0
(t),…
,
y
ns
(t)}|f(
ŷ
(t)
)
=
m
in {f(y
0
(t)),
…, f(y
ns
(t))
}
(
1
9
)
3.1. Binar
y
PSO
Some optimi
z
ation p
r
obl
e
m
s are set
in a
spa
c
e f
eaturin
g discrete. Kenne
d
y
and
Ebenha
rt [15] prop
osed bin
a
ry PSO in which e
a
ch
ele
m
ent of parti
cle’s po
sition v
e
ctor
ca
n take
on the bina
ry value 0 or 1. Ne
w velocity of
particle i
s
norm
a
lized b
y
sigmoid fun
c
tion:
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 1, March 2
016 : 245 – 2
5
3
248
t
v
ij
ij
ij
e
t
v
sig
t
v
1
1
(
2
0
)
Whe
r
e
v
ij
(
t
) is obtained fro
m
Equation (15). Using Eq
uat
ion (1
6), the po
sition u
pdate chan
ge
s to:
otherwise
0
1
if
1
1
3
t
v
sig
t
r
t
x
ij
j
ij
(
2
1
)
Whe
r
e
r
3j
(
t
) ~
U
(0,1
).
4. H
y
bridiz
in
g PSO
w
i
th
SA
A searchi
ng
algorith
m
ha
s two imp
o
rt
ant
com
pon
e
n
ts, exploration and
exploitation.
Exploration
mean
s algo
ri
thm sea
r
ch in differ
ent region of se
a
r
chi
ng spa
c
e
to find global
optimum. Exp
l
oitation mea
n
s al
go
rithm l
o
cali
ze
pro
m
i
s
ing
area to fi
nd be
st
soluti
on in that
are
a
.
A good
se
arching al
gorith
m
must
able t
o
bala
n
ce it
s
exploratio
n a
nd exploitatio
n
, able to
se
a
r
ch
entire
spa
c
e
and jum
p
out
of local opti
m
um soluti
on
. By that me
ans, it mu
st able to imp
r
o
v
e
probability and ability of fin
d
ing global optimum sol
u
tion.
Initial ran
dom
po
sition PS
O can le
ad in
to pr
e
m
ature
conve
r
ge
nce, entire
pa
rticl
e
move
toward
lo
cal optimum solu
tion
and ca
use
we
akenin
g
exploratio
n b
e
ca
use pa
rticle ca
n’t jump
out
of are
a
. It is cha
r
a
c
teri
stic and
wea
k
ne
ss of
PS
O. Mean
whil
e, SA with l
o
w va
riation
of
temperature
para
m
eter a
n
d
sea
r
ching solution re
ach equilib
rium condition, able
to guarantee
to
find global
optimum. It is enhan
ced by metropolis process, ab
ility to jump out from l
o
cal
optimum. Ho
wever, it co
st
s high
comp
u
t
ing time.
Based
on
PSO and
SA ch
ara
c
te
risti
c
a
bove, thi
s
study hyb
r
idize
s
PSO
with SA,
combi
n
e
s
PSO parallel p
r
oce
s
s and
m
o
vement me
cha
n
ism
and
SA searchi
n
g pro
c
e
dure. By
combi
n
ing th
at, this propo
sed mo
del ab
le to find goo
d solutio
n
in local a
nd glob
al optimum with
low computin
g time.
4.1. Simulated Anne
aling Algorithm
Simulated an
nealin
g is a
n
optimizatio
n
pro
c
e
ss
ba
sed on the
an
nea
lin
g process; the
cooli
ng p
r
o
c
e
ss
of a liquid
or solid an
d the analy
s
is
o
f
the behavio
r of su
bsta
nces a
s
they co
ol.
This alg
o
rith
m is introd
uced by Kirkp
a
trick [
16]
and inspi
r
ed
by Metrop
olis work about
en
ergy
distrib
u
tion [
17]. In SA algorith
m
, m
e
tropoli
s
pro
c
e
ss doe
s searchin
g sol
u
tion.
Du
ring
the
pro
c
e
ss, di
st
urba
nce me
chani
sm
(m
etropoli
s
acce
ptance rul
e
) d
e
t
er
mine
s qu
al
ity of solution
b
y
sea
r
ching a
r
ound existin
g
solution an
d
compa
r
ing n
e
ighb
or soluti
on and curre
n
t solution. T
h
is
pro
c
ed
ure affects SA abili
ty to jump o
u
t from local
optimum sol
u
tion. If neighbor
solutio
n
is
better than current solutio
n
then neigh
b
o
ring
solution
is accepted
as the ne
w current solutio
n
. If
neigh
bor
sol
u
tion is
wo
rse than
curre
n
t solutio
n
th
en SA will u
s
e a
pro
babil
i
ty to determi
ne
wheth
e
r a
c
ce
pt this neigh
b
o
ring
solutio
n
as ne
w curre
n
t solution o
r
not, or reg
e
n
e
rate for a
ne
w
neighbori
ng
solution. The probability mechani
sm for
metropoli
s
acceptance
rule is defined as
follows
:
otherwise
if
1
T
c
x
f
x
f
i
j
b
i
j
e
x
f
x
f
P
(
2
2
)
Whe
r
e
P
is probability,
f
(
x
j
) i
s
neig
h
bor
solution,
f
(
x
i
) is
cu
rr
ent solutio
n
,
c
b
>0 i
s
Boltzman
n co
nstant an
d
T
is tempe
r
ature of the syste
m
. T is derived from:
T
k+1
=
α
x
T
0
(
2
3
)
Whe
r
e
α
i
s
cooling
rate,
T
k+1
is tempe
r
ature at time
k
, an
d
T
0
is initial temperature.
While SA is
quite simpl
e
, it has bee
n succe
ssfully i
m
pleme
n
ted
to solve vario
u
s combin
ato
r
ial
probl
em [18].
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
Hybridiz
ing PSO with SA f
o
r Optimiz
i
ng SVR A
pplied to Software Effort
… (Din
d
a
No
vitas
a
ri
)
249
5. SAPSO SVR Model
5.1. Particle Repr
esen
ta
tion
In this
study,
SVR
RBF i
s
define
d
by t
he p
a
ra
meter
C
– compl
e
xity
param
eter,
ε
- the
extent to whi
c
h d
e
viations are tole
rated
,
λ
- augme
n
ting facto
r
,
σ
– width of
RB
F ke
rnel,
cL
R
–
learni
ng rate con
s
tant. The
particle i
s
co
mpri
sed of si
x parts:
C,
ε
,
λ
,
σ
, c
L
R
(co
n
tinuou
s-val
u
ed
)
and featu
r
e
s
mask (di
s
crete-value
d
).
T
able 1
sh
ows the represe
n
tation of
particl
e
i
wi
th
dimen
s
ion
n
f
+5 whe
r
e
n
f
is the n
u
m
ber
of features. Th
e feat
ure m
a
sk i
s
Boolean th
at “1”
indicates the
feature is
sel
e
cted a
nd “0
” indicate
s fea
t
ure is not
sel
e
cted.
Table 1. Parti
c
le
i
is compo
s
ed of six pa
rts: c,
ε
,
λ
,
ε
, cLR
an
d feature mask
Con
t
inu
o
u
s
-
v
al
ued
Discrete
-
v
a
lu
ed
C
ε
λ
σ
cLR
Feat
ure
ma
sk
X
i,1
X
i,2
X
i,3
X
i,4
X
i,5
X
i,6
, X
i,
7
,...,X
i,nf
5.2. Objectiv
e Functio
n
Obje
ctive function is u
s
ed
to measure how
optim
al the gene
rate
d solution. T
here a
r
e
two types of
obje
c
tive function: fitness
and cos
t. The
highe
r fitness value me
an
s better
soluti
on.
The lower cost value me
ans bette
r solution. In
this study, co
st typed is used as obj
ecti
ve
function
be
ca
use
the
pu
rp
ose
of thi
s
al
gorithm
is
to
minimize e
r
ro
r. A
ccu
ra
cy
of predictio
n
and
numbe
r
of sel
e
cted
featu
r
e
s
a
r
e
criteri
a
use
d
to
de
sig
n
cost
fun
c
tio
n
. Thu
s
, the
p
a
rticle
with hi
gh
accuracy
of
predi
ction
an
d small n
u
m
ber
of f
eatu
r
es pro
d
u
c
e
s
a
lo
w pre
d
i
c
tion error. The
predi
ction
error ha
s t
w
o p
r
edefine
d
wei
ghts:
W
A
fo
r accuracy
of p
r
edi
ction (95
%
)
and
W
F
for the
sele
cted feat
ure (5%) [19].
n
i
i
i
i
A
F
A
n
MAPE
1
1
(
2
4
)
F
n
j
j
F
A
n
f
w
MAPE
w
error
F
1
(
2
5
)
Whe
r
e
n
is n
u
mbe
r
of data,
A
i
is actua
l
value and
F
i
is predi
ction
value for da
ta,
f
j
is
value of feature ma
sk wh
ere “1” rep
r
e
s
ent
s that feature
j
is
sel
e
cted a
nd “0
” rep
r
e
s
ent
s
that
feature
j
is no
t selecte
d
an
d
n
f
is total nu
mber of featu
r
es.
5.3. SAPSO SVR Algorithms
The SAPSO SVR algorithm is started
by initia
lization of particle. Then, cal
c
ul
ate cost
and d
e
termi
n
e perso
nal b
e
s
t po
sition (
pBe
s
t
) a
nd gl
o
bal be
st po
siti
on (
gB
est
). Af
ter that, upd
ate
veloc
i
ty and
pos
ition. Us
ually, PSO aut
omatic
ally
acc
ept
new pos
ition, however SAPSO S
V
R
introdu
ce
s S
A
metropoli
s
accepta
n
ce rule in this
ste
p
. This rul
e
d
e
termin
es
wh
ether to a
c
ce
pt
new
po
sition
or rege
nera
t
e anothe
r candid
a
te
po
sition ba
sed
on cost fun
c
tion differen
c
e
betwe
en ne
w and old po
si
tions. This e
nable
s
PSO to jump out from lo
cal opt
imum, impro
v
e
quality of sol
u
tion, and i
n
cre
a
se rate
of conve
r
g
e
n
c
e. Simulate
d ann
ealin
g
explore
s
solu
tion
towards dire
ction of
pBe
s
t
an
d
gBe
s
t
. The a
c
cept
ance rule a
c
cept
s o
r
reje
cts
ne
w solu
tion
based o
n
current temp
erat
ure p
a
ramete
r and
co
st val
ue differe
nce. If candid
a
te
solutio
n
un
ab
le
pass criteri
a
then a ne
w po
sition
gene
rated
u
s
ing PSO a
nd re
peate
d
until metro
polis
accepta
n
ce rule acce
pt ne
w po
sition o
r
uppe
r bou
nd
of disturban
ce is re
ached.
By this way, the
model explo
r
es solution, improve explo
r
ation
an
d sp
end low
com
puting time si
nce u
s
in
g PSO
parall
e
l pro
c
e
ssi
ng.
Bas
ed on Figure 1, the whole proc
edur
e of SAPSO S
V
R is
des
c
ribed as
follows
:
1.
Normali
z
ing d
a
ta usin
g
min
max
min
x
x
x
x
x
n
(
2
6
)
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 1, March 2
016 : 245 – 2
5
3
250
Whe
r
e
x
is
the original data from dataset,
x
mi
n
and
x
ma
x
is the min
i
mum an
d m
a
ximum valu
e of
origin
al data, and
x
n
is normalized value.
2.
Dividing data
into k to determine traini
ng
and testin
g d
a
ta.
3.
Initializing a p
opulatio
n of particle
ran
d
o
m
ly
0
1
s
,
0
0
1
v
i=1,2,…
n
,
iter
=0.
4. Cal
c
ulating
cost
of
0
1
s
by averagin
g
erro
r o
v
er
k
SVR tra
i
ning.
5. Upd
a
ting
pBe
s
t
and
gBe
s
t
of each p
a
rticle.
6.
Upd
a
ting ine
r
tia weight.
7.
Rep
eat these
steps u
n
til meet stoppi
ng
con
d
ition
a) Upd
a
ting
velo
city
1
1
iter
v
and po
sition of each pa
rticle.
b) Cal
c
ulating
cost
of
1
1
iter
s
by averagin
g
erro
r o
v
er
k
SVR tra
i
ning.
c) Evaluate
∆
cos
t
=
iter
iter
s
s
1
1
1
cost
cost
an
d g
e
nerate
rand
o
m
num
ber
R [0,1]. If
∆
co
st
≤
0
,
then
ac
ce
p
t
new
po
s
i
tion
w
i
th
pr
o
b
a
b
i
lity O
N
E. O
t
h
e
r
w
i
se
,
1
1
iter
s
is
accepte
d
based on followi
n
g
crite
r
ion:
temp
t
e
cos
y
probabilit
≥
R. Proceed to
next step if
all new po
siti
ons a
r
e a
c
ce
pted or repea
t st
ep 7.1 until 7.3 for those particl
es fai
l
ed
to
be accepted.
Too many failu
res
(i.e. 100 in ou
r
study) for same
particl
e
will force
the last position will be accepted.
d) Upd
a
ting
pBe
s
t
and
gBe
s
t
of each p
a
rticle.
e)
Upd
a
ting ine
r
tia weight an
d temperature, set
iter
=
iter
+1.
8.
If stopping
criteria is
sati
sfied, and th
en
end iteration
.
If not, repe
at step 7. In
this
study, stoppi
ng crite
r
ia i
s
a
given numb
e
r of iteration
s
.
9.
Output the be
st solutio
n
gBest
and its
co
st value.
Figure. 1 Flowchart of SAPSO SVR algorithm
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
Hybridiz
ing PSO with SA f
o
r Optimiz
i
ng SVR A
pplied to Software Effort
… (Din
d
a
No
vitas
a
ri
)
251
6. Application SAPSO SVR in
Soft
w
a
r
e
Effor
t
Esti
m
a
tion
6.1. Sim
u
lation Setting
s
This
study si
mulates
2 alg
o
rithm
s
: PSO SVR and SA
PSO SVR progra
mmed
u
s
ing
C#.
For SAPSO
SVR s
i
mulation, we us
e the
s
a
me para
meter and datas
e
t that is obtained from [6]
that co
ndu
ct
ed PSO SV
R
simulatio
n
. For software effort e
s
ti
mation, the i
nputs of SV
R a
r
e
De
sha
r
nai
s d
a
taset [20]. T
he De
sh
arnai
s data
s
et con
s
ist
s
of 81
so
ftware p
r
oje
c
t
s
de
scrib
ed b
y
11 vari
able
s
,
9 inde
pen
den
t variable
s
a
n
d
2 d
epe
nde
nt variabl
es.
For th
e si
mul
a
tion, we
de
cide
to use 77 p
r
o
j
ects d
ue to i
n
com
p
lete p
r
ovided features an
d 7 inde
pend
ent varia
b
les
(
Team
Exp
,
Manag
erE
x
p
,
T
r
an
sa
c
t
ions
,
Entities
,
Points
Adjus
t
,
Env
e
rgure
, and
Poi
n
tsNonAdju
s
t
) an
d
1
depe
ndent va
riable
(
Effort
). The PSO paramete
r
s
we
re set as in T
able 2.. Firstl
y, we run test
to
determi
ne
be
st pa
ram
e
ter for SA
(
T
0
a
nd
α
) the
n
si
mulation
s i
s
perfo
rmed
an
d comp
are
d
to
other alg
o
rith
ms.
Table 2. PSO
param
eter settings
Number of
fold
Population of par
ticles
Number of ite
r
ati
ons
Inertia w
e
ight(
w
ma
x
, w
mi
n
)
Acceleration coefficient(c
1
, c
2
)
Parameter sea
r
c
h
ing space
10
20
40
(0,9, 0,4
)
(2, 2)
C (0,1
-1500
),
ε
(
0
,001-0,0
09),
σ
(
0
,1-4),
λ
(
0
,01-3
)
,
cLR
(0,01-
1,75)
6.2. Best Par
a
meter
In sto
c
ha
stic
algorith
m
s, p
a
ram
e
ters h
a
v
e effe
ct to q
uality of gen
e
r
ated
sol
u
tion
. In SA,
initial temperature an
d co
oling rate infl
uen
ce
its pe
rforman
c
e. By obse
r
ving p
a
ram
e
ters, we
cho
o
se be
st para
m
eter h
a
s
lowest cost
in each simul
a
tion.
Table 3
sh
o
w
ed
simul
a
tion to ch
oo
se be
st initial temperature
(
T
0
). Thi
s
s
i
mulation
con
d
u
c
ted by incre
a
si
ng tempe
r
ature b
y
10% from 50 up to 90 in each
simu
lation and u
s
e
cooli
ng rate a
t
0,5.
If
T
0
is too low then
algorithm
has
possibility to
not ex
plore, makes
converge
at local optim
um. If
T
0
is too high, then
it can increa
se co
mp
uting
time. This table sh
owed that
T
0
at 90 give lowe
st co
st.
Table
4
sh
o
w
ed
si
mulati
on to
ch
oo
se
be
st coolin
g
rate
(
α
).
Thi
s
simul
a
tion con
d
u
c
ted
by increa
sin
g
tempe
r
atu
r
e
by 10%
fro
m
0,5
up to
0,9 in
ea
ch
simulation If
α
is too l
o
w th
en
algorith
m
ha
s po
ssi
bility to fail into local optimum solution, rep
e
a
ts cal
c
ulatio
n, and increa
se
c
o
mputing time. If
α
is too
high, the
n
it increa
se
com
puting time. T
h
is tabl
e
sho
w
ed th
at
α
at 0,9
give lowes
t
cos
t.
Table 3. Para
meter setting for initial temperatu
r
e
i
-th
simulation
T
0
50 60 70
80 90
1
0,6084
0,8336
0,5801
0,6096
0,5690
2
0,5706
0,5760
0,7265
0,5975
0,5734
3
0,5992
0,5898
0,5886
0,5848
0,5682
4
0,5610
0,6177
0,6223
0,5875
0,5935
5
0,7475
0,7579
0,6161
0,5795
0,5761
Average
Cost
0,6174
0,6750
0,6267
0,5918
0,5760
Table 4. Para
meter setting for cooli
ng rate
i
-th
simulation
α
0,5
0,6 0,7 0,8
0,9
1
0,5690
0,5861
0,5776
0,5769
0,5659
2
0,5734
0,5957
0,5994
0,6163
0,5563
3
0,5682
0,5898
0,6182
0,6003
0,5748
4
0,5935
0,5765
0,6007
0,5753
0,5575
5
0,5761
0,6013
0,6026
0,5778
0,5628
Average
Cost
0,5760
0,5899
0,5997
0,5893
0,5635
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 1, March 2
016 : 245 – 2
5
3
252
6.3. Compari
s
on Wor
k
s
By us
ing bes
t
parameter,
we compare S
APSO SVR and PSO SV
R
performanc
e
.
Figure
2 s
h
owed
c
o
mparis
on of
convergenc
e betw
een PSO
SVR and SAPSO. It s
howed that SAPSO
have faster convergence than PSO SVR. In T
able 5, we can see that SAPSO has hi
gher
c
o
mputing time fas
t
er than PSO SV
R but on
the other
hand, SAPSO also c
a
n have fas
t
er
conve
r
ge
nce and lo
wer
co
st than PSO SVR. The erro
r differe
nce of erro
r is bi
g, but the hig
h
comp
uting ti
me can
be
compromised.
The
co
mput
ing time i
s
hi
gh b
e
cau
s
e t
he mo
del
m
u
st
repe
at sea
r
ching candid
a
t
e position if
they fail me
et the accept
ance rule
crit
eria an
d this is
different with
PSO automat
ically acce
pt can
d
idate p
o
s
ition.
Table 5. Simulation re
sult
Model
Time (ms
)
Op
timal (
C,
ε
,
σ
, c
L
R
,
λ
)
Selected
fea
t
ur
es
Error
PSO-SVR
50238
393,04, 0,09,
0,1
,
0,01, 1,6669
2 (
PointsAdjust
and
PointsNonAdjust
) 0,5981
SAPSO-SVR
83756
1500, 0,0401
, 0,
3996, 0,01, 0,
01
2 (
Envergu
r
e,
an
d
PointsNonAdjust
) 0,5575
Figure 2. Convergence
curve PSO SVR
and SAPSO
7. Conclusio
n
This
s
t
udy inves
t
igated the us
e of SAPSO fo
r optimal
feature
s
u
bs
et s
e
lec
t
ion and SVR
para
m
eters o
p
timization in
the probl
em o
f
software
effort es
timation. In our s
i
mulations
, we used
De
sha
r
nai
s d
a
taset. We
co
mpared ou
r result
s to PSO-SVR. From the experi
m
en
t result
s, usin
g
SA can impro
v
e perform
an
ce of PSO. The pro
p
o
s
ed
model can co
mbine the ad
vantage of bo
th
algorith
m
s a
n
d
gain lo
wer
co
st than PSO.
Referen
ces
[1]
Smola AJ, Scholk
opf B. A
T
u
toria
l
on Su
p
port Vector Re
gressi
on.
Stati
s
tics and C
o
mputin
g
. 200
4;
14(3): 19
9-2
2
2
.
[2]
W
ang W
,
Xu Z
,
Lu W
,
Z
hang X. Determ
inati
on of T
he Spre
ad Param
e
ter i
n
the Gaussi
a
n
Kerne
l
F
o
r
Classific
a
tio
n
a
nd Re
gressi
on.
Neuroc
o
m
puti
n
g
. 200
3; 55: 6
43–
66
3.
[3]
Frohlich
H, C
h
ape
lle
O, Sch
o
lko
p
f B.
F
eature Se
lecti
on f
o
r Sup
port V
e
ctor Machi
nes
by Mea
n
s of
Genetic Al
gor
i
t
hm
. In Proc
eedings
15th I
EEE
International Conf
erence on T
ools w
i
t
h
Artificial
Intelli
genc
e. 20
03: 142-
14
8.
[4]
Oliveira
ALI. E
s
timation
of s
o
ft
w
a
r
e
pr
oj
ect effort
w
i
t
h
su
pport vect
or r
egress
i
on.
N
e
uroco
m
putin
g
.
200
6; 69(1
3
-15
)
: 1749–
53.
[5]
Braga PL, Oliv
eira ALI, Meir
a
SRL.
A GA-based F
eat
ure
Selecti
on a
nd
Para
meters O
p
timi
z
a
ti
on for
Supp
ort Vector
Regr
essio
n
A
ppli
e
d
to Softw
are Effort Esti
mati
on
. In Pr
o
c
eed
ings
of th
e 20
08
ACM
S
y
mp
osi
u
m on
Appli
ed Com
p
uting. F
o
rtalez
a, Ceará, Brazi
l
, ACM. 2008: 178
8–
179
2.
[6] Novitas
a
ri
D,
C
holiss
o
d
i
n I, M
ahmu
d
y
W
F
. Optimizin
g
s
upp
ort vector r
egr
essio
n
us
in
g p
a
rticle s
w
a
r
m
optimiz
ation f
o
r soft
w
a
re effo
rt estimation.
Sub
m
itted t
o
: IAENG Interna
t
iona
l Jo
urna
l
of Co
mp
uter
Scienc
e
. 2
0
15
[7]
Shi Y, Eber
hart RC.
Empir
i
c
a
l study of p
a
r
ticle sw
arm
o
p
timi
z
a
ti
on
. In
Procee
din
g
s
of the 19
99
Con
g
ress on E
v
oluti
onar
y C
o
mputat
io
n-CE
C99. 19
99: 19
45–
19
50.
[8]
Locate
lli M. C
o
nverg
ence
pro
perties
of simul
a
t
ed a
n
n
eal
in
g
for contin
uo
us
glo
bal
optimiz
ation.
Journal
of Appli
ed Pro
bab
ility
. 19
96; 33(4): 11
27
–11
40.
[9]
Adhani G, Buono A, Faqih A. Optimization of
Support Vector Regression using Genetic Algorithm and
0
0.2
0.4
0.6
0.8
1
0
3
6
9
1
21
51
82
12
42
7
3
03
33
63
9
Cost
Iter
a
t
ion
PSO
SVR
SAPSO
SVR
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
Hybridiz
ing PSO with SA f
o
r Optimiz
i
ng SVR A
pplied to Software Effort
… (Din
d
a
No
vitas
a
ri
)
253
Particle
S
w
arm
Optimizati
on f
o
r R
a
infa
ll
Pre
d
ictio
n
i
n
Dr
y
Seaso
n
.
T
E
LK
OMNIKA Indon
esia
n Jo
urn
a
l
of Electrical En
gin
eeri
n
g
. 20
1
4
; 12(11): 7
912
–79
19.
[10]
Xu
e S, W
u
W
.
Sched
uli
ng W
o
rkflo
w
in
Clo
ud
Co
mputi
ng B
a
sed o
n
H
y
br
id
Particle S
w
a
r
m
Algorit
hm.
T
E
LKOMNIKA Indon
esi
an Jou
r
nal of Electric
al Eng
i
ne
eri
n
g
.
2012; 1
0
(7): 1
560
–1
566.
[11]
Shie
h H-L, Kuo C-C, Chia
ng
C-M. Modified particle s
w
a
rm optimizatio
n alg
o
rithm
w
i
th simulate
d
ann
eal
in
g be
h
a
vior a
nd its
numeric
al ver
i
ficatio
n
.
Appli
e
d Mathe
m
atics
and Co
mput
ation
. 20
11;
218(
8): 436
5–4
383.
[12]
Vijay
a
kumar
S
,
Wu S.
Sequ
entia
l Su
pp
ort Vector C
l
assi
fiers an
d R
egr
essio
n
. In Pro
c
eed
ings
o
f
Internatio
na
l C
onfere
n
ce o
n
Soft Co
mputin
g (SOCO ‘99). 1999: 61
0–
61
9.
[13] Kenn
ed
y
J,
Eberh
a
rt R.
Particle Sw
ar
m Opti
mi
z
a
t
i
o
n
. In Proceedings of IEEE International
Confer
ence
on
Neura
l
Net
w
or
ks. Piscata
w
a
y, NJ. 1995: 194
2–1
94
8.
[14]
Shi Y, Eberhar
t R.
A Modified Particle Sw
a
r
m Opti
mi
z
e
r
. In 1998 IEEE Internat
ional Conference on
Evoluti
onar
y
C
o
mputati
on Pr
ocee
din
g
s IEE
E
W
o
rl
d C
o
n
g
r
ess on
Com
p
utation
a
l Inte
lli
genc
e. 19
98:
69–
73.
[15]
Kenn
ed
y J, Eb
erhart RC.
A di
screte bin
a
ry versio
n of
the particle sw
arm a
l
gorit
hm
. In Procee
din
g
s o
f
the W
o
rld Mi
ult
i
confer
ence
on
S
y
stemics, C
y
bern
e
tics an
d Informatics. Pis
c
ata
w
a
y
, NJ. 1
997: 4
1
0
4
–
410
9.
[16]
Kirkpatrick S,
Gelatt CD, Vec
c
hi MP. Optimi
zation
b
y
Sim
u
l
a
ted A
nne
ali
n
g
.
Science
. 198
3;
220(
45
98)
:
671
–6
80.
[17]
Metropo
lis N,
Rose
nbl
uth A
W
, Rosenb
luth
MN, T
e
lle
r AH
,
T
e
ller E. Eq
u
a
tion
of State
Calcu
l
ati
ons
b
y
F
a
st Computin
g Machi
nes.
Jo
urna
l of Che
m
i
c
al Physics
. 19
53 ;108
7(2
1
).
[18]
Mahmu
d
y
W
F
. Improve
d
sim
u
late
d
ann
ea
li
ng for
o
p
timiz
a
tion
of v
ehic
l
e ro
uting
pr
obl
em
w
i
t
h
tim
e
w
i
ndo
w
s (VRP
TW).
Kursor
. 2014; 7(3): 1
09–
116.
[19] Guo
Y.
A
n
Int
egrated PSO
for Parame
ter
Deter
m
inati
o
n
an
d F
e
ature
Selecti
on
of S
V
R a
n
d
Its
Appl
icatio
n in
ST
LF
. In Proceed
ings of the
Eighth Intern
at
i
ona
l Confer
en
ce on Mach
ine
Learn
i
ng a
n
d
C
y
ber
netics. Baod
ing. 2
009:
12–
15.
[20]
Sa
yya
d
Shir
ab
ad J, Me
nzies
T
J
.
T
he PROMI
SE Repos
itor
y of Soft
w
a
re En
gi
neer
in
g Data
bas
e
s
[Internet]. School of Info
rmation T
e
chnology
and Engineer
ing, Un
ivers
i
t
y
of
Ottaw
a
, Canada. 2005.
Evaluation Warning : The document was created with Spire.PDF for Python.