I
nte
rna
t
io
na
l J
o
urna
l o
f
Adv
a
nces in Applie
d Science
s
(
I
J
AAS)
Vo
l.
9
,
No
.
4
,
Dec
em
b
e
r
2
0
2
0
,
p
p
.
284
~
293
I
SS
N:
2
2
5
2
-
8
8
1
4
,
DOI
:
1
0
.
1
1
5
9
1
/ijaas.v
9
.
i4
.
p
p
2
8
4
-
293
284
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
a
s
.
ia
esco
r
e.
co
m
Dev
elo
pment of
s
o
ft
wa
re de
fec
t
pr
ediction sy
stem
u
sing
a
rtif
icia
l ne
ura
l n
etwork
O
la
t
un
j
i B.
L
.
1
,
O
la
biy
is
i S.
O
.
2
,
O
y
eley
e
C.
A.
3
,
Sa
nu
s
i B
.
A.
4
,
O
lo
wo
y
e
A.
O
.
5
,
O
f
em
O
.
A.
6
1,
2,
4,
5
De
p
a
rtme
n
t
o
f
C
o
m
p
u
ter S
c
i
e
n
c
e
,
Lad
o
k
e
Ak
in
to
la
U
n
iv
e
rsit
y
o
f
Tec
h
n
o
l
o
g
y
,
Ni
g
e
ria
3
De
p
a
rtme
n
t
o
f
In
fo
rm
a
ti
o
n
S
y
ste
m
s,
Lad
o
k
e
A
k
in
t
o
la Un
iv
e
rsity
o
f
Tec
h
n
o
lo
g
y
,
Nig
e
ria
6
De
p
a
rtme
n
t
o
f
Co
m
p
u
ter
S
c
ien
c
e
,
Un
iv
e
rsit
y
o
f
Ca
lab
a
r,
Nig
e
ria
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Mar
23
,
2
0
2
0
R
ev
is
ed
J
u
n
1
1
,
2
0
20
Acc
ep
ted
J
u
n
1
8
,
2
0
20
S
o
ftwa
re
tes
ti
n
g
is
a
n
a
c
ti
v
it
y
t
o
e
n
a
b
le
a
sy
ste
m
is
b
u
g
fre
e
d
u
rin
g
e
x
e
c
u
ti
o
n
p
ro
c
e
ss
.
Th
e
so
f
twa
re
b
u
g
p
re
d
ictio
n
is
o
n
e
o
f
th
e
m
o
st
e
n
c
o
u
ra
g
i
n
g
e
x
e
rc
ise
s
o
f
th
e
tes
ti
n
g
p
h
a
se
o
f
t
h
e
so
ftwa
re
imp
ro
v
e
m
e
n
t
li
fe
c
y
c
le.
In
a
n
y
c
a
se
,
in
t
h
is
p
a
p
e
r,
a
fra
m
e
wo
rk
wa
s
c
re
a
ted
to
a
n
t
icip
a
te
t
h
e
m
o
d
u
les
th
a
t
d
e
fo
rm
it
y
in
c
li
n
e
d
i
n
o
rd
e
r
to
b
e
u
ti
li
z
e
d
to
a
l
l
t
h
e
m
o
re
li
k
e
ly
o
rg
a
n
ize
so
ftwa
re
q
u
a
li
ty
a
ffir
m
a
ti
o
n
e
x
e
rt
io
n
.
G
e
n
e
ti
c
Alg
o
r
it
h
m
wa
s
u
se
d
to
e
x
tra
c
t
re
lev
a
n
t
fe
a
tu
re
s
fro
m
th
e
a
c
q
u
i
re
d
d
a
tas
e
ts
to
e
li
m
in
a
te
t
h
e
p
o
ss
ib
il
it
y
o
f
o
v
e
rfit
ti
n
g
a
n
d
t
h
e
re
lev
a
n
t
fe
a
tu
r
e
s
we
re
c
las
sified
to
d
e
fe
c
ti
v
e
o
r
o
th
e
rwise
m
o
d
u
les
u
sin
g
t
h
e
Artifi
c
ial
Ne
u
ra
l
Ne
two
rk
.
Th
e
sy
ste
m
wa
s
e
x
e
c
u
ted
i
n
M
ATLAB
(R2
0
1
8
a
)
R
u
n
ti
m
e
e
n
v
iro
n
m
e
n
t
u
t
il
izin
g
a
sta
ti
stica
l
to
o
l
k
it
a
n
d
th
e
p
e
rfo
rm
a
n
c
e
o
f
th
e
sy
ste
m
wa
s
a
ss
e
ss
e
d
d
e
p
e
n
d
e
n
t
o
n
th
e
a
c
c
u
ra
c
y
,
p
re
c
isio
n
,
re
c
a
ll
,
a
n
d
th
e
f
-
sc
o
re
to
c
h
e
c
k
t
h
e
e
ffe
c
ti
v
e
n
e
ss
o
f
th
e
sy
ste
m
.
In
th
e
fi
n
ish
o
f
t
h
e
le
d
e
x
p
l
o
re
s,
t
h
e
o
u
tco
m
e
i
n
d
ica
ted
t
h
a
t
EC
L
IP
S
E
JD
T
CORE,
ECL
IP
S
E
P
DE
UI,
EQ
UIN
OX
F
RAME
WORK
a
n
d
L
UCENE
h
a
s
th
e
a
c
c
u
ra
c
y
,
p
re
c
isio
n
,
re
c
a
ll
a
n
d
t
h
e
f
-
sc
o
re
o
f
8
6
.
9
3
,
5
3
.
4
9
,
7
9
.
3
1
a
n
d
6
3
.
8
9
%
re
sp
e
c
ti
v
e
ly
,
8
3
.
2
8
,
3
1
.
9
1
,
4
5
.
4
5
a
n
d
3
7
.
5
0
%
re
sp
e
c
ti
v
e
ly
,
8
3
.
4
3
,
5
7
.
6
9
,
4
5
.
4
5
a
n
d
5
0
.
8
4
%
re
sp
e
c
ti
v
e
ly
a
n
d
9
1
.
3
0
,
3
3
.
3
3
,
5
0
.
0
0
a
n
d
4
0
.
0
0
%
re
sp
e
c
ti
v
e
ly
.
Th
is
p
a
p
e
r
p
re
se
n
ts
a
n
imp
r
o
v
e
d
so
ftwa
re
p
re
d
icti
v
e
sy
ste
m
fo
r
th
e
so
ftwa
re
d
e
fe
c
t
d
e
tec
ti
o
n
s.
K
ey
w
o
r
d
s
:
Ar
tific
ial
n
eu
r
al
n
etwo
r
k
Gen
etic
alg
o
r
ith
m
So
f
twar
e
d
ef
ec
t p
r
ed
ictio
n
So
f
twar
e
m
etr
ics
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Olatu
n
ji B
.
L
.
,
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
Scie
n
ce
,
L
ad
o
k
e
Ak
i
n
to
la
Un
iv
er
s
ity
o
f
T
ec
h
n
o
lo
g
y
,
Gr
a
o
k
e
aa
f
i
n
ar
ea
Og
b
o
m
o
s
o
n
o
r
th
lo
ca
l
g
o
v
e
r
n
m
e
n
t,
2
1
0
2
1
4
,
Og
b
o
m
o
s
h
o
,
Nig
er
ia
.
E
m
ail:
o
latu
n
ji_
tu
n
d
e@
y
ah
o
o
.
co
m
1.
I
NT
RO
D
UCT
I
O
N
A
s
o
f
twar
e
d
ef
ec
t
is
a
f
a
u
lt,
b
l
u
n
d
er
,
o
r
f
ailu
r
e
in
a
s
o
f
twar
e
s
y
s
tem
[
1
]
.
I
t
cr
ea
tes
eith
er
an
o
f
f
b
ase,
o
r
u
n
f
o
r
eseen
r
esu
lt,
an
d
ac
t
s
in
a
u
n
in
ten
d
ed
way
[
2
]
.
I
t
is
a
f
law
in
th
e
s
o
f
twar
e
s
y
s
tem
th
at
m
ak
es
it
p
er
f
o
r
m
o
u
t
o
f
th
e
b
l
u
e
[
3
]
.
A
s
o
f
twar
e
d
ef
ec
t
ca
n
b
e
r
ef
er
r
ed
to
as
im
p
e
r
f
ec
tio
n
d
u
r
in
g
th
e
s
o
f
twar
e
im
p
r
o
v
em
e
n
t
p
r
o
ce
s
s
th
at
m
a
k
es
th
e
s
o
f
twar
e
f
ail
an
d
n
o
t
m
ee
ts
th
e
id
ea
l
d
esire
[
4
]
.
T
h
e
d
ef
ec
t
p
r
ed
ictio
n
i
n
s
o
f
twar
e
is
th
e
way
to
war
d
d
ec
id
in
g
p
iece
s
o
f
a
s
o
f
twar
e
s
y
s
tem
th
at
m
ay
c
o
n
tain
b
u
g
s
[
5
]
.
Use
o
f
Def
ec
t
Pre
d
ictio
n
s
y
s
tem
s
in
th
e
ea
r
l
y
s
o
f
twar
e
life
-
cy
cle
p
er
m
its
th
e
p
r
o
t
o
f
o
c
u
s
th
eir
test
in
g
l
ab
o
r
in
a
way
th
at
th
e
p
ar
ts
id
e
n
tifie
d
as
m
is
tak
e
in
clin
ed
ar
e
tr
ied
i
n
s
id
e
an
d
o
u
t
in
c
o
n
tr
ast
with
d
if
f
er
en
t
p
iece
s
o
f
th
e
s
o
f
twar
e
s
y
s
tem
[
6
]
T
h
is
p
r
o
m
p
ts
th
e
d
ec
r
ea
s
e
o
f
lab
o
r
co
s
ts
d
u
r
in
g
im
p
r
o
v
em
e
n
t
an
d
f
u
r
th
er
m
o
r
e
lo
o
s
en
s
u
p
th
e
s
u
p
p
o
r
t
ef
f
o
r
t
[
7
]
.
L
ate
in
v
esti
g
atio
n
s
r
ep
o
r
t
t
h
at
th
e
ch
an
ce
o
f
b
u
g
d
is
co
v
e
r
y
b
y
t
h
e
s
o
f
twar
e
d
ef
ec
t
p
r
e
d
ictio
n
s
y
s
tem
s
m
ig
h
t
b
e
h
i
g
h
er
t
h
an
th
e
c
h
an
c
e
o
f
id
en
ti
f
icatio
n
b
y
as
o
f
n
o
w
u
tili
ze
d
s
o
f
twar
e
au
d
its
in
m
ec
h
an
ical
s
tr
ateg
i
es
[
8
]
.
T
h
u
s
ly
,
t
h
e
r
ig
h
t
p
r
e
d
ictio
n
o
f
d
ef
ec
t
-
in
clin
ed
s
o
f
twar
e
ass
is
ts
wit
h
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ad
v
Ap
p
l Sci
I
SS
N:
2
2
5
2
-
8
8
1
4
Dev
elo
p
men
t o
f so
ftw
a
r
e
d
efec
t p
r
ed
ictio
n
s
ystem
u
s
in
g
a
r
tifi
cia
l n
eu
r
a
l n
etw
o
r
k
(
Ola
tu
n
j
i B
.
L.
)
285
co
o
r
d
in
atin
g
test
ef
f
o
r
t,
to
d
e
cr
ea
s
e
co
s
ts
,
to
im
p
r
o
v
e
th
e
s
o
f
twar
e
test
in
g
p
r
o
ce
s
s
b
y
f
o
cu
s
in
g
o
n
d
ef
ec
t
-
in
clin
ed
m
o
d
u
les [
9
]
,
last
ly
to
m
ak
e
th
e
n
at
u
r
e
o
f
th
e
s
o
f
twar
e
b
etter
[
1
0
]
.
T
h
at
is
th
e
r
ea
s
o
n
to
d
ay
'
s
s
o
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
is
a
s
ig
n
if
ican
t
ex
am
in
atio
n
s
u
b
je
ct
in
th
e
s
o
f
twar
e
en
g
in
ee
r
in
g
f
ield
[
1
1
]
.
So
f
t
war
e
d
ef
ec
t
p
r
ed
ictio
n
is
a
k
ey
p
r
o
ce
d
u
r
e
in
s
o
f
twar
e
e
n
g
in
ee
r
in
g
to
m
ak
e
th
e
q
u
ality
an
d
af
f
ir
m
atio
n
o
f
s
o
f
twar
e
b
etter
in
less
tim
e
an
d
at
least
ex
p
en
s
e
[
1
2
]
.
I
t
i
s
ac
tu
alize
d
b
ef
o
r
e
th
e
test
in
g
p
h
ase
o
f
t
h
e
s
o
f
tw
ar
e
ad
v
an
ce
m
en
t
life
cy
cle.
So
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
s
y
s
tem
s
g
iv
e
d
ef
ec
ts
o
r
v
ar
io
u
s
d
ef
ec
ts
.
T
h
e
s
o
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
h
as
b
ee
n
r
o
u
s
ed
b
y
v
ar
io
u
s
an
aly
s
ts
to
g
iv
e
a
d
if
f
er
en
t
s
y
s
tem
in
s
id
e
a
task
o
r
cr
o
s
s
-
u
n
d
er
ta
k
in
g
to
im
p
r
o
v
e
d
if
f
er
en
t
q
u
ality
an
d
watc
h
in
g
af
f
ir
m
atio
n
o
f
s
o
f
tw
ar
e
[
1
2
]
.
T
h
e
r
e
ar
e
two
way
s
to
d
ea
l
with
b
u
ild
s
a
s
o
f
twar
e
d
e
f
ec
t
p
r
e
d
ictio
n
s
y
s
tem
lik
e
s
u
p
er
v
is
ed
lear
n
in
g
an
d
u
n
s
u
p
er
v
is
ed
lear
n
in
g
.
Su
p
er
v
is
ed
lear
n
in
g
h
as
an
is
s
u
e
o
f
r
eq
u
ir
in
g
h
is
t
o
r
ical
in
f
o
r
m
atio
n
to
p
r
ep
ar
e
th
e
s
o
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
s
y
s
tem
wh
ile
u
n
s
u
p
er
v
is
ed
lear
n
i
n
g
d
o
esn
'
t
r
eq
u
ir
e
h
is
to
r
ica
l
in
f
o
r
m
atio
n
o
r
s
o
m
e
k
n
o
wn
o
u
tco
m
es
[
2
]
.
T
h
e
im
p
r
o
v
em
en
t
o
f
s
o
f
twar
e
tech
n
o
lo
g
y
c
au
s
es
an
ex
p
a
n
s
io
n
in
th
e
n
u
m
b
er
o
f
s
o
f
twar
e
item
s
,
an
d
th
eir
s
u
p
p
o
r
t
h
as
b
ec
o
m
e
a
d
if
f
icu
lt
ass
ig
n
m
en
t.
B
esid
es,
h
alf
o
f
th
e
life
cy
cle
co
s
t
f
o
r
a
s
o
f
twar
e
s
y
s
tem
in
co
r
p
o
r
at
es
u
p
k
ee
p
e
x
er
cises
.
W
ith
th
e
ascen
t
in
co
m
p
lex
ity
in
s
o
f
twar
e
s
y
s
tem
s
,
th
e
lik
elih
o
o
d
o
f
h
av
in
g
d
ef
ec
tiv
e
m
o
d
u
les
in
t
h
e
s
o
f
twar
e
s
y
s
tem
s
is
g
ettin
g
h
ig
h
er
[
1
3
]
.
A
k
ey
f
o
cu
s
,
d
e
f
ec
t
p
r
ed
ictio
n
,
h
as
r
is
en
as
a
f
u
n
ctio
n
in
g
ex
am
i
n
atio
n
zo
n
e
f
o
r
d
ec
a
d
es.
Def
e
c
t
p
r
ed
ictio
n
m
et
h
o
d
s
b
u
ild
s
y
s
tem
s
d
ep
en
d
en
t
o
n
d
if
f
er
en
t
s
o
r
ts
o
f
m
etr
ics
an
d
f
o
r
esee
d
ef
ec
ts
at
d
if
f
er
en
t
g
r
an
u
lar
ity
lev
els,
e.
g
.
,
c
h
an
g
e,
file,
o
r
m
o
d
u
le
lev
els
[
1
4
]
.
T
h
ese
p
r
o
ce
d
u
r
es
ca
n
b
e
u
tili
ze
d
to
ef
f
ec
tiv
el
y
ap
p
o
r
tio
n
q
u
ality
co
n
f
ir
m
at
io
n
ass
et
s
.
I
n
s
p
ite
o
f
v
ar
i
o
u
s
d
ef
ec
ts
,
p
r
ed
i
ctio
n
co
n
tem
p
lates
r
esear
ch
o
n
d
ef
ec
t
p
r
ed
ictio
n
d
esp
ite
ev
er
y
th
in
g
in
cr
em
en
ts
ex
p
o
n
en
tially
.
T
en
d
in
g
to
th
is
is
s
u
e
ca
n
g
i
v
e
k
n
o
wled
g
e
t
o
th
e
two
ex
p
er
ts
an
d
s
cien
tis
ts
.
E
x
p
er
ts
ca
n
u
tili
ze
o
b
s
er
v
atio
n
al
p
r
o
o
f
o
n
d
ef
ec
t
p
r
ed
ictio
n
to
s
ettle
o
n
in
f
o
r
m
e
d
ch
o
ices
ab
o
u
t
wh
en
to
u
tili
z
e
d
ef
ec
t
p
r
ed
ictio
n
an
d
h
o
w
it
wo
u
ld
b
est
fit
in
to
th
eir
ad
v
an
ce
m
e
n
t
p
r
o
ce
d
u
r
e.
Sp
ec
ialis
ts
ca
n
im
p
r
o
v
e
d
ef
ec
t
p
r
e
d
ictio
n
p
r
o
ce
d
u
r
es
d
ep
en
d
en
t
o
n
th
e
d
esire
s
f
o
r
p
r
o
f
ess
io
n
als
an
d
ap
p
r
o
p
r
iatio
n
ch
allen
g
es
th
at
th
e
y
f
ac
e.
T
o
p
ick
u
p
b
its
o
f
k
n
o
wled
g
e
in
to
t
h
e
r
ea
s
o
n
ab
le
esti
m
atio
n
o
f
d
e
f
ec
t
p
r
ed
ictio
n
,
a
q
u
a
n
titativ
e
r
ep
o
r
t
was
p
er
f
o
r
m
ed
in
th
is
ex
am
in
atio
n
s
o
as to
h
elp
s
o
f
twar
e
d
esig
n
er
s
with
th
e
er
r
an
d
o
f
c
o
m
p
r
eh
en
s
io
n
,
ass
ess
in
g
,
an
d
im
p
r
o
v
i
ng
th
eir
s
o
f
twar
e
item
s
.
I
t
is
im
p
er
ativ
e
to
p
r
e
d
ict
an
d
f
i
x
th
e
d
ef
ec
ts
b
ef
o
r
e
it
is
co
n
v
ey
ed
t
o
clien
ts
in
lig
h
t
o
f
th
e
f
ac
t
th
at
th
e
s
o
f
twar
e
q
u
ality
co
n
f
ir
m
atio
n
is
a
ted
i
o
u
s
task
an
d
n
o
w
a
n
d
a
g
ain
d
o
esn
'
t
tak
e
in
t
o
co
n
s
id
er
atio
n
co
m
p
lete
test
in
g
o
f
th
e
wh
o
le
s
y
s
tem
b
ec
au
s
e
o
f
s
p
en
d
in
g
is
s
u
es.
T
h
er
e
ar
e
n
u
m
er
o
u
s
o
p
e
n
d
atasets
th
at
ar
e
ac
ce
s
s
ib
le
f
r
ee
f
o
r
s
p
ec
ialis
ts
lik
e
PR
OM
I
SE,
E
C
L
I
PS
E
,
an
d
APAC
HE
to
co
n
q
u
er
th
e
d
if
f
icu
lt
is
s
u
e
wh
en
p
r
ep
a
r
in
g
p
er
f
o
r
m
e
d
o
n
a
n
o
th
er
p
r
o
ject.
An
aly
s
ts
h
av
e
b
ee
n
cr
e
atin
g
en
th
u
s
iasm
to
b
u
ild
a
cr
o
s
s
-
p
r
o
ject
d
ef
ec
t
p
r
ed
ictio
n
s
y
s
tem
with
v
a
r
io
u
s
m
etr
ics
s
et
lik
e
class
-
lev
el
m
etr
ics,
p
r
o
ce
s
s
m
etr
ics,
s
tatic
co
d
e
m
etr
ics
y
et
th
ey
co
u
ld
n
'
t
b
u
ild
in
cr
ea
s
i
n
g
ly
f
ea
s
ib
le
s
y
s
tem
s
[
1
2
]
.
T
h
er
e
ar
e
n
u
m
er
o
u
s
class
if
ier
s
o
r
lear
n
in
g
alg
o
r
ith
m
to
ch
o
o
s
e
a
wid
e
ass
o
r
tm
en
t
o
f
s
o
f
twar
e
m
etr
ics
lik
e
Naiv
e
B
ay
es,
Su
p
p
o
r
t
Vec
to
r
Ma
ch
in
e,
K
-
Nea
r
est
Neig
h
b
o
r
,
R
an
d
o
m
Fo
r
est,
Dec
is
io
n
T
r
ee
,
Neu
r
al
Netwo
r
k
an
d
L
o
g
is
tic
R
eg
r
ess
io
n
.
Hen
ce
,
in
th
is
p
a
p
er
a
s
o
f
twar
e
d
e
f
ec
t
p
r
ed
icti
o
n
s
y
s
tem
was
d
e
v
elo
p
e
d
u
s
i
n
g
Ar
tific
ial
Neu
r
al
Netwo
r
k
as
th
e
clas
s
if
y
in
g
alg
o
r
ith
m
an
d
with
th
e
u
s
e
o
f
Gen
etic
Alg
o
r
ith
m
th
e
p
o
s
s
ib
ilit
y
o
f
o
v
er
f
itti
n
g
was
elim
in
ated
b
y
e
x
tr
ac
tin
g
th
e
r
elev
an
t
f
ea
t
u
r
es
f
r
o
m
th
e
o
r
ig
in
al
d
atasets
wh
ich
th
e
o
u
tco
m
es
g
iv
e
b
est
p
r
ed
ictiv
e
p
e
r
f
o
r
m
an
ce
.
2.
RE
L
AT
E
D
WO
RK
Fen
to
n
an
d
Neil
[
1
5
]
,
m
ak
e
u
tili
za
tio
n
o
f
B
ay
esian
n
etwo
r
k
s
f
o
r
f
o
r
ec
asti
n
g
o
f
u
n
wav
e
r
in
g
q
u
ality
an
d
d
e
f
ec
tiv
en
ess
o
f
s
o
f
twar
e
.
I
t
m
ak
es
u
tili
za
tio
n
o
f
ca
s
u
al
p
r
o
ce
s
s
f
ac
to
r
s
an
d
q
u
alitativ
e
an
d
q
u
a
n
titativ
e
m
ea
s
u
r
es,
in
th
is
m
an
n
er
t
ak
in
g
in
to
ac
co
u
n
t
th
e
co
n
s
tr
ain
ts
o
f
tr
ad
itio
n
al
s
o
f
twar
e
im
p
ed
im
en
ts
.
T
h
e
u
tili
za
tio
n
o
f
a
p
o
wer
f
u
l
d
is
cr
etiza
tio
n
m
eth
o
d
b
r
i
n
g
s
ab
o
u
t
a
b
etter
p
r
ed
ictio
n
s
y
s
tem
f
o
r
s
o
f
twar
e
d
ef
ec
ts
.
J
ie
et
al.
[
1
6
]
,
m
ak
e
u
t
ilizatio
n
o
f
d
if
f
er
e
n
t statis
tical
p
r
o
ce
d
u
r
es
,
an
d
m
ac
h
in
e
lear
n
in
g
m
eth
o
d
s
wer
e
u
tili
ze
d
to
v
er
if
y
th
e
v
alid
ity
o
f
s
o
f
twar
e
d
ef
ec
t
p
r
e
d
ictio
n
s
y
s
tem
s
.
I
n
th
is
in
v
esti
g
ati
o
n
,
th
e
n
eu
r
o
-
f
u
zz
y
m
eth
o
d
was
th
o
u
g
h
t
o
f
.
T
h
e
d
ata
f
r
o
m
I
SB
SG
wer
e
tak
en
to
ac
h
iev
e
th
e
r
esear
ch
.
Ma
n
u
[
1
7
]
,
m
ak
e
u
tili
za
tio
n
o
f
an
o
th
er
co
m
p
u
t
atio
n
al
in
s
ig
h
t
s
eq
u
en
tial
h
y
b
r
id
d
esig
n
in
clu
d
in
g
Gen
etic
Pro
g
r
am
m
in
g
(
GP)
an
d
Gr
o
u
p
Me
th
o
d
o
f
Data
H
an
d
lin
g
(
GM
DH)
v
iz.
T
h
e
GPGMD
H
h
as
b
ee
n
co
n
tem
p
late
d
.
B
e
th
at
as
it
m
ay
,
th
e
GP a
n
d
GM
DH,
a
lar
g
e
g
r
o
u
p
o
f
m
eth
o
d
s
o
n
t
h
e
I
SB
SG d
ataset
h
av
e
b
ee
n
t
r
ied
.
T
h
e
GP
-
GM
DH
an
d
GM
DH
-
GP
h
y
b
r
id
s
s
u
r
p
ass
all
o
th
er
i
n
d
ep
en
d
en
t
a
n
d
h
y
b
r
id
p
r
o
ce
d
u
r
es.
I
t
is
p
r
esu
m
ed
th
at
th
e
GPGMD
H
o
r
GM
DH
-
GP
s
y
s
tem
i
s
th
e
g
r
ea
test
s
y
s
tem
am
o
n
g
all
d
if
f
er
en
t
m
eth
o
d
s
f
o
r
s
o
f
twar
e
co
s
t
esti
m
atio
n
.
Pu
n
ee
t
an
d
Pallav
i
[
1
8
]
u
til
ize
d
d
if
f
er
e
n
t
d
ata
m
in
in
g
s
tr
ateg
ies
f
o
r
s
o
f
twar
e
m
is
tak
e
p
r
ed
ictio
n
,
lik
e
af
f
iliatio
n
m
in
in
g
,
class
if
icatio
n
,
an
d
clu
s
ter
in
g
m
et
h
o
d
s
.
T
h
is
h
as
h
elp
ed
th
e
s
o
f
twar
e
en
g
in
ee
r
s
in
g
r
o
win
g
b
etter
s
y
s
tem
s
.
Fo
r
a
s
itu
atio
n
wh
e
r
e
d
e
f
ec
t
m
ar
k
s
ar
e
ab
s
en
t,
u
n
s
u
p
e
r
v
is
ed
p
r
o
ce
d
u
r
es c
an
b
e
u
tili
ze
d
f
o
r
s
y
s
tem
ad
v
an
ce
m
en
t.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
: 2
2
5
2
-
8
8
1
4
I
n
t J Ad
v
Ap
p
l Sci
,
Vo
l.
9
,
No
.
4
,
Dec
em
b
e
r
2
0
2
0
:
2
8
4
–
2
9
3
286
I
n
2
0
1
4
,
Ma
ttias
an
d
Alex
an
d
er
wo
r
k
ed
o
n
s
o
f
twar
e
d
ef
e
ct
p
r
ed
ictio
n
u
tili
zin
g
m
ac
h
in
e
lear
n
in
g
(
R
an
d
o
m
Fo
r
est
an
d
J
4
6
)
o
n
t
est
an
d
s
o
u
r
ce
c
o
d
e
m
etr
ics.
T
h
e
g
o
al
o
f
t
h
e
p
r
o
p
o
s
al
was
to
ex
p
l
o
r
e
w
h
eth
er
a
test
,
co
m
b
in
ed
with
a
s
o
u
r
ce
co
d
e
f
ile
co
n
tai
n
ed
en
o
u
g
h
in
f
o
r
m
atio
n
to
u
p
g
r
ad
e
t
h
e
s
o
f
twar
e
d
ef
ec
t
p
er
f
o
r
m
an
ce
if
m
etr
ics
f
r
o
m
b
o
th
s
o
u
r
ce
f
iles
an
d
test
f
iles
ar
e
jo
in
ed
.
Gr
ay
et
al.
[
1
9
]
p
r
o
p
o
s
ed
an
in
v
esti
g
atio
n
u
tili
zin
g
t
h
e
s
tatic
co
d
e
m
etr
ics
f
o
r
a
g
r
o
u
p
o
f
m
o
d
u
les
co
n
tain
ed
i
n
s
id
e
elev
en
NASA
d
ata
s
et
s
an
d
m
a
k
e
u
tili
za
tio
n
o
f
a
Su
p
p
o
r
t
Vec
to
r
Ma
ch
i
n
e
class
if
ier
.
A
ca
r
ef
u
l
p
r
o
g
r
ess
io
n
o
f
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
s
tag
e
was
ap
p
lied
to
th
e
d
ata
b
ef
o
r
e
class
if
icatio
n
,
in
clu
d
i
n
g
th
e
b
alan
cin
g
o
f
th
e
two
c
lass
es
(
d
ef
ec
tiv
e
o
r
s
o
m
eth
in
g
else)
an
d
th
e
d
is
m
is
s
al
o
f
co
u
n
tles
s
r
eh
ash
in
g
e
v
en
ts
.
T
h
e
Su
p
p
o
r
t
Vec
to
r
M
ac
h
in
e
in
th
is
tr
ial
y
ield
s
a
n
o
r
m
al
ac
cu
r
ac
y
o
f
7
0
%
o
n
p
r
ev
io
u
s
ly
in
co
n
s
p
icu
o
u
s
d
ata.
Acc
o
r
d
in
g
to
th
e
r
ev
i
ewe
d
r
elate
d
wo
r
k
s
,
it
is
o
b
s
er
v
ed
th
at
th
e
p
r
ev
i
o
u
s
ly
d
e
v
elo
p
e
d
s
o
f
twar
e
p
r
ed
ictio
n
s
y
s
tem
s
h
av
e
a
li
m
itatio
n
o
f
o
v
er
f
itti
n
g
wh
ich
h
ap
p
en
s
wh
en
t
h
e
s
y
s
tem
ac
q
u
ir
e
t
h
e
d
etail
in
t
h
e
tr
ain
in
g
d
ata
t
o
th
e
e
x
ten
t
th
at
i
t
n
eg
ativ
ely
ef
f
ec
ts
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
s
y
s
tem
o
n
n
ew
d
ata.
3.
RE
S
E
ARCH
M
E
T
H
O
D
T
h
e
ar
ch
itectu
r
e
o
f
th
e
d
e
v
elo
p
ed
s
y
s
tem
in
th
is
p
ap
er
is
p
r
esen
ted
in
Fig
u
r
e
1
.
T
h
e
f
o
llo
win
g
ar
e
th
e
s
tag
es th
at
wer
e
ad
o
p
ted
in
th
is
p
ap
er
:
i.
T
h
e
f
ir
s
t
s
tag
e
is
ac
q
u
is
itio
n
o
f
d
ata.
T
h
is
s
tag
e
in
v
o
lv
es
g
at
h
er
in
g
n
ec
ess
ar
y
d
atasets
wh
ich
wer
e
u
s
ed
in
th
is
p
a
p
er
.
Ho
wev
er
,
th
e
d
atasets
wer
e
ac
q
u
ir
e
d
f
r
o
m
h
ttp
://b
u
g
.
i
n
f
.
u
s
i.c
h
/d
o
w
n
lo
ad
.
p
h
p
wh
ich
is
p
u
b
licly
av
ailab
le
f
o
r
u
s
e.
ii.
T
h
e
n
ex
t
s
tag
e
is
th
e
f
ea
tu
r
e
s
elec
tio
n
s
tag
e
wh
ich
was
ac
h
iev
ed
b
y
u
s
in
g
Gen
etic
Alg
o
r
ith
m
s
o
as
to
ex
tr
ac
t th
e
r
elev
a
n
t f
ea
tu
r
es f
r
o
m
th
e
d
atasets
ac
q
u
ir
ed
in
th
e
f
ir
s
t stag
e.
iii.
I
n
th
e
class
if
icat
io
n
s
tag
e,
th
e
ex
tr
ac
ted
f
ea
tu
r
es we
r
e
class
if
ied
u
s
in
g
Ar
tific
ial
Neu
r
al
Net
wo
r
k
.
iv
.
Fin
ally
,
th
e
r
esu
lts
o
f
th
is
wo
r
k
wer
e
ev
alu
ated
u
s
in
g
ac
c
u
r
a
cy
,
p
r
ec
is
io
n
,
r
ec
all
an
d
f
-
s
co
r
e.
Fig
u
r
e
1.
Ar
c
h
itectu
r
e
o
f
th
e
d
ev
elo
p
ed
s
y
s
tem
.
3
.
1
.
Da
t
a
co
llect
io
n
So
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
r
esear
ch
d
ep
e
n
d
s
o
n
d
ata
th
at
m
u
s
t
b
e
g
ath
er
ed
f
r
o
m
in
an
y
ca
s
e
s
ep
ar
ate
s
to
r
es.
I
n
th
is
p
ap
er
,
th
e
d
atasets
wer
e
ac
q
u
ir
ed
f
r
o
m
h
ttp
://
b
u
g
.
in
f
.
u
s
i.c
h
/d
o
w
n
lo
ad
.
p
h
p
wh
ich
is
a
s
to
r
e
f
o
r
th
e
b
u
g
p
r
ed
ictio
n
d
ataset
f
o
r
m
o
s
t
o
p
en
-
s
o
u
r
ce
s
o
f
twar
e.
“T
h
e
E
clip
s
e
J
d
t
C
o
r
e
,
E
clip
s
e
Pd
e
Ui,
E
q
u
in
o
x
Fra
m
ewo
r
k
an
d
L
u
ce
n
e”
ar
e
t
h
e
s
o
f
twar
e
s
y
s
tem
s
th
at
wer
e
co
n
s
id
er
ed
in
th
is
p
ap
e
r
.
Ho
w
ev
er
,
ea
c
h
s
o
f
twar
e
s
y
s
tem
s
in
clu
d
es
d
if
f
er
en
t
p
iece
s
o
f
in
f
o
r
m
atio
n
b
u
t
in
th
is
p
ap
er
weig
h
ted
en
tr
o
p
y
m
o
d
u
le
co
d
en
am
ed
“we
ig
h
ted
.
en
t”
was
s
elec
ted
b
ec
au
s
e
it
h
as
m
o
s
t
f
am
iliar
p
ar
am
eter
s
lik
e
lin
es
o
f
co
d
e
wh
ich
s
u
ites
th
e
aim
o
f
d
ef
ec
t p
r
ed
ictio
n
s
y
s
tem
.
W
eig
h
te
d
en
tr
o
p
y
is
th
e
p
r
o
p
o
r
tio
n
o
f
d
ata
p
r
o
v
id
ed
b
y
a
p
r
o
b
ab
ilis
tic
tes
t w
h
o
s
e
b
asic o
cc
asio
n
s
ar
e
d
escr
ib
ed
b
y
b
o
t
h
th
eir
tar
g
et
p
r
o
b
ab
ilit
i
es a
n
d
b
y
s
o
m
e
s
u
b
jectiv
e
lo
a
d
s
.
3
.
2
.
F
ea
t
ure
s
elec
t
io
n
T
h
e
co
m
p
u
tatio
n
al
c
o
m
p
lex
it
y
o
f
s
o
m
e
o
f
th
e
p
r
ev
io
u
s
ly
m
en
tio
n
ed
m
a
ch
in
e
lear
n
in
g
alg
o
r
ith
m
s
m
ak
es
th
e
b
u
ild
in
g
o
f
th
e
s
y
s
tem
in
f
ea
s
ib
le
to
u
s
e
if
all
o
f
th
e
f
ea
t
u
r
es
in
t
h
e
d
ataset
is
u
s
ed
.
Alo
n
g
t
h
ese
lin
es,
f
ea
tu
r
e
s
elec
tio
n
was
u
tili
ze
d
to
r
em
o
v
e
a
lo
t
o
f
m
o
s
t
s
ig
n
if
ican
t
f
r
ee
f
ac
to
r
s
co
n
tain
ed
in
th
e
f
ir
s
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ad
v
Ap
p
l Sci
I
SS
N:
2
2
5
2
-
8
8
1
4
Dev
elo
p
men
t o
f so
ftw
a
r
e
d
efec
t p
r
ed
ictio
n
s
ystem
u
s
in
g
a
r
tifi
cia
l n
eu
r
a
l n
etw
o
r
k
(
Ola
tu
n
j
i B
.
L.
)
287
d
ataset
to
d
is
p
en
s
e
with
f
ac
t
o
r
s
th
at
wo
n
'
t
ad
d
to
th
e
p
r
esen
t
atio
n
o
f
p
r
e
d
ictio
n
,
at
th
at
p
o
i
n
t
im
p
r
o
v
e
lear
n
in
g
p
r
o
f
icien
c
y
an
d
in
c
r
em
en
t
p
r
e
d
ictio
n
ac
cu
r
ac
y
.
Ho
wev
er
,
in
th
is
p
ap
er
Gen
etic
Alg
o
r
ith
m
(
GA)
was
u
s
ed
f
o
r
ex
tr
ac
tin
g
th
e
r
elev
an
t
f
ea
tu
r
es
in
elim
in
atin
g
th
e
p
o
s
s
ib
i
lity
o
f
o
v
er
f
itti
n
g
.
GA
is
a
v
er
s
atile
h
eu
r
is
tic
tech
n
iq
u
e
f
o
r
wo
r
l
d
wid
e
ad
v
a
n
ce
m
en
t
lo
o
k
in
g
th
r
o
u
g
h
u
s
ed
to
cr
ea
te
v
alu
ab
le
a
n
s
wer
s
f
o
r
m
ac
h
in
e
lear
n
in
g
ap
p
licatio
n
s
an
d
it
r
ee
n
ac
ts
th
e
co
n
d
u
ct
o
f
th
e
d
ev
elo
p
m
en
t
p
r
o
ce
d
u
r
e
in
n
atu
r
e.
Fig
u
r
e
2
d
ep
icts
th
e
f
lo
wch
ar
t
o
f
a
ty
p
ical
GA.
T
h
e
f
ea
tu
r
e
was u
ltima
tely
r
ed
u
ce
d
u
s
in
g
t
h
e
f
itn
ess
f
u
n
ctio
n
;
∑
1
|
(
∑
(
)
)
−
(
)
|
=
1
=
1
(
1
)
wh
er
e
is
a
×
m
atr
ix
o
f
f
ea
tu
r
e
a
n
d
is
th
e
co
r
r
esp
o
n
d
in
g
o
u
tp
u
t.
3
.
3
.
Cla
s
s
if
ica
t
io
n sta
g
e
T
h
e
ex
tr
ac
ted
r
elev
an
t
f
ea
tu
r
e
was
d
iv
id
ed
in
t
o
f
o
l
d
s
an
d
e
n
s
u
r
e
th
at
ea
ch
f
o
ld
was
u
s
ed
as
test
in
g
s
et
at
s
o
m
e
p
o
in
t
an
d
u
s
ed
t
o
tr
ain
th
e
class
if
ier
.
K
-
f
o
ld
cr
o
s
s
v
alid
atio
n
was
a
d
o
p
te
d
wh
er
e
t
h
e
ac
q
u
ir
ed
d
atasets
was
d
iv
id
ed
in
to
a
k
n
u
m
b
er
o
f
f
o
ld
s
.
Ho
wev
er
,
s
i
n
ce
f
o
u
r
o
p
en
s
o
u
r
ce
s
o
f
twar
e
wer
e
co
n
s
id
er
ed
in
th
is
p
ap
er
th
e
d
atasets
was
d
i
v
id
ed
in
to
4
f
o
ld
s
.
I
n
th
e
p
r
i
m
ar
y
cy
cle,
th
e
p
r
i
n
cip
al
f
o
l
d
was
u
tili
ze
d
to
test
th
e
f
r
am
ewo
r
k
an
d
t
h
e
r
es
t
was
u
tili
ze
d
to
p
r
ep
ar
e
th
e
f
r
am
ewo
r
k
.
I
n
th
e
s
u
b
s
eq
u
en
t
e
m
p
h
asis
,
th
e
s
u
b
s
eq
u
e
n
t
f
o
ld
was u
tili
ze
d
as
th
e
test
in
g
s
et
wh
ile
th
e
r
est
f
ill
in
as
t
h
e
p
r
ep
ar
atio
n
s
et.
T
h
is
p
r
o
ce
s
s
was
r
ep
ea
ted
u
n
til
ea
ch
f
o
ld
o
f
th
e
4
f
o
ld
s
ar
e
b
ee
n
u
s
ed
as
th
e
t
esti
n
g
s
et.
T
h
e
s
y
s
tem
h
as
a
f
lo
w
in
wh
ich
ev
er
y
u
s
er
ca
n
f
o
llo
w.
T
h
is
also
ca
n
b
e
u
s
ed
in
s
o
f
twar
e
en
g
in
ee
r
i
n
g
f
ield
wh
e
n
m
ea
s
u
r
in
g
th
e
f
lo
w
an
d
q
u
ality
o
f
a
s
o
f
twar
e
ac
co
r
d
in
g
to
s
o
f
twar
e
m
etr
ics.
C
r
o
s
s
v
alid
atio
n
was
ad
o
p
ted
s
in
ce
th
e
am
o
u
n
t
o
f
d
ata
is
lim
ited
an
d
it
h
as
a
m
er
it
o
v
e
r
th
e
e
x
is
tin
g
tech
n
iq
u
e
ca
lled
h
o
ld
o
u
t
m
eth
o
d
.
I
n
t
h
e
h
o
ld
o
u
t
m
eth
o
d
,
o
n
e
p
a
r
t
o
f
th
e
d
atasets
is
u
s
ed
f
o
r
tr
ain
in
g
an
d
th
e
o
t
h
er
f
o
r
test
in
g
.
I
n
th
is
p
ap
er
,
th
e
s
o
lu
tio
n
to
th
e
b
ias
id
ea
was
ad
o
p
ted
u
s
in
g
cr
o
s
s
v
alid
atio
n
wh
er
e
all
th
e
in
s
tan
ce
s
wer
e
u
s
ed
o
n
e
tim
e
f
o
r
test
in
g
an
d
tr
ain
in
g
.
T
h
is
s
im
p
ly
m
ea
n
s
t
h
at,
in
s
t
ea
d
o
f
co
n
d
u
ctin
g
f
o
u
r
f
o
ld
s
,
a
to
tal
o
f
1
6
f
o
ld
s
is
g
en
er
ated
an
d
th
e
e
r
r
o
r
esti
m
ate
is
th
er
ef
o
r
e
m
o
r
e
r
eli
ab
le.
Hen
ce
,
Ar
tific
ial
Neu
r
al
Netwo
r
k
(
ANN)
was a
d
o
p
ted
in
th
e
class
if
icatio
n
s
tag
e
u
s
in
g
L
ev
en
b
er
g
-
Ma
r
q
u
ar
d
t
(
L
M)
Al
g
o
r
ith
m
to
tr
ai
n
t
h
e
ANN.
T
h
e
ch
o
ice
o
f
th
e
L
M
Alg
o
r
ith
m
in
th
is
p
ap
er
is
th
at
it
is
n
o
t
th
at
m
e
m
o
r
y
e
f
f
icien
t
b
u
t
f
aster
th
a
n
o
th
er
al
g
o
r
ith
m
s
.
I
t
ap
p
r
o
x
i
m
ates
th
e
b
lu
n
d
er
o
f
th
e
n
etwo
r
k
with
a
s
ec
o
n
d
-
o
r
d
er
ar
ticu
latio
n
wh
ich
d
iv
er
g
es
f
r
o
m
th
e
b
ac
k
-
p
r
o
p
a
g
atio
n
al
g
o
r
ith
m
th
at
d
o
es
it
with
a
f
ir
s
t
-
o
r
d
er
ar
ticu
latio
n
.
L
M
r
ef
r
esh
es th
e
ANN
lo
ad
s
as f
o
llo
ws:
∆
=
[
+
∑
(
)
(
)
=
1
]
−
1
∇
(
)
(
2
)
wh
er
e
(
)
is
th
e
J
ac
o
b
ian
m
atr
ix
o
f
th
e
e
r
r
o
r
v
ec
to
r
;
(
)
ev
alu
ated
in
w
a
n
d
is
th
e
id
en
tity
m
atr
ix
.
T
h
e
v
ec
to
r
er
r
o
r
(
)
is
th
e
er
r
o
r
o
f
th
e
n
etwo
r
k
f
o
r
p
atter
,
th
at
i
s
(
)
=
−
(
)
(
3
)
T
h
e
p
ar
am
eter
is
in
cr
ea
s
ed
o
r
d
ec
r
ea
s
ed
at
ea
ch
s
tep
.
I
f
th
e
er
r
o
r
is
r
ed
u
ce
d
,
th
en
is
d
iv
id
ed
b
y
a
f
ac
to
r
an
d
it
is
m
u
ltip
lied
b
y
in
o
th
er
ca
s
e.
L
M
p
er
f
o
r
m
s
th
e
s
tep
s
d
etailed
in
Alg
o
r
ith
m
1
.
I
t
ca
lcu
lates
th
e
n
etwo
r
k
o
u
tp
u
t,
th
e
er
r
o
r
v
ec
to
r
s
an
d
th
e
J
ac
o
b
ian
m
atr
ix
f
o
r
ea
ch
p
atter
n
.
T
h
en
,
it
co
m
p
u
tes
∆
u
s
in
g
eq
u
atio
n
2
an
d
r
ec
alcu
lates
t
h
e
er
r
o
r
with
+
∆
as
n
etwo
r
k
weig
h
ts
.
I
f
th
e
er
r
o
r
h
as
d
ec
r
e
ased
,
is
d
iv
id
ed
b
y
,
th
e
n
ew
weig
h
ts
ar
e
m
ain
tain
ed
a
n
d
th
e
p
r
o
ce
s
s
s
tar
ts
ag
ain
;
o
th
er
wis
e,
is
m
u
ltip
lied
b
y
.
∆
is
ca
lcu
lated
with
a
n
ew
v
alu
e
an
d
it iter
ates a
g
ain
[
2
0
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
: 2
2
5
2
-
8
8
1
4
I
n
t J Ad
v
Ap
p
l Sci
,
Vo
l.
9
,
No
.
4
,
Dec
em
b
e
r
2
0
2
0
:
2
8
4
–
2
9
3
288
Fig
u
r
e
2
.
T
h
e
f
lo
wch
a
r
t o
f
a
ty
p
ical
g
en
etic
alg
o
r
ith
m
Alg
o
r
ith
m
1:
Ps
eu
d
o
c
o
d
e
o
f
L
ev
en
b
er
g
-
Ma
r
q
u
ar
d
t
Initialize Weights;
While not stop Criterion do
Calculates
C
P
(
w
)
for each pattern
e
1
=
∑
=
1
e
P
(
w
)
T
e
P
(
w
)
P
P
Calculates
J
P
(
w
)
for each pattern
Repeat
Calculates
∆
w
e
2
=
∑
=
e
P
(
w
+
∆
w
)
T
e
P
(
w
+
∆
w
)
P
P
If
e
1
≤
e
2
then
μ
=
μ
∗
β
End If
Until
e
1
<
e
2
μ
=
μ
/
β
w
=
w
+
∆
w
End while
3
.
4
.
P
er
f
o
rma
nce
m
et
rics
I
n
o
r
d
er
to
m
ea
s
u
r
e
d
ef
ec
t
p
r
ed
ictio
n
r
esu
lts
b
y
class
if
icatio
n
m
o
d
els,
d
if
f
er
en
t
p
er
f
o
r
m
a
n
ce
m
ea
s
u
r
es a
r
e
av
ailab
le
f
o
r
ef
f
e
ctiv
en
ess
.
I
n
th
is
p
ap
er
,
th
e
f
o
llo
win
g
p
r
e
d
ictio
n
o
u
tco
m
es w
er
e
co
n
s
id
er
ed
:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ad
v
Ap
p
l Sci
I
SS
N:
2
2
5
2
-
8
8
1
4
Dev
elo
p
men
t o
f so
ftw
a
r
e
d
efec
t p
r
ed
ictio
n
s
ystem
u
s
in
g
a
r
tifi
cia
l n
eu
r
a
l n
etw
o
r
k
(
Ola
tu
n
j
i B
.
L.
)
289
i.
T
r
u
e
p
o
s
itiv
e
(
T
P):
b
u
g
g
y
in
s
t
an
ce
s
p
r
ed
icted
as b
u
g
g
y
ii.
Fals
e
p
o
s
itiv
e
(
FP
)
: c
lean
in
s
t
an
ce
s
p
r
ed
icted
as b
u
g
g
y
iii.
T
r
u
e
n
e
g
ativ
e
(
T
N)
: c
lean
in
s
tan
ce
s
p
r
ed
icted
as c
lean
iv
.
Fals
e
n
eg
ativ
e
(
FN)
: b
u
g
g
y
in
s
tan
ce
s
p
r
ed
icted
as c
lean
W
ith
th
ese
o
u
tco
m
es,
th
e
f
o
llo
win
g
m
ea
s
u
r
es
w
h
ich
ar
e
m
o
s
tly
u
s
ed
in
th
e
s
o
f
tw
ar
e
d
ef
ec
t
p
r
ed
ictio
n
liter
atu
r
e
ar
e
d
e
f
in
e
d
:
=
+
+
+
+
(
4
)
Acc
u
r
ac
y
th
in
k
s
ab
o
u
t
b
o
th
tr
u
e
p
o
s
itiv
es
an
d
tr
u
e
n
e
g
at
iv
es
o
v
er
all
o
cc
u
r
r
en
ce
s
.
A
s
it
wer
e,
ac
cu
r
ac
y
s
h
o
ws th
e
p
r
o
p
o
r
tio
n
o
f
all
ac
cu
r
ately
class
ified
ca
s
es.
=
+
(
5
)
=
+
(
6
)
R
ec
all
m
ea
s
u
r
es c
o
r
r
ec
tly
p
r
e
d
icted
b
u
g
g
y
i
n
s
tan
ce
s
am
o
n
g
all
b
u
g
g
y
in
s
tan
ce
s
.
−
=
2
×
(
×
)
+
(
7
)
F
-
m
ea
s
u
r
e
is
a
h
ar
m
o
n
ic
m
ea
n
o
f
p
r
e
cisi
o
n
an
d
r
ec
all.
B
y
co
llectin
g
th
ese
p
e
r
f
o
r
m
a
n
ce
m
ea
s
u
r
em
en
ts
,
f
u
tu
r
e
p
r
ed
icti
o
n
s
o
n
u
n
s
ee
n
f
iles
ca
n
b
e
esti
m
ated
.
T
h
e
ca
lcu
latio
n
o
f
ac
c
u
r
ac
y
,
p
r
ec
is
i
o
n
an
d
r
ec
all
m
ak
es u
s
e
o
f
th
e
co
n
f
u
s
io
n
m
atr
ix
.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
h
e
ex
p
er
im
en
t
was
co
n
d
u
cte
d
b
y
f
ir
s
t
ex
tr
ac
tin
g
th
e
r
elev
a
n
t
f
ea
tu
r
es
f
r
o
m
t
h
e
d
atasets
u
s
ed
in
t
h
is
r
esear
ch
as
d
is
cu
s
s
ed
in
s
ec
t
io
n
3
.
2
u
s
in
g
GA.
Ho
wev
er
,
weig
h
ted
-
en
t
d
ataset
h
as
1
7
f
ea
tu
r
es
ex
clu
d
in
g
th
e
class
n
am
es
an
d
with
th
e
ad
o
p
tio
n
o
f
th
e
GA,
th
e
f
ea
tu
r
es
ar
e
r
ed
u
ce
d
to
1
3
u
s
in
g
t
h
e
f
itn
ess
f
u
n
ctio
n
d
is
cu
s
s
ed
in
s
ec
tio
n
3
.
2
.
Fig
u
r
e
3
s
h
o
ws
th
e
g
r
ap
h
ical
u
s
er
i
n
ter
f
ac
e
o
f
th
e
GA
at
th
e
f
ea
t
u
r
e
s
elec
tio
n
s
tag
e.
U
s
in
g
th
e
m
ath
e
m
atica
l
f
o
r
m
u
las
d
is
cu
s
s
ed
in
s
ec
tio
n
3
.
4
,
th
e
v
alu
es
i
n
T
ab
le
1
ar
e
ca
lcu
lated
a
n
d
b
y
co
llectin
g
th
ese
p
er
f
o
r
m
an
ce
m
ea
s
u
r
em
en
ts
,
f
u
tu
r
e
p
r
e
d
ictio
n
s
o
n
u
n
s
ee
n
f
iles
ca
n
b
e
esti
m
ated
.
Acc
o
r
d
in
g
to
th
e
co
n
d
u
cted
e
x
p
er
im
en
ts
th
e
p
er
ce
n
ta
g
e
o
f
t
h
e
T
r
u
e
Po
s
itiv
e
R
ate
(
T
P
R
)
an
d
T
r
u
e
Neg
ativ
e
R
ate
(
T
NR
)
o
f
th
e
d
atasets
u
s
ed
in
th
is
r
esear
ch
wo
r
k
;
E
C
L
I
PS
E
J
DT
C
O
R
E
,
E
C
L
I
PS
E
PDE
UI
,
E
QUI
NOX
F
R
AM
E
W
OR
K
a
n
d
L
UC
E
NE
ar
e
(
7
9
.
3
1
%
an
d
8
8
.
2
4
%),
(
4
5
.
4
5
%
an
d
8
7
.
9
7
%),
(
4
5
.
4
5
%
an
d
7
3
.
8
1
%)
an
d
(
5
0
.
0
0
%
an
d
9
3
.
8
5
%)
r
e
s
p
ec
tiv
ely
.
T
h
e
tr
ain
i
n
g
an
d
v
alid
atio
n
f
o
r
th
e
d
atasets
E
C
L
I
PS
E
J
DT
C
OR
E
,
E
C
L
I
PS
E
PDE
UI
,
E
QUI
NOX
F
R
AM
E
W
OR
K
an
d
L
UC
E
NE
was
co
n
d
u
cted
.
Ho
wev
er
,
th
e
b
est
v
alid
atio
n
p
e
r
f
o
r
m
an
ce
is
0
.
5
2
4
8
2
at
ep
o
c
h
5
,
0
.
2
1
0
3
2
at
e
p
o
ch
5
,
0
.
6
7
5
2
7
at
ep
o
c
h
9
a
n
d
0
.
0
1
3
5
6
at
ep
o
ch
1
0
r
esp
ec
tiv
ely
.
Fig
u
r
es
4
,
Fi
g
u
r
e
5
,
Fig
u
r
e
6
an
d
Fig
u
r
e
7
s
h
o
ws
th
e
ch
ar
t
r
ep
r
esen
tatio
n
o
f
t
h
e
tr
ain
i
n
g
a
n
d
v
alid
atio
n
f
o
r
ea
ch
d
ataset
r
esp
ec
tiv
ely
.
Su
m
m
ar
ily
,
K
-
f
o
ld
v
alid
ati
o
n
m
eth
o
d
was
u
s
ed
to
v
ali
d
ate
th
e
d
ataset
wh
er
e
all
t
h
e
d
atasets
p
ar
tak
es
in
b
o
th
tr
ain
in
g
an
d
test
in
g
p
r
o
ce
s
s
as
d
is
cu
s
s
ed
in
Sectio
n
3
.
3
.
Mo
r
e
s
o
,
as
s
h
o
wn
in
T
ab
le
1
th
r
o
u
g
h
o
u
t
th
e
p
er
f
o
r
m
an
ce
m
ea
s
u
r
es
th
e
d
ataset
L
UC
E
NE
h
as
th
e
h
ig
h
est
ac
cu
r
ac
y
o
f
9
1
.
3
0
%
wh
ile
E
QUI
NOX
F
R
AM
E
W
OR
K
h
as
th
e
h
ig
h
est
p
r
ec
is
io
n
o
f
5
7
.
6
9
%
wh
ich
m
ea
s
u
r
es
h
o
w
g
o
o
d
th
e
p
r
ed
ictio
n
s
y
s
tem
is
at
id
en
tify
in
g
ac
tu
al
f
au
lty
f
iles
.
Fu
r
th
er
m
o
r
e,
r
ec
all
u
s
ed
in
th
is
r
esear
ch
m
ea
s
u
r
es
th
e
p
r
o
p
o
r
tio
n
o
f
f
au
lty
f
iles
wh
ich
ar
e
co
r
r
e
ctly
id
en
tifie
d
as
f
au
lty
wh
er
e
E
C
L
I
PS
E
J
DT
C
OR
E
h
as
th
e
h
ig
h
est
r
ec
all
o
f
7
9
.
3
1
% a
n
d
h
i
g
h
est F
-
Sco
r
e
o
f
6
3
.
8
9
%.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
: 2
2
5
2
-
8
8
1
4
I
n
t J Ad
v
Ap
p
l Sci
,
Vo
l.
9
,
No
.
4
,
Dec
em
b
e
r
2
0
2
0
:
2
8
4
–
2
9
3
290
Fig
u
r
e
3
.
Desk
to
p
in
ter
f
ac
e
o
f
th
e
f
ea
tu
r
e
s
elec
tio
n
s
tag
e
T
ab
le
1
.
Sy
s
tem
r
esu
lts
o
f
th
e
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
a
n
d
F
-
Sco
r
e
D
a
t
a
s
e
t
s
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F
-
S
c
o
r
e
EC
LI
P
S
E
JD
T
C
O
R
E
8
6
.
9
3
%
5
3
.
4
9
%
7
9
.
3
1
%
6
3
.
8
9
%
EC
LI
P
S
E
P
D
E
U
I
8
3
.
2
8
%
3
1
.
9
1
%
4
5
.
4
5
%
3
7
.
5
0
%
EQ
U
I
N
O
X
F
R
A
M
EWO
R
K
8
3
.
4
3
%
5
7
.
6
9
%
4
5
.
4
5
%
5
0
.
8
4
%
LU
C
EN
E
9
1
.
3
0
%
3
3
.
3
3
%
5
0
.
0
0
%
4
0
.
0
0
%
A
V
ER
A
G
E
8
6
.
2
4
%
4
4
.
1
1
%
5
5
.
0
5
%
4
8
.
0
6
%
Fig
u
r
e
4
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
f
o
r
E
C
L
I
PS
E
J
DT
C
O
R
E
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ad
v
Ap
p
l Sci
I
SS
N:
2
2
5
2
-
8
8
1
4
Dev
elo
p
men
t o
f so
ftw
a
r
e
d
efec
t p
r
ed
ictio
n
s
ystem
u
s
in
g
a
r
tifi
cia
l n
eu
r
a
l n
etw
o
r
k
(
Ola
tu
n
j
i B
.
L.
)
291
Fig
u
r
e
5
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
f
o
r
E
C
L
I
PS
E
PDE
UI
Fig
u
r
e
6
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
f
o
r
E
QUI
NOX
FR
AM
E
W
OR
K
Fig
u
r
e
7
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
f
o
r
L
UC
E
NE
T
o
ea
s
e
th
e
co
m
p
ar
is
o
n
t
o
t
h
e
r
elate
d
s
tu
d
y
,
th
e
a
v
er
ag
e
o
f
th
e
r
esu
lts
f
o
r
all
th
e
d
a
tasets
an
d
p
er
f
o
r
m
an
ce
m
ea
s
u
r
es
ar
e
p
r
e
s
en
ted
in
Fig
u
r
e
8
.
As
ac
cu
r
a
cy
is
d
e
p
en
d
e
n
t
o
n
t
h
e
b
alan
c
e
o
f
th
e
u
n
d
er
ly
in
g
d
ataset,
it
is
f
u
r
th
er
co
m
p
a
r
ed
to
th
e
av
er
ag
e
ac
c
u
r
a
c
y
r
esu
lt
o
f
th
e
r
elate
d
s
tu
d
y
.
[
2
1
]
Pro
p
o
s
ed
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
: 2
2
5
2
-
8
8
1
4
I
n
t J Ad
v
Ap
p
l Sci
,
Vo
l.
9
,
No
.
4
,
Dec
em
b
e
r
2
0
2
0
:
2
8
4
–
2
9
3
292
a
C
o
n
Pre
d
icto
r
s
y
s
tem
to
p
r
ed
ict
d
ef
ec
ts
s
p
ec
if
ic
to
c
o
n
cu
r
r
en
t
p
r
o
g
r
am
s
b
y
co
m
b
in
i
n
g
b
o
th
s
tatic
an
d
d
y
n
am
ic
p
r
o
g
r
a
m
m
etr
ics.
As
th
is
r
esear
ch
is
co
n
d
u
cted
u
s
in
g
th
e
s
am
e
p
er
f
o
r
m
an
ce
m
ea
s
u
r
es
as
[
2
1
]
an
d
as
t
h
ey
s
u
m
m
ar
ize
m
a
n
y
s
tu
d
ies,
th
e
r
esu
lts
o
f
th
is
s
tu
d
y
ar
e
co
m
p
ar
ed
to
o
n
es c
o
m
p
iled
b
y
[
2
1
]
.
Fig
u
r
e
8
.
C
o
n
Pre
d
icto
r
co
m
p
a
r
is
o
n
r
esu
lts
ag
ain
s
t th
e
d
ev
el
o
p
ed
s
y
s
tem
5.
CO
NCLU
SI
O
N
T
h
e
d
e
v
elo
p
m
e
n
t
o
f
s
o
f
twa
r
e
p
r
o
d
u
ct
is
in
cr
ea
s
in
g
ex
p
o
n
en
tially
d
u
e
to
th
e
ir
b
e
n
ef
its
an
d
o
cc
u
r
r
e
n
ce
o
f
d
ef
ec
ts
in
th
e
s
o
f
twar
e
p
r
o
d
u
cts
is
in
ev
itab
le.
I
n
o
th
er
wo
r
d
s
,
th
is
d
ef
ec
t
n
ee
d
s
to
b
e
r
ed
u
ce
d
to
m
in
im
u
m
co
u
n
t.
So
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
e
f
f
ec
tiv
ely
im
p
r
o
v
e
th
e
q
u
ality
an
d
ef
f
icien
cy
o
f
s
o
f
twar
e
w
h
ich
en
h
an
ce
s
th
e
p
r
o
ce
d
u
r
e
o
f
f
o
ll
o
win
g
d
ef
ec
tiv
e
p
ar
ts
in
s
o
f
twar
e
p
r
ec
ed
in
g
th
e
b
e
g
in
n
in
g
o
f
th
e
test
in
g
s
tag
e.
Ho
wev
er
,
s
o
m
e
class
if
icatio
n
tech
n
iq
u
es
s
u
ch
as
Naïv
e
B
ay
es,
r
an
d
o
m
f
o
r
est
a
n
d
d
ec
i
s
io
n
tr
ee
h
as
b
ee
n
ad
o
p
ted
f
o
r
s
o
f
twar
e
d
ef
ec
t
p
r
ed
ictio
n
ac
co
r
d
in
g
to
liter
atu
r
e.
Hen
ce
,
in
th
is
p
ap
e
r
GA
was
s
u
cc
ess
f
u
lly
u
s
ed
f
o
r
f
ea
tu
r
e
s
elec
tio
n
alo
n
g
s
id
e
ANN
in
p
r
ed
ictin
g
th
e
d
ef
ec
ti
v
e
m
o
d
u
les in
a
s
o
f
twar
e
s
y
s
tem
.
T
h
is
d
ev
elo
p
ed
s
y
s
tem
was
co
m
p
ar
e
d
with
ex
is
tin
g
s
y
s
tem
wh
ich
at
t
h
e
co
m
p
letio
n
o
f
th
e
c
o
n
d
u
c
ted
ex
p
er
im
en
ts
it
out
s
h
in
es th
e
ex
is
tin
g
s
y
s
tem
b
y
g
iv
i
n
g
a
b
est p
r
ed
ictiv
e
p
e
r
f
o
r
m
an
ce
.
RE
F
E
R
E
NC
E
S
[1
]
Na
ik
,
K.
a
n
d
Tri
p
a
th
y
,
P
.
,
“
S
o
ftw
a
re
Tes
ti
n
g
a
n
d
Qu
a
li
t
y
As
su
ra
n
c
e
,”
J
o
h
n
W
il
e
y
&
S
o
n
s,
In
c
.
p
p
.
2
9
-
32
,
2
0
0
8
.
[2
]
S
a
n
u
si
B.
A.,
Ola
b
i
y
isi
S
.
O.,
Olo
wo
y
e
A.
O.
a
n
d
Ola
t
u
n
ji
B.
L.
,
“
S
o
ftwa
re
De
fe
c
t
P
re
d
ictio
n
S
y
ste
m
u
sin
g
M
a
c
h
in
e
Lea
rn
in
g
b
a
se
d
Al
g
o
ri
th
m
s
,”
J
o
u
rn
a
l
o
f
A
d
v
a
n
c
e
s
in
C
o
m
p
u
t
a
ti
o
n
a
l
In
tell
ig
e
n
c
e
T
h
e
o
ry
,
v
o
l.
1
,
n
o
.
3,
p
p
.
1
-
9
,
2
0
1
9
.
[3
]
M
c
Do
n
a
ld
M
.
,
M
u
ss
o
n
,
R
.
a
n
d
S
m
it
h
,
R.
,
“
Th
e
p
ra
c
ti
c
a
l
g
u
i
d
e
to
d
e
fe
c
t
p
re
v
e
n
ti
o
n
,
”
Co
n
tro
l
,
p
p
.
2
6
0
-
2
7
2
,
2
0
0
7
.
[4
]
Ku
m
a
re
sh
,
S
.
a
n
d
Ba
sk
a
ra
n
,
R.
,
“
De
fe
c
t
a
n
a
ly
sis
a
n
d
p
re
v
e
n
ti
o
n
fo
r
s
o
ftwa
re
p
ro
c
e
ss
q
u
a
li
ty
i
m
p
ro
v
e
m
e
n
ts,”
In
ter
n
a
t
io
n
a
l
J
o
u
rn
a
l
o
f
C
o
mp
u
ter
Ap
p
l
ica
ti
o
n
s
,
v
ol
.
8
,
no.
9
,
p
p
.
2
5
0
-
2
5
4
,
2
0
1
0
.
[5
]
Kitch
e
n
h
a
m
,
B.
A.,
”
G
u
id
e
li
n
e
s
fo
r
P
e
rfo
rm
in
g
S
y
ste
m
a
ti
c
Li
tera
tu
re
Re
v
iew
in
S
o
ftwa
re
En
g
in
e
e
r
in
g
,”
T
e
c
h
n
ica
l
Rep
o
rt E
B
S
E
-
2
0
0
7
-
0
0
1
,
Ke
e
le U
n
ive
rs
it
y
a
n
d
Du
rh
a
m
Un
ive
rs
it
y
,
S
ta
ff
o
r
d
sh
ire
,
p
p
.
1
8
4
-
2
1
0
,
2
0
0
7
.
[6
]
Ca
tal,
C.
a
n
d
Diri,
B.
,
“
A
S
y
ste
m
a
ti
c
Re
v
iew
o
f
S
o
ftwa
re
F
a
u
lt
P
r
e
d
ictio
n
st
u
d
ies
,”
Exp
e
rt
S
y
st.
A
p
p
l.
,
v
o
l.
3
6
,
p
p
.
7
3
4
6
–
7
3
5
4
,
2
0
0
9
.
[7
]
Ra
d
jen
o
v
ic,
D.,
He
rick
o
,
M
.
,
T
o
rk
a
r,
R.
a
n
d
Z
iv
k
o
v
ic
,
A.,
“
S
o
ftwa
re
fa
u
lt
p
re
d
ictio
n
m
e
tri
c
s:
A
S
y
ste
m
a
ti
c
li
tera
tu
re
re
v
i
e
w,”
In
f.
S
o
ft
w
.
T
e
c
h
n
o
l
,
v
o
l.
5
5
,
p
p
.
1
3
9
7
-
1
4
1
8
,
2
0
1
3
.
[8
]
M
e
n
z
ies
,
T.
,
M
il
t
o
n
,
Z
.
,
Tu
r
h
a
n
,
B.
,
Cu
k
ic
,
B.
,
Jia
n
g
,
Y.
,
a
n
d
Be
n
e
r,
A.,
“
De
fe
c
t
p
re
d
ictio
n
fr
o
m
sta
ti
c
c
o
d
e
fe
a
tu
re
s:
c
u
rre
n
t
re
su
l
ts,
li
m
it
a
ti
o
n
s,
n
e
w
a
p
p
ro
a
c
h
e
s,”
A
u
to
m
a
ted
S
o
f
twa
re
En
g
i
n
e
e
rin
g
,
v
o
l.
17
,
n
o
.
4
,
p
p
.
3
7
5
-
407
,
2
0
1
0
.
[9
]
Ca
ta
l,
C.
,
S
e
v
im,
U.,
a
n
d
Diri
,
B.
,
“
P
ra
c
ti
c
a
l
d
e
v
e
lo
p
m
e
n
t
o
f
a
n
E
c
li
p
se
-
b
a
se
d
so
ftwa
re
fa
u
lt
p
re
d
ic
ti
o
n
t
o
o
l
u
sin
g
Na
iv
e
Ba
y
e
s a
lg
o
rit
h
m
,
”
Exp
e
rt S
y
ste
ms
wit
h
Ap
p
li
c
a
ti
o
n
s
,
v
o
l.
38
,
n
o
.
3
,
p
p
.
2
3
4
7
-
2
3
5
3
,
2
0
1
1
.
[1
0
]
Ha
ll
,
T.
,
Be
e
c
h
a
m
,
S
.
,
Bo
we
s,
D.
,
G
ra
y
,
D.,
a
n
d
C
o
u
n
se
ll
,
S
.
,
“
A
S
y
ste
m
a
ti
c
Li
tera
tu
re
Re
v
iew
o
n
F
a
u
lt
P
re
d
ictio
n
P
e
rf
o
rm
a
n
c
e
in
S
o
f
twa
re
En
g
in
e
e
rin
g
,
”
IEE
E
T
ra
n
sa
c
ti
o
n
s
o
n
S
o
ft
wa
re
En
g
in
e
e
rin
g
,
v
o
l.
38
,
n
o
.
6
,
p
p
.
1
2
7
6
-
1
3
0
4
,
2
0
1
2
.
[1
1
]
S
o
n
g
,
Q.,
Jia
,
Z.
,
S
h
e
p
p
e
rd
,
M
.
,
Y
in
g
,
S
.
,
a
n
d
Li
u
,
J.,
“
A
G
e
n
e
ra
l
S
o
ftwa
re
De
fe
c
t
-
P
ro
n
e
n
e
ss
P
re
d
ictio
n
F
ra
m
e
wo
rk
,
”
IEE
E
T
ra
n
sa
c
ti
o
n
s
o
n
S
o
ft
wa
re
E
n
g
i
n
e
e
rin
g
,
v
o
l.
37
,
n
o
.
3
,
p
p
.
3
5
6
-
3
7
0
,
2
0
1
1
.
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
A
c
c
u
racy
Pre
c
i
s
i
o
n
R
e
c
all
F
-S
c
o
re
Co
n
Pre
d
i
c
to
r
De
ve
l
o
p
e
d
S
ys
te
m
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ad
v
Ap
p
l Sci
I
SS
N:
2
2
5
2
-
8
8
1
4
Dev
elo
p
men
t o
f so
ftw
a
r
e
d
efec
t p
r
ed
ictio
n
s
ystem
u
s
in
g
a
r
tifi
cia
l n
eu
r
a
l n
etw
o
r
k
(
Ola
tu
n
j
i B
.
L.
)
293
[1
2
]
Ra
jes
h
,
K.
a
n
d
G
u
p
ta,
D.
L.
,
“
S
o
ftwa
re
F
a
u
lt
P
re
d
ictio
n
,
”
In
ter
n
a
t
io
n
a
l
J
o
u
r
n
a
l
o
f
Co
m
p
u
ter
S
c
ien
c
e
a
n
d
M
o
b
il
e
Co
mp
u
t
in
g
,
v
o
l
.
4
,
n
o
.
9
,
p
p
.
2
5
0
-
254
,
2
0
1
5
.
[1
3
]
Xu
,
J.
Ho
,
D.
a
n
d
Ca
r
p
re
t,
L.
F
.
,
“
An
e
m
p
iri
c
a
l
st
u
d
y
o
n
th
e
p
ro
c
e
d
u
re
to
d
e
ri
v
e
so
ftwa
re
q
u
a
li
ty
e
stim
a
ti
o
n
m
o
d
e
ls,”
In
ter
n
a
ti
o
n
a
l
J
o
u
r
n
a
l
o
f
Co
m
p
u
ter
S
c
ien
c
e
&
I
n
fo
rm
a
ti
o
n
T
e
c
h
n
o
lo
g
y
(IJ
C
S
IT
)
,
v
ol
.
2
,
no.
4
,
p
p
.
1
-
16
,
2
0
1
0
.
[1
4
]
Ha
ta,
H.,
M
iz
u
n
o
,
O.
a
n
d
Kik
u
n
o
,
T
.
“
Bu
g
p
re
d
icti
o
n
b
a
se
d
o
n
fi
n
e
-
g
ra
in
e
d
m
o
d
u
le
h
isto
ries
,”
I
n
Pro
c
e
e
d
in
g
s
o
f
th
e
3
4
th
In
ter
n
a
ti
o
n
a
l
Co
n
fer
e
n
c
e
o
n
S
o
ft
wa
re
E
n
g
i
n
e
e
rin
g
,
ICS
E
’
1
2
,
p
p
.
2
0
0
-
2
1
0
,
2
0
1
2
.
[1
5
]
F
e
n
to
n
,
N.
a
n
d
Ne
il
,
M
.
,
“
Us
in
g
Ba
y
e
sia
n
n
e
two
r
k
s
t
o
p
re
d
ict
s
o
ft
wa
re
d
e
fe
c
ts
a
n
d
re
li
a
b
i
li
ty
,
”
Pro
c
.
IM
e
c
h
E
v
o
l
.
2
2
2
Pa
rt
O: J.
Ri
sk
a
n
d
Reli
a
b
il
i
t
y
,
p
p
.
7
0
2
-
7
0
3
,
2
0
0
8
.
[1
6
]
Jie
Xu
,
Da
n
n
y
H
o
a
n
d
L
u
iz
F
e
rn
a
n
d
o
,
“
An
Emp
iri
c
a
l
S
t
u
d
y
o
n
T
h
e
P
ro
c
e
d
u
re
Dri
v
e
S
o
ftwa
re
Qu
a
li
ty
E
stim
a
ti
o
n
M
o
d
e
ls,
”
In
ter
n
a
ti
o
n
a
l
j
o
u
rn
a
l
o
f
c
o
mp
u
ter
sc
ien
c
e
&
in
fo
rm
a
ti
o
n
T
e
c
h
n
o
lo
g
y
(IJ
CS
IT
)
,
v
o
l
.
2
,
n
o.
4,
pp
.
12
-
15
,
2
0
1
0
.
[1
7
]
M
a
n
u
,
B.
,
“
Co
m
p
u
tati
o
n
a
l
H
y
b
r
id
s
To
wa
rd
s
S
o
ftwa
re
De
fe
c
t
P
re
d
ictio
n
s,
”
I
n
ter
n
a
ti
o
n
a
l
J
o
u
rn
a
l
o
f
S
c
ien
ti
fi
c
En
g
i
n
e
e
rin
g
a
n
d
T
e
c
h
n
o
lo
g
y
,
v
o
.
2
,
n
o
.
5
,
p
p
.
3
1
1
-
3
1
6
,
2
0
1
3
.
[1
8
]
P
u
n
e
e
t,
J.
K.
a
n
d
P
a
ll
a
v
i
,
“
Da
ta
M
in
i
n
g
Tec
h
n
i
q
u
e
s
f
o
r
S
o
ftwa
r
e
De
fe
c
t
P
re
d
ictio
n
,
”
I
n
ter
n
a
ti
o
n
a
l
J
o
u
rn
a
l
o
f
S
o
ft
w
a
re
a
n
d
W
e
b
S
c
ien
c
e
s
,
v
o
l.
3
,
n
o
.
1
,
p
p
.
5
4
-
57
,
2
0
1
3
.
[1
9
]
G
ra
y
,
D.,
Bo
we
s,
D.,
Da
v
e
y
,
N
.
a
n
d
S
u
n
,
Y.,
“
Br
u
c
e
Ch
risti
a
n
so
n
,
Us
i
n
g
th
e
S
u
p
p
o
rt
Ve
c
to
r
M
a
c
h
in
e
a
s
a
Clas
sifica
ti
o
n
M
e
th
o
d
f
o
r
S
o
ftw
a
re
De
fe
c
t
P
re
d
ictio
n
with
S
tatic
Co
d
e
M
e
tri
c
s,”
1
1
th
In
ter
n
a
ti
o
n
a
l
Co
n
fer
e
n
c
e
,
EA
NN 2
0
1
9
,
pp
.
21
-
25
,
2
0
1
9
.
[2
0
]
Kh
a
n
,
K.
a
n
d
S
a
h
a
i,
A.,
“
Co
m
p
a
r
iso
n
o
f
BA,
G
A,
P
S
O,
BP
a
n
d
L
M
fo
r
Train
i
n
g
F
e
e
d
F
o
rwa
rd
Ne
u
ra
l
Ne
two
rk
s i
n
E
-
Lea
rn
in
g
Co
n
tex
t,
”
I
n
t
J
I
n
tel
S
y
st A
p
p
,
v
o
l.
l7
,
pp
.
23
-
29
,
2
0
1
2
.
[2
1
]
Yu
,
T.
,
Wen
,
W
.
,
Ha
n
,
X.
a
n
d
Ha
y
e
s,
J.,
“
Co
n
p
re
d
icto
r:
Co
n
c
u
rre
n
c
y
De
fe
c
t
P
re
d
icti
o
n
in
Re
a
l
-
W
o
rld
Ap
p
li
c
a
ti
o
n
s,”
In
IEE
E
In
ter
n
a
ti
o
n
a
l
Co
n
fer
e
n
c
e
o
n
S
o
ft
w
a
re
T
e
stin
g
,
Ver
if
ica
ti
o
n
a
n
d
Va
li
d
a
ti
o
n
,
pp
.
1
6
8
-
1
7
9
,
2
0
1
8
.
Evaluation Warning : The document was created with Spire.PDF for Python.