I
nd
o
ne
s
ia
n J
o
urna
l o
f
E
lect
rica
l En
g
ineering
a
nd
Co
m
pu
t
er
Science
Vo
l.
37
,
No
.
1
,
J
an
u
ar
y
20
2
5
,
p
p
.
51
8
~
5
29
I
SS
N:
2
502
-
4
7
52
,
DOI
: 1
0
.
1
1
5
9
1
/ijee
cs
.v
37.
i
1
.
p
p
5
18
-
5
29
518
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ee
cs
.
ia
esco
r
e.
co
m
A nov
el bound
a
ry
ada
ptive o
v
ersa
mpling
appro
a
ch
for
intrusio
n de
tec
tion
Rit
ind
er
K
a
ur,
Neha
G
up
t
a
S
c
h
o
o
l
o
f
C
o
m
p
u
t
e
r
A
p
p
l
i
c
a
t
i
o
n
s,
M
a
n
a
v
R
a
c
h
n
a
I
n
t
e
r
n
a
t
i
o
n
a
l
I
n
st
i
t
u
t
e
o
f
R
e
se
a
r
c
h
a
n
d
S
t
u
d
i
e
s,
F
a
r
i
d
a
b
a
d
,
I
n
d
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Ap
r
17
,
2
0
2
4
R
ev
is
ed
Sep
11
,
2
0
2
4
Acc
ep
ted
Sep
29
,
2
0
2
4
M
a
n
a
g
in
g
u
n
b
a
lan
c
e
d
d
a
tas
e
ts
is
a
sig
n
ifi
c
a
n
t
c
h
a
ll
e
n
g
e
in
in
tr
u
sio
n
d
e
tec
ti
o
n
,
si
n
c
e
u
n
c
o
m
m
o
n
a
ss
a
u
lt
s
a
re
o
ften
o
b
sc
u
re
d
b
y
th
e
b
u
l
k
o
f
re
g
u
lar
n
e
two
rk
traffic.
In
o
r
d
e
r
to
m
it
i
g
a
te
th
e
e
ffe
c
ts
o
f
c
las
s
imb
a
lan
c
e
a
n
d
imp
r
o
v
e
i
n
tr
u
sio
n
d
e
tec
ti
o
n
s
y
ste
m
(IDS)
p
e
rfo
rm
a
n
c
e
,
it
is
n
e
c
e
ss
a
ry
to
u
se
a
v
a
riety
o
f
imb
a
lan
c
e
d
lea
rn
in
g
a
l
g
o
rit
h
m
s.
M
e
th
o
d
s
o
f
d
a
ta
a
u
g
m
e
n
tatio
n
su
c
h
a
s
a
d
a
p
ti
v
e
sy
n
th
e
ti
c
sa
m
p
li
n
g
(AD
ASYN)
a
n
d
sy
n
t
h
e
ti
c
m
in
o
rit
y
o
v
e
rsa
m
p
li
n
g
tec
h
n
iq
u
e
(S
M
OTE)
a
re
u
se
fu
l
i
n
a
d
d
re
ss
in
g
c
las
s
imb
a
lan
c
e
.
Th
is
p
a
p
e
r
i
n
tr
o
d
u
c
e
s
a
n
o
v
e
l
tec
h
n
i
q
u
e
t
o
d
a
ta
r
e
sa
m
p
li
n
g
wh
e
re
d
e
c
isio
n
tree
-
g
e
n
e
ra
ted
d
e
c
isio
n
b
o
u
n
d
a
ries
a
re
u
se
d
t
o
c
o
n
d
u
c
t
AD
ASYN
o
n
c
o
m
p
li
c
a
ted
a
n
d
u
n
u
s
u
a
l
sa
m
p
les
.
Wh
e
n
th
is
m
e
th
o
d
’
s
e
ffica
c
y
wa
s
e
v
a
lu
a
ted
u
si
n
g
th
e
sta
n
d
a
rd
NSL
-
KD
D
d
a
tas
e
t,
th
e
a
c
c
u
ra
c
y
o
f
th
e
u
n
u
s
u
a
l
c
las
s u
2
r
wa
s in
c
re
a
se
d
to
4
2
%
a
n
d
,
fo
r
r
2
l
it
wa
s imp
ro
v
e
d
to
8
3
%
,
re
sp
e
c
ti
v
e
l
y
.
T
h
e
UN
S
W
-
NB
1
5
d
a
tas
e
t
h
a
s
b
e
e
n
u
se
d
f
o
r
fu
rt
h
e
r
v
a
li
d
a
ti
o
n
o
f
th
e
m
e
th
o
d
,
a
n
d
it
s
sta
ti
stica
l
sig
n
if
ica
n
c
e
h
a
s
b
e
e
n
a
ss
e
rted
b
y
c
o
m
p
a
rin
g
th
e
su
g
g
e
ste
d
m
e
t
h
o
d
to
o
th
e
r
o
v
e
rsa
m
p
li
n
g
tec
h
n
iq
u
e
s.
K
ey
w
o
r
d
s
:
Dec
is
io
n
tr
ee
s
I
m
b
alan
ce
d
d
ataset
I
m
b
alan
ce
d
lea
r
n
in
g
I
n
tr
u
s
io
n
d
etec
tio
n
Ov
er
s
am
p
lin
g
tech
n
iq
u
es
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
R
itin
d
er
Kau
r
Sch
o
o
l o
f
C
o
m
p
u
ter
A
p
p
licatio
n
s
,
Ma
n
av
R
ac
h
n
a
I
n
ter
n
atio
n
al
I
n
s
titu
te
o
f
R
esear
ch
an
d
Stu
d
ies
Secto
r
4
3
,
Far
id
a
b
ad
,
Har
y
an
a
1
2
1
0
0
4
,
I
n
d
ia
E
m
ail: r
itin
d
er
k
au
r
.
s
g
tb
im
it@
g
m
ail.
co
m
1.
I
NT
RO
D
UCT
I
O
N
R
ap
id
tech
n
o
lo
g
ical
ad
v
an
ce
m
en
ts
co
u
p
le
d
with
a
n
in
c
r
e
asin
g
r
elian
ce
o
n
n
etwo
r
k
e
d
co
m
p
u
ter
s
y
s
tem
s
h
av
e
led
to
a
s
tar
tlin
g
in
cr
ea
s
e
in
cy
b
er
th
r
ea
ts
an
d
a
ttack
s
.
I
n
tr
u
s
io
n
d
etec
tio
n
s
y
s
t
em
s
(
I
DS)
ar
e
v
ital
to
th
e
p
r
o
tectio
n
o
f
co
m
p
u
ter
n
etwo
r
k
s
b
ec
au
s
e
th
ey
ca
n
id
en
tify
an
d
s
to
p
th
ese
co
n
s
tan
tly
ch
an
g
in
g
th
r
ea
ts
.
B
u
t
it
m
ig
h
t
b
e
d
i
f
f
icu
lt
f
o
r
co
n
v
en
tio
n
al
I
DS
tech
n
iq
u
es
to
id
en
tify
n
ew
an
d
ad
v
a
n
ce
d
th
r
ea
ts
,
esp
ec
ially
wh
en
d
ea
lin
g
with
u
n
b
alan
ce
d
n
etwo
r
k
d
ata.
A
c
o
m
m
o
n
an
d
im
p
o
r
tan
t
p
r
o
b
lem
in
in
tr
u
s
io
n
d
etec
tio
n
is
d
at
a
im
b
alan
ce
,
wh
ic
h
p
r
esen
ts
a
lo
t
o
f
d
if
f
ic
u
lties
f
o
r
m
ac
h
i
n
e
lear
n
in
g
s
y
s
tem
s
.
Ma
ch
in
e
l
ea
r
n
in
g
alg
o
r
ith
m
s
f
in
d
it
ch
allen
g
in
g
to
co
r
r
ec
tl
y
ca
teg
o
r
is
e
an
d
id
e
n
tify
p
o
s
s
ib
le
in
tr
u
s
io
n
s
d
u
e
to
th
e
u
n
e
q
u
al
d
is
tr
ib
u
tio
n
o
f
class
es in
in
tr
u
s
io
n
d
etec
tio
n
d
atasets
[
1
]
.
B
ec
au
s
e
o
f
th
is
,
ML
a
lg
o
r
ith
m
s
co
u
ld
f
in
d
it
d
if
f
icu
lt
to
r
ec
o
g
n
is
e
an
d
f
o
r
ec
ast
th
e
m
i
n
o
r
ity
class
with
s
u
f
f
icien
t
ac
cu
r
ac
y
,
wh
i
ch
f
u
r
th
e
r
in
cr
ea
s
es
th
e
f
alse
-
n
eg
ativ
e
r
ate
an
d
r
ed
u
ce
th
ei
r
o
v
er
all
ef
f
icac
y
in
id
en
tify
in
g
in
tr
u
s
io
n
s
.
I
n
r
ea
l
-
wo
r
ld
ap
p
licatio
n
s
,
wh
en
t
h
e
m
in
o
r
ity
class
r
ef
lects
in
f
r
eq
u
en
t
o
cc
u
r
r
e
n
ce
s
lik
e
n
etwo
r
k
ass
au
lts
o
r
u
n
au
th
o
r
is
ed
ac
ce
s
s
attem
p
ts
,
th
is
p
r
o
b
lem
is
m
o
r
e
tr
o
u
b
lin
g
.
B
y
r
ai
s
in
g
th
e
d
etec
tio
n
r
ate
o
f
m
in
o
r
ity
class
es,
im
b
al
an
ce
d
lear
n
in
g
in
I
DS
h
as
s
h
o
wn
p
o
ten
tial
in
r
eso
lv
in
g
th
is
p
r
o
b
lem
[
2
]
.
E
ith
er
th
e
lear
n
in
g
alg
o
r
ith
m
its
elf
o
r
th
e
tr
ain
in
g
d
ata
s
am
p
les
m
ay
b
e
alter
ed
to
a
d
d
r
ess
d
ata
i
m
b
alan
ce
at
th
e
d
ata
lev
el.
T
wo
m
eth
o
d
s
ar
e
av
ailab
le
to
d
ata
-
d
r
i
v
en
tech
n
iq
u
es
to
ac
h
iev
e
class
d
is
tr
ib
u
tio
n
b
alan
cin
g
:
o
v
er
s
am
p
lin
g
an
d
u
n
d
er
s
am
p
lin
g
.
T
o
c
r
ea
te
a
b
ala
n
ce
d
d
ataset,
o
v
er
s
am
p
lin
g
d
u
p
licates
o
r
s
y
n
th
esis
es
in
s
tan
ce
s
o
f
m
in
o
r
ity
class
es.
T
ec
h
n
iq
u
es lik
e
s
y
n
th
etic
m
in
o
r
i
ty
o
v
er
s
am
p
lin
g
tech
n
iq
u
e
(
SMOT
E
)
,
ad
ap
tiv
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
A
n
o
ve
l b
o
u
n
d
a
r
y
a
d
a
p
tive
o
v
ers
a
mp
lin
g
a
p
p
r
o
a
c
h
fo
r
in
tr
u
s
io
n
d
etec
tio
n
(
R
itin
d
er K
a
u
r
)
519
s
y
n
th
etic
s
am
p
lin
g
(
ADASYN
)
,
an
d
r
a
n
d
o
m
o
v
er
s
am
p
lin
g
m
ay
b
e
u
s
ed
t
o
ac
c
o
m
p
lis
h
t
h
is
.
Alter
n
ativ
ely
,
to
g
et
a
b
ala
n
ce
d
d
ataset,
u
n
d
e
r
s
am
p
lin
g
d
e
cr
ea
s
es
th
e
o
cc
u
r
r
en
ce
s
o
f
th
e
m
ajo
r
ity
cla
s
s
.
Un
d
er
s
am
p
lin
g
m
eth
o
d
s
in
clu
d
e
to
m
e
k
lin
k
s
a
n
d
r
a
n
d
o
m
u
n
d
er
s
am
p
lin
g
.
E
v
en
th
o
u
g
h
th
ese
m
eth
o
d
s
ar
e
s
tr
aig
h
tf
o
r
war
d
an
d
s
im
p
le
to
u
s
e,
if
th
ey
a
r
e
n
o
t u
s
ed
ca
r
ef
u
lly
,
th
e
y
m
ig
h
t r
esu
l
t in
o
v
er
f
itti
n
g
an
d
r
ed
u
n
d
an
c
y
[
3
]
.
A
l
g
o
r
i
t
h
m
-
b
a
s
e
d
m
e
t
h
o
d
s
m
o
d
i
f
y
t
h
e
l
e
a
r
n
i
n
g
a
l
g
o
r
i
t
h
m
t
o
i
n
c
r
e
a
s
e
i
ts
s
e
n
s
i
ti
v
i
t
y
t
o
c
a
s
es
o
f
m
i
n
o
r
i
t
y
c
l
a
s
s
es
.
W
h
i
le
t
h
r
e
s
h
o
l
d
s
h
i
f
ti
n
g
m
o
d
i
f
i
e
s
t
h
e
c
l
a
s
s
i
f
i
c
at
i
o
n
t
h
r
e
s
h
o
l
d
t
o
f
a
v
o
u
r
o
c
c
u
r
r
e
n
c
e
s
o
f
m
i
n
o
r
i
t
y
c
l
a
s
s
es
,
c
o
s
t
-
s
e
n
s
it
i
v
e
l
e
a
r
n
i
n
g
al
l
o
c
ate
s
d
is
ti
n
c
t
m
i
s
cl
as
s
i
f
i
c
a
ti
o
n
co
s
t
s
t
o
v
a
r
i
o
u
s
c
l
as
s
es
.
A
d
d
it
i
o
n
a
l
l
y
,
e
n
s
e
m
b
l
e
m
e
t
h
o
d
s
m
a
y
b
e
u
s
e
d
t
o
e
n
h
a
n
c
e
c
l
as
s
i
f
i
c
at
i
o
n
p
e
r
f
o
r
m
a
n
c
e
.
T
h
i
s
a
p
p
r
o
a
c
h
b
at
t
l
es
w
i
t
h
e
x
t
r
e
m
e
i
m
b
a
l
a
n
ce
a
n
d
s
i
g
n
i
f
i
c
a
n
t
c
o
m
p
u
t
i
n
g
c
o
s
ts
,
b
u
t
i
t
w
o
r
k
s
w
i
t
h
a
n
y
al
g
o
r
i
t
h
m
th
a
t
d
o
e
s
n
’
t
m
o
d
i
f
y
t
h
e
d
a
t
a
.
Ad
d
itio
n
ally
,
r
esear
ch
er
s
s
u
g
g
ested
h
y
b
r
id
s
tr
ateg
ies
lik
e
SMOT
E
B
o
o
s
t,
R
US
B
o
o
s
t,
an
d
SMOT
E
-
E
NN,
wh
ich
b
len
d
al
g
o
r
ith
m
-
b
ased
an
d
d
ata
-
d
r
iv
en
m
e
th
o
d
s
to
m
ax
im
is
e
ea
ch
o
n
e
’
s
b
en
ef
its
[
4
]
–
[
6
]
.
Alth
o
u
g
h
th
ese
h
y
b
r
id
alg
o
r
i
th
m
s
in
v
o
lv
e
s
o
p
h
is
ticated
im
p
lem
en
tatio
n
a
n
d
p
a
r
am
ete
r
o
p
tim
is
atio
n
,
th
ey
in
cr
ea
s
e
th
e
p
er
f
o
r
m
an
ce
o
f
t
h
e
class
if
ier
.
E
m
p
ir
ical
s
tu
d
ies
in
th
e
liter
atu
r
e
s
u
p
p
o
r
t
th
e
u
s
e
o
f
o
v
er
s
am
p
lin
g
r
at
h
er
th
an
u
n
d
er
s
am
p
lin
g
b
ec
au
s
e
th
ey
en
h
a
n
ce
m
o
d
el
g
en
er
alis
atio
n
o
v
e
r
u
n
s
ee
n
d
ata
an
d
r
etain
in
f
o
r
m
ati
o
n
s
ee
n
in
m
in
o
r
ity
s
am
p
les
[
7
]
,
[
8
]
.
T
h
e
m
o
s
t
wid
ely
u
s
ed
o
v
er
s
am
p
lin
g
tech
n
i
q
u
e,
SMOT
E
[
9
]
,
cr
e
ates
s
y
n
th
etic
in
s
tan
ce
s
b
y
in
ter
p
o
latin
g
b
etwe
e
n
a
c
h
o
s
en
in
s
tan
ce
an
d
its
n
eig
h
b
o
u
r
af
ter
d
ete
r
m
in
in
g
th
e
k
-
n
ea
r
est
n
eig
h
b
o
u
r
s
o
f
m
in
o
r
ity
class
ex
am
p
les.
Un
til
th
e
ap
p
r
o
p
r
iate
b
alan
ce
is
r
ea
ch
ed
,
th
is
p
r
o
ce
d
u
r
e
is
r
ep
ea
t
ed
.
Desp
ite
b
ein
g
a
p
o
p
u
lar
m
eth
o
d
f
o
r
h
an
d
lin
g
u
n
b
alan
ce
d
d
ata,
SMOT
E
h
as t
h
e
f
o
llo
win
g
d
r
awb
ac
k
s
[
1
0
]
:
a)
Sy
n
th
etics th
at
co
u
ld
b
e
ar
tific
ial
an
d
in
ac
cu
r
atel
y
d
ep
ict
th
e
ac
tu
al
d
is
tr
ib
u
tio
n
o
f
th
e
m
i
n
o
r
ity
class
.
b)
T
h
e
u
n
r
ea
lis
tic
ass
u
m
p
tio
n
th
at
th
er
e
is
a
lin
ea
r
d
ec
is
io
n
b
o
u
n
d
ar
y
b
etwe
en
class
es.
c)
No
is
y
in
s
tan
ce
s
m
ay
b
e
in
clu
d
ed
in
n
ea
r
est
n
eig
h
b
o
u
r
s
,
w
h
ich
ca
u
s
es
n
o
is
e
to
b
e
i
n
clu
d
ed
in
s
y
n
th
etic
s
am
p
les.
d)
B
e
c
a
u
s
e
t
h
e
r
e
a
r
e
l
i
m
i
t
e
d
s
a
m
p
l
e
s
,
i
t
m
a
y
n
o
t
f
u
n
c
t
i
o
n
e
f
f
e
c
t
i
v
e
l
y
w
h
e
n
h
a
n
d
l
i
n
g
e
x
c
e
e
d
i
n
g
l
y
u
n
u
s
u
a
l
c
l
a
s
s
e
s
.
Nu
m
er
o
u
s
iter
atio
n
s
o
f
SMOT
E
h
av
e
b
ee
n
p
r
o
p
o
s
ed
to
o
v
er
co
m
e
th
ese
p
r
o
b
lem
s
an
d
i
m
p
r
o
v
e
th
e
task
’
s
p
er
f
o
r
m
a
n
ce
f
o
r
th
e
im
b
alan
ce
d
d
ata
class
if
icatio
n
th
r
o
u
g
h
o
u
t
th
e
y
ea
r
s
[
1
1
]
.
T
h
is
s
tu
d
y
em
p
h
asis
es
o
n
SMOT
E
v
ar
ian
ts
th
at
u
tili
s
e
d
ec
is
io
n
b
o
u
n
d
a
r
ies
as
an
ap
p
r
o
ac
h
f
o
r
o
v
er
s
am
p
lin
g
b
ec
au
s
e
th
ey
allo
w
g
en
er
atio
n
o
f
ar
tific
ial
s
am
p
l
es
th
at
ar
e
co
m
p
atib
le
with
t
h
e
o
r
ig
in
al
d
ata
d
is
tr
ib
u
tio
n
.
Dec
is
io
n
b
o
u
n
d
ar
y
-
b
ased
o
v
e
r
s
am
p
lin
g
tech
n
iq
u
e
s
g
en
er
ate
s
y
n
th
etic
s
am
p
les
wh
ich
ar
e
ac
cu
r
ate
an
d
illu
s
tr
ativ
e
o
f
th
e
m
i
n
o
r
ity
class
.
Dif
f
er
en
t
class
es
in
a
d
ataset
ar
e
ca
teg
o
r
is
ed
u
s
in
g
d
ec
is
io
n
b
o
u
n
d
ar
ies.
Va
r
io
u
s
S
MO
T
E
ad
ap
tatio
n
s
h
av
e
em
er
g
ed
to
tac
k
le
im
b
ala
n
ce
d
class
if
icatio
n
is
s
u
es a
n
d
en
h
an
ce
d
ec
is
io
n
b
o
u
n
d
a
r
y
p
r
ec
is
io
n
.
W
h
i
l
e
AD
AS
Y
N
[
1
2
]
f
o
c
u
s
s
e
s
o
n
b
u
i
l
d
i
n
g
s
y
n
t
h
e
t
i
c
s
a
m
p
l
e
s
i
n
a
r
e
a
s
w
it
h
f
e
w
e
r
m
i
n
o
r
i
t
y
c
l
a
s
s
i
n
s
t
a
n
c
es
t
o
a
d
d
r
e
s
s
c
l
as
s
o
v
er
l
a
p
p
i
n
g
,
B
o
r
d
e
r
l
i
n
e
-
SM
O
T
E
[
1
3
]
a
i
m
s
t
o
e
n
h
a
n
c
e
c
l
a
s
s
i
f
i
ca
t
i
o
n
a
c
c
u
r
a
c
y
f
o
r
b
o
r
d
e
r
l
i
n
e
c
a
s
es
b
y
t
a
r
g
e
t
i
n
g
t
h
e
m
i
n
o
r
i
t
y
c
l
a
s
s
’
s
d
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
.
T
h
i
s
i
s
f
u
r
t
h
e
r
r
e
f
i
n
e
d
b
y
s
a
f
e
-
l
e
v
e
l
S
M
O
T
E
[
1
4
]
,
w
h
i
c
h
t
a
k
e
s
i
n
t
o
a
c
c
o
u
n
t
t
h
e
s
a
f
e
-
l
e
v
e
l
r
a
t
i
o
t
o
p
r
e
v
e
n
t
p
r
o
d
u
c
i
n
g
n
o
i
s
y
s
a
m
p
le
s
.
T
h
r
o
u
g
h
s
a
m
p
l
e
w
e
i
g
h
t
i
n
g
a
n
d
g
e
o
m
e
t
r
i
c
a
n
a
l
y
s
is
,
MW
MO
T
E
[
1
5
]
,
a
n
d
G
-
S
M
OT
E
[
1
6
]
e
n
h
a
n
c
e
d
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
m
a
n
a
g
e
m
e
n
t
i
n
a
n
i
n
d
i
r
e
c
t
m
an
n
e
r
.
O
n
t
h
e
o
t
h
e
r
h
a
n
d
,
K
-
m
e
a
n
s
S
M
O
T
E
[
1
7
]
s
p
e
ci
f
i
c
a
ll
y
u
s
es
c
l
u
s
t
e
r
i
n
g
t
o
g
e
n
e
r
a
t
e
a
r
ti
f
i
c
ia
l
s
a
m
p
l
es
a
c
r
o
s
s
a
r
a
n
g
e
o
f
d
e
c
is
i
o
n
b
o
u
n
d
a
r
y
a
r
e
a
s
.
T
o
e
n
h
a
n
c
e
t
h
e
ef
f
e
c
t
i
v
e
n
e
s
s
o
f
t
h
e
c
l
a
s
s
i
f
ie
r
,
SV
M
-
SM
O
T
E
[
1
8
]
u
s
e
s
s
u
p
p
o
r
t
v
e
ct
o
r
m
a
c
h
i
n
es
t
o
g
e
n
e
r
a
t
e
a
r
t
i
f
ic
i
a
l
s
a
m
p
l
e
s
c
l
o
s
e
t
o
t
h
e
SV
M
-
d
e
f
i
n
e
d
d
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
.
M
o
r
e
r
e
c
e
n
t
t
e
c
h
n
i
q
u
e
s
i
n
cl
u
d
e
s
o
p
h
i
s
ti
c
a
t
e
d
m
et
h
o
d
s
u
s
i
n
g
n
e
u
r
a
l
n
e
t
w
o
r
k
s
a
n
d
a
l
g
o
r
i
t
h
m
s
i
n
s
p
i
r
e
d
b
y
n
a
t
u
r
e,
a
s
w
el
l
as
d
e
c
is
i
o
n
b
o
u
n
d
a
r
y
c
o
m
p
u
t
a
t
i
o
n
-
b
a
s
e
d
o
v
e
r
s
am
p
l
i
n
g
[
1
9
]
,
w
h
i
ch
c
r
e
a
t
e
s
s
y
n
t
h
e
ti
c
s
a
m
p
l
e
s
b
y
ex
a
m
i
n
i
n
g
b
o
r
d
e
r
r
e
g
i
o
n
s
.
V
a
r
io
u
s
n
e
u
r
a
l
n
e
t
w
o
r
k
a
d
a
p
t
a
ti
o
n
t
e
c
h
n
i
q
u
e
s
a
r
e
u
s
e
d
,
s
u
c
h
a
s
S
i
a
m
-
I
DS
[
2
0
]
,
w
h
i
c
h
a
d
a
p
t
s
n
e
u
r
a
l
n
e
tw
o
r
k
s
t
o
h
a
n
d
l
e
s
a
m
p
le
d
i
s
t
a
n
c
es
d
u
r
i
n
g
t
h
e
t
r
a
i
n
i
n
g
p
h
a
s
e
;
[
2
1
]
a
d
d
s
m
i
s
c
la
s
s
i
f
i
ca
t
i
o
n
c
o
s
t
a
t
te
s
t
i
n
g
p
h
a
s
e;
S
A
SM
O
T
E
[
2
2
]
,
w
h
i
c
h
i
n
t
e
g
r
a
t
e
s
a
tt
e
n
t
i
o
n
m
ec
h
a
n
i
s
m
s
f
o
r
h
i
g
h
-
q
u
a
l
i
t
y
s
a
m
p
l
e
g
e
n
e
r
a
ti
o
n
.
F
u
r
t
h
e
r
m
o
r
e
,
b
y
i
m
p
r
o
v
i
n
g
m
i
n
o
r
i
t
y
c
l
as
s
r
e
p
r
es
e
n
t
at
i
o
n
a
c
r
o
s
s
s
e
v
e
r
a
l
s
o
u
r
c
es
,
f
e
d
e
r
a
l
-
b
a
s
e
d
[
2
3
]
a
n
d
m
e
t
a
-
l
ea
r
n
i
n
g
-
b
a
s
e
d
[
2
4
]
S
M
O
T
E
t
e
ch
n
i
q
u
e
s
s
o
l
v
e
cl
a
s
s
i
m
b
al
a
n
c
e
in
r
e
m
o
t
e
d
a
t
a
b
a
s
es
.
Pre
cise
d
ec
is
io
n
b
o
u
n
d
a
r
ies
ar
e
ess
en
tial
f
o
r
d
ec
is
io
n
-
b
ased
SMOT
E
v
a
r
iatio
n
s
to
f
u
n
ctio
n
well.
B
o
u
n
d
ar
ies
th
at
a
r
e
u
n
clea
r
o
r
n
o
is
y
m
ig
h
t
im
p
air
s
y
n
th
etic
s
am
p
les
an
d
l
o
wer
p
e
r
f
o
r
m
an
ce
.
I
n
h
ig
h
-
d
im
en
s
io
n
al
d
ata
co
n
te
x
ts
,
s
o
m
e
v
e
r
s
io
n
s
,
s
u
ch
as
SMO
T
E
-
DL
[
2
5
]
,
n
ee
d
v
ar
iab
le
s
elec
tio
n
in
o
r
d
er
to
p
r
o
v
id
e
an
u
n
b
iased
class
if
icatio
n
.
An
o
v
e
r
v
iew
o
f
SMO
T
E
ap
p
r
o
ac
h
es
is
i
n
clu
d
ed
i
n
T
ab
le
1
,
with
an
em
p
h
asis
o
n
th
e
m
ai
n
tech
n
iq
u
es a
n
d
d
if
f
icu
lties
.
T
h
e
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
im
ar
y
class
if
ier
an
d
its
ad
ep
tn
es
s
to
ca
p
tu
r
e
th
e
d
ec
is
io
n
b
o
u
n
d
ar
ies
ar
e
th
e
m
ain
f
ac
to
r
s
d
eter
m
in
in
g
h
o
w
ef
f
ec
ti
v
e
th
ese
c
h
an
g
es
a
r
e.
Fu
r
th
e
r
m
o
r
e
,
g
e
n
er
atio
n
o
f
ar
tific
ial
s
am
p
les
b
ased
o
n
d
ec
is
io
n
b
o
u
n
d
a
r
ies
d
em
an
d
s
ex
tr
em
e
ca
u
tio
n
to
war
d
s
in
tr
o
d
u
ctio
n
o
f
n
o
is
e
a
n
d
ar
tef
ac
ts
to
th
e
d
ataset.
Hig
h
-
d
im
e
n
s
io
n
al
d
o
m
ain
s
f
u
r
th
er
ch
allen
g
e
th
e
estab
lis
h
m
en
t
o
f
s
u
b
s
tan
tial
d
ec
is
io
n
lim
its
.
Fu
r
th
er
m
o
r
e
,
as
th
e
a
r
tific
ial
s
am
p
les
lie
clo
s
e
to
th
e
d
e
cisi
o
n
b
o
u
n
d
ar
y
,
th
e
s
e
m
eth
o
d
s
wer
e
u
n
ab
le
t
o
ad
eq
u
ately
e
x
am
in
e
th
e
m
in
o
r
ity
class
’
s
wh
o
le
s
p
ac
e,
m
is
s
in
g
im
p
o
r
ta
n
t p
atter
n
s
o
r
v
ar
ian
ce
s
.
I
n
l
i
g
h
t
o
f
t
h
e
s
e
d
r
a
w
b
a
c
k
s
,
t
h
e
g
o
a
l
o
f
t
h
i
s
s
t
u
d
y
i
s
t
o
p
r
o
v
i
d
e
a
n
a
l
g
o
r
i
t
h
m
f
o
r
a
d
a
p
t
i
v
e
d
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
-
b
a
s
e
d
o
v
e
r
s
a
m
p
l
i
n
g
t
h
a
t
a
v
o
i
d
s
o
u
t
l
i
e
r
s
a
n
d
c
o
v
e
r
s
t
h
e
w
h
o
l
e
m
i
n
o
r
i
t
y
s
p
a
c
e
.
I
n
o
r
d
e
r
t
o
a
d
d
r
e
s
s
u
n
b
a
l
a
n
c
e
d
c
a
t
e
g
o
r
i
s
a
t
i
o
n
,
i
t
s
e
e
k
s
t
o
a
i
d
i
n
t
h
e
c
r
e
a
t
i
o
n
o
f
s
y
n
t
h
e
t
i
c
m
i
n
o
r
i
t
y
c
l
a
s
s
s
a
m
p
l
e
s
t
h
a
t
a
r
e
m
o
r
e
r
e
l
e
v
a
n
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
37
,
No
.
1
,
J
an
u
ar
y
20
25
:
51
8
-
5
29
520
a
n
d
a
c
c
u
r
a
t
e
.
B
y
t
a
k
i
n
g
i
n
t
o
a
c
c
o
u
n
t
e
a
c
h
m
i
n
o
r
i
t
y
s
a
m
p
l
e
s
e
p
a
r
a
t
e
l
y
,
t
h
e
s
u
g
g
e
s
t
e
d
m
e
t
h
o
d
o
l
o
g
y
s
e
e
k
s
t
o
s
o
l
v
e
t
h
e
i
s
s
u
e
o
f
i
n
a
d
e
q
u
a
t
e
s
a
m
p
l
e
s
.
T
h
e
d
e
c
i
s
i
o
n
t
r
e
e
,
w
h
i
c
h
h
a
s
t
h
e
i
n
n
a
t
e
c
a
p
a
c
i
t
y
t
o
p
a
r
t
i
t
i
o
n
f
e
a
t
u
r
e
s
p
a
c
e
,
i
s
s
u
g
g
e
s
t
e
d
a
s
t
h
e
u
n
d
e
r
l
y
i
n
g
c
l
a
s
s
i
f
i
e
r
.
P
r
u
n
i
n
g
d
e
c
i
s
i
o
n
t
r
e
e
s
p
r
e
v
e
n
t
s
o
v
e
r
f
i
t
t
i
n
g
a
n
d
c
a
u
s
e
s
e
a
r
l
y
t
e
r
m
i
n
a
t
i
o
n
.
T
a
r
g
e
t
e
d
a
d
a
p
t
i
v
e
s
a
m
p
l
i
n
g
b
a
s
e
d
o
n
d
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
c
l
o
s
e
n
e
s
s
c
a
n
m
a
n
a
g
e
t
h
e
c
o
m
p
l
i
c
a
t
e
d
d
a
t
s
e
t
s
.
T
h
e
r
em
ai
n
in
g
h
alf
o
f
th
is
w
o
r
k
is
s
tr
u
ct
u
r
ed
as
f
o
llo
ws.
T
h
e
s
u
g
g
ested
ap
p
r
o
ac
h
is
p
r
esen
ted
in
s
ec
tio
n
2
,
an
d
th
e
ex
p
er
im
en
t
al
f
in
d
in
g
s
ar
e
d
is
cu
s
s
ed
in
d
e
p
th
in
s
ec
tio
n
3
.
T
h
is
p
ap
er
’
s
s
tu
d
y
is
co
n
clu
d
ed
in
s
ec
tio
n
4
,
wh
ich
also
ad
d
r
es
s
es p
o
ten
tial f
u
tu
r
e
d
ir
ec
ti
o
n
s
.
T
ab
le
1
.
C
o
m
p
a
r
is
o
n
o
f
d
ec
is
io
n
-
b
ased
SMOT
E
v
a
r
ian
ts
S
a
mp
l
e
g
e
n
e
r
a
t
i
o
n
p
r
i
n
c
i
p
l
e
S
mo
t
e
v
a
r
i
a
n
t
s
D
e
scri
p
t
i
o
n
Li
mi
t
a
t
i
o
n
s
D
e
c
i
s
i
o
n
b
o
u
n
d
a
r
y
-
b
a
s
e
d
B
o
r
d
e
r
l
i
n
e
S
M
O
TE
,
S
M
O
TE
B
o
o
st
,
S
V
M
-
S
M
O
TE
G
e
n
e
r
a
t
e
sam
p
l
e
s
n
e
a
r
d
e
c
i
si
o
n
b
o
u
n
d
a
r
i
e
s
t
o
b
a
l
a
n
c
e
c
l
a
ss
e
s.
N
o
i
s
y
s
a
mp
l
e
s
i
f
b
o
u
n
d
a
r
y
i
s
u
n
c
l
e
a
r
D
e
n
si
t
y
/
d
i
s
t
r
i
b
u
t
i
o
n
-
b
a
se
d
A
D
A
S
Y
N
,
M
W
M
O
T
E,
G
-
S
M
O
TE
U
se
sam
p
l
e
d
e
n
si
t
y
a
n
d
d
i
s
t
r
i
b
u
t
i
o
n
t
o
c
r
e
a
t
e
n
e
w
i
n
st
a
n
c
e
s.
M
a
y
a
mp
l
i
f
y
n
o
i
se
i
n
sp
a
r
se
d
a
t
a
se
t
s
C
l
u
st
e
r
i
n
g
-
b
a
s
e
d
K
M
e
a
n
s
-
S
M
O
TE
C
l
u
st
e
r
s mi
n
o
r
i
t
y
sam
p
l
e
s
b
e
f
o
r
e
g
e
n
e
r
a
t
i
n
g
s
y
n
t
h
e
t
i
c
e
x
a
m
p
l
e
s.
S
t
r
u
g
g
l
e
s w
i
t
h
c
o
m
p
l
e
x
b
o
u
n
d
a
r
i
e
s
D
e
e
p
l
e
a
r
n
i
n
g
-
b
a
s
e
d
S
mo
t
e
-
DL
U
t
i
l
i
z
e
s
d
e
e
p
l
e
a
r
n
i
n
g
m
o
d
e
l
s
t
o
c
a
p
t
u
r
e
c
o
m
p
l
e
x
d
e
c
i
si
o
n
b
o
u
n
d
a
r
i
e
s.
C
o
m
p
u
t
a
t
i
o
n
a
l
l
y
i
n
t
e
n
si
v
e
A
t
t
e
n
t
i
o
n
/
h
y
b
r
i
d
m
e
t
h
o
d
s
S
A
S
M
O
TE,
F
e
d
e
r
a
l
-
b
a
s
e
d
S
M
O
TE
Le
v
e
r
a
g
e
s
a
t
t
e
n
t
i
o
n
mec
h
a
n
i
sms
o
r
f
e
d
e
r
a
t
e
d
l
e
a
r
n
i
n
g
p
r
i
n
c
i
p
l
e
s
f
o
r
sy
n
t
h
e
t
i
c
sam
p
l
e
g
e
n
e
r
a
t
i
o
n
.
R
e
s
o
u
r
c
e
-
i
n
t
e
n
s
i
v
e
,
da
t
a
h
a
r
m
o
n
i
z
a
t
i
o
n
i
ssu
e
s
2.
RE
S
E
ARCH
M
E
T
H
O
D
T
h
is
wo
r
k
p
r
esen
ted
th
e
p
r
o
x
im
ity
-
ad
a
p
tiv
e
s
y
n
th
e
tic
m
in
o
r
ity
o
v
er
-
s
am
p
lin
g
tech
n
iq
u
e
(
PASMOT
E
)
as
a
s
o
lu
tio
n
t
o
th
e
s
ev
er
al
s
h
o
r
tco
m
i
n
g
s
o
f
SMOT
E
.
T
h
is
r
esear
ch
u
s
ed
two
wid
ely
u
s
ed
in
tr
u
s
io
n
d
atasets
,
NSL
-
KDD
an
d
UNSW
_
NB
_
1
5
,
f
o
r
its
t
r
ials
.
An
ex
p
an
s
io
n
o
f
KDD
C
u
p
9
9
,
NSL
-
KDD
f
ix
es
s
o
m
e
f
u
n
d
am
en
tal
p
r
o
b
lem
s
with
its
p
r
ed
ec
ess
o
r
[
2
6
]
.
T
h
e
r
e
is
o
n
e
n
o
r
m
al
class
an
d
f
o
u
r
k
i
n
d
s
o
f
attac
k
s
:
d
en
ial
o
f
s
er
v
ice
(
Do
S),
p
r
o
b
e,
r
em
o
te
-
to
-
u
s
er
(
r
2
l)
,
an
d
u
s
er
-
to
-
r
o
o
t
(
u
2
r
)
.
T
h
o
u
g
h
,
t
h
er
e
ar
e
m
an
y
d
if
f
er
en
t
ty
p
es
o
f
ass
au
lts
av
ailab
le
in
th
ese
attac
k
class
e
s
,
th
er
e
is
an
im
b
alan
ce
in
th
e
n
u
m
b
er
o
f
in
s
t
an
ce
s
o
f
u
n
co
m
m
o
n
attac
k
class
es
s
u
ch
as
u
s
er
-
to
-
r
o
o
t
a
n
d
r
e
m
o
te
-
to
-
u
s
er
,
co
m
p
a
r
ed
to
t
y
p
ical
o
cc
u
r
r
en
ce
s
.
T
h
e
2
0
%
tr
ain
in
g
an
d
test
+
d
ataset
f
iles
ar
e
u
s
ed
in
th
e
ex
p
er
im
en
t.
T
a
b
le
2
d
is
p
lay
s
th
e
n
u
m
b
e
r
o
f
s
am
p
les
in
ea
ch
d
ata
s
et
to
h
el
p
u
n
d
er
s
tan
d
th
e
f
r
eq
u
e
n
cy
d
is
tr
ib
u
tio
n
in
th
e
v
ar
io
u
s
class
es.
T
ab
le
2
.
Dis
tr
ib
u
tio
n
o
f
f
r
eq
u
e
n
cies in
NSL
-
KDD
d
atasets
D
a
t
a
f
i
l
e
u
se
d
C
l
a
s
s
d
i
s
t
r
i
b
u
t
i
o
n
c
o
u
n
t
N
o
r
mal
D
o
s
P
r
o
b
e
u2r
r
2
l
t
ra
i
n
2
0
%
1
3
4
4
9
9
2
3
4
2
2
8
9
11
2
0
9
t
e
st
+
9
7
1
1
7
4
5
8
2
4
2
1
3
4
1
2
7
5
4
T
h
e
U
N
SW
-
N
B
1
5
d
a
ta
s
et
h
a
s
b
e
e
n
u
s
e
d
f
o
r
t
h
e
p
r
o
p
o
s
e
d
a
p
p
r
o
a
c
h
’
s
v
a
l
i
d
a
ti
o
n
a
n
d
c
o
m
p
ar
i
s
o
n
w
it
h
a
l
t
e
r
n
a
ti
v
e
o
v
e
r
s
a
m
p
l
i
n
g
te
c
h
n
i
q
u
e
s
.
T
h
is
d
a
t
as
e
t
c
o
m
b
i
n
e
s
r
e
c
e
n
t
s
y
n
t
h
e
t
i
c
as
s
a
u
l
ts
i
n
n
e
tw
o
r
k
e
d
s
y
s
t
e
m
s
wi
t
h
m
o
d
e
r
n
r
e
g
u
l
a
r
a
t
t
a
c
k
s
[
2
7
]
.
T
h
i
s
d
a
t
as
e
t
h
as
b
e
e
n
l
a
b
e
ll
e
d
an
d
c
o
n
s
i
s
ts
o
f
n
i
n
e
a
s
s
a
u
lt
t
y
p
e
s
w
it
h
4
9
a
t
t
r
i
b
u
t
es
t
o
t
a
l
,
i
n
c
l
u
d
i
n
g
t
h
e
l
a
b
e
l
.
T
h
e
t
r
a
i
n
i
n
g
a
n
d
t
e
s
t
i
n
g
s
e
ts
o
f
t
h
e
d
a
t
as
e
t
i
n
c
l
u
d
e
1
7
5
3
4
1
a
n
d
8
2
3
3
2
s
a
m
p
l
e
s
,
r
e
s
p
e
c
t
i
v
el
y
.
T
h
e
u
n
e
q
u
a
l
d
is
t
r
i
b
u
t
i
o
n
o
f
c
l
as
s
es
i
n
t
h
e
tw
o
s
e
ts
is
s
e
e
n
i
n
F
i
g
u
r
e
1
,
w
h
i
c
h
f
u
r
t
h
e
r
e
m
p
h
a
s
is
es
t
h
e
e
x
p
e
r
i
m
e
n
t
’
s
s
i
g
n
i
f
i
c
a
n
c
e
.
S
in
c
e
u
n
u
s
u
a
l
a
t
t
a
c
k
w
o
r
m
s
m
a
k
e
u
p
a
r
el
a
t
i
v
el
y
s
m
a
l
l
p
o
r
t
i
o
n
o
f
t
h
e
t
o
t
a
l
d
i
s
t
r
i
b
u
ti
o
n
,
t
h
e
y
a
r
e
d
i
f
f
i
c
u
lt
to
i
d
e
n
t
i
f
y
,
h
e
n
c
e
t
h
is
r
e
s
ea
r
c
h
a
i
m
s
t
o
e
n
h
a
n
c
e
t
h
e
i
r
d
e
t
ec
t
i
o
n
.
Py
th
o
n
p
r
o
g
r
am
m
in
g
wa
s
u
s
ed
to
ca
r
r
y
o
u
t
t
h
e
e
x
p
er
im
en
t
o
n
th
e
well
-
k
n
o
w
n
Go
o
g
le
C
o
lab
I
DE
.
I
n
itially
,
b
o
th
d
atasets
wer
e
tu
n
ed
f
o
r
th
e
s
tu
d
y
.
T
h
er
e
ar
e
4
2
ch
ar
ac
ter
is
tics
an
d
1
class
lab
el
in
NSL
-
KDD.
T
h
r
ee
o
f
t
h
ese
f
ea
tu
r
es
-
th
e
f
la
g
,
th
e
s
er
v
ice,
an
d
th
e
p
r
o
to
c
o
l
ty
p
e
-
ar
e
p
u
r
ely
n
o
m
in
al.
T
h
ese
attr
ib
u
tes
wer
e
co
n
v
er
ted
to
th
ei
r
n
u
m
e
r
ical
eq
u
iv
alen
ts
in
o
r
d
er
to
m
ax
im
is
e
th
e
lear
n
in
g
p
r
o
ce
s
s
f
o
r
th
e
m
o
d
els.
T
h
e
n
u
m
er
ical
ch
ar
ac
ter
is
tics
wer
e
th
en
n
o
r
m
alis
ed
u
s
in
g
m
in
-
m
ax
n
o
r
m
alis
atio
n
to
m
ain
tain
th
e
r
elativ
e
d
is
p
ar
ities
b
etw
ee
n
d
ata
p
o
in
ts
an
d
g
u
ar
an
tee
th
at
th
e
r
an
g
e
o
f
v
alu
es
s
tay
s
co
n
s
tan
t.
Usi
n
g
f
o
r
m
u
la
(
1
)
,
th
e
d
ata
p
o
i
n
t X
is
r
escaled
to
th
e
r
an
g
e
[
a,
b
]
,
u
s
u
ally
[
0
,
1
]
.
=
−
−
×
(
−
)
+
(
1
)
wh
er
e,
Xm
in
r
ep
r
esen
ts
th
e
m
in
im
u
m
v
alu
e
o
f
th
e
attr
ib
u
te,
Xm
ax
is
th
e
m
ax
im
u
m
v
alu
e
o
f
th
e
attr
ib
u
te,
Xscaled
d
en
o
tes th
e
r
escaled
v
alu
e
an
d
th
e
n
ew
r
an
g
e
id
d
ef
i
n
ed
as a
–
b
(
co
m
m
o
n
l
y
0
to
1
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
A
n
o
ve
l b
o
u
n
d
a
r
y
a
d
a
p
tive
o
v
ers
a
mp
lin
g
a
p
p
r
o
a
c
h
fo
r
in
tr
u
s
io
n
d
etec
tio
n
(
R
itin
d
er K
a
u
r
)
521
Fig
u
r
e
1
.
Dis
tr
ib
u
tio
n
o
f
s
am
p
les in
th
e
UNSW
-
N
B
1
5
tr
ain
in
g
an
d
test
in
g
d
ataset
E
v
er
y
o
n
e
o
f
th
e
4
9
c
h
ar
ac
ter
is
tics
in
th
e
UNS
W
-
NB
1
5
d
ataset
u
n
d
e
r
wen
t
th
e
s
a
m
e
k
in
d
o
f
p
r
etr
ea
tm
en
t.
4
7
1
5
3
m
is
s
in
g
d
ata
led
to
th
e
elim
in
atio
n
o
f
th
e
s
er
v
ice,
a
n
d
as
th
e
la
b
el
is
e
m
p
lo
y
ed
in
b
in
ar
y
in
v
esti
g
atio
n
s
,
it
was
d
estro
y
ed
.
T
h
e
ca
teg
o
r
ical
f
ea
tu
r
e
s
attac
k
(
attac
k
_
ca
t)
,
p
r
o
to
co
l
(
p
r
o
to
)
,
a
n
d
s
er
v
ice
(
attac
k
_
ca
t)
wer
e
co
n
v
er
ted
u
s
in
g
lab
el
en
co
d
e
r
s
.
Nex
t,
m
in
-
m
ax
s
ca
lin
g
was
u
s
ed
o
n
ea
ch
an
d
ev
er
y
n
u
m
er
ical
c
h
ar
ac
ter
is
tic.
T
h
e
p
r
o
b
lem
was
co
n
v
er
ted
in
to
a
b
in
ar
y
class
p
r
o
b
lem
a
n
d
t
h
e
r
ar
est
class
wo
r
m
s
an
d
n
o
r
m
al
s
am
p
les we
r
e
ex
tr
ac
ted
in
o
r
d
er
to
c
o
n
f
ir
m
th
e
s
tatis
tical
s
ig
n
if
ican
ce
o
f
th
e
r
e
s
u
lts
.
T
o
b
alan
ce
th
e
NSL
-
KDD
tr
ain
in
g
d
ataset,
th
e
r
ec
o
m
m
en
d
ed
PASM
OT
E
tech
n
iq
u
e
F
ig
u
r
e
2
was
u
s
ed
.
I
t
s
elec
tiv
ely
s
am
p
les
i
n
s
tan
ce
s
f
r
o
m
b
o
th
m
in
o
r
ity
an
d
n
o
r
m
al
class
es
to
f
ac
ili
ta
te
th
e
tr
ain
in
g
o
f
a
class
if
ier
an
d
,
th
er
eb
y
clar
if
y
in
g
th
e
d
ec
is
io
n
b
o
r
d
e
r
s
an
d
b
o
u
n
d
ar
ies.
T
h
ese
r
esu
lts
ar
e
f
u
r
th
er
u
s
ed
to
p
r
o
d
u
ce
f
ak
e
s
am
p
les d
u
r
in
g
t
h
e
r
esam
p
lin
g
p
r
o
ce
s
s
.
Fig
u
r
e
2
.
Pro
p
o
s
ed
PASMOT
E
alg
o
r
ith
m
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
37
,
No
.
1
,
J
an
u
ar
y
20
25
:
51
8
-
5
29
522
B
ec
au
s
e
d
ec
is
io
n
tr
ee
s
n
atu
r
ally
s
p
lit
th
e
f
ea
tu
r
e
d
o
m
ain
d
u
r
in
g
tr
ai
n
in
g
,
th
e
y
s
u
p
p
o
r
t
b
o
u
n
d
a
r
y
p
r
o
x
im
ity
ca
lcu
latio
n
f
o
r
ea
c
h
d
ata
p
o
in
t.
Fo
r
e
v
er
y
d
ata
p
o
in
t,
th
e
y
c
o
m
p
u
te
th
e
d
is
tan
ce
s
to
th
e
d
ec
is
io
n
b
o
r
d
er
s
an
d
p
r
o
v
i
d
e
u
s
ef
u
l
in
s
ig
h
ts
in
to
h
o
w
clo
s
e
ex
am
p
les
ar
e
to
th
e
d
ec
is
io
n
ar
ea
s
.
T
h
e
s
e
in
s
ig
h
ts
m
ay
b
e
u
s
ed
to
im
p
r
o
v
e
o
v
er
s
am
p
lin
g
tactics.
Fu
r
th
er
m
o
r
e,
th
ese
m
o
d
els
au
to
m
atica
lly
p
r
io
r
itis
e
f
ea
tu
r
es
ac
co
r
d
in
g
to
h
o
w
cr
u
cial
th
ey
ar
e
to
th
e
ca
teg
o
r
is
atio
n
p
r
o
ce
s
s
.
W
ith
t
h
is
d
ata,
o
n
e
m
ay
e
v
alu
ate
wh
eth
er
ch
a
r
ac
ter
is
tics
ar
e
im
p
o
r
ta
n
t
in
d
if
f
er
en
tiati
n
g
m
in
o
r
ity
class
o
cc
u
r
r
e
n
c
es,
wh
ich
m
ig
h
t
g
u
id
e
f
ea
tu
r
e
en
g
in
ee
r
in
g
o
r
s
elec
tio
n
p
r
o
ce
d
u
r
es
in
h
i
g
h
-
d
im
en
s
io
n
al
d
atasets
.
C
o
m
p
ar
ed
to
lin
ea
r
class
if
ier
s
,
th
ey
ar
e
als
o
m
o
r
e
s
u
cc
ess
f
u
l
at
m
o
d
ellin
g
n
o
n
lin
ea
r
co
n
n
ec
tio
n
s
b
etwe
en
f
ea
tu
r
es
an
d
th
e
tar
g
et
v
a
r
iab
le,
en
ab
lin
g
th
e
m
o
d
ellin
g
o
f
m
o
r
e
co
m
p
licated
d
ec
is
io
n
b
o
u
n
d
ar
ies.
Fo
r
ev
er
y
m
in
o
r
ity
class
s
am
p
le,
th
e
d
is
tan
ce
f
r
o
m
th
e
b
o
r
d
e
r
was
d
eter
m
i
n
ed
,
an
d
th
e
s
am
p
les
wer
e
ar
r
an
g
e
d
ac
co
r
d
in
g
to
t
h
ese
d
i
s
tan
ce
s
.
T
o
d
eter
m
i
n
e
th
e
r
esa
m
p
lin
g
r
ates,
th
r
esh
o
ld
v
alu
es
m
ay
b
e
c
o
m
p
u
te
d
b
ased
o
n
th
e
r
an
g
e
o
f
d
is
tan
c
es.
T
h
e
p
lan
is
to
r
esam
p
le
s
a
m
p
les
n
ea
r
er
th
e
d
ec
is
io
n
b
o
r
d
er
at
g
r
ea
ter
r
ates
th
an
th
o
s
e
f
u
r
th
er
awa
y
.
B
ased
o
n
t
h
e
d
is
tan
ce
th
r
esh
o
ld
v
alu
es,
th
r
ee
r
esam
p
lin
g
r
at
es
-
0
.
6
,
0
.
4
,
an
d
0
.
2
-
wer
e
u
s
ed
f
o
r
t
h
is
ex
p
e
r
im
en
t.
T
h
e
cu
b
ic
-
s
p
lin
e
i
n
ter
p
o
latio
n
tech
n
iq
u
e
was
u
s
ed
t
o
cr
ea
te
th
e
s
y
n
th
etic
s
am
p
les.
A
m
ath
em
atica
l
m
eth
o
d
ca
lled
cu
b
ic
s
p
lin
e
in
te
r
p
o
latio
n
m
a
y
b
e
u
s
ed
to
esti
m
ate
a
f
u
n
ctio
n
’
s
v
alu
es
b
etwe
en
k
n
o
wn
d
ata
p
o
in
ts
.
T
o
en
s
u
r
e
s
m
o
o
th
n
ess
an
d
co
n
tin
u
ity
,
it
b
u
ild
s
a
p
iece
wis
e
co
n
tin
u
o
u
s
cu
r
v
e
o
u
t
o
f
m
a
n
y
cu
b
ic
p
o
ly
n
o
m
ials
.
T
h
e
p
r
o
ce
s
s
g
en
er
ates
in
ter
p
o
lated
v
alu
es
b
etwe
en
th
e
k
n
o
wn
d
ata
p
o
in
ts
b
y
iter
atin
g
o
v
e
r
ea
c
h
f
ea
tu
r
e
co
lu
m
n
a
n
d
u
s
in
g
cu
b
ic
s
p
lin
e
in
ter
p
o
latio
n
.
Dis
tin
ct
v
alu
es
ar
e
r
an
d
o
m
l
y
p
ick
ed
f
r
o
m
th
e
ca
t
eg
o
r
ical
v
ar
iab
les (
p
r
o
to
co
l_
t
y
p
e,
f
lag
,
an
d
s
er
v
ice)
in
m
in
o
r
ity
class
s
am
p
les to
g
u
ar
an
tee
g
en
er
atio
n
o
f
ar
tifi
cial
s
am
p
les
with
in
th
e
o
r
i
g
in
al
class
d
is
tr
ib
u
tio
n
.
T
h
e
s
y
n
th
etic
s
am
p
les
ar
e
r
ep
r
esen
ted
b
y
th
ese
in
ter
p
o
l
ated
n
u
m
er
ical
v
alu
es
an
d
th
e
m
o
d
if
ie
d
ca
teg
o
r
y
v
ar
iab
l
es.
B
ec
au
s
e
cu
b
ic
s
p
lin
es
ca
n
m
an
ag
e
s
p
ar
s
e
o
r
ir
r
eg
u
lar
ly
s
p
ac
ed
d
atasets
a
n
d
m
ain
tain
th
e
f
o
r
m
an
d
b
e
h
av
io
u
r
o
f
s
y
n
th
etic
d
ata
b
et
wee
n
ex
is
tin
g
d
ata
p
o
in
ts
,
th
ey
ar
e
u
s
ef
u
l
f
o
r
h
an
d
l
in
g
n
o
n
lin
ea
r
an
d
co
m
p
licated
d
ata
in
ter
ac
tio
n
s
.
I
n
o
r
d
er
to
co
m
p
u
te
an
i
n
ter
p
o
lated
v
alu
e
b
etwe
en
(
x
i
,y
i
)
an
d
(
x
i+
1
,y
i+
1
)
,
th
e
g
en
e
r
ic
eq
u
atio
n
o
f
th
e
cu
b
ic
s
p
lin
e
m
ay
b
e
e
x
p
r
ess
ed
as (
2
)
.
(
)
=
+
(
−
)
+
(
−
)
2
+
(
−
)
3
(
2
)
wh
ich
h
o
ld
s
tr
u
e
f
o
r
i
=
1
…,
n
−1
,
wh
e
r
e
x
i
≤
x
≤
x
i+
1
an
d
ea
ch
cu
b
ic
f
u
n
ctio
n
’
s
co
ef
f
ici
en
ts
a
i
,
b
i
,
c
i
,
a
n
d
d
i
m
u
s
t
b
e
f
o
u
n
d
.
T
h
e
tr
ai
n
in
g
2
0
%
NSL
-
KDD
d
ataset
’
s
s
am
p
le
d
is
tr
ib
u
tio
n
is
s
h
o
wn
in
T
ab
le
3
b
o
th
b
ef
o
r
e
an
d
af
ter
r
esam
p
lin
g
.
T
ab
le
3
.
2
0
% o
f
class
es we
r
e
d
is
tr
ib
u
ted
b
ef
o
r
e
an
d
af
ter
r
esam
p
lin
g
in
NSL
-
KDD
tr
ain
in
g
T
r
a
i
n
2
0
%
d
a
t
a
f
i
l
e
C
l
a
s
s
d
i
s
t
r
i
b
u
t
i
o
n
N
o
r
mal
D
o
s
P
r
o
b
e
r
2
l
u2r
Pre
-
r
e
samp
l
i
n
g
1
3
4
4
9
9
2
3
4
2
2
8
9
2
0
9
11
P
o
st
-
r
e
s
a
mp
l
i
n
g
1
3
4
4
9
9
2
3
4
2
2
8
9
2
6
0
3
8
4
2
T
o
v
alid
ate
th
e
p
r
o
p
o
s
ed
tech
n
iq
u
e,
two
class
es
wer
e
ex
t
r
ac
ted
f
r
o
m
th
e
UNSW
-
NB
1
5
tr
ain
in
g
d
ataset
:
n
o
r
m
al
an
d
wo
r
m
s
.
T
h
ese
class
es
s
h
o
w
th
e
r
an
g
e
o
f
s
am
p
le
d
is
tr
ib
u
tio
n
a
n
d
th
e
m
ax
im
u
m
a
n
d
m
in
im
u
m
s
am
p
les
th
at
s
u
p
p
o
r
t
th
e
r
ec
o
m
m
e
n
d
ed
p
r
o
ce
d
u
r
e
’
s
ef
f
ec
tiv
en
ess
.
T
h
e
s
am
p
le
co
u
n
ts
f
o
r
th
e
two
class
es
wer
e
5
6
0
0
0
an
d
1
3
0
,
r
esp
ec
tiv
ely
,
p
r
io
r
to
d
is
tr
ib
u
t
io
n
.
On
ce
PASMOT
E
wa
s
u
s
ed
to
r
esam
p
le
th
e
m
in
o
r
ity
class
,
th
e
co
u
n
ts
wer
e
alter
ed
to
5
6
0
0
0
an
d
6
3
1
8
,
r
esp
ec
tiv
ely
.
T
o
b
e
n
c
h
m
ar
k
th
e
p
r
o
p
o
s
ed
ap
p
r
o
ac
h
ag
ain
s
t
co
n
v
en
tio
n
a
l
o
v
er
s
am
p
lin
g
s
tr
ateg
ies,
an
eq
u
al
n
u
m
b
er
o
f
f
ak
e
s
am
p
le
s
wer
e
m
ad
e
f
r
o
m
ea
ch
o
f
t
h
e
f
o
u
r
o
v
er
s
am
p
lin
g
s
tr
ateg
ies
m
en
tio
n
ed
in
s
ec
tio
n
1
:
SMOT
E
,
R
an
d
o
m
o
v
er
s
am
p
ler
,
ADASYN
an
d
,
B
o
r
d
e
r
lin
e
SMOT
E
.
I
d
en
tify
in
g
th
e
m
o
s
t
p
er
tin
e
n
t
ch
ar
ac
ter
is
tics
in
th
e
d
ata
ca
n
m
in
im
is
e
d
im
en
s
io
n
ality
an
d
b
o
o
s
t
th
e
ef
f
ec
tiv
en
ess
o
f
m
ac
h
in
e
lear
n
in
g
alg
o
r
ith
m
s
.
T
h
is
p
ap
e
r
m
a
k
es
u
s
e
o
f
a
f
ea
tu
r
e
s
elec
tio
n
t
ec
h
n
iq
u
e
ca
lled
as
C
FS
-
MH
A
wh
ich
is
an
en
s
em
b
le
o
f
cf
s
Su
b
s
etE
v
al
alg
o
r
ith
m
s
th
at
ex
p
l
o
r
e
f
ea
tu
r
e
s
p
ac
e
u
s
in
g
m
eta
-
h
eu
r
is
tic
tech
n
iq
u
es
[
2
8
]
.
E
x
p
er
im
e
n
ta
l
s
tu
d
y
in
d
icate
s
th
at
it
is
s
u
cc
ess
f
u
l
in
p
r
io
r
itis
in
g
th
e
m
o
s
t
r
ep
r
esen
tativ
e
f
ea
tu
r
es
f
r
o
m
co
m
p
lex
in
tr
u
s
i
o
n
d
etec
tio
n
d
atasets
,
h
e
n
ce
r
ed
u
cin
g
th
e
c
o
m
p
u
tatio
n
al
co
m
p
lex
ity
an
d
len
g
th
o
f
tr
ain
ed
m
o
d
els.
Af
ter
a
p
p
ly
in
g
th
e
al
g
o
r
ith
m
to
4
2
f
ea
tu
r
es
in
th
e
NSL
-
KDD
d
ataset,
1
5
r
elev
an
t
f
ea
tu
r
es
wer
e
ex
tr
ac
ted
.
T
h
e
f
ea
tu
r
es
th
at
h
av
e
b
ee
n
s
elec
ted
ar
e:
s
er
v
ice,
f
lag
,
wr
o
n
g
_
f
r
ag
m
e
n
t,
h
o
t,
lo
g
g
ed
_
i
n
,
is
_
g
u
est_
lo
g
in
,
c
o
u
n
t,
s
am
e_
s
r
v
_
r
ate,
d
if
f
_
s
er
v
_
r
ate,
d
s
t_
h
o
s
t_
s
r
v
_
co
u
n
t,
d
s
t_
h
o
s
t_
d
if
f
_
s
r
v
_
r
ate,
d
s
t_
h
o
s
t_
s
am
e_
s
r
c_
p
o
r
t_
r
ate,
d
s
t_
h
o
s
t_
s
r
v
_
d
if
f
_
h
o
s
t_
r
ate,
d
s
t_
h
o
s
t_
s
er
r
o
r
_
r
ate,
an
d
d
s
t_
h
o
s
t_
s
r
v
_
s
er
r
o
r
_
r
ate.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
A
n
o
ve
l b
o
u
n
d
a
r
y
a
d
a
p
tive
o
v
ers
a
mp
lin
g
a
p
p
r
o
a
c
h
fo
r
in
tr
u
s
io
n
d
etec
tio
n
(
R
itin
d
er K
a
u
r
)
523
T
h
is
wo
r
k
u
s
ed
a
co
s
t
-
s
en
s
iti
v
e
r
an
d
o
m
f
o
r
est
(
R
F)
alg
o
r
ith
m
ic
ad
ap
ta
tio
n
to
s
im
u
late
th
e
r
ed
u
ce
d
NSL
-
KDD
d
ataset.
A
m
o
d
if
ie
d
tech
n
iq
u
e
ca
lled
“
co
s
t
-
s
en
s
itiv
e
class
if
icatio
n
”
ad
ju
s
ts
th
e
m
o
d
el
’
s
ab
ilit
y
to
g
en
er
alis
e
b
y
ad
d
in
g
d
o
m
ain
-
s
p
ec
if
ic
co
s
ts
an
d
co
n
s
id
er
s
t
h
e
tr
u
e
im
p
ac
t
o
f
m
is
class
if
ic
atio
n
er
r
o
r
s
in
o
r
d
er
to
h
an
d
le
im
b
alan
ce
d
d
ataset
s
[
2
9
]
.
C
o
s
t
-
s
en
s
itiv
e
cla
s
s
if
i
er
s
p
r
im
ar
ily
co
n
ce
n
tr
ate
o
n
d
if
f
er
en
ce
s
in
class
d
is
tr
ib
u
tio
n
s
.
I
n
clu
d
ed
with
t
h
em
is
a
co
s
t
m
atr
ix
,
also
r
e
f
er
r
ed
to
as
a
m
is
class
if
icatio
n
co
s
t
m
atr
ix
,
th
at
p
r
o
v
i
d
e
s
a
c
l
e
a
r
b
r
e
a
k
d
o
w
n
o
f
t
h
e
c
o
s
ts
r
el
a
t
e
d
t
o
v
a
r
i
o
u
s
c
l
a
s
s
i
f
ic
a
t
i
o
n
e
r
r
o
r
s
l
i
k
e
f
a
l
s
e
p
o
s
i
t
i
v
es
a
n
d
f
a
ls
e
n
e
g
a
t
i
v
e
s
.
U
n
l
i
k
e
o
r
d
i
n
a
r
y
c
l
as
s
i
f
i
e
r
s
,
w
h
i
c
h
s
t
r
i
v
e
t
o
r
e
d
u
c
e
t
h
e
a
g
g
r
e
g
a
t
e
c
l
as
s
i
f
i
c
a
ti
o
n
e
r
r
o
r
,
c
o
s
t
-
s
e
n
s
i
ti
v
e
c
l
a
s
s
i
f
ie
r
s
i
n
t
e
n
d
t
o
m
i
n
i
m
is
e
a
c
o
s
t
f
u
n
c
t
i
o
n
f
o
r
m
e
d
f
r
o
m
t
h
e
m
i
s
c
la
s
s
i
f
i
ca
t
i
o
n
c
o
s
t
s
s
h
o
w
n
in
t
h
e
c
o
s
t
m
at
r
i
x
.
RF
wa
s
ch
o
s
en
as
th
e
p
r
im
ar
y
class
if
ier
f
o
r
b
o
th
m
o
d
e
l
tr
ain
in
g
an
d
ev
alu
atio
n
.
D
u
e
to
its
r
o
b
u
s
tn
ess
,
ad
ap
tab
ilit
y
,
an
d
h
ig
h
p
r
o
jecte
d
ac
cu
r
ac
y
,
th
is
well
-
k
n
o
wn
en
s
em
b
le
lear
n
in
g
ap
p
r
o
ac
h
p
er
f
o
r
m
s
v
er
y
well
in
a
wid
e
r
an
g
e
o
f
m
ac
h
in
e
lear
n
in
g
ap
p
licatio
n
s
[
3
0
]
.
I
t
c
o
n
s
tr
u
cts
m
an
y
d
ec
is
io
n
tr
ee
s
u
s
in
g
r
an
d
o
m
s
elec
tio
n
s
o
f
th
e
tr
ain
i
n
g
d
ata,
an
d
th
en
p
r
o
d
u
ce
s
a
class
th
at
is
th
e
av
er
ag
e
p
r
ed
ictio
n
(
r
eg
r
ess
io
n
)
o
r
th
e
m
o
d
e
o
f
th
e
class
es
(
class
if
icatio
n
)
o
f
ev
e
r
y
s
in
g
le
tr
ee
.
I
t
im
p
lies
m
o
r
e
u
n
p
r
ed
ictab
il
ity
an
d
b
o
o
s
ts
tr
ee
v
ar
iatio
n
b
y
co
n
s
id
er
i
n
g
ju
s
t a
f
r
ac
tio
n
o
f
th
e
q
u
alities
at
ea
ch
s
p
lit p
o
in
t in
th
e
d
ec
is
io
n
tr
e
e.
T
h
e
UNSW
-
NB
1
5
m
o
d
el
ev
al
u
atio
n
p
r
o
ce
d
u
r
e
u
s
ed
th
e
s
am
e
RF
ap
p
r
o
ac
h
t
o
g
u
ar
an
tee
c
o
n
s
is
ten
cy
in
th
e
r
esu
lts
.
No
ch
an
g
es
to
t
h
e
co
s
t
m
atr
ix
wer
e
m
ad
e
to
t
h
e
p
r
o
g
r
am
in
o
r
d
er
t
o
em
p
h
a
s
is
e
th
at
th
e
r
esu
lts
wer
e
o
n
ly
tr
ac
ea
b
le
to
th
e
r
ec
o
m
m
en
d
e
d
o
v
er
s
am
p
lin
g
m
eth
o
d
an
d
n
o
t th
e
lear
n
i
n
g
alg
o
r
ith
m
.
T
h
e
f
o
llo
win
g
m
etr
ics we
r
e
ta
k
en
in
to
ac
c
o
u
n
t t
o
ass
ess
th
e
m
o
d
el
’
s
class
if
icatio
n
p
er
f
o
r
m
an
ce
:
a)
C
o
n
f
u
s
io
n
m
atr
ix
-
i
t
is
a
co
m
p
r
eh
en
s
iv
e
p
r
esen
tatio
n
o
f
m
o
d
el
’
s
p
r
ed
ictio
n
s
in
a
m
atr
ix
f
o
r
m
lis
tin
g
th
e
n
u
m
b
er
o
f
tr
u
e
p
o
s
itiv
es (
T
P),
f
alse p
o
s
itiv
es (
FP
)
,
tr
u
e
n
eg
a
tiv
es (
T
N)
,
an
d
f
alse n
eg
ativ
es (
FN)
.
b)
Acc
u
r
ac
y
-
i
t
ca
lcu
lates
th
e
r
atio
o
f
co
r
r
ec
tly
p
r
e
d
icted
in
s
tan
ce
s
to
th
e
to
tal
n
u
m
b
e
r
o
f
in
s
tan
ce
s
in
th
e
d
ataset
to
esti
m
ate
th
e
m
o
d
el
’
s
p
er
f
o
r
m
a
n
ce
ac
r
o
s
s
all
class
es.
I
t is ca
lcu
lated
as
(
3
)
:
Ac
c
u
ra
c
y
=
(
+
)
(
+
+
+
)
(
3
)
c)
Pre
cisi
o
n
:
t
h
is
f
ac
to
r
f
o
cu
s
s
es
o
n
h
o
w
well
th
e
m
o
d
el
p
r
ed
icts
g
o
o
d
o
u
tco
m
es.
I
t
s
h
o
ws
th
at
th
e
m
o
d
el
ca
n
s
teer
clea
r
o
f
e
r
r
o
n
eo
u
s
p
o
s
itiv
e
p
r
ed
ictio
n
s
.
I
t is co
m
p
u
ted
as
(
4
)
:
P
re
c
isio
n
=
(
)
(
+
)
(
4
)
d)
R
ec
all
o
r
s
en
s
itiv
ity
-
i
t
f
o
cu
s
s
es
o
n
th
e
m
o
d
e
l
’
s
ab
ilit
y
to
i
d
en
tify
all
p
o
s
itiv
e
in
s
tan
ce
s
,
th
er
eb
y
r
e
d
u
cin
g
f
alse n
eg
ativ
es.
I
t is ca
lcu
lated
as
(
5
)
:
Re
c
a
ll
=
(
)
(
+
)
(
5
)
e)
F1
-
s
co
r
e
-
i
t
is
an
ef
f
ec
tiv
e
m
etr
ics
u
s
ed
with
im
b
aln
ce
d
d
a
ta
as
it
ca
lcu
lates
th
e
h
ar
m
o
n
i
c
m
ea
n
b
etwe
en
p
r
ec
is
io
n
an
d
r
ec
all.
I
t is d
eter
m
in
ed
as
(
6
)
:
F1
-
sc
o
re
=
2
(
)
(
+
)
(
6
)
f)
T
h
e
r
ec
eiv
er
o
p
er
atin
g
ch
ar
ac
t
er
is
tics
(
R
O
C
)
cu
r
v
e,
a
g
r
ap
h
i
ca
l d
ep
ictio
n
o
f
th
e
tr
ad
e
-
o
f
f
b
etwe
en
th
e
tr
u
e
p
o
s
itiv
e
r
at
e
(
s
en
s
itiv
ity
)
an
d
th
e
f
alse
p
o
s
itiv
e
r
ate
(
s
p
ec
i
f
icity
)
,
is
u
s
ed
to
ass
es
s
th
e
ef
f
ec
tiv
en
ess
o
f
b
in
ar
y
class
if
ier
s
.
As
a
s
ca
lar
n
u
m
b
er
,
AUC
p
r
o
v
id
es
an
o
v
er
v
iew
o
f
th
e
m
o
d
el
’
s
o
v
er
a
ll
p
er
f
o
r
m
a
n
ce
with
in
a
r
an
g
e
o
f
th
r
esh
o
ld
v
alu
es
v
ar
y
in
g
f
r
o
m
0
t
o
1
.
H
ig
h
er
th
e
AUC
v
alu
e,
th
e
b
et
ter
th
e
m
o
d
el
’
s
ab
ilit
y
to
d
is
cr
im
in
ate.
T
h
e
p
r
o
p
o
s
ed
PASMOT
E
ap
p
r
o
ac
h
was
s
tatis
tically
ev
alu
ated
ag
ain
s
t
f
o
u
r
o
v
er
s
am
p
ler
s
-
SMOT
E
,
ADASYN,
b
o
r
d
er
lin
e
SMOT
E
,
an
d
r
a
n
d
o
m
o
v
er
s
am
p
le
r
b
y
em
p
lo
y
i
n
g
th
r
ee
s
tatis
tical
t
ests
f
o
r
ev
alu
atin
g
m
o
d
els.
T
h
e
ex
p
er
im
e
n
t
was
co
n
d
u
cte
d
u
s
in
g
th
e
s
ec
o
n
d
d
ataset,
UNS
W
-
N
B
1
5
.
T
h
e
co
r
n
er
s
to
n
e
o
f
th
ese
test
s
is
th
e
co
n
ce
p
t
o
f
th
e
n
u
ll
h
y
p
o
th
esis
(
H0
)
an
d
t
h
e
alter
n
ativ
e
h
y
p
o
t
h
esis
(
H1
)
.
Ass
es
s
in
g
wh
eth
er
th
er
e
is
en
o
u
g
h
ev
id
e
n
ce
to
r
ef
u
te
th
e
n
u
l
l
h
y
p
o
th
esis
is
th
e
g
o
al
o
f
h
y
p
o
th
esis
test
in
g
,
wh
ich
d
eter
m
in
es
th
e
th
r
esh
o
ld
f
o
r
r
ejec
tio
n
d
ep
en
d
in
g
o
n
th
e
d
e
g
r
ee
o
f
s
ig
n
if
ican
ce
(
α
)
.
T
ab
le
4
d
escr
ib
es
th
ese
test
s
’
p
r
im
ar
y
f
ea
tu
r
es a
n
d
h
o
w
to
u
tili
s
e
th
em
to
co
m
p
ar
e
b
in
a
r
y
alg
o
r
ith
m
s
.
Ass
u
m
in
g
th
e
n
u
l
l
h
y
p
o
th
esis
is
tr
u
e,
th
e
test
s
g
en
er
ated
r
esu
lts
in
th
e
f
o
r
m
o
f
a
s
tatis
t
ical
v
alu
e
(
T
,
U,
an
d
W
)
alo
n
g
with
a
p
-
v
alu
e,
th
at
is
th
e
lik
elih
o
o
d
o
f
o
b
s
er
v
in
g
a
test
m
etr
ic
th
at
is
as
ex
tr
em
e
as
o
r
m
o
r
e
ex
tr
em
e
th
an
th
e
o
n
e
d
e
r
iv
ed
f
r
o
m
th
e
d
ata.
Sin
ce
0
.
0
5
is
th
e
th
r
esh
o
ld
lev
el
o
f
s
ig
n
if
ican
ce
,
p
-
v
al
u
es
less
th
an
th
is
m
a
y
b
e
in
ter
p
r
et
ed
as
s
u
p
p
o
r
tin
g
e
v
id
en
ce
ag
a
in
s
t
th
e
n
u
ll
h
y
p
o
t
h
esis
.
T
h
e
wh
o
le
o
p
er
atio
n
o
f
th
e
s
u
g
g
ested
m
o
d
el
is
s
h
o
wn
in
Fig
u
r
e
3
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
37
,
No
.
1
,
J
an
u
ar
y
20
25
:
51
8
-
5
29
524
T
ab
le
4
.
Sp
ec
if
ics o
f
th
e
s
tatis
tical
test
s
u
s
ed
C
r
i
t
e
r
i
o
n
P
a
i
r
e
d
T
-
t
e
s
t
M
a
n
n
-
w
h
i
t
n
e
y
U
t
e
st
W
i
l
c
o
x
o
n
s
i
g
n
e
d
-
r
a
n
k
t
e
st
N
a
t
u
r
e
o
f
t
e
st
P
a
r
a
me
t
r
i
c
N
o
n
-
p
a
r
a
m
e
t
r
i
c
N
o
n
-
p
a
r
a
m
e
t
r
i
c
T
y
p
e
o
f
d
a
t
a
P
r
e
su
m
p
t
i
v
e
l
y
d
i
s
t
r
i
b
u
t
e
d
d
i
s
p
a
r
i
t
i
e
s
C
o
m
p
a
r
i
n
g
t
w
o
i
n
d
e
p
e
n
d
e
n
t
s
a
mp
l
e
s
u
si
n
g
a
r
a
n
k
i
n
g
s
y
s
t
e
m
C
o
m
p
a
r
i
so
n
o
f
t
w
o
ma
t
c
h
e
d
samp
l
e
s
b
a
se
d
o
n
r
a
n
k
s
T
e
st
s
t
a
t
i
s
t
i
c
s
T
-
st
a
t
i
st
i
c
U
st
a
t
i
s
t
i
c
(
r
a
n
k
su
m)
W
st
a
t
i
st
i
c
(
s
u
m
o
f
s
i
g
n
e
d
r
a
n
k
s)
A
p
p
l
i
c
a
b
i
l
i
t
y
U
sed
t
o
c
o
m
p
a
r
e
ma
t
c
h
e
d
samp
l
e
mea
n
s
U
sed
t
o
c
o
m
p
a
r
e
t
h
e
me
d
i
a
n
o
f
t
w
o
i
n
d
e
p
e
n
d
e
n
t
sam
p
l
e
s
’
d
i
st
r
i
b
u
t
i
o
n
s
U
sed
t
o
c
o
m
p
a
r
e
t
h
e
t
w
o
ma
t
c
h
e
d
samp
l
e
s
’
d
i
s
t
r
i
b
u
t
i
o
n
s (m
e
d
i
a
n
)
S
a
mp
l
e
s
i
z
e
C
a
l
l
s f
o
r
a
b
i
g
sam
p
l
e
s
i
z
e
R
o
b
u
st
i
n
r
e
l
a
t
i
o
n
t
o
sa
mp
l
e
si
z
e
R
o
b
u
st
i
n
r
e
l
a
t
i
o
n
t
o
sa
mp
l
e
si
z
e
N
u
l
l
h
y
p
o
t
h
e
s
i
s
(
H
0
)
Th
e
r
e
i
s
n
o
d
i
sce
r
n
i
b
l
e
c
h
a
n
g
e
i
n
t
h
e
p
a
i
r
e
d
m
e
a
n
s
Th
e
d
i
st
r
i
b
u
t
i
o
n
s
o
f
t
h
e
t
w
o
sam
p
l
e
s
a
r
e
i
d
e
n
t
i
c
a
l
Th
e
r
e
i
s
n
o
d
i
st
i
n
c
t
i
o
n
b
e
t
w
e
e
n
t
h
e
p
a
i
r
e
d
o
b
ser
v
a
t
i
o
n
s
A
l
t
e
r
n
a
t
i
v
e
h
y
p
o
t
h
e
s
i
s (H
1
)
N
o
t
a
b
l
e
v
a
r
i
a
t
i
o
n
b
e
t
w
e
e
n
t
h
e
p
a
i
r
e
d
me
a
n
s
V
a
r
i
a
t
i
o
n
s i
n
t
h
e
d
i
st
r
i
b
u
t
i
o
n
b
e
t
w
e
e
n
t
h
e
t
w
o
s
a
mp
l
e
s
D
i
ssi
m
i
l
a
r
i
t
y
b
e
t
w
e
e
n
t
h
e
t
w
o
o
b
s
e
r
v
a
t
i
o
n
s
Fig
u
r
e
3
.
W
o
r
k
i
n
g
o
f
p
r
o
p
o
s
e
d
m
o
d
el
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
o
d
eter
m
in
e
th
e
m
o
d
el
’
s
ac
cu
r
ac
y
an
d
g
en
e
r
alis
atio
n
p
o
ten
tial,
ev
alu
atio
n
o
f
th
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
o
n
d
if
f
e
r
en
t
test
d
ata
m
u
s
t
b
e
u
s
ed
.
T
h
r
ee
m
et
h
o
d
s
h
av
e
b
ee
n
u
s
ed
in
th
is
wo
r
k
to
in
v
esti
g
ate
th
e
m
o
d
el.
Ap
p
r
o
ac
h
I
u
s
ed
th
e
f
ir
s
t
tr
ain
in
g
d
ataset,
wh
ich
h
ad
u
n
eq
u
al
class
d
is
tr
ib
u
tio
n
s
with
all
4
2
ch
ar
ac
ter
is
tics
,
to
tr
ain
th
e
m
o
d
el
u
s
in
g
t
h
e
d
ef
a
u
lt
h
y
p
e
r
p
ar
am
eter
.
Usi
n
g
C
FS
-
MH
A,
Ap
p
r
o
ac
h
I
I
ex
tr
ac
ted
1
0
r
ep
r
esen
tativ
e
f
ea
tu
r
es
f
r
o
m
th
e
u
n
b
alan
ce
d
tr
ain
in
g
s
et.
Ap
p
r
o
ac
h
I
I
I
u
s
ed
PASMOT
E
to
o
v
er
s
am
p
le
th
e
tr
ain
in
g
d
ataset,
ex
tr
ac
ted
1
5
ch
ar
ac
ter
is
tics
b
y
f
ea
tu
r
e
s
elec
tio
n
,
an
d
cr
ea
ted
th
e
as
s
e
s
s
m
en
t
m
o
d
el.
T
h
e
NSL
-
KDD
te
s
t+
d
atase
t
was
th
en
u
s
ed
to
co
m
p
ar
e
th
e
th
r
ee
m
eth
o
d
s
an
d
g
au
g
e
th
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
.
T
h
e
ef
f
ec
tiv
e
n
e
s
s
o
f
all
th
r
ee
o
f
th
ese
s
tr
ateg
ies
o
n
th
e
m
in
o
r
ity
class
es
u
2
r
an
d
r
2
l
is
s
ee
n
in
T
ab
le
5
.
T
ab
le
5
.
Ou
tco
m
es o
f
m
eth
o
d
s
tr
ied
f
o
r
m
i
n
o
r
ity
class
es in
th
e
NSL
-
KDD
ex
am
+
A
p
p
r
o
a
c
h
#
o
f
f
e
a
t
u
r
e
s
U
2
r
sam
p
l
e
s
R
2
l
sa
mp
l
e
s
O
v
e
r
a
l
l
a
c
c
u
r
a
c
y
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
I
41
7
0
.
7
3
0
.
0
0
0
.
0
0
0
.
0
0
0
.
0
0
1
1
.
0
0
0
.
0
0
1
II
10
7
1
.
3
1
0
.
1
8
0
.
8
0
0
.
1
8
0
.
0
8
0
.
6
0
0
.
0
8
III
15
7
8
.
7
6
0
.
4
2
0
.
5
3
0
.
4
2
0
.
8
3
0
.
4
8
0
.
8
3
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
A
n
o
ve
l b
o
u
n
d
a
r
y
a
d
a
p
tive
o
v
ers
a
mp
lin
g
a
p
p
r
o
a
c
h
fo
r
in
tr
u
s
io
n
d
etec
tio
n
(
R
itin
d
er K
a
u
r
)
525
T
h
e
r
esu
lts
d
em
o
n
s
tr
ate
th
at
Ap
p
r
o
ac
h
I
p
er
f
o
r
m
e
d
well
o
n
th
e
d
ataset
o
v
er
all,
with
an
o
v
er
all
ac
cu
r
ac
y
o
f
7
0
.
7
3
%,
wh
ich
is
p
r
etty
h
i
g
h
.
Ne
v
er
th
eless
,
th
e
p
er
f
o
r
m
an
ce
is
a
p
p
allin
g
f
o
r
t
h
e
m
in
o
r
ity
class
es
(
u
2
r
an
d
r
2
l
)
.
I
n
p
ar
ticu
lar
,
i
t
is
u
n
ab
le
to
ac
c
u
r
ately
id
e
n
tify
in
s
tan
ce
s
o
f
th
is
class
,
as
s
ee
n
b
y
its
0
%
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
an
d
r
ec
a
ll
wh
en
it
co
m
es
to
d
etec
tin
g
u
2
r
ass
au
lts
.
W
ith
r
esp
ec
t
to
R
2
L
ass
au
lts
,
its
ac
cu
r
ac
y
is
s
o
m
ewh
at
h
i
g
h
er
(
1
.
0
0
)
b
u
t
its
r
ec
all
is
v
er
y
lo
w
(
0
.
0
0
1
%),
in
d
icatin
g
th
at
it
d
etec
ts
th
e
b
u
lk
o
f
R
2
L
in
cid
en
ts
in
co
r
r
ec
tly
wh
il
e
ac
cu
r
ately
id
en
tify
in
g
r
elativ
ely
f
ew.
Acc
o
r
d
in
g
to
m
eth
o
d
I
I
,
th
e
ac
cu
r
ac
y
is
im
p
r
o
v
e
d
to
7
1
.
3
1
%
o
v
er
all
b
y
lim
itin
g
th
e
n
u
m
b
er
o
f
ch
ar
ac
ter
is
tics
to
1
0
,
wh
ich
is
a
litt
le
im
p
r
o
v
em
en
t
o
v
er
way
I
.
Alth
o
u
g
h
th
er
e
h
a
s
b
ee
n
s
o
m
e
p
r
o
g
r
ess
,
u
2
r
d
et
ec
tio
n
lev
els
ar
e
s
till
r
ath
er
lo
w.
Similar
ly
,
r
ec
all
an
d
ac
cu
r
ac
y
o
f
r
2
l
d
etec
tio
n
s
ig
n
if
ican
tly
in
cr
ea
s
e,
b
u
t
o
v
e
r
all
p
er
f
o
r
m
an
ce
r
em
ain
s
s
u
b
p
ar
.
T
h
e
tech
n
iq
u
e
with
th
e
b
est
o
v
e
r
all
ac
cu
r
ac
y
(
7
8
.
7
6
%)
is
tech
n
iq
u
e
I
I
I
,
wh
ich
h
as
1
5
ch
ar
ac
ter
is
tics
.
T
h
is
r
ep
r
esen
ts
a
s
u
b
s
tan
tial
im
p
r
o
v
em
en
t
o
v
er
th
e
ea
r
lier
ap
p
r
o
ac
h
es.
C
o
m
p
ar
ed
to
ea
r
lier
m
eth
o
d
s
,
U2
r
d
etec
tio
n
ex
h
ib
its
a
d
is
ce
r
n
ib
le
im
p
r
o
v
em
e
n
t
in
ac
c
u
r
ac
y
,
p
r
ec
is
io
n
,
an
d
r
ec
all.
C
o
m
p
ar
ed
to
Ap
p
r
o
ac
h
I
I
,
r
2
l
d
etec
tio
n
ex
h
ib
its
a
s
ig
n
if
ican
t
g
ain
in
ac
cu
r
ac
y
a
n
d
r
ec
all,
b
u
t
at
th
e
e
x
p
en
s
e
o
f
s
o
m
ewh
at
lo
wer
p
r
ec
is
io
n
.
W
h
en
it
ca
m
e
t
o
o
v
er
all
ac
cu
r
ac
y
an
d
ad
v
an
ce
m
en
ts
in
id
en
tify
in
g
m
in
o
r
ity
class
e
s
(
U2
R
an
d
R
2
L
ass
a
u
lts
)
,
Ap
p
r
o
ac
h
I
I
I
o
u
tp
er
f
o
r
m
ed
th
e
o
th
er
two
.
T
h
e
r
esu
lts
wer
e
co
n
tr
asted
wit
h
s
tate
-
of
-
th
e
-
a
r
t
s
tu
d
ies
in
th
e
liter
atu
r
e
to
ass
ess
th
e
ef
f
ec
tiv
en
ess
o
f
th
is
s
tr
ate
g
y
ev
e
n
f
u
r
t
h
er
.
T
a
b
le
6
p
r
esen
ts
th
e
f
in
d
in
g
s
.
T
ab
le
6
.
C
o
m
p
a
r
in
g
th
e
s
u
g
g
e
s
ted
m
eth
o
d
with
th
e
m
o
s
t r
ec
en
t r
esear
ch
A
u
t
h
o
r
Y
e
a
r
T
e
c
h
n
i
q
u
e
I
mb
a
l
a
n
c
e
d
a
p
p
r
o
a
c
h
C
l
a
s
si
f
i
c
a
t
i
o
n
st
r
a
t
e
g
y
R
e
s
u
l
t
s
u2r
r
2
l
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
D
o
u
z
a
s
e
t
a
l
.
[
1
7
]
2
0
2
0
S
i
a
m
-
IDS
A
d
a
p
t
i
v
e
n
e
u
r
a
l
n
e
t
w
o
r
k
a
p
p
r
o
a
c
h
t
h
a
t
u
se
s
Eu
c
l
i
d
i
a
n
d
i
s
t
a
n
c
e
t
o
c
a
l
c
u
l
a
t
e
sam
p
l
e
si
mi
l
a
r
i
t
y
D
N
N
,
C
N
N
1
0
.
1
1
5
6
.
7
2
5
7
.
9
4
3
3
.
2
5
F
a
r
q
u
a
d
a
n
d
B
o
s
e
[
1
8
]
2
0
2
1
GAN
-
b
a
se
d
O
v
e
r
sam
p
l
i
n
g
P
r
o
d
u
c
e
d
a
r
t
i
f
i
c
i
a
l
samp
l
e
s
u
s
i
n
g
G
e
n
e
r
a
t
i
v
e
A
d
v
e
r
sari
a
l
N
e
t
w
o
r
k
s
U
sed
a
t
h
r
e
e
-
l
a
y
e
r
e
d
,
c
o
st
-
sen
s
i
t
i
v
e
A
N
N
1
94
0
0
K
N
N
b
a
se
d
o
v
e
r
sam
p
l
i
n
g
K
N
N
i
n
t
e
r
p
o
l
a
t
e
d
b
e
t
w
e
e
n
t
h
e
m
i
n
o
r
i
t
y
samp
l
e
s
t
h
a
t
w
e
r
e
a
l
r
e
a
d
y
t
h
e
r
e
.
2
78
23
10
F
u
e
t
a
l
.
[
3
1
]
2
0
2
2
D
LN
I
D
A
D
A
S
Y
N
w
a
s u
se
d
f
o
r
o
v
e
r
sam
p
l
i
n
g
Bi
-
LST
M
-
24
-
6
5
.
7
6
W
u
e
t
a
l
.
[
3
2
]
2
0
2
2
En
h
a
n
c
e
d
R
F
K
-
mea
n
s
c
o
m
b
i
n
e
d
w
i
t
h
S
M
O
TE
I
mp
r
o
v
e
d
RF
u
si
n
g
a
si
mi
l
a
r
i
t
y
mat
r
i
x
2
6
.
5
0
2
6
.
5
0
3
0
.
6
3
3
0
.
6
3
Y
o
o
n
a
n
d
K
i
m [3
3
]
2
0
2
3
S
M
O
TE
S
M
O
TE
w
a
s
u
se
d
t
o
e
x
a
mi
n
e
t
h
e
e
f
f
e
c
t
o
f
f
e
a
t
u
r
e
r
e
d
u
c
t
i
o
n
.
RF
83
7
27
28
A
r
ı
k
a
n
d
Ç
a
v
d
a
r
o
ğ
l
u
[
3
4
]
2
0
2
3
R
O
G
O
N
G
-
IDS
T
o
a
c
h
i
e
v
e
b
a
l
a
n
c
e
,
o
v
e
r
sam
p
l
i
n
g
S
M
O
TE
a
n
d
u
n
d
e
r
sam
p
l
i
n
g
N
e
a
r
-
M
i
ss
w
e
r
e
c
o
mb
i
n
e
d
.
X
G
B
o
o
st
-
10
-
39
K
a
u
r
a
n
d
G
u
p
t
a
2
0
2
4
P
r
o
p
o
se
d
a
p
p
r
o
a
c
h
I
A
b
se
n
c
e
o
f
f
e
a
t
u
r
e
sel
e
c
t
i
o
n
a
n
d
o
v
e
r
sam
p
l
i
n
g
t
e
c
h
n
i
q
u
e
RF
0
0
1
.
0
0
.
0
0
1
K
a
u
r
a
n
d
G
u
p
t
a
2
0
2
4
P
A
S
M
O
TE
(
P
r
o
p
o
se
d
a
p
p
r
o
a
c
h
I
I
I
)
U
si
n
g
d
e
c
i
s
i
o
n
-
b
o
u
n
d
a
r
y
p
r
o
x
i
m
i
t
y
a
n
d
c
h
a
n
g
i
n
g
samp
l
i
n
g
t
h
r
e
s
h
o
l
d
v
a
l
u
e
s
t
o
a
c
h
i
e
v
e
o
v
e
r
sam
p
l
i
n
g
C
o
s
t
-
c
o
n
s
c
i
o
u
s
RF
53
42
48
83
T
h
e
co
m
p
ar
is
o
n
r
esu
lts
d
em
o
n
s
tr
ate
h
o
w
m
u
ch
wo
r
k
is
s
ti
ll
b
ein
g
d
o
n
e
in
th
e
a
r
ea
o
f
u
n
b
alan
ce
d
lear
n
in
g
in
in
tr
u
s
io
n
d
etec
tio
n
u
s
in
g
th
e
NSL
-
KDD
d
atas
et.
E
v
en
wh
ile
s
ev
er
al
o
f
th
e
ex
am
in
ed
ar
ticles
r
ep
o
r
t
s
u
p
er
io
r
o
v
er
all
ac
cu
r
a
cy
,
th
eir
b
iases
in
f
av
o
u
r
o
f
th
e
m
ajo
r
ity
class
ar
e
th
e
m
a
in
r
ea
s
o
n
f
o
r
th
eir
s
u
cc
ess
.
Do
u
za
s
et
a
l.
[
1
7
]
,
Far
q
u
ad
an
d
B
o
s
e
[
1
8
]
d
em
o
n
s
tr
ate
in
cr
ea
s
ed
r
ec
all
r
ates
f
o
r
u
2
r
s
am
p
les;
n
o
n
eth
eless
,
th
eir
ac
c
u
r
ac
y
r
at
es
p
ale
in
co
m
p
ar
is
o
n
t
o
th
e
s
u
g
g
ested
m
eth
o
d
o
l
o
g
y
.
On
th
e
o
th
er
h
an
d
,
wh
ile
[
3
3
]
h
av
e
g
r
ea
ter
ac
cu
r
ac
y
r
at
es,
th
eir
r
ec
all
lev
el
is
ju
s
t
7
%,
wh
er
ea
s
th
e
s
u
g
g
ested
m
et
h
o
d
’
s
r
ec
all
le
v
el
is
4
2
%.
T
h
e
liter
atu
r
e
s
u
g
g
est
s
th
at
ad
ap
tiv
e
alg
o
r
ith
m
s
with
co
s
t
weig
h
t
a
d
ju
s
tm
en
ts
h
av
e
also
b
ee
n
ex
ten
s
iv
ely
s
tu
d
ied
f
o
r
m
an
a
g
in
g
im
b
alan
ce
s
;
n
e
v
er
th
eless
,
th
e
s
u
g
g
ested
m
eth
o
d
,
PASMOT
E
with
co
s
t
-
s
en
s
itiv
e
lear
n
in
g
,
p
er
f
o
r
m
s
b
etter
th
an
th
e
s
tate
-
of
-
th
e
-
ar
t t
ec
h
n
iq
u
es [
1
8
]
,
[
3
2
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
37
,
No
.
1
,
J
an
u
ar
y
20
25
:
51
8
-
5
29
526
T
h
e
tr
ain
in
g
d
ataset
o
f
UNSW
-
NB
1
5
was
in
itially
o
v
er
s
a
m
p
led
u
s
in
g
all
th
e
f
iv
e
o
v
e
r
s
am
p
ler
s
:
SMOT
E
,
ADASY
N,
b
o
r
d
er
li
n
e
SMOT
E
,
r
an
d
o
m
o
v
er
s
am
p
ler
,
an
d
PASMOT
E
.
RF
cla
s
s
if
ier
s
er
v
ed
as
th
e
f
o
u
n
d
atio
n
al
class
if
ier
f
o
r
b
o
t
h
tr
ain
in
g
a
n
d
ass
ess
m
en
t.
K
-
f
o
ld
cr
o
s
s
v
alid
atio
n
was
ca
r
r
ie
d
o
u
t,
with
a
v
alu
e
o
f
k
eq
u
al
to
1
0
,
in
o
r
d
er
to
p
r
o
v
id
e
ac
c
u
r
ate
p
er
f
o
r
m
an
ce
esti
m
ates
an
d
to
g
en
er
alis
e
th
e
m
o
d
el
to
en
co
m
p
ass
all
ca
s
es
th
r
o
u
g
h
o
u
t
th
e
tr
ain
in
g
p
h
ase.
T
ab
le
7
p
r
esen
ts
th
e
co
n
f
u
s
io
n
m
atr
ix
v
alu
es
ca
lcu
lated
u
s
in
g
th
e
f
iv
e
d
is
tin
ct
o
v
er
s
am
p
lin
g
tech
n
i
q
u
es.
T
h
e
p
e
r
f
o
r
m
a
n
c
e
m
e
t
r
i
cs
f
o
r
cl
a
s
s
l
a
b
el
w
o
r
m
s
c
al
c
u
l
a
te
d
o
n
t
h
e
s
e
f
i
v
e
m
o
d
e
ls
a
r
e
s
h
o
w
n
in
T
a
b
l
e
8
.
T
h
e
f
in
d
in
g
s
clea
r
ly
r
ev
ea
l
t
h
at
PASMOT
E
h
as
th
e
g
r
ea
test
r
ec
all
v
alu
e
0
.
7
3
an
d
p
r
ec
is
io
n
o
f
0
.
8
2
lead
in
g
to
an
elev
ated
AUC
v
alu
e
o
f
0
.
8
6
an
d
a
b
alan
ce
d
F1
-
s
co
r
e
0
.
7
7
.
T
h
e
r
an
d
o
m
o
v
er
s
am
p
ler
h
as
a
g
o
o
d
r
ec
all
b
u
t
a
p
o
o
r
ac
c
u
r
ac
y
.
Alth
o
u
g
h
b
o
r
d
er
lin
e
SMOT
E
h
as
a
lar
g
er
p
r
o
b
ab
ilit
y
o
f
f
alse
p
o
s
itiv
es
d
u
e
t
o
its
p
o
o
r
er
ac
cu
r
ac
y
,
it p
er
f
o
r
m
s
well
in
r
ec
all,
s
u
g
g
esti
n
g
im
p
r
o
v
e
d
d
et
ec
tio
n
o
f
attac
k
ep
is
o
d
es.
Sim
ilar
p
er
f
o
r
m
an
ce
is
s
h
o
wn
b
y
ADASYN
an
d
SM
OT
E
,
in
d
icatin
g
a
c
o
m
p
ar
ab
le
tr
ad
e
-
o
f
f
b
etwe
en
r
ec
all
an
d
a
cc
u
r
ac
y
.
T
h
e
R
OC
cu
r
v
e
f
o
r
th
ese
m
o
d
els
is
s
h
o
wn
in
Fig
u
r
e
4
s
o
th
at
th
e
tr
u
e
p
o
s
itiv
e
a
n
d
f
als
e
p
o
s
itiv
e
r
ates
o
v
er
a
r
an
g
e
o
f
th
r
esh
o
ld
v
alu
es
ca
n
b
e
s
ee
n
.
AU
C
v
alu
es
th
at
ar
e
cl
o
s
er
to
0
.
5
in
d
icate
r
an
d
o
m
ch
an
ce
s
,
wh
ile
th
o
s
e
th
at
a
r
e
clo
s
er
to
1
,
0
in
d
icate
p
er
f
ec
t
d
is
cr
im
in
atio
n
.
As
a
r
esu
lt,
m
o
d
els
n
ea
r
er
th
e
to
p
-
lef
t c
o
r
n
er
o
f
th
e
g
r
ap
h
o
u
tp
er
f
o
r
m
o
th
e
r
m
o
d
els.
T
h
e
n
o
tio
n
th
at
PASM
OT
E
o
u
tp
er
f
o
r
m
s
o
th
er
cu
r
r
e
n
t state
-
of
-
th
e
-
a
r
t te
ch
n
iq
u
es is
s
u
p
p
o
r
ted
b
y
th
e
r
o
c
c
u
r
v
e
f
in
d
in
g
s
.
T
ab
le
7
.
UNSW
_
NB
1
5
d
ataset
’
s
co
n
f
u
s
io
n
m
at
r
ix
af
ter
b
ala
n
cin
g
u
tili
zin
g
f
iv
e
tr
ied
-
an
d
-
t
r
u
e
m
eth
o
d
s
S
M
O
TE
A
D
A
S
Y
N
B
o
r
d
e
r
l
i
n
e
S
M
O
TE
R
a
n
d
o
m
o
v
e
r
sa
mp
l
e
r
P
A
S
M
O
TE
N
o
r
mal
a
s
n
o
r
ma
l
3
6
9
6
8
3
6
9
6
6
3
6
9
6
1
3
6
9
9
3
3
6
9
9
3
N
o
r
mal
a
s
w
o
r
ms
32
34
39
7
7
W
o
r
ms a
s
n
o
r
m
a
l
17
17
16
18
12
W
o
r
ms a
s
w
o
r
ms
27
27
28
26
32
T
ab
le
8
.
Fin
d
in
g
s
u
s
in
g
a
RF
class
if
ier
o
n
th
e
UNSW
-
NB
1
5
test
s
et
f
o
r
th
e
wo
r
m
’
s
class
O
v
e
r
sam
p
l
i
n
g
t
e
c
h
n
i
q
u
e
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F1
-
sc
o
r
e
AUC
S
M
O
TE
0
.
4
6
0
.
6
1
0
.
5
2
0
.
8
1
A
D
A
S
Y
N
0
.
4
4
0
.
6
1
0
.
5
1
0
.
8
1
B
o
r
d
e
r
l
i
n
e
S
M
O
TE
0
.
4
2
0
.
6
4
0
.
5
0
0
.
8
2
R
a
n
d
o
m
o
v
e
r
sa
mp
l
e
r
0
.
7
9
0
.
5
9
0
.
6
8
0
.
8
0
P
A
S
M
O
TE
0
.
8
2
0
.
7
3
0
.
7
7
0
.
8
6
Fig
u
r
e
4
.
R
OC
cu
r
v
e
f
o
r
f
i
v
e
m
o
d
els o
n
UNSW
-
NB
1
5
test
s
et
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
A
n
o
ve
l b
o
u
n
d
a
r
y
a
d
a
p
tive
o
v
ers
a
mp
lin
g
a
p
p
r
o
a
c
h
fo
r
in
tr
u
s
io
n
d
etec
tio
n
(
R
itin
d
er K
a
u
r
)
527
T
h
ese
f
iv
e
m
o
d
els
wer
e
s
u
b
jecte
d
to
s
tatis
tical
te
s
ts
to
s
ee
if
th
e
p
er
f
o
r
m
a
n
ce
d
if
f
er
e
n
ce
s
b
etwe
en
th
em
ar
e
s
tatis
tically
s
ig
n
if
ican
t
o
r
m
ay
h
av
e
h
ap
p
en
ed
b
y
ac
cid
en
t.
T
h
e
g
en
er
alis
atio
n
o
f
th
ese
m
o
d
els
o
u
ts
id
e
o
f
p
ar
ticu
lar
d
atasets
is
also
aid
ed
b
y
th
es
e
tes
ts
.
T
h
e
th
r
ee
s
tatis
tical
test
s
co
v
er
ed
in
wer
e
u
s
ed
to
ev
alu
ate
th
e
s
u
g
g
ested
m
o
d
el
with
all
alter
n
ativ
e
o
v
er
s
am
p
lin
g
tech
n
iq
u
es.
T
a
b
le
9
p
r
es
en
ts
th
e
test
r
esu
lt
s
alo
n
g
with
th
eir
s
tatis
tics
an
d
p
-
v
alu
es.
T
ab
le
9
.
Fin
d
in
g
s
f
r
o
m
s
tatis
ti
ca
l te
s
ts
ap
p
lied
o
n
UNSW
-
N
B
1
5
o
v
er
s
am
p
led
d
ataset
C
o
m
p
a
r
e
d
mo
d
e
l
s
P
a
i
r
e
d
t
-
t
e
s
t
M
a
n
n
–
W
h
i
t
n
e
y
U
t
e
st
W
i
l
c
o
x
o
n
t
e
s
t
s
i
g
n
e
d
r
a
n
k
t
e
s
t
t
-
st
a
t
i
s
t
i
c
p
-
v
a
l
u
e
p
<
0
.
0
5
t
-
st
a
t
i
s
t
i
c
p
-
v
a
l
u
e
p
<
0
.
0
5
t
-
st
a
t
i
s
t
i
c
p
-
v
a
l
u
e
p
<
0
.
0
5
P
A
S
M
O
TE
v
s.
S
M
O
T
E
-
3
.
5
4
0
.
0
0
0
4
Y
e
s
6
8
5
7
0
.
0
4
3
Y
e
s
99
0
.
0
0
0
4
Y
e
s
P
A
S
M
O
TE
v
s.
A
D
A
S
Y
N
-
3
.
6
7
0
.
0
0
0
2
Y
e
s
6
8
5
7
0
.
0
2
7
Y
e
s
1
2
9
0
.
0
0
0
2
Y
e
s
P
A
S
M
O
TE
v
s.
B
o
r
d
e
r
l
i
n
e
S
M
O
TE
-
4
.
5
4
0
.
0
0
0
0
Y
e
s
6
8
5
6
0
.
0
0
6
Y
e
s
9
7
.
5
0
.
0
0
0
0
Y
e
s
P
A
S
M
O
TE
v
s.
R
a
n
d
o
m
o
v
e
r
sam
p
l
e
r
2
.
4
4
0
.
0
1
4
Y
e
s
6
8
6
2
0
.
4
7
9
No
0
.
0
0
.
0
1
4
3
Y
e
s
T
h
e
ap
p
lied
s
tatis
tical
test
s
d
em
o
n
s
tr
ate
th
e
PASMOT
E
m
o
d
el
’
s
r
esil
ien
ce
to
b
o
th
s
k
ewe
d
an
d
n
o
r
m
al
d
is
tr
ib
u
tio
n
s
.
Sin
ce
all
th
r
ee
test
s
an
d
m
o
d
els
h
av
e
p
-
v
alu
es
less
th
an
0
.
0
5
,
th
e
ch
o
s
en
s
ig
n
if
ican
c
e
lev
el
f
o
r
r
ejec
tin
g
th
e
n
u
ll
h
y
p
o
th
esis
,
all
th
r
ee
test
s
h
av
e
s
u
b
s
tan
tially
r
ef
u
ted
th
e
n
u
ll h
y
p
o
th
esis
,
wh
ich
h
eld
th
at
th
er
e
is
n
o
s
tatis
tical
s
ig
n
if
ican
ce
b
etwe
en
th
ese
m
o
d
el
s
.
T
h
e
lo
n
e
ex
ce
p
tio
n
is
th
e
Ma
n
n
W
h
itn
ey
test
,
wh
er
e
less
s
tatis
t
ical
s
ig
n
if
ican
ce
is
s
u
g
g
ested
in
th
ese
m
o
d
els
ac
r
o
s
s
in
d
ep
en
d
en
t
s
am
p
les
wh
en
th
e
p
-
v
alu
e
is
g
r
ea
ter
th
an
0
.
0
5
b
etwe
en
th
e
p
r
o
p
o
s
ed
m
o
d
el
an
d
r
an
d
o
m
o
v
er
s
am
p
ler
.
4
CO
NCLU
SI
O
N
AND
F
U
T
U
RE
SCO
P
E
T
h
e
o
b
jectiv
e
o
f
th
is
s
tu
d
y
is
to
in
cr
ea
s
e
th
e
s
ec
u
r
ity
o
f
co
m
p
u
ter
n
etwo
r
k
s
ag
ain
s
t
th
e
ev
er
-
in
cr
ea
s
in
g
th
r
ea
t
o
f
c
y
b
er
attac
k
s
b
y
ad
v
a
n
cin
g
I
DS
v
ia
th
e
p
r
o
p
o
s
al
o
f
an
ad
a
p
tiv
e
o
v
er
s
am
p
lin
g
tech
n
i
q
u
e.
T
h
ese
v
er
s
io
n
s
p
r
o
v
i
d
e
a
v
iab
le
way
to
o
v
er
co
m
e
u
n
b
alan
c
ed
d
ata
d
if
f
icu
lties
an
d
im
p
r
o
v
e
d
ec
is
io
n
-
m
ak
i
n
g
p
r
o
ce
s
s
es
in
s
ev
er
al
ar
ea
s
b
y
in
clu
d
in
g
d
ec
is
io
n
lim
it
s
an
d
ca
teg
o
r
is
atio
n
i
n
f
o
r
m
atio
n
in
to
t
h
e
s
am
p
lin
g
p
r
o
ce
s
s
.
T
h
e
s
u
g
g
ested
m
eth
o
d
s
h
o
we
d
an
in
cr
ea
s
e
in
u
n
co
m
m
o
n
c
lass
id
en
tific
atio
n
p
e
r
f
o
r
m
an
c
e
b
y
u
s
in
g
th
e
b
o
u
n
d
ar
y
p
r
o
x
im
ity
id
ea
i
n
o
v
e
r
s
am
p
lin
g
.
T
h
is
d
o
es
a
g
r
ea
t
jo
b
o
f
ca
p
tu
r
in
g
th
e
n
u
a
n
ce
s
an
d
co
m
p
le
x
ity
th
at
ar
e
s
p
ec
if
ic
to
th
e
m
in
o
r
ity
class
,
p
ar
ticu
lar
ly
wh
en
it
co
m
es
to
th
e
d
ec
is
io
n
b
o
r
d
e
r
.
T
h
e
ex
p
e
r
im
en
t
in
v
esti
g
ated
s
ev
er
al
th
r
esh
o
ld
s
ettin
g
s
f
o
r
th
e
r
esam
p
lin
g
p
r
o
ce
d
u
r
e
in
o
r
d
e
r
to
o
p
tim
is
e
h
y
p
er
p
a
r
am
eter
s
.
W
ith
ea
ch
cy
cle
o
f
o
v
er
s
am
p
lin
g
,
t
h
is
h
y
b
r
id
alg
o
r
ith
m
ex
h
ib
its
f
lex
ib
ilit
y
b
y
d
y
n
am
ically
ad
ju
s
tin
g
an
d
ch
o
o
s
in
g
n
ew
in
s
tan
ce
s
f
o
r
s
am
p
lin
g
.
I
t
is
n
o
tewo
r
th
y
t
h
a
t
it
p
lace
s
m
o
r
e
em
p
h
asis
o
n
cr
ea
tin
g
s
y
n
th
etic
s
am
p
les
in
ar
ea
s
n
ea
r
er
th
e
d
e
cisi
o
n
b
o
r
d
er
,
r
ed
u
ci
n
g
th
e
p
o
s
s
ib
ilit
y
o
f
o
v
er
g
en
er
a
lis
atio
n
an
d
im
p
r
o
v
in
g
t
h
e
d
if
f
er
en
tiatio
n
b
etwe
en
class
es.
An
in
tr
in
s
ic
b
en
ef
it
o
f
ea
r
ly
h
altin
g
a
n
d
m
i
n
im
is
in
g
o
v
er
f
itti
n
g
an
d
u
n
d
er
f
itti
n
g
is
p
r
o
v
id
ed
b
y
th
e
u
s
e
o
f
p
r
u
n
ed
d
ec
is
io
n
tr
ee
s
f
o
r
b
o
u
n
d
ar
y
g
e
n
er
atio
n
.
U
n
lik
e
SMOT
E
,
th
e
m
eth
o
d
d
o
es
n
o
t
ch
o
o
s
e
s
am
p
le
p
o
in
ts
at
r
an
d
o
m
f
r
o
m
th
e
f
ea
tu
r
e
s
p
ac
e,
wh
ich
lo
wer
s
th
e
p
o
s
s
ib
ilit
y
o
f
ad
d
in
g
n
o
is
e
to
th
e
s
am
p
les
th
at
ar
e
ch
o
s
en
.
T
h
e
r
esu
lts
o
f
s
tatis
tical
te
s
tin
g
h
ig
h
lig
h
t
h
o
w
r
esil
ien
t
an
d
g
en
er
alis
ab
le
th
is
tech
n
iq
u
e
is
o
v
er
a
wid
e
r
an
g
e
o
f
a
r
ea
s
.
E
v
en
wit
h
th
e
alg
o
r
ith
m
’
s
s
atis
f
ac
to
r
y
p
er
f
o
r
m
a
n
ce
,
th
e
r
e
is
s
till
r
o
o
m
f
o
r
im
p
r
o
v
em
e
n
t
i
n
ter
m
s
o
f
ac
cu
r
ately
d
etec
tin
g
t
h
ese
u
n
u
s
u
al
g
r
o
u
p
s
.
I
t
m
ay
b
e
p
o
s
s
ib
le
to
im
p
r
o
v
e
th
e
id
e
n
tific
atio
n
o
f
th
ese
m
in
o
r
it
y
class
es
b
y
f
in
e
-
tu
n
in
g
o
r
e
x
p
er
im
en
tin
g
with
o
t
h
er
m
o
d
e
llin
g
s
tr
ateg
ies.
I
t
is
p
o
s
s
ib
le
to
in
v
esti
g
ate
th
e
am
o
u
n
t
o
f
r
ea
lis
tic
d
ata
g
en
e
r
atio
n
in
f
u
r
th
er
r
esear
c
h
.
Fu
r
th
er
r
esear
ch
in
th
e
d
o
m
ain
o
f
d
ec
is
io
n
-
b
ased
SMOT
E
v
ar
ian
ts
is
an
ticip
ated
to
d
r
i
v
e
ad
v
a
n
ce
m
en
ts
in
m
ac
h
in
e
lear
n
in
g
b
r
ea
k
th
r
o
u
g
h
s
an
d
r
esu
lt
in
m
o
r
e
ac
cu
r
ate
an
d
d
e
p
en
d
a
b
le
p
r
e
d
ictio
n
m
o
d
els
as
th
e
ar
ea
o
f
u
n
b
alan
ce
d
d
ata
s
am
p
lin
g
c
o
n
tin
u
es
to
ex
p
an
d
.
T
h
r
o
u
g
h
th
e
u
s
e
o
f
ex
is
tin
g
t
o
o
ls
an
d
r
eso
u
r
ce
s
,
p
r
ac
titi
o
n
er
s
m
ay
o
p
tim
is
e
th
e
p
o
ten
tia
l
o
f
d
ec
is
io
n
-
b
ased
SMOT
E
v
ar
ian
ts
to
en
h
an
ce
u
n
b
alan
ce
d
d
ata
s
am
p
lin
g
an
d
en
h
an
ce
m
ac
h
in
e
lear
n
in
g
m
o
d
el
p
er
f
o
r
m
an
ce
in
v
ar
io
u
s
ap
p
licatio
n
s
.
RE
F
E
R
E
NC
E
S
[
1
]
Q
.
Z
h
o
u
,
Y
.
Q
i
,
H
.
Ta
n
g
,
a
n
d
P
.
W
u
,
“
M
a
c
h
i
n
e
l
e
a
r
n
i
n
g
-
b
a
s
e
d
p
r
o
c
e
ssi
n
g
o
f
u
n
b
a
l
a
n
c
e
d
d
a
t
a
se
t
s
f
o
r
c
o
m
p
u
t
e
r
a
l
g
o
r
i
t
h
m
s,
”
O
p
e
n
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
v
o
l
.
1
3
,
n
o
.
1
,
J
a
n
.
2
0
2
3
,
d
o
i
:
1
0
.
1
5
1
5
/
c
o
m
p
-
2
0
2
2
-
0
2
7
3
.
[
2
]
B
.
K
r
a
w
c
z
y
k
,
“
Le
a
r
n
i
n
g
f
r
o
m
i
m
b
a
l
a
n
c
e
d
d
a
t
a
:
o
p
e
n
c
h
a
l
l
e
n
g
e
s
a
n
d
f
u
t
u
r
e
d
i
r
e
c
t
i
o
n
s,
”
Pr
o
g
ress
i
n
Art
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
v
o
l
.
5
,
n
o
.
4
,
p
p
.
2
2
1
–
2
3
2
,
A
p
r
.
2
0
1
6
,
d
o
i
:
1
0
.
1
0
0
7
/
s
1
3
7
4
8
-
0
1
6
-
0
0
9
4
-
0.
[
3
]
D
.
El
r
e
e
d
y
a
n
d
A
.
F
.
A
t
i
y
a
,
“
A
c
o
m
p
r
e
h
e
n
si
v
e
a
n
a
l
y
s
i
s
o
f
sy
n
t
h
e
t
i
c
mi
n
o
r
i
t
y
o
v
e
r
s
a
mp
l
i
n
g
t
e
c
h
n
i
q
u
e
(
S
M
O
TE)
f
o
r
h
a
n
d
l
i
n
g
c
l
a
ss
i
m
b
a
l
a
n
c
e
,
”
I
n
f
o
rm
a
t
i
o
n
S
c
i
e
n
c
e
s
,
v
o
l
.
5
0
5
,
p
p
.
3
2
–
6
4
,
D
e
c
.
2
0
1
9
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
i
n
s
.
2
0
1
9
.
0
7
.
0
7
0
.
Evaluation Warning : The document was created with Spire.PDF for Python.