I
nte
rna
t
io
na
l J
o
urna
l o
f
I
nfo
rm
a
t
ics a
nd
Co
m
m
un
ica
t
io
n T
ec
hn
o
lo
g
y
(
I
J
-
I
CT
)
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
,
p
p
.
477
~
4
8
7
I
SS
N:
2252
-
8
7
7
6
,
DOI
:
1
0
.
1
1
5
9
1
/iji
ct
.
v
1
5
i
2
.
pp
477
-
4
8
7
477
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ict.
ia
esco
r
e.
co
m
Stacking
of
ma
chi
ne learning
clas
sif
iers for bot
de
tectio
n using
a
ccou
nt
lev
el data
J
wa
la
Sh
a
rm
a
,
Sa
ma
rj
ee
t
B
o
ra
h
D
e
p
a
r
t
me
n
t
o
f
C
o
mp
u
t
e
r
A
p
p
l
i
c
a
t
i
o
n
s,
S
i
k
k
i
m
M
a
n
i
p
a
l
I
n
st
i
t
u
t
e
o
f
T
e
c
h
n
o
l
o
g
y
,
S
i
k
k
i
m
M
a
n
i
p
a
l
U
n
i
v
e
r
si
t
y
,
G
a
n
g
t
o
k
,
I
n
d
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
an
1
6
,
2
0
2
5
R
ev
is
ed
No
v
1
4
,
2
0
2
5
Acc
ep
ted
Dec
1
4
,
2
0
2
5
S
o
c
ial
m
e
d
ia
is
a
p
latfo
rm
fo
r
in
d
i
v
id
u
a
ls
to
c
o
n
n
e
c
t,
sh
a
re
,
a
n
d
c
re
a
te
in
fo
rm
a
ti
o
n
.
S
o
c
ial
b
o
ts
p
ro
d
u
c
e
a
u
to
m
a
ted
c
o
n
te
n
t
a
n
d
i
n
tera
c
t
with
h
u
m
a
n
s;
i
n
th
e
p
r
o
c
e
ss
,
th
e
y
le
a
rn
a
n
d
m
imic
h
u
m
a
n
s’
b
e
h
a
v
i
o
u
r.
T
h
is
re
se
a
rc
h
stu
d
y
a
d
d
re
ss
e
s
th
e
c
h
a
ll
e
n
g
e
o
f
id
e
n
ti
fy
in
g
s
o
c
ial
m
e
d
ia
b
o
ts
(S
M
B)
t
h
a
t
c
a
n
ra
p
i
d
ly
d
isse
m
in
a
te
in
f
o
rm
a
ti
o
n
o
r
m
isin
f
o
r
m
a
ti
o
n
o
n
p
latfo
rm
s
li
k
e
Twi
tt
e
r.
It
c
o
n
tri
b
u
tes
to
t
h
e
field
b
y
re
v
iew
in
g
li
t
e
ra
tu
re
to
d
e
fin
e
b
o
t
b
e
h
a
v
i
o
u
rs
a
n
d
e
x
p
l
o
rin
g
a
d
v
a
n
c
e
d
m
a
c
h
in
e
lea
rn
in
g
c
las
sifiers
fo
r
e
ffe
c
ti
v
e
b
o
t
d
e
tec
ti
o
n
u
sin
g
a
c
c
o
u
n
t
-
le
v
e
l
d
a
ta.
Th
e
st
u
d
y
e
m
p
lo
y
e
d
S
p
e
a
rm
a
n
'
s
ra
n
k
c
o
rre
latio
n
c
o
e
f
ficie
n
t
to
se
lec
t
re
lev
a
n
t
fe
a
t
u
re
s
fo
r
S
M
B
c
las
sifica
ti
o
n
,
th
e
n
train
e
d
si
x
d
i
ffe
re
n
t
m
a
c
h
in
e
lea
rn
in
g
m
o
d
e
ls
:
d
e
c
isio
n
tree
(DT)
,
ra
n
d
o
m
fo
re
st
(RF
)
,
lo
g
isti
c
re
g
re
ss
io
n
(LR)
,
s
u
p
p
o
rt
v
e
c
to
r
m
a
c
h
in
e
(S
VM)
,
a
n
d
k
-
n
e
a
re
st
n
e
ig
h
b
o
u
r
(KN
N)
.
To
fu
rt
h
e
r
imp
ro
v
e
a
c
c
u
ra
c
y
,
a
c
las
sifier
sta
c
k
in
g
tec
h
n
i
q
u
e
wa
s
a
p
p
l
ied
.
Ke
y
fin
d
in
g
s
re
v
e
a
led
th
a
t
wh
il
e
i
n
d
i
v
id
u
a
l
c
las
sifiers
p
e
rfo
rm
e
d
v
a
riab
ly
,
with
RF
lea
d
i
n
g
a
t
8
9
%
acc
u
ra
c
y
,
th
e
sta
c
k
e
d
c
las
sifier
a
p
p
ro
a
c
h
o
u
t
p
e
rfo
rm
e
d
a
ll
sin
g
l
e
-
c
las
sifier
m
e
th
o
d
s
wit
h
a
n
imp
re
ss
iv
e
9
0
%
a
c
c
u
ra
c
y
ra
te.
T
h
e
re
su
l
ts
u
n
d
e
rsc
o
re
th
e
p
o
ten
ti
a
l
o
f
c
o
m
b
in
in
g
m
u
lt
ip
le
c
las
sifiers
to
e
n
h
a
n
c
e
th
e
p
re
c
isio
n
o
f
so
c
ial
m
e
d
ia b
o
t
d
e
tec
ti
o
n
e
ffo
rts.
K
ey
w
o
r
d
s
:
B
o
t d
etec
tio
n
Featu
r
e
s
elec
tio
n
Ma
ch
in
e
lear
n
in
g
So
cial
m
ed
ia
Stack
in
g
class
if
ier
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Sam
ar
jeet
B
o
r
ah
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
A
p
p
licatio
n
s
,
Sik
k
im
Ma
n
ip
al
I
n
s
t
itu
te
o
f
T
ec
h
n
o
lo
g
y
Sik
k
im
Ma
n
ip
al
Un
iv
er
s
ity
Gan
g
to
k
,
I
n
d
ia
E
m
ail:
s
am
ar
jeet.
b
@
s
m
it.e
d
u
.
in
1.
I
NT
RO
D
UCT
I
O
N
So
cial
m
ed
ia
is
a
p
latf
o
r
m
wh
er
e
in
d
iv
id
u
als,
b
u
s
in
ess
es,
in
s
titu
tio
n
s
,
an
d
in
d
u
s
tr
ies
s
h
ar
e
an
d
ex
ch
an
g
e
in
f
o
r
m
atio
n
o
f
all
k
in
d
s
.
I
t
h
as
b
ec
o
m
e
a
m
e
d
i
u
m
to
p
r
o
m
o
te
an
d
ex
ch
a
n
g
e
id
ea
s
,
wh
er
e
illi
cit
u
s
er
s
u
tili
ze
b
o
ts
to
p
r
o
m
o
te
a
ctiv
ities
o
f
th
eir
in
ter
e
s
t.
I
t
is
d
o
n
e
b
y
m
a
n
ip
u
latin
g
p
u
b
lic
o
p
in
io
n
s
,
s
p
r
ea
d
in
g
r
u
m
o
u
r
s
,
an
d
p
r
o
d
u
cin
g
f
a
k
e
r
atin
g
s
o
r
r
ev
iews,
wh
ich
ar
e
au
to
-
g
en
e
r
ated
p
o
s
ts
,
co
m
m
en
ts
,
co
n
ten
t,
an
d
in
ter
ac
tio
n
s
with
n
o
r
m
al
u
s
er
s
.
As p
er
th
e
s
tu
d
y
[
1
]
8
.
5
% o
f
u
s
er
s
o
n
T
witter
ar
e
s
im
p
ly
b
o
ts
.
B
o
ts
ar
e
r
esp
o
n
s
ib
le
f
o
r
d
is
s
em
in
atin
g
a
p
o
liti
ca
l
ag
e
n
d
a,
m
an
i
p
u
latin
g
p
u
b
lic
o
p
in
i
o
n
,
an
d
b
o
o
s
tin
g
th
e
v
o
ice
o
f
th
ei
r
p
r
o
p
a
g
an
d
a
in
v
ar
io
u
s
wo
r
ld
cr
is
is
ev
en
ts
,
s
u
ch
as
war
,
n
atu
r
al
d
is
aster
s
[
2
]
.
So
cial
b
o
ts
p
r
o
d
u
ce
au
to
m
ate
d
co
n
ten
t
an
d
in
te
r
ac
t
with
h
u
m
an
s
;
in
t
h
e
p
r
o
ce
s
s
,
th
ey
l
ea
r
n
an
d
m
im
ic
h
u
m
an
b
eh
av
io
u
r
[
3
]
.
B
o
t
ac
c
o
u
n
ts
h
a
v
e
b
ec
o
m
e
in
cr
ea
s
in
g
l
y
s
o
p
h
is
ticated
o
v
er
th
e
y
ea
r
s
,
an
d
to
s
o
m
e
ex
ten
t,
th
ey
ar
e
u
n
d
etec
ted
.
Stu
d
ies
b
y
B
ess
i
an
d
Fer
r
ar
a
[
4
]
,
r
ev
ea
ls
th
e
em
p
lo
y
m
e
n
t
o
f
s
o
cial
b
o
ts
th
r
o
u
g
h
o
u
t
t
h
e
elec
ti
o
n
p
e
r
io
d
to
e
n
h
an
ce
o
n
lin
e
n
etwo
r
k
p
o
lar
izatio
n
b
y
g
en
e
r
atin
g
f
a
k
e
twee
ts
,
lik
es,
an
d
r
etwe
etin
g
o
n
p
o
liti
ca
l
co
m
m
en
ts
.
T
h
e
g
o
al
o
f
a
s
o
cial
b
o
t
in
s
u
ch
an
ev
en
t
wo
u
ld
b
e
eith
er
to
d
is
to
r
t
th
e
ca
n
d
id
ate’
s
im
ag
e
o
r
p
r
o
m
o
te
th
em
in
f
av
o
u
r
o
f
th
e
ir
ag
en
d
a.
I
n
2
0
2
0
,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
4
7
7
-
4
8
7
478
r
esear
ch
s
h
o
ws
th
e
r
is
k
o
f
s
o
cial
m
ed
ia
b
o
ts
(
SMB
)
in
d
is
s
em
in
atin
g
m
is
in
f
o
r
m
atio
n
d
u
r
in
g
th
e
tim
e
o
f
C
OVI
D
-
1
9
p
an
d
em
ic
[
5
]
.
As
f
ar
as
th
ese
ch
allen
g
es
ar
e
co
n
ce
r
n
e
d
,
a
p
r
o
p
er
m
eth
o
d
m
u
s
t
b
e
d
esig
n
e
d
to
s
et
a
s
tr
ict
p
ar
am
eter
th
at
will
d
r
aw
a
lin
e
th
at
d
if
f
er
en
tiates
th
e
r
ea
l
ac
co
u
n
t
a
n
d
th
e
b
o
t
ac
c
o
u
n
t.
So
cial
b
o
ts
ex
h
ib
it
v
ar
io
u
s
ch
ar
ac
ter
is
tics
an
d
f
ea
tu
r
es; u
n
d
er
s
tan
d
in
g
o
f
ea
ch
ch
a
r
ac
ter
is
tic
i
s
im
p
o
r
tan
t f
o
r
th
e
ac
cu
r
ate
class
if
icat
io
n
o
f
b
o
ts
.
T
h
e
f
u
n
d
am
e
n
tal
r
esear
c
h
q
u
esti
o
n
th
a
t c
o
m
es a
cr
o
s
s
wh
en
tr
y
in
g
to
d
if
f
er
en
tiate
b
o
t a
cc
o
u
n
ts
f
r
o
m
r
ea
l
h
u
m
an
ac
c
o
u
n
ts
is
–
wh
at
ar
e
th
e
m
o
s
t
p
r
o
m
in
e
n
t
an
d
d
i
s
cr
im
in
atin
g
f
ea
tu
r
es
th
at
h
e
lp
to
id
en
tify
th
e
b
eh
av
io
u
r
o
f
a
b
o
t a
cc
o
u
n
t a
n
d
h
o
w
it is
d
if
f
er
en
t f
r
o
m
a
h
u
m
an
ac
co
u
n
t?
T
h
is
s
tu
d
y
h
as
i
n
v
esti
g
ated
th
e
f
ea
tu
r
es
o
f
b
o
ts
an
d
h
o
w
it
is
d
if
f
er
en
t
f
r
o
m
h
u
m
an
ac
co
u
n
ts
,
wh
ich
is
s
ig
n
if
ican
t in
class
if
y
in
g
b
o
t
s
an
d
h
u
m
a
n
ac
co
u
n
ts
.
C
o
n
ten
ts
o
f
th
e
p
ap
e
r
h
av
e
b
ee
n
o
r
g
a
n
ized
as f
o
llo
ws
:
−
T
h
e
in
tr
o
d
u
ctio
n
s
ec
tio
n
p
r
ese
n
ts
th
e
b
ac
k
g
r
o
u
n
d
d
etails o
f
t
h
e
r
esear
ch
wo
r
k
.
−
T
h
e
s
ec
o
n
d
s
ec
tio
n
p
r
esen
ts
s
o
m
e
in
f
lu
en
tial
wo
r
k
s
f
r
o
m
th
e
liter
atu
r
e
an
d
th
e
id
en
tific
atio
n
o
f
b
eh
av
io
u
r
al
f
ea
tu
r
es
o
f
th
e
b
o
t
.
−
T
h
e
t
h
ir
d
s
ec
tio
n
h
as p
r
o
p
o
s
ed
a
m
eth
o
d
o
lo
g
y
,
wh
e
r
e
a
s
tack
in
g
en
s
em
b
le
class
if
ier
is
im
p
lem
en
ted
.
−
T
h
e
f
o
u
r
th
s
ec
tio
n
co
n
tain
s
ev
alu
atio
n
,
r
esu
lts
,
an
d
d
i
s
cu
s
s
io
n
.
T
h
e
o
u
tco
m
e
o
f
th
e
p
r
o
p
o
s
ed
m
eth
o
d
o
l
o
g
y
is
co
m
p
ar
ed
with
o
th
er
liter
atu
r
e
r
ev
iew
r
esu
lts
.
2.
L
I
T
E
R
AT
U
RE
R
E
VI
E
W
T
h
r
o
u
g
h
v
ar
i
o
u
s
liter
atu
r
e
r
e
v
iews,
th
e
b
eh
av
io
u
r
al
f
ea
tu
r
es
o
f
b
o
t
ac
co
u
n
ts
o
n
T
witter
h
av
e
b
ee
n
p
r
o
p
er
l
y
id
e
n
tifie
d
,
w
h
ich
c
o
u
ld
b
e
u
s
ed
as
a
m
etr
ic
f
o
r
d
if
f
er
en
tiatin
g
b
etwe
en
b
o
t
a
cc
o
u
n
ts
an
d
h
u
m
an
ac
co
u
n
ts
.
T
o
p
r
ed
ict
wh
eth
e
r
a
twee
t
h
as
b
ee
n
p
o
s
ted
b
y
a
b
o
t
o
r
a
h
u
m
a
n
ac
co
u
n
t,
an
d
t
o
en
h
an
ce
ex
is
tin
g
lab
elled
d
atasets
f
o
r
s
o
cial
b
o
t
d
etec
tio
n
,
a
s
tu
d
y
h
as
p
r
o
p
o
s
ed
“a
cc
o
u
n
t
lev
el
class
if
icatio
n
”
an
d
“twe
et
lev
el”
class
if
icatio
n
wh
ich
r
esu
lted
in
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC
)
>
9
8
%
with
Ad
aBo
o
s
t
an
d
r
an
d
o
m
f
o
r
est
(
R
F)
[
6
]
.
So
cial,
d
em
o
g
r
a
p
h
ic
,
an
d
b
eh
av
io
u
r
al
asp
ec
ts
o
f
b
o
ts
an
d
h
u
m
an
s
,
an
d
th
e
im
p
a
ct
o
f
b
o
ts
in
s
o
cial
en
v
ir
o
n
m
en
ts
,
ca
n
b
e
f
o
u
n
d
i
n
[
7
]
.
B
o
tOr
No
t
r
esu
lted
in
l
ess
ac
cu
r
ac
y
with
d
if
f
er
en
t
t
h
r
esh
o
ld
s
(
4
0
%
to
6
0
%)
in
lab
ellin
g
an
ac
c
o
u
n
t
as
a
b
o
t.
T
h
e
p
r
esen
ce
o
f
T
w
itter
s
p
am
b
o
ts
h
as
b
ee
n
ev
id
e
n
t
th
r
o
u
g
h
v
a
r
io
u
s
an
aly
s
es.
A
n
ex
p
er
im
en
t
was
co
n
d
u
cted
u
s
in
g
a
d
ataset
o
f
"liv
in
g
,
d
elete
d
,
a
n
d
s
u
s
p
e
n
d
ed
ac
co
u
n
ts
"
f
o
r
m
u
ltip
le
g
r
o
u
p
i
n
g
s
o
f
le
g
itima
te
an
d
f
r
a
u
d
u
le
n
t
ac
co
u
n
ts
[
8
]
.
T
h
e
r
esu
lt
o
f
th
is
s
h
o
w
s
t
h
a
t
8
8
.
9
%
o
f
b
o
ts
o
n
T
witter
ar
e
s
till
aliv
e,
wh
ile
o
n
ly
8
.
6
%
o
f
b
o
ts
h
av
e
b
ee
n
s
u
s
p
en
d
ed
.
Acc
o
r
d
in
g
to
Yan
g
e
t
a
l.
[
9
]
,
twee
ts
ar
e
r
etr
iev
ed
f
r
o
m
T
witter
in
r
ea
l
-
tim
e
with
m
in
im
al
ac
co
u
n
t
m
etad
ata,
an
d
th
e
r
esu
lt
r
ev
ea
ls
th
at
R
F
p
er
f
o
r
m
e
d
with
f
lawless
AU
C
wh
en
tr
ain
ed
an
d
test
ed
o
n
an
y
s
in
g
le
d
at
aset.
T
o
u
n
d
e
r
s
t
a
n
d
a
n
d
i
d
e
n
t
i
f
y
s
p
am
b
e
h
a
v
i
o
u
r
a
n
d
d
e
t
e
c
t
f
a
k
e
i
d
e
n
t
i
t
i
es
,
f
il
t
e
r
i
n
g
r
u
l
es
h
a
v
e
b
ee
n
a
p
p
l
i
e
d
in
[
1
0
]
,
w
h
i
c
h
r
e
s
u
l
t
e
d
i
n
s
u
cc
e
s
s
f
u
l
l
y
d
e
t
e
ct
i
n
g
t
h
e
b
e
h
a
v
io
u
r
t
h
r
o
u
g
h
s
e
n
t
i
m
e
n
t
a
n
a
l
y
s
is
.
T
h
e
b
e
h
a
v
i
o
u
r
a
l
c
h
a
r
a
c
t
e
r
is
t
i
cs
o
f
a
b
o
t
a
r
e
c
a
t
e
g
o
r
i
z
e
d
u
n
d
e
r
“
n
u
m
e
r
i
c
,
c
a
t
e
g
o
r
i
c
a
l
,
a
n
d
s
e
r
i
es
f
e
a
t
u
r
e
s
”
.
U
s
i
n
g
m
u
l
t
i
n
a
i
v
e
b
a
y
e
s
(
NB
)
,
R
F
,
a
n
d
t
w
o
i
n
s
ta
n
c
e
s
o
f
g
e
n
e
r
al
i
z
e
d
l
i
n
e
a
r
m
o
d
e
l
,
a
n
e
x
p
e
r
i
m
e
n
t
w
as
c
o
n
d
u
c
t
e
d
w
h
i
c
h
s
h
o
ws
i
n
t
e
r
e
s
t
i
n
g
p
a
tt
e
r
n
s
s
u
c
h
t
h
at
t
h
e
p
o
p
u
l
a
r
i
t
y
,
f
o
ll
o
w
r
a
ti
o
,
an
d
r
e
c
i
p
r
o
c
i
t
y
b
o
ts
,
e
x
c
l
u
d
i
n
g
c
o
n
s
u
m
p
t
i
o
n
b
o
ts
,
h
a
v
e
m
o
r
e
f
o
l
l
o
w
e
r
s
t
h
a
n
f
o
ll
o
w
e
r
s
i
n
g
e
n
e
r
al
,
b
u
t
t
h
e
c
as
e
is
n
o
t
c
o
m
m
o
n
i
n
h
u
m
a
n
a
c
c
o
u
n
t
s
[
1
1
]
.
T
h
e
u
s
e
o
f
em
o
jis
is
p
o
p
u
la
r
in
wr
itten
co
m
m
u
n
icatio
n
,
wh
ich
p
lay
s
a
v
ital
r
o
le
in
e
x
p
r
ess
in
g
v
ar
io
u
s
em
o
tio
n
s
.
E
m
o
jis
ar
e
u
s
ed
to
t
r
ain
th
e
m
o
d
el
to
b
u
ild
a
s
en
tim
en
t
class
if
ier
u
s
in
g
MN
,
wh
ic
h
d
eter
m
in
es
a
twee
t’
s
e
m
o
tio
n
al
o
r
ien
tatio
n
[
1
2
]
.
T
h
e
r
esu
lt
an
t
ex
p
e
r
im
en
ts
s
h
o
w
th
at
th
e
m
o
d
el
p
er
f
o
r
m
e
d
well
in
class
if
y
in
g
p
o
s
itiv
e,
n
eu
tr
al,
an
d
n
eg
ativ
e
p
o
liti
ca
l
twee
ts
.
T
o
id
en
tify
th
e
co
o
r
d
in
ated
attem
p
ts
o
f
in
f
o
r
m
atio
n
d
is
s
em
in
atio
n
,
a
s
tu
d
y
was
co
n
d
u
cted
o
n
a
s
o
ci
al
b
o
t,
with
two
B
ay
esian
s
tatis
tical
m
o
d
els,
d
escr
ib
in
g
s
im
p
le
a
n
d
co
m
p
lex
co
n
tag
io
n
d
y
n
am
ics
[1
3
]
.
T
h
e
r
esu
lt
o
f
t
h
e
ex
p
e
r
im
en
t
s
h
o
ws
th
at
in
ter
d
ep
en
d
en
t
b
o
ts
wer
e
m
o
r
e
ef
f
ec
tiv
e
at
s
p
r
ea
d
in
g
in
f
o
r
m
atio
n
th
an
t
h
e
in
d
ep
en
d
en
t
b
o
ts
.
A
[
5
]
,
s
tu
d
y
h
ig
h
lig
h
ts
h
o
w
SMB
co
n
tr
ib
u
t
ed
to
s
p
r
ea
d
in
g
m
is
in
f
o
r
m
atio
n
d
u
r
in
g
t
h
e
C
OVI
D
-
1
9
c
r
is
is
.
Ar
is
ta
[
1
4
]
u
s
ed
a
d
ec
is
io
n
tr
ee
(
DT
)
f
o
r
p
r
ed
icti
o
n
an
d
f
o
u
n
d
t
h
at
it o
u
tp
e
r
f
o
r
m
ed
lo
g
is
tic
r
eg
r
ess
io
n
(
L
R
)
.
W
ith
th
e
in
cr
ea
s
in
g
n
u
m
b
er
o
f
I
o
T
d
ev
ices,
th
e
n
etwo
r
k
tr
af
f
ic
is
als
o
a
n
attac
k
f
r
o
m
b
o
ts
.
A
n
ef
f
icien
t
d
etec
tio
n
o
f
b
o
tn
et
tr
af
f
ic
b
y
f
ea
tu
r
e
s
elec
tio
n
an
d
a
DT
ca
n
b
e
f
o
u
n
d
in
[
1
5
]
.
I
n
f
o
r
m
atio
n
g
ain
an
d
Gin
i
im
p
o
r
tan
ce
ar
e
u
s
ed
f
o
r
f
ea
tu
r
e
s
elec
tio
n
to
s
elec
t
th
e
b
o
tn
et
tr
af
f
ic
f
ea
tu
r
es
th
at
m
ak
e
th
e
b
o
ts
u
n
d
etec
tab
le.
F
u
r
th
er
,
t
h
r
ee
m
ac
h
in
e
lear
n
i
n
g
class
if
ier
s
,
DT
,
R
F,
an
d
k
-
n
ea
r
est
n
ei
g
h
b
o
u
r
(
KNN
)
,
ar
e
tr
ain
ed
o
n
th
e
d
ataset
an
d
th
e
co
llected
s
et
o
f
f
ea
t
u
r
es.
T
h
e
p
er
f
o
r
m
an
ce
o
f
t
h
e
m
o
d
el
is
m
ea
s
u
r
ed
b
y
th
e
m
etr
ic
F1
-
s
co
r
e,
wh
er
e
th
e
D
T
s
co
r
ed
th
e
h
ig
h
est
ac
cu
r
ac
y
o
f
8
5
%.
V
a
r
i
o
u
s
r
e
v
i
e
ws
o
n
S
MB
[
3
]
,
[
8
]
,
[
1
2
]
,
[
1
6
]
,
s
h
o
w
t
h
e
p
r
e
s
e
n
c
e
o
f
b
o
t
t
a
m
p
e
r
i
n
g
w
i
t
h
t
h
e
i
n
f
o
r
m
a
t
i
o
n
.
T
h
e
r
e
f
o
r
e
,
t
h
e
a
m
o
u
n
t
o
f
i
m
p
ac
t
o
f
s
o
c
i
al
b
o
ts
,
w
h
i
c
h
a
r
e
r
e
s
p
o
n
s
i
b
l
e
f
o
r
s
p
r
e
a
d
i
n
g
f
a
k
e
n
ew
s
a
n
d
c
r
e
at
i
n
g
b
i
as
i
n
t
h
e
n
e
ws
o
v
e
r
t
h
e
p
as
t
y
e
ar
s
,
is
i
m
m
e
as
u
r
a
b
l
e
.
A
s
a
r
e
s
u
l
t
,
i
t
h
as
b
e
e
n
p
o
s
i
n
g
a
t
h
r
e
at
t
o
d
e
m
o
c
r
a
c
y
a
n
d
c
o
n
t
r
i
b
u
t
i
n
g
t
o
c
y
b
e
r
c
r
i
m
e
.
N
o
t
m
a
n
y
p
a
p
e
r
s
h
a
v
e
s
h
o
w
n
t
h
a
t t
h
e
b
e
h
a
v
i
o
u
r
a
l
p
a
t
t
e
r
n
s
o
f
t
h
e
b
o
t
s
a
r
e
c
o
n
s
t
a
n
tl
y
e
v
o
l
v
i
n
g
.
I
n
t
h
i
s
c
o
n
t
e
x
t
,
t
h
e
w
o
r
k
t
h
a
t
h
a
s
b
e
e
n
c
a
r
r
i
e
d
o
u
t
i
n
t
h
i
s
p
a
p
e
r
e
v
a
l
u
a
t
e
s
t
h
e
i
m
p
o
r
t
a
n
c
e
o
f
f
e
a
t
u
r
e
s
a
n
d
r
e
m
a
r
k
a
b
l
e
d
i
f
f
e
r
e
n
t
i
a
ti
o
n
b
e
t
w
e
e
n
b
o
t
s
a
n
d
h
u
m
a
n
-
o
p
e
r
a
t
e
d
a
c
c
o
u
n
t
s
b
as
e
d
o
n
t
h
e
s
e
f
e
at
u
r
e
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
S
ta
ck
in
g
o
f m
a
ch
in
e
le
a
r
n
in
g
cla
s
s
ifie
r
s
fo
r
b
o
t d
etec
tio
n
u
s
in
g
a
cc
o
u
n
t le
ve
l d
a
ta
(
Jwa
la
S
h
a
r
ma
)
479
2
.
1
.
B
eha
v
io
ura
l
f
e
a
t
ures o
f
bo
t
a
cc
o
un
t
B
eh
av
io
u
r
al
f
ea
tu
r
es
p
lay
a
p
r
im
e
r
o
le
d
u
r
in
g
th
e
b
o
t
d
etec
tio
n
p
r
o
ce
s
s
.
E
f
f
icien
t
u
tili
za
tio
n
o
f
th
e
s
am
e
m
in
im
izes
f
alse
p
o
s
itiv
es.
Mo
s
t
h
u
m
an
u
s
er
s
h
av
e
t
h
e
ir
lo
ca
tio
n
e
n
ab
led
,
allo
win
g
g
eo
-
lo
ca
tio
n
,
wh
ich
is
o
n
e
o
f
th
e
k
ey
s
ig
n
s
th
at
t
h
e
ac
co
u
n
t
is
o
p
e
r
ated
b
y
a
h
u
m
an
.
T
h
e
to
tal
n
u
m
b
e
r
o
f
twee
ts
f
r
o
m
b
o
t
ac
co
u
n
ts
i
s
h
ig
h
,
an
d
th
ey
h
av
e
f
ewe
r
f
o
llo
wer
s
co
m
p
ar
ed
t
o
h
u
m
an
ac
co
u
n
ts
.
E
ac
h
lab
elled
u
s
er
in
th
e
d
atab
ase
co
n
s
is
ts
o
f
p
r
o
f
ile
f
ea
tu
r
es,
wh
ich
g
iv
e
th
e
d
escr
ip
tio
n
s
o
f
th
eir
ac
co
u
n
t
an
d
ca
n
b
e
u
s
ed
as
a
m
etr
ic
f
o
r
class
if
icatio
n
in
to
two
ca
teg
o
r
ies
–
b
o
t
an
d
h
u
m
an
ac
co
u
n
ts
.
T
h
e
o
v
er
all
s
u
m
m
ar
y
o
f
f
ea
tu
r
es
an
d
th
ei
r
f
in
d
in
g
s
o
n
d
etec
tin
g
b
o
ts
is
s
h
o
wn
in
T
a
b
le
1
.
B
o
t
ac
co
u
n
ts
ex
h
ib
it
d
is
tin
ct
ch
ar
ac
ter
is
tics
th
at
d
if
f
er
en
tiate
th
em
f
r
o
m
g
e
n
u
in
e
u
s
er
s
.
T
h
ey
o
f
ten
lack
d
etailed
p
r
o
f
ile
i
n
f
o
r
m
atio
n
an
d
ar
e
r
ec
en
tly
cr
e
ated
f
o
r
s
h
o
r
t
-
ter
m
ag
e
n
d
as,
u
n
lik
e
l
eg
itima
te
ac
co
u
n
ts
th
at
s
h
o
w
co
n
s
is
ten
t,
lo
n
g
-
ter
m
ac
tiv
ity
.
B
o
t
ac
co
u
n
ts
u
s
u
a
lly
h
av
e
f
ew
f
o
llo
wer
s
b
u
t
f
o
llo
w
m
an
y
u
s
er
s
to
am
p
lify
th
eir
r
ea
ch
.
T
h
e
y
o
f
t
en
d
is
p
lay
l
o
w
f
o
llo
wer
r
an
k
in
g
,
lim
ited
lik
es,
a
n
d
ex
ce
s
s
iv
ely
h
ig
h
r
etwe
et
ac
tiv
ity
s
o
m
etim
es
av
er
ag
in
g
u
p
to
7
2
twee
ts
p
er
d
ay
[
1
7
]
in
d
icatin
g
au
to
m
atio
n
.
T
h
eir
s
cr
ee
n
n
am
es
ar
e
ty
p
ically
s
h
o
r
t o
r
g
e
n
er
ic,
an
d
th
e
co
n
ten
t th
ey
p
o
s
t
ten
d
to
b
e
r
ep
etitiv
e,
em
o
tio
n
less
,
o
r
f
il
led
with
UR
L
s
,
al
l
o
f
wh
ich
h
elp
in
id
e
n
tif
y
in
g
a
u
to
m
ated
b
e
h
av
io
r
o
n
s
o
cial
m
ed
ia.
T
ab
le
1
.
A
s
u
m
m
ar
y
o
f
th
e
f
ea
tu
r
es u
s
ed
an
d
t
h
eir
f
in
d
in
g
s
f
o
r
b
o
t
d
etec
tio
n
A
p
p
r
o
a
c
h
(
Ref
#)
F
e
a
t
u
r
e
s
F
i
n
d
i
n
g
s
[
7
]
A
g
e
,
t
w
e
e
t
s,
r
e
t
w
e
e
t
s,
f
a
v
o
r
i
t
e
s,
r
e
p
l
i
e
s
a
n
d
men
t
i
o
n
s
,
U
R
L
c
o
u
n
t
,
a
n
d
f
o
l
l
o
w
e
r
-
f
r
i
e
n
d
r
a
t
i
o
.
(
l
i
k
e
s
p
e
r
t
w
e
e
t
,
r
e
t
w
e
e
t
s
p
e
r
t
w
e
e
t
,
u
se
r
r
e
p
l
i
e
s
a
n
d
m
e
n
t
i
o
n
s,
a
c
t
i
v
i
t
y
s
o
u
r
c
e
c
o
u
n
t
,
t
y
p
e
o
f
a
c
t
i
v
i
t
y
s
o
u
r
c
e
s
,
a
n
d
si
z
e
o
f
c
o
n
t
e
n
t
u
p
l
o
a
d
e
d
−
B
a
se
d
o
n
t
h
e
i
r
b
e
h
a
v
i
o
r
a
l
p
a
t
t
e
r
n
s
,
a
p
p
r
o
x
i
ma
t
e
l
y
1
5
%
o
f
u
sers
a
r
e
i
d
e
n
t
i
f
i
e
d
a
s
b
o
t
s.
−
B
o
t
s
t
e
n
d
t
o
b
e
m
o
r
e
a
c
t
i
v
e
t
h
a
n
h
u
ma
n
u
s
e
r
s,
t
w
e
e
t
i
n
g
mo
r
e
f
r
e
q
u
e
n
t
l
y
a
n
d
a
t
a
h
i
g
h
e
r
r
a
t
e
,
a
n
d
b
o
t
s
t
e
n
d
t
o
h
a
v
e
a
smal
l
e
r
n
u
m
b
e
r
o
f
f
o
l
l
o
w
e
r
s
a
n
d
f
o
l
l
o
w
a
l
a
r
g
e
r
n
u
mb
e
r
o
f
u
sers.
−
B
o
t
s
u
se
d
i
f
f
e
r
e
n
t
t
y
p
e
s
o
f
c
o
n
t
e
n
t
(
b
o
t
s
b
e
i
n
g
m
o
r
e
l
i
k
e
l
y
t
o
u
se
t
w
e
e
t
l
i
n
k
s
,
h
a
s
h
t
a
g
s,
a
n
d
men
t
i
o
n
s)
t
h
a
n
h
u
m
a
n
u
sers.
−
F
e
a
t
u
r
e
s,
s
u
c
h
a
s
u
ser
a
c
t
i
v
i
t
y
,
c
o
n
t
e
n
t
,
a
n
d
s
o
c
i
a
l
n
e
t
w
o
r
k
c
h
a
r
a
c
t
e
r
i
s
t
i
c
s,
c
a
n
b
e
u
s
e
d
t
o
d
i
f
f
e
r
e
n
t
i
a
t
e
b
e
t
w
e
e
n
b
o
t
s
a
n
d
h
u
ma
n
u
sers w
i
t
h
a
h
i
g
h
d
e
g
r
e
e
o
f
a
c
c
u
r
a
c
y
.
[
8
]
F
e
a
t
u
r
e
s
:
“
F
a
k
e
f
o
l
l
o
w
e
r
f
r
a
u
d
s
,
r
e
t
w
e
e
t
f
r
a
u
d
s
,
h
a
s
h
t
a
g
p
r
o
mo
t
i
o
n
,
U
R
L
s
p
a
mm
i
n
g
,
sca
mm
i
n
g
,
a
n
d
sp
a
m
o
f
g
e
n
e
r
i
c
m
e
ss
a
g
e
s,
a
n
d
a
g
e
o
f
a
c
c
o
u
n
t
,
p
r
o
f
i
l
e
p
i
c
.
”
−
S
t
u
d
y
p
r
o
v
i
d
e
s
e
v
i
d
e
n
c
e
o
f
t
h
e
i
n
c
r
e
a
si
n
g
s
o
p
h
i
s
t
i
c
a
t
i
o
n
o
f
so
c
i
a
l
sp
a
m
b
o
t
s,
w
h
i
c
h
a
r
e
n
o
w
a
b
l
e
t
o
m
i
mi
c
h
u
m
a
n
b
e
h
a
v
i
o
r
a
n
d
d
e
c
e
i
v
e
e
v
e
n
e
x
p
e
r
i
e
n
c
e
d
u
s
e
r
s,
w
h
e
r
e
8
.
6
%
a
r
e
s
u
sp
e
n
d
e
d
b
o
t
s
a
n
d
8
8
.
9
%
a
r
e
a
c
t
i
v
e
b
o
t
s
.
[
9
]
U
ser
me
t
a
d
a
t
a
f
e
a
t
u
r
e
s
/
D
e
r
i
v
e
d
f
e
a
t
u
r
e
s
“
st
a
t
u
e
s
_
c
o
u
n
t
,
f
o
l
l
o
w
e
r
_
c
o
u
n
t
,
f
r
i
e
n
d
_
c
o
u
n
t
,
f
a
v
o
u
r
i
t
e
s
_
c
o
u
n
t
,
l
i
st
_
c
o
u
n
t
,
d
e
f
a
u
l
t
_
c
o
u
n
t
,
t
w
e
e
t
_
f
r
e
q
,
f
o
l
l
o
w
e
r
s_
g
r
o
w
t
h
_
r
a
t
e
,
f
r
i
e
n
d
s
_
g
r
o
w
t
h
_
r
a
t
e
,
f
a
v
o
u
r
i
t
e
s_
g
r
o
w
t
h
_
r
a
t
e
,
l
i
st
e
d
_
g
r
o
w
t
h
_
r
a
t
e
,
f
o
l
l
o
w
e
r
s
_
f
r
i
e
n
d
_
r
a
t
i
o
,
scree
n
_
n
a
me
_
l
e
n
g
t
h
”
−
U
si
n
g
a
l
a
r
g
e
-
sc
a
l
e
d
a
t
a
s
e
t
o
f
Tw
i
t
t
e
r
a
c
c
o
u
n
t
s
s
h
o
w
s t
h
a
t
i
t
c
a
n
a
c
h
i
e
v
e
h
i
g
h
a
c
c
u
r
a
c
y
i
n
d
e
t
e
c
t
i
n
g
s
o
c
i
a
l
b
o
t
s
,
e
v
e
n
i
n
t
h
e
p
r
e
se
n
c
e
o
f
ma
n
y
h
u
m
a
n
u
sers
a
n
d
n
o
i
se
i
n
t
h
e
d
a
t
a
.
−
R
F
r
e
s
u
l
t
e
d
i
n
a
n
AUC
o
f
0
.
8
4
.
[
1
0
]
I
d
e
n
t
i
t
y
,
b
e
h
a
v
i
o
r
,
r
e
l
a
t
i
o
n
s
h
i
p
−
Th
e
R
F
a
l
g
o
r
i
t
h
m
h
a
s
o
u
t
p
e
r
f
o
r
me
d
o
t
h
e
r
a
l
g
o
r
i
t
h
ms
u
se
d
i
n
t
h
e
s
t
u
d
y
w
h
i
l
e
c
l
a
ss
i
f
y
i
n
g
b
e
t
w
e
e
n
b
o
t
s
a
n
d
h
u
ma
n
s,
b
y
a
c
h
i
e
v
i
n
g
a
h
i
g
h
a
c
c
u
r
a
c
y
r
a
t
e
o
f
o
v
e
r
9
0
%.
[
1
1
]
N
u
meri
c
,
c
a
t
e
g
o
r
i
c
a
l
,
a
n
d
seri
e
s fe
a
t
u
r
e
s
−
Th
e
r
e
i
s
a
t
i
m
e
v
a
r
i
a
n
c
e
i
n
t
h
e
t
w
e
e
t
i
n
g
p
e
r
i
o
d
b
e
t
w
e
e
n
h
u
m
a
n
a
n
d
b
o
t
a
c
c
o
u
n
t
s.
−
M
i
s
c
e
l
l
a
n
e
o
u
s
w
e
b
l
i
n
k
s
a
n
d
t
o
p
i
c
s
a
r
e
i
n
c
l
u
d
e
d
b
y
b
o
t
s
t
h
a
n
b
y
h
u
ma
n
s.
[
1
2
]
Emo
t
i
c
o
n
s
a
n
d
e
mo
j
i
−
Th
e
s
t
u
d
y
f
o
u
n
d
t
h
a
t
t
h
e
e
m
o
j
i
-
b
a
s
e
d
a
p
p
r
o
a
c
h
p
r
o
d
u
c
e
d
c
o
m
p
a
r
a
b
l
e
r
e
s
u
l
t
s
(
F
-
m
e
a
s
u
r
e
=
6
7
.
8
,
a
c
c
u
r
a
c
y
=
7
4
.
9
)
t
o
mo
r
e
t
r
a
d
i
t
i
o
n
a
l
se
n
t
i
me
n
t
a
n
a
l
y
s
i
s
t
e
c
h
n
i
q
u
e
s
,
s
u
c
h
a
s
u
si
n
g
t
h
e
A
F
I
N
N
l
e
x
i
c
o
n
.
−
Th
e
p
a
p
e
r
d
e
mo
n
st
r
a
t
e
s
t
h
e
p
o
t
e
n
t
i
a
l
o
f
u
s
i
n
g
a
n
e
mo
j
i
t
r
a
i
n
i
n
g
h
e
u
r
i
st
i
c
f
o
r
s
e
n
t
i
me
n
t
a
n
a
l
y
s
i
s
o
f
so
c
i
a
l
me
d
i
a
d
a
t
a
2
.
2
.
Resea
rc
h g
a
p
Sev
er
al
s
tu
d
ies
lack
th
e
n
ec
e
s
s
ar
y
d
is
tin
ctio
n
o
f
attr
ib
u
tes
b
etwe
en
b
o
t
an
d
h
u
m
an
ac
c
o
u
n
ts
.
T
h
e
p
r
esen
t
wo
r
k
en
h
a
n
ce
s
th
e
ex
is
tin
g
liter
atu
r
e
in
th
is
f
ield
b
y
s
tu
d
y
in
g
b
e
h
av
io
u
r
al
p
atter
n
s
o
f
b
o
t
ac
co
u
n
ts
th
at
lead
to
f
u
r
th
e
r
id
e
n
tific
atio
n
o
f
f
ea
tu
r
es
o
n
d
if
f
er
en
t
lev
els.
Pre
v
io
u
s
ly
,
n
u
m
er
o
u
s
r
esear
ch
s
tu
d
ies
cr
ea
ted
b
o
t
d
etec
tio
n
m
eth
o
d
s
u
tili
zin
g
s
in
g
le
-
lev
el
class
if
ier
s
;
h
o
wev
er
,
d
e
p
en
d
i
n
g
o
n
th
e
p
r
ed
ictio
n
o
f
o
n
e
class
if
ier
wo
u
ld
n
o
t
b
e
s
u
f
f
ici
en
t
to
co
n
clu
d
e.
T
h
e
r
ef
o
r
e
,
to
im
p
r
o
v
e
t
h
e
ac
cu
r
ac
y
o
f
p
r
e
d
ictio
n
,
a
s
tack
in
g
ap
p
r
o
ac
h
h
as
b
ee
n
em
p
lo
y
e
d
with
class
if
ier
s
,
in
teg
r
atin
g
o
u
tp
u
ts
f
r
o
m
v
ar
i
o
u
s
class
if
ier
s
an
d
p
r
o
v
id
in
g
th
e
m
as in
p
u
t to
th
e
f
in
al
esti
m
ato
r
f
o
r
f
in
al
p
r
ed
ictio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
4
7
7
-
4
8
7
480
3.
T
H
E
P
RO
P
O
SE
D
M
E
T
H
O
D
T
h
is
s
tu
d
y
p
r
o
p
o
s
es
a
class
if
icatio
n
p
r
o
b
lem
to
d
eter
m
in
e
i
f
a
s
o
cial
m
e
d
ia
ac
co
u
n
t
is
o
p
er
ated
b
y
a
h
u
m
an
o
r
a
b
o
t
u
s
er
.
T
h
e
ex
p
er
im
en
t
h
as
two
m
o
d
u
les
–
f
i
r
s
tly
,
a
s
in
g
le
clas
s
if
ier
im
p
l
em
en
tatio
n
,
wh
er
e
class
if
ier
s
co
n
s
id
er
ed
f
o
r
th
e
ex
p
er
im
en
t
a
r
e
DT
,
R
F,
m
u
ltin
o
m
ial
Naïv
e
B
ay
es
(M
NB
)
,
L
R
,
s
u
p
p
o
r
t
v
ec
t
o
r
m
ac
h
in
e
(
SVM)
,
an
d
KNN
.
A
s
tack
in
g
class
if
ier
h
as
b
ee
n
u
s
ed
in
t
h
e
s
ec
o
n
d
m
o
d
u
le
o
f
th
e
ex
p
e
r
im
en
t
t
o
im
p
r
o
v
e
t
h
e
ac
cu
r
ac
y
o
f
th
e
cl
ass
if
icatio
n
r
esu
lt.
T
h
e
ex
p
er
im
e
n
t
h
as
u
s
ed
s
ev
e
r
al
ac
co
u
n
t
-
lev
el
f
ea
tu
r
es,
s
u
c
h
as:
“id
,
f
o
llo
wer
s
co
u
n
t,
f
r
ie
n
d
s
co
u
n
t,
lis
ted
co
u
n
t,
f
av
o
u
r
ites
co
u
n
t
,
v
er
if
ied
,
s
tatu
s
co
u
n
t,
d
ef
a
u
lt
p
r
o
f
ile,
d
e
f
au
lt
p
r
o
f
ile
im
ag
e,
s
cr
ee
n
n
am
e
,
lo
ca
tio
n
,
v
er
if
ied
”.
T
h
e
Scik
it
-
lear
n
lib
r
ar
y
o
f
Py
th
o
n
h
as b
ee
n
u
s
ed
f
o
r
th
e
im
p
lem
en
tatio
n
.
Fig
u
r
e
1
s
h
o
ws a
s
y
s
tem
atic
ap
p
r
o
ac
h
f
o
r
th
e
i
m
p
lem
en
tatio
n
o
f
t
h
e
p
r
o
p
o
s
e
d
b
o
t
d
etec
tio
n
m
ec
h
an
is
m
.
A
p
u
b
licly
a
v
ailab
le
d
ataset
f
r
o
m
Kag
g
le
h
as
b
ee
n
u
s
ed
to
tr
ain
th
e
m
o
d
el.
T
h
e
d
ataset
c
o
n
tain
s
th
e
class
if
ied
d
ata,
wh
ich
is
s
u
itab
le
f
o
r
s
u
p
er
v
is
ed
lear
n
in
g
an
d
tr
ai
n
in
g
th
e
m
o
d
el.
Fig
u
r
e
1
.
Me
th
o
d
o
lo
g
y
f
o
r
b
o
t
d
etec
tio
n
W
h
en
p
r
ep
ar
in
g
a
d
ataset
f
o
r
p
r
ed
ictiv
e
m
o
d
elin
g
,
f
ea
tu
r
e
en
g
in
ee
r
in
g
an
d
d
ata
p
r
ep
a
r
atio
n
ar
e
ess
en
tial
task
s
.
T
h
e
p
r
ep
ar
atio
n
s
tep
s
tar
ts
with
r
em
o
v
in
g
el
em
en
ts
th
at
d
o
n
o
t
s
ig
n
if
ican
t
ly
co
n
tr
i
b
u
te
to
th
e
p
r
ed
ictio
n
g
o
al,
s
u
ch
as
id
,
id
_
s
tr
,
s
cr
ee
n
_
n
am
e
,
an
d
u
r
l.
T
o
v
alid
ate
th
is
,
th
e
Sp
ea
r
m
an
co
r
r
elatio
n
co
ef
f
icien
t
h
as
b
ee
n
u
s
ed
to
ch
ec
k
th
e
co
r
r
elatio
n
an
d
th
e
d
ep
en
d
e
n
cy
o
f
f
ea
t
u
r
es
f
r
o
m
th
e
d
ataset.
As
a
r
esu
lt,
n
o
co
r
r
elatio
n
b
etwe
e
n
“id
,
s
tatu
s
es
co
u
n
t,
d
ef
a
u
lt
p
r
o
f
ile,
d
ef
au
lt
p
r
o
f
ile
im
a
g
e”
an
d
th
e
tar
g
et
v
ar
iab
le
was
o
b
s
er
v
ed
,
wh
e
r
e
as
a
s
tr
o
n
g
c
o
r
r
elatio
n
b
etwe
e
n
“v
e
r
if
ied
,
lis
ted
co
u
n
t,
f
r
ien
d
s
co
u
n
t,
f
o
llo
wer
s
co
u
n
t,
a
n
d
th
e
tar
g
et
v
ar
i
a
b
le”
was o
b
s
er
v
ed
.
I
n
th
e
s
ec
o
n
d
s
tep
o
f
d
ata
p
r
e
p
r
o
ce
s
s
in
g
,
m
is
s
in
g
v
alu
es
in
k
ey
c
o
lu
m
n
s
lik
e
l
o
ca
tio
n
,
d
escr
ip
tio
n
,
an
d
s
tatu
s
ar
e
ad
d
r
ess
ed
to
e
n
s
u
r
e
d
ata
r
eliab
ilit
y
.
T
o
e
n
ab
l
e
co
m
p
atib
ilit
y
with
m
ac
h
i
n
e
lear
n
in
g
alg
o
r
ith
m
s
,
th
e
ca
teg
o
r
ical
f
ea
tu
r
es
s
u
ch
as
lan
g
,
h
as
-
ex
ten
d
ed
-
p
r
o
f
ile
h
av
e
b
ee
n
tr
an
s
f
o
r
m
ed
in
to
n
u
m
er
ical
r
ep
r
esen
tatio
n
s
.
Ad
d
itio
n
ally
,
n
u
m
er
ical
f
ea
t
u
r
es
ar
e
n
o
r
m
al
ized
to
im
p
r
o
v
e
th
e
p
e
r
f
o
r
m
a
n
ce
o
f
m
o
d
els
lik
e
SVM
an
d
L
R
.
I
n
th
e
f
ea
tu
r
e
en
g
in
ee
r
in
g
p
h
ase,
r
elev
a
n
t
attr
ib
u
tes,
s
u
ch
as
f
o
llo
wer
s
_
co
u
n
t,
f
r
ien
d
s
_
co
u
n
t,
an
d
s
tatu
s
es_
co
u
n
t,
ar
e
s
elec
ted
to
f
o
cu
s
o
n
th
e
m
o
s
t
s
ig
n
if
i
ca
n
t
f
ea
tu
r
es.
On
e
-
h
o
t
en
c
o
d
i
n
g
h
as
b
ee
n
u
s
ed
to
en
s
u
r
e
th
at
t
h
e
ca
teg
o
r
ical
f
e
atu
r
es
in
th
e
d
ataset
ar
e
e
f
f
ec
tiv
ely
s
tr
u
ctu
r
ed
f
o
r
tr
ain
i
n
g
an
d
ev
al
u
atin
g
t
h
e
m
o
d
el
ac
cu
r
atel
y
.
3
.
1
.
Cla
s
s
if
ier
us
e
d
Dif
f
er
en
t
m
ac
h
in
e
lea
r
n
in
g
cl
ass
if
ier
s
,
DT
,
R
F,
MN
B
,
L
R
,
Ad
aBo
o
s
t,
an
d
SVM,
h
av
e
b
e
en
u
s
ed
f
o
r
s
im
ilar
k
in
d
s
o
f
class
if
icatio
n
p
r
o
b
lem
s
[
6
]
.
T
h
e
b
est
ac
c
u
r
ac
y
r
esu
lt
h
as
b
ee
n
s
h
o
wn
b
y
Ad
a
B
o
o
s
t
an
d
R
F,
with
an
ac
cu
r
ac
y
g
r
ea
ter
th
an
9
8
%.
Usi
n
g
a
m
u
lti
-
attr
ib
u
te
d
ataset
[
1
8
]
,
a
s
tu
d
y
was
co
n
d
u
cted
o
n
v
ar
i
o
u
s
m
ac
h
in
e
lear
n
i
n
g
alg
o
r
ith
m
t
h
at
in
clu
d
es
DT
,
n
eu
r
al
n
et
wo
r
k
s
,
Ko
h
o
n
en
m
ap
s
,
an
d
co
r
r
elatio
n
a
n
aly
s
is
.
Usi
n
g
v
ar
io
u
s
m
u
lti
-
attr
ib
u
te
d
ataset
attr
ib
u
tes,
th
e
p
r
o
p
o
s
e
d
m
eth
o
d
is
u
s
ed
f
o
r
p
r
ed
ictin
g
th
e
p
r
ice
s
eg
m
en
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
S
ta
ck
in
g
o
f m
a
ch
in
e
le
a
r
n
in
g
cla
s
s
ifie
r
s
fo
r
b
o
t d
etec
tio
n
u
s
in
g
a
cc
o
u
n
t le
ve
l d
a
ta
(
Jwa
la
S
h
a
r
ma
)
481
o
f
r
ea
l
estate.
T
h
e
o
v
er
all
e
x
p
er
im
en
t
r
esu
lt
s
h
o
ws
a
b
e
tter
m
ac
h
in
e
lear
n
in
g
alg
o
r
ith
m
f
o
r
p
r
ed
ictin
g
ac
cu
r
ac
y
i
n
ter
m
s
o
f
ac
cu
r
ac
y
.
T
h
e
s
tu
d
y
s
u
p
p
o
r
ts
th
at
th
e
R
FC
[
1
0
]
,
SVM
[
6
]
,
NB
,
SVM,
L
R
[
1
2
]
,
[
1
9
]
,
MN
B
C
[
9
]
,
R
FC
,
an
d
Ad
a
B
o
o
s
t
[
9
]
,
ar
e
th
e
b
est
p
er
f
o
r
m
i
n
g
class
if
ier
s
f
o
r
class
if
icatio
n
-
r
elate
d
p
r
o
b
lem
s
.
SVM
with
ar
tific
ial
n
eu
r
al
n
etw
o
r
k
(
ANN)
also
p
er
f
o
r
m
s
well
f
o
r
t
h
e
d
etec
tio
n
o
f
b
o
ts
th
r
o
u
g
h
tr
a
f
f
ic
b
eh
av
io
u
r
[
2
0
]
.
B
y
an
aly
s
in
g
th
e
b
o
tn
et
tr
af
f
ic
an
d
u
s
in
g
an
en
s
em
b
le
class
if
ier
,
th
e
s
tu
d
y
r
ev
ea
ls
th
at
u
s
in
g
a
co
m
b
in
atio
n
al
class
if
ier
wo
r
k
s
b
etter
th
an
a
s
in
g
le
class
if
ier
.
Var
io
u
s
in
f
lu
e
n
tial
wo
r
k
s
h
av
e
d
ep
icted
th
at
SVM,
DT
,
R
F,
Gr
ad
ien
t
B
o
o
s
tin
g
,
Ad
a
B
o
o
s
t,
XGB,
an
d
E
x
tr
a
T
r
ee
s
[
2
1
]
,
[
2
2
]
,
ar
e
th
e
m
o
s
t
ef
f
icien
t
a
n
d
p
o
p
u
lar
m
ac
h
in
e
lear
n
i
n
g
cla
s
s
if
ier
s
f
o
r
s
o
cial
m
ed
ia
b
o
t
d
etec
tio
n
.
T
h
e
p
e
r
f
o
r
m
an
ce
o
f
th
e
m
o
d
el
ca
n
b
e
en
h
an
ce
d
wit
h
th
e
u
s
e
o
f
v
ar
i
o
u
s
h
y
p
er
p
ar
a
m
eter
s
,
s
u
ch
as
th
e
n
u
m
b
er
o
f
tr
ee
s
,
d
ep
th
,
b
al
an
cin
g
a
n
d
s
p
litt
in
g
th
e
tr
ee
s
,
an
d
also
with
th
e
u
s
e
o
f
d
ata
f
ilter
in
g
an
d
p
r
o
ce
s
s
in
g
,
an
ef
f
ec
tiv
e
f
ea
tu
r
e
s
et,
an
d
em
p
l
o
y
in
g
th
e
s
tep
s
o
f
f
ea
tu
r
e
en
g
in
ee
r
in
g
an
d
f
ea
tu
r
e
e
x
tr
ac
tio
n
.
3
.
2
.
Alg
o
rit
hm
T
h
e
ex
p
er
im
e
n
t
h
as
b
ee
n
c
o
n
d
u
cte
d
u
s
in
g
Py
th
o
n
v
er
s
i
o
n
3
.
5
.
Scik
it
-
lear
n
h
as
b
ee
n
u
s
ed
to
im
p
lem
en
t
d
if
f
e
r
en
t
class
if
ier
s
to
ac
h
iev
e
th
e
s
am
e.
T
h
e
s
tep
-
wis
e
in
s
tr
u
ctio
n
h
as
b
ee
n
d
etailed
in
A
l
g
o
r
i
t
h
m
1
.
A
lg
o
r
ith
m
1
.
F
o
r
s
tack
in
g
e
n
s
em
b
le
class
if
ier
f
o
r
b
o
t d
etec
tio
n
Pseudocode for stacking classifier
Input:
Load and read the dataset D containing features X and target labels y
Output:
Target class (Final predicted class)
1.
Convert the target variable into categorical class l
abels (if required).
2.
Split the dataset into training and testing subsets:
o
Training data: 80%
o
Testing data: 20%
3.
Define base classifiers for stacking:
o
C
₁
, C
₂
, C
₃
, C
₄
, C
₅
, …, C
ₙ
(
e.
g.
,
SV
M,
K
NN
,
De
ci
si
on
T
re
e,
N
aï
ve
B
ay
es
,
Ra
nd
om
Forest)
4.
Train the individual
base classifiers using the training data:
o
Train each classifier Cᵢ using training features Xtrain and labels ytrain
o
Generate predictions on training data (ŷtrainᵢ)
o
Generate predictions on testing data (ŷtestᵢ)
5.
Create meta
-
training data for stacking:
o
Constr
uct a meta
-
feature matrix using base classifier predictions:
Xmeta_train = [ŷtrain
₁
, ŷtrain
₂
, …, ŷtrain
ₙ
]
6.
Define the meta
-
classifier:
o
Initialize meta
-
classifier C
ᵦ (
e.g., Logistic Regression or Decision Tree)
7.
Train the meta
-
classifier using
meta
-
features:
o
Train C
ᵦ
using Xmeta_train and original training labels ytrain
8.
Generate meta
-
test data and final prediction:
o
Construct meta
-
test features:
Xmeta_test = [ŷtest
₁
, ŷtest
₂
, …, ŷtest
ₙ
]
o
Predict final class labels using the meta
-
classifier
9.
Evaluate
the performance of the stacking classifier using:
o
Confusion matrix
o
Accuracy: overall correctness of predictions
o
Precision: accuracy of positive predictions
o
Recall: ability to detect all positive cases
o
F1
-
score: harmonic mean of precision and recall
o
ROC
–
AU
C curve: evaluates classifier performance across thresholds
10.
Save and deploy the trained stacking ensemble model.
11.
End
3
.
3
.
T
ra
ini
ng
of
t
he
cla
s
s
if
iers
Du
r
in
g
th
e
f
ir
s
t
p
h
ase
o
f
th
e
ex
p
er
im
e
n
t,
th
e
in
d
iv
id
u
al
cl
ass
if
ier
is
tr
ain
ed
with
th
e
g
i
v
en
s
et
o
f
d
ata.
T
h
e
g
e
n
er
al
s
tr
u
ctu
r
e
o
f
th
e
tr
ain
in
g
m
o
d
el,
as
in
Fig
u
r
e
2
,
s
tar
ts
with
th
e
ex
tr
ac
tio
n
o
f
f
ea
tu
r
es
(
X)
an
d
lab
els
(
y
)
f
r
o
m
th
e
tr
ain
in
g
d
a
ta.
T
h
en
,
a
class
if
ier
is
cr
ea
ted
with
s
p
ec
if
ic
p
ar
am
eter
s
,
f
o
ll
o
wed
b
y
s
ettin
g
th
e
p
ar
am
eter
s
f
o
r
ea
ch
class
if
ier
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
4
7
7
-
4
8
7
482
T
o
s
tan
d
ar
d
ize
f
ea
tu
r
es
f
o
r
b
etter
p
er
f
o
r
m
an
ce
o
f
m
u
ltin
o
m
ial
,
Stan
d
ar
d
Scaler
h
as
b
ee
n
u
s
ed
,
an
d
th
e
o
p
tim
al
v
alu
e
o
f
th
e
al
p
h
a
p
ar
am
eter
h
as
b
ee
n
u
s
in
g
Gr
id
Sear
ch
C
V.
Similar
ly
,
f
o
r
DT
an
d
R
F,
Gr
id
Sear
ch
C
V
is
u
s
ed
f
o
r
f
in
d
in
g
th
e
b
est
co
m
b
in
atio
n
o
f
c
r
iter
io
n
,
m
ax
_
d
ep
th
,
m
i
n
_
s
am
p
les_
leaf
,
an
d
en
s
u
r
in
g
th
e
tr
ain
-
test
s
p
lit
to
m
ain
tain
class
d
is
tr
ib
u
tio
n
u
s
in
g
s
tr
atif
y
=y
.
T
h
e
cr
iter
io
n
='
en
tr
o
p
y
'
in
d
icate
s
th
at
th
e
cr
iter
ia
f
o
r
m
ak
in
g
a
DT
wo
u
ld
b
e
b
ased
o
n
in
f
o
r
m
atio
n
g
ain
,
m
in
_
Sam
p
les
_
leaf
in
d
icate
s
th
e
m
in
im
u
m
n
u
m
b
er
o
f
s
am
p
les
r
eq
u
ir
ed
,
an
d
th
e
p
a
r
am
eter
m
in
_
s
am
p
les_
s
p
lit
in
d
icate
s
th
e
m
in
im
u
m
n
u
m
b
er
r
eq
u
ir
ed
to
s
p
lit an
in
ter
n
al
n
o
d
e.
T
o
en
s
u
r
e
co
n
v
e
r
g
en
ce
f
o
r
L
R
,
th
e
v
alu
e
o
f
C
h
as b
ee
n
s
et
a
s
1
0
0
0
an
d
m
ax
_
iter
=1
0
0
0
.
SVM
k
er
n
el
is
co
n
f
ig
u
r
e
d
with
a
r
ad
ial
b
asis
f
u
n
ctio
n
(
R
B
F),
wh
ich
m
an
ag
es
n
o
n
-
lin
ea
r
d
ec
is
io
n
b
o
u
n
d
ar
ies.
T
h
e
r
an
d
o
m
s
tate
i
s
s
et
to
0
,
wh
ich
g
u
ar
a
n
tees
th
at
th
e
o
u
tp
u
t
will
n
o
t
ch
an
g
e
ev
en
if
th
e
alg
o
r
it
h
m
is
ex
ec
u
ted
ag
ain
.
Ov
er
all,
f
o
r
a
c
o
m
p
r
e
h
en
s
iv
e
ev
alu
atio
n
,
a
co
n
f
u
s
io
n
m
atr
ix
a
n
d
,
class
if
icatio
n
r
ep
o
r
t
h
av
e
b
ee
n
u
s
ed
f
o
r
b
etter
in
ter
p
r
eta
b
ilit
y
o
f
ea
ch
class
if
ier
u
s
ed
.
T
h
e
R
OC
cu
r
v
e
h
as
b
ee
n
p
lo
tted
f
o
r
en
h
a
n
ce
d
v
is
u
aliza
tio
n
f
o
r
b
o
th
b
in
ar
y
an
d
m
u
lticlas
s
class
if
ier
s
.
T
h
e
s
ec
o
n
d
p
h
ase
o
f
th
e
ex
p
er
im
en
t
is
f
o
llo
wed
b
y
th
e
s
tack
in
g
o
f
th
e
class
if
ier
,
wh
ich
is
d
is
cu
s
s
ed
in
d
etail
in
th
e
f
o
llo
win
g
s
ec
tio
n
.
Fig
u
r
e
2
.
Me
th
o
d
o
lo
g
y
f
o
r
tr
ai
n
in
g
th
e
class
if
ier
s
3
.
4
.
Cre
a
t
ing
a
s
t
a
c
k
ing
cla
s
s
if
ier
Stack
in
g
is
th
e
s
im
p
le
p
r
o
ce
s
s
o
f
co
m
b
in
in
g
m
u
ltip
le
o
u
tp
u
ts
f
r
o
m
v
ar
io
u
s
class
if
ier
s
a
n
d
g
iv
in
g
th
em
as
in
p
u
t
to
th
e
f
in
al
es
tim
ato
r
f
o
r
f
i
n
al
p
r
ed
ictio
n
.
Fiv
e
d
if
f
er
en
t
class
if
ier
s
:
DT
,
L
R
,
K
-
n
eig
h
b
o
u
r
c
lass
if
ier
,
R
F,
an
d
SVM,
ar
e
u
s
ed
to
cr
ea
te
a
s
tack
i
n
g
class
if
ier
.
I
n
th
e
f
ir
s
t
s
tep
,
th
e
m
o
d
el
h
as
b
ee
n
in
s
tan
tiated
u
s
in
g
ab
o
v
e
ab
o
v
e
-
s
tated
class
if
ier
s
an
d
d
ef
in
e
d
th
e
Stack
in
g
C
lass
if
ier
s
u
s
in
g
L
R
as
th
e
f
in
al
esti
m
ato
r
.
T
h
e
m
o
d
el
is
th
en
ev
alu
ated
with
th
e
f
u
n
ctio
n
ev
alu
ate_
m
o
d
el
(
)
,
wh
ich
wil
l
ca
lcu
lat
e
v
ar
i
o
u
s
p
er
f
o
r
m
an
ce
m
etr
ics
s
u
ch
as
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
F
1
-
s
co
r
e,
an
d
AUC
-
R
OC
f
o
r
ea
ch
m
o
d
el.
At
th
e
f
in
al
s
tep
,
th
e
c
o
m
p
ar
is
o
n
o
f
all
in
d
i
v
id
u
al
class
if
ier
s
an
d
en
s
em
b
le
class
if
ier
s
is
ev
alu
ated
,
a
n
d
th
e
p
er
f
o
r
m
an
ce
o
f
ea
c
h
class
if
ier
is
s
to
r
ed
in
th
e
r
esu
lt
d
ictio
n
ar
y
(
r
esu
lt_
d
f
)
;
f
u
r
th
e
r
,
it
ca
n
b
e
u
s
ed
to
c
r
ea
te
a
b
ar
p
lo
t.
4.
RE
SU
L
T
S
AND
P
E
RF
O
RM
ANCE
M
E
T
RI
CS
On
e
o
f
th
e
s
im
p
lest
f
o
r
m
s
o
f
e
v
alu
atio
n
o
f
th
e
m
o
d
el’
s
ac
cu
r
ac
y
is
b
y
g
ettin
g
th
e
ac
c
u
r
ac
y
s
co
r
e.
T
o
d
eter
m
in
e
th
e
ac
cu
r
ac
y
,
s
co
r
e
to
tal
n
u
m
b
er
o
f
p
r
ed
ictio
n
s
is
d
iv
id
ed
b
y
th
e
n
u
m
b
er
o
f
co
r
r
ec
t
p
r
ed
ictio
n
s
m
ad
e
b
y
th
e
m
o
d
el.
Ma
th
em
at
ically
,
it is
wr
itten
as:
=
T
o
t
al
no
.
of
co
r
r
ect
p
r
ed
i
ct
i
o
n
s
T
o
t
al
n
umb
er
of
p
r
ed
i
ct
i
o
n
s
(
1
)
i
n
th
e
ca
s
e
o
f
b
in
ar
y
class
if
icatio
n
,
ac
cu
r
ac
y
m
ay
b
e
ass
ess
ed
u
s
in
g
p
o
s
itiv
e
a
n
d
n
eg
ativ
e
v
alu
es
,
wh
ic
h
ca
n
b
e
r
ep
r
esen
ted
as:
=
TP
+
TN
TP
+
TN
+
FP
+
FN
(
2
)
At
d
if
f
er
en
t
th
r
esh
o
ld
lev
els,
t
h
e
Ar
ea
u
n
d
er
th
e
c
u
r
v
e
-
r
ec
ei
v
er
o
p
e
r
atin
g
ch
ar
ac
ter
is
tic
(
AUC
-
R
O
C
)
cu
r
v
e
is
u
s
ed
to
ev
alu
ate
th
e
p
er
f
o
r
m
an
ce
o
f
a
class
if
icatio
n
m
o
d
el.
T
h
e
R
OC
cu
r
v
e
illu
s
tr
ates
h
o
w
well
th
e
m
o
d
el
d
is
tin
g
u
is
h
es
b
etwe
en
class
es
b
y
p
lo
ttin
g
th
e
tr
u
e
p
o
s
itiv
e
r
ate
ag
ain
s
t
th
e
f
alse
p
o
s
itiv
e
r
ate,
wh
ile
th
e
AUC
v
alu
e
in
d
icate
s
th
e
o
v
er
all
d
eg
r
ee
o
f
s
ep
ar
a
b
ilit
y
b
e
twee
n
class
e
s
.
A
h
ig
h
er
AUC
s
co
r
e
m
ea
n
s
th
e
m
o
d
el
is
b
etter
at
co
r
r
ec
tly
class
if
y
in
g
in
s
tan
ce
s
p
r
ed
ictin
g
0
s
as
0
an
d
1
s
as
1
in
b
in
ar
y
class
if
icatio
n
task
s
.
T
h
u
s
,
th
e
cl
o
s
er
th
e
AUC
is
t
o
1
,
th
e
m
o
r
e
e
f
f
ec
tiv
e
th
e
m
o
d
el
is
at
d
is
tin
g
u
is
h
in
g
b
etw
ee
n
b
o
t
an
d
n
o
n
-
b
o
t
ac
co
u
n
ts
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
S
ta
ck
in
g
o
f m
a
ch
in
e
le
a
r
n
in
g
cla
s
s
ifie
r
s
fo
r
b
o
t d
etec
tio
n
u
s
in
g
a
cc
o
u
n
t le
ve
l d
a
ta
(
Jwa
la
S
h
a
r
ma
)
483
W
h
en
th
e
R
OC
cu
r
v
e
r
is
es
h
ig
h
e
r
a
n
d
ap
p
r
o
ac
h
es
t
h
e
m
ax
im
u
m
th
r
esh
o
ld
v
alu
e,
it
s
ig
n
if
ies
im
p
r
o
v
e
d
class
if
icatio
n
p
er
f
o
r
m
an
ce
.
Fo
r
b
i
n
ar
y
class
if
icatio
n
,
th
e
AUC
is
ca
lcu
lated
u
s
in
g
th
e
s
ec
o
n
d
co
lu
m
n
o
f
t
h
e
p
r
o
b
ab
ilit
y
m
atr
ix
(
y
_
p
r
o
b
[
:,1
]
)
,
wh
er
ea
s
f
o
r
m
u
lticlas
s
p
r
o
b
lem
s
,
th
e
R
OC
-
AU
C
s
co
r
e
is
co
m
p
u
ted
u
s
in
g
th
e
o
n
e
-
vs
-
r
est (
Ov
R
)
s
tr
ateg
y
.
Ma
th
em
atica
lly
,
wh
ich
ca
n
b
e
d
er
i
v
ed
f
r
o
m
:
=
TP
TP
+
FN
(
3
)
T
r
u
e
p
o
s
itiv
e
d
en
o
tes
th
e
lik
elih
o
o
d
th
at
th
e
ac
tu
al
in
cid
e
n
t
will
b
e
clas
s
ed
as
p
o
s
itiv
e.
H
er
e
in
th
is
ex
p
er
im
en
t,
it
is
th
e
p
r
o
b
ab
ili
ty
th
at
ac
tu
al
b
o
t
ac
co
u
n
ts
will
b
e
class
if
ied
as
b
o
ts
,
an
d
h
u
m
an
ac
co
u
n
ts
a
s
h
u
m
an
o
r
n
o
t b
o
ts
:
=
FP
TN
+
FP
(
4
)
i
n
ca
s
e
o
f
a
f
alse
p
o
s
itiv
e,
it
r
ep
r
esen
ts
th
e
p
r
o
b
ab
ilit
y
o
r
th
e
m
ea
s
u
r
e
o
f
h
o
w
f
r
eq
u
en
tly
an
ac
tu
al
n
eg
ativ
e
in
s
tan
ce
will
b
e
class
if
ied
as
p
o
s
itiv
e.
A
class
if
icatio
n
r
ep
o
r
t
is
a
p
er
f
o
r
m
an
ce
ass
ess
m
en
t
m
ea
s
u
r
e
t
h
at
g
i
v
es
ac
cu
r
ac
y
,
r
ec
all,
F1
Sco
r
e,
a
n
d
s
u
p
p
o
r
t o
f
t
h
e
class
if
ier
s
u
s
ed
an
d
tr
ai
n
ed
d
u
r
in
g
im
p
lem
e
n
tatio
n
.
(
_
(
_
,
_
,
_
=
_
)
)
T
h
e
tar
g
et
n
a
m
e
h
er
e
d
ef
in
es
th
e
v
alu
e
o
f
p
r
e
d
icted
class
,
th
at
is
,
in
th
is
ca
s
e,
eith
er
0
o
r
1
,
an
d
th
e
f
u
n
ctio
n
class
if
icatio
n
r
ep
o
r
t
will
b
u
ild
a
tex
t
r
ep
o
r
t
th
at
s
h
o
ws
th
e
class
if
icatio
n
m
etr
ics.
T
h
e
f
o
u
r
m
ain
m
etr
ics
d
is
p
lay
ed
in
th
e
class
if
icatio
n
r
ep
o
r
t
ar
e
p
r
ec
is
io
n
a
n
d
r
ec
all.
F
-
1
s
co
r
e
a
n
d
s
u
p
p
o
r
t.
T
h
e
r
ec
all
o
f
a
class
if
ier
is
its
ab
ilit
y
to
d
et
ec
t
all
p
o
s
itiv
e
e
v
en
ts
[
1
7
]
.
T
h
e
F1
s
co
r
e
is
a
weig
h
ted
h
ar
m
o
n
ic
m
ea
n
o
f
ac
cu
r
ac
y
an
d
r
ec
all,
with
1
.
0
b
ein
g
th
e
b
est
an
d
0
.
0
r
ep
r
esen
tin
g
th
e
wo
r
s
t.
T
h
e
n
u
m
b
er
o
f
ac
tu
al
in
s
tan
ce
s
o
f
th
e
class
in
th
e
p
r
o
v
i
d
ed
d
atas
ets is
r
ef
er
r
ed
to
as su
p
p
o
r
t.
T
h
e
p
r
o
p
o
s
ed
s
tack
in
g
en
s
e
m
b
le
m
o
d
el
o
u
tp
e
r
f
o
r
m
ed
a
ll
in
d
iv
id
u
al
class
if
ier
s
,
d
em
o
n
s
tr
atin
g
s
u
p
er
io
r
r
esu
lts
ac
r
o
s
s
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
F1
s
co
r
e,
an
d
s
u
p
p
o
r
t
m
etr
ics.
A
1
0
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
was
u
s
ed
to
v
alid
ate
ea
ch
class
if
ier
’
s
ac
cu
r
ac
y
.
T
ab
le
2
p
r
esen
ts
th
e
p
er
f
o
r
m
a
n
ce
co
m
p
a
r
is
o
n
o
f
all
class
if
ier
s
,
wh
ile
Fig
u
r
e
3
v
is
u
alize
s
th
eir
r
elativ
e
p
er
f
o
r
m
an
ce
.
T
h
e
s
tack
in
g
class
if
ier
ac
h
iev
ed
th
e
h
ig
h
est
o
v
er
all
ac
cu
r
ac
y
o
f
0
.
9
0
1
wh
e
n
L
R
was
u
s
ed
as
a
b
ase
class
if
ier
.
E
f
f
ec
tiv
e
s
tack
in
g
was
ac
h
iev
ed
b
y
co
m
b
in
i
n
g
m
o
d
els
b
ased
o
n
m
u
lti
-
r
esp
o
n
s
e
lin
ea
r
r
eg
r
ess
io
n
an
d
p
r
o
b
a
b
ilit
y
d
is
tr
ib
u
tio
n
s
,
allo
win
g
R
F,
DT
,
L
R
,
a
n
d
KNN
to
co
m
p
lem
en
t
ea
ch
o
th
er
.
MN
B
was
ex
clu
d
ed
d
u
e
to
its
r
estrictio
n
to
n
o
n
-
n
e
g
ativ
e
v
alu
es.
Am
o
n
g
in
d
iv
i
d
u
al
m
o
d
els,
R
F
p
er
f
o
r
m
e
d
b
est
with
an
ac
cu
r
ac
y
o
f
0
.
8
9
8
,
f
o
llo
we
d
b
y
K
NN
(
0
.
8
7
4
)
an
d
DT
(
0
.
8
6
2
)
,
wh
ile
SVM
an
d
L
R
r
ec
o
r
d
ed
lo
wer
s
co
r
es
o
f
0
.
7
7
1
an
d
0
.
7
5
8
,
r
esp
ec
tiv
ely
.
T
h
ese
f
in
d
in
g
s
co
n
f
ir
m
th
e
s
u
p
er
io
r
ity
o
f
th
e
s
tack
in
g
en
s
em
b
le,
wh
ich
lev
er
ag
es
th
e
u
n
iq
u
e
s
tr
en
g
th
s
o
f
m
u
ltip
le
class
if
ier
s
to
ac
h
iev
e
en
h
an
ce
d
r
o
b
u
s
tn
ess
an
d
ac
cu
r
ac
y
.
T
ab
le
2
.
Su
m
m
a
r
y
o
f
ac
cu
r
ac
y
s
co
r
es
M
o
d
e
l
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F1
S
u
p
p
o
r
t
DT
0
.
8
6
2
0
.
8
7
2
0
.
8
6
2
0
.
8
5
1
8
4
0
RF
0
.
8
9
8
0
.
8
9
8
0
.
8
9
8
0
.
8
9
7
8
4
0
LR
0
.
7
5
8
0
.
7
5
8
0
.
7
5
8
0
.
7
5
8
8
4
0
KNN
0
.
8
7
4
0
.
8
7
4
0
.
8
7
4
0
.
8
7
4
8
4
0
S
V
M
0
.
7
7
1
0
.
7
7
1
0
.
7
7
1
0
.
7
7
1
8
4
0
S
t
a
c
k
i
n
g
c
l
a
ss
i
f
i
e
r
0
.
9
0
1
0
.
9
0
1
0
.
9
0
0
0
.
9
0
1
8
4
0
Fig
u
r
es
4
an
d
5
d
is
p
lay
th
e
AUC
-
R
O
C
cu
r
v
e
f
o
r
in
d
iv
id
u
al
class
if
ier
s
th
at
h
av
e
b
ee
n
tes
ted
in
th
e
ex
p
er
im
en
t
a
n
d
th
e
s
tack
in
g
class
if
ier
.
AU
C
m
ea
s
u
r
es
th
e
ab
ilit
y
o
f
a
class
if
ier
to
class
if
y
p
o
s
itiv
e
an
d
n
eg
ativ
e
class
es
ac
cu
r
ately
.
T
h
e
d
iag
o
n
al
lin
e
in
th
e
f
i
g
u
r
e
i
s
th
e
p
er
f
o
r
m
an
ce
o
f
a
r
an
d
o
m
class
if
ier
,
wh
er
e
a
p
r
ed
ictio
n
is
m
ad
e
r
an
d
o
m
ly
.
Fro
m
F
ig
u
r
es
4
an
d
5
,
it
ca
n
b
e
o
b
s
er
v
e
d
th
at
th
e
AUC
o
f
t
h
e
R
F
an
d
s
tack
in
g
class
if
ier
s
i
s
0
.
9
6
,
wh
ich
in
d
icate
s
th
at
th
ey
ca
n
p
r
ed
ict
th
e
b
o
t
lab
el
an
d
h
u
m
an
la
b
e
l
m
o
r
e
ac
cu
r
ately
.
Fo
llo
wed
th
e
p
er
f
o
r
m
an
ce
o
f
KNN
with
th
e
AU
C
o
f
0
.
9
3
,
wh
ich
s
h
o
ws
g
o
o
d
p
e
r
f
o
r
m
an
ce
as
co
m
p
ar
ed
to
SVM,
DT
,
an
d
L
R
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
4
7
7
-
4
8
7
484
F
ig
u
r
e
3
.
Per
f
o
r
m
an
c
e
co
m
p
ar
is
o
n
o
f
v
a
r
io
u
s
class
if
ier
s
Fig
u
r
e
4
.
R
OC
-
AUC cu
r
v
es o
f
v
ar
io
u
s
class
if
ier
s
Fig
u
r
e
5
.
R
OC
-
AUC cu
r
v
es o
f
s
tack
in
g
class
if
ier
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
I
SS
N:
2252
-
8
7
7
6
S
ta
ck
in
g
o
f m
a
ch
in
e
le
a
r
n
in
g
cla
s
s
ifie
r
s
fo
r
b
o
t d
etec
tio
n
u
s
in
g
a
cc
o
u
n
t le
ve
l d
a
ta
(
Jwa
la
S
h
a
r
ma
)
485
T
h
e
s
tack
in
g
class
if
ier
ef
f
ec
ti
v
ely
co
m
b
in
es
th
e
s
tr
en
g
th
s
o
f
in
d
iv
id
u
al
class
if
ier
s
,
en
h
an
cin
g
o
v
er
all
p
r
ed
ictio
n
p
er
f
o
r
m
an
ce
.
T
h
e
r
esu
lts
in
d
icate
th
at
th
e
R
F
class
if
ier
p
er
f
o
r
m
e
d
clo
s
ely
to
t
h
e
s
tack
in
g
m
o
d
el
in
ter
m
s
o
f
AUC
an
d
o
th
er
e
v
alu
atio
n
m
etr
ics.
Ho
wev
e
r
,
wh
ile
R
F
m
ay
ex
h
ib
it
s
lig
h
t
o
v
er
f
itti
n
g
d
u
r
in
g
tr
ain
in
g
,
th
e
s
tack
in
g
ap
p
r
o
ac
h
m
itig
ates
th
is
is
s
u
e
b
y
ag
g
r
eg
atin
g
p
r
ed
ictio
n
s
f
r
o
m
m
u
ltip
le
m
o
d
els.
T
h
e
in
clu
s
io
n
o
f
a
m
eta
-
lear
n
er
f
u
r
th
er
r
ef
in
es
th
e
f
in
al
o
u
tp
u
t,
i
m
p
r
o
v
i
n
g
g
en
er
aliza
tio
n
to
u
n
s
ee
n
d
ata.
Ov
er
all,
s
tack
in
g
p
r
o
b
ab
ilis
tic
m
o
d
els
with
m
u
lti
-
r
esp
o
n
s
e
lin
ea
r
r
e
g
r
ess
io
n
p
r
o
d
u
ce
s
s
u
p
er
io
r
class
if
icatio
n
r
esu
lts
.
T
h
e
p
r
o
p
o
s
ed
s
tack
in
g
class
if
ier
d
em
o
n
s
tr
ates
s
ig
n
if
ican
t
im
p
r
o
v
em
e
n
t
in
s
o
ci
al
m
e
d
ia
b
o
t
d
etec
tio
n
,
ac
h
iev
in
g
p
e
r
f
o
r
m
an
ce
c
o
m
p
a
r
ab
le
to
o
r
e
x
ce
ed
in
g
th
at
o
f
ex
is
tin
g
s
tate
-
of
-
th
e
-
ar
t
m
et
h
o
d
s
,
as
s
u
m
m
ar
ized
in
T
ab
le
3
.
T
h
e
p
r
o
p
o
s
ed
s
tack
in
g
class
if
ier
r
esu
lted
in
an
o
u
ts
tan
d
in
g
a
cc
u
r
ac
y
o
f
9
0
%
ac
r
o
s
s
v
ar
io
u
s
o
th
er
b
o
t
d
etec
tio
n
a
p
p
r
o
ac
h
es
.
Ver
y
c
lo
s
e
to
th
e
p
r
o
p
o
s
ed
s
tack
in
g
class
if
ier
,
R
F
h
as
also
r
esu
lted
in
v
er
y
g
o
o
d
p
er
f
o
r
m
an
ce
ac
r
o
s
s
v
ar
io
u
s
m
etr
ics,
wh
ich
h
as
b
ee
n
d
is
cu
s
s
ed
an
d
d
etailed
in
T
ab
le
2
.
T
h
e
r
esu
lt
d
is
cu
s
s
e
d
in
T
ab
le
3
also
em
p
h
asizes
th
e
v
ar
io
u
s
f
ea
tu
r
es
u
s
ed
f
o
r
s
o
cial
b
o
t
d
etec
tio
n
.
T
h
e
m
o
s
t
p
r
o
m
in
e
n
t
f
ea
tu
r
es
u
s
ed
ac
r
o
s
s
v
a
r
io
u
s
s
tu
d
ies
a
r
e
ac
co
u
n
t
-
lev
el
d
ata
a
n
d
te
x
tu
al
d
ata,
a
n
d
also
em
o
tico
n
s
ar
e
u
s
ed
in
s
o
m
e
s
tu
d
ies.
Du
r
in
g
s
tack
in
g
v
ar
i
o
u
s
class
if
ier
s
,
th
e
d
iv
er
s
ity
o
f
class
if
ier
s
h
as
b
ee
n
en
s
u
r
ed
b
y
co
n
s
id
e
r
i
n
g
a
v
ar
iety
o
f
class
if
ier
s
s
u
ch
as
R
F,
SVM,
DT
,
L
R
,
an
d
KNN
cl
ass
if
ier
s
.
T
h
e
s
ec
o
n
d
im
p
o
r
tan
t
f
ac
to
r
tak
e
n
ca
r
e
o
f
is
th
e
ch
o
ice
o
f
th
e
m
eta
-
c
lass
if
ier
wh
ile
b
u
ild
in
g
th
e
s
t
ac
k
in
g
class
if
ier
.
T
h
e
f
ac
t
th
at
L
R
i
s
s
im
p
le
an
d
q
u
ite
ef
f
ec
tiv
e
i
n
co
m
b
in
in
g
t
h
e
p
r
e
d
ictio
n
s
is
o
n
e
o
f
th
e
r
ea
s
o
n
s
to
s
elec
t
it
f
o
r
a
m
eta
-
class
if
ier
.
T
o
f
in
e
-
tu
n
e
th
e
m
o
d
el,
g
r
i
d
s
ea
r
ch
h
as
b
ee
n
u
s
ed
,
an
d
last
ly
,
to
e
v
alu
ate
an
d
v
a
lid
ate
th
e
r
esu
lts
,
cr
o
s
s
-
v
alid
atio
n
.
h
as b
ee
n
p
e
r
f
o
r
m
ed
.
T
ab
le
3
.
C
o
m
p
ar
is
o
n
o
f
s
tack
i
n
g
class
if
ier
with
ex
is
tin
g
m
eth
o
d
s
A
u
t
h
o
r
/
r
e
f
/
y
e
a
r
M
o
d
e
l
/
c
l
a
ssi
f
i
e
r
F
e
a
t
u
r
e
s
R
e
s
u
l
t
Lu
b
i
s
e
t
a
l
.
[
2
3
]
,
2
0
2
4
D
e
e
p
l
e
a
r
n
i
n
g
-
C
N
N
-
8
6
.
0
%
V
e
l
a
sc
o
-
mat
a
e
t
a
l
.
[
1
5
]
,
2
0
2
1
DT
,
RF
,
a
n
d
KNN
B
o
t
n
e
t
t
r
a
f
f
i
c
f
e
a
t
u
r
e
s
8
5
%
(
h
i
g
h
e
st
a
c
c
u
r
a
c
y
o
f
DT
)
P
r
a
mi
t
h
a
e
t
a
l
.
[
2
4
]
,
2
0
2
1
DT
,
K
-
N
e
a
r
e
s
t
N
e
i
g
h
b
o
r
s
,
LR
,
a
n
d
N
a
ï
v
e
B
a
y
e
s.
T
e
x
t
b
a
se
d
8
7
.
2
%
(
h
i
g
h
e
st
a
c
c
u
r
a
c
y
o
f
DT
)
M
a
n
d
l
o
i
a
n
d
P
a
t
e
l
.
[
2
5
]
,
2
0
2
0
N
a
ï
v
e
B
a
y
e
s
,
S
V
M
,
a
n
d
max
i
mu
m e
n
t
r
o
p
y
T
w
i
t
t
e
r
d
a
t
a
u
ser
d
a
t
a
a
n
d
t
e
x
t
-
b
a
s
e
d
8
3
.
3
%
Y
a
n
g
e
t
a
l
.
[
9
]
,
2
0
2
0
RF
U
ser me
t
a
d
a
t
a
f
e
a
t
u
r
e
s
/
d
e
r
i
v
e
d
f
e
a
t
u
r
e
s
8
4
%
Li
e
t
a
l
.
[
1
2
]
,
2
0
1
8
M
N
B
c
l
a
ssi
f
i
e
r
Emo
t
i
c
o
n
s
a
n
d
e
mo
j
i
7
4
.
9
%
V
a
n
D
e
r
W
a
l
t
a
n
d
El
o
f
f
[
1
0
]
,
2
0
1
8
S
V
M
,
RF
,
A
d
a
B
o
o
s
t
I
d
e
n
t
i
t
y
,
b
e
h
a
v
i
o
u
r
,
a
n
d
r
e
l
a
t
i
o
n
s
h
i
p
6
8
.
0
5
%
,
8
7
.
1
1
%,
8
5
.
9
1
%
T
e
st
e
d
DT
A
c
c
o
u
n
t
-
l
e
v
e
l
d
a
t
a
/
d
e
r
i
v
e
d
f
e
a
t
u
r
e
s
8
6
.
2
%
T
e
st
e
d
RF
-
d
o
-
8
9
.
8
%
T
e
st
e
d
K
N
N
-
d
o
-
8
7
.
4
%
T
e
st
e
d
LR
-
d
o
-
7
5
.
8
%
T
e
st
e
d
S
V
M
-
d
o
-
7
7
.
1
%
S
t
a
c
k
i
n
g
c
l
a
ss
i
f
i
e
r
[
P
r
o
p
o
se
d
]
S
t
a
c
k
i
n
g
=
{
RF
,
DT
,
LR
,
K
N
N
,
S
V
M
}
A
c
c
o
u
n
t
-
l
e
v
e
l
d
a
t
a
/
d
e
r
i
v
e
d
f
e
a
t
u
r
e
s
9
0
%
5.
CO
NCLU
SI
O
N
T
h
is
s
tu
d
y
h
elp
s
d
ef
in
e
h
o
w
b
o
t
ac
co
u
n
ts
b
eh
a
v
e
o
n
s
o
cia
l
m
ed
ia
b
y
r
ev
iewin
g
d
if
f
e
r
e
n
t
r
esear
ch
f
in
d
in
g
s
a
n
d
co
m
p
ar
in
g
th
em
with
h
u
m
a
n
u
s
er
p
atter
n
s
.
B
o
t
s
ten
d
t
o
b
e
m
o
r
e
ac
tiv
e
th
a
n
r
ea
l
u
s
er
s
,
p
o
s
tin
g
f
ar
m
o
r
e
f
r
e
q
u
en
tly
b
u
t
h
av
in
g
f
ewe
r
f
o
llo
wer
s
an
d
f
o
llo
wi
n
g
m
an
y
m
o
r
e
ac
co
u
n
ts
.
T
h
e
y
also
u
s
e
d
if
f
e
r
en
t
ty
p
es
o
f
co
n
ten
t
,
esp
ec
ially
lin
k
s
,
h
ash
tag
s
,
an
d
m
en
tio
n
s
,
m
ak
in
g
th
em
s
tan
d
o
u
t
f
r
o
m
g
en
u
in
e
u
s
er
s
.
T
h
ese
p
atter
n
s
,
alo
n
g
with
ac
tiv
ity
a
n
d
n
etwo
r
k
f
ea
tu
r
es,
ca
n
r
eliab
ly
d
is
tin
g
u
is
h
b
o
ts
f
r
o
m
h
u
m
an
s
.
I
n
ter
esti
n
g
l
y
,
s
tu
d
ies
also
s
h
o
w
th
at
m
o
d
er
n
s
o
cial
s
p
am
b
o
ts
ar
e
b
ec
o
m
in
g
m
o
r
e
s
o
p
h
is
ticated
,
o
f
ten
m
im
ick
in
g
h
u
m
an
b
eh
av
io
r
s
o
well
th
at
th
ey
ca
n
ev
en
f
o
o
l
e
x
p
er
ien
ce
d
u
s
er
s
.
Am
o
n
g
th
em
,
ab
o
u
t
8
.
6
%
ar
e
s
u
s
p
en
d
ed
b
o
ts
,
wh
ile
8
8
.
9
%
r
e
m
ain
ac
tiv
e.
I
n
th
e
n
e
x
t
p
a
r
t
o
f
th
is
wo
r
k
,
v
ar
io
u
s
m
ac
h
i
n
e
lear
n
i
n
g
m
o
d
els
wer
e
ap
p
lied
t
o
d
etec
t
s
o
cial
b
o
ts
u
s
in
g
ac
c
o
u
n
t
-
lev
el
f
ea
tu
r
es.
T
h
e
p
r
o
p
o
s
ed
s
tack
in
g
class
if
ier
o
u
tp
e
r
f
o
r
m
ed
th
e
in
d
iv
id
u
al
m
o
d
els,
ac
h
iev
in
g
h
i
g
h
er
ac
c
u
r
ac
y
,
F1
-
s
co
r
e,
r
e
ca
ll,
an
d
s
u
p
p
o
r
t
v
alu
es.
W
ith
LR
as
th
e
m
eta
-
class
if
ier
,
th
e
s
tack
in
g
m
o
d
el
r
ea
ch
e
d
a
9
0
%
ac
cu
r
ac
y
r
ate
,
p
r
o
v
in
g
th
at
co
m
b
in
in
g
m
u
ltip
le
m
o
d
el
s
an
d
tu
n
in
g
th
eir
p
ar
am
eter
s
ca
n
s
ig
n
if
ican
tly
i
m
p
r
o
v
e
p
r
e
d
ictio
n
ac
c
u
r
ac
y
.
Sin
ce
s
o
cial
m
ed
ia
d
ata
is
c
o
m
p
lex
a
n
d
d
i
v
er
s
e,
f
u
tu
r
e
wo
r
k
c
o
u
ld
e
x
p
lo
r
e
m
o
r
e
f
ea
tu
r
es
s
u
ch
as
UR
L
s
,
h
ash
tag
s
,
an
d
h
y
p
er
lin
k
s
f
r
o
m
u
s
er
p
o
s
ts
.
Fu
r
th
er
im
p
r
o
v
em
e
n
ts
co
u
ld
also
b
e
ac
h
iev
ed
b
y
ap
p
ly
i
n
g
f
ea
tu
r
e
s
elec
tio
n
an
d
h
y
p
er
p
ar
a
m
eter
o
p
tim
izatio
n
m
eth
o
d
s
to
m
a
k
e
th
e
m
o
d
els e
v
en
m
o
r
e
ef
f
ic
ien
t a
n
d
ad
ap
ta
b
le.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
7
7
6
I
n
t J I
n
f
&
C
o
m
m
u
n
T
ec
h
n
o
l
,
Vo
l.
1
5
,
No
.
2
,
J
u
n
e
20
2
6
:
4
7
7
-
4
8
7
486
F
UNDING
I
NF
O
R
M
A
T
I
O
N
T
h
e
au
th
o
r
s
d
ec
lar
e
th
at
n
o
f
u
n
d
s
,
g
r
a
n
ts
,
o
r
o
th
e
r
s
u
p
p
o
r
t
wer
e
r
ec
eiv
ed
d
u
r
in
g
t
h
e
p
r
e
p
ar
atio
n
o
f
th
is
m
an
u
s
cr
ip
t.
AUTHO
R
CO
NT
RI
B
UT
I
O
NS ST
A
T
E
M
E
N
T
Th
is
j
o
u
r
n
a
l
u
s
es
t
h
e
c
o
n
tri
b
u
to
r
ro
les
tax
o
n
o
m
y
(CRe
d
i
T)
t
o
re
c
o
g
n
ize
i
n
d
i
v
id
u
a
l
a
u
th
o
r
c
o
n
tri
b
u
ti
o
n
s,
re
d
u
c
e
a
u
th
o
rs
h
ip
d
is
p
u
tes
,
a
n
d
fa
c
il
it
a
te
c
o
ll
a
b
o
ra
ti
o
n
.
Na
m
e
o
f
Aut
ho
r
C
M
So
Va
Fo
I
R
D
O
E
Vi
Su
P
J
wala
Sh
ar
m
a
✓
✓
✓
✓
✓
✓
✓
✓
Sam
ar
jeet
B
o
r
ah
✓
✓
✓
✓
✓
✓
C
:
C
o
n
c
e
p
t
u
a
l
i
z
a
t
i
o
n
M
:
M
e
t
h
o
d
o
l
o
g
y
So
:
So
f
t
w
a
r
e
Va
:
Va
l
i
d
a
t
i
o
n
Fo
:
Fo
r
mal
a
n
a
l
y
si
s
I
:
I
n
v
e
s
t
i
g
a
t
i
o
n
R
:
R
e
so
u
r
c
e
s
D
:
D
a
t
a
C
u
r
a
t
i
o
n
O
:
W
r
i
t
i
n
g
-
O
r
i
g
i
n
a
l
D
r
a
f
t
E
:
W
r
i
t
i
n
g
-
R
e
v
i
e
w
&
E
d
i
t
i
n
g
Vi
:
Vi
su
a
l
i
z
a
t
i
o
n
Su
:
Su
p
e
r
v
i
s
i
o
n
P
:
P
r
o
j
e
c
t
a
d
m
i
n
i
s
t
r
a
t
i
o
n
Fu
:
Fu
n
d
i
n
g
a
c
q
u
i
s
i
t
i
o
n
CO
NF
L
I
C
T
O
F
I
N
T
E
R
E
S
T
ST
A
T
E
M
E
NT
O
n
b
eh
alf
o
f
all
th
e
a
u
th
o
r
s
,
th
e
co
r
r
esp
o
n
d
in
g
au
th
o
r
s
tates th
at
th
er
e
ar
e
n
o
c
o
n
f
licts
o
f
in
t
er
est.
E
T
H
I
CAL
AP
P
RO
V
AL
No
E
th
ical
ap
p
r
o
v
al
was r
eq
u
ir
ed
f
o
r
th
is
s
tu
d
y
,
s
in
ce
it r
elie
d
o
n
p
u
b
licly
a
v
ailab
le
d
ata
.
DATA AV
AI
L
AB
I
L
I
T
Y
T
h
e
d
ata
th
at
s
u
p
p
o
r
t
th
e
f
in
d
in
g
s
o
f
th
is
s
tu
d
y
ar
e
o
p
e
n
ly
av
ailab
le
on
Kag
g
le.
(
n
.
d
.
)
.
T
witter
d
ataset
-
f
ilter
ed
.
UR
L
:
h
ttp
s
://www
.
k
ag
g
le.
co
m
/
d
atasets
/k
ag
g
led
atasettb
d
/twitt
er
d
ataset
-
f
ilter
ed
.
RE
F
E
R
E
NC
E
S
[
1
]
V
.
S
.
S
u
b
r
a
h
m
a
n
i
a
n
e
t
a
l
.
,
“
Th
e
D
A
R
P
A
Tw
i
t
t
e
r
b
o
t
c
h
a
l
l
e
n
g
e
,
”
C
o
m
p
u
t
e
r
,
v
o
l
.
4
9
,
n
o
.
6
,
p
p
.
3
8
-
4
6
,
J
u
n
.
2
0
1
6
,
d
o
i
:
1
0
.
1
1
0
9
/
M
C
.
2
0
1
6
.
1
8
3
.
[
2
]
S
.
C
.
W
o
o
l
l
e
y
,
“
A
u
t
o
m
a
t
i
n
g
p
o
w
e
r
:
S
o
c
i
a
l
b
o
t
i
n
t
e
r
f
e
r
e
n
c
e
i
n
g
l
o
b
a
l
p
o
l
i
t
i
c
s,”
Fi
rs
t
Mo
n
d
a
y
,
v
o
l
.
2
1
,
n
o
.
4
,
M
a
r
.
2
0
1
6
,
d
o
i
:
1
0
.
5
2
1
0
/
f
m
.
v
2
1
i
4
.
6
1
6
1
.
[
3
]
E.
F
e
r
r
a
r
a
,
O
.
V
a
r
o
l
,
C
.
D
a
v
i
s,
F
.
M
e
n
c
z
e
r
,
a
n
d
A
.
F
l
a
mm
i
n
i
,
“
Th
e
r
i
se
o
f
so
c
i
a
l
b
o
t
s,”
C
o
m
m
u
n
i
c
a
t
i
o
n
s
o
f
t
h
e
A
C
M
,
v
o
l
.
5
9
,
n
o
.
7
,
p
p
.
9
6
–
1
0
4
,
J
u
n
.
2
0
1
6
,
d
o
i
:
1
0
.
1
1
4
5
/
2
8
1
8
7
1
7
.
[
4
]
A
.
B
e
ssi
a
n
d
E.
F
e
r
r
a
r
a
,
“
S
o
c
i
a
l
b
o
t
s
d
i
s
t
o
r
t
t
h
e
2
0
1
6
U
.
S
.
P
r
e
si
d
e
n
t
i
a
l
e
l
e
c
t
i
o
n
o
n
l
i
n
e
d
i
sc
u
ss
i
o
n
,
”
Fi
rs
t
Mo
n
d
a
y
,
v
o
l
.
2
1
,
n
o
.
1
1
,
N
o
v
.
2
0
1
6
,
d
o
i
:
1
0
.
5
2
1
0
/
f
m.
v
2
1
i
1
1
.
7
0
9
0
.
[
5
]
R
.
G
a
l
l
o
t
t
i
,
F
.
V
a
l
l
e
,
N
.
C
a
s
t
a
l
d
o
,
P
.
S
a
c
c
o
,
a
n
d
M
.
D
e
D
o
m
e
n
i
c
o
,
“
A
ssess
i
n
g
t
h
e
r
i
s
k
s o
f
‘
i
n
f
o
d
e
mi
c
s’
i
n
r
e
s
p
o
n
se
t
o
C
O
V
I
D
-
1
9
e
p
i
d
e
mi
c
s,”
Na
t
u
r
e
H
u
m
a
n
B
e
h
a
v
i
o
u
r
,
v
o
l
.
4
,
n
o
.
1
2
,
p
p
.
1
2
8
5
–
1
2
9
3
,
O
c
t
.
2
0
2
0
,
d
o
i
:
1
0
.
1
0
3
8
/
s4
1
5
6
2
-
0
2
0
-
0
0
9
9
4
-
6.
[
6
]
S
.
K
u
d
u
g
u
n
t
a
a
n
d
E
.
F
e
r
r
a
r
a
,
“
D
e
e
p
n
e
u
r
a
l
n
e
t
w
o
r
k
s
f
o
r
b
o
t
d
e
t
e
c
t
i
o
n
,
”
I
n
f
o
rm
a
t
i
o
n
S
c
i
e
n
c
e
s
,
v
o
l
.
4
6
7
,
p
p
.
3
1
2
–
3
2
2
,
O
c
t
.
2
0
1
8
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
i
n
s
.
2
0
1
8
.
0
8
.
0
1
9
.
[
7
]
Z.
G
i
l
a
n
i
,
R
.
F
a
r
a
h
b
a
k
h
s
h
,
G
.
T
y
so
n
,
a
n
d
J.
C
r
o
w
c
r
o
f
t
,
“
A
l
a
r
g
e
-
sc
a
l
e
b
e
h
a
v
i
o
u
r
a
l
a
n
a
l
y
si
s
o
f
b
o
t
s
a
n
d
h
u
man
s
o
n
t
w
i
t
t
e
r
,
”
AC
M
T
ra
n
s
a
c
t
i
o
n
s
o
n
t
h
e
We
b
,
v
o
l
.
1
3
,
n
o
.
1
,
p
p
.
1
–
2
3
,
F
e
b
.
2
0
1
9
,
d
o
i
:
1
0
.
1
1
4
5
/
3
2
9
8
7
8
9
.
[
8
]
S
.
C
r
e
s
c
i
,
A
.
S
p
o
g
n
a
r
d
i
,
M
.
P
e
t
r
o
c
c
h
i
,
M
.
Te
s
c
o
n
i
,
a
n
d
R
.
D
i
P
i
e
t
r
o
,
“
T
h
e
p
a
r
a
d
i
g
m
-
s
h
i
f
t
o
f
s
o
c
i
a
l
sp
a
m
b
o
t
s:
E
v
i
d
e
n
c
e
,
t
h
e
o
r
i
e
s,
a
n
d
t
o
o
l
s
f
o
r
t
h
e
a
r
ms
r
a
c
e
,
”
i
n
2
6
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
W
o
r
l
d
Wi
d
e
We
b
C
o
n
f
e
r
e
n
c
e
2
0
1
7
,
WW
W
2
0
1
7
C
o
m
p
a
n
i
o
n
,
N
e
w
Y
o
r
k
,
N
e
w
Y
o
r
k
,
U
S
A
:
A
C
M
P
r
e
ss
,
2
0
1
7
,
p
p
.
9
6
3
–
972
,
d
o
i
:
1
0
.
1
1
4
5
/
3
0
4
1
0
2
1
.
3
0
5
5
1
3
5
.
[
9
]
K
.
C
.
Y
a
n
g
,
O
.
V
a
r
o
l
,
P
.
M
.
H
u
i
,
a
n
d
F
.
M
e
n
c
z
e
r
,
“
S
c
a
l
a
b
l
e
a
n
d
g
e
n
e
r
a
l
i
z
a
b
l
e
s
o
c
i
a
l
b
o
t
d
e
t
e
c
t
i
o
n
t
h
r
o
u
g
h
d
a
t
a
sel
e
c
t
i
o
n
,
”
AA
AI
2
0
2
0
-
3
4
t
h
A
AAI
C
o
n
f
e
re
n
c
e
o
n
Art
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
v
o
l
.
3
4
,
n
o
.
0
1
,
p
p
.
1
0
9
6
-
1
1
0
3
,
A
p
r
.
2
0
2
0
,
d
o
i
:
1
0
.
1
6
0
9
/
a
a
a
i
.
v
3
4
i
0
1
.
5
4
6
0
.
[
1
0
]
E.
V
a
n
D
e
r
W
a
l
t
a
n
d
J
.
E
l
o
f
f
,
“
U
s
i
n
g
m
a
c
h
i
n
e
l
e
a
r
n
i
n
g
t
o
d
e
t
e
c
t
f
a
k
e
i
d
e
n
t
i
t
i
e
s:
b
o
t
s
v
s
h
u
m
a
n
s
,
”
I
EEE
Ac
c
e
ss
,
v
o
l
.
6
,
p
p
.
6
5
4
0
–
6
5
4
9
,
2
0
1
8
,
d
o
i
:
1
0
.
1
1
0
9
/
A
C
C
ESS
.
2
0
1
8
.
2
7
9
6
0
1
8
.
[
1
1
]
R
.
J.
O
e
n
t
a
r
y
o
,
A
.
M
u
r
d
o
p
o
,
P
.
K
.
P
r
a
set
y
o
,
a
n
d
E
.
P
.
Li
m,
“
O
n
p
r
o
f
i
l
i
n
g
b
o
t
s
i
n
s
o
c
i
a
l
me
d
i
a
,
”
i
n
L
e
c
t
u
r
e
N
o
t
e
s
i
n
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
(
i
n
c
l
u
d
i
n
g
s
u
b
ser
i
e
s
L
e
c
t
u
re
N
o
t
e
s
i
n
Ar
t
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
a
n
d
L
e
c
t
u
re
N
o
t
e
s
i
n
B
i
o
i
n
f
o
rm
a
t
i
c
s)
,
v
o
l
.
1
0
0
4
6
LN
C
S
,
2
0
1
6
,
p
p
.
9
2
–
1
0
9
,
d
o
i
:
1
0
.
1
0
0
7
/
9
7
8
-
3
-
3
1
9
-
4
7
8
8
0
-
7
_
6
.
[
1
2
]
M
.
L
i
,
E.
C
h
’
n
g
,
A
.
Y
.
L.
C
h
o
n
g
,
a
n
d
S
.
S
e
e
,
“
M
u
l
t
i
-
c
l
a
ss
Tw
i
t
t
e
r
se
n
t
i
m
e
n
t
c
l
a
ss
i
f
i
c
a
t
i
o
n
w
i
t
h
e
m
o
j
i
s,
”
I
n
d
u
st
ri
a
l
M
a
n
a
g
e
m
e
n
t
a
n
d
D
a
t
a
S
y
s
t
e
m
s
,
v
o
l
.
1
1
8
,
n
o
.
9
,
p
p
.
1
8
0
4
–
1
8
2
0
,
S
e
p
.
2
0
1
8
,
d
o
i
:
1
0
.
1
1
0
8
/
I
M
D
S
-
12
-
2
0
1
7
-
0
5
8
2
.
[
1
3
]
B
.
M
ø
n
s
t
e
d
,
P
.
S
a
p
i
e
ż
y
ń
s
k
i
,
E.
F
e
r
r
a
r
a
,
a
n
d
S
.
L
e
h
m
a
n
n
,
“
E
v
i
d
e
n
c
e
o
f
c
o
mp
l
e
x
c
o
n
t
a
g
i
o
n
o
f
i
n
f
o
r
mat
i
o
n
i
n
so
c
i
a
l
me
d
i
a
:
A
n
e
x
p
e
r
i
me
n
t
u
s
i
n
g
Tw
i
t
t
e
r
b
o
t
s
,
”
PLo
S
O
N
E
,
v
o
l
.
1
2
,
n
o
.
9
,
p
.
e
0
1
8
4
1
4
8
,
S
e
p
.
2
0
1
7
,
d
o
i
:
1
0
.
1
3
7
1
/
j
o
u
r
n
a
l
.
p
o
n
e
.
0
1
8
4
1
4
8.
[
1
4
]
A
.
A
r
i
s
t
a
,
“
C
o
mp
a
r
i
s
o
n
d
e
c
i
s
i
o
n
t
r
e
e
a
n
d
l
o
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
m
a
c
h
i
n
e
l
e
a
r
n
i
n
g
c
l
a
ssi
f
i
c
a
t
i
o
n
a
l
g
o
r
i
t
h
ms
t
o
d
e
t
e
r
mi
n
e
C
o
v
i
d
-
1
9
,
”
S
i
n
k
r
o
n
,
v
o
l
.
7
,
n
o
.
1
,
p
p
.
5
9
–
6
5
,
J
a
n
.
2
0
2
2
,
d
o
i
:
1
0
.
3
3
3
9
5
/
s
i
n
k
r
o
n
.
v
7
i
1
.
1
1
2
4
3
.
Evaluation Warning : The document was created with Spire.PDF for Python.