I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
1
5
,
No
.
5
,
Octo
b
er
2
0
2
6
,
p
p
.
4
4
3
2
~
4
4
4
6
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
5
.
p
p
4
4
3
2
-
4
4
4
6
4432
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
Fine
-
tuning la
ng
u
a
g
e mo
dels for so
cia
l sentimen
t:
in
sig
hts f
ro
m
sta
rtup
use
r f
eed
ba
ck
Li
-
H
ua
L
i
1
,
Ag
us
Ca
hy
o
Nu
g
ro
ho
1,
2
1
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
o
n
M
a
n
a
g
e
men
t
,
C
h
a
o
y
a
n
g
U
n
i
v
e
r
s
i
t
y
o
f
T
e
c
h
n
o
l
o
g
y
,
Ta
i
c
h
u
n
g
,
Ta
i
w
a
n
2
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
o
n
S
y
s
t
e
m,
F
a
c
u
l
t
y
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
S
o
e
g
i
j
a
p
r
a
n
a
t
a
C
a
t
h
o
l
i
c
U
n
i
v
e
r
si
t
y
,
S
e
m
a
r
a
n
g
,
I
n
d
o
n
e
s
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
u
n
2
5
,
2
0
2
5
R
ev
is
ed
J
u
l 2
0
,
2
0
2
6
Acc
ep
ted
Au
g
1
2
,
2
0
2
6
S
tartu
p
g
r
o
wth
a
n
d
su
sta
i
n
a
b
il
i
t
y
a
re
h
e
a
v
il
y
imp
a
c
ted
b
y
u
se
r
c
o
m
m
e
n
ts.
Th
is
re
se
a
rc
h
h
a
s
d
o
n
e
a
c
o
m
p
a
r
a
ti
v
e
stu
d
y
b
e
twe
e
n
larg
e
la
n
g
u
a
g
e
m
o
d
e
l
s
(LL
M
s),
sm
a
ll
lan
g
u
a
g
e
m
o
d
e
ls
(S
L
M
s)
,
a
n
d
trad
it
io
n
a
l
m
a
c
h
in
e
lea
rn
in
g
(M
L)
m
e
th
o
d
s
i
n
c
las
sify
in
g
u
se
r
c
o
m
m
e
n
ts
o
n
sta
rt
u
p
s.
Th
e
p
e
rfo
rm
a
n
c
e
s
o
f
b
id
irec
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
r
e
se
n
tatio
n
s
fr
o
m
tran
sf
o
rm
e
rs
(
BERT
)
,
In
d
o
n
e
sia
n
b
i
d
irec
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
re
se
n
tatio
n
s
fr
o
m
tra
n
sfo
rm
e
rs
(
In
d
o
BERT
)
,
d
isti
ll
e
d
b
id
ire
c
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
re
se
n
tatio
n
s
fro
m
tran
sfo
rm
e
rs
(
Distil
BERT
),
a
n
d
M
L
m
o
d
e
ls;
l
o
g
isti
c
re
g
re
ss
io
n
,
su
p
p
o
rt
v
e
c
to
r
m
a
c
h
in
e
(
S
VM
)
,
ra
n
d
o
m
fo
re
st
,
a
n
d
e
x
trem
e
g
ra
d
ie
n
t
b
o
o
sti
n
g
(
XG
Bo
o
st
)
we
re
e
v
a
lu
a
ted
wit
h
a
d
i
v
e
rse
d
a
tas
e
t
a
n
d
b
e
y
o
n
d
e
x
ten
siv
e
p
re
p
ro
c
e
ss
in
g
.
O
u
t
o
f
a
ll
t
h
e
m
o
d
e
ls
t
h
is
re
se
a
rc
h
train
e
d
,
I
n
d
o
B
ERT
o
u
t
p
e
rfo
rm
e
d
wit
h
a
n
a
c
c
u
ra
c
y
a
n
d
re
c
a
ll
9
0
.
9
9
%
a
n
d
F
1
-
sc
o
re
o
f
8
9
.
9
4
%
,
d
u
e
to
i
ts i
n
term
e
d
iate
p
re
train
i
n
g
fo
r
In
d
o
n
e
sia
n
la
n
g
u
a
g
e
tas
k
.
Di
stil
BERT
,
fro
m
th
e
S
LM
fa
m
il
y
o
f
m
o
d
e
ls
a
n
d
o
n
ly
re
q
u
iri
n
g
a
fra
c
ti
o
n
o
f
re
so
u
rc
e
s
to
m
a
k
e
p
re
d
ictio
n
s
p
e
rf
o
rm
e
d
sig
n
ifi
c
a
n
tl
y
b
e
tt
e
r
(o
v
e
r
9
0
%
in
b
o
t
h
a
c
c
u
ra
c
y
a
n
d
re
c
a
ll
).
T
h
e
se
o
u
tco
m
e
s
in
d
i
c
a
te
th
a
t
LL
M
s,
p
a
rti
c
u
larl
y
th
o
s
e
targ
e
ti
n
g
sp
e
c
ifi
c
lan
g
u
a
g
e
s,
a
re
sig
n
ifi
c
a
n
tl
y
su
p
e
rio
r
in
p
e
rfo
rm
a
n
c
e
f
o
r
se
n
ti
m
e
n
t
a
n
a
ly
sis.
Th
e
re
su
lt
s
se
e
m
t
o
im
p
ly
th
a
t
m
o
d
e
ls
li
k
e
t
h
e
se
c
a
n
b
e
u
ti
li
z
e
d
to
stu
d
y
t
h
e
p
h
e
n
o
m
e
n
o
n
a
n
d
fu
rt
h
e
r
e
n
h
a
n
c
e
u
se
r
e
n
g
a
g
e
m
e
n
t
with
sta
rtu
p
s.
F
u
tu
re
wo
r
k
will
i
n
v
o
lv
e
th
e
fu
r
t
h
e
r
d
e
v
e
lo
p
m
e
n
t
o
f
m
o
re
lan
g
u
a
g
e
sp
e
c
ifi
c
BERT
v
e
rsio
n
s a
n
d
e
x
p
a
n
d
i
n
g
L
LM
s in
o
t
h
e
r
wa
y
s with
in
sta
rt
u
p
sp
a
c
e
s.
K
ey
w
o
r
d
s
:
L
ar
g
e
lan
g
u
ag
e
m
o
d
els
Ma
ch
in
e
lear
n
in
g
Sen
tim
en
t a
n
aly
s
is
Sm
all
lan
g
u
ag
e
m
o
d
els
Star
tu
p
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Ag
u
s
C
ah
y
o
Nu
g
r
o
h
o
Dep
ar
tm
en
t o
f
I
n
f
o
r
m
atio
n
M
an
ag
em
en
t,
C
h
ao
y
an
g
Un
iv
er
s
ity
o
f
T
ec
h
n
o
lo
g
y
T
aich
u
n
g
,
T
aiwa
n
E
m
ail: a
g
u
s
.
n
u
g
r
o
h
o
@
u
n
ik
a.
a
c.
id
1.
I
NT
RO
D
UCT
I
O
N
Sen
tim
en
t
an
aly
s
is
[
1
]
with
o
u
t
a
d
o
u
b
t
o
n
e
o
f
th
e
m
o
s
t
im
p
o
r
tan
t
m
eth
o
d
/tech
n
iq
u
es
f
o
r
co
m
p
an
ies
in
I
n
d
o
n
esia
r
an
g
in
g
f
r
o
m
s
tar
t
-
u
p
to
co
r
p
o
r
ate
an
d
ev
en
g
o
v
er
n
m
e
n
t.
As
n
ew
co
m
p
an
ie
s
g
r
o
w,
it
b
ec
o
m
es
ess
en
tial
to
an
aly
ze
s
en
tim
en
t
[
2
]
f
r
o
m
m
an
y
s
o
u
r
ce
s
–
f
r
o
m
n
ews
s
to
r
ies
an
d
s
o
cial
m
ed
ia
to
co
n
s
u
m
e
r
ev
alu
atio
n
s
.
I
n
c
o
n
tr
ast,
d
esp
i
te
th
e
s
u
cc
ess
f
u
l
ap
p
licatio
n
o
f
s
en
tim
en
t
an
aly
s
is
in
m
an
y
ca
s
es,
th
er
e
is
n
o
t
m
u
ch
r
esear
ch
wo
r
k
s
th
at
co
m
b
in
es
tr
ad
itio
n
al
n
atu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
(
NL
P)
tech
n
iq
u
es
an
d
m
o
d
e
r
n
tr
an
s
f
o
r
m
er
-
b
ased
m
o
d
el
s
p
ec
if
ically
f
o
r
I
n
d
o
n
esian
s
tar
t
u
p
co
m
p
an
ies.
T
h
is
tech
g
a
p
is
co
n
s
id
er
ab
le
—
p
ar
ticu
lar
ly
with
th
e
s
ca
le
o
f
lin
g
u
is
tic
f
r
ag
m
e
n
tatio
n
,
r
ap
i
d
ly
in
c
r
ea
s
in
g
d
ig
ital
ad
o
p
tio
n
an
d
e
v
er
-
g
r
o
win
g
ec
o
n
o
m
ic
p
o
ten
tial
th
at
I
n
d
o
n
esian
s
tar
tu
p
s
r
ep
r
esen
t.
C
lo
s
in
g
th
is
g
ap
ca
n
d
eliv
e
r
b
ett
er
,
m
o
r
e
ac
tio
n
a
b
le
in
s
ig
h
ts
f
o
r
th
ese
n
ew
k
in
d
s
o
f
b
u
s
in
ess
es.
Yet,
th
er
e
ar
e
also
ch
allen
g
es
r
eg
ar
d
in
g
I
n
d
o
n
esian
lan
g
u
ag
e
s
en
tim
en
t
an
aly
s
is
s
u
ch
as
lin
g
u
is
tic
v
ar
iatio
n
s
,
in
f
o
r
m
al
ex
p
r
ess
io
n
s
an
d
lac
k
o
f
lab
ele
d
d
ata.
Stan
d
ar
d
NL
P
m
eth
o
d
s
s
u
ch
as
m
ac
h
in
e
lear
n
in
g
(
ML
)
-
b
ased
s
en
tim
e
n
t
class
if
ier
s
[
3
]
ar
e
co
m
m
o
n
ly
em
p
lo
y
ed
f
o
r
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
F
in
e
-
tu
n
in
g
la
n
g
u
a
g
e
mo
d
els
fo
r
s
o
cia
l sen
timen
t:
in
s
ig
h
ts
fr
o
m
s
ta
r
tu
p
u
s
er fee
d
b
a
ck
(
Li
-
Hu
a
Li
)
4433
s
en
tim
en
t
an
aly
s
is
.
R
ec
en
tly
,
n
o
v
el
s
m
all
lan
g
u
ag
e
m
o
d
els
(
SLM
)
an
d
lar
g
e
lan
g
u
ag
e
m
o
d
els
(
L
L
Ms)
h
av
e
allo
wed
n
ew
o
p
p
o
r
tu
n
ities
f
o
r
s
en
tim
en
t
an
aly
s
is
[
4
]
.
L
L
Ms
u
s
e
a
lar
g
e
am
o
u
n
t
o
f
c
o
m
p
u
tatio
n
al
p
o
wer
,
b
u
t
th
ey
p
r
o
v
id
e
b
etter
c
o
n
tex
t
u
a
l
u
n
d
er
s
tan
d
in
g
th
an
SLM
s
;
o
n
th
e
o
t
h
er
h
a
n
d
,
SLM
s
o
f
f
er
a
m
o
r
e
e
f
f
icien
t
tr
ad
eo
f
f
b
etwe
en
p
e
r
f
o
r
m
an
ce
an
d
r
eso
u
r
ce
u
tili
za
tio
n
.
T
h
is
p
ap
er
ev
alu
ates
th
e
p
er
f
o
r
m
an
ce
o
f
tr
ad
itio
n
al
ML
m
o
d
els,
SLM
,
an
d
L
L
Ms
f
o
r
s
en
tim
en
t
an
aly
s
is
o
n
I
n
d
o
n
esian
s
tar
t
u
p
s
.
T
h
e
o
b
jectiv
e
o
f
th
is
r
esear
ch
is
to
ev
alu
ate
th
e
m
b
ased
o
n
th
eir
p
er
f
o
r
m
an
ce
,
e
f
f
icien
cy
a
n
d
u
s
e
as
a
p
r
ac
tical
asp
ec
t
f
o
r
s
en
tim
en
t
an
aly
s
is
o
v
er
s
o
m
e
tex
tu
al
tex
ts
.
I
n
a
n
y
tr
ad
itio
n
al
s
en
tim
en
t
an
aly
s
is
,
th
e
ML
m
o
d
els
[
3
]
,
f
o
r
in
s
tan
ce
,
n
aï
v
e
B
ay
es
[
3
]
,
s
u
p
p
o
r
t
v
ec
to
r
m
ac
h
i
n
es
(
SVM)
[
3
]
,
o
r
lo
g
is
tic
r
eg
r
es
s
io
n
[
3
]
,
e
x
tr
ac
t
h
an
d
cr
af
ted
f
ea
tu
r
es
s
u
ch
as
ter
m
f
r
e
q
u
en
cy
an
d
u
s
e
th
em
to
class
if
y
tex
t
in
to
p
o
s
itiv
e,
n
e
g
ativ
e
o
r
n
eu
tr
al
s
en
tim
en
ts
.
T
h
ese
ar
e
lig
h
t
weig
h
t
a
n
d
i
n
ter
p
r
etab
le
m
o
d
els
th
o
u
g
h
th
ey
o
f
ten
f
ail
to
u
n
d
er
s
tan
d
co
n
tex
t,
s
ar
ca
s
m
o
r
th
e
n
u
an
ce
s
o
f
ex
p
r
ess
io
n
.
No
w,
I
n
d
o
n
esia
’
s
s
tar
tu
p
ec
o
s
y
s
tem
h
as
b
ee
n
g
r
o
win
g
q
u
i
ck
ly
th
r
o
u
g
h
v
a
r
io
u
s
r
o
u
n
d
s
o
f
f
u
n
d
in
g
,
esp
ec
ially
f
o
r
ty
p
es
lik
e
f
in
tech
,
e
-
co
m
m
er
ce
a
n
d
ed
tech
.
W
ith
in
cr
ea
s
in
g
co
m
p
etitio
n
,
ac
tu
al
u
s
er
f
ee
d
b
ac
k
is
p
r
o
v
in
g
m
o
r
e
p
ar
am
o
u
n
t
f
o
r
s
tar
tu
p
s
t
o
h
elp
r
ed
u
ce
th
e
tim
e
tak
e
n
to
r
ea
c
h
p
r
o
d
u
ct
-
m
ar
k
et
f
it,
im
p
r
o
v
e
u
s
er
ex
p
er
ien
ce
a
n
d
ac
h
iev
e
g
r
o
wth
in
a
co
n
s
tan
tly
c
h
an
g
in
g
m
ar
k
et.
T
h
is
r
esear
ch
p
r
o
v
id
es
a
r
ef
er
en
ce
in
to
th
e
b
est
p
r
ac
tice
o
f
s
en
tim
en
t a
n
aly
s
is
in
I
n
d
o
n
esian
s
tar
tu
p
ec
o
s
y
s
tem
b
y
c
o
n
n
ec
tin
g
class
ical
NL
P
ap
p
r
o
ac
h
to
m
o
d
er
n
lan
g
u
ag
e
m
o
d
el
s
o
th
at
s
tar
tu
p
ca
n
m
a
k
e
d
ata
d
r
i
v
en
d
ec
is
io
n
b
ased
o
n
p
u
b
lic
s
en
tim
en
t.
T
h
is
r
esear
ch
aim
s
to
f
in
d
o
u
t
th
e
b
est
an
d
m
o
s
t
f
ea
s
ib
ilit
y
m
eth
o
d
s
f
o
r
an
aly
zi
n
g
u
s
er
s
en
tim
en
t
o
n
I
n
d
o
n
esian
s
tar
tu
p
s
.
I
n
th
is
wa
y
,
th
e
s
tu
d
y
in
ten
d
s
to
h
elp
s
tar
t
-
u
p
s
g
o
in
g
th
r
o
u
g
h
d
if
f
er
e
n
t sen
tim
en
t a
n
aly
s
is
to
o
ls
b
ased
o
n
th
eir
tec
h
n
ical
ca
p
ac
ity
an
d
b
u
s
in
ess
n
ee
d
s
a
s
well
as
to
u
s
e
u
s
er
f
ee
d
b
ac
k
in
d
ec
is
io
n
-
m
a
k
in
g
.
T
h
e
f
o
llo
win
g
ar
e
t
h
is
s
tu
d
y
’
s
p
r
im
a
r
y
co
n
tr
ib
u
tio
n
s
:
i
)
a
c
o
m
p
r
eh
en
s
iv
e
co
m
p
a
r
i
s
o
n
o
f
ML
-
b
ased
,
SLM
-
b
ased
,
an
d
L
L
M
-
b
ased
s
en
tim
en
t
an
aly
s
is
ap
p
r
o
ac
h
es
;
an
d
ii)
e
v
alu
atio
n
o
f
th
e
im
p
ac
t
o
f
d
if
f
er
en
t
m
o
d
el
ar
ch
itectu
r
es o
n
s
en
tim
en
t c
lass
if
icatio
n
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
an
d
F1
-
s
co
r
e
.
T
h
is
p
ap
er
is
o
r
g
an
ize
d
as
f
o
ll
o
ws:
th
e
in
tr
o
d
u
ctio
n
o
f
SLM
,
L
L
M,
ML
,
s
tar
tu
p
in
f
o
r
m
atio
n
,
an
d
th
e
r
ev
iew
o
f
th
e
liter
atu
r
es
ar
e
co
v
er
ed
in
s
ec
tio
n
2
.
Sectio
n
3
m
eth
o
d
s
,
d
etails
ab
o
u
t
d
a
ta
co
llectio
n
,
d
ata
p
r
ep
r
o
ce
s
s
in
g
,
f
in
e
tu
n
in
g
,
e
v
alu
atio
n
m
etr
ics,
a
n
d
ex
p
e
r
im
en
tal
d
esig
n
.
Sectio
n
4
r
esu
lt
s
an
d
d
is
cu
s
s
io
n
s
.
T
h
e
k
ey
f
in
d
in
g
s
ar
e
co
n
clu
d
e
d
in
s
ec
tio
n
5
al
o
n
g
with
r
ec
o
m
m
en
d
atio
n
s
f
o
r
f
u
tu
r
e
r
esear
ch
.
2.
L
I
T
E
R
AT
U
RE
R
E
VI
E
W
2
.
1
.
O
v
er
v
iew
s
t
a
rt
up
s
in I
n
do
nes
ia
An
I
n
d
o
n
esian
e
-
co
m
m
e
r
ce
s
i
te
ca
lled
B
lib
li
at
h
ttp
s
://www.
b
lib
li.c
o
m
/
was
estab
lis
h
ed
i
n
2
0
1
1
a
n
d
p
r
o
v
id
es
a
lar
g
e
s
elec
tio
n
o
f
g
o
o
d
s
an
d
s
er
v
ices,
in
clu
d
i
n
g
q
u
ick
s
h
ip
m
en
t
an
d
s
af
e
p
ay
m
en
t
m
eth
o
d
s
.
E
s
tab
lis
h
ed
in
2
0
0
9
,
T
o
k
o
p
e
d
ia
at
h
ttp
s
://www.
to
k
o
p
e
d
ia.
co
m
/
is
a
p
r
o
m
in
en
t
e
-
co
m
m
er
ce
p
latf
o
r
m
t
h
at
f
ac
ilit
ates
b
u
s
in
ess
-
to
-
b
u
s
in
es
s
an
d
co
n
s
u
m
er
-
to
-
co
n
s
u
m
er
t
r
an
s
ac
tio
n
s
,
th
er
eb
y
e
m
p
o
wer
in
g
s
m
all
an
d
lo
ca
l
en
ter
p
r
is
es.
B
u
k
alap
ak
at
h
t
tp
s
://www.
b
u
k
alap
ak
.
co
m
/
(
f
o
u
n
d
e
d
in
2
0
1
0
)
co
n
n
ec
ts
s
m
all
an
d
m
ed
iu
m
en
ter
p
r
is
es
(
SME
s
)
with
co
n
s
u
m
er
s
,
s
ellin
g
d
iv
er
s
e
p
r
o
d
u
ct
s
an
d
o
f
f
er
in
g
f
in
an
cial
tech
n
o
lo
g
y
s
o
lu
tio
n
s
lik
e
d
ig
ital
p
ay
m
e
n
ts
.
DANA
at
h
ttp
s
://www.
d
an
a.
id
/
(
f
o
u
n
d
e
d
in
2
0
1
8
)
a
d
i
g
ital
wallet
p
r
o
v
id
i
n
g
ca
s
h
less
tr
an
s
ac
tio
n
s
,
b
ill
p
a
y
m
en
ts
a
n
d
p
er
s
o
n
-
to
-
p
er
s
o
n
(
P2
P
)
tr
an
s
f
er
s
.
Go
jek
s
tar
ted
in
2
0
1
0
as
a
co
u
r
ier
a
n
d
tr
an
s
p
o
r
tatio
n
c
o
m
p
a
n
y
,
b
ec
o
m
in
g
a
m
eg
a
a
p
p
with
m
o
r
e
th
an
2
0
s
er
v
ices
ac
r
o
s
s
So
u
th
ea
s
t
Asi
a.
OVO
at
h
ttp
s
://www.
o
v
o
.
id
/
(
estab
lis
h
ed
in
2
0
1
7
)
-
a
p
a
y
m
en
t
p
latf
o
r
m
th
at
o
f
f
er
s
d
ig
ital
p
a
y
m
en
ts
,
m
o
b
ile
to
p
-
u
p
an
d
b
ills
,
an
d
lo
y
alty
-
b
ased
r
ewa
r
d
s
p
r
o
g
r
am
b
y
wo
r
k
in
g
with
m
er
ch
an
ts
an
d
d
ig
ital
p
lay
er
p
ar
tn
er
s
.
R
u
an
g
g
u
r
u
at
h
ttp
s
://www.
r
u
an
g
g
u
r
u
.
co
m
/
f
o
u
n
d
ed
in
2
0
1
4
to
im
p
r
o
v
e
th
e
ac
ce
s
s
ib
ilit
y
o
f
e
d
u
c
atio
n
f
o
r
s
tu
d
en
ts
in
I
n
d
o
n
esia.
T
r
av
elo
k
a
at
h
ttp
s
://www.
tr
av
elo
k
a.
co
m
/en
-
en
(
estab
lis
h
ed
in
201
2
)
is
an
o
n
lin
e
tr
av
el
a
n
d
life
s
ty
le
p
latf
o
r
m
,
o
f
f
er
s
s
er
v
i
ce
s
o
n
f
lig
h
t a
n
d
h
o
tel
b
o
o
k
in
g
s
th
r
o
u
g
h
o
u
t So
u
th
ea
s
t A
s
ia.
T
h
ese
wer
e
ch
o
s
en
b
ec
au
s
e
th
ey
n
o
t
o
n
ly
r
ep
r
esen
t
a
b
r
o
ad
cr
o
s
s
s
ec
tio
n
o
f
I
n
d
o
n
esia
’
s
d
ig
ital
ec
o
n
o
m
y
in
cl
u
d
in
g
e
-
c
o
m
m
er
ce
,
f
in
tech
,
e
d
tech
,
an
d
tr
av
el
,
b
u
t
lik
e
q
u
ality
,
also
h
av
e
e
n
o
u
g
h
ac
tiv
e
u
s
er
s
g
iv
in
g
f
ee
d
b
ac
k
o
n
th
ei
r
p
latf
o
r
m
s
.
Du
e
to
th
eir
s
ig
n
if
ican
t
r
o
les
in
th
e
I
n
d
o
n
esian
s
tar
tu
p
ec
o
s
y
s
tem
,
th
ese
p
latf
o
r
m
s
m
ak
e
g
r
ea
t
ca
s
e
s
tu
d
ies
f
o
r
ap
p
ly
in
g
a
n
d
ass
ess
in
g
s
en
tim
en
t
an
aly
s
is
m
o
d
els
b
ec
au
s
e
co
m
m
en
ts
m
ad
e
b
y
u
s
er
s
o
n
th
ese
p
latf
o
r
m
s
ca
n
r
e
v
ea
l
co
n
s
u
m
er
ex
p
ec
tatio
n
s
,
s
atis
f
ac
tio
n
,
an
d
is
s
u
es.
Sen
tim
en
t
an
aly
s
is
f
r
o
m
th
ese
f
ir
m
s
p
r
o
v
id
es
th
is
r
esear
ch
with
an
o
p
p
o
r
tu
n
ity
to
in
v
esti
g
ate
p
r
ac
tical
u
s
es
o
f
ML
,
SLM
,
an
d
L
L
M
ap
p
r
o
ac
h
es in
v
ar
io
u
s
b
u
s
in
ess
s
itu
atio
n
s
.
2
.
2
.
Sentim
ent
a
na
ly
s
is
Sen
tim
en
t
an
aly
s
is
(
also
k
n
o
w
n
as
o
p
i
n
io
n
m
in
in
g
)
is
a
k
ey
ar
ea
o
f
NL
P
[
5
]
th
at
f
o
cu
s
es
o
n
f
in
d
in
g
,
ex
tr
ac
tin
g
,
an
d
class
if
y
in
g
s
u
b
jectiv
e
in
f
o
r
m
atio
n
in
tex
tu
al
d
ata
to
d
eter
m
in
e
wh
eth
er
th
e
ex
p
r
ess
ed
o
p
in
io
n
is
n
eu
tr
al,
n
eg
ativ
e
,
o
r
p
o
s
itiv
e.
T
h
is
p
r
o
ce
s
s
is
es
s
en
tial
f
o
r
d
ec
ip
h
er
in
g
co
n
s
u
m
e
r
s
en
tim
en
ts
,
m
ar
k
et
tr
en
d
s
,
an
d
p
u
b
lic
s
en
tim
en
t
to
m
a
k
e
b
etter
d
ec
is
io
n
s
th
r
o
u
g
h
a
v
a
r
iety
o
f
in
d
u
s
tr
ies.
I
t
is
th
e
u
s
e
o
f
co
m
p
u
tatio
n
al
tech
n
iq
u
es
to
d
is
co
v
er
a
n
d
ex
tr
ac
t
o
p
in
i
o
n
s
f
r
o
m
tex
t
d
ata
r
eg
ar
d
i
n
g
p
r
o
d
u
cts,
s
er
v
i
ce
s
,
o
r
g
an
izatio
n
s
,
in
d
iv
id
u
als,
o
r
e
v
en
ts
[
6
]
.
Se
v
er
al
s
tu
d
ies
6
r
ev
iew
s
en
tim
en
t
an
aly
s
is
m
eth
o
d
s
in
d
etail;
th
ey
co
v
er
b
o
th
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
5
,
Octo
b
er
2
0
2
6
:
4
4
3
2
-
4
4
4
6
4434
class
ical
ML
[
7
]
li
k
e
n
aïv
e
B
ay
es,
SVM,
an
d
lo
g
is
tic
r
eg
r
ess
io
n
,
as
wel
l
as
s
tate
-
of
-
th
e
-
ar
t
d
ee
p
lear
n
in
g
o
r
tr
an
s
f
o
r
m
er
-
b
ased
m
eth
o
d
s
u
s
in
g
b
id
ir
ec
tio
n
al
en
co
d
e
r
r
e
p
r
esen
tatio
n
s
f
r
o
m
tr
an
s
f
o
r
m
er
s
(
B
E
R
T
)
an
d
it
s
v
ar
ian
ts
.
T
h
ese
s
tu
d
ies
o
v
er
v
iew
a
d
iv
er
s
e
s
et
o
f
u
s
e
ca
s
es
ac
r
o
s
s
s
ec
to
r
s
f
r
o
m
e
-
co
m
m
er
ce
(
T
o
k
o
p
ed
ia,
B
u
k
alap
ak
,
a
n
d
B
lib
li),
f
in
t
ec
h
(
DANA
an
d
OVO)
,
tr
a
n
s
p
o
r
tatio
n
an
d
life
s
ty
le
s
er
v
ices
(
Go
jek
an
d
T
r
av
elo
k
a)
to
ed
u
ca
tio
n
(
R
u
an
g
g
u
r
u
)
.
All
th
ese
p
latf
o
r
m
s
g
et
to
n
s
o
f
u
s
er
-
g
e
n
er
ated
f
ee
d
b
ac
k
lik
e
p
r
o
d
u
ct
r
ev
iews,
co
m
m
en
ts
o
n
s
o
cial
m
ed
ia
p
o
s
ts
an
d
co
m
p
lain
ts
;
th
er
ef
o
r
e,
s
en
tim
en
t
a
n
aly
s
is
b
ec
o
m
es
a
g
r
ea
t
ass
is
tan
t
in
g
ettin
g
in
s
ig
h
ts
a
b
o
u
t
u
s
er
s
atis
f
ac
tio
n
s
,
s
er
v
ic
e
g
ap
s
a
n
d
s
tr
ateg
ic
d
ec
is
io
n
m
ak
in
g
.
T
h
is
s
tu
d
y
f
o
cu
s
ed
o
n
ca
teg
o
r
izin
g
t
h
o
s
e
d
iv
er
s
e
s
tar
tu
p
a
r
ea
s
to
s
h
o
w
th
e
m
an
y
s
h
ap
es:
it
en
co
m
p
a
s
s
es
h
o
w
s
en
tim
en
t
an
aly
s
is
-
b
ased
r
esear
ch
ca
n
b
e
ap
p
lied
in
a
co
m
p
lex
m
u
lti
-
d
im
en
s
io
n
al
s
p
ac
e
f
o
r
r
ea
l
-
wo
r
ld
co
n
tex
ts
,
th
at
b
o
th
e
n
ab
les
its
ap
p
licab
le
p
o
ten
tial
an
d
ex
p
o
s
es
th
e
ty
p
es
o
f
c
h
allen
g
es
f
ac
e
d
w
h
en
d
ep
lo
y
in
g
s
u
ch
m
o
d
els
in
I
n
d
o
n
esia d
ig
ital e
co
n
o
m
y
.
2
.
3
.
M
a
chine
lea
rning
m
et
h
o
ds
ML
is
a
b
r
an
c
h
o
f
ar
tific
ial
i
n
tellig
en
ce
,
wh
ic
h
allo
ws
m
a
ch
in
es
to
lear
n
f
r
o
m
th
e
d
ata
an
d
m
ak
e
p
r
ed
ictio
n
s
with
o
u
t
b
ein
g
e
x
p
licitly
p
r
o
g
r
am
m
ed
.
I
n
s
en
t
im
en
t
an
aly
s
is
,
clas
s
ical
ML
m
o
d
els
(
lo
g
is
tic
r
eg
r
ess
io
n
[
3
]
,
n
aïv
e
B
ay
es
[
3
]
,
SVM
[
3
]
,
r
an
d
o
m
f
o
r
ests
[
3
]
,
an
d
e
x
tr
em
e
g
r
ad
ien
t
b
o
o
s
ti
n
g
(
XGBo
o
s
t
)
)
ar
e
tr
ain
ed
o
n
p
o
s
itiv
ely
o
r
n
eg
a
tiv
ely
lab
eled
te
x
t
to
p
r
ed
ict
wh
eth
er
a
n
ew
in
s
tan
ce
o
f
tex
t
is
p
o
s
itiv
e
o
r
n
eg
ativ
e
an
d
lab
el
it
ac
co
r
d
in
g
ly
.
Du
e
to
th
eir
s
im
p
le,
in
ter
p
r
etab
le
an
d
lo
w
n
u
m
er
ic
co
s
t,
th
ese
m
o
d
els
ar
e
b
o
th
wid
ely
u
s
ed
.
ML
m
eth
o
d
s
ar
e
u
s
ef
u
l
to
s
u
m
m
ar
ize
as
t
h
ey
p
r
o
v
id
e
g
o
o
d
p
er
f
o
r
m
a
n
c
e
an
d
co
m
p
ar
is
o
n
s
with
m
o
r
e
ad
v
a
n
ce
d
SLM
s
an
d
L
L
Ms.
Ad
d
itio
n
ally
,
th
ey
s
till
m
ak
e
s
en
s
e
f
o
r
s
tar
t
-
u
p
s
o
r
o
r
g
a
n
izatio
n
s
with
lim
ited
r
eso
u
r
ce
s
.
T
h
is
s
tu
d
y
co
v
er
s
ap
p
r
o
ac
h
es
o
f
ML
to
b
en
ch
m
ar
k
s
en
tim
en
t
an
aly
s
is
m
eth
o
d
s
an
d
th
ei
r
im
p
lem
en
tatio
n
in
t
h
e
co
n
te
x
t
o
f
I
n
d
o
n
esian
s
tar
tu
p
s
.
2
.
3
.
1
.
L
o
g
is
t
ic
re
g
re
s
s
io
n
L
o
g
is
tic
r
eg
r
ess
io
n
[
8
]
is
o
n
e
o
f
th
e
m
o
s
t
u
s
ed
alg
o
r
ith
m
s
f
o
r
b
in
ar
y
class
ificatio
n
an
d
it
is
wid
ely
ap
p
lied
in
s
en
tim
en
t
an
aly
s
is
to
d
eter
m
in
e
w
h
eth
er
a
tex
t
f
alls
in
to
p
o
s
itiv
e,
n
eu
tr
al
o
r
n
eg
ativ
e
ca
teg
o
r
ies.
Du
e
to
its
s
im
p
licity
an
d
ef
f
ec
tiv
en
ess
,
it
is
v
er
y
co
m
m
o
n
ly
u
s
ed
in
b
o
th
r
esea
r
ch
an
d
r
ea
l
-
wo
r
ld
ap
p
licatio
n
s
,
s
u
ch
as
th
e
T
r
ip
Ad
v
is
o
r
r
ev
iew
an
aly
s
is
[
9
]
.
Dan
y
al
et
a
l.
[
1
0
]
f
o
u
n
d
th
at
l
o
g
is
tic
r
eg
r
ess
io
n
is
b
etter
th
an
n
ai
v
e
B
ay
es in
an
a
ly
zin
g
1
0
4
,
2
1
5
m
o
v
ie
r
e
v
iews.
L
o
g
it is
th
e
f
u
n
ctio
n
th
at
u
n
d
e
r
lies
th
e
f
o
r
m
u
la
o
f
lo
g
is
tic
r
eg
r
ess
io
n
wh
ich
is
u
s
ed
a
s
an
a
p
p
r
o
ac
h
to
s
o
lv
e
b
in
ar
y
class
if
icatio
n
p
r
o
b
lem
.
T
h
e
f
u
n
d
am
en
tal
f
o
r
m
u
l
a
o
f
lo
g
it
f
u
n
ctio
n
[
1
1
]
is
s
h
o
wn
in
(
1
)
.
(
=
1
|
)
=
1
1
+
−
(
0
+
1
1
+
2
2
+
.
.
.
+
)
(
1
)
W
h
er
e
(
=
1
∣
)
is
th
e
p
r
o
b
ab
ilit
y
th
at
th
e
d
ep
en
d
e
n
t
v
ar
iab
le
eq
u
als
1
,
g
iv
en
th
e
in
d
e
p
en
d
e
n
t
v
ar
i
ab
le
,
0
is
th
e
in
ter
ce
p
t
(
b
ias
ter
m
)
,
1
,
2
,
…
,
ar
e
th
e
r
eg
r
ess
io
n
co
ef
f
icien
t
s
f
o
r
ea
ch
in
d
ep
en
d
e
n
t
v
ar
iab
l
e
1
,
2
,
…
,
,
an
d
is
th
e
b
ase
o
f
th
e
n
atu
r
al
lo
g
ar
ith
m
(
E
u
ler
’
s
n
u
m
b
er
,
a
p
p
r
o
x
im
ately
2
.
7
1
8
)
.
2
.
3
.
2
.
Su
pp
o
rt
v
ec
t
o
r
m
a
chi
ne
I
n
s
en
tim
en
t
a
n
aly
s
is
,
SVM
[
1
2
]
a
r
e
ef
f
icie
n
t
s
u
p
er
v
is
ed
le
ar
n
in
g
tec
h
n
iq
u
es
th
at
ca
teg
o
r
ize
tex
t
as
n
eu
tr
al,
n
eg
ativ
e,
o
r
p
o
s
itiv
e.
T
h
ey
wo
r
k
b
y
f
in
d
in
g
an
o
p
tim
al
h
y
p
er
p
lan
e
in
h
ig
h
-
d
im
en
s
io
n
al
s
p
ac
e
f
o
r
class
if
icatio
n
.
SVM
h
as
p
r
o
v
en
s
u
cc
ess
f
u
l
in
an
aly
zin
g
tw
ee
ts
,
h
an
d
lin
g
s
h
o
r
t
an
d
am
b
i
g
u
o
u
s
tex
t.
Stu
d
ies
s
h
o
w
SVM
[
1
3
]
o
f
te
n
o
u
tp
er
f
o
r
m
s
m
o
d
els
lik
e
lo
g
is
tic
r
eg
r
ess
io
n
,
n
aiv
e
B
ay
es,
a
n
d
r
an
d
o
m
f
o
r
est,
esp
ec
ially
in
task
s
lik
e
class
if
y
in
g
s
en
tim
en
ts
in
Un
ited
Stat
es
air
lin
e
X
(
f
o
r
m
e
r
ly
T
witter
)
d
ata.
SVM
is
a
s
u
p
er
v
is
ed
lear
n
in
g
tech
n
iq
u
e
th
at
is
ap
p
lied
to
task
s
in
v
o
lv
i
n
g
r
eg
r
e
s
s
io
n
an
d
class
if
icatio
n
.
Fin
d
in
g
th
e
b
est
h
y
p
er
p
lan
e
to
o
p
tim
ize
th
e
m
ar
g
in
b
etwe
en
two
class
es is
it
s
aim
.
i)
E
q
u
atio
n
o
f
th
e
d
ec
is
io
n
b
o
u
n
d
ar
y
(
h
y
p
er
p
lan
e
):
f
o
r
a
li
n
ea
r
SVM
[
1
4
]
,
th
e
d
ec
is
io
n
b
o
u
n
d
ar
y
is
d
ef
in
ed
as sh
o
wn
in
(
2
)
.
(
)
=
+
=
0
(
2
)
W
h
er
e
is
th
e
in
p
u
t
f
ea
tu
r
e
v
e
cto
r
,
is
th
e
weig
h
t
v
ec
to
r
(
co
e
f
f
icien
ts
)
,
an
d
is
th
e
b
ias
(
in
ter
ce
p
t)
.
A
p
o
in
t
b
elo
n
g
s
to
class
+1
if
(
)
>
0
an
d
class
-
1
if
(
)
<
0
.
ii)
Ma
r
g
in
m
ax
im
izatio
n
co
n
d
itio
n
:
th
e
SVM
[
1
5
]
o
p
tim
izes th
e
m
ar
g
in
b
y
s
o
lv
in
g
as sh
o
wn
i
n
(
3
)
.
=
1
2
,
‖
‖
2
(
3
)
S
u
b
ject
to
th
e
co
n
s
tr
ain
t a
s
s
h
o
wn
in
(
4
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
F
in
e
-
tu
n
in
g
la
n
g
u
a
g
e
mo
d
els
fo
r
s
o
cia
l sen
timen
t:
in
s
ig
h
ts
fr
o
m
s
ta
r
tu
p
u
s
er fee
d
b
a
ck
(
Li
-
Hu
a
Li
)
4435
(
+
)
≥
1
,
∀
(
4
)
W
h
er
e
is
th
e
class
lab
el
(
+1
o
r
-
1
)
a
n
d
ar
e
th
e
tr
ain
i
n
g
p
o
in
ts
.
2
.
3
.
3
.
Ra
nd
o
m
f
o
re
s
t
An
en
s
em
b
le
lear
n
in
g
tech
n
iq
u
e
ca
lled
r
an
d
o
m
f
o
r
est
[
1
6
]
is
u
s
ed
in
s
en
tim
en
t a
n
aly
s
is
to
ca
teg
o
r
ize
tex
t
as
n
eu
tr
al,
n
e
g
ativ
e,
o
r
p
o
s
itiv
e.
T
o
in
cr
ea
s
e
ac
cu
r
ac
y
an
d
d
ec
r
ea
s
e
o
v
er
f
itti
n
g
,
it
co
n
s
tr
u
cts
n
u
m
er
o
u
s
d
ec
is
io
n
tr
ee
s
a
n
d
u
s
e
m
ajo
r
ity
v
o
tin
g
.
M
u
n
s
h
i
et
a
l.
[
1
7
]
h
a
v
e
a
p
p
lied
it
to
s
en
tim
e
n
t
class
if
icatio
n
in
s
u
p
p
o
r
t
o
f
s
o
cial
r
is
k
m
an
ag
e
m
en
t,
wh
ile
W
an
g
[
1
8
]
h
ig
h
li
g
h
ts
its
ef
f
ec
tiv
en
ess
alo
n
g
s
id
e
SVM
an
d
n
eu
r
al
n
etwo
r
k
s
i
n
b
o
th
ac
ad
e
m
ic
an
d
r
ea
l
-
wo
r
ld
ap
p
licatio
n
s
.
i)
I
n
d
iv
id
u
al
d
ec
is
io
n
tr
ee
f
o
r
m
u
la
:
as
d
em
o
n
s
tr
ated
b
y
(
5
)
,
ev
er
y
r
an
d
o
m
f
o
r
est
[
1
9
]
d
ec
is
io
n
tr
ee
co
r
r
esp
o
n
d
s
to
a
r
ec
u
r
s
iv
e
s
p
litt
in
g
r
u
le.
=
∑
(1
-
)
(
5
)
W
h
er
e
is
th
e
Gin
i
im
p
u
r
ity
(
u
s
ed
f
o
r
class
if
icatio
n
)
,
is
th
e
p
r
o
p
o
r
tio
n
o
f
class
k
in
a
n
o
d
e,
an
d
th
e
s
p
lit
aim
s
to
m
in
im
ize
t
h
e
G
in
i
im
p
u
r
ity
.
Fo
r
r
eg
r
ess
io
n
,
d
ec
is
io
n
tr
ee
s
u
s
e
t
h
e
m
ea
n
s
q
u
ar
ed
e
r
r
o
r
(
MSE
)
as sh
o
wn
in
(
6
)
.
=
1
∑
(
−
̂
)
2
=
1
(
6
)
W
h
er
e
is
th
e
co
m
p
u
te
d
o
u
t
p
u
t a
n
d
̂
is
th
e
av
er
ag
e
o
f
tar
g
et
v
alu
es in
th
e
n
o
d
e.
2
.
3
.
4
.
E
x
t
re
m
e
g
ra
dient
bo
o
s
t
ing
C
h
en
an
d
Gu
estrin
[
2
0
]
cr
ea
ted
th
e
q
u
ick
a
n
d
ef
f
ec
tiv
e
g
r
ad
ien
t
b
o
o
s
tin
g
tech
n
i
q
u
e
k
n
o
wn
as
XGBo
o
s
t
in
2
0
1
6
.
I
t
is
f
r
e
q
u
en
tly
u
tili
ze
d
i
n
ch
allen
g
es
an
d
r
ea
l
-
wo
r
ld
p
r
o
jects
b
ec
a
u
s
e
o
f
its
ex
ce
llen
t
p
er
f
o
r
m
an
ce
.
I
n
s
en
tim
en
t
a
n
aly
s
is
,
s
tu
d
ies
h
av
e
s
h
o
wn
im
p
r
o
v
e
d
r
esu
lts
b
y
co
m
b
in
i
n
g
XGBo
o
s
t
with
r
ev
iewe
r
r
eliab
ilit
y
an
d
th
e
wh
ale
o
p
tim
izatio
n
alg
o
r
ith
m
[
2
1
]
,
en
h
a
n
cin
g
p
r
o
d
u
ct
r
ec
o
m
m
e
n
d
atio
n
s
y
s
tem
s
.
i)
Pre
d
ictio
n
f
o
r
m
u
la
:
in
XGBo
o
s
t,
th
e
m
o
d
el
co
n
s
is
ts
o
f
d
ec
is
io
n
tr
ee
s
,
an
d
th
e
f
in
al
p
r
ed
i
ctio
n
[
2
2
]
is
th
e
s
u
m
o
f
all
tr
ee
o
u
tp
u
ts
as sh
o
wn
in
(
7
)
.
̂
=
∑
=
1
(
)
(
7
)
W
h
er
e
̂
is
th
e
p
r
e
d
icted
v
alu
e
f
o
r
i
n
s
tan
ce
I
,
(
)
is
th
e
o
u
tp
u
t
o
f
th
e
k
-
th
d
ec
is
io
n
tr
ee
,
an
d
is
th
e
to
tal
n
u
m
b
er
o
f
tr
ee
s
.
E
ac
h
tr
e
e
lear
n
s
f
r
o
m
th
e
r
esid
u
als (
er
r
o
r
s
)
o
f
th
e
p
r
e
v
io
u
s
tr
ee
s
.
ii)
L
o
s
s
f
u
n
ctio
n
(
o
b
jectiv
e
f
u
n
ctio
n
)
:
XGBo
o
s
t
[
2
3
]
,
[
2
4
]
m
in
i
m
izes
a
r
eg
u
lar
ize
d
lo
s
s
f
u
n
ct
io
n
as
s
h
o
w
n
in
(
8
)
.
=
∑
(
,
̂
)
=
1
+
∑
(
)
=
1
(
8
)
W
h
er
e
(
,
̂
)
is
th
e
lo
s
s
f
u
n
ctio
n
(
e.
g
.
,
s
q
u
ar
e
d
er
r
o
r
f
o
r
r
e
g
r
ess
io
n
,
lo
g
lo
s
s
f
o
r
class
if
icatio
n
)
,
an
d
(
)
is
th
e
r
eg
u
lar
izatio
n
te
r
m
to
p
r
ev
en
t o
v
er
f
itti
n
g
as sh
o
wn
in
(
9
)
.
(
)
=
+
1
2
∑
2
(
9
)
W
h
er
e
is
th
e
n
u
m
b
er
o
f
leav
es
in
th
e
tr
ee
,
ar
e
th
e
leaf
we
ig
h
ts
,
an
d
an
d
ar
e
h
y
p
e
r
p
ar
a
m
eter
s
f
o
r
r
e
g
u
lar
izatio
n
.
2
.
4
.
L
a
rg
e
la
ng
ua
g
e
m
o
del
L
L
Ms
[
2
5
]
ar
e
s
o
p
h
is
ticated
d
ee
p
lear
n
in
g
m
o
d
els
th
at
h
av
e
b
ee
n
tr
ain
ed
o
n
en
o
r
m
o
u
s
te
x
t
co
r
p
o
r
a
to
co
m
p
r
e
h
en
d
a
n
d
p
r
o
d
u
ce
la
n
g
u
ag
e
t
h
at
is
lik
e
th
at
o
f
h
u
m
an
s
.
L
L
Ms
lik
e
B
E
R
T
[
2
6
]
(
lau
n
ch
ed
b
y
Go
o
g
le
in
2
0
1
8
)
,
GPT
(
f
ir
s
t
p
u
b
lis
h
ed
b
y
Op
en
AI
in
2
0
1
8
)
,
an
d
I
n
d
o
n
esian
b
id
i
r
ec
tio
n
al
en
co
d
er
r
ep
r
esen
tatio
n
s
f
r
o
m
tr
an
s
f
o
r
m
e
r
s
(
I
n
d
o
B
E
R
T
)
[
2
7
]
,
w
h
ich
ar
e
b
ased
o
n
tr
an
s
f
o
r
m
er
ar
ch
itectu
r
es,
a
r
e
e
x
ce
llen
t
at
ca
p
tu
r
in
g
co
n
tex
tu
al
r
elatio
n
s
h
ip
s
b
etw
ee
n
wo
r
d
s
,
allo
win
g
th
e
m
to
ca
r
r
y
o
u
t
ch
allen
g
in
g
NL
P
t
ask
s
lik
e
s
en
tim
en
t
an
aly
s
is
,
tex
t
class
if
icatio
n
,
a
n
d
q
u
esti
o
n
a
n
s
wer
in
g
.
L
L
Ms
h
av
e
b
ee
n
ar
o
u
n
d
s
in
ce
a
b
o
u
t
2
0
1
8
,
w
h
en
t
h
ey
s
h
o
wed
an
ex
tr
ao
r
d
in
ar
y
,
p
r
e
-
tr
ain
ed
g
en
er
aliza
tio
n
ab
ilit
y
f
r
o
m
m
ass
iv
e
n
atu
r
al
lan
g
u
ag
e
d
ata
at
h
i
g
h
ac
cu
r
ac
y
to
m
an
y
d
if
f
er
e
n
t
d
o
m
ain
s
an
d
lan
g
u
ag
es.
L
L
Ms
p
er
f
o
r
m
b
etter
th
an
tr
a
d
itio
n
al
m
o
d
els
in
s
en
tim
en
t
an
aly
s
is
b
ec
au
s
e
t
h
ey
ca
n
d
o
m
o
r
e
f
in
e
-
g
r
ain
e
d
tex
t
u
n
d
er
s
tan
d
in
g
,
wh
ich
m
ak
es
th
em
a
g
o
o
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
5
,
Octo
b
er
2
0
2
6
:
4
4
3
2
-
4
4
4
6
4436
ca
n
d
id
ate
f
o
r
e
x
tr
ac
tin
g
in
s
ig
h
ts
f
r
o
m
u
s
er
f
ee
d
b
ac
k
,
esp
ec
ia
lly
wh
en
th
er
e
is
in
f
o
r
m
al
o
r
m
u
ltil
in
g
u
al
co
r
p
u
s
as wo
u
ld
b
e
th
e
ca
s
e
with
m
a
n
y
s
tar
tu
p
co
m
p
an
ies.
2
.
4
.
1
.
B
idi
re
ct
io
na
l e
nco
der
re
presenta
t
io
ns
f
ro
m
t
ra
ns
f
o
rm
er
s
A
n
ew
lan
g
u
ag
e
r
ep
r
esen
tati
o
n
ap
p
r
o
ac
h
b
y
Go
o
g
le
r
ese
ar
ch
er
s
ca
lled
B
E
R
T
was
in
t
r
o
d
u
ce
d
in
2
0
1
8
[
2
8
]
.
Su
ch
a
p
ar
a
d
ig
m
h
elp
s
u
s
to
u
n
d
er
s
tan
d
th
e
u
s
es
o
f
wo
r
d
s
b
ased
o
n
th
eir
co
n
tex
t
in
a
s
en
ten
ce
.
T
h
is
ca
p
ab
ilit
y
h
as b
en
e
f
its
th
at
h
av
e
r
ev
o
lu
tio
n
ize
d
NL
P.
Ar
ch
itectu
r
e
an
d
tr
ain
in
g
;
B
E
R
T
[
2
9
]
is
a
tr
an
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
el
t
h
at
o
n
l
y
m
ak
es
u
s
e
o
f
th
e
en
co
d
er
.
I
t
cr
ea
tes
em
b
ed
d
in
g
s
b
y
in
teg
r
atin
g
to
k
e
n
,
lo
ca
tio
n
,
an
d
s
eg
m
en
t
in
f
o
r
m
ati
o
n
,
to
k
e
n
izes
in
p
u
t
in
to
s
u
b
wo
r
d
s
,
an
d
ad
d
s
s
p
e
cial
to
k
en
s
lik
e
[
C
L
S]
an
d
[
SEP]
.
Fo
r
B
E
R
T
to
lear
n
r
ic
h
an
d
b
id
ir
ec
tio
n
al
co
n
tex
t,
th
e
m
o
d
el
u
s
es
1
2
t
r
an
s
f
o
r
m
er
lay
er
s
with
f
ee
d
-
f
o
r
war
d
n
etwo
r
k
s
an
d
m
u
lti
-
h
ea
d
s
elf
-
atten
tio
n
.
Fo
r
class
if
icatio
n
o
r
o
th
er
NL
P
task
s
,
th
e
f
in
al
o
u
tp
u
t
is
f
ed
in
to
a
task
-
s
p
ec
if
ic
lay
er
.
B
E
R
T
p
er
f
o
r
m
s
well
in
a
v
ar
iety
o
f
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
ap
p
licat
io
n
s
th
an
k
s
to
its
d
ee
p
ar
ch
itectu
r
e
an
d
b
id
ir
ec
ti
o
n
al
n
atu
r
e
as
s
ee
n
in
Fig
u
r
e
1
.
Fig
u
r
e
1
.
B
E
R
T
ar
ch
itectu
r
e
T
h
er
e
ar
e
two
m
ain
task
s
in
v
o
lv
ed
in
B
E
R
T
tr
ain
in
g
:
i)
Ma
s
k
ed
lan
g
u
ag
e
m
o
d
elin
g
(
ML
M)
:
th
e
p
r
o
ce
s
s
in
v
o
l
v
es
r
an
d
o
m
ly
m
ask
in
g
s
o
m
e
w
o
r
d
s
in
a
s
en
ten
ce
s
o
th
at
th
e
m
o
d
el
m
a
y
lear
n
t
o
an
ticip
ate
th
e
m
ask
ed
w
o
r
d
s
b
ased
o
n
th
eir
co
n
tex
t.
I
n
th
i
s
in
s
tan
ce
,
th
e
m
o
d
el
is
u
r
g
ed
to
g
ain
a
th
o
r
o
u
g
h
c
o
m
p
r
e
h
en
s
io
n
o
f
th
e
wo
r
d
r
elatio
n
s
h
ip
s
an
d
la
n
g
u
ag
e
co
n
tex
t,
as
in
d
icate
d
b
y
(
1
0
)
.
=
−
∑
(
|
)
∈
(
1
0
)
W
h
er
e
is
th
e
s
et
o
f
m
ask
e
d
to
k
en
s
,
is
th
e
co
r
r
ec
t
to
k
en
,
is
th
e
in
p
u
t
with
s
o
m
e
to
k
e
n
s
r
ep
lace
d
b
y
[
MA
SK]
,
an
d
is
th
e
lo
s
s
f
u
n
ctio
n
f
o
r
MLM
.
E
x
am
p
le:
in
p
u
t
:
“
T
h
e
ca
t
s
at
o
n
th
e
[
MA
SK]
”
an
d
m
o
d
el
lear
n
s
to
p
r
ed
ict
“
m
at
”
.
ii)
Nex
t
s
en
ten
ce
p
r
ed
ictio
n
(
NS
P):
th
is
task
d
eter
m
in
es
wh
eth
er
two
s
en
ten
ce
s
f
o
llo
w
ea
c
h
o
th
er
lo
g
ically
.
B
y
p
r
ac
ticin
g
o
n
t
h
is
task
,
B
E
R
T
lear
n
s
th
e
r
elatio
n
s
h
ip
b
et
wee
n
s
en
ten
ce
s
,
wh
ich
is
v
er
y
im
p
o
r
tan
t
in
task
s
s
u
ch
as q
u
esti
o
n
an
s
wer
in
g
an
d
n
atu
r
al
la
n
g
u
a
g
e
in
f
er
e
n
ce
as sh
o
wn
in
(
1
1
)
.
=
−
∑
[
(
|
1
,
2
)
+
(
1
−
)
(
|
1
,
2
)
]
(
1
1
)
W
h
er
e
1
an
d
2
ar
e
two
co
n
s
ec
u
tiv
e
tex
t
s
eg
m
en
ts
,
(
|
1
,
2
)
th
e
p
r
ed
icted
p
r
o
b
a
b
ilit
y
th
at
2
is
th
e
n
ex
t
s
en
ten
ce
a
f
ter
1
,
(
|
1
,
2
)
th
e
p
r
e
d
icted
p
r
o
b
ab
ilit
y
t
h
at
2
is
n
o
t
th
e
n
e
x
t
s
en
ten
ce
af
te
r
1
,
=
1
if
2
is
th
e
n
ex
t
s
en
ten
ce
,
o
th
e
r
wis
e
=
0
,
an
d
th
e
lo
s
s
f
u
n
ctio
n
f
o
r
t
h
e
NSP
task
.
I
t
is
a
b
in
ar
y
cr
o
s
s
-
en
tr
o
p
y
lo
s
s
th
at
m
ea
s
u
r
es
h
o
w
well
th
e
m
o
d
el
p
r
ed
icts
wh
eth
er
2
f
o
llo
w
s
1
o
r
n
o
t
.
E
x
am
p
le:
in
p
u
t
:
(
“
I
lo
v
e
d
ee
p
lear
n
in
g
.
”
,
“
I
t
is
v
er
y
p
o
wer
f
u
l.
”
)
→
L
a
b
el:
n
e
x
t
an
d
in
p
u
t
:
(
“
I
l
o
v
e
d
ee
p
lear
n
in
g
.
”
,
“
D
o
g
s
ar
e
f
r
ien
d
ly
.
”
)
→
L
ab
el:
No
t N
ex
t
.
iii)
B
E
R
T
f
in
e
tu
n
in
g
f
o
r
m
u
la:
Fo
r
d
o
wn
s
tr
ea
m
task
s
,
a
class
if
icatio
n
h
ea
d
is
ad
d
ed
as sh
o
wn
i
n
(
1
2
)
.
(
|
)
=
(
ℎ
[
]
+
)
(
1
2
)
W
h
er
e
,
ℎ
[
]
is
th
e
r
ep
r
esen
tatio
n
o
f
th
e
[
]
to
k
en
,
an
d
an
d
ar
e
tr
ain
ab
le
p
ar
am
eter
s
.
2
.
4
.
2
.
I
nd
o
nes
ia
n
B
E
R
T
I
n
d
o
B
E
R
T
[
3
0
]
is
a
tr
an
s
f
o
r
m
er
-
b
ased
la
n
g
u
a
g
e
m
o
d
el
t
ailo
r
ed
s
p
ec
if
ically
f
o
r
th
e
I
n
d
o
n
esian
lan
g
u
ag
e,
in
s
p
ir
ed
b
y
B
E
R
T
.
I
t
was
d
ev
elo
p
e
d
to
ad
d
r
ess
th
e
u
n
d
e
r
r
ep
r
esen
tatio
n
o
f
I
n
d
o
n
esian
in
NL
P
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
F
in
e
-
tu
n
in
g
la
n
g
u
a
g
e
mo
d
els
fo
r
s
o
cia
l sen
timen
t:
in
s
ig
h
ts
fr
o
m
s
ta
r
tu
p
u
s
er fee
d
b
a
ck
(
Li
-
Hu
a
Li
)
4437
r
esear
ch
,
d
esp
ite
th
e
lan
g
u
ag
e
b
ein
g
s
p
o
k
e
n
b
y
o
v
er
2
0
0
m
i
llio
n
p
eo
p
le.
I
n
d
o
B
E
R
T
was
t
r
ain
ed
o
n
I
n
d
o
4
B
,
a
lar
g
e
-
s
ca
le
co
r
p
u
s
co
n
tain
i
n
g
4
.
3
b
illi
o
n
to
k
e
n
s
co
llected
f
r
o
m
d
iv
er
s
e
s
o
u
r
ce
s
,
in
clu
d
in
g
n
ews
ar
ticles,
s
o
cial
m
ed
ia
co
n
ten
t,
I
n
d
o
n
es
ian
W
ik
ip
ed
ia,
an
d
v
ar
io
u
s
I
n
d
o
n
esian
web
p
a
g
es.
T
h
is
m
o
d
el
was
p
r
o
p
o
s
ed
with
th
e
I
n
d
o
L
E
M
b
e
n
ch
m
ar
k
at
h
ttp
s
://g
ith
u
b
.
co
m
/in
d
o
lem
/in
d
o
lem
,
a
s
et
o
f
ev
alu
atio
n
t
ask
s
f
o
r
s
ev
en
co
r
e
NL
P
f
r
o
m
7
task
s
:
p
ar
t
-
of
-
s
p
ee
ch
(
POS)
tag
g
in
g
[
3
1
]
,
n
a
m
ed
en
tity
r
ec
o
g
n
itio
n
(
NE
R
)
[
3
1
]
,
d
ep
e
n
d
en
cy
p
ar
s
in
g
[
3
1
]
,
co
n
s
titu
en
cy
p
ar
s
in
g
[
3
1
]
,
s
em
a
n
tic
r
o
le
lab
eli
n
g
(
SR
L
)
,
an
d
n
atu
r
al
lan
g
u
a
g
e
in
f
er
en
ce
(
NL
I
)
d
o
cu
m
e
n
t
class
if
icati
o
n
[
3
1
]
.
T
h
ese
task
s
ch
a
r
ac
ter
ize
a
m
o
d
el
’
s
s
y
n
tactic
k
n
o
win
g
(
POS
tag
g
in
g
a
n
d
p
ar
s
in
g
)
an
d
s
em
a
n
tic
th
in
k
i
n
g
(
SR
L
an
d
NL
I
)
,
y
ield
i
n
g
a
co
m
p
lete
p
o
r
tr
ait
o
f
lan
g
u
ag
e
u
n
d
er
s
tan
d
in
g
.
I
n
d
o
B
E
R
T
y
ield
ed
th
e
o
v
e
r
all
b
est
p
er
f
o
r
m
an
ce
o
n
m
o
s
t
o
f
th
ese
task
s
ac
h
iev
in
g
s
tate
-
of
-
th
e
-
ar
t
r
esu
lts
f
o
r
b
o
th
m
ea
s
u
r
i
n
g
s
y
n
tactic
s
tr
u
c
tu
r
e
an
d
s
em
a
n
tic
n
u
an
ce
s
o
f
t
h
e
I
n
d
o
n
esian
lan
g
u
ag
e.
2
.
5
.
S
m
a
ll
la
ng
ua
g
e
m
o
del
R
esear
ch
er
s
d
ev
elo
p
co
m
p
ac
t
v
er
s
io
n
s
o
f
LLMs
to
ad
d
r
ess
NL
P
task
s
(
SLM
s
)
[
3
2
]
,
co
n
s
u
m
in
g
less
co
m
p
u
tin
g
p
o
wer
.
E
v
en
t
h
o
u
g
h
SLM
s
[
3
3
]
h
a
v
e
f
ewe
r
p
ar
a
m
eter
s
co
m
p
ar
e
d
to
lar
g
e
m
o
d
els
s
u
ch
as
GPT
-
4
o
r
B
E
R
T
-
lar
g
e,
th
ey
m
ain
tain
r
esp
ec
tab
le
p
er
f
o
r
m
a
n
ce
o
n
tex
t
clas
s
if
icatio
n
,
s
u
m
m
ar
izatio
n
an
d
q
u
esti
o
n
an
s
wer
in
g
task
s
.
T
h
eir
co
m
p
ac
tn
ess
allo
ws
f
o
r
f
aster
in
f
er
en
ce
,
d
ec
r
ea
s
ed
m
em
o
r
y
co
n
s
u
m
p
tio
n
,
an
d
d
ep
lo
y
m
e
n
t
in
lo
w
-
r
eso
u
r
ce
c
o
n
tex
ts
o
r
ed
g
e
d
e
v
ices.
W
h
ile
SLM
s
u
s
u
ally
d
o
n
o
t
r
ea
c
h
c
o
m
p
ar
ab
le
lev
els
o
f
u
n
d
er
s
tan
d
i
n
g
as
lar
g
er
m
o
d
el
s
,
th
ey
ar
e
tr
ain
ed
o
n
a
s
p
ec
if
i
c
task
–
with
in
a
d
o
m
ain
with
h
ig
h
ef
f
icien
cy
an
d
ac
cu
r
ac
y
f
o
r
tar
g
eted
task
s
.
2
.
5
.
1
.
Dis
t
il
led
B
E
RT
B
E
R
T
was
d
is
till
ed
in
to
a
s
m
all
-
B
E
R
T
with
6
0
%
less
p
ar
am
eter
s
an
d
ar
o
u
n
d
9
7
%
o
f
B
E
R
T
’
s
p
er
f
o
r
m
an
ce
,
k
n
o
wn
as
d
is
till
ed
b
id
ir
ec
tio
n
al
e
n
co
d
e
r
r
ep
r
e
s
en
tatio
n
s
f
r
o
m
tr
an
s
f
o
r
m
er
s
(
Dis
til
B
E
R
T
)
[
3
4
]
.
I
t
p
er
f
o
r
m
s
well
with
s
en
tim
e
n
t
an
aly
s
is
o
n
Hu
g
e
s
o
cial
m
e
d
ia
d
ata
an
d
is
b
est
in
lo
w
-
r
es
o
u
r
ce
en
v
ir
o
n
m
e
n
t.
T
r
ain
ed
b
y
m
im
ick
in
g
a
lar
g
e
r
B
E
R
T
m
o
d
el
[
3
5
]
,
Dis
tilB
E
R
T
h
as
s
ix
tr
an
s
f
o
r
m
er
lay
er
s
as
s
ee
n
in
Fig
u
r
e
2
,
h
alf
o
f
B
E
R
T
-
b
ase,
u
s
in
g
s
el
f
-
atten
tio
n
an
d
f
ee
d
-
f
o
r
war
d
n
etwo
r
k
s
.
I
t
s
im
p
lifie
s
ar
ch
it
ec
tu
r
e
b
y
r
em
o
v
in
g
to
k
en
ty
p
e
em
b
ed
d
in
g
s
an
d
NSP
,
o
f
f
er
in
g
a
b
alan
ce
o
f
s
p
ee
d
,
s
im
p
licity
,
an
d
ac
c
u
r
ac
y
f
o
r
p
r
ac
tical
NL
P a
p
p
licatio
n
s
.
Fig
u
r
e
2
.
Dis
tilB
E
R
T
ar
ch
itec
tu
r
e
T
r
an
s
f
o
r
m
e
r
e
n
co
d
er
f
o
r
m
u
la
;
lik
e
B
E
R
T
an
d
Dis
tilB
E
R
T
[
2
7
]
is
b
ased
o
n
th
e
tr
an
s
f
o
r
m
er
en
c
o
d
er
ar
ch
itectu
r
e.
E
ac
h
en
co
d
er
lay
er
co
n
s
is
ts
o
f
s
elf
-
atten
tio
n
as
in
(
1
3
)
an
d
f
ee
d
f
o
r
war
d
la
y
er
s
as in
(
1
4
)
.
i)
Self
-
atten
tio
n
m
ec
h
an
is
m
(
s
ca
led
d
o
t
-
p
r
o
d
u
ct
atten
tio
n
)
(
,
,
)
=
(
√
)
(
1
3
)
W
h
er
e
,
,
ar
e
th
e
q
u
er
y
,
k
ey
,
an
d
v
alu
e
m
atr
ices
f
r
o
m
th
e
in
p
u
t
em
b
ed
d
in
g
s
,
is
th
e
d
im
en
s
io
n
ality
o
f
k
ey
s
(
s
ca
lin
g
f
ac
to
r
)
,
s
o
f
tm
a
x
en
s
u
r
es
th
at
th
e
atten
tio
n
s
co
r
es
s
u
m
to
1
,
a
n
d
e
ac
h
to
k
en
atten
d
s
to
all
o
th
er
to
k
e
n
s
in
th
e
in
p
u
t seq
u
en
ce
.
ii)
Feed
f
o
r
war
d
n
etwo
r
k
:
a
f
ter
att
en
tio
n
,
a
f
u
lly
co
n
n
ec
ted
f
ee
d
f
o
r
war
d
n
etwo
r
k
is
ap
p
lied
,
as
in
(
1
4
)
.
(
)
=
(
0
,
1
+
1
)
2
+
2
(
1
4
)
W
h
er
e
1
,
2
an
d
1
,
2
ar
e
lea
r
n
ab
le
weig
h
ts
an
d
b
iases
,
an
d
m
ax
(
0
,
⋅
)
is
th
e
r
ec
tifie
d
lin
ea
r
u
n
it
(
R
eL
U
)
ac
tiv
atio
n
f
u
n
ctio
n
.
iii)
Kn
o
wled
g
e
d
is
till
atio
n
lo
s
s
:
Dis
til
B
E
R
T
is
tr
ain
ed
u
s
in
g
a
d
is
till
atio
n
lo
s
s
,
wh
ich
co
m
b
in
e
s
:
−
C
r
o
s
s
-
en
tr
o
p
y
lo
s
s
(
)
f
o
r
lab
el
p
r
ed
ictio
n
as sh
o
wn
in
(
1
5
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
5
,
Octo
b
er
2
0
2
6
:
4
4
3
2
-
4
4
4
6
4438
=
−
∑
(
̂
)
(
1
5
)
W
h
er
e
is
th
e
tr
u
e
lab
el
an
d
̂
is
th
e
p
r
ed
icted
p
r
o
b
ab
ilit
y
.
−
Dis
till
a
tio
n
lo
s
s
(
)
f
o
r
s
o
f
t ta
r
g
et
s
as sh
o
wn
in
(
1
6
)
.
=
2
∑
ℎ
(
ℎ
)
(
1
6
)
W
h
er
e
,
is
th
e
tem
p
er
atu
r
e
(
s
ca
lin
g
f
ac
to
r
)
,
an
d
ℎ
an
d
ar
e
p
r
o
b
ab
ilit
y
d
is
tr
ib
u
tio
n
s
f
r
o
m
B
E
R
T
an
d
Dis
tilB
E
R
T
.
T
h
e
f
in
al
co
m
b
in
ed
lo
s
s
is
as s
h
o
wn
in
(
1
7
)
.
=
+
(
1
−
)
(
1
7
)
W
h
er
e
b
alan
ce
s
h
ar
d
an
d
s
o
f
t la
b
els.
Dis
til
B
E
R
T
a
ls
o
u
s
e
s
tr
an
s
f
o
r
m
er
s
lik
e
B
E
R
T
th
er
ef
o
r
e
it
h
as
th
e
s
am
e
f
o
r
m
u
la
r
eg
ar
d
in
g
s
elf
atten
tio
n
,
f
ee
d
f
o
r
war
d
n
etwo
r
k
,
a
n
d
class
if
icatio
n
lo
s
s.
3.
M
E
T
H
O
D
T
h
is
s
ec
tio
n
o
u
tlin
es
th
e
p
r
o
ce
s
s
f
o
r
c
r
ea
tin
g
s
en
tim
en
t
an
aly
s
is
m
o
d
els
b
ased
o
n
s
t
ar
tu
p
u
s
er
r
ev
iews
an
d
r
atin
g
s
u
s
in
g
ML
,
SLM
,
an
d
L
L
M.
T
h
e
o
v
e
r
all
wo
r
k
f
lo
w
o
f
th
is
s
tu
d
y
is
s
tr
u
ctu
r
ed
in
to
s
ev
er
al
s
y
s
tem
atic
p
h
ases
.
Fig
u
r
e
3
s
h
o
ws
h
o
w
th
ese
p
h
ase
s
a
r
e
o
r
g
an
ized
f
o
r
d
ata
co
llectio
n
,
p
r
ep
r
o
ce
s
s
in
g
,
tr
ain
in
g
,
an
d
ev
alu
ati
o
n
.
Fig
u
r
e
3
.
R
esear
ch
wo
r
k
f
lo
w
3
.
1
.
Da
t
a
c
o
llect
io
n
Data
s
et
i
s
co
llected
f
r
o
m
Kag
g
le
at
h
ttp
s
://www.
k
ag
g
le.
co
m
/d
atasets
/r
ez
k
y
y
ay
an
g
/r
ev
iews
-
of
-
in
d
o
n
esian
-
a
p
p
-
s
tar
tu
p
s
-
on
-
p
l
ay
s
to
r
e
an
d
th
e
licen
s
e
is
C
C
0
:
p
u
b
lic
d
o
m
ai
n
m
ea
n
s
n
o
c
o
p
y
r
ig
h
t.
T
h
e
lis
ted
I
n
d
o
n
esian
s
tar
tu
p
s
ar
e
n
o
tab
l
e
f
o
r
th
eir
d
iv
er
s
e
s
er
v
ices
an
d
s
ig
n
if
ican
t
u
s
er
en
g
ag
em
en
t
as
s
ee
n
in
T
ab
le
1
.
T
h
is
d
ataset
in
clu
d
es r
ev
iews f
r
o
m
p
o
p
u
la
r
I
n
d
o
n
esian
a
p
p
s
as seen
in
T
ab
le
2
.
T
ab
le
1
.
Su
m
m
a
r
y
o
f
s
tar
tu
p
s
ap
p
r
ev
iews
an
d
d
o
wn
l
o
ad
s
S
t
a
r
t
u
p
s
M
a
i
n
ser
v
i
c
e
s
R
e
v
i
e
w
s
D
o
w
n
l
o
a
d
s
G
o
j
e
k
R
i
d
e
-
h
a
i
l
i
n
g
,
f
o
o
d
d
e
l
i
v
e
r
y
,
a
n
d
o
n
l
i
n
e
p
a
y
me
n
t
s
1
1
6
,
0
0
0
+
1
0
0
,
0
0
0
,
0
0
0
+
To
k
o
p
e
d
i
a
O
n
l
i
n
e
mar
k
e
t
p
l
a
c
e
a
n
d
su
p
p
o
r
t
s l
o
c
a
l
e
n
t
r
e
p
r
e
n
e
u
r
s
7
1
,
0
0
0
+
1
0
0
,
0
0
0
,
0
0
0
+
D
a
n
a
D
i
g
i
t
a
l
w
a
l
l
e
t
,
p
a
y
me
n
t
s
,
a
n
d
m
o
n
e
y
t
r
a
n
sf
e
r
s
1
0
6
,
0
0
0
+
1
0
0
,
0
0
0
,
0
0
0
+
Tr
a
v
e
l
o
k
a
Tr
a
v
e
l
b
o
o
k
i
n
g
(
f
l
i
g
h
t
s,
h
o
t
e
l
s,
a
n
d
e
n
t
e
r
t
a
i
n
me
n
t
)
1
4
5
,
0
0
0
+
5
0
,
0
0
0
,
0
0
0
+
B
u
k
a
l
a
p
a
k
D
i
g
i
t
a
l
m
a
r
k
e
t
p
l
a
c
e
a
n
d
s
u
p
p
o
r
t
s
m
i
c
r
o
,
s
mal
l
,
a
n
d
me
d
i
u
m
e
n
t
e
r
p
r
i
s
e
s
(
M
S
M
Es
)
9
0
,
0
0
0
+
5
0
,
0
0
0
,
0
0
0
+
OVO
D
i
g
i
t
a
l
w
a
l
l
e
t
,
p
a
y
me
n
t
s
,
t
r
a
n
sf
e
r
s,
a
n
d
i
n
v
e
s
t
me
n
t
s
8
3
,
0
0
0
+
5
0
,
0
0
0
,
0
0
0
+
B
l
i
b
l
i
E
-
c
o
mm
e
r
c
e
p
l
a
t
f
o
r
m
1
1
6
,
0
0
0
+
1
0
,
0
0
0
,
0
0
0
+
R
u
a
n
g
g
u
r
u
Ed
u
c
a
t
i
o
n
a
l
p
l
a
t
f
o
r
m
,
o
n
l
i
n
e
l
e
a
r
n
i
n
g
,
a
n
d
l
i
v
e
c
l
a
ss
e
s
1
1
1
,
0
0
0
+
1
0
,
0
0
0
,
0
0
0
+
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
F
in
e
-
tu
n
in
g
la
n
g
u
a
g
e
mo
d
els
fo
r
s
o
cia
l sen
timen
t:
in
s
ig
h
ts
fr
o
m
s
ta
r
tu
p
u
s
er fee
d
b
a
ck
(
Li
-
Hu
a
Li
)
4439
T
ab
le
2
.
E
x
am
p
le
d
ataset
in
d
o
n
esia star
tu
p
s
Ex
a
m
p
l
e
S
c
o
r
e
O
r
i
g
i
n
a
l
c
o
n
t
e
n
t
En
g
l
i
sh
t
r
a
n
sl
a
t
i
o
n
Ex
1
5
“
Ap
l
i
k
a
si
y
a
n
g
s
a
n
g
a
t
m
e
m
b
a
n
t
u
,
p
u
a
s
b
a
n
g
e
t
d
e
n
g
a
n
r
e
sp
o
n
c
e
p
a
t
n
y
a
.
S
e
m
o
g
a
t
a
m
b
a
h
s
u
k
ses
d
a
n
b
a
g
u
s
t
e
r
u
s
u
n
t
u
k
t
ra
v
e
l
o
k
a
.
”
“
A
v
e
r
y
h
e
l
p
f
u
l
a
p
p
,
v
e
r
y
sa
t
i
sf
i
e
d
w
i
t
h
t
h
e
q
u
i
c
k
r
e
sp
o
n
s
e
.
I
w
i
sh
Tr
a
v
e
l
o
k
a
c
o
n
t
i
n
u
e
d
s
u
c
c
e
ss
a
n
d
su
c
c
e
ss.
”
Ex
2
4
“
k
n
p
b
i
n
t
a
n
g
4
?
se
b
e
n
a
r
n
y
a
si
s
t
e
m
n
y
a
u
d
h
b
a
g
u
s
,
a
n
d
sem
o
g
a
s
u
a
t
u
h
r
n
n
t
i
.
.
.
.
d
i
s
i
st
e
m
T
r
a
v
e
l
o
k
a
m
a
m
p
u
“
m
e
m
b
a
c
a
”
k
e
b
i
a
sa
a
n
u
ser
n
y
a
(
v
i
a
m
e
d
s
o
s)
,
seh
i
n
g
g
a
s
e
c
a
r
a
o
t
o
m
a
t
i
s
m
e
m
b
e
r
i
k
a
n
“
p
ro
m
o
/
d
i
s
k
o
n
”
s
e
s
u
a
i
d
g
m
i
n
a
t
/
k
e
b
i
a
sa
a
n
u
ser
n
y
a
.
T
e
ri
m
a
k
a
si
h
T
ra
v
e
l
o
k
a
,
h
o
p
e
f
o
r
l
o
n
g
l
i
f
e
a
n
d
t
h
e
b
e
s
t
t
h
i
n
g
e
v
e
r
ð
Ÿ
™
ð
Ÿ
’
ž”
“
W
h
y
4
s
t
a
r
s
?
A
c
t
u
a
l
l
y
,
t
h
e
s
y
st
e
m
i
s
a
l
r
e
a
d
y
g
o
o
d
,
a
n
d
h
o
p
e
f
u
l
l
y
o
n
e
d
a
y
.
.
.
.
t
h
e
Tr
a
v
e
l
o
k
a
s
y
st
e
m
w
i
l
l
b
e
a
b
l
e
t
o
‘
r
e
a
d
’
t
h
e
h
a
b
i
t
s
o
f
i
t
s
u
sers
(
v
i
a
s
o
c
i
a
l
m
e
d
i
a
)
,
so
t
h
a
t
i
t
a
u
t
o
m
a
t
i
c
a
l
l
y
p
r
o
v
i
d
e
s
‘
p
r
o
m
o
s/
d
i
s
c
o
u
n
t
s’
a
c
c
o
r
d
i
n
g
t
o
t
h
e
i
n
t
e
r
e
s
t
s/
h
a
b
i
t
s
o
f
i
t
s
u
s
e
r
s.
T
h
a
n
k
y
o
u
Tr
a
v
e
l
o
k
a
,
h
o
p
e
f
o
r
l
o
n
g
l
i
f
e
a
n
d
t
h
e
b
e
st
t
h
i
n
g
e
v
e
r
ð
Ÿ
™
ð
Ÿ
’
ž
”
Ex
3
3
“
c
e
p
a
t
p
ro
s
e
s
b
e
l
i
t
i
k
e
t
n
y
a
.
t
p
a
k
h
i
r
2
i
n
i
h
a
r
g
a
t
i
k
e
t
k
e
l
u
a
r
n
e
g
e
ri
y
g
d
i
t
a
m
p
i
l
k
a
n
s
u
k
a
p
a
k
e
h
a
r
g
a
h
o
a
x
.
T
a
m
p
i
l
2
j
u
t
a
p
a
s
m
a
u
b
a
y
a
r
3
j
u
t
a
.
”
“
Th
e
t
i
c
k
e
t
p
u
r
c
h
a
se
p
r
o
c
e
ss
i
s
f
a
st
.
H
o
w
e
v
e
r
,
r
e
c
e
n
t
l
y
,
t
h
e
d
i
sp
l
a
y
e
d
i
n
t
e
r
n
a
t
i
o
n
a
l
t
i
c
k
e
t
p
r
i
c
e
s
h
a
v
e
b
e
e
n
h
o
a
x
e
d
.
I
t
sh
o
w
s
2
m
i
l
l
i
o
n
w
h
e
n
y
o
u
'r
e
t
r
y
i
n
g
t
o
p
a
y
3
mi
l
l
i
o
n
.
”
Ex
4
2
“
sa
y
a
sa
n
g
a
t
k
e
c
e
w
a
k
a
r
n
a
s
a
m
p
a
i
se
k
a
r
a
n
g
p
a
y
l
a
t
e
r
n
y
a
t
i
d
a
k
a
k
t
i
f
,
p
a
d
a
h
a
l
s
a
y
a
u
n
t
u
k
p
e
m
b
a
y
a
r
a
n
sel
a
l
u
t
e
p
a
t
w
a
k
t
u
d
a
n
t
i
d
a
k
p
e
rn
a
h
s
e
k
a
l
i
p
u
n
t
e
rl
a
m
b
a
t
,
s
a
y
a
h
a
n
y
a
m
i
n
t
a
a
k
t
i
f
k
a
n
k
e
m
b
a
l
a
i
p
a
y
l
a
t
e
rn
y
a
,
j
a
d
i
s
a
y
a
k
a
si
h
b
i
n
t
a
n
g
2
d
u
l
u
u
n
t
u
k
semen
t
a
ra
w
a
k
t
u
.
t
h
a
n
k
s”
“
I
'm
v
e
r
y
d
i
s
a
p
p
o
i
n
t
e
d
b
e
c
a
u
s
e
P
a
y
L
a
t
e
r
i
s
s
t
i
l
l
i
n
a
c
t
i
v
e
,
e
v
e
n
t
h
o
u
g
h
I
a
l
w
a
y
s
m
a
k
e
p
a
y
me
n
t
s
o
n
t
i
m
e
a
n
d
h
a
v
e
n
e
v
e
r
b
e
e
n
l
a
t
e
.
I
j
u
s
t
a
s
k
e
d
f
o
r
P
a
y
La
t
e
r
t
o
b
e
r
e
a
c
t
i
v
a
t
e
d
,
so
I
'm
g
i
v
i
n
g
i
t
2
st
a
r
s
f
o
r
t
h
e
t
i
me
b
e
i
n
g
.
Th
a
n
k
s.”
Ex
5
1
“
Ap
l
i
k
a
si
m
a
k
i
n
s
u
s
a
h
d
a
n
s
u
l
i
t
u
t
k
d
i
p
a
h
a
m
i
d
i
b
a
n
d
i
n
g
k
a
n
a
p
l
i
k
a
s
i
T
r
a
v
e
l
o
k
a
s
e
b
e
l
u
m
m
a
s
a
C
O
VI
D
l
b
h
m
u
d
a
h
u
t
k
d
i
p
a
h
a
m
i
”
“
Th
e
a
p
p
l
i
c
a
t
i
o
n
i
s
g
e
t
t
i
n
g
m
o
r
e
d
i
f
f
i
c
u
l
t
a
n
d
h
a
r
d
e
r
t
o
u
n
d
e
r
s
t
a
n
d
c
o
mp
a
r
e
d
t
o
t
h
e
Tr
a
v
e
l
o
k
a
a
p
p
l
i
c
a
t
i
o
n
b
e
f
o
r
e
t
h
e
C
O
V
I
D
p
e
r
i
o
d
,
w
h
i
c
h
w
a
s
e
a
s
i
e
r
t
o
u
n
d
e
r
s
t
a
n
d
.
”
3
.
2
.
Da
t
a
prepro
ce
s
s
ing
I
n
th
is
s
tu
d
y
,
d
ata
p
r
etr
ea
t
m
en
t
is
ess
en
tial
s
in
ce
it
d
ir
ec
tly
af
f
ec
ts
h
o
w
well
s
en
tim
en
t
ca
teg
o
r
izatio
n
wo
r
k
s
f
o
r
te
x
t
ab
o
u
t
I
n
d
o
n
esian
s
tar
tu
p
s
.
T
o
less
en
tex
tu
al
n
o
is
e
th
at
d
o
e
s
n
o
t
ad
d
s
em
an
tic
m
ea
n
in
g
an
d
co
u
ld
a
d
v
er
s
ely
im
p
ac
t
f
ea
tu
r
e
ex
tr
ac
tio
n
in
ML
m
o
d
els,
p
u
n
ctu
atio
n
r
e
d
u
ctio
n
an
d
s
p
ec
ial
ch
ar
ac
ter
h
an
d
lin
g
ar
e
u
s
ed
.
L
o
wer
ca
s
in
g
is
p
er
f
o
r
m
e
d
to
k
ee
p
lex
ically
co
n
s
is
ten
t
an
d
d
o
es
n
o
t
allo
w
ca
s
e
ch
an
g
es
to
lead
to
d
u
p
licate
f
ea
tu
r
e
r
ep
r
esen
tatio
n
s
.
T
o
k
en
izatio
n
:
th
e
f
ir
s
t
s
tep
in
d
o
wn
s
tr
ea
m
-
lan
g
u
e
an
aly
s
is
,
th
is
p
iece
ca
n
b
e
ty
p
ically
d
ef
in
ed
as
a
m
ea
n
in
g
f
u
l
p
iece
o
f
tex
t
.
R
em
o
v
in
g
s
to
p
wo
r
d
s
en
ab
les
th
e
m
o
d
el
to
f
o
c
u
s
o
n
em
o
tio
n
a
lly
r
ich
wo
r
d
s
,
r
e
m
o
v
in
g
h
i
g
h
-
f
r
e
q
u
en
c
y
s
en
tim
en
t
-
n
eu
tr
al
wo
r
d
s
.
Fin
ally
,
lem
m
atiza
tio
n
an
d
s
tem
m
in
g
wer
e
b
o
th
u
s
ed
to
n
o
r
m
alis
e
th
e
wo
r
d
f
o
r
m
s
,
r
ed
u
cin
g
v
o
ca
b
u
lar
y
s
p
ar
s
ity
p
r
esen
ted
in
th
ese
m
o
r
p
h
o
lo
g
i
ca
lly
r
ich
lan
g
u
ag
es (
e.
g
.
,
I
n
d
o
n
esian
)
s
o
th
ey
g
en
e
r
alis
ed
b
etter
.
T
h
is
f
u
r
th
er
p
r
ep
r
o
ce
s
s
in
g
p
i
p
elin
e
h
elp
s
to
im
p
r
o
v
e
t
h
e
d
ata
q
u
ality
wh
ile
r
e
d
u
cin
g
n
o
is
e
an
d
r
ed
u
n
d
an
cy
,
f
ac
ilit
atin
g
b
etter
,
an
d
m
o
r
e
ac
c
u
r
ate
s
en
tim
en
t
class
if
icatio
n
.
T
h
is
also
m
ea
n
s
th
at
b
o
th
SLM
an
d
r
e
g
u
lar
ML
u
s
e
th
e
s
am
e
p
r
ep
r
o
ce
s
s
in
g
p
r
o
ce
d
u
r
es
to
en
s
u
r
e
co
n
s
is
ten
cy
in
m
eth
o
d
o
lo
g
y
.
On
th
e
o
th
er
h
an
d
,
L
L
Ms
u
s
e
in
ter
n
al
s
u
b
wo
r
d
to
k
en
izatio
n
a
n
d
c
o
n
tex
t
u
al
r
ep
r
esen
tatio
n
s
t
h
at
ar
e
d
e
s
ig
n
ed
to
wo
r
k
well
with
r
aw
tex
t,
s
o
th
ey
n
ee
d
v
e
r
y
litt
le
p
r
ep
ar
atio
n
o
th
e
r
th
an
m
in
im
al
tex
t c
lean
in
g
.
3
.
3
.
F
ine
t
un
ing
pro
ce
s
s
T
h
e
h
u
m
an
f
in
e
-
tu
n
in
g
tech
n
o
lo
g
y
o
n
p
r
etr
ain
ed
r
ep
r
esen
tatio
n
s
th
r
o
u
g
h
lan
g
u
ag
e
f
e
atu
r
es
an
d
co
n
tex
tu
al
n
u
a
n
ce
s
f
r
o
m
th
e
c
ase
s
tu
d
y
o
f
s
en
tim
en
t
an
aly
s
is
co
n
d
u
cted
b
y
eig
h
t
co
m
p
a
n
i
es
in
I
n
d
o
n
esia
f
o
r
b
o
th
L
L
Ms
an
d
SLM
s
s
u
ch
as
B
E
R
T
an
d
Dis
til
B
E
R
T
.
I
n
s
u
p
er
v
is
ed
lear
n
in
g
,
a
lab
ele
d
d
ataset
o
b
tain
e
d
f
r
o
m
th
e
s
co
r
e
c
o
lu
m
n
is
u
s
e
d
.
Fo
r
ML
an
d
th
e
SLM
m
o
d
els,
we
p
er
f
o
r
m
a
lo
t
o
f
p
r
ep
r
o
ce
s
s
in
g
to
im
p
r
o
v
e
d
ata
q
u
ality
an
d
r
e
d
u
ce
f
ea
tu
r
e
s
p
ar
s
ity
.
Su
ch
as
lo
wer
ca
s
in
g
,
to
k
en
is
atio
n
,
s
to
p
wo
r
d
r
e
m
o
v
al,
p
u
n
ctu
atio
n
an
d
s
p
ec
ial
ch
a
r
ac
ter
r
em
o
v
al,
an
d
lem
m
atiza
tio
n
o
r
s
tem
m
in
g
.
I
n
co
n
tr
ast,
L
L
Ms
in
v
o
lv
e
less
p
r
ep
r
o
ce
s
s
in
g
an
d
r
ely
o
n
c
o
n
tex
tu
al
em
b
ed
d
in
g
s
an
d
i
n
ter
n
al
s
u
b
wo
r
d
to
k
en
izatio
n
.
L
ig
h
tweig
h
t
SLM
s
ar
e
ev
alu
ated
in
co
n
ju
n
ctio
n
with
h
ea
v
i
er
L
L
Ms
to
p
r
o
v
id
e
a
b
alan
ce
b
etwe
en
co
m
p
u
tatio
n
al
ef
f
icien
cy
an
d
p
r
ed
ictiv
e
a
b
ilit
y
.
T
h
is
r
esea
r
ch
em
p
l
o
y
s
a
lear
n
i
n
g
r
ate
eq
u
als
to
5
×
10
-
1
,
th
r
ee
ep
o
ch
s
o
f
tr
ai
n
in
g
,
b
atch
s
ize
eq
u
als
to
1
6
f
o
r
tr
ain
i
n
g
an
d
6
4
f
o
r
e
v
alu
atio
n
,
5
0
0
war
m
-
u
p
s
tep
s
an
d
weig
h
t
d
ec
ay
eq
u
al
to
0
.
0
1
d
u
r
in
g
f
in
e
-
tu
n
in
g
.
T
h
ese
ar
e
d
esig
n
ed
to
p
r
o
v
id
e
s
tab
le
c
o
n
v
er
g
en
ce
wh
ile
s
till
b
ein
g
ab
le
to
ad
a
p
t
to
c
h
an
g
in
g
in
p
u
t
d
is
tr
ib
u
tio
n
s
.
T
h
ey
ar
e
ev
alu
ated
u
s
in
g
ac
cu
r
a
cy
,
p
r
e
cisi
o
n
,
r
ec
all
,
an
d
F1
-
s
co
r
e,
with
s
im
ilar
g
o
o
d
r
e
s
u
lts
f
o
r
all
m
o
d
els af
ter
f
i
n
e
-
t
u
n
in
g
.
3
.
4
.
E
v
a
lua
t
i
o
n
m
et
rics
T
h
e
p
er
f
o
r
m
an
ce
o
u
tp
u
t
f
o
llo
win
g
th
e
co
m
p
letio
n
o
f
d
ata
t
r
ain
in
g
an
d
p
r
o
d
u
ctio
n
—
wh
ic
h
lead
s
to
ev
alu
atio
n
.
T
h
e
ac
cu
r
ac
y
,
p
r
e
cisi
o
n
,
r
ec
all
an
d
F
1
-
s
co
r
es
o
f
th
e
m
o
d
els
ev
al
u
ated
ar
e
s
h
o
wn
in
(
18
)
to
(
21
)
,
r
esp
ec
tiv
ely
.
Fig
u
r
e
4
ex
p
lain
s
ea
ch
co
m
p
leted
s
tep
.
T
h
e
r
e
s
u
lts
an
d
d
is
cu
s
s
io
n
s
ec
tio
n
s
g
iv
e
d
etailed
in
s
ig
h
t
in
to
th
is
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
5
,
Octo
b
er
2
0
2
6
:
4
4
3
2
-
4
4
4
6
4440
=
+
+
+
+
(
18
)
=
+
(
19
)
=
+
(
20
)
1
−
=
2
×
×
+
(
21
)
Fig
u
r
e
4
.
C
o
m
m
e
n
ts
class
if
ier
u
s
in
g
L
L
M/SL
M
3
.
5
.
E
x
perim
ent
a
l
des
ig
n
P
o
s
it
i
v
e
,
n
e
g
a
ti
v
e
,
o
r
n
e
u
t
r
a
l
co
m
m
e
n
t
s
r
e
f
e
r
t
o
t
h
e
d
at
a
s
et
a
f
t
e
r
p
r
e
p
r
o
c
e
s
s
i
n
g
.
I
t
i
n
d
i
c
at
e
s
w
h
e
t
h
e
r
t
h
e
c
o
m
m
e
n
t
s
a
r
e
c
a
t
e
g
o
r
i
z
e
d
a
s
p
o
s
i
t
i
v
e
,
n
e
g
a
t
i
v
e
,
o
r
n
e
u
t
r
al
s
en
t
i
m
e
n
t
.
T
h
es
e
d
e
t
ai
ls
a
r
e
o
u
t
lin
e
d
i
n
T
a
b
l
e
3
.
T
h
e
tr
ain
in
g
p
ar
am
eter
s
as seen
in
T
ab
le
4
f
o
r
B
E
R
T
an
d
Dis
t
ilB
E
R
T
wer
e
o
p
tim
ized
f
o
r
ef
f
icien
cy
,
with
tr
ain
in
g
co
n
d
u
cte
d
f
o
r
t
h
r
ee
ep
o
ch
s
to
en
s
u
r
e
q
u
ick
co
n
v
er
g
e
n
ce
.
T
h
e
lear
n
in
g
r
ate
was
s
et
t
o
0
.
0
0
0
0
5
,
with
a
500
-
s
tep
war
m
-
u
p
f
o
r
g
r
ad
u
a
l
ad
ju
s
tm
en
t.
A
weig
h
t
d
ec
a
y
o
f
0
.
0
1
was
u
s
ed
to
p
r
ev
e
n
t
o
v
er
f
itti
n
g
.
T
h
e
d
ataset
was sp
lit in
to
tr
ain
in
g
,
v
alid
atio
n
,
an
d
test
in
g
s
u
b
s
ets to
en
h
an
ce
m
o
d
el
tr
ai
n
in
g
a
n
d
ev
alu
atio
n
.
T
ab
le
3
.
Descr
ip
tio
n
o
f
th
e
d
at
aset
D
a
t
a
s
e
t
F
e
a
t
u
r
e
P
o
si
t
i
v
e
N
e
g
a
t
i
v
e
N
e
u
t
r
a
l
To
t
a
l
B
l
i
b
l
i
3
8
5
,
7
7
1
1
9
,
9
4
1
1
1
,
1
9
0
1
1
6
,
9
0
2
To
k
o
p
e
d
i
a
3
3
5
,
5
9
9
3
1
,
0
5
8
4
,
5
8
5
7
1
,
2
4
2
B
u
k
a
l
a
p
a
k
3
6
2
,
5
4
9
2
3
,
4
7
3
4
,
1
2
5
9
0
,
1
4
7
DANA
3
3
1
,
1
6
3
6
6
,
5
8
4
8
,
7
1
8
1
0
6
,
4
6
5
G
o
j
e
k
3
4
9
,
3
3
1
5
6
,
6
5
9
1
0
,
0
2
7
1
1
6
,
0
1
7
OVO
3
2
4
,
0
7
5
5
3
,
5
8
0
5
,
5
2
7
8
3
,
1
8
2
R
u
a
n
g
g
u
r
u
3
9
7
,
0
6
5
1
0
,
6
8
4
4
,
0
8
9
1
1
1
,
8
3
8
Tr
a
v
e
l
o
k
a
3
1
1
8
,
9
7
7
1
8
,
4
9
0
8
,
4
8
1
1
4
5
,
9
4
8
T
ab
le
4
.
T
r
ai
n
in
g
p
ar
am
eter
s
M
o
d
e
l
p
a
r
a
me
t
e
r
s
B
ER
T
a
n
d
D
i
s
t
i
l
B
ER
T
Ep
o
c
h
s
3
Le
a
r
n
i
n
g
r
a
t
e
0
.
0
0
0
0
5
W
a
r
mu
p
_
s
t
e
p
s
5
0
0
W
e
i
g
h
t
_
D
e
c
a
y
0
.
0
1
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
F
in
e
-
tu
n
in
g
la
n
g
u
a
g
e
mo
d
els
fo
r
s
o
cia
l sen
timen
t:
in
s
ig
h
ts
fr
o
m
s
ta
r
tu
p
u
s
er fee
d
b
a
ck
(
Li
-
Hu
a
Li
)
4441
T
h
e
d
ataset
will
b
e
s
p
lit
in
to
t
r
ain
in
g
(
7
0
%),
test
in
g
(
1
5
%),
an
d
v
alid
atio
n
(
1
5
%)
s
u
b
s
ets
f
o
r
r
eliab
le
r
esu
lts
.
Af
ter
ac
q
u
is
itio
n
an
d
p
r
ep
r
o
ce
s
s
in
g
,
th
e
d
ataset
is
d
iv
id
ed
,
f
o
llo
wed
b
y
tr
a
in
in
g
,
test
in
g
,
a
n
d
v
alid
atio
n
.
Nex
t,
f
in
e
tu
n
in
g
o
f
th
e
SLM
a
n
d
L
L
M
will
b
e
p
er
f
o
r
m
ed
,
f
o
llo
we
d
b
y
m
o
d
el
tr
ain
in
g
.
Fin
ally
,
th
e
s
en
tim
en
t
an
aly
s
is
p
er
f
o
r
m
an
ce
o
f
ML
,
SLM
,
an
d
L
L
M
will
b
e
ev
alu
ated
an
d
co
m
p
ar
ed
to
ass
ess
th
e
ef
f
ec
tiv
en
ess
o
f
f
in
e
tu
n
in
g
.
A
co
m
p
ar
is
o
n
is
m
ad
e
b
etwe
en
B
E
R
T
(
r
ep
r
esen
t
L
L
M)
a
n
d
Dis
tilB
E
R
T
(
r
ep
r
esen
t
SLM
)
,
alo
n
g
with
f
o
u
r
o
t
h
er
c
o
n
v
e
n
tio
n
a
l
ML
m
o
d
els
to
m
ea
s
u
r
e
h
o
w
th
e
y
p
e
r
f
o
r
m
.
Pre
p
r
o
c
ess
in
g
:
r
em
o
v
al
o
f
p
u
n
ctu
atio
n
,
lo
wer
ca
s
in
g
,
an
d
to
k
en
izatio
n
(
th
e
p
r
o
ce
s
s
o
f
b
r
ea
k
in
g
a
s
en
ten
ce
in
to
to
k
en
s
)
,
s
to
p
wo
r
d
s
r
em
o
v
al,
lem
m
atizin
g
o
r
s
tem
m
in
g
.
Fo
r
th
at
ex
p
er
im
en
t,
th
e
h
ar
d
war
e
is
an
I
n
tel
i9
1
4
9
0
0
K
C
PU,
NVI
DI
A
GeFo
r
ce
R
T
X
4
0
6
0
GPU
an
d
3
2
GB
o
f
R
AM
.
Pro
g
r
am
m
in
g
u
s
ed
is
p
y
th
o
n
o
n
An
ac
o
n
d
a
e
n
v
ir
o
n
m
en
t.
4.
RE
SU
L
T
AND
DI
SCUS
SI
O
N
4
.
1
.
Resul
t
T
r
ad
itio
n
al
ML
m
eth
o
d
s
,
as
illu
s
tr
ated
in
T
a
b
le
5
,
o
n
l
y
th
e
lo
g
is
tic
r
eg
r
ess
io
n
m
o
d
el
f
o
r
R
u
an
g
g
u
r
u
d
ataset
g
av
e
th
e
h
ig
h
est
F1
-
s
c
o
r
e
(
8
7
.
5
4
%).
I
n
d
o
B
E
R
T
(
r
ep
r
esen
t
L
L
M)
with
th
e
h
ig
h
est
F1
-
s
co
r
e
=
8
9
.
9
4
%
f
o
r
R
u
an
g
g
u
r
u
as
s
h
o
wn
i
n
T
ab
le
6
,
a
n
d
Dis
tilB
E
R
T
(
r
ep
r
esen
t
SLM
)
g
ain
s
h
i
g
h
er
s
c
o
r
e
th
an
R
u
an
g
g
u
r
u
d
ataset
wh
ich
is
e
q
u
al
t
o
8
9
.
5
3
%
as
s
h
o
wn
in
T
ab
le
7
.
I
n
T
ab
le
8
,
th
is
r
esear
c
h
c
o
m
p
ar
es
th
e
F1
-
s
co
r
es
o
f
all
m
o
d
els with
r
esp
ec
t to
b
o
th
d
atasets
,
wh
er
e
B
E
R
T
(
r
ep
r
esen
t
L
L
M)
ac
h
iev
es su
p
er
io
r
p
er
f
o
r
m
an
ce
co
m
p
ar
e
d
to
tr
ad
itio
n
al
ML
tech
n
iq
u
es.
Dis
tilB
E
R
T
(
r
ep
r
esen
t
SLM
)
p
r
o
v
id
es
s
im
ilar
p
er
f
o
r
m
a
n
ce
with
a
s
m
aller
am
o
u
n
t
o
f
c
o
m
p
u
tatio
n
al
p
o
w
er
,
an
d
I
n
d
o
B
E
R
T
(
r
ep
r
esen
t
L
L
M)
,
th
at
is
s
p
ec
if
ically
tr
ai
n
ed
f
o
r
I
n
d
o
n
esian
o
u
tp
er
f
o
r
m
s
all
th
e
o
t
h
er
m
o
d
els at
th
e
s
am
e
s
en
tim
en
t a
n
aly
s
is
task
.
T
h
is
r
esear
ch
tak
es
o
n
e
s
a
m
p
le
f
r
o
m
th
e
h
ig
h
est
p
er
f
o
r
m
an
ce
u
s
in
g
ML
m
o
d
el
wh
ich
is
lo
g
is
tic
r
eg
r
ess
io
n
o
n
R
u
an
g
g
u
r
u
s
tar
tu
p
d
ataset.
T
h
en
v
is
u
alize
s
p
r
ec
is
o
n
-
r
ec
all
cu
r
v
es
o
f
th
e
m
o
d
el
f
r
o
m
Fig
u
r
e
5
.
B
esid
es
F1
-
s
co
r
e
th
is
r
esear
ch
also
an
al
y
ze
s
d
e
ep
er
u
s
in
g
r
ec
eiv
er
o
p
e
r
atin
g
ch
ar
ac
ter
is
tic
-
ar
ea
u
n
d
er
t
h
e
cu
r
v
e
(
R
OC
-
AUC
)
.
T
h
e
s
u
m
m
ar
y
o
f
th
e
p
e
r
f
o
r
m
an
ce
is
F1
-
s
co
r
e
8
7
.
5
4
%,
R
OC
-
AU
C
(
Ov
R
)
:
9
0
.
5
9
%,
av
er
a
g
e
p
r
ec
is
io
n
(
A
P)
(
m
ac
r
o
)
:
5
8
.
5
6
%,
an
d
AP
(
weig
h
ted
)
:
9
1
.
7
3
%.
L
o
g
is
tic
r
eg
r
ess
io
n
s
u
cc
ee
d
s
well
f
o
r
p
o
lar
ized
s
en
tim
en
ts
,
esp
ec
ially
p
o
s
itiv
e
r
ev
iews,
ac
co
r
d
in
g
to
th
e
p
r
ec
is
io
n
–
r
ec
all
an
aly
s
is
,
b
u
t
n
eu
tr
al
s
en
tim
en
t
is
s
till
d
if
f
icu
lt
b
ec
au
s
e
o
f
its
s
em
an
tic
am
b
ig
u
ity
,
u
n
d
er
s
co
r
in
g
t
h
e
s
ig
n
if
ican
ce
o
f
im
b
alan
ce
-
awa
r
e
ev
al
u
atio
n
m
etr
ics.
T
ab
le
5
.
T
h
e
p
er
f
o
r
m
a
n
ce
o
f
u
s
in
g
ML
tr
ad
itio
n
al
m
o
d
els
S
t
a
r
t
u
p
s
M
o
d
e
l
s
A
c
c
u
r
a
c
y
(
%)
P
r
e
c
i
s
i
o
n
(
%)
R
e
c
a
l
l
(
%)
F
1
(
%)
B
l
i
b
l
i
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
2
.
1
7
7
9
.
7
6
8
2
.
1
7
8
0
.
1
4
S
V
M
8
1
.
9
6
7
8
.
9
7
8
1
.
9
6
7
8
.
8
5
R
a
n
d
o
m
f
o
r
e
s
t
8
0
.
6
3
7
7
.
2
2
8
0
.
6
3
7
7
.
8
3
X
G
B
o
o
st
8
0
.
5
0
7
7
.
4
7
8
0
.
5
0
7
6
.
7
1
To
k
o
p
e
d
i
a
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
4
.
1
9
8
1
.
3
6
8
4
.
1
9
8
1
.
8
8
S
V
M
8
4
.
2
5
8
2
.
5
6
8
4
.
2
5
8
1
.
6
3
X
G
B
o
o
st
8
2
.
7
3
7
9
.
5
0
8
2
.
7
3
8
0
.
2
2
R
a
n
d
o
m
f
o
r
e
s
t
8
2
.
7
5
8
0
.
3
2
8
2
.
7
5
8
0
.
1
4
B
u
k
a
l
a
p
a
k
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
6
.
9
3
8
5
.
1
3
8
6
.
9
3
8
5
.
2
6
S
V
M
8
6
.
7
9
8
3
.
3
5
8
6
.
7
9
8
4
.
9
9
R
a
n
d
o
m
f
o
r
e
s
t
8
5
.
5
5
8
3
.
1
3
8
5
.
5
5
8
3
.
8
3
X
G
B
o
o
st
8
5
.
3
7
8
2
.
5
7
8
5
.
3
7
8
3
.
5
4
DANA
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
1
.
2
3
7
7
.
8
5
8
1
.
2
3
7
7
.
9
2
S
V
M
8
1
.
1
0
7
7
.
6
1
8
1
.
1
0
7
7
.
4
3
X
G
B
o
o
st
7
9
.
9
7
7
6
.
7
2
7
9
.
9
7
7
6
.
3
1
R
a
n
d
o
m
f
o
r
e
s
t
7
9
.
8
8
7
6
.
8
5
7
9
.
8
7
7
6
.
1
0
G
o
j
e
k
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
7
9
.
1
0
7
5
.
6
4
7
9
.
1
0
7
5
.
9
9
S
V
M
7
9
.
2
9
7
9
.
0
4
7
9
.
2
9
7
5
.
8
1
R
a
n
d
o
m
f
o
r
e
s
t
7
7
.
6
9
7
3
.
7
2
7
7
.
6
9
7
4
.
2
7
X
G
B
o
o
st
7
7
.
2
1
7
4
.
6
7
7
7
.
2
1
7
3
.
8
7
OVO
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
3
.
7
5
8
0
.
9
2
8
3
.
7
5
8
0
.
8
3
S
V
M
8
3
.
6
8
7
8
.
5
5
8
3
.
6
8
8
0
.
6
2
R
a
n
d
o
m
f
o
r
e
s
t
8
2
.
6
2
7
7
.
7
7
8
2
.
6
2
7
9
.
5
1
X
G
B
o
o
st
8
2
.
3
9
7
9
.
7
8
8
2
.
3
9
7
9
.
2
7
R
u
a
n
g
g
u
r
u
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
9
.
5
1
8
6
.
3
3
8
9
.
5
1
8
7
.
5
4
S
V
M
8
9
.
4
4
8
6
.
2
3
8
9
.
4
4
8
7
.
1
4
X
G
B
o
o
st
8
8
.
9
6
8
5
.
8
1
8
8
.
9
6
8
6
.
2
8
R
a
n
d
o
m
f
o
r
e
s
t
8
8
.
3
9
8
4
.
6
9
8
8
.
3
9
8
5
.
6
4
Tr
a
v
e
l
o
k
a
Lo
g
i
s
t
i
c
r
e
g
r
e
ssi
o
n
8
8
.
1
6
8
5
.
5
2
8
8
.
1
6
8
5
.
6
2
S
V
M
8
8
.
1
6
8
5
.
6
3
8
8
.
1
6
8
5
.
5
4
R
a
n
d
o
m
f
o
r
e
s
t
8
7
.
1
1
8
3
.
9
0
8
7
.
1
1
8
4
.
7
6
X
G
B
o
o
st
8
6
.
9
9
8
4
.
2
3
8
6
.
9
9
8
3
.
9
9
Evaluation Warning : The document was created with Spire.PDF for Python.