I
nd
o
ne
s
ia
n J
o
urna
l o
f
E
lect
rica
l En
g
ineering
a
nd
Co
m
pu
t
er
Science
Vo
l.
3
6
,
No
.
2
,
N
o
v
em
b
e
r
20
24
,
p
p
.
1
1
2
7
~
1
139
I
SS
N:
2
502
-
4
7
52
,
DOI
: 1
0
.
1
1
5
9
1
/ijee
cs
.v
3
6
.
i
2
.
pp
1
1
2
7
-
1
1
3
9
1127
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ee
cs
.
ia
esco
r
e.
co
m
Sentimen
t
a
na
ly
sis
of stude
nt
ev
a
lua
tion feedback
us
ing
trans
former
-
ba
sed la
ng
ua
g
e mo
dels
I
bn
u
Da
qiq
il
I
D
1
,
H
endy
Sa
pu
t
ra
1
,
Sy
a
m
s
ud
hu
ha
2
,
Ra
h
m
a
d K
urnia
wa
n
1
,
Ya
nti
Andriy
a
ni
1
1
D
e
p
a
r
t
e
m
e
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
N
a
t
u
r
a
l
S
c
i
e
n
c
e
,
U
n
i
v
e
r
si
t
a
s
R
i
a
u
,
P
e
k
a
n
b
a
r
u
,
I
n
d
o
n
e
s
i
a
2
D
e
p
a
r
t
e
m
e
n
o
f
M
a
t
h
,
U
n
i
v
e
r
si
t
a
s
R
i
a
u
,
P
e
k
a
n
b
a
r
u
,
I
n
d
o
n
e
si
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
an
14
,
2
0
2
4
R
ev
is
ed
J
u
n
24
,
2
0
2
4
Acc
ep
ted
J
u
l
14
,
2
0
2
4
Th
is
p
a
p
e
r
p
r
o
p
o
se
s
a
n
a
p
p
ro
a
c
h
to
se
n
ti
m
e
n
t
a
n
a
ly
sis
o
f
stu
d
e
n
t
e
v
a
lu
a
ti
o
n
fe
e
d
b
a
c
k
u
sin
g
t
ra
n
sf
o
rm
e
r
-
b
a
se
d
lan
g
u
a
g
e
m
o
d
e
ls.
Th
e
p
rima
ry
o
b
jec
ti
v
e
o
f
t
h
is
st
u
d
y
is
to
c
o
n
d
u
c
t
a
n
i
n
-
d
e
p
th
a
n
a
ly
sis
o
f
se
n
ti
m
e
n
t
e
x
p
re
ss
e
d
in
stu
d
e
n
t
e
v
a
lu
a
ti
o
n
fe
e
d
b
a
c
k
,
with
a
f
o
c
u
s
o
n
i
n
tro
d
u
c
i
n
g
c
o
n
tex
t
u
a
l
u
n
d
e
rsta
n
d
i
n
g
i
n
t
o
th
e
se
n
ti
m
e
n
t
c
las
sifica
ti
o
n
p
ro
c
e
ss
.
In
th
is
re
se
a
rc
h
,
fo
u
r
d
iffere
n
t
v
a
rian
ts
o
f
tra
n
sfo
rm
e
r
lan
g
u
a
g
e
m
o
d
e
ls
we
re
a
s
se
ss
e
d
,
n
a
m
e
l
y
m
u
lt
il
in
g
u
a
l
b
i
d
irec
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
re
se
n
tati
o
n
s
fro
m
tr
a
n
sfo
rm
e
rs
(
M
BERT
)
,
I
n
d
o
BERT
,
R
o
BER
Ta
In
d
o
n
e
sia
,
a
n
d
g
e
n
e
ra
ti
v
e
p
re
-
train
e
d
tran
sfo
rm
e
r
(
G
P
T
-
2
I
n
d
o
n
e
sia
)
.
Ad
d
it
i
o
n
a
ll
y
,
we
a
lso
c
o
m
p
a
re
d
t
h
e
p
e
rfo
rm
a
n
c
e
o
f
t
ra
n
sf
o
rm
e
r
m
o
d
e
ls
wit
h
tw
o
trad
it
i
o
n
a
l
m
o
d
e
l
s,
n
a
m
e
ly
su
p
p
o
rt
v
e
c
to
r
m
a
c
h
in
e
(S
V
M
)
a
n
d
Na
iv
e
Ba
y
e
s
(NB)
.
Th
e
e
v
a
l
u
a
ti
o
n
wa
s
c
o
n
d
u
c
ted
u
si
n
g
fe
e
d
b
a
c
k
d
a
ta
c
o
ll
e
c
ted
fro
m
th
e
E
va
lu
a
s
i
Do
s
en
o
leh
Ma
h
a
s
is
w
a
(
EDOM
)
sy
ste
m
a
t
Riau
U
n
iv
e
rsit
y
,
w
h
ich
h
a
d
b
e
e
n
c
a
teg
o
rize
d
a
s
e
it
h
e
r
p
o
siti
v
e
o
r
n
e
g
a
ti
v
e
.
Th
e
o
u
tc
o
m
e
s
in
d
ica
te
th
a
t
In
d
o
BERT
b
a
se
u
n
c
a
se
d
e
x
h
i
b
it
s
th
e
h
ig
h
e
st
p
e
rf
o
rm
a
n
c
e
,
with
p
re
c
isio
n
,
a
c
c
u
ra
c
y
,
a
n
d
re
c
a
ll
v
a
l
u
e
s
o
f
0
.
8
5
8
,
0
.
9
2
9
,
a
n
d
0
.
9
1
1
,
re
sp
e
c
ti
v
e
ly
.
Th
is
o
b
se
rv
a
ti
o
n
h
ig
h
li
g
h
ts
th
e
e
ffe
c
ti
v
e
n
e
ss
o
f
t
ra
n
sf
o
rm
e
r
-
b
a
se
d
lan
g
u
a
g
e
m
o
d
e
ls
in
se
n
ti
m
e
n
t
a
n
a
l
y
sis
o
f
stu
d
e
n
t
e
v
a
l
u
a
ti
o
n
fe
e
d
b
a
c
k
a
n
d
p
ro
v
i
d
e
s
in
sig
h
ts f
o
r
imp
ro
v
i
n
g
e
d
u
c
a
ti
o
n
a
l
a
ss
e
ss
m
e
n
t
p
ra
c
ti
c
e
s.
K
ey
w
o
r
d
s
:
B
er
t
Dee
p
lear
n
in
g
L
an
g
u
ag
e
m
o
d
el
Sen
tim
en
t a
n
aly
s
is
Stu
d
en
t e
v
alu
atio
n
an
aly
s
is
T
r
an
s
f
o
r
m
e
r
-
b
ased
m
o
d
el
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
I
b
n
u
Da
q
iq
il I
D
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
Scie
n
ce
,
Facu
lty
o
f
Natu
r
al
Scien
c
e,
Un
iv
er
s
itas
R
iau
C
am
p
u
s
B
in
a
W
id
y
a
KM
.
1
2
,
5
,
Simp
an
g
B
ar
u
,
Pek
an
b
a
r
u
C
ity
,
R
iau
2
8
2
9
3
,
I
n
d
o
n
esia
E
m
ail: d
aq
iq
il@
s
.
o
k
ay
am
a
-
u
.
ac
.
jp
1.
I
NT
RO
D
UCT
I
O
N
In
h
ig
h
er
e
d
u
ca
tio
n
,
s
tu
d
e
n
t
f
ee
d
b
ac
k
em
er
g
es
as
a
cr
u
cial
elem
en
t
in
th
e
lear
n
in
g
p
r
o
c
ess
,
alb
eit
f
r
eq
u
e
n
tly
p
o
s
in
g
ch
allen
g
es
in
b
o
th
its
p
r
o
v
is
io
n
an
d
r
ec
e
p
tio
n
.
A
s
u
b
s
tan
tial
b
o
d
y
o
f
r
esear
ch
h
ig
h
lig
h
ts
th
at
co
n
s
tr
u
ctiv
e
f
ee
d
b
ac
k
p
la
y
s
a
ce
n
tr
al
r
o
le
in
n
u
r
tu
r
in
g
t
h
e
g
r
o
wth
o
f
s
tu
d
en
ts
as
s
elf
-
r
elian
t
lear
n
er
s
an
d
s
ig
n
if
ican
tly
co
n
tr
ib
u
tes
to
i
m
p
r
o
v
i
n
g
th
eir
o
v
er
all
lear
n
i
n
g
ac
h
iev
e
m
en
ts
[
1
]
,
[
2
]
.
T
r
a
d
itio
n
al
m
eth
o
d
s
o
f
co
llectin
g
s
tu
d
en
t
f
ee
d
b
ac
k
,
s
u
ch
as
s
u
r
v
ey
s
o
r
q
u
esti
o
n
n
air
es,
f
r
eq
u
e
n
tly
ex
h
ib
it
n
o
te
wo
r
th
y
lim
itatio
n
s
.
T
h
is
ap
p
r
o
ac
h
is
o
f
te
n
p
er
ce
i
v
ed
as
in
ef
f
icien
t,
lack
in
g
tr
a
n
s
p
ar
en
cy
,
an
d
ten
d
s
to
y
ield
a
r
estricte
d
q
u
a
n
tity
o
f
f
ee
d
b
ac
k
r
esp
o
n
s
es.
As
a
s
o
lu
tio
n
to
th
ese
ch
allen
g
es,
an
“
E
va
lu
a
s
i
Do
s
en
o
leh
Ma
h
a
s
is
w
a
”
(
E
DOM
)
s
y
s
tem
was
d
ev
elo
p
ed
.
T
h
e
E
DOM
s
y
s
tem
i
s
an
in
f
o
r
m
atio
n
s
y
s
tem
s
p
ec
if
ically
d
esig
n
ed
to
co
llect
s
tu
d
en
t
f
ee
d
b
ac
k
e
f
f
icien
tly
an
d
ef
f
e
ctiv
ely
at
Un
iv
er
s
itas
R
iau
(
UNRI)
.
Alth
o
u
g
h
th
e
u
tili
za
t
io
n
o
f
th
e
E
DOM
s
y
s
tem
h
as
im
p
r
o
v
ed
th
e
p
r
o
ce
s
s
o
f
co
llectin
g
f
ee
d
b
ac
k
,
i
t
cu
r
r
en
tly
lack
s
th
e
ca
p
ab
ilit
y
to
au
to
m
atica
lly
an
aly
ze
s
tu
d
en
t
co
m
m
en
ts
.
W
ith
in
th
e
E
DOM
s
y
s
tem
,
d
ata
is
co
llected
th
r
o
u
g
h
a
f
o
r
m
f
ea
tu
r
in
g
a
L
ik
er
t
s
ca
le,
w
ith
an
ad
d
itio
n
al
s
e
ctio
n
f
o
r
co
m
m
en
ts
lo
ca
ted
at
th
e
b
o
tto
m
o
f
th
e
f
o
r
m
.
Stan
d
ar
d
d
atab
ase
p
r
o
ce
s
s
in
g
ca
n
b
e
u
s
ed
to
d
e
r
iv
e
r
esu
lts
f
r
o
m
th
e
L
ik
e
r
t
s
ca
le
v
alu
es.
No
n
eth
eless
,
th
e
d
if
f
icu
lty
ar
is
es
in
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
In
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
3
6
,
No
.
2
,
No
v
em
b
er
20
24
:
1
1
2
7
-
1
1
3
9
1128
co
m
m
en
ts
s
ec
tio
n
,
wh
er
e
ea
ch
s
u
r
v
ey
p
r
o
d
u
ce
s
ten
s
o
f
th
o
u
s
an
d
s
o
f
co
m
m
en
ts
th
at
r
eq
u
ir
e
m
an
u
al
ex
am
in
atio
n
.
T
h
is
p
r
esen
ts
a
s
u
b
s
tan
tial
ch
allen
g
e,
as
a
n
aly
zin
g
s
u
ch
a
s
u
b
s
tan
tial
v
o
lu
m
e
o
f
co
m
m
e
n
ts
ca
n
b
e
tim
e
-
co
n
s
u
m
in
g
an
d
r
eso
u
r
ce
-
in
ten
s
iv
e.
T
h
e
r
eso
lu
tio
n
to
th
is
is
s
u
e
in
v
o
lv
es
th
e
ap
p
licatio
n
o
f
a
f
ac
e
t
with
in
th
e
d
o
m
ai
n
o
f
n
atu
r
al
l
an
g
u
ag
e
p
r
o
ce
s
s
in
g
(
NL
P),
s
p
ec
if
ically
s
en
tim
en
t a
n
aly
s
is
[
3
]
-
[
7
]
.
T
h
e
p
u
r
p
o
s
e
o
f
th
is
p
ap
e
r
is
to
tack
le
th
e
h
u
r
d
les
ass
o
ciate
d
with
em
p
lo
y
in
g
s
en
tim
en
t
a
n
aly
s
is
f
o
r
th
e
ex
am
in
atio
n
o
f
s
tu
d
en
t
f
ee
d
b
ac
k
b
y
im
p
lem
e
n
tin
g
tr
a
n
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
els.
B
y
ef
f
icien
tly
an
aly
zin
g
th
e
d
ata,
th
e
r
esu
l
ts
o
f
th
is
an
aly
s
is
ar
e
ex
p
ec
ted
to
s
ig
n
if
ican
tly
c
o
n
tr
ib
u
t
e
to
in
s
titu
tio
n
s
in
im
p
r
o
v
in
g
s
er
v
ice
q
u
ality
f
o
r
s
tu
d
en
ts
.
W
h
ile
ea
r
lier
s
tu
d
ies
h
av
e
ex
p
lo
r
ed
th
e
im
p
ac
t
o
f
s
en
tim
en
t
an
aly
s
is
tech
n
iq
u
es
o
n
e
v
alu
atin
g
s
tu
d
en
t
f
ee
d
b
ac
k
,
t
h
ey
h
av
e
n
o
t
ex
p
licitly
ad
d
r
ess
ed
th
e
ef
f
ec
tiv
en
ess
a
n
d
ap
p
licab
ilit
y
o
f
t
r
an
s
f
o
r
m
er
-
b
ased
m
o
d
els
in
th
is
co
n
tex
t
.
T
o
ev
alu
ate
th
e
ca
p
ab
ilit
ies
o
f
th
ese
m
o
d
els,
we
will
f
in
e
-
tu
n
e
p
r
e
-
tr
ain
ed
tr
an
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
els,
in
clu
d
in
g
au
to
-
r
eg
r
ess
iv
e
m
o
d
els
lik
e
g
en
er
ativ
e
p
r
e
-
tr
ain
ed
tr
an
s
f
o
r
m
er
(
GPT
-
2
I
n
d
o
n
esia)
,
a
n
d
th
e
au
t
o
en
co
d
er
a
r
ch
itect
u
r
e
o
f
m
u
ltil
in
g
u
al
b
i
d
i
r
e
c
t
i
o
n
a
l
e
n
c
o
d
e
r
r
e
p
r
e
s
e
n
t
a
t
i
o
n
s
f
r
o
m
t
r
a
n
s
f
o
r
m
e
r
s
(
M
B
E
R
T
)
[
8
]
,
I
n
d
o
B
E
R
T
,
a
n
d
R
o
B
E
R
T
a
I
n
d
o
n
e
s
i
a
[
9
]
.
We
will
ev
alu
ate
th
e
p
er
f
o
r
m
an
ce
o
f
ea
ch
m
o
d
el
u
s
in
g
a
s
tu
d
en
t
f
ee
d
b
ac
k
d
ataset
o
b
tain
ed
f
r
o
m
t
h
e
E
DOM
s
y
s
tem
.
T
h
e
d
ataset
clas
s
if
ies
ea
ch
f
ee
d
b
ac
k
in
to
two
m
ain
s
en
tim
en
t
ca
teg
o
r
ies:
p
o
s
itiv
e
an
d
n
eg
ativ
e.
W
e
will
g
au
g
e
th
e
ef
f
icien
c
y
o
f
t
h
ese
tr
an
s
f
o
r
m
er
-
b
ased
lan
g
u
a
g
e
m
o
d
els b
y
em
p
lo
y
in
g
well
-
e
s
tab
lis
h
ed
r
esear
ch
m
etr
ics,
in
clu
d
in
g
ac
cu
r
ac
y
,
F1
-
s
co
r
e,
p
r
ec
is
io
n
,
a
n
d
r
ec
a
ll.
T
h
is
p
ap
e
r
is
o
r
g
an
ize
d
i
n
to
th
e
f
o
llo
win
g
s
ec
tio
n
s
:
s
ec
tio
n
2
o
u
tlin
es
r
e
ce
n
t
wo
r
k
s
;
s
ec
ti
o
n
3
p
r
esen
ts
o
u
r
p
r
o
p
o
s
ed
m
eth
o
d
o
lo
g
y
;
s
ec
tio
n
4
d
elib
er
ates
o
n
th
e
e
x
p
er
im
en
tal
o
u
tco
m
es,
an
d
last
ly
,
s
ec
tio
n
5
p
r
esen
ts
o
u
r
co
n
clu
s
io
n
.
2.
RE
C
E
NT
WO
RK
T
r
an
s
f
o
r
m
e
r
-
b
ased
m
o
d
els
h
av
e
g
ain
ed
s
ig
n
if
ican
t
p
r
o
m
i
n
en
ce
in
r
ec
en
t
y
ea
r
s
o
win
g
to
th
eir
ca
p
ac
ity
to
ca
p
tu
r
e
th
e
co
n
tex
tu
al
an
d
em
o
tio
n
al
in
tr
icac
ies
o
f
tex
t
[
1
0
]
-
[
1
2
]
.
T
r
an
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
els
r
ep
r
esen
t
a
ca
teg
o
r
y
o
f
d
ee
p
lear
n
i
n
g
m
o
d
els
th
at
ex
h
ib
it
ef
f
ec
tiv
en
ess
ac
r
o
s
s
a
r
an
g
e
o
f
NL
P
task
s
,
in
clu
d
in
g
s
en
tim
en
t
an
al
y
s
is
.
T
h
ese
m
o
d
els
in
itiall
y
g
r
asp
th
e
co
n
tex
t
o
f
a
s
en
ten
ce
an
d
s
u
b
s
eq
u
en
tly
em
p
lo
y
th
is
co
n
tex
tu
al
u
n
d
e
r
s
tan
d
in
g
to
f
o
r
ec
ast
th
e
s
en
tim
en
t
o
f
th
e
s
en
ten
ce
.
T
r
an
s
f
o
r
m
er
-
b
ased
m
o
d
els
h
av
e
s
u
r
p
ass
ed
ea
r
lier
s
tate
-
of
-
th
e
-
ar
t
m
o
d
els
in
p
er
f
o
r
m
an
ce
,
an
d
n
u
m
e
r
o
u
s
c
o
n
tem
p
o
r
ar
y
p
io
n
ee
r
in
g
m
o
d
els
ar
e
b
u
ilt
u
p
o
n
th
ei
r
f
o
u
n
d
atio
n
[
1
3
]
.
T
h
e
p
r
im
ar
y
aim
o
f
s
en
ti
m
en
t
an
aly
s
is
is
to
id
en
tify
,
an
aly
ze
,
an
d
ex
t
r
ac
t
em
o
tio
n
al
s
tates,
r
ea
ctio
n
s
,
o
r
s
en
tim
en
ts
co
n
v
ey
e
d
in
tex
tu
al
d
ata.
Ov
er
r
ec
e
n
t
y
ea
r
s
,
th
er
e
h
as
b
ee
n
a
n
in
cr
ea
s
in
g
in
ter
est in
em
p
lo
y
i
n
g
s
en
tim
en
t a
n
aly
s
is
to
au
to
m
atica
lly
ass
e
s
s
s
tu
d
en
t r
ev
iews
[
1
]
,
[
2
]
,
[
1
4
]
-
[
1
7
]
.
T
h
is
is
b
ec
au
s
e
s
en
tim
en
t
an
aly
s
is
ca
n
as
s
is
t
u
n
iv
er
s
itie
s
i
n
id
en
tify
in
g
tr
en
d
s
an
d
p
atter
n
s
with
in
s
tu
d
en
t
f
ee
d
b
ac
k
,
wh
ich
ca
n
s
u
b
s
eq
u
e
n
tly
b
e
u
tili
ze
d
to
e
n
h
an
ce
tea
ch
in
g
a
n
d
lear
n
in
g
.
Fo
r
in
s
tan
ce
,
in
s
titu
tio
n
s
m
ay
u
n
co
v
e
r
a
co
n
s
is
ten
t
p
o
s
itiv
e
s
en
tim
en
t
r
eg
ar
d
i
n
g
p
ar
ticu
lar
teac
h
in
g
m
eth
o
d
s
o
r
a
r
ec
u
r
r
in
g
n
e
g
ativ
e
s
en
tim
en
t
ass
o
ciate
d
with
a
s
p
ec
if
ic
co
u
r
s
e
elem
e
n
t.
Ar
m
e
d
with
th
is
in
f
o
r
m
ati
o
n
,
th
e
u
n
iv
er
s
ity
ca
n
m
a
k
e
d
ata
-
d
r
iv
e
n
d
ec
is
io
n
s
to
im
p
r
o
v
e
teac
h
in
g
a
n
d
lear
n
in
g
.
I
f
p
o
s
itiv
e
s
en
tim
en
ts
ar
e
ass
o
ciate
d
with
p
ar
ticu
la
r
teac
h
in
g
a
p
p
r
o
ac
h
es,
th
o
s
e
m
e
th
o
d
s
ca
n
b
e
s
tr
en
g
th
e
n
ed
an
d
ap
p
lied
in
o
th
er
co
u
r
s
es.
C
o
n
v
er
s
ely
,
if
n
eg
ati
v
e
s
en
tim
en
ts
h
ig
h
lig
h
t
r
ec
u
r
r
i
n
g
ch
allen
g
es,
f
o
c
u
s
ed
en
h
an
c
em
en
ts
ca
n
b
e
in
tr
o
d
u
ce
d
to
tack
le
th
o
s
e
is
s
u
es,
u
ltima
tely
en
h
an
ci
n
g
th
e
o
v
e
r
all
ed
u
ca
tio
n
al
e
x
p
er
ien
ce
f
o
r
s
tu
d
en
ts
.
No
n
eth
eless
,
im
p
le
m
en
tin
g
s
en
tim
en
t
an
aly
s
is
with
in
th
e
f
r
am
ewo
r
k
o
f
an
aly
zi
n
g
s
tu
d
en
t
f
ee
d
b
a
ck
p
r
esen
ts
its
o
w
n
ar
r
ay
o
f
c
h
allen
g
es.
E
x
is
tin
g
s
en
tim
en
t
an
aly
s
is
tech
n
iq
u
es
,
s
u
ch
as
Glo
Ve
an
d
W
o
r
d
2
Vec
em
b
ed
d
in
g
m
o
d
els
[
1
5
]
-
[
2
0
]
,
o
f
ten
o
v
er
l
o
o
k
th
e
s
en
tim
en
tal
an
d
co
n
tex
t
u
a
l
n
u
an
ce
s
o
f
th
e
tex
t.
T
h
ese
m
o
d
els
n
ec
ess
itate
ex
ten
s
iv
e
t
r
ain
in
g
o
n
v
ast
tex
t
co
r
p
o
r
a
to
p
r
o
d
u
ce
p
r
ec
is
e
wo
r
d
v
ec
to
r
s
an
d
m
a
y
o
m
it
o
u
t
o
f
v
o
ca
b
u
lar
y
wo
r
d
s
(
OOV)
,
lead
in
g
to
in
f
o
r
m
atio
n
lo
s
s
.
Mo
r
eo
v
er
,
th
e
lim
ited
av
ailab
ilit
y
o
f
p
r
e
-
lab
eled
d
ata
an
d
th
e
p
o
s
s
ib
ilit
y
o
f
in
co
n
s
is
ten
cies
b
etwe
en
r
ev
iews a
n
d
t
h
eir
ass
ig
n
ed
lab
els ca
n
lea
d
to
m
is
cla
s
s
if
icatio
n
[
2
1
]
.
O
n
e
o
f
t
h
e
m
o
s
t
w
i
d
e
l
y
a
d
o
p
t
e
d
t
r
a
n
s
f
o
r
m
e
r
-
b
a
s
e
d
l
a
n
g
u
a
g
e
m
o
d
e
l
s
f
o
r
s
e
n
t
i
m
e
n
t
a
n
a
l
y
s
i
s
i
s
B
E
R
T
[
2
2
]
.
B
E
R
T
d
em
o
n
s
tr
ates
r
em
ar
k
ab
le
ef
f
ec
tiv
en
ess
in
s
en
tim
en
t
an
aly
s
is
,
ev
en
wh
en
tr
ain
ed
o
n
r
elativ
ely
s
m
all
d
atasets
[
2
2
]
-
[
2
4
]
.
An
o
t
h
er
p
r
o
m
in
en
t
tr
a
n
s
f
o
r
m
e
r
-
b
ased
la
n
g
u
ag
e
m
o
d
el
is
GPT,
alth
o
u
g
h
it
is
n
o
t
ex
p
licitly
tailo
r
ed
f
o
r
s
en
tim
en
t
an
aly
s
is
.
Ho
wev
er
,
in
th
is
r
esear
ch
,
we
in
ten
d
to
in
co
r
p
o
r
ate
G
PT
-
2
to
ass
ess
its
ef
f
icac
y
in
s
en
tim
en
t
a
n
aly
s
is
.
Sev
er
al
o
th
er
tr
an
s
f
o
r
m
er
-
b
ased
lan
g
u
a
g
e
m
o
d
els
h
av
e
b
ee
n
p
u
t
f
o
r
t
h
f
o
r
s
en
tim
en
t
an
aly
s
is
,
in
clu
d
in
g
R
o
B
E
R
T
a
[
2
5
]
,
XL
NE
T
[
2
6
]
,
AL
B
E
R
T
[
2
7
]
,
a
n
d
E
R
NI
E
[
2
8
]
.
No
n
eth
eless
,
th
e
r
esear
ch
ce
n
ter
s
its
atten
tio
n
o
n
m
o
d
els ap
p
licab
le
to
th
e
I
n
d
o
n
esian
lan
g
u
ag
e
.
3.
M
E
T
H
O
D
T
h
e
m
eth
o
d
s
ch
ap
te
r
d
elin
ea
t
es
th
e
s
eq
u
en
tial
p
r
o
ce
d
u
r
es
u
n
d
er
tak
en
in
t
h
is
r
esear
ch
,
c
o
m
m
en
cin
g
with
d
ata
co
llectio
n
an
d
p
r
o
g
r
ess
in
g
th
r
o
u
g
h
d
ata
p
r
e
-
p
r
o
c
ess
in
g
,
d
ata
lab
elin
g
,
d
ata
s
p
l
itti
n
g
,
f
in
e
tu
n
in
g
,
cu
lm
in
atin
g
in
m
o
d
el
p
e
r
f
o
r
m
an
ce
ev
alu
ati
o
n
.
E
ac
h
o
f
t
h
ese
s
tag
es
h
o
ld
s
s
ig
n
if
ican
ce
in
g
u
a
r
an
teein
g
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
S
en
timen
t a
n
a
lysi
s
o
f stu
d
en
t e
va
lu
a
tio
n
fee
d
b
a
ck
u
s
in
g
tr
a
n
s
fo
r
mer …
(
I
b
n
u
Da
q
i
q
il I
D)
1129
ef
f
icien
cy
an
d
p
r
ec
is
io
n
o
f
t
h
e
co
n
s
tr
u
cted
m
o
d
el.
T
h
e
c
o
m
p
r
eh
e
n
s
iv
e
d
ep
ictio
n
o
f
th
is
en
tire
p
r
o
ce
s
s
is
elu
cid
ated
in
Fig
u
r
e
1
,
o
f
f
er
i
n
g
a
v
is
u
al
r
ep
r
esen
tatio
n
o
f
th
e
r
esear
ch
wo
r
k
f
lo
w.
Fig
u
r
e
1
.
R
esear
ch
s
tag
es
3
.
1
.
Da
t
a
c
o
llect
io
n
I
n
th
is
r
esear
ch
,
d
ata
was
g
ath
er
ed
f
r
o
m
t
h
e
lectu
r
e
r
ev
al
u
atio
n
b
y
s
tu
d
e
n
t’
s
s
u
r
v
ey
d
ata,
wh
ich
was
s
o
u
r
ce
d
f
r
o
m
th
e
e
v
alu
atio
n
in
f
o
r
m
atio
n
s
y
s
tem
at
UN
R
I
.
T
h
e
co
llected
d
ata
is
in
th
e
f
o
r
m
o
f
te
x
t,
en
co
m
p
ass
in
g
s
tu
d
en
t
co
m
m
en
ts
r
eg
ar
d
in
g
lect
u
r
er
s
an
d
lectu
r
e
ac
tiv
ities
.
Su
b
s
eq
u
en
tly
,
th
is
d
ata
was
tr
an
s
f
o
r
m
ed
in
to
C
SV f
o
r
m
at
t
o
f
ac
ilit
ate
s
u
b
s
eq
u
en
t p
r
o
ce
s
s
in
g
an
d
an
aly
s
is
.
Data
co
llectio
n
to
o
k
p
lace
b
etwe
en
2
0
2
1
an
d
2
0
2
3
,
en
co
m
p
ass
in
g
in
p
u
ts
f
r
o
m
all
f
ac
u
lties
at
UN
R
I
.
T
h
e
to
tal
am
o
u
n
t
o
f
co
llect
ed
d
ata
am
o
u
n
te
d
to
1
1
,
6
4
5
co
m
m
en
ts
.
Nev
er
th
eless
,
th
e
d
ata
co
n
tain
s
a
co
n
s
id
er
ab
le
am
o
u
n
t
o
f
n
o
is
e
,
in
clu
d
in
g
ir
r
elev
an
t
c
o
m
m
e
n
ts
,
ex
ce
s
s
iv
ely
b
r
ief
r
em
ar
k
s
,
an
d
s
en
s
itiv
e
o
r
p
r
iv
ate
in
f
o
r
m
atio
n
s
u
ch
as
th
e
lectu
r
er
’
s
n
am
e,
co
u
r
s
e,
an
d
lo
ca
tio
n
.
T
h
is
n
o
is
e
h
as
th
e
p
o
ten
tial
to
im
p
ac
t
th
e
clar
ity
an
d
q
u
ality
o
f
th
e
a
n
aly
ze
d
d
ata,
n
ec
ess
itatin
g
d
at
a
-
clea
n
in
g
p
r
o
ce
d
u
r
es
to
en
s
u
r
e
th
at
th
e
an
aly
s
is
ce
n
ter
s
s
o
lely
o
n
p
e
r
tin
en
t a
n
d
v
alu
ab
le
in
f
o
r
m
atio
n
.
3
.
2
.
Da
t
a
l
a
be
lling
T
h
e
d
ata
la
b
elin
g
p
r
o
ce
d
u
r
e
was
co
n
d
u
cted
b
y
a
team
o
f
f
iv
e
n
ativ
e
I
n
d
o
n
esian
s
p
ea
k
e
r
s
.
T
h
e
C
SV
d
ata,
wh
ich
h
ad
b
ee
n
d
o
wn
lo
ad
ed
,
was
s
u
b
s
eq
u
en
tly
p
ar
titi
o
n
ed
in
to
f
iv
e
s
ep
ar
ate
d
o
cu
m
en
ts
,
en
s
u
r
in
g
th
at
ea
ch
in
d
i
v
id
u
al
was
ass
ig
n
e
d
d
is
tin
ct
co
m
m
en
ts
to
la
b
el.
Fu
r
th
er
m
o
r
e,
th
is
team
h
ad
th
e
ad
d
itio
n
al
r
esp
o
n
s
ib
ilit
y
o
f
p
r
e
-
f
ilter
in
g
th
e
d
ata
b
ef
o
r
e
th
e
lab
elin
g
p
r
o
ce
s
s
to
g
u
ar
an
tee
th
at
o
n
ly
p
er
tin
en
t
an
d
h
i
g
h
-
q
u
ality
d
ata
r
ec
eiv
ed
lab
els.
Prio
r
to
lab
elin
g
,
th
ey
wer
e
task
ed
with
id
en
tify
in
g
a
n
d
ex
clu
d
in
g
d
ata
th
at
lack
ed
s
u
b
s
tan
tial
in
f
o
r
m
ati
o
n
.
T
h
is
in
v
o
l
v
ed
f
ilter
in
g
o
u
t
s
h
o
r
t
co
m
m
e
n
ts
th
at
o
n
ly
co
n
s
is
ted
o
f
ac
k
n
o
wled
g
m
en
ts
o
r
g
en
e
r
ic
p
h
r
ases
th
at
d
id
n
o
t
co
n
tr
i
b
u
t
e
co
n
tex
t
o
r
a
d
d
itio
n
al
v
alu
e
f
o
r
th
e
p
u
r
p
o
s
e
o
f
d
ata
an
aly
s
is
.
As
a
r
esu
lt,
t
h
is
en
d
ea
v
o
r
n
o
t
o
n
l
y
e
n
h
an
ce
d
th
e
q
u
ality
o
f
th
e
lab
ele
d
d
ataset
b
u
t
als
o
im
p
r
o
v
e
d
th
e
e
f
f
icien
cy
o
f
th
e
s
u
b
s
eq
u
en
t
m
a
ch
in
e
lear
n
in
g
p
r
o
ce
s
s
,
g
u
ar
an
teei
n
g
th
at
th
e
m
o
d
el
wo
u
ld
b
e
tr
ain
ed
o
n
g
en
u
in
ely
m
ea
n
in
g
f
u
l a
n
d
r
e
p
r
esen
tativ
e
d
ata.
T
h
e
to
tal
q
u
an
tity
o
f
d
ata
s
u
cc
ess
f
u
lly
lab
eled
am
o
u
n
ted
to
5
,
7
8
3
co
m
m
en
ts
.
Am
o
n
g
th
e
s
e,
2
,
8
4
9
wer
e
p
o
s
itiv
e
r
ev
iews,
an
d
2
,
9
3
4
wer
e
n
eg
ativ
e
r
e
v
iews.
Fig
u
r
e
2
illu
s
tr
ates
th
at
with
in
th
e
E
DOM
s
y
s
tem
,
p
o
s
itiv
e
r
ev
iews
f
r
o
m
s
tu
d
en
t
s
co
n
s
titu
te
5
0
.
7
3
%
o
f
th
e
to
tal,
wh
er
ea
s
n
e
g
ativ
e
r
e
v
iews
m
ak
e
u
p
4
9
.
2
7
%.
T
h
is
in
d
icate
s
a
n
ea
r
ly
b
alan
c
ed
d
is
tr
ib
u
tio
n
b
etwe
en
p
o
s
itiv
e
an
d
n
eg
ativ
e
r
ev
iews
with
in
th
e
clea
n
e
d
d
ata.
Fig
u
r
e
2
s
h
o
ws th
e
p
r
o
p
o
r
tio
n
o
f
p
o
s
itiv
e
an
d
n
e
g
ativ
e
d
ata
.
3
.
3
.
Da
t
a
v
er
if
ica
t
io
n
Af
ter
co
m
p
letin
g
th
e
lab
elin
g
p
r
o
ce
s
s
,
th
e
s
u
b
s
eq
u
en
t
s
t
ep
in
v
o
lv
es
v
e
r
if
y
in
g
th
e
la
b
eled
d
ata.
T
h
is
s
tag
e
h
o
ld
s
s
ig
n
if
ican
t
im
p
o
r
tan
ce
in
g
u
ar
an
teein
g
th
e
p
r
ec
is
io
n
a
n
d
c
o
n
s
is
ten
cy
o
f
th
e
d
ata.
Du
r
in
g
t
h
e
v
er
if
icatio
n
p
r
o
ce
s
s
,
a
d
is
tin
ct
team
ev
alu
ates
th
e
lab
ele
d
d
ata
s
am
p
les.
T
h
ey
m
eticu
lo
u
s
ly
ass
ess
ea
ch
ass
ig
n
ed
lab
el,
c
o
n
f
ir
m
i
n
g
its
ap
p
r
o
p
r
iaten
ess
with
in
th
e
g
iv
en
co
n
tex
t.
Data
th
at
r
aises
d
o
u
b
ts
o
r
ex
h
ib
its
in
co
n
s
is
ten
cies
in
lab
elin
g
u
n
d
er
g
o
es
s
cr
u
tin
y
an
d
,
if
d
ee
m
ed
n
ec
es
s
ar
y
,
u
n
d
er
g
o
es
r
e
-
la
b
elin
g
to
m
ain
tain
d
ata
in
teg
r
ity
.
T
h
is
v
er
if
icatio
n
p
r
o
ce
d
u
r
e
n
o
t
o
n
ly
en
h
an
ce
s
th
e
d
ataset
’
s
q
u
ality
b
u
t
also
aid
s
in
id
en
tify
in
g
ar
ea
s
th
at
m
ig
h
t
n
ec
ess
itate
im
p
r
o
v
em
e
n
ts
in
lab
elin
g
g
u
id
elin
es
o
r
ad
d
itio
n
al
tr
ain
in
g
f
o
r
th
e
lab
eler
s
.
T
h
e
u
ltima
te
o
b
jectiv
e
is
to
g
en
er
ate
a
d
ep
en
d
ab
le
d
ataset,
r
ea
d
y
f
o
r
u
tili
za
tio
n
in
tr
ain
in
g
m
ac
h
in
e
lear
n
i
n
g
m
o
d
els,
with
a
h
ig
h
lev
el
o
f
c
o
n
f
id
en
ce
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
In
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
3
6
,
No
.
2
,
No
v
em
b
er
20
24
:
1
1
2
7
-
1
1
3
9
1130
Fig
u
r
e
2
.
P
r
o
p
o
r
tio
n
o
f
p
o
s
itiv
e
an
d
n
e
g
ativ
e
d
ata
3
.
4
.
Da
t
a
p
re
-
pro
ce
s
s
ing
Data
p
r
e
-
p
r
o
ce
s
s
in
g
en
c
o
m
p
ass
es
s
ev
er
al
ess
en
tial
task
s
,
in
clu
d
in
g
d
ata
clea
n
in
g
,
to
k
en
izatio
n
,
cr
ea
tin
g
to
k
e
n
-
to
-
I
D
m
ap
p
i
n
g
s
,
in
tr
o
d
u
cin
g
s
p
ec
ial
to
k
e
n
s
,
an
d
p
e
r
f
o
r
m
in
g
p
ad
d
i
n
g
an
d
tr
u
n
ca
tio
n
,
all
o
f
wh
ich
ar
e
ca
r
r
ied
o
u
t
b
ef
o
r
e
f
in
e
-
tu
n
in
g
a
n
d
tr
ain
in
g
th
e
m
o
d
el.
Fig
u
r
e
3
p
r
o
v
i
d
es
a
v
is
u
al
r
ep
r
esen
tatio
n
o
f
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
s
tep
s
.
Fig
u
r
e
3
.
Pre
-
p
r
o
ce
s
s
in
g
illu
s
tr
atio
n
I
n
th
is
r
esear
ch
,
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
s
tag
es a
r
e
d
elin
ea
ted
as f
o
llo
ws:
a)
Data
clea
n
in
g
an
d
to
k
en
is
atio
n
.
T
h
e
in
itial
s
tep
in
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
p
h
ase
en
co
m
p
ass
e
s
d
ata
clea
n
in
g
task
s
,
s
u
ch
as
th
e
r
em
o
v
al
o
f
ex
tr
an
e
o
u
s
s
y
m
b
o
ls
,
r
ec
ti
f
icatio
n
o
f
s
p
ellin
g
m
is
tak
es,
co
n
v
er
s
io
n
o
f
ab
b
r
ev
iatio
n
s
to
th
eir
e
x
p
an
d
ed
f
o
r
m
s
,
an
d
an
o
n
y
m
izatio
n
o
f
n
am
es
o
f
in
d
iv
id
u
als,
lo
ca
t
io
n
s
,
o
r
en
titi
es
th
r
o
u
g
h
th
e
u
tili
za
tio
n
o
f
s
u
b
s
titu
tes
lik
e
‘
XYZ
’
.
Su
b
s
eq
u
en
t
to
th
e
clea
n
in
g
p
r
o
ce
s
s
,
to
k
e
n
izatio
n
o
f
th
e
d
o
cu
m
e
n
t
is
ex
ec
u
ted
.
D
u
r
in
g
to
k
en
izatio
n
,
th
e
in
p
u
t
tex
t
is
d
is
s
ec
ted
in
to
to
k
en
s
,
wh
ic
h
ca
n
en
co
m
p
ass
co
m
p
lete
wo
r
d
s
,
s
u
b
-
wo
r
d
s
,
o
r
in
d
iv
id
u
al
ch
a
r
ac
ter
s
.
Fo
r
t
h
e
p
u
r
p
o
s
es
o
f
t
h
is
r
esear
ch
,
t
h
e
t
ra
n
s
f
o
r
m
er
-
b
ased
m
o
d
el
ad
o
p
ts
th
e
W
o
r
d
Piece
to
k
en
izatio
n
ap
p
r
o
ac
h
to
s
eg
m
en
t
th
e
tex
t.
T
ab
le
1
s
h
o
ws
an
ex
am
p
le
o
f
d
ata
clea
n
s
in
g
r
esu
lts
.
I
n
th
e
ex
am
p
le,
we
ce
n
s
o
r
th
e
n
am
e
o
f
th
e
class
an
d
lectu
r
er
an
d
f
ix
s
o
m
e
ty
p
o
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
S
en
timen
t a
n
a
lysi
s
o
f stu
d
en
t e
va
lu
a
tio
n
fee
d
b
a
ck
u
s
in
g
tr
a
n
s
fo
r
mer …
(
I
b
n
u
Da
q
i
q
il I
D)
1131
an
d
g
r
a
m
m
ar
er
r
o
r
s
in
th
e
s
en
ten
ce
.
T
h
e
r
esu
ltan
t
clea
n
e
d
an
d
a
n
n
o
tated
d
ataset
is
ac
ce
s
s
ib
le
in
o
u
r
r
ep
o
s
ito
r
y
[
2
5
]
.
b)
Ad
d
s
p
ec
ial
to
k
en
s
.
W
ith
in
th
e
B
E
R
T
ar
ch
itectu
r
e,
th
e
in
co
r
p
o
r
atio
n
o
f
s
p
ec
ial
to
k
en
s
h
o
ld
s
s
ig
n
if
ican
t
im
p
o
r
tan
ce
in
c
o
n
v
e
y
in
g
s
tr
u
ctu
r
al
in
f
o
r
m
atio
n
to
th
e
m
o
d
el.
No
tab
ly
,
two
s
u
ch
to
k
en
s
ar
e
“
C
L
S
”
an
d
“
SEP.
”
T
h
e
“
[
C
L
S]
”
to
k
en
,
al
s
o
k
n
o
wn
as
th
e
class
if
icatio
n
to
k
en
,
is
p
o
s
itio
n
ed
at
t
h
e
in
c
ep
tio
n
o
f
ev
er
y
in
p
u
t
s
eq
u
en
ce
.
I
t
s
er
v
es
as
a
f
o
ca
l
p
o
in
t
f
o
r
co
n
s
o
lid
ati
n
g
th
e
co
m
p
r
eh
e
n
s
iv
e
r
ep
r
es
en
tatio
n
o
f
th
e
s
eq
u
en
ce
,
p
ar
ticu
lar
l
y
in
task
s
r
elate
d
to
class
if
icatio
n
.
T
h
e
“
[
SEP]
”
to
k
en
,
r
ef
er
r
ed
to
as
th
e
s
ep
ar
ato
r
to
k
en
,
ass
u
m
es
th
e
r
o
le
o
f
e
n
ab
lin
g
th
e
m
o
d
el
to
d
if
f
er
e
n
tiate
b
etwe
en
s
eg
m
en
ts
with
in
th
e
s
eq
u
en
ce
.
T
h
is
d
if
f
er
en
tiatio
n
p
r
o
v
es
cr
u
cial
in
s
ce
n
ar
io
s
in
v
o
lv
in
g
d
is
tin
ct
tex
t
f
r
ag
m
e
n
ts
o
r
s
ig
n
if
y
in
g
th
e
co
n
clu
s
io
n
o
f
a
s
eq
u
e
n
ce
.
c)
Pad
d
in
g
,
a
n
d
tr
u
n
ca
tio
n
p
ad
d
in
g
an
d
tr
u
n
c
atio
n
r
ep
r
esen
t
in
d
is
p
en
s
ab
le
p
r
o
ce
d
u
r
es
in
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
o
f
tex
t
d
ata
f
o
r
th
e
B
E
R
T
m
o
d
el,
in
v
o
l
v
in
g
th
e
in
c
o
r
p
o
r
a
tio
n
o
f
[
PAD]
to
k
en
s
.
Pad
d
i
n
g
in
v
o
l
v
es
th
e
ad
d
itio
n
o
f
[
PAD]
to
k
e
n
s
to
s
h
o
r
ter
s
eq
u
en
ce
s
,
th
er
e
b
y
eq
u
alizin
g
th
eir
len
g
th
with
th
e
lo
n
g
es
t
s
eq
u
en
ce
in
th
e
b
atch
.
T
h
is
e
n
s
u
r
es
t
h
e
m
o
d
el
’
s
ab
ilit
y
to
ef
f
icie
n
tly
p
r
o
ce
s
s
th
e
d
ata
with
in
a
s
in
g
le
b
atch
.
T
r
u
n
ca
tio
n
s
er
v
es
th
e
p
u
r
p
o
s
e
o
f
r
ed
u
cin
g
th
e
le
n
g
th
o
f
s
eq
u
en
ce
s
th
at
s
u
r
p
ass
th
e
m
o
d
el
’
s
m
ax
im
u
m
allo
wab
le
lim
it,
ef
f
ec
tiv
ely
tr
im
m
in
g
th
e
te
x
t
wh
ile
p
r
eser
v
in
g
th
e
f
i
x
ed
d
im
e
n
s
io
n
ality
.
No
tab
ly
,
t
h
e
[
PAD]
to
k
en
s
a
r
e
in
te
n
tio
n
all
y
en
g
in
ee
r
ed
s
o
as
n
o
t
to
im
p
ac
t
th
e
m
o
d
el
’
s
an
aly
tical
o
u
tco
m
es.
T
h
e
y
en
ab
le
th
e
m
o
d
el
to
d
is
r
eg
ar
d
th
e
p
ad
d
ed
s
ec
tio
n
s
an
d
c
o
n
ce
n
tr
ate
s
o
lely
o
n
th
e
s
u
b
s
tan
tiv
e
co
n
te
n
t
co
n
tain
ed
with
in
th
e
s
eq
u
en
c
es.
T
h
is
p
r
o
ce
d
u
r
al
ap
p
r
o
ac
h
s
tr
ik
es
a
b
alan
ce
b
etwe
en
p
r
eser
v
in
g
cr
u
cial
co
n
tex
tu
al
in
f
o
r
m
atio
n
an
d
a
d
h
er
in
g
to
th
e
m
o
d
el
’
s
tech
n
ical
co
n
s
tr
ain
ts
,
en
s
u
r
in
g
th
at
o
n
ly
p
er
tin
en
t
in
f
o
r
m
atio
n
is
in
co
r
p
o
r
ated
a
n
d
p
r
o
ce
s
s
ed
b
y
B
E
R
T.
d)
T
o
k
en
to
I
D
m
ap
p
in
g
.
T
h
e
t
o
k
en
to
I
D
m
ap
p
i
n
g
p
r
o
ce
s
s
with
in
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
w
o
r
k
f
lo
w
f
o
r
th
e
B
E
R
T
m
o
d
el
s
tan
d
s
as
a
p
iv
o
tal
s
tag
e.
I
t
i
n
v
o
lv
es
th
e
c
o
n
v
er
s
io
n
o
f
to
k
en
s
d
e
r
iv
ed
f
r
o
m
th
e
o
r
i
g
in
al
tex
t
in
to
d
is
tin
ct
id
en
tific
atio
n
n
u
m
b
er
s
(
I
Ds).
Fo
llo
win
g
th
e
s
eg
m
en
tatio
n
o
f
th
e
tex
t
in
to
t
o
k
en
s
b
ased
o
n
p
r
ed
ef
in
e
d
to
k
e
n
izatio
n
r
u
les
(
s
u
ch
as
W
o
r
d
Piece
in
th
e
ca
s
e
o
f
B
E
R
T
)
,
ea
ch
in
d
i
v
id
u
al
t
o
k
en
u
n
d
er
g
o
es
tr
an
s
f
o
r
m
atio
n
in
t
o
an
I
D
b
ase
d
o
n
th
e
m
o
d
el
’
s
v
o
ca
b
u
lar
y
.
T
o
illu
s
tr
ate,
th
e
p
h
r
ase
“
Hello
W
o
r
ld
”
m
ig
h
t
b
e
r
ep
r
esen
ted
b
y
th
e
I
D
7
5
9
2
with
in
th
e
B
E
R
T
lex
ico
n
.
T
h
is
p
r
o
ce
d
u
r
e
tr
an
s
f
o
r
m
s
th
e
tex
tu
al
s
tr
u
ctu
r
e
in
to
an
ar
r
ay
o
f
n
u
m
er
ical
v
alu
es,
en
a
b
lin
g
th
e
m
o
d
e
l
to
h
a
n
d
le
tex
t
u
al
d
ata
as
n
u
m
e
r
ic
in
p
u
t.
C
o
n
s
eq
u
en
tly
,
a
s
en
ten
ce
li
k
e
“
Hello
W
o
r
ld
”
wo
u
ld
u
n
d
e
r
g
o
co
n
v
er
s
io
n
in
to
a
s
eq
u
en
ce
o
f
n
u
m
er
ic
I
Ds
(
e.
g
.
,
[
7
5
9
2
,
9
9
9
9
]
)
,
th
er
eb
y
f
ac
ilit
atin
g
th
e
m
o
d
el
in
its
an
aly
s
is
an
d
lear
n
in
g
f
r
o
m
th
e
d
ata.
T
h
is
m
ap
p
in
g
p
r
o
ce
s
s
g
u
ar
a
n
tees
th
at
th
e
tex
t
p
r
esen
ted
to
th
e
m
o
d
el
a
d
h
er
es
to
a
u
n
if
o
r
m
f
o
r
m
at,
u
ltim
ately
en
h
an
cin
g
th
e
e
f
f
icien
cy
an
d
p
r
ec
is
io
n
o
f
p
r
o
ce
s
s
in
g
.
T
ab
le
1
.
E
x
am
p
le
o
f
d
ata
clea
n
s
in
g
r
ec
o
r
d
[
1
]
B
e
f
o
r
e
c
l
e
a
n
s
i
n
g
[
2
]
A
f
t
e
r
c
l
e
a
n
s
i
n
g
[
3
]
U
n
t
u
k
k
e
l
a
s
{n
a
me
o
f
c
l
a
ss}
d
e
n
g
a
n
b
a
p
a
k
{n
a
me
o
f
l
e
c
t
u
r
e
r
}
sa
y
a
r
a
sa
k
u
ra
n
g
e
f
e
k
t
i
f
d
i
k
a
re
n
a
k
a
n
d
o
se
n
y
a
n
g
b
e
rs
a
n
g
k
u
t
a
n
g
j
a
r
a
n
g
m
a
s
u
k
k
e
l
a
s
d
a
n
u
n
t
u
k
p
e
n
g
e
rj
a
a
n
t
u
g
a
s
k
r
g
j
e
l
a
s
d
i
b
e
ri
k
a
n
seh
i
n
g
g
a
m
a
h
a
si
s
w
a
b
i
n
g
u
n
g
t
u
g
a
s
n
y
a
se
p
e
r
t
i
a
p
a
y
a
n
g
h
a
r
u
s
d
i
k
e
rj
a
k
a
n
[
4
]
U
n
t
u
k
k
e
l
a
s
XYZ
d
e
n
g
a
n
b
a
p
a
k
YXZ
sa
y
a
ra
s
a
k
u
r
a
n
g
e
f
e
k
t
i
f
d
i
k
a
re
n
a
k
a
n
d
o
se
n
y
a
n
g
b
e
rs
a
n
g
k
u
t
a
n
j
a
ra
n
g
m
a
su
k
k
e
l
a
s
d
a
n
u
n
t
u
k
p
e
n
g
e
rj
a
a
n
t
u
g
a
s
k
u
r
a
n
g
j
e
l
a
s
d
i
b
e
r
i
k
a
n
se
h
i
n
g
g
a
m
a
h
a
s
i
sw
a
b
i
n
g
u
n
g
t
u
g
a
s
n
y
a
s
e
p
e
rt
i
a
p
a
y
a
n
g
h
a
r
u
s
d
i
k
e
r
j
a
k
a
n
3
.
5
.
Da
t
a
s
pli
t
T
h
e
p
r
o
ce
s
s
ed
d
ata
u
n
d
e
r
g
o
e
s
d
iv
is
io
n
in
to
d
is
tin
ct
s
eg
m
e
n
ts
.
I
n
t
h
e
co
n
tex
t
o
f
t
h
is
r
esear
ch
,
th
e
d
ataset
is
p
ar
titi
o
n
ed
in
to
t
h
r
ee
p
r
im
ar
y
co
m
p
o
n
en
ts
:
tr
ain
in
g
d
ata,
v
alid
atio
n
d
ata
,
an
d
test
in
g
d
ata.
T
h
e
tr
ain
in
g
d
ata,
co
m
p
r
is
in
g
8
0
%
o
f
th
e
en
tire
d
ata
s
et,
is
allo
ca
ted
f
o
r
th
e
p
u
r
p
o
s
e
o
f
m
o
d
el
tr
ain
in
g
.
T
h
is
allo
ca
tio
n
en
ab
les
th
e
m
o
d
el
to
ac
q
u
ir
e
k
n
o
wled
g
e
an
d
ad
ap
t
to
th
e
u
n
d
er
ly
in
g
p
att
er
n
s
with
in
th
e
d
ata.
T
h
e
v
alid
atio
n
d
ataset,
co
m
p
r
is
in
g
1
0
%
o
f
th
e
en
tire
d
ata
s
et,
s
er
v
es
th
e
p
u
r
p
o
s
e
o
f
ass
ess
in
g
th
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
d
u
r
i
n
g
t
h
e
tr
ain
i
n
g
p
h
ase.
I
t
f
u
r
n
is
h
es
f
ee
d
b
a
ck
o
n
th
e
m
o
d
el
’
s
ef
f
icac
y
a
n
d
ass
is
ts
in
f
in
e
-
tu
n
in
g
m
o
d
el
p
ar
am
eter
s
to
m
itig
ate
o
v
er
f
itti
n
g
.
On
th
e
o
th
e
r
h
an
d
,
th
e
test
d
ataset,
also
r
e
p
r
esen
tin
g
1
0
%
o
f
th
e
to
tal
d
ata,
s
er
v
es
a
s
t
h
e
u
ltima
te
m
etr
ic
f
o
r
ev
alu
atin
g
th
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
o
n
p
r
ev
io
u
s
ly
u
n
s
ee
n
d
ata.
T
h
is
d
ataset
co
m
p
r
is
es
n
ew
d
ata
th
at
th
e
m
o
d
el
h
as
n
o
t
en
co
u
n
ter
ed
b
ef
o
r
e
.
T
h
is
d
iv
is
io
n
o
f
th
e
d
ataset
ad
h
er
es to
an
8
0
:1
0
:1
0
r
atio
,
a
llo
ca
tin
g
p
er
ce
n
ta
g
es f
o
r
t
r
ain
i
n
g
,
v
alid
atio
n
,
an
d
test
in
g
d
ata
,
r
esp
ec
tiv
ely
.
3
.
6
.
F
ine t
un
e
mo
del
I
n
th
is
r
esear
c
h
,
th
e
p
r
e
-
tr
ai
n
ed
m
o
d
el
a
r
ch
itectu
r
e
is
ta
ilo
r
ed
f
o
r
t
h
e
p
u
r
p
o
s
e
o
f
c
o
n
d
u
ctin
g
s
en
tim
en
t
an
aly
s
is
.
T
h
ese
a
d
ap
tatio
n
s
m
ay
en
co
m
p
ass
alter
atio
n
s
to
th
e
m
o
d
el
’
s
lay
er
s
,
in
clu
d
i
n
g
th
e
ad
d
itio
n
o
r
r
em
o
v
al
o
f
lay
er
s
,
with
th
e
o
b
jectiv
e
o
f
en
h
a
n
cin
g
th
e
m
o
d
el
’
s
ca
p
ac
ity
to
e
f
f
ec
tiv
ely
lear
n
f
r
o
m
an
d
p
r
o
ce
s
s
th
e
d
is
tin
ct
ch
ar
ac
ter
is
tics
o
f
th
e
cu
r
ated
d
ataset.
Sp
ec
if
ically
,
th
is
r
esear
ch
will
em
p
lo
y
f
o
u
r
p
r
e
-
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
In
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
3
6
,
No
.
2
,
No
v
em
b
er
20
24
:
1
1
2
7
-
1
1
3
9
1132
tr
ain
ed
m
o
d
els,
n
a
m
ely
MB
E
R
T
,
I
n
d
o
B
E
R
T
,
R
o
B
E
R
T
a
I
n
d
o
n
esia,
an
d
GPT
-
2
I
n
d
o
n
esia.
E
ac
h
o
f
th
ese
m
o
d
els
b
elo
n
g
s
to
th
e
ca
teg
o
r
y
o
f
tr
an
s
f
o
r
m
er
-
b
ased
m
o
d
els,
s
h
ar
in
g
id
en
tical
attr
ib
u
tes
in
ter
m
s
o
f
th
e
n
u
m
b
er
o
f
tr
a
n
s
f
o
r
m
e
r
lay
e
r
s
,
atten
tio
n
m
ec
h
a
n
is
m
s
,
an
d
h
id
d
en
em
b
e
d
d
in
g
s
ize.
A
d
etailed
co
m
p
a
r
is
o
n
am
o
n
g
t
h
e
p
r
e
-
tr
ain
ed
m
o
d
els
is
av
ailab
le
in
T
ab
le
2
.
T
h
e
f
in
e
-
t
u
n
in
g
p
r
o
ce
s
s
in
clu
d
ed
th
e
ca
lib
r
atio
n
o
f
h
y
p
er
p
ar
am
eter
s
f
o
r
th
e
tr
an
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
els,
aim
in
g
to
o
p
tim
ize
th
eir
p
er
f
o
r
m
an
ce
i
n
th
e
task
o
f
s
en
tim
en
t
an
aly
s
is
.
T
h
e
ad
ju
s
tm
en
t
o
f
h
y
p
er
p
ar
am
eter
s
was
g
u
id
ed
b
y
t
h
e
s
p
ec
if
ic
attr
ib
u
tes
o
f
th
e
s
tu
d
en
t
f
ee
d
b
ac
k
d
ata,
t
ak
in
g
in
t
o
ac
co
u
n
t
f
ac
to
r
s
lik
e
th
e
f
ee
d
b
ac
k
’
s
len
g
th
an
d
th
e
d
is
tr
ib
u
tio
n
o
f
p
o
s
itiv
e
an
d
n
e
g
ativ
e
r
e
v
iews.
T
h
e
h
y
p
e
r
p
ar
am
eter
s
tak
en
in
to
c
o
n
s
id
er
atio
n
f
o
r
t
h
e
p
r
o
p
o
s
ed
m
o
d
el
ar
e
as f
o
llo
ws:
−
T
h
e
b
atch
s
ize
was
s
et
to
3
2
.
T
h
e
b
atc
h
s
ize
d
en
o
tes
th
e
q
u
an
tity
o
f
tr
ain
i
n
g
e
x
am
p
les
em
p
lo
y
e
d
in
a
s
in
g
le
iter
atio
n
.
A
s
m
aller
b
at
ch
s
ize
was
s
elec
ted
to
p
r
o
m
o
te
th
e
m
o
d
el
’
s
ab
ilit
y
to
g
en
er
alize
ef
f
ec
tiv
ely
to
n
ew
,
u
n
s
ee
n
d
ata.
−
T
h
e
Ad
am
W
o
p
tim
izer
was
u
s
ed
,
wh
ich
is
a
v
a
r
ian
t
o
f
th
e
A
d
am
o
p
tim
izer
th
at
p
r
o
v
id
es
w
eig
h
t
d
ec
ay
b
y
d
ec
o
u
p
lin
g
th
e
g
r
ad
ien
t
o
f
th
e
L
2
r
eg
u
lar
izatio
n
f
r
o
m
th
e
u
p
d
ate
o
f
Ad
am
[
2
9
]
.
T
h
e
lear
n
i
n
g
r
ate
(
lr
)
was
co
n
f
ig
u
r
ed
to
b
e
2
e
-
5
,
a
n
d
ep
s
il
o
n
(
ep
s
)
was
estab
lis
h
ed
at
1
e
-
8
.
T
h
e
lear
n
in
g
r
ate
g
o
v
er
n
s
th
e
m
ag
n
itu
d
e
o
f
ad
j
u
s
tm
en
ts
m
ad
e
to
th
e
m
o
d
el
in
r
esp
o
n
s
e
to
t
h
e
esti
m
ated
er
r
o
r
d
u
r
in
g
ea
c
h
u
p
d
ate
o
f
th
e
m
o
d
el
’
s
weig
h
ts
.
E
p
s
ilo
n
s
er
v
es a
s
an
ex
ce
ed
in
g
ly
s
m
all
v
alu
e,
im
p
l
em
en
ted
to
s
af
eg
u
ar
d
a
g
ain
s
t
d
iv
is
io
n
b
y
ze
r
o
.
−
T
h
e
n
u
m
b
er
o
f
ep
o
c
h
s
was
s
et
to
4
.
An
e
p
o
ch
s
ig
n
if
ies
a
s
in
g
le
iter
atio
n
th
r
o
u
g
h
th
e
c
o
m
p
lete
tr
ain
in
g
d
ataset.
T
h
e
n
u
m
b
er
o
f
e
p
o
c
h
s
s
er
v
es
as
a
h
y
p
er
p
ar
am
ete
r
,
d
eter
m
in
in
g
h
o
w
m
an
y
tim
es
th
e
lear
n
in
g
alg
o
r
ith
m
will iter
ate
o
v
e
r
t
h
e
en
tire
tr
ain
in
g
d
ataset.
T
h
e
cu
s
to
m
is
ed
m
o
d
el
ar
ch
ite
ctu
r
e
is
as f
o
llo
ws
:
T
ab
le
2
.
Par
am
eter
’
s
co
m
p
ar
is
o
n
o
f
p
r
e
-
tr
ain
ed
m
o
d
els.
L
=n
u
m
b
er
s
o
f
tr
an
s
f
o
r
m
er
lay
er
s
,
H=
n
u
m
b
er
s
o
f
h
id
d
en
e
m
b
ed
d
in
g
s
ize
,
an
d
A
=n
u
m
b
er
s
o
f
atten
tio
n
h
ea
d
s
[
5
]
Pre
-
t
r
a
i
n
e
d
m
o
d
e
l
[
6
]
L
[
7
]
H
[
8
]
A
[
9
]
To
t
a
l
p
a
r
a
me
t
e
r
[
1
0
]
La
n
g
u
a
g
e
t
y
p
e
[
1
1
]
B
ER
T
-
b
a
se
m
u
l
t
i
l
i
n
g
u
a
l
[
1
2
]
12
[
1
3
]
7
6
8
[
1
4
]
12
[
1
5
]
1
1
0
m
i
l
l
i
o
n
[
1
6
]
M
u
l
t
i
l
a
n
g
u
a
g
e
[
1
7
]
I
n
d
o
B
ER
T
-
b
a
s
e
[
1
8
]
12
[
1
9
]
7
6
8
[
2
0
]
12
[
2
1
]
1
2
4
.
5
m
i
l
l
i
o
n
[
2
2
]
M
o
n
o
l
a
n
g
u
a
g
e
[
2
3
]
R
o
B
E
R
Ta
[
2
4
]
12
[
2
5
]
7
6
8
[
2
6
]
12
[
2
7
]
1
2
5
m
i
l
l
i
o
n
[
2
8
]
M
u
l
t
i
l
a
n
g
u
a
g
e
[
2
9
]
G
P
T
-
2
I
n
d
o
n
e
si
a
[
3
0
]
12
[
3
1
]
7
6
8
[
3
2
]
12
[
3
3
]
1
1
7
m
i
l
l
i
o
n
[
3
4
]
En
g
l
i
sh
3
.
6
.
1
.
M
B
E
R
T
a
rc
hite
ct
ure
MBERT,
s
h
o
r
t
f
o
r
m
u
ltil
in
g
u
al
B
E
R
T
,
r
ep
r
esen
ts
a
v
ar
ian
t
o
f
B
E
R
T
th
at
u
n
d
er
g
o
es
tr
ain
in
g
n
o
t
s
o
lely
o
n
tex
t
f
r
o
m
a
s
in
g
le
lan
g
u
ag
e
b
u
t
o
n
an
ex
ten
s
iv
e
co
llectio
n
o
f
tex
t
s
p
an
n
in
g
m
u
ltip
le
lan
g
u
ag
es.
T
h
is
u
n
iq
u
e
tr
ain
in
g
a
p
p
r
o
ac
h
eq
u
ip
s
MBERT
with
th
e
c
ap
ab
ilit
y
to
co
m
p
r
eh
e
n
d
a
n
d
h
an
d
le
in
f
o
r
m
atio
n
ac
r
o
s
s
a
m
u
ltit
u
d
e
o
f
la
n
g
u
a
g
es,
as
o
p
p
o
s
ed
to
b
ein
g
lim
ited
to
a
s
in
g
le
lan
g
u
ag
e.
M
B
E
R
T
em
p
lo
y
s
an
id
en
tical
ar
ch
itectu
r
e
an
d
s
elf
-
atten
tio
n
m
ec
h
an
is
m
as
th
e
o
r
ig
in
al
B
E
R
T
.
Ho
wev
er
,
d
u
e
to
its
tr
ain
in
g
with
m
u
ltip
le
lan
g
u
a
g
es,
MB
E
R
T
ac
q
u
ir
es th
e
ca
p
ac
ity
f
o
r
tr
an
s
f
er
lear
n
in
g
ac
r
o
s
s
d
iv
er
s
e
lan
g
u
ag
es.
B
E
R
T
,
an
ac
r
o
n
y
m
f
o
r
B
E
R
T
s
,
co
n
s
titu
tes
a
g
r
o
u
n
d
b
r
ea
k
in
g
ad
v
an
ce
m
en
t
i
n
th
e
f
iel
d
o
f
NL
P
.
T
h
e
B
E
R
T
ar
ch
itectu
r
e
lev
er
a
g
es
tr
an
s
f
o
r
m
e
r
s
to
ac
q
u
ir
e
la
n
g
u
ag
e
r
ep
r
esen
tatio
n
s
th
r
o
u
g
h
th
e
ex
am
in
atio
n
o
f
b
i
d
ir
ec
tio
n
al
c
o
n
tex
t.
T
r
an
s
f
o
r
m
er
s
r
ep
r
esen
t
a
ca
teg
o
r
y
o
f
n
e
u
r
al
n
etwo
r
k
ar
ch
itectu
r
e
ch
ar
ac
ter
ized
b
y
a
s
elf
-
atten
tio
n
m
ec
h
a
n
is
m
.
T
h
is
m
ec
h
an
is
m
en
a
b
les
th
e
m
o
d
el
to
co
n
cu
r
r
en
tly
p
r
o
ce
s
s
wo
r
d
s
with
in
a
s
en
ten
ce
an
d
co
m
p
r
eh
e
n
d
th
e
co
n
tex
tu
al
r
elatio
n
s
h
ip
s
am
o
n
g
th
e
m
b
y
c
o
n
s
id
er
in
g
th
e
en
tire
s
en
ten
ce
.
C
o
n
s
eq
u
en
tly
,
th
is
ap
p
r
o
ac
h
f
ac
ilit
ates
m
o
r
e
r
ap
id
a
n
d
i
n
-
d
e
p
th
co
m
p
r
eh
e
n
s
io
n
wh
e
n
co
m
p
ar
ed
to
s
eq
u
en
tia
l
ar
ch
itectu
r
es th
at
p
r
o
ce
s
s
wo
r
d
s
o
n
e
af
ter
t
h
e
o
th
e
r
.
B
ef
o
r
e
em
b
ar
k
i
n
g
o
n
th
e
f
in
e
-
t
u
n
in
g
p
r
o
ce
s
s
,
it
is
im
p
er
ativ
e
to
p
r
ep
ar
e
an
d
to
k
en
ize
th
e
i
n
p
u
t
d
ata.
E
ac
h
tex
tu
al
s
eg
m
e
n
t
is
d
is
ass
em
b
led
in
to
d
is
cr
ete
to
k
en
s
,
w
h
ich
B
E
R
T
s
u
b
s
eq
u
en
tly
m
ap
s
to
co
r
r
esp
o
n
d
in
g
to
k
en
I
Ds
ac
co
r
d
in
g
t
o
its
p
r
ed
ef
in
ed
v
o
ca
b
u
lar
y
.
As
an
illu
s
tr
ativ
e
ex
am
p
le
in
Fig
u
r
e
4
,
co
n
s
id
er
th
e
s
en
ten
ce
“
P
en
jela
s
a
n
men
a
r
ik
d
a
n
m
u
d
a
h
d
ip
a
h
a
mi
,
”
wh
ic
h
wo
u
ld
b
e
s
eg
m
en
ted
i
n
to
a
to
k
en
lis
t
as
f
o
llo
ws:
[
“
[
C
L
S]
”
,
“
P
en
jela
s
a
n
”
,
“
me
n
a
r
ik
”
,
“
dan
”
,
“
mu
d
a
h
”
,
“
d
i
p
a
h
a
mi
”
,
“
[
SEP]
”
].
T
h
ese
to
k
en
s
ar
e
th
en
tr
an
s
lated
in
to
a
s
er
ies
o
f
n
u
m
er
ical
I
Ds.
W
ith
in
B
E
R
T
’
s
f
r
am
ewo
r
k
,
t
h
e
em
b
e
d
d
in
g
m
at
r
ix
E
is
u
s
e
d
to
tr
an
s
f
o
r
m
ea
c
h
to
k
en
in
to
(
)
,
a
v
ec
to
r
o
f
n
u
m
er
ic
I
Ds.
T
h
is
v
ec
t
o
r
e
n
ca
p
s
u
lates
b
o
th
th
e
s
em
an
tic
a
n
d
s
y
n
tactic
in
f
o
r
m
atio
n
o
f
th
e
t
o
k
en
s
an
d
s
er
v
es
as
th
e
in
itial
in
p
u
t
f
o
r
th
e
B
E
R
T
n
etwo
r
k
.
Su
b
s
eq
u
en
tly
,
th
e
n
etwo
r
k
u
tili
ze
s
th
is
em
b
ed
d
in
g
v
ec
to
r
to
ac
q
u
ir
e
a
n
u
n
d
er
s
tan
d
i
n
g
o
f
th
e
c
o
n
tex
t
an
d
in
ter
r
elatio
n
s
h
ip
s
am
o
n
g
to
k
en
s
with
in
a
s
en
ten
ce
th
r
o
u
g
h
o
u
t
b
o
th
th
e
p
r
e
-
tr
ain
i
n
g
an
d
f
in
e
-
tu
n
in
g
p
h
ases
.
T
h
e
em
b
ed
d
in
g
o
f
th
e
[
C
L
S]
to
k
en
,
ℎ
[
]
,
f
r
o
m
th
e
f
i
n
al
lay
er
o
f
B
E
R
T
is
u
s
ed
as
a
s
u
m
m
ar
ized
r
ep
r
esen
tatio
n
o
f
th
e
e
n
tire
in
p
u
t
s
eq
u
e
n
ce
.
Nex
t,
th
is
v
ec
to
r
u
n
d
er
g
o
es
p
r
o
ce
s
s
in
g
th
r
o
u
g
h
a
lin
ea
r
class
if
icatio
n
lay
er
,
wh
ich
in
tu
r
n
c
o
n
v
e
r
ts
it
in
to
a
two
-
d
i
m
en
s
io
n
al
v
ec
t
o
r
t
h
at
alig
n
s
with
th
e
n
u
m
b
e
r
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
S
en
timen
t a
n
a
lysi
s
o
f stu
d
en
t e
va
lu
a
tio
n
fee
d
b
a
ck
u
s
in
g
tr
a
n
s
fo
r
mer …
(
I
b
n
u
Da
q
i
q
il I
D)
1133
class
es
in
a
b
in
ar
y
class
if
icati
o
n
task
.
T
h
e
lin
ea
r
tr
an
s
f
o
r
m
atio
n
is
g
iv
en
b
y
th
e
v
ec
t
o
r
ℎ
is
u
s
ed
as
in
p
u
t
o
f
th
e
f
in
al
f
u
lly
-
co
n
n
ec
ted
class
if
icatio
n
lay
er
.
Giv
en
th
e
p
a
r
am
eter
∈
ℝ
768
o
f
th
e
class
if
icatio
n
lay
er
,
wh
er
e
K
is
th
e
n
u
m
b
er
o
f
ca
te
g
o
r
ies
.
T
h
e
o
u
tp
u
t
o
f
lin
ea
r
o
r
class
if
icatio
n
lay
er
is
ca
lcu
lated
as lo
g
it
(
1
)
.
=
ℎ
(
1
)
Mo
r
eo
v
er
,
th
e
v
alu
e
o
f
th
e
lo
g
it
Z
is
s
u
b
jecte
d
to
a
tr
an
s
f
o
r
m
atio
n
u
s
in
g
t
h
e
So
f
tMa
x
f
u
n
ctio
n
.
I
n
th
is
p
ar
ticu
lar
s
ce
n
a
r
io
,
th
e
r
e
ex
is
t
two
class
es:
n
am
ely
,
‘
p
o
s
itiv
e
,’
an
d
‘
n
e
g
ativ
e.
’
T
h
e
p
r
o
b
a
b
ilit
y
v
alu
e
ca
n
b
e
o
b
s
er
v
ed
i
n
(
2
)
.
(
c
l
a
s
s
)
=
∑
2
=
1
(
2
)
I
n
th
e
f
in
e
-
tu
n
in
g
p
r
o
ce
s
s
,
th
e
m
o
d
el
’
s
p
ar
am
eter
s
ar
e
ad
ju
s
ted
iter
ativ
ely
to
m
in
im
ize
a
lo
s
s
f
u
n
ctio
n
,
t
y
p
ically
th
e
cr
o
s
s
-
en
tr
o
p
y
l
o
s
s
,
wh
ich
is
co
m
m
o
n
ly
u
s
ed
f
o
r
class
if
icatio
n
task
s
.
T
h
e
lo
s
s
f
o
r
a
s
in
g
le
in
s
tan
ce
is
co
m
p
u
ted
(
3
)
wh
er
e
is
th
e
tr
u
e
lab
el
o
f
t
h
e
in
s
tan
ce
,
(
c
l
a
s
s
1
)
is
th
e
m
o
d
el
’
s
esti
m
a
ted
p
r
o
b
a
b
ilit
y
f
o
r
t
h
e
p
o
s
itiv
e
cla
s
s
,
an
d
(
c
l
a
s
s
2
)
i
s
th
e
p
r
o
b
ab
ilit
y
f
o
r
t
h
e
n
eg
ativ
e
class
.
=
−
[
l
og
(
(
c
l
a
s
s
1
)
)
+
(
1
−
)
l
og
(
(
c
l
a
s
s
2
)
)
]
(
3
)
An
o
v
er
v
iew
o
f
f
in
e
-
tu
n
in
g
a
B
E
R
T
-
b
ased
m
o
d
el
is
p
r
esen
ted
in
Fig
u
r
e
4
.
T
h
e
f
in
e
-
tu
n
in
g
p
r
o
ce
s
s
en
tails
s
eq
u
en
tial
iter
atio
n
s
o
v
er
th
e
co
m
p
lete
d
ataset,
en
c
o
m
p
ass
in
g
p
r
ed
ictio
n
g
e
n
er
atio
n
,
l
o
s
s
ca
lcu
latio
n
,
an
d
th
e
s
u
b
s
eq
u
en
t
ad
ju
s
tm
en
t
o
f
th
e
m
o
d
el
’
s
p
ar
am
eter
s
.
T
h
is
ad
ju
s
tm
en
t
en
co
m
p
ass
es
b
o
th
th
e
t
r
an
s
f
o
r
m
er
lay
er
s
an
d
th
e
class
if
icatio
n
h
ea
d
,
a
n
d
it
is
f
ac
ilit
ated
t
h
r
o
u
g
h
b
ac
k
p
r
o
p
ag
atio
n
alo
n
g
s
id
e
o
p
tim
izatio
n
alg
o
r
ith
m
s
s
u
ch
as
Ad
am
.
T
h
e
iter
ativ
e
p
r
o
ce
s
s
co
n
tin
u
es
f
o
r
s
ev
er
al
ep
o
ch
s
u
n
til
th
e
lo
s
s
eith
er
r
ea
ch
es
co
n
v
er
g
en
ce
o
r
s
tar
ts
to
r
is
e
o
n
a
v
alid
atio
n
s
et
th
at
is
s
et
asi
d
e,
s
ig
n
alin
g
th
e
co
n
clu
s
io
n
o
f
ef
f
ec
tiv
e
tr
ain
in
g
.
Fig
u
r
e
4
.
MBERT
ar
ch
itectu
r
e
3
.
6
.
2
.
I
nd
o
B
E
RT
s
y
s
t
em
a
rc
hite
ct
ure
I
n
d
o
B
E
R
T
an
d
MBERT
s
h
ar
e
a
co
m
m
o
n
f
o
u
n
d
atio
n
a
s
th
ey
ar
e
b
o
th
r
o
o
ted
in
t
h
e
B
E
R
T
ar
ch
itectu
r
e.
E
ac
h
o
f
th
em
em
p
lo
y
s
th
e
t
r
an
s
f
o
r
m
er
m
o
d
el,
wh
ich
en
co
m
p
ass
es
a
m
u
lti
-
h
ea
d
ed
s
elf
-
atten
tio
n
m
ec
h
an
is
m
an
d
a
f
ee
d
-
f
o
r
w
ar
d
n
eu
r
al
n
etwo
r
k
f
o
r
th
e
p
r
o
ce
s
s
in
g
o
f
in
p
u
t
t
o
k
en
s
.
B
o
th
m
o
d
els
ar
e
en
g
in
ee
r
ed
with
th
e
p
r
i
m
ar
y
o
b
jectiv
e
o
f
co
m
p
r
eh
en
d
in
g
th
e
co
n
te
x
t
o
f
i
n
d
iv
id
u
al
wo
r
d
s
th
r
o
u
g
h
t
h
e
ex
am
in
atio
n
o
f
th
e
n
eig
h
b
o
r
i
n
g
wo
r
d
s
,
lev
er
ag
i
n
g
th
e
b
id
i
r
ec
tio
n
al
tr
ain
in
g
ca
p
ab
ilit
ies
o
f
th
e
t
r
a
n
s
f
o
r
m
er
.
Ad
d
itio
n
ally
,
b
o
t
h
m
o
d
els
e
m
p
lo
y
a
co
n
s
is
ten
t
to
k
en
izati
o
n
ap
p
r
o
ac
h
,
s
u
ch
as
th
e
W
o
r
d
Piece
alg
o
r
ith
m
,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
In
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
3
6
,
No
.
2
,
No
v
em
b
er
20
24
:
1
1
2
7
-
1
1
3
9
1134
wh
ich
d
is
s
ec
ts
wo
r
d
s
in
to
s
u
b
wo
r
d
u
n
its
.
T
h
is
m
eth
o
d
o
l
o
g
y
en
h
a
n
ce
s
th
e
m
o
d
el
’
s
ab
ilit
y
to
ef
f
ec
tiv
ely
h
an
d
le
in
f
r
eq
u
e
n
t w
o
r
d
s
a
n
d
f
ac
ilit
ates m
o
r
e
r
o
b
u
s
t g
en
e
r
aliza
tio
n
.
W
h
ile
MBE
R
T
u
n
d
er
g
o
es
tr
ain
in
g
o
n
a
co
r
p
u
s
en
co
m
p
ass
in
g
tex
t
f
r
o
m
1
0
4
lan
g
u
ag
es,
in
clu
d
in
g
I
n
d
o
n
esian
,
I
n
d
o
B
E
R
T
,
in
co
n
tr
ast,
is
tr
ain
ed
ex
clu
s
iv
ely
o
n
an
ex
ten
s
iv
e
I
n
d
o
n
e
s
ian
co
r
p
u
s
.
T
h
is
s
p
ec
ialized
tr
ain
in
g
eq
u
ip
s
I
n
d
o
B
E
R
T
with
a
h
eig
h
ten
ed
s
en
s
itiv
ity
to
th
e
in
tr
icac
ie
s
o
f
th
e
I
n
d
o
n
esian
lan
g
u
ag
e,
e
n
co
m
p
ass
in
g
id
io
m
atic
ex
p
r
ess
io
n
s
,
s
lan
g
,
an
d
s
y
n
tactica
l
s
tr
u
ctu
r
es.
I
n
ter
m
s
o
f
m
ath
em
atica
l
r
ep
r
esen
tatio
n
,
th
e
d
if
f
er
en
ce
lies
in
th
e
em
b
ed
d
in
g
lay
er
,
wh
er
e
I
n
d
o
B
E
R
T
m
ay
h
av
e
a
v
o
ca
b
u
lar
y
m
atr
ix
opt
im
ized
f
o
r
I
n
d
o
n
esian
t
o
k
en
s
,
as
o
p
p
o
s
ed
to
th
e
m
o
r
e
g
en
er
al
m
atr
i
x
o
f
MBERT.
I
n
(
4
)
a
n
d
(
5
)
s
h
o
w
th
e
d
if
f
er
e
n
t o
f
th
ese
m
eth
o
d
s
.
(
)
=
(
)
+
(
)
+
(
)
(
4
)
(
)
=
(
)
+
(
)
+
(
)
(
5
)
3
.
6
.
3
.
Ro
B
E
RT
a
s
y
s
t
em
a
rc
hite
ct
ure
R
o
B
E
R
T
a
en
h
an
ce
s
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
B
E
R
T
m
o
d
el
th
r
o
u
g
h
s
ev
er
al
s
ig
n
if
ica
n
t
m
o
d
if
icatio
n
s
.
T
wo
n
o
tewo
r
th
y
d
is
tin
ctio
n
s
in
v
o
lv
e
alt
er
atio
n
s
in
th
e
l
o
s
s
f
u
n
ctio
n
an
d
th
e
r
em
o
v
al
o
f
s
e
g
m
en
t
em
b
e
d
d
in
g
s
.
R
o
B
E
R
T
a
e
lim
in
ates
th
e
s
eg
m
en
t
em
b
ed
d
i
n
g
s
an
d
th
e
n
ex
t
s
en
ten
ce
p
r
ed
ictio
n
(
NS
P)
p
r
etr
ain
in
g
task
,
th
er
eb
y
s
im
p
lify
in
g
th
e
in
p
u
t
r
ep
r
esen
tatio
n
.
B
y
co
n
ce
n
tr
a
tin
g
ex
clu
s
iv
ely
o
n
a
n
ex
te
n
d
e
d
f
o
r
m
o
f
m
ask
ed
lan
g
u
ag
e
m
o
d
elin
g
,
R
o
B
E
R
T
a
elim
in
ates
th
e
n
ec
ess
ity
to
d
if
f
er
en
tiate
b
etwe
en
m
u
ltip
le
s
eg
m
en
ts
with
in
its
in
p
u
t.
T
h
is
d
esig
n
d
ec
is
io
n
s
im
p
lifie
s
th
e
m
o
d
el
’
s
a
r
ch
itectu
r
e
a
n
d
is
r
o
o
te
d
in
th
e
r
ec
o
g
n
itio
n
th
at
s
eg
m
en
t
em
b
ed
d
in
g
s
an
d
n
e
x
t
s
en
ten
ce
p
r
ed
ictio
n
h
a
v
e
lim
ited
im
p
ac
t
o
n
th
e
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
in
d
o
wn
s
tr
ea
m
task
s
.
I
n
th
e
ab
s
en
ce
o
f
s
eg
m
en
t
em
b
ed
d
in
g
s
,
R
o
B
E
R
T
a
r
elies
ex
clu
s
iv
ely
o
n
th
e
p
o
s
itio
n
al
in
f
o
r
m
atio
n
an
d
th
e
in
h
er
en
t c
o
n
ten
t
o
f
th
e
t
o
k
en
s
th
em
s
elv
es.
I
n
(
6
)
p
r
esen
t
s
th
e
em
b
ed
d
in
g
v
ec
to
r
f
o
r
R
o
B
E
R
T
a.
(
)
=
(
)
+
(
)
(
6
)
T
h
e
co
n
s
eq
u
e
n
ce
o
f
t
h
is
d
is
tin
ctio
n
lies
in
th
e
f
ac
t th
at
wh
ile
B
E
R
T
m
ay
in
h
er
en
tly
p
e
r
f
o
r
m
b
etter
in
task
s
n
ec
ess
itat
in
g
a
p
r
o
f
o
u
n
d
co
m
p
r
eh
e
n
s
io
n
o
f
s
eg
m
e
n
t
r
elatio
n
s
h
ip
s
,
R
o
B
E
R
T
a
’
s
en
h
an
ce
d
tr
ain
in
g
ef
f
icien
cy
a
n
d
s
tr
ea
m
lin
ed
m
o
d
el
d
esig
n
p
o
s
itio
n
it
as
a
s
u
p
er
io
r
c
h
o
ice
f
o
r
task
s
wh
er
e
s
u
ch
d
if
f
er
en
tiatio
n
h
o
ld
s
less
er
s
ig
n
if
ican
ce
.
W
it
h
r
eg
ar
d
to
th
e
lo
s
s
f
u
n
ctio
n
,
R
o
B
E
R
T
a
em
p
lo
y
s
an
ad
ap
ted
v
er
s
io
n
o
f
t
h
e
m
ask
ed
lan
g
u
a
g
e
m
o
d
el
lo
s
s
,
f
ea
tu
r
in
g
d
y
n
a
m
ic
m
ask
in
g
.
T
h
e
lo
s
s
f
u
n
ctio
n
f
o
r
R
o
B
E
R
T
a,
wh
ich
ce
n
ter
s
s
o
lely
o
n
th
e
m
ask
e
d
to
k
e
n
s
r
ath
er
th
an
o
n
NSP,
is
co
m
p
u
te
d
ac
co
r
d
i
n
g
to
(
7
)
.
=
−
1
∑
(
|
;
)
=
1
(
7
)
is
th
e
n
u
m
b
er
o
f
m
ask
ed
to
k
en
s
,
is
th
e
m
a
s
k
ed
to
k
en
,
an
d
r
ep
r
esen
ts
th
e
p
ar
am
eter
s
o
f
th
e
m
o
d
el.
T
h
is
co
n
tr
ast in
lo
s
s
f
u
n
ctio
n
s
alig
n
s
with
R
o
B
E
R
T
a
’
s
tr
ain
in
g
o
b
jectiv
e
th
at
co
n
c
en
tr
ates e
x
clu
s
iv
ely
o
n
th
e
p
r
ed
ictio
n
o
f
m
ask
e
d
to
k
en
s
,
e
n
h
an
ci
n
g
its
lan
g
u
ag
e
m
o
d
ellin
g
ca
p
ab
ilit
ies
with
o
u
t
th
e
NS
P
co
n
s
tr
ain
t.
3
.
6
.
4
.
G
P
T
-
2
s
y
s
t
em
a
rc
hite
c
t
ure
GPT
-
2
,
wh
ich
s
tan
d
s
f
o
r
GPT
-
2
,
is
a
m
o
d
el
cr
ea
ted
b
y
O
p
en
AI
,
ad
v
an
cin
g
u
p
o
n
th
e
i
n
itial
GPT
ar
ch
itectu
r
e.
Fu
n
d
am
e
n
tally
,
GPT
-
2
is
s
tr
u
ctu
r
ed
as
a
g
en
er
ativ
e
m
o
d
el
with
th
e
ab
ilit
y
to
g
r
asp
co
n
tex
t
f
r
o
m
a
p
r
o
v
id
ed
tex
t a
n
d
g
e
n
er
ate
c
o
h
er
en
t a
n
d
co
n
tex
tu
ally
ap
p
r
o
p
r
iate
tex
t in
r
esp
o
n
s
e.
T
h
e
m
o
d
el
ad
h
er
es to
th
e
p
r
in
cip
les
o
f
th
e
t
r
an
s
f
o
r
m
er
ar
ch
itectu
r
e,
e
m
p
h
asizin
g
s
elf
-
atten
tio
n
m
ec
h
an
is
m
s
to
ass
e
s
s
th
e
s
ig
n
if
ican
ce
o
f
ea
ch
wo
r
d
with
in
th
e
co
n
tex
t
o
f
all
th
e
wo
r
d
s
in
a
g
iv
en
s
eq
u
en
ce
.
Hav
in
g
b
ee
n
p
r
e
-
tr
ain
e
d
o
n
an
ex
p
an
s
iv
e
an
d
d
iv
er
s
e
d
atase
t,
GPT
-
2
ac
q
u
ir
es
a
co
m
p
r
eh
en
s
iv
e
u
n
d
er
s
tan
d
i
n
g
o
f
th
e
l
an
g
u
ag
e
’
s
g
en
er
al
s
tr
u
ctu
r
e
an
d
s
u
b
tleties
.
T
h
is
p
r
o
f
icien
c
y
en
ab
les
it
to
ex
ce
l
in
a
b
r
o
ad
s
p
ec
tr
u
m
o
f
task
s
wh
en
s
u
b
s
eq
u
en
tl
y
f
in
e
-
tu
n
e
d
o
n
d
ata
th
at
is
m
o
r
e
s
p
ec
if
ic
to
th
o
s
e
task
s
.
I
n
co
n
tr
ast,
B
E
R
T
also
r
elie
s
o
n
th
e
t
r
an
s
f
o
r
m
er
ar
ch
ite
ctu
r
e,
b
u
t
its
p
r
im
ar
y
em
p
h
a
s
is
lies
in
g
en
er
atin
g
e
x
ten
s
iv
e
b
id
ir
ec
ti
o
n
al
r
ep
r
esen
tatio
n
s
b
y
co
n
c
u
r
r
en
tly
c
o
n
s
id
er
in
g
b
o
t
h
lef
t
an
d
r
ig
h
t
co
n
te
x
t
th
r
o
u
g
h
o
u
t
all
lay
er
s
o
f
t
h
e
m
o
d
el.
C
o
n
s
eq
u
en
tly
,
B
E
R
T
d
em
o
n
s
tr
ates
ex
ce
p
tio
n
al
p
r
o
f
icien
cy
in
g
r
asp
in
g
th
e
s
u
b
tleties
an
d
s
em
a
n
tics
o
f
w
o
r
d
s
with
in
th
eir
c
o
n
t
ex
tu
al
f
r
a
m
ewo
r
k
.
T
h
is
ch
ar
ac
ter
is
tic
r
en
d
er
s
it
p
ar
ticu
lar
ly
well
-
s
u
ited
f
o
r
ta
s
k
s
d
em
an
d
in
g
a
n
i
n
-
d
e
p
th
c
o
m
p
r
eh
e
n
s
io
n
o
f
lan
g
u
ag
e,
i
n
clu
d
in
g
s
en
tim
e
n
t
an
aly
s
is
,
n
am
ed
en
tity
r
ec
o
g
n
itio
n
,
an
d
q
u
esti
o
n
-
an
s
wer
in
g
.
T
h
e
p
r
im
a
r
y
d
is
tin
ctio
n
b
etwe
en
GPT
-
2
an
d
B
E
R
T
r
esid
es
in
th
eir
r
esp
ec
tiv
e
ap
p
r
o
ac
h
es
to
co
n
tex
t
an
d
t
h
e
s
p
ec
if
ic
task
s
th
ey
ar
e
o
p
ti
m
ized
f
o
r
.
GPT
-
2
’
s
u
n
id
ir
ec
tio
n
al
ap
p
r
o
ac
h
em
p
o
wer
s
it
to
f
o
r
ec
ast
th
e
s
u
b
s
eq
u
en
t
s
ec
tio
n
o
f
tex
t
s
o
lely
d
ep
en
d
e
n
t
o
n
th
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
I
SS
N:
2
5
0
2
-
4
7
52
S
en
timen
t a
n
a
lysi
s
o
f stu
d
en
t e
va
lu
a
tio
n
fee
d
b
a
ck
u
s
in
g
tr
a
n
s
fo
r
mer …
(
I
b
n
u
Da
q
i
q
il I
D)
1135
an
tece
d
en
t
c
o
n
tex
t,
a
f
ea
tu
r
e
in
h
er
e
n
tly
a
d
v
an
ta
g
eo
u
s
f
o
r
task
s
lik
e
tex
t
co
m
p
letio
n
a
n
d
tex
t
g
e
n
er
atio
n
.
C
o
n
v
er
s
ely
,
B
E
R
T
,
b
y
ex
am
in
in
g
co
n
te
x
t
f
r
o
m
b
o
th
d
ir
ec
t
io
n
s
,
f
u
r
n
is
h
es
a
co
m
p
r
e
h
en
s
i
v
e
c
o
m
p
r
e
h
en
s
io
n
th
at
p
r
o
v
es
v
alu
ab
le
f
o
r
task
s
d
em
an
d
in
g
p
r
ec
is
e
co
n
tex
tu
al
p
o
r
tr
ay
als.
Fu
n
d
a
m
en
tally
,
wh
ile
GPT
-
2
f
o
cu
s
es
o
n
tex
t
g
en
e
r
atio
n
,
B
E
R
T
ad
o
p
ts
a
m
o
r
e
d
is
cr
im
in
ativ
e
r
o
le
,
s
tr
u
ctu
r
ed
to
f
in
e
-
tu
n
e
to
war
d
a
s
p
ec
if
ic
o
u
tp
u
t
ac
co
r
d
in
g
to
th
e
in
p
u
t,
r
at
h
e
r
th
an
g
en
er
atin
g
n
o
v
el
co
n
ten
t.
T
h
is
d
if
f
e
r
en
tiatio
n
d
elin
ea
tes
th
eir
d
is
tin
ct
ap
p
licatio
n
s
in
th
e
f
ield
o
f
NL
P
.
I
n
tech
n
ical
ter
m
s
,
GPT
-
2
d
o
es
n
o
t
em
p
lo
y
s
p
ec
if
ic
to
k
en
s
lik
e
[
C
L
S]
a
n
d
[
SEP]
,
wh
ich
ar
e
co
m
m
o
n
l
y
u
s
ed
in
th
e
B
E
R
T
m
o
d
el.
I
n
s
tead
,
GPT
-
2
p
r
o
ce
s
s
es
ea
ch
in
p
u
t
to
k
en
in
d
ep
en
d
e
n
tly
.
On
c
e
to
k
en
ized
,
ea
ch
t
o
k
en
I
D
is
m
ap
p
ed
to
a
d
en
s
e
v
ec
t
o
r
s
p
a
ce
y
ield
in
g
th
e
to
k
en
e
m
b
ed
d
in
g
(
)
.
GPT
-
2
en
h
an
ce
s
th
ese
em
b
ed
d
in
g
s
b
y
in
co
r
p
o
r
atin
g
p
o
s
itio
n
al
in
f
o
r
m
atio
n
to
p
r
eser
v
e
th
e
s
eq
u
en
tial o
r
d
er
o
f
wo
r
d
s
,
a
cr
itical
asp
ec
t
f
o
r
co
m
p
r
eh
e
n
d
in
g
t
h
e
s
tr
u
ctu
r
e
o
f
lan
g
u
ag
e.
T
h
u
s
,
th
e
e
m
b
ed
d
in
g
f
o
r
a
t
o
k
en
at
p
o
s
itio
n
is
th
e
s
u
m
o
f
its
to
k
e
n
em
b
e
d
d
in
g
an
d
its
p
o
s
itio
n
e
m
b
ed
d
in
g
,
r
ep
r
esen
ted
b
y
(
)
+
(
)
.
Fo
llo
win
g
th
at,
th
e
em
b
e
d
d
in
g
s
ar
e
p
ass
ed
t
h
r
o
u
g
h
th
e
n
u
m
er
o
u
s
lay
er
s
with
in
th
e
GP
T
-
2
m
o
d
el.
W
ith
in
ea
ch
lay
er
o
f
th
e
m
o
d
el,
s
elf
-
atten
tio
n
a
n
d
n
e
u
r
al
n
etwo
r
k
o
p
er
atio
n
s
ar
e
a
p
p
lied
to
iter
ativ
ely
en
h
an
ce
an
d
r
ef
in
e
t
h
e
r
ep
r
ese
n
tatio
n
o
f
ea
ch
to
k
en
.
T
h
e
r
es
u
lt
o
b
tain
ed
f
r
o
m
th
e
u
ltima
te
lay
er
o
f
G
PT
-
2
f
o
r
th
e
g
iv
en
s
eq
u
en
ce
co
n
s
is
ts
o
f
a
co
llectio
n
o
f
v
ec
t
o
r
s
,
an
d
ty
p
ically
,
th
e
v
ec
to
r
ass
o
ci
ated
with
th
e
in
itial
to
k
en
(
o
r
an
alter
n
ativ
e
s
tr
ateg
y
b
ased
o
n
th
e
s
p
ec
if
ic
task
)
is
u
tili
ze
d
f
o
r
s
u
b
s
eq
u
en
t c
lass
if
icatio
n
task
s
.
T
h
e
ch
o
s
en
o
u
tp
u
t
v
ec
to
r
f
r
o
m
GPT
-
2
,
f
r
eq
u
en
tly
th
e
o
u
tp
u
t
co
r
r
esp
o
n
d
in
g
to
th
e
f
ir
s
t
to
k
en
,
is
s
u
b
s
eq
u
en
tly
f
o
r
war
d
ed
th
r
o
u
g
h
a
lin
ea
r
la
y
er
t
h
at
f
u
n
cti
o
n
s
as
th
e
class
if
icatio
n
h
ea
d
.
T
h
is
lay
er
p
er
f
o
r
m
s
a
d
im
en
s
io
n
ality
r
e
d
u
ctio
n
o
f
th
e
h
ig
h
-
d
im
e
n
s
io
n
al
v
ec
to
r
in
to
th
e
d
esig
n
ate
d
la
b
el
s
p
ac
e.
T
h
e
lo
g
it
z
,
wh
ich
s
ig
n
if
ies
th
e
u
n
p
r
o
ce
s
s
ed
class
if
icatio
n
s
co
r
e,
is
co
m
p
u
ted
t
h
r
o
u
g
h
th
is
lin
ea
r
tr
an
s
f
o
r
m
atio
n
.
I
n
th
e
ca
s
e
o
f
b
in
ar
y
class
if
icatio
n
task
s
,
th
e
s
ig
m
o
id
ac
tiv
atio
n
f
u
n
ctio
n
is
em
p
lo
y
ed
o
n
t
h
e
lo
g
it
to
g
en
er
ate
a
p
r
o
b
a
b
ili
ty
v
alu
e
r
an
g
in
g
f
r
o
m
0
to
1
,
in
d
icatin
g
th
e
lik
elih
o
o
d
o
f
class
m
em
b
er
s
h
ip
.
3
.
7
.
M
o
del per
f
o
rma
nce
ev
a
lua
t
io
n
A
m
o
d
el
ass
ess
m
en
t
was
ca
r
r
ied
o
u
t
to
ass
ess
th
e
ef
f
icie
n
cy
o
f
t
h
e
t
r
an
s
f
o
r
m
er
-
b
ased
lan
g
u
a
g
e
m
o
d
el
in
an
aly
zin
g
th
e
s
en
tim
en
t
ex
p
r
ess
ed
in
s
tu
d
e
n
t
r
ev
iews.
T
h
is
ev
alu
atio
n
e
n
co
m
p
ass
ed
th
e
u
tili
za
tio
n
o
f
v
ar
io
u
s
m
etr
ics
to
o
f
f
er
a
co
m
p
r
e
h
en
s
iv
e
ass
ess
m
e
n
t
o
f
th
e
m
o
d
el
’
s
p
er
f
o
r
m
a
n
ce
.
T
h
e
Ma
tth
ews
co
r
r
elatio
n
co
ef
f
icie
n
t
(
MCC
)
was
em
p
l
o
y
ed
,
co
n
s
id
er
in
g
th
e
im
b
alan
ce
d
d
is
tr
ib
u
tio
n
o
f
d
ata
lab
els.
T
h
e
co
n
f
u
s
io
n
m
atr
i
x
p
lo
t
s
er
v
e
d
as
a
v
is
u
aliza
tio
n
to
o
l
to
d
ep
ict
th
e
class
if
icatio
n
m
o
d
el
’
s
p
er
f
o
r
m
an
ce
.
Fu
r
th
er
m
o
r
e
,
ad
d
itio
n
al
e
v
al
u
atio
n
m
etr
ics,
in
clu
d
in
g
ac
c
u
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
an
d
th
e
F1
-
s
co
r
e,
wer
e
em
p
lo
y
ed
.
T
h
ese
ev
alu
atio
n
o
u
tco
m
es
wer
e
u
til
ized
to
ascer
tain
th
e
m
o
d
el
’
s
ca
p
ab
ilit
y
to
ac
cu
r
ately
class
if
y
r
ev
iews a
s
eith
er
p
o
s
itiv
e
o
r
n
eg
ativ
e
to
a
s
atis
f
ac
to
r
y
d
e
g
r
e
e.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
I
n
th
e
e
x
am
in
atio
n
o
f
s
en
tim
e
n
t
an
aly
s
is
with
in
s
tu
d
en
t
e
v
alu
atio
n
f
ee
d
b
ac
k
,
f
o
u
r
tr
a
n
s
f
o
r
m
er
-
b
ased
lan
g
u
ag
e
m
o
d
els
h
av
e
b
ee
n
u
tili
ze
d
:
MBE
R
T
b
ase,
I
n
d
o
B
E
R
T
b
ase,
R
o
B
E
R
T
a
b
ase
I
n
d
o
n
esia,
an
d
GPT
-
2
Sm
all
I
n
d
o
n
esia.
T
h
e
ef
f
ec
ti
v
en
ess
o
f
th
ese
m
o
d
els
was
th
o
r
o
u
g
h
ly
ass
ess
ed
u
s
in
g
a
v
ar
iety
o
f
m
etr
ics,
as
o
u
tlin
ed
in
T
a
b
le
3
.
Fro
m
th
ese
d
ata,
it
ca
n
b
e
o
b
s
er
v
ed
th
at
all
tr
an
s
f
o
r
m
er
-
b
ased
m
o
d
el
o
u
tp
er
f
o
r
m
s
tr
ad
itio
n
al
m
o
d
el
(
SVM
an
d
Naiv
e
B
ay
es
(
NB
)
)
an
d
th
e
I
n
d
o
B
E
R
T
b
ase
u
n
ca
s
ed
m
o
d
el
ac
h
iev
ed
th
e
h
i
g
h
est
MCC
o
f
0
.
8
5
8
,
an
ac
cu
r
ac
y
o
f
0
.
9
2
9
,
an
d
a
r
ec
all
o
f
0
.
9
1
1
.
Ad
d
itio
n
ally
,
th
e
MBERT
b
a
s
e
u
n
ca
s
ed
m
o
d
el
ex
h
ib
ited
s
tr
o
n
g
p
er
f
o
r
m
an
c
e,
r
ec
o
r
d
in
g
th
e
h
i
g
h
est
F1
-
s
co
r
e
o
f
0
.
9
3
6
.
Fu
r
th
er
m
o
r
e,
th
e
GPT
-
2
s
m
all
I
n
d
o
n
esia
u
n
ca
s
ed
m
o
d
el
d
e
m
o
n
s
tr
ated
th
e
h
ig
h
est
p
r
ec
is
io
n
at
0
.
9
5
0
,
wh
ile
th
e
R
o
B
E
R
T
a
b
ase
I
n
d
o
n
esia
u
n
ca
s
ed
m
o
d
el
ex
h
ib
ited
th
e
l
ea
s
t f
av
o
r
ab
le
p
e
r
f
o
r
m
an
ce
m
etr
ics am
o
n
g
th
e
f
o
u
r
m
o
d
els.
T
ab
le
3
.
Per
f
o
r
m
an
ce
co
m
p
a
r
is
o
n
o
f
f
i
n
e
-
tu
n
e
d
lan
g
u
ag
e
m
o
d
els
M
o
d
e
l
n
a
m
e
Le
t
t
e
r
c
a
se
Ev
a
l
u
a
t
i
o
n
p
a
r
a
me
t
e
r
s
M
C
C
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F1
-
s
c
o
r
e
M
B
ER
T
b
a
se
U
n
c
a
s
e
d
0
.
8
5
0
0
.
9
2
7
0
.
9
4
5
0
.
9
2
7
0
.
9
3
6
I
n
d
o
B
ER
T
b
a
se
U
n
c
a
s
e
d
0
.
8
5
8
0
.
9
2
9
0
.
9
5
1
0
.
9
1
1
0
.
9
3
1
R
o
B
E
R
TA
b
a
s
e
I
n
d
o
n
e
si
a
U
n
c
a
s
e
d
0
.
7
8
2
0
.
8
9
1
0
.
8
8
7
0
.
8
9
1
0
.
8
8
9
G
P
T
-
2
s
ma
l
l
I
n
d
o
n
e
si
a
U
n
c
a
s
e
d
0
.
8
3
9
0
.
9
2
0
0
.
9
5
0
0
.
9
0
5
0
.
9
2
7
S
V
M
-
0
.
7
3
2
0
.
7
4
3
0
.
7
3
6
0
.
7
4
5
0
.
7
4
4
NB
-
0
.
6
3
4
0
.
6
4
5
0
.
6
6
5
0
.
6
4
2
0
.
6
4
7
An
in
-
d
ep
th
c
o
m
p
ar
is
o
n
o
f
p
er
f
o
r
m
an
ce
m
etr
ics
f
o
r
s
en
tim
en
t
an
aly
s
is
r
ev
ea
ls
th
e
m
o
d
els
’
p
er
f
o
r
m
an
ce
ch
ar
ac
ter
is
tics
.
C
o
m
m
en
cin
g
with
th
e
MCC
,
th
e
I
n
d
o
B
E
R
T
b
ase
m
o
d
el
s
tan
d
s
o
u
t
with
an
MCC
o
f
0
.
8
5
8
,
s
lig
h
tly
s
u
r
p
a
s
s
in
g
MBE
R
T
b
ase
’
s
0
.
8
5
0
.
T
h
is
s
u
g
g
ests
a
s
lig
h
tly
h
ig
h
er
o
v
er
all
q
u
ality
in
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
5
0
2
-
4
7
52
In
d
o
n
esian
J
E
lec
E
n
g
&
C
o
m
p
Sci
,
Vo
l.
3
6
,
No
.
2
,
No
v
em
b
er
20
24
:
1
1
2
7
-
1
1
3
9
1136
b
in
ar
y
class
if
icatio
n
f
o
r
I
n
d
o
B
E
R
T
b
ase.
Alth
o
u
g
h
th
e
ad
v
an
tag
e
is
m
o
d
est,
it
ca
n
h
o
ld
s
ig
n
if
ican
ce
,
esp
ec
ially
wh
en
ass
ess
in
g
s
u
b
tle
s
en
tim
en
ts
co
n
v
ey
e
d
in
s
tu
d
en
t
f
ee
d
b
ac
k
.
C
o
n
v
e
r
s
ely
,
GPT
-
2
s
m
all
I
n
d
o
n
esia,
b
o
asti
n
g
an
MCC
o
f
0
.
8
3
9
,
an
d
R
o
B
E
R
T
a
b
ase
I
n
d
o
n
esia,
with
a
s
co
r
e
o
f
0
.
7
8
2
,
in
d
icate
t
h
at
wh
ile
th
ey
ex
h
ib
it
r
ea
s
o
n
ab
le
p
er
f
o
r
m
a
n
ce
,
th
ey
m
ig
h
t
n
o
t
ca
p
tu
r
e
s
en
tim
en
t
as
ef
f
ec
tiv
ely
as
th
e
B
E
R
T
-
b
ased
m
o
d
els d
o
.
Acc
u
r
ac
y
m
ea
s
u
r
em
en
ts
f
u
r
th
er
em
p
h
asize
th
e
tig
h
t
co
m
p
etitio
n
b
etwe
en
I
n
d
o
B
E
R
T
an
d
MBERT,
attain
in
g
s
co
r
es
o
f
0
.
9
2
9
an
d
0
.
9
2
7
,
r
esp
ec
tiv
ely
,
h
ig
h
lig
h
tin
g
th
eir
d
ep
e
n
d
ab
ilit
y
in
ac
cu
r
ately
class
if
y
in
g
s
en
tim
en
ts
.
GPT
-
2
clo
s
ely
f
o
l
lo
ws
at
0
.
9
2
0
,
estab
lis
h
in
g
its
elf
as
a
r
o
b
u
s
t
co
n
ten
d
er
,
wh
e
r
ea
s
R
o
B
E
R
T
a
lag
s
b
eh
in
d
at
0
.
8
9
1
,
wh
ic
h
,
th
o
u
g
h
lo
wer
,
s
till
s
ig
n
if
ies a
co
m
m
en
d
ab
le
lev
el
o
f
ac
cu
r
ac
y
.
Pre
ci
s
io
n
,
a
cr
u
cial
m
etr
ic
in
s
en
tim
en
t
an
aly
s
is
to
m
in
im
ize
f
alse
p
o
s
itiv
e
er
r
o
r
s
,
d
em
o
n
s
tr
ated
I
n
d
o
B
E
R
T
b
ase
as
th
e
lead
er
with
a
s
co
r
e
o
f
0
.
9
5
1
.
T
h
is
was
clo
s
ely
f
o
llo
wed
b
y
GPT
-
2
s
m
all
I
n
d
o
n
esia,
wh
ich
ac
h
iev
e
d
a
p
r
ec
is
io
n
o
f
0
.
9
5
0
,
in
d
icatin
g
its
p
r
o
f
icie
n
cy
in
co
n
f
i
d
en
tly
i
d
en
tify
in
g
p
o
s
itiv
e
s
en
tim
en
t.
MBERT
b
ase
was
n
o
t
f
ar
b
e
h
in
d
with
a
p
r
ec
is
io
n
s
co
r
e
o
f
0
.
9
4
5
,
wh
ile
R
o
B
E
R
T
a
b
as
e
I
n
d
o
n
esia
h
a
d
t
h
e
lo
west r
esu
lt a
t 0
.
8
8
7
.
C
o
n
s
id
er
in
g
r
ec
all,
MBERT
b
ase
tak
es
th
e
lead
with
a
s
co
r
e
o
f
0
.
9
2
7
,
in
d
icatin
g
i
ts
ca
p
ac
ity
to
en
co
m
p
ass
a
s
u
b
s
tan
tial
p
o
r
tio
n
o
f
p
o
s
itiv
e
s
en
tim
en
ts
an
ess
en
tial
ch
ar
ac
ter
is
tic
f
o
r
co
m
p
r
eh
en
s
iv
e
s
en
tim
en
t
an
aly
s
is
.
I
n
d
o
B
E
R
T
b
ase
an
d
R
o
B
E
R
T
a
b
ase
I
n
d
o
n
esia
e
x
h
ib
it
l
o
wer
r
ec
all
s
co
r
es
at
0
.
9
1
1
an
d
0
.
8
9
1
,
r
esp
ec
tiv
ely
,
wh
ich
co
u
ld
p
o
ten
tially
lead
to
th
e
o
m
is
s
io
n
o
f
ce
r
tain
tr
u
e
p
o
s
itiv
es.
GPT
-
2
ac
h
ie
v
es
a
r
ec
all
o
f
0
.
9
0
5
,
p
lacin
g
it
i
n
a
co
m
p
ar
ab
le
p
o
s
itio
n
t
o
I
n
d
o
B
E
R
T
b
y
s
tr
ik
in
g
a
b
ala
n
ce
b
et
wee
n
p
r
ec
is
io
n
a
n
d
th
e
ca
p
ab
ilit
y
to
id
e
n
tify
tr
u
e
p
o
s
itiv
es.
T
h
e
F1
-
s
co
r
e,
wh
ich
h
ar
m
o
n
izes
p
r
ec
is
io
n
an
d
r
ec
all,
h
ig
h
lig
h
ts
MBERT
b
ase
’
s
p
r
o
f
icien
cy
,
b
o
asti
n
g
th
e
h
ig
h
est
s
co
r
e
o
f
0
.
9
3
6
.
T
h
is
s
u
g
g
ests
th
at
it
ac
h
iev
es
a
s
u
p
er
io
r
b
alan
ce
b
etwe
en
th
ese
two
m
etr
ics.
I
n
d
o
B
E
R
T
b
ase,
wit
h
an
F1
-
s
co
r
e
o
f
0
.
9
3
1
,
in
d
ic
ates
a
s
lig
h
t
p
r
ef
er
en
ce
f
o
r
p
r
ec
is
io
n
o
v
er
r
ec
all.
GPT
-
2
s
m
all
I
n
d
o
n
esia
’
s
F1
-
s
co
r
e
o
f
0
.
9
2
7
p
o
s
itio
n
s
it
as
a
well
-
b
alan
ce
d
m
o
d
el,
wh
ile
R
o
B
E
R
T
a
b
ase
I
n
d
o
n
esia
’
s
s
co
r
e
o
f
0
.
8
8
9
s
u
g
g
ests
r
o
o
m
f
o
r
p
o
ten
tial
en
h
an
ce
m
en
ts
in
ac
h
iev
in
g
a
b
etter
b
alan
ce
b
etwe
en
p
r
ec
is
io
n
an
d
r
ec
all.
E
s
s
en
tially
,
I
n
d
o
B
E
R
T
b
ase
ex
h
ib
its
o
u
ts
tan
d
in
g
p
r
ec
is
i
o
n
,
wh
ile
MBERT
b
ase
s
h
o
wca
s
es
an
im
p
r
ess
iv
e
eq
u
ilib
r
iu
m
ac
r
o
s
s
all
m
etr
ics.
GPT
-
2
s
m
all
I
n
d
o
n
esia
co
n
s
is
ten
tly
u
p
h
o
ld
s
its
r
ep
u
tatio
n
as
a
p
r
ec
is
io
n
-
o
r
ie
n
ted
m
o
d
el,
an
d
ev
e
n
th
o
u
g
h
R
o
B
E
R
T
a
b
a
s
e
I
n
d
o
n
esia
p
er
f
o
r
m
s
less
s
atis
f
ac
to
r
ily
,
it
s
till
h
o
ld
s
p
o
ten
tial
f
o
r
im
p
r
o
v
e
m
en
t
th
r
o
u
g
h
o
p
tim
izatio
n
.
He
n
ce
,
th
e
p
a
r
ticu
lar
m
o
d
el
s
elec
ted
f
o
r
s
en
tim
en
t
an
aly
s
is
o
f
s
tu
d
en
t
ev
alu
atio
n
s
wo
u
ld
d
ep
en
d
o
n
wh
eth
er
t
h
e
task
p
r
io
r
itizes
m
in
im
izin
g
f
alse
p
o
s
itiv
es
o
r
g
u
ar
an
teein
g
t
h
e
d
etec
tio
n
o
f
all
in
s
tan
ce
s
o
f
p
o
s
it
iv
e
s
en
tim
en
t.
T
o
f
ac
ilit
ate
a
m
o
r
e
in
tu
itiv
e
co
m
p
r
eh
e
n
s
io
n
,
we
p
r
esen
t
th
is
d
ata
g
r
ap
h
ically
.
Fig
u
r
e
5
th
r
o
u
g
h
5
o
f
f
er
v
is
u
al
d
ep
i
ctio
n
s
o
f
th
e
MCC
,
ac
cu
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
a
n
d
F1
-
s
co
r
e
f
o
r
ea
ch
m
o
d
el,
r
esp
ec
tiv
e
ly
.
Fig
u
r
e
5
.
C
o
m
p
a
r
is
o
n
o
f
p
er
f
o
r
m
an
ce
r
esu
lt
Du
r
in
g
th
e
f
i
n
e
-
tu
n
i
n
g
p
r
o
ce
s
s
,
all
m
o
d
els
d
is
p
lay
ed
a
r
ed
u
ctio
n
in
tr
ai
n
in
g
lo
s
s
as
th
e
ep
o
ch
s
ad
v
an
ce
d
,
s
ig
n
if
y
in
g
s
u
cc
ess
f
u
l
lear
n
in
g
a
n
d
en
h
an
ce
d
p
er
f
o
r
m
a
n
ce
o
n
t
h
e
tr
ain
in
g
d
ata.
Nev
er
th
eless
,
th
e
v
alid
atio
n
l
o
s
s
es
f
o
r
all
m
o
d
els
s
tar
ted
to
in
cr
ea
s
e
af
t
er
th
e
th
ir
d
o
r
f
o
u
r
th
ep
o
ch
,
ev
en
as
th
e
tr
ain
i
n
g
lo
s
s
es
co
n
tin
u
ed
t
o
d
ec
r
ea
s
e.
T
h
e
d
is
cr
ep
an
c
y
o
b
s
er
v
ed
b
etwe
en
tr
ain
in
g
an
d
v
alid
atio
n
lo
s
s
im
p
lies
th
at
,
alth
o
u
g
h
th
e
m
o
d
els
ar
e
ef
f
ec
t
iv
ely
lear
n
in
g
f
r
o
m
th
e
tr
ain
i
n
g
d
ata,
th
eir
ab
ilit
y
to
g
en
e
r
alize
to
n
ew,
u
n
s
ee
n
d
ata
m
ay
b
e
lim
ited
.
Fig
u
r
e
6
illu
s
tr
ates
th
e
tr
ain
in
g
an
d
v
ali
d
atio
n
lo
s
s
f
o
r
th
ese
f
o
u
r
m
o
d
els.
I
n
co
n
tr
ast,
th
e
MBERT
b
ase
u
n
ca
s
ed
m
o
d
el
(
Fig
u
r
e
6
(
a
)
)
m
ain
tain
e
d
r
elat
iv
ely
lo
w
v
alid
atio
n
lo
s
s
f
r
o
m
th
e
s
ec
o
n
d
e
p
o
ch
Evaluation Warning : The document was created with Spire.PDF for Python.