I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
1
5
,
No
.
2
,
A
p
r
il 2
0
2
6
,
p
p
.
1
8
2
5
~
1
8
3
8
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
2
.
p
p
1
8
2
5
-
1
8
3
8
1825
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
A
hy
brid
mo
del
f
o
r
enha
nced
a
spe
ct
-
ba
sed
se
ntime
n
t
a
na
ly
sis
using
la
rg
e
la
ng
ua
g
e
mo
dels
M
o
ha
m
m
ed
Z
ia
ull
a
,
Arun
B
ira
da
r
S
c
h
o
o
l
of
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
a
n
d
E
n
g
i
n
e
e
r
i
n
g
,
R
EV
A
U
n
i
v
e
r
si
t
y
,
B
e
n
g
a
l
u
r
u
,
I
n
d
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
2
0
,
2
0
2
5
R
ev
is
ed
J
an
1
2
,
2
0
2
6
Acc
ep
ted
J
an
2
5
,
2
0
2
6
As
p
e
c
t
-
b
a
se
d
se
n
ti
m
e
n
t
a
n
a
ly
sis
(ABSA)
is
a
c
ru
c
ial
tas
k
with
in
n
a
t
u
ra
l
lan
g
u
a
g
e
p
r
o
c
e
ss
in
g
(NLP
),
e
n
a
b
li
n
g
fin
e
-
g
ra
in
e
d
o
p
in
io
n
m
in
in
g
by
id
e
n
ti
f
y
in
g
se
n
ti
m
e
n
ts
a
ss
o
c
iate
d
with
sp
e
c
ifi
c
a
sp
e
c
ts
of
a
p
ro
d
u
c
t
or
se
rv
ice
.
Wh
il
e
tran
sfo
rm
e
r
-
b
a
se
d
m
o
d
e
ls
l
ik
e
b
id
irec
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
re
se
n
tatio
n
s
fr
o
m
tran
sf
o
rm
e
rs
(
BERT
)
h
a
v
e
imp
ro
v
e
d
se
n
ti
m
e
n
t
c
las
sifica
ti
o
n
,
t
h
e
y
stil
l
stru
g
g
le
with
li
m
it
e
d
c
o
n
tex
tu
a
l
a
d
a
p
tab
il
it
y
,
e
sp
e
c
ially
in
c
u
st
o
m
e
r
re
v
iew
s
c
o
n
tai
n
i
n
g
c
o
m
p
lex
e
x
p
re
ss
io
n
s.
M
o
st
e
x
isti
n
g
a
p
p
ro
a
c
h
e
s
re
l
y
h
e
a
v
il
y
on
b
e
n
c
h
m
a
rk
d
a
tas
e
ts
su
c
h
as
se
m
a
n
ti
c
e
v
a
lu
a
ti
o
n
(
S
e
m
Ev
a
l
)
a
n
d
m
u
lt
i
-
a
sp
e
c
t
m
u
lt
i
-
se
n
ti
m
e
n
t
(
M
A
M
S
)
,
wh
ich
do
not
f
u
ll
y
c
a
p
tu
re
t
h
e
d
i
v
e
rsity
of
re
a
l
-
wo
rld
re
v
iew
sc
e
n
a
rio
s.
H
e
n
c
e
,
th
is
re
se
a
rc
h
a
d
d
re
ss
e
s
th
e
se
li
m
it
a
ti
o
n
s
by
p
ro
p
o
sin
g
a
n
o
v
e
l
h
y
b
r
i
d
m
o
d
e
l,
c
a
ll
e
d
as
h
y
b
rid
-
B
ERT
(H
-
BE
RT),
th
a
t
in
te
g
ra
tes
sp
a
n
-
a
wa
re
BERT
(
S
p
a
n
BERT
)
wi
th
b
id
irec
ti
o
n
a
l
lo
n
g
sh
o
rt
-
term
m
e
m
o
ry
(
BiL
S
TM
)
,
c
o
n
d
i
ti
o
n
a
l
ra
n
d
o
m
f
ield
(
CR
F
)
,
a
n
d
larg
e
la
n
g
u
a
g
e
m
o
d
e
ls
(LL
M
s).
T
h
e
o
b
jec
ti
v
e
is
to
e
n
h
a
n
c
e
a
sp
e
c
t
e
x
trac
ti
o
n
a
n
d
se
n
ti
m
e
n
t
c
las
sifica
ti
o
n
p
e
rfo
rm
a
n
c
e
u
si
n
g
b
o
th
a
n
n
o
tat
e
d
a
n
d
sy
n
t
h
e
ti
c
d
a
ta.
Th
e
m
e
th
o
d
o
l
o
g
y
in
c
lu
d
e
s
p
re
p
ro
c
e
ss
in
g
,
h
y
b
ri
d
m
o
d
e
l
train
i
n
g
,
a
n
d
e
v
a
lu
a
ti
o
n
u
sin
g
t
h
e
S
e
m
Ev
a
l
2
0
1
4
d
a
tas
e
t.
Ex
p
e
rim
e
n
tal
re
su
lt
s
sh
o
w
t
h
a
t
H
-
BERT
a
c
h
iev
e
d
9
0
.
5
8
%
a
c
c
u
ra
c
y
a
n
d
9
0
.
5
6
%
F
-
sc
o
re
in
t
h
e
lap
to
p
d
o
m
a
in
a
n
d
9
1
.
2
1
%
a
c
c
u
ra
c
y
with
a
9
2
.
0
3
%
F
-
sc
o
r
e
in
th
e
re
sta
u
ra
n
t
d
o
m
a
in
.
Th
e
se
re
su
lt
s
o
u
t
p
e
rfo
rm
e
x
ist
in
g
m
o
d
e
ls,
c
o
n
firmin
g
H
-
BERT
’s
ro
b
u
st
n
e
ss
a
n
d
e
ffe
c
ti
v
e
n
e
ss
.
In
c
o
n
c
lu
si
o
n
,
H
-
B
ERT
imp
ro
v
e
s
se
n
ti
m
e
n
t
u
n
d
e
rst
a
n
d
in
g
in
c
u
sto
m
e
r
re
v
iew
s.
K
ey
w
o
r
d
s
:
Asp
ec
t
-
b
ased
s
en
tim
en
t a
n
aly
s
is
B
E
R
T
C
u
s
to
m
er
r
ev
iews
Gem
in
i
Hy
b
r
id
B
E
R
T
Natu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
Sen
tim
en
t a
n
aly
s
is
T
h
is
is
an
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r
th
e
CC
BY
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Mo
h
am
m
ed
Z
iau
lla
Sch
o
o
l
of
C
o
m
p
u
ter
Scien
ce
a
n
d
E
n
g
in
ee
r
in
g
,
R
E
VA
Un
iv
er
s
ity
B
en
g
alu
r
u
,
I
n
d
ia
E
m
ail:
m
d
ziau
lla.
d
r
@
r
ed
if
f
m
ail.
co
m
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
n
a
tu
r
a
l
lan
g
u
ag
e
p
r
o
ce
s
s
i
n
g
(
NL
P)
is
a
r
ap
id
l
y
ev
o
lv
in
g
f
i
el
d
in
ar
tif
ic
ia
l
in
te
ll
ig
en
c
e
(
AI
)
th
at
f
o
cu
s
e
s
on
en
ab
lin
g
m
ac
h
in
e
s
to
u
n
d
e
r
s
tan
d
,
in
ter
p
r
et
,
an
d
g
en
er
a
te
h
u
m
an
lan
g
u
ag
e
.
W
ith
ap
p
l
ica
t
io
n
s
r
an
g
in
g
f
r
o
m
m
a
ch
i
n
e
tr
an
s
l
at
io
n
an
d
q
u
e
s
t
io
n
an
s
w
e
r
in
g
to
co
n
v
er
s
at
io
n
a
l
ag
en
ts
an
d
d
o
cu
m
en
t
s
u
m
m
ar
iza
t
io
n
,
NL
P
s
er
v
e
s
as
a
cr
it
ic
al
b
r
id
g
e
b
et
w
e
en
h
u
m
an
c
o
m
m
u
n
ic
at
io
n
an
d
co
m
p
u
t
at
io
n
al
u
n
d
er
s
t
an
d
in
g
[
1
]
,
[
2
]
.
As
d
i
g
it
al
co
n
ten
t
co
n
tin
u
e
s
to
g
r
o
w,
th
e
ab
i
li
ty
to
au
t
o
m
a
ti
ca
lly
ex
tr
a
ct
in
s
ig
h
ts
f
r
o
m
t
ex
t
h
a
s
b
ec
o
m
e
in
cr
e
a
s
i
n
g
ly
v
a
lu
ab
l
e
a
cr
o
s
s
in
d
u
s
tr
ie
s
s
u
ch
as
e
-
co
m
m
er
c
e,
h
ea
lth
ca
r
e
,
an
d
cu
s
to
m
er
s
er
v
ic
e.
On
e
of
th
e
c
o
r
e
t
a
s
k
s
in
N
L
P
is
s
e
n
t
im
en
t
an
aly
s
is
,
w
h
i
ch
in
v
o
lv
e
s
id
en
t
if
y
in
g
t
h
e
em
o
t
io
n
al
to
n
e
or
o
p
in
io
n
co
n
v
ey
ed
in
a
p
ie
ce
of
t
ex
t
[
3
]
.
T
r
ad
i
t
io
n
a
l
s
e
n
tim
en
t
an
aly
s
i
s
tec
h
n
iq
u
e
s
o
f
ten
c
la
s
s
if
y
en
t
ir
e
s
en
ten
ce
s
or
d
o
cu
m
en
t
s
i
n
to
c
ateg
o
r
i
e
s
s
u
c
h
as
p
o
s
it
iv
e,
n
e
g
at
iv
e,
or
n
eu
tr
al
[
3
]
.
W
h
il
e
e
f
f
ec
tiv
e
f
o
r
g
en
e
r
a
l
o
v
er
v
i
ew
s
,
t
h
e
s
e
m
e
th
o
d
s
f
a
ll
s
h
o
r
t
in
s
ce
n
ar
io
s
wh
er
e
u
s
e
r
s
ex
p
r
e
s
s
m
ix
ed
o
p
in
io
n
s
ab
o
u
t
d
if
f
e
r
en
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
2
,
Ap
r
il 2
0
2
6
:
1
8
2
5
-
1
8
3
8
1826
co
m
p
o
n
en
t
s
w
i
th
in
th
e
s
a
m
e
r
ev
i
ew
—
f
o
r
in
s
t
an
c
e,
l
ik
in
g
th
e
p
r
o
d
u
ct
q
u
al
i
ty
but
d
i
s
l
ik
in
g
th
e
d
el
iv
e
r
y
s
er
v
ic
e.
T
h
i
s
l
im
i
ta
t
io
n
g
av
e
r
i
s
e
to
a
more
f
in
e
-
g
r
a
in
ed
te
ch
n
iq
u
e
k
n
o
wn
as
a
s
p
ec
t
-
b
a
s
ed
s
e
n
t
im
en
t
an
al
y
s
i
s
(
A
B
SA)
.
A
B
S
A
a
i
m
s
to
i
d
en
t
i
f
y
s
p
e
ci
f
i
c
a
s
p
e
c
t
s
m
e
n
t
io
n
e
d
in
a
t
e
x
t
(
e.
g
.
,
“
b
a
t
t
e
r
y
l
i
f
e
”
an
d
“
d
i
s
p
l
ay
q
u
a
l
i
ty
”
)
an
d
d
e
t
e
r
m
i
n
e
s
en
t
im
e
n
t
ex
p
r
e
s
s
e
d
t
o
w
ar
d
s
e
a
ch
a
s
p
ec
t
.
T
h
i
s
en
a
b
le
s
a
d
e
ep
er
u
n
d
e
r
s
t
an
d
in
g
of
c
u
s
t
o
m
er
f
e
e
d
b
a
ck
by
a
s
s
o
ci
a
t
i
n
g
s
e
n
t
i
m
en
t
s
w
i
th
p
a
r
t
i
cu
l
a
r
p
r
o
d
u
c
t
or
s
e
r
v
ic
e
f
e
a
tu
r
e
s
.
De
s
p
i
t
e
it
i
s
g
r
o
w
in
g
s
i
g
n
i
f
ic
a
n
ce
,
A
B
S
A
r
e
m
a
i
n
s
a
ch
a
l
l
en
g
in
g
t
as
k
due
to
t
h
e
co
m
p
le
x
i
ty
of
n
a
t
u
r
a
l
l
an
g
u
a
g
e,
i
n
c
l
u
d
i
n
g
e
x
p
l
i
c
i
t
e
x
p
r
e
s
s
i
o
n
s
,
d
o
m
a
in
-
s
p
e
c
i
f
ic
te
r
m
in
o
lo
g
y
,
an
d
m
u
l
t
i
p
le
s
en
t
i
m
en
t
-
b
e
a
r
i
n
g
c
o
m
p
o
n
en
t
s
in
a
s
i
n
g
le
s
e
n
t
en
c
e
[
4
]
,
[
5
]
.
T
r
a
d
i
t
io
n
a
l
A
B
S
A
m
o
d
e
l
s
h
av
e
i
n
cr
e
a
s
i
n
g
l
y
em
p
lo
y
ed
b
id
i
r
e
c
t
i
o
n
a
l
en
c
o
d
e
r
r
e
p
r
e
s
e
n
t
a
t
io
n
s
f
r
o
m
t
r
an
s
f
o
r
m
e
r
s
(
B
E
R
T
)
d
u
e
to
it
i
s
a
b
i
l
i
t
y
to
c
ap
t
u
r
e
d
e
ep
c
o
n
te
x
t
u
a
l
i
n
f
o
r
m
a
t
io
n
[
6
]
−
[
8
]
.
Ho
w
ev
e
r
,
B
E
R
T
was
d
e
s
i
g
n
ed
f
o
r
g
e
n
e
r
a
l
-
p
u
r
p
o
s
e
l
a
n
g
u
a
g
e
u
n
d
er
s
t
a
n
d
i
n
g
an
d
is
n
o
t
i
n
h
er
e
n
t
ly
d
e
s
i
g
n
e
d
f
o
r
a
s
p
e
c
t
-
l
e
v
e
l
s
en
t
i
m
en
t
an
a
l
y
s
i
s
.
B
E
R
T
o
f
t
en
f
a
i
l
s
to
d
i
s
t
i
n
g
u
i
s
h
s
e
n
t
i
m
e
n
t
p
o
l
ar
i
t
y
w
h
en
m
u
l
t
i
p
le
a
s
p
e
c
t
s
a
r
e
p
r
e
s
e
n
t
,
a
n
d
it
r
eq
u
ir
e
s
ex
p
l
i
c
i
t
g
u
i
d
a
n
ce
to
f
o
c
u
s
on
a
s
p
ec
t
-
s
p
e
c
if
i
c
co
n
te
n
t
.
A
d
d
i
t
io
n
a
l
ly
,
B
E
R
T
’
s
p
r
e
-
t
r
a
i
n
i
n
g
on
g
e
n
er
i
c
c
o
r
p
o
r
a
li
m
i
t
s
its
c
a
p
ac
i
t
y
to
in
t
e
r
p
r
e
t
d
o
m
a
in
-
s
p
e
c
i
f
ic
p
h
r
a
s
e
s
,
s
u
ch
as
i
d
io
m
a
t
ic
ex
p
r
e
s
s
i
o
n
s
f
o
u
n
d
in
u
s
e
r
r
e
v
i
ew
s
.
T
h
e
s
e
l
i
m
i
t
a
t
io
n
s
m
ak
e
B
E
R
T
-
b
a
s
ed
m
o
d
e
l
s
l
e
s
s
e
f
f
e
c
t
iv
e
f
o
r
i
m
p
o
r
t
a
n
t
AB
S
A
t
a
s
k
s
.
An
o
th
er
ch
allen
g
e
in
AB
SA
r
esear
ch
is
th
e
r
elian
ce
on
tr
ad
itio
n
al
d
atasets
lik
e
s
em
an
tic
ev
alu
atio
n
(
Sem
E
v
al
)
,
m
u
lti
-
asp
ec
t
m
u
lti
-
s
en
tim
en
t
(
MA
MS)
,
an
d
d
o
m
ain
-
o
r
ien
ted
tar
g
eted
s
en
tim
en
t
an
aly
s
is
(
DOT
SA
)
,
wh
ich
a
r
e
lim
ited
in
s
ize,
d
o
m
ain
d
iv
er
s
ity
,
an
d
r
ea
l
-
wo
r
ld
v
a
r
ian
ce
.
Su
ch
d
a
tasets
o
f
ten
do
n
o
t
r
ef
lect
th
e
wid
e
r
an
g
e
of
c
u
s
to
m
er
o
p
in
io
n
s
f
o
u
n
d
in
m
o
d
er
n
d
ig
ital
p
latf
o
r
m
s
.
To
ad
d
r
ess
th
is
,
r
ec
en
t
r
esear
ch
h
as
ex
p
lo
r
ed
u
s
in
g
lar
g
e
lan
g
u
ag
e
m
o
d
els
(
L
L
Ms)
to
g
en
er
ate
s
y
n
th
etic
d
atasets
th
at
s
im
u
late
r
ea
l
-
wo
r
ld
r
ev
iews
[
9
]
,
[
1
0
]
.
W
h
ile
th
ese
LLM
-
g
en
er
ated
d
atasets
p
r
o
v
id
e
s
u
p
p
lem
en
ta
l
d
ata,
th
e
m
o
d
els
u
s
ed
f
o
r
AB
SA
s
till
s
tr
u
g
g
le
to
g
en
er
alize
well
ac
r
o
s
s
d
o
m
ain
s
an
d
m
ai
n
tain
ac
cu
r
ac
y
in
id
en
tify
in
g
asp
ec
t
-
s
en
tim
en
t
p
air
s
.
To
ad
d
r
ess
th
ese
g
ap
s
,
th
i
s
wo
r
k
p
r
o
p
o
s
es
a
h
y
b
r
id
-
B
E
R
T
(H
-
B
E
R
T
)
f
r
am
ewo
r
k
f
o
r
AB
SA.
H
-
B
E
R
T
co
m
b
in
es
th
e
co
n
tex
tu
al
p
o
wer
of
s
p
an
-
a
war
e
(
Sp
an
B
E
R
T
)
f
o
r
s
p
an
-
b
a
s
ed
asp
ec
t
d
etec
tio
n
,
b
id
ir
ec
tio
n
a
l
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
B
iLST
M)
f
o
r
m
o
d
el
in
g
s
eq
u
en
tial
d
ep
en
d
en
cies,
co
n
d
itio
n
al
r
an
d
o
m
f
ield
s
(
C
R
F
s
)
f
o
r
s
tr
u
ctu
r
ed
o
u
tp
u
t,
an
d
LLMs
f
o
r
au
x
iliar
y
s
u
p
er
v
is
io
n
.
T
h
is
m
u
lti
-
co
m
p
o
n
en
t
ar
ch
itectu
r
e
en
h
an
ce
s
th
e
ab
ilit
y
to
ex
tr
ac
t
asp
ec
t
ter
m
s
,
d
eter
m
in
e
th
eir
ass
o
ciate
d
p
o
lar
ities
,
an
d
class
if
y
s
en
ten
ce
-
lev
el
s
en
tim
en
ts
.
Ad
d
itio
n
ally
,
th
e
u
s
e
of
b
o
th
h
u
m
an
-
a
n
n
o
tate
d
(
Sem
E
v
al)
a
n
d
s
y
n
th
etic
d
ata
s
ets
(
C
h
atGPT
an
d
Gem
in
i)
im
p
r
o
v
es
r
o
b
u
s
tn
ess
an
d
g
en
e
r
aliza
tio
n
of
th
e
m
o
d
el
in
cu
s
to
m
er
r
ev
ie
w
s
ce
n
ar
io
s
.
T
h
e
co
n
tr
ib
u
tio
n
s
of
th
e
w
o
r
k
a
r
e
as
f
o
llo
ws
.
A
n
o
v
el
h
y
b
r
i
d
m
o
d
el
in
teg
r
a
tin
g
Sp
an
B
E
R
T
,
B
iLST
M,
C
R
Fs
,
an
d
LLMs
f
o
r
im
p
r
o
v
e
d
AB
SA
is
p
r
esen
ted
in
th
is
wo
r
k
.
Sy
n
th
etic
cu
s
to
m
er
r
ev
iew
d
atasets
wer
e
g
en
er
ated
u
s
in
g
C
h
atGPT
-
3
.
5
-
T
u
r
b
o
an
d
Gem
in
i
-
2
.
5
-
Flas
h
to
s
u
p
p
lem
en
t
tr
ad
itio
n
al
d
atasets
an
d
e
n
h
an
ce
g
e
n
er
aliza
tio
n
.
A
p
r
e
p
r
o
ce
s
s
in
g
s
tep
is
in
clu
d
ed
f
o
r
e
n
s
u
r
in
g
c
o
n
s
is
ten
cy
ac
r
o
s
s
d
atasets
an
d
p
r
e
p
ar
in
g
d
ata
f
o
r
d
ee
p
co
n
tex
t
u
al
an
d
s
eq
u
e
n
tial
m
o
d
elin
g
.
T
h
e
m
o
d
el
jo
in
tl
y
p
er
f
o
r
m
s
asp
ec
t
ex
tr
ac
tio
n
,
asp
ec
t
p
o
lar
ity
d
etec
tio
n
,
an
d
s
en
ten
ce
-
lev
el
s
en
tim
en
t
class
if
icat
io
n
.
T
h
e
m
o
d
el
was
ev
alu
ated
u
s
in
g
a
cc
u
r
ac
y
an
d
m
ac
r
o
-
F
-
s
co
r
e
on
b
o
th
tr
ad
itio
n
al
(
Sem
E
v
al)
an
d
LLM
-
g
e
n
er
at
ed
d
atasets
,
d
em
o
n
s
tr
atin
g
im
p
r
o
v
e
d
p
e
r
f
o
r
m
an
ce
o
v
er
s
t
an
d
ar
d
B
E
R
T
-
b
ased
m
eth
o
d
s
.
T
h
e
m
an
u
s
cr
ip
t
is
o
r
g
an
ized
in
th
e
f
o
llo
win
g
m
a
n
n
er
.
Sectio
n
2
p
r
esen
ts
liter
atu
r
e
s
u
r
v
ey
wh
ich
d
is
cu
s
s
es
ex
i
s
tin
g
AB
SA
ap
p
r
o
ac
h
es
an
d
LLM
s
y
n
th
etic
d
a
ta
g
en
er
atio
n
ap
p
r
o
ac
h
es.
Sectio
n
3
p
r
esen
ts
th
e
m
eth
o
d
o
l
o
g
y
f
o
r
t
h
e
H
-
B
E
R
T
m
o
d
el,
Sectio
n
4
p
r
esen
ts
th
e
r
esu
lts
of
H
-
B
E
R
T
an
d
co
m
p
ar
es
with
ex
is
tin
g
ap
p
r
o
ac
h
es.
Sectio
n
5
p
r
esen
ts
th
e
co
n
clu
s
io
n
an
d
f
u
tu
r
e
wo
r
k
of
H
-
B
E
R
T
.
2.
L
I
T
E
R
AT
U
RE
SU
RVE
Y
T
h
is
s
ec
tio
n
d
is
cu
s
s
es
ex
is
tin
g
AB
SA
ap
p
r
o
ac
h
es,
LLM
ap
p
r
o
ac
h
es
,
an
d
LLM
ap
p
r
o
ac
h
es
u
s
ed
f
o
r
g
en
er
atin
g
s
y
n
t
h
etic
d
atasets
.
Gu
et
a
l.
[
1
1
]
aim
e
d
at
en
h
an
c
in
g
asp
ec
t
-
lev
el
s
en
tim
en
t
-
an
a
ly
s
is
by
ad
d
r
ess
in
g
lim
itatio
n
s
in
ex
is
tin
g
g
r
ap
h
c
o
n
v
o
lu
ti
o
n
al
n
etwo
r
k
s
(
GC
N)
b
ased
a
p
p
r
o
ac
h
es,
lik
e
i
n
s
u
f
f
icien
t
u
tili
za
tio
n
of
asp
ec
t
-
s
p
ec
if
ic
in
f
o
r
m
atio
n
a
n
d
lack
of
ex
ter
n
al
s
en
tim
en
t
k
n
o
wled
g
e.
Hen
ce
,
p
r
ese
n
ted
s
y
n
tax
-
awa
r
e
g
r
ap
h
co
n
v
o
lu
tio
n
al
n
etwo
r
k
(
SAGC
N)
,
wh
ich
in
teg
r
ated
asp
ec
t
-
lev
el
f
ea
tu
r
e
in
t
o
co
n
tex
t
u
al
r
ep
r
esen
tatio
n
s
an
d
in
co
r
p
o
r
ate
d
ex
ter
n
al
s
en
tim
en
t
lex
ico
n
s
f
o
r
e
n
r
ich
in
g
s
en
tim
en
t
p
er
ce
p
tio
n
.
T
h
is
wo
r
k
also
em
p
lo
y
ed
m
u
lti
-
h
ea
d
s
elf
-
atten
tio
n
(
M
HSA)
ap
p
r
o
ac
h
alo
n
g
with
a
p
o
in
t
-
wis
e
co
n
v
o
lu
tio
n
al
-
tr
a
n
s
f
o
r
m
e
r
(
PC
T
)
f
o
r
jo
in
tly
ca
p
tu
r
in
g
s
em
an
tic
-
s
y
n
tactic
r
elatio
n
s
h
ip
s
.
Fo
r
ev
alu
atio
n
of
SAGC
N,
th
r
ee
d
atasets
,
i.e
.
,
AC
L
14
-
T
ask
T
witter
d
ataset
an
d
Sem
E
v
al
2
0
1
4
r
estau
r
a
n
t
a
n
d
Sem
E
v
al
2
0
1
4
lap
t
o
p
d
ataset
wer
e
co
n
s
id
er
ed
,
wh
er
e
ac
h
iev
e
d
7
7
.
9
7
%,
8
7
.
5
3
%
,
an
d
8
3
.
0
6
%
ac
cu
r
ac
y
.
J
eo
n
g
an
d
L
ee
[
1
2
]
aim
e
d
at
e
n
h
a
n
cin
g
asp
ec
t
-
b
ased
an
aly
s
is
of
h
o
tel
r
ev
iew
by
u
ti
lizin
g
C
h
atGPT
,
an
LLM
m
o
d
el,
f
o
r
o
v
er
co
m
in
g
ch
allen
g
es
in
in
ter
p
r
etatio
n
of
am
b
ig
u
o
u
s
an
d
c
o
m
p
lex
c
u
s
to
m
er
f
ee
d
b
ac
k
.
In
th
is
wo
r
k
,
th
ey
u
tili
ze
d
T
r
i
p
Ad
v
is
o
r
d
ataset,
wh
er
e
th
eir
ap
p
r
o
ac
h
in
v
o
lv
ed
i
d
en
tific
at
io
n
of
te
n
k
e
y
h
o
tel
attr
ib
u
te
s
an
d
g
en
e
r
atio
n
of
asp
ec
t
-
s
u
m
m
ar
izatio
n
p
air
s
h
av
in
g
d
esig
n
ed
p
r
o
m
p
ts
f
o
r
ef
f
icien
t
an
aly
s
is
.
T
h
e
C
h
atGPT
’
s
o
u
tp
u
ts
wer
e
ev
alu
ated
q
u
alitativ
ely
,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
en
h
a
n
ce
d
a
s
p
ec
t
-
b
a
s
ed
s
en
timen
t a
n
a
lysi
s
u
s
in
g
la
r
g
e
la
n
g
u
a
g
e
…
(
Mo
h
a
mme
d
Zia
u
lla
)
1827
f
o
cu
s
in
g
on
t
h
e
ab
ilit
y
of
e
x
tr
ac
tin
g
ex
p
licit
k
e
y
wo
r
d
s
an
d
s
u
m
m
ar
izin
g
s
en
tim
en
t
-
r
ich
co
n
ten
t.
T
h
e
f
in
d
i
n
g
s
s
h
o
wed
th
at
C
h
atGPT
ca
p
tu
r
ed
im
p
o
r
tan
t
in
f
o
r
m
atio
n
an
d
d
is
tin
g
u
is
h
ed
s
en
tim
en
t
p
a
tter
n
s
ac
r
o
s
s
h
o
tel
ca
teg
o
r
ies.
T
h
e
r
esu
lts
s
h
o
w
i
m
p
r
o
v
e
d
ac
cu
r
ac
y
an
d
co
n
tex
t
u
al
u
n
d
e
r
s
tan
d
in
g
.
Z
h
an
g
et
a
l.
[
1
3
]
f
o
cu
s
ed
on
im
p
r
o
v
i
n
g
AB
SA
by
ad
d
r
ess
in
g
lim
itatio
n
in
e
x
is
tin
g
g
r
a
p
h
-
b
ased
an
d
atten
tio
n
-
b
ased
m
o
d
el,
h
en
c
e,
p
r
o
p
o
s
ed
s
y
n
tactic
-
d
ep
en
d
en
cy
g
r
ap
h
co
n
v
o
lu
tio
n
al
n
etwo
r
k
(
SD
-
GC
N)
,
wh
ich
aim
ed
at
ca
p
tu
r
in
g
lo
n
g
-
r
a
n
g
e
s
y
n
tactic
-
r
elatio
n
s
h
ip
s
an
d
d
ep
e
n
d
en
c
y
b
etwe
en
o
p
in
io
n
-
wo
r
d
s
an
d
asp
ec
t
-
ter
m
s
.
T
h
is
wo
r
k
u
tili
ze
d
B
iaf
f
in
e
-
Atten
tio
n
,
wh
er
e
co
n
s
tr
u
cted
s
y
n
tactic
-
d
e
p
en
d
en
c
y
g
r
ap
h
s
f
o
r
r
ep
r
esen
tin
g
co
n
n
ec
tio
n
am
o
n
g
s
en
tim
en
t
ex
p
r
ess
io
n
s
an
d
asp
ec
ts
.
Fu
r
th
er
,
GC
N
was
ap
p
lied
f
o
r
ex
tr
ac
tin
g
r
ich
s
y
n
tactic
-
s
em
an
tic
f
ea
tu
r
es.
E
v
alu
atio
n
s
wer
e
co
n
d
u
cted
on
Sem
E
v
a
l
2014
R
estau
r
an
t
,
Sem
E
v
al
2014
L
ap
to
p
,
Sem
E
v
al
2
0
1
5
R
estau
r
an
t
,
an
d
T
witter
d
ataset,
wh
er
e
ac
h
iev
ed
8
8
.
1
4
%,
8
0
.
3
5
%
,
8
5
.
3
0
%
,
an
d
7
7
.
6
3
%
ac
cu
r
ac
y
r
esp
ec
tiv
ely
.
Mu
g
h
al
et
a
l.
[
1
4
]
aim
ed
at
ad
d
r
ess
in
g
ch
allen
g
es
in
AB
SA,
m
ain
ly
in
d
ata
d
ep
en
d
en
cy
,
s
en
s
itiv
ity
an
d
lim
ited
u
s
ag
e
of
L
L
Ms.
Un
lik
e
tr
ad
itio
n
al
s
en
tim
en
t
-
b
ased
ap
p
r
o
ac
h
es,
wh
ich
m
ain
ly
f
o
cu
s
ed
on
d
o
cu
m
e
n
t
or
s
en
ti
m
en
t
lev
el,
AB
SA
lin
k
s
s
en
tim
en
ts
to
s
p
ec
if
ic
asp
ec
ts
.
In
th
is
wo
r
k
,
th
ey
ev
alu
ated
v
ar
io
u
s
d
ee
p
lea
r
n
in
g
(
DL
)
an
d
LLM
m
o
d
e
ls
,
wh
ich
in
clu
d
ed
g
en
er
ativ
e
-
p
r
e
tr
ain
in
g
-
3
.
5
-
tu
r
b
o
(
GPR
-
3
.
5
-
T
u
r
b
o
)
,
p
at
h
way
s
-
lan
g
u
ag
e
m
o
d
els
(
PaL
M)
,
d
ec
o
d
in
g
e
n
h
an
ce
d
b
id
ir
ec
tio
n
al
-
en
c
o
d
er
-
r
ep
r
esen
tatio
n
tr
an
s
f
o
r
m
er
s
(
DeBERTa)
,
f
in
e
-
tu
n
ed
lan
g
u
ag
e
-
T5
(
FLAN
-
T
5
)
a
n
d
atten
tio
n
-
b
ased
asp
ec
t
-
ex
tr
ac
tio
n
lo
n
g
s
h
o
r
t
-
ter
m
m
em
o
r
y
(
AT
AE
-
L
STM
)
,
co
n
s
id
er
in
g
Sem
E
v
al2
0
1
6
,
MA
MS,
an
d
DOT
SA,
wh
er
e
DeBERTa
s
h
o
wed
b
etter
p
er
f
o
r
m
an
ce
,
wh
ile
PaL
M
s
h
o
we
d
b
etter
p
er
f
o
r
m
an
c
e
in
asp
ec
t
-
ter
m
s
en
tim
en
t
an
aly
s
is
.
Fa
la
to
u
r
i
et
a
l.
[
1
5
]
in
v
e
s
t
ig
ated
ef
f
ec
tiv
en
e
s
s
of
L
L
M
in
en
h
an
ce
m
en
t
of
s
e
r
v
i
ce
-
q
u
al
ity
an
d
s
en
ti
m
en
t
-
an
al
y
s
i
s
d
im
en
s
io
n
ex
tr
a
ct
io
n
f
r
o
m
u
s
er
-
g
e
n
er
a
ted
co
n
ten
t.
T
h
e
p
r
i
m
ar
y
o
b
jec
t
iv
e
of
th
i
s
wo
r
k
wa
s
to
ev
alu
at
ed
C
lau
d
e3
an
d
C
h
at
G
PT
-
3
.
5
ag
ain
s
t
th
r
e
e
N
L
P
ap
p
r
o
ac
h
e
s
u
s
in
g
b
il
in
g
u
a
l
cu
s
to
m
er
r
ev
i
ew
d
a
ta
s
et
s
in
Pe
r
s
ian
a
n
d
E
n
g
l
is
h
.
T
h
e
m
eth
o
d
o
lo
g
y
in
v
o
lv
ed
co
m
p
ar
i
n
g
m
o
d
e
l
p
er
f
o
r
m
an
ce
on
s
en
ti
m
en
t
-
c
la
s
s
if
i
ca
tio
n
an
d
s
tr
u
c
tu
r
e
in
f
o
r
m
at
io
n
ex
tr
ac
t
io
n
.
T
h
e
r
e
s
u
l
t
s
s
h
o
wed
th
at
C
h
at
GP
T
ac
h
iev
ed
76%
ac
c
u
r
a
cy
an
d
s
u
b
s
t
an
t
ia
l
ag
r
e
em
en
t
wi
th
h
u
m
an
r
a
ter
s
,
wh
er
e
as
C
l
au
d
e3
ac
h
ie
v
ed
6
8
%
ac
cu
r
ac
y
w
i
th
m
o
d
er
a
te
ag
r
ee
m
en
t
.
D
e
s
p
i
te
o
u
tp
e
r
f
o
r
m
in
g
tr
ad
it
io
n
al
ap
p
r
o
ac
h
e
s
,
b
o
th
L
L
M
s
s
h
o
we
d
in
co
n
s
i
s
t
en
c
ie
s
in
f
in
e
-
g
r
a
in
ed
ex
tr
a
ct
io
n
.
L
iu
et
a
l.
[
1
6
]
,
f
o
cu
s
ed
on
im
p
r
o
v
in
g
a
s
p
e
ct
-
o
p
in
io
n
s
en
t
im
en
t
t
r
i
p
le
t
ex
tr
ac
tio
n
f
o
r
s
en
ti
m
en
t
an
a
ly
s
i
s
by
ad
d
r
es
s
in
g
l
im
i
ta
ti
o
n
of
tr
ad
it
i
o
n
al
p
ip
el
in
e
a
n
d
tag
g
in
g
-
b
as
ed
ap
p
r
o
ac
h
es
,
wh
i
ch
o
f
t
en
f
ai
l
f
o
r
ca
p
tu
r
in
g
d
e
ep
s
y
n
ta
ct
ic
-
s
em
an
ti
c
r
e
l
at
io
n
s
h
ip
s
,
h
e
n
ce
,
p
r
e
s
en
ted
s
y
n
tac
t
ic
-
s
e
m
an
ti
c
asp
ec
t
-
s
en
ti
m
en
t
t
er
m
-
ex
t
r
a
ct
io
n
(
Sy
n
Sem
-
A
ST
E
)
,
a
m
u
l
ti
-
en
co
d
er
ap
p
r
o
a
ch
wh
i
ch
in
t
eg
r
a
ted
s
y
n
tac
t
ic
-
s
e
m
an
ti
c
en
co
d
in
g
f
o
r
ca
p
tu
r
in
g
s
tr
u
ct
u
r
a
l
an
d
co
n
tex
tu
a
l
d
ep
en
d
en
ci
e
s
.
T
h
e
ap
p
r
o
ac
h
a
l
s
o
in
co
r
p
o
r
a
ted
g
r
id
-
t
ag
g
in
g
ap
p
r
o
ac
h
f
o
r
en
ab
lin
g
ef
f
ec
t
iv
e
t
r
ip
l
et
ex
tr
a
ct
io
n
.
Sy
n
S
em
-
A
S
T
E
wa
s
ev
a
lu
a
ted
on
Sem
E
v
a
l
2016,
Sem
E
v
a
l
2015
,
a
n
d
S
em
E
v
al
2014
d
ata
s
et
s
,
wh
er
e
ac
h
ie
v
ed
m
a
cr
o
-
F
-
s
co
r
e
s
up
to
7
2
.
2
3
%
,
s
h
o
win
g
im
p
r
o
v
ed
e
x
tr
a
ct
io
n
ca
p
ab
i
li
ti
e
s
.
Hellwig
et
a
l.
[
1
7
]
in
v
esti
g
ate
d
ap
p
licatio
n
of
L
L
Ms,
m
ain
l
y
lar
g
e
lan
g
u
ag
e
m
o
d
el
m
eta
AI
-
3
-
7
0
B
(
Me
ta
L
lam
a
-
3
-
7
0
B
)
an
d
GPT
-
3
.
5
-
T
u
r
b
o
f
o
r
g
en
er
ati
n
g
a
n
n
o
tated
d
ata
f
o
r
AB
SA,
ad
d
r
ess
in
g
ch
allen
g
e
of
lim
ited
lab
eled
d
atasets
.
T
h
e
a
p
p
r
o
ac
h
in
v
o
lv
ed
f
ew
-
s
h
o
t
p
r
o
m
p
tin
g
f
o
r
cr
ea
tin
g
s
y
n
th
etic
tr
ain
in
g
d
ata
u
n
d
er
two
lo
w
-
r
eso
u
r
ce
s
ettin
g
with
25
an
d
5
0
0
m
an
u
al
-
lab
ele
d
ex
am
p
les.
Usi
n
g
d
atasets
f
o
r
asp
ec
t
-
ca
teg
o
r
y
s
en
tim
en
t
-
an
aly
s
is
(
AC
SA)
an
d
asp
ec
t
ca
te
g
o
r
y
d
etec
tio
n
(
AC
D)
,
th
e
f
in
d
i
n
g
s
s
h
o
wed
th
at
u
s
in
g
25
la
b
eled
ex
am
p
les,
F1
-
s
co
r
e
r
ea
c
h
ed
8
1
.
3
3
%
an
d
7
1
.
7
1
%
f
o
r
AC
SA
.
In
5
0
0
-
ex
a
m
p
le
s
ettin
g
,
s
y
n
t
h
etic
au
g
m
en
tatio
n
f
u
r
th
er
im
p
r
o
v
ed
AC
SA
p
er
f
o
r
m
an
ce
f
r
o
m
8
4
.
5
4
%
to
8
6
.
7
0
%.
Pan
d
ey
an
d
Sin
g
h
[
1
8
]
ad
d
r
ess
ed
lim
itatio
n
of
tr
ad
itio
n
al
p
r
o
d
u
ct
r
ev
iew
s
by
p
r
esen
tin
g
a
f
r
am
ewo
r
k
wh
ich
g
e
n
er
ated
d
etailed
te
x
tu
al
r
ev
iews
f
r
o
m
u
s
er
-
p
r
o
v
id
ed
asp
ec
t
-
wis
e
r
ati
n
g
s
u
s
in
g
L
L
Ms.
T
h
e
o
b
jectiv
e
of
th
is
wo
r
k
was
to
e
n
h
an
ce
co
m
p
lete
n
ess
an
d
q
u
ality
of
o
n
lin
e
r
e
v
iews,
wh
ich
o
f
ten
lac
k
co
v
e
r
ag
e
of
asp
ec
ts
.
T
h
e
m
eth
o
d
o
l
o
g
y
of
th
e
wo
r
k
in
v
o
lv
ed
m
ap
p
in
g
s
tr
u
ctu
r
e
L
ik
e
r
t
-
Scale
r
atin
g
f
o
r
co
h
er
en
t,
as
p
ec
t
-
r
ich
n
a
r
r
ativ
es.
T
h
e
w
o
r
k
was
ev
alu
ated
co
n
s
id
er
in
g
h
u
m
a
n
ju
d
g
em
en
t,
AI
-
g
en
e
r
ated
r
e
v
iews,
wh
ich
d
em
o
n
s
tr
ated
h
i
g
h
r
ele
v
an
ce
,
r
ea
d
ab
ilit
y
an
d
in
f
o
r
m
ativ
e
n
ess
,
wh
ich
was
o
f
ten
in
d
is
tin
g
u
is
h
ab
le
f
r
o
m
h
u
m
an
-
wr
itten
co
n
te
n
t.
T
h
e
f
in
d
in
g
s
s
h
o
wed
f
r
am
ewo
r
k
’
s
p
o
ten
tial
in
im
p
r
o
v
in
g
e
-
co
m
m
e
r
ce
r
ev
iew
s
y
s
tem
s
.
F
a
n
et
a
l
.
[
1
9
]
a
im
e
d
at
i
m
p
r
o
v
in
g
Mu
l
t
i
m
o
d
a
l
a
s
p
e
c
t
-
b
as
e
d
s
en
t
i
m
en
t
c
la
s
s
i
f
i
c
a
t
i
o
n
(
M
A
B
S
C
)
by
ad
d
r
e
s
s
i
n
g
l
i
m
i
ta
t
i
o
n
of
e
x
i
s
t
i
n
g
ap
p
r
o
ac
h
e
s
f
o
r
ca
p
t
u
r
in
g
co
n
te
x
t
f
r
o
m
s
o
c
i
a
l
-
m
e
d
ia
p
o
s
t
s
.
Fo
r
e
n
h
an
c
in
g
s
e
n
t
i
m
e
n
t
p
r
e
d
ic
t
i
o
n
,
th
i
s
w
o
r
k
p
r
e
s
e
n
te
d
m
u
l
t
i
-
m
o
d
a
l
d
u
a
l
c
au
s
e
a
n
a
l
y
s
i
s
(
M
D
C
A
)
a
r
c
h
i
t
ec
t
u
r
e
,
wh
i
c
h
in
t
r
o
d
u
ce
s
t
w
o
e
x
p
l
a
n
a
t
o
r
y
co
m
p
o
n
e
n
t
s
,
wh
i
c
h
i
n
c
lu
d
ed
d
ir
ec
t
-
c
a
u
s
e
(
D
C
)
an
d
r
e
a
s
o
n
i
n
g
-
c
au
s
e
(
R
C
)
,
f
o
r
i
d
e
n
t
i
f
y
i
n
g
u
n
d
er
l
y
in
g
m
o
t
i
v
a
ti
o
n
s
b
eh
i
n
d
u
s
e
r
s
en
t
i
m
en
t
s
.
U
s
i
n
g
LLMs
a
n
d
v
i
s
u
a
l
-
la
n
g
u
ag
e
m
o
d
e
l
s
,
M
AB
S
C
d
a
t
a
s
e
t
e
n
r
i
ch
e
d
w
i
t
h
RC
a
n
d
DC
l
a
b
e
l
s
w
e
r
e
c
o
n
s
t
r
u
c
t
e
d
.
A
m
u
l
t
i
-
t
a
s
k
l
e
a
r
n
in
g
b
en
ch
m
ar
k
M
A
B
S
C
d
a
t
a
s
e
t
s
s
h
o
w
t
h
a
t
M
D
C
A
o
u
t
p
er
f
o
r
m
s
e
x
i
s
t
i
n
g
a
p
p
r
o
a
c
h
e
s
.
Z
h
a
o
et
a
l
.
[
2
0
]
a
d
d
r
e
s
s
e
d
l
i
m
i
ta
t
i
o
n
in
DL
m
o
d
e
l
s
f
o
r
A
B
S
A
,
p
ar
t
i
c
u
l
ar
l
y
t
h
e
i
r
d
i
f
f
i
c
u
l
ty
in
a
d
a
p
t
in
g
to
v
a
r
y
i
n
g
s
e
n
t
im
e
n
t
c
l
a
s
s
e
s
a
n
d
m
o
d
e
l
in
g
c
a
t
eg
o
r
y
-
s
p
e
c
i
f
ic
in
f
o
r
m
a
t
io
n
,
h
e
n
ce
p
r
e
s
e
n
t
ed
B
E
R
T
a
p
p
r
o
a
c
h
w
i
t
h
c
l
a
s
s
-
a
w
a
r
e
w
o
r
k
M
H
S
A
(
B
E
R
T
-
CA
-
W
M
A
)
,
w
h
i
ch
in
t
r
o
d
u
c
ed
a
s
p
ec
t
-
p
r
o
j
e
c
t
io
n
l
ay
e
r
f
o
r
a
l
ig
n
in
g
a
s
p
e
c
t
-
e
m
b
e
d
d
i
n
g
w
i
t
h
c
o
n
t
e
x
tu
a
l
r
e
p
r
e
s
e
n
t
a
t
io
n
an
d
in
c
o
r
p
o
r
a
t
ed
s
e
n
t
im
e
n
t
c
l
a
s
s
i
n
f
o
r
m
a
t
i
o
n
u
s
in
g
d
y
n
a
m
i
c
a
t
te
n
t
io
n
w
e
i
g
h
t
in
g
.
T
h
e
B
E
R
T
-
CA
-
W
M
A
a
p
p
r
o
a
c
h
was
ev
a
l
u
at
e
d
u
s
i
n
g
S
e
m
E
v
a
l
2
0
1
6
,
Se
m
E
v
a
l
2
0
1
5
,
a
n
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
2
,
Ap
r
il 2
0
2
6
:
1
8
2
5
-
1
8
3
8
1828
S
e
m
E
v
a
l
2
0
1
4
,
ac
r
o
s
s
l
a
p
to
p
a
n
d
r
e
s
t
au
r
an
t
d
o
m
a
i
n
s
,
w
h
er
e
a
ch
i
e
v
e
d
h
i
g
h
ac
c
u
r
a
cy
of
9
0
.
5
4
%
,
s
h
o
w
i
n
g
g
o
o
d
p
e
r
f
o
r
m
an
c
e
a
n
d
ef
f
e
c
t
iv
e
s
e
n
t
im
e
n
t
c
l
a
s
s
i
f
i
c
a
t
io
n
.
Fro
m
ab
o
v
e
liter
atu
r
e
s
u
r
v
e
y
,
it
is
s
ee
n
th
at
s
ev
er
al
r
e
ce
n
t
s
tu
d
ies
h
av
e
p
r
o
p
o
s
ed
in
n
o
v
ativ
e
ap
p
r
o
ac
h
es
to
i
m
p
r
o
v
e
AB
SA,
y
et
ea
c
h
e
x
h
ib
its
ce
r
tain
lim
itatio
n
s
.
Fo
r
i
n
s
tan
ce
,
Gu
et
a
l.
[
1
1
]
in
tr
o
d
u
ce
d
SAGC
N
by
in
teg
r
atin
g
s
y
n
ta
x
an
d
s
en
tim
e
n
t
k
n
o
wled
g
e
but
r
elied
h
ea
v
ily
on
e
x
ter
n
a
l
lex
ico
n
s
,
lim
itin
g
ad
ap
tab
ilit
y
to
in
f
o
r
m
al
or
d
o
m
ain
-
s
p
ec
if
ic
r
e
v
iews,
J
eo
n
g
an
d
L
ee
[
1
2
]
u
tili
ze
d
C
h
atGP
T
f
o
r
asp
ec
t
an
aly
s
is
but
f
o
cu
s
ed
o
n
ly
on
q
u
alitativ
e
o
u
tp
u
ts
with
o
u
t
q
u
a
n
titativ
e
b
en
ch
m
a
r
k
in
g
,
m
ak
in
g
s
c
alab
ilit
y
u
n
ce
r
tain
.
Fu
r
th
er
,
Z
h
an
g
et
a
l.
[
1
3
]
p
r
o
p
o
s
ed
SD
-
GC
N
to
ca
p
t
u
r
e
s
y
n
tactic
d
ep
en
d
en
cies
but
o
v
er
lo
o
k
e
d
s
em
an
tic
in
co
n
s
is
ten
cies
th
at
o
f
ten
o
c
cu
r
in
u
s
er
-
g
en
e
r
ated
co
n
ten
t
.
Mu
g
h
al
et
a
l.
[
1
4
]
co
m
p
a
r
ed
LLMs
an
d
DL
m
o
d
els
f
o
r
AB
SA
but
h
ig
h
lig
h
ted
p
er
f
o
r
m
a
n
ce
in
co
n
s
is
ten
cy
ac
r
o
s
s
d
atasets
,
s
h
o
win
g
a
lack
of
d
o
m
ain
r
o
b
u
s
tn
ess
,
Falato
u
r
i
et
a
l.
[
1
5
]
f
o
u
n
d
th
at
alth
o
u
g
h
L
L
Ms
lik
e
C
h
atGPT
p
er
f
o
r
m
ed
well,
th
ey
lack
ed
p
r
ec
is
io
n
in
f
in
e
-
g
r
ain
ed
ex
tr
ac
tio
n
.
L
iu
et
a
l.
[
1
6
]
ad
d
r
ess
ed
s
tr
u
ctu
r
al
d
ep
en
d
en
cies
u
s
in
g
Sy
n
Sem
-
ASTE
but
r
eq
u
ir
e
d
co
m
p
le
x
m
u
lt
i
-
en
co
d
er
s
etu
p
s
,
m
ak
i
n
g
t
h
e
m
o
d
el
co
m
p
u
tatio
n
ally
h
ea
v
y
.
Similar
ly
,
Hellwig
et
a
l.
[
1
7
]
g
en
er
ate
d
s
y
n
th
etic
d
ata
u
s
in
g
L
L
Ms,
y
et
r
elied
on
s
m
all
an
n
o
tat
ed
s
ets,
p
o
ten
tially
in
tr
o
d
u
cin
g
b
ias.
T
h
ese
lim
it
atio
n
s
ar
e
ad
d
r
ess
ed
in
t
h
e
p
r
o
p
o
s
ed
H
-
B
E
R
T
f
r
am
ewo
r
k
,
wh
ich
co
m
b
in
es
S
p
an
B
E
R
T
f
o
r
s
p
a
n
d
etec
tio
n
,
B
iLST
M
f
o
r
s
eq
u
en
tial
co
n
te
x
t,
C
R
Fs
f
o
r
s
tr
u
ctu
r
ed
p
r
ed
ict
io
n
,
a
n
d
LLMs
f
o
r
weak
s
u
p
er
v
is
io
n
.
T
h
is
u
n
if
ie
d
ap
p
r
o
ac
h
en
h
an
ce
s
asp
ec
t
d
etec
tio
n
an
d
p
o
lar
ity
class
if
icatio
n
ac
r
o
s
s
b
o
th
s
y
n
th
etic
an
d
ex
is
tin
g
d
atasets
.
3.
M
E
T
H
O
D
T
h
i
s
s
e
ct
io
n
b
eg
i
n
s
by
p
r
e
s
e
n
tin
g
th
e
o
v
er
a
ll
ar
c
h
i
te
ctu
r
e
of
th
e
p
r
o
p
o
s
ed
s
y
s
t
em
,
f
o
l
l
o
wed
by
a
d
eta
i
led
d
i
s
cu
s
s
io
n
of
th
e
d
a
t
as
et
s
u
s
ed
an
d
t
h
e
p
r
ep
r
o
c
e
s
s
in
g
s
t
ep
s
ap
p
li
ed
.
It
th
en
o
u
t
l
in
e
s
t
h
e
l
im
it
at
io
n
s
of
tr
ad
it
io
n
al
B
E
R
T
m
o
d
e
l
s
,
in
tr
o
d
u
ce
s
t
h
e
p
r
o
p
o
s
e
d
H
-
B
E
R
T
f
r
am
ewo
r
k
,
an
d
c
o
n
clu
d
e
s
wi
th
t
h
e
p
er
f
o
r
m
an
c
e
m
e
tr
ic
s
u
s
ed
f
o
r
ev
al
u
a
tio
n
.
T
h
e
ar
ch
i
tec
tu
r
e
of
th
e
co
m
p
le
te
wo
r
k
is
p
r
e
s
en
te
d
in
F
ig
u
r
e
1,
wh
i
ch
s
h
o
ws
a
co
m
p
r
eh
en
s
iv
e
f
r
a
m
e
wo
r
k
f
o
r
A
B
S
A
u
s
in
g
H
-
B
E
R
T
ap
p
r
o
a
ch
.
In
th
i
s
a
r
ch
i
tec
tu
r
e,
f
ir
s
t
th
e
d
ata
s
et
is
co
n
s
id
er
ed
,
i
.
e.
,
Se
m
E
v
a
l
2014
T
a
s
k
4
d
a
ta
s
e
t
wh
i
ch
co
m
p
r
is
e
s
of
r
e
s
t
au
r
an
t
an
d
l
ap
to
p
.
In
th
is
wo
r
k
,
two
m
o
r
e
d
a
ta
s
e
t
s
w
er
e
cr
e
at
ed
u
s
in
g
L
L
M
s
,
i
.
e.
,
u
s
i
n
g
C
h
at
G
PT
-
3
.
5
-
T
u
r
b
o
an
d
Gem
in
i
-
2
.
5
-
Fl
a
s
h
wh
i
ch
is
d
i
s
cu
s
s
ed
in
d
e
ta
i
l
in
s
u
b
-
s
ec
tio
n
3
.
2
.
T
h
e
s
e
d
a
ta
s
e
t
s
s
ep
ar
a
te
ly
go
th
r
o
u
g
h
p
r
ep
r
o
c
es
s
in
g
,
wh
er
e
th
e
n
u
l
l
v
a
lu
e
s
ar
e
ch
e
ck
ed
an
d
th
e
r
e
v
i
ew
s
en
ten
ce
is
t
o
k
en
i
ze
d
.
T
h
e
co
m
p
le
te
p
r
e
p
r
o
ce
s
s
in
g
s
tep
s
ar
e
d
i
s
cu
s
s
ed
in
d
e
ta
i
l
in
s
u
b
-
s
ec
tio
n
3
.
3
.
Fu
r
th
er
,
th
e
p
r
e
-
p
r
o
ce
s
s
ed
t
ex
t
is
p
a
s
s
ed
on
to
th
e
H
-
B
E
R
T
m
o
d
el
,
wh
i
ch
is
d
is
cu
s
s
ed
in
d
e
ta
il
in
s
u
b
-
s
e
ct
io
n
3
.
4
.
T
h
e
m
ain
ai
m
of
H
-
B
E
R
T
is
to
ex
t
r
a
ct
a
s
p
e
ct
s
,
ex
tr
a
ct
asp
ec
t
-
t
er
m
p
o
lar
i
ty
an
d
p
r
e
d
ic
tin
g
o
v
er
a
ll
r
ev
i
ew
s
e
n
t
en
ce
s
en
ti
m
en
t
p
o
l
ar
i
ty
.
Par
al
le
l
to
th
i
s
p
r
o
ce
s
s
,
th
i
s
wo
r
k
h
as
a
ls
o
u
s
ed
L
L
Ms
(
C
h
at
G
PT
-
3
.
5
-
T
u
r
b
o
an
d
Ge
m
in
i
-
2
.
5
-
F
la
s
h
)
f
o
r
ex
tr
ac
t
in
g
a
s
p
ec
t
s
,
ex
tr
ac
t
asp
ec
t
-
t
er
m
p
o
la
r
i
ty
an
d
p
r
ed
ic
tin
g
o
v
er
a
l
l
r
ev
ie
w
s
en
ten
c
e
s
en
t
im
en
t
p
o
la
r
i
ty
,
wh
i
ch
is
d
i
s
cu
s
s
ed
in
d
e
ta
i
l
in
s
ec
tio
n
4.
Fin
al
ly
,
t
h
e
p
er
f
o
r
m
an
c
e
of
th
e
H
-
B
E
R
T
an
d
L
L
M
s
is
ev
a
lu
a
ted
u
s
in
g
t
w
o
s
t
an
d
ar
d
m
etr
ic
s
u
s
ed
f
o
r
ev
alu
at
in
g
A
B
SA,
i.
e.
,
a
cc
u
r
ac
y
an
d
m
ac
r
o
-
F
-
s
co
r
e
wh
ich
is
d
i
s
cu
s
s
ed
in
d
e
ta
i
l
in
s
u
b
-
s
ec
tio
n
3
.
6
.
T
h
i
s
ar
ch
it
ec
tu
r
e
co
m
b
in
es
m
ac
h
i
n
e
le
ar
n
i
n
g
(
ML
)
,
D
L
,
an
d
r
u
l
e
-
b
a
s
ed
ap
p
r
o
a
ch
e
s
f
o
r
p
r
o
v
id
in
g
a
cc
u
r
ac
y
s
en
ti
m
en
t
in
s
ig
h
t
s
.
Fig
u
r
e
1.
Pro
p
o
s
ed
ar
c
h
itectu
r
e
of
th
e
wo
r
k
T
h
i
s
s
e
c
t
io
n
d
i
s
c
u
s
s
e
s
th
e
d
a
ta
s
e
t
s
u
s
e
d
in
t
h
i
s
wo
r
k
f
o
r
e
v
a
l
u
a
t
i
n
g
H
-
B
E
R
T
m
o
d
e
l.
In
th
i
s
w
o
r
k
,
f
o
r
e
v
a
l
u
a
t
io
n
of
H
-
B
E
R
T
,
th
r
e
e
d
i
s
t
i
n
c
t
A
B
S
A
d
a
t
a
s
e
t
s
w
e
r
e
u
s
e
d
.
T
h
e
f
ir
s
t
d
a
t
a
s
e
t
was
S
e
m
E
v
a
l
2
0
1
4
T
a
s
k
4
d
a
t
a
s
e
t,
w
h
i
ch
is
o
p
en
a
c
c
e
s
s
i
b
l
e
an
d
can
be
d
o
wn
lo
a
d
ed
f
r
o
m
in
[
2
1
]
,
[
2
2
]
.
T
h
e
d
a
t
a
s
e
t
co
n
s
i
s
t
s
of
c
u
s
t
o
m
er
r
e
v
i
e
w
s
p
r
im
a
r
i
ly
f
r
o
m
l
a
p
to
p
a
n
d
r
e
s
t
a
u
r
a
n
t
d
o
m
a
i
n
s
.
In
t
h
i
s
d
a
t
a
s
e
t
,
e
v
er
y
a
s
p
e
c
t
was
l
ab
e
l
l
ed
w
i
t
h
o
n
e
of
t
h
r
e
e
p
o
la
r
i
t
i
e
s
,
i
.
e.
,
n
e
u
t
r
a
l
,
n
e
g
a
t
iv
e
,
a
n
d
p
o
s
i
t
iv
e
.
T
h
e
s
e
c
o
n
d
an
d
th
i
r
d
d
at
a
s
e
t
w
er
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
en
h
a
n
ce
d
a
s
p
ec
t
-
b
a
s
ed
s
en
timen
t a
n
a
lysi
s
u
s
in
g
la
r
g
e
la
n
g
u
a
g
e
…
(
Mo
h
a
mme
d
Zia
u
lla
)
1829
g
e
n
er
a
t
ed
u
s
i
n
g
C
h
a
t
G
P
T
-
3
.
5
-
T
u
r
b
o
an
d
Ge
m
i
n
in
-
2
.
5
-
F
la
s
h
,
s
i
m
i
l
ar
to
th
e
w
o
r
k
p
r
e
s
e
n
t
e
d
in
[
1
2
]
,
[
1
7
]
,
[
1
8
]
.
T
h
e
d
a
t
a
s
e
t
s
w
er
e
cr
e
a
te
d
u
s
in
g
p
r
o
m
p
t
i
n
g
a
p
p
r
o
a
ch
,
w
h
i
ch
is
p
r
e
s
e
n
te
d
in
T
ab
l
e
1.
B
o
th
th
e
LLMs
w
e
r
e
p
r
o
m
p
t
e
d
w
i
th
s
am
e
c
o
n
t
e
x
t
an
d
i
n
s
t
r
u
c
t
io
n
,
i.
e
.
,
to
c
r
ea
t
e
50
s
y
n
t
h
e
t
i
c
c
u
s
t
o
m
er
r
e
v
i
e
w
en
t
r
i
e
s
in
.
c
s
v
f
o
r
m
a
t.
In
t
h
e
.
c
s
v
f
i
l
e,
ea
c
h
e
n
tr
y
i
n
c
lu
d
e
d
a
r
ev
i
e
w
s
e
n
t
e
n
c
e
,
up
to
t
h
r
e
e
a
s
p
e
c
t
s
w
i
t
h
co
r
r
e
s
p
o
n
d
i
n
g
s
e
n
t
i
m
e
n
t
p
o
l
ar
i
t
i
e
s
an
d
o
v
er
a
l
l
s
en
t
i
m
en
t
c
l
a
s
s
i
f
i
c
a
t
i
o
n
.
T
h
e
o
u
tp
u
t
f
i
l
e
s
f
ea
t
u
r
e
c
o
l
u
m
n
s
s
u
ch
as
I
D,
r
e
v
i
e
w,
a
s
p
e
c
t
1
,
a
s
p
ec
t
1
p
o
l
ar
i
t
y
,
a
s
p
e
c
t
2
,
a
n
d
a
s
p
ec
t
2
p
o
la
r
i
ty
.
T
h
e
s
e
s
y
n
th
e
ti
c
d
a
t
a
s
e
t
s
p
r
o
v
id
e
v
a
l
u
ab
l
e
s
u
p
p
l
em
e
n
ta
l
d
a
t
a
f
o
r
ev
a
lu
a
t
i
n
g
m
o
d
e
l
r
o
b
u
s
t
n
e
s
s
a
n
d
g
e
n
er
a
l
i
z
a
t
io
n
ac
r
o
s
s
b
o
th
h
u
m
an
-
a
n
n
o
t
a
t
ed
an
d
m
a
ch
i
n
e
-
g
en
e
r
a
t
e
d
r
e
v
i
e
w
s
.
T
h
e
co
m
p
l
e
t
e
f
lo
w
of
th
e
d
a
t
a
s
e
t
co
l
l
e
c
t
io
n
u
s
i
n
g
C
h
a
t
G
P
T
a
n
d
G
e
m
i
n
i
is
p
r
e
s
e
n
t
ed
in
F
i
g
u
r
e
2.
T
ab
le
1.
Pro
m
p
t
u
s
ed
f
o
r
q
u
er
y
in
g
Gem
in
i
a
n
d
C
h
atGPT
C
o
n
t
e
x
t
G
e
n
e
r
a
t
e
a
sy
n
t
h
e
t
i
c
d
a
t
a
s
e
t
of
c
u
s
t
o
mer
r
e
v
i
e
w
s
w
i
t
h
mu
l
t
i
p
l
e
a
s
p
e
c
t
-
l
e
v
e
l
se
n
t
i
me
n
t
s
a
n
d
o
v
e
r
a
l
l
p
o
l
a
r
i
t
y
I
n
st
r
u
c
t
i
o
n
C
r
e
a
t
e
50
r
e
v
i
e
w
e
n
t
r
i
e
s
in
.
c
sv
f
o
r
mat
w
h
e
r
e
e
a
c
h
e
n
t
r
y
i
n
c
l
u
d
e
s
a
r
e
v
i
e
w
se
n
t
e
n
c
e
,
m
u
l
t
i
p
l
e
a
s
p
e
c
t
s
men
t
i
o
n
e
d
in
t
h
e
se
n
t
e
n
c
e
w
i
t
h
t
h
e
i
r
r
e
sp
e
c
t
i
v
e
s
e
n
t
i
me
n
t
p
o
l
a
r
i
t
y
(
p
o
si
t
i
v
e
/
n
e
g
a
t
i
v
e
/
n
e
u
t
r
a
l
)
,
a
n
d
t
h
e
o
v
e
r
a
l
l
sen
t
i
m
e
n
t
p
o
l
a
r
i
t
y
of
t
h
e
r
e
v
i
e
w
.
O
u
t
p
u
t
C
S
V
f
i
l
e
w
i
t
h
c
o
l
u
m
n
s:
ID,
r
e
v
i
e
w
,
a
sp
e
c
t
1
,
a
s
p
e
c
t
1
p
o
l
a
r
i
t
y
,
a
sp
e
c
t
2
,
a
sp
e
c
t
2
p
o
l
a
r
i
t
y
,
a
s
p
e
c
t
3
,
a
sp
e
c
t
3
p
o
l
a
r
i
t
y
,
o
v
e
r
a
l
l
se
n
t
i
me
n
t
Fig
u
r
e
2.
Pro
ce
s
s
of
d
ataset
cr
ea
tio
n
In
th
is
wo
r
k
,
th
e
d
atasets
wen
t
th
r
o
u
g
h
p
r
ep
r
o
ce
s
s
in
g
wh
ich
is
im
p
o
r
tan
t
f
o
r
co
n
s
is
ten
cy
b
ef
o
r
e
th
e
y
ar
e
p
ass
ed
on
to
t
h
e
H
-
B
E
R
T
.
E
ac
h
d
ataset,
i.e
.
,
Sem
E
v
al
2
1
0
4
,
C
h
atGPT
d
ataset
,
an
d
G
em
in
i
d
ataset
wer
e
p
r
o
ce
s
s
ed
in
d
ep
en
d
en
tly
f
o
r
p
r
eser
v
in
g
s
tr
u
ctu
r
al
in
teg
r
ity
a
n
d
m
an
ag
e
v
a
r
iatio
n
s
in
d
ata
f
o
r
m
at
in
[
2
3
]
,
[
2
4
]
.
T
h
e
f
ir
s
t
s
tep
in
p
r
ep
r
o
ce
s
s
in
g
in
v
o
lv
ed
ch
ec
k
i
n
g
f
o
r
n
u
l
l
or
m
is
s
in
g
v
al
u
es
in
r
ev
ie
w
s
en
ten
ce
s
,
asp
ec
t
ter
m
s
,
an
d
s
en
tim
en
t
p
o
lar
ities
.
An
y
en
tr
y
c
o
n
tain
in
g
n
u
ll
v
alu
es
wer
e
r
em
o
v
e
d
,
p
r
o
v
id
in
g
clea
n
,
an
d
r
eliab
l
e
d
ataset.
Fo
llo
win
g
th
is
,
t
h
e
r
e
v
iew
s
en
ten
ce
s
wer
e
t
o
k
en
ize
d
,
wh
ich
is
im
p
o
r
tan
t
f
o
r
H
-
B
E
R
T
m
o
d
el.
T
h
is
p
r
o
ce
s
s
en
ab
les
th
e
h
y
b
r
id
m
o
d
el
to
u
n
d
er
s
tan
d
s
em
an
ti
c
s
tr
u
ctu
r
e
of
each
r
e
v
iew
an
d
f
ac
ilit
ate
asp
ec
t
ex
tr
ac
tio
n
a
n
d
s
en
tim
en
t
an
al
y
s
is
at
g
r
an
u
la
r
lev
el.
Als
o
,
tex
t
n
o
r
m
aliza
tio
n
s
tep
s
,
wh
i
ch
in
clu
d
ed
l
o
wer
ca
s
in
g
an
d
u
n
n
ec
ess
ar
y
wh
it
esp
ac
e
wer
e
ap
p
lied
f
o
r
m
ai
n
tain
in
g
u
n
if
o
r
m
ity
ac
r
o
s
s
all
d
ataset.
T
h
is
s
tep
en
s
u
r
ed
co
n
s
is
ten
t
in
p
u
t
f
o
r
m
ats,
p
r
o
v
id
in
g
ac
cu
r
ate
a
n
d
r
o
b
u
s
t
s
en
tim
en
t
p
r
ed
ictio
n
s
[
2
5
]
.
B
E
R
T
is
a
p
r
e
-
t
r
a
in
e
d
l
an
g
u
a
g
e
r
e
p
r
e
s
e
n
t
a
t
io
n
m
o
d
e
l
d
e
v
e
lo
p
e
d
by
g
o
o
g
l
e
,
w
h
i
c
h
h
a
s
r
e
v
o
l
u
t
io
n
i
z
ed
N
L
P
t
a
s
k
s
,
i
n
cl
u
d
in
g
s
e
n
t
im
e
n
t
an
a
l
y
s
i
s
.
T
h
e
B
E
R
T
’
s
a
r
ch
i
t
e
c
tu
r
e
is
b
a
s
e
d
on
T
r
an
s
f
o
r
m
er
m
o
d
e
l
,
w
h
i
ch
u
t
i
l
i
ze
s
s
e
l
f
-
at
t
e
n
t
i
o
n
f
o
r
c
ap
t
u
r
in
g
co
n
tex
t
u
a
l
r
e
l
a
t
i
o
n
s
h
ip
a
m
o
n
g
wo
r
d
s
in
a
s
en
t
e
n
c
e
.
U
n
l
i
k
e
t
r
ad
i
t
i
o
n
a
l
w
o
r
d
-
em
b
ed
d
in
g
a
p
p
r
o
a
c
h
e
s
l
i
k
e
g
lo
b
a
l
-
v
e
ct
o
r
s
(
G
lo
V
e
)
a
n
d
w
o
r
d
-
to
-
v
e
c
to
r
(
W
o
r
d
2
V
e
c)
,
B
E
R
T
a
p
p
r
o
a
ch
is
b
i
d
ir
e
c
t
i
o
n
a
l
,
m
ea
n
in
g
it
c
o
n
s
i
d
e
r
s
b
o
t
h
r
i
g
h
t
a
n
d
l
ef
t
c
o
n
t
e
x
t
s
of
w
o
r
d
s
i
m
u
l
t
a
n
eo
u
s
ly
d
u
r
i
n
g
tr
a
in
i
n
g
.
T
h
i
s
p
r
o
v
i
d
e
s
B
E
R
T
to
g
en
e
r
a
t
e
co
n
t
ex
t
-
a
w
a
r
e
em
b
ed
d
i
n
g
s
wh
i
ch
p
r
o
v
id
e
b
e
t
t
e
r
p
e
r
f
o
r
m
an
c
e
on
t
a
s
k
s
l
i
k
e
s
e
n
t
im
e
n
t
c
la
s
s
i
f
i
c
a
t
i
o
n
.
In
s
e
n
t
im
e
n
t
a
n
a
ly
s
i
s
,
B
E
R
T
is
b
e
i
n
g
w
i
d
e
ly
u
s
e
d
f
o
r
a
s
p
e
c
t
-
l
e
v
e
l
s
e
n
t
im
en
t
a
n
d
s
e
n
te
n
ce
-
l
ev
e
l
c
l
a
s
s
i
f
ic
a
t
i
o
n
b
e
ca
u
s
e
of
its
u
n
d
e
r
s
t
an
d
i
n
g
of
l
in
g
u
i
s
t
i
c
c
o
n
t
ex
t
.
N
ev
e
r
th
e
l
e
s
s
,
h
a
v
in
g
g
o
o
d
p
er
f
o
r
m
a
n
ce
,
B
E
R
T
m
o
d
e
l
s
ex
h
ib
i
t
s
e
v
e
r
a
l
l
i
m
i
t
a
t
io
n
s
w
h
e
n
ap
p
l
ie
d
to
f
i
n
e
-
g
r
a
i
n
ed
ta
s
k
s
l
ik
e
A
B
SA
.
O
n
e
m
a
j
o
r
d
r
a
wb
a
ck
is
t
h
a
t
B
E
R
T
is
d
e
s
i
g
n
ed
f
o
r
g
en
e
r
a
l
l
an
g
u
ag
e
u
n
d
e
r
s
t
a
n
d
i
n
g
an
d
f
a
i
l
s
to
i
n
h
e
r
en
t
l
y
f
o
cu
s
on
s
p
ec
i
f
i
c
a
s
p
e
c
t
-
t
er
m
s
in
s
e
n
t
en
c
e
u
n
l
e
s
s
e
x
p
l
i
ci
t
l
y
g
u
id
e
d
in
[
2
6
]
,
[
2
7
]
.
He
n
c
e,
it
f
a
i
l
s
to
d
i
s
t
i
n
g
u
i
s
h
s
e
n
t
i
m
e
n
t
s
a
s
s
o
c
i
a
t
ed
w
i
th
m
u
l
t
ip
l
e
a
s
p
e
c
ts
in
s
am
e
r
e
v
i
e
w
.
A
d
d
i
t
io
n
a
l
ly
,
B
E
R
T
’
s
p
r
e
-
tr
a
i
n
i
n
g
on
g
en
e
r
i
c
c
o
r
p
o
r
a
l
im
i
t
s
B
E
R
T
a
b
i
l
i
t
y
f
o
r
a
cc
u
r
a
t
e
l
y
in
t
e
r
p
r
e
t
in
g
d
o
m
a
i
n
-
s
p
e
c
i
f
ic
s
en
t
i
m
en
t
cu
e
s
,
l
i
k
e
id
i
o
m
a
t
i
c
ex
p
r
e
s
s
i
o
n
or
d
o
m
a
i
n
-
s
p
e
c
if
i
c
t
e
r
m
s
.
Mo
r
e
o
v
e
r
,
f
in
e
-
t
u
n
i
n
g
B
E
R
T
f
o
r
A
B
S
A
r
eq
u
ir
e
s
s
ig
n
i
f
ic
a
n
t
co
m
p
u
t
a
t
io
n
r
e
s
o
u
r
c
e
s
a
n
d
c
a
r
e
f
u
l
h
y
p
e
r
p
a
r
a
m
et
e
r
tu
n
in
g
.
T
h
e
s
e
l
i
m
i
t
a
t
i
o
n
s
h
ig
h
l
ig
h
t
n
e
e
d
f
o
r
h
y
b
r
id
of
m
o
d
if
i
e
d
m
o
d
e
l
wh
i
c
h
i
n
co
r
p
o
r
a
t
e
s
B
E
R
T
’
s
c
o
n
t
e
x
tu
a
l
s
tr
e
n
g
t
h
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
2
,
Ap
r
il 2
0
2
6
:
1
8
2
5
-
1
8
3
8
1830
w
h
i
l
e
ad
d
r
e
s
s
s
h
o
r
t
co
m
in
g
in
a
s
p
e
c
t
-
o
r
i
en
t
ed
s
e
n
t
im
e
n
t
ta
s
k
s
.
H
en
c
e
,
in
t
h
i
s
w
o
r
k
,
a
H
-
B
E
R
T
m
o
d
e
l
is
p
r
e
s
e
n
t
e
d
f
o
r
s
o
lv
i
n
g
th
e
f
o
l
lo
w
i
n
g
i
s
s
u
e
s
.
T
h
e
ar
ch
itectu
r
e
of
th
e
H
-
B
E
R
T
is
p
r
esen
ted
in
Fig
u
r
e
3,
wh
ich
s
tar
ts
by
tak
in
g
i
n
p
u
t,
i.e
.
,
p
r
e
-
p
r
o
ce
s
s
ed
r
ev
iew
tex
ts
(
to
k
en
s
an
d
in
d
ices)
.
T
h
ese
r
ev
i
ews
ar
e
p
ass
ed
o
n
to
th
e
H
-
B
E
R
T
wh
ich
co
n
s
is
ts
of
S
p
an
B
E
R
T
,
a
p
r
e
-
tr
ain
ed
B
E
R
T
m
o
d
el
wh
ich
is
d
esig
n
ed
f
o
r
b
etter
r
e
p
r
esen
tin
g
an
d
p
r
ed
ictin
g
s
p
an
s
of
tex
t,
wh
ich
in
th
is
wo
r
k
is
u
til
ized
f
o
r
ca
p
tu
r
in
g
asp
ec
t
s
p
an
s
in
[
2
8
]
.
T
h
e
o
u
tp
u
t
of
S
p
a
n
B
E
R
T
is
p
ass
ed
on
to
th
e
B
iLST
M
f
o
r
s
eq
u
en
tial
co
n
tex
tu
alize
d
lear
n
in
g
,
f
u
r
t
h
er
p
ass
ed
on
to
d
r
o
p
o
u
t
f
o
r
r
e
g
u
lar
izatio
n
.
Af
ter
d
r
o
p
o
u
t,
a
lin
ea
r
-
lay
e
r
is
ad
d
ed
,
wh
ich
ac
ts
as
f
u
lly
-
co
n
v
o
lu
tio
n
al
lay
er
(
FC
L
)
.
T
h
e
o
u
t
p
u
ts
f
r
o
m
FC
L
ar
e
p
ass
ed
on
to
C
R
Fs
f
o
r
s
tr
u
ctu
r
ed
s
eq
u
en
ce
p
r
ed
ictio
n
an
d
L
L
Ms
(
C
h
atGPT
-
3
.
5
-
T
u
r
b
o
an
d
Gem
in
i
-
2
.
5
-
Flas
h
)
to
s
u
p
p
o
r
t
g
e
n
er
aliza
tio
n
.
T
h
e
p
r
o
ce
s
s
ed
em
b
ed
d
in
g
s
ar
e
th
en
class
if
ied
u
s
in
g
So
f
tMa
x
class
if
ier
in
to
th
r
ee
task
s
,
i.e
.
,
asp
ec
t
-
ex
tr
ac
tio
n
,
a
s
p
ec
t
-
p
o
lar
ity
,
an
d
s
en
ten
ce
-
p
o
lar
ity
.
Fig
u
r
e
3.
H
-
B
E
R
T
ar
ch
itectu
r
e
S
p
an
B
E
R
T
is
a
s
p
an
-
b
ased
ex
ten
s
io
n
of
B
E
R
T
,
wh
ich
is
s
p
ec
if
ically
tr
ain
ed
f
o
r
p
r
ed
ictin
g
s
p
an
s
of
tex
t
r
ath
er
th
a
n
in
d
i
v
id
u
al
m
a
s
k
ed
to
k
en
s
.
C
o
n
s
id
er
a
to
k
e
n
ized
s
en
ten
ce
as
=
{
1
,
2
,
…
}
be
in
p
u
t
to
S
p
an
B
E
R
T
.
T
h
e
m
o
d
el
en
co
d
es
co
n
tex
tu
al
em
b
e
d
d
in
g
s
as
=
{
1
,
2
,
…
,
}
an
d
let
d
en
o
te
em
b
ed
d
in
g
d
im
en
s
io
n
.
Hen
ce
,
f
r
o
m
th
is
,
t
h
e
S
p
an
B
E
R
T
ca
p
tu
r
es
r
ep
r
es
en
tatio
n
s
as
,
as
p
r
esen
ted
in
(
1
)
.
,
=
[
;
;
⨀
;
(
,
…
,
)
]
(
1
)
In
(
1
)
,
[
∙
]
d
en
o
tes
co
n
ca
ten
atio
n
,
⨀
d
en
o
tes
elem
en
t
-
wis
e
m
u
ltip
licatio
n
,
an
d
d
en
o
tes
m
ea
n
p
o
o
lin
g
o
v
er
s
p
an
(
,
)
.
W
h
en
co
m
p
a
r
ed
with
B
E
R
T
ap
p
r
o
ac
h
es,
wh
ich
f
o
cu
s
s
es
on
m
ask
ed
-
to
k
e
n
p
r
ed
ictio
n
,
th
e
S
p
a
n
B
E
R
T
is
f
in
e
-
tu
n
ed
on
m
ask
ed
-
s
p
an
p
r
ed
ictio
n
,
m
a
k
in
g
it
m
o
r
e
s
u
itab
le
f
o
r
asp
ec
t
ex
tr
ac
tio
n
wh
er
e
m
u
lti
-
to
k
en
s
s
p
an
s
n
ee
d
id
en
tific
atio
n
.
T
h
e
S
p
an
B
E
R
T
p
r
o
v
id
es
u
n
d
er
s
tan
d
in
g
of
s
y
n
tactic
-
s
em
an
tic
d
ep
en
d
en
cies
with
in
s
p
an
s
,
p
r
o
v
id
i
n
g
im
p
r
o
v
ed
AB
SA.
To
ca
p
tu
r
e
th
e
s
eq
u
e
n
tial
d
ep
en
d
en
cy
of
s
en
ten
ce
,
th
is
wo
r
k
h
as
u
tili
ze
d
B
iLST
M
wh
ich
is
d
is
cu
s
s
ed
in
d
etail
in
n
e
x
t
s
ec
tio
n
.
F
o
r
ca
p
tu
r
in
g
s
e
q
u
e
n
t
i
a
l
-
d
ep
e
n
d
e
n
c
y
,
a
B
i
L
S
T
M
is
u
s
e
d
on
o
u
tp
u
t
of
S
p
a
n
B
E
R
T
.
G
i
v
en
i
n
p
u
t
e
m
b
ed
d
in
g
s
=
{
1
,
2
,
…
,
}
,
th
e
B
i
L
ST
M
c
o
m
p
u
t
e
s
b
a
ck
w
a
r
d
an
d
f
o
r
w
a
r
d
h
i
d
d
e
n
s
t
a
t
e
s
u
s
i
n
g
(
2
)
a
n
d
(
3
)
.
T
h
e
f
in
a
l
r
e
p
r
e
s
en
t
a
t
i
o
n
at
e
a
ch
t
o
k
en
is
r
e
p
r
e
s
e
n
t
e
d
as
(
4
)
.
ℎ
⃗
=
(
,
ℎ
⃗
−
1
)
(
2
)
ℎ
⃖
⃗
=
(
,
ℎ
⃖
⃗
+
1
)
(
3
)
ℎ
=
[
ℎ
⃗
;
ℎ
⃖
⃗
]
∈
ℝ
2ℎ
(
4
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
en
h
a
n
ce
d
a
s
p
ec
t
-
b
a
s
ed
s
en
timen
t a
n
a
lysi
s
u
s
in
g
la
r
g
e
la
n
g
u
a
g
e
…
(
Mo
h
a
mme
d
Zia
u
lla
)
1831
In
(
4
)
,
ℎ
d
en
o
tes
s
ize
of
L
STM
h
id
d
en
u
n
its
.
T
h
e
B
iLST
M
ad
d
s
tem
p
o
r
al
u
n
d
er
s
tan
d
in
g
by
u
tili
zin
g
b
o
th
f
u
tu
r
e
a
n
d
p
ast
co
n
tex
t,
wh
ich
is
im
p
o
r
tan
t
f
o
r
p
o
lar
ity
d
is
am
b
ig
u
atio
n
in
p
h
r
ases
lik
e
“
th
o
u
g
h
t
th
e
p
ac
k
ag
in
g
co
u
ld
be
b
etter
”
.
On
ce
th
e
B
iLST
M
h
as
en
co
d
ed
t
h
e
co
m
p
lete
s
eq
u
en
ce
,
th
e
h
id
d
en
r
ep
r
esen
tatio
n
s
{
ℎ
1
,
ℎ
2
,
…
,
ℎ
}
ar
e
p
ass
ed
th
r
o
u
g
h
d
r
o
p
o
u
t
lay
e
r
f
o
r
r
eg
u
lar
izatio
n
.
T
h
e
d
r
o
p
o
u
t
is
a
s
to
ch
asti
c
r
eg
u
lar
izatio
n
ap
p
r
o
ac
h
wh
er
e
d
u
r
in
g
tr
ain
in
g
,
a
f
r
ac
tio
n
of
o
u
t
p
u
t
u
n
its
ar
e
r
an
d
o
m
l
y
s
et
to
ze
r
o
with
p
r
o
b
a
b
ilit
y
∈
[
0
,
1
]
.
T
h
is
h
elp
s
p
r
e
v
e
n
tin
g
o
v
er
f
itti
n
g
by
n
o
t
allo
wi
n
g
th
e
m
o
d
el
f
r
o
m
b
ec
o
m
in
g
to
o
r
elian
t
on
s
p
ec
if
ic
n
eu
r
o
n
s
.
T
h
e
p
r
o
ce
s
s
is
m
ath
em
atica
l
r
ep
r
esen
ted
u
s
in
g
(
5
)
.
Fo
llo
win
g
d
r
o
p
o
u
t,
th
e
r
ep
r
esen
tatio
n
s
ar
e
p
ass
ed
in
t
o
lin
ea
r
lay
er
,
i.e
.
,
FC
L
.
T
h
is
lay
er
p
er
f
o
r
m
s
an
af
f
in
e
tr
a
n
s
f
o
r
m
atio
n
wh
ic
h
p
r
o
jects
B
iLST
M
o
u
tp
u
ts
in
to
n
ew
s
p
ac
e
f
o
r
class
if
icatio
n
u
s
in
g
(
6
)
.
ℎ
′
=
(
ℎ
)
,
ℎ
ℎ
′
∈
ℝ
2ℎ
(
5
)
=
ℎ
′
+
(
6
)
In
(
6
)
,
d
en
o
tes
weig
h
t
m
atr
i
x
an
d
d
en
o
tes
b
ias
v
ec
to
r
.
T
h
e
FC
L
ac
ts
as
b
r
id
g
e
am
o
n
g
d
ee
p
s
eq
u
en
tial
f
ea
tu
r
es
ex
tr
ac
ted
by
B
iLST
M
an
d
h
ig
h
er
-
lev
el
class
if
icatio
n
wh
ich
later
p
er
f
o
r
m
ed
by
C
R
Fs
an
d
So
f
tMa
x
,
d
is
cu
s
s
ed
in
n
ex
t
s
ec
tio
n
s
r
esp
ec
tiv
ely
.
By
lear
n
in
g
lin
ea
r
c
o
m
b
in
atio
n
s
of
h
i
d
d
en
f
ea
tu
r
es,
th
is
wo
r
k
p
r
e
p
ar
es
d
ata
f
o
r
s
tr
u
ctu
r
ed
s
eq
u
en
ce
d
ec
o
d
in
g
a
n
d
f
in
al
s
en
tim
en
t
d
ec
is
io
n
-
m
ak
in
g
.
Fo
r
en
s
u
r
in
g
s
tr
u
ctu
r
ed
p
r
ed
ic
tio
n
of
asp
ec
t
ter
m
s
a
n
d
lab
els,
th
is
wo
r
k
h
as
u
s
ed
C
R
Fs
on
to
p
of
th
e
B
iLST
M
o
u
tp
u
ts
.
L
et
=
{
ℎ
1
,
ℎ
2
,
…
,
ℎ
}
d
en
o
te
h
id
d
en
s
tates
f
r
o
m
B
iLST
M
an
d
=
{
1
,
2
,
…
,
}
d
en
o
te
th
e
s
eq
u
en
ce
of
p
r
e
d
icted
tag
s
.
In
th
is
wo
r
k
,
th
e
CRF
lay
er
m
o
d
els
th
e
co
n
d
itio
n
al
p
r
o
b
ab
ilit
y
u
s
in
g
(
7
)
.
In
(
7
)
,
d
en
o
tes
tr
an
s
itio
n
-
m
at
r
ix
,
(
)
d
en
o
tes
s
co
r
e
f
r
o
m
lin
ea
r
lay
er
f
o
r
ta
g
at
p
o
s
itio
n
.
T
h
e
C
R
F
s
en
s
u
r
e
th
at
o
u
tp
u
t
s
eq
u
e
n
ce
s
ar
e
v
alid
an
d
s
em
an
tically
co
n
s
is
ten
t.
(
|
)
=
(
∑
(
−
1
,
+
(
)
)
=
1
)
∑
(
∑
(
̃
−
1
,
̃
+
(
̃
)
)
=
1
)
̃
(
7
)
T
h
e
H
-
B
E
R
T
f
r
am
ewo
r
k
als
o
in
co
r
p
o
r
ates
LLMs
(
C
h
atG
PT
-
3
.
5
-
T
u
r
b
o
a
n
d
Gem
in
i
-
2
.
5
-
Flas
h
)
as
au
x
iliar
y
s
u
p
p
o
r
t
f
o
r
k
n
o
wle
d
g
e
in
teg
r
atio
n
.
T
h
e
LLMs
in
th
is
wo
r
k
ar
e
u
tili
ze
d
f
o
r
weak
s
u
p
er
v
is
io
n
,
en
h
an
cin
g
m
o
d
el’
s
g
en
er
aliz
atio
n
.
C
o
n
s
id
er
be
a
r
e
v
iew
p
ass
ed
to
LLM.
T
h
e
o
u
tp
u
t
f
r
o
m
t
h
e
LLMs
ac
h
iev
ed
is
as
p
r
esen
ted
in
(
8
)
.
T
h
e
o
u
tp
u
ts
f
r
o
m
LLMs
ar
e
u
s
ed
f
o
r
f
i
n
e
-
tu
n
i
n
g
th
e
f
in
al
p
r
ed
ictio
n
o
u
tco
m
e,
en
h
an
cin
g
h
y
b
r
id
m
o
d
el’
s
r
o
b
u
s
tn
ess
.
T
h
e
f
in
al
r
ep
r
esen
tati
o
n
f
r
o
m
LLM
is
p
ass
ed
to
So
f
tMa
x
class
if
ier
f
o
r
mul
t
i
-
task
lear
n
in
g
,
i.e
.
,
asp
ec
t
ex
tr
ac
tio
n
,
asp
ec
t
p
o
lar
ity
,
an
d
s
en
ten
ce
-
lev
el
s
en
tim
en
t.
T
h
e
So
f
tMa
x
p
r
o
b
a
b
ilit
y
f
o
r
class
at
p
o
s
itio
n
is
d
ef
in
ed
u
s
in
g
(
9
)
.
(
)
=
{
,
,
}
(
8
)
(
=
|
ℎ
)
=
ex
p
(
ℎ
+
)
∑
ex
p
(
ℎ
+
)
=
1
(
9
)
In
(
9
)
,
an
d
d
en
o
tes
weig
h
t
f
o
r
class
an
d
,
an
d
d
en
o
tes
b
ias
an
d
d
en
o
tes
n
u
m
b
e
r
of
cl
ass
es
,
i.e
.
,
p
o
s
itiv
e,
n
eg
ativ
e
,
an
d
n
eu
tr
al.
T
h
e
So
f
tMa
x
lay
er
en
s
u
r
es
p
r
o
b
ab
ilis
tic
an
d
in
ter
p
r
etab
le
o
u
tp
u
ts
f
o
r
each
to
k
en
an
d
o
v
er
all
s
en
ten
c
e
s
en
tim
en
t.
4.
P
E
RF
O
RM
A
NCE
E
VA
L
U
AT
I
O
N
Fo
r
ev
alu
atio
n
of
s
en
tim
en
t
class
if
icat
io
n
u
s
in
g
H
-
B
E
R
T
,
th
is
wo
r
k
u
tili
ze
d
ac
cu
r
ac
y
an
d
m
ac
r
o
-
F
-
s
co
r
e.
T
h
e
ac
cu
r
ac
y
m
ea
s
u
r
es
p
r
o
p
o
r
tio
n
of
c
o
r
r
ec
tly
p
r
e
d
icted
lab
els
o
v
er
to
tal
n
u
m
b
er
of
p
r
ed
ictio
n
s
.
It
is
ev
alu
ated
u
s
in
g
(
1
0
)
.
In
AB
S
A
s
en
tim
e
n
t
class
if
icatio
n
,
th
e
m
ac
r
o
-
F
-
s
co
r
e
is
im
p
o
r
tan
t
m
etr
ics
as
it
ca
lcu
lates
F1
-
s
co
r
e
f
o
r
each
class
in
d
ep
e
n
d
en
tl
y
an
d
th
en
av
e
r
ag
es
th
em
,
p
r
o
v
id
in
g
eq
u
al
weig
h
t
f
o
r
all
class
es
r
eg
ar
d
less
of
th
eir
f
r
eq
u
e
n
cies.
It
is
ev
alu
ated
u
s
in
g
(
1
1
)
.
In
(
1
1
)
,
d
en
o
tes
class
.
T
h
e
m
etr
ics
p
r
o
v
id
e
AB
SA
ev
alu
atio
n
.
T
h
e
p
er
f
o
r
m
an
ce
of
t
h
e
H
-
B
E
R
T
is
ev
alu
ated
in
th
e
n
ex
t
s
ec
tio
n
an
d
d
is
cu
s
s
ed
in
d
etail.
=
(
1
0
)
−
1
=
1
∑
2
×
,
+
=
1
(
1
1
)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
2
,
Ap
r
il 2
0
2
6
:
1
8
2
5
-
1
8
3
8
1832
5.
RE
SU
L
T
S
AND
D
I
SCU
SS
I
O
N
T
h
e
H
-
B
E
R
T
m
o
d
e
l
was
i
m
p
l
e
m
en
t
e
d
a
n
d
te
s
t
e
d
on
a
W
i
n
d
o
w
s
11
s
y
s
t
em
f
e
a
tu
r
in
g
an
A
M
D
R
y
z
e
n
5
p
r
o
c
e
s
s
o
r
,
16
GB
R
A
M
,
a
n
d
a
4
GB
N
V
I
D
I
A
G
eF
o
r
c
e
G
T
X
1
6
5
0
G
P
U
.
D
ev
e
lo
p
m
e
n
t
was
ca
r
r
ie
d
out
u
s
i
n
g
Py
t
h
o
n
w
i
th
i
n
a
P
y
t
h
o
n
3
.
1
1
e
n
v
i
r
o
n
m
e
n
t.
T
a
b
l
e
2
d
i
s
p
l
ay
s
t
h
e
s
a
m
p
l
e
d
i
s
t
r
i
b
u
t
i
o
n
of
t
h
e
S
e
m
E
v
a
l
2
0
1
4
d
a
t
a
s
e
t
u
s
ed
f
o
r
e
v
a
lu
a
t
io
n
.
S
a
m
p
l
e
r
ev
i
e
ws
g
e
n
e
r
a
t
ed
by
C
h
a
t
G
P
T
a
n
d
G
e
m
i
n
i
a
r
e
s
h
o
w
n
in
T
a
b
l
e
s
3
an
d
4,
r
e
s
p
e
c
t
iv
e
ly
,
w
h
i
l
e
s
a
m
p
le
s
f
r
o
m
t
h
e
S
em
E
v
a
l
2
0
1
4
d
a
t
a
s
e
t
ar
e
p
r
o
v
id
e
d
in
T
ab
l
e
5.
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
H
-
B
E
R
T
m
o
d
el
in
ac
cu
r
ately
id
en
tify
in
g
asp
ec
t
ter
m
s
an
d
th
eir
ass
o
ciate
d
s
en
tim
en
t
p
o
lar
ities
.
T
h
e
m
o
d
el
was
e
v
alu
ated
o
n
a
r
a
n
g
e
o
f
r
ev
ie
w
s
am
p
les,
an
d
th
e
p
r
ed
icted
r
esu
lts
wer
e
co
m
p
ar
ed
with
th
e
ac
tu
al
asp
ec
t
-
p
o
la
r
ity
an
n
o
tatio
n
s
.
As
s
h
o
wn
in
T
ab
le
6
,
H
-
B
E
R
T
d
em
o
n
s
tr
ated
a
s
tr
o
n
g
a
b
ilit
y
to
co
r
r
ec
tly
ex
t
r
ac
t
m
u
ltip
le
a
s
p
ec
ts
an
d
th
eir
s
en
tim
en
ts
w
ith
in
co
m
p
lex
a
n
d
m
u
lti
-
s
en
tim
en
t
s
en
ten
ce
s
.
Fo
r
ex
am
p
le,
in
s
en
ten
ce
s
co
n
tai
n
in
g
b
o
th
p
o
s
itiv
e
an
d
n
e
g
ativ
e
s
en
tim
en
ts
ab
o
u
t
d
if
f
er
en
t
asp
ec
ts
,
H
-
B
E
R
T
wa
s
ab
le
to
d
is
tin
g
u
is
h
an
d
clas
s
i
f
y
th
em
ap
p
r
o
p
r
iately
.
W
h
ile
t
h
e
m
o
d
el
ac
h
iev
ed
h
ig
h
ac
cu
r
ac
y
in
s
ev
er
al
in
s
ta
n
ce
s
,
m
in
o
r
m
is
m
atch
es
wer
e
o
b
s
er
v
ed
in
a
f
ew
p
r
ed
ictio
n
s
,
s
u
ch
as
cla
s
s
if
y
in
g
“ser
v
ice”
as
n
eu
tr
al
i
n
s
tead
o
f
p
o
s
itiv
e.
T
h
ese
d
is
cr
ep
an
cies
h
ig
h
lig
h
t
th
e
ch
allen
g
es
o
f
c
o
n
tex
tu
a
l
im
p
o
r
tan
ce
i
n
s
en
tim
en
t
class
if
icatio
n
.
T
h
e
r
esu
lts
in
d
icate
t
h
at
H
-
B
E
R
T
is
ef
f
ec
tiv
e
in
h
an
d
lin
g
m
u
lti
-
asp
ec
t
s
en
tim
en
t
an
aly
s
is
an
d
o
f
f
er
s
r
o
b
u
s
t
p
er
f
o
r
m
an
ce
in
asp
ec
t
ex
tr
ac
tio
n
task
s
ac
r
o
s
s
b
o
th
g
en
er
al
an
d
n
u
a
n
ce
d
r
ev
iew
co
n
tex
ts
.
T
ab
le
2.
Sem
E
v
al
2
0
1
4
d
atase
t
D
a
t
a
s
e
t
T
r
a
i
n
T
e
st
N
e
g
a
t
i
v
e
N
e
u
t
r
a
l
P
o
si
t
i
v
e
N
e
g
a
t
i
v
e
N
e
u
t
r
a
l
P
o
si
t
i
v
e
La
p
t
o
p
8
7
0
4
6
4
9
9
4
1
2
8
1
6
9
3
4
1
R
e
st
a
u
r
a
n
t
8
0
7
6
3
7
2
1
6
4
1
9
6
1
9
6
7
2
8
T
ab
le
3.
Sam
p
les
of
C
h
atGPT
g
en
er
ated
r
ev
iews
ID
R
e
v
i
e
w
A
sp
e
c
t
1
A
sp
e
c
t
1
p
o
l
a
r
i
t
y
A
sp
e
c
t
2
A
sp
e
c
t
2
p
o
l
a
r
i
t
y
A
sp
e
c
t
3
A
sp
e
c
t
3
p
o
l
a
r
i
t
y
O
v
e
r
a
l
l
sen
t
i
m
e
n
t
1
Th
e
b
a
t
t
e
r
y
l
i
f
e
is
e
x
c
e
l
l
e
n
t
b
u
t
t
h
e
s
c
r
e
e
n
q
u
a
l
i
t
y
is
p
o
o
r
a
n
d
t
h
e
c
u
st
o
m
e
r
s
u
p
p
o
r
t
w
a
s
a
v
e
r
a
g
e
.
B
a
t
t
e
r
y
l
i
f
e
P
o
si
t
i
v
e
S
c
r
e
e
n
q
u
a
l
i
t
y
N
e
g
a
t
i
v
e
C
u
s
t
o
mer
su
p
p
o
r
t
N
e
u
t
r
a
l
N
e
u
t
r
a
l
2
I
l
o
v
e
t
h
e
c
a
m
e
r
a
a
n
d
t
h
e
d
e
s
i
g
n
,
b
u
t
t
h
e
p
e
r
f
o
r
ma
n
c
e
is
sl
u
g
g
i
s
h
.
C
a
mer
a
P
o
si
t
i
v
e
D
e
si
g
n
P
o
si
t
i
v
e
P
e
r
f
o
r
ma
n
c
e
N
e
g
a
t
i
v
e
N
e
u
t
r
a
l
3
Th
e
f
o
o
d
w
a
s
d
e
l
i
c
i
o
u
s,
t
h
e
serv
i
c
e
w
a
s
q
u
i
c
k
,
but
t
h
e
a
mb
i
a
n
c
e
w
a
s
l
a
c
k
i
n
g
.
F
o
o
d
P
o
si
t
i
v
e
S
e
r
v
i
c
e
P
o
si
t
i
v
e
A
mb
i
a
n
c
e
N
e
g
a
t
i
v
e
P
o
si
t
i
v
e
4
G
r
e
a
t
p
r
i
c
e
a
n
d
d
e
l
i
v
e
r
y
t
i
m
e
,
t
h
o
u
g
h
t
h
e
p
a
c
k
a
g
i
n
g
c
o
u
l
d
be
b
e
t
t
e
r
.
P
r
i
c
e
P
o
si
t
i
v
e
D
e
l
i
v
e
r
y
t
i
m
e
P
o
si
t
i
v
e
P
a
c
k
a
g
i
n
g
N
e
u
t
r
a
l
P
o
si
t
i
v
e
5
Th
e
st
a
f
f
w
a
s
r
u
d
e
,
t
h
e
r
o
o
m
w
a
s
d
i
r
t
y
,
a
n
d
t
h
e
c
h
e
c
k
-
in
p
r
o
c
e
ss
w
a
s
c
h
a
o
t
i
c
.
S
t
a
f
f
N
e
g
a
t
i
v
e
R
o
o
m
N
e
g
a
t
i
v
e
C
h
e
c
k
-
in
p
r
o
c
e
ss
N
e
g
a
t
i
v
e
N
e
g
a
t
i
v
e
T
ab
le
4.
Sam
p
les
of
Gem
i
n
i
g
en
er
ated
r
e
v
iews
ID
R
e
v
i
e
w
A
sp
e
c
t
1
A
sp
e
c
t
1
p
o
l
a
r
i
t
y
A
sp
e
c
t
2
A
sp
e
c
t
2
p
o
l
a
r
i
t
y
A
sp
e
c
t
3
A
sp
e
c
t
3
p
o
l
a
r
i
t
y
O
v
e
r
a
l
l
sen
t
i
m
e
n
t
1
Th
e
f
o
o
d
w
a
s
a
ma
z
i
n
g
,
e
sp
e
c
i
a
l
l
y
t
h
e
p
a
st
a
,
but
t
h
e
serv
i
c
e
w
a
s
a
b
i
t
sl
o
w
.
F
o
o
d
Q
u
a
l
i
t
y
P
o
si
t
i
v
e
S
e
r
v
i
c
e
N
e
g
a
t
i
v
e
-
-
P
o
si
t
i
v
e
2
G
r
e
a
t
a
m
b
i
a
n
c
e
a
n
d
c
o
mf
o
r
t
a
b
l
e
sea
t
i
n
g
.
T
h
e
c
o
f
f
e
e
w
a
s
d
e
c
e
n
t
,
b
u
t
n
o
t
h
i
n
g
sp
e
c
i
a
l
.
A
mb
i
a
n
c
e
P
o
si
t
i
v
e
C
o
f
f
e
e
q
u
a
l
i
t
y
N
e
u
t
r
a
l
-
-
P
o
si
t
i
v
e
3
Th
e
d
e
l
i
v
e
r
y
w
a
s
s
u
p
e
r
f
a
s
t
,
h
o
w
e
v
e
r
,
t
h
e
p
i
z
z
a
a
r
r
i
v
e
d
c
o
l
d
a
n
d
so
g
g
y
.
D
e
l
i
v
e
r
y
sp
e
e
d
P
o
si
t
i
v
e
F
o
o
d
q
u
a
l
i
t
y
N
e
g
a
t
i
v
e
-
-
N
e
g
a
t
i
v
e
4
F
r
i
e
n
d
l
y
st
a
f
f
a
n
d
a
w
i
d
e
v
a
r
i
e
t
y
of
me
n
u
o
p
t
i
o
n
s.
P
r
i
c
e
s
a
r
e
a
b
i
t
h
i
g
h
t
h
o
u
g
h
.
S
e
r
v
i
c
e
P
o
si
t
i
v
e
M
e
n
u
v
a
r
i
e
t
y
P
o
si
t
i
v
e
P
r
i
c
e
N
e
g
a
t
i
v
e
P
o
si
t
i
v
e
5
Th
e
w
a
i
t
t
i
m
e
f
o
r
a
t
a
b
l
e
w
a
s
e
x
c
e
ssi
v
e
,
b
u
t
o
n
c
e
s
e
a
t
e
d
,
t
h
e
w
a
i
t
e
r
w
a
s
v
e
r
y
a
t
t
e
n
t
i
v
e
.
W
a
i
t
t
i
m
e
N
e
g
a
t
i
v
e
S
e
r
v
i
c
e
P
o
si
t
i
v
e
-
-
N
e
u
t
r
a
l
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
A
h
yb
r
id
mo
d
el
f
o
r
en
h
a
n
ce
d
a
s
p
ec
t
-
b
a
s
ed
s
en
timen
t a
n
a
lysi
s
u
s
in
g
la
r
g
e
la
n
g
u
a
g
e
…
(
Mo
h
a
mme
d
Zia
u
lla
)
1833
T
ab
le
5
.
Sam
p
les o
f
Sem
E
v
al
2
0
1
4
T
ask
4
d
ataset
ID
R
e
v
i
e
w
A
sp
e
c
t
1
A
sp
e
c
t
1
p
o
l
a
r
i
t
y
A
sp
e
c
t
2
A
sp
e
c
t
2
p
o
l
a
r
i
t
y
A
sp
e
c
t
3
A
sp
e
c
t
3
p
o
l
a
r
i
t
y
A
sp
e
c
t
4
A
sp
e
c
t
4
p
o
l
a
r
i
t
y
O
v
e
r
a
l
l
sen
t
i
m
e
n
t
1
I
t
o
o
k
i
t
b
a
c
k
f
o
r
a
n
A
su
s
a
n
d
sa
me
t
h
i
n
g
-
b
l
u
e
s
c
r
e
e
n
w
h
i
c
h
r
e
q
u
i
r
e
d
m
e
t
o
r
e
m
o
v
e
t
h
e
b
a
t
t
e
r
y
t
o
r
e
se
t
.
B
N
-
-
-
-
-
-
N
2
W
h
e
n
I
f
i
n
a
l
l
y
h
a
d
e
v
e
r
y
t
h
i
n
g
r
u
n
n
i
n
g
w
i
t
h
a
l
l
m
y
so
f
t
w
a
r
e
i
n
s
t
a
l
l
e
d
,
I
p
l
u
g
g
e
d
i
n
my
d
r
o
i
d
t
o
r
e
c
h
a
r
g
e
a
n
d
t
h
e
s
y
st
e
m
c
r
a
sh
e
d
.
S
N
S
y
st
N
e
g
-
-
-
-
N
e
g
3
H
o
w
e
v
e
r
,
t
h
e
m
u
l
t
i
-
t
o
u
c
h
g
e
s
t
u
r
e
s
a
n
d
l
a
r
g
e
t
r
a
c
k
i
n
g
a
r
e
a
mak
e
h
a
v
i
n
g
a
n
e
x
t
e
r
n
a
l
m
o
u
se
u
n
n
e
c
e
ss
a
r
y
(
u
n
l
e
ss
y
o
u
'r
e
g
a
mi
n
g
)
.
M
TG
P
TA
P
EM
N
G
N
P
4
Th
e
f
o
o
d
i
s
u
n
i
f
o
r
ml
y
e
x
c
e
p
t
i
o
n
a
l
,
w
i
t
h
a
v
e
r
y
c
a
p
a
b
l
e
k
i
t
c
h
e
n
w
h
i
c
h
w
i
l
l
p
r
o
u
d
l
y
w
h
i
p
u
p
w
h
a
t
e
v
e
r
y
o
u
f
e
e
l
l
i
k
e
e
a
t
i
n
g
,
w
h
e
t
h
e
r
i
t
's
o
n
t
h
e
men
u
o
r
n
o
t
.
F
P
K
P
M
N
-
-
P
5
Th
e
y
d
i
d
n
o
t
h
a
v
e
may
o
n
n
a
i
s
e
,
f
o
r
g
o
t
o
u
r
t
o
a
st
,
l
e
f
t
o
u
t
i
n
g
r
e
d
i
e
n
t
s
(
i
.
e
.
c
h
e
e
se
i
n
a
n
o
me
l
e
t
)
,
b
e
l
o
w
h
o
t
t
e
mp
e
r
a
t
u
r
e
s
a
n
d
t
h
e
b
a
c
o
n
w
a
s
so
o
v
e
r
c
o
o
k
e
d
i
t
c
r
u
mb
l
e
d
o
n
t
h
e
p
l
a
t
e
w
h
e
n
y
o
u
t
o
u
c
h
e
d
i
t
.
T
N
e
g
Mn
N
e
g
B
N
C
N
N
e
g
6
Th
e
se
a
t
s
a
r
e
u
n
c
o
mf
o
r
t
a
b
l
e
i
f
y
o
u
a
r
e
s
i
t
t
i
n
g
a
g
a
i
n
s
t
t
h
e
w
a
l
l
o
n
w
o
o
d
e
n
b
e
n
c
h
e
s
.
S
N
e
g
-
-
-
-
-
-
N
e
g
N
o
t
e
s
:
B
=
b
a
t
t
e
r
y
,
N
=
n
e
u
t
r
a
l
,
S
=
s
o
f
t
w
a
r
e
,
S
y
st
=
s
y
st
e
m
,
N
e
g
=
n
e
g
a
t
i
v
e
,
M
TG
=
m
u
l
t
i
-
t
o
u
c
h
g
e
s
t
u
r
e
s,
P
=
p
o
si
t
i
v
e
,
T
A
=
t
r
a
c
k
i
n
g
a
r
e
a
,
E
M
=
e
x
t
e
r
n
a
l
m
o
u
se
,
G
=
g
a
m
i
n
g
,
F
=
f
o
o
d
,
M
=
me
n
u
,
T
=
t
o
a
st
,
M
n
=
ma
y
o
n
n
a
i
se,
B
=
b
a
c
o
n
,
C
=
c
h
e
e
se,
a
n
d
S
=
se
a
t
s
T
ab
le
6
.
H
-
B
E
R
T
asp
ec
t e
x
tr
ac
tio
n
S
a
mp
l
e
s
A
c
t
u
a
l
a
s
p
e
c
t
s (
p
o
l
a
r
i
t
y
)
P
r
e
d
i
c
t
e
d
a
s
p
e
c
t
s (
p
o
l
a
r
i
t
y
)
Th
e
b
a
t
t
e
r
y
l
i
f
e
i
s
e
x
c
e
l
l
e
n
t
b
u
t
t
h
e
scree
n
q
u
a
l
i
t
y
i
s
p
o
o
r
a
n
d
t
h
e
c
u
s
t
o
m
e
r
s
u
p
p
o
r
t
w
a
s a
v
e
r
a
g
e
.
B
a
t
t
e
r
y
l
i
f
e
(
p
o
s
i
t
i
v
e
)
,
s
c
r
e
e
n
q
u
a
l
i
t
y
(
n
e
g
a
t
i
v
e
)
,
c
u
s
t
o
mer
s
u
p
p
o
r
t
(
n
e
u
t
r
a
l
)
B
a
t
t
e
r
y
l
i
f
e
(
p
o
si
t
i
v
e
)
,
scr
e
e
n
q
u
a
l
i
t
y
(
n
e
g
a
t
i
v
e
)
,
c
u
s
t
o
mer
su
p
p
o
r
t
(
n
e
u
t
r
a
l
)
Th
e
f
o
o
d
w
a
s
d
e
l
i
c
i
o
u
s,
t
h
e
s
e
r
v
i
c
e
w
a
s
q
u
i
c
k
,
b
u
t
t
h
e
a
mb
i
a
n
c
e
w
a
s
l
a
c
k
i
n
g
.
F
o
o
d
(
p
o
si
t
i
v
e
)
,
s
e
r
v
i
c
e
(
p
o
s
i
t
i
v
e
)
,
a
mb
i
a
n
c
e
(
n
e
g
a
t
i
v
e
)
F
o
o
d
(
p
o
si
t
i
v
e
)
,
serv
i
c
e
(
p
o
s
i
t
i
v
e
)
,
a
mb
i
a
n
c
e
(
n
e
g
a
t
i
v
e
)
F
r
i
e
n
d
l
y
st
a
f
f
a
n
d
a
w
i
d
e
v
a
r
i
e
t
y
o
f
m
e
n
u
o
p
t
i
o
n
s
.
P
r
i
c
e
s
a
r
e
a
b
i
t
h
i
g
h
t
h
o
u
g
h
.
S
e
r
v
i
c
e
(
p
o
si
t
i
v
e
)
,
me
n
u
(
n
e
u
t
r
a
l
)
,
v
a
r
i
e
t
y
(
p
o
s
i
t
i
v
e
)
,
p
r
i
c
e
(
n
e
g
a
t
i
v
e
)
S
e
r
v
i
c
e
(
p
o
si
t
i
v
e
)
,
p
r
i
c
e
(
n
e
u
t
r
a
l
)
Th
e
w
a
i
t
t
i
m
e
f
o
r
a
t
a
b
l
e
w
a
s
e
x
c
e
ssi
v
e
,
b
u
t
o
n
c
e
sea
t
e
d
,
t
h
e
w
a
i
t
e
r
w
a
s
v
e
r
y
a
t
t
e
n
t
i
v
e
.
W
a
i
t
t
i
me
(
n
e
g
a
t
i
v
e
)
,
ser
v
i
c
e
(
p
o
s
i
t
i
v
e
)
S
e
r
v
i
c
e
(
n
e
u
t
r
a
l
)
I
t
o
o
k
i
t
b
a
c
k
f
o
r
a
n
A
su
s
a
n
d
s
a
me
t
h
i
n
g
-
b
l
u
e
scr
e
e
n
w
h
i
c
h
r
e
q
u
i
r
e
d
me
t
o
r
e
m
o
v
e
t
h
e
b
a
t
t
e
r
y
t
o
r
e
se
t
.
B
a
t
t
e
r
y
(
n
e
u
t
r
a
l
)
B
a
t
t
e
r
y
(
n
e
u
t
r
a
l
)
Th
e
f
o
o
d
i
s
u
n
i
f
o
r
ml
y
e
x
c
e
p
t
i
o
n
a
l
,
w
i
t
h
a
v
e
r
y
c
a
p
a
b
l
e
k
i
t
c
h
e
n
w
h
i
c
h
w
i
l
l
p
r
o
u
d
l
y
w
h
i
p
u
p
w
h
a
t
e
v
e
r
y
o
u
f
e
e
l
l
i
k
e
e
a
t
i
n
g
,
w
h
e
t
h
e
r
i
t
's
o
n
t
h
e
me
n
u
o
r
n
o
t
.
F
o
o
d
(
p
o
s
i
t
i
v
e
)
,
k
i
t
c
h
e
n
(
p
o
s
i
t
i
v
e
)
,
men
u
(
n
e
u
t
r
a
l
)
F
o
o
d
(
p
o
s
i
t
i
v
e
)
,
me
n
u
(
n
e
u
t
r
a
l
)
T
h
is
s
ec
tio
n
ev
alu
ates
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
H
-
B
E
R
T
m
o
d
el
in
d
ete
r
m
in
in
g
th
e
o
v
er
all
s
en
tim
en
t
p
o
lar
ity
o
f
cu
s
to
m
er
r
e
v
iews.
T
ab
le
7
co
m
p
ar
es
th
e
p
r
ed
ic
ted
r
ev
iew
p
o
lar
ities
with
th
e
ac
tu
al
lab
els.
T
h
e
r
esu
lts
s
h
o
w
th
at
H
-
B
E
R
T
p
er
f
o
r
m
s
well
in
m
o
s
t
ca
s
es,
co
r
r
ec
tly
id
en
tify
in
g
t
h
e
o
v
e
r
all
s
en
tim
en
t
as
p
o
s
itiv
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
2
,
Ap
r
il 2
0
2
6
:
1
8
2
5
-
1
8
3
8
1834
o
r
n
e
u
tr
al.
Ho
we
v
er
,
th
er
e
ar
e
in
s
tan
ce
s
wh
er
e
t
h
e
m
o
d
el
s
l
ig
h
tly
m
is
ju
d
g
es
th
e
r
e
v
iew's
to
n
e.
Fo
r
ex
am
p
le,
in
th
e
f
ir
s
t
s
am
p
le,
w
h
er
e
b
o
t
h
p
o
s
itiv
e
an
d
n
e
g
ativ
e
asp
ec
ts
ar
e
p
r
esen
t,
H
-
B
E
R
T
class
i
f
ied
th
e
r
ev
iew
as
n
eg
ativ
e,
wh
ile
th
e
ac
tu
al
la
b
el
was
n
eu
tr
al.
Similar
ly
,
in
an
o
th
e
r
ca
s
e,
a
r
e
v
iew
lab
eled
p
o
s
itiv
e
was
p
r
ed
icted
as
n
eu
tr
al,
lik
ely
d
u
e
to
th
e
m
en
tio
n
o
f
a
n
e
g
ativ
e
asp
ec
t
s
u
ch
as
h
ig
h
p
r
icin
g
.
T
h
ese
v
ar
iatio
n
s
s
u
g
g
est
th
at
wh
ile
H
-
B
E
R
T
ef
f
ec
tiv
ely
h
a
n
d
les g
en
er
al
s
en
tim
en
t
class
if
icatio
n
,
it o
cc
asio
n
ally
s
tr
u
g
g
les
with
b
alan
cin
g
co
n
f
lictin
g
s
en
tim
en
ts
ac
r
o
s
s
m
u
ltip
le
asp
ec
ts
.
Desp
ite
th
is
,
th
e
m
o
d
el
d
e
m
o
n
s
tr
ates
s
tr
o
n
g
p
o
ten
tial in
ca
p
tu
r
in
g
o
v
er
all
s
en
tim
en
t in
cu
s
to
m
er
f
ee
d
b
ac
k
in
[
2
9
]
,
[
3
0
]
.
T
h
is
s
ec
tio
n
h
ig
h
lig
h
ts
th
e
class
if
icatio
n
p
er
f
o
r
m
an
ce
o
f
th
e
H
-
B
E
R
T
m
o
d
el,
ev
alu
ated
u
s
in
g
ac
cu
r
ac
y
,
an
d
m
ac
r
o
-
F
-
s
co
r
e
o
n
Sem
E
v
al
2
0
1
4
d
ataset
ac
r
o
s
s
b
o
th
lap
to
p
an
d
r
estau
r
an
t
d
o
m
ain
s
.
As
illu
s
tr
ated
in
Fig
u
r
e
4
,
H
-
B
E
R
T
ac
h
iev
ed
an
ac
cu
r
ac
y
o
f
9
0
.
5
8
%
an
d
m
ac
r
o
-
F
-
s
co
r
e
o
f
9
0
.
5
6
%
in
th
e
lap
to
p
d
o
m
ain
,
an
d
an
ev
e
n
h
ig
h
er
ac
cu
r
ac
y
o
f
9
1
.
2
1
%
a
n
d
m
ac
r
o
-
F
-
s
co
r
e
o
f
9
2
.
0
3
%
i
n
th
e
r
estau
r
an
t
d
o
m
ain
.
T
h
ese
m
etr
ics
in
d
icate
t
h
at
th
e
h
y
b
r
id
a
p
p
r
o
ac
h
s
ig
n
i
f
ican
tly
im
p
r
o
v
es
b
o
th
p
r
ec
is
io
n
a
n
d
r
ec
all
in
s
en
tim
en
t
class
if
icatio
n
task
s
.
T
h
e
in
clu
s
io
n
o
f
L
L
M
-
g
en
er
ated
d
atasets
ap
p
ea
r
s
to
en
h
a
n
ce
th
e
m
o
d
el’
s
g
en
er
aliza
tio
n
ca
p
ab
ilit
ies,
allo
win
g
it to
p
er
f
o
r
m
co
n
s
is
ten
tly
ac
r
o
s
s
d
o
m
ai
n
s
.
T
ab
le
7
.
H
-
B
E
R
T
r
ev
iew
p
o
lar
ity
o
u
tco
m
e
S
a
mp
l
e
s
A
c
t
u
a
l
r
e
v
i
e
w
p
o
l
a
r
i
t
y
P
r
e
d
i
c
t
e
d
r
e
v
i
e
w
p
o
l
a
r
i
t
y
Th
e
b
a
t
t
e
r
y
l
i
f
e
i
s
e
x
c
e
l
l
e
n
t
b
u
t
t
h
e
scre
e
n
q
u
a
l
i
t
y
i
s
p
o
o
r
a
n
d
t
h
e
c
u
st
o
m
e
r
su
p
p
o
r
t
w
a
s a
v
e
r
a
g
e
.
N
e
u
t
r
a
l
N
e
g
a
t
i
v
e
Th
e
f
o
o
d
w
a
s
d
e
l
i
c
i
o
u
s,
t
h
e
s
e
r
v
i
c
e
w
a
s q
u
i
c
k
,
b
u
t
t
h
e
a
m
b
i
a
n
c
e
w
a
s l
a
c
k
i
n
g
.
P
o
si
t
i
v
e
P
o
si
t
i
v
e
F
r
i
e
n
d
l
y
s
t
a
f
f
a
n
d
a
w
i
d
e
v
a
r
i
e
t
y
o
f
m
e
n
u
o
p
t
i
o
n
s.
P
r
i
c
e
s
a
r
e
a
b
i
t
h
i
g
h
t
h
o
u
g
h
.
P
o
si
t
i
v
e
N
e
u
t
r
a
l
Th
e
w
a
i
t
t
i
me
f
o
r
a
t
a
b
l
e
w
a
s
e
x
c
e
s
si
v
e
,
b
u
t
o
n
c
e
s
e
a
t
e
d
,
t
h
e
w
a
i
t
e
r
w
a
s
v
e
r
y
a
t
t
e
n
t
i
v
e
.
N
e
u
t
r
a
l
N
e
u
t
r
a
l
I
t
o
o
k
i
t
b
a
c
k
f
o
r
a
n
A
s
u
s
a
n
d
sam
e
t
h
i
n
g
-
b
l
u
e
s
c
r
e
e
n
w
h
i
c
h
r
e
q
u
i
r
e
d
me
t
o
r
e
mo
v
e
t
h
e
b
a
t
t
e
r
y
t
o
r
e
s
e
t
.
N
e
u
t
r
a
l
N
e
u
t
r
a
l
Th
e
f
o
o
d
i
s
u
n
i
f
o
r
m
l
y
e
x
c
e
p
t
i
o
n
a
l
,
w
i
t
h
a
v
e
r
y
c
a
p
a
b
l
e
k
i
t
c
h
e
n
w
h
i
c
h
w
i
l
l
p
r
o
u
d
l
y
w
h
i
p
u
p
w
h
a
t
e
v
e
r
y
o
u
f
e
e
l
l
i
k
e
e
a
t
i
n
g
,
w
h
e
t
h
e
r
i
t
's
o
n
t
h
e
m
e
n
u
o
r
n
o
t
.
P
o
si
t
i
v
e
P
o
si
t
i
v
e
Fig
u
r
e
4
.
H
-
B
E
R
T
p
er
f
o
r
m
an
c
e
ev
alu
atio
n
o
n
Sem
E
v
al
2
0
1
4
d
ataset
A
co
m
p
ar
ativ
e
an
aly
s
is
was
co
n
d
u
cte
d
to
ev
alu
ate
th
e
p
e
r
f
o
r
m
a
n
ce
o
f
p
r
o
p
o
s
ed
H
-
B
E
R
T
m
o
d
el
ag
ain
s
t
s
ev
er
al
ex
is
tin
g
s
tate
-
of
-
th
e
-
ar
t
a
p
p
r
o
ac
h
es
in
AB
SA,
as
p
r
esen
ted
i
n
T
ab
le
8
.
T
r
ad
i
tio
n
al
g
r
ap
h
-
b
ased
m
o
d
els
s
u
ch
as
SAGC
N
[
1
1
]
an
d
SD
-
GC
N
[
1
3
]
ac
h
iev
e
d
m
o
d
er
ate
ac
c
u
r
ac
y
an
d
m
ac
r
o
-
F
-
s
co
r
e,
with
SAGC
N
s
co
r
in
g
8
3
.
0
6
%
ac
cu
r
ac
y
an
d
7
9
.
6
9
%
m
ac
r
o
-
F
-
s
co
r
e
in
th
e
lap
to
p
d
o
m
ain
.
Sy
n
Sem
-
ASTE
[
1
6
]
a
n
d
its
tr
an
s
f
o
r
m
er
-
b
ased
v
ar
ian
t
s
lik
e
B
E
R
T
,
r
o
b
u
s
tly
o
p
ti
m
ized
B
E
R
T
p
r
etr
ain
in
g
ap
p
r
o
ac
h
(
R
o
B
E
R
T
a)
,
an
d
DeBER
T
a
in
teg
r
atio
n
s
h
o
wed
lo
wer
m
ac
r
o
-
F
-
s
co
r
es,
p
ar
ticu
lar
ly
in
f
in
e
-
g
r
ain
e
d
AB
S
A
task
s
,
with
m
ac
r
o
-
F
-
s
co
r
e
r
an
g
in
g
f
r
o
m
5
9
.
7
3
%
to
7
4
.
7
3
%.
B
E
R
T
-
CA
-
W
M
A
[
2
0
]
p
er
f
o
r
m
e
d
b
etter
,
ac
h
iev
in
g
an
ac
cu
r
ac
y
o
f
8
9
.
7
2
%
an
d
m
ac
r
o
-
F
-
s
co
r
e
o
f
8
9
.
8
1
%
in
th
e
lap
to
p
d
o
m
ain
.
Ho
wev
er
,
th
e
p
r
o
p
o
s
ed
H
-
B
E
R
T
o
u
tp
er
f
o
r
m
ed
all
m
o
d
els,
ac
h
iev
in
g
9
0
.
5
8
%
ac
cu
r
ac
y
an
d
9
0
.
5
6
%
m
ac
r
o
-
F
-
s
co
r
e
f
o
r
la
p
to
p
s
,
an
d
9
1
.
2
1
%
ac
cu
r
ac
y
with
9
2
.
0
3
%
m
ac
r
o
-
F
-
s
co
r
e
f
o
r
r
estau
r
an
ts
.
T
h
e
in
teg
r
atio
n
o
f
Sp
an
B
E
R
T
,
B
iLST
M,
C
R
F
,
an
d
LLM
-
g
en
er
ated
d
ata
m
ak
es
H
-
B
E
R
T
m
o
r
e
r
o
b
u
s
t
an
d
ef
f
ec
tiv
e
in
h
an
d
lin
g
b
o
th
h
u
m
an
-
an
n
o
tated
an
d
s
y
n
th
etic
r
ev
iew
d
ata,
lead
i
n
g
to
s
u
p
er
io
r
p
er
f
o
r
m
an
ce
in
AB
SA.
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
ab
latio
n
s
tu
d
y
wh
ich
in
v
es
tig
ates
th
e
im
p
ac
t
o
f
i
n
co
r
p
o
r
atin
g
LLM
-
g
en
er
ated
d
ata
(
f
r
o
m
C
h
atGPT
an
d
Gem
in
i)
in
to
th
e
H
-
B
E
R
T
f
r
am
ewo
r
k
f
o
r
b
o
th
as
p
ec
t e
x
tr
ac
tio
n
an
d
r
ev
iew
p
o
lar
it
y
task
s
.
As
s
h
o
wn
in
T
ab
le
9
,
a
q
u
alitativ
e
co
m
p
ar
is
o
n
ac
r
o
s
s
m
o
d
els
h
i
g
h
lig
h
ts
th
at
w
h
ile
C
h
atGPT
an
d
Gem
in
i
p
er
f
o
r
m
well
in
id
en
tif
y
in
g
m
o
s
t
asp
ec
ts
an
d
p
o
lar
ities
,
th
ey
o
cc
asio
n
ally
m
is
class
if
y
th
e
o
v
e
r
all
s
en
tim
en
t
o
r
f
ail
t
o
ex
tr
ac
t
all
r
ele
v
an
t
asp
ec
ts
.
Fo
r
in
s
tan
ce
,
in
t
h
e
th
i
r
d
r
ev
i
ew
,
b
o
th
C
h
atGPT
Evaluation Warning : The document was created with Spire.PDF for Python.