Co
m
pu
t
er
Science
a
nd
I
nfo
r
m
a
t
io
n T
ec
hn
o
lo
g
ies
Vo
l.
7
,
No
.
3
,
No
v
em
b
er
20
26
,
p
p
.
4
0
4
~
4
1
8
I
SS
N:
2722
-
3
2
2
1
,
DOI
:
1
0
.
1
1
5
9
1
/csi
t
.
v
7
i
3
.
p
4
0
4
-
4
1
8
404
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ia
e
s
p
r
ime.
co
m/in
d
ex
.
p
h
p
/csi
t
A
co
mpa
ra
tive
re
v
iew
of
mo
dern
la
rg
e
la
ng
ua
g
e
mo
del
pa
ra
dig
ms
:
G
PT
-
4,
BE
RT,
G
emini
,
a
nd
Deep
Seek
K
a
v
is
h
Sa
ng
hv
i
1
,
Apa
rna
S.
Sh
a
rm
a
2
,
Su
rbhi
H
o
o
da
3
1
D
e
p
a
r
t
me
n
t
of
S
o
f
t
w
a
r
e
En
g
i
n
e
e
r
i
n
g
,
S
t
e
v
e
n
s
I
n
st
i
t
u
t
e
of
T
e
c
h
n
o
l
o
g
y
,
H
o
b
o
k
e
n
,
U
n
i
t
e
d
S
t
a
t
e
s
of
A
m
e
r
i
c
a
2
D
a
n
i
e
l
J.
Ep
s
t
e
i
n
D
e
p
a
r
t
me
n
t
of
I
n
d
u
st
r
i
a
l
a
n
d
S
y
st
e
ms
E
n
g
i
n
e
e
r
i
n
g
,
U
n
i
v
e
r
si
t
y
of
S
o
u
t
h
e
r
n
C
a
l
i
f
o
r
n
i
a
,
L
o
s
A
n
g
e
l
e
s,
U
n
i
t
e
d
S
t
a
t
e
s
of
A
meri
c
a
3
D
e
p
a
r
t
me
n
t
of
B
u
si
n
e
ss
,
U
n
i
v
e
r
s
i
t
y
of
t
h
e
P
o
t
o
mac
,
W
a
sh
i
n
g
t
o
n
,
D
.
C
.
,
U
n
i
t
e
d
S
t
a
t
e
s
of
A
mer
i
c
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
u
n
12
,
2
0
2
5
R
ev
is
ed
J
u
n
26
,
2
0
2
6
Acc
ep
ted
J
u
l
13
,
2
0
2
6
Th
is
re
v
iew
p
r
o
v
i
d
e
s
c
o
m
p
a
ra
ti
v
e
a
n
a
ly
sis
of
G
P
T
-
4,
BERT
(b
i
d
irec
ti
o
n
a
l
e
n
c
o
d
e
r
re
p
re
se
n
tatio
n
s
fro
m
tr
a
n
sfo
rm
e
rs)
,
G
e
m
in
i,
a
n
d
De
e
p
S
e
e
k
larg
e
lan
g
u
a
g
e
m
o
d
e
ls
(LL
M
)
,
fo
c
u
si
n
g
a
rc
h
it
e
c
tu
re
s,
train
i
n
g
m
e
th
o
d
o
l
o
g
ies
,
a
n
d
re
a
l
-
wo
rld
a
p
p
li
c
a
ti
o
n
s.
Th
e
p
ri
m
a
ry
re
se
a
rc
h
q
u
e
stio
n
is:
Ho
w
do
t
h
e
se
m
o
d
e
ls
d
iffer
in
d
e
sig
n
,
stre
n
g
th
s,
li
m
it
a
ti
o
n
s,
a
n
d
p
o
ten
ti
a
l
a
re
a
s
fo
r
e
n
h
a
n
c
e
m
e
n
t?
By
a
d
d
re
ss
in
g
t
h
is
q
u
e
sti
o
n
,
t
h
e
stu
d
y
a
ims
to
p
ro
v
i
d
e
in
sig
h
ts
i
n
to
th
e
trad
e
-
o
ffs
a
n
d
fu
t
u
re
d
irec
ti
o
n
s
f
o
r
o
p
t
imiz
in
g
LLM
p
e
rfo
rm
a
n
c
e
a
n
d
d
e
p
lo
y
m
e
n
t.
T
h
e
a
n
a
ly
sis
re
v
e
a
ls
G
P
T
-
4
e
x
c
e
ls
in
n
a
tu
ra
l
lan
g
u
a
g
e
g
e
n
e
ra
ti
o
n
a
n
d
c
o
m
p
le
x
re
a
so
n
in
g
,
su
p
p
o
rti
n
g
u
p
to
1
2
8
K
to
k
e
n
s
with
m
o
d
e
ra
te
late
n
c
y
a
n
d
h
ig
h
e
r
c
o
sts
m
a
k
in
g
it
e
ffe
c
ti
v
e
fo
r
c
o
n
v
e
rsa
ti
o
n
a
l
a
rti
ficia
l
in
telli
g
e
n
c
e
(AI)
.
BERT
e
x
c
e
ls
b
i
d
irec
ti
o
n
a
l
c
o
n
tex
t
u
a
l
u
n
d
e
rsta
n
d
i
n
g
wi
th
s
m
a
ll
e
r
c
o
m
p
u
tatio
n
a
l
o
v
e
rh
e
a
d
a
n
d
b
ro
a
d
o
p
e
n
-
s
o
u
rc
e
a
d
o
p
ti
o
n
,
e
ffe
c
ti
v
e
f
o
r
tex
t
c
las
sifica
ti
o
n
.
G
e
m
in
i
d
e
m
o
n
stra
tes
su
p
e
rio
r
m
u
lt
imo
d
a
l
i
n
teg
ra
ti
o
n
p
ro
c
e
ss
in
g
tex
t,
ima
g
e
,
a
u
d
i
o
,
a
n
d
c
o
d
e
with
c
o
n
tex
t
le
n
g
t
h
s
up
to
1M
t
o
k
e
n
s,
e
n
a
b
li
n
g
c
ro
ss
-
d
o
m
a
in
a
d
a
p
tab
il
it
y
.
De
e
p
S
e
e
k
e
x
c
e
ls
in
sp
e
c
ialize
d
d
o
m
a
in
s
li
k
e
fin
a
n
c
e
a
n
d
p
r
o
g
ra
m
m
in
g
,
is
o
p
ti
m
ize
d
f
o
r
e
fficie
n
c
y
a
n
d
su
p
p
o
rts
e
x
te
n
d
e
d
c
o
n
te
x
t
win
d
o
ws
e
x
c
e
e
d
in
g
2
0
0
K
t
o
k
e
n
s.
Ho
we
v
e
r,
a
ll
m
o
d
e
ls
fa
c
e
c
h
a
ll
e
n
g
e
s
re
late
d
to
c
o
m
p
u
tati
o
n
a
l
c
o
st
,
h
a
ll
u
c
in
a
t
i
o
n
s,
a
n
d
e
th
ica
l
c
o
n
c
e
rn
s,
n
e
c
e
ss
it
a
ti
n
g
fu
rth
e
r
im
p
ro
v
e
m
e
n
ts.
De
sp
it
e
a
d
v
a
n
c
e
m
e
n
ts,
LL
M
s
c
o
n
ti
n
u
e
to
g
ra
p
p
le
with
issu
e
s
su
c
h
as
d
a
ta
b
ias
,
m
o
d
e
l
in
ter
p
re
tab
il
it
y
,
a
n
d
re
sp
o
n
sib
le
AI
d
e
p
lo
y
m
e
n
t.
F
u
tu
re
re
se
a
rc
h
sh
o
u
l
d
fo
c
u
s
on
h
y
b
ri
d
m
o
d
e
l
a
p
p
r
o
a
c
h
e
s,
d
o
m
a
in
-
sp
e
c
ifi
c
fi
n
e
-
tu
n
in
g
,
a
n
d
tran
sp
a
re
n
c
y
to
m
it
i
g
a
te
risk
s
wh
il
e
m
a
x
imiz
in
g
t
h
e
tran
sfo
rm
a
ti
v
e
p
o
ten
ti
a
l
of
LL
M
s
in
re
a
l
-
wo
rl
d
a
p
p
li
c
a
ti
o
n
s.
K
ey
w
o
r
d
s
:
B
E
R
T
Dee
p
Seek
Gem
in
i
GPT
-
4
L
ar
g
e
lan
g
u
ag
e
m
o
d
els
T
h
is
is
an
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r
th
e
CC
BY
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Kav
is
h
San
g
h
v
i
Dep
ar
tm
en
t
of
So
f
twar
e
E
n
g
in
ee
r
in
g
,
Stev
e
n
s
I
n
s
titu
te
of
T
ec
h
n
o
lo
g
y
1
C
astl
e
Po
in
t
T
er
,
Ho
b
o
k
e
n
0
7
0
3
0
,
New
J
er
s
ey
,
Un
ited
States
of
Am
er
ica
E
m
ail:
s
an
g
h
v
i_
k
a
v
is
h
@
y
ah
o
o
.
in
1.
I
NT
RO
D
UCT
I
O
N
Desp
ite
th
e
r
ap
id
ad
v
an
ce
m
e
n
t
of
lar
g
e
lan
g
u
a
g
e
m
o
d
els
(
L
L
Ms)
,
s
elec
tin
g
th
e
m
o
s
t
s
u
ita
b
le
m
o
d
el
f
o
r
a
g
iv
en
task
r
em
ain
s
a
co
m
p
lex
ch
allen
g
e
d
u
e
to
v
ar
ied
ar
ch
itectu
r
al
d
esig
n
s
,
tr
ain
in
g
d
ata
r
eg
im
es,
co
s
t
s
tr
u
ctu
r
es,
an
d
m
o
d
ality
s
u
p
p
o
r
t.
E
x
is
tin
g
s
tu
d
ies
o
f
te
n
p
r
o
v
id
e
i
n
d
iv
id
u
al
p
er
f
o
r
m
an
ce
r
ep
o
r
ts
or
b
en
ch
m
ar
k
s
,
b
u
t
a
co
m
p
r
eh
e
n
s
iv
e
an
d
p
r
ac
tical
co
m
p
a
r
is
o
n
,
esp
ec
ially
co
v
e
r
in
g
r
ec
en
t
m
o
d
els
lik
e
GPT
-
4,
Gem
in
i,
an
d
Dee
p
Seek
,
is
lack
in
g
.
T
h
is
p
ap
er
ad
d
r
ess
es
th
is
g
ap
by
ev
alu
atin
g
lead
in
g
LLMs
ac
r
o
s
s
co
r
e
d
im
en
s
io
n
s
s
u
ch
as
ar
c
h
itectu
r
e,
d
o
m
ain
s
p
ec
ializatio
n
,
m
u
ltimo
d
al
ca
p
a
b
ilit
ies,
an
d
r
ea
l
-
wo
r
ld
u
s
ab
ilit
y
.
Evaluation Warning : The document was created with Spire.PDF for Python.
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
I
SS
N:
2722
-
3
2
2
1
A
co
mp
a
r
a
tive
r
ev
iew
o
f m
o
d
e
r
n
la
r
g
e
la
n
g
u
a
g
e
mo
d
el
p
a
r
a
d
ig
ms:
GP
T
-
4
,
B
E
R
T,
…
(
K
a
v
is
h
S
a
n
g
h
vi
)
405
T
h
e
n
o
v
elty
of
th
is
wo
r
k
lies
in
its
s
y
n
th
esis
of
tech
n
ical
ch
ar
ac
ter
is
tics
w
ith
d
ep
lo
y
m
en
t
s
u
itab
ilit
y
,
m
ak
in
g
it
a
p
r
ac
tical
r
ef
er
en
ce
f
o
r
r
esear
ch
er
s
an
d
i
n
d
u
s
tr
y
p
r
ac
titi
o
n
er
s
alik
e
[
1
]
–
[
5
]
.
On
e
of
th
e
b
ig
g
est
ad
v
an
ce
s
in
ar
tific
ial
in
tellig
en
ce
(
AI
)
is
LLMs
,
p
ar
ticu
lar
ly
n
atu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
(
NL
P).
B
ein
g
ab
le
to
co
n
s
tr
u
ct
h
u
m
an
-
lik
e
tex
t
in
a
v
ar
iety
of
u
s
e
ca
s
es
an
d
co
n
tex
ts
h
as
p
r
o
v
en
to
be
in
v
al
u
ab
le
to
in
d
u
s
tr
ie
s
s
u
ch
as
f
in
an
ce
,
h
ea
lth
ca
r
e,
an
d
e
d
u
ca
tio
n
.
LLMs
r
ely
lar
g
ely
on
n
eu
r
al
n
etwo
r
k
ar
c
h
itectu
r
e,
d
ee
p
le
ar
n
in
g
,
a
n
d
lar
g
e
d
atasets
.
L
an
g
u
ag
e
m
o
d
els
p
r
ed
ict
wo
r
d
s
in
a
s
en
ten
ce
an
d
,
af
ter
s
ee
in
g
en
o
u
g
h
d
ata,
can
s
u
g
g
est
th
e
m
o
s
t
u
s
ef
u
l
an
d
c
o
n
tex
t
-
r
elate
d
r
esp
o
n
s
e.
NL
P
LLMs
r
eq
u
ir
e
a
lo
t
of
co
m
p
u
tin
g
r
eso
u
r
ce
s
.
T
r
a
n
s
f
o
r
m
er
a
r
ch
itectu
r
e
is
a
n
o
v
el
n
eu
r
al
ar
ch
itectu
r
e
an
d
lear
n
in
g
f
r
am
ewo
r
k
in
tr
o
d
u
ce
d
by
Vaswan
i
et
a
l
.
[
6
]
m
a
k
in
g
it
ea
s
ier
to
s
ca
le
th
ese
m
o
d
els.
LLMs
also
u
s
e
a
m
ec
h
an
is
m
ca
lled
s
elf
-
atten
tio
n
,
wh
ich
allo
ws
t
h
e
ca
p
tu
r
e
of
c
o
m
p
lex
r
elatio
n
s
h
ip
s
b
etwe
en
co
m
p
o
n
en
ts
of
a
tex
t,
allo
win
g
th
em
to
r
eso
lv
e
m
u
ltip
le
am
b
i
g
u
ities
.
Am
o
n
g
th
e
m
o
s
t
p
r
o
m
in
e
n
t
L
L
Ms
in
co
n
tem
p
o
r
a
r
y
AI
r
ese
ar
ch
an
d
a
p
p
licatio
n
ar
e
GPT
-
4,
B
E
R
T
(
b
id
ir
ec
tio
n
al
e
n
co
d
e
r
r
ep
r
esen
tatio
n
s
f
r
o
m
tr
an
s
f
o
r
m
er
s
)
,
Gem
in
i,
an
d
Dee
p
Seek
.
E
a
ch
of
t
h
ese
m
o
d
els
em
b
o
d
ies
u
n
iq
u
e
ar
c
h
itectu
r
al
in
n
o
v
atio
n
s
a
n
d
tr
ain
in
g
m
eth
o
d
o
lo
g
ies,
tailo
r
ed
to
s
p
ec
if
ic
task
s
an
d
p
er
f
o
r
m
an
ce
o
b
jectiv
es.
GPT
-
4,
an
au
to
r
e
g
r
ess
iv
e
m
o
d
el,
b
u
ild
s
upon
its
p
r
ed
ec
ess
o
r
s
by
em
p
lo
y
in
g
a
d
ec
o
d
e
r
-
o
n
l
y
tr
an
s
f
o
r
m
er
ar
ch
itectu
r
e
o
p
tim
ized
f
o
r
l
o
n
g
-
c
o
n
tex
t
u
n
d
er
s
tan
d
in
g
a
n
d
g
e
n
er
ativ
e
tex
t
p
r
o
d
u
ctio
n
.
It
ex
ce
ls
in
task
s
r
eq
u
ir
in
g
cr
ea
tiv
ity
,
p
r
o
b
lem
-
s
o
lv
in
g
,
an
d
n
u
a
n
ce
d
lan
g
u
a
g
e
g
en
er
atio
n
[
7
]
.
In
co
n
t
r
ast,
B
E
R
T
ad
o
p
ts
a
b
id
ir
ec
tio
n
al
tr
a
n
s
f
o
r
m
e
r
en
co
d
er
,
wh
ic
h
p
r
o
ce
s
s
es
tex
t
in
b
o
th
d
ir
ec
tio
n
s
s
im
u
ltan
eo
u
s
ly
.
T
h
is
b
id
ir
ec
tio
n
al
p
r
o
ce
s
s
in
g
en
h
a
n
ce
s
its
co
n
tex
tu
al
u
n
d
er
s
tan
d
in
g
,
m
ak
in
g
B
E
R
T
p
ar
ticu
lar
ly
ef
f
ec
tiv
e
f
o
r
ap
p
licatio
n
s
s
u
ch
as
s
en
tim
en
t
an
aly
s
is
,
q
u
esti
o
n
an
s
wer
in
g
,
an
d
en
tity
r
ec
o
g
n
itio
n
[
8
]
.
Gem
in
i
r
ep
r
esen
ts
an
ev
o
lu
tio
n
in
LLMs
by
in
c
o
r
p
o
r
atin
g
m
u
ltimo
d
al
ca
p
a
b
ilit
ies,
en
ab
lin
g
th
e
in
teg
r
atio
n
of
tex
t,
v
is
u
al,
an
d
au
d
ito
r
y
d
ata.
T
h
is
cr
o
s
s
-
m
o
d
al
r
ea
s
o
n
in
g
ab
ilit
y
f
ac
ilit
ates
more
co
m
p
r
eh
e
n
s
iv
e
AI
ap
p
licatio
n
s
in
ar
ea
s
s
u
ch
as
in
ter
ac
tiv
e
ed
u
ca
tio
n
an
d
m
e
d
ical
d
i
ag
n
o
s
tics
.
Gem
in
i
’
s
ar
ch
itectu
r
e
ex
te
n
d
s
tr
ad
itio
n
al
tr
an
s
f
o
r
m
er
s
to
ac
co
m
m
o
d
ate
m
u
ltimo
d
al
em
b
ed
d
i
n
g
s
,
d
r
awin
g
in
s
p
ir
atio
n
f
r
o
m
m
o
d
els
lik
e
co
n
tr
asti
v
e
lan
g
u
ag
e
–
im
ag
e
p
r
e
-
tr
ain
in
g
(
C
L
I
P
)
[
9
]
,
wh
ich
s
u
cc
ess
f
u
lly
m
er
g
e
v
is
u
al
an
d
tex
tu
al
r
ep
r
esen
tatio
n
s
.
On
t
h
e
o
th
e
r
h
an
d
,
Dee
p
Seek
,
th
o
u
g
h
less
p
u
b
licly
d
o
cu
m
en
t
ed
,
is
d
esig
n
e
d
f
o
r
d
o
m
ain
-
s
p
ec
if
ic
ap
p
licatio
n
s
.
By
in
co
r
p
o
r
atin
g
tar
g
eted
tr
ain
in
g
s
tr
ateg
ies
an
d
ar
c
h
itectu
r
al
o
p
tim
izatio
n
s
,
Dee
p
Seek
ac
h
iev
es
h
ig
h
ef
f
i
cien
cy
in
s
p
ec
ialized
task
s
s
u
ch
as
f
in
an
cial
m
o
d
elin
g
an
d
co
de
a
n
aly
s
is
.
I
ts
d
o
m
ain
-
ad
a
p
tiv
e
d
esig
n
a
lig
n
s
with
p
r
in
cip
les
of
tr
an
s
f
er
lear
n
in
g
,
wh
er
e
f
i
n
e
-
tu
n
i
n
g
en
h
a
n
ce
s
m
o
d
el
p
er
f
o
r
m
an
ce
on
p
a
r
ticu
lar
ap
p
licatio
n
s
[
1
0
]
.
T
h
e
in
cr
ea
s
in
g
r
elian
ce
on
LLMs
r
aises
cr
u
cial
q
u
esti
o
n
s
ab
o
u
t
th
eir
u
n
d
er
l
y
in
g
ar
ch
itectu
r
es,
tr
ain
in
g
m
eth
o
d
o
lo
g
ies,
an
d
r
ea
l
-
wo
r
ld
ef
f
ec
tiv
en
ess
.
T
h
e
p
r
im
ar
y
r
esear
ch
q
u
esti
o
n
g
u
id
in
g
th
is
co
m
p
ar
ativ
e
r
e
v
iew
is
:
Ho
w
do
GPT
-
4,
B
E
R
T
,
Gem
in
i,
an
d
Dee
p
Seek
d
if
f
e
r
in
th
ei
r
s
tr
u
ctu
r
al
d
esig
n
,
lear
n
in
g
p
ar
a
d
ig
m
s
,
an
d
a
p
p
l
icatio
n
-
s
p
ec
if
ic
p
er
f
o
r
m
an
ce
,
an
d
wh
at
en
h
a
n
ce
m
en
ts
can
be
im
p
lem
en
ted
to
ad
d
r
ess
th
eir
r
esp
ec
tiv
e
lim
ita
tio
n
s
?
By
s
y
s
tem
at
ically
ex
am
in
in
g
th
ese
m
o
d
els,
th
is
p
ap
er
aim
s
to
elu
cid
ate
th
e
tr
ad
e
-
o
f
f
s
in
h
er
e
n
t
in
each
d
esig
n
,
p
r
o
v
id
in
g
i
n
s
ig
h
ts
in
t
o
th
eir
o
p
tim
al
u
s
e
ca
s
es
an
d
f
u
tu
r
e
d
ev
el
o
p
m
en
t
tr
ajec
to
r
ies.
T
h
is
r
e
v
iew
p
r
o
c
ee
d
s
with
a
d
etailed
ex
p
lo
r
atio
n
of
k
ey
asp
ec
ts
th
at
d
ef
in
e
LLM
p
er
f
o
r
m
a
n
ce
an
d
ap
p
licab
ilit
y
.
Firstl
y
,
it
d
elv
es
in
to
th
e
tech
n
ical
u
n
d
er
p
in
n
in
g
s
of
each
m
o
d
el,
in
clu
d
in
g
th
eir
a
r
ch
itectu
r
al
f
r
a
m
ewo
r
k
s
,
s
elf
-
atten
tio
n
m
ec
h
an
is
m
s
,
em
b
ed
d
in
g
s
tr
ateg
ies,
an
d
to
k
e
n
izatio
n
m
eth
o
d
s
.
Seco
n
d
ly
,
it
an
aly
ze
s
th
eir
tr
ain
in
g
m
eth
o
d
o
lo
g
ies,
s
u
ch
as
th
e
n
atu
r
e
of
th
eir
d
atasets
,
p
r
e
tr
ain
in
g
o
b
jectiv
es,
f
in
e
-
tu
n
in
g
ap
p
r
o
ac
h
es,
an
d
th
e
r
o
le
of
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
in
o
p
tim
izin
g
p
er
f
o
r
m
an
ce
.
T
h
ir
d
ly
,
th
e
r
ev
iew
ev
al
u
ates
th
e
p
r
ac
t
ical
ap
p
licatio
n
s
of
th
ese
m
o
d
els,
co
n
s
id
er
in
g
t
h
eir
s
tr
en
g
th
s
an
d
lim
itatio
n
s
in
v
ar
io
u
s
d
o
m
ai
n
s
.
GPT
-
4
’
s
g
en
er
ativ
e
p
r
o
wess
,
B
E
R
T
’
s
d
ee
p
co
n
tex
tu
al
u
n
d
er
s
tan
d
in
g
,
Ge
m
in
i’
s
m
u
ltimo
d
al
in
teg
r
atio
n
,
a
n
d
Dee
p
Seek
’
s
d
o
m
ain
-
s
p
ec
if
ic
a
d
ap
tatio
n
s
a
r
e
cr
itically
co
m
p
ar
e
d
to
h
ig
h
lig
h
t
th
eir
r
elativ
e
ad
v
an
tag
es.
Fu
r
th
er
m
o
r
e
,
th
is
p
ap
er
in
v
e
s
tig
ates
th
e
ch
allen
g
es
ass
o
c
iated
with
d
e
p
lo
y
in
g
L
L
Ms,
in
clu
d
in
g
co
m
p
u
tatio
n
al
co
s
ts
,
eth
ical
c
o
n
s
id
er
atio
n
s
,
b
ias
m
itig
atio
n
,
an
d
i
n
ter
p
r
etab
ilit
y
.
T
h
e
c
o
m
p
u
tatio
n
al
d
e
m
an
d
s
of
tr
ain
in
g
lar
g
e
-
s
ca
le
tr
an
s
f
o
r
m
er
m
o
d
els
n
ec
ess
itate
s
ig
n
i
f
ican
t
h
ar
d
war
e
r
eso
u
r
ce
s
,
r
aisi
n
g
co
n
ce
r
n
s
ab
o
u
t
ac
ce
s
s
ib
ilit
y
an
d
en
v
ir
o
n
m
en
t
al
im
p
ac
t
[
1
1
]
.
E
th
ical
co
n
s
id
e
r
atio
n
s
,
s
u
ch
as
b
ias
in
tr
ain
in
g
d
ata
an
d
p
o
ten
tial
m
is
u
s
e,
r
em
ain
p
iv
o
tal
ch
all
en
g
es
th
at
d
em
an
d
r
o
b
u
s
t
m
itig
atio
n
s
tr
ateg
ies.
Ad
d
itio
n
ally
,
en
h
a
n
cin
g
th
e
in
ter
p
r
etab
ilit
y
of
LLM
o
u
tp
u
ts
is
cr
u
cial
f
o
r
f
o
s
ter
in
g
tr
u
s
t
an
d
r
eliab
ilit
y
in
AI
-
d
r
iv
en
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
es.
T
h
is
co
m
p
ar
ativ
e
r
ev
iew
aim
s
to
p
r
o
v
id
e
a
co
m
p
r
eh
en
s
iv
e
an
aly
s
is
of
GPT
-
4,
B
E
R
T
,
Gem
in
i,
an
d
Dee
p
Seek
,
d
elin
ea
tin
g
th
eir
a
r
ch
itectu
r
al
d
if
f
er
en
ce
s
,
t
r
ain
i
n
g
m
eth
o
d
o
l
o
g
ies,
an
d
r
ea
l
-
wo
r
ld
ap
p
licatio
n
s
.
B
y
ad
d
r
ess
in
g
th
e
ce
n
tr
al
r
esear
ch
q
u
esti
o
n
,
th
is
s
tu
d
y
s
ee
k
s
to
in
f
o
r
m
r
esear
ch
e
r
s
,
d
e
v
elo
p
er
s
,
a
n
d
in
d
u
s
tr
y
p
r
ac
titi
o
n
er
s
ab
o
u
t
th
e
s
tr
en
g
t
h
s
,
lim
itatio
n
s
,
an
d
p
o
ten
tial
e
n
h
an
ce
m
e
n
ts
of
th
ese
L
L
Ms.
T
h
e
in
s
ig
h
ts
g
ain
e
d
f
r
o
m
th
is
s
tu
d
y
not
o
n
ly
co
n
tr
ib
u
te
to
th
e
o
n
g
o
i
n
g
ad
v
an
ce
m
en
t
of
NL
P
tech
n
o
lo
g
ies
b
u
t
also
p
av
e
th
e
way
f
o
r
f
u
tu
r
e
in
n
o
v
atio
n
s
in
lar
g
e
-
s
ca
le
AI
s
y
s
tem
s
.
T
h
r
o
u
g
h
c
o
n
tin
u
o
u
s
r
e
f
in
em
en
t
a
n
d
r
esp
o
n
s
ib
le
d
ep
lo
y
m
e
n
t,
LLMs
can
be
h
ar
n
ess
ed
to
d
r
iv
e
tr
an
s
f
o
r
m
ativ
e
p
r
o
g
r
ess
ac
r
o
s
s
d
iv
er
s
e
s
ec
to
r
s
,
en
h
an
ci
n
g
th
e
ef
f
icac
y
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
7
2
2
-
3
2
2
1
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
,
Vo
l.
7
,
No
.
3
,
No
v
em
b
er
20
26
:
4
0
4
-
418
406
eth
ical
co
n
s
id
er
atio
n
s
of
AI
ap
p
licatio
n
s
in
th
e
m
o
d
er
n
wo
r
ld
.
T
a
b
le
1
p
r
o
v
id
es
a
h
ig
h
-
lev
el
co
m
p
ar
ativ
e
o
v
er
v
iew
of
GPT
-
4,
B
E
R
T
,
Gem
in
i,
an
d
Dee
p
Seek
ac
r
o
s
s
f
iv
e
k
ey
p
ar
am
eter
s
:
ar
c
h
itectu
r
e
ty
p
e,
to
k
en
lim
it,
m
u
ltimo
d
al
s
u
p
p
o
r
t,
c
o
s
t
-
ef
f
i
cien
cy
,
an
d
ac
ce
s
s
ib
ilit
y
.
T
h
i
s
s
u
m
m
ar
y
tab
le
en
a
b
les
r
ap
i
d
id
en
tific
atio
n
of
f
u
n
d
am
e
n
tal
d
if
f
e
r
en
ce
s
in
m
o
d
el
d
esig
n
an
d
d
ep
l
o
y
m
en
t
c
h
ar
ac
ter
is
tics
.
T
ab
le
1
.
C
o
m
p
r
eh
en
s
iv
e
c
o
m
p
ar
is
o
n
b
etwe
en
LLM
m
o
d
els
a
cr
o
s
s
k
ey
p
ar
am
eter
s
M
o
d
e
l
A
r
c
h
i
t
e
c
t
u
r
e
To
k
e
n
l
i
m
i
t
M
u
l
t
i
m
o
d
a
l
C
o
s
t
-
e
f
f
i
c
i
e
n
c
y
A
c
c
e
ss
G
P
T
-
4
D
e
c
o
d
e
r
-
o
n
l
y
1
2
8
K
P
a
r
t
i
a
l
(GPT
-
4o)
H
i
g
h
c
o
s
t
C
l
o
se
d
A
P
I
B
ER
T
B
i
d
i
r
e
c
t
i
o
n
a
l
e
n
c
o
d
e
r
5
1
2
No
H
i
g
h
e
f
f
i
c
i
e
n
c
y
O
p
e
n
-
so
u
r
c
e
G
e
mi
n
i
M
u
l
t
i
m
o
d
a
l
Tr
a
n
sf
o
r
m
e
r
1
M
+
Y
e
s
M
o
d
e
r
a
t
e
C
l
o
se
d
A
P
I
D
e
e
p
S
e
e
k
H
y
b
r
i
d
e
n
c
o
d
e
r
-
d
e
c
o
d
e
r
2
0
0
K
+
No
H
i
g
h
e
f
f
i
c
i
e
n
c
y
O
p
e
n
-
so
u
r
c
e
Mo
d
er
n
LLM
d
e
v
elo
p
m
e
n
t
h
as
co
n
v
er
g
ed
t
o
war
d
f
o
u
r
d
o
m
in
an
t
ar
c
h
itectu
r
al
p
a
r
ad
ig
m
s
r
ef
lectin
g
d
is
tin
ct
o
p
tim
izatio
n
p
r
io
r
itie
s
:
g
en
er
ativ
e
r
ea
s
o
n
in
g
,
c
o
n
te
x
tu
al
u
n
d
e
r
s
tan
d
in
g
,
m
u
ltimo
d
al
in
teg
r
atio
n
,
an
d
d
o
m
ain
-
e
f
f
icien
t
s
p
ec
ializatio
n
.
T
h
e
au
to
r
eg
r
ess
iv
e
p
ar
ad
i
g
m
,
ex
em
p
lifie
d
by
GPT
-
s
er
ies
m
o
d
els,
p
r
io
r
itizes
g
en
er
ativ
e
r
ea
s
o
n
in
g
an
d
c
o
n
v
er
s
atio
n
al
in
telli
g
en
ce
t
h
r
o
u
g
h
d
ec
o
d
er
-
o
n
ly
tr
an
s
f
o
r
m
er
s
[
7
]
,
[
8
]
.
T
h
e
en
co
d
er
p
ar
ad
ig
m
o
r
ig
in
atin
g
f
r
o
m
B
E
R
T
f
o
cu
s
es
on
b
id
i
r
ec
tio
n
al
co
n
tex
tu
al
r
ep
r
esen
tatio
n
a
n
d
ef
f
icien
t
s
em
an
tic
u
n
d
er
s
tan
d
i
n
g
[
9
]
.
Mu
ltimo
d
a
l
ar
ch
itectu
r
es
s
u
ch
as
Gem
in
i
ex
ten
d
tr
an
s
f
o
r
m
e
r
s
to
in
teg
r
ate
h
eter
o
g
en
e
o
u
s
m
o
d
alities
,
in
clu
d
in
g
v
is
io
n
an
d
au
d
io
[
1
0
]
,
[
1
1
]
.
Do
m
ai
n
-
ef
f
icien
t
LLMs
s
u
ch
as
Dee
p
Seek
em
p
h
asize
s
p
ec
ialized
r
ea
s
o
n
in
g
an
d
r
e
d
u
ce
d
in
f
e
r
en
ce
co
s
t
f
o
r
tech
n
ic
al
d
o
m
ain
s
[
1
2
]
,
[
1
3
]
.
2.
M
E
T
H
O
D
T
h
is
co
m
p
ar
ativ
e
r
ev
iew
e
m
p
lo
y
s
a
s
y
s
tem
atic
liter
atu
r
e
r
ev
iew
m
eth
o
d
o
l
o
g
y
to
a
n
aly
ze
an
d
s
y
n
th
esize
th
e
s
tate
-
of
-
th
e
-
a
r
t
in
L
L
Ms.
T
h
e
m
et
h
o
d
o
lo
g
ical
f
r
am
ewo
r
k
was
d
e
s
ig
n
ed
to
en
s
u
r
e
co
m
p
r
eh
e
n
s
iv
e
co
v
er
ag
e
,
o
b
j
ec
tiv
e
ev
alu
atio
n
,
an
d
r
ep
r
o
d
u
cib
le
f
in
d
in
g
s
ac
r
o
s
s
f
o
u
r
p
r
o
m
i
n
en
t
L
L
Ms:
GPT
-
4,
B
E
R
T
,
Gem
in
i,
an
d
D
ee
p
Seek
.
2
.
1
.
Rev
iew
f
ra
m
ew
o
rk
a
nd
re
s
ea
rc
h
qu
estio
n
s
T
h
e
r
ev
iew
was
s
tr
u
ctu
r
ed
ar
o
u
n
d
a
ce
n
tr
al
r
esear
ch
q
u
esti
o
n
:
h
o
w
do
GPT
-
4,
B
E
R
T
,
Gem
in
i,
an
d
Dee
p
Seek
d
if
f
er
in
th
eir
s
tr
u
ctu
r
al
d
esig
n
,
lear
n
in
g
p
a
r
ad
i
g
m
s
,
an
d
ap
p
licatio
n
-
s
p
ec
if
ic
p
er
f
o
r
m
a
n
ce
,
an
d
wh
at
en
h
an
ce
m
e
n
ts
can
be
i
m
p
lem
en
ted
to
a
d
d
r
ess
th
eir
r
esp
ec
tiv
e
lim
itatio
n
s
?
To
ad
d
r
ess
th
is
q
u
esti
o
n
co
m
p
r
eh
e
n
s
iv
ely
,
th
e
f
o
llo
win
g
s
u
b
-
q
u
esti
o
n
s
wer
e
f
o
r
m
u
lat
ed
:
−
Ar
ch
itectu
r
al
co
m
p
a
r
is
o
n
:
w
h
a
t
ar
e
th
e
f
u
n
d
a
m
en
tal
ar
ch
itectu
r
al
d
if
f
er
en
ce
s
am
o
n
g
t
h
e
f
o
u
r
m
o
d
els,
in
clu
d
in
g
t
h
eir
tr
an
s
f
o
r
m
er
c
o
n
f
ig
u
r
atio
n
s
,
atten
tio
n
m
ec
h
an
is
m
s
,
an
d
to
k
en
izatio
n
s
tr
ateg
ies?
−
T
r
ain
in
g
m
eth
o
d
o
lo
g
y
:
how
do
tr
ain
in
g
o
b
jectiv
es,
d
ataset
co
m
p
o
s
itio
n
s
,
a
n
d
f
i
n
e
-
tu
n
in
g
a
p
p
r
o
ac
h
es
d
if
f
er
ac
r
o
s
s
th
e
m
o
d
els?
−
Per
f
o
r
m
an
ce
ev
al
u
atio
n
:
wh
a
t
ar
e
th
e
r
elativ
e
s
tr
en
g
th
s
an
d
lim
itatio
n
s
of
each
m
o
d
el
ac
r
o
s
s
v
ar
io
u
s
p
er
f
o
r
m
an
ce
m
etr
ics
an
d
b
en
c
h
m
ar
k
d
atasets
?
−
Ap
p
licatio
n
s
u
itab
ilit
y
:
h
o
w
do
th
ese
m
o
d
els
p
er
f
o
r
m
in
d
o
m
ain
-
s
p
ec
if
ic
ap
p
licatio
n
s
,
in
clu
d
in
g
s
o
f
twar
e
en
g
in
ee
r
in
g
,
f
in
a
n
ce
,
h
ea
lth
ca
r
e,
an
d
e
d
u
ca
tio
n
?
−
Dep
lo
y
m
en
t
co
n
s
id
er
atio
n
s
:
wh
at
ar
e
th
e
t
r
ad
e
-
o
f
f
s
in
ter
m
s
of
co
m
p
u
tatio
n
al
co
s
t,
late
n
cy
,
ac
ce
s
s
ib
ilit
y
,
an
d
eth
ical
co
n
s
id
er
atio
n
s
?
2
.
2
.
L
a
rg
e
la
ng
ua
g
e
m
o
dels
a
rc
hite
ct
ure
2
.
2
.
1
.
T
he
t
ra
ns
f
o
rm
er
f
r
a
mewo
rk
At
th
e
co
r
e
of
m
o
d
er
n
LLMs
lies
th
e
tr
a
n
s
f
o
r
m
er
ar
c
h
itectu
r
e,
wh
ic
h
h
as
f
u
n
d
am
en
tally
r
ed
ef
i
n
ed
how
m
ac
h
in
es
p
r
o
ce
s
s
s
eq
u
en
tial
d
ata.
I
n
tr
o
d
u
ce
d
by
Vaswan
i
et
a
l
.
[
6
]
,
th
e
tr
an
s
f
o
r
m
er
d
is
p
en
s
es
with
tr
ad
itio
n
al
r
ec
u
r
r
en
t
n
eu
r
al
n
e
two
r
k
s
(
R
NNs)
an
d
i
n
s
tead
e
m
p
lo
y
s
m
u
lti
-
h
ea
d
s
elf
-
atten
tio
n
m
ec
h
an
is
m
s
to
ca
p
tu
r
e
d
e
p
en
d
e
n
cies
ac
r
o
s
s
lo
n
g
s
eq
u
e
n
ce
s
.
Key
elem
en
ts
of
th
e
tr
a
n
s
f
o
r
m
er
f
r
am
ewo
r
k
in
clu
d
e:
−
Self
-
atten
tio
n
m
ec
h
an
is
m
s
:
s
elf
-
atten
tio
n
allo
ws
m
o
d
els
to
d
y
n
am
ically
weig
h
th
e
r
elev
an
ce
of
each
to
k
en
in
th
e
in
p
u
t
s
eq
u
e
n
ce
.
T
h
is
m
ec
h
an
is
m
,
o
p
er
atin
g
o
v
er
m
u
ltip
le
“
h
ea
d
s
,
”
e
n
ab
l
es
th
e
m
o
d
el
to
ca
p
tu
r
e
d
if
f
e
r
en
t
asp
ec
ts
of
s
em
an
tic
an
d
s
y
n
tactic
r
elatio
n
s
h
ip
s
s
im
u
ltan
eo
u
s
ly
.
T
h
e
ab
i
lity
to
co
n
s
id
er
all
p
o
s
itio
n
s
in
th
e
s
eq
u
en
c
e
co
n
cu
r
r
en
tly
r
e
p
r
esen
ts
a
m
ajo
r
d
ep
a
r
tu
r
e
f
r
o
m
s
eq
u
e
n
tial
p
r
o
ce
s
s
in
g
p
ar
ad
ig
m
s
[
6
]
.
−
Po
s
itio
n
al
en
co
d
in
g
s
:
g
iv
en
t
h
at
tr
an
s
f
o
r
m
er
s
do
not
in
h
er
e
n
tly
en
co
d
e
p
o
s
itio
n
al
in
f
o
r
m
atio
n
,
p
o
s
itio
n
al
en
co
d
in
g
s
ar
e
ad
d
e
d
to
t
h
e
in
p
u
t
em
b
e
d
d
in
g
s
[
6
]
.
T
h
ese
en
co
d
in
g
s
p
r
o
v
id
e
th
e
m
o
d
el
wi
th
in
f
o
r
m
atio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
I
SS
N:
2722
-
3
2
2
1
A
co
mp
a
r
a
tive
r
ev
iew
o
f m
o
d
e
r
n
la
r
g
e
la
n
g
u
a
g
e
mo
d
el
p
a
r
a
d
ig
ms:
GP
T
-
4
,
B
E
R
T,
…
(
K
a
v
is
h
S
a
n
g
h
vi
)
407
ab
o
u
t
th
e
r
elativ
e
or
a
b
s
o
lu
te
p
o
s
itio
n
s
of
to
k
en
s
in
th
e
s
eq
u
en
ce
,
en
s
u
r
in
g
th
at
th
e
s
eq
u
en
tial
n
atu
r
e
of
lan
g
u
ag
e
is
n
o
t
lo
s
t.
−
Feed
-
f
o
r
war
d
n
etwo
r
k
s
:
each
lay
er
of
th
e
tr
an
s
f
o
r
m
er
ar
ch
itectu
r
e
co
n
tain
s
a
f
u
lly
co
n
n
ec
ted
f
ee
d
-
f
o
r
war
d
n
eu
r
al
n
etwo
r
k
wo
r
k
in
g
on
th
e
o
u
tp
u
ts
g
en
e
r
ated
by
th
e
atten
tio
n
p
r
o
ce
s
s
[
6
]
.
T
h
ese
n
etwo
r
k
s
h
elp
to
ca
p
t
u
r
e
co
m
p
lex
n
o
n
-
li
n
ea
r
tr
an
s
f
o
r
m
atio
n
s
an
d
im
p
r
o
v
e
th
e
m
o
d
el’
s
r
ep
r
esen
tatio
n
al
p
o
wer
.
−
L
ay
er
n
o
r
m
aliza
tio
n
an
d
r
esid
u
al
co
n
n
ec
tio
n
s
:
to
en
s
u
r
e
th
e
s
tab
ilit
y
of
d
ee
p
a
r
ch
itectu
r
es
d
u
r
in
g
tr
ain
in
g
,
tr
an
s
f
o
r
m
er
s
in
co
r
p
o
r
ate
lay
er
n
o
r
m
aliza
tio
n
[
1
4
]
an
d
r
esid
u
al
co
n
n
ec
tio
n
s
[
1
5
]
.
Su
ch
ar
ch
itectu
r
al
p
r
o
p
er
ties
h
elp
in
m
ain
tain
in
g
th
e
g
r
ad
ien
t
f
lo
w
an
d
m
ak
e
it
ea
s
y
to
tr
ain
v
er
y
d
ee
p
m
o
d
els
with
o
u
t
s
u
f
f
er
in
g
f
r
o
m
v
an
is
h
in
g
an
d
ex
p
lo
d
in
g
g
r
a
d
ien
t
p
r
o
b
lem
s
.
2
.
2
.
2
.
E
m
bedd
ing
t
ec
hn
iq
ues
E
m
b
ed
d
in
g
tech
n
iq
u
es
ar
e
cr
itical
in
tr
an
s
f
o
r
m
in
g
d
is
cr
ete
to
k
en
s
in
to
co
n
tin
u
o
u
s
v
ec
to
r
r
ep
r
esen
tatio
n
s
th
at
ca
p
tu
r
e
s
em
an
tic
m
ea
n
in
g
.
Pre
v
io
u
s
ap
p
r
o
ac
h
es
u
s
ed
s
tatic
e
m
b
ed
d
in
g
s
s
u
ch
as
W
o
r
d
2
Vec
[
1
2
]
an
d
Glo
Ve
[
1
3
]
,
lead
in
g
to
c
o
n
s
tan
t
r
e
p
r
e
s
en
tatio
n
s
ir
r
esp
ec
tiv
e
of
th
e
co
n
tex
t.
In
m
o
d
e
r
n
LLMs
,
h
o
wev
er
,
d
y
n
am
ic
em
b
ed
d
in
g
s
ar
e
u
tili
ze
d
,
allo
win
g
f
o
r
a
d
ap
tatio
n
in
ac
c
o
r
d
a
n
c
e
with
th
e
lex
ical
co
n
tex
t.
T
h
e
ev
o
lu
tio
n
of
em
b
ed
d
in
g
tec
h
n
iq
u
es
in
clu
d
es:
−
Static
em
b
ed
d
in
g
s
:
ea
r
ly
ap
p
r
o
ac
h
es
s
u
ch
as
W
o
r
d
2
Vec
[
1
2
]
an
d
Glo
Ve
[
1
3
]
r
ep
r
esen
t
wo
r
d
s
as
f
ix
ed
v
ec
to
r
s
.
T
h
ese
m
o
d
els
ca
p
tu
r
e
s
em
an
tic
r
elatio
n
s
h
ip
s
but
f
ail
to
ac
co
u
n
t
f
o
r
p
o
ly
s
em
y
an
d
co
n
tex
tu
al
n
u
an
ce
.
−
C
o
n
tex
tu
al
em
b
ed
d
i
n
g
s
:
with
th
e
ad
v
en
t
of
tr
an
s
f
o
r
m
er
s
[
6
]
,
em
b
e
d
d
in
g
s
ar
e
g
en
er
ate
d
in
a
co
n
tex
t
-
d
ep
en
d
e
n
t
m
an
n
er
.
Mo
d
els
lik
e
B
E
R
T
[
8
]
an
d
GPT
-
4
[
1
]
u
p
d
ate
th
e
r
ep
r
esen
tatio
n
of
e
ac
h
to
k
en
b
ased
on
its
co
n
tex
t,
th
e
r
eb
y
ca
p
tu
r
i
n
g
s
u
b
tle
s
h
if
ts
in
m
ea
n
i
n
g
.
−
S
u
b
w
o
r
d
t
o
k
e
n
i
z
a
t
i
o
n
:
m
e
t
h
o
d
s
l
i
k
e
b
y
t
e
p
a
i
r
e
n
c
o
d
i
n
g
(
B
P
E
)
[
1
6
]
a
n
d
S
e
n
t
e
n
c
e
P
i
ec
e
[
1
7
]
a
l
l
o
w
m
o
d
e
l
s
to
b
r
e
a
k
d
o
w
n
u
n
c
o
m
m
o
n
or
c
o
m
p
l
e
x
w
o
r
d
s
i
n
t
o
s
u
b
w
o
r
d
s
,
m
a
k
i
n
g
t
h
e
m
e
as
i
e
r
to
m
a
n
a
g
e
.
T
h
i
s
s
t
r
a
te
g
y
d
o
e
s
not
o
n
l
y
r
e
d
u
c
e
t
h
e
v
o
c
a
b
u
l
a
r
y
s
i
z
e
but
m
a
k
es
t
h
e
m
o
d
e
l
b
et
t
er
at
d
e
a
l
i
n
g
wi
t
h
out
-
of
-
v
o
c
a
b
u
l
a
r
y
w
o
r
d
s
.
2
.
2
.
3
.
T
ra
ini
ng
da
t
a
s
et
s
a
nd
pre
-
t
ra
ini
ng
o
bje
ct
iv
es
T
h
e
ef
f
ec
tiv
en
ess
of
an
L
L
M
is
lar
g
ely
d
eter
m
in
ed
by
th
e
q
u
ality
an
d
d
iv
e
r
s
ity
of
its
tr
ain
in
g
d
ata
,
as
well
as
th
e
o
b
jectiv
es
u
s
ed
d
u
r
in
g
p
r
e
-
tr
ain
in
g
.
C
o
m
m
o
n
s
tr
ateg
ies
in
clu
d
e:
i)
t
r
ain
in
g
d
atasets
:
m
o
d
er
n
LLMs
ar
e
tr
ain
e
d
on
v
ast
d
ata
s
ets
co
m
p
r
is
in
g
web
p
ag
es,
b
o
o
k
s
,
ac
a
d
em
ic
ar
ticles,
an
d
s
p
ec
ialized
tex
ts
[
7
]
.
Su
ch
d
iv
er
s
e
d
atasets
en
s
u
r
e
b
r
o
ad
lan
g
u
ag
e
co
v
er
ag
e
a
n
d
th
e
ab
ilit
y
to
g
en
er
alize
ac
r
o
s
s
d
o
m
ain
s
;
an
d
ii)
p
re
-
tr
ain
in
g
o
b
jectiv
es
:
−
Au
to
r
eg
r
ess
iv
e
m
o
d
elin
g
:
m
o
d
els
lik
e
GPT
-
4
u
s
e
an
au
t
o
r
e
g
r
ess
iv
e
o
b
jectiv
e,
lear
n
i
n
g
to
p
r
ed
ict
th
e
n
ex
t
to
k
en
in
a
s
eq
u
en
ce
.
It
d
ev
e
lo
p
s
p
o
wer
f
u
l
g
en
e
r
ativ
e
ab
il
ities
in
th
e
m
o
d
el
to
co
n
s
tr
u
ct
co
n
tex
tu
ally
r
elev
an
t
an
d
co
n
s
is
ten
t
tex
t
[
7
]
.
−
Ma
s
k
ed
lan
g
u
a
g
e
m
o
d
elin
g
(
ML
M)
:
B
E
R
T
u
tili
ze
s
MLM,
wh
er
e
th
e
m
o
d
el
p
r
e
d
icts
m
is
s
in
g
to
k
en
s
th
at
ar
e
m
ask
ed
in
a
s
eq
u
en
ce
g
iv
en
th
e
s
u
r
r
o
u
n
d
in
g
c
o
n
tex
t.
T
h
is
m
eth
o
d
r
elies
on
a
b
id
ir
ec
tio
n
al
ap
p
r
o
ac
h
,
im
p
r
o
v
in
g
th
e
m
o
d
el
’
s
co
m
p
r
eh
en
s
io
n
of
th
e
s
tr
u
ctu
r
e
of
la
n
g
u
ag
e
an
d
c
o
n
tex
tu
al
m
ea
n
in
g
[
8
]
.
−
C
r
o
s
s
-
m
o
d
al
o
b
jectiv
es:
in
m
u
ltimo
d
al
m
o
d
els
lik
e
Gem
in
i,
tr
ain
in
g
is
ex
ten
d
ed
to
in
clu
d
e
s
tim
u
li
b
ey
o
n
d
tex
t,
f
o
r
e
x
am
p
le,
im
ag
es
an
d
s
o
u
n
d
s
.
C
r
o
s
s
-
m
o
d
al
o
b
jectiv
es
m
a
k
e
ce
r
tain
th
at
th
e
m
o
d
el
is
tau
g
h
t
to
ass
o
ciate
an
d
i
n
co
r
p
o
r
ate
d
if
f
e
r
en
t
d
ata
m
o
d
alities
[
2
]
,
[
1
8
]
.
2
.
3
.
LLMs
ca
pa
bil
it
y
co
m
pa
riso
n
T
h
is
s
ec
tio
n
co
m
p
ar
es
r
ep
r
esen
tativ
e
L
L
Ms,
in
clu
d
in
g
GPT
-
4
,
Gem
in
i,
B
E
R
T
,
an
d
De
ep
Seek
,
in
ter
m
s
o
f
p
er
f
o
r
m
an
ce
an
d
ca
p
ab
ilit
ies,
co
s
t
an
d
laten
cy
,
ac
ce
s
s
ib
ilit
y
an
d
o
p
en
n
ess
,
an
d
m
u
ltimo
d
ality
an
d
u
s
e
-
ca
s
e
alig
n
m
en
t
.
−
Per
f
o
r
m
an
ce
an
d
ca
p
ab
ilit
ie
s
:
GPT
-
4
d
em
o
n
s
tr
ates
s
u
p
er
io
r
g
en
er
ativ
e
r
ea
s
o
n
in
g
an
d
lo
n
g
-
c
o
n
tex
t
h
an
d
lin
g
[
1
]
,
[
4
]
,
wh
ile
B
E
R
T
is
more
ef
f
ec
tiv
e
in
class
if
icatio
n
an
d
s
em
an
tic
s
im
ilar
ity
task
s
[
8
]
,
[
1
9
]
,
[
2
0
]
.
Gem
i
n
i
ex
ce
l
s
in
m
u
ltimo
d
al
in
f
er
en
ce
[
2
]
,
[
2
1
]
,
a
n
d
Dee
p
Seek
p
er
f
o
r
m
s
s
tr
o
n
g
ly
in
d
o
m
ain
-
s
p
ec
if
ic
b
en
ch
m
a
r
k
s
s
u
ch
as
f
in
an
cial
a
n
d
co
d
e
g
e
n
e
r
atio
n
d
atasets
[
3
]
,
[
2
2
]
.
−
C
o
s
t
an
d
laten
cy
:
co
s
t
an
d
in
f
er
en
ce
s
p
ee
d
p
lay
a
cr
itical
r
o
le
in
d
ep
lo
y
m
en
t.
GPT
-
4
v
ar
ian
ts
(
e.
g
.
,
GPT
-
4
.
1
)
o
f
f
e
r
h
i
g
h
p
er
f
o
r
m
an
ce
but
co
m
e
with
h
ig
h
er
p
er
-
to
k
en
ap
p
licatio
n
p
r
o
g
r
am
m
in
g
in
ter
f
ac
e
(
API
)
c
o
s
ts
(
~$
3
0
-
60
p
er
m
illi
o
n
to
k
e
n
s
)
an
d
m
o
d
er
ate
laten
cy
(
~0
.
9
s
in
itial
r
esp
o
n
s
e
tim
e)
[
1
]
,
[
4
]
.
Gem
i
n
i’
s
to
p
-
tier
r
ea
s
o
n
in
g
an
d
lo
n
g
-
co
n
tex
t
c
ap
ab
ilit
ies
in
cu
r
s
h
ig
h
er
co
m
p
u
te
co
s
t
an
d
s
tr
u
g
g
les
with
lo
w
-
laten
cy
e
d
g
e
d
e
p
lo
y
m
e
n
t;
its
h
ea
v
y
m
u
ltimo
d
al
ar
c
h
itectu
r
e
m
a
k
es
it
r
eso
u
r
ce
-
in
ten
s
iv
e
[
2
]
,
[
2
1
]
.
Dee
p
See
k
,
b
ein
g
o
p
tim
ized
f
o
r
d
o
m
a
in
-
s
p
ec
if
ic
r
ea
s
o
n
in
g
,
o
f
f
er
s
a
co
s
t
-
ef
f
ec
tiv
e
alter
n
ativ
e
[
3
]
,
[
2
2
]
.
−
Acc
ess
ib
ilit
y
an
d
o
p
en
n
ess
:
wh
ile
GPT
-
4
an
d
Gem
i
n
i
ar
e
cu
r
r
en
tly
clo
s
ed
-
s
o
u
r
ce
with
API
-
o
n
ly
ac
ce
s
s
[
1
]
,
[
2
]
,
[
2
1
]
,
B
E
R
T
an
d
Dee
p
Seek
p
r
o
v
id
e
o
p
en
-
s
o
u
r
ce
v
ar
ian
ts
th
at
en
co
u
r
a
g
e
r
esear
ch
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
7
2
2
-
3
2
2
1
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
,
Vo
l.
7
,
No
.
3
,
No
v
em
b
er
20
26
:
4
0
4
-
418
408
cu
s
to
m
izatio
n
[
8
]
,
[
3
]
,
[
2
2
]
.
Dee
p
Seek
’
s
API
ac
ce
s
s
is
n
o
tab
ly
ch
ea
p
er
an
d
f
aster
f
o
r
in
f
er
en
ce
in
s
p
ec
if
ic
d
o
m
ain
s
lik
e
f
in
a
n
ce
or
s
o
f
twar
e
en
g
in
ee
r
in
g
.
−
Mu
ltimo
d
ality
an
d
u
s
e
-
ca
s
e
alig
n
m
en
t:
Gem
in
i’
s
ab
ilit
y
to
p
r
o
ce
s
s
an
d
r
ea
s
o
n
o
v
er
te
x
t,
im
ag
es,
an
d
au
d
io
p
r
o
v
id
es
s
ig
n
if
ica
n
t
lev
er
ag
e
f
o
r
ed
u
ca
tio
n
al,
ass
is
tiv
e,
an
d
cr
ea
tiv
e
to
o
ls
[
2
]
,
[
1
8
]
,
[
2
1
]
.
GPT
-
4,
th
r
o
u
g
h
p
lu
g
i
n
s
an
d
im
ag
e
in
p
u
t
(
in
GPT
-
4
o
)
,
ex
p
an
d
s
th
e
tex
t
-
ce
n
tr
ic
p
ar
ad
ig
m
in
to
lim
ited
m
u
ltimo
d
al
ter
r
ito
r
y
[
1
]
,
[
5
]
.
Dee
p
Seek
is
tu
n
ed
p
r
i
m
ar
ily
f
o
r
tech
n
ical
d
o
m
ain
s
lik
e
m
ath
em
atics
an
d
p
r
o
g
r
am
m
in
g
,
but
lack
s
b
r
o
ad
er
m
u
ltimo
d
al
i
n
p
u
t
s
u
p
p
o
r
t
[
3
]
,
[
2
2
]
.
2
.
4
.
G
P
T
-
4
GPT
-
4
r
ep
r
esen
ts
th
e
lar
g
e
-
s
c
ale
au
to
r
eg
r
ess
iv
e
g
en
er
ativ
e
p
ar
ad
ig
m
in
m
o
d
e
r
n
LLM
r
es
ea
r
ch
.
B
u
ilt
on
a
d
ec
o
d
er
-
o
n
ly
tr
a
n
s
f
o
r
m
e
r
ar
ch
itectu
r
e,
GPT
-
4
d
em
o
n
s
tr
ates
s
tr
o
n
g
r
ea
s
o
n
in
g
,
l
o
n
g
-
co
n
tex
t
co
h
e
r
en
ce
,
an
d
co
n
v
er
s
atio
n
al
in
tellig
en
c
e
ac
r
o
s
s
d
iv
er
s
e
task
s
[
1
]
.
T
h
e
m
o
d
el’
s
d
esig
n
f
o
c
u
s
es
on
o
p
tim
izin
g
in
f
er
en
ce
s
p
ee
d
an
d
r
e
d
u
cin
g
is
s
u
es
s
u
c
h
as
h
allu
cin
atio
n
s
,
a
p
h
en
o
m
en
o
n
wh
er
e
th
e
m
o
d
el
g
e
n
er
at
es
tex
t
th
at
ap
p
ea
r
s
f
ac
tu
al
but
is
ac
tu
ally
in
co
r
r
e
ct
or
f
ab
r
icate
d
,
wh
ile
p
r
eser
v
in
g
its
cr
ea
tiv
e
lan
g
u
ag
e
g
e
n
er
atio
n
ca
p
a
b
ilit
ies
[
7
]
,
[
2
3
]
,
[
2
4
]
.
I
ts
m
u
ltimo
d
al
ex
ten
s
io
n
GPT
-
4
in
teg
r
a
tes
v
is
u
al
an
d
au
d
i
o
in
p
u
ts
wh
ile
m
ain
tain
in
g
g
en
er
ativ
e
p
e
r
f
o
r
m
an
ce
.
Alth
o
u
g
h
n
ewe
r
m
o
d
els
h
av
e
em
er
g
ed
,
GPT
-
4
r
em
ain
s
a
ca
n
o
n
ical
b
aselin
e
f
o
r
lar
g
e
-
s
ca
le
g
en
e
r
ativ
e
LLM
ev
alu
atio
n
an
d
d
e
p
lo
y
m
e
n
t
s
tu
d
ies.
GPT
-
4
u
s
es
d
y
n
a
m
ic,
co
n
te
x
t
-
awa
r
e
em
b
ed
d
in
g
s
to
ca
p
t
u
r
e
th
e
n
u
an
ce
s
of
lan
g
u
ag
e.
I
ts
s
u
b
wo
r
d
to
k
en
izatio
n
m
eth
o
d
allo
ws
th
e
m
o
d
el
to
h
a
n
d
le
r
ar
e
or
co
m
p
o
u
n
d
wo
r
d
s
ef
f
i
cien
tly
,
en
s
u
r
in
g
t
h
at
ev
en
in
f
r
eq
u
e
n
t
ter
m
s
ar
e
r
ep
r
esen
te
d
ad
eq
u
ately
in
th
e
g
en
er
ated
tex
t
[
9
]
.
Pre
-
tr
ain
in
g
f
o
r
GPT
-
4
i
n
v
o
lv
es
an
ex
te
n
s
iv
e,
h
ete
r
o
g
e
n
eo
u
s
d
ataset
th
at
i
n
clu
d
es
w
eb
p
a
g
es,
liter
atu
r
e,
s
cien
tific
tex
ts
,
an
d
o
th
er
p
u
b
licly
av
ailab
le
co
r
p
o
r
a.
T
h
e
a
u
to
r
eg
r
ess
iv
e
o
b
jectiv
e
p
r
ed
ictin
g
th
e
n
ex
t
to
k
en
b
ased
on
p
r
e
v
io
u
s
co
n
tex
t
f
o
r
m
s
th
e
b
ac
k
b
o
n
e
of
its
tr
ain
in
g
,
e
n
ab
lin
g
th
e
m
o
d
el
to
ex
ce
l
in
tex
t
g
en
er
atio
n
task
s
.
Ov
er
s
u
cc
e
s
s
iv
e
v
er
s
io
n
s
,
im
p
r
o
v
e
m
en
ts
h
av
e
b
ee
n
m
a
d
e
in
s
ca
lin
g
t
h
e
m
o
d
el
s
ize
an
d
r
ef
in
in
g
tr
ai
n
in
g
tech
n
iq
u
es
to
en
h
an
ce
b
o
th
r
ea
s
o
n
in
g
an
d
f
ac
tu
al
ac
cu
r
ac
y
[
1
]
,
[
2
5
]
.
Fro
m
GPT
-
1
to
GPT
-
4,
each
iter
atio
n
h
as
in
tr
o
d
u
ce
d
r
e
f
in
e
m
en
ts
in
m
o
d
el
ar
c
h
itectu
r
e,
p
ar
am
eter
s
ca
lin
g
,
an
d
tr
ain
in
g
p
r
o
to
co
l
s
.
T
h
ese
im
p
r
o
v
em
en
ts
h
av
e
r
esu
lted
in
en
h
a
n
ce
d
p
er
f
o
r
m
an
ce
on
co
m
p
lex
r
ea
s
o
n
in
g
task
s
,
lo
n
g
er
co
n
te
x
t
r
eten
tio
n
,
an
d
a
r
ed
u
cti
o
n
in
co
m
m
o
n
p
itfa
lls
s
u
ch
as
f
ac
tu
al
h
allu
cin
atio
n
.
R
ec
en
t
s
tu
d
ies
h
av
e
also
h
ig
h
l
ig
h
ted
th
e
im
p
o
r
ta
n
ce
of
f
in
e
-
tu
n
in
g
on
d
o
m
ain
-
s
p
ec
if
ic
d
ata
to
m
itig
ate
b
iases
an
d
im
p
r
o
v
e
r
eliab
ilit
y
in
s
p
ec
ialized
ap
p
licatio
n
s
[
7
]
,
[
2
6
]
.
2
.
5
.
B
E
RT
B
E
R
T
’
s
ar
ch
itectu
r
e
is
d
ef
in
ed
by
its
b
id
ir
ec
tio
n
al
e
n
co
d
e
r
s
in
ce
it
p
r
o
ce
s
s
es
th
e
en
tire
s
e
q
u
en
ce
at
o
n
ce
in
s
tead
of
s
eq
u
en
tially
.
T
h
is
m
ak
es
it
p
o
s
s
ib
le
f
o
r
B
E
R
T
to
u
n
d
er
s
tan
d
th
e
co
n
tex
tu
al
n
u
an
ce
s
of
th
e
r
elatio
n
s
h
ip
b
etwe
en
p
r
ec
e
d
in
g
an
d
s
u
b
s
eq
u
e
n
t
to
k
en
s
,
wh
ich
m
a
k
es
B
E
R
T
an
ef
f
ec
tiv
e
to
o
l
in
u
n
d
er
s
tan
d
i
n
g
lan
g
u
ag
e.
B
E
R
T
h
as
p
r
o
v
en
its
u
s
ef
u
ln
ess
in
NL
P
in
task
s
lik
e
s
en
tim
en
t
an
aly
s
is
an
d
q
u
esti
o
n
an
s
wer
in
g
[
8
]
,
[
2
7
]
,
[
2
8
]
.
B
E
R
T
em
p
lo
y
s
W
o
r
d
Piece
to
k
en
izatio
n
to
b
r
ea
k
tex
t
i
n
to
s
u
b
wo
r
d
u
n
its
.
T
h
is
ap
p
r
o
ac
h
e
n
s
u
r
es
th
at
ev
en
r
ar
e
wo
r
d
s
or
out
-
of
-
v
o
c
ab
u
lar
y
ter
m
s
ar
e
r
ep
r
esen
ted
ef
f
ec
tiv
ely
th
r
o
u
g
h
a
co
m
b
i
n
atio
n
of
s
u
b
wo
r
d
s
.
C
o
n
tex
tu
al
em
b
e
d
d
in
g
s
a
r
e
g
e
n
er
ated
d
y
n
am
ically
,
allo
win
g
B
E
R
T
to
ca
p
tu
r
e
th
e
s
em
an
ti
c
n
u
an
ce
s
of
w
o
r
d
s
in
v
ar
ied
c
o
n
tex
ts
[
2
9
]
.
T
h
e
tr
ain
in
g
task
s
f
o
r
B
E
R
T
in
clu
d
e
MLM
an
d
n
e
x
t
s
en
ten
ce
p
r
ed
ictio
n
(
NSP).
MLM
in
v
o
lv
es
m
ask
in
g
a
ce
r
tain
n
u
m
b
er
of
t
o
k
en
s
in
a
s
en
ten
ce
an
d
th
en
p
r
ed
ictin
g
t
h
em
f
r
o
m
th
e
co
n
t
ex
t
s
u
r
r
o
u
n
d
in
g
th
e
s
en
ten
ce
.
T
h
e
tr
ain
in
g
c
o
r
p
u
s
in
clu
d
es
d
iv
er
s
e
s
o
u
r
ce
s
s
u
ch
as
b
o
o
k
s
,
W
ik
ip
ed
ia,
an
d
web
tex
ts
,
en
s
u
r
in
g
th
at
B
E
R
T
is
well
-
g
r
o
u
n
d
e
d
in
g
en
er
al
lan
g
u
ag
e
u
s
e
[
8
]
,
[
3
0
]
.
Sin
ce
its
r
elea
s
e,
B
E
R
T
h
as
s
ee
n
n
u
m
er
o
u
s
ad
ap
tatio
n
s
an
d
d
e
r
iv
ativ
es.
Var
ian
ts
s
u
ch
as
R
o
B
E
R
T
a,
Dis
tilB
E
R
T
,
an
d
d
o
m
a
in
-
s
p
ec
if
ic
m
o
d
els
(
e.
g
.
,
B
io
B
E
R
T
)
h
av
e
em
er
g
ed
to
o
p
tim
ize
p
er
f
o
r
m
a
n
ce
f
o
r
d
if
f
e
r
en
t
task
s
,
r
an
g
in
g
f
r
o
m
co
m
p
u
tatio
n
al
ef
f
icien
cy
to
s
p
ec
ialized
a
p
p
li
ca
tio
n
s
in
b
io
m
ed
ical
te
x
t
an
al
y
s
is
[
2
7
]
.
2
.
6
.
G
em
ini
Gem
in
i
r
ep
r
esen
ts
an
in
n
o
v
at
iv
e
leap
by
in
teg
r
atin
g
m
u
ltimo
d
al
ca
p
ab
ilit
ies
in
to
th
e
tr
an
s
f
o
r
m
er
f
r
am
ewo
r
k
.
Un
lik
e
tr
ad
itio
n
a
l
LLMs
th
at
o
p
er
ate
s
o
lely
on
tex
tu
al
d
ata,
Gem
in
i
in
co
r
p
o
r
ates
s
p
ec
ialized
m
o
d
u
les
f
o
r
p
r
o
ce
s
s
in
g
v
is
u
al
an
d
au
d
ito
r
y
i
n
p
u
ts
alo
n
g
s
id
e
tex
t.
T
h
is
cr
o
s
s
-
m
o
d
al
a
p
p
r
o
ac
h
lev
er
a
g
es
b
o
t
h
co
n
v
o
l
u
tio
n
al
tech
n
iq
u
es
(
as
s
ee
n
in
v
is
io
n
tr
a
n
s
f
o
r
m
e
r
s
)
an
d
s
p
ec
tr
o
g
r
am
-
b
ased
m
eth
o
d
s
f
o
r
a
u
d
io
,
allo
win
g
Gem
in
i
to
h
an
d
le
a
d
i
v
er
s
e
r
an
g
e
of
d
ata
ty
p
es
co
n
c
u
r
r
e
n
tly
[
2
]
,
[
1
8
]
,
[
3
1
]
.
Gem
in
i
1
.
5
f
u
r
th
e
r
ex
te
n
d
s
co
n
tex
t
len
g
th
to
m
illi
o
n
-
to
k
en
s
ca
les,
en
ab
lin
g
lo
n
g
-
d
o
c
u
m
en
t
m
u
ltimo
d
al
r
ea
s
o
n
in
g
an
d
lar
g
e
-
c
o
n
tex
t
p
r
o
ce
s
s
in
g
.
Fo
r
tex
t,
Gem
in
i
em
p
lo
y
s
s
im
ilar
co
n
tex
tu
al
em
b
ed
d
in
g
tec
h
n
iq
u
es
as
f
o
u
n
d
in
B
E
R
T
an
d
GPT
-
4.
Fo
r
im
ag
es
an
d
au
d
io
,
m
o
d
al
ity
-
s
p
ec
if
ic
em
b
ed
d
in
g
s
ar
e
g
en
er
ated
u
s
in
g
n
etwo
r
k
s
th
at
ar
e
o
p
tim
ized
f
o
r
Evaluation Warning : The document was created with Spire.PDF for Python.
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
I
SS
N:
2722
-
3
2
2
1
A
co
mp
a
r
a
tive
r
ev
iew
o
f m
o
d
e
r
n
la
r
g
e
la
n
g
u
a
g
e
mo
d
el
p
a
r
a
d
ig
ms:
GP
T
-
4
,
B
E
R
T,
…
(
K
a
v
is
h
S
a
n
g
h
vi
)
409
each
d
ata
ty
p
e.
T
h
ese
em
b
ed
d
in
g
s
ar
e
th
en
alig
n
e
d
th
r
o
u
g
h
cr
o
s
s
-
atten
tio
n
m
ec
h
an
is
m
s
,
e
n
ab
lin
g
in
teg
r
ated
r
ea
s
o
n
in
g
o
v
er
h
eter
o
g
e
n
eo
u
s
in
p
u
ts
[
3
2
]
.
T
h
e
tr
ain
in
g
r
eg
im
e
f
o
r
Gem
i
n
i
in
v
o
lv
es
c
u
r
ated
d
atasets
th
at
s
p
an
m
u
ltip
le
m
o
d
alities
.
An
n
o
tated
im
ag
e
co
llectio
n
s
an
d
au
d
io
f
iles
co
m
p
lem
en
t
tex
tu
al
d
ata,
e
n
s
u
r
in
g
th
at
th
e
m
o
d
el
lea
r
n
s
r
o
b
u
s
t
c
r
o
s
s
-
m
o
d
al
r
ep
r
esen
tatio
n
s
.
Pre
-
tr
ain
in
g
o
b
jectiv
es
ar
e
ex
ten
d
ed
to
in
cl
u
d
e
task
s
th
at
r
eq
u
ir
e
th
e
m
o
d
el
to
co
r
r
elate
v
is
u
al
or
a
u
d
ito
r
y
cu
es
with
tex
t,
s
u
c
h
as
g
e
n
er
atin
g
d
escr
ip
tiv
e
ca
p
tio
n
s
f
o
r
im
a
g
es
or
p
r
o
v
id
i
n
g
s
en
tim
en
t
an
al
y
s
is
f
r
o
m
au
d
i
o
v
is
u
al
in
p
u
ts
[
9
]
.
Gem
in
i
is
s
til
l
ev
o
lv
in
g
,
with
iter
ativ
e
en
h
an
ce
m
en
ts
aim
ed
at
r
ed
u
cin
g
m
o
d
ality
-
s
p
ec
if
ic
b
iases
an
d
im
p
r
o
v
in
g
cr
o
s
s
-
m
o
d
al
c
o
h
er
en
ce
.
Fu
tu
r
e
iter
atio
n
s
ar
e
e
x
p
ec
ted
to
f
u
r
th
er
o
p
tim
ize
th
e
f
u
s
io
n
of
d
if
f
e
r
e
n
t
d
ata
t
y
p
es,
p
o
ten
tially
in
te
g
r
atin
g
a
d
d
itio
n
al
m
o
d
alities
(
e.
g
.
,
s
en
s
o
r
d
ata)
to
wid
en
its
ap
p
licab
ilit
y
[
3
3
]
.
2
.
7
.
Dee
pS
ee
k
Dee
p
Seek
ex
em
p
lifie
s
th
e
em
er
g
in
g
d
o
m
ain
-
e
f
f
icien
t
L
L
M
p
ar
ad
ig
m
,
em
p
h
asizin
g
s
p
ec
ialized
r
ea
s
o
n
in
g
an
d
r
ed
u
ce
d
co
m
p
u
tatio
n
al
co
s
t
f
o
r
tech
n
ical
d
o
m
ain
s
s
u
ch
as
m
ath
em
atics
an
d
p
r
o
g
r
a
m
m
in
g
[
2
2
]
.
Dee
p
Seek
-
R1
in
tr
o
d
u
ce
s
r
ein
f
o
r
ce
m
en
t
-
lear
n
in
g
-
b
ased
r
ea
s
o
n
in
g
o
p
tim
izatio
n
,
ac
h
ie
v
in
g
s
tr
o
n
g
p
er
f
o
r
m
a
n
ce
in
co
d
e
an
d
m
ath
em
atica
l
b
en
ch
m
ar
k
s
w
h
ile
m
ain
tain
in
g
ef
f
icien
c
y
ad
v
an
tag
es.
W
h
ile
it
is
b
u
ilt
on
a
tr
an
s
f
o
r
m
er
-
b
ased
ar
c
h
itectu
r
e,
its
d
esig
n
in
clu
d
es
m
o
d
if
icatio
n
s
tailo
r
ed
to
s
p
ec
ialized
task
s
s
u
ch
as
co
d
e
an
aly
s
is
an
d
f
in
an
cial
m
o
d
elin
g
.
T
h
ese
ad
ap
tatio
n
s
f
o
cu
s
on
r
ed
u
cin
g
c
o
m
p
u
tatio
n
al
o
v
er
h
ea
d
an
d
en
h
a
n
cin
g
p
er
f
o
r
m
an
ce
in
n
ar
r
o
wly
d
ef
i
n
ed
ap
p
licatio
n
a
r
ea
s
[
2
3
]
–
[
2
5
]
.
Dee
p
Seek
em
p
lo
y
s
a
h
y
b
r
i
d
em
b
ed
d
in
g
s
tr
ateg
y
th
at
co
m
b
in
es
g
en
er
al
-
p
u
r
p
o
s
e
co
n
tex
tu
al
em
b
ed
d
in
g
s
with
d
o
m
ain
-
s
p
e
cif
ic
to
k
en
izatio
n
s
ch
em
es.
F
o
r
ex
a
m
p
le,
in
co
d
e
an
al
y
s
is
,
its
to
k
en
izatio
n
m
a
y
in
co
r
p
o
r
ate
p
r
o
g
r
am
m
in
g
lan
g
u
ag
e
s
y
n
tax
to
p
r
eser
v
e
s
tr
u
ctu
r
al
in
f
o
r
m
atio
n
,
wh
ile
in
f
i
n
an
cial
ap
p
licatio
n
s
,
s
p
ec
ialized
v
o
ca
b
u
lar
y
is
h
a
n
d
led
with
cu
s
to
m
ized
s
u
b
wo
r
d
u
n
its
[
3
4
]
.
T
h
e
m
o
d
el
is
p
r
e
-
tr
ain
ed
on
ca
r
ef
u
lly
cu
r
ated
d
atasets
th
at
ar
e
r
ef
le
ctiv
e
of
its
tar
g
et
d
o
m
ain
s
.
Fo
r
co
d
e
-
r
elate
d
task
s
,
r
ep
o
s
ito
r
ies
of
o
p
e
n
-
s
o
u
r
c
e
co
d
e
a
n
d
tech
n
ical
d
o
cu
m
en
t
atio
n
ar
e
u
s
ed
.
In
f
in
a
n
cial
m
o
d
elin
g
,
s
tr
u
ctu
r
e
d
f
in
an
cial
r
ep
o
r
ts
an
d
m
ar
k
et
d
ata
f
o
r
m
th
e
co
r
e
tr
ain
in
g
co
r
p
u
s
.
T
h
is
f
o
cu
s
ed
ap
p
r
o
ac
h
a
llo
ws
Dee
p
Seek
to
ex
ce
l
in
s
p
ec
if
ic
ar
ea
s
wh
ile
m
ain
tain
in
g
a
g
en
er
al
lev
el
of
lan
g
u
a
g
e
u
n
d
er
s
tan
d
in
g
[
3
5
]
,
[
3
6
]
.
Alth
o
u
g
h
less
p
u
b
lici
ze
d
,
Dee
p
Seek
h
as
u
n
d
er
g
o
n
e
iter
ativ
e
r
e
f
in
em
en
ts
th
at
p
r
io
r
itize
ef
f
icien
cy
an
d
d
o
m
ain
ac
c
u
r
ac
y
.
L
ater
v
er
s
io
n
s
in
teg
r
ate
more
r
ef
in
ed
to
k
e
n
izatio
n
s
tr
ateg
ies
an
d
lev
er
ag
e
co
n
tin
u
o
u
s
f
e
ed
b
ac
k
f
r
o
m
r
ea
l
-
wo
r
l
d
ap
p
li
ca
tio
n
s
to
im
p
r
o
v
e
task
-
s
p
ec
if
ic
p
er
f
o
r
m
an
ce
[
3
7
]
.
3.
RE
SU
L
T
S
AND
D
I
SCU
SS
I
O
N
3
.
1
.
Co
m
pa
ra
t
iv
e
dis
cus
s
io
n
of
LLMs
m
o
dels
T
h
e
a
r
c
h
it
ec
t
u
r
e
of
L
L
Ms
is
t
h
e
k
e
y
to
t
h
ei
r
e
f
f
ic
ie
n
c
y
a
n
d
u
ti
lit
y
in
v
ar
io
u
s
a
r
ea
s
.
T
h
is
s
ec
ti
o
n
is
a
co
m
p
r
eh
e
n
s
i
v
e
an
al
y
s
is
of
t
h
e
ar
ch
ite
ct
u
r
es
of
GPT
-
4,
B
E
R
T
,
Ge
m
i
n
i
,
a
n
d
D
ee
p
Se
e
k
,
em
p
l
o
y
i
n
g
a
s
y
s
te
m
a
tic
ap
p
r
o
ac
h
to
r
e
v
iew
r
el
ev
a
n
t
r
e
s
ea
r
c
h
,
ca
te
g
o
r
ize
f
i
n
d
in
g
s
,
a
n
d
h
i
g
h
li
g
h
t
k
e
y
s
t
r
e
n
g
th
s
a
n
d
l
im
it
ati
o
n
s
.
T
ab
le
2
p
r
ese
n
ts
a
d
et
ail
ed
c
o
m
p
a
r
a
ti
v
e
an
al
y
s
is
of
m
o
d
el
f
r
a
m
e
wo
r
k
s
,
i
n
c
lu
d
i
n
g
a
r
ch
ite
ct
u
r
e
t
y
p
e
,
t
r
a
in
in
g
d
a
ta
co
m
p
o
s
iti
o
n
,
c
o
s
t/
lat
en
c
y
ch
ar
ac
t
er
is
tics
,
u
s
e
-
ca
s
e
f
it
,
m
u
lt
i
m
o
d
al
s
u
p
p
o
r
t
,
an
d
AP
I
/a
cc
es
s
m
o
d
els
.
T
h
is
ta
b
l
e
f
a
cili
tat
es
s
y
s
t
em
ati
c
co
m
p
a
r
is
o
n
of
a
r
ch
ite
ct
u
r
al
a
n
d
d
e
p
l
o
y
m
e
n
t
f
ea
t
u
r
es
a
cr
o
s
s
t
h
e
f
o
u
r
m
o
d
els.
T
ab
le
2
.
C
o
m
p
a
r
ativ
e
an
aly
s
is
of
LLM
m
o
d
els
f
r
am
ewo
r
k
s
F
e
a
t
u
r
e
G
P
T
-
4
B
ER
T
G
e
mi
n
i
D
e
e
p
S
e
e
k
A
r
c
h
i
t
e
c
t
u
r
e
D
e
c
o
d
e
r
-
o
n
l
y
En
c
o
d
e
r
-
o
n
l
y
M
u
l
t
i
m
o
d
a
l
t
r
a
n
sf
o
r
m
e
r
H
y
b
r
i
d
t
r
a
n
sf
o
r
m
e
r
Tr
a
i
n
i
n
g
d
a
t
a
W
e
b
,
c
o
d
e
,
b
o
o
k
s
W
i
k
i
p
e
d
i
a
+
b
o
o
k
s
M
u
l
t
i
m
o
d
a
l
w
e
b
-
s
c
a
l
e
C
o
d
e
a
n
d
f
i
n
a
n
c
i
a
l
d
a
t
a
C
o
s
t
/
l
a
t
e
n
c
y
H
i
g
h
Lo
w
M
o
d
e
r
a
t
e
Lo
w
U
se
-
c
a
se
f
i
t
G
e
n
e
r
a
l
-
p
u
r
p
o
s
e
C
l
a
s
si
f
i
c
a
t
i
o
n
M
u
l
t
i
m
o
d
a
l
AI
C
o
d
e
,
f
i
n
a
n
c
e
M
u
l
t
i
m
o
d
a
l
su
p
p
o
r
t
Li
mi
t
e
d
(GPT
-
4o)
No
F
u
l
l
No
A
P
I
/
a
c
c
e
ss
C
l
o
se
d
O
p
e
n
C
l
o
se
d
O
p
e
n
-
so
u
r
c
e
3
.
2
.
Co
m
pa
ra
t
iv
e
a
na
ly
s
is
o
f
L
L
M
m
o
dels
T
ab
le
3
p
r
o
v
id
es
a
co
m
p
r
eh
en
s
iv
e
s
tr
u
ctu
r
ed
co
m
p
ar
is
o
n
ac
r
o
s
s
1
7
k
e
y
p
ar
am
eter
s
,
in
clu
d
in
g
d
ev
elo
p
er
,
m
o
d
el
ty
p
e,
co
n
te
x
t
len
g
th
,
atten
tio
n
m
ec
h
an
is
m
,
to
k
en
izatio
n
,
p
r
e
-
tr
ain
in
g
o
b
jectiv
es,
tr
ain
in
g
d
ata,
f
in
e
-
tu
n
in
g
a
p
p
r
o
ac
h
es,
s
ca
lab
ilit
y
,
p
e
r
f
o
r
m
an
ce
ch
ar
ac
ter
is
tics
,
s
ec
u
r
ity
co
n
s
id
e
r
atio
n
s
,
f
lex
i
b
ilit
y
,
co
d
e
g
en
e
r
atio
n
ca
p
a
b
ilit
y
,
in
f
er
en
ce
s
p
ee
d
,
ac
ce
s
s
ib
ilit
y
,
s
tr
en
g
th
s
,
wea
k
n
ess
es,
an
d
b
est
u
s
e
ca
s
es.
T
h
is
d
etailed
co
m
p
ar
is
o
n
s
er
v
es a
s
a
r
ef
er
en
ce
g
u
id
e
f
o
r
m
o
d
el
s
e
lectio
n
an
d
d
ep
lo
y
m
en
t d
ec
is
i
o
n
s
.
Fig
u
r
e
1
p
r
esen
ts
a
r
a
d
ar
c
h
a
r
t
v
is
u
aliza
tio
n
c
o
m
p
ar
i
n
g
th
e
ca
p
ab
ilit
ies
o
f
GPT
-
4
,
B
E
R
T
,
Gem
in
i,
an
d
Dee
p
Seek
ac
r
o
s
s
m
u
ltip
le
p
er
f
o
r
m
a
n
ce
d
im
en
s
io
n
s
,
in
clu
d
in
g
r
ea
s
o
n
in
g
ca
p
ab
ilit
y
,
co
n
tex
tu
al
u
n
d
er
s
tan
d
i
n
g
,
m
u
ltimo
d
al
in
teg
r
atio
n
,
d
o
m
ai
n
s
p
ec
iali
za
tio
n
,
an
d
d
e
p
lo
y
m
e
n
t
f
lex
ib
ilit
y
.
T
h
e
r
ad
ar
ch
ar
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
7
2
2
-
3
2
2
1
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
,
Vo
l.
7
,
No
.
3
,
No
v
em
b
er
20
26
:
4
0
4
-
418
410
p
r
o
v
id
es
a
h
o
lis
tic
v
is
u
al
r
ep
r
esen
tatio
n
o
f
th
e
r
elativ
e
s
tr
en
g
th
s
an
d
wea
k
n
ess
es
o
f
ea
ch
m
o
d
el,
en
ab
lin
g
r
ap
id
co
m
p
ar
is
o
n
o
f
th
eir
ca
p
a
b
ilit
y
p
r
o
f
iles
.
T
ab
le
3.
Stru
ctu
r
ed
d
etailed
c
o
m
p
ar
is
o
n
b
etwe
en
LLM
m
o
d
els
ac
r
o
s
s
k
ey
p
ar
am
eter
s
F
e
a
t
u
r
e
G
P
T
-
4
B
ER
T
G
e
mi
n
i
D
e
e
p
S
e
e
k
D
e
v
e
l
o
p
e
r
O
p
e
n
A
I
G
o
o
g
l
e
G
o
o
g
l
e
D
e
e
p
M
i
n
d
D
e
e
p
S
e
e
k
A
I
M
o
d
e
l
t
y
p
e
D
e
c
o
d
e
r
-
o
n
l
y
t
r
a
n
sf
o
r
mer
En
c
o
d
e
r
-
o
n
l
y
t
r
a
n
sf
o
r
mer
M
u
l
t
i
m
o
d
a
l
t
r
a
n
sf
o
r
m
e
r
D
o
ma
i
n
-
s
p
e
c
i
f
i
c
t
r
a
n
sf
o
r
me
r
C
o
n
t
e
x
t
l
e
n
g
t
h
1
2
8
K
t
o
k
e
n
s
5
1
2
t
o
k
e
n
s
1
M
t
o
k
e
n
s
2
0
0
K
+
t
o
k
e
n
s
M
u
l
t
i
m
o
d
a
l
Y
e
s (t
e
x
t
,
v
i
s
i
o
n
)
No
Y
e
s (t
e
x
t
,
v
i
s
i
o
n
,
a
u
d
i
o
,
a
n
d
c
o
d
e
)
No
A
t
t
e
n
t
i
o
n
mec
h
a
n
i
sm
A
u
t
o
r
e
g
r
e
ssi
v
e
se
l
f
-
a
t
t
e
n
t
i
o
n
B
i
d
i
r
e
c
t
i
o
n
a
l
s
e
l
f
-
a
t
t
e
n
t
i
o
n
C
r
o
ss
-
m
o
d
a
l
i
t
y
a
t
t
e
n
t
i
o
n
O
p
t
i
mi
z
e
d
se
l
f
-
a
t
t
e
n
t
i
o
n
To
k
e
n
i
z
a
t
i
o
n
B
P
E
W
o
r
d
P
i
e
c
e
M
o
d
a
l
i
t
y
-
S
p
e
c
i
f
i
c
H
y
b
r
i
d
(
c
o
n
t
e
x
t
u
a
l
a
n
d
d
o
m
a
i
n
-
s
p
e
c
i
f
i
c
)
Pre
-
t
r
a
i
n
i
n
g
o
b
j
e
c
t
i
v
e
N
e
x
t
t
o
k
e
n
p
r
e
d
i
c
t
i
o
n
M
L
M
,
N
S
P
C
r
o
ss
-
m
o
d
a
l
i
t
y
l
e
a
r
n
i
n
g
Ta
sk
-
s
p
e
c
i
f
i
c
l
e
a
r
n
i
n
g
Tr
a
i
n
i
n
g
d
a
t
a
I
n
t
e
r
n
e
t
-
sca
l
e
d
a
t
a
set
,
d
i
v
e
r
se
W
i
k
i
p
e
d
i
a
,
B
o
o
k
sC
o
r
p
u
s
M
u
l
t
i
m
o
d
a
l
,
w
e
b
,
a
n
d
c
o
d
e
La
r
g
e
-
sca
l
e
w
e
b
c
o
r
p
u
s
F
i
n
e
-
t
u
n
i
n
g
S
u
p
p
o
r
t
e
d
v
i
a
O
p
e
n
A
I
A
P
I
s
P
r
e
t
r
a
i
n
e
d
,
f
i
n
e
-
t
u
n
e
d
mo
d
e
l
s
S
u
p
p
o
r
t
e
d
v
i
a
G
o
o
g
l
e
A
I
F
i
n
e
-
t
u
n
e
d
f
o
r
r
e
a
so
n
i
n
g
S
c
a
l
a
b
i
l
i
t
y
H
i
g
h
,
b
u
t
r
e
so
u
r
c
e
-
i
n
t
e
n
si
v
e
M
o
d
e
r
a
t
e
,
o
p
t
i
mi
z
e
d
f
o
r
N
LP t
a
s
k
s
H
i
g
h
,
b
u
t
e
x
p
e
n
s
i
v
e
t
r
a
i
n
i
n
g
c
o
s
t
s
H
i
g
h
f
o
r
d
o
ma
i
n
-
sp
e
c
i
f
i
c
a
p
p
l
i
c
a
t
i
o
n
s
P
e
r
f
o
r
ma
n
c
e
Ex
c
e
l
l
e
n
t
f
o
r
g
e
n
e
r
a
t
i
v
e
AI
B
e
st
f
o
r
s
e
ma
n
t
i
c
u
n
d
e
r
s
t
a
n
d
i
n
g
S
t
r
o
n
g
mu
l
t
i
m
o
d
a
l
c
a
p
a
b
i
l
i
t
i
e
s
B
e
st
i
n
s
p
e
c
i
a
l
i
z
e
d
d
o
m
a
i
n
s
S
e
c
u
r
i
t
y
a
n
d
mai
n
t
a
i
n
a
b
i
l
i
t
y
M
o
d
e
r
a
t
e
,
v
u
l
n
e
r
a
b
l
e
t
o
h
a
l
l
u
c
i
n
a
t
i
o
n
s
H
i
g
h
,
r
o
b
u
s
t
b
i
d
i
r
e
c
t
i
o
n
a
l
t
r
a
i
n
i
n
g
M
o
d
e
r
a
t
e
,
i
n
c
r
e
a
s
e
d
c
o
m
p
l
e
x
i
t
y
H
i
g
h
,
t
a
s
k
-
o
p
t
i
m
i
z
e
d
a
n
d
e
f
f
i
c
i
e
n
t
F
l
e
x
i
b
i
l
i
t
y
H
i
g
h
,
d
i
v
e
r
se
t
e
x
t
-
b
a
se
d
a
p
p
l
i
c
a
t
i
o
n
s
M
o
d
e
r
a
t
e
,
l
i
m
i
t
e
d
t
o
N
LP
V
e
r
y
h
i
g
h
,
t
e
x
t
,
i
m
a
g
e
,
a
n
d
a
u
d
i
o
s
u
p
p
o
r
t
Lo
w
,
sp
e
c
i
a
l
i
z
e
d
t
a
sk
s
C
o
d
e
g
e
n
e
r
a
t
i
o
n
S
t
r
o
n
g
(
v
i
a
G
P
T
-
4
Tu
r
b
o
)
N
o
t
d
e
s
i
g
n
e
d
f
o
r
c
o
d
e
g
e
n
e
r
a
t
i
o
n
S
t
r
o
n
g
(
su
p
p
o
r
t
s
mu
l
t
i
p
l
e
l
a
n
g
u
a
g
e
s)
G
o
o
d
,
o
p
t
i
mi
z
e
d
f
o
r
c
o
d
i
n
g
I
n
f
e
r
e
n
c
e
sp
e
e
d
F
a
st
(
w
i
t
h
o
p
t
i
m
i
z
a
t
i
o
n
s)
F
a
st
e
r
(
l
i
g
h
t
e
r
mo
d
e
l
)
O
p
t
i
mi
z
e
d
f
o
r
sp
e
e
d
a
n
d
sca
l
e
F
a
st
,
o
p
t
i
mi
z
e
d
f
o
r
l
a
r
g
e
c
o
n
t
e
x
t
s
A
c
c
e
ss
i
b
i
l
i
t
y
A
P
I
v
i
a
O
p
e
n
A
I
(
p
a
i
d
)
O
p
e
n
-
so
u
r
c
e
(
B
E
R
T
-
b
a
s
e
,
l
a
r
g
e
)
G
o
o
g
l
e
C
l
o
u
d
A
I
(
p
a
i
d
)
Li
mi
t
e
d
p
u
b
l
i
c
a
c
c
e
ss
S
t
r
e
n
g
t
h
s
S
t
r
o
n
g
g
e
n
e
r
a
l
A
I
,
d
e
e
p
r
e
a
so
n
i
n
g
,
c
o
d
i
n
g
e
x
p
e
r
t
i
se
C
o
n
t
e
x
t
u
a
l
e
m
b
e
d
d
i
n
g
s,
st
r
o
n
g
N
LP a
p
p
l
i
c
a
t
i
o
n
s
H
i
g
h
m
u
l
t
i
m
o
d
a
l
c
a
p
a
b
i
l
i
t
i
e
s
,
l
o
n
g
c
o
n
t
e
x
t
u
n
d
e
r
s
t
a
n
d
i
n
g
O
p
t
i
mi
z
e
d
f
o
r
l
o
n
g
c
o
n
t
e
x
t
a
n
d
d
e
e
p
r
e
a
so
n
i
n
g
W
e
a
k
n
e
sses
Ex
p
e
n
si
v
e
,
p
o
t
e
n
t
i
a
l
h
a
l
l
u
c
i
n
a
t
i
o
n
s
N
o
t
g
e
n
e
r
a
t
i
v
e
,
l
i
m
i
t
e
d
c
o
n
t
e
x
t
w
i
n
d
o
w
R
e
q
u
i
r
e
s
h
i
g
h
r
e
s
o
u
r
c
e
s
,
l
i
m
i
t
e
d
p
u
b
l
i
c
a
c
c
e
ss
S
t
i
l
l
e
mer
g
i
n
g
,
l
e
ss
k
n
o
w
n
B
e
st
u
se
c
a
s
e
s
C
h
a
t
b
o
t
s,
c
o
d
i
n
g
,
c
o
n
t
e
n
t
g
e
n
e
r
a
t
i
o
n
,
r
e
a
so
n
i
n
g
N
LP t
a
s
k
s (s
e
n
t
i
m
e
n
t
a
n
a
l
y
si
s
,
Q
&A
,
c
l
a
ss
i
f
i
c
a
t
i
o
n
)
M
u
l
t
i
m
o
d
a
l
t
a
s
k
s,
A
I
r
e
sea
r
c
h
,
g
e
n
e
r
a
l
k
n
o
w
l
e
d
g
e
A
I
r
e
a
so
n
i
n
g
,
c
o
d
i
n
g
a
ssi
s
t
a
n
c
e
Fig
u
r
e
1.
LLM
m
o
d
els
ca
p
a
b
ilit
y
co
m
p
ar
ativ
e
r
ad
ar
ch
ar
t
Evaluation Warning : The document was created with Spire.PDF for Python.
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
I
SS
N:
2722
-
3
2
2
1
A
co
mp
a
r
a
tive
r
ev
iew
o
f m
o
d
e
r
n
la
r
g
e
la
n
g
u
a
g
e
mo
d
el
p
a
r
a
d
ig
ms:
GP
T
-
4
,
B
E
R
T,
…
(
K
a
v
is
h
S
a
n
g
h
vi
)
411
3
.
3
.
B
enchm
a
r
k
s
a
nd
re
f
er
ence
lea
derbo
a
rds
T
ab
le
4
s
u
m
m
ar
izes
b
e
n
ch
m
a
r
k
p
er
f
o
r
m
a
n
ce
d
ata
f
o
r
GPT
-
4
/B
E
R
T
/Gem
in
i/Deep
Seek
ac
r
o
s
s
m
ajo
r
ev
alu
atio
n
f
r
am
ewo
r
k
s
in
clu
d
in
g
,
HE
L
M
B
en
c
h
m
ar
k
,
G
L
UE
L
ea
d
er
b
o
ar
d
,
SQu
AD,
Hu
g
g
in
g
Face
Op
en
LLM
L
ea
d
er
b
o
a
r
d
,
an
d
in
ter
n
al
ev
alu
atio
n
s
.
T
h
e
tab
le
in
clu
d
es
s
p
ec
if
ic
b
en
ch
m
ar
k
s
co
r
es,
r
an
k
in
g
p
o
s
itio
n
s
,
an
d
ap
p
r
o
p
r
iate
citatio
n
s
to
en
ab
le
v
er
if
icatio
n
of
r
e
p
o
r
te
d
p
er
f
o
r
m
an
ce
m
et
r
ics.
Fig
u
r
e
2
illu
s
tr
ates
th
e
co
m
p
ar
ativ
e
b
e
n
ch
m
a
r
k
p
er
f
o
r
m
an
ce
of
GPT
-
4,
B
E
R
T
,
Ge
m
in
i,
Dee
p
Seek
,
L
L
aM
A
3,
a
n
d
Mix
tr
al/Mistra
l
ac
r
o
s
s
th
r
ee
m
ajo
r
ev
alu
atio
n
f
r
am
ewo
r
k
s
:
HE
L
M
B
en
ch
m
ar
k
,
Hu
g
g
in
g
Face
Op
en
LLM
L
ea
d
er
b
o
ar
d
,
an
d
Ar
en
a
-
Har
d
L
ea
d
er
b
o
ar
d
.
T
h
e
b
ar
ch
ar
t
v
is
u
aliza
tio
n
en
a
b
le
s
d
ir
ec
t
co
m
p
ar
is
o
n
of
m
o
d
el
p
er
f
o
r
m
an
ce
ac
r
o
s
s
s
tan
d
ar
d
ized
ev
al
u
atio
n
m
etr
ic
s
,
h
ig
h
lig
h
tin
g
r
elativ
e
s
tr
en
g
t
h
s
an
d
id
en
tif
y
in
g
p
er
f
o
r
m
an
c
e
g
ap
s
.
T
ab
le
4
.
B
en
ch
m
a
r
k
s
an
d
r
ef
e
r
en
ce
lead
er
b
o
ar
d
s
u
m
m
ar
y
M
o
d
e
l
B
e
n
c
h
mar
k
p
e
r
f
o
r
m
a
n
c
e
S
o
u
r
c
e
s
G
P
T
-
4
/
G
P
T
-
4o
To
p
-
r
a
n
k
e
d
i
n
H
EL
M
B
e
n
c
h
mar
k
(
h
o
l
i
st
i
c
e
v
a
l
u
a
t
i
o
n
)
,
e
x
c
e
l
s
i
n
r
e
a
s
o
n
i
n
g
,
su
mm
a
r
i
z
a
t
i
o
n
,
M
M
LU
,
a
n
d
B
B
H
.
S
t
a
n
f
o
r
d
H
EL
M
[
4
]
,
O
p
e
n
A
I
Te
c
h
n
i
c
a
l
R
e
p
o
r
t
[
1
]
B
ER
T
(
b
a
se
/
l
a
r
g
e
)
C
o
m
p
e
t
i
t
i
v
e
o
n
G
LU
E
B
e
n
c
h
mar
k
a
n
d
S
Q
u
A
D
b
e
n
c
h
m
a
r
k
s
f
o
r
c
l
a
ss
i
f
i
c
a
t
i
o
n
,
Q
A
,
a
n
d
N
E
R
.
G
LU
E
Le
a
d
e
r
b
o
a
r
d
[
1
9
]
,
S
Q
u
A
D
[
2
0
]
G
e
mi
n
i
1
.
5
N
o
t
y
e
t
p
u
b
l
i
c
i
n
H
E
LM
B
e
n
c
h
mark
o
r
H
u
g
g
i
n
g
F
a
c
e
,
b
u
t
sh
o
w
s s
t
r
o
n
g
p
e
r
f
o
r
m
a
n
c
e
o
n
m
u
l
t
i
mo
d
a
l
t
a
s
k
s
i
n
i
n
t
e
r
n
a
l
D
e
e
p
M
i
n
d
e
v
a
l
u
a
t
i
o
n
s.
G
o
o
g
l
e
D
e
e
p
M
i
n
d
[
2
1
]
D
e
e
p
S
e
e
k
-
C
o
d
e
r
To
p
3
i
n
H
u
g
g
i
n
g
F
a
c
e
O
p
e
n
LL
M
L
e
a
d
e
r
b
o
a
r
d
f
o
r
c
o
d
e
a
n
d
r
e
a
s
o
n
i
n
g
;
e
x
c
e
l
s
i
n
l
o
n
g
-
c
o
n
t
e
x
t
t
a
sk
s
(
2
0
0
K
t
o
k
e
n
s)
.
H
F
O
p
e
n
L
LM
L
e
a
d
e
r
b
o
a
r
d
[
5
]
,
D
e
e
p
S
e
e
k
A
I
G
i
t
H
u
b
[
2
2
]
LLa
M
A
3
(
8
B
/
7
0
B
)
S
t
r
o
n
g
a
c
r
o
ss
o
p
e
n
l
e
a
d
e
r
b
o
a
r
d
s
,
e
sp
e
c
i
a
l
l
y
i
n
m
u
l
t
i
l
i
n
g
u
a
l
a
n
d
r
e
a
s
o
n
i
n
g
e
v
a
l
u
a
t
i
o
n
s
.
H
u
g
g
i
n
g
F
a
c
e
A
r
e
n
a
[
5
]
,
M
e
t
a
A
I
[
3
8
]
M
i
x
t
r
a
l
/
M
i
st
r
a
l
M
i
x
t
r
a
l
-
M
o
E
a
c
h
i
e
v
e
s
t
o
p
r
e
s
u
l
t
s
i
n
O
p
e
n
LL
M
A
r
e
n
a
a
n
d
A
r
e
n
a
-
H
a
r
d
;
M
i
s
t
r
a
l
7
B
p
e
r
f
o
r
ms w
e
l
l
o
n
c
o
s
t
-
e
f
f
i
c
i
e
n
t
se
t
u
p
s.
H
u
g
g
i
n
g
F
a
c
e
A
r
e
n
a
[
5
]
,
M
i
s
t
r
a
l
A
I
[
3
9
]
Fig
u
r
e
2.
LLM
m
o
d
els
co
m
p
a
r
ativ
e
b
en
ch
m
ar
k
r
atin
g
s
(
HE
L
M,
HF,
an
d
Ar
en
a
)
3
.
4
.
Appl
ica
t
io
ns
a
cr
o
s
s
do
m
a
ins
T
h
e
t
r
a
n
s
f
o
r
m
a
t
i
v
e
p
o
t
e
n
t
i
a
l
of
LLMs
s
u
c
h
as
GP
T
-
4
[
7
]
,
B
E
R
T
[
8
]
,
G
e
m
i
n
i
[
2
]
,
a
n
d
D
ee
p
S
e
e
k
[
3
]
is
e
v
i
d
e
n
c
e
d
by
t
h
e
i
r
r
a
p
i
d
l
y
e
x
p
a
n
d
i
n
g
r
o
l
e
a
c
r
o
s
s
d
i
v
e
r
s
e
d
o
m
a
i
n
s
.
In
c
o
n
t
e
m
p
o
r
a
r
y
r
e
s
e
a
r
c
h
a
n
d
p
r
a
c
t
i
c
e
,
t
h
e
s
e
m
o
d
e
l
s
a
r
e
r
e
v
o
l
u
ti
o
n
i
z
i
n
g
i
n
d
u
s
t
r
i
es
by
a
u
t
o
m
a
t
i
n
g
c
o
m
p
l
e
x
t
a
s
k
s
,
e
n
h
a
n
c
i
n
g
d
e
c
is
i
o
n
-
m
ak
i
n
g
p
r
o
c
e
s
s
es
,
a
n
d
e
n
a
b
l
i
n
g
i
n
n
o
v
a
t
i
v
e
a
p
p
r
o
a
c
h
es
to
d
a
t
a
a
n
a
l
y
s
i
s
.
T
h
e
i
r
i
n
te
g
r
a
t
i
o
n
i
n
t
o
d
o
m
a
i
n
s
s
u
c
h
as
s
o
f
t
w
a
r
e
e
n
g
i
n
e
e
r
i
n
g
[
4
0
]
,
f
i
n
a
n
c
e
[
4
1
]
,
h
e
a
lt
h
c
a
r
e
[
4
2
]
,
a
n
d
e
d
u
c
a
t
i
o
n
[
4
3
]
not
o
n
l
y
u
n
d
e
r
s
c
o
r
e
s
t
h
ei
r
v
e
r
s
a
t
il
i
t
y
but
a
ls
o
h
i
g
h
li
g
h
ts
e
m
e
r
g
i
n
g
p
a
t
t
e
r
n
s
in
m
o
d
e
l
d
e
p
l
o
y
m
e
n
t
,
d
o
m
a
i
n
-
s
p
e
c
i
f
i
c
t
u
n
i
n
g
,
a
n
d
t
h
e
c
o
n
v
e
r
g
e
n
c
e
of
m
u
l
t
i
m
o
d
a
l
d
a
t
a
p
r
o
c
e
s
s
i
n
g
[
4
4
]
.
T
h
i
s
s
e
c
ti
o
n
p
r
o
v
i
d
e
s
an
in
-
d
e
p
t
h
a
n
a
l
y
s
i
s
of
k
e
y
a
p
p
l
i
c
a
t
i
o
n
s
,
s
t
r
e
n
g
t
h
s
,
l
im
i
t
a
t
i
o
n
s
,
a
n
d
f
u
t
u
r
e
d
i
r
e
c
t
i
o
n
s
,
d
r
aw
i
n
g
on
t
h
e
la
t
est
l
i
te
r
a
t
u
r
e
a
n
d
e
m
p
i
r
i
c
a
l
f
i
n
d
i
n
g
s
,
r
e
v
e
a
l
i
n
g
s
e
v
e
r
a
l
k
e
y
t
h
e
m
es
:
t
h
e
d
r
i
v
e
t
o
w
a
r
d
a
u
t
o
m
a
t
i
o
n
[
4
3
]
,
t
h
e
n
e
e
d
f
o
r
e
x
p
l
a
i
n
a
b
i
l
it
y
a
n
d
t
r
u
s
t
[
4
5
]
,
an
d
t
h
e
i
m
p
o
r
t
a
n
c
e
of
d
o
m
a
i
n
-
s
p
e
c
i
f
i
c
f
i
n
e
-
t
u
n
i
n
g
[
3
7
]
.
B
e
l
o
w
,
we
r
e
v
i
e
w
a
p
p
li
c
at
i
o
n
s
in
f
o
u
r
c
r
i
ti
c
a
l
s
e
c
t
o
r
s
.
3
.
5
.
So
f
t
wa
re
eng
ineering
3
.
5
.
1
.
Use
ca
s
es
a
nd
inte
g
ra
t
io
n
In
th
e
r
ea
lm
of
s
o
f
twar
e
en
g
in
ee
r
in
g
,
LLMs
h
av
e
b
eg
u
n
to
r
ev
o
lu
tio
n
ize
tr
a
d
itio
n
al
wo
r
k
f
l
o
ws.
On
e
p
r
im
ar
y
u
s
e
is
co
d
e
g
en
er
ati
o
n
an
d
d
eb
u
g
g
in
g
.
GPT
-
4,
f
o
r
in
s
tan
ce
,
is
in
cr
ea
s
in
g
ly
d
ep
lo
y
ed
to
g
en
er
at
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
7
2
2
-
3
2
2
1
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
,
Vo
l.
7
,
No
.
3
,
No
v
em
b
er
20
26
:
4
0
4
-
418
412
b
o
iler
p
late
co
d
e
,
o
f
f
e
r
s
y
n
tax
s
u
g
g
esti
o
n
s
,
an
d
ev
e
n
ass
is
t
in
r
ea
l
-
tim
e
d
eb
u
g
g
in
g
.
T
h
is
a
u
to
m
atio
n
n
o
t
o
n
ly
s
p
ee
d
s
up
th
e
d
e
v
elo
p
m
e
n
t
cy
cle
but
also
p
r
o
v
id
es
im
m
ed
ia
te
f
ee
d
b
ac
k
t
h
at
can
h
elp
d
ev
e
lo
p
er
s
id
en
tify
a
n
d
co
r
r
ec
t
er
r
o
r
s
ea
r
ly
on
[
6
]
,
[
4
5
]
,
[
4
6
]
.
An
o
th
er
p
r
o
m
is
in
g
ap
p
licatio
n
is
in
d
o
cu
m
en
tatio
n
an
d
co
d
e
an
aly
s
is
.
Mo
d
els
lik
e
B
E
R
T
[
8
]
an
d
Dee
p
Seek
lev
er
ag
e
th
eir
b
i
d
ir
ec
tio
n
al
co
n
tex
t
u
n
d
er
s
ta
n
d
in
g
to
p
a
r
s
e
co
m
p
lex
c
o
d
eb
ases
,
g
en
er
ate
m
ea
n
in
g
f
u
l
d
o
cu
m
e
n
tatio
n
,
a
n
d
p
in
p
o
in
t
p
o
ten
tial
v
u
ln
er
a
b
ilit
ies.
T
h
is
ca
p
ab
ilit
y
is
e
s
p
ec
ially
v
alu
ab
le
in
m
ain
tain
in
g
co
n
s
is
ten
cy
an
d
e
n
s
u
r
in
g
th
at
c
r
itical
d
etails
ar
e
not
o
v
er
lo
o
k
ed
d
u
r
in
g
r
ap
i
d
d
ev
elo
p
m
en
t
c
y
cles
[
6
]
,
[
4
7
]
.
Mo
r
eo
v
e
r
,
th
e
in
teg
r
atio
n
of
LLMs
with
d
ev
elo
p
m
en
t
en
v
ir
o
n
m
en
ts
f
u
r
th
e
r
ex
e
m
p
lifie
s
th
eir
u
tili
ty
.
T
o
o
ls
s
u
ch
as
GitHu
b
C
o
p
ilo
t
[
3
5
]
,
p
o
wer
ed
by
GPT
-
4,
p
r
o
v
id
e
co
n
tex
tu
al
c
o
d
in
g
ass
is
tan
ce
d
ir
ec
tly
with
in
in
teg
r
ated
d
e
v
elo
p
m
e
n
t
en
v
ir
o
n
m
en
ts
(
I
DE
s
)
[
3
5
]
.
Gem
in
i
’
s
m
u
ltimo
d
al
ab
ilit
ies
also
e
x
ten
d
to
g
en
er
atin
g
UI
m
o
ck
u
p
s
an
d
v
is
u
ally
an
n
o
tated
co
d
e
r
ev
iews,
th
er
eb
y
en
r
ich
in
g
th
e
d
e
v
elo
p
m
e
n
t
p
r
o
ce
s
s
with
b
o
th
tex
tu
al
an
d
v
is
u
al
f
ee
d
b
ac
k
[
4
4
]
.
3
.
5
.
2
.
Str
eng
t
hs
a
nd
lim
it
a
t
i
o
ns
T
h
e
in
teg
r
atio
n
of
LLMs
in
s
o
f
twar
e
e
n
g
in
ee
r
i
n
g
o
f
f
e
r
s
s
ev
er
al
a
d
v
an
tag
es.
Au
to
m
ated
co
d
e
g
en
er
atio
n
an
d
d
eb
u
g
g
i
n
g
en
h
an
ce
p
r
o
d
u
ctiv
ity
,
r
e
d
u
cin
g
th
e
co
g
n
itiv
e
lo
ad
on
d
ev
el
o
p
er
s
an
d
allo
win
g
th
em
to
f
o
c
u
s
on
h
ig
h
er
-
lev
el
d
esig
n
an
d
p
r
o
b
lem
-
s
o
lv
in
g
[
4
0
]
.
R
ap
id
p
r
o
to
ty
p
in
g
f
ac
ilit
ated
by
th
ese
m
o
d
els
ac
ce
ler
ates
th
e
ag
ile
d
ev
elo
p
m
en
t
p
r
o
ce
s
s
,
wh
ile
au
to
m
ated
d
o
cu
m
en
tatio
n
p
r
o
m
o
tes
u
n
if
o
r
m
c
o
d
in
g
p
r
ac
tices
ac
r
o
s
s
team
s
.
Ho
wev
er
,
ch
allen
g
es
r
em
ain
.
A
n
o
tab
le
lim
itatio
n
is
th
e
p
r
o
p
en
s
ity
f
o
r
h
allu
cin
atio
n
s
an
d
er
r
o
r
s
th
at
LLMs
can
g
en
er
ate
co
d
e
th
at,
wh
ile
s
y
n
tactica
lly
co
r
r
ec
t,
m
ay
co
n
tain
lo
g
ical
or
s
ec
u
r
ity
f
laws
[
4
6
]
.
T
h
e
r
is
k
of
in
tr
o
d
u
ci
n
g
v
u
ln
e
r
ab
ilit
ies
th
r
o
u
g
h
au
to
m
ated
s
u
g
g
esti
o
n
s
n
ec
ess
itates
r
o
b
u
s
t
v
alid
atio
n
an
d
h
u
m
an
o
v
er
s
ig
h
t
[
6
]
.
Ad
d
itio
n
ally
,
d
ep
en
d
en
c
y
on
e
x
ter
n
al
LLM
s
er
v
ices
r
aises
co
n
ce
r
ns
ab
o
u
t
laten
cy
,
co
s
t,
an
d
d
ata
p
r
iv
ac
y
,
wh
ich
m
u
s
t
be
m
itig
ated
by
im
p
r
o
v
e
d
in
teg
r
ati
o
n
p
r
o
to
co
ls
an
d
lo
ca
l
f
in
e
-
tu
n
in
g
s
tr
ateg
ies
[
4
4
]
.
3
.
5
.
3
.
F
uture
enha
ncem
ent
s
L
o
o
k
in
g
ah
ea
d
,
s
ev
er
al
av
e
n
u
es
can
f
u
r
th
e
r
en
h
an
ce
th
e
ap
p
licatio
n
of
LLMs
in
s
o
f
twar
e
en
g
in
ee
r
in
g
:
−
Do
m
ain
-
s
p
ec
if
ic
f
in
e
-
tu
n
in
g
:
t
r
ain
in
g
on
c
u
r
ated
co
d
e
r
ep
o
s
i
to
r
ies
an
d
in
teg
r
atin
g
s
ec
u
r
e
c
o
d
in
g
p
r
ac
tices
co
u
ld
s
ig
n
if
ican
tly
en
h
a
n
ce
ac
cu
r
ac
y
[
4
0
]
.
−
Hy
b
r
id
a
p
p
r
o
ac
h
es:
co
m
b
i
n
in
g
LLM
o
u
tp
u
ts
with
tr
a
d
itio
n
al
s
tatic
an
aly
s
is
to
o
ls
an
d
f
o
r
m
al
v
er
if
icatio
n
m
eth
o
d
s
p
r
o
m
is
es
to
im
p
r
o
v
e
r
eliab
ilit
y
[
46
],
[
47
].
−
Ad
ap
tiv
e
lear
n
in
g
s
y
s
tem
s
:
in
co
r
p
o
r
atin
g
co
n
tin
u
o
u
s
f
ee
d
b
ac
k
lo
o
p
s
f
r
o
m
d
ev
elo
p
er
s
co
u
ld
h
elp
iter
ativ
ely
r
ef
in
e
th
e
m
o
d
els
a
n
d
r
ed
u
ce
er
r
o
r
r
ates
[
6
]
.
3
.
6
.
F
ina
nce
3
.
6
.
1
.
Use
ca
s
es
a
nd
inte
g
ra
t
io
n
In
f
in
a
n
ce
,
LLMs
ar
e
in
cr
ea
s
in
g
ly
lev
er
ag
e
d
to
a
u
to
m
ate
an
d
e
n
h
an
ce
th
e
an
al
y
tical
p
r
o
ce
s
s
es
cr
itical
to
d
ec
is
io
n
-
m
ak
in
g
.
On
e
of
th
e
m
o
s
t
co
m
p
ellin
g
ap
p
licatio
n
s
is
au
to
m
ated
r
ep
o
r
t
g
en
e
r
atio
n
.
Fo
r
ex
am
p
le,
GPT
-
4
is
ca
p
ab
l
e
of
s
y
n
th
esizin
g
v
ast
am
o
u
n
t
s
of
f
in
an
cial
d
ata
i
n
to
co
h
er
e
n
t,
co
m
p
r
eh
en
s
iv
e
r
ep
o
r
ts
,
f
ac
ilit
atin
g
r
a
p
id
d
ec
is
io
n
-
m
ak
in
g
in
f
ast
-
p
ac
ed
m
ar
k
ets
[
4
1
]
.
Ad
d
itio
n
ally
,
r
is
k
ass
ess
m
en
t
an
d
s
en
tim
en
t
a
n
aly
s
is
ar
e
s
ig
n
if
ican
tly
im
p
r
o
v
e
d
th
r
o
u
g
h
m
o
d
els
lik
e
B
E
R
T
.
I
ts
co
n
tex
tu
al
s
en
s
itiv
ity
allo
ws
f
o
r
t
h
e
n
u
an
ce
d
a
n
aly
s
is
of
m
ar
k
et
s
en
tim
en
t
d
er
iv
ed
f
r
o
m
n
ews
ar
ticles,
s
o
cial
m
ed
ia,
an
d
f
in
an
cial
r
ep
o
r
ts
.
T
h
is
ca
p
ab
ilit
y
aid
s
in
id
en
tif
y
in
g
e
m
er
g
in
g
tr
en
d
s
an
d
p
o
ten
tial
r
is
k
s
,
m
ak
in
g
it
an
i
n
v
alu
ab
le
to
o
l
f
o
r
tr
ad
er
s
an
d
r
is
k
m
an
a
g
er
s
[
4
1
]
.
LLMs
also
p
lay
a
k
ey
r
o
le
in
alg
o
r
ith
m
ic
tr
a
d
in
g
.
By
an
aly
zin
g
h
is
to
r
ical
f
in
a
n
cial
d
ata
an
d
id
en
tify
in
g
p
atter
n
s
an
d
an
o
m
alies,
th
ese
m
o
d
els
s
u
p
p
o
r
t
th
e
cr
ea
tio
n
of
p
r
e
d
ictiv
e
tr
ad
in
g
alg
o
r
ith
m
s
[
4
8
]
.
Gem
in
i’
s
m
u
ltimo
d
al
d
ata
p
r
o
ce
s
s
in
g
f
u
r
th
e
r
en
h
an
ce
s
th
ese
an
aly
s
es
by
in
co
r
p
o
r
atin
g
v
is
u
al
in
f
o
r
m
atio
n
f
r
o
m
ch
ar
ts
a
n
d
g
r
ap
h
s
al
o
n
g
s
id
e
tex
tu
al
d
ata,
lead
in
g
to
more
co
m
p
r
eh
e
n
s
iv
e
m
ar
k
et
in
s
ig
h
ts
[
4
4
]
.
Fu
r
t
h
er
m
o
r
e,
s
p
ec
ialized
f
in
an
cial
ap
p
licatio
n
s
s
u
ch
as
p
o
r
tf
o
lio
m
an
ag
e
m
en
t
an
d
r
is
k
an
aly
s
is
ar
e
b
ein
g
e
x
p
lo
r
e
d
with
Dee
p
Seek
,
wh
ich
can
be
f
in
e
-
tu
n
ed
to
ad
d
r
ess
th
e
u
n
iq
u
e
c
h
allen
g
e
s
of
f
in
an
cial
d
ata
an
aly
s
is
[
4
1
]
.
3
.
6
.
2
.
Str
eng
t
hs
a
nd
lim
it
a
t
i
o
ns
T
h
e
p
r
in
cip
al
s
tr
en
g
th
s
of
L
L
M
ap
p
licatio
n
s
in
f
in
an
ce
i
n
clu
d
e
th
e
s
p
ee
d
a
n
d
ef
f
icien
cy
with
wh
ich
lar
g
e
d
atasets
can
be
p
r
o
ce
s
s
e
d
,
en
ab
lin
g
r
ea
l
-
tim
e
d
ec
is
io
n
-
m
ak
in
g
.
T
h
e
i
n
co
r
p
o
r
atio
n
of
u
n
s
tr
u
ctu
r
ed
d
ata
p
r
o
v
id
es
d
ee
p
er
an
aly
tical
i
n
s
ig
h
ts
,
an
d
th
e
a
u
to
m
atio
n
of
r
ep
o
r
t
g
e
n
er
atio
n
an
d
s
en
tim
en
t
an
aly
s
is
s
ig
n
if
ican
tly
r
ed
u
ce
s
o
p
er
atio
n
al
co
s
ts
[
4
1
]
.
Nev
e
r
th
eless
,
th
er
e
ar
e
s
ig
n
if
ica
n
t
ch
allen
g
es
:
Evaluation Warning : The document was created with Spire.PDF for Python.
C
o
m
p
u
t Sci
I
n
f
T
ec
h
n
o
l
I
SS
N:
2722
-
3
2
2
1
A
co
mp
a
r
a
tive
r
ev
iew
o
f m
o
d
e
r
n
la
r
g
e
la
n
g
u
a
g
e
mo
d
el
p
a
r
a
d
ig
ms:
GP
T
-
4
,
B
E
R
T,
…
(
K
a
v
is
h
S
a
n
g
h
vi
)
413
−
Data
s
en
s
itiv
ity
:
f
in
an
cial
d
e
cisi
o
n
s
b
ased
on
L
L
M
o
u
t
p
u
ts
r
eq
u
ir
e
e
x
ce
p
tio
n
ally
h
ig
h
d
ata
q
u
ality
,
as
n
o
is
y
or
b
iased
in
p
u
ts
m
ay
lead
to
er
r
o
n
eo
u
s
ass
ess
m
en
t
s
[
4
9
]
.
−
L
ac
k
of
ex
p
lain
ab
ilit
y
:
th
e
“
b
lack
b
o
x
”
n
atu
r
e
of
m
an
y
L
L
Ms
co
m
p
licates
r
eg
u
lato
r
y
co
m
p
lian
ce
an
d
r
aises
co
n
ce
r
n
s
am
o
n
g
s
tak
eh
o
ld
er
s
r
eg
ar
d
in
g
t
h
e
tr
an
s
p
ar
e
n
cy
of
d
ec
is
io
n
-
m
a
k
in
g
p
r
o
ce
s
s
es
[
5
0
]
.
−
R
eg
u
lato
r
y
co
n
s
tr
ain
ts
:
g
iv
en
th
e
s
tr
ict
r
eg
u
lato
r
y
en
v
ir
o
n
m
en
t
of
f
in
an
ce
,
an
y
er
r
o
r
or
m
is
in
ter
p
r
etatio
n
can
h
av
e
c
o
s
tly
im
p
licatio
n
s
[
5
1
]
.
3
.
6
.
3
.
F
uture
enha
ncem
ent
s
To
o
v
e
r
co
m
e
th
ese
ch
alle
n
g
es,
th
e
f
o
llo
win
g
en
h
a
n
ce
m
en
ts
ar
e
r
ec
o
m
m
e
n
d
ed
:
−
Hy
b
r
id
f
in
an
cial
m
o
d
els:
in
teg
r
atin
g
LLMs
with
tr
ad
iti
o
n
al
q
u
an
titativ
e
m
o
d
els
an
d
ec
o
n
o
m
etr
ic
tech
n
iq
u
es
co
u
l
d
p
r
o
d
u
ce
m
o
r
e
r
o
b
u
s
t
p
r
ed
ictio
n
s
[
5
2
]
.
−
E
x
p
lain
ab
ilit
y
f
r
am
ewo
r
k
s
:
d
ev
elo
p
in
g
m
eth
o
d
s
to
elu
cid
ate
LLM
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
es
w
ill
be
cr
u
cial
in
b
u
ild
i
n
g
tr
u
s
t
with
r
eg
u
lato
r
s
an
d
s
tak
eh
o
ld
er
s
[
5
3
]
.
−
R
ea
l
-
tim
e
d
ata
in
teg
r
atio
n
:
th
e
in
co
r
p
o
r
atio
n
of
liv
e
d
ata
f
e
ed
s
can
f
u
r
th
er
b
o
o
s
t
th
e
p
r
ed
ictiv
e
ac
cu
r
ac
y
an
d
r
esp
o
n
s
iv
en
ess
of
f
in
a
n
cial
m
o
d
els
[
5
4
]
.
3
.
7
.
H
ea
lt
hca
re
3
.
7
.
1
.
Use
ca
s
es
a
nd
inte
g
ra
t
io
n
In
h
e
a
l
t
h
c
a
r
e
,
t
h
e
d
e
p
l
o
y
m
e
n
t
of
LLMs
s
u
c
h
as
GP
T
-
4,
B
E
R
T
,
G
e
m
i
n
i
,
a
n
d
D
e
e
p
S
ee
k
is
t
r
a
n
s
f
o
r
m
i
n
g
c
l
i
n
i
c
al
a
n
d
a
d
m
i
n
i
s
t
r
at
i
v
e
wo
r
k
f
l
o
w
s
.
O
n
e
of
t
h
e
m
o
s
t
im
p
a
c
t
f
u
l
a
p
p
l
i
c
a
ti
o
n
s
is
c
l
i
n
ic
a
l
d
e
c
i
s
i
o
n
s
u
p
p
o
r
t
.
G
P
T
-
4
can
ass
is
t
cl
i
n
i
c
ia
n
s
by
s
u
m
m
a
r
i
z
i
n
g
p
at
i
e
n
t
h
is
t
o
r
i
es
,
s
u
g
g
e
s
t
i
n
g
p
r
el
i
m
i
n
ar
y
d
i
a
g
n
o
s
e
s
,
a
n
d
r
e
c
o
m
m
e
n
d
i
n
g
t
r
e
a
t
m
e
n
t
p
la
n
s
,
t
h
e
r
e
b
y
s
t
r
e
a
m
li
n
i
n
g
t
h
e
d
ec
i
s
i
o
n
-
m
a
k
i
n
g
p
r
o
c
es
s
a
n
d
p
o
t
e
n
t
i
a
ll
y
i
m
p
r
o
v
i
n
g
p
a
t
i
e
n
t
o
u
t
c
o
m
e
s
[
5
5
]
.
A
n
o
t
h
er
k
e
y
a
p
p
l
i
c
a
t
i
o
n
is
t
h
e
a
u
t
o
m
a
t
e
d
g
e
n
e
r
a
t
i
o
n
of
m
e
d
i
cal
r
e
p
o
r
t
s
.
T
h
e
s
e
m
o
d
e
ls
can
p
r
o
d
u
c
e
d
i
s
c
h
a
r
g
e
s
u
m
m
a
r
i
e
s
,
p
r
o
g
r
es
s
n
o
te
s
,
a
n
d
o
t
h
e
r
cl
i
n
i
c
a
l
d
o
c
u
m
e
n
t
a
ti
o
n
,
r
e
d
u
c
i
n
g
t
h
e
a
d
m
i
n
is
t
r
at
i
v
e
b
u
r
d
e
n
on
h
e
a
l
t
h
c
a
r
e
p
r
o
v
i
d
e
r
s
a
n
d
a
l
l
o
w
i
n
g
t
h
e
m
to
f
o
c
u
s
m
o
r
e
on
p
a
t
i
e
n
t
c
a
r
e
[
5
6
]
.
LLMs
ar
e
also
p
iv
o
tal
in
th
e
r
ea
lm
of
p
er
s
o
n
alize
d
m
e
d
icin
e.
By
an
aly
zin
g
in
d
iv
id
u
al
p
atien
t
d
ata,
th
ese
m
o
d
els
can
o
f
f
er
tailo
r
ed
tr
ea
tm
en
t
r
ec
o
m
m
en
d
ati
o
n
s
an
d
r
is
k
ass
ess
m
en
ts
.
Gem
in
i’
s
ab
ilit
y
to
in
teg
r
ate
m
u
ltimo
d
al
d
ata,
s
u
ch
as
r
ad
io
lo
g
ical
im
a
g
es
with
tex
tu
al
p
atien
t
r
ec
o
r
d
s
,
en
h
an
ce
s
d
iag
n
o
s
tic
ac
cu
r
ac
y
,
wh
ile
Dee
p
Seek
’
s
ca
p
ac
ity
f
o
r
p
ar
s
in
g
co
m
p
l
ex
m
e
d
ical
liter
atu
r
e
s
u
p
p
o
r
ts
d
r
u
g
in
ter
ac
tio
n
an
aly
s
is
an
d
ev
id
en
ce
-
b
ased
d
ec
is
io
n
-
m
ak
in
g
[
5
7
]
.
3
.
7
.
2
.
Str
eng
t
hs
a
nd
lim
it
a
t
i
o
ns
T
h
e
p
r
im
ar
y
ad
v
an
tag
e
of
LLMs
in
h
ea
lth
ca
r
e
is
th
e
d
r
am
atic
im
p
r
o
v
em
e
n
t
in
ef
f
icien
cy
,
au
to
m
atin
g
r
o
u
tin
e
d
o
cu
m
e
n
t
atio
n
an
d
d
ata
a
n
aly
s
is
task
s
,
wh
ich
f
r
ee
s
up
v
al
u
ab
le
clin
ic
ian
tim
e
[
5
8
]
,
[
5
9
]
.
E
n
h
an
ce
d
p
atien
t
c
o
m
m
u
n
ic
atio
n
th
r
o
u
g
h
v
ir
tu
al
ass
is
ta
n
ts
an
d
im
p
r
o
v
e
d
r
eso
u
r
ce
o
p
tim
izatio
n
f
u
r
th
er
u
n
d
er
s
co
r
es
th
e
p
o
ten
tial
of
th
ese
m
o
d
els
[
6
0
]
.
Ho
wev
er
,
th
e
h
ea
lth
ca
r
e
d
o
m
ain
p
r
esen
ts
d
is
tin
ct
ch
allen
g
es:
−
R
is
k
of
m
is
in
f
o
r
m
atio
n
:
in
ac
cu
r
ate
or
h
allu
cin
ate
d
o
u
tp
u
t
s
can
h
av
e
s
er
io
u
s
im
p
licatio
n
s
f
o
r
p
atien
t
s
af
ety
[
6
1
]
.
−
Data
p
r
iv
ac
y
an
d
s
ec
u
r
ity
:
th
e
h
an
d
lin
g
of
s
en
s
itiv
e
p
atien
t
d
ata
n
ec
ess
itate
s
r
ig
o
r
o
u
s
co
m
p
lian
ce
with
p
r
iv
ac
y
r
eg
u
latio
n
s
an
d
r
o
b
u
s
t
cy
b
er
s
ec
u
r
ity
m
ea
s
u
r
es
[
6
2
]
.
−
E
th
ical
an
d
leg
al
co
n
ce
r
n
s
:
is
s
u
es
of
liab
ilit
y
,
co
n
s
en
t,
a
n
d
p
o
ten
tial
alg
o
r
ith
m
ic
b
ias
r
eq
u
ir
e
ca
r
ef
u
l
m
an
ag
em
en
t
to
en
s
u
r
e
t
h
at
AI
-
d
r
iv
en
d
ec
is
io
n
s
do
not
ad
v
er
s
ely
im
p
ac
t
p
atien
t
ca
r
e
[
6
3
]
.
3
.
7
.
3
.
F
uture
enha
ncem
ent
s
Fu
tu
r
e
im
p
r
o
v
em
en
ts
in
h
ea
lth
ca
r
e
ap
p
licatio
n
s
of
LLMs
co
u
ld
f
o
c
u
s
on:
−
Mu
ltimo
d
al
d
ata
f
u
s
io
n
:
in
teg
r
atin
g
d
iv
er
s
e
d
ata
s
o
u
r
ce
s
,
s
u
ch
as,
g
en
etic,
lab
o
r
at
o
r
y
,
a
n
d
im
ag
in
g
ca
n
lead
to
more
h
o
lis
tic
an
d
ac
c
u
r
ate
p
atien
t
ass
ess
m
en
ts
[
6
4
]
.
−
R
eg
u
lato
r
y
-
co
m
p
lian
t
m
o
d
els
:
d
ev
elo
p
in
g
h
ea
lth
ca
r
e
-
s
p
ec
if
ic
LLMs
th
at
s
tr
ictly
ad
h
e
r
e
to
r
eg
u
lato
r
y
s
tan
d
ar
d
s
will
be
ess
en
tial
f
o
r
s
af
e
an
d
ef
f
ec
tiv
e
d
ep
lo
y
m
en
t
[
6
5
]
.
−
I
n
ter
p
r
etab
ilit
y
an
d
tr
an
s
p
ar
e
n
cy
:
en
h
an
ci
n
g
th
e
ex
p
lain
a
b
i
lity
of
m
o
d
el
o
u
tp
u
ts
will
f
o
s
ter
g
r
ea
ter
tr
u
s
t
am
o
n
g
clin
ician
s
an
d
im
p
r
o
v
e
co
llab
o
r
ativ
e
d
ec
is
io
n
-
m
ak
i
n
g
[
6
6
]
.
−
C
o
n
tin
u
o
u
s
clin
ical
v
alid
atio
n
:
o
n
g
o
in
g
r
ea
l
-
wo
r
ld
test
in
g
a
n
d
iter
ativ
e
u
p
d
ates
will
be
cr
itical
to
en
s
u
r
e
th
at
LLM
r
ec
o
m
m
e
n
d
atio
n
s
r
e
m
ain
ac
cu
r
ate
a
n
d
r
elev
a
n
t
[
6
7
]
.
3
.
8
.
E
du
ca
t
io
n
3
.
8
.
1
.
Use
ca
s
es
a
nd
inte
g
ra
t
io
n
LLMs
ar
e
also
p
o
is
ed
to
r
ev
o
lu
tio
n
ize
th
e
e
d
u
ca
tio
n
s
ec
to
r
by
e
n
ab
lin
g
p
er
s
o
n
alize
d
lea
r
n
in
g
a
n
d
au
to
m
atin
g
ad
m
i
n
is
tr
ativ
e
task
s
.
GPT
-
4,
f
o
r
ex
a
m
p
le,
is
u
s
ed
to
cr
ea
te
p
er
s
o
n
alize
d
le
ar
n
in
g
co
n
ten
t
an
d
p
r
o
v
id
e
r
ea
l
-
tim
e
tu
to
r
in
g
,
ad
ap
tin
g
in
s
tr
u
ctio
n
al
m
ater
ials
to
in
d
iv
id
u
al
s
tu
d
en
t
n
ee
d
s
[
6
8
]
.
T
h
is
Evaluation Warning : The document was created with Spire.PDF for Python.