I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
15
,
No
.
4
,
A
u
g
u
s
t 2
0
2
6
,
p
p
.
3
3
6
5
~
3
3
7
5
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
4
.
p
p
3
3
6
5
-
3
3
7
5
3365
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
WVisio
nBE
RT
-
V
L:
a
multimo
da
l
mo
del architec
tur
e f
o
r
toxicit
y
cla
ss
ificat
io
n on so
cia
l medi
a
platform
s usin
g
larg
e
la
ng
ua
g
e mo
dels
Wit
t
a
L
is
t
iy
a
Ning
rum
,
Achm
a
d B
enny
M
utia
ra
,
Dia
na
I
k
a
s
a
ri
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
m
a
t
i
o
n
Te
c
h
n
o
l
o
g
y
,
G
r
a
d
u
a
t
e
P
r
o
g
r
a
m
,
G
u
n
a
d
a
r
ma
U
n
i
v
e
r
s
i
t
y
,
D
e
p
o
k
,
I
n
d
o
n
e
s
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
an
3
1
,
2
0
2
5
R
ev
is
ed
Ma
y
1
6
,
2
0
2
6
Acc
ep
ted
Ma
y
2
4
,
2
0
2
6
Th
e
in
c
re
a
sin
g
p
re
v
a
len
c
e
o
f
t
o
x
ic
c
o
n
ten
t
o
n
so
c
ial
m
e
d
ia,
c
o
n
v
e
y
e
d
th
ro
u
g
h
tex
t,
ima
g
e
s,
a
n
d
v
id
e
o
s,
p
o
se
s
sig
n
if
ica
n
t
c
h
a
ll
e
n
g
e
s
fo
r
a
u
to
m
a
ted
c
o
n
ten
t
m
o
d
e
ra
ti
o
n
s
y
ste
m
s.
Alth
o
u
g
h
p
ri
o
r
st
u
d
ies
h
a
v
e
re
p
o
rted
p
ro
m
isin
g
re
su
lt
s
in
u
n
imo
d
a
l
a
n
d
b
imo
d
a
l
se
tt
in
g
s,
th
e
y
o
fte
n
fa
il
to
c
a
p
tu
r
e
imp
li
c
it
a
n
d
c
o
n
tex
tu
a
l
to
x
icit
y
e
m
e
rg
in
g
fr
o
m
i
n
tera
c
ti
o
n
s
a
c
ro
ss
m
u
lt
i
p
le
m
o
d
a
li
ti
e
s,
p
a
rti
c
u
larl
y
i
n
n
o
n
-
E
n
g
li
s
h
e
n
v
iro
n
m
e
n
ts.
Th
is
p
a
p
e
r
p
ro
p
o
se
s
WVisio
n
BERT
-
V
L,
a
n
e
n
d
-
to
-
e
n
d
m
u
lt
imo
d
a
l
fra
m
e
wo
rk
f
o
r
to
x
icity
d
e
tec
ti
o
n
t
h
a
t
in
teg
ra
tes
tex
t,
im
a
g
e
,
a
n
d
v
id
e
o
m
o
d
a
li
ti
e
s
wit
h
in
a
u
n
ifi
e
d
a
rc
h
it
e
c
tu
re
.
Th
e
p
ro
p
o
se
d
m
o
d
e
l
in
c
o
r
p
o
ra
tes
m
o
d
a
li
t
y
-
sp
e
c
ifi
c
e
n
c
o
d
e
rs,
b
id
irec
ti
o
n
a
l
m
u
lt
i
-
h
e
a
d
c
ro
ss
a
tt
e
n
ti
o
n
(BM
HCA
)
f
o
r
c
r
o
ss
-
m
o
d
a
l
sy
n
c
h
r
o
n
iza
ti
o
n
,
a
n
d
a
n
a
d
a
p
ti
v
e
fu
sio
n
g
a
te
t
o
d
y
n
a
m
ica
ll
y
b
a
lan
c
e
m
o
d
a
li
ty
c
o
n
tri
b
u
ti
o
n
s.
A
b
a
lan
c
e
d
m
u
lt
im
o
d
a
l
d
a
tas
e
t
is
c
o
n
str
u
c
ted
fro
m
so
c
ial
m
e
d
ia
p
latfo
rm
s,
i
n
c
lu
d
in
g
X,
In
sta
g
ra
m
,
a
n
d
Ti
k
T
o
k
,
a
n
d
re
fin
e
d
u
sin
g
a
m
o
d
e
l
-
b
a
se
d
lab
e
li
n
g
stra
teg
y
with
l
imited
h
u
m
a
n
-
in
-
th
e
-
lo
o
p
v
a
li
d
a
ti
o
n
.
E
x
p
e
rime
n
tal
re
su
lt
s
o
n
a
c
u
sto
m
In
d
o
n
e
sia
n
d
a
tas
e
t
d
e
m
o
n
stra
te
stro
n
g
i
n
-
d
o
m
a
in
p
e
rfo
rm
a
n
c
e
,
a
c
h
iev
in
g
a
n
a
c
c
u
ra
c
y
o
f
9
4
.
1
2
%
,
a
ma
c
ro
-
F
1
o
f
0
.
9
4
0
7
,
a
n
d
a
re
c
e
iv
e
r
o
p
e
ra
ti
n
g
c
h
a
ra
c
teristic
-
a
re
a
u
n
d
e
r
th
e
c
u
r
v
e
(
ROC
-
AUC
)
o
f
0
.
9
7
2
1
,
with
r
o
b
u
stn
e
ss
fu
rth
e
r
v
a
li
d
a
ted
t
h
ro
u
g
h
fiv
e
-
fo
l
d
c
ro
ss
-
v
a
li
d
a
ti
o
n
.
Cro
ss
-
d
a
tas
e
t
e
v
a
lu
a
ti
o
n
h
i
g
h
li
g
h
ts
c
h
a
ll
e
n
g
e
s
re
late
d
to
d
o
m
a
in
s
h
ift
,
u
n
d
e
rsc
o
rin
g
t
h
e
n
e
e
d
fo
r
fu
t
u
re
re
se
a
rc
h
o
n
r
o
b
u
st
a
n
d
d
o
m
a
in
-
a
d
a
p
ti
v
e
m
u
lt
imo
d
a
l
to
x
i
c
it
y
d
e
tec
ti
o
n
.
K
ey
w
o
r
d
s
:
C
o
n
ten
t m
o
d
er
atio
n
C
r
o
s
s
-
m
o
d
al
atten
tio
n
Mu
ltimo
d
al
to
x
icity
d
etec
tio
n
So
cial
m
ed
ia
an
aly
s
is
T
r
an
s
f
o
r
m
e
r
-
b
ased
f
u
s
io
n
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
W
itta L
is
tiy
a
Nin
g
r
u
m
Dep
ar
tm
en
t
o
f
I
n
f
o
r
m
atio
n
T
e
ch
n
o
lo
g
y
,
Gr
a
d
u
ate
Pro
g
r
am
,
Gu
n
ad
ar
m
a
U
n
iv
er
s
ity
Dep
o
k
,
I
n
d
o
n
esia
E
m
ail: w
ita_
lis
tiy
a
@
s
taf
f
.
g
u
n
ad
ar
m
a.
ac
.
id
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
ev
o
lu
tio
n
o
f
in
f
o
r
m
atio
n
an
d
co
m
m
u
n
icatio
n
tech
n
o
lo
g
y
h
as
co
n
tr
i
b
u
ted
to
th
e
r
is
e
o
f
a
d
ig
itally
co
n
n
ec
ted
er
a
d
o
m
i
n
ated
b
y
o
n
lin
e
p
latf
o
r
m
s
[
1
]
.
So
cial
m
ed
ia
h
as
b
ec
o
m
e
a
p
r
im
ar
y
m
ed
iu
m
f
o
r
o
n
lin
e
co
m
m
u
n
icatio
n
,
e
n
ab
lin
g
r
ap
id
an
d
lar
g
e
-
s
ca
le
in
f
o
r
m
atio
n
d
is
s
em
in
atio
n
as
well
as
s
o
cial
in
ter
ac
tio
n
[
2
]
.
Ho
wev
e
r
,
th
e
h
ig
h
in
ten
s
ity
o
f
o
n
lin
e
ac
tiv
i
ties
h
as
a
ls
o
co
n
tr
ib
u
ted
to
th
e
in
cr
ea
s
in
g
s
p
r
ea
d
o
f
to
x
ic
co
n
te
n
t,
s
u
ch
as
h
ate
s
p
ee
ch
,
h
ar
ass
m
en
t,
an
d
o
f
f
en
s
iv
e
lan
g
u
a
g
e,
wh
ich
ca
n
n
eg
ativ
ely
af
f
ec
t
in
d
iv
id
u
als
an
d
d
ig
ital
co
m
m
u
n
ities
[
3
]
,
[
4
]
.
T
h
ese
ch
allen
g
es
h
ig
h
lig
h
t
th
e
n
ee
d
f
o
r
a
n
in
tellig
en
t
to
x
icity
d
etec
tio
n
m
ec
h
a
n
is
m
to
f
o
s
ter
a
s
af
er
an
d
m
o
r
e
in
cl
u
s
iv
e
s
o
cial
n
etwo
r
k
in
g
e
n
v
ir
o
n
m
en
t.
R
esear
ch
o
n
to
x
icity
d
etec
t
io
n
in
s
o
cial
m
ed
ia
h
as
c
o
n
tin
u
ed
t
o
p
r
o
g
r
ess
in
p
a
r
allel
with
d
ev
elo
p
m
e
n
ts
in
m
ac
h
in
e
lear
n
in
g
a
n
d
th
e
in
cr
ea
s
in
g
av
aila
b
ilit
y
o
f
lar
g
e
-
s
ca
le
d
atasets
.
E
ar
ly
s
tu
d
ies
m
ain
ly
em
p
lo
y
ed
tr
ad
itio
n
al
m
ac
h
in
e
lear
n
in
g
tech
n
iq
u
es
th
at
u
tili
ze
d
h
an
d
cr
a
f
ted
tex
tu
al
f
ea
tu
r
es
to
id
en
tify
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
3
6
5
-
3
3
7
5
3366
class
if
y
h
ar
m
f
u
l
c
o
n
ten
t
[
5
]
.
W
ith
th
e
g
r
o
win
g
co
m
p
lex
it
y
o
f
lan
g
u
a
g
e
an
d
ex
p
r
ess
iv
e
v
ar
iatio
n
s
,
r
ec
e
n
t
s
tu
d
ies
h
av
e
tr
an
s
itio
n
ed
to
war
d
d
ee
p
lear
n
in
g
ap
p
r
o
a
ch
es
ca
p
ab
le
o
f
au
to
m
atica
lly
lear
n
in
g
laten
t
r
ep
r
esen
tatio
n
s
in
an
en
d
-
to
-
en
d
f
r
am
ewo
r
k
[
6
]
–
[
8
]
.
M
o
s
t
ea
r
ly
wo
r
k
f
o
cu
s
ed
o
n
tex
t
-
b
ased
to
x
icity
d
etec
tio
n
,
r
ef
lectin
g
th
e
d
o
m
in
an
ce
o
f
tex
tu
al
ex
p
r
ess
io
n
o
n
s
o
cial
m
ed
ia
p
latf
o
r
m
s
[
9
]
–
[
1
1
]
.
Alth
o
u
g
h
th
ese
u
n
im
o
d
al
a
p
p
r
o
ac
h
es
a
ch
iev
ed
c
o
m
p
etitiv
e
p
er
f
o
r
m
an
ce
,
th
eir
ef
f
ec
tiv
en
ess
b
ec
o
m
es
lim
ited
wh
e
n
h
ar
m
f
u
l
co
n
ten
t
is
co
n
v
ey
e
d
m
u
ltimo
d
ally
th
r
o
u
g
h
co
m
b
in
atio
n
s
o
f
tex
t,
im
ag
es,
an
d
v
i
d
eo
s
,
wh
er
e
im
p
licit
cr
o
s
s
-
m
o
d
al
co
n
te
x
t is o
f
ten
o
v
er
lo
o
k
e
d
[
1
2
]
.
T
h
e
d
ev
elo
p
m
en
t
o
f
tr
an
s
f
o
r
m
er
-
b
ased
ar
ch
itectu
r
es
h
as
c
o
n
tr
ib
u
ted
to
th
e
r
is
e
o
f
lar
g
e
lan
g
u
ag
e
m
o
d
els
(
L
L
Ms)
,
wh
ich
h
a
v
e
d
em
o
n
s
tr
ated
o
u
ts
tan
d
in
g
ca
p
ab
ilit
ies
ac
r
o
s
s
a
wid
e
r
an
g
e
o
f
n
atu
r
al
lan
g
u
ag
e
p
r
o
ce
s
s
in
g
task
s
,
in
clu
d
in
g
h
ar
m
f
u
l
lan
g
u
ag
e
d
etec
tio
n
,
s
en
tim
en
t
an
aly
s
is
,
an
d
cy
b
er
b
u
lly
in
g
[
1
3
]
–
[
1
6
]
.
Nev
er
th
eless
,
m
o
s
t L
L
M
-
b
ased
s
tu
d
ies r
em
ain
tex
t
-
ce
n
tr
ic
a
n
d
d
o
n
o
t f
u
lly
e
x
p
lo
it th
e
v
is
u
al
in
f
o
r
m
atio
n
th
at
f
r
eq
u
e
n
tly
ac
co
m
p
an
ies s
o
cial
m
ed
ia
co
n
ten
t
[
1
7
]
.
T
o
a
d
d
r
es
s
th
is
lim
itatio
n
,
L
MM
s
h
av
e
b
ee
n
in
tr
o
d
u
ce
d
b
y
in
teg
r
atin
g
in
f
o
r
m
atio
n
f
r
o
m
m
u
ltip
le
m
o
d
alities
[
1
8
]
–
[
2
3
]
.
Mu
ltimo
d
al
ap
p
r
o
ac
h
es
h
av
e
b
ee
n
s
h
o
wn
to
en
r
ich
s
em
an
tic
r
ep
r
esen
tati
o
n
s
an
d
im
p
r
o
v
e
class
if
icati
o
n
p
er
f
o
r
m
an
ce
th
r
o
u
g
h
ef
f
ec
tiv
e
cr
o
s
s
-
m
o
d
a
l
alig
n
m
en
t
[
2
4
]
–
[
2
7
]
.
Desp
it
e
th
ese
ad
v
a
n
ce
s
,
ex
is
tin
g
r
esear
ch
p
r
e
d
o
m
in
an
tly
f
o
cu
s
es
o
n
tex
t
–
im
ag
e
in
teg
r
atio
n
,
wh
ile
ex
p
lo
r
atio
n
o
f
th
e
v
id
eo
m
o
d
ality
,
wh
ic
h
co
n
tain
s
r
ich
er
v
is
u
al
an
d
tem
p
o
r
al
co
n
tex
t,
r
em
ain
s
r
elativ
ely
lim
ited
,
p
a
r
ticu
lar
ly
f
o
r
t
o
x
icity
d
etec
tio
n
task
s
.
T
o
f
a
c
i
l
i
t
at
e
c
r
o
s
s
-
m
o
d
a
l
i
n
t
e
r
a
c
t
i
o
n
,
a
t
t
e
n
ti
o
n
m
e
c
h
a
n
i
s
m
s
s
u
c
h
a
s
m
u
l
t
i
-
h
e
a
d
c
r
o
s
s
a
tt
en
t
i
o
n
h
a
v
e
b
e
e
n
w
i
d
e
l
y
a
d
o
p
t
e
d
a
n
d
p
r
o
v
e
n
e
f
f
e
c
t
i
v
e
i
n
m
o
d
e
l
i
n
g
r
e
l
a
t
io
n
s
h
i
p
s
b
e
t
w
ee
n
m
o
d
a
l
r
e
p
r
e
s
e
n
t
a
t
i
o
n
s
[
2
8
]
,
[
2
9
]
.
T
r
an
s
f
o
r
m
e
r
-
b
ased
atten
tio
n
m
ec
h
an
is
m
s
h
av
e
also
b
ee
n
r
ep
o
r
ted
to
en
h
a
n
ce
m
u
ltimo
d
al
r
ep
r
esen
tatio
n
s
in
co
m
p
lex
d
ata
s
ce
n
ar
io
s
[
3
0
]
,
en
ab
lin
g
m
o
d
els
to
atten
d
to
d
if
f
er
en
t
p
a
r
ts
o
f
t
h
e
in
p
u
t
s
eq
u
en
ce
wh
en
m
ak
in
g
p
r
ed
ictio
n
s
[
3
1
]
–
[
3
3
]
.
I
n
th
e
co
n
tex
t
o
f
s
o
cial
m
ed
ia,
to
x
ic
co
n
ten
t
is
o
f
ten
ex
p
r
ess
ed
im
p
licitly
th
r
o
u
g
h
th
e
in
ter
p
lay
o
f
tex
t,
im
ag
es,
an
d
v
id
eo
s
[
3
4
]
,
[
3
5
]
.
C
o
n
s
eq
u
en
tly
,
m
u
ltimo
d
al
ap
p
r
o
ac
h
es
h
av
e
co
n
s
is
ten
tly
o
u
tp
er
f
o
r
m
ed
u
n
im
o
d
al
m
eth
o
d
s
in
d
etec
tin
g
co
n
tex
tu
al
a
n
d
am
b
ig
u
o
u
s
to
x
ic
co
n
ten
t
[
3
6
]
.
Desp
ite
th
ese
f
in
d
in
g
s
,
th
e
s
im
u
ltan
eo
u
s
in
teg
r
atio
n
o
f
te
x
t,
im
ag
e,
an
d
v
id
eo
with
in
a
u
n
if
ied
en
d
-
to
-
e
n
d
f
r
am
ewo
r
k
r
em
ai
n
s
u
n
d
er
e
x
p
lo
r
e
d
,
p
a
r
ticu
lar
l
y
f
o
r
n
o
n
-
E
n
g
lis
h
lan
g
u
ag
es
s
u
ch
as
I
n
d
o
n
esian
.
T
h
is
g
ap
is
f
u
r
th
er
am
p
lifie
d
b
y
th
e
lim
ited
n
u
m
b
er
o
f
m
u
lti
m
o
d
al
to
x
icity
d
etec
tio
n
s
tu
d
i
es in
th
e
I
n
d
o
n
esian
lan
g
u
ag
e
co
n
tex
t.
Mo
r
eo
v
er
,
v
id
eo
as
a
m
o
d
ality
p
r
o
v
id
es
tem
p
o
r
al
d
y
n
am
ics
th
at
ca
n
ca
p
tu
r
e
n
u
a
n
ce
d
to
x
ic
cu
es
n
o
t
ad
eq
u
ately
r
e
p
r
esen
t
ed
b
y
s
tatic
tex
t
o
r
im
ag
es
al
o
n
e.
T
h
is
s
tu
d
y
p
r
o
p
o
s
es
W
Vis
io
n
B
E
R
T
-
VL
as
a
s
o
lu
tio
n
to
th
e
id
en
tifie
d
ch
a
llen
g
es,
a
m
u
ltimo
d
al
to
x
icity
class
if
icatio
n
ar
ch
itectu
r
e
th
at
in
teg
r
ates
tex
t,
im
ag
e,
an
d
v
id
eo
r
ep
r
esen
t
atio
n
s
in
an
en
d
-
to
-
en
d
m
an
n
er
to
ac
h
iev
e
a
m
o
r
e
h
o
lis
tic
co
n
tex
tu
al
u
n
d
er
s
tan
d
i
n
g
o
f
I
n
d
o
n
esian
s
o
cial
m
ed
ia
co
n
te
n
t.
2.
M
E
T
H
O
D
T
h
is
s
ec
tio
n
d
escr
ib
es
th
e
r
esear
ch
m
eth
o
d
o
lo
g
y
em
p
l
o
y
ed
to
d
esig
n
,
tr
ain
,
an
d
e
v
alu
ate
th
e
p
r
o
p
o
s
ed
m
u
ltimo
d
al
to
x
icity
d
etec
tio
n
m
o
d
el.
T
h
e
o
v
er
al
l
wo
r
k
f
l
o
w
co
m
p
r
is
es
d
ataset
co
n
s
tr
u
ctio
n
,
d
ata
p
r
ep
r
o
ce
s
s
in
g
an
d
lab
elin
g
,
m
u
ltimo
d
al
m
o
d
el
d
esig
n
,
a
n
d
tr
ain
in
g
an
d
ev
alu
atio
n
s
tr
ateg
ies.
T
h
ese
s
tag
es a
r
e
tig
h
tly
in
teg
r
ated
to
s
u
p
p
o
r
t
th
e
p
r
im
ar
y
o
b
jectiv
e
o
f
d
etec
t
in
g
co
n
tex
tu
al
to
x
icity
th
r
o
u
g
h
th
e
f
u
s
io
n
o
f
tex
t,
im
ag
e,
an
d
v
id
eo
m
o
d
alities
.
2
.
1
.
I
dentif
y
ing
da
t
a
s
o
urce
s
Data
co
llectio
n
was
co
n
d
u
cted
th
r
o
u
g
h
a
web
cr
awlin
g
p
r
o
ce
s
s
ac
r
o
s
s
th
r
ee
p
o
p
u
lar
s
o
cial
m
ed
ia
p
latf
o
r
m
s
r
ep
r
esen
tin
g
d
iv
er
s
e
m
u
ltimo
d
al
ch
ar
ac
ter
is
tics
.
T
ex
tu
al
d
ata
wer
e
co
llected
f
r
o
m
th
e
s
o
cial
m
ed
ia
p
latf
o
r
m
X,
r
esu
ltin
g
i
n
4
,
5
4
1
tex
t
en
tr
ies
th
at
r
ef
lect
s
h
o
r
t
an
d
c
o
n
tex
t
-
d
ep
en
d
en
t
lin
g
u
is
tic
ex
p
r
ess
io
n
s
co
m
m
o
n
l
y
f
o
u
n
d
in
o
n
lin
e
in
t
er
ac
tio
n
s
.
Vis
u
al
d
ata
wer
e
o
b
tain
ed
f
r
o
m
I
n
s
tag
r
am
,
co
m
p
r
is
in
g
3
,
0
5
8
im
ag
es
with
h
ig
h
s
em
an
tic
a
n
d
v
is
u
al
v
ar
iab
ilit
y
.
Vid
eo
d
ata
wer
e
c
o
llected
f
r
o
m
T
i
k
T
o
k
,
c
o
n
s
is
tin
g
o
f
1
,
1
3
8
v
id
eo
s
co
n
tain
in
g
c
o
m
p
lex
m
u
ltimo
d
al
in
f
o
r
m
atio
n
,
in
clu
d
in
g
v
is
u
al,
tem
p
o
r
al,
a
n
d
im
p
licit tex
t
u
al
cu
es.
Alth
o
u
g
h
t
h
e
d
ata
wer
e
co
llec
ted
f
r
o
m
d
if
f
e
r
en
t
p
latf
o
r
m
s
w
ith
v
ar
y
in
g
d
ata
v
o
lu
m
es,
th
e
n
u
m
b
er
o
f
s
am
p
les
u
s
ed
f
o
r
tr
ain
in
g
was
s
tan
d
ar
d
ized
to
1
,
1
3
8
in
s
tan
ce
s
p
er
m
o
d
ality
.
T
h
is
s
tan
d
ar
d
i
za
tio
n
was
ap
p
lied
to
m
itig
ate
class
im
b
alan
ce
an
d
p
r
ev
e
n
t
b
ias
ar
is
in
g
f
r
o
m
u
n
eq
u
al
d
ata
d
is
tr
ib
u
tio
n
ac
r
o
s
s
m
o
d
alities
.
B
y
b
alan
cin
g
th
e
d
ataset
s
ize,
ea
ch
m
o
d
ality
co
n
tr
i
b
u
tes
p
r
o
p
o
r
tio
n
ally
d
u
r
in
g
m
u
ltimo
d
al
lear
n
in
g
,
en
a
b
lin
g
a
f
air
er
ev
alu
atio
n
o
f
m
o
d
ality
f
u
s
io
n
ef
f
ec
tiv
en
ess
.
I
n
ad
d
itio
n
to
th
e
cr
awle
d
d
ataset,
th
is
s
tu
d
y
em
p
lo
y
s
p
u
b
licly
av
ailab
le
b
e
n
ch
m
ar
k
d
atasets
to
ev
alu
ate
cr
o
s
s
-
d
o
m
ain
g
en
er
a
lizatio
n
.
T
h
e
jig
s
aw
to
x
ic
co
m
m
en
t
d
atas
et
[
3
7
]
is
u
s
ed
f
o
r
tex
t
-
b
ased
to
x
icity
ev
alu
atio
n
,
w
h
ile
th
e
h
atef
u
l
m
em
es
d
atase
t
[
3
8
]
is
u
tili
ze
d
to
ass
ess
m
u
ltimo
d
al
p
er
f
o
r
m
an
ce
in
v
o
lv
in
g
tex
t
an
d
im
ag
e
m
o
d
alities
.
T
h
e
in
clu
s
io
n
o
f
th
ese
b
en
c
h
m
ar
k
s
en
s
u
r
es
th
at
th
e
p
r
o
p
o
s
ed
m
o
d
el
is
ev
alu
ated
n
o
t
o
n
ly
o
n
in
ter
n
al
d
ata
b
u
t a
ls
o
ag
ain
s
t w
id
ely
ad
o
p
te
d
s
tan
d
a
r
d
d
atasets
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
WVisi
o
n
B
E
R
T
-
V
L:
a
mu
ltimo
d
a
l m
o
d
el
a
r
ch
itectu
r
e
fo
r
to
xi
city
cla
s
s
ifica
tio
n
…
(
Wit
ta
Li
s
tiya
N
in
g
r
u
m
)
3367
2
.
2
.
Da
t
a
prepro
ce
s
s
ing
a
nd
la
belin
g
Prio
r
to
m
o
d
el
in
p
u
t,
all
d
ata
u
n
d
er
g
o
es
p
r
ep
r
o
ce
s
s
in
g
to
en
h
an
ce
q
u
ality
an
d
c
o
n
s
is
ten
cy
ac
r
o
s
s
m
o
d
alities
.
T
ex
tu
al
p
r
ep
r
o
ce
s
s
in
g
in
clu
d
es
n
o
is
e
r
em
o
v
al,
s
u
ch
as
u
n
if
o
r
m
r
eso
u
r
c
e
lo
ca
to
r
(
UR
L
s
)
,
ex
ce
s
s
iv
e
em
o
jis
,
an
d
n
o
n
-
i
n
f
o
r
m
ativ
e
s
y
m
b
o
ls
,
f
o
llo
we
d
b
y
ca
s
e
n
o
r
m
aliza
tio
n
to
r
ed
u
ce
lin
g
u
is
tic
am
b
ig
u
ity
an
d
im
p
r
o
v
e
r
e
p
r
esen
tatio
n
s
tab
ilit
y
.
Fo
r
im
ag
e
d
ata,
r
esizin
g
an
d
p
ix
el
n
o
r
m
al
izatio
n
ar
e
ap
p
lie
d
to
m
ee
t
t
h
e
in
p
u
t
r
eq
u
ir
em
en
t
s
o
f
t
h
e
v
is
u
al
en
co
d
er
,
en
s
u
r
i
n
g
u
n
if
o
r
m
d
im
en
s
io
n
s
a
n
d
m
in
im
izin
g
ir
r
elev
an
t
v
is
u
al
v
ar
iatio
n
s
.
Fo
r
v
id
eo
d
a
ta,
a
f
r
am
e
s
am
p
lin
g
s
tr
ateg
y
is
em
p
lo
y
e
d
to
ex
tr
ac
t
r
e
p
r
esen
tativ
e
f
r
am
es f
r
o
m
ea
ch
v
id
eo
,
p
r
eser
v
i
n
g
ess
en
tial te
m
p
o
r
al
in
f
o
r
m
atio
n
wh
ile
m
ain
tain
in
g
co
m
p
u
tatio
n
al
e
f
f
icien
cy
.
T
o
en
s
u
r
e
d
ataset
r
eliab
ilit
y
an
d
m
itig
ate
an
n
o
tato
r
b
ias,
t
h
is
s
tu
d
y
a
d
o
p
ts
a
h
y
b
r
id
m
o
d
el
-
b
ased
lab
elin
g
s
tr
ateg
y
in
co
r
p
o
r
atin
g
h
u
m
an
-
in
-
th
e
-
lo
o
p
v
alid
atio
n
.
Do
m
ain
-
s
p
ec
if
ic
p
r
etr
ain
ed
m
o
d
els
f
o
r
tex
t,
im
ag
e,
an
d
v
i
d
eo
m
o
d
alities
a
r
e
f
ir
s
t
u
s
ed
to
g
en
er
ate
p
s
eu
d
o
-
lab
els,
an
d
cr
o
s
s
-
m
o
d
al
co
n
s
is
ten
cy
is
an
aly
ze
d
to
id
en
tify
d
is
cr
ep
a
n
cies.
Hu
m
an
v
alid
atio
n
is
th
e
n
ap
p
lied
to
a
p
p
r
o
x
im
ately
3
0
%
o
f
t
h
e
s
am
p
les
to
v
e
r
if
y
to
x
icity
lab
els
an
d
g
en
er
ate
im
ag
e
an
d
v
i
d
eo
d
escr
ip
tio
n
s
,
f
o
cu
s
in
g
o
n
lo
w
-
c
o
n
f
id
e
n
c
e
an
d
cr
o
s
s
-
m
o
d
al
d
is
ag
r
ee
m
en
t c
ases
to
en
s
u
r
e
a
n
n
o
tatio
n
q
u
ality
wh
ile
m
ain
ta
in
in
g
s
ca
lab
ilit
y
.
2
.
3
.
P
r
o
po
s
ed
m
o
del
:
WVis
io
nB
E
R
T
-
VL
T
h
i
s
s
t
u
d
y
p
r
o
p
o
s
e
s
W
V
i
s
i
o
n
B
E
R
T
-
VL
,
a
v
i
s
u
al
l
y
o
r
i
e
n
te
d
m
u
l
t
i
m
o
d
a
l
a
r
c
h
i
t
ec
t
u
r
e
f
o
r
t
o
x
i
c
i
t
y
d
e
t
e
c
t
i
o
n
i
n
s
o
c
ia
l
m
e
d
i
a
c
o
n
te
n
t
t
h
a
t
i
n
t
e
g
r
a
t
es
t
e
x
t
,
i
m
a
g
e
,
a
n
d
v
i
d
e
o
m
o
d
a
l
i
ti
e
s
.
T
h
e
p
r
o
p
o
s
e
d
f
r
a
m
e
w
o
r
k
is
d
e
s
i
g
n
e
d
t
o
c
a
p
t
u
r
e
c
o
m
p
l
em
e
n
t
a
r
y
i
n
f
o
r
m
a
t
i
o
n
a
c
r
o
s
s
m
o
d
a
l
i
t
i
es
w
h
i
l
e
r
e
m
ai
n
i
n
g
r
o
b
u
s
t
t
o
n
o
i
s
y
o
r
i
n
c
o
m
p
l
e
t
e
s
i
g
n
a
l
s
.
T
o
t
h
is
e
n
d
,
W
V
i
s
i
o
n
B
E
R
T
-
VL
c
o
m
p
r
i
s
es
f
o
u
r
m
a
i
n
c
o
m
p
o
n
e
n
t
s
:
i)
m
o
d
a
l
i
t
y
-
s
p
e
ci
f
i
c
e
n
c
o
d
e
r
s
,
ii)
c
r
o
s
s
-
m
o
d
a
l
s
y
n
c
h
r
o
n
i
z
a
t
i
o
n
v
i
a
b
i
d
i
r
e
c
t
i
o
n
a
l
m
u
l
t
i
-
h
e
a
d
c
r
o
s
s
a
t
t
e
n
t
i
o
n
(
B
M
HC
A
)
,
i
ii
)
a
n
a
d
a
p
t
i
v
e
f
u
s
i
o
n
g
a
t
e
,
a
n
d
i
v
)
a
t
o
x
i
c
i
t
y
cl
as
s
i
f
i
c
a
ti
o
n
m
o
d
u
l
e
.
T
h
e
o
v
e
r
a
l
l
a
r
c
h
i
t
e
ct
u
r
e
o
f
t
h
e
p
r
o
p
o
s
e
d
i
s
s
h
o
w
n
i
n
F
i
g
u
r
e
1
,
i
l
l
u
s
t
r
at
i
n
g
i
n
t
e
r
a
c
ti
o
n
b
e
t
w
e
e
n
m
o
d
a
l
i
t
y
-
s
p
e
ci
f
i
c
r
e
p
r
e
s
e
n
t
a
ti
o
n
s
a
n
d
t
h
e
s
u
b
s
e
q
u
e
n
t
f
u
s
i
o
n
p
r
o
c
e
s
s
.
T
h
i
s
m
o
d
u
l
a
r
a
r
c
h
i
t
ec
t
u
r
e
e
n
ab
l
e
s
e
f
f
i
c
i
e
n
t
c
r
o
s
s
-
m
o
d
a
l
a
l
i
g
n
m
e
n
t
w
h
i
le
m
a
i
n
t
a
i
n
i
n
g
t
h
e
u
n
i
q
u
e
c
h
a
r
a
c
t
e
r
i
s
ti
cs
o
f
e
a
c
h
m
o
d
a
l
i
t
y
,
t
h
e
r
e
b
y
c
r
e
a
ti
n
g
a
n
i
n
t
e
g
r
a
t
e
d
e
n
d
-
to
-
e
n
d
f
r
a
m
e
w
o
r
k
f
o
r
c
o
n
t
e
x
t
u
a
l
t
o
x
i
c
ity
d
e
t
e
c
t
i
o
n
.
Fig
u
r
e
1
.
Ov
e
r
v
iew
o
f
th
e
p
r
o
p
o
s
ed
W
Vis
io
n
B
E
R
T
-
VL
m
u
l
tim
o
d
al
ar
ch
itectu
r
e
2
.
3
.
1
.
E
nco
der
m
o
du
les
T
h
e
en
c
o
d
er
m
o
d
u
le
is
d
esig
n
ed
to
ex
tr
ac
t
m
o
d
ality
-
s
p
ec
i
f
ic
r
ep
r
esen
tatio
n
s
f
r
o
m
tex
tu
al,
v
is
u
al,
an
d
v
id
eo
in
p
u
ts
.
E
ac
h
m
o
d
a
lity
is
p
r
o
ce
s
s
ed
u
s
in
g
a
d
e
d
icate
d
p
r
etr
ai
n
ed
e
n
co
d
e
r
t
o
ca
p
tu
r
e
its
u
n
iq
u
e
s
em
an
tic
ch
ar
ac
ter
is
tics
wh
i
le
m
ain
tain
in
g
co
m
p
ati
b
ilit
y
f
o
r
cr
o
s
s
-
m
o
d
al
in
ter
ac
ti
o
n
.
T
h
e
r
esu
ltin
g
r
ep
r
esen
tatio
n
s
ar
e
s
u
b
s
eq
u
en
t
ly
p
r
o
jecte
d
i
n
to
a
s
h
ar
e
d
late
n
t sp
ac
e
to
f
ac
ilit
ate
ef
f
ec
tiv
e
m
u
ltimo
d
al
f
u
s
io
n
.
i)
T
ex
t
en
co
d
er
:
te
x
tu
al
in
p
u
ts
ar
e
en
co
d
ed
u
s
in
g
I
n
d
o
B
E
R
T
[
3
9
]
,
w
h
ich
is
p
r
etr
ain
ed
o
n
lar
g
e
-
s
ca
le
I
n
d
o
n
esian
co
r
p
o
r
a
to
ca
p
tu
r
e
co
n
tex
tu
al
s
em
an
tics
an
d
in
f
o
r
m
al
lan
g
u
a
g
e
co
m
m
o
n
ly
f
o
u
n
d
in
s
o
cial
m
ed
ia.
T
h
is
en
co
d
er
en
a
b
le
s
r
o
b
u
s
t
r
ep
r
esen
tatio
n
o
f
li
n
g
u
is
tic
v
ar
iatio
n
s
,
s
lan
g
,
a
n
d
s
h
o
r
t
-
f
o
r
m
ex
p
r
ess
io
n
s
ty
p
ical
o
f
o
n
lin
e
d
is
co
u
r
s
e.
T
h
e
r
esu
ltin
g
c
o
n
tex
tu
al
em
b
ed
d
in
g
s
s
er
v
e
a
s
th
e
p
r
im
a
r
y
s
em
an
tic
b
ac
k
b
o
n
e
f
o
r
to
x
icit
y
d
etec
tio
n
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
3
6
5
-
3
3
7
5
3368
ii)
I
m
ag
e
e
n
co
d
e
r
:
v
is
u
al
f
ea
tu
r
es
ar
e
ex
t
r
ac
ted
u
s
in
g
a
lar
g
e
lan
g
u
ag
e
an
d
v
is
io
n
ass
is
t
an
t
(
L
L
aVA
)
-
On
eVisio
n
,
wh
ich
p
r
o
d
u
ce
s
l
an
g
u
ag
e
-
alig
n
ed
v
is
u
al
em
b
e
d
d
in
g
s
an
d
allo
ws
s
tr
o
n
g
tr
a
n
s
f
er
lear
n
in
g
ac
r
o
s
s
d
if
f
er
en
t
m
o
d
alities
[
4
0
]
.
T
h
is
d
esig
n
allo
ws
v
is
u
al
co
n
ten
t
to
b
e
r
ep
r
esen
te
d
in
a
s
em
an
tically
co
m
p
atib
le
s
p
ac
e
with
tex
tu
a
l
em
b
ed
d
in
g
s
,
th
er
eb
y
f
ac
ilit
atin
g
ef
f
ec
tiv
e
cr
o
s
s
-
m
o
d
al
a
lig
n
m
en
t.
B
y
lev
er
ag
in
g
a
v
is
io
n
–
lan
g
u
ag
e
p
r
etr
ain
ed
m
o
d
el
,
th
e
im
ag
e
e
n
co
d
er
ca
p
tu
r
es
h
ig
h
-
lev
el
v
is
u
al
s
em
an
tics
th
at
ar
e
r
elev
an
t t
o
co
n
te
x
tu
al
to
x
icity
cu
es
.
iii)
Vid
eo
en
co
d
er
:
v
i
d
eo
in
p
u
ts
ar
e
p
r
o
ce
s
s
ed
u
s
in
g
a
f
r
am
e
-
b
ased
s
am
p
lin
g
s
tr
ateg
y
,
in
wh
ich
s
elec
ted
f
r
am
es
ar
e
t
r
an
s
f
o
r
m
e
d
in
t
o
lan
g
u
a
g
e
-
o
r
ie
n
ted
r
ep
r
esen
t
atio
n
s
u
s
in
g
Qwe
n
-
2
[
4
1
]
.
T
h
is
ap
p
r
o
ac
h
ca
p
tu
r
es
v
is
u
al
ap
p
ea
r
an
ce
an
d
im
p
licit
tem
p
o
r
al
cu
es
wh
ile
m
ain
tain
in
g
co
m
p
u
tatio
n
al
e
f
f
icien
cy
.
T
h
e
s
am
p
led
f
r
am
e
r
ep
r
esen
tatio
n
s
ar
e
ag
g
r
eg
ated
to
f
o
r
m
a
co
m
p
ac
t
v
id
eo
em
b
e
d
d
in
g
th
at
p
r
eser
v
es
ess
en
tial te
m
p
o
r
al
in
f
o
r
m
atio
n
.
All
en
co
d
er
o
u
tp
u
ts
ar
e
p
r
o
je
cted
in
to
a
u
n
if
ied
laten
t
s
p
ac
e
with
co
n
s
is
ten
t
d
im
en
s
io
n
ality
,
en
s
u
r
in
g
th
at
r
ep
r
esen
tatio
n
s
f
r
o
m
d
if
f
er
e
n
t
m
o
d
alities
ca
n
in
ter
ac
t
e
f
f
ec
tiv
ely
d
u
r
i
n
g
cr
o
s
s
-
m
o
d
al
s
y
n
ch
r
o
n
izatio
n
.
C
o
n
s
eq
u
en
tly
,
th
e
en
co
d
er
m
o
d
u
le
p
r
o
v
id
es
a
b
alan
ce
d
an
d
m
o
d
ality
-
awa
r
e
f
o
u
n
d
a
tio
n
f
o
r
s
u
b
s
eq
u
e
n
t
m
u
ltimo
d
al
p
r
o
ce
s
s
in
g
.
2
.
3
.
2
.
Cro
s
s
-
m
o
da
l sy
nchro
niza
t
io
n
us
ing
bid
irec
t
io
na
l m
ulti
-
hea
d c
ro
s
s
a
t
t
ent
io
n
T
o
alig
n
in
f
o
r
m
atio
n
ac
r
o
s
s
m
o
d
alities
,
th
e
p
r
o
p
o
s
ed
m
o
d
el
,
B
MH
C
A
as
th
e
co
r
e
m
u
ltimo
d
al
s
y
n
ch
r
o
n
izatio
n
m
ec
h
a
n
is
m
.
Un
lik
e
u
n
id
ir
ec
tio
n
al
cr
o
s
s
-
atten
tio
n
,
B
MH
C
A
en
ab
les
ea
ch
m
o
d
ality
t
o
alter
n
ately
f
u
n
ctio
n
as
q
u
er
y
,
k
ey
,
an
d
v
alu
e,
f
ac
ilit
atin
g
b
id
ir
ec
tio
n
al
i
n
f
o
r
m
atio
n
ex
c
h
an
g
e.
T
h
is
d
esig
n
ca
p
tu
r
es
co
m
p
lex
s
em
an
tic
d
e
p
en
d
en
cies
ac
r
o
s
s
m
o
d
alities
,
s
u
ch
as
n
eu
tr
al
tex
tu
al
e
x
p
r
e
s
s
io
n
s
th
at
b
ec
o
m
e
to
x
ic
wh
en
in
ter
p
r
eted
i
n
co
n
j
u
n
ctio
n
with
s
p
ec
if
ic
v
is
u
al
co
n
tex
ts
.
Alg
o
r
ith
m
1
d
escr
ib
es
th
e
B
MH
C
A
m
ec
h
an
is
m
u
s
ed
to
s
y
n
ch
r
o
n
ize
r
e
p
r
esen
tatio
n
s
a
cr
o
s
s
tex
t,
im
ag
e,
an
d
v
id
e
o
m
o
d
alitie
s
.
E
ac
h
m
o
d
ality
alter
n
ately
atten
d
s
to
th
e
o
th
er
s
,
en
ab
lin
g
b
id
ir
ec
tio
n
al
in
f
o
r
m
atio
n
e
x
ch
an
g
e
a
n
d
ca
p
tu
r
in
g
im
p
licit
c
r
o
s
s
-
m
o
d
al
d
ep
en
d
e
n
cies.
T
h
is
d
esig
n
al
lo
ws
th
e
m
o
d
el
t
o
b
etter
alig
n
s
em
an
tic
cu
es a
cr
o
s
s
m
o
d
alities
b
ef
o
r
e
th
e
f
u
s
io
n
s
tag
e.
Alg
o
r
ith
m
1
.
B
id
ir
ec
tio
n
al
m
u
lti
-
h
ea
d
cr
o
s
s
atten
tio
n
I
nput
:
T = text embeddings
I = image embeddings
V = video embeddings
Output
: T_att, I_att, V_att = synchronized multimodalrepresentations
1:
// Text attends to Image and Video
2:
T_I ← MultiHeadAttention (Q=T, K=I, V=I)
3:
T_V ← MultiHeadAttention (Q=T, K=V, V=V)
4:
T_att ← T_I + T_V
5:
// Image attends to Text and Video
6:
I_T ← MultiHeadAttention (Q=I, K=T, V=T)
7:
I_V ← MultiHeadAttention (Q=I, K=V, V=V)
8:
I_att ← I_T + I_V
9:
// Video attends to Text and Image
10:
V_T ← MultiHeadAttention (Q=V, K=T, V=T)
11:
V_I ← MultiHeadAttention (Q=V, K=I, V=I)
12:
V_att ← V_T + V_I
13:
return T_att, I_att, V_att
2
.
3
.
3
.
Ada
ptiv
e
f
us
io
n g
a
t
e
Fo
llo
win
g
cr
o
s
s
-
m
o
d
al
s
y
n
c
h
r
o
n
izatio
n
,
m
u
ltimo
d
al
r
ep
r
esen
tatio
n
s
ar
e
i
n
teg
r
ated
u
s
in
g
a
n
ad
a
p
tiv
e
f
u
s
io
n
g
ate
th
at
d
y
n
am
ically
b
alan
ce
s
m
o
d
ality
co
n
tr
ib
u
tio
n
s
f
o
r
to
x
icity
p
r
ed
ictio
n
.
R
esid
u
al
co
n
n
ec
tio
n
s
,
lay
er
n
o
r
m
aliza
tio
n
,
an
d
d
r
o
p
o
u
t
ar
e
ap
p
lied
to
p
r
eser
v
e
m
o
d
ality
-
s
p
ec
if
ic
in
f
o
r
m
atio
n
an
d
im
p
r
o
v
e
r
o
b
u
s
tn
ess
.
T
h
e
c
o
n
ca
ten
ate
d
m
u
ltimo
d
al
r
e
p
r
esen
tatio
n
s
ar
e
s
u
b
s
eq
u
en
tly
f
ed
in
to
f
u
lly
c
o
n
n
ec
ted
lay
er
s
f
o
r
f
in
al
em
b
ed
d
in
g
g
en
er
atio
n
.
Alg
o
r
ith
m
2
d
escr
ib
es
th
e
ad
a
p
tiv
e
f
u
s
io
n
g
ate
em
p
lo
y
ed
to
in
teg
r
ate
th
e
s
y
n
ch
r
o
n
ized
m
u
ltimo
d
al
r
ep
r
esen
tatio
n
s
f
o
r
t
o
x
icity
c
lass
if
icatio
n
.
R
es
id
u
al
co
n
n
e
ctio
n
s
an
d
n
o
r
m
aliza
tio
n
lay
er
s
ar
e
ap
p
lied
to
p
r
eser
v
e
m
o
d
ality
-
s
p
ec
if
ic
i
n
f
o
r
m
atio
n
an
d
s
tab
ilize
th
e
tr
ain
in
g
p
r
o
ce
s
s
,
wh
ile
d
r
o
p
o
u
t
is
u
s
ed
as
a
r
eg
u
lar
izatio
n
m
ec
h
a
n
is
m
.
T
h
e
r
esu
ltin
g
f
u
s
ed
r
ep
r
esen
tat
io
n
is
s
u
b
s
eq
u
e
n
tly
p
ass
ed
to
a
f
u
lly
co
n
n
ec
ted
lay
er
to
p
r
o
d
u
ce
to
x
icity
class
p
r
o
b
a
b
ilit
ies.
Alg
o
r
ith
m
2
.
F
u
s
io
n
g
ate
f
o
r
t
o
x
icity
class
if
icatio
n
I
nput
: Synchronized multimodal representations
T = text embeddings
I = image embeddings
V = video embeddings
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
WVisi
o
n
B
E
R
T
-
V
L:
a
mu
ltimo
d
a
l m
o
d
el
a
r
ch
itectu
r
e
fo
r
to
xi
city
cla
s
s
ifica
tio
n
…
(
Wit
ta
Li
s
tiya
N
in
g
r
u
m
)
3369
O
utput
: y = Fused multimodal representations
1:
// Cross
-
modal synchronization
2:
T_att, I_att, V_att ← BMHCA (T,
I,
V)
3:
// Residual connections
4:
T_ress ← T_att + T
5:
I_ress ← I_att + I
6:
V_ress ← V_att + V
7:
// Layer normalization
8:
T_norm ← Dropout(T_norm)
9:
I_norm ← Dropout(I_norm)
10:
V_norm ← Dropout(V_norm)
11:
// Dropout regularization
12:
T_drop ← Dropout (T_norm)
13:
I_drop ← Dropout (I_norm)
14:
V_drop ← Dropout (V_norm)
15:
// Feature fusion
16:
F ← Concatenate (T_drop, I_drop, V_drop)
17:
// Classification
18:
logits ← FullyConnected(F)
19:
y ← Softmax(logits)
20:
return y
2
.
3
.
4
.
M
ultit
a
s
k
lea
rning
o
bj
ec
t
iv
e
T
h
e
f
u
s
ed
m
u
ltimo
d
al
r
ep
r
es
en
tatio
n
is
p
r
o
jecte
d
in
to
t
h
e
o
u
tp
u
t
s
p
ac
e
th
r
o
u
g
h
a
f
u
lly
co
n
n
ec
ted
class
if
icatio
n
lay
er
,
en
ab
li
n
g
to
x
icity
class
if
icatio
n
in
to
t
h
r
ee
ca
teg
o
r
ies:
to
x
ic,
n
o
n
-
to
x
i
c,
an
d
n
eu
t
r
al.
T
h
e
en
tire
f
r
am
ewo
r
k
is
tr
ain
ed
en
d
-
to
-
e
n
d
u
s
in
g
a
cr
o
s
s
-
en
t
r
o
p
y
l
o
s
s
f
u
n
ctio
n
.
I
n
ad
d
itio
n
to
th
e
p
r
im
ar
y
class
if
icatio
n
o
b
jectiv
e,
th
e
m
o
d
el
in
co
r
p
o
r
ates
a
m
u
ltit
ask
lear
n
in
g
co
m
p
o
n
e
n
t
th
at
g
en
er
ates
tex
tu
al
d
escr
ip
tio
n
s
f
o
r
im
ag
e
an
d
v
id
eo
in
p
u
ts
as
an
a
u
x
iliar
y
c
ap
tio
n
in
g
task
.
T
h
is
ca
p
tio
n
i
n
g
o
u
tp
u
t
p
r
o
v
id
es
co
m
p
lem
en
tar
y
s
em
an
tic
in
f
o
r
m
atio
n
th
at
en
r
ich
es
m
u
ltimo
d
al
r
ep
r
esen
tatio
n
s
an
d
f
ac
ilit
ates
m
o
r
e
ef
f
ec
tiv
e
cr
o
s
s
-
m
o
d
al
f
u
s
io
n
,
r
ath
er
th
a
n
s
er
v
in
g
as a
s
tan
d
al
o
n
e
g
e
n
e
r
ativ
e
o
b
jectiv
e.
2
.
4
.
T
ra
ini
ng
a
nd
ev
a
lua
t
io
n str
a
t
eg
y
W
Vis
io
n
B
E
R
T
-
VL
i
s
tr
ain
ed
o
n
th
e
in
ter
n
ally
cr
awle
d
d
a
taset.
Mo
d
el
p
er
f
o
r
m
an
ce
is
ev
alu
ated
u
n
d
er
two
p
r
im
ar
y
s
ce
n
a
r
io
s
:
in
-
d
o
m
ain
e
v
alu
atio
n
u
s
in
g
th
e
cu
s
to
m
d
ataset
an
d
cr
o
s
s
-
d
a
taset
ev
alu
atio
n
o
n
p
u
b
lic
b
e
n
ch
m
ar
k
d
atasets
(
ji
g
s
aw
an
d
h
ate
f
u
l
m
em
es
)
.
Per
f
o
r
m
a
n
ce
is
ass
ess
ed
u
s
in
g
ac
cu
r
ac
y
,
m
ac
r
o
-
F1
,
m
icr
o
-
F1
,
an
d
r
ec
eiv
e
r
o
p
e
r
atin
g
ch
ar
ac
ter
is
tic
-
ar
ea
u
n
d
er
th
e
c
u
r
v
e
(
R
OC
-
AU
C
)
,
co
m
p
lem
en
ted
b
y
p
er
-
class
an
aly
s
is
to
e
x
am
in
e
b
eh
av
io
r
u
n
d
er
class
im
b
alan
c
e.
An
ab
latio
n
s
tu
d
y
is
co
n
d
u
c
ted
to
q
u
a
n
tify
th
e
co
n
tr
ib
u
tio
n
o
f
ea
ch
m
o
d
ality
an
d
th
e
o
v
er
all
m
u
ltimo
d
al
f
r
am
ewo
r
k
.
C
r
o
s
s
-
d
ataset
ev
alu
atio
n
is
p
er
f
o
r
m
e
d
with
o
u
t r
etr
ain
in
g
to
ass
ess
g
en
er
aliza
tio
n
u
n
d
er
d
o
m
ain
a
n
d
lan
g
u
ag
e
s
h
if
ts
[
4
2
]
,
[
4
3
]
.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
ex
p
er
im
en
tal
r
esu
lts
an
d
d
is
cu
s
s
es
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
o
p
o
s
ed
m
u
ltimo
d
al
m
o
d
el
f
o
r
to
x
icity
d
etec
tio
n
b
ased
o
n
tex
t,
im
ag
es,
an
d
v
id
eo
s
.
T
h
e
ev
alu
atio
n
is
co
n
d
u
cted
o
n
a
cu
s
to
m
d
ataset
as
well
as
p
u
b
lic
b
en
ch
m
ar
k
d
atasets
to
ass
es
s
m
o
d
el
p
er
f
o
r
m
an
ce
,
g
en
e
r
aliza
tio
n
ab
ilit
y
,
an
d
r
o
b
u
s
tn
ess
.
I
n
ad
d
itio
n
,
ab
lat
io
n
s
tu
d
ies,
ca
p
tio
n
in
g
p
er
f
o
r
m
an
ce
e
v
alu
atio
n
,
an
d
q
u
ali
tativ
e
an
aly
s
is
ar
e
p
er
f
o
r
m
ed
to
p
r
o
v
id
e
a
c
o
m
p
r
eh
en
s
iv
e
u
n
d
er
s
tan
d
in
g
o
f
t
h
e
p
r
o
p
o
s
ed
ap
p
r
o
ac
h
.
3
.
1
.
E
x
perim
ent
a
l
s
et
up
T
o
en
s
u
r
e
co
n
s
is
ten
t
an
d
f
air
ev
alu
atio
n
ac
r
o
s
s
ex
p
er
im
en
t
s
,
id
en
tical
h
y
p
er
p
ar
am
eter
s
e
ttin
g
s
ar
e
ap
p
lied
th
r
o
u
g
h
o
u
t
th
e
tr
ain
in
g
p
r
o
ce
s
s
.
T
h
e
m
o
d
el
is
tr
ain
ed
f
o
r
1
0
e
p
o
ch
s
u
s
in
g
a
b
at
ch
s
ize
o
f
4
a
n
d
a
lear
n
in
g
r
ate
of
3
×
10
-
4
,
wh
ile
co
n
s
id
er
in
g
th
e
co
m
p
u
tatio
n
al
lim
itatio
n
s
o
f
th
e
av
ailab
le
A1
0
0
GPU.
Fo
r
ev
alu
atio
n
,
th
e
d
ataset
is
d
iv
id
ed
in
to
7
0
%
tr
ain
in
g
d
ata,
1
5
%
v
alid
atio
n
d
ata,
a
n
d
1
5
%
test
in
g
d
ata.
T
o
im
p
r
o
v
e
th
e
r
eliab
ilit
y
o
f
th
e
ev
alu
atio
n
an
d
r
ed
u
ce
b
ias
in
tr
o
d
u
ce
d
b
y
a
s
in
g
le
d
ata
s
p
lit,
a
5
-
f
o
ld
c
r
o
s
s
-
v
alid
atio
n
s
tr
ateg
y
is
ap
p
lied
to
th
e
cu
s
to
m
d
ataset.
I
n
ea
ch
f
o
ld
,
th
e
d
ataset
is
p
ar
titi
o
n
ed
in
to
f
iv
e
s
u
b
s
ets
with
b
alan
ce
d
class
d
is
tr
ib
u
tio
n
,
wh
er
e
o
n
e
s
u
b
s
et
is
u
s
ed
f
o
r
test
in
g
,
an
d
th
e
r
em
ain
i
n
g
s
u
b
s
ets
ar
e
u
s
ed
f
o
r
tr
ain
in
g
an
d
v
alid
atio
n
.
T
h
is
p
r
o
to
co
l a
llo
ws th
e
ass
ess
m
en
t o
f
m
o
d
el
s
tab
ilit
y
u
n
d
er
d
if
f
e
r
en
t d
ata
p
ar
titi
o
n
s
.
3
.
2
.
P
er
f
o
r
m
a
nce
o
n
cus
t
o
m da
t
a
s
et
Mo
d
el
p
e
r
f
o
r
m
an
ce
o
n
th
e
cu
s
to
m
d
ataset
is
ev
alu
ate
d
u
s
in
g
s
tan
d
ar
d
class
if
icatio
n
m
etr
ics,
in
clu
d
in
g
ac
cu
r
ac
y
,
m
ac
r
o
-
F1
,
m
ic
ro
-
F1
,
a
n
d
R
OC
-
A
UC
.
T
h
ese
m
etr
ics
ar
e
c
h
o
s
en
to
p
r
o
v
id
e
co
m
p
r
eh
e
n
s
iv
e
ass
ess
m
en
t
o
f
o
v
er
all
p
r
ed
ictiv
e
p
er
f
o
r
m
an
ce
wh
ile
ac
co
u
n
tin
g
f
o
r
p
o
ten
tial
class
im
b
alan
ce
[
4
4
]
.
T
h
e
e
v
alu
atio
n
r
esu
lts
ar
e
r
ep
o
r
ted
at
b
o
t
h
th
e
o
v
er
all
a
n
d
p
er
-
class
lev
el
s
to
o
f
f
er
d
etailed
in
s
ig
h
ts
in
to
m
o
d
el
b
eh
av
io
r
ac
r
o
s
s
d
if
f
er
en
t
to
x
icity
ca
teg
o
r
ies.
T
h
is
m
u
lti
-
lev
el
ev
alu
atio
n
s
tr
ateg
y
en
ab
les
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
3
6
5
-
3
3
7
5
3370
a
m
o
r
e
r
eliab
le
an
aly
s
is
o
f
class
if
icatio
n
ef
f
ec
tiv
en
ess
,
p
ar
ticu
lar
ly
in
d
is
tin
g
u
is
h
i
n
g
b
etwe
en
to
x
ic,
non
-
to
x
ic,
an
d
n
eu
t
r
al
co
n
ten
t.
3
.
2
.
1
.
O
v
er
a
ll
perf
o
rma
nce
m
et
rics
T
ab
le
1
r
ep
o
r
ts
th
e
o
v
er
all
p
e
r
f
o
r
m
a
n
ce
o
f
th
e
p
r
o
p
o
s
ed
W
Vis
io
n
B
E
R
T
-
VL
m
o
d
el
o
n
th
e
cu
s
to
m
m
u
ltimo
d
al
d
ataset.
T
h
e
m
o
d
el
ac
h
iev
es
an
ac
cu
r
ac
y
o
f
0
.
9
4
1
2
,
i
n
d
icatin
g
s
tr
o
n
g
class
if
icatio
n
ca
p
ab
ilit
y
ac
r
o
s
s
all
class
es.
T
h
e
clo
s
e
ag
r
ee
m
en
t
b
etwe
en
m
ac
r
o
-
F1
(
0
.
9
4
0
7
)
a
n
d
m
ic
ro
-
F1
(
0
.
9
4
1
2
)
s
u
g
g
ests
th
at
th
e
m
o
d
el
p
er
f
o
r
m
s
c
o
n
s
is
ten
tly
ac
r
o
s
s
b
o
th
m
ajo
r
ity
an
d
m
in
o
r
ity
class
es.
Fu
r
th
er
m
o
r
e,
t
h
e
h
ig
h
R
OC
-
AUC
s
co
r
e
o
f
0
.
9
7
2
1
d
e
m
o
n
s
tr
ates
th
e
m
o
d
el’
s
s
tr
o
n
g
d
is
cr
im
i
n
ativ
e
ab
ilit
y
in
s
ep
ar
atin
g
t
o
x
ic
an
d
n
o
n
-
to
x
ic
co
n
ten
t.
T
h
ese
r
esu
lts
in
d
ica
te
th
at
th
e
p
r
o
p
o
s
ed
m
u
ltimo
d
al
ar
c
h
itectu
r
e
ef
f
ec
tiv
ely
ca
p
tu
r
es
co
n
te
x
tu
al
in
f
o
r
m
atio
n
r
eq
u
ir
ed
f
o
r
r
eliab
le
to
x
icity
d
etec
tio
n
.
T
ab
le
1
.
Ov
e
r
all
p
er
f
o
r
m
an
ce
m
etr
ics o
f
th
e
p
r
o
p
o
s
ed
W
Vis
i
o
n
B
E
R
T
-
VL
m
o
d
el
o
n
th
e
cu
s
to
m
d
ataset
M
e
t
r
i
c
s
V
a
l
u
e
A
c
c
u
r
a
c
y
0
.
9
4
1
2
M
a
c
r
o
-
F1
0
.
9
4
0
7
M
i
c
r
o
-
F1
0
.
9
4
1
2
R
O
C
-
AUC
0
.
9
7
2
1
3
.
2
.
2
.
P
er
-
cla
s
s
perf
o
rma
nce
a
na
ly
s
is
T
ab
le
2
p
r
esen
ts
th
e
p
er
-
cla
s
s
p
er
f
o
r
m
an
ce
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
o
n
t
h
e
cu
s
to
m
d
a
taset.
T
h
e
non
-
to
x
ic
class
ac
h
iev
es
th
e
h
ig
h
est
p
r
ec
is
io
n
(
0
.
9
9
)
an
d
F1
-
s
co
r
e
(
0
.
9
6
)
,
in
d
icatin
g
r
eliab
le
id
en
tific
atio
n
o
f
b
en
ig
n
c
o
n
ten
t w
ith
m
in
im
al
f
alse p
o
s
itiv
es.
T
h
is
s
u
g
g
ests
t
h
at
th
e
m
o
d
el
is
ef
f
ec
tiv
e
in
a
v
o
id
in
g
u
n
n
ec
ess
ar
y
m
is
class
if
icatio
n
o
f
n
o
n
-
h
ar
m
f
u
l
co
n
ten
t
.
T
h
e
T
o
x
ic
cla
s
s
ex
h
ib
its
b
alan
ce
d
p
r
ec
is
io
n
(
0
.
9
5
)
an
d
r
ec
all
(
0
.
9
0
)
,
r
ef
lectin
g
t
h
e
m
o
d
el’
s
ab
ilit
y
to
d
etec
t
h
ar
m
f
u
l
c
o
n
te
n
t
wh
ile
co
n
tr
o
llin
g
f
alse
alar
m
s
.
Me
an
wh
ile,
th
e
Neu
tr
al
class
d
em
o
n
s
tr
ates
h
i
g
h
r
ec
all
(
0
.
9
9
)
,
in
d
icatin
g
ef
f
ec
tiv
e
co
v
er
ag
e
o
f
am
b
ig
u
o
u
s
s
am
p
les
th
at
d
o
n
o
t
clea
r
ly
f
all
in
to
to
x
ic
o
r
n
o
n
-
to
x
ic
ca
teg
o
r
ies.
T
h
e
m
ac
r
o
-
a
v
er
ag
e
F1
-
s
co
r
e
o
f
0
.
9
4
f
u
r
th
er
co
n
f
ir
m
s
s
tab
le
p
er
f
o
r
m
an
ce
ac
r
o
s
s
all
class
e
s
d
esp
ite
class
im
b
alan
ce
.
T
ab
le
2
.
Per
-
class
p
er
f
o
r
m
an
c
e
o
f
th
e
p
r
o
p
o
s
ed
W
Vis
io
n
B
E
R
T
-
VL
m
o
d
el
o
n
th
e
cu
s
to
m
d
ataset
C
l
a
s
s
P
r
e
c
i
s
i
o
n
R
e
c
a
l
l
F1
-
sc
o
r
e
To
x
i
c
0
.
9
5
0
.
9
0
0
.
9
3
N
o
n
-
t
o
x
i
c
0
.
9
9
0
.
9
2
0
.
9
6
N
e
u
t
r
a
l
0
.
8
8
0
.
9
9
0
.
9
4
M
a
c
r
o
a
v
g
0
.
9
4
0
.
9
4
0
.
9
4
3
.
3
.
Cro
s
s
-
v
a
lid
a
t
io
n r
esu
lt
s
T
o
en
h
an
ce
ev
alu
atio
n
r
eliab
ilit
y
,
a
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
[
4
5
]
is
co
n
d
u
cted
o
n
th
e
cu
s
to
m
d
ataset.
I
n
ea
ch
f
o
l
d
,
o
n
e
s
u
b
s
et
is
u
s
ed
f
o
r
test
in
g
w
h
ile
th
e
r
em
ai
n
in
g
s
u
b
s
ets
ar
e
u
s
ed
f
o
r
tr
ai
n
in
g
a
n
d
v
alid
atio
n
,
en
s
u
r
in
g
b
alan
ce
d
class
d
is
tr
ib
u
tio
n
ac
r
o
s
s
f
o
ld
s
[
4
6
]
.
T
h
is
e
v
alu
atio
n
s
tr
ateg
y
m
itig
ates
p
o
ten
tial
b
ias
ar
is
in
g
f
r
o
m
a
s
in
g
le
t
r
ain
–
test
s
p
lit.
As
s
h
o
wn
in
T
ab
le
3
,
th
e
m
o
d
el
d
em
o
n
s
tr
ates
co
n
s
is
ten
t
p
er
f
o
r
m
an
ce
ac
r
o
s
s
all
f
iv
e
f
o
ld
s
.
T
h
e
av
er
ag
e
ac
c
u
r
ac
y
r
ea
ch
es
0
.
9
6
0
6
with
a
s
tan
d
ar
d
d
ev
iatio
n
o
f
0
.
0
2
1
7
,
wh
ile
th
e
av
er
a
g
e
R
OC
-
AUC
i
s
0
.
9
8
4
1
with
a
s
tan
d
ar
d
d
ev
iatio
n
o
f
0
.
0
1
0
2
.
T
h
e
r
elativ
ely
s
m
all
s
tan
d
ar
d
d
ev
iatio
n
ac
r
o
s
s
m
etr
i
cs
in
d
icate
s
th
at
th
e
m
o
d
el
ex
h
i
b
its
s
tr
o
n
g
s
tab
ilit
y
an
d
r
o
b
u
s
tn
ess
u
n
d
er
d
i
f
f
er
en
t
d
ata
p
ar
titi
o
n
s
.
T
h
ese
r
esu
lts
v
alid
ate
th
e
r
eliab
ilit
y
o
f
th
e
r
ep
o
r
te
d
p
er
f
o
r
m
an
ce
a
n
d
s
u
p
p
o
r
t su
b
s
eq
u
en
t a
n
aly
s
es o
n
m
o
d
el
g
e
n
er
al
izatio
n
.
T
ab
le
3
.
Fiv
e
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
r
esu
lts
o
f
th
e
p
r
o
p
o
s
ed
W
Vis
io
n
B
E
R
T
-
VL
m
o
d
el
o
n
th
e
cu
s
to
m
d
ataset
F
o
l
d
A
c
c
u
r
a
c
y
M
a
c
r
o
-
F1
M
i
c
r
o
-
F1
R
O
C
-
AUC
F
o
l
d
1
0
.
9
3
5
0
.
9
4
0
7
0
.
9
4
1
2
0
.
9
7
2
1
F
o
l
d
2
0
.
9
4
0
.
9
4
4
8
0
.
9
4
6
3
0
.
9
7
4
9
F
o
l
d
3
0
.
9
6
8
0
.
9
6
9
4
0
.
9
7
1
7
0
.
9
8
6
2
F
o
l
d
4
0
.
9
8
0
.
9
8
1
3
0
.
9
8
2
1
0
.
9
9
3
4
F
o
l
d
5
0
.
9
8
0
.
9
7
9
8
0
.
9
8
0
9
0
.
9
9
4
1
M
e
a
n
±
S
t
d
0
.
9
6
0
6
±
0
.
0
2
1
7
0
.
9
6
3
2
±
0
.
0
1
9
2
0
.
9
6
4
4
±
0
.
0
1
9
3
0
.
9
8
4
1
4
±
0
.
0
1
0
2
3
.
4
.
Abla
t
io
n study
An
ab
latio
n
s
tu
d
y
was
co
n
d
u
c
ted
o
n
th
e
cu
s
to
m
d
ataset
to
a
n
aly
ze
th
e
s
p
ec
if
ic
c
o
n
tr
ib
u
tio
n
o
f
ea
c
h
m
o
d
ality
t
o
th
e
o
v
er
all
p
er
f
o
r
m
an
ce
,
as
s
u
m
m
ar
ized
[
4
7
]
in
T
ab
le
4
.
T
h
e
ev
alu
atio
n
p
r
o
g
r
ess
iv
ely
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
WVisi
o
n
B
E
R
T
-
V
L:
a
mu
ltimo
d
a
l m
o
d
el
a
r
ch
itectu
r
e
fo
r
to
xi
city
cla
s
s
ifica
tio
n
…
(
Wit
ta
Li
s
tiya
N
in
g
r
u
m
)
3371
in
co
r
p
o
r
ated
v
is
u
al
an
d
tem
p
o
r
al
in
f
o
r
m
atio
n
to
ass
ess
th
eir
im
p
ac
t
o
n
to
x
icity
d
etec
t
io
n
.
T
h
e
tex
t
-
o
n
ly
v
ar
ian
t
estab
lis
h
ed
a
s
o
lid
b
aselin
e
with
an
ac
cu
r
ac
y
o
f
0
.
9
0
an
d
a
m
ac
r
o
-
F1
o
f
0
.
8
7
.
T
h
e
in
teg
r
atio
n
o
f
s
tatic
im
ag
e
f
ea
tu
r
es
(
tex
t
+
im
a
g
e)
f
u
r
th
e
r
en
h
an
ce
d
p
er
f
o
r
m
a
n
c
e,
y
ield
in
g
th
e
h
ig
h
est
ac
cu
r
a
cy
o
f
0
.
9
7
,
w
h
ich
s
u
g
g
ests
th
at
v
is
u
al
co
n
tex
t c
o
m
p
lem
en
ts
tex
tu
al
s
em
an
tics
in
r
eso
lv
in
g
ex
p
licit to
x
icity
.
Ho
wev
er
,
th
e
f
u
ll
m
u
ltimo
d
al
m
o
d
el
(
W
Vis
io
n
B
E
R
T
-
VL
)
ac
h
iev
ed
th
e
h
i
g
h
est
m
ac
r
o
-
F
1
o
f
0
.
9
4
,
d
esp
ite
a
m
ar
g
in
al
d
ec
r
ea
s
e
i
n
ac
cu
r
ac
y
c
o
m
p
ar
e
d
to
th
e
b
im
o
d
al
b
aselin
e.
T
h
is
tr
ad
e
-
o
f
f
in
d
icate
s
th
at
th
e
in
clu
s
io
n
o
f
v
i
d
eo
m
o
d
ality
p
r
im
ar
ily
en
h
a
n
ce
s
th
e
r
ec
o
g
n
itio
n
o
f
m
in
o
r
ity
an
d
am
b
ig
u
o
u
s
class
e
s
,
th
er
eb
y
im
p
r
o
v
in
g
class
-
lev
el
b
alan
ce
an
d
s
af
ety
-
cr
itical
p
er
f
o
r
m
an
c
e,
r
ath
er
th
an
o
p
tim
izin
g
m
ajo
r
ity
-
class
ac
cu
r
ac
y
alo
n
e.
I
t
is
wo
r
th
n
o
tin
g
th
at
in
th
e
co
n
tex
t
o
f
to
x
icity
d
etec
tio
n
,
wh
er
e
d
atasets
ar
e
in
h
e
r
en
tly
im
b
alan
ce
d
,
th
e
F1
-
s
co
r
e
p
r
o
v
id
es
a
m
o
r
e
r
o
b
u
s
t
m
etr
ic
th
an
r
aw
ac
cu
r
ac
y
.
T
h
e
s
ig
n
if
ican
t
im
p
r
o
v
e
m
en
t
in
m
ac
r
o
-
F1
(
r
is
in
g
f
r
o
m
0
.
8
8
in
th
e
tex
t
+
im
ag
e
v
ar
ian
t
to
0
.
9
4
)
in
d
icat
es
th
at
tem
p
o
r
al
v
id
eo
f
ea
tu
r
e
s
en
ab
le
th
e
m
o
d
e
l
to
b
etter
g
en
er
alize
ac
r
o
s
s
m
i
n
o
r
ity
class
es
an
d
r
ed
u
ce
f
als
e
n
eg
ativ
es.
T
h
u
s
,
th
e
in
clu
s
io
n
o
f
v
id
eo
m
o
d
ality
p
r
io
r
itizes m
o
d
el
r
o
b
u
s
tn
ess
an
d
s
af
ety
in
r
ea
l
-
wo
r
ld
a
p
p
lica
tio
n
s
o
v
er
m
e
r
ely
o
p
tim
izin
g
g
lo
b
al
ac
cu
r
ac
y
.
T
ab
le
4
.
Ab
latio
n
s
tu
d
y
o
f
m
o
d
ality
co
n
tr
ib
u
tio
n
s
o
n
th
e
cu
s
to
m
d
ataset
M
o
d
a
l
i
t
y
A
c
c
u
r
a
c
y
M
a
c
r
o
-
F1
T
e
x
t
-
o
n
l
y
0
.
9
0
0
.
8
7
T
e
x
t
+
i
ma
g
e
0
.
9
7
0
.
8
8
T
e
x
t
+
i
ma
g
e
+
v
i
d
e
o
(
W
V
i
si
o
n
B
E
R
T
-
V
L)
0
.
9
4
1
2
0
.
9
4
0
7
3
.
5
.
Cro
s
s
-
da
t
a
s
et
e
v
a
lua
t
io
n a
nd
ro
bu
s
t
ne
s
s
T
o
ass
es
s
th
e
g
en
er
aliza
tio
n
ca
p
ab
ilit
y
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
b
ey
o
n
d
th
e
cu
s
to
m
d
ataset,
a
cr
o
s
s
-
d
ataset
ev
alu
atio
n
was
c
o
n
d
u
cte
d
o
n
p
u
b
licly
av
ailab
l
e
b
en
ch
m
ar
k
s
,
n
am
ely
jig
s
aw
an
d
h
atef
u
l
m
em
es
,
as
s
u
m
m
ar
ized
in
T
ab
le
5
.
I
n
th
is
ze
r
o
-
s
h
o
t
s
ettin
g
,
t
h
e
m
o
d
el
ex
h
ib
its
s
u
b
s
tan
tially
l
o
wer
p
er
f
o
r
m
a
n
ce
,
ac
h
iev
in
g
a
n
ac
cu
r
ac
y
o
f
0
.
4
0
(
ma
cr
o
-
F1
0
.
2
7
5
9
,
R
OC
-
AUC
0
.
4
4
6
9
)
o
n
th
e
jig
s
aw
d
ataset
an
d
a
n
ac
cu
r
ac
y
o
f
0
.
2
9
4
1
(
m
ac
ro
-
F1
0
.
1
7
2
8
)
o
n
th
e
h
atef
u
l
m
em
es
d
at
aset.
T
h
is
p
er
f
o
r
m
an
ce
d
eg
r
ad
atio
n
r
e
f
lects
a
p
r
o
n
o
u
n
ce
d
d
o
m
ain
s
h
if
t
in
la
n
g
u
ag
e
s
ty
le,
a
n
n
o
tatio
n
p
o
lic
y
,
an
d
class
d
is
tr
ib
u
tio
n
,
r
ath
e
r
th
an
ar
c
h
itectu
r
al
lim
itatio
n
s
.
No
tab
ly
,
s
in
ce
W
Vis
io
n
B
E
R
T
-
VL
em
p
lo
y
s
I
n
d
o
B
E
R
T
as
its
tex
tu
al
b
ac
k
b
o
n
e,
its
ze
r
o
-
s
h
o
t
tr
an
s
f
er
ab
ilit
y
to
E
n
g
lis
h
-
lan
g
u
ag
e
d
atasets
is
in
h
er
en
tly
co
n
s
tr
ain
ed
.
T
h
ese
f
in
d
i
n
g
s
h
ig
h
lig
h
t
th
e
m
o
d
el’
s
s
tr
o
n
g
s
p
ec
ializatio
n
in
th
e
I
n
d
o
n
esian
lin
g
u
is
tic
co
n
te
x
t
an
d
u
n
d
e
r
s
co
r
e
th
e
b
r
o
a
d
er
ch
allen
g
es
o
f
cr
o
s
s
-
lin
g
u
al
m
u
ltimo
d
al
tr
an
s
f
er
with
o
u
t ta
r
g
eted
f
in
e
-
tu
n
in
g
o
r
d
o
m
ain
-
ad
ap
tiv
e
tr
ai
n
in
g
s
tr
ateg
ies.
T
ab
le
5
.
C
r
o
s
s
-
d
ataset
ev
alu
at
io
n
r
esu
lts
o
n
th
e
p
u
b
lic
b
en
ch
m
ar
k
d
ataset
D
a
t
a
s
e
t
M
o
d
a
l
i
t
y
A
c
c
u
r
a
c
y
M
a
c
r
o
-
F1
R
O
C
-
AUC
Ji
g
s
a
w
Te
x
t
0
.
4
0
0
0
0
.
2
7
5
9
0
.
4
4
6
9
H
a
t
e
f
u
l
m
e
m
e
s
Te
x
t
+
i
ma
g
e
0
.
2
9
4
1
0
.
1
7
2
8
-
3
.
6
.
Ca
pti
o
nin
g
perf
o
rm
a
nc
e
ev
a
lua
t
io
n
T
h
is
s
u
b
s
ec
tio
n
ev
al
u
ates
th
e
ca
p
tio
n
in
g
co
m
p
o
n
e
n
t
o
f
th
e
p
r
o
p
o
s
ed
f
r
am
ewo
r
k
,
wh
ich
f
u
n
ctio
n
s
as
an
au
x
iliar
y
task
to
e
n
r
ich
v
is
u
al
r
ep
r
esen
tatio
n
s
f
o
r
to
x
icit
y
class
if
icatio
n
.
B
y
g
en
er
atin
g
tex
tu
al
d
escr
ip
tio
n
s
f
r
o
m
im
ag
e
a
n
d
v
i
d
eo
in
p
u
ts
,
th
is
m
o
d
u
le
p
r
o
v
id
es
s
u
p
p
le
m
en
tar
y
s
em
an
tic
cu
es
th
at
r
e
in
f
o
r
ce
m
u
ltimo
d
al
f
u
s
io
n
,
r
ath
er
th
an
o
p
e
r
atin
g
as
a
s
tan
d
alo
n
e
g
en
e
r
ativ
e
s
y
s
tem
.
T
h
e
q
u
an
titativ
e
r
esu
lts
o
f
th
e
ca
p
tio
n
in
g
p
er
f
o
r
m
an
ce
ev
al
u
atio
n
ar
e
s
u
m
m
ar
ized
in
T
a
b
le
6
.
T
a
b
l
e
6
.
C
a
p
ti
o
n
i
n
g
p
e
r
f
o
r
m
a
n
c
e
o
f
i
m
a
g
e
a
n
d
v
i
d
e
o
m
o
d
a
l
i
t
ie
s
e
v
a
l
u
at
e
d
u
s
i
n
g
B
L
E
U
,
ME
T
E
O
R
,
a
n
d
C
I
D
E
r
Ta
sk
B
LEU
-
1
B
LEU
-
2
B
LEU
-
3
B
LEU
-
4
M
ET
EO
R
C
I
D
Er
I
mag
e
c
a
p
t
i
o
n
i
n
g
0
.
6
1
7
6
0
.
5
8
1
5
0
.
5
5
7
0
0
.
5
3
9
2
0
.
5
8
8
2
4
.
8
9
0
6
V
i
d
e
o
c
a
p
t
i
o
n
i
n
g
0
.
1
7
2
7
0
.
0
6
3
8
0
.
0
2
5
4
0
.
0
1
0
8
0
.
1
9
5
1
0
.
1
1
7
2
Fo
r
im
ag
e
ca
p
tio
n
in
g
,
th
e
m
o
d
el
d
em
o
n
s
tr
ates
r
o
b
u
s
t
p
er
f
o
r
m
an
ce
,
ac
h
iev
i
n
g
a
b
ilin
g
u
al
ev
alu
atio
n
u
n
d
er
s
tu
d
y
(
B
L
E
U)
-
4
s
co
r
e
o
f
0
.
5
3
9
2
,
a
m
etr
ic
f
o
r
e
v
alu
atio
n
o
f
tr
a
n
s
latio
n
with
ex
p
licit
o
r
d
er
in
g
(
ME
T
E
OR
)
s
co
r
e
o
f
0
.
5
8
8
2
,
an
d
a
co
n
s
en
s
u
s
-
b
ased
im
ag
e
d
escr
ip
tio
n
ev
al
u
atio
n
(
C
I
DE
r
)
s
co
r
e
o
f
4
.
8
9
0
6
,
in
d
icatin
g
its
ab
ilit
y
to
g
e
n
er
at
e
s
em
an
tically
r
elev
an
t
an
d
lin
g
u
is
tically
co
h
er
en
t d
escr
ip
tio
n
s
f
o
r
s
tatic
v
is
u
al
co
n
ten
t.
I
n
co
n
tr
ast,
v
id
e
o
ca
p
tio
n
in
g
p
er
f
o
r
m
a
n
ce
is
s
u
b
s
tan
tially
lo
wer
,
with
a
B
L
E
U
-
4
s
co
r
e
o
f
0
.
0
1
0
8
an
d
a
C
I
DE
r
s
co
r
e
o
f
0
.
1
1
7
2
,
r
ef
lectin
g
th
e
in
h
er
en
t
ch
allen
g
es
o
f
m
o
d
elin
g
te
m
p
o
r
al
d
y
n
am
ics,
r
ap
id
s
ce
n
e
tr
an
s
itio
n
s
,
an
d
lim
ited
tem
p
o
r
al
s
u
p
er
v
is
io
n
.
I
m
p
o
r
tan
tly
,
v
id
eo
ca
p
tio
n
i
n
g
is
n
o
t
o
p
tim
i
ze
d
as
a
s
tan
d
alo
n
e
g
en
er
ativ
e
o
b
jectiv
e;
in
s
tead
,
it
is
d
elib
er
ately
in
co
r
p
o
r
ated
as
an
au
x
iliar
y
s
e
m
an
tic
m
ec
h
an
is
m
.
Desp
ite
lo
wer
q
u
an
titativ
e
s
c
o
r
es,
th
e
g
en
er
ated
v
id
eo
ca
p
t
io
n
s
p
r
o
v
id
e
c
o
m
p
lem
en
ta
r
y
co
n
tex
tu
al
cu
es
th
at
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
3
6
5
-
3
3
7
5
3372
en
h
an
ce
cr
o
s
s
-
m
o
d
al
alig
n
m
e
n
t
an
d
c
o
n
tr
ib
u
te
t
o
m
o
r
e
ef
f
ec
tiv
e
m
u
ltimo
d
al
f
u
s
io
n
f
o
r
th
e
p
r
im
ar
y
task
o
f
to
x
icity
class
if
icatio
n
.
3
.
7
.
Q
ua
lit
a
t
iv
e
a
na
ly
s
is
a
nd
f
a
ilu
re
ca
s
es
T
h
is
s
ec
tio
n
co
m
p
lem
en
ts
t
h
e
q
u
an
titativ
e
ev
alu
atio
n
b
y
p
r
o
v
id
in
g
d
ee
p
er
in
s
ig
h
ts
in
to
th
e
b
eh
av
io
r
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
el
ac
r
o
s
s
d
iv
er
s
e
m
u
ltimo
d
a
l
s
ce
n
ar
io
s
.
T
h
e
an
aly
s
is
ex
am
in
es
h
o
w
tex
tu
al,
v
is
u
al,
an
d
tem
p
o
r
al
i
n
f
o
r
m
atio
n
ar
e
jo
i
n
tly
lev
er
a
g
ed
t
o
s
u
p
p
o
r
t
to
x
icity
p
r
e
d
ictio
n
.
I
t
also
h
ig
h
lig
h
ts
r
ep
r
esen
tativ
e
p
atter
n
s
o
f
b
o
th
co
r
r
ec
t
an
d
in
c
o
r
r
ec
t
p
r
e
d
ic
tio
n
s
to
d
elin
ea
te
th
e
s
tr
en
g
th
s
an
d
lim
itatio
n
s
o
f
th
e
p
r
o
p
o
s
ed
ar
c
h
itectu
r
e.
3
.
7
.
1
.
Q
ua
lita
t
iv
e
re
s
ults o
n
t
he
cus
t
o
m
da
t
a
s
et
T
ab
le
7
p
r
esen
ts
r
ep
r
esen
tativ
e
q
u
alitativ
e
ex
am
p
les
illu
s
tr
a
tin
g
th
e
jo
in
t
an
aly
s
is
o
f
tex
tu
al,
v
is
u
al,
an
d
v
id
eo
in
p
u
ts
,
alo
n
g
wi
th
th
e
g
en
er
ate
d
ca
p
tio
n
s
a
n
d
p
r
ed
icted
to
x
icity
lab
els.
T
h
ese
e
x
am
p
les
d
em
o
n
s
tr
ate
th
e
m
o
d
el’
s
ab
ilit
y
to
s
y
n
th
esize
co
n
tex
tu
al
in
f
o
r
m
atio
n
ac
r
o
s
s
m
o
d
alities
an
d
p
r
o
d
u
ce
co
h
er
en
t
m
u
ltimo
d
al
in
ter
p
r
etatio
n
s
.
N
o
tab
ly
,
th
e
r
esu
lts
h
ig
h
lig
h
t
t
h
e
r
o
le
o
f
cr
o
s
s
-
m
o
d
al
c
u
es
in
d
is
am
b
ig
u
atin
g
to
x
ic
an
d
n
o
n
-
to
x
ic
co
n
ten
t
in
co
m
p
le
x
an
d
co
n
tex
t
-
d
e
p
en
d
en
t
s
ce
n
ar
io
s
.
Ov
e
r
all,
th
is
q
u
alitativ
e
an
aly
s
is
u
n
d
er
s
co
r
es
th
e
ef
f
ec
tiv
e
n
ess
o
f
th
e
p
r
o
p
o
s
ed
ap
p
r
o
ac
h
in
h
an
d
lin
g
th
e
n
u
an
ce
d
an
d
co
n
tex
t
-
r
ich
n
atu
r
e
o
f
s
o
cial
m
ed
ia
co
n
ten
t.
T
ab
le
7
.
R
ep
r
esen
tativ
e
q
u
alitativ
e
ex
am
p
les o
f
m
u
ltimo
d
al
to
x
icity
class
if
icatio
n
p
r
o
d
u
ce
d
b
y
th
e
p
r
o
p
o
s
ed
W
Vis
io
n
B
E
R
T
-
VL
f
r
am
ewo
r
k
Te
x
t
i
n
p
u
t
I
mag
e
i
n
p
u
t
V
i
d
e
o
i
n
p
u
t
I
mag
e
c
a
p
t
i
o
n
i
n
g
o
u
t
p
u
t
V
i
d
e
o
c
a
p
t
i
o
n
i
n
g
o
u
t
p
u
t
To
x
i
c
i
t
y
l
a
b
e
l
Y
o
u
’
r
e
so
w
e
a
k
,
l
i
t
t
l
e
o
n
e
,
y
o
u
a
l
r
e
a
d
y
h
a
v
e
so
man
y
r
e
sp
o
n
s
i
b
i
l
i
t
i
e
s
.
D
o
n
’
t
f
o
r
g
e
t
t
o
w
a
sh
y
o
u
r
f
a
c
e
.
Th
e
c
i
r
c
u
mci
se
d
k
i
d
p
r
e
t
e
n
d
e
d
t
o
f
a
i
t
h
h
a
h
a
N
e
u
t
r
a
l
N
o
w
a
d
a
y
s,
man
y
p
e
o
p
l
e
d
o
n
’
t
k
n
o
w
t
h
e
mse
l
v
e
s
I
l
o
v
e
t
h
e
r
e
c
e
p
t
i
o
n
o
f
t
h
e
d
r
a
ma
,
b
u
t
i
t
’
s
d
e
f
i
n
i
t
e
l
y
n
o
t
r
e
a
l
.
S
o
r
r
y
,
b
u
t
t
h
i
s
i
s
p
r
o
h
i
b
i
t
e
d
c
o
n
t
e
n
t
.
I
c
a
n
’
t
h
e
l
p
w
i
t
h
t
h
i
s.
N
o
n
-
t
o
x
i
c
3
.
7
.
2
.
F
a
ilu
re
ca
s
e
a
na
ly
s
is
a
nd
lim
it
a
t
io
ns
Desp
ite
th
e
s
tr
o
n
g
o
v
er
all
p
e
r
f
o
r
m
a
n
ce
,
s
ev
er
al
lim
itatio
n
s
an
d
f
ailu
r
e
ca
s
es
wer
e
id
en
tifie
d
.
T
h
e
m
o
d
el
o
cc
asio
n
ally
s
tr
u
g
g
les
to
d
etec
t
to
x
ic
in
ten
t
ex
p
r
ess
ed
im
p
licitly
th
r
o
u
g
h
s
ar
ca
s
m
,
ir
o
n
y
,
o
r
c
u
ltu
r
ally
s
p
ec
if
ic
s
lan
g
,
p
ar
ticu
lar
ly
in
tex
t
-
o
n
ly
s
ce
n
ar
io
s
wh
er
e
e
x
p
licit
lex
ical
cu
es
ar
e
ab
s
en
t.
Su
ch
ca
s
es
p
o
s
e
ch
allen
g
es f
o
r
s
em
a
n
tic
in
ter
p
r
etatio
n
ev
en
w
h
en
u
s
in
g
co
n
t
ex
tu
alize
d
em
b
e
d
d
in
g
s
.
I
n
m
u
ltimo
d
al
s
ettin
g
s
,
m
is
class
if
icatio
n
s
p
r
im
ar
ily
o
cc
u
r
wh
en
s
em
an
tic
s
ig
n
als
ac
r
o
s
s
m
o
d
alities
ar
e
wea
k
o
r
m
is
alig
n
ed
.
Fo
r
e
x
am
p
le,
n
eu
tr
al
tex
tu
al
co
n
te
n
t p
air
ed
with
s
u
b
tly
s
u
g
g
esti
v
e
o
r
s
y
m
b
o
lic
v
is
u
al
cu
es
ca
n
lead
to
in
co
r
r
ec
t
p
r
e
d
ictio
n
s
,
u
n
d
er
s
co
r
i
n
g
th
e
d
if
f
icu
lty
o
f
r
eso
lv
in
g
am
b
i
g
u
ity
wh
en
cr
o
s
s
-
m
o
d
al
ev
id
en
ce
r
e
m
ain
s
im
p
licit.
Vid
eo
-
b
ased
i
n
p
u
ts
in
tr
o
d
u
ce
a
d
d
itio
n
al
ch
allen
g
es
r
elate
d
t
o
tem
p
o
r
al
s
p
ar
s
ity
:
s
h
o
r
t
v
id
eo
d
u
r
atio
n
s
,
r
a
p
id
s
ce
n
e
tr
an
s
itio
n
s
,
an
d
p
a
r
tial o
cc
lu
s
io
n
s
m
ay
h
in
d
er
th
e
m
o
d
el’
s
ab
ilit
y
to
ca
p
tu
r
e
co
n
s
is
ten
t
tem
p
o
r
al
p
atter
n
s
.
T
h
is
lim
itatio
n
is
co
n
s
is
ten
t
with
th
e
lo
wer
q
u
an
titativ
e
p
e
r
f
o
r
m
a
n
ce
o
b
s
er
v
ed
in
th
e
v
id
e
o
ca
p
tio
n
in
g
task
,
in
d
icatin
g
th
at
th
e
cu
r
r
en
t
tem
p
o
r
al
m
o
d
elin
g
s
tr
ateg
y
m
a
y
b
e
in
s
u
f
f
icien
t
wh
e
n
s
alien
t v
is
u
al
cu
es a
p
p
ea
r
b
r
ie
f
ly
o
r
i
n
ter
m
itten
tly
.
Fin
ally
,
cr
o
s
s
-
d
ataset
ev
alu
atio
n
r
ev
ea
ls
s
en
s
itiv
ity
to
d
o
m
ain
s
h
if
t.
Du
e
to
its
r
elian
ce
o
n
I
n
d
o
B
E
R
T
as
th
e
te
x
tu
al
b
ac
k
b
o
n
e,
t
h
e
m
o
d
el
e
x
h
ib
its
r
ed
u
ce
d
p
er
f
o
r
m
an
ce
o
n
E
n
g
lis
h
-
lan
g
u
ag
e
b
en
ch
m
ar
k
s
(
jig
s
aw
an
d
h
atef
u
l
m
em
es
)
in
th
e
ab
s
en
ce
o
f
f
in
e
-
tu
n
in
g
.
T
h
ese
f
in
d
in
g
s
h
ig
h
lig
h
t
th
e
n
ee
d
f
o
r
f
u
tu
r
e
wo
r
k
to
f
o
c
u
s
o
n
e
n
h
a
n
ce
d
tem
p
o
r
al
m
o
d
elin
g
,
s
tr
o
n
g
er
cr
o
s
s
-
m
o
d
al
alig
n
m
e
n
t,
a
n
d
t
h
e
d
ev
elo
p
m
en
t
o
f
d
o
m
ain
-
ad
a
p
tiv
e
an
d
cr
o
s
s
-
lin
g
u
al
s
tr
ateg
ies to
im
p
r
o
v
e
g
en
er
aliza
tio
n
ac
r
o
s
s
d
iv
er
s
e
li
n
g
u
is
tic
co
n
tex
ts
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
WVisi
o
n
B
E
R
T
-
V
L:
a
mu
ltimo
d
a
l m
o
d
el
a
r
ch
itectu
r
e
fo
r
to
xi
city
cla
s
s
ifica
tio
n
…
(
Wit
ta
Li
s
tiya
N
in
g
r
u
m
)
3373
4.
CO
NCLU
SI
O
N
T
h
is
s
tu
d
y
i
n
tr
o
d
u
ce
s
W
Vis
io
n
B
E
R
T
-
VL
,
a
co
m
p
r
eh
en
s
iv
e
m
u
ltimo
d
al
to
x
icity
d
etec
tio
n
f
r
am
ewo
r
k
th
at
in
teg
r
ates te
x
t,
im
ag
e,
an
d
v
id
eo
m
o
d
alities
to
ad
d
r
ess
th
e
lim
itatio
n
s
o
f
u
n
im
o
d
al
ap
p
r
o
ac
h
es
in
ca
p
tu
r
in
g
c
o
n
tex
tu
al
a
n
d
im
p
licit
to
x
icity
.
B
y
lev
er
ag
i
n
g
B
MH
C
A
an
d
an
ad
ap
tiv
e
f
u
s
io
n
g
ate,
th
e
p
r
o
p
o
s
ed
ar
ch
itectu
r
e
ef
f
ec
tiv
ely
s
y
n
ch
r
o
n
izes
cr
o
s
s
-
m
o
d
al
in
f
o
r
m
atio
n
an
d
b
alan
ce
s
m
o
d
ality
co
n
tr
ib
u
tio
n
s
with
in
a
u
n
if
ied
e
n
d
-
to
-
en
d
s
y
s
tem
.
E
x
p
er
im
en
tal
r
esu
lts
o
n
a
cu
s
to
m
I
n
d
o
n
esian
s
o
cial
m
ed
ia
d
ataset
d
em
o
n
s
tr
ate
s
tr
o
n
g
in
-
d
o
m
ain
p
er
f
o
r
m
an
ce
,
with
th
e
m
o
d
el
ac
h
iev
in
g
h
ig
h
ac
c
u
r
ac
y
an
d
m
ac
r
o
-
F1
.
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
f
u
r
th
er
co
n
f
i
r
m
s
th
e
s
tab
ilit
y
an
d
r
o
b
u
s
tn
ess
o
f
th
e
p
r
o
p
o
s
ed
a
p
p
r
o
ac
h
.
Ab
latio
n
s
tu
d
ies
p
r
o
v
id
e
an
im
p
o
r
ta
n
t
in
s
ig
h
t:
wh
ile
tex
tu
al
in
f
o
r
m
atio
n
estab
lis
h
es
a
s
tr
o
n
g
b
aselin
e,
th
e
in
clu
s
io
n
o
f
v
is
u
al
an
d
tem
p
o
r
al
cu
es
,
p
ar
ticu
lar
ly
f
r
o
m
v
id
eo
,
s
u
b
s
tan
tially
i
m
p
r
o
v
es
class
-
lev
el
b
alan
ce
an
d
r
o
b
u
s
tn
ess
f
o
r
am
b
ig
u
o
u
s
an
d
m
in
o
r
ity
class
es,
d
esp
ite
a
m
ar
g
i
n
al
tr
ad
e
-
o
f
f
in
o
v
er
all
ac
cu
r
ac
y
.
I
n
a
d
d
it
io
n
,
th
e
ca
p
tio
n
in
g
co
m
p
o
n
en
t
s
er
v
es
as
an
ef
f
ec
tiv
e
au
x
iliar
y
m
ec
h
an
is
m
to
en
r
ich
v
is
u
al
r
ep
r
esen
tatio
n
s
an
d
f
ac
ilit
ate
m
u
ltimo
d
al
f
u
s
io
n
,
r
at
h
er
th
a
n
f
u
n
ctio
n
i
n
g
as
a
s
tan
d
alo
n
e
g
en
er
ativ
e
o
b
jectiv
e.
Nev
er
th
eless
,
ch
allen
g
es
r
em
ain
in
h
an
d
lin
g
im
p
licit
ex
p
r
ess
io
n
s
s
u
ch
as
s
ar
ca
s
m
,
ad
d
r
ess
in
g
wea
k
cr
o
s
s
-
m
o
d
al
alig
n
m
en
t,
an
d
ac
h
iev
in
g
ze
r
o
-
s
h
o
t g
e
n
er
aliza
tio
n
ac
r
o
s
s
lan
g
u
ag
es d
u
e
to
d
o
m
ain
s
h
if
ts
.
Fu
tu
r
e
wo
r
k
will
th
er
ef
o
r
e
f
o
cu
s
o
n
en
h
an
ce
d
tem
p
o
r
al
m
o
d
elin
g
,
s
tr
o
n
g
er
d
o
m
ain
-
ad
ap
tiv
e
s
tr
ateg
ies,
an
d
ex
ten
d
in
g
th
e
f
r
a
m
ewo
r
k
to
b
r
o
a
d
er
lin
g
u
is
tic
an
d
cu
ltu
r
al
co
n
tex
t
s
.
Ov
er
all,
th
is
s
tu
d
y
co
n
tr
ib
u
t
es to
th
e
d
ev
elo
p
m
e
n
t o
f
s
af
er
an
d
m
o
r
e
i
n
clu
s
iv
e
d
ig
ital e
n
v
ir
o
n
m
en
ts
f
o
r
lo
w
-
r
eso
u
r
ce
lan
g
u
ag
es
.
ACK
NO
WL
E
DG
M
E
N
T
S
T
h
e
au
th
o
r
s
ac
k
n
o
wled
g
e
th
e
s
u
p
p
o
r
t p
r
o
v
i
d
ed
b
y
Gu
n
a
d
ar
m
a
Un
iv
er
s
ity
.
F
UNDING
I
NF
O
R
M
A
T
I
O
N
Au
th
o
r
s
s
tate
n
o
f
u
n
d
in
g
in
v
o
lv
ed
.
AUTHO
R
CO
NT
RI
B
UT
I
O
NS ST
A
T
E
M
E
N
T
T
h
is
jo
u
r
n
al
u
s
es
th
e
C
o
n
tr
ib
u
to
r
R
o
les
T
ax
o
n
o
m
y
(
C
R
ed
iT)
to
r
ec
o
g
n
ize
in
d
iv
id
u
al
au
th
o
r
co
n
tr
ib
u
tio
n
s
,
r
ed
u
ce
au
th
o
r
s
h
ip
d
is
p
u
tes,
an
d
f
ac
ilit
ate
co
llab
o
r
atio
n
.
Na
m
e
o
f
Aut
ho
r
C
M
So
Va
Fo
I
R
D
O
E
Vi
Su
P
Fu
W
itta L
is
tiy
a
Nin
g
r
u
m
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Ach
m
ad
B
en
n
y
M
u
tiar
a
✓
✓
✓
✓
✓
✓
✓
✓
Dian
a
I
k
asar
i
✓
✓
✓
✓
✓
✓
✓
✓
✓
C
:
C
o
n
c
e
p
t
u
a
l
i
z
a
t
i
o
n
M
:
M
e
t
h
o
d
o
l
o
g
y
So
:
So
f
t
w
a
r
e
Va
:
Va
l
i
d
a
t
i
o
n
Fo
:
Fo
r
mal
a
n
a
l
y
s
i
s
I
:
I
n
v
e
s
t
i
g
a
t
i
o
n
R
:
R
e
so
u
r
c
e
s
D
:
D
a
t
a
C
u
r
a
t
i
o
n
O
:
W
r
i
t
i
n
g
-
O
r
i
g
i
n
a
l
D
r
a
f
t
E
:
W
r
i
t
i
n
g
-
R
e
v
i
e
w
&
E
d
i
t
i
n
g
Vi
:
Vi
su
a
l
i
z
a
t
i
o
n
Su
:
Su
p
e
r
v
i
s
i
o
n
P
:
P
r
o
j
e
c
t
a
d
mi
n
i
st
r
a
t
i
o
n
Fu
:
Fu
n
d
i
n
g
a
c
q
u
i
si
t
i
o
n
CO
NF
L
I
C
T
O
F
I
N
T
E
R
E
S
T
ST
A
T
E
M
E
NT
T
h
e
au
th
o
r
s
d
ec
lar
e
th
at
th
e
r
e
is
n
o
co
n
f
lict o
f
in
ter
est r
eg
ar
d
in
g
th
e
p
u
b
licatio
n
o
f
th
is
p
a
p
er
.
I
NF
O
RM
E
D
CO
NS
E
N
T
T
h
is
s
tu
d
y
d
o
es
n
o
t
in
v
o
lv
e
h
u
m
an
p
ar
ticip
an
ts
o
r
an
y
p
er
s
o
n
ally
id
e
n
tifia
b
le
in
f
o
r
m
atio
n
;
th
er
ef
o
r
e,
in
f
o
r
m
e
d
co
n
s
en
t w
as n
o
t
r
eq
u
ir
ed
.
E
T
H
I
CAL AP
P
RO
V
AL
T
h
is
r
es
ea
r
c
h
d
o
es
n
o
t
in
v
o
l
v
e
h
u
m
a
n
o
r
a
n
im
al
s
u
b
j
ec
ts
,
t
h
e
r
e
f
o
r
e
et
h
ic
al
a
p
p
r
o
v
al
w
as
n
o
t
r
eq
u
i
r
ed
.
DATA AV
AI
L
AB
I
L
I
T
Y
T
h
e
cu
s
to
m
d
ataset
s
u
p
p
o
r
tin
g
th
e
f
in
d
in
g
s
o
f
th
is
s
tu
d
y
is
p
u
b
licly
av
ailab
le
o
n
H
u
g
g
i
n
g
f
ac
e
at:
h
ttp
s
://h
u
g
g
in
g
f
ac
e.
co
/wittalis
tiy
an
in
g
r
u
m
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
3
6
5
-
3
3
7
5
3374
RE
F
E
R
E
NC
E
S
[
1
]
B
.
D
.
S
h
i
f
e
r
a
w
e
t
a
l
.
,
“
I
mp
a
c
t
o
f
d
i
g
i
t
a
l
a
d
d
i
c
t
i
o
n
o
n
y
o
u
t
h
h
e
a
l
t
h
:
a
s
y
st
e
m
a
t
i
c
r
e
v
i
e
w
a
n
d
me
t
a
-
a
n
a
l
y
si
s,
”
J
o
u
rn
a
l
o
f
Be
h
a
v
i
o
r
a
l
Ad
d
i
c
t
i
o
n
s
,
v
o
l
.
1
4
,
n
o
.
3
,
p
p
.
1
1
2
9
–
1
1
5
8
,
S
e
p
.
2
0
2
5
,
d
o
i
:
1
0
.
1
5
5
6
/
2
0
0
6
.
2
0
2
5
.
0
0
0
8
1
.
[
2
]
H
.
F
a
n
e
t
a
l
.
,
“
S
o
c
i
a
l
m
e
d
i
a
t
o
x
i
c
i
t
y
c
l
a
ss
i
f
i
c
a
t
i
o
n
u
si
n
g
d
e
e
p
l
e
a
r
n
i
n
g
:
r
e
a
l
-
w
o
r
l
d
a
p
p
l
i
c
a
t
i
o
n
U
K
B
r
e
x
i
t
,”
El
e
c
t
r
o
n
i
c
s
,
v
o
l
.
1
0
,
n
o
.
1
1
,
J
u
n
.
2
0
2
1
,
d
o
i
:
1
0
.
3
3
9
0
/
e
l
e
c
t
r
o
n
i
c
s
1
0
1
1
1
3
3
2
.
[
3
]
A
.
K
u
m
a
r
a
n
d
N
.
S
a
c
h
d
e
v
a
,
“
M
u
l
t
i
mo
d
a
l
c
y
b
e
r
b
u
l
l
y
i
n
g
d
e
t
e
c
t
i
o
n
u
si
n
g
c
a
p
su
l
e
n
e
t
w
o
r
k
w
i
t
h
d
y
n
a
mi
c
r
o
u
t
i
n
g
a
n
d
d
e
e
p
c
o
n
v
o
l
u
t
i
o
n
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
,
”
Mu
l
t
i
m
e
d
i
a
S
y
st
e
m
s
,
v
o
l
.
2
8
,
n
o
.
6
,
p
p
.
2
0
4
3
–
2
0
5
2
,
D
e
c
.
2
0
2
2
,
d
o
i
:
1
0
.
1
0
0
7
/
s0
0
5
3
0
-
0
2
0
-
0
0
7
4
7
-
5.
[
4
]
A
.
El
s
a
y
a
r
y
,
R
.
A
.
A
l
z
a
f
f
i
n
,
a
n
d
M
.
E.
A
l
su
w
a
i
d
i
,
“
N
a
v
i
g
a
t
i
n
g
d
i
g
i
t
a
l
i
d
e
n
t
i
t
i
e
s
:
t
h
e
i
n
f
l
u
e
n
c
e
o
f
o
n
l
i
n
e
i
n
t
e
r
a
c
t
i
o
n
s
o
n
y
o
u
t
h
c
y
b
e
r
p
s
y
c
h
o
l
o
g
y
a
n
d
c
y
b
e
r
b
e
h
a
v
i
o
r
,
”
i
n
1
8
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
M
u
l
t
i
-
C
o
n
f
e
r
e
n
c
e
o
n
S
o
c
i
e
t
y
,
C
y
b
e
rn
e
t
i
c
s
a
n
d
I
n
f
o
rm
a
t
i
c
s
(
I
M
S
C
I
)
,
S
e
p
.
2
0
2
4
,
p
p
.
1
2
3
–
1
3
0
.
d
o
i
:
1
0
.
5
4
8
0
8
/
I
M
S
C
I
2
0
2
4
.
0
1
.
1
2
3
.
[
5
]
C
.
Ja
n
i
e
sc
h
,
P
.
Z
sc
h
e
c
h
,
a
n
d
K
.
H
e
i
n
r
i
c
h
,
“
M
a
c
h
i
n
e
l
e
a
r
n
i
n
g
a
n
d
d
e
e
p
l
e
a
r
n
i
n
g
,
”
E
l
e
c
t
r
o
n
i
c
Ma
r
k
e
t
s
,
v
o
l
.
3
1
,
n
o
.
3
,
p
p
.
6
8
5
–
6
9
5
,
S
e
p
.
2
0
2
1
,
d
o
i
:
1
0
.
1
0
0
7
/
s
1
2
5
2
5
-
021
-
0
0
4
7
5
-
2.
[
6
]
J.
C
h
a
i
,
H
.
Ze
n
g
,
A
.
L
i
,
a
n
d
E
.
W
.
T.
N
g
a
i
,
“
D
e
e
p
l
e
a
r
n
i
n
g
i
n
c
o
m
p
u
t
e
r
v
i
s
i
o
n
:
a
c
r
i
t
i
c
a
l
r
e
v
i
e
w
o
f
e
mer
g
i
n
g
t
e
c
h
n
i
q
u
e
s
a
n
d
a
p
p
l
i
c
a
t
i
o
n
s
c
e
n
a
r
i
o
s,
”
Ma
c
h
i
n
e
L
e
a
r
n
i
n
g
w
i
t
h
Ap
p
l
i
c
a
t
i
o
n
s
,
v
o
l
.
6
,
D
e
c
.
2
0
2
1
,
d
o
i
:
1
0
.
1
0
1
6
/
j
.
ml
w
a
.
2
0
2
1
.
1
0
0
1
3
4
.
[
7
]
I
.
D
.
M
i
e
n
y
e
a
n
d
T.
G
.
S
w
a
r
t
,
“
A
c
o
mp
r
e
h
e
n
s
i
v
e
r
e
v
i
e
w
o
f
d
e
e
p
l
e
a
r
n
i
n
g
:
a
r
c
h
i
t
e
c
t
u
r
e
s
,
r
e
c
e
n
t
a
d
v
a
n
c
e
s
,
a
n
d
a
p
p
l
i
c
a
t
i
o
n
s,
”
I
n
f
o
rm
a
t
i
o
n
,
v
o
l
.
1
5
,
n
o
.
1
2
,
N
o
v
.
2
0
2
4
,
d
o
i
:
1
0
.
3
3
9
0
/
i
n
f
o
1
5
1
2
0
7
5
5
.
[
8
]
B
.
M
i
n
e
t
a
l
.
,
“
R
e
c
e
n
t
a
d
v
a
n
c
e
s i
n
n
a
t
u
r
a
l
l
a
n
g
u
a
g
e
p
r
o
c
e
ssi
n
g
v
i
a
l
a
r
g
e
p
r
e
-
t
r
a
i
n
e
d
l
a
n
g
u
a
g
e
m
o
d
e
l
s
:
a
s
u
r
v
e
y
,
”
AC
M
C
o
m
p
u
t
i
n
g
S
u
r
v
e
y
s
,
v
o
l
.
5
6
,
n
o
.
2
,
p
p
.
1
–
4
0
,
F
e
b
.
2
0
2
4
,
d
o
i
:
1
0
.
1
1
4
5
/
3
6
0
5
9
4
3
.
[
9
]
M
.
C
i
n
e
l
l
i
,
A
.
P
e
l
i
c
o
n
,
I
.
M
o
z
e
t
i
č
,
W
.
Q
u
a
t
t
r
o
c
i
o
c
c
h
i
,
P
.
K
.
N
o
v
a
k
,
a
n
d
F
.
Zo
l
l
o
,
“
D
y
n
a
mi
c
s
o
f
o
n
l
i
n
e
h
a
t
e
a
n
d
mi
si
n
f
o
r
m
a
t
i
o
n
,”
S
c
i
e
n
t
i
f
i
c
Re
p
o
r
t
s
,
v
o
l
.
1
1
,
n
o
.
1
,
N
o
v
.
2
0
2
1
,
d
o
i
:
1
0
.
1
0
3
8
/
s
4
1
5
9
8
-
0
2
1
-
0
1
4
8
7
-
w.
[
1
0
]
F
.
I
scu
s
a
n
d
A
.
S
.
G
i
r
s
a
n
g
,
“
S
e
n
t
i
me
n
t
a
n
a
l
y
si
s
o
f
C
O
V
I
D
-
1
9
p
u
b
l
i
c
a
c
t
i
v
i
t
y
r
e
s
t
r
i
c
t
i
o
n
(
P
P
K
M
)
i
m
p
a
c
t
u
si
n
g
B
ER
T
met
h
o
d
,
”
I
n
t
e
r
n
a
t
i
o
n
a
l
J
o
u
rn
a
l
o
f
E
n
g
i
n
e
e
ri
n
g
T
re
n
d
s
a
n
d
T
e
c
h
n
o
l
o
g
y
,
v
o
l
.
7
0
,
n
o
.
1
2
,
p
p
.
2
8
1
–
2
8
8
,
D
e
c
.
2
0
2
2
,
d
o
i
:
1
0
.
1
4
4
4
5
/
2
2
3
1
5
3
8
1
/
I
JETT
-
V
7
0
I
1
2
P
2
2
6
.
[
1
1
]
J
.
Z
e
n
g
,
L
.
Y
a
n
g
,
Z
.
W
a
n
g
,
Y
.
S
u
n
,
a
n
d
H
.
L
i
n
,
“
S
h
e
e
p
’
s
s
k
i
n
,
w
o
l
f
’
s
d
e
e
d
s
:
a
r
e
L
L
M
s
r
e
a
d
y
f
o
r
m
e
t
a
p
h
o
r
i
c
a
l
i
m
p
l
i
c
i
t
h
a
t
e
s
p
e
e
c
h
?
,
”
i
n
6
3
r
d
A
n
n
u
a
l
M
e
e
t
i
n
g
o
f
t
h
e
A
s
s
o
c
i
a
t
i
o
n
f
o
r
C
o
m
p
u
t
a
t
i
o
n
a
l
L
i
n
g
u
i
s
t
i
c
s
,
2
0
2
5
,
p
p
.
1
6
6
5
7
–
16677
,
d
o
i
:
1
0
.
1
8
6
5
3
/
v
1
/
2
0
2
5
.
a
c
l
-
l
o
n
g
.
8
1
4
.
[
1
2
]
M
.
W
a
n
g
e
t
a
l
.
,
“
S
TA
N
D
-
g
u
a
r
d
:
a
s
mal
l
t
a
sk
-
a
d
a
p
t
i
v
e
c
o
n
t
e
n
t
m
o
d
e
r
a
t
i
o
n
m
o
d
e
l
,
”
I
n
t
e
rn
a
t
i
o
n
a
l
C
o
n
f
e
re
n
c
e
o
n
C
o
m
p
u
t
a
t
i
o
n
a
l
L
i
n
g
u
i
st
i
c
s
,
C
O
L
I
N
G
.
p
p
.
1
–
2
0
,
2
0
2
5
.
[
1
3
]
M
.
U
.
H
a
d
i
e
t
a
l
.
,
“
A
s
u
r
v
e
y
o
n
l
a
r
g
e
l
a
n
g
u
a
g
e
m
o
d
e
l
s:
a
p
p
l
i
c
a
t
i
o
n
s
,
c
h
a
l
l
e
n
g
e
s
,
l
i
m
i
t
a
t
i
o
n
s,
a
n
d
p
r
a
c
t
i
c
a
l
u
s
a
g
e
.
”
T
e
c
h
R
x
i
v
,
J
u
l
.
1
0
,
2
0
2
3
.
d
o
i
:
1
0
.
3
6
2
2
7
/
t
e
c
h
r
x
i
v
.
2
3
5
8
9
7
4
1
.
v
1
.
[
1
4
]
J.
H
u
a
n
g
a
n
d
K
.
C
.
-
C
.
C
h
a
n
g
,
“
T
o
w
a
r
d
s
r
e
a
so
n
i
n
g
i
n
l
a
r
g
e
l
a
n
g
u
a
g
e
mo
d
e
l
s
:
a
s
u
r
v
e
y
,
”
i
n
F
i
n
d
i
n
g
s
o
f
t
h
e
Asso
c
i
a
t
i
o
n
f
o
r
C
o
m
p
u
t
a
t
i
o
n
a
l
L
i
n
g
u
i
s
t
i
c
s:
A
C
L
2
0
2
3
,
2
0
2
3
,
p
p
.
1
0
4
9
–
1
0
6
5
.
d
o
i
:
1
0
.
1
8
6
5
3
/
v
1
/
2
0
2
3
.
f
i
n
d
i
n
g
s
-
a
c
l
.
6
7
.
[
1
5
]
J.
A
.
D
.
‐
G
a
r
c
i
a
a
n
d
J
.
P
.
C
a
r
v
a
l
h
o
,
“
A
l
i
t
e
r
a
t
u
r
e
r
e
v
i
e
w
o
f
t
e
x
t
u
a
l
c
y
b
e
r
a
b
u
s
e
d
e
t
e
c
t
i
o
n
u
si
n
g
c
u
t
t
i
n
g
‐
e
d
g
e
n
a
t
u
r
a
l
l
a
n
g
u
a
g
e
p
r
o
c
e
ss
i
n
g
t
e
c
h
n
i
q
u
e
s
:
l
a
n
g
u
a
g
e
mo
d
e
l
s
a
n
d
l
a
r
g
e
l
a
n
g
u
a
g
e
m
o
d
e
l
s
,
”
WI
REs
D
a
t
a
M
i
n
i
n
g
a
n
d
K
n
o
w
l
e
d
g
e
D
i
sco
v
e
ry
,
v
o
l
.
1
5
,
n
o
.
3
,
S
e
p
.
2
0
2
5
,
d
o
i
:
1
0
.
1
0
0
2
/
w
i
d
m.
7
0
0
2
9
.
[
1
6
]
H
.
To
u
v
r
o
n
e
t
a
l
.
,
“
L
La
M
A
:
o
p
e
n
a
n
d
e
f
f
i
c
i
e
n
t
f
o
u
n
d
a
t
i
o
n
l
a
n
g
u
a
g
e
m
o
d
e
l
s
,
”
2
0
2
3
,
a
rX
i
v
:
2
3
0
2
.
1
3
9
7
1
.
[
1
7
]
A
.
K
u
c
h
a
r
a
v
y
,
O
.
P
l
a
n
c
h
e
r
e
l
,
V
.
M
u
l
d
e
r
,
A
.
M
e
r
m
o
u
d
,
a
n
d
V
.
Le
n
d
e
r
s,
“
L
a
r
g
e
l
a
n
g
u
a
g
e
mo
d
e
l
s
i
n
c
y
b
e
r
sec
u
r
i
t
y
:
t
h
r
e
a
t
s,
e
x
p
o
s
u
r
e
a
n
d
m
i
t
i
g
a
t
i
o
n
,
”
L
a
rg
e
L
a
n
g
u
a
g
e
Mo
d
e
l
s
i
n
C
y
b
e
rse
c
u
ri
t
y
:
T
h
re
a
t
s
,
E
x
p
o
s
u
re
a
n
d
Mi
t
i
g
a
t
i
o
n
,
p
p
.
1
–
2
4
7
,
2
0
2
4
,
d
o
i
:
1
0
.
1
0
0
7
/
9
7
8
-
3
-
031
-
5
4
8
2
7
-
7.
[
1
8
]
H
.
Li
u
,
C
.
Li
,
Q
.
W
u
,
a
n
d
Y
.
J.
Le
e
,
“
V
i
su
a
l
i
n
st
r
u
c
t
i
o
n
t
u
n
i
n
g
,
”
3
7
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
r
e
n
c
e
o
n
N
e
u
r
a
l
I
n
f
o
r
m
a
t
i
o
n
Pr
o
c
e
ssi
n
g
S
y
s
t
e
m
s
,
2
0
2
3
,
p
p
.
3
4
8
9
2
-
3
4
9
1
6
,
d
o
i
:
1
0
.
5
5
5
5
/
3
6
6
6
1
2
2
.
3
6
6
7
6
3
8
.
[
1
9
]
H
.
Li
u
,
C
.
Li
,
Y
.
L
i
,
a
n
d
Y
.
J
.
L
e
e
,
“
I
mp
r
o
v
e
d
b
a
s
e
l
i
n
e
s wi
t
h
v
i
su
a
l
i
n
st
r
u
c
t
i
o
n
t
u
n
i
n
g
,
”
2
0
2
4
I
EEE
/
C
VF
C
o
n
f
e
r
e
n
c
e
o
n
C
o
m
p
u
t
e
r
Vi
si
o
n
a
n
d
Pa
t
t
e
rn
Re
c
o
g
n
i
t
i
o
n
(
C
V
P
R)
,
p
p
.
2
6
2
8
6
–
2
6
2
9
6
,
2
0
2
4
,
d
o
i
:
1
0
.
1
1
0
9
/
C
V
P
R
5
2
7
3
3
.
2
0
2
4
.
0
2
4
8
4
.
[
2
0
]
W
.
D
a
i
e
t
a
l
.
,
“
I
n
st
r
u
c
t
B
LI
P
:
t
o
w
a
r
d
s
g
e
n
e
r
a
l
-
p
u
r
p
o
se
v
i
s
i
o
n
-
l
a
n
g
u
a
g
e
mo
d
e
l
s
w
i
t
h
i
n
st
r
u
c
t
i
o
n
t
u
n
i
n
g
,
”
A
d
v
a
n
c
e
s
i
n
N
e
u
r
a
l
I
n
f
o
rm
a
t
i
o
n
Pr
o
c
e
ssi
n
g
S
y
st
e
m
s
,
v
o
l
.
3
6
,
p
p
.
4
9
2
5
0
–
4
9
2
6
7
,
D
e
c
.
2
0
2
3
,
d
o
i
:
1
0
.
5
2
2
0
2
/
0
7
5
2
8
0
-
2
1
4
2
.
[
2
1
]
S
.
To
n
g
e
t
a
l
.
,
“
C
a
m
b
r
i
a
n
-
1
:
a
f
u
l
l
y
o
p
e
n
,
v
i
s
i
o
n
-
c
e
n
t
r
i
c
e
x
p
l
o
r
a
t
i
o
n
o
f
mu
l
t
i
m
o
d
a
l
LL
M
s,
”
A
d
v
a
n
c
e
s
i
n
N
e
u
ra
l
I
n
f
o
rm
a
t
i
o
n
Pro
c
e
ssi
n
g
S
y
s
t
e
m
s
,
v
o
l
.
3
7
,
2
0
2
4
,
d
o
i
:
1
0
.
5
2
2
0
2
/
0
7
9
0
1
7
-
2
7
7
1
.
[
2
2
]
M
.
D
e
i
t
k
e
e
t
a
l
.
,
“
M
o
l
m
o
a
n
d
P
i
x
M
o
:
o
p
e
n
w
e
i
g
h
t
s
a
n
d
o
p
e
n
d
a
t
a
f
o
r
st
a
t
e
-
of
-
t
h
e
-
a
r
t
v
i
si
o
n
-
l
a
n
g
u
a
g
e
mo
d
e
l
s,
”
2
0
2
5
I
EEE
/
C
V
F
C
o
n
f
e
re
n
c
e
o
n
C
o
m
p
u
t
e
r
Vi
s
i
o
n
a
n
d
Pa
t
t
e
r
n
Re
c
o
g
n
i
t
i
o
n
(
C
VP
R)
,
p
p
.
9
1
–
1
0
4
,
2
0
2
5
,
d
o
i
:
1
0
.
1
1
0
9
/
C
V
P
R
5
2
7
3
4
.
2
0
2
5
.
0
0
0
1
8
.
[
2
3
]
S
.
T.
Er
u
k
u
d
e
,
S
.
R
.
V
e
l
u
r
u
,
a
n
d
V
.
C
.
M
a
r
e
l
l
a
,
“
M
u
l
t
i
mo
d
a
l
d
e
e
p
l
e
a
r
n
i
n
g
:
a
su
r
v
e
y
o
f
mo
d
e
l
s,
f
u
s
i
o
n
s
t
r
a
t
e
g
i
e
s,
a
p
p
l
i
c
a
t
i
o
n
s,
a
n
d
r
e
s
e
a
r
c
h
c
h
a
l
l
e
n
g
e
s,”
I
n
t
e
r
n
a
t
i
o
n
a
l
J
o
u
r
n
a
l
o
f
C
o
m
p
u
t
e
r
A
p
p
l
i
c
a
t
i
o
n
s
,
v
o
l
.
1
8
7
,
n
o
.
1
9
,
p
p
.
1
–
7
,
J
u
l
.
2
0
2
5
,
d
o
i
:
1
0
.
5
1
2
0
/
i
j
c
a
2
0
2
5
9
2
5
2
6
4
.
[
2
4
]
E.
N
e
p
o
v
i
n
n
y
k
h
,
V
.
I
mm
o
n
e
n
,
T
.
E
e
r
o
l
a
,
C
.
V
.
S
t
e
w
a
r
t
,
a
n
d
H
.
K
ä
l
v
i
ä
i
n
e
n
,
“
R
e
‐
i
d
e
n
t
i
f
i
c
a
t
i
o
n
o
f
p
a
t
t
e
r
n
e
d
a
n
i
ma
l
s
b
y
m
u
l
t
i
‐
i
ma
g
e
f
e
a
t
u
r
e
a
g
g
r
e
g
a
t
i
o
n
a
n
d
g
e
o
m
e
t
r
i
c
s
i
mi
l
a
r
i
t
y
,
”
I
ET C
o
m
p
u
t
e
r
Vi
s
i
o
n
,
v
o
l
.
1
9
,
n
o
.
1
,
Ja
n
.
2
0
2
5
,
d
o
i
:
1
0
.
1
0
4
9
/
c
v
i
2
.
1
2
3
3
7
.
[
2
5
]
S
.
C
u
i
e
t
a
l
.
,
“
S
h
i
e
l
d
V
L
M
:
saf
e
g
u
a
r
d
i
n
g
t
h
e
m
u
l
t
i
m
o
d
a
l
i
m
p
l
i
c
i
t
t
o
x
i
c
i
t
y
v
i
a
d
e
l
i
b
e
r
a
t
i
v
e
r
e
a
s
o
n
i
n
g
w
i
t
h
LV
L
M
s:
S
h
i
e
l
d
V
LM
,
”
3
3
r
d
AC
M
I
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
r
e
n
c
e
o
n
M
u
l
t
i
m
e
d
i
a
,
2
0
2
5
,
p
p
.
1
1
6
7
7
–
1
1
6
8
6
,
d
o
i
:
1
0
.
1
1
4
5
/
3
7
4
6
0
2
7
.
3
7
5
5
7
1
1
.
[
2
6
]
M
.
T
a
l
e
b
,
A
.
H
a
mz
a
,
M
.
Z
o
u
i
t
n
i
,
N
.
B
u
r
ma
n
i
,
S
.
L
a
f
k
i
a
r
,
a
n
d
N
.
En
-
N
a
h
n
a
h
i
,
“
D
e
t
e
c
t
i
o
n
o
f
t
o
x
i
c
i
t
y
i
n
so
c
i
a
l
me
d
i
a
b
a
s
e
d
o
n
n
a
t
u
r
a
l
l
a
n
g
u
a
g
e
p
r
o
c
e
ssi
n
g
m
e
t
h
o
d
s
,
”
2
0
2
2
I
n
t
e
rn
a
t
i
o
n
a
l
C
o
n
f
e
re
n
c
e
o
n
I
n
t
e
l
l
i
g
e
n
t
S
y
st
e
m
s
a
n
d
C
o
m
p
u
t
e
r
V
i
si
o
n
,
I
S
C
V
2
0
2
2
,
2
0
2
2
,
d
o
i
:
1
0
.
1
1
0
9
/
I
S
C
V
5
4
6
5
5
.
2
0
2
2
.
9
8
0
6
0
9
6
.
[
2
7
]
X
.
Q
i
n
,
Y
.
Z
h
o
u
,
a
n
d
J.
Li
,
“
M
u
l
t
i
-
mo
d
a
l
e
m
o
t
i
o
n
r
e
c
o
g
n
i
t
i
o
n
f
o
r
o
n
l
i
n
e
e
d
u
c
a
t
i
o
n
u
si
n
g
e
m
o
j
i
p
r
o
m
p
t
s
,
”
Ap
p
l
i
e
d
S
c
i
e
n
c
e
s
,
v
o
l
.
1
4
,
n
o
.
1
2
,
J
u
n
.
2
0
2
4
,
d
o
i
:
1
0
.
3
3
9
0
/
a
p
p
1
4
1
2
5
1
4
6
.
[
2
8
]
Z.
W
e
n
,
W
.
Li
n
,
T
.
W
a
n
g
,
a
n
d
G
.
X
u
,
“
D
i
st
r
a
c
t
y
o
u
r
a
t
t
e
n
t
i
o
n
:
mu
l
t
i
-
h
e
a
d
c
r
o
ss
a
t
t
e
n
t
i
o
n
n
e
t
w
o
r
k
f
o
r
f
a
c
i
a
l
e
x
p
r
e
ssi
o
n
r
e
c
o
g
n
i
t
i
o
n
,
”
Bi
o
m
i
m
e
t
i
c
s
,
v
o
l
.
8
,
n
o
.
2
,
M
a
y
2
0
2
3
,
d
o
i
:
1
0
.
3
3
9
0
/
b
i
o
m
i
me
t
i
c
s8
0
2
0
1
9
9
.
[
2
9
]
D
o
so
v
i
t
s
k
i
y
e
t
a
l
.
,
“
A
n
i
ma
g
e
i
s
w
o
r
t
h
1
6
x
1
6
w
o
r
d
s
:
t
r
a
n
sf
o
r
m
e
r
s
f
o
r
i
m
a
g
e
r
e
c
o
g
n
i
t
i
o
n
a
t
s
c
a
l
e
,
”
I
C
L
R
2
0
2
1
-
9
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
re
n
c
e
o
n
L
e
a
rn
i
n
g
Re
p
res
e
n
t
a
t
i
o
n
s
,
2
0
2
1
,
p
p
.
1
-
21
.
[
3
0
]
J.
D
h
a
n
i
t
h
,
S
.
V
e
n
k
a
t
r
a
ma
n
,
M
.
N
a
r
e
n
d
r
a
,
V
.
S
h
a
r
m
a
,
S
.
M
a
l
a
r
v
a
n
n
a
n
,
a
n
d
A
.
H
.
G
a
n
d
o
mi
,
“
M
u
l
t
i
m
o
d
a
l
e
m
o
t
i
o
n
r
e
c
o
g
n
i
t
i
o
n
u
si
n
g
a
u
d
i
o
-
v
i
d
e
o
t
r
a
n
sf
o
r
mer f
u
si
o
n
w
i
t
h
c
r
o
ss
a
t
t
e
n
t
i
o
n
,
”
2
0
2
4
,
a
rX
i
v
:
2
4
0
7
.
1
8
5
5
2
.
[
3
1
]
Y
.
H
a
o
,
L.
D
o
n
g
,
F
.
W
e
i
,
a
n
d
K
.
X
u
,
“
S
e
l
f
-
a
t
t
e
n
t
i
o
n
a
t
t
r
i
b
u
t
i
o
n
:
i
n
t
e
r
p
r
e
t
i
n
g
i
n
f
o
r
ma
t
i
o
n
i
n
t
e
r
a
c
t
i
o
n
s
i
n
si
d
e
t
r
a
n
sf
o
r
m
e
r
,
”
3
5
t
h
AAAI C
o
n
f
e
re
n
c
e
o
n
Art
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
AAAI
2
0
2
1
,
p
p
.
1
2
9
6
3
–
1
2
9
7
1
,
2
0
2
1
,
d
o
i
:
1
0
.
1
6
0
9
/
a
a
a
i
.
v
3
5
i
1
4
.
1
7
5
3
3
.
[
3
2
]
X
.
Z
h
a
n
g
,
Z.
W
u
,
K
.
Li
u
,
Z
.
Zh
a
o
,
J
.
W
a
n
g
,
a
n
d
C
.
W
u
,
“
T
e
x
t
se
n
t
i
me
n
t
c
l
a
ss
i
f
i
c
a
t
i
o
n
b
a
se
d
o
n
B
E
R
T
e
mb
e
d
d
i
n
g
a
n
d
s
l
i
c
e
d
mu
l
t
i
-
h
e
a
d
sel
f
-
A
t
t
e
n
t
i
o
n
B
i
-
G
R
U
,
”
S
e
n
s
o
rs
,
v
o
l
.
2
3
,
n
o
.
3
,
J
a
n
.
2
0
2
3
,
d
o
i
:
1
0
.
3
3
9
0
/
s
2
3
0
3
1
4
8
1
.
[
3
3
]
O
.
N
d
a
m
a
,
I
.
B
e
n
s
a
ssi
,
S
.
N
d
a
ma
,
a
n
d
E.
M
.
E
n
-
N
a
i
m
i
,
“
U
n
i
f
i
e
d
B
ER
T
-
L
S
TM
f
r
a
m
e
w
o
r
k
e
n
h
a
n
c
e
s
mac
h
i
n
e
l
e
a
r
n
i
n
g
i
n
f
r
a
u
d
d
e
t
e
c
t
i
o
n
,
f
i
n
a
n
c
i
a
l
se
n
t
i
me
n
t
,
a
n
d
b
i
o
me
d
i
c
a
l
c
l
a
ss
i
f
i
c
a
t
i
o
n
,
”
I
AE
S
I
n
t
e
r
n
a
t
i
o
n
a
l
J
o
u
r
n
a
l
o
f
Ar
t
i
f
i
c
i
a
l
I
n
t
e
l
l
i
g
e
n
c
e
,
v
o
l
.
1
4
,
n
o
.
6
,
D
e
c
.
2
0
2
5
,
d
o
i
:
1
0
.
1
1
5
9
1
/
i
j
a
i
.
v
1
4
.
i
6
.
p
p
5
0
8
1
-
5
0
9
5
.
Evaluation Warning : The document was created with Spire.PDF for Python.