I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
p
ute
r
E
ng
in
ee
ring
(
I
J
E
CE
)
Vo
l.
9
,
No
.
3
,
J
u
n
e
201
9
,
p
p
.
2025
~
2
0
3
2
I
SS
N:
2088
-
8708
,
DOI
: 1
0
.
1
1
5
9
1
/
i
j
ec
e
.
v
9
i
3
.
p
p
2
0
2
5
-
2032
2025
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ia
e
s
co
r
e
.
co
m/
jo
u
r
n
a
ls
/in
d
ex
.
p
h
p
/
I
JE
C
E
Ara
bic
n
a
m
ed
e
nt
ity
r
ecog
nition
u
si
ng
d
eep
l
ea
rni
ng
a
ppro
a
ch
I
s
m
a
il
E
l B
a
zi,
Na
bil
L
a
a
chf
o
ub
i
IR2
M
L
a
b
o
ra
to
ry
,
F
S
T
,
Un
iv
Ha
ss
a
n
1
st,
S
e
tt
a
t,
M
o
ro
c
c
o
Art
icle
I
nfo
AB
ST
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
A
p
r
27
,
2
0
1
8
R
ev
i
s
ed
No
v
2
0
,
2
0
1
8
A
cc
ep
ted
Dec
1
0
, 2
01
8
M
o
st
o
f
th
e
A
r
a
b
ic
Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
(NER)
s
y
ste
m
s
d
e
p
e
n
d
m
a
ss
iv
e
l
y
o
n
e
x
tern
a
l
re
so
u
rc
e
s
a
n
d
h
a
n
d
m
a
d
e
f
e
a
tu
re
e
n
g
in
e
e
rin
g
to
a
c
h
iev
e
sta
te
-
of
-
th
e
-
a
rt
re
su
lt
s.
T
o
o
v
e
rc
o
m
e
su
c
h
li
m
it
a
ti
o
n
s,
w
e
p
ro
p
o
se
d
,
in
th
is
p
a
p
e
r,
t
o
u
se
d
e
e
p
lea
rn
in
g
a
p
p
r
o
a
c
h
t
o
tac
k
le
th
e
A
ra
b
ic
NER
tas
k
.
W
e
in
tro
d
u
c
e
d
a
n
e
u
ra
l
n
e
tw
o
rk
a
rc
h
it
e
c
tu
re
b
a
se
d
o
n
b
id
irec
ti
o
n
a
l
L
o
n
g
S
h
o
rt
-
T
e
rm
M
e
m
o
r
y
(L
S
T
M
)
a
n
d
Co
n
d
i
ti
o
n
a
l
Ra
n
d
o
m
F
ield
s
(CRF
)
a
n
d
e
x
p
e
ri
m
e
n
ted
w
it
h
v
a
rio
u
s
c
o
m
m
o
n
l
y
u
se
d
h
y
p
e
rp
a
ra
m
e
ters
to
a
ss
e
ss
th
e
ir
e
ffe
c
t
o
n
t
h
e
o
v
e
ra
ll
p
e
rf
o
r
m
a
n
c
e
o
f
o
u
r
s
y
ste
m
.
Ou
r
m
o
d
e
l
g
e
ts
t
w
o
so
u
rc
e
s
o
f
in
f
o
rm
a
ti
o
n
a
b
o
u
t
w
o
rd
s
a
s
in
p
u
t:
p
re
-
trai
n
e
d
w
o
rd
e
m
b
e
d
d
i
n
g
s
a
n
d
c
h
a
ra
c
ter
-
b
a
se
d
re
p
re
se
n
tatio
n
s
a
n
d
e
li
m
in
a
ted
th
e
n
e
e
d
f
o
r
a
n
y
ta
sk
-
sp
e
c
i
f
ic
k
n
o
w
led
g
e
o
r
f
e
a
tu
re
e
n
g
in
e
e
rin
g
.
We
o
b
tain
e
d
sta
te
-
of
-
th
e
-
a
rt
re
su
lt
o
n
t
h
e
sta
n
d
a
rd
A
NERc
o
rp
c
o
r
p
u
s w
it
h
a
n
F
1
sc
o
re
o
f
9
0
.
6
%
.
K
ey
w
o
r
d
s
:
A
r
ab
ic
Dee
p
l
ea
r
n
in
g
Na
m
ed
e
n
ti
t
y
r
ec
o
g
n
itio
n
NL
P
W
o
r
d
e
m
b
ed
d
in
g
s
Co
p
y
rig
h
t
©
2
0
1
9
In
stit
u
te o
f
A
d
v
a
n
c
e
d
E
n
g
i
n
e
e
rin
g
a
n
d
S
c
ien
c
e
.
Al
l
rig
h
ts re
se
rv
e
d
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
I
s
m
ail
E
l B
az
i,
I
R
2
M
L
ab
o
r
ato
r
y
,
F
ST
,
Hass
a
n
1
s
t U
n
iv
er
s
it
y
,
C
asab
lan
ca
Stre
et,
B
o
x
5
7
7
,
S
ettat,
2
6
0
0
0
,
Mo
r
o
cc
o
.
E
m
ail:
i
s
m
a
ilelb
az
i@
g
m
ail.
co
m
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
Nam
ed
E
n
tit
y
R
ec
o
g
n
iti
o
n
(
NE
R
)
task
ai
m
s
to
id
en
tify
an
d
ca
teg
o
r
ize
p
r
o
p
er
n
o
u
n
s
a
n
d
i
m
p
o
r
tan
t
n
o
u
n
s
i
n
a
te
x
t
in
to
a
s
et
o
f
p
r
ed
ef
in
ed
ca
te
g
o
r
ies
o
f
in
ter
es
t
s
u
ch
a
s
p
er
s
o
n
s
,
o
r
g
a
n
izatio
n
s
,
lo
ca
tio
n
s
,
etc.
[
1
]
NE
R
i
s
a
m
an
d
ato
r
y
p
r
ep
r
o
ce
s
s
in
g
m
o
d
u
le
in
s
e
v
er
al
n
a
tu
r
al
la
n
g
u
a
g
e
p
r
o
ce
s
s
i
n
g
(
N
L
P
)
ap
p
licatio
n
s
s
u
ch
a
s
s
y
n
tac
ti
c
p
ar
s
in
g
[
2
]
,
q
u
esti
o
n
a
n
s
wer
in
g
[
3
]
an
d
en
ti
t
y
co
r
ef
er
e
n
ce
r
eso
lu
t
io
n
[
4
]
.
A
c
h
ie
v
in
g
t
h
e
b
est
p
er
f
o
r
m
a
n
ce
o
n
NE
R
tas
k
r
eq
u
ir
es
lar
g
e
a
m
o
u
n
ts
o
f
e
x
ter
n
al
r
eso
u
r
ce
s
s
u
c
h
as
g
az
et
teer
s
,
p
len
t
y
o
f
h
an
d
-
cr
af
ted
f
ea
tu
r
e
en
g
i
n
ee
r
in
g
an
d
e
x
ten
s
i
v
e
d
ata
p
r
e
-
p
r
o
ce
s
s
in
g
.
Ho
w
ev
e
r
,
d
ev
elo
p
in
g
s
u
c
h
task
-
s
p
ec
i
f
ic
r
eso
u
r
ce
s
an
d
f
e
atu
r
es
is
co
s
tl
y
a
n
d
n
ee
d
s
a
lo
t
o
f
ti
m
e.
Fo
r
m
o
r
p
h
o
lo
g
icall
y
r
ich
lan
g
u
ag
e
s
l
ik
e
A
r
ab
ic,
th
is
ta
s
k
b
ec
o
m
es
ev
e
n
m
o
r
e
ch
alle
n
g
i
n
g
d
u
e
to
its
u
n
iq
u
e
ch
ar
ac
ter
i
s
tics
.
T
h
e
h
ig
h
l
y
ag
g
l
u
ti
n
a
ti
v
e
n
atu
r
e
o
f
A
r
ab
ic
allo
w
s
f
o
r
th
e
s
a
m
e
w
o
r
d
to
h
av
e
d
i
f
f
er
e
n
t
m
o
r
p
h
o
lo
g
ical
f
o
r
m
s
w
h
ich
g
en
er
ate
a
lo
t
o
f
d
ata
s
p
ar
s
en
es
s
.
A
ls
o
,
th
e
ab
s
e
n
ce
o
f
d
iacr
itics
in
m
o
s
t
m
o
d
er
n
s
tan
d
ar
d
A
r
ab
ic
tex
t
s
cr
ea
tes
a
lo
t
o
f
am
b
ig
u
it
y
s
in
ce
m
a
n
y
w
o
r
d
s
ca
n
s
h
ar
e
t
h
e
s
a
m
e
s
u
r
f
ac
e
f
o
r
m
w
it
h
o
u
t
d
iacr
itics
b
u
t
h
a
v
e
d
if
f
er
en
t
n
a
m
ed
e
n
tit
y
(
NE
)
tag
s
.
F
u
r
th
er
m
o
r
e,
u
n
l
ik
e
m
o
s
t
E
u
r
o
p
ea
n
lan
g
u
a
g
es,
t
h
er
e
is
n
o
ca
p
italizatio
n
i
n
A
r
ab
ic.
T
h
er
ef
o
r
e,
it
is
n
o
t
p
o
s
s
ib
le
to
u
s
e
ca
p
italizatio
n
as
f
ea
t
u
r
e
in
d
icato
r
to
d
etec
t
n
a
m
ed
en
titi
e
s
.
Fi
n
all
y
,
th
er
e
ar
e
a
v
er
y
li
m
i
ted
n
u
m
b
er
o
f
lin
g
u
is
tic
r
eso
u
r
ce
s
s
u
c
h
as
g
az
e
tteer
s
an
d
NE
an
n
o
tated
co
r
p
o
r
a,
f
r
ee
ly
av
a
il
ab
le
f
o
r
r
esear
ch
er
s
to
b
u
ild
d
ec
en
t A
r
ab
ic
NE
R
s
y
s
te
m
s
.
Ma
in
l
y
,
th
e
r
esear
c
h
er
s
i
n
ter
e
s
ted
in
NE
R
f
o
r
t
h
e
A
r
ab
ic
lan
g
u
a
g
e
f
o
llo
w
t
h
r
ee
ap
p
r
o
ac
h
es:
r
u
le
-
b
ased
[
5
]
,
[
6
]
,
m
ac
h
i
n
e
lear
n
i
n
g
(
ML
)
-
b
ased
[
7
]
,
[
8
]
an
d
h
y
b
r
id
ap
p
r
o
ac
h
es
[9
]
-
[
11]
.
T
h
ese
th
r
ee
ap
p
r
o
ac
h
es
s
u
f
f
er
f
r
o
m
t
h
e
s
a
m
e
is
s
u
e
s
s
in
ce
it
n
ee
d
s
a
lo
t
o
f
la
n
g
u
ag
e
-
s
p
ec
if
ic
k
n
o
w
led
g
e
a
n
d
an
e
x
ten
s
i
v
e
f
ea
t
u
r
e
en
g
i
n
ee
r
i
n
g
to
o
b
tain
u
s
e
f
u
l
r
esu
lt
s
.
T
h
is
is
ev
e
n
m
o
r
e
ac
ce
n
tu
a
ted
b
y
t
h
e
lac
k
o
f
li
n
g
u
i
s
t
ic
r
eso
u
r
ce
s
an
d
th
e
co
m
p
le
x
m
o
r
p
h
o
lo
g
y
o
f
t
h
e
la
n
g
u
a
g
e.
R
ec
en
t
l
y
,
t
h
e
Dee
p
L
ea
r
n
i
n
g
(
DL
)
[
1
2
]
p
a
r
ad
ig
m
h
as
e
m
er
g
ed
an
d
m
ad
e
i
m
p
r
es
s
iv
e
ac
h
i
ev
e
m
en
t
s
in
f
ield
s
s
u
c
h
as
s
p
ee
ch
p
r
o
ce
s
s
i
n
g
[
1
3
]
an
d
i
m
ag
e
r
ec
o
g
n
i
tio
n
[
1
4
]
.
Fo
r
NL
P
,
th
e
ap
p
licatio
n
o
f
d
ee
p
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8708
I
n
t J
E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
9
,
No
.
3
,
J
u
n
e
201
9
:
2
0
2
5
-
2
0
3
2
2026
lear
n
in
g
h
as
p
r
o
v
en
to
b
e
v
er
y
e
f
f
ec
tiv
e
y
ield
in
g
s
tate
-
of
-
t
h
e
-
ar
t
in
v
ar
io
u
s
co
m
m
o
n
N
L
P
task
s
as
s
eq
u
e
n
ce
lab
elin
g
[
1
5
]
,
s
en
ti
m
e
n
t
an
a
l
y
s
i
s
[
1
6
]
,
[
1
7
]
an
d
m
ac
h
i
n
e
tr
an
s
lat
i
o
n
[
1
8
]
f
o
r
th
e
E
n
g
lis
h
lan
g
u
a
g
e.
Un
l
ik
e
tr
ad
itio
n
al
ap
p
r
o
ac
h
es,
DL
is
an
en
d
-
to
-
en
d
m
o
d
el
th
at
d
i
d
n
o
t
r
ely
o
n
d
ata
p
r
ep
r
o
ce
s
s
in
g
,
m
an
u
al
f
ea
tu
r
e
en
g
i
n
ee
r
i
n
g
o
r
lar
g
e
a
m
o
u
n
t
s
task
-
s
p
ec
i
f
ic
r
eso
u
r
ce
s
an
d
ca
n
b
e
ad
ap
ted
t
o
v
ar
io
u
s
lan
g
u
ag
es
an
d
d
o
m
ai
n
s
.
T
h
is
m
ak
e
s
it a
v
er
y
attr
ac
ti
v
e
s
o
lu
tio
n
f
o
r
co
m
p
lex
a
n
d
lo
w
r
eso
u
r
ce
lan
g
u
a
g
e
li
k
e
A
r
ab
ic.
Mo
tiv
ated
b
y
th
e
s
u
cc
ess
o
f
d
ee
p
lear
n
in
g
i
n
s
ev
er
al
N
L
P
ap
p
licatio
n
s
,
w
e
i
n
tr
o
d
u
ce
a
n
Ar
ab
ic
NE
R
s
y
s
te
m
b
ased
o
n
d
ee
p
n
eu
r
al
n
et
w
o
r
k
s
.
I
n
th
e
D
L
liter
at
u
r
e
tw
o
n
e
u
r
al
n
et
w
o
r
k
ar
ch
itect
u
r
es
ar
e
w
id
el
y
u
s
ed
:
co
n
v
o
lu
tio
n
al
n
e
u
r
al
n
et
w
o
r
k
s
(
C
N
N)
[
1
9
]
an
d
lo
n
g
-
s
h
o
r
t
-
ter
m
m
e
m
o
r
y
(
L
ST
M)
[
2
0
]
.
T
h
u
s
t
h
e
n
e
u
r
al
n
et
w
o
r
k
ar
ch
itect
u
r
e
t
h
at
w
e
in
tr
o
d
u
ce
o
n
th
i
s
p
ap
er
e
m
b
r
ac
es
b
o
th
m
o
d
els.
W
e
e
m
p
l
o
y
C
NN
to
in
d
u
ce
ch
ar
ac
ter
-
le
v
el
r
ep
r
esen
ta
tio
n
s
o
f
w
o
r
d
s
an
d
w
e
f
ee
d
it
in
co
n
j
u
n
ctio
n
w
i
th
w
o
r
d
e
m
b
ed
d
in
g
s
to
a
b
id
ir
ec
tio
n
al
L
ST
M
n
et
w
o
r
k
(
B
iL
ST
M)
th
at
p
er
f
o
r
m
t
h
e
tr
ai
n
in
g
.
Fin
a
ll
y
,
w
e
u
s
e
a
co
n
d
iti
o
n
al
r
an
d
o
m
f
ie
ld
s
(
C
R
F)
[
2
1
]
lay
er
to
d
o
th
e
d
ec
o
d
in
g
o
f
t
h
e
in
p
u
t seq
u
e
n
ce
.
Sin
ce
t
h
e
ca
r
e
f
u
l
s
elec
tio
n
o
f
o
p
ti
m
al
p
ar
a
m
e
ter
s
ca
n
o
f
ten
m
a
k
e
a
h
u
g
e
d
i
f
f
er
e
n
ce
in
t
h
e
p
er
f
o
r
m
a
n
ce
o
f
n
e
u
r
al
n
et
w
o
r
k
ar
c
h
itect
u
r
e,
w
e
t
h
o
r
o
u
g
h
l
y
in
v
est
ig
ated
t
h
e
i
m
p
ac
t
o
f
d
i
v
er
s
e
h
y
p
er
p
ar
a
m
eter
s
o
n
th
e
o
v
er
a
ll
p
er
f
o
r
m
an
ce
o
f
t
h
e
ch
o
s
e
n
n
eu
r
al
ar
c
h
itect
u
r
e
an
d
s
elec
t
ed
th
e
b
est
o
n
e
s
f
o
r
o
u
r
f
i
n
al
m
o
d
el.
Ou
r
m
ai
n
co
n
tr
ib
u
tio
n
s
o
f
t
h
is
p
a
p
er
a
r
e
as f
o
llo
w
s
:
a.
P
r
o
p
o
s
in
g
a
d
ee
p
lear
n
in
g
ap
p
r
o
ac
h
to
ad
d
r
ess
th
e
A
r
ab
ic
N
E
R
tas
k
.
b.
E
v
alu
a
tin
g
a
n
d
s
elec
ti
n
g
t
h
e
o
p
ti
m
al
h
y
p
er
p
ar
a
m
eter
s
f
o
r
th
e
p
r
o
p
o
s
ed
n
eu
r
al
n
et
w
o
r
k
ar
c
h
itect
u
r
e.
c.
C
o
n
f
ir
m
in
g
t
h
e
ad
v
an
ta
g
e
o
f
i
n
teg
r
ati
n
g
c
h
ar
ac
ter
-
b
ase
d
r
ep
r
esen
tatio
n
s
f
o
r
m
o
r
p
h
o
lo
g
icall
y
r
ic
h
lan
g
u
a
g
es li
k
e
A
r
ab
ic.
d.
A
c
h
ie
v
in
g
s
tate
-
of
-
th
e
-
ar
t
r
es
u
lts
o
n
th
e
s
tan
d
ar
d
ANE
R
C
o
r
p
co
r
p
u
s
w
it
h
o
u
t
th
e
n
ee
d
o
f
an
y
f
ea
t
u
r
e
en
g
i
n
ee
r
i
n
g
o
r
d
o
m
ain
-
s
p
ec
if
i
c
k
n
o
w
led
g
e
.
2.
P
RO
P
O
SE
D
AP
P
RO
ACH
I
n
th
is
s
ec
tio
n
,
w
e
o
u
tl
in
e
th
e
d
ee
p
lear
n
in
g
ap
p
r
o
ac
h
th
at
we
ad
o
p
ted
to
tack
le
th
e
NE
R
t
ask
f
o
r
t
h
e
A
r
ab
ic
lan
g
u
ag
e.
W
e
p
r
o
p
o
s
e
n
eu
r
al
n
et
w
o
r
k
ar
ch
itect
u
r
e
co
m
p
o
s
ed
o
f
a
B
iL
ST
M
la
y
e
r
an
d
a
C
R
F
la
y
er
.
First,
w
e
co
m
p
u
te
th
e
c
h
ar
ac
t
er
r
ep
r
esen
tatio
n
f
o
r
ea
ch
w
o
r
d
u
s
i
n
g
eit
h
er
C
NN
o
r
B
iL
ST
M
(
s
ee
Sectio
n
2
.
5
f
o
r
d
etails),
th
e
n
w
e
co
n
ca
te
n
ate
it
w
it
h
th
e
w
o
r
d
e
m
b
ed
d
i
n
g
s
b
ef
o
r
e
f
ee
d
i
n
g
in
to
t
h
e
B
iL
ST
M
la
y
er
.
T
h
is
la
y
er
i
s
co
m
p
o
s
ed
o
f
t
w
o
L
S
T
M
n
et
w
o
r
k
s
.
T
h
e
f
o
r
w
ar
d
L
ST
M
r
ea
d
s
th
e
w
o
r
d
s
eq
u
e
n
c
e
f
r
o
m
t
h
e
b
e
g
in
n
i
n
g
w
h
e
n
th
e
b
ac
k
w
ar
d
L
ST
M
r
ea
d
s
it
in
o
p
p
o
s
ite
o
r
d
er
.
Fin
all
y
,
th
e
o
u
tp
u
t
v
ec
to
r
s
o
f
b
o
t
h
L
ST
M
n
et
w
o
r
k
s
ar
e
co
n
ca
ten
ated
a
n
d
s
en
t
as
i
n
p
u
t
to
t
h
e
C
R
F
la
y
er
to
g
e
n
er
at
e
th
e
tag
s
p
r
ed
ictio
n
f
o
r
t
h
e
i
n
p
u
t
s
eq
u
e
n
ce
.
T
h
e
ar
ch
itect
u
r
e
o
f
o
u
r
n
e
u
r
al
n
et
w
o
r
k
is
illu
s
tr
ated
i
n
d
etail
i
n
Fig
u
r
e
1
.
W
e
b
r
ief
l
y
d
esc
r
ib
e
th
e
la
y
er
s
o
f
o
u
r
m
o
d
el
i
n
t
h
e
f
o
llo
w
i
n
g
s
ec
tio
n
s
.
Fig
u
r
e
1
.
T
h
e
m
ai
n
ar
ch
itect
u
r
e
o
f
o
u
r
n
eu
r
al
B
i
L
ST
M
-
C
R
F
n
et
w
o
r
k
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J
E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8708
A
r
a
b
ic
n
a
med
e
n
tity reco
g
n
itio
n
u
s
in
g
d
ee
p
lea
r
n
in
g
a
p
p
r
o
a
ch
(
I
s
ma
il E
l B
a
z
i
)
2027
2
.
1
.
L
ST
M
L
o
n
g
-
s
h
o
r
t
ter
m
m
e
m
o
r
y
(
L
ST
M)
n
et
w
o
r
k
s
ar
e
v
ar
ia
n
ts
o
f
r
ec
u
r
r
en
t
n
e
u
r
al
n
et
w
o
r
k
s
(
R
NN
)
s
p
ec
iall
y
d
esi
g
n
ed
to
ad
d
r
es
s
s
o
m
e
w
ell
-
k
n
o
w
n
is
s
u
e
s
r
elate
d
to
ex
p
lo
d
in
g
an
d
v
a
n
is
h
i
n
g
g
r
ad
ien
t
b
y
ap
p
en
d
in
g
a
n
e
x
tr
a
m
e
m
o
r
y
-
c
ell.
L
ST
Ms
ar
e
v
er
y
e
f
f
ec
ti
v
e
to
ca
p
tu
r
e
lo
n
g
-
d
i
s
tan
ce
d
ep
en
d
en
cie
s
.
T
h
e
y
ta
k
e
as
in
p
u
t
a
s
eq
u
e
n
ce
o
f
v
ec
to
r
s
(
x
1
,x
2
,
…,
x
n
)
o
f
le
n
g
th
n
a
n
d
r
etu
r
n
a
n
o
u
tp
u
t
s
eq
u
e
n
ce
o
f
v
ec
to
r
s
(
h
1
,h
2
,
…,
h
n
)
ca
lled
h
id
d
en
s
tate
s
.
T
h
e
L
ST
M
i
m
p
le
m
en
ta
tio
n
u
s
ed
is
r
ep
r
esen
ted
b
y
t
h
e
f
o
llo
w
i
n
g
f
o
r
m
u
las at
ti
m
e
t:
=
(
+
ℎ
ℎ
−
1
+
−
1
)
+
(
1
)
=
(
1
−
)
ʘ
−
1
+
ʘ
ta
n
h
(
+
ℎ
−
1
+
)
(
2
)
=
(
+
ℎ
ℎ
−
1
+
+
)
(
3
)
ℎ
=
ʘ
ta
n
h
(
)
(
4
)
w
h
er
e
σ
d
en
o
tes
t
h
e
ele
m
e
n
t
-
w
i
s
e
s
i
g
m
o
id
f
u
n
ctio
n
an
d
ʘ
th
e
ele
m
en
t
-
w
i
s
e
p
r
o
d
u
ct.
i
t
is
th
e
i
n
p
u
t
g
ate
v
ec
to
r
,
c
t
th
e
ce
ll st
ate
v
ec
to
r
an
d
o
t
th
e
o
u
tp
u
t
g
ate
v
ec
to
r
.
A
ll W
an
d
b
ar
e
tr
ain
ab
le
p
ar
am
eter
s
.
2
.
2
.
B
iL
ST
M
Desp
ite
th
e
ir
ca
p
ab
ilit
y
to
c
ap
tu
r
e
lo
n
g
-
d
is
tan
ce
d
ep
en
d
en
cies,
s
tan
d
ar
d
L
ST
Ms
ar
e
n
o
t
v
er
y
ef
f
ec
tiv
e
o
n
s
eq
u
e
n
ce
ta
g
g
in
g
tas
k
s
l
ik
e
NE
R
.
I
n
f
ac
t,
a
n
L
ST
M
u
n
it
ca
n
tak
e
in
f
o
r
m
at
io
n
o
n
l
y
f
r
o
m
p
a
s
t
co
n
tex
t,
b
u
t
f
o
r
s
eq
u
en
ce
tag
g
in
g
is
v
er
y
u
s
ef
u
l
to
r
etr
iev
e
b
o
th
p
ast
an
d
f
u
tu
r
e
in
f
o
r
m
a
tio
n
.
T
o
o
v
er
co
m
e
th
is
co
n
s
tr
ai
n
t
w
e
u
s
e
b
id
ir
ec
t
io
n
al
L
ST
M.
T
h
e
b
asic
id
ea
i
s
th
at
w
e
w
ill
u
s
e
t
w
o
s
ep
ar
ate
L
ST
M
u
n
its
.
T
h
e
f
ir
s
t
o
n
e
is
a
f
o
r
w
ar
d
L
ST
M
th
at
r
ea
d
s
th
e
s
eq
u
en
ce
o
f
w
o
r
d
s
an
d
in
d
u
ce
s
a
r
ep
r
ese
n
tatio
n
o
f
t
h
e
p
ast
co
n
tex
t.
T
h
e
s
ec
o
n
d
o
n
e
is
a
b
ac
k
w
ar
d
L
ST
M
th
at
tak
e
s
th
e
s
a
m
e
s
eq
u
e
n
ce
b
u
t
i
n
r
e
v
er
s
e
an
d
in
d
u
ce
s
a
r
ep
r
esen
tati
o
n
o
f
th
e
f
u
t
u
r
e
co
n
tex
t.
T
h
e
f
in
a
l
r
ep
r
esen
tati
o
n
o
f
a
w
o
r
d
is
t
h
e
co
m
b
i
n
at
io
n
o
f
it
s
p
ast
a
n
d
f
u
tu
r
e
co
n
te
x
t r
ep
r
esen
ta
tio
n
s
.
2
.
3
.
CRF
l
a
y
er
T
o
p
r
ed
ict
th
e
f
in
al
ta
g
s
eq
u
en
ce
f
o
r
t
h
e
i
n
p
u
t
s
e
n
ten
ce
,
w
e
f
ee
d
th
e
o
u
tp
u
t o
f
t
h
e
B
i
L
ST
M
la
y
er
to
a
class
i
f
ie
r
.
A
v
er
y
s
i
m
p
le
ex
a
m
p
le
o
f
class
i
f
ier
la
y
er
i
s
s
o
f
t
m
ax
.
I
t is s
u
itab
le
f
o
r
s
i
m
p
le
tas
k
s
w
h
er
e
th
e
o
u
tp
u
t
tag
s
ar
e
in
d
ep
en
d
en
t.
Fo
r
m
o
r
e
co
m
p
le
x
s
eq
u
e
n
ce
tag
g
in
g
tas
k
s
li
k
e
NE
R
,
w
h
er
e
w
e
h
a
v
e
s
tr
o
n
g
d
ep
en
d
en
cies
b
et
w
ee
n
o
u
tp
u
t
tag
s
,
th
e
in
d
ep
en
d
e
n
ce
a
s
s
u
m
p
tio
n
s
ar
e
n
o
t
v
alid
.
A
ct
u
all
y
,
in
NE
R
w
it
h
I
OB
2
f
o
r
m
at
I
-
L
OC
ca
n
n
o
t
f
o
llo
w
B
-
P
E
R
.
Hen
ce
,
i
n
s
tead
o
f
d
ec
o
d
in
g
ea
ch
tag
i
n
d
ep
en
d
en
tl
y
,
w
e
j
o
in
tl
y
d
ec
o
d
e
th
e
tag
p
r
ed
ictio
n
s
u
ti
lizi
n
g
a
co
n
d
itio
n
al
r
an
d
o
m
f
ield
co
m
p
o
n
en
t
w
h
ich
m
a
x
i
m
izes
t
h
e
t
ag
s
p
r
o
b
ab
ilit
ies
o
f
th
e
w
h
o
le
s
en
te
n
ce
.
2
.
4
.
Wo
rd
e
m
bedd
i
ng
s
W
o
r
d
em
b
ed
d
in
g
s
ar
e
d
e
n
s
e
l
o
w
-
d
i
m
e
n
s
io
n
al
r
ea
l
-
v
alu
ed
v
ec
to
r
s
lear
n
ed
o
v
er
u
n
lab
eled
d
ata
u
s
i
n
g
u
n
s
u
p
er
v
is
ed
ap
p
r
o
ac
h
es.
E
ac
h
w
o
r
d
in
an
i
n
p
u
t
s
e
n
te
n
ce
ca
n
b
e
m
ap
p
ed
to
a
p
r
e
-
tr
ain
ed
w
o
r
d
em
b
ed
d
in
g
.
Fo
r
u
n
s
ee
n
w
o
r
d
s
,
w
o
r
d
e
m
b
ed
d
in
g
h
as
a
v
er
y
g
o
o
d
g
en
er
aliza
tio
n
s
in
ce
it
p
o
ten
tial
l
y
ca
p
tu
r
es
u
s
e
f
u
l
s
e
m
a
n
tic
an
d
s
y
n
tact
ic
p
r
o
p
er
ties
b
et
w
ee
n
w
o
r
d
s
.
T
h
ese
in
t
er
esti
n
g
c
h
ar
ac
ter
is
tic
s
,
allo
w
it
to
s
i
g
n
i
f
ica
n
tl
y
b
o
o
s
t
th
e
p
er
f
o
r
m
a
n
ce
o
f
v
ar
io
u
s
NL
P
tas
k
s
[
1
5
]
,
[
2
2
]
.
Fo
r
o
u
r
n
eu
r
al
n
et
w
o
r
k
ar
ch
itect
u
r
e,
w
e
u
s
e
p
r
etr
ain
ed
w
o
r
d
e
m
b
ed
d
in
g
s
a
s
in
p
u
t to
ef
f
icie
n
tl
y
in
it
ialize
th
e
lo
o
k
u
p
tab
le
o
f
o
u
r
m
o
d
el.
2
.
5
.
Cha
ra
c
t
er
r
epre
s
ent
a
t
io
ns
T
h
e
u
s
e
o
f
w
o
r
d
e
m
b
ed
d
in
g
s
is
u
s
u
all
y
s
u
f
f
icie
n
t
to
g
et
th
e
b
est
p
er
f
o
r
m
a
n
ce
f
o
r
th
e
E
n
g
lis
h
lan
g
u
a
g
e.
Fo
r
m
o
r
p
h
o
lo
g
ica
ll
y
r
ic
h
lan
g
u
a
g
e
s
lik
e
A
r
ab
ic,
th
e
r
ich
n
e
s
s
o
f
t
h
e
m
o
r
p
h
o
lo
g
i
ca
l
f
o
r
m
s
m
a
k
e
th
e
v
o
ca
b
u
lar
y
s
izes
lar
g
er
an
d
t
h
e
o
u
t
-
of
-
v
o
ca
b
u
lar
y
(
OOV)
r
a
te
r
elativ
el
y
h
ig
h
er
.
He
n
ce
th
e
n
ee
d
s
o
f
a
n
o
th
er
r
ep
r
esen
tatio
n
o
f
w
o
r
d
b
ased
o
n
it
s
c
h
ar
ac
ter
s
to
e
f
f
ec
ti
v
el
y
ca
p
t
u
r
e
t
h
e
o
r
th
o
g
r
ap
h
ic
an
d
m
o
r
p
h
o
lo
g
ical
in
f
o
r
m
atio
n
s
u
ch
as
p
r
e
-
an
d
s
u
f
f
i
x
e
s
o
f
w
o
r
d
s
an
d
e
n
co
d
e
it
in
to
n
e
u
r
al
r
ep
r
esen
tat
io
n
s
t
h
at
ca
n
b
e
u
s
ed
b
y
o
u
r
m
o
d
el.
Ma
i
n
l
y
,
t
h
er
e
ar
e
t
w
o
w
a
y
s
to
lear
n
c
h
ar
ac
ter
r
ep
r
esen
tatio
n
s
.
W
e
ca
n
u
s
e
c
o
n
v
o
l
u
tio
n
al
n
e
u
r
al
n
et
w
o
r
k
s
[
1
5
]
to
en
co
d
e
a
ch
ar
ac
ter
-
b
ased
r
ep
r
esen
tatio
n
o
f
a
w
o
r
d
.
Fi
g
u
r
e
2
s
h
o
w
s
th
e
C
NN
ar
ch
itec
tu
r
e
u
s
ed
.
O
n
th
e
o
t
h
er
h
a
n
d
,
w
e
ca
n
al
s
o
u
s
e
b
id
ir
ec
tio
n
al
L
ST
Ms
[
2
2
]
to
g
en
er
ate
a
ch
ar
ac
ter
-
b
ased
r
ep
r
esen
tatio
n
o
f
a
w
o
r
d
f
r
o
m
its
ch
ar
ac
ter
s
.
Fi
g
u
r
e
3
d
escr
ib
es th
e
B
i
L
ST
M
ar
ch
itectu
r
e.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8708
I
n
t J
E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
9
,
No
.
3
,
J
u
n
e
201
9
:
2
0
2
5
-
2
0
3
2
2028
Fig
u
r
e
2
.
C
h
ar
ac
ter
-
b
ased
r
ep
r
esen
tat
io
n
u
s
i
n
g
C
NN
Fig
u
r
e
3
.
C
h
ar
ac
ter
-
b
ased
r
ep
r
esen
tat
io
n
u
s
i
n
g
B
iL
ST
M
3.
E
XP
E
R
I
M
E
NT
S
T
h
is
s
ec
tio
n
p
r
o
v
id
es
d
etail
s
ab
o
u
t
th
e
tr
ain
in
g
o
f
o
u
r
n
e
u
r
al
n
et
w
o
r
k
.
Si
n
ce
t
h
e
ac
h
ie
v
e
m
en
t
o
f
s
tate
-
of
-
th
e
-
ar
t
r
es
u
lt
s
u
s
i
n
g
n
e
u
r
al
n
et
w
o
r
k
s
r
eq
u
ir
es
th
e
s
elec
tio
n
an
d
o
p
ti
m
izatio
n
o
f
m
a
n
y
h
y
p
er
p
ar
a
m
eter
s
,
w
e
w
il
l
also
s
tu
d
y
t
h
e
i
m
p
ac
t
o
f
t
h
e
h
y
p
er
p
ar
am
eter
s
an
d
th
e
p
ar
a
m
e
ter
in
itial
izatio
n
o
n
t
h
e
o
v
er
all
p
er
f
o
r
m
a
n
ce
o
f
o
u
r
m
o
d
els.
W
e
w
ill
p
r
ec
is
el
y
e
v
al
u
ate
th
e
i
m
p
ac
t
o
f
t
h
e
f
o
llo
w
in
g
h
y
p
er
p
ar
a
m
eter
s
:
p
r
e
-
tr
ain
ed
w
o
r
d
e
m
b
ed
d
in
g
s
,
ch
ar
ac
ter
r
ep
r
esen
tatio
n
,
d
r
o
p
o
u
t,
an
d
o
p
ti
m
izer
s
.
3
.
1
.
Net
wo
rk
t
ra
ini
ng
Ou
r
n
e
u
r
al
m
o
d
el
is
i
m
p
le
m
en
ted
u
s
i
n
g
Ker
as
A
P
I
w
it
h
th
e
T
h
ea
n
o
lib
r
ar
y
as
a
b
ac
k
en
d
[
2
3
]
.
T
h
e
tr
ain
in
g
is
d
o
n
e
u
s
i
n
g
t
h
e
b
ac
k
-
p
r
o
p
ag
atio
n
alg
o
r
it
h
m
w
it
h
t
h
e
A
d
a
m
o
p
ti
m
izer
.
W
e
u
s
e
g
r
ad
ien
t
n
o
r
m
aliza
t
io
n
o
f
1
to
d
ea
l
w
i
th
“g
r
ad
i
en
t
e
x
p
lo
d
in
g
”.
Fo
r
all
o
u
r
ex
p
er
i
m
en
t
s
,
w
e
r
u
n
t
h
e
tr
ain
in
g
w
i
th
t
h
e
m
i
n
i
-
b
atch
s
ize
o
f
8
f
o
r
5
0
ep
o
ch
s
a
n
d
ap
p
l
y
ea
r
l
y
s
to
p
p
in
g
o
f
5
b
ased
o
n
t
h
e
p
er
f
o
r
m
a
n
ce
o
n
t
h
e
v
alid
atio
n
s
et.
T
h
e
r
em
ai
n
i
n
g
d
ef
au
lt sett
in
g
s
o
f
t
h
e
h
y
p
er
p
ar
a
m
eter
s
ar
e
s
u
m
m
ar
ized
in
T
ab
le
1
.
T
ab
le
1
.
T
h
e
Def
au
lt H
y
p
er
p
ar
a
m
eter
s
o
f
t
h
e
N
et
w
o
r
k
L
a
y
e
r
H
y
p
e
r
p
a
r
a
me
t
e
r
V
a
l
u
e
C
N
N
w
i
n
d
o
w
si
z
e
3
n
u
m
b
e
r
o
f
f
i
l
t
e
r
s
30
L
S
T
M
st
a
t
e
s
i
z
e
50
n
u
m
b
e
r
o
f
l
a
y
e
r
s
2
D
r
o
p
o
u
t
D
r
o
p
o
u
t
t
y
p
e
N
a
i
v
e
d
r
o
p
o
u
t
r
a
t
e
0
.
5
3
.
2
.
P
re
-
t
ra
ined wo
rd
e
m
be
dd
ing
s
W
e
e
m
p
lo
y
p
r
etr
ain
ed
w
o
r
d
r
ep
r
esen
tatio
n
s
to
in
i
tialize
o
u
r
lo
o
k
u
p
tab
le.
W
e
lear
n
ed
o
u
r
o
w
n
w
o
r
d
e
m
b
ed
d
in
g
s
u
s
i
n
g
t
h
e
A
r
ab
ic
W
ik
ip
ed
ia
d
u
m
p
o
f
Dec
e
m
b
er
2
0
1
6
w
it
h
a
d
i
m
e
n
s
io
n
o
f
5
0
.
T
o
ass
ess
i
f
t
h
e
ch
o
ice
o
f
th
e
lear
n
i
n
g
al
g
o
r
it
h
m
is
r
elev
a
n
t,
w
e
e
x
p
er
i
m
e
n
t
w
it
h
5
m
o
d
els
n
a
m
el
y
,
S
k
ip
Gr
a
m
[2
4
]
,
C
B
OW
[2
5
]
,
Glo
Ve
[2
6
]
,
Fas
tT
ex
t
[2
7
]
an
d
Helli
n
g
er
P
C
A
(
H
-
P
C
A
)
[2
8
]
.
W
e
also
a
s
s
es
s
t
h
e
i
m
p
ac
t
o
f
t
h
e
v
ec
to
r
s
ize
b
y
v
ar
y
in
g
it
f
o
r
th
e
b
est
p
er
f
o
r
m
in
g
al
g
o
r
ith
m
b
et
w
ee
n
5
0
an
d
5
0
0
.
3
.
3
.
Cha
ra
c
t
er
r
epre
s
ent
a
t
io
ns
I
n
th
is
e
x
p
er
i
m
e
n
t,
w
e
ch
ec
k
if
t
h
e
u
s
e
o
f
c
h
ar
ac
ter
r
ep
r
esen
tatio
n
is
h
el
p
f
u
l
an
d
ca
n
r
ea
l
l
y
h
av
e
a
tan
g
ib
le
i
m
p
ac
t
o
n
t
h
e
p
er
f
o
r
m
an
ce
o
f
t
h
e
n
et
w
o
r
k
.
A
d
d
itio
n
all
y
,
w
e
co
m
p
ar
e
t
h
e
C
NN
an
d
B
iL
ST
M
ap
p
r
o
ac
h
es
o
f
lear
n
in
g
c
h
ar
ac
ter
-
b
ased
r
ep
r
esen
tatio
n
s
a
n
d
an
al
y
ze
w
h
ic
h
o
n
e
to
b
e
p
r
ef
er
r
ed
in
r
eg
ar
d
to
p
er
f
o
r
m
a
n
ce
.
3
.
4
.
Dro
po
ut
Dr
o
p
o
u
t
is
a
k
ey
m
et
h
o
d
to
r
e
g
u
lar
ize
t
h
e
n
e
u
r
al
m
o
d
el
an
d
m
iti
g
ate
o
v
er
f
it
tin
g
.
I
n
th
i
s
ex
p
er
i
m
e
n
t,
w
e
e
v
alu
a
te
th
r
ee
s
et
u
p
s
:
N
o
d
r
o
p
o
u
t,
n
aiv
e
d
r
o
p
o
u
t,
an
d
v
ar
iatio
n
al
d
r
o
p
o
u
t
[2
9
]
.
T
h
e
d
r
o
p
o
u
t
r
ate
is
s
elec
ted
f
r
o
m
t
h
e
s
et
{0
.
2
5
,
0
.
5
,
0
.
7
5
}.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J
E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8708
A
r
a
b
ic
n
a
med
e
n
tity reco
g
n
itio
n
u
s
in
g
d
ee
p
lea
r
n
in
g
a
p
p
r
o
a
ch
(
I
s
ma
il E
l B
a
z
i
)
2029
3
.
5
.
O
ptim
izer
T
h
e
o
p
tim
izer
is
an
alg
o
r
it
h
m
th
at
h
elp
s
u
s
to
m
i
n
i
m
ize
th
e
o
b
j
ec
tiv
e
f
u
n
ctio
n
o
f
t
h
e
n
eu
r
al
n
et
w
o
r
k
.
T
h
e
ch
o
ice
o
f
a
n
o
p
ti
m
izer
ca
n
in
f
l
u
e
n
ce
b
o
th
t
h
e
p
er
f
o
r
m
a
n
ce
a
n
d
t
h
e
tr
ai
n
i
n
g
ti
m
e
o
f
o
u
r
m
o
d
el.
W
e
ex
p
er
im
e
n
t
w
it
h
6
p
o
p
u
la
r
o
p
tim
izer
s
.
Na
m
e
l
y
,
Sto
c
h
as
tic
Gr
ad
ien
t
Descen
t
(
SGD)
,
A
d
a
g
r
ad
,
A
d
ad
elta,
A
d
a
m
,
Nad
a
m
,
an
d
R
MSP
r
o
p
.
3
.
6
.
Da
t
a
s
et
s
T
o
ev
alu
ate
th
e
i
m
p
ac
t
o
f
h
y
p
er
p
ar
a
m
et
er
s
w
e
u
s
e
t
h
e
A
r
ab
ic
W
ik
ip
ed
ia
n
a
m
ed
e
n
t
it
y
co
r
p
u
s
(
A
QM
AR
)
[
30
]
.
I
t
is
a
s
m
all
an
n
o
tated
co
r
p
u
s
o
f
7
4
K
to
k
en
th
a
t
w
e
ch
o
o
s
e
it
f
o
r
co
n
v
en
ien
ce
d
u
e
to
th
e
li
m
ited
co
m
p
u
tatio
n
p
o
w
er
th
at
w
e
h
av
e
to
r
u
n
o
u
r
ex
p
er
i
m
en
ts
.
T
h
e
co
r
p
u
s
s
tati
s
tics
ar
e
d
ep
icted
in
T
a
b
le
2
.
T
ab
le
2
.
A
QM
A
R
C
o
r
p
u
s
S
tat
is
tics
D
a
t
a
se
t
S
e
n
t
e
n
c
e
s
W
o
r
d
s
En
t
i
t
i
e
s
T
r
a
i
n
1
9
7
6
5
2
6
5
0
3
7
8
1
D
e
v
3
3
6
1
0
6
4
0
1
0
9
9
T
e
st
3
7
6
1
0
5
6
4
9
7
4
Fo
r
th
e
co
m
p
ar
is
o
n
w
i
th
p
r
ev
io
u
s
l
y
s
tate
-
of
-
th
e
-
ar
t
A
r
ab
ic
NE
R
s
y
s
te
m
s
,
w
e
u
s
e
th
e
A
NE
R
C
o
r
p
co
r
p
u
s
.
I
t
is
a
p
u
b
licl
y
av
a
ila
b
le
d
ataset
a
n
d
co
n
s
id
er
ed
th
e
s
ta
n
d
ar
d
b
en
c
h
m
ar
k
f
o
r
th
e
A
r
ab
ic
NE
R
tas
k
.
T
h
e
co
r
p
u
s
s
tati
s
tics
ar
e
s
u
m
m
ar
ized
in
T
ab
le
3
.
T
h
e
tr
ain
in
g
o
f
n
e
u
r
al
n
et
wo
r
k
s
is
a
v
er
y
n
on
-
d
eter
m
i
n
is
tic
p
r
o
ce
s
s
as
it
t
y
p
icall
y
d
ep
en
d
s
o
th
e
r
an
d
o
m
n
u
m
b
er
g
e
n
er
ato
r
to
in
itialize
t
h
e
w
ei
g
h
ts
o
f
t
h
e
n
et
w
o
r
k
[
3
0
]
.
T
o
m
iti
g
ate
th
e
i
m
p
ac
t
o
f
t
h
is
o
b
s
er
v
ed
r
an
d
o
m
n
es
s
in
t
h
e
ev
alu
a
tio
n
o
f
o
u
r
n
e
u
r
al
n
et
w
o
r
k
,
w
e
e
x
ec
u
te
a
ll
th
e
ex
p
er
i
m
e
n
ts
5
ti
m
e
s
an
d
u
s
e
th
e
av
er
a
g
e
o
f
F1
s
co
r
es a
s
t
h
e
co
m
p
ar
is
o
n
m
etr
ic.
T
ab
le
3
.
A
NE
R
C
o
r
p
C
o
r
p
u
s
S
tatis
tic
s
D
a
t
a
se
t
S
e
n
t
e
n
c
e
s
W
o
r
d
s
En
t
i
t
i
e
s
T
r
a
i
n
4
7
5
6
1
2
0
0
1
1
1
2
8
3
3
D
e
v
5
8
5
1
4
9
7
6
1
7
2
6
T
e
st
5
4
6
1
5
0
1
9
1
6
0
5
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
ab
le
4
s
h
o
w
s
th
e
i
m
p
ac
t
o
f
v
ar
io
u
s
p
r
etr
ain
ed
w
o
r
d
e
m
b
e
d
d
in
g
s
o
n
t
h
e
A
r
ab
ic
NE
R
ta
s
k
.
Desp
ite
th
at
w
e
r
u
n
all
th
e
f
i
v
e
lear
n
i
n
g
al
g
o
r
ith
m
s
u
s
in
g
th
e
d
e
f
au
lt
s
etti
n
g
o
n
t
h
e
s
a
m
e
u
n
lab
el
ed
d
ata,
w
e
ca
n
s
ee
th
at
Fa
s
tT
ex
t
h
a
s
co
n
s
is
te
n
t
l
y
o
u
tp
er
f
o
r
m
ed
th
e
o
t
h
er
m
o
d
e
l
s
w
it
h
a
n
av
er
a
g
e
F1
s
co
r
e
o
f
7
0
.
8
6
%.
T
h
e
s
ec
o
n
d
b
est
m
o
d
el
is
S
k
ip
Gr
a
m
w
i
th
a
6
1
.
9
1
%
in
F1
s
co
r
e.
I
n
f
ac
t,
Fa
s
tT
ex
t
i
s
an
e
x
te
n
s
io
n
o
f
Sk
ip
Gr
a
m
,
b
u
t
in
s
tead
o
f
u
s
in
g
w
o
r
d
s
d
ir
ec
tl
y
,
it
lear
n
s
w
o
r
d
e
m
b
ed
d
in
g
u
s
i
n
g
ch
ar
ac
ter
n
-
g
r
a
m
s
.
T
h
is
s
i
m
p
le
tr
ic
k
allo
w
s
it
to
tak
e
th
e
m
o
r
p
h
o
lo
g
y
o
f
w
o
r
d
s
in
to
ac
co
u
n
t
a
n
d
h
elp
s
to
d
ea
l
w
it
h
r
ar
e
an
d
o
u
t
o
f
v
o
ca
b
u
lar
y
w
o
r
d
s
w
h
ic
h
is
al
w
a
y
s
t
h
e
ca
s
e
f
o
r
m
o
r
p
h
o
lo
g
y
r
ich
la
n
g
u
a
g
e
li
k
e
Ar
ab
ic.
Hen
ce
,
o
u
r
e
m
p
ir
ical
r
es
u
lt
s
s
h
o
w
t
h
at
t
h
e
Fas
tT
ex
t
i
s
m
o
r
e
s
u
itab
le
f
o
r
th
ese
t
y
p
es
o
f
la
n
g
u
a
g
es
i
n
co
m
p
ar
is
o
n
w
i
th
o
th
er
lear
n
in
g
al
g
o
r
ith
m
s
.
T
ab
le
4
.
R
esu
lts
w
i
th
D
i
f
f
er
en
t Ch
o
ices o
f
W
o
r
d
E
m
b
ed
d
in
g
s
M
o
d
e
l
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
F
a
st
T
e
x
t
7
0
,
8
2
7
0
,
7
9
6
8
,
8
4
7
0
,
8
9
7
2
,
9
4
7
0
,
8
6
S
k
i
p
G
r
a
m
6
6
,
0
4
5
6
,
3
3
5
9
,
7
9
6
1
,
5
3
6
5
,
9
6
1
,
9
1
C
B
O
W
5
5
,
9
4
5
9
,
4
4
6
1
,
7
8
5
1
,
1
7
5
2
,
3
8
5
6
,
1
4
G
l
o
v
e
5
5
,
7
4
6
3
,
9
8
6
4
,
0
5
5
8
,
2
5
6
1
,
6
3
6
0
,
7
3
H
P
C
A
3
6
,
3
8
4
1
,
1
8
3
9
,
3
8
3
8
,
9
3
4
0
,
9
8
3
9
,
3
7
I
n
T
ab
le
5
,
w
e
v
ar
y
t
h
e
s
ize
o
f
th
e
Fas
tT
ex
t
w
o
r
d
e
m
b
ed
d
in
g
s
to
s
ee
if
it
i
n
f
lu
e
n
ce
s
th
e
p
er
f
o
r
m
a
n
ce
o
f
o
u
r
s
y
s
te
m
.
Su
r
p
r
is
i
n
g
l
y
,
i
n
cr
ea
s
i
n
g
t
h
e
v
ec
to
r
s
ize
d
id
n
o
t
f
u
r
t
h
er
en
h
an
ce
t
h
e
p
er
f
o
r
m
an
ce
ev
e
n
w
it
h
b
ig
g
er
v
alu
e
s
as
5
0
0
,
r
ath
er
it
d
ec
r
ea
s
es
it.
So
v
ec
to
r
d
i
m
e
n
s
io
n
5
0
w
a
s
o
p
ti
m
al
in
o
u
r
ca
s
e.
A
c
tu
al
l
y
,
w
h
ile
in
tr
i
n
s
ic
tas
k
s
lik
e
w
o
r
d
s
i
m
i
lar
it
y
h
a
v
e
u
s
u
a
ll
y
clea
r
te
n
d
en
c
y
to
p
r
ef
er
h
i
g
h
er
v
ec
to
r
d
i
m
en
s
io
n
al
it
y
to
ef
f
ec
tiv
e
l
y
ca
p
tu
r
e
s
e
m
a
n
tic
r
elatio
n
s
h
ip
s
b
et
w
ee
n
w
o
r
d
s
,
t
h
e
e
x
tr
in
s
ic
ta
s
k
s
li
k
e
NE
R
u
s
u
all
y
r
eq
u
ir
e
m
o
r
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8708
I
n
t J
E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
9
,
No
.
3
,
J
u
n
e
201
9
:
2
0
2
5
-
2
0
3
2
2030
ca
r
ef
u
l
t
u
n
in
g
to
f
i
n
d
th
e
o
p
ti
m
al
d
i
m
e
n
s
io
n
alit
y
an
d
te
n
d
to
f
av
o
r
lo
w
er
v
ec
to
r
s
iz
e
[3
1
]
.
T
h
er
ef
o
r
e,
th
e
c
h
o
ice
o
f
v
ec
to
r
s
ize
b
et
w
ee
n
5
0
an
d
1
0
0
s
h
o
u
ld
u
s
u
a
ll
y
b
e
e
n
o
u
g
h
f
o
r
s
i
m
ilar
ta
s
k
s
a
s
NE
R
to
g
e
t
t
h
e
b
est r
esu
lt
s
.
T
ab
le
5
.
T
h
e
P
e
r
f
o
r
m
a
n
ce
C
o
m
p
ar
i
s
o
n
w
h
e
n
Var
y
i
n
g
t
h
e
Si
ze
o
f
Fas
tT
ex
t W
o
r
d
E
m
b
ed
d
in
g
s
S
i
z
e
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
50
7
0
,
8
2
7
0
,
7
9
6
8
,
8
4
7
0
,
8
9
7
2
,
9
4
7
0
,
8
6
1
0
0
6
9
,
6
5
6
9
,
2
5
7
0
,
3
9
7
0
,
5
2
6
9
,
3
2
6
9
,
8
2
2
0
0
6
7
,
45
6
8
,
2
5
6
7
,
9
5
6
6
,
8
9
6
8
,
6
7
6
7
,
8
4
3
0
0
6
9
,
6
6
6
8
,
8
7
6
9
,
3
7
6
6
,
5
8
6
7
,
8
9
6
8
,
4
7
4
0
0
6
8
,
6
1
6
8
,
5
7
6
9
,
6
1
6
8
,
5
4
6
7
,
9
7
6
8
,
6
6
5
0
0
6
7
,
5
7
6
9
,
2
6
6
9
,
7
5
6
9
,
9
5
6
9
,
5
6
9
,
2
C
o
n
ce
r
n
i
n
g
c
h
ar
ac
ter
r
ep
r
esen
tatio
n
s
,
T
ab
le
6
s
h
o
w
s
t
h
at
u
s
i
n
g
it
y
ield
s
to
s
ig
n
i
f
ica
n
tl
y
b
etter
pe
r
f
o
r
m
a
n
ce
o
n
th
e
A
r
ab
ic
N
E
R
tas
k
.
P
r
ec
is
el
y
,
t
h
e
C
N
N
ap
p
r
o
ac
h
w
as
s
u
p
er
io
r
to
th
e
B
iL
ST
M
o
n
e
in
all
th
e
5
r
u
n
s
o
f
o
u
r
s
et
u
p
.
T
h
u
s
,
w
e
ad
o
p
t
it
as
th
e
d
ef
au
lt
s
etti
n
g
f
o
r
all
u
p
co
m
i
n
g
ex
p
er
i
m
en
ts
d
u
e
to
its
s
u
p
er
io
r
it
y
a
n
d
its
h
i
g
h
er
co
m
p
u
tatio
n
al
e
f
f
icie
n
c
y
.
I
n
ter
e
s
ti
n
g
l
y
,
r
ec
en
t
s
tu
d
ie
s
[3
1
]
s
u
g
g
est
th
at
t
h
er
e
is
n
o
s
tatis
t
ical
d
if
f
er
e
n
ce
o
f
u
s
i
n
g
ch
ar
ac
ter
r
ep
r
esen
ta
tio
n
s
w
h
e
n
ap
p
lied
t
o
th
e
E
n
g
li
s
h
NE
R
tas
k
.
I
n
d
ee
d
,
f
o
r
lan
g
u
a
g
es
li
k
e
E
n
g
lis
h
w
h
ich
d
id
n
o
t
ex
h
ib
it
m
o
r
p
h
o
lo
g
y
r
ic
h
n
ess
,
t
h
e
u
s
e
o
f
ch
ar
ac
te
r
r
ep
r
esen
tatio
n
s
is
n
o
m
a
n
d
ato
r
y
to
g
et
t
h
e
b
est
r
esu
lt
s
,
b
u
t
f
o
r
m
o
r
p
h
o
lo
g
y
r
i
ch
lan
g
u
ag
e
s
li
k
e
A
r
ab
ic
it
is
cr
u
cial
to
u
s
e
it
t
o
d
ea
l
w
it
h
th
e
co
m
p
lex
i
t
y
a
n
d
th
e
h
i
g
h
er
n
u
m
b
er
o
f
r
ar
e
an
d
o
u
t
-
of
-
v
o
ca
b
u
lar
y
w
o
r
d
s
o
b
s
e
r
v
ed
.
T
a
b
le
6
.
C
o
m
p
ar
is
o
n
o
f
n
o
t u
s
in
g
C
h
ar
ac
ter
R
ep
r
esen
tat
io
n
s
an
d
u
s
i
n
g
C
NN
o
r
B
i
L
ST
M
to
I
n
d
u
ce
C
h
ar
ac
ter
-
b
ased
R
ep
r
esen
ta
tio
n
s
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
N
o
n
e
6
5
,
9
6
7
,
1
5
6
5
,
3
1
6
5
,
6
8
6
5
,
7
9
6
5
,
9
6
C
N
N
7
0
,
8
2
7
0
,
7
9
6
8
,
8
4
7
0
,
8
9
7
2
,
9
4
7
0
,
8
6
B
i
L
S
T
M
6
8
,
8
6
7
,
9
2
6
7
,
6
5
6
8
,
1
6
6
7
,
6
6
6
8
,
0
4
I
n
T
ab
le
7
,
w
e
s
t
u
d
y
t
h
e
i
m
p
ac
t
o
f
d
r
o
p
o
u
t.
W
e
ev
al
u
ate
t
h
r
ee
o
p
tio
n
s
:
n
ai
v
e
d
r
o
p
o
u
t,
v
ar
iatio
n
al
d
r
o
p
o
u
t,
an
d
n
o
d
r
o
p
o
u
t
a
n
d
s
elec
t
t
h
e
d
r
o
p
o
u
t
r
ates
f
r
o
m
t
h
e
s
et
{0
.
2
5
,
0
.
5
,
0
.
7
5
}.
W
e
o
b
s
er
v
e
t
h
e
b
est
p
er
f
o
r
m
a
n
ce
w
i
th
a
d
r
o
p
o
u
t
r
ate
o
f
0
.
2
5
.
T
h
e
n
aiv
e
d
r
o
p
o
u
t
p
r
o
d
u
ce
s
th
e
b
est
r
es
u
lt
s
w
it
h
an
a
v
er
ag
e
F1
o
f
7
1
.
0
8
%.
T
h
e
v
ar
iatio
n
al
d
r
o
p
o
u
t
y
ie
ld
s
a
co
m
p
etiti
v
e
r
esu
lt
o
f
7
0
.
5
2
%.
T
ab
le
7
.
R
esu
lts
w
i
th
a
n
d
w
ith
o
u
t D
r
o
p
o
u
t u
s
i
n
g
d
if
f
er
en
t
R
ates
D
r
o
p
o
u
t
D
r
o
p
o
u
t
r
a
t
e
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
N
o
n
e
n
/
a
7
0
,
3
1
6
9
,
8
3
7
1
,
4
9
7
1
,
0
8
6
9
,
5
8
7
0
,
4
6
N
a
i
v
e
0
.
2
5
7
0
,
5
3
7
1
,
5
7
1
,
5
5
7
2
,
6
1
6
9
,
1
9
7
1
,
0
8
N
a
i
v
e
0
.
5
7
0
,
8
2
7
0
,
7
9
6
8
,
8
4
7
0
,
8
8
7
2
,
9
4
7
0
,
8
5
N
a
i
v
e
0
.
7
5
6
9
,
1
9
7
1
,
6
2
6
9
,
8
6
6
6
,
6
6
2
,
2
1
6
7
,
9
V
a
r
i
a
t
i
o
n
a
l
0
.
2
5
7
0
,
1
2
7
1
,
6
5
7
0
,
1
5
7
0
,
1
9
7
0
,
4
8
7
0
,
5
2
V
a
r
i
a
t
i
o
n
a
l
0
.
5
6
9
,
0
7
6
7
,
6
6
6
5
,
7
5
6
0
,
1
7
5
8
,
4
6
6
4
,
2
2
V
a
r
i
a
t
i
o
n
a
l
0
.
7
5
5
5
,
5
5
8
,
2
7
5
7
,
8
4
5
6
,
8
3
5
5
,
1
7
5
6
,
7
2
T
ab
le
8
d
e
p
icts
th
e
r
esu
lt
s
f
o
r
th
e
d
if
f
er
en
t
o
p
ti
m
izer
s
ap
p
lied
to
o
u
r
n
eu
r
al
n
et
w
o
r
k
.
W
e
u
s
ed
th
e
s
etti
n
g
s
r
ec
o
m
m
en
d
ed
b
y
th
e
au
t
h
o
r
s
o
f
ea
c
h
o
p
ti
m
izer
.
A
d
a
m
s
h
o
w
s
t
h
e
b
est
p
er
f
o
r
m
an
ce
,
y
ield
i
n
g
th
e
h
ig
h
e
s
t
s
co
r
e
f
o
r
7
0
.
8
6
%.
Na
d
am
w
h
ich
i
s
a
v
ar
ian
t
o
f
A
d
a
m
(
A
d
a
m
w
it
h
Nester
o
v
m
o
m
en
tu
m
)
ac
h
iev
e
s
a
v
er
y
co
m
p
et
iti
v
e
p
er
f
o
r
m
a
n
ce
o
f
7
0
.
5
7
%.
R
e
m
ar
k
ab
l
y
,
SG
D
p
r
o
d
u
ce
s
th
e
w
o
r
s
t
s
co
r
e
o
f
3
3
.
8
2
%.
A
ct
u
all
y
,
SGD
i
s
v
er
y
s
e
n
s
i
tiv
e
to
t
h
e
c
h
o
ice
o
f
t
h
e
lear
n
in
g
r
ate
a
n
d
s
in
ce
w
e
d
id
n
o
t
f
i
n
e
t
u
n
e
it
m
an
u
al
l
y
,
it
f
ailed
to
co
n
v
er
g
e
to
a
m
in
i
m
u
m
.
F
u
r
th
er
m
o
r
e
,
ap
p
ly
i
n
g
ea
r
l
y
s
to
p
p
in
g
d
id
n
o
t
h
elp
as
SGD
n
ee
d
s
u
s
u
all
y
m
o
r
e
ep
o
ch
s
to
f
in
d
t
h
e
g
lo
b
al
m
in
i
m
u
m
o
f
th
e
o
b
j
ec
tiv
e
f
u
n
c
tio
n
.
I
n
o
r
d
er
to
co
m
p
ar
e
o
u
r
n
eu
r
al
n
et
w
o
r
k
w
it
h
t
h
e
b
est p
er
f
o
r
m
i
n
g
A
r
ab
ic
NE
R
s
y
s
te
m
s
,
we
ap
p
l
y
o
u
r
B
iL
ST
M
-
C
R
F
m
o
d
el
to
th
e
s
tan
d
ar
d
A
NE
R
co
r
p
d
ataset
u
s
i
n
g
t
h
e
b
est
h
y
p
er
p
ar
a
m
e
ter
s
ev
al
u
ated
an
d
s
elec
ted
d
u
r
in
g
o
u
r
p
r
ev
io
u
s
ex
p
er
i
m
e
n
ts
.
S
in
ce
t
h
e
p
er
f
o
r
m
an
ce
o
f
b
o
th
n
ai
v
e
an
d
v
ar
iatio
n
al
d
r
o
p
o
u
t
w
a
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J
E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8708
A
r
a
b
ic
n
a
med
e
n
tity reco
g
n
itio
n
u
s
in
g
d
ee
p
lea
r
n
in
g
a
p
p
r
o
a
ch
(
I
s
ma
il E
l B
a
z
i
)
2031
q
u
ite
clo
s
e
an
d
it
is
also
th
e
ca
s
e
f
o
r
th
e
A
d
a
m
a
n
d
Nad
a
m
o
p
ti
m
izer
s
,
w
e
d
ec
id
ed
to
ex
p
er
im
e
n
t
w
it
h
d
if
f
er
e
n
t
s
e
tti
n
g
s
o
f
t
h
ese
h
y
p
er
p
ar
am
eter
s
i
n
co
m
b
in
a
tio
n
w
it
h
t
h
e
o
th
er
b
est
h
y
p
er
p
ar
am
eter
s
to
b
e
s
u
r
e
th
at
w
e
h
a
v
e
th
e
o
p
ti
m
al
s
et
u
p
f
o
r
o
u
r
m
o
d
el.
T
ab
le
9
s
u
m
m
ar
izes
t
h
e
r
es
u
lt
s
.
T
h
e
b
est
p
er
f
o
r
m
an
ce
o
f
o
u
r
B
iL
ST
M
-
C
R
F
m
o
d
el
is
ac
h
ie
v
ed
u
s
i
n
g
Nad
a
m
as
a
n
o
p
ti
m
izer
an
d
v
ar
iatio
n
al
d
r
o
p
o
u
t
w
it
h
a
n
av
er
ag
e
F1
s
co
r
e
o
f
9
0
.
6
0
%.
T
ab
le
8
.
P
er
f
o
r
m
a
n
ce
C
o
m
p
ar
is
o
n
f
o
r
Var
io
u
s
O
p
ti
m
izer
s
O
p
t
i
mi
z
e
r
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
A
d
a
m
7
0
,
8
2
7
0
,
7
9
6
8
,
8
4
7
0
,
8
9
7
2
,
9
4
7
0
,
8
6
N
a
d
a
m
7
1
,
1
7
1
,
5
7
7
1
,
2
8
6
7
,
9
3
7
0
,
9
6
7
0
,
5
7
R
msp
r
o
p
6
9
,
1
9
6
8
,
0
8
6
9
,
5
3
6
8
,
9
6
9
,
4
2
6
9
,
0
2
A
d
a
d
e
l
t
a
5
5
,
9
7
5
8
,
9
1
6
5
,
2
5
8
,
8
2
5
3
,
5
1
5
8
,
4
8
A
d
a
g
r
a
d
5
7
,
3
4
5
7
,
7
7
5
2
,
5
4
5
3
,
6
2
6
0
,
7
4
5
6
,
4
S
G
D
3
6
,
3
5
2
4
,
2
4
3
7
,
2
7
3
7
,
4
7
3
3
,
7
8
3
3
,
8
2
Ta
b
le
9
.
R
esu
lts
o
n
t
h
e
A
NE
R
co
r
p
Data
s
et
u
s
i
n
g
t
h
e
b
est H
y
p
er
p
ar
am
eter
s
S
e
t
t
i
n
g
s
F1
R
u
n
1
R
u
n
2
R
u
n
3
R
u
n
4
R
u
n
5
A
v
e
r
a
g
e
N
a
d
a
m
+
V
a
r
i
a
t
i
o
n
a
l
D
r
o
p
o
u
t
9
0
,
5
6
9
0
,
3
8
9
0
,
3
6
9
0
,
7
8
9
0
,
9
1
9
0
,
6
0
A
d
a
m +
V
a
r
i
a
t
i
o
n
a
l
D
r
o
p
o
u
t
8
9
,
5
1
9
0
,
2
2
8
9
,
9
7
9
0
,
0
7
9
0
,
1
2
8
9
,
9
8
A
d
a
m +
N
a
i
v
e
D
r
o
p
o
u
t
8
8
,
0
5
8
8
,
7
5
8
8
,
2
7
8
8
,
1
1
8
7
,
7
1
8
8
,
1
8
N
a
d
a
m
+
N
a
i
v
e
D
r
o
p
o
u
t
89
,
6
2
8
8
,
2
5
8
9
,
1
6
8
8
,
9
9
8
9
,
3
8
8
9
,
0
8
I
n
T
ab
le
1
0
,
w
e
p
r
esen
t
t
h
e
r
esu
lt
s
o
f
o
u
r
s
y
s
te
m
i
n
co
m
p
ar
is
o
n
w
it
h
th
r
ee
p
r
ev
io
u
s
to
p
p
er
f
o
r
m
i
n
g
s
y
s
te
m
s
f
o
r
A
r
ab
ic
NE
R
.
O
u
r
s
y
s
te
m
ac
h
ie
v
es
s
ig
n
i
f
ica
n
t
i
m
p
r
o
v
e
m
en
t
s
o
v
er
[
7
]
an
d
[
9
]
o
n
th
e
s
ta
n
d
ar
d
A
NE
R
co
r
p
d
ataset
w
it
h
an
F1
s
co
r
e
o
f
9
0
.
6
%.
W
e
o
b
tain
s
tate
-
of
-
t
h
e
-
ar
t
r
esu
lt
i
n
co
m
p
ar
is
o
n
w
it
h
[
1
0
]
.
Ou
r
m
o
d
e
l i
s
s
li
g
h
tl
y
lo
w
er
w
i
th
0
.
0
6
%.
I
n
f
ac
t,
th
e
s
y
s
te
m
i
n
tr
o
d
u
ce
d
b
y
S
h
aa
la
n
an
d
Ou
d
ah
[
1
0
]
is
a
h
y
b
r
id
m
o
d
el
th
at
co
m
b
i
n
es
m
ac
h
i
n
e
lear
n
in
g
-
b
ased
co
m
p
o
n
en
t
a
n
d
r
u
le
-
b
ased
co
m
p
o
n
e
n
t.
I
t
r
elies
h
ea
v
il
y
o
n
t
h
e
tas
k
-
s
p
ec
if
ic
an
d
lan
g
u
ag
e
d
ep
en
d
en
t
k
n
o
w
led
g
e
p
r
o
v
id
ed
b
y
t
h
e
r
u
le
-
b
ased
co
m
p
o
n
en
t
a
n
d
u
s
es
a
lo
t
o
f
h
an
d
cr
af
ted
en
g
i
n
ee
r
ed
f
ea
t
u
r
es
i
n
cl
u
d
in
g
m
o
r
p
h
o
lo
g
ical
f
e
at
u
r
es,
P
OS
tag
s
,
ca
p
it
aliza
tio
n
f
ea
t
u
r
es
a
n
d
g
az
e
tte
er
s
to
ac
h
ie
v
e
s
tate
-
of
-
t
h
e
-
ar
t
p
er
f
o
r
m
a
n
ce
.
O
n
t
h
e
o
th
er
h
a
n
d
,
O
u
r
B
iLST
M
-
C
R
F
m
o
d
el
h
a
s
t
h
e
ad
v
a
n
ta
g
e
o
f
b
ein
g
a
tr
u
e
e
n
d
-
to
-
en
d
s
y
s
te
m
t
h
at
d
o
es
n
o
t
r
eq
u
ir
e
an
y
f
ea
tu
r
e
e
n
g
i
n
ee
r
i
n
g
,
d
ata
p
r
e
-
p
r
o
ce
s
s
in
g
o
r
ex
te
r
n
al
r
eso
u
r
ce
s
an
d
th
er
ef
o
r
e
ca
n
b
e
ea
s
il
y
ex
ten
d
ed
to
o
th
er
d
o
m
ain
s
w
it
h
m
i
n
i
m
al
t
w
ea
k
i
n
g
.
T
ab
le
1
0
.
C
o
m
p
ar
is
o
n
w
it
h
P
r
ev
io
u
s
T
o
p
P
er
f
o
r
m
an
ce
A
r
ab
ic
NE
R
S
y
s
te
m
s
o
n
A
NE
R
co
r
p
D
ataset
M
o
d
e
l
F1
CRF
-
b
a
se
d
sy
st
e
m
[
7
]
7
5
.
6
6
A
b
d
a
l
l
a
h
e
t
a
l
.
[
9
]
8
8
.
3
3
S
h
a
a
l
a
n
a
n
d
O
u
d
a
h
[
1
0
]
9
0
.
6
6
O
u
r
sy
st
e
m
9
0
.
6
0
5.
CO
NCLU
SI
O
N
T
h
is
p
ap
er
p
r
o
p
o
s
es
n
eu
r
al
n
e
t
w
o
r
k
ar
c
h
itect
u
r
e
f
o
r
A
r
ab
ic
NE
R
b
ased
o
n
b
id
ir
ec
tio
n
al
L
ST
Ms.
W
e
ev
alu
a
ted
d
if
f
er
en
t
co
m
m
o
n
l
y
u
s
ed
h
y
p
er
p
ar
a
m
eter
s
f
o
r
o
u
r
B
iL
ST
M
-
C
R
F
ar
ch
itect
u
r
e
to
ass
es
s
t
h
eir
i
m
p
ac
t
o
n
th
e
o
v
er
all
p
er
f
o
r
m
a
n
ce
.
Ou
r
b
est
m
o
d
el
o
b
tain
s
s
tat
e
-
of
-
t
h
e
-
ar
t
r
es
u
lt
s
w
it
h
an
F
1
o
f
9
0
.
6
%
u
s
in
g
Fas
tT
ex
t
p
r
e
-
tr
ai
n
ed
w
o
r
d
e
m
b
ed
d
i
n
g
s
,
C
NN
C
h
ar
ac
ter
R
ep
r
esen
tat
io
n
s
,
a
v
ar
iatio
n
al
d
r
o
p
o
u
t
an
d
Nad
a
m
o
p
tim
izer
.
I
n
co
m
p
ar
is
o
n
w
it
h
p
r
ev
io
u
s
l
y
s
tate
-
of
-
th
e
-
ar
t
A
r
ab
ic
NE
R
s
y
s
te
m
s
,
o
u
r
n
e
u
r
al
m
o
d
el
is
tr
u
l
y
e
n
d
-
to
-
en
d
an
d
d
o
es
n
o
t
d
ep
en
d
o
n
an
y
d
ata
p
r
ep
r
o
ce
s
s
in
g
,
ex
te
r
n
al
tas
k
-
s
p
ec
if
ic
r
eso
u
r
ce
s
o
r
h
an
d
cr
af
ted
f
ea
t
u
r
e
en
g
i
n
ee
r
i
n
g
.
I
t
is
a
ls
o
v
er
y
f
l
ex
ib
le
b
y
a
llo
w
in
g
t
h
e
e
f
f
o
r
tl
ess
ad
d
itio
n
o
f
a
n
o
th
er
t
y
p
e
o
f
n
a
m
ed
e
n
tit
ies
a
s
n
u
m
er
al
a
n
d
te
m
p
o
r
al
NE
s
,
f
a
cilities
a
n
d
g
eo
-
p
o
liti
ca
l N
E
s
,
etc.
Ou
r
o
n
g
o
i
n
g
w
o
r
k
is
to
ex
p
lo
r
e
m
u
lti
-
ta
s
k
lear
n
i
n
g
ap
p
r
o
ac
h
es
an
d
s
ee
if
it
ca
n
f
u
r
t
h
er
im
p
r
o
v
e
o
u
r
m
o
d
el.
A
l
s
o
,
w
e
h
o
p
e
th
at
w
e
ca
n
ex
te
n
d
th
is
w
o
r
k
to
o
th
er
d
o
m
ai
n
s
li
k
e
n
o
is
y
u
s
er
-
g
e
n
e
r
ated
tex
t
w
h
ic
h
is
m
o
r
e
c
h
alle
n
g
in
g
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8708
I
n
t J
E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
9
,
No
.
3
,
J
u
n
e
201
9
:
2
0
2
5
-
2
0
3
2
2032
RE
F
E
R
E
NC
E
S
[1
]
I.
El
b
a
z
i
a
n
d
N
.
L
a
a
c
h
f
o
u
b
i,
“
Ex
p
lo
rin
g
th
e
Ef
f
e
c
ts
o
f
S
te
m
m
in
g
o
n
A
ra
b
ic
Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
,
”
In
ter
n
a
t
io
n
a
l
J
o
u
rn
a
l
o
f
Art
if
icia
l
In
telli
g
e
n
c
e
a
n
d
A
p
p
li
c
a
ti
o
n
s
,
v
o
l
/i
ss
u
e
:
7
(
1
)
,
p
p
.
3
3
–
4
3
,
2
0
1
6
.
[2
]
A
.
S
h
a
h
ro
u
r
,
e
t
a
l.
,
“
Ca
m
e
lp
a
rse
r:
A
s
y
ste
m
f
o
r
A
ra
b
ic
s
y
n
tac
t
ic
a
n
a
l
y
sis
a
n
d
m
o
rp
h
o
lo
g
ica
l
d
isa
m
b
ig
u
a
ti
o
n
,
”
Pro
c
e
e
d
in
g
s
o
f
COLING
2
0
1
6
,
th
e
2
6
t
h
In
ter
n
a
ti
o
n
a
l
Co
n
fe
re
n
c
e
o
n
Co
mp
u
t
a
ti
o
n
a
l
L
in
g
u
isti
c
s:
S
y
ste
m
De
mo
n
stra
ti
o
n
s
,
p
p
.
2
2
8
–
2
3
2
,
2
0
1
6
.
[3
]
L
.
A
b
o
u
e
n
o
u
r,
e
t
a
l.
,
“
IDRA
A
Q:
Ne
w
A
ra
b
ic
q
u
e
stio
n
a
n
sw
e
rin
g
s
y
ste
m
b
a
s
e
d
o
n
q
u
e
ry
e
x
p
a
n
sio
n
a
n
d
p
a
ss
a
g
e
re
tri
e
v
a
l,
”
2
0
1
2
.
[4
]
M
.
Be
se
iso
a
n
d
A
.
A
l
-
A
l
w
a
n
i,
“
A
Co
re
fe
re
n
c
e
R
e
so
lu
ti
o
n
A
p
p
ro
a
c
h
u
sin
g
M
o
r
p
h
o
lo
g
ica
l
F
e
a
tu
re
s
in
A
ra
b
ic,”
In
ter
n
a
t
io
n
a
l
J
o
u
rn
a
l
o
f
A
d
v
a
n
c
e
d
Co
m
p
u
ter
S
c
ie
n
c
e
a
n
d
A
p
p
li
c
a
t
io
n
s
,
v
o
l
/i
ss
u
e
:
7
(
10
)
,
p
p
.
1
0
7
–
1
1
3
,
2
0
1
6
.
[5
]
S
.
M
e
sf
a
r,
“
Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
f
o
r
A
ra
b
ic
Us
in
g
S
y
n
tac
ti
c
G
r
a
m
m
a
rs,”
Pro
c
e
e
d
in
g
s
o
f
th
e
1
2
t
h
In
ter
n
a
t
io
n
a
l
C
o
n
fer
e
n
c
e
o
n
A
p
p
l
ica
ti
o
n
s o
f
Na
t
u
ra
l
L
a
n
g
u
a
g
e
to
I
n
fo
rm
a
t
io
n
S
y
ste
ms
,
p
p
.
3
0
5
–
3
1
6
,
2
0
0
7
.
[6
]
W
.
Zag
h
o
u
a
n
i,
“
RENA
R:
A Ru
le
-
Ba
se
d
A
ra
b
ic Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
S
y
ste
m
,
”
v
o
l
/i
ss
u
e
:
11
(
1
)
,
p
p
.
2
:
1
–
2
:
1
3
,
2
0
1
2
.
[7
]
Y.
Be
n
a
ji
b
a
a
n
d
P
.
Ro
ss
o
,
“
A
ra
b
ic
n
a
m
e
d
e
n
ti
ty
re
c
o
g
n
it
io
n
u
sin
g
c
o
n
d
i
ti
o
n
a
l
ra
n
d
o
m
f
i
e
ld
s,”
Pro
c
.
o
f
W
o
rk
sh
o
p
o
n
HL
T
&
NL
P
wit
h
in
t
h
e
Ara
b
ic
W
o
rld
,
L
R
EC
,
v
o
l
.
8
,
p
p
.
1
4
3
–
1
5
3
,
2
0
0
8
.
[8
]
I.
El
b
a
z
i
a
n
d
N.
L
a
a
c
h
f
o
u
b
i,
“
A
r
a
b
ic
Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
Us
in
g
T
o
p
ic
M
o
d
e
li
n
g
,
”
In
ter
n
a
t
io
n
a
l
J
o
u
r
n
a
l
o
f
In
telli
g
e
n
t
E
n
g
i
n
e
e
rin
g
a
n
d
S
y
ste
ms
,
v
o
l
/i
ss
u
e
:
11
(
1
)
,
p
p
.
2
2
9
–
2
3
8
,
2
0
1
8
.
[9
]
S
.
A
b
d
a
ll
a
h
,
e
t
a
l.
,
“
In
teg
ra
ti
n
g
Ru
le
-
Ba
se
d
S
y
st
e
m
w
it
h
Clas
si
fica
ti
o
n
f
o
r
A
ra
b
ic
Na
m
e
d
En
ti
ty
Re
c
o
g
n
it
io
n
,
”
Co
mp
u
t
a
ti
o
n
a
l
L
in
g
u
isti
c
s
a
n
d
In
telli
g
e
n
t
T
e
x
t
Pro
c
e
ss
in
g
,
p
p
.
3
1
1
–
3
2
2
,
2
0
1
2
.
[1
0
]
K.
S
h
a
a
lan
a
n
d
M
.
Ou
d
a
h
,
“
A
h
y
b
rid
a
p
p
ro
a
c
h
t
o
A
ra
b
ic
n
a
m
e
d
e
n
ti
ty
re
c
o
g
n
it
io
n
,
”
J
o
u
rn
a
l
o
f
In
f
o
rm
a
ti
o
n
S
c
ien
c
e
,
v
o
l
/i
ss
u
e
:
40
(
1
)
,
p
p
.
6
7
–
8
7
,
2
0
1
4
.
[1
1
]
M
.
O
u
d
a
h
a
n
d
K.
S
h
a
a
lan
,
“
NERA
2
.
0
:
Im
p
ro
v
in
g
c
o
v
e
ra
g
e
a
n
d
p
e
rf
o
rm
a
n
c
e
o
f
ru
le
-
b
a
se
d
n
a
m
e
d
e
n
ti
ty
re
c
o
g
n
it
io
n
f
o
r
A
ra
b
ic,”
Na
tu
ra
l
L
a
n
g
u
a
g
e
E
n
g
in
e
e
rin
g
,
v
o
l
/i
ss
u
e
:
23
(
3
)
,
p
p
.
4
4
1
–
4
7
2
,
2
0
1
7
.
[1
2
]
C.
M
ish
ra
a
n
d
D.
G
u
p
ta,
“
De
e
p
M
a
c
h
in
e
L
e
a
rn
in
g
a
n
d
Ne
u
ra
l
Ne
t
w
o
rk
s:
A
n
O
v
e
r
v
ie
w
,
”
IAE
S
In
ter
n
a
t
io
n
a
l
J
o
u
rn
a
l
o
f
Arti
fi
c
i
a
l
I
n
telli
g
e
n
c
e
,
v
o
l
/i
ss
u
e
:
6
(
2
)
,
p
p
.
6
6
–
7
3
,
2
0
1
7
.
[1
3
]
A
.
v
a
n
d
e
n
Oo
rd
,
e
t
a
l
.
,
“
W
a
v
e
n
e
t:
A
g
e
n
e
ra
ti
v
e
m
o
d
e
l
f
o
r
ra
w
a
u
d
io
,
”
a
rXiv
p
re
p
ri
n
t
a
r
X
iv
:
1
6
0
9
.
0
3
4
9
9
,
2
0
1
6
.
[1
4
]
J.
Hu
,
e
t
a
l.
,
“
S
q
u
e
e
z
e
-
a
n
d
-
Ex
c
it
a
ti
o
n
Ne
tw
o
rk
s,”
a
rX
iv
p
re
p
rin
t
a
r
X
iv
:
1
7
0
9
.
0
1
5
0
7
,
2
0
1
7
.
[1
5
]
X
.
M
a
a
n
d
E
.
Ho
v
y
,
“
En
d
-
to
-
e
n
d
se
q
u
e
n
c
e
lab
e
li
n
g
v
ia
b
i
-
d
i
re
c
ti
o
n
a
l
lstm
-
c
n
n
s
-
c
rf
,
”
a
rX
iv
p
re
p
rin
t
a
rX
iv
:
1
6
0
3
.
0
1
3
5
4
,
2
0
1
6
.
[1
6
]
C.
d
o
s
S
a
n
t
o
s
a
n
d
M
.
G
a
tt
i,
“
De
e
p
c
o
n
v
o
lu
ti
o
n
a
l
n
e
u
ra
l
n
e
t
w
o
rk
s
f
o
r
se
n
ti
m
e
n
t
a
n
a
ly
sis
o
f
s
hor
t
tex
ts,”
Pro
c
e
e
d
in
g
s
o
f
COLING
2
0
1
4
,
th
e
2
5
t
h
I
n
ter
n
a
ti
o
n
a
l
Co
n
fer
e
n
c
e
o
n
C
o
mp
u
ta
ti
o
n
a
l
L
i
n
g
u
isti
c
s:
T
e
c
h
n
ica
l
Pa
p
e
rs
,
p
p
.
6
9
–
78
,
2
0
1
4
.
[1
7
]
S
.
S
h
a
h
,
e
t
a
l.
,
“
S
e
n
t
im
e
n
tal
An
a
ly
sis
o
f
Tw
it
ter
D
a
ta
Us
in
g
Clas
sif
ier
A
lg
o
rit
h
m
s,”
In
ter
n
a
ti
o
n
a
l
J
o
u
rn
a
l
o
f
El
e
c
t
ric
a
l
a
n
d
C
o
mp
u
ter
En
g
in
e
e
rin
g
(
IJ
ECE
)
,
v
o
l
/i
ss
u
e
:
6
(
1
)
,
p
p
.
3
5
7
–
3
6
6
,
2
0
1
6
.
[1
8
]
A
.
V
a
sw
a
n
i,
e
t
a
l.
,
“
A
tt
e
n
ti
o
n
is
A
ll
Yo
u
Ne
e
d
,
”
2
0
1
7
.
[1
9
]
Y.
L
e
Cu
n
,
e
t
a
l.
,
“
Ba
c
k
p
ro
p
a
g
a
ti
o
n
A
p
p
li
e
d
t
o
Ha
n
d
w
rit
ten
Zi
p
Co
d
e
Re
c
o
g
n
it
io
n
,
”
Ne
u
ra
l
C
o
mp
u
t.
,
v
o
l
/i
ss
u
e
:
1
(
4
)
,
p
p
.
5
4
1
–
5
5
1
,
1
9
8
9
.
[2
0
]
S
.
Ho
c
h
re
it
e
r
a
n
d
J.
S
c
h
m
id
h
u
b
e
r,
“
L
o
n
g
S
h
o
rt
-
T
e
rm
M
e
m
o
r
y
,
”
Ne
u
ra
l
Co
mp
u
t.
,
v
o
l
/i
ss
u
e
:
9
(
8
)
,
p
p
.
1
7
3
5
–
1
7
8
0
,
1
9
9
7
.
[2
1
]
J.
D.
L
a
ff
e
rt
y
,
e
t
a
l.
,
“
Co
n
d
i
ti
o
n
a
l
Ra
n
d
o
m
F
ield
s:
P
ro
b
a
b
i
li
stic
M
o
d
e
ls
f
o
r
S
e
g
m
e
n
ti
n
g
a
n
d
L
a
b
e
li
n
g
S
e
q
u
e
n
c
e
Da
ta,”
Pro
c
e
e
d
in
g
s o
f
t
h
e
Ei
g
h
tee
n
th
I
n
ter
n
a
ti
o
n
a
l
Co
n
fer
e
n
c
e
o
n
M
a
c
h
in
e
L
e
a
r
n
in
g
,
p
p
.
2
8
2
–
2
8
9
,
2
0
0
1
.
[2
2
]
G
.
La
m
p
le,
e
t
a
l.
,
“
Ne
u
ra
l
a
rc
h
it
e
c
tu
re
s f
o
r
n
a
m
e
d
e
n
ti
t
y
re
c
o
g
n
it
io
n
,
”
a
rX
iv
p
re
p
ri
n
t
a
r
X
iv
:
1
6
0
3
.
0
1
3
6
0
,
2
0
1
6
.
[2
3
]
h
tt
p
s:/
/g
it
h
u
b
.
c
o
m
/UKP
L
a
b
/em
n
lp
2
0
1
7
-
b
il
stm
-
c
n
n
-
c
rf
[2
4
]
T
.
M
ik
o
lo
v
,
e
t
a
l
.
,
“
Ef
f
icie
n
t
e
sti
m
a
ti
o
n
o
f
w
o
rd
re
p
re
se
n
tatio
n
s
in
v
e
c
to
r
sp
a
c
e
,
”
a
rX
iv
p
re
p
rin
t
a
rX
iv
:
1
3
0
1
.
3
7
8
1
,
2
0
1
3
.
[2
5
]
T
.
M
ik
o
lo
v
,
e
t
a
l.
,
“
Distri
b
u
ted
Re
p
re
se
n
tatio
n
s
o
f
W
o
rd
s
a
n
d
P
h
ra
se
s
a
n
d
th
e
ir
C
o
m
p
o
siti
o
n
a
li
t
y
,
”
Ad
v
a
n
c
e
s
i
n
Ne
u
ra
l
In
fo
rm
a
t
io
n
Pro
c
e
ss
in
g
S
y
ste
ms
2
6
,
L
a
k
e
T
a
h
o
e
,
Ne
v
a
d
a
:
Cu
rr
a
n
Asso
c
ia
tes
,
I
n
c
.
,
p
p
.
3
1
1
1
–
3
1
1
9
,
2
0
1
3
.
[2
6
]
J.
P
e
n
n
i
n
g
to
n
,
e
t
a
l.
,
“
G
lo
V
e
:
G
lo
b
a
l
V
e
c
to
rs
f
o
r
W
o
rd
Re
p
re
se
n
tatio
n
,
”
Emp
iric
a
l
M
e
th
o
d
s
in
Na
t
u
ra
l
L
a
n
g
u
a
g
e
Pro
c
e
ss
i
n
g
(
EM
NL
P)
,
p
p
.
1
5
3
2
–
1
5
4
3
,
2
0
1
4
.
[2
7
]
P
.
Bo
ja
n
o
w
sk
i,
e
t
a
l.
,
“
E
n
rich
i
n
g
W
o
rd
V
e
c
to
rs
w
it
h
S
u
b
w
o
rd
I
n
f
o
rm
a
ti
o
n
,
”
a
rXiv
p
re
p
ri
n
t
a
r
X
i
v
:
1
6
0
7
.
0
4
6
0
6
,
2
0
1
6
.
[2
8
]
R.
L
e
b
re
t
a
n
d
R.
Co
l
lo
b
e
rt,
“
W
o
rd
Em
b
e
d
d
i
n
g
s
th
ro
u
g
h
He
ll
i
n
g
e
r
P
CA
,
”
Pro
c
e
e
d
in
g
s
o
f
t
h
e
1
4
t
h
Co
n
fer
e
n
c
e
o
f
th
e
Eu
r
o
p
e
a
n
C
h
a
p
ter
o
f
th
e
Asso
c
ia
ti
o
n
f
o
r Co
mp
u
t
a
ti
o
n
a
l
L
i
n
g
u
i
stics
,
p
p
.
4
8
2
–
4
9
0
,
2
0
1
4
.
[2
9
]
Y.
G
a
l
a
n
d
Z.
G
h
a
h
ra
m
a
n
i,
“
A
T
h
e
o
re
ti
c
a
ll
y
G
ro
u
n
d
e
d
A
p
p
li
c
a
t
io
n
o
f
Dro
p
o
u
t
i
n
Re
c
u
rre
n
t
Ne
u
ra
l
Ne
tw
o
rk
s,”
Pro
c
e
e
d
in
g
s
o
f
th
e
3
0
t
h
I
n
ter
n
a
ti
o
n
a
l
C
o
n
fer
e
n
c
e
o
n
Ne
u
ra
l
I
n
f
o
rm
a
ti
o
n
Pro
c
e
ss
in
g
S
y
ste
ms
,
p
p
.
1
0
2
7
–
1
0
3
5
,
2
0
1
6
.
[3
0
]
B.
M
o
h
it
,
e
t
a
l.
,
“
Re
c
a
ll
-
o
rien
te
d
L
e
a
rn
in
g
o
f
Na
m
e
d
En
ti
t
ies
i
n
A
ra
b
ic
W
ik
ip
e
d
ia,”
Pro
c
e
e
d
in
g
s
o
f
th
e
1
3
th
Co
n
fer
e
n
c
e
o
f
th
e
E
u
ro
p
e
a
n
C
h
a
p
ter
o
f
t
h
e
Asso
c
i
a
ti
o
n
f
o
r Co
m
p
u
ta
ti
o
n
a
l
L
i
n
g
u
isti
c
s
,
p
p
.
1
6
2
–
1
7
3
,
2
0
1
2
.
[3
1
]
N
.
Re
ime
rs
a
n
d
I.
G
u
re
v
y
c
h
,
“
R
e
p
o
rti
n
g
S
c
o
re
Distrib
u
t
io
n
s
M
a
k
e
s
a
Diffe
re
n
c
e
:
P
e
rf
o
r
m
a
n
c
e
S
tu
d
y
o
f
L
S
T
M
-
n
e
tw
o
rk
s
f
o
r
S
e
q
u
e
n
c
e
T
a
g
g
in
g
,
”
Pro
c
e
e
d
in
g
s
o
f
th
e
2
0
1
7
Co
n
fer
e
n
c
e
o
n
Emp
irica
l
M
e
th
o
d
s
in
Na
t
u
ra
l
L
a
n
g
u
a
g
e
Pro
c
e
ss
in
g
(
EM
NL
P)
,
p
p
.
3
3
8
–
3
4
8
,
2
0
1
7
.
Evaluation Warning : The document was created with Spire.PDF for Python.