Inter
national
J
our
nal
of
Electrical
and
Computer
Engineering
(IJECE)
V
ol.
9,
No.
6,
December
2019,
pp.
5386
5394
ISSN:
2088-8708,
DOI:
10.11591/ijece.v9i6.pp5386-5394
r
5386
A
computational
analysis
of
short
sentences
based
on
ensemble
similarity
model
Arifah
Che
Alhadi,
Aziz
Deraman,
Masita
Masila
Abdul
J
alil,
W
an
Nural
J
awahir
W
an
Y
ussof,
Rosmayati
Mohemad
Softw
are
T
echnology
Research
Group
(SofT
ech),
School
of
Informatics
and
Applied
Mathematics,
Uni
v
ersiti
Malaysia
T
erengg
anu,
Malaysia
Article
Inf
o
Article
history:
Recei
v
ed
Apr
14,
2019
Re
vised
May
29,
2019
Accepted
Jun
10,
2019
K
eyw
ords:
Ensemble
T
e
xt
similarity
V
ector
space
model
Edit
distance
Short
sentences
ABSTRA
CT
The
rapid
de
v
elopment
of
Internet
along
with
the
wide
use
of
social
media
applica-
tions
produce
huge
v
olume
of
unstructured
data
in
short
te
xt
form
such
as
tweets,
te
xt
snippets
and
instant
messages.
This
form
of
data
rarely
contains
repeated
w
ord.
It
presents
challenge
in
sentences
similarity
analysis
as
the
standard
te
xt
similarity
models
merely
rely
on
the
number
of
w
ord
occurrence,
often
resulting
unreliable
similarity
v
alue.
Besides,
the
use
of
abbre
viation,
acron
yms,
slang,
smile
y
,
jar
gon,
symbol
or
non-standard
s
hort
form
also
contrib
utes
to
the
dif
ficulty
in
similarity
analysis.
Thus,
an
e
xtended
ensemble
similarity
model
approach
is
proposed.
An
e
xperimental
study
has
been
conducted
using
datasets
of
English
short
sentences.
The
findings
are
v
ery
enc
ouraging
in
impro
ving
the
similarity
v
alue
for
short
sentences.
Copyright
c
2019
Institute
of
Advanced
Engineering
and
Science
.
All
rights
r
eserved.
Corresponding
A
uthor:
Arif
ah
Che
Alhadi,
Senior
Lecturer
,
Softw
are
T
echnology
Research
Group
(SofT
ech),
School
of
Informatics
and
Applied
Mathematics,
Uni
v
ersiti
Malaysia
T
erengg
anu,
21030
K
uala
Nerus,
T
erengg
anu,
Malaysia.
Email:
arif
ah
hadi@umt.edu.my
1.
INTR
ODUCTION
Measuring
the
similarity
are
important
in
v
arious
natural
language
processing
applications
and
information
retrie
v
al
applications
such
as
information
e
xtraction,
document
clustering
[1,
2],
cate
gorization
or
classification
[3],
language
modeling
[4]
and
ontology
mapping
[5].
Similarity
measure
can
represent
the
similarity
between
tw
o
documents,
tw
o
queries
or
one
document
and
one
query
.
It
is
possible
to
rank
retrie
v
ed
documents
in
order
of
presumed
importance.
A
similarity
function
which
computes
the
de
gree
of
similarity
between
pair
of
te
xt
objects.
The
similarity
analysis
has
been
done
between
queries
[6],
documents
[1],
te
xt
snippets
[7,
8],
short
se
gment
[6,
9],
tweets
[10,
11]
or
question
answer
(QA)
[12,
13].
Most
of
the
preceding
w
orks
on
semantic
similarity
or
combining
the
semantic
and
le
xical
model
relies
on
additional
information
deri
v
ed
from
lar
ge
corpora,
dictionary
[14,
15]
or
background
kno
wledge
such
as
W
ordNet
or
ConceptNet
[16,
17].
Thus
these
w
ork
highly
dependence
on
third
source
information.
The
increasing
ease
of
access
to
the
Internet
is
cause
of
recent
e
xplosion
of
users
acti
vity
in
online
communication
through
social
media.
The
amount
of
unstructured
data
generated
from
user’
s
interaction
is
gro
wing
rapidly
and
publicly
a
v
ailable
[18].
Detecting
and
retrie
ving
such
information
can
f
acilitate
people
from
o
v
erwhelming
or
o
v
erload
information.
J
ournal
homepage:
http://iaescor
e
.com/journals/inde
x.php/IJECE
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
5387
The
nature
of
social
media
platform
which
has
the
limitation
of
messages
which
mak
e
the
user
tend
to
compact
the
te
xt
by
using
abbre
viation,
slangs,
jar
gon,
symbol
or
non
standard
s
hort
form
[19,
20].
The
e
xisting
similarity
model
based
on
le
xical,
semantic
and
ensemble
based
model
still
suf
fer
from
data
sparsity
and
noise.
It
is
because
the
dependenc
y
of
this
model
on
NLP
tools
or
third
source
such
as
corpus,
dictionary
or
background
kno
wledge.
This
w
ork
proposes
an
ensemble
similarity
model
that
uses
multiple
le
xical-based
similarity
model
to
o
v
ercome
this
issue
as
well
as
to
demonstrate
the
applicability
and
significant
contrib
ution
of
the
proposed
model.
2.
RELA
TED
W
ORK
A
wide
v
ariety
of
te
xt
similarity
model
e
xists
in
the
literature.
Referring
to
a
surv
e
y
conducted
by
[21],
w
ords
can
be
identical
similar
through
le
xical
and
semantic.
Le
xical-based
identify
the
w
ords
similarity
through
its
le
xical
which
it
has
similar
character
sequence
or
w
ord
matching.
Whereas
semantic
w
ords
similarity
is
based
on
the
meaning
of
the
w
ord
rather
than
character
matching.
Semantic
similarity
is
computed
based
on
corpus
and
kno
wledge
information
g
ather
from
lar
ge
corpora
or
semantic
netw
orks
such
as
W
ordNet
[22,
23].
The
ensembles
of
similarity
model
is
proposed
by
emplo
ying
ensemble
or
h
ybrid
model
to
get
a
comprehensi
v
e
measure
which
inte
grates
dif
ferent
conte
xt
feat
ures
for
similarity
measuring
including
le
xical
or
semantic
model.
In
preceding
w
ork,
ensemble
models
ha
v
e
been
recommended
by
the
researchers
which
pro
v
en
gi
v
e
record
of
better
performance
when
applying
combine
similarity
model,
compare
to
single
model
of
measuring
of
similarity
.
The
aim
of
an
ensemble
based
similarity
model
is
to
ensemble
multiple
models
of
classifiers
or
features
to
solv
e
v
arious
problems
[24].
Existing
combination
models
can
be
cate
gorized
into
dif
ferent
types
such
as
combine
classifier
or
models
[16,
25,
26]
combine
features
[11,
27,
28,
29]
and
combine
feature
and
classifier
or
models
[7,
30,
31].
M.
A.
Sultan,
S.
Bethard,
and
T
.
Sumner
in
[28]
measured
the
similarities
of
te
xt
by
combining
a
v
ector
similarity
feature
deri
v
ed
from
the
w
ord
embedding
with
alignment
based
similarity
.
The
other
model
for
the
semantic
measurement
between
short
te
xts
w
as
proposed
by
[32]
using
combinations
of
tw
o
dif
ferent
features:
(1)
distrib
uted
w
ord
representation,
(
2
)
corpus
and
kn
o
wl
edge-based
metrics.
Later
,
the
presented
method
w
as
tested
and
e
v
aluated
by
using
datasets
of
Microsoft
Research
P
araphrase
Corpus
and
SemEv
al2015.
This
w
ork
is
dif
ferent
with
[30]
where
the
h
ybrid
model
were
used
to
reduce
the
impact
of
less
infor
-
mati
v
e
terms.
The
h
ybrid
model
is
based
on
tf-idf
information
and
semantic
w
ord
embeddi
ng.
These
combination
leads
to
a
greater
model
for
short
te
xt
semantic
content.
Combinations
of
dif
ferent
le
xical,
syntactic
and
semantic
similarity
measures
ha
v
e
been
carried
out
by
[16,
31]
in
their
research
to
tackle
problem
in
identifying
paraphrase
and
assessing
sentence
similarity
.
R.
Ferreira
et
al
in
[16]
also
combined
with
statistical
model
where
syntacti
c
similarity
between
sentences
w
as
measured
using
the
relation
of
the
syn-
tactic
layer
calculated
by
matching
the
v
ertices
of
the
RDF
triples.
Semanti
c
Role
Annotation
(SRA)
w
as
used
to
identify
the
semantic
function
of
each
RDF
graph
entities
and
to
identify
the
meaning
in
the
sentence.
The
focus
of
the
w
ork
by
[25]
used
kno
wledge
and
corpus-based
similarity
model
to
estimate
the
similarities
of
the
corresponding
w
ords.
The
y
presented
tw
o
systems
for
automated
measuring
of
semantic
similarity
of
short
te
xts
that
is
submitted
to
SemEv
al-2012
T
ask
6.
Ho
we
v
er
,
their
w
ork
w
as
relied
on
semantic
netw
ork
(W
ordNet)
and
the
Latent
Semantic
Analysis
(LSA)
o
v
er
a
lar
ge
cor
p
us
to
estimate
the
distrib
ution.
Dataset
from
W
ordSim353
w
as
later
used
to
compared
and
e
v
aluate
the
model.
J.
Oli
v
a
et
al
in
[33]
introduced
ne
w
method
namely
SyMSS
that
used
a
combination
of
syntactic
and
semantic
information
from
W
ordNet
to
compute
sentence
semantic
similarity
of
tw
o
sentences.
The
sentences
were
represented
as
syntactic
dependence
tree
which
the
idea
w
as
the
meaning
of
terms
can
be
seen
through
its
syntactic
connections
among
terms.
The
semantic
information
w
as
obtained
using
W
ordNet.
The
statistical
information
within
the
short
te
xt
snippets
pair
contained
in
the
corpus
in
v
olv
ed
is
com-
bined
with
semantic
information
by
[7].
The
W
ordNet
w
as
emplo
yed
as
le
xical
database
to
compare
with
w
ord
similarity
measure.
CMU
ne
wsgroup
dataset
w
as
used
to
simulate
short
te
xt
clustering
scenarios.
Through
the
proposed
method,
initial
similarities
were
established
between
w
ords
via
le
xical
database.
Later
,
the
method
calculates
iterati
v
el
y
w
ords
similarities
and
short
te
xt
similarities
and
finally
,
the
proximity
metric
w
as
con-
structed
and
used
to
con
v
ert
the
ra
w
te
xt
snippets
into
v
ectors.
Ho
we
v
er
,
in
this
short
te
xt
modelling
method,
the
corpus
that
pro
vides
the
conte
xt
for
understanding
the
particular
meanings
of
the
w
ords
usually
needs
to
contain
se
v
eral
thousands
of
te
xt
snippets
Ho
we
v
er
,
the
corpus
need
to
contain
thousands
of
te
xt
snippet
to
A
computational
analysis
of
short
sentences
based
on...
(Arifah
Che
Alhadi)
Evaluation Warning : The document was created with Spire.PDF for Python.
5388
r
ISSN:
2088-8708
pro
vides
the
meaning
of
particular
conte
xt
of
the
w
ords.
The
combine
model
also
been
used
to
analyze
Malay
sentences
by
Noah
et
al.
[26]
where
the
w
ord
order
similarity
w
as
ensemble
with
semantic
similarity
.
The
w
ord
order
similarities
w
as
relied
on
the
calcula-
tion
of
v
ector
space
model
(cosine),
ho
we
v
er
base
on
the
e
xperimental
results,
this
model
is
not
ef
fecti
v
ely
to
be
used
alone.
Open
dictionary
w
as
use
as
a
corpus
for
semantic
analysis
and
the
its
sho
ws
the
dependenc
y
of
another
third
resource.
Although,
significant
progress
has
been
made
by
some
well-kno
wn
te
xt
similarity
model,
such
as
the
le
xical
and
semantic
similarity
model,
the
y
are
not
truly
supporting
the
short
te
xt
retrie
v
al
with
respect
to
the
limitation
of
the
length
and
changes
the
w
ay
of
user’
s
writing
the
sentence
to
compact
with
the
length
which
limit
their
applicability
.
In
an
ef
fort
to
address
this
limitation,
this
w
ork
has
focused
on
de
v
eloping
an
e
xtended
ensemble
similarity
model
that
consist
of
both
le
xical
and
spelling
error
model
as
briefly
discussed
in
the
follo
wing
section.
3.
LEXICAL-B
ASED
SIMILARITY
MODEL
Le
xical-based
similarity
model
were
originally
applied
to
identify
the
w
ords
with
similar
string
sequences
and
character
composition
[34].
In
contrast
on
relying
on
a
single
measure,
our
model
relies
on
combination
of
le
xical
si
milarity
models
which
is
combination
of
VSM
(cosine)
and
edit
dis
tance
(Damerau-
Le
v
einstein
Distance)
model.
The
proposed
methodology
is
applied
to
benchmark
dataset
by
[27].
This
dataset
has
been
used
in
man
y
studies
[16,
22,
35].
This
dataset
consist
of
65
pairs
or
noun
definitions
of
terms.
Ho
we
v
er
,
the
pre
vious
w
ork
by
[22,
35]
and
[36]
generally
considered
only
a
subset
of
30
pairs.
3.1.
V
ector
space
model
Figure
1.
sho
ws
the
proces
s
for
computing
the
sentence
similarity
between
tw
o
short
te
xt
using
cosine
similarity
model.
In
adopting
the
cosine
model,
each
sentences
is
tok
enized
and
a
set
of
joint
distinct
w
ord
if
formed.
A
joint
distinct
w
ord
set
of
W
is
formed
between
s
and
t
as
W
=
s
[
t
,
where
s
=
w
1
;
w
2
;
:::::::
;
w
n
;
and
t
=
v
1
;
v
2
;
:::::::;
v
n
.
The
joint
distinct
w
ord
set
is
used
to
construct
term-document
matrix.
Figure
1.
The
process
for
Cosine
similarity
between
sentences
In
a
term-document
matrix,
a
document
v
ectors
is
represent
the
frequenc
y
of
terms
occurrence
in
document
collections.
The
ro
ws
of
the
matrix
represent
to
the
documents
in
the
collections
and
columns
corresponds
to
term.
The
matrix
v
alue
indicate
the
terms
appearance
in
pairwise
short
te
xt
i.e
zero
v
alue
are
set
for
una
v
ailable
terms
and
non-zero
v
alue
for
the
occurrence
terms
in
a
pairwise
short
te
xt.
The
non-zero
v
alues
of
each
entry
in
the
matrix
are
set
based
on
frequenc
y
of
terms
occurrence
within
a
document
using
tf-idf
scheme.
The
frequencies
of
terms
in
a
document
tend
to
indicate
the
rele
v
ance
or
similarity
of
the
document
to
each
other
.
A
pair
of
sentences
midday:noon,
s
and
t
from
[27]
dataset
which:
(a)
s
:
Midday
is
12
o’cloc
k
in
the
middle
of
the
day
(b)
t
:
Noon
is
12
o’cloc
k
in
the
midle
of
the
de
y
Int
J
Elec
&
Comp
Eng,
V
ol.
9,
No.
6,
December
2019
:
5386
–
5394
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
5389
where
t
is
used
as
the
input
sentences
and
s
as
the
pair
sentences
from
the
database.
The
s
is
represent
as
sentence
with
correct
spelling
paired
with
the
spelling
errors
of
the
sentence
t
.
From
this
pair
of
sentences,
the
join
set
is
generate
st
=
[midday
,
noon,
is,
12,
o’clock,
in,
the,
middle,
midle,
of,
the,
day
,
de
y].
W
ith
the
gi
v
en
of
tw
o
v
ectors,
the
similarity
v
alue
of
s
and
t
can
be
measure
by
calculating
their
cosine
product
based
on
matrix
constructed.
Thus
the
cosine
similarity
v
alue
for
aforementioned
sentences
are
0.67.
Ho
we
v
er
the
cosine
similarity
v
alue
is
much
lo
wer
than
the
m
u
t
ual
agreement
of
32
raters
which
indicate
that
those
similarity
v
alue
of
that
particular
pair
of
sentences
is
0.96.
3.2.
Edit
distance
The
edit
distance
between
sentence
s
and
t
is
defined
as
the
minimum
number
of
edit
required
to
transforming
s
into
t
or
vise
v
ersa.
The
DLD
allo
ws
insertion,
de
letion,
substitution
and
transposition
for
tw
o
adjacent
characters.
Damerau
stated
that
these
four
operations
correspond
to
more
than
80%
of
all
human
misspellings
[37].
Figure
2
sho
ws
the
process
of
calculati
ng
the
DLD
between
tw
o
sentences.
It
in
v
olv
es
three
main
steps
which
are
constructing
the
matrix,
mapping
the
character
to
the
suitable
conditions
and
finally
calculating
the
similarity
score.
Figure
2.
The
process
for
DLD
similarity
between
sentences
The
aforementioned
pair
of
sentences
w
as
ag
ain
used
to
calculate
the
DLD.
The
bigger
edit
distance
v
alue
implies
that
the
sentences
s
and
t
are
not
similar
.
Then
the
similarity
v
alue
is
0.82
w
as
calculated
as
follo
w:
S
im
D
amer
auLev
enshtein
(
s;
t
)
=
1
D
LD
(
s;
t
)
max
(
j
s
j
;
j
t
j
)
(1)
4.
ENSEMBLE
MODEL
Ensemble
based
models
ha
v
e
recently
g
ain
attention
due
to
their
reported
results
which
is
better
t
han
using
single
model
alone
[6,
15,
26,
31].
The
aim
of
these
ensemble
model
is
to
combine
multiple
models
to
solv
e
the
short
te
xt
similarity
problems
containing
noise
(spelling
error
,
short
form,
jar
gon
etc).
The
models
can
be
combined
by
se
v
eral
approaches
such
a
v
eraging
[27,
26],
majority
v
ote,
weighted
majority
v
ote
[38]
and
boosting
[39].
The
proposed
ensemble
model
is
based
on
a
v
eraging
of
VSM
and
edit-distance,
where
both
uncertainty
and
reliability
of
each
single
model
are
tak
en
into
account.
The
proposed
ensemble
models
is
a
combinations
of
VSM
(cosine
simil
arity)
and
edit-distance
sim
i-
larity
(DLD)
model
for
analysing
the
te
xt
similarity
contents
of
social
media
platform.
The
proposed
ensemble
model
will
calculate
the
o
v
erall
similarity
between
tw
o
sentences
by
a
linear
combinations
as
follo
ws:
S
im
combinr
=
S
im
cosine
+
(1
)
S
im
D
LD
(2)
The
ensemble
a
v
eraging
model
is
same
as
used
by
[27]
and
[26].
A
computational
analysis
of
short
sentences
based
on...
(Arifah
Che
Alhadi)
Evaluation Warning : The document was created with Spire.PDF for Python.
5390
r
ISSN:
2088-8708
5.
RESUL
T
AND
DISCUSSION
The
e
xperimental
result
is
based
on
dataset
by
[27].
The
lack
of
suitable
e
v
aluation
dataset
is
grea
test
obstacle
that
hinders
for
e
v
aluating
our
proposed
EXSIMO.
Y
.
Li
et
al
in
[27]
used
this
benchmark
dataset
for
e
v
aluating
algorithms
to
measure
the
semantic
similarity
of
the
short
te
xt.
The
dataset
w
as
modified
as
sho
wn
in
T
able
1
for
pair
number
tw
o
and
three
which
contain
spelling
errors
in
the
sentence
and
the
changes
of
the
sentence
structure.
The
testing
results
of
the
selecti
v
e
sentence
s
are
as
illustrated
in
T
able
1.
T
able
sho
ws
the
simil
arity
v
alue
of
cosine
similarity
,
DLD
and
combination
of
both
model
for
pairwise
analysis
on
short
sentence
(
s
)
with
sentence
in
database
(
t
).
The
X
denote
the
mean
v
alue
of
32
raters
agreement.
The
dif
ferent
pair
of
sentences
w
as
tested
to
sho
w
the
applicability
of
proposed
combine
model
with
single
model.
The
full
result
w
as
represented
by
graph
as
sho
wn
in
Figure
3.
In
general,
the
proposed
ensemble
model
gi
v
es
superior
results
and
its
satisf
actory
as
compared
to
the
single
baseline
model.
The
result
of
the
proposed
ensemble
model
in
T
able
1
indicates
a
consistent
similarity
v
alue
compared
with
X
with
v
ery
minimal
dif
ferences.
The
similarity
v
alue
produced
by
the
proposed
ensemble
model
is
significant
under
dissimilar
pair
.
In
the
case
of
most
select
ion
sentences
pair
,
the
similarity
v
alue
of
dissimilar
sentence
is
decreased.
F
or
e
xample
for
the
pair
of
cor
d:smile
,
the
proposed
ensemble
model
producing
0.13
of
similarity
v
alue
com-
pared
with
cosine
0.11
and
DLD
are
0.15.
The
DLD
gi
v
e
higher
similarity
v
alue
than
cosine,
e
v
en
though
the
sentences
are
totally
dissimilar
in
w
ord
and
meaning.
Then
the
proposed
model
o
v
ercome
these
problems
by
a
v
eraging
the
similarity
v
alue
and
automatically
decrease
the
similarity
v
alue
for
dissimilar
pair
of
sentences.
T
able
1.
Comparison
of
Similarity
Models
using
Dataset
by
[27]
W
ords
pair
Sentence
pair
X
C
os
DLD
Ensemble
1.
midday:noon
-
Midday
is
12
o’clock
in
the
middle
of
the
day
.
-
Noon
is
12
o’clock
in
the
middle
of
the
day
.
0.95
0.89
0.87
0.88
2.
midday:noon
-
Midday
is
12
o’clock
in
the
middle
of
the
day
.
-
Noon
is
12
o’clock
in
the
midle
of
the
de
y
.
-
0.67
0.82
0.75
3.
midday:noon
-
Midday
is
12
o’clock
in
the
middle
of
the
day
.
-
12
o’clock
in
the
midle
of
the
de
y
is
noon.
-
0.67
0.56
0.62
4.
cor
d:smile
-
Cord
is
strong,
thick
string.
-
A
smile
is
the
e
xpression
that
you
ha
v
e
on
your
f
ace
when
you
are
pleased
or
amused,
or
when
you
are
being
friendly
.
0.01
0.11
0.15
0.13
5.
r
ooster:voya
g
e
-
A
rooster
is
an
adult
male
chick
en.
-
A
v
o
yage
is
a
long
journe
y
on
a
ship
or
in
a
spacecraft.
0.01
0.24
0.29
0.27
6.
g
em:
je
wel
-
A
gem
is
a
je
wel
or
stone
that
is
used
in
je
wellery
.
-
A
je
wel
is
a
precious
stone
used
to
decorate
v
aluable
things
that
you
wear
,
such
as
rings
or
necklaces.
0.65
0.52
0.32
0.42
7.
coc
k:r
ooster
-
A
cock
is
an
adult
male
chick
en
-
A
rooster
is
an
adult
male
chick
en.
0.86
0.86
0.83
0.85
8.
cemetery:
gr
ave
yar
d
-
A
cemetery
is
a
place
where
dead
people’
s
bodies
or
their
ashes
are
b
uried.
-
A
gra
v
e
yard
is
an
area
of
land,
sometimes
near
a
church,
where
de
ad
people
are
b
uried.
0.77
0.36
0.39
0.38
9.
automobile:wizar
d
-
An
automobile
is
a
car
.
-
In
le
gends
and
f
airy
stories,
a
wizard
is
a
man
who
has
magic
po
wers.
0.02
0.25
0.23
0.24
10.
mound:sto
ve
-
A
mound
of
something
is
a
lar
ge
rounded
pile
of
it.
-
A
sto
v
e
is
a
piece
of
equipment
which
pro
vides
heat,
either
for
cooking
or
for
heating
a
room.
0.01
0.26
0.29
0.28
Int
J
Elec
&
Comp
Eng,
V
ol.
9,
No.
6,
December
2019
:
5386
–
5394
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
5391
Figure
3.
Comparison
Similarity
V
alue
by
Cosine,
DLD
and
Proposed
Model
6.
CONCLUSION
This
paper
demonstrates
the
combination
of
le
xical-based
model
sho
ws
the
significant
in
anal
ysing
the
short
te
xt.
The
e
xperimental
results
also
ha
v
e
sho
wn
the
significant
use
of
the
proposed
ensemble
model
in
analysing
the
short
sentences.
This
research
is
to
o
v
ercome
the
limitations
of
bo
t
h
le
xical-based
models
used.
The
e
xperimental
results
also
ha
v
e
indicated
the
potential
use
of
our
combination
similarity
model
in
o
v
ercome
the
weakness
of
both
models
by
resolving
the
spelling
error
for
the
input
query
Both
techniques
are
only
consider
the
similarities
in
le
xical
terms,
without
taking
into
account
the
semantic
conte
xt
of
the
terms
in
sentences.
Ho
we
v
er
,
the
adv
antage
of
both
single
models
also
influence
the
attaining
better
si
milarity
v
alue
and
sho
w
con
vincing
results.
Comparison
with
single
similarity
model
indicates
that
the
proposed
model
produces
a
mark
ed
impro
v
ement
in
short
te
xt
similarity
retrie
v
al.
7.
A
CKNO
WLEDGMENT
This
research
is
supported
by
the
Fundamental
Research
Grant
Scheme
(FRGS)
funded
by
Malays
ian
Ministry
of
Higher
Education,
under
grant
number
59467.
REFERENCES
[1]
A.
Huang,
“Similarity
measures
for
te
xt
document
clustering,
”
in
Pr
oceedings
of
the
Sixth
Ne
w
Zealand
Computer
Science
Resear
c
h
Student
Confer
ence
(NZCSRSC2008),
Christc
hur
c
h,
Ne
w
Zealand
,
2008,
pp.
49–56.
[2]
M.
John
Basha
and
K.
Kaliyamurthie,
“
An
impro
v
ed
similarity
matching
based
clustering
frame
w
ork
for
short
and
sentence
le
v
el
te
xt,
”
International
J
ournal
of
Electrical
and
Computer
Engineering
,
v
ol.
7,
pp.
551–558,
02
2017.
[3]
Y
.
Y
.
Andreas
Lianos,
“Classifying
unstructured
te
xt
using
structured
training
instances
and
an
ensemble
of
classifiers,
”
J
ournal
of
Intellig
ent
Learning
Systems
and
Applications
,
v
ol.
07,
pp.
58–73,
2015.
[4]
I.
Dag
an,
L.
Lee,
and
F
.
C.
N.
Pereira,
“Similarity-based
models
of
w
ord
cooccurrence
probabilities,
”
Mac
hine
Learning
,
v
ol.
34,
no.
1,
pp.
43–69,
Feb
1999.
[5]
Q.
Ji,
P
.
Haase,
and
G.
Qi,
Combination
of
Similarity
Measur
es
in
Ontolo
gy
Matc
hing
Using
the
O
W
A
Oper
ator
.
Berlin,
Heidelber
g:
Springer
Berlin
Heidelber
g,
2011,
pp.
281–295.
[6]
D.
Metzler
,
S.
Dumais,
and
C.
Meek,
“Similarity
measures
for
short
se
gments
of
te
xt,
”
in
Pr
oceedings
of
the
29th
Eur
opean
Confer
ence
on
IR
Resear
c
h
,
ser
.
ECIR’07.
Berlin,
Heidelber
g:
Springer
-V
erlag,
2007,
pp.
16–27.
[Online].
A
v
ailable:
http://dl.acm.or
g/citation.cfm?id=1763653.1763660
[7]
L.
W
en
yin,
X.
Quan,
M.
Feng,
and
B.
Qiu,
“
A
short
te
xt
modeling
method
combining
semantic
and
statistical
information,
”
Inf
.
Sci.
,
v
ol.
180,
no.
20,
pp.
4031–4041,
Oct.
2010.
[8]
M.
Sahami
and
T
.
D.
Heilman,
“
A
web-based
k
ernel
function
for
measuring
the
similari
ty
of
short
te
xt
snippets,
”
in
Pr
oceedings
of
the
15th
International
Confer
ence
on
W
orld
W
ide
W
eb
,
ser
.
WWW
’06.
Ne
w
A
computational
analysis
of
short
sentences
based
on...
(Arifah
Che
Alhadi)
Evaluation Warning : The document was created with Spire.PDF for Python.
5392
r
ISSN:
2088-8708
Y
ork,
NY
,
USA:
A
CM,
2006,
pp.
377–386.
[9]
R.
Mihalcea,
C.
Corle
y
,
and
C.
Strappara
v
a,
“Corpus-based
and
kno
wledge-based
measures
of
te
xt
se-
mantic
similarity
,
”
in
Pr
oceedings
of
the
21st
National
Confer
ence
on
Artificial
Intellig
ence
-
V
olume
1
,
ser
.
AAAI’06.
AAAI
Press,
2006,
pp.
775–780.
[10]
M.
AL-Smadi,
Z.
Jaradat,
M.
AL-A
yyoub,
and
Y
.
Jararweh,
“P
araphrase
identification
and
semantic
te
xt
similarity
analysis
in
arabic
ne
ws
tweets
using
le
xical,
syntactic,
and
semantic
features,
”
Inf
.
Pr
ocess.
Mana
g
e
.
,
v
ol.
53,
no.
3,
pp.
640–652,
May
2017.
[11]
M.
Rizzo
Irf
an,
M.
F
auzi
,
T
.
T
ibyani,
and
N.
Dyah
Mentari,
“T
witter
sentiment
analysis
on
2013
curricu-
lum
using
ensemble
features
and
k-nearest
neighbor
,
”
International
J
ournal
of
Electrical
and
Computer
Engineering
(IJECE)
,
v
ol.
8,
p.
5409,
12
2018.
[12]
A.
Se
v
eryn
and
A.
Mos
chitti,
“Learning
to
rank
short
te
xt
pairs
with
con
v
olutional
deep
neural
netw
orks,
”
in
Pr
oceedings
of
the
38th
International
A
CM
SIGIR
Confer
ence
on
Resear
c
h
and
De
velopment
in
Infor
-
mation
Retrie
val
,
ser
.
SIGIR
’15.
Ne
w
Y
ork,
NY
,
USA:
A
CM,
2015,
pp.
373–382.
[13]
A.
Shtok,
G.
Dror
,
Y
.
Maare
k
,
and
I.
Szpektor
,
“Learning
from
the
past:
answering
ne
w
questions
with
past
answers,
”
in
Pr
oceedings
of
the
21st
international
confer
ence
on
W
orld
W
ide
W
eb
,
ser
.
WWW
’12,
2012,
pp.
759–768.
[14]
Y
.
Gu,
Z.
Y
ang,
J.
Zhou,
W
.
Qu,
J.
W
ei,
and
X.
Shi,
“
A
f
ast
approach
for
semantic
similar
short
te
xts
retrie
v
al,
”
in
Pr
oceedings
of
the
54th
Annual
Meeting
of
the
Association
for
Computational
Linguistics,
A
CL
2016,
A
ugust
7-12,
2016,
Berlin,
Germany
,
V
olume
2:
Short
P
aper
s
,
2016.
[15]
S.
A.
Noah,
A.
Y
.
Am
ruddin,
and
N.
Omar
,
“Semantic
similarity
measures
for
malay
sentences,
”
in
Pr
oceedings
of
the
10th
International
Confer
ence
on
Asian
Digital
Libr
aries:
Looking
Bac
k
10
Y
ear
s
and
F
or
ging
Ne
w
F
r
ontier
s
,
ser
.
ICADL
’07.
Berlin,
Heidelber
g:
Springer
-V
erlag,
2007,
pp.
117–126.
[16]
R.
Ferreira,
R.
D.
Lins,
S.
J.
Simsk
e,
F
.
Freitas,
,
and
M.
Riss,
“
Asses
sing
sentence
similarity
through
le
xical,
syntactic
and
semantic
analysis,
”
Comput.
Speec
h
Lang
.
,
v
ol.
39,
no.
C,
pp.
1–28,
Sep.
2016.
[17]
T
.
W
ei,
Y
.
Lu,
H.
Chang,
Q.
Zhou,
and
X.
Bao,
“
A
semantic
approach
for
te
xt
clustering
using
w
ordnet
and
le
xical
chains,
”
Expert
Systems
with
Applications
,
v
ol.
42,
no.
4,
pp.
2264–2275,
2015.
[18]
P
.
Nak
o
v
,
S.
Rosenthal,
S.
Kiritchenk
o,
S.
M.
Mohammad,
Z.
K
ozare
v
a,
A.
Ritter
,
V
.
Sto
yano
v
,
and
X.
Zhu,
“De
v
eloping
a
successful
seme
v
al
task
in
sentiment
analysis
of
twitter
and
other
social
media
te
xts,
”
Langua
g
e
Resour
ces
and
Evaluation
,
v
ol.
50,
no.
1,
pp.
35–65,
Mar
2016.
[19]
S.
Anson,
H.
W
atson,
K.
W
adhw
a,
and
K.
Metz,
“
Analysing
social
media
data
for
disaster
prepared-
ness:
Understanding
the
opportunities
and
barriers
f
aced
by
humanitarian
actors,
”
International
J
ournal
of
Disaster
Risk
Reduction
,
v
ol.
21,
pp.
131
–
139,
2017.
[20]
J.
V
.
Lochter
,
R.
F
.
Zanetti,
D.
Reller
,
and
T
.
A.
Almeida,
“Short
te
xt
opinion
detection
using
ensemble
of
classifiers
and
semantic
inde
xing,
”
Expert
Systems
with
Applications
,
v
ol.
62,
pp.
243
–
249,
2016.
[21]
W
.
H.
Gomaa
and
A.
A.
F
ahmy
,
“
A
surv
e
y
of
te
xt
similarity
approaches,
”
International
J
ournal
of
Com-
puter
Applications
,
v
ol.
68,
no.
13,
pp.
13–18,
April
2013.
[22]
S.
Harispe,
S.
Ranwez,
S.
Janaqi,
and
J.
Montmain,
“Semantic
similarity
from
natural
language
and
ontology
analysis,
”
CoRR
,
v
ol.
abs/1704.05295,
2017.
[23]
K.
Abdalg
ader
and
A.
Skabar
,
“Short-te
xt
similarity
measurement
using
w
ord
sense
disambiguation
and
synon
ym
e
xpansion,
”
in
AI
2010:
Advances
in
Artificial
Intellig
ence
,
J.
Li,
Ed.
Berlin,
Heidelber
g:
Springer
Berlin
Heidelber
g,
2011,
pp.
435–444.
[24]
T
.
G.
Dietterich,
“
An
e
xperimental
comparison
of
three
methods
for
constructing
ensembles
of
decision
trees:
Bagging,
boosting,
and
randomization,
”
Mac
hine
Learning
,
v
ol.
40,
no.
2,
pp.
139–157,
Aug
2000.
[25]
F
.
Sari
´
c,
G.
Gla
v
a
ˇ
s,
M.
Karan,
J.
ˇ
Snajder
,
and
B.
D.
Ba
ˇ
si
´
c,
“T
ak
elab:
Systems
for
measuring
semantic
te
xt
similarity
,
”
in
Pr
oceedings
of
the
F
ir
st
J
oint
Confer
ence
on
Le
xical
and
Computational
Semantics
-
V
olume
1:
Pr
oceedings
of
the
Main
Confer
ence
and
the
Shar
ed
T
ask,
and
V
olume
2:
Pr
oceedings
of
the
Sixth
International
W
orkshop
on
Semantic
Evaluation
,
ser
.
SemEv
al
’12.
Stroudsb
ur
g,
P
A,
USA:
Association
for
Computational
Linguistics,
2012,
pp.
441–448.
[26]
S.
A.
Noah,
N.
Omar
,
and
A.
Y
.
Amruddin,
“Ev
aluation
of
le
xical-based
approaches
to
the
semantic
similarity
of
malay
sentences,
”
J
ournal
of
Quantitative
Linguistics
,
v
ol.
22,
no.
2,
pp.
135–156,
April
2015
2015.
[27]
Y
.
Li,
D.
McLean,
Z.
A.
Bandar
,
J.
D.
O’Shea,
and
K.
Crock
ett,
“Sentence
similarity
based
on
semantic
nets
and
corpus
statistics,
”
IEEE
T
r
ans.
on
Knowl.
and
Data
Eng
.
,
v
ol.
18,
no.
8,
pp.
1138–1150,
Aug.
2006.
Int
J
Elec
&
Comp
Eng,
V
ol.
9,
No.
6,
December
2019
:
5386
–
5394
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
5393
[28]
M.
A.
Sultan,
S.
Bethard,
and
T
.
Sumner
,
“Dlscu
at
seme
v
al-2016
task
1:
Supervised
models
of
sentence
similarity
,
”
in
Pr
oceedings
of
the
10th
International
W
orkshop
on
Semantic
Evaluation,
SemEvalN
AA
CL-
HL
T
2016,
San
Die
go,
CA,
USA,
J
une
16-17,
2016
,
2016,
pp.
650–655.
[29]
A.
A.
Hasan,
S.
T
iun,
M.
M.
Y
usof,
U.
A.
Mokhtar
,
and
D.
I.
Jambari,
“Enhanced
feature
for
short
document
classification,
”
J
ournal
of
Engineering
and
Applied
Sciences
,
v
ol.
12,
no.
13,
pp.
3534–3540,
2017.
[30]
C.
D.
Boom,
S.
V
.
Canne
yt,
S.
Bohez,
T
.
Demeester
,
and
B.
Dhoedt,
“Learning
semantic
similarity
for
v
ery
short
te
xts,
”
CoRR
,
v
ol.
abs/1512.00765,
2015.
[31]
R.
Ferreira,
F
.
Ca
v
alcanti,
Geor
ge
D.C.
Freitas,
R.
D.
Lins,
S.
J.
Simsk
e,
,
and
M.
Riss,
“Combining
sentence
similarities
measures
to
identify
paraphrases,
”
Comput.
Speec
h
Lang
.
,
v
ol.
47,
pp.
59–73,
Jan.
2018.
[32]
P
.
H.
Duong
and
N.-T
.
Nguyen,
Hien
T
.and
Huynh,
“Measuring
similarity
for
short
te
xts
on
social
media,
”
in
Pr
oceeding
of
the
5th
Int
ernational
Confer
ence
on
Computational
Social
Networks,
CSoNet
2016,
Ho
Chi
Minh
City
,
V
ietnam,
A
ugust
2-4,
2016
.
Springer
International
Publishing,
2016,
pp.
249–259.
[33]
J.
Oli
v
a,
J.
I.
Serrano,
M.
D.
del
Castillo,
and
A.
Iglesias,
“Symss:
A
syntax-based
measure
for
short-te
xt
semantic
similarity
,
”
Data
Knowl.
Eng
.
,
v
ol.
70,
no.
4,
pp.
390–405,
Apr
.
2011.
[34]
W
.
H.
Gomaa
and
A.
A.
F
ahmy
,
“
A
surv
e
y
of
te
xt
similarity
approaches,
”
International
J
ournal
of
Com-
puter
Applications
,
v
ol.
68,
no.
13,
pp.
13–18,
April
2013.
[35]
S.
Hassan
and
R.
Mihalcea,
“Semantic
relatedness
using
salient
semantic
analysis,
”
in
Pr
oceedings
of
the
T
wenty-F
ifth
AAAI
Confer
ence
on
Artificial
Intellig
ence
,
ser
.
AAAI’11.
AAAI
Press,
2011,
pp.
884–889.
[Online].
A
v
ailable:
http://dl.acm.or
g/citation.cfm?id=2900423.2900564
[36]
G.
Tsatsaronis,
I.
V
arlamis,
and
M.
V
azir
giannis,
“T
e
xt
relatedness
based
on
a
w
ord
the-
saurus,
”
J
.
Artif
.
Int.
Res.
,
v
ol.
37,
no.
1,
pp.
1–40,
Jan.
2010.
[Online].
A
v
ailable:
http://dl.acm.or
g/citation.cfm?id=1861751.1861752
[37]
F
.
J.
Damerau,
“
A
technique
for
computer
detection
and
correction
of
spelling
errors,
”
Commun.
A
CM
,
v
ol.
7,
no.
3,
pp.
171–176,
Mar
.
1964.
[38]
A.
Husin
and
K.
Ruhana
K
u-Mahamud,
“
Ant
system
and
weighted
v
oting
method
for
multiple
classifier
systems,
”
International
J
ournal
of
Electrical
and
Computer
Engineering
(IJECE)
,
v
ol.
8,
p.
4705,
12
2018.
[39]
M.
Whitehead
and
L.
Y
ae
ger
,
Sentiment
Mining
Using
Ensemble
Classification
Models
.
Dordrecht:
Springer
Netherlands,
2010,
pp.
509–514.
BIOGRAPHY
OF
A
UTHORS
Arifah
Che
Alhadi
is
currently
a
senior
lecturer
at
School
of
Informatics
and
Applied
Mathematics,
Uni
v
ersiti
Malaysia
T
erengg
anu
(UMT).
She
recei
v
ed
her
Master
de
gree
in
Information
T
echnology
from
National
Uni
v
ersity
of
Malaysia
in
2005.
Her
recent
research
w
ork
focuses
on
Information
Retrie
v
al
especially
in
the
short
te
xt
analysis
and
retrie
v
al.
Besides
that,
she
also
in
v
olv
ed
in
the
research
on
ontology
de
v
elopment
and
sentiment
analysis.
She
is
acti
v
ely
conduc
ting
her
research
in
her
field
of
interests
through
the
supervision
of
under
graduate
students.
She
is
also
acti
v
ely
in
v
olv
ed
as
paper
re
vie
wers
for
national
and
international
journals,
conferences,
seminars
and
symposiums
in
her
field.
Aziz
Deraman
recei
v
ed
his
Bachelor
from
UKM
in
1982,
Master
from
Glasgo
w
Uni
v
ersity
in
1984
and
PhD
from
UMIST
in
1992
and
has
serv
ed
Uni
v
ersiti
K
ebangsaan
Malaysia
from
1984
to
2007
and
Uni
v
ersiti
Malaysia
T
erengg
anu
since
2007.
He
is
presently
a
senior
professor
of
Softw
are
Engineer
-
ing
specialising
in
softw
are
process,
softw
are
management
and
certification.
He
maintains
a
di
v
erse
research
interest
including
IT
strate
gic
planning,
Softw
are
process
and
certification,
reusability
in
multimedia
object,
medical
computing,
smart
education
management
and
community
computing.
A
computational
analysis
of
short
sentences
based
on...
(Arifah
Che
Alhadi)
Evaluation Warning : The document was created with Spire.PDF for Python.
5394
r
ISSN:
2088-8708
Masita
Masila
Abdul
J
alil
recei
v
ed
her
B.Eng.
(Hons)
de
gree
in
Computer
System
Engineering
from
Uni
v
ersity
of
W
arwick,
UK
in
1997.
Upon
graduation,
she
joined
Celcom
(M)
Sdn
Bhd,
a
mobile
service
pro
vider
,
as
System
Engineer
.
She
later
pursued
her
study
in
engineering
b
usiness
management
at
Uni
v
ersity
of
W
arwick
and
obtained
her
Master’
s
de
gree
in
2000.
In
2013,
she
re-
cei
v
ed
her
Ph.D
de
gree
in
Information
System
from
Uni
v
ersiti
K
ebangsaan
Malaysia
(UKM).
She
is
currently
a
senior
lecturer
at
the
School
of
Informatics
and
Applied
Mathematics,
Uni
v
ersiti
Malaysia
T
erengg
anu.
Her
research
interests
include
Information
System,
softw
are
reuse
and
computer
science
education.
W
an
Nural
J
awahir
W
an
Y
ussof
recei
v
ed
her
B.IT
in
Softw
are
Engineering
and
M.Sc.
in
Artificial
Intelligence
from
K
olej
Uni
v
ersiti
Sains
dan
T
eknologi
Malaysia.
In
2014,
she
obtained
her
Ph.D.
from
Uni
v
ersiti
Malaysia
T
erengg
anu.
She
is
currently
a
senior
lecturer
at
School
of
Informatics
and
Applied
Mathematics,
Uni
v
ersiti
Malaysia
T
erengg
anu.
Her
research
interests
are
in
2D/3D
image
analysis
and
underw
ater
video
processing.
Rosmayati
Mohemad
is
currently
a
senior
lecturer
a
t
School
of
Informatics
and
Applied
Mathe-
matics,
Uni
v
ersiti
Mal
aysia
T
erengg
anu
(UMT).
She
recei
v
ed
her
PhD
de
gree
in
Computer
Science
from
National
Uni
v
ersity
of
Malaysia
in
2013.
Her
research
interes
ts
are
in
kno
wledge
engineering
and
decision
support
system
and
currently
focusing
on
modelling
ontology
for
supporting
decision-
making
process
in
the
domains
of
special
education
and
fore
nsics
science.
She
is
an
editor
of
books,
conference
proceedings
and
also
re
vie
wer
of
international
journals
and
conference
papers.
She
is
a
profesional
technologist
of
Malaysian
Board
of
T
echnologies
and
also
a
member
of
IEEE
society
.
Int
J
Elec
&
Comp
Eng,
V
ol.
9,
No.
6,
December
2019
:
5386
–
5394
Evaluation Warning : The document was created with Spire.PDF for Python.