Inter
national
J
our
nal
of
Electrical
and
Computer
Engineering
(IJECE)
V
ol.
9,
No.
3,
June
2019,
pp.
2164
2176
ISSN:
2088-8708,
DOI:
10.11591/ijece.v9i3.pp2164-2176
2164
Pr
ediction
of
Answer
K
eyw
ords
using
Char
-RNN
Pratheek
I
and
J
oy
P
aulose
Dept.
of
Computer
Science,
Christ
Uni
v
ersity
,
India
Article
Inf
o
Article
history:
Recei
v
ed
Jan
20,
2018
Re
vised
May
31,
2018
Accepted
No
v
16,
2018
K
eyw
ords:
Natural
Language
Processing
Recurrent
Neural
Netw
ork
Long
Short
T
erm
Memory
Gated
Recurrent
Units
Neural
Netw
orks
ABSTRA
CT
Generating
sequences
of
characters
using
a
Recurrent
Neural
Netw
ork
(RNN)
is
a
tried
and
tested
method
for
creating
unique
a
nd
conte
xt
a
w
are
w
ords,
and
is
fundamental
in
Natural
Language
Processing
tasks.
These
type
of
Neural
Netw
orks
can
also
be
used
a
question-answering
system.
The
main
dra
wback
of
most
of
these
systems
is
that
the
y
w
ork
from
a
f
actoid
database
of
information,
and
when
queried
about
ne
w
and
current
information,
the
responses
are
usually
bleak.
In
this
paper
,
the
author
proposes
a
no
v
el
approach
to
finding
answer
k
e
yw
ords
from
a
gi
v
en
body
of
ne
ws
te
xt
or
headline,
based
on
the
query
that
w
as
ask
ed,
where
the
query
w
ould
be
of
the
nature
of
current
af
f
airs
or
recent
ne
ws,
with
the
use
of
Gated
Recurrent
Unit
(GR
U)
v
ariant
of
RNNs.
Thus,
this
ensures
that
the
answers
pro
vided
are
rele
v
ant
to
the
content
of
query
that
w
as
put
forth.
Copyright
©
2019
Institute
of
Advanced
Engineering
and
Science
.
All
rights
r
eserved.
Corresponding
A
uthor:
Pratheek
I,
Department
of
Computer
Science,
Christ
Uni
v
ersity
Hosur
Road,
Bang
alore,
Karnataka,
India
560029.
Email:
pratheek.i@cs.christuni
v
ersity
.in
1.
INTR
ODUCTION
In
the
modern
w
orld
the
amount
of
information
that
is
presented
to
the
user
,
or
e
v
en
generated
is
quite
o
v
erwhelming.
Ev
en
though
there
are
v
arious
applications
that
f
acilitate
the
consolidation
of
the
information,
there
is
a
possibility
that
some
information
might
be
missed
which
may
pro
v
e
t
o
be
vital
in
other
scenarios.
One
w
ay
to
compress
the
information
while
ha
ving
it
retain
conte
xtual
information
is
through
the
use
of
Recurrent
Neural
Netw
orks
(RNNs)
RNNs
are
dynamic
systems
which
are
able
to
generate
sequences
of
information,
not
only
in
te
xtual
domain
(1),
b
ut
also
in
v
arious
other
domains,
including
b
ut
not
limited
to
music
(2),
(3),
image
(4)
among
se
v
eral
others.
RNNs
w
ork
on
the
basic
principle
that,
a
certain
data
is
pro
vided
to
the
system,
which
it
processes
and
trains
itself
on,
and
after
a
certain
number
of
iterations
of
training,
it
be
gins
to
generate
sequences
of
te
xt,
or
other
information
that
is
pro
vided
initially
.
In
theory
,
this
is
considered
to
be
good,
b
ut
when
practically
using
the
system
one
of
the
mai
n
fla
ws
of
the
system
is
e
xposed,
which
is
that
as
the
system
generates
te
xt,
it
be
gins
to
lose
conte
xt
and
thus,
the
information
it
subsequently
generates
is
only
based
on
the
fe
w
pre
vious
w
ords
it
has
learnt,
b
ut
not
from
the
original
content
from
which
it
started
to
train.
T
o
o
v
ercome
this
fundamental
fla
w
,
the
author
uses
a
v
ariant
of
the
standard
RNN
called
as
GR
U
(Gated
Recurrent
Unit).
When
compared
with
a
standard
RNN,
Long
Short
T
erm
Memory
(LSTM)
based
v
ariant
performs
better
since
it
is
able
to
retain
information,
from
the
original
conte
xt
that
it
w
as
trai
ned
on,
and
on
the
ne
wer
conte
xt
that
it
learns,
thus
being
able
to
generate
a
much
more
di
v
erse
and
natural
sequences
of
information.
A
GR
U
based
RNN
is
considered
to
be
the
ne
xt
step
of
e
v
olution
for
LSTMs.
The
closest
implementation
of
this
nature
w
as
attempted
by
the
F
acebook
Research
team(5).
In
their
paper
,
the
y
are
using
a
lar
ge
data
corpus
to
predict
answers
for
f
actual
questions.
The
system
proposed
in
this
paper
attempts
to
pro
vide
answers
to
questions
whose
answers
are
considered
to
be
currently
‘in
ne
ws’,
and
J
ournal
Homepage:
http://iaescor
e
.com/journals/inde
x.php/IJECE
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISSN:
2088-8708
2165
may
not
necessarily
ha
v
e
an
enc
yclopaedic
entry
.
Section
2.
contains
the
Literature
Re
vie
w
of
the
papers
that
are
used
as
reference
material
for
this
paper
.
Section
3.1.
consists
of
the
GR
U
based
RNN
model
that
is
used
in
this
paper
.
Section
3.2.
includes
the
proposed
method,
a
model,
and
its
e
v
aluation.
The
results
are
recorded
in
section
4.
Section
5.
comprises
the
future
scope
and
conclusion.
2.
RELA
TED
W
ORK
2.1.
Smart
Reply:
A
utomated
Response
Suggestion
f
or
Email
Replying
to
an
Em
ail
is
sometimes
a
trick
y
situation,
especially
with
choosing
the
right
w
ords
to
con
v
e
y
the
right
meaning
which
is
some
what
a
tedious
task.
In
the
past
there
ha
v
e
been
man
y
automated
response
systems
b
uilt
within
email
clients
b
ut
most
of
those
are
just
static
sentences
that
w
ouldn’
t
be
fle
xible
to
a
gi
v
en
scenario.
The
y
w
ould
just
be
simple
sentences.
In
this
paper
,
The
authors
of
the
paper
,
de
vise
a
system
that
w
ould
pro
vide
an
automated
response
to
a
recei
v
ed
email,
based
on
the
content
of
the
email.
In
this
system,
the
authors
create
a
sequence-to-sequence
long
short
term
memory
netw
ork
to
predict
the
sequences
of
te
xt.
The
input
is
an
incoming
message,
and
the
output
is
a
series
of
responses
that
are
generated
based
on
the
pro
vided
te
xt
corpus.
The
LSTMs
were
first
originally
applied
to
Machine
T
ranslation
b
ut
has
since
seen
success
in
other
domains
such
as
image
captioning
and
speech
recognition.
(6)
2.2.
Impr
o
ving
Context
A
war
e
Language
Models
In
the
current
age,
man
y
automated
te
xtual
systems
use
the
LSTM
model
to
generate
te
xt
res
ponses.
The
authors
of
this
paper
propose
an
alternati
v
e
approach
to
the
standard
RNN
used
for
te
xt
generation,
which
is
Recurrent
Neural
Netw
ork
Language
Model
(RNNLM).
What
the
y
propose
is,
to
replace
the
domains
which
are
used
to
pro
vide
a
sort
of
an
inadequate
conte
xt
for
the
te
xt
generation,
with
conte
xt
based
v
ariables.
These
conte
xt
v
ariables
are
used
to
describe
certain
aspects
of
the
language,
such
as
topic,
time,
or
other
language.
These
conte
xt
v
ariables
are
then
dynamically
combined
to
create
to
create
a
more
coherent
te
xt.
The
combina-
tion
of
the
conte
xt
v
ariables
with
conte
xt
embedding
creates
the
aforementioned
RNNLM.
The
data
used
to
test
the
model
is
obtained
from
reddit,
twitter
and
SCO
TUS.
On
the
reddit
data,
the
testing
is
done
based
on
a
gi
v
en
comment,
and
the
objecti
v
e
for
the
model
is
to
identify
the
subreddit
(discussion
group
of
one
topic),
from
which
the
sentence
might
ha
v
e
originated.
There
are
eight
specific
subreddits,
and
nine
general
subreddits,
and
the
results
sho
w
that
the
RNNLM
model
is
comparati
v
ely
better
on
specific
subreddits,
compared
to
general
subreddits,
thus
sho
wing
that
conte
xt
based
language
models
were
much
more
ef
fecti
v
e.
(7)
2.3.
Contextual
LSTM
(CLSTM)
models
f
or
Lar
ge
scale
NLP
tasks
T
e
xt
presented
in
an
y
format
is
usually
in
the
form
of
phrases,
sentences
,
paragraphs,
sections
among
others.
These
formats
are
a
w
ay
of
abstracting
sentences
into
a
combined
entity
to
present
a
cohesi
v
e
meaning.
In
this
paper
,
the
authors
create
a
Conte
xtual
LSTM
(CLSTM)
model
,
and
it
is
tested
ag
ainst
a
normal
LSTM
model
on
Natural
Language
Processing
tasks,
such
as
w
ord
selection,
ne
xt
w
ord
prediction,
and
ne
xt
sentence
prediction.
The
data
used
on
this
model
w
as
from
English
W
ikipedia,
and
English
Google
Ne
ws
site.
The
results
on
the
English
Google
Ne
ws
data,
sho
wed
impro
v
ements
across
the
board
when
comparing
LSTM
with
CLSTM.
In
terms
of
ne
xt
w
ord
prediction,
LSTM
using
w
ords
as
features
had
the
perple
xity
of
37,
and
the
CLSTM
impro
v
ement
w
as
about
2
percent.
In
the
case
of
ne
xt
sentence
selection,
the
impro
v
ement
w
as
about
39
percent
for
the
LSTM,
when
compared
with
CLSTM,
which
had
an
accurac
y
of
46
percent
and,
finally
ne
xt
sentence
topic
prediction
in
LSTM,
using
current
sentence
topic
as
fea
ture,
the
perple
xity
w
as
about
5,
and
CLSTM
impro
v
ed
on
it
by
9
percent.
Thus
one
can
see
that
the
usage
of
conte
xt
along
with
an
LSTM
has
an
impro
ving
ef
fect
o
v
er
a
standard
LSTM,
for
natural
language
processing
tasks.
(8)
2.4.
Context-awar
e
Natural
Language
Generation
with
Recurr
ent
Neural
Netw
orks
Natural
Language
generation
is
a
useful
system
in
v
arious
applications,
such
as
response
generation
in
messaging
systems,
te
xt
summarization
and
image
captioning.
Most
of
the
natural
language
generation
Pr
ediction
of
Answer
K
e
ywor
ds
using
Char
-RNN
(Pr
atheek)
Evaluation Warning : The document was created with Spire.PDF for Python.
2166
ISSN:
2088-8708
w
orks
only
with
the
pro
vided
amount
of
content,
while
ignoring
the
conte
xtual
information
that
is
present.
Ho
we
v
er
,
in
real
life
scenarios,
these
systems
ignore
the
conte
xtual
component,
and
only
focus
on
the
current
data
to
generat
e
te
xt.
The
authors
of
the
paper
,
propose
tw
o
approaches
for
this.
The
first
is
a
C2S
(conte
xt
to
sequences)
model,
which
encodes
a
set
of
conte
xts
into
a
continuous
representation
and
then
decode
the
representation
into
a
te
xt
sequence,
through
a
recurrent
neural
netw
ork.
Ho
we
v
er
not
all
the
w
ords
may
depend
on
the
conte
xts,
some
of
which
may
only
depend
on
their
preceding
w
ords.
T
o
res
olv
e
this,
a
g
ating
mechanism
is
introduced
to
control
when
the
information
from
the
conte
xts
are
accessed.
The
second
model
that
the
y
propose,
is
called
gC2S
(Gated
Conte
xts
to
Sequences).
The
data
used
for
this
is
customer
re
vie
ws
from
sites
such
as
Amazon
and
T
ripAdvisor
(T
ra
v
el
website,
where
tra
v
elers
can
re
vie
w
places).
In
Amazon
the
re
vie
ws
were
focused
from
the
book,
electronic,
and
mo
vie
cate
gories;
and
for
T
ripAdvisor
,
the
cate
gory
w
as
hotels.
Upon
testing,
the
authors
found
that,
the
gC2S
model
significantly
outperforms
the
C2S
model
as
it
adds
skip-connections
between
the
conte
xt
representations
and
the
w
ords
in
the
sequences,
allo
wing
the
information
from
the
conte
xts
to
be
able
to
directly
af
fect
the
generation
of
w
ords.
(9)
2.5.
Generating
Sentences
fr
om
a
Continuous
Space
The
standard
Recurrent
Neural
Netw
ork
Language
Model
(RNNLM)
generates
a
single
w
ord
at
a
time
and
does
not
w
ork
from
an
e
xplicit
global
sentence
v
ariational
autoencoder
generati
v
e
model
that
incorporates
distrib
uted
latent
representations
of
entire
sentences.
This
f
actorization
allo
ws
it
to
e
xplicitly
model
holistic
properties
of
sentences
such
as
style,
topic,
and
high-le
v
el
syntactic
features.
The
RNNLM
is
a
probabilis-
tic
m
od
e
l
with
no
significant
assumptions
since
it
generates
sentences
w
ord-by-w
ord
based
on
an
e
v
olving
distrib
uted
state
representation.
Ho
we
v
er
,
when
the
RNNLM
model
is
brok
en
do
wn
into
a
series
of
ne
xt-step
predictions,
it
does
not
e
xpose
interpretable
representation
of
global
features.
T
o
o
v
ercome
such
short
comings,
the
authors
of
the
papers
propose
a
concept,
which
the
y
refer
to
as
v
ariational
autoencoder
,
through
which
the
y
hope
to
capture
the
global
features
in
a
continuous
latent
v
ariable.
A
standard
RNN
predicts
w
ords
based
on
condi
tioning
of
the
pre
vious
w
ord
and
an
e
v
olving
hidden
state.
Although
ef
fecti
v
e,
it
does
not
learn
about
the
v
ector
representation
of
the
full
sentence.
In
order
to
represent
a
continuous
latent
sentence,
one
should
first
find
a
method
to
map
sentences
and
distrib
uted
repre-
sentations
that
can
be
trained
in
an
unsupervised
setting.
Sequence
autoencoders
are
some
what
successful,
in
generating
complete
documents,
an
autoencoder
consists
of
an
encoder
function
and
a
probabilistic
decoder
model,
in
the
case
of
a
sequence
autoencoder
,
both
encoder
and
decoder
are
RNNs
and
e
xamples
are
tok
en
sequences.
The
V
ariational
Autoencoder
(V
AE)
is
a
generati
v
e
model
that
is
based
on
a
re
gularized
v
ersion
of
the
standard
autoencoder
.
This
model
imposes
a
prior
distrib
ution
which
enforces
a
re
gular
geometry
and
mak
es
it
possible
to
dra
w
proper
samples
from
the
model
using
ancestral
sampling.
The
dataset
used
for
this
is
a
collection
of
te
xt
from
K12
e-books,
which
contains
approximately
80
million
sentences.
One
of
the
test
w
as
to
see
the
imputation
of
missing
w
ords
and
the
authors
observ
ed
that
a
standard
RNN
can
only
predict
from
a
pre
vious
set
of
pro
vided
tok
ens
where
as
the
RNN
with
V
AE
is
much
more
ef
fecti
v
e
and
can
pro
vide
an
intuiti
v
e
based
completion
which
fit
the
conte
xt
much
better
and
pro
vided
a
more
cohesi
v
e
meaning
to
a
sentence.
(10)
2.6.
A
Neural
Attention
Model
f
or
Abstracti
v
e
Sentence
Summarization
Summarization
is
one
of
the
most
important
components
in
the
current
w
orld,
it
is
especially
required
by
the
ne
ws
outlets
and
discussion
forums,
among
others
to
summarize
contents
that
are
otherwise
too
long
to
read
by
users
who
w
ould
not
ha
v
e
the
time
to
do
s
o.
The
objecti
v
e
is
to
create
a
condensed
form
of
the
te
xt
corpus
that
is
able
to
capture
the
whole
meaning
of
the
corpus.
Most
common
form
of
summarization
is
e
xtraction
style,
where
certain
parts
of
the
te
xt
is
e
xtracted
out
and
then
stitched
together
.
The
other
form
is
abstracti
v
e
summarization
where
the
approach
is
to
do
the
summarization
from
the
bottom
up.
The
authors
of
this
paper
propose
a
ne
w
concept
of
summarization
called
as
attention
based
summarization.
It
incorporates
linguistic
structure
when
compared
with
other
approaches,
and
can
scale
easily
to
train
on
lar
ge
amounts
of
data.
Since
the
s
ystem
mak
es
no
assumptions
about
the
v
ocab
ulary
of
the
generated
summary
,
it
can
be
trained
directly
on
an
y
document
pairs,
and
thus
can
be
used
to
summarize
a
gi
v
en
data
into
a
headline.
IJECE
V
ol.
9,
No.
3,
June
2019
:
2164
–
2176
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISSN:
2088-8708
2167
The
dataset
used
by
the
authors
w
as
a
collection
of
500
ne
ws
articles
from
The
Ne
w
Y
ork
T
im
es
and
Associate
d
Press,
paired
with
4
dif
ferent
human
generated
reference
summaries.
The
authors
fi
nd
that,
the
proposed
model
is
able
to
perform
comparati
v
ely
much
better
than
the
older
approaches
and
in
a
more
abstracti
v
e
summaries.
(11)
2.7.
Neural
T
uring
Machines
The
neural
netw
ork
is
an
e
xcellent
concept.
It
is
able
to
mimic
the
human
brain
quite
well.
But
one
of
the
missing
components
is
the
memory
part
of
it,
the
neural
netw
ork
is
not
able
to
remember
the
acti
vity
it
has
completed.
It
trains
for
a
gi
v
en
task,
and
e
x
ecutes
the
task
b
ut
once
completed,
i
t
for
gets
the
method,
and
if
the
said
task
is
to
be
performed
ag
ain,
it
w
ould
ha
v
e
to
be
trained
yet
ag
ain
and
then
the
task
could
be
performed.
The
authors
of
this
paper
propose
a
concept
in
wh
i
ch
the
neural
netw
ork
(in
this
case,
an
LSTM)
is
paired
with
a
memory
bank.
This
neural
net
w
ork
is
able
to
interact
with
both
standard
input
and
output
v
ectors,
and
also
with
a
memory
bank.
The
authors
train
the
model
in
performing
tw
o
simple
tasks;
reading
and
writing,
and
then
store
the
concept
of
the
task
into
its
memory
banks.
Then
the
y
perform
a
fe
w
tests
,
first
being
the
cop
y
test.
The
model
is
pro
vided
with
random
v
alues
of
8-bit
binary
v
alues
of
random
lengths,
and
then
it
is
to
output
a
v
alue,
that
is
a
cop
y
of
the
initial
sequence.
The
authors
find
that
after
training
both
the
LSTM
model
and
an
NTM
model,
the
NTM
model
i
s
able
to
perform
subsequent
tasks
of
cop
ying
in
much
lesser
time,
and
it
need
not
be
trained
ag
ain.
Another
test
that
the
authors
performed
w
as
the
sorting.
The
model
w
as
pro
vided
with
random
binary
v
alues,
along
with
priority
which
is
between
-1
and
1.
The
y
found
that
the
NTM
w
as
able
to
understand
the
concept
of
sorting
and
w
as
able
to
perform
the
task
much
more
cohesi
v
ely
.
It
w
as
able
to
approach
this
concept
as
a
human
mind
w
ould,
compared
to
an
LSTM
model.
(12)
After
e
xamining
v
arious
papers
which
are
aforementioned,
it
w
as
found
that
the
system
that
the
au-
thors
were
looking
for
does
not
e
xist,
and
thus,
through
this
paper
the
authors
w
ants
to
propose
such
a
system.
3.
PR
OPOSED
W
ORK
3.1.
RNN
Model
In
this
section,
the
authors
describe
the
model
that
is
being
used
in
this
paper
.
An
e
xample
of
this
RNN
Model
is
sho
wn
in
Figure
1.
The
first
thing
to
note
is
the
input
which
is
a
v
ector
X
=
(
x
1
;
:
:
:
;
x
T
)
(1)
This
along
with
certain
weights
is
passed
as
input,
to
a
stack
of
recurrently
connected
and
hidden
layers,
represented
by
A
,
to
calculate
the
first
hidden
v
ector
sequence
of
h
n
=
(
h
n
1
;
:
:
:
;
h
n
T
)
(2)
and
the
output
v
ector
sequence
is
y
=
(
y
1
;
:
:
:
;
y
T
)
(3)
This
output
v
ector
y
is
parameterized,
looped
and
redirected
back
into
the
netw
ork
as
a
set
of
inputs.
This
looping
feature
is
what
pro
vides
the
neural
netw
ork,
its
k
e
y
recurrence
nature.
T
o
observ
e
the
actual
nature
of
the
RNN
one
can
see
that
the
‘unrolled’
RNN
is
just
chained
normal
neural
netw
ork,
which
processes
the
data
as
required
and
passes
the
information
to
the
ne
xt
successor
.
No
w
,
this
can
be
considered
as
a
single
layer
netw
ork.
T
o
mak
e
it
a
‘deep’
RNN,
one
can
stack
them
one
on
top
of
the
other
.
No
w
,
one
of
the
main
problems
that
occurs
as
the
information
tra
v
erses
the
layers
is
that,
the
components
in
the
subsequent
layers
are
learning
f
aster
compared
to
the
pre
vious
layers.
This
problem
is
referred
to
as
the
‘V
anishing
Gradient
Problem’
(13).
Here,
gradients
are
the
v
alues
that
refer
to
speed
at
which
netw
orks
are
able
to
pick
up
and
learn
the
information.
This
is
an
important
f
actor
in
the
case
of
RNNs
since
the
data
is
propag
ated
back
into
the
netw
ork
and
if
the
v
alues
k
eep
reducing
as
the
y
propag
ate
through
the
layers,
then
e
v
entually
it
w
ould
tend
to
zero,
and
tha
t
w
ould
mak
e
the
model
useless.
Thus
to
o
v
ercome
this,
the
output
is
directly
repropag
ated
to
input
layers,
and
not
to
the
middle
layers.
Pr
ediction
of
Answer
K
e
ywor
ds
using
Char
-RNN
(Pr
atheek)
Evaluation Warning : The document was created with Spire.PDF for Python.
2168
ISSN:
2088-8708
No
w
as
this
process
is
happening,
the
other
problem
is
that,
as
it
learns
ne
w
information,
the
system
f
ails
to
k
eep
track
of
the
pre
viously
learnt
information.
T
o
o
v
ercome
this
ca
v
eat,
an
LSTM
style
of
RNN
is
used.
LSTM
(14).
The
LSTM
v
ariant
of
RNN
essentially
eliminates
the
main
dra
wback
of
the
standard
RNN,
which
is
its
ability
to
remember
,
and
this
v
ariant
w
orks
on
a
lar
ge
class
of
problems.
Figure
1.
Recurrent
Neural
Netw
ork
Figure
2.
RNN
e
xpanded
3.2.
Pr
oposed
W
ork
Man
y
algorithms
e
xist
to
predict
k
e
yw
ords,
b
ut
most
of
such
systems
e
xist
for
data
which
is
quite
lar
ge.
The
authors
w
ant
to
design
a
system
that
has
a
lo
w-o
v
erhead,
in
terms
of
model
b
uildi
ng
or
the
amount
of
data
required
to
predict.
The
model
the
authors
ha
v
e
chosen
is
an
GR
U
v
ariant
of
the
Recurrent
Neural
Netw
ork.
The
authors
split
the
o
v
erall
process
as
three
subprocesses
and
then
e
xpand
each
of
those
subprocesses’
indi
vidually
as
sho
wn
in
Figure
3
Data
Collecting
and
Preprocessing
Building
the
model
Ev
aluating
the
model
Figure
3.
Ov
erall
process
IJECE
V
ol.
9,
No.
3,
June
2019
:
2164
–
2176
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISSN:
2088-8708
2169
3.3.
Sub
Pr
ocess
1:
Data
Collecting
and
Pr
e-pr
ocessing
F
or
this,
the
data
is
of
homemade
or
self-made
dataset.
It
is
a
collection
of
questions,
stored
in
a
te
xt
file,
interlea
v
ed
with
a
ne
w
line.
An
e
xample
of
the
dataset
is
sho
wn
in
Figure
4.
Figure
4.
Example
of
the
homemade
dataset
First,
the
question
is
searched
on
a
normal
internet
search
engine,
and
then
the
results
obtained
are
processed.
This
is
done
by
first
con
v
erting
all
of
the
content
to
lo
wercase,
and
then
the
common
stop
w
ords
are
remo
v
ed.
F
or
this
task,
one
uses
a
certain
application
c
alled
SpaCy(15),
and
NL
TK(
16
)
,
(17)
which
is
used
for
natural
language
processing,
to
clear
out
the
said
w
ords
and
the
punctuation
marks
as
well
(While,
these
punctuation
marks
are
useful
for
a
normal
human
reader
,the
y
only
add
noise
to
the
model).
Then,
depending
on
the
type
of
the
question,
the
user
can
also
filter
out
numeric
v
alues
which
may
be
present
(Ha
ving
numeric
v
alues
is
useful,
only
if
the
answer
k
e
yw
ords
that
are
being
predicted
ha
v
e
them,
else
the
y
add
noise
to
our
model).
Once
this
pre-processing
is
repea
ted
for
all
the
fetched
data,
it
is
stored
into
a
file,
and
this
becomes
our
input
data
to
the
model.
The
process
for
which
is
sho
wn
belo
w
,
Figure
5.
Query
pro
vided
by
the
user
Perform
an
internet
search
on
the
query
Extract
all
the
links
from
the
response
Perform
a
request
on
each
of
those
links,
and
store
the
responses
Extract
the
ne
ws
headlines
from
each
of
the
responses
Figure
5.
Data
pre-processing
flo
w
diagram
Pr
ediction
of
Answer
K
e
ywor
ds
using
Char
-RNN
(Pr
atheek)
Evaluation Warning : The document was created with Spire.PDF for Python.
2170
ISSN:
2088-8708
3.4.
Sub
Pr
ocess
2:
Building
the
model
Here,
the
input
data
is
split
into
chunks
of
certain
size.
These
chunks
form
the
input
v
ectors
for
the
model.
Each
chunk
forms
a
single
input
v
ector
and
each
v
alue
from
the
v
ector
is
pro
vided
as
an
input
for
the
first
layer
of
the
RNN.
So,
logically
,
once
these
v
alues
are
“encoded”,
the
user
can
start
to
train
the
model
on
these
v
alues.
After
the
y
are
processed
by
the
first
layer
,
the
output
is
captured.
This
becomes
the
first
state.
This
state
is
retropropag
ated
as
input,
back
into
the
model.
As
the
model
goes
o
v
er
a
ne
w
encoded
v
ector
each
time,
it
remembers
what
it
has
trained
on
as
well
as
the
ne
wer
information
that
it
has
learnt.
Suppose,
the
first
input
is
cons
idered
to
ha
v
e
an
error
and
the
user
isn’
t
allo
wed
to
modify
it,
then
the
magnitude
of
the
error
multiplies
as
the
model
b
uilds
and
it
causes
problems.
But,
with
an
RNN
one
can
tweak
the
data
that
is
propag
ated
from
the
subsequent
layers,
thus
being
able
to
dimini
sh
the
error
rate
in
the
system.
This
is
because,
one
can
control
the
amount
of
information
that
is
being
retropropag
ated
back
into
the
system.
So,
if
the
first
input
contains
error
,
then
one
can
tak
e
the
output
after
the
first
iteration,
tweak
that
output
and
then
pro
vide
that
as
the
input
for
the
second
iteration.
So,
using
the
datas
et
once
can
train
the
whole
model,
and
after
this
one
can
start
e
v
aluating
the
model.
3.5.
Sub
Pr
ocess
3:
Ev
aluating
the
model
There
are
tw
o
possible
w
ays
to
e
v
aluate
the
model.
The
first
method
is
the
manual
method,
where
in
the
authors
manually
v
erify
the
predictions,
by
noting
the
answer
k
e
yw
ords
and
then,
manually
searching
for
the
query
and
v
erifying
the
results.
This
method
is
a
bit
tedious.
The
other
method
is
where
one
can
ha
v
e
a
program
that
automates
this.
This
w
ould
mak
e
it
much
simpler
.
In
the
case
of
automation,
the
process
w
ould
be
similar
to
the
manual
process.
The
authors
pro
vide
a
set
of
questions
at
a
time,
fi
v
e
questions,
for
e
xample.
No
w
,
the
y
perform
the
operation
for
each
of
the
question,
and
capture
the
output.
Then,
the
y
tok
enize
the
captured
output,
and
compare
this
with
a
set
answer
k
e
yw
ords,
that
the
authors
themselv
es
ha
v
e
pro
vided.
If
the
pro
vided
answer
k
e
yw
ords
is
said
to
be
found
in
the
tok
enized
w
ords,
then
the
model
is
said
to
ha
v
e
predicted
the
answer
correctly
,
else
the
model
has
predicted
inaccurately
.
4.
RESUL
TS
As
mentioned
in
the
section
3.5.,
for
the
automated
process,
one
has
obtained
a
set
of
questions
from
the
user
,
and
pro
vi
ded
this
as
the
testing
dataset
for
the
m
odel.
After
each
complete
run
of
the
model,
the
output
w
as
captured
and
compared
with
the
answers
that
w
as
pro
vided
by
the
users
themselv
es.
One
of
the
w
ays
in
which
the
system
w
as
tested,
w
as
by
manually
pro
viding
questions.
Here
the
au-
thors
consider
tw
o
questions,
the
first
being,
“
Who
won
the
2017
Nobel
prize
for
economics?
”
and
the
second
question
being,
“
Which
company
has
launched
the
Indian
Pale
Ale
(IPA),
its
fifth
beer
in
India
”.
F
or
the
first
question
the
obtained
results
are
sho
wn
in
Figure
6.
After
pro
viding
this
as
the
dataset
for
the
model
to
train,
the
system
is
able
to
predict
certain
sequence
of
w
ords
which
w
ould
match
the
results
that
w
ould
be
obtained
from
an
internet
search
query
.
Thi
s
is
sho
wn
in
the
follo
wing
figures.
Answer
k
e
yw
ord
patterns
predicted
by
the
system
is
sho
wn
in
Figure
7
and
the
answers
obtained
from
internet
searc
h
is
sho
wn
in
Figure
8.
The
results
of
the
second
question
w
as
also
v
erified
in
a
similar
manner
.
The
results
dataset
is
as
sho
wn
in
the
Figure
9.
This
is
pro
vided
as
input
for
the
system
to
train
on,
and
once
trained
the
system
be
gins
to
generate
answer
k
e
yw
ords
patterns,
similar
to
the
pre
vious
e
xample
and
is
sho
wn
in
the
Figure
10.
The
observ
ation
to
note
here
is
that,
when
queried
via
the
internet
search
engines,
similar
results
are
yielded,
as
sho
wn
in
the
Figure11
The
observ
ed
training
loss
for
each
of
the
questions
is
sho
wn
in
the
follo
wing
figures.
The
graph
of
the
training
loss
for
the
first
question
is
sho
wn
in
Figure
12
and
similarly
for
the
second
question
in
Figure
13.
One
general
observ
ation
to
note
in
both
the
cases
is
that
for
each
iteration,
the
amount
of
te
xt
being
processed
increases,
and
the
training
loss
decreases;
indicating
that
the
model
is
getting
more
ef
fecti
v
e
in
predicting
k
e
yw
ords.
IJECE
V
ol.
9,
No.
3,
June
2019
:
2164
–
2176
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISSN:
2088-8708
2171
Figure
6.
Example
of
the
fetch
results,
for
the
pro
vided
query
Figure
7.
Answer
k
e
yw
ord
patterns
predicted
by
the
system
Pr
ediction
of
Answer
K
e
ywor
ds
using
Char
-RNN
(Pr
atheek)
Evaluation Warning : The document was created with Spire.PDF for Python.
2172
ISSN:
2088-8708
Figure
8.
Answers
as
obtained
from
an
Internet
search
Figure
9.
Example
of
the
fetch
results
for
the
second
question
IJECE
V
ol.
9,
No.
3,
June
2019
:
2164
–
2176
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISSN:
2088-8708
2173
Figure
10.
Answer
k
e
yw
ords
as
predicted
by
the
system
Figure
11.
Answers
as
obtained
from
an
Internet
search
Pr
ediction
of
Answer
K
e
ywor
ds
using
Char
-RNN
(Pr
atheek)
Evaluation Warning : The document was created with Spire.PDF for Python.