Inter
national
J
our
nal
of
Electrical
and
Computer
Engineering
(IJECE)
V
ol.
10,
No.
4,
August
2020,
pp.
3826
3838
ISSN:
2088-8708,
DOI:
10.11591/ijece.v10i4.pp3826-3838
r
3826
Classification
of
emotions
induced
by
horr
or
and
r
elaxing
mo
vies
using
single-channel
EEG
r
ecordings
Amir
J
alilifard
1
,
Amir
Rastegar
nia
2
,
Ednaldo
Bir
gante
Pizzolato
3
,
Md
Kafiul
Islam
4
1,3
Department
of
Computer
Sceince,
Federal
Uni
v
ersity
of
Sao
Carlos,
Brazil
2
Department
of
Electrical
Engineering,
Malayer
Uni
v
ersity
,
Iran
4
Department
of
Electrical
and
Electronic
Engineering,
Independent
Uni
v
ersity
,
Bangladesh
Article
Inf
o
Article
history:
Recei
v
ed
Jun
12,
2019
Re
vised
Feb
2,
2019
Accepted
Feb
9,
2019
K
eyw
ords:
EEG
dynamics
Electroencephalograph
y
Emotion
classification
Fear
K-nearest
neighbors
Relaxation
Short-time
fourier
transform
Single-channel
Stationary
w
a
v
elet
Support
v
ector
machine
ABSTRA
CT
It
has
been
observ
ed
from
recent
studies
that
corticolimbic
Theta
rh
ythm
from
EEG
recordings
percei
v
ed
as
fear
or
threatening
scene
during
neural
processing
of
visual
stimuli.
In
additions,
neural
oscillations’
patterns
in
Theta,
Alpha
and
Beta
sub-bands
also
play
important
role
in
brain’
s
emotiona
l
processing.
Inspired
from
these
find-
ings,
in
this
paper
we
attempt
to
classify
tw
o
dif
ferent
emotional
states
by
analyzing
single-channel
EEG
recordings.
A
video
clip
that
can
e
v
ok
e
3
dif
ferent
emotional
states:
neutral,
relaxation
and
scary
is
sho
wn
to
19
colle
ge-aged
subjects
and
the
y
were
ask
ed
t
o
score
their
emotional
outcome
by
gi
ving
a
number
between
0
to
10
(where
0
means
not
scary
at
all
and
10
means
the
most
scary).
First,
rec
orded
EEG
data
were
preprocessed
by
stationary
w
a
v
elet
transform
(SWT)
based
artif
act
remo
v
al
algorithm.
Then
po
wer
distrib
ution
in
simultaneous
time-frequenc
y
domain
w
as
analyzed
using
short-time
F
ourier
transform
(STFT)
follo
wed
by
calculating
the
a
v
erage
po
wer
during
each
0.2s
time-se
gment
for
each
brain
sub-band.
Finally
,
46
features,
as
the
mean
po
wer
of
freque
nc
y
bands
between
4
and
50
Hz
during
each
time-se
gment,
con-
taining
689
instances—for
each
subject
—were
collected
for
classification.
W
e
found
that
relaxation
and
fear
emotions
e
v
ok
ed
during
w
a
tching
scary
and
relaxing
mo
vies
can
be
classified
with
a
v
erage
classification
rate
of
94.208%
using
K-NN
by
applying
methods
and
materials
proposed
in
this
paper
.
W
e
also
classified
the
dataset
using
SVM
and
we
found
out
that
K-NN
classifier
(when
k
=
1
)
outperforms
SVM
in
classifying
EEG
dynamics
induced
by
horror
and
relaxing
mo
vies,
ho
we
v
er
,
for
K
>
1
in
K-NN,
SVM
has
better
a
v
erage
classification
rate.
Copyright
c
2020
Insitute
of
Advanced
Engineeering
and
Science
.
All
rights
r
eserved.
Corresponding
A
uthor:
Md
Kafiul
Islam,
Senior
Member
of
IEEE,
Department
of
Electrical
and
Electronic
Engineering,
Independent
Uni
v
ersity
,
Bangladesh,
Academic
Block,
Le
v
el
5
6,
Plot-16,
Block-B,
Bashundhara,
Dhaka-1229.
T
el:
+880-1730871457
Email:
kafiul
islam@iub
.edu.bd
1.
INTR
ODUCTION
In
recent
years,
emotional
interaction
between
humans
and
machines
has
been
established
as
an
inter
-
esting
research
area
in
brain-computer
interf
ace
(BCI)
applications
[1].
Out
of
man
y
studies
for
emotion
recog-
nition,
tw
o
major
cate
gories
of
studies
are
most
common;
one
of
them
relies
on
tracking
the
f
acial
e
xpressions
[2,
3],
and
the
other
one
records
the
brain
signals
from
the
central
nerv
ous
system
by
dif
ferent
brain
record-
ing
techniques
such
as
electroencephalograph
y
(EEG)
,
electrocorticograph
y
(ECoG),
and
functional
magnetic
resonance
imaging
(fMRI)
during
e
xperiencing
dif
ferent
emotional
states.
Among
these
recording
techniques,
J
ournal
homepage:
http://ijece
.iaescor
e
.com/inde
x.php/IJECE
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
3827
EEG
is
one
of
the
most
suitable
because
it’
s
non-in
v
asi
v
e,
easy-to-use,
and
lo
w-cost
compared
to
others
and
it
has
also
been
pro
v
en
to
sho
w
informati
v
e
features
in
response
to
a
particular
emotional
state
[4–6].
The
frontal
brain’
s
acti
vity
has
been
found
to
be
responsible
for
emotional
response
[7].
Another
study
reported
similar
findings
that
frontal
EEG
asym
metry
can
serv
e
as
both
moderator
and
mediator
of
emotions
[5].
Inspired
from
these
findings,
adv
anced
engineering
techniques
such
as
signal
processing,
machine
learning
and
human-computer
interaction
ha
v
e
been
started
to
be
applied
in
order
to
in
v
estig
ate
EEG-
based
emotion
recognition
[8,
9].
Such
in
v
estig
ations
focus
broadly
on
tw
o
aspects:
studying
the
EEG
signal’
s
dynamics
through
dif
ferent
emotional
e
xperiences
[7,
10],
and
processing
or
classification
of
EEG
signals
during
each
emotional
state
[11,
12].
Dif
ferent
emotions
such
as
sadness,
happiness,
fear
,
disgust,
anger
,
jo
y
,
and
relaxation
ha
v
e
been
st
udied
by
tracking
EEG
dynamics
during
such
emotional
states.
In
some
of
the
studies,
emotions
are
class
ified
in
tw
o
broad
groups:
(1)
ne
g
ati
v
e
emotions
(i.e.
sadness,
disgust,
anger
,
fear);
and
(2)
positi
v
e
emoti
o
ns
(i.e.
jo
y
,
happiness,
relaxation)
[13–15].
Out
of
these
emotions,
fear
and
relaxation
ha
v
e
been
considered
in
fe
w
recent
literature
[10,
16,
17].
In
order
to
establish
emotional
interaction
with
smart-TV
or
computers,
emotions
must
be
detec
ted
with
reasonable
accurac
y
by
a
reliable
system
which
is
rob
ust
ag
ainst
artif
acts
and
other
interference.
There
are
se
v
eral
studies
found
that
concentrated
on
EEG
band
po
wer
for
classification
of
dif
ferent
emotions.
Iaco
viello
et
al.
proposed
a
real-time
EEG-based
algorithm
for
classifying
self-induced
emotions
[11]
where
the
subjects
were
s
ho
wn
a
group
of
symbols
to
induce
dif
ferent
emotions
such
as
disgust,
fear
,
anger
,
and
hunger
and
finally
the
y
achie
v
ed
an
a
v
erage
classification
accurac
y
of
90%
using
support
v
ector
machine
(SVM).
In
[18],
a
mo
vie
induction
e
xperiment
w
as
performed
which
spontaneously
led
the
subjects
to
get
real
emotional
states
and
classification
accurac
y
of
87.53%
using
SVM
with
linear
k
ernel
w
as
reported
at
the
end.
Naji
et
al.
performed
another
emotion
classification
e
xperiment
through
a
music
listening
task
by
recording
EEG
from
forehead
and
reached
an
a
v
erage
classificat
ion
rate
of
87.05%
using
SVM,
K-NN
and
CFNN
while
appl
ying
CFNN,
t
he
best
accurac
y
of
93.66%
w
as
obtained
[12].
Using
a
deep
learning
netw
ork
as
classifier
in
[19]
along
with
principal
component
analysis
(PCA)
as
feature
e
xtraction,
the
authors
reported
classification
accurac
y
of
46.03%
and
49.52%
to
separate
v
alence
and
arousal
respecti
v
ely
.
On
the
other
hand,
Mu
et
al.
used
a
62-channel
EEG
recorder
f
or
classifying
between
happiness
and
sadness
on
10
subjects
e
v
ok
ed
by
the
images
of
smile
and
cry
f
acial
e
xpressions
and
using
SVM,
the
y
achie
v
ed
classification
accuracies
of
93.5%
6.7%
for
3s-trials
and
93.0%
6.2%
for
1s-trials.
The
y
also
observ
ed
that
gamma
band
is
more
suitable
for
EEG-based
emotion
classification
[20].
In
this
paper
,
we
used
a
single
dry
electrode
based
EEG
recorder
placed
o
v
er
left
frontal
corte
x
(as
a
simplest
lo
w-cost
solution)
to
recognize
fear
and
relaxation
through
mo
vie
screening.
Then
we
attempted
to
classify
EEG
dynamics
related
to
fear
and
relaxation
with
an
acceptable
accurac
y
.
W
e
also
propose
an
artif
act
remo
v
al
algorithm
based
on
stationary
w
a
v
elet
transform
that
has
been
found
to
impro
v
e
the
final
classification
rate
in
emotion
recognition
task.
2.
B
A
CKGR
OUND
2.1.
Short
time
f
ourier
transf
orm
In
order
to
see
the
frequenc
y
v
ariances
in
time,
we
need
a
mathematical
tool
that
helps
to
understand
the
frequenc
y
changes
during
a
specific
time.
The
time-frequenc
y
representation
of
signal
can
be
obtained
in
dif
ferent
manners.
Short
duration
of
signal
with
windo
w
function
are
e
xtracted
using
short-time
F
ourier
transform
(STFT)
and
then,
its
frequenc
y
representation
is
calculated:
F
x
(
t;
f
;
h
)
=
Z
+
1
1
x
(
u
)
h
(
u
t
)
e
i
2
uf
du
(1)
where
h(t)
refers
to
the
sliding
analysis
windo
w
of
STFT
.
F
or
a
finite
ener
gy
windo
w
it
can
be
repre-
sented
as:
x
(
t
)
=
1
E
h
Z
+
1
1
Z
+
1
1
F
x
(
u;
f
;
h
)
h
(
t
u
)
e
i
2
tf
dud
f
(2)
Where
E
h
=
Z
+
1
1
j
h
(
t
)
j
2
dt
(3)
Classification
of
emotions
induced
by
...
(Amir
J
alilifar
d)
Evaluation Warning : The document was created with Spire.PDF for Python.
3828
r
ISSN:
2088-8708
Consequently
,
STFT
is
considered
as
a
tool
that
decomposes
the
signal
into
w
a
v
eforms,
and
used
to
find
the
ener
gy
distrib
ution
in
time-frequenc
y
space.
Since
STFT
is
easy
to
use
and
to
understand
[21],
in
this
study
it
is
used
for
decomposing
EEG
sub-bands
and
presenting
the
time-frequenc
y
dynamics.
2.2.
Stationary
wa
v
elet
transf
orm
The
recorded
EEG
data
are
contaminated
by
artif
acts
with
electrooculographic
(EOG)
and
electromyo-
graphic
(EMG)
or
igins.
In
order
to
minimize
the
so-cal
led
artif
acts,
stationary
w
a
v
elet
t
ransform
w
as
applied.
In
recent
years,
W
a
v
elet
based
denoising
methods
ha
v
e
attracted
much
attention
for
biomedical
signal
analysis,
because
it
is
suitable
for
non-linear
and
non-stationary
signals
[22],
such
as
EEG.
The
choice
of
w
a
v
elet
trans-
form
o
v
er
other
methods
(e.g.
independent
component
analysis
(ICA),
empirical
mode
decomposition(EMD),
adapti
v
e
filtering,
etc.)
is
moti
v
ated
by
its
capability
to
decompose
single-channel
EEG
data
into
dif
ferent
frequenc
y
bands
with
high
temporal
resolution
follo
wed
by
easier
denoising
technique
[22].
Another
adv
an-
tage
of
w
a
v
elet-based
methods
is
its
lo
wer
computational
comple
xity
compared
to
ICA
and
EMD.
In
addition,
unlik
e
adapti
v
e
filtering
or
re
gression
based
techniques,
it
doesn’
t
require
an
y
separate
reference
artif
act
channel.
There
are
se
v
eral
types
of
w
a
v
elet
transform
such
as
discrete
w
a
v
elet
transform
(D
WT),
contin-
uous
w
a
v
elet
transform
(CWT)
,
w
a
v
elet
pack
et
transform
(WPT),
stationary
w
a
v
e
let
transform
(SWT),
etc.
D
WT
is
the
simplest
w
a
v
elet
transform
technique,
alt
h
ough,
we
chose
SWT
for
its
adv
antage
o
v
er
D
WT
due
its
translational
in
v
ariance
property
(small
shift
in
signal
doesn’
t
cause
significant
changes
in
w
a
v
elet
coef
fi-
cients
and
lar
ge
v
ariations
in
distrib
ution
of
ener
gy
in
dif
ferent
w
a
v
elet
scales)
[23].
As
a
result,
during
signal
reconstruction,
no
(or
minimal)
distortion
in
signal
occurs
[24–26].
2.3.
Classification
2.3.1.
K-near
est
neighbors
The
nearest
neighbors
searching
is
the
follo
wing:
gi
v
en
a
set
S
of
n
data
points
in
the
feature
space,
X
D
,
the
task
is
to
process
thes
e
data
points,
so
that
gi
v
en
an
y
query
point
q
X
D
,
the
K
nearest
points
f
s
1
;
s
2
;
:::;
s
k
g
S
to
q
be
reported
as
quick
as
possible.
If
x
is
an
arbit
rary
instance
defined
by
feature
v
ector
as
<
a
1
(
x
)
;
a
2
(
x
)
;
a
3
(
x
)
;
:::::;
a
n
(
x
)
>
,where
a
r
(
x
)
indicates
the
v
alue
of
r
th
attrib
ute
of
x,
therefore
the
distance
is
calculated
by
d
(
x
i
;
x
j
)
where:
d
(
x
i
;
x
j
)
=
v
u
u
t
n
X
r
=1
(
a
r
(
x
i
)
a
r
(
x
j
))
2
(4)
Gi
ving
labeled
training,
K-NN
partitions
the
feature
space
to
polygonal-lik
e
sections
where
each
polygon
contains
all
the
nearest
neighbors
to
a
specific
point
in
feature
space.
This
algorithm
can
be
refined
by
gi
ving
a
weight
coef
ficient
based
on
the
distance
of
each
points
to
the
query
point.
Dudani
et
al.
introduced
a
weighted
v
oting
method
for
K-NN,
called
the
distance-weighted
k-nearest
neighbor
rule
(WKNN)
[27].
In
WKNN,
the
closer
neighbors
are
weighted
more
hea
vily
than
the
f
arther
ones.
2.3.2.
Support
v
ector
machine
Support
v
ector
machine
(SVM)
is
a
promising
method
to
classify
both
linear
and
non-linear
dat
asets.
It
uses
a
non-linear
mapping
in
order
to
transform
data
into
a
higher
dimension.
The
idea
is
to
find
the
max-
imum
mar
ginal
h
yperplanes
(MMH)
or
widest
street
[28]
between
separable
areas
among
infinite
number
of
separating
lines,
in
o
r
der
to
minimize
the
classificati
o
n
error
.
The
area
between
tw
o
separating
lines
is
called
plane
and
the
term
hyperplane
is
used
to
refer
the
ideal
decision
boundary
[29].
Hyperplane
can
be
written
as
W
:X
+
b
=
0
where
W
is
weight
v
ector
,
X
is
training
tuple
and
b
is
the
of
fset
parameter
as
sho
wn
in
Figure
1.
Considering
b
as
an
additional
weight,
the
points
that
are
abo
v
e
and
under
the
separating
plane
satisfy
y
i
(
w
0
+
w
1
x
1
+
w
2
x
2
)
1
;
8
i:
(5)
where
y
i
is
+1
if
the
point
is
abo
v
e
the
h
yperplane
and
-1
when
it
f
alls
under
the
h
yperplane.
Using
Lagrangian
formulation
the
MMH
is
written
as
a
decision
boundary
.
d
(
X
T
)
=
l
X
i
=1
y
i
i
X
i
X
T
+
b
0
(6)
where
X
i
is
support
v
ector
,
X
T
is
the
test
tuple
and
y
i
is
the
class
label
of
i
th
support
v
ector
.
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
3826
–
3838
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
3829
Gi
v
en
the
test
tuple
X
T
,
the
output
tells
in
which
part
of
the
plane
the
test
instance
f
alls.
If
the
result
is
a
black
point,
then
it
f
alls
on
the
top
of
plane
and
belongs
to
the
black
class
and
in
case
of
being
a
white,
it
f
alls
under
h
yperplane
and
belongs
to
white
class.
When
instances
aren’
t
linearly
separable,
SVM
transforms
data
to
a
ne
w
dimension
in
which
instances
related
to
each
class
will
be
separable
linearly
.
Rather
than
the
size
of
data,
the
comple
xity
of
the
learned
class
is
characterized
by
support
v
ectors.
Hence,
SVM
is
less
prone
to
o
v
er
-fitting
problem
[29].
Se
v
eral
in
v
estig
ations
sho
w
that
in
competition
among
sophisticated
algorithms
—lik
e
neural
netw
ork
and
support
v
ector
machine
—and
simple
ones
—for
e
xample,
k-nearest
neighbors
—,
the
clear
winner
isn’
t
necessarily
the
more
elaborate
one
[30,
31].
In
this
study
we
intend
to
classi
fy
emotions
using
SVM
and
K-NN,
so
as
to
find
which
algorithm
performs
better
in
emotion
classification.
Figure
1.
SVM
tends
to
find
the
widest
h
yperplane.
Support
v
ectors
are
sho
wn
with
a
red
border
[32]
3.
METHODS
AND
MA
TERIALS
3.1.
P
articipants
and
en
vir
onmental
pr
otocol
Nineteen
health
y
colle
ge/uni
v
ersity
students
aged
19-32
years
old
(14
males,
5
females)
partici
pated
in
this
e
xperiment
of
EEG
recording.
The
EEG
data
from
11
subjects
were
e
xcluded
from
analysis
later
due
to
presence
of
e
xcessi
v
e
noise
or
major
mo
v
ement
artif
acts
—such
as
touching
forehead
and/or
EEG
equipment
(6
subjects)—or
because
the
subject
reported
that
the
mo
vies
were
already
w
atched
by
them
bef
o
r
e
and
thus
fear
emotion
w
as
not
elicited
in
them
at
all
(5
subjects).
Before
starting
the
e
xperiment,
all
the
participants
were
ask
ed
to
sign
a
consent
form
after
careful
reading
and
then
the
y
were
requested
to
sit
still.
Also,
in
order
to
a
v
oid
bias,
no
technical
or
general
information
w
as
gi
v
en
to
the
participants
about
the
e
xperiment.
This
is
dif
ferent
from
some
other
in
v
estig
ations
where
the
y
ha
v
e
been
ask
ed
to
focus
on
each
corresponding
emotion
to
a
picture
or
mo
vie
clip,
or
the
feeding
beha
viors
are
controlled
24
hours
before
data
acquisition
[33,
14].
Our
final
cohort
comprised
of
8
subjects
(6
males
and
2
females).
The
e
xperiment
w
as
carried
out
in
a
dark
small
room
using
an
LCD
monitor
placed
50
cm
a
w
ay
from
the
subject.
The
place
w
as
de
v
oid
of
an
y
noise
or
interruption.
The
subjects
were
requested
not
to
stand
up
and
not
to
remo
v
e
the
EEG
headset
immediately
after
finishing
the
e
xperiment.
At
the
end,
subjects
themselv
es
e
v
aluated
their
le
v
el
of
fear
for
horror
mo
vie
and
their
general
relaxation
state
during
the
relaxing
mo
vie,
by
gi
ving
a
number
between
0-10
where
0
means
no
fear
at
all
and
10
means
highest
le
v
el
of
fear
.
3.2.
Stimuli
Acquiring
meaningful
data
is
critical
in
an
y
signal
processing
application.
Ho
we
v
er
,
acquiring
data
corresponding
to
a
particular
emotional
state
i
s
challenging
due
to
the
subjecti
v
e
nature
of
emotions
as
well
as
cogniti
v
e
dependenc
y
of
ph
ysiol
o
gi
cal
signals
that
requires
a
specific
emotion
must
be
elicited
internally
in
participants
[34].
Thus,
in
order
to
stimulate
fear
and
relaxation
emotional
states
in
subjects,
a
set
of
combined
mo
vie
clips
with
a
total
duration
of
288
s
ec
w
as
used
without
an
y
break
time
during
the
e
xperi-
ment.
This
e
xperiment
w
as
consistent
with
fe
w
pre
vious
studies
where
video
clips
were
used
as
emotional
stimuli
[35–37].
As
sho
wn
in
T
able
1,
the
video
sho
wn
to
the
subjects
consists
of
2
mo
vie
clips
which
elicit
tw
o
tar
get
emotional
states
such
as
positi
v
e
(relaxation)
and
ne
g
ati
v
e
(fear).
The
selecti
on
criteria
for
mo
vie
clips
are:
(a)
the
mo
vie
needs
to
be
easily
understood
without
e
xplanation;
and
(b)
both
mo
vies
ha
v
e
to
elicit
single
desired
tar
get
emotion.
Classification
of
emotions
induced
by
...
(Amir
J
alilifar
d)
Evaluation Warning : The document was created with Spire.PDF for Python.
3830
r
ISSN:
2088-8708
T
able
1.
V
ideo
Clip
Description
Number
Film
title
T
ar
get
emotion
Y
ear
1
The
Ela
v
ator
Neutral
2011
2
Matr
a
Positi
v
e
(Relaxation)
2012
3
Lights
out
Ne
g
ati
v
e
(Fear)
2013
Before
starting
the
EEG
recording
and
data
acquisition
process,
se
v
eral
e
xperimental
arrangem
ents
were
carried
out
in
order
to
find
the
best
situation
for
obtaining
the
best
possible
data.
Through
these
e
xperi-
ments,
we
found
out
that
during
the
first
minute
of
test,
users
tend
to
ha
v
e
some
mo
v
ements,
such
as
trying
to
touch
the
EEG
de
vice,
strong
head
mo
v
ements
and
rubbing
e
yes,
that
cause
major
artif
acts.
In
order
to
a
v
oid
this
phase,
we
dec
ided
to
us
e
a
neutr
al
mo
vie
du
r
ing
t
he
first
90
seconds,
which
neither
arouses
ne
g
ati
v
e
nor
positi
v
e
emotion.
The
neutral
mo
vie
is
follo
wed
by
relaxing
(60
seconds)
and
horror
mo
vies
(138
seconds).
The
process
of
our
e
xperiment
is
depicted
in
Figure
2.
Figure
2.
The
process
of
mo
vie
broadcasting
3.3.
EEG
r
ecording
One
of
the
issues
in
EEG-related
e
xperiments
is
that
the
setup
of
the
EEG
recorder
is
not
straight
forw
ard
because
of
wet-electrodes.
thus
we
ha
v
e
used
a
wireless
EEG
headset
[38]
from
NeuroSk
y
with
a
single
dry-electrode
attached
to
the
forehead
at
position
Fp1
as
sho
wn
in
Figure
3),
a
reference
electrode
on
the
ear
clip
and
the
sampling
frequenc
y
w
as
512
Hz.
Figure
3.
Electrode
placement
for
single-channel
EEG.
Channel
Label:
Fp1
[39]
3.4.
Methods
The
recorded
data
were
analyzed
by
step-by-step
processes
including
feature
smoothing,
signal
processing,
feature
e
xtraction
and
finally
classification
for
emotional
state
as
depicted
in
Figure
4.
One
of
the
purposes
of
this
study
is
also
to
e
v
aluate
the
ef
fect
of
the
artif
act
remo
v
al
algorithm
on
the
classification
stage.
In
order
to
do
that,
tw
o
types
of
training
data
were
collected:
(1)
artif
actual
and
(2)
artif
act-free.
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
3826
–
3838
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
3831
Figure
4.
The
flo
wchart
of
emotion
classification
3.5.
EEG
pr
epr
ocessing
EEG
data
were
preprocessed
on
MA
TLAB
(v8.3)
platform
(The
MathW
orks,
Inc.)
for
correction
of
the
stereotyped
artif
acts
including
e
ye
blinks,
e
ye
mo
v
ements,
muscle
artif
acts
and
line
noise,
by
denois-
ing
with
the
help
of
Stationary
W
a
v
elet
T
ransform
(SWT).
A
6-le
v
el
decomposition
of
SWT
(with
Haar
as
mother/basis
w
a
v
elet)
w
as
applied
on
the
recorded
ra
w
signal.
The
output
of
SWT
is
a
set
of
detail
(d1-d6)
and
final
approximate
coef
ficients
(a6),
representing
non-o
v
erlapping
high
and
lo
w
frequenc
y
bands
respecti
v
ely
as
sho
wn
in
T
able2.
T
able
2.
Frequenc
y
bands
and
the
corresponding
labels
in
training
dataset
SWT
Coef.(Le
v
el
=
5,
Fs
=
512
Hz)
d1
d2
d3
d4
d5
d6
a5
Freq.
Band
(Hz)
128—256
64—128
32—64
16—32
8-16
4–8
0–4
EEG
Rh
ythm
Component
Gamma
Beta
Alpha
Theta
Delta
A
modified
uni
v
ersal
threshold
proposed
by
[40]
w
as
then
applied
on
dif
ferent
decomposition
le
v
els
of
coef
ficients
in
order
to
separate
potential
artif
acts
from
actual
EEG
rh
ythms.
During
denoising,
the
threshold
v
alue
and
threshold
function
are
carefully
selected
according
to
the
w
ork
by
Islam
et
al.
[41].
The
threshold
v
alue
is
calculated
by
belo
w
equation
follo
wed
by
the
e
xpression
of
g
arrote
t
hreshold
function
46
as
gi
v
en
belo
w:
Threshold
v
alue
at
le
v
el
i
,
T
i
0
=
K
i
sq
r
t
(2
l
n
(
N
));
(7)
N
is
the
length
of
data,
i
is
the
estimated
noise
v
ariance
for
W
i
which
is
the
w
a
v
elet
coef
ficients
at
the
i
th
decomposition
le
v
el
(
W
i
=
a
i
for
approximation
coef
ficient
and
W
i
=
a
i
for
detail
coef
ficients)
Threshold
function,
i
=
x
;
j
x
j
<
=
T
0
i
T
0
2
i
=x
;
j
x
j
>
T
0
i
(8)
Where
i
is
the
g
arrote
threshold
function
at
each
decomposition
le
v
el
of
i
and
x
is
the
signal
v
alue
of
the
w
a
v
elet
coef
ficients.
During
denoising,
the
choice
of
threshold
parameter
K
is
v
ery
critical
in
order
to
mak
e
sure
that
no
distortion
in
the
desired
signal
of
interest
[41]
occurs.
This
is
because
the
distinct
frequenc
y
bands
represent
dif
ferent
EEG
rh
ythms
and
some
coef
ficients
are
more
lik
ely
to
contain
artif
acts
than
others.
Finally
,
the
artif
act-reduced
EEG
signal
is
reconstructed
back
using
the
ne
w
set
of
w
a
v
elet
coef
ficients
after
application
of
in
v
erse
SWT
.
The
complete
process
flo
w
of
artif
act
remo
v
al
is
sho
wn
in
Figure
5.
Figure
5.
Artif
act
Remo
v
al
Process
Flo
w
3.6.
Signal
pr
ocessing
and
featur
e
extraction
EEG
rh
ythms
or
sub-bands
consisting
of
Theta
(4–8
Hz),
Alpha
(8–12
Hz),
Beta
(12–30
Hz)
and
Gamma
(30–50
Hz),
related
to
relaxing
and
horror
mo
vies,
were
decomposed
using
Short-time
F
ourier
trans-
form
(STFT).
Each
input
time-series
w
as
di
vided
into
a
sequence
of
se
gments
with
duration
of
0.2
sec
and
then
Classification
of
emotions
induced
by
...
(Amir
J
alilifar
d)
Evaluation Warning : The document was created with Spire.PDF for Python.
3832
r
ISSN:
2088-8708
the
mean
signal
po
wer
w
as
calculated
for
each
frequenc
y
band
during
each
of
the
time-se
gments.
The
self-
reported
forms
from
the
subjects
were
used
to
e
xclude
data
related
to
the
parts
of
horror
mo
vie
that
didn’
t
carry
an
y
fear
emotion.
Finally
,
46
features,
as
the
mean
po
wer
of
frequenc
y
bands
between
4
and
50
Hz
during
each
time-se
gment,
containing
689
instances
(388
for
scary
and
301
for
relaxing
mo
vie)
for
each
subject,
labeled
with
either
’Scary’
or
’Relaxing’,
formed
tw
o
training
data-sets
(artif
act-free
and
artif
actual).
4.
EXPERIMENT
AL
RESUL
TS
AND
DISCUSSIONS
K-nearest
neighbors
wi
th
Euclidean
distance
w
as
used
for
emotions
classification
task.
In
order
to
a
v
oid
the
o
v
er
-fitting
problem,
a
10-folds
cross-v
alidation
w
as
applied
on
the
datasets.
In
k-fold
cross-
v
alidation,
the
dataset
is
se
gmented
into
K
equal
portions
and
then
through
k
iterations,
each
time
one
of
the
folds
is
used
as
v
alidation
set
and
the
k-1
remaining
folds
are
used
for
training.
W
e
tested
the
classification
rate
for
k
ranges
between
k
=
1
and
k
=
p
n
where
n
is
the
number
of
instances
in
the
dataset
[42].
W
e
also
chose
the
K
parameter
to
be
odd
numbers
in
order
to
a
v
oiding
ties
[43].
T
able
3.
Frequenc
y
bands
and
the
corresponding
labels
in
training
dataset
Frequenc
y
band
Frequenc
y
ranges(Hz)
Features
considered
for
trianing
Meaning
Theta
4-7
F4
-
F7
The
mean
ener
gy
for
time
se
gmentation
of
0.2s
Alpha
8-12
F8
-
F12
The
mean
ener
gy
for
time
se
gmentation
of
0.2s
Beta
13-30
F13
-
F30
The
mean
ener
gy
for
time
se
gmentation
of
0.2s
Gamma
30-50
F30
-
F50
The
mean
ener
gy
for
time
se
gmentation
of
0.2s
W
e
compared
the
results
and
found
k
=
1
as
the
best
K-NN
parameter
for
both
datasets
with
and
without
using
artif
act
remo
v
al
algorithm
as
sho
wn
in
T
able
4
and
T
able
5).
F
or
7
subjects,
applying
SWT
algorithm
impro
v
ed
the
classification
rates
between
1.32
%
and
9.43%.
Also,
for
1
subject
an
insignificant
decrease
in
classification
rate
(-0.58%)
is
seen.
Ho
we
v
er
,
the
datasets
are
unbalanced
(388
instances
of
Scary
mo
vie
and
301
instances
of
relaxing
mo
vie),
b
ut
observing
the
confusion
matrix
sho
ws
an
almost
balanced
state
of
wrongly
classified
instances
for
both
’Scary’
and
’Relaxing’
classes.
Based
on
confusion
matrix,
we
may
calculate
the
true
positi
v
e
(TP)
and
f
alse
positi
v
e(FP)
rates,
as
well.
W
e
sho
w
the
results
in
details
for
both
datasets
—artif
act-free
and
artif
actual
data
—for
K-NN
algorithm
since
it
gi
v
es
the
best
classification
result.
F
or
SVM
the
r
esults
are
sho
wn
just
for
artif
act-free
dat
aset,
ho
we
v
er
using
SWT
increased
the
classification
rate
in
SVM,
as
well.
T
able
4.
The
K-NN
classification
result
for
dataset
without
using
SWT
,
k=1
and
cross-v
alidated
with
10
folds
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
91.146%
0.911
0.092
2
Male
91.001%
0.91
0.087
3
Male
88.679%
0.887
0.109
4
Male
93.468%
0.935
0.069
5
Female
90.420
%
0.915
0.11
6
Male
96.661%
0.96
0.033
7
Female
93.468%
0.935
0.065
8
Male
84.325%
0.843
0.843
Classification
a
v
erage
rate:
91.146%
T
able
5.
The
K-NN
classification
result
for
dataset
after
using
SWT
,
k=1
and
cross-v
alidated
with
10
folds
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
90.566%
0.906
0.104
2
Male
95.791%
0.958
0.045
3
Male
93.759%
0.938
0.061
4
Male
95.791%
0.958
0.042
5
Female
95.21
%
0.952
0.051
6
Male
97.933%
0.979
0.021
7
Female
96.371%
0.964
0.036
8
Male
88.243%
0.882
0.119
Classification
a
v
erage
rate:
94.208%
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
3826
–
3838
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
3833
The
best
res
ult
w
as
obtained
for
k
=
1
,
ho
we
v
er
,
we
check
ed
the
results
for
all
K
’
s
between
k
=
1
and
k
=
p
(
n
)
2
weighted
by
1
=distance
.
In
T
ables
6,7,8,9,10
and
table
11
the
results
are
sho
wn
for
k
=
3
,
k
=
5
and
k
=
7
.
T
able
6.
The
K-NN
classification
result
for
datasets
without
using
SWT
,
k=3
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
86.792%
0.868
0.137
2
Male
87.227%
0.872
0.124
3
Male
82.293%
0.823
0.179
4
Male
91.436%
0.914
0.088
5
Female
84.905
%
0.849
0.158
6
Male
90.566%
0.906
0.093
7
Female
84.18%
0.842
0.158
8
Male
79.825%
0.798
0.212
Classification
a
v
erage
rate:
85.903%
T
able
7.
The
K-NN
classification
result
for
datasets
after
using
SWT
,
k=3
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
85.341%
0.853
0.158
2
Male
93.468%
0.935
0.062
3
Male
88.679%
0.887
0.109
4
Male
93.033%
0.93
0.069
5
Female
90.420
%
0.90
0.099
6
Male
96.661%
0.967
0.033
7
Female
93.468%
0.935
0.065
8
Male
84.325%
0.843
0.163
Classification
a
v
erage
rate:
90.674%
T
able
8.
The
K-NN
classification
result
for
datasets
without
using
SWT
,
k=5
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
84.18%
0.842
0.167
2
Male
86.937%
0.869
0.126
3
Male
80.69%
0.807
0.199
4
Male
87.373%
0.874
0.126
5
Female
81.422
%
0.814
0.198
6
Male
88.824%
0.888
0.111
7
Female
80.261%
0.807
0.203
8
Male
75.181%
0.752
0.255
Classification
a
v
erage
rate:
83.108%
T
able
9.
The
K-NN
classification
result
for
datasets
after
using
SWT
,
k=5
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
81.422%
0.814
0.197
2
Male
88.969%
0.89
0.106
3
Male
84.470%
0.845
0.155
4
Male
89.114%
0.891
0.108
5
Female
88.098
%
0.814
0.198
6
Male
92.527%
0.925
0.075
7
Female
91.291%
0.913
0.086
8
Male
80.841%
0.808
0.197
Classification
a
v
erage
rate:
87.091%
Classification
of
emotions
induced
by
...
(Amir
J
alilifar
d)
Evaluation Warning : The document was created with Spire.PDF for Python.
3834
r
ISSN:
2088-8708
T
able
10.
The
K-NN
classification
result
for
datasets
without
using
SWT
,
k=7
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
83.599%
0.836
0.173
2
Male
85.195%
0.852
0.142
3
Male
78.229%
0.782
0.782
4
Male
86.647%
0.866
0.136
5
Female
81.277
%
0.813
0.199
6
Male
87.518%
0.875
0.124
7
Female
79.1%
0.791
0.2
8
Male
74.165%
0.742
0.27
Classification
a
v
erage
rate:
81.966%
T
able
11.
The
K-NN
classification
result
for
datasets
after
using
SWT
,
k=7
and
cross-v
alidated
with
10
folds
and
distance-weighted
Subject
Gender
Classification
rate
A
vg.
TP
rate
A
vg.
FP
rate
1
Male
79.10%
0.791
0.223
2
Male
87.08%
0.871
0.124
3
Male
82.873%
0.829
0.172
4
Male
87.373%
0.874
0.127
5
Female
87.317
%
0.873
0.123
6
Male
91.891%
0.919
0.081
7
Female
88.824%
0.888
0.109
8
Male
77.358%
00.774
0.233
Classification
a
v
erage
rate:
85.227%
F
or
k
=
1
,
k
=
3
,
k
=
5
and
k
=
7
,
using
the
SWT
caused
ha
ving
better
a
v
erage
classification
accuracies
of
+3.06%,
+4.771%,
+3.983%
and
+3.261%,
respecti
v
ely
.
F
or
all
the
subjects
e
xcept
subject1,
the
classification
rate
after
artif
act
remo
v
al
w
as
raised.
Also
we
had
the
best
result
for
subject7
with
11.3%
impro
v
ement
(for
k=7)
in
classification
rate
as
sho
wn
in
Figure
6,
Figure
7,
Figure
8
and
Figure
9.
Figure
6.
The
classification
rate
for
k=1
before
and
after
using
SWT
for
artif
act
remo
v
al
[39]
Figure
7.
The
classification
rate
for
k=3
before
and
after
using
SWT
for
artif
act
remo
v
al
[39]
Int
J
Elec
&
Comp
Eng,
V
ol.
10,
No.
4,
August
2020
:
3826
–
3838
Evaluation Warning : The document was created with Spire.PDF for Python.
Int
J
Elec
&
Comp
Eng
ISSN:
2088-8708
r
3835
Figure
8.
The
classification
rate
for
k=5
before
and
after
using
SWT
for
artif
act
remo
v
al
[39]
Figure
9.
The
classification
rate
for
k=7
before
and
after
using
SWT
for
artif
act
remo
v
al
[39]
In
order
to
find
the
best
po
wer
spectrum
feature
for
emotion
classification,
we
repeated
the
classifi
ca-
tion
tas
k
for
each
of
the
features
mentioned
in
T
able
3
for
all
the
subjects,
using
the
artif
act-free
datasets,
when
k
=
1
.
As
it
is
sho
wn
in
T
able
12,
the
best
accurac
y
w
as
obtained
when
a
combination
of
all
po
wer
spectrum
features
were
eng
aged
in
emotion
classification.
Furthermore,
the
w
orst
and
the
best
a
v
erage
classification
accuracies
were
obtained
by
Theta
and
Gamma
bands
with
65.368%
and
94.208%,
respecti
v
ely
.
This
finding
is
consistent
with
the
result
of
the
other
studies
in
which
it
w
as
mentioned
that
high
frequenc
y
bands
ha
v
e
a
major
role
in
emotional
acti
vities
[18,
20].
T
able
12.
Classification
accurac
y
of
K-NN
(K=1),
using
dif
ferent
po
wer
spectrum
features
Subject
Gender
Theta
Alpha
Beta
Gamma
All
1
Male
59.07%
63.28%
76.052%
79.680%
90.566%
2
Male
60.232%
60.522%
80.87%
84.323%
91.001%
3
Male
56.603%
63.86%
78.229%
85.229%
93.759%
4
Male
63.57%
64.876%
81.132%
82.873%
95.791%
5
Female
64.005%
66.763%
85.050%
86.792%
95.210%
6
Male
76.947%
76.629%
91.07%
90.938%
97.933%
7
Female
76.777%
75.907%
86.937%
85.486%
96.371%
8
Male
65.747%
66.037%
80.406%
82.148%
88.243%
Classification
a
v
erage
rate:
65.368%
67.234%
82.468%
84.683%
94.208%
In
order
to
get
the
classification
accurac
y
using
SVM,
the
Pearson
VI
I
function-based
uni
v
ersal
k
ernel
[44]
w
as
used
along
with
10-folds
cross-v
alidation
and
the
cost
parameters
C
:
C
10
5
;
10
4
;
:::;
10
2
.
Fear
and
relaxation
were
classified
with
90.816%
a
v
erage
classification
rate
as
sho
wn
in
T
able
13.
The
classification
rate
of
K-NN,
for
K
=
1
,
is
higher
than
SVM,
although,
we
don’
t
claim
that
using
e
xact
nearest
neighbors
is
the
best
classifier
for
emotion
classification.
This
is
because
there
is
a
main
point
of
concern
about
the
laziness
of
K-NN
and
its
performance
when
the
size
—or
the
dimensionality
—of
training
dataset
is
big.
One
w
ay
to
o
v
ercome
the
so-called
laziness
problem
can
be
using
another
type
of
nearest
neighbor
algorithm
called
approximate
nearest
neighbors
(ANN)
lik
e
locality
sensiti
v
e
hashing
(LSH),
in
order
to
reduce
the
search
space
and
consequently
,
ha
ving
a
f
ast
instance-based
classifier
[45].
Classification
of
emotions
induced
by
...
(Amir
J
alilifar
d)
Evaluation Warning : The document was created with Spire.PDF for Python.