TELK
OMNIKA
T
elecommunication,
Computing,
Electr
onics
and
Contr
ol
V
ol.
19,
No.
1,
February
2021,
pp.
173
181
ISSN:
1693-6930,
accredited
First
Grade
by
K
emenristekdikti,
No:
21/E/KPT/2018
DOI:
10.12928/TELK
OMNIKA.v19i1.16281
r
173
T
raffic
sign
detection
optimization
using
color
and
shape
segmentation
as
pr
e-pr
ocessing
system
Handok
o,
J
ehoshua
Hank
y
Pratama,
Banu
W
irawan
Y
ohanes
Department
of
Electronic
and
Computer
Engineering,
Uni
v
ersitas
Kristen
Satya
W
acana,
Indonesia
Article
Inf
o
Article
history:
Recei
v
ed
Apr
8,
2020
Re
vised
Jun
13,
2020
Accepted
Jul
6,
2020
K
eyw
ords:
Color
se
gmentation
Optimization
Shape
se
gmentation
T
raf
fic
sign
detection
ABSTRA
CT
One
of
performance
indicators
in
an
autonomous
v
ehicle
(A
V)
is
its
ability
to
accom-
modate
rapid
en
vironment
changing;
and
performance
of
traf
fic
sign
detection
(TSD)
system
is
one
of
them.
A
lo
w
frame
rate
of
TSD
impacts
to
late
decision
making
and
may
cause
to
a
f
atal
accident.
Meanwhile,
adding
an
y
GPU
to
TSD
will
significantly
increases
its
cost
and
mak
e
it
unaf
fordable.
This
paper
proposed
a
pre-processing
system
for
TSD
which
implement
a
color
and
a
shape
s
e
gmentation
to
increase
the
system
speed.
These
se
gmentat
ion
systems
filter
input
frames
such
that
the
number
of
frames
sent
to
artificial
intelligence
(AI)
system
is
reduced.
As
a
result,
w
orkload
of
AI
system
is
decreased
and
its
frame
rate
increases.
HSV
threshold
is
used
in
color
se
gmentation
to
filter
frames
with
no
desired
color
.
This
algorithm
ignores
the
satura-
tion
when
performing
color
detection.
Further
,
an
edge
detection
feature
is
emplo
yed
in
shape
se
gmentation
to
count
the
total
contours
of
an
object.
Using
German
traf
fic
sign
recognition
benchmark
dataset
as
model,
the
pre-processing
system
filters
97%
of
frames
with
no
traf
fic
sign
objects
and
has
an
accurac
y
of
88%.
TSD
system
proposed
allo
ws
a
frame
rate
impro
v
ement
up
to
32
frame
per
second
(FPS)
when
Y
ou
Only
Look
Once
(Y
OLO)
algorithm
is
used.
This
is
an
open
access
article
under
the
CC
BY
-SA
license
.
Corresponding
A
uthor:
Handok
o
Department
of
Electronic
and
Computer
Engineering
Uni
v
ersitas
Kristen
Satya
W
acana
Dipone
goro
52-60,
Salatig
a,
Central
Ja
v
a,
Indonesia
Email:
handok
o@uksw
.edu
1.
INTR
ODUCTION
In
industrial
re
v
olution
4.0
era,
research
in
autonomous
v
ehicle
(A
V)
are
e
xtensi
v
ely
encouraged
to
b
uild
a
modern
and
safer
transportation.
An
A
V
which
is
equipped
with
an
enormous
number
of
sensors
(infrared,
camera,
proximity
,
etc)
allo
ws
sensiti
v
e
sense
for
the
change
of
its
surrounding
to
pro
vi
de
maximum
safety
for
human
being.
Since
it
is
controlled
by
computational
de
vices
without
an
y
human
as
a
dri
v
er
,
A
V
is
claimed
to
lo
wer
do
wn
the
accident
rate
by
90%
[1].
One
of
the
crucial
parts
in
an
A
V
is
its
detection
system,
which
has
to
be
v
ery
responsi
v
e
to
an
y
changes
of
its
surrounding,
including
traf
fic
signs.
The
speed
of
processing
in
the
detection
system,
in
term
of
frame
rate
number
,
will
define
correct
or
incorrect
decisions.
The
lo
w
frame
rate
indicates
slo
w
response
of
the
traf
fic
sign
dete
ction
(TSD)
system
to
respond
to
the
e
xternal
changes
including
the
e
xistence
of
traf
fic
signs.
Since
2015,
a
number
of
accidents
which
in
v
olv
e
A
Vs
had
been
reported
and
the
lo
w
frame
rate
w
as
responsible
for
most
of
these
accidents.
Most
of
the
accidents
w
as
caused
by
the
system
f
ailure
of
the
A
V
to
J
ournal
homepage:
http://journal.uad.ac.id/inde
x.php/TELK
OMNIKA
Evaluation Warning : The document was created with Spire.PDF for Python.
174
r
ISSN:
1693-6930
mak
e
a
f
ast
and
accurate
decision
[2,
3].
Therefore,
a
good
TSD
system
must
pro
vide
a
frame
rate
of
minimum
25
frame
per
second
(FPS)
in
order
to
react
instantly
b
ut
still
preserv
e
its
accurac
y
.
A
high
frame
rate
TSD
can
be
obtained
in
tw
o
w
ays
either
using
additional
graphics
processing
unit
(GPU)
or
by
impro
v
ement
of
its
algorithm.
An
additional
GPU
will
instantly
boost
the
frame
rate
of
TSD
since
the
detection
process
will
be
performed
using
additional
processors
and
memories.
Ho
we
v
er
,
GPU
e
xtension
are
costly
and
as
the
consequence
the
v
ehicle
price
will
not
af
fordable.
In
contrast,
impro
v
ement
of
algorithm
for
the
current
system
may
produces
a
higher
speed
TSD
with
a
lo
wer
cost.
As
one
of
the
detection
systems,
the
TSD
system
requires
an
e
xtensi
v
e
computation
resources
at
the
serv
er
and
produces
a
poor
frame
rate.
Meanwhile,
not
all
frames
sent
for
processing
at
the
detection
system
contains
traf
fic
sign
objects.
Therefore,
application
of
a
preliminary
system
which
is
able
to
determine
frames
to
pass
for
processing
at
the
artificial
intelligence
(AI)
system
and
which
ones
should
be
denied,
will
significantly
increase
TSD
frame
rate.
Since
the
traf
fic
signs
ha
v
e
a
specific
characteristic
in
colors
and
shapes,
the
se
gmentation
of
color
and
shape
systems
are
good
option
to
perform
frame
filtering
in
the
pre-processing
system
and
allo
ws
a
lo
wer
w
orkload
for
artificial
intelligence
system
[3].
2.
PREVIOUS
W
ORK
T
raf
fic
sign
detection
is
typical
ly
classified
into
three
fundamental
cate
gories:
color
-based,
shape-
based
and
learning
based
methods
[4].
M
eanwhile,
Liu
[5]
add
tw
o
mode
cate
gories:
color
and
shape
based
methods
and
light
detection
and
ranging
(LID
AR)
based
methods.
Among
these
methods,
detection
system
based
on
learning
is
the
best
because
it
achie
v
es
a
detection
rate
to
more
than
90%.
Using
a
deep
learning
method
such
as
con
v
olutional
neural
netw
ork
(CNN)
e
v
en
increases
the
detection
rate
to
more
than
98%
[6].
Ho
we
v
er
,
the
e
xisting
algorithms
are
still
f
ar
from
a
real
time
classified
system
[4].
Balado
[7]
proposed
an
algorithm
to
achie
v
e
a
better
detection
rate,
b
ut
still
could
not
reach
a
real
time
processing
frame
rate.
Research
on
traf
fic
sign
detection
based
on
CNN
proposed
by
W
ang
[8]
produced
a
detection
rate
of
5
FPS
which
is
f
ar
from
the
real
time
processing
standard.
A
multi-scale
cascaded
R-CNN
is
proposed
by
Zhang
[9]
and
Liu
[10]
to
o
v
ercome
the
problem
of
f
alse
detection
when
traf
fic
signs
are
in
smal
l
sizes.
Moreo
v
er
,
a
real
time
object
detection
named
Y
OLO
and
its
modification
are
often
used
to
tackle
the
real
time
TSD
problems
[9,
11,
12].
Sphurti
More
in
[13],
found
that
Y
OLO
sho
wed
a
better
accurac
y
than
support
v
ector
machine
(SVM)
based
algorithm,
although
it
run
slo
wer
.
On
the
other
hand,
classification
using
color
-based
and
shape-based
algorithms
are
usually
f
ast
b
ut
the
y
perform
a
lo
wer
det
ection
rate
which
is
not
suitable
for
A
V
system.
Color
-based
se
gmentation
algorithms
can
be
applied
for
TSD
since
the
traf
fic
si
gns
ha
v
e
specific
characteristic
colors:
red,
bl
ue
and
yello
w
[14].
The
red
sign
is
used
for
signs
which
represent
a
danger
or
prohibition,
yello
w
sign
sho
ws
construction
signs
and
blue
sign
is
for
information
[14].
Color
thresholding,
re
gion
gro
wing,
color
inde
xing,
dynamic
pix
el
aggre
g
ation
are
most
popular
color
-based
detection
methods
[15].
Ho
we
v
er
,
the
v
ariation
of
color
due
to
the
age
of
signs,
v
ariation
of
light
mak
es
color
-based
se
gmentation
algorithms
produce
lo
w
detection
rates
[4].
Some
researches
on
color
se
gmentation
algorithms
sho
w
that
using
HSV
as
the
color
space
produces
better
results
although
it
requires
a
longer
processing
time
[16,
17,
18].
Meanwhile,
shape-based
se
gmentation
algorithms
are
commonly
used
t
o
identify
objects.
In
the
TSD
system,
there
are
four
specific
shapes
to
identify
since
traf
fic
signs
can
be
classified
into
rectangular
,
circular
,
octagonal
and
triangular
signs.
W
ali
[15]
listed
a
number
of
shape-based
methods
such
as:
Hough
transfor
-
mation,
similarity
detection,
distance
transform
matching,
edge
detection
features
and
Haar
-lik
e
fea
tures.
The
combination
of
an
adapti
v
e
color
threshold
se
gmentation
method
and
a
shape
symmetry
detection
algorithm
w
as
proposed
in
[19]
to
get
higher
detection
rate
in
TSD.
3.
PR
OPOSED
SYSTEM
Based
on
the
performance
of
color
-based,
shape-based
and
learning-based
method
discused
earlier
,
we
proposed
a
system
to
increase
the
detection
rate
of
learning-based
TSD
by
application
of
a
frame
filtering
system.
In
the
real
implementation
of
TSD,
the
number
of
frames
without
an
y
traf
fic
sign
object
typically
more
than
the
ones
with
traf
fic
signs.
Therefore,
processing
only
the
fram
es
which
contain
traf
fic
signs
will
significantly
increase
its
o
v
erall
detection
rate.
The
proposed
system
consists
of
tw
o
important
processes:
pre-
processing
and
AI
system.
Figure
1
sho
ws
the
diagram
of
proposed
TSD
system.
The
pre-processing
part
applies
a
color
se
gmentation
and
a
shape
se
gmentation
algorithm.
The
y
are
used
to
pass
only
the
candidate
TELK
OMNIKA
T
elecommun
Comput
El
Control,
V
ol.
19,
No.
1,
February
2021
:
173
–
181
Evaluation Warning : The document was created with Spire.PDF for Python.
TELK
OMNIKA
T
elecommun
Comput
El
Control
r
175
frames
which
contain
an
y
traf
fic
sign
to
the
AI
system.
Then,
the
system
will
continue
the
processing
to
detect
whether
the
candidate
frame
is
actually
ha
ving
traf
fic
sign
in
it
by
application
of
learning-based
algorithm
(AI).
Figure
1.
The
proposed
A
V
system
diagram
Input
from
camera
is
resized
using
a
bilinear
int
erpolation
algorithm
which
performs
linear
i
nterpo-
lation
algorithm
twice
in
dif
ferent
directions.
This
m
ethod
uses
four
nearest
point
to
calculate
the
point
in
the
center
of
four
points
chosen
[20,
8].
Resizing
is
used
to
reduce
the
number
of
iteration
when
performing
con
v
olution
in
a
frame.
Bilinear
interpolation
is
chosen
because
it
produces
a
smoother
interpolation
compares
to
the
other
resizing
methods
[21,
22].
Since
traf
fic
signs
ha
v
e
specific
characteristic
of
shapes
and
colors,
we
limit
the
detection
of
shapes
for
square,
triangle,
circle
and
octagon;
and
the
detection
of
colors
for
red,
blue
and
yello
w
.
A
frame
which
has
none
of
t
h
e
se
shapes
and
colors
can
be
classified
as
a
frame
without
an
y
traf
fic
sign,
then
it
can
be
e
xcluded
from
further
processing.
3.1.
Color
segmentation
In
this
w
ork,
we
use
HSV
threshold
for
color
se
gmentation
to
identify
the
desired
color
a
v
ailable
in
a
frame.
Before
being
processed
by
HSV
threshold,
the
particular
frame
is
filtered
using
Gaussian
blur
to
smoothen
and
enhance
the
frame’
s
color
feature.
It
remo
v
es
unw
anted
salt
noises
of
the
images
and
mak
es
the
color
detection
easier
and
more
accurate.
Gaussian
blur
uses
the
k
ernel
windo
w
to
perform
blurring
of
an
image.
The
blur
le
v
el
can
be
adjusted
by
modification
of
k
ernel
size.
The
bigger
k
ernel
size
will
produce
a
higher
blur
of
image
which
is
unlik
ely
can
be
detected
correctly
[23].
Figure
2.
sho
ws
the
5x5
Gaussian
blur
k
ernel
windo
w
.
Figure
2.
Gaussian
blur
k
ernel
windo
w
[24]
HSV
color
space
is
used
because
the
color
can
be
detected
according
to
its
hue
and
v
alue
wi
thout
considering
the
saturation
of
the
color
,
and
according
to
the
pre
vious
w
orks,
using
the
HSV
color
space
sho
wed
a
better
performance
than
RGB.
The
HSV
threshold
is
set
to
the
color
desired
such
as
red,
blue
and
yello
w
.
The
upper
and
lo
wer
bound
of
the
thresholds
are
chosen
based
on
the
e
xperiment
done
by
Adrian
from
PyIm-
ageSearch
[25].
The
threshold
[110,
50,
50]
to
[130,
255,
255]
is
used
for
blue
color
,
[20,
100,
100]
to
[30,
255,
255]
is
for
yello
w
color
.
Meanwhile,
the
threshold
[160,
20,
70]
to
[190,
255,
255]
is
for
red
color
[25].
Algorithm
1
sho
ws
an
implementation
of
color
se
gmentation
method.
3.2.
Shape
segmentation
In
this
w
ork,
we
use
an
edge
detect
ion
feature
method
for
shape
se
gmentation
of
the
frames.
A
mor
-
phological
filter
is
emplo
yed
to
reduce
the
number
of
noises
in
a
frame
and
to
add
pix
els
between
important
T
r
af
fic
sign
detection
optimization
using
color
and...
(Handok
o)
Evaluation Warning : The document was created with Spire.PDF for Python.
176
r
ISSN:
1693-6930
features.
This
technique
uses
a
nonlinear
operation
[26].
There
are
fi
v
e
morphological
filter
operations:
dila-
tion
,
er
osion
,
opening
,
closing
and
hit
or
miss
transform.
Dilation
and
er
osion
are
the
main
operations
in
the
morphological
filter
.
Meanwhile,
the
opening
,
closing
and
hit
or
miss
transforms
are
formed
by
the
process
of
combination
between
dilation
and
er
osion
[27].
Dilation
is
an
operation
which
can
thick
en
the
structure
of
object
in
a
frame
by
adding
e
xtra
pix
els
between
important
features.
The
1
is
used
for
dilation
operation
[28].
In
addition,
the
er
osion
is
an
operation
which
in
contrast
to
the
dilation
operation.
In
this
operation,
the
structure
of
an
object
will
be
reduced
by
elimination
of
pix
els
around
the
object.
The
2
sho
ws
the
equation
of
er
osion
operation
[28].
H
(
p
+
q
)
j
for
e
v
ery
p
2
I
;
q
2
H
(1)
H
p
2
Z
2
j
(
p
+
q
)
2
I
;
for
e
v
ery
q
2
H
(2)
These
operations
are
used
to
enhance
the
features
of
an
object
insi
d
e
the
fram
e.
Morphological
fi
lter
allo
ws
an
easier
detection
of
the
objects
in
an
image.
After
the
frame
being
processed
by
morphological
filter
,
the
contour
of
object
in
the
frame
is
detected
and
counted.
The
Douglass
Peuck
er
algorithm
is
used
to
count
the
contour
of
object.
It
reduces
the
number
of
curv
e
in
an
image
and
transform
it
to
a
straight
line
between
tw
o
points.
As
a
result,
the
contour
of
object
can
be
identified
easier
[29].
If
the
number
of
contours
is
4,
then
the
object
is
classified
as
a
rectangle
object.
The
total
contours
of
3,
15
and
8
represent
a
triangle,
circle
and
octagon
object
repecti
v
ely
.
The
frame
which
is
classified
ha
vi
ng
one
of
the
desired
shapes
will
be
processed
by
the
AI
system.
Algorithm
2
sho
ws
an
implementation
of
the
shape
se
gmentation
method.
Algorithm
1
Color
se
gmentation
1:
Set
threshold
for
red,
yello
w
and
blue
colors
2:
while
!(end
of
frame)
do
3:
Get
a
frame
from
input
camera
4:
Apply
Resizing
for
the
current
frame
5:
Apply
a
Gaussian
Blur
filter
to
the
frame
6:
if
(the
frame
contains
at
least
one
color
desired)
then
7:
Send
the
current
frame
to
the
shape
se
gmentation
system
8:
else
9:
get
the
ne
xt
frame
10:
end
if
11:
end
while
Algorithm
2
Shape
se
gmentation
1:
Set
desired
contour
v
alues
2:
while
!(end
of
frame)
do
3:
Get
a
frame
from
color
se
gmentation
system
4:
Apply
Mor
phological
filter
to
the
current
frame
5:
Apply
a
Douglass
P
euck
er
algorithm
to
the
frame
6:
if
(the
number
of
contour
macth
one
of
the
desired
shape)
then
7:
Send
the
current
frame
to
the
AI
system
8:
else
9:
Get
the
ne
xt
frame
10:
end
if
11:
end
while
3.3.
Artificial
intelligence
(AI)
system
A
frame
which
is
successfully
pass
pre-processing
system
is
processed
by
the
AI
system
for
detection
and
classification
.
In
AI
system,
the
frame
is
resized
to
a
smaller
size
to
reduce
the
number
of
neural
netw
ork
processing.
In
this
w
ork,
Y
ou
Only
Look
Once
(Y
OLO)
algorithm
is
emplo
yed
as
the
processing
of
the
AI
system.
Y
OLO
is
an
open
source
system
which
has
se
v
eral
v
ersions:
Y
OLO,
Y
OLOv2,
Y
OLOv3
and
T
in
y
Y
OLO.
Furthermore,
Y
OLOv2
is
used
because
it
is
f
aster
and
accurate
enough
for
a
TSD
system
[30].
Y
OLO
TELK
OMNIKA
T
elecommun
Comput
El
Control,
V
ol.
19,
No.
1,
February
2021
:
173
–
181
Evaluation Warning : The document was created with Spire.PDF for Python.
TELK
OMNIKA
T
elecommun
Comput
El
Control
r
177
has
a
feature
netw
ork
with
24
con
v
olutional
layers
and
2
fully
connected
layers
[31].
Con
v
olutional
layer
is
used
to
e
xtract
the
important
feature
in
a
frame
by
application
of
a
con
v
olution
using
5x5
mask.
This
mask
will
k
eep
mo
ving
through
the
a
v
ailable
position
in
a
fram
e
to
get
its
important
features
[32].
The
output
of
this
process
is
a
smaller
matrix
with
important
feature
pix
els.
Figure
3
describes
the
process
in
a
con
v
olution
layer
.
The
output
of
this
process
is
a
list
of
neurons
which
includes
all
the
important
v
alues
from
the
layer
.
This
list
will
be
connected
in
a
fully
connected
layer
and
become
one
final
feature
map.
The
final
feature
map
will
be
compared
to
the
feature
map
from
the
model
which
has
been
obtained
from
the
traini
ng
phase.
If
the
feature
map
of
the
frame
matches
to
one
of
the
feature
maps
in
the
model,
then
the
system
is
success
to
detect
and
classify
the
particular
object
[33].
Figure
3.
The
con
v
olution
process
[32]
Max
pooling
will
be
performed
after
finishing
the
con
v
olution
process.
Max
pooling
will
tak
e
the
biggest
v
alue
in
the
matrix
which
is
assumed
as
the
feature
which
being
searched.
Similar
to
the
con
v
olutional
layer
,
max
pooling
emplo
ys
a
matrix
which
will
mo
v
e
throughout
the
frame
and
compare
the
v
alues
in
the
matrix
[34,
35].
Max
pooling
is
chosen
because
of
the
condition
of
the
dataset
where
most
of
the
images
ha
v
e
a
bright
color
.
4.
RESUL
T
AND
DISCUSSION
In
this
w
ork,
we
borro
w
the
German
T
raf
fic
Sign
Recognition
Benchmark
(GTSRB)
dataset
to
con-
struct
our
model.
Figure
4
sho
ws
the
sample
images
of
the
dataset.
W
e
di
vide
our
e
xperiments
into
tw
o
parts,
the
pre-processing
part
and
the
full
system
part.
The
pre-processing
e
xperiments
are
done
with
5
dif
ferent
configurations.
Meanwhile,
the
full
system
is
used
to
measure
the
frame
rate
of
entire
system
with
and
without
pre-processing.
Figure
4.
Sample
of
dataset
images
4.1.
Pr
e-pr
ocessing
The
first
part
of
the
e
xperiments
in
v
olv
es
43
traf
fic
signs
of
the
GTSRB
dataset
.
This
e
xperiment
run
with
dif
ferent
combination
of
parameter
settings
including
the
size
of
open
k
ernel
,
size
of
close
k
ernel
and
size
of
Gaussian
blur
.
In
this
e
xperiment
the
still
images
from
the
dataset
are
used
as
input
of
the
system.
Figure
5
sho
ws
an
e
xample
of
pre-processing
of
an
image
where
the
color
se
gmentation
system
catch
a
yello
w
object
and
T
r
af
fic
sign
detection
optimization
using
color
and...
(Handok
o)
Evaluation Warning : The document was created with Spire.PDF for Python.
178
r
ISSN:
1693-6930
the
shape
se
gmentation
identifies
a
rectangular
sign.
T
able
1
sho
ws
the
percentage
of
the
successful
detection
and
filtering
of
the
e
xper
iment
using
43
traf
fic
signs
with
v
arious
parameter
v
alues.
The
successful
detection
v
alue
represents
ho
w
good
the
system
can
recognize
traf
fic
signs
by
using
the
shape
and
color
of
the
signs.
Meanwhile,
the
successful
filter
v
alue
sho
ws
the
total
frame
successfully
pass
to
the
ne
xt
processing
compare
to
the
total
of
input
frames.
As
we
can
see
from
the
result,
changes
on
parameters
do
ef
fect
the
accurac
y
of
detection
and
filtering
of
the
frames.
Figure
5.
Pre-processing
of
a
frame;
(a)
original
image,
(b)
shape
and
color
se
gmentation
result,
(c)
detection
result
T
able
1.
Pre-processing
e
xperiment
results
Experiment
1
Experiment
2
Experiment
3
Experiment
4
Experiment
5
Open
K
ernel
size
5
x
5
px
11
x
11
px
5
x
5
px
5
x
5
px
7
x
7
px
Close
K
ernel
size
5
x
5
px
5
x
5
px
11
x
11
px
5
x
5
px
5
x
5
px
Gaussian
Blur
size
3
x
3
px
3
x
3
px
3
x
3
px
5
x
5
px
3
x
3
px
Frame
T
otal
1835
frames
1835
frames
1835
frames
1835
frames
1835
frames
T
rue
Positi
v
e
47
frames
40
frames
46
frames
53
frames
53
frames
T
rue
Ne
g
ati
v
e
9
frames
7
frames
9
frames
11
frames
16
frames
F
alse
Positi
v
e
1772
frames
1774
frames
1772
frames
1765
frames
1760
frames
F
alse
Ne
g
ati
v
e
7
frames
14
frames
8
frames
1
frames
1
frames
Successful
Detection
Percentage
84%
77%
81%
88%
88%
Successful
Filter
Percentage
97,05%
97,34%
97,08%
97,18%
96,53%
In
e
xperiment
1,
the
open
and
close
k
ernel
were
set
to
5x5
and
Gaussian
blur
w
as
set
to
3x3.
In
this
configuration,
there
were
a
lot
of
salt
noises
found
in
the
frame
and
will
produce
inef
ficient
frame
filtering.
In
e
xperiment
2,
the
open
k
ernel
size
were
set
to
11x11
and
the
Gaussian
blur
w
as
set
to
3x3.
Using
this
configuration,
we
obtain
a
better
result
than
the
first
e
xperiment
since
the
open
k
ernel
can
filter
more
frames.
Ho
we
v
er
the
detection
accurac
y
in
e
xperiment
2
decreased
to
77%.
This
de
gradation
w
as
caused
by
the
size
of
the
open
k
ernel
which
is
too
big
and
therefore
will
eliminate
some
of
the
important
features
in
the
frame.
In
e
xperiment
3,
the
close
k
ernel
w
as
set
to
11x11
and
open
k
ernel
to
5x5.
The
result
when
using
this
configuration
is
slightly
higher
than
the
e
xperim
ent
2
with
the
detection
accurac
y
up
to
81%.
It
is
better
because
the
close
k
ernel
capable
to
close
the
pix
el
g
aps
in
the
frame
which
will
enhance
the
feature
of
traf
fic
sign
in
a
frame.
In
the
ne
xt
e
xperiment,
the
Gaussian
blur
size,
k
ernel
open
and
k
ernel
close
were
set
to
5x5.
Using
these
parameter
v
alues
enable
the
system
to
get
a
higher
percentage
in
detection
accurac
y
and
filtering.
The
bigger
size
of
Gaussian
blur
mak
es
the
feature
in
a
frame
can
be
easily
detected.
In
the
last
e
xperiment,
the
size
of
open
k
ernel
w
as
increased
to
7x7
where
the
close
k
ernel
and
the
Gaussian
blur
were
maintained
to
5x5
and
3x3.
The
result
of
using
these
parameter
v
alues
w
as
not
as
e
xpected
where
the
performance
of
filtering
frame
were
lo
wer
than
e
xperiment
4.
Since
the
optimal
results
are
reached
in
e
xperiment
4,
then
these
parameters
are
used
for
further
e
xperiments.
T
able
2
sho
ws
the
a
v
erage
time
needed
to
process
a
single
frame
in
the
pre-processing
system.
TELK
OMNIKA
T
elecommun
Comput
El
Control,
V
ol.
19,
No.
1,
February
2021
:
173
–
181
Evaluation Warning : The document was created with Spire.PDF for Python.
TELK
OMNIKA
T
elecommun
Comput
El
Control
r
179
T
able
2.
Pre-processing
time
elapse
for
a
single
frame
Processing
A
v
erage
processing
time
for
1
frame)
Color
Se
gmentation
11.3
ms
Shape
Se
gmentation
0.4
ms
T
otal
pre-processing
time
11.7
ms
4.2.
T
raffic
sign
detection
(TSD)
system
In
the
ne
xt
e
xperiment,
we
us
e
some
videos
as
inputs
of
the
TSD
system.
Figures
6
(a
and
c)
sho
ws
an
e
xample
of
processing
a
video
which
is
tak
en
from
a
car
dashboard.
The
o
v
erall
processing
time
for
e
v
ery
processing
step
and
the
total
frame
processed
are
displayed
on
the
Figures
6
(b
and
d).
Figure
6.
(a,
c)
TSD
with
pre-processing
system,
(b,
d)
Number
of
frames
processed
and
the
time
processing
In
the
ne
xt
step,
we
measure
a
v
erage
time
needed
to
process
a
single
frame.
The
time
process
ing
includes
pre-processing
system,
transfer
file,
and
artificial
intelligence
system,
and
can
be
sho
wn
in
T
able
3.
It
sho
ws
that
pre-processing
system
tak
es
1.012
seconds
in
a
v
erage
to
process
a
single
frame.
Meanwhile,
without
the
pre-processing
system
tak
es
on
a
v
erage
1.0003
second
which
includes
the
transfer
file.
This
means
application
of
pre-process
ing
system
increases
the
time
processing
of
the
TSD
system
up
to
1.17%
when
an
y
candidate
traf
fic
sign
object
is
identified.
On
the
other
hand,
when
no
candidate
traf
fic
sign
object
is
identified,
the
system
tak
es
0.0117
second
which
is
the
pre-processing
time
only
since
the
transfer
file
and
AI
processing
are
not
required.
Using
10
frame
videos
with
10%
of
the
frames
ha
ving
a
traf
fic
sign
object,
we
s
ummaries
the
time
processing
of
the
sys
tem
as
sho
wn
in
T
able
4.
The
processing
time
is
reduced
from
10.003
to
1.12
second
T
r
af
fic
sign
detection
optimization
using
color
and...
(Handok
o)
Evaluation Warning : The document was created with Spire.PDF for Python.
180
r
ISSN:
1693-6930
(88%)
and
as
the
consequences,
the
frame
rate
will
increase.
The
result
on
Figure
6
sho
ws
that
the
FPS
v
alues
v
ary
from
6.8
FPS
to
105.64
FPS.
The
lo
w
frame
rate
indicates
the
corresponding
frame
has
an
y
candidate
traf
fic
sign
object.
On
the
other
hand,
system
will
g
ain
a
high
frame
rate
when
no
candidate
traf
fic
sign
object
is
detected.
The
e
xperiment
results
sho
ws
that
proposed
algorithm
with
pre-proce
ssing
system
can
boost
up
the
speed
of
TSD
system
to
the
a
v
erage
of
32
FPS.
T
able
3.
Processing
time
of
full
system
for
a
single
frame
A
v
erage
time
for
processing
1
frame
(second)
Pre-processing
system
0.0117
s
T
ransfer
file
via
TCP
0.0003
s
AI
System
1.0000
s
T
otal
T
ime
1.0120
s
T
able
4.
T
ime
processing
for
10
frame
videos
with
10%
contains
an
y
traf
fic
sign
object
Pre-processing
(s)
T
ransfer
File
(s)
AI
(s)
T
otal
(s)
W
ithout
Pre-processing
0
10*0.0003
10*1.0000
10.0030
W
ith
Pre-processing
10*0.0117
10*0.0003
1*1.0000
1.1200
5.
CONCLUSION
AND
FUTURE
W
ORK
Using
color
and
shape
se
gmentation
as
the
pre-processing
system
allo
ws
a
frame
rate
impro
v
ement
of
the
traf
fic
sign
detection
system.
Using
open
k
ernel,
close
k
ernel
and
Gaussian
blur
matrix
size
of
5x5
pix
els,
this
system
is
able
to
filter
up
to
97.18%
of
the
frames
and
pass
less
than
3%
for
processing
at
the
AI
system.
On
a
v
erage,
the
filtering
system
r
eaches
88%
detection
accurac
y
.
As
the
number
of
frames
sent
to
the
AI
system
is
reduced,
the
w
orkload
of
the
artificial
intelligence
system
is
decreased
and
as
a
result
,
the
frame
rate
of
this
detection
system
increases
up
to
32
FPS
without
using
an
y
graphics
processing
uni
t
(GPU).
In
the
future,
refinement
of
color
and
shape
se
gmentation
algorithms
can
be
used
to
increase
the
accurac
y
of
pre-processing
and
therefore
impro
v
e
accurac
y
of
the
whole
system.
REFERENCES
[1]
K.
Ser
geant,
“
Autonom
ous
v
ehicles:
Risk
management
issues
and
concern,
”
Loc
kton
Companies
,
March
2017.
[2]
V
.
V
.
Dixit,
S.
Chand,
and
D.
J.
Nair
,
“
Autonomous
v
ehicles:
Diseng
agements,
accidents
and
reaction
times,
”
PLOS
ONE
,
v
ol.
11,
no.
12,
pp.
1–14,
2016.
[3]
J.
H.
Pratama,
Handok
o,
and
B.
W
.
Y
ohanes,
“Frame
rat
e
optimization
in
traf
fic
sign
detection,
”
in
International
Confer
ence
on
Cybernatics
and
Intellig
ent
System
,
2019.
[4]
Y
.
Saadna
and
A.
Behloul,
“
An
o
v
ervie
w
of
traf
fic
sign
dete
ction
and
classification
methods,
”
International
J
ournal
of
Multimedia
Information
Retrie
val
,
v
ol.
6,
pp.
1–18,
2017.
[5]
C.
Liu,
S.
Li,
F
.
Chang,
and
Y
.
W
ang,
“Machine
vision
based
traf
fic
sign
detection
methods:
Re
vi
e
w
,
analyses
and
perspecti
v
es,
”
IEEE
Access
,
v
ol.
7,
pp.
86
578–86
596,
2019.
[6]
J.
Jiang,
X.
Feng,
F
.
Liu,
Y
.
Xu,
and
H.
Huang,
“Multi-spectral
r
gb-nir
image
classification
using
double-channel
cnn,
”
IEEE
Access
,
v
ol.
7,
pp.
20
607–20
613,
2019.
[7]
J.
Balado
Frias,
E.
Gonz
´
alez,
P
.
Arias,
and
D.
Castro,
“No
v
el
approach
to
automatic
traf
fic
sign
in
v
entory
based
on
mobile
mapping
system
data
and
deep
learning,
”
Remote
Sensing
,
v
ol.
12,
no.
3,
pp.
442–457,
2020.
[8]
Z.
W
ang
and
H.
Guo,
“Research
on
traf
fic
sign
detection
based
on
con
v
olutional
neural
netw
ork,
”
in
Pr
oc.
of
the
12th
Int.
Symposium
on
V
isual
Information
Communication
and
Inter
action
,
2019.
[9]
J.
Zhang,
Z.
Xie,
J.
Sun,
X.
Zou,
and
J.
W
ang,
“
A
cascaded
r
-cnn
with
multiscale
attention
and
imbalanced
samples
for
traf
fic
sign
detection,
”
IEEE
Access
,
v
ol.
8,
pp.
29
742–29
754,
2020.
[10]
Z.
Liu,
D.
Li,
S.
Ge,
and
et
al.
,
“Small
traf
fic
sign
detection
from
lar
ge
image,
”
Applied
Intellig
ence
,
v
ol.
50,
pp.
1–13,
2019.
[11]
X.
Liu
and
F
.
Xi
ong,
“
A
real-time
traf
fic
sign
detection
model
based
on
impro
v
ed
yolo
v3,
”
IOP
Confer
ence
Series:
Materials
Science
and
Engineering
,
v
ol.
787,
p.
012034,
2020.
[12]
Q.
Xu,
R.
Lin,
H.
Y
ue,
H.
Huang,
Y
.
Y
ang,
and
Z.
Y
ao,
“Research
on
smal
l
tar
get
detection
in
dri
ving
scenarios
based
on
impro
v
ed
yolo
netw
ork,
”
IEEE
Access
,
v
ol.
8,
pp.
27
574–27
583,
2020.
[13]
S.
More
and
J.
Bos,
“Comparing
machine
learning
and
neural
netw
ork-based
approaches
for
sign
detection
and
classification
in
autonomous
v
ehicles,
”
in
A
utonomous
Systems:
Sensor
s,
Pr
ocessing
,
and
Security
for
V
ehicles
and
Infr
astructur
e
2020
,
M.
C.
Dudzik
and
S.
M.
Jameson,
Eds.,
v
ol.
11415,
International
Society
for
Optics
and
Photonics.
SPIE,
2020,
pp.
129–134.
[Online].
A
v
ailable:
https://doi.or
g/10.1117/12.2558966
TELK
OMNIKA
T
elecommun
Comput
El
Control,
V
ol.
19,
No.
1,
February
2021
:
173
–
181
Evaluation Warning : The document was created with Spire.PDF for Python.
TELK
OMNIKA
T
elecommun
Comput
El
Control
r
181
[14]
N.
Mohd
Ali,
S.
Karis,
A.
F
.
Zainal
Abidin,
B.
Bakri,
E.
F
.
Shair
,
and
N.
Razif,
“T
raf
fic
sign
detection
and
recognition:
Re
vie
w
and
analysis,
”
J
urnal
T
eknolo
gi
,
v
ol.
77,
no.
20,
pp.
107–113,
2015.
[15]
S.
W
ali,
M.
Abdullah,
M.
A.
Hannan,
A.
Hussain,
S.
Samad,
P
.
J.
K
er
,
and
M.
Mansor
,
“V
ision-based
traf
fic
sign
detection
and
recognition
systems:
Current
trends
and
challenges,
”
Sensor
s
,
v
ol.
19,
pp.
2093–2120,
2019.
[16]
X.
W
ang,
R.
H
¨
ansch,
L.
Ma,
and
O.
Hell
wich,
“Comparison
of
dif
ferent
color
spaces
for
image
se
gmentation
using
graph-cut,
”
2014
International
Confer
ence
on
Computer
V
ision
Theory
and
Applications
(VISAPP)
,
v
ol.
1,
pp.
301–
308,
2014.
[17]
M.
Hassan,
R.
Ema,
and
T
.
Islam,
“Color
image
se
gmentation
using
automated
k-means
clustering
with
r
gb
and
hsv
color
spaces,
”
Global
Jpurnal
Inc.
(USA)
,
v
ol.
17,
no.
2,
pp.
33–41,
2017.
[18]
S.
Sural,
Gang
Qian,
and
S.
Pramanik,
“Se
gmentation
and
histogram
generation
using
the
hsv
color
space
for
image
retrie
v
al,
”
Pr
oceedings.
International
Confer
ence
on
Ima
g
e
Pr
ocessing
,
v
ol.
2,
pp.
II–II,
2002.
[19]
X.
Xu,
J.
Jin,
S.
Zhang,
L.
Zhang,
S.
Pu,
and
Z.
Chen,
“Smart
data
dri
v
en
traf
fic
sign
detection
method
based
on
adapti
v
e
color
threshold
and
shape
symmetry
,
”
Futur
e
Gener
ation
Computer
Systems
,
v
ol.
94,
pp.
381–391,
2018.
[20]
P
.
A.
Dilip,
K.
Rameshbab
u,
K.
P
.
Ashok,
and
S.
A.
Shi
vdas,
“Bilinear
interpolation
i
mage
scaling
processor
for
vlsi
bilinear
interpolation
image
scaling
proces
sor
for
vlsi
architecure,
”
International
J
ournal
of
Reconfigur
able
and
Embedded
Systems
(IJRES)
,
v
ol.
3,
no.
3,
pp.
104–113,
No
v
ember
2014.
[21]
A.
C.
Bo
vik,
“Chapter
4
-
basic
binary
image
processing,
”
pp.
69
–
96,
2009.
[22]
X.
W
ang,
X.
Jia,
W
.
Zhou,
X.
Qin,
and
H.
Guo,
“Correction
for
color
art
if
acts
using
the
r
gb
intersection
and
the
weighted
bilinear
interpolation,
”
Appl.
Opt.
,
v
ol.
58,
no.
29,
pp.
8083–8091,
Oct
2019.
[23]
E.
Gedraite
and
M.
Hadad,
“In
v
estig
ation
on
the
ef
fect
of
a
g
aussian
blur
in
image
filtering
and
se
gmentation,
”
pp.
393–396,
2011.
[24]
A.
P
athmanabhan
and
S.
Dinesh,
“The
ef
fect
of
g
aussian
blurring
on
the
e
xtraction
of
peaks
and
pits
from
digital
ele
v
ation
models,
”
Discr
ete
Dynamics
in
Natur
e
and
Society
,
v
ol.
2007,
pp.
1–12,
No
v
ember
2007.
[25]
A.
Rosebrock,
“Opencv
and
p
ython
color
detection,
”
2014.
[Online].
A
v
ailable:
https://www
.p
yimagesearch.com/2014/08/04/opencv-p
ython-color
-detection/
[26]
R.
K.
P
ande
y
and
S.
S.
Mathurkar
,
“
A
re
vie
w
on
morphological
filter
and
its
implementation,
”
Internatinal
J
ournal
of
Science
and
Resear
c
h
,
v
ol.
6,
no.
1,
pp.
69–71,
2015.
[27]
A.
M.
Raid,
W
.
M.
Khedr
,
M.
A.
El-dosuk
y
,
and
M.
Aoud,
“Image
restoration
based
on
morphological
operations,
”
International
J
ournal
of
Computer
Science
,
Engineering
and
Information
T
ec
hnolo
gy
(IJCSEIT)
,
v
ol.
4,
no.
3,
pp.
9–21,
June
2014.
[28]
R.
S
and
A.
M.
Khan,
“Morphological
operati
ons
for
image
processing
:
Understanding
and
its
applications,
”
National
Confer
ence
on
VLSI,
Signal
pr
ocessing
Communications
,
pp.
17–19,
December
2013.
[29]
S.-T
.
W
u,
A.
C.
G.
d.
Silv
a,
and
M.
R.
G.
Marquez,
“The
Douglas-peuck
er
algorithm:
suf
ficienc
y
conditions
for
non-self-intersections,
”
J
ournal
of
the
Br
azilian
Computer
Society
,
v
ol.
9,
no.
3,
pp.
67
–
84,
2004.
[30]
J.
Zhang,
M.
Huang,
and
X.
L.
Xiaokang
Jin,
“
A
real-time
chinese
traf
fic
sign
detection
algorithm
based
on
modified
yolo
v2,
”
Algorithms
2017
,
v
ol.
10,
no.
4,
pp.
1–13,
2017.
[31]
C.
Y
.
W
ang
and
R.
C
.
Y
ue,
“T
raf
fic
sign
detection
using
you
only
look
once
frame
w
ork,
”
Stanford
Uni
v
ersity
,
T
ech.
Rep.,
2016.
[32]
J.
Deepika,
“Image
classification
using
con
v
olutional
neural
netw
orks,
”
International
J
ournal
of
Advancements
in
Resear
c
h
and
T
ec
hnolo
gy
,
v
ol.
6,
no.
3,
pp.
22–26,
2014.
[33]
Q.
Zhang,
M.
Zhang,
T
.
Chen,
Z.
Sun,
Y
.
Ma,
and
B.
Y
u,
“Recent
adv
ances
in
con
v
olutional
neural
netw
ork
acceleration,
”
CoRR
,
v
ol.
abs/1807.08596,
2018.
[Online].
A
v
ailable:
http://arxi
v
.or
g/abs/1807.08596
[34]
K.
Y
ue,
F
.
Xu,
and
J.
Y
u,
“Shallo
w
and
wide
fractional
max-pooling
netw
ork
for
image
classification,
”
Neur
al
Com-
puting
and
Applications
,
v
ol.
31,
no.
5,
p.
409–419,
2017.
[35]
D.
Y
u,
H.
W
ang,
P
.
Chen,
and
Z.
W
ei,
“Mix
ed
pooling
for
con
v
olutional
neural
netw
orks,
”
The
9th
International
Confer
ence
on
Rough
Sets
and
Knowledg
e
T
ec
hnolo
gy
(RSKT’14)
,
pp.
364–375,
2014.
T
r
af
fic
sign
detection
optimization
using
color
and...
(Handok
o)
Evaluation Warning : The document was created with Spire.PDF for Python.