Indonesian
J
our
nal
of
Electrical
Engineering
and
Computer
Science
V
ol.
36,
No.
3,
December
2024,
pp.
1640
1654
ISSN:
2502-4752,
DOI:
10.11591/ijeecs.v36.i3.pp1640-1654
r
1640
Fusion
algorithms
on
identifying
v
acant
parking
spots
using
vision-based
appr
oach
Ginanjar
Suwasono
Adi
1
,
Hertog
Nugr
oho
2
,
Griffani
Megiyanto
Rahmatullah
2
,
Muhammad
Y
usuf
F
adhlan
2
,
Dinan
Mutamaddin
2
1
Department
of
Electrical
Engineering,
Politeknik
Ne
geri
Malang,
Malang,
Indonesia
2
Department
of
Electrical
Engineering,
Politeknik
Ne
geri
Bandung,
Bandung,
Indonesia
Article
Inf
o
Article
history:
Recei
v
ed
Jul
3,
2024
Re
vised
Aug
7,
2024
Accepted
Aug
19,
2024
K
eyw
ords:
Computer
vision
Fusion
detection
method
Object
detection
Occupanc
y
state
algorithms
P
arking
management
system
Y
OLOv7
ABSTRA
CT
In
densely
populated
cities,
parking
space
scarcity
results
in
issues
lik
e
traf
fic
congestion
and
dif
ficulty
finding
parking
spots.
Recent
adv
ancements
in
computer
vision
ha
v
e
introduced
methods
to
address
parking
lot
manage-
ment
challenges.
The
a
v
ailability
of
publ
ic
image
datasets
and
rapid
gro
wth
in
deep
learning
technology
has
led
to
vision-based
parking
management
studies,
of
fering
adv
antages
o
v
er
sensor
-based
systems
in
comprehensi
v
e
area
co
v
erage,
cost
reduction,
and
additional
functionalities.
This
study
presents
an
inno
v
ati
v
e
fusion
algorithm
that
inte
grates
object
detection
with
occupanc
y
state
algorithms
to
accurately
identify
v
acant
parking
spaces.
The
emplo
ym
ent
of
the
Y
OLOv7
frame
w
ork
for
v
ehicl
e
instance
se
gmentation,
combined
with
three
occupanc
y
algorithms
Euclidean
distance
(ED),
intersection
o
v
er
reference
(IoR),
and
intersection
o
v
er
union
(IoU)
are
compared
to
determine
the
occupanc
y
state
of
observ
ed
areas.
The
proposed
method
is
e
v
aluated
using
the
CNRP
ark-EXT
dataset,
and
its
performance
is
compared
with
state-of-the-art
methods.
As
a
result,
the
proposed
approach
demonstrates
rob
ustness
under
v
arying
conditions.
It
outperforms
e
xisting
methods
in
terms
of
system
e
v
aluation
performance,
achie
ving
accuracies
of
98.88%,
97.99%,
and
90.04%
for
ED,
IoR,
and
IoU,
respecti
v
ely
.
This
fusion
detection
method
enhances
adaptability
and
addresses
occlusions,
emphasizing
Y
OLOv7’
s
adv
antages
and
accurate
shape
approxima-
tion
for
slot
annotation.
This
study
contrib
utes
v
aluable
insights
for
ef
fecti
v
e
parking
management
systems
and
has
potential
usage
in
the
real-w
orld
imple-
mentation
of
intelligent
transportation
systems.
This
is
an
open
access
article
under
the
CC
BY
-SA
license
.
Corresponding
A
uthor:
Hertog
Nugroho
Department
of
Electrical
Engineering,
Politeknik
Ne
geri
Bandung
Bandung
40012,
W
est
Ja
v
a,
Indonesia
Email:
hertog@polban.ac.id
1.
INTR
ODUCTION
The
scarcity
of
parking
spaces
in
highly
populated
cities
is
causing
dif
ficulties
for
people
to
park
their
v
ehicles,
leading
to
issues
lik
e
traf
fic
congestion,
incon
v
enienc
y
finding
suitable
parking
spots,
and
lim-
ited
accessibility
[1],
[2].
This
also
impacts
their
o
v
erall
v
ehicle
trip
e
xperience
[3].
Consequently
,
ef
fecti
v
e
parking
lot
management
necessitates
real-time
monitoring
of
v
ehicle
parking
occupanc
y
[4],
[5].
Being
a
w
are
of
a
v
ailable
spots
and
pro
viding
this
information
to
users
can
decrease
bottlenecks,
impro
v
e
scalability
,
and
quick
er
identification
of
v
acant
parking
areas.
Dif
ferent
approaches
ha
v
e
been
suggested
to
deal
with
the
issue
of
parking
management
in
urban
areas.
Solutions
using
wireless
sensor
netw
orks
(WSN)
are
well-re
g
arded
for
J
ournal
homepage:
http://ijeecs.iaescor
e
.com
Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian
J
Elec
Eng
&
Comp
Sci
ISSN:
2502-4752
r
1641
their
ease
of
implementation,
af
fordabili
ty
,
and
ability
to
w
ork
with
dif
ferent
sensors
that
can
monitor
park
ed
v
ehicles
[6].
Sensors
such
as
ultrasonic
[7],
[8]
and
inducti
v
e
loop
sensors
[7],
[9]–[12]
are
frequently
em-
plo
yed
in
the
WSN
research
for
smart
parking
management
systems
[4].
On
the
other
hand,
the
e
xtended
observ
ation
areas
and
the
enormous
num
b
e
r
of
parking
slots
might
create
real-w
orld
deplo
yment
challenges,
leading
to
high
maintenance
e
xpenses.
Additionally
,
in
WSN-based
systems,
the
algorithm
for
determining
the
state
of
a
parking
spot
is
relati
v
ely
straightforw
ard.
Each
sensor
is
being
assessed
using
thresholds
to
decide
the
occupanc
y
state.
Despite
this
simplicity
,
the
sensors’
limits
in
object
identification
and
lack
of
intelligence
de
grade
accurac
y
,
leading
to
potential
f
alse
detection.
In
recent
years,
v
arious
computer
vision
m
ethods
ha
v
e
been
proposed
to
tackle
challenges
assoc
iated
with
parking
lot
management
[5].
These
approaches
focus
on
analyzing
images
captured
from
parking
areas
and
address
v
arious
goals,
such
as
identifying
t
he
boundaries
of
each
parking
space
[13]–[19];
classifying
indi
vidual
parking
spaces
to
determine
if
the
y
are
occupied
or
unoccupied
[6],
[20]–[26];
and
counting
the
number
of
v
ehicles
in
images
[27]–[30].
Furthermore,
the
a
v
ailability
of
public
image
datasets
[23],
[31],
[32]
and
rapid
gro
wth
of
deep
learning
technology
[33]–[41]
ha
v
e
produced
se
v
eral
state-of-the-arts
(SoT
As)
in
vision-based
parking
management
studies
[6],
[23],
[25],
[26],
[31],
[32],
[42],
[43].
Computer
vision
techniques
ha
v
e
an
edge
o
v
er
indi
vidual
sensors
in
WSN-base
d
systems.
These
methods
can
co
v
er
e
xtensi
v
e
parking
areas
using
just
one
camera,
a
v
oiding
the
necessity
of
a
separate
sensor
per
spot.
This
minimizes
setup
and
maintenance
costs.
Furthermore,
cameras
can
aid
in
tasks
lik
e
surv
eillance
and
identifying
unusual
beha
vior
.
This
study
addresses
the
problem
of
recent
occupanc
y
detection
methods
in
parking
surv
eillance
s
ys-
tems.
Existing
methods
e
xhibit
limitations,
including
v
ague
slot
definitions,
high
computational
costs,
and
potential
bias
issues
[6],
[31],
[32],
[42].
K
e
et
al.
[6]
proposed
a
method
relying
on
a
ma
tching
algorithm
and
lea
ving
it
unclear
ho
w
parking
slots
are
defined.
Another
approach
emplo
ys
computationally
comple
x
detectors,
incurring
significant
e
xpenses
for
slot
annotation
[32].
Additionally
,
some
methods
proposed
by
Almeida
et
al.
[31]
e
xhibit
bias,
performing
well
only
under
specific
conditions.
Moreo
v
er
,
certain
classifi-
cation
approaches
utilize
patches
for
slot
assessment
[23],
[25],
[26],
[31],
[43],
introducing
problems
related
to
o
v
erlapping
annotations
and
limited
feature
information
for
distant
slots.
These
limitations
highlight
the
necessity
for
a
more
accurate
and
adaptable
approach
to
occupanc
y
detection
in
parking
surv
eillance
systems.
T
o
o
v
ercome
these
challenges,
we
de
v
eloped
a
fusion
detection
method
to
identify
the
occupanc
y
state
of
parking
slots
using
the
publicly
a
v
ailable
CNR-EXT
dataset
[23].
This
method
achie
v
es
a
substantial
breakthrough
in
system
performance
compared
to
state-of-the-art
(SoT
A)
methods.
W
e
contrib
ute
to
sho
wing
that
our
method
has
better
dealt
with
accurac
y
,
performance,
and
rob
ustness.
W
e
ha
v
e
de
vised
a
pipeline
that
combines
object
detection
through
computer
vision
techniques
with
an
occupanc
y
state
algorithm
to
enable
detection
under
e
xtreme
lighting
conditions
and
occlusions.
Furthermore,
our
comprehensi
v
e
e
xperimental
results
and
findings
serv
e
as
a
v
aluable
resource
for
future
research.
This
paper
is
or
g
anized
as
follo
ws.
In
section
2,
we
describe
details
of
the
complete
system
of
the
proposed
method.
Section
3
presents
the
e
xperiments
and
results
obtained
by
the
system.
This
section
presents
and
e
xplains
the
e
v
aluation
frame
w
ork
(fine-tuning
h
yperparameters)
used
to
achie
v
e
optimized
results
of
the
Y
OLOv7
best
model,
the
system
performance
of
occupanc
y
detection
methods,
and
the
positioning
of
this
research.
Finally
,
section
4
describes
the
conclusions
of
the
paper
and
the
possible
future
research
that
could
b
uild
upon
these
findings.
2.
METHOD
The
proposed
frame
w
ork’
s
o
v
ervie
w
is
presented
as
a
flo
w
diagram
in
Figure
1.
It
consists
of
tw
o
main
components:
(i)
data
preprocessing
and
training
and
(ii)
data
testing
and
occupanc
y
detection.
The
system’
s
architecture
is
tuned
to
balance
computing
ef
ficienc
y
,
reliability
,
and
scalability
.
Assuming
that
each
camera
position
is
fix
ed
to
monitor
the
desired
area
in
the
parking
lot,
we
be
gin
by
manually
annotating
and
labeling
each
parking
slot
to
obtain
re
gions
of
interest
(R
OIs)
for
indi
vidual
parking
spots
from
9
distinct
camera
positions,
which
were
acquired
from
the
CNRP
ark-EXT
dataset
[23].
Once
all
the
desired
slots
ha
v
e
been
annotated,
the
resul
ting
coordinate
v
ertices
for
these
slots
are
sa
v
ed
as
blobs
in
our
system
for
future
use.
As
part
of
the
v
ehicle
detection
method,
we
utilized
reno
wned
computer
vision
algorithms,
Y
OLOv7
[44]
due
to
their
optimal
accurac
y
and
ef
ficienc
y
for
object
detection
tasks.
T
o
impro
v
e
the
rob
ustness
and
Fusion
algorithms
on
identifying
vacant
parking
spots
using
vision-based
...
(Ginanjar
Suwasono
Adi)
Evaluation Warning : The document was created with Spire.PDF for Python.
1642
r
ISSN:
2502-4752
accurac
y
of
our
syste
m’
s
performance,
we
propose
a
combination
of
object
detection
and
occupanc
y
state
algo-
rithms
that
use
intersection
o
v
er
union
(IoU),
intersection
o
v
er
reference
(IoR),
and
Euclidean
distance
(ED),
which
will
be
e
xplained
in
the
follo
wing
subsection.
Figure
1.
Ov
ervie
w
of
the
end-to-end
pipeline
design
and
methodology
.
It
is
made
up
of
tw
o
primary
parts:
(i)
data
preprocessing
and
training
and
(ii)
data
testing
and
occupanc
y
detection
2.1.
Slot
initialization
Gi
v
en
an
image
of
a
parking
area,
each
parking
slot
is
manually
annotated
to
gi
v
e
a
close
approxi-
mation
of
its
true
form
based
on
four
coordinates
as
sho
wn
in
Figure
2;
for
k
=
f
1,
2,
3,
4
g
,
where
(
x
k
,
y
k
)
represent
the
horizontal
and
v
ertical
axis
coordinates
of
each
parking
spot,
respecti
v
ely
.
Then,
a
unique
ID
is
assigned
to
each
parking
space
based
on
the
i
th
ro
w
and
j
th
column
indices.
W
e
represent
each
parking
space
with
its
ID
as
a
list,
a
ij
2
N
;
i
=
1
;
2
;
:
:
:
;
I
j
m
=
1
;
2
;
:
:
:
;
J
;
m
=
1
;
2
;
:
:
:
;
M
(1)
(
x
k
;
y
k
)
!
a
:=
ff
x
1
;
y
1
g
;
f
x
2
;
y
2
g
;
f
x
3
;
y
3
g
;
f
x
4
;
y
4
g
;
f
a
ij
gg
(2)
where
the
components
of
list
a
ij
denote
the
location
of
parking
slot
in
ro
w
i
and
column
j
.
It
should
be
emphasized
that
the
v
alue
of
J
for
the
j
th
column
may
v
ary
across
dif
ferent
ro
ws,
which
is
indicated
by
M
.
As
a
result,
a
tuple
may
represent
the
complete
specified
slot
ID
along
with
its
coordinates
as
e
xpressed
in
(2).
Figure
3
sho
ws
the
comparison
between
a
ra
w
image
from
camera
eight
[23],
represented
in
Figure
3(a),
and
the
outcome
of
the
slot
initialization
step,
sho
wn
in
Figure
3(b).
In
the
latter
,
yello
w
lines
designate
each
parking
slot,
and
white
te
xt
indicates
the
ID
of
each
slot.
The
resulting
tuple
of
a
is
then
sa
v
ed
in
our
system
for
future
procedures
once
the
appropriate
slots
ha
v
e
been
defined.
Figure
2.
Annotation
process
to
obtain
slot
coordinates
Indonesian
J
Elec
Eng
&
Comp
Sci,
V
ol.
36,
No.
3,
December
2024:
1640–1654
Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian
J
Elec
Eng
&
Comp
Sci
ISSN:
2502-4752
r
1643
(a)
(b)
Figure
3.
The
comparison
(a)
ra
w
image
from
camera
eight
[23]
and
(b)
slot
initialization
based
on
the
same
camera
vie
w
2.2.
T
ransfer
lear
ning
f
or
car
detection
2.2.1.
Dataset
pr
eparation
CNRP
ark-EXT
dataset
used
in
this
research
consists
of
810
parking
lot
images
g
athered
from
9
cam-
eras
with
dif
ferent
vie
wpoints
and
v
arious
en
vironmental
conditions:
sunn
y
,
o
v
ercast,
and
rain
y
.
This
dataset
contains
information
on
the
binary
classification
of
2
labeled
classes:
class
1:
occupied
slot,
which
has
a
car
image;
class
2:
v
acant
slot,
which
is
the
annotated
image
of
an
empty
slot.
Ho
we
v
er
,
since
we
w
ant
to
increase
the
rob
ustness
and
accurac
y
in
our
system
instead
of
using
binary
classific
ation
as
in
[23],
we
used
instance
se
gmentation
to
detect
v
ehicles
in
the
observ
ed
image
and
then
e
v
aluate
their
position
in
the
slot
area.
Thus,
we
need
to
re-annotate
our
dataset
for
data
preparation.
Roboflo
w
platform
w
as
emplo
yed
to
resize
the
input
image
into
640
640
pix
els
so
that
this
input
will
be
compatible
with
the
Y
OLOv7
detector
.
Additionally
,
we
annotated
v
ehicles
indi
vidually
as
polygon
shapes
for
se
gmentation
purposes.
T
o
v
ary
our
dataset,
we
used
augmentation
tools
in
the
Roboflo
w
platform
by
rotating
our
train
set
randomly
between
15
clockwise
and
counter
-clockwise
so
that
we
g
athered
1,243
image
frames
with
a
total
of
16,653
car
instances
from
the
CNRP
ark-EXT
dat
aset.
Then,
we
randomly
di
vided
all
datasets
from
1,243
images
with
the
portion
of
70%,
20%,
and
10%
for
the
train,
v
alidation,
and
test
sets,
respecti
v
ely
.
2.2.2.
T
ransfer
lear
ning
Y
OLOv7
In
this
research,
we
emplo
y
the
Y
OLOv7
architecture
[44]
to
perform
instance
se
gmentation
for
v
ehi
-
cle
detection.
Y
OLOv7
stands
out
in
v
ehicle
se
gmentation
due
to
its
inno
v
ati
v
e
enhancements,
as
highlighted
in
se
v
eral
studies.
The
Y
OLOv7
algorithm
introduces
impro
v
ements
such
as
lightweight
models
with
high
de-
tection
accurac
y
,
lo
w
comple
xity
,
and
reduced
parameter
count
[45].
Addi
tionally
,
the
Y
OLOv7-based
method
[46]
also
e
xcels
in
instance
se
gmentation,
resulting
in
superior
se
gmentation
accurac
y
and
f
aster
inference
speeds
compared
to
other
popular
algorithms
li
k
e
mask
re
gion-based
con
v
olutional
neural
netw
ork
(R-CNN)
[37]
and
Y
OLA
CT
[41].
Moreo
v
er
,
the
Y
OLOv7-based
v
ehicle
detection
method
addresses
challenges
such
as
lo
w
accurac
y
in
detecting
small
and
occluded
tar
gets
by
utilizing
a
p
yramid
pooling
structure
and
paral-
lel
channel-spatial
attention
modules,
achie
ving
high
a
v
erage
precision
and
processing
speed
[47].
Therefore,
these
collecti
v
e
adv
ancements
highlight
Y
OLOv7’
s
superiority
in
v
ehicle
se
gmentation
o
v
er
other
models,
making
it
suitable
for
this
study
.
After
data
preparation
w
as
completed,
we
processed
the
training
set
for
transfer
learning
to
achie
v
e
the
best
Y
OLOv7
model.
W
e
then
v
alidated
the
model
using
the
v
alidation
set
to
e
xamine
the
performance
of
our
inference
model.
Finally
,
the
best
weights
from
the
e
v
aluation
process
were
applied
to
our
system
for
future
use.
2.3.
Occupancy
state
algorithms
W
e
compared
three
occupanc
y
state
algorithms:
IoU,
IoR,
and
ED.
In
IoU
w
as
implemented
to
es-
timate
the
state
of
each
slot.
The
k
e
y
to
this
algorithm
is
to
compare
the
intersection
area
of
a
specific
slot
R
O
I
S
ij
and
the
area
of
a
selected
car
R
O
I
C
n
.
Since
the
shape
of
e
v
ery
parking
space
might
v
ary
,
taking
on
the
form
of
a
polygon,
we
can
measure
the
R
OIs
of
each
parking
space
by
using
the
quadrilateral
equation
as,
R
O
I
S
ij
=
(
x
1
y
2
y
1
x
2
)
+
+
(
x
4
y
1
y
4
x
1
)
2
(3)
Fusion
algorithms
on
identifying
vacant
parking
spots
using
vision-based
...
(Ginanjar
Suwasono
Adi)
Evaluation Warning : The document was created with Spire.PDF for Python.
1644
r
ISSN:
2502-4752
where
R
O
I
S
ij
is
a
list
containing
all
the
calculated
areas
of
each
parking
space.
Whereas
R
O
I
C
n
is
a
list
of
e
v
ery
area
of
detected
cars
that
can
be
obtained
from
Y
OLOv7
instance
se
gmentation
inference.
Hence,
the
IoU
can
be
computed
as,
I
oU
ij
=
I
;J
m
max
i
=1
;j
=1
R
O
I
S
ij
\
R
O
I
C
n
R
O
I
S
ij
[
R
O
I
C
n
(4)
The
output
of
the
slot
state
after
performing
the
IoU
algorithm
can
be
e
xpressed
as,
P
r
ed
(
a
ij
)
=
occupied;
I
oU
ij
T
h
v
acant;
other
w
ise
(5)
where
T
h
is
the
ratio
of
(
R
O
I
S
ij
[
R
O
I
C
n
).
Ho
we
v
er
,
since
this
algorithm
depends
on
the
threshold
v
alue
corresponding
to
the
percentage
of
o
v
erlap
between
the
e
v
aluated
R
O
I
S
ij
and
R
O
I
C
n
,
the
IoU
algorithm
may
f
ail
to
detect
occupied
slots
that
are
f
ar
a
w
ay
from
the
camera
when
higher
threshold
v
alues
are
set.
Therefore,
we
propose
tw
o
other
algorithms,
ED
and
IoR,
to
minimize
the
error
of
f
alse
detection
in
occupanc
y
state
detection.
The
details
of
these
algorithms
are
described
in
the
follo
wing
subsection.
2.3.1.
Intersection
o
v
er
r
efer
ence
In
Algorithm
1,
we
proposed
the
IoR.
This
algorithm
aims
to
maximize
the
occupanc
y
area
of
a
detected
car
onto
an
indi
vidual
slot
by
modifying
the
di
visor
of
IoU
as
a
f
actor
of
reference
area
R
O
I
S
ij
.
By
e
v
oking
(3),
The
IoR
formula
can
be
e
xpressed
as,
I
oR
ij
=
I
;J
m
max
i
=1
;j
=1
R
O
I
S
ij
\
R
O
I
C
n
R
O
I
S
ij
(6)
Algorithm
1.
IoR
to
determine
slot
state
Input:
parking
slot
image,
slot
coordinate,
Y
OLOv7
best
model
Output:
Pr
ed
(
a
ij
)
Initialisation:
compute
Y
OLOv7
instance
se
gmentation
compute
R
OIS
ij
,
R
OIC
n
for
all
slot
in
ro
w
do
set
T
h
compute
IoR
sort
max
IoR
if
max
IoR
T
h
then
set
Pr
ed
(
a
ij
)
occupied
else
set
Pr
ed
(
a
ij
)
v
acant
end
if
update
Pr
ed
(
a
ij
)
end
for
return
Pr
ed
(
a
ij
)
P
r
ed
(
a
ij
)
=
occupied;
I
oR
ij
T
h
v
acant;
ot
h
er
w
ise
(7)
In
(7)
can
be
applied
to
predict
the
slot
state
of
this
algorithm,
where
the
threshold
v
alue
T
h
can
be
set
as
the
proportion
of
R
O
I
S
ij
.
Additionally
,
we
also
apply
a
sorting
mechanism
to
pre
v
ent
redundanc
y
of
a
certain
v
ehicle
occup
ying
more
than
one
slot
using
argmax
function
for
IoU
and
IoR
algorithms
in
this
study
.
2.3.2.
Euclidean
distance
Algorithm
2
sho
ws
the
implementation
of
ED
to
predict
slot
state
in
an
observ
ed
image.
T
o
perform
the
ED
algorithm,
the
centroid
of
e
v
ery
indi
vidual
slot
will
be
assessed
for
all
detected
centroid
cars.
Recalling
(2),
where
each
element
represents
the
coordinates
of
e
v
ery
distincti
v
e
slot,
the
center
of
gra
vity
of
each
slot
can
be
calculated
as,
C
ij
=
x
1
+
x
2
+
x
3
+
x
4
4
;
y
1
+
y
2
+
y
3
+
y
4
4
(8)
Indonesian
J
Elec
Eng
&
Comp
Sci,
V
ol.
36,
No.
3,
December
2024:
1640–1654
Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian
J
Elec
Eng
&
Comp
Sci
ISSN:
2502-4752
r
1645
Algorithm
2.
ED
to
determine
slot
state
Input:
parking
slot
image,
slot
coordinate,
Y
OLOv7
best
model
Output:
Pr
ed
(
a
ij
)
Initialisation:
compute
Y
OLOv7
instance
se
gmentation
compute
C
ij
,
C
n
for
all
slot
in
ro
w
do
set
T
h
compute
ED
sort
min
ED
if
min
ED
T
h
then
set
Pr
ed
(
a
ij
)
occupied
else
set
Pr
ed
(
a
ij
)
v
acant
end
if
update
Pr
ed
(
a
ij
)
end
for
return
Pr
ed
(
a
ij
)
Then,
the
firs
t-order
image
moments
are
calculated
to
pro
vide
the
centroid
of
each
v
ehicle
detec
ted
in
the
frame
after
running
Y
OLOv7
inference.
This
requires
adapting
a
gre
yscale
image
with
pix
el
intensities
represented
as
I
(
x;
y
)
,
which
allo
ws
for
the
computation
of
image
moments
M
v
w
as,
M
v
w
=
X
x
X
y
x
v
y
w
I
(
x;
y
)
(9)
the
sum
of
all
pix
el
intensities
in
the
image
of
the
gre
y
le
v
el
is
defined
as
the
zeroth-order
moment
M
00
.
The
first-order
moments,
M
10
and
M
01
,
pro
vide
information
about
the
spatial
distrib
ution
of
the
pix
el
intensities
along
the
x
and
y
ax
es,
respect
i
v
ely
.
These
moments
are
used
to
calculate
the
centroid
of
the
instance
car
in
the
directions
x
and
y
.
Thus,
the
centroid
of
all
detected
cars
can
be
calculated
as
follo
ws
(10).
C
n
(
x;
y
)
=
M
10
M
00
;
M
01
M
00
(10)
After
obtaining
C
ij
and
C
n
we
perform
Euclidean
distance
for
all
centroid
slots
to
e
v
ery
centroid
of
detected
cars
by
the
follo
wing
equation,
d
(
p;
q
)
=
I
X
i
=1
J
X
j
=1
N
X
n
=1
(
q
i;j
p
n
)
2
!
1
=
2
(11)
where
p
2
C
n
centroid
of
e
v
ery
car
and
q
2
C
ij
centroid
of
each
indi
vidual
slot.
Let
f
T
h
2
min
f
diag
(
a
ij
)
gg
be
the
shortest
diagonal
for
all
indi
vidual
slots
of
slot
j
in
ro
w
i
,
where
the
components
of
v
ector
T
h
ij
represent
the
v
alue
of
thresholds
for
e
v
ery
slot
j
in
ro
w
i
.
Thus,
the
output
of
the
slot
state
whether
the
parking
slot
is
v
acant
or
occupied
can
be
represented
as,
P
r
ed
(
a
ij
)
=
occupied;
min
f
d
(
p;
q
)
g
T
h
v
acant;
other
w
ise
(12)
Note
that
to
a
v
oid
redundanc
y
of
a
particular
slot
occupied
by
more
than
one
v
ehicle,
we
implement
a
simple
sort
ing
procedure
that
will
e
v
al
u
a
te
those
s
pecific
slots
only
with
t
he
closest
v
ehicle.
T
o
ai
d
comprehen-
sion
of
these
algorithms’
functionality
,
Figure
4
of
fers
a
visual
representation
of
t
he
proposed
methods.
The
top
ro
w
of
Figure
4(a)
corresponds
to
the
system
input,
which
consists
of
the
input
image,
Y
OLOv7
inference,
and
the
slot
area
R
O
I
S
ij
.
The
second
ro
w
Figure
4(b)
depicts
the
IoU
algorithm,
where
it
sho
ws
a
con
v
entional
method
for
e
v
aluating
the
occupanc
y
st
ate
of
a
specific
slot
by
sho
wing
the
o
v
erlap
between
the
intersection
of
(
R
O
I
S
ij
\
R
O
I
C
n
)
and
the
union
of
(
R
O
I
S
ij
[
R
O
I
C
n
).
F
ollo
wing
this,
the
third
and
fourth
ro
ws
illus-
trate
our
proposed
algorithms:
Figure
4(c),
the
IoR
algorithm,
which
focuses
on
the
proportion
of
(
R
O
I
S
ij
\
R
O
I
C
n
)
to
the
desired
R
O
I
S
ij
,
and
Figure
4(d),
the
ED
algorithm,
which
measures
the
spatial
accurac
y
of
the
detected
v
ehicle
to
the
designated
slot
by
calculating
the
Euclidean
distance
between
the
centroid
of
the
slot
Fusion
algorithms
on
identifying
vacant
parking
spots
using
vision-based
...
(Ginanjar
Suwasono
Adi)
Evaluation Warning : The document was created with Spire.PDF for Python.
1646
r
ISSN:
2502-4752
C
ij
and
the
centroid
of
the
car
C
n
.
T
ogether
,
these
visual
elements
in
Figure
4
pro
vide
a
comprehensi
v
e
guide
to
understanding
the
input
system
and
the
progression
from
the
IoU
algorithm
t
o
the
proposed
IoR
and
ED
methods,
highlighting
their
dis
tinct
approaches
and
contrib
utions
to
fusion
algorithms
for
identifying
v
acant
parking
spots.
(a)
(b)
(c)
(d)
Figure
4.
V
isual
illustration
of
the
proposed
methods.
T
op
ro
w:
(a)
depicts
the
input
system,
which
consists
of
the
input
image,
Y
OLOv7
inference,
and
the
R
OI
slot;
(b)
represents
the
IoU
algorithm;
(c)
illustrates
the
IoR
algorithm;
and
(d)
sho
ws
the
ED
algorithm
3.
RESUL
TS
AND
DISCUSSIONS
3.1.
Y
OLOv7
perf
ormance
In
this
study
,
we
utilized
transfer
learning
instead
of
training
our
deep
learning
model
from
scratch.
This
approac
h
allo
wed
us
to
achi
e
v
e
impro
v
ed
model
performance
by
le
v
eraging
our
data
dist
rib
ution
while
reducing
the
computational
cost.
In
Figure
5,
the
comparison
between
Y
OLOv7
performance
with
def
ault
h
yperparameters
and
optimized
h
yperparameters
during
the
training
phase
is
portrayed.
W
e
emplo
yed
the
def
ault
h
yperparameters
setup
on
Y
OLOv7
for
the
initial
training
process
using
50
epochs
to
our
dataset.
Figures
5(a)
and
5(b)
portrays
the
loss
performance
of
the
training
v
ersus
v
ali
d
a
tion
process
and
the
result
of
model
performance
for
def
aul
t
h
yperparameters
Y
OLOv7,
respecti
v
ely
.
Based
on
the
graph
in
Figure
5(a)
we
can
analyze
where
at
about
17
th
epoch,
the
v
alidation
loss
starts
to
appear
saturated,
while
the
training
loss
sho
ws
a
slight
do
wntrend
until
the
end
of
the
learning
process.
Ev
en
though
the
model
performance
of
this
setup
produced
good
marks
on
the
mean
a
v
erage
precision
(mAP@50)
of
97.5%,
precision
of
96.9%,
and
recall
of
94.9%,
this
model
implies
o
v
erfitting
to
our
data
distrib
ution.
Indonesian
J
Elec
Eng
&
Comp
Sci,
V
ol.
36,
No.
3,
December
2024:
1640–1654
Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian
J
Elec
Eng
&
Comp
Sci
ISSN:
2502-4752
r
1647
Gi
v
en
this
circumstance,
we
tried
to
optimize
the
h
yperparameters
and
the
learning
process
of
Y
OLOv7
to
a
v
oid
o
v
erfitting
conditions
that
may
cause
the
model
to
only
remember
our
data
train
distrib
ution
instead
of
ha
ving
the
ability
to
accomplish
generalization.
W
e
disco
v
ered
that
the
best
setup
for
Y
OLOv7
h
yperpa-
rameters
to
our
data
distrib
ution,
as
stated
in
T
able
1
is
by
using
500
epochs,
early
stopping
of
20
patience,
and
applying
the
stochastic
gradient
descent
(SGD)
optimizer
.
It
is
essential
to
highlight
that
the
unspecified
h
yperparameters
listed
in
T
able
1
are
determined
using
the
def
ault
configuration.
An
early
stopping
mechanism
is
intended
to
discontinue
the
learning
process
if
there
is
no
impro
v
ement
in
the
training
loss
of
as
man
y
as
20
epochs.
Hence,
we
do
not
need
to
w
ait
until
the
learning
process
of
500
epochs
is
finished.
Y
OLOv7
allo
ws
us
to
use
adapti
v
e
gradient
algorithms
(AD
AM)
and
SGD
optimizers.
AD
AM
can
adjust
the
learning
rate
for
each
parameter
during
training.
It
can
progress
f
aster
in
the
training
phase
b
ut
may
not
al
w
ays
result
in
the
best
gen-
eralization
performance.
The
geometry
adaptation
in
AD
AM,
through
adapti
v
ely
scaling
gradient
coordinates,
reduces
anisotropic
gradient
noise
and
increases
the
Radon
measure
of
a
basin.
As
a
result,
AD
AM
tends
to
tak
e
longer
to
escape
sharp
minima
with
small
Radon
measures.
In
contrast,
SGD
uses
a
fix
ed
learning
rate
for
all
parameters
that
leads
to
impro
v
ed
model
perfor
-
mance
slo
wly
b
ut
could
achie
v
e
higher
test
performance.
The
SGD’
s
local
ins
tability
allo
ws
it
to
better
na
vi-
g
ate
to
w
ards
flatter
minima
with
more
considerable
Radon
measure.
Flatter
minima,
commonly
associ
ated
with
generalized
solutions,
e
xplain
wh
y
AD
AM
usually
suf
fers
from
w
orse
generalization
performance
than
SGD
[48]–[51].
W
ith
the
rapid
adv
ancement
of
parallel
computing
technologies,
such
as
graphics
processing
units
(GPUs)
and
tensor
processing
units
(TPUs),
the
time
and
computational
cost
of
training
deep
learning
models
are
no
longer
significant
obstacles.
As
a
result,
obtaining
the
parameters
required
for
training
data
has
become
significantly
f
aster
and
more
af
fordable.
This
a
v
ailability
has
also
bee
n
supported
by
man
y
cloud
platforms
so
that
man
y
users
can
easily
design
and
deplo
y
their
deep
learning
models
in
the
cloud
serv
er
without
necessarily
o
wning
a
local
serv
er
.
All
the
computations
for
the
transfer
learning
stage
of
this
research
were
performed
using
Google
Colab
and
the
Nvidia
R
T
esla
T4
GPU
accelerator
.
T
able
1.
Optimized
Y
OLOv7
h
yperparameters
Hyperparameters
Description
V
alue
Def
ault
Optimized
lr0
Initial
learning
rate
(lr)
at
the
be
ginning
of
training
process
0.01
0.001
lrf
Final
lr
(during
training,
lr
is
gradually
decreased
till
reaching
final
v
alue)
0.01
10e-6
momentum
Determining
contrib
ution
of
the
pre
vious
gradient
update
to
the
current
update.
As
the
model
learns
to
recognize
cars,
momentum
helps
accelerate
con
v
er
gence
and
pre
v
ents
the
model
from
getting
stuck
in
suboptimal
local
minima.
(a
higher
momentum
v
alue
leading
to
more
stable
and
consistent
updates)
0.937
0.9
w
armup
epoch
Number
of
epochs
for
which
the
learning
rate
is
gradually
increased
from
0
to
lr0.
3.0
1.0
mixup
The
probability
of
applying
the
mixing
images
for
each
training
batch
augmentation
technique
during
training.
The
mixup
coef
ficient
combines
tw
o
images
into
a
single
image
to
pro
vide
ne
w
training
e
xamples.
It
helps
to
pre
v
ent
o
v
erfitting
and
can
lead
to
impro
v
ed
performance
while
enhancing
the
model
to
learn
more
rob
ust
and
generalized
features
by
training
on
di
v
erse
combinations
of
images.
0.0
0.5
As
depicted
in
Figure
5(c),
the
results
of
tuning
h
yperparameters
for
Y
OLOv7
sho
w
a
noticeable
trend
in
the
v
alidation
loss,
which
closely
follo
ws
the
training
loss.
The
v
alidat
ion
loss
is
lo
wer
than
the
training
loss
because
of
Y
OLOv7’
s
tendenc
y
to
learn
more
ef
fecti
v
ely
during
the
traini
ng
process.
This
discrepanc
y
can
be
attrib
uted
to
the
training
set’
s
data
distrib
ution
v
arying
more
significantly
than
the
v
alidation
set,
o
wing
to
the
data
augmentation
process.
Furthermore,
optimizing
h
yperpar
ameters,
particularly
selecting
a
lo
wer
final
learning
rate
(lrf),
has
enabled
the
SGD
optimizer
to
enhance
the
model’
s
performance,
resulting
in
smoother
and
more
generalized
outcomes.
The
performance
of
the
optimized
Y
OLOv7
h
yperparameters,
as
illustrated
in
Figure
5(d),
yields
comparable
results
to
the
def
ault
setup
with
the
mAP@50
of
97.4%,
precision
of
98%,
and
recall
of
93.9%.
Ho
we
v
er
,
it
is
w
orth
noting
that
the
def
ault
model
might
still
e
xhibit
some
o
v
erfitting
to
our
specific
data
distrib
ution.
Despite
the
slight
dif
ferences
observ
ed
in
mAP@50
and
recall
between
the
optimized
and
def
ault
setups,
the
optimi
zed
performance
e
xcels
in
precision,
ensuring
greater
consistenc
y
in
inferencing.
This
emphasizes
the
significance
of
fine-tuning
h
yperparameters
to
achie
v
e
more
rob
ust
and
reliable
results.
Fusion
algorithms
on
identifying
vacant
parking
spots
using
vision-based
...
(Ginanjar
Suwasono
Adi)
Evaluation Warning : The document was created with Spire.PDF for Python.
1648
r
ISSN:
2502-4752
(a)
(b)
(c)
(d)
Figure
5.
Performance
metrics
comparison
of:
(a)
loss
v
alue
in
def
ault
Y
OLOv7
h
yperparameters;
(b)
loss
v
alue
in
optimized
Y
OLOv7
h
yperparameters;
(c)
model
performance
with
def
ault
Y
OLOv7
h
yperparameters;
and
(d)
model
performance
with
optimized
Y
OLOv7
h
yperparameters
3.2.
Ev
aluation
perf
ormance
on
occupancy
state
algorithms
T
o
e
v
aluate
our
proposed
algorithms,
we
split
10%
of
a
total
dataset
based
on
our
data
preparation
stage.
This
results
in
125
unique
images
from
CNRP
ark-EXT
of
9
dif
ferent
vie
w
cameras
in
3
conditions
(sunn
y
,
o
v
ercast,
and
rain
y).
Ev
aluation
performance
metrics
of
accurac
y
(13),
precision
(14),
recall
(15),
and
F1-score
(16)
were
applied
to
predict
ho
w
our
proposed
method
performed
in
this
study
based
on
the
ground
truth
label
from
CNRP
ark-EXT
using
the
confusion
matrix
gi
v
en
in
T
able
2.
Here,
TP
indicates
true
positi
v
e,
TN
refers
to
true
ne
g
ati
v
e,
while
FP
and
FN
represent
f
alse
positi
v
e
and
f
alse
ne
g
ati
v
e,
respecti
v
ely
.
Accur
acy
=
T
P
+
T
N
T
P
+
T
N
+
F
P
+
F
N
(13)
P
r
ecision
=
T
P
T
P
+
F
P
(14)
R
ecal
l
=
T
P
T
P
+
F
N
(15)
F
1
scor
e
=
2
P
r
ecision
R
ecal
l
P
r
ecision
+
R
ecal
l
(16)
T
able
2.
Confusion
matrix
of
occupanc
y
detection
algorithms
Ground
truth
Prediction
Ev
aluation
Occupied
Occupied
TP
Empty
Empty
TN
Empty
Occupied
FP
Occupied
Empty
FN
In
this
study
,
it
is
essential
to
note
that
we
emplo
yed
v
arious
threshold
v
alues,
denoted
as
f
T
h
2
f
0
:
25
;
0
:
5
;
0
:
75
;
1
gg
,
for
each
of
the
proposed
algorithms
in
order
to
determine
the
optimal
solution
for
each
Indonesian
J
Elec
Eng
&
Comp
Sci,
V
ol.
36,
No.
3,
December
2024:
1640–1654
Evaluation Warning : The document was created with Spire.PDF for Python.
Indonesian
J
Elec
Eng
&
Comp
Sci
ISSN:
2502-4752
r
1649
method.
In
the
case
of
the
ED
algorithm,
as
outlined
in
(12),
the
threshold
v
alue
is
defined
as
the
proportion
of
the
shortest
diagonal
distance
for
each
slot.
Con
v
ersely
,
for
the
IoU
algorithm,
as
described
in
(5),
the
threshold
v
alue
corresponds
to
the
percentage
of
o
v
erlap
between
the
e
v
aluated
R
O
I
S
ij
and
R
O
I
C
n
.
Lastly
,
the
IoR
algorithm
determines
the
threshold
v
alue
as
the
ratio
of
the
e
v
aluated
R
O
I
S
ij
,
as
indicated
in
(7).
As
a
result,
Figure
6
illustrates
the
e
v
aluation
performance
of
this
study’
s
proposed
methods.
Generally
,
the
ED
algorithm
demonstrates
an
upw
ard
trend
as
its
threshold
v
alue
increases
with
respect
to
accurac
y
,
recall,
and
F1-score
metrics.
In
contrast,
the
IoU
and
the
IoR
algorithms
e
xhibit
higher
scores
for
these
metrics
at
lo
wer
threshold
v
alues,
and
their
performance
tends
to
decline
as
the
threshold
v
alue
increases.
Specifically
,
Figure
6(a)
illustrates
that
the
ED
algorithm
attains
its
highest
accurac
y
of
98.88%
at
a
threshold
v
alue
of
0.75.
Figure
6(b)
demonstrates
that
the
ED
algorithm
achie
v
es
its
optimal
precision
of
99.57%
at
a
threshold
of
0.5.
Furthermore,
Figure
6(c)
sho
ws
that
the
ED
algorithm
reaches
its
highest
recall
of
99.19%
at
a
threshold
of
1.
Figure
6(d)
highlights
that
the
ED
algorithm
achie
v
es
the
highest
F1-score
of
98.99%
at
a
threshold
v
alue
of
0.75.
The
IoU
algorithm’
s
peak
performance
is
observ
ed
at
a
threshold
v
alue
of
0.25,
achie
ving
an
accurac
y
of
90.04%,
as
depicted
in
Figure
6(a),
a
recall
of
82.17%,
as
sho
wn
in
Figure
6(c),
and
an
F1-score
of
89.92%,
as
indicated
in
Figure
6(d).
Additionally
,
Figure
6(b)
re
v
eals
that
the
IoU
algorithm
achie
v
es
a
precision
of
99.77%
at
a
threshold
v
alue
of
1.
Lastly
,
there
are
intere
sting
outcomes
for
the
IoR
algorithm.
At
a
threshold
v
alue
of
1,
the
results
sho
w
0%
in
precision
sho
wn
in
Figure
6(b),
recall
displayed
in
Figure
6(c),
and
F1-score
represented
in
Figure
6(d)
because
there
are
no
true
positi
v
e
detections.
According
to
(6),
this
threshold
requires
the
(
R
O
I
S
ij
\
R
O
I
C
n
)
to
perfectly
o
v
erlap
100%
with
the
R
O
I
S
ij
.
The
0%
precision
and
the
0%
recall
signifies
that
no
actual
occupied
slots
were
correctl
y
detected.
Consequently
,
the
F1-score
is
also
0%,
confirming
the
algorithm’
s
f
ailure
to
ide
n
t
ify
an
y
true
positi
v
es
at
this
threshold.
This
strict
requirement
for
perfect
o
v
erlap
is
lik
ely
impractical,
and
adjus
ting
the
threshold
to
allo
w
for
less
than
1
could
impro
v
e
the
IoR
algorithm’
s
performance.
the
IoR
algorithm
demonstrates
its
best
performance
at
a
threshold
v
alue
of
0.25,
achie
ving
an
accurac
y
of
97.99%,
as
depicted
in
Figure
6(a),
a
precision
of
99.64%,
as
sho
wn
in
Figure
6(b),
a
recall
of
96.74%,
as
illustrated
in
Figure
6(c),
and
an
F1-score
of
98.15%,
a
s
indicated
in
Figure
6(d).
The
sample
results
of
occupanc
y
detection
from
proposed
methods
using
best
threshold
parameters
are
depicted
in
Figure
7.
The
re
gions
in
red
are
indicated
as
occupied
slots,
while
the
green
areas
are
specified
as
v
acant
slots
and
the
blue
areas
represent
detected
v
ehicles
through
instance
se
gmentation
using
the
Y
OLOv7.
(a)
(b)
(c)
(d)
Figure
6.
The
comparison
of
performance
analysis
for
proposed
methods
using
dif
ferent
metric:
(a)
accurac
y
,
(b)
precision,
(c)
recall,
and
(d)
F1-score
Fusion
algorithms
on
identifying
vacant
parking
spots
using
vision-based
...
(Ginanjar
Suwasono
Adi)
Evaluation Warning : The document was created with Spire.PDF for Python.