Indonesi
an
Journa
l
of El
ect
ri
cal Engineer
ing
an
d
Comp
ut
er
Scie
nce
Vo
l.
23
,
No.
3
,
Septem
ber
20
21
,
pp.
135
0
~
135
6
IS
S
N: 25
02
-
4752, DO
I: 10
.11
591/ijeecs
.v
23
.i
3
.
pp
1350
-
135
6
1350
Journ
al h
om
e
page
:
http:
//
ij
eecs.i
aesc
or
e.c
om
Advanced
UI
t
est aut
omation (A
UT
A) fo
r BIOS
valid
atio
n
usin
g OpenCV a
nd
OCR
Ei
ssa
Abdull
ah Aw
ad
h
Moh
ammed
1
, Mus
li
m Mu
s
tapa
2
,
Hasl
iz
a
Rahim
3
, Mohd
Nat
as
h
ah
N
oriz
an
4
1,2,3,4
Facul
t
y
of
E
le
c
troni
c
Eng
ineeri
ng
Technol
og
y
,
Univer
si
ti Ma
lay
s
ia Perl
is
(UniMA
P),
02600
Arau,
Pe
rli
s,
Mal
a
y
sia
2
Advanc
ed
Com
puti
ng,
Cent
r
e
of
Excel
l
ence
(Ad
vCom
p),
Univer
siti
Mal
a
y
sia
Per
li
s (UniMA
P)
3
Advanc
ed
Com
m
unic
at
ion
Enginee
ring
,
C
ent
re
of
Excel
l
ence
(A
CE),
U
niv
ersit
i
Malay
s
ia Perl
is
(
UniMA
P)
4
Cent
re
of Excel
le
nc
e
Geopo
l
y
m
er
and
Gree
n
T
e
chnol
og
y
(C
eGe
oGTe
ch)
,
Unive
rsiti
Ma
lay
sia
Pe
rli
s (UniMA
P)
Art
ic
le
In
f
o
ABSTR
A
CT
Art
ic
le
history:
Re
cei
ved
Feb
18, 202
1
Re
vised
A
pr 14, 2
021
Accepte
d
Apr 20,
2021
B
a
s
i
c
i
n
p
u
t
o
u
t
p
u
t
s
y
s
t
e
m
(
B
I
O
S
)
v
a
l
i
d
a
t
i
o
n
i
s
p
e
r
f
o
r
m
e
d
o
n
b
o
t
h
g
r
a
p
h
i
c
a
l
u
s
e
r
i
n
t
e
r
f
a
c
e
(
G
U
I
)
a
n
d
c
o
m
m
a
n
d
-
l
i
n
e
i
n
t
e
r
f
a
c
e
(
C
L
I
)
b
y
a
t
e
s
t
e
n
g
i
n
e
e
r
.
K
e
y
b
o
a
r
d
a
n
d
m
o
u
s
e
a
r
e
u
s
e
d
t
o
i
n
s
e
r
t
t
e
s
t
c
a
s
e
s
c
o
m
m
a
n
d
s
i
n
t
o
s
y
s
t
e
m
u
n
d
e
r
t
e
s
t
(
S
U
T
)
.
T
e
s
t
e
n
g
i
n
e
e
r
m
o
n
i
t
o
r
s
t
e
s
t
c
a
s
e
s
p
r
o
g
r
e
s
s
o
n
a
m
o
n
i
t
o
r
f
o
r
v
a
l
i
d
a
t
i
o
n
.
T
h
i
s
m
e
t
h
o
d
i
s
t
i
m
e
-
c
o
n
s
u
m
i
n
g
a
n
d
r
e
l
a
t
i
v
e
l
y
m
o
r
e
e
x
p
e
n
s
i
v
e
t
h
a
n
a
u
t
o
m
a
t
i
o
n
.
I
n
t
h
i
s
p
r
o
j
e
c
t
w
e
d
e
s
i
g
n
e
d
a
n
i
n
d
e
p
e
n
d
e
n
t
a
u
t
o
m
a
t
i
o
n
s
y
s
t
e
m
t
h
a
t
a
b
l
e
t
o
m
i
m
i
c
h
u
m
a
n
i
n
t
e
r
a
c
t
i
o
n
i
n
B
I
O
S
v
a
l
i
d
a
t
i
o
n
.
T
h
e
a
p
p
r
o
a
c
h
c
a
n
b
e
d
i
v
i
d
e
d
i
n
t
o
t
w
o
m
a
i
n
p
a
r
t
s
.
T
h
e
f
i
r
s
t
p
a
r
t
i
s
t
h
e
i
n
p
u
t
d
e
v
i
c
e
t
o
e
n
t
e
r
c
o
m
m
a
n
d
s
i
n
t
o
S
U
T
a
n
d
t
h
e
s
e
c
o
n
d
p
a
r
t
i
s
t
h
e
a
d
v
a
n
c
e
d
i
m
a
g
e
r
e
c
o
g
n
i
z
e
r
.
T
h
e
k
e
y
b
o
a
r
d
a
n
d
m
o
u
s
e
e
m
u
l
a
t
o
r
i
s
u
s
e
d
a
s
a
n
i
n
p
u
t
d
e
v
i
c
e
t
o
r
e
p
r
o
d
u
c
e
t
e
s
t
c
o
m
m
a
n
d
s
a
n
d
s
e
n
d
t
h
e
m
t
o
a
n
S
U
T
.
T
h
e
i
m
a
g
e
a
n
a
l
y
z
e
r
a
l
g
o
r
i
t
h
m
i
s
d
e
v
e
l
o
p
e
d
u
s
i
n
g
O
p
e
n
C
V
a
n
d
o
p
t
i
c
a
l
c
h
a
r
a
c
t
e
r
r
e
c
o
g
n
i
z
e
r
(
O
C
R
)
t
o
o
l
s
t
o
h
e
l
p
a
u
t
o
m
a
t
e
s
o
m
e
t
e
s
t
c
h
a
l
l
e
n
g
e
s
.
O
u
r
r
e
s
u
l
t
s
h
o
w
s
t
h
a
t
a
d
v
a
n
c
e
d
u
s
e
r
i
n
t
e
r
f
a
c
e
(
U
I
)
t
e
s
t
a
u
t
o
m
a
t
i
o
n
(
A
U
T
A
)
c
a
n
p
e
r
f
o
r
m
a
1
2
5
t
e
s
t
c
a
s
e
s
w
i
t
h
i
n
5
h
o
u
r
s
c
o
m
p
a
r
e
d
t
o
4
8
h
o
u
r
s
f
o
r
a
h
u
m
a
n
t
o
c
o
m
p
l
e
t
e
t
h
e
j
o
b
.
Ke
yw
or
ds:
BIOS vali
datio
n
Com
pu
te
r
visi
on
OCR
Test
au
tom
at
ion
UI
aut
om
ation
This
is an
open
acc
ess arti
cl
e
un
der
the
CC
BY
-
SA
l
ic
ense
.
Corres
pond
in
g
Aut
h
or
:
Muslim
Mustapa
Faculty
of Elec
tro
nic Engi
nee
rin
g
Tec
hnol
ogy
Un
i
ver
sit
i M
al
ay
sia
Per
li
s
,
K
a
m
pu
s
Alam
Pau
h P
utra, 0
26
00 Ara
u,
Mal
a
ysi
a
Em
a
il
:
m
us
lim@unim
ap.
edu.
m
y
1.
INTROD
U
CTION
T
h
e
c
o
n
t
i
n
u
o
u
s
g
r
o
w
t
h
in
e
l
e
c
t
r
o
n
i
c
p
r
o
d
u
c
t
s
h
a
s
d
e
m
a
n
d
ed
a
d
v
a
n
c
e
d
a
p
p
l
i
c
a
t
i
o
n
s
t
o
s
u
p
p
o
r
t
t
h
e
r
i
g
o
r
o
u
s
e
n
g
i
n
e
e
r
i
n
g
d
e
s
i
g
n
a
n
d
d
e
v
e
l
o
p
m
e
n
t
p
h
a
s
e
.
A
n
e
n
g
i
n
e
e
r
i
n
g
p
r
o
d
u
c
t
t
h
e
n
n
e
e
d
t
o
b
e
v
a
l
i
d
a
t
e
d
o
r
t
e
s
t
e
d
b
e
f
o
r
e
i
t
b
e
e
n
r
e
l
e
a
s
e
d
i
n
t
o
t
h
e
m
a
r
k
e
t
.
T
e
s
t
e
n
g
i
n
e
e
r
[
1
]
n
e
e
d
s
t
o
e
n
h
a
n
c
e
c
u
r
r
e
n
t
v
a
l
i
d
a
t
i
o
n
s
y
s
t
e
m
t
o
c
a
t
c
h
u
p
w
i
t
h
t
h
e
r
a
p
i
d
c
h
a
n
g
e
s
i
n
t
e
c
h
n
o
l
o
g
y
[
2
]
.
T
e
d
i
o
u
s
a
n
d
l
a
b
o
r
-
i
n
t
e
n
s
i
v
e
m
a
n
u
a
l
t
e
s
t
s
i
n
e
n
g
i
n
e
e
r
i
n
g
v
a
l
i
d
a
t
i
o
n
t
e
a
m
a
r
e
g
r
a
d
u
a
l
l
y
m
i
s
s
i
n
g
t
h
e
d
e
a
d
l
i
n
e
a
s
t
h
e
e
l
e
c
t
r
o
n
i
c
p
r
o
d
u
c
t
s
a
r
e
b
e
c
o
m
i
n
g
m
o
r
e
c
o
m
p
l
i
c
a
t
e
d
a
n
d
i
n
v
o
l
v
i
n
g
a
n
a
g
i
l
e
d
e
v
e
l
o
p
m
e
n
t
p
r
o
c
e
s
s
.
T
h
e
r
e
w
e
r
e
e
f
f
o
r
t
s
d
o
n
e
b
y
s
o
m
e
t
e
c
h
n
o
l
o
g
y
c
o
m
p
a
n
y
to
d
e
v
e
lop
a
t
e
s
t
a
u
t
o
m
a
t
i
o
n
t
h
a
t
f
o
c
u
s
e
s
o
n
i
n
c
r
e
a
s
i
n
g
t
e
s
t
e
f
f
i
c
i
e
n
c
y
,
c
o
v
e
r
a
g
e
,
a
n
d
e
f
f
e
c
t
i
v
e
n
e
s
s
[
3
]
.
A
n
a
u
t
o
m
a
t
e
d
t
e
s
t
c
a
n
b
e
r
u
n
r
e
p
e
a
t
e
d
l
y
w
i
t
h
c
o
m
p
a
r
a
b
l
y
l
o
w
e
r
c
o
s
t
s
w
i
t
h
f
a
s
t
e
r
s
p
e
e
d
[
3
]
,
[
4
]
a
n
d
f
i
t
w
e
l
l
w
i
t
h
t
h
e
r
e
q
u
i
r
e
m
e
n
t
o
f
a
n
a
g
i
l
e
s
o
f
t
w
a
r
e
a
n
d
h
a
r
d
w
a
r
e
d
e
v
e
l
o
p
m
e
n
t
p
r
o
c
e
s
s
.
T
h
e
r
e
c
e
n
t
t
e
c
h
n
o
l
o
g
i
c
a
l
a
d
v
a
n
c
e
s
i
n
s
y
s
t
e
m
o
n
c
h
i
p
(
S
o
C
)
[
5
]
,
i
n
t
e
r
n
e
t
o
f
t
h
i
n
g
s
(
I
o
T
)
[
6
]
,
[
7
]
,
c
o
m
p
u
t
e
r
v
i
s
i
o
n
(
C
V
)
[
8
]
,
c
l
o
u
d
c
o
m
p
u
t
i
n
g
[
9
]
,
r
e
a
l
-
t
i
m
e
t
e
c
h
n
o
l
o
g
y
[
10
]
,
[
1
1
]
a
n
d
n
e
t
w
o
r
k
e
d
c
o
n
t
r
o
l
s
y
s
t
e
m
s
(
N
C
S
s
)
[
1
2
]
h
a
v
e
m
a
d
e
t
h
e
f
u
l
l
a
u
t
o
m
a
t
i
o
n
i
n
v
a
l
i
d
a
t
i
o
n
a
n
d
t
e
s
t
i
n
d
u
s
t
r
y
b
e
c
o
m
e
p
o
s
s
i
b
l
e
.
Most
of
t
he
te
s
t
cond
ucted
in
sem
ic
on
duct
or
industry
i
nvol
ve
hard
war
e
a
nd
s
of
t
war
e
inte
gr
at
io
n
a
nd
is
of
te
n
pe
rfo
r
m
ed
by
a
hu
m
an
te
ste
r
us
in
g
a
keybo
a
rd
a
nd
m
ou
se.
A
ny
te
st
inv
ol
ving
us
e
r
interfac
e
(
UI
)
need
a
key
bo
a
rd
a
nd
m
ou
se
to
dri
ve
a
te
st
run
[
13]
.
Test
eng
i
neer
nee
d
to
co
ntr
ol
the
keyb
oard
a
nd
m
ou
se
al
ong
with
m
on
it
or
i
ng
the
sy
stem
beh
a
vio
r
s
to
validat
e
th
e
te
st
run
[9]
.
Since
the
ste
ps
in
the
te
st
r
un
ar
e
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Adv
an
ce
d UI t
est
auto
ma
ti
on
(AUTA
)
for
B
I
OS va
li
dati
on
us
in
g
… (
Eiss
a Ab
dull
ah Awa
dh M
ohamme
d
)
1351
of
te
n
re
petit
ive
an
d
m
on
ot
onous,
te
ste
rs
oft
en
try
t
o
fi
nd
a
s
uitable
te
sti
ng
to
ol
to
a
uto
m
at
e
the
ta
sk
.
The
basic
re
qu
i
rem
ent
of
a
s
oft
wa
re
te
sti
ng
a
utom
at
ion
is
the
abili
ty
to
rep
r
oduce
a
key
bo
a
rd
and
m
ou
se
e
ve
nts
to
dr
i
ve
a test
on
a
n
syst
em
u
nder test (
SU
T
)
with acc
ur
acy
at
h
ig
her sp
ee
d
.
Ther
e
a
re
tw
o
ty
pes
of
us
e
r
interface
(
UI)
that
are
m
ai
nly
us
e
in
in
dustr
ia
l
app
li
cat
ions
wh
ic
h
are
gr
a
phic
al
us
er
interface
(
G
UI
)
a
nd
c
omm
and
-
li
ne
i
nt
erf
ace
(
CLI
)
.
Exam
ples
of
G
UI
are
de
sk
to
p
platfo
rm
s
[1
4
]
,
[15]
,
we
bs
it
e
[1
6
]
,
[
17]
,
a
nd
sm
artph
one
a
pp
li
cat
io
ns
[18]
,
[19]
.
T
he
e
xam
ples
for
CL
I
ar
e
Ba
sic
inp
ut
ou
tpu
t
syst
e
m
(B
IO
S
)
inter
face,
Linux
te
rm
inal
[20]
,
an
d
e
xtensi
ble
firm
war
e
inter
face
(
EFI
)
sh
el
l.
BI
OS
va
li
dation
e
ngineer
usual
ly
interact
s
m
anual
ly
with
UI
to
en
sure
that
the
BI
OS
fir
m
war
e
com
plete
d
it
s
job
with
out
a
ny
erro
r
(
e.
g.
c
he
ck
for
s
uccess
f
ul
bo
otin
g
to
s
ever
al
operati
ng
syst
em
s
or
c
hec
k
how
it
handles
the
po
wer
m
an
agem
ent
)
. Th
is
p
r
oject
foc
us
e
d
m
or
e on t
he
UI
i
nvolv
i
ng B
IO
S
v
al
id
at
ion,
su
c
h
as
W
i
ndows,
L
inux, EFI
-
s
hell, an
d
B
IOS int
erf
ace.
2.
THE
PR
OPO
SED
METHO
D
Ther
e
are
not
m
any
wo
r
ks
done
on
UI
base
d
BI
OS
validat
io
n
or
BIO
S
te
sti
ng
proce
dure
s
autom
at
ion
can
be
f
ound
in
any
scho
la
rly
arti
cl
e
[
13
]
,
[2
1
]
.
On
the
ot
her
ha
nd
the
re
are
m
any
wo
r
k
s
done
on
UI
aut
om
at
ion
in
gen
eral
w
her
e
s
om
e
of
the
m
are
app
li
cable
in
BIOS
validat
io
n
[22]
.
U
I
aut
om
ation
appr
oach can
be divi
ded int
o
t
wo
asp
ect
s
that
are
SU
T
-
dep
e
nd
e
nt a
nd Host
-
de
pe
nd
e
nt.
SU
T
-
de
pe
nd
e
nt
is
a
to
ol
or
m
et
ho
d
t
hat
r
e
qu
i
res
S
UT
to
be
a
ble
to
exe
cute
the
aut
oma
te
d
scri
pts
.
Alm
os
t al
l autom
at
ed
UI
test
ing
t
oo
ls
are
de
sign
e
d base
d o
n
S
UT
-
de
pe
nd,
su
c
h
as:
Siku
li
[1
5
]
is a
f
am
ou
s
GUI a
uto
m
at
ion
to
ol
that uses
im
ag
e rec
ogniti
on to
sea
rch f
or GUI
c
om
ponen
ts
on
the
disp
la
y
an
d
react
to
it
.
Sik
uli
le
ts
a
te
ste
r
sel
ect
a
reg
io
n
of
inte
rest
on
the
scree
n,
s
ubm
it
the
i
m
age
i
n
the
reg
i
on
as
a
qu
ery
to
the
search
en
gine
,
and
br
ow
se
the
search
re
su
lt
s.
Sik
uli
su
pp
or
ts
dif
fer
e
nt
op
e
rati
ng syst
e
m
s
su
ch
as
Wi
ndows,
Lin
ux, a
nd Mac
intos
h.
T
h
e
p
r
o
p
o
s
e
d
s
y
s
t
e
m
in
[
2
3]
a
d
d
r
e
s
s
e
s
t
h
e
d
e
s
k
t
o
p
v
e
r
s
i
o
n
o
f
p
r
o
g
r
a
m
m
i
n
g
b
y
d
e
m
o
n
s
t
r
a
t
i
o
n
p
r
o
b
l
e
m
.
I
t
s
t
a
n
d
s
a
t
t
h
e
c
r
o
s
s
w
a
y
s
o
f
i
n
t
e
n
t
i
o
n
-
i
n
f
e
r
e
n
c
e
,
s
o
f
t
w
a
r
e
u
s
a
b
i
l
i
t
y
,
a
n
d
a
c
t
i
o
n
i
d
e
n
t
i
f
y
i
n
g
a
n
d
p
r
e
d
i
c
t
i
n
g
.
I
t
a
l
l
o
w
s
a
t
e
s
t
e
r
t
o
t
e
a
c
h
a
b
o
t
t
o
d
o
a
r
e
p
e
t
i
t
i
v
e
t
a
s
k
.
T
h
e
p
r
o
c
e
s
s
i
s
s
i
m
i
l
a
r
t
o
t
e
a
c
h
i
n
g
a
h
u
m
a
n
f
o
r
t
h
e
f
i
r
s
t
t
i
m
e
.
T
h
e
t
o
o
l
p
r
e
d
i
c
t
s
a
n
d
r
e
c
o
v
e
r
s
a
u
s
e
r
'
s
i
n
t
e
n
d
e
d
l
o
o
p
s
b
y
a
n
a
l
y
z
i
n
g
t
h
e
d
a
t
a
f
r
o
m
t
h
e
i
r
i
n
i
t
i
a
l
d
e
m
o
n
s
t
r
a
t
i
o
n
.
The
w
ork
i
n
[
2
4]
pro
pose
d
a
m
et
ho
d
us
in
g
a
m
achine
le
ar
ning
m
et
ho
d
that
a
uto
m
at
ic
a
ll
y
identifie
s
GUI
widget
s
in
scr
eens
ho
ts
t
o
i
m
pr
ov
e
G
UI
te
sti
ng
.
The
s
tud
y
fou
nd
th
at
recog
nizing
G
UI
wi
dg
et
s
i
n
screen
shots
a
nd
us
in
g
t
his
i
nfor
m
at
ion
to
guide
ra
ndom
te
sti
ng
ac
hieved
sig
nifica
ntly
higher
branc
h
cov
e
ra
ge wh
e
n com
par
ed
to
t
r
aditi
on
al
rand
om
te
sti
ng
.
Chall
eng
e:
Th
ere
a
re
a
fe
w
c
halle
ng
e
s
in
a
pply
ing
the
SUT
-
de
pe
nd
e
nt
m
et
hod
for
B
IOS
vali
datio
n
autom
at
ion
.
T
his
m
et
ho
d
ca
nnot
run
a
utom
at
ed
scripts
without
OS
,
w
hich
m
eans
S
UT
ca
nnot
be
con
t
ro
l
l
e
d
durin
g
the
booting
process
or
in
B
IO
S
i
nt
erf
aces.
Anot
her
c
halle
nge
is
SU
T
-
de
penden
t
m
et
ho
d
does
no
t
su
pp
or
t a
CLI
su
c
h
as
EFI
-
s
he
ll
an
d Term
inals, w
hich
are e
ssentia
l i
n
B
I
OS
validat
io
n.
Ho
st
-
de
pe
nd
e
nt
is
a
m
e
tho
d
that
ru
ns
auto
m
at
ed
scripts
on
a
n
ind
e
pe
ndent
platf
or
m
cal
le
d
ho
st
m
achine
an
d
us
ua
ll
y
has
a
hard
war
e
c
onne
ct
ion
betwee
n
the
host
a
nd
SU
T
.
T
his
m
et
hod
is
ha
ving
f
ul
l
con
t
ro
l
of
an
S
UT
an
d
can
r
un
an
aut
om
at
e
d
scripts.
It
do
es
no
t
rely
on
SU
T
O
S
.
F
e
w
past
stud
ie
s
ha
ve
bee
n
done
on the
ho
st
-
de
pende
nt m
et
hod
s
uc
h
as:
The
work
i
n
[
25
]
pr
opos
e
d
a
n
aut
om
at
ed
GU
I
te
sti
ng
m
eth
od
base
d
on
im
age
detect
io
n
w
hich
us
es
t
he
i
m
age
detect
ion
te
ch
no
l
og
y
t
o
rec
ognize
c
om
po
nen
ts
in
t
he
GUI
im
age.
The
in
put
de
vice
si
m
ulate
s
inp
ut
sign
al
s
into
S
UT.
Usi
ng
this
m
et
ho
d,
they
dev
el
ope
d
a
GUI
te
sti
ng
pl
at
fo
rm
cal
le
d
AG
T
P,
a
univ
ersal
and
no
n
-
i
ntrus
ive
G
U
I
te
sti
ng
to
ol
for
S
UT
.
Wh
e
n
the
te
s
t
script
runs,
it
will
cal
l
the
i
m
age
rec
ogniti
on
a
nd
i
nput
de
vi
ce
si
m
ulati
on
to
com
plete
the
correspo
ndin
g
op
e
rati
ons
acc
ordin
g
to
t
he
instr
uctions.
Aft
er
execu
ti
ng
the
te
st
script,
te
st
res
ults
inf
or
m
at
ion
will
be
rec
orde
d
and
sto
red
in
the
database
f
or
su
bse
que
nt p
la
yback a
nd
regr
ession t
est
ing.
A
c
a
p
t
u
r
e
/
r
e
p
l
a
y
t
e
s
t
i
n
g
t
o
o
l
c
a
l
l
e
d
K
O
R
A
T
[
2
1
]
.
K
O
R
A
T
a
d
o
p
t
s
a
h
a
r
d
w
a
r
e
c
o
m
p
o
n
e
n
t
t
o
i
n
t
e
r
c
e
p
t
a
n
d
e
m
u
l
a
t
e
k
e
y
b
o
a
r
d
/
m
o
u
s
e
s
i
g
n
a
l
s
t
o
d
r
i
v
e
a
S
U
T
a
s
i
f
t
h
e
S
U
T
i
s
i
n
t
e
r
a
c
t
i
n
g
w
i
t
h
a
h
u
m
a
n
.
A
t
e
s
t
e
r
c
a
n
d
e
s
i
g
n
a
n
d
o
p
e
r
a
t
e
a
t
e
s
t
c
a
s
e
t
o
r
e
c
o
r
d
t
h
e
i
n
t
e
n
d
e
d
b
e
h
a
v
i
o
r
s
i
n
t
o
K
O
R
A
T
t
e
s
t
c
a
s
e
s
c
r
i
p
t
o
n
a
c
o
r
r
e
c
t
S
U
T
w
i
t
h
o
u
t
p
r
o
g
r
a
m
m
i
n
g
s
k
i
l
l
s
.
I
n
a
r
e
g
r
e
s
s
i
o
n
r
u
n
,
t
h
e
t
e
s
t
c
a
s
e
i
s
r
e
p
l
a
y
e
d
,
a
n
d
t
h
e
c
o
r
r
e
c
t
n
e
s
s
i
s
a
s
s
e
r
t
e
d
a
u
t
o
m
a
t
i
c
a
l
l
y
b
y
a
n
a
l
y
z
i
n
g
S
U
T
'
s
v
i
d
e
o
o
u
t
p
u
t
(
i
m
a
g
e
s
)
a
n
d
s
e
n
d
i
n
g
k
e
y
b
o
a
r
d
a
n
d
m
o
u
s
e
s
i
g
n
a
l
s
s
m
a
r
t
l
y
.
S
i
n
c
e
K
O
R
A
T
o
n
l
y
i
n
t
e
r
f
a
c
e
s
t
h
e
v
i
d
e
o
o
u
t
p
u
t
o
f
a
S
U
T
,
t
h
i
s
p
l
a
t
f
o
r
m
i
s
c
o
n
s
i
d
e
r
e
d
i
n
d
e
p
e
n
d
e
n
t
a
n
d
n
o
n
-
i
n
t
r
u
s
i
v
e
,
w
h
i
c
h
m
e
a
n
s
t
h
e
r
e
i
s
n
o
p
e
r
f
o
r
m
a
n
c
e
i
n
t
e
r
f
e
r
e
n
c
e
c
a
u
s
e
d
b
y
K
O
R
A
T
'
s
c
a
p
t
u
r
e
a
n
d
r
e
p
l
a
y
.
KO
R
AT
is
int
rod
uced
a
gain
in
[
13
]
with
m
ajor
im
pr
ovem
ents.
ARM
-
co
r
te
x
M4
dev
el
opm
ent
bo
ar
d
i
s
us
e
d
as
a
USB
e
m
ulator
an
d
at
the
sam
e
tim
e
the
wor
k
com
bin
ed
both
S
UT
-
de
pe
nd
e
nt
an
d
H
ost
-
dep
e
ndent
ap
proac
hes
f
or
m
ou
se
autom
at
ion
.
The
im
pr
ov
ed
KO
R
AT
ha
s
i
m
pr
ov
e
d
the
detect
ion
rate
of
op
ti
cal
c
har
act
er r
ec
ognize
r (
OCR)
from
n
early
25% to
n
e
arly
85%.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c
En
g
&
Co
m
p
Sci,
Vo
l.
23
, N
o.
3
,
Se
ptem
ber
20
21
:
13
50
-
135
6
1352
Chall
eng
e:
Th
e
host
-
dep
e
nd
ent
m
et
ho
d
is
su
it
able
t
o
a
uto
m
at
e
BIOS
validat
io
n
be
cause
it
ca
n
con
t
ro
l
an
S
U
T
at
ever
y
sta
ge,
sta
rting
f
ro
m
the
BIOS
interface,
bo
ot
up,
an
d
O
S
le
ve
l.
Howev
e
r,
th
ere
ar
e
few
c
halle
nges
face
d
in
past
works
s
uch
as
the
capt
ur
e/
r
ep
la
y
m
et
ho
d
in
GUI
a
uto
m
at
ion
[
26]
.
T
his
m
et
hod
has
a
high
ris
k
of
fail
ur
e
be
cause
a
syst
e
m
or
s
of
twa
re
with
var
io
us
loading
ti
m
es
that
causes
a
ut
om
at
ed
scripts
can
be
easi
ly
m
isti
m
e
d.
The
oth
e
r
chall
eng
e
faced
is
the
i
m
age
reco
gnit
ion
to
ol
s
based
on
the
ho
st
-
dep
e
ndent
m
eth
od
can
only
autom
at
e
the
m
ou
se
po
sit
io
n
an
d
do
e
s
no
t
su
pport
any
UI
syst
em
m
e
t
hod
that
on
ly
acce
pts
keyb
oard
as
a
n
in
pu
t
dev
ic
e
su
ch
as
B
IOS
.
O
ur
pro
pos
ed
m
et
ho
d
off
ers
a
so
l
ution
to
al
l
lim
it
at
ion
s f
ac
ed by p
re
vious
pro
po
se
d
m
et
ho
ds.
3.
RESEA
R
CH MET
HO
D
T
h
i
s
s
e
c
t
i
o
n
e
x
p
l
a
i
n
s
our
p
r
o
p
o
s
e
d
m
e
t
h
o
d
for
U
I
a
u
t
o
m
a
t
i
o
n
p
r
o
c
e
s
s
on
B
I
O
S
v
a
l
i
d
a
t
i
o
n
.
F
i
g
u
r
e
1
s
h
o
w
s
t
h
e
o
v
e
r
v
i
e
w
o
f
t
h
e
p
r
o
p
o
s
e
d
m
e
t
h
o
d
. T
he
e
x
p
l
a
n
a
t
i
o
n
a
b
o
u
t
t
h
e
A
U
T
A
p
r
o
j
e
c
t
a
r
e
d
i
v
i
d
e
d
i
n
t
o
t
h
r
e
e
p
r
i
m
a
r
y
p
h
a
s
e
s
:
P
h
a
s
e
1
:
K
M
e
m
u
l
a
t
o
r
,
P
h
a
s
e
2
:
O
C
R
i
m
p
r
o
v
e
m
e
n
t
i
n
B
I
O
S
i
n
t
e
r
f
a
c
e
,
a
n
d
P
h
a
s
e
3
:
A
U
T
A
l
i
b
r
a
r
y
.
Figure
1. O
verview
of the
AUTA
pr
oj
ect
3.1.
P
ha
se
1:
KM
e
mula
to
r
Keyb
oard
a
nd
m
ou
se
em
ulator
is
de
vel
oped
in
ph
a
se
1
to
s
en
d
keyb
oard
keyst
roke
s
an
d
m
ou
se
m
ot
ion
s
from
a
host
m
achine
to
a
n
S
UT
.
A
ti
ny
m
ic
ro
con
t
ro
ll
er
(
pro
m
ic
ro
)
with
the
ATm
ega32U4
c
hip
i
s
us
e
d
to
em
ulate
keyb
oard
ke
yst
ro
ke
s
a
nd
m
ou
se
m
otion
s.
US
B
to
TT
L
co
nverter
m
odule
is
re
qu
i
red
to
connect
the
m
ic
ro
c
ontrolle
r
with
a
host
m
achine.
CP2
10
2
m
od
ule
with
300
bps
to
1.
5
Mb
ps
Ba
ud
rates
i
s
us
e
d
to
tr
ansm
it
data
from
a
ho
st
m
achine
t
o
the
pro
m
ic
r
o
.
T
he
m
ic
ro
c
on
t
ro
ll
er
is
c
ontr
olled
f
r
om
t
he
ho
st
m
achine
th
rou
gh
serial
c
omm
un
ic
at
ion
by
us
in
g
a
set
of
pr
e
def
i
ned
inst
ru
ct
io
ns
.
I
nc
om
ing
inst
ru
ct
io
ns
from
the
serial
port
are
sp
li
t
into
t
wo
par
ts
as
he
ad
an
d
body.
T
he
first
th
ree
c
har
act
er
s
are
c
on
si
der
e
d
as
he
ad
an
d
the
rem
ai
nin
g
char
act
e
rs
are
consi
der
e
d
as
body.
Hea
d
-
da
ta
is
us
ed
to
de
scribe
t
he
re
qu
ired
functi
on
f
r
om
th
e
m
ic
ro
co
ntro
ll
e
r.
Ta
ble
1
s
ho
ws
th
e c
omm
a
nd li
ne
in
struc
ti
on
e
xam
ples.
Table
1
. H
ow the
host m
achine u
ses
ser
ia
l c
om
m
un
ic
at
ion
Fu
n
ctio
n
s
Predef
in
ed
Head
-
d
ata
Co
m
m
an
d
-
lin
e
Exa
m
p
le
Key
b
o
ard p
rintin
g
Kp
:
Kp
:tex
t
Kp
:h
ello
word
Key
b
o
ard b
u
tto
n
s
Kb
:
Kb
:b
u
tto
n
1
+b
u
tto
n
2
+….
Kb
:ctrl+alt+delet
e
Key
b
o
ard h
o
ld
in
g
Kh
:
Kh
:b
u
tto
n
,state
Kb
:ctrl,
1
Mou
se
m
o
v
in
g
m
m
:
Kp
:d
irection
,pix
cel
m
m
:
u
p
,25
Mou
se click
in
g
m
c
:
Kb
:b
u
tto
n
m
c
:r
Mou
se p
o
sitio
n
m
p
:
m
p
:
X ,
Y
m
p
:2
6
5
,80
1
3.2.
P
ha
se
2:
OCR impr
ove
ment
in
BIOS
interf
ace
An
OCR
is
use
d
to
rec
ogniz
e
te
xt
in
BIO
S
m
enu
as
show
n
in
Fi
gure
2
.
It
re
quires
an
im
pr
ov
e
d
i
m
age
to
f
unct
ion
ac
cu
ratel
y.
I
n
s
uch
a
ca
se
,
im
age
analy
zi
ng
a
nd
im
pr
ovin
g
a
re
tw
o
m
ai
n
i
m
po
rtan
t
thin
gs
in
this
pr
oj
ect
.
Op
e
nC
V
is
use
d
to
ca
pture
the
vi
deo
fr
am
e
from
the
ta
rg
et
de
vice
an
d
exec
ute
the
f
r
a
m
e
'
s
i
m
age
processi
ng.
Tesse
ract
-
OCR
is
us
e
d
to
rec
ognize
th
e
char
act
e
r
in
the
im
age.
Im
a
ge
im
pr
ov
em
ent
in
the
BIOS
i
nterf
ac
e
can
be
di
vid
e
d
into
t
hr
ee
m
ai
n
pa
rts
w
hic
h
are:
1)
im
age
analy
zi
ng
on
sel
ect
ed
te
xt
(
I
AS
T
),
2)
im
age an
al
yz
ing
on text
m
enu
(
IATM
), a
nd 3)
im
age anal
yz
in
g
on
pop
-
up m
enu
(IAP
M).
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Adv
an
ce
d UI t
est
auto
ma
ti
on
(AUTA
)
for
B
I
OS va
li
dati
on
us
in
g
… (
Eiss
a Ab
dull
ah Awa
dh M
ohamme
d
)
1353
Figure
2. The
targ
et
te
xt on
t
he
BIOS
gr
a
phic
u
se
r
inte
rf
ace
3.2.1.
I
AS
T
Im
age
analy
zin
g
on
sel
ect
ed
te
xt
(I
AS
T
)
f
un
ct
io
n
is
devel
op
e
d
to
recogn
iz
e
the
sel
ect
ed
te
xt
in
the
BIOS
GUI,
usual
ly
hig
hlig
ht
ed
with
a
blac
k
box.
The
f
unct
ion
has
sev
en
sta
ges
be
fore
go
i
ng
to
O
CR
,
as
sh
ow
n
in
Fig
ure
3.
Eac
h
fr
am
e
has
to
go
th
r
ough
al
l
the
sta
ges.
C
onve
rt
to
gr
ay
scal
e
sta
ge
is
conve
rting
each
fr
am
e
to
gr
ay
scal
e
i
m
age.
The
n
it
has
to
un
derg
o
no
ise
rem
ov
al
to
inc
rease
the
accuracy
in
i
m
age
thres
ho
l
ding
a
nd
fin
d
co
ntou
rs
sta
ges.
T
he
appr
oxPo
ly
D
P
functi
on
is
use
d
to
detect
po
ly
go
n
with
f
our
-
si
ded
sh
a
pe
an
d
heig
ht
that
is
no
t
exceed
i
ng
25
pi
xels.
Th
e
detec
te
d
po
ly
gon
wi
ll
be
cro
ppe
d
a
nd
rev
e
rse
col
ur
to
increase
the
de
te
ct
ion
acc
ur
ac
y
in
OCR.
T
he
outp
uts
of
thi
s
f
un
ct
i
on
are
the
locat
io
n
of
the
highli
ghte
d
te
xt
and it
s ex
tra
ct
ion.
Figure
3. I
AS
T
stages
3.2.2.
I
AT
M
Im
age
analy
zin
g
on
te
xt
m
e
nu
(IATM)
functi
on
recog
ni
zes
al
l
the
te
xt
m
enu
in
the
BIOS
GUI
excep
t
the
sel
e
ct
ed
te
xt.
This
fu
ncti
on
c
on
t
ai
ns
four
sta
ge
s,
as
sh
ow
n
in
Figu
re
4,
w
hi
ch
is
le
ss
than
IA
ST
because
it
do
e
s
not
nee
d
m
uch
pr
e
-
processi
ng.
Af
te
r
the
f
ram
e
ha
s
bee
n
cov
e
rted
to
gra
ysc
al
e
the
data
from
char
act
e
r
f
ro
m
the
im
ages
are
extracte
d.
T
he
te
xt
posit
ion
will
be
locat
e
d
an
d
ar
ra
ng
e
d
i
n
order
t
o
assi
st
the
sel
ect
ion
proce
ss lat
er.
Figure
4. I
AT
M st
ages
3.2.3.
I
AP
M
Im
age
analy
zin
g
on
po
p
-
up
m
enu
(
IAPM)
functi
on
rec
ognizes
the
te
xt
on
the
pop
-
up
m
enu
from
the
BIOS
G
UI.
T
hi
s
functi
on
c
on
ta
ins
te
n
sta
ge
s
to
im
pr
ove
OCR
accu
racy
be
fore
goin
g
t
o
OCR,
as
s
hown
in
Figure
5.
Mo
st
of
t
he
sta
ges
us
e
d
are
the
s
a
m
e
excep
t
in
IA
PM
t
her
e
is
an
er
os
io
n
sta
ge
use
d
t
o
re
m
ov
es
pix
el
s
on selec
te
d
ob
j
ect
bounda
ry.
3.3.
P
ha
se
3:
AU
T
A
li
br
ary
This
sect
ion
discuss
es
f
unct
io
ns
in
the
A
UT
A
li
br
ary
de
ve
lop
e
d
us
i
ng
py
thon
la
ngua
ge,
wh
ic
h
will
help
a
te
ste
r
to
create
an
aut
om
at
ed
script
by
cal
li
ng
avail
able
f
un
ct
i
on
s
.
Pyt
hon
is
on
e
of
the
m
os
t
popula
r
and po
werfu
l
pro
gr
am
m
ing
la
ngua
ges wit
h
t
he
ca
pab
il
it
ie
s
on
cr
os
s
-
platf
orm
interp
reter
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c
En
g
&
Co
m
p
Sci,
Vo
l.
23
, N
o.
3
,
Se
ptem
ber
20
21
:
13
50
-
135
6
1354
Figure
5. I
APM
stages
3.3.1. Bi
os
_g
oto
f
u
ncti
on
On
e
of
t
he
fir
m
war
e
te
st
en
gin
ee
r
routine
is
to
go
th
r
ough
BI
OS
m
enu
s
t
o
ver
i
fy
B
IO
S
s
et
ti
ngs.
Diff
e
re
nt
te
st
cases
in
BI
OS
validat
io
n
re
quires
diff
e
re
nt
set
ti
ng
s.
BI
OS
m
enu
s
a
nd
se
tt
ing
s
do
no
t
ha
ve
a
fixe
d
posit
ion
,
and
with
ever
y
new
firm
war
e
release
,
it
m
a
y
chan
ge
their
po
sit
io
n
or
ord
er.
H
um
an
ey
e
s
can
easi
ly
reco
gniz
e
a
change
i
n
a
po
sit
ion,
but
a
m
achine
do
e
s
not
ha
ve
this
recog
niti
on
a
bili
ty
.
Th
e
i
m
age
analy
zer
an
d
ke
yboa
rd
em
ulator
are
c
om
bin
ed
to
so
l
ve
the
recogn
it
io
n
pr
ob
le
m
and
al
lows
the
host
m
a
chine
to
detect
an
d
s
el
ect
the
cor
r
e
ct
m
enu
.
T
wo
functi
ons
ha
ve
been
c
reated
us
in
g
Pyt
hon
c
od
e
t
hat
are
se
le
ct
ing
the r
e
quired
BI
OS
m
enu
a
nd s
el
ect
ing
the
po
p
-
up m
enu
'
s r
e
qu
i
red set
ti
ng.
Ther
e
a
re
fe
w
ste
ps
involve
d
in
sel
ect
ing
the
BIOS
m
enu
.
T
he
first
st
ep
is
to
scan
the
cu
rr
e
ntly
sel
ect
ed
m
enu
us
in
g
the
I
AST
functi
on.
If
i
t
is
m
at
ched
w
it
h
the
require
d
m
enu
,
then
the
f
un
ct
io
n
wil
l
retur
n
Tru
e
Value
.
I
f
it
do
es
not
m
atch
,
the
black
box
l
ocati
on
will
be
save
d
insi
de
the
var
ia
ble
and
goes
to
th
e
next
ste
p,
sca
nnin
g
the
w
hole
cu
rrent
pa
ge
us
i
ng
the
I
ATM
f
unct
ion
.
If
the
re
i
s
a
m
at
ch,
the
I
ATM
f
un
ct
io
n
will
return
t
he
posit
ion
of
the
m
at
ched
m
enu.
T
he
keyb
oard
e
m
ula
tor
will
send
ei
ther
ar
row
up
or
a
rro
w
do
w
n
keyst
roke,
de
pe
nd
i
ng
on
the
require
d
m
enu
's
locat
ion.
I
AST
will
be
us
e
d
ever
y
ti
m
e
the
keyb
oard
em
ulato
r
sen
ds
a
keyst
r
ok
e
to
e
ns
ure
th
at
the
requir
ed
m
enu
has
been
sel
ect
ed
.
If
there
is
no
m
at
ch,
the
keybo
a
rd
e
m
ulator
will
sen
d
a
pag
e
dow
n
butt
on
an
d
re
peat
al
l
th
e
ste
ps
un
ti
l
the
la
st
pag
e.
I
f
al
l
pag
es
ha
ve
bee
n
check
e
d
a
nd
no m
at
ched
m
enu
, t
hen the test
will
stop, a
nd the test
r
e
su
lt
is
conside
red as
fail
ing
.
To
sel
ect
the
BIOS
set
ti
ng
i
n
the
pop
-
up
m
enu
,
the
first
ste
p
is
to
use
the
I
AP
M
f
unc
ti
on
to
detect
and
cr
op
pop
-
up
m
enu
a
nd
do
so
m
e
i
m
age
enh
a
ncem
ent
on
the
te
xt
to
m
a
ke
it
cl
eare
r.
T
he
sec
ond
ste
p
us
e
s
the
I
AS
T
funct
ion
to
scan
the
sel
ect
ed
te
xt
a
nd
c
om
par
e
it
with
the
requir
ed
set
ti
ng.
I
f
it
is
m
at
ched
,
th
en
th
e
functi
on
will
return
true
val
ue
.
If
it
do
es
not
m
at
ch,
the
black
bo
x
or
s
el
ect
ed
te
xt
locat
ion
will
be
save
d
inside
the
va
riable
and
go
e
s
to
the
ne
xt
ste
p,
scan
ning
the
pop
-
up
m
enu
usi
ng
I
AP
M
an
d
IATM
functi
on
s
.
I
f
there
is
a
m
at
c
h
the
IA
TM
functi
on
retu
r
ns
the
posit
io
n
of
t
he
re
quired
set
ti
ng
.
By
com
par
in
g
th
os
e
pos
it
ions
(actual
a
nd
ta
r
get),
t
he
keyb
oard
em
ulator
will
send
the
ke
yst
ro
ke
to
t
he
require
d
set
ti
ng.
I
AST
f
un
c
ti
on
is
us
e
d
upon
e
ve
ry
keyst
roke
t
o
ens
ure
the
r
equ
i
red
set
ti
ng
is
sel
ect
ed.
If
there
is
no
m
at
ched
,
the
t
est
i
s
consi
der
e
d
a
s fai
lure
a
nd the
f
un
ct
io
n
is
term
inate
d.
3.3.2. M
ou
se
_m
ovet
ot
em
pla
te
f
unc
tion
The
ta
sk
to
i
nc
lud
e
a
m
ou
se
po
sit
io
n
f
un
ct
ion
wit
hin
an
autom
at
ed
script
is
no
t
an
ea
sy
ta
sk
for
a
te
ste
r
due
to
th
e
diff
ic
ulty
in
ob
ta
ini
ng
the
t
arg
et
e
d
area
e
xact
co
ordinat
es.
To
so
l
ve
th
is
issue,
the
te
m
pla
te
m
at
ching
m
et
h
od
is
us
e
d.
T
he
thresholdi
ng
value
of
0.8
is
us
ed
to
recogn
iz
e
the
re
gi
on
of
int
e
rest
on
th
e
screen
cove
rin
g
te
xt,
sym
bo
l, an
d bu
tt
on.
T
he
n
the
m
ou
se
posit
ion f
un
ct
i
on
will
p
oi
nt to
wards t
he
locat
ion
.
3.3.3. Wai
tfor
_t
em
pla
te
f
un
ction
In
ste
a
d
of
us
i
ng
est
im
at
ed
t
i
m
e
or
delay
t
i
m
e
fo
r
the
ne
xt
ste
p,
this
f
unct
ion
is
us
ed
to
keep
t
he
autom
at
ed
scri
pt in
a
hold
sta
te
an
d co
ns
ist
e
ntly
ch
eck
in
S
UT'
s scr
ee
n unt
il
m
at
ching
th
e te
m
plate
.
4.
RESU
LT
S
AND A
N
ALYSIS
This
sect
ion
di
scusses
the
A
UTA
'
s
res
ult
a
nd
it
s
com
par
ison
wit
h
the
m
anu
al
te
ste
r
i
m
plem
entat
ion
in
te
rm
s
of
the
com
pleti
on
ti
m
e.
The
res
ults
are
obta
ine
d
thr
ough
se
ver
a
l
sp
eci
fic
te
st
c
ases
im
ple
m
en
te
d
on
the
real
te
st
en
vir
on
m
ent.
A
n
ap
plica
ti
on
wi
th
a
G
UI
inter
face
is
de
velo
ped
ba
sed
on
the
AUTA
li
brary
t
o
te
st
KM
e
m
ulator
f
unct
ions
on
diff
e
re
nt
platfo
rm
s.
The
te
st
resu
lt
s
sh
owe
d
that
K
M
e
m
ulator
f
un
ct
io
ns
worked
p
e
rf
ect
ly
o
n dif
fer
e
nt
platfo
rm
s su
ch
as BI
OS
i
nter
f
ace, EF
I
-
s
hell,
W
i
ndows
, a
nd
Linux
.
Fo
r
the
OCR
im
pr
ov
em
ents,
al
l
three
fu
ncti
on
s
a
re
te
ste
d
on
BI
OS
m
enus
and
co
ns
ist
en
t
resu
lt
s
are
ob
ta
ine
d
with
high
acc
ur
acy
detect
ion
in
al
ph
a
bet
c
har
act
ers
reachi
ng
90.
32%,
w
hich
cov
e
rs
c
apita
l
le
tt
ers
A
-
Z
,
sm
al
l
le
tters
a
-
z
a
nd
nu
m
ber
s
0
-
9.
In
(
1
)
is
us
e
d
to
m
easur
e
OCR
accuracy.
Alth
ough
the
fr
am
e
has
been
en
ha
nced
with
m
ulti
ple
m
et
ho
ds,
s
om
e
of
t
he
al
pha
be
t
and
num
ber
char
act
er
s
ha
ve
si
m
il
ar
sh
ape
wh
ic
h
conf
us
es t
he O
CR
to
disti
ng
uish
betwee
n
t
hose cha
racters.
The
c
onf
us
in
g characte
r
li
st i
s
:
Evaluation Warning : The document was created with Spire.PDF for Python.
Ind
on
esi
a
n
J
E
le
c Eng &
Co
m
p
Sci
IS
S
N:
25
02
-
4752
Adv
an
ce
d UI t
est
auto
ma
ti
on
(AUTA
)
for
B
I
OS va
li
dati
on
us
in
g
… (
Eiss
a Ab
dull
ah Awa
dh M
ohamme
d
)
1355
I
1
---
>
I
(alp
hab
et
)
1(n
um
ber
)
.
O
0
---
>
O(
al
ph
a
bet)
0(n
um
ber
).
U
V
---
>
U(
al
ph
a
bet)
V(al
phabet)
.
.
%
=
(
1
−
ℎ
ℎ
)
×
100
(1)
BIOS
m
enu
s
c
on
ta
ini
ng
the
conf
us
in
g
c
harac
te
rs
will
re
duce
th
e
OCR
accuracy.
To
im
pr
ov
e
t
he
OCR
accu
racy
,
we
i
nvest
iga
te
OCR
outp
ut
that
involve
s
co
nfusing
c
har
act
er
s
to
fi
nd
any
relat
io
ns
hi
p
betwee
n
OCR
input
an
d
out
put.
W
e
te
ste
d
OCR
on
di
ff
e
r
ent
BIO
S
m
enu
s
a
nd
fou
nd
out
that
each
i
nc
orrect
char
act
e
r
has
only
two
ou
t
put
possibil
it
ie
s:
a
ct
ual
char
act
e
r
and
sim
il
ar
sh
ape
cha
racter
.
To
s
olv
e
t
his
issue
,
we
create
d
a
functi
on
that
can
gen
e
rate
al
l
po
ssi
ble
outc
om
es,
as
s
how
n
in
Fig
ure
6.
T
he
num
ber
of
po
s
sibil
it
ie
s w
il
l i
ncr
ease ex
pone
ntial
ly
b
ased on
the
nu
m
ber
of inc
orrect characte
rs
2
n
w
her
e
n
is eq
ual to th
e
nu
m
ber
of inc
orrect c
har
act
e
rs
.
To
te
st
t
he
AUTA
li
brary,
on
e
aut
om
at
ed
script
hav
e
be
en
create
d
us
i
ng
our
li
brary
f
or
15
ra
ndom
te
st
cases
in
BIOS
validat
io
n.
It
to
ok
ar
ound
42
m
inu
te
s
to
com
plete
al
l
15
te
st
c
ases.
Each
te
st
cas
e
ha
s
diff
e
re
nt
com
pleti
on
ti
m
e.
T
he
est
im
at
ed
t
i
m
e
to
com
plete
125
ca
ses
us
in
g
AUTA
is
ar
ound
4
-
6
hour
s
com
par
ed
t
o
48
ho
ur
s
com
pleti
on
ti
m
e
by
a
te
ste
r,
as
show
n
in
Ta
ble
2.
Test
a
uto
m
at
ion
has
m
any
adv
a
ntage
s
co
m
par
ed
to
th
e
te
st
per
f
orm
ed
m
anu
al
ly
.
Au
t
om
at
ed
te
sts
can
be
run
re
petit
ively
with
com
par
ably
lo
wer costs
, less
tim
e, an
d
l
ow
error.
Figure
6. Exa
m
ple o
n
ho
w
t
o
c
reate al
l p
ossi
ble outcom
es f
r
om
the text
with s
uspect
ed
ch
a
racters
Table
2
.
C
om
par
iso
n betwee
n m
anu
al
an
d a
ut
om
a
te
d
te
sts
Manu
al test
Au
to
m
ated
test
Ti
m
e
-
co
n
su
m
in
g
(
av
erage)
4
8
ho
u
rs
4
-
6
ho
u
rs
Ef
f
icien
cy
Low
Hig
h
Testin
g
ti
m
e
f
lex
ib
ility
W
o
rkin
g
ho
u
rs on
l
y
An
y
ti
m
e
Co
st
Hig
h
Low
5.
CONCL
US
I
O
N
In
t
his
w
ork,
we
ha
ve
pro
posed
a
m
et
ho
d
t
o
aut
om
at
e
BI
OS
validat
ion
us
in
g
O
pe
nCV
and
OCR
.
This
wor
k
has
sho
wn
that
by
ha
ving
a
f
ully
autom
at
ion
BIOS
vali
datio
n
it
c
ou
l
d
redu
ce
the
te
st
ti
m
e
by
a
factor
of
ei
ght.
The
im
age
enh
a
ncin
g
a
nd
processi
ng
te
c
hn
i
qu
e
pro
pos
ed
in
our
m
eth
od
has
inc
rea
se
the
accuracy
up to
90%.
REFERE
NCE
S
[1]
S.
Braun,
F.
Elberz
hag
er,
and
K.
Holl,
"A
uto
m
at
ion
Support
for
Mobil
e
A
pp
Quali
t
y
As
suranc
e
-
A
To
ol
La
ndsca
p
e,
"
in
P
roce
dia
Comput
er
Scienc
e
,
vol.
110,
pp
.
117
–
12
4,
2017
,
doi
:
10
.
1016/j
.
pro
cs.
201
7.
06.
129
.
[2]
M.
Portola
n,
"A
utomate
d
T
esti
n
g
Flow:
the
Pre
sent
and
the
Fut
ure
,
"
IE
EE
Tr
ans.
Comput.
Des.
Inte
gr.
Circu
it
s
Syst.
,
2019,
doi
: 10.1109/
t
ca
d
.
20
19.
2961328.
[3]
A.
Ism
ai
l,
S.
Int
u,
S.
Za
m
bri,
"
A
GU
I
-
drive
n
prototy
pe
for
s
y
n
the
sizi
ng
se
lf
-
ad
apt
a
ti
on
de
ci
sio
n,
"
in
Bul
l
et
in
o
f
El
e
ct
rica
l
Eng
in
ee
ring a
nd
Infor
matic
s
,
vo
l. 9, n
o.
2
,
pp
.
792
-
80
0,
2020
,
doi
:
10
.
11591/e
e
i.
v9i2
.
1
716
[4]
S.
Jam
al
udin,
N.
Za
inal,
and
W
.
M.
D.
W
.
Za
ki,
"D
ebl
urring
of
Nois
y
Iris
Rec
o
gnit
ion,
"
in
Bu
lleti
n
of
Elec
tric
a
l
Engi
ne
ering
and
Informatic
s
,
vol
.
10
,
no
.
1
,
pp
.
1
56
-
159
,
2021
,
d
oi
:
10
.
11591/eei
.
v10i1.
246
7
[5]
S.
Suhartono,
F.
Nugroho,
M.
F
ai
sal
,
M.A.
Yaqi
n,
and
S.
Suvant
a,
“
Speak
er
Re
c
ognit
io
n
in
Cont
ent
-
base
d
Im
age
Ret
ri
eva
l
for
a
High
Degre
e
of
Acc
ura
c
y
,
"
in
Bul
letin
of
Elec
t
rical
Engi
n
ee
rin
g
and
Informatic
s
,
v
ol.
7,
n
o.
3
,
pp.
350
-
358
,
Se
pte
m
ber
2018
,
d
oi
:
10
.
11591/eei
.
v7i3.
957
Evaluation Warning : The document was created with Spire.PDF for Python.
IS
S
N
:
2502
-
4752
Ind
on
esi
a
n
J
E
le
c
En
g
&
Co
m
p
Sci,
Vo
l.
23
, N
o.
3
,
Se
ptem
ber
20
21
:
13
50
-
135
6
1356
[6]
N.
Moreir
a,
J.
L
áz
aro
,
U.
Bida
r
t
e,
J.
Jim
ene
z
,
an
d
A.
As
ta
rloa
,
"
On
the
Util
i
za
t
io
n
of
Sy
st
em
-
on
-
Chip
Plat
form
s
to
Achie
ve
Nanose
cond
S
y
nchr
oni
za
t
ion
Acc
ura
cies
in
Subs
ta
ti
on
Autom
at
ion
Sy
stems
,
"
IEE
E
Trans.
Smar
t
G
rid
,
2017,
doi
:
10
.
11
09/T
SG
.
2015.
25
12440.
[7]
N.
H.
M.
I.
Za
i
hani
,
R.
Rosl
an,
Z.
Ibra
him,
K.
A.
F.
A.
Sam
ah
,
"A
utomate
d
segm
ent
at
ion
and
det
e
ct
ion
of
T1
-
weight
ed
m
agnetic
r
esona
nce
imaging
bra
in
ima
ges
of
gli
om
a
br
ai
n
tumor,"
Bull
et
in
of
Elec
tri
cal
Engi
ne
ering
an
d
Informatic
s
,
vo
l. 9, no
.
3,
pp.
103
2
-
1037,
June
20
20,
doi
:
10
.
1159
1/e
e
i.
v9i3
.
2079.
[8]
H.
Dibows
ki,
J.
Ploenni
gs,
and
M.
W
oll
schla
e
ger
,
“
Sem
ant
ic
Devic
e
and
S
y
s
te
m
Modeli
ng
f
or
Autom
at
ion
S
y
stems
and
Sen
sor Net
works
,
”
I
EE
E
Tr
ans.
Ind.
Informatic
s
,
201
8,
doi
:
10
.
1109/
TII
.
2018.
27
9686
1
.
[9]
M.
Nara
y
a
na
,
R
.
R.
Redd
y
,
H
yndavi
R
.
N.
,
"H
igh
spee
d
script
execut
ion
for
GU
I
aut
om
at
ion
using
compute
r
vision,
"
in
Inte
r
nati
onal
Journa
l
of
El
e
ct
rica
l
and
Computer
Engi
nee
ring
(
IJECE)
,
vol.
9,
no.
1,
pp.
231
-
236,
Februa
r
y
2019
,
doi
:
10
.
11591/ij
ec
e
.
v
9i1
.
pp231
-
236
.
[10]
H.
Ham
le
dar
i,
B.
McCabe,
an
d
S.
Dava
ri,
"A
utomate
d
computer
vision
-
bas
ed
det
e
ct
ion
of
components
of
under
-
constructi
on
indoor
p
artitions
,
"
Aut
om.
Co
nstr.
,
2017
,
doi
: 10.1016/
j
.
autcon
.
2016.
11
.
009.
[11]
C.
Jitt
awir
i
y
anu
koon,
"P
roposed
al
gorit
hm
for
image
cl
assifi
c
at
ion
using
reg
r
ession
-
base
d
pre
-
proc
essing
an
d
rec
ogni
ti
on
m
odel
s,"
in
Int
ernat
ional
Journal
o
f
El
e
ct
ri
cal
and
Computer
Enginee
ring
(
IJE
CE
)
,
vol.
9,
no
.
2
,
pp.
1021
-
1027
,
April
2019,
doi
: 10.11
591/ijece.
v
9i2
.
pp1021
-
102
7.
[12]
H.
J.
Park
,
K.
B.
Kim
,
"D
ept
h
image
cor
re
ct
i
on
for
intel
realsense
dep
th
c
a
m
era
,
"
in
Indon
esian
Journal
o
f
El
e
ct
rica
l
Eng
ine
ering
and
Computer
S
ci
en
ce
,
vol
.
19,
no
.
2,
pp.
1021
-
1027
,
Augus
t
202
0,
doi
:
10
.
11591/ij
ee
cs.
v19
.
i2
.
pp10
21
-
1027.
[13]
W
.
Dai,
L.
Ril
iskis,
P.
W
ang,
V.
V
y
a
tki
n
,
and
X.
Guan,
"A
Cloud
-
Based
Dec
ision
Support
S
y
st
em
for
Self
-
He
al
ing
in
Distributed
Autom
at
ion
S
y
s
te
m
s
Us
ing
Fault
Tr
ee
Anal
y
sis
,
"
IEEE
Tr
ans.
Ind.
Informati
cs
,
vol
.
14
,
no
.
3
,
pp.
989
–
1000
,
Mar.
2018
,
doi
: 10.1109/
TII
.
201
8.
2791503.
[14]
S.
Huang,
C
.
Zh
ou,
N.
Xiong
,
S. H
.
Yang,
Y
.
Qin
,
and
Q.
Zh
ang,
"A
gene
ral real
-
t
ime
cont
ro
l
appr
oac
h
of intrusio
n
response
for
industri
al
a
utomati
on
s
y
s
te
m
s,"
IEE
E
Tr
ans.
Syst
.
Man,
Cyb
ern.
Syst.
,
2016,
doi:
10
.
1109/T
S
MC.2015.
24696
88.
[15]
A.
F.
H.
Alhar
a
n,
H.
K.
Fatlawi,
and
N.
S.
Al
i
,
"
A
cl
uster
-
ba
sed
fea
tur
e
select
ion
m
et
hod
fo
r
image
te
x
tur
e
cl
assifi
ca
t
ion,
"
in
Indone
sian
Journal
of
El
e
ct
rical
Engi
ne
e
ring
and
Com
pute
r
Sci
en
ce
,
vol.
14,
no
.
3
,
pp.
1433
-
144
2
,
J
une
2019,
DO
I: 10.11591/
i
je
e
cs.
v14.
i3.
[16]
L.
Qiu
,
F.
Yao,
G.
Xu,
S.
Li
,
and
B.
Xu,
"O
utpu
t
fee
dba
ck
guar
a
nte
ed
cost
cont
r
ol
for
net
worked
cont
rol
s
y
stems
with
ran
dom
packet
dropouts
and
ti
m
e
del
a
y
s
in
f
orward
and
fee
d
bac
k
comm
unicati
on
l
inks,"
IEEE
Tr
ans.
Aut
om.
Sci
.
Eng
.
,
2016,
doi:
10
.
1109/T
A
SE.
2014.
235365
7.
[17]
A.
AlQaisi,
M.
AlTa
rawne
h
,
Z.
A.
Alqadi,
A.
A.
Shara
dqah
,
"
Anal
y
sis
of
col
o
r
image
f
ea
tur
es
ext
ra
ction
usin
g
te
xtur
e
m
et
hods,"
in
TEL
KOMNIKA
Tele
com
municat
ion
,
Computing,
Elec
tronic
s
and
Cont
rol
,
vol.
17
,
no
.
3,
pp.
1220
-
1225
,
J
une
2019
,
doi
:
1
0.
12928/telkomnika.
v17
i3.
9922
.
[18]
Y.
P.
Cheng,
C.
W
.
Li
,
and
Y.
C.
Chen,
"A
ppl
y
computer
v
isi
on
in
GU
I
aut
om
at
ion
for
industri
al
appl
i
cations
,
"
Math.
Bi
osci
.
En
g.
,
vol. 16, no. 6, 2019, doi
:
10.
39
34/mbe.
2019378
.
[19]
T.
Yeh
,
T
.
H.
C
hang,
and
R.
C.
Mill
er
,
"S
ikul
i
:
Us
ing
GU
I
scre
e
nshots
for
sea
rc
h
and
au
tomati
o
n,
"
in
UIST
200
9
-
Proce
ed
ings
of
the
22nd
Annual
ACM
Sy
mpos
ium
on
U
ser
Inte
rface
S
oft
ware
and
Technol
og
y
,
200
9,
doi:
10
.
1145/16
22176.
1622211.
[20]
P.
Ram
y
a
,
V.
Sindhura,
and
P.
V.
Sagar
,
"Te
sti
ng
using
sele
n
iu
m
web
drive
r,
"
in
Proceedi
ngs
of
the
2017
2nd
IEE
E
In
te
rnatio
nal
Confe
ren
ce
on
El
e
ct
rica
l,
C
omputer
and
Comm
unic
ati
on
Te
chnol
ogi
es,
ICE
CCT
2017
,
2017,
doi:
10
.
1109/IC
ECCT
.
2017
.
811
7878.
[21]
Y.
P.
Ch
eng,
D
.
L
i
ang,
and
W
.
J.
W
ang,
"K
ORA
T
-
A
pl
at
for
m
inde
pende
n
t
t
est
au
tomati
on
t
ool
b
y
emulat
in
g
ke
y
bo
ard
/mous
e
har
dwar
e
sig
nal
s,"
in
AUT
OT
ESTCON
(
Proce
edi
ngs)
,
O
ct
.
2016,
vol
.
2016
-
Octobe
r,
doi:
10
.
1109/AUTEST.
2016
.
758
9572.
[22]
F.
Nata
lia,
H.
Meidi
a
,
N.
Afril
ia
n
a
,
J.C.
Youn
g,
S.
Sudirm
an,
"Contour
evol
u
ti
on
m
et
hod
for
pre
ci
se
bounda
r
y
del
in
ea
t
ion
of
m
edi
ca
l
imag
es,
"
in
TEL
KOM
NIKA
Tele
com
municat
ion
,
Computing,
Elec
tronic
s
and
Control
,
vol.
18
,
no
.
3
,
pp
.
1621
-
1632
,
Jun
e
2020,
doi
:
10
.
1
2928/T
EL
KO
MN
IKA
.
v18i3.
14746
.
[23]
L.
Bel
h
ade
f
,
Z
.
M.
Maa
za,
"Loss
le
ss
4D
Medic
al
Im
age
s
Com
pre
ss
ion
Us
ing
A
dapt
iv
e
Inte
r
Sli
ce
s
Filt
er
ing,
"
i
n
Inte
rnational
Jo
urnal
of
Adv
ance
s
in
Appl
ie
d
Sci
en
ce
s
(
IJA
AS
)
,
vol.
7,
no.
4,
pp.
361
-
368,
Dec
ember
2018,
doi
:
10
.
11591/ij
aa
s.v7.i4.
pp361
-
368
.
[24]
T.
In
tha
r
ah,
M.
Firm
an,
and
G.
J.
Brostow,
"Rec
urBot
:
Learn
to
au
to
-
co
m
ple
te
GU
I
tas
ks
from
hu
m
a
n
demons
tra
ti
ons,"
in
Confe
renc
e
on
Hum
an
Fac
tors
in
Computing
Syste
ms
-
Proce
ed
ings
,
2018,
doi:
10
.
1145/31
70427.
3188532.
[25]
T.
D.
W
hit
e
,
G.
Fraser,
a
nd
G.
J
.
Brown,
"Im
proving
ran
dom
G
UI
te
sting
with
i
m
age
-
base
d
wid
get
d
et
e
ct
ion
,
"
i
n
ISSTA
2019
-
Pr
oce
ed
ings
of
the
28th
ACM
SIGSO
FT
Inte
rnation
al
Symposium
on
Soft
ware
Testi
ng
and
Anal
ysis
,
2019,
doi
:
10
.
11
45/3293882.
333
0551.
[26]
Z.
Yu,
P.
Xiao
,
Y.
W
u,
B.
Liu,
and
L.
W
u,
"A
Novel
Autom
at
ed
GU
I
Te
s
ti
ng
E
chnol
og
y
Based
on
Im
ag
e
Rec
ognition,"
i
n
Proce
ed
ings
-
18th
IEE
E
In
t
ernati
onal
Conf
ere
nce
on
Hig
h
Pe
rform
ance
Computing
and
Comm
unic
ati
ons,
14th
I
EE
E
Inter
nati
onal
Conf
e
renc
e
on
Smar
t
Cit
y
and
2nd
IE
EE
Int
er
nati
ona
l
Confe
ren
ce
o
n
Data
Scienc
e
an
d
Syste
ms
,
HPC
C/SmartCit
y/
DS
S
2016
,
2017,
do
i:
10
.
1109/HPC
C
-
Sm
art
Cit
y
-
DS
S.2016.
0031.
Evaluation Warning : The document was created with Spire.PDF for Python.