TELKOM
NIKA
, Vol.14, No
.2, June 20
16
, pp. 715~7
2
4
ISSN: 1693-6
930,
accredited
A
by DIKTI, De
cree No: 58/DIK
T
I/Kep/2013
DOI
:
10.12928/TELKOMNIKA.v14i1.3116
715
Re
cei
v
ed Fe
brua
ry 7, 201
6; Revi
se
d
May 3, 201
6; Accepted Ma
y
15, 2016
A Systematic Design towards Usability for Novice
Designers
Hoo Me
ei Ha
o*
1
, Az
iz
ah Jaafar
2
1
Lee Kon
g
Ch
i
an F
a
cult
y of Engi
neer
in
g and
Scienc
e, Un
iv
ersiti T
unku Abdul R
ahma
n
, Mala
ysi
a
2
Institute of Visual Informatics,
Univ
ersiti K
e
b
angs
aa
n Mala
ysia, Mala
ys
ia
*Corres
p
o
ndi
n
g
author, e-ma
i
l
: hoomh
@
utar
.edu.m
y
1
, aziz
a
h
j@ukm.e
du.m
y
2
A
b
st
r
a
ct
Dep
end
enc
e o
n
intu
itive
and
exper
ienc
es
w
ould
a
dd
mo
re co
mp
lexity i
n
the
desi
gn
process
.
Novice d
e
si
gn
ers w
ould foll
o
w
the design of
any exis
tin
g
systems w
i
thout
directio
n. This paper pr
opos
e
d
a
system
atic
des
i
gn approach t
hat pr
ov
ides guidance
for novice desi
g
ners
in achi
eving
the
targeted usability
goa
l us
in
g p
a
tterns, an
d
ev
alu
a
ting
the
p
r
ototypes
in a
c
hievi
n
g
the
d
e
termin
ed usa
b
ility go
als.
T
h
e
prop
osed ap
pr
oach
i
n
vo
lved the
coo
perat
i
o
n of experts i
n
provid
in
g dire
ction to the n
o
vice des
ig
ners
in
achi
eving us
ability g
oal
in the system
des
ign. A coll
ective
decisi
on both fr
om users
and
experts w
o
uld
be
gather to g
i
ve
a clear dec
is
ion of a ch
os
en protot
y
pe. T
w
o experi
m
e
n
ts t
hat follow
ed the pr
opos
ed
appr
oach w
e
r
e
cond
ucted w
i
th 7 gr
oups
or
33 u
nder
gra
d
uate stud
ents.
T
en in
dustry
experts, 10 fi
e
l
d
experts an
d 29
prospectiv
e
us
ers w
e
re involv
ed in ev
al
uat
in
g the ap
propr
ia
te use of the to
ol to assist the
m
i
n
m
a
kin
g
d
e
c
isi
o
n
d
u
r
in
g
com
p
a
r
a
t
i
v
e
p
r
o
t
o
t
yp
e
e
v
a
l
ua
ti
on
. Th
e
re
sul
t
fro
m
b
o
t
h
e
x
p
e
r
im
en
ts co
n
c
l
u
ded
the syste
m
atic
appr
oach
co
ul
d h
e
lp
the
nov
i
c
e d
e
sig
ner
s
to e
m
phas
i
z
e
u
s
abil
i
ty a
l
on
g t
he
desi
g
n
proc
ess.
It
w
a
s observe
d that the achi
eve
m
e
n
t of
important g
oals i
n
their d
e
sig
n
e
d
prototyp
e co
uld b
e
eas
ily d
one i
f
pattern s
e
lecti
on w
a
s
provi
d
ed to
the
nov
ic
e d
e
sig
ners.
T
he
desi
g
n
ed to
ol to
assist
no
vice
desi
g
n
e
rs
has
hig
h
re
lia
bil
i
ty an
d hi
gh
acc
eptanc
e l
e
vel
of ap
pr
opr
iate
of use
in
ma
king
dec
isio
n
for the pr
eferr
e
d
prototype.
Ke
y
w
ords
:
usab
ility go
al; usab
ility patter
n
; desig
n dec
is
ion;
co
mp
arativ
e prototype
ev
alu
a
tion
; a
nalyt
ical
hier
archic
al pr
ocess
Copy
right
©
2016 Un
ive
r
sita
s Ah
mad
Dah
l
an
. All rig
h
t
s r
ese
rved
.
1. Introduc
tion
Experien
c
e
d
desi
gne
rs rely on
the availa
ble
desi
gn
gu
ideline
s
, pa
st de
sign
experie
nces, templates an
d
pro
b
lem
-
so
lution
sets
from
previou
s
experie
nces. This con
c
u
r
s
to
the study clai
med that pra
c
titioners
were
depend
ed
on
their intuition
and experi
e
n
c
e
s
gaine
d [1].
Thus,
novi
c
e
de
sign
ers
woul
d find
difficultie
s in
de
signi
ng i
n
tuitively if experie
nce
and
k
n
owledge are not
s
u
fficiently gained. Subs
eq
uently, further res
e
arc
h
work
s
have been
explore
d
on
how
kno
w
l
edge a
nd e
x
perien
c
e
were u
s
e
d
a
m
ong exp
e
ri
enced an
d novice
desi
gne
rs
su
ch a
s
in [2, 3
]
and [4]. Ref
e
ren
c
e [4]
su
gge
sted that
desi
gn
strate
gies,
kno
w
led
ge,
and inform
ation sho
u
ld be
include
d in developin
g
suppo
rt metho
d
s for novice
design
e
rs a
s
a
guide to them
.
Previou
s
works i
n
[5, 6] e
x
plored th
e u
s
e of g
u
ideli
n
es a
nd p
a
tterns a
s
a
n
aid
to teach
desi
gn. The result from [5]
has shown the use
of design pattern
s
displ
a
yed a g
r
eate
r
impa
ct on
the novice
de
sign
er’
s
pe
rfo
r
man
c
e tha
n
guidelin
es.
We con
s
id
ered
all desi
gn rel
a
ted kn
owl
e
d
g
e
and i
n
form
ation
su
ch
as
desi
gn
de
cisi
on a
nd
de
si
gn p
a
ttern
in
the p
r
o
p
o
s
e
d
meth
od th
at
leadin
g
to the developme
n
t of the tool.
This re
sea
r
ch
aims
to
un
d
e
rsta
nd ho
w the
propo
se
d
assi
stan
ce t
ool an
d meth
od d
r
ive
the novice de
sign
ers in de
signi
ng a usa
b
le syst
em, a
nd there
b
y co
ntribute towards develo
p
in
g a
reliabl
e tool
a
nd meth
od fo
r them
in m
a
king de
sig
n
d
e
c
isi
on, pa
rticu
l
arly in
determining
usabili
ty
goal
s and p
a
tterns, an
d selectin
g the b
e
st prot
ot
ype. This pape
r
wa
s extende
d from [7] wh
ich
inclu
ded
2 e
x
perime
n
ts
condu
cted foll
owin
g t
he p
r
opo
sed
app
roach. The
propo
sed to
ol
wa
s
adoptin
g the
application
of AHP tech
nique in p
r
io
ritizing th
e common u
s
a
b
ility goals an
d
prototype
s
. AHP, a multi-criteria te
chniq
ue, ha
s
be
en
applie
d in va
riou
s di
sciplin
es. Recently, in
engin
eeri
ng it
self, the AHP
techni
que
wa
s ap
plied fo
r the pu
rpo
s
e
of eco
nomi
c
efficien
cy [8] an
d
planni
ng [9].
Rel
e
vant to
the
study, it
is al
so
appl
ied in
p
r
od
u
c
t de
sig
n
su
ch
as in
[10]
. A
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 2, June 20
16 : 715 – 72
4
716
prop
osed ta
sk-m
appi
ng te
mplate
drive
s
the
novice
desi
gne
r to
a
c
hieve
the ta
rgeted
u
s
abili
ty
goal
s. The d
e
sig
n
and de
velopment of
the tool were
to capture use
r
s’ a
nd ex
perts’ d
e
ci
sio
n
in
sele
cting the
best p
r
ototype, thereafter
helpin
g
t
he n
o
vice de
sig
n
e
r to kn
ow th
e best choi
ce
of
prototype ba
sed on the targeted u
s
abilit
y goals.
This p
ape
r is stru
cture
d
a
s
follows. Firs
t, the pape
r
gives an ove
r
view of the p
r
opo
se
d
method an
d tool used in th
is study. Thi
s
is follo
we
d b
y
a descriptio
n
of the experiment an
d d
a
ta
colle
ction me
thods. In the next sectio
n, findi
ngs a
r
e pre
s
e
n
ted
and discu
s
sed. Finally, we
provide the
concl
u
si
on an
d future wo
rk.
2. Rese
arch
Metho
d
In the beginni
ng, we a
c
q
u
ired the pri
o
rity
ran
k
of the u
s
ability goal f
o
r the sy
stem
desig
n
f
r
om
ex
p
e
rt
s.
The re
sult
wa
s se
rv
ed
as a
dire
ction
in t
he
system
de
sign
for the
n
o
vice
de
sign
e
r
s
whi
c
h a
ssi
st them to determine the sel
e
cti
on of patterns on the
sel
e
cted d
e
si
gn
tasks.
Prior to
de
si
gning th
e int
e
rface, novi
c
e
de
sign
ers were cond
ucting
user an
d
task
analysi
s
with
the
pro
s
p
e
ct
ive users, in
relating
to th
e de
sig
ned
system. Base
d on
the
u
s
e
r
s
analysi
s
resul
t, they chose
the impo
rtant
task th
at
the
pro
s
pe
ctive u
s
ers
req
u
ire
d
. The de
sig
n
e
d
tasks
were used in their f
o
cus
in apply
i
ng usability patterns. Th
i
s
was followed by designing an
intera
ctive prototype usin
g a pr
ototyp
e tool. In th
e evaluation
stage, they were
req
u
ired to
arrang
e the
p
o
tential u
s
e
r
s to provide fe
edba
cks on
t
heir de
sign
a
nd sele
ction of
prototype
fro
m
the com
p
a
r
ati
v
e prototype
evaluat
ion. T
he de
ci
sion
s
of usa
b
ility
goal pri
o
riti
zati
on an
d prototype
sele
ction
were assiste
d
by the prop
osed
tool.
The de
sign
of the propo
sed tool was divided into four main module
s
. Th
ere we
re
namely: u
s
ab
ility goal p
r
ioritization, p
r
ot
otype sele
ctio
n to a
ggregat
e all th
e
con
s
istent d
e
ci
sio
n
s
made by u
s
ers
and
exp
e
rts afte
r wa
lk-th
r
ou
gh al
l
prototype
s
, the cal
c
ulatio
n to che
c
k for
con
s
i
s
ten
c
y deci
s
io
n and
determine t
he most in
consi
s
ten
c
y deci
s
ion in th
e matrix for re-
evaluation, a
nd admi
n
ist
r
ation mod
u
le
to setup
project d
e
scri
ption an
d
eval
uators’ p
r
ofil
e of
use
r
s an
d ex
perts.
The
sy
stem
woul
d h
e
lp the
de
sig
ner to
efficie
n
tly kno
w
the
ran
k
in
g of t
he
usa
b
ility goal and determi
ne the protot
ype sele
ctio
n
.
The 3 main activities in the propo
sed
approa
ch in
clude of prio
ritization u
s
a
b
ility goal, desig
ning of use
r
interfa
c
e following
pre
-
determi
ned u
s
ability goal,
and evalu
a
tin
g
and sele
cting the be
st prototype.
Industry exp
e
rts from
sof
t
ware d
e
velo
pment
and fi
eld expert
s
related to intera
ction
desi
gn, usabi
lity, or interface
design
were invite
d to
prioritize the
four
comm
on usability goal
s
based o
n
a
system
de
sig
n
. The
sele
ct
ion of the
ch
ose
n
u
s
abilit
y goal for pri
o
ritizatio
n
wa
s
adopte
d
fro
m
the a
nalysi
s
made
by [11]
that sho
w
ed
learnability, efficien
cy
in use, relia
bility
in
use
an
d subj
ective satisfa
c
tion we
re
th
e
mo
st comm
only
cite
d.
Th
e
meth
od of prio
ritizatio
n
and
sele
cting prot
otype
we
re a
dapted
f
r
om [
12, 13]
called
Analytical
Hi
era
r
chical Proce
s
s (A
HP),
a
multi-criteri
a
analysi
s
a
p
p
r
oach. Fu
rther discu
s
si
on
o
n
the m
e
thod
ology in th
e p
r
ioriti
zation
a
nd
prototype sel
e
ction
can b
e
found in [14]. The result
of the prioritiza
tion wa
s se
rved as a g
oal for
novice d
e
si
gn
ers to target.
Followi
ng ap
proa
ch
es in d
e
sig
n
for u
s
a
b
ility discu
ssed in [15], we
requi
red the
stude
nts
or novice de
sign
ers to follo
w user-ce
n
tered d
e
si
g
n
approa
ch. The de
sign
assignm
ent wa
s
started to un
derstand u
s
e
r
s an
d tasks i
n
the sy
stem
desig
n. The novice d
e
sig
ners gathe
r u
s
e
r
informatio
n a
nd p
e
rfo
r
med
the ta
sk ana
lysis
of
the
system. Du
rin
g
the
develo
p
ment of
de
sign,
the
novi
c
e
d
e
s
ign
e
rs starte
d
with sketch
es or wi
refra
m
e p
r
ototype
and i
m
prove
d
their de
sign
as
they went on
desi
gning th
e
prototype
s
to a highe
r
fid
e
lity. The novice de
sig
ner
were provide
d
a
list of usabili
ty patterns suppo
rted wit
h
some
u
s
a
b
le de
sign solution
s. Based on the gi
ven
template, the
novice de
si
gners foll
owed the
se
lected patterns that match
ed the
targ
e
t
ed
usa
b
ility goal set by the expert in the previou
s
a
c
tivity, for all identified impo
rtant tasks. T
he
usa
b
ility pattern
s served
as a referen
c
e an
d gui
d
e
d
them to achieve the usability goal. The
usability patterns
were adopted from
19 user-perspective
patterns of intera
ction patterns from
[6], 21 a
r
chit
ectural u
s
a
b
il
ity patterns b
y
[16],
and
1
0
fun
c
tional
u
s
ability p
a
ttern from
[17],
are
categ
o
ri
zed i
n
to the 4 co
mmon usabili
ty goal or
usability attributes . Figure 1
shows a pa
rtial
template
scre
en of a ta
sk-pattern m
app
ing for
efficie
n
cy go
al. Th
e task-pattern map
p
ing i
s
to
drive their d
e
s
ign
solutio
n
to meet the targeted u
s
a
b
ili
ty goals.
A matrix of task
scenario
a
nd usability goal was used to a
ssi
st the novice desi
gners to
determi
ne th
e possibl
e usability goal to be desi
gn
e
d
to achieve t
he dete
r
mine
d perce
ntage
of
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A System
atic Design towards Usability fo
r Novi
ce Designer
s (Hoo
Meei Hao)
717
usa
b
ility goal
s. Th
e p
e
rce
n
tage fo
r
ea
ch
usability
goal
relate
d
pattern
that
adopte
d
in
the
desi
gne
d tasks i
s
define
d
as:
∗
100
(
1
)
Whe
r
e
UGP i
s
the
n
u
mbe
r
of patterns rel
a
ted to
a
usa
b
ility goal, IUGP is the
nu
mber of
patte
rns
that ha
s inve
rse
effe
ct wit
h
the
relat
e
d
usability go
al, and
P i
s
total patte
rn
appli
ed i
n
t
he
desi
gne
d tasks fo
r the system.
Figure 1. An example of a partial scree
n
of the task-p
a
ttern map
p
in
g for efficien
cy
This
wa
s
se
rved a
s
a
n
ap
proa
ch
for th
e novi
c
e
desi
gners to
pla
n
their
de
sign.
Table
1
sho
w
s an exa
m
ple of a re
sult of the usa
b
ility goal
fulfillment determ
i
ned by the n
o
vice de
sig
n
e
r
s
in a team. Novice de
sign
ers
woul
d ref
e
r to t
he percenta
ge weig
hting given b
y
the experts via
prio
ritizatin
g
4 main u
s
abil
i
ty goals, and
the ran
k
ing
o
f
usability go
al that determined the lev
e
l of
importa
nce. De
sign
ers ca
n start thei
r work
wi
th the
s
e go
als i
n
m
i
nd and
base
d
on the de
ci
ded
usa
b
ility pattern in all de
sig
ned tasks.
Table 1. Sum
m
ary of usa
b
i
lity goal fulfillment in desi
g
n planni
ng
Usabili
t
y
Goal
Efficie
n
c
y
Learna
bilit
y
Reliabili
t
y
Satisfac
tio
n
Total
pa
ttern
s
Pattern
27
11
26
16
Pattern affected
(
negative effect)
7
0
0
0
7
Net total patte
rn
(positive effect)
20
11
26
16
73
% achieved
27.4
15.1
35.6
21.9
100
Targete
d
(
%
)
23
9.5
58
9.5
100
The novi
c
e
desi
gne
rs
we
re bri
e
fed on
cond
uctin
g
usa
b
ility testing. They ha
ve been
practiced
on conducting us
ability test during the
pr
e-testing
day in order to fam
iliarize
with t
he
step
s and d
a
t
a to collect
durin
g user o
b
se
rvation.
In orde
r to con
duct the p
r
ot
otype sele
cti
on,
invited use
r
s
and expe
rts
were wal
k
-through in
all p
r
e-dete
r
min
e
d
prototype de
sign
s ba
se
d on
the testing scenari
o
in the observato
ry usability
testin
g. Use
r
s a
nd
experts h
ad e
v
aluated 2 an
d
4 p
r
ototypes
respe
c
tively in the
co
mpa
r
ative ev
alu
a
tion. Qu
antitative mea
s
u
r
e
s
su
ch
a
s
tim
e
were taken t
o
com
p
lete t
he tasks, errors
mad
e
an
d com
m
ent
s
from the eval
uators were
also
recorded. Ho
wever, b
a
sed
on the u
s
abil
i
ty measur
em
ents, novi
c
e
desi
gne
rs m
a
y find difficult to
deci
de
whi
c
h best prototype to
sel
e
ct f
o
r further improvement.
We extended t
he
short usability
testing
and
wal
k
-th
r
ou
gh
de
sign to
le
t the eval
uat
ors jud
ged
their
de
cisi
on
on the
prototype
selection using AHP.
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 2, June 20
16 : 715 – 72
4
718
The m
a
jo
rity of the
use
r
s
were
not fami
liar
putting th
eir
prefe
r
en
ces
scale
in
th
e AHP’
s
matrix. We p
r
ovided a
gui
ded dial
ogue
for the eval
u
a
tors to ma
ke their de
ci
si
on in a de
sig
ned
tool. Figure
2 sho
w
s a
guide
d dialo
gue of pai
r-wise co
mpa
r
i
s
on
scale fo
r 4 prototype
s
in
efficien
cy in use u
s
ing the
design
ed to
ol. Beside
s calcul
ating for
con
s
i
s
ten
c
y ratio, the system
wa
s al
so
che
c
ked fo
r in
co
nsi
s
ten
c
y rati
o, whi
c
h
wa
s more
than
0.10. Inco
nsi
s
t
ency
ratio fo
und
in a deci
s
ion
matrix will allow the evalu
a
tor to re
-eva
luate their de
cisi
on
s made.
This is ba
se
d on
the discu
s
sio
n
on the Sa
a
t
y’s theory o
n
sati
sfyi
ng consi
s
ten
c
y m
a
trix in [12]. We a
dopte
d
the
theory a
nd i
d
entified three
location
s o
n
the matrix
to
evaluate
ba
sed o
n
the
hig
hest
differen
c
es
value betwee
n
the use
r
preferen
ce
s scale and the
satisfacto
ry value from the
con
s
i
s
tent pa
ir-
wise comparis
on
matrix. After c
o
mp
letin
g
re
ceiving
al
l the evaluat
ors’
de
cisi
on,
the co
nsi
s
te
nt
evaluators’ d
e
ci
sion result will be agg
regated
a
nd the ch
osen p
r
ototype will
be determine
d
based
on the
highe
st total
weig
hted eval
uation. Thi
s
i
s
a
c
hi
eved b
y
multiplying t
he fa
ctor
wei
ght
for each usab
ility goal with factor ev
alu
a
tion for all prototype desi
g
n
s
.
Figure 2. A guided di
alogu
e to assi
st an
expert
made
deci
s
io
n on the evaluate
d
prototype in a
desi
gne
d tool
3. Rese
arch
Metho
d
In unde
rstan
d
ing the
pro
posed a
p
p
r
o
a
ch,
we
we
re co
ndu
cting
2 expe
rime
nts with
private institu
t
ion stude
nts
who
we
re taking the subj
e
c
t of desig
nin
g
use
r
interfa
c
e. They we
re
stude
nts i
n
Y
ear
2
softwa
r
e en
ginee
rin
g
pro
g
ramme
.
Figure 3
give
s a
co
mbin
ation of a
n
ove
r
all
flow of the proposed app
roach to driving novice designers
design towards usability, in both
experim
ent d
e
sig
n
s.
In ord
e
r to
un
derstand
the
usa
ge of th
e
pr
op
osed
ap
proa
ch
an
d tool, the
stude
nts-cum
-
novice de
sig
ners we
re re
quire
d to follow a use
r
-ce
n
tered a
pproa
ch in desig
nin
g
use
r
interfa
c
e
s
for stu
dent’
s
orga
nization
manag
eme
n
t system
ba
sed o
n
the
gi
ven ge
neral
descri
p
tion.
Data
colle
ction
of the effect of t
he u
s
ag
e of the
propo
se
d
approa
ch a
n
d
tool were
condu
cted fo
r
all
partici
pant
s i
n
volved in th
e experim
ent
s. Throug
hou
t the implem
entation of the expe
rime
nts,
conte
n
t analy
s
is
wa
s cond
ucted
on de
sign dia
r
ie
s a
nd group
de
sign
refle
c
tio
n
written
by the
novice
de
sig
ners, an
d o
n
prototy
pe
de
signs to a
s
se
ss the
u
s
ed
of
pattern
s.
De
sign
dia
r
ie
s a
nd
grou
p de
sig
n
reflectio
n
were coll
ect
ed to unde
rstand the p
r
oblem
s face
d by the novice
desi
gne
rs fol
l
owin
g the
p
r
opo
se
d a
p
p
r
oa
ch. F
u
rt
h
e
rmo
r
e, eval
uation
of the
tool
wa
s al
so
con
d
u
c
ted a
m
ong u
s
e
r
s
and exp
e
rts
who i
n
volved
in the comp
arative evalu
a
tion prototype.
Table 2 sho
w
s the su
mma
ry of data collecti
on meth
o
d
con
d
u
c
ted i
n
the experim
ents.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A System
atic Design towards Usability fo
r Novi
ce Designer
s (Hoo
Meei Hao)
719
Figure 3. Overall flow of the prop
osed a
ppro
a
ch for u
s
er inte
rfa
c
e
desi
g
n
Table 2. Sum
m
ary of data colle
ction met
hod
s
Participa
n
ts
Metho
d
/
a
n
al
y
s
i
s
Purpose
Novice designers
Content anal
y
s
is:
Grou
p design refl
ection
Individual’s desig
n diaries
To evaluate the
approp
riateness of the prop
osed
approach in designing the prot
ot
ypes.
Prospective users and expe
rts
(field and industr
y e
x
perts)
Quantitative and
qualitative
sur
v
ey
To evaluate the
approp
riateness of the prop
osed
tool assist
ing their decisions.
Protot
ypes designed b
y
novices
Content anal
y
s
is:
Review
th
e application of
patterns in the p
r
otot
y
pes
To evaluate the
pattern usage in
fulfilling the
goal.
Two expe
rim
ents were
co
ndu
cted in de
signi
ng t
he u
s
er inte
rfa
c
e
of different sy
stem
s in
different sem
e
ster. In the
first expe
rime
nt, there were 4 group
s, in whi
c
h 6 m
e
mbe
r
s in
ea
ch
grou
p, involved in the user interfa
c
e d
e
sig
n
for a student com
m
unity system
for a universi
t
y.
Whe
r
ea
s i
n
experim
ent 2
,
a total of
1
0
novi
c
e
de
signers
we
re
grou
ped
in
3
,
involved in
the
desi
gn of user interfa
c
e for man
aging
statione
ry
. The differen
c
e
in the numb
e
r of partici
p
ants
involved in th
e expe
riment
s de
pen
ds
o
n
the nu
mb
e
r
of stu
dent
s in the
se
con
d
year
software
engin
eeri
ng t
a
kin
g
the
de
signi
ng u
s
e
r
interface u
n
it in the
sem
e
ster. Both
experim
ents foll
ow
the same
ap
proa
ch
an
d t
ool a
s
d
e
scribed in
se
cti
on 2,
except
the d
e
ci
sio
n
for
sel
e
cti
ng
approp
riate p
a
tterns.
Novi
ce d
e
si
gne
rs in exper
i
m
e
n
t 1 we
re freely de
cided
the app
rop
r
i
a
te
patterns for t
he identifi
ed
desi
gned tasks and mat
c
hed the ta
rget
ed usability goal
set from t
h
e
result of usa
b
ility goal pri
o
ritizatio
n
. While nov
ice d
e
sig
ners in e
x
perime
n
t 2 followed the
pre
-
determi
ned p
a
tterns th
at deeme
d
app
ropriate fo
r
the desi
gne
d tasks. Th
e p
a
tterns
used
in
experim
ent 2 were de
cide
d
by the resea
r
che
r
.
4. Results a
nd Discu
ssi
on
Data
of seve
n g
r
oup
s of
novice
de
sig
ners
from t
w
o expe
rime
nts
we
re
colle
cted. Ten
indu
stry expe
rts, 10 field
experts, a
n
d
29 pro
s
p
e
cti
v
e use
r
s fro
m
experim
en
t 1 or su
rrog
ate
use
r
s f
r
om ex
perim
ent 2 were involve
d
in eval
uating t
he tool. Evaluation of the a
ppro
p
ri
atene
ss
of the propo
sed method a
n
d
tool are
di
scu
s
sed in the
following
se
ction.
4.1. Appropr
iatene
ss of
the Propose
d
Approa
ch in Designing the Proto
t
y
p
e
s
Submissio
n
s
of individual
diary log a
n
d
grou
ps’ d
e
si
gn refle
c
tion
were colle
cte
d
and
each
conte
n
t
wa
s a
nalyze
d
.
Similar g
r
ou
ps’ th
eme
th
a
t
discu
s
sed
a
nd hi
ghlighte
d
by the
novi
c
e
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 2, June 20
16 : 715 – 72
4
720
desi
gne
rs in t
he written lo
g
s
and
repo
rts, was
summa
rize
d in Tabl
e
3. Based on
the re
sult, the
sub
s
tantial p
r
oble
m
s occur were
difficult
in
u
s
in
g
prototyping
tool and
difficult in a
c
hi
e
v
ing
targeted
ran
k
goal usi
ng t
he usability goal fulfillm
e
n
t. The difficulty in using
prototyping
tool
occurred because novices
took some ti
me to familiar with the
tool
to compl
e
te their prototypes.
Ho
wever th
e
probl
em
s were red
u
ci
ng aft
e
r 2 h
o
u
r
s
ha
nds-on
and
d
e
mon
s
tratio
n
wa
s cond
ucte
d
in the exp
e
ri
ment 2 i
n
o
r
der to
famili
arize the
fun
c
tion
s of the
tool p
r
ovide
d
. Similarly,
the
probl
em of
achi
eving targeted ran
k
goal was
re
duci
ng trem
e
ndou
sly after the deci
s
io
n of
sele
cting p
a
ttern
s was
given to novice
desi
gne
rs
as
a guide i
n
the
experim
ent 2
comp
are to the
s
a
me
pr
ob
lem w
a
s
r
e
po
rte
d
mo
r
e
th
an
3
0
%
in
experim
ent 1. No aid was
given to novice
desi
gne
rs in
experim
ent 1
to determin
e
the pa
ttern
s for ea
ch d
e
sig
ned ta
sks. They have
to
deci
de them
selves the usability pa
ttern that would appropri
ate
for the designed tasks. These
had
sh
owe
d
t
hat gui
de to
sele
ct p
a
ttern
s u
s
e
d
for th
e de
sign
ed ta
sk was nee
d
ed o
n
top
of the
usa
b
ility goal fulfillment.
Ho
wever, de
sign follo
win
g
usability p
a
tterns h
ad
given the no
vice
desi
gne
rs
an
approa
ch to
trigge
r idea
s
in their
d
e
si
g
n
. This h
ad b
een repo
rted
by 3 gro
u
p
s
i
n
both experi
m
ents.
Table 3. Nu
m
ber of theme
s
discusse
d in the r
epo
rts a
nd log
s
for experim
ent 1 an
d experim
ent 2
The
me
Experime
nt 1
Experime
nt 2
Total
Grou
p
1
Grou
p
2
Grou
p
3
Grou
p
4
Grou
p
1
Grou
p 2
Grou
p 3
A B
A B
A
B
A B
A
B
A
B
A
B
Protot
yping
tool
2 1
2 0
5
0
3 0
2
1
4
0
3
0
23
Difficult to
achieve
targeted rank
goal
2 0
3 0
1
0
0 1
3
1
0
0
0
0
11
Easy
t
o
trigge
r
design idea
4 0
0 0
0
1
0 0
0
0
0
0
5
0
10
Changes of
design
impacted
protot
ype
0 0
0 0
0
0
0 0
1
0
0
0
0
0
1
Difficult to get a
convinced
design
0 0
0 0
0
0
1 0
0
0
0
0
0
0
1
Total
8 1
5 0
6
1
4 1
6
2
4
0
8
0
46
Gui
d
e
:
A: data obtained
from novice designers’ diaries
B: data obtained
from gro
ups’ design reflection reports
4.2. Appropr
iatene
ss of
the Propose
d
Tool
Assists Ev
aluators
’ Making Dec
i
sions
A total of 49 returne
d
and
complete form
s we
re u
s
ed t
o
analyze the
desig
ned too
l
for its
approp
riaten
ess in u
s
ag
e and its
u
s
ability. 29
resp
ond
ents
or ap
proxim
ate 59% of total
respon
dent
s were
fro
m
u
n
iversity stud
ents who
be
came
the p
r
osp
e
ctive u
s
ers or surrog
ate
use
r
s i
n
eval
uating the p
r
ototypes d
u
ri
ng u
s
abilit
y testing
and p
r
ototype sel
e
ction pro
c
e
s
s
usin
g
the desi
gne
d
tool. The de
sign
ed tool was u
s
ed to
a
ssi
st evaluat
ors
and exp
e
rts to ma
ke the
deci
s
io
n in
the
com
parative evaluatio
n p
r
otot
ype and agg
reg
a
t
e
all con
s
i
s
tent
de
ci
sion
s
t
o
provide
an
a
b
sol
u
te sele
ction re
sult, a
s
Ta
ble
4.
Approximatel
y 20% or
10
re
spo
ndent
s for
each ind
u
stry
expert
s
an
d
field expe
rts are i
n
vo
lved
in the eval
u
a
tion. Fou
r
q
uestio
n
s
and
3
que
stion
s
we
re de
sig
ned t
o
evaluate th
e app
rop
r
ia
te
use
and
usa
b
ility of the tool re
spe
c
tivel
y
.
Que
s
tion
s rel
a
ted to the usag
e of the tool co
m
p
ri
se
d of 1 binary que
stion and
3 questio
n
s o
f
5-
point Li
ke
rt scale. A
bina
ry typed qu
est
i
on
wa
s re
lat
ed to th
e mat
c
hin
g
of i
ndiv
i
dual’
s
de
ci
si
on
with the de
ci
sion
sug
g
e
s
ted by the de
sign
ed tool
. W
h
er
e
a
s
qu
es
tio
n
s
in
5
-
po
in
t L
i
k
e
r
t
sca
le
were rel
a
ted
to the use of
the system i
n
prot
otype
selectio
n de
ci
sion
-ma
k
in
g, need of h
e
lp
in
usin
g the tool
, and comfort
ablen
ess to reco
nsi
der
th
e
i
r incon
s
iste
n
t
deci
s
ion
s
. Result
related t
o
the u
s
ag
e
a
ppro
p
ri
atene
ss of
the
de
si
gned
tool
wa
s
sho
w
n
in
Figure 4.
Th
ree q
u
e
s
tion
s to
asse
ss th
e u
s
ability of the
tool com
p
ri
sed of
suffici
e
n
t and un
derstandi
ng the
que
stion
s
asked
as well as learnability of the tool. The result of
the usability of the tool
was shown in Figure 5.
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A System
atic Design towards Usability fo
r Novi
ce Designer
s (Hoo
Meei Hao)
721
Table 4. Que
s
tionn
aire Structure
Purpose of
questions
H
y
poth
e
sis Questions
T
o
assess the
approp
riate use
- The
result gene
rated b
y
the desi
gned tool is matched
w
i
th the r
e
sult of
an evaluator’s decision.
Q1
- The d
e
signed t
ool assist
s evalu
a
tors to make de
cision in the preferred
protot
ype.
Q2
- Decision could not make
w
i
thout
the help of the t
ool.
Q7
- It is not uncomf
o
rtable to re
eval
uate inconsistent decisions.
Q6
T
o
assess the
usability
of tool
- The q
uestions asked b
y
the
tool
are unde
rstanda
ble.
Q3
- The q
uestions are sufficient to
determine the deci
s
ions made.
Q4
- Evaluator
w
oul
d not need h
e
lp in making decisio
n to use the s
y
st
em.
Q5
Figure 4. Survey result on
approp
riate u
s
e of the tool
The de
sign
e
d
tool has hi
gh relia
bility as 96% o
r
4
7
re
spon
dent
s stated thei
r deci
s
ion
prio
rities were eq
ual
with t
he
re
sult g
e
n
e
rate
d
by th
e
de
sign
ed to
o
l
. The
de
sign
ed tool
ha
s hi
gh
usa
ge a
p
p
r
o
p
riaten
ess i
n
terms of its a
ssi
stan
ce
i
n
helpin
g
evalu
a
tors to ma
ke de
cisi
on b
a
s
ed
on the perce
ption of evaluated us
ability goals. A total of 86% of
respon
dent
s were ag
reed
th
e
desi
gne
d tool
coul
d hel
p them to ma
ke
deci
s
io
n
in choo
sing
th
e prefe
rre
d
de
signed prototy
pe.
In contra
ry, 51% of the res
pon
dent
s agre
ed that they could
e
a
sily kn
ow
whi
c
h preferred
prototype to
cho
o
se with
o
u
t the hel
p of
the tool
co
m
pare
to o
n
ly
20% in
nee
d
of help f
r
om t
h
e
tool. This
wa
s be
cau
s
e th
ey would
use
their visu
al
impre
s
sion t
o
sele
ct
t
heir preferred
p
r
otot
ype
in whi
c
h thi
s
wa
s not the p
u
rpo
s
e
of the
study.
A lot of evaluators’
commente
d
fo
r improveme
n
t
s
related
to th
e ae
stheti
c
v
a
lue
s
in
the
desi
gn th
at compri
sin
g
col
o
rs,
graphi
cs, and
pictu
r
e
s
.
Thus,
we
beli
e
ve the meth
od in the to
ol
woul
d a
ssi
st
them to jud
g
e
their
de
cisi
on mainly b
a
s
ed
on the usabili
ty goals. More than
7% of the resp
ond
ents did n
o
t sa
tisfy with their in
con
s
i
s
tent
deci
s
io
n to
be
re-evaluat
ed
com
pare
to only
30%
re
sp
ond
ents ag
ree
d
to
reevaluate
th
eir
inco
nsi
s
tent
d
e
ci
sion
s.
We
find that thi
s
i
s
n
e
cessa
r
y i
n
o
r
de
r to
ha
ve co
nsi
s
tent
de
cisi
on
s to
be
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 2, June 20
16 : 715 – 72
4
722
aggregate
d
to give a reli
able ab
solut
e
result
in selectin
g the prototype du
ring com
p
a
r
at
ive
evaluation.
The de
sig
n
e
d
tool also h
a
s hig
h
u
s
abi
lity
in terms
of unde
rsta
n
dable of the
que
stion
s
asked,
with
92% of th
e t
o
tal resp
ond
ents, a
nd
su
fficient of the
que
stion
s
(69% of th
e t
o
tal
respon
dent
s).
Neverth
e
le
ss, about 53%
of the tota
l re
spo
nde
nts ne
ed help i
n
u
s
i
ng the tool.
We
observed
that
2 expe
rts we
re h
a
ving
difficultie
s to
re
consi
der their i
n
co
nsi
s
t
ent
d
e
ci
sion
s.
Th
e
y
informe
d
that
they did n
o
t
kno
w
what
n
eed
s to
b
e
chang
ed to
achieve a
co
nsi
s
tent result. This
explained
the
high f
r
eq
uen
cy of hel
p
wa
s n
eede
d in
usin
g the to
ol
. Ho
wever, th
e tool p
r
ovid
es
the flexibility to chang
e
the evaluato
r
s’ d
e
ci
si
on
s
according
to
the qu
estio
n
s
aske
d o
r
to
maintain th
ei
r in
con
s
i
s
ten
t
deci
s
io
ns.
Detail
ed a
ssi
stan
ce
to
help
to a
c
hieve
con
s
i
s
tent
deci
s
io
ns d
o
e
s
not re
pre
s
e
n
t the deci
s
io
n from an eva
l
uator.
Figure 5. Survey result
on
the usability of the tool
4.3. Ev
aluation of the
Us
age of
Usabi
lit
y
Pattern
Figure 6 summari
zes the
usability patterns
use i
n
the desi
gn of a protot
y
pe in each
novice d
e
sig
ner’
s
group i
n
achi
eving the targete
d
p
r
iority of usa
b
ility goal. All
group
s coul
d not
meet the targeted p
r
iority
of the usa
b
i
lity goal
. All
4 gro
u
p
s
ha
ve more p
a
ttern
s related
to
satisfa
c
tion t
han lea
r
na
bil
i
ty. Among all group
s, onl
y one gro
u
p
(gro
up 1
)
h
a
s a
c
hieve
d
the
highe
st prio
rity in patterns relate to efficiency
in use. Whil
st other
3 grou
ps focuse
d on pattern
s
related to
rel
i
ability in use more than
other goal
s.
This result
occurred
as novice
designers
indep
ende
ntly cho
s
e the
pattern
s relat
e
to their d
e
s
ign
ed tasks.
Improveme
n
t
s we
re ma
d
e
in
the methodol
ogy of choo
si
ng usability patterns
we
re
con
d
u
c
ted an
d evaluated i
n
experim
ent 2.
Figure 6. Percenta
ge of usability pattern
app
lied in p
r
ototype desi
g
n for experim
ent 1
Evaluation Warning : The document was created with Spire.PDF for Python.
TELKOM
NIKA
ISSN:
1693-6
930
A System
atic Design towards Usability fo
r Novi
ce Designer
s (Hoo
Meei Hao)
723
Selection
s
of
usability pat
terns related
to the
de
sig
ned ta
sks
were
revie
w
e
d
to d
e
termi
n
e its
relevan
c
y a
s
well a
s
in a
c
hieving the ta
rgeted
usabili
ty goal. We h
a
ve put emp
hasi
z
in
g on t
h
e
u
s
e o
f
pa
tte
r
n
for
th
e de
s
i
gn
e
d
ta
sks
th
a
t
w
o
u
l
d b
e
e
v
a
l
u
a
t
e
d
in us
a
b
ility te
s
t
in
g
and
comparative prototype
ev
aluation. Table 5 shows the su
mmari
zation of the planned usability
pattern an
d the implem
ent
ation of the target
ed p
r
io
rity of usability goal in experi
m
ent 2.
Re
sult in
exp
e
rime
nt 2 fo
und th
at not
all
group
s
ca
n completely
meet e
a
ch t
a
rgete
d
perc
entage of pattern in their
des
i
gned prototype
as
planned.
Novic
e
des
i
gners were new
and
requi
re time to maste
r
their skill
s in the con
s
tru
c
tion
of design
ed
prototype u
s
i
ng the tool. Two
grou
ps,
whi
c
h we
re g
r
o
u
p
1 and
2, co
u
l
d pla
c
e the
hi
ghe
st pri
o
rit
y
target of rel
i
ability in use
in
their de
sign
e
d
prototype with
the achi
evement of 24 pattern
s and 20 patte
rns respe
c
tively.
Whil
st group
3 ha
s 1
patte
rn diffe
ren
c
e
betwe
en th
e
goal
s of
relia
bility in use a
nd effici
en
cy in
use. T
he mai
n
finding from the improvement of
the usage of usability pattern in experi
m
ent 2
wa
s the a
c
hi
evement of importa
nt goa
ls in t
heir d
e
s
ign
ed p
r
otot
ype coul
d be
easily don
e
if
pattern sele
ction wa
s pro
v
ided to the novice d
e
sig
ners rath
er than ind
epen
dently deci
d
e
d
by
them.
Table 5. Total
and perce
nta
ge of usa
b
ility pattern
u
s
ag
e in prototype
desig
n for experim
ent 2
No
v
i
ce
desig
n
er
grou
ps
Pattern
s relate
d
to u
sabili
t
y
g
o
a
l
Total
patt
ern
related to
other
patt
erns
Total
net
patt
ern
Efficie
n
c
y
in
use
(to
tal/ %
)
Learna
bilit
y
(to
tal/ %
)
Reliabili
t
y
in
us
e
(to
tal/ %
)
Satisfac
tio
n
(to
tal/ %
)
Target
achie
v
e
men
t
(%
)
23.0
%
9.5
%
58.0
%
9.5
%
SG
1
(planned)
20
29.3%
11
15.1%
26
35.6%
16
21.9%
73 34
SG
1
(designed)
21
28.8%
9
12.3%
24
32.9%
19
26.0%
73 34
SG
2
(planned)
21
31.8%
8
12.2%
25
37.9%
12
18.2%
66 31
SG
2
(designed)
19
35.2%
6
11.1%
20
37.0%
9
16.7%
54 25
SG
3
(planned)
20
23.5%
11
15.1%
26
35.6%
16
21.9%
73 34
SG
3
(designed)
19
32.2%
9
15.3%
18
30.5%
13
22.0%
59 25
Re
sult from t
he a
nalysi
s
o
f
the de
sig
n
e
d
p
r
ototypes and com
p
a
r
ison with
the
p
l
anne
d
pattern fo
und
that som
e
pa
tterns
we
re n
o
t desi
gne
d i
n
a p
r
ototype
of a group
bu
t were de
sign
ed
in anothe
r p
r
ototype of an
other g
r
o
up. Inequ
ality of the implem
ent
ation of u
s
abi
lity pattern in
the
desi
gne
d p
r
o
t
otype with t
he pla
n
wou
l
d de
rive
so
me rea
s
on
s.
One
of the
m
wa
s
novi
c
e
desi
gne
rs m
a
y not al
way
s
refer to th
e
patterns
in t
he pl
an
whil
e
implem
entin
g the
prototype.
Other rea
s
on
s may incl
ud
e novice d
e
si
gners could
not dominate
the skill in u
s
ing the tool
to
develop the p
r
ototype follo
wing the pla
n
ned patterns.
Novice d
e
sig
ners in group
1 were a
b
le
to
reali
z
e the i
m
portan
c
e
of trade-off usa
b
il
ity goal
follo
wing th
e targ
eted pri
o
rity i
n
usability go
al
.
Some patte
rns that d
eem
ed relevant t
o
the de
sig
n
ed tasks were relat
ed to
efficien
cy in
use,
reliability in
use,
and
sat
i
sfactio
n
. Th
ese
patte
rn
s we
re
ad
ded
in th
eir
de
signed
p
r
ototype
without affecti
ng the po
sitio
n
of
highe
st priority in usab
ility goal.
5. Conclusio
n
Achieving
bal
ance of
usabi
lity goal i
s
im
portant
in th
e
de
sign
p
r
oce
s
s towards u
s
ability.
Furthe
rmo
r
e, novice de
sig
n
e
rs
woul
d not follow b
lindly the design from any
existing system
s if a
method
of a
c
hievin
g u
s
a
b
ility along d
e
sig
n
p
r
o
c
e
s
s is impo
se
d
.
Theref
ore, the
sy
stemat
ic
approa
ch to
desi
gn
ba
se
d on
u
s
abilit
y pattern
s
di
scusse
d in
this
pap
er
be
come
the
de
sign
rational
e an
d guide to
ward
s achievi
ng the targ
e
t
ed usa
b
ility goals. AHP
techniq
ue
wa
s
cu
stomized i
n
the
desi
gn
pro
c
e
s
s to a
s
sist i
n
co
lle
cting d
a
ta p
r
iori
tization
of u
s
ability goal
s
an
d
prototype
s
.
The pro
p
o
s
e
d
meth
od ha
d
sho
w
n a co
operation of
experts in
providing
dire
ct
ion to t
h
e
novice
de
sig
ners in a
c
hi
e
v
ing usability requi
rem
ent
in the sy
stem
desi
gn. It provides
a way to
Evaluation Warning : The document was created with Spire.PDF for Python.
ISSN: 16
93-6
930
TELKOM
NIKA
Vol. 14, No. 2, June 20
16 : 715 – 72
4
724
novice designers
how trade-off of usability pattern
is ma
de in fulfilling the ta
rgeted priority of
usa
b
ility goal
s. However,
guida
nce to
select
app
rop
r
i
a
te pattern n
eed
s to
com
e
from in
du
stry
or
field expert to ensure it
s suitability and releva
ncy of chosen patterns
with the designed tasks
and a
c
hieve
m
ent of usab
ility goal. In d
e
cidi
ng the
b
e
st prot
otype in fulfilling the usa
b
ility goal
s
durin
g the co
mparative evaluati
on p
r
o
c
ess, it had shown a colle
ctive deci
s
ion
both from users
and
expert
s
. This could
give a
clea
r de
cisi
on
of
a p
a
rticular prototype
was
ch
osen t
hat
sup
porte
d wit
h
the weig
ht of pr
eferen
ce
of each u
s
ab
ility goals.
The re
sult fro
m
both expe
riments
con
c
l
uded the
pro
posed sy
ste
m
atic ap
pro
a
c
h u
s
ing
usa
b
ility pattern
s to
achi
eve u
s
ability
goal
s
co
uld
help
the
no
vice d
e
si
gne
rs t
o
em
pha
size
usability in designing. Novice de
signers found that
the usability patterns coul
d easily trigger
desi
gn idea
s.
The desig
n
ed tool has
high reli
ab
ility due to high percenta
g
e
of similarity of
deci
s
io
n pri
o
rity with the individual
evaluators’
de
ci
sion. It also h
a
s hi
gh a
c
ce
ptance level
of
approp
riate o
f
use
in
deci
s
ion-m
a
ki
ng fo
r the
pr
eferre
d prototype a
nd hi
gh a
c
ce
ptance level f
o
r
usa
b
ility in terms
of und
erstand
able
an
d suffici
ent
q
uestio
n
s. Su
pport
s
fro
m
the tool to
col
l
ect
deci
s
io
n data
and to gui
d
e
evaluato
r
s
to rea
s
sess t
heir in
co
nsi
s
t
ency de
ci
sio
n
to achi
eve
the
validity result
of the sel
e
cti
on of prototype were
also
incl
u
ded
alth
ough evaluat
ors may
find the
difficulty in a
c
hievin
g co
n
s
iste
ncy in th
eir de
ci
sion
s.
The sy
stem
atic de
sig
n
p
r
ovide
s
gui
da
nce
for novice de
sign
ers in p
r
i
o
ritizin
g
the u
s
abilit
y goal,
achi
eving the
targeted
usa
b
ility goal usi
n
g
pattern
s, a
n
d
evaluatin
g t
he p
r
ototype
s
in
a
c
hi
eving the
u
s
abilit
y goal
s. Mo
re stu
d
ie
s
will
be
requi
re
d to determin
e
if the app
roa
c
h
could hel
p in
d
e
livering g
o
o
d
quality
of prototype that has
less number
of iteration and less
number of usability
probl
em
s found.
Referen
ces
[1]
Rogers Y. NE
W
T
H
EORE
T
I
CAL APPROA
CHES FOR HCI.
Annua
l Re
view
of Information Sc
ienc
e
and T
e
ch
no
log
y
. 2004; 38: 87
-143.
[2]
Waldro
n MB, Jelin
ek W, O
w
e
n
D, Waldro
n
KJ.
A study of visual r
e
cal
l
dif
f
erences betw
een experts
and n
a
iv
e desi
gners
. Intern
ati
ona
l Conf
erenc
e of Engin
eer
in
g Desi
gn (ICE
D 87). 198
7.
[3]
Kavalki M, Ge
ro JS.
Strategic Know
ledg
e Differenc
es be
tw
een an Exp
e
rt and a Novi
ce Desig
ner
.
Huma
n Beh
a
vi
our in D
e
sig
n
Indivi
du
als, T
eams,
T
ools. Berlin, Sprin
ger. 20
03: 42-5
2
.
[4]
Ahmed S, W
a
llac
e
KM. U
ndersta
ndi
ng t
he d
i
fferenc
es
bet
w
e
en h
o
w
n
o
vic
e
a
n
d
experi
enc
ed
desi
gners a
ppr
oach d
e
si
gn ta
sks.
Research in
Eng
i
ne
eri
ng Desig
n
. 20
03; 14: 1-11.
[5]
Kouko
u
lets
os
K, Khaza
e
i B,
Dear
den A, O
zcan
M. T
eaching
Usab
ilit
y P
r
incip
l
es
w
i
th
Patterns a
n
d
Guide
lin
es. In:
Kotzé P, W
o
ng W
,
Jor
ge
J, Di
x A, Si
lv
a P.
Editors
. Creativit
y an
d
HCI:
From
Exp
e
ri
ence to
Desig
n
in Ed
uc
ation. Spri
ng
er US; 2009: 1
59-
174.
[6]
W
e
lie M V
an,
Veer GC V
an
Der, Eli
ëns A.
Patterns as T
ools for Us
er In
terface Des
i
g
n
.
T
ools for
W
o
rking w
i
th Guid
eli
nes
. 20
01
: 313-32
4.
[7]
Hao HM, Jaaf
ar A.
Novice
Assistance T
o
ol an
d Metho
d
o
lo
gy: Desig
n
Decisi
on a
nd
T
a
sk-Pattern
Mapp
ing
. Proc
eed
ing
of the Electrical E
ngi
ne
erin
g
Comp
ute
r
Science a
nd I
n
formatics. 20
15; 2.
[8]
Ci T
,
Liu
X. T
he Stud
y
of Sel
e
ction
of Co
al
F
i
red Su
pp
lier
i
n
T
hermal Po
wer Enterpr
i
se B
a
sed
on t
h
e
Exte
nsio
n An
al
ysis
Metho
d
.
T
E
LKOMNIKA Indo
nesi
an J
o
u
r
nal
of Electric
al En
gi
neer
in
g
. 201
3; 1
1
:
387
4-38
85.
[9]
Shen
g W
,
Zha
ng
L, T
ang W
,
W
ang
J, Fan
g
H.
Optima
l M
u
ltiDistri
bute
d
Generators
Pl
a
nni
ng
Un
der
Uncerta
i
nt
y
usi
ng AHP an
d GA.
T
E
LKOMNIKA Indones
ia
n
Journa
l of Ele
c
trical Eng
i
ne
e
r
ing
. 20
14;
12: 258
2-2
591.
[10]
Ding
l
e
i
W
,
Jian
yon
g
L, W
ens
hen
g X. T
he Methods of F
a
ctor W
e
i
ght’s Determin
e
in t
he Process of
Cluster.
T
E
LK
OMNIKA Indon
esia
n Journ
a
l o
f
Electrical Eng
i
ne
erin
g
. 201
2; 10: 113
7-11
41
.
[11]
Folmer E, Gur
p
J Va
n, Bosc
h J. A frame
w
or
k for ca
pturi
ng th
e rel
a
tio
n
s
hip
bet
w
e
en
usab
ilit
y
and
soft
w
a
re architecture.
Softw
are Process.
2
003
.
[12]
Saat
y T
L
. T
he Ana
l
y
t
ic H
i
era
r
ch
y Proc
ess:
Plan
nin
g
, Pri
o
r
i
t
y
S
e
tting,
Re
source
All
o
cati
on. McGra
w
Hill Inter
natio
n
a
l. 198
0.
[13]
Saat
y T
L
. Decision
maki
ng
w
i
t
h
the
an
al
ytic hier
arch
y
p
r
ocess.
Intern
ation
a
l J
ourn
a
l
of Service
s
Scienc
es
. 200
8; 1: 83.
[14]
Hao HM, Jaaf
ar A. Goal prioritizati
on an
d
pre
limi
nar
y u
s
abil
i
t
y
eval
uat
ion: An ap
pro
a
ch to assist
novic
e desi
g
n
e
r
.
Journal of T
h
eoretic
al an
d A
ppli
ed Infor
m
at
ion T
e
ch
nol
ogy
. 2014; 67: 69
0
-
700.
[15]
Kein
one
n T
.
One-dim
ensi
o
n
a
l usa
b
i
lit
y
:
i
n
f
l
ue
nce of us
a
b
ilit
y o
n
co
ns
um
er’s pr
od
uc
t preferenc
e.
Univers
i
t
y
of Art and Desi
gn; 1
998.
[16]
F
o
lmer E, Bos
c
h J.
Usab
ility
patterns i
n
soft
w
a
re architect
u
re
. Proce
edi
n
g
s of the
10th
Internatio
na
l
Confer
ence on
.
2003.
[17]
Pattern Factory
.
Pa
tternr
y
: Start buil
d
in
g con
s
istent
w
e
b i
n
terfaces. 201
4.
Evaluation Warning : The document was created with Spire.PDF for Python.