Internati
o
nal
Journal of Ele
c
trical
and Computer
Engineering
(IJE
CE)
V
o
l.
6, N
o
. 3
,
Ju
n
e
201
6, p
p
. 1
038
~ 10
47
I
S
SN
: 208
8-8
7
0
8
,
D
O
I
:
10.115
91
/ij
ece.v6
i
3.9
836
1
038
Jo
urn
a
l
h
o
me
pa
ge
: h
ttp
://iaesjo
u
r
na
l.com/
o
n
lin
e/ind
e
x.ph
p
/
IJECE
Academi
c
Cl
oud ERP Quality Assessm
ent Model
Krida
n
to
Surendro, Oliv
i
a
Information S
y
s
t
ems & Technolo
g
y
, Inst
itute of
Technolog
y
Band
ung,
Indonesia
Article Info
A
B
STRAC
T
Article histo
r
y:
Received Dec 30, 2015
Rev
i
sed
Mar
16
, 20
16
Accepte
d Apr 1, 2016
In the pas
t
f
e
w decad
es
, e
ducat
ional ins
t
i
t
utions
have b
een us
ing
conventional academic
ERP s
y
s
t
em to
integr
ate and
optimize th
eir
busin
ess
process. In th
is deliv
er
y
m
odel,
each
educ
ation
a
l
institut
i
ons are
responsibl
e
of thei
r own da
t
a
,
instal
lat
i
on,
a
nd also m
a
int
e
na
nce.
For som
e
in
stitutions,
it
might cause not only
waste of resources
, but als
o
problems in
management
and financial aspects. Cloud-b
a
sed
Academic ERP, a
SaaS-based ERP
s
y
stem, begin to come as a
so
lution w
ith is virtualization technolog
y
.
It
allows institu
tio
ns to use onl
y
the
needed E
R
P
resources, without
an
y
s
p
ecifi
c ins
t
all
a
tion
,
in
tegr
ati
on, or
m
a
int
e
nanc
e n
eeds
.
As
th
e
implementation of Cloud ERP increases
,
problems arise on how to evalu
a
te
this
s
y
s
t
em
. Cu
rrent
evalu
a
t
i
on
approa
ches
are
eith
er on
l
y
ev
alua
ting
the
cloud
computing
aspects or on
ly
evalu
a
tin
g
the so
ftware qu
ality
aspects. This
paper proposes
an assessment mo
del fo
r
Cloud
ERP s
y
stem,
considering bo
th
software quality
char
acte
ristics and cloud computing attrib
utes to help
s
t
rateg
i
c d
e
c
i
s
i
o
n
m
a
kers
ev
alua
t
e
a
cad
em
ic Clo
u
d
ERP
s
y
s
t
em
.
Keyword:
Assessm
ent m
odel
C
l
ou
d c
o
m
put
i
n
g
Ent
e
r
p
ri
se res
o
urce
pl
a
nni
ng
So
ft
ware qu
ality
Copyright ©
201
6 Institut
e
o
f
Ad
vanced
Engin
eer
ing and S
c
i
e
nce.
All rights re
se
rve
d
.
Co
rresp
ond
i
ng
Autho
r
:
K
r
id
a
n
to
Su
r
e
n
d
ro
,
In
fo
rm
ation Sy
stem
s & Tech
n
o
lo
gy
,
In
stitu
te
o
f
Tech
no
log
y
Ban
dun
g,
Jal
a
n
Ganesa
1
0
, B
a
n
d
u
n
g
40
13
2,
I
n
do
nesi
a.
Em
a
il: en
d
r
o
@
in
fo
rm
atik
a.o
r
g
1.
INTRODUCTION
Academ
ic enterprise res
o
urce
planning (ER
P
) syst
em
s are
m
eant to integrate the se
pa
rate busi
n
ess
p
r
o
cess and
fun
c
tio
ns with
i
n
an
ed
u
cation
a
l
in
stitu
tio
n
t
o
h
e
lp
st
ream
l
i
n
i
n
g
t
h
e
p
r
o
cess and
to
p
r
ov
ide real
t
i
m
e
, on dem
a
nd i
n
f
o
rm
at
ion
needs
[1]
.
Ho
weve
r, as
t
h
e busi
n
e
ss cont
i
n
ues t
o
g
r
o
w
, m
o
re an
d m
o
re
resources
n
eed to
b
e
m
a
n
a
g
e
d
in th
e system
. In
ed
u
cation
a
l in
stitu
tion
s
, th
ere are in
creasing
n
u
m
b
e
rs
of
stude
nts, de
pa
rtm
e
nts, and othe
r acad
em
ic data each year. This c
ont
inuo
us growth requires cont
inuous
scaling and im
provem
ent of the academ
ic ERP system
.
Cloud com
puting technol
ogy
ca
n m
e
et several of the
ad
v
a
n
ce and
fu
turistic requ
ire
m
en
ts o
f
ER
P i
m
p
l
e
m
en
tatio
n
s
in
h
i
gh
er edu
catio
n
a
l i
n
stitu
tio
ns and
can cater
t
o
i
n
creasi
ng
vol
um
e and ra
nge
o
f
ser
v
i
c
e
s
[2]
.
T
h
e
r
ef
o
r
e, i
n
or
de
r t
o
adapt
t
o
t
h
i
s
cont
i
n
u
o
u
s
g
r
owt
h
,
academ
ic ERP syste
m
begins
to be
constr
uct
e
d
base
d
on cloud c
o
m
puting platform
.
C
l
ou
d com
put
i
n
g
,
t
h
r
o
ug
h i
t
s
Soft
ware
-as-a
-
Ser
v
i
ce (Saa
S
)
ser
v
i
ce
m
ode
l
,
enabl
e
s c
ons
um
er t
o
use
ap
p
lication
s
rem
o
te
ly, with
o
u
t an
y sp
ecific in
stallatio
n
o
r
i
n
t
e
g
r
at
i
on
nee
d
ed
. The a
ppl
i
cat
i
ons are
bei
ng
ru
n
on a cl
o
ud p
r
ovi
der
’
s i
n
f
r
as
t
r
uct
u
re. T
h
es
e appl
i
cat
i
ons
can be use
d
by
cons
um
ers from
vari
ou
s
cl
i
e
nt
devices
, suc
h
as web
browse
r, or a progra
m interface.
T
h
e cons
um
er does
n
’t nee
d
to
manage or control the
un
de
rl
y
i
ng i
n
f
r
ast
r
uct
u
re
by
t
h
em
sel
v
es. SaaS ap
p
licatio
n
s
u
s
u
a
lly allo
ws
o
n
l
y limited
u
s
er-specific
application confi
g
uration
settings
[3]. M
one
[4] propo
se
d a
m
u
lty-user se
curity policies to access data,
while
R
a
dha
[
5
]
p
r
o
p
o
se
d a se
r
v
i
ce l
e
vel
ag
reem
ents of cloud c
o
mputing.
ER
P sy
st
em
t
h
at
i
s
im
p
l
em
ent
e
d as a soft
w
a
re-as
-
a-se
r
v
i
ce has bee
n
m
e
nt
i
one
d i
n
vari
ous
nam
e
s,
fr
om
servi
ce-b
a
sed ER
P, ER
P III
, t
o
cl
ou
d ER
P [6]
.
I
n
t
h
i
s
pape
r, i
t
i
s
refer
r
ed as cl
o
u
d
ER
P. C
l
oud E
R
P i
s
a
fl
exi
b
l
e
, y
e
t
p
o
w
er
ful
i
n
fo
rm
at
i
on sy
st
em
i
n
cor
p
orat
ed
we
b-
base
d SO
A a
n
d cl
o
ud c
o
m
p
u
t
i
ng ve
rsi
o
n,
w
h
i
c
h
en
ab
les
v
i
rtu
a
l
en
terprises to
o
f
fer in
creasi
n
g
d
e
g
r
ees of fl
ex
ib
ility, ag
ilit
y an
d
d
y
n
a
m
i
c
a
m
o
r
ph
ou
sn
ess [7
].
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
Acad
emic Cl
o
u
d
ERP
Qua
lity Assessmen
t Mod
e
l (Krid
a
n
t
o
S
u
r
end
r
o
)
1
039
This ERP
syste
m
applies cloud com
put
i
n
g
charact
e
r
i
s
t
i
c
s. It
s o
n
-
dem
a
nd-
se
rvice cha
r
acteristic allows ER
P
i
m
p
l
e
m
en
tatio
n
to
b
e
m
o
re flex
ib
le, ad
ap
t
a
b
l
e, and
scalable. Its res
o
urce pooling a
nd m
easured
service
ch
aracteristic allo
ws ER
P im
p
l
e
m
en
tatio
n
to
b
e
m
o
re efficien
t and
afford
able [8
].
M
a
ny
or
ga
ni
za
t
i
ons a
r
e shi
f
t
i
ng
o
r
st
art
i
n
g t
o
s
h
i
f
t
ER
P
de
pl
oy
m
e
nt
, fr
o
m
on-
pr
em
i
s
e
t
o
o
n
-cl
o
u
d
.
Research s
h
ows that the m
a
in reas
o
n
s a
r
e
be
cause ER
P i
m
pl
em
ent
a
t
i
on i
s
becom
i
ng m
o
re com
p
l
e
x an
d
m
o
re
chal
l
e
ngi
n
g
t
o
m
a
nage, an
d or
ga
ni
zat
i
ons
onl
y
ha
ve l
i
m
i
t
e
d reso
urce
s f
o
r t
h
e i
m
pl
ement
a
t
i
on. T
h
ey
do
n’t
want
t
o
spe
n
d m
u
ch on b
u
y
i
ng t
h
e se
rve
r
s
and
hi
re t
h
e I
T
reso
urce
s. T
h
ey
are l
o
o
k
i
n
g fo
r easi
e
r an
d m
o
re
econom
i
cal ERP im
ple
m
entation [9]. Clou
d ERP is the answe
r
of his problem
. It is co
st-savi
n
g, and easy to
u
s
e. B
u
t, as th
e im
p
l
e
m
en
tatio
n
of clou
d
ERP in
cr
eases, in
clud
ing
th
e o
n
e
s
for ed
ucatio
n
a
l in
stitutio
n
s
,
problem
arises on how to eval
uate the
system
.
Recent resea
r
c
h
[10],[11] s
h
ows that t
h
ere a
r
e a
fe
w c
h
allenge
s on cl
oud
ERP im
ple
m
entation.
One
o
f
th
e
p
r
im
ary
ch
allen
g
e
is on
m
o
n
ito
ring
,
an
alysis, and
bu
ild
ing
trust [10
]
. ERP is a v
e
ry larg
e and
critical
appl
i
cat
i
o
n, a
n
d i
t
s
o
u
t
s
o
u
rci
ng
re
qui
re
s co
nst
a
nt
e
v
al
ua
tio
n, m
o
n
ito
r
i
ng, and
r
e
po
r
ting
.
It is th
erefore,
a
m
odel to asses
s
and e
v
aluate
academ
ic cloud ERP
system
is nee
d
e
d
.
This pa
per
propos
es a
m
odel to assess and
eval
uate acade
m
ic cloud ER
P system
. Initial
m
odel is
b
a
sed
on
prev
i
o
u
s
research
[1
2
]
,[13
] on
ERP q
u
a
n
titativ
e assessm
en
t
m
o
d
e
l. Th
e main
d
i
fferen
ce
is in
th
is
research
p
a
p
e
r, th
e m
o
d
e
l is
m
o
re sp
ecialized
for so
ft
w
a
re qu
ality asp
ects of acad
e
m
i
c
ERP system
an
d
cloud
com
puting c
h
a
r
acteristics, es
pecially for
SaaS. Base
d on studies
on these areas, the
assessm
ent
m
odel is
devel
ope
d a
n
d
pr
o
pose
d
.
Th
e m
a
in
o
b
j
e
ctiv
e of th
is
pap
e
r is to
p
r
ov
id
e a m
e
th
o
d
for strateg
i
c
d
ecision
m
a
k
e
rs (with
no
t
echni
cal
back
gr
o
u
n
d
)
t
o
ev
al
uat
e
an
aca
dem
i
c cl
oud
ER
P sy
st
em
. Ho
pe
ful
l
y
, t
h
i
s
pa
per
c
oul
d
hel
p
educational ins
titutions to m
e
asure
the
quali
ty of their academ
ic cloud E
R
P system
. W
ith proper eval
uation
an
d m
easu
r
emen
t, ed
u
cati
o
n
a
l in
stitu
tio
n
s
can
tak
e
ac
tion
s
t
o
im
p
r
ov
e th
ei
r system
an
d
their bu
sin
e
ss.
The p
a
pe
r i
s
or
ga
ni
zed as
f
o
l
l
o
w
s
, i
n
sect
i
on
2 t
h
e aca
d
e
m
i
c cl
oud E
R
P sy
st
em
i
s
defi
ned a
n
d
ex
p
l
o
r
ed
, Software-as-a-Service
q
u
a
lity
assessm
en
t criteri
as are d
i
scu
ssed
in
section
3
.
Th
e co
m
b
in
ati
o
n
and
propose
d
academ
ic cloud ERP assessm
ent
m
odel is described in section
4,
m
odel im
plem
entati
on is shown in
sect
i
on
5,
an
d
f
i
nal
l
y
t
h
e res
u
l
t
i
s
co
ncl
u
ded
i
n
sect
i
o
n
6.
2.
AC
A
D
EMI
C
CLOU
D
E
R
P
2.
1.
Academic Cloud
ERP Archi
t
ecture
To be able to deliver its func
tionalities, academ
ic cloud E
R
P system
is s
u
pporte
d
by a
client/server
architecture. T
h
e comm
only used m
ode
l fo
r cloud
ERP is th
e th
ree-tier
client/server architecture
.
This
archi
t
ect
u
r
e co
nsi
s
t
s
of
3 l
a
y
e
rs:
appl
i
cat
i
o
n/
prese
n
t
a
t
i
on l
a
y
e
r, pr
ocessi
ng
l
a
y
e
r, and exe
c
ut
i
on/
st
ora
g
e l
a
y
e
r.
The fi
rst layer
is located on t
h
e device i
n
terface on the c
lient’s si
de,
whe
r
eas the rest two layers are l
o
c
a
ted in
th
e clou
d serv
er
[9
],
[10
]
.
Fi
gu
re
1.
Aca
d
em
i
c
C
l
oud
E
R
P A
r
chi
t
ect
u
r
e
App
licatio
n layer m
a
n
a
g
e
s
user in
terface, so
the
users
ca
n c
o
n
d
u
ct
vari
ous
t
r
a
n
sact
i
o
ns
wi
t
h
t
h
e
system
. Co
m
pone
nts of this l
a
yer is used t
o
set data fo
rm
at and i
n
terface
for the
use
r
s and ha
ndle use
r
inputs
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJEC
E
V
o
l
.
6,
No
. 3,
J
u
ne 2
0
1
6
:
10
3
8
– 10
47
1
040
and
res
p
onse from
the proce
ssing layer. T
h
is layer al
so
manages user
access to en
s
u
re security within the
academ
ic cloud ERP
system
[10].
Pro
cessi
n
g
l
a
y
e
r m
a
nages t
h
e fu
nct
i
onal
m
odul
es of ac
adem
i
c
cl
oud
ER
P sy
st
em
.
In
p
u
t
s
fr
om
appl
i
cat
i
o
n l
a
y
e
r are p
r
oces
se
d usi
ng t
h
e rel
a
t
e
d dat
a
fr
om
st
orage l
a
y
e
r.
C
o
m
pone
nt
s o
f
t
h
i
s
l
a
y
e
r act
as t
h
e
brai
n of t
h
e sy
st
em
, cont
rol
l
i
ng
dat
a
and
fu
nct
i
o
n
s
t
o
be u
s
ed by
rel
a
t
e
d
users
.
Pr
ocessi
ng l
a
y
e
r al
so i
n
cl
u
d
es
fu
nct
i
o
ns
fo
r
p
o
l
i
c
y
and
sec
u
r
i
t
y
, aut
h
ent
i
cat
i
on,
an
d
vi
rt
ual
i
zat
i
on c
ont
r
o
l
[1
0]
.
St
ora
g
e l
a
y
e
r
con
s
i
s
t
s
of c
o
m
ponent
s
han
d
l
i
ng
or
de
rs f
o
r OLT
P
dat
a
i
n
p
u
t
an
d st
o
r
a
g
e i
n
phy
si
cal
st
ora
g
e de
vi
ces. Dat
a
st
ora
g
e i
s
m
a
naged
usi
n
g a rel
a
t
i
onal
dat
a
base
whi
c
h i
s
used
t
o
st
ore
m
a
ster an
d
transactional data. To m
a
ximize the
proce
s
s
efficiency of
the academ
ic
cloud ERP syste
m
,
usually similar
dat
a
base
m
odel
i
s
use
d
fo
r e
v
ery
m
odul
e i
n
t
h
e sy
st
em
.
2.
2.
Academic Cloud
ERP Requirements
Sim
ilar with any othe
r syst
e
m
, academ
ic
cloud
ERP s
y
ste
m
has its
own
require
m
ents. Thes
e
requ
irem
en
ts hav
e
to
b
e
m
e
t
in
ord
e
r fo
r t
h
e syste
m
to
wo
rk
.
Acad
em
ic
clo
u
d
ERP sy
ste
m
h
a
s to
com
p
l
y
wi
t
h
bot
h aca
d
e
m
i
c ER
P req
u
i
rem
e
nt
s and cl
ou
d
ER
P at
t
r
i
b
ut
es.
Saba
u, et al [14] defi
ned
re
quirem
ents for academ
ic
ERP syst
e
m
.
An integrate
d
unive
r
sity
inform
ation syste
m
m
u
st provide:
Inte
gration
.
A
cadem
i
c
ER
P sy
st
em
m
u
st
pr
ovi
de
dat
a
t
h
at
i
s
in
tegrated
en
terp
rise-w
id
e. Th
is i
n
tegratio
n
is n
e
ed
ed to
p
r
ov
id
e co
m
p
lete cov
e
rag
e
o
f
all n
e
ed
s an
d av
ailab
ility
of info
rm
atio
n
and
av
o
i
d
data
in
con
s
isten
c
y so
th
e pro
cesses in
th
e in
stitu
tio
n
s
co
u
l
d
b
e
do
n
e
well.
Flex
ibility
.
O
p
erat
i
o
ns i
n
ac
adem
i
c
ent
e
rpr
i
ses no
wa
day
s
i
s
very
c
o
m
p
l
e
x. C
h
an
ges a
r
e do
ne
re
gul
ar
l
y
and freque
ntly, va
ries from
the aca
dem
i
c sites (curric
u
lum
,
courses, etc.)
or re
gulat
ory sites (aca
de
mic
pol
i
c
y
)
.
The
r
e
f
ore
,
t
h
e
ER
P s
y
st
em
shoul
d
b
e
fl
exi
b
l
e
f
o
r
t
h
e c
h
an
ges
t
o
be a
ppl
i
e
d
i
n
t
h
e sy
st
em
so i
t
ca
n
m
a
t
c
h t
h
e e
vol
ut
i
o
n
o
f
or
ga
ni
zat
i
on.
Supp
ort in de
cision m
a
kin
g
. The
academ
ic ERP system m
u
st prov
ide
accurate a
n
d good m
a
terials to
su
ppo
rt t
h
e strateg
i
c an
alysis and
d
ecision
mak
i
n
g
i
n
institu
tio
n
a
l gov
ern
a
n
c
e pro
cess. Data
v
a
lid
ity,
reliab
ility an
d
av
ailab
ility is t
h
e co
re req
u
i
remen
t
s to
su
pport th
is
g
o
v
e
rn
an
ce
p
r
o
cess.
Service e
volution
.
One
of the m
a
in purpose for im
ple
m
e
n
tin
g ERP in
the academ
ic i
n
stitutions is t
o
provide better service for
the
institu
tion's academ
ic co
mmunity. There
f
ore,
the se
rvice i
n
the
ERP shoul
d
be a
b
le to
be
im
prove
d for time to tim
e
, e.
g. could be
acce
ssed
from
diffe
rent
de
vices.
Sha
o
[15] defi
ned
attributes for
m
u
lti-tenant
arc
h
itecture, whic
h
is use
d
for the
academ
i
c
cloud ERP
syste
m
. It h
a
s t
o
co
m
p
ly with
4
attribu
t
es:
Resource isol
ati
o
n
. T
h
e
res
o
urce se
paration bet
w
een te
na
nts is
done i
n
a fair m
a
nner s
o
each te
na
nt get
the fair i
n
f
r
astr
uctu
re an
d s
o
ft
ware se
r
v
ices.
It pre
v
e
n
t
s
i
m
pr
o
p
er se
rvi
ce
usa
g
e f
r
om
on
e t
e
nant
t
h
at
c
a
n
interfe
re with
t
h
e
se
rvice f
o
r othe
r
tena
nts.
Cus
t
omi
z
ati
o
n
. C
l
ou
d ap
pl
i
cat
i
ons us
ual
l
y
need c
u
st
om
i
z
at
i
on t
o
fi
t
wi
t
h
cust
om
er’s n
eeds.
Ho
we
ver
,
i
n
cl
ou
d ER
P, c
o
m
p
l
e
x cust
om
izat
i
on i
s
n
o
t
r
ecom
m
e
nde
d as i
t
can l
ead t
o
sl
ow
pr
oces
si
ng
. A
ddi
t
i
o
n
a
l
co
m
p
u
t
atio
n
will b
e
n
eed
ed
at ru
n
tim
e, an
d
ad
ds co
m
p
lex
ity to
th
e d
a
tab
a
se as in
d
i
v
i
d
u
al cu
sto
m
iza
tion
need
s t
o
be
st
o
r
ed
i
n
a
d
di
t
i
on
t
o
va
ri
o
u
s
dat
a
.
Security
. I
n
cl
ou
d ER
P, t
e
na
nt
s sha
r
e t
h
ei
r
dat
a
st
ora
g
e an
d ot
he
r co
de p
r
oces
si
n
g
. Thi
s
shari
n
g sc
hem
a
m
i
ght rise som
e
significa
nt security
risks.
Data access protection is on
e
of the m
o
st vital co
m
pone
nt in
cl
ou
d ER
P
sec
u
ri
t
y
.
Sca
l
ability
.
A
ppl
i
cat
i
o
n de
v
e
l
opm
ent
on c
l
ou
d ER
P m
i
ght
be m
o
re l
i
m
i
t
e
d t
h
an t
h
e
on
-
p
rem
i
se ER
P
.
C
l
ou
d servi
c
e
pro
v
i
d
e
r
s ha
v
e
t
o
have m
u
lt
i
p
l
e
copi
es of
t
h
e sam
e
soft
ware t
h
at
can
be dy
nam
i
call
y
created
t
o
p
r
ovid
e
serv
ices, in ord
e
r to m
a
in
t
a
in
th
e
system
scalab
ility.
The aspects a
b
ove m
u
st be fulfilled so the
academ
ic c
l
oud ERP syste
m
can provide the expecte
d
bene
fits and a
d
vanta
g
es
for t
h
e academ
ic institutions.
3.
SOFTW
ARE
-
A
S-
A
-
SER
V
I
C
E Q
UALIT
Y
ASSES
S
M
E
NT
Th
e q
u
a
lity
o
f
a
Software-as-a-Serv
i
ce (SaaS) ap
p
licatio
n is related
to
its
p
r
od
u
c
t
q
u
a
lity an
d serv
ice
q
u
a
lity. Prod
uct q
u
a
lity is related
to
th
e
fun
c
tio
n
a
lities of
th
e app
lication
itself.
On the o
t
h
e
r
h
a
nd
, serv
ice
q
u
a
lity is related
to
th
e
rate o
f
co
m
p
lian
ce an
d
serv
ices prov
id
ed
b
y
th
e clo
u
d
co
m
p
u
t
in
g
p
r
o
v
i
d
e
rs.
Th
ese
two types
of
quality nee
d
to be m
e
t by academ
ic
cloud ERP system
, as this
syste
m
requires not only
academ
ic ERP functional capa
b
ilities, but also
bene
fits of cl
oud c
o
m
puting services.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
Acad
emic Cl
o
u
d
ERP
Qua
lity Assessmen
t Mod
e
l (Krid
a
n
t
o
S
u
r
end
r
o
)
1
041
3.
1.
Software
Quality
Meas
ure
ment
So
ft
ware
p
r
odu
ct q
u
a
lity cou
l
d
b
e
m
easu
r
ed
with
sev
e
ral fram
e
wo
rk
s
[16
]
-[18
]. On
e o
f
th
e m
o
st
si
m
p
le, yet o
n
e o
f
t
h
e m
o
st
p
o
werfu
l
on
e
is th
e
software q
u
a
lity m
eas
u
r
em
en
t fram
e
work
fro
m
Wag
n
e
r.
Wagn
er [19
]
propo
sed th
e
step
s
for
d
e
v
e
lop
i
n
g
soft
ware
quality
m
o
d
e
l, evalu
a
tio
n
,
an
d
measu
r
em
en
t.
1.
So
ft
ware Qu
ality Plan
n
i
ng
: Th
ese are th
e step
s
requ
ired
t
o
d
e
v
e
l
o
p a so
ft
ware qu
ality
mo
d
e
l.
Fig
u
re
2
.
So
ft
ware Qu
ality Plan
n
i
ng
Step
s [1
9
]
Th
e fi
rst step
is to
id
en
tify
an
d
list all stak
eho
l
d
e
rs related
to
th
e syst
e
m
/so
f
tware.
Th
en
, defin
e
gene
ral
goal
f
o
r eac
h st
ake
h
ol
der
.
T
h
e
goal
s
de
fi
ne
d ca
n
b
e
refi
ned
by
a
n
al
y
z
i
ng
rel
e
v
a
nt
d
o
c
u
m
e
nt
s t
o
get
m
o
re in
fo
rm
ati
o
n. After th
at, list act
iv
ities
b
e
tween
stak
eh
o
l
d
e
r an
d
t
h
e syste
m
th
at
is n
eed
ed
to
reach
th
e
goals. Quality goals ca
n
be
define
d by
a
n
al
yzing activities, prioritizing,
and
deci
ding on how deep
is
each
activity going
to be supporte
d
by the
syste
m
. The next step is to ide
n
tif
y artefacts related to each
quality
goals
, the im
pact from
the s
y
stem
, and choos
e the entitie
s to be evaluated from
each artefacts. M
a
terial
analysis can be use
d
to
get
product fact
or from
each
e
n
tities.
Defi
ned product
factors a
r
e c
h
ecke
d
aga
i
n a
nd
ad
d
e
d
or d
e
du
cted
if n
e
ed
ed. Fin
a
lly,
qu
ality
req
u
i
rem
e
n
t
s can
b
e
sp
ecified
[19
]
.
2.
So
ft
ware Qu
ality Ev
alu
a
tion
an
d Measurem
e
n
t
After th
e
q
u
ality req
u
i
rem
e
n
t
s are d
e
fi
n
e
d, th
e n
e
x
t
step
to d
o
is to
ev
aluate an
d
m
easu
r
e syste
m
’s
fu
lfillm
en
t o
f
each
so
ft
ware qu
ality requ
iremen
t.
Th
e
requ
ired
step
s are sho
w
n
i
n
Fi
g
u
re
3
.
Fig
u
re
3
.
So
ft
ware Qu
ality Ev
alu
a
tion
and
Measu
r
em
en
t [1
9
]
Data is co
llect
ed
for each
p
r
o
d
u
c
t facto
r
fro
m
q
u
a
lity p
l
an
n
i
n
g
. Th
en
, t
h
e
d
e
gree
o
f
ex
isten
ce
for
each product factor is evaluat
e
d from
the data collected. Ea
ch
product factor is gi
ven a weight num
b
er,
base
d
o
n
its im
p
act o
n
th
e
p
r
o
d
u
c
t qu
ality. Th
en, th
e ev
al
u
a
tio
n
resu
lt is ag
gregated
with
th
e
weigh
t
, so
we
can
g
e
t
th
e q
u
a
lity
m
e
asu
r
em
en
t. Aft
e
r th
at, we n
e
ed
to
d
e
v
e
lop
an
in
terp
retatio
n
sch
e
m
a
, in
o
r
der to
b
e
ab
le to
in
ter
p
r
e
t th
e measu
r
em
en
t r
e
su
lt, is it a
g
ood or
b
a
d nu
m
b
er
[1
9
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJEC
E
V
o
l
.
6,
No
. 3,
J
u
ne 2
0
1
6
:
10
3
8
– 10
47
1
042
3.
2.
Software
-as-a-Ser
v
ice Eval
uati
on
Fo
llowing
the
in
creasing u
s
e of SaaS, t
h
e
q
u
a
lity ev
al
u
a
tio
n
m
e
th
od
s are also arisi
n
g.
W
e
n
,
et al
[20
]
p
r
op
osed
a
m
o
d
e
l to
ev
alu
a
te SaaS qu
ality. Th
e p
r
o
posed
m
o
d
e
l is b
a
sed
on
SaaS Matu
rity Mo
d
e
l fro
m
Fo
rrester and
SaaS qu
ality m
o
d
e
l, wh
ich
co
v
e
rs secu
rity
m
easu
r
es,
Quality-o
f-Serv
i
ce (Qo
S
) m
easu
r
es, an
d
so
ft
ware qu
ality
m
easu
r
es.
Fig
u
re 4
.
SaaS Qu
ality
Ev
al
u
a
tio
n
[20
]
There a
r
e f
o
ur
l
e
vel
s
on t
h
e
eval
uat
i
n
g m
odel
.
Th
ese four levels are B
a
sic SaaS, Sta
nda
rd SaaS,
Op
tim
ized
Sa
aS, and
In
teg
r
ated
SaaS. SaaS th
at is
cat
eg
orized
as B
a
sic SaaS fu
lfills so
ftware
q
u
a
lity
criterias, SaaS b
a
sic
features (m
u
lti-ten
a
n
t
), and
co
n
f
i
g
urab
le UI, d
e
d
i
cated
datab
a
se,
sin
g
l
e/m
u
lti in
stan
ce
tech
n
i
cal architectu
r
e. Stand
a
rd
SaaS
fulfills all
Bas
i
c SaaS criteri
as, with
m
e
a
s
u
r
ed
Serv
ice Lev
e
l
Agreem
en
t (SLA) and
standard
ized
app
licatio
n
secu
rity.
Op
tim
ized
SaaS fu
l
f
ills all St
an
d
a
rd
SaaS criterias,
ad
d
e
d
with
stan
d
a
rd
ized
n
e
twork,
u
s
er, and
d
a
ta secur
ity, also
o
p
tim
ize
d
SLA.
In
teg
r
ated
SaaS
fu
l
f
ills all
Op
tim
ized
SaaS criterias
with Qu
ality o
f
Serv
ice (Qo
S
) on
co
st-sav
ing
and
risk
con
t
ro
l [2
0
]
.
Bau
e
r, et al
[21
]
g
a
t
h
ered
k
e
y qu
ality in
d
i
cato
rs
(KQI), sub
s
et
o
f
k
e
y
p
e
rf
orm
a
n
ce in
d
i
cato
rs (KPI)
acr
o
s
s th
e cu
sto
m
er
f
acin
g
ser
v
ice
bo
undar
y
ch
ar
acter
ize k
e
y asp
ects o
f
th
e cu
stomer
’
s
exp
e
r
i
en
ce and
p
e
rcep
tion
o
f
qu
ality. For cloud
-b
ased
ap
p
licati
o
n
,
KQIs can
co
v
e
r t
h
e ch
aracteristics b
e
l
o
w.
Serv
ice Av
ailab
ility: Th
e service is on
lin
e and
av
ailab
l
e to users.
Ser
v
i
ce Lat
e
nc
y
:
The se
rvi
ce
pr
om
pt
l
y
respo
nds
t
o
use
r
req
u
est
s
.
Serv
ice Reliab
ility
: Th
e serv
ice correctly resp
ond
s t
o
u
s
er req
u
e
sts.
Service Accessibility:
The pr
obability that an indivi
dual
use
r
can
prom
ptly access the service or resourc
e
th
at th
ey desire.
Serv
ice Retain
ab
ility: Th
e p
r
o
b
a
b
ility th
at a serv
ice sessio
n
w
ill con
tinu
o
u
s
ly b
e
rend
ered
w
ith
good
serv
ice
qu
ality u
n
til norm
a
l te
rm
in
atio
n
of that session
.
Serv
ice Thro
ug
hpu
t: Meetin
g serv
ice thr
ough
pu
t co
mmit
m
en
ts to cu
st
o
m
er
s.
Serv
ice Tim
e
st
a
m
p
Accu
racy:
Meetin
g
b
illin
g
o
r
reg
u
l
atory
co
m
p
lian
ce accu
racy
requ
iremen
ts.
KQIs usu
a
lly co
v
e
r h
i
g
h
-level b
u
s
in
ess con
s
id
era
tio
ns, in
clud
ing
serv
i
ce q
u
a
lities th
at i
m
p
act u
s
er
sat
i
s
fact
i
on an
d ch
ur
n [
21]
. T
h
e K
Q
Is ca
n b
e
adj
u
st
ed
base
d o
n
t
h
e ap
pl
i
cat
i
on’s e
nvi
ro
n
m
ent
and cust
o
m
er’s
foc
u
s.
4.
ASSES
S
ME
N
T
MO
DEL
The academ
ic
cloud ERP assessm
ent
m
odel
is deve
lope
d base
d on softwa
re quality
measurem
ent
fram
e
work and SaaS
quality evaluation
m
odel, in
accorda
n
ce with
academ
ic c
l
oud ERP re
qui
re
m
e
nts.
Soft
ware
qual
i
t
y
m
easurem
ent
cov
e
rs t
h
e
pr
o
duct
asp
ect
s of aca
dem
i
c cl
oud ER
P sy
st
em
, wherea
s SaaS
q
u
a
lity ev
aluatio
n
cov
e
rs th
e
serv
ice asp
ects o
f
th
e syst
em
. If th
e acad
e
m
i
c clo
u
d
ER
P syste
m
sco
r
es
well in
bot
h c
r
i
t
e
ri
as,
we ca
n
hi
g
h
l
y
expect
t
h
e sy
st
em
i
s
ru
nni
ng
very
wel
l
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
Acad
emic Cl
o
u
d
ERP
Qua
lity Assessmen
t Mod
e
l (Krid
a
n
t
o
S
u
r
end
r
o
)
1
043
4.
1.
Academic Cloud
ERP Appli
c
ati
o
n Qu
ality
Asses
s
men
t
App
licatio
n
q
u
ality
assess
m
e
n
t
can
b
e
d
o
n
e
b
a
sed
o
n
so
ft
ware
p
r
o
d
u
c
t facto
r
s. Pro
d
u
c
t
facto
r
s can
b
e
d
e
fi
n
e
d from
s
t
ak
eho
l
d
e
r
g
o
a
ls
and
activities. Fro
m
stak
eho
l
d
e
rs’ activ
ities to
ach
i
ev
e
g
o
a
ls, we can
fi
nd
the related art
e
facts to be e
x
am
ined
. Each artefact has its own as
pect that can be ass
e
ssed,
whic
h c
a
lled
product factor.
The
table below pr
ovides product
factor
s for academic cloud ERP application
quality
assessm
ent.
The aca
dem
i
c
cl
ou
d ER
P a
p
pl
i
cat
i
on q
u
al
i
t
y
m
odel
cons
i
s
t
s
of 5 m
a
jo
r cri
t
e
ri
as t
o
b
e
eval
uat
e
d.
Tho
s
e criterias are fun
c
tion
a
l
ities, u
s
er-frien
d
lin
ess, secu
rity, b
u
s
in
ess su
ppo
rt, an
d
st
ab
ility. In
to
tal, th
ey
mak
e
up
1
5
pro
d
u
c
t
facto
r
s t
o
b
e
ev
alu
a
ted. Th
ese
facto
r
s are
d
e
ri
v
e
d fro
m
stak
eh
o
l
d
e
rs’
go
als, activities,
and
arte
facts. Each factor
ha
s one
or m
o
re
m
easure to be chec
ke
d in t
h
e ass
e
ssm
ent of eac
h
factor.
The
wei
g
hi
n
g
o
f
t
hos
e m
easures i
s
based o
n
ho
w cr
uci
a
l thos
e aspects a
r
e in affe
cting the perform
a
nce of
academ
ic cloud ERP
system
.
Tab
l
e 1
.
Ap
p
l
i
catio
n
Qu
ality
P
r
oduct
F
a
ct
or
Measure
Weight
(
1
-
10)
1.
Functionaliti
e
s
Accessibility Existence
10
Co
m
p
leteness
Functional
im
pl
e
m
entation co
m
p
leten
e
ss
10
Functional im
ple
m
enta
tion cover
a
ge
10
Cor
r
ectness Co
m
putational
Accur
acy
10
Pr
ecision 10
Accur
a
cy
to expectation
10
Appr
opr
iateness Functional
adequa
cy
10
Functional specific
a
tion
stability (vol
atility)
5
2.
User-friendliness
T
i
m
e
-
b
ehavior
Response
tim
e
10
T
h
r
oughp
ut 7
T
u
r
n
ar
ound tim
e
5
UI
aestheticness
M
e
ssage Clar
ity
10
Interf
ace ele
m
ent c
l
arity
10
Attractive inte
racti
o
n
6
Por
ting user
fr
iendliness
5
3.
Security
Conceal
m
e
nt Data
encr
yption
10
L
i
m
itedness
Session
contr
o
l
10
Authentication Access
auditability
7
Access controllabi
lity
10
Guar
diness
Data corr
uption pr
evention
10
4.
Business support
Dependency
Data-
c
oupled
m
o
d
u
le
r
a
tio
10
Autono
m
y
10
Cohesion
Unified data r
e
fer
e
nce
10
Analyzabilit
y Activity
recording
10
Diagnostic functio
n suppor
t
8
5.
Stability
Operability Operational
consistency
10
Erro
r
co
rre
ctio
n
8
Operational error rec
overability in us
e
5
Custo
m
izabilit
y 5
Reliability Fault
detection
10
Test adequacy
8
I
n
cor
r
ect operation avoidance
8
Failure density
against test cases
10
Fun
c
tio
n
a
lities, bu
sin
e
ss su
pp
ort, an
d security h
a
s
m
o
st h
i
gh
weigh
i
ng
. Th
ese th
ree criterias h
a
v
e
th
e m
o
st i
m
p
a
ct o
n
system
p
e
rfo
r
m
a
n
ce and
qu
ality. Fun
c
tio
n
a
lities are related
to
th
e fun
c
tio
n
a
l m
o
du
les o
f
the system
. Without prope
r
functionalities, system
cannot deliver the
expected outc
om
es. Busines
s
support
in
clu
d
e
s m
o
d
u
larity an
d
an
al
yzab
ility wh
ich
co
n
t
ri
b
u
t
es to
system
co
m
p
lex
ity an
d
troub
leshoo
tin
g
activ
ities.
Security
also plays a big role
in academ
ic
E
R
P perform
a
n
ce, as this syste
m
is
used by a big num
ber of users
and
st
o
r
es a l
a
r
g
e am
ount
of
e
n
t
e
r
p
ri
se
dat
a
.
A si
n
g
l
e
sec
u
ri
t
y
fl
aw co
ul
d
l
ead t
o
m
a
ssi
ve per
f
o
rm
ance d
e
l
a
y
of the aca
dem
i
c cloud
ERP sy
ste
m
.
Th
ese
factors co
v
e
r the acad
e
mic ERP requ
i
r
em
en
ts
. Fun
c
tio
n
a
lities and
stab
ility criteria
cov
e
rs th
e
in
teg
r
ation
req
u
i
rem
e
n
t
s. Bu
sin
e
ss su
pport criteria co
vers
flex
ib
ility and
sup
p
o
r
t
in
d
ecision
mak
i
n
g
requirem
ents. User-friendline
ss an
d se
cu
ri
t
y
cove
rs t
h
e se
rv
i
ce evol
ut
i
o
n r
e
qui
rem
e
nt
s.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJEC
E
V
o
l
.
6,
No
. 3,
J
u
ne 2
0
1
6
:
10
3
8
– 10
47
1
044
4.
2.
Academic
Cloud ERP Ser
v
i
ce Quality As
s
e
ssment
Serv
ice qu
ality assessm
en
t c
a
n
b
e
don
e
b
a
sed
o
n
clou
d
co
m
p
u
tin
g
service asp
ects. Th
ese asp
ects
i
n
cl
ude
s Saa
S
feat
ure
s
a
n
d
ar
chi
t
ect
ure.
T
h
e
t
a
bl
e bel
o
w
p
r
o
v
i
d
es
ser
v
i
c
e
aspect
s
f
o
r ac
adem
i
c
cl
oud
ER
P
serv
ice qu
ality
assessm
en
t.
Th
ere are three m
a
in
asp
e
cts n
e
ed
t
o
be con
s
id
ered
o
n
acad
e
m
i
c
clo
u
d
ER
P serv
ice
qu
ality
assessm
en
t
mo
d
e
l. Tho
s
e criterias are tech
n
i
cal arch
it
ectu
r
e,
q
u
a
lity of serv
ice, and secu
rity. Th
ere are
sev
e
ral m
easu
r
es th
at can
b
e
u
s
ed
t
o
ev
aluate th
e asp
ects. Techn
i
cal arch
itectu
r
e co
vers th
e m
u
lti-t
e
n
a
n
c
y
d
e
gree, d
a
ta iso
l
atio
n
lev
e
l (ded
icat
ed
/sh
a
red
)
, ap
p
licatio
n
iso
l
atio
n
lev
e
l
(sing
l
e in
stan
ce/
m
u
lti in
stan
ce), and
co
nfigu
r
ab
ility (on
UI,
b
u
sin
e
ss log
i
c,
workflow). Qu
ality o
f
Serv
ice
co
v
e
rs th
e scalab
ility, av
aila
b
ility,
reliab
ility (
p
ro
cess usab
ility, effectiv
en
ess and
efficien
cy
), Serv
ice Lev
e
l Ag
reem
en
t
(ex
i
sted
/m
an
aged
/op
t
i
m
ized
), and
in
terop
e
rab
ility (
m
u
lti-
d
e
v
i
ce-u
se) o
f
th
e
system
.
Secu
rity cov
e
rs asp
ects
ens
u
ring security on c
u
st
om
e
r
(SL
A
a
n
d ris
k
m
a
nagem
e
nt), a
p
plication
(authe
n
tication
and access
control,
pre
v
e
n
t
i
on
o
n
com
m
on securi
t
y
t
h
reat
,
pri
v
at
e dat
a
encry
p
t
i
o
n
)
,
net
w
or
k
(en
s
ure
succe
ssf
ul
dat
a
transm
ission), data (confidentiality
, integrity, accessibility, reliabi
lity
), and managem
e
nt (security
m
a
nagem
e
nt
sy
st
em
).
Tab
l
e 2
.
Serv
ice
Qu
ality
Service Aspe
cts
Measure
Weight (1
-10)
Technical architect
ure
Multi-tenancy
10
Data isolation leve
l
7
Application isolation level
7
Confi
gurability 5
Quality of Service
Scalability
10
Service availabilit
y
10
Service reliability
10
Service Level
Agr
e
e
m
ent
7
Interoperability 7
Security Custo
m
er
securit
y
7
Application security
10
Networ
k secur
i
ty
10
Data securit
y
10
M
a
nagem
e
nt security
5
Qu
ality o
f
Serv
ice and
security h
a
s h
i
g
h
e
r wei
g
h
i
ng
amo
u
n
t
. Th
ese two
criterias are i
m
p
actin
g
di
rect
l
y
on h
o
w
ap
pl
i
cat
i
on s
e
rvi
ce i
s
percei
ved
by
t
h
e cust
om
ers. C
u
st
om
ers usi
n
g cl
o
u
d
com
put
i
ng ser
v
i
ces
are exp
ecting
t
o
g
e
t all th
e b
e
n
e
fits
o
f
its tech
no
lo
g
y
,
wh
ich
m
a
in
ly are scalab
ility, in
tero
p
e
rab
ility, easin
ess
to use, a
n
d c
o
s
t
-saving
(whic
h
a
r
e stated i
n
Quality
of Se
rvice).
W
ithout
thes
e bene
fits, academ
ic
cloud
ERP
system
has no additional point from
c
onve
nt
ional academ
ic ERP system
.
Secur
ity is als
o
ve
ry im
porta
nt in
academ
ic c
l
oud ERP syste
m
, as the syste
m
is used by m
u
ltiple tenants at the sam
e
time, and the m
a
nage
m
e
nt
of the system
is outsourced with
the cl
oud se
rvice
provide
r
s
.
Technical arc
h
itecture is als
o
im
portant, alt
h
ough it
is
not
directly affect
ing t
h
e s
e
rvice. Tec
hnical
arch
itecture
o
f
th
e system
affects scalab
ility
and
u
s
ab
ility
o
f
t
h
e system
. Th
e system
’s tech
n
i
cal arch
it
ectu
r
e
h
a
v
e
to
fu
lfill th
e clo
u
d
ERP arch
itecture attribu
t
es, wh
ic
h
are resource iso
l
atio
n
,
cu
stomizatio
n
,
secu
rity, an
d
scalab
ility. In th
is m
o
d
e
l,
resource iso
l
atio
n
is co
v
e
red
b
y
d
a
ta and
ap
p
lication iso
l
atio
n m
e
asu
r
es.
Cu
sto
m
izatio
n
is co
v
e
red
b
y
co
nfigu
r
ab
ility
m
easu
r
es. S
ecu
rity is co
v
e
red
o
n
secu
rit
y
serv
ice asp
e
cts, an
d
scalab
ility is co
v
e
red
o
n
Qu
ality o
f
Serv
ice
asp
ects.
4.
3.
Assessme
n
t Value
Each m
easures
on eve
r
y aspe
cts need to be
evalua
t
e
d o
n
t
h
e sy
st
em
. The eval
uat
i
o
n can
be do
ne by
perform
i
ng syste
m
testing. From
the test re
sults,
one
ca
n
defi
ne the
de
gree
of existenc
e for
each m
e
asure
fr
om
t
h
e scal
e
0-
1,
wi
t
h
0 t
h
e l
o
west
de
g
r
ee (
n
o
n
-
exi
s
t
e
nt
) a
nd
1 t
h
e
hi
g
h
es
t
degree
(f
ul
l
y
per
f
o
r
m
i
ng). E
v
ery
d
e
gree m
easu
r
e
m
en
t is th
en
ag
greg
ated
u
s
i
n
g
th
e
weigh
i
ng
. It is
m
u
ltip
l
i
ed
with
th
e asso
ciated
wei
g
ht, and
th
en
av
erag
ed
t
o
g
e
t th
e
qu
ality assessm
en
t v
a
lu
e.
Pr
ov
i
d
ed
b
e
low
is th
e m
a
th
ematic eq
u
a
tio
n f
o
r
t
h
e pr
oposed
m
o
d
e
l. Let
be the score
value
of a
pr
o
duct
fact
o
r
,
b
e
t
h
e
n
u
m
b
e
r
o
f
m
easu
r
es of a pro
d
u
c
t
fact
o
r
(for
ap
p
l
icatio
n
q
u
a
lity) o
r
serv
ice
asp
ects
(fo
r
p
r
od
u
c
t
qu
ality),
be t
h
e
wei
g
ht
of t
h
e cor
r
es
po
n
d
i
n
g m
easure, a
nd
be t
h
e sc
ore
va
l
u
e of t
h
e sy
st
em
related
to th
e co
rresp
ond
ing
measu
r
e.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
Acad
emic Cl
o
u
d
ERP
Qua
lity Assessmen
t Mod
e
l (Krid
a
n
t
o
S
u
r
end
r
o
)
1
045
Th
e
fin
a
l score v
a
lue for th
e app
licatio
n
quality /
serv
ice
q
u
a
lity can b
e
d
e
term
in
ed
b
y
find
ing
t
h
e
avera
g
e of
all
val
u
es
.
Th
e acad
e
m
i
c
clo
u
d
ERP
qu
ality assess
men
t
v
a
lu
e
shou
ld
b
e
a nu
mb
er i
n
0-1
rang
e,
b
o
t
h
fo
r
ap
p
lication
q
u
a
lity an
d
serv
ice qu
ality.
5.
R
E
SU
LTS AN
D ANA
LY
SIS
In
t
h
is research
, th
e
propo
sed
m
o
d
e
l was i
m
p
l
e
m
en
ted
to
ev
al
u
a
te th
e q
u
a
lity o
f
an Indo
n
e
sian
academ
ic c
l
oud ERP system
,
which is
Cloud SisfoKam
pus [13]. The results
of the
qua
lity assess
m
e
nt
using
th
is m
o
d
e
l are
as fo
llo
ws.
5.
1.
Application Qua
lity
Assessment
Following the
propose
d
m
odel, application qu
ality assessm
ent was done
by exam
ining quality
requ
irem
en
ts o
f
th
e syste
m
in
relatio
n
to
its p
r
od
u
c
t
factors
and m
easures. These are a fe
w exam
ples of the
p
r
od
u
c
t
qu
ality
req
u
i
rem
e
n
t
s,
th
e related assessm
en
t criterias, an
d th
e m
easu
r
em
en
t v
a
lu
e
resu
lts.
Tab
l
e 3
.
Ap
p
l
i
catio
n
q
u
a
lity assessm
en
t
Exa
m
p
l
e
Quality Re
quire
m
e
n
t
Product
Factor
Measure
Value
There is
a H
T
ML
page for each
m
odule of
the sy
ste
m
accessibility
existence
1
Or
ganization data is stor
ed based on needs
to operate the syst
e
m
appr
opr
iateness
Functional adequa
cy
1
Functional specific
a
tion
stability (vol
atility)
0.5
Or
ganization data is stor
ed cor
r
ectly
b
a
sed
on user
input
Cor
r
ectness Co
m
putational
Accur
acy
1
Pr
ecision 1
Accur
a
cy
to expectation
1
Private user dat
a
is
stored accord
ingly Conceal
m
e
nt
Data
encr
yption
1
Assessm
en
t v
a
lu
es are determin
ed
b
y
testin
g. Fo
r ex
amp
l
e for th
e
first q
u
a
lity requ
irem
en
t, to
measure e
x
istence, t
h
ere a
r
e
sets of test cases to
c
h
ec
k
HTM
L
page
e
x
i
s
t
e
nce
fo
r al
l
12 m
o
d
u
l
e
s
of t
h
e
system
. The s
y
stem
received
value sc
ore
1
due t
o
48 HTML page
s existence for all
12
m
odule
s
of
the
sy
st
em
. Funct
i
onal
a
d
eq
uacy
val
u
es ca
n be
det
e
rm
i
n
ed b
y
t
e
st
i
ng t
h
e n
u
m
b
er of
fu
nct
i
ons t
h
at
are s
u
i
t
a
bl
e
for th
e syste
m
fro
m
all
relat
e
d
fun
c
tion
s
.
Fun
c
tio
n
a
l specificatio
n
stab
ility
can
b
e
sco
r
ed
b
y
testin
g
th
e
num
ber o
f
f
u
n
c
t
i
ons t
h
at
are
unc
ha
nge
d f
r
o
m
t
h
e requi
r
e
m
e
nt
s t
o
im
pl
em
ent
a
t
i
on an
d m
a
i
n
t
e
nance. Thi
s
syste
m
received sc
ore
0.5 due
to
20 out
of 40 data
functions change
d on
th
e
way to im
ple
m
entation.
Al
l
t
h
e defi
ne
d sco
r
e val
u
es
are t
h
en a
g
g
r
egat
ed
based
on t
h
e
wei
g
hi
ng
defi
ned i
n
t
h
e pr
o
pose
d
m
o
d
e
l. Th
ey are first agg
r
eg
ated
p
e
r qu
ality req
u
i
re
m
e
n
t
s, th
en
av
erag
ed
for all q
u
a
lity req
u
i
remen
t
s.
Following this
assessm
ent m
odel,
the ac
adem
ic cloud
ERP system
r
ecei
ved
t
h
e
sc
ore of 0.
978 out of
m
a
xim
u
m
scor
e 1.
5.
2.
Service Quality
Assessme
n
t
Si
m
ilar to
th
e
p
r
od
u
c
t
q
u
a
lit
y assessm
en
t, serv
ice
qu
ality
assessm
en
t v
a
lu
es can b
e
d
e
termin
ed
b
y
perform
i
ng test to the s
p
ecified se
rvice as
pects and m
easures
. T
h
ese a
r
e a fe
w e
x
a
m
ples of the
service
assessm
ent criterias, a
n
d the
measurem
ent va
l
u
e res
u
l
t
s
f
o
r
C
l
ou
d
Si
sf
oK
am
pus
[1
3]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
IJEC
E
V
o
l
.
6,
No
. 3,
J
u
ne 2
0
1
6
:
10
3
8
– 10
47
1
046
Tab
l
e
4
.
Serv
ice qu
ality assessm
en
t Ex
am
p
l
e
Service Aspe
cts
Measure
Value
Quality of Service
Scalability
1
Service availabilit
y
0.814
Service reliability
0.915
Service Level
Agr
e
e
m
ent
0
Interoperability 0.25
Security Custo
m
er
securit
y
0
Application security
0
Networ
k secur
i
ty
1
Data securit
y
1
M
a
nagem
e
nt security
1
Values for technical arc
h
itecture an
d sec
u
ri
ty service aspe
cts can be
done using syste
m
test. On the
othe
r hand, sc
ore val
u
es for quality of service aspect
s need to be de
fined
using use
r
acceptance test. For
exam
ple, the availability of
the syste
m
is
tested by checking up
ti
m
e
from
access
by several use
r
s. The
interopera
bility is
tested by user accessing the syste
m
from various
devi
ces (desktop
a
nd m
obile). The syste
m
is no
t ru
nn
ing v
e
ry well
o
n
m
o
b
ile d
e
v
i
ce, h
e
n
ce t
h
e i
n
tero
p
e
rab
ility score is l
o
w. Th
e cu
st
o
m
e
r
an
d
application se
curity as
pects
have
n’t bee
n
taken care
pr
operly because the
system
is running
on e
x
ternal
pl
at
fo
rm
(not
m
a
naged
by
t
h
e sy
st
em
).
Aft
e
r al
l
t
h
e sc
ore
val
u
es
defi
ned
,
t
h
ese
val
u
es are ag
gre
g
a
t
ed usi
n
g t
h
e
d
e
si
gnat
e
d wei
g
hi
n
g
f
r
om
th
e m
o
d
e
l, an
d th
en
av
erag
ed. Th
e serv
ice
q
u
a
lity sco
r
e
fo
r Cloud
SisfoKam
p
u
s
is 0
.
75
2
o
u
t
o
f
m
a
x
i
m
u
m
score
1.
6.
CO
NCL
USI
O
N
Th
e
u
s
e
o
f
acad
e
m
i
c ERP sys
t
e
m
in
ed
u
cati
o
n
a
l i
n
stitu
tio
n is v
e
ry
v
ital. Howev
e
r, so
m
e
edu
catio
n
a
l
institutions ca
nnot i
m
ple
m
e
n
t the syste
m
because of
lack in budget or resour
ces, and
other e
d
ucational
institutions ne
e
d
m
o
re scalable academ
ic
ER
P syste
m
to
adapt with their e
vol
ving
busine
ss. Aca
d
em
ic
c
l
oud
ERP system
en
ab
les th
ese featu
r
es and
q
u
i
ck
ly b
e
co
m
e
a so
lu
tion
.
An
d as th
e im
p
l
e
m
e
n
tatio
n of Cloud
ER
P
increases
, problem
s
arise on how t
o
eval
uate this system
. Therefore,
th
is p
a
p
e
r prov
id
es th
e assessm
en
t
m
odel for academ
ic cloud
ERP quality evaluation, to
s
u
pport project
m
a
nagers in
m
a
king decisi
on
on
im
pl
em
ent
i
ng
cl
ou
d-
base
d ac
adem
i
c
ER
P sy
st
em
.
The assessm
ent
m
odel is develope
d base
d on academ
ic c
l
oud ERP re
qui
rem
e
nt
s, software produc
t
q
u
a
lity m
easu
r
em
en
t, an
d
So
ft
ware-as-a-Serv
ice ev
alu
a
ti
o
n
. Th
e assessm
e
n
t
asp
ects are fo
cu
sed
o
n
th
e
characte
r
istics of academ
ic cloud
ERP sy
ste
m
as a product
and as
a se
rvi
ce. T
h
e
weighi
ng for these
as
pects
are
determ
ined on
how c
r
ucial these as
p
ects
are in affecting
ER
P per
f
o
r
m
a
nce.
Th
e three m
o
st i
m
p
o
r
tan
t
asp
ects to b
e
co
n
s
i
d
ered
on
p
r
od
u
c
t
q
u
a
lit
y sid
e
s are fun
c
tio
n
a
lities,
b
u
s
i
n
ess su
ppo
rt, an
d
secu
ri
ty. Oth
e
r asp
e
cts fro
m
p
r
odu
ct qu
ality
me
asu
r
em
en
t in
clu
d
e
s u
s
er-friend
lin
ess
an
d
stab
ility. On
th
e serv
ice q
u
a
lity sid
e
s, th
e
m
a
in
asp
ects to
b
e
co
n
s
idered
are Qu
ality o
f
Serv
ice an
d
also
security. T
h
e
othe
r as
pect need to
be c
o
ns
idere
d
is
the t
echnical archit
ecture
of
t
h
e s
y
st
em
. Eval
uat
i
on
on
these
as
pects
c
a
n be done
by perform
i
ng
sys
t
e
m
testi
ng and
user acce
ptance testing. From
the evaluation a
nd
weigh
i
ng
agg
r
eg
atio
n,
qu
ality assessm
en
t v
a
lu
e can b
e
d
e
term
in
ed
. Th
e
fin
a
l
resu
lt
s will b
e
2
assessm
en
t
v
a
lu
es,
o
n
e
for so
ft
ware qu
ality, a
n
d
ano
t
her for serv
ice
q
u
a
lity. Th
ese assessm
en
t v
a
lu
e rang
es
from
0
-
1
,
with 0 m
eans
the academ
ic
cloud ERP sy
ste
m
is not
pe
rform
i
ng, a
nd
1 m
eans the academ
ic cloud ERP
syste
m
has the
best
perform
a
nce.
ACKNOWLE
DGE
M
ENTS
Th
is r
e
sear
ch
w
a
s su
ppo
r
t
ed b
y
th
e Research
Gr
an
ts
I
n
st
itu
te o
f
Tech
no
log
y
Ban
dung
, Ind
o
n
e
si
a
(N
o 02
6
5d/
I1
.
C
07/
PL/
2
0
1
5
)
.
REFERE
NC
ES
[1]
C.
Murphy
,
“ERP:
T
h
e
Once
and Fut
u
re King
of Campus Computing,” in
Campus Technology, Syllabus M
e
dia
Group
, 2004. http
://campustec
hnolog
y
.
com/articles
/2004/01/erp
-
the-
once-and-fu
ture-king-of-campus-
computing.aspx
[2]
S.
Goe
l
,
et al.
, “
I
mpact of C
l
oud
Computing on
ERP im
plementations in
Higher
Education,”
In
ternational Journ
a
l
of Ad
vanced
Co
mputer
Science and
Applications
, vol/issue: 2(6),
pp. 146-148
, 20
11.
[3]
National Institute of Standards
and
Technolog
y
(
N
IST), “NIST Definition of
Clou
d Computing: R
ecommendations
of the Nat
i
onal
I
n
stitute of St
and
a
rds and
Tec
hno
log
y
,” U.S. Dep
a
rtm
e
nt of
Com
m
e
rce, 2011.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
Acad
emic Cl
o
u
d
ERP
Qua
lity Assessmen
t Mod
e
l (Krid
a
n
t
o
S
u
r
end
r
o
)
1
047
[4]
S. P. Mone
and
S. S. Dhotre,
“Enforcing Multi-u
s
er Security
Policies
in Cloud
C
o
mputing,”
In
ter
national Journal
of Electrical and
Comput
er
Eng
i
neer
ing (
I
JEC
E
)
, vol/issue: 3(4),
pp. 504~508, 20
13.
[5]
K. Radha,
et a
l
.
, “Service Lev
e
l Agreements
in Cloud Comp
uting and
Big
Data,”
International Journal o
f
Electrica
l
and
C
o
mputer Engin
e
ering (
I
JECE)
, v
o
l/issue: 5
(
1), pp
. 158~165, 2015.
[6]
I. Saeed,
et al.
,
“Cloud Enterpr
i
se Resource Planni
ng Adoption: Motives & Bar
r
iers,”
CONFEN
IS
, pp. 99-122,
2011.
[7]
Y. Wan,
and B
.
Clegg, “En
t
er
pris
e Manag
e
ment
and ERP D
e
velopment:
Case Stud
y
of Zo
omlion Using the
Dy
na
mi
c E
n
te
rpri
se
Ref
e
ren
c
e Grid,”
Communications in
Comp
uter and In
formation S
c
ien
c
e,
v
o
l/issue: 109
(3),
191-198, 2010
.
[8]
G. Raih
ana and
F. Haseen
, “Cloud ERP - A S
o
lution Model,”
Internationa
l Journal
of
Comp
uter Science an
d
Information Technology
&
Security
, vol/issue: 2(
1), pp
. 76-79
, 20
12.
[9]
G.
N.
Purohit,
et al.
, “Challeng
es Involved in I
m
plementation
of
ERP on Demand Solution: C
l
oud Computing,”
International Jo
urnal
of Computer Science,
vol/issue: 9(4), pp. 4
81-489, 2012
.
[10]
A. Chaudhar
y
, “Database Archi
t
ectur
e of OLTP in the S
aaS-based Multi-ten
a
nt
Educa
tional Ent
e
rprise Resource
Planning (Java Enterprise
Edition (J
EE) Bas
e
d Engineering
approach)
,
”
P
r
oceedings of
2012 Internatio
nal
Conference on Education and e-Learning Innovations
, 2012.
http://ieeexp
l
ore.ieee.org/stamp/s
tamp.jsp
?
t
p=&ar
number= 63606
36.
[11]
S.
Roy
,
et al.
, “A cognitive Ap
proach for Eval
uating th
e Usabilit
y
of Storage
as a Service in C
l
oud Com
puting
Environment,”
International Jo
urnal of Electr
i
ca
l and Computer Engineering (
I
JECE)
, vol/issue: 6(2), pp.
759~769, 2016.
[12]
Olivia
and K
.
Surendro, “
G
en
eric
Quanti
tat
i
v
e
A
ssessment Model for
Enterpri
se Resource Planning
(ER
P
)
Sy
s
t
e
m
,
”
ICSIIT - Int
e
l
ligen
ce
in
the
Era o
f
B
i
g
Data. Ba
li
,
vol.
4, pp
. 563-571
,
2015.
[13]
Olivia, “Cloud
Computing Implementati
on
for Academic Enter
p
rise
Resource
Planning (ERP)
S
y
stem in High
er
Educa
tiona
l Inst
ituti
ons in Indo
nesia,”
Undergraduate thesis (
i
n Bahasa)
.
Bandung: Institute
of Technology
Bandung,
2015
.
[14]
G. Sabau
,
et al.
, “An Evaluatio
n
Framework for
Higher
Education ERP S
y
s
t
em,”
WSEAS
T
R
ANSACT
IONS on
COMPUT
ERS
, vol/issue: 8(11), pp.
1790-1799
, 2009.
[15]
Q. Shao,
“
T
owar
ds effe
ctiv
e
and
intel
ligen
t m
u
lt
i-
tenan
c
y Sa
aS,”
PHD thesis,
Arizona State Univ
ersity
, 2011.
[16]
G. O’Regan
,
“
A
P
r
acti
cal
Appr
oa
ch to
Software
Qualit
y,
”
Springer
,
2002
.
[17]
W.
Sury
n,
“Soft
w
a
r
e
Qual
i
t
y
E
n
gi
n
eer
ing: A
Practitioner’s Appr
oach,”
W
ile
y,
20
14.
[18]
C. R.
Pandi
an,
“
S
oftware Metr
ic
s: A Guide
to Planning, Analy
s
is
, and
Appli
cat
io
n,”
C
R
C Press
LCC,
2004.
[19]
S. Wagner, “Software Pr
oduct Q
u
ality
Contro
l,”
Springer,
2013
.
[20]
P. X. Wen and L. Dong,
“Quality
Model for
Eval
u
a
ting SaaS Service,”
Fou
r
th Internationa
l Conference o
n
Emerging Intelligent Data
and W
e
b Techno
logies
. Fukuo
k
a,
2013.
[21]
E. B
a
uer
and
R.
Adam
s
,
“
S
ervice
Qu
ality
of
Clou
d-Based Applications,”
IEEE
Press and Wiley,
20
14.
BIOGRAP
HI
ES OF
AUTH
ORS
Kridanto Surend
ro is a lecture
in
the School of
El
ectr
i
cal Engin
eer
ing and Inform
at
ics – Institut
e
of Technolog
y
B
a
ndung, Indones
i
a.
. He graduated
from doctoral degree in Comp
uter Scien
ce at
Keio University, Japan
.
His r
e
search
area
in
cludes Cloud Co
mputing, Infor
m
ation S
y
stems
Quality
,
Information Govern
ance,
and
IT Service
Management.
Olivia gr
aduated from bachelo
r
degree in
Inf
o
rmation S
y
stems & Technolog
y
,
School of
Electrical Eng
i
n
eering
& Info
rmatics - Instit
ute o
f
Technolog
y
B
a
ndung, Indonesia in
2015. Her
res
earch ar
ea i
n
cludes
:
Cloud
Qualit
y
As
s
e
s
s
m
ent, Inform
ation S
y
s
t
em
s
,
and S
o
ftware
Engineering.
Evaluation Warning : The document was created with Spire.PDF for Python.