Internati
o
nal
Journal of Ele
c
trical
and Computer
Engineering
(IJE
CE)
V
o
l.
6, N
o
. 5
,
O
c
tob
e
r
201
6, p
p
. 2
415
~242
4
I
S
SN
: 208
8-8
7
0
8
,
D
O
I
:
10.115
91
/ij
ece.v6
i
5.1
063
9
2
415
Jo
urn
a
l
h
o
me
pa
ge
: h
ttp
://iaesjo
u
r
na
l.com/
o
n
lin
e/ind
e
x.ph
p
/
IJECE
An Effiecient Approach for
Resource Auto-Scaling in
Cloud Environments
Bahar Asgari
1
,
Mo
st
afa
Ghoba
e
i
A
r
ani
1
,
Sa
m Ja
b
b
e
hd
ari
2
1
Departm
e
nt
of
Com
puter Engin
eering
,
M
a
h
a
ll
at
Branch,
Is
lam
i
c
Azad Unive
r
s
i
t
y
, M
a
ha
ll
at,
Iran
2
Departm
e
nt
of
Com
puter Engin
eering
,
North
T
e
hran Bran
ch,
Is
l
a
m
i
c Az
ad Univ
ers
i
t
y
,
T
e
hran
, Ir
an
Article Info
A
B
STRAC
T
Article histo
r
y:
Received
Mar 27, 2016
Rev
i
sed
Ju
l 6
,
2
016
Accepte
d
J
u
l 28, 2016
Cloud services have b
ecome more
popular
among users these d
a
y
s
.
Automatic resou
r
ce provisioning
for cloud
services is one of th
e important
challenges in
cloud environmen
ts. In
the
cloud
computing env
i
ronment,
resource provid
e
rs shall offer require
d resour
ces to users automatically
without an
y
limitations
. It means whenev
er a user needs more resources, th
e
required r
e
sources should be ded
i
cated to
th
e user
s without an
y
pr
oblems. On
the other h
a
nd,
if resources are
more th
an user’s needs extr
a reso
urces should
be turn off temporarily
and turn
back on
whenev
er the
y
need
ed. I
n
this
paper,
we propose an automatic r
e
source prov
isioning appro
ach
based o
n
reinforcement learning for au
to-sca
ling resour
ces according to Markov
Decision Process (MDP). Si
mulation Resu
lts show that the rate
of Service
Level Agreement (SLA) violation and st
ability that th
e
proposed approach
bett
er pe
rform
ance
com
p
ared
to
t
h
e s
i
m
ilar
appro
aches
.
Keyword:
A
u
t
o
-
s
caling
C
l
ou
d c
o
m
put
i
n
g
Markov decision
process
Reinforcem
ent learni
ng
Scalab
ility
Copyright ©
201
6 Institut
e
o
f
Ad
vanced
Engin
eer
ing and S
c
i
e
nce.
All rights re
se
rve
d
.
Co
rresp
ond
i
ng
Autho
r
:
B
a
har As
gari
,
Depa
rt
m
e
nt
of
C
o
m
put
er E
ngi
neeri
n
g
,
Mahallat
Branch, Islam
i
c
Azad Uni
v
ersity,
Mah
a
llat, Iran
.
Em
a
il: Bah
a
r_asg
a
ri
8
8
@yaho
o
.co
m
1.
INTRODUCTION
C
l
ou
d com
p
u
t
i
ng i
s
t
h
e
num
ber
of
v
i
rt
ual
i
zed co
n
n
ect
ed c
o
m
p
u
t
ers w
h
i
c
h
o
ffe
rs si
n
g
l
e
com
put
at
i
onal
res
o
u
r
ce
dy
n
a
m
i
cal
ly
and t
o
c
o
m
put
e co
m
p
l
e
x com
put
at
i
on
[1]
-
[
3]
.
In
ot
h
e
r
w
o
r
d
,
cl
ou
d
com
put
i
ng st
at
es t
o
b
o
t
h
ap
p
l
i
cabl
e
pr
og
ra
m
s
offere
d
as
services
on t
h
e Internet,
ha
rdware
and s
o
ftware
syste
m
s in
d
a
ta cen
ters. By d
e
fi
n
itio
n
,
in
data cen
ter h
a
rdware an
d
so
ft
ware are called "clo
ud
". Scalab
ility i
s
one
o
f
t
h
e
ba
si
c conce
p
t
s
i
n
cl
o
ud c
o
m
put
i
ng
w
h
i
c
h i
s
im
port
a
nt
i
n
usi
n
g
hi
g
h
er
effi
ci
ency
of
cl
ou
d
co
m
p
u
tin
g
[4
]. Scalab
ility
is
referred
to
in
crease syste
m
fu
n
c
tion
a
l po
wer to
h
a
v
e
su
it
ab
le respon
se
ag
ain
s
t
increase
d
work loa
d
of course
by adding software
and hardware
res
o
urces
[5].
Whereas
applications
,
esp
ecially ap
p
l
icatio
n
program o
n
web
,
do
n
o
t
h
a
v
e
re
g
u
l
ar wo
rk
load
pattern
s so
th
at
scalab
ility fu
nctio
n
s
(i
ncrea
s
e o
r
d
ecrease
of sc
a
l
e) sh
o
u
l
d
hav
e
be
do
ne i
m
m
e
di
at
el
y
wi
th m
i
nim
u
m
hum
an i
n
t
e
rve
n
t
i
on t
o
pr
o
v
i
d
e
res
o
u
r
ces f
o
r a
p
pl
i
cat
i
ons
as s
o
o
n
a
s
p
o
ssi
bl
e.
R
e
so
urce
scal
i
n
g
wi
t
h
m
i
nim
u
m
h
u
m
a
n i
n
t
e
rve
n
t
i
on i
s
called
au
to-
s
calin
g
[6
]-
[8
].
Var
i
ou
s wo
rkl
o
a
d
s are
of the
biggest c
h
allenges
in
d
i
fferen
t
ti
m
e
s, so
wh
enev
er
p
r
ov
id
er wan
t
s to
m
eet all
th
e req
u
i
rem
e
n
t
s in
a
ll tim
e
s
, it sho
u
l
d
reserv
e m
a
x
i
m
u
m
n
eed
ed
reso
urces
p
r
ev
iou
s
ly for p
eak
wo
rk
lo
ad
to
su
ppo
rt th
e
m
. In
th
is situ
atio
n
p
r
o
v
i
d
e
r so
m
e
t
i
m
es will
b
e
o
v
e
r-
pr
o
v
i
s
i
oni
ng a
nd i
t
i
s
goi
n
g
t
o
be ve
ry
cost
l
y
for t
h
em
(t
o
buy
m
a
xim
u
m
reso
u
r
ces at
peak t
i
m
e
s) whi
c
h l
ead
s
to
lo
wer
p
r
o
f
it. Th
erefo
r
e
functio
n
a
l exp
e
n
s
es will b
e
re
d
u
ced
b
y
turn
ing o
f
f id
le nod
es o
n
i
d
le ti
m
e
s, b
u
t
it
cann
o
t
d
ecreas
e fi
na
nci
a
l
ex
p
e
nses
rel
a
t
e
d p
u
rc
hasi
n
g
a
n
d
host
i
ng
IT e
q
u
i
pm
ent
’
s and t
h
ei
r
dep
r
eci
at
i
o
n
.
I
f
p
r
ov
id
er po
ssesses on
ly enou
gh
r
e
so
ur
ces
(
a
v
e
r
a
g
e
capacity)
to
supp
or
t av
er
ag
e num
b
e
r
o
f
r
e
q
u
e
sts, the
p
r
ov
id
ers m
a
y
b
e
u
tilized
, bu
t th
e
p
r
ov
id
er mig
h
t
no
t h
a
ve eno
ugh
lo
cal
reso
urces to
meet clien
t
s’ req
u
e
st
whi
c
h l
eads t
o
un
der
-
p
r
o
v
i
s
i
oni
n
g
i
n
s
o
m
e
si
t
u
at
i
ons so
pr
ovi
der ha
s t
o
reject
ne
w
cust
om
ers or
cancel
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
I
J
ECE
Vo
l. 6
,
N
o
. 5
,
O
c
tob
e
r
20
16
:
241
5
–
24
24
2
416
p
r
ev
iou
s
serv
ices op
erating
on
system
. W
e
sh
ou
l
d
d
e
sign
a syste
m
wh
ich
will b
e
ab
le t
o
m
a
n
a
g
e
u
n
c
ertain
ty
and
rem
ove a
n
y
pr
obl
em
s i
n
cl
ou
d e
nvi
ro
n
m
ent
.
Al
so
i
t
sho
u
l
d
be
abl
e
t
o
i
m
pact
para
m
e
t
e
rs l
i
k
e ex
pen
s
e,
efficiency, SL
A
violation etc
.
We offe
r auto-scaling accordi
ng to
rei
n
force
m
ent learning. Reinforcem
ent learning (RL)
is a kind of
deci
si
o
n
m
a
king t
h
at
det
e
r
m
i
n
es a goal
per
f
o
r
m
i
ng funct
i
o
nal
m
odel
,
appl
i
e
s p
o
l
i
c
y
wi
t
hout
p
r
evi
o
u
s
i
n
f
o
rm
at
i
on. R
L
has
bee
n
pe
rf
orm
e
d succe
ssful
l
y
i
n
ext
e
nsi
v
e
fi
el
ds
t
o
su
pp
o
r
t
aut
o
-
c
ont
rol
a
n
d
de
di
cat
e
reso
u
r
ces [
9
]
-
[
12]
whi
c
h
wo
r
k
s
on
t
h
e
basi
c assum
p
t
i
on
of
pe
nal
t
y
and
rewa
r
d
s
o
t
h
e
fact
o
r
s m
ove
t
o
wa
rd
ope
rat
i
o
ns
whi
c
h l
ead
t
o
hi
gh
est
pr
o
f
i
t
.
M
a
j
o
r
pa
rt
o
f
R
L
is on
t
h
e
b
a
sis
o
f
d
e
term
in
atio
n of
op
ti
m
a
l
p
o
licies
i
n
M
a
r
k
o
v
[
13]
,[
14]
.
In t
h
i
s
pa
per
we wa
nt
t
o
pr
op
ose a
u
t
o
-sca
l
i
ng
ap
p
r
oac
h
usi
n
g M
D
P t
o
m
a
nage SL
A
vi
ol
at
i
o
n
a
n
d
scalin
g
exp
e
n
s
e and
to preserv
e
system
stab
ilit
y. RL h
a
s the cap
acity to
respo
n
s
e su
itab
l
y u
s
ing
env
i
ron
m
en
t
expe
ri
ences
. R
L
l
eads
t
o
bet
t
e
r m
a
nagem
e
nt
of
com
p
r
o
m
i
se SL
A
vi
ol
at
i
o
n a
n
d
n
u
m
b
er
of
scal
es
but
i
t
causes
h
i
gh
er
ex
p
e
n
s
es. Th
e
r
e
st o
f
t
h
is p
a
p
e
r
is org
a
n
i
zed
as fo
llo
w
s
:
w
e
r
e
v
i
ew
r
e
lated
w
o
r
k
s ab
ou
t RL in
second
part
;
t
h
e
pr
op
o
s
ed ap
p
r
oac
h
c
o
m
e
s i
n
t
h
i
r
d
part
i
n
det
a
i
l
.
The
per
f
o
r
m
a
nce eval
uat
i
o
n
of
pr
o
pose
d
a
p
pr
oac
h
will b
e
ex
p
l
ai
ned
in fou
r
th p
a
rt. Fi
n
a
lly co
n
c
lu
sion
an
d sugg
estion
will b
e
p
r
esen
ted in
fifth
section
.
2.
RELATED WORKS
Vari
ous st
u
d
i
e
s have bee
n
carri
ed o
u
t
abo
u
t
aut
o
-sc
a
l
i
ng an
d i
t
s
im
pl
em
ent
a
t
i
on
. C
u
r
r
ent
app
r
oaches
ha
ve ad
vant
a
g
es and
di
sad
v
ant
a
ges. As t
h
e p
r
op
ose
d
ap
pr
oa
ch i
n
t
h
i
s
pape
r i
s
based u
p
o
n
R
L
,
we
revie
w
re
se
arches
related to
th
is techn
i
que in
th
is section
.
•
En
da B
a
r
r
et
t
e
t
al
. [
14]
ha
ve
bee
n
c
o
n
s
i
d
e
r
ed t
h
e
p
a
ral
l
e
l
Q l
e
a
r
ni
ng
t
o
re
duce
t
i
m
e
of
det
e
rm
i
n
at
i
on
ab
ou
t
o
p
tim
al
p
o
licies an
d onlin
e learn
i
n
g
. Th
eir
proposed approach
use
s
MD
P
along
w
i
th
RL.
•
Fou
a
d Bahr
p
e
y
m
a et al. [
1
5] su
gg
ests RL-D
RP ap
pr
o
ach
.
They
use ne
ural
net
w
or
ks i
n
t
h
ei
r pr
o
p
o
s
ed
mechanism
.
The approac
h
e
n
able cl
oud s
e
rvice
prov
id
ers to
m
eet h
i
g
h
vo
lu
m
e
o
f
requ
ests
witho
u
t
wasting
an
y time, v
a
lu
ab
le
wo
rk
and
at th
e sa
m
e
ti
m
e
co
n
t
ro
l resou
r
ces opti
m
a
lly.
•
Xav
i
er Du
treil
h
et al.
[16
]
h
a
v
e
p
r
o
p
o
s
ed
u
s
in
g
p
r
op
er i
n
itializatio
n
in
p
r
i
m
ary stag
es also
in
creasing
t
h
e
rat
e
o
f
co
n
v
er
g
e
nce i
n
pr
oces
s of l
e
a
r
ni
ng
t
o
sol
v
e
p
r
o
b
l
e
m
.
They
have
o
f
fere
d e
xpe
ri
m
e
nt
s res
u
l
t
s
.
Al
so
t
h
ey
ha
ve i
n
t
r
od
uce
d
a
n
ef
fi
ci
ent
m
odel
t
o
det
ect
cha
nge
s t
h
en c
o
m
p
l
e
t
e
d l
ear
ni
n
g
pr
ocess m
a
nage
m
e
nt
base
d on
t
h
at
.
•
B
a
uer et
al
. [1
7]
pr
op
ose
d
us
i
ng R
L
t
o
m
a
nage t
h
res
h
ol
d
or
der
s
. Fi
rst
cont
rol
l
e
r ap
pl
i
e
s t
h
ese or
ders
t
o
th
e go
al
p
r
ogram
to
rein
force its
q
u
a
lity features.
Seco
nd
con
t
ro
ller sup
e
rv
ises the ord
e
rs, ad
ap
ts
t
h
res
hol
ds a
n
d
chan
ges
co
n
d
i
t
i
ons
, al
so
i
t
de
act
i
v
at
es u
n
rel
a
t
e
d o
r
ders
.
•
Jia Rao et al.
[18]
represe
n
t a
RL awa
r
e
virt
uali
zed m
achine c
o
nfigurati
o
n (V
C
O
NF
).
C
e
nt
ral
de
si
g
n
of
VCONF is
pre
p
are
d
base
d
on RL awa
r
e m
odel to
scale and a
d
apt.
•
Am
oui et al. used RL successfully in m
a
nagem
e
nt
quality of
web
progra
m
s
to optim
ize
program's out
put
[19
]
.
Using
si
m
u
la
tio
n
in in
i
tializat
io
n
o
f
learn
i
n
g
f
u
n
c
ti
o
n
s is on
e
of th
e in
teresting
asp
e
cts o
f
it.
Fi
nal
l
y
Tabl
e
1
sh
ow
s t
h
e
co
m
p
ari
s
on
of
ab
ove
t
ech
ni
q
u
es
.
Table 1.
C
o
m
p
arison of
Techniques
Ref
e
rence
Auto-scaling
technique
Advantages and di
sadvantages
Contr
i
butio
n
E
nda Bar
r
e
tt [14]
Parallel
Q
lear
ning
Decr
easing ti
m
e
of optim
al policy determ
ination
and online lear
ning
Disadvantages: challenges
in determ
ination of
initial policies
I
t
uses inher
e
nt parallelis
m
in
distr
i
buted co
m
puting platform
s
like cloud
Fouad
Bahr
peym
a [15]
RL
Fast conver
g
ence
pr
ocess Higher
utili
zation
I
t
intr
oduces a new decision
m
a
king
process to use predictably anal
ysis
of dem
a
nd which consider
s
par
a
m
e
ter
s
of offer
and dem
a
nd
Xavier Dutreilh
[16]
RL
Hor
i
zontal
scaling
I
n
cr
ease in
conver
g
ence r
a
te in lear
ning stages
I
n
tegr
ation in a r
eal cloud contr
o
ller
and auto pr
ogr
am
m
i
ng
Bahati [17]
RL
I
t
li
m
its situation as pair
of oper
a
tion-
condition
and provides the possibility
of re-use of learned
m
odels
in an or
der set for
next stage
Reinforce
m
ent of load
based on ef
fective lim
i
t
T
h
ey
pr
oposed using RL
to
m
a
nage
thr
e
shold or
der
s
.
Fir
s
t contr
o
ller
applies or
der
to the goal pr
ogr
am
to
i
m
p
r
ove the features of quality
JiaRao [
18]
VCONF
VCONF is
good adaptation with onli
n
e auto
config
ur
ation polic
ies with heter
ogeneous VM
s
VCONF is
enable
to guide initial setting without
decr
easing in funct
i
on of VM
s
Centr
a
l design of VCONF using RL
aware
m
odel work
s to scale and
adapt
Am
oui [19]
RL Quality
m
a
nage
m
e
nt
in
application of new web
design to optim
ize
pr
ogr
am
output
Using si
m
u
lation f
o
r initialization
of lear
ning fu
nctio
ns
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
An
Efficien
t
App
r
oa
ch
f
o
r Reso
u
r
ce Au
to
S
c
alin
g
in Clou
d En
vironmen
ts (Ba
h
a
r
Asga
ri)
2
417
3.
PROP
OSE
D
APP
R
O
A
CH
Fin
a
l
g
o
a
l is
to
m
a
k
e
au
to
scalin
g
system
to
h
a
v
e
th
e
ab
ility o
f
d
ecreasin
g
co
sts an
d
in
creasin
g
system
stab
ility; at th
e sam
e
tim
e with
SLA req
u
i
rem
e
n
t
s an
d
system
efficien
cy
It
m
eans t
o
use
an onl
i
n
e pol
i
c
y
t
o
dedi
cat
e
resources
wi
t
h
scal
i
ng
aut
o
m
a
t
i
cal
l
y
. Proposed
approach
will
be introduced according to RL and MDP.
The offered MDP constitutes from
4 categories
in
clu
d
e
d
co
n
d
itio
n
s
,
o
p
e
ratio
n
s
, tran
sm
itted
p
o
ssib
ilities an
d
reward
s so
th
at d
ecisio
n
m
a
k
i
n
g
about scale up/ scale down will be accom
p
lished based on it.
3.
1.
Reinforcement Lear
ning (RL)
R
L
[7]
,
[14]
,[15]
i
s
a com
put
at
i
onal
approach t
o
underst
a
nd aut
o
m
a
t
i
c
base l
earni
ng
t
o
m
a
ke t
h
e best
deci
si
ons. It
i
n
si
st
s on l
earni
ng vi
a di
rect
i
nvol
vem
e
nt
of agent
and
envi
ronm
ent
.
Decision
m
a
ker refers to
the agent who
learns from
experi
ence and
i
t
s
best
act
i
on
i
s
t
o
perform
at
its
m
a
xi
m
u
m
i
n
any
envi
ronm
ent
.
An aut
o
scal
er i
s
responsi
b
l
e
for deci
si
ons
about
scal
i
ng
wi
t
hout
hum
an i
nvol
vem
e
nt
and i
t
s
object
i
v
e i
s
t
o
adap
t
resources dy
nam
i
cal
l
y
t
o
t
h
e
appl
i
cat
i
ons
according
to input workload. It
d
ecides to allocate or
deallocate
resources to the applications
based on workl
o
ad. In
any
t
t
i
m
e whi
c
h
t
=
0,1,2,… t
i
m
e sequences
are separat
e
d, agent
shows
condi
t
i
on of envi
ronm
ent
s
ts
where
i
s
al
l
possi
bl
e condi
t
i
ons and i
t
sel
ect
s
()
t
t
aA
s
where
()
t
As
is all v
a
riab
les in
th
e co
n
d
itio
n
o
f
t
s
but in a determ
ined tim
e, sequence of these
fu
n
c
tio
n
s
an
d
ag
en
t
will b
e
th
e n
e
x
t
reward
1
t
r
whi
c
h
fi
nds i
t
s
el
f i
n
new condi
t
i
on
of
1
t
s
. Agent
will
select from
condition possibilities then operate
s the possible action. This will be agent'
s
p
o
licy th
at sh
o
w
s
π
t in
wh
ich
π
t (s, a) as
t
aa
at th
e co
n
d
itio
n
t
ss
.
So
MDP can
b
e
sh
o
w
n
in
fo
u
r
categ
o
r
ies
in
clu
d
e
d
co
n
d
itio
n
s
, o
p
e
ratio
n
s
,
tran
sm
itted
p
o
ssib
ilities an
d
reward
s:
S:
E
n
vironm
ental state space
A
: total action
space
P(.|
s, a)
d
e
fin
e
s d
i
stribu
tion
o
f
go
v
e
rn
ed po
ssib
ilities o
n
transmitted
co
nd
itio
n
s
t
tt
~p
(
.
s|
s
1,
a
+)
Q (.|
s
, a)
d
e
fi
nes d
i
stri
b
u
tion
o
f
gov
ern
e
d possib
ilities o
n
receiv
e
d
reward
.
tt
tt
R(
s
~q
,a
.
|
s
)(
,
a
)
The object
i
v
e of l
earni
ng process i
n
si
de l
earni
ng Q i
s
t
o
achi
e
ve t
h
e opt
i
m
al
pol
i
c
y
whi
c
h
refl
ect
s by
Q am
ount
i
n
general
reward
and con
t
i
nues by
operat
i
ng i
n
current
si
t
u
at
i
on.
The
am
ount of Q will be
calculated by equation 1
which includes discounted reward
(decreased
reward) and shows R
L
process pol
i
c
y
.
1
(,
)
(
,
)
(m
a
x
(
,
)
(
,
)
)
tt
t
t
ta
t
t
t
Qs
a
Q
s
a
rQ
s
a
Q
s
a
(1)
Whe
r
e
1
t
r
is m
e
dium
received
re
ward
of selecting
t
a
in
t
s
cond
ition
is learn
i
n
g
rate an
d
γ
i
s
di
s
c
ou
nt
coef
fi
ci
ent
(R
e
duct
i
o
n
)
. T
h
e
o
v
eral
l
pr
ocess
of
R
L
has
been
sh
ow
n i
n
Al
g
o
r
i
t
h
m
1:
Alg
o
rithm
1
:
Reinforcem
ent
Learning
A
l
gor
ith
m
(
Q
-
learnin
g
)
1.
In
itialize Q(s,a) arb
itrarily
2.
Repeat ( for each episode)
3.
In
itialize s
4.
Repeat
5.
C
hoose a from
s usi
ng pol
i
c
y
deri
ved from
Q (€- greedy
)
6.
Take act
i
on a and observe r,s’
7.
tt
tt
t+
1
a
t
t
t
Qs
,
a
Qs
,
a
+
ar
+
m
a
x
Q
s
,
a
-
Q
s
,
a
8.
s
←
s’;
9.
Un
til s is term
in
al
s
a
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
I
JECE
Vo
l. 6
,
N
o
. 5
,
O
c
tob
e
r
20
16
:
241
5
–
24
24
2
418
3.
2.
Prop
osed
Al
g
o
ri
thm
The proposed algorithm
has
been offered ba
sed
on RL, that
is defined according
to
M
a
rkov
process for aut
o
scal
i
ng a M
D
P. Upper
a
nd l
o
wer t
h
reshol
d have
been defi
ned t
oo and
cloud service operation will be m
onito
red after introduction proposed MDP.
C
onfi
gurat
i
on of proposed M
D
P has been consi
d
ered as fol
l
o
ws:
S
: the space of condition: Full utilization,
Under utillization, Norm
al utilization.
A
: the space of operation: Scale up, Scale down, No- op.
P (.|
s
, a)
defines distributi
onpossibility governed on transm
itted conditions.
Q (.|
s
, a)
defines distribution possibility
governed on received reward.
As
we know M
I
PS m
eans t
h
e num
ber
of i
n
st
ruct
i
ons per second. There has been
i
n
t
r
oduced t
w
o vari
abl
e
s
for proposed approach
i
n
cl
uded avai
l
a
bl
e M
I
PS
and R
e
quest
ed
M
I
PS,
both are variables of service i
nputs. Q Updating
will be done usi
ng local regerssion according
to
history of instructions. Divisi
on of two am
ounts shows the am
ount
of utillization (equation 2)
and
com
p
arison of
upper and
lower th
resholds determ
ine
space of
condition. The
full utilization and
the under utilization condition show in
Equation 3 and 4, respectively.
Av
a
i
l
a
b
l
e
M
I
P
S
U
t
i
l
iz
a
t
io
n
R
e
que
s
t
e
d
M
I
P
S
(2)
R
e
qu
e
s
ted
/
A
v
a
i
l
a
bl
e
H
i
g
h-
T
h
re
sh
o
l
d
Fu
l
l
-
U
t
i
l
i
zat
i
o
n
U
n
d
er
-
P
r
o
vi
s
i
o
n
i
n
g
MI
P
S
MI
P
S
(3
)
R
e
que
s
t
e
d
/
A
v
a
il
a
b
l
e
L
o
w-
T
h
r
e
s
hol
d
Un
de
r
-
uti
l
i
z
at
ion
O
v
e
r
-
Pr
ov
isi
onin
g
MI
P
S
MI
P
S
(4)
After
de
fini
ng
full utilization, unde
r utilizati
on a
n
d norm
a
l conditions
, and
operating e
quation
1
(Q(s, a
)
), S
L
A violation am
ount
will be ac
quire
d
by
Re
quested MIP
S
a
n
d a
v
ailable MIPS the
n
decision will
be m
a
de according to a
b
ove
functions
t
o
do curre
nt action, it m
eans to
i
n
crease
or dec
r
ease virtual
machine
or
n
o
o
p
erat
i
o
n. Tabl
e
2
re
p
r
esent
s
p
r
oces
s
o
f
deci
si
o
n
m
a
ki
ng
an
d Fi
gu
re 1 sh
o
w
s a
di
a
g
ram
i
n
cl
ud
e
d
p
r
ov
id
er cond
itio
n
ch
an
g
e
s reg
a
rd
ed
to u
tilizatio
n
p
a
ram
e
te
r.
Tabl
e
2.
Deci
si
on
M
a
ki
ng
B
y
M
D
P
Utilization>High-
T
h
r
e
shold
Low-Threshold< Utilization
< High -
T
hreshold
Utilization <Low
-
T
h
r
e
shold
State(t) Full-Utilization
(
U
nder
-
P
r
ovisioni
ng)
Norm
al
-Utiliza
tion
(
N
orm
a
l-
Pr
ovisioning)
Under-Utilization
(
O
ver
-
P
r
ovisionin
g
)
Next-
Action(
t+1) Scale_up
No-
op
Scale_down
Fig
u
re 1
.
Con
d
itio
n
of p
r
ov
ider
ch
ang
e
s regard
ed
t
o
u
tiliza
tio
n
p
a
ram
e
ter
Proposed
al
gori
t
h
m
i
n
t
r
oduced i
n
t
h
i
s
paper i
s
re
present
e
d as sem
i
code offered i
n
al
gori
t
h
m
2
according to Markov and deci
sion m
a
king in Table 2.
Al
gori
t
hm2:
Reinforcem
ent Learning (Q-Learning
)
1.
In
itialize Q (s, a)=0
,s=0
, a=0
,
h
i
g
h
Ran
g
e
Q=0
.
8
,
lo
w Ran
g
e
Q=0
.
2
.
2.
Observe the Available MIPS and Requested MIPS.
3.
Observe the current state S.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
J
ECE
I
S
SN
:
208
8-8
7
0
8
An
Efficien
t
App
r
oa
ch
f
o
r Reso
u
r
ce Au
to
S
c
alin
g
in Clou
d En
vironmen
ts (Ba
h
a
r
Asga
ri)
2
419
4.
If (req
u
e
sted
MIPS/av
ailab
l
e MIPS) > h
i
g
h
Ran
g
e
Q) , state [0
]= 0
;
/*
Fu
ll-Utilizatio
n
state*
/
5.
Else
if (req
u
e
sted
MIPS/av
ailab
l
e
MIPS) <Lo
w
Ran
g
e
Q),
state[1
]
= 1
;
/*
Un
d
e
r-Utilizatio
n
state */
6.
Else state [2
]= 2
;
/*
No
rm
al-Utilizatio
n
state*
/
7.
Loop
8.
Select action, choose for state ,based one
of the action selection policy Utilization
9.
Take action, observe r, as well as the new state, s’.
10
.
Updat
e
Q val
u
e for t
h
e st
at
e usi
ng t
h
e R
e
gressi
on and observed r and t
h
e m
a
xi
m
u
m
reward
possi
bl
e for t
h
e next
st
at
e.
11
.
tt
tt
t+
1
a
t
t
t
Qs
,
a
Q
s
,
a
+
ar
+
m
a
x
Q
s
,
a
-
Q
s
,
a
12
.
Set the state s to the new state s’ , s
←
s’
13
.
Un
til s is term
in
al
4.
PERFO
R
MA
NCE E
V
ALU
A
TIO
N
There
has bee
n
use
d
of
C
l
o
udsi
m
[2
0]
si
m
u
l
a
t
o
r fo
r
si
m
u
l
a
t
i
on. Fo
u
r
ki
n
d
s of
vi
rt
ual
m
achi
n
e
cor
r
es
po
n
d
ed t
o
Am
azon EC
2 [
2
1]
ha
ve b
een p
e
r
f
o
r
m
e
d whi
c
h t
h
ei
r s
p
eci
fi
cat
i
o
n
s
o
ffe
red i
n
Ta
bl
e II
I
.
There
ha
ve be
en used
f
o
u
r
k
i
nds o
f
ser
v
i
c
e
s
re
gar
d
ed
to t
h
e
variety of a
v
ailable servic
es in cloud a
n
d
we
have
n
o
t
foc
u
s
e
d o
n
t
y
pe o
f
servi
ce
or s
p
ec
i
a
l
pro
g
ram
so t
h
at
used se
r
v
i
ces are i
nde
pe
nde
nt
t
o
pr
og
r
a
m
s
.
These s
e
rvices
are c
o
m
b
ina
tio
n of
all h
e
ter
o
g
e
n
e
ou
s
p
r
og
ra
m
s
lik
e H
P
C,
W
e
b and
so
on. A
l
so
w
o
rk
load
h
a
s
been m
odeled
according to norm
al distri
bution to be close
r
to real world.
Scaling will be done in 24
hour
peri
od a
n
d i
n
5
m
i
nut
es i
n
t
e
r
v
al
s (2
8
8
fi
v
e
m
i
nut
es)
,
L
o
w
-
T
h
res
h
ol
d i
s
c
o
n
s
i
d
ere
d
0.
2 an
d
Hi
g
h
-
T
h
r
es
ho
l
d
i
s
con
s
i
d
ere
d
0
.
8
St
andar
d
de
vi
at
i
on i
s
30
00
M
I
PS an
d Di
f
f
R
a
nge i
s
0.4
There
has bee
n
consi
d
e
r
e
d
a funct
i
o
n
for in
itializatio
n
co
st.
As co
st
fu
n
c
tion
is com
p
u
t
ed
b
y
ho
ur an
d we h
a
v
e
5
m
i
n
u
t
es in
terv
als so
t
h
at we h
a
ve
to
m
u
ltip
le o
v
e
rall co
sts
b
y
3
0
0
/
36
00
.
Table 3. Speci
fication
of Virt
ual
Machi
n
e
Type Of
V
i
rtual
Machine
MIPS
(C
P
U
)
Core
RAM
(M
B
)
Price
(C
e
n
t
)
M
i
cr
o
500
1
633
0.
026
Sm
all 1,
000
1
1,
700
0.
070
E
x
tr
a Lar
g
e
2,
000
1
3,
750
0.
280
High-CPU
Med
i
u
m
2,
500
1
850
0.
560
Algo
rith
m
work
s
b
y
up
d
a
ti
n
g
Q.
we h
a
ve d
o
n
e
Q
up
datin
g
and
ob
tain
in
g
u
tilizatio
n
b
y
lo
cal
regression. Updating Q
will be acc
om
plished acc
ording t
o
inst
ruction
history; it
m
eans ne
xt am
ount
will be
det
e
rm
i
n
ed ac
cor
d
i
n
g t
o
p
r
e
d
i
c
t
i
on
o
f
p
r
e
v
i
o
us am
ount
.
Pre
d
i
c
t
e
d am
ou
nt
s
h
o
u
l
d
b
e
m
u
l
t
i
p
l
i
e
d b
y
0.
7,
because error
possibility has
been conside
r
ed as 30 perce
n
t.
Regressi
on function helps us to scale VMs in the
way
that decre
a
se
failed case along with
m
i
nim
u
m
cost.
The am
ount
of
Avai
l
a
bl
e M
I
PS an
d R
e
q
u
es
t
e
d M
I
PS i
s
ca
l
c
ul
at
ed i
n
t
h
e
m
a
i
n
fu
nct
i
on
of
pr
o
pos
e
d
app
r
oach
. Al
so
t
h
e am
ount
of
SLA vi
ol
at
i
o
n
i
s
cal
cul
a
t
e
d usi
ng t
h
ei
r
di
ffe
rence
.
R
e
que
st
ed M
I
PS i
s
di
v
i
de
d
to
Av
ailab
l
e
MIPS to
i
n
trod
u
ce t
w
o
d
i
m
e
n
s
ion
a
l array r [rstate] [0
],
r
[rstate] [1
] and
r
[rsatate] [2] wh
ich
d
e
term
in
es
und
eru
tilizatio
n
,
fu
ll u
tilizatio
n
an
d
norm
a
l
f
u
n
c
tio
ns.
Th
en
t
h
e am
o
u
n
t
of
r will b
e
upd
ated
an
d
the
am
ount of equation Q (s, a) will
be
calculated.
Ove
r
al
l
M
I
PS am
ount
i
s
o
b
t
a
i
n
ed by
di
vi
si
o
n
of
vi
ol
at
i
o
n r
a
t
e
t
o
t
h
e t
o
t
a
l
M
I
PS t
h
e
n
o
p
t
i
m
al
current
actio
n
will b
e
selected
u
s
ing
Current Ac
tion()
and
Sel
ect
acti
on(
)
acc
ordi
ng t
o
the a
m
ount of
utilization.
Th
en
d
ecision
for scale down an
d
scale up
o
r
nu
ll acti
o
n
will b
e
m
a
d
e
.Propo
sed
app
r
oach
wh
ich
is learn
i
n
g
au
to
m
a
ta aware [22
]
will b
e
co
m
p
ared
to
co
st aware
au
t
o
scalin
g
ap
pro
ach
wh
ich
is a si
m
p
le, au
to
m
a
t
a
ap
pro
ach
b
y
param
e
ters lik
e co
st, SLA
v
i
o
l
atio
n
,
i
n
itializa
tio
n
co
st and
n
u
m
b
e
r
of scal
i
ng.
The
r
e has
bee
n
defi
ned
t
h
re
e s
cenari
o
f
o
r e
v
a
l
uat
i
on
p
r
o
p
o
s
e
d a
p
p
r
oach
i
n
Tabl
e
4.
Tabl
e
4. E
v
al
u
a
t
i
on Sce
n
a
r
i
o
s
Scenario
Goal
Fir
s
t scenar
io
M
i
nim
i
zation SL
A violation
Second scenar
io
M
i
nim
i
zation
T
o
tal Cost
T
h
ir
d scenar
io
M
i
nim
i
zation nu
m
b
er
of scaling
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
I
JECE
Vo
l. 6
,
N
o
. 5
,
O
c
tob
e
r
20
16
:
241
5
–
24
24
2
420
4.
1.
First Sce
n
ari
o
Eval
uat
i
o
n
of
SLA
vi
ol
at
i
on
has
bee
n
con
s
i
d
ere
d
i
n
fi
rst
sce
n
ari
o
com
p
are
d
t
o
t
w
o
ot
her
ap
pro
ach
es. SLA v
i
o
l
atio
n
will b
e
h
a
pp
ened
wh
en
p
r
ov
id
er can
n
o
t
p
r
ov
id
e pred
efined
m
easu
r
es in
SLA fo
r
users
.
Som
e
exam
pl
es of SLA vi
ol
at
i
o
n i
s
t
h
e num
ber o
f
l
o
st
deadl
i
n
es
,
l
ack of wa
rra
nt
y
on ag
reed
M
I
PS
,
l
ack of
war
r
an
t
y
on agree
d
b
a
nd
wi
dt
h, n
u
m
b
er
of re
ject
ed
req
u
est
s
beca
use o
f
n
o
t
ha
vi
ng e
n
o
u
gh
res
o
u
r
c
e
s
at th
e p
eak
times. In
creasing
rate of SLA v
i
o
l
atio
n
ca
u
s
es lo
wer
qu
ality in
p
r
ov
id
ing
serv
ices fo
r
u
s
er. If
Req
u
e
sted
MIPS is n
o
t
m
a
tc
h
with
av
ailab
l
e MIPS, SLA
v
i
o
l
atio
n
will h
a
pp
en
. Fi
g
u
re 2
rep
r
esen
ts resu
lts o
f
com
p
arison
of
SLA vi
olation
in three com
p
ared a
p
proac
h
es
for
4 services. As you can se
e SLA violation rate
i
n
p
r
op
ose
d
a
p
pr
oac
h
i
s
l
e
ss t
h
an
t
h
e
ot
he
rs.
Fi
gu
re 2.
C
o
m
p
ari
s
on
o
f
SL
A vi
ol
at
i
o
n
i
n
servi
ces
Fi
gu
re
3 s
h
o
w
s t
h
e c
o
m
p
ari
s
on
o
f
ove
ral
l
S
L
A
vi
ol
at
i
o
n f
o
r
ser
v
i
ces i
n
c
l
ude
d c
o
st
awa
r
e, l
ear
ni
n
g
aut
o
m
a
t
a
and pr
o
pose
d
ap
pr
oach
. As y
o
u
can see resul
t
s of pr
o
p
o
sed
appr
oac
h
sim
u
l
a
t
i
on com
p
ared t
o
l
earni
n
g
aut
o
m
a
t
a
and cost
aware ap
p
r
oac
h
has l
o
wer r
a
t
e
of SLA
vi
ol
at
i
on at
t
h
e t
i
m
e
of sim
u
l
a
t
i
on so t
h
at
usi
n
g Q l
ear
ni
ng t
e
c
hni
que i
n
aut
o
scal
i
ng
l
eads t
o
re
du
ce
SLA vi
ol
at
i
o
n
.
So
whe
n
e
v
er
SLA i
s
i
m
port
a
nt
fo
r
au
to
scalin
g, we
can
u
s
e
pr
opo
sed
ap
pr
o
a
ch
.
Fi
gu
re
3.
C
o
m
p
ari
s
on
o
f
o
v
er
al
l
SLA
vi
ol
at
i
o
n
i
n
t
h
ree
ap
p
r
oac
h
es
4.
2.
Second Sce
n
ario
We address evalu
a
tio
n
o
f
co
st
m
easu
r
e and
co
m
p
ari
n
g
it with
o
t
h
e
r approach
es.
Serv
ice
co
st will b
e
calculated
acc
ordi
ng
t
o
hours of utility.
It m
e
ans
user pays t
h
e c
o
st accordin
g to spee
d,
power a
n
d ca
pacity of
requ
ested
resou
r
ce (CPU, M
e
m
o
ry an
d
d
i
sk
and
…) also
ti
me o
f
u
s
ing
resource. Nat
u
rally co
st will
b
e
lo
w
whe
n
we use
resource
with lowe
r speed
and
cap
acity in
lo
wer in
terv
als. It
can dec
r
eas
e cost but affe
cts other
q
u
a
lity factors. So to
h
a
v
e
a
hig
h
qu
ality serv
ice we h
a
v
e
to
in
crease
co
st
. Co
st is on
e
of th
e m
o
st i
m
p
o
r
tan
t
factors for users. It m
eans that
use
r
always st
ruggle
s to
accom
p
l
ish
th
e
requ
est wit
h
m
i
n
i
m
u
m
co
st. Para
m
e
ter
o
f
co
st in
trod
uces in
three categ
o
ries: In
itializatio
n
co
st,
Ru
n
tim
e co
st an
d
To
tal cost. In
itializatio
n
co
st is
initial cost for
setting up
VM
s. Runtim
e cos
t
equals t
o
cost according to
utility pe
r hour whic
h
will be pai
d
fo
r
VM
s
ope
ra
t
i
on.
Tot
a
l
c
o
st
cal
cul
a
t
e
s by
e
quat
i
o
n
5:
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISS
N
:
2088-8708
An
Efficien
t
App
r
oa
ch
f
o
r Reso
u
r
ce Au
to
S
c
alin
g
in Clou
d En
vironmen
ts (Ba
h
a
r
Asga
r
i)
2
421
Co
s
=
Co
s
C
o
s
T
o
ta
l
t
I
n
i
tia
l
i
z
a
ti
o
n
t
R
u
n
tim
e
t
(5
)
Runtim
e cost of VM increases nearly 20 percen
t in
sim
u
latio
n
o
f
v
e
rtical scale u
p
.
Co
st
m
easure
in sim
u
lation is calculated according to
addition of VM initia
lization cost and VM
ru
n
tim
e
co
st. Fig
u
r
e 4
sh
o
w
s VM in
itializatio
n
co
st. It sp
ecifies
th
at a Q
aware ap
p
r
o
ach
h
a
s h
i
g
h
in
itializatio
n
co
st wh
ile an
au
to
m
a
ta aware ap
p
r
o
ach
will sav
e
in
itializatio
n
co
st su
b
s
tan
tially.
Fig
u
re
4
.
Co
mp
ariso
n
of i
n
itializatio
n
co
st i
n
three app
r
o
a
ch
es
Fi
gure
5 represent
s
VM
runt
i
m
e cost
i
n
3 servi
ces i
n
24 hours. R
e
sul
t
s
of si
m
u
l
a
t
i
on shows
proposed approach has l
o
wer runt
i
m
e cost
.
Fi
gu
re
5.
C
o
m
p
ari
s
on
o
f
VM
ru
nt
i
m
e cost
i
n
t
h
ree
ap
pr
oac
h
es
Figure 6 represents results
of sim
u
lation accordi
ng
to total cost of
scaling for 3
com
p
ared
approaches i
n
a 24 hour peri
od.
Fi
gu
re
6.
C
o
m
p
ari
s
on
o
f
t
o
t
a
l cost i
n
three a
p
proaches
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
I
JECE
Vo
l. 6
,
N
o
. 5
,
O
c
tob
e
r
20
16
:
241
5
–
24
24
2
422
R
e
sul
t
s
of
si
m
u
l
a
t
i
on acc
or
di
ng t
o
t
a
l
cost
i
n
2
4
ho
u
r
s f
o
r
fo
ur s
e
r
v
i
ces
have
bee
n
re
p
r
esent
e
d i
n
Fi
gu
re 7.
Fig
u
re
7
.
Co
mp
ariso
n
of
ov
erall th
etotal cost
in three a
p
proaches
As i
t
i
s
o
bvi
ou
s i
n
Fi
g
u
r
e 6 a
nd
7,
scal
i
n
g
a
w
are
of
l
earni
ng a
u
t
o
m
a
t
a
h
a
s l
o
we
r c
o
st
c
o
m
p
ared t
o
pr
o
pose
d
a
p
pr
oach
a
n
d
cost
aware
ap
p
r
oac
h
.
Pr
o
pose
d
a
p
p
r
oach
i
n
t
h
i
s
pa
per
has t
h
e hi
ghest
t
o
t
a
l
cost
i
n
co
m
p
ariso
n
.
Q aware app
r
o
a
ch
h
a
s h
i
g
h
in
i
tializat
io
n
and
run
tim
e co
sts. Fin
a
lly to
tal co
st wh
ich
is ad
d
ition
o
f
t
w
o
m
e
n
tion
e
d co
sts shows th
at t
h
e app
r
o
a
ch
will not b
e
pro
p
e
r app
r
o
a
ch
co
m
p
ared
to
learn
i
n
g
au
to
m
a
ta
and cost awa
r
e
approac
h
es
whene
v
e
r
the
cost m
easure is consi
d
ere
d
.
4.
3.
Third Scen
ari
o
In t
h
ird scena
r
io
we addres
s co
m
p
aring num
b
er of scales with
t
h
e
ot
her t
w
o a
p
pr
o
aches.
Th
e
num
ber
of
el
i
m
i
n
at
i
on o
r
a
d
di
n
g
VM
s i
s
o
n
e
of t
h
e i
m
port
a
nt
fact
ors
i
n
dy
nam
i
c scal
ing
.
It
a
ffect
s
s
p
eed
o
f
resp
o
n
se i
n
c
o
m
put
i
ng e
n
vi
ro
nm
ent
.
Al
so
i
t
can cause
t
o
o
p
erat
i
o
nal
ove
rl
oa
d a
n
d
im
poses c
o
st
t
o
t
h
e
sy
st
em
. Pro
p
er
m
a
nagem
e
nt
of
t
h
e m
easur
e hel
p
s
us t
o
achi
e
ve
m
i
nimum
cost
, i
n
c
r
e
a
se rat
e
of
re
s
p
o
n
se
,
con
s
eq
ue
nt
l
y
reduct
i
o
n t
h
e ra
t
e
of SLA
vi
ol
at
i
on. O
v
e
r
al
l
scal
i
ng f
unct
i
o
n cal
cul
a
t
e
s t
o
t
a
l
num
ber of s
cal
es.
The n
u
m
b
er o
f
scal
i
ng f
u
nct
i
ons
f
o
r f
o
ur se
rvi
ces
has bee
n
sh
o
w
n i
n
Fi
g
u
re
8.
As y
o
u
can see t
h
e
nu
m
b
er of
scalin
g
fun
c
tion
s
for propo
sed
appro
a
ch
will n
o
t
ch
ang
e
d
r
am
atical
ly s
o
th
at system
will h
a
v
e
a
pro
p
e
r
stab
ility.
Fi
gu
re
8.
C
o
m
p
ari
s
on
t
h
e
n
u
m
ber of
scal
i
n
g i
n
t
h
ree a
p
p
r
oache
s
As i
t
i
s
represent
e
d i
n
Fi
gure 9, t
h
e
num
ber of scal
i
ng funct
i
ons has been decreased
i
n
proposed
approach com
p
ared to two other appr
oaches according to results of sim
u
lation.
Red
u
c
tio
n
h
e
lp
s to
o
p
tim
ize SLA v
i
o
l
atio
n
rate, lo
wer co
st an
d
h
i
g
h
e
r system
stab
ility.
Evaluation Warning : The document was created with Spire.PDF for Python.
IJECE
ISS
N
:
2088-8708
An
Efficien
t
App
r
oa
ch
f
o
r Reso
u
r
ce Au
to
S
c
alin
g
in Clou
d En
vironmen
ts (Ba
h
a
r
Asga
r
i)
2
423
Fig
u
re
9
.
Co
mp
ariso
n
of
ov
erall th
e nu
m
b
er
of scaling in t
h
ree a
p
proac
h
es
5.
CO
NCL
USI
O
N
Cloud
services are distributed
infrastructures which exte
nd
space of com
m
unication and
service. The resource
providing has
been very
im
portant because of
daily
grow of
cloud services
and
scal
i
ng i
ssue has been wel
c
om
ed
as one of t
h
e m
o
st
i
m
port
a
nt
feat
ures of cl
oud com
put
at
i
on.
In t
h
i
s
paper we have represent
e
d an appro
ach based upon rei
n
forcem
ent
l
earni
ng al
so
have
addressed
M
a
rkov m
odel
.
There
are 3 i
m
port
a
nt
fact
ors i
n
proposed
approaches i
n
cl
udi
ng SLA
vi
ol
at
i
on rat
e
, scal
i
ng
cost
and
num
ber of scal
es.
Regarding cost
m
easure, Q aware
approach is
not
proper approach
com
p
ared t
o
t
h
e aut
o
m
a
t
a
and
cost
aware
approaches. B
u
t
proposed approach
red
u
ces
n
u
m
b
e
r o
f
scales wh
ich
lead
s to
o
p
tim
ize rate
o
f
SLA
v
i
o
l
atio
n
an
d
system
stab
ility. Also
proposed approach decreases SLA vi
ol
at
i
on and opt
i
m
i
z
i
ng SLA l
eads t
o
i
n
crease cost
. As a resul
t
i
t
m
a
kes di
ffi
cul
t
havi
ng m
i
ni
m
u
m
cost
.
On t
h
e ot
her hand focusi
ng
on t
h
e m
i
ni
m
u
m
cost
l
eads
t
o
SLA
vi
ol
at
i
on. So, we can
observe subst
a
nt
i
a
l
reduct
i
on i
n
SLA vi
ol
at
i
on and hi
gher sy
st
em
stab
ility b
y
u
s
in
g
Q-learn
i
n
g
tech
n
i
q
u
e
in
au
to
scalin
g
.
Th
erefo
r
e it is po
ssib
l
e t
o
con
tin
u
e
stud
ies
ab
ou
t
a
u
to
-sca
ling
rega
rde
d
othe
r ef
fective
facto
r
s a
n
d
othe
r a
p
proac
h
es
for e
x
ample the c
o
ndit
i
on space
w
ill be c
h
a
nge
d
according t
o
utilization or
we c
a
n
represe
n
t a novice approach
in
au
to-scalin
g u
s
in
g
p
a
rallel Q learn
i
n
g
an
d co
m
b
in
atio
n
of p
a
rallel factor and
n
e
w cond
itio
n
. Also
we can
ap
p
l
y RL to
pred
icate lo
ad
in
web aware
software’s. Also
it
is po
ssi
b
l
e to
merg
e
RL and m
achine learning. Overl
o
ad
i
n
pr
o
p
o
se
d a
p
p
r
oac
h
sho
u
l
d
be c
ons
i
d
er ca
ref
u
l
l
y
t
o
o
.
REFERE
NC
ES
[1]
R. Bu
yy
a,
et
al.
,
“Cloud computing: prin
ci
ples an
d parad
i
gms,” Jo
hn
Wiley
& Son
s
, vol. 87, 2010.
[2]
A. Vijay
a
and
V. Neelan
aray
a
n
an, “A Model Dr
iven Framework for Portable Cloud Services,”
Int
e
rnation
a
l
Journal of Electrical and
Co
mputer
Eng
i
neer
ing
(
I
JECE)
, vol/iss
u
e: 6(2)
, pp
. 708
-716, 2016
.
[3]
M. G. Arani and M. Shamsi, “An Extended
Approach
for
Efficien
t Data Storage
in Cloud Computing
Environment,”
I
n
ternational Jou
r
nal of Computer
Network and
I
n
formation Secu
rity
, vol/issue: 7(
8), pp
. 30
, 2015
.
[4]
M.
G.
Arani,
et al.
, “An autono
mic approach f
o
r resource provis
i
oning of cloud
services,”
Cluster Computing
, pp.
1-20, 2016
.
[5]
N. Ro
y
,
et
al.
,
“Efficien
t au
toscaling in th
e cloud using pred
ictiv
e models for
workload forecasting,” in
C
l
oud
Computing (
C
LOUD)
,
2011 IEEE In
ternational Conference
on
,
pp. 500-507
, 20
11.
[6]
K. Mogouie,
et
al.
, “A Novel Approach for
Optimizatio
n Auto
-Scaling
in
Cloud Computing Environmen
t,”
International Jo
urnal of Mod
e
rn
Educa
tion and
Computer Scien
c
e
, vol/issue: 7(8
)
, pp
. 9
,
2015
.
[7]
A. Liu
,
“Theor
etical Analy
s
is for
Scale-down-Aw
are Serv
ice Allo
cation in C
l
oud
Storage S
y
s
t
ems,”
In
ternationa
l
Journal of Electrical and
Computer
Eng
i
neer
ing
,
vol/issue:
3(1), p
p
. 21
, 2013
.
[8]
H.
Ghiasi and
M.
G.
Arani,
“Sma
rt Virtual
M
achine P
l
ac
e
m
ent Us
ing Le
arning Autom
a
t
a
to Redu
ce P
o
wer
Consumption in
Cloud Data C
e
nters.”
[9]
R. Hu,
et a
l
.
, “
E
ffic
i
ent R
e
sources Provisionin
g
Based on Lo
ad Forecast
i
ng
in Cloud,”
The
Scient
ifi
c Worl
d
Journal
, 2014
.
[10]
Y.
Che
v
al
ey
re
,
et
al.
,
“
I
ssues in
m
u
ltiagent
resou
r
ce
allo
ca
tion,
”
I
n
formatica
, vol/issue: 30(1), 200
6.
[11]
M. Jacy
no,
et
al.
, “
U
nderstandin
g
dec
e
ntra
lised
c
ontrol of r
e
sourc
e
al
loc
a
tion
in
a
m
i
nim
a
l m
u
lti-a
g
ent s
y
s
t
em
,”
in
Proceed
ings of
t
h
e 6th
int
e
rnatio
nal join
t
conf
ere
n
ce on
Au
tonomous agents and
multiagent
syste
m
s,
pp. 208
, 200
7.
[12]
E. S
c
alas
,
et
al.
, “
G
rowth and
allo
ca
tion of
r
e
s
ources
in
eco
nom
ics
:
The
ag
ent-bas
e
d
appro
ach,
”
Physica
A
:
Statistical Mech
anics and
its
Ap
plications
, vo
l/is
sue: 370(1), pp
.
86-90, 2006
.
[13]
E. Barr
ett
,
et a
l
.
, “Apply
i
ng r
e
inforcement lear
ning towards automating reso
ur
ce al
loc
a
tion
and appli
cat
ion
s
calab
ili
t
y
in th
e cloud
,”
Con
c
u
rrency and Computation:
Practice and
Experien
ce
,
vol/issue: 25
(12), pp. 1656-
1674, 2013
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
SN
:
2
088
-87
08
I
JECE
Vo
l. 6
,
N
o
. 5
,
O
c
tob
e
r
20
16
:
241
5
–
24
24
2
424
[14]
B. B. G. Abad
i
and M. G
.
Aran
i, “Resource Ma
nagement of
IaaS Provide
rs in
Cloud Feder
a
tio
n,”
In
ternationa
l
Journal of Grid
and Distributed
Computing
, vo
l/issue: 8(5), pp. 3
27-336, 2015
.
[15]
F. Bahrpey
m
a,
et
al
.
, “An adaptive RL based app
r
oach for d
y
n
a
mi
c resource provisioning in
Cloud virtualized data
cent
e
rs
,”
Computing
, pp. 1-26, 2
015.
[16]
X.
Dutreilh,
et
al.
, “Using reinf
o
rcement
learning for autonomic re
source allo
cation
in clouds: Towards a fully
autom
a
ted work
flow,” in
IC
AS
2011, The Seventh Internation
a
l Conferen
ce
on Autonomic
and Autonomous
Sy
ste
m
s,
pp
. 67-
74, 2011
.
[17]
R. M. Bahati
and M. Baue
r
,
“
T
owards
adaptive pol
ic
y
-
b
a
sed management,” in
Network Operations
and
Management S
y
mposium (
N
OM
S)
, 2010 IEEE,
p
p
. 511-518
, 201
0.
[18]
J. Rao,
et al.
, “
V
CONF
: a reinforcem
ent le
arning
approach to virt
ual m
achines
aut
o
-configura
tion
,
” in
Pr
oceedings
of th
e 6th
in
tern
ational
conf
eren
ce on
Au
tonomic computing
,
pp
.
137-146, 2009
.
[19]
M. Amoui,
et al.
, “Adaptive action selection in autonomic so
ftware using reinforcement learn
i
ng,” in
Autonomic
and Autonomous
Systems, 2008. I
C
AS 2008. F
ourth International
Conference on
,
pp. 175-181
, 20
08.
[20]
R. N. C
a
lh
eiros,
et
al.
, “CloudSim: a too
l
kit for
modeling and
simula
tion of
clo
ud computing
environments and
evalu
a
tion of resource provisioning algorithms,”
Software: Practice and Experience
, vol/issue: 41(1), pp. 23-50
,
2011.
[21]
Am
azon EC2
in
stance
t
y
pes
,
htt
p
://aws.Am
azon
.
com
/
ec2/
.
[22]
K. Mogouie,
et
al.
, “A Novel Approach for
Optimizatio
n Auto
-Scaling
in
Cloud Computing Environmen
t,”
International Jo
urnal of Mod
e
rn
Educa
tion
&
Co
mputer Science
, vol/issue: 7(8),
p
p
. 9-16
, 2015
.
BIOGRAP
HI
ES OF
AUTH
ORS
Bahar Asgari receiv
e
d the B.S.C degree in Info
rmation Techno
log
y
from PNU
University
, Iran
in 2012, and M.S.C degree from Azad University
of mahallat, Ir
an in 2015, respectively
.
Her
research
inter
e
s
t
s include C
l
ou
d Computing,
Dis
t
ributed S
y
s
t
em
s
,
Bi
g Data and Software
Engineering.
Mostafa Ghobaei Arani r
eceived
the B
.
S.C degr
ee
in
Software
Engineering fro
m University
o
f
Kashan, Iran in
2009, and M.S.C degree fro
m A
zad Univer
sity
of Tehr
an, Iran in 2011,
res
p
ect
ivel
y.
He
is
a P
h
D Candi
date in Is
lam
i
c Azad Univers
i
t
y
, S
c
ienc
e and R
e
s
earch Br
anch,
Tehran
, Iran
.
His research in
terests include
G
r
id Computing, Cloud Computing, Pervasiv
e
Computing, Distributed
S
y
stem
s and Software Development.
Sam Jabbehdari curren
t
ly
work
ing
as
an as
s
i
s
t
ant prof
es
s
o
r at
the d
e
par
t
m
e
nt
of Com
puter
Engineering in IAU (Islamic Az
ad University
),
North Tehran Branch, in Tehran
, since 1993. He
received his both B.Sc. and M.S. degrees in
El
ectr
i
ca
l Engin
e
e
r
i
ng Telecommunication from
Khajeh Nasir Toosi University
of Technolog
y
,
a
nd IAU, South
Tehran branch
in Tehran
, Iran,
respectively
.
He was honored Ph.D. degree in
Computer Engin
eering from IAU, Science and
Research Br
anch, Tehr
an, Iran
in 2005. His curre
nt research
inter
e
sts are Scheduling
,
QoS,
MANETs, Wireless Sensor Netw
orks and Cloud
Computing.
Evaluation Warning : The document was created with Spire.PDF for Python.