I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
,
p
p
.
1175
~
1
1
8
7
I
SS
N:
2088
-
8
7
0
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijece.
v
1
6
i
3
.
pp
1
1
7
5
-
1
1
8
7
1175
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
Residua
l rein
forc
ement l
ea
rning
f
o
r distur
ba
nce
-
resi
lient
co
ntrol unde
r mo
deling
uncer
tainti
es
Abo
la
nle A
det
if
a
,
Rex
cha
rles
E
ny
in
na
Do
na
t
us
,
Da
niel U
dek
we
D
e
p
a
r
t
me
n
t
o
f
A
e
r
o
s
p
a
c
e
En
g
i
n
e
e
r
i
n
g
,
F
a
c
u
l
t
y
o
f
A
i
r
E
n
g
i
n
e
e
r
i
n
g
,
A
i
r
F
o
r
c
e
I
n
st
i
t
u
t
e
o
f
Te
c
h
n
o
l
o
g
y
,
K
a
d
u
n
a
,
N
i
g
e
r
i
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Ma
r
4
,
2
0
2
6
R
ev
is
ed
Ap
r
2
,
2
0
2
6
Acc
ep
ted
Ap
r
2
6
,
2
0
2
6
M
o
d
e
r
n
c
o
n
tro
l
sy
ste
m
s
m
u
st
o
p
e
ra
te
re
li
a
b
ly
i
n
t
h
e
p
re
se
n
c
e
o
f
m
o
d
e
li
n
g
u
n
c
e
rtain
ti
e
s
a
n
d
e
x
tern
a
l
d
ist
u
rb
a
n
c
e
s,
c
o
n
d
it
io
n
s u
n
d
e
r
w
h
ich
c
o
n
v
e
n
t
io
n
a
l
fix
e
d
-
g
a
i
n
c
o
n
tro
ll
e
rs
o
f
ten
e
x
h
ib
it
p
e
rf
o
rm
a
n
c
e
d
e
g
ra
d
a
ti
o
n
.
T
h
is
p
a
p
e
r
p
ro
p
o
se
s
a
re
sid
u
a
l
re
i
n
fo
rc
e
m
e
n
t
lea
rn
i
n
g
fra
m
e
wo
rk
f
o
r
d
i
stu
rb
a
n
c
e
-
re
sili
e
n
t
p
i
tch
-
ra
te
c
o
n
tro
l
o
f
a
n
a
ircra
ft
lo
n
g
i
tu
d
in
a
l
m
o
d
e
l.
A
c
las
sic
a
l
p
ro
p
o
rti
o
n
a
l
-
i
n
teg
ra
l
-
d
e
riv
a
ti
v
e
(
P
ID)
c
o
n
tro
ll
e
r
is
e
m
p
l
o
y
e
d
a
s
a
sta
b
il
izin
g
b
a
se
li
n
e
,
wh
il
e
a
d
e
e
p
d
e
term
in
i
stic
p
o
li
c
y
g
ra
d
ien
t
(DD
P
G
)
a
g
e
n
t
lea
rn
s
a
b
o
u
n
d
e
d
re
sid
u
a
l
c
o
n
tro
l
si
g
n
a
l
t
o
c
o
m
p
e
n
sa
te
fo
r
u
n
m
o
d
e
led
d
y
n
a
m
ics
a
n
d
e
x
tern
a
l
p
e
rtu
r
b
a
ti
o
n
s.
T
o
p
r
o
m
o
t
e
fa
v
o
ra
b
le
tran
sie
n
t
b
e
h
a
v
io
r,
th
e
lea
rn
in
g
p
ro
c
e
ss
in
c
o
rp
o
ra
tes
tran
sie
n
t
-
a
wa
re
a
n
d
re
fe
re
n
c
e
-
m
o
d
e
l
-
b
a
se
d
re
wa
rd
sh
a
p
in
g
,
wh
il
e
a
c
tu
a
to
r
c
o
n
stra
i
n
ts
a
re
e
n
f
o
rc
e
d
wi
th
i
n
th
e
e
n
v
iro
n
m
e
n
t
d
y
n
a
m
ics
.
S
imu
lati
o
n
re
su
lt
s
d
e
m
o
n
stra
te
t
h
a
t
t
h
e
p
ro
p
o
se
d
re
sid
u
a
l
c
o
n
tro
ll
e
r
a
c
h
iev
e
s
a
su
p
e
rio
r
b
a
lan
c
e
b
e
twe
e
n
re
sp
o
n
se
sp
e
e
d
,
o
v
e
rsh
o
o
t
,
a
n
d
trac
k
in
g
a
c
c
u
ra
c
y
c
o
m
p
a
re
d
with
b
o
th
t
h
e
sta
n
d
a
l
o
n
e
P
ID
c
o
n
tro
l
ler
a
n
d
a
p
u
re
DD
P
G
-
b
a
se
d
c
o
n
tro
ll
e
r.
In
p
a
rti
c
u
lar,
t
h
e
re
sid
u
a
l
a
r
c
h
it
e
c
tu
re
sig
n
ifi
c
a
n
t
ly
re
d
u
c
e
s
o
v
e
rsh
o
o
t
a
n
d
trac
k
i
n
g
e
rro
r
w
h
il
e
p
re
se
r
v
in
g
fa
st
tran
sie
n
t
re
sp
o
n
se
a
n
d
p
ro
v
id
in
g
ro
b
u
st
d
istu
r
b
a
n
c
e
re
jec
ti
o
n
u
n
d
e
r
larg
e
p
it
c
h
i
n
g
m
o
m
e
n
t
d
ist
u
rb
a
n
c
e
s.
Th
e
se
re
su
lt
s
in
d
ica
te
th
a
t
re
sid
u
a
l
re
in
fo
rc
e
m
e
n
t
lea
rn
i
n
g
o
ffe
rs
a
p
ra
c
ti
c
a
l
a
n
d
e
ffe
c
ti
v
e
a
p
p
r
o
a
c
h
f
o
r
e
n
h
a
n
c
in
g
ro
b
u
st
n
e
ss
a
n
d
p
e
rfo
rm
a
n
c
e
in
sa
fe
ty
-
c
rit
ica
l
fli
g
h
t
c
o
n
tr
o
l
a
p
p
li
c
a
ti
o
n
s.
K
ey
w
o
r
d
s
:
Dee
p
d
eter
m
in
is
tic
p
o
licy
g
r
ad
ien
t
Dis
tu
r
b
an
ce
r
ejec
tio
n
Fli
g
h
t c
o
n
tr
o
l
Pit
ch
-
r
ate
tr
ac
k
in
g
R
esid
u
al
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Dan
iel
Ud
ek
we
Dep
ar
tm
en
t o
f
Aer
o
s
p
ac
e
E
n
g
in
ee
r
in
g
,
Facu
lty
o
f
Air
E
n
g
in
ee
r
in
g
,
Air
Fo
r
ce
I
n
s
titu
te
o
f
T
ec
h
n
o
lo
g
y
Nig
er
ian
Air
Fo
r
ce
B
ase,
Ma
n
d
o
,
Kad
u
n
a,
Nig
er
ia
E
m
ail:
d
au
d
ek
we@af
it.e
d
u
1.
I
NT
RO
D
UCT
I
O
N
T
h
e
in
cr
ea
s
in
g
co
m
p
le
x
ity
an
d
u
n
ce
r
tain
ty
o
f
m
o
d
er
n
co
n
t
r
o
l
s
y
s
tem
s
h
av
e
e
x
p
o
s
ed
lim
itatio
n
s
in
tr
ad
itio
n
al
m
o
d
el
-
b
ased
co
n
tr
o
l
s
tr
ateg
ies
[
1
]
.
C
las
s
ical
ap
p
r
o
ac
h
es
s
u
ch
as
p
r
o
p
o
r
tio
n
al
-
in
teg
r
al
-
d
e
r
iv
ativ
e
(
PID
)
co
n
tr
o
l,
lin
ea
r
q
u
ad
r
atic
r
eg
u
latio
n
(
L
QR
)
,
an
d
m
o
d
el
p
r
ed
ictiv
e
co
n
tr
o
l
(
MPC
)
p
er
f
o
r
m
r
eliab
ly
wh
e
n
s
y
s
tem
d
y
n
am
ics
ar
e
ac
cu
r
at
ely
m
o
d
eled
a
n
d
d
is
tu
r
b
an
ce
s
ar
e
p
r
ed
ictab
le.
I
n
p
r
ac
tic
e,
h
o
wev
er
,
m
an
y
s
y
s
tem
s
ar
e
n
o
n
lin
ea
r
,
tim
e
v
ar
y
in
g
,
a
n
d
af
f
ec
ted
b
y
m
o
d
elin
g
u
n
ce
r
tain
ties
o
r
ex
ter
n
al
d
is
tu
r
b
an
ce
s
[
2
]
,
[
3
]
.
Un
d
er
s
u
ch
co
n
d
itio
n
s
,
co
n
v
e
n
tio
n
al
co
n
tr
o
ller
s
m
ay
ex
p
e
r
ien
ce
d
eg
r
ad
e
d
r
o
b
u
s
tn
ess
an
d
lim
ited
ad
ap
tab
ilit
y
[
4
]
.
R
ein
f
o
r
ce
m
en
t
lear
n
i
n
g
(
R
L
)
h
as
em
er
g
ed
as
a
p
r
o
m
is
in
g
alter
n
ativ
e
b
ec
a
u
s
e
it
en
ab
les
co
n
tr
o
l
p
o
licies
to
b
e
lear
n
ed
d
ir
ec
tly
f
r
o
m
in
ter
ac
tio
n
with
th
e
e
n
v
ir
o
n
m
en
t.
R
L
d
o
es
n
o
t
r
eq
u
ir
e
an
ex
p
licit
m
o
d
el
an
d
ca
n
a
d
ap
t
to
c
o
m
p
lex
o
r
p
ar
tially
u
n
k
n
o
wn
d
y
n
am
ics
[
5
]
.
Desp
ite
th
is
f
lex
ib
ilit
y
,
r
ea
l
-
wo
r
ld
d
ep
l
o
y
m
en
t
r
em
ain
s
ch
allen
g
in
g
.
Dee
p
R
L
alg
o
r
ith
m
s
s
u
ch
as
d
ee
p
d
et
er
m
in
is
tic
p
o
licy
g
r
ad
ie
n
t
(
D
DPG)
an
d
p
r
o
x
im
al
p
o
licy
o
p
tim
izatio
n
(
PP
O)
o
f
ten
s
u
f
f
er
f
r
o
m
h
i
g
h
s
am
p
le
co
m
p
lex
ity
,
u
n
s
tab
le
co
n
v
er
g
en
ce
,
an
d
u
n
s
af
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
7
5
-
1
1
8
7
1176
ex
p
lo
r
atio
n
[
6
]
.
T
h
ese
lim
itatio
n
s
r
estrict
th
eir
ap
p
licab
ilit
y
in
s
af
ety
-
cr
itical
d
o
m
ain
s
wh
er
e
s
tab
ilit
y
an
d
co
n
s
tr
ain
t satis
f
ac
tio
n
ar
e
ess
en
tial.
T
o
o
v
er
c
o
m
e
th
ese
d
r
awb
ac
k
s
,
r
ec
en
t
r
esear
ch
h
as
f
o
cu
s
ed
o
n
h
y
b
r
id
co
n
tr
o
l
f
r
am
e
wo
r
k
s
th
at
co
m
b
in
e
R
L
with
estab
lis
h
ed
co
n
tr
o
ller
s
.
R
esid
u
al
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
(
R
R
L
)
is
a
p
r
o
m
in
en
t
ex
am
p
le,
in
wh
ich
th
e
R
L
ag
en
t
lea
r
n
s
a
c
o
r
r
ec
tiv
e
s
ig
n
al
th
at
au
g
m
e
n
ts
a
s
tab
ilizin
g
b
aselin
e
co
n
tr
o
ller
[
7
]
.
T
h
e
r
esid
u
al
ac
tio
n
is
ty
p
ically
co
n
s
tr
ain
ed
to
p
r
eser
v
e
n
o
m
in
al
s
tab
ilit
y
wh
ile
im
p
r
o
v
in
g
tr
ac
k
in
g
o
r
d
is
tu
r
b
an
ce
r
ejec
tio
n
.
E
x
p
e
r
im
en
tal
r
esu
lts
in
r
o
b
o
tic
s
y
s
tem
s
h
av
e
s
h
o
w
n
th
at
r
esid
u
al
p
o
licies
ca
n
en
h
an
ce
p
er
f
o
r
m
a
n
ce
wh
ile
m
ain
tain
in
g
s
tr
u
ctu
r
e
d
an
d
in
ter
p
r
etab
le
c
o
n
tr
o
l
b
e
h
av
io
r
[
8
]
.
B
r
o
a
d
er
an
aly
s
es
also
in
d
icate
th
at
h
y
b
r
id
ar
ch
itectu
r
es o
f
ten
o
u
tp
er
f
o
r
m
s
tan
d
alo
n
e
R
L
ag
en
ts
in
r
ea
l
-
tim
e
an
d
s
af
ety
-
s
en
s
itiv
e
ap
p
licatio
n
s
[
9
]
.
Ad
d
itio
n
al
co
n
t
r
ib
u
tio
n
s
h
av
e
f
u
r
th
e
r
r
ein
f
o
r
ce
d
th
is
d
ir
e
ctio
n
b
y
in
teg
r
atin
g
R
L
with
s
tr
u
ctu
r
e
-
p
r
eser
v
in
g
f
ilter
s
an
d
th
e
o
r
etica
l
s
af
ety
g
u
ar
an
tees.
Piccin
elli
et
a
l.
[
1
0
]
p
r
o
p
o
s
ed
a
p
ass
iv
e
R
L
f
r
am
ewo
r
k
th
at
en
s
u
r
es saf
e
p
o
licy
co
n
v
e
r
g
en
ce
u
s
in
g
L
i
p
s
ch
itz
co
n
tin
u
ity
co
n
s
tr
ain
ts
an
d
o
p
tim
al
c
o
n
tr
o
l g
u
id
an
ce
.
I
n
a
s
im
ilar
ef
f
o
r
t,
Ko
s
telac
et
a
l.
[
1
1
]
in
tr
o
d
u
ce
d
a
L
ip
s
ch
itz
-
f
i
lter
ed
R
L
s
ch
em
e
g
u
id
ed
b
y
MPC
th
at
b
alan
ce
s
ad
ap
tab
ilit
y
an
d
co
n
s
tr
ain
t
a
d
h
er
en
ce
i
n
n
o
n
lin
ea
r
s
ettin
g
s
.
T
h
ese
ap
p
r
o
ac
h
es
em
p
h
asize
th
e
v
alu
e
o
f
em
b
ed
d
in
g
p
r
i
o
r
co
n
tr
o
l k
n
o
w
led
g
e
an
d
co
n
s
tr
ain
t h
a
n
d
lin
g
d
ir
ec
tly
in
to
th
e
lea
r
n
in
g
ar
ch
i
tectu
r
e.
Ad
d
itio
n
al
s
tu
d
ies
h
av
e
s
tr
en
g
th
en
ed
t
h
is
ap
p
r
o
ac
h
b
y
em
b
ed
d
in
g
t
h
eo
r
etica
l
s
af
ety
g
u
ar
an
tees
an
d
s
tr
u
ctu
r
al
co
n
s
tr
ain
ts
in
to
th
e
lear
n
i
n
g
p
r
o
ce
s
s
.
Fo
r
e
x
am
p
le,
p
ass
iv
e
an
d
L
ip
s
ch
itz
-
co
n
s
tr
ain
ed
R
L
f
o
r
m
u
latio
n
s
h
av
e
b
ee
n
p
r
o
p
o
s
ed
to
p
r
o
m
o
te
s
af
e
p
o
licy
co
n
v
er
g
en
ce
a
n
d
b
o
u
n
d
ed
c
o
n
tr
o
l
u
p
d
ates
[
1
2
]
.
Similar
ly
,
R
L
s
ch
em
es
g
u
id
e
d
b
y
MPC
in
co
r
p
o
r
ate
p
r
io
r
co
n
tr
o
l
k
n
o
wled
g
e
to
b
alan
c
e
ad
ap
tab
ilit
y
with
co
n
s
tr
ain
t
ad
h
er
en
ce
in
n
o
n
l
in
ea
r
s
y
s
tem
s
[
1
3
]
.
I
n
d
is
tu
r
b
an
ce
-
r
ic
h
e
n
v
ir
o
n
m
en
ts
,
r
e
ce
n
t
co
n
t
r
ib
u
tio
n
s
d
em
o
n
s
tr
ate
th
at
s
af
ety
-
awa
r
e
R
L
ca
n
im
p
r
o
v
e
r
o
b
u
s
tn
e
s
s
an
d
d
is
tu
r
b
an
ce
r
ejec
tio
n
wh
en
ap
p
r
o
p
r
iate
f
ilter
in
g
an
d
c
o
n
s
tr
ain
t
m
ec
h
an
is
m
s
ar
e
in
teg
r
ate
d
in
to
th
e
co
n
tr
o
l
lo
o
p
[
1
4
]
,
[
1
5
]
.
T
h
ese
d
ev
elo
p
m
en
ts
s
u
p
p
o
r
t
t
h
e
v
iew
th
at
lear
n
in
g
-
b
ased
co
n
tr
o
ller
s
ar
e
m
o
s
t
e
f
f
ec
tiv
e
wh
en
co
m
b
in
ed
with
estab
lis
h
ed
co
n
tr
o
l
p
r
in
cip
les
[
1
6
]
.
Mo
tiv
ated
b
y
th
is
lin
e
o
f
r
ese
ar
ch
,
we
p
r
o
p
o
s
e
a
s
af
e
r
esid
u
al
r
ein
f
o
r
ce
m
e
n
t
lear
n
in
g
f
r
a
m
ewo
r
k
f
o
r
d
is
tu
r
b
an
ce
-
r
esil
ien
t
co
n
t
r
o
l.
A
co
n
v
en
tio
n
al
PID
co
n
tr
o
ller
p
r
o
v
id
es
b
aselin
e
s
tab
ilit
y
,
wh
ile
a
DDPG
-
tr
ain
ed
r
esid
u
al
p
o
licy
c
o
m
p
en
s
ates
f
o
r
ex
ter
n
al
d
is
tu
r
b
an
ce
s
an
d
m
o
d
elin
g
in
ac
c
u
r
ac
ies.
T
r
ain
in
g
is
co
n
d
u
cte
d
in
a
co
n
s
tr
ain
ed
a
n
d
d
is
tu
r
b
a
n
ce
-
in
ten
s
iv
e
en
v
ir
o
n
m
en
t
to
p
r
o
m
o
te
r
o
b
u
s
t
an
d
s
af
e
b
eh
av
io
r
.
T
h
e
f
r
am
ewo
r
k
is
ev
alu
ated
u
s
in
g
a
lo
n
g
itu
d
in
al
air
cr
af
t
d
y
n
am
ics
m
o
d
el,
d
em
o
n
s
tr
atin
g
im
p
r
o
v
e
d
tr
ac
k
in
g
p
er
f
o
r
m
an
ce
an
d
d
is
tu
r
b
a
n
ce
r
ejec
tio
n
co
m
p
ar
ed
t
o
b
o
t
h
class
ical
co
n
tr
o
l
an
d
s
tan
d
alo
n
e
R
L
ap
p
r
o
ac
h
es,
wh
ile
m
ain
tain
in
g
s
m
o
o
t
h
an
d
b
o
u
n
d
ed
c
o
n
tr
o
l a
ctio
n
s
.
T
h
e
r
em
ain
d
er
o
f
th
is
m
an
u
s
cr
ip
t
is
o
r
g
an
ized
as
f
o
llo
ws.
Sectio
n
2
d
escr
ib
es
th
e
p
r
o
p
o
s
e
d
m
eth
o
d
o
l
o
g
y
,
in
clu
d
in
g
th
e
m
ath
em
atica
l
m
o
d
el
o
f
th
e
air
cr
af
t
lo
n
g
itu
d
i
n
al
d
y
n
am
ics
an
d
th
e
co
n
tr
o
ller
d
esig
n
p
r
o
ce
d
u
r
e.
Sectio
n
3
p
r
esen
ts
th
e
s
im
u
latio
n
r
esu
lts
an
d
d
is
cu
s
s
io
n
,
f
o
c
u
s
in
g
o
n
tim
e
-
d
o
m
ai
n
r
esp
o
n
s
es,
p
itch
r
ate
tr
ac
k
in
g
p
er
f
o
r
m
an
ce
,
an
d
d
is
tu
r
b
an
c
e
r
ejec
tio
n
.
Sectio
n
4
co
n
clu
d
es
th
e
p
a
p
er
a
n
d
o
u
tlin
es th
e
m
ain
lim
itatio
n
s
o
f
th
e
s
tu
d
y
.
2.
M
E
T
H
O
D
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
m
o
d
elin
g
o
f
th
e
air
cr
af
t lo
n
g
itu
d
i
n
al
d
y
n
am
ics,
f
o
llo
wed
b
y
th
e
d
er
iv
atio
n
o
f
th
e
lin
ea
r
ize
d
m
o
d
el
an
d
a
d
escr
ip
tio
n
o
f
th
e
p
r
o
p
o
s
ed
co
n
tr
o
ller
s
.
2
.
1
.
M
o
delin
g
o
f
t
he
lo
ng
i
t
ud
ina
l dy
na
m
ics
T
h
e
lo
n
g
itu
d
in
al
m
o
tio
n
o
f
a
f
ix
ed
-
win
g
air
c
r
af
t
d
escr
ib
es
it
s
tr
an
s
latio
n
al
an
d
r
o
tatio
n
al
b
eh
av
io
r
in
th
e
v
er
tical
p
lan
e
an
d
is
g
o
v
er
n
ed
b
y
t
h
e
f
o
r
war
d
v
elo
cit
y
,
a
n
g
le
o
f
attac
k
,
p
itch
an
g
le,
an
d
p
itch
r
ate.
I
n
th
is
s
tu
d
y
,
th
e
lo
n
g
itu
d
in
al
d
y
n
a
m
ics
o
f
an
air
cr
af
t
a
r
e
ad
o
p
t
ed
as
th
e
p
lan
t
m
o
d
el
f
o
r
c
o
n
tr
o
ller
d
ev
el
o
p
m
en
t,
f
o
llo
win
g
s
tan
d
ar
d
ae
r
o
d
y
n
am
ic
f
o
r
m
u
latio
n
s
r
ep
o
r
te
d
in
th
e
liter
atu
r
e
[
1
7
]
.
T
h
e
ae
r
o
d
y
n
am
ic
f
o
r
ce
s
an
d
m
o
m
en
ts
ac
tin
g
o
n
th
e
a
ir
cr
af
t
ca
n
b
e
ex
p
r
ess
ed
in
ter
m
s
o
f
n
o
n
d
im
e
n
s
io
n
al
ae
r
o
d
y
n
am
ic
co
ef
f
icien
ts
o
b
tain
ed
f
r
o
m
wi
n
d
-
tu
n
n
el
test
in
g
.
T
h
ese
f
o
r
ce
s
an
d
m
o
m
en
ts
ar
e
g
iv
en
b
y
(
1
)
an
d
(
2
)
.
=
,
=
,
=
,
(
1
)
=
,
=
,
=
,
(
2
)
wh
er
e
̅
d
en
o
tes
th
e
d
y
n
am
ic
p
r
ess
u
r
e,
is
th
e
w
in
g
r
ef
er
en
ce
ar
ea
,
is
th
e
win
g
s
p
an
,
an
d
̅
is
th
e
m
ea
n
ae
r
o
d
y
n
am
ic
ch
o
r
d
.
T
h
e
co
e
f
f
icien
ts
,
,
,
,
,
an
d
r
ep
r
esen
t
th
e
d
im
e
n
s
io
n
less
ae
r
o
d
y
n
a
m
ic
f
o
r
ce
an
d
m
o
m
en
t c
o
ef
f
icien
ts
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
R
esid
u
a
l rein
fo
r
ce
men
t le
a
r
n
in
g
fo
r
d
is
tu
r
b
a
n
ce
-
r
esil
ien
t c
o
n
tr
o
l
…
(
A
b
o
l
a
n
le
A
d
etifa
)
1177
Ass
u
m
in
g
s
y
m
m
etr
ic
f
lig
h
t
co
n
d
itio
n
s
an
d
n
e
g
lectin
g
later
al
-
d
ir
ec
tio
n
al
co
u
p
lin
g
an
d
th
r
u
s
t
v
ec
to
r
in
g
e
f
f
ec
ts
,
th
e
n
o
n
lin
ea
r
lo
n
g
itu
d
in
al
eq
u
atio
n
s
o
f
m
o
tio
n
o
f
th
e
air
cr
af
t c
a
n
b
e
wr
itt
en
as
[
1
8
]
:
̇
=
=
2
[
(
)
c
os
+
(
)
s
in
]
−
s
in
(
−
)
+
[
(
,
)
c
os
+
(
,
)
s
in
]
+
c
os
(
3
)
̇
=
[
1
+
2
2
(
(
)
c
os
−
(
)
s
in
)
]
+
[
(
,
)
c
os
−
(
,
)
s
in
]
+
c
os
(
−
)
−
s
in
(
4
)
̇
=
(
5
)
̇
=
2
[
(
)
+
Δ
(
)
]
+
[
(
,
)
+
Δ
(
,
)
]
(
6
)
Her
e,
d
en
o
tes th
e
air
s
p
ee
d
,
is
th
e
an
g
le
o
f
attac
k
,
is
th
e
p
itch
an
g
le,
is
th
e
p
itch
r
ate,
r
ep
r
esen
ts
th
e
elev
ato
r
d
ef
lectio
n
,
is
th
e
th
r
u
s
t
f
o
r
ce
,
is
th
e
air
cr
af
t
m
ass
,
is
th
e
g
r
av
itatio
n
al
ac
ce
ler
a
tio
n
,
an
d
is
th
e
m
o
m
en
t
o
f
i
n
er
tia
ab
o
u
t
t
h
e
p
itch
a
x
is
.
T
h
ese
n
o
n
lin
ea
r
eq
u
atio
n
s
ca
p
t
u
r
e
th
e
co
u
p
led
ae
r
o
d
y
n
am
ic
a
n
d
in
er
tial e
f
f
ec
ts
g
o
v
er
n
in
g
t
h
e
l
o
n
g
itu
d
i
n
al
m
o
tio
n
o
f
th
e
air
cr
af
t.
2
.
1
.
1
.
L
inea
rize
d lo
ng
it
ud
ina
l m
o
del
Fo
r
co
n
tr
o
ller
s
y
n
t
h
esis
an
d
s
tab
ilit
y
an
aly
s
is
,
th
e
n
o
n
lin
ea
r
lo
n
g
itu
d
i
n
al
d
y
n
a
m
ics
ar
e
lin
ea
r
ized
ab
o
u
t
a
s
tead
y
,
win
g
s
-
lev
el
tr
im
co
n
d
itio
n
u
s
in
g
a
f
ir
s
t
-
o
r
d
er
T
ay
lo
r
s
er
ies
ex
p
an
s
io
n
wh
ile
n
eg
lectin
g
h
ig
h
er
-
o
r
d
e
r
ter
m
s
[
1
7
]
.
Def
i
n
in
g
th
e
s
tate
v
ec
to
r
as
s
h
o
wn
in
(
7
)
an
d
th
e
co
n
tr
o
l
in
p
u
t
as
th
e
elev
at
o
r
d
ef
lectio
n
,
th
e
lin
ea
r
ized
s
tate
-
s
p
ac
e
r
ep
r
esen
tatio
n
is
g
i
v
en
b
y
(
8
)
an
d
(
9
)
.
=
[
]
(
7
)
[
Δ
˙
Δ
˙
Δ
˙
Δ
˙
]
=
[
−
0
.
022
−
0
.
002
0
3
×
10
−
7
−
1
.
395
−
0
.
582
0
0
.
324
−
9
.
828
0
0
0
−
0
.
672
0
.
908
1
−
0
.
708
]
[
]
+
[
−
1
.
139
−
0
.
072
0
−
4
.
301
]
(
8
)
[
]
=
[
0
0
0
1
0
0
1
0
]
[
]
(
9
)
E
ig
en
v
alu
e
a
n
aly
s
is
o
f
th
e
r
e
s
u
ltin
g
lin
ea
r
ized
s
y
s
tem
r
e
v
e
als
th
e
p
r
esen
ce
o
f
u
n
s
tab
le
m
o
d
es
in
th
e
o
p
e
n
-
lo
o
p
d
y
n
am
ics,
in
d
icatin
g
th
at
th
e
air
cr
af
t
is
n
o
t
in
h
e
r
en
tly
s
tab
le
in
th
e
lo
n
g
itu
d
i
n
al
ax
is
.
T
h
is
m
o
tiv
ates
th
e
n
ee
d
f
o
r
a
co
n
tr
o
l
au
g
m
e
n
tatio
n
s
y
s
tem
ca
p
ab
le
o
f
s
tab
ilizin
g
th
e
air
cr
af
t
wh
ile
en
s
u
r
in
g
ac
cu
r
ate
p
itch
-
r
ate
tr
ac
k
in
g
an
d
d
is
tu
r
b
a
n
ce
r
ejec
tio
n
,
wh
ich
is
ad
d
r
ess
ed
in
th
e
s
u
b
s
eq
u
en
t sectio
n
s
.
2
.
1
.
2
.
Co
ntr
o
l
s
urf
a
ce
a
ct
ua
t
io
n m
o
del
Acc
u
r
ate
m
o
d
elin
g
o
f
ac
t
u
ato
r
d
y
n
am
ics
an
d
co
n
s
tr
ain
ts
is
ess
en
tial
in
-
f
lig
h
t
co
n
t
r
o
l
s
y
s
tem
d
esig
n
to
en
s
u
r
e
r
ea
lis
tic
s
im
u
latio
n
an
d
s
af
e
co
m
m
an
d
g
en
er
atio
n
.
T
h
e
air
cr
a
f
t'
s
elev
ato
r
co
n
tr
o
l
s
u
r
f
ac
e,
d
en
o
ted
,
is
m
o
d
eled
with
b
o
th
d
y
n
a
m
ic
r
esp
o
n
s
e
an
d
n
o
n
lin
ea
r
p
h
y
s
ical
lim
its
to
ca
p
tu
r
e
r
ea
l
-
wo
r
ld
b
eh
av
i
o
r
as
s
h
o
wn
in
Fig
u
r
e
1
.
Fig
u
r
e
1
.
E
le
v
ato
r
ac
tu
atio
n
m
o
d
el
in
clu
d
i
n
g
lag
d
y
n
a
m
ics,
r
ate
lim
itin
g
an
d
d
e
f
lectio
n
s
atu
r
atio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
7
5
-
1
1
8
7
1178
T
h
e
ac
tu
atio
n
m
o
d
el
in
clu
d
e
s
a
f
ir
s
t
-
o
r
d
er
lag
,
a
r
ate
li
m
iter
,
an
d
a
d
e
f
lectio
n
lim
iter
,
ap
p
lie
d
s
eq
u
en
tially
to
th
e
co
m
m
a
n
d
ed
in
p
u
t.
T
h
e
elev
ato
r
ac
t
u
ato
r
d
y
n
am
ics
ar
e
r
ep
r
esen
ted
b
y
a
f
ir
s
t
-
o
r
d
er
la
g
s
y
s
tem
with
th
e
tr
an
s
f
er
f
u
n
ctio
n
1
+
1
(
1
0
)
w
h
er
e
=
49
.
5
×
10
−
3
is
th
e
ac
tu
ato
r
tim
e
co
n
s
tan
t.
T
h
is
m
o
d
els
th
e
i
n
ter
n
al
r
esp
o
n
s
e
d
ela
y
o
f
th
e
ac
tu
ato
r
to
a
co
n
tr
o
l
in
p
u
t.
T
h
e
o
u
tp
u
t
o
f
th
e
lag
b
lo
c
k
is
th
en
p
ass
ed
th
r
o
u
g
h
a
r
ate
lim
iter
to
en
f
o
r
ce
th
e
m
ax
im
u
m
allo
wab
le
r
ate
o
f
c
h
an
g
e
in
th
e
elev
ato
r
d
ef
lectio
n
:
|
|
≤
60
∘
/
s
(
1
1
)
Af
ter
th
e
r
ate
lim
it is
ap
p
lied
,
th
e
s
ig
n
al
is
co
n
s
tr
ain
ed
b
y
th
e
elev
ato
r
'
s
d
ef
lectio
n
lim
its
:
−
25
∘
≤
≤
+
25
∘
(
1
2
)
T
h
e
s
ig
n
co
n
v
en
tio
n
u
s
ed
d
ef
in
es
a
p
o
s
itiv
e
as
a
tr
ailin
g
e
d
g
e
d
o
wn
d
e
f
lectio
n
.
T
h
is
p
r
o
d
u
ce
s
a
n
eg
ativ
e
p
itch
in
g
m
o
m
en
t,
ca
u
s
in
g
th
e
air
cr
af
t
to
p
itch
n
o
s
e
-
d
o
wn
.
T
h
ese
co
n
s
tr
ain
ts
en
s
u
r
e
th
at
th
e
elev
ato
r
d
ef
lectio
n
r
em
ain
s
with
in
th
e
p
h
y
s
ical
an
d
m
ec
h
an
ical
b
o
u
n
d
s
o
f
th
e
ac
tu
ato
r
,
wh
ile
th
e
lag
d
y
n
am
ics s
m
o
o
t
h
th
e
in
p
u
t
r
esp
o
n
s
e.
T
h
is
co
m
p
o
s
ite
m
o
d
el
p
lay
s
an
ess
en
tial
r
o
le
in
en
s
u
r
in
g
t
h
at
g
en
e
r
ated
co
n
tr
o
l
i
n
p
u
ts
ar
e
b
o
th
f
ea
s
ib
le
an
d
r
ef
lectiv
e
o
f
r
ea
lis
tic
ac
tu
ato
r
b
eh
av
io
r
.
T
h
e
co
m
p
lete
co
n
tr
o
l a
u
g
m
en
t
atio
n
s
y
s
tem
(
C
AS)
in
teg
r
ates
th
e
in
n
er
-
lo
o
p
p
itc
h
r
ate
co
n
tr
o
ller
with
th
e
n
o
n
lin
ea
r
air
cr
af
t
d
y
n
a
m
ic
s
an
d
ac
tu
ato
r
m
o
d
el
in
a
clo
s
ed
-
lo
o
p
co
n
f
ig
u
r
atio
n
.
As
illu
s
tr
ated
in
Fig
u
r
e
2
,
th
e
s
y
s
tem
is
d
esig
n
ed
to
r
eg
u
late
th
e
p
itch
r
ate
ab
o
u
t
a
s
p
ec
if
ied
tr
im
c
o
n
d
itio
n
.
T
h
e
er
r
o
r
s
ig
n
al
Δ
is
co
m
p
u
ted
b
y
s
u
b
tr
ac
tin
g
th
e
tr
im
r
ate
f
r
o
m
t
h
e
m
ea
s
u
r
e
d
p
itch
r
ate
an
d
is
th
e
n
f
ed
i
n
to
th
e
co
n
tr
o
ller
(
)
wh
ich
g
en
er
ates a
d
esire
d
elev
ato
r
d
ef
lectio
n
in
cr
em
e
n
t
Δ
.
Fig
u
r
e
2
.
B
lo
ck
d
iag
r
am
o
f
th
e
co
n
tr
o
l a
u
g
m
en
tatio
n
s
y
s
tem
f
o
r
p
itc
h
r
ate
s
tab
ilizatio
n
T
h
is
co
n
tr
o
l
s
ig
n
al
is
s
u
m
m
ed
with
a
tr
im
b
ias
Δ
,
to
f
o
r
m
th
e
to
tal
elev
ato
r
co
m
m
an
d
,
wh
ich
th
en
p
ass
es
th
r
o
u
g
h
th
e
ac
tu
ato
r
m
o
d
el
d
escr
ib
ed
ea
r
l
ier
.
T
h
e
air
c
r
af
t’
s
d
y
n
am
ic
r
es
p
o
n
s
e
is
o
b
s
er
v
e
d
th
r
o
u
g
h
b
o
th
th
e
p
itc
h
r
ate
an
d
an
g
le
o
f
attac
k
.
T
h
e
a
n
g
le
o
f
attac
k
is
f
u
r
th
e
r
f
e
d
b
ac
k
th
r
o
u
g
h
a
p
r
o
p
o
r
tio
n
al
g
ain
to
p
r
o
v
i
d
e
a
n
ad
d
itio
n
al
s
tab
ilizin
g
i
n
f
lu
en
ce
o
n
th
e
c
o
n
tr
o
ller
in
p
u
t.
T
h
is
au
g
m
en
ted
f
ee
d
b
ac
k
l
o
o
p
en
h
a
n
ce
s
p
itch
ax
is
s
tab
ilit
y
an
d
im
p
r
o
v
es
d
y
n
am
ic
r
e
s
p
o
n
s
e
b
y
s
h
ap
in
g
th
e
clo
s
ed
-
lo
o
p
b
eh
a
v
io
r
ar
o
u
n
d
t
h
e
tr
im
m
ed
o
p
er
at
in
g
co
n
d
itio
n
.
Mo
r
eo
v
e
r
,
th
e
a
ctu
ato
r
co
n
s
tr
ain
ts
em
b
ed
d
e
d
in
th
e
m
o
d
el
en
s
u
r
e
th
at
all
co
n
tr
o
l
in
p
u
ts
r
em
ai
n
with
in
p
h
y
s
ical
lim
its
.
T
h
e
a
r
ch
itectu
r
e
s
u
p
p
o
r
ts
m
o
d
u
lar
d
esig
n
an
d
e
n
ab
les
t
h
e
in
teg
r
atio
n
o
f
lear
n
in
g
-
b
as
ed
au
g
m
en
tatio
n
m
o
d
u
les
o
r
d
is
tu
r
b
an
ce
r
ejec
tio
n
m
ec
h
an
is
m
s
with
in
th
e
ex
is
tin
g
co
n
tr
o
l f
r
am
ewo
r
k
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
R
esid
u
a
l rein
fo
r
ce
men
t le
a
r
n
in
g
fo
r
d
is
tu
r
b
a
n
ce
-
r
esil
ien
t c
o
n
tr
o
l
…
(
A
b
o
l
a
n
le
A
d
etifa
)
1179
2
.
2
.
Co
ntr
o
ller
d
esig
n
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
co
n
tr
o
l
ar
c
h
itectu
r
e
d
e
v
elo
p
e
d
f
o
r
s
tab
ilizin
g
an
d
r
eg
u
l
atin
g
th
e
lo
n
g
itu
d
in
al
d
y
n
am
ics
o
f
th
e
air
cr
af
t.
T
h
r
ee
co
n
tr
o
ller
s
ar
e
co
n
s
id
er
ed
:
a
class
ical
PID
c
o
n
tr
o
ller
,
a
DDPG
-
b
ased
co
n
tr
o
ller
,
an
d
a
r
esid
u
a
l
lear
n
in
g
ar
ch
itectu
r
e
th
at
co
m
b
in
es
PID
co
n
tr
o
l
with
DDPG.
T
h
e
o
b
jectiv
e
in
all
ca
s
es
i
s
to
en
s
u
r
e
s
tab
le
p
itch
-
r
ate
r
eg
u
latio
n
,
ac
c
u
r
ate
r
e
f
er
en
ce
tr
ac
k
i
n
g
,
a
n
d
r
o
b
u
s
tn
e
s
s
ag
ain
s
t
ex
ter
n
al
d
is
tu
r
b
an
ce
s
.
2
.
2
.
1
.
P
I
D
co
ntr
o
l d
esig
n
T
h
e
p
r
o
p
o
r
tio
n
al
-
i
n
teg
r
al
-
d
e
r
i
v
ativ
e
(
PID
)
co
n
tr
o
ller
is
ad
o
p
ted
as
a
b
aselin
e
co
n
tr
o
l
s
tr
at
eg
y
d
u
e
to
its
s
im
p
licity
an
d
wid
esp
r
ea
d
u
s
e
in
f
lig
h
t
c
o
n
tr
o
l
ap
p
lica
tio
n
s
[
1
9
]
.
I
n
th
is
s
tu
d
y
,
th
e
PID
co
n
tr
o
ller
is
d
esig
n
ed
to
r
eg
u
late
th
e
p
itch
r
ate
b
y
g
en
er
atin
g
th
e
elev
ato
r
d
ef
lectio
n
co
m
m
a
n
d
b
ased
o
n
th
e
tr
ac
k
in
g
er
r
o
r
b
etwe
en
th
e
d
esire
d
a
n
d
m
ea
s
u
r
ed
p
itch
r
ates
[
2
0
]
.
L
et
th
e
tr
ac
k
in
g
er
r
o
r
b
e
d
ef
in
ed
as
(
1
3
)
.
(
)
=
r
e
f
(
)
−
(
)
,
(
1
3
)
w
h
er
e
(
)
d
en
o
tes
th
e
r
ef
e
r
en
ce
p
itch
-
r
ate
co
m
m
a
n
d
an
d
(
)
r
ep
r
esen
ts
th
e
m
ea
s
u
r
ed
p
itch
r
a
te.
T
h
e
PID
co
n
tr
o
l la
w
is
ex
p
r
ess
ed
a
s
(
1
4
)
.
PID
(
)
=
(
)
+
∫
(
)
0
+
(
)
(
1
4
)
wh
er
e
,
,
an
d
d
en
o
te
th
e
p
r
o
p
o
r
tio
n
al,
in
teg
r
al,
an
d
d
e
r
iv
at
iv
e
g
ain
s
,
r
esp
ec
tiv
ely
.
T
h
e
co
n
tr
o
ller
g
ain
s
ar
e
tu
n
ed
u
s
in
g
th
e
M
AT
L
AB
PID
tu
n
in
g
to
o
l
to
a
ch
iev
e
a
s
atis
f
ac
to
r
y
tr
ad
e
-
o
f
f
b
etwe
en
r
is
e
tim
e,
o
v
er
s
h
o
o
t,
an
d
s
tead
y
-
s
tate
er
r
o
r
.
Alth
o
u
g
h
th
e
PID
co
n
tr
o
ller
o
f
f
er
s
ac
ce
p
tab
le
p
er
f
o
r
m
an
ce
u
n
d
er
n
o
m
i
n
al
o
p
er
atin
g
co
n
d
itio
n
s
,
its
f
ix
ed
-
g
ai
n
s
tr
u
ctu
r
e
lim
its
ad
a
p
tab
ilit
y
wh
en
th
e
air
c
r
af
t is su
b
jecte
d
to
n
o
n
lin
ea
r
ities
,
m
o
d
elin
g
u
n
ce
r
tain
ties
,
o
r
ex
ter
n
al
d
is
tu
r
b
an
ce
s
.
T
h
ese
lim
itatio
n
s
m
o
tiv
ate
th
e
u
s
e
o
f
lear
n
in
g
-
b
ased
c
o
n
tr
o
l stra
t
eg
ies.
2
.
2
.
2
.
DDP
G
-
ba
s
ed
deep
re
info
rc
em
ent
lea
rning
co
ntr
o
l
Dee
p
d
eter
m
in
is
tic
p
o
licy
g
r
ad
ien
t
(
DDPG)
is
a
m
o
d
el
-
f
r
ee
,
o
f
f
-
p
o
licy
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
alg
o
r
ith
m
d
esig
n
e
d
f
o
r
c
o
n
tin
u
o
u
s
ac
tio
n
s
p
ac
es,
m
ak
in
g
it
a
s
u
itab
le
ch
o
ice
f
o
r
air
cr
a
f
t
c
o
n
tr
o
l
ap
p
licatio
n
s
[
1
9
]
,
[
2
0
]
.
I
t
co
m
b
i
n
es
th
e
a
cto
r
-
cr
itic
ar
c
h
itectu
r
e
with
d
ete
r
m
in
is
tic
p
o
licy
u
p
d
ates,
wh
er
e
th
e
ac
t
o
r
n
etwo
r
k
p
r
o
p
o
s
es
co
n
tr
o
l
ac
tio
n
s
an
d
t
h
e
cr
itic
ev
alu
ates
th
eir
ex
p
ec
ted
p
er
f
o
r
m
an
ce
[
2
1
]
.
B
y
le
v
e
r
ag
in
g
d
ee
p
n
eu
r
al
n
etwo
r
k
s
to
a
p
p
r
o
x
im
ate
b
o
th
th
e
p
o
licy
an
d
v
al
u
e
f
u
n
ct
io
n
s
,
DDPG
en
ab
les
th
e
lear
n
in
g
o
f
co
m
p
lex
,
n
o
n
lin
ea
r
co
n
tr
o
l
s
tr
ateg
ies
d
ir
ec
tly
f
r
o
m
in
ter
ac
tio
n
with
th
e
en
v
ir
o
n
m
en
t
[
2
2
]
,
[
2
3
]
.
I
t
s
ab
ilit
y
to
h
an
d
le
h
ig
h
-
d
im
e
n
s
io
n
al
s
tate
s
p
ac
es
an
d
p
r
o
d
u
ce
s
m
o
o
th
,
r
ea
l
-
v
a
lu
ed
ac
tio
n
s
m
ak
es
it
p
ar
ticu
lar
ly
ef
f
ec
tiv
e
f
o
r
s
y
s
tem
s
s
u
ch
as
f
ix
ed
-
win
g
air
cr
af
t,
wh
er
e
co
n
tin
u
o
u
s
an
d
p
r
ec
is
e
ac
tu
atio
n
is
r
eq
u
ir
ed
.
T
h
e
tr
ain
in
g
f
r
am
ewo
r
k
o
f
th
e
DDPG
ag
en
t
s
h
o
win
g
th
e
ac
to
r
-
cr
itic
n
etwo
r
k
s
,
ex
p
er
ien
ce
r
e
p
lay
,
an
d
in
ter
ac
tio
n
with
th
e
en
v
ir
o
n
m
en
t is sh
o
wn
in
Fig
u
r
e
3
ac
co
r
d
in
g
to
Af
za
li
et
a
l.
[
2
4
]
.
T
o
f
o
r
m
alize
th
e
co
n
tr
o
l
p
r
o
b
l
em
,
th
e
lear
n
in
g
task
is
f
r
am
e
d
as
a
Ma
r
k
o
v
d
ec
is
io
n
p
r
o
ce
s
s
(
MD
P),
ch
ar
ac
ter
ized
b
y
a
d
e
f
in
ed
s
tate
s
p
ac
e,
ac
tio
n
s
p
ac
e,
en
v
ir
o
n
m
en
t
d
y
n
am
ics,
an
d
tr
an
s
it
io
n
s
tr
u
ctu
r
e
[
2
5
]
.
W
ith
in
th
is
f
r
am
ewo
r
k
,
th
e
ag
en
t
o
b
s
er
v
es
th
e
cu
r
r
e
n
t
s
tat
e
o
f
th
e
s
y
s
tem
an
d
s
elec
ts
an
ap
p
r
o
p
r
iate
co
n
tr
o
l
ac
tio
n
to
m
ax
im
ize
cu
m
u
lativ
e
f
u
tu
r
e
r
ewa
r
d
s
.
T
h
e
ac
t
io
n
,
co
r
r
esp
o
n
d
i
n
g
to
th
e
e
lev
ato
r
d
ef
lectio
n
co
m
m
an
d
,
is
th
en
ap
p
lied
to
t
h
e
air
cr
af
t m
o
d
el
to
i
n
f
lu
en
ce
its
lo
n
g
itu
d
in
al
m
o
tio
n
.
a.
State
r
ep
r
esen
tatio
n
: T
h
e
s
tate
v
ec
to
r
is
d
ef
in
e
d
as
(
1
5
)
:
=
[
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
˙
(
)
,
(
−
1
)
]
(
1
5
)
w
h
er
e
is
th
e
p
itch
r
ate,
is
th
e
p
itch
an
g
le,
is
th
e
an
g
le
o
f
at
tack
,
is
th
e
f
o
r
war
d
v
elo
city
,
(
)
=
(
)
−
(
)
is
th
e
tr
ac
k
in
g
er
r
o
r
,
(
)
̇
is
th
e
er
r
o
r
d
e
r
iv
ativ
e,
(
−
1
)
is
th
e
p
r
ev
io
u
s
co
n
tr
o
l in
p
u
t.
b.
Actio
n
s
p
ac
e
an
d
co
n
tr
o
l
s
ig
n
al:
T
h
e
ac
tio
n
g
en
e
r
ated
b
y
th
e
ag
en
t
co
r
r
esp
o
n
d
s
to
th
e
elev
ato
r
co
n
tr
o
l
co
m
m
an
d
,
=
(
)
(
1
6
)
wh
ich
is
ap
p
lied
d
ir
ec
tly
t
o
th
e
lo
n
g
itu
d
in
al
air
cr
a
f
t
m
o
d
el.
Du
r
in
g
tr
ai
n
in
g
,
ex
p
lo
r
atio
n
n
o
is
e
is
ad
d
ed
to
en
co
u
r
a
g
e
s
u
f
f
icien
t e
x
p
lo
r
ati
o
n
:
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
7
5
-
1
1
8
7
1180
=
(
∣
)
+
,
(
1
7
)
w
h
er
e
(
∙
)
d
en
o
tes th
e
ac
to
r
n
etwo
r
k
an
d
is
Gau
s
s
ian
n
o
is
e.
Fig
u
r
e
3
.
Ar
c
h
itectu
r
e
an
d
tr
ai
n
in
g
wo
r
k
f
lo
w
o
f
th
e
DDPG
b
ased
co
n
tr
o
ller
c.
R
ewa
r
d
d
esig
n
a
n
d
t
r
an
s
ien
t
a
war
e
s
h
ap
in
g
:
T
o
m
itig
ate
a
s
lo
w
tr
an
s
ien
t
r
esp
o
n
s
e,
th
e
r
ew
ar
d
f
u
n
ctio
n
is
d
esig
n
ed
to
ex
p
licitly
en
co
d
e
b
o
th
tr
a
n
s
ien
t
an
d
s
tead
y
-
s
tate
p
er
f
o
r
m
an
ce
o
b
jectiv
es.
I
n
s
tead
o
f
em
p
h
asizin
g
o
n
ly
s
tead
y
-
s
tate
er
r
o
r
m
in
im
izatio
n
,
th
e
p
r
o
p
o
s
ed
r
ewa
r
d
in
co
r
p
o
r
ates
p
e
n
alties
r
elate
d
to
r
is
e
tim
e,
s
ettlin
g
b
eh
av
io
r
,
an
d
ex
ce
s
s
iv
e
co
n
tr
o
l a
ctiv
it
y
.
T
h
e
in
s
tan
tan
eo
u
s
r
ewa
r
d
is
d
ef
in
ed
as
(
1
8
)
:
=
−
(
1
2
(
)
+
2
e
2
(
)
+
3
Δ
2
(
)
+
4
|
(
)
|
+
5
out
(
)
)
(
1
8
)
w
h
er
e
(
)
is
th
e
tr
ac
k
in
g
er
r
o
r
,
(
)
̇
p
en
alize
s
s
lo
w
o
r
o
s
cillato
r
y
tr
an
s
ien
t
b
eh
av
i
o
r
,
Δ
(
)
=
(
)
−
(
−
1
)
d
is
co
u
r
ag
es
ab
r
u
p
t
ac
tu
ato
r
m
o
tio
n
,
|
(
)
|
lim
its
ex
ce
s
s
iv
e
co
n
tr
o
l
m
ag
n
itu
d
e,
(
)
is
an
in
d
icato
r
th
at
p
e
n
alize
s
tim
e
s
p
en
t o
u
ts
id
e
a
p
r
ed
ef
i
n
ed
to
ler
an
ce
b
an
d
ar
o
u
n
d
t
h
e
r
ef
e
r
en
c
e.
d.
R
ef
er
en
ce
m
o
d
el
-
b
ased
r
ewa
r
d
s
h
ap
in
g
:
T
o
f
u
r
th
er
g
u
id
e
t
h
e
lear
n
in
g
p
r
o
ce
s
s
to
war
d
d
e
s
ir
ab
le
tr
an
s
ien
t
ch
ar
ac
ter
is
tics
,
a
r
ef
er
en
ce
-
m
o
d
el
tr
ac
k
in
g
f
o
r
m
u
latio
n
is
in
tr
o
d
u
ce
d
.
A
s
ec
o
n
d
-
o
r
d
er
r
ef
er
en
ce
m
o
d
el
with
d
esire
d
n
atu
r
al
f
r
eq
u
e
n
cy
an
d
d
a
m
p
in
g
r
atio
is
d
ef
in
e
d
as
(
1
9
)
:
¨
+
2
˙
+
2
=
2
c
md
(
1
9
)
w
h
er
e
r
ep
r
esen
ts
th
e
id
ea
l
p
itch
-
r
ate
r
esp
o
n
s
e.
T
h
e
tr
ac
k
in
g
er
r
o
r
u
s
ed
i
n
th
e
r
ewa
r
d
ca
n
th
en
b
e
r
ed
ef
in
ed
as
(
2
0
)
:
(
)
=
(
)
−
(
)
(
2
0
)
wh
ich
ex
p
licitly
en
c
o
u
r
ag
es
t
h
e
lear
n
ed
p
o
licy
to
m
im
ic
a
well
-
d
am
p
ed
s
ec
o
n
d
-
o
r
d
er
r
esp
o
n
s
e
co
m
m
o
n
ly
u
s
ed
in
f
lig
h
t
co
n
tr
o
l
d
esig
n
.
T
h
is
r
ef
er
e
n
ce
-
m
o
d
el
f
o
r
m
u
l
atio
n
h
elp
s
r
e
co
n
cile
f
ast
tr
an
s
ien
t
r
esp
o
n
s
e
with
s
tab
ilit
y
an
d
s
m
o
o
th
n
ess
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
R
esid
u
a
l rein
fo
r
ce
men
t le
a
r
n
in
g
fo
r
d
is
tu
r
b
a
n
ce
-
r
esil
ien
t c
o
n
tr
o
l
…
(
A
b
o
l
a
n
le
A
d
etifa
)
1181
e.
C
r
itic a
n
d
ac
to
r
u
p
d
ates
:
T
h
e
cr
itic n
etwo
r
k
esti
m
ates th
e
ac
tio
n
–
v
alu
e
f
u
n
ctio
n
(
,
∣
)
(
2
1
)
an
d
is
tr
ain
ed
b
y
m
in
im
izin
g
t
h
e
tem
p
o
r
al
-
d
if
f
er
e
n
ce
lo
s
s
with
tar
g
et
v
alu
es g
i
v
en
in
(
2
2
)
a
n
d
(
2
3
)
.
=
1
∑
(
−
(
,
∣
)
)
2
=
1
(
2
2
)
=
+
′
(
+
1
,
′
(
+
1
)
)
(
2
3
)
T
h
e
ac
to
r
p
ar
am
eter
s
ar
e
u
p
d
a
ted
v
ia
th
e
d
eter
m
in
is
tic
p
o
licy
g
r
ad
ie
n
t
an
d
s
o
f
t ta
r
g
et
u
p
d
a
tes ar
e
ap
p
lied
as sh
o
wn
in
(
2
4
)
an
d
(
2
5
)
-
(
2
6
)
.
≈
1
∑
(
,
∣
∣
)
|
=
(
)
(
)
(
2
4
)
′
←
(
1
−
)
′
+
(
2
5
)
′
←
(
1
−
)
′
+
(
2
6
)
T
o
f
u
r
th
er
im
p
r
o
v
e
tr
ai
n
in
g
s
tab
ilit
y
an
d
tr
an
s
ien
t
r
esp
o
n
s
e,
a
r
esid
u
al
lear
n
in
g
ar
ch
itectu
r
e
is
ad
o
p
ted
.
I
n
s
tead
o
f
d
ir
ec
tly
lear
n
i
n
g
t
h
e
f
u
ll
c
o
n
tr
o
l
s
ig
n
al,
t
h
e
DDPG
ag
en
t
lea
r
n
s
a
co
r
r
ec
tiv
e
r
e
s
id
u
al
o
n
to
p
o
f
a
b
aselin
e
PID
co
n
tr
o
ller
.
T
h
e
t
o
tal
co
n
tr
o
l in
p
u
t is d
ef
in
e
d
as
:
(
)
=
(
)
+
(
)
(
2
7
)
wh
er
e
(
)
p
r
o
v
id
es
b
aselin
e
s
tab
ilizatio
n
an
d
(
)
is
th
e
lear
n
ed
r
esid
u
al.
I
n
ad
d
itio
n
to
r
e
war
d
s
h
ap
in
g
,
p
h
y
s
ical
an
d
o
p
er
ati
o
n
al
co
n
s
tr
ain
ts
ar
e
en
f
o
r
ce
d
d
ir
ec
tly
with
in
th
e
en
v
ir
o
n
m
e
n
t
d
y
n
am
ics
r
ath
er
th
an
b
ein
g
h
an
d
led
s
o
lely
th
r
o
u
g
h
p
en
alties.
T
h
ese
in
clu
d
e
ac
tu
ato
r
s
atu
r
atio
n
lim
its
:
|
|
≤
,
,
ac
tu
ato
r
r
ate
lim
its
:
|
Δ
|
≤
̇
an
d
s
af
ety
en
v
elo
p
es
o
n
,
an
d
.
W
h
en
th
ese
lim
its
ar
e
v
io
lated
,
th
e
en
v
ir
o
n
m
e
n
t
clip
s
th
e
co
n
tr
o
l
s
ig
n
al
o
r
ter
m
in
ates
th
e
ep
is
o
d
e.
T
h
is
s
ep
ar
atio
n
o
f
s
af
ety
co
n
s
tr
ain
ts
f
r
o
m
r
ewa
r
d
s
h
ap
in
g
im
p
r
o
v
es lea
r
n
i
n
g
s
tab
ilit
y
an
d
en
s
u
r
es p
h
y
s
ically
r
ea
lis
tic
co
n
tr
o
l b
e
h
av
io
r
.
Ov
er
all,
th
e
p
r
o
p
o
s
ed
r
esid
u
al
DDPG
f
r
am
ewo
r
k
co
m
b
in
es
th
e
s
tab
ilit
y
o
f
class
ical
co
n
tr
o
l
with
th
e
ad
ap
tab
ilit
y
o
f
d
ata
-
d
r
iv
en
l
ea
r
n
in
g
,
r
esu
ltin
g
in
im
p
r
o
v
ed
tr
an
s
ien
t
p
er
f
o
r
m
an
ce
,
en
h
an
ce
d
d
is
tu
r
b
a
n
ce
r
ejec
tio
n
,
an
d
b
etter
g
en
er
aliz
atio
n
ac
r
o
s
s
o
p
er
atin
g
co
n
d
iti
o
n
s
co
m
p
ar
e
d
with
b
o
th
s
tan
d
alo
n
e
PID
an
d
p
u
r
e
DDPG
co
n
tr
o
ller
s
.
T
h
e
tr
ai
n
in
g
p
a
r
am
eter
s
u
s
ed
f
o
r
th
e
r
ein
f
o
r
ce
m
e
n
t
lear
n
in
g
ag
e
n
ts
ar
e
s
u
m
m
ar
ize
d
i
n
T
ab
le
1
.
T
h
e
lear
n
in
g
cu
r
v
es
s
h
o
wn
in
Fig
u
r
e
4
illu
s
tr
ate
th
e
t
r
ain
in
g
p
r
o
g
r
ess
io
n
o
f
th
e
ag
e
n
ts
,
wh
er
e
Fig
u
r
e
4
(
a)
p
r
esen
ts
th
e
lear
n
in
g
b
eh
a
v
io
r
o
f
th
e
s
tan
d
alo
n
e
DDPG
-
q
C
AS
co
n
tr
o
ller
an
d
Fig
u
r
e
4
(
b
)
s
h
o
ws
th
e
tr
ain
in
g
p
er
f
o
r
m
a
n
ce
o
f
th
e
R
es
-
DDPG
-
q
C
AS
ag
en
t.
T
h
e
r
esid
u
al
ag
e
n
t
d
em
o
n
s
tr
ates
a
m
o
r
e
s
tab
le
an
d
co
n
s
is
ten
t
im
p
r
o
v
em
en
t
in
c
u
m
u
lativ
e
r
ewa
r
d
d
u
r
in
g
tr
ain
i
n
g
,
in
d
icatin
g
m
o
r
e
ef
f
icien
t
lear
n
in
g
d
u
e
to
th
e
s
tab
ilizin
g
in
f
lu
en
ce
o
f
th
e
PI
D
b
aselin
e.
T
ab
le
1
.
DDPG
tr
ain
in
g
p
ar
a
m
eter
s
P
a
r
a
me
t
e
r
V
a
l
u
e
M
a
x
i
m
u
m
e
p
i
s
o
d
e
s
1
0
0
0
Ep
i
s
o
d
e
d
u
r
a
t
i
o
n
40s
S
a
mp
l
e
t
i
m
e
0
.
1
s
D
i
sco
u
n
t
f
a
c
t
o
r
0
.
9
9
M
i
n
i
-
b
a
t
c
h
s
i
z
e
64
R
e
p
l
a
y
b
u
f
f
e
r
si
z
e
1
×
10
6
Ta
r
g
e
t
s
mo
o
t
h
i
n
g
f
a
c
t
o
r
0
.
0
0
1
A
c
t
o
r
l
e
a
r
n
i
n
g
r
a
t
e
1
×
10
−
4
C
r
i
t
i
c
l
e
a
r
n
i
n
g
r
a
t
e
1
×
10
−
3
I
n
i
t
i
a
l
n
o
i
se
st
d
.
d
e
v
0
.
6
N
o
i
se
d
e
c
a
y
r
a
t
e
1
×
10
−
5
H
i
d
d
e
n
l
a
y
e
r
s
3
N
e
u
r
o
n
s
p
e
r
l
a
y
e
r
3
5
0
A
c
t
i
v
a
t
i
o
n
f
u
n
c
t
i
o
n
R
e
LU
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
7
5
-
1
1
8
7
1182
(
a)
(
b
)
Fig
u
r
e
4
.
T
r
ain
in
g
c
u
r
v
es o
f
d
ee
p
r
ein
f
o
r
ce
m
en
t le
ar
n
in
g
a
g
en
ts
,
(
a)
DDPG
-
q
C
AS a
n
d
(
b
)
R
es
-
DDP
G
-
q
C
AS
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
T
h
is
s
ec
tio
n
p
r
esen
ts
th
e
r
esu
l
ts
o
f
th
e
tim
e
r
esp
o
n
s
e
an
al
y
s
is
,
as
well
as
th
e
p
itch
r
ate
tr
a
ck
in
g
a
n
d
d
is
tu
r
b
an
ce
r
ejec
tio
n
p
er
f
o
r
m
an
ce
o
f
b
o
th
th
e
class
ical
co
n
tr
o
ller
an
d
th
e
d
ee
p
r
ei
n
f
o
r
ce
m
e
n
t
lear
n
in
g
co
n
tr
o
ller
.
3
.
1
.
T
im
e
r
esp
o
ns
e
Fig
u
r
e
5
p
r
esen
ts
th
e
s
tep
r
esp
o
n
s
es
o
f
th
e
th
r
ee
c
o
n
tr
o
ller
s
,
wh
ile
T
ab
le
2
s
u
m
m
ar
izes
th
eir
co
r
r
esp
o
n
d
in
g
tim
e
-
d
o
m
ain
p
er
f
o
r
m
a
n
ce
in
d
ices.
As
illu
s
tr
ated
in
Fig
u
r
e
5
(
a)
,
th
e
PID
-
q
C
AS
ex
h
ib
its
th
e
f
astes
t
r
i
s
e
tim
e
(
=
0
.
2390
)
b
u
t
s
u
f
f
er
s
f
r
o
m
a
v
er
y
lar
g
e
o
v
e
r
s
h
o
o
t
(
=
18
.
9541
%
)
an
d
n
o
ticea
b
le
o
s
cillato
r
y
tr
an
s
ien
t
b
eh
av
i
o
r
.
T
h
ese
ch
ar
ac
ter
is
tics
r
esu
lt
i
n
r
elativ
ely
h
ig
h
er
r
o
r
m
etr
ic
s
(
MA
E
,
I
SE,
an
d
MSSE)
,
in
d
icatin
g
an
a
g
g
r
ess
iv
e
y
et
p
o
o
r
ly
d
am
p
e
d
r
esp
o
n
s
e.
As
illu
s
tr
ated
in
Fig
u
r
e
5
(
b
)
,
th
e
DDPG
-
q
C
AS
s
ig
n
if
ican
tly
r
e
d
u
ce
s
o
v
er
s
h
o
o
t
(
=
0
.
6580
%
)
an
d
p
r
o
d
u
ce
s
a
s
m
o
o
th
r
esp
o
n
s
e;
h
o
wev
er
,
th
is
im
p
r
o
v
e
m
en
t
co
m
es
at
th
e
e
x
p
en
s
e
o
f
v
er
y
s
lo
w
d
y
n
am
ics,
with
a
r
is
e
tim
e
o
f
=
9
.
4061
an
d
a
s
ett
lin
g
tim
e
o
f
=
17
.
7502
.
T
h
e
p
r
o
lo
n
g
ed
tr
an
s
ien
t
r
esp
o
n
s
e
ex
p
lain
s
th
e
lar
g
e
I
SE
v
alu
e
(
0
.
8
8
3
8
)
,
d
esp
ite
m
o
d
er
ate
MA
E
an
d
MSSE
v
alu
es.
I
n
co
n
tr
ast,
Fig
u
r
e
5
(
c)
s
h
o
ws
th
at
th
e
p
r
o
p
o
s
ed
R
es
-
DDPG
-
q
C
AS
ac
h
iev
es
a
m
o
r
e
f
av
o
r
ab
le
b
ala
n
ce
b
etwe
en
r
esp
o
n
s
e
s
p
ee
d
an
d
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
R
esid
u
a
l rein
fo
r
ce
men
t le
a
r
n
in
g
fo
r
d
is
tu
r
b
a
n
ce
-
r
esil
ien
t c
o
n
tr
o
l
…
(
A
b
o
l
a
n
le
A
d
etifa
)
1183
ac
cu
r
ac
y
.
I
t
m
ai
n
tain
s
a
f
ast
r
is
e
tim
e
(
=
0
.
4020
)
,
th
e
s
h
o
r
test
s
ettlin
g
tim
e
(
=
0
.
9825
)
,
an
d
m
in
im
al
o
v
er
s
h
o
o
t
(
=
0
.
2731
%
)
.
Fu
r
th
er
m
o
r
e,
it
ac
h
iev
es
th
e
lo
west
er
r
o
r
m
etr
ics,
with
MA
E
=
0
.
0043
,
I
SE
=
0
.
0212
,
an
d
MSSE
=
0
.
0061
,
d
em
o
n
s
t
r
atin
g
s
u
p
er
i
o
r
tr
a
n
s
ien
t
p
er
f
o
r
m
an
ce
a
n
d
tr
ac
k
in
g
ac
cu
r
ac
y
co
m
p
ar
ed
with
b
o
th
PID
-
q
C
AS
an
d
DDPG
-
q
C
AS.
T
h
ese
r
esu
lts
co
n
f
ir
m
th
at
r
esid
u
al
lear
n
in
g
ef
f
ec
tiv
ely
c
o
m
b
in
es
th
e
f
ast
tr
an
s
ien
t
r
esp
o
n
s
e
o
f
PID
c
o
n
tr
o
l
with
th
e
s
m
o
o
th
an
d
r
o
b
u
s
t
b
eh
av
io
r
o
f
r
ein
f
o
r
ce
m
e
n
t le
ar
n
in
g
.
(
a)
(
b
)
(
c)
Fig
u
r
e
5
.
Step
r
esp
o
n
s
e
f
o
r
(
a)
PID
-
q
C
AS,
(
b
)
DDPG
-
q
C
AS
,
an
d
(
c
)
R
es
-
DDPG
-
q
C
AS
c
o
n
tr
o
ller
s
T
ab
le
2
.
C
o
m
p
a
r
is
o
n
o
f
tim
e
-
d
o
m
ain
p
er
f
o
r
m
an
ce
m
et
r
ics f
o
r
PID
-
q
C
AS,
DDPG
-
q
C
AS,
an
d
R
es
-
DDP
G
-
q
C
AS
C
o
n
tr
o
ller
s
M
e
t
h
o
d
(
s)
(
)
(
%
)
P
I
D
-
q
C
A
S
0
.
2
3
9
0
1
.
4
1
6
7
1
8
.
9
5
4
1
0
.
1
0
2
5
0
.
1
1
0
6
0
.
0
8
7
9
D
D
P
G
-
q
C
A
S
9
.
4
0
6
1
1
7
.
7
5
0
2
0
.
6
5
8
0
0
.
0
1
0
0
0
.
8
8
3
8
0
.
0
1
3
8
Res
-
D
D
P
G
-
q
C
A
S
0
.
4
0
2
0
0
.
9
8
2
5
0
.
2
7
1
3
0
.
0
0
4
3
0
.
0
2
1
2
0
.
0
0
6
1
3
.
2
.
P
i
t
ch
ra
t
e
co
mm
a
nd
t
ra
ck
ing
Fig
u
r
e
6
illu
s
tr
ates
th
e
p
itch
-
r
ate
co
m
m
an
d
tr
ac
k
in
g
p
e
r
f
o
r
m
an
ce
o
f
th
e
PID
-
q
C
AS,
DDPG
-
q
C
AS,
an
d
R
es
-
DDPG
-
q
C
AS
co
n
tr
o
ller
s
u
n
d
er
s
u
cc
ess
iv
e
s
tep
ch
an
g
es
in
th
e
r
e
f
er
en
ce
s
i
g
n
al.
As
s
h
o
wn
in
Fig
u
r
e
6
(
a)
,
th
e
PID
-
q
C
AS
r
esp
o
n
d
s
r
ap
id
l
y
to
co
m
m
an
d
ch
an
g
es
b
u
t
ex
h
ib
its
n
o
ticea
b
le
o
v
er
s
h
o
o
t
an
d
o
s
cillatio
n
s
,
p
ar
ticu
lar
ly
at
th
e
r
is
in
g
a
n
d
f
allin
g
e
d
g
es
o
f
t
h
e
co
m
m
an
d
.
T
h
ese
o
s
cillatio
n
s
in
d
icate
lim
ited
d
am
p
in
g
an
d
r
esu
lt
in
in
cr
ea
s
ed
tr
an
s
ien
t
tr
ac
k
in
g
er
r
o
r
.
I
n
co
n
tr
ast,
Fig
u
r
e
6
(
b
)
s
h
o
ws
th
at
th
e
DDPG
-
q
C
AS
p
r
o
d
u
ce
s
a
m
u
c
h
s
m
o
o
t
h
er
r
e
s
p
o
n
s
e
with
m
in
im
al
o
s
cillati
o
n
;
h
o
wev
er
,
its
co
n
v
er
g
en
ce
to
th
e
c
o
m
m
an
d
ed
v
alu
e
is
r
elativ
ely
s
lo
w,
lead
in
g
to
a
s
lu
g
g
is
h
tr
a
n
s
ien
t d
u
r
i
n
g
b
o
th
t
h
e
s
tep
-
u
p
an
d
s
tep
-
d
o
wn
tr
an
s
itio
n
s
.
T
h
e
p
r
o
p
o
s
ed
R
es
-
DDPG
-
q
C
AS
ac
h
iev
es
a
m
o
r
e
f
av
o
r
ab
l
e
tr
ad
e
-
o
f
f
b
etwe
en
s
p
ee
d
a
n
d
s
tab
ilit
y
.
As
s
h
o
wn
in
Fig
u
r
e
6
(
c
)
,
it
c
lo
s
ely
f
o
llo
ws
th
e
r
ef
er
e
n
ce
with
n
eg
lig
ib
le
o
v
e
r
s
h
o
o
t
a
n
d
with
o
u
t
o
s
cillato
r
y
b
eh
av
io
r
,
wh
ile
m
ain
tain
in
g
a
s
ig
n
if
ican
tly
f
aster
r
esp
o
n
s
e
th
an
th
e
s
tan
d
alo
n
e
DDPG
co
n
tr
o
ller
.
T
h
e
s
m
o
o
th
tr
an
s
itio
n
s
an
d
r
ap
id
s
ettlin
g
d
em
o
n
s
tr
ate
th
at
th
e
r
esid
u
al
le
ar
n
in
g
ar
ch
itectu
r
e
ef
f
ec
tiv
ely
co
m
b
in
es
th
e
f
ast
r
esp
o
n
s
e
o
f
th
e
PID
b
aselin
e
with
th
e
d
am
p
in
g
an
d
r
o
b
u
s
tn
ess
p
r
o
v
id
ed
b
y
th
e
lear
n
ed
r
esid
u
al
p
o
licy
,
r
esu
ltin
g
in
s
u
p
er
i
o
r
p
itch
-
r
ate
co
m
m
an
d
tr
ac
k
in
g
p
er
f
o
r
m
a
n
ce
.
(
a)
(
b
)
(
c)
Fig
u
r
e
6
.
Pit
ch
r
ate
tr
ac
k
in
g
f
o
r
(
a)
PID
-
q
C
AS,
(
b
)
DDPG
-
q
C
AS,
an
d
(
c)
R
es
-
DDPG
-
q
C
A
S
c
o
n
tr
o
ller
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
7
5
-
1
1
8
7
1184
3
.
3
.
Dis
t
urba
nce
r
ej
ec
t
io
n
I
n
th
is
test
,
t
h
e
air
c
r
af
t
is
r
eq
u
ir
ed
t
o
m
ai
n
tain
ze
r
o
p
itch
-
r
ate
d
ev
iatio
n
in
th
e
p
r
esen
ce
o
f
a
lar
g
e
p
itch
in
g
m
o
m
en
t
d
is
tu
r
b
a
n
ce
o
f
3
0
k
Nm
ap
p
lied
at
=
200
.
Fig
u
r
e
7
co
m
p
a
r
es
th
e
d
is
tu
r
b
an
ce
r
ejec
tio
n
p
er
f
o
r
m
an
ce
o
f
th
e
PID
-
q
C
AS,
DDPG
-
q
C
A
S,
an
d
R
es
-
D
DPG
-
q
C
AS
co
n
tr
o
ller
s
.
As
s
h
o
wn
in
Fig
u
r
e
7
(
a)
,
th
e
PID
-
q
C
AS
ex
h
ib
its
a
r
a
p
id
in
itial
r
esp
o
n
s
e
b
u
t
s
h
o
ws
n
o
ticea
b
le
o
s
cillatio
n
s
b
o
th
d
u
r
in
g
th
e
in
itial
tr
an
s
ien
t
an
d
f
o
llo
win
g
th
e
d
i
s
tu
r
b
an
ce
in
jectio
n
,
in
d
icatin
g
lim
ited
d
am
p
i
n
g
an
d
s
en
s
itiv
ity
to
th
e
e
x
ter
n
al
p
er
tu
r
b
atio
n
.
Alth
o
u
g
h
th
e
p
itch
r
ate
ev
en
tu
ally
r
etu
r
n
s
to
th
e
r
ef
er
en
ce
,
th
e
o
s
cillato
r
y
r
ec
o
v
er
y
im
p
lies
in
cr
ea
s
ed
tr
an
s
ien
t e
r
r
o
r
an
d
r
ed
u
ce
d
r
o
b
u
s
tn
ess
u
n
d
e
r
s
ev
er
e
d
is
tu
r
b
an
ce
c
o
n
d
itio
n
s
.
As
s
h
o
wn
in
Fig
u
r
e
7
(
b
)
,
th
e
DDPG
-
q
C
AS
d
em
o
n
s
tr
ates
s
m
o
o
th
er
b
eh
av
io
r
with
r
ed
u
ce
d
o
s
cillatio
n
s
;
h
o
wev
er
,
its
r
ec
o
v
er
y
f
r
o
m
th
e
d
is
tu
r
b
an
ce
is
r
elativ
ely
s
lo
w,
an
d
a
n
o
ticea
b
le
d
ev
iatio
n
f
r
o
m
th
e
r
ef
e
r
en
ce
p
er
s
is
ts
f
o
r
a
l
o
n
g
er
d
u
r
atio
n
.
I
n
co
n
tr
ast,
F
ig
u
r
e
7
(
c)
s
h
o
ws
th
at
th
e
p
r
o
p
o
s
ed
R
es
-
DDPG
-
q
C
AS
ac
h
iev
es
th
e
m
o
s
t
ef
f
ec
tiv
e
d
is
tu
r
b
an
ce
r
ejec
tio
n
,
with
o
n
ly
a
s
m
all
tr
an
s
ien
t
d
ev
iatio
n
at
th
e
d
is
tu
r
b
an
ce
o
n
s
et
an
d
r
ap
id
r
esto
r
atio
n
o
f
th
e
p
itch
r
ate
to
th
e
r
ef
er
en
ce
v
alu
e.
T
h
e
r
es
p
o
n
s
e
r
em
ain
s
well
d
am
p
ed
an
d
f
r
ee
o
f
s
u
s
tain
ed
o
s
cillatio
n
s
,
co
n
f
ir
m
in
g
t
h
at
th
e
r
esid
u
al
lear
n
in
g
ar
c
h
itectu
r
e
en
h
an
ce
s
r
o
b
u
s
tn
ess
to
lar
g
e
e
x
ter
n
al
d
i
s
tu
r
b
an
ce
s
wh
ile
p
r
eser
v
in
g
s
tab
le
an
d
ac
c
u
r
ate
r
eg
u
latio
n
o
f
th
e
p
itch
r
ate.
(
a)
(
b
)
(
c)
Fig
u
r
e
7.
Dis
tu
r
b
a
n
ce
r
ejec
tio
n
p
er
f
o
r
m
an
ce
f
o
r
(
a)
PID
-
q
C
AS,
(
b
)
DDPG
-
q
C
AS,
an
d
(
c)
R
es
-
DDPG
-
q
C
A
S
c
o
n
tr
o
ller
s
4.
CO
NCLU
SI
O
N
T
h
is
s
tu
d
y
p
r
esen
ted
a
d
is
tu
r
b
an
ce
-
r
esil
ien
t
co
n
tr
o
l
f
r
am
ewo
r
k
f
o
r
air
cr
af
t
p
itch
-
r
ate
r
eg
u
latio
n
b
ased
o
n
r
esid
u
al
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
in
teg
r
ate
d
with
in
a
class
ical
co
n
tr
o
l
ar
ch
itectu
r
e.
T
h
e
p
r
o
p
o
s
ed
ap
p
r
o
ac
h
co
m
b
in
es
a
s
tab
ilizi
n
g
PID
co
n
tr
o
ller
with
a
DD
PG
ag
en
t
th
at
lear
n
s
a
b
o
u
n
d
ed
r
esid
u
al
co
n
tr
o
l
s
ig
n
al.
B
y
au
g
m
en
tin
g
r
at
h
er
t
h
an
r
ep
lacin
g
th
e
co
n
v
en
tio
n
a
l c
o
n
tr
o
ller
,
th
e
f
r
am
ewo
r
k
p
r
eser
v
es th
e
s
tab
ilit
y
an
d
r
eliab
ilit
y
o
f
class
ical
co
n
tr
o
l
wh
ile
in
t
r
o
d
u
cin
g
a
d
a
p
tiv
e,
d
ata
-
d
r
iv
en
co
m
p
e
n
s
atio
n
f
o
r
m
o
d
elin
g
u
n
ce
r
tain
ties
an
d
e
x
ter
n
al
d
is
t
u
r
b
an
ce
s
.
T
h
e
o
b
jectiv
e
o
f
th
is
wo
r
k
was
to
in
v
esti
g
ate
wh
eth
e
r
a
h
y
b
r
id
c
o
n
tr
o
l
s
tr
ateg
y
th
at
in
teg
r
ates
class
ical
f
ee
d
b
ac
k
co
n
t
r
o
l
with
d
ee
p
r
ein
f
o
r
ce
m
en
t
lea
r
n
in
g
ca
n
im
p
r
o
v
e
p
itch
-
r
at
e
r
eg
u
latio
n
u
n
d
er
u
n
ce
r
tain
an
d
d
is
tu
r
b
a
n
ce
-
r
ich
o
p
e
r
atin
g
co
n
d
itio
n
s
.
T
o
ac
h
i
ev
e
th
is
,
t
h
e
air
c
r
af
t
lo
n
g
itu
d
i
n
al
d
y
n
am
ics
wer
e
m
o
d
eled
an
d
lin
ea
r
ized
ar
o
u
n
d
a
tr
im
m
ed
o
p
er
atin
g
p
o
in
t,
an
d
a
co
n
tr
o
l
au
g
m
en
tatio
n
s
y
s
tem
in
co
r
p
o
r
atin
g
ac
tu
ato
r
d
y
n
am
ics
an
d
co
n
s
tr
ain
ts
was
d
ev
elo
p
ed
.
T
h
r
ee
co
n
tr
o
l
s
tr
ateg
ies
wer
e
ev
a
lu
ated
with
in
th
is
f
r
am
ewo
r
k
:
a
class
ical
PID
co
n
tr
o
ller
,
a
s
tan
d
alo
n
e
DDP
G
co
n
tr
o
ller
,
a
n
d
th
e
p
r
o
p
o
s
ed
r
esid
u
al
DDPG
co
n
tr
o
ller
.
T
h
e
r
ein
f
o
r
ce
m
en
t
lear
n
in
g
f
o
r
m
u
latio
n
e
m
p
lo
y
ed
a
Ma
r
k
o
v
Dec
is
io
n
Pro
c
ess
r
ep
r
esen
tatio
n
,
tr
an
s
ien
t
-
awa
r
e
r
ewa
r
d
s
h
ap
i
n
g
,
an
d
a
r
ef
er
e
n
ce
-
m
o
d
el
-
b
a
s
ed
r
ewa
r
d
d
esig
n
to
g
u
id
e
t
h
e
lear
n
in
g
p
r
o
ce
s
s
to
war
d
d
esira
b
le
f
lig
h
t
-
co
n
t
r
o
l
d
y
n
am
ics.
Simu
latio
n
r
esu
lts
d
em
o
n
s
tr
ated
th
at
t
h
e
r
esid
u
al
lear
n
i
n
g
a
r
ch
itectu
r
e
s
ig
n
if
ican
tly
im
p
r
o
v
es
o
v
er
all
co
n
tr
o
l
p
er
f
o
r
m
a
n
ce
.
W
h
ile
th
e
PID
co
n
tr
o
ller
p
r
o
v
id
ed
a
f
ast
r
esp
o
n
s
e
with
co
n
s
id
er
ab
le
o
v
er
s
h
o
o
t
an
d
o
s
cillato
r
y
b
eh
av
i
o
r
,
th
e
s
tan
d
alo
n
e
DDPG
co
n
tr
o
ller
p
r
o
d
u
ce
d
s
m
o
o
th
er
r
esp
o
n
s
es
at
th
e
ex
p
en
s
e
o
f
s
lo
wer
tr
an
s
ien
t d
y
n
am
ics.
I
n
co
n
tr
as
t,
th
e
p
r
o
p
o
s
ed
R
es
-
DDPG
-
q
C
AS a
ch
iev
ed
a
b
alan
ce
d
p
er
f
o
r
m
an
ce
,
co
m
b
i
n
in
g
f
ast
r
esp
o
n
s
e
with
m
in
im
al
o
v
er
s
h
o
o
t
an
d
th
e
lo
west
tr
ac
k
in
g
er
r
o
r
m
etr
ics.
C
o
m
m
an
d
tr
ac
k
in
g
an
d
d
is
tu
r
b
an
ce
r
ejec
tio
n
ex
p
e
r
i
m
en
ts
f
u
r
th
er
co
n
f
ir
m
ed
t
h
at
th
e
r
esid
u
al
co
n
tr
o
ller
p
r
o
v
id
es
s
m
o
o
th
,
well
-
d
am
p
ed
r
esp
o
n
s
es a
n
d
r
ap
id
r
ec
o
v
er
y
f
r
o
m
lar
g
e
e
x
ter
n
al
d
i
s
tu
r
b
an
ce
s
.
Desp
ite
th
ese
p
r
o
m
is
in
g
r
esu
lts
,
s
ev
er
al
lim
itatio
n
s
s
h
o
u
ld
b
e
ac
k
n
o
wled
g
ed
.
First,
th
e
e
v
alu
atio
n
was
co
n
d
u
cted
e
n
tire
ly
in
s
im
u
latio
n
,
an
d
ad
d
itio
n
al
r
o
b
u
s
tn
ess
s
tu
d
ies
u
n
d
er
v
ar
y
in
g
d
is
tu
r
b
an
ce
p
r
o
f
iles
,
p
ar
am
eter
u
n
ce
r
tain
ties
,
an
d
s
en
s
o
r
n
o
is
e
co
n
d
itio
n
s
ar
e
r
e
q
u
ir
ed
to
f
u
lly
ass
ess
th
e
co
n
tr
o
ller
’
s
r
eliab
ilit
y
.
Evaluation Warning : The document was created with Spire.PDF for Python.