T
E
L
KO
M
N
I
KA
T
e
lec
om
m
u
n
icat
ion
,
Com
p
u
t
i
n
g,
E
lec
t
r
on
ics
an
d
Cont
r
ol
Vol.
18
,
No.
3
,
J
une
2020
,
pp.
1
382
~
1
388
I
S
S
N:
1693
-
6930,
a
c
c
r
e
dit
e
d
F
ir
s
t
G
r
a
de
by
Ke
me
nr
is
tekdikti
,
De
c
r
e
e
No:
21/E
/KP
T
/2018
DO
I
:
10.
12928/
T
E
L
KO
M
NI
KA
.
v18i3.
14868
1382
Jou
r
n
al
h
omepage
:
ht
tp:
//
jour
nal.
uad
.
ac
.
id/
index
.
php/T
E
L
K
OM
N
I
K
A
T
r
an
sf
e
r
le
a
r
n
in
g
w
ith
m
u
lti
p
le
p
r
e
-
t
r
ai
n
e
d
n
e
t
w
o
r
k
f
or
f
u
n
d
u
s
c
la
ssi
f
ic
at
io
n
Wah
yu
d
i
S
e
t
iawan
1
,
M
oh
.
I
m
a
m
Ut
oyo
2
,
Rir
ies
Rul
an
in
gt
yas
3
1
In
fo
rma
t
i
c
s
D
ep
ar
t
men
t
,
U
n
i
v
er
s
i
t
y
o
f
T
ru
n
o
j
o
y
o
Ma
d
u
ra,
In
d
o
n
e
s
i
a
2
Mat
h
ema
t
i
c
s
D
ep
ar
t
men
t
,
U
n
i
v
er
s
i
t
as
A
i
r
l
an
g
g
a,
I
n
d
o
n
es
i
a
3
Ph
y
s
i
cs
D
ep
ar
t
men
t
,
U
n
i
v
er
s
i
t
as
A
i
r
l
an
g
g
a,
In
d
o
n
es
i
a
Ar
t
icle
I
n
f
o
AB
S
T
RA
CT
A
r
ti
c
le
h
is
tor
y
:
R
e
c
e
ived
S
e
p
10,
2019
R
e
vis
e
d
De
c
14,
2019
Ac
c
e
pted
De
c
22,
2019
T
r
a
n
s
f
e
r
l
e
a
r
n
i
n
g
(
T
L
)
i
s
a
t
e
c
h
n
i
q
u
e
o
f
r
e
u
s
e
a
n
d
m
o
d
i
f
y
a
p
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
.
It
r
e
u
s
e
s
f
e
a
t
u
r
e
e
x
t
r
a
c
t
i
o
n
l
a
y
e
r
a
t
a
p
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
.
A
t
a
r
g
e
t
d
o
m
a
i
n
i
n
T
L
o
b
t
a
i
n
s
t
h
e
f
e
a
t
u
r
e
s
k
n
o
w
l
e
d
g
e
f
r
o
m
t
h
e
s
o
u
r
c
e
d
o
m
a
i
n
.
T
L
m
o
d
i
f
i
e
d
c
l
a
s
s
i
f
i
c
a
t
i
o
n
l
a
y
e
r
a
t
a
p
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
.
T
h
e
t
a
r
g
e
t
d
o
m
a
i
n
c
a
n
d
o
n
e
w
t
a
s
k
s
a
c
c
o
r
d
i
n
g
t
o
a
p
u
r
p
o
s
e
.
I
n
t
h
i
s
a
r
t
i
c
l
e
,
t
h
e
t
a
r
g
e
t
d
o
m
a
i
n
i
s
f
u
n
d
u
s
i
m
a
g
e
c
l
a
s
s
i
f
i
c
a
t
i
o
n
i
n
c
l
u
d
e
s
n
o
r
m
a
l
a
n
d
n
e
o
v
a
s
c
u
l
a
r
i
z
a
t
i
o
n
.
D
a
t
a
c
o
n
s
i
s
t
o
f
1
0
0
p
a
t
c
h
e
s
.
T
h
e
c
o
m
p
a
r
i
s
o
n
o
f
t
r
a
i
n
i
n
g
a
n
d
v
a
l
i
d
a
t
i
o
n
d
a
t
a
w
a
s
7
0
:
3
0
.
T
h
e
s
e
l
e
c
t
i
o
n
o
f
t
r
a
i
n
i
n
g
a
n
d
v
a
l
i
d
a
t
i
o
n
d
a
t
a
i
s
d
o
n
e
r
a
n
d
o
m
l
y
.
S
t
e
p
s
o
f
T
L
i
.
e
l
o
a
d
p
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
s
,
r
e
p
l
a
c
e
f
i
n
a
l
l
a
y
e
r
s
,
t
r
a
i
n
t
h
e
n
e
t
w
o
r
k
,
a
n
d
a
s
s
e
s
s
n
e
t
w
o
r
k
a
c
c
u
r
a
c
y
.
F
i
r
s
t
,
t
h
e
p
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
i
s
a
l
a
y
e
r
c
o
n
f
i
g
u
r
a
t
i
o
n
o
f
t
h
e
c
o
n
v
o
l
u
t
i
o
n
a
l
n
e
u
r
a
l
n
e
t
w
o
r
k
a
r
c
h
i
t
e
c
t
u
r
e
.
P
r
e
-
t
r
a
i
n
e
d
n
e
t
w
o
r
k
u
s
e
d
a
r
e
A
l
e
x
N
e
t
,
V
G
G
1
6
,
V
G
G
1
9
,
R
e
s
N
e
t
5
0
,
R
e
s
N
e
t
1
0
1
,
G
o
o
g
L
e
N
e
t
,
I
n
c
e
p
t
i
o
n
-
V
3
,
I
n
c
e
p
t
i
o
n
R
e
s
N
e
t
V
2
,
a
n
d
s
q
u
e
e
z
e
n
e
t
.
S
e
c
o
n
d
,
r
e
p
l
a
c
e
t
h
e
f
i
n
a
l
l
a
y
e
r
i
s
t
o
r
e
p
l
a
c
e
t
h
e
l
a
s
t
t
h
r
e
e
l
a
y
e
r
s
.
T
h
e
y
a
r
e
f
u
l
l
y
c
o
n
n
e
c
t
e
d
l
a
y
e
r
,
s
o
f
t
m
a
x
,
a
n
d
o
u
t
p
u
t
l
a
y
e
r
.
T
h
e
l
a
y
e
r
i
s
r
e
p
l
a
c
e
d
w
i
t
h
a
f
u
l
l
y
c
o
n
n
e
c
t
e
d
l
a
y
e
r
t
h
a
t
c
l
a
s
s
i
f
i
e
s
a
c
c
o
r
d
i
n
g
t
o
n
u
m
b
e
r
o
f
c
l
a
s
s
e
s
.
F
u
r
t
h
e
r
m
o
r
e
,
i
t
'
s
f
o
l
l
o
w
e
d
b
y
a
s
o
f
t
m
a
x
a
n
d
o
u
t
p
u
t
l
a
y
e
r
t
h
a
t
m
a
t
c
h
e
s
w
i
t
h
t
h
e
t
a
r
g
e
t
d
o
m
a
i
n
.
T
h
i
r
d
,
w
e
t
r
a
i
n
e
d
t
h
e
n
e
t
w
o
r
k
.
N
e
t
w
o
r
k
s
w
e
r
e
t
r
a
i
n
e
d
t
o
p
r
o
d
u
c
e
o
p
t
i
m
a
l
a
c
c
u
r
a
c
y
.
I
n
t
h
i
s
s
e
c
t
i
o
n
,
w
e
u
s
e
g
r
a
d
i
e
n
t
d
e
s
c
e
n
t
a
l
g
o
r
i
t
h
m
o
p
t
i
m
i
z
a
t
i
o
n
.
F
o
u
r
t
h
,
a
s
s
e
s
s
n
e
t
w
o
r
k
a
c
c
u
r
a
c
y
.
T
h
e
e
x
p
e
r
i
m
e
n
t
r
e
s
u
l
t
s
s
h
o
w
a
t
e
s
t
i
n
g
a
c
c
u
r
a
c
y
b
e
t
w
e
e
n
8
0
%
a
n
d
1
0
0
%
.
K
e
y
w
o
r
d
s
:
C
las
s
if
ica
ti
on
C
onvolut
ional
ne
ur
a
l
ne
twor
k
M
ult
ipl
e
pr
e
-
tr
a
ined
ne
twor
k
Ne
ova
s
c
ular
iza
t
ion
T
r
a
ns
f
e
r
lea
r
ning
Th
i
s
i
s
a
n
o
p
en
a
c
ces
s
a
r
t
i
c
l
e
u
n
d
e
r
t
h
e
CC
B
Y
-
SA
l
i
ce
n
s
e
.
C
or
r
e
s
pon
din
g
A
u
th
or
:
R
ir
ies
R
ulaningtyas
,
P
hys
ics
De
pa
r
tm
e
nt,
Unive
r
s
it
a
s
Air
langga
,
S
ur
a
ba
ya
,
I
ndon
e
s
ia.
E
mail:
r
ir
ies
-
r
@f
s
t.
una
ir
.
a
c
.
id
1.
I
NT
RODU
C
T
I
ON
A
s
ys
tem
r
e
quir
e
s
lea
r
ning
p
r
oc
e
s
s
to
pe
r
f
o
r
m
c
e
r
tain
tas
ks
.
T
he
tas
ks
include
im
a
ge
e
nha
nc
e
ment,
c
las
s
if
ica
ti
on,
c
lus
ter
ing,
r
e
c
ognit
ion,
a
nd
de
tec
ti
on.
Da
ta
pr
oc
e
s
s
ing
ne
e
ds
to
do
it
.
Da
ta
is
divi
de
d
int
o
two
pa
r
ts
,
t
r
a
ini
ng
a
nd
tes
ti
ng
da
ta.
I
n
c
onve
nti
on
a
l
s
ys
tems
,
tr
a
ini
ng
da
ta
pr
oc
e
s
s
e
d
to
ge
t
kn
owle
dge
.
T
he
pr
oblem
a
r
is
e
s
whe
n
the
a
mount
of
tr
a
ini
ng
da
ta
is
li
mi
ted,
the
lea
r
ning
pr
oc
e
s
s
doe
s
n’
t
we
ll
pe
r
f
or
m.
An
a
lt
e
r
na
ti
ve
s
olut
ion
to
the
p
r
oble
m
is
tr
a
ns
f
e
r
lea
r
ning.
I
t
is
a
mac
hine
lea
r
ning
method
that
wor
ks
by
uti
li
z
ing
e
xis
ti
ng
models
.
T
r
a
ns
f
e
r
lea
r
ning
mod
if
i
e
s
a
nd
upda
tes
p
a
r
a
mete
r
s
on
the
model.
T
r
a
ns
f
e
r
lea
r
ning
make
s
modi
f
ied
models
a
s
lea
r
ning
with
dif
f
e
r
e
n
t
tas
ks
.
T
he
model
us
e
d
f
or
tr
a
ns
f
e
r
lea
r
ning
ha
s
lea
r
ne
d
f
r
om
other
da
ta
,
s
o
lea
r
nin
g
is
not
ne
e
de
d
f
r
om
s
c
r
a
tch.
T
he
model
ha
s
r
e
c
ognize
d
f
e
a
tur
e
s
s
uc
h
a
s
textur
e
s
,
s
ha
pe
s
,
a
nd
c
olor
s
a
s
a
r
e
s
ult
of
pr
e
vious
lea
r
ning
.
Evaluation Warning : The document was created with Spire.PDF for Python.
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
T
r
ans
fer
lear
ning
w
it
h
multi
ple
pr
e
-
tr
ained
ne
tw
or
k
fo
r
fundus
c
las
s
if
ication
(
W
ahy
udi
Se
ti
aw
an
)
1383
T
he
be
ne
f
it
of
tr
a
ns
f
e
r
lea
r
ning
is
we
ll
lea
r
ning
e
ve
n
though
li
mi
ted
tr
a
ini
ng
da
ta.
C
ontr
a
s
t
to
tr
a
dit
ional
mac
hine
lea
r
ning
,
e
ve
r
y
lea
r
ning
p
r
oc
e
s
s
a
lwa
y
s
r
e
quir
e
s
r
e
latively
lar
ge
a
moun
ts
of
da
ta
[
1]
.
T
he
dif
f
e
r
e
nc
e
be
twe
e
n
t
r
a
dit
ional
mac
hine
lea
r
ni
ng
a
nd
t
r
a
n
s
f
e
r
lea
r
ning
is
f
ound
in
F
igur
e
1
.
No
wa
da
ys
,
tr
a
ns
f
e
r
lea
r
ning
ha
s
be
e
n
a
ppli
e
d
to
r
oboti
c
s
[2
,
3]
im
a
g
e
c
las
s
if
ica
ti
on
[4
,
5]
,
s
e
nti
ment
c
las
s
i
f
ica
ti
on
[
6]
,
ga
me
tec
hnology
[7
,
8]
a
nd
text
c
las
s
if
ica
ti
on
[
9]
.
Ge
ne
r
a
ll
y,
the
type
of
t
r
a
ns
f
e
r
lea
r
ning
us
e
d
in
de
e
p
lea
r
ning
is
a
pr
e
-
tr
a
ined
ne
twor
k
.
T
he
pha
s
e
f
o
r
c
o
nduc
ti
ng
tr
a
ns
f
e
r
lea
r
n
ing
a
s
f
oll
ows
:
-
S
e
lec
t
a
s
pe
c
if
ic
model.
P
r
e
-
tr
a
ined
ne
twor
k
model
s
a
r
e
take
n
f
r
o
m
e
xis
ti
ng
models
.
-
R
e
us
e
d
model.
P
r
e
-
tr
a
ined
models
c
a
n
be
us
e
d
a
s
a
s
tar
ti
ng
point
f
or
c
a
r
r
ying
out
a
ne
w
tas
k.
A
ne
w
tas
k
c
a
n
us
e
the
whole
pa
r
t
o
f
a
pr
e
-
tr
a
ined
model
o
r
pa
r
tl
y
de
pe
nds
on
s
ys
tem
r
e
quir
e
ments
.
(
a
)
(
b)
F
igur
e
1.
(
a
)
T
r
a
dit
ional
mac
hine
lea
r
ning,
(
b)
T
r
a
ns
f
e
r
l
e
a
r
ning,
modi
f
ied
f
r
o
m
-
M
odif
ica
ti
on
of
the
model.
M
odif
ica
ti
ons
a
r
e
mad
e
a
t
the
las
t
f
ull
y
c
onne
c
ted
laye
r
T
his
pa
pe
r
dis
c
us
s
e
d
c
las
s
if
ica
ti
on
of
f
undus
im
a
ge
s
.
C
las
s
if
ica
ti
on
f
or
d
is
ti
nguis
hing
nor
mal
a
nd
ne
ova
s
c
ular
iza
ti
on.
Ne
ova
s
c
ular
iza
ti
on
is
the
a
p
pe
a
r
a
nc
e
of
ne
w
ve
s
s
e
ls
in
opti
c
dis
k
or
other
s
ur
f
a
c
e
s
of
r
e
ti
na
.
Ne
ova
s
c
ular
iza
ti
on
f
e
a
tur
e
s
of
the
blo
od
ve
s
s
e
ls
a
r
e
br
it
tl
e
,
i
r
r
e
gular
in
s
ha
pe
,
a
nd
e
a
s
il
y
los
t.
Ne
ova
s
c
ular
iza
ti
on
is
a
s
e
ve
r
e
diabe
ti
c
r
e
ti
nopa
thy
(
DR
)
.
Ne
ova
s
c
ular
iza
ti
on
c
ons
is
ts
of
two
c
a
t
e
gor
ies
:
ne
ova
s
c
ular
iza
ti
on
on
the
dis
c
(
NV
D)
a
nd
ne
o
va
s
c
ular
iza
ti
on
e
ls
e
wh
e
r
e
(
NV
E
)
.
NV
D
is
a
ne
w
ve
s
s
e
l
in
the
opti
c
dis
c
while
NV
E
is
a
ne
w
ve
s
s
e
l
in
the
e
nti
r
e
s
ur
f
a
c
e
of
the
r
e
ti
na
e
xc
e
pt
in
opt
ic
d
is
c
[
10]
.
n
e
ova
s
c
ular
iza
ti
on
of
the
f
undus
i
mage
s
is
s
hown
in
F
igur
e
2
.
T
he
r
e
a
r
e
pr
e
vi
ous
s
tudi
e
s
that
c
las
s
if
ied
f
undus
i
mage
s
.
T
e
nna
koon
e
t
a
l.
c
las
s
if
ied
two
c
a
tegor
ies
:
gr
a
da
ble
a
nd
ungr
a
da
ble
ba
s
e
d
on
i
mage
qua
li
ty
f
undus
.
T
he
model
us
e
s
s
ha
ll
owN
e
t
a
nd
a
modi
f
ied
Ale
xNe
t
model.
A
f
ull
y
c
onne
c
ted
laye
r
(
F
C
L
)
f
c
7
is
a
laye
r
f
or
f
e
a
tur
e
e
xtr
a
c
ti
on
.
F
C
L
f
c
8
is
the
f
ine
-
tuni
ng
laye
r
f
or
c
las
s
if
ica
ti
on.
C
las
s
if
ica
ti
on
us
ing
S
VM
,
boos
ted
tr
e
e
,
a
nd
k
-
NN
methods
.
T
he
da
ta
c
ons
is
ts
of
463
im
a
ge
s
.
T
he
highes
t
a
c
c
ur
a
c
y
is
98.
27%
us
ing
s
ha
ll
owN
e
t
[
11]
.
L
i
e
t
a
l
.
c
las
s
if
ied
f
undus
im
a
ge
s
us
ing
d
a
ta
f
r
om
DR
1
a
nd
M
E
S
S
I
DO
R
.
T
he
a
mount
of
da
ta
f
or
e
a
c
h
da
tas
e
t
is
1,
014
a
nd
1
,
200
i
mage
s
.
T
he
r
e
a
r
e
th
r
e
e
s
teps
f
or
tr
a
ns
f
e
r
lea
r
ning,
f
ine
-
tuni
ng
a
ll
laye
r
s
on
pr
e
-
tr
a
ined
C
NN
models
a
c
c
or
ding
to
their
f
unc
ti
ons
,
f
ine
-
tuni
ng
pr
e
-
tr
a
ined
C
NN
models
on
a
ddit
ional
laye
r
s
,
then
f
e
a
tur
e
e
xtr
a
c
ti
on
a
nd
c
las
s
if
ica
ti
on
us
ing
S
VM
.
(a)
(b
)
F
igur
e
2.
Ne
ova
s
c
ular
iza
ti
on
in
f
undus
pa
tch
;
(
a
)
NV
E
,
(
b)
NV
D
[
12]
T
he
e
xpe
r
im
e
nt
us
e
s
s
e
ve
r
a
l
models
,
Ale
x
Ne
t,
g
oogleN
e
t,
a
nd
VG
G.
T
he
pa
r
a
mete
r
s
us
e
d
we
r
e
maximum
e
poc
h
30,
mi
nibatc
h
s
ize
50,
lea
r
ning
r
a
tes
0.
1
to
0.
0001,
we
ight
s
0.
0005
a
nd
mom
e
nt
um
0.
95.
Optim
iza
ti
on
us
ing
s
tocha
s
ti
c
gr
a
dient
de
s
c
e
nt
wit
h
mom
e
ntum
(
S
GD
M
)
a
lgor
it
h
m.
T
he
tes
t
r
e
s
ult
s
s
howe
d
th
e
be
s
t
a
c
c
ur
a
c
y
is
the
modi
f
ied
VG
G
-
m
model
o
f
95.
49
%
f
o
r
the
DR
1
da
tas
e
t
a
nd
Google
Ne
t
mod
if
ica
ti
on
of
79.
37
%
f
or
the
M
E
S
S
I
DO
R
da
tas
e
t
[1
3
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
1693
-
6930
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
,
Vol.
18
,
No
.
3
,
J
une
2020:
1
382
-
1
388
1384
C
hoi
e
t
a
l.
[
14]
c
las
s
if
ied
10
c
las
s
e
s
of
diabe
ti
c
r
e
ti
nopa
thy
(
DR
)
.
Da
ta
c
ons
is
ts
of
10,
000
im
a
ge
s
.
E
a
c
h
c
a
tegor
y
ha
s
1
,
000
im
a
ge
s
.
T
he
model
us
e
d
f
or
tr
a
ns
f
e
r
lea
r
ning
is
VG
G19
a
nd
Ale
xNe
t
.
T
he
opti
mi
z
a
ti
on
a
lgo
r
it
hm
us
e
s
S
GD
M
,
mom
e
n
tum
0.
9
lea
r
ning
r
a
te
10
-
6,
a
nd
max
e
poc
h
50.
T
he
tes
t
s
c
e
na
r
io
is
va
r
ied,
with
3
a
nd
5
c
las
s
e
s
.
T
he
tes
t
r
e
s
ult
s
s
howe
d
th
e
be
s
t
a
c
c
ur
a
c
y
is
VG
G19
f
or
c
las
s
if
ica
ti
on
of
th
r
e
e
c
a
tegor
ies
a
t
80.
8%
,
while
c
las
s
if
ica
ti
on
o
f
5
c
a
tegor
ies
s
howe
d
the
h
ighes
t
a
c
c
ur
a
c
y
of
59.
1%
[1
4
]
.
M
a
s
ood
e
t
a
l.
c
las
s
if
ied
4
DR
c
las
s
e
d
a
s
mi
ld
,
moder
a
te,
s
e
ve
r
e
non
-
pr
oli
f
e
r
a
ti
ve
d
iabe
ti
c
r
e
ti
nopa
thy
(
NPDR
)
,
a
nd
P
DR
.
T
he
da
tas
e
t
is
take
n
f
r
om
e
ye
P
a
c
s
.
T
he
s
teps
f
o
r
tr
a
ini
ng
lea
r
ning
a
r
e
pr
e
p
r
oc
e
s
s
ing
a
nd
r
e
tr
a
ini
ng
I
nc
e
pti
on
-
V3.
R
e
s
ult
s
hows
48.
2%
f
o
r
a
c
c
ur
a
c
y
[1
5
]
.
Okta
lor
a
e
t
a
l.
[
16]
c
las
s
if
y
f
or
e
xuda
te.
E
xuda
te
is
a
s
ympt
om
in
the
f
or
m
of
a
ye
ll
ow
s
pot,
ir
r
e
gular
s
ha
pe
,
a
r
is
ing
f
r
o
m
l
ipi
d
inf
il
tr
a
ti
on
in
the
r
e
ti
na
.
E
xuda
te
is
a
s
ympt
om
of
diabe
ti
c
r
e
ti
nopa
thy
.
T
h
is
s
tudy
us
e
s
a
L
e
Ne
t
model
with
s
e
ve
n
laye
r
s
.
E
xpe
r
i
ment
da
ta
us
ing
Optha
da
tas
e
t.
T
he
s
ize
of
da
ta
is
48x4
8
pixels
.
T
he
c
las
s
if
ica
ti
on
c
ons
is
ts
of
two
c
a
tegor
ies
:
nor
mal
a
nd
e
xuda
te
[1
6
]
.
S
a
de
k
e
t
a
l.
bu
il
d
tr
a
ns
f
e
r
l
e
a
r
ning
to
cl
a
s
s
if
y
3
c
a
tegor
ies
include
nor
mal,
e
xuda
tes
,
a
n
d
dr
us
e
n.
T
he
da
tas
e
t
us
e
s
a
r
e
S
T
AR
E
,
HR
F
,
D
r
i
s
onDB
,
Optha
,
HE
I
M
E
D
a
nd
M
E
S
S
I
DO
R
da
tas
e
t.
T
r
a
ns
f
e
r
lea
r
ning
us
e
s
modi
f
ied
VG
G,
Google
Ne
t
a
nd
R
e
s
Ne
t
models
.
R
e
s
ult
s
hows
a
ve
r
a
ge
a
c
c
ur
a
c
y
f
r
om
91
.
2
3%
to
92
%
[1
7
]
.
T
he
a
bove
s
tudi
e
s
ha
ve
not
r
e
a
c
he
d
the
op
ti
mal
a
c
c
ur
a
c
y.
C
ha
r
a
c
ter
is
ti
c
s
of
diabe
ti
c
r
e
ti
nopa
thy
(
DR
)
d
is
e
a
s
e
ha
ve
not
be
e
n
f
ull
y
c
las
s
if
ied.
T
he
c
ha
r
a
c
ter
is
ti
c
s
of
DR
a
r
e
mi
c
r
oa
ne
ur
ys
m,
he
mor
r
ha
ge
s
,
e
xuda
tes
,
c
ott
on
wool
s
pots
,
a
n
d
ne
ova
s
c
ular
iza
ti
on.
T
he
nove
lt
y
of
thi
s
s
tudy
is
c
las
s
if
ica
ti
on
o
f
f
undus
im
a
ge
s
to
dis
ti
nguis
h
nor
mal
a
nd
ne
ova
s
c
ular
iza
ti
on
us
ing
tr
a
ns
f
e
r
lea
r
ning.
B
e
s
ides
,
nove
lt
y
is
a
l
s
o
f
ound
in
C
NN
modi
f
ica
ti
on
tec
hnique
by
uti
li
z
ing
the
la
s
t
thr
e
e
laye
r
s
of
e
a
c
h
model
.
T
he
r
e
s
ult
s
of
mea
s
ur
e
ment
a
c
c
ur
a
c
y
f
r
om
t
r
a
ns
f
e
r
lea
r
ning
a
r
e
c
ompar
e
d
in
the
opti
mi
z
a
ti
on
of
g
r
a
dient
de
s
c
e
nt
s
uc
h
a
s
s
t
oc
ha
s
ti
c
gr
a
dient
de
s
c
e
nt
with
mom
e
ntum
(
S
GD
M
)
,
r
oot
mea
n
s
qua
r
e
pr
opa
ga
ti
on
(
R
M
S
P
r
op)
,
a
nd
a
da
pti
v
e
mom
e
nt
opti
mi
z
a
ti
on
(
Ada
m)
.
2.
RE
S
E
AR
CH
M
E
T
HO
D
T
he
e
xpe
r
im
e
nt
da
ta
c
ons
is
ts
of
2
c
las
s
e
s
includ
e
nor
mal
a
nd
ne
ova
s
c
ular
iza
ti
on.
E
a
c
h
c
las
s
ha
s
50
pa
tche
s
,
s
o
the
tot
a
l
da
ta
is
100
pa
tche
s
.
I
t
is
take
n
f
r
om
the
M
E
S
S
I
DO
R
[1
8
]
a
nd
r
e
ti
n
a
im
a
ge
ba
nk
[
12]
.
T
he
pr
e
-
tr
a
ined
ne
twor
k
is
a
C
NN
mo
de
l.
C
NN
is
the
s
a
me
a
s
the
other
ne
ur
a
l
n
e
twor
ks
,
c
ons
is
ti
ng
of
we
ight
,
bias
a
nd
a
c
ti
va
ti
on
f
unc
ti
o
ns
.
C
NN
ha
s
2
big
pa
r
ts
of
the
laye
r
,
laye
r
f
or
f
e
a
tur
e
e
xtr
a
c
ti
on
a
nd
laye
r
f
or
c
las
s
if
ica
ti
on.
T
he
laye
r
f
or
f
e
a
tur
e
e
xt
r
a
c
ti
on
c
ons
is
ts
of
a
c
onvolut
iona
l
laye
r
,
pooli
ng
laye
r
,
s
tr
ide
,
a
nd
pa
dding.
W
hil
e
laye
r
f
o
r
c
las
s
if
ica
ti
on
c
ons
is
ts
of
f
ull
y
c
onne
c
ted
laye
r
,
s
of
tm
a
x,
a
nd
output
laye
r
[1
9
]
.
P
r
e
-
tr
a
ined
ne
two
r
k
be
c
om
e
s
a
pa
r
t
o
f
t
r
a
ns
f
e
r
lea
r
ning
.
P
ha
s
e
of
tr
a
ns
f
e
r
le
a
r
ning
is
im
por
t
pr
e
-
tr
a
ined
ne
twor
k,
r
e
plac
e
c
las
s
if
ica
ti
on
laye
r
,
tr
a
in
ne
twor
k
on
the
f
undus
im
a
ge
da
ta,
a
nd
ge
t
pe
r
f
or
manc
e
mea
s
ur
e
of
a
c
c
ur
a
c
y.
2.
1.
I
m
p
or
t
p
r
e
-
t
r
ain
e
d
n
e
t
wor
k
W
e
us
e
Ale
xNe
t
[1
9
]
,
VG
G16,
VG
G19
[
20
]
,
R
e
s
Ne
t50,
R
e
s
Ne
t101
[2
1
]
,
Google
Ne
t
[2
2
]
,
I
nc
e
pti
on
-
V3
[2
3
]
,
I
nc
e
pti
on_R
e
s
Ne
tV2
[2
4
]
,
a
n
d
S
que
e
z
e
ne
t
[2
5
]
a
s
pr
e
-
tr
a
ined
ne
twor
k.
T
he
p
r
e
-
tr
a
ined
ne
twor
k
ha
s
tr
a
ined
in
I
mage
Ne
t
c
ompetit
ion.
I
t
ha
s
mor
e
than
a
mi
ll
ion
im
a
ge
s
a
nd
20,
000
c
las
s
e
s
[2
6
]
.
E
a
c
h
pr
e
-
tr
a
ined
ne
twor
k
ha
s
a
dif
f
e
r
e
nt
laye
r
c
o
nf
igur
a
ti
on.
T
he
laye
r
a
t
the
b
e
ginni
ng
a
nd
mi
dd
l
e
c
a
ll
e
d
a
f
e
a
tur
e
e
xtr
a
c
ti
on
laye
r
.
T
he
s
e
laye
r
s
pr
oduc
e
s
im
ple
f
e
a
tur
e
s
s
uc
h
a
s
br
ight
ne
s
s
a
nd
e
dg
e
s
,
to
c
ompl
e
x
unique
f
e
a
tur
e
s
s
uc
h
a
s
c
olor
s
a
nd
s
ha
pe
s
.
T
he
r
e
s
ult
s
of
f
e
a
tur
e
e
xtr
a
c
ti
on
laye
r
a
t
s
our
c
e
do
m
a
in
c
a
n
be
tr
a
ns
f
e
r
r
e
d
f
o
r
f
e
a
tur
e
e
xtr
a
c
ti
o
n
laye
r
a
t
tar
ge
t
domain.
F
e
a
tur
e
e
xt
r
a
c
ti
on
lea
r
n
ing
on
ta
r
ge
t
do
m
a
ins
a
ls
o
knows
tr
a
ini
ng
f
undus
da
ta
im
a
ge
s
.
F
igur
e
3
s
ho
ws
a
pr
opos
e
d
f
r
a
mew
or
k
of
tr
a
ns
f
e
r
lea
r
ning
f
o
r
f
undus
im
a
ge
c
las
s
if
ica
ti
on.
F
igur
e
3.
T
he
pr
opos
e
d
f
r
a
mew
o
r
k
of
tr
a
ns
f
e
r
lea
r
ning
f
or
f
undus
im
a
ge
c
las
s
if
ica
ti
on
Evaluation Warning : The document was created with Spire.PDF for Python.
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
T
r
ans
fer
lear
ning
w
it
h
multi
ple
pr
e
-
tr
ained
ne
tw
or
k
fo
r
fundus
c
las
s
if
ication
(
W
ahy
udi
Se
ti
aw
an
)
1385
2.
2.
Re
p
lace
c
las
s
if
icat
ion
layer
T
he
c
las
s
if
ica
ti
on
laye
r
is
known
a
s
3
f
inal
laye
r
s
i.
e
f
ull
y
c
onne
c
ted
laye
r
,
s
of
tm
a
x,
a
nd
a
n
output
laye
r
.
I
t
r
e
plac
e
d
by
a
p
r
e
-
tr
a
ined
ne
twor
k
a
nd
s
ubs
ti
tut
e
s
wit
h
ne
w
c
las
s
if
ica
ti
o
n
laye
r
that
matc
he
d
with
a
ne
w
c
las
s
if
ica
ti
on
tas
k.
I
t
include
s
ne
w
number
of
c
las
s
e
s
a
nd
a
s
e
t
lea
r
ning
r
a
te
in
the
ne
w
ne
twor
k.
T
he
r
e
a
r
e
e
xc
e
pti
ons
f
o
r
s
que
e
z
e
ne
t,
a
laye
r
that
mus
t
be
r
e
plac
e
d
c
ons
is
ts
of
f
ive
laye
r
s
.
T
a
ble
1
s
how
s
the
c
las
s
if
ica
ti
on
laye
r
that
r
e
plac
e
s
the
ne
twor
k.
T
a
ble
1
.
C
las
s
if
ica
ti
on
l
a
ye
r
of
the
pr
e
-
tr
a
ined
ne
t
wor
k
P
r
e
-
T
r
a
in
e
d N
e
twor
k
C
la
s
s
if
ic
a
ti
on L
a
ye
r
A
le
xN
e
t
f
c
8, pr
ob, output
V
G
G
16
f
c
8, pr
ob, output
V
G
G
19
f
c
8, pr
ob, output
R
e
s
N
e
t5
0
f
c
1000,
f
c
1000
_s
of
tm
a
x, c
la
s
s
if
ic
a
ti
onl
a
ye
r
_f
c
1000
R
e
s
N
e
t1
01
f
c
1000, pr
ob, c
la
s
s
if
ic
a
ti
onl
a
ye
r
_pr
e
di
c
ti
ons
G
oogl
e
N
e
t
lo
s
s
3
-
c
la
s
s
if
ie
r
, pr
ob, output
I
nc
e
pt
io
n
-
V3
pr
e
di
c
ti
ons
, pr
e
di
c
ti
ons
_s
of
tm
a
x, c
la
s
s
if
ic
a
ti
onl
a
ye
r
_pr
e
di
c
ti
on
s
I
nc
e
pt
io
n
-
R
e
s
N
e
tV2
pr
e
di
c
ti
ons
, p
r
e
di
c
t
io
ns
_s
of
tm
a
x, c
la
s
s
if
ic
a
ti
onl
a
ye
r
_pr
e
di
c
ti
on
s
S
que
e
z
e
ne
t
c
onv10, r
e
lu
_c
onv10, pool10, pr
ob, c
la
s
s
if
ic
a
ti
onl
a
ye
r
_pr
e
di
c
ti
ons
2.
3.
T
r
ain
n
e
t
wor
k
on
f
u
n
d
u
s
im
age
Da
ta
is
pr
oc
e
s
s
e
d
with
r
e
s
e
a
r
c
h
method
a
s
s
hown
i
n
F
igur
e
3
.
I
t
is
a
pha
s
e
of
t
r
a
ns
f
e
r
lea
r
n
ing
with
a
pr
e
-
tr
a
ined
ne
twor
k
f
or
c
las
s
if
ica
ti
on
of
f
und
us
im
a
ge
s
.
At
the
top,
s
our
c
e
domains
a
r
e
pr
e
-
tr
a
ined
ne
twor
ks
that
ha
ve
c
las
s
if
ied
da
ta
on
I
mage
Ne
t.
T
he
t
r
a
in
ne
twor
k
a
ls
o
ne
e
ds
a
n
opti
mi
z
a
ti
on
a
lgor
it
hm.
W
e
us
e
a
n
opti
mi
z
a
ti
on
gr
a
dient
de
s
c
e
nt
a
lgor
it
hm.
Gr
a
dient
d
e
s
c
e
nt
(
GD
)
obtain
opti
mal
pa
r
a
mete
r
we
ight
s
,
r
e
duc
e
pr
e
diction
e
r
r
o
r
s
a
nd
im
pr
ove
pr
e
diction
s
of
a
c
c
ur
a
c
y.
GD
pe
r
f
o
r
ms
pa
r
a
mete
r
opti
mi
z
a
ti
on
o
n
the
ne
twor
k.
B
e
s
ides
,
GD
ha
s
a
li
ne
a
r
c
ompl
e
xit
y
of
da
ta
incr
e
ment
.
GD
c
a
n
be
c
omput
e
d
in
pa
r
a
ll
e
l
by
uti
li
z
ing
a
gr
a
phica
l
pr
oc
e
s
s
ing
unit
(
GPU)
.
T
he
a
ppli
c
a
ti
on
of
GD
on
the
C
NN
model
p
r
ove
s
that
GD
c
a
n
do
tr
a
ini
ng
with
mi
ll
ions
o
f
da
ta
[2
7
]
.
2.
4
.
Gradi
e
n
t
d
e
s
c
e
n
t
wi
t
h
m
om
e
n
t
u
m
M
omentum
is
a
method
f
or
GD
a
c
c
e
ler
a
ti
on
by
uti
li
z
ing
gr
a
dient
inf
o
r
mation
in
the
pr
e
vious
s
teps
.
Ac
c
umul
a
ti
on
o
f
gr
a
dients
is
us
e
f
ul
f
or
c
on
tr
oll
ing
os
c
il
lator
y
e
f
f
e
c
ts
.
F
u
r
ther
mor
e
,
it
is
e
xpe
c
ted
that
the
opti
mi
z
a
ti
on
pa
th
c
a
n
be
mor
e
s
table
[2
8
]
.
Algor
it
hm
1
.
(
Gr
a
dient
de
s
c
e
nt
with
mom
e
ntu
m
)
1
.
0
= 0
2
.
∶
=
∇
−
1
(
−
1
)
3
.
≔
+
−
1
4
.
∶
=
−
1
−
with
=
gr
a
dient
loos
f
unc
ti
on
to
−
1
,
=
ne
xt
pa
r
a
mete
r
,
=
lea
r
ning
r
a
te.
T
he
c
ons
tant
c
ontr
ols
the
s
ize
of
the
c
ontr
ibut
ion
f
r
om
the
p
r
e
vious
gr
a
dient.
Ge
ne
r
a
ll
y,
s
e
t
t
o
0.
9
is
the
be
s
t
va
lue
of
the
e
xpe
r
im
e
nt
that
ha
s
be
e
n
c
a
r
r
ied
out
.
I
f
s
e
t
to
0
,
then
the
GD
M
r
e
s
ult
s
a
r
e
the
s
a
me
a
s
GD
.
S
tocha
s
ti
c
gr
a
dient
de
s
c
e
nt
with
mom
e
ntu
m
(
S
GD
M
)
is
a
va
r
iant
of
GD
M
.
T
he
dif
f
e
r
e
nc
e
is
da
ta
a
c
c
e
s
s
.
I
f
in
GD
M
the
da
ta
is
pr
oc
e
s
s
e
d
a
ll
the
da
ta
a
t
the
s
a
me
ti
me.
Da
ta
on
S
GD
M
will
be
p
r
oc
e
s
s
e
d
s
uit
a
bly
with
mi
nibatc
h
s
ize
[2
7
]
.
2.
4.
1.
AdaG
r
ad
an
d
RM
S
P
r
op
Ada
pti
ve
s
ubgr
a
dient
de
s
c
e
nt
(
Ada
Gr
a
d)
[2
9
]
ge
t
GD
im
pr
ove
ments
by
p
r
ovidi
ng
di
f
f
e
r
e
nt
upda
te
s
pe
e
ds
f
or
e
a
c
h
ve
c
tor
dim
e
ns
ion.
T
he
Ada
Gr
a
d
a
lgor
it
hm
is
f
ound
in
a
lgor
it
hm
2
[2
8
]
.
C
ons
tants
pr
ovide
inf
or
mation
a
bout
c
ha
nging
the
va
lue
o
f
a
n
e
leme
nt
in
the
gr
a
dient
ve
c
tor
.
I
f
the
va
lue
in
a
c
e
r
tain
d
im
e
ns
ion
de
c
r
e
a
s
e
s
,
the
upda
te
s
pe
e
d
in
c
e
r
tain
dim
e
ns
ions
i
nc
r
e
a
s
e
s
a
n
d
vice
ve
r
s
a
.
T
his
wi
ll
ba
lanc
e
the
c
ont
r
ibut
ion
of
e
a
c
h
dim
e
ns
ion
of
the
g
r
a
dient
ve
c
tor
s
o
that
th
e
opti
mi
z
a
ti
on
pa
th
be
c
omes
mo
r
e
s
table
Algor
it
hm
2
.
(
Ada
Gr
a
d)
1
.
0
= 0
2
.
∶
=
∇
−
1
(
−
1
)
3
.
≔
2
+
−
1
4
.
∶
=
−
1
−
√
+
with
=
adapti
v
e
s
ubgr
adient
,
=
a
c
ons
tant
1e
-
6
T
he
pr
oblem
with
Ada
Gr
a
d
is
the
va
lue
c
a
n
be
ve
r
y
la
r
ge
a
t
c
e
r
tain
ti
me.
I
t
will
s
low
down
the
opti
mi
z
a
ti
on
pr
oc
e
s
s
ti
me.
T
he
s
olut
ion
to
thes
e
pr
oblems
is
to
modi
f
y
by
a
dding
c
ons
tant
s
.
T
he
s
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
1693
-
6930
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
,
Vol.
18
,
No
.
3
,
J
une
2020:
1
382
-
1
388
1386
c
ons
tants
a
r
e
us
e
d
to
s
e
t
va
r
iable
qua
nti
ti
e
s
.
T
he
r
oot
mea
ns
s
qua
r
e
pr
opa
ga
ti
on
(
R
M
S
P
r
op)
a
lgor
it
hm
is
f
ound
in
a
lgo
r
it
hm
3
[2
8
,
30
]
.
Algor
it
hm
3
.
(
R
M
S
P
r
op)
1
.
0
= 0
2
.
∶
=
∇
−
1
(
−
1
)
3
.
≔
(
1
−
)
2
+
−
1
4
.
∶
=
−
1
−
√
+
with
=
k
oe
fi
s
ien
de
c
ay
r
ate
0,
95
2.
4.
2.
Adap
t
ive
m
o
m
e
n
t
op
t
i
m
izat
ion
(
Adam
)
Ada
m
a
lgor
it
hm
c
ombi
ne
s
the
two
a
ppr
oa
c
he
s
to
im
p
r
ove
GD
,
mom
e
ntum
a
nd
a
da
pti
ve
s
ub
gr
a
dient.
T
his
a
lgor
it
hm
c
ombi
ne
s
GD
M
with
R
M
S
P
r
op
.
Ada
m's
a
lgor
it
hm
is
s
hown
in
a
lgor
it
hm
4
[2
8
]
.
L
ine
4
is
a
n
e
leme
nt
of
mom
e
ntum
,
li
ne
6
is
a
n
a
da
pti
ve
s
ubgr
a
dient
e
leme
nt.
Ada
m
ha
s
a
c
or
r
e
c
ti
on
bias
tec
hnique
with
a
be
tt
e
r
a
ppr
oxim
a
ti
on
[3
1
]
.
Algor
it
hm
4
.
Ada
m
1
.
0
= 0
2
.
∶
=
∇
−
1
(
−
1
)
3
.
≔
(
1
−
)
+
−
1
4
.
̂
∶
=
1
−
5
.
≔
(
1
−
)
2
+
−
1
6
.
̂
∶
=
1
−
7
.
∶
=
−
1
−
̂
√
̂
+
=
mom
e
ntum
,
=
a
da
pti
ve
s
ubgr
a
dient,
̂
=
mom
e
ntum
e
s
ti
mation
with
c
or
r
e
c
ted
bias
a
t
ti
me
t
,
̂
=
=
a
da
pti
ve
s
ubgr
a
dient
e
s
ti
mation
with
c
or
r
e
c
ted
bias
a
t
ti
me
t
.
3.
RE
S
UL
T
AN
D
DI
S
CU
S
S
I
ON
T
he
e
xpe
r
im
e
nt
inc
ludes
the
f
oll
owing
s
c
e
na
r
ios
:
-
Da
ta
divi
de
two
pa
r
ts
,
70%
f
o
r
tr
a
ini
ng
a
nd
30
%
f
or
tes
ti
ng.
T
otal
da
ta
is
100
pa
tche
s
,
70
pa
t
c
he
s
f
or
t
r
a
ini
ng
a
nd
30
pa
tche
s
f
or
tes
ti
ng
-
T
r
a
ini
ng
pha
s
e
.
M
a
ke
s
ur
e
the
im
a
ge
s
ize
a
t
the
tr
a
ini
ng
a
nd
va
li
da
ti
on
s
uit
a
ble
with
a
pr
e
-
tr
a
ined
model.
I
f
it
is
not
ye
t
s
ize
d,
then
r
e
s
ize
the
im
a
ge
.
Anothe
r
a
lt
e
r
na
ti
ve
is
to
ge
t
a
n
a
ugmenta
ti
on
im
a
ge
to
a
utom
a
ti
c
a
ll
y
s
uit
a
ble
with
the
im
a
ge
inpu
t
s
ize
.
-
T
he
t
r
a
ini
ng
pa
r
a
mete
r
s
a
r
e
s
e
t
a
s
f
ol
lows
:
le
a
r
ning
r
a
te
1e
-
4,
m
ini
ba
tch
s
ize
4
,
max
e
poc
h
5,
va
li
da
ti
on
f
r
e
qu
e
nc
y
3
.
T
he
r
e
s
ult
s
of
a
n
e
x
pe
r
im
e
nt
a
r
e
s
hown
in
T
a
bles
2,
3,
a
nd
4
.
T
a
ble
3
s
hows
va
li
da
ti
on
us
ing
S
GD
M
a
lgor
it
hm
pr
oduc
e
s
the
be
s
t
a
c
c
ur
a
c
y
up
to
100%
us
ing
VG
G16
with
a
ti
me
of
16,
572
s
e
c
onds
.
I
n
T
a
ble
4,
va
li
da
ti
on
with
R
M
S
P
r
op
pr
oduc
e
s
the
be
s
t
a
c
c
ur
a
c
y
va
lue
o
f
up
to
93
.
3%
with
a
ti
me
of
164
.
38
s
e
c
onds
.
P
r
e
-
tr
a
ined
Ne
twor
k
us
e
d
is
R
e
s
ne
t50.
T
a
ble
5
s
hows
that
va
li
da
ti
on
us
ing
Ada
m
a
lgor
it
hm
pr
oduc
e
s
be
s
t
a
c
c
ur
a
c
y
of
96.
7
%
.
T
he
e
xpe
r
i
ment
us
ing
Ale
xne
t
with
a
pr
oc
e
s
s
ing
ti
me
o
f
36
,
274
s
e
c
onds
.
T
he
ini
ti
a
li
z
a
ti
on
of
lea
r
ning
r
a
te
,
mi
nibatc
h
s
ize
,
max
e
poc
h,
va
li
da
ti
on
f
r
e
que
nc
y,
a
nd
g
r
a
dient
de
s
c
e
nt
opti
mi
z
a
ti
on
a
lgor
it
hm
a
r
e
f
a
c
tor
s
that
inf
luenc
e
r
e
s
ult
s
of
va
li
da
ti
on
a
c
c
ur
a
c
y
a
nd
pr
oc
e
s
s
i
ng
ti
me.
L
e
a
r
ning
r
a
te
c
a
n
be
i
nit
ialize
d
s
tar
ti
ng
f
r
om
big
va
lue
unti
l
it
gr
a
dua
ll
y
s
hr
inks
.
L
e
a
r
ning
r
a
te
is
be
twe
e
n
0
a
nd
1.
I
f
lea
r
ning
r
a
te
too
lar
ge
,
it
will
c
a
us
e
ov
e
r
f
it
ti
ng,
while
lea
r
n
ing
r
a
te
va
lue
is
too
s
mall
wh
ich
will
c
a
us
e
a
longer
pr
oc
e
s
s
ing
ti
me.
T
a
ble
2
.
Ac
c
ur
a
c
y
a
nd
ti
me
pr
oc
e
s
s
ing
with
S
GD
M
P
r
e
-
tr
a
in
e
d N
e
twor
k
A
c
c
ur
a
c
y (
%
)
T
im
e
(
S
e
c
ond)
A
le
x N
e
t
93.3
8.9537
googL
e
N
e
t
86.7
12.876
R
e
s
N
e
t5
0
100
49.411
V
G
G
16
100
16.572
V
G
G
19
80
18.163
R
e
s
N
e
t1
01
93.3
126.61
I
nc
e
pt
io
n
-
V3
96.7
95.715
I
nc
e
pt
io
nR
e
s
N
e
tV2
70
352.05
S
que
e
z
e
ne
t
96.7
4.0363
T
a
ble
3
.
Ac
c
ur
a
c
y
a
nd
ti
me
pr
oc
e
s
s
ing
with
R
M
S
P
r
op
P
r
e
-
tr
a
in
e
d N
e
twor
k
A
c
c
ur
a
c
y (
%
)
T
im
e
(
S
e
c
ond)
A
le
xN
e
t
83.3
26.957
googL
e
N
e
t
90
73.062
R
e
s
N
e
t5
0
93.3
164.38
V
G
G
16
53.3
246.06
V
G
G
19
50
225.00
R
e
s
N
e
t1
01
93.3
305.7
I
nc
e
pt
io
n
-
V3
90
230
I
nc
e
pt
io
nR
e
s
N
e
tV2
90
530.06
S
que
e
z
e
ne
t
53.3
18.896
Evaluation Warning : The document was created with Spire.PDF for Python.
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
T
r
ans
fer
lear
ning
w
it
h
multi
ple
pr
e
-
tr
ained
ne
tw
or
k
fo
r
fundus
c
las
s
if
ication
(
W
ahy
udi
Se
ti
aw
an
)
1387
T
a
ble
4
.
Ac
c
ur
a
c
y
a
nd
ti
me
pr
oc
e
s
s
ing
with
a
da
m
P
r
e
-
tr
a
in
e
d N
e
twor
k
A
c
c
ur
a
c
y (
%
)
T
im
e
(
S
e
c
ond)
A
le
xN
e
t
96.7
36.274
googL
e
N
e
t
93.3
64.961
R
e
s
N
e
t5
0
90
149.93
V
G
G
16
50
126.31
V
G
G
19
50
3
88.24
R
e
s
N
e
t1
01
86.7
196.91
I
nc
e
pt
io
n
-
V3
86.7
252.1
I
nc
e
pt
io
nR
e
s
N
e
tV2
93.3
448.99
S
que
e
z
e
ne
t
96.7
44.925
T
a
ble
5
.
R
e
s
ult
c
ompar
is
on
with
the
pr
e
vious
s
tud
y
A
ut
hor
C
la
s
s
P
r
e
-
tr
a
in
e
d N
e
twor
k
A
c
c
ur
a
c
y (
%
)
T
e
nna
koon e
t
a
l.
[
11]
2
A
le
xN
e
t
98.27
L
i
e
t
a
l.
[1
3
]
2
V
G
G
m,
G
oogl
e
N
e
t
95.49
79.39
C
hoi
e
t
a
l.
[1
4
]
3
V
G
G
19
80.8
5
V
G
G
19
59.1
M
a
s
ood e
t
a
l.
[1
5
]
4
I
nc
e
pt
io
n V
3
48.2
S
a
de
k e
t
a
l.
[1
7
]
3
V
G
G
,
G
oogl
e
N
e
t,
R
e
s
N
e
t
91.23
-
92
P
r
opos
e
d M
e
th
od
2
A
le
xN
e
t
96.7
G
oogL
e
N
e
t
93.3
R
e
s
N
e
t5
0
100
V
G
G
16
100
V
G
G
19
80
R
e
s
N
e
t1
01
93.3
I
nc
e
pt
io
n
-
V3
96.7
I
nc
e
pt
io
nR
e
s
N
e
tV2
93.3
S
que
e
z
e
ne
t
96.7
M
ini
ba
tch
s
ize
will
a
f
f
e
c
t
memor
y
us
a
ge
dur
ing
pr
oc
e
s
s
ing.
S
maller
mi
nibatc
h
s
ize
r
e
qui
r
e
s
les
s
memor
y
whe
n
pr
oc
e
s
s
ing.
Ge
ne
r
a
ll
y,
mi
nibatc
h
s
ize
is
2
n
.
M
a
x
e
poc
h
va
lue
is
maximum
va
lue
that
c
a
n
be
done
to
pr
oc
e
s
s
one
f
e
e
df
or
wa
r
d
on
C
NN
.
I
ter
a
ti
on
s
tops
whe
n
a
n
e
r
r
or
is
c
ons
tant
or
whe
n
ma
x
e
poc
h
is
r
e
a
c
he
d.
Va
li
da
ti
on
f
r
e
que
nc
y
is
va
lue
given
f
or
the
number
of
va
li
da
ti
on
f
r
e
que
nc
y.
T
he
s
e
va
lues
c
a
n
be
va
r
ied
to
obtain
opti
mal
a
c
c
ur
a
c
y
a
nd
mi
nim
a
l
pr
oc
e
s
s
ing
ti
me.
T
a
ble
5
s
hows
c
ompar
is
on
be
twe
e
n
the
methods
in
thi
s
a
r
ti
c
le
with
pr
e
vious
s
tudi
e
s
.
I
n
thi
s
a
r
ti
c
le,
tr
a
ns
f
e
r
lea
r
n
ing
wa
s
done
with
mul
t
ipl
e
pr
e
-
tr
a
ined
ne
twor
ks
include
mul
ti
ple
pr
e
-
tr
a
ined
ne
twor
ks
.
T
he
r
e
s
ult
s
s
howe
d
va
li
da
ti
on
of
up
to
100%
a
c
c
ur
a
c
y
us
ing
R
e
s
Ne
t50
a
nd
VG
G16.
4.
CONC
L
USI
ON
T
r
a
ns
f
e
r
lea
r
ning
us
ing
mul
ti
ple
pr
e
-
tr
a
ined
ne
t
wor
ks
ha
s
be
e
n
made
to
de
ter
mi
ne
the
c
a
tegor
y
of
f
undus
im
a
ge
s
including
nor
mal
a
nd
ne
ova
s
c
ular
iza
ti
on.
I
t
is
us
e
d
100
pa
tche
s
take
n
f
r
om
M
E
S
S
I
DO
R
a
nd
R
e
ti
na
I
mage
B
a
nk.
T
r
a
ns
f
e
r
lea
r
ning
c
a
n
be
us
e
d
a
s
a
n
o
pti
on
to
incr
e
a
s
e
va
li
da
ti
on
a
c
c
ur
a
c
y.
T
he
e
xpe
r
im
e
nt
r
e
s
ult
s
hows
the
be
s
t
c
las
s
if
ica
ti
on
is
f
ound
in
tr
a
ns
f
e
r
lea
r
ning
us
ing
p
r
e
-
tr
a
ined
ne
twor
k
VG
G16
with
va
li
da
ti
on
a
c
c
ur
a
c
y
up
to
100
%
a
nd
ti
me
p
r
oc
e
s
s
ing
16,
572
s
e
c
onds
.
F
o
r
f
ur
ther
r
e
s
e
a
r
c
h,
we
c
a
n
us
e
ow
n
C
NN
model.
T
he
a
mount
of
da
ta
a
nd
number
of
c
las
s
e
s
ne
e
d
to
be
e
nlar
ge
d
f
or
v
a
li
da
ti
on
r
e
li
a
bil
it
y
of
C
NN
model.
RE
F
E
RE
NC
E
S
[1
]
S.
J
.
Pan
an
d
Q
.
Y
an
g
,
“A
s
u
r
v
ey
o
n
t
ra
n
s
fer
l
ear
n
i
n
g
,
”
IE
E
E
Tr
a
n
s
a
c
t
i
o
n
s
o
n
K
n
o
wl
ed
g
e
a
n
d
D
a
t
a
E
n
g
i
n
eer
i
n
g
,
v
o
l
.
2
2
,
n
o
.
1
0
,
p
p
.
1
3
4
5
-
1
3
5
9
,
O
c
t
o
b
er
2
0
1
0
.
[2
]
M.
K
.
H
el
w
a
an
d
A
.
P.
Sch
o
e
l
l
i
g
,
“Mu
l
t
i
-
R
o
b
o
t
T
ran
s
fer
L
earn
i
n
g
:
A
D
y
n
am
i
cal
S
y
s
t
em
Pers
p
ec
t
i
v
e,
”
IE
E
E
/
R
S
J
In
t
er
n
a
t
i
o
n
a
l
C
o
n
f
er
e
n
ce
o
n
I
n
t
e
l
l
i
g
e
n
t
R
o
b
o
t
s
a
n
d
S
y
s
t
e
m
s
(IR
O
S
)
,
p
p
.
4
7
0
2
-
4
7
0
8
,
2
0
1
7
.
[3
]
B.
Bo
t
o
n
d
an
d
J
.
Pet
ers
,
“A
l
i
g
n
me
n
t
-
b
as
e
d
T
ra
n
s
fer
L
earn
i
n
g
fo
r
Ro
b
o
t
Mo
d
el
s
,
”
Th
e
2
0
1
3
In
t
er
n
a
t
i
o
n
a
l
J
o
i
n
t
Co
n
f
er
e
n
ce
Neu
r
a
l
Ne
t
wo
r
ks
(IJCNN),
2
0
1
3
.
[4
]
Y
.
Z
h
u
,
Y
.
Ch
en
,
an
d
Z
.
L
u
,
“H
et
ero
g
en
e
o
u
s
T
ra
n
s
f
er
L
earn
i
n
g
f
o
r
Imag
e
Cl
a
s
s
i
fi
ca
t
i
o
n
,
”
Twen
t
y
-
F
i
f
t
h
A
A
A
I
Co
n
f
er
e
n
ce
o
n
A
r
t
i
f
i
c
i
a
l
In
t
e
l
l
i
g
e
n
ce
H
e
t
er
o
g
e
n
eo
u
s
,
p
p
.
1
3
0
4
-
1
3
0
9
,
2
0
0
8
.
[5
]
B.
Pet
ro
v
s
k
a,
I.
St
o
j
a
n
o
v
i
c,
an
d
T
.
A
t
a
n
as
o
v
a
-
p
acems
k
a,
“Cl
as
s
i
f
i
cat
i
o
n
o
f
Smal
l
Set
s
o
f
Imag
es
w
i
t
h
Pre
-
t
ra
i
n
ed
N
eu
ra
l
N
et
w
o
r
k
s
,
”
In
t
.
J.
E
n
g
.
M
a
n
u
f
.
,
v
o
l
.
4
,
p
p
.
4
0
-
5
5
,
2
0
1
8
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N
:
1693
-
6930
T
E
L
KO
M
NI
KA
T
e
lec
omm
un
C
omput
E
l
C
ontr
o
l
,
Vol.
18
,
No
.
3
,
J
une
2020:
1
382
-
1
388
1388
[6
]
Y
.
Y
o
s
h
i
d
a,
T
.
H
i
rao
,
T
.
Iw
a
t
a,
M.
N
ag
at
a,
an
d
Y
.
Mat
s
u
mo
t
o
,
“T
ra
n
s
fer
L
earn
i
n
g
f
o
r
Mu
l
t
i
p
l
e
-
D
o
mai
n
Sen
t
i
m
en
t
A
n
a
l
y
s
i
s
-
Id
e
n
t
i
fy
i
n
g
D
o
ma
i
n
D
ep
en
d
en
t
/
I
n
d
e
p
en
d
en
t
W
o
r
d
Po
l
ari
t
y
,
”
A
A
A
I
Co
n
f
e
r
en
ce
o
n
A
r
t
i
f
i
c
i
a
l
In
t
e
l
l
i
g
en
ce
,
p
p
.
1
2
8
6
-
1
2
9
1
,
2
0
1
1
.
[7
]
M.
Sh
arma,
M.
H
o
l
mes
,
J
.
Sa
n
t
amar
i
a,
A
.
Ira
n
i
,
C.
Is
b
el
l
,
an
d
A
.
Ram,
“T
ran
s
fer
L
earn
i
n
g
i
n
Real
-
T
i
me
St
ra
t
eg
y
G
ames
U
s
i
n
g
H
y
b
ri
d
CBR/
RL
,
”
IJCA
I
,
p
p
.
1
0
4
1
-
1
0
4
6
,
2
0
0
5
.
[8
]
T
.
R.
H
i
n
ri
c
h
s
an
d
K
.
D
.
Fo
r
b
u
s
,
“T
ra
n
s
fer
L
earn
i
n
g
t
h
ro
u
g
h
A
n
al
o
g
y
i
n
G
ames
,
”
A
i
M
a
g
a
z
i
n
e
,
v
o
l
.
3
2
,
n
o
.
1
,
p
p
.
7
0
-
8
3
,
2
0
1
1
.
[9
]
C.
B.
D
o
an
d
A
.
Y
.
N
g
,
“T
ra
n
s
fer
l
earn
i
n
g
fo
r
t
ex
t
c
l
as
s
i
fi
ca
t
i
o
n
,
”
Co
n
f
e
r
en
ce:
A
d
v
a
n
c
es
i
n
Ne
u
r
a
l
I
n
f
o
r
m
a
t
i
o
n
P
r
o
ces
s
i
n
g
S
y
s
t
e
m
s
1
8
[
Neu
r
a
l
I
n
f
o
r
m
a
t
i
o
n
P
r
o
ce
s
s
i
n
g
S
ys
t
em
s
]
,
2
0
0
5
.
[1
0
]
W
.
Set
i
a
w
an
,
M.
U
t
o
y
o
,
an
d
R.
Ru
l
a
n
i
n
g
t
y
a
s
,
“Cl
as
s
i
fi
ca
t
i
o
n
o
f
n
eo
v
as
c
u
l
ar
i
zat
i
o
n
u
s
i
n
g
co
n
v
o
l
u
t
i
o
n
al
n
e
u
ral
n
et
w
o
r
k
mo
d
el
,
”
TE
LK
O
M
NIK
A
Tel
eco
m
m
u
n
i
ca
t
i
o
n
Co
m
p
u
t
i
n
g
E
l
ect
r
o
n
i
c
s
a
n
d
Co
n
t
r
o
l
,
v
o
l
.
1
7
,
n
o
.
1
,
p
p
.
4
6
3
-
4
7
3
,
2
0
1
9
.
[1
1
]
R.
T
en
n
a
k
o
o
n
an
d
P.
Ro
y
,
“Imag
e
Q
u
a
l
i
t
y
C
l
as
s
i
f
i
cat
i
o
n
fo
r
D
R
Scree
n
i
n
g
U
s
i
n
g
Co
n
v
o
l
u
t
i
o
n
al
N
e
u
ra
l
Ne
t
w
o
r
k
s
,
”
P
r
o
cee
d
i
n
g
s
o
f
t
h
e
O
p
h
t
h
a
l
m
i
c
M
ed
i
ca
l
Im
a
g
e
A
n
a
l
y
s
i
s
In
t
er
n
a
t
i
o
n
a
l
W
o
r
k
s
h
o
p
,
p
p
.
1
1
3
-
1
2
0
,
2
0
1
6
.
[1
2
]
A
meri
ca
n
So
ci
e
t
y
o
f
Ret
i
n
a
Sp
ec
i
al
i
s
t
s
,
“Ret
i
n
a
Ima
g
e
Ban
k
,
”
[O
n
l
i
n
e].
A
v
ai
l
ab
l
e:
h
t
t
p
s
:
/
/
i
ma
g
eb
a
n
k
.
a
s
rs
.
o
r
g
[1
3
]
X
.
L
i
,
T
.
Pan
g
,
B.
X
i
o
n
g
,
W
.
L
i
u
,
P.
L
i
an
g
,
an
d
T
.
W
an
g
,
“Co
n
v
o
l
u
t
i
o
n
a
l
N
eu
ral
N
e
t
w
o
rk
s
Bas
ed
T
ra
n
s
fer
L
earn
i
n
g
fo
r
D
i
ab
e
t
i
c
Ret
i
n
o
p
a
t
h
y
Fu
n
d
u
s
Imag
e
Cl
as
s
i
fi
cat
i
o
n
,
”
10
th
In
t
er
n
a
t
i
o
n
a
l
Co
n
g
r
es
s
o
n
Im
a
g
e
a
n
d
S
i
g
n
a
l
P
r
o
ces
s
i
n
g
,
B
i
o
M
ed
i
ca
l
E
n
g
i
n
eer
i
n
g
a
n
d
In
f
o
r
m
a
t
i
cs
(C
IS
P
-
B
M
E
I)
,
n
o
.
9
7
8
,
2
0
1
7
.
[1
4
]
J
.
Y
.
Ch
o
i
,
T
.
K
.
Y
o
o
,
J
.
G
.
Seo
,
J
.
K
w
ak
,
T
.
T
.
U
m
,
an
d
T
.
H
.
Ri
m,
“M
u
l
t
i
-
ca
t
eg
o
ri
ca
l
d
ee
p
l
ear
n
i
n
g
n
eu
ral
n
et
w
o
r
k
t
o
c
l
as
s
i
f
y
ret
i
n
al
i
mag
e
s
:
A
p
i
l
o
t
s
t
u
d
y
emp
l
o
y
i
n
g
s
mal
l
d
a
t
ab
a
s
e,
”
P
Lo
S
O
n
e
,
p
p
.
1
-
1
6
,
2
0
1
7
.
[1
5
]
S.
Mas
o
o
d
an
d
T
.
L
u
t
h
ra,
“I
d
e
n
t
i
fi
ca
t
i
o
n
o
f
D
i
a
b
et
i
c
Ret
i
n
o
p
a
t
h
y
i
n
E
y
e
Imag
e
s
U
s
i
n
g
T
ran
s
fer
L
earn
i
n
g
,
”
In
t
e
r
n
a
t
i
o
n
a
l
Co
n
f
er
e
n
ce
o
n
Co
m
p
u
t
i
n
g
,
Co
m
m
u
n
i
c
a
t
i
o
n
a
n
d
A
u
t
o
m
a
t
i
o
n
(ICCC
A
2
0
1
7
),
n
o
.
2
,
p
p
.
1
1
8
3
-
1
1
8
7
,
2
0
1
7
.
[1
6
]
S.
O
k
t
al
o
ra,
O
.
Perd
o
m
o
,
F.
G
o
n
zal
es
,
an
d
H
.
Mu
l
l
er,
“
T
rai
n
i
n
g
D
ee
p
Co
n
v
o
l
u
t
i
o
n
a
l
N
eu
ra
l
N
e
t
w
o
rk
s
w
i
t
h
A
c
t
i
v
e
L
earn
i
n
g
fo
r
E
x
u
d
at
e
C
l
as
s
i
f
i
cat
i
o
n
i
n
E
y
e
Fu
n
d
u
s
Imag
e
s
,
”
CV
II
-
S
TE
NT
/
LA
B
E
L
S
2
0
1
7
,
LNCS
1
0
5
5
2
,
p
p
.
1
4
6
-
1
5
4
,
2
0
1
7
.
[1
7
]
I.
Sad
ek
,
M.
E
l
aw
a
d
y
,
A
.
E
l
,
an
d
R.
Sh
ab
a
y
ek
,
“
A
u
t
o
mat
i
c
Cl
as
s
i
f
i
cat
i
o
n
o
f
Br
i
g
h
t
Re
t
i
n
al
L
es
i
o
n
s
v
i
a
D
eep
N
et
w
o
r
k
Feat
u
re
s
,
”
A
r
X
i
v
,
p
p
.
1
-
2
0
,
2
0
1
7
.
[1
8
]
E
.
D
ecen
ci
ère
et
al
.
,
“Feed
b
ack
o
n
a
Pu
b
l
i
c
l
y
D
i
s
t
ri
b
u
t
ed
Imag
e
D
at
ab
a
s
e:
t
h
e
Mes
s
i
d
o
r
D
at
ab
a
s
e,
”
Im
a
g
e
A
n
a
l
.
S
t
e
r
eo
l
.
,
v
o
l
.
3
3
,
n
o
.
3
,
p
p
.
2
3
1
,
2
0
1
4
.
[1
9
]
A
.
K
ri
zh
e
v
s
k
y
a
n
d
G
.
E
.
H
i
n
t
o
n
,
“Imag
e
N
et
Cl
a
s
s
i
fi
cat
i
o
n
w
i
t
h
D
eep
Co
n
v
o
l
u
t
i
o
n
al
N
eu
ra
l
N
et
w
o
r
k
s
,”
A
d
v
a
n
ces
in
n
e
u
r
a
l
i
n
f
o
r
m
a
t
i
o
n
p
r
o
ces
s
i
n
g
s
ys
t
em
s
,
v
o
l
.
2
5
,
n
o
.
2
,
pp
. 1
-
9
,
2
0
1
2
.
[2
0
]
K
.
Si
mo
n
y
an
an
d
A
.
Z
i
s
s
erman
,
“V
ery
D
ee
p
Co
n
v
o
l
u
t
i
o
n
a
l
N
et
w
o
rk
s
fo
r
L
arg
e
-
Scal
e
Imag
e
Reco
g
n
i
t
i
o
n
,
”
ICLR
,
p
p
.
1
-
14,
2
0
1
5
.
[2
1
]
K
.
H
e,
X
.
Z
h
an
g
,
S.
Ren
an
d
J
.
Su
n
,
"
D
eep
Res
i
d
u
a
l
L
earn
i
n
g
fo
r
I
mag
e
Reco
g
n
i
t
i
o
n
,
"
IE
E
E
Co
n
f
er
e
n
ce
o
n
Co
m
p
u
t
er
V
i
s
i
o
n
a
n
d
P
a
t
t
er
n
R
ec
o
g
n
i
t
i
o
n
(CV
P
R
),
p
p
.
7
7
0
-
7
7
8
,
L
as
V
eg
a
s
,
N
V
,
2
0
1
6
.
[2
2
]
C.
Szeg
ed
y
et
al
.
,
"
G
o
i
n
g
d
eep
er
w
i
t
h
co
n
v
o
l
u
t
i
o
n
s
,
"
IE
E
E
Co
n
f
e
r
en
ce
o
n
Co
m
p
u
t
e
r
V
i
s
i
o
n
a
n
d
P
a
t
t
er
n
R
eco
g
n
i
t
i
o
n
(CV
P
R
)
,
p
p
.
1
-
9,
2
0
1
5
.
[2
3
]
Szeg
ed
y
,
V
.
V
an
h
o
u
c
k
e,
S.
Io
ffe,
J
.
Sh
l
en
s
an
d
Z
.
W
o
j
n
a,
"
Ret
h
i
n
k
i
n
g
t
h
e
In
cep
t
i
o
n
A
r
ch
i
t
ec
t
u
re
fo
r
Co
m
p
u
t
er
V
i
s
i
o
n
,
"
2
0
1
6
IE
E
E
C
o
n
f
er
e
n
ce
o
n
C
o
m
p
u
t
er
V
i
s
i
o
n
a
n
d
P
a
t
t
er
n
R
ec
o
g
n
i
t
i
o
n
(C
V
P
R
),
L
as
V
eg
as
,
N
V
,
p
p
.
2
8
1
8
-
2
8
2
6
,
2
0
1
6
.
[2
4
]
C.
Szeg
ed
y
,
S.
Io
ffe,
a
n
d
V
.
V
an
h
o
u
ck
e,
“I
n
cep
t
i
o
n
-
v
4
,
In
cep
t
i
o
n
-
Re
s
N
e
t
an
d
t
h
e
Imp
ac
t
o
f
Re
s
i
d
u
a
l
Co
n
n
ec
t
i
o
n
s
o
n
L
earn
i
n
g
,
”
A
r
X
i
v
,
p
p
.
1
-
1
2
,
2
0
1
6
.
[2
5
]
F.
N
.
Ian
d
o
l
a,
S.
H
an
,
M.
W
.
Mo
s
k
ew
i
cz,
K
.
A
s
h
raf,
W
.
J
.
D
al
l
y
,
an
d
K
.
K
e
u
t
zer,
“Sq
u
eeze
N
et
:
A
l
e
x
N
e
t
-
l
e
v
e
l
accu
racy
w
i
t
h
5
0
x
few
er
p
aramet
er
s
an
d
<
0
.
5
MB
mo
d
e
l
s
i
ze,
”
ICLR
,
p
p
.
1
-
1
3
,
2
0
1
7
.
[2
6
]
J
.
D
en
g
,
W
.
D
o
n
g
,
R.
So
c
h
er,
L
.
-
J
.
L
i
,
K
.
L
i
,
an
d
F.
-
F.
L
i
,
“Imag
eN
e
t
:
A
l
ar
g
e
-
sc
a
l
e
h
i
erarch
i
ca
l
i
ma
g
e
d
a
t
ab
a
s
e
,
”
IE
E
E
Co
n
f
.
Co
m
p
u
t
.
V
i
s
.
p
a
t
t
e
r
n
R
eco
g
n
i
t
,
p
p
.
2
4
8
-
2
5
5
,
2
0
0
9
.
[2
7
]
S.
Ru
d
er,
“A
n
o
v
erv
i
ew
o
f
g
rad
i
en
t
d
e
s
cen
t
o
p
t
i
m
i
za
t
i
o
n
,
”
A
r
X
i
v,
p
p
.
1
-
1
4
,
2
0
1
7
.
[2
8
]
M.
G
h
i
fary
,
“D
eep
L
earn
i
n
g
O
p
t
i
mi
za
t
i
o
n
,
”
2
0
1
7
.
[O
n
l
i
n
e].
A
v
ai
l
ab
l
e
:
h
t
t
p
s
:
/
/
g
h
i
f.
g
i
t
h
u
b
.
i
o
/
a
i
ml
/
2
0
1
7
/
0
4
/
1
1
/
o
p
t
i
mi
s
as
i
-
p
ad
a
-
d
eep
-
l
earn
i
n
g
.
h
t
m
l
.
[A
cce
s
s
e
d
:
2
0
-
A
u
g
-
2
0
1
8
].
[2
9
]
J
.
D
u
c
h
i
,
E
.
H
aza
n
,
an
d
Y
.
Si
n
g
er,
“A
d
ap
t
i
v
e
s
u
b
g
ra
d
i
e
n
t
me
t
h
o
d
s
fo
r
o
n
l
i
n
e
l
earn
i
n
g
a
n
d
s
t
o
ch
a
s
t
i
c
o
p
t
i
mi
z
at
i
o
n
,
”
J.
M
a
ch
.
Lea
r
n
.
R
e
s
.
,
v
o
l
.
1
2
,
p
p
.
2
1
2
1
-
2
1
5
9
,
2
0
1
1
.
[3
0
]
G
.
E
.
H
i
n
t
o
n
,
N
.
Sri
v
as
t
av
a,
an
d
K
.
Sw
ers
k
y
,
“L
ect
u
re
6
a
-
o
v
er
v
i
e
w
o
f
mi
n
i
-
b
at
c
h
g
ra
d
i
e
n
t
d
es
ce
n
t
,
”
CO
U
R
S
E
R
A
Neu
r
a
l
Net
w
o
r
k
s
M
a
ch
.
Lea
r
n
.
, p
p
.
3
1
,
2
0
1
2
.
[3
1
]
D
.
P.
K
i
n
g
ma
a
n
d
J
.
Ba,
“A
d
am:
A
Me
t
h
o
d
fo
r
S
t
o
c
h
as
t
i
c
O
p
t
i
m
i
zat
i
o
n
,
”
ICLR
,
p
p
.
1
-
15
,
2
0
15
.
Evaluation Warning : The document was created with Spire.PDF for Python.