I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
1
5
,
No
.
4
,
A
u
g
u
s
t 2
0
2
6
,
p
p
.
3
6
4
6
~
3
6
5
9
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
4
.
p
p
3
6
4
6
-
3
6
5
9
3646
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
Co
mp
a
r
a
t
iv
e e
v
a
l
ua
ti
o
n
o
f
t
ra
nsfe
r
lea
rni
ng
model
s
a
nd
G
ra
d
-
CA
M
int
er
preta
bil
it
y
f
o
r br
a
in t
um
o
r d
etec
ti
o
n f
r
o
m
M
R
I
M
d.
F
iro
z
H
a
s
a
n
1
,
M
d.
Awa
l H
a
di
1
,
Su
m
a
i
ya
Na
s
rin
1
,
M
d.
Ra
is
ul I
s
la
m
1
,
M
d.
At
ik
Sh
a
hria
r
1
,
Md
.
H
a
s
a
n M
o
o
n
1
,
T
a
nv
ir
A
hm
ed
M
o
m
in
1
,
Dew
a
n M
a
m
un
Ra
za
1
D
e
p
a
r
t
me
n
t
o
f
C
o
mp
u
t
e
r
S
c
i
e
n
c
e
a
n
d
En
g
i
n
e
e
r
i
n
g
,
F
a
c
u
l
t
y
o
f
S
c
i
e
n
c
e
a
n
d
I
n
f
o
r
mat
i
o
n
T
e
c
h
n
o
l
o
g
y
,
D
a
f
f
o
d
i
l
I
n
t
e
r
n
a
t
i
o
n
a
l
U
n
i
v
e
r
si
t
y
,
D
h
a
k
a
,
B
a
n
g
l
a
d
e
s
h
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
J
an
1
2
,
2
0
2
5
R
ev
is
ed
J
u
n
9
,
2
0
2
6
Acc
ep
ted
J
u
l 9
,
2
0
2
6
Bra
in
tu
m
o
r
c
las
sifica
ti
o
n
p
lay
s
a
n
imp
o
rtan
t
r
o
le
i
n
e
a
rly
d
iag
n
o
sis
a
n
d
trea
tme
n
t
p
lan
n
i
n
g
.
T
h
e
c
u
rre
n
t
stu
d
y
a
ims
to
e
v
a
lu
a
te
a
n
d
c
o
m
p
a
re
th
e
p
e
rfo
rm
a
n
c
e
o
f
fi
v
e
p
re
-
train
e
d
c
o
n
v
o
lu
t
io
n
a
l
n
e
u
ra
l
n
e
two
r
k
(CNN
)
m
o
d
e
ls,
n
a
m
e
ly
VG
G
1
6
,
VG
G
1
9
,
M
o
b
il
e
Ne
t,
Xc
e
p
ti
o
n
,
a
n
d
I
n
c
e
p
ti
o
n
V
3
u
sin
g
m
a
g
n
e
ti
c
re
so
n
a
n
c
e
ima
g
i
n
g
(
M
RI
)
ima
g
e
s
c
a
teg
o
rize
d
in
to
g
li
o
m
a
,
m
e
n
in
g
i
o
m
a
,
p
i
tu
it
a
r
y
t
u
m
o
r
,
a
n
d
n
o
t
u
m
o
r
c
las
se
s.
To
e
n
h
a
n
c
e
m
o
d
e
l
p
e
rfo
rm
a
n
c
e
a
n
d
a
d
d
re
ss
c
las
s
imb
a
lan
c
e
,
tran
sfe
r
lea
rn
in
g
a
n
d
d
a
ta
a
u
g
m
e
n
tatio
n
tec
h
n
iq
u
e
s
we
re
e
m
p
lo
y
e
d
.
T
o
b
o
o
st
m
o
d
e
l
in
terp
re
tab
il
it
y
,
h
e
a
tma
p
s
o
f
imp
o
rta
n
t
a
re
a
s
in
t
u
m
o
r
c
las
sifica
ti
o
n
we
re
p
ro
d
u
c
e
d
th
r
o
u
g
h
g
ra
d
ien
t
-
we
ig
h
ted
c
las
s
a
c
ti
v
a
ti
o
n
m
a
p
p
in
g
(G
ra
d
-
CAM).
M
o
b
il
e
Ne
t
wa
s
th
e
m
o
st
a
c
c
u
ra
te
wit
h
9
7
%
a
n
d
wa
s
m
o
re
p
re
c
ise
a
n
d
m
o
re
se
n
si
ti
v
e
.
T
h
e
G
ra
d
-
CAM
v
isu
a
li
z
a
ti
o
n
s
sh
o
we
d
t
h
e
m
o
d
e
ls
we
re
a
tt
e
n
d
i
n
g
to
c
li
n
ica
ll
y
re
lev
a
n
t
fe
a
tu
re
s,
wh
ic
h
in
c
re
a
se
d
th
e
i
n
terp
re
tab
i
li
ty
.
T
h
is
c
o
m
p
a
ra
ti
v
e
stu
d
y
d
e
m
o
n
stra
tes
th
e
e
ffe
c
ti
v
e
n
e
ss
o
f
th
e
in
te
g
ra
ti
o
n
o
f
e
x
p
lai
n
a
b
le
a
rti
ficia
l
in
telli
g
e
n
c
e
(
X
A
I
)
in
d
e
e
p
lea
rn
in
g
p
i
p
e
li
n
e
s
f
o
r
re
li
a
b
le
b
ra
in
tu
m
o
r
d
iag
n
o
sis.
K
ey
w
o
r
d
s
:
B
r
ain
tu
m
o
r
class
if
icatio
n
E
x
p
l
ai
n
a
b
l
e
ar
tif
ici
al
i
n
te
lli
g
e
n
ce
Hea
lth
ca
r
e
ar
tific
ial
in
tellig
en
ce
Me
d
ical
im
ag
in
g
T
r
an
s
f
er
lear
n
i
n
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Md
.
Firo
z
Hasan
Dep
ar
tm
en
t
o
f
C
o
m
p
u
te
r
Scien
ce
an
d
E
n
g
in
ee
r
in
g
,
Facu
lty
o
f
Scien
ce
an
d
I
n
f
o
r
m
at
io
n
T
ec
h
n
o
l
o
g
y
Daf
f
o
d
il I
n
te
r
n
atio
n
al
Un
i
v
er
s
ity
Daf
f
o
d
il Sm
ar
t Ci
ty
,
B
ir
u
lia,
Sav
ar
,
Dh
ak
a,
B
an
g
la
d
esh
E
m
ail:
f
ir
o
z.
cse@
d
iu
.
ed
u
.
b
d
1.
I
NT
RO
D
UCT
I
O
N
B
r
ain
tu
m
o
r
class
if
icatio
n
is
an
im
p
o
r
ta
n
t
task
in
th
e
a
r
ea
o
f
m
ed
ical
im
ag
in
g
r
esear
ch
b
ec
au
s
e
o
f
its
s
ig
n
if
ican
t
r
o
le
in
ea
r
ly
d
ia
g
n
o
s
is
an
d
tr
ea
tm
en
t
p
lan
n
i
n
g
[
1
]
.
T
r
ad
itio
n
al
m
et
h
o
d
s
r
ely
in
g
o
n
m
an
u
al
in
ter
p
r
etatio
n
a
r
e
o
f
ten
s
u
b
je
ctiv
e
an
d
in
c
o
n
s
is
ten
t
[
2
]
.
T
h
is
h
as
p
r
o
m
p
ted
th
e
d
ev
elo
p
m
en
t
o
f
ar
tific
ial
in
tellig
en
ce
(
AI
)
,
esp
ec
ially
d
ee
p
lear
n
in
g
,
to
im
p
r
o
v
e
th
e
a
cc
u
r
ac
y
an
d
r
eliab
ilit
y
o
f
d
iag
n
o
s
is
[
3
]
.
C
o
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
s
(
C
NNs)
h
av
e
s
h
o
w
n
ex
c
ellen
t
p
er
f
o
r
m
an
ce
f
o
r
th
e
a
n
aly
s
is
o
f
co
m
p
lex
m
ag
n
etic
r
eso
n
an
ce
im
ag
in
g
(
MRI)
d
ata.
R
ec
en
tly
,
p
r
e
-
tr
ain
e
d
m
o
d
els
s
u
ch
as
VGG1
6
,
VGG1
9
,
Mo
b
ileNet,
Xce
p
tio
n
,
an
d
I
n
ce
p
tio
n
V3
h
a
v
e
b
ee
n
a
p
p
li
ed
with
h
ig
h
ac
cu
r
ac
y
f
o
r
t
h
e
task
s
o
f
tu
m
o
r
class
if
icatio
n
[
4
]
,
[
5
]
.
T
h
ese
m
o
d
els
h
av
e
b
ee
n
f
u
r
th
er
im
p
r
o
v
e
d
th
r
o
u
g
h
tr
an
s
f
e
r
lear
n
in
g
,
wh
ich
allo
ws
ef
f
icien
t
f
ea
tu
r
e
ex
tr
ac
tio
n
f
r
o
m
s
m
aller
m
ed
ical
d
atasets
[
6
]
,
[
7
]
.
Ho
wev
e
r
,
ch
allen
g
es
r
e
m
ain
to
en
s
u
r
e
h
ig
h
g
en
er
aliza
b
ilit
y
ac
r
o
s
s
d
if
f
er
e
n
t
tu
m
o
r
ty
p
es
[
8
]
.
T
h
e
p
r
es
en
t
s
tu
d
y
aim
s
to
o
v
er
co
m
e
s
u
ch
lim
itatio
n
s
b
y
co
m
p
ar
in
g
m
u
ltip
le
f
in
e
-
tu
n
e
d
C
NN
m
o
d
els
f
o
r
th
e
class
if
icatio
n
o
f
g
lio
m
a,
m
en
in
g
io
m
a,
p
itu
itar
y
tu
m
o
r
s
,
an
d
n
o
tu
m
o
r
ca
s
es
[
9
]
u
s
in
g
d
ata
au
g
m
en
tatio
n
an
d
o
v
er
s
am
p
lin
g
to
h
an
d
le
class
im
b
alan
c
e
[
1
0
]
.
T
h
e
s
tu
d
y
also
in
cl
u
d
es
ex
p
l
ain
ab
le
ar
tific
ial
in
tellig
en
ce
(
XAI
)
in
th
e
f
o
r
m
o
f
g
r
ad
ie
n
t
-
weig
h
ted
class
ac
tiv
atio
n
m
ap
p
in
g
(
G
r
ad
-
C
AM
)
h
ea
tm
ap
s
to
h
ig
h
lig
h
t
th
e
k
e
y
ar
ea
s
th
at
in
f
l
u
en
ce
p
r
ed
ictio
n
s
,
th
u
s
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
o
mp
a
r
a
tive
ev
a
lu
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
mo
d
els a
n
d
Gr
a
d
-
C
A
M in
terp
r
eta
b
ilit
y
…
(
Md
.
F
ir
o
z
Ha
s
a
n
)
3647
in
cr
ea
s
in
g
in
ter
p
r
etab
ilit
y
f
o
r
clin
ician
s
.
Mo
b
ileNet
ac
h
iev
e
d
th
e
b
est
p
e
r
f
o
r
m
an
ce
with
9
7
%
ac
cu
r
ac
y
a
n
d
it
also
ex
ce
lled
in
p
r
ec
is
io
n
an
d
r
ec
all.
T
h
e
Gr
a
d
-
C
AM
o
u
t
p
u
ts
co
n
f
i
r
m
ed
th
e
r
eliab
ilit
y
o
f
t
h
e
m
o
d
el
b
y
in
d
icatin
g
th
e
im
p
o
r
tan
t
r
eg
io
n
s
o
f
th
e
im
a
g
es th
at
co
n
tr
ib
u
t
ed
to
its
d
ec
is
io
n
s
.
T
h
e
r
e
v
iewin
g
liter
atu
r
e
h
ig
h
l
ig
h
ts
r
ap
id
p
r
o
g
r
ess
an
d
in
cr
e
asin
g
im
p
ac
t
o
f
d
ee
p
lear
n
in
g
tech
n
iq
u
es
in
m
ed
ical
im
a
g
e
class
if
icatio
n
,
s
p
ec
if
ically
f
o
r
d
etec
tio
n
o
f
b
r
ain
tu
m
o
r
s
.
B
o
u
h
af
r
a
an
d
E
l
B
ah
i
[
1
1
]
,
r
ev
iewe
d
6
0
s
tu
d
ies
(
2
0
2
0
-
2
0
2
4
)
a
n
d
h
ig
h
lig
h
te
d
th
e
ef
f
ec
tiv
en
ess
o
f
m
eth
o
d
s
s
u
ch
as
tr
an
s
f
er
lear
n
in
g
,
au
to
en
co
d
er
s
,
tr
an
s
f
o
r
m
er
s
,
a
n
d
atten
tio
n
m
ec
h
an
is
m
s
in
en
h
an
cin
g
d
iag
n
o
s
tic
p
r
ec
is
io
n
an
d
co
m
p
u
tatio
n
al
ef
f
icien
cy
,
d
esp
ite
c
h
allen
g
es
s
u
ch
as
d
ata
h
eter
o
g
en
eity
an
d
lim
ited
i
n
ter
p
r
etab
ilit
y
.
B
h
a
g
y
alax
m
i
et
a
l.
[
1
2
]
em
p
h
asized
th
e
u
s
ef
u
ln
ess
o
f
d
ee
p
lear
n
in
g
in
m
u
lti
-
g
r
ad
e
tu
m
o
r
class
if
icatio
n
.
Ag
ar
wal
et
a
l.
[
1
3
]
r
ep
o
r
ted
9
9
.
5
4
%
ac
cu
r
ac
y
f
o
r
b
r
ain
tu
m
o
r
d
etec
tio
n
u
s
in
g
R
esNet5
0
.
Nea
m
ah
et
a
l.
[
1
0
]
s
im
ilar
ly
id
en
tifie
d
s
o
m
e
o
f
th
e
k
ey
c
h
allen
g
es
o
f
C
NN
-
b
ased
ap
p
r
o
ac
h
es,
s
u
c
h
as
th
e
n
ee
d
f
o
r
lar
g
e
an
n
o
tated
d
atasets
an
d
h
ig
h
m
o
d
el
c
o
m
p
lex
ity
.
Du
e
to
th
e
ir
ab
ilit
y
t
o
a
u
to
m
atica
lly
lear
n
co
m
p
lex
f
ea
tu
r
es
[
1
4
]
,
[
1
5
]
,
C
NNs
h
av
e
s
h
o
w
n
g
r
ea
t
s
u
cc
ess
in
d
is
tin
g
u
is
h
in
g
th
e
ty
p
es
o
f
tu
m
o
r
s
as
co
m
p
ar
ed
to
tr
a
d
itio
n
al
m
ac
h
in
e
l
ea
r
n
in
g
alg
o
r
ith
m
s
,
s
u
ch
as
g
lio
m
a,
m
en
in
g
i
o
m
a
,
an
d
p
itu
itar
y
.
B
y
u
s
in
g
tr
a
n
s
f
er
lear
n
i
n
g
,
I
m
p
r
o
v
e
d
C
N
N
m
o
d
els
s
u
c
h
as
Go
o
g
L
eNe
t
an
d
R
esNet5
0
ac
h
iev
ed
ac
cu
r
ac
ies
o
f
9
9
.
6
7
%
an
d
9
9
.
9
1
%
,
r
esp
ec
tiv
ely
with
b
etter
p
er
f
o
r
m
a
n
ce
an
d
co
n
v
er
g
e
n
ce
[
1
6
]
,
[
1
7
]
.
So
m
e
s
tu
d
ies
also
lo
o
k
ed
in
to
th
e
g
r
ad
in
g
o
f
g
lio
m
as,
wh
ic
h
s
h
o
ws
th
e
v
er
s
atility
o
f
d
ee
p
lear
n
in
g
in
class
if
y
in
g
b
r
ain
tu
m
o
r
s
.
T
h
e
s
tan
d
ar
d
class
if
icatio
n
p
ip
elin
e
co
n
s
is
ts
o
f
p
r
e
p
r
o
ce
s
s
in
g
o
f
MRI
im
ag
es,
f
ea
tu
r
e
ex
tr
ac
tio
n
a
n
d
class
if
icatio
n
alg
o
r
ith
m
s
[
1
8
]
.
Me
th
o
d
s
lik
e
d
is
cr
ete
wav
elet
tr
an
s
f
o
r
m
(
DW
T
)
an
d
g
r
ey
lev
el
co
-
o
cc
u
r
r
en
ce
m
at
r
ix
(
GL
C
M)
ar
e
u
s
ef
u
l
f
o
r
p
r
ep
r
o
ce
s
s
in
g
an
d
tex
t
u
r
e
an
al
y
s
is
,
an
d
p
r
in
cip
al
co
m
p
o
n
en
t
an
aly
s
is
(
PC
A)
is
u
s
ed
to
r
ed
u
ce
th
e
d
im
en
s
io
n
a
lity
o
f
f
ea
tu
r
es
with
o
u
t
lo
s
s
o
f
ac
cu
r
ac
y
.
Su
p
p
o
r
t
v
ec
to
r
m
ac
h
in
es
(
SVMs)
h
av
e
b
ee
n
s
h
o
wn
to
b
e
h
ig
h
ly
ef
f
ec
tiv
e
f
o
r
b
o
th
b
in
a
r
y
an
d
m
u
lti
-
class
class
if
icatio
n
task
s
with
th
e
r
ig
h
t
k
er
n
els
[
1
9
]
–
[
2
1
]
.
B
esid
es
b
r
ain
tu
m
o
r
class
if
icati
o
n
,
m
ed
ical
im
ag
e
class
if
icatio
n
h
as
b
ee
n
ex
ten
d
ed
to
o
th
er
m
o
d
alities
s
u
ch
as
MRI,
co
m
p
u
ted
to
m
o
g
r
ap
h
y
(
CT
)
,
X
-
r
ay
,
a
n
d
f
u
n
d
u
s
im
ag
in
g
u
s
in
g
m
o
d
els
lik
e
SVMs,
B
ay
esian
n
etwo
r
k
s
,
n
eu
r
al
n
etwo
r
k
s
,
k
-
n
ea
r
est
n
eig
h
b
o
r
s
(
KNN)
,
an
d
f
u
zz
y
l
o
g
ic
[
2
2
]
.
Stan
d
a
r
d
class
if
icatio
n
p
ip
elin
es
ar
e
co
m
p
o
s
ed
o
f
d
ata
ac
q
u
is
itio
n
,
p
r
ep
r
o
ce
s
s
in
g
,
f
ea
tu
r
e
s
elec
tio
n
,
an
d
m
o
d
el
t
r
ain
in
g
[
2
3
]
.
T
h
e
u
s
u
al
tech
n
iq
u
es
ar
e
te
x
tu
r
e
an
aly
s
is
,
n
e
u
r
al
n
etwo
r
k
s
,
an
d
d
ata
m
in
in
g
,
an
d
th
e
ev
alu
ati
o
n
is
u
s
u
ally
b
ased
o
n
ac
cu
r
a
cy
,
s
en
s
itiv
ity
,
s
p
ec
if
icity
,
an
d
r
u
n
tim
e
[
2
4
]
,
[
2
5
]
.
C
o
llectiv
ely
,
th
ese
s
tu
d
ies
h
ig
h
lig
h
t
th
e
p
o
ten
tial
o
f
d
ee
p
lea
r
n
in
g
f
o
r
im
p
r
o
v
i
n
g
th
e
ac
c
u
r
ac
y
,
ef
f
icien
cy
,
an
d
clin
ical
u
tili
ty
o
f
b
r
ain
tu
m
o
r
d
iag
n
o
s
is
,
wh
ile
ac
k
n
o
wled
g
in
g
th
e
o
n
g
o
in
g
ch
alle
n
g
es
ass
o
ciate
d
with
d
ata
v
ar
iab
ilit
y
an
d
c
o
m
p
u
tatio
n
al
r
eq
u
ir
em
e
n
ts
.
T
h
e
s
tu
d
y
s
h
o
ws
th
e
tr
a
n
s
f
o
r
m
ativ
e
p
o
ten
tial
o
f
AI
-
d
r
i
v
en
s
o
lu
tio
n
s
f
o
r
m
ed
ical
d
iag
n
o
s
tics
.
I
t
h
elp
s
t
o
b
u
ild
s
tr
o
n
g
,
ac
c
u
r
ate
,
a
n
d
in
ter
p
r
eta
b
le
b
r
ain
tu
m
o
r
class
if
icatio
n
s
y
s
tem
s
th
at
ca
n
s
u
p
p
o
r
t
r
ad
io
lo
g
is
ts
an
d
r
eliev
e
d
iag
n
o
s
tic
wo
r
k
lo
ad
s
i
n
clin
ical
s
e
ttin
g
s
b
y
c
o
m
b
in
i
n
g
s
o
p
h
is
ticated
d
ee
p
lear
n
in
g
m
o
d
els
with
ex
p
lain
ab
ilit
y
te
ch
n
iq
u
es.
Fu
tu
r
e
wo
r
k
will
f
o
cu
s
o
n
ex
p
a
n
d
in
g
th
e
d
ataset,
ex
p
lo
r
in
g
n
o
v
e
l
ar
ch
itectu
r
es,
an
d
im
p
r
o
v
i
n
g
in
ter
p
r
eta
b
ilit
y
m
eth
o
d
s
to
f
u
r
t
h
er
im
p
r
o
v
e
class
if
icatio
n
ac
cu
r
ac
y
an
d
clin
ical
ap
p
licab
ilit
y
.
2.
M
E
T
H
O
D
2
.
1
.
Da
t
a
s
et
des
cr
iptio
n
T
h
e
b
r
ain
tu
m
o
r
class
if
icatio
n
d
ataset
in
clu
d
es
MRI
im
ag
es
o
f
f
o
u
r
class
es:
g
lio
m
a,
m
en
in
g
io
m
a,
n
o
tu
m
o
r
,
a
n
d
p
itu
itar
y
tu
m
o
r
.
T
h
is
class
if
icatio
n
f
ac
ilit
ates
s
u
p
er
v
is
ed
lear
n
i
n
g
b
y
ass
o
ciatin
g
ea
ch
im
ag
e
with
its
lab
el
an
d
th
u
s
allo
win
g
t
h
e
m
o
d
els
to
r
ec
o
g
n
ize
p
atte
r
n
s
s
p
ec
if
ic
to
ea
c
h
ty
p
e
o
f
tu
m
o
r
.
T
h
e
d
ataset
co
n
tain
s
2
2
,
8
4
8
im
a
g
es.
T
h
e
n
u
m
b
er
o
f
im
a
g
es
f
o
r
ea
c
h
tu
m
o
r
is
6
,
3
8
0
im
ag
es
f
o
r
n
o
tu
m
o
r
,
5
,
8
2
8
im
ag
es
f
o
r
p
itu
itar
y
,
5
,
3
5
6
im
ag
es f
o
r
m
en
in
g
io
m
a
,
an
d
5
,
2
8
4
im
a
g
es f
o
r
g
lio
m
a.
2
.
2
.
Da
t
a
s
et
prepro
ce
s
s
ing
2
.
2
.
1
.
Resizing
a
nd
re
s
ca
lin
g
R
esizin
g
an
d
r
escalin
g
n
o
r
m
alize
th
e
in
p
u
t
im
ag
es
an
d
h
elp
in
co
m
p
u
tatio
n
al
ef
f
icien
cy
.
I
n
co
n
s
is
ten
t
in
p
u
t
s
h
ap
es
d
u
e
to
d
if
f
er
e
n
t
im
ag
e
s
izes
ca
n
m
ak
e
m
o
d
el
tr
ai
n
in
g
m
o
r
e
d
if
f
icu
lt.
R
escalin
g
p
ix
el
in
ten
s
ity
v
alu
es
to
a
co
m
m
o
n
s
ca
le
im
p
r
o
v
es
n
u
m
er
ical
s
tab
ilit
y
an
d
s
p
ee
d
s
u
p
c
o
n
v
er
g
en
ce
d
u
r
in
g
tr
ain
in
g
o
f
th
e
m
o
d
el.
Fo
r
d
e
ep
lear
n
in
g
m
o
d
els
s
u
ch
as
VGG
19
,
I
n
ce
p
tio
n
,
an
d
Mo
b
ile
Net,
all
th
e
im
ag
es
ar
e
r
esized
to
2
2
4
×
2
2
4
p
ix
els.
T
h
is
g
u
ar
a
n
tees
a
u
n
if
o
r
m
i
n
p
u
t
d
im
en
s
io
n
f
o
r
th
e
d
ataset,
wh
ich
is
n
ec
ess
ar
y
f
o
r
b
atch
p
r
o
ce
s
s
in
g
an
d
efficien
t m
em
o
r
y
u
s
ag
e.
I
m
ag
es a
r
e
r
escaled
s
u
ch
th
at
p
ix
el
v
alu
es r
an
g
e
f
r
o
m
0
to
1
to
n
o
r
m
alize
th
em
.
T
h
e
in
ten
s
ity
o
f
ea
c
h
p
i
x
el
is
s
ca
led
to
[
0
,
1
]
b
y
d
i
v
id
in
g
b
y
2
5
5
(
t
h
e
m
ax
im
u
m
p
ix
el
in
ten
s
ity
v
alu
e
in
an
8
-
b
it
im
ag
e)
.
T
h
is
p
r
e
v
en
ts
lar
g
e
in
p
u
t
v
alu
es
,
wh
ich
ca
n
h
i
n
d
er
t
h
e
lear
n
in
g
o
f
th
e
n
eu
r
al
n
etwo
r
k
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
6
4
6
-
3
6
5
9
3648
2
.
2
.
2
.
L
a
bel
enco
di
ng
T
h
e
m
o
d
el
r
eq
u
ir
es
lab
el
en
c
o
d
in
g
to
co
n
v
er
t
th
e
ca
teg
o
r
i
ca
l
lab
els
(
g
lio
m
a
,
m
e
n
in
g
io
m
a
,
an
d
no
tu
m
o
r
p
itu
itar
y
)
t
o
n
u
m
er
ic
v
alu
es
to
p
r
o
ce
s
s
.
Ma
ch
in
e
lea
r
n
in
g
m
o
d
els
ex
p
ec
t
n
u
m
er
ic
in
p
u
ts
,
s
o
we
will
co
n
v
er
t th
e
lab
els to
in
teg
e
r
v
alu
es th
at
th
e
m
o
d
el
ca
n
wo
r
k
with
.
E
ac
h
ca
teg
o
r
y
is
ass
ig
n
ed
a
s
ep
ar
ate
in
teg
er
u
s
in
g
‘
L
a
b
elE
n
co
d
er
’
f
r
o
m
‘
Sk
lear
n
’
lib
r
ar
y
.
Fo
r
ex
am
p
le,
g
lio
m
a
ca
n
b
e
en
c
o
d
ed
as
0
,
m
en
in
g
io
m
a
as
1
,
no
tu
m
o
r
as
2
,
an
d
p
it
u
itar
y
as
3
.
T
h
is
m
ap
p
in
g
f
r
o
m
a
ca
teg
o
r
ical
lab
el
to
its
en
co
d
ed
in
teg
er
v
alu
e
is
ex
p
r
ess
ed
in
(
1
)
.
=
(
)
(
1
)
W
h
er
e
r
ep
r
esen
ts
th
e
ca
teg
o
r
i
ca
l
lab
el
(
e.
g
.
,
'
g
lio
m
a'
)
an
d
f
is
th
e
f
u
n
ctio
n
th
at
m
ap
s
it
to
a
n
in
teg
er
v
alu
e
.
2
.
2
.
3
.
Da
t
a
a
ug
m
ent
a
t
io
n
Data
au
g
m
en
tatio
n
ca
n
in
cr
e
ase
th
e
d
iv
er
s
ity
o
f
th
e
tr
ai
n
i
n
g
s
et
with
o
u
t
th
e
n
ee
d
to
c
o
llect
m
o
r
e
d
ata.
I
t
also
h
elp
s
th
e
m
o
d
el
to
g
en
er
alize
b
etter
(
b
y
tr
ain
in
g
o
n
d
if
f
e
r
en
t
tr
an
s
f
o
r
m
atio
n
s
o
f
th
e
s
am
e
im
ag
e)
.
Au
g
m
en
tatio
n
tech
n
iq
u
es
s
u
c
h
as
s
h
if
tin
g
,
zo
o
m
in
g
,
an
d
f
lip
p
in
g
cr
ea
te
n
ew
im
a
g
e
v
ar
i
atio
n
s
wh
ich
r
ed
u
ce
o
v
er
f
itti
n
g
a
n
d
im
p
r
o
v
e
r
o
b
u
s
tn
ess
.
T
h
e
im
ag
e
is
tr
an
s
lated
b
y
a
r
atio
o
f
th
e
wid
th
in
h
o
r
izo
n
tal
s
h
if
tin
g
.
Fo
r
ex
am
p
le,
if
th
e
wid
th
s
h
if
t
r
an
g
e
is
0
.
2
,
th
e
im
ag
es
ca
n
b
e
s
h
if
ted
lef
t
o
r
r
ig
h
t
b
y
u
p
to
2
0
%
o
f
t
h
e
im
a
g
e
wid
th
.
T
h
is
au
g
m
e
n
tatio
n
s
tr
ateg
y
f
o
llo
ws estab
lis
h
ed
p
r
ac
tice
f
o
r
im
p
r
o
v
in
g
d
ee
p
lear
n
in
g
g
en
e
r
aliza
tio
n
o
n
lim
ited
d
atasets
[
2
6
]
.
T
h
is
h
o
r
izo
n
tal
s
h
if
tin
g
o
p
e
r
atio
n
i
s
ex
p
r
ess
ed
in
(
2
)
.
′
=
+
×
ℎ
(
2
)
Her
e
is
th
e
o
r
ig
in
al
h
o
r
izo
n
t
al
p
o
s
itio
n
,
an
d
α
is
a
r
an
d
o
m
f
ac
to
r
in
th
e
r
an
g
e
o
f
s
h
if
t
.
Ver
tical
s
h
if
tin
g
tr
an
s
lates
th
e
im
ag
e
u
p
o
r
d
o
wn
b
y
a
f
r
ac
tio
n
o
f
th
e
im
ag
e
h
eig
h
t.
T
h
e
r
a
n
g
e
is
0
.
2
,
m
ea
n
in
g
th
at
im
ag
es
ca
n
b
e
s
h
if
ted
u
p
o
r
d
o
wn
b
y
u
p
to
2
0
% o
f
th
e
im
ag
e
h
eig
h
t.
T
h
is
v
er
tical
s
h
if
tin
g
o
p
er
atio
n
is
ex
p
r
ess
ed
in
(
3
)
.
′
=
+
×
ℎ
ℎ
(
3
)
W
h
er
e
is
th
e
o
r
ig
in
al
v
er
tical
p
o
s
itio
n
an
d
is
a
r
an
d
o
m
f
ac
t
o
r
in
th
e
r
an
g
e
o
f
th
e
s
h
if
t.
Z
o
o
m
in
g
in
o
r
o
u
t
alter
s
th
e
p
er
ce
iv
ed
d
is
tan
ce
o
f
th
e
im
ag
e.
T
h
e
zo
o
m
r
atio
is
0
.
2
,
wh
ic
h
m
ea
n
s
th
at
th
e
im
ag
e
ca
n
b
e
zo
o
m
e
d
in
o
r
o
u
t b
y
u
p
t
o
2
0
%.
T
h
is
zo
o
m
in
g
tr
an
s
f
o
r
m
atio
n
is
ex
p
r
ess
ed
in
(
4
)
.
=
(
,
(
1
±
)
×
)
(
4
)
W
h
er
e
is
a
r
an
d
o
m
f
ac
to
r
i
n
th
e
zo
o
m
r
an
g
e
.
T
h
is
m
eth
o
d
r
ev
er
s
es
th
e
im
ag
e
h
o
r
izo
n
tally
,
cr
ea
tin
g
a
m
ir
r
o
r
im
a
g
e.
Ho
r
izo
n
tal
f
lip
p
in
g
ass
is
ts
th
e
m
o
d
el
in
lear
n
in
g
i
n
v
ar
ian
ce
to
o
r
ien
tatio
n
.
T
h
is
h
o
r
izo
n
tal
f
lip
p
in
g
o
p
er
atio
n
is
ex
p
r
ess
ed
in
(
5
)
.
=
(
,
=
1
)
(
5
)
2
.
2
.
4
.
Cla
s
s
ba
la
ncing
(
o
v
er
s
a
m
pli
ng
)
C
las
s
b
alan
cin
g
is
a
tech
n
iq
u
e
to
h
an
d
le
th
e
im
b
alan
ce
s
in
t
h
e
s
am
p
le
s
izes
am
o
n
g
m
o
d
el
s
to
av
o
id
b
iased
p
r
ed
ictio
n
.
Ov
e
r
s
am
p
lin
g
d
u
p
licates
m
in
o
r
ity
in
s
tan
ce
s
to
en
s
u
r
e
th
at
all
cl
ass
es
ar
e
eq
u
ally
r
ep
r
esen
ted
.
T
h
is
s
tu
d
y
u
s
e
t
h
e
“Ran
d
o
m
Ov
e
r
Sam
p
ler
”
f
r
o
m
th
e
“
I
m
b
lear
n
”
lib
r
a
r
y
t
o
d
u
p
licate
s
am
p
les
f
r
o
m
u
n
d
er
-
r
ep
r
esen
te
d
class
es u
n
til all
class
e
s
h
av
e
eq
u
al
n
u
m
b
er
s
.
I
t h
el
p
s
to
in
cr
ea
s
e
th
e
s
ize
o
f
th
e
d
ataset
wh
ile
k
ee
p
in
g
th
e
class
b
alan
ce
,
th
u
s
r
ed
u
ci
n
g
th
e
p
o
s
s
ib
ilit
y
o
f
th
e
m
o
d
el
f
a
v
o
r
in
g
a
n
y
o
n
e
class
.
T
h
is
o
v
er
s
am
p
lin
g
s
tr
ateg
y
is
co
n
s
is
ten
t
with
s
tan
d
ar
d
ap
p
r
o
a
ch
es
f
o
r
h
an
d
lin
g
class
im
b
alan
ce
in
m
ac
h
in
e
lear
n
in
g
p
ip
elin
es
[
2
7
]
.
T
h
is
o
v
er
s
am
p
lin
g
p
r
o
ce
s
s
,
wh
ich
e
q
u
alize
s
th
e
n
u
m
b
er
o
f
s
am
p
le
s
ac
r
o
s
s
all
class
e
s
,
is
ex
p
r
ess
ed
in
(
6
)
.
=
(
6
)
W
h
er
e
is
th
e
n
ew
n
u
m
b
er
o
f
s
am
p
les
f
o
r
class
,
eq
u
al
is
ed
am
o
n
g
all
class
es.
T
h
ese
p
r
ep
r
o
ce
s
s
in
g
tech
n
iq
u
es
ar
e
c
o
m
b
in
e
d
to
ef
f
ec
tiv
ely
p
r
e
p
ar
e
th
e
d
ataset,
en
ab
lin
g
b
etter
m
o
d
el
p
er
f
o
r
m
an
ce
a
n
d
m
o
r
e
r
o
b
u
s
t r
esu
lts
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
o
mp
a
r
a
tive
ev
a
lu
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
mo
d
els a
n
d
Gr
a
d
-
C
A
M in
terp
r
eta
b
ilit
y
…
(
Md
.
F
ir
o
z
Ha
s
a
n
)
3649
2
.
3
.
Xce
ptio
n
m
o
del w
it
h a
t
t
ent
io
n m
ec
ha
nis
m
Xce
p
tio
n
is
a
d
ee
p
C
NN
[
2
8
]
,
wh
ich
is
ef
f
icien
t
in
ex
tr
ac
tin
g
co
m
p
lex
p
atter
n
s
.
I
t
r
ep
lace
s
th
e
s
tan
d
ar
d
I
n
ce
p
tio
n
m
o
d
u
les
w
ith
d
ep
th
wis
e
s
ep
ar
ab
le
co
n
v
o
lu
tio
n
s
,
wh
ich
d
ec
o
u
p
le
s
p
atia
l
an
d
c
h
an
n
el
-
wis
e
f
ea
tu
r
e
lear
n
i
n
g
to
ca
p
tu
r
e
v
is
u
al
p
atter
n
s
m
o
r
e
e
f
f
icien
tl
y
.
Af
ter
th
e
f
ea
tu
r
e
ex
tr
ac
ti
o
n
,
th
e
o
u
tp
u
ts
o
f
Xce
p
tio
n
ar
e
r
esh
ap
ed
to
f
ee
d
in
to
a
m
u
lti
-
h
ea
d
atten
tio
n
lay
er
.
T
h
is
atten
tio
n
m
ec
h
an
is
m
is
u
s
ef
u
l
to
f
o
cu
s
o
n
s
p
ec
ific
r
eg
io
n
s
in
t
h
e
b
r
ai
n
MRI
s
ca
n
s
th
at
ar
e
m
o
r
e
li
k
ely
to
co
n
tain
in
f
o
r
m
atio
n
a
b
o
u
t
t
h
e
tu
m
o
r
.
T
h
e
m
u
lti
-
h
ea
d
atten
tio
n
m
ec
h
a
n
is
m
u
s
ed
h
er
e
f
o
llo
ws
th
e
f
o
r
m
u
latio
n
in
tr
o
d
u
ce
d
f
o
r
tr
an
s
f
o
r
m
er
-
b
ased
ar
ch
itectu
r
es
[
2
9
]
,
as e
x
p
r
ess
ed
in
(
7
)
.
=
(
/
√
)
(
7
)
W
h
er
e
,
,
an
d
ar
e
t
h
e
v
ec
t
o
r
s
o
f
t
h
e
q
u
er
ies,
k
ey
s
,
an
d
v
alu
es
,
r
esp
ec
tiv
ely
,
an
d
is
th
e
d
i
m
en
s
io
n
o
f
th
e
k
ey
v
ec
to
r
s
.
T
h
e
C
NN
m
o
d
el
u
s
es
a
g
lo
b
al
av
er
ag
e
p
o
o
lin
g
lay
er
to
r
ed
u
ce
th
e
s
p
atial
d
im
en
s
io
n
s
wh
ile
r
etain
in
g
f
ea
tu
r
e
i
n
f
o
r
m
atio
n
.
I
t
also
u
s
es
d
r
o
p
o
u
t
lay
er
s
f
o
r
r
eg
u
lar
is
atio
n
,
a
s
o
f
tm
a
x
o
u
tp
u
t
lay
e
r
f
o
r
class
if
icatio
n
in
to
f
o
u
r
class
es
an
d
a
m
u
lti
-
h
ea
d
atten
tio
n
lay
er
f
o
r
en
h
an
ce
d
f
ea
tu
r
e
ex
tr
ac
tio
n
.
I
t
u
tili
ze
s
d
r
o
p
o
u
t
la
y
e
r
s
,
G
a
u
s
s
ia
n
n
o
is
e
a
n
d
b
atc
h
n
o
r
m
ali
za
ti
o
n
an
d
r
ec
ti
f
i
ed
l
in
ea
r
u
n
it
(
R
e
L
U
)
ac
t
iv
a
ti
o
n
s
t
o
p
r
e
v
e
n
t
o
v
e
r
f
itti
n
g
a
n
d
t
o
r
et
ai
n
th
e
f
e
atu
r
e
i
n
f
o
r
m
a
ti
o
n
.
Sp
a
r
s
e
ca
te
g
o
r
i
ca
l
cr
o
s
s
-
e
n
t
r
o
p
y
l
o
s
s
a
n
d
Ad
a
m
o
p
t
im
ize
r
a
r
e
u
s
e
d
f
o
r
m
o
d
el
tr
ai
n
i
n
g
.
T
h
e
o
u
t
p
u
t
o
b
t
ai
n
e
d
b
y
c
o
m
b
i
n
i
n
g
m
u
lti
p
l
e
att
en
ti
o
n
h
ea
d
s
is
e
x
p
r
ess
e
d
i
n
(
8
)
.
=
(
,
,
)
(
8
)
W
h
er
e
,
,
an
d
m
ec
h
an
is
m
s
r
ep
r
esen
t th
e
q
u
er
ies,
k
ey
s
an
d
v
alu
es m
atr
ices u
s
ed
in
atten
tio
n
m
o
d
els
.
T
h
e
Ad
am
o
p
tim
izer
a
n
d
ca
teg
o
r
i
ca
l
cr
o
s
s
-
en
tr
o
p
y
ar
e
u
s
ed
to
m
in
im
ize
class
if
icatio
n
er
r
o
r
s
.
T
h
is
co
m
b
in
atio
n
h
elp
s
to
h
av
e
s
tab
le
an
d
ef
f
ec
tiv
e
g
r
ad
ien
t
u
p
d
ates.
T
h
e
Ad
am
o
p
tim
izer
is
wid
ely
ad
o
p
ted
f
o
r
its
ad
ap
tiv
e
lear
n
in
g
r
ate
a
n
d
f
ast
co
n
v
er
g
en
ce
p
r
o
p
er
ties
[
3
0
]
.
T
h
e
s
p
ar
s
e
ca
teg
o
r
ical
c
r
o
s
s
-
en
tr
o
p
y
l
o
s
s
f
u
n
ctio
n
u
s
ed
t
o
tr
ain
th
e
m
o
d
el
is
ex
p
r
ess
ed
in
(
9
)
.
=
−
∑
.
(
̂
)
=
1
(
9
)
W
h
er
e
̂
is
th
e
p
r
ed
icted
p
r
o
b
a
b
ilit
y
f
o
r
class
an
d
is
th
e
tr
u
e
lab
el.
T
h
e
tr
ain
in
g
is
m
o
n
it
o
r
ed
an
d
ea
r
ly
s
to
p
p
in
g
is
u
s
ed
to
s
to
p
tr
ain
i
n
g
if
th
er
e
is
n
o
im
p
r
o
v
e
m
en
t
in
v
alid
atio
n
lo
s
s
o
v
er
a
s
et
n
u
m
b
er
o
f
ep
o
ch
s
.
T
h
is
p
r
ev
en
ts
o
v
er
f
itti
n
g
b
y
r
e
tu
r
n
in
g
t
o
th
e
weig
h
ts
th
at
wer
e
s
ee
n
b
est d
u
r
in
g
tr
ain
in
g
.
2
.
4
.
G
ra
d
-
CAM
hea
t
m
a
p v
i
s
ua
liza
t
io
n
Gr
ad
-
C
AM
[
3
1
]
is
a
v
is
u
al
ex
p
lan
atio
n
m
eth
o
d
f
o
r
m
o
d
el
p
r
ed
ictio
n
s
th
at
h
ig
h
lig
h
ts
th
e
m
o
s
t
im
p
o
r
tan
t
a
r
ea
s
o
f
th
e
im
ag
e
f
o
r
th
e
p
r
ed
icted
class
,
p
r
o
v
id
in
g
in
s
ig
h
ts
in
to
wh
ich
im
ag
e
r
eg
io
n
s
th
e
m
o
d
e
l
d
ee
m
s
im
p
o
r
ta
n
t.
T
h
e
Gr
ad
-
C
AM
lo
ca
lizatio
n
m
ap
,
wh
i
ch
h
ig
h
lig
h
ts
th
e
r
e
g
io
n
’
s
m
o
s
t
r
elev
an
t
to
th
e
p
r
ed
icted
class
,
is
ex
p
r
ess
ed
in
(
1
0
)
.
−
=
∆
(
∑
)
(
1
0
)
W
h
er
e
ar
e
th
e
f
ea
tu
r
e
m
ap
s
o
f
th
e
last
co
n
v
o
lu
tio
n
al
lay
e
r
an
d
α
c
ar
e
t
h
e
weig
h
ts
wh
i
ch
r
ep
r
esen
t
th
e
im
p
o
r
tan
ce
o
f
ea
ch
f
ea
tu
r
e
m
ap
.
T
h
is
m
eth
o
d
is
a
h
o
lis
tic
ap
p
r
o
ac
h
f
o
r
b
r
ain
tu
m
o
r
class
if
icatio
n
,
wh
ich
in
clu
d
es
f
ea
tu
r
e
e
x
tr
ac
tio
n
,
atten
tio
n
m
ec
h
an
is
m
s
,
an
d
r
eg
u
lar
is
atio
n
to
im
p
r
o
v
e
th
e
m
o
d
el
’
s
ac
cu
r
ac
y
,
in
ter
p
r
etab
ilit
y
,
a
n
d
r
o
b
u
s
tn
e
s
s
.
E
ac
h
m
o
d
el
is
tr
ain
e
d
f
o
r
th
e
b
r
ain
tu
m
o
r
class
ificati
o
n
task
b
y
ad
d
in
g
m
u
lti
-
h
ea
d
atten
tio
n
m
ec
h
an
is
m
s
an
d
r
eg
u
lar
is
atio
n
lay
er
s
o
n
p
r
e
-
tr
ain
e
d
I
m
ag
eNe
t
weig
h
ts
.
T
h
e
m
o
d
els
ar
e
tr
ain
ed
to
class
if
y
f
o
u
r
t
y
p
es
o
f
b
r
ain
tu
m
o
r
s
:
g
lio
m
a,
m
en
in
g
io
m
a,
n
o
tu
m
o
r
,
an
d
p
i
tu
itar
y
.
Gr
ad
-
C
AM
o
u
tp
u
ts
v
is
u
al
ex
p
lan
atio
n
s
o
f
m
o
d
el
d
ec
is
io
n
s
b
y
v
is
u
a
lizin
g
p
ar
ts
o
f
th
e
im
a
g
e
m
o
s
t
r
elev
an
t
to
t
h
e
p
r
ed
icted
class
.
T
h
e
co
r
r
esp
o
n
d
in
g
h
ea
tm
a
p
o
b
tain
ed
f
r
o
m
t
h
ese
weig
h
ted
f
ea
tu
r
e
m
ap
s
is
ex
p
r
ess
ed
in
(
1
1
)
.
=
(
∑
)
(
1
1
)
W
h
er
e
ar
e
th
e
weig
h
ts
o
b
ta
in
ed
b
y
a
v
er
ag
i
n
g
t
h
e
g
r
ad
ie
n
ts
an
d
ar
e
th
e
f
ea
tu
r
e
m
ap
s
o
f
t
h
e
last
co
n
v
o
l
u
tio
n
al
lay
er
.
T
h
is
m
eth
o
d
o
lo
g
y
p
r
o
v
id
es
a
co
m
p
lete
d
ee
p
lear
n
in
g
m
eth
o
d
f
o
r
b
r
ain
tu
m
o
r
class
if
icatio
n
.
E
ac
h
p
h
ase
is
f
o
cu
s
ed
o
n
im
p
r
o
v
in
g
th
e
ac
cu
r
ac
y
,
av
o
id
in
g
o
v
er
f
itti
n
g
,
an
d
u
n
d
er
s
tan
d
in
g
th
e
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
o
f
th
e
m
o
d
el.
2
.
5
.
VG
G
1
6
m
o
del
VGG1
6
an
d
VGG1
9
[
3
2
]
ar
e
p
r
e
-
tr
ain
ed
m
o
d
els
th
at
h
a
v
e
s
tack
ed
co
n
v
o
lu
tio
n
al
lay
er
s
,
wh
ich
m
ak
es
th
em
m
o
r
e
ef
f
icien
t
in
ca
p
tu
r
in
g
lo
ca
l
an
d
d
etailed
f
ea
tu
r
es
in
im
ag
es.
T
h
ey
ar
e
r
ef
in
ed
with
a
m
u
lti
-
h
ea
d
atten
tio
n
la
y
er
,
r
eg
u
lar
izatio
n
b
y
b
atch
n
o
r
m
aliza
tio
n
an
d
Gau
s
s
ian
n
o
is
e
to
im
p
r
o
v
e
m
o
d
e
l
r
o
b
u
s
tn
ess
.
Den
s
e
lay
er
s
wit
h
5
1
2
u
n
its
(
R
eL
U
ac
tiv
atio
n
)
allo
w
lear
n
in
g
co
m
p
lex
p
atter
n
s
an
d
d
r
o
p
o
u
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
6
4
6
-
3
6
5
9
3650
lay
er
s
p
r
ev
en
t
o
v
er
f
itti
n
g
.
T
h
e
s
p
ar
s
e
ca
teg
o
r
ical
cr
o
s
s
-
en
tr
o
p
y
lo
s
s
f
u
n
ctio
n
is
m
in
im
ized
u
s
in
g
th
e
Ad
am
o
p
tim
izer
,
wh
ich
m
ea
s
u
r
es th
e
d
if
f
er
en
ce
b
etwe
en
th
e
tr
u
e
la
b
els an
d
th
e
p
r
ed
icted
p
r
o
b
a
b
ilit
ies.
I
n
th
is
wo
r
k
,
th
e
VGG1
6
m
o
d
el
with
th
e
p
r
etr
ain
ed
weig
h
ts
o
n
I
m
ag
e
Net
is
u
s
ed
a
s
a
f
ea
tu
r
e
ex
tr
ac
to
r
,
th
e
lay
er
s
ar
e
f
r
o
ze
n
d
u
r
in
g
tr
ai
n
in
g
.
VGG1
9
m
o
d
el
lik
e
VGG1
6
h
as m
o
r
e
lay
er
s
to
in
cr
ea
s
e
th
e
d
ep
th
an
d
ab
ilit
y
to
ex
tr
ac
t
f
ea
tu
r
es.
B
atch
n
o
r
m
aliza
tio
n
an
d
d
r
o
p
o
u
t
ar
e
well
-
estab
lis
h
ed
r
eg
u
lar
izatio
n
tech
n
i
q
u
es
f
o
r
s
tab
ilizin
g
tr
ain
in
g
an
d
r
ed
u
cin
g
o
v
er
f
itti
n
g
in
d
ee
p
n
etwo
r
k
s
[
3
3
]
,
[
3
4
]
.
T
h
e
o
u
tp
u
t
c
o
m
p
u
te
d
b
y
t
h
e
d
en
s
e
lay
er
with
in
th
is
ar
ch
itectu
r
e
is
ex
p
r
ess
ed
i
n
(
1
2
)
.
=
(
+
)
(
1
2
)
T
h
e
p
r
e
-
tr
ain
ed
lay
er
s
ar
e
f
r
o
ze
n
d
u
r
in
g
tr
ain
in
g
,
s
o
th
at
th
e
m
o
d
el
ca
n
f
o
c
u
s
o
n
th
e
n
ew
task
s
an
d
it
ap
p
lies
g
lo
b
al
av
er
a
g
e
p
o
o
lin
g
,
d
en
s
e
lay
er
s
,
Gau
s
s
ian
n
o
is
e
,
an
d
d
r
o
p
o
u
t t
o
a
f
in
al
s
o
f
tm
a
x
class
if
icatio
n
lay
er
.
2
.
6
.
M
o
bil
eNe
t
m
o
de
l
T
h
is
s
tu
d
y
u
s
e
Mo
b
ileNet
[
3
5
]
with
p
r
e
-
tr
ai
n
ed
weig
h
ts
an
d
n
o
n
-
t
r
ain
ab
le
f
ea
tu
r
es
to
b
u
ild
th
e
m
o
d
el.
T
h
en
,
th
e
o
u
tp
u
t
is
p
a
s
s
ed
to
a
m
u
lti
-
h
ea
d
atten
tio
n
lay
er
,
f
o
llo
wed
b
y
g
l
o
b
al
a
v
er
ag
e
p
o
o
lin
g
,
d
en
s
e
lay
er
,
b
atch
n
o
r
m
aliza
tio
n
,
Gau
s
s
ian
n
o
is
e,
d
r
o
p
o
u
t
,
an
d
s
o
f
tm
ax
lay
er
to
o
u
tp
u
t
th
e
p
r
o
b
ab
ilit
ies.
T
h
e
lo
s
s
f
u
n
ctio
n
u
s
ed
to
tr
ain
th
e
Mo
b
ileNet
-
b
ased
m
o
d
el
is
ex
p
r
ess
ed
in
(
1
3
)
.
=
−
∑
.
(
̂
)
=
1
(
1
3
)
T
h
is
s
tu
d
y
u
s
es
th
e
Ad
am
o
p
tim
izer
f
o
r
s
tab
le
lea
r
n
in
g
an
d
atten
tio
n
m
ec
h
an
is
m
s
to
f
o
cu
s
o
n
tu
m
o
r
-
r
ele
v
an
t
r
eg
i
o
n
s
o
f
th
e
im
ag
e.
B
atch
n
o
r
m
aliza
tio
n
r
ed
u
ce
s
in
ter
n
al
co
v
ar
iate
s
h
i
f
t,
wh
ich
s
tab
ilizes
th
e
lear
n
in
g
p
r
o
ce
s
s
.
T
h
e
Gau
s
s
ian
n
o
is
e
lay
er
s
ad
d
r
an
d
o
m
n
o
is
e
to
th
e
d
ata
wh
ich
h
elp
s
th
e
n
etwo
r
k
to
g
en
er
alize
.
R
eg
u
la
r
izatio
n
is
a
tech
n
iq
u
e
o
f
r
a
n
d
o
m
ly
p
ick
in
g
n
eu
r
o
n
s
d
u
r
in
g
th
e
tr
ai
n
in
g
f
o
r
o
v
e
r
f
itti
n
g
r
ed
u
ctio
n
.
T
h
e
c
o
m
b
in
atio
n
o
f
b
atch
n
o
r
m
aliza
tio
n
,
Gau
s
s
ian
n
o
is
e,
an
d
d
r
o
p
o
u
t
im
p
r
o
v
es
g
en
er
aliza
tio
n
a
n
d
p
r
ev
en
ts
o
v
e
r
f
itti
n
g
.
T
h
is
en
s
u
r
es
r
o
b
u
s
tn
ess
an
d
p
r
e
v
en
ts
o
v
er
f
itti
n
g
b
y
ig
n
o
r
in
g
n
eu
r
o
n
s
d
u
r
in
g
tr
ain
in
g
T
h
e
m
o
d
el
p
e
r
f
o
r
m
an
ce
f
o
r
d
i
f
f
er
en
t
t
u
m
o
r
class
es
is
ev
alu
ated
u
s
in
g
im
p
o
r
ta
n
t
class
if
icatio
n
m
etr
ics
s
u
ch
as
p
r
ec
is
io
n
,
r
ec
all,
F1
-
s
co
r
e,
an
d
ac
cu
r
a
cy
.
T
h
ese
m
etr
ics
a
r
e
ca
lcu
lated
b
ased
o
n
th
e
an
aly
s
is
o
f
tr
u
e
p
o
s
itiv
es,
f
alse
p
o
s
itiv
es,
tr
u
e
n
eg
ativ
es
,
an
d
f
alse
n
eg
ativ
es
f
o
r
ea
c
h
class
.
Pre
cis
io
n
is
th
e
p
r
o
p
o
r
tio
n
o
f
c
o
r
r
ec
tly
p
r
ed
icted
p
o
s
itiv
e
ca
s
es,
r
ec
all
is
th
e
ef
f
ec
tiv
en
ess
o
f
th
e
m
o
d
el
in
i
d
en
tify
in
g
p
o
s
itiv
e
ca
s
es,
F1
-
s
co
r
e
is
a
b
alan
ce
d
ass
ess
m
en
t,
an
d
ac
cu
r
ac
y
is
th
e
o
v
e
r
all
co
r
r
ec
t
n
ess
o
f
th
e
m
o
d
el.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
Fig
u
r
e
1
was
cr
ea
ted
t
o
v
is
u
a
lize
th
e
class
d
is
tr
ib
u
tio
n
,
s
h
o
win
g
th
e
im
b
alan
ce
i
n
th
e
d
at
aset.
T
h
is
v
is
u
aliza
tio
n
r
ev
ea
led
th
e
n
e
ed
o
f
b
ala
n
cin
g
tech
n
iq
u
es
as
th
e
class
es
wer
e
im
b
alan
ce
d
.
As
s
h
o
wn
in
Fig
u
r
e
1
,
th
e
n
o
-
tu
m
o
r
class
i
s
th
e
lar
g
est
g
r
o
u
p
(
6
,
3
8
0
im
a
g
es),
f
o
llo
wed
b
y
p
itu
itar
y
(
5
,
8
2
8
)
,
m
e
n
in
g
io
m
a
(
5
,
3
5
6
)
,
an
d
g
lio
m
a
(
5
,
2
8
4
)
,
a
d
if
f
er
e
n
ce
o
f
ab
o
u
t
1
,
1
0
0
im
a
g
es
b
etwe
en
th
e
lar
g
est
an
d
s
m
allest
class
e
s
th
at
m
o
tiv
ated
th
e
o
v
er
s
am
p
lin
g
s
tr
ateg
y
d
escr
ib
e
d
in
s
u
b
s
ec
tio
n
2
.
2
.
5
.
Fig
u
r
e
1
.
Dis
tr
ib
u
tio
n
o
f
class
lab
els
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
o
mp
a
r
a
tive
ev
a
lu
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
mo
d
els a
n
d
Gr
a
d
-
C
A
M in
terp
r
eta
b
ilit
y
…
(
Md
.
F
ir
o
z
Ha
s
a
n
)
3651
T
h
e
d
ataset
in
Fig
u
r
e
2
was
s
t
r
u
ctu
r
ed
in
t
o
a
Data
Fra
m
e
wit
h
ea
ch
r
o
w
as
an
im
ag
e
p
ath
an
d
lab
el.
L
ab
els
wer
e
en
co
d
ed
n
u
m
er
ic
ally
to
b
e
co
m
p
atib
le
with
m
o
d
el
tr
ain
in
g
.
R
an
d
o
m
o
v
er
s
a
m
p
lin
g
was
u
s
ed
to
tack
le
th
e
class
im
b
ala
n
ce
p
r
o
b
lem
.
Af
ter
o
v
er
s
am
p
lin
g
,
ea
ch
class
h
ad
th
e
a
m
o
u
n
t
o
f
6
,
3
8
0
im
ag
es.
T
h
e
s
tu
d
y
h
as
ex
am
in
e
d
C
NN
f
o
r
class
if
y
in
g
b
r
ain
MRI
im
ag
es
in
to
4
ca
teg
o
r
ies:
g
l
io
m
a,
m
e
n
in
g
io
m
a
,
n
o
tu
m
o
r
,
a
n
d
p
itu
itar
y
.
T
r
an
s
f
er
lear
n
in
g
was
ap
p
lied
b
y
u
s
in
g
th
e
p
r
e
-
tr
ain
e
d
C
NN
m
o
d
els,
wh
ich
h
av
e
alr
ea
d
y
lear
n
e
d
th
e
p
atter
n
s
f
r
o
m
th
e
la
r
g
e
-
s
ca
le
d
atasets
.
T
h
e
m
o
d
els
VGG1
6
,
V
GG1
9
,
Mo
b
ileNet,
Xce
p
tio
n
,
an
d
I
n
ce
p
tio
n
V
3
wer
e
test
ed
.
T
h
e
m
o
d
els
wer
e
o
p
tim
ized
with
Ad
am
o
p
ti
m
izer
an
d
a
s
p
ar
s
e
ca
teg
o
r
ical
cr
o
s
s
-
en
tr
o
p
y
lo
s
s
f
u
n
ctio
n
f
o
r
m
u
lti
-
class
class
if
icatio
n
.
Fig
u
r
e
2
.
Vis
u
al
r
ep
r
esen
tatio
n
o
f
im
a
g
es
o
f
d
i
f
f
er
en
t sizes
T
h
e
o
b
jectiv
e
is
to
ev
al
u
ate
th
e
p
er
f
o
r
m
an
ce
o
f
v
ar
io
u
s
C
NN
ar
ch
itectu
r
es
in
th
e
class
if
icatio
n
o
f
b
r
ain
tu
m
o
r
s
.
T
r
a
n
s
f
er
lear
n
in
g
was
u
s
ed
to
tak
e
ad
v
an
tag
e
o
f
p
r
e
-
tr
ain
ed
C
NN
m
o
d
els
to
ac
h
iev
e
th
e
b
est
p
er
f
o
r
m
an
ce
with
a
s
m
all
d
ataset.
E
ac
h
m
o
d
el
was
r
esized
to
th
e
in
p
u
t
lay
er
s
an
d
f
in
e
-
tu
n
ed
with
th
e
in
p
u
t
s
ize
(
2
2
4
,
2
2
4
,
3
)
p
i
x
els.
T
o
en
h
an
ce
th
e
g
en
er
aliza
tio
n
o
f
th
e
m
o
d
el,
d
ata
a
u
g
m
e
n
tatio
n
o
p
e
r
atio
n
s
lik
e
h
o
r
izo
n
tal
f
lip
p
in
g
,
z
o
o
m
in
g
,
an
d
wid
th
/h
eig
h
t
s
h
if
tin
g
w
er
e
p
er
f
o
r
m
e
d
.
All
t
h
e
m
o
d
e
ls
wer
e
tr
ain
ed
f
o
r
5
-
1
0
e
p
o
ch
s
with
ea
r
l
y
s
to
p
p
i
n
g
to
p
r
ev
en
t
o
v
er
f
itti
n
g
.
T
ab
le
1
s
h
o
ws
th
e
b
r
ai
n
tu
m
o
r
class
if
icatio
n
with
m
u
ltip
le
m
etr
ics.
Acc
u
r
ac
y
,
p
r
ec
is
io
n
,
r
ec
all,
an
d
F1
-
s
co
r
e
ar
e
th
e
m
etr
ics
o
f
t
h
is
s
tu
d
y
.
T
h
e
b
est
m
o
d
el,
Mo
b
ileNet,
attain
ed
9
7
%
ac
c
u
r
ac
y
an
d
F1
-
s
co
r
e
wh
ich
is
ac
ce
p
tab
le
f
o
r
r
ea
l
-
t
im
e
ap
p
licatio
n
s
.
Fo
llo
win
g
clo
s
ely
,
VGG1
9
ac
h
iev
ed
an
ac
cu
r
ac
y
o
f
9
2
%,
s
h
o
win
g
its
ab
ilit
y
to
ca
p
tu
r
e
co
m
p
lex
MRI
p
atter
n
s
.
A
s
im
p
ler
v
er
s
io
n
VGG1
6
h
a
d
a
s
lig
h
tly
lo
wer
p
er
f
o
r
m
an
ce
with
ac
cu
r
ac
y
9
1
%.
Xce
p
tio
n
was
as
ac
c
u
r
ate
as
VGG1
6
,
b
u
t
s
lig
h
t
ly
less
p
r
ec
is
e
in
d
if
f
er
en
tiatin
g
s
o
m
e
tu
m
o
r
ty
p
es.
T
h
e
lo
west
p
er
f
o
r
m
an
ce
was
ac
h
iev
ed
b
y
I
n
ce
p
tio
n
V3
with
8
5
%
ac
cu
r
ac
y
,
p
o
s
s
ib
ly
d
u
e
to
th
e
c
o
m
p
le
x
ity
an
d
th
e
r
e
q
u
ir
em
en
t
o
f
lar
g
e
r
d
atasets
.
I
n
ce
p
tio
n
V3
s
h
o
wed
v
is
ib
le
m
is
class
if
icatio
n
am
o
n
g
s
im
ilar
tu
m
o
r
ty
p
es
lik
e
g
lio
m
a
an
d
m
en
in
g
io
m
a,
p
r
o
b
a
b
ly
d
u
e
to
th
e
v
is
u
al
s
im
ilar
ity
in
Fig
u
r
e
3
o
f
th
ese
tu
m
o
r
s
in
MRI
im
ag
es.
M
o
b
il
eNe
t
h
ad
t
h
e
h
i
g
h
est
ac
cu
r
ac
y
o
f
9
7
%
th
at
m
ea
n
s
th
at
alm
o
s
t a
ll o
f
th
e
tu
m
o
r
im
ag
es we
r
e
class
if
ied
co
r
r
ec
tly
.
VGG1
9
was a
ls
o
q
u
ite
g
o
o
d
with
an
ac
cu
r
ac
y
o
f
9
2
%.
I
n
ce
p
tio
n
V3
was
a
b
it
b
eh
in
d
with
an
ac
cu
r
ac
y
o
f
8
5
%
wh
ich
in
d
icate
s
th
at
it
m
ig
h
t
n
o
t
h
av
e
g
en
er
alize
d
well
f
r
o
m
th
e
d
ata
s
et.
T
ab
le
1
.
C
lass
if
icatio
n
r
esu
lt f
o
r
ap
p
lied
m
o
d
el
M
o
d
e
l
P
r
e
c
i
s
i
o
n
(
%)
R
e
c
a
l
l
(
%)
F1
-
sc
o
r
e
(
%)
A
c
c
u
r
a
c
y
(
%)
V
G
G
1
6
0
.
9
1
0
.
9
1
0
.
9
1
91
V
G
G
1
9
0
.
9
2
0
.
9
2
0
.
9
2
92
M
o
b
i
l
e
N
e
t
0
.
9
7
0
.
9
7
0
.
9
7
97
X
c
e
p
t
i
o
n
0
.
9
1
0
.
9
1
0
.
9
0
91
I
n
c
e
p
t
i
o
n
V
3
0
.
8
7
0
.
8
5
0
.
8
4
85
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
6
4
6
-
3
6
5
9
3652
Fig
u
r
e
3
.
Vis
u
aliza
tio
n
o
n
p
r
e
cisi
o
n
,
r
ec
all
,
F1
-
s
co
r
e
,
a
n
d
ac
cu
r
ac
y
T
h
e
Mo
b
ileNet
an
d
VGG1
9
m
o
d
els
ac
h
iev
ed
h
ig
h
p
r
ec
is
io
n
an
d
r
ec
all
in
p
r
ed
ictin
g
t
h
e
ty
p
es
o
f
tu
m
o
r
s
,
th
er
eb
y
r
e
d
u
cin
g
th
e
r
is
k
o
f
m
is
clas
s
if
icatio
n
an
d
en
s
u
r
in
g
r
eliab
le
id
en
tific
atio
n
o
f
tu
m
o
r
s
.
T
h
e
Mo
b
ileNet
an
d
VGG1
9
m
o
d
el
s
h
ad
h
ig
h
r
ec
all
v
alu
es,
s
h
o
win
g
th
at
th
ey
wer
e
ab
le
to
co
r
r
ec
tly
id
en
tify
m
o
s
t
ca
s
es
o
f
ea
ch
ty
p
e
o
f
tu
m
o
r
.
B
u
t
I
n
ce
p
tio
n
V3
h
ad
r
ec
all
is
s
u
es,
p
ar
ticu
lar
ly
in
d
is
cr
im
in
atin
g
m
en
in
g
io
m
a
b
ec
au
s
e
it
is
s
en
s
itiv
e
to
co
m
p
lex
p
atter
n
s
an
d
s
m
all
d
atasets
.
F1
-
s
co
r
e:
a
co
m
b
in
atio
n
o
f
p
r
ec
is
io
n
an
d
r
ec
all
to
k
n
o
w
h
o
w
well
a
m
o
d
el
ca
n
av
o
id
f
alse
p
o
s
itiv
es
an
d
f
al
s
e
n
eg
ativ
es.
Mo
b
ileNet
h
as
th
e
h
ig
h
est
F1
-
s
co
r
e
o
f
0
.
9
7
wh
ich
is
a
g
o
o
d
b
ala
n
ce
f
o
r
ca
p
tu
r
in
g
r
elev
a
n
t
ca
s
es
wh
ile
also
av
o
id
in
g
m
is
cla
s
s
if
icatio
n
.
VGG1
9
ca
m
e
n
ex
t
with
a
h
ig
h
F1
-
s
co
r
e
o
f
0
.
9
2
,
co
n
f
i
r
m
in
g
its
r
eliab
ilit
y
f
o
r
p
r
ac
tical
ap
p
licatio
n
s
.
I
n
ce
p
tio
n
V3
d
i
d
n
o
t
p
er
f
o
r
m
co
n
s
is
ten
tly
well
in
ter
m
s
o
f
p
r
ec
is
io
n
an
d
r
e
ca
ll,
esp
ec
ially
in
d
is
tin
g
u
is
h
i
n
g
v
is
u
ally
s
im
ilar
tu
m
o
r
class
es
lik
e
g
lio
m
a
an
d
m
en
in
g
io
m
a,
with
a
r
elativ
el
y
lo
wer
F1
-
s
co
r
e.
T
h
e
co
n
f
u
s
i
o
n
m
atr
ices
o
f
f
e
r
ed
m
o
r
e
in
s
ig
h
t in
t
o
th
e
m
is
class
i
f
icatio
n
p
atter
n
s
o
f
th
e
m
o
d
els.
Fig
u
r
e
4
s
h
o
ws
th
e
m
o
d
el
p
er
f
o
r
m
an
ce
e
v
alu
atio
n
f
o
r
VGG1
6
,
VGG1
9
,
Mo
b
ileNet,
Xce
p
tio
n
,
an
d
I
n
ce
p
tio
n
V3
;
wh
e
r
e
Fig
u
r
e
4
(
a)
s
h
o
ws
th
e
r
ec
eiv
er
o
p
er
at
in
g
ch
ar
ac
ter
is
tic
(
R
OC
)
cu
r
v
e
co
m
p
ar
is
o
n
a
n
d
Fig
u
r
e
4
(
b
)
s
h
o
ws
th
e
s
co
r
es
ac
r
o
s
s
ep
o
ch
s
f
o
r
d
if
f
e
r
en
t
m
o
d
els
.
T
h
e
R
OC
cu
r
v
es
ar
e
u
s
ed
to
ev
al
u
ate
th
e
p
er
f
o
r
m
an
ce
o
f
th
e
m
o
d
els
in
d
is
cr
im
in
atin
g
b
etwe
en
class
es.
T
h
e
R
OC
cu
r
v
es
o
f
VGG1
6
an
d
VGG1
9
a
r
e
m
o
d
er
ate
in
c
r
ea
s
in
g
,
in
d
icatin
g
a
g
o
o
d
s
en
s
itiv
ity
an
d
s
p
ec
if
icity
with
a
s
o
lid
m
o
d
el
p
er
f
o
r
m
an
ce
.
Mo
b
ileNet
h
as
a
s
teep
cu
r
v
e
n
ea
r
th
e
o
r
i
g
in
,
ac
h
ie
v
in
g
a
h
ig
h
tr
u
e
p
o
s
itiv
e
r
ate
with
s
m
all
in
cr
em
e
n
ts
o
f
f
alse
p
o
s
itiv
e
r
ate
an
d
h
as
th
e
h
ig
h
est
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC
)
o
f
all
t
h
e
m
o
d
els.
T
h
is
s
h
o
ws
th
at
Mo
b
ileNet
is
g
o
o
d
at
d
is
cr
im
in
atin
g
b
etwe
en
class
es
an
d
also
h
as
h
ig
h
r
ec
all
an
d
p
r
ec
is
io
n
with
m
in
im
al
tr
ad
e
-
o
f
f
o
n
f
alse
p
o
s
itiv
es.
Xce
p
tio
n
’
s
R
OC
c
u
r
v
e
is
m
o
d
er
ately
s
teep
wit
h
a
s
tead
y
in
cr
ea
s
e
b
u
t
a
litt
le
lo
wer
AUC
th
an
Mo
b
ileNet.
I
n
ce
p
tio
n
V3
’
s
c
u
r
v
e
also
in
cr
ea
s
es
s
tead
ily
b
u
t
less
s
teep
ly
at
th
e
b
e
g
i
n
n
in
g
,
in
d
icatin
g
a
m
o
d
er
ate
r
esu
lt,
wh
er
e
it
g
ets
a
lo
t
o
f
t
r
u
e
p
o
s
itiv
es
b
u
t
p
o
s
s
ib
ly
with
a
h
ig
h
er
f
alse
p
o
s
itiv
e
r
ate
ea
r
ly
o
n
.
T
h
e
R
OC
cu
r
v
e
an
d
AUC
o
f
Mo
b
ileNet
s
h
o
w
th
at
it
i
s
th
e
m
o
s
t
ca
p
ab
le
m
o
d
el
f
o
r
th
is
class
if
icatio
n
task
.
VGG1
6
,
VGG1
9
,
Xce
p
tio
n
,
an
d
I
n
ce
p
tio
n
V3
also
p
e
r
f
o
r
m
well
with
s
lig
h
t
d
if
f
er
en
ce
s
in
th
e
b
alan
ce
o
f
s
en
s
itiv
ity
an
d
s
p
ec
if
icity
.
A
UC
is
a
g
o
o
d
m
ea
s
u
r
e
t
o
co
m
p
ar
e
th
e
p
er
f
o
r
m
a
n
ce
o
f
m
o
d
els
in
class
if
icatio
n
p
r
o
b
lem
s
.
T
h
is
g
iv
es
a
b
alan
ce
d
v
iew
o
f
th
e
m
o
d
el'
s
ab
ilit
y
to
d
is
cr
im
in
ate,
wh
ich
is
u
s
e
f
u
l
if
y
o
u
wan
t
to
ch
o
o
s
e
a
m
o
d
el
th
at
d
o
es
well
at
all
th
r
esh
o
ld
s
,
an
d
t
h
u
s
b
e
r
o
b
u
s
t
an
d
r
eliab
le
in
th
e
r
ea
l
wo
r
ld
.
I
f
class
s
ep
ar
atio
n
is
n
ee
d
ed
,
a
m
o
d
el
with
a
b
etter
AU
C
ca
n
be
o
f
ten
th
e
b
est
o
p
tio
n
,
s
u
ch
as
Mo
b
ileNet.
R
OC
an
d
AUC r
em
ain
s
tan
d
ar
d
to
o
ls
f
o
r
ev
alu
atin
g
a
n
d
c
o
m
p
ar
i
n
g
cl
ass
if
ier
p
er
f
o
r
m
an
ce
ac
r
o
s
s
th
r
esh
o
ld
s
[
3
5
].
Fig
u
r
e
5
is
a
lin
e
ch
a
r
t
th
at
s
h
o
ws
th
e
tr
ain
in
g
an
d
v
alid
atio
n
lo
s
s
in
ea
ch
m
o
d
el
(
VGG1
6
,
VGG1
9
,
Mo
b
ileNet,
Xce
p
tio
n
,
an
d
I
n
c
ep
tio
n
V3
)
o
v
er
ep
o
ch
s
.
Mo
b
i
leNe
t
is
a
s
u
itab
le
m
o
d
el
f
o
r
task
s
n
ee
d
in
g
h
ig
h
class
if
icatio
n
ac
cu
r
ac
y
an
d
a
b
alan
ce
b
etwe
en
p
r
ec
is
io
n
an
d
r
ec
all.
I
t
also
h
as
th
e
least
tr
ain
in
g
tim
e
in
d
icatin
g
e
f
f
icien
cy
in
co
m
p
u
tatio
n
al
r
eso
u
r
ce
s
.
VGG1
6
an
d
VGG1
9
ar
e
co
n
s
is
ten
tly
im
p
r
o
v
in
g
th
e
p
r
ec
is
io
n
,
r
ec
all
an
d
F1
-
s
co
r
e,
with
m
o
d
er
ate
AUC v
alu
es a
n
d
lo
w
ten
d
en
cy
o
f
o
v
er
f
itti
n
g
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
o
mp
a
r
a
tive
ev
a
lu
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
mo
d
els a
n
d
Gr
a
d
-
C
A
M in
terp
r
eta
b
ilit
y
…
(
Md
.
F
ir
o
z
Ha
s
a
n
)
3653
(
a)
(
b
)
Fig
u
r
e
4
.
Mo
d
el
p
er
f
o
r
m
a
n
ce
ev
alu
atio
n
o
f
(
a)
R
OC
cu
r
v
e
c
o
m
p
ar
is
o
n
a
n
d
(
b
)
s
co
r
es a
cr
o
s
s
ep
o
ch
s
f
o
r
d
if
f
er
en
t m
o
d
els
Fig
u
r
e
5
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
lo
s
s
o
v
er
e
p
o
ch
s
f
o
r
d
i
f
f
er
e
n
t m
o
d
els
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
1
5
,
No
.
4
,
Au
g
u
s
t 2
0
2
6
:
3
6
4
6
-
3
6
5
9
3654
Xce
p
tio
n
d
o
es
well
b
u
t
s
ee
m
s
a
b
it
o
v
er
f
itted
.
I
n
ce
p
tio
n
V3
o
b
tain
s
r
ea
s
o
n
ab
le
s
co
r
es
f
o
r
all
m
etr
ics
in
Fig
u
r
e
6
b
u
t
it
h
as
t
h
e
lo
we
s
t
p
r
ec
is
io
n
an
d
r
ec
all
am
o
n
g
m
o
d
els
,
in
d
icatin
g
th
at
it
c
an
b
e
f
u
r
th
er
im
p
r
o
v
ed
b
y
tu
n
i
n
g
.
M
o
b
ileNet
is
ch
o
s
en
b
ec
au
s
e
o
f
its
ef
f
icien
c
y
a
n
d
h
ig
h
p
e
r
f
o
r
m
an
ce
.
Misclass
if
icatio
n
h
ap
p
en
s
wh
en
th
e
m
o
d
el
p
r
e
d
icts
th
e
tu
m
o
r
ca
teg
o
r
y
o
f
a
n
im
ag
e
wr
o
n
g
ly
as
g
lio
m
a
an
d
m
en
in
g
io
m
a
.
Fig
u
r
e
6
.
C
o
n
f
u
s
io
n
m
atr
i
x
f
o
r
d
if
f
er
e
n
t m
o
d
els
I
n
T
a
b
le
2
,
ca
s
es
wh
er
e
th
e
m
o
d
els
f
ailed
to
co
r
r
ec
tly
cla
s
s
if
y
ar
e
lis
ted
.
Fo
r
ea
ch
en
t
r
y
,
th
e
im
ag
e
I
D,
th
e
tr
u
e
tu
m
o
r
ty
p
e,
th
e
i
n
co
r
r
ec
t
p
r
e
d
ictio
n
,
th
e
m
o
d
e
l
u
s
ed
,
an
d
th
e
p
o
s
s
ib
le
r
ea
s
o
n
f
o
r
th
e
er
r
o
r
ar
e
p
r
o
v
id
e
d
.
All
th
ese
r
ea
s
o
n
s
ar
e
v
is
u
al
ju
d
g
em
e
n
ts
.
T
y
p
ical
p
r
o
b
lem
s
ar
e
th
at
th
e
t
u
m
o
r
s
h
ap
es
o
r
s
izes
ar
e
s
im
ilar
an
d
th
at
th
e
im
ag
e
h
as
ar
tef
ac
ts
th
at
m
ay
m
i
s
lead
th
e
m
o
d
el.
T
o
b
etter
u
n
d
er
s
tan
d
th
ese
m
is
class
if
icatio
n
s
,
let’
s
tak
e
a
lo
o
k
at
s
o
m
e
o
f
th
ese
i
m
ag
es.
Her
e
is
an
ex
am
p
le
o
f
a
f
ew
s
elec
ted
m
is
class
if
ied
im
ag
es
with
tr
u
e
an
d
p
r
ed
icted
lab
els
s
h
o
wn
u
n
d
er
ea
ch
im
ag
e
to
h
elp
v
i
s
u
alize
th
e
ar
ea
s
o
f
co
n
f
u
s
io
n
f
o
r
ea
ch
m
o
d
el.
T
ab
le
2.
Misclass
ified
im
ag
e
s
u
m
m
ar
y
ta
b
le
I
mag
e
I
D
Tr
u
e
l
a
b
e
l
P
r
e
d
i
c
t
e
d
l
a
b
e
l
M
o
d
e
l
3
0
0
0
.
j
p
g
G
l
i
o
ma
M
e
n
i
n
g
i
o
ma
I
n
c
e
p
t
i
o
n
V
3
1
9
7
9
.
j
p
g
M
e
n
i
n
g
i
o
ma
G
l
i
o
ma
V
G
G
1
6
4
4
2
1
.
j
p
g
N
o
t
u
m
o
r
P
i
t
u
i
t
a
r
y
M
o
b
i
l
e
N
e
t
5
2
5
4
.
j
p
g
P
i
t
u
i
t
a
r
y
G
l
i
o
ma
V
G
G
1
9
2
5
9
9
.
j
p
g
G
l
i
o
ma
M
e
n
i
n
g
i
o
ma
M
o
b
i
l
e
N
e
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
C
o
mp
a
r
a
tive
ev
a
lu
a
tio
n
o
f tra
n
s
fer lea
r
n
in
g
mo
d
els a
n
d
Gr
a
d
-
C
A
M in
terp
r
eta
b
ilit
y
…
(
Md
.
F
ir
o
z
Ha
s
a
n
)
3655
Fig
u
r
e
7
p
r
esen
ts
th
ese
m
is
class
if
ied
MRI
im
ag
es
with
o
u
t
Gr
ad
-
C
AM
o
v
er
lay
s
,
g
iv
in
g
a
b
aselin
e
v
iew
o
f
th
e
r
aw
in
p
u
ts
b
ef
o
r
e
th
e
m
o
d
el'
s
atten
tio
n
r
eg
io
n
s
ar
e
v
is
u
alize
d
in
Fig
u
r
e
8
.
T
h
e
tr
u
e
an
d
p
r
ed
icted
lab
els
f
o
r
ea
ch
m
is
class
i
fied
i
m
ag
e
ar
e
s
h
o
wn
,
with
th
e
s
p
ec
ific
r
eg
io
n
s
wh
er
e
th
e
m
o
d
el
’
s
f
o
cu
s
m
ig
h
t
h
av
e
led
to
an
in
c
o
r
r
ec
t
p
r
ed
ictio
n
.
T
h
e
m
ain
s
o
u
r
ce
o
f
m
is
class
if
icatio
n
in
th
is
s
tu
d
y
is
th
e
v
is
u
al
s
im
ilar
ity
o
f
tu
m
o
r
ty
p
es
an
d
am
b
ig
u
o
u
s
i
m
ag
e
f
ea
tu
r
es.
Ov
er
lap
p
i
n
g
t
ex
tu
r
es
an
d
ir
r
eg
u
lar
b
o
u
n
d
ar
ies
o
f
g
lio
m
a
a
n
d
m
en
in
g
io
m
a
o
f
ten
c
o
n
f
u
s
ed
t
h
em
.
Fre
q
u
en
t
f
alse
p
r
ed
ictio
n
s
as
g
lio
m
a
wer
e
m
ad
e
f
o
r
m
en
in
g
io
m
a
im
ag
es
th
at
h
ad
b
l
u
r
o
r
in
d
is
tin
ctn
es
s
.
I
n
s
o
m
e
ca
s
es,
n
o
n
-
tu
m
o
r
o
u
s
im
ag
es
with
f
ain
t
ar
te
f
ac
ts
wer
e
in
co
r
r
ec
tly
id
en
tifie
d
as
p
itu
itar
y
tu
m
o
r
s
,
s
u
g
g
esti
n
g
th
e
n
ee
d
f
o
r
b
et
ter
f
ea
tu
r
e
d
is
cr
im
in
atio
n
.
S
m
all
an
d
lo
ca
lized
p
itu
itar
y
tu
m
o
r
s
wer
e
also
m
i
s
class
if
ied
as
g
lio
m
a,
illu
s
tr
atin
g
lim
itatio
n
s
in
s
p
atial
r
ec
o
g
n
itio
n
.
T
h
e
r
esu
lts
in
d
icate
th
at
f
u
r
th
er
p
r
ep
r
o
c
ess
in
g
,
f
ea
tu
r
e
lo
ca
lizatio
n
,
an
d
m
o
d
el
tu
n
in
g
ar
e
r
e
q
u
ir
ed
to
en
h
an
ce
th
e
class
if
icatio
n
ac
cu
r
ac
y
.
Fig
u
r
e
7
.
I
m
ag
e
with
o
u
t G
r
ad
-
C
AM
Fig
u
r
e
8
.
I
m
ag
e
af
te
r
ap
p
l
y
in
g
Gr
ad
-
C
AM
Misclas
s
if
ied
im
ag
es
d
ep
ict
lim
itatio
n
s
o
f
a
m
o
d
el.
Fo
r
ex
am
p
le,
MRI
im
ag
es
o
f
g
l
io
m
a
an
d
m
en
in
g
io
m
a
ca
n
h
a
v
e
s
im
ilar
v
is
u
al
tr
aits
,
wh
ich
ca
u
s
ed
I
n
ce
p
tio
n
V3
an
d
VGG1
6
to
m
i
s
class
if
y
g
lio
m
a
as
m
en
in
g
io
m
a
.
T
h
ese
er
r
o
r
s
m
ay
b
e
d
u
e
t
o
co
n
v
o
lu
tio
n
al
lay
er
s
tr
ea
tin
g
s
h
ar
ed
tex
tu
r
es
a
n
d
ed
g
es
as
s
im
ilar
f
ea
tu
r
es.
T
h
e
Mo
b
ileNet
ar
ch
itectu
r
e
was
n
o
t
ab
le
to
p
er
f
o
r
m
well
in
th
e
p
r
esen
ce
o
f
s
lig
h
t
ar
tef
ac
ts
an
d
s
h
o
u
ld
b
e
ad
ju
s
ted
f
o
r
th
e
s
en
s
itiv
ity
to
tex
tu
r
e
a
n
d
s
h
ap
e,
p
er
h
ap
s
b
y
ad
ju
s
tin
g
th
e
t
r
ain
in
g
p
r
o
ce
d
u
r
e
o
r
th
e
atten
tio
n
m
ec
h
an
is
m
s
.
Gr
ad
-
C
AM
i
s
a
m
eth
o
d
f
o
r
p
r
o
v
id
i
n
g
v
is
u
al
ex
p
lan
atio
n
s
o
f
d
ec
i
s
io
n
s
m
ad
e
b
y
d
ee
p
lear
n
in
g
m
o
d
els
b
y
u
tili
zin
g
t
h
e
r
eg
i
o
n
s
o
f
an
im
ag
e
th
at
a
r
e
m
o
s
t
r
elev
a
n
t
to
th
e
m
o
d
el’
s
class
if
icatio
n
.
An
ex
p
er
im
en
t
o
n
b
r
ain
tu
m
o
r
cla
s
s
if
icatio
n
with
Gr
ad
-
C
AM
v
i
s
u
aliza
tio
n
s
p
r
o
v
id
es
a
h
ea
tm
ap
o
v
er
la
y
o
n
ea
c
h
im
ag
e,
in
d
icatin
g
r
e
g
io
n
s
r
el
ev
an
t
to
class
if
y
in
g
th
e
tu
m
o
r
ty
p
es.
T
h
is
in
ter
p
r
eta
b
ilit
y
is
im
p
o
r
tan
t
f
o
r
m
ed
ical
ap
p
licatio
n
s
,
as
it
g
u
ar
an
tees
th
at
th
e
m
o
d
el
atten
d
s
to
th
e
r
elev
an
t
a
n
ato
m
ical
s
tr
u
ctu
r
es
an
d
d
o
es
n
o
t
m
a
k
e
m
is
class
if
icatio
n
s
.
Gr
ad
-
C
AM
h
ea
tm
ap
s
ca
n
also
u
n
c
o
v
er
p
atter
n
s
i
n
th
e
m
o
d
e
l’
s
d
ec
is
io
n
p
r
o
ce
s
s
th
at
co
u
ld
s
u
g
g
est
ch
an
g
es
in
th
e
m
o
d
el’
s
s
en
s
itiv
ity
o
r
p
r
ep
r
o
ce
s
s
in
g
to
b
etter
d
etec
t
tr
u
e
tu
m
o
r
f
ea
tu
r
es.
T
h
is
in
ter
p
r
etativ
e
to
o
l
allo
ws
d
ev
elo
p
er
s
to
id
e
n
tify
an
d
f
i
x
s
p
ec
if
ic
atten
tio
n
er
r
o
r
s
,
th
e
r
eb
y
in
cr
ea
s
in
g
t
h
e
r
eliab
ilit
y
o
f
th
e
m
o
d
el.
W
ith
th
e
aid
o
f
XAI
m
eth
o
d
s
s
u
ch
as
Gr
ad
-
C
AM
in
Fig
u
r
e
8
,
th
e
tr
an
s
p
ar
en
cy
a
n
d
tr
u
s
two
r
th
in
ess
o
f
d
ee
p
lear
n
in
g
m
o
d
els
ca
n
b
e
im
p
r
o
v
e
d
,
esp
ec
ially
in
h
ea
lth
ca
r
e.
Gr
ad
-
C
AM
p
r
o
v
id
es
in
s
ig
h
t
in
to
th
e
d
ec
is
io
n
-
m
ak
in
g
p
r
o
ce
s
s
o
f
th
e
m
o
d
el
an
d
h
elp
s
in
v
alid
atin
g
t
h
e
clin
ic
al
ac
cu
r
ac
y
o
f
th
e
m
o
d
el
an
d
id
e
n
tify
in
g
er
r
o
r
s
th
at
m
ay
a
r
is
e
d
u
e
to
in
h
er
e
n
t
lim
itatio
n
s
o
r
c
h
allen
g
es
o
f
th
e
d
ataset.
Su
ch
in
ter
p
r
etab
ilit
y
r
e
d
u
ce
s
r
is
k
s
in
m
e
d
ical
d
iag
n
o
s
is
,
wh
er
e
m
is
class
if
icatio
n
o
f
tu
m
o
r
s
m
ay
lead
t
o
in
ap
p
r
o
p
r
iate
tr
ea
tm
en
ts
.
T
h
is
v
is
u
al
o
v
er
lap
b
etwe
en
g
li
o
m
a
an
d
m
en
in
g
io
m
a
ca
s
es
h
as
b
ee
n
s
im
ilar
ly
r
ep
o
r
ted
i
n
o
th
er
m
u
lti
-
class
b
r
ain
tu
m
o
r
class
if
icatio
n
s
tu
d
ies
[
3
6
]
,
wh
ile
s
y
s
tem
atic
r
ev
iews
o
f
XAI
in
m
ed
ical
im
ag
in
g
co
n
f
i
r
m
t
h
at
Gr
ad
-
C
AM
-
b
ased
in
ter
p
r
etab
ilit
y
is
am
o
n
g
t
h
e
m
o
s
t
wid
ely
ad
o
p
ted
ap
p
r
o
ac
h
es f
o
r
b
u
ild
in
g
clin
ician
tr
u
s
t in
d
ee
p
lear
n
in
g
d
iag
n
o
s
tic
to
o
ls
[
3
7
]
.
Gr
ad
-
C
AM
v
is
u
aliza
tio
n
s
h
ig
h
lig
h
t
th
e
d
if
f
e
r
en
ce
s
in
atte
n
tio
n
,
g
iv
in
g
ac
tio
n
a
b
le
s
u
g
g
esti
o
n
s
to
en
h
an
ce
th
e
m
o
d
el'
s
ac
cu
r
ac
y
an
d
r
eliab
ilit
y
.
No
w,
with
th
e
Gr
ad
-
C
AM
v
is
u
aliza
tio
n
s
in
teg
r
ated
in
to
th
e
m
o
d
el
ev
al
u
atio
n
p
r
o
ce
s
s
,
d
ev
elo
p
er
s
ca
n
s
ee
th
e
m
o
d
el
’
s
s
p
atial
atten
tio
n
d
ir
ec
tly
an
d
a
n
iter
ativ
e
f
ee
d
b
ac
k
lo
o
p
ar
is
es.
I
n
co
n
clu
s
io
n
,
Gr
ad
-
C
AM
an
d
XAI
tech
n
iq
u
es
ar
e
im
p
o
r
tan
t
to
en
h
a
n
ce
d
e
ep
lear
n
in
g
m
o
d
els
f
o
r
m
e
d
ical
ap
p
licatio
n
s
.
Evaluation Warning : The document was created with Spire.PDF for Python.