I
AE
S In
t
er
na
t
io
na
l J
o
urna
l o
f
Art
if
icia
l In
t
ellig
ence
(
I
J
-
AI
)
Vo
l.
15
,
No
.
4
,
A
u
g
u
s
t
20
26
,
p
p
.
3
7
9
2
~
3
8
0
4
I
SS
N:
2
2
5
2
-
8
9
3
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijai.v
15
.i
4
.
p
p
3
7
9
2
-
3
8
0
4
3792
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
a
i
.
ia
esco
r
e.
co
m
Deep
lea
rni
ng
-
ba
sed pro
g
no
stic m
o
deling
of brain
t
umo
rs
L
a
ia
li Alm
a
za
y
deh
1
,
Ara
r
Al
T
a
wil
2
1
C
o
l
l
e
g
e
o
f
I
n
f
o
r
ma
t
i
o
n
Te
c
h
n
o
l
o
g
y
,
A
l
-
H
u
sse
i
n
B
i
n
Ta
l
a
l
U
n
i
v
e
r
si
t
y
,
M
a
’
a
n
,
Jo
r
d
a
n
2
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
,
F
a
c
u
l
t
y
o
f
I
n
f
o
r
ma
t
i
o
n
Te
c
h
n
o
l
o
g
y
,
A
p
p
l
i
e
d
S
c
i
e
n
c
e
P
r
i
v
a
t
e
U
n
i
v
e
r
si
t
y
,
A
m
man
,
J
o
r
d
a
n
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Feb
8
,
2
0
2
6
R
ev
is
ed
J
u
l
15
,
2
0
2
6
Acc
ep
ted
J
u
l
21
,
2
0
2
6
He
a
d
li
n
e
a
c
c
u
ra
c
y
is
n
o
t
e
n
o
u
g
h
t
o
m
o
v
e
a
d
e
e
p
-
lea
rn
i
n
g
b
r
a
in
tu
m
o
r
m
a
g
n
e
ti
c
re
so
n
a
n
c
e
ima
g
i
n
g
(
M
RI
)
c
las
sifier
in
to
th
e
c
li
n
ic.
A
m
o
d
e
l
a
lso
n
e
e
d
s
re
p
e
a
ted
-
sp
li
t
v
a
li
d
a
ti
o
n
,
tr
u
stwo
rth
y
p
r
o
b
a
b
il
it
ies
,
a
sta
ti
sti
c
a
l
sa
n
it
y
c
h
e
c
k
,
a
n
d
a
sm
a
ll
e
n
o
u
g
h
fo
o
t
p
r
in
t
f
o
r
h
o
sp
i
tal
h
a
rd
wa
re
.
T
h
is
st
u
d
y
c
o
v
e
r
a
ll
fo
u
r
i
n
o
n
e
fra
m
e
wo
rk
.
F
i
v
e
c
o
n
v
o
l
u
ti
o
n
a
l
n
e
u
ra
l
n
e
two
r
k
(
CNN
)
b
a
c
k
b
o
n
e
s
(
v
isu
a
l
g
e
o
m
e
try
g
r
o
u
p
(
VGG
)
-
1
6
,
re
sid
u
a
l
n
e
two
rk
(
Re
sN
e
t
)
-
5
0
V2
,
M
o
b
il
e
Ne
tV2
,
Eff
icie
n
t
Ne
tB0
,
a
n
d
d
e
n
se
c
o
n
v
o
lu
t
io
n
a
l
n
e
two
rk
(
De
n
se
Ne
t
)
-
1
2
1
)
a
re
train
e
d
o
n
4
,
6
0
0
p
u
b
li
c
b
ra
i
n
M
RI
sc
a
n
s
(2
,
5
1
3
t
u
m
o
r,
2
,
0
8
7
h
e
a
lt
h
y
)
u
n
d
e
r
a
sh
a
re
d
two
-
p
h
a
se
tran
sfe
r
-
lea
rn
in
g
re
c
ip
e
.
Th
e
u
su
a
l
sin
g
le
s
p
li
t
is
re
p
lac
e
d
b
y
stra
ti
fie
d
5
-
f
o
l
d
c
ro
ss
-
v
a
li
d
a
ti
o
n
(CV)
,
w
it
h
p
a
ired
M
c
Ne
m
a
r
a
n
d
De
Lo
n
g
tes
ts
a
n
d
1
,
0
0
0
b
o
o
tstrap
re
sa
m
p
les
.
Ca
li
b
ra
ti
o
n
is
ju
d
g
e
d
b
y
e
x
p
e
c
ted
c
a
li
b
ra
ti
o
n
e
rro
r
(
ECE
)
,
Brier,
re
li
a
b
il
i
ty
d
ia
g
ra
m
s,
a
n
d
tem
p
e
ra
tu
re
sc
a
li
n
g
.
VG
G
1
6
win
s
m
e
a
n
a
c
c
u
ra
c
y
(9
8
.
7
2
±
0
.
4
2
%
);
Re
sN
e
t5
0
V2
h
a
s
th
e
t
ig
h
tes
t
a
re
a
u
n
d
e
r
th
e
c
u
r
v
e
(
AUC
)
(0
.
9
9
8
6
±
0
.
0
0
0
6
);
th
e
two
a
re
sta
ti
stica
ll
y
ti
e
d
.
M
o
b
il
e
Ne
tV2
is
th
e
b
e
st
-
c
a
li
b
ra
ted
m
o
d
e
l
o
u
t
o
f
th
e
b
o
x
(ECE
=
0
.
0
0
2
1
)
with
o
n
l
y
2
.
5
9
M
p
a
ra
m
e
ters
,
m
a
k
in
g
it
th
e
m
o
st
d
e
p
lo
y
a
b
le.
A
f
o
u
r
-
le
v
e
l
c
o
n
f
i
d
e
n
c
e
-
b
a
se
d
risk
sc
o
re
t
u
rn
s
c
a
li
b
ra
ted
p
ro
b
a
b
il
it
ies
in
to
tri
a
g
e
tag
s.
T
h
is
stu
d
y
a
lso
flag
a
m
e
a
su
re
d
3
.
2
3
%
fo
rm
a
t
-
d
u
p
li
c
a
te
lea
k
a
g
e
in
th
e
p
u
b
li
c
d
a
tas
e
t
a
n
d
p
re
se
n
t
t
h
e
fra
m
e
wo
rk
a
s
a
ra
d
io
l
o
g
ist
c
o
-
p
il
o
t,
n
o
t
a
p
ro
g
n
o
s
ti
c
m
o
d
e
l
.
K
ey
w
o
r
d
s
:
B
r
ain
tu
m
o
r
m
a
g
n
etic
r
eso
n
a
n
ce
im
ag
in
g
C
alib
r
atio
n
C
o
n
f
i
d
e
n
c
e
-
b
a
s
e
d
r
i
s
k
s
t
r
a
t
i
f
i
c
a
t
i
o
n
C
o
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
b
en
ch
m
ar
k
in
g
C
r
o
s
s
-
v
alid
atio
n
Gr
ad
-
C
AM
T
r
an
s
f
er
lear
n
i
n
g
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
L
aiali
Alm
az
ay
d
eh
C
o
lleg
e
o
f
I
n
f
o
r
m
atio
n
T
ec
h
n
o
lo
g
y
,
Al
-
H
u
s
s
ein
B
in
T
alal
Un
iv
er
s
ity
Ma
’
an
,
J
o
r
d
a
n
E
m
ail:
laiali.
alm
az
ay
d
eh
@
ah
u
.
ed
u
.
j
o
1.
I
NT
RO
D
UCT
I
O
N
B
r
ain
tu
m
o
r
s
s
it
at
th
e
to
p
o
f
th
e
lis
t
wh
en
clin
ician
s
talk
ab
o
u
t
ca
n
ce
r
s
with
th
e
wo
r
s
t
o
u
tco
m
es.
C
atch
in
g
o
n
e
ea
r
ly
c
h
an
g
e
e
v
er
y
th
in
g
.
Su
r
g
eo
n
s
ca
n
p
lan
a
clea
n
er
r
esectio
n
,
th
e
r
a
d
io
th
e
r
ap
y
b
ea
m
is
aim
ed
with
m
o
r
e
co
n
f
id
e
n
ce
,
an
d
th
e
ch
em
o
th
e
r
ap
y
r
eg
im
en
is
p
ick
ed
with
b
etter
in
f
o
r
m
atio
n
.
All
o
f
th
o
s
e
m
a
p
s
d
ir
ec
tly
to
s
u
r
v
iv
al
a
n
d
q
u
ality
o
f
life
[
1
]
,
[
2
]
.
T
h
e
im
ag
in
g
s
id
e
r
elies
alm
o
s
t
en
tire
ly
o
n
m
ag
n
etic
r
eso
n
an
ce
im
ag
in
g
(
MRI)
.
I
t
s
h
o
ws
s
o
f
t
tis
s
u
e
in
a
way
co
m
p
u
ted
t
o
m
o
g
r
a
p
h
y
(
C
T
)
ca
n
n
o
t,
an
d
th
e
p
atien
t
is
n
o
t
ex
p
o
s
ed
to
io
n
izin
g
r
a
d
iatio
n
.
On
to
p
o
f
th
at,
d
ee
p
lear
n
in
g
h
as
q
u
ietly
r
esh
ap
ed
wh
at
r
ad
io
lo
g
is
ts
ca
n
lean
o
n
.
A
s
p
ec
ialis
t
-
lev
el
s
ec
o
n
d
r
ea
d
er
,
av
ailab
le
ev
e
n
in
h
o
s
p
it
als
with
o
u
t
a
s
en
io
r
r
ad
io
lo
g
y
team
,
is
n
o
lo
n
g
e
r
s
cien
ce
f
ictio
n
[
3
]
–
[
5
]
.
T
h
r
ee
th
in
g
s
s
till
h
o
ld
th
e
f
ield
b
a
ck
:
i)
p
ap
e
r
s
u
s
u
ally
r
e
p
o
r
t
r
esu
lts
o
n
a
s
in
g
le
tr
ain
/tes
t
s
p
lit
,
y
ield
in
g
a
n
u
m
b
er
r
at
h
er
th
a
n
a
co
n
f
id
en
ce
i
n
ter
v
al
[
6
]
,
ii)
th
e
r
aw
p
r
o
b
ab
i
liti
es
co
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
s
(
C
NNs)
h
an
d
b
ac
k
is
o
f
ten
m
is
ca
lib
r
ated
.
A
0
.
9
f
r
o
m
a
s
o
f
tm
ax
d
o
es
n
o
t
alwa
y
s
m
ea
n
90%
ch
an
ce
,
an
d
with
o
u
t
an
ex
p
lici
t
ca
lib
r
atio
n
ch
ec
k
n
o
b
o
d
y
s
h
o
u
ld
tr
ea
t
it
as
clin
ical
r
is
k
[
7
]
,
an
d
iii)
h
ea
v
y
an
d
lig
h
tweig
h
t
b
ac
k
b
o
n
es
ar
e
s
e
ld
o
m
tr
ain
ed
s
id
e
b
y
s
id
e
u
n
d
er
th
e
s
am
e
r
ec
ip
e.
W
ith
o
u
t
th
at,
th
e
ac
tu
al
ac
cu
r
ac
y
v
e
r
s
u
s
ef
f
icien
cy
f
r
o
n
tier
o
n
r
ea
l
ed
g
e
h
ar
d
war
e
s
tay
s
h
id
d
en
[
8
]
.
T
h
e
n
et
r
esu
lt
is
th
at
b
en
ch
m
ar
k
p
ap
er
s
an
d
d
e
p
lo
y
ab
le
s
o
f
tw
ar
e
liv
e
in
d
if
f
e
r
en
t
wo
r
ld
s
.
T
h
e
ar
ch
itectu
r
es
av
ailab
le
t
o
d
ay
a
r
e
f
a
r
m
o
r
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Dee
p
lea
r
n
in
g
-
b
a
s
ed
p
r
o
g
n
o
s
t
ic
mo
d
elin
g
o
f
b
r
a
in
tu
mo
r
s
…
(
La
ia
li A
lma
z
a
yd
eh
)
3793
d
iv
er
s
e
th
an
th
ey
wer
e
f
iv
e
y
ea
r
s
ag
o
.
C
lass
ic
co
n
v
o
lu
tio
n
al
f
am
ilies
lik
e
v
is
u
al
g
e
o
m
etr
y
g
r
o
u
p
(
VGG)
,
r
esid
u
al
n
etwo
r
k
(
R
esNet)
,
d
en
s
e
co
n
v
o
lu
tio
n
al
n
etwo
r
k
(
Den
s
eNe
t)
,
Mo
b
ileNet
,
an
d
E
f
f
icien
tNet
ar
e
s
till
ev
er
y
wh
er
e
.
Vis
io
n
tr
an
s
f
o
r
m
er
s
(
ViT
s
)
,
s
w
in
tr
an
s
f
o
r
m
er
s
,
th
e
C
o
n
v
NeX
t
r
ev
iv
al
o
f
p
u
r
e
co
n
v
o
lu
tio
n
s
an
d
h
y
b
r
id
C
NN
-
t
r
an
s
f
o
r
m
er
d
esi
g
n
s
h
av
e
all
ea
r
n
e
d
a
p
lace
o
n
th
e
lead
er
b
o
ar
d
s
.
Self
-
s
u
p
er
v
is
ed
p
r
e
-
tr
ai
n
in
g
an
d
m
ed
ical
im
ag
in
g
f
o
u
n
d
at
io
n
m
o
d
els
ar
e
n
o
w
s
tar
tin
g
to
tak
e
o
v
er
b
en
ch
m
a
r
k
s
to
o
[
9
]
–
[
1
4
]
.
On
th
e
in
ter
p
r
etab
ilit
y
s
id
e,
g
r
ad
ien
t
-
weig
h
ted
class
ac
tiv
atio
n
m
ap
p
in
g
(
Gr
ad
-
C
AM
)
[
1
5
]
a
n
d
S
h
ap
ley
a
d
d
itiv
e
ex
p
lan
atio
n
s
(S
HAP
)
[
1
6
]
ar
e
b
asically
s
tan
d
ar
d
.
Fo
r
tr
ain
in
g
ac
r
o
s
s
m
u
ltip
le
h
o
s
p
itals
with
o
u
t
m
o
v
in
g
r
aw
s
ca
n
s
,
f
ed
er
ated
lear
n
i
n
g
[
1
7
]
h
as b
ec
o
m
e
th
e
d
ef
au
lt a
n
s
wer
.
Sectio
n
2
ties
th
ese
th
r
ea
d
s
t
o
g
eth
er
.
Fro
m
th
is
s
tate
o
f
th
e
ar
t,
t
h
r
ee
co
n
cr
ete
g
ap
s
em
er
g
e
th
at
m
o
tiv
ate
th
is
s
tu
d
y
:
i)
a
f
air
,
id
en
tical
-
p
r
o
to
c
o
l
b
en
ch
m
ar
k
o
f
f
iv
e
wid
ely
u
s
ed
C
NN
b
ac
k
b
o
n
es
o
n
th
e
s
am
e
b
r
ain
tu
m
o
r
MRI
d
ataset
is
s
till
m
is
s
in
g
;
ii)
c
alib
r
atio
n
is
r
ar
ely
r
ep
o
r
ted
ev
e
n
th
o
u
g
h
p
r
o
b
a
b
ilit
y
q
u
ality
d
eter
m
in
es
wh
eth
er
a
m
o
d
el'
s
co
n
f
id
en
ce
ca
n
g
u
id
e
clin
ical
tr
iag
e
;
an
d
iii)
t
h
e
co
m
b
in
atio
n
o
f
cr
o
s
s
-
v
alid
atio
n
(
C
V)
,
s
tatis
tical
s
ig
n
if
ican
ce
test
in
g
,
ca
lib
r
atio
n
an
aly
s
is
,
ex
p
lain
ab
ilit
y
,
an
d
ef
f
icien
c
y
b
en
ch
m
ar
k
i
n
g
,
ea
c
h
in
d
iv
id
u
ally
well
k
n
o
wn
,
h
as
n
o
t
b
ee
n
ass
em
b
led
in
to
a
s
in
g
le,
d
ep
lo
y
m
en
t
-
o
r
ien
ted
ev
al
u
atio
n
s
u
ite
f
o
r
b
r
ai
n
tu
m
o
r
MRI.
W
e
clo
s
e
th
ese
g
ap
s
with
a
u
n
if
ied
ev
alu
atio
n
f
r
am
ewo
r
k
th
at
d
eliv
er
s
f
i
v
e
co
n
tr
ib
u
tio
n
s
:
i)
a
5
-
f
o
ld
s
tr
atif
ied
CV
b
e
n
ch
m
ar
k
o
f
VGG1
6
,
R
esNet5
0
V2
,
Mo
b
ileNetV2
,
E
f
f
icien
tNetB
0
,
a
n
d
Den
s
eNe
t1
2
1
u
n
d
er
a
s
in
g
le
tr
ain
in
g
r
ec
ip
e
;
ii)
p
air
wis
e
s
tatis
t
ical
s
ig
n
if
ican
ce
an
aly
s
is
u
s
in
g
Mc
Nem
ar
an
d
DeL
o
n
g
test
s
with
b
o
o
ts
tr
ap
9
5
%
co
n
f
id
en
ce
in
ter
v
als
(
C
I
s
)
;
iii)
a
ca
lib
r
at
io
n
s
tu
d
y
r
ep
o
r
tin
g
E
C
E
,
B
r
ier
s
co
r
e,
r
eliab
ilit
y
d
iag
r
am
s
,
an
d
tem
p
er
atu
r
e
s
ca
lin
g
;
iv
)
Gr
a
d
-
C
AM
b
ased
ex
p
lain
ab
ilit
y
an
d
a
f
o
u
r
-
l
ev
el
co
n
f
id
en
ce
-
b
ased
r
is
k
s
tr
atif
icatio
n
s
ch
em
e
(
lo
w,
m
ild
,
m
o
d
er
ate,
h
ig
h
)
in
ten
d
ed
f
o
r
clin
ical
tr
ia
g
e
;
an
d
v
)
a
n
e
f
f
icien
cy
b
en
c
h
m
ar
k
(
t
r
ain
in
g
tim
e,
in
f
er
en
ce
laten
c
y
,
p
ea
k
m
e
m
o
r
y
,
p
ar
am
eter
co
u
n
t)
th
at
lo
ca
tes
ea
ch
b
ac
k
b
o
n
e
o
n
t
h
e
ac
cu
r
ac
y
–
e
f
f
icien
cy
f
r
o
n
tier
.
W
e
also
q
u
an
tif
y
an
d
d
is
clo
s
e
a
s
m
all
(
3
.
2
3
%)
f
o
r
m
at
-
d
u
p
licate
leak
ag
e
in
th
e
p
u
b
lic
d
ataset
an
d
d
is
cu
s
s
its
im
p
licatio
n
s
.
T
h
is
wo
r
k
is
im
p
o
r
tan
t
i
n
f
o
u
r
way
s
.
T
h
e
ca
lib
r
ated
p
r
o
b
ab
ilit
ies
u
s
ed
f
o
u
r
-
lev
el
r
i
s
k
s
co
r
e
o
f
p
ac
k
ag
e
C
NN
o
u
tp
u
t
was
d
esig
n
ed
to
p
r
o
d
u
ce
ac
tio
n
ab
le
tr
iag
e
u
p
o
n
wh
ich
th
e
r
a
d
io
lo
g
is
t
ca
n
u
s
e
as
a
s
tr
u
ctu
r
ed
s
ec
o
n
d
o
p
in
io
n
.
H
o
s
p
itals
lack
in
g
s
u
f
f
icien
t
r
es
o
u
r
ce
s
f
in
d
Mo
b
ileNetV2
an
id
ea
l
ca
n
d
id
ate
f
o
r
d
ep
lo
y
m
e
n
t
s
in
ce
it
ac
h
ie
v
es
n
ea
r
-
b
est
ac
cu
r
ac
y
wit
h
th
e
b
est
-
in
-
class
o
u
t
-
of
-
t
h
e
-
b
o
x
ca
lib
r
atio
n
(
E
C
E
=
0
.
0
0
2
1
)
.
I
t
also
h
as
a
s
m
all
m
o
d
el
s
ize
o
f
2
.
5
9
M
p
ar
am
eter
s
an
d
1
0
.
5
m
in
t
r
ain
i
n
g
tim
e.
Acc
o
r
d
in
g
to
th
e
r
esear
ch
co
m
m
u
n
ity
,
t
h
e
p
r
o
to
co
l,
f
iv
e
b
ac
k
b
o
n
es,
id
en
tical
h
y
p
er
p
a
r
am
eter
s
,
r
e
p
ea
ted
C
V,
p
ai
r
ed
s
tatis
t
ical
tes
ts
,
an
d
ca
lib
r
atio
n
an
aly
s
is
d
ef
in
es
a
tr
an
s
p
ar
en
t
ev
alu
atio
n
tem
p
late
r
eu
s
ab
le
f
o
r
f
u
tu
r
e
ar
ch
itectu
r
e
in
cl
u
d
in
g
tr
an
s
f
o
r
m
er
s
an
d
f
o
u
n
d
atio
n
m
o
d
els.
I
n
co
n
clu
s
io
n
,
th
e
f
r
am
ewo
r
k
is
ex
p
licitly
p
o
s
itio
n
ed
as
a
r
a
d
io
lo
g
is
t
co
-
p
ilo
t
an
d
n
o
t
as
an
au
to
n
o
m
o
u
s
d
iag
n
o
s
tic
o
r
p
r
o
g
n
o
s
tic
d
ev
ice
to
ad
d
r
ess
s
af
e
h
u
m
an
-
in
-
th
e
-
lo
o
p
a
r
tific
ial
in
tellig
en
ce
(
AI
)
co
n
ce
r
n
s
.
T
h
e
f
o
llo
win
g
s
ec
tio
n
s
o
f
th
is
p
ap
er
will
b
e
o
r
g
an
ized
as
f
o
llo
ws.
Sect
io
n
2
p
r
esen
ts
liter
atu
r
e
r
elate
d
to
C
NN
-
b
ased
b
r
ain
t
u
m
o
r
class
if
icatio
n
,
tr
an
s
f
er
l
ea
r
n
in
g
,
e
x
p
lain
ab
ilit
y
,
m
o
d
er
n
ar
ch
itectu
r
e,
a
n
d
id
en
tifie
s
a
p
r
o
s
p
ec
tiv
e
r
esear
ch
g
ap
.
Sectio
n
3
d
escr
ib
es
o
u
r
ap
p
r
o
ac
h
in
d
etail,
in
clu
d
i
n
g
th
e
MRI
d
ataset
an
d
th
o
r
o
u
g
h
leak
ag
e
an
al
y
s
is
,
MRI
d
ata
p
r
ep
r
o
ce
s
s
in
g
,
m
ain
m
o
d
el
ar
c
h
itectu
r
es,
tr
a
in
in
g
a
n
d
s
tatis
tical
p
r
o
to
co
l
u
s
ed
,
ca
lib
r
atio
n
an
al
y
s
is
,
an
d
h
ea
t
m
ap
ex
p
lain
a
b
ilit
y
,
an
d
th
e
ass
o
ciate
d
r
is
k
-
s
tr
atif
icatio
n
s
ch
em
e.
T
h
is
s
ec
tio
n
co
n
tai
n
s
r
esu
lts
r
elate
d
to
CV
,
s
tatis
tical
test
in
g
,
ca
lib
r
atio
n
,
e
f
f
icien
cy
,
Gr
a
d
-
C
AM
an
aly
s
is
an
d
r
is
k
s
tr
atif
icatio
n
.
Fin
ally
,
s
ec
tio
n
6
c
o
n
clu
d
es
th
is
r
esear
ch
.
2.
L
I
T
E
R
AT
U
RE
R
E
VI
E
W
T
h
is
s
ec
tio
n
r
ev
iews
th
e
m
o
s
t
r
elev
an
t
p
r
io
r
wo
r
k
alo
n
g
f
iv
e
t
h
em
es
th
at
d
ir
ec
tly
m
o
tiv
ate
th
is
s
tu
d
y
:
C
NN
-
b
ased
b
r
ain
tu
m
o
r
class
if
icatio
n
,
tr
an
s
f
er
lear
n
in
g
with
p
r
e
-
tr
ain
e
d
b
ac
k
b
o
n
es,
ex
p
lain
ab
ilit
y
m
eth
o
d
s
f
o
r
clin
ical
tr
a
n
s
latio
n
,
m
o
d
e
r
n
ar
c
h
itectu
r
es
(
ViT
s
,
C
o
n
v
NeX
t
,
h
y
b
r
i
d
d
esig
n
s
,
s
elf
-
s
u
p
er
v
is
ed
,
an
d
f
o
u
n
d
atio
n
m
o
d
els),
an
d
th
e
r
e
s
ea
r
ch
g
ap
th
at
t
h
is
p
ap
er
clo
s
es.
2
.
1
.
Co
nv
o
lutio
na
l neura
l net
wo
rk
-
ba
s
ed
bra
in t
um
o
r
cl
a
s
s
if
ica
t
io
n
I
n
th
e
ar
ea
o
f
b
r
ai
n
tu
m
o
r
in
p
ar
ticu
lar
,
q
u
ite
a
f
ew
C
NN
-
b
ased
s
tu
d
ies h
av
e
p
r
o
v
ed
th
e
f
e
asib
ilit
y
o
f
au
to
m
atio
n
.
Per
eir
a
et
a
l.
[
1
8
]
u
s
ed
a
d
ee
p
C
NN
with
s
m
all
3
-
by
-
3
co
n
v
o
lu
tio
n
al
k
e
r
n
els
an
d
ac
h
iev
ed
m
u
lti
-
class
b
r
ain
tu
m
o
r
s
eg
m
en
tatio
n
o
n
MRI
with
a
c
o
m
p
etitiv
e
Dice
s
co
r
e
o
n
th
e
B
r
aT
S
ch
allen
g
e.
T
h
e
two
p
ath
way
s
C
NN
ar
ch
itect
u
r
e
in
tr
o
d
u
ce
d
b
y
Hav
ae
i
et
a
l.
[
1
9
]
ca
p
tu
r
es
lo
ca
l
as
wel
l
g
lo
b
al
co
n
tex
tu
al
in
f
o
r
m
atio
n
.
Fo
r
th
e
class
if
icatio
n
task
,
C
h
en
g
et
a
l.
[
2
0
]
ac
h
iev
ed
co
n
s
id
er
a
b
le
g
ain
s
f
r
o
m
u
s
in
g
tu
m
o
r
-
r
e
g
io
n
au
g
m
en
tatio
n
a
n
d
p
a
r
titi
o
n
in
g
wh
er
ea
s
Dee
p
ak
an
d
Am
ee
r
[
2
1
]
wer
e
ab
le
to
d
e
m
o
n
s
tr
ate
th
e
ef
f
ec
tiv
en
ess
o
f
u
s
in
g
tr
a
n
s
f
er
lear
n
in
g
o
n
a
Go
o
g
L
eNe
t
p
r
e
-
tr
ain
ed
f
o
r
th
r
ee
-
class
b
r
ain
t
u
m
o
r
class
if
icatio
n
o
n
MRI
with
a
h
i
g
h
ac
c
u
r
ac
y
r
ate
o
f
~9
8
%.
Su
r
v
ey
s
i
n
[
4
]
,
[
5
]
s
h
o
wca
s
e
a
b
r
o
ad
er
tr
en
d
i
n
m
ed
ical
im
ag
in
g
wh
er
e
C
NNs tak
e
o
v
er
h
an
d
-
c
r
af
ted
f
ea
tu
r
es.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
26
:
3
7
9
2
-
3
8
0
4
3794
2
.
2
.
T
ra
ns
f
er
lea
rning
a
nd
pre
-
t
ra
ined ba
ck
bo
nes
Me
d
ical
im
ag
in
g
co
m
m
o
n
l
y
u
s
es
tr
an
s
f
er
lear
n
in
g
f
r
o
m
I
m
a
g
eNe
t
-
p
r
etr
ain
ed
b
ac
k
b
o
n
es
a
s
s
tan
d
ar
d
p
r
ac
tice
[
6
]
.
VGG1
6
[
9
]
an
d
R
esNet5
0
V2
[
2
2
]
h
av
e
b
e
en
wid
ely
u
s
ed
b
en
c
h
m
ar
k
m
o
d
els
s
in
ce
th
en
.
Mo
b
ileNetV2
[
1
0
]
p
r
o
p
o
s
ed
in
v
er
ted
r
esid
u
al
b
lo
ck
s
co
n
tain
in
g
t
h
e
lin
ea
r
b
o
ttlen
ec
k
to
in
v
o
l
v
e
less
co
m
p
u
tatio
n
al
co
s
t
alo
n
g
w
ith
f
ewe
r
p
ar
am
eter
s
,
th
u
s
p
r
o
v
id
i
n
g
g
r
ea
t
f
lex
i
b
ilit
y
f
o
r
m
o
b
ile
-
f
r
ien
d
ly
in
f
er
en
ce
.
E
f
f
icien
tNetB
0
[
1
1
]
p
r
o
p
o
s
ed
a
co
m
p
o
u
n
d
s
ca
li
n
g
r
u
le
th
at
jo
in
tly
b
alan
c
es
d
ep
th
,
wid
th
,
an
d
in
p
u
t
r
eso
lu
tio
n
wh
ile
Den
s
e
Net1
2
1
[
1
2
]
s
tr
en
g
th
en
s
f
ea
tu
r
e
r
e
u
s
e
th
r
o
u
g
h
d
en
s
e
c
o
n
n
ec
tiv
ity
.
T
h
e
r
e
is
a
d
is
tin
ct
ac
cu
r
ac
y
–
ef
f
icien
cy
t
r
ad
e
-
o
f
f
with
ea
c
h
b
ac
k
b
o
n
e,
b
u
t
v
e
r
y
f
ew
s
tu
d
ies
p
r
io
r
to
th
is
b
en
ch
m
a
r
k
all
f
iv
e
f
am
ilies
u
n
d
er
a
s
in
g
le,
c
o
n
tr
o
lled
tr
ain
in
g
p
r
o
to
co
l
o
n
th
e
s
am
e
b
r
ain
tu
m
o
r
MRI
d
ataset,
a
co
n
tr
ib
u
tio
n
o
f
th
is
wo
r
k
.
2
.
3
.
E
x
pla
ina
bil
it
y
a
nd
clinica
l t
ra
ns
la
t
io
n
T
h
e
s
ea
m
less
in
teg
r
atio
n
o
f
d
e
ep
lear
n
in
g
in
to
r
ad
io
lo
g
y
wo
r
k
f
lo
ws
an
d
in
ter
p
r
eta
b
ilit
y
ar
e
ess
en
tial
f
o
r
its
clin
ical
ac
ce
p
tan
ce
n
o
t
ju
s
t
p
r
e
d
ictiv
e
p
e
r
f
o
r
m
an
ce
.
Me
th
o
d
s
lik
e
G
r
ad
-
C
AM
[
1
5
]
an
d
SHAP
-
b
ased
attr
ib
u
tio
n
s
[
1
6
]
,
u
s
ed
f
o
r
class
ac
tiv
atio
n
m
ap
p
in
g
,
h
ig
h
lig
h
t
th
e
lo
ca
tio
n
/r
eg
io
n
o
r
f
ea
tu
r
e
th
at
is
r
esp
o
n
s
ib
le
f
o
r
ea
ch
p
r
e
d
ictio
n
r
en
d
er
in
g
an
o
p
aq
u
e
m
o
d
el
in
to
a
u
s
ef
u
l
d
ec
is
io
n
s
u
p
p
o
r
t
s
y
s
tem
.
T
h
e
wid
er
m
ed
ical
-
AI
liter
atu
r
e
[
3
]
,
[
7
]
,
[
8
]
ec
h
o
es
th
em
es
f
r
o
m
elsewh
er
e:
th
e
r
e
ar
e
co
m
p
u
ter
-
v
is
io
n
p
r
o
jects
f
o
r
p
at
h
o
lo
g
y
an
d
n
atu
r
al
-
lan
g
u
ag
e
-
p
r
o
ce
s
s
in
g
p
r
o
jects
f
o
r
r
ad
io
l
o
g
y
r
e
p
o
r
ts
,
a
n
d
th
ey
all
p
o
i
n
t
to
s
am
e
n
ee
d
h
ig
h
-
p
e
r
f
o
r
m
an
ce
m
o
d
els
th
at
r
ad
io
lo
g
is
ts
ca
n
in
s
p
ec
t,
o
v
er
r
id
e
an
d
tr
u
s
t.
B
o
th
eth
ical
a
n
d
p
r
iv
ac
y
co
n
s
id
er
atio
n
s
ar
e
im
p
o
r
tan
t;
f
r
am
ewo
r
k
s
f
o
r
f
e
d
er
ated
lear
n
in
g
[
1
7
]
,
[
2
3
]
p
r
o
v
id
e
th
e
lead
in
g
p
ar
ad
i
g
m
f
o
r
tr
ain
in
g
ac
r
o
s
s
m
u
ltip
le
in
s
titu
tio
n
s
with
o
u
t e
x
ch
an
g
i
n
g
r
aw
p
atien
t d
ata.
2
.
4
.
M
o
dern
a
rc
hite
ct
ures a
nd
f
o
un
da
t
io
n m
o
dels
T
h
e
n
ew
a
r
ch
itectu
r
al
f
a
m
ilies
s
in
ce
2
0
2
2
ar
e
wo
r
th
m
e
n
tio
n
in
g
ex
p
licitly
.
ViT
[
1
3
]
is
ab
le
to
ch
ar
ac
ter
ize
an
im
a
g
e
b
y
b
r
ea
k
in
g
th
e
im
ag
e
in
t
o
p
atc
h
es
an
d
u
s
in
g
th
em
as
in
p
u
t
f
o
r
tr
a
n
s
f
o
r
m
e
r
.
Swin
t
r
an
s
f
o
r
m
e
r
[
1
4
]
an
d
Sw
in
V2
[
2
4
]
le
v
er
ag
e
h
ier
ar
c
h
ic
al
s
witch
ed
-
win
d
o
w
s
elf
-
atten
tio
n
f
o
r
e
f
f
icien
cy
o
n
d
en
s
e
p
r
ed
ictio
n
task
s
.
C
o
n
v
NeX
t
[
2
5
]
a
n
d
C
o
n
v
NeX
t
V2
[
2
6
]
b
r
in
g
p
u
r
e
-
co
n
v
o
lu
ti
o
n
al
d
esig
n
s
u
p
t
o
d
ate
u
s
in
g
d
e
p
th
wis
e
co
n
v
o
lu
tio
n
s
,
lar
g
er
k
e
r
n
els,
an
d
s
elf
-
s
u
p
er
v
is
ed
p
r
e
-
tr
ai
n
in
g
to
cl
o
s
e
th
e
tr
a
n
s
f
o
r
m
e
r
g
ap
.
Ar
c
h
itectu
r
es
o
f
h
y
b
r
id
C
NNs
an
d
t
r
an
s
f
o
r
m
e
r
s
co
m
b
in
e
th
e
v
ar
iatio
n
s
o
f
c
o
n
v
o
lu
tio
n
s
with
th
e
atten
tio
n
o
f
g
lo
b
al
m
o
d
ellin
g
.
T
ec
h
n
iq
u
es
lik
e
m
ask
ed
a
u
to
e
n
co
d
er
s
(
MA
E
)
[
2
7
]
o
r
DI
NOv
2
[
2
8
]
u
s
in
g
s
elf
-
s
u
p
er
v
is
ed
p
r
e
-
t
r
ain
in
g
less
en
lab
el
d
ep
en
d
e
n
ce
an
d
p
r
o
d
u
ce
g
en
er
ic
v
is
u
al
r
ep
r
esen
tatio
n
s
f
r
o
m
u
n
lab
eled
d
ata.
As
o
f
n
o
w,
m
e
d
ical
-
im
ag
in
g
f
o
u
n
d
atio
n
m
o
d
els
i
n
clu
d
in
g
Me
d
SAM
[
2
9
]
f
o
r
s
eg
m
en
tatio
n
a
n
d
B
io
m
ed
C
L
I
P
[
3
0
]
f
o
r
v
is
i
o
n
–
la
n
g
u
a
g
e
a
li
g
n
m
e
n
t
ar
e
att
a
in
i
n
g
2
0
2
3
–
2
0
2
5
s
t
ate
-
of
-
t
h
e
-
a
r
t
f
o
r
r
ad
io
lo
g
y
wo
r
k
f
l
o
ws
.
I
n
t
h
is
w
o
r
k
,
n
o
n
e
o
f
t
h
ese
f
a
m
il
ies
a
r
e
b
e
n
ch
m
a
r
k
e
d
;
t
h
e
y
a
r
e
ex
p
l
icit
ly
id
en
t
if
ie
d
as
p
r
io
r
i
ty
tar
g
ets
f
o
r
n
e
x
t
b
e
n
c
h
m
a
r
k
.
2
.
5
.
Resea
rc
h g
a
p a
nd
o
ur
co
ntr
ibu
t
io
n
W
ea
v
in
g
to
g
eth
er
th
e
s
tr
an
d
s
ab
o
v
e,
th
r
ee
g
ap
s
em
er
g
e
.
Firstl
y
,
th
e
m
ajo
r
ity
o
f
p
r
io
r
wo
r
k
s
u
tili
zin
g
C
NNs
f
o
r
b
r
ain
tu
m
o
r
d
etec
ti
o
n
an
d
class
if
icatio
n
o
n
ly
e
v
alu
ate
a
s
in
g
le
ar
ch
itectu
r
e
,
u
n
d
er
a
s
in
g
le
tr
ain
/tes
t
s
p
lit.
T
h
is
p
r
o
v
id
es
v
er
y
litt
le
in
s
ig
h
t
in
to
h
o
w
m
o
d
er
n
b
ac
k
b
o
n
es
co
m
p
a
r
e
u
n
d
e
r
id
en
ti
ca
l
co
n
d
itio
n
s
an
d
h
o
w
s
tab
le
th
e
m
etr
ics
ar
e
ac
r
o
s
s
s
p
lits
.
Mo
r
eo
v
er
,
ca
lib
r
atio
n
,
s
tatis
tical
s
ig
n
if
ican
ce
,
ef
f
icien
cy
b
en
ch
m
ar
k
in
g
,
ex
p
lain
a
b
ilit
y
an
d
a
clin
ical
r
is
k
-
s
tr
atif
icatio
n
o
u
tp
u
t
ar
e
r
a
r
ely
r
ep
o
r
ted
t
o
g
eth
er
,
t
h
o
u
g
h
all
ar
e
r
eq
u
ir
ed
f
o
r
clin
ical
d
e
p
lo
y
m
en
t.
T
h
ir
d
,
lig
h
tweig
h
t
m
o
d
els
(
lik
e
Mo
b
ileNetV2
)
ten
d
to
n
o
t
b
e
e
v
alu
ated
to
g
eth
er
with
a
h
ea
v
ier
b
ac
k
b
o
n
e
o
n
th
e
s
am
e
b
r
ai
n
tu
m
o
r
d
ata
n
o
r
f
o
r
ca
lib
r
atio
n
q
u
ality
.
T
h
is
r
esear
ch
p
ap
er
a
p
p
ea
r
s
to
clo
s
e
th
ese
g
ap
s
as
it
b
en
ch
m
a
r
k
s
f
iv
e
f
am
ilies
o
f
C
NNs
u
n
d
er
a
u
n
if
ied
5
-
f
o
ld
CV
p
r
o
to
c
o
l
with
p
air
ed
s
tatis
tical
test
s
,
an
aly
s
is
o
f
ca
lib
r
atio
n
,
ex
p
lan
atio
n
s
v
ia
Gr
a
d
-
C
AM
,
ef
f
icie
n
cy
m
etr
ics,
an
d
a
co
n
f
id
en
ce
-
b
ased
r
is
k
s
co
r
e
.
3.
M
E
T
H
O
D
T
h
e
p
r
o
p
o
s
ed
p
ip
elin
e
co
m
p
r
i
s
es
s
ev
en
s
tag
es:
d
ata
ac
q
u
is
itio
n
,
p
r
ep
r
o
ce
s
s
in
g
,
m
o
d
el
tr
ai
n
in
g
u
n
d
er
s
tr
atif
ied
5
-
f
o
ld
CV
,
m
u
lti
-
m
etr
ic
ev
alu
atio
n
with
s
tatis
tical
s
ig
n
if
ican
ce
test
in
g
,
ca
lib
r
atio
n
an
aly
s
is
,
ex
p
lain
ab
ilit
y
th
r
o
u
g
h
Gr
ad
-
C
AM
,
an
d
co
n
f
id
en
ce
-
b
ased
r
is
k
s
tr
atif
icatio
n
.
3
.
1
.
Da
t
a
s
et
T
h
e
p
u
b
lic
b
r
ain
tu
m
o
r
MRI
co
llectio
n
co
n
s
is
t
s
o
f
4
,
6
0
0
s
ca
n
s
(
2
,
5
1
3
tu
m
o
r
-
p
o
s
itiv
e
an
d
2
,
0
8
7
h
ea
lth
y
)
s
to
r
ed
in
J
PG,
T
I
FF
an
d
PNG
f
iles
[
3
1
]
.
Fig
u
r
e
1
p
r
esen
ts
ex
am
p
les
o
f
th
e
b
r
ain
MRI
s
ca
n
s
u
s
ed
in
th
is
s
tu
d
y
,
in
clu
d
in
g
tu
m
o
r
-
p
o
s
itiv
e
s
ca
n
s
in
Fig
u
r
e
1
(
a)
an
d
h
ea
lth
y
co
n
tr
o
ls
in
Fig
u
r
e
1
(
b
)
.
T
h
e
m
etad
ata
in
d
icate
s
th
at
th
e
b
r
ain
im
ag
es
wer
e
s
ca
n
n
ed
f
r
o
m
p
atien
ts
d
iag
n
o
s
ed
with
a
b
r
ain
t
u
m
o
r
.
T
h
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Dee
p
lea
r
n
in
g
-
b
a
s
ed
p
r
o
g
n
o
s
t
ic
mo
d
elin
g
o
f
b
r
a
in
tu
mo
r
s
…
(
La
ia
li A
lma
z
a
yd
eh
)
3795
d
im
en
s
io
n
s
o
f
th
e
im
a
g
es
ar
e
b
etwe
en
2
5
0
-
by
-
2
0
1
to
6
2
0
-
by
-
6
2
0
p
ix
els
wh
ich
in
d
icate
s
t
h
at
th
e
im
ag
es
wer
e
tak
en
b
y
MRI
s
ca
n
in
s
tead
o
f
X
-
r
ay
.
E
ac
h
p
h
o
to
h
as
a
b
ase
I
D
s
u
ch
as
“c
an
ce
r
1
”
o
r
“n
o
r
m
al
1
.
”
T
h
e
m
etad
ata
ch
ec
k
s
h
o
wed
th
at
9
2
o
f
th
e
4
,
5
0
6
b
ase
I
Ds (
r
o
u
g
h
ly
2
%)
ar
e
s
to
r
ed
in
m
u
ltip
le
f
o
r
m
ats (
jp
g
+
tif
o
r
jp
g
+
tif
+
p
n
g
)
.
T
h
er
ef
o
r
e,
th
er
e
will
b
e
a
m
i
n
o
r
f
o
r
m
at
-
d
u
p
licate
leak
ag
e
if
th
e
f
iles
ar
e
s
p
lit
r
an
d
o
m
ly
s
in
ce
f
iles
with
th
e
s
am
e
b
ase
I
D
w
ill
n
o
t
b
e
g
r
o
u
p
ed
to
g
eth
er
.
T
o
m
ea
s
u
r
e
th
e
p
r
ac
tical
ef
f
ec
t,
we
ca
lcu
lated
th
e
p
r
o
p
o
r
tio
n
o
f
test
b
ase
id
en
tifi
er
s
wh
ich
ap
p
ea
r
e
d
in
th
e
r
esp
ec
tiv
e
tr
ain
in
g
f
o
ld
,
o
b
tain
in
g
an
av
er
ag
e
o
v
er
la
p
o
f
3
.
2
3
%
ac
r
o
s
s
th
e
f
iv
e
f
o
ld
s
(
r
an
g
e
2
.
7
2
%
–
3
.
9
5
%.
T
h
e
d
i
v
is
io
n
o
f
d
ata
in
to
s
u
b
s
ets
is
d
o
n
e
c
o
n
s
id
er
in
g
all
f
ea
tu
r
es in
th
e
d
ata
s
et.
A
f
u
tu
r
e
g
r
o
u
p
-
awa
r
e
s
p
lit f
o
r
CV
h
as
b
ee
n
d
esig
n
ed
w
h
ich
will b
e
d
is
cu
s
s
ed
in
d
etail
in
s
ec
tio
n
5
.
T
h
is
s
ec
tio
n
will d
is
cu
s
s
m
eth
o
d
o
f
e
n
s
u
r
in
g
p
a
r
ticip
an
ts
ar
e
in
th
e
s
am
e
f
o
ld
.
(
a)
(
b
)
Fig
u
r
e
1
.
R
ep
r
esen
tativ
e
MRI
s
am
p
les (
a)
tu
m
o
r
-
p
o
s
itiv
e
ca
s
es
an
d
(
b
)
h
ea
lth
y
c
o
n
tr
o
ls
3
.
2
.
P
re
pro
ce
s
s
ing
a
nd
a
ug
m
ent
a
t
io
n
All
im
ag
es
ar
e
r
esized
t
o
2
2
4
p
ix
els
b
y
2
2
4
p
ix
els
.
I
n
ca
s
e
n
ec
ess
ar
y
,
we
h
av
e
also
d
o
n
e
g
r
ay
s
ca
le
-
to
-
R
GB
co
n
v
er
s
io
n
.
T
h
e
im
a
g
e
p
ix
el
v
alu
es
ar
e
s
ca
led
i
n
th
e
[
0
,
1
]
r
a
n
g
e.
T
h
e
n
etwo
r
k
tak
es
ca
r
e
o
f
m
o
d
el
s
p
ec
if
ic
p
r
ep
r
o
ce
s
s
_
in
p
u
t
in
te
r
n
ally
in
th
e
n
etwo
r
k
b
y
cr
ea
tin
g
a
lay
er
o
f
ty
p
e
L
am
b
d
a.
(
No
te:
ea
r
lier
in
th
e
p
ip
elin
e,
th
e
p
r
ep
r
o
ce
s
s
_
in
p
u
t
was
o
m
itted
,
wh
ich
m
ad
e
th
e
E
f
f
icien
tNetB
0
p
er
f
o
r
m
q
u
i
te
b
ad
ly
.
T
h
e
o
th
er
m
o
d
els
d
id
n
’
t
h
a
v
e
as
b
ad
i
m
p
ac
t
with
th
at
o
m
is
s
io
n
,
b
u
t
co
r
r
ec
tio
n
in
cl
u
d
ed
n
o
n
et
h
eless
)
.
T
o
en
s
u
r
e
r
o
b
u
s
tn
ess
ag
ain
s
t
ac
q
u
is
itio
n
v
ar
iab
ilit
y
d
u
r
in
g
tr
ain
in
g
,
r
ea
l
-
tim
e
d
ata
au
g
m
en
tatio
n
was
ap
p
lied
to
th
e
d
at
a
in
clu
d
in
g
s
ca
n
n
er
d
if
f
e
r
en
ce
s
an
d
im
a
g
in
g
ar
tifa
cts.
W
h
ile
q
u
an
tify
in
g
th
e
r
e
g
io
n
o
f
in
te
r
est,
we
p
er
f
o
r
m
e
d
f
lip
p
in
g
v
er
tically
/h
o
r
izo
n
tall
y
,
r
an
d
o
m
r
o
tatio
n
(
u
p
-
to
±
1
5
°),
b
r
ig
h
tn
ess
p
er
tu
r
b
atio
n
(
f
r
o
m
0
.
8
to
1
.
2
)
,
zo
o
m
(
u
p
-
t
o
±
1
0
%)
,
an
d
s
h
e
ar
.
T
h
e
u
s
e
o
f
g
eo
m
etr
ic
a
n
d
p
h
o
to
m
etr
ic
au
g
m
en
tatio
n
im
p
licitly
r
eg
u
lar
izes
th
e
m
o
d
el
wh
ils
t a
ls
o
ac
tin
g
a
s
a
s
o
f
t f
o
r
m
o
f
d
o
m
ain
r
a
n
d
o
m
izatio
n
.
3
.
3
.
C
o
nv
o
lutio
na
l neura
l net
wo
rk
a
rc
hite
ct
ures
Fiv
e
p
o
p
u
lar
C
NN
b
a
ck
b
o
n
es
wer
e
b
e
n
ch
m
ar
k
ed
u
n
d
e
r
th
e
s
am
e
tr
ain
in
g
r
ec
ip
e:
VGG1
6
,
R
esNet5
0
V2
,
Mo
b
ileNetV2
,
E
f
f
icien
tNetB
0
,
an
d
Den
s
eNe
t1
2
1
.
E
ac
h
n
etwo
r
k
was
in
itiali
ze
d
with
I
m
ag
eNe
t
p
r
e
-
tr
ain
ed
weig
h
ts
.
T
h
e
class
if
icatio
n
h
ea
d
was
r
ep
lace
d
b
y
g
lo
b
al
av
e
r
ag
e
p
o
o
lin
g
f
o
llo
wed
b
y
b
atch
n
o
r
m
aliza
tio
n
,
a
2
5
6
-
u
n
it
d
en
s
e
r
ec
tifie
d
lin
ea
r
u
n
it
(
R
eL
U
)
lay
er
,
a
d
r
o
p
o
u
t
lay
er
(
r
ate
0
.
4
)
,
an
d
a
s
in
g
le
s
ig
m
o
id
-
ac
tiv
ated
o
u
t
p
u
t.
Mo
b
ileNetV2
was
ex
p
licitly
in
cl
u
d
ed
as
a
lig
h
tweig
h
t
ca
n
d
id
a
te
f
o
r
lo
w
-
r
eso
u
r
ce
clin
ical
d
ep
lo
y
m
e
n
ts
,
with
o
n
l
y
2
.
5
9
M
tr
ain
ab
le
p
ar
am
eter
s
.
3
.
4
.
T
ra
ini
ng
pro
t
o
co
l
T
r
ain
in
g
u
s
ed
s
tr
atif
ied
5
-
f
o
l
d
CV
.
Fo
r
ev
er
y
f
o
ld
,
th
e
tr
a
in
in
g
p
o
r
tio
n
was
f
u
r
th
er
d
iv
i
d
ed
9
0
/
1
0
in
to
a
tr
ain
i
n
g
a
n
d
a
n
in
ter
n
al
v
alid
atio
n
s
et.
E
ac
h
m
o
d
el
was
tr
ain
ed
in
two
p
h
ases
o
v
er
1
5
e
p
o
ch
s
(
7
f
r
o
ze
n
+
8
f
in
e
-
t
u
n
e)
.
I
n
p
h
ase
1
,
th
e
b
ac
k
b
o
n
e
was
f
r
o
ze
n
an
d
th
e
h
ea
d
was
tr
ain
ed
with
th
e
Ad
am
o
p
tim
izer
[
3
2
]
at
lear
n
in
g
r
ate
1
e
-
3
.
I
n
p
h
ase
2
,
th
e
to
p
2
0
l
ay
er
s
o
f
th
e
b
ac
k
b
o
n
e
wer
e
u
n
f
r
o
ze
n
an
d
tr
ai
n
ed
with
Ad
am
at
lear
n
i
n
g
r
ate
1
e
-
5
.
T
h
e
lo
s
s
f
u
n
ctio
n
was
b
i
n
ar
y
c
r
o
s
s
-
en
tr
o
p
y
with
class
weig
h
ts
in
v
er
s
ely
p
r
o
p
o
r
tio
n
al
to
class
f
r
eq
u
e
n
cy
.
Mix
e
d
-
p
r
ec
is
io
n
(
f
lo
at1
6
)
tr
ain
in
g
an
d
a
b
atch
s
ize
o
f
3
2
wer
e
u
s
ed
.
E
ar
ly
Sto
p
p
in
g
m
o
n
ito
r
ed
v
ali
d
atio
n
ar
ea
u
n
d
er
th
e
cu
r
v
e
(
AUC
)
with
p
atien
ce
4
a
n
d
r
e
s
to
r
ed
b
est
weig
h
ts
,
an
d
R
ed
u
ce
L
R
On
Plateau
h
alv
ed
th
e
lear
n
in
g
r
ate
ev
er
y
2
ep
o
ch
s
o
f
s
tag
n
atin
g
v
alid
atio
n
lo
s
s
.
T
ab
le
1
s
u
m
m
ar
izes
th
e
co
m
p
lete
h
y
p
er
p
ar
am
ete
r
co
n
f
ig
u
r
a
tio
n
.
All
ex
p
er
im
en
ts
r
an
o
n
a
Kag
g
le
n
o
teb
o
o
k
b
ac
k
ed
b
y
a
s
in
g
le
NVI
DI
A
T
esla
P1
0
0
GPU
(
1
6
GB
VR
A
M)
an
d
3
0
GB
o
f
s
y
s
tem
r
an
d
o
m
ac
ce
s
s
m
em
o
r
y
(
R
AM
)
,
u
s
in
g
T
en
s
o
r
Flo
w
2
.
1
5
,
Ker
as
3
.
0
,
s
cik
it
-
lear
n
1
.
7
,
an
d
Py
th
o
n
3
.
1
0
.
E
ac
h
e
p
o
ch
to
o
k
r
o
u
g
h
ly
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
26
:
3
7
9
2
-
3
8
0
4
3796
3
0
to
4
5
s
ec
o
n
d
s
d
ep
en
d
in
g
o
n
th
e
b
ac
k
b
o
n
e,
an
d
a
f
u
ll
1
5
-
ep
o
ch
two
-
p
h
ase
r
u
n
f
in
is
h
ed
in
1
0
to
1
2
m
in
u
tes
p
er
m
o
d
el.
T
ab
le
1
.
Hy
p
er
p
ar
a
m
eter
co
n
f
ig
u
r
atio
n
u
s
ed
f
o
r
ev
er
y
C
NN
b
ac
k
b
o
n
e
(
f
u
ll r
ep
r
o
d
u
ci
b
ilit
y
d
is
clo
s
u
r
e)
H
y
p
e
r
p
a
r
a
me
t
e
r
V
a
l
u
e
I
mag
e
si
z
e
2
2
4
×
2
2
4
B
a
t
c
h
si
z
e
32
M
i
x
e
d
p
r
e
c
i
s
i
o
n
f
l
o
a
t
1
6
Ep
o
c
h
s
p
h
a
se
1
(
f
r
o
z
e
n
)
7
Ep
o
c
h
s
p
h
a
se
2
(
f
i
n
e
-
t
u
n
e
)
8
O
p
t
i
mi
z
e
r
A
d
a
m
Le
a
r
n
i
n
g
r
a
t
e
(
p
h
a
se
1
)
1e
-
3
Le
a
r
n
i
n
g
r
a
t
e
(
p
h
a
se
2
)
1e
-
5
U
n
f
r
o
z
e
n
t
o
p
l
a
y
e
r
s
20
Lo
ss
B
i
n
a
r
y
c
r
o
ss
-
e
n
t
r
o
p
y
+
c
l
a
ss
w
e
i
g
h
t
s
C
l
a
s
s we
i
g
h
t
i
n
g
I
n
v
e
r
se
f
r
e
q
u
e
n
c
y
D
e
n
se
h
e
a
d
G
A
P
+
B
N
+
D
e
n
s
e
(
2
5
6
,
R
e
LU
)
+
D
r
o
p
o
u
t
(
0
.
4
)
Ea
r
l
y
st
o
p
p
i
n
g
mo
n
i
t
o
r
v
a
l
_
a
u
c
,
p
a
t
i
e
n
c
e
4
,
r
e
s
t
o
r
e
b
e
st
R
e
d
u
c
e
L
R
O
n
P
l
a
t
e
a
u
mo
n
i
t
o
r
v
a
l
_
l
o
ss,
f
a
c
t
o
r
0
.
5
,
p
a
t
i
e
n
c
e
2
A
u
g
m
e
n
t
a
t
i
o
n
H
/
V
f
l
i
p
,
r
o
t
a
t
i
o
n
±
1
5
°
,
b
r
i
g
h
t
n
e
ss
0
.
8
–
1
.
2
,
z
o
o
m
±
0
.
1
,
s
h
e
a
r
±
0
.
1
CV
5
-
f
o
l
d
s
t
r
a
t
i
f
i
e
d
R
a
n
d
o
m s
e
e
d
42
B
o
o
t
s
t
r
a
p
i
t
e
r
a
t
i
o
n
s
1
,
0
0
0
3
.
5
.
St
a
t
is
t
ica
l a
na
ly
s
is
T
o
ch
ec
k
wh
eth
er
th
e
g
ap
s
b
etwe
en
m
o
d
els
ar
e
r
ea
l,
we
p
o
o
led
test
p
r
e
d
ictio
n
s
ac
r
o
s
s
th
e
5
f
o
ld
s
an
d
r
an
th
r
ee
ch
ec
k
s
.
First,
p
a
ir
ed
Mc
Nem
ar
test
s
[
3
3
]
with
co
n
tin
u
ity
co
r
r
ec
tio
n
c
o
m
p
ar
ed
b
est
-
p
er
f
o
r
m
in
g
m
o
d
el
(
b
y
m
ea
n
AUC)
ag
ain
s
t
ev
er
y
o
th
er
m
o
d
el
o
n
p
er
-
s
am
p
le
co
r
r
ec
t
n
ess
.
Seco
n
d
,
DeL
o
n
g
test
s
[
3
4
]
co
m
p
ar
ed
ea
ch
p
ai
r
o
f
AUCs
wh
ile
ac
co
u
n
tin
g
f
o
r
th
e
co
v
ar
ian
ce
b
etwe
en
p
r
ed
ictio
n
s
o
n
s
h
ar
ed
s
am
p
les.
T
h
ir
d
,
n
o
n
p
ar
a
m
etr
ic
b
o
o
ts
tr
a
p
r
esam
p
lin
g
with
1
,
0
0
0
iter
atio
n
s
p
r
o
d
u
ce
d
9
5
%
CI
s
f
o
r
ac
cu
r
ac
y
an
d
AUC
f
o
r
ev
e
r
y
m
o
d
el.
Fo
r
r
ea
d
er
s
u
n
f
am
iliar
with
t
h
ese
test
s
:
M
cNe
m
ar
ch
ec
k
s
wh
eth
er
two
class
if
ier
s
d
is
ag
r
ee
o
n
th
e
s
am
e
p
atien
ts
in
a
wa
y
th
at
ch
an
ce
alo
n
e
ca
n
n
o
t
e
x
p
lain
,
an
d
DeL
o
n
g
test
s
wh
eth
er
two
r
ec
eiv
er
o
p
er
atin
g
ch
ar
ac
te
r
is
tic
(
R
OC
)
cu
r
v
es
d
if
f
er
o
n
ce
we
ac
c
o
u
n
t
f
o
r
th
e
f
ac
t
th
at
th
e
y
a
r
e
c
o
m
p
u
ted
o
n
th
e
s
am
e
p
atien
ts
.
B
o
th
ar
e
s
tan
d
ar
d
t
o
o
ls
in
clin
ical
m
ac
h
in
e
lear
n
in
g
ev
alu
atio
n
.
3
.
6
.
Ca
lib
ra
t
i
o
n a
na
ly
s
is
T
h
r
ee
m
ea
s
u
r
es
ar
e
u
s
ed
t
o
ju
d
g
e
p
r
o
b
ab
ilit
y
q
u
ality
.
E
x
p
ec
ted
ca
lib
r
atio
n
e
r
r
o
r
(
E
C
E
)
[
3
5
]
is
co
m
p
u
ted
o
v
er
1
0
e
q
u
al
-
wid
th
p
r
o
b
ab
ilit
y
b
in
s
u
s
in
g
th
e
s
tan
d
ar
d
d
ef
in
itio
n
b
ased
o
n
th
e
b
in
-
a
v
er
ag
e
d
f
r
ac
tio
n
o
f
p
o
s
itiv
es.
T
h
e
B
r
ie
r
s
co
r
e
was
co
m
p
u
ted
as
t
h
e
m
ea
n
s
q
u
ar
e
d
er
r
o
r
b
etwe
en
p
r
ed
icted
p
r
o
b
a
b
ilit
y
an
d
tr
u
e
lab
el,
p
r
o
v
id
i
n
g
a
s
tr
i
ctly
p
r
o
p
er
s
co
r
in
g
r
u
le
s
en
s
itiv
e
to
b
o
th
ca
lib
r
atio
n
a
n
d
s
h
ar
p
n
ess
.
R
eliab
ilit
y
d
iag
r
am
s
v
is
u
alize
th
e
b
in
-
wi
s
e
f
r
ac
tio
n
o
f
p
o
s
itiv
es
ag
ain
s
t
p
r
ed
icted
p
r
o
b
ab
ilit
y
.
T
h
is
s
tu
d
y
ad
d
itio
n
ally
f
it
a
s
in
g
le
-
p
ar
am
eter
tem
p
er
at
u
r
e
s
ca
lin
g
[
3
2
]
o
n
a
h
eld
-
o
u
t
f
o
ld
b
y
m
i
n
im
izin
g
n
eg
ati
v
e
lo
g
-
lik
elih
o
o
d
an
d
r
ep
o
r
t c
alib
r
atio
n
m
etr
ics b
o
th
b
ef
o
r
e
an
d
a
f
ter
s
ca
lin
g
.
3
.
7
.
E
v
a
lua
t
i
o
n m
et
rics a
nd
ex
pla
ina
bil
it
y
T
h
e
m
etr
ics
r
ep
o
r
ted
i
n
th
is
s
tu
d
y
in
clu
d
e
,
ac
c
u
r
ac
y
,
p
r
ec
is
io
n
,
s
en
s
itiv
ity
(
r
ec
all)
,
s
p
ec
if
icity
,
F1
-
s
co
r
e,
an
d
AUC
-
R
OC
,
ar
e
co
m
p
u
ted
p
er
f
o
l
d
an
d
s
u
m
m
a
r
ized
as
m
ea
n
±
s
tan
d
ar
d
d
ev
i
atio
n
.
E
f
f
icien
cy
i
s
ch
ar
ac
ter
ized
b
y
f
o
u
r
a
d
d
itio
n
al
in
d
icato
r
s
:
to
tal
tr
ain
in
g
t
im
e
in
m
in
u
tes,
p
e
r
-
im
ag
e
i
n
f
er
en
ce
laten
cy
in
m
illi
s
ec
o
n
d
s
,
p
ea
k
i
n
f
er
en
ce
m
em
o
r
y
in
m
eg
a
b
y
tes,
an
d
tr
ain
ab
le
-
p
ar
am
eter
co
u
n
t
in
m
illi
o
n
s
.
Fo
r
in
ter
p
r
etab
ilit
y
we
a
p
p
lied
G
r
ad
-
C
AM
[
1
5
]
to
th
e
f
in
al
c
o
n
v
o
lu
ti
o
n
al
b
l
o
ck
o
f
ea
ch
m
o
d
el.
Gr
a
d
-
C
AM
p
r
o
d
u
ce
s
a
co
a
r
s
e
lo
ca
lizatio
n
h
ea
tm
ap
h
ig
h
lig
h
tin
g
t
h
e
r
e
g
io
n
s
wh
o
s
e
g
r
ad
ien
ts
m
o
s
t
s
tr
o
n
g
ly
in
f
lu
en
ce
th
e
p
r
ed
icted
class
.
T
h
e
r
esu
ltin
g
v
is
u
aliza
tio
n
s
wer
e
o
v
er
lai
d
o
n
th
e
o
r
ig
in
al
MRI
s
ca
n
to
v
er
if
y
clin
ically
m
ea
n
in
g
f
u
l a
tten
tio
n
.
3
.
8
.
Co
nfidence
-
ba
s
ed
risk
s
t
ra
t
if
ica
t
i
o
n
B
ey
o
n
d
b
in
ar
y
class
if
icatio
n
,
a
f
o
u
r
-
lev
el
c
o
n
f
i
d
en
ce
-
b
ased
r
is
k
s
co
r
e
is
d
e
r
iv
ed
f
r
o
m
th
e
ca
lib
r
ated
s
ig
m
o
id
p
r
o
b
a
b
ilit
y
p
o
f
th
e
b
est
-
p
er
f
o
r
m
i
n
g
m
o
d
el.
T
h
e
f
o
u
r
b
in
s
ar
e
l
o
w
r
is
k
(
p
<
0
.
2
5
)
,
m
ild
r
is
k
(
0
.
2
5
≤
p
<
0
.
5
0
)
,
m
o
d
er
ate
r
is
k
(
0
.
5
0
≤
p
<
0
.
7
5
)
,
a
n
d
h
ig
h
r
is
k
(
p
≥
0
.
7
5
)
.
T
h
e
th
r
esh
o
ld
s
wer
e
tu
n
e
d
o
n
th
e
v
alid
atio
n
p
o
r
tio
n
o
f
th
e
f
ir
s
t f
o
ld
s
o
as to
m
in
im
ize
th
e
n
u
m
b
er
o
f
h
ig
h
-
r
is
k
ca
s
es in
co
r
r
ec
tly
lab
elled
as
l
o
w,
p
r
io
r
itizin
g
clin
ical
s
af
ety
.
T
h
is
s
ch
em
e
is
d
elib
er
ately
lab
e
lled
“
co
n
f
i
d
en
ce
-
b
ased
r
is
k
s
t
r
atif
icatio
n
”
r
ath
e
r
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Dee
p
lea
r
n
in
g
-
b
a
s
ed
p
r
o
g
n
o
s
t
ic
mo
d
elin
g
o
f
b
r
a
in
tu
mo
r
s
…
(
La
ia
li A
lma
z
a
yd
eh
)
3797
th
an
“
p
r
o
g
n
o
s
tic
m
o
d
elin
g
,
”
b
ec
au
s
e
n
o
lo
n
g
itu
d
in
al
o
u
tco
m
e
d
ata
(
s
u
r
v
i
v
al,
r
ec
u
r
r
e
n
ce
,
an
d
tr
ea
tm
e
n
t
r
esp
o
n
s
e)
ar
e
u
s
ed
;
th
e
s
co
r
e
d
is
cr
etize
s
th
e
m
o
d
el'
s
ca
lib
r
ated
co
n
f
id
en
ce
in
to
clin
ical
ly
ac
tio
n
ab
le
tr
ia
g
e
tag
s
th
at
co
m
p
lem
en
t t
h
e
b
in
a
r
y
d
iag
n
o
s
is
.
4.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
Fo
r
th
e
r
esu
lts
,
th
e
an
aly
s
i
s
is
s
tr
u
ctu
r
ed
in
to
8
s
u
b
s
ec
tio
n
s
wh
ich
m
atch
th
e
n
ew
ev
alu
atio
n
p
illar
s
:
5
-
f
o
ld
class
if
icatio
n
p
er
f
o
r
m
an
ce
,
s
tatis
tical
s
ig
n
if
ican
ce
,
R
OC
an
aly
s
is
,
ca
lib
r
atio
n
,
tr
ain
in
g
d
y
n
am
ics,
ef
f
icien
cy
b
e
n
ch
m
ar
k
s
,
Gr
ad
-
C
AM
v
is
u
aliza
tio
n
,
an
d
co
n
f
id
en
ce
-
b
ased
r
is
k
s
tr
atif
icatio
n
.
4
.
1
.
Cro
s
s
-
v
a
lid
a
t
io
n c
la
s
s
if
ica
t
io
n per
f
o
rma
nce
T
ab
le
2
s
h
o
ws
5
-
f
o
ld
CV
n
u
m
b
er
s
.
VGG1
6
o
b
tain
ed
t
h
e
h
ig
h
est
m
ea
n
ac
cu
r
ac
y
(
0
.
9
8
7
2
±
0
.
0
0
4
2
)
an
d
th
e
s
tr
o
n
g
est
m
ea
n
F1
-
s
c
o
r
e
(
0
.
9
8
8
3
±
0
.
0
0
3
8
)
,
wh
ile
R
esNet5
0
V2
h
ad
th
e
tig
h
test
AUC
ac
r
o
s
s
f
o
ld
s
(
0
.
9
9
8
6
±
0
.
0
0
0
6
)
with
th
e
lo
west
ac
r
o
s
s
-
f
o
ld
s
tan
d
ar
d
d
ev
i
atio
n
.
Mo
b
ileNetV2
r
ea
c
h
ed
a
m
ea
n
ac
cu
r
ac
y
o
f
0
.
9
7
9
3
±
0
.
0
0
4
8
with
o
n
l
y
2
.
5
9
M
p
ar
am
eter
s
.
E
f
f
icien
tNe
tB
0
,
wh
ich
p
r
ev
io
u
s
ly
u
n
d
er
p
er
f
o
r
m
ed
at
6
1
%
ac
cu
r
ac
y
b
ec
a
u
s
e
o
f
a
m
is
s
in
g
.
T
h
ese
ac
cu
r
ac
y
lev
els
ar
e
co
n
s
is
ten
t
with
p
r
ev
io
u
s
ly
r
ep
o
r
ted
tr
an
s
f
er
-
lear
n
in
g
r
esu
lts
o
n
b
r
ain
tu
m
o
r
MRI:
Dee
p
ak
an
d
Am
e
er
[
2
1
]
r
ea
ch
ed
a
b
o
u
t 9
8
% with
a
p
r
e
-
tr
ain
e
d
Go
o
g
L
eNe
t,
an
d
C
h
en
g
et
a
l.
[
2
0
]
r
ep
o
r
ted
co
m
p
a
r
ab
le
g
ain
s
a
f
ter
tu
m
o
r
-
r
eg
io
n
au
g
m
en
tati
o
n
.
T
h
e
co
n
tr
i
b
u
tio
n
is
th
at
th
e
s
am
e
lev
el
o
f
ac
cu
r
ac
y
is
n
o
w
d
em
o
n
s
tr
ated
ac
r
o
s
s
f
iv
e
b
ac
k
b
o
n
es
u
n
d
er
o
n
e
p
r
o
t
o
co
l
with
r
ep
ea
ted
s
p
l
its
,
r
ath
er
th
an
o
n
a
s
in
g
le
s
p
lit
o
f
o
n
e
ar
ch
itectu
r
e
[
6
]
.
Pre
p
r
o
ce
s
s
_
in
p
u
t
s
tep
,
n
o
w
r
ea
ch
es
0
.
9
8
1
7
±
0
.
0
0
3
7
o
n
ce
th
e
p
r
ep
r
o
ce
s
s
in
g
is
co
r
r
ec
tly
ap
p
lied
.
Den
s
eNe
t1
2
1
tr
ails
s
li
g
h
tly
at
0
.
9
7
5
0
±
0
.
0
0
6
4
.
T
h
e
f
u
ll
p
er
-
m
etr
ic
b
r
ea
k
d
o
wn
is
s
h
o
wn
in
T
ab
le
2
(
m
ea
n
±
s
td
)
.
Per
-
class
b
eh
a
v
io
r
is
wo
r
th
n
o
tin
g
f
o
r
clin
ica
l saf
ety
: sen
s
it
iv
ity
f
o
r
th
e
tu
m
o
r
class
s
tay
s
ab
o
v
e
9
7
.
9
%
f
o
r
ev
e
r
y
m
o
d
el,
wh
ich
is
wh
at
m
atter
s
m
o
s
t
wh
en
th
e
c
o
s
t
o
f
a
m
is
s
ed
tu
m
o
r
is
h
ig
h
;
s
p
ec
if
icity
o
n
h
ea
lth
y
s
ca
n
s
is
9
6
.
4
%
to
9
8
.
5
%,
with
Den
s
eNe
t1
2
1
t
h
e
s
lig
h
t
o
u
tlier
o
n
th
e
lo
w
en
d
.
T
ab
le
2
.
5
-
f
o
ld
CV
r
esu
lts
(
m
ea
n
±
s
td
)
M
o
d
e
l
A
c
c
u
r
a
c
y
P
r
e
c
i
s
i
o
n
S
e
n
s
i
t
i
v
i
t
y
S
p
e
c
i
f
i
c
i
t
y
F1
-
s
c
o
r
e
AUC
B
r
i
e
r
V
G
G
1
6
0
.
9
8
7
2
±
0
.
0
0
4
2
0
.
9
8
7
3
±
0
.
0
0
4
5
0
.
9
8
9
3
±
0
.
0
0
5
7
0
.
9
8
4
7
±
0
.
0
0
5
6
0
.
9
8
8
3
±
0
.
0
0
3
8
0
.
9
9
8
5
±
0
.
0
0
1
3
0
.
0
1
0
2
±
0
.
0
0
3
9
R
e
sN
e
t
5
0
V
2
0
.
9
8
5
7
±
0
.
0
0
1
9
0
.
9
8
7
3
±
0
.
0
0
4
5
0
.
9
8
6
5
±
0
.
0
0
3
9
0
.
9
8
4
7
±
0
.
0
0
5
6
0
.
9
8
6
9
±
0
.
0
0
1
7
0
.
9
9
8
6
±
0
.
0
0
0
6
0
.
0
1
0
9
±
0
.
0
0
1
4
M
o
b
i
l
e
N
e
t
V
2
0
.
9
7
9
3
±
0
.
0
0
4
8
0
.
9
8
3
6
±
0
.
0
0
6
4
0
.
9
7
8
5
±
0
.
0
0
5
5
0
.
9
8
0
4
±
0
.
0
0
7
8
0
.
9
8
1
1
±
0
.
0
0
4
4
0
.
9
9
6
4
±
0
.
0
0
1
5
0
.
0
1
6
1
±
0
.
0
0
3
4
Ef
f
i
c
i
e
n
t
N
e
t
B
0
0
.
9
8
1
7
±
0
.
0
0
3
7
0
.
9
8
2
6
±
0
.
0
0
5
7
0
.
9
8
4
1
±
0
.
0
0
4
6
0
.
9
7
8
9
±
0
.
0
0
7
0
0
.
9
8
3
3
±
0
.
0
0
3
4
0
.
9
9
6
6
±
0
.
0
0
1
7
0
.
0
1
4
8
±
0
.
0
0
2
5
D
e
n
seN
e
t
1
2
1
0
.
9
7
5
0
±
0
.
0
0
6
4
0
.
9
7
0
6
±
0
.
0
0
6
0
0
.
9
8
4
1
±
0
.
0
0
8
8
0
.
9
6
4
1
±
0
.
0
0
7
6
0
.
9
7
7
3
±
0
.
0
0
5
9
0
.
9
9
6
0
±
0
.
0
0
1
8
0
.
0
1
9
0
±
0
.
0
0
4
8
4
.
2
.
St
a
t
is
t
ica
l sig
nifica
nce
t
esting
T
ab
le
3
lis
ts
th
e
p
air
wis
e
test
s
co
m
p
ar
in
g
R
esNet5
0
V2
(
b
e
s
t
b
y
m
ea
n
AUC)
ag
ain
s
t
th
e
o
th
er
f
o
u
r
m
o
d
els
o
n
t
h
e
p
o
o
led
CV
p
r
e
d
ictio
n
s
.
Pair
ed
s
ig
n
if
ican
ce
t
esti
n
g
o
f
th
is
k
in
d
is
s
tan
d
ar
d
in
clin
ical
m
ac
h
in
e
lear
n
in
g
b
u
t
is
r
ar
ely
r
ep
o
r
te
d
in
b
r
ain
tu
m
o
r
class
if
icatio
n
s
tu
d
ies;
we
f
o
llo
w
th
e
r
ec
o
m
m
en
d
atio
n
s
o
f
DeL
o
n
g
et
a
l.
[3
4
]
f
o
r
co
r
r
elate
d
R
OC
cu
r
v
es
an
d
Mc
Nem
ar
[
33
]
f
o
r
p
air
ed
cla
s
s
if
ier
co
m
p
ar
is
o
n
.
Mc
Nem
ar
'
s
test
r
ejec
ts
th
e
n
u
ll
h
y
p
o
th
esis
o
f
eq
u
i
v
alen
t
class
if
icatio
n
p
er
f
o
r
m
a
n
ce
a
g
ain
s
t
Mo
b
ileNetV2
(
p
=
0
.
0
0
7
9
)
,
E
f
f
icien
tNetB
0
(
p
=
0
.
0
8
9
,
b
o
r
d
er
lin
e)
,
a
n
d
Den
s
eNe
t1
2
1
(
p
<
0
.
0
0
0
1
)
,
b
u
t
ca
n
n
o
t
r
ejec
t
a
tie
with
VGG1
6
(
p
=
0
.
4
9
)
.
D
eL
o
n
g
'
s
AUC
test
co
n
f
ir
m
s
a
s
ig
n
if
ican
t
AUC
g
ap
ag
ain
s
t
Mo
b
ileNetV2
(
p
=
0
.
0
0
1
3
)
,
E
f
f
icien
tNetB
0
(
p
=
0
.
0
0
0
6
)
,
a
n
d
De
n
s
eNe
t1
2
1
(
p
=
0
.
0
0
0
2
)
,
b
u
t
a
g
ain
f
in
d
s
n
o
d
if
f
e
r
en
ce
b
etwe
en
R
esNet5
0
V2
an
d
V
GG1
6
(
p
=
0
.
9
9
7
)
.
B
o
o
ts
tr
ap
9
5
%
CI
s
in
T
ab
le
3
a
r
e
tig
h
t
f
o
r
ac
cu
r
ac
y
(
wid
th
s
o
f
0
.
0
0
5
–
0
.
0
0
9
)
an
d
v
er
y
tig
h
t
f
o
r
AUC
(
wid
th
s
u
n
d
er
0
.
0
0
3
)
,
co
n
f
i
r
m
in
g
t
h
e
s
tab
ilit
y
o
f
th
e
r
an
k
in
g
s
.
T
h
e
1
,
0
0
0
-
iter
atio
n
n
o
n
p
a
r
am
etr
ic
b
o
o
ts
tr
ap
CI
s
f
o
r
AUC
ar
e
v
is
u
alize
d
in
Fig
u
r
e
2
,
wh
er
e
ea
ch
m
o
d
el'
s
p
o
in
t e
s
tim
ate
an
d
9
5
% C
I
ar
e
s
h
o
wn
s
id
e
b
y
s
id
e
.
T
ab
le
3
.
Pair
wis
e
s
tatis
tical
te
s
ts
ag
ain
s
t Re
s
Net5
0
V2
(
b
est b
y
m
ea
n
AUC)
P
a
i
r
(
R
e
sN
e
t
5
0
V
2
v
s)
M
c
N
e
mar χ
²
M
c
N
e
mar
p
D
e
Lo
n
g
ΔA
U
C
D
e
Lo
n
g
p
V
e
r
d
i
c
t
V
G
G
1
6
0
.
4
7
0
.
4
9
4
+
0
.
0
0
0
0
0
2
0
.
9
9
7
S
t
a
t
i
st
i
c
a
l
l
y
e
q
u
i
v
a
l
e
n
t
M
o
b
i
l
e
N
e
t
V
2
7
.
0
6
0
.
0
0
8
+
0
.
0
0
2
1
2
3
0
.
0
0
1
3
S
i
g
n
i
f
i
c
a
n
t
Ef
f
i
c
i
e
n
t
N
e
t
B
0
2
.
8
9
0
.
0
8
9
+
0
.
0
0
2
0
6
8
0
.
0
0
0
6
B
o
r
d
e
r
l
i
n
e
/
s
i
g
n
i
f
i
c
a
n
t
D
e
n
seN
e
t
1
2
1
1
6
.
5
8
<
0
.
0
0
1
+
0
.
0
0
2
6
6
8
0
.
0
0
0
2
H
i
g
h
l
y
s
i
g
n
i
f
i
c
a
n
t
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
26
:
3
7
9
2
-
3
8
0
4
3798
Fig
u
r
e
2
.
B
o
o
ts
tr
ap
9
5
% c
o
n
f
id
en
ce
in
ter
v
als f
o
r
AUC (
1
,
0
0
0
r
esam
p
les)
4
.
3
.
RO
C
a
na
ly
s
is
T
h
e
p
o
o
led
R
OC
cu
r
v
es
f
o
r
all
f
iv
e
m
o
d
els
ar
e
s
h
o
wn
in
Fig
u
r
e
3
.
R
esNet5
0
V2
an
d
VGG1
6
s
it
clo
s
est
to
th
e
to
p
-
lef
t
wit
h
m
ea
n
test
AUC
s
o
f
0
.
9
9
8
6
an
d
0
.
9
9
8
5
r
esp
ec
tiv
el
y
.
E
f
f
icien
tNetB
0
,
Mo
b
ileNetV2
,
an
d
Den
s
eNe
t1
2
1
f
o
llo
w
with
AUCs
ab
o
v
e
0
.
9
9
6
.
T
h
e
o
r
d
er
o
f
th
e
c
u
r
v
es
is
co
n
s
i
s
ten
t
with
th
e
s
tatis
tical
te
s
ts
an
d
with
t
h
e
b
o
o
ts
tr
ap
C
I
s
.
T
h
e
AUC
v
alu
es
we
o
b
tain
ar
e
i
n
th
e
s
am
e
r
an
g
e
as
th
o
s
e
r
ep
o
r
ted
f
o
r
C
NN
-
b
ased
b
r
ai
n
tu
m
o
r
p
ip
elin
es
in
[
1
8
]
,
[
1
9
]
,
wh
ich
s
u
p
p
o
r
ts
th
e
v
iew
th
at
th
e
d
is
cr
im
in
ativ
e
ce
ilin
g
o
n
th
is
task
is
n
o
w
s
et
b
y
d
ata
q
u
ality
r
ath
e
r
th
an
b
y
ar
ch
itectu
r
e
ch
o
ice
[
4
]
,
[
5
]
.
Fig
u
r
e
3
.
R
OC
cu
r
v
es p
o
o
led
ac
r
o
s
s
5
C
V
f
o
ld
s
4
.
4
.
Ca
lib
ra
t
i
o
n r
esu
lt
s
C
alib
r
atio
n
r
esu
lts
ar
e
s
u
m
m
ar
ized
in
T
a
b
le
4
an
d
v
is
u
alize
d
in
Fig
u
r
e
4
.
Mo
b
ileNetV2
is
,
s
o
m
ewh
at
u
n
e
x
p
ec
ted
ly
,
th
e
b
est
-
ca
lib
r
ated
m
o
d
el
o
u
t
o
f
th
e
b
o
x
with
an
E
C
E
o
f
0
.
0
0
2
1
a
n
d
a
n
o
p
tim
al
tem
p
er
atu
r
e
o
f
1
.
0
4
,
clo
s
e
to
1
.
R
esNet5
0
V2
an
d
VGG1
6
f
o
llo
w
with
E
C
E
s
o
f
0
.
0
0
5
9
an
d
0
.
0
0
6
1
,
a
n
d
E
f
f
icien
tNetB
0
s
its
at
0
.
0
0
7
1
.
Den
s
eNe
t1
2
1
h
as
th
e
h
ig
h
est
r
aw
E
C
E
at
0
.
0
1
5
5
,
b
u
t
its
t
em
p
er
atu
r
e
s
ca
lin
g
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Dee
p
lea
r
n
in
g
-
b
a
s
ed
p
r
o
g
n
o
s
t
ic
mo
d
elin
g
o
f
b
r
a
in
tu
mo
r
s
…
(
La
ia
li A
lma
z
a
yd
eh
)
3799
T
h
is
f
in
d
in
g
is
r
elev
an
t
b
ec
au
s
e
m
o
d
er
n
d
ee
p
n
etwo
r
k
s
ar
e
k
n
o
wn
to
b
e
s
y
s
tem
atica
lly
o
v
er
-
co
n
f
id
en
t
[
7
]
,
an
d
th
e
tem
p
e
r
atu
r
e
-
s
ca
lin
g
co
r
r
ec
tio
n
we
ap
p
ly
f
o
llo
ws
th
e
p
r
o
ce
d
u
r
e
in
tr
o
d
u
ce
d
f
o
r
e
x
ac
tly
th
at
p
r
o
b
lem
[
7
]
,
[
3
4
]
.
(
T
=
0
.
7
6
)
s
h
r
in
k
s
th
e
er
r
o
r
t
o
0
.
0
1
1
9
.
T
em
p
er
atu
r
e
s
ca
lin
g
also
r
ed
u
ce
s
E
C
E
f
o
r
R
esNet5
0
V2
(
0
.
0
0
5
9
→
0
.
0
0
3
1
)
an
d
E
f
f
icien
tNetB
0
(
0
.
0
0
7
1
→
0
.
0
0
4
5
)
.
B
r
ier
s
co
r
es
ac
r
o
s
s
all
m
o
d
els
lie
b
etwe
en
0
.
0
1
0
a
n
d
0
.
0
1
9
,
in
d
icatin
g
th
at
all
f
iv
e
b
ac
k
b
o
n
es
p
r
o
d
u
ce
ca
lib
r
ated
a
n
d
s
h
ar
p
p
r
o
b
ab
ilit
ies
s
u
itab
le
f
o
r
tr
iag
e.
R
eliab
ilit
y
d
iag
r
am
s
ar
e
p
r
o
v
i
d
ed
in
t
h
e
c
alib
r
atio
n
f
ig
u
r
es
.
T
ab
le
4
.
C
alib
r
atio
n
m
etr
ics b
ef
o
r
e
an
d
af
ter
tem
p
er
atu
r
e
s
ca
lin
g
M
o
d
e
l
T
EC
E
(
r
a
w
)
EC
E
(
T)
B
r
i
e
r
(
r
a
w
)
B
r
i
e
r
(
T)
V
G
G
1
6
1
.
5
0
0
.
0
0
6
1
0
.
0
0
7
3
0
.
0
1
0
2
0
.
0
1
0
5
R
e
sN
e
t
5
0
V
2
1
.
2
3
0
.
0
0
5
9
0
.
0
0
3
1
0
.
0
1
0
9
0
.
0
1
0
8
M
o
b
i
l
e
N
e
t
V
2
1
.
0
4
0
.
0
0
2
1
0
.
0
0
2
4
0
.
0
1
6
1
0
.
0
1
6
1
Ef
f
i
c
i
e
n
t
N
e
t
B
0
1
.
2
9
0
.
0
0
7
1
0
.
0
0
4
5
0
.
0
1
4
8
0
.
0
1
4
9
D
e
n
seN
e
t
1
2
1
0
.
7
6
0
.
0
1
5
5
0
.
0
1
1
9
0
.
0
1
9
0
0
.
0
1
8
7
Fig
u
r
e
4
.
E
C
E
an
d
B
r
ier
s
co
r
e
b
ef
o
r
e
v
s
af
ter
tem
p
e
r
atu
r
e
s
c
alin
g
f
o
r
all
f
iv
e
b
ac
k
b
o
n
es
4
.
5
.
T
ra
ini
ng
dy
na
m
ics
T
h
e
p
er
-
m
o
d
el
tr
ain
in
g
an
d
v
alid
atio
n
cu
r
v
es
(
lo
s
s
an
d
A
UC
)
ar
e
s
h
o
wn
in
Fig
u
r
e
5
.
All
m
o
d
els
co
n
v
er
g
ed
s
m
o
o
th
l
y
an
d
d
o
n
o
t
s
h
o
w
an
y
o
v
er
f
itti
n
g
b
eh
av
io
r
with
in
1
5
e
p
o
ch
s
,
t
h
u
s
v
alid
atin
g
th
e
two
-
p
h
ase
f
in
e
-
tu
n
in
g
s
ch
e
d
u
le
an
d
th
e
ea
r
ly
-
s
to
p
p
in
g
cr
it
er
io
n
.
T
h
e
ab
s
en
ce
o
f
d
iv
er
g
en
ce
b
etwe
en
th
e
tr
ain
in
g
an
d
v
alid
atio
n
cu
r
v
es
in
d
icate
s
th
at
th
e
au
g
m
en
tatio
n
an
d
class
-
weig
h
tin
g
s
tr
ateg
y
-
co
n
tr
o
lled
o
v
er
f
itti
n
g
e
f
f
ec
tiv
ely
,
i
n
lin
e
with
th
e
s
y
s
tem
atic
ev
id
en
ce
r
ep
o
r
ted
b
y
B
ejan
i a
n
d
Gh
atee
[
6
]
.
Fig
u
r
e
5
.
T
r
ain
in
g
a
n
d
v
alid
atio
n
cu
r
v
es f
o
r
R
esNet5
0
V2
(
r
e
p
r
esen
tativ
e)
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
2
5
2
-
8
9
3
8
I
n
t J Ar
tif
I
n
tell
,
Vo
l.
15
,
No
.
4
,
Au
g
u
s
t
20
26
:
3
7
9
2
-
3
8
0
4
3800
4
.
6
.
E
f
f
iciency
benchm
a
r
k
s
Fro
m
T
ab
le
5
a
n
d
Fig
u
r
e
6
,
i
t
is
o
b
s
er
v
ed
Mo
b
ileNetV2
h
as
th
e
lo
west
n
u
m
b
er
o
f
p
ar
a
m
eter
s
i.e
.
2
.
5
9
M
an
d
th
e
lo
west
tr
ain
in
g
tim
e
i.e
.
1
0
.
5
m
in
u
tes.
T
h
e
co
m
b
in
ed
f
i
n
d
in
g
s
in
d
icate
th
at
Mo
b
ileNetV2
i
s
th
e
o
p
tim
al
c
h
o
ice
f
o
r
b
ed
s
id
e
d
ep
lo
y
m
en
t.
T
h
e
R
esNet5
0
V2
m
o
d
el
ac
h
iev
es
th
e
h
ig
h
est
AUC
s
co
r
e,
b
u
t
it
r
eq
u
ir
es
2
4
.
1
m
illi
o
n
p
ar
a
m
eter
s
an
d
ea
ch
i
m
ag
e
h
a
s
an
in
f
er
en
ce
laten
cy
o
f
5
0
.
8
6
m
illi
s
ec
o
n
d
s
.
Den
s
eNe
t1
2
1
an
d
VGG1
6
ar
e
r
an
k
ed
in
th
e
m
id
d
le.
T
h
e
in
f
er
en
ce
p
ea
k
m
em
o
r
y
o
f
all
th
e
m
o
d
els
is
m
o
d
est
with
a
v
alu
e
≤
0
.
2
MB m
ak
in
g
it f
ea
s
ib
le
o
n
a
m
o
d
er
n
h
o
s
p
ital w
o
r
k
s
tatio
n
.
T
h
is
is
co
n
s
i
s
t
en
t w
ith
th
e
d
esig
n
in
ten
t
o
f
th
e
in
v
er
ted
-
r
esid
u
al
ar
ch
itectu
r
e
[
1
0
]
,
an
d
it
co
m
p
lem
en
ts
th
e
co
m
p
o
u
n
d
-
s
ca
lin
g
ar
g
u
m
en
t
m
a
d
e
f
o
r
E
f
f
icien
tNet
[
1
1
]
;
in
o
u
r
s
ettin
g
th
e
s
m
aller
m
o
d
el
is
p
r
ef
er
ab
le
o
n
ce
ca
lib
r
atio
n
a
n
d
h
ar
d
war
e
co
s
t
ar
e
co
n
s
id
er
ed
alo
n
g
s
id
e
ac
cu
r
ac
y
[
8
]
.
T
h
e
e
f
f
icien
cy
p
r
o
f
ile
o
f
ea
ch
b
ac
k
b
o
n
e
(
tr
ain
i
n
g
tim
e,
in
f
er
en
ce
laten
cy
,
p
ea
k
m
em
o
r
y
,
p
ar
am
eter
co
u
n
t,
an
d
FLOPs)
is
v
is
u
alize
d
in
Fig
u
r
e
6
.
T
ab
le
5
.
E
f
f
icien
cy
b
en
ch
m
a
r
k
s
p
er
b
ac
k
b
o
n
e
M
o
d
e
l
Tr
a
i
n
(
mi
n
)
La
t
e
n
c
y
(
ms)
M
e
m
o
r
y
(
M
B
)
P
a
r
a
ms (
M
)
F
LO
P
s (G
)
V
G
G
1
6
1
2
.
3
4
5
.
5
3
0
.
2
0
1
4
.
8
5
1
5
.
4
7
R
e
sN
e
t
5
0
V
2
1
2
.
3
5
0
.
8
6
0
.
2
0
2
4
.
1
0
4
.
1
0
M
o
b
i
l
e
N
e
t
V
2
1
0
.
5
4
9
.
0
3
0
.
2
0
2
.
5
9
0
.
3
1
Ef
f
i
c
i
e
n
t
N
e
t
B
0
1
2
.
4
5
2
.
3
1
0
.
2
0
4
.
3
8
0
.
3
9
D
e
n
seN
e
t
1
2
1
1
0
.
7
5
8
.
0
2
0
.
1
8
7
.
3
0
2
.
8
7
Fig
u
r
e
6
.
E
f
f
icien
cy
b
en
ch
m
ar
k
s
: tr
ain
in
g
tim
e,
in
f
e
r
en
ce
lat
en
cy
,
m
em
o
r
y
,
an
d
p
ar
am
eter
co
u
n
t
4
.
7
.
G
ra
dient
-
weig
hte
d c
la
s
s
a
ct
iv
a
t
io
n
m
a
pp
ing
v
is
ua
liza
t
io
n
Fig
u
r
e
6
s
h
o
ws
Gr
ad
-
C
AM
h
ea
tm
ap
s
o
f
R
esNet5
0
V2
o
v
e
r
laid
o
n
e
x
em
p
lar
y
tu
m
o
r
-
p
o
s
itiv
e
im
ag
es.
I
n
tu
itiv
ely
,
th
e
ac
tiv
atio
n
m
ap
s
an
d
o
cc
ip
ital
p
atch
in
g
c
o
n
s
i
s
ten
tly
f
o
r
m
a
d
ip
o
le
in
th
e
r
e
g
io
n
o
f
i
n
tr
ac
r
an
ial
lesi
o
n
wh
ile
av
o
id
in
g
an
y
ac
tiv
atio
n
in
th
e
ir
r
elev
a
n
t
b
ac
k
g
r
o
u
n
d
.
T
h
is
r
ea
s
s
u
r
ed
u
s
t
h
at
th
e
m
o
d
el
was
r
ely
in
g
o
n
clin
ically
m
ea
n
in
g
f
u
l
f
ea
tu
r
es
f
o
r
d
ec
is
io
n
-
m
ak
in
g
.
T
h
ese
v
is
u
aliza
tio
n
s
m
a
y
b
e
d
ir
ec
tly
u
s
ab
le
i
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J Ar
tif
I
n
tell
I
SS
N:
2252
-
8
9
3
8
Dee
p
lea
r
n
in
g
-
b
a
s
ed
p
r
o
g
n
o
s
t
ic
mo
d
elin
g
o
f
b
r
a
in
tu
mo
r
s
…
(
La
ia
li A
lma
z
a
yd
eh
)
3801
r
ad
io
lo
g
is
ts
’
d
ash
b
o
ar
d
s
an
d
f
o
r
m
th
e
b
asis
o
f
a
h
u
m
an
-
A
I
co
-
p
ilo
t
wo
r
k
f
l
o
w.
R
ep
r
esen
tativ
e
Gr
ad
-
C
AM
s
alien
cy
m
ap
s
f
o
r
R
esNet5
0
V2
o
v
er
laid
o
n
t
u
m
o
r
-
p
o
s
itiv
e
MRI
s
ca
n
s
ar
e
s
h
o
wn
in
Fig
u
r
e
7
.
Fig
u
r
e
7
.
Gr
a
d
-
C
AM
h
ea
tm
ap
s
f
r
o
m
R
esNet5
0
V2
o
v
er
laid
o
n
r
ep
r
esen
tativ
e
tu
m
o
r
-
p
o
s
itiv
e
MRI
s
ca
n
s
4
.
8
.
Co
nfidence
-
ba
s
ed
risk
s
t
ra
t
if
ica
t
i
o
n
T
h
e
f
o
u
r
–
lev
el
r
is
k
s
co
r
e
ass
o
ciate
d
with
c
o
n
f
id
e
n
ce
in
th
e
ca
lib
r
ated
p
r
o
b
a
b
ilit
y
o
f
R
esNet5
0
V2
is
g
iv
en
in
T
ab
le
6
.
T
h
e
s
co
r
e
al
lo
ws
clin
ician
s
to
p
r
io
r
itize
ca
s
es
b
y
u
r
g
en
cy
.
B
ased
o
n
f
o
r
e
ca
s
ted
v
alu
es
f
r
o
m
p
o
o
led
CV
,
4
7
%
o
f
ca
s
es
g
o
to
th
e
lo
w
-
r
is
k
b
in
,
5
%
to
th
e
m
ild
b
in
,
6
%
to
th
e
m
o
d
er
ate
b
in
,
an
d
4
2
%
to
th
e
h
ig
h
b
in
.
T
h
e
f
i
g
u
r
es
lie
v
er
y
clo
s
e
to
th
e
ac
tu
al
u
n
d
er
ly
in
g
tu
m
o
r
p
r
e
v
alen
ce
.
I
n
t
h
e
v
i
e
w
,
n
o
t
u
m
o
r
-
p
o
s
i
ti
v
e
o
c
c
u
r
r
e
n
c
e
i
s
a
s
s
i
g
n
e
d
t
o
t
h
e
l
ow
-
r
i
s
k
b
i
n
,
s
u
g
g
e
s
t
i
n
g
t
h
a
t
th
e
t
u
n
i
n
g
o
f
a
n
y
p
a
r
t
i
c
u
l
a
r
t
h
r
e
s
h
o
l
d
e
f
f
e
c
t
i
v
e
l
y
p
r
e
f
e
r
s
c
l
i
n
i
ca
l
s
a
f
et
y
o
v
e
r
s
t
an
d
a
r
d
i
z
a
t
i
o
n
p
r
e
s
e
n
t
i
n
g
a
c
a
li
b
r
a
t
e
d
p
r
o
b
a
b
i
l
i
t
y
a
s
a
d
is
c
r
e
t
e
tr
i
a
g
e
t
a
g
r
a
t
h
e
r
t
h
a
n
a
s
a
r
a
w
s
c
o
r
e
f
o
l
l
o
w
s
t
h
e
g
u
i
d
a
n
c
e
o
n
c
l
i
n
i
c
ia
n
-
f
a
c
i
n
g
A
I
o
u
tp
u
t
g
i
v
e
n
b
y
[
8
]
,
[
3
]
.
T
ab
le
6
.
Fo
u
r
-
lev
el
co
n
f
id
e
n
c
e
-
b
ased
r
is
k
s
co
r
e
an
d
clin
ical
ac
tio
n
R
i
s
k
l
e
v
e
l
P
r
o
b
a
b
i
l
i
t
y
r
a
n
g
e
C
l
i
n
i
c
a
l
a
c
t
i
o
n
Lo
w
p
<
0
.
2
5
R
o
u
t
i
n
e
f
o
l
l
o
w
-
u
p
;
n
o
i
mm
e
d
i
a
t
e
i
n
t
e
r
v
e
n
t
i
o
n
M
i
l
d
0
.
2
5
≤
p
<
0
.
5
0
S
c
h
e
d
u
l
e
a
d
d
i
t
i
o
n
a
l
i
m
a
g
i
n
g
w
i
t
h
i
n
4
w
e
e
k
s
M
o
d
e
r
a
t
e
0
.
5
0
≤
p
<
0
.
7
5
U
r
g
e
n
t
r
a
d
i
o
l
o
g
i
st
r
e
v
i
e
w
a
n
d
c
o
n
f
i
r
mat
o
r
y
M
R
I
H
i
g
h
p
≥
0
.
7
5
I
mm
e
d
i
a
t
e
s
p
e
c
i
a
l
i
st
c
o
n
s
u
l
t
a
t
i
o
n
a
n
d
t
r
e
a
t
me
n
t
p
l
a
n
n
i
n
g
4
.
9
.
Dis
cus
s
io
n
T
h
er
e
ar
e
a
f
ew
th
i
n
g
s
w
o
r
th
f
lag
g
in
g
.
T
h
e
m
o
r
e
p
o
wer
f
u
l
r
esid
u
al
an
d
d
en
s
e
f
am
ilies
(
R
esNet5
0
V2
,
Den
s
eNe
t1
2
1
,
VGG1
6
)
ar
e
th
e
to
p
p
er
f
o
r
m
e
r
s
o
n
r
aw
ac
c
u
r
ac
y
,
a
n
d
t
h
at
e
x
ac
tly
m
atch
es
th
e
b
r
o
ad
e
r
m
ed
ical
im
a
g
in
g
lit
er
atu
r
e.
Mo
b
ileNetV2
is
an
in
ter
esti
n
g
o
u
tlier
.
W
ith
ab
o
u
t
1
0
tim
es
f
ewe
r
p
ar
am
eter
s
,
it
p
er
m
its
a
9
7
.
9
3
%
ac
cu
r
ac
y
with
th
e
b
est
-
in
-
c
lass
ca
lib
r
atio
n
o
u
t
o
f
th
e
b
o
x
(
E
C
E
=
0
.
0
0
2
1
)
.
T
h
is
co
m
b
i
n
atio
n
is
im
p
o
r
tan
t
in
a
clin
ic
th
at
d
o
es
n
o
t
h
av
e
a
GPU
s
er
v
e
r
i
n
th
e
b
asem
en
t.
I
n
c
o
n
clu
s
io
n
,
d
o
n
o
t
s
elec
t
a
m
o
d
el
b
ased
o
n
ly
o
n
its
ac
cu
r
ac
y
.
T
h
e
d
ec
is
io
n
r
e
g
ar
d
in
g
ca
lib
r
atio
n
a
n
d
h
ar
d
war
e
b
u
d
g
et
is
a
s
in
g
le
o
n
e.
T
h
e
s
am
e
o
r
d
e
r
in
g
o
f
r
esid
u
al
an
d
d
en
s
ely
co
n
n
ec
ted
f
am
ilies
h
as
b
ee
n
o
b
s
er
v
e
d
in
th
e
m
ed
ical
-
im
ag
in
g
s
u
r
v
ey
s
o
f
[
4
]
,
[
5
]
.
R
o
b
u
s
tn
ess
to
im
ag
in
g
v
a
r
iab
ilit
y
was
ad
d
r
ess
ed
in
d
ir
ec
tly
v
ia
th
e
p
r
e
p
r
o
ce
s
s
in
g
an
d
au
g
m
en
tatio
n
p
ip
elin
e.
T
h
e
n
etwo
r
k
was
m
a
d
e
awa
r
e
o
f
a
v
ar
iety
o
f
in
ten
s
ity
s
ca
les
an
d
v
iew
a
n
g
les
d
u
e
to
t
h
e
g
r
ay
s
ca
le
to
R
GB
n
o
r
m
aliza
tio
n
,
m
o
d
el
-
s
p
ec
if
ic
p
r
ep
r
o
ce
s
s
_
in
p
u
t
tr
an
s
f
o
r
m
s
,
an
d
r
an
d
o
m
p
h
o
to
m
etr
ic
an
d
g
eo
m
etr
i
c
au
g
m
en
tatio
n
s
.
I
n
ess
en
ce
,
th
ese
au
g
m
en
tatio
n
s
m
ay
b
e
s
ee
n
as
a
f
o
r
m
o
f
d
o
m
ain
r
an
d
o
m
izatio
n
b
u
t
s
o
f
ter
.
T
o
co
n
d
u
ct
a
m
o
r
e
f
o
r
m
al
r
o
b
u
s
tn
ess
ev
alu
atio
n
u
n
d
e
r
Gau
s
s
ian
n
o
is
e,
b
lu
r
,
an
d
b
r
ig
h
tn
ess
s
h
if
t,
it
is
n
ec
ess
ar
y
to
r
e
-
ev
alu
ate
s
av
ed
m
o
d
el
weig
h
ts
;
h
o
wev
er
,
th
e
s
e
weig
h
ts
wer
e
n
o
t
s
av
ed
f
o
r
th
e
cu
r
r
en
t
r
u
n
,
s
o
th
is
will
b
e
ev
alu
ated
in
s
ec
tio
n
5
.
T
h
e
p
er
t
u
r
b
atio
n
s
p
ec
if
ic
atio
n
is
s
u
p
p
lied
(
Gau
s
s
ian
n
o
is
e
σ
∈
{0
.
0
2
,
0
.
0
5
,
0
.
1
0
},
Ga
u
s
s
ian
b
lu
r
σ
∈
{1
,
2
},
b
r
ig
h
t
n
ess
s
h
if
t ±
0
.
2
)
s
o
th
at
th
e
ex
p
er
im
e
n
tal
r
esu
lts
ca
n
b
e
r
ep
r
o
d
u
ce
d
.
T
h
e
f
r
am
ewo
r
k
is
b
est
u
n
d
e
r
s
to
o
d
as
a
co
-
p
ilo
t
o
f
th
e
r
ad
io
lo
g
is
t
r
ath
er
th
a
n
an
au
to
n
o
m
o
u
s
d
iag
n
o
s
tic
s
y
s
tem
f
r
o
m
a
d
e
p
l
o
y
m
en
t
p
e
r
s
p
ec
tiv
e,
an
d
ex
p
li
citly
n
o
t
as
a
p
r
o
g
n
o
s
tic
m
o
d
e
l.
T
h
e
p
r
o
jectio
n
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.