I
nte
rna
t
io
na
l J
o
urna
l o
f
E
lect
rica
l a
nd
Co
m
pu
t
er
E
ng
ineering
(
I
J
E
CE
)
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
,
p
p
.
1199
~
1
2
1
2
I
SS
N:
2088
-
8
7
0
8
,
DOI
: 1
0
.
1
1
5
9
1
/ijece.
v
1
6
i
3
.
pp
1
1
9
9
-
1
2
1
2
1199
J
o
ur
na
l ho
m
ep
a
g
e
:
h
ttp
:
//ij
ec
e.
ia
esco
r
e.
co
m
O
ptimi
zed
ResNe
t
-
5
0
f
ra
mewo
rk
f
o
r ma
mm
o
g
ra
m
-
ba
sed brea
st
ca
ncer c
la
ss
ificati
o
n:
a
com
pa
ra
tive eva
lua
tion wit
h
Efficien
tNe
t
-
B0
M
uh
a
m
m
a
d Sub
a
li
1
,
L
u
lu
M
a
wa
dd
a
h Wi
s
ud
a
wa
t
i
2
,
T
e
re
s
a
3
1
D
e
p
a
r
t
me
n
t
o
f
C
o
m
p
u
t
e
r
S
c
i
e
n
c
e
a
n
d
I
n
f
o
r
ma
t
i
o
n
T
e
c
h
n
o
l
o
g
y
,
C
e
n
d
e
k
i
a
A
b
d
i
t
a
ma
U
n
i
v
e
r
s
i
t
y
,
Ta
n
g
e
r
a
n
g
,
I
n
d
o
n
e
si
a
2
D
e
p
a
r
t
me
n
t
o
f
I
n
f
o
r
mat
i
c
s
,
G
u
n
a
d
a
r
ma
U
n
i
v
e
r
si
t
y
,
D
e
p
o
k
,
I
n
d
o
n
e
si
a
3
D
e
p
a
r
t
me
n
t
o
f
N
u
r
si
n
g
,
C
e
n
d
e
k
i
a
A
b
d
i
t
a
m
a
U
n
i
v
e
r
si
t
y
,
T
a
n
g
e
r
a
n
g
,
I
n
d
o
n
e
si
a
Art
icle
I
nfo
AB
S
T
RAC
T
A
r
ticle
his
to
r
y:
R
ec
eiv
ed
Au
g
2
6
,
2
0
2
5
R
ev
is
ed
J
an
2
4
,
2
0
2
6
Acc
ep
ted
Ma
r
1
6
,
2
0
2
6
Bre
a
st
c
a
n
c
e
r
re
m
a
in
s
o
n
e
o
f
t
h
e
m
o
st
p
re
v
a
len
t
m
a
li
g
n
a
n
c
ies
wo
rld
wi
d
e
,
u
n
d
e
rsc
o
ri
n
g
th
e
n
e
e
d
f
o
r
a
c
c
u
ra
te
a
n
d
re
li
a
b
le
m
a
m
m
o
g
ra
p
h
ic
in
terp
re
tatio
n
.
Co
m
p
u
ter
-
a
id
e
d
d
i
a
g
n
o
sis
(CAD
)
b
a
se
d
o
n
d
e
e
p
lea
rn
in
g
h
a
s
e
m
e
rg
e
d
a
s
a
p
ro
m
isi
n
g
a
p
p
r
o
a
c
h
t
o
imp
r
o
v
e
b
o
th
sc
re
e
n
in
g
p
e
r
fo
rm
a
n
c
e
a
n
d
d
ia
g
n
o
stic
c
o
n
siste
n
c
y
,
y
e
t
fa
irn
e
ss
-
d
riv
e
n
c
o
m
p
a
ris
o
n
s
b
e
twe
e
n
p
o
p
u
lar
c
o
n
v
o
l
u
ti
o
n
a
l
b
a
c
k
b
o
n
e
s
o
n
p
u
b
li
c
m
a
m
m
o
g
ra
m
b
e
n
c
h
m
a
rk
s
re
m
a
in
li
m
it
e
d
.
Th
is
stu
d
y
p
r
o
v
i
d
e
s
a
sta
ti
stica
ll
y
v
a
li
d
a
ted
,
fa
irn
e
ss
-
d
riv
e
n
c
o
m
p
a
riso
n
o
f
tw
o
wid
e
ly
u
se
d
c
o
n
v
o
lu
ti
o
n
a
l
n
e
u
ra
l
n
e
two
r
k
a
rc
h
it
e
c
tu
re
s,
Re
sN
e
t
-
5
0
a
n
d
Eff
icie
n
tNe
t
-
B0
,
fo
r
m
a
m
m
o
g
ra
m
-
b
a
se
d
b
re
a
st
c
a
n
c
e
r
c
las
sifica
ti
o
n
u
n
d
e
r
a
rig
o
ro
u
sly
c
o
n
tro
ll
e
d
,
c
li
n
ica
ll
y
m
o
ti
v
a
ted
p
ro
t
o
c
o
l.
Th
e
p
ro
p
o
se
d
“
o
p
t
imiz
e
d
Re
sN
e
t
-
5
0
”
fra
m
e
wo
rk
is
d
e
fi
n
e
d
b
y
p
a
t
ien
t
-
lev
e
l
stra
ti
fied
u
n
d
e
rsa
m
p
li
n
g
,
p
a
ired
5
-
fo
l
d
c
ro
ss
-
v
a
li
d
a
ti
o
n
wi
th
id
e
n
ti
c
a
l
p
a
rti
ti
o
n
s,
h
a
rm
o
n
ize
d
a
u
g
m
e
n
tatio
n
a
n
d
trai
n
in
g
c
o
n
fi
g
u
ra
ti
o
n
s,
a
n
d
d
u
a
l
sta
ti
stica
l
tes
ti
n
g
(
p
a
ired
t
-
te
sts
a
n
d
Wi
lc
o
x
o
n
sig
n
e
d
-
ra
n
k
tes
ts),
e
m
p
h
a
siz
in
g
m
e
th
o
d
o
lo
g
ica
l
ri
g
o
r
ra
th
e
r
th
a
n
a
rc
h
it
e
c
tu
ra
l
n
o
v
e
lt
y
.
Ac
ro
ss
M
IAS
a
n
d
CBI
S
-
DD
S
M
b
e
n
c
h
m
a
rk
s,
t
h
e
m
o
d
e
ls
d
e
m
o
n
stra
ted
c
o
m
p
lem
e
n
tary
stre
n
g
th
s,
with
Eff
icie
n
tNe
t
-
B0
e
x
c
e
ll
in
g
i
n
sc
re
e
n
in
g
-
o
rien
ted
tas
k
s
(n
o
rm
a
l
v
s.
a
b
n
o
r
m
a
l)
a
n
d
Re
sN
e
t
-
5
0
o
ffe
rin
g
m
o
re
ro
b
u
st
p
e
rfo
rm
a
n
c
e
fo
r
d
ia
g
n
o
stic
-
o
rie
n
ted
tas
k
s
(b
e
n
ig
n
v
s.
m
a
li
g
n
a
n
t).
Th
e
se
fin
d
i
n
g
s
h
ig
h
li
g
h
t
th
e
v
a
l
u
e
o
f
fa
irn
e
ss
-
d
riv
e
n
e
v
a
lu
a
ti
o
n
p
ro
t
o
c
o
l
s
in
CAD
re
se
a
rc
h
a
n
d
su
p
p
o
rt
t
h
e
fe
a
sib
il
i
ty
o
f
i
n
teg
ra
ti
n
g
l
ig
h
twe
ig
h
t
c
o
n
v
o
l
u
ti
o
n
a
l
n
e
u
ra
l
n
e
two
r
k
s
(CNN
s
)
in
t
o
ti
e
re
d
c
li
n
ica
l
wo
rk
fl
o
ws
,
wh
e
re
d
iffere
n
t
b
a
c
k
b
o
n
e
s
a
re
stra
teg
ica
ll
y
d
e
p
lo
y
e
d
fo
r
i
n
it
ial
sc
re
e
n
in
g
a
n
d
c
o
n
firma
to
ry
a
ss
e
ss
m
e
n
t.
K
ey
w
o
r
d
s
:
B
r
ea
s
t c
an
ce
r
class
if
icatio
n
C
o
m
p
u
ter
-
aid
ed
d
iag
n
o
s
is
Dee
p
lear
n
in
g
E
f
f
icien
tNet
-
B0
Ma
m
m
o
g
r
a
p
h
y
R
esNet
-
50
T
h
is i
s
a
n
o
p
e
n
a
c
c
e
ss
a
rticle
u
n
d
e
r th
e
CC B
Y
-
SA
li
c
e
n
se
.
C
o
r
r
e
s
p
o
nd
ing
A
uth
o
r
:
Mu
h
am
m
ad
Su
b
ali
Dep
ar
tm
en
t o
f
C
o
m
p
u
ter
Scie
n
ce
an
d
I
n
f
o
r
m
atio
n
T
ec
h
n
o
lo
g
y
,
C
en
d
ek
ia
A
b
d
itam
a
Un
iv
e
r
s
ity
I
s
lam
ic
R
ay
a
St.,
Kela
p
a
Du
a,
T
an
g
er
a
n
g
,
B
an
ten
,
I
n
d
o
n
esia
E
m
ail: su
b
ali@
u
ca
.
ac
.
id
1.
I
NT
RO
D
UCT
I
O
N
B
r
ea
s
t
ca
n
ce
r
r
em
ain
s
o
n
e
o
f
th
e
m
o
s
t
p
r
ev
ale
n
t
an
d
lif
e
-
th
r
ea
ten
in
g
m
alig
n
a
n
cies
wo
r
ld
wid
e,
im
p
o
s
in
g
a
s
u
b
s
tan
tial
b
u
r
d
en
o
f
in
cid
en
ce
an
d
m
o
r
tality
a
m
o
n
g
wo
m
en
[
1
]
.
T
h
is
b
u
r
d
e
n
ca
lls
f
o
r
r
eso
u
r
ce
-
s
tr
atif
ied
s
tr
ateg
ies
to
s
tr
en
g
th
en
ea
r
ly
d
etec
tio
n
a
n
d
ca
r
e
p
at
h
way
s
ac
r
o
s
s
d
iv
er
s
e
h
ea
lth
-
s
y
s
tem
co
n
tex
ts
[
2
]
,
with
ad
d
itio
n
al
g
lo
b
al
-
b
u
r
d
en
ev
id
e
n
ce
f
u
r
th
e
r
u
n
d
er
s
co
r
in
g
th
e
n
ee
d
f
o
r
tim
el
y
in
ter
v
e
n
tio
n
[
3
]
.
Ma
m
m
o
g
r
a
p
h
y
is
th
e
m
o
s
t
wid
ely
ad
o
p
te
d
s
cr
ee
n
in
g
to
o
l
an
d
is
s
tan
d
ar
d
ized
th
r
o
u
g
h
th
e
B
I
-
R
ADS
®
r
ep
o
r
tin
g
f
r
am
ew
o
r
k
; h
o
wev
e
r
,
in
ter
p
r
etatio
n
v
a
r
iab
ilit
y
an
d
th
e
s
u
b
tlety
o
f
ce
r
tain
lesi
o
n
p
atter
n
s
ca
n
lea
d
to
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
9
9
-
1
2
1
2
1200
f
alse
p
o
s
itiv
es
an
d
f
alse
n
eg
ativ
es
with
d
o
wn
s
tr
ea
m
clin
i
ca
l
an
d
ec
o
n
o
m
ic
co
n
s
eq
u
en
ce
s
[
4
]
.
Po
p
u
latio
n
-
lev
el
esti
m
ates
al
s
o
co
n
f
ir
m
b
r
ea
s
t
ca
n
ce
r
’
s
lar
g
e
co
n
tr
ib
u
tio
n
to
f
e
m
ale
ca
n
ce
r
in
ci
d
en
ce
an
d
d
ea
th
s
,
r
ein
f
o
r
cin
g
th
e
u
r
g
en
cy
o
f
e
f
f
ec
tiv
e
s
cr
ee
n
in
g
p
r
o
g
r
am
s
[
5
]
.
R
ec
en
t
ev
al
u
atio
n
s
s
h
o
w
th
at
d
ee
p
lear
n
in
g
–
b
ased
co
m
p
u
ter
-
aid
e
d
d
ia
g
n
o
s
is
(
C
AD)
ca
n
im
p
r
o
v
e
m
am
m
o
g
r
a
p
h
ic
ass
ess
m
en
t
an
d
f
u
n
ctio
n
as
a
co
m
p
lem
en
tar
y
aid
in
clin
ical
wo
r
k
f
lo
ws
[
6
]
,
[
7
]
.
B
r
o
ad
e
r
d
ee
p
-
lear
n
i
n
g
liter
atu
r
e
h
ig
h
li
g
h
ts
th
e
ca
p
ac
ity
o
f
n
eu
r
al
n
etwo
r
k
s
to
lear
n
m
u
lti
-
lev
el
r
e
p
r
esen
tatio
n
s
a
n
d
to
s
u
p
p
o
r
t
d
is
co
v
er
y
ac
r
o
s
s
h
eter
o
g
e
n
eo
u
s
b
io
m
ed
ical
d
ata
[
8
]
,
[
9
]
.
T
r
an
s
f
er
lear
n
in
g
,
in
p
ar
ticu
lar
,
f
a
cilitates
r
o
b
u
s
t
p
er
f
o
r
m
an
ce
in
m
ed
ical
im
ag
in
g
d
esp
ite
lim
ited
an
n
o
tated
d
ata
s
ets
[
1
0
]
.
W
ith
in
th
is
co
n
tex
t,
C
AD
lev
er
ag
in
g
m
ac
h
in
e
lear
n
in
g
(
ML
)
an
d
d
ee
p
lear
n
in
g
(
D
L
)
en
ab
les
au
to
m
ated
e
x
tr
ac
tio
n
o
f
h
ier
a
r
ch
ical,
d
is
cr
im
in
ativ
e
f
ea
tu
r
es
th
at
o
f
ten
s
u
r
p
ass
h
an
d
c
r
a
f
ted
d
escr
ip
to
r
s
in
m
am
m
o
g
r
a
p
h
y
an
d
r
elate
d
m
o
d
alities
[
1
1
]
,
wh
ile
p
r
etr
ai
n
e
d
co
n
v
o
lu
tio
n
al
n
eu
r
al
n
etwo
r
k
s
(
C
NNs)
tailo
r
ed
to
m
am
m
o
g
r
ap
h
y
co
n
tin
u
e
t
o
m
atu
r
e
to
war
d
s
tate
-
of
-
th
e
-
a
r
t
p
r
ac
tice
[
1
2
]
.
E
f
f
icien
tNet
v
ar
ian
ts
h
av
e
b
ee
n
ex
p
lo
r
ed
f
o
r
b
r
ea
s
t
-
im
ag
in
g
t
ask
s
with
p
r
o
m
is
in
g
ac
cu
r
ac
y
an
d
s
ca
lab
ilit
y
ch
ar
ac
ter
is
tics
[
1
3
]
,
[
1
4
]
.
I
n
th
is
s
tu
d
y
,
we
th
er
ef
o
r
e
f
o
cu
s
o
n
o
p
tim
izin
g
R
esNet
-
5
0
f
o
r
m
a
m
m
o
g
r
am
-
b
ased
b
r
ea
s
t
-
ca
n
ce
r
class
if
icatio
n
an
d
p
r
o
v
id
e
a
d
ir
ec
t c
o
m
p
ar
is
o
n
to
E
f
f
icien
tNet
-
B
0
.
At
th
e
b
ac
k
b
o
n
e
le
v
el,
r
esid
u
al
co
n
n
ec
tio
n
s
i
n
R
esNet
s
tab
ilize
d
ee
p
o
p
tim
izatio
n
a
n
d
e
n
ab
le
ef
f
ec
tiv
e
f
ea
tu
r
e
r
eu
s
e,
an
d
R
esNet
-
5
0
h
as
d
em
o
n
s
tr
ated
r
o
b
u
s
t
p
er
f
o
r
m
a
n
ce
in
b
r
ea
s
t
-
ca
n
ce
r
im
ag
in
g
ap
p
licatio
n
s
[
1
5
]
–
[
1
7
]
.
B
ey
o
n
d
ar
c
h
itectu
r
es,
f
ea
tu
r
e
en
g
in
ee
r
in
g
a
n
d
b
aselin
e
m
o
d
ellin
g
r
em
ai
n
r
elev
a
n
t.
Prio
r
wo
r
k
s
p
an
s
h
an
d
cr
a
f
te
d
tex
tu
r
e
d
escr
ip
to
r
s
an
d
C
N
N
-
b
ased
r
ep
r
esen
tatio
n
s
,
ea
c
h
s
h
o
win
g
tan
g
ib
le
g
ain
s
in
d
is
cr
im
in
ab
ilit
y
ac
r
o
s
s
d
atasets
an
d
task
s
[
1
8
]
–
[
2
0
]
.
Dim
en
s
io
n
ality
r
ed
u
ctio
n
a
n
d
f
ea
t
u
r
e
s
elec
tio
n
(
e.
g
.
,
L
ASSO)
h
av
e
also
b
ee
n
r
ep
o
r
ted
t
o
im
p
r
o
v
e
p
e
r
f
o
r
m
a
n
ce
an
d
c
o
m
p
u
tatio
n
al
ef
f
icie
n
cy
in
b
r
ea
s
t
-
ca
n
ce
r
class
if
icatio
n
p
ip
elin
es
[
2
1
]
.
C
las
s
ical
ML
b
aselin
es
—
s
u
ch
as
s
u
p
p
o
r
t
v
ec
to
r
m
ac
h
in
es
an
d
en
s
em
b
le
lear
n
er
s
—
h
av
e
ac
h
iev
ed
s
tr
o
n
g
r
esu
lts
in
s
ev
er
al
s
tu
d
ies
an
d
d
atasets
,
alth
o
u
g
h
th
eir
r
elian
ce
o
n
en
g
in
ee
r
ed
f
ea
tu
r
es
ca
n
lim
it
g
en
er
aliza
ti
o
n
ac
r
o
s
s
h
eter
o
g
e
n
eo
u
s
c
o
h
o
r
ts
[
2
2
]
–
[
2
5
]
.
A
b
r
o
ad
er
r
ev
iew
o
f
ML
/DL
f
o
r
m
ed
ical
im
ag
e
an
aly
s
is
f
u
r
t
h
e
r
em
p
h
asizes
th
e
b
e
n
ef
its
o
f
l
ea
r
n
ed
r
ep
r
esen
tatio
n
s
f
o
r
en
d
-
to
-
en
d
d
iag
n
o
s
tic
m
o
d
ellin
g
[
2
6
]
,
an
d
r
ec
e
n
t
m
am
m
o
g
r
a
p
h
y
ap
p
licatio
n
s
o
n
p
u
b
lic
d
atasets
co
n
tin
u
e
to
s
h
o
w
d
etec
tio
n
im
p
r
o
v
em
e
n
ts
u
n
d
er
p
r
ac
tical
s
ettin
g
s
[
2
7
]
.
C
o
m
p
ar
ativ
e
s
tu
d
ies
o
f
ML
al
g
o
r
ith
m
s
also
p
r
o
v
id
e
p
er
f
o
r
m
an
ce
co
n
tex
t a
cr
o
s
s
v
ar
ie
d
s
ce
n
ar
io
s
an
d
ev
alu
atio
n
p
r
o
to
co
ls
[
2
8
]
.
Desp
ite
th
ese
ad
v
an
ce
s
,
class
im
b
alan
ce
—
wh
er
e
b
en
i
g
n
c
ases
o
f
ten
o
u
tn
u
m
b
er
m
alig
n
an
t
o
n
es
—
r
em
ain
s
a
ch
allen
g
e
t
h
at
ca
n
s
h
if
t
d
ec
is
io
n
b
o
u
n
d
ar
ies
an
d
r
ed
u
ce
s
en
s
itiv
ity
to
m
in
o
r
ity
c
lass
es
in
s
cr
ee
n
in
g
co
n
tex
ts
.
R
ec
en
t
p
ip
elin
es
h
av
e
th
er
ef
o
r
e
in
c
o
r
p
o
r
ated
d
if
f
er
en
t
b
alan
cin
g
s
tr
ateg
ies,
s
u
ch
as
s
tr
atif
ied
u
n
d
er
s
am
p
lin
g
,
class
-
weig
h
ted
lo
s
s
es,
an
d
s
y
n
th
etic
o
v
e
r
s
am
p
lin
g
(
e.
g
.
,
SMOT
E
)
,
with
d
em
o
n
s
tr
ated
ef
f
ec
tiv
en
ess
in
b
o
th
R
esNet
-
50
-
b
ased
c
o
n
f
ig
u
r
atio
n
s
an
d
co
n
tem
p
o
r
ar
y
d
ee
p
-
lear
n
in
g
f
r
am
ewo
r
k
s
f
o
r
b
r
ea
s
t
-
ca
n
ce
r
d
etec
tio
n
an
d
cl
ass
if
icatio
n
[
2
9
]
,
[
3
0
]
.
I
n
th
is
s
tu
d
y
,
we
em
p
lo
y
s
tr
atif
ied
u
n
d
er
s
am
p
lin
g
at
t
h
e
p
atien
t
lev
el
to
m
itig
ate
m
aj
o
r
ity
-
class
b
ias
wh
ile
en
s
u
r
in
g
f
air
r
ep
r
esen
tatio
n
ac
r
o
s
s
tr
ai
n
in
g
an
d
v
alid
atio
n
f
o
ld
s
.
T
o
r
ed
u
ce
p
o
ten
tial
in
f
o
r
m
atio
n
lo
s
s
,
u
n
d
e
r
s
am
p
lin
g
is
co
m
b
in
ed
with
s
tan
d
ar
d
ize
d
au
g
m
en
tatio
n
an
d
r
ig
o
r
o
u
s
ev
alu
atio
n
p
r
o
t
o
co
ls
.
I
n
th
is
s
tu
d
y
,
we
d
esig
n
an
d
v
alid
ate
an
o
p
tim
ized
R
esNet
-
5
0
f
r
am
ewo
r
k
f
o
r
m
am
m
o
g
r
am
-
b
ased
b
r
ea
s
t
-
ca
n
ce
r
class
if
icatio
n
,
with
a
d
ir
ec
t
co
m
p
ar
is
o
n
to
E
f
f
icien
tNet
-
B0
.
Ou
r
m
eth
o
d
o
lo
g
y
co
m
p
r
is
es:
i)
R
ig
o
r
o
u
s
p
r
ep
r
o
c
ess
in
g
an
d
au
g
m
en
tatio
n
tailo
r
ed
to
m
a
m
m
o
g
r
a
p
h
ic
v
a
r
iab
ilit
y
;
ii)
I
n
teg
r
atio
n
o
f
p
atien
t
-
lev
el
s
tr
atif
ied
u
n
d
er
s
am
p
lin
g
with
a
f
ix
ed
r
an
d
o
m
s
ee
d
to
b
alan
ce
class
es
r
ep
r
o
d
u
cib
ly
ac
r
o
s
s
f
o
ld
s
;
iii)
Stan
d
ar
d
ized
tr
a
n
s
f
er
lear
n
in
g
with
f
r
o
ze
n
b
ac
k
b
o
n
es
an
d
an
id
en
tical
m
in
im
al
class
if
ier
h
ea
d
.
Her
e
,
“o
p
tim
ized
”
h
ig
h
lig
h
ts
m
eth
o
d
o
lo
g
ical
im
p
r
o
v
em
en
ts
(
b
ala
n
ce
d
s
p
litt
in
g
,
co
n
tr
o
lled
class
if
ier
s
tr
u
ctu
r
e,
an
d
s
y
s
tem
atic
h
y
p
er
p
ar
am
eter
tu
n
in
g
)
d
esig
n
e
d
to
en
s
u
r
e
r
ep
r
o
d
u
cib
ilit
y
an
d
clin
ical
r
ele
v
an
ce
,
r
ath
e
r
th
an
p
r
o
p
o
s
in
g
a
n
o
v
el
n
etwo
r
k
ar
ch
itectu
r
e
;
an
d
iv
)
C
o
m
p
r
eh
en
s
iv
e
b
en
ch
m
a
r
k
in
g
ag
ai
n
s
t
E
f
f
icien
tNet
-
B
0
ac
r
o
s
s
ac
cu
r
ac
y
,
s
en
s
itiv
ity
,
s
p
ec
if
icity
,
F1
-
s
co
r
e,
a
n
d
R
OC
–
AUC
.
T
o
en
s
u
r
e
r
o
b
u
s
tn
es
s
,
p
er
f
o
r
m
a
n
ce
was
r
ep
o
r
ted
as
m
ea
n
±
s
tan
d
ar
d
d
ev
iatio
n
(
SD)
ac
r
o
s
s
cr
o
s
s
-
v
alid
atio
n
f
o
ld
s
,
9
5
%
co
n
f
id
en
ce
in
ter
v
als
(
C
I
)
f
o
r
R
OC
–
AU
C
es
tim
ated
u
s
in
g
th
e
t
-
d
is
tr
ib
u
tio
n
ac
r
o
s
s
f
o
ld
s
,
an
d
s
tatis
tical
v
alid
atio
n
u
s
in
g
p
air
ed
t
-
test
s
an
d
W
ilco
x
o
n
s
ig
n
ed
-
r
a
n
k
test
s
.
I
n
s
u
m
m
a
r
y
,
th
is
s
tu
d
y
m
a
k
e
s
th
r
ee
m
ai
n
c
o
n
tr
ib
u
tio
n
s
.
F
ir
s
t
,
it
p
r
o
p
o
s
es
a
r
ep
licab
le,
p
r
ac
tice
-
m
in
d
ed
o
p
tim
izatio
n
p
r
o
to
co
l
f
o
r
R
esNet
-
5
0
tailo
r
ed
to
m
a
m
m
o
g
r
a
p
h
y
,
r
e
f
lectin
g
b
o
th
te
ch
n
ical
an
d
clin
ical
co
n
s
tr
ain
ts
.
Seco
n
d
,
it
p
r
o
v
id
es
h
ea
d
-
to
-
h
ea
d
co
m
p
ar
ativ
e
in
s
ig
h
ts
b
etwe
en
R
e
s
Net
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
with
s
tatis
t
ical
v
alid
atio
n
ac
r
o
s
s
p
air
ed
cr
o
s
s
-
v
alid
atio
n
f
o
ld
s
u
n
d
er
a
f
air
n
ess
-
d
r
iv
e
n
e
x
p
er
im
en
tal
d
esig
n
.
T
h
ir
d
,
it
em
p
h
asizes
clin
ically
m
ea
n
in
g
f
u
l
ev
alu
atio
n
—
in
clu
d
in
g
p
atien
t
-
lev
el
v
alid
atio
n
,
s
en
s
itiv
ity
-
f
o
cu
s
ed
o
p
er
atin
g
p
o
in
ts
,
an
d
r
o
b
u
s
t
co
n
f
id
en
ce
-
in
ter
v
al
esti
m
atio
n
—
to
war
d
r
ea
l
-
wo
r
l
d
C
AD
in
teg
r
atio
n
.
Un
lik
e
p
r
io
r
in
cr
e
m
en
tal
co
m
p
ar
is
o
n
s
,
th
e
n
o
v
elty
o
f
th
is
wo
r
k
lie
s
in
th
is
f
air
n
ess
-
d
r
iv
en
ex
p
er
im
en
tal
d
esig
n
an
d
clin
ically
o
r
ien
ted
e
v
alu
atio
n
p
r
o
to
co
l
r
ath
er
th
a
n
in
p
r
o
p
o
s
in
g
a
n
ew
n
eu
r
al
a
r
ch
itectu
r
e.
T
h
r
o
u
g
h
th
ese
co
n
t
r
ib
u
tio
n
s
,
th
e
s
tu
d
y
aim
s
to
b
r
id
g
e
th
e
g
ap
b
etwe
en
alg
o
r
ith
m
ic
ad
v
an
ce
s
an
d
clin
ical
d
ep
lo
y
m
en
t,
en
s
u
r
i
n
g
th
at
C
AD
s
y
s
tem
s
n
o
t
o
n
ly
ac
h
iev
e
h
ig
h
ac
cu
r
ac
y
b
u
t
also
d
em
o
n
s
tr
ate
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
ed
R
es
N
et
-
5
0
fr
a
mew
o
r
k
fo
r
ma
mmo
g
r
a
m
-
b
a
s
ed
b
r
ea
s
t c
a
n
ce
r
…
(
Mu
h
a
mma
d
S
u
b
a
li
)
1201
r
eliab
ilit
y
,
in
ter
p
r
etab
ilit
y
,
a
n
d
ef
f
icien
c
y
i
n
b
r
ea
s
t
ca
n
ce
r
s
cr
ee
n
in
g
.
T
h
e
r
em
ain
d
er
o
f
th
i
s
p
ap
er
is
o
r
g
an
ized
as
f
o
llo
ws.
Sectio
n
2
p
r
esen
ts
th
e
m
ater
ials
an
d
m
eth
o
d
s
,
s
ec
tio
n
3
r
ep
o
r
ts
th
e
ex
p
er
i
m
en
tal
r
esu
lts
an
d
d
is
cu
s
s
es th
e
f
in
d
in
g
s
,
an
d
s
ec
tio
n
4
co
n
clu
d
es th
e
s
tu
d
y
.
2.
M
E
T
H
O
D
T
h
e
o
v
er
all
wo
r
k
f
lo
w
o
f
th
e
p
r
o
p
o
s
ed
f
r
am
ewo
r
k
is
illu
s
tr
ated
in
Fig
u
r
e
1
.
T
h
e
s
y
s
tem
b
e
g
in
s
with
th
e
ac
q
u
is
itio
n
o
f
m
a
m
m
o
g
r
a
m
im
ag
es
o
b
tain
ed
f
r
o
m
two
p
u
b
lic
b
en
c
h
m
ar
k
d
atasets
.
T
wo
ex
p
er
im
e
n
tal
s
ce
n
ar
io
s
ar
e
c
o
n
s
id
er
ed
:
i)
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
class
if
icatio
n
u
s
in
g
th
e
MI
AS
d
ataset,
an
d
ii)
b
en
ig
n
v
s
.
m
alig
n
an
t
class
if
icatio
n
u
s
in
g
th
e
C
B
I
S
-
DDSM
d
ataset.
All
im
ag
es
ar
e
s
u
b
jecte
d
to
p
r
ep
r
o
ce
s
s
in
g
an
d
r
e
g
io
n
o
f
in
ter
est
(
R
OI
)
e
x
tr
ac
tio
n
.
T
h
is
s
tep
in
clu
d
es
g
r
ay
s
ca
le
co
n
v
er
s
io
n
,
n
o
is
e
s
u
p
p
r
ess
io
n
w
ith
m
ed
ian
f
ilter
in
g
,
b
r
ea
s
t
b
o
u
n
d
a
r
y
d
etec
tio
n
,
r
e
m
o
v
al
o
f
n
o
n
-
b
r
ea
s
t
s
tr
u
ctu
r
es
(
e.
g
.
,
p
ec
to
r
al
m
u
s
cle)
,
an
d
cr
o
p
p
in
g
o
f
r
elev
a
n
t
r
eg
io
n
s
o
f
in
ter
est.
Fo
r
th
e
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
s
ce
n
ar
io
,
th
e
R
OI
co
r
r
esp
o
n
d
s
to
th
e
wh
o
le
b
r
ea
s
t a
r
ea
,
wh
ile
f
o
r
th
e
b
en
ig
n
v
s
.
m
alig
n
a
n
t scen
ar
io
,
ca
n
d
id
ate
m
ass
r
eg
io
n
s
ar
e
s
eg
m
en
ted
a
n
d
cr
o
p
p
e
d
.
Fo
llo
win
g
R
OI
ex
tr
ac
tio
n
,
all
im
ag
es
ar
e
s
tan
d
ar
d
ized
f
o
r
C
NN
in
p
u
t.
Sp
ec
if
ically
,
ea
ch
R
OI
is
r
esized
to
2
2
4
×
2
2
4
p
i
x
els,
c
o
n
v
er
ted
to
th
r
ee
-
ch
an
n
el
R
GB
,
an
d
n
o
r
m
alize
d
to
th
e
[
0
,
1
]
r
an
g
e.
Data
au
g
m
en
tatio
n
is
th
en
a
p
p
lied
d
u
r
in
g
t
r
ain
in
g
to
in
cr
ea
s
e
v
ar
iab
ilit
y
an
d
r
ed
u
ce
o
v
er
f
itti
n
g
.
T
o
a
d
d
r
ess
th
e
is
s
u
e
o
f
class
im
b
alan
ce
,
we
ap
p
lied
p
atien
t
-
lev
el
s
tr
atif
i
ed
u
n
d
er
s
am
p
lin
g
with
a
f
ix
ed
r
an
d
o
m
s
ee
d
to
g
en
er
ate
r
e
p
r
o
d
u
cib
le
b
alan
ce
d
s
p
lits
.
A
p
air
ed
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
d
esig
n
was
th
en
e
m
p
lo
y
ed
,
en
s
u
r
in
g
th
at
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
wer
e
tr
ain
ed
an
d
e
v
alu
ated
o
n
id
e
n
tical
d
ataset
p
ar
titi
o
n
s
f
o
r
a
f
air
co
m
p
ar
is
o
n
.
I
n
th
e
f
ea
tu
r
e
ex
t
r
ac
tio
n
s
tag
e,
two
d
ee
p
C
NN
b
ac
k
b
o
n
es
wer
e
co
n
s
id
er
ed
:
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
,
b
o
th
p
r
etr
ai
n
ed
-
o
n
I
m
ag
eNe
t.
R
esNet
-
5
0
em
p
lo
y
s
r
esid
u
al
co
n
n
ec
tio
n
s
to
en
ab
le
s
tab
le
tr
ain
in
g
o
f
d
ee
p
e
r
ar
ch
itectu
r
e
s
,
wh
ile
E
f
f
icie
n
tNet
-
B
0
lev
er
ag
es
co
m
p
o
u
n
d
s
ca
lin
g
to
b
al
an
ce
d
e
p
th
,
wid
th
,
an
d
r
eso
lu
tio
n
.
Fo
r
b
o
th
b
ac
k
b
o
n
es,
th
e
o
r
ig
in
al
f
u
lly
c
o
n
n
ec
ted
class
if
icatio
n
lay
er
s
wer
e
r
em
o
v
ed
a
n
d
r
ep
lace
d
with
a
n
ew
d
en
s
e
lay
er
f
o
llo
wed
b
y
a
So
f
tMa
x
f
u
n
ctio
n
to
ad
a
p
t to
th
e
b
in
ar
y
class
if
icatio
n
task
s
.
T
h
e
class
if
icatio
n
s
tag
e
o
u
tp
u
ts
p
r
o
b
ab
ilit
y
esti
m
ates
f
o
r
ea
ch
class
,
ag
g
r
eg
ated
ac
r
o
s
s
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
.
Mo
d
el
p
er
f
o
r
m
a
n
ce
was
q
u
an
tifie
d
u
s
in
g
m
u
ltip
le
m
etr
ics,
in
clu
d
in
g
ac
cu
r
ac
y
,
s
en
s
itiv
ity
,
s
p
ec
if
icity
,
p
r
ec
is
io
n
,
F1
-
s
co
r
e,
an
d
R
OC
–
AUC.
R
esu
lt
s
ar
e
r
ep
o
r
ted
as
m
ea
n
±
s
tan
d
a
r
d
d
ev
iatio
n
ac
r
o
s
s
f
o
ld
s
,
with
9
5
%
co
n
f
id
en
ce
in
ter
v
als
f
o
r
R
OC
–
AU
C
esti
m
ated
u
s
in
g
th
e
t
-
d
is
tr
ib
u
tio
n
ac
r
o
s
s
f
o
ld
s
.
T
o
f
u
r
th
er
en
s
u
r
e
r
o
b
u
s
tn
ess
,
s
tatis
tical
s
ig
n
if
ican
ce
o
f
p
er
f
o
r
m
an
ce
d
i
f
f
er
en
ce
s
b
etwe
e
n
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
was
ass
ess
ed
u
s
in
g
p
air
e
d
Stu
d
e
n
t’
s
t
-
test
s
an
d
W
ilco
x
o
n
s
ig
n
e
d
-
r
an
k
test
s
.
T
h
is
ar
ch
itectu
r
e
en
ab
les
a
r
ig
o
r
o
u
s
,
r
e
p
r
o
d
u
ci
b
le,
an
d
s
tatis
tically
v
alid
ated
co
m
p
a
r
is
o
n
b
etwe
en
two
r
ep
r
esen
tativ
e
C
NN
f
am
ilies
ac
r
o
s
s
b
o
th
s
cr
ee
n
in
g
-
o
r
ien
ted
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
a
l)
an
d
d
iag
n
o
s
tic
-
o
r
ien
ted
(
b
e
n
ig
n
v
s
.
m
alig
n
an
t)
m
am
m
o
g
r
a
m
class
if
icatio
n
task
s
.
As s
u
m
m
ar
ized
in
Fig
u
r
e
1
,
th
is
wo
r
k
f
lo
w
ex
p
licitly
co
m
b
in
es p
atien
t
-
lev
el
s
p
litt
in
g
,
p
air
ed
cr
o
s
s
-
v
alid
atio
n
,
an
d
h
a
r
m
o
n
ize
d
tr
ain
in
g
c
o
n
f
ig
u
r
atio
n
s
to
c
o
n
tr
o
l
c
o
m
m
o
n
s
o
u
r
ce
s
o
f
b
ias,
wh
ich
u
n
d
e
r
p
in
s
th
e
f
air
n
ess
-
d
r
iv
en
p
er
f
o
r
m
an
ce
c
o
m
p
ar
is
o
n
s
r
ep
o
r
ted
in
th
e
r
esu
lts
.
Fig
u
r
e
1
.
W
o
r
k
f
lo
w
o
f
th
e
p
r
o
p
o
s
ed
m
am
m
o
g
r
am
class
if
icat
io
n
f
r
am
ew
o
r
k
u
s
in
g
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B0
2
.
1
.
Da
t
a
s
et
T
h
is
s
tu
d
y
e
m
p
lo
y
s
two
wi
d
ely
u
s
ed
p
u
b
lic
m
a
m
m
o
g
r
a
m
d
atasets
.
T
h
e
MI
AS
d
ata
b
ase
[
3
1
]
p
r
o
v
id
es
l
o
w
-
r
eso
lu
tio
n
m
am
m
o
g
r
am
s
a
n
d
was
u
s
ed
f
o
r
t
h
e
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
clas
s
if
icatio
n
s
ce
n
ar
io
,
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
9
9
-
1
2
1
2
1202
co
m
p
r
is
in
g
1
0
7
n
o
r
m
al
a
n
d
8
9
ab
n
o
r
m
al
ca
s
es
(
3
9
b
en
i
g
n
,
5
0
m
alig
n
an
t)
.
I
n
co
n
tr
ast
,
th
e
C
B
I
S
-
DDSM
d
ataset
[
3
2
]
o
f
f
er
s
h
ig
h
-
r
eso
l
u
tio
n
m
am
m
o
g
r
am
s
an
d
was
u
s
ed
f
o
r
th
e
b
en
ig
n
v
s
.
m
ali
g
n
an
t
class
if
icatio
n
s
ce
n
ar
io
,
co
n
tain
in
g
4
8
1
b
e
n
ig
n
a
n
d
5
2
7
m
alig
n
an
t
ca
s
es.
T
h
ese
d
atasets
r
ep
r
ese
n
t
co
m
p
lem
en
tar
y
b
en
ch
m
ar
k
s
,
s
u
p
p
o
r
tin
g
b
o
th
s
cr
ee
n
in
g
-
o
r
ien
ted
a
n
d
d
iag
n
o
s
tic
-
o
r
ien
ted
ev
al
u
atio
n
task
s
.
2
.
2
.
P
re
pro
ce
s
s
ing
a
nd
RO
I
e
x
t
ra
ct
io
n
T
o
en
s
u
r
e
co
n
s
is
ten
cy
an
d
co
m
p
ar
ab
ilit
y
ac
r
o
s
s
d
atasets
,
w
e
ad
o
p
ted
a
s
tan
d
ar
d
ized
p
r
e
-
p
r
o
ce
s
s
in
g
p
r
o
to
co
l
ad
ap
ted
f
r
o
m
p
r
i
o
r
m
am
m
o
g
r
a
p
h
y
C
AD
s
tu
d
ies
[
1
8
]
,
[
1
9
]
.
As
illu
s
tr
ated
in
Fig
u
r
e
2
,
all
m
am
m
o
g
r
a
m
im
ag
es
wer
e
f
ir
s
t
co
n
v
er
ted
to
8
-
b
it
g
r
a
y
s
ca
le,
h
ar
m
o
n
izin
g
in
ten
s
ity
s
ca
les
b
etwe
en
f
o
r
m
ats.
No
is
e
s
u
p
p
r
ess
io
n
was
p
er
f
o
r
m
ed
u
s
in
g
a
3
×3
m
ed
ian
f
ilter
,
wh
ich
ef
f
ec
tiv
ely
r
e
d
u
ce
s
s
alt
-
an
d
-
p
e
p
p
er
n
o
is
e
with
o
u
t
b
lu
r
r
i
n
g
lesi
o
n
b
o
u
n
d
ar
ies.
Fo
r
b
r
ea
s
t
r
eg
io
n
s
eg
m
e
n
tatio
n
,
g
lo
b
al
th
r
esh
o
ld
i
n
g
(
i
n
ten
s
ity
lev
el
f
ix
ed
at
0
.
0
8
af
ter
m
ed
ian
f
ilter
in
g
)
was
ap
p
lied
to
s
ep
ar
ate
f
o
r
e
g
r
o
u
n
d
tis
s
u
e
f
r
o
m
th
e
b
ac
k
g
r
o
u
n
d
.
T
h
e
r
esu
ltin
g
b
in
ar
y
m
ask
was
r
ef
in
ed
th
r
o
u
g
h
h
o
le
f
illi
n
g
,
r
em
o
v
al
o
f
s
m
all
s
p
u
r
io
u
s
co
m
p
o
n
en
ts
,
an
d
m
o
r
p
h
o
lo
g
ical
o
p
en
in
g
with
a
d
is
k
s
tr
u
ctu
r
i
n
g
elem
en
t.
T
h
e
lar
g
est
co
n
n
ec
ted
co
m
p
o
n
en
t
to
u
c
h
in
g
th
e
im
ag
e
b
o
r
d
er
was
th
en
id
en
tifie
d
,
a
n
d
its
b
o
u
n
d
i
n
g
b
o
x
was
u
s
ed
to
au
to
m
atic
ally
cr
o
p
th
e
b
r
ea
s
t
r
eg
io
n
,
p
r
o
d
u
cin
g
th
e
R
OI
–
b
r
ea
s
t
ar
ea
.
T
h
is
p
r
o
ce
d
u
r
e
is
cr
itical
f
o
r
th
e
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
class
if
icatio
n
s
ce
n
ar
io
,
as
it
en
s
u
r
es
th
at
o
n
ly
r
elev
a
n
t b
r
ea
s
t tis
s
u
e
is
an
aly
ze
d
wh
ile
ex
cl
u
d
in
g
b
ac
k
g
r
o
u
n
d
ar
tifa
cts.
B
ec
au
s
e
n
o
n
-
b
r
ea
s
t
s
tr
u
ctu
r
es,
p
ar
ticu
lar
ly
th
e
p
ec
to
r
al
m
u
s
cle,
m
ay
in
ter
f
er
e
with
lesi
o
n
an
aly
s
is
,
th
ey
wer
e
r
em
o
v
ed
u
s
in
g
an
ed
g
e
-
an
d
lin
e
-
b
ased
d
etec
tio
n
s
tr
ateg
y
.
C
an
n
y
ed
g
e
d
etec
tio
n
f
o
llo
wed
b
y
a
Ho
u
g
h
tr
a
n
s
f
o
r
m
co
n
s
tr
ain
ed
to
o
b
liq
u
e
o
r
ien
tatio
n
s
(
2
0
°
–
7
0
°)
was
ap
p
lied
in
th
e
u
p
p
er
co
r
n
er
r
e
g
io
n
;
th
e
d
etec
ted
lin
e
d
e
f
in
ed
a
tr
ian
g
u
lar
m
ask
co
v
er
in
g
th
e
p
ec
to
r
al
ar
ea
,
wh
ich
was
s
lig
h
tly
d
i
lated
to
en
s
u
r
e
f
u
ll
s
u
p
p
r
ess
io
n
.
T
h
is
m
ask
i
n
g
s
te
p
was
ap
p
lied
r
eg
a
r
d
less
o
f
p
r
o
jectio
n
to
m
in
im
ize
t
h
e
ch
a
n
ce
o
f
b
r
i
g
h
t
n
o
n
-
p
ar
en
ch
y
m
al
s
tr
u
ctu
r
es
b
ein
g
m
is
in
ter
p
r
eted
as
s
u
s
p
icio
u
s
m
ass
es.
Fo
r
m
ass
s
eg
m
en
tatio
n
with
in
t
h
e
cr
o
p
p
e
d
R
OI
,
a
h
is
to
g
r
am
-
g
u
id
ed
th
r
esh
o
ld
in
g
a
p
p
r
o
ac
h
was
u
s
ed
to
h
ig
h
lig
h
t
d
e
n
s
e
r
eg
io
n
s
.
T
h
e
th
r
esh
o
l
d
was
d
er
iv
ed
ad
ap
tiv
el
y
f
r
o
m
th
e
d
o
m
i
n
an
t
p
ea
k
in
th
e
i
n
ten
s
ity
d
is
tr
ib
u
tio
n
,
f
o
llo
wed
b
y
m
o
r
p
h
o
lo
g
ical
r
ef
in
em
en
t
(
d
ilatio
n
,
er
o
s
io
n
,
h
o
le
f
illi
n
g
,
an
d
ar
ea
o
p
en
i
n
g
)
.
C
an
d
id
ate
b
lo
b
s
wer
e
f
u
r
th
er
f
ilter
ed
u
s
in
g
s
h
ap
e
d
escr
ip
to
r
s
(
cir
cu
lar
ity
,
s
o
lid
ity
,
ex
te
n
t)
an
d
lo
ca
ti
o
n
r
u
les
to
r
ejec
t
el
o
n
g
ate
d
o
r
co
r
n
er
-
t
o
u
ch
in
g
ar
tifa
cts.
T
h
e
s
u
r
v
iv
in
g
b
lo
b
s
wer
e
en
clo
s
ed
b
y
b
o
u
n
d
in
g
b
o
x
es
an
d
cr
o
p
p
e
d
to
p
r
o
d
u
ce
R
OI
–
Ma
s
s
p
atch
es,
wh
ich
s
er
v
ed
as th
e
in
p
u
t f
o
r
t
h
e
b
en
ig
n
v
s
.
m
alig
n
an
t c
lass
if
icatio
n
task
.
I
n
s
u
m
m
ar
y
,
th
e
p
r
e
-
p
r
o
ce
s
s
in
g
p
ip
elin
e
co
m
b
in
es
class
ica
l
im
ag
e
p
r
o
ce
s
s
in
g
tech
n
iq
u
e
s
—
m
ed
ian
f
ilter
in
g
,
th
r
esh
o
ld
in
g
,
m
o
r
p
h
o
lo
g
ical
an
aly
s
is
,
an
d
ed
g
e
-
b
a
s
ed
m
ask
in
g
—
with
task
-
s
p
ec
if
ic
R
OI
ex
tr
ac
tio
n
.
Fo
llo
win
g
p
r
io
r
wo
r
k
s
[
1
8
]
,
[
1
9
]
,
we
tailo
r
ed
th
e
R
OI
d
ef
in
itio
n
to
th
e
class
if
icatio
n
s
c
en
ar
io
:
b
r
ea
s
t
ar
ea
R
OI
s
f
o
r
n
o
r
m
al
v
s
.
ab
n
o
r
m
al
an
d
m
ass
R
OI
s
f
o
r
b
en
ig
n
v
s
.
m
alig
n
a
n
t
.
T
h
is
tailo
r
ed
R
OI
s
tr
ateg
y
,
s
u
m
m
ar
ized
in
Fig
u
r
e
2
,
co
n
c
en
tr
ates le
ar
n
in
g
o
n
clin
ically
r
elev
an
t b
r
ea
s
t tis
s
u
e
an
d
co
n
tr
ib
u
tes to
th
e
s
tab
le
R
OC
–
AU
C
v
alu
es r
ep
o
r
ted
in
T
ab
le
1
,
p
ar
ticu
lar
ly
f
o
r
t
h
e
m
o
r
e
ch
allen
g
in
g
b
en
ig
n
v
s
.
m
alig
n
an
t ta
s
k
.
Fig
u
r
e
2
.
Pre
-
p
r
o
ce
s
s
in
g
p
i
p
elin
e
f
o
r
m
am
m
o
g
r
am
R
OI
ex
tr
ac
tio
n
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
ed
R
es
N
et
-
5
0
fr
a
mew
o
r
k
fo
r
ma
mmo
g
r
a
m
-
b
a
s
ed
b
r
ea
s
t c
a
n
ce
r
…
(
Mu
h
a
mma
d
S
u
b
a
li
)
1203
T
ab
le
1
.
C
r
o
s
s
-
v
alid
atio
n
p
er
f
o
r
m
an
ce
(
m
ea
n
±
SD
; 9
5
% C
I
)
Ta
sk
M
o
d
e
l
LR
Ep
A
C
C
S
e
n
s
S
p
e
c
P
r
e
c
F1
AUC
(
mea
n
±
S
D
)
9
5
%
C
I
(
A
U
C
)
N
o
r
mal
v
s
.
A
b
n
o
r
ma
l
(
M
I
A
S
)
R
e
sN
e
t
-
50
0
.
0
0
0
1
30
0
.
8
9
3
±
0
.
0
3
0
0
.
9
2
2
±
0
.
0
4
9
0
.
8
6
5
±
0
.
0
5
1
0
.
8
7
4
±
0
.
0
4
2
0
.
8
9
6
±
0
.
0
2
9
0
.
9
5
9
±
0
.
0
2
7
0
.
9
2
5
–
0
.
9
9
3
E
f
f
i
c
i
e
n
t
N
e
t
-
B0
0
.
0
0
1
30
0
.
8
9
9
±
0
.
0
3
1
0
.
8
8
8
±
0
.
0
6
8
0
.
9
0
9
±
0
.
0
9
8
0
.
9
1
8
±
0
.
0
8
0
0
.
8
9
9
±
0
.
0
2
9
0
.
9
8
1
±
0
.
0
1
7
0
.
9
6
0
–
1
.
0
0
0
B
e
n
i
g
n
v
s
.
M
a
l
i
g
n
a
n
t
(
C
B
I
S
-
D
D
S
M
)
R
e
sN
e
t
-
50
0
.
0
0
1
15
0
.
8
3
3
±
0
.
0
2
3
0
.
8
3
8
±
0
.
0
6
8
0
.
8
2
7
±
0
.
0
4
0
0
.
8
3
1
±
0
.
0
2
5
0
.
8
3
3
±
0
.
0
2
9
0
.
9
1
5
±
0
.
0
1
3
0
.
8
9
9
–
0
.
9
3
1
Ef
f
i
c
i
e
n
t
N
e
t
-
B0
0
.
0
0
0
1
30
0
.
8
0
3
±
0
.
0
1
9
0
.
8
3
8
±
0
.
0
4
6
0
.
7
6
7
±
0
.
0
5
2
0
.
7
8
4
±
0
.
0
3
4
0
.
8
0
9
±
0
.
0
1
9
0
.
9
0
2
±
0
.
0
1
6
0
.
8
8
2
–
0
.
9
2
2
N
o
t
e
:
9
5
%
C
I
s v
i
a
t
-
d
i
s
t
r
i
b
u
t
i
o
n
a
c
r
o
s
s fo
l
d
s (
p
a
i
r
e
d
5
-
f
o
l
d
C
V
)
.
L
R
=
L
e
a
r
n
i
n
g
R
a
t
e
;
Ep
=
Ep
o
c
h
s
;
A
C
C
=
A
c
c
u
r
a
c
y
;
S
e
n
s
=
S
e
n
s
i
t
i
v
i
t
y
;
S
p
e
c
=
S
p
e
c
i
f
i
c
i
t
y
;
P
r
e
c
=
P
r
e
c
i
s
i
o
n
;
F
1
=
F
1
-
s
c
o
r
e
;
A
U
C
=
R
O
C
–
AUC
2
.
3
.
Aug
m
ent
a
t
io
n
Af
ter
R
OI
ex
tr
ac
tio
n
,
th
e
im
ag
es
wer
e
s
tan
d
ar
d
ized
f
o
r
C
NN
in
p
u
t,
as
s
h
o
wn
in
Fig
u
r
e
3
.
All
R
OI
s
wer
e
r
esized
to
2
2
4
×2
2
4
p
ix
els
to
m
atch
th
e
in
p
u
t
r
eso
lu
tio
n
r
eq
u
ir
ed
b
y
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
,
b
o
th
p
r
etr
ain
e
d
-
o
n
I
m
ag
eNe
t.
Pix
el
in
ten
s
ity
n
o
r
m
aliza
tio
n
was
ap
p
lied
b
y
c
o
n
v
e
r
tin
g
g
r
ay
s
ca
le
im
ag
es
to
th
r
ee
-
ch
an
n
el
R
GB
an
d
s
ca
lin
g
p
ix
el
v
alu
es
to
th
e
[
0
,
1
]
r
a
n
g
e,
th
e
r
eb
y
alig
n
in
g
th
e
in
p
u
t
d
is
tr
ib
u
tio
n
with
th
at
ex
p
ec
ted
b
y
th
e
p
r
etr
ain
e
d
weig
h
ts
an
d
f
a
cilitatin
g
ef
f
e
ctiv
e
tr
an
s
f
er
lear
n
in
g
.
Fig
u
r
e
3
.
R
OI
s
tan
d
ar
d
izatio
n
an
d
au
g
m
en
tatio
n
w
o
r
k
f
l
o
w
T
o
f
u
r
th
er
im
p
r
o
v
e
r
o
b
u
s
tn
es
s
an
d
r
e
d
u
ce
o
v
er
f
itti
n
g
,
d
ata
au
g
m
e
n
tatio
n
was
in
co
r
p
o
r
at
ed
d
u
r
in
g
tr
ain
in
g
.
T
h
e
ap
p
lied
tr
an
s
f
o
r
m
atio
n
s
in
clu
d
ed
r
an
d
o
m
r
o
ta
tio
n
s
with
in
±
1
5
°,
r
an
d
o
m
h
o
r
izo
n
tal
r
ef
lectio
n
s
(
f
lip
s
)
,
r
an
d
o
m
s
ca
lin
g
b
etwe
en
9
0
%
an
d
1
1
0
%,
an
d
r
an
d
o
m
tr
an
s
latio
n
s
u
p
to
±
8
p
ix
els
in
b
o
th
h
o
r
izo
n
tal
an
d
v
er
tical
d
ir
ec
tio
n
s
.
T
h
e
s
e
au
g
m
en
tatio
n
s
s
im
u
late
r
ea
lis
tic
v
ar
iab
ilit
y
in
b
r
ea
s
t
p
o
s
itio
n
in
g
an
d
ac
q
u
is
itio
n
s
ettin
g
s
,
th
er
eb
y
en
r
ich
in
g
th
e
d
iv
er
s
ity
o
f
tr
ain
in
g
s
am
p
les
with
o
u
t
alter
in
g
th
e
u
n
d
e
r
ly
in
g
p
ath
o
lo
g
y
.
Similar
au
g
m
en
tati
o
n
s
tr
ateg
ies
h
av
e
b
ee
n
s
u
cc
e
s
s
f
u
lly
em
p
lo
y
ed
in
r
ec
e
n
t
m
am
m
o
g
r
a
p
h
y
C
AD
s
tu
d
ies
[
6
]
,
[
7
]
.
T
h
is
co
m
p
r
e
h
en
s
iv
e
au
g
m
en
t
atio
n
p
ip
elin
e
e
n
s
u
r
ed
t
h
at
th
e
C
NN
m
o
d
els
r
ec
eiv
ed
co
n
s
is
ten
t,
h
ig
h
-
q
u
ality
in
p
u
ts
ac
r
o
s
s
b
o
t
h
d
at
asets
an
d
class
if
icatio
n
s
ce
n
ar
io
s
,
en
ab
lin
g
a
f
ai
r
an
d
r
e
p
r
o
d
u
cib
le
co
m
p
ar
is
o
n
b
etwe
en
R
esNet
-
5
0
an
d
E
f
f
i
cien
tNet
-
B
0
.
I
n
p
r
ac
tice,
t
h
is
s
tan
d
ar
d
ized
i
n
p
u
t
an
d
au
g
m
en
tatio
n
s
tr
ateg
y
s
u
p
p
o
r
ted
th
e
r
elativ
ely
n
ar
r
o
w
co
n
f
id
e
n
ce
in
ter
v
als
an
d
co
n
s
is
ten
t
f
o
ld
-
wis
e
p
er
f
o
r
m
an
ce
o
b
s
er
v
e
d
in
T
ab
le
1
,
esp
ec
ially
o
n
th
e
s
m
a
ller
MI
AS d
ataset.
2
.
4
.
Cla
s
s
ba
la
ncing
a
nd
ex
perim
ent
a
l desig
n
T
o
ad
d
r
ess
class
im
b
alan
ce
in
b
o
th
MI
AS
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
an
d
C
B
I
S
-
DDSM
(
b
en
ig
n
v
s
.
m
alig
n
an
t)
,
we
ap
p
lied
p
ati
en
t
-
lev
el
s
tr
atif
ied
u
n
d
e
r
s
am
p
lin
g
with
a
f
ix
ed
r
an
d
o
m
s
ee
d
to
g
en
er
ate
r
ep
r
o
d
u
cib
le,
b
alan
ce
d
s
p
lits
.
Mo
d
el
ev
alu
atio
n
f
o
llo
wed
a
p
air
ed
5
-
f
o
l
d
cr
o
s
s
-
v
alid
atio
n
p
r
o
t
o
co
l,
i
n
wh
ich
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
s
h
ar
ed
id
en
tical
tr
ain
–
test
p
ar
titi
o
n
s
f
o
r
a
f
ai
r
,
h
ea
d
-
to
-
h
ea
d
c
o
m
p
ar
is
o
n
.
E
ac
h
f
o
ld
co
m
p
r
is
ed
f
o
u
r
p
ar
titi
o
n
s
f
o
r
tr
ain
in
g
an
d
o
n
e
f
o
r
test
in
g
,
en
s
u
r
in
g
th
at
ev
er
y
ca
s
e
was
ev
alu
ated
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
9
9
-
1
2
1
2
1204
ex
ac
tly
o
n
ce
i
n
th
e
test
r
o
le.
T
h
is
p
air
ed
d
esig
n
r
ed
u
c
ed
s
am
p
lin
g
b
ias
an
d
allo
w
ed
d
ir
ec
t
s
tatis
tical
co
m
p
ar
is
o
n
b
etwe
en
th
e
two
b
ac
k
b
o
n
es.
Mo
d
el
p
er
f
o
r
m
an
ce
was
r
ep
o
r
ted
as
m
ea
n
±
s
tan
d
ar
d
d
ev
i
atio
n
ac
r
o
s
s
f
o
ld
s
,
with
9
5
%
co
n
f
id
en
ce
in
ter
v
als
f
o
r
R
OC
–
AUC
esti
m
ated
u
s
in
g
th
e
t
-
d
is
tr
ib
u
tio
n
.
T
h
is
ex
p
er
im
e
n
tal
s
etu
p
en
s
u
r
ed
r
e
p
r
o
d
u
cib
ilit
y
an
d
p
r
o
v
i
d
ed
s
tatis
tically
r
o
b
u
s
t
co
n
clu
s
io
n
s
f
o
r
b
o
th
s
cr
ee
n
in
g
-
o
r
ien
ted
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
a
n
d
d
iag
n
o
s
tic
-
o
r
ien
te
d
(
b
e
n
ig
n
v
s
.
m
alig
n
an
t)
class
if
icatio
n
task
s
.
2
.
5
.
M
o
del
a
rc
hite
ct
ures
T
h
is
s
tu
d
y
co
m
p
ar
es
two
wid
ely
u
s
ed
C
NN
b
ac
k
b
o
n
es
,
R
e
s
Net
-
5
0
an
d
E
f
f
icien
tNet
-
B0
,
p
r
etr
ain
e
d
o
n
I
m
ag
eNe
t
an
d
ad
a
p
ted
f
o
r
b
in
ar
y
m
am
m
o
g
r
a
m
class
if
ica
tio
n
.
T
o
en
s
u
r
e
a
co
n
tr
o
lled
an
d
f
air
co
m
p
a
r
is
o
n
,
b
o
th
b
ac
k
b
o
n
es
ar
e
u
s
ed
as
f
i
x
ed
f
ea
tu
r
e
ex
tr
ac
to
r
s
:
th
e
p
r
e
tr
ain
ed
weig
h
ts
ar
e
f
r
o
ze
n
,
an
d
a
s
in
g
le,
id
e
n
tical,
m
in
im
al
class
if
ier
h
ea
d
is
attac
h
ed
to
ea
ch
b
ac
k
b
o
n
e.
T
h
is
h
ea
d
m
ap
s
ex
tr
ac
ted
f
ea
tu
r
es
to
two
o
u
tp
u
t
class
es
an
d
is
f
o
llo
wed
b
y
a
So
f
tMa
x
to
p
r
o
d
u
ce
p
r
o
b
ab
ilit
y
esti
m
ates.
B
y
h
o
ld
in
g
th
e
b
ac
k
b
o
n
e
p
ar
am
eter
s
co
n
s
tan
t
an
d
u
s
in
g
t
h
e
s
am
e
lig
h
tweig
h
t
h
ea
d
,
we
is
o
late
ar
c
h
itectu
r
al
d
if
f
er
en
ce
s
:
r
esid
u
al
c
o
n
n
e
ctio
n
s
in
R
esNet
-
50
v
er
s
u
s
co
m
p
o
u
n
d
s
ca
lin
g
in
E
f
f
icien
tNet
-
B0
—
with
o
u
t
co
n
f
o
u
n
d
in
g
e
f
f
ec
ts
f
r
o
m
class
if
ier
co
m
p
lex
ity
o
r
tu
n
in
g
d
ep
th
.
Fro
m
a
r
e
p
r
ese
n
t
ati
o
n
al
s
t
an
d
p
o
i
n
t
,
R
esNet
-
5
0
le
v
e
r
a
g
es
r
esi
d
u
al
c
o
n
n
ec
ti
o
n
s
t
o
s
ta
b
i
liz
e
i
n
f
o
r
m
at
io
n
f
l
o
w
i
n
d
ee
p
e
r
s
t
ac
k
s
,
en
a
b
li
n
g
r
o
b
u
s
t
f
e
at
u
r
e
e
x
t
r
a
cti
o
n
ev
en
w
it
h
li
m
it
ed
d
a
ta
[
3
3
]
.
E
f
f
icien
tNet
-
B
0
,
in
co
n
tr
ast,
ap
p
lies
co
m
p
o
u
n
d
s
ca
lin
g
ac
r
o
s
s
d
ep
th
,
wid
th
,
an
d
r
eso
lu
tio
n
u
s
in
g
MBC
o
n
v
b
lo
ck
s
,
y
ield
in
g
an
ef
f
icien
t
ac
cu
r
ac
y
–
p
e
r
f
o
r
m
an
ce
tr
ad
e
-
o
f
f
[
3
4
]
.
Op
tim
izati
o
n
was
in
ten
tio
n
ally
co
n
s
er
v
ativ
e
to
p
r
io
r
itize
r
ep
r
o
d
u
cib
ilit
y
.
W
e
em
p
lo
y
ed
th
e
Ad
am
o
p
tim
izer
with
ea
r
ly
s
to
p
p
in
g
a
n
d
ex
p
lo
r
ed
a
co
m
p
ac
t
h
y
p
er
p
ar
am
eter
g
r
i
d
(
lear
n
in
g
r
ates
∈
{1
e
-
3
,
1
e
-
4
};
ep
o
c
h
s
∈
{
1
5
,
3
0
}
)
.
U
n
less
o
th
er
wi
s
e
n
o
ted
,
s
ec
tio
n
3
r
ep
o
r
ts
th
e
b
est
-
p
e
r
f
o
r
m
in
g
c
o
n
f
ig
u
r
atio
n
f
o
r
ea
ch
b
ac
k
b
o
n
e
an
d
d
ataset.
Fo
r
clar
ity
,
T
ab
le
2
s
u
m
m
a
r
izes
th
e
k
ey
ar
ch
itectu
r
al
an
d
tr
ai
n
in
g
s
ettin
g
s
,
h
ig
h
lig
h
tin
g
th
e
s
y
m
m
etr
y
b
etwe
en
th
e
two
p
ip
elin
es
—
id
en
tical
class
if
ier
h
ea
d
s
,
f
r
o
ze
n
b
ac
k
b
o
n
es,
s
h
ar
ed
o
p
tim
izer
an
d
s
to
p
p
in
g
cr
iter
ia,
an
d
a
co
m
p
ac
t
h
y
p
er
p
ar
am
eter
s
ea
r
c
h
s
p
ac
e
—
th
er
eb
y
en
s
u
r
in
g
a
clea
n
ar
ch
itectu
r
al
co
m
p
ar
is
o
n
.
T
ab
le
2
.
Ar
c
h
itectu
r
al
an
d
tr
ain
in
g
s
ettin
g
s
o
f
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
f
o
r
m
am
m
o
g
r
am
class
if
icatio
n
A
sp
e
c
t
R
e
sN
e
t
-
50
Ef
f
i
c
i
e
n
t
N
e
t
-
B0
P
r
e
t
r
a
i
n
i
n
g
I
mag
e
N
e
t
I
mag
e
N
e
t
B
a
c
k
b
o
n
e
r
o
l
e
F
r
o
z
e
n
b
a
c
k
b
o
n
e
(
f
i
x
e
d
f
e
a
t
u
r
e
e
x
t
r
a
c
t
o
r
)
F
r
o
z
e
n
b
a
c
k
b
o
n
e
(
f
i
x
e
d
f
e
a
t
u
r
e
e
x
t
r
a
c
t
o
r
)
C
l
a
s
si
f
i
e
r
h
e
a
d
M
i
n
i
m
a
l
l
i
n
e
a
r
l
a
y
e
r
→
S
o
f
t
M
a
x
(
2
c
l
a
sses)
M
i
n
i
m
a
l
l
i
n
e
a
r
l
a
y
e
r
→
S
o
f
t
M
a
x
(
2
c
l
a
sses)
K
e
y
a
r
c
h
i
t
e
c
t
u
r
a
l
t
r
a
i
t
R
e
si
d
u
a
l
c
o
n
n
e
c
t
i
o
n
s fo
r
st
a
b
l
e
d
e
e
p
t
r
a
i
n
i
n
g
C
o
m
p
o
u
n
d
sc
a
l
i
n
g
w
i
t
h
M
B
C
o
n
v
b
l
o
c
k
s
O
p
t
i
mi
z
e
r
a
n
d
c
o
n
t
r
o
l
A
d
a
m
;
e
a
r
l
y
st
o
p
p
i
n
g
(
b
e
st
v
a
l
i
d
a
t
i
o
n
l
o
ss re
t
a
i
n
e
d
)
A
d
a
m
;
e
a
r
l
y
st
o
p
p
i
n
g
(
b
e
st
v
a
l
i
d
a
t
i
o
n
l
o
ss re
t
a
i
n
e
d
)
H
y
p
e
r
p
a
r
a
me
t
e
r
s
t
e
s
t
e
d
Le
a
r
n
i
n
g
r
a
t
e
s {
1
e
-
3
,
1
e
-
4
};
e
p
o
c
h
s
{
1
5
,
3
0
}
Le
a
r
n
i
n
g
r
a
t
e
s {
1
e
-
3
,
1
e
-
4
};
e
p
o
c
h
s
{
1
5
,
3
0
}
2
.
6
.
E
v
a
lua
t
i
o
n
m
et
rics
Mo
d
el
p
er
f
o
r
m
a
n
ce
was
ass
ess
ed
u
s
in
g
a
co
m
p
r
eh
en
s
iv
e
s
et
o
f
s
tan
d
ar
d
class
if
icatio
n
m
etr
ics
wid
ely
ad
o
p
ted
in
m
ed
ical
im
ag
in
g
.
T
h
ese
in
clu
d
e
d
o
v
er
all
ac
cu
r
ac
y
(
AC
C
)
,
wh
ich
m
ea
s
u
r
es
th
e
p
r
o
p
o
r
tio
n
o
f
co
r
r
ec
tly
class
if
ied
s
am
p
les,
s
en
s
itiv
ity
(
r
ec
all)
,
wh
ic
h
q
u
an
tifie
s
th
e
a
b
ilit
y
to
co
r
r
ec
t
ly
id
en
tify
p
o
s
itiv
e
ca
s
es
s
u
ch
as
ab
n
o
r
m
al
o
r
m
alig
n
an
t
lesi
o
n
s
,
s
p
ec
if
icity
(
Sp
ec
)
,
wh
ich
ca
p
tu
r
es
th
e
ab
ilit
y
to
c
o
r
r
ec
tl
y
id
en
tify
n
eg
ativ
e
ca
s
es
(
n
o
r
m
al
o
r
b
en
ig
n
)
,
p
r
ec
is
io
n
(
Pre
c)
,
wh
ich
m
ea
s
u
r
es
th
e
p
r
o
p
o
r
ti
o
n
o
f
tr
u
e
p
o
s
itiv
es
am
o
n
g
all
p
r
ed
icted
p
o
s
itiv
es,
an
d
th
e
F1
-
s
co
r
e,
d
ef
i
n
ed
as th
e
h
ar
m
o
n
ic
m
ea
n
o
f
p
r
ec
is
io
n
an
d
r
ec
all.
I
n
ad
d
itio
n
,
th
e
r
ec
eiv
e
r
o
p
er
atin
g
ch
ar
ac
ter
is
tic
ar
ea
u
n
d
e
r
th
e
cu
r
v
e
(
R
OC
–
AUC)
wa
s
co
m
p
u
ted
f
r
o
m
p
r
ed
icted
class
p
r
o
b
a
b
ilit
ies
to
q
u
an
tify
r
an
k
i
n
g
p
er
f
o
r
m
an
ce
(
i.e
.
,
th
e
p
r
o
b
a
b
ilit
y
th
at
a
r
an
d
o
m
l
y
ch
o
s
en
p
o
s
itiv
e
s
am
p
le
is
r
a
n
k
ed
h
ig
h
e
r
th
a
n
a
n
eg
ativ
e
o
n
e)
.
R
OC
cu
r
v
es
wer
e
g
e
n
er
ated
,
a
n
d
m
ea
n
R
OC
–
AU
C
v
alu
es
ar
e
r
ep
o
r
ted
to
g
et
h
er
with
9
5
%
co
n
f
id
en
ce
in
ter
v
als
(
C
I
)
ac
r
o
s
s
f
o
ld
s
u
s
in
g
th
e
t
-
d
is
tr
ib
u
tio
n
.
All
o
th
er
m
etr
i
cs
(
AC
C
,
Sen
s
,
Sp
ec
,
Pre
c,
an
d
F1
)
wer
e
d
er
iv
ed
f
r
o
m
f
o
ld
-
wis
e
co
n
f
u
s
io
n
m
atr
ices
an
d
ar
e
s
u
m
m
a
r
ized
as
m
ea
n
±
s
tan
d
ar
d
d
ev
iatio
n
ac
r
o
s
s
f
o
ld
s
.
T
h
is
co
m
b
i
n
atio
n
o
f
m
etr
ics
en
s
u
r
ed
a
co
m
p
r
eh
e
n
s
iv
e
ev
alu
atio
n
o
f
b
o
th
o
v
er
all
class
if
icatio
n
ac
cu
r
ac
y
an
d
clin
ically
cr
itical
p
er
f
o
r
m
a
n
ce
d
im
en
s
io
n
s
,
with
p
ar
ticu
lar
e
m
p
h
asis
o
n
s
en
s
itiv
ity
f
o
r
s
c
r
ee
n
in
g
s
ce
n
ar
io
s
a
n
d
s
p
ec
if
i
city
f
o
r
d
iag
n
o
s
tic
co
n
tex
ts
.
2
.
7
.
St
a
t
is
t
ica
l
a
na
ly
s
is
T
o
v
er
i
f
y
wh
et
h
er
th
e
p
er
f
o
r
m
an
ce
d
if
f
er
en
ce
s
b
etwe
en
R
esNet
-
5
0
an
d
E
f
f
icien
tNe
t
-
B
0
wer
e
s
tatis
t
ically
m
ea
n
in
g
f
u
l,
we
p
er
f
o
r
m
e
d
p
air
e
d
h
y
p
o
th
esis
test
in
g
at
th
e
f
o
ld
le
v
el,
en
s
u
r
in
g
d
ir
ec
t
co
m
p
a
r
is
o
n
ac
r
o
s
s
id
en
tical
d
ata
p
ar
titi
o
n
s
.
Fo
r
ea
ch
lear
n
i
n
g
-
r
ate
×
e
p
o
ch
c
o
n
f
ig
u
r
atio
n
in
t
h
e
c
o
m
p
ac
t
g
r
id
,
f
o
ld
-
wis
e
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
ed
R
es
N
et
-
5
0
fr
a
mew
o
r
k
fo
r
ma
mmo
g
r
a
m
-
b
a
s
ed
b
r
ea
s
t c
a
n
ce
r
…
(
Mu
h
a
mma
d
S
u
b
a
li
)
1205
m
etr
ic
v
alu
es
(
ac
cu
r
ac
y
,
F1
-
s
co
r
e
,
s
en
s
itiv
ity
,
s
p
ec
if
icity
,
an
d
R
OC
–
AUC)
f
r
o
m
b
o
th
m
o
d
els
wer
e
co
m
p
a
r
ed
u
s
in
g
a
p
air
ed
Stu
d
en
t’
s
t
-
tes
t;
to
ac
co
m
m
o
d
ate
p
o
ten
tial
n
o
n
-
n
o
r
m
ality
,
a
W
ilco
x
o
n
s
ig
n
ed
-
r
an
k
test
was
also
ap
p
lied
.
Statis
tical
s
ig
n
if
i
ca
n
ce
was
s
et
at
p
<
0
.
0
5
.
I
n
ad
d
itio
n
to
r
ep
o
r
tin
g
m
ea
n
±
s
tan
d
ar
d
d
ev
iatio
n
,
9
5
%
co
n
f
id
en
ce
in
ter
v
als
f
o
r
R
OC
–
AUC
wer
e
co
m
p
u
ted
u
s
in
g
th
e
t
-
d
is
tr
ib
u
tio
n
ac
r
o
s
s
f
o
ld
s
.
T
h
is
d
u
al
-
test
in
g
f
r
am
ewo
r
k
e
n
s
u
r
es
th
at
p
er
f
o
r
m
an
ce
d
if
f
er
e
n
ce
s
ar
e
s
tatis
tically
v
alid
ated
r
ath
er
th
an
p
u
r
el
y
d
escr
ip
tiv
e.
3.
RE
SU
L
T
S AN
D
D
I
SCU
SS
I
O
N
3
.
1
.
Cro
s
s
-
v
a
lid
a
t
io
n per
f
o
rm
a
n
ce
Al
l
r
e
s
u
l
t
s
r
ep
o
r
ted
h
er
e
w
e
r
e
o
b
ta
in
ed
u
n
d
er
th
e
s
tan
d
ar
d
i
ze
d
au
g
m
en
ta
tio
n
an
d
p
at
ien
t
-
l
ev
e
l
s
t
r
a
tif
ied
u
n
d
er
s
a
m
p
l
in
g
p
r
o
to
co
l
d
e
s
cr
ib
ed
in
s
e
ct
io
n
2
,
wi
th
p
a
ir
ed
5
-
f
o
ld
cr
o
s
s
-
v
al
id
a
tio
n
en
s
u
r
in
g
id
en
ti
ca
l
p
ar
t
it
io
n
s
ac
r
o
s
s
b
ac
k
b
o
n
e
s
.
Am
o
n
g
th
e
te
s
ted
co
n
f
ig
u
r
at
io
n
s
(
L
R
∈
{
1e
-
3
,
1
e
-
4
},
ep
o
ch
s
∈
{1
5
,
3
0
})
,
w
e
r
ep
o
r
t
th
e
b
e
s
t
-
p
er
f
o
r
m
in
g
s
e
tu
p
f
o
r
e
ac
h
b
a
ck
b
o
n
e
an
d
d
at
a
s
e
t
to
p
r
o
v
id
e
a
b
es
t
-
c
as
e
co
m
p
ar
i
s
o
n
an
d
en
s
u
r
e
co
n
c
i
s
e
p
r
e
s
en
ta
ti
o
n
o
f
r
e
s
u
l
t
s
.
T
ab
l
e
1
p
r
e
s
en
t
s
th
e
cr
o
s
s
-
v
al
id
a
t
io
n
r
e
s
u
lt
s
o
f
R
es
Ne
t
-
5
0
an
d
E
f
f
i
ci
en
t
Ne
t
-
B
0
ac
r
o
s
s
t
h
e
t
wo
b
en
ch
m
ar
k
t
as
k
s
:
n
o
r
m
a
l
v
s
.
ab
n
o
r
m
al
(
M
I
A
S)
an
d
b
en
ig
n
v
s
.
m
a
lig
n
an
t
(
C
B
I
S
-
DD
S
M)
.
On
M
I
A
S,
E
f
f
i
ci
en
t
Ne
t
-
B
0
ac
h
ie
v
ed
a
m
ea
n
R
O
C
–
A
U
C
o
f
0
.
9
8
1
±
0
.
0
1
7
(
9
5
%
C
I
:
0
.
9
6
0
−
1
.
0
0
0
)
,
h
ig
h
er
th
an
0
.
9
5
9
±
0
.
0
2
7
(
9
5
%
C
I
:
0
.
9
2
5
−
0
.
9
9
3
)
f
o
r
R
e
s
Ne
t
-
5
0
.
B
ey
o
n
d
R
O
C
–
A
U
C
,
E
f
f
i
ci
en
t
Ne
t
-
B
0
a
ls
o
o
u
tp
er
f
o
r
m
ed
R
e
s
N
et
-
5
0
in
ac
c
u
r
a
cy
(
0
.
8
9
9
v
s
.
0
.
8
9
3
)
,
s
p
ec
if
ici
ty
(
0
.
9
0
9
v
s
.
0
.
8
6
5
)
,
p
r
ec
is
io
n
(
0
.
9
1
8
v
s
.
0
.
8
7
4
)
,
an
d
F1
-
s
co
r
e
(
0
.
8
9
9
v
s
.
0
.
8
9
6
)
.
R
e
s
Ne
t
-
5
0
s
h
o
we
d
s
l
i
g
h
tly
h
ig
h
er
m
ea
n
s
en
s
it
iv
ity
in
i
t
s
b
e
s
t
co
n
f
i
g
u
r
at
io
n
(
0
.
9
2
2
v
s
.
0
.
8
8
8
)
,
al
th
o
u
g
h
th
i
s
d
if
f
er
en
ce
w
as
n
o
t
s
t
at
i
s
t
ic
al
ly
s
i
g
n
if
ic
an
t
.
As
s
h
o
wn
in
Fig
u
r
e
s
4
(
a)
an
d
4
(
b
)
,
AUC
v
ar
ies
s
y
s
tem
atica
lly
with
th
e
L
R
–
ep
o
c
h
co
m
b
in
atio
n
:
E
f
f
icien
tNet
-
B
0
b
en
ef
its
f
r
o
m
lo
n
g
er
tr
ai
n
in
g
at
a
h
i
g
h
e
r
L
R
(
1
e
-
3
,
3
0
ep
o
ch
s
)
,
r
ea
c
h
in
g
0
.
9
8
1
±
0
.
0
1
7
,
wh
er
ea
s
R
esNet
-
5
0
p
ea
k
s
at
a
lo
wer
L
R
with
lo
n
g
er
tr
ain
in
g
(
1
e
-
4
,
3
0
ep
o
ch
s
)
at
0
.
9
5
9
±
0
.
0
2
7
.
T
h
e
lar
g
er
er
r
o
r
b
ar
s
f
o
r
s
o
m
e
E
f
f
icien
tNet
-
B
0
s
ettin
g
s
in
d
icate
h
ig
h
er
f
o
ld
-
to
-
f
o
ld
v
a
r
iab
ilit
y
,
b
u
t its
p
ea
k
co
n
f
i
g
u
r
atio
n
s
till
o
u
tp
er
f
o
r
m
s
R
esNet
-
5
0
o
n
MI
AS,
alig
n
in
g
with
T
ab
le
1
.
Fig
u
r
e
s
5
(
a
)
a
n
d
5
(
b
)
s
h
o
w
th
e
R
OC
cu
r
v
es
f
o
r
th
e
b
est co
n
f
ig
u
r
atio
n
s
.
Fo
r
C
B
I
S
-
DDSM,
p
er
f
o
r
m
an
ce
m
ar
g
in
s
wer
e
n
ar
r
o
wer
.
R
esNet
-
5
0
o
b
tain
ed
a
m
ea
n
R
OC
–
AUC
o
f
0
.
9
1
5
±
0
.
0
1
3
(
9
5
%
C
I
:
0
.
8
9
9
−
0
.
9
3
1
)
,
s
lig
h
tly
ab
o
v
e
E
f
f
ic
ien
tNet
-
B
0
at
0
.
9
0
2
±
0
.
0
1
6
(
9
5
%
C
I
:
0
.
8
8
2
–
0
.
9
2
2
)
,
with
e
q
u
al
s
en
s
itiv
ity
(
b
o
th
0
.
8
3
8
)
.
E
f
f
icien
tNet
-
B
0
r
em
ain
ed
c
o
m
p
etitiv
e
o
n
ac
cu
r
ac
y
(
0
.
8
0
3
)
,
p
r
ec
is
io
n
(
0
.
7
8
4
)
,
a
n
d
F1
-
s
co
r
e
(
0
.
8
0
9
)
,
th
o
u
g
h
tr
ailin
g
s
lig
h
tly
.
Fig
u
r
es
5
(
a)
a
n
d
5
(
b
)
illu
s
tr
ate
th
e
im
p
ac
t
o
f
lear
n
in
g
r
ate
a
n
d
t
r
ain
in
g
ep
o
ch
s
o
n
R
OC
–
AUC
f
o
r
C
B
I
S
-
DDSM:
R
e
s
Net
-
5
0
is
b
est
at
1
e
-
3
,
1
5
e
p
o
ch
s
(
lo
n
g
er
tr
ain
in
g
s
lig
h
tly
d
e
g
r
a
d
es
p
er
f
o
r
m
an
c
e)
,
wh
ile
E
f
f
ic
ien
tNet
-
B
0
p
r
ef
er
s
1
e
-
4
,
3
0
e
p
o
ch
s
.
Fig
u
r
e
s
7
(
a)
an
d
7
(
b
)
p
r
esen
t
th
e
R
OC
cu
r
v
es f
o
r
th
e
b
est co
n
f
ig
u
r
atio
n
s
,
h
ig
h
lig
h
tin
g
th
e
r
elativ
ely
s
tab
le
y
et
o
v
er
lap
p
i
n
g
p
er
f
o
r
m
an
ce
tr
e
n
d
s
.
Fig
u
r
e
4
.
C
o
m
p
a
r
is
o
n
o
f
m
ea
n
R
OC
–
AU
C
(
±
SD)
o
v
er
p
air
e
d
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
o
n
M
I
AS
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
f
o
r
R
esNet
-
5
0
(
lef
t)
a
n
d
E
f
f
icien
tNet
-
B
0
(
r
ig
h
t)
u
n
d
er
f
o
u
r
h
y
p
er
p
a
r
am
eter
s
ettin
g
s
:
L
R
∈
{1
e
-
3
,
1
e
-
4}
a
n
d
ep
o
ch
s
∈
{1
5
,
3
0
}
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
9
9
-
1
2
1
2
1206
(
a)
(
b
)
Fig
u
r
e
5
.
R
OC
cu
r
v
es (
m
ea
n
±
SD; p
air
ed
,
p
atien
t
-
le
v
el
5
-
f
o
ld
C
V)
f
o
r
th
e
b
est
-
p
er
f
o
r
m
i
n
g
co
n
f
ig
u
r
atio
n
s
o
n
MI
AS (
n
o
r
m
al
v
s
.
a
b
n
o
r
m
al)
:
(
a)
R
esNet
-
5
0
,
L
R
=
1
e
-
4
,
3
0
ep
o
ch
s
; (
b
)
E
f
f
icien
tNet
-
B
0
,
L
R
=
1
e
-
3
,
3
0
ep
o
ch
s
.
So
lid
lin
es sh
o
w
th
e
m
ea
n
; sh
ad
ed
b
an
d
s
d
e
n
o
te
±
1
SD.
9
5
% C
I
s
f
o
r
AUC ar
e
r
ep
o
r
ted
in
T
a
b
le
1
Fig
u
r
e
6.
C
o
m
p
a
r
is
o
n
o
f
m
ea
n
R
OC
–
AU
C
(
±
SD)
o
v
er
p
air
e
d
5
-
f
o
ld
cr
o
s
s
-
v
alid
atio
n
o
n
C
B
I
S
-
DDSM
(
b
en
ig
n
v
s
.
m
alig
n
a
n
t
)
f
o
r
R
esNet
-
5
0
(
lef
t)
a
n
d
E
f
f
icien
tNet
-
B
0
(
r
ig
h
t)
u
n
d
er
f
o
u
r
h
y
p
er
p
a
r
am
eter
s
ettin
g
s
:
L
R
∈
{1
e
-
3
,
1
e
-
4
}
a
n
d
ep
o
ch
s
∈
{1
5
,
3
0
}
(
a)
(
b
)
Fig
u
r
e
7
.
R
OC
cu
r
v
es (
m
ea
n
±
SD; p
air
ed
,
p
atien
t
-
le
v
el
5
-
f
o
ld
C
V)
f
o
r
th
e
b
est
-
p
er
f
o
r
m
i
n
g
co
n
f
ig
u
r
atio
n
s
o
n
C
B
I
S
-
DD
SM
(
b
en
ig
n
v
s
.
m
ali
g
n
an
t)
: (
a
)
R
esNet
-
5
0
,
L
R
=
1
e
-
3
,
1
5
ep
o
c
h
s
; (
b
)
E
f
f
icien
tN
et
-
B
0
,
L
R
=
1
e
-
4
,
3
0
ep
o
ch
s
.
So
lid
lin
es sh
o
w
th
e
m
ea
n
;
s
h
ad
ed
b
an
d
s
d
en
o
te
±
1
SD.
9
5
% C
I
s
f
o
r
AUC ar
e
r
ep
o
r
ted
in
T
ab
le
1
Evaluation Warning : The document was created with Spire.PDF for Python.
I
n
t J E
lec
&
C
o
m
p
E
n
g
I
SS
N:
2088
-
8
7
0
8
Op
timiz
ed
R
es
N
et
-
5
0
fr
a
mew
o
r
k
fo
r
ma
mmo
g
r
a
m
-
b
a
s
ed
b
r
ea
s
t c
a
n
ce
r
…
(
Mu
h
a
mma
d
S
u
b
a
li
)
1207
3
.
2
.
St
a
t
is
t
ica
l
s
ig
nifica
nce
a
na
ly
s
is
T
o
v
er
i
f
y
wh
et
h
er
th
e
p
er
f
o
r
m
an
ce
d
if
f
er
en
ce
s
b
etwe
en
R
esNet
-
5
0
an
d
E
f
f
icien
tNe
t
-
B
0
wer
e
s
tatis
t
ically
m
ea
n
in
g
f
u
l,
we
a
p
p
lied
b
o
th
p
air
e
d
Stu
d
e
n
t’
s
t
-
test
s
an
d
W
ilco
x
o
n
s
ig
n
ed
-
r
a
n
k
test
s
ac
r
o
s
s
th
e
5
-
f
o
ld
c
r
o
s
s
-
v
alid
atio
n
.
T
h
ese
co
m
p
lem
en
tar
y
test
s
ac
co
u
n
t f
o
r
th
e
d
ep
e
n
d
en
c
y
o
f
p
ai
r
ed
f
o
ld
s
an
d
r
ed
u
ce
th
e
r
is
k
o
f
b
ias
ca
u
s
ed
b
y
n
o
n
-
n
o
r
m
al
m
etr
ic
d
is
tr
ib
u
tio
n
s
.
Statis
tical
test
in
g
was
p
er
f
o
r
m
ed
u
n
d
er
an
id
e
n
tical
LR
–
ep
o
ch
s
ettin
g
f
o
r
b
o
th
b
ac
k
b
o
n
es,
s
elec
ted
as
th
e
b
est
s
h
ar
ed
co
n
f
ig
u
r
atio
n
p
e
r
d
ataset
f
r
o
m
th
e
h
y
p
er
p
ar
am
eter
g
r
i
d
in
s
ec
tio
n
2
.
5
(
s
ee
T
ab
le
2
)
.
W
h
ile
T
ab
le
1
r
ep
o
r
ts
th
e
b
est
-
p
er
f
o
r
m
i
n
g
co
n
f
ig
u
r
atio
n
f
o
r
ea
ch
b
ac
k
b
o
n
e,
T
ab
le
3
p
r
o
v
i
d
es
a
co
n
tr
o
lled
s
am
e
-
s
ettin
g
p
air
ed
co
m
p
a
r
is
o
n
(
f
o
l
d
-
wis
e,
s
am
e
-
s
ettin
g
;
th
u
s
,
Δ
Me
an
v
alu
es
a
r
e
n
o
t
d
er
iv
ed
f
r
o
m
T
ab
le
1
)
t
o
ass
ess
wh
eth
er
th
e
o
b
s
er
v
ed
d
if
f
er
e
n
ce
s
ar
e
s
tatis
tically
m
ea
n
in
g
f
u
l u
n
d
er
id
e
n
tical
tr
ain
in
g
co
n
d
itio
n
s
.
As
s
u
m
m
ar
ized
in
T
ab
le
3
,
o
n
th
e
MI
AS
d
ataset
(
n
o
r
m
al
v
s
.
ab
n
o
r
m
al)
,
E
f
f
icien
tNet
-
B
0
d
em
o
n
s
tr
ated
s
m
all
b
u
t
s
tatis
ti
ca
lly
s
ig
n
if
ican
t
im
p
r
o
v
em
en
t
s
o
v
er
R
esNet
-
5
0
in
ac
c
u
r
ac
y
(
Δ
Me
an
=
+0
.
0
4
5
,
p
=
0
.
0
1
6
)
,
F1
-
s
co
r
e
(
Δ
Me
an
=
+0
.
0
4
8
,
p
=
0
.
0
1
9
)
,
a
n
d
R
OC
–
AUC
(
Δ
Me
an
=
+
0
.
0
2
7
,
p
=
0
.
0
3
6
)
b
y
p
air
ed
t
-
test
,
wh
ile
th
e
W
ilco
x
o
n
s
ig
n
ed
-
r
a
n
k
test
d
id
n
o
t
r
ea
ch
s
ig
n
if
ican
ce
(
p
=
0
.
0
6
6
)
.
Sen
s
itiv
ity
an
d
s
p
ec
if
icity
also
s
h
o
wed
p
o
s
itiv
e
g
ain
s
(
Δ
Me
an
=
+0
.
0
5
6
an
d
Δ
Me
an
=
+0
.
0
3
3
,
r
esp
ec
tiv
ely
)
,
b
u
t
th
ese
d
if
f
er
en
ce
s
wer
e
n
o
t
s
tatis
tically
s
ig
n
if
ican
t
(
p
>
0
.
0
5
)
.
T
h
is
in
d
icate
s
th
at
E
f
f
icien
tNet
-
B
0
p
r
o
v
id
es
m
o
r
e
r
eliab
le
o
v
er
all
d
is
cr
im
in
atio
n
o
n
MI
AS,
p
ar
t
icu
lar
ly
in
ter
m
s
o
f
b
alan
ce
d
p
er
f
o
r
m
an
ce
(
AC
C
,
F1
)
an
d
p
r
o
b
a
b
ilit
y
r
an
k
in
g
(
AUC).
T
ab
le
3.
Statis
tical
s
ig
n
if
ican
ce
test
in
g
o
f
R
esNet
-
5
0
v
er
s
u
s
E
f
f
icien
tNet
-
B
0
ac
r
o
s
s
cr
o
s
s
-
v
alid
atio
n
f
o
l
d
s
u
s
in
g
p
air
ed
t
-
test
an
d
W
ilco
x
o
n
s
ig
n
ed
-
r
an
k
test
Ta
sk
/
M
e
t
r
i
c
Δ M
e
a
n
(
Ef
f
B
0
–
R
e
sN
e
t
5
0
)
t
-
t
e
s
t
(
p
)
W
i
l
c
o
x
o
n
(
p
)
S
i
g
n
i
f
i
c
a
n
c
e
N
o
r
mal
v
s
.
A
b
n
o
r
ma
l
(
M
I
A
S
)
A
c
c
u
r
a
c
y
+
0
.
0
4
5
0
.
0
1
6
0
.
0
6
6
p
<
0
.
0
5
(
t
)
F1
-
sc
o
r
e
+
0
.
0
4
8
0
.
0
1
9
0
.
0
6
8
p
<
0
.
0
5
(
t
)
S
e
n
s
i
t
i
v
i
t
y
(
R
e
c
a
l
l
)
+
0
.
0
56
0
.
2
2
6
0
.
1
4
4
n
.
s.
S
p
e
c
i
f
i
c
i
t
y
+
0
.
0
3
3
0
.
4
2
6
0
.
2
8
5
n
.
s.
R
O
C
–
AUC
+
0
.
0
2
7
0
.
0
3
6
0
.
0
6
3
p
<
0
.
0
5
(
t
)
B
e
n
i
g
n
v
s
.
M
a
l
i
g
n
a
n
t
(
C
B
I
S
-
D
D
S
M
)
A
c
c
u
r
a
c
y
–
0
.
0
3
3
0
.
0
5
4
0
.
0
6
3
n
.
s.
(
b
o
r
d
e
r
l
i
n
e
)
F1
-
sc
o
r
e
–
0
.
0
2
7
0
.
1
5
2
0
.
1
8
8
n
.
s.
S
e
n
s
i
t
i
v
i
t
y
(
R
e
c
a
l
l
)
+
0
.
0
0
2
0
.
9
6
0
1
.
0
0
0
n
.
s.
S
p
e
c
i
f
i
c
i
t
y
–
0
.
0
6
9
0
.
0
7
3
0
.
1
2
5
n
.
s.
R
O
C
–
AUC
–
0
.
0
2
0
0
.
0
2
7
0
.
0
6
3
p
<
0
.
0
5
(
t
)
N
o
t
e
s
:
Δ
M
e
a
n
=
p
e
r
f
o
r
m
a
n
c
e
d
i
f
f
e
r
e
n
c
e
(
Ef
f
i
c
i
e
n
t
N
e
t
-
B
0
–
R
e
sN
e
t
-
5
0
)
.
S
i
g
n
i
f
i
c
a
n
c
e
l
e
v
e
l
s:
p
<
0
.
0
5
=
si
g
n
i
f
i
c
a
n
t
;
p
<
0
.
0
1
=
h
i
g
h
l
y
s
i
g
n
i
f
i
c
a
n
t
;
n
.
s
.
=
n
o
t
si
g
n
i
f
i
c
a
n
t
.
S
a
m
e
-
set
t
i
n
g
c
o
m
p
a
r
i
s
o
n
w
a
s
u
se
d
f
o
r
s
t
a
t
i
st
i
c
a
l
t
e
s
t
i
n
g
:
M
I
A
S
(
LR
=
1e
-
4
,
E
p
=
1
5
)
a
n
d
C
B
I
S
-
D
D
S
M
(
L
R
=
1e
-
3
,
E
p
=
1
5
)
f
o
r
b
o
t
h
b
a
c
k
b
o
n
e
s
.
Fo
r
th
e
C
B
I
S
-
DDSM
d
ataset
(
b
en
ig
n
v
s
.
m
alig
n
an
t)
,
th
e
tr
en
d
r
e
v
er
s
ed
:
R
esNet
-
5
0
o
u
t
p
er
f
o
r
m
ed
E
f
f
icien
tNet
-
B
0
in
ter
m
s
o
f
R
OC
–
AU
C
(
Δ
Me
an
=
–
0
.
0
2
0
,
p
=
0
.
0
2
7
)
,
in
d
icatin
g
a
s
tatis
tically
s
ig
n
if
ican
t
ad
v
an
tag
e
b
y
p
air
ed
t
-
test
,
wh
ile
th
e
W
ilco
x
o
n
s
ig
n
ed
-
r
an
k
test
d
id
n
o
t
r
ea
ch
s
ig
n
if
ican
ce
(
p
=
0
.
0
6
6
)
.
Dif
f
er
en
ce
s
in
ac
cu
r
ac
y
(
Δ
M
ea
n
=
–
0
.
0
3
3
,
p
=
0
.
0
5
4
)
a
n
d
s
p
ec
if
icity
(
Δ
Me
an
=
–
0
.
0
6
9
,
p
=
0
.
0
7
3
)
f
a
v
o
r
e
d
R
esNet
-
5
0
b
u
t
d
i
d
n
o
t
r
ea
c
h
s
tatis
tical
s
ig
n
if
ican
ce
.
Sen
s
itiv
ity
d
if
f
er
e
n
ce
s
wer
e
n
eg
li
g
ib
le
(
Δ
=
+
0
.
0
0
2
,
p
=
0
.
9
6
0
)
.
T
h
e
s
e
r
e
s
u
l
ts
s
u
g
g
e
s
t
t
h
a
t
w
h
i
l
e
E
f
f
i
ci
e
n
t
N
et
-
B
0
g
e
n
e
r
a
l
i
z
es
b
e
t
t
e
r
o
n
t
h
e
M
I
A
S
d
a
t
a
s
e
t
,
R
es
N
e
t
-
5
0
r
e
m
a
i
n
s
m
o
r
e
r
o
b
u
s
t
f
o
r
t
h
e
p
at
h
o
l
o
g
y
-
c
o
n
f
i
r
m
e
d
C
B
I
S
-
D
DSM
t
as
k
,
p
a
r
t
i
c
u
la
r
l
y
i
n
d
is
c
r
i
m
in
a
t
i
n
g
b
e
n
i
g
n
f
r
o
m
m
a
l
i
g
n
a
n
t
c
as
e
s
.
Ov
er
all,
th
e
s
tatis
tical
an
aly
s
is
in
d
icate
s
th
at
E
f
f
icien
tNet
-
B
0
y
ield
s
g
ain
s
o
n
MI
AS,
m
a
k
in
g
it
well
-
s
u
ited
f
o
r
s
cr
ee
n
in
g
-
o
r
ien
ted
c
lass
if
icatio
n
wh
er
e
b
alan
ce
d
m
etr
ics (
AC
C
,
F1
)
an
d
p
r
o
b
ab
ilit
y
r
an
k
in
g
(
AUC)
ar
e
ess
en
tial,
wh
er
ea
s
R
esNe
t
-
5
0
m
ain
tain
s
an
ad
v
an
ta
g
e
o
n
C
B
I
S
-
DD
SM,
s
u
p
p
o
r
tin
g
its
s
u
itab
ili
ty
f
o
r
d
iag
n
o
s
tic
co
n
tex
ts
wh
er
e
h
ig
h
er
AUC r
eliab
ilit
y
is
r
eq
u
ir
ed
.
3
.
3
.
Co
m
pa
riso
n wit
h
prev
io
us
s
t
ud
ie
s
T
ab
le
4
p
r
esen
ts
a
c
o
n
s
o
lid
at
ed
co
m
p
ar
is
o
n
o
f
t
h
e
p
r
o
p
o
s
ed
m
o
d
els
with
p
r
e
v
io
u
s
wo
r
k
s
o
n
b
r
ea
s
t
ca
n
ce
r
class
if
icatio
n
ac
r
o
s
s
d
if
f
er
en
t
d
atasets
an
d
m
o
d
alities
.
C
lass
ical
m
ac
h
in
e
lear
n
in
g
s
tu
d
ies,
s
u
ch
as
th
o
s
e
u
s
in
g
th
e
W
is
co
n
s
in
d
i
ag
n
o
s
tic
b
r
ea
s
t
ca
n
ce
r
(
W
DB
C
)
d
ataset
[
2
2
]
,
[
2
4
]
,
[
2
5
]
,
c
o
n
s
is
ten
tly
r
ep
o
r
ted
h
ig
h
ac
cu
r
ac
ies
in
th
e
r
an
g
e
o
f
9
6
%
–
9
9
%
d
u
e
to
th
e
r
elativ
ely
s
m
all
s
ize
an
d
h
o
m
o
g
e
n
eo
u
s
tab
u
lar
f
ea
tu
r
es.
Similar
ly
,
h
an
d
cr
af
ted
tex
t
u
r
e
-
b
ased
ap
p
r
o
ac
h
es,
in
clu
d
in
g
th
e
u
s
e
o
f
2
D
-
DW
T
an
d
GL
C
M
co
m
b
in
ed
with
ar
tific
ial
n
eu
r
al
n
etwo
r
k
s
(
AN
N)
,
d
em
o
n
s
tr
ated
co
m
p
etitiv
e
p
er
f
o
r
m
an
ce
o
n
m
am
m
o
g
r
ap
h
y
d
atasets
s
u
ch
as
MI
AS a
n
d
C
B
I
S
-
DDSM,
ac
h
i
ev
in
g
ac
cu
r
ac
ies ab
o
v
e
9
0
%
[
1
8
]
,
[
1
9
]
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SS
N
:
2
0
8
8
-
8
7
0
8
I
n
t J E
lec
&
C
o
m
p
E
n
g
,
Vo
l.
1
6
,
No
.
3
,
J
u
n
e
20
2
6
:
1
1
9
9
-
1
2
1
2
1208
T
ab
le
4.
C
o
m
p
a
r
is
o
n
o
f
th
e
p
r
o
p
o
s
ed
m
o
d
els with
p
r
ev
i
o
u
s
s
tu
d
ies o
n
b
r
ea
s
t c
an
ce
r
class
if
icatio
n
D
a
t
a
s
e
t
M
e
t
h
o
d
A
c
c
u
r
a
c
y
/
A
U
C
N
o
t
e
s
M
a
mm
o
g
r
a
ms (
D
D
S
M
)
[
6
]
C
N
N
(
C
A
D
s
y
st
e
m)
C
N
N
:
9
4
.
5
%;
R
e
sN
e
t
-
5
0
:
9
5
.
8
%
;
I
n
c
e
p
t
i
o
n
R
e
sN
e
t
-
V
2
:
9
7
.
5
%
D
L
o
u
t
p
e
r
f
o
r
me
d
c
o
n
v
e
n
t
i
o
n
a
l
C
A
D
;
i
mp
r
o
v
e
d
l
e
si
o
n
d
e
t
e
c
t
i
o
n
.
C
B
I
S
-
D
D
S
M
,
I
N
b
r
e
a
st
[
7
]
En
d
-
to
-
e
n
d
C
N
N
c
l
a
ss
i
f
i
e
r
A
U
C
:
0
.
9
1
(
C
B
I
S
-
D
D
S
M
)
,
0
.
9
8
(
I
N
b
r
e
a
st
)
S
e
n
s
i
t
i
v
i
t
y
:
8
6
–
8
7
%
,
s
p
e
c
i
f
i
c
i
t
y
:
80
–
9
6
%.
H
i
st
o
p
a
t
h
o
l
o
g
y
(
1
3
6
1
i
ma
g
e
s
)
[
1
4
]
Ef
f
i
c
i
e
n
t
N
e
t
(
C
N
N
)
A
c
c
u
r
a
c
y
:
9
4
.
2
6
%
P
r
e
c
i
s
i
o
n
:
9
4
%
,
R
e
c
a
l
l
:
9
4
%
.
M
R
I
(
1
4
8
0
s
c
a
n
s,
K
a
g
g
l
e
)
[
1
5
]
R
e
sN
e
t
-
50
A
c
c
u
r
a
c
y
:
9
2
.
0
1
%
M
R
I
c
l
a
ssi
f
i
c
a
t
i
o
n
(
h
e
a
l
t
h
y
v
s.
mal
i
g
n
a
n
t
)
.
M
a
mm
o
g
r
a
ms
[
1
7
]
Tr
a
n
sf
e
r
l
e
a
r
n
i
n
g
(
R
e
sN
e
t
-
5
0
v
2
)
A
c
c
u
r
a
c
y
:
9
3
.
4
%
F
i
n
e
-
t
u
n
e
d
R
e
sN
e
t
;
i
mp
r
o
v
e
d
sen
s
i
t
i
v
i
t
y
.
M
I
A
S
,
U
D
I
A
T
[
1
8
]
2D
-
D
W
T
+
G
LC
M
+
A
N
N
A
c
c
u
r
a
c
y
:
9
3
.
8
%
(
3
-
c
l
a
ss)
C
l
a
s
si
c
a
l
h
a
n
d
c
r
a
f
t
e
d
t
e
x
t
u
r
e
+
ANN.
M
a
mm
o
g
r
a
ms (
b
e
n
i
g
n
v
s.
mal
i
g
n
a
n
t
)
[
1
9
]
Te
x
t
u
r
e
+
A
N
N
A
c
c
u
r
a
c
y
:
9
5
.
8
3
%
Te
x
t
u
r
e
f
e
a
t
u
r
e
s
c
o
mb
i
n
e
d
w
i
t
h
ANN.
M
I
A
S
[
2
0
]
S
h
e
a
r
l
e
t
+
G
L
C
M
/
G
L
R
L
M
+
B
i
L
S
T
M
-
C
N
N
A
c
c
u
r
a
c
y
:
9
7
.
1
4
%
H
y
b
r
i
d
h
a
n
d
c
r
a
f
t
e
d
+
d
e
e
p
seq
u
e
n
t
i
a
l
f
e
a
t
u
r
e
s.
W
D
B
C
(
t
a
b
u
l
a
r
)
[
2
2
]
S
V
M
(
b
e
st
mo
d
e
l
)
A
c
c
u
r
a
c
y
:
9
6
.
2
5
%;
A
U
C
:
0
.
9
9
4
C
o
m
p
a
r
e
d
mu
l
t
i
p
l
e
M
L
c
l
a
ss
i
f
i
e
r
s.
W
D
B
C
[
2
4
]
M
L
c
l
a
ssi
f
i
e
r
s
(
S
V
M
,
R
F
,
N
B
)
A
c
c
u
r
a
c
y
:
9
6
.
5
%
R
F
a
n
d
S
V
M
sh
o
w
e
d
s
t
r
o
n
g
e
s
t
p
e
r
f
o
r
m
a
n
c
e
.
K
a
g
g
l
e
(
5
6
9
sa
mp
l
e
s)
[
2
5
]
S
V
C
,
R
F
,
X
G
B
o
o
st
A
c
c
u
r
a
c
y
:
9
8
.
7
%
(
X
G
B
o
o
st
)
M
L
b
a
s
e
l
i
n
e
w
i
t
h
f
e
a
t
u
r
e
-
b
a
se
d
l
e
a
r
n
i
n
g
.
C
B
I
S
-
D
D
S
M
[
2
7
]
D
e
e
p
C
N
N
(
t
u
n
e
d
w
i
t
h
a
u
g
me
n
t
a
t
i
o
n
)
A
c
c
u
r
a
c
y
:
9
9
.
4
8
%
C
l
o
u
d
-
b
a
se
d
t
r
a
i
n
i
n
g
;
h
y
p
e
r
p
a
r
a
me
t
e
r
o
p
t
i
m
i
z
a
t
i
o
n
.
M
a
mm
o
g
r
a
ms
[
2
9
]
R
e
sN
e
t
-
5
0
+
S
M
O
TE
A
c
c
u
r
a
c
y
:
9
9
%
(
b
a
l
a
n
c
e
d
)
,
9
0
%
(
i
mb
a
l
a
n
c
e
d
)
A
d
d
r
e
ss
e
d
c
l
a
ss
i
m
b
a
l
a
n
c
e
e
x
p
l
i
c
i
t
l
y
.
B
r
e
a
k
H
i
s
(
h
i
st
o
p
a
t
h
)
[
3
0
]
F
a
st
L
e
a
k
y
R
e
sN
e
t
-
C
I
R
A
c
c
u
r
a
c
y
:
9
8
.
9
4
%
O
u
t
p
e
r
f
o
r
m
e
d
R
e
sN
e
t
1
8
/
5
0
,
I
n
c
e
p
t
i
o
n
,
V
G
G
.
M
a
m
mo
g
r
a
ms
(
M
IA
S
)
R
e
sN
e
t
-
50
A
c
c
u
r
a
c
y
:
8
9
.
3
%
±
3
.
0
;
A
U
C
:
0
.
9
5
9
±
0
.
0
2
7
N
o
r
m
a
l
v
s
.
a
b
n
o
r
m
a
l
c
l
a
ssi
f
i
c
a
t
i
o
n
.
M
a
m
mo
g
r
a
ms
(
C
B
IS
-
D
D
S
M
)
R
e
sN
e
t
-
50
A
c
c
u
r
a
c
y
:
8
3
.
3
%
±
2
.
3
;
A
U
C
:
0
.
9
1
5
±
0
.
0
1
3
B
e
n
i
g
n
v
s
.
m
a
l
i
g
n
a
n
t
c
l
a
ssi
f
i
c
a
t
i
o
n
.
M
a
m
mo
g
r
a
ms
(
M
IA
S
)
E
f
f
i
c
i
e
n
t
N
e
t
-
B0
A
c
c
u
r
a
c
y
:
8
9
.
9
%
±
3
.
1
;
A
U
C
:
0
.
9
8
1
±
0
.
0
1
7
N
o
r
m
a
l
v
s
.
a
b
n
o
r
m
a
l
;
b
e
st
R
OC
–
AUC
.
M
a
m
mo
g
r
a
ms
(
C
B
IS
-
D
D
S
M
)
E
f
f
i
c
i
e
n
t
N
e
t
-
B0
A
c
c
u
r
a
c
y
:
8
0
.
3
%
±
1
.
9
;
A
U
C
:
0
.
9
0
2
±
0
.
0
1
6
B
e
n
i
g
n
v
s
.
m
a
l
i
g
n
a
n
t
c
l
a
ssi
f
i
c
a
t
i
o
n
.
N
o
t
e
s
:
R
e
s
u
l
t
s
a
r
e
r
e
p
o
r
t
e
d
f
r
o
m
h
e
t
e
r
o
g
e
n
e
o
u
s
d
a
t
a
set
s
(
mamm
o
g
r
a
p
h
y
,
h
i
st
o
p
a
t
h
o
l
o
g
y
,
M
R
I
,
W
D
B
C
)
.
D
i
r
e
c
t
c
o
mp
a
r
i
s
o
n
s a
r
e
n
o
t
st
r
i
c
t
l
y
v
a
l
i
d
d
u
e
t
o
d
a
t
a
se
t
c
o
m
p
l
e
x
i
t
y
,
b
u
t
t
h
e
t
a
b
l
e
i
l
l
u
s
t
r
a
t
e
s
p
e
r
f
o
r
ma
n
c
e
t
r
e
n
d
s
a
c
r
o
s
s c
l
a
ss
i
c
a
l
M
L
,
h
a
n
d
c
r
a
f
t
e
d
f
e
a
t
u
r
e
s,
a
n
d
d
e
e
p
C
N
N
s.
T
h
i
s
st
u
d
y
f
o
c
u
s
e
s s
p
e
c
i
f
i
c
a
l
l
y
o
n
m
a
mm
o
g
r
a
p
h
y
(
M
I
A
S
a
n
d
C
B
I
S
-
D
D
S
M
)
.
W
ith
th
e
ad
v
e
n
t
o
f
d
ee
p
lea
r
n
in
g
,
C
NN
-
b
ased
ap
p
r
o
ac
h
es
h
av
e
s
ig
n
if
ica
n
tly
a
d
v
an
ce
d
t
h
e
s
tate
o
f
b
r
ea
s
t
ca
n
ce
r
C
AD.
Al
-
an
tar
i
et
a
l.
[
6
]
ac
h
iev
e
d
u
p
to
9
7
.
5
%
ac
cu
r
ac
y
u
s
in
g
C
NN
v
a
r
ian
ts
o
n
th
e
DDSM
d
ataset,
wh
ile
Sh
en
et
a
l.
[
7
]
d
em
o
n
s
tr
ated
th
at
en
d
-
to
-
en
d
C
NN
class
if
ier
s
tr
ain
ed
o
n
C
B
I
S
-
DDSM
an
d
I
Nb
r
ea
s
t
y
ield
ed
AUC
v
alu
e
s
o
f
0
.
9
1
an
d
0
.
9
8
,
r
esp
ec
tiv
ely
,
s
u
r
p
ass
in
g
m
a
n
y
h
an
d
c
r
af
ted
ap
p
r
o
ac
h
es.
R
ec
en
t
s
tu
d
ies
lev
er
ag
in
g
E
f
f
icien
tNet
ar
ch
itectu
r
es
also
a
ch
iev
ed
s
tr
o
n
g
r
esu
lts
:
T
asy
a
et
a
l.
[
1
4
]
r
e
p
o
r
te
d
9
4
.
2
6
%
ac
cu
r
ac
y
o
n
h
is
to
p
ath
o
lo
g
y
im
ag
es,
an
d
B
h
u
v
a
n
eswar
i
et
a
l.
[
1
3
]
s
h
o
we
d
n
ea
r
9
9
%
p
er
f
o
r
m
a
n
ce
o
n
m
am
m
o
g
r
a
m
s
u
s
in
g
E
f
f
icien
t
Net
-
B
3
.
Fu
r
th
er
m
o
r
e,
a
d
v
an
c
ed
R
esNet
v
ar
ian
ts
h
av
e
also
b
ee
n
ef
f
ec
tiv
e
i
n
o
th
er
im
ag
i
n
g
m
o
d
alities
s
u
ch
as
MRI
an
d
h
is
to
p
ath
o
l
o
g
y
,
with
Das
et
a
l.
[
1
5
]
an
d
Z
en
g
et
a
l.
[
3
0
]
r
e
p
o
r
tin
g
ac
cu
r
ac
ies ab
o
v
e
9
2
% a
n
d
9
8
%,
r
esp
ec
tiv
ely
.
C
o
m
p
ar
ed
t
o
th
ese
wo
r
k
s
,
o
u
r
s
tu
d
y
p
r
o
v
id
es
a
d
ir
ec
t
ev
alu
atio
n
o
f
R
esNet
-
5
0
an
d
E
f
f
icien
tNet
-
B
0
o
n
p
u
b
lic
m
am
m
o
g
r
ap
h
y
d
atasets
(
MI
AS
an
d
C
B
I
S
-
DDS
M)
.
On
t
h
e
MI
AS
d
ataset,
R
esNet
-
5
0
ac
h
iev
e
d
8
9
.
3
%
ac
c
u
r
ac
y
with
a
n
R
OC
–
AUC
o
f
0
.
9
5
9
,
wh
ile
E
f
f
i
cien
tNet
-
B
0
s
lig
h
tly
o
u
t
p
er
f
o
r
m
ed
it
with
8
9
.
9
%
ac
cu
r
ac
y
an
d
th
e
h
ig
h
est
R
O
C
–
AUC
o
f
0
.
9
8
1
.
On
th
e
m
o
r
e
ch
allen
g
in
g
C
B
I
S
-
DDSM
d
ataset
(
b
en
ig
n
v
s
.
m
alig
n
an
t)
,
R
esNet
-
5
0
o
b
tai
n
ed
8
3
.
3
%
ac
cu
r
ac
y
an
d
0
.
9
1
5
R
OC
–
AUC
,
o
u
tp
er
f
o
r
m
in
g
E
f
f
icien
tNet
-
B
0
,
wh
ich
ac
h
iev
ed
8
0
.
3
%
ac
cu
r
ac
y
an
d
0
.
9
0
2
R
OC
–
AUC
.
T
h
ese
r
esu
lts
h
ig
h
lig
h
t
th
at
wh
ile
E
f
f
icien
tNet
-
B
0
ac
h
iev
ed
h
i
g
h
er
R
OC
–
AUC
o
n
n
o
r
m
al
v
s
.
a
b
n
o
r
m
al
class
if
icatio
n
,
R
esNet
-
5
0
r
em
ain
s
m
o
r
e
r
o
b
u
s
t
f
o
r
d
is
cr
im
in
atin
g
b
en
i
g
n
an
d
m
al
ig
n
an
t c
ases
.
I
t
i
s
i
m
p
o
r
t
a
n
t
t
o
n
o
t
e
t
h
a
t
T
a
b
l
e
4
i
n
c
l
u
d
e
s
r
es
u
l
ts
o
b
t
a
i
n
e
d
f
r
o
m
h
e
t
e
r
o
g
e
n
e
o
u
s
d
a
t
a
s
et
s
s
u
c
h
a
s
W
DB
C
(
ta
b
u
l
a
r
)
,
h
is
t
o
p
a
t
h
o
l
o
g
y
,
M
R
I
,
a
n
d
m
a
m
m
o
g
r
a
p
h
y
.
D
i
r
e
ct
c
o
m
p
a
r
is
o
n
a
c
r
o
s
s
t
h
es
e
m
o
d
a
li
t
i
es
i
s
n
o
t
s
t
r
i
ct
l
y
v
a
li
d
,
as
d
a
ta
s
et
s
i
z
e
,
c
o
m
p
l
e
x
i
t
y
,
a
n
d
i
m
a
g
i
n
g
c
h
a
r
a
c
t
e
r
is
t
i
cs
d
i
f
f
e
r
s
u
b
s
t
a
n
ti
a
l
l
y
.
I
n
g
e
n
e
r
a
l
,
h
i
g
h
e
r
a
c
c
u
r
a
c
i
es
r
e
p
o
r
t
e
d
o
n
W
DB
C
a
n
d
h
i
s
t
o
p
a
t
h
o
l
o
g
y
d
a
t
as
e
ts
o
f
te
n
r
e
f
l
e
c
t
l
o
w
e
r
i
n
t
e
r
-
p
a
t
i
e
n
t v
a
r
i
a
b
i
l
it
y
c
o
m
p
a
r
e
d
t
o
m
a
m
m
o
g
r
a
p
h
y
,
w
h
i
c
h
is
c
li
n
i
c
a
l
l
y
m
o
r
e
c
h
a
l
le
n
g
i
n
g
d
u
e
t
o
s
u
b
t
le
l
es
i
o
n
a
p
p
e
a
r
a
n
c
es
a
n
d
i
m
a
g
i
n
g
n
o
i
s
e.
N
e
v
e
r
t
h
e
l
es
s
,
t
h
is
c
o
n
s
o
l
i
d
ate
d
c
o
m
p
a
r
i
s
o
n
i
l
l
u
s
t
r
a
te
s
t
h
e
p
e
r
f
o
r
m
a
n
c
e
l
a
n
d
s
c
a
p
e
a
cr
o
s
s
c
l
as
s
i
c
a
l
M
L
,
h
a
n
d
c
r
a
f
t
e
d
t
e
x
t
u
r
e
m
e
t
h
o
d
s
,
a
n
d
m
o
d
e
r
n
C
N
N
a
r
c
h
i
t
e
c
t
u
r
es
.
W
it
h
i
n
t
h
is
l
a
n
d
s
c
a
p
e
,
o
u
r
s
t
u
d
y
d
e
m
o
n
s
t
r
a
t
e
s
Evaluation Warning : The document was created with Spire.PDF for Python.