In
te
r
n
ation
a
l Jou
rn
al
o
f E
v
al
u
a
t
i
on
a
n
d
R
e
se
arc
h
in
Ed
u
c
ation
(
IJERE
)
Vol.
8
, No. 2, J
une
201
9
,
p
p
.
3
56
~3
62
ISSN
: 2252-
88
22,
D
O
I
:
10.11
59
1
/ijer
e
.
v8i
2.
187
02
356
Jou
rn
a
l
h
o
me
pa
ge
:
ht
tp:
//i
a
e
score
.
com
/
j
o
u
r
na
l
s
/
i
n
d
e
x
.
p
hp/IJ
ERE
O
u
t-of-s
chool time STEM pr
o
gr
am
: St
udents’ at
t
itudes tow
a
rd
and ca
reer interests i
n
m
athematics and science
Gu
an
Kun
g S
a
w
1
,
Bren
dan
S
w
agert
y
2
, Sho
n
B
r
ewing
t
on
3
,
Ch
i
-
Ni
n
g
C
h
a
ng
4
, Ryan Culbertson
5
1,
2
,
3
De
pa
rtme
nt o
f
Edu
c
a
tio
na
l
Psy
c
ho
lo
g
y
,
Un
iv
e
r
sity o
f
Te
x
a
s a
t
Sa
n
A
n
t
o
ni
o,
U
S
A
4
Depart
m
e
nt
o
f E
d
u
catio
nal
Psycho
lo
gy,
Tex
as A
&M
U
nivers
ity,
U
S
A
5
Inst
itute for
P
-
2
0 Initi
a
t
i
v
es,
U
n
iv
ersity
o
f T
e
x
a
s
at S
an
Anto
nio,
U
S
A
Ar
ticle
In
fo
ABSTRACT
A
r
t
i
cl
e hi
sto
r
y:
Rec
e
i
ve
d Mar
8,
201
9
Re
vise
d A
p
r
25,
201
9
A
c
cepte
d
Ma
y 4,
201
9
In
ternat
io
nally,
o
u
t
-
of
-sch
oo
l
time
(
OST
)
s
ci
ence,
t
ech
no
log
y
,
eng
i
n
eering
,
an
d
m
a
th
em
a
tic
s
(STE
M)
p
rog
r
ams
abo
und
.
Howev
e
r,
r
ig
orous
e
v
i
d
e
nce
of
t
h
ei
r
im
p
acts
o
n
s
t
u
d
e
nt
o
u
t
com
e
s
is
s
carce.
T
his
stu
d
y
ev
al
uat
ed
t
he
r
e
l
a
t
i
o
n
s
hi
ps
b
et
w
e
en
O
S
T
S
T
E
M
p
r
o
g
ra
m
pa
rt
ic
ip
a
tion
a
n
d
s
t
ud
e
n
t
m
o
tiv
a
ti
on
al
f
a
c
to
rs
i
n
m
a
t
h
a
n
d
s
ci
ence
b
y
a
n
a
lyzi
ng
s
u
r
vey
an
d
ad
m
i
n
i
s
t
rati
ve
d
ata
of
1
.0
17
midd
le
s
cho
o
l
st
ud
en
ts
w
h
o
p
artici
pat
e
d
i
n
t
he
s
e
ven-w
eek,
S
TEM
-f
ocu
s
ed
P
ref
r
es
h
m
an
E
n
g
ineerin
g
P
r
o
g
ram
(P
REP
)
i
n
Sa
n
Anton
i
o
,
T
e
x
a
s
,
from
2
01
5
t
o
2
0
1
7
.
Mu
l
tiple
r
e
g
re
ssio
n
re
su
lt
s
in
d
i
cat
ed
t
h
at
t
he
P
R
E
P
partici
p
ation
was
positive
l
y
associ
ated
w
i
t
h
s
t
ud
ents
’
a
t
tit
ud
es
t
o
w
a
rd
m
ath
a
nd
i
n
t
e
rests
in
m
ath-rel
a
ted
careers
,
w
h
e
rea
s
t
h
e
effects
o
n
students’
at
titud
e
s
t
o
ward
s
cie
n
c
e
a
nd
car
eer
i
n
t
erests
i
n
sc
ie
n
ce
were
n
e
gl
ig
i
b
l
e
.
No
e
vi
den
ce
was
f
o
u
n
d
t
o
s
ug
gest
t
h
a
t
the
a
s
so
cia
t
i
ons
b
etw
een
P
R
EP
p
art
i
c
i
p
a
ti
on
and
stud
ent
m
o
tiv
ati
onal
f
acto
r
s
in
m
ath
and
s
c
i
ence
d
i
ff
ered
by
gend
er, race/et
hn
icity, or soci
oeco
no
mi
c st
a
t
u
s
.
Key
w
ords:
Mi
ddl
e sc
h
ool
stu
d
e
nt
s
Mu
l
tip
le
r
egre
ssion
S
TEM
m
ot
ivation
S
TEM
s
umm
e
r progr
am
Co
pyr
i
g
h
t
©
2019
Instit
u
t
e of Advan
ced E
n
g
i
neer
in
g a
nd Sci
e
nce.
All rights
reserv
e
d
.
C
o
rre
s
po
n
d
in
g Au
t
h
or:
Gu
an
Saw,
D
e
partm
e
nt o
f
Ed
uca
t
i
ona
l P
s
ych
o
l
o
gy,
Uni
v
er
si
ty o
f Texa
s at
S
an
A
nt
o
n
io,
50
1 W.
C
e
s
a
r
E
. Chave
z
Bl
v
d
.
S
a
n A
n
to
n
i
o
,
TX
782
07
,
U
S
A.
Em
ail
:
gua
n.
sa
w
@
u
t
sa
.e
du.
1.
I
N
TR
OD
U
C
TI
O
N
En
ha
nc
i
ng
sc
ie
nc
e,
t
ec
h
nol
og
y, eng
inee
r
i
n
g
, and
m
a
the
m
a
tics (S
T
EM
)
education
a
n
d
workforce
has
bee
n
a
p
riori
t
y
f
or
m
a
ny
c
o
u
n
t
r
i
e
s
to
c
om
p
e
te
i
n
t
h
e
gl
ob
a
l
e
co
n
o
m
y
[
1
,
2
]
.
T
h
e
i
n
t
e
r
e
s
t
i
n
i
m
p
r
o
v
i
n
g
S
T
E
M
lear
n
i
n
g
an
d
pa
r
tic
ipa
t
ion ha
s l
e
d
t
o
the
c
rea
t
i
o
n
a
n
d
gr
o
w
t
h
o
f
various
in-
and
out-of-school
p
rogr
am
s
aim
e
d
at inc
rea
s
i
ng t
h
e
num
ber of yo
ung st
ude
nts p
u
rsu
i
ng a co
lle
ge d
e
g
re
e and ult
i
ma
te
ly
a
c
a
r
ee
r i
n
STEM fiel
d
s.
In
t
he
U
n
i
t
e
d
S
t
ates,
for
exa
m
ple,
e
stab
l
i
s
h
i
n
g
more
i
nc
l
u
s
i
ve
S
TEM
h
i
g
h
sc
hoo
ls
r
e
p
res
e
nt
s
on
e
rap
i
d
l
y
expa
n
d
i
n
g
in-
s
c
h
o
o
l
S
TEM
r
eform
progr
a
m
[
3
,
4
]
.
I
n
t
h
e
ou
t-of-s
c
h
o
o
l
c
on
te
xts,
m
any
pub
li
c
an
d
pri
v
ate
orga
niza
t
i
o
n
s
offer
S
TEM
o
utrea
c
h
p
rogra
m
s
dur
i
ng
sch
o
o
l
s
umm
e
r
hol
i
d
ay
s
(
J
une
t
o
A
ugus
t)
t
h
a
t
ar
e
d
e
si
gn
ed
t
o
s
t
imu
l
at
e
o
r
m
ai
n
t
a
i
n
an
i
n
t
e
r
est
i
n
S
TEM
c
a
r
ee
rs
a
mong
stude
n
t
s
a
t
t
he
e
lem
e
nt
ary,
s
e
c
on
dar
y
,
and
p
o
s
t
se
c
o
n
d
ary
leve
l.
T
he
se
S
TEM
s
umm
e
r
pr
o
g
r
a
m
s
t
y
p
i
cal
l
y
i
nc
lu
de
S
TEM-foc
u
s
e
d
l
e
a
rn
in
g
acti
v
i
t
ies
suc
h
a
s
m
a
th
/sc
i
ence
e
nr
ic
h
m
e
n
t
cur
r
ic
ul
u
m
,
S
TEM
c
a
r
e
e
r
se
mina
r
s,
a
nd
fi
e
l
d
tri
p
s
t
o
S
TEM
p
r
o
fe
ssi
ona
l
set
tin
gs [5-7].
A
l
t
h
o
u
gh
o
u
t-
of-sc
h
o
o
l
time
(
O
S
T
)
S
TEM
p
r
ogr
am
s
a
b
o
und
i
n
t
he
U
S
and
inter
n
at
iona
l
l
y,
r
igoro
u
s
rese
arc
h
on t
h
e
i
r
e
ffe
ct
s on s
t
ude
n
t
o
utc
o
m
e
s i
s
li
m
i
t
e
d
. The e
xisting
evi
d
ence is
m
i
x
e
d
a
t bes
t
. W
hil
e
several
stud
ies
ge
ne
ral
l
y
fo
un
d
p
o
sit
i
v
e
effec
t
s
o
n
s
tude
nt
c
og
nit
i
v
e
a
nd
m
o
tiva
t
i
ona
l
o
u
t
come
s
[7-13],
othe
rs
f
ai
le
d
to de
t
e
c
t
a
n
e
f
f
ec
t
o
f
O
ST STEM
progra
m
p
artic
ipa
t
io
n on des
i
ra
ble s
t
ude
nt
S
TE
M o
u
t
c
o
me
s,
s
uc
h
as S
TEM
in
t
e
rest a
nd c
o
l
l
ege
m
a
j
o
r [1
4
-
16].
The
mixed
fi
n
d
in
gs
o
f p
r
ior
s
t
u
d
i
es
s
t
e
mm
ed
i
n
part
from
re
search
d
esign
s
tha
t
r
elie
d
on
s
m
all
sam
p
l
e
s
iz
es
a
nd
im
prec
ise
ou
tc
om
e
measure
s.
T
he
re
i
s
cle
a
rl
y
the
n
e
e
d
f
or
m
ore
e
m
pirica
l
Evaluation Warning : The document was created with Spire.PDF for Python.
Int J
Eval & R
es E
d
u
c
.
IS
S
N
:
2252-
88
22
O
u
t-o
f
-sch
o
o
l t
i
m
e
ST
EM
program
:
St
u
d
en
ts’
att
i
t
u
des
tow
a
rd a
n
d
c
a
reer
inte
re
st
s in
…
(G
uan S
a
w
)
35
7
stud
ies
on
t
h
e
i
m
pac
t
o
f
O
S
T
S
TEM
p
ro
gram
s
w
i
t
h
l
a
r
ger
st
ud
e
n
t
sam
p
les
an
d
m
o
re
v
ali
d
a
nd
relia
ble
m
easure
s
.
2.
LI
T
E
R
A
TU
R
E
R
EV
I
E
W
O
S
T
S
TEM
p
rogra
m
s
in
t
he
U
S
A
a
nd
ac
ross
the
g
l
o
b
e
t
y
p
i
c
a
l
l
y
s
e
e
k
to
i
n
c
re
a
s
e
st
ud
ent
s
’
a
t
ti
tud
e
s
tow
a
rd
a
n
d
car
eer
i
nter
es
ts
i
n
S
TEM
t
hro
u
gh
var
i
o
u
s
a
ppr
oac
h
e
s
.
M
ost
O
S
T
S
TEM
p
ro
gr
am
s
de
velo
p
a
S
TEM-ba
s
e
d
c
urricu
lum
t
o
i
m
p
rove
s
t
u
d
e
nts’
c
r
i
tica
l
t
hin
k
i
n
g
sk
i
l
l
s
a
nd
k
n
o
w
l
e
d
ge
a
s
w
e
ll
a
s
t
he
ir
moti
va
t
i
o
n
a
l
fa
c
tors i
n
STEM.
These
curr
i
c
u
l
a
ten
d
t
o
inc
l
u
d
e
a
ser
i
e
s of
t
e
a
c
h
in
g
an
d le
a
r
ni
ng ac
tiv
it
i
e
s such
as
a
dva
nc
ed
c
o
u
rsew
ork,
h
an
ds-o
n
ac
t
i
v
itie
s,
a
nd
gr
ou
p
proje
c
t
s
[
17-
1
9
]
.
F
or
e
xam
p
l
e
,
N
i
em
ann,
M
i
l
l
e
r
,
a
nd
D
a
vis
[1
2]
r
ep
orte
d
tha
t
t
he
S
umm
e
r
S
c
ien
c
e
Inst
i
t
u
t
e,
o
ff
ere
d
by
the
Uni
v
e
r
s
ity
o
f
Alabam
a
at
B
irm
i
ng
ham
,
A
l
abam
a,
w
a
s
a
t
hre
e
-ye
a
r
progress
i
on
of
s
umm
e
r
progra
m
s
desig
n
e
d
t
o
intr
od
uc
e
h
i
gh
sc
h
ool
s
tu
d
e
nt
s
to
incre
a
s
i
ng
ly
m
or
e
rigoro
u
s
sc
i
e
n
tific
c
o
n
ce
p
t
s
an
d
resea
r
ch
s
ki
l
l
s
t
h
r
o
ug
h
l
a
bo
rato
ry
-b
a
s
ed
a
dv
an
c
e
d
sci
e
n
ce
course
w
o
r
k
o
v
e
r
the
co
ur
se
o
f
t
h
re
e
su
mmer
s
(si
x
-we
e
k
,
t
hree
-d
a
y
pe
r
w
eek co
u
r
s
e
ea
ch
s
um
m
e
r)
. The st
u
d
y
fo
u
nd
tha
t
stud
e
nts w
ho
r
e
turned ea
c
h year
w
e
r
e
rate
d
a
s
bein
g
mo
re
m
o
t
i
v
at
ed
i
n
l
e
a
r
nin
g
s
c
i
e
n
c
e
co
mp
are
d
w
ith first-ye
a
r atte
n
d
ee
s b
y
m
ent
o
rs,
a
n
d s
t
ude
n
t
s overa
l
l
d
eve
l
o
ped a dee
p
u
nder
s
ta
nd
in
g of
w
ha
t
i
t
i
s
l
i
ke
to
be
a
s
cien
t
i
st
.
Sim
i
l
a
rl
y,
s
e
v
era
l
o
the
r
s
tu
die
s
b
ased
i
n
va
ri
o
us
s
tate
s
i
n
t
h
e
U
S
and
foc
u
sin
g
o
n
S
TEM
l
ear
ning
[9,
1
7
,
20]
c
o
n
cl
u
d
ed
t
hat
t
h
eir
OST
S
T
EM
p
ro
gram
s’
c
u
r
ricu
lum
p
o
s
i
tive
l
y
im
pact
ed
s
t
u
den
t
s’
a
t
tit
ude
s
toward
a
nd career
int
e
rest in
STEM.
I
n
a
ddit
i
on
to
r
i
g
or
ou
s
cour
se
w
o
rk
o
r
e
n
ric
h
m
e
nt
l
e
a
rn
in
g
a
c
t
i
v
itie
s,
m
any
O
S
T
STEM
p
ro
gra
m
s
cre
a
te
o
p
p
or
t
u
ni
t
i
es
f
or
s
tud
e
nts
to
b
e
ex
p
o
s
e
d
t
o
r
ea
l
-
w
o
rl
d
S
T
E
M
p
r
o
f
e
s
s
i
o
n
a
l
s
a
n
d
s
e
t
t
i
n
g
s
,
w
h
i
c
h
a
r
e
o
f
t
e
n
una
va
ila
b
l
e
d
u
r
ing
regu
lar
scho
o
l
h
ours
[6].
T
hese
r
ea
l-w
o
rld
S
T
E
M
e
xpo
s
u
r
e
a
cti
v
iti
e
s
i
n
cl
ud
e
STEM c
are
e
r
sem
i
na
rs
f
e
a
t
u
rin
g
p
r
o
fess
i
o
nal
s
f
rom
various
S
TEM
f
ie
ld
s
and
S
TEM-
r
ela
t
e
d
f
iel
d
t
r
i
ps
s
uc
h
a
s
v
is
it
ing
scie
nc
e
m
u
seu
m
s,
n
ature
ce
n
t
e
r
s,
a
nd
inf
o
r
m
a
t
i
o
n
tech
n
o
l
o
g
y
c
om
pa
n
i
e
s
.
A
grow
in
g
bod
y
o
f
r
ese
a
rc
h
has
repor
t
e
d
t
h
a
t
t
h
e
se
oppor
tun
i
tie
s
of
e
xpos
ure
to
r
e
a
l-w
o
rld
S
T
E
M
pr
ofe
ssi
ona
l
s
a
nd
se
tti
ng
s
can
boost
stu
d
en
ts
’
att
itu
des
tow
a
r
d
a
n
d
c
are
e
r
in
tere
st
s
in
S
TE
M
[9,
21,
2
2]
.
For
insta
n
ce
,
Mo
hr-S
chroe
d
er
a
nd
c
o
llea
g
ues
[22]
dem
o
n
s
t
r
ate
d
t
hat
m
i
dd
le
s
ch
oo
l
st
u
d
en
ts
w
ho
pa
r
t
i
c
ipa
t
e
d
i
n
a
S
TEM
s
umm
e
r
progra
m
i
n
K
e
ntuc
ky
e
i
t
h
er
deve
l
ope
d
an
i
nt
e
r
es
t
i
n
S
TEM
f
ie
lds
or
t
he
i
r
i
nter
es
t
in
S
TEM
w
a
s
he
i
g
h
t
ene
d
t
hr
oug
h
t
our
in
g
an
d
c
o
n
d
uc
t
i
n
g
han
d
s-o
n
l
e
a
rn
in
g
act
iv
it
ies
in
e
ng
i
n
ee
ri
ng
l
abora
t
ori
e
s
a
nd
fa
c
i
lit
ies,
l
ed
by
e
n
gine
e
r
in
g
e
xpert
s.
E
lam
,
D
o
n
h
am
,
and
S
o
loma
n
[6]
d
o
cum
e
n
t
e
d
t
ha
t
midd
le
a
nd
hi
g
h
s
cho
o
l
st
ude
n
t
s
w
ho
att
e
n
d
e
d
a
t
w
o
-w
e
e
k
eng
i
nee
r
i
n
g
su
m
m
er
p
r
ogram
i
n
Texa
s
re
ported
an
i
nc
rea
s
e
i
n
c
a
r
ee
r
i
n
t
e
r
e
sts
in
s
c
i
en
ce
a
nd
e
ngi
ne
e
r
in
g,
w
h
ic
h
co
u
l
d
p
a
rtia
ll
y
be
e
x
p
l
a
ine
d
by
t
h
e
i
r
posi
t
i
ve
e
xpe
r
i
e
n
ce
s
from
interacting
w
i
th
p
rofe
ssio
n
a
l
s
ci
ent
i
s
t
s
and
e
n
g
i
n
e
ers
as
w
ell
as
v
i
s
i
tin
g
a
fo
od
m
a
nu
fa
c
t
uri
n
g
p
l
ant
a
n
d
a
co
ns
truc
ti
on
site
.
T
a
ken
t
o
ge
t
h
e
r
,
the
s
e
st
u
d
i
e
s
off
e
r
evi
d
en
c
e
t
h
a
t
expo
si
ng
s
tud
e
nt
s
to
S
TEM
p
rof
e
ssio
n
a
l
s
a
n
d
e
n
v
i
r
o
n
m
e
n
t
s
c
a
n
b
o
o
s
t
t
h
e
i
r
a
t
t
i
t
u
d
e
s
toward
a
nd career
int
e
rests
in
S
TEM.
D
e
spite
i
nform
a
ti
v
e
f
in
d
i
n
g
s,
p
re
vi
o
u
s
st
ud
i
e
s
a
r
e
l
i
m
i
t
e
d
in
t
erm
s
o
f
meth
odo
l
o
g
i
ca
l
is
sues.
They
w
e
r
e
l
arge
ly
d
e
s
crip
ti
ve
i
n
nat
u
re,
and
t
h
e
i
r
st
ud
ie
d
sam
p
l
e
s
t
en
de
d
t
o
b
e
s
m
all
(ty
p
i
c
a
l
l
y
c
om
pose
d
o
f
n
o
m
ore
tha
n
a
f
e
w
doze
n
s
of
p
a
r
tic
i
p
ant
s
)
.
M
os
t
stu
d
i
e
s
used
o
n
l
y
si
ng
le-i
tem
surve
y
m
e
a
sure
s
to
a
ssess
the
im
pact
o
f
S
TEM
s
umm
e
r
progr
am
s
on
st
ude
n
t
o
ut
c
o
me
s,
w
hi
c
h
h
a
v
e
l
o
w
leve
l
o
f
m
e
a
s
u
r
e
m
e
n
t
r
e
l
i
a
b
i
l
i
t
y
a
n
d
v
a
l
i
d
i
t
y
.
More
ove
r, stud
e
nt
sam
p
l
e
s
in pr
ior
stu
d
i
e
s
w
e
r
e not
s
oc
io
de
m
ogr
a
p
h
i
c
a
l
l
y
d
i
v
erse
,
a
n
d
t
h
e
r
ef
o
r
e,
h
ad
l
imi
t
ed
gene
ra
liza
b
il
i
t
y
a
n
d
o
f
fer
e
d
no
est
i
ma
tes
for
di
ffere
n
t
soc
i
od
e
m
ogr
a
p
h
i
c
s
u
b
g
r
o
ups,
w
h
ich
c
o
ul
d
be
in
form
ative
evi
d
enc
e
f
or
unde
r
s
t
a
n
d
in
g
the
cha
n
ges
i
n
S
TEM
moti
va
ti
ons
a
mong
u
n
d
err
e
pre
s
e
n
ted
gro
u
p
s
[2
3,
24].
Impr
ovin
g
on
pr
ior
w
o
rk,
this
s
tud
y
a
ime
d
t
o
a
sses
s
t
he
a
s
so
ci
a
tio
n
s
b
et
we
en
O
ST
S
TEM
p
ro
g
r
a
m
part
ici
p
a
t
i
o
n
and
stude
n
t
s
a
t
t
i
t
ude
s
t
o
w
a
rd
a
nd
ca
ree
r
i
nte
r
e
s
t
s
in
S
TEM
by
analyz
i
ng
m
u
l
t
i
p
l
e
y
ea
rs
o
f
dat
a
from
a
S
TEM sum
me
r pr
ogram
ba
s
e
d
in S
a
n
A
n
t
o
ni
o,
T
exa
s
,
U
S
A
,
w
i
th a si
zable
a
nd div
er
se
stude
n
t
sa
m
pl
e
and
a r
i
c
h
set
o
f m
e
a
s
ures on
st
ude
n
t
s’
a
tt
i
t
u
d
es
t
ow
ard
and
c
a
re
e
r
i
n
t
e
r
est
s
in m
a
th a
n
d
scie
n
ce.
3.
METHOD
To in
v
es
t
i
ga
te
the rela
t
i
ons
h
i
ps betw
e
e
n
O
S
T
S
TEM progr
am par
t
i
c
i
pa
tio
n a
nd stu
d
e
n
t m
o
tiva
tio
na
l
fa
ct
ors,
t
hi
s
stu
dy
em
pl
oye
d
mult
ipl
e
r
e
g
ressi
on
m
ode
ls
w
i
t
h
co
v
a
ri
at
e
a
d
j
u
st
me
nt
s
t
o
a
nal
y
ze
m
u
l
ti
pl
e
y
e
a
r
s
of
p
artic
i
p
a
n
t d
a
t
a
fr
o
m t
h
e P
r
e
f
reshm
a
n En
g
i
ne
er
in
g P
r
ogra
m
(P
R
E
P
)
i
n San
Antoni
o,
T
exas, USA.
P
R
EP is
a
se
ven-w
e
ek
S
TEM
s
umm
e
r
progra
m
o
ffe
r
e
d
t
o
m
idd
l
e
a
nd
h
i
g
h
s
c
h
oo
l
stude
n
t
s
in
g
ra
des
six
t
o
e
le
ve
n,
over
t
h
e
c
o
u
r
s
e
o
f
f
o
u
r
s
u
m
m
e
r
s
w
i
t
h
d
i
f
f
e
r
e
n
t
l
e
v
e
l
s
o
f
S
T
E
M
c
u
r
r
i
c
ulum
a
nd
lea
r
n
i
ng
a
c
tiv
it
ies.
P
REP
w
a
s
fo
u
nde
d
at
t
he
U
niver
s
it
y
of
T
e
x
as
a
t
S
a
n
A
n
t
oni
o
(U
TS
A
)
i
n
19
7
9
t
o
p
repa
re
y
o
u
n
g
stud
en
ts,
espe
c
i
a
l
l
y
w
o
m
e
n,
unde
rrepr
esente
d
ra
cial
/e
t
h
n
i
c
min
o
ri
ti
e
s
(
U
R
M
,
w
ho
w
e
re
m
ost
l
y
B
l
ac
ks,
H
i
s
p
an
ic
s,
a
nd
N
a
ti
ve
A
m
erica
n
s,
a
s
op
p
o
se
d
to
W
hites
a
n
d
A
s
i
a
ns),
a
nd
lo
w
-
soci
oec
o
n
o
m
i
c
s
t
a
t
u
s
(
S
E
S
)
s
t
u
d
e
n
t
s
,
f
o
r
s
u
c
c
e
s
s
i
n
adva
nc
e
d
s
tu
d
i
e
s
l
e
a
di
n
g
t
o
car
eer
s
i
n
S
TEM.
I
n
t
he
s
umm
e
r
of
2
01
8,
t
he
re
w
e
r
e
40
P
R
EP
s
i
t
es
a
c
r
oss
the
state
of
T
e
x
as,
N
e
w
Me
xic
o
,
A
r
izona
,
Cal
i
forn
ia,
U
t
ah,
and
N
o
rt
h
D
a
ko
t
a
,
en
ro
ll
ing
ov
e
r
4
,0
00
st
ud
e
n
t
s
.
Thi
s
s
t
udy
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-8822
Int.
J
.
Ev
al. & R
es. Educ.
Vol.
8, No. 2, June
2
0
19:
3
56
- 3
62
35
8
foc
u
se
d
o
n
ly
o
n
t
h
e
P
R
EP
s
it
es
i
n
S
a
n
A
n
t
oni
o,
w
hi
c
h
s
u
r
veye
d
S
TEM
m
oti
v
a
t
i
o
ns
o
f
pa
rtic
i
p
an
ts
i
n
e
v
ery
summ
er since
20
1
5
.
P
R
EP
pursues
i
t
s
’
m
i
ssi
o
n
b
y
p
ro
vi
din
g
s
tud
e
nts
a
r
i
gor
o
u
s
S
TEM
-base
d
c
urricu
lum
ta
ug
ht
b
y
co
lle
ge
profe
s
sor
s
o
r
c
e
rtifie
d
tea
c
h
er
s
for
seve
n
w
e
eks
(fo
u
r
da
y
s
p
e
r
w
e
e
k
;
140
i
nst
r
u
c
ti
on
a
l
h
o
u
r
s
in
t
ot
a
l
)
d
u
r
i
ng
the
sum
m
e
r,
w
here
s
tu
den
t
s
a
t
te
n
d
p
rogra
m
a
c
tiv
i
t
i
e
s
M
o
n
d
ay
s
t
h
r
o
u
gh
T
hurs
d
ays
an
d
t
h
e
a
v
era
g
e
da
y
is
s
ix
to
s
eve
n
h
o
u
r
s
l
on
g
pl
us
h
o
m
ew
ork.
T
he
r
ig
oro
u
s
curr
i
c
ulum
c
o
n
s
is
ts
o
f
var
i
ou
s
ha
n
d
s-
o
n
a
c
t
i
v
itie
s,
g
roup
pro
j
ec
t
s
,
a
n
d
S
TEM
c
ourse
w
o
rk
t
ha
t
faci
l
i
ta
tes
the
grow
t
h
o
f
es
se
n
tia
l
a
c
a
d
em
i
c
a
n
d
c
og
n
iti
ve
s
k
i
ll
s
ne
ede
d
for
S
TEM
s
uc
c
e
ss.
I
n
p
a
rticu
l
ar,
P
R
EP
e
xp
os
e
s
m
idd
l
e
an
d
h
i
gh
s
c
h
o
o
l
s
t
ude
nts
to
a
dva
nc
e
d
c
o
u
rse
s
,
incl
u
d
in
g
lo
gic
an
d
a
p
plica
t
ion
s
t
o
m
a
them
at
ics,
i
ntr
o
d
u
c
t
i
o
n
t
o
e
n
g
i
n
ee
r
i
n
g
,
i
n
t
rod
u
c
tio
n
to
p
h
y
s
i
cs,
st
a
tis
tic
s
and
pro
b
ab
i
lit
y,
a
nd
in
tro
d
u
ct
i
on
to
c
om
p
u
t
e
r
s
cience
,
to
h
e
l
p
pre
p
a
r
e
st
ud
ent
s
f
o
r
a
d
v
a
n
ceme
n
t
i
n
S
TEM
.
P
R
E
P
a
l
so
p
ro
v
i
d
e
s
op
po
rtun
iti
e
s
f
or
s
t
u
d
e
nt
s
to
e
ng
a
g
e
with
S
TEM
p
r
ofe
s
si
on
a
l
s
t
h
roug
h
STEM
-re
l
a
t
e
d
f
i
e
ld
t
ri
ps
and
da
il
y
S
TEM
c
are
e
r
sem
i
nars.
O
v
e
r
t
he
c
ourse
o
f
the
s
e
ve
n
w
e
e
k
p
ro
gra
m
,
stude
nts
e
nga
ge
w
i
t
h
d
i
ffe
ren
t
S
T
E
M
p
r
o
f
e
ss
io
n
a
l
s
th
r
ou
gh
r
o
ugh
ly
t
w
o
d
oze
n
o
f
c
a
r
e
er
s
em
i
n
a
r
s
a
nd
at
l
ea
st
o
n
e
S
TEM-rela
t
e
d
fiel
d
trip
t
o
enha
nc
e
the
i
r
at
t
i
t
u
d
e
s
t
owar
d
and
car
eer in
t
e
r
e
s
ts in
STEM.
3.1.
Sa
mp
le
B
y
u
t
iliz
in
g
t
h
re
e
ye
ars
of
s
tude
n
t
d
ata
fro
m
P
REP
si
te
s
in
S
a
n
A
nto
n
i
o
(
201
5
to
2
0
17)
,
r
i
sin
g
8
th
grade
r
s
(7
th
g
r
a
der
s
i
n
pri
o
r
scho
ol
y
ea
r)
w
h
o
partic
ipa
t
e
d
i
n
P
R
EP
f
or
t
he
s
e
c
o
nd
tim
e
wer
e
i
de
n
t
i
f
i
e
d
a
s
t
h
e
“
t
re
atme
nt
g
ro
up,
”
w
h
ere
a
s
t
h
e
“
c
om
pariso
n
gr
oup”
c
o
n
si
ste
d
o
f
ri
s
i
ng
8
th
g
raders
w
ho
w
e
re
f
irst-tim
e
PREP
part
ici
p
a
n
t
s
.
In
o
the
r
w
ords,
st
ude
n
t
s
in
t
he
t
r
eatm
e
n
t
g
r
o
up
attended one
additional s
e
ven-
w
eek PREP
pro
g
ram
,
compar
ed
w
it
h
the
c
o
mpa
r
iso
n
g
rou
p
.
The
a
n
alytic
s
am
pl
e
of
t
h
i
s
s
tud
y
in
c
l
u
d
e
s
a
t
o
t
a
l
o
f
1
.017
rising
8
th
g
ra
de
r
s
,
cons
i
s
t
i
n
g
o
f
6
4
1
t
r
eate
d
a
nd
3
7
6
c
ompa
r
i
son
ca
se
s.
T
ab
le
1
s
h
o
w
s
d
es
crip
ti
ve
s
ta
tis
tic
s
on
t
h
e
d
e
mog
r
a
phi
c ch
a
r
ac
t
e
ri
sti
c
s o
f
st
udy
sa
m
p
l
es by
t
r
ea
tmen
t
s
ta
tus.
C
hi-s
quare
(
) test r
esul
t
s
i
nd
ica
t
e that
there
w
a
s
no
sig
n
i
f
i
c
a
n
t
dif
f
e
r
e
n
ce
by
gen
d
e
r
be
tw
een
t
re
ated
a
n
d
c
om
p
a
riso
n
grou
ps
(
56%
v
e
r
sus
5
2
%)
.
H
o
w
e
ve
r, the c
om
paris
on gro
up had dispr
o
p
o
rt
io
na
tel
y
m
o
r
e
under
repr
es
ente
d rac
i
a
l
/
e
thn
i
c m
i
nor
it
ies
(
81%)
and
low
-
S
E
S
stu
d
e
n
t
s
(
37%)
,
c
ompa
red
w
ith
t
he
tre
atme
n
t
gro
up (
76
% a
n
d 31
%,
r
e
s
pe
ctive
l
y).
T
a
b
l
e 1.
D
emogr
ap
hic c
h
ara
c
t
eris
t
i
cs of
s
t
u
d
y
s
am
p
l
e
(N
=1
.017)
T
r
e
a
t
e
d
C
om
pa
ri
s
o
n
n
=
641
n
=
376
p
-v
a
l
ue
M
ea
n
S
.
D.
M
ea
n
S.
D
.
(
t
e
s
t)
D
e
m
ogra
phi
c
s
Fe
m
a
le
.
56
.
50
.
52
.
50
.
207
U
R
M
.
76
.
43
.
81
.
39
.
071
L
o
w-SE
S
.
31
.
46
.
37
.
48
.
049
N
o
te
. N
/n
=
s
a
m
pl
e
s
i
ze
;
S
.
D
. =
s
ta
n
d
a
r
d
de
via
tion;
UR
M = unde
rre
prese
n
t
e
d r
a
c
i
a
l
m
inoritie
s;
SE
S
=
so
c
i
o
ec
ono
m
i
c
s
tatus.
3.2.
Mea
s
u
r
es
I
n
t
h
i
s
st
u
dy,
f
o
u
r
me
asure
s
on
st
u
d
en
t
S
TEM
m
ot
i
v
a
t
i
ons
a
re
u
s
e
d
a
s
ou
tcom
es.
The
f
i
r
s
t
tw
o
ou
tc
ome
m
easure
s
a
re
ma
t
h
a
t
ti
tud
e
s
and
s
c
ie
nce a
t
ti
tude
s
,
assessed
by
us
in
g
the
sca
l
e
s
o
f
stu
d
e
n
t
at
t
i
t
u
des
tow
a
rd
S
TEM,
de
ve
l
ope
d
b
y
U
nfr
i
e
d
a
n
d
c
o
l
le
a
g
ues
[25]
.
B
o
th
s
c
a
les
of
t
he
m
a
t
h
an
d
scienc
e
a
t
t
i
tude
s
con
s
ist
of se
v
en ite
ms w
i
t
h a five-p
o
i
nt
L
i
k
ert sca
l
e (“
strong
ly di
s
a
g
r
ee”
up to “
stron
g
l
y
a
gree
”).
Two exa
m
p
l
e it
e
m
s
for
m
easur
i
n
g
m
a
th
a
tt
it
u
d
es
a
re
:
“I
a
m
the
t
ype
o
f
s
t
u
d
e
n
t
to
do
w
e
l
l
i
n
m
a
th”
an
d
“
I
can
g
e
t
g
oo
d
gr
a
d
e
s
i
n
m
a
th.
”
T
w
o
e
x
a
m
p
le i
tem
s
f
or
a
ss
e
s
sin
g
s
c
i
e
n
ce
at
t
itu
de
s
ar
e: “
I
am sure of m
ysel
f when
I
do scie
nc
e” a
nd “
I
kn
ow
I
c
a
n
do w
e
ll i
n
scien
c
e
”
(
see Ta
bl
e
2
for
t
h
e
com
p
le
t
e
s
e
t of
i
t
e
ms
f
or
b
o
t
h
ma
t
h
a
nd
sc
ie
nce
at
t
itude
s)
.
The
t
h
ird
an
d
fo
urt
h
o
u
t
com
e
m
e
a
sures
are
interes
t
i
n
m
a
th-re
l
a
t
e
d
care
e
r
s
a
nd
i
n
terest i
n
sc
ie
nc
e-rela
te
d
caree
r
s
,
wh
i
c
h
were
a
sse
sse
d
b
y
ask
i
n
g
stude
n
t
s
t
o
r
e
s
p
ond
t
o
t
h
e
f
o
l
l
ow
in
g
t
w
o
sta
t
e
m
e
n
t
s
:
“I
w
oul
d
co
ns
id
e
r
cho
o
s
i
ng
a
c
a
re
er
t
ha
t
uses
m
ath”
a
n
d
“
I
w
oul
d
co
ns
i
d
e
r
a
c
a
r
ee
r
in
s
cie
n
ce”
(
five
-p
o
i
n
t
L
i
k
er
t
scale
:
“
st
r
o
ngl
y
di
sa
gree
”
u
p
t
o
“
s
t
r
on
g
l
y
ag
re
e”).
These
tw
o
item
s
w
e
r
e
code
d
i
n
t
o
bi
nary
v
aria
b
l
e
s
,
ind
i
ca
tin
g
w
h
e
t
her
a
stude
n
t
i
s
in
t
e
r
e
ste
d
i
n
ma
th- or
s
cienc
e
-r
ela
t
ed
car
eer
s.
3.3.
A
n
alytic
strat
egy
C
o
n
f
irm
a
t
o
ry
f
ac
tor
a
n
a
l
ys
i
s
(
CFA
)
w
as
a
pp
lie
d
t
o
t
e
s
t
the
val
i
d
i
t
y
a
nd
to
c
om
pu
te
t
he
s
t
a
nda
r
d
ize
d
com
p
o
s
ite
s
c
o
r
e
s
of
m
ath
a
n
d
scienc
e
a
t
t
i
t
u
d
e
s
[2
6].
To
e
valua
t
e
t
h
e
g
o
o
dne
ss-of-
f
i
t
o
f
CF
A
mode
l
,
H
u
an
d
Be
ntle
r
[2
7]
r
e
c
om
me
nde
d
tha
t
a
R
oo
t
Me
an
S
qua
re
E
rro
r
of
A
ppro
x
i
m
a
tio
n
(R
M
S
EA
)
at
o
r
belo
w
0.06
in
dic
a
t
e
s
a
goo
d
fi
t,
w
he
rea
s
C
ompa
rati
ve
F
it
I
nde
x
(
C
F
I
)
a
nd
T
uc
ke
r-Le
w
is
I
ndex
(
TLI)
gr
eate
r
t
han
0.
95
ar
e
preferr
e
d.
R
e
lia
bil
i
t
y
t
es
ts
w
er
e
c
onduc
te
d
t
o
c
alcu
la
te
C
ro
nba
c
h
’
s
a
lp
ha
c
oef
f
ic
ie
n
t
s
for
the
tw
o
cons
t
r
uct
s
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.
Int J
Eval & R
es E
d
u
c
.
IS
S
N
:
2252-
88
22
O
u
t-o
f
-sch
o
o
l t
i
m
e
ST
EM
program
:
St
u
d
en
ts’
att
i
t
u
des
tow
a
rd a
n
d
c
a
reer
inte
re
st
s in
…
(G
uan S
a
w
)
35
9
ma
th
a
n
d
s
cie
n
ce
a
tt
it
ude
s
[28
]
.
To
q
ua
n
t
i
f
y
the
r
e
la
t
i
on
sh
ips
of
S
TE
M
sum
m
e
r
progra
m
p
arti
ci
p
ati
o
n
an
d
stude
n
t
s’
a
tt
itu
de
s
tow
a
r
d
a
n
d
c
a
r
eer
i
nte
r
est
s
i
n
ma
t
h
a
nd
s
c
i
enc
e
,
m
u
l
t
i
p
le
r
egre
ssion
mode
l
s
w
ith
c
o
v
a
riate
adj
u
s
t
m
e
n
t
s
w
e
re
u
sed.
F
or
t
he
d
e
p
en
de
nt
v
a
r
iab
l
es
o
f
i
n
tere
sts
i
n
ma
th/sc
i
e
n
c
e
-r
elat
ed
c
are
e
rs
t
ha
t
we
re
d
i
cho
t
o
m
ou
s,
r
e
g
res
s
ion
mo
de
l
s
w
e
r
e
e
s
t
i
m
at
ed
w
it
h
a
l
i
n
ear
p
ro
b
a
bi
l
ity
m
o
d
e
l
[2
9
]
.
To
t
est
t
h
e
he
t
e
roge
ne
ou
s
e
f
f
e
c
t
s
o
f
P
R
E
P
,
a
s
e
r
i
e
s
o
f
r
e
g
ression
m
odels
wer
e
r
e-e
s
timat
e
d
by
a
d
d
in
g
t
h
e
i
n
tera
ctio
n
ter
m
b
e
t
w
e
en
trea
t
m
en
t
st
a
t
us
a
nd
soci
ode
m
ograp
hic
c
h
a
r
a
c
terist
i
c
s,
d
efine
d
by
gen
d
e
r
,
r
a
ce/e
t
h
n
i
c
i
ty,
and
S
E
S
.
R
ob
ust
st
a
ndard
e
rrors
w
e
r
e
c
o
mputed
a
nd
use
d
f
or
s
t
a
t
i
s
t
ical
s
i
gnific
a
n
ce
t
e
s
t
s
.
A
l
l
data
w
as
a
nalyz
e
d
u
s
i
n
g
S
T
A
T
A
14.2.
4.
RESULT
S
A
N
D
DISCU
SSIO
N
4.1.
R
e
su
l
t
s
of
c
onf
i
rmator
y fac
t
or
a
n
a
l
y
ses fo
r
mat
h
a
n
d
s
cien
ce
at
titudes
Co
nf
i
r
ma
to
ry
f
ac
t
o
r
a
n
a
l
ysi
s
r
e
s
u
l
t
s
s
ho
wed
a
n
a
d
e
qu
at
e
fit
o
f
t
h
e
m
ode
l
(
(
6
6
)
=
1
7
5
.
3
,
p
<
.
001
;
RM
S
E
A
=
0
.
04
0;
C
F
I
=
0
.9
85
;
TF
I
=
0
.
980)
f
o
r
c
o
n
s
t
r
u
c
t
i
n
g
t
w
o
l
a
t
e
n
t
fa
ctors-m
a
th
a
n
d
s
cie
n
ce
a
t
ti
tu
des-
from
14
sur
v
e
y
i
te
ms,
w
ith
t
he
e
xce
p
tio
n
o
f
a
s
i
g
n
i
fic
a
n
t
v
a
l
ue
d
ue
t
o
t
h
e
large
sa
mpl
e
s
ize
.
A
s
show
n
in
Tab
l
e
2,
m
ost
of
t
he
f
ac
tor
loa
d
in
gs
w
er
e
large
(>.60)
a
nd
a
l
l
of
t
he
m
w
e
re
s
ta
t
i
s
t
i
c
al
l
y
s
i
g
ni
fica
nt
.
The
ca
l
c
u
l
a
t
e
d
C
ro
nbac
h
’s
a
lp
ha
c
oef
f
ic
ie
n
t
s
fo
r
m
a
th
a
nd
sc
i
e
nce
a
tt
i
t
ude
s
w
e
re
both
0.
88,
e
x
c
ee
din
g
t
he
0
.6
0
thres
h
o
l
d.
T
a
k
en
t
o
g
e
t
he
r,
t
h
e
se
r
esults
s
u
g
g
e
s
t
the
tw
o
cons
tr
uc
ts
o
f
m
a
th
a
n
d
s
cienc
e
a
t
ti
t
udes
p
o
s
s
essed
ade
qua
te r
elia
b
ili
ty a
nd
va
l
i
d
i
t
y
,
as ind
ica
t
e
d
in t
h
e
U
n
frie
d
and
co
l
l
ea
gue
s
[21].
Tab
l
e 2.
Resu
l
t
s
o
f
con
f
irm
a
to
ry factor
ana
l
y
sis a
nd
re
l
i
a
b
i
l
i
t
y
t
e
st
F
a
c
tor
L
o
a
d
ing
A
l
pha
Ma
th Attitud
e
s
.
88
a
M
a
th
h
a
s
b
e
e
n
my w
o
r
st su
b
j
ect
.
8
0
a
M
a
th
is h
a
rd
f
o
r
m
e
.7
6
I
a
m
t
h
e
ty
p
e
o
f
st
ude
nt
t
o
do
we
ll
i
n
m
a
t
h
.
79
a
I
c
a
n
h
a
ndle m
o
st
s
ubjec
ts
w
ell,
but
I
c
a
nnot
do
a
good
j
ob
w
i
t
h m
a
th
.
77
I
a
m
s
u
re
I
c
ould
do
a
dva
n
c
e
d
w
ork
in m
a
t
h
.
66
I
c
a
n
g
e
t
good
gra
d
e
s
i
n m
a
t
h
.
67
I
a
m
good
a
t
m
a
t
h
.
84
S
c
i
e
n
c
e
At
titud
e
s
.
88
I
a
m
s
u
re
o
f
m
y
s
e
lf
w
he
n
I
do
sc
i
e
n
c
e
.
66
I
e
xp
e
c
t
t
o use
s
c
i
e
n
c
e
w
h
e
n
I
g
e
t
out
o
f
sc
hool
.
81
Knowing sc
i
e
n
c
e
will h
e
l
p
me e
a
r
n a li
ving
.
80
I will n
e
e
d
s
c
i
e
n
c
e
for
my
future
wo
r
k
.84
I
know
I
c
a
n
do
w
e
ll
i
n
sc
i
e
n
c
e
.
66
S
c
i
e
n
ce
wi
ll
b
e
i
mpor
t
a
nt
t
o m
e
i
n my
lif
e
’
s
w
o
r
k
.
83
I
a
m
s
u
re
I
c
ould
do
a
dva
n
c
e
d
w
ork
in
s
c
i
e
n
ce
.
70
No
t
e
.
a
Re
ve
rs
e
d
c
o
d
e
d
i
n
fac
t
or
a
n
a
lysis.
S
a
m
pl
e
size
i
s
1.
017
stude
nt
s.
4.2.
S
TEM su
mmer
p
r
ogram
p
ar
ticip
a
t
i
on
an
d
a
ttitu
d
e
s toward
m
ath
and
scien
ce
H
a
vi
n
g
c
o
n
s
t
r
u
cte
d
t
h
e
t
w
o
outc
o
m
e
m
e
a
sures
of
s
tude
nt
m
ot
iv
at
i
ona
l
fac
t
or
s-
ma
th
a
nd
scienc
e
att
itu
des,
w
e
exam
i
n
e
d
t
heir
r
ela
tio
ns
hips
w
ith
t
he
S
TEM
s
um
me
r
p
rogram
p
a
r
tic
ip
ati
on.
T
a
b
l
e
3
d
i
s
p
l
a
y
s
results
f
r
om
e
sti
m
ating
a
series
o
f
m
u
l
tip
le
r
e
g
ressi
o
n
m
ode
ls
t
h
at
p
re
di
ct
m
a
t
h
an
d
sci
e
n
ce
a
t
ti
tud
e
s
wi
th
P
R
E
P
part
ici
p
a
t
i
o
n,
c
on
tro
l
l
i
n
g
f
or
s
o
c
i
odem
ogr
a
p
h
i
c
char
acter
ist
i
cs
.
It
a
ppea
r
ed
t
ha
t
pa
rt
i
c
i
p
a
tin
g
in
a
S
TEM
summ
er
p
rogra
m
i
s
positi
ve
ly
a
ssoc
ia
t
e
d
w
i
t
h
s
tude
n
t
s’
a
tt
itude
s
tow
a
r
d
m
a
t
h
(
β
=
0
.1
12
,
p
<
0
.
0
1
)
b
u
t
h
a
s
n
o
effec
t
o
n
sc
ien
c
e
att
i
t
u
de
s.
T
h
e
e
stim
at
es
o
n
a
cou
p
l
e
o
f
s
oc
i
o
-
d
em
ograp
h
i
c
cha
r
ac
teris
tic
s
we
re
i
n
t
h
e
e
xpe
cte
d
direc
t
i
o
n.
F
or
e
xa
mple,
w
o
m
e
n
and
U
R
Ms
h
a
d
a
s
ig
ni
fican
t
l
y
l
o
w
e
r
level
of
a
tti
t
u
d
e
s
tow
a
r
d
m
a
t
h
.
F
u
r
t
he
rm
o
r
e,
t
he
i
nte
r
ac
tio
n
e
ffe
c
t
s
m
odel
s
(
Mode
ls
2
t
o
4)
s
ug
ge
ste
d
t
ha
t
t
h
e
r
e
lat
i
o
n
shi
p
s
a
m
ong
S
TEM
summ
er
p
r
ogram
p
artici
p
a
t
i
o
n
and
ma
th
a
n
d
s
c
i
ence
a
tti
tu
des
wer
e
not
v
a
r
ie
d
ac
ross
soc
i
o
d
e
mogr
ap
hic
gr
ou
ps,
defi
ne
d
b
y
ge
nde
r, ra
ce/et
hn
i
c
it
y,
a
nd S
E
S
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-8822
Int.
J
.
Ev
al. & R
es. Educ.
Vol.
8, No. 2, June
2
0
19:
3
56
- 3
62
36
0
Ta
b
l
e
3.
Resu
lts
o
f m
u
lti
pl
e
l
i
nea
r
r
egre
ssio
n
m
odels (
N
=
1.
01
7)
M
ode
l
1
M
ode
l
2
M
ode
l
3
M
ode
l
4
β
RS
E
β
RS
E
β
RS
E
β
RS
E
Pa
n
e
l
1:
M
a
t
h A
tti
tude
s
S
T
E
M
S
P
.
112**
(
.
0
38
)
.
074
(
.
052
)
.
198*
(
.
086
)
.
131*
(
.
051
)
F
e
m
al
e
-.
15
7
***
(
.
0
36
)
-
.
201
***
(
.
058
)
-
.
156
***
(
.
036
)
-
.
1
5
9
***
(
.
036
)
U
R
M
-
.
1
2
4
**
(
.
0
45
)
-
.
124
**
(
.
045
)
-
.
052
(
.
079
)
-
.
124
**
(
.
0
4
5)
L
ow
-SE
S
.
027
(
.
0
42
)
.
025
(
.
042
)
.
028
(
.
042
)
.
027
(
.
066
)
S
TE
MS
P
x
F
em
a
l
e
.
071
(
.
074
)
S
TE
MS
P
x
U
R
M
-
.
109
(
.
095
)
S
TE
MS
P
x
L
ow
-
S
E
S
.
002
(
.
085
)
I
nte
r
ce
pt
.
079
(
.
0
55
)
.
102
(
.
059
)
.
021
(
.
079
)
.
069
(
.
062
)
R
2
.
038
.
038
.
039
.
0
3
9
Pa
n
e
l
2: Sc
i
e
n
ce
Attitude
s
S
T
E
M
S
P
.
012
(
.
0
22
)
.
015
(
.
031
)
-
.
007
(
.
046
)
.
024
(
.
029
)
F
e
m
al
e
-.
02
6
(.
0
2
2
)
-.
023
(
.
033
)
-
.
027
(
.
022
)
-
.
026
(
.
022
)
U
R
M
-
.
0
4
0
(
.
0
27
)
-
.
040
(
.
027
)
-
.
056
(
.
039
)
-
.
039
(
.
027
)
L
ow
-SE
S
-
.
0
1
4
(
.
0
25
)
-
.
013
(
.
025
)
-
.
014
(
.
025
)
.
003
(
.
037
)
S
TE
MS
P
x
F
em
a
l
e
-
.
005
(
.
043
)
S
TE
MS
P
x
UR
M
.
024
(
.
052
)
S
TE
MS
P
x
L
ow
-
S
E
S
-
.
027
(
.
050
)
I
n
t
e
rce
p
t
.
045
(
.
0
30
)
.
044
(
.
033
)
.
058
(
.
038
)
.
037
(
.
034
)
R
2
.
006
.
006
.
006
.
0
0
7
No
t
e
.
N
= s
a
m
p
le
size;
β
=
c
o
e
f
f
ic
i
e
nt;
RS
E
=
r
obust
st
a
nda
rd
e
rror
;
S
T
E
M
SP =
S
T
EM
s
u
m
me
r
progr
a
m
;
U
R
M
=
unde
rre
prese
n
t
e
d
r
a
c
i
a
l
/
e
thni
c
m
i
nor
i
tie
s;
S
E
S
=
s
oc
io
e
c
o
no
m
i
c
sta
t
us.
***
p
<
.
001;
**
p
<
.
01;
*
p
<
.
05
(
tw
o-t
a
ile
d
t
e
st).
4.3.
STEM su
mmer p
r
o
g
r
a
m
pa
r
ticipa
t
i
o
n
a
nd
i
n
terests
i
n
m
a
t
h- a
nd sc
i
e
nc
e-r
e
lated c
a
ree
rs
O
u
r
se
cond
set
of
r
egr
e
ssi
o
n
ana
l
yse
s
e
sti
m
ated
t
he
a
ssocia
t
io
n
s
a
m
ong
S
TEM
s
um
me
r
progra
m
part
ici
p
a
t
i
o
n
and
s
t
u
d
e
n
ts’
i
n
t
e
res
t
s
i
n
m
ath-
a
n
d
s
cie
n
c
e
-
r
ela
t
e
d
c
a
r
e
e
r
s
.
T
a
b
l
e
4
r
e
p
o
r
t
s
r
e
s
u
l
t
s
f
r
o
m
est
i
ma
t
i
n
g
li
n
e
a
r
pr
o
ba
bil
i
t
y
m
odel
s
tha
t pr
edic
t c
a
re
er
i
nt
e
r
e
sts in m
ath and scie
nce
w
i
t
h
P
REP
participa
t
io
n,
c
o
nt
roll
i
n
g
f
o
r
so
ci
od
emog
rap
h
i
c
c
hara
cteri
s
t
i
c
s
.
Similar
to
t
h
e
fi
nd
ing
s
a
bove,
S
T
EM
s
um
m
e
r
progra
m
atte
n
d
an
c
e
i
s
p
o
sit
i
v
el
y
rela
t
e
d
to
s
tu
de
nts’
i
ntere
s
t
in
m
at
h
-
re
lated
c
a
re
ers
(
β
=
0
.
0
7
0
,
p
<
0
.
0
5
)
b
u
t
h
a
s
n
o
effec
t
on
s
t
u
d
e
nt
s’
i
nte
r
est
in
s
c
i
enc
e
-re
la
ted
ca
ree
r
s.
T
he
re
wa
s
a
l
so
n
o
e
v
id
en
ce
t
o
su
gg
est
th
at
t
he
assoc
i
at
i
o
n
s
a
m
ong S
T
E
M
s
um
me
r program
partici
p
a
t
i
o
n and s
t
u
d
e
n
t
s
’ i
n
tere
sts
i
n
ma
t
h-
a
nd
sc
ien
c
e-r
e
late
d
c
a
re
ers d
i
f
f
e
r
ed
b
y
soc
i
od
e
m
o
g
r
ap
hi
c c
h
aract
eri
s
t
i
c
s
, i
n
c
l
ud
ing
gen
de
r,
r
a
c
e/et
h
n
i
ci
ty,
a
nd SES.
T
a
b
l
e 4.
Resu
lts
o
f li
nea
r
p
r
oba
bi
lit
y m
ode
ls (N
=
1.01
7)
M
ode
l
1
M
ode
l
2
M
ode
l
3
M
ode
l
4
β
RS
E
β
RS
E
β
RS
E
β
RS
E
P
a
n
e
l
1
:
M
a
t
h
C
a
r
eer
I
n
t
er
es
t
S
T
E
M
S
P
.
070*
(
.
0
32
)
.
027
(
.
045
)
.
040
(
.
069
)
.
051
(
.
042
)
F
e
m
al
e
-.
1
7
3
***
(
.030
)
-
.
223
***
(
.
050
)
-
.
173
***
(
.
030
)
-
.
1
7
4
***
(
.
030
)
U
R
M
.
008
(
.037
)
.
007
(
.
037
)
-
.
017
(
.
063
)
.
006
(
.
038
)
L
ow
-SE
S
-
.
0
02
(
.036
)
-
.
004
(
.
036
)
-
.
003
(
.
036
)
-
.
038
(
.
057
)
S
TE
MS
P
x
F
em
a
l
e
.
080
(
.
063
)
S
TE
MS
P
x
U
R
M
.
037
(
.
077
)
S
TE
MS
P
x
L
ow
-
S
E
S
.
058
(
.
072
)
I
nte
r
ce
pt
.
639**
*
(.
0
4
6
)
.
665**
*
(.
050
)
.
659**
*
(
.
061
)
.
654
**
*
(.
051
)
R
2
.
035
.
036
.
035
.
035
P
a
n
e
l
2
:
S
ci
e
n
c
e
C
ar
eer
I
n
t
er
es
t
S
T
E
M
S
P
-.
0
0
5
(.
0
3
1
)
-.
043
(
.
044
)
-
.
040
(
.
064
)
.
028
(
.
040
)
F
ema
l
e
-
.
0
5
1
†
(
.030
)
-
.
094
†
(
.
059
)
-
.
051
†
(
.
030
)
-
.
050
†
(
.
030
)
U
R
M
-
.
0
35
(
.036
)
-
.
036
(
.
036
)
-
.
064
(
.
059
)
-
.
034
(
.
035
)
L
ow
-SE
S
-
.
0
40
(
.034
)
-
.
042
(
.
034
)
-
.
041
(
.
034
)
.
004
(
.
054
)
S
TE
MS
P
x
F
em
a
l
e
.
070
(
.
061
)
S
TE
MS
P
x
U
R
M
.
044
(
.
073
)
S
TE
MS
P
x
L
ow
-
S
E
S
-.
070
(
.
069
)
I
nte
r
ce
pt
.
764**
*
(.
0
4
3
)
.
786**
*
(.
047
)
.
787**
*
(
.
057
)
.
739
**
*
(.
048
)
R
2
.
008
.
009
.
008
.
010
No
te
. N
=
s
am
p
l
e
siz
e
;
β
= c
o
e
ffic
i
e
n
t;
RS
E
=
r
obust
sta
nda
rd
e
rr
or;
S
T
E
M
S
P
=
S
TEM
s
u
m
m
e
r
pr
ogram
;
U
R
M
=
unde
rre
prese
n
t
e
d
r
a
c
i
a
l
/
e
thni
c
m
i
nor
i
tie
s;
S
E
S
=
s
oc
io
e
c
o
no
m
i
c
sta
t
us.
***
p
<
.
001;
**
p
<
.
01;
*
p
<.
0
5
;
†
p
<
.
10
(tw
o
-
t
a
ile
d
t
e
s
t
)
.
Evaluation Warning : The document was created with Spire.PDF for Python.
Int J
Eval & R
es E
d
u
c
.
IS
S
N
:
2252-
88
22
O
u
t-o
f
-sch
o
o
l t
i
m
e
ST
EM
program
:
St
u
d
en
ts’
att
i
t
u
des
tow
a
rd a
n
d
c
a
reer
inte
re
st
s in
…
(G
uan S
a
w
)
36
1
5.
CONCL
U
S
ION
To
r
ec
ap,
the
fi
n
d
i
n
g
s
o
f
t
h
i
s
stud
y
i
ndic
a
t
e
that
p
ar
tic
i
p
at
in
g
i
n
P
R
E
P
-
a
n
O
S
T
S
T
E
M
p
r
o
g
r
a
m
-
i
s
pos
it
ive
l
y
asso
cia
t
e
d
w
i
t
h
hig
h
e
r
l
eve
l
s
o
f
s
t
ude
n
t
s’
a
t
t
i
t
ud
e
s
t
ow
ard
ma
t
h
a
nd
i
n
tere
st
i
n
ma
t
h
-
r
elate
d
c
a
r
ee
rs,
bu
t
has
no
sig
n
i
f
ica
n
t
rela
tio
nsh
i
ps
w
ith
s
t
ude
n
t
s’
s
c
i
enc
e
att
i
t
ude
s
an
d
c
a
re
e
r
i
ntere
s
t
s
.
These
result
s
we
r
e
enc
our
ag
i
ng fo
r
O
S
T S
TEM
p
ro
gra
m
provi
ders and ed
uca
t
or
s.
H
ow
e
v
er, it is i
m
p
orta
nt to n
o
te
th
a
t e
s
tim
ates
wer
e
n
o
t
c
ausa
l
a
s
t
here
m
ay
b
e
con
f
ou
n
d
i
n
g
fa
ct
ors
tha
t
w
e
r
e
n
o
t
a
c
c
ou
nte
d
f
or
,
w
h
i
c
h
coul
d
b
i
as
our
r
e
s
ul
ts.
N
one
the
l
ess,
b
y
a
n
al
yz
in
g
a
large
sa
mple
o
f
midd
le
s
cho
o
l
s
t
u
de
nts
fr
om
m
ul
ti
p
l
e
cohor
ts
a
nd
soc
i
o-
dem
ogra
p
h
i
ca
lly
d
i
v
erse
b
ac
kgr
ou
n
d
s,
t
h
i
s
stud
y
o
f
fers
n
e
w
e
vid
e
n
ce
s
u
gge
sti
ng
tha
t
O
S
T
S
TEM
p
rogra
m
s
cou
l
d
be
a
p
ro
m
i
si
ng
in
t
e
rve
n
t
i
on
t
o
i
n
c
r
ease
st
u
d
e
n
t
m
o
t
i
va
tio
na
l
fac
t
o
r
s
in
S
TE
M
a
r
ea
s.
F
urtherm
o
re,
our
fi
n
d
in
gs
i
nd
i
c
ate
t
h
at
O
S
T
S
TEM
p
rogr
am
s
have
n
o
diffe
r
en
t
i
al
e
ffec
ts
f
or
s
tu
den
t
s
a
c
ross
gen
d
er
,
rac
i
al
/et
h
n
i
c
,
o
r
S
E
S
grou
ps.
F
u
t
u
r
e
i
ni
tia
ti
v
e
s
aim
e
d
a
t
i
m
p
ro
vi
n
g
S
TEM
l
e
a
rni
ng
a
nd
pa
rt
i
c
ipa
tio
n
a
m
ong
un
derr
epre
se
n
t
e
d
g
ro
u
p
s,
i
nc
lud
i
ng
w
o
m
e
n,
U
RM,
an
d
l
o
w
-
S
E
S
stud
e
n
ts,
cou
l
d
be
d
e
s
i
gne
d
t
o
a
nd
te
st
f
or
gen
d
er
-spec
i
f
i
c
or
c
ult
u
ral
l
y
re
spo
n
s
i
ve
i
n
t
e
r
ven
tio
ns
o
r progr
am
s
in S
TEM
.
ACKNOW
LEDG
E
MEN
T
S
Th
is
s
tud
y
w
as
f
un
ded
b
y
a
n
A
m
erica
n
E
duca
t
io
na
l
Re
s
ear
ch
A
ssoc
i
a
ti
on
R
es
ea
rch
Gran
t
(wi
t
h
sup
por
t
fr
om
t
he
U
S
N
a
tio
nal
S
c
ie
nce
Fou
n
d
a
t
i
on)
a
n
d
a
r
ese
a
rc
h
g
r
a
n
t
from
th
e
Gra
n
t
s
f
or
R
esea
rch
A
dva
nc
em
ent
and
Trans
f
or
ma
t
i
o
n
p
rogr
am
by
the
O
f
fic
e
of
t
he
V
ic
e
P
r
esiden
t
for
Resea
r
ch,
Ec
onom
ic
D
e
ve
lopm
en
t and K
now
le
dge
En
t
er
prise a
t
th
e
U
ni
ve
rsi
t
y o
f
Texas
at
S
a
n A
n
ton
i
o (U
TSA
)
. The
auth
o
r
s
a
l
s
o
tha
n
k
Josep
h
K
ul
hane
k and K
r
ystal
P
e
ra
l
e
z from t
he
U
TS
A
Instit
u
te fo
r
P-20 Init
i
a
t
i
v
e
s for their suppor
t
and
con
s
truc
t
i
ve
f
e
e
dbac
k
o
n
th
i
s
s
t
u
d
y
.
REFE
RENCES
[1]
M
a
rgin
so
n
S.,
Tyt
l
er
R
.,
F
reem
an
B
.,
&
R
ob
erts
K
.,
"
S
T
EM
: Co
u
n
t
r
y com
parisons: Intern
atio
nal com
parison
s of
sci
e
nce, t
echn
o
logy,
eng
i
neer
in
g
a
n
d
math
ema
t
ic
s
(
S
TEM)
ed
ucatio
n
, "
.
F
i
na
l
re
po
rt, 2
01
3.
[2]
Whi
t
e
Hous
e.
"
A
s
t
r
a
t
e
gy f
o
r
Amer
ican
inn
o
va
ti
on:
Secu
ri
ng
o
u
r
econ
o
m
i
c
growth a
n
d
pro
s
perit
y
,
".
W
as
h
i
ngton
,
DC: Ex
ecutive Office o
f
t
he P
re
si
den
t
,
201
1
.
[3]
M
e
a
n
s
,
B
.
,
W
a
n
g
,
H
.
,
W
e
i
,
X
.
,
L
y
n
c
h
,
S
.
,
P
e
t
e
r
s
,
V
.
,
Y
o
u
n
g
,
V
.
,
&
A
l
l
e
n
,
C
.,
"
E
x
p
a
n
ding
S
TE
M
op
po
r
t
un
it
ie
s
th
rou
gh
in
c
l
usive
S
T
E
M
-fo
c
us
ed
h
ig
h
s
c
ho
ols,
"
Sc
i
e
nce E
d
u
c
at
io
n
,
v
o
l
.
1
01(5
)
,
pp.
6
81
-71
5
,
2017.
[4]
Saw,
G
.
K
.,
"Th
e
i
mpact
o
f
inclusi
v
e
STEM
h
igh
schools
on
s
tud
e
n
t
out
co
m
e
s
:
A
s
tatew
i
d
e
l
o
ngitud
i
nal
ev
alua
t
i
on
of
T
exas
S
TE
M acad
em
ies
,
" Int
e
rn
ati
o
n
a
l J
o
u
r
nal of S
c
i
ence an
d
M
a
th
em
ati
c
s E
d
u
c
a
t
io
n, in pres
s.
[5]
Bey
m
er,
P
.
N
.,
R
osenb
e
rg
,
J
.
M
.,
S
ch
mid
t
,
J
.
A
.,
&
N
a
f
tzger,
N
.
J
.
,
"
E
x
amin
in
g
relati
onsh
i
ps
a
m
ong
c
ho
ice,
a
ff
ect,
a
n
d e
n
ga
g
e
me
n
t
in
su
m
m
e
r
s
te
m
p
r
og
ra
ms,
"
Jo
urna
l of Yo
ut
h
a
nd Ad
o
l
escence
,
v
o
l.
47
, p
p.
11
7
8
-1
19
1, 2
01
8.
[6]
E
l
am
,
M.
E
.
,
D
o
n
h
am
,
B
.
L
.,
&
S
o
l
o
m
o
n
,
S
.
R.
,
"An
en
gi
neer
i
n
g
s
u
m
m
e
r
pro
g
ra
m
f
o
r
un
derrep
r
esent
e
d
stud
ent
s
f
r
o
m
ru
r
al s
cho
o
l
d
i
s
t
ri
cts,"
J
o
urna
l
o
f
S
T
EM
Ed
u
c
atio
n
:
In
no
vati
on
s a
n
d
Res
e
arch
,
v
ol.
1
3(2),
p
p
.
3
5-44
,
2
0
1
2
.
[7]
Kno
x
,
K.,
Mo
yn
ih
an,
J.
,
&
Marko
w
i
t
z,
D
.,
"
E
v
a
l
u
at
io
n
o
f
s
ho
r
t
-t
erm
i
m
pact
o
f
a
h
i
g
h
s
cho
o
l
summ
er
s
cien
ce
p
rogram
on
s
tu
dent
s’
p
ercei
ved
kn
ow
ledge
a
n
d
s
k
i
lls,
"
Jou
r
n
a
l o
f
Sci
e
nce E
d
u
c
ation
a
nd T
ech
nol
og
y
,
vol.
12
(4
),
p
p
p
.
47
1
-
47
8,
200
3.
[8]
Chen
,
C.,
Tom
s
ov
ic,
K
.,
&
Ay
d
e
ni
z,
M
.
,
"
Filling
t
h
e
p
i
pel
i
ne
:
P
ower
s
ystem
an
d
en
ergy
c
u
rricul
a
f
or
m
id
d
l
e
and
hi
gh
s
ch
ool
s
tud
e
nt
s
t
h
ro
ugh
sum
m
e
r
pro
g
ram
s
,
"
I
E
E
E
Tran
sac
t
io
ns
o
n
Po
we
r
Sy
s
t
e
m
s
,
vol.
29
(4
),
pp
.
1
874
-18
7
9
, 2
014
.
[9]
Go
on
atil
ake,
R
.
,
&
B
achn
a
k,
R
.,
"
P
ro
m
o
t
i
n
g
e
n
g
ineerin
g
ed
ucati
o
n
am
ong
high
s
ch
oo
l
an
d
m
i
d
d
l
e
s
ch
oo
l
stu
d
ents,
"
J
o
urna
l o
f
S
T
EM
Ed
u
c
at
io
n
,
vo
l. 1
3(1
)
,
p
p
. 1
5-2
1
, 2
01
2.
[10]
Kitch
e
n,
J
.
A.,
Son
n
ert
,
G
.,
&
S
a
d
l
e
r,
P
.,
"Th
e
i
mpact
o
f
coll
e
ge-
and
un
iv
ersity-ru
n
h
i
g
h
sch
ool
s
um
m
e
r
prog
rams
on
s
t
u
d
e
nt
s’
e
n
d
o
f
h
i
gh
sch
o
o
l
S
T
E
M
career
a
sp
irat
io
ns,
"
Science Education
,
vol.
102
(3),
pp.
1
-1
9,
201
8.
[11]
M
a
rko
w
it
z,
D
.,
"
E
v
alu
a
ti
on
o
f
the
l
o
n
g
‐term
im
pact
o
f
a
u
n
iv
ers
ity
h
igh
sc
ho
ol
s
um
me
r
sc
ie
nc
e
pro
g
ra
m
on
s
tu
de
n
t
s’
interest
a
nd
p
e
r
ceived
abilit
i
e
s
in
s
ci
ence,"
J
o
urna
l o
f
Sc
i
e
n
c
e
Ed
uc
a
t
ion
an
d Te
c
h
no
lo
g
y
,
v
o
l
.
1
3(
3)
,
pp
.
3
95-4
0
7
,
2
0
0
4.
[12]
Ni
emann
,
M
.
A.
,
M
i
l
l
er,
M.
L
.,
&
D
av
i
s
,
T.,
"T
he
U
niv
e
rsi
t
y
o
f
A
l
ab
am
a
at
B
irm
i
n
g
h
a
m
Center
f
o
r
C
o
m
m
u
n
i
t
y
Ou
treach
D
ev
elop
m
e
nt
S
u
m
m
e
r
S
c
ience
Ins
t
itu
te
p
rog
r
am
:
A
3-yr
l
ab
orato
r
y
research
e
xp
erie
nce
f
o
r
inn
e
r-city
se
c
o
n
d
a
r
y le
ve
l stud
e
n
t
s
,"
C
e
ll
Bi
ol
o
g
y E
d
u
c
at
i
o
n
,
v
o
l
.
3
,
p
p.
1
62-18
0,
2
00
4.
[13]
Timme,
N., Bair
d
, M
.,
B
en
n
e
t
t
, J
.,
F
ry
,
J
., Lan
ce,
G
., & Ma
l
t
e
s
e,
A
.
,
"
A
summ
er
m
ath
and
ph
y
s
ics
prog
ram
f
o
r
hig
h
sc
ho
ol
s
tu
de
n
t
s
:
S
tu
de
n
t
p
e
r
fo
r
m
a
n
c
e
a
n
d
le
sso
n
s
l
e
a
r
ne
d
in
t
he
s
e
c
o
nd
y
ear,
"
Th
e Ph
ysics T
e
ach
e
r
,
v
o
l.
51,
pp.
2
8
0
-
28
4,
201
3.
[14]
Bachm
a
n,
N
.,
B
i
s
ch
o
f
f
,
P
.,
G
a
l
l
agh
e
r,
H
.
,
L
ab
roo
,
S
.,
&
S
ch
aum
l
off
e
l,
J
.,
"
P
R
2EP
S
:
P
r
eparat
ion
,
r
ecrui
tm
e
n
t
reten
tio
n
and
ex
cel
lence
in
t
h
e
p
hys
ical
s
ci
ences
,
i
n
c
l
udi
ng
en
g
i
n
eerin
g.
A
r
ep
ort
on
t
h
e
2
0
0
4
,
200
5,
a
nd
2
006
,
sci
e
nce s
u
mm
er cam
ps
,"
Jou
r
na
l of STEM
Ed
uca
t
i
o
n
,
vol.
9(1
-
2),
p
p
.
30-3
9
,
20
08.
[15]
Kab
acof
f
,
C
.,
S
ri
vas
t
ava,
V
.,
&
Rob
i
n
s
o
n
,
D.,
"
A
s
ummer
acade
m
i
c
research
e
x
p
e
rien
c
e
f
or
d
i
s
a
d
vant
aged
y
out
h
,
"
CBE—
L
i
fe Sc
iences E
d
ucat
i
o
n
,
vol
.
12
,
p
p
.
410
-4
1
8
,
201
3.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-8822
Int.
J
.
Ev
al. & R
es. Educ.
Vol.
8, No. 2, June
2
0
19:
3
56
- 3
62
36
2
[16]
M
i
l
l
er,
K.,
Carrick,
T.,
M
a
rtin
ez‐S
u
ssm
ann
,
C
.,
L
evi
n
e,
R
.,
A
nd
ron
i
cos
,
C
.,
&
L
an
gf
ord,
R
.,
"Eff
ecti
v
en
e
s
s
o
f
a
su
mmer exp
e
ri
ence f
o
r in
spiri
ng
interest
i
n g
e
os
cience amo
n
g
Hi
sp
ani
c
‐A
m
e
rican
h
i
gh
sch
ool student
s,
"
Jo
ur
nal
of
Geo
s
cien
ce Ed
uca
tio
n
,
vol.
55
(6),
p
p.
596
-60
3
,
2
0
07.
[17]
Salt
o
,
L
.
M
.,
R
i
g
g
s,
M
.
L
.
, De
L
e
o
n
,
D.
D.
,
C
asi
a
n
o
,
C
.
A
.
,
&
D
e
Leon
,
M.
,
"U
nderrep
r
esent
e
d
m
i
n
o
rity
h
igh
sch
ool
and
coll
ege
stu
d
ent
s
r
ep
ort
ST
EM
-pi
p
elin
e
s
u
stai
ni
ng
g
a
i
n
s
a
fte
r
parti
c
ipating
i
n
t
he
L
om
a
Lin
d
a
Un
iv
ers
i
ty
s
umm
e
r
heal
th
d
i
s
p
a
riti
es
res
earch p
rog
r
am
,"
PL
o
S
O
n
e
, v
ol. 9
(9),
2
01
4.
[18]
Nu
gen
t
,
G.,
Bark
er,
B.,
Grand
g
en
ett,
N
.,
&
A
d
am
chu
k
,
V.
I
.,
"
Im
p
act
o
f
robot
ics
and
geos
p
a
t
i
al
t
ech
nol
og
y
in
terven
ti
on
s
on
yo
ut
h
S
T
EM
l
earn
i
ng
a
n
d
a
ttitu
d
e
s
,
"
Jou
r
n
a
l
of R
e
sea
r
ch
on
T
ech
nol
og
y i
n
E
d
u
c
at
ion
,
vol.
42(4
)
,
pp
.
3
91-4
0
8
,
2
0
1
0.
[19]
Con
s
tan
,
Z
.,
&
S
pi
cer,
J.
J
.
,
"
Maxi
mizi
ng
F
utu
r
e
P
o
tenti
a
l
i
n
P
h
y
s
i
c
s
a
n
d
S
T
E
M
:
E
v
a
l
u
a
t
i
n
g
a
S
u
m
m
e
r
P
r
o
g
r
a
m
th
rou
g
h
a
Part
ners
hi
p
betw
een
S
cience
Ou
treach
a
n
d
E
du
cati
on
Re
search
,
"
J
o
urna
l of
Hig
h
e
r Ed
uc
a
t
io
n
Ou
tre
a
c
h
an
d E
n
g
agem
e
nt
, vo
l
. 1
9(2
)
,
p
p
.
1
17
-13
6
, 2
01
5.
[20]
Nu
gen
t
,
G.,
Bark
er,
B.,
G
r
and
g
en
ett,
N
.,
&
A
d
am
chu
k
,
V.
I
.,
"
Im
p
act
o
f
robot
ics
and
geos
p
a
t
i
al
t
ech
nol
og
y
in
terven
ti
on
s
on
yo
ut
h
S
T
EM
l
earn
i
ng
a
n
d
a
ttitu
d
e
s
,
"
Jou
r
n
a
l
of R
e
sea
r
ch
on
T
ech
nol
og
y i
n
E
d
u
c
at
ion
,
vol.
42(4
)
,
pp
.
3
91-4
0
8
,
2
0
1
0.
[21]
Martinez,
E
.
,
L
indl
ine,
J
.,
Pet
r
onis,
M
.
S
.,
&
P
ilotti
,
M
.
,
"Ef
f
ectiveness
o
f
a
s
cience agricu
lt
ural sum
mer experi
ence
(S
ASE)
i
n
recru
i
t
i
n
g
s
tud
e
nt
s
t
o
n
atu
r
al
r
eso
u
rces
m
anagem
en
t,"
J
o
urna
l o
f
S
c
ie
nc
e
Edu
c
at
ion
an
d
Te
c
h
no
log
y
,
vol.
21
(6),
p
p.
7
1
3
-721,
2
0
1
2
.
[22]
M
o
h
r
‐S
ch
roed
er,
M.
J
.,
Jack
so
n,
C
.,
M
i
ll
er
M
.,
W
a
l
co
tt
B
.,
L
itt
le
D
.
L.,
S
p
eler
L
.,
S
c
h
o
o
l
er
W
.,
S
ch
roed
er
D
.
C.,
"Devel
op
ing
M
i
dd
le
S
cho
o
l
S
t
udent
s'
I
nteres
ts
i
n
S
T
E
M
v
ia
S
um
m
e
r
Learn
i
n
g
E
xperi
e
n
ces:
S
ee
Blu
e
S
TEM
Cam
p
,"
S
c
ho
ol Sc
ie
n
c
e
a
n
d
Ma
the
m
a
tic
s
,
v
o
l.
1
14(6
)
,
p
p
.
291
-30
1
,
201
4.
[23]
Saw,
G
.
K
.
,
Ch
a
n
g
,
C
.-N
.
,
&
Ch
an
,
H.
-Y
.,
"
C
ros
s
-secti
o
nal
an
d
l
ongit
u
d
i
nal
disparities
i
n
S
TEM
c
a
reer
a
spi
r
ati
ons
at
t
h
e
i
nt
ersection
of
g
end
e
r,
r
ace/eth
n
i
c
ity,
a
n
d
s
o
ci
oecon
o
mi
c
stat
us,"
E
duca
t
i
o
n
a
l R
e
sear
cher
,
vo
l.
4
7(
8),
pp
.
5
25-5
3
2
,
2
0
1
8.
[24]
S
a
w,
G
.
K
.
,
&
Chang
,
C
.-N
.
,
"Cross
-lag
ged
mo
del
s
o
f
m
a
t
h
em
ati
c
s
ach
ievemen
t
a
n
d
m
otiv
ational
f
acto
r
s
amon
g
Hi
sp
a
n
ic
a
n
d
n
on
-His
panic
h
i
gh
s
ch
oo
l
s
t
udent
s,
"
Hispa
n
i
c
J
o
u
r
na
l o
f
Be
ha
v
i
oral
Sc
ie
n
c
e
s
,
vo
l.
40(2),
pp
.
2
40-2
5
6
,
2
0
1
8.
[25]
Unf
r
ied
,
A
.
,
F
ab
er,
M.
,
S
t
an
ho
pe,
D.
S
.,
&
W
ie
b
e
,
E
.,
"
T
h
e
d
ev
e
lo
pm
en
t
an
d
val
i
d
a
ti
on
o
f
a
m
e
a
su
re
o
f
stu
d
ent
attit
udes
t
o
ward
s
c
i
ence, technolo
gy, engi
neeri
ng,
a
nd
m
a
t
h
(S-
STEM),"
Jour
nal of Ps
y
c
hoed
ucational Ass
e
ssment
,
vo
l.
33(7
)
,
p
p
.
6
2
2-63
9,
201
5.
[26]
Jack
so
n,
D
.
L.
,
G
i
l
l
asp
y
,
J.
A
.
,
&
P
u
rc-S
t
e
phenso
n
,
R.,
"Repo
r
t
in
g
practi
ces
i
n
c
o
nf
i
r
m
a
tory
f
acto
r
a
nal
y
s
i
s
:
A
n
ov
ervi
e
w
a
nd
s
o
m
e
recom
m
e
nd
ati
ons,
"
P
s
ycho
l
ogi
cal M
e
th
od
s
,
v
o
l
.
1
4(
1
)
,
p
p
.
6
-
23
,
2
009
.
[27]
Hu
,
L.
-t.,
&
B
en
tl
er,
P
.
M
.,
"
Cu
toff
criteri
a
f
o
r
fi
t
i
ndex
e
s
i
n
co
varian
ce
st
ruct
ure
anal
ys
is:
Co
n
v
en
ti
onal
crit
eria
vers
us
n
ew
a
lt
ernati
ves
,
"
S
t
ru
c
t
u
r
al Eq
uati
on M
odeli
ng
,
v
o
l.
6
(1
),
p
p.
1
-55,
1
999.
[28]
Cron
bach,
L.
J
., "
C
o
e
f
f
i
c
i
en
t alp
h
a and
th
e i
n
ternal stru
c
tu
re
of
t
es
t
s
,
"
Psycho
m
et
r
i
k
a
,
v
o
l
.
1
6
,
2
97-33
4,
1
95
1.
[29]
Wo
old
r
id
ge,
J
.
M.
,
"
Int
r
od
uct
o
ry
Eco
n
o
m
etrics
: A M
o
d
e
rn
Ap
proach
(5
t
h
.
)
,
"
Ma
so
n, O
H:
S
ou
th
-
W
e
s
te
rn,
20
12
.
Evaluation Warning : The document was created with Spire.PDF for Python.