In
te
r
n
ation
a
l Jou
rn
al
o
f E
v
al
u
a
t
i
on
a
n
d
R
e
se
arc
h
in
Ed
u
c
ation
(
IJERE
)
V
o
l.
8
, N
o. 1, Mar
c
h
201
9,
pp.
39~4
6
ISSN
: 2252-
88
22,
D
O
I
:
10.11
59
1
/ijer
e
.
v8i
1.
155
94
39
Jou
rn
a
l
h
o
me
pa
ge
:
ht
tp:
//i
a
e
score
.
com
/
j
o
u
r
na
l
s
/
i
n
d
e
x
.
p
hp/IJ
ERE
The influen
ce of problem based
learning towards social scie
n
ce
learning outcomes viewed
from
learning interest
Bert
i Dyah
P
er
matas
a
r
i
1
,
G
u
na
r
h
a
d
i
2
, Riyadi
3
1,
3
D
e
p
a
rtm
e
n
t
of El
em
entary
S
cho
o
l
Teach
er Trai
n
in
g an
d Edu
cation
,
S
ebel
as M
aret
U
nivers
it
y,
I
ndo
nes
i
a
2
D
e
partm
e
n
t
o
f Ed
ucat
ion
a
l S
c
i
e
nce,
S
eb
elas
M
a
r
et U
n
i
versity
, In
don
esi
a
Art
i
cl
e In
fo
ABSTRACT
A
r
tic
le hist
o
r
y
:
R
e
c
e
i
v
e
d
Sep
1
7
,
2
018
Re
vise
d O
c
t
2
9
,
2018
Ac
ce
p
t
ed
No
v
1
5
,
2
018
Th
e
ai
m
of
t
his
s
t
u
d
y
i
s
t
o
determ
in
e
t
h
e
in
fl
uen
ces
o
f
P
r
obl
em
B
ased
Learn
i
ng
a
nd
l
earni
ng
i
n
t
e
res
t
a
t
im
pro
v
i
ng
t
h
e
cog
n
i
t
i
v
e
learn
ing
ou
tco
m
es
in
s
o
c
ial
s
c
ience
o
f
f
o
u
rt
h-g
r
ade
elem
ent
a
ry
s
choo
l
s
t
ud
en
t
s
.
T
his
st
ud
y
i
s
a
qu
asi
-
experi
m
e
n
t
s
tu
dy
wit
h
p
re-t
e
s
t
an
d
po
st-t
est
co
nt
ro
l
g
r
ou
p
des
i
g
n
.
Th
e
sam
p
le
c
oll
ecti
o
n
i
s
t
ak
e
n
w
it
h
s
t
rati
fied
c
lu
st
er
r
an
do
m
sam
p
l
i
ng
t
e
c
h
ni
qu
e.
Th
e
sam
p
l
e
o
f
t
h
is
s
t
u
d
y
i
s
1
0
9
fo
urt
h
-g
rade
e
lem
e
n
t
ary
s
c
ho
ol
students
,
con
s
i
s
t
e
d
o
f
5
0
stu
d
en
ts
f
ro
m
experi
m
e
nt
g
ro
up
a
nd
5
9
stu
d
en
ts
f
r
om
c
on
trol
g
r
ou
p.
T
h
e
s
tud
e
nts
fro
m
e
xp
e
r
ime
n
t
g
r
ou
p
a
r
e
giv
e
n
th
e
a
p
p
lic
a
t
i
on
o
f
P
r
ob
lem
Bas
e
d
L
earnin
g
,
wh
ile
th
e
st
ud
ents
i
n
co
nt
rol
gro
up
are
g
i
v
e
n
t
h
e
app
l
i
catio
n
of
D
i
r
ect
I
ns
tru
c
tio
n
.
T
he
d
ata
co
ll
ec
t
i
o
n
i
s
carri
ed
b
y
us
in
g
th
e
ins
t
rumen
t
s
of
s
ocial
science
l
earni
ng
o
u
t
comes
tes
t
a
nd
s
o
c
ial
s
c
i
e
n
ce
learn
i
n
g
i
nterest
q
u
esti
onn
a
i
re.
Th
e
valid
i
t
y
of
t
est
and
quest
ionnaire
in
s
t
rum
e
n
t
s
is
carried
b
y
ex
pert
j
u
dgemen
t
.
Th
e
difficu
lty
l
eve
l,
di
s
t
ingu
i
s
hing
p
ower,
and
rel
i
abili
ty
o
f
tes
t
i
nst
r
ument
are
te
s
t
ed
b
y
usi
ng
ITEMAN
a
pplicati
on.
T
he
i
nternal
cons
ist
e
ncy
and
reliabi
l
i
ty
o
f
qu
est
i
on
na
i
r
e
instrum
e
nt
i
s
t
e
sted
b
y
us
in
g
S
P
S
S
app
l
i
c
a
t
io
n
.
T
he
d
ata
is
a
n
a
l
y
s
e
d
b
y
u
s
in
g
two-
wa
y
a
n
ov
a
.
T
h
e
o
u
t
c
o
me
s
of
t
he
s
tu
dy
s
ho
w
t
ha
t
P
B
L
and
l
earn
i
n
g
i
n
t
erest
co
ntri
bu
te
s
i
gnifican
t
im
p
act
t
oward
s
s
oc
ia
l
s
c
ie
nc
e
learn
i
n
g
o
u
t
c
o
mes
o
f
e
lem
e
ntary
scho
ol
s
tud
e
nt
s.
K
eyw
ord
s
:
Lea
r
ni
ng i
n
t
e
rest
Lea
r
ni
ng o
u
t
c
o
me
s
P
r
ob
le
m
based
l
e
arni
ng
So
ci
al
sci
e
n
ce
Co
pyri
gh
t © 2
019 In
stit
u
t
e
of Advanced
En
gi
neeri
n
g
an
d
Scien
ce.
All
rights
res
e
rv
ed.
Corres
pon
d
i
n
g
Au
th
or:
B
e
rt
i D
y
ah
P
e
r
m
a
tasar
i
,
D
e
pa
rtme
nt
o
f
El
e
m
e
n
t
a
ry S
cho
o
l
Te
ache
r
T
rain
in
g an
d
Ed
ucat
i
on,
Se
belas Ma
ret
U
n
i
v
e
r
sity,
Jl
.
I
r
. S
u
tam
i
N
o.
36A
,
Jebres,
S
u
raka
rta,
J
awa
Te
n
g
a
h
,
Indo
nes
i
a (
571
26)
.
Em
ail:
bert
id
y
a
hp
@s
t
u
de
nt
.
u
ns.ac
.
id
1.
I
N
TR
OD
U
C
TI
O
N
O
n
e
of
t
he
r
ol
es
o
f
nat
i
o
n
a
l
e
duca
t
io
n
i
n
I
n
d
o
n
e
s
i
a
i
s
t
o
p
re
pa
re
p
erc
e
pt
ive
stu
d
e
n
t
s
o
n
fa
ci
ng
ha
r
d
cha
lle
ng
e
i
n
g
lo
ba
l
s
o
cie
t
y
life
t
h
at
c
o
n
sta
n
tl
y
c
h
an
ges.
T
h
i
s
g
o
es
i
n
line
w
ith
A
c
t
o
f
T
h
e
Re
pu
b
l
ic
o
f
Ind
ones
i
a
N
u
m
b
er
20,
Y
ear
200
3
on
N
a
ti
o
n
al
E
duc
a
t
i
on
S
y
s
t
e
m
,
Ar
tic
le
1
S
e
c
t
i
on
2
w
h
ic
h
trans
l
ate
s
“
N
a
t
iona
l
e
duc
ati
o
n
me
ans
educ
a
t
i
o
n
ba
se
d
o
n
P
anc
a
si
la
a
nd
the
1
945
Con
s
t
i
t
u
t
i
on
,
an
d
i
s
r
oo
t
e
d
i
n
t
h
e
reli
gi
o
u
s
va
lue
s
,
nat
i
o
n
a
l
c
u
l
t
u
res
o
f
I
ndo
ne
sia
,
a
n
d
o
ne
t
h
a
t
i
s
r
e
s
pons
i
v
e
to
t
he
n
ee
ds
o
f
the
ever
-cha
ng
ing
e
r
a.”
[1
].
O
n
e
o
f
t
h
e
su
bj
e
c
t
s
i
n
e
l
ement
a
ry
s
ch
ool
w
h
i
ch
i
s
d
e
s
i
g
n
e
d
t
o
prepa
r
e
stu
d
e
n
t
s
t
o
face
t
he
c
ha
ng
in
g
age
is
s
ocia
l sc
ience
,
as sta
t
e
d
b
y
N
u
rca
hyo
and
H
arto
no
th
a
t
s
ocia
l
sc
i
e
nc
e ed
uc
at
ion in elem
e
n
t
a
r
y
sc
h
o
o
l
i
s
aim
e
d
for
stud
e
n
ts
t
o
de
ve
l
o
p
k
now
led
g
e
a
nd
ba
si
c
s
k
il
l
s
w
hich
bene
fit
them
sel
v
es
i
n
the
i
r
dai
l
y
li
fe
[
2]
.
So
ci
al
s
c
i
en
c
e
i
t
s
elf
i
s
d
e
f
i
n
ed
a
s
e
d
u
c
ati
o
n
a
l
p
r
o
g
r
a
m
t
h
a
t
ch
ooses
e
duc
a
t
i
o
na
l
m
a
t
e
r
i
al
f
rom
soc
i
al
a
n
d
h
u
m
ani
t
i
e
s
s
cie
n
c
e
w
hi
ch
a
re
s
ci
e
n
t
i
fi
ca
ll
y
an
d
p
s
y
c
ho
l
o
g
i
cal
ly
orga
n
ize
d
and
serve
d for educ
a
t
i
o
na
l
p
u
r
pose
[3].
T
h
i
s
de
fi
nit
i
o
n
s
h
o
w
s
t
ha
t
soc
i
al
s
ci
enc
e
’s
o
ne
o
f
it
s
m
a
i
n
c
h
a
r
a
c
terist
ics
is
c
oo
p
e
rat
i
on
a
m
on
g
e
d
u
cati
o
n
a
l
sci
e
n
ce
and
so
ci
a
l
s
c
i
e
n
c
e
s
.
I
n
Mi
n
i
st
ri
al
R
eg
ul
at
i
on
N
u
m
b
e
r
22
,
Ye
ar
2
006
o
n
Co
nt
ent
St
a
n
d
a
rd
,
it
i
s
men
t
ion
e
d
th
at
“
So
ci
a
l
S
c
i
ence
s
ubj
ec
t
i
s
d
e
s
ig
ne
d
to
d
e
v
e
l
op
t
h
e
kn
ow
l
e
dge,
c
o
mpr
e
h
e
ns
io
n,
a
n
d
a
n
aly
s
i
s
a
b
ili
ty
o
n
so
ci
et
y
so
ci
al
c
on
dit
i
on
i
n
e
n
t
e
ring
d
y
na
mic
c
i
v
iliza
tio
n
life
.
”
[4]
.
T
he
o
bj
ec
tive
o
f
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
Int.
J
.
Eval. & R
es. Educ.
Vol.
8, No. 1, March
2
019
:
39
- 4
6
40
soc
i
al
s
c
i
e
n
ce
i
s
t
he
n
form
ul
ated
i
nt
o
f
our
c
ateg
ories
:
(
1)
K
no
w
l
e
dge
,
t
h
a
t
h
e
l
ps
s
tude
nts
st
ud
y
t
h
em
sel
v
es
and
t
h
eir
en
v
i
ronm
en
t,
i
ncl
u
d
e
s
ge
o
g
rap
h
y
,
hist
ory,
p
oli
tic
s,
e
conom
y,
a
nt
h
r
o
p
o
lo
gy
a
nd
s
o
c
i
o-
psyc
h
o
l
o
g
y
;
(2)
Skil
l
,
w
hic
h
i
n
th
is
c
a
s
e
i
n
cl
u
d
es
t
h
i
n
k
i
ng
s
k
i
l
ls
;
(3)
A
t
t
it
ude
s,
g
r
o
upe
d
i
n
to
t
w
o
:
inte
l
l
ec
t
u
a
l
b
e
h
avi
o
ur
a
n
d
so
ci
al
b
ehav
i
o
u
r
;
(4
)
Va
lu
e
,
w
h
i
ch
i
n
t
h
i
s
c
ase
i
s
v
alu
e
c
o
n
t
a
i
ne
d
i
n
s
ocie
t
y
o
b
t
a
i
ne
d
from
l
oca
l
s
ocie
t
y
or
g
o
v
ernm
en
t
organ
i
sa
ti
on
(na
t
i
on
ph
ilo
so
ph
y)
[
2].
H
e
nce
,
s
oci
al
s
c
i
en
ce
e
duca
t
i
o
n
i
n
e
lem
e
n
t
ar
y
s
c
hoo
l
em
phas
i
ze
mor
e
on
h
o
w
t
o
e
duca
t
e
s
t
u
d
e
n
t
s
t
o
rec
o
gnise,
com
p
re
h
en
d,
a
nd
to
b
e
ab
le
t
o
a
p
ply
kn
o
w
led
g
e
,
sk
i
l
ls,
val
u
es,
and m
o
ra
l
i
n
to
liv
i
ng a
m
on
g s
o
c
i
e
t
y
and
na
t
i
on.
Referr
ing
to t
he
sa
i
d
ob
j
e
c
t
i
v
e
s
, soc
ial sc
ie
nc
e has a
ve
ry i
m
p
orta
nt ro
l
e for stude
nts
'
liv
es,
bot
h n
o
w
and
i
n
t
he f
utu
r
e. Bu
t
un
f
or
tu
nate
l
y
, t
h
e pr
o
cess of lea
rn
i
n
g
s
oc
ial
stu
d
i
es
i
n e
l
em
en
tary sch
oo
ls st
ill
la
c
k
s t
h
e
atte
n
t
ion
o
f
b
o
t
h tea
c
her
s
a
n
d
st
u
d
e
n
t
s
.
The
pro
b
lem
in
s
oc
ial
st
udi
e
s
l
ea
rn
ing
t
h
at
i
s
o
f
te
n
e
n
c
o
unt
ere
d
i
s
t
h
e
w
e
a
kness
o
f
th
e
lea
r
n
i
ng
p
r
o
c
e
ss.
L
ear
ning
i
s
sti
l
l
teac
her-
cen
t
e
red
,
t
ea
chers
a
r
e
l
e
ss
enc
o
u
r
ag
e
d
t
o
d
e
v
e
l
op
t
h
e
ab
ilit
y
to
t
h
i
n
k
,
l
e
a
r
n
i
ng
i
s
onl
y
di
re
c
t
e
d
a
t
th
e
c
h
il
d'
s
a
b
il
it
y
to
m
e
m
orise
in
for
m
a
t
io
n
[5]
.
T
e
ache
r
-
ce
nt
e
r
e
d
l
ea
rn
i
ng
a
l
so
m
ea
ns
t
ha
t
te
ac
hers
t
en
d
to
b
e
d
o
m
inan
t
(a
ct
i
v
e
l
y
i
n
v
o
l
ve
d)
i
n
ca
rryi
ng
t
h
e
le
ss
on
con
t
e
n
t
t
o
t
he
s
tude
nts
an
d
te
a
c
h
t
h
em
d
irec
t
l
y
t
o
s
t
u
de
nts
in
t
h
e
cl
a
ssroom
t
h
r
o
ugh
l
ectu
r
e
s
,
d
i
s
c
u
ssi
on
a
nd
assi
gnme
n
ts.
C
onve
nti
ona
l
le
arni
ng
i
s
c
a
l
l
e
d
d
i
rec
t
l
ear
nin
g
/
d
i
rec
t
i
n
s
truc
t
i
o
n
[
6].
C
onve
nti
ona
l
l
e
arni
ng
w
h
ic
h
is
a
ls
o
d
o
m
i
na
ted
b
y
t
he
v
iew
t
h
at
k
now
led
g
e
is
a
f
ac
t
th
a
t
m
us
t
be
m
e
m
or
i
s
e
d
c
an
r
e
s
u
l
t
in
s
tude
n
t
s
bec
o
mi
n
g
p
as
sive
i
n
lear
n
i
ng,
w
a
y
s
of
t
h
i
n
k
ing
a
r
e
les
s
d
e
v
elo
p
e
d
b
ec
a
u
se
t
he
y
on
ly
f
oc
u
s
on
t
h
e
the
o
r
y
b
u
t
are
no
t
stim
u
l
a
t
ed
t
o
t
h
in
k
c
r
itic
al
ly
o
n
p
r
oble
m
s
tha
t
o
c
c
ur
i
n
eve
r
y
d
a
y
l
i
f
e
.
T
hi
s
i
s
w
hat
m
a
kes
soci
al
scie
nc
e
le
a
r
ning
se
em
m
o
n
ot
o
n
o
u
s,
b
ori
ng,
a
nd
m
e
m
o
rise
d,
t
he
ref
or
e
students
h
ave
less
interest
i
n
lear
n
i
n
g
s
oc
ia
l st
ud
i
e
s.
The
lea
r
n
i
n
g
p
ara
d
i
g
m
s
h
o
u
l
d
be
c
ha
nge
d
from
t
ea
c
h
er
-ce
n
t
e
re
d
t
o
stu
d
e
nt
-ce
n
t
e
red
so
t
ha
t
th
e
pro
b
lem
o
f
s
o
c
ial
scie
nce
le
a
r
ni
ng
c
a
n
b
e
ove
rc
om
e.
T
he
refore,
stude
n
t
s
are
no
t
on
ly
u
sed
a
s
l
ea
rn
in
g
ob
jec
t
s,
b
u
t
a
s
sub
j
ec
t
s
o
f
l
e
a
r
ni
ng.
S
tude
n
t
s
a
r
e
rea
l
ly
g
u
i
de
d
a
n
d
fac
ili
ta
ted
to
b
u
ild
t
heir
o
w
n
k
n
o
w
l
ed
ge
,
no
t
me
rel
y
m
ov
ing
t
h
e
c
o
nte
n
ts
o
f
the
t
e
a
c
he
r's
hea
d
t
o
t
h
e
s
t
u
d
e
nt'
s
h
ea
d
.
L
e
a
r
ni
ng
s
h
o
ul
d
no
t
o
n
ly
b
e
foc
u
se
d
o
n
p
ro
vi
d
i
ng
t
h
e
o
re
t
i
c
a
l
know
le
d
g
e
,
b
u
t
s
t
ude
n
t
s
a
r
e
a
l
so
g
ive
n
l
ear
ning
e
xpe
ri
ence
s
t
h
a
t
a
r
e
a
lway
s
rela
t
e
d
to
t
he
actua
l
pr
ob
lem
s
t
ha
t
occ
u
r
i
n
t
he
ir
e
nv
iro
n
m
e
n
t
[
7
]
.
T
h
u
s
,
t
h
e
s
e
l
e
c
t
i
o
n
a
n
d
a
p
p
l
i
c
a
t
i
o
n
o
f
lear
n
i
n
g
a
p
p
ro
a
c
hes
a
n
d
mo
de
ls
g
rea
tly
i
nfl
u
e
n
ce
t
he
a
c
h
i
e
vem
e
n
t
o
f
l
ea
rni
ng
g
o
a
l
s.
T
he
s
t
ude
n
t
-ce
n
te
re
d
educ
a
t
i
o
n
proc
e
s
s,
w
h
i
c
h
a
i
m
s
to
d
e
v
e
l
o
p
ke
y
s
k
ills,
ne
e
d
s
t
o
i
n
v
o
l
v
e
a
pr
oact
i
v
e
a
ppr
oa
ch
t
ha
t
fac
ili
ta
tes
st
u
d
e
nt
i
nvo
l
v
emen
t
in
r
e
s
earc
h
,
sci
e
n
t
i
f
i
c
i
nq
ui
ry
,
an
a
l
ysi
s
o
f
pr
o
b
l
e
ma
t
i
c
s
i
tua
t
ions
o
r
so
l
v
i
n
g
r
eal
p
r
o
b
l
e
m
s
o
r
j
u
s
t
h
y
p
o
t
h
e
t
i
c
a
l
p
r
o
b
l
e
m
s
.
T
h
r
o
u
g
h
i
t
s
s
p
e
c
i
f
i
c
i
t
y
,
P
r
ob
le
m
Base
d
Lea
r
ni
n
g
(
P
B
L)
c
a
n
b
e
a
ma
jor c
o
mpo
n
e
nt
i
n
suc
h
tea
chi
n
g appr
oach
e
s
[
8].
P
r
oblem
B
ased
L
earning
(PBL)
is
o
ften
r
ef
erred
t
o
a
s
Inqui
r
y
B
a
se
d
Lea
r
n
i
n
g
.
Cogn
i
t
i
v
e-
con
s
truc
t
i
v
i
s
t
v
ie
w
s
,
w
h
i
c
h
un
der
lie
p
rob
l
em
-base
d
l
ea
r
n
in
g
(P
B
L),
mos
tly
f
o
llow
Pi
a
g
e
t
.
T
h
i
s
v
ie
w
states
t
h
a
t
s
t
u
d
e
n
t
s
a
t
a
n
y
a
g
e
a
r
e
a
c
t
i
v
e
l
y
i
n
v
o
l
v
e
d
i
n
t
h
e
p
r
o
c
e
s
s
o
f
ob
ta
in
i
n
g
inf
o
rm
ation
a
nd
bu
i
l
d
i
ng
t
he
i
r
o
w
n
kn
ow
le
d
g
e.
W
hile
V
y
got
sk
y
bel
i
e
v
e
t
h
a
t
i
nte
l
l
i
g
e
n
ce
d
e
v
el
o
p
s
be
ca
us
e
in
div
i
dua
ls
f
ac
e
co
nfus
i
n
g
new
expe
r
i
ence
s,
so
t
h
e
y
tr
y
to
resol
v
e
g
a
ps t
hat
a
r
ise from
thi
s
e
xper
i
ence
[9].
A
l
o
ng
w
i
t
h
t
h
e
c
han
g
i
n
g
o
f
the
time
s
,
m
a
ny
e
xpe
r
t
s
pro
v
ide
d
d
e
f
i
n
iti
o
n
s
ab
o
u
t
the
PBL
m
ode
l.
P
r
ob
le
m
Base
d
Lea
r
nin
g
(
P
B
L)
i
s
a
lea
r
nin
g
m
ode
l
t
h
a
t
i
s
orie
n
t
e
d
t
o
the
a
c
ti
ve
r
ole
of
s
tude
n
t
s
b
y
e
x
pos
in
g
stude
n
t
s
t
o
a
p
rob
l
em
w
it
h
th
e
aim
of
s
tu
de
nts
be
i
ng
a
b
le
t
o
a
c
t
i
ve
ly
s
ol
ve
e
x
i
s
t
in
g
pr
o
b
l
e
ms
a
nd
t
he
n
dr
aw
conc
l
u
s
i
o
n
s
b
y
determ
in
ing
w
h
at
s
t
e
p
s
t
he
y
sho
u
l
d
be
d
one
[
10].
T
he
p
r
o
ble
m
p
re
se
nt
e
d
i
s
a
prob
lem
r
e
late
d
to
t
he
r
e
a
l
w
o
r
l
d.
T
h
e
c
l
o
ser
t
o
t
he
r
ea
l
w
o
r
l
d,
t
he
b
e
tter
t
h
e
e
f
f
ec
t
w
i
l
l
b
e
on
im
prov
i
ng
lear
ner
sk
i
l
l
s
[
11]
.
Ba
se
d
on
t
h
e
pro
b
lem
g
i
ve
n
,
s
tu
den
t
s
w
o
rk
t
o
g
e
t
her
in
g
ro
ups
t
o
s
olv
e
t
hese
p
ro
ble
m
s
by
referr
in
g
to
t
he
kn
ow
le
d
g
e
t
h
e
y
a
lrea
d
y
h
a
v
e
and
from
r
ele
v
a
n
t
new
i
n
for
m
atio
n.
Wi
t
h
t
hi
s
P
B
L
m
ode
l
,
s
tude
n
t
s
a
r
e
direc
tly
in
vol
ve
d
i
n
l
e
a
r
ni
ng
a
c
tiv
i
t
i
e
s
so
t
ha
t
t
h
e
i
r
know
led
g
e
c
a
n
b
e
a
b
s
o
rbe
d
p
ro
pe
rl
y.
I
n
add
i
t
i
o
n
,
th
i
s
l
ear
ning
mode
l
a
l
l
o
w
s
s
tude
n
t
s
t
o
i
nt
era
c
t
a
nd
c
o
l
l
abora
t
e
w
ith
g
rou
p
f
r
i
e
n
d
s
so
t
ha
t
it
i
s
e
x
pecte
d
t
o
enc
our
a
g
e
stude
n
t
s
'
in
t
er
e
s
t
i
n
lea
rnin
g.
V
a
rious
s
t
u
die
s
i
n
t
h
e
w
o
rl
d
o
f
e
duca
t
i
o
n
rec
o
gn
i
z
e
t
h
e
suc
c
e
s
s
o
f
th
e
i
r
l
e
a
r
ni
ng
b
y
appl
yi
n
g
Pro
b
l
e
m
Ba
se
d
Lea
r
ni
ng
.
S
t
ud
en
t
s
w
ho
s
t
u
dy
w
ith
P
BL
s
c
o
re
h
ig
h
after
l
e
arni
ng
a
n
d
th
e
i
r
sc
ien
t
i
f
i
c
p
r
o
ce
ss
sk
i
l
ls
a
lso
incr
e
a
se
[
12]
.
In
a
dd
i
tio
n
to
s
c
o
r
e
s,
h
i
g
h
sc
h
o
o
l
s
tu
d
e
nt
s
ma
jori
ng
i
n
so
ci
a
l
s
t
udi
e
s
s
ub
j
ect
t
o
PBL
mode
l
s
a
l
s
o
d
e
m
onstra
t
e
t
h
e
a
b
i
l
ity
t
o
so
lve
ge
ogra
p
h
i
c
pro
b
lem
s
t
h
a
t
a
r
e
h
i
g
h
e
r
t
h
a
n
t
h
e
c
l
a
s
s
e
s
sub
j
e
c
t
t
o
c
o
n
v
e
nt
i
ona
l
m
o
del
s
[
1
3
].
A
t
the
e
nd of the
P
BL proc
ess,
s
tu
den
t
s
can i
den
t
i
f
y
a
n
d
s
o
lve
pr
o
b
le
ms
w
ith
t
h
e
ir
o
w
n
i
de
as
a
n
d
a
bi
l
ities
a
nd
d
e
ve
l
op
t
h
eir
c
r
ea
t
i
ve
t
h
i
n
k
i
n
g
,
w
h
i
c
h
is
o
ne
o
f
the
hi
gh
e
r-or
d
e
r
th
ink
i
ng s
k
i
lls [1
4
].
The
r
i
g
h
t
l
e
ar
ni
n
g
m
o
d
e
l
i
s
on
l
y
one
o
f
th
e
fac
t
ors
th
at
can
i
n
flue
nce
t
h
e
s
u
cc
ess
o
f
t
he
l
ear
nin
g
proce
s
s.
L
e
a
rnin
g
m
ode
ls
f
a
l
l
int
o
t
he
c
a
t
e
gor
y
of
e
x
t
er
n
a
l
fa
c
tors,
w
h
i
l
e
t
h
ere
is
s
t
ill
o
n
e
m
o
re
f
ac
tor,
nam
e
ly
t
he
i
n
t
erna
l
fac
t
or,
w
h
i
c
h
co
ver
s
t
hree
m
or
e
fa
ctors.
T
h
ese
thre
e
fac
t
ors
are
physic
a
l
f
a
c
tors,
psy
c
h
o
l
og
ic
al
f
act
ors,
a
nd
f
at
i
g
ue
f
ac
t
o
rs.
P
hysi
c
a
l
f
actors,
i
nc
lu
di
ng
he
al
t
h
a
n
d
d
isa
b
i
lit
y
fac
t
or
s;
P
s
ycho
l
o
gica
l
fa
ct
ors,
i
nc
lu
d
i
n
g
i
n
t
e
l
l
i
ge
nc
e,
a
tten
t
ion,
i
nte
r
ests,
ta
len
t
s,
m
otive
s
,
ma
turity,
an
d
rea
d
i
n
e
s
s;
F
a
t
i
g
u
e
fac
t
ors,
i
nc
lu
di
n
g
p
h
y
s
i
ca
l
fat
i
gue
a
nd
sp
iri
t
ua
l
fati
g
ue
[
15]
.
F
r
om
s
ome
of
t
h
e
se
i
n
t
e
r
na
l
fa
ctors,
Evaluation Warning : The document was created with Spire.PDF for Python.
Int.
J
. Eva
l
. & Re
s
. Educ
.
IS
S
N
:
2252-
88
22
T
h
e
in
f
l
u
en
c
e
of
pro
b
l
em
bas
ed lea
r
n
i
ng t
o
wards s
o
c
i
a
l
sc
i
e
nce
lear
n
i
ng.
.. (B
erti
D
y
ah
Perm
at
asar
i)
41
there
is
a
n
in
t
e
resti
n
g
o
n
e
to
s
tud
y
,
nam
e
l
y
t
he
i
n
t
ere
s
t
fa
c
t
o
r.
I
nt
e
r
es
t
i
n
l
e
a
rn
in
g
i
s
a
n
i
n
d
i
vi
d
u
al
's
t
e
n
d
e
nc
y
t
o
h
av
e
pl
e
a
s
ure
wi
t
hou
t
c
o
erc
i
on
s
o
th
at
i
t
c
a
n
cau
se
c
h
a
n
g
e
s
in
know
l
e
d
g
e
,
s
ki
l
l
s
a
n
d
be
ha
v
i
o
u
r.
I
ntere
s
t
ca
n
affec
t
t
he
qua
l
ity
o
f
ach
i
e
ve
me
nt
o
f
s
t
u
d
en
t
lea
r
n
i
ng
o
u
tc
o
m
e
s
in
c
e
r
tai
n
f
iel
d
s
of
s
tud
y
b
e
c
a
u
se
h
e
w
ill
f
o
cu
s
mo
re
o
n
t
h
e
f
i
eld
t
h
a
n
o
t
h
er
s
tu
de
nts
[1
6
]
.
Th
ere
f
o
r
e
,
t
e
ac
hers
m
ust
rec
o
g
n
i
z
e
h
o
w
m
u
ch
l
ear
ning
in
t
e
rest
s
t
u
den
t
s
ha
ve
a
n
d
g
r
o
w
t
h
em
s
o
th
a
t
t
he
l
e
a
r
n
i
n
g
pr
oce
ss
can
r
u
n
e
ffect
ive
l
y
a
nd
a
c
h
iev
e
d
succe
ss
i
n lea
r
ni
ng.
S
u
cc
ess
i
n
l
e
a
r
ni
ng
ca
n
be
m
easure
d
fro
m
three
d
o
m
a
in
s;
t
hey
a
r
e
cog
n
i
t
i
ve
,
affe
cti
v
e,
a
n
d
p
s
yc
ho
mot
o
r.
T
h
e
c
og
nit
i
v
e
do
main
i
n
c
lud
e
s
in
t
e
ll
ec
tu
al
a
b
i
lit
i
e
s
a
ss
ocia
te
d
w
ith
m
e
n
t
a
l
proce
s
se
s
or
a
c
t
i
vit
i
es
f
ro
m
l
o
w
t
o
h
i
g
h
cat
e
g
o
r
i
e
s;
a
f
f
ect
iv
e
d
o
m
ai
n
s
a
re
r
el
at
e
d
t
o
fe
el
ing
s
,
emo
tio
n
s
,
at
tit
ud
es,
d
e
g
r
e
e
s
of
a
c
c
e
p
ta
nc
e
or
r
ejec
tio
n
o
f
a
n
o
b
jec
t
;
an
d
t
h
e
psy
c
h
o
m
o
t
o
r
do
m
a
in
i
s
r
e
late
d
t
o
t
he
c
o
m
petenc
e
to
d
o
w
o
rk
tha
t
i
nv
olv
e
s
me
mbe
r
s,
a
nd
com
p
ete
n
c
i
es
r
e
l
a
t
e
d
t
o
ph
ys
ic
al
m
o
v
em
en
t
[17]
.
Le
arni
ng
o
utc
o
m
e
s
m
e
a
s
ure
d
in
t
hr
ee
d
o
m
a
i
n
s
by
Bl
oom
w
ere
later
revi
sed
by
A
n
d
e
r
s
on
&
K
r
at
hw
o
h
l
.
T
he
c
ogn
it
i
v
e
dom
ai
n
c
o
n
s
ists
o
f
rem
e
mbe
r
ing,
unde
rsta
n
d
i
ng,
a
pp
ly
ing,
a
na
ly
si
n
g
,
e
v
al
uat
i
n
g
,
a
n
d
c
r
ea
t
i
n
g
;
a
ffe
c
t
i
ve
doma
i
n
c
ons
is
t
s
o
f
re
ce
i
v
i
n
g
,
r
e
s
p
o
ndi
ng
,
e
v
al
u
a
t
i
ng
,
o
r
g
a
ni
z
i
ng
,
an
d
fo
rmi
ng
c
h
ar
ac
t
e
riza
t
i
on
;
ps
yc
h
o
m
o
t
o
r
doma
i
ns
c
o
n
s
i
st
o
f
perc
ep
ti
o
n
,
mani
p
u
la
t
i
ng,
p
e
r
for
m
in
g
pr
oc
edure
s
(
pr
eci
si
on),
do
ing
w
e
ll
a
nd
c
o
r
r
ec
t
l
y
(artic
u
l
a
t
i
on),
a
n
d
ac
t
i
n
g
na
t
ur
al
l
y
(
na
tura
lisa
t
i
o
n)
[
18].
Th
e
l
e
arni
ng
o
ut
co
me
s
h
e
re
a
s
t
h
e
d
e
p
e
nd
e
n
t
v
a
ri
abl
e
a
re
t
h
e
e
f
fec
t
s
of
t
he
a
p
p
l
i
c
a
ti
o
n
o
f
t
h
e
lear
n
i
n
g
m
ode
l
and
stu
d
e
n
t
i
n
tere
st
i
n
s
o
cia
l
s
c
i
ence
.
The
i
n
te
n
d
ed
l
ea
rn
i
ng
ou
tc
om
es
a
re
c
ogn
i
t
i
v
e
d
o
m
a
i
ns
w
h
ic
h co
nsist of C1-
C6
(
r
e
mem
b
erin
g to cre
ate)
. R
e
me
mb
e
r
in
g
is
r
e
late
d
t
o
stu
de
n
t
s
'
ab
ili
ty to rec
o
gni
se an
d
rec
a
ll
;
u
n
d
er
sta
n
d
i
ng
is
r
ela
t
ed
t
o
the
a
b
il
i
t
y
t
o
i
n
t
er
pret,
e
xem
p
li
fy,
cla
ssify,
summ
ar
i
s
e
,
c
onc
lu
de
,
com
p
are
,
and
ex
p
l
ai
n
;
a
pp
lyi
n
g
is
a
bout
t
he
a
b
i
l
ity
t
o
im
plem
en
t,
e
xe
cu
t
e
/
im
p
l
e
m
e
n
t;
ana
l
y
s
e
r
e
late
d
to
t
he
a
bi
lit
y
t
o
di
st
i
n
g
u
i
sh,
organ
i
se,
a
n
d
as
soc
i
ate
;
a
sse
ss
r
e
late
d
t
o
t
he
a
bi
lit
y
t
o
ex
ami
n
e
and
c
r
i
tic
i
z
e;
a
n
d
c
rea
t
i
n
g
i
s
rela
t
e
d to t
he
a
bi
l
i
t
y
t
o
pro
d
u
ce,
plan,
a
nd prod
uce
[19]
.
Rese
arc
h
es
t
ha
t
ha
ve
b
ee
n
c
o
nd
uc
ted
s
o
le
ly
a
r
e
t
o
i
m
pro
v
e
t
he
q
u
a
l
ity
o
f
lea
r
n
i
ng
in
w
a
y
s
tha
t
c
an
a
t
t
r
a
c
t
stud
e
n
ts
'
i
n
t
e
re
sts
and
sti
m
ul
a
t
e
stu
d
e
nt
s
to
t
hink
c
ri
tica
lly.
P
r
evi
o
us
s
tu
d
i
es,
espec
i
a
l
l
y
f
or
P
BL
mode
l
s
h
ave
show
n
th
e
e
f
fe
c
tive
n
ess
of
t
h
i
s
m
ode
l
in
i
m
p
ro
vin
g
t
h
e
abi
l
it
y
a
n
d
l
e
arnin
g
out
co
mes
of
h
ig
h
sch
o
o
l
G
eo
gra
p
h
y
s
ubje
c
t
s
[13]
;
S
c
ience
[14],
[8],
[
2
0
];
a
nd
C
h
em
is
t
r
y
[1
2],
[21].
F
r
om
t
he
v
ari
o
u
s
s
t
u
d
i
es
abo
v
e,
l
e
a
r
n
i
n
g
w
i
th
P
BL
m
ode
l
s
,
in
a
d
d
i
t
i
o
n
t
o
i
m
p
rov
i
ng
l
e
a
r
n
ing
ou
t
c
o
m
e
s
c
a
n
a
l
s
o
i
m
p
r
o
v
e
ot
h
e
r
a
b
ili
ti
e
s
a
nd
s
ki
ll
s
in
s
t
u
d
e
n
t
s
su
ch
a
s
p
r
o
b
l
e
m-s
o
l
v
i
n
g
sk
i
lls
,
scien
t
ific
p
r
o
c
e
ss
s
k
i
l
ls,
cr
itica
l
t
h
i
nk
ing
sk
il
ls
and
c
r
ea
t
i
ve
.
A
m
ong
t
h
ese
stud
ies,
P
BL
i
s
w
i
de
l
y
u
se
d
for
sc
ie
n
ce
l
e
a
r
n
ing,
w
h
i
le
f
or
s
oc
ial
scie
nce
i
s
view
e
d
f
rom
t
h
e
in
ter
e
s
t
o
f
s
t
u
d
e
n
t
s
i
n
ele
m
entar
y
s
c
h
oo
l
lea
r
n
i
n
g
i
s
s
t
i
l
l
r
ar
e.
T
here
fore
,
it
i
s
nec
e
s
s
ary
t
o
con
d
u
ct
r
e
s
ear
ch
t
o
e
x
am
ine
the
e
f
fec
tive
n
e
s
s
of
P
B
L
i
n
s
o
cia
l
s
t
ud
i
e
s
le
arni
ng
i
n
e
lem
e
ntar
y
sch
o
o
l
s
b
ase
d
on s
t
u
d
e
n
ts
'
lea
r
n
i
ng
in
ter
e
sts.
Th
e
a
i
m
o
f
t
his
stud
y
i
s
t
o
de
t
e
rmi
n
e
di
ff
ere
n
ce
s
i
n
t
h
e
e
ff
ec
t
o
f
P
r
o
b
le
m
Base
d
Le
ar
ni
n
g
(
P
B
L)
mode
l
an
d
D
i
rec
t
I
nstruc
tio
n
(D
I)
on
the
re
su
l
t
s
o
f
s
ocia
l
s
c
ie
n
ces.
Thi
s
s
t
u
dy
a
l
s
o
ex
a
m
i
n
es
t
h
e
d
if
feren
c
e
s
i
n
s
oc
i
a
l
sci
e
nce
l
e
arnin
g
out
co
mes
ob
t
a
in
ed
b
y
g
r
o
u
p
s
o
f
st
ud
e
nts
w
h
o
ha
d
d
i
ffe
r
ent
le
ve
l
s
o
f
le
ar
nin
g
in
t
e
rest
;
a
nd
t
h
e
i
n
ter
a
c
t
i
on
b
e
tw
e
e
n
the
l
e
a
r
ni
ng
m
o
de
l
a
nd
t
h
e
lear
n
i
n
g
i
n
t
ere
s
t
of
s
t
ude
n
t
s
t
o
wa
rds
soc
i
al
sc
i
e
n
c
e
l
e
a
r
ni
ng
o
ut
co
me
s.
2.
RESEARCH
M
ETH
O
D
Th
is
s
t
u
dy
is
a
qua
si-e
x
p
eri
m
ental
st
ud
y
w
ith
p
re
-tes
t
a
nd
po
st
-te
s
t
c
o
n
t
r
o
l
gro
u
p
d
es
ig
n.
T
he
varia
b
l
e
s
of
t
h
i
s
stu
dy
c
o
ns
iste
d
of
l
ear
nin
g
m
ode
ls
a
s
inde
pe
n
de
n
t
v
a
r
iab
l
es,
whi
c
h
ar
e
Problem
B
ase
d
Lea
r
ni
ng
(P
B
L
)
a
nd
D
i
re
ct
I
nstruct
i
o
n
(D
I)
m
odels;
l
e
arni
ng
in
t
e
r
est
a
s
m
ode
rat
o
r
va
riab
les
w
i
t
h
t
he
dime
n
s
i
o
ns
o
f hi
g
h
l
ea
r
n
i
n
g
i
n
tere
st (H
L
I),
m
ode
rate lear
n
i
n
g
i
n
t
e
r
est
(
M
LI
), and low
le
a
r
ning i
n
t
e
res
t
(LLI);
a
n
d
so
ci
al
s
c
i
en
c
e
l
ea
rn
ing
ou
t
c
o
m
es
a
s
d
e
p
e
nd
en
t
v
a
ri
abl
e
.
In
ad
di
t
i
on
,
t
h
e
r
e
a
r
e
con
f
o
und
i
n
g
v
a
r
i
abl
e
s
that
are
not
i
nc
lude
d
in
t
he
s
tud
y
,
but
c
an
a
f
f
ec
t
t
h
e
de
pe
nde
n
t
v
a
r
i
a
bl
e
.
C
onfo
und
ing
v
a
ri
a
b
l
e
s
i
n
t
hi
s
s
t
udy
a
re
l
e
a
r
ni
ng
t
i
m
e
an
d
h
e
a
l
th
.
Co
nf
oun
di
n
g
v
a
r
i
a
b
l
e
s
"
Lea
r
nin
g
T
i
m
e
"
c
a
n
b
e
minimi
ze
d
b
y
c
a
rryin
g
out
l
e
a
r
n
in
g
in
a
ll
classes in
t
he
m
orn
i
n
g
.
Mea
n
wh
i
l
e
,
"
Healt
h
"
can
n
o
t
be
c
o
nt
r
o
lled
;
t
he
ref
o
re
t
he
e
ffe
c
t i
s
c
o
n
s
i
der
e
d
t
h
e
sam
e
o
r ignored.
Th
is
r
e
s
e
a
rc
h
w
a
s
c
o
n
duc
t
e
d
at
t
he
P
ol
o
k
ar
t
o
D
istr
ict
E
l
e
m
entar
y
S
c
ho
o
l
,
S
uko
harj
o
Re
genc
y.
T
h
e
po
p
u
l
a
tio
n
of
t
h
i
s
st
u
dy
w
a
s
a
ll
fo
ur
t
h
-
g
rade
s
tu
de
nt
s
o
f
p
ub
l
i
c
el
e
m
e
n
t
a
ry
s
c
h
oo
l
s
i
n
Po
lok
a
rt
o
Di
st
ri
ct
,
S
uko
har
j
o
Re
g
e
ncy
c
ons
is
tin
g
of
43
sc
ho
o
l
s.
T
he
s
a
m
ple
in
t
h
i
s
st
ud
y
to
o
k
6
c
las
s
es
from
6
e
l
e
m
e
n
tar
y
sc
ho
ol
s
tot
a
ll
in
g
109
s
t
u
d
e
nt
s,
w
i
t
h
5
0
s
t
u
d
e
nt
s
f
r
o
m
t
h
e
e
xp
er
ime
n
t
a
l
gro
u
p
t
r
ea
te
d
us
ing
P
r
ob
l
e
m
Bas
e
d
Lea
r
ni
ng
m
o
d
e
l
a
n
d
5
9
s
t
u
d
e
nt
s
from
the
con
t
ro
l
gr
ou
p
w
e
re
t
re
a
ted
u
s
ing
D
i
rec
t
I
ns
truc
ti
o
n
m
ode
l
s
.
The
rese
arc
h
s
am
ple
w
a
s
take
n
by
s
tra
t
i
f
ied
c
l
u
s
te
r
ra
n
dom
s
am
p
l
in
g
t
ec
hn
ique
.
S
t
rat
i
fied
c
l
u
ster
r
a
n
d
o
m
sam
p
lin
g
is
a
c
om
bi
na
t
i
on
o
f
s
tra
tif
i
e
d
ran
d
o
m
sam
p
l
i
n
g
m
etho
ds
w
it
h
c
l
u
s
ter
r
a
nd
om
s
a
m
ple
s
,
bec
a
u
se
t
he
po
p
u
l
a
tio
n
o
r
t
ar
g
e
t p
o
p
u
l
at
io
n
i
s
q
uite
l
arge
and
w
i
de,
is rar
ely r
a
nd
om
or
has the
sam
e
chara
cter
i
s
t
i
cs [
22].
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
Int.
J
.
Eval. & R
es. Educ.
Vol.
8, No. 1, March
2
019
:
39
- 4
6
42
D
a
ta
c
ol
l
e
c
t
io
n
uses
t
es
t
a
n
d
que
st
i
o
n
n
a
i
re
i
ns
t
r
ume
n
ts.
Mul
tip
le
c
h
o
i
ce
t
est
ins
t
rum
e
n
t
s
w
ith
a
t
o
t
a
l
of
2
5
q
u
est
i
o
ns
w
e
r
e
u
s
ed
t
o
fi
n
d
o
u
t
t
he
r
e
s
ult
s
o
f
st
u
d
en
ts
'
soc
i
al
s
tu
d
i
es
l
e
a
r
ni
ng.
Q
ues
t
io
nna
ir
e
in
strum
e
n
t
s
i
n
t
he
f
orm
o
f
1
5
s
t
atem
en
ts
r
e
p
resent
t
he
m
agni
tud
e
o
f
s
t
ude
n
t
s
'
i
n
t
e
r
est
i
n
s
oc
i
a
l
sc
ience
.
Q
u
est
i
onna
ire
w
i
t
h
L
ikert
sca
l
e
(1-
5
)
t
o
c
las
s
ify
s
t
u
d
e
n
t
s
'
l
e
a
rni
ng
inte
res
t
i
n
t
o
t
h
re
e
le
v
e
l
s
,
w
h
ic
h
ar
e
hi
g
h
,
me
dium,
an
d
l
o
w
.
B
ef
ore
be
ing
u
s
ed
f
or
d
a
t
a
c
o
lle
c
t
i
o
n,
e
ac
h
in
s
t
r
u
m
e
n
t
h
a
s
b
e
e
n
t
e
s
t
e
d
f
i
r
s
t
.
F
u
r
t
h
e
r
m
o
r
e
,
the
tes
t
i
ns
tru
m
ents
w
ere
t
e
st
e
d
f
or
c
ont
e
n
t
val
i
d
i
t
y
,
l
e
ve
l
of
d
ifficu
l
t
y,
d
i
s
ti
ng
u
i
shi
ng
pow
er
,
and
r
e
lia
bi
lit
y,
w
h
i
l
e
th
e
ques
t
io
n
n
a
i
re
i
nstru
m
ents
w
ere
tes
t
ed
f
or
c
on
te
nt
v
al
idi
t
y,
i
n
t
e
r
n
a
l
c
o
n
s
iste
nc
y,
a
n
d
r
el
ia
bi
l
ity.
Test
the
va
li
di
ty
o
f
the
t
e
s
t
r
esu
lts
o
f
lear
n
i
n
g
o
ut
com
e
s
an
d
q
u
e
s
t
i
onn
a
i
res
o
f
i
n
t
e
r
e
s
t
in
l
e
a
r
ni
ng
i
s
d
o
n
e
t
h
r
o
ugh
expe
r
t
j
u
d
g
me
nt,
bo
th
i
n
te
rm
s
of
m
at
er
i
a
l
,
c
o
n
s
t
ruct
i
o
n
,
a
nd
l
a
n
gua
g
e
.
Te
st
o
f
t
h
e
le
vel
of
d
iffi
cul
t
y
,
di
st
i
n
g
u
i
sh
i
n
g
pow
er
a
nd
r
e
l
i
a
b
il
it
y
o
f
t
he
t
e
s
t
in
st
r
u
me
n
t
w
a
s
c
arr
i
ed
o
ut
w
i
t
h
the
IT
EMAN
ap
p
lic
ati
o
n,
w
h
i
l
e
t
h
e
i
n
terna
l
c
on
si
s
t
e
n
c
y
t
es
t
an
d
t
h
e
r
e
lia
bi
lit
y
o
f
t
he
inte
res
t
l
ear
nin
g
q
ue
sti
o
nn
aire
i
n
s
trum
en
t
s
w
er
e
co
ndu
cted
w
ith
t
h
e
h
elp
of
t
h
e
S
PSS
ap
plication
.
B
ased
o
n
th
e
r
esult
s
o
f
a
ser
i
e
s
o
f
tests
of
v
alidity
and
relia
bi
l
ity,
t
h
e
tes
t
i
ns
t
r
ume
n
t
s
f
or
s
oc
ial
s
t
u
d
i
e
s
lea
r
ni
ng
o
utc
o
m
e
s
a
n
d
q
u
es
ti
o
n
n
a
ires
o
f
i
n
tere
st
i
n
so
cia
l
stud
ies
ha
ve
m
e
t
the
c
ri
teria
for
re
v
i
e
w
ing
i
t
e
m
s
t
h
a
t
a
r
e
go
o
d
a
nd
suit
a
bl
e fo
r re
se
arc
h
.
A
t
t
he
d
a
t
a
a
n
aly
s
i
s
s
ta
ge,
b
e
fore
b
e
i
ng
gi
ve
n
a
d
i
ffe
r
en
t
t
r
ea
tm
en
t,
t
es
ts
w
e
r
e
ca
rried
o
u
t
f
or
t
he
prer
equ
i
s
i
te
t
e
s
ts
o
f
va
r
i
ance
a
na
ly
sis
a
n
d
bala
nce
tes
t
s
i
n
t
h
e
e
x
p
e
ri
men
t
a
l
g
roup
a
nd
t
h
e
c
ont
rol
g
r
ou
p
.
P
r
e
r
e
quisi
te
t
e
s
t
i
n
cl
u
d
es
p
op
u
l
at
ion
nor
ma
l
i
t
y
t
e
s
t
us
i
n
g
Li
l
l
i
e
fors
m
etho
d
an
d
po
pu
la
ti
o
n
v
arianc
e
homo
g
e
n
ei
ty
t
est
us
i
ng
t
h
e
Ba
rt
l
e
t
t
m
et
ho
d,
w
hi
le
t
he
b
alance
t
es
t
use
s
one-
w
ay
A
n
ova.
The
re
su
l
t
i
s
t
h
a
t
bo
th
g
r
o
u
p
s
c
o
m
e
f
r
o
m
p
o
p
u
la
t
i
o
n
s
t
h
at
a
re
nor
ma
l
l
y
di
stri
bu
te
d
a
nd
hom
o
g
e
n
eo
us
;
a
n
d
h
a
ve
a
b
ala
n
c
e
d
in
itia
l
a
b
i
l
ity.
A
f
ter
the
sam
p
l
e
w
as
g
ive
n
a
d
iffer
e
n
t
t
r
eat
m
e
nt,
the
c
o
l
l
e
c
ted
or
a
na
lyse
d
w
a
s
t
h
e
n
a
na
ly
sed
us
i
n
g
t
w
o-w
a
y
A
nova w
i
t
h u
n
e
q
ua
l c
e
l
l
s.
3.
RESULT
S
A
N
D
DISCU
SSIO
N
Be
fo
re
b
ei
ng
g
iv
en
t
re
a
t
men
t
,
bo
t
h
g
roups
w
ere
t
e
st
ed
t
h
r
o
ugh
p
re-te
s
t.
A
ft
er
b
eing
treat
e
d
by
app
l
yi
ng
P
B
L
m
o
d
e
l
f
o
r
the
e
xper
i
m
e
nta
l
g
r
oup
a
nd
D
I
f
o
r
t
he
c
o
ntro
l
gro
up,
t
he
a
bi
l
i
t
y
o
f
s
t
u
d
e
n
ts
w
as
tes
t
ed
a
ga
i
n
b
y
g
i
v
i
ng
a
post-
t
e
st
t
o
f
i
n
d
o
ut
t
he
l
e
a
rn
in
g
ou
t
com
e
s
i
n
s
ocia
l
s
t
u
d
i
e
s
l
e
a
rni
ng.
B
ot
h
t
h
e
pr
e-
tes
t
a
nd
pos
t-te
st
d
a
t
a
w
e
r
e
t
e
s
te
d
for
n
o
rm
a
l
i
t
y
us
in
g
t
h
e
Li
l
i
e
f
o
rs
m
e
t
ho
d
a
n
d
t
e
s
t
ed
f
o
r
h
o
m
og
e
n
e
i
ty
u
si
ng
the
B
a
rt
l
e
tt
te
s
t
.
From
t
he
r
esul
t
s
o
f
t
h
e
pre-te
st,
the
m
ean
w
a
s
obta
i
ned
X
=
4
7.8
4
i
n
the
expe
r
i
me
nt
a
l
g
rou
p
and
X
=
43.
9
3
i
n
the
c
o
nt
r
o
l
g
r
ou
p.
T
hen
fro
m
t
he
r
esults
o
f
t
h
e
po
s
t
-te
s
t,
t
h
e
m
ea
n
w
a
s
ob
ta
ine
d
X
= 7
4
.
2
4
in t
he
e
x
p
erim
enta
l gr
o
u
p
a
n
d
X
= 51.
80
i
n
t
he c
on
tro
l
g
ro
u
p
c
lass.
D
a
ta
t
hat
has
b
een
k
n
o
w
n
t
o
be
n
orma
l
an
d
homo
g
e
n
e
o
us,
th
e
n
t
e
s
t
e
d
the
hy
p
o
t
h
es
is
w
ith
t
w
o
w
a
y
An
ov
a
di
ff
e
r
en
t
cell
p
a
th
s.
T
h
e
r
esu
l
t
s
o
f
h
ypo
th
e
s
i
s
t
e
s
ti
ng
w
i
t
h
tw
o-w
a
y
A
n
o
v
a
ca
n
be
s
e
e
n
i
n
T
a
ble
1.
Thro
ug
h hy
p
o
t
h
es
is
t
e
s
t
i
n
g,
o
bta
i
ned F
a
=
5
4
.
929 >
3.9
3
3
;
F
b
=
5
.
4
09 > 3.
08
5
;
F
ab
= 0.
3
3
0
<
3.
0
8
5
s
o t
h
at t
h
e
dec
i
si
o
n
o
f
the
H
0A
t
e
s
t
i
s
r
e
j
e
c
t
e
d
,
H
0B
i
s
re
j
e
ct
ed,
an
d
H
0AB
i
s
a
cce
p
t
e
d
.
H
0A
r
ej
ec
t
e
d
me
a
n
s
th
at
t
h
e
re
i
s
an
in
flue
nce
betw
e
e
n
P
B
L
a
n
d
D
I
l
ea
rnin
g
m
o
del
s
o
n
so
c
i
a
l
s
cie
n
ce
l
e
a
r
ni
n
g
o
ut
c
o
mes;
H
0B
i
s
re
j
e
ct
ed
m
e
a
n
i
ng
tha
t
ther
e
a
re d
iffe
rence
s
in t
h
e inf
l
ue
nce
of
l
e
a
rni
ng i
n
ter
e
s
t o
n
s
oc
ia
l
sc
ie
nce
le
ar
ni
n
g
o
utc
o
m
e
s; a
nd
H
0AB
i
s
ac
cept
e
d,
m
e
a
ni
n
g
t
hat
the
r
e
is
n
o
i
n
t
e
rac
t
i
o
n
be
tw
ee
n
lea
r
ni
n
g
m
o
d
e
l
s
a
n
d
le
arni
n
g
i
nt
ere
s
t
in
s
oc
i
a
l
scie
nc
e
l
e
a
r
ni
ng
ou
t
c
o
mes
.
Tab
l
e
1. A
nov
a
test r
esul
t
s
on
soc
i
al sc
i
e
n
c
e
l
ear
ning
o
u
tc
ome
s
Sourc
e
F
co
u
n
t
F
ta
b
l
e
T
e
st Dec
ision
Lea
r
nin
g
M
ode
l
(
A
)
54.
929
3.
933
H
0A
is
re
j
ec
t
e
d
L
ear
n
i
n
g
I
n
t
er
es
t
(
B
)
5.
409
3.
085
H
0B
is r
e
j
ec
te
d
In
t
e
r
a
c
t
i
o
n
(A
B
)
0.
330
3.
085
H
0A
B
i
s
ac
ce
p
t
e
d
3.1.
Effec
t
of lear
ning
m
ode
l
s
on
s
o
c
ial
scien
ce
l
e
arn
i
n
g
ou
t
co
me
s
Ana
v
a
test
r
es
ul
ts
s
h
o
w
F
a
=
54.9
2
9
>
3
.9
3
3
,
it
ca
n
be
s
a
i
d
tha
t
t
her
e
i
s
a
n
e
ffect
o
f
t
h
e
a
p
p
lica
t
ion
of
l
ea
r
n
in
g
m
o
del
s
o
n
le
arni
n
g
o
u
t
c
o
me
s.
T
he
r
e
s
u
lts
o
f
pre
-
tes
t
a
n
d
po
st-t
es
t
in
t
h
e
e
xpe
ri
me
n
t
al
g
roup
a
n
d
the
c
o
n
t
ro
l
gr
oup
c
a
n
b
e
se
e
n
i
n
de
t
a
il
i
n
T
a
b
l
e
2
.
Ba
se
d
o
n
t
h
e
resul
t
s
of
pos
t-tes
t
o
n
so
cial
s
tud
i
es
s
u
b
j
e
c
t
s,
the
a
v
era
g
e
s
t
ude
nt
t
a
u
g
h
t
u
sing
t
he
P
BL
m
odel
w
a
s
h
i
g
h
er
a
n
d
h
i
gher
t
h
a
n
t
he
a
ve
rage
s
t
ude
n
t
w
ho
w
as
tau
g
h
t
us
i
n
g
D
I
(
D
i
re
ct
I
nst
r
u
c
tio
n)
m
ode
l,
a
lt
h
o
u
gh bo
t
h
e
xpe
r
ienc
e
d
a
n
i
n
cr
ease
fr
om t
he
r
esu
l
t
s
of th
e
pre
-
tes
t
.
Ther
efor
e,
i
t
ca
n
be
c
o
n
c
l
u
d
e
d
t
ha
t
t
h
e
resul
t
s
o
f
s
oc
i
a
l
s
ci
en
c
e
o
f
st
u
d
e
nt
s
wh
o
were
t
re
a
t
ed
w
i
t
h
PB
L
mode
l w
e
r
e
b
ett
e
r
tha
n
st
ude
nts
w
ho w
e
re
t
rea
t
e
d
w
it
h t
h
e
DI m
o
del.
A
sign
ific
an
t
i
n
cre
a
se
i
n
s
o
c
i
al
s
cie
n
ce
l
e
a
r
ni
ng
o
u
tc
om
es
o
f
s
tud
e
nt
s
ta
u
ght
u
sing
P
BL
m
od
e
l
show
s
t
h
e
effe
cti
v
e
n
ess
of
t
h
e
m
odel
in
h
e
l
p
i
ng
s
t
u
d
e
n
t
s
l
ea
rn
so
c
i
al
s
ci
e
n
ce
.
Th
e
e
x
is
t
e
n
c
e
of
p
rob
l
ems
un
derl
y
i
n
g
t
he
l
ear
nin
g
p
r
o
c
e
ss
has
s
timu
l
ate
d
s
tude
nts
t
o
t
h
i
n
k
crit
i
cal
ly
a
n
d
an
a
l
ytic
al
ly
t
o
sol
v
e
t
h
ese
Evaluation Warning : The document was created with Spire.PDF for Python.
Int.
J
. Eva
l
. & Re
s
. Educ
.
IS
S
N
:
2252-
88
22
T
h
e
in
f
l
u
en
c
e
of
pro
b
l
em
bas
ed lea
r
n
i
ng t
o
wards s
o
c
i
a
l
sc
i
e
nce
lear
n
i
ng.
.. (B
erti
D
y
ah
Perm
at
asar
i)
43
pro
b
lem
s
t
hro
u
g
h
P
BL
s
te
ps
a
nd
lear
n
i
n
g
i
n
sma
ll
gro
u
p
s
can
f
os
ter
in
ter
act
io
n
betw
ee
n
st
u
d
e
n
ts
s
o
t
h
a
t
l
e
a
r
ni
ng
f
e
e
l
s
g
oo
d
a
n
d
st
ud
en
ts'
l
e
arn
i
ng
i
nt
ere
s
t
ca
n
i
n
cre
a
s
e.
Tab
l
e
2.
D
iffe
renc
es of
pre
-
te
st an
d
post-te
st
r
esult
s
o
f e
xpe
r
im
enta
l c
l
a
ss and
cla
ss c
o
n
t
r
o
ls
Te
s
t
Re
sul
t
C
l
a
ss
n
X
Mo
M
e
M
a
x
Min
St
an
d
a
r
d
Pre
-
te
st
Expe
r
i
m
e
nt
5
0
47.
84
56.
00
48.
00
76.
00
12.
00
13.33
C
ontrol
59
43.
93
44.
00
44.
00
72.
00
12.
00
13.26
Po
s
t
-
t
es
t
Expe
r
i
m
e
nt
5
0
74.
16
68.
00
72.
00
100
24.
00
16.80
C
ontrol
59
51.
80
60.
00
52.
00
80.
00
24.
00
12.77
The
f
i
r
s
t
fi
n
d
ing
s
h
ow
s
t
h
at
P
BL
m
ode
l
i
s
m
or
e
effect
i
v
e
th
a
n
D
I
m
odel.
T
hi
s
f
i
n
d
i
n
g
is
p
ara
lle
l
w
ith
t
he
r
e
s
u
lts
o
f
pre
v
i
o
us
s
tu
d
i
es
t
ha
t
t
h
e
le
ar
ni
n
g
o
ut
c
o
me
s
o
f
st
ude
n
t
s
stu
d
y
i
n
g
w
i
th
P
B
L
m
ode
ls
sign
ifica
n
tl
y
i
n
cre
a
sed
base
d
o
n
p
re
-test
and
p
o
st-
t
e
s
t
and
t
h
e
s
ci
en
t
i
fic
proc
ess
s
k
i
l
l
s
o
f
s
tu
de
nt
s
a
l
s
o
incre
a
se
d
[12].
Similar
r
e
sea
r
ch
s
h
o
w
s
t
h
a
t
t
h
ere
are
statis
ti
c
ally
s
ig
n
i
f
i
c
a
n
t
differ
e
n
ce
s
i
n
t
he
m
e
a
n
post-te
st
ac
hi
e
v
em
en
t
s
c
ore
s
o
n
Tea
c
her
-
Made
T
e
s
t
(TM
T
)
an
d
t
h
e
a
v
era
g
e
p
ost
-
te
st
achi
e
v
e
me
nt
s
co
r
e
s
a
t
Re
sear
cher
-D
esi
gne
d
Tes
t
(
RD
T)
b
etw
e
e
n
s
t
u
de
nts
w
h
o
stud
y
w
i
t
h
P
r
ob
lem
Base
d
Lea
r
ni
ng
(P
BL)
an
d
th
ose
w
h
o
st
udy
w
i
t
h
t
ra
d
iti
o
n
al
m
e
t
ho
ds
o
r
Tradit
iona
l
M
e
th
o
d
(
T
M)
[
2
3
].
A
nother
st
u
dy
is
a
lso
sh
o
w
n
th
a
t
the P
B
L
m
e
th
o
d
is m
o
re
e
ffecti
v
e
tha
n
TT
M
in
stu
d
e
n
ts
' c
onc
ep
t
u
a
l
dev
elo
p
m
ent
[2
4
]
.
Bes
i
de
s
be
in
g
a
b
l
e
t
o
si
gn
i
f
ica
n
t
l
y
im
pro
v
e
l
ear
nin
g
o
u
t
c
o
me
s,
P
B
L
m
odel
i
s
a
lso
be
lie
ved
t
o
impro
v
e
pro
b
l
em
s
ol
vi
n
g
s
ki
ll
s
an
d
deve
l
o
p
cr
itica
l
a
nd
c
r
e
a
ti
v
e
thi
nki
ng
.
Thi
s
i
s
re
i
n
fo
rce
d
b
y
th
e
fin
d
i
n
gs
tha
t
c
la
sse
s
t
ha
t
ar
e
sub
j
ec
t
to
P
BL
m
ode
ls
h
ave
a
h
i
g
h
er
p
r
o
b
l
em-so
l
vin
g
ab
ili
t
y
t
h
a
n
c
l
as
ses
subj
ect
e
d
t
o
con
v
e
n
t
i
ona
l
m
odel
s
[
13].
A
n
o
t
her
re
sear
ch
s
h
o
w
s
t
he
s
a
m
e
thi
n
g
,
n
a
m
e
l
y
a
t
t
h
e
e
n
d
o
f
P
B
L
p
r
o
c
e
s
s
,
stude
n
t
s
ca
n
i
d
e
n
t
i
fy
a
nd
so
l
v
e
pro
b
l
e
m
s
w
i
t
h
t
he
ir
o
w
n
i
deas
a
n
d
a
bi
l
ities
a
n
d
d
e
ve
lo
p
t
h
e
i
r
cr
eati
v
e
th
ink
i
ng
[
14].
P
B
L
m
o
de
l
is
i
nde
e
d
e
ffec
ti
v
e
t
o
im
pro
v
e
l
e
arni
n
g
o
u
t
c
ome
s
a
nd
t
h
e
ab
i
l
i
t
y
t
o
t
h
i
n
k
a
nd
s
ol
ve
pro
b
lem
s
,
b
u
t
i
t
r
e
q
u
i
re
s
ca
reful
pl
a
n
nin
g
a
nd
g
o
o
d
c
om
munica
ti
on
be
tw
een
t
eac
hers
a
nd
st
u
d
en
ts.
This
i
s
bec
a
u
s
e
t
h
e
q
u
al
i
t
y
o
f
c
om
m
u
nic
a
t
ion
be
tw
ee
n
tea
c
h
er
s
an
d
s
t
u
d
e
n
t
s
is
v
ery
imp
o
rta
n
t
,
t
ea
che
r
s
ar
e
con
s
i
d
ere
d
a
s
part
ne
rs,
as
a
ctive
par
tici
p
a
n
t
s
,
dur
i
n
g
tra
i
n
i
ng
a
c
tiv
it
i
e
s
[2
0].
F
u
rther
m
or
e,
t
ea
che
r
s
must
p
a
y
mor
e
a
tt
e
n
tio
n
t
o
f
ee
d
b
ack
r
ec
eive
d
from
stu
den
t
s, to
co
n
t
rol a
n
d
a
d
j
u
s
t
w
e
ll t
o
t
he
t
rai
n
i
ng pr
oc
ess.
3.2.
The
eff
e
c
t
of
learn
i
ng in
t
er
est
on le
a
rnin
g out
c
ome
s
A
n
a
v
a
tes
t
r
esults
s
how
e
d
F
b
=
5
.
4
0
9
>
3
.
0
8
5
s
o
t
h
e
t
e
s
t
d
e
c
i
s
i
o
n
w
a
s
r
e
j
e
c
t
e
d
b
y
H
0B
.This
m
ean
s
tha
t
t
he
re
i
s
a
n
i
n
f
l
u
e
n
ce
o
f
st
ude
n
t
s
'
l
e
a
r
n
i
n
g
i
n
te
rest
o
n
so
c
i
a
l
s
t
u
d
i
e
s
l
e
a
r
n
i
n
g
o
u
t
c
o
m
e
s
.
T
h
i
s
f
i
n
d
i
n
g
i
s
i
n
line
w
i
th
t
he
f
in
di
n
g
s
of
N
urhasa
na
h
&
S
oba
n
d
i
w
hi
c
h
s
h
o
w
s
t
ha
t
i
ntere
s
t
in
l
e
a
rni
n
g
ha
s
a
sig
n
i
f
ica
n
t
in
flue
nce
o
n
l
ea
rni
ng
ou
tc
o
m
es.
S
t
ude
nt
l
ea
rni
ng
ou
tc
o
m
es
c
an
b
e
i
m
prove
d
t
h
rou
gh
i
n
c
r
ea
s
i
n
g
s
t
ude
n
t
lear
n
i
n
g
i
nt
e
r
est,
w
h
i
ch
m
e
a
ns
t
ha
t
t
h
e
be
tter
t
h
e
stu
d
e
n
t
'
s
le
arni
ng
i
n
t
ere
s
t
w
i
l
l
h
a
v
e
a
n
i
m
p
act
on
s
t
ude
nt
l
e
a
r
ni
ng
ou
t
c
o
m
e
s
t
ha
t
a
r
e
ge
tt
in
g
be
t
t
e
r
[
25].
O
t
he
r
fi
nd
ings
a
l
s
o
show
s
imilar
r
e
sul
t
s
tha
t
i
nt
er
est
in
l
ear
nin
g
h
as
a
p
os
iti
v
e
e
ffe
c
t
o
n
lea
r
n
i
n
g
out
co
me
s
in
E
ngl
ish
su
bj
e
c
ts
[
2
6
],
A
r
a
b
i
c
[27],
Econ
om
ics [
28],
[29].
Th
e
d
eci
si
on
o
f
t
h
e
H
0B
t
e
s
t
w
a
s
rejec
t
e
d
a
lso
m
ean
t
th
a
t
t
he
s
oc
i
a
l
sc
i
e
nc
e
l
e
arn
i
n
g
outc
o
m
e
s
o
f
stude
n
t
s
c
o
ul
d
be
d
iffere
n
t
i
n
e
a
c
h
c
a
t
e
gor
y
of
h
i
gh,
m
e
d
i
u
m
a
n
d
lo
w
l
e
arni
ng
i
n
t
e
r
es
t.
T
herefore,
furth
e
r
tes
tin
g
is
n
e
e
d
ed
a
fter
A
na
va
t
o
fi
nd
o
u
t
w
h
a
t
s
tu
de
nts
w
ith
l
e
ar
nin
g
i
nte
r
es
t
ha
ve
b
et
t
e
r
lear
n
i
n
g
o
u
t
c
o
me
s.
T
he
r
esult
s
o
f
th
e
c
a
lc
ula
t
io
n
of
t
he
a
ver
a
g
e
c
o
m
p
ar
at
i
v
e
te
st
b
etwee
n
c
olum
ns
a
re
prese
n
t
e
d i
n
T
a
b
l
e
3.
Ta
b
l
e
3.
S
ummary
o
f
i
n
ter
col
u
m
n
a
vera
g
e
c
ompa
rison
te
st
r
e
s
ul
t
s
No
.
I
n
t
e
r
act
io
n
F
ob
s
F
ta
ble
D
ec
ision
Ma
r
g
ina
l
me
a
n
1
.
.
9.
801
6
.
169
H
0
is
r
e
j
ec
t
e
d
2
10.
259
6
.
169
H
0
is
r
e
j
ec
t
e
d
3
.
.
0.
734
6
.
169
H
0
is ac
c
e
p
te
d
.
.
F
r
o
m
T
a
b
l
e
3
i
t
c
a
n
b
e
s
e
e
n
t
h
a
t
t
h
e
r
e
H
0
i
s
rej
e
c
t
ed
,
a
n
d
i
s
b
a
s
ed
o
n
t
h
e
a
v
erag
e
marg
i
n
al
a
s
prese
n
t
e
d i
n
T
a
b
l
e
3.
It can b
e
conc
l
ude
d th
a
t
t
he resu
lts o
f s
oc
i
a
l scie
nc
e
stu
d
e
n
ts w
it
h a
n
i
n
t
ere
s
t in
l
ear
ning
the
h
i
gh
be
tte
r
t
h
an
t
he
r
e
s
ul
t
s
o
f
s
o
cia
l
s
cie
n
c
e
s
tude
n
t
s
w
i
t
h
a
n
i
ntere
s
t
i
n
l
ear
nin
g
m
ed
ium.
T
he
r
e
s
u
lts
o
f
soc
i
al
s
c
i
e
n
ce
stude
n
t
s w
ith h
igh i
n
t
e
rest i
n le
arni
ng
b
e
tte
r
t
h
a
n t
h
e
re
sul
t
s
o
f
so
c
i
a
l
s
c
i
ence
s
tud
e
nt
s
with
l
o
w
l
e
a
r
n
i
n
g
i
n
t
e
r
e
s
t
,
w
h
i
l
e
t
h
e
r
e
s
u
l
t
s
o
f
s
o
c
i
a
l
s
c
i
e
n
c
e
o
f
s
t
u
d
e
n
t
s
w
ith
l
e
a
r
n
ing
in
ter
e
st
a
r
e
a
s
go
o
d
a
s
soc
i
a
l
scie
nc
e lea
r
n
i
n
g
ou
t
c
o
m
e
s of st
ude
n
t
s
w
ith
l
ow
lea
rni
ng
in
te
res
t.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N: 2
2
5
2
-
88
22
In
t
.
J
.
E
v
a
l
.
&
R
e
s
.
E
d
u
c
.
V
o
l.
8
,
N
o
.
1
,
M
ar
c
h
2
0
1
9
:
3
9
-
4
6
44
3.
3.
In
tera
ctio
n
betw
een
lea
r
n
i
ng
mo
d
els
a
nd
lea
r
n
i
ng
in
t
erest
B
a
se
d
on
the
A
n
ava
tes
t
,
ob
t
a
ine
d
F
ab
=
0
.
3
3
0
<
3
.
0
8
5
s
o
t
h
a
t
t
h
e
H
0AB
t
e
s
t
d
eci
si
on
i
s
a
cc
ept
e
d
.
H
0A
B
i
s
ac
ce
p
t
ed
w
h
i
c
h
m
ea
ns
t
ha
t
t
h
ere
is
n
o
in
tera
ct
i
on
be
twee
n
l
ea
rn
in
g
m
ode
ls
a
n
d
l
ea
rn
in
g
in
t
e
rest
i
n
so
ci
al
s
c
i
en
ce
l
e
a
r
n
i
n
g
out
comes
.
T
h
e
ref
o
re,
t
h
e
c
o
mp
ari
s
o
n
b
e
t
w
e
e
n
t
h
e
u
s
e
o
f
P
B
L
a
n
d
D
I
m
o
d
e
l
s
f
o
r
e
a
c
h
leve
l
of
l
ea
rn
i
ng
inte
res
t
f
ol
lows
t
he
r
a
t
i
o
o
f
the
m
a
rg
i
n
al
m
e
a
n
.
G
r
aph
i
ca
l
l
y,
t
he
a
bse
n
c
e
o
f
i
n
te
r
a
c
t
i
on
be
t
w
ee
n
le
ar
n
i
ng
m
od
e
l
s
an
d
i
n
t
e
r
e
st
i
n
l
e
a
r
ni
n
g
can
b
e
s
e
e
n
i
n
F
ig
ur
e
1.
I
n
the
f
i
g
u
r
e
,
the
pr
o
f
iles
of
P
BL
a
n
d
D
I
m
ode
ls
d
o
no
t
i
n
ter
s
e
c
t
.
F
r
om
t
ha
t
p
r
ofi
l
e,
i
t
ca
n
a
l
so
be
s
e
e
n
t
hat
t
h
e
a
v
era
g
e
f
o
r
st
u
d
en
ts
w
h
o
a
r
e
sub
j
ec
t
e
d
t
o
P
B
L
m
o
d
e
l
i
s
a
l
w
a
ys
h
i
gher
t
h
an
t
he
a
ver
a
ge
f
or
s
t
ude
n
t
s
w
h
o
ar
e
su
b
j
e
c
te
d
to
D
I
m
odel,
b
ot
h
f
o
r
s
t
u
d
e
n
t
s
w
i
t
h
h
i
g
h
,
m
e
d
i
u
m
,
a
n
d
l
o
w
l
e
a
r
n
i
n
g
i
n
t
e
r
e
s
t
.
T
h
i
s
m
eans
tha
t
t
he
s
oc
ia
l
sc
ie
nc
e
lea
r
n
i
n
g
ou
tc
om
es
o
f s
t
ude
n
t
s w
h
o
st
u
d
y
w
ith
P
BL m
ode
ls
a
r
e
a
lw
a
y
s
be
tte
r tha
n
t
he s
oc
ia
l scie
n
c
e
le
arn
i
ng
o
u
t
com
e
s
of
s
t
ude
n
t
s
w
h
o
st
ud
y
w
i
t
h
D
I
m
odel
a
t
a
n
y
l
e
v
e
l
o
f
lear
n
i
n
g
.
Fi
g
u
re
1
.
E
f
f
e
ct
p
r
o
f
i
l
e
o
f
l
e
a
r
n
i
n
g
m
o
d
e
l
v
a
r
i
a
b
l
e
s
4.
CONCLUSION
P
r
obl
e
m
B
a
s
e
d
L
ear
nin
g
(
PBL)
m
ode
l
has
m
u
ch
h
ig
her
eff
e
c
t
i
v
e
n
e
s
s
t
h
a
n
c
onve
nti
o
nal
l
e
ar
n
i
ng
m
odel
s
n
a
m
e
l
y
D
i
r
e
c
t
I
nstr
u
c
ti
o
n
(
D
I
)
in
i
m
p
r
o
v
i
ng
soc
i
al
s
cie
n
c
e
l
e
a
r
n
i
n
g
o
u
tc
om
es.
This
P
BL
m
odel
c
a
n
be
u
se
d
as
a
n
alter
n
a
t
i
v
e
in
l
e
a
r
n
i
n
g
t
h
at
i
s
mor
e
i
n
n
o
v
a
ti
ve,
fu
n,
a
nd
c
ha
llen
g
i
ng
f
o
r
st
u
d
e
n
ts
i
n
soci
a
l
sc
i
e
nce
beca
us
e
stu
d
e
n
ts
c
a
n
b
u
i
l
d
t
he
ir
o
w
n
k
n
o
w
l
e
d
ge
by
so
lvi
n
g
p
r
o
b
l
em
s
t
h
at
a
re
s
imilar
with
t
h
e
r
e
a
l
w
o
r
l
d.
I
n
ad
di
tio
n
t
o
t
h
e
l
ea
r
n
i
ng
mo
de
l
us
ed,
t
h
e
le
ve
l
o
f
i
n
t
ere
s
t
in
l
e
a
rn
in
g
al
so
i
n
f
lu
e
n
ces
t
h
e
s
u
cce
ss
o
f
stu
d
e
nt
s
in
l
e
a
rn
in
g
so
ci
al
s
c
i
e
n
c
e
,
al
th
ough
i
n
t
h
i
s
s
tu
dy
t
h
e
r
e
w
as
n
o
inter
a
c
t
i
o
n
be
t
w
ee
n
lea
r
n
i
ng
mode
ls
a
nd
i
n
ter
e
st
i
n
lear
nin
g
.
S
t
u
d
e
n
t
s
w
ith
h
ig
h
le
ar
ni
ng
i
n
ter
e
s
t
ar
e
mor
e
e
age
r
to
lea
r
n
an
d
g
e
t
be
t
t
er
r
esul
ts
t
ha
n
st
ude
n
t
s
w
i
t
h
m
oder
a
t
e
a
nd
low
lea
r
n
i
n
g
i
nt
e
r
es
ts.
F
u
r
t
h
e
r
m
or
e
,
P
r
oble
m
B
a
s
ed
L
e
a
r
ni
ng
mo
de
l
c
a
n
be
de
ve
l
ope
d
ac
c
o
r
d
in
g
to
t
he
n
e
e
d
s
o
f
s
t
ude
nt
s
or
d
e
v
el
o
p
e
d
i
n
t
h
e
fo
rm
o
f
t
e
achin
g
mat
e
ri
a
l
s
t
o
f
aci
lit
at
e
tea
c
he
r
s
i
n
tea
c
hi
n
g
t
h
i
s
m
ode
l.
REFERE
NC
E
S
[1]
Ministry
o
f
Edu
cat
ion.
Ac
t
o
f
T
h
e
Re
p
u
b
l
ic
o
f
Ind
o
n
e
s
ia
Nu
mb
e
r
2
0
,
Y
e
ar 2
003
on
N
a
tio
n
a
l
Ed
u
c
atio
n
Sy
ste
m
,
Arti
c
l
e 1 Section
2.
J
akart
a
: M
i
n
i
s
t
ry
o
f Ed
ucati
o
n
, 2
003
.
[2]
A.
N
u
r
cahyo
a
nd
Y
.
H
arto
no
.
Basi
c Con
cept
a
n
d D
evelop
men
t
o
f
S
o
ci
a
l
Science-El
ement
a
r
y
Sch
o
o
l
.
M
ag
eta
n
:
Lembag
a E
d
uk
a
s
i Swast
i
k
a
Press,
p
p
. 6
-7
, 73
,
2
0
1
1
.
[3]
M
.
N
.
S
o
emantri
.
S
o
cial
Scien
ces E
d
u
c
a
t
ion Vie
w
ed
fro
m
the
Pers
pect
ive o
f
A
c
tu
aliz
a
t
i
o
n
.
J
akarta:
IK
IP
J
akarta,
p
p
.3, 2
00
1
.
[4]
M
i
nistri
a
l
R
egulati
o
n
N
u
m
b
er
2
2,
Y
ear
2
0
0
6
o
n
C
on
ten
t
S
tand
ard.
J
akarta:
D
i
rect
orate
Gen
e
r
a
l
of
P
rim
a
ry
a
n
d
S
econ
d
ary
Ed
ucati
o
n
M
a
nag
e
m
e
nt.
[5]
S
u
t
o
m
o
,
“
E
ff
ect
o
f
Jig
s
aw
C
o
operat
i
v
e
L
earnin
g
S
trat
egy
and
S
o
c
ial
S
k
ills
o
n
S
o
cial
S
cien
ce
L
e
a
r
nin
g
O
u
t
comes
,
”
Jur
n
a
l
Il
mu Pendi
di
k
a
n
,
vol.
23
, is
s
u
e
.
1
,
p
p.
1
1-1
8
,
201
7.
[6]
A.
S
up
rijo
no
,
Ag
us
.
Coo
p
er
a
tive L
e
ar
ni
ng
: PAIKEM
T
h
eo
ry a
n
d
Ap
p
l
i
c
a
t
io
n
.
Yog
y
ak
arta:
P
u
st
aka
P
e
laj
a
r,
2
0
1
3
,
pp
.
4
7.
[7]
Rus
m
an.
Le
arn
i
ng
M
o
de
ls
. Jak
arta:
P
T
Raj
a Grafi
n
d
o
P
ers
a
da, p
p
.
1
8
7
, 2
0
1
3
.
0
10
20
30
40
50
60
70
80
90
Hi
gh Le
a
r
n
i
ng
In
t
e
r
e
s
t
Me
di
um
Learning
In
t
e
r
e
s
t
Lo
w L
e
ar
nin
g
Inter
e
st
PBL
DI
Evaluation Warning : The document was created with Spire.PDF for Python.
Int.
J
. Eva
l
. & Re
s
. Educ
.
IS
S
N
:
2252-
88
22
T
h
e
in
f
l
u
en
c
e
of
pro
b
l
em
bas
ed lea
r
n
i
ng t
o
wards s
o
c
i
a
l
sc
i
e
nce
lear
n
i
ng.
.. (B
erti
D
y
ah
Perm
at
asar
i)
45
[8]
Drăgh
i
ces
cu
et al
,
A
p
p
l
i
c
atio
n
of
P
rob
l
em
-Bas
ed
L
earnin
g
S
trat
egy
i
n
S
c
i
ence
les
so
ns
-
E
x
a
mp
les
of
G
oo
d
P
r
act
ice,”
Pro
cedi
a
- S
o
ci
a
l
an
d
B
e
ha
vio
r
al S
c
i
e
nces
,
v
ol.
14
9
,
p
p
.
2
9
7
–
3
01,
2014
.
[9]
R. I.
Aren
d
s
.
Le
arnin
g
to
Te
ac
h
.
J
a
kart
a: S
al
e
m
ba
H
am
an
ika,
pp.
1
04
-1
05
,
2
013
.
[10]
S
.
M
. W
ar
doy
o.
Res
e
arch
Based
L
e
ar
ni
ng
.
Jakart
a:
A
k
a
demia Per
m
at
a
,
p
p
. 40
, 20
1
3
.
[11]
M
.
T
.
Amir.
Educat
io
nal In
nova
t
i
on thro
ugh
Prob
lem
Ba
sed
Learn
i
n
g
: H
o
w
Educa
t
o
r
s
Em
power
St
ud
ent
s
i
n
t
h
e
Knowl
e
dg
e Ag
e.
Jak
arta:
P
r
enada
Med
i
a Grou
p,
p
p
. 22
,
2
00
9.
[12]
A.
G
u
r
ses,
C
.
Do
gar,
&
E
.
G
e
yi
k,
“
Teach
in
g
Of
T
he
C
on
ce
p
t
O
f
En
tha
l
p
y
Usin
g
Prob
le
m
Ba
se
d
Le
a
r
ning
App
r
oach
,”
Pro
c
e
d
ia - S
o
c
i
a
l
an
d Be
ha
v
i
ora
l
Sc
i
e
nc
e
s
,
v
ol
.
1
9
7
, pp
.
2
39
0
–
23
94
,
2
015
.
[13]
S
u
ji
on
o,
H
an
doyo,
&
R
u
j
a,
“
Sol
v
i
ng
Geo
g
rap
h
i
c
P
rob
l
ems
throu
g
h
P
r
ob
lem
Ba
s
e
d
Learn
i
ng
,
”
Ju
rn
al
T
e
or
i da
n
Pra
k
si
s P
e
mb
ela
j
ar
an
IPS
, vo
l
. 2
,
issue.
2
, p
p.
14
-
20
, 20
1
7
.
[14]
E
.
E
r
s
o
y
a
n
d
N
.
B
a
s
e
r
,
“
T
h
e
E
f
f
ects
o
f
P
robl
em
-Based
L
earni
ng
M
et
ho
d
in
H
i
gher
E
d
u
cati
on,
”
Pro
cedi
a
- So
cia
l
and
Beh
a
vi
or
a
l
Sci
e
nces
,
vo
l.
1
1
6
,
p
p.
349
4
–
3
4
9
8
,
2
01
4.
[15]
Sla
m
e
t
o.
L
e
ar
n
i
ng
an
d
A
f
f
ecti
n
g
Fact
o
r
s
.
Jakart
a
:
P
T
Rin
e
ka
C
i
p
ta,
p
p.
5
4-7
2
,
2
013.
[16]
M.
S
yah
.
Ed
uca
t
i
o
n
a
l Ps
y
c
h
o
l
ogy.
Ba
nd
un
g
:
P
T
Rem
aj
a
Rosd
akary
a
, pp
.
1
34
, 20
1
0
.
[17]
B. S
.
Bl
o
o
m
, B.
S
.
T
a
xon
o
my
of
Ed
ucatio
n o
b
j
ecti
ve.
Ha
nd
boo
k.
I: Co
gniti
ve do
main
. New
York: Mc
Ka
y. 1
98
1.
[18]
L.W
.
A
nd
erso
n
and
D.
R
.
Krathw
ohl
.
A T
a
xo
n
o
my of
L
e
a
r
n
i
ng,
T
e
ach
i
n
g
and Asses
s
i
n
g
,
a
Revi
si
on
o
f
Bl
o
o
m
T
a
xon
o
m
y
of Educa
tio
nal O
b
j
ecti
ve.
Ne
w
York
:
L
on
gm
a
n
,
20
01
.
[19]
L.
W
.
An
ders
on.
Classroom Assess
ment:
E
nhan
c
i
n
g the
Qua
lity
of Teacher
Decision
M
a
ki
ng
.
Lo
nd
on
:
Law
r
ence
Erlbaum Associa
t
e
s,
2
003
.
[20]
G.
G
orgh
i
u
,
et al
,
“
P
roblem
-Based
L
earnin
g
-
A
n
Eff
i
c
i
ent
Le
arn
i
ng
S
t
r
ategy
i
n
T
he
S
cien
ce
Les
s
o
n
s
Contex
t,”
Pro
cedi
a
- S
o
ci
a
l
an
d
B
e
ha
vior
a
l
Sci
e
nces
,
vol.
1
9
1,
pp.
1
8
6
5
–
187
0,
2
0
15.
[21]
S
.
T
emel,
“T
he
E
ff
ects
of
P
rob
l
em
-Based
L
earnin
g
o
n
P
r
e-S
e
rvi
c
e
T
each
ers’
C
ri
ti
cal
T
hi
nkin
g
D
is
positio
ns
a
nd
Pe
rcepti
ons of P
robl
em-So
l
v
i
ng Abi
lity,”
So
uth
Afric
a
n
J
o
u
r
n
a
l
of Ed
uc
a
tion
,
v
o
l
.
3
4,
i
ss
ue.
1,
p
p
.
1
-2
0,
201
4.
[22]
N. S
. S
ukmad
in
a
t
a.
Edu
c
atio
nal R
e
sear
ch M
e
th
o
d
s
. Band
u
n
g
:
P
T Remaj
a
Ro
s
d
a
kary
a, pp
.
2
59
, 20
1
3
.
[23]
A.
O
.
Fatade,
D.
M
ogari
,
&
A.A
.
A
r
i
gbabu,
“
Eff
ect
o
f
Pro
b
lem-Ba
sed
Lea
r
ni
ng
o
n
S
e
nio
r
S
econ
d
ary
Sch
o
o
l
S
t
ud
ents’ A
c
h
i
evem
ent
s
In F
u
rther M
a
t
h
em
ati
c
s
,
”
A
c
ta D
i
da
cti
c
a Nap
o
cen
si
a
,
vol
.
6
,
i
ss
ue.
3
,
p
p
2
7
-4
3,
2
01
3
.
[24]
A.
K
.
Tas
ogl
u
an
d
M
.
B
akac,
“
T
h
e
Effect
s
of
P
ro
bl
em
B
as
ed
L
earn
ing
and
Traditional
T
eaching
Met
hods
on
S
t
ud
ents’
A
cadem
ic
A
chi
e
vem
e
nt
,
Co
ncep
tual
D
evelo
p
m
e
nt
s
and
S
c
i
entifi
c
P
r
oces
s
S
k
ills
A
cco
rdi
ng
to
T
h
e
ir
Gradu
a
ted
Hi
gh S
c
ho
ol
T
yp
es,
”
Pr
ocedi
a
S
o
c
i
al
an
d B
e
ha
vior
a
l
Sci
e
nce
,
vol
.
2
, p
p.
2
40
9-2
4
1
3
, 2
01
0.
[25]
S
.
N
u
r
hasan
a
h
&
S
o
band
i,
“
L
earni
ng
In
teres
t
a
s
Determi
n
an
t
S
t
u
d
en
t
L
earn
i
ng
O
u
t
c
om
es,”
J
u
rn
al
Pend
idikan
Ma
na
je
me
n
Pe
rk
an
to
ra
n
,
v
o
l
.
1
,
i
s
su
e.
1
.
p
p
. 1
35
-14
2
, 2
01
6.
[26]
P
.
Z
hang
,
Z
.
W
a
n
g
,
a
n
d
O
.
A
d
esop
e,
“
Th
e
Effects
o
f
G
oa
l
Ty
pe
,
L
earn
i
n
g
I
n
t
erest,
a
nd
Tas
k
D
ifficul
t
y
on
Learn
i
ng
En
gl
ish
Wo
rds
,
”
In
ter
natio
na
l J
o
u
r
n
a
l
o
f
L
e
a
r
n
i
ng
,
T
e
ach
i
n
g
an
d Ed
uca
t
i
onal Resea
r
ch
,
v
o
l
.
1
5
,
i
s
s
u
e
.
2
,
p
p
.
3
2
-
4
6
,
2
01
6.
[27]
M.
A
.
Kurniawan,
“
Contrib
u
tions
o
f
T
eacher
P
ro
f
e
ssion
ali
s
m
a
n
d
In
terest
i
n
Learn
i
n
g
t
o
w
ards
A
rabi
c
L
e
arn
i
ng
Ou
t
c
o
m
es,”.
Jurna
l
Penel
i
tia
n dan
Peni
laian Pendidikan
,
2
(
2),
2
5
7
-
27
3,
201
7.
[28]
I.
R
.
Nuraini,
“
T
h
e
Eff
ect
o
f
Fa
th
er's
I
n
c
om
e
an
d
Learni
ng
I
nte
rest
o
n
Eco
nomi
c
L
ea
rn
in
g
Outco
m
es
o
f
S
t
u
d
ents
i
n
X
G
r
ade
S
M
A
N
e
geri
1
P
AKEM
201
5/
20
16
Ac
ad
em
ic
Y
ear,
”
J
u
rn
al
P
e
ndi
d
i
kan
d
an
Eko
n
o
m
i
,
vo
l
.
6
,
i
s
su
e.
1
.
pp
.
28-3
7
,
20
17.
[29]
Z.
A
.
Suk
a
rn
o
&
P.
H
a
r
dinto
,
“
Th
e
In
flu
e
nc
e
of
t
h
e
U
se
o
f
Ga
d
g
e
ts,
L
earni
ng
Interest
s
and
Emot
iona
l
Intel
ligence
on
t
h
e
Learni
ng
O
ut
co
m
e
s
of
X
I
G
r
a
d
e
IIS
S
t
u
d
ents
i
n
Eco
n
o
m
ic
S
ub
j
ects
a
t
S
MAN
1
Kepanj
en,”
Jur
n
a
l
Pendidikan E
k
o
n
omi
,
v
o
l
.
1
1
,
i
ss
ue,
1
.
p
p
.
9-
1
9
, 2
01
8.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N: 2
2
5
2
-
88
22
In
t
.
J
.
E
v
a
l
.
&
R
e
s
.
E
d
u
c
.
V
o
l.
8
,
N
o
.
1
,
M
ar
c
h
2
0
1
9
:
3
9
-
4
6
46
BIOGRAPHI
E
S
OF
AUT
HORS
Berti
Dy
ah
P
erm
a
t
a
sari
g
rad
u
ated
f
ro
m
El
em
en
t
a
ry
S
ch
oo
l
teacher
T
rain
in
g
and
Ed
ucat
ion
at
S
e
belas
M
a
ret
U
n
iv
ersi
ty
i
n
2
0
15.
S
he
h
as
b
een
d
o
i
ng
h
er
m
as
ter
’s
d
egree
at
t
he
s
am
e
dep
a
rtm
e
nt
a
t
Sebel
a
s
M
a
ret
Uni
v
ers
i
t
y
.
H
e
r
research
i
nt
erest
i
s
i
m
p
l
emen
ta
tio
n
of
l
earni
n
g
m
odel
f
o
r
elem
e
n
t
a
ry s
cho
o
l st
u
d
ents
.
P
r
of.
Drs.
G
u
n
arh
a
di
c
om
p
l
eted
h
is
M
A
degree
at
I
nst
i
t
u
t
e
o
f
S
p
eci
a
l
E
du
catio
n,
N
o
r
way,
S
p
ecial
E
d
u
cation
f
o
r
Th
e
P
h
y
i
call
y
H
an
di
capped
Chil
dren
P
rogra
m
and
hi
s
P
h
.
D
d
eg
ree
at
Universiti
Utara
Mal
a
ysi
a
,
Spe
c
i
a
l
E
d
ucat
i
on
Pr
ogram.
He
h
as
b
e
en
w
o
r
ki
ng
a
t
S
e
b
e
l
a
s
M
a
ret
Un
iversit
y
,
Departm
e
nt
o
f
E
d
u
c
ati
onal
S
c
i
e
nce.
H
is
r
esearch
i
n
t
eres
ts
a
re
i
nc
l
u
s
i
v
e
e
d
u
cati
on
policy
and
its
i
mpl
e
mentation,
i
ncl
u
si
ve
e
d
u
cation
s
t
rat
e
gy
,
tr
eatm
e
n
t
s
f
o
r
chil
dren
w
it
h
sp
ecia
l
need
s,
t
reatm
e
nts
f
o
r
s
t
reet
c
hil
d
ren
,
a
nd
d
evel
op
men
t
o
f
susta
in
a
b
l
e
t
r
a
in
in
g
m
o
d
e
l
fo
r
s
p
e
c
i
a
l
s
u
p
e
r
v
is
in
g
t
e
a
c
he
r
.
Dr.
Riy
a
di
c
omplet
ed
h
i
s
M
aster
o
f
S
ci
ence
a
n
d
Do
cto
r
al
D
eg
ree
in
M
at
hemat
i
cs
S
cien
ce
at
Gad
j
ah
M
ada
Uni
v
ersity
.
He
w
o
r
ks
i
n
D
e
partm
e
nt
o
f
Elem
en
tary
S
c
h
o
o
l
T
each
e
r
T
rainin
g
an
d
Ed
ucati
on,
S
ebelas
M
aret
U
niv
e
rs
i
t
y.
H
is
r
esearch
i
nt
erest
s
a
re
c
u
r
ri
cul
u
m
devel
o
p
m
ent,
learni
ng
m
ode
l
and
m
e
d
i
a,
P
J
J
P
ro
gram
i
m
p
le
m
e
nt
ati
o
n
ev
aluati
on
,
and
l
earning
c
reat
iv
it
y
an
d
motivation
impr
ovement
Evaluation Warning : The document was created with Spire.PDF for Python.