In
te
r
n
ation
a
l Jou
rn
al
o
f E
v
al
u
a
t
i
on
a
n
d
R
e
se
arc
h
in
Ed
u
c
ation
(
IJERE
)
V
o
l.7,
N
o.1,
M
a
r
ch
2018,
pp.
72~
7
6
ISSN
: 2252-
88
22,
D
O
I
:
10.11
59
1
/ijer
e
.
v
7
.
i
1
.11
3
4
2
72
Jou
rn
a
l
h
o
me
pa
ge
:
ht
tp:
//i
a
e
score
.
com
/
j
o
u
r
na
l
s
/
i
n
d
e
x
.
p
hp/IJ
ERE
Stude
nt Engage
m
ent and A
ssessment M
odes: a Study
An
i
l
P
a
t
hak
Cen
t
re f
o
r
Comm
u
n
i
catio
n, T
ea
chi
ng
and
Learning
, U
n
i
v
e
rs
i
t
i
Te
k
no
log
i
B
ru
nei,
Bru
nei
Darus
s
a
l
am
Art
i
cl
e In
fo
ABSTRACT
A
r
tic
le hist
o
r
y
:
Re
ce
i
v
e
d
Jan
1
, 2018
Re
vise
d Jan
3
1
, 2018
A
c
c
e
pte
d
F
eb 27,
2
0
1
8
Th
e
aim
o
f
t
his
proj
ect
i
s
to
a
ttem
pt
a
f
acto
r
ia
l
an
a
l
ys
is
o
f
t
h
e
c
on
gru
e
nc
e
am
on
gst
t
h
ree
l
a
yers
o
f
as
sessm
ent
s
o
f
Oral
P
resen
t
at
ion:
Exp
e
r
t
,
Self,
a
nd
Peer
.
P
a
rti
c
ip
an
ts
i
nclu
ded
grad
uate
a
n
d
u
nd
erg
r
adu
a
te
s
tu
den
t
s
of
Asian
back
gro
und
s
t
u
d
y
i
n
g
Research
W
ritin
g
at
a
t
ech
no
lo
gical
u
n
i
v
e
rs
ity.
The
research
i
n
s
t
r
u
m
ent
co
ns
is
t
e
d
of
a
s
et
o
f
ass
e
ss
m
e
n
t
c
heckl
i
s
t
s
w
ith
w
eighted
and
sp
ecified
c
rit
e
ria
f
o
r
E
x
p
e
rt,
S
e
lf
a
nd
Peer
a
s
s
es
sm
ent
of
res
earch
pres
ent
a
t
i
o
n
s.
B
ro
ader
c
rit
e
ri
a
w
e
re
b
ased
o
n
th
e
fo
llowi
ng
f
o
u
r
fa
c
t
o
r
s:
U
se
of
s
poken
m
e
d
i
a,
U
se
o
f
v
i
sua
l
s,
I
n
t
errel
a
ti
on
shi
p
s,
a
nd
Com
m
un
i
cativ
e
Eff
ect
.
P
a
rt
ic
i
p
ant
s
r
eceiv
e
d
trainin
g
a
nd
w
ere
inv
o
l
v
ed
i
n
a
p
ra
cti
c
e
s
e
ss
i
on
on
s
elf
an
d
p
eer
a
sses
s
m
e
nt
b
efo
r
e
engag
i
n
g
i
n
th
e
ass
e
ssm
e
n
t
t
asks
.
Wh
il
e
the
pi
lot
study
is
u
na
ble
t
o
c
on
cl
us
ive
l
y
es
tab
lish
cong
ruen
ce
am
ong
t
h
e
m
o
d
e
s
o
f
a
s
s
ess
m
en
t,
i
t
i
s
e
xp
ec
t
e
d
th
at
r
esults
fro
m
a
w
i
der
d
atab
ase
wou
l
d
in
di
cate
the
nat
u
re
o
f
wel
l
-de
f
i
n
e
d
a
s
s
ess
m
ent
criteria
and
t
h
e
w
a
y
s
t
h
e
y
c
a
n
be eff
ect
iv
ely com
m
unicat
ed
t
o
as
ses
s
o
r
s
.
T
he presen
t
stu
dy
s
u
g
ge
st
s
t
h
at
t
he
diff
erences
i
n
ratin
gs
g
iv
en
by
p
eers
and
ex
pe
rts
are
n
o
t
d
u
e
t
o
a
lack
o
f
capab
il
ity
o
n
th
e
part
o
f
st
ud
en
t
s
,
bu
t
du
e
t
o
l
ack
o
f
o
r
ientat
i
o
n
and
trai
ni
ng.
It
i
s
necess
a
ry
t
o
ev
ol
ve
c
riteri
a
t
h
at
a
re
w
el
l-d
o
cum
e
n
t
ed
t
o
enab
le
s
t
uden
t
s
to
e
valu
ate
t
h
e
i
r
o
w
n
perf
orm
a
n
c
e
as
w
ell
as
t
he
p
erf
o
rm
an
ce
of
t
h
e
i
r
p
e
e
r
s
as eff
ect
iv
e
l
y
as
d
on
e by
exp
erts.
K
eyw
ord
:
Com
m
uni
c
a
t
ion
Oral
p
resen
t
a
t
io
n
Peer
a
ssessm
e
n
t
Co
pyri
gh
t © 2
018 In
stit
u
t
e
of Advanced
En
gi
neeri
n
g
an
d
Scien
ce.
All
rights
res
e
rv
ed.
Corres
pon
d
i
n
g
Au
th
or:
A
n
i
l
P
a
t
ha
k,
Ce
ntre
f
or
C
o
m
munica
tio
n,
Tea
ch
in
g a
nd
Lea
r
ni
ng,
Un
iv
e
r
sit
i
Tekn
o
lo
gi
B
run
e
i
,
Br
u
n
e
i
D
a
r
us
s
a
l
a
m
Em
ail:
ani
l
.
p
a
t
hak
@
ut
b.e
d
u.b
n
1.
I
N
TR
OD
U
C
TI
O
N
Thi
s
p
ap
er
o
utl
i
n
e
s
a
re
sea
r
ch
p
ropo
sa
l
t
o
c
o
ndu
c
t
a
n
inv
e
st
ig
a
tio
n
i
n
t
o
a
s
pec
i
fic
as
pe
ct
o
f
the
dy
nam
i
c
s
o
f
or
al
p
r
e
sen
t
at
i
o
n
s
.
It
a
ims
to
a
tt
e
m
pt
a
f
a
c
t
or
ia
l
a
n
al
ys
i
s
o
f
th
e
co
ngr
uenc
e
a
m
ong
st
t
hree
l
a
y
ers
of
a
ssessment
s
o
f
Or
al
P
resent
at
ion: Expert, S
elf
,
and Peer.
Co
mp
et
en
c
e
r
e
l
at
e
d
t
o
Ora
l
P
re
se
n
t
at
ion
is
u
s
u
a
l
l
y
c
o
n
s
i
d
e
r
ed
t
o
co
ns
ist
of
t
he
v
erb
a
l
,
t
he
n
o
n
-
verba
l
,
a
n
d
the
vis
u
al.
In
p
ro
fess
io
na
l
c
onte
x
t
s
,
h
o
w
e
ve
r,
f
our
o
the
r
c
ompo
ne
nt
s
be
com
e
r
eleva
n
t
fro
m
t
he
del
i
v
ery
aspe
c
t
s
[1].
T
he
y
a
r
e:
1
)
P
r
ofessi
on
ali
s
m
:
T
he
w
ay
a
p
r
e
sen
t
er
m
ana
g
es
t
he
t
e
c
hn
ica
l
c
onte
n
t
of
t
he
prese
n
t
a
ti
on
;
2)
P
rac
t
i
c
al
a
s
p
ec
t
s
:
T
h
e
w
a
y
a
dem
o
nst
r
a
tio
n
is
pla
n
ne
d
a
nd
e
x
e
c
u
t
e
d
;
3)
I
dea
s
:
P
r
esen
ti
ng
t
h
e
idea
s
so
a
s
t
o
h
i
ghl
i
g
h
t
n
o
v
e
lty,
in
ge
nu
i
t
y,
a
nd
cre
a
tiv
it
y
;
4
)
C
ol
la
bora
t
i
v
e
e
f
for
t:
C
o
-o
p
e
ra
t
i
on
a
m
o
n
g
t
eam
me
mbe
r
s a
nd i
n
te
gra
t
i
on o
f
ski
lls a
n
d
abi
l
iti
es
[
2].
The
pro
j
ec
t
ha
s
spe
c
i
fica
lly
d
ef
ine
d
t
he
t
h
r
ee
m
ode
s
of
a
ssessm
e
n
t
und
e
r
d
i
s
cu
ssi
on
a
s
f
ol
lo
ws.
Sel
f-a
sse
ssme
n
t
i
s
d
ef
in
ed
f
ol
lo
wing
K
l
e
nowsk
i
(1
995
)
as
“
t
h
e
e
va
lua
t
i
o
n
or
j
u
dgm
e
n
t
of
‘
the
w
o
r
t
h’
o
f
one’s
p
e
rform
anc
e
a
n
d
t
he
i
de
n
tific
a
tio
n
o
f
one’s
s
tren
g
t
hs
a
n
d
w
e
a
k
n
e
s
s
e
s
w
i
t
h
a
v
i
e
w
t
o
i
m
p
r
o
v
i
n
g
o
n
e
’
s
l
e
a
r
ni
ng
ou
t
c
o
mes
”
[3
]
.
P
e
e
r
a
ssessment
,
i
n
t
h
e
c
o
n
t
ex
t
of
t
h
i
s
projec
t
has
bee
n
d
ef
i
n
e
d
a
s
“
an
a
rr
angem
e
nt
i
n
w
h
i
c
h
in
div
i
dua
ls
c
o
n
s
i
der
the
a
m
ou
n
t
,
le
ve
l,
v
a
l
u
e
,
w
o
rt
h,
q
u
a
l
i
t
y
,
or
s
ucc
e
ss
of
t
he
p
ro
d
u
ct
s
or
o
ut
c
o
me
s
of
lear
n
i
n
g
o
f pe
e
r
s of similar
st
atu
s
[4]
.”
Evaluation Warning : The document was created with Spire.PDF for Python.
IJERE
I
S
S
N
:
2252-
88
22
St
u
d
en
t
E
n
g
a
g
e
m
e
nt
a
nd Ass
e
ssm
e
nt
M
odes
:
a
Stu
d
y (Ani
l
Pa
th
a
k
)
73
S
i
milarly,
E
xpert
a
ssessment
has
been
d
e
f
ined
a
s
assess
m
e
nt
u
su
a
l
l
y
c
a
r
r
i
e
d
o
u
t
b
y
t
h
e
s
u
b
j
e
c
t
teac
her
/lec
t
ur
er
o
r
a
person
i
d
e
n
ti
fi
e
d
a
s
a
n
e
xpe
r
t
i
n
t
h
e
fie
l
d
u
si
ng
p
re
-d
et
ermi
n
e
d
cri
t
eri
a
c
o
m
mu
ni
c
a
t
e
d
to
t
h
e
ass
e
sse
well
b
e
fo
re
the ass
essm
ent [5]
.
2.
E
A
RL
IER RESEARCH
A
st
ud
y
b
y
L
u
c
e
t
al.
foc
u
se
d
on
t
h
e
a
g
ree
m
e
n
t
betw
e
e
n
p
rofe
ssi
o
n
al
a
sse
ssm
ent
an
d
sel
f
-
an
d
pe
er
assessm
ent
of
o
ral
prese
n
t
a
ti
on
s
k
i
l
ls
a
nd
inve
st
i
g
ates
i
nto
st
u
d
en
t
perc
ept
i
on
s
ab
ou
t
peer
a
ssessme
n
t
[6]
.
S
i
nce
the
ma
in
f
oc
us
o
f
t
h
is
s
tu
d
y
i
s
o
n
t
he
i
nt
e
r-r
ater
r
e
l
i
a
b
il
it
y,
it
pre
s
e
n
ts
a
c
om
p
a
ris
o
n
of
t
he
t
e
a
c
h
e
r
a
nd
peer
a
ssessment
scor
es
a
nd
p
oint
s
t
o
w
a
rds
a
posi
t
ive
rela
ti
ons
h
i
p
.
H
o
w
e
v
e
r
,
t
h
e
d
a
t
a
a
l
s
o
i
n
d
i
c
a
t
e
s
e
x
i
s
t
e
n
c
e
of
s
om
e
cr
ucia
l
d
i
ffere
nces.
In
g
e
n
e
r
al,
the
st
ud
y
su
g
g
e
s
ts
t
ha
t
p
e
ers
and
t
e
ac
h
e
rs
i
nt
erp
r
et
t
h
e
c
rit
e
ria
t
h
e
rubric
f
r
o
m
fu
ndam
e
n
t
a
l
ly
d
i
f
fere
n
t
p
er
spe
c
t
i
v
es.
Wi
t
h
r
egard
t
o
t
h
e
co
m
p
ar
i
s
o
n
o
f
se
lf-asse
ssm
e
n
t
s
c
o
res
a
n
d
t
e
a
c
h
e
r
sc
o
r
e
s
,
t
h
e
st
udy
c
o
n
c
l
u
d
e
s
th
a
t
t
h
e
re
a
re
s
ig
ni
fi
c
a
n
t
dif
f
ere
n
ce
s
betwe
e
n
t
he
se
s
core
s,
i
n
d
i
ca
t
i
n
g
an
unde
s
i
r
a
b
l
e
sta
t
e
o
f
a
ffa
i
rs. A
s is t
he
g
ene
r
al e
x
p
ec
ta
ti
on
,
self-a
s
se
ssm
e
nt
scores ar
e
, in m
a
n
y
c
ases,
hi
g
h
er
tha
n
t
he
s
core
s
aw
a
r
ded
by ex
pe
rts.
M
c
G
a
r
r
&
C
l
i
f
f
o
r
d
r
e
p
o
r
t
e
d
o
n
t
h
e
u
s
e
o
f
p
e
e
r
l
e
a
r
n
i
n
g
a
n
d
p
e
e
r
a
s
s
e
s
s
m
e
n
t
w
i
t
h
a
c
o
h
o
r
t
o
f
f
o
u
r
-
yea
r
u
n
d
ergr
ad
uate
phys
i
o
t
h
e
r
a
p
y
stu
d
e
n
ts
[
7]
.
The
st
ud
y
obs
erv
e
d
t
he
s
t
ude
n
t
s’
e
n
g
age
m
e
n
t
in
t
he
p
roce
ss
base
d
o
n
a
s
u
r
vey
a
n
d
f
o
c
u
s
gr
o
up
disc
u
ssion.
I
t
w
a
s
di
sc
o
v
ere
d
t
ha
t
mos
t
o
f
the
st
ude
n
t
s
va
lu
e
d
t
h
e
expe
r
i
ence
o
f
assessi
n
g
t
he
ir
p
eer
s.
H
owever
,
when
a
ske
d
w
he
the
r
it
w
as
a
f
air
m
o
de
o
f
as
s
e
s
s
m
e
nt,
their
responses
were
d
iverse
.
S
i
m
ila
rly,
w
he
n
a
s
ke
d
w
h
e
t
her
P
e
e
r
A
sse
ssmen
t
sh
ould
h
a
v
e
a
s
ig
ni
fi
can
t
weig
ht
i
n
the
over
a
ll
as
sessme
n
t,
t
here
w
as
a
g
ene
r
al
d
isa
g
re
em
ent.
T
he
s
tud
y
hi
gh
l
i
g
h
t
ed
t
he
v
a
l
ue
o
f
stu
d
e
n
t
enga
ge
me
n
t
i
n
peer
l
ear
nin
g
a
nd
s
ugge
s
t
s
tha
t
t
he
e
x
p
eri
e
nce
do
es
a
d
d
p
ed
ago
g
i
cal
v
a
l
u
e
t
o
t
h
e
ov
e
r
al
l
educ
a
t
i
o
na
l
ex
perie
n
c
e
.
P
a
nader
o
&
J
o
n
sso
n
p
l
ac
e
d
t
he
ir
f
oc
us
o
n
t
h
e
use
o
f
r
ub
ri
cs
t
o
ac
hi
ev
e
c
ong
ru
en
ce
betw
ee
n
se
lf
a
nd
p
ee
r
as
sess
m
e
nt
[
8].
The
i
r
w
o
rk
h
i
g
h
lig
ht
s
som
e
im
p
o
r
t
an
t
p
o
i
n
ts
r
ega
r
d
i
ng
r
u
brics
.
W
h
i
l
e
rubrics
are
sc
or
ing
t
o
o
l
s
t
h
at
n
ee
d
t
o
b
e
base
d
o
n
s
pe
c
i
f
i
c
c
r
i
t
e
ria,
t
hey
ca
n
be
u
se
d
for
sum
m
a
t
i
v
e
or
form
ati
v
e
a
s
se
ssm
e
nt
p
urp
o
s
e
s.
I
t
is
g
ener
all
y
s
e
e
n
t
hat
ru
bric
s
he
l
p
i
mp
rove
t
r
a
n
s
pare
ncy
in
t
he
a
sse
s
sm
en
t
proce
dure.
F
orm
a
tiv
e
e
ffe
c
t
s of r
ubrics are m
ore lasti
n
g
t
h
an
t
he s
umm
a
ti
ve effe
c
t
s
. The
stu
d
y
c
o
n
c
l
u
d
e
d t
h
a
t
effec
t
s
of d
i
f
fe
re
n
c
es
i
n
perso
n
al
a
t
t
r
i
b
u
te
s
o
n
the
u
se
of
ru
b
rics
n
ee
d
furt
her
re
se
ar
ch.
While
S
elf-Assessment
a
nd
Peer
A
sse
s
s
ment
i
s
a
well
-
r
e
searched
are
a
,
i
t
s
a
ppl
ica
t
i
on
to
o
ral
prese
n
t
a
t
i
on
s
h
a
s
n
o
t
bee
n
s
e
e
n
t
o
be
i
m
p
l
e
m
e
nt
e
d
i
n
a
ri
go
rous
m
a
n
ner.
S
ec
on
d
l
y,
o
n
l
y
a
han
d
f
u
l
o
f
stud
ie
s
lai
d
e
m
p
has
i
s
on
t
h
e
de
ve
l
o
pm
ent
o
f
c
rit
e
ria,
d
o
c
ume
n
ta
tio
n,
o
r
ie
nta
t
ion,
a
nd
trai
n
i
n
g
.
T
he
p
r
e
sen
t
s
tu
d
y
wishes
t
o
ad
dre
s
s
some
o
f
t
h
ese
iss
u
es
i
n
grea
ter
de
tai
l
.
Th
e
s
i
g
ni
fi
can
c
e
o
f
t
h
e
p
r
e
s
ent
st
u
d
y
is
r
el
a
t
ed
t
o
t
h
e
grea
ter
w
e
igh
t
b
ei
n
g
a
ss
ig
ne
d
to
p
a
r
t
i
c
i
p
a
t
o
ry
e
va
l
u
a
t
i
o
n.
T
he
are
a
o
f
t
h
e
st
udy
is
s
ign
i
fi
ca
nt
a
lso
due
t
o
t
h
e
fa
ct
t
h
a
t
by
i
n
vo
l
v
in
g
st
u
d
en
ts
i
n
a
sses
s
me
nt
,
t
h
e
dy
na
mi
cs
o
f
in
struc
t
i
on
ca
n
b
e
r
em
ar
k
a
bl
y
impr
ove
d.
However,
i
f
the
assessm
ent admini
stered
b
y
st
udent
s
i
s
not
s
ys
t
e
m
ic,
such
a
n
exerc
i
se
is li
ke
ly t
o be
fut
i
l
e.
Th
e
ai
m
o
f
t
he
p
rese
nt
s
t
udy
i
s
t
o
i
nv
est
i
g
a
t
e
t
h
e
c
ong
ru
e
n
ce
(
o
r
l
ac
k
t
h
e
r
eo
f
)
a
mon
g
th
e
th
ree
mode
s
of
a
sse
s
sm
ent
(S
elf,
P
e
e
r,
a
nd
Ex
p
e
rt).
T
he
s
tu
dy
al
so
w
i
shes
t
o
f
o
cus
on
t
he
e
f
f
ects
of
a
s
s
essor
trai
ni
n
g
on a
c
h
i
ev
i
ng s
u
ch
c
o
ngr
uenc
e
am
on
g
the t
h
re
e
mode
s.
3.
METHOD
P
a
rti
c
ipa
n
t
s
i
n
c
lude
d gra
d
uat
e
s
tu
den
t
s o
f
A
sia
n
ba
c
k
g
ro
un
d
st
u
d
y
i
n
g
Res
ea
rch
Writ
ing
at a
tec
h
n
o
l
og
i
cal
u
n
i
v
ersit
y
. Th
e res
e
a
r
ch
i
n
s
t
r
umen
t
c
o
nsi
s
t
e
d o
f
a
s
et
o
f
a
sse
s
sme
n
t che
c
k
lists w
it
h w
e
ig
ht
e
d
a
nd
spe
c
i
f
i
e
d
criter
i
a
for
Exp
e
r
t
,
S
e
lf
a
nd P
e
e
r
a
sse
s
sm
ent
of r
esea
rch
prese
n
t
at
i
ons.
Bro
a
der
criter
i
a
ha
ve
be
e
n ba
sed
on
the
fo
llow
i
n
g
four fact
ors: U
se
of
spo
k
e
n
m
e
d
ia,
U
s
e
of
v
isua
ls
,
Inter
r
ela
t
i
ons
h
i
ps,
and
Com
m
uni
c
a
t
ive
Effec
t
.
P
a
rtic
ip
ant
s
re
ceive
d train
i
ng an
d
w
e
r
e
inv
o
l
ve
d i
n
a
p
r
actice session on se
l
f
and
p
eer
asses
s
m
e
nt
befor
e
e
nga
gi
n
g
in t
h
e
assess
me
nt tas
ks.
Table
1 de
sc
ribe
s
the
criterias
u
s
e
d for teache
r (exper
t
)
assessm
ent.
T
a
b
l
e 1.
Criteri
a
used for Te
a
c
h
e
r
(
Expert)
Assessme
n
t
C
r
it
e
r
ion
W
e
i
ght (
%)
Pla
nning a
nd
O
r
ga
nisa
tion
-
P
l
a
nni
n
g
-
S
e
qu
e
n
c
i
ng
-
O
r
ga
nisa
tion
40
De
live
r
y
-
U
s
e
of
body
l
a
n
g
u
a
g
e
-
Ora
l
d
e
liv
e
r
y
40
O
v
e
r
al
l
I
m
p
act
-
A
c
h
i
e
v
e
m
e
n
t
of
purpose
-
I
m
p
act
o
n
a
u
d
i
en
ce
20
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
IJERE
V
ol
.
7,
N
o.
1,
Ma
r
ch 201
8
: 72
– 76
74
4.
RESULT
The
fol
l
ow
in
g sam
p
le da
t
a an
aly
s
is is base
d
on
t
he re
s
u
l
ts o
f a
P
il
ot
S
t
u
dy
c
a
r
ri
e
d
o
ut
o
n
3
0
s
ubj
e
c
t
s
(Ta
b
le
2).
The
data
w
a
s
subje
c
ted t
o
a
P
aired t-tes
t
(
A
l
pha
=0.
0
5).
Ta
ble
2.
C
om
p
a
r
i
so
n
of S
elf-
A
sse
ssm
e
nt w
i
t
h P
e
e
r
A
ssessm
e
nt
Se
lf
Pee
r
(A
VG)
Me
a
n
69.6
71.9
V
a
ri
a
n
ce
35.69
13.54
Obse
r
v
a
tions
3
0
3
0
Pe
a
r
son
C
o
rr
e
l
a
t
io
n
0.
4
0
H
ypot
he
si
z
e
d
Me
a
n
D
i
ff
e
r
e
n
ce
0
Df
2
9
t S
t
a
t
2
.24
P(T<
=
t
)
on
e
-
ta
il
0
.
016
t
C
r
i
t
i
c
a
l
one
-
t
a
i
l
1.
69
P
(
T
<
=
t
) two-
ta
il
0
.032
t C
r
it
i
c
a
l
two-
t
a
il
2.045
Ta
b
l
e
2
s
h
ow
s
tha
t
t
he
c
orre
la
tio
n
be
tw
e
e
n
S
e
lf
a
n
d
P
e
e
r
is
0
.
4
0
i
n
d
i
ca
te
s
a
w
e
ak
linea
r
relat
i
on
sh
ip
am
ong
the
tw
o
mode
s
o
f
a
s
s
essm
ent.
T
h
i
s
m
eans
that
t
h
e
re
i
s
a
g
e
n
era
l
l
ac
k
of
a
gr
eem
ent
or
c
on
gr
uenc
e
am
ong
the
tw
o
sets
o
f
da
ta
i
n
d
ica
t
in
g
t
h
a
t
t
he
w
a
y
s
tude
n
t
s
sc
o
r
e
t
h
e
m
s
e
l
v
e
s
i
s
s
i
g
n
i
f
i
c
a
n
t
l
y
d
i
f
f
e
r
e
n
t
f
r
o
m
the wa
y t
h
eir
p
eer
s score
them
.
Ta
b
l
e
3
sh
ow
s
a
P
e
a
r
son
C
o
rr
elat
ion
Coe
ffi
cien
t
o
f
0.
6
3
s
ugge
s
tin
g
a
s
tro
ng
pos
i
tive
relat
i
o
n
s
h
i
p
betw
ee
n
S
e
lf-
A
ssessme
n
t
a
nd
Ex
per
t
A
ssessm
ent.
T
hi
s
f
i
nd
in
g
is
s
i
gnifi
cantly
different
f
r
o
m
the
findings
repor
t
e
d in a
n
um
ber
of e
arlie
r
re
sear
ch stud
i
es.
Ta
b
l
e
3.
Comp
a
riso
n
of S
elf-
A
s
se
ssm
e
nt
w
ith
T
eac
her
(Exper
t
)
A
s
sessm
e
n
t
Se
lf
Ex
pe
rt
Mea
n
69.
6
60.
63
V
a
r
i
a
n
ce
35.
69
14.
72
O
b
s
e
rva
tions
3
0
30
P
e
a
r
son
C
o
rre
l
a
tio
n
0.
63
H
y
pothe
size
d
Mea
n
D
iffe
re
n
c
e
0
Df
2
9
t
St
a
t
10.
59
P(T
<
=
t
)
on
e
-
t
a
il
8
.
7
2
t C
r
iti
c
a
l
one
-t
a
il
1.69
P(T<
=
t
)
tw
o-t
a
il
1.
74
t C
r
iti
c
a
l
two-ta
i
l
2.045
I
n
t
er
ms
o
f
t
h
e
re
l
a
t
i
on
sh
i
p
b
etwe
en
P
ee
r
an
d
Ex
pe
rt
a
s
s
e
ssm
ent
,
th
e
si
tu
at
ion
i
s
s
lig
h
t
l
y
b
e
t
t
e
r.
Th
e
Pea
r
so
n
Co
rrela
t
i
o
n
Co
e
f
f
i
ci
en
t
h
e
re
(
Tab
l
e
4
)
i
s
0
.
32
s
ugge
sti
n
g
a
v
ery
w
e
ak
l
i
n
ea
r
re
l
a
tio
ns
hi
p.
I
t
ca
n
be
sai
d
t
ha
t
s
t
ude
nts
nee
d
f
ur
the
r
t
rain
i
ng
r
e
gar
d
i
n
g
th
e
use
o
f
c
rit
e
ria
a
n
d
ru
bric
w
he
n
i
t
c
o
m
e
s
t
o
ass
e
ss
i
n
g
t
h
e
perform
ance of
their peers.
Tab
l
e
4.
C
om
p
a
riso
n of P
eer
A
ssessm
ent w
i
th Tea
c
h
e
r
(Expe
rt)
A
s
sess
m
e
nt
Pee
r
(A
VG)
Ex
pe
rt
Mea
n
71.9
60.
63
Va
r
i
a
n
ce
13.
54
14.
72
O
b
se
rva
tions
3
0
30
P
e
a
r
son
C
o
rre
l
a
tio
n
0.
32
H
y
pothe
size
d Mea
n
D
iffe
re
n
c
e
0
Df
2
9
t
St
a
t
14.
06
P(T
<
=
t
)
on
e
-
t
a
il
8.
65
t C
r
iti
c
a
l
one
-t
a
il
1.70
P
(
T<
=t)
t
w
o
-
t
a
il
1
.7
3
t C
r
iti
c
a
l
t
wo-ta
i
l
2.045
Evaluation Warning : The document was created with Spire.PDF for Python.
IJERE
I
S
S
N
:
2252-
88
22
St
u
d
en
t
E
n
g
a
g
e
m
e
nt
a
nd Ass
e
ssm
e
nt
M
odes
:
a
Stu
d
y (Ani
l
Pa
th
a
k
)
75
Le
t
us
n
ow
c
ons
ide
r
o
ther
a
spe
c
t
s
o
f
s
ta
tis
ti
c
s
a
s
seen
i
n
t
h
e
ta
b
l
e
s
a
bo
ve.
T
i
s
t
he
c
alcu
la
te
d
di
ffe
re
nce
repr
e
s
en
ted
i
n
u
ni
t
s
o
f
s
t
an
dar
d
e
rror.
T
he
l
arg
e
r
i
s
the
va
l
u
e
of
T
(
i
t
c
a
n
b
e
ei
ther
p
osit
i
v
e
or
nega
t
i
ve)
,
t
he
s
tro
nger
w
o
ul
d
be
t
he
s
up
p
o
rt
a
gai
n
st t
he
n
ul
l
h
y
p
o
t
h
es
is
t
ha
t
the
r
e
is n
o
si
gn
i
f
ica
n
t
d
i
f
f
er
ence
.
A
value
of
T
c
lo
se
r
to
0
i
n
d
ic
ates
l
a
c
k
o
f
sig
n
i
f
ica
n
t
d
i
ffe
r
e
n
c
e
.
I
n
t
he
a
na
lysis
presen
ted
in
t
he
t
a
b
le
s
abo
v
e
,
for
Se
lf
a
n
d
P
ee
r
the
T
is
2
.2
4
.
I
f
this
T
v
a
l
u
e
i
s
c
o
mpa
r
ed to
t
he
v
a
l
ue
o
f
T
in
o
t
h
er
2
s
et
s
(14.
06
a
nd
1
0
.
5
9)
,
it
i
s
m
u
ch
l
ow
e
r
a
nd
clo
s
er
t
o
0.
T
hi
s
m
e
a
n
s
tha
t
t
he
re
i
s
a
la
c
k
o
f
s
i
gn
ifica
n
t
di
ffere
n
c
e
i
n
t
h
e
tw
o
sets
o
f
data
(
S
e
lf
a
n
d
P
eer).
H
ow
eve
r
,
there
is
a
n
o
t
he
r
i
n
tere
s
t
i
n
g
fac
t
to
o
bserv
e
h
er
e.
I
n
t
h
e
c
a
se
o
f
t
h
e
da
t
a
se
t
s
rela
t
e
d
t
o
S
e
l
f
and
P
e
er
(
Ta
ble
2),
since
the
p-va
lue
is
v
ery
l
ow
(
<
a
l
pha
l
e
v
e
l
o
f
0.
05),
w
e
n
e
e
d
t
o
r
ej
ect
t
he
nu
ll
h
y
p
o
t
he
s
i
s
and
co
nc
l
u
de
t
ha
t
t
h
ere
is
a
s
t
a
t
i
s
t
ical
l
y
s
ig
n
ifica
n
t
d
i
ffe
r
en
c
e
i
n
t
h
ese
tw
o
sets
o
f
da
t
a
.
Thus
,
the da
t
a
(
and,
possi
b
l
y,
i
t
s
in
t
erpr
etat
i
o
n)
is a
noma
l
o
u
s a
n
d
n
ee
ds fur
the
r
i
nve
st
i
g
at
ion.
5.
DISC
USSION
C
ons
ide
r
i
ng
t
h
e
anom
al
ies
fa
ce
d
i
n
t
he
d
at
a
anal
ysi
s
o
f
t
h
e
p
i
l
o
t
s
tud
y
,
a
m
o
re
s
yste
ma
t
i
c
da
ta
col
l
ec
t
i
o
n
ove
r
lar
g
er
m
u
l
t
i
pl
e
sam
p
les
is
r
equ
i
red.
I
t
i
s
e
xpe
c
t
e
d
t
ha
t
re
su
lt
s
o
f
s
uc
h
s
t
ud
y
w
o
u
l
d
i
n
di
c
a
t
e
w
h
at
w
e
l
l
-
defi
ned
assessme
n
t
cri
t
er
i
a
s
h
o
u
ld
i
nc
l
u
de
a
nd
h
ow
i
t
c
a
n
b
e
ef
fe
cti
v
el
y
co
mmun
i
cat
e
d
t
o
assessors.
By
i
m
p
li
c
a
t
ion,
s
uc
h
c
r
iteria
m
a
y
enab
le
s
tu
de
nt
s
to
e
va
l
u
a
t
e
t
h
eir
ow
n
perf
orm
a
nce
a
s
w
e
l
l
a
s
t
he
perform
ance
of
their pe
ers
as e
ffe
c
t
i
v
e
l
y as
done b
y
ex
per
t
s.
V
a
lidi
t
y
i
n
se
lf-a
ssessme
n
t
u
s
ual
l
y
im
p
lie
s
c
ongr
ue
nce
w
i
t
h
t
he
s
c
o
res
aw
a
r
ded
b
y
t
he
e
xper
t
o
r
t
h
e
teac
her
or
a
n
a
v
e
r
age
o
f
s
cor
e
s
aw
a
r
ded
b
y
t
he
p
e
e
r
gro
u
p
.
The
as
su
mpt
i
o
n
h
e
re
i
s
tha
t
t
h
e
t
ea
ch
er
h
as
a
n
expe
r
t
i
s
e
t
h
a
t
p
ro
vide
s
an
a
b
s
o
l
u
t
e
sta
nda
r
d
,
a
n
d
(w
ith
p
eer
r
a
nki
n
g
s),
t
h
e
me
an
o
f
mult
iple
r
ank
i
n
g
s
or
score
s
is like
l
y
to be
mor
e
reli
a
ble
tha
n
t
he
s
c
o
res
aw
a
r
ded by
a
s
i
n
gl
e
a
s
se
s
s
o
r
.
I
t
i
s
usua
ll
y
se
en
t
ha
t
se
l
f-as
s
essm
ent
sc
ore
s
h
ave
a
h
i
g
h
e
r
c
o
n
g
r
u
e
n
c
e
w
i
t
h
p
e
e
r
j
udgmen
t
s
t
h
a
n
with
s
core
s
a
w
arde
d
b
y
t
he
t
ea
cher
o
r
t
h
e
expert.
An
e
x
p
la
na
ti
o
n
of
t
hi
s
ph
en
o
m
e
non
m
i
g
h
t
b
e
th
at
p
ee
rs
in
t
e
rpre
t
a
sse
ssm
ent
cri
t
er
i
a
i
n
a
s
i
gn
ifica
n
tl
y
di
ffere
n
t
m
a
nn
er
[
9].
I
t
i
s
also
p
o
ssible
that
p
eers
f
oc
us
on
fe
at
ures
o
f t
h
e
O
r
al
P
re
sentat
i
on w
h
ich
mig
h
t
be
supe
rfic
i
a
l
[1
0],
[
1
1
].
A
numbe
r
of
s
tud
i
es
h
a
v
e
fo
cuse
d
o
n
t
he
e
ffec
t
s
o
f
trai
ni
n
g
s
tu
de
nts
t
o
w
a
rds
s
e
lf
a
n
d
p
ee
r
assessm
ent.
T
he
o
rientation
and
t
r
aining
u
sua
l
ly
c
onsis
t
s
of
s
y
st
ema
t
i
c
e
xp
l
a
n
a
t
i
on
o
f
ea
ch
o
f
the
com
p
o
n
e
n
t
s
t
h
a
t
de
fi
ne
t
he
a
ssessm
ent.
A
t
the
firs
t
sta
g
e,
i
t
m
igh
t
b
e
w
o
r
t
hw
h
i
le
t
o
inv
o
l
v
e
a
s
se
s
s
or
s
i
n
defi
ni
n
g
a
n
d
e
la
bora
t
i
n
g
o
n
t
he
c
om
p
one
n
t
s
of
t
he
a
sse
ssm
ent
c
r
iter
i
a.
T
h
i
s
sta
g
e
w
o
ul
d
esse
nt
ia
l
l
y
i
n
v
o
l
v
e
deve
l
opm
en
t o
f
assessm
en
t fram
e
w
o
r
k
an
d
d
isc
u
ss
i
o
n o
n
t
he
w
e
i
g
h
t
a
ssi
gne
d
t
o
ea
c
h
of
t
h
e c
o
m
p
o
n
e
n
t in
t
he
f
r
amewo
r
k
[
5
].
I
t
w
ou
ld a
ls
o b
e
usef
u
l
t
o
ha
v
e
a disc
ussi
on
w
i
t
h
the
s
tu
de
n
t
s
re
gard
in
g th
e app
l
i
c
at
i
on o
f
the
c
ri
t
e
ri
a
to
t
he
a
c
t
ua
l
p
e
rfor
ma
nce.
F
or
s
elf-a
s
se
ssm
ent,
p
erha
p
s
i
t
w
o
u
l
d
be
u
se
fu
l
t
o
a
s
k
s
t
ude
n
t
s
to
c
onsi
d
er
u
si
n
g
vi
de-re
corde
d
pre
sen
t
at
ion
s
.
Mo
de
l a
p
p
lic
at
i
o
ns of t
h
e
cr
it
e
r
ia
a
n
d
t
he ru
b
ric ca
n
be
pre
se
n
t
ed
a
s
e
x
am
p
l
e
s
o
f
perform
ance
a
ssessme
n
t
.
I
t
is
a
lso
w
o
rt
h
w
hi
le
t
o
pro
v
ide
fee
dba
ck
o
n
s
e
lf
-as
s
es
sm
ents
a
nd
peer
a
ss
essment
s
con
d
u
cte
d
b
y
the
st
ude
n
t
s.
T
hi
s
w
oul
d
nec
e
ssitate
e
n
g
a
g
i
ng
s
t
u
d
ent
s
i
n
ev
id
en
c
e
-b
a
s
e
d
d
isc
u
ssi
o
n
s
u
si
n
g
t
h
e
assessm
ent
dat
a
t
ha
t
part
ic
ula
r
l
y
r
epre
se
n
t
s
c
o
n
g
rue
n
ce
(
or
l
ac
k
the
r
eo
f)
am
ong
t
h
e
t
h
r
e
e
differe
n
t
m
o
d
es
o
f
assessm
ent [12]
.
Ea
rlier
r
e
sea
r
ch
s
h
o
w
s
t
ha
t
in
tera
ct
ive
me
th
ods
o
f
teac
hin
g
h
a
v
e
a
posi
t
i
ve
e
ffec
t
on
st
ude
n
t
e
n
g
a
g
e
m
e
n
t
[
1
3
]
.
A
s
i
g
n
i
f
i
c
a
n
t
a
n
d
d
e
s
i
r
a
b
l
e
f
e
a
t
u
r
e
o
f
a
n
y
a
s
s
essment
is
t
hat
the
w
o
rth
of
a
n
assess
m
e
nt
i
s
deter
m
i
n
ed
by
its
e
ffec
t
s
o
n
t
h
e
o
v
era
ll
lea
r
nin
g
e
xpe
rie
n
c
e
.
T
his
f
e
atur
e
of
a
ssessm
ent
is
u
sua
l
ly
r
efe
rre
d
t
o
as Bene
f
ic
ia
l
Bac
k
w
a
sh.
6.
CONCL
U
S
ION
To
c
o
n
c
l
u
d
e
,
t
he
d
a
t
a
r
e
late
d
to
t
v
a
l
ue
i
s
anom
al
o
u
s
t
o
our
i
n
ter
p
re
ta
tions
b
ase
d
on
p
va
lue
a
n
d
Pe
arson
corr
elati
o
n
coe
f
fic
i
e
n
t
.
F
urther
i
nv
est
i
ga
tio
n
i
s
ne
e
de
d
to
e
xpre
ss
this
a
n
o
m
a
l
y
.
Wh
il
e
the
pi
lo
t
stud
y
is
u
na
ble
to
c
onc
l
u
s
i
v
e
l
y
e
st
a
b
lis
h
co
n
g
rue
n
c
e
a
m
o
n
g
t
he
m
odes
o
f
a
sse
ssmen
t
,
i
t
i
s
e
x
p
ect
ed
t
h
a
t
re
su
lt
s
from
a
w
i
d
e
r
d
ataba
s
e
w
o
ul
d
in
d
i
ca
te
w
ha
t
w
e
l
l
-de
f
i
n
e
d
a
ssessme
nt
c
r
ite
r
i
a
sh
ou
l
d
i
nc
l
ude
a
n
d
h
ow
i
t
can
b
e
effec
t
i
v
e
l
y
co
m
m
unica
ted
to
a
ssess
ors.
B
y
im
p
l
i
c
a
t
ion,
s
uc
h
c
r
i
te
ria
m
a
y
e
n
ab
le
s
t
u
d
e
nt
s
t
o
e
va
lua
t
e
t
h
ei
r
ow
n pe
rform
anc
e
a
s w
e
l
l
a
s the
pe
rform
ance of
their peers
a
s
ef
fe
ct
i
v
e
l
y as
don
e
b
y
e
x
p
er
t
s
.
REFE
RENCES
[1]
A.
P
athak,
“
Teach
ing
an
d
Ass
es
s
i
n
g
M
u
l
tim
ed
i
a
-bas
ed
O
ral
P
r
es
ent
a
tio
ns
,”
B
u
sin
e
s
s
Co
m
m
unica
ti
on
Qu
arterl
y
,
vo
l/issue:
6
4(4),
p
p
.
6
3
-
71
,
2
001
.
[2]
A.
P
a
t
h
a
k
and
M
LeV
a
san
,
“
Devel
opi
ng
O
ral
P
r
esen
tati
on
Com
p
eten
c
e
in
P
r
o
fe
ss
io
n
a
l
C
o
n
t
e
x
ts
:
A
De
s
i
g
n
-
B
a
se
d
Col
l
abo
r
ati
v
e
A
p
proach
,”
In
ter
n
a
tio
nal Jou
r
nal
of
E
v
a
l
ua
tio
n
and Res
e
arch
i
n
Ed
ucat
io
n
(
I
JERE)
,
vo
l/issue:
4
(4)
,
pp
.
1
79-1
8
4
,
2
0
1
5.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
IJERE
V
ol
.
7,
N
o.
1,
Ma
r
ch 201
8
: 72
– 76
76
[3]
V.
K
l
e
now
ski,
“S
t
ud
ent
self
-evaluati
o
n
pro
cess
e
s i
n
stu
den
t
-cen
t
r
ed t
eachi
ng and
learn
i
ng
co
n
tex
t
s of
Au
s
tral
ia an
d
En
gl
a
n
d,
”
Assessment i
n
Education
,
v
o
l
/i
ss
ue:
2(2
)
,
p
p
.
1
45
-1
63
, 1
99
5.
[4]
K.
T
op
pi
ng,
“
P
e
er
A
ss
essmen
t
betw
ee
n
S
t
u
d
ents
i
n
Coll
eges
a
n
d
U
nive
rs
it
ie
s
,
”
Rev
i
ew of
Edu
c
atio
nal Res
e
arch
,
vo
l/issue:
6
8(3),
p
p
.
2
4
9
-2
76
,
1998
.
[5]
A.
P
at
hak,
“
P
r
esentat
i
o
n
M
edi
a
:
Trai
ni
ng
t
h
e
Tech
no
log
i
s
t
s
,
”
T
e
achi
ng
and
L
e
arnin
g
,
vo
l/
issue
:
2
1
(
1
)
,
pp
.
45
-5
2,
20
00
.
[6]
L.
D
e
Grez,
et a
l
.,
“
H
o
w
eff
ecti
v
e
are
sel
f
-
an
d
peer
a
s
s
es
sm
ent
o
f
o
ral
p
r
esent
ation
s
k
ills
co
m
p
ared
w
i
t
h
t
e
ach
ers
’
ass
e
ssm
ents
?
”
Act
i
ve L
e
a
r
n
i
ng
in
H
i
gh
er Edu
c
atio
n,
vo
l
. 13
, pp
.
1
29
-14
2
,
2
01
2.
[7]
O.
M
c
G
a
rr
a
n
d
A.
M
.
Cl
iffo
rd
,
“Jus
t
En
ou
gh
t
o
Ma
k
e
Y
o
u
T
a
k
e
it
S
e
ri
ou
sly,
”
Exp
l
o
r
i
ng s
t
uden
t
s
’
Attit
ud
es
T
o
wa
rds Peer
asses
s
m
e
nt Hi
gher E
duca
t
i
on,
vol.
65
,
p
p.
6
77
,
2
013.
[8]
E.
P
an
adero
an
d
A.
J
on
ss
on,
“
Th
e
u
s
e
o
f
s
cori
n
g
r
u
b
ri
cs
f
or
f
o
r
m
a
ti
ve
a
ss
essmen
t
p
u
rpo
s
es
r
e
v
is
it
ed:
A
revi
ew,”
Ed
uca
t
io
na
l R
e
s
e
a
r
ch
Review
,
v
o
l.
9
,
p
p
.
12
9-144,
2
0
13.
[9]
A.
P
a
t
h
a
k
and
N
.
S
.
Ch
audh
ari,
“
M
e
di
um
i
s
t
h
e
M
e
ssag
e
:
W
e
b
-
b
a
se
d
L
ectu
r
e
P
r
esen
tatio
ns
i
n
D
i
stan
ce
E
d
u
c
at
io
n
in
S
ingapore,”
Asi
an
Jou
r
na
l of
Di
st
ance Edu
c
a
tio
n
,
vo
l/
issu
e
:
3
(
2
), p
p.
13
-
2
2
,
20
05
.
[10]
A.
P
athak
an
d
S
.
K
ath
p
alia,
“
D
is
cou
r
se
s
trateg
ie
s
f
o
r
e
n
su
rin
g
inv
o
l
v
em
ent
in
e
lect
ron
i
c
l
ectu
r
es
,”
International
jo
ur
nal
of W
e
b B
a
s
e
d
Commu
niti
e
s
, vo
l/issue:
1
(
3
), pp.
330-
3
45,
2
004.
[11]
A.
P
ath
a
k,
“
A
n
aly
s
i
s
o
f
a
Conv
e
r
sat
i
on
al
F
lo
or:
Im
p
l
i
catio
n
s
o
f
T
eachi
n
g
an
d
A
s
s
es
sin
g
,
”
O
n
l
i
n
e
D
i
scussion EL
T
Quar
ter
l
y
,
vo
l/
i
s
sue: 12
(
3)
, pp
.
1
-1
0, 20
1
1
.
[12]
A.
P
at
hak
a
n
d
S
.
B
.
Bee,
“
Ident
i
f
y
ing
Learn
i
ng
N
eed
s:
U
s
i
n
g
N
ar
rati
ve
A
naly
sis,
”
H
u
ma
n
i
si
ng
La
ng
ua
ge
T
e
ach
i
ng
, vo
l/is
sue:
13(4)
, 2010.
[13]
A.
P
ath
a
k
and
F.
C
aval
laro,
“T
ea
ch
in
g
Conflict
Res
o
l
u
tion:
A
S
t
udy
o
f
Two
In
t
e
racti
v
e
M
e
t
h
o
d
s,”
The Asian ESP
J
o
urna
l
, v
ol
/issu
e: 2(
2
),
pp
.
5
-1
1,
20
0
6
.
Evaluation Warning : The document was created with Spire.PDF for Python.