In
te
r
n
ation
a
l Jou
rn
al
o
f E
v
al
u
a
t
i
on
a
n
d
R
e
se
arc
h
in
Ed
u
c
ation
(
IJERE
)
V
o
l.7,
N
o.1,
M
a
r
ch
2018,
pp.
32~
3
8
ISSN
: 2252-
88
22,
D
O
I
:
10.11
59
1
/ijer
e
.
v
7
.
i
1
.11
4
8
8
32
Jou
rn
a
l
h
o
me
pa
ge
:
ht
tp:
//i
a
e
score
.
com
/
j
o
u
r
na
l
s
/
i
n
d
e
x
.
p
hp/IJ
ERE
Effects of Test Level Discri
mination
and Difficulty on
Answer-
Copy
i
n
g Indices
O
n
der
S
unb
u
l
, Se
h
a
Yo
rmaz
D
e
partm
e
n
t
o
f M
easu
r
emen
t an
d
Eval
ua
t
i
on i
n
E
du
catio
n
,
M
ersin
U
niv
e
rsi
t
y,
Tu
r
key
Art
i
cl
e In
fo
ABSTRACT
A
r
tic
le hist
o
r
y
:
R
e
c
e
i
v
e
d
No
v
1
9
,
2
017
Re
vise
d Jan
2
7
, 2018
A
c
c
e
pte
d
F
eb 14,
2
0
1
8
In
t
his
s
t
ud
y
Typ
e
I
E
rror
an
d
t
h
e
pow
er
r
ates
o
f
om
eg
a
(ω)
and
G
B
T
(gen
eralized
b
in
om
ia
l
t
e
s
t
)
in
d
i
ces
w
ere
i
n
v
e
sti
g
at
ed
f
o
r
s
ever
al
n
o
m
ina
l
alp
h
a
lev
e
ls
a
n
d
f
o
r
4
0
and
80
-ite
m
t
e
st
l
ength
s
w
ith
1
0,
000-ex
am
in
ee
sam
p
le
s
ize under several t
e
st l
e
v
el res
tri
c
tio
n
s.
A
s a resu
l
t
,
T
yp
e
I err
o
r
rates
of
b
o
t
h
ind
i
ces
w
ere
fo
un
d
to
b
e
bel
o
w
t
h
e
ac
ce
ptab
le
n
omin
al
a
lp
ha
l
ev
els
.
Th
e
po
wer
stu
d
y
sh
ow
ed
t
hat
averag
e
tes
t
d
ifficul
t
y
was
very
e
f
f
ecti
v
e
f
o
r
po
wer
(tru
e
dete
ct
io
n)
r
a
t
es
o
f
i
n
di
ces.
Cl
ear
p
atterns
were
o
b
served
f
or
t
he
in
crease
o
f
t
e
s
t
diffi
culty
i
n
f
a
v
o
r
o
f
b
oth
ω
an
d
G
B
T
pow
er
r
a
t
e
.
Contrary
t
o
exp
ectati
o
n
s
;
average
tes
t
d
i
s
crimi
n
a
t
io
n
was
n
o
t
as
e
f
f
ective
a
s
a
ve
ra
ge
t
e
s
t
diff
icu
l
ty.
Th
e
res
u
l
t
s
o
f
t
he
i
nteracti
o
n
ef
f
ects
of
item
d
i
sc
r
i
m
i
na
tion
a
n
d
diff
icu
l
ty
s
how
e
d
t
hat
f
o
r
th
e
cases
w
h
o
s
e
b
p
aram
eters
were
l
o
w
e
r
than
0
with
w
eak
d
is
crim
i
n
a
t
i
on,
i
nd
i
ces
h
ad
w
eak
p
o
w
er
f
o
r
b
o
t
h
ω
an
d
G
BT.
In
addition,
f
or
t
he
cases
w
hose
b
p
ar
amet
er
l
evel
s
were
b
elow
zer
o
w
i
t
h
h
igh
di
scriminatio
n
in
di
ces,
th
e
p
o
w
e
r
perform
ance
of
b
oth
answer-co
py
ing
in
di
ces
w
ere
v
e
ry
w
eak.
Resu
lt
s
fo
r
t
e
st
l
engt
h
show
e
d
t
hat
w
i
t
h
the
in
creas
e
o
f
t
e
s
t
l
e
n
g
t
h
t
h
e
p
o
w
e
r
r
a
t
e
o
f
b
o
t
h
ω
a
n
d
G
B
T
t
e
n
d
e
d
t
o
i
n
c
r
e
a
s
e
.
A
l
s
o
,
ω
perf
ormed
s
lig
h
t
l
y
b
e
t
ter
t
h
an
G
BT
o
r
very
c
lose
t
o
GB
T
f
o
r
8
0
-it
e
m
t
e
st
len
g
t
h
h
o
w
ever,
ω
perf
orm
e
d
b
e
tter
t
h
an
G
BT
i
n
term
s
of
p
ow
e
r
r
ate
f
o
r
th
e
cases
with
40
-
it
e
m
tes
t l
e
ng
th.
K
eyw
ord
:
A
n
sw
er
c
opy
in
g
C
h
ea
tin
g
Copy
d
e
t
ec
ti
on
GBT
Om
ega
(ω)
Co
pyri
gh
t © 2
018 In
stit
u
t
e
of Advanced
En
gi
neeri
n
g
an
d
Scien
ce.
All
rights
res
e
rv
ed.
Corres
pon
d
i
n
g
Au
th
or:
O
nder
S
unb
u
l
,
Fa
cult
y
o
f
E
duc
ati
o
n, D
epar
t
m
ent
o
f
Mea
sur
e
m
e
nt a
nd Ev
alua
t
i
o
n
in
Educa
t
io
n
,
Mersi
n
U
ni
ver
s
i
t
y,
Mersin,
Turkey.
Em
ail:
onde
rsu
n
b
u
l
@
m
e
r
sin.
edu.tr
1.
I
N
TR
OD
U
C
TI
O
N
Mu
lt
ip
l
e
c
ho
i
c
e
i
t
em
t
e
s
ts
h
ave
be
en
u
se
d
for
d
e
cade
s
f
or
d
ec
is
io
n
m
a
ki
n
g
i
n
h
i
g
h
-sta
ke
t
e
s
ts
.
D
e
spi
t
e
the
fa
c
t
t
ha
t
m
u
lti
p
l
e
cho
i
ce
item
tes
t
s
ha
ve
n
ume
r
ous
a
d
v
a
n
t
ag
es
f
o
r
l
arg
e
s
cal
e
e
x
amin
a
t
io
n
s
;
t
h
e
y
als
o
h
a
v
e
se
ve
ra
l
w
e
aknesse
s
w
h
ic
h
espec
i
a
l
l
y
t
hrea
te
n
tes
t
s
e
c
ur
it
y
iss
u
es.
Te
st
s
ecur
ity
i
s
vi
t
a
l
for
va
li
d
dec
i
si
o
n
s
t
h
a
t
d
epe
n
d
o
n
t
e
s
t
sc
ores.
Wi
th
t
he
i
ncre
ase
of
p
o
p
u
la
t
i
o
n
i
n
c
o
un
tries,
c
ompe
ti
ti
on
a
m
ong
exa
m
ine
e
s
o
r
e
d
uc
a
t
io
na
l
or
gan
i
za
tio
ns
t
o
cove
r
a
requi
re
me
nt
,
to
p
as
s
a
cuto
ff
poi
nt,
to
f
in
d
a
j
ob
has
incre
a
se
d.
T
he
i
ncr
ease
of
p
o
p
u
l
a
t
i
o
n
ac
ro
ss
lim
ite
d
s
ource
s
fo
rce
s
i
n
d
iv
i
dua
ls
o
r
e
d
uca
t
i
o
nal
orga
niz
a
tio
ns
t
o
c
h
e
a
t
i
n
ex
a
m
i
n
a
tio
ns
.
C
h
ea
t
i
ng
i
n
e
x
amin
at
io
ns
c
a
n
b
e
c
o
ndu
cte
d
i
n
se
vera
l
w
a
ys
:
T
r
ansm
i
t
ti
n
g
a
n
s
w
e
rs
of
q
ues
t
io
ns,
m
e
m
o
r
i
zin
g
que
s
t
i
o
ns
f
or
f
u
t
ur
e
exa
m
i
n
ee
s
w
h
i
c
h
r
esu
l
t
i
n
i
tem
exp
o
s
u
re
,
coll
us
i
o
n
dur
in
g
exa
m
ina
t
i
ons,
e
r
asure
s
a
fter
e
xa
mina
t
i
ons,
ob
ta
in
i
ng
que
s
t
i
o
ns
be
fo
r
e
e
xam
i
na
t
i
ons
t
o
have
p
re
-k
no
w
l
ed
g
e
abo
u
t
i
tem
s
e
t
c
.
Te
chn
o
l
o
gic
a
l
deve
l
opme
n
ts
h
ave
ma
de
c
om
munic
a
ti
o
n
e
a
s
ier
.
H
ow
eve
r
,
one
o
f
t
h
e
s
i
de
effec
t
s
of
t
he
se
i
mpro
ve
me
n
t
s
is
t
he
t
hrea
t
s
f
or
t
est
secu
r
i
t
y
.
W
he
n
w
e
c
om
pare
i
t
w
i
t
h
t
he
l
as
t
de
c
a
des
now
a
d
a
y
s
i
t
i
s
very
e
a
s
y
to
t
ransm
it
or
b
roa
d
c
a
s
t
t
he
a
nsw
e
rs
o
f
a
n
e
x
a
m
i
n
a
t
i
o
n
.
T
e
s
t
s
e
c
u
r
i
t
y
a
r
e
a
,
w
h
i
c
h
i
s
a
su
bdom
ai
n
of
e
duca
t
io
na
l
m
e
a
s
u
r
em
ent,
h
as
t
o
improv
e
i
t
se
lf
b
ec
au
se
o
f
t
h
e
s
e
t
h
rea
t
s.
L
it
erat
u
r
e
re
v
i
e
w
prese
n
t
s
num
e
r
ou
s
c
h
ea
t
i
n
g
d
e
t
e
c
t
ion
proc
edur
es
[
1]
-[2
3
]
.
A
s
pr
e
v
ious
l
y
m
en
ti
one
d,
t
he
re
a
re
s
ever
al
w
ays
to
c
hea
t
i
n
te
sts
;
h
ow
e
v
er,
t
h
i
s
s
tud
y
o
n
l
y
foc
u
ses
o
n
a
n
s
w
e
r
c
opy
in
g.
S
e
v
era
l
a
nsw
e
r-c
opy
i
n
g
det
e
c
t
io
n
Evaluation Warning : The document was created with Spire.PDF for Python.
IJERE
I
S
S
N
:
2252-
88
22
Ef
fec
t
s of T
e
st L
e
v
e
l
D
i
scrim
i
na
tio
n and D
i
f
f
icu
l
ty on
A
n
s
w
e
r
-
C
o
p
y
i
n
g
Ind
i
ces (O
nder Su
nb
u
l
)
33
in
dic
e
s
focu
s
on
m
a
tc
hi
n
g
e
xa
mine
e
s
’
in
corre
ct
r
e
s
po
n
s
es
a
nd
so
m
e
o
f
t
h
e
m
f
o
c
u
s
o
n
m
a
t
c
h
i
n
g
b
o
t
h
exa
m
ine
e
s
’
co
rr
ect
a
nd
inc
o
r
r
ec
t
r
e
spo
n
ses.
T
he
p
e
rfor
m
a
n
ces
o
f
a
nsw
e
r-c
opy
i
n
g
de
t
e
cti
on
st
at
is
t
i
c
s
m
ay
v
a
ry
d
ep
endi
ng
o
n
a
s
su
mp
tio
n
s
o
f
in
di
c
e
s
as
w
ell
as
t
he
t
e
s
t
an
d
i
t
e
m
s
pec
i
fic
a
t
i
ons.
In
o
t
h
e
r
w
ords,
d
e
ci
sion
s
made
f
o
r
e
x
a
mi
n
ees
f
o
r
a
n
s
we
r
c
o
pyi
ng
m
ust
be
e
v
a
l
u
at
e
d
w
ith
p
a
y
in
g
a
t
te
n
tio
n
to
a
ss
um
p
t
i
o
n
of
i
n
d
i
ce
s,
t
e
s
t
and
e
x
amin
ee
s
pe
c
i
fi
cat
i
o
n
s
a
s
an
a
dd
it
i
o
n
a
l
e
v
id
en
ce
o
f
ch
ea
t
i
ng
.
Li
t
e
ra
t
u
r
e
r
ev
i
e
w
sh
o
w
s
t
h
at
Omeg
a
(ω
) a
n
d G
e
nera
l
i
ze
d B
i
nom
ia
l Te
st
(
G
B
T)
i
nd
ice
s
ar
e
qui
te
r
em
a
r
kab
l
e.
Ome
g
a
(ω)
i
n
de
x
wa
s
p
r
opo
se
d
by
W
ol
l
a
ck
[
23
].
B
ri
efly
,
ω
co
mp
ar
e
s
t
he
e
x
p
e
ct
e
d
m
at
ch
w
i
t
h
t
h
e
obs
erve
d m
a
tc
h b
e
tw
ee
n tw
o exa
m
i
n
ees’
re
sp
on
se
v
ect
o
r
s. Ori
g
i
n
a
l for
m
ula
of ω
i
s
base
d on
i
t
e
m
para
m
e
t
ers
ob
ta
ine
d
f
r
o
m
the
n
o
mi
nal
r
e
sp
onse
m
o
d
e
l
[24]
o
f
Item
R
es
po
nse
T
h
e
o
ry.
The
ex
pe
cted
v
a
l
ue
s
of
e
a
c
h
respo
n
se
a
l
t
e
r
n
a
ti
ve
w
i
l
l
be
e
st
im
ated
b
y
us
in
g
n
o
mina
l
r
e
spo
n
s
e
mo
d
e
l
e
s
ti
mat
i
on
s
und
e
r
t
h
e
r
e
s
t
r
i
c
ti
on
o
f
abi
l
i
t
y
p
ara
m
e
t
e
r
s
o
f
p
r
oba
b
l
e
c
o
p
i
er
a
nd
sou
r
ce.
D
espit
e
t
h
e
f
a
c
t
t
h
at
ω
’
s
o
ri
gi
na
l
form
ul
a
w
a
s
deve
lop
e
d
b
y
us
i
n
g
N
o
m
i
nal
Re
sp
o
n
se
M
o
d
e
l
a
n
d
u
s
i
ng
op
t
i
on
m
a
tri
x
o
f
a
mu
lt
i
pl
e
c
h
o
i
ce
item
tes
t
,
it
i
s
a
l
s
o
p
o
ss
ible
t
o
com
p
u
t
e
ω
on
di
c
h
otom
o
u
s
d
a
ta
s
e
t
s
b
y
u
s
i
ng
1P
L,
2
P
L
o
r
3P
L
m
odels
o
f
I
R
T
[
25],[26]
.
Literature
r
eview
prese
n
t
s
s
e
v
e
r
al
s
tu
die
s
w
hi
ch
c
om
par
e
ω
f
or
nom
i
n
a
l
r
esp
o
n
ses
an
d
d
i
chot
o
m
ou
s
re
sponse
s
.
Be
ca
u
s
e
of
t
h
e
hard
ines
s
of
u
s
i
n
g
o
r
w
o
rki
n
g
o
n
o
pt
i
o
n
m
a
trix,
rese
arc
h
ers
in
sec
u
ri
ty
c
ompa
n
i
e
s
t
en
d
t
o
w
ork
o
n
dic
h
ot
o
m
o
u
s r
e
sp
onse
s
ra
t
he
r tha
n
n
om
ina
l
r
esp
onses
[
2
7
].
G
e
nera
li
z
e
d
B
i
nomia
l
Te
st
i
s
a
n
o
t
he
r
in
de
x
de
ve
l
ope
d
by
va
n
der
L
in
de
n
an
d
S
o
ta
rid
o
n
a
[28]
a
nd
use
d
f
or
a
nsw
e
r-
copy
in
g
det
e
c
tio
n.
L
i
k
e
-
ω
inde
x,
it
t
a
ke
s
i
n
to
a
c
c
o
un
t
ide
n
tica
l
c
orre
ct
a
n
d
i
nc
orr
ect
answe
r
s
o
f
a
s
uspe
cte
d
p
air
o
f
c
o
p
i
e
r
a
n
d
s
o
u
rce
exam
i
n
ee.
G
BT
u
s
es
c
omp
o
und
b
i
n
o
m
i
a
l
di
st
rib
u
ti
on.
T
h
i
s
tes
t
a
lso
r
e
q
u
i
r
es
e
st
im
atio
ns
o
bta
i
ne
d
fr
o
m
I
RT
m
ode
ls
(
N
o
m
i
na
l
R
e
s
p
o
n
s
e
M
o
d
e
l
o
r
1
P
L
,
2
P
L
o
r
3
P
L
mode
l
s
o
f
IRT
)
.
In
a
dd
i
tio
n
t
o
d
is
t
r
ibu
t
io
na
l
d
i
ffere
n
ces
b
etw
e
e
n
G
BT
a
nd
ω
,
a
no
t
h
er
d
i
s
ti
nct
i
on
be
tw
e
e
n
t
h
e
m
i
s
p
o
in
te
d
b
y
Zopl
uog
lu
[
29
]
th
at
G
BT
i
s
a
si
mi
l
a
r
i
ty
s
t
a
t
is
tic
s
(sy
m
m
e
tric
)
how
e
v
er
ω
i
s
a
n
an
swer-
cop
y
i
n
g
i
nde
x (
a
sym
m
e
t
r
i
c
)
. Ther
e a
r
e
very li
m
i
t
e
d
st
u
d
i
es
i
n l
i
t
era
t
ur
e for GBT.
1.1.
Pu
r
p
ose
of the
S
tu
d
y
N
u
m
e
r
ous
s
tu
die
s
h
a
v
e
be
e
n
c
on
d
u
cte
d
t
o
c
o
m
p
ar
e
ans
w
e
r-copyi
n
g
d
e
tec
t
i
o
n
i
n
di
c
e
s
for
seve
ral
conditions
i
n
t
erm
s
o
f
Type
I
e
rror
and
pow
er
s
tudies
[
5]-[7],[
12],[13]
,[1
5
]
,
[19]
,[2
0
],
[2
3],
[
25],
[
2
6
],
[2
8],[30]-
[3
3].
Lit
e
rat
u
re
r
evie
w
p
o
in
t
s
t
ha
t
tw
o
o
f
t
he
a
nsw
e
r
c
o
p
y
i
n
g
i
n
d
i
c
es,
ω
[23]
a
n
d
G
B
T
[
2
8
],
a
re
r
e
m
ar
kable
with
l
ow
T
ype
I
error
rates
and
hi
gh
d
et
e
c
tio
n
po
wer
ra
t
i
o
s
[
1
9
],[2
0],[23
],[25
]
,[26]
,[3
0
]
-
[
34].
H
o
w
e
ve
r
ther
e
a
r
e
l
i
m
it
ed
s
tud
i
es
w
hi
ch
c
omp
a
re
t
h
e
p
erfo
rma
n
ce
o
f
t
h
e
s
e
in
d
i
c
e
s
i
n
l
i
t
e
r
at
ure
.
Z
opl
uo
g
l
u
an
d
D
a
v
e
np
or
t
[2
6]
i
nve
st
i
g
at
ed
t
he
p
e
rformanc
e
o
f
ω
an
d
G
B
T
for
sever
a
l
am
o
u
nt
o
f
co
pyi
n
g
an
d
c
o
p
i
e
r
&
s
ou
rce
a
b
ilit
y
c
r
o
ssing
.
Zo
pl
u
ogl
u
[25
]
i
nve
sti
g
at
ed
t
h
e
p
erf
o
rma
n
ce
o
f
ω
,
G
BT
,
K,
,
,
S
1
, S
2
o
n
seve
ral si
m
u
late
d da
ta
se
t
s
a
nd
r
ea
l
da
t
a
s
e
t
.
Su
nb
ul
a
nd
Y
o
r
maz
[3
4
]
i
nv
esti
g
a
t
e
d
t
h
e
p
e
rform
anc
e
o
f
ω
i
n
dex
on
t
he
8
0-ite
m
five
op
tio
n
m
u
l
t
i
p
le
c
ho
ice
te
st
w
it
h
10,
0
0
0
e
xa
m
i
nees.
The
d
a
ta
s
e
t
c
ont
a
i
ns
f
ou
r
di
ff
ere
n
t
i
t
em
t
yp
e
s
:
20
e
a
s
y
-
low
d
i
scrim
i
na
tive
i
t
em
s,
2
0
ha
rd
-
l
ow
d
isc
r
imina
t
i
v
e
i
t
e
m
s,
2
0
ea
sy
-
h
ig
h
d
i
scrim
i
na
tive
i
t
em
s
a
nd
2
0
h
ard
-hi
gh
disc
rimi
nat
i
v
e
i
t
em
s.
I
n
t
h
e
i
r
st
ud
y
the
y
i
nve
s
t
i
g
ate
d
t
h
e
pe
rfor
ma
nce
o
f
ω
f
or
c
hea
t
i
ng
o
v
er
s
pe
ci
fi
e
d
it
e
m
g
ro
ups.
Liter
a
ture
r
ev
ie
w
pre
s
e
n
t
s
v
er
y
l
i
m
i
t
e
d
s
t
ud
i
e
s
w
h
i
c
h
com
p
are
the
pe
rfor
m
anc
e
s
o
f
ω
a
nd
G
BT
by
t
a
k
i
n
g
i
n
t
o
a
ccou
n
t
t
es
t
a
n
d
i
t
e
m
p
ro
per
t
i
e
s
a
n
d
pr
ev
io
u
s
s
t
ud
i
e
s
sh
o
w
t
h
a
t
it
em
p
ro
pert
i
e
s
and
e
s
p
eci
al
ly
di
ffic
ul
t
y
le
v
el
s a
ffe
ct t
he
p
er
form
ance
of
i
n
d
i
c
e
s
[25]
.
Li
ter
a
t
u
re
r
e
v
i
e
w
show
s
n
o
s
t
u
d
y
f
or
p
erfor
m
a
n
ce
in
ves
tiga
t
i
o
n
o
n
t
es
t
l
e
ve
l
w
h
ere
tes
t
s
ha
ve
homo
g
e
n
eo
us
i
tem
gr
ou
ps
i
n
ter
m
s
of
item
diffic
u
l
t
y
a
n
d
dis
c
r
i
m
ina
t
i
on.
I
n
this
s
tu
dy,
w
e
ge
ne
rated
1
2
di
ffe
re
nt
hom
o
g
ene
ous
(
i
n
t
e
r
m
s
o
f
i
t
em
p
r
o
pert
ies)
i
t
e
m
g
r
ou
ps
f
o
r
4
0
an
d
80
-i
t
e
m
t
e
st
l
e
ngt
hs.
By
c
ro
ssi
ng
h
o
m
og
e
n
e
o
u
s
i
t
e
m
g
r
oup
s
wi
t
h
t
e
s
t
l
e
ngt
h
s
;
24
d
i
f
f
e
re
nt
d
i
c
ho
to
mo
us
(
1-
0)
t
es
t
da
ta
w
ere
obta
i
ne
d
a
nd
w
e
obs
erve
d
t
h
e
performa
n
ce
o
f
ω
and
G
B
T
on
se
ver
a
l
te
st
d
a
t
a
se
ts
.
Ma
i
n
pur
pos
e
of
t
h
i
s
st
ud
y
w
a
s
to
in
ves
tiga
t
e
per
f
orm
a
nce
o
f
ω
a
nd
G
B
T
for
answ
er
-cop
yi
n
g
d
e
t
e
c
t
i
o
n
o
n
di
ffe
ren
t
t
est
s
i
n
te
rms
of
a
vera
ge
it
e
m
d
i
f
fic
u
lt
y and
d
i
sc
rim
i
na
ti
o
n
.
2.
RESEARCH
M
ETH
O
D
2.1.
Rese
arch
De
s
ign
D
i
c
h
o
t
om
o
u
s
(1-
0
)
da
ta
w
ere
ge
n
e
rate
d
u
nder
the
fra
m
e
w
ork
o
f
2
P
L
mo
d
e
l
of
I
t
e
m
Re
spon
se
The
o
ry
b
y
u
s
i
ng
R
pr
ogram
ming
l
a
ng
ua
ge
.
To
m
a
n
ipu
l
a
t
e
tes
t
a
ver
age
i
t
em
d
i
f
ficulty;
f
our
differ
e
nt
b
para
me
ter
range
s
(-2.
00
–
-1.
10,
-
1.00
–
-0.10,
0
.0
0
–
0.9
0
a
n
d
1
.0
0
–
2
.
00)
w
ere
used.
To
m
anip
ula
t
e
te
st
ave
r
age
i
t
em
d
i
s
c
r
imi
n
a
t
i
o
n;
t
hre
e
d
i
ffe
ren
t
a
p
ar
am
eter
r
ange
s
w
e
r
e
use
d
(
0.
4
0
–
0
.
7
9
,
0
.8
0
–
1.1
9
a
n
d
1
.
2
0
–
1.6
0
).
a
a
nd
b
par
a
m
e
ters
w
er
e
sa
m
p
led
from
u
n
i
for
m
d
i
s
t
r
ib
ut
i
ons
d
epe
n
din
g
o
n
t
h
e
re
lated
mi
ni
m
u
m
and
ma
ximum
va
lu
es
o
f
re
le
va
n
t
p
ar
am
eter
r
ang
e
s.
B
ase
d
on
a
a
nd
b
pa
ram
e
ter
restric
t
i
o
n
s
,
dich
o
t
om
ous
i
tem
respo
n
ses
w
e
re
ge
n
era
t
e
d
for
40-
item
a
n
d
8
0
-
item
test le
n
gt
h fo
r 10,
0
00-
exam
inee
sam
p
l
e
size
.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
IJERE
V
ol
.
7,
N
o.
1,
Ma
r
ch 201
8
: 32
– 38
34
Ra
nd
om
i
tem
cop
y
i
n
g
p
roc
e
dure
w
a
s
a
p
p
l
i
e
d
for
f
i
xe
d
c
opy
i
n
g
am
o
un
t
of
4
0%
f
or
e
a
c
h
r
ep
l
i
cat
i
o
n.
A
m
ount
o
f
c
o
p
y
in
g
w
a
s
t
a
ke
n
an
d
fix
e
d
as
4
0%
f
or
eac
h
c
o
n
d
i
t
i
on
b
e
cau
se
p
re
vio
u
s
s
t
u
dies
[
2
3
]
,[
26
][
30
],[
3
1
]
sho
we
d
t
h
a
t
p
o
w
e
r
rat
es o
f
i
ndi
c
e
s were
q
ui
t
e
sat
i
sfac
t
o
ry
f
or
30% a
n
d
m
ore
.
A
b
i
l
i
t
y
le
ve
ls
o
f
c
o
p
i
er
a
nd
sourc
e
a
re
i
m
por
tan
t
f
or
t
h
e
p
erfo
r
m
a
n
ce
of
a
nsw
e
r-c
op
yin
g
i
ndic
e
s
.
Se
vera
l
s
t
u
d
i
e
s
i
n
t
h
e
liter
a
t
u
r
e
h
a
v
e
poi
n
t
ed
t
o
the
im
p
o
rta
n
c
e
o
f
e
x
a
m
i
n
ee
a
b
i
l
i
t
y
le
ve
ls
o
n
a
n
sw
er
c
o
p
y
i
ng
[2
6],
[
3
4
].
F
or
t
hi
s
s
t
u
dy
the
a
b
i
lit
ies
o
f
e
xa
m
i
ne
es
w
e
r
e
o
b
ta
i
ned
b
y
u
s
i
ng
i
r
t
o
y
s
[3
5]
R
p
ac
ka
ge
d
e
p
e
n
d
i
ng
on
2P
L
Ite
m
R
e
s
p
o
nse
M
o
del.
F
or
e
a
c
h
r
e
pl
icat
i
on,
c
o
p
iers
w
ere
sa
mpled
from
f
ir
st
q
uart
il
e
o
f
a
bi
l
ity
di
stri
b
u
t
i
on
a
n
d
s
ou
r
c
e
s
w
ere
sa
mple
d
fr
o
m
f
our
th
q
uart
ile
o
f
a
bil
i
t
y
d
i
s
trib
u
tio
n.
I
t
i
s
a
ssume
d
tha
t
l
ow
abi
l
i
t
y
l
e
v
e
l
e
xa
mine
es te
nd
to
c
he
at fr
o
m
h
i
gh
ab
i
l
i
t
y
le
v
e
l
e
xa
minee
s
.
F
our
d
i
f
fer
e
n
t
nom
i
n
a
l
a
lpha
l
e
v
e
l
s
(0.
0
0
1
,
0.
0
05,
0
.0
1
a
n
d
0.0
5)
w
er
e
use
d
f
o
r
T
ype
I
error
and
pow
er
s
t
u
d
y
.
F
i
ve
h
u
ndre
d
s
re
pli
c
a
t
ions
w
e
r
e
cond
uc
t
e
d
f
o
r
eac
h
e
x
p
e
rim
e
nta
l
c
e
ll
of
c
on
d
iti
on
c
r
ossi
ng.
O
m
ega
a
nd
G
B
T
i
ndic
e
s
w
e
r
e
eva
l
ua
te
d
i
n
t
erm
s
o
f
Ty
pe
I
e
rr
or
and
pow
e
r
s
tud
y
.
Ta
b
l
e
1
prese
n
ts
si
m
u
lat
i
on de
si
gn r
e
gard
in
g r
e
sea
r
ch
c
on
di
ti
o
n
s a
n
d its
l
e
v
e
l
s.
Tab
l
e
1. S
imulati
o
n D
e
si
g
n
C
on
d
iti
on
s a
nd
Leve
l
s
C
onditi
on
Nu
m
b
e
r
o
f
Le
v
e
ls
L
e
v
el
V
a
l
u
e
s
S
a
m
p
le
S
iz
e
1
1
0
0
0
0
Te
st
L
e
ngth
(
T
)
2
40 (
T1
)
80 (
T2
)
Am
ount
o
f C
opy
ing
1
40%
Item
Dis
c
r
i
m
in
a
tio
n (
a
)
3
0.
40
–
0.
79
(
a1
)
0.
80
–
1.
19
(
a2
)
1.
20
–
1.
60
(
a3
)
Item
D
iffic
u
lt
y (
b
)
4
-2
–
-1
.
1
(
b1
)
-1
.0
–
-0
.
1
(
b2
)
0.
0
–
0.
9 (
b3
)
1.
0
–
2.
0 (
b4
)
Indic
e
s
2
ω
a
nd
G
B
T
N
u
m
b
e
r
o
f
R
e
plica
tions
5
00
I
n
t
he
c
urre
nt
s
tu
dy,
c
on
d
iti
on
s
and
its
l
eve
l
s
w
e
re
d
eterm
i
n
e
d
b
y
pay
i
ng
a
t
t
e
n
t
i
o
n
t
o
t
he
c
on
d
i
t
i
on
s
of rese
a
rc
hes
[
19]-[20],
[
2
3
],
[26]
,
[
3
0
] to m
ake com
p
ar
iso
n
s
. The
e
m
p
i
r
i
c
al
T
ype
I
er
ror
rate
w
as
c
om
puted
by
d
e
t
e
rmin
ing
t
h
e
ra
t
i
o
o
f
hon
e
s
t
ex
amin
e
e
p
ai
rs
w
hi
ch
w
e
r
e
f
l
a
g
g
e
d
a
s
c
o
p
i
e
r
i
nc
orre
ct
l
y
b
y
i
n
di
c
e
s
o
v
er
5
00
repl
ica
t
i
on
(
hone
s
t
p
a
i
rs).
F
o
r
t
he
p
ow
er
s
tud
y
,
p
o
w
e
r
of
i
n
d
i
c
e
s
w
as
c
om
pute
d
by
de
te
rm
in
i
ng
t
h
e
rati
o
o
f
exa
m
ine
e
pa
irs
w
ho
w
e
re
f
la
gge
d
as
c
op
ier
cor
r
ec
t
l
y
b
y
i
n
d
i
ce
s
o
v
er
500
r
e
plic
a
t
i
o
n.
I
n
c
o
n
t
rast
w
i
t
h
T
ype
I
err
o
r stu
dy, co
p
y
i
ng
sce
nar
i
o
s
w
e
r
e implem
ente
d
t
o
the e
x
a
m
i
nee
p
airs’ (copier
-source)
respo
nse
vec
t
ors.
To
cre
a
te
c
o
p
i
ed
e
xam
i
ne
e
pa
i
r
s,
t
he
f
ixe
d
a
m
o
u
n
t
40%
o
f
t
h
e
ran
d
o
m
ite
m
response
s
o
f
co
p
i
er
e
xa
mi
nee
w
a
s
altere
d
w
ith
t
he
r
espo
nses
o
f
s
ource
e
xa
m
i
nee
for
e
ach
e
x
p
erim
e
n
t
a
l
c
e
ll
an
d
r
e
p
l
i
c
a
t
i
on.
C
opy
D
e
tect
pac
k
a
g
e
o
f
R
[
3
6
]
w
a
s
u
s
ed
f
or
c
omp
u
t
i
n
g
the
pr
oba
b
i
l
i
t
ie
s
of
i
ndic
e
s.
B
o
t
h
for
T
y
pe
I
e
rr
or
a
n
d
p
o
w
e
r
stud
y
resul
t
s
w
e
r
e
a
ggrega
t
e
d
by
sp
e
c
i
f
ie
d
n
o
m
i
nal a
l
pha
le
v
e
l
s
.
3.
RESULT
S
A
N
D
ANALY
S
IS
3.1.
T
y
pe
I
e
rr
o
r
s
t
u
dy
T
y
p
e
I
e
r
r
o
r
r
a
t
e
s
w
e
r
e
g
i
v
e
n
i
n
T
a
b
l
e
2
.
T
h
e
r
e
s
u
l
t
s
o
f
t
h
e
T
y
pe
I
e
rror
stu
d
y
s
how
e
d
t
hat,
m
ost
o
f
the Ty
pe
I
e
rror rate
s w
e
re zero
or
v
er
y
c
l
os
e to
z
ero.
A
ny Ty
p
e I
e
r
ror
infla
t
io
n
f
o
r
a
ny con
d
i
t
i
o
n c
o
u
l
dn
't
b
e
obs
erve
d
rega
r
d
le
ss
o
f
t
e
s
t
a
v
er
age
d
i
fficu
l
t
y
a
nd
d
i
scrim
i
na
t
i
on.
Most
o
f
the
result
s
w
e
re
0
.00
and
seve
ral
di
sc
repa
nc
ies
from
0.
00
w
e
r
e
obs
erve
d
f
o
r
ω
inde
x.
F
or
t
he
m
ost
d
i
f
f
i
c
u
l
t
t
es
t
(
b4)
,
on
ly
one
e
xc
eeda
n
c
e
w
a
s
obs
erve
d
f
o
r
n
o
mina
l
a
l
p
h
a
le
vel
0.05
f
or
ω
i
n
d
ex.
G
B
T
per
f
orm
e
d
ver
y
w
el
l
for
ea
ch
c
o
n
d
i
t
i
o
n
i
n
t
e
rm
s
of
Type
I
error
rat
e
.
Evaluation Warning : The document was created with Spire.PDF for Python.
IJERE
I
S
S
N
:
2252-
88
22
Ef
fec
t
s of T
e
st L
e
v
e
l
D
i
scrim
i
na
tio
n and D
i
f
f
icu
l
ty on
A
n
s
w
e
r
-
C
o
p
y
i
n
g
Ind
i
ces (O
nder Su
nb
u
l
)
35
Tab
l
e
2.
T
ype I
Err
or r
ates of ω
a
n
d
G
B
T fo
r
S
i
m
u
lat
i
on
Con
d
i
t
i
on
s
Al
pha
Level
T1
T
2
b
1
b
2
b3
b4
b1
b
2
b
3
b4
ω
GBT
ω
GBT
ω
GB
T
ω
GB
T
ω
GBT
ω
GBT
ω
GBT
ω
GB
T
a1
0.00
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.00
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
.
01
0
0
0
0
0
0
0
0
0
0
0
.
01
0
0
0
0
.01
0
0
.
05
0.01
0
.0
1
0.03
0
0
.04
0
.
01
0
.
0
4
0
0
.
0
1
0.01
0
.04
0.0
2
0
.03
0
.
0
1
0.06
0
.01
a2
0.00
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.00
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
.
01
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.01
0
0
.
05
0
0
0.02
0
.
0
1
0.04
0
.01
0
.
0
3
0
0
0
0
.01
0.0
1
0
.03
0
0.04
0
a3
0.00
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.00
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
.
01
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.01
0
0
.
05
0
0
0
0
0.0
1
0
0
.
0
2
0
0
0
0
0
0.01
0
0
.03
0
3.2.
Pow
e
r study
I
t
i
s
ob
se
rved
i
n
F
i
g
u
re
1
t
hat
the
p
o
w
e
r
of
ω
a
nd
G
B
T
in
dic
e
s
t
e
n
d
ed
t
o
incr
ease
w
i
t
h
t
he
i
ncre
ase
of
t
es
t
le
n
g
t
h
a
nd
t
h
i
s
re
s
u
lt
i
s
co
ns
is
t
e
n
t
w
it
h
the
lite
rat
u
r
e
[19]
,[2
0
]
,
[23],
[
3
0
]-
[3
2].
A
n
o
t
her
c
l
e
a
r
pat
t
e
r
n
w
a
s
observe
d
for
incre
a
ses
i
n
t
e
s
t
d
i
ffic
ul
t
y
.
Wi
th
t
he
i
nc
re
a
s
e
o
f
b
p
ara
m
et
ers
o
f
t
est
i
t
e
ms,
p
o
w
e
r
o
f
ω
a
n
d
G
B
T
i
n
d
i
c
e
s
t
e
nde
d
t
o
i
ncr
e
ase
,
t
oo.
T
h
i
s
r
e
sul
t
i
s
a
l
so
c
on
sis
te
nt
w
i
t
h
p
rev
i
ous
s
tu
d
i
e
s
i
n
t
e
rm
s
of
t
he
rela
tio
ns
hip
be
tw
een
t
he
d
i
f
ficu
l
t
y
an
d
c
o
p
y
d
e
t
e
c
t
ion
pow
er
[
2
5],[34]
.
Howe
ver
a
cl
e
a
r
incr
ease
p
a
tter
n
cou
l
dn
't
b
e
o
b
s
erve
d
w
i
t
h
t
h
e
i
n
c
re
ase
of
a
p
ar
am
eter
s
o
f
t
est
it
e
m
s.
T
he
d
iffer
e
nce
be
t
w
ee
n
ω
a
n
d
G
B
T
in
dic
e
s
i
n
cre
a
s
e
d
in
f
av
or
o
f
ω
w
ith
t
he
i
nc
rea
s
e
of
a
p
a
r
a
m
ete
r
s
e
s
p
e
c
i
a
l
l
y
f
o
r
t
h
e
t
e
s
t
s
w
i
t
h
h
a
r
d
e
r
i
t
e
m
s
.
A
n
o
t
her
i
n
fe
renc
e
m
a
y
i
n
cl
u
d
e
t
h
e
in
ter
a
c
tio
n
be
tw
e
e
n
t
est
le
n
g
t
h
a
n
d
i
t
e
m
d
i
f
f
i
c
u
l
t
y
.
B
o
t
h
ω
a
n
d
G
B
T
in
dic
e
s
w
e
re
o
bser
ve
d
t
o
b
e
mor
e
capa
b
l
e
t
o
detec
t
c
o
p
i
er
s
w
h
e
n
t
he
y
c
h
ea
te
d
i
n
l
o
n
g
e
r
t
e
sts
w
i
th
h
ar
d
e
r
it
e
m
s
espec
i
a
l
l
y
f
or
0
.05 a
n
d
0.01
n
o
mi
nal
a
l
p
h
a le
ve
ls.
Whe
n
w
e
c
o
m
p
are
t
h
e
re
sul
t
s
of
ω
a
n
d
G
BT
,
ω
w
a
s
foun
d
to
p
e
rfor
m
bet
t
e
r
t
ha
n
G
B
T
n
ear
ly
f
or
a
l
l
c
o
n
d
i
t
i
o
n
s
.
ω
i
n
d
e
x
c
l
e
a
r
l
y
p
e
r
f
o
r
m
e
d
b
e
t
t
e
r
t
h
a
n
G
B
T
e
s
p
e
c
i
a
l
l
y
fo
r
the
40
-i
t
e
m
t
e
st
l
e
ngt
h
who
s
e
it
ems'
b
para
me
ters
w
er
e
hi
g
h
er
t
ha
n
0.00.
T
he
d
i
f
fe
r
e
nce
be
tw
ee
n
the
p
o
wer
ra
t
e
s
of
t
wo
i
nd
i
ces
got
c
l
o
ser
wit
h
t
h
e
i
n
c
r
e
a
s
e
o
f
t
e
s
t
l
e
n
g
t
h
.
I
n
g
e
n
e
r
a
l
,
t
h
e
p
e
r
f
o
r
m
a
n
c
e
o
f
ω
a
n
d
G
BT
i
n
d
ice
s
w
er
e
un
sa
t
i
s
f
a
c
tor
y
f
or
t
es
t
s
w
hose
it
e
m
s
'
b
p
ara
m
e
t
ers
w
e
re low
er
than
0.0
0
for
a
l
l
n
o
m
i
n
al a
lpha
le
ve
l
s
F
i
gur
e 1.
The
Inte
r
ac
ti
o
n
Effe
c
t
s
of
C
o
n
d
it
io
ns for
P
ow
er Stu
d
y
Evaluation Warning : The document was created with Spire.PDF for Python.
I
SSN: 2252-
8822
IJERE
V
ol
.
7,
N
o.
1,
Ma
r
ch 201
8
: 32
– 38
36
4.
DISC
USSION
The
s
t
ud
ies
[1
9],
[
20],[2
3
],[2
5],[26
]
,
[3
0]
-[3
4
]
sh
ow
ed
ω
i
nde
x
t
o
ha
ve
l
ow
e
r
T
y
p
e
I
er
ror
rate
a
n
d
hi
gh
de
t
e
ct
i
o
n
pow
er
f
or
a
n
s
w
e
r
c
opy
i
n
g
un
der
ma
ny
con
d
iti
on
s.
L
i
t
e
r
a
tu
r
e
r
e
v
ie
w
p
r
e
s
e
n
ts
t
h
a
t
a
n
s
w
e
r
-
cop
y
i
n
g
d
e
t
e
c
t
i
o
n
pow
er
o
f
ω
tende
d
t
o
i
n
c
re
ase
w
i
t
h
t
he
i
nc
rea
s
e
o
f
t
e
s
t
l
e
n
g
t
h
.
T
h
e
r
e
s
u
l
t
s
o
f
t
h
i
s
s
t
u
d
y
w
e
r
e
f
ou
n
d
t
o
be
s
im
ila
r
t
o
t
hos
e
in
l
i
t
era
t
ur
e
in
t
e
r
m
s
o
f
l
o
w
T
ype
I
e
rror
r
a
tes
an
d
p
o
w
er
i
nc
rea
s
e
w
ith
t
he
incre
a
se
o
f tes
t
le
ngt
h.
Zop
l
uo
gl
u
an
d
Da
v
e
npo
rt
[
26
]
co
mp
ared
t
h
e
p
erfo
rma
n
ce
s
of
ω
a
n
d
G
B
T
w
i
t
h
c
r
o
s
s
i
n
g
t
h
e
a
b
i
l
i
t
y
leve
ls
o
f
c
o
p
i
e
r
a
nd
s
o
u
rce
a
n
d
sh
ow
e
d
t
ha
t
G
B
T
w
a
s
s
l
i
ght
ly
b
e
t
t
er
t
h
a
n
ω
in
g
e
n
era
l
.
I
n
c
on
trast,
p
resen
t
stud
y
sh
ow
ed
t
hat
ω
p
e
rfor
m
e
d
b
e
t
t
e
r
tha
n
G
BT
i
n
ter
m
s
of
a
nsw
e
r
c
opy
de
t
e
c
t
i
o
n
p
o
w
e
r
e
spe
c
ial
l
y
f
or
4
0-
it
em
t
es
t length with har
der
items.
S
unb
u
l
a
n
d
Y
orm
a
z
[34]
r
ep
or
t
e
d
seve
ral
Ty
pe
I
e
rror
i
n
fl
a
tio
ns
f
o
r
l
o
w
a
b
i
l
i
t
y
l
e
v
e
l
s
o
u
r
c
e
f
o
r
ω
in
de
x.
T
he
y
a
l
so
r
ep
orte
d
sa
tisfa
c
t
ory
pow
e
r
r
a
t
es
e
xcep
t
w
h
e
n
low
a
b
i
l
ity
l
e
v
e
l
c
o
p
i
er
s
chea
ted
fr
om
h
ig
h
di
sc
rimina
t
i
v
e
and
ea
sy
item
s
.
The
resu
l
t
s
o
f
t
he
c
urr
e
nt
s
tu
dy
w
ere
f
oun
d
to
b
e
c
onsis
te
n
t
w
it
h
the
r
e
su
l
t
s
of
S
unb
u
l
a
nd
Y
o
rm
az’
s
st
ud
y
[3
4]
a
s
w
e
l
l
.
Chea
t
i
ng
in
e
a
s
y
item
s
mi
g
h
t
b
e
u
nd
et
e
c
t
a
bl
e
by
a
n
s
w
e
r-co
pyi
ng
detec
t
i
on
i
n
d
i
c
e
s
.
A
n
o
t
he
r
im
por
tan
t
p
o
i
n
t
t
o
be
c
i
t
e
d
i
s
t
h
e
f
a
c
t
th
at
t
he
c
u
r
re
n
t
s
tudy
t
ri
e
d
t
o
ma
ni
pul
a
t
e
it
e
m
d
iffic
u
lt
y
a
n
d
d
i
scrim
i
na
tio
n
in
t
est
l
e
v
e
l.
I
n
othe
r
w
o
rd
s,
homo
g
e
n
e
ous
t
est
s
w
ere
ge
n
e
ra
te
d
i
n
t
e
r
ms
o
f
di
ffic
ul
t
y
a
nd
disc
rimina
t
i
on.
H
ow
eve
r
S
un
bu
l
a
nd
Y
o
rm
az
[
3
4
]
g
e
n
e
r
ated
a
n
80
i
t
em
h
eter
oge
ne
ou
s
te
st
w
h
ic
h
c
overe
d
sever
a
l
i
t
em
t
ype
s
(e
asy-ha
r
d
,
mor
e
d
i
s
cri
m
inat
iv
e
-
less
d
i
s
crim
i
n
a
t
i
v
e
)
i
n
a
s
i
ng
le
t
es
t
s
i
nc
e
the
y
w
er
e
in
tere
ste
d
i
n
c
h
ea
t
i
ng
from
s
peci
fic
t
y
pes
o
f
ite
m
s
.
The
ma
i
n
r
eason
f
o
r
this
m
ig
h
t
b
e
the
am
ount
of
item
infor
m
a
tio
n
w
h
i
c
h w
a
s
prov
ide
d
b
y
i
t
e
m
i
n
f
orm
a
tio
n
fu
nc
ti
o
n
for
t
he
easy
item
s
.
5.
CONCL
U
S
ION
Bot
h
o
f
ω
and
G
B
T
performe
d
w
ell
ne
arl
y
f
or
a
l
l
c
o
n
d
i
t
i
ons
i
n
t
e
r
m
s
o
f
T
y
p
e
I
e
r
r
o
r
r
a
t
e
s
.
T
h
e
resul
t
s
of
pow
e
r
s
t
u
dy
for
t
h
e
bo
t
h
o
f
ω
and
G
B
T
s
h
ow
ed
t
ha
t
t
h
e
y
p
e
r
for
m
e
d
w
e
ll
for
te
sts
w
h
ic
h
ha
ve
harde
r
ite
ms
h
ow
eve
r
t
he
r
esults
f
or
t
he
t
est
s
w
hic
h
h
a
v
e
e
a
sy
item
s
w
ere
unsa
tisfa
c
t
ory.
D
etec
tin
g
c
h
e
a
tin
g
f
r
o
m
e
a
s
y
it
em
t
e
s
t
s
m
i
g
ht
b
e
a
p
r
obl
em.
In
t
e
r
ms
o
f
re
al
l
if
e
ap
p
lica
t
i
ons,
it
is
n
o
t
c
om
m
on
t
o
c
he
at
f
r
o
m
ea
sy i
t
e
ms sinc
e
it mi
gh
t n
o
t
b
e
a bi
g
ga
p for
answ
er
dete
c
t
ion
i
n
d
i
ce
s. However
pra
cti
tio
ner
s
and rese
arc
h
ers
sho
u
l
d
b
e
m
o
re
c
are
f
ul
w
h
i
l
e
i
n
v
e
s
ti
ga
ti
n
g
a
nsw
e
r
c
o
pyi
n
g
o
n
ea
s
y
t
e
s
t
s
.
M
o
r
e
s
t
u
d
i
e
s
a
r
e
n
e
e
d
e
d
t
o
in
ves
itga
t
e
t
h
e
effec
t
of i
t
em
inform
a
ti
on a
m
ou
n
t
for a
nsw
e
r
co
py
ing.
ACKNOW
LEDG
E
MEN
T
S
Th
is
s
t
u
d
y
i
s
t
h
e
re
v
i
se
d
and
impr
ove
d
ve
rsio
n
of
t
he
o
ra
l
pre
s
e
n
t
a
ti
on
i
n
V.
C
on
g
r
e
ss
on
Meas
ure
m
e
n
t
and
E
v
a
l
ua
t
i
o
n
in
E
duc
a
t
io
n
and
P
s
yc
h
o
l
o
g
y
2
016
as
I
n
ve
stiga
t
in
g
t
h
e
P
e
r
f
orm
a
nce
of
C
op
y
D
e
tec
t
i
o
n
Ind
i
ces
A
ccor
d
i
n
g
t
o
I
tem
D
i
scr
i
m
i
nat
i
on a
nd
D
i
ffic
ul
ty
REFE
RENCES
[1]
An
go
ff
W
.
H
.,
“
T
he
d
ev
elop
m
e
nt
o
f
stat
isti
cal
i
n
d
i
ces
f
or
d
et
ec
tin
g
ch
ea
t
e
rs,”
Jo
urn
a
l
of
A
m
e
r
ican
Stati
s
t
i
cal
Assoc
i
ation,
vol.
6
9
,
pp.
4
4
-
49,
197
4.
[2]
A
n
i
k
e
e
f
A
.
M
.
,
“
I
n
d
e
x
o
f
c
o
l
l
a
b
o
rati
on
f
or
t
es
t
admi
n
i
s
t
rators
,
”
J
o
urna
l o
f
Ap
p
l
ie
d
Psy
c
h
olog
y
,
v
o
l
.
38,
pp.
1
7
4
-
17
7,
195
4.
[3]
Argenal
R.
N
.,
et
al.
,
“A
n
ew
i
ndex
f
o
r
detecting
col
l
u
s
i
o
n
an
d
i
t
s
statisti
cal
p
rop
ert
i
es,
”
N
a
t
i
onal
Con
v
en
tio
n
on
Sta
tist
i
c
s
,
20
04
.
[4]
Arm
s
tron
g
R.
D
.
an
d
S
h
i
M
.
A
.,
“P
aram
et
ric
cum
u
lat
i
ve
s
um
s
ta
t
i
st
ic
f
o
r
p
e
r
s
o
n
fi
t,
”
Ap
p
l
ie
d P
s
y
c
h
olo
g
i
c
al
Me
asu
r
e
m
e
n
t
,
v
o
l
/
i
s
su
e: 3
3(5
)
, p
p
.
3
91
-4
10
, 2
00
9.
[5]
Bay
M
.
L
.
G.
,
“Det
ectio
n
of
c
op
yi
ng
on
m
u
lt
iple-cho
ice
ex
aminat
ions,”
[
Doctoral
d
i
s
ser
t
a
t
i
on],
Southern
I
llinois
Unive
r
sit
y
,
1987
.
D
i
s
s
ert
a
tion Abst
rac
t
s
Int
e
rnational
,
vo
l/is
su
e
:
5
6(3
-
A),
p
p
.
899,
1
99
4.
[6]
Bell
ezz
a
F
.
S
.
a
n
d
Bell
ezza
S
.
F
.,
“Detectio
n
of
c
heat
ing
on
m
u
lti
p
l
e
-
ch
oi
ce
t
e
st
s
by
usin
g
error-s
im
il
arit
y
anal
ys
is
,
”
T
e
ach
i
ng
of
P
s
ycho
l
o
g
y,
v
ol.
1
6
,
p
p
.
151-1
55,
1
9
8
9
.
[7]
Belo
v
D.
I
.
,
“Detect
io
n
of
a
n
s
w
e
r
co
py
ing
b
a
sed
o
n
t
he
s
t
r
uct
u
r
e
o
f
a
h
i
g
h
-
sta
k
es
t
es
t,”
Ap
plied Ps
ycho
l
og
ica
l
Me
asu
r
e
m
e
n
t
,
v
o
l
/
i
s
s
u
e:
3
5
(
7),
pp.
4
9
5–5
17
,
2
011.
[8]
Belo
v
D.
I
.
and
Arm
s
t
r
on
g
R.
D
.
,
“
A
u
t
om
ati
c
d
e
t
ect
ion
o
f
a
ns
we
r
c
op
yin
g
v
ia
kul
lb
ack-l
e
i
b
l
er
d
iv
ergen
c
e
an
d
K-
index,”
App
lied Psychol
ogic
al M
e
asu
r
em
ent
,
vol/is
su
e
:
3
4(6),
pp.
3
79
–39
2,
2
0
10.
[9]
Bird C
.,
“T
h
e de
tect
io
n
o
f
ch
ea
ting in objective examinat
i
o
ns
,”
Scho
ol
an
d
s
o
ci
e
t
y,
v
o
l
. 25
,
p
p
. 2
61
-26
2
, 1
92
7.
[10]
Bird
C
.,
“
An
i
mprov
e
d
m
e
tho
d
o
f
detectio
n
cheati
n
g
i
n
o
b
j
ecti
v
e
e
xam
i
n
a
ti
ons,
”
Jour
na
l of
Ed
uca
t
i
onal
Res
e
ar
c
h
,
vo
l.
25,
p
p
.
2
61
-262
,
1
92
9.
[11]
Clark
J.
M
.,
“Aberrant
r
esponse
patt
erns
a
s
a
multidimensional
ph
eno
m
e
n
on
:
U
s
i
n
g
f
acto
r-analy
ti
c
m
o
d
e
l
com
p
ari
s
o
n
t
o
d
e
tect
cheat
in
g,”
ProQuest
L
LC
,
Unive
r
sit
y
of
Ka
n
sas
,
2
010.
Evaluation Warning : The document was created with Spire.PDF for Python.
IJERE
I
S
S
N
:
2252-
88
22
Ef
fec
t
s of T
e
st L
e
v
e
l
D
i
scrim
i
na
tio
n and D
i
f
f
icu
l
ty on
A
n
s
w
e
r
-
C
o
p
y
i
n
g
Ind
i
ces (O
nder Su
nb
u
l
)
37
[12]
Frary
R.
B
.,
et a
l
.
,
“
I
n
d
i
ces
o
f
cheatin
g
on
m
u
l
ti
p
l
e-c
h
oi
ce
t
e
sts,”
Jo
ur
na
l o
f
Ed
ucat
io
nal
S
t
a
t
i
s
tics,
v
o
l
.
6
,
p
p
.
1
5
2
-
16
5,
197
7.
[13]
Ha
nson
B
.
A
.
,
et
a
l
.
,
“
A
c
o
m
pa
ri
son
of
s
ev
eral
s
tatisti
cal
m
et
hods
f
or
e
xamin
i
ng
al
leg
a
ti
on
s
of
c
o
pyi
ng
,
”
I
o
w
a
Cit
y
,
I
A
:
A
m
eric
an Co
l
leg
e
Testi
n
g
,
19
87. ACT
Research
Report
S
eries No. 87-1
5.
[14]
Harpp
D
.
N
.
,
et a
l
.
,
“Cri
m
e
i
n
the
cl
assroom
–
Pa
rt
II,
a
n
update,”
Jo
ur
na
l of
Che
m
i
c
al E
duca
t
i
on,
v
ol/issu
e:
73(4
)
,
pp
.
3
49-3
5
1
,
1
9
9
6.
[15]
Ho
ll
and
P.
W
.,
“
A
s
s
e
s
s
i
n
g
u
nusual
agreem
ent
b
e
tween
t
he
i
n
c
orre
ct
a
ns
wers
o
f
tw
o
ex
ami
n
ees
u
sing
t
he
K
-in
d
ex
:
st
ati
s
tical
t
heo
r
y
an
d
em
p
i
rical
s
u
p
p
o
rt
,”
P
rin
ceton
,
N
J
,
Edu
c
a
tio
na
l
Te
stin
g
Se
rv
ic
e
,
1
99
6
.
R
e
s
e
a
rc
h
Re
po
rt
N
o:
94
-4.
[16]
Hu
i
H.
F
.
,
“S
tabi
li
ty
a
n
d
s
en
si
tivit
y
o
f
a
m
o
d
e
l-bas
e
d
pers
on
-f
it
i
ndex
in
d
etect
ing
it
em
p
re-kno
wl
edge
i
n
com
p
uteri
zed
a
dap
t
i
v
e
tes
t
,”
D
i
s
s
e
rtation
Abstract
s
Int
e
rnat
ion
al
S
ecti
o
n
A
:
H
u
m
aniti
e
s
and
S
o
cial
S
ci
ence,
Un
iv
e
r
sit
y
o
f
Hong
K
o
n
g
,
2
01
0.
[17]
S
a
up
e
J
.
L
.
,
“A
n
em
piri
cal
m
od
e
l
f
o
r
t
he
c
orrob
o
rat
i
on
o
f
s
u
s
p
e
cted
c
heating
on
m
ultipl
e
-choi
ce
t
e
st
s,
”
Ed
uca
t
io
na
l a
nd Ps
ycho
l
ogica
l M
e
as
ur
ement,
vol
.
20
,
p
p
.
475
-4
8
9
,
19
60.
[18]
S
h
u
Z.,
“
D
et
ecting
t
e
st
c
h
eati
ng
u
s
i
ng
a
det
e
rminist
i
c,
g
a
t
ed
i
tem
resp
on
se
t
h
e
ory
m
o
del,
”
[Doctor
a
l
dis
s
ertation
]
,
The
Un
i
v
ersity
o
f
Nort
h
Car
o
l
i
n
a
a
t
Greensboro,
2
010.
Diss
e
r
t
a
t
ion
A
b
stracts
Intern
atio
nal
S
ect
ion
A:
H
u
m
aniti
e
s
and
S
o
ci
al S
ci
en
ces,
20
11
.
[19]
Sot
a
ridona
L
.
S.
a
nd
M
ei
jer
R.
R
.,
“S
tatistical
p
ropert
ies
of
t
he
K
-i
nd
e
x
f
o
r
d
ete
c
t
i
n
g
a
ns
wer
copy
in
g,
”
J
o
urna
l of
Ed
uca
t
io
na
l M
e
as
ur
e
m
ent
,
v
o
l
. 39
,
p
p.
115
-13
2
,
200
2.
[20]
S
o
t
a
r
i
d
o
n
a
L
.
S
.
a
n
d
M
e
i
j
e
r
R
.
R
.
,
“
T
w
o
n
e
w
s
t
a
t
i
s
t
i
c
s
t
o
d
e
t
e
c
t
an
swer
cop
y
i
ng,
”
Jo
urn
a
l
o
f
Ed
ucatio
nal
Me
asu
r
e
m
e
n
t
,
v
ol
.
4
0
,
pp
.
53
-69,
2
003
.
[21]
S
o
t
a
rido
na
L
.
S.
,
et a
l
.
,
“
D
et
ecti
n
g
ans
w
er
c
opy
in
g
us
ing
t
h
e
kapp
a
s
t
atist
i
c,”
Ap
plied Ps
ycho
l
ogica
l
M
e
as
ur
ement,
vo
l.
30,
p
p
.
4
12
-431
,
2
00
6.
[22]
van
d
e
r
L
i
n
d
en
W
.
J.
a
n
d
S
ot
ari
d
o
n
a
L.
S
.,
“
A
s
t
a
ti
st
ical
t
es
t
for
det
ecti
ng
ans
w
er
c
op
yin
g
o
n
m
u
lt
iple-cho
ice
te
st
s,”
Jou
r
n
a
l
o
f
Ed
uca
t
i
ona
l Meas
ur
ement
,
vo
l
.
41
, pp
.
3
61
-3
78
,
2
0
0
4
.
[23]
Wo
ll
a
c
k
J.
A
.
,
“
A
nomi
n
a
l
r
e
s
po
ns
e
m
o
del
ap
pro
a
c
h
t
o
d
e
tect
a
n
s
we
r
c
o
py
in
g,”
Applied Ps
ycho
l
og
ic
a
l
Me
asu
r
e
m
e
n
t
,
v
ol
.
2
1
,
pp
.
30
7-320
,
1
9
9
7
.
[24]
Bock
R
.
D
.
,
“E
st
i
m
a
ting
i
t
em
p
aramet
ers
and
l
a
t
e
nt
a
bi
lity
w
h
e
n
r
es
pon
ses
are
scored
i
n
two
o
r
m
o
r
e
nom
inal
categ
ories
,
”
Psycho
m
et
rika
,
vo
l
.
46
, pp
. 4
43
-4
59
, 1
97
2
.
[25]
Zo
pl
uoglu
C.
,
“Cl
a
ssif
i
catio
n
pe
rf
o
r
man
ce
o
f
a
n
swer-co
p
y
i
ng
i
n
d
ices
u
nder
d
i
f
f
e
rent
t
ype
s
o
f
ir
t
models,”
Appli
e
d
Ps
ychol
og
ical M
e
as
urem
ent
,
vol/is
su
e
:
4
0(8
)
,
pp.
5
92
-607
,
2
01
6.
[26]
Zo
pl
uoglu
C.
a
n
d
D
av
enpo
rt
E
.
C.,
“
T
he
e
m
p
irical
p
o
w
er
a
nd
t
yp
e
I
e
r
r
o
r
r
a
t
e
s
o
f
t
h
e
G
B
T
a
n
d
ω
i
n
d
i
c
e
s
i
n
det
ectin
g
ans
w
er
c
o
p
y
i
ng
o
n
mu
lt
iple-ch
o
ice
tes
t
s,
”
Ed
uca
t
i
onal a
n
d Psych
ol
og
ica
l
M
e
asurem
ent
,
v
ol
/issu
e:
72(6
)
,
pp
.
9
75-1
0
0
0
,
2
0
12.
[27]
As
sessm
ent Sy
s
t
em
s
,
“Us
er’s m
anua
l f
o
r
SIFT: Soft
ware f
or
i
nves
ti
gati
ng
f
raud
in
testi
ng,
”
2
016.
[28]
van
der
Li
nden
W
.
J
.
an
d
So
tarido
na
L
.
S
.
,
“Detecti
n
g
ans
w
er
c
o
py
in
g
w
h
en
t
h
e
r
eg
ular
r
esp
ons
e
p
r
ocess
f
o
llows
a
kn
ow
n
res
pon
se m
o
d
el
,
”
Jou
r
n
a
l of
Ed
uca
t
i
ona
l
a
n
d
B
e
h
a
vi
or
a
l
S
t
at
istics
,
v
o
l
.
31
, p
p.
2
8
3
-30
4
,
2006
.
[29]
Zo
pl
uoglu
C.
,
“Sim
il
arit
y,
a
ns
wer
cop
y
i
ng,
a
n
d
aberran
ce:
U
nd
e
r
stan
di
ng
t
he
s
t
a
tu
s
qu
o,
”
in
G
.
J.
C
i
z
e
k
a
nd
J
.
A
.
Wol
l
a
c
k,
“
Handbook
o
f
quan
titat
i
ve
m
et
hods
f
or
d
etect
ing
cheati
ng
on
t
e
s
t
s,”
New
Y
o
rk,
NY:
R
outledge,
p
p
.
2
5
-
46
,
2
017
.
[30]
Wol
l
a
c
k
J.
A
.,
“
C
omp
a
riso
n
o
f
a
nswe
r
c
o
p
y
in
g
i
n
d
i
c
e
s
w
i
t
h
re
a
l
dat
a
,
”
Jo
ur
na
l
o
f
E
d
u
c
ati
o
n
a
l
M
e
as
urem
ent
,
v
o
l
.
40
,
p
p.
1
89
-20
5
,
2
0
0
3
.
[31]
Wol
l
ack
J
.
A.
,
“S
imul
t
a
neous
use
of
m
ultiple
a
nswer
copying
ind
exes
t
o
im
p
r
o
v
e
det
ecti
o
n
rates
,
”
Appl
i
e
d
Me
asu
r
e
m
e
n
t
in
Ed
uc
a
tio
n,
vo
l
.
19
, pp
. 2
65
-2
88
, 2
00
6
.
[32]
Wo
llack
J
.
A
.
a
nd
C
oh
en
A
.
S
.
,
“
D
etect
io
n
of
a
n
s
w
e
r
copy
in
g
wit
h
un
kn
own
item
a
n
d
t
rait
p
aram
eters,
”
Appli
e
d
Ps
ychol
og
ical M
e
asu
r
em
ent
,
v
o
l
. 2
2, p
p
. 14
4
-1
52
, 19
9
8
.
[33]
Wol
l
a
c
k
J.
A
.,
et al
.
,
“D
efinin
g
erro
r
r
a
t
e
s
an
d
po
we
r
for
de
tec
t
io
n
of
a
n
s
we
r
c
o
p
y
i
ng
,”
Ap
plied Ps
ycho
l
og
ic
a
l
Me
asu
r
e
m
e
n
t
,
v
ol
.
2
5
,
pp
.
38
5–
4
0
4,
2
00
1.
[34]
S
u
n
b
u
l
O
.
and
Yo
rm
az
S
.
,
“
I
n
v
est
i
g
a
tin
g
th
e
p
e
rf
orm
a
nce
of
ω
i
n
dex
acco
rdi
n
g
to
item
param
e
ters
a
nd
a
b
ilit
y
levels,”
Eu
rasi
a
n
Jour
na
l
o
f
Educat
io
nal
R
e
sear
ch,
in p
r
ess
, 20
1
7
.
[35]
P
a
rtch
e
v
I. I.
,
“Sim
p
le i
nt
erf
ace
to
th
e
es
tim
ation an
d p
l
otti
n
g
o
f
I
RT m
od
els,” R pack
ag
e
versi
o
n
0
.
1.
7
,
2
014
.
[36]
Zo
pl
uoglu
C.
,
“Cop
yDet
ect:
Co
m
p
uting
s
t
a
t
istic
al
i
nd
i
ces
t
o
det
ect
a
n
s
w
e
r
cop
y
in
g
on
m
u
l
ti
p
l
e
-
ch
oi
ce
tes
t
s,”
R
pack
ag
e v
e
rsio
n 1
.
2
, 20
16.
Evaluation Warning : The document was created with Spire.PDF for Python.
I
S
S
N: 2
2
5
2
-
88
22
I
J
ERE
V
ol.
7,
N
o.
1
,
Mar
c
h 20
1
8
:
32
–
38
38
BIOGRAPHI
E
S
OF
AUT
HORS
On
der
S
unb
ul
i
s
an
A
ssistant
P
r
of
ess
o
r
in
M
ers
i
n
Uni
v
ersty
i
n
t
h
e
D
epart
m
ent
of
M
eas
urem
ent
and
Ev
aluat
i
o
n
i
n
E
d
u
catio
n.
H
e
reci
ved
his
M
S
c
and
P
h
D
deg
r
ee
f
r
om
D
epartm
en
t
of
M
easurem
en
t
and
Ev
alu
a
ti
on
i
n
E
du
cati
o
n
,
M
ers
i
n
U
n
iv
ersi
ty.
His
m
ai
n
in
terests
are
Ed
ucati
o
n
a
l
M
easurem
en
t,
C
o
m
p
u
t
a
tio
n
al
P
s
y
ch
om
etri
cs,
Tes
t
S
ecu
ri
ty,
It
e
m
R
esp
o
n
s
e
Th
eory,
and
S
t
atistical
P
rog
r
am
ming
.
S
e
haYo
rm
az
is
a
P
hD
s
tud
e
n
t
.
Sh
e
w
a
s
born
i
n
Mers
in
o
n
March
6,
1
98
3
.
S
he
obt
ain
e
d
her
BCs
deg
r
ee
(
2
005
)
i
n
A
n
k
ara
at
H
ace
t
t
epe
Univ
ersity
,
F
acu
lt
y
o
f
E
d
u
cati
o
n
,
a
nd
MS
c
deg
r
ee
(20
1
4
)
at
M
ersin
Univ
e
r
sity
,
D
e
partm
e
nt
o
f
M
eas
urem
ent
and
E
v
al
uati
on
in
E
d
u
c
a
t
io
n.
C
urr
e
ntly
s
h
e
i
s
a
P
h
D
st
uden
t
a
t
M
e
rsi
n
U
n
i
versity
,
Dep
a
rtm
e
n
t
o
f
M
easurem
en
t
a
n
d
E
valu
ati
on
in
E
d
u
cati
on
and
w
o
rk
s
as
a
s
cience
t
eac
h
e
r
in
M
ers
i
n,
T
u
r
key.
H
er
m
ain
i
n
t
e
r
e
s
t
s
ar
e
CT
T,
I
RT
,
R
pro
g
ram
m
in
g
and
ch
eati
ng.
Evaluation Warning : The document was created with Spire.PDF for Python.