The National Spanish Exam 2020

Reliability

NSE recognizes that test reliability is defined as the degree to which the test gives consistent results each time it is given.

In other words, reliability answers the following questions:

  1. Can I depend on the test to measure the same outcomes consistently?
  2. Given all the other variables being the same, will the test produce the same results again?

NSE uses the Kuder-Richardson 21 formula to calculate reliability coefficients:

r (reliability) = (K)(SD2)-M(K-M)⁄(SD2)(K-1)

K = the number of items in the list
SD = the standard deviation of the scores
M = the mean of the scores

The scores of reliability are judged against a perfect score of 1.00. The closer the reliability coefficient is to 1.00, the better it is. Most standardized tests usually have a reliability coefficient of .90 or above.1

Reliability Coefficients

for the 2020 National Spanish Exam

Level Pre 01 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=521)
  Mean Raw Score 75.4 0.0 154.4 39.0 40.0 0.0 154.4
  Standard Deviation 16.4 0.0 30.6 10.5 9.3 0.0 30.6
  Reliability coefficient 0.941 ? 0.967 0.792 0.731 ? 0.901
Outside Experience (N=114)
  Mean Raw Score 81.5 0.0 166.6 42.5 42.6 0.0 166.6
  Standard Deviation 13.4 0.0 24.6 8.5 7.6 0.0 24.6
  Reliability coefficient 0.925 ? 0.959 0.666 0.587 ? 0.841
Bilingual (N=19)
  Mean Raw Score 88.9 0.0 176.0 43.2 43.9 0.0 176.0
  Standard Deviation 18.1 0.0 37.7 10.1 11.0 0.0 37.7
  Reliability coefficient 0.980 ? 0.990 0.765 0.805 ? 0.933
Level 01 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=3404)
  Mean Raw Score 71.5 64.9 136.4 70.9 78.4 149.3 285.8
  Standard Deviation 19.2 20.2 36.7 22.3 22.4 41.9 70.9
  Reliability coefficient 0.954 0.954 0.973 0.968 0.976 0.983 0.986
Outside Experience (N=906)
  Mean Raw Score 76.6 69.7 146.3 75.8 84.6 160.3 306.6
  Standard Deviation 18.3 19.1 35.2 20.6 20.0 37.9 66.3
  Reliability coefficient 0.956 0.952 0.973 0.966 0.977 0.983 0.986
Bilingual (N=211)
  Mean Raw Score 89.6 76.7 166.3 83.8 89.0 172.8 339.1
  Standard Deviation 13.4 15.9 26.9 19.7 20.6 38.5 58.1
  Reliability coefficient 0.957 0.939 0.966 0.975 0.987 0.989 0.987
Level 1 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=15943)
  Mean Raw Score 72.6 56.7 129.4 70.2 68.3 138.6 267.9
  Standard Deviation 20.3 20.6 38.1 23.2 24.3 44.6 75.7
  Reliability coefficient 0.962 0.952 0.973 0.971 0.973 0.983 0.987
Outside Experience (N=2695)
  Mean Raw Score 79.5 62.5 142.0 75.1 75.1 150.2 292.2
  Standard Deviation 18.0 19.9 35.1 23.1 24.4 45.0 72.7
  Reliability coefficient 0.959 0.951 0.971 0.975 0.978 0.986 0.988
Bilingual (N=1118)
  Mean Raw Score 88.5 71.0 159.5 76.8 81.0 157.9 317.4
  Standard Deviation 16.0 19.6 33.2 25.7 27.3 51.2 73.2
  Reliability coefficient 0.970 0.956 0.976 0.983 0.989 0.992 0.990
Level 2 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=17955)
  Mean Raw Score 71.4 46.5 117.9 68.9 73.8 142.6 260.5
  Standard Deviation 21.8 20.6 39.0 22.8 24.0 43.9 75.6
  Reliability coefficient 0.967 0.951 0.973 0.969 0.976 0.984 0.987
Outside Experience (N=1126)
  Mean Raw Score 81.2 53.3 134.5 71.6 79.1 150.7 285.2
  Standard Deviation 20.4 22.1 39.0 24.6 25.9 47.8 76.5
  Reliability coefficient 0.973 0.959 0.976 0.976 0.985 0.989 0.988
Bilingual (N=2046)
  Mean Raw Score 92.0 65.6 157.6 74.5 85.6 160.0 317.6
  Standard Deviation 13.2 19.6 29.6 24.9 24.9 47.4 65.5
  Reliability coefficient 0.968 0.951 0.967 0.979 0.990 0.991 0.987
Level 3 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=15916)
  Mean Raw Score 69.3 53.2 122.5 66.5 70.4 137.0 259.5
  Standard Deviation 21.3 19.3 36.5 18.7 22.2 37.3 66.2
  Reliability coefficient 0.963 0.943 0.969 0.946 0.967 0.974 0.982
Outside Experience (N=1135)
  Mean Raw Score 79.9 57.6 137.6 67.2 73.6 140.9 278.4
  Standard Deviation 20.6 20.0 36.5 19.3 22.8 38.5 64.0
  Reliability coefficient 0.972 0.948 0.973 0.950 0.972 0.977 0.982
Bilingual (N=2278)
  Mean Raw Score 92.0 69.3 161.3 68.1 75.5 143.6 304.9
  Standard Deviation 12.9 18.0 27.8 21.2 23.2 41.6 58.1
  Reliability coefficient 0.966 0.944 0.964 0.961 0.975 0.982 0.981
Level 4 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=8998)
  Mean Raw Score 69.4 48.7 118.0 59.2 60.7 119.9 237.9
  Standard Deviation 19.6 21.4 36.8 20.5 21.3 38.0 68.6
  Reliability coefficient 0.954 0.955 0.969 0.952 0.957 0.972 0.982
Outside Experience (N=606)
  Mean Raw Score 79.5 52.1 131.6 64.2 66.0 130.2 261.8
  Standard Deviation 18.0 20.2 33.6 20.0 20.4 36.7 62.1
  Reliability coefficient 0.959 0.948 0.965 0.952 0.955 0.971 0.979
Bilingual (N=2055)
  Mean Raw Score 86.6 58.7 145.3 67.2 67.0 134.2 279.5
  Standard Deviation 13.7 18.6 28.4 20.8 19.8 37.3 55.8
  Reliability coefficient 0.948 0.939 0.955 0.958 0.953 0.973 0.975
Level 5 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=3109)
  Mean Raw Score 68.5 53.4 121.8 71.5 73.3 144.8 266.7
  Standard Deviation 20.5 20.8 37.5 21.1 21.7 39.5 70.4
  Reliability coefficient 0.958 0.952 0.971 0.964 0.968 0.979 0.985
Outside Experience (N=266)
  Mean Raw Score 79.4 58.0 137.4 74.1 76.8 150.9 288.3
  Standard Deviation 17.4 19.3 32.7 21.2 21.7 39.9 63.5
  Reliability coefficient 0.956 0.944 0.965 0.967 0.972 0.982 0.982
Bilingual (N=871)
  Mean Raw Score 85.2 69.2 154.4 75.8 79.4 155.2 309.6
  Standard Deviation 15.5 20.0 32.1 19.9 19.6 36.1 58.7
  Reliability coefficient 0.957 0.956 0.971 0.963 0.967 0.978 0.982
Level 6 Vocabulary Grammar Achievement Reading Listening Proficiency Total
Classroom Experience (N=285)
  Mean Raw Score 69.8 57.0 126.8 72.1 66.1 138.2 265.0
  Standard Deviation 21.4 22.6 40.9 20.0 24.0 40.1 76.2
  Reliability coefficient 0.963 0.961 0.977 0.959 0.971 0.978 0.987
Outside Experience (N=56)
  Mean Raw Score 78.8 59.2 138.0 74.1 75.8 149.9 288.0
  Standard Deviation 17.9 18.2 31.1 16.1 19.4 32.5 57.6
  Reliability coefficient 0.957 0.937 0.961 0.935 0.961 0.969 0.978
Bilingual (N=257)
  Mean Raw Score 85.1 73.2 158.3 73.7 75.5 149.2 307.6
  Standard Deviation 14.7 17.2 28.5 20.0 22.3 39.1 58.5
  Reliability coefficient 0.951 0.943 0.964 0.961 0.972 0.980 0.982

1John A. Kaufhold, Basic Statistics for Educational Research (New York: iUniverse, Inc., 2007) pp. 43-46.