National Portuguese Exam (2024)

Reliability

npe recognizes that test reliability is defined as the degree to which the test gives consistent results each time it is given.

In other words, reliability answers the following questions:

  1. Can I depend on the test to measure the same outcomes consistently?
  2. Given all the other variables being the same, will the test produce the same results again?

npe uses the Kuder-Richardson 21 formula to calculate reliability coefficients:

r (reliability) = (K)(SD2)-M(K-M)⁄(SD2)(K-1)

K = the number of items in the list
SD = the standard deviation of the scores
M = the mean of the scores

The scores of reliability are judged against a perfect score of 1.00. The closer the reliability coefficient is to 1.00, the better it is. The relationship between variables is commonly considered strong over .7. Most standardized tests usually have a reliability coefficient of .90 or above.1

Reliability Coefficients

for the National Portuguese Exam (2024)

Population Size Number of Questions (K) Mean Score (M) Standard Deviation (SD) Reliability Coefficient (r)
Level Name Category Name Exam Part
Level 1 Bilingual Grammar 11 20 15.545455 2.871922 0.610748
Listening 11 11 24.636364 0.771389 1.000000
Reading 11 14 31.909090 2.314167 1.000000
Vocabulary 11 20 17.727272 2.135880 0.587814
Total 11 65 89.818184 6.492208 1.000000
Classroom Experience Grammar 85 20 12.717647 3.774876 0.710558
Listening 85 11 20.376471 4.286714 1.000000
Reading 84 14 27.035715 5.733042 1.000000
Vocabulary 85 20 14.235294 3.383862 0.675438
Total 85 65 74.047058 15.352261 1.000000
Outside Experience Grammar 17 20 14.294118 2.844286 0.522016
Listening 16 11 23.125000 2.260393 1.000000
Reading 17 14 28.823530 4.475230 1.000000
Vocabulary 17 20 16.588236 2.498442 0.575446
Total 17 65 81.470589 12.645006 1.000000
Level 2 Bilingual Grammar 31 20 16.322580 2.798468 0.649230
Listening 31 11 23.483871 2.226975 1.000000
Reading 31 14 28.451612 5.977586 1.000000
Vocabulary 31 20 17.000000 2.016064 0.392230
Total 31 65 85.258064 10.140840 1.000000
Classroom Experience Grammar 110 20 12.954545 3.783664 0.717085
Listening 109 11 18.889908 4.741655 1.000000
Reading 109 14 22.513762 7.693218 1.000000
Vocabulary 110 20 14.309091 3.891121 0.769564
Total 110 65 68.290909 18.082312 1.000000
Outside Experience Grammar 26 20 14.538462 3.565117 0.723831
Listening 27 11 21.074074 3.516373 1.000000
Reading 27 14 25.888889 6.442528 1.000000
Vocabulary 26 20 15.230769 3.905693 0.802009
Total 27 65 75.629631 16.559988 1.000000
Level 3 Bilingual Grammar 17 20 14.176471 3.347260 0.664819
Listening 17 11 22.000000 1.533930 1.000000
Reading 17 14 24.882353 4.873486 1.000000
Vocabulary 17 20 17.411764 2.030903 0.477570
Total 17 65 78.470589 9.017093 1.000000
Classroom Experience Grammar 62 20 10.564516 2.797585 0.382294
Listening 63 11 17.825397 4.725949 1.000000
Reading 62 14 21.403225 6.880219 1.000000
Vocabulary 62 20 13.129032 3.735116 0.712311
Total 63 65 62.206348 16.481022 1.000000
Outside Experience Grammar 17 20 11.647058 2.471988 0.214699
Listening 17 11 20.941177 2.979747 1.000000
Reading 17 14 24.764706 4.557210 1.000000
Vocabulary 17 20 14.470589 3.145822 0.627089
Total 17 65 71.823532 9.382307 1.000000
Level 4 Bilingual Grammar 11 20 13.454545 3.473631 0.668493
Listening 11 11 19.090910 6.748125 1.000000
Reading 11 14 24.454546 7.439008 1.000000
Vocabulary 11 20 14.818182 4.018552 0.802376
Total 11 65 71.818184 18.770472 1.000000
Classroom Experience Grammar 55 20 8.709091 3.398785 0.604608
Listening 55 11 12.545455 5.429945 1.000000
Reading 55 14 17.927273 6.741606 1.000000
Vocabulary 55 20 10.454545 4.142104 0.746501
Total 55 65 49.636364 16.348883 0.971045
Outside Experience Grammar 4 20 12.000000 2.549510 0.275304
Listening 4 11 20.500000 3.840573 1.000000
Reading 4 14 23.750000 8.982622 1.000000
Vocabulary 4 20 15.750000 2.772634 0.594352
Total 4 65 72.000000 17.677670 1.000000

1John A. Kaufhold, Basic Statistics for Educational Research (New York: iUniverse, Inc., 2007) pp. 43-46.