User's Guide to the Composite System
The Composite program was written to perform the record keeping functions associated with classroom testing and student grade assignment. Up to 30 scores from tests, quizzes, projects and written assignments are stored in a unique instructor file managed by staff at the Evaluation and Examination Service (EES). The program provides instructors a flexible system for combining student evaluation components to form a final composite score. This program can be used with classes of any size but, is particularly helpful in classes with a large enrollment and/or multiple sections.
An instructor who would like to have a Composite file created for them must indicate this to the EES receptionist when the first set of answer sheets is submitted for scoring. At the same time that the first test analysis is run a master composite file is created that includes each students' name, ID, and test score. Each time an additional set of scores from a test or project is brought to EES the composite file is automatically updated, and one printed or email copy of the results is returned to the instructor. Scores from projects, written assignments, etc., must be given to EES on answer sheets (gridded scores) or through an alternate electronic medium. Additional copies of the composite can be provided for a duplication fee. If test scores are submitted prior to 3 p.m., the results are ready the following day after 9:30 a.m. Instructors must specify each time answer sheets are submitted how the grade components are to be combined.
Composite Program Options
The Composite program provides a number of options to address instructor needs. There are currently two versions of the Composite program; a short form and, a long form. The short form is provided after each set of answer sheets is scanned during the semester; the long form is printed after the last test component has been submitted and final grades are being calculated. The following information describes basic options which instructors may choose to include in their composite; standard features of the program are explained in the section describing the sample Composite printout that appears at the end of this bulletin.
Composite Program Sample
The computer printout that results from the Composite program will vary in its appearance depending on the options selected by the user. Some sections are routinely included in the program while others will appear only if requested. Each section of the printout is described below and illustrated in a sample that appears at the end of this bulletin. Those options marked with an asterisk do not appear in the short form but, are included in the long form for the final Composite.
Identification Information (p. 1) (sample)
This page includes descriptive information about the instructors' master file and the options that have been used to process the results for the most recent set of scores. Unique EES job numbers are assigned to each processing request and are shown on this page along with the instructor's name, course number, instructor selected options, and number of students included in the analysis. For example, the instructor for this sample has four separate score components in the Composite, has chosen to have the results reported by section, has indicated specific cut-off scores to assign student grades, and is using weights (1,1,2,4) to indicate the relative contribution of each score component to the Composite.
Missing Scores Roster (p. 2) (sample)
An alphabetical roster is printed of students who have one or more missing test scores. Scores are missing when students fail to take a test or when their ID number on the most recent test does not match an ID number in the master file. This same information is printed on p. 12 using the last five digits of the students' ID number. Some instructors post the ID roster and require students to monitor the list to verify corrections and note the need for make-up tests.
*Component Test Score Frequency Distribution Tables (p. 3) (sample)
The Composite program can accommodate up to 30 separate scores per course. The maximum composite score is 9999. The frequency distribution tables display raw scores for each test from high to low, the number of students obtaining each score, a cumulative frequency, and percentile rank.
Composite Score Frequency Distribution (p. 4) (sample)
A statement on this page indicates whether raw scores or converted scores were weighted to form the composite score. The weights by which the individual test scores were multiplied and the constants that were added appear on the identification (p. 1) and summary pages. The composite frequency distribution table shows the frequency of each composite score, cumulative frequency, percentile rank, and includes a histogram. If the histogram extends beyond two printed pages a proportional table is also printed which fits on one page.
Summary (p. 5) (sample)
The first table on this page gives descriptive information about each test and the composite score including: the mean, median, standard deviation, number of students in the analysis, weights, constants, and converted scores. The converted scores are the weighted raw or standard scores that are added together to form the composite score. If the composite score has been weighted or if a constant has been added to it, all statistics involving the composite score will be based on the adjusted composite scores.
The second table shown in the sample appears only if a grading option has been selected. The score ranges for each letter grade are shown, the percentile rank of each cut score, and the frequency distribution for the grades.
*The third table is a correlation matrix listing the correlation coefficients for each pair of component test scores and between each component and the composite score.
*A coefficient alpha reliability value is printed directly below the third table. Coefficient Alpha is a reliability estimate for the set of composite scores. It is interpreted in the same manner as the KR-20 reliability estimate provided in all test analysis output. (Users may refer to EES Technical Bulletin #17 for further information about using and interpreting the reliability estimate).
Name Rosters (p. 6,7,8) (sample)
This section of the printout includes an alphabetical roster of students either by section or total class. Section rosters can only be printed if students have gridded section numbers in the optional codes portion of their answer sheets. If the rosters-by-section option has been chosen, the first table lists students who misgridded their section numbers; the numbers were not coded completely or were coded with an invalid section number. The roster includes the student name, ID number, composite score, grade (if the grade option was selected), percentile rank of the composite score, and the converted and raw scores for each grade component. The table is followed by the mean, standard deviation, median and N for the composite.
ID Roster (p. 9,10,11) (sample)
The ID rosters provide all of the information found on the name roster with the exception of the students' name. On the ID roster only the last five digits of the social security number have been printed. The ID order is determined by the last five digits only.
Correction Page (p. 12,13,14) (sample)
Pages 12 & 13 lists students who appear to be missing one or more test scores; page 14 allows additional space for instructors to update student records. Instructors are given directions for correcting score problems and are then asked to sign and date the last page before handing it in to the EES receptionist. If the course has several sections, this list can be sorted by section or by alphabetical order. It is very important that the correction sheet be signed so that if a question arises concerning the authenticity of score changes, EES has a verified record. In order for score updates to appear on subsequent composites, corrections should be mailed or brought to EES prior to the submission of answer sheets for scoring. Poor student gridding of their ID numbers is the greatest problem with the composite system.
Using the Composite Program Options
Student grades typically are based on a number of components (e.g., exams, papers, projects, quizzes) and instructors often weight some components more heavily than others. Once it has been decided how much weight each grading component should have, the instructor should ensure that the composite score is actually formed with the desired weights. This task is not as simple as it first appears. An example of weighting will illustrate the potential problem.
When several sets of scores are combined to form a composite score, the amount of variability in each set is likely to be the most influential factor that determines how much weight each set will have in forming the composite. (The correlation between component scores also is a factor). Though the standard deviation is the most useful indicator of the amount of variability in a set of scores, the concept of weighting can be illustrated more easily by using the range, which is a somewhat grosser indicator of variability.
Suppose that an instructor wanted the final in a course to count twice as much as the midterm and that the range of scores on the midterm was 0-100 and the final exam range was 75-100. The midterm score variability is four times that of the final. Because there is so little variability on the final, the scores have very little weight in the total score. The net effect is like adding a constant value to each student's midterm score; the students maintain essentially the same relative standing that they had on the midterm. By doubling the final exam score, the range changes to 150-200, still only half that of the range for the midterm scores. To achieve the desired weights, the final exam scores would need to be multiplied by eight to yield a final exam score range that is double the range of the midterm scores.
To avoid the problems described in the example, instructors are strongly encouraged to use standard scores to represent adjusted raw scores in forming a weighted composite. The Composite program converts raw scores to T-scores which have a mean of 50 and a standard deviation of 10. The function of the transformation is to give scores a constant meaning across all different sets of grade components for the same group of students. The range of the standard scores for each test in a given class will be approximately the same each time. The desired scoring weights can be applied directly to the standard scores in forming a composite that reflects the desired weights.
The composite program provides an option for eliminating one or more of each student's lowest scores prior to forming the composite score. A missing score is considered to be a score of 0 for elimination purposes. The instructor specifies the number of scores per student that may be dropped as well as the specific tests from which scores can be eliminated. In the simplest case, the program examines all scores for each student and drops the lowest score no matter which test it represents.
Instructors who use this option reason that anyone can have a bad day and, therefore, perform below their actual level of competence. They feel that eliminating the lowest score will yield more accurate information about the students' "true" levels of achievement.
There are several reasons why eliminating at least one test score is not sound educational practice. First, measurement errors that account for students performing better or worse than they actually should are random. When several measures of achievement from different occasions are combined, the random errors from the different tests should cancel one another. That is, the inflated and deflated scores should yield a total or composite score which is a much more accurate measure of true performance than any one component measure. If the lowest score in a set is eliminated, the composite score for each student would tend to represent a somewhat higher level of performance than is actually the case.
Second, the more information available for making a decision, the more accurate that decision is likely to be. Instructors who labor to build tests that provide useful information about the differential achievement levels of their students should not be willing to throw such useful information away. Just as a 50-item test will help to discriminate better between high and low achievers than a 30-item test, five test scores will be more useful for assigning grades accurately than four test scores.
Third, the elimination of the lowest test score likely will create composite scores that are not comparable. For example, some composite scores will be based on tests 1-3, some on 2-4, some on 1-2 and 4, etc. Such composites do not represent measures of achievement of the same instructional outcomes. This circumstance is parallel to the use of optional essay questions on a test. When students respond to different sets of questions, their scores reflect achievement of different sets of instructional objectives. The lack of comparability may necessitate using a different set of grading standards for each possible combination of components so that the difference in grading criteria will not result in unfair grading practices for some students.
Course grades can be assigned to students on the basis of the composite scores using one of two optional methods. With the cut-off score method, the instructor provides the minimum composite score required for each letter grade (A - F, or A+ - F). Each student's grade is reported on the alphabetical and ID rosters with the composite score. With the percentage method, the instructor indicates the percentage of the class that should be assigned each letter grade. This procedure is what many refer to as grading-on-the-curve.
Both of these methods have advantages and disadvantages that should be weighed very carefully. Instructors who would like a reference to help in establishing grading policies should refer to EES Technical Bulletin #5; Assigning Course Grades.
Questions concerning the Composite program should be directed to EES scoring room personnel at 335-0356.