Eighteen months ago, I wrote an op-ed arguing that the revised SAT essay, which appeared on tests last week, is a superficial exercise that encourages students to write formulaically and adds little to a college application. Today, the good news is that fewer colleges are requiring students to submit an SAT or ACT essay score. The bad news for those that do is that the redesigned essays may be even worse than their predecessors.
Less than 9 percent of colleges require the SAT essay, and a similar share will require the ACT essay next year. This handful of colleges that continues to have faith in the essay tests should have lost it this past fall, when the ACT essay had a terrible rollout. Difficulties getting the essays graded delayed the delivery of scores in time for students hoping to be admitted through early-decision and early-action programs. As The Washington Post recently recounted, once the scores arrived, the news got worse. Scores, which on all ACT sections range from 1 to 36, came in surprisingly low for the essay. Students scoring in the 30s on the other sections were receiving scores in the low 20s on the essay.
In January, ACT issued a report intended to assuage concerns over the essay, but it ended up confirming them. The report acknowledges that “students with extremely high ACT scores in [other sections] may be receiving noticeably lower scores in writing.” For many students, this discrepancy created serious concerns that their lower essay scores would harm their chances of getting into highly competitive schools.
In the report, ACT admits that essay scores are indeed less reliable than scores on multiple-choice sections. “Every test score includes some level of imprecision,” the authors write, which is why test makers calculate the standard error of measurement, or SEM, “a metric for reporting the extent that a typical observed score varies from the true score.” If a student takes a test multiple times, without doing anything between tests to increase her test skills, the SEM predicts the range between which her score will vary on additional tests.
For example, the SEM for the composite score is about 1, so a student who gets a 33 and takes the test multiple times has a two-out-of-three chance that her scores will fall between 32 and 34. The essay, in contrast, has an SEM of 4, so a “score of 20 on ACT Writing would indicate that there is a two-out-of-three chance that the student’s true score would be between 16 and 24.” That’s the difference between the 44th and the 88th percentile. With such a wide range, how can colleges rely on the ACT essay to help them distinguish strong applicants from weak?
Too bad for this year’s seniors who had to live with their scores while the graders learned how to do their jobs.
Perhaps most damning of all is what happened in the aftermath of the release of the scores. As the Washington Post story explains, some students paid $50 to get their essays rescored and saw drastic improvements, including one student whose essay went from a 19 (63rd percentile) to a 31 (98th percentile). A change like that indicates serious problems with the graders. All the ACT could muster as a defense in its report was the promise that “as raters become more familiar and experienced in scoring with the new domain-based rubrics, these issues will be mitigated.” Too bad for this year’s seniors who had to live with their scores while the graders learned how to do their jobs.
The irony of the ACT essay fiasco is that the new scoring on the ACT and SAT essays is supposed to make the scores more objective and standardized. In September, the ACT switched from holistic scoring, which assigns a single score to the whole work, to an analytic rubric, which assigns individual scores to several writing competencies. The SAT will do the same this month. The switch brings the tests in line with what became the dominant practice in elementary and secondary schools after No Child Left Behind and other legislation put assessments at the center of the curriculum. The two main assessments directly tied to the Common Core, the PARCC and the Smarter Balanced exam, both use four-point rubrics, as will the SAT.
With only four points to distinguish students’ ability to read, analyze, and write, colleges are unlikely to gain little useful information about applicants from the SAT essay.
That four-point scale, applied to the three domains of reading, analysis, and writing, could make the SAT essay even more problematic than the ACT essay, which uses a six-point scale. With only four points to distinguish students’ ability to read, analyze, and write, colleges are unlikely to gain little useful information about applicants from the SAT essay. The College Board, which is already on record that “one single essay historically has not contributed significantly to the overall predictive power of the exam,” has so little faith in the value of the essay for colleges that, in addition to not converting the essay scores into the 200-800 scale of the rest of the exam, it will not even add the three subscores together.
In all likelihood, this grading switch is not for colleges at all. The College Board and ACT are campaigning to have their exams replace existing state assessments tied to the Common Core, as has happened in New Hampshire, Connecticut, Illinois, and elsewhere. The new essays are designed to allow teachers and school districts to adapt them quickly and clearly to the work they and their students are already doing, which is just another reason for colleges to disregard the essay.
Standardized-test essays will disappear only when enough colleges reject them. You might think that over 90 percent of colleges saying no to the essay would be sufficient to kill it, but the holdouts, including Harvard and the University of California at Berkeley, have a great deal of influence. The essay requirement at these and other colleges was born, I suspect, of a genuine belief in the importance of writing and an understandable lack of knowledge about the essays and their scoring, but neither a noble principle nor ignorance should be a helpmate to bad tests. The time has come for all colleges to say no to the essay.
