Zen Buddhism famously employs paradoxical riddles to demonstrate the inadequacy of logical reasoning and to provoke enlightenment. I’m sure the monks would have a field day with this koan describing the current state of the SAT: When is a standardized test not standardized?
You don’t need to have achieved nirvana to guess the answer to this one. The SAT stops being truly standardized when different forms of the test are administered without clarity on which questions actually count. In a sense, the SAT lost its claim to standardization a while ago.
The December 2018 SAT definitely delivered that unfortunate point to an entire cohort of testers. Not only did the test seemed easier than practice tests, which had been the disappointing trend in 2018, but most test takers under standard group testing conditions who skipped the optional essay section–which, these days, is most of them–faced a mysterious fifth section of multiple choice questions. Section 5 lasted 20 minutes for everyone, but each test taker could see content from any of the other four test sections.
Section 5 of the SAT is widely believed to be an experimental section, an opportunity for the College Board to pretest questions with a captive audience under test conditions. However, the test maker has been unusually and persistently mysterious about what should be a transparent aspect of the exam. Back when this version of the SAT was released, College Board shared (or, rather, buried in very fine print) the following guidance:
The SAT will be given in a standard testing room (to students with no testing accommodations) and consist of four components — five if the optional 50-minute Essay is taken — with each component timed separately. The timed portion of the SAT with Essay (excluding breaks) is three hours and 50 minutes. To allow for pretesting, some students taking the SAT with no Essay will take a fifth, 20-minute section. Any section of the SAT may contain both operational and pretest items.
However, the March 2019 SAT introduced a new wrinkle to this drama. High schoolers in my area who sat for the Essay on the March SAT still faced that fifth 20-minute section. Imagine how freaked out, not to mention exhausted, these students were to be administered an unexpected section that may or may not have counted! Who knew that the policy had changed? If you carefully combed the Spring and Summer 2019 The SAT and SAT Subject Tests Supervisor Manual, you’d have found the following:
At some centers, certain administrations will include an additional 20-minute section to be completed by all SAT test takers, including students taking the SAT with Essay. The
Standard Testing Room Manual informs associate supervisors precisely when to administer it.
Since the inception of the current version of the SAT, the College Board rebuffed efforts to gain clarity on this fifth section. The general consensus accepted Section 5 as a purely experimental section, with no bearing on a student’s test scores. Some have hypothesized ancillary uses for this section, such as testing to make sure a student isn’t cheating, but nobody–including the test maker–has addressed the elephant in the room: “Any section of the SAT may contain both operational and pretest items.”
In the absence of guidance from the test maker, we can assume that operational items contribute to a score while pretest items serve as experimental material that does not contribute to a score. Interspersing experimental questions throughout the test makes far more sense for statistical validity than informing testers that certain questions have no bearing on test scores. The last iteration of the SAT executed that requirement well: Out of the ten test sections, one extra Math, Reading, or Writing section was mixed in with the rest but would not contribute to the official score. Since students had no real way to identify the experimental section–not for lack of trying–they were forced to act as if all test questions counted.
Contrast the effectiveness of that approach to one where test takers are informed that a certain block of questions has no bearing on the official score. No wonder students routinely, by their own accounts, blow off that last, presumably meaningless test section. Yet, SAT Section 5 may not be as innocuous as widely believed. Some of our students reported that their proctors at the December 2018 SAT informed their groups that some of the questions on Section 5 do count towards the official score. Specifically, a certain number of questions on that section would count, while some questions from the previous section covering the same content would not.
Either these proctors were ill-informed, which happens far too often, or someone finally shared critical information that College Board has failed to clarify. Both possibilities, unfortunately, lead to the same crisis of confidence in the validity of this test.
Does SAT Section 5 count in any way towards official scores? College Board refuses to directly answer that question. The most recent statement from a College Board rep does nothing to clarify the situation:
The SAT (as of March 2016) and SAT with Essay (as of March 2019) contain some questions that won’t be used to compute student scores. These questions may appear in any section. To give students the extra time to answer more questions, the tests include a fifth section with regular and pretest questions.
Until we all receive the definitive explanation that any competent, credible organization would provide about such an influential standardized exam, test takers would be wise to take Section 5 of the SAT as seriously as they do the first four sections. Play to the final whistle to ensure that every question that might contribute to your score receives your best focus, commitment, and effort.
(This article was first published in December 2018 and revised in March 2019.)