Chariot Learning
  • About
    • FAQ
    • Our Curriculum
    • Areas Chariot Learning Serves
  • Services
    • SAT & ACT Prep
    • SAT/ACT TestFlip
    • Academic Coaching
    • Subject Tutoring
    • Strategic Reading Club
    • Proctored Practice Tests
    • GRE Prep
    • College Essay Coaching
    • Roots2Words
  • Calendar
  • Testimonials
  • Blog
  • Contact
July 30, 2020 by Mike Bergin

Anatomy of a Test Question

How is a test like a duck? They may appear unruffled and serene on the surface, but underneath they are usually paddling frantically. Great tests, meaning those designed to be valid, fair, and reliable, require tremendous effort and insight to put together. Casual observers may see a random assemblage of items, but those in the know can glimpse the many hands–not to mention reams of data and decades of experience–that go into crafting entire tests, specific sections, and even individual questions for standardized exams like the SAT and ACT.

Basically, assessment design goes very, very deep. If you are going to spend weeks, months, or perhaps even years of your life analyzing test questions, some insight into testing terminology can’t hurt.

ITEM
A test question–both the problem itself and any answer choices–is called an item. Tests can feature all sorts of objective and subjective items. The standardized tests used for admissions predominantly employ multiple choice items.

STEM
The problem posed by a test question is called the stem. An effective stem should present a definite problem in the form of a question or a partial sentence.

ALTERNATIVES
In a multiple choice test item, the stem is followed by a list of answer choices, known as alternatives or options. Some testing taxonomies also refer to alternatives as the foil, though that term can also be applied strictly to wrong answers. Ideally, alternatives will be presented clearly and concisely.

KEY
The right answer choice is called the key. Sometimes, the key will be the best possible answers. In instances where facts, formulas, or equations are tested, the key will be the only correct answer.

DISTRACTORS
Every choice apart from the key is considered a distractor or, sometimes, foil. These incorrect or inferior alternatives should all be plausible to some degree but not so similar to the right response that a reasonable case can be made for multiple choices.

Tests like the SAT and ACT follow many of the fundamentals of effective item design, except in those instances where deviations from best practices add complexity to the testing process. For example, one guideline suggests that a stem should be negatively stated only when significant learning outcomes require it. The SAT and ACT are not learning tests but rather assessments. These exams often include phrased negatively stems in order to increase difficulty or add extra time to the solution process. Even the inclusion of four or even five alternatives when research indicates that three choices will often do illustrates how norm-referenced assessment design might deviate from that of criterion-referenced assessments.

A brilliant test item is a thing of complex beauty, even if most test takers lack either the time or inclination to appreciate its intricate design. Don’t let the tiny print and thin paper fool you. First, paper-and-pencil tests may not be around much longer anyway, if remote proctoring manages to earn the confidence of admissions offices. Second and more important, though, well-crafted test items are labors of love (ask a true test architect) floated by lots and lots of research and money. You may not like a test, but when you recognize a truly well-designed one, you can’t help but respect it.

question difficulty standardized tests

Previous articleFor All Intensive PurposesNext article The Most Important Vocabulary Words for the SAT & ACT
Mike Bergin
Tens of thousands of students a year prep for the SAT & ACT through programs Mike Bergin created or organized. After more than 25 years of intensive experience in the education industry, he's done it all as a teacher, tutor, director, curriculum developer, blogger, podcaster, and best-selling author. Mike founded Chariot Learning in 2009 to deliver on the promise of what truly transformative individualized education can and should be.

Leave a Reply

Your email address will not be published. Required fields are marked *

Helping high schoolers with tests, school, and life is what Chariot Learning is all about. Let us know how we can help you!

Latest Posts

  • When Should You Take The SAT or ACT?
  • Growing That Growth Mindset
  • How Fair is Test Optional?
  • Why Reading Skills Matter
  • Sal Khan on Testing, Admissions, and Equity

What is TestFlip?

Learn about the ultimate in self-directed supplemental SAT/ACT prep, powered by Chariot Learning:

  • Instead of making the same mistakes over and over, try to make some different ones!
    by chariotlearning 2 days ago
    Instead of making the same mistakes over and over, try to make some different ones!
  • Why do you write?
    by chariotlearning 6 hours ago
    Why do you write?
  • What kind of reader are you? http://ow.ly/14g050J1CP0
    by chariotlearning 2 weeks ago
    What kind of reader are you? http://ow.ly/14g050J1CP0
  • How you respond to failure will either make you or break you.
    by chariotlearning 5 days ago
    How you respond to failure will either make you or break you.
  • Keep making progress!
    by chariotlearning 7 days ago
    Keep making progress!
  • The world is changed by your example, not your opinion.
    by chariotlearning 2 weeks ago
    The world is changed by your example, not your opinion.
  • Maybe life isn
    by chariotlearning 2 weeks ago
    Maybe life isn't about avoiding the bruises...
  • The most effective way to do it is to do it!
    by chariotlearning 1 week ago
    The most effective way to do it is to do it!

Subscribe to Mike’s Podcast

Subscribe to Tests and the Rest

PARENT GROUP AND NEWSLETTER

Are you an Upstate New York parent with questions about college admissions, testing, and scholarships? The Upstate NY College Conversations Facebook group is for you. Join today!

And also sign up for College Road, our free email newsletter delivering expert educational advice, test news, admissions action steps, special offers, and more.

Upcoming Events

  1. Free Practice SAT (5/29/22)

    May 29 @ 10:00 am - 2:00 pm
  2. June SAT (Official Test)

    June 4 @ 8:00 am - 1:00 pm
  3. Free Practice ACT (6/5/22)

    June 5 @ 10:00 am - 2:00 pm
  4. SAT/ACT Online Flipped Class – June 2022

    June 8 @ 6:00 pm - 8:00 pm
  5. June ACT (Official Test)

    June 11 @ 8:00 am - 1:00 pm

View All Events

Testimonials

We highly recommend... We considered classroom instruction given by national programs, but having one-on-one instruction, on our schedule, in our home, with the focus on areas and skills critical to our daughter’s success was important to us. Teaming up with Chariot Learning was definitely the right decision for our daughter and my family.

Eileen, Rochester mom

Read more testimonials...
© 2022 Chariot Learning, LLC. All rights reserved.