Assessment-oriented instruction, as I like to call test prep when I’m feeling particularly bombastic, usually follows utilitarian principles. Basically, if something works, keep doing it. Thanks to rich quantitative feedback loops, we can track in real time what allows either an individual student or an entire cohort to more quickly and accurately solve different types of problems.
Couple the emphatic pragmatism of test prep with the fact that many practitioners have backgrounds in fields far outside of education and it’s no wonder that theoretical frameworks are rarely primary considerations in tutoring sessions. Nonetheless, educational professionals can learn a lot from educational theory and models, which is why I recently asked expert Erik Francis of Maverik Education to teach a group of test prep teachers about Depth of Knowledge levels.
As far as theoretical educational frameworks go, Depth of Knowledge certainly sounds rigorous. Cooked up by Dr. Norman Webb in 1997 to facilitate the alignment of standards to assessments, the DOK model categorizes activities according to the level of complexity in thinking. Depth of knowledge is not a proxy for difficulty. Instead, a DOK level describes the kind of thinking required by a task separate from perceived difficulty.
DOK 1 – RECALL
DOK level 1 items require either basic recall or a rote operation. Look for requirements to restate or reproduce.
DOK 2 – SKILLS AND CONCEPTS
DOK level 2 items require the use of concepts, skills, and basic reasoning, often involving two or more steps.
DOK 3 – STRATEGIC THINKING
DOK level 3 items require complex reasoning supported by evidence, often allowing more than one answer. Expect to have to develop a plan or follow a sequence of steps.
DOK 4 – EXTENDED THINKING
DOK level 4 items require investigation, comparison, and synthesis taking more than ten minutes.
Assigning an accurate DOK level to a task or test item is more of an art than a science, especially because different activities can be structured to address a range of levels. However, savvy educators can find value in understanding how students deal with increasing levels of cognitive demand.
Exams like the SAT and ACT do not include DOK levels in score reports, but review of the different question types on each section reveals some persistent trends. Most items are either DOK level 2 or 3 with a smattering of DOK 1 items throughout. DOK 4 cannot really be assessed through the kind of multiple-choice questions favored by most admissions tests. Also, now that the SAT and ACT are used as state assessments across the US, the test design teams absolutely incorporate DOK allocations in their internal blueprints to meet varying state standards. This probably explains why the latest revision of the SAT included new question types like Linked Evidence questions on the Reading section: these items are almost certainly DOK 3. Furthermore, math word problems are, just by nature of the requirement to translate English to math, usually at least DOK 2.
If Depth of Knowledge identifies the context and cognitive requirements of a task, effective test prep can be seen as a mechanism that helps lower levels of cognitive demand on high stakes exams by tackling certain challenges in advance. Real preparation is all about providing students rich opportunities to, for example, build deep conceptual understanding and master heuristics or strategic sequences to predictable problems ahead of time. Even better, once these students encounter reviewed problems in a test setting, they can lean on lower level DOK requirements of recall and application rather than having to derive novel solutions to unoriginal problems.
Clearly, insight into the concept of Depth of Knowledge opens doors to better teaching and learning. Even the most practical practitioners can find value in this theoretical framework, especially when designing deep learning in advance of testing to encode automaticity in sophisticated sequences and solutions. Effective test prep leads to a reduction of cognitive load during a high stakes test, which in turn facilitates better performance and higher scores.
Mike Bergin