Depth of knowledge.
It’s a concept we in education have heard a lot about since our states transitioned to college and career ready standards, be it the Common Core State Standards, the Next Generation Science Standards, or whatever each individual state has decided to call their academic standards that promote college and career readiness.
It’s also one of the most misinterpreted and misrepresented concepts in education that is not only confusing but also frustrating us educators. Most of it is due to the infamous DOK Wheel.
Perhaps you are familiar with this wheel. Perhaps you were presented a copy of the wheel as part of the Race to the Top training that addressed transitioning to the Common Core State Standards. Perhaps you were provided a copy of this graphic as a poster or instructional tool you can use develop and deliver lessons that not only address depth of knowledge but also promote cognitive rigor.
This graphic is an effective and useful tool – for teaching and learning for higher order thinking. It categorizes the levels of thinking students are expected to demonstrate, which is what cognitive taxonomies such as Anderson and Krathwohl’s revision of Bloom’s Taxonomy, Cognitive and Metacognitive Systems of Marzano’s New Taxonomy, and Biggs and Collis’s SOLO Taxonomy. In fact, out of all the taxonomies that categorize higher order thinking, the DOK Wheel is most aligned to the SOLO Taxonomy and its four quadrants. This should be called the HOT Wheel or even the SOLO Wheel and could be used to plan and provide instruction that marks and measures higher order thinking.
However, it does not designate the depth of knowledge students are expected to communicate – or, at least, how Webb’s Depth of Knowledge Model designates these levels.
(You can click here to read further about why the DOK Wheel does not accurately depict Webb’s Depth of Knowledge.)
Now before you look wide-eyed or with a narrowed gaze at your school leaders or the professional development coordinators of your local education agency for giving you the DOK Wheel and having you use it to plan and provide your instruction and assessment, don’t blame them! They were only using the resource they were provided in their own training by either an academic organization or professional development provider who specialized in training that addresses the cognitive rigor of college and career ready standards. Also the DOK Wheel is everywhere! Just conduct a Google search for depth of knowledge or DOK, and you’ll find The DOK Wheel linked to state and local education agencies as an instructional resource for educators.
So from where did the DOK wheel come?
The origins of the DOK Wheel are unknown. No one seems to know from where it came or who designed it. What is confirmed, however, is that it was not designed by Norman Webb and he does not endorse it as a tool for educators to use to plan and provide complex instruction and assessments (Walkup, 2014a).
Unfortunately, the DOK Wheel has been widely addressed and included in professional development trainings and materials on college and career ready standards. The widespread saturation of the DOK Wheel and other poorly designed visuals and maps has also been due to educators and professional development providers downloading these visuals and incorporating them into their presentations and trainings with the assumption the visual was a credible source that was developed by Dr. Webb. There’s a citation at the bottom of the DOK Wheel graphic that references the developer of depth of knowledge, Norman Webb along with a url link, which is why many people believe this wheel was authorized by Dr. Webb. However, if you try the link, you’ll not only find it has no connection to the DOK wheel whatsoever but is also a dead link.
The concept of depth of knowledge that is addressed in the college and career ready standards was developed by Norman Webb (1997; 2002). Webb designed his model as a means of increasing the cognitive complexity and demand of standardized assessments. Traditionally, standardized assessments measured students to think deeply about the academic content, concepts, ideas, and procedures they were learning. However, these assessments were limited in measuring students ability to transfer and use what they were learning in different contexts. They were also limited in measuring the depth of understanding students must develop and demonstrate. Students were challenged to demonstrate – or show – the ability to think deeply about how to answer questions, address problems, accomplish tasks, and analyze texts and topics. However, they were not being challenged to communicate – or tell – how and why they could transfer and use what they were learning in different contexts.
Webb first introduced the concept of depth of knowledge in his paper “Research Monograph No. 8: Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education” (1997). The purpose of this monograph was “to define criteria for judging the alignment between expectations and assessments” (Webb, 1997). He explains how depth of knowledge consistency serves as attribute under the criteria of content focus, explaining how DOK can vary on a number of dimensions such as the following
- the level of cognitive complexity of information students should be expected to know
- how well they should be able to transfer this knowledge to different contexts
- how well they should be able to form generalizations
- how much prerequisite knowledge they must have in order to grasp ideas
Essentially, depth of knowledge designates how deeply students must know, understand, and be aware of what they are learning in order to attain and explain answers, outcomes, results, and solutions. It also designates how extensively students are expected to transfer and use what they have learned in different academic and real world contexts.
In 2002 Webb published his paper “Depth of Knowledge in the Four Content Areas” in which he not only describes how depth of knowledge can be addressed in all the content areas but also established the following levels within his model:
- DOK-1: Recall and reproduce data, definitions, details, facts, information, and procedures. (knowledge acquisition)
- DOK-2: Use academic concepts and cognitive skills to answer questions, address problems, accomplish tasks, and analyze texts and topics. (knowledge application)
- DOK-3: Think strategically and reasonably about how and why concepts, ideas, operations, and procedures can be used to attain and explain answers, conclusions, decisions, outcomes, reasons, and results. (knowledge analysis)
- DOK-4: Think extensively about what else can be done, how else can learning be used, and how could the student personally use what they have learned in different academic and real world contexts. (knowledge augmentation)
The focus of these levels are not on the type of thinking or even the kind of knowledge students are expected to demonstrate. That’s what taxonomies such as Bloom’s, SOLO, or Marzano’s do – establish the level of thinking students will be expected to demonstrate as part of a learning experience. Webb’s establishes the context – the scenario, the setting, or the situation – which students will express and share the depth and extent of their learning. Are they expected to acquire knowledge (DOK-1)? Apply knowledge (DOK-2)? Analyze knowledge (DOK-3)? Augment knowledge (DOK-4)?
These levels may seem to scaffold similarly to taxonomies like Bloom’s. However, they establish how in-depth students will express and share their learning. DOK-1 is content-specific, focusing on the specific text or topic being taught and learned. DOK-2 and DOK-3 is item-oriented, focusing on how and why learning can be transferred and used to attain and explain reasons, relationships, and results. DOK-4 is extensive and practical, focusing on how and why learning can be transferred and used across the curriculum and beyond the classroom. Karin Hess (2006) describes these levels not as steps but rather “ceilings” that establish how far or in-depth students will study and share knowledge and thinking.
It was Karin Hess along with Dennis Carlock, Ben Jones, and John Walkup who extended the concept of using depth of knowledge to instruct and assess student learning by superimposing Webb’s Depth-of-Knowledge Model with Bloom’s Revised Taxonomy to mark and measure the cognitive rigor of learning experiences (2009).
In teaching and learning for cognitive rigor, Bloom’s determines the cognition or thinking students are expected to demonstrate as part of a learning experience. That’s the verb that starts the educational objective or academic standard. Webb’s designates the context – the scenario, setting, and situation – students are expected to express and share what they are learning. To align Bloom’s and Webb’s, Hess, Carlock, Jones, and Walkup created the Cognitive Rigor Matrix / Hess Matrix to categorize educational objectives and questions based upon their cognitive complexity.
The DOK Wheel represents a poor attempt to visualize depth of knowledge similar to how Bloom’s Taxonomy has often been graphically represented as a pyramid. According to John Walkup’s blog (2014a), the DOK Wheel “most likely it grew out of crude attempts to map the familiar six levels of Bloom’s Taxonomy into the four levels of Depth of Knowledge, therefore creating a shortcut to understanding the new rigor measure”. In a phone interview with Walkup, Webb himself commented how “[t]he only possible use of the chart I can see is if someone took a verb and asked how it could be placed in each of the four sectors” (Walkup, 2014a). Hess refers to the DOK Wheel as the “DOK wheel of misfortune” and tell people in my workshops to discard it. It flies in the face of what DOK is about: What comes AFTER the verb” (Walkup, 2014b).
My own work with depth of knowledge and analysis of the wheel led me to design a visual I hope not only teachers will use to plan and provide their instruction and assessment but also administrators will use as a guide for observing and evaluating teacher instruction and effectiveness. I have attempted to create a graphic representation that not only embodies Webb’s concept of depth of knowledge but also how Hess, Carlock, Jones, and Walkup explain how the Webb’s DOK model superimposes with Bloom’s Revised Taxonomy in their Cognitive Matrix. I call this visual Webb’s Depth of Model Context Ceilings. Each block represents how in-depth or extensively students will develop, demonstrate, and discuss their learning. I also incorporated my strategy of asking good questions to set the instructional focus and serve as an assessment – formative, summative, or authentic – for student learning.
DOK-1: What is the knowledge? At this level, students are asked to acquire and gather the information they need to develop deeper knowledge and thinking. They are asked mostly factual questions (who, what, where, when) about the texts and topics they are reading and reviewing. They might also might be asked to recall or reproduce how or why a concept or procedure works or is used. The answers to these good questions are either correct or incorrect. Good questions at this level ask students to describe what are the ideas and information presented in texts and explain how concepts and procedures work.
DOK-2: How can the knowledge be used? At this level, students are asked to demonstrate and communicate conceptual and procedural knowledge. They are asked analytical questions that challenge them to examine and explain how can the concepts and procedures they are learning be used to answer questions, address problems, accomplish tasks, or analyze texts and topics. They also begin to show and tell self-knowledge and personal understanding of how can and could they (or you) use what they (you) are learning. They also begin to think critically about how would you use the concepts and procedures to answer a question, address a problem, accomplish a task, or analyze a text or topic. Good questions at this level ask students to show and tell how concepts and procedures are used. The emphasis is more on the application of ideas and information rather than the item being addressed.
DOK-3: Why can the knowledge be used? Students learning at this level are still demonstrating and communicating conceptual and procedural understanding. However, the instructional focus and assessments shift from applying to analyze and evaluating how and why concepts and procedures can be transferred and used to attain and explain certain scenarios, settings, situations, and solutions. Students are also asked hypothetical questions that prompt them to think strategically and creatively about how could you use what they are learning. They are also asked argumentative questions that engage them to think reasonably about the credibility and validity of ideas and theories, critique different perspectives and points of view, and defend or refute conclusions and decisions.
DOK-4: How else can the knowledge be used? At this level, students are encouraged to extend their thinking deeper within the subject they are learning, across the curriculum, and even beyond the classroom. These learning experiences focus heavily on developing and demonstrating metacognition – specifically, conditional and contextual knowledge and self-knowledge. Students are asked to think critically about the impact, implications, and influence ideas and information have on a much grander scale. They are also encouraged to express and share their own perspectives and points of view about a text or topic using oral, written, creative, or technical communication. These learning experiences are time and thought-intensive and are typically presented and provided as active and authentic learning experiences such as project-based or problem-based learning that require in-depth research, examinations, investigations, and demonstrations of learning through design.
The Webb’s Depth of Knowledge Ceiling Context and the driving questions at the top of each ceiling provides administrators, teachers, and hopefully even students and parents an understanding of how deeply and extensively students will be expected to demonstrate and communicate their learning. The question stems within the ceilings actually encourage more flexibility in the level of thinking students are expected to demonstrate within these levels. However, as Walkup (2014a) says in his blog, “With Depth of Knowledge, context (notably absent in the wheel chart) is everything” – or, as Hess simply puts it, “what comes AFTER the verb” in a performance objective or academic standard establishes the DOK. When planning for teaching and learning for depth of knowledge, consider how deeply or extensively you want your students to go with their learning. Establish the scenario, setting, or situation teaching and learning will be developed, delivered, and demonstrated.
That is teaching and learning that promotes depth of knowledge.
