The Rigor/Relevance Framework®

The Rigor/Relevance Framework is a tool developed by staff of the International Center for Leadership in Education to examine curriculum, instruction, and assessment. The Rigor/Relevance Framework is based on two dimensions of higher standards and student achievement.

First, there is the Knowledge Taxonomy, a continuum based on the six levels of Bloom's Taxonomy, which describes the increasingly complex ways in which we think. The low end involves acquiring knowledge and being able to recall or locate that knowledge. The high end labels the more complex ways in which individuals use knowledge, such as taking several pieces of knowledge and combining them in both logical and creative ways.

The second continuum, known as the Application Model, is one of action. Its five levels describe putting knowledge to use. While the low end is knowledge acquired for its own sake, the high end signifies use of that knowledge to solve complex real-world problems and to create unique projects, designs, and other works for use in real-world situations.

The Rigor/Relevance Framework has four quadrants. Each is labeled with a term that characterizes the learning or student performance at that level.

The Rigor/Relevance Framework is easy to understand. With its simple, straightforward structure, it can serve as a bridge between school and the community. It offers a common language with which to express the notion of a more rigorous and relevant curriculum.

The Rigor/Relevance Framework is versatile; it can be used in the development of instruction and assessment. Likewise, teachers can use it to measure their progress in adding rigor and relevance to instruction and to select appropriate instructional strategies to meet learner needs and higher achievement goals.

Click here to read more about the Rigor/Relevance Framework.
 

Knowledge Taxonomy Verb List

Rigor refers to academic rigor - learning in which students demonstrate a thorough, in-depth mastery of challenging tasks to develop cognitive skills through reflective thought, analysis, problem solving, evaluation, or creativity. Rigorous learning can occur at any school grade and in any subject. A versatile way to define the level of rigor of curriculum, objectives, instructional activities, or assessments is the Knowledge Taxonomy Verb List. The Verb List can be used to create a desired level of expected student performance in a performance task and can also be used to evaluate the level of existing curriculum, instruction or assessment.

When creating performance tasks, selecting the proper word from the verb list can help to describe the appropriate performance. Simply start with a verb from the desired level and finish the statement with a specific description of that skill or knowledge area. The verb list can also be used to evaluate existing performance tasks. Looking for verbs and identifying their level will give a good indication of the level of student performance in that instruction.

Click here to read more about the Knowledge Taxonomy Verb List.

Quadrant A Verbs: calculate choose count define describe find identify label list locate match memorize name point to recall recite record say select spell view

Quadrant B Verbs: adjust apply build collect construct demonstrate display dramatize draw fix follow illustrate interpret interview look up maintain make measure model operate play practice produce relate role play sequence show solve tune use

Quadrant C Verbs: analyze categorize classify compare conclude contrast debate defend diagram differentiate discriminate evaluate examine explain express generate infer judge justify prove research study summarize

Quadrant D Verbs: adapt compose conclude create design develop discover explore formulate invent modify plan predict prioritize propose rate recommend revise teach

The Application Model

Defining Relevance
Relevance refers to learning in which students apply core knowledge, concepts, or skills to solve real-world problems. Relevant learning is interdisciplinary and contextual. Student work can range from routine to complex at any school grade and in any subject. Relevant learning is created, for example, through authentic problems or tasks, simulation, service learning, connecting concepts to current issues, and teaching others.

Identifying the level of relevance in a performance task is a little more difficult than determining the Knowledge Taxonomy level because there is no verb list. However, just as the Knowledge Taxonomy categorizes increasing levels of thinking, the Application Model describes increasingly complex applications of knowledge. Any student performance can be expressed as one of the five levels of the Application Model. The Application Model Decision Tree can assist in setting the desired level of expected student performance in application.

Teachers can use the Application Model Decision Tree to clarify where a task belongs on the Application Model.

Click here to read more about the Application Model and view the Application Model Decision Trees.
 

Teacher Questions

In order to ask the appropriate questions at the right times, it is important to understand the different types of questions and the kinds of information/response which they elicit. Questions can be grouped into six major types based upon the purpose in teaching and learning. In the context of teaching and learning, questions pose the greatest potential for creating learning conversations. By using such questions, the teacher can stimulate thoughtful student reflection and learning. Teacher questions can be a useful strategy in many aspects of teaching to increase student engagement in learning, enhance the quality and depth of student thinking, and develop a questioning attitude in students whereby they learn to ask questions and seek answers. The following question stems are helpful to support teachers in assessing student learning.

Quadrant A Ask questions to recall facts, make observations, or demonstrate understanding: What is/are _____? How many _____? How do/does _____? What did you observe _____? What else can you tell me about _____? What does it mean _____? What can you recall _____? Where did you find that _____? Who is/was _____? In what ways _____? How would you define that in your own terms? What do/did you notice about this _____? What do/did you feel/see/hear/smell _____? What do/did you remember about _____? What did you find out about _____?

Quadrant B Ask questions to apply or relate: How would you do that? Where will use that knowledge? How does that relate to your experience? How can you demonstrate that? What observations relate to _____? Where would you locate that information? Calculate that for _____? How would you illustrate that? How would you interpret that? Who could you interview? How would you collect that data? How do you know it works? Can you show me? Can you apply what you know to this real-world problem? How do you make sure it is done correctly?

Quadrant C Ask questions to summarize, analyze, organize, or evaluate: How are these similar/different? How is this like _____? What's another way we could say/explain/express that? What do you think are some reasons/causes that _____? Why did _____ changes occur? How can you distinguish between _____? What is a better solution to _____? How would you defend your position about _____? What changes to _____would you recommend? What evidence can you offer? How do you know? Which ones do you think belong together? What things/events lead up to _____? What is the author’s purpose?

Quadrant D Ask questions to predict, design, or create: How would you design a _____ to _____? How would you compose a song about _____? How would you rewrite the ending to the story? What would be different today, if that event occurred as _____? Can you see a possible solution to _____? How could you teach that to others? If you had access to all the resources, how would you deal with _____? How would you devise your own way to deal with _____? What new and unusual uses would you create for _____? Can you develop a proposal that would _____? How would you have handled _____? How would you do it differently?

Click here to see the question stems on the Rigor/Relevance Framework.
 

Student Work

Student work is at the heart of learning. Focusing on student work is also an excellent means of measuring the quality of instruction. Student work is defined as the observable effort or tangible products produced by a student. Student work provides the most tangible evidence of the learning process. The best way to judge the quality of teaching and learning is by looking at the work that students are producing in the classroom.

• Is the work meaningful and challenging?
• Are all students actively engaged?
• Do students have a clear understanding of what constitutes outstanding work?
• Do students who commitment to and enthusiasm for their work?

Answers to these questions provide rich evidence of the quantity and quality of learning taking place. Teachers should spend time thinking about what significant pieces of work students will produce and not limit themselves by simply defining the content and objectives for what students will learn. The following list of student work is a good reference for defining student work as part of assessment planning.

Advice letter Analysis of painting Analyzing primary sources Argument analysis Article reviews Biography analysis Cartoon Character analysis Chart Complaint letter Data analysis

Debate Error analysis Field guide Film analysis Geometric analysis Graph Interview Questions Journal entry Letter writing Literary analysis Logical sequences

Map Memo News report Oral history Persuasive letter Planning for a task Poem Poster Preparing for a discussion Proposal

Proposals and criteria Questionnaire Questions Real-world problem solutions Road trip directions Rules Scale model Speech critiques Survey Taxonomy Timelines

Click here to see more student work products.
 

Next Generation Assessments

Next Generation Assessments will be performance-based tasks. A performance based task requires that students think critically and apply their learning to a real-life situation. The performance task provides is a description of how a student is expected to demonstrate mastery of learning aligned to the Common Core State Standards. The task description includes the context for the task and expectations for the student work that will be produced or performed for a given length of time. The task requires that students think critically and apply their learning to a real-life situation.

Appendix B of the Common Core State Standards (http://www.corestandards.org/assets/Appendix_B.pdf) includes sample texts and performance tasks for teachers that align to specific standards. An example of a performance task from appendix B is below:

Students (with prompting and support from the teacher) describe the relationship between key events of the overall story of Little Bear by Else Holmelund Minarik to the corresponding scenes illustrated by Maurice Sendak.[RL.K.7]

The sample next generation assessments developed by the International Center are positioned for high levels of rigor and relevance as described by the Rigor/Relevance Framework. They include:

• Student work that will be produced or performed
• A learning context that is interdisciplinary, real-world predictable or unpredictable

What sets Next Navigator apart from performance tasks in Appendix B of the Common Core State Standards?
All samples in Next Navigator include student work and scoring guides. The example from Appendix B does not provide how the students will complete their description, nor does it include a scoring guide that describes what an exemplary description includes.

What does an exemplary next generation assessment look like?
Teachers and instructional leaders can use the following rubrics to reflect on existing next generation assessments and as they create their own.


 

Designing Performance Assessments

Various types of assessments are used to measure what a student knows and is able to do. A conscious effort to mirror instruction in assessment will enhance the student’s ability to perform. The more frequently used types of assessment are:

• Multiple choice
• Constructed Response
• Extended Response
• Process Performance
• Product Performance
• Portfolio
• Interview
• Self-reflection

The sample next generation assessments included in Next Navigator are considered process or product performance-based assessments.

The analytic scoring guide is the most popular for performance-based assessments. The analytic scoring guide uses several broad categories. For each category, specific criteria describe the highest and lowest levels of performance and several levels in between. Analytic scoring guides can be developed and refined to evaluate student performance on a consistent and accurate basis. Defining criteria for complex tasks is not easy. It involves analyzing what is essential in the task as performed by the real-world expert, not merely what a student must do to complete the task. Criteria help students formulate a working definition of what is considered "expert performance" and why. Using the criteria, teachers can work with students to develop a scoring guide for a task that describes what is competent, highly proficient, etc. Such discussions help student understand how to become "expert" themselves. Everyone knows, from the beginning, exactly what a student must do to excel. A student version of the sample next generation assessment should be printed and distributed to students.
 

Instructional Strategies

The appropriateness of using any instructional strategy in a particular situation depends upon matching the characteristics of the strategy, the learners, and what needs to be learned. All of these factors should be considered when selecting the best strategy for the learning situation. The more familiar you are with the strategies, the more likely you are to select the best strategy. Oftentimes, several strategies are used during one class and/or unit. Keeping in mind the strengths of each strategy will help you to create effective instructional experiences for your students. The first criterion to consider in selecting strategies is the level of student performance. When students are expected to demonstrate high level use of knowledge, then the instructional strategy chosen must give them experience with complex use of knowledge.

Artistic Expression Brainstorming Compare and Contrast Cooperative Learning Demonstration Digital Media Production Feedback and Reflection Games Guided Practice

Inquiry Instructional Technology – Any Time Instructional Technology – Real Time Learning Centers Lecture Logical and Independent Thinking

Manipulatives and Models Memorization Note-taking/Graphic Organizers Physical Movement Play Presentations/Exhibitions Problem-based Learning Project Design Research Service Learning Simulation/Role-playing

Socratic Seminar Storytelling Summarizing Teacher Questions Teaching Others Test Preparation Video Work-based Learning Writing to Learn

Click here to see a chart of the instructional strategies aligned to the quadrants of the Rigor/Relevance Framework.
 

Essential Skills

The National Essential Skills Study asks participants to identify what they believe are the 20 to 30 most important topics in each of four subject areas: English language arts, mathematics, science, and social studies.

The topics compiled for each content area were adapted from national standards as identified by the National Council of Teachers of English, the National Council of Teachers of Mathematics, the National Research Council, and the National Council for the Social Studies, and from various state standards. All topics were reviewed by teams of subject-specialist teachers. In all, 50 English language arts, 70 mathematics, 85 science, and 60 social studies topics were identified for incorporation in NESS.

The results show the national rankings as well as the rankings by various groups: Business/Industry, Other Non-educators, Subject Area Educators, and Other Educators. As of December 2007, nearly 14,000 educators, parents, business people, and other stakeholders nationwide had responded to the NESS survey. Residents of all 50 states and the Virgin Islands have participated.