Jesse Steif: All right. Well then, let’s go ahead and get started. Welcome again, everyone, to our webinar here Measure Smarter, Not More: Understanding Assessment Types, Uses and Common Misuses. 

My name is Jesse Steif. I am a literacy content and implementation manager here at the Lassinger Center, and my background is in elementary education and school psychology. 

Assessment and assessment-driven instruction is a topic that is near and dear to my heart, and maybe one of the sort of lesser discussed domains here as everyone is on their journey to, you know, towards more evidence-based practices in their practice.

Let’s talk about today’s goals here. We’re going to understand the types of assessment that are out there, what each one of them measures and what assessment questions they can answer.

The next goal is to clarify common misconceptions about how different assessments are used.

We are big on this concept of explicit instruction here at the Lastinger Center and in evidence-based practice, and one of the main levers to pull in explicit instruction is to use examples and non-examples of concepts. I always find it helpful, and participants always find it helpful, when we talk about a concept and how it applies, but then also how it does not apply. 

So we’ll also emphasize some common assessment mistakes, and then what to do instead, we’re going to connect the assessment types with the types of instructional decisions that they support, and then provide you with some prompts and ways to think about how to apply this assessment information to your assessment ecosystem.

That’s a term that I sort of like because today, maybe more than ever in 2026, we are awash in this sort of sea of data. Sometimes I think about a thick jungle full of weeds and trees and leaves and branches, and our job here as educators is to sort of hack and slash our way through and find out to try to determine what is usable information versus what is information that we can sort of ignore or information that is maybe applied or not applied in the correct context.

We have universal screeners, we have diagnostic assessments, we have progress monitoring probes, end of unit tests, state exams. We have so much data at our fingertips.

And what is needed is this useful framework or heuristic to help us understand and put this data into the right box, or the right frame of thinking. Because each type of assessment data is really designed to answer different questions, and we don’t want to try to use one piece of data to answer a question that it wasn’t actually designed to answer.

And we also want to say at the outset here that there is no such thing as a single assessment that can answer every question that you need about your students, or about your school systems, or your grade levels, if we have leaders here in the audience today, or who are watching after the fact.

So we do need, from a leadership perspective and from a teacher perspective, we do need a fair amount of data at our fingertips to help drive instruction and improve student outcomes.

But there absolutely does come a point, particularly now, when we have data reports that are 50 pages long or 30 pages long, where we may be either over-assessing students or generating more data than we know certainly what to do with.

So, today we’re gonna try to kind of cut through some of the noise and try to find where the signal is in the noise here by talking about these assessment types and the questions that each one is built to answer.

When we’re talking with schools who are dealing with more data than they know what to do with, we get to the point where teachers or leaders are kind of looking at their pages and pages of data reports and kind of wondering what they can do with it.

And looking at their scores and kind of wondering what they can glean from that. We would like to encourage folks to flip a mode of thinking and say, instead of looking at our scores first, and saying, okay, well, what can we glean from this? We want to ask, what question do I actually need to answer? And then, using that question to, you know, to guide you to look at all of your different data sources. 

So first, we need to figure out:

• What do I need to know about an individual student?
• About a Tier 2 intervention group? 
• About a particular classroom in a grade level? 
• About a particular grade level, or a particular school? 

And once you’ve kind of articulated that question to serve whatever purposes that you need, then you want to figure out:

• what action will I take when I find out the information, 
• or when we sort of boil it down and answer it? 
• Is this going to lead to a change in my intervention group?
• Is this going to lead to the change or am I not going to reteach a particular lesson? 
• From a leadership perspective, can I identify a certain percentage of students who are falling below benchmark that would necessitate us to have to take a look at what’s going on in Tier 1 instruction, and make some tough changes to our curriculum and instruction?

We want to identify those decision points and then look at which data answers it and then take action. So we’re sort of flipping the way that we think about it, and starting with the question in mind deciding on what action you’re going to take, then diving into your data.

Let’s take a look at the four types of assessments and four domains of questions that they can answer.

We’ll start with screening assessments. Screening assessments are created to answer the question, “Which students are at risk for future reading failure, or who are actively struggling in reading now, and need additional intervention?”

That’s the question from a specific student standpoint. If we have leaders in the audience here, or even classroom teachers, screening data when we aggregate it and we look at class at whole classrooms and grade levels and schools and we look at the proportion of students who are falling below, you know, below a benchmark, and indicating that they need intervention, it tells us a lot about how effective my core Tier 1 instruction is. 

So, a big job of a literacy leadership team, or a school-based leadership team, we call them lots of different things throughout the state, is to first look at your screening data and use that to determine is there are systemic changes that need to happen here that are either causing this higher level of risk in our students who are at risk for future reading failure. We’ll talk more about this concept of risk when we get in more depth about screeners.

We have diagnostic assessments, and a diagnostic assessment is meant to answer the question of what specific skills do students need instruction or intervention in.

Then there are progress monitoring assessments, which are our tools that help us understand how students are responding to instruction or intervention over time. They have specific characteristics that make them useful for answering that question.

And then we have outcome or summative assessments. And these are our standards-based end-of-unit or, or end-of-year, assessments that give us a high-level picture of whether or not students have mastered the standards that we’ve taught towards.

So we’re going to spend most of the time talking about each one of these individual assessment types. There are some more specifics about their uses and their maybe misuses or mistakes that we make when interpreting them, and we’ll do some multiple-choice activities here as well, where you can put your answer in the chat.

Alright, so first let’s jump into screening assessments here.

And again, as I mentioned, screening assessments should be brief, and they should identify and sort students who are at-risk for reading difficulties and need additional support to sort of lower that risk. It’s also a systems-level risk indicator, meaning is there something about the Tier 1 instruction that is causing more students than need be to fall below benchmark.

So, I wanna talk a little bit more about this concept, of risk, and what screeners actually do. So, screeners like STAR reading, other computer adaptive tests like, like MAP and iReady, or curriculum-based measures like DIBELS Oral Reading Fluency or Acadian’s Oral Reading Fluency, I think about them when I’m thinking about early literacy, I think about them as the closest thing we have to a crystal ball to predict where a student is headed if they are a young student.

Oftentimes, it’s really hard to know, in the absence of screening data, when a student’s entering kindergarten, or first grade, or even second grade, whether they are on track to meet standards by the end of third grade. 

And screenings assessments can do that really well because of how they’re created. They have a property to them called predictive validity, meaning that we know if you fall above or below a certain cut point, that most students who go on, you know, who are scoring in that range either do or do not go on to fail at an end-of-year outcome measure.

So when we look at something like STAR Reading, or FAST in K-2, we know that it has really, really strong predictive validity for third grade reading outcomes. We know pretty well based on how you score in kindergarten or first grade how on-track or off-track you are to be a proficient reader by the end of third grade. 

Now, that doesn’t mean that you are doomed to that if you are at risk in kindergarten or first grade. It’s far from that. Actually, it’s the exact opposite. You are never “doomed” to be a non-proficient reader in third grade, but it shows us where a student’s trajectory is headed if we don’t provide intervention. 

And of course, effective intervention can be life-changing and can absolutely affect a student’s trajectory, regardless of when we provide it, but intervention is most effective in those primary K-2 years.

So, risk in screening doesn’t mean that a student has a disability. It just means that based on their current performance, they’re unlikely to meet later reading benchmarks without additional support.

That’s the whole point of using the term risk is that risk can be reduced. If I am at risk for hypertension, and I have high blood pressure, I can take action by lowering the amount of salt that I eat and eating more kale in my diet. I can reduce my risk for future hypertension, just like we can reduce risk for literacy difficulties.

That concept of risk also applies to our Tier 1. So we know that when, say, 80% of students in a class, or in a grade level, or at a school, are meeting a benchmark. You know our Tier 1 instruction is healthy, and it’s meeting the needs of most students. 

But when fewer than that proportion of students are, the instructional system itself is at risk. And the fix for that is strengthening Tier 1 core instruction. So we always want to look at curriculum and instruction first, before suspecting that it’s a student-level difficulty.

Because the vast majority of students don’t have a disability. Some of them absolutely do, and there will always be a need for specially designed instruction and Tier 2 and Tier 3 intervention, but we know that the vast majority of students don’t, you know, don’t need, or shouldn’t need intensive intervention and specially designed instruction.

I think about screening data as a mirror for instruction. If students aren’t climbing those performance bands and raising their achievement levels, we look at curriculum, instruction and implementation, and not just jump right into assuming something is wrong with the student.

So, they are meant to answer the questions of which of my students need intervention, and is our Tier 1 instruction working for most students?

So, let’s talk about some common screening mistakes.

The first one is using multiple screeners,and that can cause a couple of issues. 

I want to be clear where we’re talking about two screeners that measure reading the exact same way, like two computer adaptive tests. That’s what that acronym stands for, CAT, Computer Adaptive Test.

Using two computer adaptive tests, doesn’t actually improve the accuracy with which we are identifying students who are at risk. It actually reduces accuracy because you might have one screener that says where a student scores in the 20th percentile, and maybe the other screener may indicate the student is scoring slightly above where that cut score for where you would provide Tier 2 or Tier 3 intervention.

And, what do you do when you have that scenario? It doesn’t mean that one is right or one is wrong, but it has to do with how that computer adaptive test actually measures student reading. 

Whereas if you are pairing something like a computer-adaptive test that measures overall reading skills, you know, with a little bit of vocabulary, a little bit of phonics, a little bit of oral language, a little bit of comprehension with something like an oral reading fluency test, where a student has one minute to read a text out loud, and we measure their words correct per minute, which can tell us a ton about how a student is reading. That actually can improve identification of students who are at risk, particularly in the elementary grades.

The next one is using screeners to identify exactly what to teach. There’s no such thing as a screening assessment that will tell you everything that you need to know about what a student needs for Tier 1 instruction and for small group intervention.

Screeners vary in terms of the level of detail that they provide to you, but there is no such thing as a screener that will tell you exactly what you need to know in order to instruct them the next day. That’s just not what a screener was made to do.

It’s very challenging to figure out all of the granular, specific skills that students need from a computer-adaptive test, where they’re answering multiple choice questions, and no two students are really getting the same set of questions. It’s very challenging to group students based on screening data.

And so the primary question, again, that it can answer is who needs intervention, not necessarily what or everything that they need in terms of their intervention.

The next one is related: forming small groups based only on screening. Screening will tell us who may need help in the elementary grades, screening will tell us who needs small groups, Tier 2 and Tier 3 intervention, but again, just not precise enough by themselves to form those particular groups. We need other pieces of data for that. 

And the other one, is specific to English language learners. Sometimes we see students in districts where we may delay, or we just don’t screen students because, depending on their English language learner status, but research would indicate that is not a practice that we want to engage in. We want to screen every student. The term is universal screening. Every student gets screened.

And because the same risk indicators apply to English language learners as for English-speaking students. And when we say risk indicators, we mean difficulties in key skills that we know to be very important for reading proficiency. 

On the early learning side, it’s things like their skills in phonological awareness, letter sound identification, you know, segmenting and blending sounds, decoding words, and their reading rate. 

As students get older, it’s their oral language proficiency that becomes much more important. Which is not to say that oral language proficiency isn’t important in the early language years, or in the early learning years, of course it is, but word-reading skills are actually a bigger indicator of risk in the early grades because most kids are learning how to read in K-2. And students’ oral language in the later years becomes a bigger factor in their reading comprehension.

So, we want to screen all students, regardless of whether they’re a student with a disability, or an English language learner or anyone else.

Alright, we have a couple of these pause and predict questions built in, throughout the session. So, these are some multiple-choice questions. We’re gonna take a look at a question.  I’ll show you A, B, C or D, and then in the chat, I’d like you to put your answer. If you’re watching this after the fact, go ahead and pause it here and think, and come up with your answer. 

We have a fictional district here that has a district assessment team, or a school, or a school assessment team.

They’re planning next year’s reading screening systems, and talking about what assessments they’re going to use, and maybe what assessments they’re going to get rid of, what assessments they may add. 

Currently, they use two computer adaptive screener, beginning, middle, and end of the year, that will identify students as at risk, and will identify the same students about 75% of the time. There are differences in how these two computer adaptive screeners measure students, and that result in differences in how they score, in say, their percentile rank.

And it’s leading to situations where teachers feel uncertain about how to act when the results conflict. What do we think the committee should recommend? There’s a little overlap there.

A. keep both screeners and develop additional rules for resolving conflicts when screener. A says, yep, we’re at risk, we’re in the 20th percentile, and B says, nope, we’re not at risk, we’re in the 40th percentile, and we’ve identified it as, like, the 35th percentile as the risk line.

B. Or we can keep both, but reduce the number of screenings to twice a year instead of three times. 

C. We can adopt one screener and then use diagnostics to guide instruction.

Or we can D, add a third screener to break ties.

So go ahead and take a moment.

All right. The answer? C. Adopt one screener, and use diagnostics to then guide instruction.

Answer choice A, two computer adaptive test screeners will result in less accuracy, and more false positives, and more false negatives. By false positives, we mean we have actually identified a student as at risk, but they actually don’t need intervention. Maybe this is a student who kind of blew through the test, but is actually a proficient reader. Or this is a student who had some small issues, but actually, but all things considered, even without intervention, would go on to be a proficient reader at the end of the year. 

And more false negatives, which are not so great with a screener, meaning we’ve identified a student as not at risk, as being on track to reading proficiency, but they’re actually not. They’re struggling.

One of the most common cases of false negatives happens for students who have word-reading difficulties, but a really strong kind of oral language comprehension.

And so, on a computer adaptive test, they may score in the 40th or the 50th percentile, because they can sort of glean enough meaning from the text, despite reading actually pretty inaccurately. 

Maybe they’re reading, like, 75-80% of the words correct and they have real reading issues, but they have strong enough oral language that they can kind of get the gist of a passage and pass with an average score on a test of reading comprehension.

Screening twice a year instead of three times a year with two screeners is still gonna be redundant. And adding a third screener would compound the problem, would increase the number of false positives and negatives, and not solve it.

So that brings us to diagnostic assessments, and the first thing here, to set some context, when we use the term diagnostic, we’re not talking about diagnosing a disability, like a specific learning disability, or dyslexia, or something like that. 

We think of a medical diagnostic test, like a blood test, it says, yes, you have the blood markers for this disability, or you don’t have the blood markers for this disability. That’s not what we’re talking about here with diagnostic assessments. 

We are diagnosing skill gaps, or specific skills that students have or do not have, that may need instruction. So we would administer diagnostic assessments to students who were found to be at risk on screeners. We don’t give diagnostic assessments to every student, like we screen every student. We just reserve the diagnostics for students who need intervention.

And to figure out what they need that intervention in. What skills do we need instruction in? So there are a lot of diagnostic assessments out there, including things like phonics surveys that are able to assess specific decoding or phonics skills that students do and do not have. Spelling inventories, where we have students spell a series of words and do some good error analysis to figure out what phonics skills they can and can’t spell.

When we look at students reading aloud and we do some oral error analysis on that, that’s a diagnostic. Listening to a student read out loud is a really wonderful driver of instruction. We can kind of see how they read and what types of errors they make.

And those are more on the foundational skills side of phonics and spelling and reading fluency. On the language side, on the more comprehension-oriented side, there aren’t as many. There are some new ones that are, like, just starting to come out in the last, maybe, year or two years.

There’s one, the Acadians Comprehension, Fluency, and Oral Language Diagnostic Assessment that will let you understand how your students understand, for example, narrative language and text structures, their ability to make inferences, their vocabulary, and a wide variety of other comprehension skills.

It’s a bit of a longer test, and so we don’t want to give it to everyone, and it is free, and open source available.

And we’ll also get into specific intervention placement tests, here in just a moment. But we do also consider these intervention placement tests to be diagnostic. So, you know, we don’t endorse any particular curricula or program, but many good evidence-based intervention programs will have students take a test in order to place them in a lesson within the lesson sequence. 

Some of the ones that are not computer-adaptive, interventions that are more teacher-directed and paper will have students read words and read passages, and we’ll say, okay, you have the skills that we usually teach in lessons one through 50, so we’re not gonna reteach those to you. 

You can decode CVC words with short vowels, you kind of understand a lot about early decoding instruction, so we’re gonna put you into lesson 51 here. And that is considered a diagnostic observation assessment because it’ll answer the question, what does this student need for instruction right here, right now?

So the questions: 

• What skills has this student mastered? 
• What are our next instructional targets? 

And diagnostic assessments are really where we want to look after screening to form our small groups for intervention.

We want to group students who have similar instructional needs when we’re talking about Tier 2 or Tier 3 intervention, and the best way to do that is to provide everyone who’s at risk and needs intervention, diagnostic assessments, and look at students who have similar profiles of strengths and weaknesses, and they will likely be students who would benefit from the same types of instruction.

So I just have an example here. This is a small portion of a comprehensive phonics survey. If you’re joining us and you’re from a secondary grade. Then you may not be familiar with a phonics survey. This is just one small piece of a broader one.

And you can see that we have students reading CCVC words, we were measuring their ability to recognize consonant digraphs in initial and final positions, reading words with Y as a final vowel and open syllable words.

Phonics diagnostics get very specific about the types of phonics patterns that students need. Similarly, you know, spelling inventories get very specific about the types of spelling instruction that students need, and so on.

Some diagnostic mistakes that we see happening often, maybe a little less now, and, but as opposed to a couple of years ago, this first one, using text levels as diagnostic assessments. 

So I’m trained in elementary education. I was a second and fifth grade teacher. And a big part of my training, you know, close to 30 years ago, was  to find a student’s reading level A through Z, and I was trained that we then put students in those levels of books for their instruction, because those levels of books, you know, contain the skills that the student needs, because they are placed into that level of text.

We know now that that is off-base. There’s a ton more complexity and nuance to these concepts of instructional level and independent and frustrational level, but we don’t want to get in this habit of labeling a student as a particular level of text. A level K refers to the level of difficulty of a single book, not to the skills of a student. 

Same thing with a lexile. You have a lexile of 1,100. That just refers to a text, not the capability of what a student is able to read. Books within a particular level, or books within a particular lexile range, actually vary really widely, in terms of kids’ ability to read them. 

I might assess a student at quote-unquote, level K, but if they are reading a book at an L, an M, an N, or an O, that they are interested in, or they have some background knowledge on, there’s nothing in the research that would indicate they’re not capable of that. So students do not have one instructional level in terms of these letter levels or lexiles.

It’s much more complex than that, and we don’t want to be in the business of holding students to texts that are “at their level.”

Treating summative scores or outcome assessments as diagnostics, looking at an end-of-unit comprehension test, you know, a 10-question multiple choice, or another kind of summative tests, they measure standards, and some of them measure standards mastery really well. 

But hey don’t tell us why a student did or did not master a standard. If you answer a multiple choice question incorrectly on the computer that measures a certain standard, we know pretty well that you haven’t mastered that standard, but there are all kinds of reasons why students answer those questions incorrectly.

Maybe they couldn’t read the question accurately enough, or the answer choices accurately enough to actually answer the question correctly. Maybe there’s a word or two or three within the question or within the passage that a student didn’t know the definition of.

It’s more of a vocabulary issue, rather than an author’s perspective, or any other standard. So, we don’t want to take a summative assessment, a standards-based assessment, and then jump directly to providing intervention in that specific standard without diving deeper into why they may be struggling.

We don’t want to administer diagnostics to all students. They can be very, very time-intensive. Some of, you know, single diagnostic assessments can take anywhere between 5 to 15, sometimes 30 minutes, and so that it’s just prohibitively long and expensive to do for every student. So this is just for students who are at risk and need intervention, and we need to know more.

And on that same subject, if you are in a district that has specific interventions that you are using for, say, second grade students who are at risk, we know that we have these two interventions to choose from.

Both of those interventions have placement tests. You likely do not need to do a full diagnostic assessment because that placement test is the diagnostic. It’ll tell you where to start in that intervention, and there’s no need to spend additional time assessing a student and likely not using that information. Time is always of the essence when it comes to students who are at risk and need intervention. 

We need to assess as little as possible, and only as much as necessary to drive our instruction.

We have our next pause and predict. Mr. Lee’s third graders just finished their winter screening, and 10 of his 25 were flagged as at-risk and in need of intervention. They scored below, just for, just say, for example, below the 30th percentile, when the 30th percentile and below is kind of our Tier 2 and Tier 3 criteria.

He pulls their guided reading levels, their end-of-unit assessment scores, and plans to administer a phonics diagnostic, because a lot of third graders do struggle in reading because they still can’t read accurately and fluently enough. So he’s going to administer them to the whole class next week to get a complete picture before moving into the grouping.

What’s the problem here?

We have A: Levels and unit tests aren’t diagnostic, only the at-risk students need a diagnostic. 

B: He should add a second screener before diagnosing, just to be sure.

C: He should skip the diagnostic and group by his screening scores because that’s going to take too much time. 

D: He should diagnose the whole class to compare at-risk students to their peers, to see if everyone has an issue with decoding.

Alright, and our answer was… A.

What I would do, if I were, if I were in Mr. Lee’s class, I would skip the reading levels. We would want to collect that unit test data to figure out how we need to sort of drive our Tier 1 instruction.

And only administer diagnostics to those 10 students who were at risk.

Hopefully he has some additional assistance from a reading coach, an interventionist, a school psychologist, to help kind of shoulder the load of that.

So again, adding a second screener is not going to help drive the intervention. Screening scores alone don’t determine group members.

They determined that those students may need small group intervention, but not what intervention they need and how to group them. And then we don’t need to administer diagnostics to everyone, just the students who are at risk.

Alright, so let’s talk about progress monitoring here.

So when we talk about progress monitoring assessments, there’s kind of two categories of how we monitor students’ progress over time.

We have something called general outcome measures. You might be familiar with general outcome measures if you have administered a DIBELS or an Acadian’s oral reading fluency, phoneme segmentation fluency, nonsense word fluency.

Any word-reading fluency, there’s lots of them, and generally they are all free and downloadable. And what they do is they assess overall growth in reading in response to intervention. So they are sort of these regular temperature checks that indicate to us whether or not our intervention is working or how effectively it’s working. 

What is a student’s rate of progress? Are they making up for those missing skills? Are we changing their trajectory? Are we changing their path and moving it from “oh, boy, this student is really in trouble for this end-of-year test” to “the student is lessening and lessening their risk for issues on an end-of-year assessment”?

And then the other type of progress monitoring are known as mastery measures, which are typically curriculum-specific and built into structured interventions at regular intervals to sort of assess how well a student has learned the specific skills that are taught in that intervention. 

So, for example, if you’re familiar with UFLI Foundations, the phonics curriculum, you know that every week there’s a spelling assessment in which students spell words that contain the patterns that were taught that week, and that integrate previously taught patterns. 

It’s a way of measuring the specific sub-skills that are taught in an intervention.

There’s lots of other programs or intervention curricula out there that will have a mastery test every 10 lessons, or every 15 lessons that will cumulatively assess what those previous 10 or 15 lessons actually taught the student.

Both of them are not exactly the same. General outcome measures have to do with “is the student lessening their overall level of risk in response to intervention,” and a mastery measure tells us, “has the student learned these specific skills that we taught them”.

And many good intervention curricula will provide guidance for if a student doesn’t do so hot on these mastery measures, we need to go back and reteach lessons two, five and seven, for example.

We administer general outcome measures or ongoing progress monitoring every one to four weeks, depending on the tier of intervention. So, if a student is receiving Tier 2 intervention, it might be every two, three or four weeks [that] we might monitor their progress.

If they’re in Tier 3 intensive intervention, or if you’re monitoring progress through an IEP, it might be every one to two weeks. The nice thing about general outcome measures, like DIBELS, oral reading fluency, or Acadian’s Oral Reading Fluency, is that they take just one minute, generally, most of them.

An oral reading fluency assessment is one of the most powerful little assessments. It tells us so much in such little time about how a student is responding, where they’re headed in terms of their literacy intervention. 

The oral reading fluency in particular if you’re older than first grade, is such a good indicator of general reading proficiency. 

It doesn’t sound like it should be that powerful when we just have a student read a passage for one minute, and we track how many words correct per minute they’re able to read and the percentage of accuracy of words, but, if you’re a second or third grader, there’s such a high correlation between how you’re performing in oral reading fluency and your comprehension, that it’s almost the same thing. 

So there’s a (forgive me for talking a little bit about statistics here for a second,) but correlation talks about the relationship between two variables. So “it is raining outside” and “it is wet on the ground” has a perfect correlation. It’s a correlation of one.

Right? So every time it’s raining, it’s gonna be wet on the ground. 

Oral reading fluency has a correlation with end-of-year reading comprehension. In some grades as high as, like 0.94, which is about as high of a correlation as you can get. 

So oftentimes, we’re, you know, we were asked about what are really good measures of reading comprehension? How do we track reading comprehension over time, other than these sort of curriculum-specific, you know, formative or summative assessments? And how, like, how do we know a student is making gains in their reading comprehension? 

And if you’re in elementary school, despite it being a little bit counterintuitive, oral reading fluency is an excellent way to really understand a student’s general reading skill because your ability to fluently read a passage samples all of these super important skills, like your reading accuracy, your reading rate. Faster is not necessarily better at a certain point.

We don’t want to be speed reading, but if you can read at a reasonable rate, and you can read with good expression, you’re attending to what you read, you’re understanding what you’re reading as you’re reading it, even though you’re not necessarily isolating reading comprehension. So, general outcome measures for use of progress monitoring can be super, super helpful.

And so mastery measures, again, when we do them, they’re built into specific intervention curriculum according to whoever wrote the curriculum themselves. Sometimes they’re every 10 lessons, sometimes they’re every 5, sometimes they’re every 15 lessons.

So the questions that progress monitoring assessments can answer:

Is the intervention that we are implementing leading to accelerated growth in the student? Are they reliably reducing that risk for reading failure? That is the key, important question that we want to know about students receiving intervention.

But then, from a systems level, examining our progress monitoring data, aggregating it from a school leadership perspective, understanding the proportion of students who are in Tier 2 and Tier 3 intervention who are making progress, who are reducing their risk, or who are not, tells us a lot about how effective our Tier 2 and our Tier 3 system actually is.

Are we consistently using the right interventions?

It spurs the absolutely necessary conversation from a leadership perspective of whether or not our Tier 2 and Tier 3 interventions are having the intended effect of accelerating our students back to grade-level proficiency. 

And if about 75% of students who are in Tier 2 and Tier 3 interventions in your school, in your grade, in your class, are not making really strong, accelerated progress, your first assumption and course of action should be to examine how we are providing that intervention, what curriculum you’re using, whether it’s the right intervention, whether that intervention is being done with fidelity.

Or according to how it should be implemented and how it was implemented in research, rather than saying, oh, you know, looking more at student factors and thinking, well, 75% of your students in intervention are not, you know, they’re not reading disabled.

That’s just, it’s just not that high of a prevalence rate in the population.

You know, we work with schools and districts where, for example, they may have an intervention that’s evidence-based, super strong, good for the students, and in research studies, led to really strong effects for K-5 students.

The intervention itself takes, say, 30 minutes to administer the five steps of the lesson.

And then in talking with the leadership, they have a 20-minute intervention window.

And so the first question is every single student in that 20-minute intervention window is not getting a third of the intervention. So 33% of the instruction, they’re automatically missing because our intervention window is too short for the intervention that we have chosen. 

And so it’s no surprise that 80-90% of the students who are getting that intervention are not making satisfactory growth.

You could either look at changing your master schedule to allow for 30 minutes, looking at an intervention that’s shorter in length. There’s all sorts of ways to problem solve around what we would consider an unhealthy Tier 2 or Tier 3 system.

Okay. We’ll go a little bit more quickly through these.

Some common mistakes that we see, using end-of-unit comprehension tests as general outcome measures. When we think about summative classroom tests, like an end-of-unit test, they’re not sensitive enough to growth to inform quick, you know, Tier 2 or Tier 3 decisions. So, you may have students who score a 70%, then a 90%, then a 40%, then a 80%.

There’s more variation in the test scores that are due to those test scores actually measuring totally different things, rather than it being a lack of progress, on the student factor. So, the end of unit Test 1 might be assessing inferencing and point an author’s perspective, and end of unit test 2 is assessing a totally different skill. And the end of Unit 3 is assessing a totally different skill.

When you’re using progress monitoring assessments, you want to make sure that each assessment that you’re doing is measuring mostly the same thing. You know, when you look at an oral reading fluency for those one-minute oral reading fluency, DIBELS or Acadians probes, they’re all roughly equal in difficulty. The student is doing the same thing each time. They’re reading for a minute, you’re marking for their accuracy and their rate. 

Those sorts of general outcome measures, GOM, are what we want to use when we’re making high-stakes decisions in, you know, about a student’s response to intervention, like, who needs to move up to Tier 3, who can move from Tier 2 to Tier 1, and certainly, one of the highest stakes decisions about whether we need to assess a student for special education or their eligibility for special education.

And that’s not to say that end-of-unit comprehension tests aren’t useful data, they certainly are, but they’re probably the least reliable, and kids’ performance tends to be kind of all over the place.

We also want to be reviewing grade-level progress monitoring data as a team. You know, systems matter a lot, and sometimes we see, you know, individual teachers who are responsible for collecting, monitoring, interpreting and making decisions all by themselves. 

And intervention is a team sport. MTSS is a team sport because these are often systems-level issues, and leadership teams should be involved in analyzing why interventions are or are not reliably improving students.

We also don’t want to use reading levels to measure growth. There are a lot of different reasons why that’s the case, but there’s tons and tons of overlap between consecutive levels, so a level M text and a level N text may not actually be that much different from one another.

And a level M text may actually be more difficult than a level N for some students. And so, if we assess a student as moving from an M to an N, that may not actually mean anything, depending on the books that we choose, or the types of decisions that we make based on that growth or lack thereof. We don’t see a lot of schools doing this anymore, building in “by the end of kindergarten, you need to be at a level D, and by the end of first grade, you need to be at a level whatever it is.”

We have largely moved away from that, as a state, at least on our comprehensive, reading plans, which we would say is a good thing.

Similar to screening, we want to be in the business of progress monitoring closely English language learners. Because evidence-based instruction works well for English language learners, we want to be monitoring them on the same schedules as their monolingual peers. 

You know, oral reading fluency is a great indicator of general reading skill, whether you’re an English language learner, or English is your first language, for example.

We’re coming to the end, this is our last pause and predict.

So, we have Lila who’s a third grader. She’s been in Tier 2 reading intervention, and her teacher’s been using oral reading fluency assessments to monitor her progress every week. This is Lila’s intervention graph.

And so we have the blue line, which are her weekly oral reading fluency scores. So you see in week one, she was reading at a rate of about 30 words per minute.

Then at week 12, at the end of, at the end of that graph, she was, she was somewhere a little bit above 60, words correct per minute.

The aim line is her goal line. We’ve set a goal for her to be reading at a little bit above 70 words per minute.

That AIM line tracks the progress that we would expect to see if she were responding at an accelerated rate.

So we want to know: how should the team respond to this? 

A: Continue the current intervention as is. She’s showing steady growth, and just needs a little bit more time.

B: We want to exit Lila from intervention. She’s making consistent progress and is no longer at risk. 

C: We want to check intervention fidelity and consider how to intensify. Lila’s improving, but not enough to close the gap with her grade-level peers, as represented by the AIM line.

D: refer Lila for a special education evaluation because the intervention simply is not working.

Take a moment…

The answer there is C.

She is improving. She has definitely increased her reading rate, but we see this gap between Lila’s scores and the AIM line. That gap is not narrowing, right? It’s not getting wider. Her improvement is certainly not nothing.

She has effectively doubled her reading rate, but if we extend those lines and extrapolate that data, there will never come a point where Lila is meeting her goal. So she is making some improvement, but we need to do something, either intensify her intervention, if she’s getting it as planned, or look at fidelity. 

Maybe there’s a piece to the intervention that is not being implemented, or not being implemented consistently. Maybe she’s 5 minutes late half the time and is missing a piece of the intervention. Something like that we need to do to intensify her intervention because she’s just not gonna be able to meet that goal.

We’re not totally unhappy with what’s going on with Lila, but we need to make some kind of tweak. This type of data-based decision-making is really best done as a team, as a grade-level team, or a teacher in conjunction with others.

Many heads are better than one because we know from research that teachers generally do not get the necessary level of database decision-making training in their teacher training programs, particularly for how to interpret progress monitoring data and make instructional decisions in response to kids who are needing intervention.

This is exactly where database decision-making is super important.

We’re coming to the end of our time here. We’ll end with outcome measures.

Or summative measures. These are broad indicators of achievement at the end of major instructional units.

 They are generally designed for system-level evaluation, meaning looking at how effective our instructional systems are, not to guide specific individual instructional decisions.

They happen at the end of the year, or end of semester, or end of unit, and are things like FAST, or end of course data or final unit assessments. 

So the questions that it can answer are:

• Which students are or are not mastering grade-level standards? 
• Is our curriculum and instruction working to get students to master standards across classrooms and subgroups and schools, if you’re at a district level? 

So there is some overlap here in terms of the questions that it answers with screening data, in that it is looking at the effectiveness of our systems of instruction.

So again, summative data, like screening data, though, it cannot tell you exactly why a student didn’t master a standard, or what you need to teach on Monday. Again, there’s just, there’s too many variables and too many ways that we assess mastery of standards to really get at the level of specific instructional skills measurement that we need to drive our intervention.

So, putting it all together here we have a summary slide here that we’ll end with. I know we’re a bit over time.

If the question is about who is at risk and may need intervention, we look at our screening data, and when we’re looking at the health of our Tier 1 instructional system, our curriculum, our instruction, how we are implementing it, screening data is the data we look at.

Diagnostic data tells us what skills I should target in intervention, and how you regroup students for small group instruction.

Progress monitoring data, both in terms of our general outcome measures and our mastery measures, tell us how students are responding to intervention that we’re providing over time, our temperature checks, and it also tells us from a systems perspective about the health of our intervention system. About 75% of students should be reliably getting better, being accelerated in their literacy outcomes in response to intensive intervention.

 And then our students, classes, whole schools, meeting broad standards, we look at the proportion of students who are taking outcomes and summative measures. 

So I know we are, we are over time here. I thank you for your time. 

We thank everyone for your time. If you have questions about this, if you are looking, if you want more information, we have many ways to provide professional learning to you around assessment and data-based decision making to your school, to your district. 

We will be rolling out a series of new professional learning sessions, in the fall. We have an excellent new one that we’re calling, Literacy Lab, where we spend quite a bit of time digging into your specific data, looking at progress monitoring data and training folks on how to make these sorts of key instructional decisions. 

This is, you know, an hour is not even close to enough to really do anything other than scratch the surface and kind of reveal the sorts of concepts that are involved in assessment and database decision making. 

So, go ahead and we’re always wanting to hear from the folks that we work with, or the folks that we don’t work with, about what is needed there and what kind of questions you’re grappling with. We’re always looking for new subjects to provide these sorts of free technical assistance webinars, or to partner directly with school districts. 

So, thank you so much, we really appreciate your time, have a great rest of your May and have a great summer.

Thank you.