lies, damned lies, and Tiers: how it is possible to say “evidence-based practice” and not mean it

It is only now that parents and families are beginning to realize how wrong it is for school officials to say “evidence-based practice” without clarifying the Tier level of research support. 

The importance of thorough communication by school officials on the evidence-based framework is the focus of this E3 newsletter, along with insights into the unfolding story on the de-tracking experiment in Oak Park and River Forest, IL.    

The Every Student Succeeds Act (ESSA) was signed into law by President Obama in December of 2015 with goal of improving the previous No Child Left Behind Act of 2022.  ESSA was intended to reauthorize a commitment to equal opportunity for all students.  ESSA introduced the four Tiers of an “evidence-based” framework for evaluating practices that would bring about increased achievement for all students.  The four Tiers are as follows:

• Tier 1 (Strong Evidence) requires evidence from studies that have had a positive, statistically significant impact on student outcomes without any negative findings from well-designed, well-implemented experimental or quasi-experimental studies examining the same interventions and outcomes.

  
  

• Tier 2 (Moderate Evidence) requires evidence from studies that have had a positive, statistically significant impact on student outcomes without any negative findings from well-designed, well-implemented experimental or quasi-experimental studies examining the same interventions and outcomes.

  
  

• Tier 3 (Promising Evidence) requires evidence from at least one correlational study with statistical controls for selection bias, which generally includes a statistical model for determining the relationship between two factors or variables.

  
  

If it stopped there, you might say well done.  I’d prefer to see my children, my schools, my community using practices that were evidence-based at Tier 1, or maybe Tier 2.  Tier 3 relies on correlation and not causation, and those practices probably either need more study or more thinking.   Now look at evidence-based practice Tier 4. 

• Tier 4 (Gathering Evidence Over Time) relies on logic models and creates the opportunity to implement potentially innovative practices that do not yet have a rigorous evidence base (i.e., where researchers have not yet demonstrated that these practices have had a significant positive impact on the desired outcomes).

  
  

If you were not aware of the four Tiers, had high trust in public schools, and only heard someone claiming this is an “evidence-based practice”, then you may be inclined not to think or ask questions.  This likely was the mistake of residents in Oak Park and River Forest, IL around de-tracking as an intervention that would help black students.

By way of example, this is former OPRF high school district 200 board member and secretary describing de-tracking freshman year as an intervention that will close the gap in test scores.  De-tracking is stopping the practice of differentiating instruction for ability level, and at OPRF it meant eliminating separate standalone honors classes for freshman and combining students under one lesson plan. 

“You made that case. You made an evidence-based case. Congratulations. I think it’s the right thing to do”.  Did you hear it?  The quote comes from Ralph Martire in local media.  Martire led the 2016 de-tracking effort in K-8 River Forest, IL D90, a feeder district to OPRF high school, before he joined the OPRF school board and led de-tracking there which began in 2022. It might have been interesting if anyone knew to ask which Tier of evidence supported this intervention.  It might have been more honest if it was clear to residents that de-tracking at OPRF was Gathering Evidence Over Time, or an experiment.

Only recently, E3 learned about the Evidence-Supported Interventions Associated with Black Students' Education Outcomes: Findings from a Systematic Review of Research.  The literature review, completed in 2018 by the U.S. Dept. Education, screened 3,917 studies holding possible interventions and found 22 that met Tier 1 or Tier 2.  De-tracking wasn’t one of these 22, it wasn’t Tier 3 either, and at best was a Tier 4 experiment. 

So, next time a school official calls something “evidence based”, don’t tear-up, just ask them to clarify the Tier level of research support according to the ESSA framework.

Stay tuned, E3 will highlight how many of these 22 proven interventions were part of the Equal Opportunity Schools program dropped by the OPRF administration.

The U.S Dept. of Education de-tracking research at OPRF is a story that is unfolding fast and it has implications for feeder districts 90 and 97.  The Manhattan Institute has covered the larger story of U.S. Dept. of Ed. funding research, and local media has details on de-tracking research at OPRF high school.

By the way, watch out for unsupported claims of “best practice” too. 

It’s E3, where equity still means fairness.

Like what you’re learning?  SUBSCRIBE and send to a friend – it’s free.